Develop Reference

How to include a global file type declaration in a TypeScript (Node.js) package

I'm working on a package that I am planning to publish publicly on npmjs. Let's call it the "text package".
I would like that by default when installing that package, you can import .txt files directly and get the correct type (out of the box), like this:
import text from './file.txt'
The text variable would be of type string because the package would have defined its type, using something like this (in a global.d.ts):
declare module '*.txt' {
export const text: string;
export default text;
}
If I include that global.d.ts in my package, and that I import something from this package, then I will automatically get the correct type when importing a .txt file.
But the problem is sometimes I would just need to import a .txt file without importing anything from the "text package", which is why I was wondering if there is some sort of way, as you install a package to install a global type that does not require to import anything else for the type to apply.
In other words, as soon as you install my "txt package" the declare module '*.txt' would apply to my entire project out of the box.
Is there even a way to do this, or whoever installs my package would have to declare their own global type (e.g., declarations.d.ts) to be able to import .txt files globally?
I know that even if the import type works, it will still require Webpack or another bundler to really work but this question is just about the type.

The short answer is:
TypeScript does not support Global types without importing the file referring to the type.
More details:
One example that I found doing this was Next.js - when creating a TypeScript app using npx create-next-app#latest --typescript you can start importing *.css files (for example) and get the correct type.
Where I got confused is that I originally thought that the type was coming from the next-env.d.ts but even when I deleted the file, *.css import was still working in Visual Studio code. But the reason it was, is because a file in the pages directory were importing Next.js' index.d.ts file.
Basically, in Visual Studio Code, as soon as your import a type somewhere in your project, if it's global, it will be accessible everywhere.
Workaround
So what can be done with the current TypeScript capabilities? To support new file types, you will need a file loader such as Webpack. The logical thing to do would be to add a reference to the file type declaration in the file loader itself. This way, as soon as you configure your file loader to be able to import the file, you will inherit the type:
create a txt.d.ts in our package's source directory (e.g. src) - you can use any name for the file, it's not important
if you are using eslint, add an entry to ignore the type file (e.g. 'src/*.d.ts' in your ignorePatterns option
Since you are adding a d.ts file in your source that is not managed by tsc, you need to add a script that will perform the following actions:
Copy txt.d.ts in the target directory of the compiled files for your package
Add this line at the top of your package's file loader (e.g. loader/index.d.ts: /// <reference types="../txt" />\r\n - this will link the declaration file back into your package. Note that you can add this reference to any file of your package.
This workaround will only work once you import the file referencing back to the declaration - this is the only way TypeScript can be made aware that this type exists (see https://github.com/microsoft/TypeScript/issues/49124).
Another alternative could also be to add manual steps (in a readme file) to add a global type declaration file.

to bundle global types, do the following. form typescript
specify default typings directory at typeRoots. .e.g "typeRoots": ["./src/types"].
create a file /src/types/global.d.ts in the specified directory
declare your types in the file inside declare global {} and make sure to have export {} if you don't already export anything.

import() in ts is not dynamic? what's the cause?

According to MDN, import() is a function-like dynamic method. However, I found it not dynamic in my ts project.
Say I have an appleShare.json:
{
price: 123
}
And then, there's an index.ts:
console.log("update the price to 456...")
// manually modify the json file content making price 456
let currentPrice = await import("./appleShare.json").then(obj=obj.price)
console.log(currentPrice)
I ran index.ts directly in vs-code, the result in console:
update the price to 456...
123
I expect 456, but got 123. From my limited knowledge I guess there are two possible reasons:
I have a misunderstanding in import() and dynamic importing.
My understanding is right but vs-code compiled all the code to js before executing them. So, I will never get the newly modified price.
I want to ask, what's exactly the cause of the issue, and how to resolve it?

The "dynamic" import() has the following different behaviors from the regular import which is often referred to as the "static" import.
You can construct a module filename in code and can then load that module from the filename you built. You cannot dynamically build module filenames with the regular import. Filenames for the regular import must be statically specified so they are known by anyone who parses the file, but does not run the code in it. This static declaration enables code analysis for things like tree-shaking and bundling. The dynamic import cannot be used as effectively with features like that.
The import() can happen anywhere in your code (not only at the top of your module). The regular import cannot be just anywhere in your code. In this sense, it is "dynamically" loaded upon demand, not only at the beginning of the module.
A dynamic import() statement can be used to load an ESM module into a CommonJS module. The regular import statement cannot be used in a CommonJS Module at all.
Modules, even dynamically loaded ones are cached. Once they are loaded, subsequent import() statements using the same filename just load the module from the cache, they do not re-read the file. That's why your subsequent import() is not picking up the modified JSON.
If you want to re-read the file, then don't use import - use something like fs.promises.readFile() and then parse the JSON. This will read a fresh copy of the data each time you call it.

How to tune babel loader to properly import a module named macro.js when using CRA

I'm using a library vtk.js. VTK has a special design of classes (https://kitware.github.io/vtk-js/docs/develop_class.html). When I write a class I need to import macro.js module which exposes several base methods such as getters/setters/newInstance etc (https://github.com/Kitware/vtk-js/blob/master/Sources/macro.js).
Looks like CRA uses some special loader for macro.js files. I get an error when trying to import such a file (SyntaxError: Cannot use import statement outside a module). If I copy an original file macro.js from #kitware/vtk.js to my local source folder I still get an error. Even if I remove all the contents of macro.js file I get an error (The macro imported from "../macro" must be wrapped in "createMacro" which you can get from "babel-plugin-macros". Please refer to the documentation to see how to do this properly: https://github.com/kentcdodds/babel-plugin-macros/blob/master/other/docs/author.md#writing-a-macro). However, when I rename macro.js to macro2.js the error is gone.
Is there some way how can I disable this macro loader?

How to invoke a javascript function (generated from typescript) trapped within "System.register()" module while using Google protobuf?

Update: It seems that, the problem is coming due to protobuf. I am fine with other solution as well, which help me to fix the Google protobuf issues. This problem boils down to:
How to integrate Google protobuf with Typescript/Javascript for the browser?
I am retaining below question for the future purpose.
We have moved our application from Javascript to Typescript for obvious advantages of OOP etc..
Earlier invoking a direct javascript function from Html was as straight forward as:
<script>window.MyFunction()</script>
Now with Typescript, all the files are combined into a single autogenerated .js file.
In this single file, individual code of every file are isolated within System.register(). It typically looks something like:
System.register("<filename>", ["<import_1>", ..., "<import_N>"],
function (exports_13, context_13) {
"use strict";
...
function MyFunction () { ... } // somewhere inside the external function
}
In short, everything written within the .ts file is wrapped in an unnamed function after running the tsc compiler.
Now, I don't know how to invoke a function, which is trapped inside another function, which is in turn listed under System.register(...)
Question: What is the correct syntax to invoke such function externally from an Html file?
<script> ??? </script>
Update:
The HTML tries to invoke in following way in the body tag:
<script>
System.import("Main").then( // Main.ts is one of the file
function (module)
{
throw 0; // Temporary, to see if we reach till here
module.main(); // "main()" is the function, which is the entry point
});
</script>
In my code, I am using "browserify" to be able to use the Google protobuf for JS. The error comes for the protobuf related files only. Those definition and source files are present in .d.ts and .js formats.
The error is something like below:
js: Uncaught (in promise) Error: Fetch error: 404 NOT FOUND
Instantiating http://localhost:50000/folder/external/Server_pb
Loading http://localhost:50000/folder/external/_External
Loading Main
Note that, 50000 is a temporary port and the "folder" is just any folder where the .js are kept. The "Server_pb" is a custom protobuf file generated.
My problem can be aptly described quite similar as this link.
Related:
What is mean by System.register in JS file?
How to call a named module from a bundle (<-- can be helpful, but don't know the syntax as a newbie)
How to start a Typescript app with SystemJS modules? (nearly duplicate, but unable to solve the problem with this approach yet)
How do I get TypeScript to bundle a 3rd party lib from node_modules? (seems like another close match; trying to dig into this right now to fix the protobuf problem)

With "google-protobuf" there are issues when used in the fashion of systemjs. It seems that Google has created it only for the nodejs. :-)
To be able to use the protobuf in Javascript for the browser, there are few things which we have to do manually. Such manual boilerplate work can be done using some scripts as well.
I am giving an iterative way, on how to achieve this:
The first step is to generate the protobuf for both JS and TS. Use following command for the same:
protoc <file1.proto> <file2.proto> ... <fileN.proto>
--proto_path=<proto_folder> \
--cpp_out=<cpp_folder> \
--js_out=import_style=commonjs,binary:<js_folder> \
--ts_out=import_style=commonjs,binary:<ts_folder>
Note that, we are using the commonjs (and not systemjs). Legends:
<proto_folder> = folder path where all these file1/2/N.proto files are stored
<cpp_folder> = folder path where you want the c++ file1/2/N.pb.cc/h files to be stored
<js_folder> = folder where you want the file1/2/N_pb.js files to be stored
<ts_folder> = folder where you want the file1/2/N_pb.d.ts files to be stored
Now in all the .d.ts (Typescript definition) files, there are certain code lines, which will give compiler errors. We need to comment these lines. Doing manually, is very cumbersome. Hence you may use sed (or ssed in Windows, gsed in Mac). For example, the lines starting with,
sed -i "s/^ static extensions/\/\/ static extensions/g" *_pb.d.ts;
same as above for static serializeBinaryToWriter
same as above for static deserializeBinaryFromReader
sed -i "s/google-protobuf/\.\/google-protobuf/g" *_pb.d.ts; // "./google-protobuf" is correct way to import
Now, while generating the *_pb.d.ts, the protoc compiler doesn't follow the packaging for Typescript. For example, if in your fileN.proto, you have mentioned package ABC.XYZ, then the fileN.pb.h will be wrapped in namespace ABC { namespace XYZ { ... } }. The same doesn't happen in the case of Typescript. So we have to manually add these in the file. However, here it won't be a simple find/replace as above. Rather, we have to find only the first occurance of any export class (which is of generated proto) and wrap the namespaces. So below is the command:
sed -i "0,/export class/{s/export class/export namespace ABC { export namespace XYZ {\\n &/}" fileN_pb.d.ts;
sed -i -e "\$a} }" fileN_pb.d.ts;
Initial importing of the google-protobuf package has to be prefixed with ./ in the case of generated _pb.js file as well
sed -i "s/google-protobuf/\.\/google-protobuf/g" *_pb.js;
Now compile the the custom Typescript files with tsc -p "<path to the tsconfig.json>", where the tsconfig.json may look like (see arrow):
{
"compileOnSave": true,
"compilerOptions": {
"removeComments": true,
"preserveConstEnums": true,
"module": "CommonJS", <=======
"outDir": "<path to generated js folder>",
},
"include": ["../*"],
"files": ["<path to file1.ts>", ..., "<path to file2.ts>"
}
Now a very important step. All the references to the the generated *_pb.d.ts files, should be referred in 1 of your custom file. That custom file may contain the wrappers around the generated classes if it's required. This will help in limiting string replacement only in that file, which is explained in the upcoming step. For example, create a custom file name as MyProtobuf.ts and import your proto as following:
import * as proto from './fileN; // from fileN.d.ts
In above step, it's important to note that the name "proto" is crucial. With that name, the .js files are auto generated. If there are several proto files in your project, then you may have to create yet 1 more file which exports all of them and then import that 1 file:
// in 'MyProtobufExports.ts' file
export * from './file1'
export * from './file2'
export * from './fileN'
import * as proto from './MyprotobufExports // in MyProtobuf.ts file
With above 2 steps, the usage of the protobuf as, var myClass = new proto.ABC.XYZ.MyClass;
Now the continuation of the important step we discussed above. When we generate the equivalent _pb.js and our custom .js files, still the special name-symbol proto will not be found somehow. Even though everything is wrapped. This is because the autogenerated JS files (from TS files), will declare a var proto. If we comment that then, that issue is gone.
sed -i "s/var proto = require/\/\/ &/g" Protobuf.js;
The final step is to put the browserify on all the .js files into a single file, as below. Due to this, there will be only single .js file, we have to deal with [good or bad]. In this command, the ordering is very important. file1_pb.js should come before file2_pb.js, if file1.proto is imported by file2.proto or vice a versa. If there is no import then the order doesn't matter. In any case the _pb.js should come before the custom .js files.
browserify --standalone file1_pb.js fileN_pb.js MyProtobuf.js myfile1.js myfileN.js -o=autogen.js
Since the code is browserified, the calling of function can be done in following way:
window.main = function (...) { ... } // entry point somewhere in the fileN.ts file
<script>main(...)</script> // in the index.html
With the above steps only, I am able to make the "google-protobuf" work within my project for the browser.

RequireJS: Cannot find module 'domReady'

I have quite an annoying, but probably simple, problem that I just cannot figure out.
In a TypeScript file I have defined the following line:
import test1 = require('domReady');
This "domReady" module is defined in a main.js file that is loaded as the entry point for RequireJS. The definition is as followed:
require.config({
paths: {
'domReady': '../domReady',
}
However... in my TypeScript file I simply get a "cannot find module 'domReady'" and it is driving me insane, as I have double checked the pathing to the file and it is indeed in the correct location with the correct name. Additionally, I am fairly certain that the domReady.js file IS AMD compatible, so it should define an external module just fine! (domReady GitHub Link).
I seriously can't understand why the module can't be found in the import statement. Does anyone have any ideas to what the problem may be?
EDIT 1
The directory structure is as follows:
.
+--App
| +--main.js
| +--dashboard.js
+--domReady.js
The import statement takes place in the "dashboard.js" file, and the config for require.js happens in "main.js".

In order for TypeScript to find a module, you must actually provide module information to TypeScript.
TypeScript doesn’t yet support AMD-style paths configuration, it doesn’t ever use calls within your JavaScript code (like require.config()) to configure itself, and it won’t treat JavaScript files on disk as modules when compiling. So right now, you aren’t doing anything to actually give the compiler the information it needs to successfully process the import statement.
For your code to compile without error, you have to explicitly declare an ambient declaration for the module you’re importing within the compiler, in a separate d.ts file:
// in domReady.d.ts
declare module 'domReady' {
function domReady(callback: () => any): void;
export = domReady;
}
Then, include this d.ts in the list of files you pass to the compiler:
tsc domReady.d.ts App/main.ts App/dashboard.ts
Any other third party JavaScript code that you import also needs ambient declarations to compile successfully; DefinitelyTyped provides d.ts files for many of these.

I've had problems before when the path key and the directory name or file name are the same (in your case, domReady). Might not work, but worth giving it a quick try, i.e.
either
'domReadyModule': '../domReady',
require('domReadyModule')
or rename domReady.js to e.g. domReady-1.0.js and use
'domReady': '../domReady-1.0',
require('domReady')
If that doesn't work, I'd check the relative directories between main.js and the file that is doing the require, or else try importing another library in the same fashion, or finally compare with other libraries that you are importing successfully.
Good luck, hope you resolve the problem!