So, I'm dabbling a bit with Typescript and Grunt at the moment to see if it's worth it for me. The thing is: Typescript does not compile to *.mjs files but only regular *.js files. Node does support ES6 Modules but only if you either mark them as '*.jsm' files or by setting "type": "module". Setting this top-level field in package.json however has global scope for any *.js file in the same directory and any following ones.
This breaks the Gruntfile.js file as it seems since it uses CommonJS modules, see my very basic Gruntfile as example:
module.exports = function (grunt) {
grunt.initConfig({
ts: {
default: {tsconfig: "./tsconfig.json"}
}
})
grunt.loadNpmTasks("grunt-ts");
grunt.registerTask("default", ["ts"]);
}
Without expecting much success I naively changed the export syntax from module.exports = to export default which expectedly did no work since it didn't make much sense.
Questions
Is there any option to use Grunt with ES6 modules enabled in node?
Is there a proper way to tell TypeScript to compile to *.mjs files?
If you set "type": "module" in your package.json, you need to rename Gruntfile.js to Gruntfile.cjs, and run it with grunt --gruntfile Gruntfile.cjs.
The suggested approach with Babel running before Grunt makes Grunt a bit redundant. Since TypeScript does not yet support exporting ES6 modules to *.mjs files (and you have to use the *.mjs suffix in your import when node should still be running with its CommonJS system) and will probably never fully (see Design Meeting Notes 11/22/2019) I have to conclude that ES6 modules still have serious implications and issues. Changing the file extension is not enough since the extension-less imports fail with node. You'd need to go through every compiled file and change the import to specifically load *.mjs files.
However, the TypeScript Compiler can be set up in a way that it does understand ES6 module syntax and to compile to CommonJS (see TS handbook).
{
"compilerOptions": {
"module": "CommonJS",
// [...]
},
}
This way the TypeScript code be written with ES6 module syntax and the output can be CommonJS compatible without braking other code. As a bonus you can skip the Babel approach and grunt can run TS compiler.
You can using babel-node to compile first. Which will resolve ES6 export and import problem.
npm install --save-dev #babel/core #babel/node
npx babel-node server.js
Related
I have a number of published npm packages that I have upgraded to provide both commonjs and esm builds. Some of the packages might be for both node and the browser. All packages compiled with webpack or rollup. All are written in typescript and transpiled into a dist directory.
I create a commonjs index.js file that looks like this:
'use strict'
if (process.env.NODE_ENV === 'production') {
module.exports = require('./react-abortable-fetch.cjs.production.min.js')
} else {
module.exports = require('./react-abortable-fetch.cjs.development.js')
}
I set the package.json main field to the above index.js file.
I also generate a .esm.js file for each package and I set both browser and module fields to the esm.js file and set the type file to be module.
The end result is something like this:
"type": "module",
"main": "dist/index.js",
"browser": "dist/react-abortable-fetch.esm.js",
"module": "dist/react-abortable-fetch.esm.js",
"types": "dist/index.d.ts",
The problem with this approach is that only esm packages can consume it (unless I am wrong).
What is the best way to configure the package.json file so that packages that have not made the leap yet (and that is quite a few) can still consume the package?
The idea is to leverage Node.js conditional exports to customize the import behavior depending on how you import the module (require or import).
In order to have two different interfaces for CommonJS and ESM, you could either:
Transpile to both CommonJS and ESM (allows to accidentally use both import and require for your library in the same application which could cause unwanted and unpredictable behaviors)
Have the implementation as ESM and a CommonJS wrapper (not possible because you can't require an ECMAScript module when it has top-level await)
Have the implementation as CommonJS and an ESM wrapper (best actual solution)
So you have to set CommonJS as a TypeScript transpilation target, then create an ESM wrapper (preferably in a dedicated folder). Finally, map the CommonJS and the ESM entry points to the corresponding files in the package.json:
"exports": {
"require": "./index.js",
"import": "./esm/index.js"
}
I have published a minimal working project here: https://github.com/Guerric-P/demo-commonjs-esm
In order to use the libraries in TypeScript, you also need one .d.ts file aside every JavaScript file.
I'm building a discord.js Discord bot. Now for some reason, discord.js doesn't work with ESM modules (a totally separate issue), so my bot app uses CommonJS modules. Now I have another project on my system called Lib, which has a lot of utility functions that I plan to use in several different projects so I don't have to rewrite them. This Lib project uses ESM modules. Since I have to import Lib from DiscordBot, I use the dynamic import syntax in typescript. Now, whenever I transpile my DiscordBot project, the dynamic imports get converted into some ugly javascript module code, and that ugly module code ultimately ends up using require(). Since require() can't import ESM modules, my bot ends up crashing.
I tried however to stop my ts compiler, copy the code from my ts file that imports Lib then pasting that code into the corresponding JS file manually (and removing TS-exclusive features like type annotations and interfaces). Then I ran my bot app, and it worked perfectly fine. But I don't want to have to do this every time. So it's tsc's compiling that's the problem. How do I fix this?
So I understand the purpose is:
Develop the code in TypeScript
Run the compiled code in CommonJS package
Import and use an ES Module
Option 1:
If "module" in tsconfig.json is set to "commonjs", currently there's no way to prevent TypeScript from transpiling dynamic import() into require() - except that you hide the code in a string and use eval to execute it. Like this:
async function body (pMap:any){
// do something with module pMap here
}
eval ("import('p-map').then(body)");
No way TypeScript transpiles a string!
Option 2
Set "module" in tsconfig.json to "es2020". By doing this, dynamic import would not be transpiled into require(), and you can use dynamic import to import a CommonJS or ES Module. Or, you can use the const someModule = require("someModule") syntax to import a CommonJS module (would not be transpiled to ES6 import syntax). You cannot use the ES6 import syntax such as import * as someModule from "someModule" or import someModule from "someModule". These syntaxes will emit ES Module syntax imports ("module" is set to "es2020") and cannot be run in CommonJS package.
Below is a bit information:
If "module" is set to "es2020": dynamic import import() is not transpiled.
If "module" is set to `"es2015": there's an error:
TS1323: Dynamic imports are only supported when the '--module' flag is set to 'es2020', 'esnext', 'commonjs', 'amd', 'system', or 'umd'.
If "module" is set to "commonjs": dynamic imports are transpiled.
Quote tsconfig.json reference for "module" field:
If you are wondering about the difference between ES2015 and ES2020,
ES2020 adds support for dynamic imports, and import.meta.
This is currently not possible. There is a very new issue at GitHub (https://github.com/microsoft/TypeScript/issues/43329), but that is not implemented yet. So everything you can do now is to switch from ESM to CommonJS with your Lib project.
Update 2022
The issue has been closed and there is now a new option for "module" called node12. That should fix the problem
The node12 setting others are talking about did not work for me, but these compilerOptions did, using Typescript 4.7.2:
"module": "CommonJS",
"moduleResolution": "Node16",
This saved my backside, I did not have to migrate all import requires to imports to be able to use an ESM npm lib.
Typescript input source:
import Redis = require('redis');
import * as _ from 'lodash';
export async function main() {
const fileType = await import('file-type');
console.log(fileType, _.get, Redis);
}
CommonJS output:
...
const Redis = require("redis");
const _ = __importStar(require("lodash"));
async function main() {
const fileType = await import('file-type');
console.log(fileType, _.get, Redis);
}
exports.main = main;
What compiler/bundler are you using? I am assuming tsc based on context.
I recommend using esbuild to compile and bundle your TS. You can also use it simply to transform it after using tsc. It has an option called "format" that can remove any module-style imports. See https://esbuild.github.io/api/#format.
Here is a simple example of using.
build.js
const esbuild = require("esbuild");
esbuild.build({
allowOverwrite: true,
write: true,
entryPoints: ["my-main-file.ts"],
outfile: "some-file.bundle.js",
format: "cjs", //format option set to cjs makes all imports common-js style
bundle: true,
}).then(() => {
console.log("Done!");
});
You can then add something like this to your package.json
"scripts": {
"build": "node build.js",
...rest of scripts
Here is an additional link about some caveats using esbuild with typescript. None of these should really be a problem for you. https://esbuild.github.io/content-types/#typescript-caveats
This has been fixed with the addition of the node12 option for the module setting. From the docs:
Available in nightly builds, the experimental node12 and nodenext modes integrate with Node’s native ECMAScript Module support. The emitted JavaScript uses either CommonJS or ES2020 output depending on the file extension and the value of the type setting in the nearest package.json. Module resolution also works differently. You can learn more in the handbook.
If you use this setting without a nightly build, however, it currently produces the following error:
error TS4124: Compiler option 'module' of value 'node12' is unstable. Use nightly TypeScript to silence this error. Try updating with 'npm install -D typescript#next'.
I'm using a variant of the already mentioned eval-based hack to overcome this issue.
So for example, parse-domain is distributed as an ESM module, so importing it like this breaks in a CJS-based node app:
import { fromUrl, parseDomain } from 'parse-domain';
const parseDomainFromUrl = (url: string) => {
return parseDomain(fromUrl(url));
}
And this is how I have managed to get it working:
const dynamicImport = new Function('specifier', 'return import(specifier)');
const parseDomainFromUrl = (url: string) => {
return dynamicImport('parse-domain').then((module: any) => {
const { fromUrl, parseDomain } = module;
return parseDomain(fromUrl(url));
})
};
(Note that parseDomainFromUrl became asynchronous in the process, so it would need to be awaited by the caller.)
I'm using some files from another repository which is written in es6 syntax.
The current project also is written with es6 syntax, but Babel is configured so tests don't fail on the imports.
but when it tries to import file from that repository it fails.
When I rewrite es6 imports and exports into CommonJS require syntax tests does not fail.
Do I need some additional configuration of Jest or Babel?
Yes, you just need some additional configuration of Jest.
By default Jest does not transform anything in node_modules.
That behavior is controlled by the configuration option transformIgnorePatterns which defaults to["/node_modules/"].
If there is code within node_modules that needs to be transformed then you can whitelist those modules by modifying transformIgnorePatterns in your Jest config:
"transformIgnorePatterns": [
"node_modules/(?!(first-module-to-transform|second-module-to-transform)/)"
]
I am trying to understand the typescript module compiler option.
I went through typescript documentation - docs
It says module option is to Specify module code generation.
What does that mean?
Does it mean if I put module option as commonjs, then the compiler compiles the code to commonjs? But then we have options like esnext, es16. After I went through Docs: Difference between esnext, es6, es2015 module targets, I understood that import() expressions are understood in esnext. Anyway the purpose of compiler is to compile the code into browser understandable syntax(commonjs). So compiling code to the module type given doesn't make sense.
So does it mean the module type you give tells the compiler in what syntax the code is written? Meaning from which code it has to compile it to commonjs? But then we have module type commonjs which is frequently used but we almost never write code in pure commonjs syntax.
what is the purpose of tsconfig.json? stackoverflow answer says module specifies module manager. I don't understand what that means.
I also went through Understanding “target” and “module” in tsconfig and tsconfig module options - does 'System' refer to SystemJS?.
None of these seem to answer my question properly.
tsconfig.json
{
"compilerOptions: {
"module": "esnext"
}
}
TLDR; module in tsconfig.json tells the compiler what syntax to use for the modules in the emitted .js files. Frequently used values are "commonjs" (require/module.exports) or "ES2015" (import/export keywords), but there are other module systems. module affects the module syntax of emitted code while target affects the rest.
What does Specify module code generation mean?
"module" in tsconfig.json tells the Typescript (TS) compiler what module syntax
to use when the files are compiled to Javascript (JS).
When you set "module" to "commonjs" in tsconfig.json, this means that the modules in the compiled .js files will use the commonJS (CJS) syntax, so var x = require(...) and module.exports = {...} to import and export.
If you changed "module" to "ES2015" for example, the compiled code would use the import and export keywords used in ES2015 module syntax. For an overview of the other syntaxes you can take a look here.
There are several different module systems with CJS and the
native ES Module (ESM) format probably being the ones most widely used.
What to choose depends on your requirements. If it's for a server-side project
that uses Node.js then probably CJS, if it's for an Angular front-end application
than perhaps ESM (or their own NgModules but that's going beyond scope here).
A somewhat similar situation is library and package designs and how you would
like to expose them to consumers. This depends on what sort of users are going to consume
the code, what are they working with (browser, Node) and which of the module systems
is best suited for the job?
ES Modules are now the built-in standard for importing/exporting modules in JS but back when there was no native solution other module systems were designed: This is why we also have CJS, AMD and UMD modules around. They are not all obsolete, CJS is still used a lot in Node.js and the AMD module loader for example allows non-JS imports which can be useful in some cases.
Nowadays, all the modern browsers and Node 13.2.0+ support the ESM format (see this page for compatibility data and more background on modules).
But then we have options like esnext
Newer JS versions sometimes contain more features for module import/export.
setting "module" to "ESNext" enables support for these features which often are not added to official specifications yet. Such as the import(...) expression which is a dynamic import.
Does it mean if I put module option as commonjs, then the compiler compiles the code to commonjs?
The "module" setting does not affect the rest of the code, "target" is used for that instead and specifies what JS version the outputs should be compatible with.
This has been explained in other threads, I'm just adding it here for clarity.
Say you want to use require(...) and module.exports = {...} in a Node project but also want the code to utilise ES2015 features like let and const in the code (for readability/performance/other reasons).
In this case you would set "module" to "commonjs" and "target" to "ES2015" in tsconfig.
Anyway the purpose of compiler is to compile the code into browser understandable syntax(commonjs).
Yes, the compiler has to turn TS code into JS that a browser understands.
However, JS is no longer limited to browsers, Node runs in other environments (servers) for example. CJS was in fact intended as a server-side module format while AMD modules were/are used for browser imports/exports.
So does it mean the module type you give tells the compiler in what syntax the code is written?
It tells the compiler in what syntax to write the modules in the output .js files
I have created some npm modules and compile them to:
commonJS (using exports.default =) and
esm (using export default)
I set up my package.json like so:
main: "index.cjs.js",
module: "index.esm.js"
When I npm install the package and I simple import it like:
import myPackage from 'my-package'
It will automatically choose the main file, not the module.
My question:
Is there a way to import the module file instead when doing import myPackage from 'my-package' in a JavaScript file?
Why I choose the commonJS file for "main":
I noticed that using Node, importing an esm file is not possible because of export default, it has to be commonJS. I have some simple helper JS functions like this and this, and I would want them to be usable to the widest audience. That's why I chose cjs for the "main" path in package.json.
Why I define a separate "module" in package.json:
Lots of famous libraries like Vue.js are already doing this. See further information on this Stackoverflow thread:
What is the "module" package.json field for?
Why do I want to be able to import the "module" file instead of the "main" file:
There is currently a bug in Rollup where it will not properly show JSDoc comments when coding after having imported a cjs file, but it does work when importing a es file.
The workaround I want to avoid:
Just set "main" to the esm file in package.json, right? But then all users who are using my packages in Node apps will not be able to use it anymore...
→ I'm really confused about all this as well, but I think I did enough research to make sense of all it. That being said, if anyone knows a better approach or any other advice, please do tell me in the comments down below!!
Just don't use extension for main file and have es6 and CommonJS version as two separate files with the same name and in the same directory, but with different extension, so:
index.js // transpiled CommonJS code for old nodejs
index.mjs // es6 module syntax
and in package.json:
{
"main": "index"
}
If node is launched with --experimental-modules flag, it would use *.mjs file, otherwise *.js.
Nodejs does not support "module" but does support the newer "exports" spec.
https://nodejs.org/api/packages.html#exports
https://github.com/nodejs/node/blob/v16.14.0/lib/internal/modules/esm/resolve.js#L910
"exports": {
"import": "./main-module.js",
"require": "./main-require.cjs"
},