I'm looking for a way to converting a full node-project's Babelimports into CommonJS-style require(). The goal is to get rid of Babel.
Considering node.js has things like async/await built-in nowadays it feels redundant to run Babel. The only thing left that Babel does currently is that it converts the ES6-style imports into require().
I've been searching but can't find any elegant solution to do it semi-automatically. The output when compiling Babel isn't clean enough to just copy without a lot of manual work.
If I have a file with input like this:
import express from 'express'
import bodyParser from 'body-parser'
import authMiddleware from './middlewares/auth'
import { get } from 'lodash'
export const myVar = 1
export default function doSomething() {
// ...
}
.. I'd want an output similar to this
const express = require('express')
const bodyParser = require('body-parser')
const authMiddleware = require('./middlewares/auth').default
const { get } = require('lodash')
export.myVar = 1
export.default = function doSomething() {
// ...
}
Alternatively that it converted the files to the .mjs-syntax for the relative ones and used require() for external stuff.
It's not the first time I have an old node project running Babel where it's turned more-and-more redundant with time, so I'm sure someone has done neat solution to this before.
I dig up the source code of babel-plugin-transform-modules-commonjs. Looks like it's impossible to config babel to output your desired result.
Reason behind is the necessity of helpers like _interopRequireDefault still holds strong, because ES module is not backward-compat to commonjs, notably the export default thing.
Take for example:
// input
import bodyParser from 'body-parser'
import authMiddleware from './middlewares/auth'
// your desired output
const bodyParser = require('body-parser') // <-- no default
const authMiddleware = require('./middlewares/auth').default // <-- default
// actual babel output
var _bodyParser = _interopRequireDefault(require("body-parser"));
var _auth = _interopRequireDefault(require("./middlewares/auth"));
You have no way to tell when to add .default and when not to. Only proper way to handle this is by wrapping require() with _interopRequireDefault and do runtime check.
If compiler does trace the required module and check if it's a ES module or commonjs module, then it can tell if .default is needed. However babel is designed around a single-file-at-a-time model, so no chance it can do that for you.
I think if you can figure out a reliable rule to tell when to add a .default and when not to then perhaps a simple regex-replace will solve your problem.
Side note. I do have some idea to hack it out with a customized babel plugin.
You can fork the babel-plugin-transform-modules-commonjs source, remove the _interopRequireDefault wrapping logic, then you use a resolver to do the aforementioned check-if-requiree-is-esmodule job, then see if .default is needed in output.
But easier said than done, this requires some serious effort.
The simple solution, open your source code in an editor that can go through all files I use VSCode and set a ignore on the node_modules folder and do I regex replace on all files the full way if you need the multiple exports is below.
RegEx way
Search for:
import[\s*]([a-zA-Z0-9,]*)[\s*]from[\s*]['|"]([a-zA-Z0-9\{\},\.\/\\]*)['|"][\s*]
replace with
const $1 = require('$2')
if you use as do this also.
Search for:
import[\s*][a-zA-Z0-9,]*[\s*]as[\s*]([a-zA-Z0-9]*)[\s*]from[\s*]['|"]([a-zA-Z0-9\{\},\.\/\\]*)['|"][\s*]
replace with
const $1 = require('$2')
there are some drawbacks here you can't use multiple exports for that you need The full way
Long way
Ok so for anyone who is interested, here is the process I used to work this out, you can then copy your source out of the build folder into a new location as a new source or overwrite your old src and the remove all of babel from your project (npm prune).
This will leave all the support stuff that module needs including the support for export default _interopRequireDefault() the only way to get rid of this is to create your own plugin that does not do this.
Step 1
Identify what ECMA babel was using for that. So I went to https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import
it shows that import has been part of the spec since ES6 (also known as ECMA2015)
Step 2
The presets are just groups of packages so identify the package for that particular transpile.
Opened my package.json and looked for babel-preset-es2015 found it. went to node_moduels\babel-preset-es2015, opened it's package.json to find
"babel-plugin-transform-es2015-modules-amd": "^6.24.1",
"babel-plugin-transform-es2015-modules-commonjs": "^6.24.1",
"babel-plugin-transform-es2015-modules-systemjs": "^6.24.1",
"babel-plugin-transform-es2015-modules-umd": "^6.24.1",
Step 3
Some testing so using --plugins= argument for babel I tested what each of them did on a small set of 2 files one requiring the other to and test each one I worked out it was the commonjs version that was needed for require();
Step 4
Do the conversion
So make sure you have the following node modules installed babel-cli, babel-core, babel-plugin-transform-es2015-modules-commonjs
Then fire up the CLI and do,
babel --plugins=transform-es2015-modules-commonjs ./src/ --out-dir build/
taken from https://babeljs.io/docs/en/babel-cli
You can use plugin of putout called #putout/plugin-convert-esm-to-commonjs for this purpose. It converts:
import {readFile} from 'fs/promises';
To:
const readFile = require('fs/promises');
To get things done set up the base line with:
npx putout . --disable-all
Enable convert-esm-to-commonjs in configuration file .putout.json:
{
"rules": {
"convert-esm-to-commonjs": "on"
}
}
And apply fixes:
npx putout . --fix
Related
The tutorial of webpack configuration for typescript shows something like this:
const path = require('path');
module.exports = { ... }
Would it not be better to use it as es module and configure it with e.g. imports. Or is there a reason why it is configured like above? I cant find any example where it is configured like this:
import webpack from "webpack";
import path from "path";
export default () => { ... }
TypeScript and the newer ES standard are supersets of normal JavaScript. Writing config files using widely supported syntax and features makes it more widely available and acceptable without requiring additional setup.
TypeScript is a better practice in some ways, but you need to introduce extra dependencies and configuration to use it, in some organizations you don't even have that freedom. Similar to ES, Node.js didn't have native support for mjs until v12.
The good news is that Webpack also supports writing configurations in multiple languages including TypeScript, see https://webpack.js.org/configuration/configuration-languages/
It's also available to get features like IntelliSense by using TypeScript JSDoc annotations if for some reason you can't write TypeScript directly:
/** #type { import('webpack').Configuration } */
const config = {...};
module.exports = config;
I'm building a discord.js Discord bot. Now for some reason, discord.js doesn't work with ESM modules (a totally separate issue), so my bot app uses CommonJS modules. Now I have another project on my system called Lib, which has a lot of utility functions that I plan to use in several different projects so I don't have to rewrite them. This Lib project uses ESM modules. Since I have to import Lib from DiscordBot, I use the dynamic import syntax in typescript. Now, whenever I transpile my DiscordBot project, the dynamic imports get converted into some ugly javascript module code, and that ugly module code ultimately ends up using require(). Since require() can't import ESM modules, my bot ends up crashing.
I tried however to stop my ts compiler, copy the code from my ts file that imports Lib then pasting that code into the corresponding JS file manually (and removing TS-exclusive features like type annotations and interfaces). Then I ran my bot app, and it worked perfectly fine. But I don't want to have to do this every time. So it's tsc's compiling that's the problem. How do I fix this?
So I understand the purpose is:
Develop the code in TypeScript
Run the compiled code in CommonJS package
Import and use an ES Module
Option 1:
If "module" in tsconfig.json is set to "commonjs", currently there's no way to prevent TypeScript from transpiling dynamic import() into require() - except that you hide the code in a string and use eval to execute it. Like this:
async function body (pMap:any){
// do something with module pMap here
}
eval ("import('p-map').then(body)");
No way TypeScript transpiles a string!
Option 2
Set "module" in tsconfig.json to "es2020". By doing this, dynamic import would not be transpiled into require(), and you can use dynamic import to import a CommonJS or ES Module. Or, you can use the const someModule = require("someModule") syntax to import a CommonJS module (would not be transpiled to ES6 import syntax). You cannot use the ES6 import syntax such as import * as someModule from "someModule" or import someModule from "someModule". These syntaxes will emit ES Module syntax imports ("module" is set to "es2020") and cannot be run in CommonJS package.
Below is a bit information:
If "module" is set to "es2020": dynamic import import() is not transpiled.
If "module" is set to `"es2015": there's an error:
TS1323: Dynamic imports are only supported when the '--module' flag is set to 'es2020', 'esnext', 'commonjs', 'amd', 'system', or 'umd'.
If "module" is set to "commonjs": dynamic imports are transpiled.
Quote tsconfig.json reference for "module" field:
If you are wondering about the difference between ES2015 and ES2020,
ES2020 adds support for dynamic imports, and import.meta.
This is currently not possible. There is a very new issue at GitHub (https://github.com/microsoft/TypeScript/issues/43329), but that is not implemented yet. So everything you can do now is to switch from ESM to CommonJS with your Lib project.
Update 2022
The issue has been closed and there is now a new option for "module" called node12. That should fix the problem
The node12 setting others are talking about did not work for me, but these compilerOptions did, using Typescript 4.7.2:
"module": "CommonJS",
"moduleResolution": "Node16",
This saved my backside, I did not have to migrate all import requires to imports to be able to use an ESM npm lib.
Typescript input source:
import Redis = require('redis');
import * as _ from 'lodash';
export async function main() {
const fileType = await import('file-type');
console.log(fileType, _.get, Redis);
}
CommonJS output:
...
const Redis = require("redis");
const _ = __importStar(require("lodash"));
async function main() {
const fileType = await import('file-type');
console.log(fileType, _.get, Redis);
}
exports.main = main;
What compiler/bundler are you using? I am assuming tsc based on context.
I recommend using esbuild to compile and bundle your TS. You can also use it simply to transform it after using tsc. It has an option called "format" that can remove any module-style imports. See https://esbuild.github.io/api/#format.
Here is a simple example of using.
build.js
const esbuild = require("esbuild");
esbuild.build({
allowOverwrite: true,
write: true,
entryPoints: ["my-main-file.ts"],
outfile: "some-file.bundle.js",
format: "cjs", //format option set to cjs makes all imports common-js style
bundle: true,
}).then(() => {
console.log("Done!");
});
You can then add something like this to your package.json
"scripts": {
"build": "node build.js",
...rest of scripts
Here is an additional link about some caveats using esbuild with typescript. None of these should really be a problem for you. https://esbuild.github.io/content-types/#typescript-caveats
This has been fixed with the addition of the node12 option for the module setting. From the docs:
Available in nightly builds, the experimental node12 and nodenext modes integrate with Node’s native ECMAScript Module support. The emitted JavaScript uses either CommonJS or ES2020 output depending on the file extension and the value of the type setting in the nearest package.json. Module resolution also works differently. You can learn more in the handbook.
If you use this setting without a nightly build, however, it currently produces the following error:
error TS4124: Compiler option 'module' of value 'node12' is unstable. Use nightly TypeScript to silence this error. Try updating with 'npm install -D typescript#next'.
I'm using a variant of the already mentioned eval-based hack to overcome this issue.
So for example, parse-domain is distributed as an ESM module, so importing it like this breaks in a CJS-based node app:
import { fromUrl, parseDomain } from 'parse-domain';
const parseDomainFromUrl = (url: string) => {
return parseDomain(fromUrl(url));
}
And this is how I have managed to get it working:
const dynamicImport = new Function('specifier', 'return import(specifier)');
const parseDomainFromUrl = (url: string) => {
return dynamicImport('parse-domain').then((module: any) => {
const { fromUrl, parseDomain } = module;
return parseDomain(fromUrl(url));
})
};
(Note that parseDomainFromUrl became asynchronous in the process, so it would need to be awaited by the caller.)
I'm searching a Flask-Assets filter that allows me to uglify javascript and support ES6 syntax. I tried to use uglifyjs-es binary instead of uglifyjs but I can't figure out how to configure my filter to use the uglifyjs-es binary.
I've this:
my_app_js = Bundle(
'js/MyApp.js',
filters='uglifyjs',
output='my_app_js.js'
)
From the Webassets documentation :
UglifyJS is an external tool written for NodeJS; this filter assumes
that the uglifyjs executable is in the path. Otherwise, you may define
a UGLIFYJS_BIN setting
Maybe the solution is there but I can't figure out where and how to change that UGLIFYJS_BIN setting, any idea ?
Also, I read here that uglifyjs-es project isn't maintenained anymore. terser seems to be the alternative, but could it be used as a filter too ?
Edit
If you know a good alternative to uglifyjs-es with a code example, you win a bounty ;)
If you have installed uglifyjs using npm, it should be in the node_modules folder in your project.
You could configure flask as follows:
app = Flask(__name__)
app.config['UGLIFYJS_BIN'] = 'path/to/node_modules/uglify-js/bin/uglifyjs'
Regarding uglifyjs-es, you know it's unmaintained. But if your code gets minified using it, it's still a good option.
As mentioned here:
uglify-js only supports ES5 code as input.
uglify-es also supports ES6, but is buggy and has been abandoned.
Terser's command line is almost as baroque as with ls. I found that easiest for me is to use Rollup with Terser plugin, and there's ready made Rollup filter for webassets. This way all Terser configuration is done in Rollup's config that you specify in filter extra args. Minimal Terser's configuration for ES6 modules:
{
compress: {ecma: 2015, module: true},
mangle: {module: true},
output: {ecma: 2015},
parse: {ecma: 2015},
rename: {},
}
With configured Terser plugin you may now use it as if it was both bundle and minify filters applied:
from flask_assets import Bundle
from webassets.filter import register_filter
from webassets_rollup import Rollup
register_filter(Rollup)
all_css = Bundle(
'css/app.scss', filters='node-scss,cleancss', output='dist/all.%(version)s.min.css',
)
all_js = Bundle(
'js/main.js', filters='rollup', output='dist/all.%(version)s.min.js',
)
Looks like you need to set an environment variable named UGLIFYJS_BIN. See here for Linux instructions, or over here for Windows instructions.
I am trying to convert my npm package imports from ES5 (require) to ES6 (import...from) but I keep getting unexpected identifier errors. What should I be looking out for?
I have read through forums but honestly not sure what to try. I checked the documentation in MDN but none of those worked. Every single package has been tried individually but none of them work.
// changed version
import fs from 'fs';
// old imports
// const fs = require('fs')
Because they're NPM modules - IIRC you have to use require for those.
I think you can change the command you use slightly and it should work if your Node version is high enough. You'll also need to change the file type from .js to .mjs (module JS):
node --experimental-modules moduleApp.mjs
I am looking for a convenient way to access files in the root of my application while avoiding require() strings that look like:
require('../../../../myModule')
There are some good solutions out there for Node (https://gist.github.com/branneman/8048520) but I haven't seen a way to use global variables in React Native.
Does anyone have a clean solution to this problem?
From Marc Shilling's answer on https://github.com/facebook/react-native/issues/3099
You can use an absolute path on imports/requires:
import {ProximaNovaText} from 'MyApp/src/components';
require('MyApp/src/utils/moment-twitter');
where 'MyApp' is whatever name you registered in your index.ios.js file
Note for VS Code: This works, but be warned that you might lose intellisense and cmd/ctrl + click. Thanks Johan for the info about CS code
You can mark a directory as a package by adding a package.json within the root directory you want to resolve.
e.g:
- app
- package.json // ← Add this package.json file
- config
- components
- ... (etc)
package.json should look like this:
{ "name": "app" }
Restart your packager
react-native start --reset-cache
Now you can use the following in all of you project files:
import store from 'app/config/store';
import Button from 'app/components/Button';
You can use this same method across other directories in your project, I don't think this works via require, although image paths seemed work.
As noted in the comments, you may lose auto-complete in your editor (VSCode).
For Jetbrains IDE's there are some ongoing tickets here:
https://youtrack.jetbrains.com/issue/WEB-17254
https://youtrack.jetbrains.com/issue/WEB-20104#comment=27-1486526
https://intellij-support.jetbrains.com/hc/en-us/community/posts/205434510-Configure-custom-modules-resolve-folder
This might help with Jetbrains IDE's in the meantime.
// A slash may allow auto-complete to work in your IDE (but will fail to resolve)
import Button from '/app/components/Button'; // Cannot be resolved
Put code below on top of your myModule file:
/**
* #providesModule myModule
*/
Then you can use require('myModule') in any other files.
Complementing #Tiagojdferreira
You can use his solution with babel-plugin-module-resolver library.
Install with:
npm install --save-dev babel-plugin-module-resolver
Configure .babelrc adding plugins property like this:
{
"presets": ["react-native"],
"plugins": [
["module-resolver", {
"alias": {
"#src": "MyApp/src",
"#otherAlias": "MyApp/src/other/path",
}
}]
]
}
Usage:
require('#src/utils/moment-twitter');
Hope this helps!
You can use global variables in react native, same as node, properties defined on global are accessible globally.
e.g.
global.foo = "blah";
console.log(foo); //logs "blah"
Most of the node solutions in the gist above should work correctly.
One I've used in the past is defining a global function at the top directory level, usually on the first line like
global.rootRequire = function(path) { return require(path); }
Which simply allows deeply nested requires to be from the root, and avoids all of the ../../ business.
However the other comment is true, if this is really an issue, there is probably something structurally deficient with the project.