Bundling non-native npm modules with Webpack - javascript

Description
I have a nodejs + TypeScript + express project and currently the source *.ts files are being bundled with webpack and node_modules is ignored with webpack-node-externals.
When I deploy bundle.js in Docker, I would need to still run npm i --production on the target image to install the dependencies, which installs all the modules listed in package.json
The Problem:
If I am using only one function from lodash which does not have native parts, the whole lodash module (4.8MB) is installed nonetheless (which is intended).
This results in a huge node_modules folder where functions inside packages aren't always necessarily used in bundle.js. This problem is especially prevalent when containerizing the application with Docker.
Is there any way to bundle non-native modules with Webpack while leaving native modules alone?

This is very similar to https://stackoverflow.com/a/54393299/2234013 - I believe you're looking for nodeExternals({ whitelist }) and babel-loader exclude:
// excerpt from https://stackoverflow.com/a/54393299/2234013
externals: [
nodeExternals({
whitelist: [/lodash/]
})
],
...
module: {
rules: [
{
...
exclude: /node_modules\/(?!(lodash).*/,
use: {
loader: 'babel-loader',
...
}
}
]
}

Related

React, Redux , Typescript and Sass bundled using Webpack and Vite

So I created two projects using React + Redux + SASS + Typescript with
Vite and Webpack.
I was amazed using Vite as the configuration part was pre handled as compared to Webpack. But again my project is big so I'd prefer Webpack over Vite.
LINK FOR VITE PROJECT : https://codesandbox.io/p/github/MrIndra/ReactRedux/csb-n4u7e3/draft/reverent-bohr?file=%2Fdist%2Findex.html&selection=%5B%7B%22endColumn%22%3A8%2C%22endLineNumber%22%3A6%2C%22startColumn%22%3A8%2C%22startLineNumber%22%3A6%7D%5D
LINK FOR WEBPACK PROJECT: https://codesandbox.io/p/github/MrIndra/react-sass-typescript-webpack/draft/affectionate-cookies?import=true&file=%2Fbuild%2Fmain.css&selection=%5B%7B%22endColumn%22%3A36%2C%22endLineNumber%22%3A4%2C%22startColumn%22%3A36%2C%22startLineNumber%22%3A4%7D%5D
,
Note : All the images test: /\.(?:ico|gif|png|jpg|jpeg)$/i, generated after build will be in their same parent folders as before build.
Current Problem
I have index.module.scss in the root directory which contains all the root level variables. Now again in every components, I have placed folder/component folder.module.scss files. Now npm run build is generating weird kind of css file.[image below]
The snapshot for the configuration of SCSS test: /\.s[ac]ss$/i, is
.
The folder.module.scss looks something like this
And the output where the buttons is not styled with the scss provided.
I assume you are trying to bundle all css/scss into one single css file. If so try following.
// install sass-loader and sass implementation of your choice
npm i sass-loader css-loader sass --save-dev
//install this plugin to extract CSS/Sass into separate or single file
npm i --save-dev mini-css-extract-plugin
and add following to your webpack config,
const MiniCssExtractPlugin = require("mini-css-extract-plugin");
module.exports = {
plugins: [new MiniCssExtractPlugin()],
module: {
rules: [
{
test: /\.s[ac]ss$/i,,
use: [MiniCssExtractPlugin.loader, "css-loader"],
},
],
},
};

Using local npm libraries with es6, babel, and webpack

I'm running into an issue that appears to be due to my lack of understanding with webpack. I have created a file structure that looks like this:
|-serverless-cloud-functions
||-my-local-libs
|||-twilioClient
||-service1
||-service2
twilioClient is a library that I made, that needs to be included in service1 and service2. Because of limitations with the serverless framework, you can not bundle files outside of the service, so the only option (I think) is to use a npm install ../my-local-libs/twilioClient from inside a service folder. This works for installing the module, but now it resides in node_modules. Currently, I am using webpack and babel as well.
I believe my root issue is that my webpack config looks like this:
const slsw = require("serverless-webpack");
const nodeExternals = require("webpack-node-externals");
module.exports = {
entry: slsw.lib.entries,
target: "node",
externals: [nodeExternals()],
module: {
rules: [
{
test: /\.js$/,
loader: "babel-loader",
exclude: /node_modules/
}
]
}
};
Which excludes my es6 twilioClient lib, because it is in the node_modules folder.
I saw a couple people suggest that this was the way to accomplish 'exclude everything in node modules besides twilioClient':
module.exports = {
entry: slsw.lib.entries,
target: "node",
externals: [nodeExternals()],
module: {
rules: [
{
test: /\.js$/,
loader: "babel-loader",
exclude: /node_modules\/(?!(twilioClient))/
}
]
}
};
But this did not work in my case. Any help is greatly appreciated.
Instead of trying to exclude twilioClient, you could use Babel to compile it separately. Something like this (in the twilioClient directory):
npx babel src --out-dir lib
In twilioClient/package.json, you could then set main to lib/index.js instead of src/index.js so that importing scripts will get the compiled version:
"main": "lib/index.js",
Then instead of hosting twilioClient alongside service1 and service2, you could just push it to github, and install it in each client using npm:
npm install --save http://github.com/your_github_username/twilioclient.git
Now you can use twilioClient as if it were any other npm dependency.
I have encountered a similar problem of using local common package with babel.
If you would like to not change the main since you also actively edit the package and do not want to run the build each time you make a change, then you should use the below babel.config.js
module.exports = function(api) {
api.cache(true);
const plugins = [
'#babel/plugin-transform-modules-commonjs',
];
return {
plugins,
ignore: [
/node_modules\/(?!(your-pkg)\/)/,
],
};
};
This transpiles the code for your-pkg into node_modules dir in dist.

How to config eslint-loader to lint NPM linked modules?

I am working on a large JavaScript project that's broken down into a few sub modules, each sub modules have their own git repository and Node dependencies. And we also have an entry point module, App, which has Webpack and ESLint configuration.
Since most of the developers would work on a few modules in tandem with each other, instead of waiting for each module is published, we use npm link to connect the sub-modules to the App module, so the changes to submodule during development time would be immediately visible to the App module.
We have run into issues to get eslint-loader to lint not only the App module source code, but all the submodules which are linked through npm link. When linting the submodule's code, eslint-loader failed to resolve the paths local to the submodule.
Is there a way for me to get around this problem?
Here is a sample structure of the project:
--- App
| src
| node_modules
|- Sub_module_x
| src
| node_modules
|- Sub_module_y
src
node_modules
Here is the relevant section of eslint-loader in webpack.config.js:
module: {
preloaders: [
{
test: /\.js$/,
loader: 'eslint-loader',
exclude: [
path.resolve(__dirname, 'node_modules', 'Sub_module_x'), // If this is commented, the loader would lint Sub_module_x, but cannot resolve any files which import other files
/node_modules/
]
}
]
}

React webpack / browserify "unexpected token"

I have this npm module that I created and every time I try to include it to see if it works I get this error:
Unexpected token <
You may need an appropriate loader to handle this file type.
I've used react-starterkit and included it in main.js like so
var ReactDOM = require('react-dom');
var ColorPicker = require('color-picker-react');
ReactDOM.render(<ColorPicker />, document.getElementById('app'));
then when i run gulp which runs webpack I get the error. Here's the webpack.config.js
module.exports.getConfig = function(type) {
var isDev = type === 'development';
var config = {
entry: './app/scripts/main.js',
output: {
path: __dirname,
filename: 'main.js'
},
debug : isDev,
module: {
loaders: [{
test: /\.jsx?$/,
exclude: /node_modules/,
loader: 'babel',
query: {
presets: ['react', 'es2015']
}
}]
}
};
if(isDev){
config.devtool = 'eval';
}
return config;
}
I've tried everything I could think of and still can't get it to work. I'm not using ES6 anywhere and I've tried many different react starter kits but I just can't get it to work. Please help!!!
P.S. I am able to get the project to run when I clone it locally and build out app.js with browserify like so: browserify -t [ babelify --presets [ react ] ] app.js -o bundle.js
To solve the problem you need to remove the line exclude: /node_modules/ if you are not the author of the npm module (But you should go with another module).
The component color-picker-react doesn't seem to have a release build or script that compile the jsx. So you need to do it by your own and compile the jsx file on the fly using wepack.
Instead of just removing the exclude: /node_modules/
You can exclude all /node_modules/ except the /node_modules/color-picker-react folder by using a regex pattern :
//will exclude all modules except `color-picker-react`
exclude: /node_modules\/(?!color-picker-react).*\//,
EDIT Basics for creating a npm module:
A correct setup for a npm module is to add a prepublish script to
ensure compilation happens automatically before uploading to NPM.
Thus when you push your module to npm the users doesn't need to compile the module they can just require it.
Taking an example of a node_module:
https://github.com/securingsincity/react-ace/blob/master/package.json
The package.json file is saying which file is the entry point when you required the module
"main": "lib/ace.js",
You can see in the github repository that the lib folder doesn't exist because added to the .gitignore but the line
"prepublish": "npm run clean && npm run build"
is run before uploading to NPM so on the npm repository the lib/ folder exist and you can see it when you do npm install --save react-ace the lib folder appears in the node_modules/react-ace/ folder
A great link that explains how to build npm modules in es6 for example http://javascriptplayground.com/blog/2015/10/authoring-modules-in-es6/
EDIT explain what needs to be done on react-color-picker module :
Sorry i didn't see that you was the author of the module so you should go with the solution below.
The react-color-picker for example doesn't have a prepublish script and the main file is index.js which is
var ColorPicker = require('./colorpicker.js'); // require a file with jsx will throw an error if not precompiled
module.exports = ColorPicker;
So a syntax error is thrown.
To be able to use the npm module in your other applications :
Create a webpack config for the npm module to handle the conversion of your react component written using jsx (you can take some of the webpack configs of this module https://github.com/securingsincity/react-ace set libraryTarget: 'umd' you module will be more easy to consume from various module systems (global, AMD, CommonJS).)
add a prepublish script that output a precompiled version of the color picker (lib/pickedprecompiled.js)
change the main to "main": "lib/pickedprecompiled.js",

"ReferenceError: Can't find variable: require" (using babel.js output) [duplicate]

When this code (generated from babel) runs I get an error exports is undefined
Object.defineProperty(exports, '__esModule', {
any ideas?
You are most likely not executing the code in an environment that supports CommonJS modules. You could use a bundler, such as Browserify or webpack
to bundle your modules into something that can be run in different environments.
Or you could choose a different module transformer.
With webpack
Run npm install -g webpack; npm install -D babel-loader. Then with this webpack configuration:
// webpack.config.js
module.exports = {
entry: "./path/to/entry/module.js",
output: {
path: __dirname,
filename: "bundle.js"
},
module: {
loaders: [
{ test: /\.js$/, exclude: /node_modules/, loader: "babel-loader"}
]
}
};
running the webpack command will convert all *.js files reachable via the entry file with babel and bundle them together into bundle.js.
I read an article about how ES6 import and export are only supposed to work in browser with "statically analyzable files" and Babel removed import and export support in the browser because of this. Something to do with async or possibly security?
If you want to skip the server-side bundling for dev purposes, you can put
window.MyModule = MyModule at the bottom, then import
var MyModule = window.MyModule at the top of your next file

Categories

Resources