I have a modularised application based on NodeJS.
The structure of the application looks like
server.js
controllers
controller1.js
controller2.js
dao
dao1.js
dao2.js
node_modules
Now I want to minify and concatenate the entire project into a single file. The challenge I am facing is every controller and dao files are using "require" (relative paths) to include the node modules
Is there any easy way to achieve this?
Edit
Tried webpack module
My webpack.config.js looks like
var webpack = require('webpack');
var path = require('path');
var fs = require('fs');
var nodeModules = {};
fs.readdirSync(path.resolve(__dirname, 'node_modules'))
.filter(x => ['.bin'].indexOf(x) === -1)
.forEach(mod => { nodeModules[mod] = `commonjs ${mod}`; });
module.exports = {
name: 'server',
target: 'node',
entry: './server.js',
output: {
path: path.join(__dirname, 'build'),
filename: 'bundle.js'
},
externals: nodeModules,
module: {
loader: [
{
test: /\.json$/, loader: 'json-loader'
}
]
}
}
It is giving this error:
ERROR in ./~/npm/~/npm-registry-client/test/unpublish-scoped.js
Module not found: Error: Cannot resolve 'file' or 'directory' ./fixtures/#npm/np
m-registry-client/cache.json in D:\OTTAFW\node_modules\npm\node_modules\npm-regi
stry-client\test
Did you take a look at webpack?
https://webpack.js.org/concepts/
You can use a task runner like gulp to stitch and minify. See here if you want to bluntly stitch and minify files. In order to manage dependency you have to define a dependency tree and write the logic so that multiple dependencies are not added in the final build.
server.js
controllers
|-- controller1.js
require('lodash')
|-- controller2.js
require('./controller1')
dao
|-- dao1.js
require('lodash')
require('../controllers/controller2')
|-- dao2.js
require('../controllers/controller1')
node_modules
Hypothetically, if this is the case you have to careful of the fact that the files gets attached only once.
On the contrary, you can use rollup
Related
I'm trying to move from Gulp to Webpack. In Gulp I have task which copies all files and folders from /static/ folder to /build/ folder. How to do the same with Webpack? Do I need some plugin?
Requiring assets using the file-loader module is the way webpack is intended to be used (source). However, if you need greater flexibility or want a cleaner interface, you can also copy static files directly using my copy-webpack-plugin (npm, Github). For your static to build example:
const CopyWebpackPlugin = require('copy-webpack-plugin');
module.exports = {
context: path.join(__dirname, 'your-app'),
plugins: [
new CopyWebpackPlugin({
patterns: [
{ from: 'static' }
]
})
]
};
Compatibility note: If you're using an old version of webpack like webpack#4.x.x, use copy-webpack-plugin#6.x.x. Otherwise use latest.
You don't need to copy things around, webpack works different than gulp. Webpack is a module bundler and everything you reference in your files will be included. You just need to specify a loader for that.
So if you write:
var myImage = require("./static/myImage.jpg");
Webpack will first try to parse the referenced file as JavaScript (because that's the default). Of course, that will fail. That's why you need to specify a loader for that file type. The file- or url-loader for instance take the referenced file, put it into webpack's output folder (which should be build in your case) and return the hashed url for that file.
var myImage = require("./static/myImage.jpg");
console.log(myImage); // '/build/12as7f9asfasgasg.jpg'
Usually loaders are applied via the webpack config:
// webpack.config.js
module.exports = {
...
module: {
loaders: [
{ test: /\.(jpe?g|gif|png|svg|woff|ttf|wav|mp3)$/, loader: "file" }
]
}
};
Of course you need to install the file-loader first to make this work.
If you want to copy your static files you can use the file-loader in this way :
for html files :
in webpack.config.js :
module.exports = {
...
module: {
loaders: [
{ test: /\.(html)$/,
loader: "file?name=[path][name].[ext]&context=./app/static"
}
]
}
};
in your js file :
require.context("./static/", true, /^\.\/.*\.html/);
./static/ is relative to where your js file is.
You can do the same with images or whatever.
The context is a powerful method to explore !!
One advantage that the aforementioned copy-webpack-plugin brings that hasn't been explained before is that all the other methods mentioned here still bundle the resources into your bundle files (and require you to "require" or "import" them somewhere). If I just want to move some images around or some template partials, I don't want to clutter up my javascript bundle file with useless references to them, I just want the files emitted in the right place. I haven't found any other way to do this in webpack. Admittedly it's not what webpack originally was designed for, but it's definitely a current use case.
(#BreakDS I hope this answers your question - it's only a benefit if you want it)
Webpack 5 adds Asset Modules which are essentially replacements for common file loaders. I've copied a relevant portion of the documentation below:
asset/resource emits a separate file and exports the URL. Previously achievable by using file-loader.
asset/inline exports a data URI of the asset. Previously achievable by using url-loader.
asset/source exports the source code of the asset. Previously achievable by using raw-loader.
asset automatically chooses between exporting a data URI and emitting a separate file. Previously achievable by using url-loader with asset size limit.
To add one in you can make your config look like so:
// webpack.config.js
module.exports = {
...
module: {
rules: [
{
test: /\.(jpe?g|gif|png|svg|woff|ttf|wav|mp3)$/,
type: "asset/resource"
}
]
}
};
To control how the files get output, you can use templated paths.
In the config you can set the global template here:
// webpack.config.js
module.exports = {
...
output: {
...
assetModuleFilename: '[path][name].[hash][ext][query]'
}
}
To override for a specific set of assets, you can do this:
// webpack.config.js
module.exports = {
...
module: {
rules: [
{
test: /\.(jpe?g|gif|png|svg|woff|ttf|wav|mp3)$/,
type: "asset/resource"
generator: {
filename: '[path][name].[hash][ext][query]'
}
}
]
}
};
The provided templating will result in filenames that look like build/images/img.151cfcfa1bd74779aadb.png. The hash can be useful for cache busting etc. You should modify to your needs.
Above suggestions are good. But to try to answer your question directly I'd suggest using cpy-cli in a script defined in your package.json.
This example expects node to somewhere on your path. Install cpy-cli as a development dependency:
npm install --save-dev cpy-cli
Then create a couple of nodejs files. One to do the copy and the other to display a checkmark and message.
copy.js
#!/usr/bin/env node
var shelljs = require('shelljs');
var addCheckMark = require('./helpers/checkmark');
var path = require('path');
var cpy = path.join(__dirname, '../node_modules/cpy-cli/cli.js');
shelljs.exec(cpy + ' /static/* /build/', addCheckMark.bind(null, callback));
function callback() {
process.stdout.write(' Copied /static/* to the /build/ directory\n\n');
}
checkmark.js
var chalk = require('chalk');
/**
* Adds mark check symbol
*/
function addCheckMark(callback) {
process.stdout.write(chalk.green(' ✓'));
callback();
}
module.exports = addCheckMark;
Add the script in package.json. Assuming scripts are in <project-root>/scripts/
...
"scripts": {
"copy": "node scripts/copy.js",
...
To run the sript:
npm run copy
The way I load static images and fonts:
module: {
rules: [
....
{
test: /\.(jpe?g|png|gif|svg)$/i,
/* Exclude fonts while working with images, e.g. .svg can be both image or font. */
exclude: path.resolve(__dirname, '../src/assets/fonts'),
use: [{
loader: 'file-loader',
options: {
name: '[name].[ext]',
outputPath: 'images/'
}
}]
},
{
test: /\.(woff(2)?|ttf|eot|svg|otf)(\?v=\d+\.\d+\.\d+)?$/,
/* Exclude images while working with fonts, e.g. .svg can be both image or font. */
exclude: path.resolve(__dirname, '../src/assets/images'),
use: [{
loader: 'file-loader',
options: {
name: '[name].[ext]',
outputPath: 'fonts/'
},
}
]
}
Don't forget to install file-loader to have that working.
You can write bash in your package.json:
# package.json
{
"name": ...,
"version": ...,
"scripts": {
"build": "NODE_ENV=production npm run webpack && cp -v <this> <that> && echo ok",
...
}
}
Most likely you should use CopyWebpackPlugin which was mentioned in kevlened answer. Alternativly for some kind of files like .html or .json you can also use raw-loader or json-loader. Install it via npm install -D raw-loader and then what you only need to do is to add another loader to our webpack.config.js file.
Like:
{
test: /\.html/,
loader: 'raw'
}
Note: Restart the webpack-dev-server for any config changes to take effect.
And now you can require html files using relative paths, this makes it much easier to move folders around.
template: require('./nav.html')
I was stuck here too. copy-webpack-plugin worked for me.
However, 'copy-webpack-plugin' was not necessary in my case (i learned later).
webpack ignores root paths
example
<img src="/images/logo.png'>
Hence, to make this work without using 'copy-webpack-plugin'
use '~' in paths
<img src="~images/logo.png'>
'~' tells webpack to consider 'images' as a module
note:
you might have to add the parent directory of images directory in
resolve: {
modules: [
'parent-directory of images',
'node_modules'
]
}
Visit https://vuejs-templates.github.io/webpack/static.html
The webpack config file (in webpack 2) allows you to export a promise chain, so long as the last step returns a webpack config object. See promise configuration docs. From there:
webpack now supports returning a Promise from the configuration file. This allows to do async processing in you configuration file.
You could create a simple recursive copy function that copies your file, and only after that triggers webpack. E.g.:
module.exports = function(){
return copyTheFiles( inpath, outpath).then( result => {
return { entry: "..." } // Etc etc
} )
}
lets say all your static assets are in a folder "static" at the root level and you want copy them to the build folder maintaining the structure of subfolder, then
in your entry file) just put
//index.js or index.jsx
require.context("!!file?name=[path][name].[ext]&context=./static!../static/", true, /^\.\/.*\.*/);
In my case I used webpack for a wordpress plugin to compress js files, where the plugin files are already compressed and need to skip from the process.
optimization: {
minimize: false,
},
externals: {
"jquery": "jQuery",
},
entry: glob.sync('./js/plugin/**.js').reduce(function (obj, el) {
obj[path.parse(el).name] = el;
return obj
}, {}),
output: {
path: path.resolve(__dirname, './js/dist/plugin'),
filename: "[name].js",
clean: true,
},
That used to copy the js file as it is to the build folder. Using any other methods like file-loader and copy-webpack create issues with that.
Hope it will help someone.
So I have a Vue 3 + Typescript app. npm run build of course takes the app and compiles it into the dist folder so it can be deployed. I have a web worker typescript file that I would like to be compiled separately so that it ends up in the root of the dist folder with the name worker.js. Here is what I'm looking for:
dist
|- worker.js
src
|- worker.ts // This gets compiled to js file in dist folder
I tried doing this by using webpack's DefinePlugin in my vue.config.js like so:
const webpack = require('webpack')
module.exports = {
configureWebpack: {
plugins: [
new webpack.DefinePlugin({
entry: `${__dirname}/src/worker.ts`,
module: {
rules: [
{
test: /worker\.ts$/,
use: 'ts-loader',
exclude: /node-modules/
}
]
},
resolve: {
extensions: ['.ts']
},
output: {
filename: 'worker.js',
path: `${__dirname}/dist`
}
})
],
resolve: {
alias: {
vue$: 'vue/dist/vue.esm-bundler.js'
}
}
}
}
Unfortunately this doesn't work, npm run build just completely ignores the worker.ts file and it doesn't show up in the dist folder anywhere, not even as a chunk. Any suggestions? Or is what I'm wanting to do even possible? Thanks for any help!
I was able to get the desired result using esbuild. Now I can write my web worker in Typescript and use classes and functions from my Vue app. It also compiles lightning fast. I just added the node ./esbuild.js command to my npm run dev and npm run build scripts, and now it compiles to the public folder before the Vue app builds. Here is my esbuild.js file.
const esbuild = require('esbuild')
const { nodeExternalsPlugin } = require('esbuild-node-externals')
const config = {
entryPoints: ['./src/worker.ts'],
outfile: 'public/worker.js',
bundle: true,
minify: false,
logLevel: 'silent',
plugins: [nodeExternalsPlugin()]
}
esbuild.build(config).catch(() => process.exit(1))
Let me know if you have any questions, I'd be happy to help anyone out getting this working.
I have an existing node js backend that need to deploy on customer premise. I would like to compile/obfuscate the code before passing to client.
I do not know whether it is "compilable".
But at very least i want to have only single merged code into index.js file (other javascript file all remove) that is obfuscated before passing to client.
Is there any existing npm module that does that and how reliable that an obfuscated code work as it is then original code.
What are your company/ ways to deal with it when u need to pass an existing node js source code deployable at client premise.
I am much more looking at existing library that can automate the whole process.
eg: npx run obfuscate-code --entrypoint.js (it will search all the import/require from node js and compile everything.
Folder structure is as follow.
home
- controllers (folder)
- file1.js
- file2.js
- languages (folder)
- en.js
- services (folder)
- service1.js
- service2.js
- index.js (entry point)
You can make an executable using PKG (npm package) see => https://github.com/zeit/pkg
or webpack if you want to make a single js file that is minimized.
// webpack.config.js
const nodeExternals = require('webpack-node-externals');
module.exports = {
mode: 'production',
target: 'node',
externals: [nodeExternals()],
entry: {
'build/output': './src/index.js'
},
output: {
path: __dirname,
filename: '[name].bundle.js',
libraryTarget: 'commonjs2'
},
module: {
rules: [
{
test: /\.js$/,
use: {
loader: 'babel-loader',
options: {
presets: [
['env', {
'targets': {
'node': 'current'
}
}]
]
}
}
}]
}
};
By using nodeExternals you don't put external dependencies in the main js file but you refer to node_modules.
You can use both those 2 solutions using npm run ... by adding them to your scripts section in package.json
OK, i have searched high and low but cannot reliably deterrmine if this is or is not possible with webpack.
https://github.com/webpack/webpack/tree/master/examples/require.context
Appears to indicate that one can pass a string to a function and it load a module...
But my attempt is just not working:
webpack.config.js
'use strict';
let webpack = require('webpack'),
jsonLoader = require("json-loader"),
path = require("path"),
fs = require('fs'),
nodeModules = {};
fs.readdirSync('node_modules')
.filter(function(x) {
return ['.bin'].indexOf(x) === -1;
})
.forEach(function(mod) {
nodeModules[mod] = 'commonjs ' + mod;
});
let PATHS = {
app: __dirname + '/src'
};
module.exports = {
context: PATHS.app,
entry: {
app: PATHS.app+'/server.js'
},
target: 'node',
output: {
path: PATHS.app,
filename: '../build/server.js'
},
externals: nodeModules,
performance: {
hints: "warning"
},
plugins: [
jsonLoader
],
resolve: {
modules: [
'./node_modules',
path.resolve(__dirname),
path.resolve(__dirname + "/src"),
path.resolve('./config')
]
},
node: {
fs: "empty"
}
};
The server.js
let _ = require('lodash');
let modules = [ "modules/test" ];
require( 'modules/test' )();
_.map( modules, function( module ){
require( module );
});
The module in modules/ named test.js
module.exports = () => {
console.log('hello world');
};
But the result is always the same... the pm2 logs just say hello world for the static require... but for the dynamic load of the same module
Error: Cannot find module "."
All i want to be able to do is loop through an array of paths to modules and load then...
You cannot use a variable as argument to require. Webpack needs to know what files to bundle at compile time. As it does no program flow analysis, it can't know what you pass to the function. In that case it might be obvious, but this could go as far as using user input to decide what module to require, and there is no way webpack can possibly know which modules to include at compile time, so webpack does not allow it.
The example you posted is a bit different. You could use require with a concatenated string. For example:
require(`./src/${moduleName}/test`);
Which modules does webpack need to include in the bundle? The variable moduleName could be anything, so the exact module is not known at compile time. Instead it includes all modules that could possibly match the above expression. Assuming the following directory structure:
src
├─ one
│ └─ test.js
├─ two
│ ├─ subdir
│ │ └─ test.js
│ └─ test.js
└─ three
└─ test.js
All of these test.js files will be included in the bundle, because moduleName could be one or something nested like two/subdir.
For more details see require with expression of the official docs.
You cannot loop through an array and import every module of the array, with the above exception by concatenating a string, but that has the effect of including all possible modules and should generally be avoided.
I ran into this problem in an electron environment. My use case was being able to require dynamically created files in an IDE like application. I wanted to use the electron require, which is basically a NodeJS Common module loader. After some back and forth I landed on a solution that uses webpack's noParse module configuration.
First create a module that that will be ignored by webpack's parser:
// file: native-require.js
// webpack replaces calls to `require()` from within a bundle. This module
// is not parsed by webpack and exports the real `require`
// NOTE: since the module is unparsed, do not use es6 exports
module.exports = require
In my webpack config, under module, instruct the bundler not to parse this module:
{
module: {
noParse: /\/native-require.js$/,
}
}
Lastly, in any bundle where you want to access the original require:
import nativeRequire from './native-require'
const someModule = nativeRequire('/some/module.js') // dynamic imports
A bit late....but... since you are bundling to target: 'node', there is a workaround to dynamic requiring modules, and bypassing the "the effect of including all possible modules".
The solution is lifted from:
Using dynamic require on node targets WITHOUT resolve or bundle the target module · Issue #4175 · webpack/webpack
Quoted from that comment:
const requireFunc = typeof __webpack_require__ === "function" ? __non_webpack_require__ : require;
const foo = requireFunc(moduleName);
Bundles to:
const requireFunc = true ? require : require;
const foo = requireFunc(moduleName);
OK, i have searched high and low but cannot reliably deterrmine if this is or is not possible with webpack.
https://github.com/webpack/webpack/tree/master/examples/require.context
Appears to indicate that one can pass a string to a function and it load a module...
But my attempt is just not working:
webpack.config.js
'use strict';
let webpack = require('webpack'),
jsonLoader = require("json-loader"),
path = require("path"),
fs = require('fs'),
nodeModules = {};
fs.readdirSync('node_modules')
.filter(function(x) {
return ['.bin'].indexOf(x) === -1;
})
.forEach(function(mod) {
nodeModules[mod] = 'commonjs ' + mod;
});
let PATHS = {
app: __dirname + '/src'
};
module.exports = {
context: PATHS.app,
entry: {
app: PATHS.app+'/server.js'
},
target: 'node',
output: {
path: PATHS.app,
filename: '../build/server.js'
},
externals: nodeModules,
performance: {
hints: "warning"
},
plugins: [
jsonLoader
],
resolve: {
modules: [
'./node_modules',
path.resolve(__dirname),
path.resolve(__dirname + "/src"),
path.resolve('./config')
]
},
node: {
fs: "empty"
}
};
The server.js
let _ = require('lodash');
let modules = [ "modules/test" ];
require( 'modules/test' )();
_.map( modules, function( module ){
require( module );
});
The module in modules/ named test.js
module.exports = () => {
console.log('hello world');
};
But the result is always the same... the pm2 logs just say hello world for the static require... but for the dynamic load of the same module
Error: Cannot find module "."
All i want to be able to do is loop through an array of paths to modules and load then...
You cannot use a variable as argument to require. Webpack needs to know what files to bundle at compile time. As it does no program flow analysis, it can't know what you pass to the function. In that case it might be obvious, but this could go as far as using user input to decide what module to require, and there is no way webpack can possibly know which modules to include at compile time, so webpack does not allow it.
The example you posted is a bit different. You could use require with a concatenated string. For example:
require(`./src/${moduleName}/test`);
Which modules does webpack need to include in the bundle? The variable moduleName could be anything, so the exact module is not known at compile time. Instead it includes all modules that could possibly match the above expression. Assuming the following directory structure:
src
├─ one
│ └─ test.js
├─ two
│ ├─ subdir
│ │ └─ test.js
│ └─ test.js
└─ three
└─ test.js
All of these test.js files will be included in the bundle, because moduleName could be one or something nested like two/subdir.
For more details see require with expression of the official docs.
You cannot loop through an array and import every module of the array, with the above exception by concatenating a string, but that has the effect of including all possible modules and should generally be avoided.
I ran into this problem in an electron environment. My use case was being able to require dynamically created files in an IDE like application. I wanted to use the electron require, which is basically a NodeJS Common module loader. After some back and forth I landed on a solution that uses webpack's noParse module configuration.
First create a module that that will be ignored by webpack's parser:
// file: native-require.js
// webpack replaces calls to `require()` from within a bundle. This module
// is not parsed by webpack and exports the real `require`
// NOTE: since the module is unparsed, do not use es6 exports
module.exports = require
In my webpack config, under module, instruct the bundler not to parse this module:
{
module: {
noParse: /\/native-require.js$/,
}
}
Lastly, in any bundle where you want to access the original require:
import nativeRequire from './native-require'
const someModule = nativeRequire('/some/module.js') // dynamic imports
A bit late....but... since you are bundling to target: 'node', there is a workaround to dynamic requiring modules, and bypassing the "the effect of including all possible modules".
The solution is lifted from:
Using dynamic require on node targets WITHOUT resolve or bundle the target module · Issue #4175 · webpack/webpack
Quoted from that comment:
const requireFunc = typeof __webpack_require__ === "function" ? __non_webpack_require__ : require;
const foo = requireFunc(moduleName);
Bundles to:
const requireFunc = true ? require : require;
const foo = requireFunc(moduleName);