I've created git repo with simple example of my project
https://github.com/paveleremin/webpack-build
The question is: how to build it with multiple chunks, like:
components (all js files from /app/components/ folder that used in two or more modules)
vendors (all js files from /node_modules/ and /app/vendor/ folders that used in two or more modules)
manifest (webpack js code, babel-polifill etc)
per module js files
Right now build had a problems with:
new webpack.optimize.CommonsChunkPlugin({
async: 'components',
children: true,
minChunks({ resource }, count) {
return resource && resource.includes(paths.components) && count > 1;
}
}),
new webpack.optimize.CommonsChunkPlugin({
async: 'vendors',
children: true,
minChunks({ resource }, count) {
console.log(resource, count);
return resource && (resource.includes(paths.nodeModules) || resource.includes(paths.vendor)) && count > 1;
}
}),
new webpack.optimize.CommonsChunkPlugin({
name: 'manifest',
}),
source of scUsedTwice is copy-paste and exists in login.js and dashboard.js as well - SOLVED
all vendors are in app.js
I've tried do it myself, but have a problem with async modules.
There are a lot of questions in here!
I'll answer the vendor and manifest questions since they're related.
There are multilple ways of doing this depending on how you dev, how frequent the flies are likely to change and what is highest priority for you (build time?).
You could use a 'dll' (https://webpack.js.org/plugins/dll-plugin/) which is good for when you know the exactly what files you want to go into the vendor/dll bundle and you know the vendor files will rarely change. This method has a super fast build time as it only builds it once / when needed.
The way i have done it by explicitly having a 'vendor.js' file which includes my known 3rd party packages. I then setup webpack to treat this file differently. The benefit of this is the vendor file is more explicit and easier to update, but the build will be slightly slower.
entry: {
app: [`${SRC}/client-entry.js`],
vendor: [`${SRC}/vendor.js`]
}
plugins: [
new webpack.optimize.CommonsChunkPlugin({ names: ['vendor'], minChunks: Infinity }),
]
To create a Manifest, the method is similar to the above. You need to also use the common chunks plugin and add:
new webpack.optimize.CommonsChunkPlugin({ name: "manifest", minChunks: Infinity }
The best source i've found, if you've time to go through it is : https://survivejs.com/webpack/optimizing/separating-manifest/
May I suggest posting you async and copy-paster problem in a separate question?
Related
I have a Gatsby site that consumes a number of packages. One of those packages is published from our monorepo: #example/forms. That package contains a number of named exports, one for each form component that we use on our site. There are quite a large number of forms and some are relatively complex multistep forms of a non-trivial size.
Currently, Gatsby/Webpack does a good job of treeshaking, and does produce a large number of bundles, some common bundles and one for each page of the site, containing any local assets or components that are only used on that page. However, all the components from #example/forms are being added to the commons bundle, despite the fact that most are used on only a single page. This is causing unnecessary bloat of the commons bundle which is loaded on every page.
If feels like it should be possible for the individual components within #example/forms to be split out into the page-specific bundles, but I'm not sure if I'm hoping for too much. For example, if Form A is only used on page 4, I'd like Form A to only be added to the bundle for page 4.
Is this something that is supported, and if so, what could be preventing this from happening.
#example/forms is transpiled with ES6 exports left intact, and sideEffects is set to false in its package.json.
Its main file is index.js which (re)exports all the form components from their own files as separate named exports:
export {default as FormA} from './forms/formA'
export {default as FormB} from './forms/formB'
...
Another thing that might be relevant is that all the exports from #example/forms are used within the Gatsby site, just on separate pages. It appears that perhaps tree-shaking cannot be used across bundles, i.e. tree shaking is performed first, then what is left is split into bundles. Using that logic, #example/forms would have been used on multiple pages and would be moved to commons. However this is definitely not optimal, and hopefully isn't what is happening.
Tree-shaking process is part of the minification step which is really at late stage, since that it is not possible to reverse the order, first tree-shake then chunk split.
But you can split your forms lib before hand using some heuristics
The basic one is to use splitChunks in order to split this #example/forms module into a separate chunk as whole, this will decrease the commons bloat & on the first form usage all forms will be loaded.
// webpack.config.js
module.exports = {
...
optimization: {
splitChunks: {
cacheGroups: {
formComponents: {
chunks: 'all',
enforce: true,
minChunks: 1,
name: 'form-components',
priority: 10,
test: /[\\\/]node_modules[\\\/]#example[\\\/]forms[\\\/]/,
},
},
},
},
...
};
Make a chunk for each form inside the #example/forms module based on some heuristics, in my example I'm "grouping" all items based on the form path.
// webpack.config.js
module.exports = {
...
optimization: {
splitChunks: {
cacheGroups: {
formComponents: {
chunks: 'all',
minChunks: 1,
name(module) {
const libPath = path.resolve(path.join('node_modules/#example/forms'))
const folderBasedChunk = path.relative(libPath, module.context);
const hash = crypto.createHash('sha1');
hash.update(folderBasedChunk)
return hash.digest('hex').substring(0, 8)
},
priority: 10,
reuseExistingChunk: true,
enforce: true,
test: /[\\\/]node_modules[\\\/]#example[\\\/]forms[\\\/]src[\\\/]forms[\\\/]/,
},
},
},
...
};
You can checkout a small example that I've created that mimics the scenario.
https://github.com/felixmosh/webpack-module-split-example
Hope this helps
using webpack v2, i have two entry points which should generate two bundles that can be loaded on the same page or different pages separately. I'm also using the CommonsChunkPlugin to extract all dependencies from the node_modules folder to a 3rd bundle which is only needed by the first bundle. When loading the second bundle alone on a page, it does not get initialized, because it seems to need the commonChucnks bundle to have been loaded before (hosting the webpackJsonp function) is there an option other than having two webpack configs to solve my problem?
module.exports = {
context: path.resolve(__dirname),
entry: {
"first": "./src/js/first.js",
"second": "./src/js/second.js",
},
output: {
path: path.resolve(__dirname, "target/js"),
filename: "[name].js",
libraryTarget: "umd"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({
names: "vendor",
// automatically determine which files are from the node modules. And put them in a separate "lib" bundle
// #see https://webpack.js.org/guides/code-splitting-libraries/#commonschunkplugin
// this assumes your vendor imports exist in the node_modules directory
minChunks: module => module.context && module.context.indexOf("node_modules") !== -1
}),
"(two entrypoints)... that can be loaded on the same page".
This is not how it supposed to be. Entry point - is an entry point - it can be only once loaded.
What you need is: two entrypoints chunks, with "specific" code chunks" and one common code chunk.
webpack version
2.2.1
I'm trying to add an additional manifest file in order to bypass webpacks runtime code injection issue and enable caching:
https://webpack.js.org/guides/code-splitting-libraries/#manifest-file
however, when passing a function into minChunks, in this case - in order to automatically get all the files that are in node_modules inside the vendor.js chunk - will result an unexpected outcome: only the last file in the array (manifest, in the example below) is generated.
webpack.config.js
entry: {
bundle: "./src/index.tsx",
},
output: {
filename: "[name].js?[chunkhash]",
path: `${projectRoot}/dist`
},
plugins: [new webpack.optimize.CommonsChunkPlugin({
names: ['vendor', 'manifest'],
minChunks: function (module) {
return module.context && module.context.indexOf('node_modules') !== -1;
}
})]
expected output
3 files: bundle.js, vendor.js and manifest.js
actual output
2 files: bundle.js, manifest.js
I observed that with the config above, the webpack v2.2.1, would create just the last one (i.e. manifest in your case) as it first thinks vendor as parent for the 'bundle' bundle and process ... next it thinks manifest as parent of 'bundle' bundle and process which would overwrite the previous processing. Finally while generating bundle, it create two bundle, the 'bundle' bundle and its parent - manifest bundle. I am not sure why it behaves like this with v2.2.1.
You would want to write your config like this
plugins: [new webpack.optimize.CommonsChunkPlugin({
name: 'vendor',
chunks: ["bundle"], //optional
minChunks: function (module) {
return module.context && module.context.indexOf('node_modules') !== -1;
}
}),
new webpack.optimize.CommonsChunkPlugin({
name: "manifest",
chunks: ["vendor"],
minChunks: Infinity
})]
The trick here is that with the first plugin instance, you will be extracting the modules coming from node_modules to the vendor bundle. Since this is the parent common bundle, webpack will add its runtime code into vendor bundle (if the second instance of plugin is not added). Then with the application of second instance of the plugin, using minChunks as Infinity, you will not be extracting any module, but you will make the manifest bundle as parent bundle for vendor, hence, webpack will add its runtime code into the manifest bundle.
You need to make sure that manifest script should get executed first, then vendor script and finally bundle script. In other words, load them sequentially to avoid unexpected errors.
I'm trying out the multi-compiler option in webpack and am following the example at their github. However, I can't seem to understand how I can split out the common code amongst the multiple configurations.
For example, I may have same vendor libraries used in the different set of configurations. I would like to have these shared codes to be bundled to one single common file.
I tried the following but it ended up creating an individual bundles of the vendors entry for each compile configuration.
var path = require("path");
var webpack = require("webpack");
module.exports = [
{
name: "app-mod1",
entry: {
vendors: ['jquery', 'react', 'react-dom'],
pageA: ['./mod1/pageA'],
pageB: ['./mod1/pageB']
},
output: {
path: path.join(__dirname, "/mod1/js"),
filename: "[name].bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({
names: ['vendors'],
minChunks: Infinity
})
]
},
{
name: "app-mod2",
entry: {
vendors: ['lodash', 'react', 'react-dom'],
pageA: ['./mod2/pageA'],
pageB: ['./mod2/pageB']
},
output: {
path: path.join(__dirname, "/mod2/js"),
filename: "[name].bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({
names: ['vendors'],
minChunks: Infinity
})
]
}
];
Since react, react-dom are shared between the 2 compilations, my intention is for them to be bundled as a single file which can be shared instead of creating a same bundle for each compilation.
How can I extract the common chunks out of multiple compiler configurations?
Brief answer
You can't do that job in the way you want.
TL;DR
#Carven, I am afraid that you can't achieve your goal via MultiCompiler of Webpack, MultiCompiler is not meant to do that job, at least for the near feature.
See the source code for initiating the MultiCompiler instance, it actually initiates separate Compiler instances. These compiler have no data shared between.
See also the source of running MultiCompiler instance, the compilers instance also run separately without sharing data.
The only thing these compilers share is the Stats instance they produce and merge into a MultiStats.
By the way, there is no clue in the example you mentioned that some modules are shared between multi-compilers.
Alternative
As described by #Tzook-Bar-Noy, IMHO, you have to use muti-entries to achieve MPA(Multi-page Application) bundling.
Other worth mentioning
I noticed a library called webpack-multi-configurator is using the multi-compiler feature. But I don't think it will share common chunk(s).
You can extract the shared code into another compilation, and bundle it with DllBundlesPlugin.
later consume this DLL via DLLReferencePlugin and add it to your page either manually or via HTMLWebpackPlugin's add-asset-html-webpack-plugin
Bolierplate can be reduced by using autodll-webpack-Plugin
I learned about it now, and this topic seems quite hard to understand in webpack docs. I managed to create something that works, as it created 2 separate files and extracted the common dependencies to another file.
Here is my webpack config:
{
entry: {
pageA: "./first/first",
pageB: "./second/second"
},
output: {
path: path.join(__dirname, "js"),
filename: "[name].js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({
names: ["vendor", "common"],
})
]
};
the output of this will be:
./js/
common.js
vendor.js
pageA.js
pageB.js
I created a repo with the example I worked on: https://github.com/tzookb/webpack-common-vendor-chunks
when I open a new html file I load these files:
first page:
common.js
vendor.js
pageA.js
sec page:
common.js
vendor.js
pageB.js
Folder Structure:
app.js, benchmark.js, board.js all require jquery. I just want to extract jquery as vender.js and three other bundles only contain application code:
Webpack Config:
The result is not what I expected:
app.js, benchmark.js, board.js still contains jquery code (as you can see from the huge file size)
Is there anything wrong with my webpack configuration?
I just followed the example in :
https://github.com/webpack/webpack/tree/master/examples/two-explicit-vendor-chunks
https://github.com/webpack/webpack/tree/master/examples/multiple-entry-points
plugins should be an object array outside of modules.
Also, I don't think you need the minChunks or chunks options for this use case scenario. Your vendor entry chunk should be sufficient.
entry: {
vendor: ['jquery']
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({
name: "vendor",
filename:"vendor.js",
minChunks: Infinity
})
];