Develop Reference

Webpack imports a very large existing chunk even if only a few modules are used

I’m working with Webpack 5 and trying to optimize the splitChunks configuration in a multi page application with a PHP backend and a Vue frontend.
In order to reduce the size of the vendor file I have started excluding some libraries by customizing the test function of the vendor cacheGroup.
test(module /* , chunk */) {
if (module.context) {
// only node_modules are needed
const isNodeModule = module.context.includes('node_modules');
// but we exclude some specific node_modules
const isToBundleSeparately = [
'marked', // only needed in one component
'braintree-web', // only payment page
'bootstrap/js',
].some(str => module.context.includes(str));
if (isNodeModule && !isToBundleSeparately) {
return true;
}
}
return false;
},
By doing so, some libraries that are not used in all pages are only imported in the components that need them, because those components are imported via dynamic imports and are extracted to separate chunks.
This has proven to work as expected until I encountered a strange behavior with one specific chunk and with one specific library (BootstrapVue).
If I add 'bootstrap-vue' to the list of excluded libraries, what happens is that two components of the library, BTab and BTabs, are extracted to a very large chunk which also includes the whole code for the payment page and all the libraries used in that page.
If you look at the screenshot, the file is the one whose name starts with “init-initPaymentGateway”.
So now all the pages that need those two BootstrapVue components are loading that large file, including the product page and other pages that only need the two tiny BootstrapVue components.
Here you can see that the chunk is imported in the product page:
I would expect that with my current configuration those two components would go to a separate chunk, or, if they are too small, they should be duplicated. Using Webpack Bundle Analyzer I see both very small files and duplicated library files, so I don’t understand why this is not happening with those specific components. The BootstrapVue BAlert component, for example, which is also small, is duplicated in different components.
I suspect that the issue comes from the fact that the components are small, but by setting minSize to 0 (or 10) I would expect to not have a minimum size for the creation of chunks.
Here are the imports in the payment page:
import { BTab, BTabs } from 'bootstrap-vue';
import dataCollector from 'braintree-web/data-collector';
(Then other inner components import other files from braintree-web).
Here is the import in one component of the product page:
import { BTabs, BTab } from 'bootstrap-vue';
And here is the complete splitChunks configuration (I have removed some of the excluded libraries from the list in the test function as they are not relevant).
splitChunks: {
chunks: 'all',
minSize: 10,
minChunks: 1,
maxAsyncRequests: 30,
maxInitialRequests: 30,
enforceSizeThreshold: 50000,
name(module, chunks /* , cacheGroupKey */) {
const allChunksNames = chunks
.filter(item => item.name !== null)
.map(item => item.name.replace('View', ''))
.join('~');
return allChunksNames.slice(0, 30);
},
cacheGroups: {
default: {
minChunks: 2,
priority: -20,
reuseExistingChunk: true,
},
defaultVendors: {
name: 'vendor',
priority: -10,
chunks: 'all',
minChunks: 1,
reuseExistingChunk: true,
test(module /* , chunk */) {
if (module.context) {
// only node_modules are needed
const isNodeModule = module.context.includes('node_modules');
// but we exclude some specific node_modules
const isToBundleSeparately = [
'marked', // only needed in one component
'bootstrap-vue',
'braintree-web',
'bootstrap/js',
].some(str => module.context.includes(str));
if (isNodeModule && !isToBundleSeparately) {
return true;
}
}
return false;
},
},
},
},
To fix the issue I removed 'bootstrap-vue' from the exclusion in the test function, but I’d like to understand if I can improve my configuration or at least the reason for this behavior.

By trying to fix another issue caused by a naming collision, I have modified the name function and fixed the issue.
Basically the slice function was causing a naming collision, which resulted in a chunk containing the code for more than one module, because both had the same name.
An explanation is given in the Webpack docs:
https://webpack.js.org/plugins/split-chunks-plugin/#splitchunksminsize, but it wasn't very clear to me until I experienced this issue:
Warning
When assigning equal names to different split chunks, all vendor modules are placed into a single shared chunk, though it's not recommend since it can result in more code downloaded.
A simple fix was to increase the length of the name by changing the last line from
return allChunksNames.slice(0, 30);
to
return allChunksNames.slice(0, 40);
But I still had the feeling that in some moment another naming clash could come up. I tried removing the slice part but it resulted in a Webpack error complaining about a name that was too long.
So in the end the long term solution was to simply delete the name function, and let Webpack assign a name.
name(module, chunks /* , cacheGroupKey */) {
const allChunksNames = chunks
.filter(item => item.name !== null)
.map(item => item.name.replace('View', ''))
.join('~');
return allChunksNames.slice(0, 40);
}
The resulting names are less clear, but I can always figure out what each chunk contains by using the Webpack Bundle Analyzer plugin.
Here is the chunk containing only the two modules that are needed in the product page.

What is the difference between multi-compiler and multiple-entry-points in webpack?

guys!! I'm trying to learn webpack and optimize my webpack project(a mutil-page project) configuration, but I got confused in some problems.hope someone can help me.
By the way,please forgive me for not having a good English, but I think Google Translate should basically be able to let me explain my problem. If there is something unclear in the description, please point it out and I will modify it.
Related Links
https://github.com/webpack/webpack/blob/main/examples/multiple-entry-points/webpack.config.js
https://github.com/webpack/webpack/blob/main/examples/multi-compiler/webpack.config.js
Q1.What are the advantages and disadvantages of each in mutil-html page config?
maybe multi-compiler have more flexible configuration but speed slower?
Q2.If I use the configuration of mutil-entry-point, how can I get the current entry name in the plugin or loader?
I know that the author of webpack said that this can't be done on the loader,
https://github.com/webpack/webpack/issues/6124#issuecomment-351647892
but how about in plugin?
Q3.Is there any way to get the entire dependency graph from the entry?
like webpack-bundle-analyzer?

The MultiCompiler allows you to create different rulesets and use different plugins for different files where the multi-target just allows you to compile multiple files and outputs. As for your questions:
Basically yes, multi-compiler does not run in parallel and needs to load the configuration, plugins and rulesets for each compilation. So it is more flexible, but runs slower, there are however alternatives to run it in parallel like parallel-webpack
You can create a custom plugin for your webpack and use their compiler-hooks to get what you want, more specifically I would check out the asset-emitted hook, that looks as follows:
compiler.hooks.assetEmitted.tap(
'MyPlugin',
(file, { content, source, outputPath, compilation, targetPath }) => {
console.log(content); // <Buffer 66 6f 6f 62 61 72>
}
);
If you wish to create your own plugin, this guide of theirs is essential: webpack writing-a-plugin
The best I can come up with is this from webpack-bundle-analyzer npm package :
When opened, the report displays all of the Webpack chunks for your project. It's possible to filter to a more specific list of chunks by using the sidebar or the chunk context menu.
Sidebar
The Sidebar Menu can be opened by clicking the > button at the top left of the report. You can select or deselect chunks to display under the "Show chunks" heading there.
Chunk Context Menu
The Chunk Context Menu can be opened by right-clicking or Ctrl-clicking on a specific chunk in the report. It provides the following options:
Hide chunk: Hides the selected chunk
Hide all other chunks: Hides all chunks besides the selected one
Show all chunks: Un-hides any hidden chunks, returning the report to its initial, unfiltered view
They clarify it in their documentation:
MultiCompiler:
The MultiCompiler module allows webpack to run multiple configurations in separate compilers. If the options parameter in the webpack's NodeJS api is an array of options, webpack applies separate compilers and calls the callback after all compilers have been executed.
var webpack = require('webpack');
webpack([
{ entry: './index1.js', output: { filename: 'bundle1.js' }, ruleset1, pluginsA},
{ entry: './index2.js', output: { filename: 'bundle2.js' }, ruleset2, pluginsB }
], (err, stats) => { // [Stats Object](#stats-object)
process.stdout.write(stats.toString() + '\n');
})
Multiple-entrypoints
If your configuration creates more than a single "chunk" (as with multiple entry points or when using plugins like CommonsChunkPlugin), you should use substitutions to ensure that each file has a unique name.
module.exports = {
entry: {
app: './src/app.js', // Will export to app.js
search: './src/search.js', // Will export to search.js
},
output: {
filename: '[name].js',
path: __dirname + '/dist',
},
};
Also, check out webpack-code splitting

How can I exclude multiple files from splitchunk webpack?

Question
I need to exclude multiple files from webpack optimization from splitchunk plugin.
What I've tried
const excludedFiles = ['file 1', 'file 2'];
module.exports = {
//...
optimization: {
splitChunks: {
chunks (chunk) {
// exclude `my-excluded-chunk`
return !excludedFiles.includes(chunk.name);
}
}
}
};
Result
It seems the files are not excluded.
How can achieve that?

You cannot do that. Note that chunk is not equal to a file. Modules (1 module ~ 1 file) are combined into chunks, chunks into chunk groups and groups into chunk graph. So, a filename doesn't mean it is chunk.
The options chunks in optimization.splitChunks means that which chuck to exclude or include during the optimization process. So, it means you can exclude a chunk but not individual file.
If you want to do what you are trying to achieve then, using externals is a way to go. It is not exactly same but almost equivalent to your needs.

Code-splitting separate exports from a package to different bundles

I have a Gatsby site that consumes a number of packages. One of those packages is published from our monorepo: #example/forms. That package contains a number of named exports, one for each form component that we use on our site. There are quite a large number of forms and some are relatively complex multistep forms of a non-trivial size.
Currently, Gatsby/Webpack does a good job of treeshaking, and does produce a large number of bundles, some common bundles and one for each page of the site, containing any local assets or components that are only used on that page. However, all the components from #example/forms are being added to the commons bundle, despite the fact that most are used on only a single page. This is causing unnecessary bloat of the commons bundle which is loaded on every page.
If feels like it should be possible for the individual components within #example/forms to be split out into the page-specific bundles, but I'm not sure if I'm hoping for too much. For example, if Form A is only used on page 4, I'd like Form A to only be added to the bundle for page 4.
Is this something that is supported, and if so, what could be preventing this from happening.
#example/forms is transpiled with ES6 exports left intact, and sideEffects is set to false in its package.json.
Its main file is index.js which (re)exports all the form components from their own files as separate named exports:
export {default as FormA} from './forms/formA'
export {default as FormB} from './forms/formB'
...
Another thing that might be relevant is that all the exports from #example/forms are used within the Gatsby site, just on separate pages. It appears that perhaps tree-shaking cannot be used across bundles, i.e. tree shaking is performed first, then what is left is split into bundles. Using that logic, #example/forms would have been used on multiple pages and would be moved to commons. However this is definitely not optimal, and hopefully isn't what is happening.

Tree-shaking process is part of the minification step which is really at late stage, since that it is not possible to reverse the order, first tree-shake then chunk split.
But you can split your forms lib before hand using some heuristics
The basic one is to use splitChunks in order to split this #example/forms module into a separate chunk as whole, this will decrease the commons bloat & on the first form usage all forms will be loaded.
// webpack.config.js
module.exports = {
...
optimization: {
splitChunks: {
cacheGroups: {
formComponents: {
chunks: 'all',
enforce: true,
minChunks: 1,
name: 'form-components',
priority: 10,
test: /[\\\/]node_modules[\\\/]#example[\\\/]forms[\\\/]/,
},
},
},
},
...
};
Make a chunk for each form inside the #example/forms module based on some heuristics, in my example I'm "grouping" all items based on the form path.
// webpack.config.js
module.exports = {
...
optimization: {
splitChunks: {
cacheGroups: {
formComponents: {
chunks: 'all',
minChunks: 1,
name(module) {
const libPath = path.resolve(path.join('node_modules/#example/forms'))
const folderBasedChunk = path.relative(libPath, module.context);
const hash = crypto.createHash('sha1');
hash.update(folderBasedChunk)
return hash.digest('hex').substring(0, 8)
},
priority: 10,
reuseExistingChunk: true,
enforce: true,
test: /[\\\/]node_modules[\\\/]#example[\\\/]forms[\\\/]src[\\\/]forms[\\\/]/,
},
},
},
...
};
You can checkout a small example that I've created that mimics the scenario.
https://github.com/felixmosh/webpack-module-split-example
Hope this helps

Load chunks/bundles as needed (like SystemJS)

Using Webpack, I have multiple chunks/bundles being created so that the entire app is not loaded at once. I've hand-chosen which dependencies I want to be moved into their own chunks. Here is the important part of my config:
module.exports = {
devtool: 'inline-source-map',
mode: process.env.NODE_ENV,
entry: {
main: './src/index.tsx',
},
optimization: {
runtimeChunk: {
name: 'runtime',
},
splitChunks: {
cacheGroups: {
...makeChunkCacheGroup('chunk_1', /\/node_modules\/(... list of deps ...)(\/|$)/),
...makeChunkCacheGroup('chunk_2', /\/node_modules\/(... list of deps ...)(\/|$)/),
},
},
},
// ...
};
function makeChunkCacheGroup(name, ...moduleNameRegexps) {
return {
[name]: {
name,
test: module => moduleNameRegexps.some(pattern => pattern.test(module.context)),
chunks: 'all',
minChunks: 1,
minSize: 0,
maxSize: Infinity,
reuseExistingChunk: true,
enforce: true,
},
};
}
This config gives me runtime, main, chunk_, and chunk_2. However, all of these chunks are injected into index.html, thus they all load during the initial page load instead of dynamically (as I naively expected).
I've used SystemJS in the past to bundle things up into multiple bundles and it would only download a given bundle as it was required by the app. I now realize that Webpack does not work this way.
Is there a way to make Webpack only download the runtime and main bundles initially, and then download the other bundles as they're needed?
Note 1: I realize that I can use dynamic imports e.g. import('some-dep').then(...), but it's not reasonable to do so based on the size of the codebase, and also, I think this sort of thing is better left to configuration (a module shouldn't have to pick and choose which deps it should load dynamically).
Note 2: I did try to specify multiple entry points but never got it working. The app really only has a single entry point. But, for instance, we have multiple directories under src/app/elements/, and it'd be perfect if each of those directories ended up in its own bundle which was then dynamically loaded. I couldn't get this working in an automated/smart way.