Develop Reference

WebPack configuration- what is the proper code and what does it mean

I'm learning React and want to understand how web pack is configured for a project.
It would be great if someone can tell me what the following lines of code are doing.
const fs = require('fs')
const path = require('path')
const webpack = require('webpack')
function isDirectory(dir) {
return fs.lstatSync(dir).isDirectory()
}
const SubjectsDir = path.join(__dirname, 'subjects')
const SubjectDirs = fs.readdirSync(SubjectsDir).filter(function (dir) {
return isDirectory(path.join(SubjectsDir, dir))
})
module.exports = {
devtool: 'source-map',
entry: SubjectDirs.reduce(function (entries, dir) {
if (fs.existsSync(path.join(SubjectsDir, dir, 'exercise.js')))
entries[dir + '-exercise'] = path.join(SubjectsDir, dir, 'exercise.js')
if (fs.existsSync(path.join(SubjectsDir, dir, 'solution.js')))
entries[dir + '-solution'] = path.join(SubjectsDir, dir, 'solution.js')
if (fs.existsSync(path.join(SubjectsDir, dir, 'lecture.js')))
entries[dir + '-lecture'] = path.join(SubjectsDir, dir, 'lecture.js')
return entries
}, {
shared: [ 'react', 'react-dom' ]
}),
output: {
path: '__build__',
filename: '[name].js',
chunkFilename: '[id].chunk.js',
publicPath: '__build__'
},
resolve: {
extensions: [ '', '.js', '.css' ]
},
module: {
loaders: [
{ test: /\.css$/, loader: 'style!css' },
{ test: /\.js$/, exclude: /node_modules|mocha-browser\.js/, loader: 'babel' },
{ test: /\.woff(2)?$/, loader: 'url?limit=10000&mimetype=application/font-woff' },
{ test: /\.ttf$/, loader: 'file' },
{ test: /\.eot$/, loader: 'file' },
{ test: /\.svg$/, loader: 'file' },
{ test: require.resolve('jquery'), loader: 'expose?jQuery' }
]
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({ name: 'shared' })
],
devServer: {
quiet: false,
noInfo: false,
historyApiFallback: {
rewrites: [
{ from: /ReduxDataFlow\/exercise.html/,
to: 'ReduxDataFlow\/exercise.html'
}
]
},
stats: {
// Config for minimal console.log mess.
assets: true,
colors: true,
version: true,
hash: true,
timings: true,
chunks: false,
chunkModules: false
}
}
}
This informaiton is coming from a training course, but they do not explain what the lines are doing.

Webpack is what we call a module bundler for JavaScript applications. You can do a whole slew of things with it that help a client browser download and run your code. In the case of React, it helps convert JSX code into plain JS so that the browser can understand it. JSX itself will not run in the browser. We can even use plugins to help minify code, inject HTML, bundle various groups of code together, etc. Now that the introduction to Webpack is out of the way, let's take a look at the code. I will be starting from the very top. Feel free to skip down to #3 if you are only interested in the Webpack configuration object.
The following code will require the modules that are needed in this file. fs is short for "filesystem" and is a module that gives you functions to run that can access the project's filesystem. path is a common module used to resolve or create pathnames to files and is very easy to use! And then we have the webpack module through which we can access webpack specific functions (ie: webpack plugins like webpack.optimize.UglifyJsPlugin).
const fs = require('fs')
const path = require('path')
const webpack = require('webpack')
These next few lines first set up a helper function to determine whether or not something in the filesystem being read is a directory.
function isDirectory(dir) {
return fs.lstatSync(dir).isDirectory()
}
const SubjectsDir = path.join(__dirname, 'subjects')
const SubjectDirs = fs.readdirSync(SubjectsDir).filter(function (dir) {
return isDirectory(path.join(SubjectsDir, dir))
})
devtool specifies which developer tool you want to use to help with debugging. Options are listed here : https://webpack.github.io/docs/configuration.html#devtool. This can be very useful in helping you determine exactly which files and lines and columns errors are coming from.
devtool: 'source-map'
These next few lines tell Webpack where to begin bundling your files. These initial files are called entry points. The entry property in a Webpack configuration object should be an object whose keys determine the name of a bundle and values point to a relative path to the entry file or the name of a node_module. You can also pass in an array of files to each entry point. This will cause each of those files to be bundled together into one file under the name specified by the key - ie: react and react-dom will each be parsed and have their outputs bundled under the name shared.
entry: SubjectDirs.reduce(function (entries, dir) {
if (fs.existsSync(path.join(SubjectsDir, dir, 'exercise.js')))
entries[dir + '-exercise'] = path.join(SubjectsDir, dir, 'exercise.js')
if (fs.existsSync(path.join(SubjectsDir, dir, 'solution.js')))
entries[dir + '-solution'] = path.join(SubjectsDir, dir, 'solution.js')
if (fs.existsSync(path.join(SubjectsDir, dir, 'lecture.js')))
entries[dir + '-lecture'] = path.join(SubjectsDir, dir, 'lecture.js')
return entries
}, {
shared: [ 'react', 'react-dom' ]
}),
In the reduce function we simply read through the SubjectsDir, determine whether files exercise.js, lecture.js & solution.js exist, and then provide the path to those files as values associated with the key names identified by dir + '-' + filename (ie: myDir-exercise). This may end up looking like the following if only exercise.js exists:
entry : {
'myDir-exercise': 'subjectDir/myDir/exercise.js',
share: ['react', 'react-dom']
}
After we provide entry points to the Webpack configuration object, we must specify where we want Webpack to output the result of bundling those files. This can be specified in the output property.
output: {
path: '__build__',
filename: '[name].js',
chunkFilename: '[id].chunk.js',
publicPath: '__build__'
},
The path property defines the absolute path to the output directory. In this case we call it __build__.
The filename property defines the output name of each entry point file. Webpack understands that by specifying '[name]' you are referring to the key you assigned to each entry point in the entry property (ie: shared or myDir-exercise).
The chunkFilename property is similar to the filename property but for non-entry chunk files which can be specified by the CommonChunksPlugin (see below). The use of [id] is similar to the use of [name].
The publicPath property defines the public URL to where your files are located, as in the URL from which to access your files through a browser.
The resolve property tells Webpack what how to resolve your files if it can not find them for some reason. There are several properties we can pass here with extensions being one of them. The extensions property tells Webpack which file extensions to try on a file if one is not specified in your code.
resolve: {
extensions: [ '', '.js', '.css' ]
},
For example, let's say we have the following code
const exercise = require('./exercise');
We can leave out the .js because we have provided that string in the resolve property of the webpack configuration and Webpack will try and append .js to this at bundling time to find your file. As of Webpack 2 we also no longer need to specify an empty string as the first element of the resolve property.
The module property tells Webpack how modules within our project will be treated. There are several properties we can add here and I suggest taking a look at the documentation for more details. loaders is a common property to use and with that we can tell Webpack how to parse particular file types within our project. The test property in each loader is simply a Regex that tells Webpack which files to run this loader on. This /\.js$/ for example will run the specified loader on files that end with .js. babel-loader is a widely used JavaScript + ES6 loader. The exclude property tells Webpack which files to not run with the specified loader. The loader property is the name of the loader. As of Webpack 2 we are no longer able to drop the -loader from the string as we see here.
Plugins have a wide range of functions. As mentioned earlier we can use plugins to help minify code or to build chunks that are used across our entire application, like react and react-dom. Here we see the CommonChunksPlugin being used which will bundle the files under the entry name shared together as a chunk so that they can be separated from the rest of the application.
Finally we have the devServer property which specifies certain configurations for the behavior of webpack-dev-server, which is a separate module from webpack. This module can be useful for development in that you can opt out of building your own web server and allow webpack-dev-server to serve your static files. It also does not write the outputs to your filesystem and serves the bundle from a location in memory at the path specified by the publicPath property in the output property of the Webpack configuration object (see #5). This helps make development faster. To use it you would simply run webpack-dev-server in place of webpack. Take a look at the documentation online for more details.
The configuration object that we've taken a look at follows the Webpack 1 standard. There are many changes between Webpack 1 and 2 in terms of configuration syntax that would be important to take a look at. The concepts are still the same, however. Take a look at the documentation for information on migrating to Webpack 2 and for further Webpack 2 details.

webpack2 CommonsChunkPlugin with manifest file, only generates the last file in the "names[]" array if minChunks is a function

webpack version
2.2.1
I'm trying to add an additional manifest file in order to bypass webpacks runtime code injection issue and enable caching:
https://webpack.js.org/guides/code-splitting-libraries/#manifest-file
however, when passing a function into minChunks, in this case - in order to automatically get all the files that are in node_modules inside the vendor.js chunk - will result an unexpected outcome: only the last file in the array (manifest, in the example below) is generated.
webpack.config.js
entry: {
bundle: "./src/index.tsx",
},
output: {
filename: "[name].js?[chunkhash]",
path: `${projectRoot}/dist`
},
plugins: [new webpack.optimize.CommonsChunkPlugin({
names: ['vendor', 'manifest'],
minChunks: function (module) {
return module.context && module.context.indexOf('node_modules') !== -1;
}
})]
expected output
3 files: bundle.js, vendor.js and manifest.js
actual output
2 files: bundle.js, manifest.js

I observed that with the config above, the webpack v2.2.1, would create just the last one (i.e. manifest in your case) as it first thinks vendor as parent for the 'bundle' bundle and process ... next it thinks manifest as parent of 'bundle' bundle and process which would overwrite the previous processing. Finally while generating bundle, it create two bundle, the 'bundle' bundle and its parent - manifest bundle. I am not sure why it behaves like this with v2.2.1.
You would want to write your config like this
plugins: [new webpack.optimize.CommonsChunkPlugin({
name: 'vendor',
chunks: ["bundle"], //optional
minChunks: function (module) {
return module.context && module.context.indexOf('node_modules') !== -1;
}
}),
new webpack.optimize.CommonsChunkPlugin({
name: "manifest",
chunks: ["vendor"],
minChunks: Infinity
})]
The trick here is that with the first plugin instance, you will be extracting the modules coming from node_modules to the vendor bundle. Since this is the parent common bundle, webpack will add its runtime code into vendor bundle (if the second instance of plugin is not added). Then with the application of second instance of the plugin, using minChunks as Infinity, you will not be extracting any module, but you will make the manifest bundle as parent bundle for vendor, hence, webpack will add its runtime code into the manifest bundle.
You need to make sure that manifest script should get executed first, then vendor script and finally bundle script. In other words, load them sequentially to avoid unexpected errors.

Extracting common chunks amongst multiple compiler configurations in webpack?

I'm trying out the multi-compiler option in webpack and am following the example at their github. However, I can't seem to understand how I can split out the common code amongst the multiple configurations.
For example, I may have same vendor libraries used in the different set of configurations. I would like to have these shared codes to be bundled to one single common file.
I tried the following but it ended up creating an individual bundles of the vendors entry for each compile configuration.
var path = require("path");
var webpack = require("webpack");
module.exports = [
{
name: "app-mod1",
entry: {
vendors: ['jquery', 'react', 'react-dom'],
pageA: ['./mod1/pageA'],
pageB: ['./mod1/pageB']
},
output: {
path: path.join(__dirname, "/mod1/js"),
filename: "[name].bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({
names: ['vendors'],
minChunks: Infinity
})
]
},
{
name: "app-mod2",
entry: {
vendors: ['lodash', 'react', 'react-dom'],
pageA: ['./mod2/pageA'],
pageB: ['./mod2/pageB']
},
output: {
path: path.join(__dirname, "/mod2/js"),
filename: "[name].bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({
names: ['vendors'],
minChunks: Infinity
})
]
}
];
Since react, react-dom are shared between the 2 compilations, my intention is for them to be bundled as a single file which can be shared instead of creating a same bundle for each compilation.
How can I extract the common chunks out of multiple compiler configurations?

Brief answer
You can't do that job in the way you want.
TL;DR
#Carven, I am afraid that you can't achieve your goal via MultiCompiler of Webpack, MultiCompiler is not meant to do that job, at least for the near feature.
See the source code for initiating the MultiCompiler instance, it actually initiates separate Compiler instances. These compiler have no data shared between.
See also the source of running MultiCompiler instance, the compilers instance also run separately without sharing data.
The only thing these compilers share is the Stats instance they produce and merge into a MultiStats.
By the way, there is no clue in the example you mentioned that some modules are shared between multi-compilers.
Alternative
As described by #Tzook-Bar-Noy, IMHO, you have to use muti-entries to achieve MPA(Multi-page Application) bundling.
Other worth mentioning
I noticed a library called webpack-multi-configurator is using the multi-compiler feature. But I don't think it will share common chunk(s).

You can extract the shared code into another compilation, and bundle it with DllBundlesPlugin.
later consume this DLL via DLLReferencePlugin and add it to your page either manually or via HTMLWebpackPlugin's add-asset-html-webpack-plugin
Bolierplate can be reduced by using autodll-webpack-Plugin

I learned about it now, and this topic seems quite hard to understand in webpack docs. I managed to create something that works, as it created 2 separate files and extracted the common dependencies to another file.
Here is my webpack config:
{
entry: {
pageA: "./first/first",
pageB: "./second/second"
},
output: {
path: path.join(__dirname, "js"),
filename: "[name].js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({
names: ["vendor", "common"],
})
]
};
the output of this will be:
./js/
common.js
vendor.js
pageA.js
pageB.js
I created a repo with the example I worked on: https://github.com/tzookb/webpack-common-vendor-chunks
when I open a new html file I load these files:
first page:
common.js
vendor.js
pageA.js
sec page:
common.js
vendor.js
pageB.js

Webpack CommonsChunkPlugin not working as expected

Folder Structure:
app.js, benchmark.js, board.js all require jquery. I just want to extract jquery as vender.js and three other bundles only contain application code:
Webpack Config:
The result is not what I expected:
app.js, benchmark.js, board.js still contains jquery code (as you can see from the huge file size)
Is there anything wrong with my webpack configuration?
I just followed the example in :
https://github.com/webpack/webpack/tree/master/examples/two-explicit-vendor-chunks
https://github.com/webpack/webpack/tree/master/examples/multiple-entry-points

plugins should be an object array outside of modules.
Also, I don't think you need the minChunks or chunks options for this use case scenario. Your vendor entry chunk should be sufficient.
entry: {
vendor: ['jquery']
},
plugins: [
new webpack.optimize.CommonsChunkPlugin({
name: "vendor",
filename:"vendor.js",
minChunks: Infinity
})
];

Simple solution to share modules loaded via NPM across multiple Browserify or Webpack bundles

Pulling my hair out here looking for a simple solution to share code, required via NPM, across multiple Browserify or Webpack bundles. Thinking, is there such a thing as a file "bridge"?
This isn't due to compile time (I'm aware of watchify) but rather the desire to extract out all of my vendor specific libs into vendor.js so to keep my app.js filesize down and to not crash the browser with massive sourcemaps. Plus, I find it way cleaner should the need to view the compiled js arise. And so:
// vendor.js
require('react');
require('lodash');
require('other-npm-module');
require('another-npm-module');
Its very important that the code be loaded from NPM as opposed to Bower, or saved into some 'vendor' directory in order to be imported via a relative path and identified via a shim. I'd like to keep every library reference pulled via NPM except for my actual application source.
In app.js I keep all of my sourcecode, and via the externals array, exclude vendor libraries listed above from compilation:
// app.js
var React = require('react');
var _ = require('lodash');
var Component = React.createClass()
// ...
And then in index.html, I require both files
// index.html
<script src='vendor.js'></script>
<script src='app.js'></script>
Using Browserify or Webpack, how can I make it so that app.js can "see" into those module loaded via npm? I'm aware of creating a bundle with externals and then referencing the direct file (in, say, node_modules) via an alias, but I'm hoping to find a solution that is more automatic and less "Require.js" like.
Basically, I'm wondering if it is possible to bridge the two so that app.js can look inside vendor.js in order to resolve dependencies. This seems like a simple, straightforward operation but I can't seem to find an answer anywhere on this wide, wide web.
Thanks!

Listing all the vendor files/modules and using CommonChunkPlugin is indeed the recommended way. This gets pretty tedious though, and error prone.
Consider these NPM modules: fastclick and mprogress. Since they have not adopted the CommonJS module format, you need to give webpack a hand, like this:
require('imports?define=>false!fastclick')(document.body);
require('mprogress/mprogress.min.css');
var Mprogress = require('mprogress/mprogress.min.js'),
Now assuming you would want both fastclick and mprogress in your vendor chunk, you would probably try this:
module.exports = {
entry: {
app: "./app.js",
vendor: ["fastclick", "mprogress", ...]
Alas, it doesn't work. You need to match the calls to require():
module.exports = {
entry: {
app: "./app.js",
vendor: [
"imports?define=>false!fastclick",
"mprogress/mprogress.min.css",
"mprogress/mprogress.min.js",
...]
It gets old, even with some resolve.alias trickery. Here is my workaround. CommonChunkPlugin lets you specify a callback that will return whether or not you want a module to be included in the vendor chunk. If your own source code is in a specific src directory, and the rest is in the node_modules directory, just reject the modules based on their path:
var node_modules_dir = path.join(__dirname, 'node_modules'),
app_dir = path.join(__dirname, 'src');
module.exports = {
entry: {
app: "./app.js",
},
output: {
filename: "bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin(
/* chunkName= */"vendor",
/* filename= */"vendor.bundle.js"
function (module, count) {
return module.resource && module.resource.indexOf(app_dir) === -1;
}
)
]
};
Where module.resource is the path to the module being considered. You could also do the opposite, and include only the module if it is inside node_modules_dir, i.e.:
return module.resource && module.resource.indexOf(node_modules_dir) === 0;
but in my situation, I'd rather say: "put everything that is not in my source source tree in a vendor chunk".
Hope that helps.

With webpack you'd use multiple entry points and the CommonChunkPlugin.
Taken from the webpack docs:
To split your app into 2 files, say app.js and vendor.js, you can require the vendor files in vendor.js. Then pass this name to the CommonChunkPlugin as shown below.
module.exports = {
entry: {
app: "./app.js",
vendor: ["jquery", "underscore", ...],
},
output: {
filename: "bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin(
/* chunkName= */"vendor",
/* filename= */"vendor.bundle.js"
)
]
};
This will remove all modules in the vendor chunk from the app chunk. The bundle.js will now contain just your app code, without any of it’s dependencies. These are in vendor.bundle.js.
In your HTML page load vendor.bundle.js before bundle.js.
<script src="vendor.bundle.js"></script>
<script src="bundle.js"></script>

// vendor anything coming from node_modules
minChunks: module => /node_modules/.test(module.resource)
Source: https://github.com/webpack/webpack/issues/2372#issuecomment-213149173