Develop Reference

Sharing node_modules chunks across builds (webpack)

I want to setup a webpack.config file that is able to dynamically accept entry points, and build them, whilst outputting all of the used node modules as chunks. So for example if we had:
// main-a.js
import { a, b, c } from 'node-module-a'
import hello from 'node-module-b'
I'd like to have an output folder that contains:
main-a.js
node-module-a.js
node-module-b.js
Note that node-module-a.js should contain all of its dependencies as well, so that node-module-a contains the entire imported library. So then if I used the same webpack configuration file with another entrypoint as shown:
// main-b.js
import { b, d } from 'node-module-a'
import there from 'node-module-c'
My output folder would now contain:
main-a.js
main-b.js
node-module-a.js
node-module-b.js
node-module-c.js
Loading both main-a.js and main-b.js from the browser should work.
What I tried
I tried achieving this with the optimisation.splitChunks option, to detect the node_modules and to place them into their own chunks. This seems to have a few issues:
It doesn't reliably seem to work with async modules (but that could just be a misconfiguration)
On production builds, the modules from the first build are not recognised by the second build
Is there a better way to achieve this? I get the hint that this isn't what splitChunks is designed for. The purpose here is to be able to use the node_modules that I've installed almost like modules, since I will have lots of different entry points, and don't want to include any of the node modules code in those entry points.

How to split code into several bundles with Vue CLI3

I have a Vue project using TypeScript and built by Vue-CLI3.
What I'm trying to achieve is to get Webpack to build separate bundles for my workers. I've read about Webpack Code Splitting and about configureWebpack in vue.config.js, but so far had no luck in putting them together.
The project setup is the standard vue create type. I have a ./src/main.ts as the main entry point and a bunch of TypeScript modules, I want as separate bundles with their own dependency trees (I'm fine with code duplication if it can't be avoided).
I'd like to get
./dist/js/all main stuff
./dist/js/workers/worker1.6e3ebec8.js
./dist/js/workers/worker2.712f2df5.js
./dist/js/workers/worker3.83041b4b.js
So I could do new Worker(worker1.6e3ebec8.js) in the main code.
I could launch workers from the main package by generating javascript code, putting it into a blob and instantiating from that, but it looks rather awkward. Besides, my worker code import other modules, so it doesn't seem to be an option anyway.
I'm quite new to all of this, so maybe I'm not even heading in the right direction.
What is the usual way of doing that on this stack?

You can use import(), it will return a Promise and will resolve your module.
As you are using Vue-CLI 3, webpack is ready and it should split your bundle automatically.
const moduleName = 'coolModuleName'
import (
/* webpackChunkName: "[moduleName]" */
`#/my/module/path/${moduleName}.js`
).then(moduleCode => {
// use your module
})
// load them in parallel
const getModuleDynamically(path, moduleName) => import(
/* webpackChunkName: "[moduleName]" */
`#/${path}/${moduleName}.js`
)
Promise.all([
getModuleDynamically(path, moduleName1),
getModuleDynamically(path, moduleName2),
getModuleDynamically(path, moduleName3)
])

Got there! #aquilesb's answer did help, although I've failed to get getModuleDynamically() from the answer working after plenty of experimenting.
Update: Looks like with this solution I'm not able to use imports of npm modules. I've tried experimenting with worker-loader but haven't got anywhere so far.
Here are a few takeaways:
Create a separate webpack config for packing workers. The target: 'webworker' must be there. Call it with webpack --config ./webpack.config.workers.js, as Vue wouldn't know about that.
Create a separate tsconfig.json for workers TypeScript. The lib setting for workers must be there: "lib": ["esnext","webworker","scripthost"] as well as the proper include:[...]/exclude:[...] settings.
You may need to tell Vue to use the main tsconfig.json that has it's own "lib":["esnext","dom","dom.iterable","scripthost"] and include/exclude. This is done in vue.config.js, you will probably need to create it. I use chainWebpack configuration option of Vue config.
Let Webpack know you have dynamic loading by making calls to import() with static (i.e. not variable) names. I haven't found a way to do so in config file, but it doesn't matter: you can't help hardcoding the names somewhere, how else Webpack would know it has to bundle and split the code?
Somehow get the name(s) of generated files, as you must have at least one of them at runtime to do new Worker(filename). I used the --json option of Webpack CLI for that.
There are many ways all of this can be achieved. This is what this ended up looking like in my project:
Folder structure:
webpack.config.workers.js
vue.config.js
tsconfig.base.json
src/main/
src/main/tsconfig.json -- extends tsconfig.base.json
src/shared/ -- this code may be duplicated by the Vue app bundles and by workers bundle
src/workers/
src/workers/tsconfig.json -- extends tsconfig.base.json
webpack.config.workers.js: contains a single entry – the main worker file, that loads the other stuff.
entry: {
worker: './src/workers/worker.ts'
}
build.workers.sh: the script calls Webpack CLI and produces a JSON file with the resulting workers names (trivial actions on folders are omitted). The only one I need is called "worker". The rest is to be dynamically loaded by it.
#!/bin/bash
# Map entry name -> bundle file name
# "assetsByChunkName":{"entryN":"entryN.[hash].js", ...}
json=$(webpack --config ./webpack.config.workers.js --json $#|tr -d "\n\r\t "|grep -Eo '"assetsByChunkName":.+?}')
# Remove "assetsByChunkName"
json=$(echo "${json:20}")
echo $json
echo $json > "$target/$folder/workers.json"
Load workers.json at runtime. The other option would be to use it at compile time by providing Vue config with const VUE_APP_MAIN_WORKER = require("path to/workers.json").worker and using this env constant.
Now that we have the name of the main worker file, we can do new Worker("main worker file path we've got from webpack").
The main worker file contains the function that statically references other modules and dynamically loads them. This way Webpack knows what to bundle and how to split the code.
enum WorkerName {
sodium = "sodium",
socket = "socket"
}
function importModule(name: WorkerName): Promise<any> {
switch (name) {
case WorkerName.sodium:
return import(
/* webpackChunkName: "sodium" */
"workers/sodium"
);
case WorkerName.socket:
return import(
/* webpackChunkName: "socket" */
"workers/socket"
);
}
}
Use the postMessage/message event API to tell your main worker code what to load.
const messageHandler = (e: MessageEvent) => {
// here goes app-specific implementation of events
// that gets you the moduleName in the end
importModule(moduleName).then((work) => {
// do smth
});
};

Now, to the correct answer.
To achieve the following:
Using webworkers
Using both dynamic imports and normal imports in webworker code
Sharing code between webworkers and main app
I had to add a separate rule for worker-loader in vue.config.js and also to add babel-loader. It took me some time to find the correct solution, but I dropped the previous one (in my other answer) in the end. I still use separate tsconfig.js for main and for webworkers.
What I'm still not happy with, is that vue-cli–or rather fork-ts-checker plugin–doesn't seem to know the webworker-specific types in my worker classes (so I can't use DedicatedWorkerScope, for instance).

How to bundle in Webpack assets in multiple pages that don't link to every other assets?

There is something I couldn't understand about Webpack.
Most of the webpack examples show a main entry point app.js which imports all other react components for Webpack to recursively build a resultant file. If there were multiple entry points, like pageA.js and pageB.js, we put them into an array in the entry argument.
My problem however is that my "main" entry point doesn't use and import every single components. Maybe just some. On PageA, I might only import ComponentA and ComponentB. On PageB, I might only import ComponentB and ComponentC. Then in the main.js on my MainPage, I might only import ComponentD.
I can put main.js, PageA and PageB all into my entry points. But does that mean I have to add to the entry list in webpack config every time I have a new page?
How should I approach this scenario using Webpack?

Sounds like your setup is a typical webpack multi-page app, which is usually consist of common/vendor bundle (entry point), and separate bundles (entry points) for each page, like your PageA and PageB.
For each page you will need to include the common/vendor bundle first, and then include the page specific bundle.
<script src="common.js">
<script src="PageA.js">
Have a look at multiple-entry-points and multiple-commons-chunks. Webpack can automatically extract the common deps and bundle them together (1st example), or you can specify which deps should go into which common chucks (2nd example). Also have a look at common-chunk-and-vendor-chunk, which perfectly explained how common chuck works.
According to your description, your setup is very similar to the first example (multiple-entry-points).
And yes, you will probably need to add a new entry when you have a new page, like PageC.

Webpack's config is actually a JS module. This means that the object you return can be built programmatically rather than manually.
What you want is for webpack to handle chunks optimization properly for you. This, along with programmatically determining the pages you have, should allow you to generate all your assets.

Using webpack to generate typescript libraries with typing files

Currently my process for building is:
Write lots of typescript files with ES6 module syntax
Generate an index.ts which re-exports all modules from one point
Compile to CommonJS + System
Output Descriptor/Typing files
This results in an index.js file which re-exports all the internal files without the developer consuming it needing to know about it, as well as a lot of d.ts files which mirror the file structure.
Now this works, but if I were to take this approach to the browser I would need to webpack up all the js or it would be a http request nightmare pulling in all the individual files. Currently this library would be consumed as a dependency for other libraries, so it is not an end point for logic or anything it is a module/library.
Now the main question is with webpack I know I can load TS in and get a commonJS module out, however I cannot find any way to generate d.ts files with webpack. So is there a way for me to use webpack as the compiler/packager in this scenario and have an output my-lib.js and my-lib.d.ts rather than the current approach which yields lots of individual files.
== Extra Clarification on Use Case ==
Just to try and make sure everyone is on the same page here when I say it is a library that would be re-used what I mean is that this is something that would be loaded via npm or jspm or something as a module dependency for other modules.
So for example let us pretend jquery doesn't exist and I am going to create it but write it in typescript for other developers to consume in both JS and TS. Now typescript outputs both js files and d.ts files, the js files are to be used as you would expect, but the d.ts files explain to other typescript files what the types contained within the library are.
So assuming I have developed jquery in TS as listed above, I would then want to publish this output (be it created by webpack or tsc) on npm/jspm/bower etc. So then others can re-use this library in their own projects.
So webpack typically is used to package an "application" if you will, which contains logic and business concerns and is consumed directly as an entry point to a larger set of concerns. In this example it would be used as a compilation and packaging step for a library and would be consumed via var myLib = require("my-lib"); or similar.

Generating the .d.ts files is not related to webpack. With webpack you can use either ts-loader or awesome-typescript-loader. Both of them make use of tsconfig.json. What you need to do is to add declaration: true in your tsconfig.json.
I'd also suggest you to take a look at typescript-library-starter. You'll find how's set up there, including UMD bundle and type definitions :).

Angular2 - require module on client side

In the context of a Node.js / Express / Angular2 / typescript (IDE=Visual Studio) app, I am trying to load a third party .js utility (packery) onto the client side (for use in a directive). Someone made typescript definitions for it. The d.ts file looks like:
declare module "packery" {
interface PackeryOptions { stuff... }
class Packery { stuff .... }
export = Packery;
}
I refer to this d.ts file, tell the browser where the .js packery script lives, and then import the module as such:
import Packery = require('packery');
This compiles without complaint. However, upon running, the browser attempts (and fails) to find "packery" at http://localhost/packery as opposed to knowing packery is an imported library. This is in contrast to the other import statements I have made on the client such as:
import {Http, HTTP_PROVIDERS} from 'angular2/http';
which work - as far as I can tell the only two pieces of information I gave it for those were also a d.ts file and the location of the .js file, just like packery. But, I must be missing something. Have tried many combinations of file locations and linking and can't get it to work. How can I get the proper linking to "packery"?
Thanks!

I found a workaround for this and thought I'd post in case it helps anyone, although I am still having difficulty with the setup posed in the original question, that is, getting statements of the type:
import foo = require('foo')
to run on the CLIENT side. These work for me in node.js on the server, but on the client, for third party libraries that have been loaded via a script tag, I cannot get it to work, even if I add mapping entries to the system.js config file, irrespective of if I point to a .js file or a d.ts file.
Anyway, what does work is if you load the library using the script tag, then in your IDE put a reference path as such at the top of the CLIENT side code
/// <reference path="foo.d.ts" />
and ensure that your d.ts file does not declare a module/namespace but rather exports methods etc. directly. This lets the IDE compile without complaint, and the client side code is able to access the third party library.
However, I'm not sure if it is preferable / best practices to do what I did or if one should be configuring System.js somehow.

Typings are empty definitions of js libraries that aren't written in a typed language. They are only useful in development for IDEs hints and stuff, in your app, you'll still use the library as you normally would, adding the js file in your index.html or w/e you load your js files from.