Develop Reference

Converting legacy namespace code to modules from the top down

I'm trying to convert out codebase which is comprised of multiple repositories to use modules instead of namespaces.
Sorry ahead of time if my understanding is fundamentally wrong, my main expirience in javascript is in this company so this is all I've ever known.
The codebase has heiharchies where certain repositories inherit from the base one so for instance if we have A,B,C,D,E then B,C,D,E all know A.
C might know B, but D only knows A, while E knows D and C. (Attached image to be clear)
Right now we have every repository compile into 1 js file using tsconfig outFile, and we load all of it in our html file one by one in order in script tags.
I've started converting the "upper" repos as modules can use namespaces, (so in my example E and D). I changed all of the code inside the repositories themselves to be modular and not use namespaces.
It now compiles properly and works (individually - the issue I'm having is with making them import from one another).
Since I'm trying to preserve the behavior it seems like I need to use a bundler.
I'm trying to use webpack but I'm having some problems with it.
I created a package.json (we didn't have one because we just directly made one js file and put it in the html until now yes that means we couldn't import things properly it's a nightmare it's why I'm trying to change it).
Installed webpack ts-loader and yarg.
I've then made the webpack.config.js I linked below and tried compiling.
Some of my repos don't have good entry files because they are initialized by other repos so I have to make a massive list and check that all files are at least written in the JS (there has to be a solution for this i'm missing right?)
The other issue is I don't understand how to make the other repos import from the repo I just bundled.
I put the bundled.js in the node_modules but when I attempt to import a class from it I get the following error:
"TS2305: Module '"../../node_modules/#types/hybridPanel.js"' has no exported member 'test'.
(I am trying to load D in E in this instance both have been converted to modules and are not using namespaces so this should work).
Do I need to publish the repo somewhere and npm install it in repo2 (I thought that's the same thing as moving the js over).
So the questions I have are:
How do I make this work?
What do I do about the entry files issue?
Am I even going about this the right way (will I even get types if I just import the js)?
Will we need to change our tags in the html to be type=module instead of javascript?
If I were to add a module (using npm or similar) will I then need to add it to the externals tag in the webpack? I'm not confident in my understanding just yet.
Is this the correct way of converting to modules from namespaces? Our plan is if I can get this to work to convert the rest of the higher ones next and then do the lower ones all at once (the base ones multiple people use) and fix the imports in the higher ones once we get there.
webpack config link :
https://pastebin.com/iNqV1dEV
const webpack = require("webpack");
const path = require("path");
const yargs = require("yargs");
const env = yargs.argv.env; // use --env with webpack 2
const pkg = require("./package.json");
const shouldExportToAMD = yargs.argv.amd;
let libraryName = pkg.name;
let outputFile, mode;
if (shouldExportToAMD) {
libraryName += ".amd";
}
if (env === "build") {
mode = "production";
outputFile = libraryName + ".min.js";
} else {
mode = "development";
outputFile = libraryName + ".js";
}
const config = {
mode: mode,
entry: [__dirname + "/src/panel/MoPanelManager.ts", __dirname + "/src/panel/chat/MoSingleChat.ts", __dirname + "/src/panel/booth/MoBoothDisplays.ts", __dirname + "/src/panel/chat/MoGifs.ts", __dirname + "/src/settings/MoSettings.ts"],
devtool: "source-map",
output: {
path: __dirname + "/www/module",
filename: outputFile,
library: libraryName,
libraryTarget: "umd",
libraryExport: "default",
umdNamedDefine: true,
globalObject: "typeof self !== 'undefined' ? self : this",
},
module: {
rules: [
{
test: /\.ts?$/,
use: {
loader: 'ts-loader',
},
exclude: /(node_modules|bower_components)/,
},
],
},
resolve: {
modules: [path.resolve("./node_modules"), path.resolve("./src")],
extensions: [".ts", ".js"]
},
};
module.exports = config;
Image of the repo example

Change domain of images that webpack generates for imported images

Although my dev server is running on localhost:3000, I have set up my host file to point www.mysite.com to localhost. In my JavaScript, I have code like:
import myImage from '../assets/my-image.jpg'
const MyCmp => <img src={myImage} />
Using Webpack's file-loader, it transforms that import into a URL to the hosted image. However, it uses the localhost path to that image, but I'd like it to use the www.mysite.com domain. I looked at both the publicPath and postTransformPublicPath options for file-loader, but those only appear to allow you to modify the part of the path that comes after the domain.

I personally don't like the notion of defining host-information statically in the build output. This is something that should be determined in runtime based on where you actually put your files.
If you are like me then there are two options here.
Both involve you calling a global method that you have defined on i.e. window / global scope.
The purpose of the global method is to resolve the root path (the domain, etc) in runtime.
Define a global method
So lets say you define a method on the global scope somewhere in your startup code like so:
(<any>window).getWebpackBundleRootPath = function (webpackLibraryId) {
if (webpackLibraryId == null) return throw "OOOPS DO SOMETHING HERE!";
// Preferably these variables should be loaded from a config-file of sorts.
if(webpackLibraryId == "yourwebpacklibrary1") return "https://www.yoursite.com/";
// If you have other libraries that are hosted somewhere else, put them here...
return "...some default path for all other libraries...";
};
The next step is to configure webpack to call this global method when it tries to resolve the path.
As I mentioned there are two ways, one that manipulates the output of the webpack and one that is more integrated in webpacks configuration (although only for file-loader but I think it should suffice).
It's worth mentioning that you don't need a global method if you only have one bundle or if you host all your bundles in one place. Then it would be enough to use a global variable instead. It should be quite easy to modify the example below to accommodate this.
First option: configure webpack file-loader to call your method when resolving path
This solution will not require something to be done post build. If this fits your need and covers all scenarios I would go for this option.
Edit your webpack config file
var path = require('path');
let config = {
entry: {
'index': path.join(__dirname, '/public/index.js')
},
output: {
path: path.join(__dirname, '/dist/'),
filename: 'index-bundle.js',
publicPath: 'https://localhost:3000/',
library: 'yourwebpacklibrary1',
...
},
module: {
rules: [{
// Please note that this only defines the resolve behavior for ttf. If you want to resolve other files you need to configure the postTransformPublicPath for those too. This is a big caveat in my opinion and might be a reason for using second option.
test: /\.ttf(\?v=\d+\.\d+\.\d+)?$/,
use: [{
loader: 'file-loader',
options: {
outputPath: 'assets/fonts', // The folder where you store your fonts.
name: '[name].[ext]',
// This is where the magic happens. This lets you override the output of webpack file resolves.
postTransformPublicPath: function (p) {
// Because of the way webpack file-loader works the input from to this method will look something like this: __webpack_public_path__ + "/assets/fonts/yourfont.ttf"
// But we are not interested in using the __webpack_public_path__ variable so lets remove that.
p = p.replace('__webpack_public_path__ + ', '');
// Return a stringified call to our global method and append the relative path to the root path returned.
return `getWebpackBundleRootPath("${config.output.library}") + ${p}`;
}
}
}]
},
},
...
};
module.exports = config;
As you might have noticed in the comments in the webpack config file you need to specify the resolve behavior for each file-loader that you add (if someone knows a better way, please let me know). This is why I still use the second option.
Second option: manipulate the output of the webpack in a postbuild step
Example webpack.config.js file
For completeness sake here is an example of a webpack.config.js file that contains the variables used in the postbuild script.
var path = require('path');
module.exports = {
entry: {
'index': path.join(__dirname, '/public/index.js')
},
output: {
path: path.join(__dirname, '/dist/'),
filename: 'index-bundle.js',
publicPath: 'https://localhost:3000/',
library: 'yourwebpacklibrary1',
...
},
...
}
Create a postbuild.js file
Create a file postbuild.js next to your package.json with the following content:
const fs = require('fs');
// We take the path to the webpack config file as input so that we can read settings from it.
const webpackConfigFile = process.argv[2];
// Read the webpack config file into memory.
const config = require(webpackConfigFile);
// The file to manipulate is the output javascript bundle that webpack produces.
const inputFile = config.output.path + config.output.filename;
// Load the file into memory.
let fileContent = fs.readFileSync(inputFile, 'utf8');
// Replace the default public path with the call to the method. Please note that if you specify a publicPath as '/' or something very common you might end up with a problem so make sure it is unique in the file to avoid other unrelated stuff being replaced as well.
fileContent = fileContent.replace('"' + config.output.publicPath + '"', 'getWebpackBundleRootPath("' + config.output.library + '")');
// Save the manipulated file back to disk.
fs.writeFileSync(inputFile, fileContent, 'utf8');
Call the postbuild.js automatically on build
Next step is to actually call the postbuild.js script after each build.
This can be done in a postscript in package.json like so (in the script section in your package.json):
{
"scripts": {
"build": "webpack",
"postbuild": "node postbuild.js ./webpack.config.js"
}
}
From now on whenever you run the build script it will also run the postbuild script (from npm or yarn, etc).
You can of course also manually run the postbuild.js script manually after each build instead.

but those only appear to allow you to modify the part of the path that comes after the domain.
Not really, you can give it an URL that includes the domain.
In your case, assuming your images are under the assets directory, you will have something like this in your webpack.config.js
...
module: {
rules: [
...
{
test: /\.(png|jpe?g|gif|svg)$/,
use: {
loader: 'file-loader',
options: {
publicPath: 'https://www.example.com/assets',
outputPath: 'assets'
}
}
},
...
]
}
...

How to use google-closure-compiler-js for a node.js app without gulp/grunt/webpack?

The docs don't have any examples of using this on its own but they do say this:
Unless you're using the Gulp or Webpack plugins, you'll need to specify code via flags. Both jsCode and externs accept an array containing objects in the form {src, path, sourceMap}. Using path, you can construct a virtual filesystem for use with ES6 or CommonJS imports—although for CommonJS, be sure to set processCommonJsModules: true.
I've created a "compile.js" file based on the docs:
const compile = require('google-closure-compiler-js').compile;
const flags = {
jsCode: [{path: './server/server.js'}],
processCommonJsModules: true
};
const out = compile(flags);
console.info(out.compiledCode);
In my "./server/server.js" file, I put a console.log but it doesn't output. Not sure where to go from here...

Borrowing from icidasset/quotes.
It appears, to me, that path is not intended to be used as you are using it.
Quote:
Using path, you can construct a virtual filesystem for use with ES6 or CommonJS imports—although for CommonJS, be sure to set processCommonJsModules: true.
So instead you must expand your own sources, something webpack and gulp must be doing for you when you go that route.
files=['./server/server.js']
files.map(f => {
const out = compile({
jsCode: [{ src: f.content }],
assumeFunctionWrapper: true,
languageIn: 'ECMASCRIPT5'
});
return out;
}

Simple solution to share modules loaded via NPM across multiple Browserify or Webpack bundles

Pulling my hair out here looking for a simple solution to share code, required via NPM, across multiple Browserify or Webpack bundles. Thinking, is there such a thing as a file "bridge"?
This isn't due to compile time (I'm aware of watchify) but rather the desire to extract out all of my vendor specific libs into vendor.js so to keep my app.js filesize down and to not crash the browser with massive sourcemaps. Plus, I find it way cleaner should the need to view the compiled js arise. And so:
// vendor.js
require('react');
require('lodash');
require('other-npm-module');
require('another-npm-module');
Its very important that the code be loaded from NPM as opposed to Bower, or saved into some 'vendor' directory in order to be imported via a relative path and identified via a shim. I'd like to keep every library reference pulled via NPM except for my actual application source.
In app.js I keep all of my sourcecode, and via the externals array, exclude vendor libraries listed above from compilation:
// app.js
var React = require('react');
var _ = require('lodash');
var Component = React.createClass()
// ...
And then in index.html, I require both files
// index.html
<script src='vendor.js'></script>
<script src='app.js'></script>
Using Browserify or Webpack, how can I make it so that app.js can "see" into those module loaded via npm? I'm aware of creating a bundle with externals and then referencing the direct file (in, say, node_modules) via an alias, but I'm hoping to find a solution that is more automatic and less "Require.js" like.
Basically, I'm wondering if it is possible to bridge the two so that app.js can look inside vendor.js in order to resolve dependencies. This seems like a simple, straightforward operation but I can't seem to find an answer anywhere on this wide, wide web.
Thanks!

Listing all the vendor files/modules and using CommonChunkPlugin is indeed the recommended way. This gets pretty tedious though, and error prone.
Consider these NPM modules: fastclick and mprogress. Since they have not adopted the CommonJS module format, you need to give webpack a hand, like this:
require('imports?define=>false!fastclick')(document.body);
require('mprogress/mprogress.min.css');
var Mprogress = require('mprogress/mprogress.min.js'),
Now assuming you would want both fastclick and mprogress in your vendor chunk, you would probably try this:
module.exports = {
entry: {
app: "./app.js",
vendor: ["fastclick", "mprogress", ...]
Alas, it doesn't work. You need to match the calls to require():
module.exports = {
entry: {
app: "./app.js",
vendor: [
"imports?define=>false!fastclick",
"mprogress/mprogress.min.css",
"mprogress/mprogress.min.js",
...]
It gets old, even with some resolve.alias trickery. Here is my workaround. CommonChunkPlugin lets you specify a callback that will return whether or not you want a module to be included in the vendor chunk. If your own source code is in a specific src directory, and the rest is in the node_modules directory, just reject the modules based on their path:
var node_modules_dir = path.join(__dirname, 'node_modules'),
app_dir = path.join(__dirname, 'src');
module.exports = {
entry: {
app: "./app.js",
},
output: {
filename: "bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin(
/* chunkName= */"vendor",
/* filename= */"vendor.bundle.js"
function (module, count) {
return module.resource && module.resource.indexOf(app_dir) === -1;
}
)
]
};
Where module.resource is the path to the module being considered. You could also do the opposite, and include only the module if it is inside node_modules_dir, i.e.:
return module.resource && module.resource.indexOf(node_modules_dir) === 0;
but in my situation, I'd rather say: "put everything that is not in my source source tree in a vendor chunk".
Hope that helps.

With webpack you'd use multiple entry points and the CommonChunkPlugin.
Taken from the webpack docs:
To split your app into 2 files, say app.js and vendor.js, you can require the vendor files in vendor.js. Then pass this name to the CommonChunkPlugin as shown below.
module.exports = {
entry: {
app: "./app.js",
vendor: ["jquery", "underscore", ...],
},
output: {
filename: "bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin(
/* chunkName= */"vendor",
/* filename= */"vendor.bundle.js"
)
]
};
This will remove all modules in the vendor chunk from the app chunk. The bundle.js will now contain just your app code, without any of it’s dependencies. These are in vendor.bundle.js.
In your HTML page load vendor.bundle.js before bundle.js.
<script src="vendor.bundle.js"></script>
<script src="bundle.js"></script>

// vendor anything coming from node_modules
minChunks: module => /node_modules/.test(module.resource)
Source: https://github.com/webpack/webpack/issues/2372#issuecomment-213149173

How can I exclude code path when bundling with webpack/browserify?

I have a library that can be used with both node.js and the browser. I am using CommonJS then publishing for the web version using webpack. My code looks like this:
// For browsers use XHR adapter
if (typeof window !== 'undefined') {
// This adapter uses browser's XMLHttpRequest
require('./adapters/xhr');
}
// For node use HTTP adapter
else if (typeof process !== 'undefined') {
// This adapter uses node's `http`
require('./adapters/http');
}
The problem I am encountering is that when I run webpack (browserify would do the same) the generated output includes http along with all it's dependencies. This results in a HUGE file which is not optimal for browser performance.
My question is how can I exclude the node code path when running a module bundler? I solved this temporarily by using webpack's externals and just returning undefined when including './adapters/http'. This doesn't solve the use case where other developers depend on my library using CommonJS. Their build will end up with the same problem unless they use similar exclude config.
I've looked at using envify, just wondering if there is another/better solution.
Thanks!

You may use IgnorePlugin for Webpack. If you are using a webpack.config.js file, use it like this:
var webpack = require('webpack')
var ignore = new webpack.IgnorePlugin(/^(canvas|mongoose|react)$/)
module.exports = {
//other options goes here
plugins: [ignore]
}
To push it further, you may use some flags like process.env.NODE_ENV to control the regex filter of IgnorePlugin

In order to exclude node_modules and native node libraries from being bundled, you need to:
Add target: 'node' to your webpack.config.js. This will exclude native node modules (path, fs, etc.) from being bundled.
Use webpack-node-externals in order to exclude all other node_modules.
So your result config file should look like:
var nodeExternals = require('webpack-node-externals');
...
module.exports = {
...
target: 'node', // in order to ignore built-in modules like path, fs, etc.
externals: [nodeExternals()], // in order to ignore all modules in node_modules folder
...
};

This worked best for me
var _process;
try {
_process = eval("process"); // avoid browserify shim
} catch (e) {}
var isNode = typeof _process==="object" && _process.toString()==="[object process]";
as Node will return true and not only will the browser return false, but browserify will not include its process shim when compiling your code. As a result, your browserified code will be smaller.
Edit: I wrote a package to handle this more cleanly: broquire

You can use require.ensure for bundle splitting. For example
if (typeof window !== 'undefined') {
console.log("Loading browser XMLHttpRequest");
require.ensure([], function(require){
// require.ensure creates a bundle split-point
require('./adapters/xhr');
});
} else if (typeof process !== 'undefined') {
console.log("Loading node http");
require.ensure([], function(require){
// require.ensure creates a bundle split-point
require('./adapters/http');
});
}
See code splitting for more information and a sample demo usage here