I am trying to optimise my Webpack bundle and I have identified a large chunk of data that is being bundled from one of my dependencies. The dependency is world-countries, which is just a huge JSON array of objects containing country data. I am however only using a fraction of this data in my app (a simple React app).
So what I am thinking is that I want to add something into my Webpack config that will effectively let me map the large objects down into just the properties I am using, and avoid having all the rest of the data end up in my bundle.
My only idea currently on how to do this would be to write a node script that runs postinstall or prebuild that imports the module, maps it and saves it back out to disk in a new JSON file. Then that JSON file is what my app imports.
I'm looking for any advice or ideas on the best way to implement this, preferably in a way that doesn't affect my apps code and is just part of the Webpack config.
You can write a custom loader which will be applied on the Jason import, inside it filter out what you need.
i am using DefinePlugin now to create only the JSON I need. In my example i am using the countries from country-data:
Webpack config:
import { countries } from 'country-data';
...
plugins: [
new webpack.DefinePlugin({
COUNTRIES: webpack.DefinePlugin.runtimeValue(function() {
return JSON.stringify(countries.all.filter(function (c) {
// Has "United Kingdom" twice, #see https://github.com/OpenBookPrices/country-data/issues/72
var assigned = c.status === 'assigned';
// cleanup JSON
['countryCallingCodes', 'languages', 'currencies', 'emoji', 'ioc', 'alpha3', 'status'].forEach(function (key) {
delete c[key];
});
return assigned;
}));
}),
}),
]
You then can use COUNTRIES in your code!
Related
I'm building a custom Webpack loader. What the loader does is unimportant, but it transforms JSON in some way and uses paths from the JSON in order to resolve certain other details. In my loader.js I need a way of getting the original path of the JSON file being loaded such that I can resolve other paths properly.
Take this simple loader and config:
loader.js
module.exports = function (source) {
/* Do some file lookups based on source and modify source */
this.callback(null, source)
}
webpack.config.js
module.exports = {
/* ... */
module: {
rules: [
{
test: /\.json$/i,
use: ['loader'],
},
],
},
};
The loader is working (being used), but any business logic I add needs to be path-aware such that it can do lookups on the file system.
Because this is a JSON loader, the source var in the loader function is passed as raw JSON content, with no details about which file the JSON was loaded from. How does one go about getting path information from within the loader such that I can perform other file lookups based on the content from source?
It turns out the property I was looking for is available via the execution context property this, as resourcePath.
module.exports = function (source) {
console.log('The original file was here:', this.resourcePath)
this.callback(null, source)
}
Here is the (rather sparse) documentation.
If anybody can think of a better/more future-proof way of getting this path then feel free to comment or post another answer.
I'm trying to build a modular application in Vue via the vue-cli-service. The main app and the modules are separated projects living in different folders, the structure is something like this:
-- app/package.json
/src/**
-- module1/package.json
/src**
-- module2/package.json
/src**
The idea is to have the Vue app completely agnostic about the application modules that can be there at runtime, the modules themself are compiled with vue-cli-service build --target lib in a local moduleX/dist folder, pointed with the package.json "main" and "files" nodes.
My first idea (now just for development speed purposes) was to add the modules as local NPM packages to the app, building them with a watcher and serving the app with a watcher itself, so that any change to the depending modules would (I think) be distributed automatically to the main app.
So the package.json of the app contains dependencies like:
...
"module1": "file:../module1",
"module2": "file:../module2",
...
This dependencies are mean to be removed at any time, or in general be composed as we need, the app sould just be recompiled and everything should work.
I'm trying to understand now how to dynamically load and activate the modules in the application, as I cannot use the dynamic import like this:
import(/* webpackMode: "eager" */ `module1`).then(src => {
src.default.boot();
resolve();
});
Because basically I don't know the 'module1', 'module2', etc...
In an OOP world I would just use dependency injection retrieving classes implementing a specific interface, but in JS/TS I'm not sure it is viable.
There's a way to accomplish this?
Juggling with package.json doesn't sound like a good idea to me - doesn't scale. What I would do:
Keep all available "modules" in package.json
Create separate js file (or own prop inside package.json) with all available configurations (for different clients for example)
module.exports = {
'default': ['module1', 'module2', 'module3'],
'clientA': ['module1', 'module2', 'module4'],
'clientB': ['module2', 'module3', 'module4']
}
tap into VueCLI build process - best example I found is here and create js file which will run before each build (or "serve") and using simple template (for example lodash) generate new js file which will boot configured modules based on the value of some ENV variable. See following (pseudo)code (remember this runs inside node during build):
const fs = require('fs')
const _ = require('lodash')
const modulesConfig = require(`your module config js`)
const configurationName = process.env.MY_APP_CONFIGURATION ?? 'default'
const modules = modulesConfig[configurationName]
const template = fs.loadFileSync('name of template file')
const templateCompiled = _.template(template)
const generatedJS = templateCompiled({ `modules`: modules })
fs.writeFileSync('bootModules.js', generatedJS)
Write your template for bootModules.js. Simplest would be:
<% _.forEach(modules , function(module) { %>import '<%= module %>' as <%= module %><% }); %>;
import bootModules.js into your app
Use MY_APP_CONFIGURATION ENV variable to switch desired module configuration - works not just during development but you can also setup different CI processes targeting same repo with just different MY_APP_CONFIGURATION values
This way you have all configurations at one place, you don't need to change package.json before every build, you have simple mechanism to switch between different module configurations and every build (bundle) contains only the modules needed....
In an OOP world I would just use dependency injection retrieving classes implementing a specific interface, but in JS/TS I'm not sure it is viable.
Why not?
More than this, with JS/TS you are not restricted to use classes implementing a specific interface: you just need to define the interface (i.e. the module.exports) of your modules and respecting it in the libraries entries (vue build lib).
EDIT: reading comments probably I understood the request.
Each module should respect following interface (in the file which is the entry of the vue library)
export function isMyAppModule() {
return true;
}
export function myAppInit() {
return { /* what you need to export */ };
}
Than in your app:
require("./package.json").dependencies.forEach(name => {
const module = require(name);
if(! module.isMyAppModule || module.isMyAppModule() !== true) return;
const { /* the refs you need */ } = module.myAppInit();
// use your refs as you need
});
I have a rather complex scenario.
We are building a desktop application with React which is wrapped with Electron, Webpack takes care of the Babel transpilation and chunking.
The application receives configuration data from a cms.
Part of the configuration may be a javascript class that needs to override one that resides in the application. The JS code as specified in the CMS will be vanilla Javascript code (ES6/7/8 same as what we use for the application)
I see 2 problems here:
How to transpile just this one class and
How to replace it runtime in the application
Is this even possible?
Regards
If with "The application receives configuration data from a cms." you mean runtime data, then, because Webpack acts at compile time, it cannot help you to transpile/replace your code (Runtime vs Compile time).
if your data from a CMS can be fetched at compile time, then, notice that you can return a promise from webpack.config.js.
module.exports = function webpackConfig(env) {
const configs = {
context: __dirname,
plugins: []
// etc...
};
return CMS
.fetchConfig()
.then(cmsConfigs => {
const vars = {
replaceClass: JSON.stringify(cmsConfigs.classINeed.toString())
};
configs.plugins.push(new webpack.DefinePlugin(vars));
return configs;
})
;
}
My page includes several components that exist as separate AMD modules. Each of these components is turned into a single file by the Require.js optimiser. Because several of these components share dependencies (e.g. jQuery and d3), the optimiser paths config uses CDN URLs for those dependencies, rather than bundling them into the optimised file.
Here's where it gets tricky. I've written a module loader plugin for Ractive.js called rvc.js, which allows me to include Ractive components that are defined in HTML files. (Yes, I'm asking for help on how to use my own library.)
This works fine - code like this gets optimised as you'd expect:
define( function ( require ) {
var ChartView = require( 'rvc!views/Chart' );
var view = new ChartView({ el: 'chart' });
});
Because Ractive is used by several of the components, it should be served from a CDN like jQuery and d3. But it's used by rvc.js during the optimisation process, which means that the Ractive entry for the optimiser's paths config can't point to a CDN - it has to point to a local file.
Is there a way to tell Require.js 'use the local file during optimisation, but load from CDN at runtime'?
So here's the solution I eventually settled on. It feels somewhat kludgy, but it works:
Stub out the loaders and the library you don't want bundled
Add an onBuildWrite function that rewrites modules depending on the library, so that they think they're requiring something else entirely - in this case Ractive_RUNTIME
Add an entry to your runtime AMD config's paths object, so that Ractive_RUNTIME points to the CDN
My optimiser config now looks like this:
{
baseUrl: 'path/to/js/',
out: 'build/js/app.js',
name: 'app',
optimize: 'none',
paths: {
'amd-loader': 'loaders/amd-loader',
'rvc': 'loaders/rvc',
'Ractive': 'lib/Ractive'
},
stubModules: [ 'amd-loader', 'rvc', 'Ractive' ],
onBuildWrite: function ( name, path, contents ) {
if ( contents === "define('Ractive',{});" ) {
// this is the stub module, we can kill it
return '';
}
// otherwise all references to `Ractive` need replacing
return contents.replace( /['"]Ractive['"]/g, '"Ractive_RUNTIME"' );
}
}
Meanwhile, the script that loads the app.js file created by the optimiser has a config entry that points to the CDN:
require.config({
context: uniqueContext,
baseUrl: baseUrl,
paths: {
'amd-loader': 'loaders/amd-loader',
'rvc': 'loaders/rvc',
'Ractive': 'lib/Ractive',
'Ractive_RUNTIME': 'http://cdn.ractivejs.org/releases/0.3.9/Ractive.min'
}
});
I am currently working myself into requirejs, to do this I started to write a small package. The package consists of one main.js, the entry point for the application and several other modules that will be loaded into main.js through a call to require.
The package has several options that can be configured during runtime, to ease maintenance I have a central configuration.js that is loaded whenever needed. The contents of configuration.js is just an object that maps key to value, no functionality or anything. Just a key-value map.
Now requirejs allows to pass options to packages, the configuration is done through the requirejs config. The problem I have is, that it is only available in the main.js and not in other modules in the package, also I could not find a way to set default values unless I hardcode them into the main.js.
My current approach (the one with the configuration.js) is to merge the configuration given to requirejs with the contents of my configuration.js in the main.js.
main.js
define(function(require, exports, module) {
var $ = require('jquery'); // Could be any library that offers an "extend"
// feature that allows "deep" copies.
var config = require('configuration'); // Load configuration file
$.extend(true, config, module.config()); // Overwrite default values with
// set values
var otherfile = require('anyotherfile);
otherfile();
});
anyotherfile.js
define(function(require, exports, module) {
var config = require('config'); // Loads configuration, package-wide
// options are set
// module.config() will be empty. Unless I explicicy specify the
// configuration for this one module in the "requirejs.config"
return function() {
// Something that can be configured
};
});
This way of setting configuration options makes me dependent on an additional library or my own implementation of an extend functionality (it's not much code, but still more code than no code) plus there are two points where I have to look for possible configuration mistakes which makes it harder to find errors.
So now I am searching for a better way to pass a package-wide configuration all while trying to avoid more than one point of failure. (This might be hard cause somewhere I have to store the default values if I don't want to hardcode them.) And of course the solution should not depend on any additional libraries or my own extend functionality.
After talking with jrburke about the problem the best solution is to do the extending of the config array in the configuration.js.
Example of the file configuration.js
define(function(require, exports, module) {
var $ = require('jquery'); // need jQuery for extend
var config = {
key: value,
...
};
return $.extend(true, config, module.config());
});
So there is no more need to do this in the main.js.
The only disadvantage of this approach is that in the requirejs.config you have to address
package/configuration and not package.
Example of the requirejs.config
requirejs.config({
config: {
"package/configuration": {
key: value
}
}
});