Dynamic bundle loading at runtime - javascript

Consider a bundle in some project packed with webpack and all dependencies to bundle1.js:
// entry.ts
import * as _ from 'lodash';
// other imported libraries
import { something } from './util';
// other imported bundle project local code
export function doBundleSomething(args: any): any {
// actually does something necessary
console.log('doing things');
return 'whatever result';
}
Shell program is another project also packed with webpack and all dependencies to main.js, except bundle1.js, because it is decided at run time from environment:
// main.ts
import * as _ from 'lodash';
// other imported libraries
import { something } from './shell-util';
// other imported shell program local code
// do something useful
const ifBundleNeeded = true; // decided by useful something
if (ifBundleNeeded) {
const bundlePath = process.env.BUNDLE_PATH; // could be CLI argument
// somehow load bundle
const doBundleSomething = ???
const result = doBundleSomething({arg: 'hello-bundle'});
console.log('bundle said', result);
}
Should be executed like:
BUNDLE_PATH=/opt/path/some-bundle.js node /srv/program/path/main.js
Notes:
bare minimum possible configurations are used for everything
tsconfig targets for both bundles is es6
typescript >= 3.x with webpack awesome-typescript-loader (i.e. latest and greatest)
duplication of code in both bundles is unavoidable and acceptable
security issues are not of concern (like code injection etc.)
How to achieve this behavior with bare minimum configs?
If BUNDLE_PATH is known (i.e. hard-coding path, so that webpack sees it) at webpacking of shell program, it works in different variations with require, import() including eval("require")().
Otherwise, using different alternatives fail at different stages with various errors from Cannot find module to undefined when used.

Related

How can I use switch between different environments using the new cypress.config.js in Cypress 10x?

I used to have json files under my config folder containing variables for different environments. For example:
local.env.json would contain:
{
"baseUrl": "localhost:8080"
}
then another one called uat.env.json would contain:
{
"baseUrl": "https://uat.test.com"
}
and its configured on my plugins/index.ts as:
const version = config.env.version || 'uat'; // if version is not defined, default to this stable environment
config.env = require(`../../config/${version}.env.json`); // load env from json
I will then call it on my tests with cy.visit(Cypress.env().baseUrl)) then pass it on the CI with CYPRESS_VERSION=uat npx cypress run
However, with the new Cypress 10x version, the plugin file has been deprecated and just relies on cypress.config.js. I can't find any example on their documentation on how this can be done (I remember they used to have a page with these scenarios but can't find it now).
It's possible to use the old plugins/index.ts in the new cypress.config.ts by importing it.
This is the simplest example (with no new config in cypress.config.ts)
import { defineConfig } from 'cypress'
import legacyConfig from './cypress/plugins/index.js'
export default defineConfig({
e2e: {
baseUrl: 'http://localhost:1234',
setupNodeEvents(on, config) {
return legacyConfig(on, config) // call legacy config fn and return result
}
}
})
The typescript and modules may cause you grief about typings etc. I've not tried it in a typescript project, but it does work in a javascript project.
Alternatively, copy/paste everything from plugins/index.ts instead of calling the legacy function. This might be better as you add more plugins in the future.

How to use the TypeScript Compiler API to type-check modules imported using `require()`?

I am using TypeChecker from the TypeScript Compiler API in order to extract (inferred) type information for every node in the AST of my program. In particular, I try to find out the return values from imported module functions such as:
var vec3 = require('gl-matrix/vec3')
var myVec = vec3.fromValues(1, 2, 3) // $ExpectedType vec3
This works well for modules that were imported using the import { … } from '…' statement, but unfortunately, modules that were imported using require() like above are not recognized correctly, I only receive the type any for them. However, I have set both compiler options allowJs and checkJs.
Why are the types form require()d modules not inferred correctly? VS Code (which AFAIK relies on the same API?) is able to infer the types from require() statements as well, so I'd guess that in general, tsc is able of handling them. Are there any other compiler options that I need to set differently? Or is this indeed not supported and I need to use some other package for this?
Here is a minimum script to reproduce, I have also put it on repl.it together with two example files: https://replit.com/#LinqLover/typecheck-js
var ts = require("typescript")
// Run `node index.js sample-import.js`` to see the working TypeScript analysis
const files = process.argv[1] != "/run_dir/interp.js" ? process.argv.slice(2) : ["sample-require.js"]
console.log(`Analyzing ${files}:`)
const program = ts.createProgram(files, {
target: ts.ScriptTarget.ES5,
module: ts.ModuleKind.CommonJS,
allowJs: true,
checkJs: true
})
const checker = program.getTypeChecker()
for (const sourceFile of program.getSourceFiles()) {
if (!sourceFile.isDeclarationFile) {
ts.forEachChild(sourceFile, visit)
}
}
function visit(node) {
try {
const type = checker.getTypeAtLocation(node)
console.log(checker.typeToString(type))
} catch (e) {
// EAFP
}
ts.forEachChild(node, visit)
}
Thank you so much in advance!
For follow-up, this turned out to be an issue with the type definitions for gl-matrix. I should better have tried out multiple packages before suspecting that the TypeScript engine itself could be broken ...
gl-matrix issue: https://github.com/toji/gl-matrix/issues/429

Application modularity with Vue.js and local NPM packages

I'm trying to build a modular application in Vue via the vue-cli-service. The main app and the modules are separated projects living in different folders, the structure is something like this:
-- app/package.json
/src/**
-- module1/package.json
/src**
-- module2/package.json
/src**
The idea is to have the Vue app completely agnostic about the application modules that can be there at runtime, the modules themself are compiled with vue-cli-service build --target lib in a local moduleX/dist folder, pointed with the package.json "main" and "files" nodes.
My first idea (now just for development speed purposes) was to add the modules as local NPM packages to the app, building them with a watcher and serving the app with a watcher itself, so that any change to the depending modules would (I think) be distributed automatically to the main app.
So the package.json of the app contains dependencies like:
...
"module1": "file:../module1",
"module2": "file:../module2",
...
This dependencies are mean to be removed at any time, or in general be composed as we need, the app sould just be recompiled and everything should work.
I'm trying to understand now how to dynamically load and activate the modules in the application, as I cannot use the dynamic import like this:
import(/* webpackMode: "eager" */ `module1`).then(src => {
src.default.boot();
resolve();
});
Because basically I don't know the 'module1', 'module2', etc...
In an OOP world I would just use dependency injection retrieving classes implementing a specific interface, but in JS/TS I'm not sure it is viable.
There's a way to accomplish this?
Juggling with package.json doesn't sound like a good idea to me - doesn't scale. What I would do:
Keep all available "modules" in package.json
Create separate js file (or own prop inside package.json) with all available configurations (for different clients for example)
module.exports = {
'default': ['module1', 'module2', 'module3'],
'clientA': ['module1', 'module2', 'module4'],
'clientB': ['module2', 'module3', 'module4']
}
tap into VueCLI build process - best example I found is here and create js file which will run before each build (or "serve") and using simple template (for example lodash) generate new js file which will boot configured modules based on the value of some ENV variable. See following (pseudo)code (remember this runs inside node during build):
const fs = require('fs')
const _ = require('lodash')
const modulesConfig = require(`your module config js`)
const configurationName = process.env.MY_APP_CONFIGURATION ?? 'default'
const modules = modulesConfig[configurationName]
const template = fs.loadFileSync('name of template file')
const templateCompiled = _.template(template)
const generatedJS = templateCompiled({ `modules`: modules })
fs.writeFileSync('bootModules.js', generatedJS)
Write your template for bootModules.js. Simplest would be:
<% _.forEach(modules , function(module) { %>import '<%= module %>' as <%= module %><% }); %>;
import bootModules.js into your app
Use MY_APP_CONFIGURATION ENV variable to switch desired module configuration - works not just during development but you can also setup different CI processes targeting same repo with just different MY_APP_CONFIGURATION values
This way you have all configurations at one place, you don't need to change package.json before every build, you have simple mechanism to switch between different module configurations and every build (bundle) contains only the modules needed....
In an OOP world I would just use dependency injection retrieving classes implementing a specific interface, but in JS/TS I'm not sure it is viable.
Why not?
More than this, with JS/TS you are not restricted to use classes implementing a specific interface: you just need to define the interface (i.e. the module.exports) of your modules and respecting it in the libraries entries (vue build lib).
EDIT: reading comments probably I understood the request.
Each module should respect following interface (in the file which is the entry of the vue library)
export function isMyAppModule() {
return true;
}
export function myAppInit() {
return { /* what you need to export */ };
}
Than in your app:
require("./package.json").dependencies.forEach(name => {
const module = require(name);
if(! module.isMyAppModule || module.isMyAppModule() !== true) return;
const { /* the refs you need */ } = module.myAppInit();
// use your refs as you need
});

How do I Resolve an Alias with Node Sass in a .mjs Script?

My project's structure has a SCSS library for global settings and such, and main component scss modules are elsewhere with their corresponding component files. I have my Webpack set up to resolve a path alias # when it builds, so my usage of lines like #import "#/src/scss/variables/blah-blah.scss" works out just fine when running the project.
Conversely, I have some utility scripts i'm using to do SCSS benchmarking, just some compile and analyze operations to help illustrate our performance gains while we work on style changes. I have these built using the .mjs file extension, which I don't think is important here, it's just helping play nice with Typescript.
So when Node Sass encounters a path like #import "#/src/scss/variables/blah-blah.scss" it won't resolve the path. No surprise there, but I can't seem to locate a best solution to just help resolve the path within my script, rather than add on NPM packages to do a seemingly simple job. How should I be approaching this?
Error comes up from await sass.render()
import { writeFileSync } from "fs";
import sass from "node-sass";
import tildeImporter from "node-sass-tilde-importer";
import { resolve } from "path";
export const compileCSS = async module => {
const input = `src/components/${module}/scss/module.scss`;
const output = `stats/css/${module}.css`;
await sass.render(
{
file: resolve(input),
importer: tildeImporter,
outputStyle: "expanded",
outFile: resolve(output),
sourceMap: true
},
async function(error, result) {
if (error) {
await console.log(module, "COMPILE ERROR", error.message);
} else {
writeFileSync(resolve(output), result);
return result;
}
}
);
};
Further research shows me that Node Sass does not currently have a great way to enable this, though some ideas appear in the works.
I made a work around by converting my stats gathering operations into a Webpack loader module that analyzes and spits out my data in between steps to pack the compiled styles into the bundle. This way I get the alias path resolved and simplify the dev experience. It's a better solution than I had set out for so I'm pleased.

Packaging-up Browser/Server CommonJS modules with dependancies

Lets say I'm writing a module in JavaScript which can be used on both the browser and the server (with Node). Lets call it Module. And lets say that that Module would benefit from methods in another module called Dependancy. Both of these modules have been written to be used by both the browser and the server, à la CommonJS style:
module.js
if (typeof module !== 'undefined' && module.exports)
module.exports = Module; /* server */
else
this.Module = Module; /* browser */
dependancy.js
if (typeof module !== 'undefined' && module.exports)
module.exports = Dependancy; /* server */
else
this.Dependancy = Dependancy; /* browser */
Obviously, Dependancy can be used straight-out-of-the-box in a browser. But if Module contains a var dependancy = require('dependency'); directive in it, it becomes more of a hassle to 'maintain' the module.
I know that I could perform a global check for Dependancy within Module, like this:
var dependancy = this.Dependancy || require('dependancy');
But that means my Module has two added requirements for browser installation:
the user must include the dependency.js file as a <script> in their document
and the user must make sure this script is loaded before module.js
Adding those two requirements throws the idea of an easy-going modular framework like CommonJS.
The other option for me is that I include a second, compiled script in my Module package with the dependency.js bundled using browserify. I then instruct users who are using the script in the browser to include this script, while server-side users use the un-bundled entry script outlined in the package.json. This is preferable to the first way, but it requires a pre-compilation process which I would have to run every time I changed the library (for example, before uploading to GitHub).
Is there any other way of doing this that I haven't thought of?
The two answers currently given are both very useful, and have helped me to arrive at my current solution. But, as per my comments, they don't quite satisfy my particular requirements of both portability vs ease-of-use (both for the client and the module maintainer).
What I found, in the end, was a particular flag in the browserify command line interface that can bundle the modules and expose them as global variables AND be used within RequireJS (if needed). Browserify (and others) call this Universal Module Definition (UMD). Some more about that here.
By passing the --standalone flag in a browserify command, I can set my module up for UMD easily.
So...
Here's the package.js for Module:
{
"name": "module",
"version": "0.0.1",
"description": "My module that requires another module (dependancy)",
"main": "index.js",
"scripts": {
"bundle": "browserify --standalone module index.js > module.js"
},
"author": "shennan",
"devDependencies": {
"dependancy": "*",
"browserify": "*"
}
}
So, when at the root of my module, I can run this in the command line:
$ npm run-script bundle
Which bundles up the dependancies into one file, and exposes them as per the UMD methodology. This means I can bootstrap the module in three different ways:
NodeJS
var Module = require('module');
/* use Module */
Browser Vanilla
<script src="module.js"></script>
<script>
var Module = module;
/* use Module */
</script>
Browser with RequireJS
<script src="require.js"></script>
<script>
requirejs(['module.js'], function (Module) {
/* use Module */
});
</script>
Thanks again for everybody's input. All of the answers are valid and I encourage everyone to try them all as different use-cases will require different solutions.
Of course you could use the same module with dependency on both sides. You just need to specify it better. This is the way I use:
(function (name, definition){
if (typeof define === 'function'){ // AMD
define(definition);
} else if (typeof module !== 'undefined' && module.exports) { // Node.js
module.exports = definition();
} else { // Browser
var theModule = definition(), global = this, old = global[name];
theModule.noConflict = function () {
global[name] = old;
return theModule;
};
global[name] = theModule;
}
})('Dependency', function () {
// return the module's API
return {
'key': 'value'
};
});
This is just a very basic sample - you can return function, instantiate function or do whatever you like. In my case I'm returning an object.
Now let's say this is the Dependency class. Your Module class should look pretty much the same, but it should have a dependency to Dependency like:
function (require, exports, module) {
var dependency = require('Dependency');
}
In RequireJS this is called Simplified CommonJS Wrapper: http://requirejs.org/docs/api.html#cjsmodule
Because there is a require statement at the beginning of your code, it will be matched as a dependency and therefore it will either be lazy loaded or if you optimize it - marked as a dependency early on (it will convert define(definition) to define(['Dependency'], definition) automatically).
The only problem here is to keep the same path to the files. Keep in mind that nested requires (if-else) won't work in Require (read the docs), so I had to do something like:
var dependency;
try {
dependency = require('./Dependency'); // node module in the same folder
} catch(err) { // it's not node
try {
dependency = require('Dependency'); // requirejs
} catch(err) { }
}
This worked perfectly for me. It's a bit tricky with all those paths, but at the end of the day, you get your two separate modules in different files which can be used on both ends without any kind of checks or hacks - they have all their dependencies are work like a charm :)
Take a look at webpack bundler.
You can write module and export it via module exports. Then You can in server use that where you have module.export and for browser build with webpack. Configuration file usage would be the best option
module.exports = {
entry: "./myModule",
output: {
path: "dist",
filename: "myModule.js",
library: "myModule",
libraryTarget: "var"
}
};
This will take myModule and export it to myModule.js file. Inside module will be assigned to var (libraryTarget flag) named myModule (library flag).
It can be exported as commonJS module, war, this, function
Since bundling is node script, this flag values can be grammatically set.
Take a look at externals flag. It is used if you want to have special behavior for some dependencies. For example you are creating react component and in your module you want to require it but not when you are bundling for web because it already is there.
Hope it is what you are looking for.

Categories

Resources