Develop Reference

Application modularity with Vue.js and local NPM packages

I'm trying to build a modular application in Vue via the vue-cli-service. The main app and the modules are separated projects living in different folders, the structure is something like this:
-- app/package.json
/src/**
-- module1/package.json
/src**
-- module2/package.json
/src**
The idea is to have the Vue app completely agnostic about the application modules that can be there at runtime, the modules themself are compiled with vue-cli-service build --target lib in a local moduleX/dist folder, pointed with the package.json "main" and "files" nodes.
My first idea (now just for development speed purposes) was to add the modules as local NPM packages to the app, building them with a watcher and serving the app with a watcher itself, so that any change to the depending modules would (I think) be distributed automatically to the main app.
So the package.json of the app contains dependencies like:
...
"module1": "file:../module1",
"module2": "file:../module2",
...
This dependencies are mean to be removed at any time, or in general be composed as we need, the app sould just be recompiled and everything should work.
I'm trying to understand now how to dynamically load and activate the modules in the application, as I cannot use the dynamic import like this:
import(/* webpackMode: "eager" */ `module1`).then(src => {
src.default.boot();
resolve();
});
Because basically I don't know the 'module1', 'module2', etc...
In an OOP world I would just use dependency injection retrieving classes implementing a specific interface, but in JS/TS I'm not sure it is viable.
There's a way to accomplish this?

Juggling with package.json doesn't sound like a good idea to me - doesn't scale. What I would do:
Keep all available "modules" in package.json
Create separate js file (or own prop inside package.json) with all available configurations (for different clients for example)
module.exports = {
'default': ['module1', 'module2', 'module3'],
'clientA': ['module1', 'module2', 'module4'],
'clientB': ['module2', 'module3', 'module4']
}
tap into VueCLI build process - best example I found is here and create js file which will run before each build (or "serve") and using simple template (for example lodash) generate new js file which will boot configured modules based on the value of some ENV variable. See following (pseudo)code (remember this runs inside node during build):
const fs = require('fs')
const _ = require('lodash')
const modulesConfig = require(`your module config js`)
const configurationName = process.env.MY_APP_CONFIGURATION ?? 'default'
const modules = modulesConfig[configurationName]
const template = fs.loadFileSync('name of template file')
const templateCompiled = _.template(template)
const generatedJS = templateCompiled({ `modules`: modules })
fs.writeFileSync('bootModules.js', generatedJS)
Write your template for bootModules.js. Simplest would be:
<% _.forEach(modules , function(module) { %>import '<%= module %>' as <%= module %><% }); %>;
import bootModules.js into your app
Use MY_APP_CONFIGURATION ENV variable to switch desired module configuration - works not just during development but you can also setup different CI processes targeting same repo with just different MY_APP_CONFIGURATION values
This way you have all configurations at one place, you don't need to change package.json before every build, you have simple mechanism to switch between different module configurations and every build (bundle) contains only the modules needed....

In an OOP world I would just use dependency injection retrieving classes implementing a specific interface, but in JS/TS I'm not sure it is viable.
Why not?
More than this, with JS/TS you are not restricted to use classes implementing a specific interface: you just need to define the interface (i.e. the module.exports) of your modules and respecting it in the libraries entries (vue build lib).
EDIT: reading comments probably I understood the request.
Each module should respect following interface (in the file which is the entry of the vue library)
export function isMyAppModule() {
return true;
}
export function myAppInit() {
return { /* what you need to export */ };
}
Than in your app:
require("./package.json").dependencies.forEach(name => {
const module = require(name);
if(! module.isMyAppModule || module.isMyAppModule() !== true) return;
const { /* the refs you need */ } = module.myAppInit();
// use your refs as you need
});

Exclude folders from builds in Brocfile

Is there a way to exclude a folder from a build in a Brocfile (or any other place).
The use case is packaging, where I have an app made of sub-apps within pods. eg.
/app/modules/components
/app/modules/app1
/app/modules/app2
/app/modules/app3
I'd like to build them all when environment is set to 'development' or only eg. 'app1' when environment is 'app1'. Any suggestions?
I have tried different combinations of broccoli-file-remover, broccoli-funnel and broccoli-merge-trees to no avail.
var removeFile = require('broccoli-file-remover');
module.exports = removeFile(app.toTree(), {
paths: ['app/modules/pod1/', 'app/modules/pod2/']
});

Ah, so after actually thinking about this clearly, everything is actually working exactly as expected in my previous example.
I clearly wasn't paying enough attention. app.toTree() is far too late to perform this operation, as everything has already been built and concated.
Luckily, ember-cli does enable addons to modify the appropriate trees at various life cycle milestones.
See: https://github.com/ember-cli/ember-cli/blob/master/ADDON_HOOKS.md for more details on which hooks are currently available.
The hook that should do the trick is Addon.prototype.postprocessTree. Now we have two choices, we can build a standalone addon, via ember addon or we can create a light-weight in-repo addon via ember g in-repo-addon. Typically for these types of situations, I prefer in-repo-addons as they don't require a second project, but otherwise they are the same.
ember g in-repo-addon remove
we need to install broccoli-stew via npm install --save broccoli-stew
include it var stew = require('broccoli-stew');
add hook postprocessTree to the add-on
when the postprocessTree is for the type we care about, use broccoli-stew to remove the directories we no longer care care.
The resulting pull request: https://github.com/WooDzu/ember-exclude-pod/pull/1
Note: I noticed template wasn't one of the types available in postprocess, so I added it: https://github.com/ember-cli/ember-cli/pull/4263 (should be part of the next ember-cli release)
Note: we really do want an additional hook
Addon.prototype.preprocessTree, as to ignore the files before we
even build them. I have opened a related issue:
https://github.com/ember-cli/ember-cli/issues/4262
output of the above steps
var stew = require('broccoli-stew');
module.exports = {
name: 'remove',
isDevelopingAddon: function() {
return true;
},
postprocessTree: function(type, tree){
if (type === 'js' || type === 'template') {
return stew.rm(tree, '*/modules/pod{1,2}/**/*');
} else {
return tree;
}
}
};

I am pretty confident broccoli-stew's rm will handle this correctly.
https://github.com/stefanpenner/broccoli-stew/blob/master/lib/rm.js#L4-L40 there are even tests that test a very similar scenario: https://github.com/stefanpenner/broccoli-stew/blob/master/tests/rm-test.js#L48-L57
var stew = require('broccoli-stew');
module.exports = stew.rm(app.tree(), 'app/modules/{pod1,pod2}');
If this doesn't work, feel free to open an issue on broccoli-stew. Be sure to provide a running example though

This is really late, but I created a Broccoli plugin to do just this. It's available at https://www.npmjs.com/package/broccoli-rm.
(The trick is to detect whether an excluded path is a folder, and then use a glob match to make sure that none of the children of the folder get symlinked during copying.)
var rm = require('broccoli-rm');
var input = app.toTree();
module.exports = output = rm([input], {
paths: ['app/modules/pod1', 'app/modules/pod2']
});

browserify - exclude code blocks?

I'm building an app with shared React components in the browser and server-side Node.
Right now, I'm using Marty.js to do this:
function getUser() {
if (Marty.isBrowser) {
/* Get user using some client method */
} else {
/* otherwise, use some secret server code */
}
}
I'm bundling those functions up via Browserify, so they can run on the client as well as the server.
What I'd like to do is remove the else block from the bundle entirely, so I'm not leaking sensitive server-side code.
Is there a way to exclude blocks of code from the bundle?

I would create separate modules, one for the browser and one for the server. Then in your package.json, you tell browserify to use the browser module:
"browser": {
"./path/to/node-module.js": "./path/to/browser-module.js"
}
Now, whereever you call require('path/to/node-module'), browserify will load the other module instead.
More information from the docs:
browser field
There is a special "browser" field you can set in your package.json on a per-module basis to override file resolution for browser-specific versions of files.
For example, if you want to have a browser-specific module entry point for your "main" field you can just set the "browser" field to a string:
"browser": "./browser.js"
or you can have overrides on a per-file basis:
"browser": {
"fs": "level-fs",
"./lib/ops.js": "./browser/opts.js"
}
Note that the browser field only applies to files in the local module, and like transforms, it doesn't apply into node_modules directories.

While I'm not sure if it possible with Browserify, you can do it with Webpack using its DefinePlugin
From the docs (little modified):
Example:
new webpack.DefinePlugin({
DEBUG: false,
PRODUCTION: true,
...
})
...
Example:
if(DEBUG)
console.log('Debug info')
if(PRODUCTION)
console.log('Production log')
After passing through webpack with no minification results in:
if(false)
console.log('Debug info')
if(true)
console.log('Production log')
and then after a minification pass results in:
console.log('Production log')

You can use an environment variable, envify and uglify to do this.
if ('browser' === process.env.ENVIRONMENT) {
...
}
else {
...
}
Set process.env.ENVIRONMENT = 'browser' when doing your browser build, use the envify transform to substitute references to process.env with their current values and uglify will then perform dead code elimination to remove the branches which will never be hit.

Be more explicit about your intent, and put your code in their own files:
function getUser(options, callback) {
var fn;
if (Marty.isBrowser) {
fn = require("./lib/users/get.browser");
} else {
fn = require("./lib/users/get.server");
}
fn(options, callback);
}
and then as a browserify option you can say "replace require("./lib/users/get.server") with this variable instead, when you see it: ..." so that you don't build in that server file when you build for the browser.
However, if getUser can do different things based on where it's running, it feels far more likely that you're doing something wrong here: maybe that getUser should be a REST call to your server from the browser instead, but without more information, that's always hard to determine.

What about putting the code in a module for example UserServer and then exclude that module when you are compiling for the client? Your code becomes:
function getUser() {
if (Marty.isBrowser) {
/* Get user using some client method */
} else {
require('UserServer').getUser();
}
}
Browserify provides the following option to exclude files from the bundle:
--exclude, -u Omit a file from the output bundle. Files can be globs.

How to run node js application programmatically using forever

I need to run the set of node js files which contains the configuration information where It has to run typically port number and IP address then using the forever in node.js I need to run the script in the terminal with the configuration without having any manual input.

For Programmatic approach , you can use Forever-Moniter
var forever = require('forever-monitor');
var child = new (forever.Monitor)('your-filename.js', {
max: 3,
silent: true,
options: []
});
child.on('exit', function () {
console.log('your-filename.js has exited after 3 restarts');
});
child.start();

You could make use of the child_process module. Check the doc, there're some useful information there: http://nodejs.org/api/child_process.html
To give a brief example
var exec = require('child_process').exec;
exec('forever', function callback(error, stdout, stderr){
// cb
});
If you don't need a callback / don't want to wait for the execution:
var exec = require('child_process').exec('forever').unref();
Was that helpful?
Best
Marc
Edit: Ok, not sure if I really got the point of your question, but my answer combined with https://stackoverflow.com/a/23871739/823851 might offer a good solution.

Usage:
forever start hello.js to start a process.
forever list to see list of all processes started by forever
forever stop hello.js to stop the process, or forever stop 0 to stop the process with index 0 (as shown by forever list).

node-config is a good module for managing different configurations of a Node.js app.
For example, to make a "production" config, /config/production.json would contain:
{
"port" : "3000",
"ip" : "192.168.1.1"
}
In one of your node application JS files:
config = require('config')
....
var port = config.port;
var ip = config.ip;
And to launch the app in this configuration, just first set your NODE_ENV to production from the shell before running your app.
export NODE_ENV=production
forever start app.js
Make additional config JSON files as needed for each of your environments. default.json is used when no environment is specified.

JavaScript require() on client side

Is it possible to use require() (or something similar) on client side?
Example
var myClass = require('./js/myclass.js');

You should look into require.js or head.js for this.

I've been using browserify for that. It also lets me integrate Node.js modules into my client-side code.
I blogged about it here: Add node.js/CommonJS style require() to client-side JavaScript with browserify

If you want to have Node.js style require you can use something like this:
var require = (function () {
var cache = {};
function loadScript(url) {
var xhr = new XMLHttpRequest(),
fnBody;
xhr.open('get', url, false);
xhr.send();
if (xhr.status === 200 && xhr.getResponseHeader('Content-Type') === 'application/x-javascript') {
fnBody = 'var exports = {};\n' + xhr.responseText + '\nreturn exports;';
cache[url] = (new Function(fnBody)).call({});
}
}
function resolve(module) {
//TODO resolve urls
return module;
}
function require(module) {
var url = resolve(module);
if (!Object.prototype.hasOwnProperty.call(cache, url)) {
loadScript(url);
}
return cache[url];
}
require.cache = cache;
require.resolve = resolve;
return require;
}());
Beware: this code works but is incomplete (especially url resolving) and does not implement all Node.js features (I just put this together last night).
YOU SHOULD NOT USE THIS CODE in real apps but it gives you a starting point. I tested it with this simple module and it works:
function hello() {
console.log('Hello world!');
}
exports.hello = hello;

I asked myself the very same questions. When I looked into it I found the choices overwhelming.
Fortunately I found this excellent spreadsheet that helps you choice the best loader based on your requirements:
https://spreadsheets.google.com/lv?key=tDdcrv9wNQRCNCRCflWxhYQ

Take a look at requirejs project.

I have found that in general it is recommended to preprocess scripts at compile time and bundle them in one (or very few) packages with the require being rewritten to some "lightweight shim" also at compile time.
I've Googled out following "new" tools that should be able to do it
http://mixu.net/gluejs/
https://github.com/jrburke/almond
https://github.com/component/builder2.js
And the already mentioned browserify should also fit quite well - http://esa-matti.suuronen.org/blog/2013/04/15/asynchronous-module-loading-with-browserify/
What are the module systems all about?
Older Stack Overflow explanation - Relation between CommonJS, AMD and RequireJS?
Detailed discussion of various module frameworks and the require() they need is in Addy Osmani - Writing Modular JavaScript With AMD, CommonJS & ES Harmony

You can create elements to the DOM, which loads items.
Like such:
var myScript = document.createElement('script'); // Create new script element
myScript.type = 'text/javascript'; // Set appropriate type
myScript.src = './js/myclass.js'; // Load javascript file

Simply use Browserify, what is something like a compiler that process your files before it go into production and packs the file in bundles.
Think you have a main.js file that require the files of your project, when you run browserify in it, it simply process all and creates a bundle with all your files, allowing the use of the require calls synchronously in the browser without HTTP requests and with very little overhead for the performance and for the size of the bundle, for example.
See the link for more info: http://browserify.org/

Some answers already - but I would like to point you to YUI3 and its on-demand module loading. It works on both server (node.js) and client, too - I have a demo website using the exact same JS code running on either client or server to build the pages, but that's another topic.
YUI3: http://developer.yahoo.com/yui/3/
Videos: http://developer.yahoo.com/yui/theater/
Example:
(precondition: the basic YUI3 functions in 7k yui.js have been loaded)
YUI({
//configuration for the loader
}).use('node','io','own-app-module1', function (Y) {
//sandboxed application code
//...
//If you already have a "Y" instance you can use that instead
//of creating a new (sandbox) Y:
// Y.use('moduleX','moduleY', function (Y) {
// });
//difference to YUI().use(): uses the existing "Y"-sandbox
}
This code loads the YUI3 modules "node" and "io", and the module "own-app-module1", and then the callback function is run. A new sandbox "Y" with all the YUI3 and own-app-module1 functions is created. Nothing appears in the global namespace. The loading of the modules (.js files) is handled by the YUI3 loader. It also uses (optional, not show here) configuration to select a -debug or -min(ified) version of the modules to load.

Here's a solution that takes a very different approach: package up all the modules into a JSON object and require modules by reading and executing the file content without additional requests.
https://github.com/STRd6/require/blob/master/main.coffee.md
STRd6/require depends on having a JSON package available at runtime. The require function is generated for that package. The package contains all the files your app could require. No further http requests are made because the package bundles all dependencies. This is as close as one can get to the Node.js style require on the client.
The structure of the package is as follows:
entryPoint: "main"
distribution:
main:
content: "alert(\"It worked!\")"
...
dependencies:
<name>: <a package>
Unlike Node a package doesn't know it's external name. It is up to the pacakge including the dependency to name it. This provides complete encapsulation.
Given all that setup here's a function that loads a file from within a package:
loadModule = (pkg, path) ->
unless (file = pkg.distribution[path])
throw "Could not find file at #{path} in #{pkg.name}"
program = file.content
dirname = path.split(fileSeparator)[0...-1].join(fileSeparator)
module =
path: dirname
exports: {}
context =
require: generateRequireFn(pkg, module)
global: global
module: module
exports: module.exports
PACKAGE: pkg
__filename: path
__dirname: dirname
args = Object.keys(context)
values = args.map (name) -> context[name]
Function(args..., program).apply(module, values)
return module
This external context provides some variable that modules have access to.
A require function is exposed to modules so they may require other modules.
Additional properties such as a reference to the global object and some metadata
are also exposed.
Finally we execute the program within the module and given context.
This answer will be most helpful to those who wish to have a synchronous node.js style require statement in the browser and are not interested in remote script loading solutions.

I find the component project giving a much more streamlined workflow than other solutions (including require.js), so I'd advise checking out https://github.com/component/component . I know this is a bit late answer but may be useful to someone.

Here's a light weight way to use require and exports in your web client. It's a simple wrapper that creates a "namespace" global variable, and you wrap your CommonJS compatible code in a "define" function like this:
namespace.lookup('org.mydomain.mymodule').define(function (exports, require) {
var extern = require('org.other.module');
exports.foo = function foo() { ... };
});
More docs here:
https://github.com/mckoss/namespace

The clientside-require library provides an asynchronous load() function that can be used to load any JS file or NPM module (which uses module.exports), any .css file, any .json, any .html, any any other file as text.
e.g.,
npm install clientside-require --save
<script src = '/node_modules/clientside-require/dist/bundle.js'></script>
<script>
load('color-name') // an npm module
.then(color_name=>{
console.log(color_name.blue); // outputs [0, 0, 255]
})
</script>
A really cool part of this project is that inside of any load()ed script, you can use the synchronous require() function the same way you would expect in node.js!
e.g.,
load('/path/to/functionality.js')
and inside /path/to/functionality.js:
var query_string = require("qs") // an npm module
module.exports = function(name){
return qs.stringify({
name:name,
time:new Date()
}
}
That last part, implementing the synchronous require() method, is what enables it to utilize NPM packages built to run on the server.
This module was designed to implement the require functionality as closely as possible in the browser. Disclaimer: I have written this module.

Yes it is very easy to use, but you need to load javascript file in browser by script tag
<script src="module.js"></script>
and then user in js file like
var moduel = require('./module');
I am making a app using electron and it works as expected.