Develop Reference

Share code between two local node projects

I have the following directory structure
./app1/app1.js
./app1/package.json
./app1/node_modules
./app2/app2.js
./app2/package.json
./app2/node_modules
./shared/share.js
Both app1 and app2 pull in share.js
// app1.js
const share = require('../shared/share.js')
share.js uses a module defined in both app1 and app2's package.json
// share.js
const toml = require('toml')
When running node app1.js
Error: Cannot find module 'toml'
How can I share the share.js file between these apps? I'd prefer not to use symlinks if possible.

One of the solutions that you could use here would be using yarn workspaces.
That would allow you to share your packages across apps so you can have structure like:
- project1
-- package.json (deps for project1)
- project2
-- package.json (deps for project2)
- sharedPackage
-- package.json (deps for shared)
- package.json (your root of project deps where you define workspaces)
And now you are able to use your shared package as regular dependency!
Somewhere inside you project1/package.json
"dependencies": {
"sharedPackage": "1.0.0"
}
And use it in project project1/index.js
const shared = require('sharedPackage');
shared.doSomething();
Yarn will know about your dependencies inside of your workspaces, so it will let you include them and do all of the linking without pain. More to that you can even publish your shared package to some package registry (NPM/Github/whatever) and make it just a package.
That will do the work, but if you are looking for some more scalable solution to manage dependencies I'd really recommend to try lerna or rush.

Using local modules with Webpack 4 is bundling the same dependency multiple times

I am attempting to find a good way to use local modules in npm, or a way of structuring a large application so it can be bundled off into modules which may or may not be in a separate repository.
Each local module has it's own package.json and dependencies which are installed.
My requirements are that the modules are written in ES6 and only compiled as part of the main project being built (so I don't have lots of dependencies being indiependently built constantly).
Project structure
/root
/main-module
... main js files <- entry point
webpack.config.js
package.json
/module-1
... module 1 js files
package.json
/module-2
... module 2 js files
package.json
/module-3
... module 3 js files
package.json
I'm currently investigating using local modules via specifying a local file in my package.json like so:
...
"dependencies": {
"lodash": "^4.17.10",
"module-1": "../module-1",
"module-2": "../module-2",
"module-3": "../module-3",
"normalize.css": "^8.0.0"
}
...
You can see the whole project here: https://github.com/SamStonehouse/webpack-local-modules-test
I'm using webpack with the babel-loader which doesn't need any extra setup in order to use this form and even watches the module file for changes and rebuilds when they're complete which is amazing.
Issue: once this has built lodash is included in the built bundle 4 times over, one for each module which requires it, even though they all require the same version and all the sources are compiled at the same time.
I've tried using the splitChunkPlugin but to no avail
I've tried setting lodash as a devDependency in the local modules (this was something I didn't want to do but it didn't work anyway)
Does anyone have a solution for this?
Or an alternative way of bundling local modules in a similar fashion

Change each of the modules to have lodash as a peer dependency instead of a direct dependency. So in the package.json file, change this:
"dependencies": {
"lodash": "^4.17.5"
}
To:
"peerDependencies": {
"lodash": "^4.17.5"
}

webpack not copying images

I trying to import local package, that already bundled in webpack. When I build an app everything looks good, except webpack not copying images, fonts, etc. from that internal package. Any ideas what's going wrong?
When I try to reference my custom library, that is already bundled:
- dist
-- asdfjlsjdflasdjflsdaf.ttf
-- sdjflsjdfasjdflsjdfsd.eot
-- mylibrary.js
only mylibrary.js is copied. So parent app looks like:
- dist
-- mylibrary.js
-- parentapp.js
- packages
-- mylibrary
--- src...
--- dist...
Problem could be solved:
by increasing the loader limit and everything that converted to
base64 and included in mylibrary.js is available in my parent app
copy plugin, which copy fonts from mylibrary, but this solution looks ugly, since everyone who using mylibrary should care about copying fonts, images, etc.

A work around for copying images and fonts using webpack would be the use of GlobCopyWebpackPlugin.
How it works (I am not quite sure how your current webpack configuration looks like) but let`s go with a blank slate and see if this might help you. So, you can keep separate webpack files for different environments - say for now a prod and dev. Once you have created webpack.config.dev.js, inside the "plugins" property you can use GlobCopyWebpackPlugin. It would look something like below:
new GlobCopyWebpackPlugin({
"patterns": [
"Assets"
],
"globOptions": {
"cwd": path.join(process.cwd(), "src"),
"dot": true,
"ignore": "**/.gitkeep"
}
}),
"Assets" is the folder where you have kept fonts, icons, multimedia etc.
And then you can configure your tasks inside package.json
"scripts": {
"serve": "npm run -s generatelocalefiles && node --max_old_space_size=4096 ./node_modules/webpack-dev-server/bin/webpack-dev-server.js --config webpack.config.dev.js --port=4300",
}
And the folder structure looks like this (just in case you are wondering)
-src
-app
-assets
-package.json
-webpack.config.dev.js
-webpack.config.prod.js
Hope this helps!

The best way to understand is to see how webpack takes care of images, fonts etc.
Here is a module that uses bootstrap and has the required concepts in action.
The index.html uses bootstrap styles and one glyph as well. The glyph (and bootstrap in all) uses one image from bootstrap binaries.
After you build the module, you can compare the paths of the image in original bootstrap binary (i.e. node_modules/bootstrap/dist/css/bootstrap.css) and the bundled output file (i.e. dist/bundle.js). You can search for glyphicons-halflings-regular.svg in those files.
The webpack simply
Copies images/fonts to your configured (in loader config) output folders
Updates those resource paths in the bundled output files (i.e. js, css etc)
(1) above need to be configured by you via a loader for ex. (2) will be automatically taken care by webpack provided the module.rules in webpack config are working properly for the images/fonts.
Hope that makes things a bit clear.

How to use the 'main' parameter in package.json?

I have done quite some search already. However, still having doubts about the 'main' parameter in the package.json of a Node project.
How would filling in this field help? Asking in another way, can I start the module in a different style if this field presents?
Can I have more than one script filled into the main parameter? If yes, would they be started as two threads? If no, how can I start two scripts in a module and having them run in parallel?
I know that the second question is quite weird. It is because I have hosted a Node.js application on OpenShift but the application consists of two main components. One being a REST API and one being a notification delivering service.
I am afraid that the notification delivering process would block the REST API if they were implemented as a single thread. However, they have to connect to the same MongoDB cartridge. Moreover, I would like to save one gear if both the components could be serving in the same gear if possible.
Any suggestions are welcome.

From the npm documentation:
The main field is a module ID that is the primary entry point to your
program. That is, if your package is named foo, and a user installs
it, and then does require("foo"), then your main module's exports
object will be returned.
This should be a module ID relative to the root of your package
folder.
For most modules, it makes the most sense to have a main script and
often not much else.
To put it short:
You only need a main parameter in your package.json if the entry point to your package differs from index.js in its root folder. For example, people often put the entry point to lib/index.js or lib/<packagename>.js, in this case the corresponding script must be described as main in package.json.
You can't have two scripts as main, simply because the entry point require('yourpackagename') must be defined unambiguously.

To answer your first question, the way you load a module is depending on the module entry point and the main parameter of the package.json.
Let's say you have the following file structure:
my-npm-module
|-- lib
| |-- module.js
|-- package.json
Without main parameter in the package.json, you have to load the module by giving the module entry point: require('my-npm-module/lib/module.js').
If you set the package.json main parameter as follows "main": "lib/module.js", you will be able to load the module this way: require('my-npm-module').

If you have for instance in your package.json file:
{
"name": "zig-zag",
"main": "lib/entry.js",
...
}
lib/entry.js will be the main entry point to your package.
When calling
require('zig-zag');
in node, lib/entry.js will be the actual file that is required.

As far as I know, it's the main entry point to your node package (library) for npm. It's needed if your npm project becomes a node package (library) which can be installed via npm by others.
Let's say you have a library with a build/, dist/, or lib/ folder. In this folder, you got the following compiled file for your library:
-lib/
--bundle.js
Then in your package.json, you tell npm how to access the library (node package):
{
"name": "my-library-name",
"main": "lib/bundle.js",
...
}
After installing the node package with npm to your JS project, you can import functionalities from your bundled bundle.js file:
import { add, subtract } from 'my-library-name';
This holds also true when using Code Splitting (e.g. Webpack) for your library. For instance, this webpack.config.js makes use of code splitting the project into multiple bundles instead of one.
module.exports = {
entry: {
main: './src/index.js',
add: './src/add.js',
subtract: './src/subtract.js',
},
output: {
path: `${__dirname}/lib`,
filename: '[name].js',
library: 'my-library-name',
libraryTarget: 'umd',
},
...
}
Still, you would define one main entry point to your library in your package.json:
{
"name": "my-library-name",
"main": "lib/main.js",
...
}
Then when using the library, you can import your files from your main entry point:
import { add, subtract } from 'my-library-name';
However, you can also bypass the main entry point from the package.json and import the code splitted bundles:
import add from 'my-library-name/lib/add';
import subtract from 'my-library-name/lib/subtract';
After all, the main property in your package.json only points to your main entry point file of your library.

One important function of the main key is that it provides the path for your entry point. This is very helpful when working with nodemon. If you work with nodemon and you define the main key in your package.json as let say "main": "./src/server/app.js", then you can simply crank up the server with typing nodemon in the CLI with root as pwd instead of nodemon ./src/server/app.js.

From the Node.js getting started documentation, it states;
An extra note: if the filename passed to require is actually a directory, it will first look for package.json in the directory and load the file referenced in the main property. Otherwise, it will look for an index.js.

For OpenShift, you only get one PORT and IP pair to bind to (per application). It sounds like you should be able to serve both services from a single nodejs instance by adding internal routes for each service endpoint.
I have some info on how OpenShift uses your project's package.json to start your application here: https://www.openshift.com/blogs/run-your-nodejs-projects-on-openshift-in-two-simple-steps#package_json

How do I connect bower components with sails.js?

I'd like to be able to install Javascript dependencies through bower and use them in a sails.js app, but I can't figure out a way to do this with out just copying an pasting files from the bower_components folder to the Sails assets folder.
Ideally I think I'd like to use requirejs and point to the bower components in the main.js file. I may be trying to pound a square peg in a round hole, please let me know if so. Any thoughts on managing components/libraries with Sails are welcome.

In Sails 0.10 the 'assets/linker' directory no longer exists, however I found a lead on simple solution that gives some automation to the bower -> asset linker workflow while also allowing some granular control over what exactly ends up getting linked.
The solution is adding grunt-bower to your Sails.js compileAssets task
grunt.registerTask('compileAssets', [
'clean:dev',
'bower:dev',
'jst:dev',
'less:dev',
'copy:dev',
'coffee:dev'
]);
Then configure your grunt-bower task like so (as tasks/config/bower.js):
module.exports = function(grunt) {
grunt.config.set('bower', {
dev: {
dest: '.tmp/public',
js_dest: '.tmp/public/js',
css_dest: '.tmp/public/styles'
}
});
grunt.loadNpmTasks('grunt-bower');
};
This will automatically copy your bower js and css files to the proper place for Sail's asset linker to find and automatically add to your project's layout template. Any other js or css files will still automatically be added after your bower files.
However this is still no silver bullet as this setup has 2 big caveats to it:
1 - The files that are added through bower-grunt have to be listed in bower.json's main array. If you see a file isn't being loaded you would expect to be, you must either edit that packages bower.json file OR add the dependency manually through grunt-bower's packageSpecific options.
2 - The order of bower files in the asset linker is currently alphabetical. My only recourse to adjust this order so far is tinkering around with an additional grunt task to manually re-order files prior to the rest of Sail's compileAssets task. However this one I'm confident there is something grunt-bower could do by supporting package copy ordering.

Note: the following answer is no longer completely relevant to the current version of SailsJS because there is no support for the linker folder as of SailsJS 0.10.
See: Sails not generating linker
Original answer:
I was able to figure out a solution for this, which is actually pretty simple. I had not realized you could configure where bower places it's files.
Create a .bowerrc file and change the directory where bower components are installed, in the case of Sailjs they should be put into the assets folder.
/*
* Create a file called .bowerrc and put the following in it.
* This file should be in the root directory of the sails app.
*/
{
"directory": "assets/linker/bower_components"
}
Sails will then use grunt to copy them to the .tmp/public/assets folder whenever a file is change. If you don't wish to have sails continually deleting and then recopying those files you can exclude them in the grunt file.
/*
* This is not necessary, but if you have a lot of components and don't want
* them constantly being deleted and copied at every file change you can update
* your Gruntfile.js with the below.
*/
clean: {
dev: ['.tmp/public/**',
'!.tmp/public',
'!.tmp/public/bower_components/**'],
build: ['www']
},
One tip on using requirejs with sails. By default you will get an error from the socket.io file since sails will load it without using requirejs. This will throw an error since socket.io supports amd style loading, more details here http://requirejs.org/docs/errors.html#mismatch.
The simplest way to fix this is to just comment out the lines near the end of the socket.io.js.
/*
* Comment the below out in the file assets/js/socket.io.js, if using requirejs
* and you don't want to modify the default sails setup or socket.io.
*/
if (typeof define === "function" && define.amd) {
define([], function () { return io; });
}
The other way would be to recode the sails files in assets/js named "socket.io.js", "sails.io.js" and app.js to be amd modules.

The simplest way I've found of achieving this is simply to add the individual Bower components to your tasks/pipeline.js file. For example, here's how you might add Modernizr:
// Client-side javascript files to inject in order
// (uses Grunt-style wildcard/glob/splat expressions)
var jsFilesToInject = [
// Load sails.io before everything else
'js/dependencies/sails.io.js',
// Dependencies like jQuery, or Angular are brought in here
'js/dependencies/**/*.js',
// =========================================================
// Modernizr:
'bower_components/modernizr/modernizr.js',
// =========================================================
// All of the rest of your client-side js files
// will be injected here in no particular order.
'js/**/*.js'
];
A link to bower_components/modernizr/modernizr.js then gets inserted in the <!--SCRIPTS--> placeholder in your layout template, and it also gets minified, etc, when you run sails lift --prod.

Sorry for my late.
I think include bower_components in linker it's a bad idea, because when you will lift sails, everything in it will be copied in .tmp and include in tags.
For example, if you have include Angular with bower, you will have two inclusions in your script: one for angular.js and one for angular.min.js. And having both is a mistake.
Here is my solution on my projects :
I have created a folder bower_component in my root directory.
I have added this line in my Gruntfile.js in the array files in copy:dev
{ '.tmp/public/linker/js/angular.js': './bower_components/angular/angular.js' }
And for each files I want to include, I need to manually add a new line in this array. I haven't find an another automatic solution. If someone finds a better solution... ;)

There is more than one approach to hooking up SailsJS and Bower.
A package for SailsJS that integrates Bower via a custom generator exists here:
https://www.npmjs.com/package/sails-generate-bower
There is one for Gulp as well.