I m actually studying continuous integration and I m actually facing a (little) problem when dealing with the build sequence of the process.
Actually, I have an application that has the following directory at the root of the project:
src
doc
dist
tests
node_modules
My question is : when I m at the building step (building the last artifact to put in production, after testing process) should I copy the node_modules directory inside of the dist folder ? So this dist folder could work in standalone (with minification etc... etc...) and so I have only to deploy this folder in my prod environnement ?
How can I move only "dependencies" modules and not "devDependencies" modules ?
You don't need to copy anything, because Node when you require a module within your Node app, it will search for node_modules in current directory, and if the dependency is not found, it will try to search in its parent and so on.
Check out how Node looks for a package here: http://mycodesmells.com/post/node-basics-looking-for-package/
If you don't want to have development dependencies in your production environment, you can install only non-dev ones:
npm install --production
Source: https://docs.npmjs.com/cli/install
Related
even if it seems a simple task I'm having some trouble finding a solution. I know that with Nest CLI I can use the command "nest build" in order to create a dist folder that contains the production files of my project.
The problem is when I move the folder on my Raspberry and i try to run the project with the command "node dist/main" following NestJs instructions. Nothing starts because node says that it cannot find #nestjs/core and other modules.
I did't find nothing clear in the official guide about deploying the app, so my question is: what do I need to move onto my rasperry in addition to dist folder? Do I need to reinstall all node_modules folder or it's possible to have a running project without have to reinstall 800Mb of modules?
Yes you need to run yarn or npm install on your production environment, as your dist folder only contains your own code.
Because unlike compiled language like Golang where all dependencies are bundled and compiled in you executable file, Javascript bundlers don't. Your bundled code in dist still contains require or import statement to get dependencies from node_modules.
You can also run npm prune --production to remove any developpement dependencies and thus reduce the size of your node_modules folder. (I believe that yarn does it by default.)
I'm trying to publish a module I have created.
The module has multiple entries in it and I want to publish from within my build folder.
Every time I try to publish with yarn. I get:
Output:
The following paths are ignored by one of your .gitignore files:
build/package.json
Use -f if you really want to add them.
I tried yarn publish build
And also cd build && yarn publish where I copy the package.json and npmignore and all the relevant files.
It's always the same result.
I have a .gitignore in my root and an .npmignore in my root.
If I publish the build folder from the root it works but I dont want the path to include the build/lib folder.
If I do yarn build && cd build && npm publish && cd ../ that will also work.
But I'd rather use yarn.
Does anyone have a solution for it?
All I want is to publish my already created build folder content. I have all the needed files there.
EDIT
After much research and looking into other packages,
I ended up just creating a small copy script that copies my essential files to a build folder and clears up my package.json from all unwanted items.
Then my ci publishes from the build folder.
That was our solution that we've implemented
How does my node.js server know how to find the express.js file?
In my server.js file I have a requirement for express:
var express = require('express');
var app = express();
My server.js file is in the app folder while express.js is in the app\node_modules\express\lib folder:
express.js is here in the lib directory
Does node.js automatically know to look in node_modules for dependencies? There is no direct enumeration of the path anywhere - I don't see it in the code.
This is a sample project from scotch.io and I'm trying to pick it apart and learn. I'm pretty new to the MEAN stack and I'm trying to understand this at a basic level.
FYI this is node v 4.5.0
The comments already cover the main answer, but I'll try to break it out here for a slightly more complete overview of the subject (not all encompassing). The short version is that node looks in node's core (built in) packages, then your project's node_modules path for modules (packages, but we generally require modules) matching the name. 1
Package Installation, Saving, and Location
Node uses npm to install dependencies, which can be either a "dependency" or "devDependency"; the latter is used for development concerns that shouldn't be required for just normally using the module. We save these to our project's package.json using the flag --save or --save-dev (such as npm install express --save). 2
The package.json file is what sits at the root of your project tree (the project's folder/directory) and contains the dependency information, along with other information. 3 This is defines a "package". When a person publishes a package to npmjs, the (default) registry for packages to install via npm, they should include a well formed package.json which lists its own dependencies, files to include, and what is the "main" file to start with, should it be used in a require statement.
The dependencies you install by running npm install after cloning down the project's repository, will install the packages specified in the package.json, into the node_modules path in the root of your project (where you should be running the install command from).
Side Note
After checking the GitHub repo listed by the article you referenced, you appear to have created each of the files inside the app/ directory.
Require-ing
The use of a require statement in node is in the style of CommonJS and for node, looks first (after core packages) in the node_modules/ path or, if you specify a relative path to a folder or file, you can get it directly. Path relative means that it starts with ./ for a prefix to the current working directory (that the file is executing from), ../ for the directory above the current one, etc. There are other ways of handling paths, such as as the path module that is built into node 4. For example, the following are valid:
require('./some-other.js') require in that file's module.exports, from the some-other.js file 5, in the current, relative path
require('./some.json') will bring in JSON formatted content from a .json file 6, in the current, relative path
require('./routes') will/can also bring in exported content from the routes/ path (directory), which will by default start with the directory's index.js, should it exist 7
That last method is a nice way to be able to bring in a more complex requirement, without keeping everything in a single, overly busy file.
Let's consider the possible sources of modules:
Core modules provided by Node.js.
Package modules from doing npm install. These modules are stored in node_modules folder which is usually located in the root of the project.
Modules in any other location in your project (Usually modules created by you)
If you require modules without any prefix e.g require('a_module'), the core modules are searched first, If it's not found, the package modules are searched next. See the Node.js docs here
If you require modules with prefix / or ./, e.g require('/another_module') , require(./another_module), another_module is considered relative to the location of the requiring file. This is how you would require modules in any other location.
Check Node.js modules docs for further reading.
Whenever I make projects, I have to download all dependencies of node modules. Without copying the node_modules, Is there anyway to share the central node_modules in multiple projects?
like the followings, I have to run many commands every time..
npm install gulp-usemin
npm install gulp-wrap
npm install gulp-connect
npm install gulp-watch
npm install gulp-minify-css
npm install gulp-uglify
npm install gulp-concat
npm install gulp-less
npm install gulp-rename
npm install gulp-minify-html
You absolutely can share a node_modules directory amongst projects.
From node's documentation:
If the module identifier passed to require() is not a native module,
and does not begin with '/', '../', or './', then node starts at the
parent directory of the current module, and adds /node_modules, and
attempts to load the module from that location.
If it is not found there, then it moves to the parent directory, and
so on, until the root of the file system is reached.
For example, if the file at '/home/ry/projects/foo.js' called
require('bar.js'), then node would look in the following locations, in
this order:
/home/ry/projects/node_modules/bar.js /home/ry/node_modules/bar.js
/home/node_modules/bar.js /node_modules/bar.js
So just put a node_modules folder inside your projects directory and put in whatever modules you want. Just require them like normal. When node doesn't find a node_modules directory in your project folder, it will check the parent folder automatically. So make your directory structure like this:
-myProjects
--node_modules
--myproject1
---sub-project
--myproject2
So like this, even your sub-project's dependencies can draw on your main node_modules repository.
One drawback to doing it this way is you will have to build out your package.json file manually (unless someone knows a way to automate this with grunt or something). When you install your packages and add the --save arg to an npm install command it automatically appends it to the dependencies section or your package.json, which is convenient.
Try pnpm instead of npm.
pnpm uses hard links and symlinks to save one version of a module only ever once on a disk.
If you have npm installed, you can install in your terminal with:
npm install -g pnpm
To update your existing installations (and sub-directories) use:
pnpm recursive install
Or use the shorthand command (leave off -r if you need to target only one directory)
pnpm -r i
One helpful note: You may find some rare packages don't have all their dependencies defined. They might rely on the flat node_modules file directory structure of npm or yarn installs. If you run into issues of missing dependencies, use this command to hoist all the sub dependencies into a flat-file structure:
pnpm install --shamefully-hoist
It's best to avoid using the --shamefully-hoist flag as it defeats the purpose of using pnpm in the first place, so try using the command pnpm i your-missing-package first (See pnpm FAQ).
I found a trick, just take a look at the Symbolic Links (symlinks) on Windows or Linux, it is working just like shortcuts but more powerful.
Simply you need to make a Junction for your node_modules folder anywhere you want. The junction is nothing but a short cut to your original node_modules folder. Create it inside your project folder where the actual node_modules would have been created if used npm install.
To achieve this you need at least one node_modules real folder then make a Junction to it in the other projects.
On Windows, you can either use the Command Prompt, or use an application. Using the Command Prompt gives you a bit more control, using an application is easier I suggest Link Shell Extension.
Main directory should look like this
node_modules
Project 1
Project 2
Project 3
Project 4
just open the file Project 1/.angular-cli.json
change the schema
"$schema": "./node_modules/#angular/cli/lib/config/schema.json",
to
"$schema": "./../node_modules/#angular/cli/lib/config/schema.json"
and don't forget to create node_modules empty folder inside your project directory
See also npm v7.0.0's support for workspaces
RFC
https://github.com/npm/rfcs/blob/latest/implemented/0026-workspaces.md
Documentation
https://docs.npmjs.com/cli/v7/using-npm/workspaces
By looking at some articles it seems that Lerna
is a good tool for managing multiple projects inside a single directory (monorepo). It supports modules sharing without duplicating the entire packages in every folder and commands to install them in multiple projects.
Javascript monorepos
Monorepos by example
Building large scale apps in a monorepo
pnpm is also a simple and efficient tool, which doesn't duplicate those modules which are already installed for other projects.
Let's assume that having a single node_modules it should contain all the packages for all applications. thus your apps will also share most of the unique package.json entries (just the name should change)
my idea would be to have a single root and multiple src level as below
root\package.json
root\node_modules
root\\..
root\app1\src\\..
root\app2\src\\..
the only issue you might face would be having a backup of json (or tsconfig) for any app and restore them when you work on it or setup your startup scripts to serve any app
this is probably a silly question but am new to Meteor and struggling a bit. I want to build a stellar app that tweets when you get stellar. There is a nice Javascript API stellar-lib that works on node, but im unsure how to access the modules in Meteor...
I'm also building an app with Meteor and stellar-lib and have found two options:
1) You can manually copy over the built stellar-lib.js or stellar-lib-min.js from the build/ directory from either the github repo or the node_modules folder you installed it to when you ran the npm install command.
If you do this, you will have to copy the .js file to client/compatibility, otherwise stellar-lib will not work (note: this means you can only use Stellar on the client).
2) If you need it on the server, you can also have browserify wrap stellar-lib for you, then copy the output to lib/ in your Meteor app's root directory. I did this in my repo here with gulp.
Here's a short explanation to how 2) works:
.gulp is where I'll install my NPM modules where they will be ignored by the Meteor build environment (because the folder is hidden).
In deps.js, I require the modules I would want to use in my Meteor app as I would if I was using them in a traditional node.js app. package.json defines the modules I'll install with NPM and the gulpfile.js describes a build task that will resolve my require statements and output a single deps.js file that includes my dependencies to my Meteor app's lib/ folder.