We have more than 10 instances of prod servers and each time we update our dependencies, so cleaning and re-installing sounds more controlled, but also a bit slower.
Problem is the devops team complain about the time taken to do a clean (after removing existing node_modules) npm install everytime package.json changes.
We have noticed sometimes our build breaks on prod if we do run update or install on existing node_modules.
Are there any best practices for production deployment?
How can i optimize the process of updating the node_modules safely here?
In newer npm version, there is a feature for locking the version of your dependencies.
There is a file called package-lock.json along with package.json. That lock file
will lock the dependency version while you install it on dev environment. So, when you install it on production from package.json using npm install, it will fetch locked version from package-lock.json file and will install specific version same as dev environment.
That means you don't need to clean node_module folder every time on production. You can just install new added dependency from package.json file and its version will be taken from package-lock.json file.
There is another package manager called "yarn" is there which provide same feature, but if you want to stick to npm, then its now possible with new npm version.
After a couple of months breaking my head on this. I came across this package on NPM:
npm-check-updates.
npm-check-updates allows you to upgrade your package.json dependencies to the latest versions.
All you have to do is run
npm install npm-check-updates --save
ncu -u
npm install
This works very nicely for me.
I just switched to gulp task runner to automate my workflow, but there is this problem whenever i want to start a new project i have to install all packages required in gulpfile.js using the following command:
npm install --save-dev {package name}
Imagine there are 20 of them, it's a bit boring. How can simplify this?
You can add multiple package names to npm install:
npm install --save-dev package1 package2 package3
npm will install and save the specified packages in your package.json.
Personally I use mostly the same gulp plugins for all of my projects. I copy the devDependencies bit from the package.json of one of my previous projects into my newly created package.json, then I run npm i which installs all dependencies listed in package.json. It's a huge timesaver, especially since I usually copy my gulpfile.js as well.
Note: don't forget to run npm outdated if it's been a while since your previous project started, to check if any of the dependencies have been updated in the meantime.
You can also use brace expansion for installing many similarly named packages:
npm i -D babel-{core,preset-es2015,preset-react}
Guys I know that using npm install -g we can install the node modules/packages globally, but I am not sure about the options --save and --save-dev
I have Googled it but still not clear about it. Please share your thoughts.
--save adds the third-party package to the package's dependencies. It will be installed together with the package whenever someone runs npm install yourPackage.
--save-dev adds the third-party package to the package's development dependencies. It won't be installed when someone installs your package. It's typically only installed if someone clones your source repository and runs npm install in it.
Dev dependencies, as the same suggests, are those dependencies that are only needed for developing the package. That can include test runners, compilers, packagers, etc.
Both types of dependencies are stored in the package's package.json file. --save adds to dependencies, --save-dev adds to devDependencies. From the documentation:
devDependencies
If someone is planning on downloading and using your module in their program, then they probably don't want or need to download and build the external test or documentation framework that you use.
In this case, it's best to map these additional items in a devDependencies object.
These things will be installed when doing npm link or npm install from the root of a package, and can be managed like any other npm configuration param. See npm-config(7) for more on the topic.
For build steps that are not platform-specific, such as compiling CoffeeScript or other languages to JavaScript, use the prepublish script to do this, and make the required package a devDependency.
Edit: As of npm 5.0.0 the installed modules are added as a dependency by default, so the --save option is no longer needed.
--save-dev is used to save the package for development purpose. Example: unit tests, minification.
--save is used to save the package required for the application to run.
--save-dev will save npm modules in development dependencies in package.json i.e it will save inside devDependencies object.
--save will save npm modules dependencies in package.json i.e it will save inside dependencies object.
I'm new and I need to use node for off-line use so, I'm trying to understand how the install modules work.
It's the same if I use npm install express or included it in the package.json?
The way a package is installed will be the same whether you manually type npm install express or put it in your package.json and then do npm install. The difference comes when you try to install your Node project elsewhere.
For example, if your code was checked into GitHub and you didn't include a package.json with all of the dependencies listed, then when the project was downloaded you would have to manually re-install all of the dependencies on the command line in order for it to work. But if you had checked in a package.json with the code, then you could run npm install to install all of the dependencies at once, and not have to remember which ones were necessary.
In addition, the package.json allows you to specify an "approximate version" of a dependency to use. This way if a few packages in your project share a dependency and they all specify similar "approximate versions", only one version will be installed and it will be shared between packages. This saves some install time.
Nothing actually. But you don't want to do that again and again. So you might as well put your module dependencies in your package.json
Can someone tell me where can I find the Node.js modules, which I installed using npm?
Global libraries
You can run npm list -g to see which global libraries are installed and where they're located. Use npm list -g | head -1 for truncated output showing just the path. If you want to display only main packages not its sub-packages which installs along with it - you can use - npm list --depth=0 which will show all packages and for getting only globally installed packages, just add -g i.e. npm list -g --depth=0.
On Unix systems they are normally placed in /usr/local/lib/node or /usr/local/lib/node_modules when installed globally. If you set the NODE_PATH environment variable to this path, the modules can be found by node.
Windows XP - %USERPROFILE%\AppData\npm\node_modules
Windows 7, 8 and 10 - %USERPROFILE%\AppData\Roaming\npm\node_modules
Non-global libraries
Non-global libraries are installed the node_modules sub folder in the folder you are currently in.
You can run npm list to see the installed non-global libraries for your current location.
When installing use -g option to install globally
npm install -g pm2 - pm2 will be installed globally. It will then typically be found in /usr/local/lib/node_modules (Use npm root -g to check where.)
npm install pm2 - pm2 will be installed locally. It will then typically be found in the local directory in /node_modules
The command npm root will tell you the effective installation directory of your npm packages.
If your current working directory is a node package or a sub-directory of a node package, npm root will tell you the local installation directory. npm root -g will show the global installation root regardless of current working directory.
Example:
$ npm root -g
/usr/local/lib/node_modules
See the documentation.
For globally-installed modules:
The other answers give you platform-specific responses, but a generic one is this:
When you install global module with npm install -g something, npm looks up a config variable prefix to know where to install the module.
You can get that value by running npm config get prefix
To display all the global modules available in that folder use npm ls -g --depth 0 (depth 0 to not display their dependencies).
If you want to change the global modules path, use npm config edit and put prefix = /my/npm/global/modules/prefix in the file or use npm config set prefix /my/npm/global/modules/prefix.
When you use some tools like nodist, they change the platform-default installation path of global npm modules.
On windows I used npm list -g to find it out. By default my (global) packages were being installed to C:\Users\[Username]\AppData\Roaming\npm.
If you are looking for the executable that npm installed, maybe because you would like to put it in your PATH, you can simply do
npm bin
or
npm bin -g
If a module was installed with the global (-g) flag, you can get the parent location by running:
npm get prefix
or
npm ls -g --depth=0
which will print the location along with the list of installed modules.
Not direct answer but may help ....
The npm also has a cache folder, which can be found by running npm config get cache (%AppData%/npm-cache on Windows).
The npm modules are first downloaded here and then copied to npm global folder (%AppData%/Roaming/npm on Windows) or project specific folder (your-project/node_modules).
So if you want to track npm packages, and some how, the list of all downloaded npm packages (if the npm cache is not cleaned) have a look at this folder. The folder structure is as {cache}/{name}/{version}
This may help also https://docs.npmjs.com/cli/cache
In earlier versions of NPM modules were always placed in /usr/local/lib/node or wherever you specified the npm root within the .npmrc file. However, in NPM 1.0+ modules are installed in two places. You can have modules installed local to your application in /.node_modules or you can have them installed globally which will use the above.
More information can be found at https://github.com/isaacs/npm/blob/master/doc/install.md
To get a compact list without dependencies simply use
npm list -g --depth 0
The easiest way would be to do
npm list -g
to list the package and view their installed location.
I had installed npm via chololatey, so the location is
C:\MyProgramData\chocolatey\lib\nodejs.commandline.0.10.31\tools\node_modules
C:\MyProgramData\ is chocolatey repo location.
As the other answers say, the best way is to do
npm list -g
However, if you have a large number of npm packages installed, the output of this command could be very long and a big pain to scroll up (sometimes it's not even possible to scroll that far back).
In this case, pipe the output to the more program, like this
npm list -g | more
I was beginning to go mad while searching for the real configuration, so here is the list of all configuration files on linux:
/etc/npmrc
/home/youruser/.npmrc
/root/.npmrc
./.npmrc in the current directory next to package.json file (thanks to #CyrillePontvieux)
on windows:
c/Program\ Files/nodejs/node_modules/npm/npmrc
Then in this file the prefix is configured:
prefix=/usr
The prefix is defaulted to /usr in linux, to ${APPDATA}\npm in windows
The node modules are under $prefix tree, and the path should contain $prefix/bin
There may be a problem :
When you install globally, you use "sudo su" then the /root/.npmrc may be used!
When you use locally without sudo: for your user its the /home/youruser/.npmrc.
When your path doesn't represent your prefix
When you use npm set -g prefix /usr it sets the /etc/npmrc global, but doesn't override the local
Here is all the informations that were missing to find what is configured where. Hope I have been exhaustive.
You can find globally installed modules by the command
npm list -g
It will provide you the location where node.js modules have been installed.
C:\Users\[Username]\AppData\Roaming\npm
If you install node.js modules locally in a folder, you can type the following command to see the location.
npm list
Echo the config: npm config ls or npm config list
Show all the config settings: npm config ls -l or npm config ls --json
Print the effective node_modules folder: npm root or npm root -g
Print the local prefix: npm prefix or npm prefix -g
(This is the closest parent directory to contain a package.json file or node_modules directory)
npm-config | npm Documentation
npm-root | npm Documentation
npm-prefix | npm Documentation
Expanding upon other answers.
npm list -g
will show you the location of globally installed packages.
If you want to output that list to a file that you can then easily search in your text editor:
npm list -g > ~/Desktop/npmfiles.txt
From the docs:
In npm 1.0, there are two ways to install things:
globally —- This drops modules in {prefix}/lib/node_modules, and puts executable files in {prefix}/bin, where {prefix} is usually
something like /usr/local. It also installs man pages in
{prefix}/share/man, if they’re supplied.
locally —- This installs your package in the current working directory. Node modules go in ./node_modules, executables go in
./node_modules/.bin/, and man pages aren’t installed at all.
You can get your {prefix} with npm config get prefix. (Useful when you installed node with nvm).
From the docs:
Packages are dropped into the node_modules folder under the prefix.
When installing locally, this means that you can
require("packagename") to load its main module, or
require("packagename/lib/path/to/sub/module") to load other modules.
Global installs on Unix systems go to {prefix}/lib/node_modules.
Global installs on Windows go to {prefix}/node_modules (that is, no
lib folder.)
Scoped packages are installed the same way, except they are grouped
together in a sub-folder of the relevant node_modules folder with the
name of that scope prefix by the # symbol, e.g. npm install
#myorg/package would place the package in
{prefix}/node_modules/#myorg/package. See scope for more details.
If you wish to require() a package, then install it locally.
You can get your {prefix} with npm config get prefix. (Useful when you installed node with nvm).
Read about locally.
Read about globally.
Windows 10: When I ran npm prefix -g, I noticed that the install location was inside of the git shell's path that I used to install. Even when that location was added to the path, the command from the globally installed package would not be recognized. Fixed by:
running npm config edit
changing the prefix to 'C:\Users\username\AppData\Roaming\npm'
adding that path to the system path variable
reinstalling the package with -g.
In Ubuntu 14.04 they are installed at
/usr/lib/node_modules
For Windows 7, 8 and 10 -
%USERPROFILE%\AppData\Roaming\npm\node_modules
Note:
If you are somewhere in folder type cd .. until you are in C: directory. Then, type cd %USERPROFILE%\AppData\Roaming\npm\node_modules. Then, magically, %USERPROFILE% will change into Users\YourUserProfile\.
I just wanted to clarify on ideas referred by Decko in the first response. npm list -g will list all the bits you have globally installed. If you need to find your project related npm package then cd 'your angular project xyz', then run npm list. It will show list of modules in npm package. It will also give you list of dependencies missing, and you may require to effectively run that project.
Btw, npm will look for node_modules in parent folders (up to very root) if can not find in local.
If you're trying to access your global dir from code, you can backtrack from process.execPath. For example, to find wsproxy, which is in {NODE_GLOBAL_DIR}/bin/wsproxy, you can just:
path.join(path.dirname(process.execPath), 'wsproxy')
There's also how the npm cli works # ec9fcc1/lib/npm.js#L254 with:
path.resolve(process.execPath, '..', '..')
See also ec9fcc1/lib/install.js#L521:
var globalPackage = path.resolve(npm.globalPrefix,
'lib', 'node_modules', moduleName(pkg))
Where globalPrefix has a default set in ec9fcc1/lib/config/defaults.js#L92-L105 of:
if (process.env.PREFIX) {
globalPrefix = process.env.PREFIX
} else if (process.platform === 'win32') {
// c:\node\node.exe --> prefix=c:\node\
globalPrefix = path.dirname(process.execPath)
} else {
// /usr/local/bin/node --> prefix=/usr/local
globalPrefix = path.dirname(path.dirname(process.execPath))
// destdir only is respected on Unix
if (process.env.DESTDIR) {
globalPrefix = path.join(process.env.DESTDIR, globalPrefix)
}
}
If you have Visual Studio installed, you will find it comes with its own copy of node separate from the one that is on the path when you installed node yourself - Mine is in C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VisualStudio\NodeJs.
If you run the npm command from inside this directory you will find out which node modules are installed inside visual studio.