When installing anything via npm, it downloads dozens of not needed files. Usually I am looking for a library final build, a *.min.js file or anything like that but the rest is useless.
How do you handle all these useless files? Do you remove them by hand or generate the final app with any build tool like gulp or grunt?
I'm quite confused as I have plenty of npm modules installed in my webapp and the folder size is about 50 megabytes but it could be 2mb only.
npm install --production
Just doing an npm install brings in both development and runtime dependencies. You could also set the ENV to production globally for the server: npm config set production.
See this github issue. Note that this won't get you only the final minified build of everything, but will greatly reduce the bloat. For instance, a library might rely on babel-cli, babel-preset-es2015, and uglifyjs to be built (devDependency), but you don't need any of that if it also includes the transpiled minified file.
Managing Packages
For front end non-development packages I prefer Bower. It maintains the minified and non-minified version of your packages.
Build Tool
Use either Gulp or Grunt. Gulp would be my tool of choice.
Gulp task that will greatly improve your code are:
minification of both css and js
optimization/compression of images
concatenation and caching to reduce the number of calls to the server
package versioning
automatic injection of project dependencies
automatic injection of external dependencies
static analysis of js and css
automatic builds on code changes
deployment
testing
Node
If you can, leave to node all your development tools and leave to bower all your release plugins. Most node packages that are used in released apps have a bower installation counterpart.
Edit
Don't delete anything from Node manually as you don't know which packages have other packages as dependencies. If you are afraid that you may have junk in there, use npm rimraf to delete the node_modules folder, and then run npm install. Most importantly check your package.json for unnecessary saved packages.
Related
I have an NPM package that can be used with the browser. But in order to use it in the browser, I pre-package it using Webpack and put the browserified code in the /dist directory.
Normally, I don't feel it's necessary to include the /dist directory when publishing to NPM, unless someone wants to use the browser version instead of the Node.js version (most of my customers will be using my lib for Node.js not for front-end).
The dist is a huge directory (all the project's code, plus NPM deps) and I want to save people the disk space and install time.
Should I just create a separate package for the browser code, or is there some flag I can use for conditionally including the dist directory when people install my package?
I believe it's better to create two separate packages.
I am learning to use webpack and generally getting in to the Javascript world, including npm.
Several answers deal with --save vs --save-dev when using npm install. My understanding is that their use (and updates to package.json) is actually useful when recreating either a run or a dev environment via npm install <the package being developed or ran>
--save is used to save packages needed to run the app in node.js, that is on a server
--save-dev is used to save packages needed to develop the app
a bare npm install <module> just installs the package, without enabling the possibility to install it somewhere else though the appropriate entry in package.json
Therefore, in a webpack context, is --save ever used? I belive not, because what is created is a JS bundle which is then included in the HTML file, and ran in a browser. In that sense, there is never a need to "save modules needed to run your app".
In the same vein, --save-dev is useful (again, in a webpack context) at it allows someone to develop elsewhere (in that case both modules in the app (say, moment.js) and logistical ones (say, gulp) should be installed with --save-dev, right?)
Finally, a bare npm install <module> is also possible (although less useful) is the development is not intended to be done elsewhere (the modules are still installed but no mention of this fact is made in package.json).
Is this correct? Specifically, is the assumption of a lack of --save in a webpack context true?
Anything that it utilised in the production build of your application should be listed in save. For example if you use React, your application utilises React in the final production build. It doesn't matter that your file is bundled but the fact that it's heavily relied upon running when compiled.
Anything that is used during development should be listed under devDependency. In this example, once WebPack has finished bundling your files, we no longer care about WebPack because it's not apart of the final compiled file.
--save-dev : Anything that is used during development such as Unit testing frameworks or bundlers etc.
--save : Anything that is used within your application such as Axios, React, Vue or Chart.JS etc.
Is it possible to exclude files from a list of NPM packages in my package.json?
I have a non-browser environment that works a bit differently: every file in node_modules dir becomes part of the production package. So there's no smart treeshaking that imports only the files that I use in my code.
For instance, I use some packages which also carry a lot of tests and i18n files, most of which I don't need and like to remove from my packaged production version. However, they are still included in the end package because the whole package folder is included in the build.
I'm trying to remove as many files from the packages as I can (without doing it manually each time) to save space and compilation time. The environment I use is looping all files in the node_modules directory and adds them to the production package (all packaged using Javascript). I would like a JavaScript solution to remove these files on compilation so the end package is as small as it can be.
I would use bower to manage client-side javascript modules, instead of using npm directly.
Bower packages are simpler than npm equivalents and don't have subfolders with module dependencies. Most will include a "dist" folder with pre-minified javascript. Right out of the box your packages will be smaller than if you use npm.
If you want to go further, you can include some processing in your gulpjs or gruntjs scripts to either manually copy needed files to lib and css folder(s), or use a plugin like gulp-bower-normalize to (somewhat) automatically do the same thing.
I would like to deploy a nodejs project with frequent updates. npm is not available at the site so I must package the node_modules. This works ok but takes a long time to send to the customer over the available ftp connection (80MB of mostly node_module files every time). My workflow looks like this:
git clone project
npm install # installs all my dev tools which I need for packaging
grunt build
tar xvzf build.tar.gz build/
The build step minfifies my code packaging only what is needed. The node_modules folder is copied into the build folder. If I use npm install --production, I get a smaller footprint but miss the tools I need to build it in the first place. So in the end I go to some effort to make my code footprint small but all my work is undone by having to package such a large node_modules tree.
Is my approach wrong? Is there a simpler way to deploy where npm is not available on the production server or is there a good way to reduce the size of the node_modules folder?
Update: Since writing this answer, npm3 (and yarn) arrived, and
flattened npm dependencies. This reduces the size of the
node_modules folder considerably (perhaps 20% - 30% for a
typical project). Nevertheless, some of the tips below that will
reduce your footprint by an order of magnitude.
I have compiled the list of findings for anyone wanting to
deploy without npm on the server or
reduce the footprint of the node_modules folder
Smaller node_modules footprint:
Use npm prune --production to remove devDependencies and purge additional modules
In my case this node_modules folder size by about 20%.
The bulk of large files under node_modules folders is confined to a small number of modules that are unused at runtime!. Purging/deleting these reduces the footprint by a factor of 10! e.g: karma, bower, less and grunt. Many of these are used by the modules themselves and have no place in a
production build. The drawback is that npm install has to be run before each build.
Use partial npm packages
Many npm packages are available in parts. For example, of
installing all of async or lodash install only the bits you
need: e.g.
Bad: npm install -save lodash async
Good: npm install --save async.waterfall async.parallel lodash.foreach
Typically, individual lodash modules are 1/100th the size of the full package.
npm-package-minifier may be used to reduce the size of the node_modules tree
Compacting node_modules for client-side deployment
This basically deletes a lot of unused files in the
node_modules tree. This tool will reduce the size of
devDependencies also so it should be run on a 'production'
version of node_modules.
Reducing size of updates
Differential deployment
As mentioned in the comments, updates may be split into updates where dependency changes are required or only business logic changes. I have tried this approach and it greatly reduces the footprint for most updates. However, it also increases the complexity of deployment.
I just installed Node with npm to use it for frontend dependency management. I know there is also bower but why would I need another package manager that is built on top of another?
When installing a package, npm seems to always load the full source of the js library into the node_modules directory. Just as it's downloading the complete github repository.
How do I install only the minifed (distribution) version of a javascript lib with npm?
There is no standard way to ask npm to get the minified version of a library. Some developers will produce packages that contain both minified and unminified versions (that's what I've done for one of my projects, which is web-only but can be installed through npm) or will create a package that contains an unminified version and another package that contains the minified version. This is done on a case by case basis, varies from package to package, and has to be determined by looking at a project's documentation.
If a developer has not cared to provide a minified code base through npm then you'll have to perform the minification yourself or get the "official" minified code through some other means.
You can use packages like node-prune and ModClean to clean up the unwanted dependencies.
Read this post for more details.