I am coding a lot of individual plugins for my websites and I am using grunt to manage the final distribution of them.
I'm used to write grunt.loadNpmTasks to import a specific plugin from the node_modules directory right into my Gruntfile performing the tasks.
However, in order to do that, i always npm install <package> --save-dev to make the plugin available in the specific plugin i am coding. But as they are dozens of plugins i am maintaining now, i found out the node_modules directories are growing quite bigger, and my backup gets more and more slowly as the node_modules directories are full of files.
Is there a way to centralize the plugins ? so I can reunite all the node_modules directories in one ? and tell grunt where this central repository stands as to load a particular plugin ?
edit: I tried to install grunt-contrib-less globally (-g) for the test, but it still persist to say Local Npm module "grunt-contrib-less" not found. Is it installed?
I haven't tested this. but it's worth trying. Instead of using grunt.loadNpmTasks try using load-grunt-tasks.
You should use the config option which allows you to specify the path of your pacakge.json.
require('load-grunt-tasks')(grunt, {config: '../package'});
Related
I installed the same package for both my projects. That package(won't link, it's private one) has react-popper as dependency(which in order has create-react-context as dependency), so, when I run project one - everything is ok, but error appears for project two:
ERROR in ./node_modules/react-popper/lib/esm/Manager.js
Module not found: Error: Can't resolve 'create-react-context' in '/../node_modules/react-popper/lib/esm'
After some investigation, I got that node_modules structure is different:
for project one, all react-popper dependencies are saved in project one
node modules, and local folder contain only warning package:
for project two all react-popper dependencies saved in ../react-popper/node_modules local folder:
I've tried some common approaches like reinstalling node modules, clear cache and so on, but the structure is the same. Actually I had a thoughts about webpack and babel versions, but I don't think it can affect node_modules structure itself.
So the question is, which factors can affect it? What should I check?
NOTE: If I manually add create-react-context to project two, it works fine, but it's not a solution.
NOTE: I found out similar issue, but no suggestions there - Why does npm install packages in different directories?, im my case re-creating of yarn.lock also helps, but it's also doesn't look like a right way to solve it. Hope my description is more complete and will help to figure it out.
This is very likely because of the way yarn (as well as npm) tries to deduplicate dependencies. Let's say there are modules A and B which exist in 2 versions (1.0.0 and 2.0.0). B depends on Version 1.0.0 of Module A.
If you install only module B, you will get a node_modules folder like this:
node_modules
- A#1.0.0
- B#2.0.0
But what if you install module A in its latest version (2.0.0)? If npm just updated the version of module A, your existing module B would (potentially) no longer work as it depends on module A. So your node_modules folder will instead look like this (A#1.0.0 is moved inside B's node_modules folder)
node_modules
- A#2.0.0
- B#2.0.0
-- A#1.0.0
Your 2 projects likely have further dependencies, which somehow overlap with react-popper or its dependencies. Due to the nodeJs module resolution mechanism, this usually shouldn't be a problem.
TLDR: The exact structure of the node_modules folder depends on all your dependencies (and devDependencies). yarn/npm will look into every package.json/package-lock.json file of your projects dependencies (and their dependencies) and use this information to calculate a dependency tree with minimal duplication.
I have an NPM package that can be used with the browser. But in order to use it in the browser, I pre-package it using Webpack and put the browserified code in the /dist directory.
Normally, I don't feel it's necessary to include the /dist directory when publishing to NPM, unless someone wants to use the browser version instead of the Node.js version (most of my customers will be using my lib for Node.js not for front-end).
The dist is a huge directory (all the project's code, plus NPM deps) and I want to save people the disk space and install time.
Should I just create a separate package for the browser code, or is there some flag I can use for conditionally including the dist directory when people install my package?
I believe it's better to create two separate packages.
Is it possible to exclude files from a list of NPM packages in my package.json?
I have a non-browser environment that works a bit differently: every file in node_modules dir becomes part of the production package. So there's no smart treeshaking that imports only the files that I use in my code.
For instance, I use some packages which also carry a lot of tests and i18n files, most of which I don't need and like to remove from my packaged production version. However, they are still included in the end package because the whole package folder is included in the build.
I'm trying to remove as many files from the packages as I can (without doing it manually each time) to save space and compilation time. The environment I use is looping all files in the node_modules directory and adds them to the production package (all packaged using Javascript). I would like a JavaScript solution to remove these files on compilation so the end package is as small as it can be.
I would use bower to manage client-side javascript modules, instead of using npm directly.
Bower packages are simpler than npm equivalents and don't have subfolders with module dependencies. Most will include a "dist" folder with pre-minified javascript. Right out of the box your packages will be smaller than if you use npm.
If you want to go further, you can include some processing in your gulpjs or gruntjs scripts to either manually copy needed files to lib and css folder(s), or use a plugin like gulp-bower-normalize to (somewhat) automatically do the same thing.
I'm completely new to frontend web dev with a very basic question. Once I npm install something, how do I actually use it? For example, I just did npm install bootstrap, and I would now like to be able to use the CSS and Javascript that it downloaded. I'm sure I shouldn't have to dig through the directories to find some entry point... so how do I now use bootstrap in my webpage?
Most modules on NPM are used in Node.js, for the server (backend). Node.js has a built-in function require('your-module') to make use of the module. This function is not present on the frontend in the browser. However, there are tools like browserify or webpack and probably others to make the NPM modules and the require function work in the frontend.
If you're just starting out I suggest you take a look at Bower first. With Bower (installed with NPM though) you can pull in all your frontend libraries like jQuery, Bootstrap, etc. to your project folder and you can point your script tags in your HTML to the bower_components/ directory, e.g. <script src="/bower_components/jquery/jquery.min.js"></script>. You can save a list of all libraries used with a version number in a json file called bower.json in the root of your project folder.
Based on this file you can pull in or update all the libraries listed with the use of the command line.
As a really general rule, npm is used for assets your node app will use on the server, while bower (and others) are the equivalent for dependencies that you want to use on the client.
That said, the use is basically the same.
npm (and bower) install the files into your project directory in a standard location. All you really have to do is make sure that location is accessible via a web request (typically, node_modules is not; which is why bower came about), and then embed link and script tags as appropriate in your html:
<script src='/node_modules/bootstrap/js/bootstrap.min.js'></script>
I use Node.js (via browserify) for each of my web apps, all of which have some dependencies in common and others specific to themselves. Each of these apps has a package.json file that specifies which versions of which modules it needs.
Right now, I have a /node_modules directory in the parent folder of my apps for modules that they all need to reference, and then I put app-specific modules in a node_modules folder in that app's directory. This works fine in the short term, since my require() statements are able to keep looking upward in the file structure until they find the node_modules directory with the correct app in it.
Where this gets tricky is when I want to go back to an old project and run npm install to make sure it can still find all the dependencies it needs. (Who knows what funny-business has occurred since then at the parent directory level.) I was under the impression that npm install did this:
for each module listed in package.json, first check if it's present, moving up the directory the same way require does. If it's not, install it to the local node_modules directory (creating that directory if necessary).
When I run npm install inside an app folder, however, it appears to install everything locally regardless of where else it may exist upstream. Is that the correct behavior? (It's possible there's another reason, like bad version language in my package.json). If this IS the correct behavior, is there a way for me to have npm install behave like the above?
It's not a big deal to widely replicate the modules inside every app, but it feels messy and prevents me from make small improvements to the common modules and not having to update every old package.json file. Of course, this could be a good thing...
When I run npm install inside an app folder, however, it appears to install everything locally regardless of where else it may exist upstream. Is that the correct behavior? (It's possible there's another reason, like bad version language in my package.json). If this IS the correct behavior, is there a way for me to have npm install behave like the above?
Yes, that is what npm install does. In node.js code, the require algorithm has a particular sequence of places it looks, including walking up the filesystem. However, npm install doesn't do that. It just installs in place. The algorithms it uses are all constrained to just a single node_modules directory under your current directory and it won't touch anything above that (except for with -g).
It's not a big deal to widely replicate the modules inside every app, but it feels messy and prevents me from make small improvements to the common modules and not having to update every old package.json file. Of course, this could be a good thing...
Yeah basically you're doing it wrong. The regular workflow scales well to the Internet. For your use case it creates some extra tedious work, but you can also just use semantic versioning as intended and specify "mylib": "^1.0.0" in your package.json for your apps and be OK with automatically getting newer versions next time you npm install.