Why the same package are installed in a different way? - javascript

I installed the same package for both my projects. That package(won't link, it's private one) has react-popper as dependency(which in order has create-react-context as dependency), so, when I run project one - everything is ok, but error appears for project two:
ERROR in ./node_modules/react-popper/lib/esm/Manager.js
Module not found: Error: Can't resolve 'create-react-context' in '/../node_modules/react-popper/lib/esm'
After some investigation, I got that node_modules structure is different:
for project one, all react-popper dependencies are saved in project one
node modules, and local folder contain only warning package:
for project two all react-popper dependencies saved in ../react-popper/node_modules local folder:
I've tried some common approaches like reinstalling node modules, clear cache and so on, but the structure is the same. Actually I had a thoughts about webpack and babel versions, but I don't think it can affect node_modules structure itself.
So the question is, which factors can affect it? What should I check?
NOTE: If I manually add create-react-context to project two, it works fine, but it's not a solution.
NOTE: I found out similar issue, but no suggestions there - Why does npm install packages in different directories?, im my case re-creating of yarn.lock also helps, but it's also doesn't look like a right way to solve it. Hope my description is more complete and will help to figure it out.

This is very likely because of the way yarn (as well as npm) tries to deduplicate dependencies. Let's say there are modules A and B which exist in 2 versions (1.0.0 and 2.0.0). B depends on Version 1.0.0 of Module A.
If you install only module B, you will get a node_modules folder like this:
node_modules
- A#1.0.0
- B#2.0.0
But what if you install module A in its latest version (2.0.0)? If npm just updated the version of module A, your existing module B would (potentially) no longer work as it depends on module A. So your node_modules folder will instead look like this (A#1.0.0 is moved inside B's node_modules folder)
node_modules
- A#2.0.0
- B#2.0.0
-- A#1.0.0
Your 2 projects likely have further dependencies, which somehow overlap with react-popper or its dependencies. Due to the nodeJs module resolution mechanism, this usually shouldn't be a problem.
TLDR: The exact structure of the node_modules folder depends on all your dependencies (and devDependencies). yarn/npm will look into every package.json/package-lock.json file of your projects dependencies (and their dependencies) and use this information to calculate a dependency tree with minimal duplication.

Related

How do I replace Bower with Yarn --flat with nested dependencies?

Bower is no longer maintained and the recommendation is to move to npm.
However, npm is designed for Node and has a nested dependency model that's completely unsuitable for the web.
So (for instance) given component-X and component-Y that both need library-A Bower would download something like:
/bower-components
/library-A
/component-X
/component-Y
While npm produces:
/node_modules
/component-X
/node_modules
/library-A
/component-Y
/node_modules
/library-A
As this is a web application multiple copies of library-A are a significant problem - we need to load it exactly once.
Potentially Yarn solves this with yarn install --flat as that flattens the structure and has a resolution mechanism for different versions.
However, the nested references inside the components don't point to the new location. For instance, inside component-X there will be a line like:
import {moduleB} from 'library-A/module-B.js';
This now fails, it needs to point to the new flattened location:
import {moduleB} from '../library-A/module-B.js';
It seems that without this change yarn install --flat is pretty useless, as nested dependencies will always be lost.
How do I make Yarn update these resolutions as it flattens?
Is there another way? I realise a custom Gulp/Grunt/Webpack/Rollup task can do it, but that seems like a workaround for something supposedly obsolete Bower did out of the box.

Is there a way to tell grunt where the node_modules is?

I am coding a lot of individual plugins for my websites and I am using grunt to manage the final distribution of them.
I'm used to write grunt.loadNpmTasks to import a specific plugin from the node_modules directory right into my Gruntfile performing the tasks.
However, in order to do that, i always npm install <package> --save-dev to make the plugin available in the specific plugin i am coding. But as they are dozens of plugins i am maintaining now, i found out the node_modules directories are growing quite bigger, and my backup gets more and more slowly as the node_modules directories are full of files.
Is there a way to centralize the plugins ? so I can reunite all the node_modules directories in one ? and tell grunt where this central repository stands as to load a particular plugin ?
edit: I tried to install grunt-contrib-less globally (-g) for the test, but it still persist to say Local Npm module "grunt-contrib-less" not found. Is it installed?
I haven't tested this. but it's worth trying. Instead of using grunt.loadNpmTasks try using load-grunt-tasks.
You should use the config option which allows you to specify the path of your pacakge.json.
require('load-grunt-tasks')(grunt, {config: '../package'});

Change Directory in Node REPL not working?

Just earlier, I posted my question:
https://stackoverflow.com/questions/28336443/how-to-not-put-my-js-files-in-user-myuser-for-node-js
I have a file, hello.js, located in /Users/MyUser/Desktop/Node/
I can see that my default directory is /Users/MyUser/
Okay, so I get that I need to change my working directory. What I have been able to find so far is to use >process.chrdir('/Users/MyUser/Desktop/Node/');
Cool, that works, but now when I get out of the REPL shell, the directory resets.
The person who responded to my question said that I needed to run >node init and later npm install <name of dependency> --save
My first question: I have ran >node init and see that I can create this package.json file, what does this do exactly?
Secondly: I was told that I need to add dependancies. Could someone please explain to me what this means in Node terms? Does a dependancy simply mean a folder that I want node to include? Do I want to add this Node folder on my Desktop to be able to run my scripts?
I am currently trying to go through the learnyounode courses, however I do not want to have to save all of these test files in my /User/MyUser directory, so any advice would be greatly appreciated.
Thanks
I have ran >node init and see that I can create
this package.json file, what does this do exactly?
npm init is used to create a package.json file interactively. This will ask you a bunch of questions, and then write a package.json for you.
package.json is just a file that handle the project's dependencies and holds various metadata relevant to the project[ project description, version, license information etc]
I was told that I need to add dependencies. Could someone please
explain to me what this means in Node terms?
Lets say you're building an application that is dependent on a number of NPM modules, you can specify them in your package.json file this way:
"dependencies": {
"express": "2.3.12",
"jade": ">= 0.0.1",
"redis": "0.6.0"
}
Now doing npm install would install a package, and any packages that it depends on.
A package is:
a folder containing a program described by a package.json file
a gzipped tarball containing (1)
a url that resolves to (2)
a # that is published on the registry with (3)
a # that points to (4)
a that has a "latest" tag satisfying (5)
a that resolves to (2)
If you need to install a dependency that haven't been included in package.json, simply do npm install <packageName>. Whether or not you want to include this newly installed package in package.json is completely your take. You can also decide how this newly installed package shall appear in your package.json
npm install <packageName> [--save|--save-dev|--save-optional]:
--save: Package will appear in your dependencies.
--save-dev: Package will appear in your devDependencies.
--save-optional: Package will appear in your optionalDependencies.
Does a dependency simply mean a folder that I want node to include?
Umm, partly yes. You may consider dependencies as folders, typically stored in node_modules directory.
Do I want to add this Node folder on my Desktop to be able to run my
scripts?
No, node manages it all. npm install will automatically create node_modules directory and you can refer to those dependencies with
require() in your .js files
var express = require('express');
Node REPL simply provides a way to interactively run JavaScript and see the results. It can be used for debugging, testing, or just trying things out.
process.cwd() points to the directory from which REPL itself has been initiated. You may change it using process.chdir('/path'), but once you close the REPL session and restart, it would always re-instantiate process.cwd() to the directory from which it has been started.
If you are installing some packages/dependencies in node project1 and think those dependencies can also be useful for node project2,
install them again for project2 (to get independentnode_modules directory)
install them globally [using -g flag]. see this
reference packages in project2 as
var referencedDependency = require('/home/User/project1/node_modules/<dependency>')
Simply doing process.chdir('/home/User/project1/node_modules/') in REPL and referencing as
var referencedDependency = require('<dependency>') in your js file wont work.
>process.chdir('/Users/MyUser/Desktop/Node/'); change the working directory only for that particular REPL session.
Hope it helps!
This has nothing to do with node.js but is rather inherent in the design of Unix (which in turn influences the design of shells on other operating systems).
Processes inherit values from their parent's environment but their environments are distinct.
That terse description of how process environments work has a sometimes unexpected behavior: you cannot change your parent's environment. It was designed this way explicitly for security reasons.
What this means is, when you change the working directory in a process and quits that process your shell's working directory will not be affected. Indeed, your shell's working directory isn't affected even when the process (in this case, node REPL) is running.
This exact question is often asked by people writing shell scripts wanting to write a script that CDs into someplace. But it's also common to find this question asked by people writing other languages such as Perl, Tcl, Ruby etc. (even C).
The answer to this question is always the same regardless of language: it's not possible to CD from another program/script/process.
I'm not sure how Windows handles it so it may be possible to do it there. But it's not possible on Unixen.

How npm install works

I use Node.js (via browserify) for each of my web apps, all of which have some dependencies in common and others specific to themselves. Each of these apps has a package.json file that specifies which versions of which modules it needs.
Right now, I have a /node_modules directory in the parent folder of my apps for modules that they all need to reference, and then I put app-specific modules in a node_modules folder in that app's directory. This works fine in the short term, since my require() statements are able to keep looking upward in the file structure until they find the node_modules directory with the correct app in it.
Where this gets tricky is when I want to go back to an old project and run npm install to make sure it can still find all the dependencies it needs. (Who knows what funny-business has occurred since then at the parent directory level.) I was under the impression that npm install did this:
for each module listed in package.json, first check if it's present, moving up the directory the same way require does. If it's not, install it to the local node_modules directory (creating that directory if necessary).
When I run npm install inside an app folder, however, it appears to install everything locally regardless of where else it may exist upstream. Is that the correct behavior? (It's possible there's another reason, like bad version language in my package.json). If this IS the correct behavior, is there a way for me to have npm install behave like the above?
It's not a big deal to widely replicate the modules inside every app, but it feels messy and prevents me from make small improvements to the common modules and not having to update every old package.json file. Of course, this could be a good thing...
When I run npm install inside an app folder, however, it appears to install everything locally regardless of where else it may exist upstream. Is that the correct behavior? (It's possible there's another reason, like bad version language in my package.json). If this IS the correct behavior, is there a way for me to have npm install behave like the above?
Yes, that is what npm install does. In node.js code, the require algorithm has a particular sequence of places it looks, including walking up the filesystem. However, npm install doesn't do that. It just installs in place. The algorithms it uses are all constrained to just a single node_modules directory under your current directory and it won't touch anything above that (except for with -g).
It's not a big deal to widely replicate the modules inside every app, but it feels messy and prevents me from make small improvements to the common modules and not having to update every old package.json file. Of course, this could be a good thing...
Yeah basically you're doing it wrong. The regular workflow scales well to the Internet. For your use case it creates some extra tedious work, but you can also just use semantic versioning as intended and specify "mylib": "^1.0.0" in your package.json for your apps and be OK with automatically getting newer versions next time you npm install.

Running grunt on dependencies

I have 3 projects all using grunt.
Project a depends on project c and b
Project b depends on project c
Project c depends on nothing
Project a and b both require a step that compiles project c (which is a style repo that contains global styles for our org).
I am attempting to run grunt post install for project b.
There are a couple problems here.
Project a and b both try to build project c in their build processes. This takes sometime and I'd rather avoid it.
Project b expects different paths when running alone. loadNpmTasks fails unless I grunt.file.setBase, but then I have other paths that are broken as well. This means I need to manually track all of those down and make sure they are correct in both situations. This is rather flimsy.
As a result, I am thinking I am not doing this correctly or in a "normal way". What is the appropriate way to manage dependencies that use grunt?
Update
The main problem is that I get the following errors no matter what solution I've been choosing:
Local Npm module "grunt-contrib-compass" not found. Is it installed?
Local Npm module "grunt-contrib-handlebars" not found. Is it installed?
Local Npm module "grunt-contrib-requirejs" not found. Is it installed?
Local Npm module "grunt-contrib-watch" not found. Is it installed?
Local Npm module "grunt-notify" not found. Is it installed?
Local Npm module "grunt-curl" not found. Is it installed?
Local Npm module "grunt-shell" not found. Is it installed?
I tried spawn a task and calling grunt via the command line. I tried using various plugins to help. I think the issue is that both the top level and the dependencies require those tasks. This means they get pulled up into the parent node_modules folder. As a result the dependencies are missing the above modules in their node_modules folder.
I've built grunt-grunt with exactly this kind of scenario in mind.
grunt.initConfig({
grunt: {
lintsome: {
gruntfile: 'node_modules/some/Gruntfile.js',
task: 'jshint'
}
}
});
The README contains enough documentation to get you going.

Categories

Resources