I have problem with npm install. I want to install all dependencies used in my project. In package.json there aren't any dependencies, but in my project files I have for example const mongo=require('mongoose') and in another file const morgan=require('morgan') etc.
I know that when typing npm i --save <dependency_name> it will update my package.json. But I would like to install all dependencies without typing their names explicitly anywhere.
Is there any way to install all dependencies used in the whole project, but which there aren't in package.json?
It is not intended to write your program code first and then let NPM find all dependencies that are used. This is because of a simple reason: Before you can use an external library / package, you'll have to download it, otherwise you could hardly use it at all.
Now, I only see two reasons for your use case:
you copied and pasted some foreign code where you don't have all files, especially no package.json, or
you "inherited" some code of a former employee or team that is not documented and consists of unrelated, inconsistent files.
Anyways, there would be some kinds of solutions for your problem:
The nasty solution: Run npm start and see what error messages you get. All uninstalled dependencies will not be found, so you'll see the package names and can add them manually to your package.json. This will become nasty because you'll have to re-run your program each time. To avoid this, you could have a look at Nodemon which automatically re-starts your program.
The better solution: Open a web IDE of your favor and use the "Global Search" function to find all occurrences of the string require(, or as a Regex: require\((.+)\). This will list you all dependency imports in your program files. If there should be also ECMA 6 imports, also search for import (.+) from (.+). - However, you'll still have to copy and paste all dependency names from all files manually into your package.json file.
The best but most complex solution: Write a Node.js script that scans all your files recursively, beginning in your root project directory. Create a dependency memory variable, like let dependencies = []. Read all *.js files (sync or async) and every time the require or import statement is matched, check whether the dependency is already in your dependencies array. If not, push it. Finally, all your project dependencies will be listed in the dependencies array and you can copy and paste them into your package.json.
Pseudo Node.js Code:
const lineReader = require('line-reader');
let dependencies = [];
const regex = /require\(['|"](.+)['|"]\)/g;
lineReader.eachLine('/path/to/file', function(line) {
const match = regex.exec(line);
if(match) {
if(dependencies.indexOf(match[1]) === -1) {
dependencies.push(match[1]);
}
}
});
No, you can't do that - you will have to add all dependencies by installing them explicitly, like npm install morgan. NPM is not aware of the dependencies you're importing in your files. Another thing is that requiring dependencies that are not listed in package.json is simply wrong and should never take place.
Short answer: You can't
How npm install works is checking all your dependencies listed in the package.json and install them by once. So either you get the package.json from the tutor or you install them one by one
You can scan your project for all required modules.
Assuming your project only uses common.js (require) you can get a list of all modules by doing something like:
egrep -R --exclude-dir=node_modules '=\s*require\s*\(' | awk '{gsub(/^.+require\s*\(\s*./,""); gsub(/.\s*\).*$/,""); print $0}'
A more readable version of the above command is:
#! /bin/bash
egrep -R --exclude-dir=node_modules '=\s*require\s*\(' |
awk '{
gsub(/^.+require\s*\(\s*./,"");
gsub(/.\s*\).*$/,"");
print $0
}'
You can save the script above in a file and execute it as a shell script.
To automatically install the modules just pipe it to xargs:
egrep -R --exclude-dir=node_modules '=\s*require\s*\(' |
awk '{
gsub(/^.+require\s*\(\s*./,"");
gsub(/.\s*\).*$/,"");
print $0
}' |
xargs npm install
I leave supporting ES6 module as homework for the reader.
Related
I have two directory structure:
The first one, which is my utils project is like this
mock-utils:
app-mock (symlinked to app-mock)
common
utils.js
node_modules
package.json
package-lock.json
I have another one which is the app-mock which is somewhere locally,
app-mock:
tests
test.js
test.js contains the following import, and the entry point for the execution is inside the mock-utils since package.json is located there:
var mocha = require('mocha');
var shield = require(‘../../common/utils.js’)
However, executing test.js from mock-utils will give the error of
Error: Cannot find module
Originally, app-mock is inside the mock-utils but it have to be separated for git. How do I adjust it so that it would be possible to execute test.js while retaining this kind of structure?
You can find a solution here:
Installing a local module using npm?
Please note that due to recent NPM updates you might need to remove you package-lock.json file for this to work. Avoid running npm install during your development because this will recreate the package-lock.json file and npm will try to replace your symlinked package with the real one which, with your current setup, from what i understand, will throw a package not found exception.
There's an error being thrown in of my node.js projects, and the stack trace doesn't seem to point back to any of the libraries I'm using, is there a simple method of finding out what packages depend on this package in my node_modules directory?
Ideally this method doesn't involve manually checking the package.json of every module in my node_modules directory.
Use npm ls <module-name>.
You can use a bit of bash scripting to automate this
who_depends_on() {
local dependency=$1;
for file in $(ls node_modules); do
local match=$(grep $dependency "node_modules/$file/package.json");
if [[ $match ]]; then
echo "'$file' is dependant in '$dependency'";
fi
done
}
Put the function wherever you store your shell functions, and then run like so
who_depends_on "your-package-here"
The main caveat with this package is it'll pick up the package.json package you're looking for as well.
Whenever I make projects, I have to download all dependencies of node modules. Without copying the node_modules, Is there anyway to share the central node_modules in multiple projects?
like the followings, I have to run many commands every time..
npm install gulp-usemin
npm install gulp-wrap
npm install gulp-connect
npm install gulp-watch
npm install gulp-minify-css
npm install gulp-uglify
npm install gulp-concat
npm install gulp-less
npm install gulp-rename
npm install gulp-minify-html
You absolutely can share a node_modules directory amongst projects.
From node's documentation:
If the module identifier passed to require() is not a native module,
and does not begin with '/', '../', or './', then node starts at the
parent directory of the current module, and adds /node_modules, and
attempts to load the module from that location.
If it is not found there, then it moves to the parent directory, and
so on, until the root of the file system is reached.
For example, if the file at '/home/ry/projects/foo.js' called
require('bar.js'), then node would look in the following locations, in
this order:
/home/ry/projects/node_modules/bar.js /home/ry/node_modules/bar.js
/home/node_modules/bar.js /node_modules/bar.js
So just put a node_modules folder inside your projects directory and put in whatever modules you want. Just require them like normal. When node doesn't find a node_modules directory in your project folder, it will check the parent folder automatically. So make your directory structure like this:
-myProjects
--node_modules
--myproject1
---sub-project
--myproject2
So like this, even your sub-project's dependencies can draw on your main node_modules repository.
One drawback to doing it this way is you will have to build out your package.json file manually (unless someone knows a way to automate this with grunt or something). When you install your packages and add the --save arg to an npm install command it automatically appends it to the dependencies section or your package.json, which is convenient.
Try pnpm instead of npm.
pnpm uses hard links and symlinks to save one version of a module only ever once on a disk.
If you have npm installed, you can install in your terminal with:
npm install -g pnpm
To update your existing installations (and sub-directories) use:
pnpm recursive install
Or use the shorthand command (leave off -r if you need to target only one directory)
pnpm -r i
One helpful note: You may find some rare packages don't have all their dependencies defined. They might rely on the flat node_modules file directory structure of npm or yarn installs. If you run into issues of missing dependencies, use this command to hoist all the sub dependencies into a flat-file structure:
pnpm install --shamefully-hoist
It's best to avoid using the --shamefully-hoist flag as it defeats the purpose of using pnpm in the first place, so try using the command pnpm i your-missing-package first (See pnpm FAQ).
I found a trick, just take a look at the Symbolic Links (symlinks) on Windows or Linux, it is working just like shortcuts but more powerful.
Simply you need to make a Junction for your node_modules folder anywhere you want. The junction is nothing but a short cut to your original node_modules folder. Create it inside your project folder where the actual node_modules would have been created if used npm install.
To achieve this you need at least one node_modules real folder then make a Junction to it in the other projects.
On Windows, you can either use the Command Prompt, or use an application. Using the Command Prompt gives you a bit more control, using an application is easier I suggest Link Shell Extension.
Main directory should look like this
node_modules
Project 1
Project 2
Project 3
Project 4
just open the file Project 1/.angular-cli.json
change the schema
"$schema": "./node_modules/#angular/cli/lib/config/schema.json",
to
"$schema": "./../node_modules/#angular/cli/lib/config/schema.json"
and don't forget to create node_modules empty folder inside your project directory
See also npm v7.0.0's support for workspaces
RFC
https://github.com/npm/rfcs/blob/latest/implemented/0026-workspaces.md
Documentation
https://docs.npmjs.com/cli/v7/using-npm/workspaces
By looking at some articles it seems that Lerna
is a good tool for managing multiple projects inside a single directory (monorepo). It supports modules sharing without duplicating the entire packages in every folder and commands to install them in multiple projects.
Javascript monorepos
Monorepos by example
Building large scale apps in a monorepo
pnpm is also a simple and efficient tool, which doesn't duplicate those modules which are already installed for other projects.
Let's assume that having a single node_modules it should contain all the packages for all applications. thus your apps will also share most of the unique package.json entries (just the name should change)
my idea would be to have a single root and multiple src level as below
root\package.json
root\node_modules
root\\..
root\app1\src\\..
root\app2\src\\..
the only issue you might face would be having a backup of json (or tsconfig) for any app and restore them when you work on it or setup your startup scripts to serve any app
I would like to use npm modules within a Typescript program. Fortunately, I came across this which looks very easy and does indeed seem to work. I figured I would try and add another package, just to get the hang of it. So I forked it and made one relatively simple commit.
Here's all the things I did in my commit:
Added pubsub-js to my dependencies in package.json, so it will install on npm install.
Set up tsd, used it to install the TypeScript definition for pubsub-js, and set it to automatically run on npm install.
Modified index.ts to include the installed definition:
/// <reference path="./typings/pubsubjs/pubsub.d.ts" />
and to import the installed package:
import PubSub = require('pubsub-js');
Unfortunately, this doesn't work. I get this error:
$ npm install
$ npm test
> demo-typescript-node-minimal#0.0.1 test /home/dumbmatter/projects/mini/demo-typescript-node-minimal
> tsc index.ts --module commonjs && node ./index
index.ts(10,25): error TS2307: Cannot find module 'pubsub-js'.
npm ERR! Test failed. See above for more details.
(If you want to it for yourself, clone my repo, run npm install, and then npm test.)
And I want to reiterate, the original version (without my commit, straight from the original repo) does in fact work:
$ git checkout d002c0dffc9d9f65aca465b0fc6a279bcd23202d
$ npm test
> demo-typescript-node-minimal#0.0.1 test /home/dumbmatter/projects/mini/demo-typescript-node-minimal
> tsc index.ts --module commonjs; node ./index
[ 'abc', index: 0, input: 'abcdefgh' ]
Hello Dave
So what gives? Why did my attempt fail so miserably?
I would also appreciate any advice on the wisdom of using npm packages in TypeScript in general. Is it just to error prone for real use? If so, and you found yourself in the situation where you wanted some generic pubsub library in a TypeScript program you were using... what would you do? Write your own?
The problem is that pubsub.d.ts doesn't contain a definition for the CommonJS module "pubsub-js" (instead it only defines a global object PubSubJS).
The best fix would be to edit that file; at the bottom add:
declare module "pubsub-js" {
export = PubSub;
}
Most .d.ts files already include such a definition when a CommonJS or AMD module is available for a given package.
I created a small test program for web applications that uses jasmine, and I'm preparing it for easy downloads. Before installing my package, the user's project should look something like this:
myProject/
app/
lib/
...
I want to be able to have the user cd to myProject in the terminal, issue a single command that points to the app and lib folders, and then end up with this:
myProject/
app/
lib/
requirejs
test/
lib/
node_modules/
specs/
SpecRunner.html
server.js
...
app/ should contain the js project files, lib/ should contain all the external js dependencies for the project, and test/lib/ should contain all the external dependencies for the tests. server.js runs with nodejs and depends on apps installed in node_modules/.
What's the best way to go about doing this? I could make a bash script, but I'd rather use a package manager. I'm not sure how I'd do this in bower or npm. And am I right in thinking it's better to have two libs, one for the project and one for testing, rather than one? I know I can declare certain packages as test packages in bower, but it seems like they should live in a separate libraries.
And am I right in thinking it's better to have two libs, one for the project and one for testing, rather than one?
No. The idiomatic way in the npm-verse is to have tests in the same package in the test folder. Since bower is based on npm I'd say the same applies there too. If you don't want bower users to have to download test-stuff you should be able to ignore the test folder in the bower.json file (according to this answer). You should also specify node modules that are only used for tests as devDependencies.
Developers who want to run your test should IMO install it directly from source using e.g. git clone git#github.com/your/repo.git (and then just run npm install). Or simply npm install x if it's available on npm. Even if you really want the tests in their own package, I'd still suggest not using a package manager but ask the developer to clone it from the repo into the test folder.
Anyway to answer the question, the following one-liner should work (assuming npm, I'm not too familiar with bower):
npm install x-test && mv node_modules/x-test test