Develop Reference

How to install npm dependencies which aren't in package.json?

I have problem with npm install. I want to install all dependencies used in my project. In package.json there aren't any dependencies, but in my project files I have for example const mongo=require('mongoose') and in another file const morgan=require('morgan') etc.
I know that when typing npm i --save <dependency_name> it will update my package.json. But I would like to install all dependencies without typing their names explicitly anywhere.
Is there any way to install all dependencies used in the whole project, but which there aren't in package.json?

It is not intended to write your program code first and then let NPM find all dependencies that are used. This is because of a simple reason: Before you can use an external library / package, you'll have to download it, otherwise you could hardly use it at all.
Now, I only see two reasons for your use case:
you copied and pasted some foreign code where you don't have all files, especially no package.json, or
you "inherited" some code of a former employee or team that is not documented and consists of unrelated, inconsistent files.
Anyways, there would be some kinds of solutions for your problem:
The nasty solution: Run npm start and see what error messages you get. All uninstalled dependencies will not be found, so you'll see the package names and can add them manually to your package.json. This will become nasty because you'll have to re-run your program each time. To avoid this, you could have a look at Nodemon which automatically re-starts your program.
The better solution: Open a web IDE of your favor and use the "Global Search" function to find all occurrences of the string require(, or as a Regex: require\((.+)\). This will list you all dependency imports in your program files. If there should be also ECMA 6 imports, also search for import (.+) from (.+). - However, you'll still have to copy and paste all dependency names from all files manually into your package.json file.
The best but most complex solution: Write a Node.js script that scans all your files recursively, beginning in your root project directory. Create a dependency memory variable, like let dependencies = []. Read all *.js files (sync or async) and every time the require or import statement is matched, check whether the dependency is already in your dependencies array. If not, push it. Finally, all your project dependencies will be listed in the dependencies array and you can copy and paste them into your package.json.
Pseudo Node.js Code:
const lineReader = require('line-reader');
let dependencies = [];
const regex = /require\(['|"](.+)['|"]\)/g;
lineReader.eachLine('/path/to/file', function(line) {
const match = regex.exec(line);
if(match) {
if(dependencies.indexOf(match[1]) === -1) {
dependencies.push(match[1]);
}
}
});

No, you can't do that - you will have to add all dependencies by installing them explicitly, like npm install morgan. NPM is not aware of the dependencies you're importing in your files. Another thing is that requiring dependencies that are not listed in package.json is simply wrong and should never take place.

Short answer: You can't
How npm install works is checking all your dependencies listed in the package.json and install them by once. So either you get the package.json from the tutor or you install them one by one

You can scan your project for all required modules.
Assuming your project only uses common.js (require) you can get a list of all modules by doing something like:
egrep -R --exclude-dir=node_modules '=\s*require\s*\(' | awk '{gsub(/^.+require\s*\(\s*./,""); gsub(/.\s*\).*$/,""); print $0}'
A more readable version of the above command is:
#! /bin/bash
egrep -R --exclude-dir=node_modules '=\s*require\s*\(' |
awk '{
gsub(/^.+require\s*\(\s*./,"");
gsub(/.\s*\).*$/,"");
print $0
}'
You can save the script above in a file and execute it as a shell script.
To automatically install the modules just pipe it to xargs:
egrep -R --exclude-dir=node_modules '=\s*require\s*\(' |
awk '{
gsub(/^.+require\s*\(\s*./,"");
gsub(/.\s*\).*$/,"");
print $0
}' |
xargs npm install
I leave supporting ES6 module as homework for the reader.

Single makefile to build multiple subdirectories

So inspired by http://www.olioapps.com/blog/the-lost-art-of-the-makefile/ I wanted to try to build a repo of several JS projects using make.
What I'm trying to do initially is just the rule to run yarn install whenever either package.json or yarn.lock changes. The rule if it's just a static rule is explained in the link and is just
node_modules: package.json yarn.lock
yarn install
However I'm trying to do a similar rule that runs for multiple projects.
I have a directory structure like similar to this:
Makefile
rocks/js/projectA/package.json
rocks/js/projectA/yarn.lock
rocks/js/projectB/package.json
rocks/js/projectB/yarn.lock
...
And so forth with several projects. I've tried to use static pattern rules to write this out, but so far have failed quite miserably. What my Makefile looks like currently is
js_rocks = $(addprefix rocks/js/,$(shell ls rocks/js))
%/node_modules: %/package.json %/yarn.lock
cd $(dir $#) && yarn install
$(js_rocks): %: %/node_modules
all: $(js_rocks)
.PHONY: all
This however only seems to detect and run for the first project in the list.

Try this change:
%/node_modules: %/package.json %/yarn.lock
cd $(dir $#) && ( yarn install ; cd - )
After the first one, the current directory is left in the node_modules directory. From there, make cannot find any more targets.

So turns out the main problem was not really in the targets themselves, but in the fact that I had misunderstood make's default target to always be all whereas it is actually always the first target (thank you #shawnhcorey).
So with the feedback I got and a bit more reading up on make, this is the Makefile I have now which can run yarn install for all my sub-projects :)
js_rocks = $(wildcard rocks/js/*)
node_modules = $(addsuffix /node_modules,$(js_rocks))
all: $(node_modules)
clean:
rm -rf $(node_modules)
$(node_modules): %/node_modules: %/package.json %/yarn.lock
cd $*; yarn install
.PHONY: all

NPM - Scripts - How Do They Work?

I cant get my head around how scripts are running within package.json & would appreciate some insight for us newbies.
Is it the case that they are bash scripts that are run by node having loaded the various dependencies?
If yes, then how does it process the javascript code?

Is it the case that they are bash scripts
yes
that are run by node
no, they are run by sh.
having loaded the various dependencies?
no, no js files are loaded, the only thing npm does for you is to prepare the environment. Among other things, it adds ./node_modules/.bin to PATH so you can invoke installed modules immediately.
When you run npm run-script whatever, this is what npm does:
reads the corresponding command line from package.json
prepares the environment
invokes sh (or comspec on win) and gives it the command and the env. No big magic here.

This may not be 100% accurate so I implore other, more qualifies, experts to chime in.
NPM is a program, installed as part of the Node.JS environment. It's two main uses (as describe here) are for searching for node.js packages and installing node.js packages.
However, NPM is also capable of understanding "simple" (a relative term) scripts.
When you write a script in your package.json, and issue the NPM command, say "npm start", NPM will read and interpret the script. NPM then searches your node_modules structure for the accompanying binary and executes that binary with the necessary start parameters.
An example would be
"test": "mocha --reporter spec test"
when you issue "npm test", NPM will look for the mocha binary in your node_modules structure. NPM finds mocha initiates the call, passing the reporter command arg (--reporter spec) and the name of the file to be read and executed for the test.

How to run specific tests with frisby?

We are using frisby to run our integration tests and while developing them, it would be handy to execute one specific one or a group of tests, without having run all of them and see extra noise. Right now I am commenting out all the ones I don't want to run, which is getting tedious.
Is there a way, from the command line, to run specific frisby tests?
So basically, instead of
npm test
I want to be able to say
npm test --name: posts
Or something like that. I found this post about jasmine-only, but I'm not sure it will satisfy my needs.
Thanks so much!

I'm not sure if you're still looking for answer, but this is pretty simple.
Firstly install latest version of jasmine-node from command line: npm install jasmine-node -g
Then to run particular test use: jasmine-node --coffee putTestNameHere

Install jasmine-node module. Execute one file at a time - you can group your test cases in specific file:
jasmin-node moduleTestCases_spec.js
Also, if you want to specify exact test case name to be executed, you can make use of sequenty module. It is a nodejs module, which you can specify the order(and thus the exact test cases to execute).

To run specific test in Frishby just run:
npm test ./folder/filename.js
So lets assume you have an folder say test under that you have a file called api.spec.js
then you will execute like this:
npm test ./test/api.spec.js
Parallely don't forget to specify these below things into your package.json file
"scripts": {
"test": "mocha" }

Writing a script to automate npm command

I am working on an angular js project and I would like to automate The following two commands.
./node_modules/protractor/bin/webdriver-manager update
./node_modules/protractor/bin/webdriver-manager start
The issue is that I am working on a small angular project on github. I added all dependencies required to my package.json, However when my friend pulled it from git he was able to install protractor but he could not get webdriver to start unless he ran the above two commands. So i wanted to write some script to automate it and better yet even add protractor ./conf.js to it.
So I did research and I am aware that I can write a npm script but I was not able to find a proper document that showed where to write the script and how to execute it. I would appreciate all suggestions.

You can add a scripts property to your package.json with the command you wish you run.
"scripts": {
"prostart": "./node_modules/protractor/bin/webdriver-manager start",
"proupdate": "./node_modules/protractor/bin/webdriver-manager update"
}
you would then run these by typing npm run prostart or npm run proupdate which would look for those commands in your package.json.

In addition to Josh's answer, the script start could be run as npm start as start is a special keyword, but update should be run as npm run update because npm update is another npm command entirely.
For any other command besides start and test (I think), you have to preface it with npm run ...