Develop Reference

How to install a npm package from github requiring a build step, e.g. when forking a library?

Assume you use a library like vue3-datepicker. You realize you need to customize something, and as as first step you want to use a custom fork of it.
The issue is, there is a build step when the package is pushed to npm's registry since the project doesn't use plain JavaScript, but may have vue or typescript files.
In this case, that would be npm run build:component, though that depends on the project.
Just installing the fork from github via:
yarn add <GitHub user name>/<GitHub repository name>#<branch/commit/tag>
hence doesn't suffice as then the ./dist folder doesn't exist.
You'll get really strange errors like:
error: [plugin: vite:dep-scan] Failed to resolve entry for package "vue3-datepicker". The package may have incorrect main/module/exports specified in its package.json: Failed to resolve entry for package "vue3-datepicker". The package may have incorrect main/module/exports specified in its package.json.
As a quick and dirty solution, I removed in my fork the ./dist/ folder from the .gitignore, ran the npm i && npm run build:component in my fork, and pushed it.
Huge downside is, the ./dist/ folder is now part of that repository, after each change in my fork I also have to build the files again and push those as well.
I rather have the build process triggered in my application using my fork. Is there a way from my application to say:
When you install that library, you have to run a certain script once you downloaded all the files?
The solution should be usable for both npm and yarn, in the sense that the fork my be installed by either one in different applications.

A quote from npm-install Docs
If the package being installed contains a prepare script, its dependencies and devDependencies will be installed, and the prepare script will be run, before the package is packaged and installed.
so in your fork's package.json you can add
"scripts": {
// ...
"build:component": "rollup -c build/rollup.config.js",
"prepare": "yarn build:component || npm run build:component"
}

If you want to trigger builds after installation, you can use the postinstall or a build script in your package.json. In this script, you can create directories and do other setups, using shell commands or javascript programs:
{
"scripts": {
"build": "mkdir dist && npm run build:component",
"build:component": "some command"
}
}

Running js tests - getting "Cannot use import statement outside of a module"

I added the type: module but that didn't help.
I am trying to run mocha or jest tests that use import and export for the source files.
The existing questions about this have specifics that are different from mine and I also find them confusing to follow for someone with my specific situation, especially since I have developed a specific answer with details not relevant to the existing questions, but relevant to other people in my situation.

In the past, you could not use ES modules (i.e. import/export) in Node without transpiling your code using Babel. However, support for ES modules in Node is now a reality, and both Jest and Mocha have also recently added support for using ES modules natively.
It takes more than just adding "type": "module" to your package.json, however.
Steps for using native ES Modules in Jest
As already mentioned, add "type": "module" to your package.json
Install either jest-environment-node or jest-environment-jsdom-sixteen to your development dependencies. For example:
$ npm i -D jest-environment-node
Update the Jest configuration in package.json and add the testEnvironment setting. For example:
"jest": {
"testEnvironment": "jest-environment-jsdom-node"
}
If you are using a version of Node earlier than 13.2, then you will need to add two additional flags when running Node: --experimental-modules and experimental-vm-modules. I use npx to execute the commands, although it's a little verbose:
$ npx --node-arg=--experimental-modules --node-arg=--experimental-vm-modules jest
This will run all your Jest tests using the appropriate Node flags. I'd recommend making this your test script in package.json if this is the way you go.
Now you should be able to use import/export without Babel!
One last important point: when using native ES modules in Node, you have to use the entire import path to your local modules, including the file extensions. For example:
import lib from "./my/lib.js"
Here is the Node documentation on native ES modules, if you want to read about this in more detail: https://nodejs.org/dist/latest-v14.x/docs/api/esm.html
I'd also recommend reading through this Github issue for more details on the Jest implementation of native ES modules: https://github.com/facebook/jest/issues/9430

Per the Jest documentation, if you make sure babel-jest is installed and supply your Babel configuration per the Babel documentation, in a config file or package.json e.g.:
.babelrc.json
{
"presets": [
"#babel/env"
]
}
Then babel-jest will pick it up automatically and you don't need to explicitly pre-build the files to test them. This also means you don't have to set flags on the command line when calling Babel.
Note that if you do want to explicitly pre-build, I'd recommend:
Using a pre<script> script rather than having multiple steps in one line; and
Re-using the build script so you don't have to make changes in two places.
In your case:
"scripts": {
"build": "babel src/ --out-dir lib",
"pretest": "npm run build",
"test": "jest lib/*.test.js"
}

You need to use a compiler and then use the compiled files when running tests.
Many of the references say to add type: module but don't say much more.
To be clear, the basic message:
SyntaxError: Cannot use import statement outside a module
is because you are using import/export and you are trying to run the files directly without compilation.
The mindshift here is getting used to editing the files in one directory and running the tests in another. Alternatively, some solutions offer "in'flight" compilation so this detail is essentially hidden and only the source files are used.
There are a few different approaches to doing this depending on specific needs. Here is one of the simplest approach I've found so far, using Babel for the compilation step:
Install babel npm install babel --save-dev
Add babel commands in package.json scripts, for example:
"scripts": {
"test": "babel src/ --out-dir lib --presets=#babel/env; jest lib/*.test.js",
"build": "babel src/ --out-dir lib --presets=#babel/env"
}, // this was for jest but you can use mocha, etc as needed
// Note that using preset this way eliminates the need for a specific .babel.config.json file
Now, if you run:
jest .
you get SyntaxError: Cannot use import statement outside a module, but if you run npm t you get
Successfully compiled 2 files with Babel.
PASS lib/app.test.js
All tests
✓ Canary test (2ms)
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Going forward, just remember to:
Edit files in src/
Run tests in lib/

How does npm scripts prioritise local dependency over global ones?

I understand npm scripts adds ./node_modules/.bin to your PATH, therefore you can simply run npm test using the package.json below, and npm will automagically use the local version of mocha found in ./node_modules/.bin
"scripts": {
"test": "mocha"
}
This is a nice feature, because it saves me writing package.json files like this:
"scripts": {
"test": "./node_modules/.bin/mocha"
}
BUT what if I bring on a new developer who has mocha installed globally? or I need to push this to an environment with preconfigured global packages? If I am using the short-hand mocha, rather than ./node_modules/.bin/mocha in my package.json, What takes precedence, the global or local package?

Node.js will try to run first your locally installed packages.
If you require a module, Node.js looks for it by going through all node_modules/ directories in ancestor directories (./node_modules/, ../node_modules/, ../../node_modules/, etc.). The first appropriate module that is found is used.
For a more detailed explanation about how Node.js resolves required modules, here is a nice breakdown.

Is there source map support for typescript in node / nodemon?

I have a node project written in typescript#2.
My tsconfig has sourceMap set to true and the *.map.js files are generated. When I execute my transpiled *.js JavaScript files via node or nodemon, I only see the error messages relative to the js file and not to the mapped typescript files; I assume it's completely ignored.
Is sourceMap support only intended for browser-support? Or can I use it together with node or nodemon? If the latter, how would I enable it?
I want to see runtime errors detected from an executed javascript file relative to the original typescript file.

🚩 for Node versions since v12.12, there is an easier and better solution.
I recently got this working in my express app. Steps as follows:
Install the required library:
npm install --save-dev source-map-support
In your entry point (eg app.ts):
require('source-map-support').install();
In your app.ts, you may also require better logging for errors within promises:
process.on('unhandledRejection', console.log);
In your tsconfig, under compilerOptions:
"inlineSourceMap": true

The answers here are correct for Node versions before v12.12.0, which added the (experimental) --enable-source-maps flag. With that enabled, source maps are applied to stack traces without an additional dependency. As demonstrated in this article, it has the slightly different and possibly beneficial behavior of including both the generated .js file location and the source file location. For example:
Error: not found
at Object.<anonymous> (/Users/bencoe/oss/source-map-testing/test.js:29:7)
-> /Users/bencoe/oss/source-map-testing/test.ts:13:7

Install source map support:
npm install source-map-support
(I run in in production as well, as it immensely helps finding bugs from the logs of when an error an occurs. I did not experience a large performance impact, yet your experience may be different.)
Add to your tsconfig.json:
{
"compilerOptions": {
"sourceMap": true
}
}
When running your JavaScript file, add the require parameter:
nodemon -r source-map-support/register dist/pathToJson.js
node -r source-map-support/register dist/pathToJson.js
Alternatively, you add in your entry call:
require('source-map-support').install()
yet I find this tedious is projects with multiple entry points.
Sidenote: mocha also supports the --require / -r option, so to have the sourcemap support in mocha you can also call your tests with it, e.g. similar to:
NODE_ENV=test npx mocha --forbid-only --require source-map-support/register --exit --recursive ./path/to/your/tests/

I found this npm module which seems to do the trick:
https://github.com/evanw/node-source-map-support
run npm install source-map-support --save at the root of your node project and add import 'source-map-support/register' to your main.ts or index.ts file.
That's it.

Source map support works perfectly fine with node
All you need to do is add
"source-map-support": "0.4.11",
to dependencies or dev-dependencies in package.json by running
npm install --save source-map-support
And in your entry point ts file, simply add at the top
require('source-map-support').install()
(note: this is calling nodeJS require - there is no need for source-map-support definition files)

For Node versions from v12.12.0 use the --enable-source-maps flag when you run node.
Example: node --enable-source-maps main.js
Do not install "source-map-support" for Node versions from v12.12.0

What's the js equivalent of RailsInstaller?

I want to build a small js plugin, and I want to try that with ReactJS.
ReactJS recommends installing using npm and browserify.
In my experience with Ruby on Rails, there are always a lot of things to install, and using Windows introduces additional problems.
With Ruby it is Rails Installer. What is JS equivalent of Rails Installer? i.e. a tool that lets me install all necessary packages with one step?
I did a little bit of search, find that I need to install:
NodeJS, NPM
NVM
Webpack or Browerify
Babel
I think there may be others that I need.

To start the project you need node and npm because webpack, browserify and babel is npm packages.
The only way to install all packages in one step is to install package that actually depend on all packages you need. There is a lot of ReactJS starter kits in github like this https://github.com/kriasoft/react-starter-kit
I, personally recommend you doing such a things by yourself - its not a big deal if you understand what you want. In the root of all nodejs projects there is a file package.json every time you need a new module (package) just type npm install -S <module_name> in the root directory of your project.
This new module appears in node_modules directory and because of -S flag it name also will be stored in dependencies of your project in package.json file so that in the future you can just type npm install to install all dependencies (aka modules/packages) of your project.