Ignore `nsp check` when doing `yarn/npm publish`? - javascript

I'm trying to publish a javascript package to an internal repository. When I do yarn publish (or npm publish) it now runs nsp check (otherwise wonderful feature!) to look for known vulnerabilities in any of the dependencies in my project. It finds one and exits in error:
[3/4] Publishing...
$ nsp check
(+) 1 vulnerability found
...
│ More Info │ https://nodesecurity.io/advisories/nnn |
...
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/publish for documentation about this command.
I'd like to ignore this vulnerability and proceed with the publish anyway. How can I do that?
I understand that introducing a way to ignore a vulnerability may be misused, but in this instance the package I'm publishing is an internal tool. I've looked at the dependency's vulnerability and in no way could it be taken advantage of in the context in which I'm using it.
Sidenote: The vulnerability is nested 3 layers deep in a pretty commonly used package (request#2.87.0). The vulnerability itself has been fixed in its own latest version, but because this dependency is three layers deep we have to wait for each subsequent dependent to upgrade. I suspect it will take some time- there's been a ticket open in their Github for 3 months- and I don't want to wait on that.

I've found the answer to this:
Add a file called .nsprc at the top of your project. In the file add:
{
"exceptions": ["https://nodesecurity.io/advisories/nnn"]
}
The URL should match the URL of the vulnerability presented in the nsp check command.

Related

How to create a common code module for use with cloud functions or docker

I have a typescript monorepo with front end, cloud functions, and back end (docker) packages in it. I'd like to add a "common" package that can be used by all of those.
Some production environments like Google Cloud Functions and Docker (at least in one common usage) package up the current app's directory and send it to a remote server, where it does "yarn install" to install all the dependencies. This means the obvious ways of referencing the common package (import * as common from "../../common" and the like) don't work because the build server won't find common there.
Also, sadly, docker and GCF don't follow symlinks, so the yarn link: protocol isn't helpful.
And since I'm working on the common code just as much as the apps, I really don't want to publish it to a private NPM registry every time I make a change. I'd like that change process to be as seamless as possible.
The most seamless way is to just symlink common into each app's node_modules, but (a) yarn deletes those, and (b) the builders don't follow symlinks. So I assume I'll need some kind of build/push process when changing the common code.
It seems like a kind of common thing to want to abstract out common code into its own module that can be used anywhere in the monorepo, but I can't find a solution that works. (For extra points I'd like "jump to definition" in my IDE to go to the actual definition, not a local copy.) Any ideas? Can Yarn 2 help with this use case?

Am I meant to commit yarn's `.pnp.js` file?

Yarn includes an optional "Plug'n'Play" feature which moves node_modules out of the project directory. In doing so it creates a .pnp.js file with references to various dependency paths on the hard drive.
The file is auto-generated, is several thousand lines long, and appears to reference paths specific to the machine that ran yarn install.
Is .pnp.js meant to be commited with the rest of the code? I can't seem to find any information about this despite the file appearing to be useless to commit. What is conisdered best practice and why?
I found this comment by Arcanis (the lead maintainer of Yarn) on an issue asking about what to include in a .gitignore. It may answer your question:
.yarn/plugins and .yarn/releases contain the Yarn releases used in the current repository (as defined by yarn set version). You will want to keep them versioned (this prevents potential issues if, say, two engineers use different Yarn versions with different features).
.yarn/unplugged should likely always be ignored, since it may contain native builds
.yarn/build-state.yml should likely be ignored as well, as it contains the build infos
If for some reason you version unplugged, it may make sense to keep build-state as well
.yarn/cache may be ignored, but you'll need to run yarn install to regenerate it
Versioning it unlocks what we call Zero-Installs - it's optional, though
.pnp.js (and potentially .pnp.data.json) are in the same boat as the cache. Add it to your repository if you keep your cache in your repository, and ignore it otherwise.
yarn.lock should always be stored within your repository (even if you develop a library)
So to summarize:
If you're using Zero-Installs:
.yarn/unplugged
.yarn/build-state.yml
If you're not using Zero-Installs:
.yarn/*
!.yarn/releases
!.yarn/plugins
.pnp.*
As one of the commenters noted, the official .gitignore recommendations are here, along with excellent commentary. The only critique I have is in the way they showed it there, I think they should have combined the common ways into one, and then separated the special cases for zero-install vs not. Here then is how I have it. But again, note that article, as some of these (e.g. .yarn/sdk) are optionally added or not.
# ------- yarn -------
# see excellent notes at: https://yarnpkg.com/getting-started/qa#which-files-should-be-gitignored
# Also: `yarn.lock` and `.yarnrc.yml` (or it's older counterpart .yarnrc) "should always be stored in your repo"
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions
## --> ADD if using zero-install, otherwise do NOT:
#!.yarn/cache
## --> ELSE ADD if NOT using yarn's zero-install:
.pnp.*
# ------- end yarn -------

test requirements for published npm packages [duplicate]

What exactly should I put in .npmignore?
Tests? Stuff like .travis.yml, .jshintrc? Anything that isn't needed when running the module (except the readme)?
I can't find any guidance on this.
As you probably found, NPM doesn't really state specifically what should go in there, rather they have a list of ignored-by-default files. Many people don't even use it as everything in your .gitignore is ignored in npm by default if .npmignore doesn't exist. Additionally, many files are already ignored by default regardless of settings and some files are always excluded from being ignored, as outlined in the link above.
There is not much official on what always should be there because it is basically a subset of .gitignore, but from what I gather from using node for 5-ish years, here's what I've come up with.
Note: By production I mean any time where your module is used by someone and not to develop on the module itself.
Pre-release cross-compiled sources
Pros: If you are using a language that cross-compiles into JavaScript, you can precompile before release and not include .coffee files in your package but keep tracking them in your git repository.
Build file leftovers
Pros: People using things like node-gyp might have object files that get generated during a build that never should go into the package.
Cons: This should always go into the .gitignore anyway. You must place these things inside here if you are using a .npmignore file already as it overrides .gitignore from npm's point of view.
Tests
Pros: Less baggage in your production code.
Cons: You cannot run tests on live environments in the slim chance there is a system-specific failure, such as an out of date version of node running that causes a test to fail.
Continuous integration settings/Meta files
Pros: Again, less baggage. Things such as .travis.yml are not required for using, testing, or viewing the code.
Non-readme docs and code examples
Pros: Less baggage. Some people exist in the school-of-thought where if you cannot express at least minimum viable functionality in your Readme, your module is too big.
Cons: People cannot see exhaustive documentation and code examples on their own file system. They would have to visit the repository (which also requires an internet connection).
Github-pages objects
Pros: You certainly don't need to litter your releases with CNAME files or placeholder index.htmls if you use your module serves double-duty as a gh-pages repository as well.
bower.json and friends
Pros: If you decide to build in your dependencies prior to release, you don't need the end-user to install bower then install more things with that. I would, personally, keep that stuff in the package. When I do an npm install, I should only be relying on npm and no other external sources.
Basically, you should ever use it if there is something you wish to keep out of your npm package but checked-in to your module's repo. It's not a long list of items, but npm would rather build in the functionality than having people stuck with irrelevant objects in their package.
I agree with lante's short and syntetic answer and SamT's big answer:
You should not include your tests in your package.
Your package should only contains production runtime files.
That will make your package more straightforward and faster to be dowloaded.
My contribution to those answers:
.npmignore is the blacklist way to achieve package file selection. But in a more practical way, you can whitelist files you need to include in your package using the files field in your package.json:
{
"files": [
"lib/",
"index.js"
]
}
I think that's simpler, future proof and have better semantics ;)
Just to clarify, anytime someone do npm install your-library, npm will download all source files that the package includes. Those files that were included in the .npmignore file in the source code of the package your-library will be excluded when publishing the lib, so users of your-library won't download them.
Know that people installing your library will need just your library running, anything else will be not necessary.
For example, when someone installs a library, its probably that he/she doesn't care about your .travis.yml or your .jshintrc files, or even some images, Grunt files, documentation, etc.
.npmignore could let your npm package to have less files, and faster to be downloaded
Don't include your tests. Oftentimes tests are like 5x the size of the actual codebase. As long as your tests are on Github, etc, that's good enough.
But what you absolutely should do is test your NPM package in its published format. Create some smoke tests that reside in the actual codebase, but are not part of the test suite.
You can read about testing your package after tarballing it, here:
https://github.com/ORESoftware/r2g
How to test an `npm publish` result, without actually publishing to NPM?

SonarQube: Integrate ESLint for JavaScript in SonarQube?

We have determined our rules, which should be used for JavaScript code, with ESLint. Now we want to integrate ESLint to SonarQube as we did it before the same way with Checkstyle for JavaCode.
Under the following link it is described why SonarQube doesn't want to provide a plugin for ESLint:
http://www.sonarqube.org/sonarqube-javascript-plugin-why-compete-with-jslint-and-jshint/
Is there still no plugin fir ESLint in SonarQube? Isn't this part of a marketing strategy? There is also a plugin for Checkstyle, FindBugs etc... Why does SonarQube suddenly stop to support the integration of other code analysing tools?
Yes, there is still no plugin for ESLint, and this is part of the strategy, but in the other direction.
In fact, our first plugins were for external analyzers, and over time we realized that simply aggregating other tools' results didn't truly serve the community because that community was coming to us with rule bugs, requests and suggestions for improvement - and all we could do was refer them on the tools' makers.
So we started writing our own rules instead for better responsiveness and, we believe, enhanced accuracy.
I urge you to take the rules you feel are missing to the SonarQube Google Group
Edit The strategy has come full circle. SonarJS now imports ESLint reports.
I have not tried it yet, but I've just discovered this plugin that seems to be very promising for any front end project:
https://github.com/groupe-sii/sonar-web-frontend-plugin
It may worth giving a try
You have a good option here:
https://github.com/sleroy/SonarEsLintPlugin
From the docs:
• Install Node.js
• Install EsLint (3+) with npm install -g eslint, or ensure it is installed locally against your project
• If you're installing globally, find the path to EsLint and copy it - will be similar to C:\Users\
[Username]\AppData\Roaming\npm\node_modules\eslint\bin\eslint.js on Windows
• Copy .jar file (downloaded from https://github.com/sleroy/SonarEsLintPlugin/releases page) to SonarQube extensions folder
• Restart SonarQube server
• Browse to SonarQube web interface, login as Admin, hit up Settings
• Hit the Rules tab, then the EsLint rule set, then apply it to your project - alter rule activation as required
I came across a similar requirement and the following are the steps that I followed to achieve the ESLint report output integration into the Sonarqube dashboard:
Switch to the project directory.
Run: npx eslint <source_code_to_be_scanned> --format json --output-file <file_name>.json
For e.g. npx eslint ./src/**/*.js --format json --output-file eslint-result.json
Cope the relative path of the generated report.
Add an additional property in your sonar-project.properties file as below:
sonar.eslint.reportPaths=app/eslint-result.json
Perform the scan
Output: I was able to see the expected output within my Sonarqbue dashboard.
I hope some of you find it helpful and save a bit of time.

How to add tab completion to a Nodejs CLI app

I want to add tab completion to a Nodejs CLI app (And preferably generate the tab completion dynamically).
I found a few npm modules but not sure how to really implement them:
https://github.com/hij1nx/complete
https://github.com/mklabs/node-tabtab
So what I am looking for is so I can have a nodejs file that is something like:
my-cmd create arg1 arg2
But then I might want to autocomplete like:
my-cmd cr<tab> -> create
Thanks!
Use omelette package that I built. If you have any questions, please contact me.
Edit - fast answer
After I answered, I kept reading tabtab source a bit and noticed that I can also run
pkgname completion install
to install the completion. since my environment was already dirty, I don't know if it actually did anything, but seems to me like it did..
Longer answer
#CameronLittle has given great documentation.
For the impatient, you can start by running
sudo bash -c 'pkgname completion > /etc/bash_completion.d/pkgname'
source /etc/bash_completion.d/pkgname
This will add completion to your current bash session.
As far as I know, new sessions will get the completion automatically.
To make the process seamless for user, you can use the install and postinstall hooks in package.json
https://docs.npmjs.com/misc/scripts
Make sure to not print anything by default. means running pkgname should result in no output, or otherwise it will not work.
important! install tabtab only from master
It seems tabtab has an annoying bug that was resolved in master but never got into a release..
The relevant commit to fix it is this:
https://github.com/mklabs/node-tabtab/commit/f8473555bf7278a300eae31cbe3377421e2eeb26
which handles completion for strings starting with --.
The commit if from february 2014, however the latest release as of (Jan. 2015) is 0.0.2 from Jan. 2014.. I assume there will not be more releases.
So if you want to get this fix, and you should(!), install tabtab only from master.
don't waste 2 hours figuring out what you did wrong like me :)
How did i reach this answer? TL;DR
While #CameronLittle's answer gives the explanation behind the scene, I would like to explain how to I reached the answer.
I tried using the package tabtab which has an explicit section about installing it. see https://www.npmjs.com/package/tabtab#completion-install
However, that didn't seem to work for me.
Looking at the code they instruct to add, I see the following process.argv.slice(2)[0] === 'completion' which made me run the command pkgname completion, which outputs something that starts with
###-begin-pkgname-completion-###
### credits to npm, this file is coming directly from isaacs/npm repo
#
# Just testing for now. (trying to learn this cool stuff)
#
# npm command completion script
#
# Installation: pkgname completion >> ~/.bashrc (or ~/.zshrc)
#
the words this file is coming directly from isaacs/npm repo made me wonder more. following the other answer here, I looked at /etc/bash_completion.d/npm - which showed the same exact content.. and so the comment.
I decided to run
pkgname completion > /etc/bash_completion.d/pkgname
however that requires sudo permissions and so becomes
sudo bash -c "pkgname completion > /etc/bash_completion.d/pkgname
and then, in order to apply it to current bash session I had to run
source /etc/bash_completion.d/pkgname
and voila! it works!
when I tried to open another terminal, it still worked, so I assume it will apply to all users. if not - you should add it to .bashrc or something..
I would just like to add that there is a
npm package yargs that enables bash-completion shortcuts for commands and options.
It has the option to output a .bashrc completion script. Bash completions are then enabled by sourcing the generated script.
It is currently an actively maintained package on npm with over a million downloads a month.

Categories

Resources