What exactly should I put in .npmignore?
Tests? Stuff like .travis.yml, .jshintrc? Anything that isn't needed when running the module (except the readme)?
I can't find any guidance on this.
As you probably found, NPM doesn't really state specifically what should go in there, rather they have a list of ignored-by-default files. Many people don't even use it as everything in your .gitignore is ignored in npm by default if .npmignore doesn't exist. Additionally, many files are already ignored by default regardless of settings and some files are always excluded from being ignored, as outlined in the link above.
There is not much official on what always should be there because it is basically a subset of .gitignore, but from what I gather from using node for 5-ish years, here's what I've come up with.
Note: By production I mean any time where your module is used by someone and not to develop on the module itself.
Pre-release cross-compiled sources
Pros: If you are using a language that cross-compiles into JavaScript, you can precompile before release and not include .coffee files in your package but keep tracking them in your git repository.
Build file leftovers
Pros: People using things like node-gyp might have object files that get generated during a build that never should go into the package.
Cons: This should always go into the .gitignore anyway. You must place these things inside here if you are using a .npmignore file already as it overrides .gitignore from npm's point of view.
Tests
Pros: Less baggage in your production code.
Cons: You cannot run tests on live environments in the slim chance there is a system-specific failure, such as an out of date version of node running that causes a test to fail.
Continuous integration settings/Meta files
Pros: Again, less baggage. Things such as .travis.yml are not required for using, testing, or viewing the code.
Non-readme docs and code examples
Pros: Less baggage. Some people exist in the school-of-thought where if you cannot express at least minimum viable functionality in your Readme, your module is too big.
Cons: People cannot see exhaustive documentation and code examples on their own file system. They would have to visit the repository (which also requires an internet connection).
Github-pages objects
Pros: You certainly don't need to litter your releases with CNAME files or placeholder index.htmls if you use your module serves double-duty as a gh-pages repository as well.
bower.json and friends
Pros: If you decide to build in your dependencies prior to release, you don't need the end-user to install bower then install more things with that. I would, personally, keep that stuff in the package. When I do an npm install, I should only be relying on npm and no other external sources.
Basically, you should ever use it if there is something you wish to keep out of your npm package but checked-in to your module's repo. It's not a long list of items, but npm would rather build in the functionality than having people stuck with irrelevant objects in their package.
I agree with lante's short and syntetic answer and SamT's big answer:
You should not include your tests in your package.
Your package should only contains production runtime files.
That will make your package more straightforward and faster to be dowloaded.
My contribution to those answers:
.npmignore is the blacklist way to achieve package file selection. But in a more practical way, you can whitelist files you need to include in your package using the files field in your package.json:
{
"files": [
"lib/",
"index.js"
]
}
I think that's simpler, future proof and have better semantics ;)
Just to clarify, anytime someone do npm install your-library, npm will download all source files that the package includes. Those files that were included in the .npmignore file in the source code of the package your-library will be excluded when publishing the lib, so users of your-library won't download them.
Know that people installing your library will need just your library running, anything else will be not necessary.
For example, when someone installs a library, its probably that he/she doesn't care about your .travis.yml or your .jshintrc files, or even some images, Grunt files, documentation, etc.
.npmignore could let your npm package to have less files, and faster to be downloaded
Don't include your tests. Oftentimes tests are like 5x the size of the actual codebase. As long as your tests are on Github, etc, that's good enough.
But what you absolutely should do is test your NPM package in its published format. Create some smoke tests that reside in the actual codebase, but are not part of the test suite.
You can read about testing your package after tarballing it, here:
https://github.com/ORESoftware/r2g
How to test an `npm publish` result, without actually publishing to NPM?
Npm allows several versions of the same package to be used in a project. This is a powerful feature.
However in most front-end projects, I would argue that it is not desired to have dependencies on the same library in different versions.
Having dependencies on the same library in different versions means, that the end-user would have to load the given library several times (as seperate requests or as part of a bigger bundle).
However if npm is used to manage dependencies of a frontend-project, you can end up very quickly with dependencies on the same library in different versions.
I think in most cases this is undesired and most of the time we are not even aware of the situation.
A simple case where we end up in such a situation:
At some point in time you install react-router and history in your project:
npm i -S react-router#1.0.0-rc1
npm i -S history#1.17.0
At that time react-router has a dependency to history#1.17.0. As a result your project as a whole has only a dependency to this version of history.
Later you decide to upgrade to the latest version of react-router:
npm i -S react-router#2
Now react-router has a dependency on history#2.
As a consequence your project has now a dependency on history#1.17.0 and a transitive dependency on history#2.
Both versions of history are included in your npm_modules.
If your are using a module bundler like Webpack or Browerify, you end up with a bundle that contains both versions of history.
Most probably you don't notice. But if you would notice, you would probably upgrade your direct dependency to `history#2.
How can we deal with this problem?
How can we find out if our front-end project has dependencies on the same library in different versions?
How can we avoid ending up with bundles that are bigger than they should be?
I know that technically npm/Webpack/Browserify are correct in including dependencies on the same library in different versions in a bundle. But I am looking for a way to make it easily visible that this is happening, so that the developer can take steps to optimize the dependencies.
I also recreated the example in this repo: https://github.com/jbandi/npm-package-problems
The find-duplicate-dependencies tool can be used to examine a node_modules directory and report upon packages that are present with multiple versions.
I guess it makes some sense to tackle the problem at a node_modules-level, as doing so works in both WebPack and Browserify situations.
I've come my way from java development to development of complex Javascript projects. In Java build systems you don't checkin your dependencies (your jars) to sourcvode repository but let a build tool (maven, gradle, ...) manage that dependencies for you.
Now bower seems for me to be something like Maven to handle that dependency things for me. What I wonder about: Should I checkin the libs downloaded by bower or should I just checkin a script describing the dependencies I have and keep my repository free of thrid party code? Whats best practise? (And why?)
There are some pros and cons when you check the dependencies in your source control. The main reason for checking your dependencies would be to avoid issues while deploying your app. For example if Github is having network issues you can't deploy to production because your dependencies can't be fetched.
On the other hand the benefit of only having your bower.json file in your project instead of the whole bower_components folder is that it will keep your repository smaller. If you use CI, you can run bower install, run tests to see if everything works, and then deploy the exact copy in your production server.
Ultimately it is up to you to decide the best approach.
What is the fundamental difference between bower and npm? Just want something plain and simple. I've seen some of my colleagues use bower and npm interchangeably in their projects.
All package managers have many downsides. You just have to pick which you can live with.
History
npm started out managing node.js modules (that's why packages go into node_modules by default), but it works for the front-end too when combined with Browserify or webpack.
Bower is created solely for the front-end and is optimized with that in mind.
Size of repo
npm is much, much larger than bower, including general purpose JavaScript (like country-data for country information or sorts for sorting functions that is usable on the front end or the back end).
Bower has a much smaller amount of packages.
Handling of styles etc
Bower includes styles etc.
npm is focused on JavaScript. Styles are either downloaded separately or required by something like npm-sass or sass-npm.
Dependency handling
The biggest difference is that npm does nested dependencies (but is flat by default) while Bower requires a flat dependency tree (puts the burden of dependency resolution on the user).
A nested dependency tree means that your dependencies can have their own dependencies which can have their own, and so on. This allows for two modules to require different versions of the same dependency and still work. Note since npm v3, the dependency tree will be flat by default (saving space) and only nest where needed, e.g., if two dependencies need their own version of Underscore.
Some projects use both: they use Bower for front-end packages and npm for developer tools like Yeoman, Grunt, Gulp, JSHint, CoffeeScript, etc.
Resources
Nested Dependencies - Insight into why node_modules works the way it does
This answer is an addition to the answer of Sindre Sorhus. The major difference between npm and Bower is the way they treat recursive dependencies. Note that they can be used together in a single project.
On the npm FAQ: (archive.org link from 6 Sep 2015)
It is much harder to avoid dependency conflicts without nesting
dependencies. This is fundamental to the way that npm works, and has
proven to be an extremely successful approach.
On Bower homepage:
Bower is optimized for the front-end. Bower uses a flat dependency
tree, requiring only one version for each package, reducing page load
to a minimum.
In short, npm aims for stability. Bower aims for minimal resource load. If you draw out the dependency structure, you will see this:
npm:
project root
[node_modules] // default directory for dependencies
-> dependency A
-> dependency B
[node_modules]
-> dependency A
-> dependency C
[node_modules]
-> dependency B
[node_modules]
-> dependency A
-> dependency D
As you can see it installs some dependencies recursively. Dependency A has three installed instances!
Bower:
project root
[bower_components] // default directory for dependencies
-> dependency A
-> dependency B // needs A
-> dependency C // needs B and D
-> dependency D
Here you see that all unique dependencies are on the same level.
So, why bother using npm?
Maybe dependency B requires a different version of dependency A than dependency C. npm installs both versions of this dependency so it will work anyway, but Bower will give you a conflict because it does not like duplication (because loading the same resource on a webpage is very inefficient and costly, also it can give some serious errors). You will have to manually pick which version you want to install. This can have the effect that one of the dependencies will break, but that is something that you will need to fix anyway.
So, the common usage is Bower for the packages that you want to publish on your webpages (e.g. runtime, where you avoid duplication), and use npm for other stuff, like testing, building, optimizing, checking, etc. (e.g. development time, where duplication is of less concern).
Update for npm 3:
npm 3 still does things differently compared to Bower. It will install the dependencies globally, but only for the first version it encounters. The other versions are installed in the tree (the parent module, then node_modules).
[node_modules]
dep A v1.0
dep B v1.0
dep A v1.0 (uses root version)
dep C v1.0
dep A v2.0 (this version is different from the root version, so it will be an nested installation)
For more information, I suggest reading the docs of npm 3
TL;DR: The biggest difference in everyday use isn't nested dependencies... it's the difference between modules and globals.
I think the previous posters have covered well some of the basic distinctions. (npm's use of nested dependencies is indeed very helpful in managing large, complex applications, though I don't think it's the most important distinction.)
I'm surprised, however, that nobody has explicitly explained one of the most fundamental distinctions between Bower and npm. If you read the answers above, you'll see the word 'modules' used often in the context of npm. But it's mentioned casually, as if it might even just be a syntax difference.
But this distinction of modules vs. globals (or modules vs. 'scripts') is possibly the most important difference between Bower and npm. The npm approach of putting everything in modules requires you to change the way you write Javascript for the browser, almost certainly for the better.
The Bower Approach: Global Resources, Like <script> Tags
At root, Bower is about loading plain-old script files. Whatever those script files contain, Bower will load them. Which basically means that Bower is just like including all your scripts in plain-old <script>'s in the <head> of your HTML.
So, same basic approach you're used to, but you get some nice automation conveniences:
You used to need to include JS dependencies in your project repo (while developing), or get them via CDN. Now, you can skip that extra download weight in the repo, and somebody can do a quick bower install and instantly have what they need, locally.
If a Bower dependency then specifies its own dependencies in its bower.json, those'll be downloaded for you as well.
But beyond that, Bower doesn't change how we write javascript. Nothing about what goes inside the files loaded by Bower needs to change at all. In particular, this means that the resources provided in scripts loaded by Bower will (usually, but not always) still be defined as global variables, available from anywhere in the browser execution context.
The npm Approach: Common JS Modules, Explicit Dependency Injection
All code in Node land (and thus all code loaded via npm) is structured as modules (specifically, as an implementation of the CommonJS module format, or now, as an ES6 module). So, if you use NPM to handle browser-side dependencies (via Browserify or something else that does the same job), you'll structure your code the same way Node does.
Smarter people than I have tackled the question of 'Why modules?', but here's a capsule summary:
Anything inside a module is effectively namespaced, meaning it's not a global variable any more, and you can't accidentally reference it without intending to.
Anything inside a module must be intentionally injected into a particular context (usually another module) in order to make use of it
This means you can have multiple versions of the same external dependency (lodash, let's say) in various parts of your application, and they won't collide/conflict. (This happens surprisingly often, because your own code wants to use one version of a dependency, but one of your external dependencies specifies another that conflicts. Or you've got two external dependencies that each want a different version.)
Because all dependencies are manually injected into a particular module, it's very easy to reason about them. You know for a fact: "The only code I need to consider when working on this is what I have intentionally chosen to inject here".
Because even the content of injected modules is encapsulated behind the variable you assign it to, and all code executes inside a limited scope, surprises and collisions become very improbable. It's much, much less likely that something from one of your dependencies will accidentally redefine a global variable without you realizing it, or that you will do so. (It can happen, but you usually have to go out of your way to do it, with something like window.variable. The one accident that still tends to occur is assigning this.variable, not realizing that this is actually window in the current context.)
When you want to test an individual module, you're able to very easily know: exactly what else (dependencies) is affecting the code that runs inside the module? And, because you're explicitly injecting everything, you can easily mock those dependencies.
To me, the use of modules for front-end code boils down to: working in a much narrower context that's easier to reason about and test, and having greater certainty about what's going on.
It only takes about 30 seconds to learn how to use the CommonJS/Node module syntax. Inside a given JS file, which is going to be a module, you first declare any outside dependencies you want to use, like this:
var React = require('react');
Inside the file/module, you do whatever you normally would, and create some object or function that you'll want to expose to outside users, calling it perhaps myModule.
At the end of a file, you export whatever you want to share with the world, like this:
module.exports = myModule;
Then, to use a CommonJS-based workflow in the browser, you'll use tools like Browserify to grab all those individual module files, encapsulate their contents at runtime, and inject them into each other as needed.
AND, since ES6 modules (which you'll likely transpile to ES5 with Babel or similar) are gaining wide acceptance, and work both in the browser or in Node 4.0, we should mention a good overview of those as well.
More about patterns for working with modules in this deck.
EDIT (Feb 2017): Facebook's Yarn is a very important potential replacement/supplement for npm these days: fast, deterministic, offline package-management that builds on what npm gives you. It's worth a look for any JS project, particularly since it's so easy to swap it in/out.
EDIT (May 2019)
"Bower has finally been deprecated. End of story." (h/t: #DanDascalescu, below, for pithy summary.)
And, while Yarn is still active, a lot of the momentum for it shifted back to npm once it adopted some of Yarn's key features.
2017-Oct update
Bower has finally been deprecated. End of story.
Older answer
From Mattias Petter Johansson, JavaScript developer at Spotify:
In almost all cases, it's more appropriate to use Browserify and npm over Bower. It is simply a better packaging solution for front-end apps than Bower is. At Spotify, we use npm to package entire web modules (html, css, js) and it works very well.
Bower brands itself as the package manager for the web. It would be awesome if this was true - a package manager that made my life better as a front-end developer would be awesome. The problem is that Bower offers no specialized tooling for the purpose. It offers NO tooling that I know of that npm doesn't, and especially none that is specifically useful for front-end developers. There is simply no benefit for a front-end developer to use Bower over npm.
We should stop using bower and consolidate around npm. Thankfully, that is what is happening:
With browserify or webpack, it becomes super-easy to concatenate all your modules into big minified files, which is awesome for performance, especially for mobile devices. Not so with Bower, which will require significantly more labor to get the same effect.
npm also offers you the ability to use multiple versions of modules simultaneously. If you have not done much application development, this might initially strike you as a bad thing, but once you've gone through a few bouts of Dependency hell you will realize that having the ability to have multiple versions of one module is a pretty darn great feature. Note that npm includes a very handy dedupe tool that automatically makes sure that you only use two versions of a module if you actually have to - if two modules both can use the same version of one module, they will. But if they can't, you have a very handy out.
(Note that Webpack and rollup are widely regarded to be better than Browserify as of Aug 2016.)
Bower maintains a single version of modules, it only tries to help you select the correct/best one for you.
Javascript dependency management : npm vs bower vs volo?
NPM is better for node modules because there is a module system and you're working locally.
Bower is good for the browser because currently there is only the global scope, and you want to be very selective about the version you work with.
My team moved away from Bower and migrated to npm because:
Programmatic usage was painful
Bower's interface kept changing
Some features, like the url shorthand, are entirely broken
Using both Bower and npm in the same project is painful
Keeping bower.json version field in sync with git tags is painful
Source control != package management
CommonJS support is not straightforward
For more details, see "Why my team uses npm instead of bower".
Found this useful explanation from http://ng-learn.org/2013/11/Bower-vs-npm/
On one hand npm was created to install modules used in a node.js environment, or development tools built using node.js such Karma, lint, minifiers and so on. npm can install modules locally in a project ( by default in node_modules ) or globally to be used by multiple projects. In large projects the way to specify dependencies is by creating a file called package.json which contains a list of dependencies. That list is recognized by npm when you run npm install, which then downloads and installs them for you.
On the other hand bower was created to manage your frontend dependencies. Libraries like jQuery, AngularJS, underscore, etc. Similar to npm it has a file in which you can specify a list of dependencies called bower.json. In this case your frontend dependencies are installed by running bower install which by default installs them in a folder called bower_components.
As you can see, although they perform a similar task they are targeted to a very different set of libraries.
For many people working with node.js, a major benefit of bower is for managing dependencies that are not javascript at all. If they are working with languages that compile to javascript, npm can be used to manage some of their dependencies. however, not all their dependencies are going to be node.js modules. Some of those that compile to javascript may have weird source language specific mangling that makes passing them around compiled to javascript an inelegant option when users are expecting source code.
Not everything in an npm package needs to be user-facing javascript, but for npm library packages, at least some of it should be.
So to preface my question, I'm coming from a Java back-end developer perspective, where we use Maven to build. I have worked on testing on a server-side Node project we recently developed, but now I'm moving on to setting up testing on our front-end JavaScript client. I'm not very well-versed in front-end development and this is really my first foray into that.
All that being said, I'm thinking I'm going to use the following technology stack for our front-end testing: Eclipse IDE, Maven build process, Mocha testing framework, Chai assertion framework, Nock HTTP mocking framework, Sinon mocking/spying/stubbing framework, Rewire dependency injection framework. That all should be fine, and since that's the stack we use for our Node project, I would like to keep the front-end setup as similar as possible.
So, this is where my knowledge breaks down, though. I cannot seem to understand the difference in dependency management between our Node project and our front-end JavaScript project. I cannot see why I would not continue to use NPM, integrated into our Maven build, to handle dependency management and installation.
I see many people advocating Bower, but after looking at its page and docs, I still am not seeing what niche it fills that NPM would not. I am seeing a lot of adamant rejection of NPM insofar as browser dependency management goes; the main reasoning being that NPM is designed for server-side Node projects, and not for the front-end space. But who cares? Regardless of its initial design's intentions, if it does what I need it to do, where is the downside?
Please approach this "question" as if I am a complete newbie. I have realized as I've gone through this process and research, that I have very large dearths of knowledge in regards to the front-end side of things. With that being said, be as specific and thorough as possible in your answers, please. I would be happy to share project configuration and such, as needed, in order to help paint a picture of the space I'm in. Thanks for any feedback!
Bower and NPM do work differently.
NPM is very powerful and great at what it does. However, you won't find many client-side packages in there; most of them are there because they work in Node too. (For example, Underscore.JS and the JADE template engine.)
That's why you have Bower, which has the majority of the client-side packages. It has many jQuery plugins, templating engines, CSS frameworks, etc. Don't expect to find such packages in NPM.
You can work fine with both. I do so. :)
Traditionally, many web development projects combined npm and Bower. npm was used to manage back-end dependencies, while Bower was used for front-end dependencies. In fact, you needed to use npm in order to install Bower in the first place.
Although Bower’s advantages were compelling, they are now provided by other tools, namely npm, Yarn and webpack. While the open source project is still maintained, its creators decided to deprecate it, and advise how to migrate to other solutions—namely Yarn and webpack.
more explanation