I've come my way from java development to development of complex Javascript projects. In Java build systems you don't checkin your dependencies (your jars) to sourcvode repository but let a build tool (maven, gradle, ...) manage that dependencies for you.
Now bower seems for me to be something like Maven to handle that dependency things for me. What I wonder about: Should I checkin the libs downloaded by bower or should I just checkin a script describing the dependencies I have and keep my repository free of thrid party code? Whats best practise? (And why?)
There are some pros and cons when you check the dependencies in your source control. The main reason for checking your dependencies would be to avoid issues while deploying your app. For example if Github is having network issues you can't deploy to production because your dependencies can't be fetched.
On the other hand the benefit of only having your bower.json file in your project instead of the whole bower_components folder is that it will keep your repository smaller. If you use CI, you can run bower install, run tests to see if everything works, and then deploy the exact copy in your production server.
Ultimately it is up to you to decide the best approach.
Related
What exactly should I put in .npmignore?
Tests? Stuff like .travis.yml, .jshintrc? Anything that isn't needed when running the module (except the readme)?
I can't find any guidance on this.
As you probably found, NPM doesn't really state specifically what should go in there, rather they have a list of ignored-by-default files. Many people don't even use it as everything in your .gitignore is ignored in npm by default if .npmignore doesn't exist. Additionally, many files are already ignored by default regardless of settings and some files are always excluded from being ignored, as outlined in the link above.
There is not much official on what always should be there because it is basically a subset of .gitignore, but from what I gather from using node for 5-ish years, here's what I've come up with.
Note: By production I mean any time where your module is used by someone and not to develop on the module itself.
Pre-release cross-compiled sources
Pros: If you are using a language that cross-compiles into JavaScript, you can precompile before release and not include .coffee files in your package but keep tracking them in your git repository.
Build file leftovers
Pros: People using things like node-gyp might have object files that get generated during a build that never should go into the package.
Cons: This should always go into the .gitignore anyway. You must place these things inside here if you are using a .npmignore file already as it overrides .gitignore from npm's point of view.
Tests
Pros: Less baggage in your production code.
Cons: You cannot run tests on live environments in the slim chance there is a system-specific failure, such as an out of date version of node running that causes a test to fail.
Continuous integration settings/Meta files
Pros: Again, less baggage. Things such as .travis.yml are not required for using, testing, or viewing the code.
Non-readme docs and code examples
Pros: Less baggage. Some people exist in the school-of-thought where if you cannot express at least minimum viable functionality in your Readme, your module is too big.
Cons: People cannot see exhaustive documentation and code examples on their own file system. They would have to visit the repository (which also requires an internet connection).
Github-pages objects
Pros: You certainly don't need to litter your releases with CNAME files or placeholder index.htmls if you use your module serves double-duty as a gh-pages repository as well.
bower.json and friends
Pros: If you decide to build in your dependencies prior to release, you don't need the end-user to install bower then install more things with that. I would, personally, keep that stuff in the package. When I do an npm install, I should only be relying on npm and no other external sources.
Basically, you should ever use it if there is something you wish to keep out of your npm package but checked-in to your module's repo. It's not a long list of items, but npm would rather build in the functionality than having people stuck with irrelevant objects in their package.
I agree with lante's short and syntetic answer and SamT's big answer:
You should not include your tests in your package.
Your package should only contains production runtime files.
That will make your package more straightforward and faster to be dowloaded.
My contribution to those answers:
.npmignore is the blacklist way to achieve package file selection. But in a more practical way, you can whitelist files you need to include in your package using the files field in your package.json:
{
"files": [
"lib/",
"index.js"
]
}
I think that's simpler, future proof and have better semantics ;)
Just to clarify, anytime someone do npm install your-library, npm will download all source files that the package includes. Those files that were included in the .npmignore file in the source code of the package your-library will be excluded when publishing the lib, so users of your-library won't download them.
Know that people installing your library will need just your library running, anything else will be not necessary.
For example, when someone installs a library, its probably that he/she doesn't care about your .travis.yml or your .jshintrc files, or even some images, Grunt files, documentation, etc.
.npmignore could let your npm package to have less files, and faster to be downloaded
Don't include your tests. Oftentimes tests are like 5x the size of the actual codebase. As long as your tests are on Github, etc, that's good enough.
But what you absolutely should do is test your NPM package in its published format. Create some smoke tests that reside in the actual codebase, but are not part of the test suite.
You can read about testing your package after tarballing it, here:
https://github.com/ORESoftware/r2g
How to test an `npm publish` result, without actually publishing to NPM?
We have built a small-ish application using Aurelia, and would like to be able to integrate the app into a larger codebase. For example, it may be desirable to publish the Aurelia app on NPM, so other projects could integrate our code.
How can we build/publish the Aurelia app, so that it can be instantiated in a larger JavaScript application?
Typically, you wouldn't publish an entire Aurelia application to NPM, rather you would push plugins to NPM.
The simplest way to learn how to do this is to follow the example of the various Aurelia repos. We build the code in to multiple formats, but the most important are AMD and CommonJS (which is more important for your company is dependent on what tooling your company is using). Once you've built the code, the next step is to make sure your package.json file is set up correctly. It's best to copy the example of what we're doing in our own repos.
I would start by looking at either the dialog plugin or the templating-resources plugin. Both of these show good examples of how to publish your Aurelia plugin to NPM, whether it is a public or private npm feed.
https://github.com/aurelia/dialog/blob/master/package.json
https://github.com/aurelia/templating-resources/blob/master/package.json
Both of these plugins are set up to support Webpack, JSPM/SystemJS, and the Aurelia CLI out of the box.
I totally agree with #Ashley in not publishing larger applications to the global NPM registry. The big advantage of it is the simplicity of all that small packages which can be used to build large applications.
If you feel you got some reusable code, put in an own package and publish it.
Anyway, to give you an answer which does not require you to publish your complete app: you can include a complete repository (which is probobly what you are lookig for) in a different application via NPM.
npm install git://github.com/somegit/hubrepo.git
Or directly via the package.json:
"dependencies": {
"private-repo": "git+ssh://git#github.com:somegit/hubrepo.git#v1.0.0",
}
You can also do something similiar e.g. using JSPM. This would be simply:
jspm install your-app=github:OrgOrUser/your-app#your-branch
If you are facing long relative paths in your imports now, depending on your tooling, you may be interested in something like e.g. Resolving require paths with webpack which shows how to alias relative paths.
Sources/Links:
How to install a private NPM module without my own registry?
npm install private github repositories by dependency in package.json
I'm a backend developer, and slightly confused by npm, bower, gulp, grunt, and Yeoman. Whenever I ask someone what their purpose is, the answer tends to boil down to dependency manager - for all of them. Surely, we don't need four different tools that all do the same?
Can someone please explain what each of these is good for in as few sentences as possible - if possible just one per tool, using language a five year old (with development skills) could understand?
For example:
SVN stores, manages, and keeps track of changes to our source code
I have used maven, Jenkins, nexus and ant in the past; maybe you could compare the tools above to these?
Also feel free to add other front-end tools to the list.
Here is what I have found out so far - not sure it's correct, though:
bower dependency manager for front-end development/JS libraries, uses a flat dependency list
npm dependency manager for node.js server, can resolve transitive dependencies/dependency trees
grunt runs tasks, much like Jenkins, but locality on the command line
Yeoman provided scaffolding, i.e skeleton projects
gulp same as grunt, but written in js only
node.js server for js apps?
git decentralized SCM/VCS, counterpart to svn/cvs
Am I close? :)
You are close!
Welcome to JavaScript :)
Let me give you a short description and one feature that most developers spend some time with.
bower
Focuses on packages that are used in the browser. Each bower install <packagename> points to exactly one file to be included for (more can be downloaded). Due to the success of webpack, browserify and babel it's mostly obsolete as a first class dependency manager.
2018 Update: bower is mostly deprecated in favour of NPM
npm
Historically focuses on NodeJS code but has overthrown bower for browser modules. Don't let anyone fool you: NPM is huge. NPM also loads MANY files into your project and a fresh npm install is always a good reason to brew a new cup of coffee. NPM is easy to use but can break your app when changing environments due to the loose way of referencing versions and the arbitrariness of module publishing. Research Shrink Wrap and npm install --save-exact
2018 Update: NPM grew up! Lot's of improvements regarding safety and reproducibility have been implemented.
grunt
Facilitates task automation. Gulps older and somewhat more sluggish brother. The JavaScript community used to hang out with him in 2014 a lot. Grunt is already considered legacy in some places but there is still a great amount of really powerful automation to be found. Configuration can be a nightmare for larger use-cases. There is a grunt module for that though.
2018 Update: grunt is mostly obsolete. Easy to write webpack configurations have killed it.
gulp
Does the same thing as grunt but is faster.
npm run-script
You might not need task runners at all. NodeJS scripts are really easy to write so most use-cases allow for customizedtask-automation workflow. Run scripts from the context of your package.json file using npm run-script
webpack
Don't miss out on webpack. Especially if you feel lost on the many ways of writing JavaScript into coherent modular code. Webpack packages .js files into modules and does so splendidly. Webpack is highly extensible and offers a good development environment too: webpack-dev-server
Use in conjunction with babel for the best possible JavaScript experience to date.
Yeoman
Scaffolding. Extremly valuable for teams with different backgrounds as it provides a controllable common ground for your projects architecture. There even is a scaffolding for scaffolds.
So, Since you have a good idea what each is, I will give you a simple workflow.
I use yeoman to scaffold a basic skeleton.
I am using node as the runtime for my application. ie. run node appname
I use npm to install node modules for helping me write the application in node
I might need some component from bower like front-end libraries so use bower to fetch these.
Now to do some repetitive task I will use grunt or gulp to write some tasks. So everytime I want to repeat it, say minimify my js files I call grunt/gulp and make them do it. Difference you ask, Gulp is stream based while grunt is task based.
I do version control using git to keep track of changes
Gulp vs Grunt: Gulp provides more flexibility with task automation, Grunt comes built in with a lot of functionality as per the common development practices. There are two main differences between Grunt and Gulp:
Grunt focuses on configuration, while Gulp focuses on code
Grunt was built around a set of built-in, and commonly used tasks, while Gulp came around with the idea of enforcing nothing, but how community-developed micro-tasks should connect to each other Read here
NodeJS: It is a Non-blocking server-side scripting language. Which means operations won't block further execution until current operation finishes.
Git: As you mentioned it is an SCM tool, a widely used one. As per the GitHub docs it is different from other SCM tools as data is never deleted.
Git thinks of its data more like a set of snapshots of a mini filesystem. Every time you commit, or save the state of your project in Git, it basically takes a picture of what all your files look like at that moment and stores a reference to that snapshot.
When you do actions in Git, nearly all of them only add data to the Git database. It is very difficult to get the system to do anything that is not undoable or to make it erase data in any way. As in any VCS, you can lose or mess up changes you haven’t committed yet; but after you commit a snapshot into Git, it is very difficult to lose, especially if you regularly push your database to another repository.
Read More
Bower vs NPM : Bower and NPM are dependency managers but Bower modules are for front-end development. NPM is a huge collection of modules to be used with NodeJS backend. This SO answer covers it better
I added some details:
npm is a package manager for javascript, npm is nodejs's package ecosystem, but it can be used only for front-end projects.
grunt & gulp are useful to separate and automate tasks like minification, compilation, unit testing on command line, it's a way lighter solution than (for example) visual studio as the process is only a separated (and usually lightweight) command line/process.
Regarding the differences between gulp, grunt and bower there is already a ticket: What are the differences between Grunt, Gulp.js and Bower? Why & when to use them?
Nodejs is more a javascript runtime. Node.js allows the creation of web servers and networking tools using js and a collection of "modules" that handle various core functionality and other core functions. Source
This ticket resumes the differences between Git and Subversion: Why is Git better than Subversion?
I would like you to share your way of deploying complicated JS projects, where Grunt or Gulp are used.
For example, grunt build command concats css, js files, puts minified bower dependencies into dist folder. As far as i know, we should not store build results in version control repo, shouldn't we? Also, development environment is not needed on production server.
That is why flow like: git push production, and then grunt build on production server, then restarting it is not a good practice, isn't it?
The purpose o question is to find out, how should I deploy complicated JS projects when:
Building is necessary.
Building should not be done on production server.(Or it is a normal practice?)
Build results should not be indexed by version control system.
Deployment should not be done manually.
So to preface my question, I'm coming from a Java back-end developer perspective, where we use Maven to build. I have worked on testing on a server-side Node project we recently developed, but now I'm moving on to setting up testing on our front-end JavaScript client. I'm not very well-versed in front-end development and this is really my first foray into that.
All that being said, I'm thinking I'm going to use the following technology stack for our front-end testing: Eclipse IDE, Maven build process, Mocha testing framework, Chai assertion framework, Nock HTTP mocking framework, Sinon mocking/spying/stubbing framework, Rewire dependency injection framework. That all should be fine, and since that's the stack we use for our Node project, I would like to keep the front-end setup as similar as possible.
So, this is where my knowledge breaks down, though. I cannot seem to understand the difference in dependency management between our Node project and our front-end JavaScript project. I cannot see why I would not continue to use NPM, integrated into our Maven build, to handle dependency management and installation.
I see many people advocating Bower, but after looking at its page and docs, I still am not seeing what niche it fills that NPM would not. I am seeing a lot of adamant rejection of NPM insofar as browser dependency management goes; the main reasoning being that NPM is designed for server-side Node projects, and not for the front-end space. But who cares? Regardless of its initial design's intentions, if it does what I need it to do, where is the downside?
Please approach this "question" as if I am a complete newbie. I have realized as I've gone through this process and research, that I have very large dearths of knowledge in regards to the front-end side of things. With that being said, be as specific and thorough as possible in your answers, please. I would be happy to share project configuration and such, as needed, in order to help paint a picture of the space I'm in. Thanks for any feedback!
Bower and NPM do work differently.
NPM is very powerful and great at what it does. However, you won't find many client-side packages in there; most of them are there because they work in Node too. (For example, Underscore.JS and the JADE template engine.)
That's why you have Bower, which has the majority of the client-side packages. It has many jQuery plugins, templating engines, CSS frameworks, etc. Don't expect to find such packages in NPM.
You can work fine with both. I do so. :)
Traditionally, many web development projects combined npm and Bower. npm was used to manage back-end dependencies, while Bower was used for front-end dependencies. In fact, you needed to use npm in order to install Bower in the first place.
Although Bower’s advantages were compelling, they are now provided by other tools, namely npm, Yarn and webpack. While the open source project is still maintained, its creators decided to deprecate it, and advise how to migrate to other solutions—namely Yarn and webpack.
more explanation