I would like you to share your way of deploying complicated JS projects, where Grunt or Gulp are used.
For example, grunt build command concats css, js files, puts minified bower dependencies into dist folder. As far as i know, we should not store build results in version control repo, shouldn't we? Also, development environment is not needed on production server.
That is why flow like: git push production, and then grunt build on production server, then restarting it is not a good practice, isn't it?
The purpose o question is to find out, how should I deploy complicated JS projects when:
Building is necessary.
Building should not be done on production server.(Or it is a normal practice?)
Build results should not be indexed by version control system.
Deployment should not be done manually.
Related
I am writing a nodejs application with Angular as my front end.
For this I am having Git as my code management server.
For client, I am doing minification and it is ready for production.
But I am not sure, how to prepare server side files for production.
Do we need to just copy all the Git folders into production server?.
Let me know the best way to deploy nodejs server application.
You could use pm2 as your daemon to keep your nodejs app up all the time.
Try not to include node_modules in the repo, cause different machines have different setups/installations, you cannot tell if one package would work before you run it unless you npm install them.
If you are familiar with Docker, use it, pre-bundle all (include node_modules) files into the docker image, and you do not need pm2 here, Docker itself can restart automatically. This is the ideal approach.
It really depends on how you (or your company) want to organize the workflow and the size of the project.
Sometimes I too use a GIT repository, because then is really simple to update: just a git pull and (if server files got edits) a pm2 restart N command.
In this way, you dont have to install the whole development stack in order to compile (and minify) the bundles - I guess you work on your local machine where all the development tools are installed.
Keep in mind to use the --dev flag while installing packages that are only required in development mode, so you can keep the production server as slim as possible.
A good practice I found is to add some random tokens inside the final bundle filename (both for js and css) that get then injected inside the final html static files, to avoid the refresh the page loop.
Once you have the bundle files on your dev machine, just upload them to the server (ftp, git, rsync, sshfs mount, whatever you like) and (if server files got edits) restart/reload the node process (Im using pm2 for this, its really great). If you only edited client files, no reload is needed.
Starting from here, there a lot of ways more or less sophisticated to do the job, like git pipelines for example.. but depends on the situation.
Edit: this is a good article about task runner (gulp vs grunt vs vanilla npm), while may be a little off topic, it analyze some aspect of the common deployment process
I've got two projects which I've created:
A web UI built using webpack
A Vert.x webserver written in java built using Gradle
I want to find a way to bring the resulting build dir contents of the first project into the second as the webroot which will be server up using the StaticHandler.
Is anyone aware of a clean way to do this? I want to preserve the two git projects as they are because I like using the webpack dev server for development of the UI and it generally feels cleaner to have them separated.
I was looking at potentially using the bitbucket pipelines build on my repo, however bringing the assets generated by the first project into the build of the second is where I'm facing issues.
You could create a gradle task that before that depends on the jar task (so it runs before it) executes webpack compile into the resources directory. So when your jar task runs it bundles the compiled webpack code.
I'm a backend developer, and slightly confused by npm, bower, gulp, grunt, and Yeoman. Whenever I ask someone what their purpose is, the answer tends to boil down to dependency manager - for all of them. Surely, we don't need four different tools that all do the same?
Can someone please explain what each of these is good for in as few sentences as possible - if possible just one per tool, using language a five year old (with development skills) could understand?
For example:
SVN stores, manages, and keeps track of changes to our source code
I have used maven, Jenkins, nexus and ant in the past; maybe you could compare the tools above to these?
Also feel free to add other front-end tools to the list.
Here is what I have found out so far - not sure it's correct, though:
bower dependency manager for front-end development/JS libraries, uses a flat dependency list
npm dependency manager for node.js server, can resolve transitive dependencies/dependency trees
grunt runs tasks, much like Jenkins, but locality on the command line
Yeoman provided scaffolding, i.e skeleton projects
gulp same as grunt, but written in js only
node.js server for js apps?
git decentralized SCM/VCS, counterpart to svn/cvs
Am I close? :)
You are close!
Welcome to JavaScript :)
Let me give you a short description and one feature that most developers spend some time with.
bower
Focuses on packages that are used in the browser. Each bower install <packagename> points to exactly one file to be included for (more can be downloaded). Due to the success of webpack, browserify and babel it's mostly obsolete as a first class dependency manager.
2018 Update: bower is mostly deprecated in favour of NPM
npm
Historically focuses on NodeJS code but has overthrown bower for browser modules. Don't let anyone fool you: NPM is huge. NPM also loads MANY files into your project and a fresh npm install is always a good reason to brew a new cup of coffee. NPM is easy to use but can break your app when changing environments due to the loose way of referencing versions and the arbitrariness of module publishing. Research Shrink Wrap and npm install --save-exact
2018 Update: NPM grew up! Lot's of improvements regarding safety and reproducibility have been implemented.
grunt
Facilitates task automation. Gulps older and somewhat more sluggish brother. The JavaScript community used to hang out with him in 2014 a lot. Grunt is already considered legacy in some places but there is still a great amount of really powerful automation to be found. Configuration can be a nightmare for larger use-cases. There is a grunt module for that though.
2018 Update: grunt is mostly obsolete. Easy to write webpack configurations have killed it.
gulp
Does the same thing as grunt but is faster.
npm run-script
You might not need task runners at all. NodeJS scripts are really easy to write so most use-cases allow for customizedtask-automation workflow. Run scripts from the context of your package.json file using npm run-script
webpack
Don't miss out on webpack. Especially if you feel lost on the many ways of writing JavaScript into coherent modular code. Webpack packages .js files into modules and does so splendidly. Webpack is highly extensible and offers a good development environment too: webpack-dev-server
Use in conjunction with babel for the best possible JavaScript experience to date.
Yeoman
Scaffolding. Extremly valuable for teams with different backgrounds as it provides a controllable common ground for your projects architecture. There even is a scaffolding for scaffolds.
So, Since you have a good idea what each is, I will give you a simple workflow.
I use yeoman to scaffold a basic skeleton.
I am using node as the runtime for my application. ie. run node appname
I use npm to install node modules for helping me write the application in node
I might need some component from bower like front-end libraries so use bower to fetch these.
Now to do some repetitive task I will use grunt or gulp to write some tasks. So everytime I want to repeat it, say minimify my js files I call grunt/gulp and make them do it. Difference you ask, Gulp is stream based while grunt is task based.
I do version control using git to keep track of changes
Gulp vs Grunt: Gulp provides more flexibility with task automation, Grunt comes built in with a lot of functionality as per the common development practices. There are two main differences between Grunt and Gulp:
Grunt focuses on configuration, while Gulp focuses on code
Grunt was built around a set of built-in, and commonly used tasks, while Gulp came around with the idea of enforcing nothing, but how community-developed micro-tasks should connect to each other Read here
NodeJS: It is a Non-blocking server-side scripting language. Which means operations won't block further execution until current operation finishes.
Git: As you mentioned it is an SCM tool, a widely used one. As per the GitHub docs it is different from other SCM tools as data is never deleted.
Git thinks of its data more like a set of snapshots of a mini filesystem. Every time you commit, or save the state of your project in Git, it basically takes a picture of what all your files look like at that moment and stores a reference to that snapshot.
When you do actions in Git, nearly all of them only add data to the Git database. It is very difficult to get the system to do anything that is not undoable or to make it erase data in any way. As in any VCS, you can lose or mess up changes you haven’t committed yet; but after you commit a snapshot into Git, it is very difficult to lose, especially if you regularly push your database to another repository.
Read More
Bower vs NPM : Bower and NPM are dependency managers but Bower modules are for front-end development. NPM is a huge collection of modules to be used with NodeJS backend. This SO answer covers it better
I added some details:
npm is a package manager for javascript, npm is nodejs's package ecosystem, but it can be used only for front-end projects.
grunt & gulp are useful to separate and automate tasks like minification, compilation, unit testing on command line, it's a way lighter solution than (for example) visual studio as the process is only a separated (and usually lightweight) command line/process.
Regarding the differences between gulp, grunt and bower there is already a ticket: What are the differences between Grunt, Gulp.js and Bower? Why & when to use them?
Nodejs is more a javascript runtime. Node.js allows the creation of web servers and networking tools using js and a collection of "modules" that handle various core functionality and other core functions. Source
This ticket resumes the differences between Git and Subversion: Why is Git better than Subversion?
Question 1 :
I am installing my project dependency libraries using npm and it gets stored in the npm_modules folder. Is it necessary to keep the copy of library like angular.js,angular-route.js in lib folder or vendor folder? I could see few people are using lib folder or vendor folders to store the library in the permanent manner. I am confused by seeing this.
Question 2:
Do I need to copy/paste the node_modules folder to production or just run the npm install command on the project folder's command prompt to install all the dependencies in production. How does a dependency library get promoted to production?
Thank you kindly for your advice.
It all depends on how you need to deploy your site to production, really. Ultimately, you will probably want to bundle all your JS files into one or a few files, which are minified and sent with gzip compression.
How you bundle them is up to you. There are quite a few options:
Browserify
Webpack
Grunt / gulp build process
And many more besides
As to whether you need to keep a copy of these bundled javascript files under version control, well I think that boils down to 1 key question: can you run a build process (such as one of the tools using NodeJS) on the production server, or on a build server that creates a zip file or installer? If so, then you don't need to include them, just get the build server or production server to check out the latest copy from version control, npm install and then run the build process.
But if the best you could do is have the production server check files out from source control, then you would want to include the final versions of the files to use in the repository.
Keeping generated files, such as your bundled javascript files, in your source control repo should be avoided where possible. Because otherwise, every commit has to contain the changes to the source files, and the corresponding change to the generated files as well. And the latter is just noise, and has to be ignored by every developer looking at a diff/patch for a commit.
I've come my way from java development to development of complex Javascript projects. In Java build systems you don't checkin your dependencies (your jars) to sourcvode repository but let a build tool (maven, gradle, ...) manage that dependencies for you.
Now bower seems for me to be something like Maven to handle that dependency things for me. What I wonder about: Should I checkin the libs downloaded by bower or should I just checkin a script describing the dependencies I have and keep my repository free of thrid party code? Whats best practise? (And why?)
There are some pros and cons when you check the dependencies in your source control. The main reason for checking your dependencies would be to avoid issues while deploying your app. For example if Github is having network issues you can't deploy to production because your dependencies can't be fetched.
On the other hand the benefit of only having your bower.json file in your project instead of the whole bower_components folder is that it will keep your repository smaller. If you use CI, you can run bower install, run tests to see if everything works, and then deploy the exact copy in your production server.
Ultimately it is up to you to decide the best approach.