I am working on a pretty large node web application and realize that my workflow is suboptimal. The server side code is quite well organized into node modules. However most of the heavy lifting in the application happens on the client side code.
The project consists of jade, less and javascript/coffeescript code. Furthermore there is a bunch of css/js includes from libraries (jQuery, D3, Bootstrap, etc). What I am looking for, but have a hard time to find, is a nice tool chain that helps me organize my project and switch between development and production:
Developing mode: compile jade, less and coffeescript on the fly in node without a need for rebuilding. Include compiled code into the html as separate files. Set some debugging parameters in the server side code.
Production mode: a build script (make/ant) that compiles compresses and combines the code into static html/css/js.
Currently I am doing a lot manually with some node server code for development, and a make file for production, e.g. things like:
cat javascript/*.js > build/client.js
However this is a bit annoying and I am not at all a make/ant expert. Also a source of problems has been that I am maintaining separate html files for development and production, because they include different css/js files.
Is there any good frameworks or example projects that can help me improve my workflow? Google gave me things like hem, buildr.npm, grunt, but I am not quite sure how mature these projects are, and if they really solve my problem or add more complexity.
You can also have a look at brunch, with many plugin support
As per the website it supports the following, you can also add a new plugin if you need to...
Programming languages: JavaScript, CoffeeScript, IcedCoffeeScript, LiveScript, Roy
Pre-compiled templates: Handlebars.js, Jade, Mustache, Eco
Styles: CSS, Stylus, LESS, Sass
Linters: JSHint, CoffeeLint
Minifiers: uglify.js, clean-css
I would take a look at grunt. We use it to handle the deployment and testing of our projects.
Related
I'm wondering whether there is a nice way to continuously bundle complex javascript files in a single, customized, JS file that is then directly served to clients, sort of a "smart" CDN.
My use case is really simple: serve JS files to different web applications from a microservice, like RUM scripts (or also Google AdSense scripts, or anything else that works by importing a JS file from a service in your web application).
Problem is, at every request the JS file will be different, based on several parameters (for example, IP address of the request, query params in the URI, back-end configuration, and so on).
Solution can be trivial, for example I could manually bundle JS files by concatenating strings, and then serve the resulting file, with minimal logic or with the help of a standard template engine that handles placeholders/pre-processing within the source JS files.
I was wondering if there is a better way to do this, like using JS bundlers as Webpack or Parcel.js.
My experience in using JS bundlers is limited to their standard usage, so that use them in the build process (or at application startup), and make them write to disk the compiled JS file.
Is there anything better to use, or a best practice to suggest for my use case?
Programming language makes no difference, I added the node.js tag because ideally the backend would be developed using Express.js, but also Java or other compiled languages would be ok.
Thanks in advance,
Sure, that's definitely possible.
Esbuild (a new, fast bundler) has a well-documented Node.js API: https://esbuild.github.io/api/#js-specific-details
The two other bundlers you mention are similarly available:
Webpack has a well-documented Node.js API: https://webpack.js.org/api/node/
Parcel has a well-documented Node.js API: https://parceljs.org/api.html
Then it's just (always "just" ;-) ) a matter of hooking those up to the web framework of your choice and going to town.
I'd recommend thinking about caching in advance, though – even if e.g. Esbuild and Parcel are pretty fast, not doing work is always faster than doing work.
I'm looking for a solution to have server settings in a config file of some sort separate from the JS app (which I could be running in dev or prod (built) mode) but allow for the server to replace the file with a new one at random point in time (when settings have changed) and allow the JS app to pick it up and refresh it's settings.
This is mainly for setting URLs to API endpoints.
My choice of JS framework is Angular 1 and 4 but answers geared generally towards various JS frameworks/libraries are applicable.
to have server settings in a config file of some sort separate from the JS app
If you are thinking about having your server settings exposed to the front you are gonna have a bad time ;)
I will assume that you think you will keep API URLs and maybe some config along with it in a separate file while you develop. Then I can advise two approaches:
1.) SIMPLE - Have a single JS file with a single object with all the necessary API URLs and settings in a tree like structure for organisational purposes. So you have a dev, a test and production property with everything in them. Once you load app 1st thing that should the app do is load the correct settings. This is fast and doesn't require any additional knowledge. Everywhere you had something configurable hard coded you replace it with a variable that derives from that file. This should suffice for simple applications.
2.) POWERFULL - For bigger applications with loads of different dependencies and more formal and professional approach do what most people do, use NodeJs, leverage Grunt or Gulp to help you version dependencies and write scripts for different builds. It requires you to install and learn to use new stuff but offers you way more power.
Even better combine this two approaches, link the external dependencies dynamically with the build and internal via including the correct JS file with all the API information.
You can also use other npm modules to assist you with your development like precompiling CSS, lint your code, rearrange and organise files by types into folders, auto-name files or folders.
Basically, MOST of the REPETITIVE tasks you do while developing can be automated this way. This truly shines on bigger projects when all this is handled by someone who is proficient and experienced with the 2nd approach, it can save months of repetitive tasks across the team.
You could use gulp (or even npm) as a task runner to do this kind of work. Maybe this article would help? https://www.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool/
The main point of a task runner is that you can set a specific variable ie prod or development, and the task runner can then switch out your config files with the present definitions for prod or dev config.
I'm creating a little SPA framework (might be a full-feature framework).
Frontend is based on native javascript (including ES6 features and web-components).
Backend is written in Go.
Here is a list of my needs, that I'd like to do automatically:
Minify my javascript files
Transpile ES6 code to ES5 (with something like babel.js)
Polyfill my web-components
Is there a way to achieve this without using node.js? Are there already go packages that can make those 3 things happen, with a "simple" grunt/gulp-like way?
I would like to avoid installing node.js, npm, bower... etc.
In my (not very classified) oppinion, using those node.js tools makes my application a mess adding a lot of unnecessery overhead to my application folder and makes my framework dependant on a lot of stuff (that application programmer has to learn, understand and know how to use).
Thanks for any tips/oppinions.
If you're doing a SPA you should just stick with gulp and others. You'll probably end up using npm stuff like browserify, autoprefix, etc, and you'll basically end up reinventing the wheel with your own asset pipeline.
With that said I've seen a couple, but none of them looked very mature last time I looked:
https://github.com/jbowens/assets
https://github.com/shaoshing/train
and you're going to need npm installed to use them regardless.
In order to build static web pages to serve interactive news applications (mostly data visualizations scripted with something like D3 or lightweight apps built with Bootstrap.js), is it worth the effort to install and configure a build system like Middleman or Grunt.js?
The reason why I am asking is because I am not familiar with Ruby and Ruby Gems at all, and do not have the time and willingness at the moment to dwelve deeper into it, as I already know Python & Javascript pretty well.
Do I need to know Ruby in order to use Middleman productively? Or should I just use Grunt? I am asking because I want to reuse the same templates over and over again and heard that this is easy with Middleman.
There is a distinct difference between Middleman and Grunt.
Grunt is a generic task runner while Middleman is a static site generator.
Middleman for example would be used to create a site with multiple URLs. You put some files in a folder and Middleman takes care of arranging them, creating subfolders for URLs, rendering layouts, compressing assets and in the end you have a bunch of files that represent the site. All this is built in.
You don’t necessarily need to know a lot of Ruby to get started with Middleman, but to get the most out of it, understand how it works and change configurations to your needs, basic Ruby knowledge is key.
Grunt on the other hand works by using plugins and telling them exactly what to do. By default it does nothing. You could think of Grunt as "framework", ready for you to add tasks. Compress these two css files. Combine these JS files and move theme over here. Things like that.
For building a complete website I recommend Middleman. In order to be able to use the templates you create in another language like JS, you could use Jade as a templating language. By default Middleman uses ERB (part of the Ruby standard library). Another popular choice is HAML, but I’m not sure if there is a decent JS compiler for it.
If you want to have really fine grained control over the output of your JS files (you’re talking about JS client side apps in your question) you can even use Grunt and Middleman together. Grunt would take care of the assets (you can have really detailed configurations there) and Middleman would handle templating, URL generation and all the other "default website related" stuff.
If you're keen to use Grunt as a base for this sort of build, you may like to look at Assemble.io.
This is a static-site generator like Middleman, but one that is completely (currently) dependant upon Grunt tasks in order to do its thing.
I believe the learning curve for Assemble.io is rather higher than Middleman. It is based on Node, rather than Ruby. However I don't believe it's necessary to know Ruby in order to get a lot from Middleman.
You may have a look to http://wintersmith.io/ a static page generator like Jekill or MiddleMan (I don't really know this one)
With Grunt you have something at a lower level: Grunt provides you tasks to automate things so you'll have to find or write the task that fits your needs.
I am in the process of writing a heavy Javascript app, which will ultimately be used by injecting one script into clients websites.
As of now I am writing all the modules in one JS file, however I am quickly finding that to be ineffective, as it feels very messy and cluttered, and I feel like the modules should all be in separate files.
My question is, what is a good approach to managing this. Should I write all the apps modules in separate files, and than compile them into one on the Server?
If it matters, I am using Node.js for my server.
First point : don't try to code everything in one file. Most big javascript applications contain dozens of files.
Use some kind of makefile to concatenate your js (and css) files. And after that use a minifier (I use Google Closure Compiler). To help debug, my deployement scripts always make two versions in parallel : one non concatenated/minified and one concatenated/minified. The uncompressed version enables the development/test onsite without any deployement operation.
This means that, as for all big application development, you need some kind of deployement toolchain to orchestrate the operations. This may be based on shell scripts, maven, ant, etc.
Secondly : use classes (without abuse, javascript isn't really OOP) and namespaces to clearly isolate your functions.
Yes, keep all your files logically separate and then minify and combine them as a publish step or on the fly when serving them. Scott Hansleman wrote a very good blog post on why you should do this here.