I have had some thoughts recently on how to handle shared javascript and css files across a web application.
In a current web application that I am working on, I got quite a large number of different javascripts and css files that are placed in an folder on the server. Some of the files are reused, while others are not.
In a production site, it's quite stupid to have a high number of HTTP requests and many kilobytes of unnecessary javascript and redundant css being loaded. The solution to that is of course to create one big bundled file per page that only contains the necessary information, which then is minimized and sent compressed (GZIP) to the client.
There's no worries to create a bundle of javascript files and minimize them manually if you were going to do it once, but since the app is continuously maintained and things do change and develop, it quite soon becomes a headache to do this manually while pushing out new updates that features changes to javascripts and/or css files to production.
What's a good approach to handle this? How do you handle this in your application?
I built a library, Combres, that does exactly that, i.e. minify, combine etc. It also automatically detects changes to both local and remote JS/CSS files and push the latest to the browser. It's free & open-source. Check this article out for an introduction to Combres.
I am dealing with the exact same issue on a site I am launching.
I recently found out about a project named SquishIt (see on GitHub). It is built for the Asp.net framework. If you aren't using asp.net, you can still learn about the principles behind what he's doing here.
SquishIt allows you to create named "bundles" of files and then to render those combined and minified file bundles throughout the site.
CSS files can be categorized and partitioned to logical parts (like common, print, vs.) and then you can use CSS's import feature to successfully load the CSS files. Reusing of these small files also makes it possible to use client side caching.
When it comes to Javascript, i think you can solve this problem at server side, multiple script files added to the page, you can also dynamically generate the script file server side but for client side caching to work, these parts should have different and static addresses.
I wrote an ASP.NET handler some time ago that combines, compresses/minifies, gzips, and caches the raw CSS and Javascript source code files on demand. To bring in three CSS files, for example, it would look like this in the markup...
<link rel="stylesheet" type="text/css"
href="/getcss.axd?files=main;theme2;contact" />
The getcss.axd handler reads in the query string and determines which files it needs to read in and minify (in this case, it would look for files called main.css, theme2.css, and contact.css). When it's done reading in the file and compressing it, it stores the big minified string in server-side cache (RAM) for a few hours. It always looks in cache first so that on subsequent requests it does not have to re-compress.
I love this solution because...
It reduces the number of requests as much as possible
No additional steps are required for deployment
It is very easy to maintain
Only down-side is that all the style/script code will eventually be stored within server memory. But RAM is so cheap nowadays that it is not as big of a deal as it used to be.
Also, one thing worth mentioning, make sure that the query string is not succeptible to any harmful path manipulation (only allow A-Z and 0-9).
What you are talking about is called minification.
There are many libraries and helpers for different platforms and languages to help with this. As you did not post what you are using, I can't really point you towards something more relevant to yourself.
Here is one project on google code - minify.
Here is an example of a .NET Http handler that does all of this on the fly.
Related
I've come across sites with CSS and JS filenames like this:
css_pbm0lsQQJ7A7WCCIMgxLho6mI_kBNgznNUWmTWcnfoE.css
What's causing this or why would you do it?
Edit: Some of each answer below could apply to this scenario, but given the sites I've found this on, serving/caching methods seems the most accurate.
Versioning and making sure that correct version of static resources is being served.
If you have a high traffic website and you serve lots of users you will have several layers of caching: CDN, caching headers on files, etc.
Sometimes it can be hard invalidating the caches with the same filename. Server might pass the correct headers, but client might disregard them and still load cached version. Serving different file name prevents that and ensures that you have correct version of css/js and other static resources.
As you can probably tell, no human came up with that name.
Typically it's
the result of combining multiple CSS files into a single file. This is
done for performance reasons (requesting one file is faster than requesting two.)
The name is likely to be the result of a deterministic algorithm on the
input (i.e. a hash), such that if you perform the combination again but haven't changed the CSS, the output will be given the same name.
When the content (CSS) changes, the name of the output file will change.
This is useful because it makes it impossible for a browser to cache
the old version.
It looks like the file was generated, server-side, for minification.
The website you're visiting might have had multiple CSS files (perhaps combined with #import statements) and JS files (jQuery, jQuery UI, jQuery plugins, some custom code, etc) - rather than have the developer manually minify and combine the files the server might do it for them (ASP.NET 4.5 does this, for example). In this case it uses an arbitrary (random? GUID-based?) filename to ensure it doesn't conflict with anything.
It may be the technology used by the website.
i.e. if you use gwt (it's some java compiled in javscript) or something else that preprocess some code and outputs javascript, you will likely to get weird filenames.
In my project each page has a bunch of dependent Javascript and Css. Whilst developing I just dumped this code right into the page but now I'm looking to clean it up...
it appears that the general approach out there is to package all the Javascript/CSS for an application into two big files that get minimised.
This approach has the benefit that it reduces bandwidth since all the front-end code gets pulled in just once from the server... however, I'm concerned I will be increasing the memory footprint of the application by defining a whole ton of functions for each page that I don't actually need - which is why I had them on a per-page basis to begin with.
is that something anyone else cares about or is there some way to manage this issue?
yes, I have thought of doing conditional function creation since I need to run code conditionally for each page anyway - though that starts to get a bit hackish in my view.
also, is there much cost to defining a whole ton of Css that is never used?
Serving the javascript/CSS in one big hit for the application, allows the browser to cache all it needs for all your pages. If the standard use case for your site is that users will stay and navigate around for a while then this is a good option to use.
If, however, you wish your landing page to load quickly, since there is a chance that the user will navigate away, consider only serving the CSS/javascript required for this page.
In terms of a performance overhead of a large CSS file - there will be none that is noticeable. All modern browsers are highly optimised for applying styles.
As for your javascript - try not to use conditional function creation, conditional namespace creation is acceptable and required, but your functions should be declared only in one place.
The biggest thing you can do for bandwidth is make sure your server is compressing output. Any static document type should be compressed (html, js, css, etc.).
For instance the jQuery Core goes from approx. 90KB to 30KB only because of the compressed output the server is sending to browsers.
If you take into account the compression, then you have to create some mammoth custom JS includes to really need to split-up your JS files.
I really like minifying and obfuscating my code because I can put my documentation right into the un-minified version and then the minification process removes all the comments for the production environment.
One approach would be to have all the shared javascript minified and compressed into one file and served out on each page. Then the page-specific javascript can be compressed/minified to its own files (although I would consider putting any very common page's javascript into the main javascript file).
I've always been in the habit of compressing/minifying all of the CSS into one file, rather than separate files for each page. This is because some of the page-specific files can be very small, and ideally we share as much css across the site as possible.
Like Jasper mentioned the most important thing would be to make sure that your sever is GZIPing the static resources (such as javascript and css).
If you have a lot of javascript code you can take a look on asynchronous loading of js files.
Some large project like ExtJs or Qooxdoo have build in loaders to load only required code, but here is a lot of libs which simplify this, and you can use in your project (e.g. head.js, LAB.js).
Thanks to them you can build application which loads only necessary files, not whole javascript code which in case of big apps can be a heavy stuff for browser.
In order to improve performance of our web pages, we are recommended to use CDNs to serve .js files on our web pages. That makes sense.
Also, we are recommended to bundle our .js files in order to reduce the number of requests which are being made to server on load.
So, we need to sit down and make decision between if we use CDN or bundle .js files.
What are the pros and cons? Which ones make more sense?
Why can't you bundle them and place them are the CDN? It should hardly be a decision of one or the other?
If you have to choose one or the other, it depends on how many .js files you are including. For a small number of files, I'd suggest that a CDN would be quicker, where-as for a greater number of files, a bundle of .js files would definitely be quicker. Where the switch-over would be, is something for you to experiment with.
My answer: both. Bundle them and place them on a CDN.
The downside of doing this? Depends. What does you build process look like? Can you easily automate the bundling and minification? Are you using Yahoo YUI or Google Closure or something else?
Also, if there is a lot of GUI dependent jQuery there might be some time consuming friction due to constantly changing elements/effects/css.
Testing is important too because due to possible minification quirks.
Bottom line: 5 javascript files safely bundled into 1 file === 4 fewer requests.
A page with just plain old Html and one external javascript reference === 2 requests to your server. However, a page with just plain old Html and one external javascript reference on a CDN === 1 request to your server.
Currently we are using the Google Closure tools. The Google Closure Inspector helps with the following:
Closure Compiler modifies your original JavaScript code and produces code that's smaller and more efficient than the original, but harder to read and debug. Closure Inspector helps by providing a source mapping feature, which identifies the line of original source code that corresponds to the compiled code.
As others have already stated, the answer is both if possible. Bundled (and minifying) gives a benefit to your users because it decreases the page weight. The CDN benefits your servers because you are offloading work. Generally speaking, you need not optimize either unless you have observed performance issues or you just have nothing better to do.
There's a few things you need to think about...
How much of the JS do you need to load early in the page load, and how much can you delay until later?
If you can delay loading JS (e.g. put it at the bottom of the page) or load it asynchronously as Google Analytics does, then you will minimise the amount of time downloading the JS spends blocking the UI thread.
After working out how the load of the JS can be split, I'd deal with the merge / minify of the various JS files - cutting down HTTP requests is key to improving performance.
Then look at moving to the CDN and ensure the CDN can serve the JS content compressed and allow you to set headers so it's "cached forever" (you'll need to version the files if you cache forever). A CDN helps reduce the latency but will also reduce size by being cookieless
Other thing you might want to consider is setting up a separate domain for static content, point it to your server(s) while you sort things out and then switch to a CDN if it looks worthwhile.
Andy
When dealing with websites with large amount of javascript, i see that these are still usually served to the client as one large javascript file.
In the development phase, are the javascript files usually split up (say there are >300 lines of js) to make things abit more manageable, and then merged when the website is 'put live'? Or do the developers just put up with working in one long large file?
We place different modules/classes/parts in separate files and use a proper build process to
validate the code using eg. jslint
concatenate
instrument (replace, wrap etc)
minify
An example of how to use Ant for this can be found in one of my projects here
http://github.com/oyvindkinsey/easyXDM/blob/master/build.xml.
I also have projects where the webserver automatically merges the files, localizes and then minifies them before serving the client.
So stick with whats manageable, using separate files, but do remember that if you use error reporting then the line numbers will point to the concatenated version.
In my experience — having separate files in development is the norm. It certainly makes life easier when you need to hunt for code or have multiple people working on different parts of the system.
It would be ideal to have multiple javascript files depending on the class and functionality (like you have them for java project ) in development environment.
However when you are deploying the js file in production, you should concatenate all js file in a single file and have them referred by your web application. That will make thing easy
Note: It would also be advisable to use javascript compressor to reduce the actual size and hence saving bandwidth.
Developement differs from company to company and from developer team to developer team.
I for myself am used to the approach of implementing functionality step by step, storing those functionalities in seperate files and merging everything together in most cases - at lease when i am not the only one working on a given project.
What are some standard practices for managing a medium-large JavaScript application? My concerns are both speed for browser download and ease and maintainability of development.
Our JavaScript code is roughly "namespaced" as:
var Client = {
var1: '',
var2: '',
accounts: {
/* 100's of functions and variables */
},
orders: {
/* 100's of functions and variables and subsections */
}
/* etc, etc for a couple hundred kb */
}
At the moment, we have one (unpacked, unstripped, highly readable) JavaScript file to handle all the business logic on the web application. In addition, there is jQuery and several jQuery extensions. The problem we face is that it takes forever to find anything in the JavaScript code and the browser still has a dozen files to download.
Is it common to have a handful of "source" JavaScript files that gets "compiled" into one final, compressed JavaScript file? Any other handy hints or best practices?
The approach that I've found works for me is having seperate JS files for each class (just as you would in Java, C# and others). Alternatively you can group your JS into application functional areas if that's easier for you to navigate.
If you put all your JS files into one directory, you can have your server-side environment (PHP for instance) loop through each file in that directory and output a <script src='/path/to/js/$file.js' type='text/javascript'> in some header file that is included by all your UI pages. You'll find this auto-loading especially handy if you're regularly creating and removing JS files.
When deploying to production, you should have a script that combines them all into one JS file and "minifies" it to keep the size down.
Also, I suggest you to use Google's AJAX Libraries API in order to load external libraries.
It's a Google developer tool which bundle majors JavaScript libraries and make it easier to deploy, upgrade and make them lighter by always using compressed versions.
Also, it make your project simpler and lighter because you don't need to download, copy and maintain theses libraries files in your project.
Use it this way :
google.load("jquery", "1.2.3");
google.load("jqueryui", "1.5.2");
google.load("prototype", "1.6");
google.load("scriptaculous", "1.8.1");
google.load("mootools", "1.11");
google.load("dojo", "1.1.1");
Just a sidenode - Steve already pointed out, you should really "minify" your JS files. In JS, whitespaces actually matter. If you have thousand lines of JS and you strip only the unrequired newlines you have already saved about 1K. I think you get the point.
There are tools, for this job. And you should never modify the "minified"/stripped/obfuscated JS by hand! Never!
In our big javascript applications, we write all our code in small separate files - one file per 'class' or functional group, using a kind-of-like-Java namespacing/directory structure. We then have:
A compile-time step that takes all our code and minifies it (using a variant of JSMin) to reduce download size
A compile-time step that takes the classes that are always or almost always needed and concatenates them into a large bundle to reduce round trips to the server
A 'classloader' that loads the remaining classes at runtime on demand.
For server efficiency's sake, it is best to combine all of your javascript into one minified file.
Determine the order in which code is required and then place the minified code in the order it is required in a single file.
The key is to reduce the number of requests required to load your page, which is why you should have all javascript in a single file for production.
I'd recommend keeping files split up for development and then create a build script to combine/compile everything.
Also, as a good rule of thumb, make sure you include your JavaScript toward the end of your page. If JavaScript is included in the header (or anywhere early in the page), it will stop all other requests from being made until it is loaded, even if pipelining is turned on. If it is at the end of the page, you won't have this problem.
Read the code of other (good) javascript apps and see how they handle things. But I start out with a file per class. But once its ready for production, I would combine the files into one large file and minify.
The only reason, I would not combine the files, is if I didn't need all the files on all the pages.
My strategy consist of 2 major techniques: AMD modules (to avoid dozens of script tags) and the Module pattern (to avoid tightly coupling of the parts of your application)
AMD Modules: very straight forward, see here: http://requirejs.org/docs/api.html also it's able to package all the parts of your app into one minified JS file: http://requirejs.org/docs/optimization.html
Module Pattern: i used this Library: https://github.com/flosse/scaleApp you asking now what is this ? more infos here: http://www.youtube.com/watch?v=7BGvy-S-Iag