compressing entire webpages (HTML and JS) - javascript

I have found some tools like this one that let me create "auto-extracting" javascript for javascript code in a web page, which employ a variety of techniques to minimize transfer size.
I have a webpage which does have a rather large chunk of javascript code in it. But since I haven't gotten around to optimizing the filesize yet I was thinking about doing the same sort of thing with the HTML bits of my website too. On my blog page the PHP script pulls HTML snippets from a large number of text files, and concatenates them into one giant HTML file which is sent out. Chrome tells me that compressing it with gzip would reduce the filesize by two-thirds.
However I did turn off the gzip compression because what was happening was if you downloaded any of my zip archives I hosted via Internet Explorer, it would herp derp neglect to gunzip them so the file downloaded by IE is always corrupted. I guess I can turn gzip back on if I fix this little issue, but for the time being I'd like to try to see if I can make a self-extracting HTML page. Is it possible to have javascript extract a giant HTML string and add the entire chunk as child of the body element? Would that work?

It will be slower to do that, and very error prone. Any Javascript error will cause the entire page to not render, and your SEO will be absolutely destroyed. Stick to regularly rendered HTML: as the browser is downloading / parsing the HTML, it will begin fetching other resources (images, scripts, css) and rendering the layout. Don't be so focused on strictly smallest download size, but rather quickest overall experience.
Make heavy use of the freely available CDNs. There are the big two: Google and Microsoft, that host a variety of scripts like jQuery and Modernizr. Stick to Google where possible, they seem to have a much higher adoption than Microsoft and thusly a higher chance of a warm cache. Past that, use CDNJS for other publically-available libraries -- they have a lot.
Minify your existing Javascript, and enable content compression for static and dynamic pages. Don't force it on, let the browser request it. What version of IE are you seeing corruption on? I haven't seen that be an issue since IE6...
Using the Javascript packers will make your site appear slower at the expense of saving a few more bytes of transfer on your end. Not only does a script have to run, but now you're asking the users' browser to perform an additional (potentially large) extra step before your scripts can run.
If you're trying to download individual files (with the Save-As dialog), you can't use gzip with a content type of 'application/zip'. The actual Zip format is available with PHP, use those libraries instead.

As a quick win Cloudflare has an auto minify feature for HTML, JS and CSS. We've been using it for a little while now with good results. Definitely worth a look.

Related

Minification for Css/Js - right way?

In my project each page has a bunch of dependent Javascript and Css. Whilst developing I just dumped this code right into the page but now I'm looking to clean it up...
it appears that the general approach out there is to package all the Javascript/CSS for an application into two big files that get minimised.
This approach has the benefit that it reduces bandwidth since all the front-end code gets pulled in just once from the server... however, I'm concerned I will be increasing the memory footprint of the application by defining a whole ton of functions for each page that I don't actually need - which is why I had them on a per-page basis to begin with.
is that something anyone else cares about or is there some way to manage this issue?
yes, I have thought of doing conditional function creation since I need to run code conditionally for each page anyway - though that starts to get a bit hackish in my view.
also, is there much cost to defining a whole ton of Css that is never used?
Serving the javascript/CSS in one big hit for the application, allows the browser to cache all it needs for all your pages. If the standard use case for your site is that users will stay and navigate around for a while then this is a good option to use.
If, however, you wish your landing page to load quickly, since there is a chance that the user will navigate away, consider only serving the CSS/javascript required for this page.
In terms of a performance overhead of a large CSS file - there will be none that is noticeable. All modern browsers are highly optimised for applying styles.
As for your javascript - try not to use conditional function creation, conditional namespace creation is acceptable and required, but your functions should be declared only in one place.
The biggest thing you can do for bandwidth is make sure your server is compressing output. Any static document type should be compressed (html, js, css, etc.).
For instance the jQuery Core goes from approx. 90KB to 30KB only because of the compressed output the server is sending to browsers.
If you take into account the compression, then you have to create some mammoth custom JS includes to really need to split-up your JS files.
I really like minifying and obfuscating my code because I can put my documentation right into the un-minified version and then the minification process removes all the comments for the production environment.
One approach would be to have all the shared javascript minified and compressed into one file and served out on each page. Then the page-specific javascript can be compressed/minified to its own files (although I would consider putting any very common page's javascript into the main javascript file).
I've always been in the habit of compressing/minifying all of the CSS into one file, rather than separate files for each page. This is because some of the page-specific files can be very small, and ideally we share as much css across the site as possible.
Like Jasper mentioned the most important thing would be to make sure that your sever is GZIPing the static resources (such as javascript and css).
If you have a lot of javascript code you can take a look on asynchronous loading of js files.
Some large project like ExtJs or Qooxdoo have build in loaders to load only required code, but here is a lot of libs which simplify this, and you can use in your project (e.g. head.js, LAB.js).
Thanks to them you can build application which loads only necessary files, not whole javascript code which in case of big apps can be a heavy stuff for browser.

Put CSS and JavaScript in files or main HTML?

Although it is always recommended to put JavaScript and CSS code into appropriate files (as .js and .css), most of major websites (like Amazon, facebook, etc.) put a significant part of their JavaScript and CSS code directly within the main HTML page.
Where is the best choice?
Place your .js in multiple files, then package, minify and gzip that into one file.
Keep your HTML into multiple seperate files.
Place your .css in multiple files, then package, minify and gzip that into one file.
Then you simply send one css file and one js file to the client to be cached.
You do not inline them with your HTML.
If you inline them then any change to the css or html or js forces to user to download all three again.
The main reason major websites have js & cs in their files, is because major websites code rot. Major companies don't uphold standards and best practices, they just hack it until it works then say "why waste money on our website, it works doesn't it?".
Don't look at examples of live websites, because 99% of all examples on the internet show bad practices.
Also for the love of god, Separation of concerns please. You should never ever use inline javascript or inline css in html pages.
http://developer.yahoo.com/performance/rules.html#external
Yahoo (even though they have many inline styles and scripts), recommends making them external. I believe google page speed used to (or still does?) do the same as well.
It's really a logical thing to have them separate. There are so many benefits to keeping CSS and JS separate to the HTML. Things like logical code management, caching of those pages, lower page size (would you rather a ~200ms request for a 400kb cached resource, or a 4000ms delay from having to download that data on every page?), SEO options (less crap for google to look through when scripts/styles are external), easier to minify external scripts (online tools etc), can load them synchronously from different servers....
That should be your primary objective in any website. All styles that make up your whole website should be in the one file (or files for each page, then merged and minified when updated), the same for javascript.
In the real world (not doing a project for yourself, doing one for a client or stakeholder that wants results), the only time where it doesn't make sense to load in another javascript resource or another stylesheet (and thus use inline styles/javascript) is if there's some kind of dynamic information that is on a kind of per-user, per-session or per-time-period that can't be accomplished as simply any other way. Example: when my website has a promotion, we dump a script tag with a small JSON object of information. Because we don't minify and merge multiple files, it makes more sense to just include it in the page. Sure there are other ways to do this, but it costs $20 to do that, whereas it could cost > $100 to do it another way.
Perhaps Amazon/Facebook/Google etc use so much inline code is so their servers aren't taxed so much. I'm not too sure on the benchmarking between requesting a 1MB file in one hit or requesting 10 100KB files (presuming 1MB/10 = 100KB for examples' sake), but what would be faster? Potentially the 1MB file, BUT smaller requests can be loaded synchronously, meaning each one of those 10 requests could come from a separate server/domain potentially, thus reducing overall load time.
Further, google homepages for example seem to dump a JSON array of information for the widgets, presumably because it compiles all that information from various sources, minifies it, caches it, then puts in on the page, then the javascript functions build the layout (client side processing power rather than server-side).
An interesting investigation might be whether they include various .css files regardless of the style blocks you're also seeing. Perhaps it's overhead or perhaps it's convenience.
I've found that while working with different styles of interface developer (and content deployers) that convenience/authority often wins in the face of deadlines and "getting the job done". In a project of a large scale there could be factors involved like "No, you ain't touching our stylesheets", or perhaps if there isn't a stylesheet using an http request already then convenience has won a battle against good practice.
If your css and javascript code is for a global usage, then it is best to put them into appropriate files.
Otherwise, if the code is used just by a certain page, like the home page, put them directly into html is acceptable, and is good for maintenance.
Our team keeps it all seperate. All resources like this goes into a folder called _Content.
CSS goes into _Content/css/xxx.js
JS goes into _Content/js/lib/xxx.js (For all the library packages)
Custom page events and functions get called from the page, but are put into a main JS file in _Content/js/Main.js
Images will go into the same place under _Content/images/xxx.x
This is just how we lay it out as it keeps the HTML markup as it should be, for markup.
I think putting css and js into the main html makes the page loads fast.

Is using inline JavaScript preferred to an external include if the script is really short?

I use External JavaScripts in a website as I always try to keep JavaScript at bottom and external.
But Google page speed is giving this suggestion
The following external resources have small response bodies. Inlining
the response in HTML can reduce blocking of page rendering.
http://websiteurl/ should inline the following small resources:
http://websiteurl/script.js
This external js file has only this content
$(document).ready(function() {
$("#various2").fancybox({
'width': 485,
'height': 691,
});
});
But in Yslow I get this suggestion
Grade n/a on Make JavaScript and CSS external
Only consider this if your property is a common user home page.
There are a total of 3 inline scripts
JavaScript and CSS that are inlined in HTML documents get downloaded
each time the HTML document is requested. This reduces the number of
HTTP requests but increases the HTML document size. On the other hand,
if the JavaScript and CSS are in external files cached by the browser,
the HTML document size is reduced without increasing the number of
HTTP requests.
Which is right Google or Yahoo?
This is a bit of a problematic example, on quite a few fronts.
You can organise your scripts in such a way that you do not need to inline that JS. For example you could have a common.js file that runs that snippet, other similar snippets and simplifies your code.
Additionally, this seems to have awoken "never inline any JavaScript EVER" architecture police. Turns out that sometimes it is a best practice to inline JavaScript, for example look at the common snippet from Google analytics.
Why are Google suggesting you should inline this tiny script?
Because 20% of the page visits you get have an unprimed cache
If you have a cache miss, it is likely a new connection to your site will need to be opened (1 round trip) and then the data delivered in the 2nd round trip. (if you are lucky you get to use a keepalive connection and it is cut to 1 round trip.
For a general "global" English web application you are looking at a typical 110ms round trip time for a service hosted in the US. If you are using a CDN the number would probably be halved.
Even if the resource is local, the web browser may still need to access the disk to grab that tiny file.
Non async or defer JavaScript script tags are blocking, if this script is somewhere in the middle of your page, it will be stuck there until the script downloads.
From a performance perspective if the only 2 options are:
Place a 50 char JavaScript bit inline
Place the 50 chars in a separate file and serve it.
Considering that you are a good web citizen and compress all your content, the amount of additional payload this adds is negligible compared to the 20 percent risk of giving people a considerable delay. I would always choose #1.
In an imperfect world it is very rare to have such a clear and easy set of options. There is an option 3 that involved async loading jQuery and grabbing this functionality from a common area.
Making scripts inline can have some detrimental effects -
a) Code organization - Your code gets scattered in between your markup, thus affecting readability
b) Code Minification and obfuscation becomes difficult
Its best to keep your js in seperate files, and then at build time integrate all of them into a single file, and minify and obfuscate this.
This is not quite true. You can configure the web server (well atleast apache) to make the scrips/ccs inlined when they are served.
Here is a useful link
http://www.websiteoptimization.com/speed/tweak/mod_pagespeed/
There are two factors to consider here. One is download time, the other is maintainability. Both of these are impacted by how many times a piece of Javascript is needed.
With respect to download time, you obviously have two choices: include the JS in the body of the page, or as an external file. Including the JS in the body does save an extra HTTP request, although it also bloats the HTML a bit and can be a pain to maintain if you have several scripts you're putting inline on several different pages.
Another important consideration is whether or not the JS is needed immediately on the page. If a small piece of JS is needed as soon as the page loads, then putting it inline may be a good idea. If it's being used for something asynchronous in the future, then putting it an external file may still be a good choice.
I usually write javascript inline, especially if the script is this small. I would say just paste it in your code. It won't increase the http document size by a lot.
While inlining the script will save a request, as Yslow suggests it increases the HTML document size, and mixes content/markup with code/logic, which you generally want to avoid from as much as possible.
The reason Yslow gives this caveat:
Only consider this if your property is a common user home page.
Is that if the page is loaded frequently, it's worth it to have the javascript external, since the files will be cached in the browser. So, if you combine your JS into one file, on the first request you incur one extra request, and on subsequent requests the file is loaded from the cache.
Aaron Peters talk from last year's Velocity EU gives a good insight into the options, and course you should choose - http://www.slideshare.net/startrender/fast-loading-javascript
For really small snippet of js it's really not worth putting them in an external file as the network overhead of retrieving them will dwarf the benefits.
Depending on the latency it may be ever worth including large scripts e.g. Bind mobile has loads of js in the first page loaded which it then cached in localstorage for later pages.
Addy Osmani recently put together a experimental library to help people play with caching scripts in localstorage - http://addyosmani.github.com/basket.js/

How do I optimize my website for slow data connections?

The solutions here worked fine however they were quite labour intensive. To anyone looking to perform similar enhancements on old asp.net solutions I would highly recommend switching the project to MVC just to take advantage of the script and style bundling. .aspx files work as expected in MVC projects.
I'm about to start work on performing some performance enhancements for one of our products.
Our users connect to the network using radio which is extremely slow. The main bottlenecks in the application are the network and the database. I am going to be focusing on reducing the network footprint of the application.
I am going to start with a few "quick wins" before I get down to the nitty gritty of tearing apart UpdatePanels, removing unnecessary content and whatever else I can think of.
Right now I have a few things that I think I'm ready to implement
These include
Minifying and combine css Using This
Minifying and combine js same as above
Removing excess whitespace from html sent to client. Using this
Edit : The assets minification and white space cleaning tools work quite well together.
However I have a few things that I'm not sure how I'll address.
Some microsoft resources (WebResource.axd?d=blahblah and ScriptResource.axd?d=blahblah) are not minified. This and This and a few others depending on the page. Microsoft.Ajax is fine though. How can I manually minify these files if they aren't being minified automatically? Am I missing a setting somewhere?
Is it possible to combine the microsoft resources into a single js file with my javascript?
401 errors, In fiddler I can see that my first hit to the website always gives a 401 error it is immediately followed by the normal 200. Also other resources will randomly have a 401 on their first call as well. Is this some sort of IIS setting that needs to be configured to remove this unneeded call?
Javascript inside aspx files. Unfortunately we have a lot of js inside our aspx files as well as a lot of javascript that gets rendered using ScriptManager.RegisterStartupScript in our code behinds. How would I go about minifying javascript within <script> tags in the aspx markup?
Favicon, can this be diabled? If not what's the next best thing?
Update
Mads Kristensen's combiner works great. However I've found that there are issues with some pages that include 14+ axd references produce a 404.15 error (query string is too long, ie only bug) My solution for this was to gzip and base64 encode the query string.
I've found that combining my js includes with the .axd files is a fruitless task as the .axd files are different for each page. Having my static js files seperate produces an extra service request but it will remain cached on the client instead of having the client redownload those scripts as a part of the combined js axd file.
I enabled anonymous authentication. No more issues.
No progress.
I've found that putting favicon.ico at the root is necessary. I think this may be just because of the way my application has been designed though.
Merging Microsoft script resources: Check out my ContentGator project which I've used to intercept requests for the WebResource (and other scripts and css) files and merge them together. I haven't updated it in a couple years, so I can't speak to how well it'll work out of the box, you should at least be able to reuse some of the code. I don't think I remember adding minification, but you should be able to add it in pretty easily. I think it also has either hooks into RegisterStartupScript, or an alternative to it, where again you should be able to wire in minification.
Favicon, as far as I know, cannot be disabled, as it is requested by default by the browser. If you really don't want it, you could probably just put up a 1x1 pixel ico so you aren't serving a 404, and subsequent requests will result in a 304. It wouldn't hurt to use a CDN for this and all your other static resources as well.
Additionally, check out http://developer.yahoo.com/yslow/ for other more general web optimization tips.
Other things off the top of my head:
Use sprites for images when possible
Output Caching
1 and 2) Optimize .axd: http://madskristensen.net/post/Optimize-WebResourceaxd-and-ScriptResourceaxd.aspx EDIT dead link Compress Script Resource .zip Google Cache of the article
3) HTTP 401 Unauthorized: You're configured authentication mechanism is doing this. If you have Windows authentication enabled but are not using it...
4) Embedded JS: MS AJAX Minifier
http://www.codeproject.com/Articles/81317/Automatically-compress-embedded-JavaScript-resourc
http://stephenwalther.com/blog/archive/2009/10/16/using-the-new-microsoft-ajax-minifier.aspx
There's not much you can do for JS mixed in with your markup. You could make your own utility to parse it out of the ASPX(s) with RegEx and create a file that contains all of it per page then minify that file and insert the 1 script reference. The regular expressions to capture everything within SCRIPT tags will end up being fairly complex because of corner cases like...
<script type="text/javascript">
document.write("<script>Dynamica, RegEx don't stop here -></script>");
</script>
5) Favicon: you either have a LINK tag on your page(s) that reference it with REL="shortcut icon" or you have a "favicon.ico" file sitting at the root of your web site. If you don't have the LINK tags then the browser will check for the favicon.ico at the root of your website automatically.
You should also consider enabling compression in IIS.
IIS6 Compression
IIS7 Compression
From gtmetrix.com:
Avoid bad requests
Avoid CSS #import
Avoid CSS expressions (deprecated)
Avoid document.write
Combine external CSS
Combine external JavaScript
Combine images using CSS sprites
Defer loading of JavaScript
Defer parsing of JavaScript
Enable gzip compression
Enable Keep-Alive
Inline small CSS
Inline small JavaScript
Leverage browser caching
Leverage proxy caching (deprecated)
Make landing page redirects cacheable
Minify CSS
Minify HTML
Minify JavaScript
Minimize cookie size (deprecated)
Minimize DNS lookups
Minimize redirects
Minimize request size
Optimize images
Optimize the order of styles and scripts
Parallelize downloads across hostnames
Prefer asynchronous resources
Put CSS in the document head
Remove query strings from static resources
Remove unused CSS
Serve resources from a consistent URL
Serve scaled images
Serve static content from a cookieless domain
Specify a cache validator
Specify a character set early
Specify a Vary: Accept-Encoding header
Specify image dimensions
Use efficient CSS selectors
You can use the gtmetrix tool, ySlow, or google's Page Speed to see how all of these impact it, but this gtmetrix tool is generally awesome and combines features for you, as well as doing some auto-generations that give you the improved versions of CSS files, etc.
http://wiki.asp.net/page.aspx/80/aspnet-optimization/
has a great set of resources on the various elements that you can / should tweek to make speedster web apps on asp.net! Njoy :)
I think that website should be optimized for best performance regardless of user connection speed.
Website performance/speed affects user experience which on the other hand affects overall website goal/conversion, so creating fast responsive websites and speeding up existing ones should be one of the primary goals of every web developer/front end engineer etc.
Anyway, these are two great resource to start with and comes from two giants:
http://developer.yahoo.com/performance/rules.html
http://code.google.com/speed/
Best
Have you enabled client-side caching for static resources such as site images and styles? They won't help with first page view but would speed up things a lot in subsequence views.
Favicon cannot be disabled but the request itself can be eliminated in modern browsers by using a data:url. For example this would cause a page to have slashdot's favicon without sending any request:
<link rel="shortcut icon" href="data:image/x-icon;base64,AAABAAEAEBAQAAEABAAoAQAAFgAAACgAAAAQAAAAIAAAAAEABAAAAAAAgAAAAAAAAAAAAAAAEAAAAAAAAAB4eE0AX18OAP///wBeXisAYWETANPTxACrq4cAgYEaAEhJEgBKSiYAkJF3AL29pgBiYjAAVFQQADQ0CgBCQg4AWe7u7u7u7pWe7u7u7u7u6e7u7u7u7u7u7/ZVr/+rz/7v+iIp8CJf/v//UibwIl////8CIj+mz//4iIUiuIiIj/iIjCIgiIiPjd3dsiXd3diN3d1CIq3d2I3d3dYiLd3Y0RERFGZsERHUERERERERRDd3d3d3d3dzXERERERERMUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" type="image/x-icon" />
Keep in mind that enabling client-side caching for favicon should save more bytes then embeding it in every page you send.

HTML + JavaScript + CSS compact tool

I need a tool which can minify, optimize and munge many files of those possible types
HTML (minify only)
JavaScript (minify, optimize and munge)
CSS (minify)
The final result should be one HTML file only with all JavaScript and CSS embedded or in the worst scenario 3 files respectively for HTML, JS, and CSS.
I am aware of tools like YUI Compressor etc. What I currently have not found is the type of compressor which will work on all those file types and merge them in one huge file only.
If you're not using gzip yet, start using it already and we can close this thing out ;)
My suggestion would be to put all of the JavaScrpt inside a <script> in the header, and put all of the CSS in a <style> tag as well. Then optimize to the best of your ability and then minimize it on your own. No software will ever be able to handle every case. If you are trying to minimize the file size then just remove the newlines and non-necessary spaces.
I also found this, which should work on your file with all 3 in it.
Also be aware that if you start removing optional tags you could run into some problems. Some browsers don't cooperate as well as they are supposed to.
Gad, consider the drawbacks to minifying CSS. If you don't have a system where you edit normal/minify/ then deploy it can make later editing of CSS rather dicey.
I went through this whole argument on a big UI refresh project for an international bank. One site in particular was doing 1million+ visitors/day and bandwidth numbers were insane despite all our efforts to keep it minimal (every little bit adds up on a site that heavily trafficked) Upon business analysis by a large team of very talented minds, it was determined that we would take several steps, but NOT css minifying due to the added time it would take engineers to undo the minifying before fixing, minifying and redeploying for simple CSS tweaks. The numbers showed that even with a 5gb/day bandwidth improvement, it was still cheaper to not pay a UI engineer for the extra time.
We don't know your site's specifics, but there aren't terribly many that have to worry about the traffic my example uses. Run your site in Firebug's new speed analyzer and see what the real benefit could be from minifying.....now multiply that by your traffic. Usually, that number isn't too scary. Spend your time doing image sprites, combining css and js into respective files (better than mashing into individual php files due to caching benefits) to limit http requests, and ensuring that caching is properly setup. Run gzip compression. If after those steps you're not good, then take the site to a new level.
Keep it simple....it makes a huge difference in the update and maintenance portions of the site's lifecycle. With the time and headache saved, you can help us out with our questions :)
As far as I know there isn't such a tool. If you use PHP on the server side, however, you can try PHP Speedy (not actively developed, but works) or Minify for automated minification, caching and gzipping (HTML, CSS, JS).
edit: kangax's HTML minifier (as Badger suggested) only does HTML minification.
you can use jsCompressor to compress and minify multiple JavaScript files and CSS Compressor to compress/minify multiple Css files into one file.
and to optimize your HTML files you may send post request of your code to Google Clouser server which sends response with optimized Html code.
http://prettydiff.com/?m=minify will do exactly what you need, except that it only minify JavaScript and not obfuscate it.
I suggest you look at the build script that's included in the html5 Boilerplate: http://html5boilerplate.com/. With some minimal tweaking it'll do what you're looking to do.

Categories

Resources