How do I write JavaScript which is compression-friendly? - javascript

I want to minimize the size of the Javascript code I have to send over the wire for my personal website. Are there any tricks to writing code that will compress better? I'm thinking about both min-ification and gzip.
Do I need to use short variable names? Or can min-ification just find all instances of those and shorten them? Or is it fine because gzip will detect them and compress over the wire? For functions, is it better to pass in an object with properties? Or just have individual parameters for each value needed? (I can imagine that parameters would be safe for min-ification to shorten because they are always local to the function)
I have a cursory understanding of what these tools do, but it's not enough to be confident in how I should write my Javascript to compress as small as possible.

If you are using a minification tool, you don't need to use short names for your variables, the minifier will take care of that.
As for function arguments, from a minification point of view, it's better to use individual parameters instead of object properties, because the minifier can't rename object properties, but will rename the parameters.
I don't feel qualified to advise you on how to write more compressable code from a Gzip point of view, but I can give you some advice on writing more minifiable Javascript code.
My first advice is that you run your favorite minifier on your code and then compare the pretty printed version of it, with the original code. That way you will learn what kind of changes your minifier makes on your code.
Usually, minifiers rename variables an function names to shorter ones, but can't rename object properties.
So avoid this:
foo.bar.baz.A = 1;
foo.bar.baz.B = 2;
foo.bar.baz.C = 3;
And instead, do this:
var shortcut = foo.bar.baz;
shortcut.d = 1;
shortcut.e = 2;
shortcut.f = 3;
A minifier will rename shortcut to a one or two letters variable.
Another good practice is using method chaining as much as possible (specially if you are using JQuery)
For example, avoid this:
$div.css('background', 'blue'); // set BG
$div.height(100); // set height
$div.fadeIn(200); // show element
And do this instead:
$('#my-div')
.css('background', 'blue')
.height(100)
.fadeIn(200);

You should use both minification and gzipping, see https://css-tricks.com/the-difference-between-minification-and-gzipping/ for some results.
You should also consider using client side caching, as this will eliminate over the wire cost altogether on cache hits.
Also make sure you're not sending any redundant http headers on your response.

Related

Is eval in Javascript considered safe if not using variable code? [duplicate]

I'm writing some JavaScript code to parse user-entered functions (for spreadsheet-like functionality). Having parsed the formula I could convert it into JavaScript and run eval() on it to yield the result.
However, I've always shied away from using eval() if I can avoid it because it's evil (and, rightly or wrongly, I've always thought it is even more evil in JavaScript, because the code to be evaluated might be changed by the user).
So, when it is OK to use it?
I'd like to take a moment to address the premise of your question - that eval() is "evil". The word "evil", as used by programming language people, usually means "dangerous", or more precisely "able to cause lots of harm with a simple-looking command". So, when is it OK to use something dangerous? When you know what the danger is, and when you're taking the appropriate precautions.
To the point, let's look at the dangers in the use of eval(). There are probably many small hidden dangers just like everything else, but the two big risks - the reason why eval() is considered evil - are performance and code injection.
Performance - eval() runs the interpreter/compiler. If your code is compiled, then this is a big hit, because you need to call a possibly-heavy compiler in the middle of run-time. However, JavaScript is still mostly an interpreted language, which means that calling eval() is not a big performance hit in the general case (but see my specific remarks below).
Code injection - eval() potentially runs a string of code under elevated privileges. For example, a program running as administrator/root would never want to eval() user input, because that input could potentially be "rm -rf /etc/important-file" or worse. Again, JavaScript in a browser doesn't have that problem, because the program is running in the user's own account anyway. Server-side JavaScript could have that problem.
On to your specific case. From what I understand, you're generating the strings yourself, so assuming you're careful not to allow a string like "rm -rf something-important" to be generated, there's no code injection risk (but please remember, it's very very hard to ensure this in the general case). Also, if you're running in the browser then code injection is a pretty minor risk, I believe.
As for performance, you'll have to weight that against ease of coding. It is my opinion that if you're parsing the formula, you might as well compute the result during the parse rather than run another parser (the one inside eval()). But it may be easier to code using eval(), and the performance hit will probably be unnoticeable. It looks like eval() in this case is no more evil than any other function that could possibly save you some time.
eval() isn't evil. Or, if it is, it's evil in the same way that reflection, file/network I/O, threading, and IPC are "evil" in other languages.
If, for your purpose, eval() is faster than manual interpretation, or makes your code simpler, or more clear... then you should use it. If neither, then you shouldn't. Simple as that.
When you trust the source.
In case of JSON, it is more or less hard to tamper with the source, because it comes from a web server you control. As long as the JSON itself contains no data a user has uploaded, there is no major drawback to use eval.
In all other cases I would go great lengths to ensure user supplied data conforms to my rules before feeding it to eval().
Let's get real folks:
Every major browser now has a built-in console which your would-be hacker can use with abundance to invoke any function with any value - why would they bother to use an eval statement - even if they could?
If it takes 0.2 seconds to compile 2000 lines of JavaScript, what is my performance degradation if I eval four lines of JSON?
Even Crockford's explanation for 'eval is evil' is weak.
eval is Evil, The eval function is the most misused feature of
JavaScript. Avoid it
As Crockford himself might say "This kind of statement tends to generate irrational neurosis. Don't buy it."
Understanding eval and knowing when it might be useful is way more important. For example, eval is a sensible tool for evaluating server responses that were generated by your software.
BTW: Prototype.js calls eval directly five times (including in evalJSON() and evalResponse()). jQuery uses it in parseJSON (via Function constructor).
I tend to follow Crockford's advice for eval(), and avoid it altogether. Even ways that appear to require it do not. For example, the setTimeout() allows you to pass a function rather than eval.
setTimeout(function() {
alert('hi');
}, 1000);
Even if it's a trusted source, I don't use it, because the code returned by JSON might be garbled, which could at best do something wonky, at worst, expose something bad.
Bottom Line
If you created or sanitized the code you eval, it is never evil.
Slightly More Detailed
eval is evil if running on the server using input submitted by a client that was not created by the developer or that was not sanitized by the developer.
eval is not evil if running on the client, even if using unsanitized input crafted by the client.
Obviously you should always sanitize the input, as to have some control over what your code consumes.
Reasoning
The client can run any arbitrary code they want to, even if the developer did not code it; This is true not only for what is evaled, but the call to eval itself.
Eval is complementary to compilation which is used in templating the code. By templating I mean that you write a simplified template generator that generates useful template code which increases development speed.
I have written a framework, where developers don't use EVAL, but they use our framework and in turn that framework has to use EVAL to generate templates.
Performance of EVAL can be increased by using the following method; instead of executing the script, you must return a function.
var a = eval("3 + 5");
It should be organized as
var f = eval("(function(a,b) { return a + b; })");
var a = f(3,5);
Caching f will certainly improve the speed.
Also Chrome allows debugging of such functions very easily.
Regarding security, using eval or not will hardly make any difference,
First of all, the browser invokes the entire script in a sandbox.
Any code that is evil in EVAL, is evil in the browser itself. The attacker or anyone can easily inject a script node in DOM and do anything if he/she can eval anything. Not using EVAL will not make any difference.
It is mostly poor server-side security that is harmful. Poor cookies validation or poor ACL implementation on the server causes most attacks.
A recent Java vulnerability, etc. was there in Java's native code. JavaScript was and is designed to run in a sandbox, whereas applets were designed to run outside a sandbox with certificates, etc. that lead to vulnerabilities and many other things.
Writing code for imitating a browser is not difficult. All you have to do is make a HTTP request to the server with your favourite user agent string. All testing tools mock browsers anyway; if an attacker want to harm you, EVAL is their last resort. They have many other ways to deal with your server-side security.
The browser DOM does not have access to files and not a user name. In fact nothing on the machine that eval can give access to.
If your server-side security is solid enough for anyone to attack from anywhere, you should not worry about EVAL. As I mentioned, if EVAL would not exist, attackers have many tools to hack into your server irrespective of your browser's EVAL capability.
Eval is only good for generating some templates to do complex string processing based on something that is not used in advance. For example, I will prefer
"FirstName + ' ' + LastName"
As opposed to
"LastName + ' ' + FirstName"
As my display name, which can come from a database and which is not hardcoded.
When debugging in Chrome (v28.0.1500.72), I found that variables are not bound to closures if they are not used in a nested function that produces the closure. I guess, that's an optimization of the JavaScript engine.
BUT: when eval() is used inside a function that causes a closure, ALL the variables of outer functions are bound to the closure, even if they are not used at all. If someone has the time to test if memory leaks can be produced by that, please leave me a comment below.
Here's my test code:
(function () {
var eval = function (arg) {
};
function evalTest() {
var used = "used";
var unused = "not used";
(function () {
used.toString(); // Variable "unused" is visible in debugger
eval("1");
})();
}
evalTest();
})();
(function () {
var eval = function (arg) {
};
function evalTest() {
var used = "used";
var unused = "not used";
(function () {
used.toString(); // Variable "unused" is NOT visible in debugger
var noval = eval;
noval("1");
})();
}
evalTest();
})();
(function () {
var noval = function (arg) {
};
function evalTest() {
var used = "used";
var unused = "not used";
(function () {
used.toString(); // Variable "unused" is NOT visible in debugger
noval("1");
})();
}
evalTest();
})();
What I like to point out here is, that eval() must not necessarily refer to the native eval() function. It all depends on the name of the function. So when calling the native eval() with an alias name (say var noval = eval; and then in an inner function noval(expression);) then the evaluation of expression may fail when it refers to variables that should be part of the closure, but is actually not.
I saw people advocate to not use eval, because is evil, but I saw the same people use Function and setTimeout dynamically, so they use eval under the hoods :D
BTW, if your sandbox is not sure enough (for example, if you're working on a site that allow code injection) eval is the last of your problems. The basic rule of security is that all input is evil, but in case of JavaScript even JavaScript itself could be evil, because in JavaScript you can overwrite any function and you just can't be sure you're using the real one, so, if a malicious code start before you, you can't trust any JavaScript built-in function :D
Now the epilogue to this post is:
If you REALLY need it (80% of the time eval is NOT needed) and you're sure of what you' re doing, just use eval (or better Function ;) ), closures and OOP cover the 80/90% of the case where eval can be replaced using another kind of logic, the rest is dynamically generated code (for example, if you're writing an interpreter) and as you already said evaluating JSON (here you can use the Crockford safe evaluation ;) )
The only instance when you should be using eval() is when you need to run dynamic JS on the fly. I'm talking about JS that you download asynchronously from the server...
...And 9 times of 10 you could easily avoid doing that by refactoring.
On the server side eval is useful when dealing with external scripts such as sql or influxdb or mongo. Where custom validation at runtime can be made without re-deploying your services.
For example an achievement service with following metadata
{
"568ff113-abcd-f123-84c5-871fe2007cf0": {
"msg_enum": "quest/registration",
"timely": "all_times",
"scope": [
"quest/daily-active"
],
"query": "`SELECT COUNT(point) AS valid from \"${userId}/dump/quest/daily-active\" LIMIT 1`",
"validator": "valid > 0",
"reward_external": "ewallet",
"reward_external_payload": "`{\"token\": \"${token}\", \"userId\": \"${userId}\", \"amountIn\": 1, \"conversionType\": \"quest/registration:silver\", \"exchangeProvider\":\"provider/achievement\",\"exchangeType\":\"payment/quest/registration\"}`"
},
"efdfb506-1234-abcd-9d4a-7d624c564332": {
"msg_enum": "quest/daily-active",
"timely": "daily",
"scope": [
"quest/daily-active"
],
"query": "`SELECT COUNT(point) AS valid from \"${userId}/dump/quest/daily-active\" WHERE time >= '${today}' ${ENV.DAILY_OFFSET} LIMIT 1`",
"validator": "valid > 0",
"reward_external": "ewallet",
"reward_external_payload": "`{\"token\": \"${token}\", \"userId\": \"${userId}\", \"amountIn\": 1, \"conversionType\": \"quest/daily-active:silver\", \"exchangeProvider\":\"provider/achievement\",\"exchangeType\":\"payment/quest/daily-active\"}`"
}
}
Which then allow,
Direct injection of object/values thru literal string in a json, useful for templating texts
Can be use as a comparator, say we make rules how to validate quest or events in CMS
Con of this:
Can be errors in the code and break up things in the service, if not fully tested.
If a hacker can write script on your system, then you are pretty much screwed.
One way to validate your script is keep the hash of your scripts somewhere safe, so you can check them before running.
Eval isn't evil, just misused.
If you created the code going into it or can trust it, it's alright.
People keep talking about how user input doesn't matter with eval. Well sort of~
If there is user input that goes to the server, then comes back to the client, and that code is being used in eval without being sanitized. Congrats, you've opened pandora's box for user data to be sent to whoever.
Depending on where the eval is, many websites use SPAs, and eval could make it easier for the user to access application internals that otherwise wouldn't have been easy. Now they can make a bogus browser extension that can tape into that eval and steal data again.
Just gotta figure what's the point of you using the eval. Generating code isn't really ideal when you could simply make methods to do that sort of thing, use objects, or the like.
Now a nice example of using eval.
Your server is reading the swagger file that you have created. Many of the URL params are created in the format {myParam}. So you'd like to read the URLs and then convert them to template strings without having to do complex replacements because you have many endpoints. So you may do something like this.
Note this is a very simple example.
const params = { id: 5 };
const route = '/api/user/{id}';
route.replace(/{/g, '${params.');
// use eval(route); to do something
eval is rarely the right choice. While there may be numerous instances where you can accomplish what you need to accomplish by concatenating a script together and running it on the fly, you typically have much more powerful and maintainable techniques at your disposal: associative-array notation (obj["prop"] is the same as obj.prop), closures, object-oriented techniques, functional techniques - use them instead.
As far as client script goes, I think the issue of security is a moot point. Everything loaded into the browser is subject to manipulation and should be treated as such. There is zero risk in using an eval() statement when there are much easier ways to execute JavaScript code and/or manipulate objects in the DOM, such as the URL bar in your browser.
javascript:alert("hello");
If someone wants to manipulate their DOM, I say swing away. Security to prevent any type of attack should always be the responsibility of the server application, period.
From a pragmatic standpoint, there's no benefit to using an eval() in a situation where things can be done otherwise. However, there are specific cases where an eval SHOULD be used. When so, it can definitely be done without any risk of blowing up the page.
<html>
<body>
<textarea id="output"></textarea><br/>
<input type="text" id="input" />
<button id="button" onclick="execute()">eval</button>
<script type="text/javascript">
var execute = function(){
var inputEl = document.getElementById('input');
var toEval = inputEl.value;
var outputEl = document.getElementById('output');
var output = "";
try {
output = eval(toEval);
}
catch(err){
for(var key in err){
output += key + ": " + err[key] + "\r\n";
}
}
outputEl.value = output;
}
</script>
<body>
</html>
Since no one has mentioned it yet, let me add that eval is super useful for Webassembly-Javascript interop. While it's certainly ideal to have pre-made scripts included in your page that your WASM code can invoke directly, sometimes it's not practicable and you need to pass in dynamic Javascript from a Webassembly language like C# to really accomplish what you need to do.
It's also safe in this scenario because you have complete control over what gets passed in. Well, I should say, it's no less safe than composing SQL statements using C#, which is to say it needs to be done carefully (properly escaping strings, etc.) whenever user-supplied data is used to generate the script. But with that caveat it has a clear place in interop situations and is far from "evil".
It's okay to use it if you have complete control over the code that's passed to the eval function.
Code generation. I recently wrote a library called Hyperbars which bridges the gap between virtual-dom and handlebars. It does this by parsing a handlebars template and converting it to hyperscript. The hyperscript is generated as a string first and before returning it, eval() it to turn it into executable code. I have found eval() in this particular situation the exact opposite of evil.
Basically from
<div>
{{#each names}}
<span>{{this}}</span>
{{/each}}
</div>
To this
(function (state) {
var Runtime = Hyperbars.Runtime;
var context = state;
return h('div', {}, [Runtime.each(context['names'], context, function (context, parent, options) {
return [h('span', {}, [options['#index'], context])]
})])
}.bind({}))
The performance of eval() isn't an issue in a situation like this too because you only need to interpret the generated string once and then reuse the executable output many times over.
You can see how the code generation was achieved if you're curious here.
There is no reason not to use eval() as long as you can be sure that the source of the code comes from you or the actual user. Even though he can manipulate what gets sent into the eval() function, that's not a security problem, because he is able to manipulate the source code of the web site and could therefore change the JavaScript code itself.
So... when to not use eval()? Eval() should only not be used when there is a chance that a third party could change it. Like intercepting the connection between the client and your server (but if that is a problem use HTTPS). You shouldn't eval() for parsing code that is written by others like in a forum.
If it's really needed eval is not evil. But 99.9% of the uses of eval that I stumble across are not needed (not including setTimeout stuff).
For me the evil is not a performance or even a security issue (well, indirectly it's both). All such unnecessary uses of eval add to a maintenance hell. Refactoring tools are thrown off. Searching for code is hard. Unanticipated effects of those evals are legion.
My example of using eval: import.
How it's usually done.
var components = require('components');
var Button = components.Button;
var ComboBox = components.ComboBox;
var CheckBox = components.CheckBox;
...
// That quickly gets very boring
But with the help of eval and a little helper function it gets a much better look:
var components = require('components');
eval(importable('components', 'Button', 'ComboBox', 'CheckBox', ...));
importable might look like (this version doesn't support importing concrete members).
function importable(path) {
var name;
var pkg = eval(path);
var result = '\n';
for (name in pkg) {
result += 'if (name !== undefined) throw "import error: name already exists";\n'.replace(/name/g, name);
}
for (name in pkg) {
result += 'var name = path.name;\n'.replace(/name/g, name).replace('path', path);
}
return result;
}
I think any cases of eval being justified would be rare. You're more likely to use it thinking that it's justified than you are to use it when it's actually justified.
The security issues are the most well known. But also be aware that JavaScript uses JIT compilation and this works very poorly with eval. Eval is somewhat like a blackbox to the compiler, and JavaScript needs to be able to predict code ahead of time (to some extent) in order to safely and correctly apply performance optimisations and scoping. In some cases, the performance impact can even affect other code outside eval.
If you want to know more:
https://github.com/getify/You-Dont-Know-JS/blob/master/scope%20%26%20closures/ch2.md#eval
Only during testing, if possible. Also note that eval() is much slower than other specialized JSON etc. evaluators.
My belief is that eval is a very powerful function for client-side web applications and safe... As safe as JavaScript, which are not. :-) The security issues are essentially a server-side problem because, now, with tool like Firebug, you can attack any JavaScript application.
When is JavaScript's eval() not evil?
I'm always trying to discourage from using eval. Almost always, a more clean and maintainable solution is available. Eval is not needed even for JSON parsing. Eval adds to maintenance hell. Not without reason, it is frowned upon by masters like Douglas Crockford.
But I found one example where it should be used:
When you need to pass the expression.
For example, I have a function that constructs a general google.maps.ImageMapType object for me, but I need to tell it the recipe, how should it construct the tile URL from the zoom and coord parameters:
my_func({
name: "OSM",
tileURLexpr: '"http://tile.openstreetmap.org/"+b+"/"+a.x+"/"+a.y+".png"',
...
});
function my_func(opts)
{
return new google.maps.ImageMapType({
getTileUrl: function (coord, zoom) {
var b = zoom;
var a = coord;
return eval(opts.tileURLexpr);
},
....
});
}
Eval is useful for code generation when you don't have macros.
For (a stupid) example, if you're writing a Brainfuck compiler, you'll probably want to construct a function that performs the sequence of instructions as a string, and eval it to return a function.
While there may be numerous instances where you can accomplish what you need to accomplish by concatenating a script together and running it on the fly, you typically have much more powerful and maintainable techniques at your disposal. eval is rarely the right choice.: associative-array notation (obj["prop"] is the same as obj.prop), closures, object-oriented techniques, functional techniques - use them instead.
When you parse a JSON structure with a parse function (for example, jQuery.parseJSON), it expects a perfect structure of the JSON file (each property name is in double quotes). However, JavaScript is more flexible. Therefore, you can use eval() to avoid it.

How to Encode JavaScript files?

How can I encode my JavaScript file like DLL files?
I mean nobody can understand the code like Dll created from CS file in C#.
I need this because I want to give my functions to some company, but I do not want they to understand inside my functions....just can call the functions.
I see the jQuery files are encode to variables (a,b,c,d,....).
for example encode this simple code:
function CookiesEnabled() {
var result = false;
createCookie("testing", "Hello", 1);
if (readCookie("testing") != null) {
result = true;
eraseCookie("testing");
} return result;
};
There really isn't any way to encrypt/encode code like that in JS (at least I do not know of any way), but the standard way is to use good minifiers i.e. programs that collapse your code, remove comments rename local variables from good long names to stuff like width and height to stuff like a and b. And even re-structure your code so its as compact as possible. They usually end up non-human readable.
Minifing is usually even called JS compiling, but its not really. As with is a good one, well not going to go there, there are so many, but for my purposes I've been using the Microsoft official bundler:
http://www.asp.net/mvc/tutorials/mvc-4/bundling-and-minification
You should also check out this question (all of the big names that I know are all there.):
Is there a good JavaScript minifier?

Passing a JSON object to a remote LESS compiler to use as LESS variables

I have an admin panel where users customize the look of a static website (mostly fonts and colors). This panel generate a JSON object with the user values. What I would need to do is passing this JSON to the LESS compiler, so that it can dynamically generate a CSS file from a LESS one using the JSON content as LESS variables. The filename should be different every time, something line file-ID.css (the ID is for the user and it could be passed via JSON too).
Is it technically possible (without extending LESS)? I noticed, for example, that you can pass functions to the parser object when you create it, could I use this functions to evaluate the JSON and passing the variables to the compiler?
Obviously I don't need to know the details, just if it is doable and possibly some link to related information if you have it.
Thanks in advance.
The best way I've found to do what I was trying to accomplish was to use a server side LESS library like PHPLESS to parse the variables from the JSON before compiling. Regular LESS compiler doesn't allow to dynamically inject variables.
To my knowledge the LESS compiler doesn't support any other input than LESS. It would be trivial to make your own pre-parser that mixes-in the variables from JSON .. not even a parser, more of a string-replacer.

How to allow minimifying of public functions names in Javascript?

I don't know whether there is a solution to this issue, but I have a large set of Javascript functions bearing long descriptive names, something like:
function getTimeFromTimezoneInMilliseconds(...) { ... };
function computeDifferenceFromUTCInMilliseconds(...) { ... };
...
These long names help explaining what the code does, since some operations are complex and not obvious to understand when reading the code only. It helps maintaining the code too.
I am minimifying this Javascript code, but of course, those names are not minimified.
Is there a refactoring trick in Javascript that would allow minimifiers to pick smaller function names and reduce the code size?
You should wrap your code in an IIFE.
This way, you won't have any public members at all, and the minifier will be able to do whatever it wants.
This has the added advantage of not polluting the global scope.
Don't minify yourself! Let the machine do the hard parts.
There are many different options.
You can use an online site where you paste your code and you get the minified back (manual).
You can automate and use a server-side language to minify your JavaScript.
You can use Google CC or Yahoo YUI Compressor to minify and greatly optimize your code.

Any advanced tool to compress a JS file?

I'm looking for a tool to compress my JS files.
What I want is not just simply remove unneccessary characters;
I prefer to tool to be able to reduce the lengths of
local variables in functions also.
Even better I the tool can reduce the lengths of all other
stuff like global variables, function names, etc. This is to protect the code from being used by web viewers without our consent.
For example, the tool should rename a variable or function name from "some_variable", "some_function" to "sv", "sf", etc. And then refactoring all related references.
I've tried to dig the net but couldn't find something similar.
I think that this one can help you : http://www.minifyjavascript.com/
I've used it in the past and it does a good job!
The Google Closure Compiler does this. It has various settings, but even the "simple optimizations" setting will shorten variable names and (in cases where it knows the function will never be called outside the script) function names. It will even inline functions when there's a savings to be had.
For example, given this script:
jQuery(function($) {
var allDivsOnThePage = $("div"),
someParagraphsAsWell = $("p");
setColor(allDivsOnThePage, "blue");
setColor(someParagraphsAsWell, "green");
function setColor(elms, color) {
return elms.css("color", color);
}
});
Using the closure compiler with simple optimizations (and telling it we're using jQuery) yields:
jQuery(function(a){var b=a("div"),a=a("p");b.css("color","blue");a.css("color","green")});
Note how it's not only shortened the identifiers, but reused one where it detected it could (in this case that didn't save us anything, but in some other cases it could), and inlined the setColor function since that resulted in a savings.
Try YUI Compressor:
http://developer.yahoo.com/yui/compressor/
For me it seems to be one of the most powerful at the moment.

Categories

Resources