The Google Closure Compiler is a powerful compiler and minifier for JS, which gives a lot of optimization options such as renaming variables, removing dead codes, collapsing variable declarations, rewriting control flow structures and etc.
What I want is to separately apply one or some of these optimizations on an input JS program. For example, I may want to rename variables with short names, but not to remove dead codes. How can I achieve this kind of detailed compilation pass control? Does the source code of CC expose specific interfaces to do this customization, or I should write my own pass(If so, how am I supposed to start?).
The command line features do offer several options for controlling the compilation, but are insufficient to fit what I want above. Since the source code is kinda complicated and few detailed design documentation can be found, I am truly stuck here. Any insights would be appreciated, thanks :)
Take a look at DefaultPassConfig. That class lists all the passes that are run during the compilation, based on what options are set in the CompilerOptions. Some of the CompilerOptions can be controlled from the command line, but we generally try to keep the compiler relatively simple and easy to use, and not ask users to make decisions about a bunch of different compiler flags. Plus, there are some passes that actually increase the code size, but they do it in such a way that it makes it easier for some later pass to decrease it afterwards.
Of course if you're just experimenting with the compiler or trying to understand how it works, you can turn on and off whichever passes you want, either by adding new flags, or just modifying DefaultPassConfig directly.
Related
I come from the Java world, i.e. a typesafe world and am right now doing a couple of things for which I need client side execution using javascript.
I keep running into pretty hard to detect errors at times due to the non typification of JS and am wondering if there is any way to prevent it beforehand. E.g. setting sth like "use typification;" or via some tool that does these checks before executing like a compiler does.
E.g. last time was when I was creating a face in three.js. There depening on order of vertices a face is front-facing or not. I had mixed that up and then copy pasted parameters in which case I also copied a bracket too much so it ended up in the wrong place with just calling the method with one instead of three vertices which of course resulted in an error. However in line 2107 of three.js code and it took a while to figure out this little copy paste issue. Comparing to java the compiler would have directly complained that i try calling the method with 1 instead of 3 parameters...
Hope there is sth like it. Or do have some tips how to spot such things faster?
Cheers
Tom
There are various linting tools which you can use to scan your javascript files before you actually use them. The popular ones in the industry are JSLint, JSHint, JSCS and ESLint.
They come inbuilt with various rule sets which you can configure and you can also add your own rules.
You can compare these to checkstyles and PMD from the JAVA world.
You have a number of answers. But first, need to clarify: Java is not type-safe (see: NullPointerException, history of).
But to get closer to type-safety in any dynamic language you have the option to pepper your code with asserts. This can to some degree be automated, it may cause performance issues. This is the route I usually take, but I certainly wouldn't do it with three.js.
For JavaScript specifically, you have two additional options: TypeScript and Flow.
TypeScript is a dialect of JavaScript with type annotations that gets compiled down to plain JS. Flow is a static analyzer written in OCaml that tries to infer types in your JS code and check them.
I am looking for a solution that helps me achieve this:
I would like to have at 'build' time an error throwed if I try doing this:
var ciao = {
wow: 'ciaoooo',
}
console.log(ciao.wowe); // this should be an error because the prop does not exist
I have tried Typescript and it does the jobs, but it seems an overkill use it only for this. I was wondering if there was something for ESLint or something else that can help me with this type of problem.
You're asking for features of a statically typed language. If you want reliable information about variables on build time, you'll need this "overkill" of defining interfaces and types. TypeScript, however, is optionally typed, which might be a good compromise if you want static types for your public interfaces.
If you want type checking but don't want to use a transpiler, you could also try Flow which tries to derive type information just from your code as much as possible. However, its capabilities without additional type information are very limited.
Concerning your actual question: I think it would be possible to derive type information within module (or file) boundaries with ESLint or Babel. As long as you are using that object just within these boundaries, static analysis can probably help you. But beyond that, I don't think it's reliably feasible because of the dynamic nature of JavaScript. Unfortunately, I don't know such a rule or plugin.
ESLint doesn't have a rule like that. It would be very hard to create a rule that would be able to catch all possible cases for this. ESLint relies on Escope to track declared variables, but Escope doesn't track object properties, and there are just too many ways to add a property to a declared object.
I'm trying to make a generic table-top RPG helper web app. It has to be generic because Wizards of the Coast is very protective of their copyrights. So, in order to avoid cease-and-desists, the system has to be capable of loading arbitrary rules. That said, my buddies and I (along with most any other user) will be using it for D&D.
In D&D, modifiers are based on a character's stats. To get a modifier, you take the stat, subtract 10, divide by 2, and round down.
function getModifier(statValue) {
return Math.floor((statValue - 10) / 2);
}
My app will be capable to loading a game's rules from a .json file. I want to be able to make this modifier function user-definable. The easiest way would be to just eval() whatever they provide in the .json file, but obviously that's a terrible idea due to security issues.
Unfortunately, I can't think of a simple way to get around this in a secure manner. Obviously I could write my own parser, but that's more complicated than I'd like/am currently capable of.
Ideas?
I did a quick google search and found two options - http://mathjs.org and http://jsep.from.so
btw. writing your own parser is not that difficult. You may be able to write one that's sufficient for your purpose in less than 100 lines of code
First, bear in mind that this is all on the client-side. So long as you trust the JSON (it came from your server or it is controlled alongside the application itself), then you can use it. That doesn't completely rule out some trojan inserting malicious rules, but I don't think botnets have learned to play D&D yet.
To literally avoid eval and provide some modicum of security, you can use new Function. It takes a list of argument names followed by the function body, allowing you a large amount of control over how the function is defined and where it can be referenced from. You only need to store the body in the JSON file, allowing you to largely control the parameters and make it difficult to assign to the global scope, minimizing your attack surface (such as it is) quite a lot.
If you want to have fun with this and now your rules will largely be mathematical, you may consider using a tool like pegjs to generate a parser for your rules. Calculators are one of the classic introductions to parsers and compilers, so this could be an opportunity to play with some fun tech. The rules and parsers built by pegjs are very easy to use in a JS project (I have an example using PegJS and ES6 via Babel).
There are two options as I see it. One, if you 100% want to define the function in .json, you will need to parse. JSON does not allow functions, so you need to store the function as a string and eval it.
The better way would be to also provide a config.js ability. Perhaps the .json would have a "configUrl": property, and then expose well known apis, eg RPGHelper.setModifer(someFunction) that you can call from inside the config.js file. The user is then free to define whatever arbitrary logic there.
One solution to your problem is to use sandboxed iframes (current supported by all major browsers
You run user code on a separate page, with limited priviliges, in such a way that the code does not interfere with the rest of the application. You can communicate with the sandboxed page using postMessage() and window.onmessage.
I'm using TypeScript, and I want to use Closure-Compiler to minify and obfuscate the JS output that I get after building my TS code.
I read that GCC is capable of obfuscating based on type definitions. As far as I can tell (and please correct me if I'm wrong) this means that if I have type annotations on my code then GCC will use them to do a better obfuscation.
For example, for obj.someProp GCC currently finds all instances of the someProp property name in my code, without regarding which object it's on, and replaces all of them to the same obfuscated name (e.g. o.a).
But if I had type annotations on my code GCC would instead be able to know which object is of which type and obfuscate it accordingly - so the same property name on two different types will be obfuscated to two different names.
Questions:
Did I understand this correctly?
Is "type-safe" obfuscation a good thing?
I mean, what are the benefits of it? It seems like it would not have an impact on the resulting file size, and might even have a negative impact on the gzipped file size (since there are potentially more different keys across different types).
Can I use TypeScript to somehow create the GCC annotations automatically?
since TS is already type-safe, I believe it is possible, however I'm skeptical that it is a feature. It would probably require knowledge of the TS-compiler internals. Any insights on this?
EDIT
I actually just found this and this. I'll check it out and post an update soon.
Update
As of the 20150315 release of Closure-compiler, the type based optimizations are enabled by default.
"use types for optimizations" triggers several compiler optimizations:
"disambiguate properties"
"ambiguate properties"
"inline properties"
Disambiguation occurs early in the optimization process and enables the type unaware optimizations to remove unused properties, devirtualization and other optimizations
Ambiguation occurs just before renaming and is specifically to improve code size by leveraging shorter names and making them more common (which improves gzipping)
Inline properties uses type information to inline properties that wouldn't otherwise be devirtualized. This pass is less effective and thus less interesting to this discussion.
These optimizations are very effective for larger projects but there are some risks involved as there are isn't usually any runtime type enforcement. It is possible to lie about the types in a variety of unintentional ways. I describe the risks in more detail in this project wiki page:
https://github.com/google/closure-compiler/wiki/Type-Based-Property-Renaming
The risks are low but can be mitigated by using the "runtime type check" instrumentation that the compiler supports. If you are trying to retrofit a larger project or having trouble isolating a "bad" disambiguation, look into the "runtime type check"
The TypeScript issue is a separate and should be really be a separate question. Converting TypeScript to the Closure Compiler type system is possible to some extend but there are differences between the TypeScript and Closure Compiler type systems (TypeScript is distinctly unsound and allows many unsafe assignments).
Your understanding is mostly correct. The advantage to type based renaming (which requires the --use_types_for_optimization flag) is that properties that are named the same as any property in an extern are no longer blocked from renaming.
var foo = {};
foo.src = 'test'; //only renameable with type based optimizations
As for typescript, closure-compiler is being modified to understand type-script style type notations. However this project is in the very early stages. The plugins you linked are the best option I know of for now.
The type systems in Typescript and Closure-Compiler are not completely compatible right now. That is also being actively worked on.
I think Google Closure Compiler will support TypeScript annotations in the near future. Look here
Is there a way to read a set of JavaScript files, and output a description of where every function/method is defined?
I realize that this is likely impossible in full generality, due to the extreme dynamic nature of the language. What I'm imagining is something which gets the (relatively) straightforward cases. Ideally, I'd want it figure out where, e.g. some method got attached to string or hash or some other fundamental class (and also just let you find all the classes/functions that get defined once in one place).
Does such a tool exist?
Give Eclipse (with JSDT) or Aptana a try.
The Outline view of a JavaScript file gives great view of functions and object hierarchies.
Granted this will only work for a single js file, and it sounds like your looking for more of a report, so I guess I need to ask what your ultimate goal is.
I'm not sure if it will output anything without documentation comments, but you could give YUI Doc a try.