We are creating a javascript library from scratch and we need some utilities functions, just like what dojo provides, e.g. xhr wrapper, simple inheritances and array manipulation, etc. We don't want to write these functions ourselves from scratch, but we also don't want to just copy and paste code from dojo.
Dojo provides the custom build mechanism, but it works on the module level. For example, in the custom build, you can specify to only add the lang module in the dojo base. But we may only need the dojo.hitch function in the lang module, but not the dojo.clone function that also included in the lang module. We have a tight constraint on the size of the library file, so we need to remove any unused code.
What we are seeking is a way to extract certain functions from a module and build these functions into a single javascript file. For example, say we need dojo.mixin, dojo.declare and dojo.hitch functions, then these functions' declaration and their dependent inner functions should be built into a single file.
Any suggestions?
Update from my test result
Thanks Stephen Chung for your great suggestion on how to solve this problem.
So basically I created a custom dojo build with four modules : dojo._base.xhr, dojo._base.json, dojo._base.declare and dojo._base.lang. The built dojo.js file is 149kb, after using Google Closure compiler with advanced mode, the size has reduced to 24kb and the unused functions have been removed. If we want to keep some functions, just add another javascript file like below:
window['dojo'] = dojo;
window['dojo']['requireLocalization'] = dojo.requireLocalization;
window['dojo']['moduleUrl'] = dojo.moduleUrl;
and compile it with dojo.js.
Unfortunately the size is still a little bigger. We have a size limit of 40k in total. But this is still a very good solution for other cases.
Dojo Core is written in a very compact manner, with a lot of cross callings (i.e. one function calling other functions) in order to save download bytes and reduce code size.
Therefore, you need a minimal set of Dojo Core functions, together with the functions that it calls.
In other words, what you need is a build that includes Dojo Core, but then removes any dead-code that is never called.
I'd suggest that you don't go about creating a minimal-footprint library by writing it from scratch yourself. What you need is to use the Closure Compiler's Advanced Mode to process your Dojo application. It removes dead code, optimizes the entire app, and fully obfuscates it.
And yes, it is possible to use the Closure Compiler with Dojo in Advanced Mode -- Dojo is probably the only popular library that can do it. Read it here.
Some statistics: A Closure-compiled Dojo app is typically around 25-30% smaller than the equivalent Dojo app in a Shrinksafe build. It depends on how many Dojo Core features you use -- typically you may reduce Dojo Core up to 40% of the original size (i.e. reduce code size by up to 60%) from dead code removal alone. Further reductions are more difficult, as you may have to start removing features like the Dojo loader etc.
Are you fixed on dojo? Google's closure library offers a very similar package management system to dojo, as well as all the necessary scripts and tools you need to compile the javascript down to a single "executable". The closure library compiler is arguably one of the most advanced javascript minifiers you will find so if tight, concise javascript is your goal, this tool could help tremendously.
Related
I'm building a project in javascript, using paper.js in some features.
Most of my project is built in vanilla Javascript, just some minimum but important features are built with paper.js
The problem is that the library (.min library) is 200kb.
The normal library is 300kb, I was wondering if there is an automatic way to see which functions are being used in the main paper.js library, in order to delete the useless functions.
If there is no program or automatic way to do this, maybe some advice of how to do it manually, or which tools you recommend for me and my team in order to delete useless functions, then minify the file and run it smaller.
Thank you all guys, I did not added any specific code because I want this anwser to be global.
Greetings
You have to do it manually but it's not an easy (and certainly not quick) process. You'll have to find which functions you're using and then find whatever classes or functions those functions reference. You would probably have an easier time creating a new script then copy/pasting what you're using (and any referenced content) then running it with your script, log errors, and repeat.
When you're done there's many minify libraries and services online you can use to minify the new script.
I used the paper-core version and then minified.
I saved 140kb by doing this.
There's still no way to see useless functions in this library
I'm using a nice framework for Adobe ExtendScript called Extendables. I forked the project here: https://github.com/daluu/Extendables
A problem though is that in some ways using the framework is worse than not because the framework extends javascript objects with more functionality. And on an initial review of the code files, they seem rather interdependent such that it will take some work to uncouple the strict dependencies to make it optional/configurable to load only what you need and skip the rest in case of issues with particular features (i.e. you can just not load/include what you don't use - I don't think that's currently possible, although I might be mistaken). See the issue tracker in my project for details but in general the issues encountered using the full framework is failure of try/catch blocks and object iteration includes unintented properties.
For me, I'd just like at a minimum to make all functionality optional and just load the logger and Jasmine test framework as those are the only two feature/modules that I really use with Extendables. I don't care for the extensions to strings, files, object, arrays, etc.
As I'm a novice in javascript/ExtendScript, and this is not a trivially simple javascript framework, I could use suggestions on how to decouple the dependencies so that every module (baring it's dependendencies) can be optionally loaded, and where there are dependencies we can group into sets as in you can load or not load this set of features.
Sorry that I can't include code snippets as its too much to post, you can find it in my Github fork.
Not sure if this is best StackExchange site to post but starting here.
You should be able to extract the log module from this file:
https://github.com/daluu/Extendables/blob/master/core-packages/logging/lib/index.jsx
Try to use it like this (not tested):
#include "core-packages/logging/lib/index.jsx"
Log.debug("Log this");
You might need to adjust some things in there e.g. Folder.extendables does not exist in ExtendScript. Also exports.Log at the end will throw an error.
I've recently created a JavaScript library using Google's Closure Compiler: https://github.com/bvaughn/task-runner
I intend for this library to be used by applications that also require the full Closure Library, and so I want the built version of my library (Task Runner) not to include a subset of Closure Library. If Task Runner includes a subset of Closure Library, and another application includes the full Closure Library, a race condition will exist between which loads Closure last (last in wins). It also bloats the size of the Task Runner file.
However I'm having trouble. If I don't require any of the Closure library classes, Task Runner builds fine (obviously). However if I require something (goog.dom for instance) then my "compiled" JavaScript file also includes a portion of the Closure library. Is there a way to tell the Closure Compiler to leave certain JavaScript files/modules/whatever out of the built result?
FWIW, the Plovr experimental-exclude-closure-library option seems to somewhat describe the functionality I'm looking for.
On the surface what you are asking makes no sense. You want to depend on/use code from Closure-library, but not include it in your output. This isn't really possible nor how the library and compiler function together.
There is a rather small list of primitive functions defined in Closure-library that are completely removed/replaced when compiled with Closure-compiler. goog.require and goog.provide are the two most prominent of those.
For the vast majority of the Closure-Library, if you use or depend on a class, method or object that specific code will appear in the compiled output. And because that library code itself may depend on other parts of the library, even more code may be included in the compiled result.
The difference between using Closure-compiler with Closure-library as compared to more traditional JavaScript libraries is that ONLY the parts of the code determined to be called and used are included in the output. This is much more granular than a simple file inclusion - prototypes, variables, constants, etc will all be excluded because the compiler can determine that they are never used.
Distributing a Library
If you are building a library which depends on Closure-library, you have two options.
Distribute a compiled/built version
You would compile your library using Closure-library; exporting any public API methods and properties. Others who utilize your library with Closure-compiler or Closure-library will need to use an externs file and include your library AFTER compilation.
Distribute your library as source
You would simply distribute your library source code. Others would goog.require your library as part of their source and build process. If Closure-library is used in both projects, it will only be included once in the build process.
There is no hybrid approach where you compile your code but exclude Closure-library. This violates the principle that all of the source code will be compiled simultaneously.
You can peruse my Geolocation-marker library to see an example. I provide a compiled standalone version of the code for use, but the uncompiled source can also be included in other projects which use Closure-library.
Does anybody have to share a file for extern definitions for dojo and dgrid ?
I am trying to include those in my clojurescript project and to use even 'simple' clsure compiler optimization option.
At the moment I have found dgrid and dojo toolkit incompatible with Clojurescript compilation process.
In other words it is not possible to
a) include dojo or dgrid with :foreign-libs option and then leverage the
closure compiler to create a 'minified release using closure compiler :advanced or even :simple option
b) it is also not possible to include the dojo or dgrid libraries with :extern option
because for that one has to create a list of extern definitions (which does not come in dojo/dgrid releases).
I am pretty sure option ( a ) is a dead-end, because for it to work dojo team needs to make changes to the toolkit, and it is probably low priority. I have read Stephen Chung's 50+ pages document of how to modify dojo to work with google-closure, but it is well beyond my capabilities to re-implement for Dojo 1.9 and above.
I am thinking ( b ) might eventually work, but for that I need to find the externs library.
Hoping somebody has done it for dojo 1.9 and the latest dgrid.
Prior to posting here I had checked the repository of currently available 'externs' for the closure compiler,and dojo is not there
https://code.google.com/p/closure-compiler/source/browse/#git%2Fcontrib%2Fexterns
I had also found a online extractor of externs
http://www.dotnetwise.com/Code/Externs/index.html
and http://www.dotkam.com/2013/07/15/clojurescript-use-any-javascript-library/
However, I am not even sure what dgrid/dojo files to do this for.
Therefore looking if anybody has created these externs already.
You can use their own source file as the externs definition. Obviously this means that the compiler can't optimize the library, but if as you say, you don't need that, it's a good working option.
I am not experienced with dojo/dgrid in ClojureScript but I've used several javascript libraries with ClojureScript already.
The best way to use complicated libraries with ClojureScript is just not to touch them, and not run Google Closure on them. (what you call (b)).
Therefore, you will need to put very few externs definitions in externs.js
Only for the specific vars/functions which you access from your ClojureScript code. You don't need externs.js for EVERYTHING they declare.
The Google Closure compiler will run only on your code, not the external js library.
You need to put externs only when you:
1. Call your cljs function from js code (to avoid calling a munged function name that is not what you have in your js)
2. Call the js library function from your cljs code (to avoid having munged the function name in the function call).
So there's no prior work that needs to be done, you need to put a small externs file exactly for the specific calls you make.
In the process of evaluating the various approaches available to developers to use javascript modules, module loaders and build tools, i'd like some suggestions on what tools you use, and why.
I'm currently after something that is able to:
-encourage modular code
-allow features to be added based on necessity to a given module [think mixins/inheritance]
-produce a BUILD that contains a development release, and at the very minimum a production release with different layers (say, i want a layer [a script] which contains my bootstrap code, module 1, 2 and 3; and then another layer which contains modules 4,5 and 6. This way I can defer loading of code based on what's actually going on in the application.)
-Work [when built for production] in an extremely low bandwidth scenario, with xfer speeds of 1kbps and high latency (think worst case mobile connection over GPRS to get the picture).
I've seen the following:
Using prototype inheritance, as in:
myNS.thing = function(){};
myns.thing.prototype = {
something: "foo"
}
Which can be built by simply taking the contents of this script, and appending it to the next one one wants to include in a production optimized package as a single script. Loaders in this case are simple script tag injections/eval's or similar, based on a known file.
I've also seen other approaches, such as:
function(){
return function(){
something: "foo"
}
}();
Building this already gets more complex because one has to manipulate the script, removing the wrapping self executing function and combining the return values into one object. I am not aware of an "easy" way to use available build tools. The loader approach works the same as above.
Both of these approaches lack dependencies.
Then we have AMD:
define("mymodule", ["dep1"], function(dep1){
return {something: dep1}
});
Some might be nauseated by its indenting, and its "ceremony", but still its quite effective, the google closure compiler knows about it natively, it knows about dependencies, and seems to have widespread adoption across the board. There are a bunch of module loaders available for it (https://docs.google.com/spreadsheet/ccc?key=0Aqln2akPWiMIdERkY3J2OXdOUVJDTkNSQ2ZsV3hoWVE#gid=2) and quite a few build tools as well.
What other options do you know of, or have you seen used in production?
As said, i'm interested in a combination of code syntax, loader tools and build tools. These three must exist and be working together properly. The rest is an academic excercise i'm not interested in.
I personally use RequireJS, an AMD solution. If I'm whipping up quick proof-of-concepts I won't bother setting that up, but the most common source/dep-mapping solutions I know of now include:
RequireJS
CommonJS
Google Closure
YepNope (a conditional loader, can be used in combination with the others)
I have started a boilerplate that uses Require in combination with Backbone to get all the ugly setup code out of the way:
https://github.com/nick-jonas/assemblejs
So you can type assemble init to scaffold the basic project and assemble build to run the compilers and get a final production-ready build.
You might be interested in looking at Grunt a command line tool with various modules for building javascript projects. It uses npm for dependencies and it will work with amd modules, but you can just configure it to concatenate the files you need using grunt-buildconcat.