In the process of evaluating the various approaches available to developers to use javascript modules, module loaders and build tools, i'd like some suggestions on what tools you use, and why.
I'm currently after something that is able to:
-encourage modular code
-allow features to be added based on necessity to a given module [think mixins/inheritance]
-produce a BUILD that contains a development release, and at the very minimum a production release with different layers (say, i want a layer [a script] which contains my bootstrap code, module 1, 2 and 3; and then another layer which contains modules 4,5 and 6. This way I can defer loading of code based on what's actually going on in the application.)
-Work [when built for production] in an extremely low bandwidth scenario, with xfer speeds of 1kbps and high latency (think worst case mobile connection over GPRS to get the picture).
I've seen the following:
Using prototype inheritance, as in:
myNS.thing = function(){};
myns.thing.prototype = {
something: "foo"
}
Which can be built by simply taking the contents of this script, and appending it to the next one one wants to include in a production optimized package as a single script. Loaders in this case are simple script tag injections/eval's or similar, based on a known file.
I've also seen other approaches, such as:
function(){
return function(){
something: "foo"
}
}();
Building this already gets more complex because one has to manipulate the script, removing the wrapping self executing function and combining the return values into one object. I am not aware of an "easy" way to use available build tools. The loader approach works the same as above.
Both of these approaches lack dependencies.
Then we have AMD:
define("mymodule", ["dep1"], function(dep1){
return {something: dep1}
});
Some might be nauseated by its indenting, and its "ceremony", but still its quite effective, the google closure compiler knows about it natively, it knows about dependencies, and seems to have widespread adoption across the board. There are a bunch of module loaders available for it (https://docs.google.com/spreadsheet/ccc?key=0Aqln2akPWiMIdERkY3J2OXdOUVJDTkNSQ2ZsV3hoWVE#gid=2) and quite a few build tools as well.
What other options do you know of, or have you seen used in production?
As said, i'm interested in a combination of code syntax, loader tools and build tools. These three must exist and be working together properly. The rest is an academic excercise i'm not interested in.
I personally use RequireJS, an AMD solution. If I'm whipping up quick proof-of-concepts I won't bother setting that up, but the most common source/dep-mapping solutions I know of now include:
RequireJS
CommonJS
Google Closure
YepNope (a conditional loader, can be used in combination with the others)
I have started a boilerplate that uses Require in combination with Backbone to get all the ugly setup code out of the way:
https://github.com/nick-jonas/assemblejs
So you can type assemble init to scaffold the basic project and assemble build to run the compilers and get a final production-ready build.
You might be interested in looking at Grunt a command line tool with various modules for building javascript projects. It uses npm for dependencies and it will work with amd modules, but you can just configure it to concatenate the files you need using grunt-buildconcat.
Related
In our AngularJS application we currently have a lot (400+) of files, which get included via <script>-tags. The order these files is something like this:
AngularJS script files
3rd party plugins / modules
business Logic files
modules
services
controllers / components / directives
We would like to move to a better approach utilizing a module bundler and TypeScript. New files are already written in TypeScript but don't make use of import/export. In order to make things easier, we could convert every JavaScript file into a TypeScript file and fix the resulting errors in a feasible time.
However, before we do this, we would like to have a viable strategy on how we could gradually make use of import/export. I'm thinking of something like rewriting one module from time to time, starting with modules deep down in the dependency tree.
However I was not able to achieve this, but I'm quiet sure that others already had to solve this before.
Once you decide on a module bundler, you'll need to learn about its facilities for (1) allowing other JavaScript on the page to access things defined in the bundle and (2) allowing the bundle to access things defined by other JavaScript on the page. (If you're able to migrate in strict dependency order, you might never need #2.) For example, for Webpack, #1 would be the library* output options and #2 would be externals. Then just move the code into modules little by little, adjusting the configuration as necessary so that each part of the code has access to the things it needs from the other part of the code. Since Webpack only supports a single library export module, during the transition, you may have to maintain a dummy module that just re-exports all modules that you need to access from code outside the bundle. This is a little tedious and represents extra work that you wouldn't have to do if you migrated all at once, but you may decide it's worth paying that cost in order to be able to migrate gradually.
If you have issues getting type information from TypeScript module files in non-module TypeScript files, see this answer for a workaround.
I have a two different javascript files that I am using with my GAE Python app, and each of these depend on common functionality that I don't want to repeat in two different places.
In my development environment, I'd like to have three files:
script1.js
script2.js
library.js
where each of script1.js and script2.js need functionality in library.js.
My javascript uses the "revealing module pattern", e.g.,
var my_module = (function() {
var my = {};
my.do_stuff = function() {...};
return my;
})();
But in production, I want clients to be able to download one static file that has all needed javascript. For example a client could download script1all.js that will include script1.js and library.js.
How can I implement this in GAE/P? Is there an existing tool that does this?
I've been reading about javascript package managers (npm, bower, etc), but these seem related to handling third party javascript, where I just need to package my own javascript.
I've spent the last 3 hours reading about this, and I'm somewhat blown away by the complexity and variations that exist with javascript modules and bundling.
I found tools like Browserify and Webpack, but as I learned about them I had to refresh myself about javascript modules. This is a great resource for understanding javascript module options.
For my javascript, I've been using the "revealing module pattern" discussed in the linked article. For this style of javascript modules, it seems that the simple concatenate and minify approach is a perfectly good solution. This is another article that lays out bundling options.
So after hours of educating myself, I'll do the simple concatenate and minify approach, and I'll eventually upgrade to ES6.
P.S.: This answer was also very helpful in understanding the different categories of javascript processing that one might need.
I would like to create a (require.js style) AMD module that can be used in the browser and in node. What is the best way to do this? I keep seeing references to r.js, but still not 100% sure on how to use it, or if it is necessary for my situation.
Also, when including this module in node, do I still run require('module-name'), or will this change as well?
First things first: AMD basics, What all you can do with them, How to optimize them
In extremely simple terms
AMD modules are reusable JS code. Think of them as functions kept in separate files.
AMD loaders are central functions which call all other functions (modules). Think of them as "main" method in C or Java.
RequireJS is a framework which pulls all this scattered code and stitches it in a usable form.
RequireJS works in a browser. As a result, all your code is "stitched" together in a web browser.
r.js works offline (in a web server or on your development machine) to "stitch" all the code offline so that when it reaches a web browser, it's already "stitched".
Use of RequireJS lib is a must no matter you want to "stitch" your code in browser or you want to serve your code "pre-stitched".
Use of r.js is optional. It's needed only if you want to increase performance and decrease HTTP calls.
I'm trying to figure out how to best manage Javascript file dependencies and have that drive the packaging of a 100% front-end app. In short, I'm building an application using backbone.js along with some other libraries. I want an organized codebase and would like to be able to declare dependencies within each file. Ideally those declarations would a) drive the order in which files are loaded by the browser (while in development I want the files to load separately) and drive the order in which the packaging scripts load the scripts for concatenation (I'm aiming to serve a single file for the entire app).
I've been reading on requirejs and commonjs but I'm not convinced.
I have a simple shell script right now that uses cat <file> <file> <file> <file> > concatenated.file to do what I want but it's a pain to maintain that list of files up to date and in the right order. It'd be much easier to be able to declare the dependency at the begining of each javascript file and have the packager and loaders be smart about using that information to concatenate/load scripts.
Any suggestions?
Thanks you,
Luis
I am partial to stealjs myself. It's part of JavascriptMVC but no reason why you can't use it with Backbone.js
The nice part about this one is that it builds your app for production including minifying your css and js and neatly packing all of it into 2 files: production.css and production.js.
It can handle loading non JS files too so you can do things like steal('somefile.css').then(function() {...});
For files, its very much like you would do in other languages:
steal(dep1, dep2, dep3).then(function () {
// code
});
For complex frontend apps Asynchronous Module Definition (AMD) format is best choice. And it's alot of great loaders that supports AMD (curl.js, RequireJS).
I recomend this articles to learn about modern approaches in javascript dependecy management:
Writing Modular JavaScript With AMD, CommonJS & ES Harmony
Why AMD?
For packaging take into account CommonJS specifications, there are few implementations and it's a matter of taste, but in any case I recommend to choose tools, that is compliant with some of that specifications.
It'd be much easier to be able to declare the dependency at the begining of each javascript file and have the packager and loaders be smart about using that information to concatenate/load scripts.
I have had the same idea several months ago and are working on a dependency resolver for my Resource Builder which already makes it easier for me (including the need to distinuish between development and deployed version, with the optional debug parameter).
JsDoc Toolkit (and related efforts), which syntax is supported e. g. by Eclipse JSDT, provides a #requires tag, so you could use that. But resolving dependencies is still not a trivial task (as you can see in ResourceBuilder::resolveDeps()). (The ultimate goal is to use static code analysis to resolve dependencies automatically, without any tags.)
This would reduce the current
<script type="text/javascript" src="builder?src=object,types,dom,dom/css"></script>
to a mere
<script type="text/javascript" src="builder?src=dom/css"></script>
As for asynchronous loaders: The good thing about asynchronous loaders is that they are fast. The bad thing about asynchronous loaders is that – if they work; they are all based on a non-standard approach – they are so fast that you cannot be sure that the features the scripts provide are available in following scripts. So you have to have your code executed by their listeners. I recommend avoiding them unless you really have features in your application that are only needed on demand.
We are creating a javascript library from scratch and we need some utilities functions, just like what dojo provides, e.g. xhr wrapper, simple inheritances and array manipulation, etc. We don't want to write these functions ourselves from scratch, but we also don't want to just copy and paste code from dojo.
Dojo provides the custom build mechanism, but it works on the module level. For example, in the custom build, you can specify to only add the lang module in the dojo base. But we may only need the dojo.hitch function in the lang module, but not the dojo.clone function that also included in the lang module. We have a tight constraint on the size of the library file, so we need to remove any unused code.
What we are seeking is a way to extract certain functions from a module and build these functions into a single javascript file. For example, say we need dojo.mixin, dojo.declare and dojo.hitch functions, then these functions' declaration and their dependent inner functions should be built into a single file.
Any suggestions?
Update from my test result
Thanks Stephen Chung for your great suggestion on how to solve this problem.
So basically I created a custom dojo build with four modules : dojo._base.xhr, dojo._base.json, dojo._base.declare and dojo._base.lang. The built dojo.js file is 149kb, after using Google Closure compiler with advanced mode, the size has reduced to 24kb and the unused functions have been removed. If we want to keep some functions, just add another javascript file like below:
window['dojo'] = dojo;
window['dojo']['requireLocalization'] = dojo.requireLocalization;
window['dojo']['moduleUrl'] = dojo.moduleUrl;
and compile it with dojo.js.
Unfortunately the size is still a little bigger. We have a size limit of 40k in total. But this is still a very good solution for other cases.
Dojo Core is written in a very compact manner, with a lot of cross callings (i.e. one function calling other functions) in order to save download bytes and reduce code size.
Therefore, you need a minimal set of Dojo Core functions, together with the functions that it calls.
In other words, what you need is a build that includes Dojo Core, but then removes any dead-code that is never called.
I'd suggest that you don't go about creating a minimal-footprint library by writing it from scratch yourself. What you need is to use the Closure Compiler's Advanced Mode to process your Dojo application. It removes dead code, optimizes the entire app, and fully obfuscates it.
And yes, it is possible to use the Closure Compiler with Dojo in Advanced Mode -- Dojo is probably the only popular library that can do it. Read it here.
Some statistics: A Closure-compiled Dojo app is typically around 25-30% smaller than the equivalent Dojo app in a Shrinksafe build. It depends on how many Dojo Core features you use -- typically you may reduce Dojo Core up to 40% of the original size (i.e. reduce code size by up to 60%) from dead code removal alone. Further reductions are more difficult, as you may have to start removing features like the Dojo loader etc.
Are you fixed on dojo? Google's closure library offers a very similar package management system to dojo, as well as all the necessary scripts and tools you need to compile the javascript down to a single "executable". The closure library compiler is arguably one of the most advanced javascript minifiers you will find so if tight, concise javascript is your goal, this tool could help tremendously.