I've been reading the Medium.js Open Source Project that tries to mock Medium.com.
While reading, I noticed a pattern in the code. Mainly, an IIFE is called every so often to do a variety of stuff.
(function(Medium) {
"use strict";
//do stuff
})(Medium);
(function(Medium) {
"use strict";
//do stuff
})(Medium);
(function(Medium) {
"use strict";
//do stuff
etc.
Why is the code compartmentalized like this? Couldn't it all be invoked at the same time, why call a series of IFFE statements? Is this a common pattern? You can view my link above for the full code I am referring to.
If you look through the source files, you will see that each individual component is developed in a separate JavaScript file. A build script is used to concatenate all of these scripts together, and produce the file you are looking at.
By developing it this way, you could build your own custom build with only the components you need. By wrapping groups of functions together in an IIFE, he can also avoid setting "use strict" globally, and for each individual function. If there was a need to a variable to be shaded between a set of API's, this allows a variable to be declared inside the IIFE, without colliding with variables in other modules or the global space.
But the main advantage is the developer can develop the library as multiple different files, and you can optionally only include those you need in your project.
If you look in the src/Medium directory you can see that each of these IIFE's are separate files and many of them have matching files in the tests directory.
They separate their code like this so they can split their code up into small testable modules and they use IIFE's to extend an existin Medium object with new functionality.
The Medium.js file is a result of a build process that concatinates all the files in the src/Medium folder.
Related
Uncompressed jQuery file: http://code.jquery.com/jquery-2.0.3.js
jQuery Source code: https://github.com/jquery/jquery/blob/master/src/core.js
What are they doing to make it seem like the final output is not using Require.js under the hood? Require.js examples tells you to insert the entire library into your code to make it work standalone as a single file.
Almond.js, a smaller version of Require.js also tell you to insert itself into your code to have a standalone javascript file.
When minified, I don't care for extra bloat, it's only a few extra killobytes (for almond.js), but unminified is barely readable. I have to scroll all the way down, past almond.js code to see my application logic.
Question
How can I make my code to be similar to jQuery, in which the final output does not look like a Frankenweenie?
Short answer:
You have to create your own custom build procedure.
Long answer
jQuery's build procedure works only because jQuery defines its modules according to a pattern that allows a convert function to transform the source into a distributed file that does not use define. If anyone wants to replicate what jQuery does, there's no shortcut: 1) the modules have to be designed according to a pattern which will allow stripping out the define calls, and 2) you have to have a custom conversion function. That's what jQuery does. The entire logic that combines the jQuery modules into one file is in build/tasks/build.js.
This file defines a custom configuration that it passes to r.js. The important option are:
out which is set to "dist/jquery.js". This is the single
file produced by the optimization.
wrap.startFile which is set to "src/intro.js". This file
will be prepended to dist/jquery.js.
wrap.endFile which is set to "src/outro.js". This file will
be appended to dist/jquery.js.
onBuildWrite which is set to convert. This is a custom function.
The convert function is called every time r.js wants to output a module into the final output file. The output of that function is what r.js writes to the final file. It does the following:
If a module is from the var/ directory, the module will be
transformed as follows. Let's take the case of
src/var/toString.js:
define([
"./class2type"
], function( class2type ) {
return class2type.toString;
});
It will become:
var toString = class2type.toString;
Otherwise, the define(...) call is replace with the contents of the callback passed to define, the final return statement is stripped and any assignments to exports are stripped.
I've omitted details that do not specifically pertain to your question.
You can use a tool called AMDClean by gfranko https://www.npmjs.org/package/amdclean
It's much simpler than what jQuery is doing and you can set it up quickly.
All you need to do is to create a very abstract module (the one that you want to expose to global scope) and include all your sub modules in it.
Another alternative that I've recently been using is browserify. You can export/import your modules the NodeJS way and use them in any browser. You need to compile them before using it. It also has gulp and grunt plugins for setting up a workflow. For better explanations read the documentations on browserify.org.
TL;DR at the bottom of the question, for those who don't want to read all my junk.
My current method of JavaScript modularization is to simply create a global anchor, like "csc" (my company acronym) and then start tacking modules onto that.
So I'll end up with a global structure like:
csc.
Utils
Map
.Geolocation
Storage.
Cookie
DB
And each of these files are stored in a directory structure:
csc.js
csc.Utils.js
csc.Map.js
csc.Storage.js
This eliminates the need to load my entire codebase all the time.
I'm trying to transition toward using RequireJS, but the methodology employed by that library seems to be a bit different.
In order to maintain my namespaced structure, I could define modules around all of my modules, but still tack them into the global "csc" reference. However this seems to go against the core principles of Require.
Without keeping them global, however, I lose my nice namespacing, such as "csc.Map.Geolocation" because I now have a bunch of separate variables, like so:
require(['csc', 'csc.Utils', 'csc.Map'], function (csc, utils, map) {
});
I'll strip my question down to its essence, in case my horrible description above didn't suffice:
Is there a way, perhaps within the module definitions, to combine these three variables back into the structure defined at the top of this question? Or am I going about this all wrong, and should I instead be adhering to the Require way of doing things?
I'd love to follow the Require methodology, but I also love the ability to have all of my modules chainable and namespaced.
Don't be discouraged by the documentation's example path hierarchy, notice that require does not strictly enforce any particular convention. You are free to design and follow your own convention.
Utils, Map, and Storage all become directories. The base actions that they perform should be named module.js in their respective directories, like so:
core.js
Utils/
module.js
Map/
module.js
geolocation.module.js
Storage/
module.js
cookie.module.js
db.module.js
The module.js files include and return their children. Here is an example of Storage/module.js:
require(["Storage/cookie", "Storage/db"], function (cookie, db) {
var Storage = {};
// do whatever you need with Storage
Storage.cookie = cookie
Storage.db = db
return Storage
});
Notice also the core.js file in root. This file would work just the same,
require(["Utils/module", "Storage/module", "Map/module"], function (utils, storage, map) {
var Core = {};
// do whatever you need with Storage
Core.Utils = utils
Core.Storage = storage
Core.Map = map
return Core
});
Now, require core.js wherever you will need access to these modules (every file basically). Yes, this will load all of the involved files all of the time in development, but when you compile your app, all of your variables will be inside the scope of an anonymous function, and not directly accessible via the window object.
Again, tweak this as you see fit, it's your own convention.
I'm writing client-side code and would like to write multiple, modular JS files that can interact while preventing global namespace pollution.
index.html
<script src="util.js"></script>
<script src="index.js"></script>
util.js
(function() {
var helper() {
// Performs some useful utility operation
}
});
index.js
(function () {
console.log("Loaded index.js script");
helper();
console.log("Done with execution.");
})
This code nicely keeps utility functions in a separate file and does not pollute the global namespace. However, the helper utility function will not be executed because 'helper' exists inside a separate anonymous function namespace.
One alternative approach involves placing all JS code inside one file or using a single variable in the global namespace like so:
var util_ns = {
helper: function() {
// Performs some useful utility operation.
},
etc.
}
Both these approaches have cons in terms of modularity and clean namespacing.
I'm used to working (server-side) in Node.js land where I can 'require' one Javascript file inside another, effectively injecting the util.js bindings into the index.js namespace.
I'd like to do something similar here (but client-side) that would allow code to be written in separate modular files while not creating any variables in the global namespace while allowing access to other modules (i.e. like a utility module).
Is this doable in a simple way (without libraries, etc)?
If not, in the realm of making client-side JS behave more like Node and npm, I'm aware of the existence of requireJS, browserify, AMD, and commonJS standardization attempts. However, I'm not sure of the pros and cons and actual usage of each.
I would strongly recommend you to go ahead with RequireJS.
The modules support approach (without requires/dependencies):
// moduleA.js
var MyApplication = (function(app) {
app.util = app.util || {};
app.util.hypotenuse = function(a, b) {
return Math.sqrt(a * a + b * b);
};
return app;
})(MyApplication || {});
// ----------
// moduleB.js
var MyApplication = (function(app) {
app.util = app.util || {};
app.util.area = function(a, b) {
return a * b / 2;
};
return app;
})(MyApplication || {});
// ----------
// index.js - here you have to include both moduleA and moduleB manually
// or write some loader
var a = 3,
b = 4;
console.log('Hypotenuse: ', MyApplication.util.hypotenuse(a, b));
console.log('Area: ', MyApplication.util.area(a, b));
Here you're creating only one global variable (namespace) MyApplication, all other stuff is "nested" into it.
Fiddle - http://jsfiddle.net/f0t0n/hmbb7/
**One more approach that I used earlier in my projects - https://gist.github.com/4133310
But anyway I threw out all that stuff when started to use RequireJS.*
You should check out browserify, which will process a modular JavaScript project into a single file. You can use require in it as you do in node.
It even gives a bunch of the node.js libs like url, http and crypto.
ADDITION: In my opinion, the pro of browserify is that it is simply to use and requires no own code - you can even use your already written node.js code with it. There's no boilerplate code or code change necessary at all, and it's as CommonJS-compliant as node.js is. It outputs a single .js that allows you to use require in your website code, too.
There are two cons to this, IMHO: First is that two files that were compiled by browserify can override their require functions if they are included in the same website code, so you have to be careful there. Another is of course you have to run browserify every time to make change to the code. And of course, the module system code is always part of your compiled file.
I strongly suggest you try a build tool.
Build tools will allow you to have different files (even in different folders) when developing, and concatenating them at the end for debugging, testing or production. Even better, you won't need to add a library to your project, the build tool resides in different files and are not included in your release version.
I use GruntJS, and basically it works like this. Suppose you have your util.js and index.js (which needs the helper object to be defined), both inside a js directory. You can develop both separately, and then concatenate both to an app.js file in the dist directory that will be loaded by your html. In Grunt you can specify something like:
concat: {
app: {
src: ['js/util.js', 'js/index.js'],
dest: 'dist/app.js'
}
}
Which will automatically create the concatenation of the files. Additionally, you can minify them, lint them, and make any process you want to them too. You can also have them in completely different directories and still end up with one file packaged with your code in the right order. You can even trigger the process every time you save a file to save time.
At the end, from HTML, you would only have to reference one file:
<script src="dist/app.js"></script>
Adding a file that resides in a different directory is very easy:
concat: {
app: {
src: ['js/util.js', 'js/index.js', 'js/helpers/date/whatever.js'],
dest: 'dist/app.js'
}
}
And your html will still only reference one file.
Some other available tools that do the same are Brunch and Yeoman.
-------- EDIT -----------
Require JS (and some alternatives, such as Head JS) is a very popular AMD (Asynchronous Module Definition) which allows to simply specify dependencies. A build tool (e.g., Grunt) on the other hand, allows managing files and adding more functionalities without relying on an external library. On some occasions you can even use both.
I think having the file dependencies / directory issues / build process separated from your code is the way to go. With build tools you have a clear view of your code and a completely separate place where you specify what to do with the files. It also provides a very scalable architecture, because it can work through structure changes or future needs (such as including LESS or CoffeeScript files).
One last point, having a single file in production also means less HTTP overhead. Remember that minimizing the number of calls to the server is important. Having multiple files is very inefficient.
Finally, this is a great article on AMD tools s build tools, worth a read.
So called "global namespace pollution" is greatly over rated as an issue. I don't know about node.js, but in a typical DOM, there are hundreds of global variables by default. Name duplication is rarely an issue where names are chosen judiciously. Adding a few using script will not make the slightest difference. Using a pattern like:
var mySpecialIdentifier = mySpecialIdentifier || {};
means adding a single variable that can be the root of all your code. You can then add modules to your heart's content, e.g.
mySpecialIdentifier.dom = {
/* add dom methods */
}
(function(global, undefined) {
if (!global.mySpecialIdentifier) global.mySpecialIdentifier = {};
/* add methods that require feature testing */
}(this));
And so on.
You can also use an "extend" function that does the testing and adding of base objects so you don't replicate that code and can add methods to base library objects easily from different files. Your library documentation should tell you if you are replicating names or functionality before it becomes an issue (and testing should tell you too).
Your entire library can use a single global variable and can be easily extended or trimmed as you see fit. Finally, you aren't dependent on any third party code to solve a fairly trivial issue.
You can do it like this:
-- main.js --
var my_ns = {};
-- util.js --
my_ns.util = {
map: function () {}
// .. etc
}
-- index.js --
my_ns.index = {
// ..
}
This way you occupy only one variable.
One way of solving this is to have your components talk to each other using a "message bus". A Message (or event) consists of a category and a payload. Components can subscribe to messages of a certain category and can publish messages. This is quite easy to implement, but there are also some out of the box-solutions out there. While this is a neat solution, it also has a great impact on the architecture of your application.
Here is an example implementation: http://pastebin.com/2KE25Par
http://brunch.io/ should be one of the simplest ways if you want to write node-like modular code in your browser without async AMD hell. With it, you’re also able to require() your templates etc, not just JS files.
There are a lot of skeletons (base applications) which you can use with it and it’s quite mature.
Check the example application https://github.com/paulmillr/ostio to see some structure. As you may notice, it’s written in coffeescript, but if you want to write in js, you can — brunch doesn’t care about langs.
I think what you want is https://github.com/component/component.
It's synchronous CommonJS just like Node.js,
it has much less overhead,
and it's written by visionmedia who wrote connect and express.
I'm working on a project with Node.js and the server-side code is becoming large enough that I would like to split it off into multiple files. It appears this has been done client-side for ages, development is done by inserting a script tag for each file and only for distribution is something like "Make" used to put everything together. I realize there's no point in concatting all the server-side code so I'm not asking how to do that. The closest thing I can find to use is require(), however it doesn't behave quite like script does in the browser in that require'd files do not share a common namespace.
Looking at some older Node.js projects, like Shooter, it appears this was once not the case, that or I'm missing something really simple in my code. My require'd files cannot access the global calling namespace at compile time nor run time. Is there any simple way around this or are we forced to make all our require'd JS files completely autonomous from the calling scope?
You do not want a common namespace because globals are evil. In node we define modules
// someThings.js
(function() {
var someThings = ...;
...
module.exports.getSomeThings = function() {
return someThings();
}
}());
// main.js
var things = require("someThings");
...
doSomething(things.getSomeThings());
You define a module and then expose a public API for your module by writing to exports.
The best way to handle this is dependency injection. Your module exposes an init function and you pass an object hash of dependencies into your module.
If you really insist on accessing global scope then you can access that through global. Every file can write and read to the global object. Again you do not want to use globals.
re #Raynos answer, if the module file is next to the file that includes it, it should be
var things = require("./someThings");
If the module is published on, and installed through, npm, or explicitly put into the ./node_modules/ folder, then the
var things = require("someThings");
is correct.
I am currently maintaining a large number of JS files and the dependency issue is growing over my head. Right now I have each function in a separate file and I manually maintain a database to work out the dependencies between functions.
This I would like to automate. For instance if I have the function f
Array.prototype.f = function() {};
which is referenced in another function g
MyObject.g = function() {
var a = new Array();
a.f();
};
I want to be able to detect that g is referencing f.
How do I go about this? Where do I start? Do I need to actually write a compiler or can I tweak Spidermonkey for instance? Did anyone else already do this?
Any pointers to get me started is very much appreciated
Thanks
Dok
Whilst you could theoretically write a static analysis tool that detected use of globals defined in other files, such as use of MyObject, you couldn't realistically track usage of prototype extension methods.
JavaScript is a dynamically-typed language so there's no practical way for any tool to know that a, if passed out of the g function, is an Array, and so if f() is called on it there's a dependency. It only gets determined what variables hold what types at run-time, so to find out you'd need an interpreter and you've made yourself a Turing-complete problem.
Not to mention the other dynamic aspects of JavaScript that completely defy static analysis, such as fetching properties by square bracket notation, the dreaded eval, or strings in timeouts or event handler attributes.
I think it's a bit of a non-starter really. You're probably better of tracking dependencies manually, but simplifying it by grouping related functions into modules which will be your basic unit of dependency tracking. OK, you'll pull in a few more functions that you technically need, but hopefully not too much.
It's also a good idea to namespace each module, so it's very clear where each call is going, making it easy to keep the dependencies in control manually (eg. by a // uses: ThisModule, ThatModule comment at the top).
Since extensions of the built-in prototypes are trickier to keep track of, keep them down to a bare minimum. Extending eg. Array to include the ECMAScript Fifth Edition methods (like indexOf) on browsers that don't already have them is a good thing to do as a basic fixup that all scripts will use. Adding completely new arbitrary functionality to existing prototypes is questionable.
Have you tried using a dependency manager like RequireJS or LabJS? I noticed no one's mentioned them in this thread.
From http://requirejs.org/docs/start.html:
Inside of main.js, you can use require() to load any other scripts you
need to run:
require(["helper/util"], function(util) {
//This function is called when scripts/helper/util.js is loaded.
//If util.js calls define(), then this function is not fired until
//util's dependencies have loaded, and the util argument will hold
//the module value for "helper/util".
});
You can nest those dependencies as well, so helper/util can require some other files within itself.
As #bobince already suggested, doing static analysis on a JavaScript program is a close to impossible problem to crack. Google Closure compiler does it to some extent but then it also relies on external help from JSDoc comments.
I had a similar problem of finding the order in which JS files should be concatenated in a previous project, and since there were loads of JS files, manually updating the inclusion order seemed too tedious. Instead, I stuck with certain conventions of what constitutes a dependency for my purposes, and based upon that and using simple regexp :) I was able to generated the correct inclusion order.
The solution used a topological sort algorithm to generate a dependency graph which then listed the files in the order in which they should be included to satisfy all dependencies. Since each file was basically a pseudo-class using MooTools syntax, there were only 3 ways dependencies could be created for my situation.
When a class Extended some other class.
When a class Implemented some other class.
When a class instantiated an object of some other class using the new keyword.
It was a simple, and definitely a broken solution for general purpose usage but it served me well. If you're interested in the solution, you can see the code here - it's in Ruby.
If your dependencies are more complex, then perhaps you could manually list the dependencies in each JS file itself using comments and some homegrown syntax such as:
// requires: Array
// requires: view/TabPanel
// requires: view/TabBar
Then read each JS file, parse out the requires comments, and construct a dependency graph which will give you the inclusion order you need.
It would be nice to have a tool that can automatically detect those dependencies for you and choose how they are loaded. The best solutions today are a bit cruder though. I created a dependency manager for my particular needs that I want to add to the list (Pyramid Dependency Manager). It has some key features which solve some unique use cases.
Handles other files (including inserting html for views...yes, you can separate your views during development)
Combines the files for you in javascript when you are ready for release (no need to install external tools)
Has a generic include for all html pages. You only have to update one file when a dependency gets added, removed, renamed, etc
Some sample code to show how it works during development.
File: dependencyLoader.js
//Set up file dependencies
Pyramid.newDependency({
name: 'standard',
files: [
'standardResources/jquery.1.6.1.min.js'
]
});
Pyramid.newDependency({
name:'lookAndFeel',
files: [
'styles.css',
'customStyles.css',
'applyStyles.js'
]
});
Pyramid.newDependency({
name:'main',
files: [
'createNamespace.js',
'views/buttonView.view', //contains just html code for a jquery.tmpl template
'models/person.js',
'init.js'
],
dependencies: ['standard','lookAndFeel']
});
Html Files
<head>
<script src="standardResources/pyramid-1.0.1.js"></script>
<script src="dependencyLoader.js"></script>
<script type="text/javascript">
Pyramid.load('main');
</script>
</head>
It does require you to maintain a single file to manage dependencies. I am thinking about creating a program that can automatically generate the loader file for you based on includes in the header but since it handles many different types of dependencies, maintaining them in one file might actually be better.
JSAnalyse uses static code analysis to detect dependencies between javascript files:
http://jsanalyse.codeplex.com/
It also allows you to define the allowed dependencies and to ensure it during the build, for instance. Of course, it cannot detect all dependencies because javascript is dynamic interpretet language which is not type-safe, like already mentioned. But it at least makes you aware of your javascript dependency graph and helps you to keep it under control.
I have written a tool to do something like this: http://github.com/damonsmith/js-class-loader
It's most useful if you have a java webapp and you structure your JS code in the java style. If you do that, it can detect all of your code dependencies and bundle them up, with support for both runtime and parse-time dependencies.