So I thought about ES modules lately and this is how I think they work:
There's a global object that can't be accessed by a user (let's call it #moduleMap). It maps modules absolute urls to their exports:
#moduleMap = {
"https://something.com/module.mjs": { exportName: "value" }
}
The procedure for evaluating a module is as follows:
module is fetched
module is parsed to ast
ast is modified as follows:
import { x } from "./module1.mjs" => all x references are replaced with
#moduleMap["abs path module1.mjs"].x (and imported module is being fetched)
export const y = "some value" => #moduleMap["abs path to this module"].y = "some value"
(as #Bergi pointed out it's not that simple with exports because exports are not hoisted so dead zone for consts and hoisting for functions are not reflected with just property assignments)
(the above is what's called binding that produce 'live bindings')
the operation is repeated for each module imported
when all modules are fetched, parsed and modified the evaluation starts from the entry module (each module is executed in isolation (~wapped in an IIFE with strict mode on).
As #Bergi pointed out, modules are evaluated eagerly starting from the entry module and evaluating module's imports before the module code itself is executed (with exception for circular dependencies) which practically means the import that was required last will be executed first.
when evaluation reaches any code that accesses #moduleMap["some module"], the browser checks if the module was evaluated
if it wasn't evaluated it is evaluated at this point after which the evaluation returns to this place (now the module (or its exports to be exact) is 'cached' in #moduleMap)
if it was evaluated the import is accessible from #moduleMap["some module"].someImport
for situations where an import is reassigned from other module the browser throws an error
That's basically all is happening AFAIK. Am I correct?
You have a pretty good understanding but there are a few anomalies that should be corrected.
ES Modules
In ECMA-262, all modules will have this general shape:
Abstract Module {
Environment // Holds lexical environment for variables declared in module
Namespace // Exotic object that reaches into Environment to access exported values
Instantiate()
Evaluate()
}
There are a lot of different places that modules can come from, so there are "subclasses" of this Abstract Module. The one we are talking about here is called a Source Text Module.
Source Text Module : Abstract Module {
ECMAScriptCode // Concrete syntax tree of actual source text
RequestedModules // List of imports parsed from source text
LocalExportEntries // List of exports parsed from source text
Evaluate() // interprets ECMAScriptCode
}
When a variable is declared in a module (const a = 5) it is stored in the module's Environment. If you add an export declaration to that, it will also show up in LocalExportEntries.
When you import a module, you are actually grabbing the Namespace object, which has exotic behaviour, meaning that while it appears to be a normal object, things like getting and setting properties might do different things than what you were expecting.
In the case of Module Namespace Objects, getting a property namespace.a, actually looks up that property as a name in the associated Environment.
So if I have two modules, A, and B:
// A
export const a = 5;
// B
import { a } from 'A';
console.log(a);
Module B imports A, and then in module B, a is bound to A.Namespace.a. So whenever a is accessed in module b, it actually looks it up on A.Namespace, which looks it up in A.Environment. (This is how live bindings actually work).
Finally onto the subject of your module map. All modules will be Instantiated before they can be Evaluated. Instantiation is the process of resolving the module graph and preparing the module for Evaluation.
Module Map
The idea of a "module map" is implementation specific, but for the purposes of browsers and node, it looks like this: Module Map <URL, Abstract Module>.
A good way to show how browsers/node use this module map is dynamic import():
async function import(specifier) {
const referrer = GetActiveScriptOrModule().specifier;
const url = new URL(specifier, referrer);
if (moduleMap.has(url)) {
return moduleMap.get(url).Namespace;
}
const module = await FetchModuleSomehow(url);
moduleMap.set(url, module);
return module.Namespace;
}
You can actually see this exact behaviour in Node.js: https://github.com/nodejs/node/blob/e24fc95398e89759b132d07352b55941e7fb8474/lib/internal/modules/esm/loader.js#L98
export const y = "some value" => #moduleMap["abs path to this module"].y = "some value"
(the above is what's called binding that produce 'live bindings')
Yeah, basically - you correctly understood that they all reference the same thing, and when the module reassigns it the importers will notice that.
However, it's a bit more complicated as const y stays a const variable declaration, so it still is subject to the temporal dead zone, and function declarations are still subject to hoisting. This isn't reflected well when you think of exports as properties of an object.
when evaluation reaches any code that accesses #moduleMap["some module"], the browser checks if the module was evaluated
if it wasn't evaluated it is evaluated at this point after which the evaluation returns to this place (now the module (or its exports to be exact) is 'cached' in #moduleMap)
if it was evaluated the import is accessible from #moduleMap["some module"].someImport
No. Module evaluation doesn't happen lazily, when the interpreter comes across the first reference to an imported value. Instead, modules are evaluated strictly in the order of the import statements (starting from the entry module). A module does not start to be evaluated before all of its dependencies have been evaluated (apart from when it has a circular dependency on itself).
Related
In a CommonJS implementation of a module through Node, I have this infantModule.js:
filename: infantModule.js
var infant = function(gender) {
this.gender = gender;
//technically, when passed though this line, I'm born!
};
var infantInstance = new infant('female');
module.exports = infantInstance;
My question is:
When is this module's constructor function really executed, considering other module consuming this infantModule, such as:
filename: index.js -- entry point of the application
var infantPerson = require('./infantModule');
// is it "born" at this line? (1st time it is "required")
console.log(infantPerson);
// or is it "born" at this line? (1st time it is referenced)
Since my infantModule exposes a ready-made instantiated object, all other future requires of this module, by any other modules besides the index.js entry point, will reference this same object, that behaves like a shared instance in the application, is it correct to put it that way?
If there's an additional line of code in index.js at the bottom, such as:
infantInstance.gender = 'male';
Any other module in my application besides index.js, that require infantModule at a future point in time, will get the object with the gender property changed, is it the correct assumption?
require returns a normal object. Nothing magical happens when you access that object.
Specifically, the first time you call require(), Node will execute the entire contents of the required file, and will then return the value of its module.exports property.
I was browsing through the source code here: http://js-dos.com/games/doom2.exe.html and noticed a few things:
if (typeof Module === 'undefined')
{
Module = eval('(function() {try { return Module || {} } catch(e) { return {} }})()');
}
The Module function is defined with an inline script tag
It is later declared again with var in another inline tag, this time it checks if the Module exists.
My question: What is the point of declaring Module with a self invoking function if it'll only try to return the Module again? Hasn't it already been proven that it doesn't exist? Why not just explicitly declare Module as {}?
typeof Module might be undefined if Module is a local variable that happens to contain undefined. This code is meant to support a few cases, Module might be local or global, and defined or undefined. We want to avoid polluting the global scope, so we don't just do Module = ... if it's undefined.
First, the usual case is emscripten-generated code in the global scope. In this case, Module may or may not be defined, and may be local but still undefined, so we need to handle both.
Second, emscripten code may be just a module, like a game that uses ammo.js. In that case, the usage is
function Ammo(Module) {
// emscripten-generated code, uses the Module
return something;
}
so Module in this case is a function local, given as a param already defined for us.
We can't just declare var Module because that means Module is a local variable. So we need eval. For eval, we need a function that returns a value, because we need a try-catch. The try-catch uses Module, and will throw if Module is not a local (regardless of whether it contains undefined or not), which is exactly what we want.
It's possible this code could be simplified, though!
I've been working on a commonjs implementation on ExtendScript Toolkit lately and I am stuck with this "dependency cycle" thing. My code passed most of the commonjs compliance tests except on these: cyclic, determinism, exactExports, monkeys.
The wiki states that:
If there is a dependency cycle, the foreign module may not have finished executing at the time it is required by one of its transitive dependencies; in this case, the object returned by "require" must contain at least the exports that the foreign module has prepared before the call to require that led to the current module's execution.
Can somebody please explain to me further how this specification should be implemented? Do I throw an exception if it detects a dependency cycle?
You can check my code at: https://github.com/madevelopers/estk
Only tested on ExtendScript Toolkit CS6
In CommonJS, you attach the things you're exporting onto an export object. The intention of the spec is that if there is a 'require' statement part way through a file and then part way through that required file there is a require of the original file, then the second require gets the state of the exports object as it is at that point. I'll provide an example.
// file A.js
exports.A1 = {};
exports.A2 = {};
// The object returned from this require call has B1, B2 and B3
var B = require('./B.js');
exports.A3 = B.B1;
And in file B.js:
// fie B.js
exports.B1 = {};
exports.B2 = {};
// This require call causes a cycle. The object returned has only A1 and A2.
// A3 will be added to the exports object later, *after* this file is loaded.
var A = require('./A.js');
exports.B3 = A.A1;
This example would have worked correctly, even though there is a cycle in the code. Here's another example that would work even though it is circular:
var OtherModule = require('./OtherModule.js');
// even if OtherModule requires this file and causes a circular dependency
// this will work, since doAThing is only defined and not called by requiring this
// this file. By the time doAThing is called, OtherModule will have finished
// loading.
exports.doAThing = function() {
return OtherModule.doSomething() + 3;
}
Even though OtherModule.doSomething doesn't exist when this files code is executed and doAThing is defined, as long as doAThing doesn't get called until later, then everything is fine.
I've got some function that allows to merge namespace, very similar to import when the module contains lot's of function (I expose an API with dozens of combinators)
It generates lots of var f = target.f; for every item from the export
function getNamespace(name, exports){
var output='';
for(var item in exports){
output += 'var ' + item + ' = '+name+ '.'+item + ';';
}
return output;
}
and usage:
var paco = require('./paco.js');
eval(paco.getNamespace('paco', paco));
// instead of paco.between(paco.start(),paco.content(),paco.end())
between(start(), content(), end())
Question:
I there a way to 'hide' the eval into the some function ? I don't want neither to mutate global namespace nor to call vm.runInThisContext, just need to add some local variables into the calling context after call function similar to require.
I mean I need something like
import('./paco');
// this should work like this
// var paco = require('./paco.js');
// var between = paco.between;
but without mutation of global and without eval in the calling scope.
tl;dr: No.
In order to understand why this is impossible, it's important to understand what Node is doing behind the scenes.
Let's say we define a function in test.js:
function foo() {
var msg = 'Hello world';
console.log(msg);
}
In traditional browser JavaScript, simply putting that function declaration in a file and pulling the file in with a <script> tag would cause foo to be declared in the global scope.
Node does things differently when you require() a file.
First, it determines exactly which file should be loaded based on a somewhat complex set of rules.
Assuming that the file is JS text (not a compiled C++ addon), Node's module loader calls fs.readFileSync to get the contents of the file.
The source text is wrapped in an anonymous function. test.js will end up actually looking like this:
(function (exports, require, module, __filename, __dirname) {
function foo() {
var msg = 'Hello world';
console.log(msg);
}
});
This should look familiar to anyone who has ever wrapped their own code in an anonymous function expression to keep variables from leaking into global scope in a browser. It should also start making sense how "magic" variables in Node work.
The module loader evals1 the source text from step 3 and then invokes the resulting anonymous function, passing in a fresh exports object. (See Module#_compile.)
1 - Really vm.runInThisContext, which is like eval except it does not have access to the caller's scope
After the anonymous wrapper function returns, the value of module.exports is cached internally and then returned by require. (Subsequent calls to require() return the cached value.)
As we can see, Node implements "modules" by simply wrapping a file's source code in an anonymous function. Thus, it is impossible to "import" functions into a module because JavaScript does not provide direct access to the execution context of a function – that is, the collection of a function's local variables.
In other words, there is no way for us to loop over the local variables of a function, nor is there a way for us to create local variables with arbitrary names like we can with properties of an object.
For example, with objects we can do things like:
var obj = { key: 'value' };
for (var k in obj) ...
obj[propertyNameDeterminedAtRuntime] = someValue;
But there is no object representing the local variables of a function, which would be necessary for us to copy the properties of an object (like the exports of a module) into the local scope of a function.
What you've done is generate code inside the current scope using eval. The generated code declares local variables using the var keyword, which is then injected into the scope where eval was called from.
There is no way to move the eval call out of your module because doing so would cause the injected code to be inserted into a different scope. Remember that JavaScript has static scope, so you're only able to access the scopes lexically containing your function.
The other workaround is to use with, but you should avoid with.
with (require('./paco.js')) {
between(start(), content(), end())
}
with should not be used for two reasons:
It absolutely kills performance because V8 cannot perform name lookup optimizations.
It is deprecated, and is forbidden in strict mode.
To be honest, I'd recommend that rather than doing something tricky with eval, do your future maintainers a favor and just follow the standard practice of assigning a module's exports to a local variable.
If you're typing it that often, make it a single-character name (or use a better editor).
According to this answer Global variables for node.js standard modules? there is global object the same as in browser there is window. So you can add key to that object
function getNamespace(exports) {
for(var item in exports){
global[item] = exports[item];
}
}
and use it as:
paco.getNamespace(paco);
no need for eval at all.
No. It's not possible to modify the local scope from an external module. Reason being, when eval is called in the external module, its context will be the external module, not the scope requiring the module.
In addition, vm.runInThisContext does not have access to the local scope, so that wont help you either.
Since require.js is AMD module loader, while defining a new module like:
define(["jquery"],function($){
var _private;
var obj = {
pubFunc:...
}
return obj;
});
or
define(["jquery","exports"],function($,exports){
var pubFunc;
exports.pubFunc = pubFunc;
});
Is it the second is inappropriate in Require.js's defining new module?
TL;DR: the first is how RequireJS has been designed to work (use that one); the second isn't.
More info:
It looks like you're confusing RequireJS with Node's require.
In the second, you're almost reversing the direction of the dependency-chain by exporting backwards (into what should be a dependency).
This will actually work most of the time:
exports.pubFunc = pubFunc;
insofar that:
JavaScript Objects are passed by reference, so the property will be added to the exports Object held by RequireJS; and
RequireJS will continue to distribute the modified exports Object whenever exports is required.
However, this is entirely reliant on RequireJS's caching. As an optimisation, RequireJS stores modules' exports, rather than loading the file every time; hence the cached version of exports is redistributed (complete with the property you added).
Consequently, this would break when the cached Object is not passed around, such as in different RequireJS contexts.
You should be treating imports as immutable, even if you can modify them.