I have the following situation.
EntryA
require("./test.js");
EntryB
require("./test.js");
test.js
module.exports = "something";
I want tuse webpack to compile these javascript files. On the html page i just want to include the EntryA.js. Every thing that is common between EntryA and EntryB should come in a sperate file. Now, when test.js is required, that common file should be downloaded from the net.
is this possible and ghow should i proceed?
Webpack offers two ways to split out your JS.
Use multiple entry points (As you have done).
Define split points via require.ensure (or System.import in Webpack 2).
Approach 1 is really intended to be used by traditional 'multi' page websites and requires you to import the appropriate entry point on the correct page. NOTE: You can use the CommonsChunkPlugin to extract out all the shared code but it still must be manually referenced.
Approach 2 is more geared towards single page apps and will automatically load in new scripts as necessary.
Related
Using Webpack 5, in one entry file I have multiple dynamic imports setup like so: import(/* webpackExports: ["moduleName"] */ 'package name'); each of these imports are done under different conditions. Likely on a page load only one of the 5 or so imports should actually be called. These imports are spread over multiple different packages.
The problem is for some reason Webpack is setting all these imports into one, rather larger, chunk instead of individual chunks as I would expect them to be.
Just to make sure there wasn't some cross dependency conflict or something I tried clearing the imported modules out to be practically nothing for testing purposes. I've also tried using webpackChunkName. Nothing has worked.
Why would webpack be making these one chunk instead of multiple? How can I fix this?
Thanks.
The issue is related to "state". Conditional loading does not mean that we can have multiple states for the same module. Issues with side effects and live bindings force webpack (or any loader) to aggregate all the exports of a module that are being used, conditionally or not, into one chunk.
Therefore, it's a limitation with tree shaking of dynamically imported modules.
I have written more about this here: https://blog.hotstar.com/how-to-dynamically-import-esmodules-and-tree-shake-them-too-aa24ee4885f5
I'm trying to load multiple modules on the fly via chokidar (watchdog) using Meteor 1.6 beta, however after doing extensive research on the matter I just can't seem to get it to work.
From what I gather require by design will not take in anything other than static strings, i.e.
require("test/string/here")
Since if I try:
var path = "test/string/here"
require(path)
I just get Error: Cannot find module, even though the strings are identical.
Now the thing is I'm uncertain how to go on about this, am I really forced to either use import or static strings when using meteor or is there some workaround this?
watchdog(cmddir, (dir) => {
match = "." + regex_cmd.exec(dir);
match = dir;
loader.emit("loadcommand", match)
});
loader.on('loadcommand', (file) => {
require(file);
});
There is something intrinsically weird in what you describe.
chokidar is used to watch actual files and folders.
But Meteor compiles and bundles your code, resulting in an app folder after build that is totally different from your project structure.
Although Meteor now supports dynamic imports, the mechanism is internal to Meteor and does not rely on your actual project files, but on Meteor built ones.
If you want to dynamically require files like in Node, including with dynamically generated module path, you should avoid import and require statements, which are automatically replaced by Meteor built-in import mechanism. Instead you would have to make up your own loading function, taking care of the fact that your app built folder is different from your project folder.
That may work for example if your server is watching files and/or folders in a static location, different from where your app will be running.
In the end, I feel this is a sort of XY problem: you have not described your objective in the first place, and the above issue is trying to solve a weird solution that does not seem to fit how Meteor works, hence which may not be the most appropriate solution for your implicit objective.
#Sashko does a great job of explaining Meteor's dynamic imports here. There are also docs
A dynamic import is a function that returns a promise instead of just importing statically at build time. Example:
import('./component').then((MyComponent) => {
render(MyComponent);
});
The promise runs once the module has been loaded. If you try to load the module repeatedly then it only gets loaded once and is immediately available on subsequent requests.
afaict you can use a variable for the string to import.
Uncompressed jQuery file: http://code.jquery.com/jquery-2.0.3.js
jQuery Source code: https://github.com/jquery/jquery/blob/master/src/core.js
What are they doing to make it seem like the final output is not using Require.js under the hood? Require.js examples tells you to insert the entire library into your code to make it work standalone as a single file.
Almond.js, a smaller version of Require.js also tell you to insert itself into your code to have a standalone javascript file.
When minified, I don't care for extra bloat, it's only a few extra killobytes (for almond.js), but unminified is barely readable. I have to scroll all the way down, past almond.js code to see my application logic.
Question
How can I make my code to be similar to jQuery, in which the final output does not look like a Frankenweenie?
Short answer:
You have to create your own custom build procedure.
Long answer
jQuery's build procedure works only because jQuery defines its modules according to a pattern that allows a convert function to transform the source into a distributed file that does not use define. If anyone wants to replicate what jQuery does, there's no shortcut: 1) the modules have to be designed according to a pattern which will allow stripping out the define calls, and 2) you have to have a custom conversion function. That's what jQuery does. The entire logic that combines the jQuery modules into one file is in build/tasks/build.js.
This file defines a custom configuration that it passes to r.js. The important option are:
out which is set to "dist/jquery.js". This is the single
file produced by the optimization.
wrap.startFile which is set to "src/intro.js". This file
will be prepended to dist/jquery.js.
wrap.endFile which is set to "src/outro.js". This file will
be appended to dist/jquery.js.
onBuildWrite which is set to convert. This is a custom function.
The convert function is called every time r.js wants to output a module into the final output file. The output of that function is what r.js writes to the final file. It does the following:
If a module is from the var/ directory, the module will be
transformed as follows. Let's take the case of
src/var/toString.js:
define([
"./class2type"
], function( class2type ) {
return class2type.toString;
});
It will become:
var toString = class2type.toString;
Otherwise, the define(...) call is replace with the contents of the callback passed to define, the final return statement is stripped and any assignments to exports are stripped.
I've omitted details that do not specifically pertain to your question.
You can use a tool called AMDClean by gfranko https://www.npmjs.org/package/amdclean
It's much simpler than what jQuery is doing and you can set it up quickly.
All you need to do is to create a very abstract module (the one that you want to expose to global scope) and include all your sub modules in it.
Another alternative that I've recently been using is browserify. You can export/import your modules the NodeJS way and use them in any browser. You need to compile them before using it. It also has gulp and grunt plugins for setting up a workflow. For better explanations read the documentations on browserify.org.
I'm building a web app (react app written in es6) that is starting to get pretty big. As a result, I'm seeing unacceptably long download times for my JS file on mobile. I'm trying to wrap my mind around chunking large JS applications into chunks that are loaded on-demand. I'm using webpack, and have read this article:
https://webpack.github.io/docs/code-splitting.html
Using this article, I've split my code into app.js and vendor.js, where vendor.js contains all third party modules/plugins.
I'd like to go further and break up the app.js file into a several entry points, which would then download chunks as needed. The article above describes how to do this with CommonJS or AMD. However, I'm using ES6's native modules instead of these two options and can't find the syntax to define dependencies per file (basically, the ES6 version of .ensure() ).
My questions:
Can I take advantage of webpack's on-demand chunking using ES6 modules, or do I need to use AMD or CommonJS to accomplish this?
If I need to use AMD/CommonJS, how can I avoid a refactor of the entire app?
What do I need to do to ensure dependencies will be loaded asynchronously?
Does anyone have a link to a tutorial/guide/code example to help illustrate what I need?
To answer your first question: Yes. You can definitely use ES6 modules and still load them asynchronously, but you must use the require() function at whatever point you need the code instead of putting imports at the top of the module like normal.
Also keep in mind if you are using export default and using babel 6, you will have to invoke the module using Module.default (Babel 5 treats the Module itself as the default export as a short cut, but the new behaviour is more direct. More info here
there seems to be 3 potential ingredients:
entry points
chunking
async loading
You could set separate entry points and just include the resulting builds separately in your html. But you can also use async loading based on other things (such as scrolling to a certain point, existence of certain classes/IDs).
There is a succinct guide to these at the bottom of Pete Hunt's how-to that's much easier to make sense of than the official webpack documentation.
Jonathan Creamer also has a great walkthrough in the two parts of his Advanced Webpack series of posts.
ES6 modules are implemented by augmenting JS with special syntax which can not be easily extended within javascript in the way webpack extends require for AMD/CommonJS.
However, you can use CommonsChunkPlugin to externally specify chunks for code-splitting. You will have to include these chunks manually though.
Example from Documentation:
Split your code into vendor and application.
entry: {
vendor: ["jquery", "other-lib"],
app: "./entry"
}
new CommonsChunkPlugin({
name: "vendor",
// filename: "vendor.js"
// (Give the chunk a different name)
minChunks: Infinity,
// (with more entries, this ensures that no other module
// goes into the vendor chunk)
})
<script src="vendor.js" charset="utf-8"></script>
<script src="app.js" charset="utf-8"></script>
I am currently maintaining a large number of JS files and the dependency issue is growing over my head. Right now I have each function in a separate file and I manually maintain a database to work out the dependencies between functions.
This I would like to automate. For instance if I have the function f
Array.prototype.f = function() {};
which is referenced in another function g
MyObject.g = function() {
var a = new Array();
a.f();
};
I want to be able to detect that g is referencing f.
How do I go about this? Where do I start? Do I need to actually write a compiler or can I tweak Spidermonkey for instance? Did anyone else already do this?
Any pointers to get me started is very much appreciated
Thanks
Dok
Whilst you could theoretically write a static analysis tool that detected use of globals defined in other files, such as use of MyObject, you couldn't realistically track usage of prototype extension methods.
JavaScript is a dynamically-typed language so there's no practical way for any tool to know that a, if passed out of the g function, is an Array, and so if f() is called on it there's a dependency. It only gets determined what variables hold what types at run-time, so to find out you'd need an interpreter and you've made yourself a Turing-complete problem.
Not to mention the other dynamic aspects of JavaScript that completely defy static analysis, such as fetching properties by square bracket notation, the dreaded eval, or strings in timeouts or event handler attributes.
I think it's a bit of a non-starter really. You're probably better of tracking dependencies manually, but simplifying it by grouping related functions into modules which will be your basic unit of dependency tracking. OK, you'll pull in a few more functions that you technically need, but hopefully not too much.
It's also a good idea to namespace each module, so it's very clear where each call is going, making it easy to keep the dependencies in control manually (eg. by a // uses: ThisModule, ThatModule comment at the top).
Since extensions of the built-in prototypes are trickier to keep track of, keep them down to a bare minimum. Extending eg. Array to include the ECMAScript Fifth Edition methods (like indexOf) on browsers that don't already have them is a good thing to do as a basic fixup that all scripts will use. Adding completely new arbitrary functionality to existing prototypes is questionable.
Have you tried using a dependency manager like RequireJS or LabJS? I noticed no one's mentioned them in this thread.
From http://requirejs.org/docs/start.html:
Inside of main.js, you can use require() to load any other scripts you
need to run:
require(["helper/util"], function(util) {
//This function is called when scripts/helper/util.js is loaded.
//If util.js calls define(), then this function is not fired until
//util's dependencies have loaded, and the util argument will hold
//the module value for "helper/util".
});
You can nest those dependencies as well, so helper/util can require some other files within itself.
As #bobince already suggested, doing static analysis on a JavaScript program is a close to impossible problem to crack. Google Closure compiler does it to some extent but then it also relies on external help from JSDoc comments.
I had a similar problem of finding the order in which JS files should be concatenated in a previous project, and since there were loads of JS files, manually updating the inclusion order seemed too tedious. Instead, I stuck with certain conventions of what constitutes a dependency for my purposes, and based upon that and using simple regexp :) I was able to generated the correct inclusion order.
The solution used a topological sort algorithm to generate a dependency graph which then listed the files in the order in which they should be included to satisfy all dependencies. Since each file was basically a pseudo-class using MooTools syntax, there were only 3 ways dependencies could be created for my situation.
When a class Extended some other class.
When a class Implemented some other class.
When a class instantiated an object of some other class using the new keyword.
It was a simple, and definitely a broken solution for general purpose usage but it served me well. If you're interested in the solution, you can see the code here - it's in Ruby.
If your dependencies are more complex, then perhaps you could manually list the dependencies in each JS file itself using comments and some homegrown syntax such as:
// requires: Array
// requires: view/TabPanel
// requires: view/TabBar
Then read each JS file, parse out the requires comments, and construct a dependency graph which will give you the inclusion order you need.
It would be nice to have a tool that can automatically detect those dependencies for you and choose how they are loaded. The best solutions today are a bit cruder though. I created a dependency manager for my particular needs that I want to add to the list (Pyramid Dependency Manager). It has some key features which solve some unique use cases.
Handles other files (including inserting html for views...yes, you can separate your views during development)
Combines the files for you in javascript when you are ready for release (no need to install external tools)
Has a generic include for all html pages. You only have to update one file when a dependency gets added, removed, renamed, etc
Some sample code to show how it works during development.
File: dependencyLoader.js
//Set up file dependencies
Pyramid.newDependency({
name: 'standard',
files: [
'standardResources/jquery.1.6.1.min.js'
]
});
Pyramid.newDependency({
name:'lookAndFeel',
files: [
'styles.css',
'customStyles.css',
'applyStyles.js'
]
});
Pyramid.newDependency({
name:'main',
files: [
'createNamespace.js',
'views/buttonView.view', //contains just html code for a jquery.tmpl template
'models/person.js',
'init.js'
],
dependencies: ['standard','lookAndFeel']
});
Html Files
<head>
<script src="standardResources/pyramid-1.0.1.js"></script>
<script src="dependencyLoader.js"></script>
<script type="text/javascript">
Pyramid.load('main');
</script>
</head>
It does require you to maintain a single file to manage dependencies. I am thinking about creating a program that can automatically generate the loader file for you based on includes in the header but since it handles many different types of dependencies, maintaining them in one file might actually be better.
JSAnalyse uses static code analysis to detect dependencies between javascript files:
http://jsanalyse.codeplex.com/
It also allows you to define the allowed dependencies and to ensure it during the build, for instance. Of course, it cannot detect all dependencies because javascript is dynamic interpretet language which is not type-safe, like already mentioned. But it at least makes you aware of your javascript dependency graph and helps you to keep it under control.
I have written a tool to do something like this: http://github.com/damonsmith/js-class-loader
It's most useful if you have a java webapp and you structure your JS code in the java style. If you do that, it can detect all of your code dependencies and bundle them up, with support for both runtime and parse-time dependencies.