I am trying to port a library from grunt/requirejs to webpack and stumbled upon a problem, that might be a game-breaker for this endeavor.
The library I try to port has a function, that loads and evaluates multiple modules - based on their filenames that we get from a config file - into our app. The code looks like this (coffee):
loadModules = (arrayOfFilePaths) ->
new Promise (resolve) ->
require arrayOfFilePaths, (ms...) ->
for module in ms
module ModuleAPI
resolve()
The require here needs to be called on runtime and behave like it did with requireJS. Webpack seems to only care about what happens in the "build-process".
Is this something that webpack fundamentally doesn't care about? If so, can I still use requireJS with it? What is a good solution to load assets dynamically during runtime?
edit: loadModule can load modules, that are not present on the build-time of this library. They will be provided by the app, that implements my library.
So I found that my requirement to have some files loaded on runtime, that are only available on "app-compile-time" and not on "library-compile-time" is not easily possible with webpack.
I will change the mechanism, so that my library doesn't require the files anymore, but needs to be passed the required modules. Somewhat tells me, this is gonna be the better API anyways.
edit to clarify:
Basically, instead of:
// in my library
load = (path_to_file) ->
(require path_to_file).do_something()
// in my app (using the 'compiled' libary)
cool_library.load("file_that_exists_in_my_app")
I do this:
// in my library
load = (module) ->
module.do_something()
// in my app (using the 'compiled' libary)
module = require("file_that_exists_in_my_app")
cool_library.load(module)
The first code worked in require.js but not in webpack.
In hindsight i feel its pretty wrong to have a 3rd-party-library load files at runtime anyway.
There is concept named context (http://webpack.github.io/docs/context.html), it allows to make dynamic requires.
Also there is a possibility to define code split points: http://webpack.github.io/docs/code-splitting.html
function loadInContext(filename) {
return new Promise(function(resolve){
require(['./'+filename], resolve);
})
}
function loadModules(namesInContext){
return Promise.all(namesInContext.map(loadInContext));
}
And use it like following:
loadModules(arrayOfFiles).then(function(){
modules.forEach(function(module){
module(moduleAPI);
})
});
But likely it is not what you need - you will have a lot of chunks instead of one bundle with all required modules, and likely it would not be optimal..
It is better to define module requires in you config file, and include it to your build:
// modulesConfig.js
module.exports = [
require(...),
....
]
// run.js
require('modulesConfig').forEach(function(module){
module(moduleAPI);
})
You can also try using a library such as this: https://github.com/Venryx/webpack-runtime-require
Disclaimer: I'm its developer. I wrote it because I was also frustrated with the inability to freely access module contents at runtime. (in my case, for testing from the console)
Related
I am moderately new to webpack and have been struggling with this for a while, and need some help.
This repo: github.com/gkindel/CSV-JS
I cannot get to be built with webpack. It throws the error:
WARNING in ./bower_components/CSV-JS/csv.js
Critical dependencies:
203:20-27 require function is used in a way in which dependencies cannot be statically extracted
Here is how I try to use it:
// CSV-JS used by angular ui-grid CSV importer
window.CSV = require('../bower_components/CSV-JS');
The line it is failing on looks like this:
CSV.stream = function () {
var s = new require('stream').Transform({objectMode: true});
...
I am pretty sure ui-grid does not use this function in the library, but webpack has to parse the entire file. I am guessing this is functionality to use in a node environment.
Anyway, anyone have a way of loading this dependency? Or perhaps an alternative library? (not sure if this is possible though, because ui-grid depends on the CSV.parse() function).
I have a scenario where I want to export data to CSV from the client-side. I will have a textbox/area or whatever where the user can input data and then ideally with one click, a local CSV file will be updated with that data.
This is easily achievable with NodeJS with server interaction, and its core modules (specifically fs module), but apparently not so much from a browser.
I found out that certain node modules (for example underscore), support RequireJS's method of making a particular module work in the browser. So for underscore I did this:
methods.js
define(['underscore'],function(_) {
var Methods = {
doSomething: function() {
var x = _.size({one: 1, two: 2, three: 3, xuz: 3});
alert(x);
}
};
return Methods;
});
common.js
requirejs.config({
baseURL: 'node_modules',
paths: {
underscore: 'underscore/underscore',
}
});
require(['methods'], function(y){
y.doSomething();
});
index.html
<script data-main="common" src="require.js"></script>
<script>
require(['common'], function() {
require(['methods.js']);
});
</script>
The above works fine and will show an alert: 4.
But when I try the same with the fs module it won't work. It will show this error:
Module name "util" has not been loaded yet for context: _. Use require([])
As far as I can understand, this is because fs requires several other modules, one of them being util.
So I proceeded to add all these modules to the RequireJS config. But still no luck, so I specifically tested util module by itself as this doesn't seem to require other modules to work.
And now I am stuck on this error: Uncaught ReferenceError: exports is not defined
I tried modularizing this util module by encapsulating the whole module source code in this:
define([], function() {})
But it didn't work either... I've also tried copying underscore's model but still no luck.
So I was wondering if anyone managed to use util & fs modules (or any other core NodeJS modules) within the browser with libraries such as RequireJS or Browserify.
That's right, exports is node-specific JS (used to make part of your module available outside the module) and isn't supported by web-browsers. Even though NodeJS is technically JS, there are client specific properties (like the window property for browsers, and exports for NodeJS apps) that can't be interchanged.
That said, here's a client side JS answer to the CSV problem.
Your best bet (and probably only one) is to use HTML5 FileSystem API. I am not aware of any other browser feature that would allow to work with files on client computer except maybe Flash and similar solution.
I am bit confused by your browserify tag since you are not obviously using Browserify. That would fix your issue with "exports is not defined".
I am working on a javascript module/library that should work in 3 environments:
in node.js
in requirejs
when simply included using tags into the webpage. In this case the whole module should be hooked up under window.myModule
Do you have any suggestions as to how to write the structure of the library so that it works in all these environments?
EDIT: basically I mean some sort of wrapper code around the library so that I can call the file form any of those three methods and I'm fine...
This requirement and its solution is known as Universal Module Definition (UMD). It is currently a draft proposal. Background and current status is described in Addy Osmani - Writing Modular JavaScript With AMD, CommonJS & ES Harmony article. Look for "UMD" link pointing to various templates you can use.
Quite many other templates can be found on the web - UMD is the search keyword.
(did not find the final link myself yet :)
We're working on the same thing, I think.
And we have some success. We have library (we call it 'slib'), compiled to AMD js files. It does not depend on npm modules or browser, so it can be called from node and from browser.
1) To call it from node, we use requirejs:
file require.conf.js
module.exports = function(nodeRequire){
global.requirejs = require('requirejs');
requirejs.config({
baseUrl: __dirname+"/../web/slib/",
paths: {
slib: "."
},
nodeRequire: nodeRequire
});
}
In any other serverside (nodejs) file we add this line at the beginning
require("./require.conf")(require);
then we call slib's code by:
var Computation = requirejs("slib/Computation");
2) To call slib from browser, we just use requirejs. It handles everything fine.
3) We do not need to call slib from < script > directly.
For production, we use r.js to make a bundle js file with most of dependencies and use it on the page with one < script >. And this script downloads all other deps, if they are not included, using standard requirejs and it does not need requirejs (as far as I remember), it just works alone. This is very flexible for large projects: use requirejs while development, use r.js to bundle core files in production to speed up page load, use whole bundle if you need only one < script > without any other requests. r.js bundles all dependencies correctly, including old js libraries, which were commonly loading using only < script > and accessible by window.myOldLibrary using shim param on config.
It seems you can use browserfy to make some npm modules accessible from slib's code, but we did not tried yet.
Also, using requirejs on node's side, I think, can be simpler (why we need second 'requirejs' function together with node's one?) We just have not investigated it well, but this works.
In any slib module you can write
if (window)
window.module1 = this // or whatever
and it will be exported as old js lib upon load
So, I have an app that is using requireJS. Quite happily. For the most part.
This app makes use of Socket.IO. Socket.IO is being provided by nodejs, and does not run on the same port as the main webserver.
To deal with this, in our main js file, we do something like this:
var hostname = window.location.hostname;
var socketIoPath = "http://" + hostname + ":3000/socket.io/socket.io";
requirejs.config({
baseUrl: "/",
paths: {
app : "scripts/appapp",
"socket.io" : socketIoPath
}
});
More complicated than this, but you get the gist.
Now, in interactive mode, this works swimingly.
The ugliness starts when we try to use r.js to compile this (technically we're using grunt to run r.js, but that's besides the point).
In the config for r.js, we set an empty path for socket.io (to avoid it failing to pull in), and we set our main file as the mainConfigFile.
The compiler yells about this, saying:
Running "requirejs:dist" (requirejs) task
>> Error: Error: The config in mainConfigFile /…/client.js cannot be used because it cannot be evaluated correctly while running in the optimizer. Try only using a config that is also valid JSON, or do not use mainConfigFile and instead copy the config values needed into a build file or command line arguments given to the optimizer.
>> at Function.build.createConfig (/…/r.js:23636:23)
Now, near as I can figure, this is due to the fact that I'm using a variable to set the path for "socket.io". If i take this out, require runs great, but i can't run the raw from a server. If I leave it is, my debug server is happy, but the build breaks.
Is there a way that I can lazily assign the path of "socket.io" at runtime so that it doesn't have to go into the requirejs.config() methos at that point?
Edit: Did some extensive research on this. Here are the results.
Loading from CDN with RequireJS is possible with a build. However, if you're using the smaller Almond loader, it's not possible.
This leaves you with two options:
Use almond along with a local copy of the file in your build.
Use the full require.js loader and try to use a CDN.
Use a <script> tag just for that resource.
I say try for #2 because there are some caveats. You'll need to include require.js in your HTML with the data-main attribute for your built file. But if you do this, require and define will be global functions, allowing users to require any of your internal modules and mess around with them. If you're okay with this, you'll need to follow the "empty: scheme" in your build config (but not in your main config).
But the fact remains that you now have another HTTP request. If you only want one built file, which includes the require.js loader, you'll need to optimize for only one file.
Now, if you want to avoid users being able to require your modules, you'll have to do something like wrap:true in your build. But as far as I can tell, once your module comes down from CDN, if it's AMD, it's going to look for a global define function to register itself with, and that won't exist because it's now wrapped in a closure.
The lesson I took away from all this: inline your resources to your build. It makes sense. You reduce HTTP requests, minify it all and get gzip compression. You don't expose your modules to the world and everything is a lot simpler. If you cache your resources properly you won't even need to worry about it.
But since new versions of socket.io don't like AMD, here's how I did it. Make sure to include the socket.io <script> tag before requirejs. Then create a requirejs module named socket.io with the following contents:
define([], function () {
var io = window.io;
window.io = null;
return io;
});
Set the path like so: 'socket.io': 'core/socket.io' or wherever you want.
And require it as normal! The build works fine this way.
Original answer
Is it possible that you could make use of the path config fallbacks specified in the RequireJS API? Maybe you could save the file locally as a fallback so your build will work.
The socket.io GitHub repository specifies that you can serve the client with the files in the socket.io-client package's dist/ directory.
I am building a web application based on Rails and, on the client-side Backbone.js. For structuring my Coffeescript-Code, I used RequireJS and requirejs-rails. Each of my Backbone classes lives in its own RequireJS module.
I recently refactored a lot of code into some base classes and want to package them somehow to be able to easily reuse them in other projects (Rails and/or Javascript/Coffeescript, possible even without RequireJS) and share it on GitHub as a separate project from my Rails application. I read the RequireJS documentation on packages, but it doesn't go into the details very much. So this is what I would like to do:
Move my shared code into its own package, so views/base_collection_view becomes commons/views/base_collection_views and so on
Include the package into my requirejs-rails setup in my rails applications, and provide a compiled my-commons.js file to use within non-requirejs setups (I guess the latter would be done using almond fairly easily once I figured out how to layout the package)
A full example of a reusable RequireJS-package would really help me a lot at this point, along with some ideas how this could be transfered to requirejs-rails.
Not sure about requirejs-rails, but with RequireJS it's pretty easy.
define(['dep1', 'dep2'] , function ($, otherLibrary) {
return function () {
// your module code
};
});
Where 'dep1' and 'dep2' are other RequireJS modules that your module depends on. You can depend on as few or as many as you like. The var names you pass to the actual function ($ and otherLibrary in this example) are the names that those libraries will be assigned to within your module.
Anyone using RequireJS will be able to require your module this way, based on how the file is named and the folders it's in.
For instance, if this file was called "my-super-lib.js" inside of the libs directory, another module could just pass libs/my-super-lib to its dependency array and everything will be set up.
Update: just remembered you mentioned coffeescript. Same idea, but to be clear:
define ['dep1', 'dep2'], ($, otherLibrary) ->
() ->
// your module code
If you're into that. ;)