Develop Reference

How to debug Rollup build output with a massive encoding log?

I recently built a library with Rollup that has a few non-usual bits. That includes for instance, loading up a wasm module, workers with importScripts and a few occurences of eval() in the global scope.
Now I used the rollup-starter-app to create a demonstrator and client app for that library. The repo is https://github.com/frantic0/sema-engine-rollup
I managed to get everything working, after hitting a few walls and adding the following rollup plugins
import { wasm } from "#rollup/plugin-wasm";
import workerLoader from "rollup-plugin-web-worker-loader";
import dynamicImportVars from "#rollup/plugin-dynamic-import-vars";
import copy from "rollup-plugin-copy";
However, in the build output, I'm getting this massive log of what seems to be some encoding...
I'm not sure where this log is coming from and it is so massive that it clears out all the information of the build in the terminal...
What is the best way to tackle this issue and how to debug it effectively?

based on the suggestion #lukastaegert on the rollup issues, one solution is to redirect stderr into a file to read the log.
To do that you can add the following to your rollup command
"rollup -cw 2>err.log 1>out.log"
this allows to further inspect the build log but doesn't solve the error
[EDIT]
After a bit of peeking around Rollup's github issues and source, I found the warning categories and how to deactivate warnings.
Basically, we need to add a function onwarn to rollup.config.js. The first code section below shows the function. The second one show where we should add it on the rollup.config.js
const onwarn = (warning) => {
// Silence warning
if (
warning.code === 'CIRCULAR_DEPENDENCY' ||
warning.code === 'EVAL'
) {
return
}
console.warn(`(!) ${warning.message}`)
}
export default {= {
inlineDynamicImports: !dynamicImports,
preserveEntrySignatures: false,
onwarn,
input: `src/main.js`,
output: {

ES6 Dynamic Imports using Webpack and Babel

I've been using Webpack for my ES6 JS project and has been going well until I started to play with dynamic imports.
What I had that worked (router.js):
import { navigo } from "Navigo"; // router
import { clients } from "Controllers/clients.js";
const navigo = new Navigo();
navigo_router.on({
'/clients': () => {
clients.init();
}
});
But the more pages/routes I add, the more imports get stacked up in the head of the module. This is a relatively large app and I have a lot of pages/routes to add and therefore I need to load them dynamically to reduce the size of the initial page load.
So, following Webpack's documentation for dynamic imports, I tried the following which loads the controller module only when the relative route is called:
import { navigo } from "Navigo"; // router
const navigo = new Navigo();
navigo_router.on({
'/clients': () => {
import("Controllers/clients.js").then((clients) => {
clients.init();
});
}
});
But saving this in my editor resulted in a Babel transpiling error; SyntaxError: 'import' and 'export' may only appear at the top level, and clients.init() is not being called when tested in browser.
After a bit of reading, I discovered I needed a Babel plugin to transpile dynamic import() to require.ensure. So, I installed the plugin using the following command:
npm install babel-plugin-dynamic-import-webpack --save-dev
And declared the plugin in my babel.rc file
{ "plugins": ["dynamic-import-webpack"] }
After installing the plugin, the transpiling error disappeared and checking my transpiled code I found that the dynamic import()s has in fact been changed to require.ensure as expected. But now I get the following browser errors when testing:
Error: Loading chunk 0 failed.
Stack trace:
u#https://<mydomain.com>/js/app.bundle.js:1:871
SyntaxError: expected expression, got '<' 0.app.bundle.js:1
Error: Loading chunk 0 failed.
I didn't understand why it was referencing 0.app.bundle.js with the 0. prefix, so I checked my output/dist folder and I now have a new file in there called 0.app.bundle.js:
0.app.bundle.js 1,962bytes
app.bundle.js 110,656bytes
I imagine this new bundled file is the dynamically imported module, clients.js.
I only added dynamic importing to that one route and have left all the other routes as they were. So, during testing, I can view all routes except that one /clients route that now throws the above errors.
I'm totally lost at this point and hoped somebody could help push me over the finish line. What is this new file 0.app.bundle.js and how am I supposed to be using it/including it in my application?
I hope I've explained myself clearly enough and look forward to any responses.

I managed to fix my own problem in the end, so I will share what I discovered in an answer.
The reason the chunk file wasn't loading was because Webpack was looking in the wrong directory for it. I noticed in the Network tab of my developer console that the the chunk file/module was being called from my root directory / and not in /js directory where it belongs.
As per Webpack's documentation, I added the following to my Webpack config file:
output: {
path: path.resolve(__dirname, 'dist/js'),
publicPath: "/js/", //<---------------- added this
filename: 'app.bundle.js'
},
From what I understand, path is for Webpack's static modules and publicPath is for dynamic modules.
This made the chunk load correctly but I also had further issues to deal with, as client.init() wasn't being called and yielded the following error:
TypeError: e.init is not a function
To fix this, I also had to change:
import("Controllers/clients.js").then((clients) => {
clients.init();
});
To:
import("Controllers/clients.js").then(({clients}) => {
clients.init();
});
Note the curly braces in the arrow function parameter.
I hope this helps somebody else.

For debugging, you need to do
import("Controllers/clients.js").then((clients) => {
console.log(clients);
});
maybe working
import("Controllers/clients.js").then((clients) => {
clients.default.init();
});

Typescript guide gives "Duplicate function implementation" warning

I'm getting started with TypeScript and at moment I'm following the TypeScript in 5 minutes guide. I'm receiving a strange warning in Visual Studio Code when I hover the mouse over the greeter function name, as shown in the below image. The alert is:
[ts] Duplicate function implementation.
function greeter(person: Person): string (+1 overload)
But there is no other implementation of this unique function in my single file! When I run tsc greeter.ts all works fine and the js file is generated.
The complete greeter.ts file:
interface Person {
firstName: string;
lastName: string;
}
function greeter(person: Person) {
return "Hello, " + person.firstName + " " + person.lastName;
}
var user = { firstName: "Jane", lastName: "User" };
console.log(greeter(user));
Why am I receiving this alert? How to solve it? I took a look in this question, but I believe it isn't related.

Looks like this is a bug in Visual Studio Code. There are a few issues on GitHub about this, such as here and here. The comments on the issues imply that it was an issue, then was fixed, and has just become an issue again in v1.12.1.
It looks as if the solution is to run tsc --init to initialize the tsconfig.json in the folder.

When we open both file.ts and the transpiled file.js files and do TSC, this error occurs.
Please close the transpiled file.js and try again.

If you have both the src file (typescript) and the transpiled file (javascript) in the same directory and open up the javascript file in VS Code then you'll get the error. Output the transpiled file into a directory and it will not error. Use the --outDir flag:
tsc --outDir ./dist greeter.ts
Got this problem in version 1.26.1 of VS Code. Generating the tsconfig.json file did not make the error go away for me.

According to this article, add this simple line in the top of your Typescript file
export { };
[index.ts]
export { };
declare const signalR: any;
declare const moment: any;

This might be because you don't have a tsconfig.json file for your TypeScript project.
Try creating a tsconfig file and write a default "compilerOptions".
It worked for me.
The tsconfig.json file with default code that I used is :
{
"compilerOptions": {
"module": "commonjs"
},
"exclude": [
"node_modules"
]
}
For more info on VS TypeScript Compiling please refer
https://code.visualstudio.com/docs/typescript/typescript-compiling

In my case, it seems that if the file didn't import/export anything it wasn't considered a module and VS Code would assume that all the files would get concatenated together. You can fix it by adding an import/export.
I also went ahead and set my tsconfig to include isolatedModules so that I would get a more helpful error message.
{
"isolatedModules": true,
}

Old post but hope this helps someone:
I encountered this problem when I had my code written like this:
function myFunc() { ... }
module.exports = {
myFunc
}
I fixed the problem by writing it this way instead:
export function myFunc() { ... }

Exclude folders from builds in Brocfile

Is there a way to exclude a folder from a build in a Brocfile (or any other place).
The use case is packaging, where I have an app made of sub-apps within pods. eg.
/app/modules/components
/app/modules/app1
/app/modules/app2
/app/modules/app3
I'd like to build them all when environment is set to 'development' or only eg. 'app1' when environment is 'app1'. Any suggestions?
I have tried different combinations of broccoli-file-remover, broccoli-funnel and broccoli-merge-trees to no avail.
var removeFile = require('broccoli-file-remover');
module.exports = removeFile(app.toTree(), {
paths: ['app/modules/pod1/', 'app/modules/pod2/']
});

Ah, so after actually thinking about this clearly, everything is actually working exactly as expected in my previous example.
I clearly wasn't paying enough attention. app.toTree() is far too late to perform this operation, as everything has already been built and concated.
Luckily, ember-cli does enable addons to modify the appropriate trees at various life cycle milestones.
See: https://github.com/ember-cli/ember-cli/blob/master/ADDON_HOOKS.md for more details on which hooks are currently available.
The hook that should do the trick is Addon.prototype.postprocessTree. Now we have two choices, we can build a standalone addon, via ember addon or we can create a light-weight in-repo addon via ember g in-repo-addon. Typically for these types of situations, I prefer in-repo-addons as they don't require a second project, but otherwise they are the same.
ember g in-repo-addon remove
we need to install broccoli-stew via npm install --save broccoli-stew
include it var stew = require('broccoli-stew');
add hook postprocessTree to the add-on
when the postprocessTree is for the type we care about, use broccoli-stew to remove the directories we no longer care care.
The resulting pull request: https://github.com/WooDzu/ember-exclude-pod/pull/1
Note: I noticed template wasn't one of the types available in postprocess, so I added it: https://github.com/ember-cli/ember-cli/pull/4263 (should be part of the next ember-cli release)
Note: we really do want an additional hook
Addon.prototype.preprocessTree, as to ignore the files before we
even build them. I have opened a related issue:
https://github.com/ember-cli/ember-cli/issues/4262
output of the above steps
var stew = require('broccoli-stew');
module.exports = {
name: 'remove',
isDevelopingAddon: function() {
return true;
},
postprocessTree: function(type, tree){
if (type === 'js' || type === 'template') {
return stew.rm(tree, '*/modules/pod{1,2}/**/*');
} else {
return tree;
}
}
};

I am pretty confident broccoli-stew's rm will handle this correctly.
https://github.com/stefanpenner/broccoli-stew/blob/master/lib/rm.js#L4-L40 there are even tests that test a very similar scenario: https://github.com/stefanpenner/broccoli-stew/blob/master/tests/rm-test.js#L48-L57
var stew = require('broccoli-stew');
module.exports = stew.rm(app.tree(), 'app/modules/{pod1,pod2}');
If this doesn't work, feel free to open an issue on broccoli-stew. Be sure to provide a running example though

This is really late, but I created a Broccoli plugin to do just this. It's available at https://www.npmjs.com/package/broccoli-rm.
(The trick is to detect whether an excluded path is a folder, and then use a glob match to make sure that none of the children of the folder get symlinked during copying.)
var rm = require('broccoli-rm');
var input = app.toTree();
module.exports = output = rm([input], {
paths: ['app/modules/pod1', 'app/modules/pod2']
});

requireJS an entire folder

Is it possible to "require" an entire folder using requireJS.
For example, I have a behaviors folder with a ton of behavior js files. I'd really like to be able to simply use require(['behaviors/*'], function() {...}); to load everything in that folder rather than having to keep that list up to date. Once compressed and optimized I'd have all those files lump together, but for development it's easier to work with them individually.

javascript in browser has no filesystem access and so it can't scan a directory for files. If you are building your app in a scripting language like php or ruby you could write a script that scans the directory and adds the file names to the require() call.

I don't know if I can recommend this approach anymore. I think the more explicit way to do this is by manually "requiring"/"exporting" the functionality you need. The exception I think is if you have a "namespace" of files that you want exported see below "Babel and ES6 Module Import Declarations (export-namespace-from) or see below "Babel and ES6 Module Import Declarations.
These solutions also assume that you have a meaningful file structure - where file names become part of that "require" * definition.
However, if you still need to do this there are a few existing tools and methods that might provide the behavior that you're looking for.
Possible Solutions
Babel and ES6 Module Import Declarations (plugin-export-namespace-from)
Have a setup that is ES6 compliant.
You need to update your .babelrc file to include babel-plugin-proposal-export-namespace-from.
Use export namespace plugin by writing syntax like the following:
common/index.js
export * from './common/a'; // export const a = false;
export * from './common/b'; // export const b = true;
main.js
import { a, b } from './common';
console.log(a); // false
console.log(b); // true
Babel and ES6 Module Import Declarations (plugin-wildcard)
Have a setup that is ES6 compliant.
You need to update your .babelrc file to include babel-plugin-wildcard.
Use wildcard namespace plugin by writing syntax like the following:
main.js
import { a, b } from './common/*'; // imports './common/a.js' and './common/b.js'
console.log(a); // false
console.log(b); // true
RequireJS (Now Outdated)
Download and install require-wild npm install require-wild
Configure the declaration as follows
grunt.initConfig({
requireWild: {
app: {
// Input files to look for wildcards (require|define)
src: ["./**/*.js"],
// Output file contains generated namespace modules
dest: "./namespaces.js",
// Load your require config file used to find baseUrl - optional
options: { requireConfigFile: "./main.js" }
}
}
});
grunt.loadNpmTasks("require-wild");
grunt.registerTask('default', ['requireWild']);
Then run the grunt task. Your file will be generated. Modify your setup to load namespaces.js
require(['namespaces'], function () { ... });
This now allows modules under src to use dependencies glob pattern matching.
require(['behaviors/**/*'], function (behaviors) { }

I know this is old, but I'd like to share my solution:
For this solution you need JQuery
1) Create a bash script that will list all the js files in
"MyDirectory/", and save it to "directoryContents.txt":
#!/bin/bash
#Find all the files in that directory...
for file in $( find MyDirectory/ -type f -name "*.js" )
do
fileClean=${file%.js} #Must remove .js from the end!
echo -n "$fileClean " >> MyDirectory/directoryContents.txt
done
File will look like this:
MyDirectory/FirstJavascriptFile MyDirectory/SecondJavascriptFile
MyDirectory/ThirdJavascriptFile
Problem with my script! Puts an extra " " at the end, that messes things up! Make sure to remove the excess space at the end of directoryContents.txt
2) Then in your Client side JS code:
do a "GET" request to retrieve the text file
For each entry (split by the space), 'require' that file:
.
$.get( "MyDirectory/directoryContents.txt", {}, function( data ) {
var allJsFilesInFolder = data.split(" ");
for(var a=0; a<allJsFilesInFolder.length; a++)
{
require([allJsFilesInFolder[a]], function(jsConfig)
{
//Done loading this one file
});
}
}, "text");
I was having a problem with this code not finishing before my other code, so Here's my extended answer:
define([''], function() {
return {
createTestMenu: function()
{
this.loadAllJSFiles(function(){
//Here ALL those files you need are loaded!
});
},
loadAllJSFiles: function(callback)
{
$.get( "MyDirectory/directoryContents.txt", {}, function( data ) {
var allJsFilesInFolder = data.split(" ");
var currentFileNum = 0;
for(var a=0; a<allJsFilesInFolder.length; a++)
{
require([allJsFilesInFolder[a]], function(jsConfig)
{
currentFileNum++;
//If it's the last file that needs to be loaded, run the callback.
if (currentFileNum==allJsFilesInFolder.length)
{
console.log("Done loading all configuration files.");
if (typeof callback != "undefined"){callback();}
}
});
}
}, "text");
}
}
});
What I ended up doing was everytime my Node server boots, it will run the bash script, populating directoryContents.txt. Then My client side just reads directoryContents.txt for the list of files, and requires each in that list.
Hope this helps!

There isn't really a way to do this conceptually on the fly (that I know of).
There's a few work arounds though:
Use grunt and concat and then just require that behemoth...I know, kinda sucky.
What I think is a better solution... use a require hierarchy like so:
require('/js/controllers/init', function(ctrls){
ctrls(app, globals);
});
// /js/controllers/init.js
define('js/controllers/index', 'js/controllers/posts', function(index, posts){
return function protagonist(app, globals){
var indexModule = index(app, globals);
var indexModule = posts(app, globals);
return app || someModule;
};
});
// /js/controllers/index.js
define('js/controllers/index', 'js/controllers/posts', function(index, posts){
return function protagonist(app, globals){
function method1(){}
function method2(){}
return {
m1: method1,
m2: method2
};
};
});
Note that "protagonist" function. That allows you to initialize modules before their use, so now you can pass in a 'sandbox' -- in this case app and globals.
Realistically, you wouldn't have /js/controllers/index.js... It should probably be something like /js/controllers/index/main.js or /js/controllers/index/init.js so that there is a directory adjacent to (sibling of) /js/controllers/init.js called "index". This will make your modules scalable to a given interface -- you can simply swap modules out and keep your interface the same.
Hope this helps! Happy coding!

I wrote a library to solve this problem. Eventually someone else came along and improved my library, here it is:
https://github.com/smartprocure/directory-metagen
You can use my lib with Gulp or whatever - it generates metadata for your project and RequireJS can use that metadata to require the desired files from the filesystem.
Using this lib will produce a RequireJS module that looks something like this:
define(
[
"text!app/templates/dashboardTemplate.ejs",
"text!app/templates/fluxCartTemplate.ejs",
"text!app/templates/footerTemplate.ejs",
"text!app/templates/getAllTemplate.ejs",
"text!app/templates/headerTemplate.ejs",
"text!app/templates/homeTemplate.ejs",
"text!app/templates/indexTemplate.ejs",
"text!app/templates/jobsTemplate.ejs",
"text!app/templates/loginTemplate.ejs",
"text!app/templates/overviewTemplate.ejs",
"text!app/templates/pictureTemplate.ejs",
"text!app/templates/portalTemplate.ejs",
"text!app/templates/registeredUsersTemplate.ejs",
"text!app/templates/userProfileTemplate.ejs"
],
function(){
return {
"templates/dashboardTemplate.ejs": arguments[0],
"templates/fluxCartTemplate.ejs": arguments[1],
"templates/footerTemplate.ejs": arguments[2],
"templates/getAllTemplate.ejs": arguments[3],
"templates/headerTemplate.ejs": arguments[4],
"templates/homeTemplate.ejs": arguments[5],
"templates/indexTemplate.ejs": arguments[6],
"templates/jobsTemplate.ejs": arguments[7],
"templates/loginTemplate.ejs": arguments[8],
"templates/overviewTemplate.ejs": arguments[9],
"templates/pictureTemplate.ejs": arguments[10],
"templates/portalTemplate.ejs": arguments[11],
"templates/registeredUsersTemplate.ejs": arguments[12],
"templates/userProfileTemplate.ejs": arguments[13]
}
});
You can then require modules in your front-end like so:
var footerView = require("app/js/jsx/standardViews/footerView");
however, as you can see this is too verbose, so the magic way is like so:
name the dependency above as allViews!
now you can do:
var allViews = require('allViews');
var footerView = allViews['standardViews/footerView'];
There are two advantages to requiring directories whole:
(1) in production, with the r.js optimizer, you can point to one dependency (module A) and it can then easily trace all of A's dependencies that represent a entire directory
(2) in development, you can require whole directories up front and then use synchronous syntax to require dependencies because you know they have already been loaded
enjoy "RequireJS-Metagen"
https://github.com/smartprocure/directory-metagen
https://www.npmjs.com/package/requirejs-metagen
https://github.com/ORESoftware/requirejs-metagen