Project Intro
My project is a single page storefront. The project has multiple modules, and each module contains a set of controller.js, view.js and model.js files, as well as a template.html file. And uses requirejs to manage dependencies.
Problem Statement
I want to use mainConfigFile to provide paths to reference modules in grunt-requirejs.
Part of my mainConfigFile's require.config is stored in separate file (base.dependency.config), and require.config.paths are pieced together by underscore at runtime.
base.dependency.config
config = {
baseDependencyConfig: {
paths: { ... }
shim: { ... }
}
}
main.js
var dependencies = config.baseDependencyConfig;
var basePaths = config.baseDependencyConfig.paths;
var extensionPaths = {
// extra sets of paths
};
// combine base paths and extension paths at runtime using underscore
var dependencyPaths = _.extend(basePaths, extensionPaths);
dependencies.paths = dependencyPaths;
require.config(dependencies);
// application startup
require(['app', 'eventbus']) {
// code
}
Error
However, grunt requirejs is ignoring mainConfigFile, grunt requirejs tries to find 'app.js' under root, when in fact, 'app' is defined under require.config paths as
'app': 'modules/base/app/base.app.controller'
my gruntFile:
module.exports = function (grunt) {
grunt.initConfig({
// ... other plugin config
requirejs: {
options: {
baseUrl: 'public',
// the paths for the named modules such as 'app' are defined
// in main.js under require.config paths
name: 'main',
include: [
'app',
'cart',
'category'
],
out: 'public/build/app-optimized.js',
mainConfigFile: 'public/main.js',
findNestedDependencies: true,
optimizeCss: 'none',
cssImportIgnore: 'style/style.css, style/mocha.css',
}
}
})
}
my file structure
public
|--modules/
| |--base/
| | |--cart
| | |--category
| | |--category.controller.js
| | |--category.view.js
| | |--category.model.js
| | └-category.template.html
| |
| └--extension/
|
|--style/
|--image/
|--main.js <-- main config file
|--other .js files
mainConfigFile, main.js lives in root, along with a few other application startup js files
main bulk of application files lives inside modules folder
each module folder contains its controller, view and model js file, as well as a template.html file
Edit
the gruntFile worked before, with different mainConfigFile (main.js) setup:
require.config({
paths: {...}
shim: {...}
})
// application startup
require(['app', 'eventbus']) {
// code
}
r.js uses Esprima as Javascript parser to extract the config object from the specified mainConfigFile. It only looks for certain signatures in the code.
Look at
hasRequire(): determine the AST node is a configuration call candidate
findConfig(): calls the above deciding how to extract the config
I've created a patch making it aware of recognizing
requirejs.config(_VariableToExport = { ... });
This is limited and the Javascript parser approach makes it very complicated to make r.js able to extract configurations that were created by functions (like in your code) or so. This patch has not been submitted to the project yet. I'm struggling with bureaucracy yet.
I think the only working workaround so far is
not to use mainConfigFile
exporting the config as NodeJS module
requiring the main.js/config.js in Node (Grunt)
passing the object as value to the config attribute or method
See my comment to this issue for a scribble.
This approach is proven in another, a bit older project I'm working on.
Related to r.js 2.1.11.
Related
OK, i have searched high and low but cannot reliably deterrmine if this is or is not possible with webpack.
https://github.com/webpack/webpack/tree/master/examples/require.context
Appears to indicate that one can pass a string to a function and it load a module...
But my attempt is just not working:
webpack.config.js
'use strict';
let webpack = require('webpack'),
jsonLoader = require("json-loader"),
path = require("path"),
fs = require('fs'),
nodeModules = {};
fs.readdirSync('node_modules')
.filter(function(x) {
return ['.bin'].indexOf(x) === -1;
})
.forEach(function(mod) {
nodeModules[mod] = 'commonjs ' + mod;
});
let PATHS = {
app: __dirname + '/src'
};
module.exports = {
context: PATHS.app,
entry: {
app: PATHS.app+'/server.js'
},
target: 'node',
output: {
path: PATHS.app,
filename: '../build/server.js'
},
externals: nodeModules,
performance: {
hints: "warning"
},
plugins: [
jsonLoader
],
resolve: {
modules: [
'./node_modules',
path.resolve(__dirname),
path.resolve(__dirname + "/src"),
path.resolve('./config')
]
},
node: {
fs: "empty"
}
};
The server.js
let _ = require('lodash');
let modules = [ "modules/test" ];
require( 'modules/test' )();
_.map( modules, function( module ){
require( module );
});
The module in modules/ named test.js
module.exports = () => {
console.log('hello world');
};
But the result is always the same... the pm2 logs just say hello world for the static require... but for the dynamic load of the same module
Error: Cannot find module "."
All i want to be able to do is loop through an array of paths to modules and load then...
You cannot use a variable as argument to require. Webpack needs to know what files to bundle at compile time. As it does no program flow analysis, it can't know what you pass to the function. In that case it might be obvious, but this could go as far as using user input to decide what module to require, and there is no way webpack can possibly know which modules to include at compile time, so webpack does not allow it.
The example you posted is a bit different. You could use require with a concatenated string. For example:
require(`./src/${moduleName}/test`);
Which modules does webpack need to include in the bundle? The variable moduleName could be anything, so the exact module is not known at compile time. Instead it includes all modules that could possibly match the above expression. Assuming the following directory structure:
src
├─ one
│ └─ test.js
├─ two
│ ├─ subdir
│ │ └─ test.js
│ └─ test.js
└─ three
└─ test.js
All of these test.js files will be included in the bundle, because moduleName could be one or something nested like two/subdir.
For more details see require with expression of the official docs.
You cannot loop through an array and import every module of the array, with the above exception by concatenating a string, but that has the effect of including all possible modules and should generally be avoided.
I ran into this problem in an electron environment. My use case was being able to require dynamically created files in an IDE like application. I wanted to use the electron require, which is basically a NodeJS Common module loader. After some back and forth I landed on a solution that uses webpack's noParse module configuration.
First create a module that that will be ignored by webpack's parser:
// file: native-require.js
// webpack replaces calls to `require()` from within a bundle. This module
// is not parsed by webpack and exports the real `require`
// NOTE: since the module is unparsed, do not use es6 exports
module.exports = require
In my webpack config, under module, instruct the bundler not to parse this module:
{
module: {
noParse: /\/native-require.js$/,
}
}
Lastly, in any bundle where you want to access the original require:
import nativeRequire from './native-require'
const someModule = nativeRequire('/some/module.js') // dynamic imports
A bit late....but... since you are bundling to target: 'node', there is a workaround to dynamic requiring modules, and bypassing the "the effect of including all possible modules".
The solution is lifted from:
Using dynamic require on node targets WITHOUT resolve or bundle the target module · Issue #4175 · webpack/webpack
Quoted from that comment:
const requireFunc = typeof __webpack_require__ === "function" ? __non_webpack_require__ : require;
const foo = requireFunc(moduleName);
Bundles to:
const requireFunc = true ? require : require;
const foo = requireFunc(moduleName);
As we know, when use RequireJS in the configuration file we must define 'paths' and 'shim'. When writing tests (Karma, Jasmine), we need to create an additional configuration file for RequireJS and re-define the same data.
I try to extract common parts and load them dynamically. In Angular JS application, everything works without problems, but in test i have always error '404'. But let's get to the beginning. Simple example structure:
.
|-- newApp
| |-- app
| | `-- app.require.js
| |-- newApp.require.js
| `-- index.html
`-- test
|-- test.karma.js
`-- test.require.js
index.html load RequireJS config file
<script data-main="newApp.require.js" src="bower_components/requirejs/require.js"></script>
newApp.require.js init RequireJS
require([
'app/app.require'
], function (appRequire) {
"use strict";
require.config({
baseUrl: 'app',
paths: appRequire.paths,
shim: appRequire.shim,
deps: [
'NewAppBootstrap'
]
});
});
app.require.js Module / object with paths and shim
define([
'some/others/components.require'
], function (componentsRequire) {
"use strict";
var appRequire = {
'NewApp': 'app.module',
'NewAppBootstrap': 'app.bootstrap',
'NewAppRoute': 'app.routes'
};
var vendorRequire = {
'jQuery': {
exports: '$'
},
'angular': {
exports: 'angular'
}
};
return {
paths: Object.assign(
appRequire,
componentsRequire
),
shim: vendorRequire
};
});
Up to this point everything works fine. Now I would like to file app.require.js load into test.require.js. And the problems begin...
test.require.js
var TEST_REGEXP = /(spec|test)\.js$/i;
var allTestFiles = [];
Object.keys(window.__karma__.files).forEach(function (file) {
'use strict';
if (TEST_REGEXP.test(file)) {
var normalizedTestModule = file.replace(/^\/base\/|\.js$/g, '');
allTestFiles.push(normalizedTestModule);
}
});
require([
'app/app.require'
], function (appRequire) {
"use strict";
require.config({
baseUrl: '/base/newApp',
waitSeconds: 200,
paths: appRequire.paths,
shim: appRequire.shim,
deps: allTestFiles,
callback: window.__karma__.start
});
});
Unfortunately, I still have error:
WARN [web-server]: 404: /app/app.require.js
PhantomJS 2.1.1 (Linux 0.0.0) ERROR: 'There is no timestamp for app/app.require.js!'
PhantomJS 2.1.1 (Linux 0.0.0) ERROR
Error: Script error for "app/app.require"
http://requirejs.org/docs/errors.html#scripterror at node_modules/requirejs/require.js:143
I tried different paths, but still nothing. Does anyone know how to load this file? Whether it is at all possible load file in this place? I would be grateful for any tips.
By the time your code hits this require call:
require([
'app/app.require'
],
There is no RequireJS configuration in effect, and data-main is not used, so by default RequireJS takes the directory that contains the HTML page that loads RequireJS as the baseUrl (See the documentation.) Since index.html sits at the root, then RequireJS resolves your module name to /app/app.require.js and does not find it.
You can work around it by using a full path:
require([
'base/newApp/app/app.require'
],
Or if it so happens that other code later is going to try to access this same module as app/app.require, then you should have a minimal configuration before your first require call:
require.config({
baseUrl: '/base/newApp',
});
It is perfectly fine to call require.config multiple times. Subsequent calls will override values that are atomic (like baseUrl) and marge values that can be merged. (An example of the latter would be paths. If the first call sets a paths for the module foo and a 2nd call sets a paths for the module bar, then the resulting configuration will have paths for both foo and bar.)
I use webpack path aliases for ES6 module loading.
E.g. If I define an alias for utils instead of something like
import Foo from "../../../utils/foo", I can do
import Foo from "utils/foo"
The problem is that once I start using aliases, WebStorm looses track of the import and I'm left with warnings and no auto-completion.
Is there a way to instruct WebStorm to use such aliases?
Yes, there is.
In fact, Webstorm can't automatically parse and apply Webpack config, but you can set up aliases the same way.
You just have to mark the parent folder of "utils" (in your example) as a resource root (right-click, mark directory as / resource root).
We just managed to do with the following structure :
/src
/A
/B
/C
We have A B and C folders declared as alias in Webpack.
And in Webstorm we marked "src" as "Resource Root".
And now we can simply import :
import A/path/to/any/file.js
instead of
import ../../../../../A/path/to/any/file.js
while still having Webstorm correctly parsing and indexing all code, link to files, autocompleting and so on ...
I managed to set up aliases for WebStorm 2017.2 within webpack like this:
For the record: in PHPSTORM, working with laravel mix, I managed to solve this by creating a webpack.config.js file separately like:
const path = require('path')
const webpack = require('webpack')
module.exports = {
...
resolve: {
extensions: ['.js', '.json', '.vue'],
alias: {
'~': path.resolve(__dirname, './resources/assets/js')
}
},
...
}
And then importing it in the webpack.mix.js like:
const config = require('./webpack.config')
...
mix.webpackConfig(config)
Make sure the webpack configuration file is pointed correctly in the configuration of the PhpStorm in: Settings > Languages & Frameworks > Javascript > Webpack
You can define custom paths, so WebStorm/PhpStorm can understand your aliases. But make sure, they are identical with your aliases. Create file in your root directory and call it something like this: webStorm.config.js (any js file will be ok). Then configure your paths inside:
System.config({
"paths": {
"components/*": "./src/components/*",
"core/*": "./src/core/*",
...
}
});
WebStorm/PhpStorm will recognize System as it's own module and will treat this file as configuration.
This is answered in a comment but to save people digging into comments and link only information, here it is:
As of WS2017.2 this will be done automatically. The information is here.
According to this, webstorm will automatically resolve aliases that are included within the webpack.config in the root of the project. If you have a custom structure and your webpack.config isn't in the root folder then go to Settings | Languages & Frameworks | JavaScript | Webpack and set the option to the config you require.
Note: Most setups have a base config which then call a dev or prod version. In order for this to work properly, you need to tell webstorm to use the dev one.
Not right now, We were also using path aliases for the files in our react project. The import names were shorter but we lost a lot on static checking of webstorm as well as completion features.
We later came up with a decision to reduce the code to only 3 levels of depth, as well a single level for the common parts. The path completion feature of webstom (ctrl + space) even helps reduce the typing overhead. The production build does not use longer names, so hardly makes any difference in final code.
I will suggest please reconsider your decision about aliases. You loose semantic meaning of modules coming from node_modules and your own code, as well as referencing the alias files again and again to make sense of your code, is a much bigger overhead.
add jsconfig.js on your project root
{
"compilerOptions": {
"baseUrl": ".",
"paths": {
"~/*": ["./src/*"]
}
}
}
In PHPStorm (using 2017.2 currently), I have not been able to get webpack configs to work properly in regards to aliases.
My fix involves using the "Directories" section of the main settings. I just had to mark each folder referenced by an alias as a sources root, then click the properties dropdown for each and specify the alias as a "Package prefix". This made everything link up for me.
Not sure if the Directories section exists in WebStorm, but if it does, this seems to be a fool-proof method for getting import aliases working.
For anyone struggling: path.resolve() must be called with "__dirname" first argument for Idea (Websorm) to be able to resolve the path correctly.
Will work for Idea (Websorm):
alias: {
'#someAlias': pathLib.resolve(__dirname, 'path/to/directory')
}
Will not work for Idea (Websorm) (while still being valid webpack alias):
alias: {
'#someAlias': pathLib.resolve('path/to/directory')
}
Webstorm can't read webpack.config if module.exports return a function.
For example
module.exports = function (webpackEnv) {
return {
mode: isEnvProduction ? 'production' : isEnvDevelopment && 'development',
...
}
}
Check your config file, maybe this cause you are a problem.
There is a lot of discussion here about Laravel Mix, so I'll leave this here to help out future readers. I solved this by creating a separate (fake) webpack config file which is only used by my IDE (PHPStorm).
1. Create a separate alias.js file (e.g. /webpack/alias.js)
const path = require('path');
const assets = path.join(__dirname,'..','resources','assets');
module.exports = {
'#js' : path.resolve(assets, 'js'),
'#c' : path.resolve(assets, 'js', 'components'),
'#errors' : path.resolve(assets, 'js', 'errors'),
'#utils' : path.resolve(assets, 'js', 'utils'),
'#store' : path.resolve(assets, 'js', 'store'),
'#api' : path.resolve(assets, 'js', 'api'),
'#less' : path.resolve(assets, 'less')
}
2. Require the alias.js file into webpack.mix.js
const mix = require('laravel-mix');
mix.alias(require('./webpack/alias'))
// ... The rest of your mix, e.g.
.js('app.js')
.vue()
.less('app.less');
3. Create the fake webpack config for your IDE (e.g. /webpack/ide.config.js)
Here, import the laravel-mix webpack config, plus your aliases, and any other config that the IDE might need help finding. Also include the prefixed ~ aliases for importing styles into your Vue components.
/*
|--------------------------------------------------------------------------
| A fake config file for PhpStorm to enable aliases
|--------------------------------------------------------------------------
|
| File > Settings... > Languages & Frameworks > Javascript > Webpack
|
| Select "Manually" and set the configuration file to this
|
*/
const path = require('path');
const mixConfig = require('./../node_modules/laravel-mix/setup/webpack.config')();
module.exports = {
...mixConfig,
resolve: {
alias: {
...require('./alias'),
'~#less' : path.resolve('#less'), // <--
},
...mixConfig.resolve
}
}
4. Set your IDE to use webpack/ide.config.js as your webpack config file.
Had the same problem on a new Laravel project with Jetstream. The webpack.config.js was present and correct. But PHPStorm still didn't recognize the # symbol as a resource root.
After opening the webpack config, I got a notification:
After Clicking on Trust project and run, the # symbol became recognized.
I know that this isn't the solution or use-case for everyone. But I still found it worthy to note on this post, because it helped me in my situation.
Using
laravel/framework:8.77.1
npm:8.3.0
node:v14.18.1
Is there any way to create Grunt task for Browserify to use fallback path?
Eg.
require('./module.js)
Check if module.js is in current directory, if not there, check some other directory?
project1
|- dir1
|- mod1.js - has: require('./mod2.js')
|- bundle.js - has: require('./mod1.js') and require('./mod2.js')
project2
|- dir2
|- mod1.js
|- mod2.js
I already tried remapify, but it can only check one specific directory.
And I don't want to change require paths inside modules, everything should be done trough Grunt.
I want to override some app (dir2), so I want to get (copy to dir1) only modules which I need to change, and to leave all others in app directory.
Gruntfile is in project1. Modules are CommonJS. I'm using grunt-browserify.
I know this is a little old now but it would be good to have an answer.
So, I asked myself a very similar question a couple of days ago and ended up doing basically what you want to do.
Using browserifyOptions from grunt-browserify like:
// Using grunt-browserify
grunt.loadNpmTasks("grunt-browserify");
grunt.initConfig({
browserify : {
options : {
browserifyOptions : {
// keep in mind that paths are relative to Gruntfile.js
paths: ["./first/path", "./second/path", "./and/so/on"]
}
},
dist : {
files: { "./path/to/dist.js" : "./path/to/src.js" }
}
},
});
And you will now be able to do:
var myModule = require("myModule");
opts.paths is an array of directories that browserify searches when
looking for modules which are not referenced using relative path. Can
be absolute or relative to basedir. Equivalent of setting NODE_PATH
environmental variable when calling browserify command. - Browserify
Theoretically, it should follow the paths in order, the only problem I've had is that it looks through native node modules (util, fs, etc..) before even searching those paths.
What the quote is saying is that if you do:
var myModule = require("./myModule");
Browserify will look for the module following the relative path. When you remove the relative path:
var myModule = require("myModule");
it can search through the stablished paths in the options.
Edit
I tested this and it works as expected. I have a module called myModule.js that is in "./current/directory", and I also have a module called myModule.js that is in "./modules/directory". Also, in current directory, I have the "src.js" that requires the module: require("myModule");. It looks in order through the paths and the first found is the one injected into bundle.js in this case.
grunt.initConfig({
browserify : {
dist : {
options : {
browserifyOptions : {
// keep in mind that paths are relative to Gruntfile.js
paths: ["./current/directory", "./modules/directory"]
}
},
files: { "./current/directory/bundle.js" : "./current/directory/src.js" }
}
},
});
Pulling my hair out here looking for a simple solution to share code, required via NPM, across multiple Browserify or Webpack bundles. Thinking, is there such a thing as a file "bridge"?
This isn't due to compile time (I'm aware of watchify) but rather the desire to extract out all of my vendor specific libs into vendor.js so to keep my app.js filesize down and to not crash the browser with massive sourcemaps. Plus, I find it way cleaner should the need to view the compiled js arise. And so:
// vendor.js
require('react');
require('lodash');
require('other-npm-module');
require('another-npm-module');
Its very important that the code be loaded from NPM as opposed to Bower, or saved into some 'vendor' directory in order to be imported via a relative path and identified via a shim. I'd like to keep every library reference pulled via NPM except for my actual application source.
In app.js I keep all of my sourcecode, and via the externals array, exclude vendor libraries listed above from compilation:
// app.js
var React = require('react');
var _ = require('lodash');
var Component = React.createClass()
// ...
And then in index.html, I require both files
// index.html
<script src='vendor.js'></script>
<script src='app.js'></script>
Using Browserify or Webpack, how can I make it so that app.js can "see" into those module loaded via npm? I'm aware of creating a bundle with externals and then referencing the direct file (in, say, node_modules) via an alias, but I'm hoping to find a solution that is more automatic and less "Require.js" like.
Basically, I'm wondering if it is possible to bridge the two so that app.js can look inside vendor.js in order to resolve dependencies. This seems like a simple, straightforward operation but I can't seem to find an answer anywhere on this wide, wide web.
Thanks!
Listing all the vendor files/modules and using CommonChunkPlugin is indeed the recommended way. This gets pretty tedious though, and error prone.
Consider these NPM modules: fastclick and mprogress. Since they have not adopted the CommonJS module format, you need to give webpack a hand, like this:
require('imports?define=>false!fastclick')(document.body);
require('mprogress/mprogress.min.css');
var Mprogress = require('mprogress/mprogress.min.js'),
Now assuming you would want both fastclick and mprogress in your vendor chunk, you would probably try this:
module.exports = {
entry: {
app: "./app.js",
vendor: ["fastclick", "mprogress", ...]
Alas, it doesn't work. You need to match the calls to require():
module.exports = {
entry: {
app: "./app.js",
vendor: [
"imports?define=>false!fastclick",
"mprogress/mprogress.min.css",
"mprogress/mprogress.min.js",
...]
It gets old, even with some resolve.alias trickery. Here is my workaround. CommonChunkPlugin lets you specify a callback that will return whether or not you want a module to be included in the vendor chunk. If your own source code is in a specific src directory, and the rest is in the node_modules directory, just reject the modules based on their path:
var node_modules_dir = path.join(__dirname, 'node_modules'),
app_dir = path.join(__dirname, 'src');
module.exports = {
entry: {
app: "./app.js",
},
output: {
filename: "bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin(
/* chunkName= */"vendor",
/* filename= */"vendor.bundle.js"
function (module, count) {
return module.resource && module.resource.indexOf(app_dir) === -1;
}
)
]
};
Where module.resource is the path to the module being considered. You could also do the opposite, and include only the module if it is inside node_modules_dir, i.e.:
return module.resource && module.resource.indexOf(node_modules_dir) === 0;
but in my situation, I'd rather say: "put everything that is not in my source source tree in a vendor chunk".
Hope that helps.
With webpack you'd use multiple entry points and the CommonChunkPlugin.
Taken from the webpack docs:
To split your app into 2 files, say app.js and vendor.js, you can require the vendor files in vendor.js. Then pass this name to the CommonChunkPlugin as shown below.
module.exports = {
entry: {
app: "./app.js",
vendor: ["jquery", "underscore", ...],
},
output: {
filename: "bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin(
/* chunkName= */"vendor",
/* filename= */"vendor.bundle.js"
)
]
};
This will remove all modules in the vendor chunk from the app chunk. The bundle.js will now contain just your app code, without any of it’s dependencies. These are in vendor.bundle.js.
In your HTML page load vendor.bundle.js before bundle.js.
<script src="vendor.bundle.js"></script>
<script src="bundle.js"></script>
// vendor anything coming from node_modules
minChunks: module => /node_modules/.test(module.resource)
Source: https://github.com/webpack/webpack/issues/2372#issuecomment-213149173