Develop Reference

Resolving dynamic module with Webpack and Laravel Mix

Good time of the day,
Recently I've been trying to implement dynamic module loading functionality for my project. However, I'm failing for past few hours. To give you an idea of what I'm trying to achieve, here is the structure of the project
plugins
developer
assets
scss
developer.scss
js
developer.js
themes
theme_name
webpack.mix.js
node_modules/
source
js
application.js
bootstrap.js
scss
application.scss
_variables.scss
So, in order to get the available plugins, I've made the following function
/**
* Get all plugins for specified developer
* which have 'assets' folder
* #param developerPath
* #param plugins
*/
function getDeveloperPlugins(developerPath, plugins) {
if (fs.existsSync(developerPath)) {
fs.readdirSync(developerPath).forEach(entry => {
let pluginPath = path.resolve(developerPath, entry),
assetsPath = path.resolve(pluginPath, 'assets');
if (fs.existsSync(assetsPath))
plugins[entry] = assetsPath;
});
}
}
This function loads all the available plugins for the specified developer, then goes inside and looks for the assets folder, if it exists, then it returns it and we can work with the provided directory later.
The next step is to generate the reference for every plugin (direct path to the developer_name.js file) which later should be 'mixed' into one plugins.bundle.js file.
In order to achieve this, the following piece of code 'emerged'
_.forEach(plugins, (directory, plugin) => {
let jsFolder = path.resolve(directory, 'js'),
scssFolder = path.resolve(directory, 'scss');
if (fs.existsSync(jsFolder)) {
webpackModules.push(jsFolder);
let possibleFile = path.resolve(jsFolder, plugin + '.js');
if (fs.existsSync(possibleFile))
pluginsBundle.js[plugin] = possibleFile;
}
if (fs.existsSync(scssFolder)) {
webpackModules.push(scssFolder);
let possibleFile = path.resolve(scssFolder, plugin + '.scss');
if (fs.existsSync(possibleFile))
pluginsBundle.scss[plugin] = possibleFile;
}
});
And the last step before I'm starting to edit the configuration of the Webpack is to get the folders for both scss and js files for all plugins and all developers:
let jsPluginsBundle = _.values(pluginsBundle.js),
scssPluginsBundle = _.values(pluginsBundle.scss);
And here is where the problems start to appear. I've tried many solutions offered either here on GitHub (in respective repositories), but I've failed so many times.
The only error I'm having now is this one:
ERROR in F:/Web/Projects/TestProject/plugins/developer/testplugin/assets/js/testplugin.js
Module build failed: ReferenceError: Unknown plugin "transform-object-rest-spread" specified in "base" at 0, attempted to resolve relative to "F:\\Web\\Projects\\TestProject\\plugins\\developer\\testplugin\\assets\\js"
Yes, i know that webpack.mix.js file should be in the root folder of the project, however, i'm just developing theme, which uses modules developed by other members of the team.
So, idea was to:
Start build process: npm run dev|prod
Load plugins for all needed developers automatically
Use methods and html tags provided by the plugin (it is a mix of PHP for API routing and Vue.js for Components, etc) as follows: <test-component></test-component>
Any help is really appreciated, i just cant get my head around that error. If you need extra information, i'm ready to help since i myself need help to solve this issue =)
Update: The latest Webpack config used by mix.webpackConfig() (still failing though)
let webpackConfiguration = {
module: {
rules: [{
test: /\.js$/,
exclude: /(node_modules|bower_components)/,
use: {
loader: require.resolve('babel-loader'),
options: {
presets: [
'babel-preset-env'
].map(require.resolve),
plugins: [
'babel-plugin-transform-object-rest-spread'
].map(require.resolve)
}
}
}]
},
resolve: {
modules: webpackModules
}
};
mix.webpackConfig(webpackConfiguration);
And this is the content of the webpackModules variable:
[
'F:\\Web\\Projects\\TestProject\\themes\\testtheme\\node_modules',
'F:\\Web\\Projects\\TestProject\\themes\\testtheme',
'F:\\Web\\Projects\\TestProject\\plugins\\developer\\testplugin\\assets\\js',
'F:\\Web\\Projects\\TestProject\\plugins\\developer\\testplugin\\assets\\scss'
]

Okay, after 7 hours I've decided to try the most obvious method to solve the problem, to create node_modules folder in the root of the project and install laravel-mix there, and it worked like a charm.
Looks like, if it cant find the module in the directory outside the root scope of the Webpack, it will go up the tree to find the node_modules folder.
Developers should allow us to set the root folder for Webpack to fetch all the modules i guess, but well, problem is solved anyways.

Webpack 2 - multiple output files with the same content

I'm trying to build 2 output files with the same content.
One tagged with the version number (taken from package.json)
and second tagged with "latest".
My (simplified) configuration looks like this:
var webpack = require('webpack');
var path = require('path');
var version = require('./package.json').version;
module.exports = {
entry: {
js: './src/main.js'
},
output: {
path: path.resolve('./dist/sdk'),
filename: [`oc-sdk-${version}.js`, 'oc-sdk-latest.js']
}
}
But this isn't currently supported by webpack. I'm getting this error:
configuration.output.filename should be a string
Is there a way to do this? Using a plugin or something?
Thanks for any advice or suggestion!

I think it would fall outside Webpack's duties.
Instead I'd suggest you to add a couple of lines to your build setup to copy/rename your files.
on-build-webpack plugin, for example, provides you a callback which is fired after the build task is completed.

Path aliases for imports in WebStorm

I use webpack path aliases for ES6 module loading.
E.g. If I define an alias for utils instead of something like
import Foo from "../../../utils/foo", I can do
import Foo from "utils/foo"
The problem is that once I start using aliases, WebStorm looses track of the import and I'm left with warnings and no auto-completion.
Is there a way to instruct WebStorm to use such aliases?

Yes, there is.
In fact, Webstorm can't automatically parse and apply Webpack config, but you can set up aliases the same way.
You just have to mark the parent folder of "utils" (in your example) as a resource root (right-click, mark directory as / resource root).
We just managed to do with the following structure :
/src
/A
/B
/C
We have A B and C folders declared as alias in Webpack.
And in Webstorm we marked "src" as "Resource Root".
And now we can simply import :
import A/path/to/any/file.js
instead of
import ../../../../../A/path/to/any/file.js
while still having Webstorm correctly parsing and indexing all code, link to files, autocompleting and so on ...

I managed to set up aliases for WebStorm 2017.2 within webpack like this:

For the record: in PHPSTORM, working with laravel mix, I managed to solve this by creating a webpack.config.js file separately like:
const path = require('path')
const webpack = require('webpack')
module.exports = {
...
resolve: {
extensions: ['.js', '.json', '.vue'],
alias: {
'~': path.resolve(__dirname, './resources/assets/js')
}
},
...
}
And then importing it in the webpack.mix.js like:
const config = require('./webpack.config')
...
mix.webpackConfig(config)
Make sure the webpack configuration file is pointed correctly in the configuration of the PhpStorm in: Settings > Languages & Frameworks > Javascript > Webpack

You can define custom paths, so WebStorm/PhpStorm can understand your aliases. But make sure, they are identical with your aliases. Create file in your root directory and call it something like this: webStorm.config.js (any js file will be ok). Then configure your paths inside:
System.config({
"paths": {
"components/*": "./src/components/*",
"core/*": "./src/core/*",
...
}
});
WebStorm/PhpStorm will recognize System as it's own module and will treat this file as configuration.

This is answered in a comment but to save people digging into comments and link only information, here it is:
As of WS2017.2 this will be done automatically. The information is here.
According to this, webstorm will automatically resolve aliases that are included within the webpack.config in the root of the project. If you have a custom structure and your webpack.config isn't in the root folder then go to Settings | Languages & Frameworks | JavaScript | Webpack and set the option to the config you require.
Note: Most setups have a base config which then call a dev or prod version. In order for this to work properly, you need to tell webstorm to use the dev one.

Not right now, We were also using path aliases for the files in our react project. The import names were shorter but we lost a lot on static checking of webstorm as well as completion features.
We later came up with a decision to reduce the code to only 3 levels of depth, as well a single level for the common parts. The path completion feature of webstom (ctrl + space) even helps reduce the typing overhead. The production build does not use longer names, so hardly makes any difference in final code.
I will suggest please reconsider your decision about aliases. You loose semantic meaning of modules coming from node_modules and your own code, as well as referencing the alias files again and again to make sense of your code, is a much bigger overhead.

add jsconfig.js on your project root
{
"compilerOptions": {
"baseUrl": ".",
"paths": {
"~/*": ["./src/*"]
}
}
}

In PHPStorm (using 2017.2 currently), I have not been able to get webpack configs to work properly in regards to aliases.
My fix involves using the "Directories" section of the main settings. I just had to mark each folder referenced by an alias as a sources root, then click the properties dropdown for each and specify the alias as a "Package prefix". This made everything link up for me.
Not sure if the Directories section exists in WebStorm, but if it does, this seems to be a fool-proof method for getting import aliases working.

For anyone struggling: path.resolve() must be called with "__dirname" first argument for Idea (Websorm) to be able to resolve the path correctly.
Will work for Idea (Websorm):
alias: {
'#someAlias': pathLib.resolve(__dirname, 'path/to/directory')
}
Will not work for Idea (Websorm) (while still being valid webpack alias):
alias: {
'#someAlias': pathLib.resolve('path/to/directory')
}

Webstorm can't read webpack.config if module.exports return a function.
For example
module.exports = function (webpackEnv) {
return {
mode: isEnvProduction ? 'production' : isEnvDevelopment && 'development',
...
}
}
Check your config file, maybe this cause you are a problem.

There is a lot of discussion here about Laravel Mix, so I'll leave this here to help out future readers. I solved this by creating a separate (fake) webpack config file which is only used by my IDE (PHPStorm).
1. Create a separate alias.js file (e.g. /webpack/alias.js)
const path = require('path');
const assets = path.join(__dirname,'..','resources','assets');
module.exports = {
'#js' : path.resolve(assets, 'js'),
'#c' : path.resolve(assets, 'js', 'components'),
'#errors' : path.resolve(assets, 'js', 'errors'),
'#utils' : path.resolve(assets, 'js', 'utils'),
'#store' : path.resolve(assets, 'js', 'store'),
'#api' : path.resolve(assets, 'js', 'api'),
'#less' : path.resolve(assets, 'less')
}
2. Require the alias.js file into webpack.mix.js
const mix = require('laravel-mix');
mix.alias(require('./webpack/alias'))
// ... The rest of your mix, e.g.
.js('app.js')
.vue()
.less('app.less');
3. Create the fake webpack config for your IDE (e.g. /webpack/ide.config.js)
Here, import the laravel-mix webpack config, plus your aliases, and any other config that the IDE might need help finding. Also include the prefixed ~ aliases for importing styles into your Vue components.
/*
|--------------------------------------------------------------------------
| A fake config file for PhpStorm to enable aliases
|--------------------------------------------------------------------------
|
| File > Settings... > Languages & Frameworks > Javascript > Webpack
|
| Select "Manually" and set the configuration file to this
|
*/
const path = require('path');
const mixConfig = require('./../node_modules/laravel-mix/setup/webpack.config')();
module.exports = {
...mixConfig,
resolve: {
alias: {
...require('./alias'),
'~#less' : path.resolve('#less'), // <--
},
...mixConfig.resolve
}
}
4. Set your IDE to use webpack/ide.config.js as your webpack config file.

Had the same problem on a new Laravel project with Jetstream. The webpack.config.js was present and correct. But PHPStorm still didn't recognize the # symbol as a resource root.
After opening the webpack config, I got a notification:
After Clicking on Trust project and run, the # symbol became recognized.
I know that this isn't the solution or use-case for everyone. But I still found it worthy to note on this post, because it helped me in my situation.
Using
laravel/framework:8.77.1
npm:8.3.0
node:v14.18.1

Using Gulp to create angular $templateCache per module/directory

So, I'm moving from grunt to gulp (or trying to anyway), and I'm having trouble getting gulp to do what I'm doing in grunt. Specifically the $templateCache stuff.
My angular app is broken up into several components/modules. Each module contains everything it needs to run (controllers, directives, partials, scss, etc.).
Using Grunt, I've been able to boil each module down into 5 files:
module.min.css // all module scss files compiled and concatenated
module.min.js // all module controllers, directives, services, etc. concatenated
module.tpls.min.js // all partials in $templateCache for this module
module.mocks.min.js // all unit test mock objects for this module
module.specs.min.js // all unit test specs for this module
This has worked really well for 2 years now and been a cornerstone of my modular architecture. My only reasons to try out gulp was 1) Curiosity, 2) My grunt file is getting kinda hairy as we add in deployment and environment specific stuff and so far gulp has really slimmed that down.
For the most part, I've figured out how to do all my grunt tasks in gulp, but I'm having trouble figuring out how to generate a template cache file for each module. All the gulp-ng|angular-templates|templatecache plugins take all my partials and create one file. I'd like to take all my files under module/partials/*.html and create a single module.tpls.min.js; and do that for each module.
This was actually a problem with grunt too, but I figured it out with grunt.file.expand().forEach() like this:
grunt.registerTask('prepModules', '...', function(){
// loop through our modules directory and create subtasks
// for each module, modifying tasks that affect modules.
grunt.file.expand("src/js/modules/*").forEach(function (dir) {
// get the module name by looking at the directory we're in
var mName = dir.substr(dir.lastIndexOf('/') + 1);
// add ngtemplate subtasks for each module, turning
// all module partials into $templateCache objects
ngtemplates[mName] = {
module: mName,
src: dir + "/partials/**/*.html",
dest: 'dev/modules/' + mName + '/' + mName + '.tpls.min.js'
};
grunt.config.set('ngtemplates', ngtemplates);
});
});
My current gulp for this same task:
var compileTemplates = gulp.src('./src/js/modules/**/partials/*.html', {base:'.'})
.pipe(ngTemplates())
.pipe(gulp.dest('.'));
I've only really looked at the options, but none of them seemed to do what I wanted. They were all around changing the file name, or the final destination of the file, or a module name, or whatever else; nothing that said anything about doing it for only the directory it happens to be in.
I had thought about using gulp-rename because it worked well for me when doing the CSS compilation:
var compileScss = gulp.src('./src/js/modules/**/scss/*.scss', {base:'.'})
.pipe(sass({includePaths: ['./src/scss']}))
.pipe(rename(function(path){
path.dirname = path.dirname.replace(/scss/,'css');
}))
.pipe(gulp.dest('.'));
However, when I pipe rename() after doing ngTemplates() it only has the path of the final output file (one log entry). When you console.log() path after sass(), it has all the paths of all the files that it found (lots of log entries).
Any ideas? Thanks!

This SO post has the correct answer, but the wasn't coming up in my searches for this specific usage. I was going to vote to close my question, but since someone else might search using my own specific terms (since I did), it seems more appropriate to leave it alone and just redirect to the original question as well as show how I solved my own particular problem.
var fs = require('fs');
var ngTemplates = require('gulp-ng-templates');
var rename = require('gulp-rename');
var modulesDir = './src/js/modules/';
var getModules = function(dir){
return fs.readdirSync(dir)
.filter(function(file){
return fs.statSync(path.join(dir, file)).isDirectory();
});
};
gulp.task('default', function(){
var modules = getModules(modulesDir);
var moduleTasks = modules.map(function(folder){
// get all partials for this module
// parse into $templateCache file
// rename to be /dev/modules/_____/______.tpls.min.js
return gulp.src(modulesDir + folder + '/partials/*.html', {basedir:'.'})
.pipe(ngTemplates({module:folder}))
.pipe(rename(function(path){
path.dirname = './dev/apps/' + folder + '/';
path.basename = folder + '.tpls.min';
}))
.pipe(gulp.dest('.'));
});
});
It's essentially like the tasks per folder recipe but with a change to use gulp-ng-templates. I'll probably be using this same pattern for my SCSS and JS now that I'm more aware of it.
Seems like the gulp equivalent of grunt.file.expand().forEach().

Whenever I deal with scss/sass for gulp tasks, I will only put one scss file as the source parameter. This parameter file is composed of a list of imports. This way you don't need to rely on gulp to concat the scss file contents for you.
//in gulpfile
gulp.src('./src/js/modules/**/scss/main.scss', {base:'.'})
//in main.scss
#import 'a', 'b', 'c';
a, b, and c would represent your other scss files.

Simple solution to share modules loaded via NPM across multiple Browserify or Webpack bundles

Pulling my hair out here looking for a simple solution to share code, required via NPM, across multiple Browserify or Webpack bundles. Thinking, is there such a thing as a file "bridge"?
This isn't due to compile time (I'm aware of watchify) but rather the desire to extract out all of my vendor specific libs into vendor.js so to keep my app.js filesize down and to not crash the browser with massive sourcemaps. Plus, I find it way cleaner should the need to view the compiled js arise. And so:
// vendor.js
require('react');
require('lodash');
require('other-npm-module');
require('another-npm-module');
Its very important that the code be loaded from NPM as opposed to Bower, or saved into some 'vendor' directory in order to be imported via a relative path and identified via a shim. I'd like to keep every library reference pulled via NPM except for my actual application source.
In app.js I keep all of my sourcecode, and via the externals array, exclude vendor libraries listed above from compilation:
// app.js
var React = require('react');
var _ = require('lodash');
var Component = React.createClass()
// ...
And then in index.html, I require both files
// index.html
<script src='vendor.js'></script>
<script src='app.js'></script>
Using Browserify or Webpack, how can I make it so that app.js can "see" into those module loaded via npm? I'm aware of creating a bundle with externals and then referencing the direct file (in, say, node_modules) via an alias, but I'm hoping to find a solution that is more automatic and less "Require.js" like.
Basically, I'm wondering if it is possible to bridge the two so that app.js can look inside vendor.js in order to resolve dependencies. This seems like a simple, straightforward operation but I can't seem to find an answer anywhere on this wide, wide web.
Thanks!

Listing all the vendor files/modules and using CommonChunkPlugin is indeed the recommended way. This gets pretty tedious though, and error prone.
Consider these NPM modules: fastclick and mprogress. Since they have not adopted the CommonJS module format, you need to give webpack a hand, like this:
require('imports?define=>false!fastclick')(document.body);
require('mprogress/mprogress.min.css');
var Mprogress = require('mprogress/mprogress.min.js'),
Now assuming you would want both fastclick and mprogress in your vendor chunk, you would probably try this:
module.exports = {
entry: {
app: "./app.js",
vendor: ["fastclick", "mprogress", ...]
Alas, it doesn't work. You need to match the calls to require():
module.exports = {
entry: {
app: "./app.js",
vendor: [
"imports?define=>false!fastclick",
"mprogress/mprogress.min.css",
"mprogress/mprogress.min.js",
...]
It gets old, even with some resolve.alias trickery. Here is my workaround. CommonChunkPlugin lets you specify a callback that will return whether or not you want a module to be included in the vendor chunk. If your own source code is in a specific src directory, and the rest is in the node_modules directory, just reject the modules based on their path:
var node_modules_dir = path.join(__dirname, 'node_modules'),
app_dir = path.join(__dirname, 'src');
module.exports = {
entry: {
app: "./app.js",
},
output: {
filename: "bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin(
/* chunkName= */"vendor",
/* filename= */"vendor.bundle.js"
function (module, count) {
return module.resource && module.resource.indexOf(app_dir) === -1;
}
)
]
};
Where module.resource is the path to the module being considered. You could also do the opposite, and include only the module if it is inside node_modules_dir, i.e.:
return module.resource && module.resource.indexOf(node_modules_dir) === 0;
but in my situation, I'd rather say: "put everything that is not in my source source tree in a vendor chunk".
Hope that helps.

With webpack you'd use multiple entry points and the CommonChunkPlugin.
Taken from the webpack docs:
To split your app into 2 files, say app.js and vendor.js, you can require the vendor files in vendor.js. Then pass this name to the CommonChunkPlugin as shown below.
module.exports = {
entry: {
app: "./app.js",
vendor: ["jquery", "underscore", ...],
},
output: {
filename: "bundle.js"
},
plugins: [
new webpack.optimize.CommonsChunkPlugin(
/* chunkName= */"vendor",
/* filename= */"vendor.bundle.js"
)
]
};
This will remove all modules in the vendor chunk from the app chunk. The bundle.js will now contain just your app code, without any of it’s dependencies. These are in vendor.bundle.js.
In your HTML page load vendor.bundle.js before bundle.js.
<script src="vendor.bundle.js"></script>
<script src="bundle.js"></script>

// vendor anything coming from node_modules
minChunks: module => /node_modules/.test(module.resource)
Source: https://github.com/webpack/webpack/issues/2372#issuecomment-213149173