Webpack: ModuleNotFoundError for an unimported module - javascript

The problem
My project consists of a yarn monorepo in which, among various packages, there is a NextJS app and a configuration package (which is also shared with the server and a react-native application). In the configuration module what I do is to export the production keys if the environment is production, while if the project is under development, the development ones.
import { merge } from "lodash";
import { IConfig, RecursivePartial } from "./interfaces";
import { defaultKeys } from "./default.keys";
import { existsSync } from "fs";
const secretKeys: RecursivePartial<IConfig> = (function (env) {
switch (env) {
case "production":
return require("./production.keys").productionKeys;
default:
try {
if (!existsSync("./development.keys")) {
return require("./production.keys").productionKeys;
} else {
return require("./development.keys").developmentKeys;
}
} catch (e) {
}
}
})(process.env.NODE_ENV);
export const keys = merge(defaultKeys, secretKeys) as IConfig;
Of course, the development configuration file is included in the .gitignore file and therefore does not exist during production.
This has always worked perfectly (with the server and the mobile app), and indeed, there was never the need to check with the fs module if the development.keys file exists (check which I added later).
However, I recently added a NextJs application to the project. I encountered no problems during development, however when I tried to deploy the app on Heroku I encountered this error:
ModuleNotFoundError: Module not found: Error: Can't resolve './development.keys' in '/tmp/build_43d652bc/packages/config/dist/keys'
What I did
Initially, I thought it was a problem with the environment variables and that in production the development file was wrongly required.
However, I later realized that this was not the problem. And indeed, even placing the import of the configuration file for development in any place of the code, even following a return, the error continues to occur.
import { merge } from "lodash";
import { IConfig, RecursivePartial } from "./interfaces";
import { defaultKeys } from "./default.keys";
import { existsSync } from "fs";
const secretKeys: RecursivePartial<IConfig> = (function (env) {
switch (env) {
case "production":
return require("./production.keys").productionKeys;
default:
try {
if (!existsSync("./development.keys")) {
return require("./production.keys").productionKeys; // <-------
} else {
return require("./production.keys").productionKeys; // <-------
}
require("./development.keys").developmentKeys; // <------- This line should not be executed
} catch (e) {
return require("./production.keys").productionKeys;
}
}
})(process.env.NODE_ENV);
export const keys = merge(defaultKeys, secretKeys) as IConfig;
It is as if during the build, nextjs (or probably webpack) controls all the imports, without following the "flow" of the code.
I hope someone shows me where I am wrong because at the moment I am stuck. Thank you!
Update
Thanks to the ideas of this discussion I changed my file which now looks like this:
const getConfigPath = (env?: string): string => {
console.log(env);
if (env === "production") return "production.keys";
else return "development.keys";
};
const secretKeys: RecursivePartial<IConfig> = require("./" +
getConfigPath(process.env.NODE_ENV)).keys;
export const keys = merge(defaultKeys, secretKeys) as IConfig;
However, now I'm running into another webpack error:
Module parse failed: Unexpected token (2:7)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| import { IConfig, RecursivePartial } from "../interfaces";
> export declare const keys: RecursivePartial<IConfig>;
It is as if webpack does not recognize declaration files generated by typescript. However, the error is new and I have never encountered it. I believe it's due to the behavior of webpack pointed out in the linked discussion.
I trust in some help, since I know little about webpack
Edit
This is my next.config.js:
const path = require("path");
module.exports = {
distDir: '../../.next',
sassOptions: {
includePaths: [path.join(__dirname, 'styles')],
prependData: `#import "variables.module.scss";`
},
webpack: (config, { isServer }) => {
if (!isServer) {
config.resolve.fallback.fs = false;
}
return config;
},
};
Basically they are the defult configurations. The only thing I have changed is the path for the build, I have made a sass file globle and I have momentarily added a piece of code to allow the use of the fs module, which however as you can see above I do not use it more. So I could take this configuration out.

You need the loader for typescript configured in your next.config.js
npm install || yarn add awesome-typescript-loader
module.exports = {
distDir: '../../.next',
sassOptions: {
includePaths: [path.join(__dirname, 'styles')],
prependData: `#import "variables.module.scss";`
},
webpack: (config, options) => {
const { dir, defaultLoaders, isServer } = options;
if (!isServer) {
config.resolve.fallback.fs = false;
}
config.pageExtensions.push(".ts", ".tsx");
config.resolve.extensions.push(".ts", ".tsx");
config.module.rules.push({
test: /\.tsx?$/,
include: [dir],
exclude: /node_modules/,
use: [
defaultLoaders.babel,
{
loader: "awesome-typescript-loader",,
},
],
});
return config;
}
}

Related

How to configure webpack 5 to bundle the module Web Worker script correctly which imports other modules?

As per the documentation, I am not using worker-loader, I am trying to use the native way suggested by the webpack-5 documentation.
Below is the usage of the worker script in the main thread.
const worker = new window.Worker(
new URL("../workers/listOperation.worker.js", import.meta.url),
{
type: "module",
},
);
worker.postMessage({ list: hugeList, params: reqData });
worker.onerror = err => console.error(err);
worker.onmessage = e => {
const { list } = e.data;
// Usage of `list` from the response
worker.terminate();
};
return worker;
It works fine if there are no imports used in the script. But when I import any node modules (e.g. loadash/get) or any other function/constants from local, it does not work as the output webWorker bundle file doesn't transpile and bundle the imported code. It keeps the "import" statement as it is.
Below is the worker script (listOperation.worker.js)
import get from "lodash/get";
import { ANY_CONSTANT } from "../constants"; // This is some local constant
addEventListener("message", e => {
const { list, params } = e.data;
// Here I have some usage of `get` method from `lodash/get` and ANY_CONSTANT
self.postMessage({
list: list,
});
});
Webpack outputs the bundle file like below which won't be usable by the browser, if I put the /\.worker.js$/ pattern in the the exclude of babel-loader rule.
import get from "lodash/get";import{ANY_CONSTANT}from"../constants";addEventListener("message",(e=>{const{list:s,params:t,.......
And even if I don't put the /\.worker.js$/ pattern in the the exclude of babel-loader rule, the output bundle still doesn't include the implementation of get from lodash/get or the value of the constant. It just outputs it in cjs using require.
Also, I made use of asset module so that I can put the output file inside a directory, not directly in the root of the dist folder. Configuration changes in my webpack.config looks like this.
module.exports = {
entry: {...},
module: {
rules: [
{
test: /\.js$/,
exclude: [/(node_modules)/, /\.worker.js$/],
use: {
loader: "babel-loader", // This uses the config defined in babel.config.js
},
},
{
test: /\.worker.js$/,
exclude: /(node_modules)/,
type: "asset/resource",
generator: {
filename: "js/workers/[hash][ext][query]",
},
},
],
},
}

Yarn Workspaces, workspace does not emit errors or warnings

I have followed the following post in order to create a monorepo using yarn workspaces and craco.
It works really well except one thing: the errors/warnings of the common (components )library are not emitted to the console.
The structure is very simple:
monorepo
|-packages
|-components
|-fe
Fe is the main webApp that uses the components library.
The FE emits all warnings correctly, components does not.
How to make the shared component emit warnings/errors?
Updated:
Steps to reproduce in this repo:
https://github.com/sofoklisM/my-monorepo.git
What you need to change is the context option of the underlying ESLint Webpack plugin that is used by Create React App.
In this case I changed the context of ESLint to the root of the monorepo (yarn workspace root).
Here is an updated craco.config.js that should do the trick:
// craco.config.js
const path = require("path");
const { getLoader, loaderByName } = require("#craco/craco");
const { getPlugin, pluginByName } = require("#craco/craco/lib/webpack-plugins")
const absolutePath = path.join(__dirname, "../components");
module.exports = {
webpack: {
alias: {},
plugins: [],
configure: (webpackConfig, { env, paths }) => {
const { isFound, match } = getLoader(
webpackConfig,
loaderByName("babel-loader")
);
if (isFound) {
const include = Array.isArray(match.loader.include)
? match.loader.include
: [match.loader.include];
match.loader.include = include.concat([absolutePath]);
}
// Change context of ESLint Webpack Plugin
const { match: eslintPlugin } = getPlugin(webpackConfig, pluginByName("ESLintWebpackPlugin"));
eslintPlugin.options['context'] = path.join(__dirname, "../..");
return webpackConfig;
}
}
};
I've also made an updated fork of your reproduction repo here: https://github.com/ofhouse/stackoverflow-65447779

NuxtJS: Disable console.log in production env

I am looking for a way to disable console.log() for production env. Something like putting the below code to nuxt.config.js or index.js:
if (process.env.NODE_ENV !== "development") {
console.log = () => {};
}
I tried it, but it doesn't work. Any help would be appreciated.
My nuxt.config.js is here
https://gist.github.com/somaria/9a2b0e06497d13a35fe9eee141a15d07
Nuxt's build process includes terser, which can be configured to automatically remove console statements from your production build. You could set build.terser.terserOptions:
// nuxt.config.js
export default {
build: {
terser: {
// https://github.com/terser/terser#compress-options
terserOptions: {
compress: {
drop_console: true
}
}
}
}
}
As an alternative, this can also be done with Plugins.
Under Plugins folder, we can create a file called disableLogs.js which can look like so:
// plugins/disableLogs.js
export function disableLogs() {
console.log = () => {};
// or you can override any other stuff you want
}
process.env.NODE_ENV === "production" ? disableLogs() : null;
Then we can register this plugin to be used inside nuxt.config.js
// nuxt.config.js
plugins: [
{ src: "~/plugins/disableLogs.js" },
{ src: "~/plugins/any-other-plugin.js"
],
This will run before instantiating the root Vue.js Application.
There are other things where you can configure it to run either client or server side, etc. More info here - https://nuxtjs.org/guide/plugins#vue-plugins

webpack: import + module.exports in the same module caused error

I'm developing a website with webpack. When I have a code like this:
import $ from 'jquery';
function foo() {};
module.exports = foo;
I got the error Uncaught TypeError: Cannot assign to read only property 'exports' of object '#<Object>'.
Turns out that changing import $ from 'jquery' to var $ = require('jquery') don't cause any errors.
Why import with module.exports causes this error? Is anything wrong in using require instead?
You can't mix import and module.exports. In the import world, you need to export things.
// Change this
module.exports = foo;
// To this
export default foo;
This happens if other modules down stream have an unexpected require tree. Babel changes require to import where it isn't supposed to which causes the aforementioned issue #Matthew Herbst. To solve this add "sourceType": "unambiguous" to your babelrc file or babel.config.js so that #babel/plugin-transform-runtime won't do this change of require expression to import in your commonjs files. eg:
module.exports = {
presets: [
'#quasar/babel-preset-app'
],
"sourceType": "unambiguous"
}
You can use require with export. But not import and module.exports.
In my react-native-web case, just use an additional webpack rule, then the TypeError: Cannot assign to read only property 'exports' of object is fixed. Maybe you can ref to it.
npm install --save-dev react-app-rewired
Create a config-overrides.js in your project root
// used by react-app-rewired
const webpack = require('webpack');
const path = require('path');
module.exports = {
webpack: function (config, env) {
config.module.rules[1].use[0].options.baseConfig.extends = [
path.resolve('.eslintrc.js'),
];
// To let alias like 'react-native/Libraries/Components/StaticRenderer'
// take effect, must set it before alias 'react-native'
delete config.resolve.alias['react-native'];
config.resolve.alias['react-native/Libraries/Components/StaticRenderer'] =
'react-native-web/dist/vendor/react-native/StaticRenderer';
config.resolve.alias['react-native'] = path.resolve(
'web/aliases/react-native',
);
// Let's force our code to bundle using the same bundler react native does.
config.plugins.push(
new webpack.DefinePlugin({
__DEV__: env === 'development',
}),
);
// Need this rule to prevent `Attempted import error: 'SOME' is not exported from` when `react-app-rewired build`
// Need this rule to prevent `TypeError: Cannot assign to read only property 'exports' of object` when `react-app-rewired start`
config.module.rules.push({
test: /\.(js|tsx?)$/,
// You can exclude the exclude property if you don't want to keep adding individual node_modules
// just keep an eye on how it effects your build times, for this example it's negligible
// exclude: /node_modules[/\\](?!#react-navigation|react-native-gesture-handler|react-native-screens)/,
use: {
loader: 'babel-loader',
},
});
return config;
},
paths: function (paths, env) {
paths.appIndexJs = path.resolve('index.web.js');
paths.appSrc = path.resolve('.');
paths.moduleFileExtensions.push('ios.js');
return paths;
},
};
Also create a web/aliases/react-native/index.js
// ref to https://levelup.gitconnected.com/react-native-typescript-and-react-native-web-an-arduous-but-rewarding-journey-8f46090ca56b
import {Text as RNText, Image as RNImage} from 'react-native-web';
// Let's export everything from react-native-web
export * from 'react-native-web';
// And let's stub out everything that's missing!
export const ViewPropTypes = {
style: () => {},
};
RNText.propTypes = {
style: () => {},
};
RNImage.propTypes = {
style: () => {},
source: () => {},
};
export const Text = RNText;
export const Image = RNImage;
// export const ToolbarAndroid = {};
export const requireNativeComponent = () => {};
Now you can just run react-app-rewired start instead of react-scripts start

Compiling Webpack in memory but resolving to node_modules on disk

I'm trying to use web pack to compile an in memory string of valid javascript code. I'm using memory fs as outlined here: https://webpack.github.io/docs/node.js-api.html#compile-to-memory.
So I'm taking a string containing raw javascript, writing that to memory fs, and then web pack resolves to that entry point. But the compilation fails on the first require statement, presumably because it's not able to look in the real fs for node_modules.
Any ideas on how can I accomplish this?
import webpack from 'webpack';
import MemoryFS from 'memory-fs';
import thenify from 'thenify';
function* compile(code) {
const fs = new MemoryFS();
fs.writeFileSync('/file.js', code);
const compiler = webpack({
entry: { file: '/file.js' },
output: {
path: '/build',
filename: '[name].js'
},
module: {
loaders: [
{ test: /\.json$/, loader: 'json' }
],
}
});
compiler.run = thenify(compiler.run);
compiler.inputFileSystem = fs;
compiler.resolvers.normal.fileSystem = fs; //this is needed for memfs
compiler.outputFileSystem = fs;
const stats = yield compiler.run();
//retrieve the output of the compilation
const res = stats.compilation.assets['file.js'].source();
return res;
}
Usage
var code = "var _ = require('underscore'); console.log(_);";
var bundle = yield compile(code); //should be a bundle containing the underscore source.
The error is
ModuleNotFoundError: Module not found: Error: Cannot resolve module
underscore in /
This question indicates that others have tried the same thing: https://github.com/webpack/webpack/issues/1562. there's a gist referenced at https://gist.github.com/DatenMetzgerX/2a96ebf287b4311f4c18 that I believe was intended to do what I'm hoping to accomplish, but in it's current form I don't see how. It assigns an instance of MemoryFs to all of the resolvers. I've tried assigning node's fs module, but no dice.
So in short, I'm trying to set an entry point to an in memory string of raw javascript, but still have require and import statements resolved to node_modules on disk.
UPDATE
I've been able to get the result I'm looking for but it's not pretty. I'm basically overriding the implementation of #stat and #readFile in MemoryFS
to check the real filesystem if it gets any request for a file that doesn't exist in memory. I could clean this up a bit by subclassing MemoryFS instead of swapping method implementations at runtime, but the idea would still be the same.
Working solution
import webpack from 'webpack';
import JsonLoader from 'json-loader';
import MemoryFS from 'memory-fs';
import UglifyJS from "uglify-js";
import thenify from 'thenify';
import path from 'path';
import fs from 'fs';
import root from 'app-root-path';
/*
* Provide webpack with an instance of MemoryFS for
* in-memory compilation. We're currently overriding
* #stat and #readFile. Webpack will ask MemoryFS for the
* entry file, which it will find successfully. However,
* all dependencies are on the real filesystem, so any require
* or import statements will fail. When that happens, our wrapper
* functions will then check fs for the requested file.
*/
const memFs = new MemoryFS();
const statOrig = memFs.stat.bind(memFs);
const readFileOrig = memFs.readFile.bind(memFs);
memFs.stat = function (_path, cb) {
statOrig(_path, function(err, result) {
if (err) {
return fs.stat(_path, cb);
} else {
return cb(err, result);
}
});
};
memFs.readFile = function (path, cb) {
readFileOrig(path, function (err, result) {
if (err) {
return fs.readFile(path, cb);
} else {
return cb(err, result);
}
});
};
export default function* compile(code) {
// Setup webpack
//create a directory structure in MemoryFS that matches
//the real filesystem
const rootDir = root.toString();
//write code snippet to memoryfs
const outputName = `file.js`;
const entry = path.join(rootDir, outputName);
const rootExists = memFs.existsSync(rootDir);
if (!rootExists) {
memFs.mkdirpSync(rootDir);
}
memFs.writeFileSync(entry, code);
//point webpack to memoryfs for the entry file
const compiler = webpack({
entry: entry,
output: {
filename: outputName
},
module: {
loaders: [
{ test: /\.json$/, loader: 'json' }
]
}
});
compiler.run = thenify(compiler.run);
//direct webpack to use memoryfs for file input
compiler.inputFileSystem = memFs;
compiler.resolvers.normal.fileSystem = memFs;
//direct webpack to output to memoryfs rather than to disk
compiler.outputFileSystem = memFs;
const stats = yield compiler.run();
//remove entry from memory. we're done with it
memFs.unlinkSync(entry);
const errors = stats.compilation.errors;
if (errors && errors.length > 0) {
//if there are errors, throw the first one
throw errors[0];
}
//retrieve the output of the compilation
const res = stats.compilation.assets[outputName].source();
return res;
}
Usage
var code = "var _ = require('underscore'); console.log(_);";
var bundle = yield compile(code); //is a valid js bundle containing the underscore source and a log statement logging _.
If there's not a better way, then I'll definitely encapsulate this into a subclass of MemoryFS, but I'm hoping there's a more sane way to accomplish this with Webpack's api.
Instead of memory-fs, the combination of unionfs/memfs/linkfs should help.
https://npmjs.com/unionfs
https://npmjs.com/memfs
https://npmjs.com/linkfs
I have created this snippet untested. I think you want the inputFS to be the real one and the output fs to be the in memory one. On the other hand you want all the dependencies of file.js to be constructed separately. For that I figured the webpack.optimize.CommonsChunkPlugin plugin could help. I expect webpack to write everything to the memory. I hope it works.
import webpack from 'webpack';
import MemoryFS from 'memory-fs';
import thenify from 'thenify';
import realFS from 'fs';
function* compile(code) {
const fs = new MemoryFS();
const compiler = webpack({
entry: {
file: '/file.js',
vendor: [
'underscore',
'other-package-name'
]
},
output: {
path: '/build',
filename: '[name].js'
},
module: {
loaders: [
{ test: /\.json$/, loader: 'json' }
],
},
plugins: [
new webpack.optimize.CommonsChunkPlugin('vendor', 'vendor.js')
]
});
compiler.run = thenify(compiler.run);
compiler.inputFileSystem = realFS;
compiler.resolvers.normal.fileSystem = fs; //this is needed for memfs
compiler.outputFileSystem = fs;
const stats = yield compiler.run();
//retrieve the output of the compilation
const res = stats.compilation.assets['file.js'].source();
return res;
}
You're using MemoryFS, which is a JavaScript reimplementation of a feature normally handled by the Operating System. I wonder, could you mount a directory using tmpfs at the Operating System level, then use that? webpack would then not know or care that the input file is actually stored in memory.
Assuming that you have mounted a memory-based filesystem at /media/memory, the webpack configuration code could be as simple as this:
resolve: {
root: ['/media/memory', ...other paths...],
},
output: {
path: '/wherever/you/want/the/output/files'
}
}
This approach also has a hidden benefit: If you want to debug the input code, you just mount /media/memory with a non-RAM-based filesystem and you can see what's being generated.
I know it's late but for the record here comes a code snippet.
import * as fs from 'fs';
import { resolve } from 'path';
import { Volume } from 'memfs';
import { ufs } from 'unionfs';
const volume = Volume.fromJSON({
[resolve(process.cwd(), 'test.js')]: 'this file is on memory not on disk'
});
ufs.use(fs).use(volume);
// Reads from memory
console.log(ufs.readFileSync(resolve(process.cwd(), 'test.js'), 'utf8'));
// Reads from disk
console.log(ufs.readFileSync(resolve(process.cwd(), 'package.json'), 'utf8'));
// Writing into memory
volume.writeFileSync(resolve(process.cwd(), 'test.memory'), 'This should be
on memory');
console.log(ufs.readFileSync(resolve(process.cwd(), 'test.memory'), 'utf8'));
// Writing into disk
ufs.writeFileSync(resolve(process.cwd(), 'test.disk'), 'This should be on disk');
console.log(ufs.readFileSync(resolve(process.cwd(), 'test.disk'), 'utf8'));
Hers the console output:
user1#pc playground % node inMem.mjs
this file is on memory not on disk
{
"name": "playground",
"version": "1.0.0",
"main": "index.js",
"license": "MIT",
"dependencies": {
"memfs": "^3.3.0",
"unionfs": "^4.4.0"
}
}
This should be on memory
This should be on disk
user1#pc playground % ls .
inMem.mjs node_modules package.json yarn.lock

Categories

Resources