Develop Reference

Next.js Scripts Error: Cannot find module '../../webpack-runtime.js'

I want to create RSS script using Next.js.
So I put up a script in a subfolder inside the root folder scripts/ & named it build-rss.js
next.config.js
module.exports = {
webpack: (config, options) => {
config.module.rules.push({
test: /\.svg$/,
issuer: { and: [/\.(js|ts|md)x?$/] },
use: [
{
loader: '#svgr/webpack',
options: {
prettier: false,
svgo: true,
svgoConfig: { plugins: [{ removeViewBox: false }] },
titleProp: true,
},
},
],
})
if (!options.dev && options.isServer) {
const originalEntry = config.entry
config.entry = async () => {
const entries = { ...(await originalEntry()) }
entries['./scripts/build-rss'] = './scripts/build-rss.js'
return entries
}
}
if (!options.isServer) {
config.resolve.fallback.fs = false
}
return config
},
}
When I try to run my script npm run build:development which in package.json represents:
"scripts": {
"clean": "rimraf .next",
"dev": "next dev",
"export": "next export",
"start": "next start",
"lint": "next lint",
"build:development": "next build && npm run export && npm run rss:development",
"build": "next build && npm run export && npm run rss",
"rss:development": "node ./.next/server/scripts/build-rss.js",
"rss": "node ./.next/serverless/scripts/build-rss.js"
}
It throws an error saying:
Error: Cannot find module '../../webpack-runtime.js'
But I checked. The file does exist.
The blunder is this used to work earlier. Probably few versions ago when my other project used the same combination.
I have made a complete reproduction showcasing the error → https://github.com/deadcoder0904/next-script-rss-error
Just clone it, install it & try the script npm run build:development in the terminal to see the error.

Based on our conversation:
entry: path.join(__dirname, '..', 'path/to/file')
That's what a webpack entry looks like. It can also be an array or an object:
entry: [
path.join(__dirname, '..', 'path/to/file'),
// other entries here
]
Whereas you're already getting the whole webpack config:
webpack: (config, options)
So doing:
const originalEntry = config.entry
config.entry = async () => {
const entries = { ...(await originalEntry()) }
entries['./scripts/build-rss'] = './scripts/build-rss.js'
return entries
}
Makes no sense to me if you can just do:
config.entry.push('./scripts/build-rss')
// config.entry['./scripts/build-rss'] = './scripts/build-rss.js'
Unless I miss something with how nextjs is loading the webpack config.
Even then I'd suggest that you use path.join in order to ensure it's loaded to the correct location, because that relative root will execute from wherever webpack is compiled from.
Along with that in your first project you used nextjs v10 and now you're using nextjs v11, which has an upgrade from webpack 4 to 5, which is a major upgrade. I don't know the details, I can only speculate, but under no conditions should you assume that "because your previous project was working this one should using the same stuff", it won't necessarily (especially not in this case).
The first intuitive thing I thought was that webpack should by default bundle everything to a single output file, unless the configuration for that was changed by nextjs (I don't know). So using a script you added to entries didn't make sense to me, because it wouldn't exist. But you're saying that it does exist so I can only assume that webpack is configured to do code splitting and outputs each entry to a different file. In which case I have no idea. As far as I'm aware in webpack 5 (I don't know about webpack 4) code splitting is disabled in dev and enabled in production so your problem is likely a discrepancy between dev and production.
Perhaps the last thing you can try is to change your !options.dev, because right now you're only adding that script when it's production but you're trying to run the script using development.
If you really just have to get it working you can downgrade your nextjs to the previous version you were using (v10), even though that's not really a solution.
Other than that I'm out of ideas.

Not sure if you are still looking for an answer, but simply changing the webpack entry as follows seems to have fixed the problem.
entries['scripts/build-rss'] = './scripts/build-rss.js';
// instead of entries['./scripts/build-rss']

I had the same Error! I deleted the .next Folder and did an npm run dev, It started to work for me!

Fail Gatsby build if environment variable missing

I have experimented with adding environment variables to my Gatsby project using .env.development and .env.production files and it's working great.
I would like to have my builds fail if one of the environment variables is missing, however I can't seem to see how to enable this functionality.
I have read through the Gatsby environment variables documentation, but can't seem to see how this would work? is this possible?
I believe it uses dotenv/webpack define plugin under the hood.

I’m sure there are other ways to do this, but with some quick tests, this approach seems to be working well for me.
In your gatsby-config.js file, you can choose to explicitly require the dotenv, so you can use those environment variables in your config.
I added the following, and now the Gatsby build will fail unless the specified environment variables are present.
// Load the environment variables, per
// https://www.gatsbyjs.org/docs/environment-variables/#server-side-nodejs
require('dotenv').config({
path: `.env.${process.env.NODE_ENV}`,
})
function checkEnv(envName) {
if (typeof process.env[envName] === 'undefined' || process.env[envName] === '') {
throw `Missing required environment variables: ${envName}`
}
}
try {
checkEnv('NODE_ENV')
checkEnv('EXAMPLE_MISSING_ENV')
checkEnv('EXAMPLE_API_KEY')
} catch (e) {
throw new Error(e)
}
// The rest of the config file
I could imagine customizing this further, ex. logging a warning for a variable with a fallback versus throwing an error for one that is required by your content sourcing plugin or theme. Hope this is helpful as a starting point!

I couldn't find built-in solution for this on Gatsby neither. You may do it manually, but still not too easy.
First problem: If you wanna load your environment from file while running npm script; it can not be loaded right away. But you may trigger a script file, and it can load this environment variables before your check.
lets say build.sh on root directory of project :
source ./.env.development # this line will set env variables
if [ "$API_KEY" = 927349872349798 ] ; then
npm run build
fi
Another problem rises; some developers might want to run it on windows maybe. So better use famous cross-env package.
npm i cross-env
Then everything is ready, add your secure-build :
"scripts": {
"build": "gatsby build",
"develop": "gatsby develop",
"format": "prettier --write \"**/*.{js,jsx,json,md}\"",
"start": "npm run develop",
"serve": "gatsby serve",
"clean": "gatsby clean",
"test": "echo \"Write tests! -> https://gatsby.dev/unit-testing\" && exit 1",
"secure-build": "cross-env-shell \"./build.sh\""
},
And run it :
npm run secure-build
This solution looks too much for me as we created a build.sh and install a new package. Maybe there is cleaner solution. I am not Gatsby Guru after all.

I added env checking to the onPreInit life cycle hook in gatsby-node.ts:
const envVariablesList = [
"ENV1",
"ENV2",
"ENV3",
];
function envVarChecker(vars: string[]): string | undefined {
return vars.find(
(item) => process.env[item] === undefined || process.env[item] === ""
);
}
export const onPreInit: GatsbyNode["onPreInit"] = ({ actions }) => {
const emptyEnv = envVarChecker(envVariablesList);
if (emptyEnv !== undefined) {
throw new Error(`Env variable: ${emptyEnv} is empty!`);
}
};
It fails build almost at the very beginning (during pre-bootstrap phase) if any of the declared variables is missing

node dotenv won't work with pm2

I have an application where locally (without pm2) all the environment variables in the .env file work just fine using dotenv.
But on the server where I'm using pm2 to run the app, the environment variables remain undefined.
The pm2 commands I'm using to run the app on server are:
pm2 start myapp/app.js
pm2 startup
pm2 save

dotenv will read .env file located in the current directory.
When you call pm2 start myapp/app.js it won't search for myapp/.env.
.env // It will try to load this, which doesn't exist
myapp/
app.js
So you have two solutions
use path option:
const path = require('path');
require('dotenv').config({ path: path.join(__dirname, '.env') });
Or call your script from inside myapp/
pm2 start app.js

A good pattern here is to remove dotenv from your code and "require" it on the command line. This makes your code nicely transportable between any environment (including cloud-based) - which is one of the main features of environment variables.
Note: you will still need to install dotenv in your project via npm when running it on a server.
a) code up your .env file alongside your script (e.g. app.js)
b) to run your script without pm2:
node -r dotenv/config app.js
c) in pm2.config.js:
module.exports = {
apps : [{
name : 'My Application',
script : 'app.js',
node_args : '-r dotenv/config',
...
}],
}
and then
pm2 start pm2.config.js
note: the use of dotenv/config on the command line is one of the best practices recommended by dotenv themselves
edit 2021: for completeness - as my answer has got some ticks, I wanted to add a 4th option to the list:
d) combined pm2/env config
module.exports = { apps : [{
name : 'My Application',
script : 'app.js',
env : {
PORT: 5010,
DB_STRING: 'mongodb://localhost:27017',
...
},
}]};
This will be useful if you are treating your pm2.config as environmental configuration and outside of git etc. It just negates the need for a separate .env, which may suit you. It negates the need for dotenv completely as pm2 injects the env variables into your script's process

you have kill you pm2 process first
try
pm2 kill
then restart pm2 using
pm2 start app.js

I had the same problem but it wasnt explained clearly so here is the solution based on
github user vmarchaud comment.
This also fixes the issue people had with #Andy Lorenz solution.
In my case i wanted to create an ecosystem file for multiple apps but i was keep getting
Error: Cannot find module 'dotenv/config'
The solution was easy.
You have to declar cwd, aka the project folder where the dotenv/config will be read from.
module.exports = {
apps: [{
name: 'app1 name',
script: 'app1.js',
cwd: '/path/to/folder/',
exec_mode: 'fork_mode',
node_args: '-r dotenv/config',
}, {
name: 'app2 name',
script: 'app2.js',
cwd: '/path/to/folder/',
instances: 'max',
exec_mode: 'cluster',
node_args: '-r dotenv/config',
}],
};

You can parse .env using dotenv lib end set them manually in ecosystem.config.js
ecosystem.config.js:
const { calcPath, getEnvVariables } = require('./helpers');
module.exports = {
apps: [
{
script: calcPath('../dist/app.js'),
name: 'dev',
env: getEnvVariables(),
},
],
};
helpers.js:
const path = require('path');
const dotenv = require('dotenv');
const fs = require('fs');
function calcPath(relativePath) {
return path.join(__dirname, relativePath);
}
// this function will parce `.env` file but not set them to `process.env`
const getEnvVariables = () => {
const envConfig = dotenv.parse(fs.readFileSync(calcPath('.env')));
const requiredEnvVariables = ['MODE'];
for (envVariable of requiredEnvVariables) {
if (!envConfig[envVariable]) {
throw new Error(`Environment variable "${envVariable}" is not set`);
}
}
return envConfig;
};

None of this worked for me because I was using cluster mode.
I installed dotenv as dev dependency at the root (I was using yarn workspaces too).
Then I did this:
require('dotenv').config({ path: 'path/to/your/.env' })
module.exports = {
apps: [
{
name: 'app',
script: 'server/dist/index.js',
instances: 2,
exec_mode: 'cluster',
instance_var: 'APP_INSTANCE_SEQ',
// listen_timeout: 10000,
// restart_delay: 10000,
}
]
}

I use a much simpler version of #Marcos answer:
.env
app.js
for example we need to store token in .env file and pass it right to app.js:
inside .env
token=value
inside app.js:
require('dotenv').config();
console.log(process.env.token)
Also, don't forget. If you add .env file to .gitignore and then git pull you repo on VPS or smth, you need to copy .env file manually, otherwise your app won't work.
And in some cases it's important in what area you are using your config, so make sure that NODE_ENV=production string is added to your .env file.
After all you could use pm2 start app.js right from your app's folder.

This was my project setup..
/src/app.ts
which than compiled into dist folder.
/dist/app.js
my .env file was outside dist folder so it wasn't accessible.
this is the command i tried.
pm2 start app.js --env=.env

Setting environment variables in Gatsby

I used this tutorial: https://github.com/gatsbyjs/gatsby/blob/master/docs/docs/environment-variables.md
Steps I followed:
1) install dotenv#4.0.0
2) Create two files in root folder: ".env.development" and ".env.production"
3) "follow their setup instructions" (example on dotenv npm docs)
In gatsby-config.js:
const fs = require('fs');
const dotenv = require('dotenv');
const envConfig =
dotenv.parse(fs.readFileSync(`.env.${process.env.NODE_ENV}`));
for (var k in envConfig) {
process.env[k] = envConfig[k];
}
Unfortunately, when i run gatsby develop, NODE_ENV isn't set yet:
error Could not load gatsby-config
Error: ENOENT: no such file or directory, open 'E:\Front-End Projects\Gatsby\sebhewelt.com\.env.undefined'
It works when I set it manually:
dotenv.parse(fs.readFileSync(`.env.development`));
I need environment variables in gatsby-config because I put sensitive data in this file:
{
resolve: `gatsby-source-contentful`,
options: {
spaceId: envConfig.CONTENTFUL_SPACE_ID,
accessToken: envConfig.CONTENTFUL_ACCESS_TOKEN
}
}
How to make it work?
PS: Additional question - As this made me think, I know I shouldn't put passwords and tokens on github, but as netlify builds from github, is there other safe way?

I had a similar issue, I created 2 files in the root ".env.development" and ".env.production" but was still not able to access the env file variables - it was returning undefined in my gatsby-config.js file.
Got it working by npm installing dotenv and doing this:
1) When running gatsby develop process.env.NODE_ENV was returning undefined, but when running gatsby build it was returning 'production' so I define it here:
let env = process.env.NODE_ENV || 'development';
2) Then I used dotenv but specify the filepath based on the process.env.NODE_ENV
require('dotenv').config({path: `./.env.${env}`});
3) Then you can access your variables for your config:
module.exports = {
siteMetadata: {
title: `Gatsby Default Starter`,
},
plugins: [
`gatsby-plugin-react-helmet`,
{
resolve: `gatsby-source-contentful`,
options: {
spaceId: `${process.env.CONTENTFUL_ID}`,
accessToken: `${process.env.CONTENTFUL_TOKEN}`,
},
},
],
}

You should only use env files when you're comfortable checking those into git. For passwords/tokens/etc. add them to Netlify or whatever build tool you use through their dashboard.
These you can access in gatsby-config.js & gatsby-node.js via process.env.ENV_VARIABLE.
You can't access environment variables added this way in the browser however. For this you'll need to use .env.development & .env.production.

I really dislike the .env.production file pattern, our build system sets up and uses env variables and having extra build steps to write those into a file is weird. But Gatsby only whitelists GATSBY_ of the env vars, with no obvious way of adding your own.
But doing that isn't so hard, you can do it by adding something like this in the gatsby-node.js file:
exports.onCreateWebpackConfig = ({ actions, getConfig }) => {
const config = getConfig();
// Allow process.env.MY_WHITELIST_PREFIX_* environment variables
const definePlugin = config.plugins.find(p => p.definitions);
for (const [k, v] of Object.entries(process.env)) {
if (k.startsWith("MY_WHITELIST_PREFIX_")) {
definePlugin.definitions[`process.env.${k}`] = JSON.stringify(v);
}
}
actions.replaceWebpackConfig(config);
};

After doing a few searches, I found that we can set environment variables through netlify website, here are the steps:
Under your own netlify console platform, please go to settings
Choose build & deploy tab (can be found on sidebar)
Choose environment sub-tab option
Click edit variables and add/put your credentials in
Done!

How do you use inline mode with the webpack-dev-server API and Gulp

I've been using the webpack-dev-server with it's --inline and --host flags. This all works fine.
webpack-dev-server --inline --host example.com
I then looked at wrapping up this task using gulp and the webpack-dev-server API.
var gulp = require('gulp');
var gutil = require('gulp-util');
var Webpack = require('webpack');
var WebpackDevServer = require('webpack-dev-server');
var WebpackConfig = require('./webpack.config.js');
gulp.task('default', ['webpack-dev-server']);
gulp.task('webpack-dev-server', function(callback) {
new WebpackDevServer(Webpack(WebpackConfig), {
host: 'example.com',
inline: true,
publicPath: WebpackConfig.output.publicPath,
}).listen(port, host, function(err) {
if(err) throw new gutil.PluginError('webpack-dev-server', err);
gutil.log('[webpack-dev-server]', 'http://example.com:8080');
});
});
This does not seem to work, I believe there is no inline or host for the API.
Any idea if this is possible?

In the current Webpack version ( 1.13.2 ) this CAN be done.
From the documentation:
To use the inline mode, either
specify --inline on the command line.
specify devServer: { inline: true } in your webpack.config.js

The inline option can't be enabled via a flag in the options passed to the Server. However by taking a look at the command line script you can see that's it just appending additional entry scripts to the options passed to the webpack compiler.
You can repeat the same thing in your code.
WebpackConfig.entry = [
WebPackConfig.entry, // assuming you have a single existing entry
require.resolve("webpack-dev-server/client/") + '?http://localhost:9090',
'webpack/hot/dev-server'
]

The inline option in not available when using API approach to create webpack-dev-server, instead we need to manually define
webpack-dev-server/client?http://<path>:<port>/
to (all) entry point(s). Reason being the webpack-dev-server module
has no access to the webpack configuration.
https://webpack.github.io/docs/webpack-dev-server.html#inline-mode-with-node-js-api

It seems that that the answers got outdated and I did not manage to use any of them to add the inline via gulp, so I opened webpack-dev-server.js and copied the method that does that to my gulp file and modified it a bit. It works (even though it's a bit nasty):
function addInline(config, hot) {
var devClient = [require.resolve("webpack-dev-server/client/") + "?http://localhost:8080"];
if (hot) {
devClient.push("webpack/hot/dev-server");
}
[].concat(config).forEach(function (config) {
if (typeof config.entry === "object" && !Array.isArray(config.entry)) {
Object.keys(config.entry).forEach(function (key) {
config.entry[key] = devClient.concat(config.entry[key]);
});
} else {
config.entry = devClient.concat(config.entry);
}
});
}
You will need to pass the config in there before you pass it to webpack:
var webpackDevelopmentConfig = require('./Source/config/webpack.config.development.js');
addInline(webpackDevelopmentConfig)
var compiler = webpack(webpackDevelopmentConfig);

Inline mode is enabled by default - go to http://host:port/webpack-dev-server/