I have experimented with adding environment variables to my Gatsby project using .env.development and .env.production files and it's working great.
I would like to have my builds fail if one of the environment variables is missing, however I can't seem to see how to enable this functionality.
I have read through the Gatsby environment variables documentation, but can't seem to see how this would work? is this possible?
I believe it uses dotenv/webpack define plugin under the hood.
I’m sure there are other ways to do this, but with some quick tests, this approach seems to be working well for me.
In your gatsby-config.js file, you can choose to explicitly require the dotenv, so you can use those environment variables in your config.
I added the following, and now the Gatsby build will fail unless the specified environment variables are present.
// Load the environment variables, per
// https://www.gatsbyjs.org/docs/environment-variables/#server-side-nodejs
require('dotenv').config({
path: `.env.${process.env.NODE_ENV}`,
})
function checkEnv(envName) {
if (typeof process.env[envName] === 'undefined' || process.env[envName] === '') {
throw `Missing required environment variables: ${envName}`
}
}
try {
checkEnv('NODE_ENV')
checkEnv('EXAMPLE_MISSING_ENV')
checkEnv('EXAMPLE_API_KEY')
} catch (e) {
throw new Error(e)
}
// The rest of the config file
I could imagine customizing this further, ex. logging a warning for a variable with a fallback versus throwing an error for one that is required by your content sourcing plugin or theme. Hope this is helpful as a starting point!
I couldn't find built-in solution for this on Gatsby neither. You may do it manually, but still not too easy.
First problem: If you wanna load your environment from file while running npm script; it can not be loaded right away. But you may trigger a script file, and it can load this environment variables before your check.
lets say build.sh on root directory of project :
source ./.env.development # this line will set env variables
if [ "$API_KEY" = 927349872349798 ] ; then
npm run build
fi
Another problem rises; some developers might want to run it on windows maybe. So better use famous cross-env package.
npm i cross-env
Then everything is ready, add your secure-build :
"scripts": {
"build": "gatsby build",
"develop": "gatsby develop",
"format": "prettier --write \"**/*.{js,jsx,json,md}\"",
"start": "npm run develop",
"serve": "gatsby serve",
"clean": "gatsby clean",
"test": "echo \"Write tests! -> https://gatsby.dev/unit-testing\" && exit 1",
"secure-build": "cross-env-shell \"./build.sh\""
},
And run it :
npm run secure-build
This solution looks too much for me as we created a build.sh and install a new package. Maybe there is cleaner solution. I am not Gatsby Guru after all.
I added env checking to the onPreInit life cycle hook in gatsby-node.ts:
const envVariablesList = [
"ENV1",
"ENV2",
"ENV3",
];
function envVarChecker(vars: string[]): string | undefined {
return vars.find(
(item) => process.env[item] === undefined || process.env[item] === ""
);
}
export const onPreInit: GatsbyNode["onPreInit"] = ({ actions }) => {
const emptyEnv = envVarChecker(envVariablesList);
if (emptyEnv !== undefined) {
throw new Error(`Env variable: ${emptyEnv} is empty!`);
}
};
It fails build almost at the very beginning (during pre-bootstrap phase) if any of the declared variables is missing
Related
I'm new the world of automating/testing/bunding with JS and I've got parcel setup for the most part but I noticed that when it builds files, it does not actually save them with the .min.js part in the file name. I'm wondering if theres a way to do this without having to rename the build file manually.
I'm also trying to find a way to have parcel go through the original source files(the ones that you work on) and lint and beautify them for me
Here's what my package.json looks like
{
"name": "lpac",
"version": "1.3.1",
"description": "",
"dependencies": {},
"devDependencies": {
"parcel": "^2.0.0-rc.0"
},
"scripts": {
"watch": "parcel watch --no-hmr",
"build": "parcel build"
},
"targets": {
"lite-maps": {
"source": ["./path/file1.js", "./path/file2.js", "./path/file3.js"],
"distDir": "./path/build/"
}
},
"browserslist": "> 0.5%, last 2 versions, not dead",
"outputFormat" : "global",
}
I checked out the docs but I couldn't find anything on linting or beautifying with parcel. How can i go about doing that? If you have tutorial links to doing so please also share because resources/tutorials seem scarce for anything other than the basic watching and building files
Unfortunately, there is no out-of-the-box setting that can cause parcel javascript output look like [fileName].[hash].min.js instead of [fileName].[hash].js. The .min.js extension is just a convention to keep output files distinct from source files, though - it has no effect at runtime - and the fact that parcel does automatic content hashing makes it easy enough to tell this. And even though they don't have a .min.js extension, these output files are definitely still minified and optimized by default.
However, if you really, really want this anyways, it's relatively simple to write a Namer plugin for parcel that adds .min.js to all javascript output:
Here's the code:
import { Namer } from "#parcel/plugin";
import path from "path";
export default new Namer({
name({ bundle }) {
if (bundle.type === "js") {
const filePath = bundle.getMainEntry()?.filePath;
if (filePath) {
let baseNameWithoutExtension = path.basename(filePath, path.extname(filePath));
// See: https://parceljs.org/plugin-system/namer/#content-hashing
if (!bundle.needsStableName) {
baseNameWithoutExtension += "." + bundle.hashReference;
}
return `${baseNameWithoutExtension}.min.js`;
}
}
// Returning null means parcel will keep the name of non-js bundles the same.
return null;
},
});
Then, supposing the above code was published in a package called parcel-namer-js-min, you would add it to your parcel pipeline with this .parcelrc:
{
"extends": "#parcel/config-default",
"namers": ["parcel-namer-js-min", "..."]
}
Here is an example repo where this is working.
The answer to your second question (is there "a way to have parcel go through the original source files(the ones that you work on) and lint and beautify them for me") is unfortunately, no.
However, parcel can work well side-by-side with other command line tools that do this do this. For example, I have most of my projects set up with a format command in the package.json, that looks like this:
{
...
"scripts": {
...
"format": "prettier --write src/**/* -u --no-error-on-unmatched-pattern"
}
...
{
You can easily make that command automatically run for git commits and pushes with husky.
I want to create RSS script using Next.js.
So I put up a script in a subfolder inside the root folder scripts/ & named it build-rss.js
next.config.js
module.exports = {
webpack: (config, options) => {
config.module.rules.push({
test: /\.svg$/,
issuer: { and: [/\.(js|ts|md)x?$/] },
use: [
{
loader: '#svgr/webpack',
options: {
prettier: false,
svgo: true,
svgoConfig: { plugins: [{ removeViewBox: false }] },
titleProp: true,
},
},
],
})
if (!options.dev && options.isServer) {
const originalEntry = config.entry
config.entry = async () => {
const entries = { ...(await originalEntry()) }
entries['./scripts/build-rss'] = './scripts/build-rss.js'
return entries
}
}
if (!options.isServer) {
config.resolve.fallback.fs = false
}
return config
},
}
When I try to run my script npm run build:development which in package.json represents:
"scripts": {
"clean": "rimraf .next",
"dev": "next dev",
"export": "next export",
"start": "next start",
"lint": "next lint",
"build:development": "next build && npm run export && npm run rss:development",
"build": "next build && npm run export && npm run rss",
"rss:development": "node ./.next/server/scripts/build-rss.js",
"rss": "node ./.next/serverless/scripts/build-rss.js"
}
It throws an error saying:
Error: Cannot find module '../../webpack-runtime.js'
But I checked. The file does exist.
The blunder is this used to work earlier. Probably few versions ago when my other project used the same combination.
I have made a complete reproduction showcasing the error → https://github.com/deadcoder0904/next-script-rss-error
Just clone it, install it & try the script npm run build:development in the terminal to see the error.
Based on our conversation:
entry: path.join(__dirname, '..', 'path/to/file')
That's what a webpack entry looks like. It can also be an array or an object:
entry: [
path.join(__dirname, '..', 'path/to/file'),
// other entries here
]
Whereas you're already getting the whole webpack config:
webpack: (config, options)
So doing:
const originalEntry = config.entry
config.entry = async () => {
const entries = { ...(await originalEntry()) }
entries['./scripts/build-rss'] = './scripts/build-rss.js'
return entries
}
Makes no sense to me if you can just do:
config.entry.push('./scripts/build-rss')
// config.entry['./scripts/build-rss'] = './scripts/build-rss.js'
Unless I miss something with how nextjs is loading the webpack config.
Even then I'd suggest that you use path.join in order to ensure it's loaded to the correct location, because that relative root will execute from wherever webpack is compiled from.
Along with that in your first project you used nextjs v10 and now you're using nextjs v11, which has an upgrade from webpack 4 to 5, which is a major upgrade. I don't know the details, I can only speculate, but under no conditions should you assume that "because your previous project was working this one should using the same stuff", it won't necessarily (especially not in this case).
The first intuitive thing I thought was that webpack should by default bundle everything to a single output file, unless the configuration for that was changed by nextjs (I don't know). So using a script you added to entries didn't make sense to me, because it wouldn't exist. But you're saying that it does exist so I can only assume that webpack is configured to do code splitting and outputs each entry to a different file. In which case I have no idea. As far as I'm aware in webpack 5 (I don't know about webpack 4) code splitting is disabled in dev and enabled in production so your problem is likely a discrepancy between dev and production.
Perhaps the last thing you can try is to change your !options.dev, because right now you're only adding that script when it's production but you're trying to run the script using development.
If you really just have to get it working you can downgrade your nextjs to the previous version you were using (v10), even though that's not really a solution.
Other than that I'm out of ideas.
Not sure if you are still looking for an answer, but simply changing the webpack entry as follows seems to have fixed the problem.
entries['scripts/build-rss'] = './scripts/build-rss.js';
// instead of entries['./scripts/build-rss']
I had the same Error! I deleted the .next Folder and did an npm run dev, It started to work for me!
Is it possible to call out to retrieve a key from yargs when using as a npm script argument?
User types in the OSX terminal:
npm run scaffold --name=blah
which executes in package.json:
"scaffold" : "node ./scaffold/index.js -- "
This results in
const yargs = require('yargs').argv
if (yargs) {
console.log(yargs);
console.log(yargs.name);
process.exit(1)
}
...
result:
{ _: [], '$0': 'scaffold/index.js' }
undefined
This only works if I hard code in package.json "scaffold" : "node scaffold/index.js --name=blah", but I need this to be configurable.
As I stated I am using args, as it appears to make it easy to retrieve keys by name ( as opposed to an array ). Open to suggestions.
What am I missing?
update 11-07-2017
Related: Sending command line arguments to npm script
However, passing in the commandline 1: npm run scaffold name=hello
OR 2: npm run scaffold --name=hello yields:
1: { _: [], '$0': 'scaffold/index.js' }
2: { _: [ 'name=hello' ], '$0': 'scaffold/index.js' }
Still can't see a way to retrieve the yargs.name property. Still undefined.
Update 13-07-2017
For the time being, I have given up. It just seem impossible. I run the script manually in the terminal.
E.g.
node ./scaffold/index.js --name=blah
Image below shows executing of a node script directly as opposed to running through npm scripts. I have added https://www.npmjs.com/package/nopt node module to see if it helps ( it doesn't ). process.argv.name is still undefined when running through npm scripts.
Update 18-07-2017
Added github example: https://github.com/sidouglas/stackoverflow-node-arguments
Update 24-07-2017
Adding the variables before the start of the command works
myvar="hello npm run scaffold as opposed to npm run scaffold myvar="hello world"
As of npm#2.0.0, you can use custom arguments when executing scripts. The special option -- is used by getopt to delimit the end of the options. npm will pass all the arguments after the -- directly to your script:
npm run test -- --grep="pattern"
https://docs.npmjs.com/cli/run-script
I'm not sure that it matters where the variables are added on the command line, and if this is of no concern to you, then this works:
//package.json
{
"name": "npm-test",
"version": "1.0.0",
"description": "",
"main": "index.js",
"dependencies": {},
"devDependencies": {},
"scripts": {
"start": "node index.js"
},
"author": "",
"license": "ISC"
}
Your JS file:
//index.js
console.log('myvar', process.env.myvar);
And your command line command:
myvar="hello world" npm run start
So in the end, just prefix your npm script command with your argument list.
For me the following works on Node 10, 12, 14
npm run yourscript -- -- --name=bla
I do need to use -- --
and
"yourscript": "node bla.js"
I'm creating a Node.JS project based on Express and I'm trying to setup environment variables to separate the develop from production flux. I created a shell script file with some env variables like NODE_ENV or DEBUG to be executed by the predevelop script when I run the npm run develop on command line, but the variables are being lost after predevelop finish his execution and getting undefined when the Express app runs.
This is the ./scripts/development.sh
#!/bin/sh
echo 'Setting up development env';
export NODE_ENV='development';
export NODE_DEBUG='http';
export DEBUG='project:server';
echo 'NODE_ENV => '$NODE_ENV;
echo 'NODE_DEBUG => '$NODE_DEBUG;
echo 'DEBUG => '$DEBUG;
exit 0;
This is my package.json scripts
{
// ...
"scripts": {
"predevelop": "/bin/bash ./scripts/development.sh",
"develop": "echo 'NODE_ENV => '$NODE_ENV && echo 'NODE_DEBUG => '$NODE_DEBUG && echo 'DEBUG => '$DEBUG; node ./bin/www"
}
// ...
}
The console output from npm run develop:
┌─ [ jcorradi ] ~/Sites/pocs/project-server
└─• npm run develop
> project-server#0.0.0 predevelop /home/jcorradi/Sites/pocs/project-server
> /bin/bash ./scripts/development.sh
Setting up development env
NODE_ENV => development
NODE_DEBUG => http
DEBUG => project:server
> project-server#0.0.0 develop /home/jcorradi/Sites/pocs/project-server
> echo 'NODE_ENV => '$NODE_ENV && echo 'NODE_DEBUG => '$NODE_DEBUG && echo 'DEBUG => '$DEBUG; node ./bin/www
NODE_ENV =>
NODE_DEBUG =>
DEBUG =>
// From ./bin/www
NODE_ENV => undefined
NODE_DEBUG => undefined
DEBUG => undefined
Someone knows if this is the best pratice or if there are other ways to make this work?
Obs: I have already tried set the env var directly on npm script and this still keeps failing.
Ex:
{
// ...
"scripts": {
"predevelop": "NODE_ENV='development'; NODE_DEBUG='http'; DEBUG='project:server';",
"develop": "echo 'NODE_ENV => '$NODE_ENV && echo 'NODE_DEBUG => '$NODE_DEBUG && echo 'DEBUG => '$DEBUG; node ./bin/www"
}
// ...
}
Someone knows if this is the best pratice or if there are other ways to make this work?
The best practice is to avoid setting environment variables in package.json, as that should environment agnostic, in order to support environments that don't have bash, or have bash installed in /usr/bin/bash
Assuming you care not about that constraint, the way to make this work is to execute your environment in the actual run command:
{
"scripts": {
"develop": "NODE_ENV=development NODE_DEBUG=http DEBUG=project:server node ./bin/www"
}
}
If you don't want to just pass the params as mentioned above you can always do something like this.
package.json
"scripts": {
"start": ". development.sh | node path/to/app/www"
}
Within project root run "npm start" (also assuming development.sh is in your project root). This will execute the development.sh bash script and then your node script.
Two Notes:
I remember reading the syntax above might be slightly dependent on your OS/version of node. So if you run into any issues take that into consideration.
Also, within your node script, environmental vars will be key/values on the process object. This will be helpful to know for debugging purposes.
For Windows, my Node scripts should look like this:
"scripts": {
"start-docs": "SET NODE_ENV=development&&babel-node ./docs/Server.js"
}
But on Linux there's no SET, so it would be like this:
"scripts": {
"start-docs": "NODE_ENV=development&&babel-node ./docs/Server.js"
}
Is there a way to declare environment variables in a way that is consistent and cross-platform?
I recently came across the cross-env project. It's pretty straight-forward
{
"scripts": {
"build": "cross-env NODE_ENV=production webpack --config build/webpack.config.js"
}
}
That will set the build environment variable to production regardless of the OS.
I would vote against setting this in package.json because environment variables should be set dependent on your environment while package.json is most likely the same for every environment (you commit it to your version control system, right?). Instead you should use something like dotenv if you are looking for a clean and generic solution.
If you don't wanna use any third-party tools there is an easy way to achieve this using a nodejs script.
Step-by-step guide:
Create ./scripts/set-env.js file with the following content:
import { argv, env } from "node:process"
import { spawn } from "node:child_process"
// Set your custom env variables here
const extenv = {
NODE_ENV: "development",
}
spawn(argv[2], argv.slice(3), {
env: { ...env, ...extenv },
stdio: "inherit",
})
Notes:
argv[2] is a passed-in command to run in a child shell
argv.slice(3) - passed-in arguments
...env - used to preserve current shell environment variables
stdio: "inherit" - used to pipe the child shell output into the current shell
Edit your package.json's scripts section:
"scripts": {
"set-env": "node ./scripts/set-env.js",
"start-docs": "npm run set-env -- babel-node ./docs/Server.js"
}
Notes: -- is used to pass command and arguments to the set-env script.
Advantages
Cross-platform
No third-party dependencies
Fully customisable