Develop Reference

What's the proper way of setting environment variable in Netlify functions?

What's the proper way of setting environment variables in netlify? I would like to be able to set different values for the variables depending on the environment.
Pseudo code:
let host;
if (process.env.GATSBY_CUSTOM_CONTEXT === 'production') {
host = process.env.PRODUCTION_HOST
} else if (process.env.GATSBY_CUSTOM_CONTEXT === 'development') {
host = process.env.DEVELOPMENT_HOST
}
I have tried passing env variable thru CLI, like GATSBY_CUSTOM_CONTEXT=production gatsby build and I also tried using same command with cross-env.
My other attempt used netlify.toml:
[build]
base = "/"
publish = "public"
command = "yarn build"
functions = "src/functions"
[context.production]
[context.production.environment]
GATSBY_CUSTOM_CONTEXT = "production"
All of these options worked with netlify dev locally, but in production GATSBY_CUSTOM_CONTEXT is always undefined.

The reason you can't resolve the environment variables in your Netlify functions is because as of the time of your question, Netlify does not transfer the environment variables from the netlify.toml file.
You must put them into the admin panel in your site settings in the app.netlify.com dashboard.

Unfortunately, what you're looking to doesn't seem to be currently supported. Though they provide an alternative approach.
I found this snippet on their docs:
CALLING ENVIRONMENT VARIABLES
Using environment variables directly as
values ($VARIABLENAME) in your netlify.toml file is not supported.
However, the following workflow can be used to substitute values in
the file with environment variable values, assuming you are only
trying to change headers or redirects. The rest of the file is read
BEFORE your build — but those sections are read AFTER the build
process.
Add a placeholder like HEADER_PLACEHOLDER somewhere in the netlify.toml redirects or headers sections.
Create an environment variable, for example PROD_API_LOCATION, with the desired value. You can create environment variables in the
toml file or in our UI. You might use the latter to keep sensitive
values out of your repository.
Prepend a replacement command to your build command. Here’s an example for a site using yarn build to build: sed -i
s/HEADER_PLACEHOLDER/${PROD_API_LOCATION}/g netlify.toml && yarn build
Taken from here: https://www.netlify.com/docs/netlify-toml-reference/

How to define different vars for different environment in node.js?

I start to use dotenv for defining my env vars. I can easily specify my common vars for dev and prod environment in one .env file. But how to specify different vars for development and production environment.
For example I want to use DB_NAME = dev_db for development and DB_NAME = prod_db for production. How can I achieve this with dotenv?
I'm working with node.js.
Thank you in advance!

The .env file is supposed to be different on different environments, and hence should not be committed to your source control repo (for Git this would mean .gitignore).
For example, you'd have a config.env on production with this:
DB_NAME=prod_db
And on development, like this:
DB_NAME=dev_db
On both environments, the files would be named config.env and would reside in the same location so that the code will read the correct file.
In my project, the committed file is called config.env.sample that contains all the necessary values so that developers know what they need to add to the config.env in their environment:
# config.env.sample
# Sample configuration file for Project XYZ
# Name of the database
DB_NAME=dev_db
Here is more information from the FAQ:
Should I commit my .env file?
No. We strongly recommend against committing your .env file to version control. It should only include environment-specific values such as database passwords or API keys. Your production database should have a different password than your development database.
Should I have multiple .env files?
No. We strongly recommend against having a "main" .env file and an "environment" .env file like .env.test. Your config should vary between deploys, and you should not be sharing values between environments.

How to manage configuration for Webpack/Electron app?

I am using Webpack 2 and Electron to build nodejs application on Mac.
In my project in the root I have directory 'data' where I store configuration in a json like data/configurations/files.json (in practices there are different files with dynamic names)
After webpackaing though when I call: fs.readdirSync(remote.app.getAppPath()); to get files in the root I get only these packed: [ "default_app.js", "icon.png", "index.html", "main.js", "package.json", "renderer.js" ]
path.join(remote.app.getAppPath(), 'data/tests/groups.json'); called with FS ReadSync leads to an issue Error: ENOENT, data/tests/groups.json not found in /Users/myuser/myproject/node_modules/electron/dist/Electron.‌​app/Contents/Resourc‌​es/default_app.asar. So it seems that the whole data folder is not picked up by webpacker.
Webpack config is using json-loader and I did not find any documentation mentioning anything special about including specific files or jsons. Or do I have to reference json files in my code differently as they might be packed under main.js.
What is the best practice for Electron/Webpack for managing JSON config files? Am I doing something wrong when webpacking the project?
My project is based of https://github.com/SimulatedGREG/electron-vue using webpack/electron/vue

The Webpack Misconception
One thing to understand upfront is that webpack does not bundle files required through fs or other modules that ask for a path to a file. These type of assets are commonly labeled as Static Assets, as they are not bundled in any way. webpack will only bundle files that are required or imported (ES6). Furthermore, depending on your webpack configuration, your project root may not always match what is output within your production builds.
Based on the electron-vue documentation's Project Structure/File Tree, you will find that only webpack bundles and the static/ directory are made available in production builds. electron-vue also has a handy __static global variable that can provide a path to that static/ folder within both development and production. You can use this variable similar to how one would with __dirname and path.join to access your JSON files, or really any files.
A Solution to Static Assets
It seems the current version of the electron-vue boilerplate already has this solved for you, but I'm going to describe how this is setup with webpack as it can apply to not only JSON files and how it can also apply for any webpack + electron setup. The following solution assumes your webpack build outputs to a separate folder, which we'll use dist/ in this case, assumes your webpack configuration is located in your project's root directory, and assumes process.env.NODE_ENV is set to development during development.
The static/ directory
During development we need a place to store our static assets, so let's place them in a directory called static/. Here we can put files, such as JSONs, that we know we will need to read with fs or some other module that requires a full path to the file.
Now we need to make that static/ assets directory available in production builds.
But webpack isn't handling this folder at all, what can we do?
Let's use the simple copy-webpack-plugin. Within our webpack configuration file we can add this plugin when building for production and configure it to copy the static/ folder into our dist/ folder.
new CopyWebpackPlugin([
{
from: path.join(__dirname, '/static'),
to: path.join(__dirname, '/dist/static'),
ignore: ['.*']
}
])
Okay so the assets are in production, but how do I get a path to this folder in both development and production?
Creating a global __static variable
What's the point of making this __static variable?
Using __dirname is not reliable in webpack + electron setups. During development __dirname could be in reference to a directory that exists in your src/ files. In production, since webpack bundles our src/ files into one script, that path you formed to get to static/ doesn't exist anymore. Furthermore, those files you put inside src/ that were not required or imported never make it to your production build.
When handling the project structure differences from development and production, trying to get a path to static/ will be highly annoying during development having to always check your process.env.NODE_ENV.
So let's simplify this by creating one source of truth.
Using the webpack.DefinePlugin we can set our __static variable only in development to yield a path that points to <projectRoot>/static/. Depending if you have multiple webpack configurations, you can apply this for both a main and renderer process configuration.
new webpack.DefinePlugin({
'__static': `"${path.join(__dirname, '/static').replace(/\\/g, '\\\\')}"`
})
In production, we need to set the __static variable manually in our code. Here's what we can do...
index.html (renderer process)
<!-- Set `__static` path to static files in production -->
<script>
if (process.env.NODE_ENV !== 'development') window.__static = require('path').join(__dirname, '/static').replace(/\\/g, '\\\\')
</script>
<!-- import webpack bundle -->
main.js (main process)
// Set `__static` path to static files in production
if (process.env.NODE_ENV !== 'development') {
global.__static = require('path').join(__dirname, '/static').replace(/\\/g, '\\\\')
}
// rest of application code below
Now start using your __static variable
Let's say we have a simple JSON file we need to read with fs, here's what we can accomplish now...
static/someFile.json
{"foo":"bar"}
someScript.js (renderer or main process)
import fs from 'fs'
import path from 'path'
const someFile = fs.readFileSync(path.join(__static, '/someFile.json'), 'utf8')
console.log(JSON.parse(someFile))
// => { foo: bar }
Conclusion
webpack was made to bundle assets together that are required or imported into one nice bundle. Assets referenced with fs or other modules that need a file path are considered Static Assets, and webpack does not directly handle these. Using copy-webpack-plugin and webpack.DefinePlugin we can setup a reliable __static variable that yields a path to our static/ assets directory in both development and production.
To end, I personally haven't seen any other webpack + electron boilerplates handle this situation as it isn't a very common situation, but I think we can all agree that having one source of truth to a static assets directory is a wonderful approach to alleviate developer fatigue.

I think the confusion, (if there is any), might come from the fact that webpack not only "packs", embeds, things, code, etc... but also process content with its plugins.
html plugin being a good example, as it simply generates an html file at build-time.
And how this relates to the config file issue?,
well depending on how you are "requiring" the "config" file, what plug-in you are using to process that content.
You could be embedding it, or simply loading it as text, from file system or http, or else...
In the case of a config file, that I guess you want it to be parsed at runtime,
otherwise it's just fancy hardcoding values that perhaps you could be better simply typing it in your source code as simple objects.
And again in that case I think webpack adds little to nothing to the runtime needs, as there is nothing to pre-pack to read at later use,
so I would possibly instead or "require"it, i'll read it from the file system, with something like :
// read it parse it relative to appPath/cwd,
const config = JSON.parse(
fs.readfileSync(
path.join( app.getAppPath(), "config.json" ),
"utf-8"
))
//note: look fs-extra, it does all that minus the app.path plus async
and electron will read it from the file system , or if using Electron.require will read it from asar|fileSystem (in that order if I remember correctly, I could be wrong),

Webpack design philosophy is focused around very simple yet powerful concept:
Transform and bundle everything that is actually used by your app.
To achieve that webpack introduces a powerful concept of dependency graph, which is able to manage virtually any kind of dependencies (not only *.js modules) by the means of so-called loaders.
The purpose of a loader is to transform your dependency in a way that makes statement import smth from 'your_dependency' meaningful. For instance, json-loader calls JSON.parse(...) during loading of *.json file and returns configuration object. Therefore, in order to take advantage of webpack dependency resolution system for managing JSONs, start from installing json-loader:
$ npm install --save-dev json-loader
Then modify your webpack.config.js in the following way:
module.exports = {
...
module: {
rules: [
{test: /\.json$/, use: 'json-loader'}
]
}
...
};
At this point webpack should be able to resolve your JSON dependencies by their absolute paths, so the following should work (I assume here that you have a subdirectory config of your root context dir, containing file sample.json):
import sampleCfg from './config/sample.json';
But importing physical paths doesn't lead to elegant, robust and maintainable code (think of testability, for example), so it is considered a good practice to add aliases to your webpack.config.js for abstracting away your physical .config/ folder from your import statements
module.exports = {
...
resolve: {
alias: {
cfg: './config'
}
}
...
}
Then you'll be able to import your JSON config like that:
import sampleCfg from 'cfg/sample.json'
Finally, if you use SimulatedGREG/electron-vue Electron project template (as you mentioned in your post), then you have three webpack configuration files:
.electron-vue/webpack.web.config.js - use this config file if you use this template just for ordinary web development (i.e. not for building native Electron projects);
.electron-vue/webpack.main.config.js - use this file to configure webpack module that will run inside Electron's main process;
.electron-vue/webpack.renderer.config.js - use this file for Electron's renderer process.
You can find more information on main and renderer processes in the official Electron documentation.

Managing JavaScript config files for different build environments (like QA, UAT, Production)

So you are building a stand alone Single Page Application in JavaScript that is served statically. It connects to an API.
Your application has a JSON object in its main.js like this:
var config = {
apiPath: 'http://dev.api.example.com',
appTitle: 'hello'
};
You use Grunt to create a release build, this transpiles all your SASS, ES6, AMD Modules etc code into one or two JavaScript files, along with this config object.
Your production API url is http://api.example.com and when you do a release build, you want just this value in the config to change. (Maybe you would want to change other config parameters too). You have a release config
var config = {
apiPath: 'http://api.example.com'
};
How can a JavaScript config file be altered depending on different types of release environments during a build. Notice that I want to merge the release config into the default, not maintain totally seperate files with all the settings. While there are grunt plugins for setting a build mode, I've not found any that have the ability to update config files, only hacks that change which file is pointed to.
ASP.NET has web.config transforms that do exactly this, any property in the release transform for example, will overwrite the 'default' config on packaging. Does such a workflow exist using grunt or something?

In the end I solved the question myself by using a combination of environment variables and an amazing Node called Confidence which where you place all your different envs config settings in a single file, and they are resolved to a single configuration based on filters (such as an environment variable) - these can then be injected into the build process using tools like WebPack.

Node.js: How to setup different variables for prod and staging

I'm using Express and I need to use different credentials for each server (staging and production).
I could setup the variables in the server.coffee file but then I'd need to access those variables in different files.
server.coffee:
app.configure 'production', () ->
app.use express.errorHandler()
What's the solution? Setup the variables and then export them?

./config.js
var development = {
appAddress : '127.0.0.1:3000',
socketPort : 4000,
socketHost : '127.0.0.1',
env : global.process.env.NODE_ENV || 'development'
};
var production = {
appAddress : 'someDomain.com',
socketPort : 4000,
socketHost : '8.8.8.8',
env : global.process.env.NODE_ENV || 'production'
};
exports.Config = global.process.env.NODE_ENV === 'production' ? production : development;
./app.js
var cfg = require('./config').Config;
if (cfg.something) { // switch on environment
// your logic
}

This might be a good place to use npm-config.
When running scripts (see npm-scripts(7)) the package.json "config" keys are overwritten in the environment if there is a config param of <name>[#<version>]:<key>
I would not use them for every type of variable configuration setting, but I think it's a good solution for simple cases like URLs and ports because:
You put them directly into package.json.
In addition, you can specify them on the command line or as ENV variables.
Anything run through npm can refer to them (e.g., scripts).
You can set them per-user with `npm config set foo:port 80
The one caveat is that the config parameter in your package.json is only automatically exported into the environment when you run your code through npm. So, if you just run it with node, like, node ./myapp.js, then you can't expect that process.env.npm_package_config_foo will contain your value. However, you can always var pkg = require('./package.json'); and access the values at pkg.config.
Because it might not be immediately obvious, I'd also add that the npm_package_config environment variables do not bubble up to apps that depend on your npm package. So, if your depended-on package refers to process.env.npm_package_config_foo, then the dependent package would have to define that in its own package.json. I guess since it's an "npm_package_config" it wouldn't make sense to push them all the way up the tree.
So, how would I use one npm config variable and have it work all the way up the tree, in both the base package and the packages that depend on it? It's actually a little confusing, and I had to figure this out through trial and error.
Let's say you have a package connector and package client. client depends on connector and you want to specify a config parameter for connector that can be used or overwritten in client. If you use process.env.npm_package_config.port in your connector module, then when it's depended on in client module, then that variable won't be exported and it will end up as undefined.
However, if you instead use process.env.npm_config_connector_port (notice the first one starts with npm_package_config and the other with npm_config_packagename), then you can at least set that in your .npmrc using npm config set connector:port 80 and it will be "namespaced" as process.env.npm__config_connector_port everywhere that you run npm, including scripts that you run in client that depend on connector, and you'll always be able to overwrite it on the command line, in your ENV, or in your .npmrc. You just have to keep in mind that, as with any environment variable, it may not always be set. So, I would use the default operator with the process.env.npm_config_connector_port as the first (preferred) value:
var port = process.env.npm_config_connector_port || sane_default
Here, sane_default could be populated from one of the other recommended methods. Personally, I like keeping configuration data like these in JSON files at the very least, and package.json seems like the best JSON file to put them in. Store them in data instead of code and then you can easily use the static JSON in-line, generate them dynamically, or pull them from the filesystem, URLs or databases.

I have uploaded an implementation into https://github.com/qiangyu/nodejsconfig. I believe it will satisfy your needs. Basically, you only need to provide one configuration file:
dev.appAddress = '127.0.0.1:3000';
prod.appAddress = 'someDomain.com';
Then, you use following code to read the appAddress in prod environments:
var xnconfig = require('nodejsconfig');
var fs = require('fs');
var data = fs.readFileSync(__dirname+"/config.properties", "UTF8");
// assume we will be using environment "prod"
var config = xnconfig.parse("prod", data);
// the output will be someDomain.com
console.log(config.getConfig("appAddress"));

If you don't want the logic for determining which config to use in each file (which would look pretty ugly), you'll have to export it somewhere.
What I would suggest: Have a config.json file containing the different configs. The main file requires it and does something like config.default = config.(condition ? 'production':'development'). In all other files, you can now just do require('./config').default.

I have an app which uses three different methods of declaring config variables (uris, api keys, credentials, etc.) depending upon the environment (production = environment variables; staging = command line args; local = config files.)
I wrote a little "config" module to handle merging all of these options into one object that I can use in my app and uploaded it as a gist: https://gist.github.com/1616583
It might not be the best implementation, but it's been working pretty well so far :).