I'm working on a Node/Express project. Most of the code is contained in files that are inside folders, so that a folder named 'controllers' contains all of the project's controllers, a folder named 'services' contains all of the project's services etc. The code that starts the server is inside a file called app.js which is directly in the project root.
All controllers and services are declared as JS classes and contain a constructor. Dotenv is loaded inside app.js and env variables work nicely everywhere BUT inside class constructors, as those seem to be loaded before dotenv is initialized.
An example of the class syntax used:
export default class exampleService {
constructor() {
console.log('This is not working', process.env.EXAMPLE); // process.env.EXAMPLE is undefined
}
myFunction() {
console.log('This works just fine', process.env.EXAMPLE); // process.env.EXAMPLE is exactly how it's defined in .env
}
// All other class methods are here.
}
Is there a way I could get dotenv variables to work inside class constructors without having to import and init dotenv in the beginning of each class file?
If I do resort to importing dotenv individually to each class file, will it have any other cons besides making the code less clean?
Try loading the application with preload on dotenv (copied from the source):
Preload
You can use the --require (-r) command line option to preload dotenv. By doing this, you do not need to require and load dotenv in your application code. This is the preferred approach when using import instead of require.
$ node -r dotenv/config your_script.js
The configuration options below are supported as command line arguments in the format dotenv_config_<option>=value
$ node -r dotenv/config your_script.js dotenv_config_path=/custom/path/to/your/env/vars
Additionally, you can use environment variables to set configuration options. Command line arguments will precede these.
$ DOTENV_CONFIG_<OPTION>=value node -r dotenv/config your_script.js
$ DOTENV_CONFIG_ENCODING=latin1 node -r dotenv/config your_script.js dotenv
I assume PM2 appends env vars the 'native' system way at startup, something like:
MYVAR=hey; node app.js
The difference with the dotenv npm package is it MUST append vars another way, because it works inside the script (it can't do MYVAR=someothervar; node app.js because the program is already started), so it works like this:
dotenv.config() //reads .env file and appends stuff to process.env at runtime
Now say PM2 launches MYVAR=hey; node app.js and then inside app.js we run dotenv.config() that reads an .env file containing MYVAR=foo. Which var will be in process.env?
ecosystem.config.js
{
//...standard pm2 config above
env: {
MYVAR: 'ecosystem',
},
}
.env/dotenv
MYVAR=dotenv
Code
dotenv.config()
console.log(process.env.MYVAR)
dotenv.config() will not overwrite variables if it sees they already exist in the process.env (that they've been assigned the PM2 MYVAR=foo; node app.js way.
So process envs set before launch will take precedence.
This is actually in the README of dotenv.
What happens to environment variables that were already set?
We will never modify any environment variables that have already been set. In particular, if there is a variable in your .env file which collides with one that already exists in your environment, then that variable will be skipped. This behavior allows you to override all .env configurations with a machine-specific environment, although it is not recommended.
https://www.npmjs.com/package/dotenv#what-happens-to-environment-variables-that-were-already-set
If you absolutely need to override existing env vars - use the dotenv-override package.
I am new to Node js, started developing the Angular application using Angular 1.2 and Node js. As of now, I have hardcoded the REST API(Java) endpoints in the node services.js. Now I want to load the base endpoint URI specific to the environment. I have tried few ways by setting a new key value for the process.env, a env file and load it. Can anyone please help me.
I have tried below approach.
Created devEnv.env file under root folder.
Added 3 key-value pairs
hostname = xyz
apikey = 123
devUrl = xyz/xyz/xyz.com/
Then in terminal, I am trying to add it to the source.
$ source denEnv.env
I am getting source not found.
Another way I have added the script in package.json file
{
"start-dev": "source devEnv.env; node server.js"
}
In terminal I executed
$ npm start-dev
It's also failing. Can anyone please let me know what mistake I am doing and what is the correct approach.
Thanks in advance.
There are three methods known to me:
1) .env file
You need to install dotenv package using npm install/yarn add and on top of your main file (e.g. index.js) put require('dotenv').config(). That should load your variables to node.
2) passed on a start
If you want pass a small amount of environmental variables you can try something like this in your package.json:
{
"start-dev": "hostname=xyz apikey=123 devUrl=xyz/xyz/xyz.com node server.js"
}
Advice: environmental variables should look like HOSTNAME, API_KEY or DEV_URL.
3) system environmental variables
Solution: Set environment variables from file
Your variables are most likely not being exported to the shell. To be able to source your devEnv.env script, try to modify it as follows:
#!/bin/bash
export hostname=xyz
export apikey=123
export devUrl=xyz/xyz/xyz.com/
You most likely need to give it executable rights:
chmod +x devEnv.env
And then source it by running:
. devEnv.env
Another example can be found here: Set environment variables from file
I am using Webpack 2 and Electron to build nodejs application on Mac.
In my project in the root I have directory 'data' where I store configuration in a json like data/configurations/files.json (in practices there are different files with dynamic names)
After webpackaing though when I call: fs.readdirSync(remote.app.getAppPath()); to get files in the root I get only these packed: [ "default_app.js", "icon.png", "index.html", "main.js", "package.json", "renderer.js" ]
path.join(remote.app.getAppPath(), 'data/tests/groups.json'); called with FS ReadSync leads to an issue Error: ENOENT, data/tests/groups.json not found in /Users/myuser/myproject/node_modules/electron/dist/Electron.app/Contents/Resources/default_app.asar. So it seems that the whole data folder is not picked up by webpacker.
Webpack config is using json-loader and I did not find any documentation mentioning anything special about including specific files or jsons. Or do I have to reference json files in my code differently as they might be packed under main.js.
What is the best practice for Electron/Webpack for managing JSON config files? Am I doing something wrong when webpacking the project?
My project is based of https://github.com/SimulatedGREG/electron-vue using webpack/electron/vue
The Webpack Misconception
One thing to understand upfront is that webpack does not bundle files required through fs or other modules that ask for a path to a file. These type of assets are commonly labeled as Static Assets, as they are not bundled in any way. webpack will only bundle files that are required or imported (ES6). Furthermore, depending on your webpack configuration, your project root may not always match what is output within your production builds.
Based on the electron-vue documentation's Project Structure/File Tree, you will find that only webpack bundles and the static/ directory are made available in production builds. electron-vue also has a handy __static global variable that can provide a path to that static/ folder within both development and production. You can use this variable similar to how one would with __dirname and path.join to access your JSON files, or really any files.
A Solution to Static Assets
It seems the current version of the electron-vue boilerplate already has this solved for you, but I'm going to describe how this is setup with webpack as it can apply to not only JSON files and how it can also apply for any webpack + electron setup. The following solution assumes your webpack build outputs to a separate folder, which we'll use dist/ in this case, assumes your webpack configuration is located in your project's root directory, and assumes process.env.NODE_ENV is set to development during development.
The static/ directory
During development we need a place to store our static assets, so let's place them in a directory called static/. Here we can put files, such as JSONs, that we know we will need to read with fs or some other module that requires a full path to the file.
Now we need to make that static/ assets directory available in production builds.
But webpack isn't handling this folder at all, what can we do?
Let's use the simple copy-webpack-plugin. Within our webpack configuration file we can add this plugin when building for production and configure it to copy the static/ folder into our dist/ folder.
new CopyWebpackPlugin([
{
from: path.join(__dirname, '/static'),
to: path.join(__dirname, '/dist/static'),
ignore: ['.*']
}
])
Okay so the assets are in production, but how do I get a path to this folder in both development and production?
Creating a global __static variable
What's the point of making this __static variable?
Using __dirname is not reliable in webpack + electron setups. During development __dirname could be in reference to a directory that exists in your src/ files. In production, since webpack bundles our src/ files into one script, that path you formed to get to static/ doesn't exist anymore. Furthermore, those files you put inside src/ that were not required or imported never make it to your production build.
When handling the project structure differences from development and production, trying to get a path to static/ will be highly annoying during development having to always check your process.env.NODE_ENV.
So let's simplify this by creating one source of truth.
Using the webpack.DefinePlugin we can set our __static variable only in development to yield a path that points to <projectRoot>/static/. Depending if you have multiple webpack configurations, you can apply this for both a main and renderer process configuration.
new webpack.DefinePlugin({
'__static': `"${path.join(__dirname, '/static').replace(/\\/g, '\\\\')}"`
})
In production, we need to set the __static variable manually in our code. Here's what we can do...
index.html (renderer process)
<!-- Set `__static` path to static files in production -->
<script>
if (process.env.NODE_ENV !== 'development') window.__static = require('path').join(__dirname, '/static').replace(/\\/g, '\\\\')
</script>
<!-- import webpack bundle -->
main.js (main process)
// Set `__static` path to static files in production
if (process.env.NODE_ENV !== 'development') {
global.__static = require('path').join(__dirname, '/static').replace(/\\/g, '\\\\')
}
// rest of application code below
Now start using your __static variable
Let's say we have a simple JSON file we need to read with fs, here's what we can accomplish now...
static/someFile.json
{"foo":"bar"}
someScript.js (renderer or main process)
import fs from 'fs'
import path from 'path'
const someFile = fs.readFileSync(path.join(__static, '/someFile.json'), 'utf8')
console.log(JSON.parse(someFile))
// => { foo: bar }
Conclusion
webpack was made to bundle assets together that are required or imported into one nice bundle. Assets referenced with fs or other modules that need a file path are considered Static Assets, and webpack does not directly handle these. Using copy-webpack-plugin and webpack.DefinePlugin we can setup a reliable __static variable that yields a path to our static/ assets directory in both development and production.
To end, I personally haven't seen any other webpack + electron boilerplates handle this situation as it isn't a very common situation, but I think we can all agree that having one source of truth to a static assets directory is a wonderful approach to alleviate developer fatigue.
I think the confusion, (if there is any), might come from the fact that webpack not only "packs", embeds, things, code, etc... but also process content with its plugins.
html plugin being a good example, as it simply generates an html file at build-time.
And how this relates to the config file issue?,
well depending on how you are "requiring" the "config" file, what plug-in you are using to process that content.
You could be embedding it, or simply loading it as text, from file system or http, or else...
In the case of a config file, that I guess you want it to be parsed at runtime,
otherwise it's just fancy hardcoding values that perhaps you could be better simply typing it in your source code as simple objects.
And again in that case I think webpack adds little to nothing to the runtime needs, as there is nothing to pre-pack to read at later use,
so I would possibly instead or "require"it, i'll read it from the file system, with something like :
// read it parse it relative to appPath/cwd,
const config = JSON.parse(
fs.readfileSync(
path.join( app.getAppPath(), "config.json" ),
"utf-8"
))
//note: look fs-extra, it does all that minus the app.path plus async
and electron will read it from the file system , or if using Electron.require will read it from asar|fileSystem (in that order if I remember correctly, I could be wrong),
Webpack design philosophy is focused around very simple yet powerful concept:
Transform and bundle everything that is actually used by your app.
To achieve that webpack introduces a powerful concept of dependency graph, which is able to manage virtually any kind of dependencies (not only *.js modules) by the means of so-called loaders.
The purpose of a loader is to transform your dependency in a way that makes statement import smth from 'your_dependency' meaningful. For instance, json-loader calls JSON.parse(...) during loading of *.json file and returns configuration object. Therefore, in order to take advantage of webpack dependency resolution system for managing JSONs, start from installing json-loader:
$ npm install --save-dev json-loader
Then modify your webpack.config.js in the following way:
module.exports = {
...
module: {
rules: [
{test: /\.json$/, use: 'json-loader'}
]
}
...
};
At this point webpack should be able to resolve your JSON dependencies by their absolute paths, so the following should work (I assume here that you have a subdirectory config of your root context dir, containing file sample.json):
import sampleCfg from './config/sample.json';
But importing physical paths doesn't lead to elegant, robust and maintainable code (think of testability, for example), so it is considered a good practice to add aliases to your webpack.config.js for abstracting away your physical .config/ folder from your import statements
module.exports = {
...
resolve: {
alias: {
cfg: './config'
}
}
...
}
Then you'll be able to import your JSON config like that:
import sampleCfg from 'cfg/sample.json'
Finally, if you use SimulatedGREG/electron-vue Electron project template (as you mentioned in your post), then you have three webpack configuration files:
.electron-vue/webpack.web.config.js - use this config file if you use this template just for ordinary web development (i.e. not for building native Electron projects);
.electron-vue/webpack.main.config.js - use this file to configure webpack module that will run inside Electron's main process;
.electron-vue/webpack.renderer.config.js - use this file for Electron's renderer process.
You can find more information on main and renderer processes in the official Electron documentation.
I understood it was possible to natively use CoffeeScript instead of JavaScript, without compiling the CoffeeScript files to JavaScript (at least not compiled them as files; Maybe in-memory or on-the-fly compilation).
I have a basic app with a main app.coffee file as following:
coffee = require 'coffee-script'
coffee = require 'coffee-script/register'
express = require 'express'
compression = require 'compression'
sockets = require 'socket.io'
http = require 'http'
server = http.createServer express
app = express()
# [...]
In my package.json I have the following:
// ...
"dependencies": {
"coffee-script": "^1.9.3"
// ...
}
"scripts": {
"start": "coffee app.coffee --nodejs"
// ...
}
// ...
I can run the app using $> nodemon app.coffee
or $> coffee app.coffee
or $> npm start
At some point when I re-run the application or save a CoffeeScript file, CoffeeScript file is compiled to a JavaScript file, and therefor every file in the folder tree end up getting duplicated in both a .js and a .coffee version, which I find pretty disturbing.
Also once there is a .js file for a module, I sometime feel like the application will use it in priority, and that changes in the CoffeeScript file are no longer taken in account.
How can I avoid this behavior, and avoid file to be compiled every time ? Is it possible for the node engine to natively use the CoffeeScript files without creating a .js copy in the file tree?
Of course I understand that the Node.js engine is a JavaScript engine, but is there a way to maybe compile/run files directly in-memory or in a different folder (since it might still be interesting to see the JavaScript output)?
What is the exact interaction between the Node.js engine, the CoffeeScript compiler, and how can I understand this behavior ?
I often use a small .js file to launch my code making the command to start my applications node index.js which contains this:
CoffeeScript = require('coffee-script');
CoffeeScript.register();
require('./src/main');
Change ./src/main to the path to your coffee file.
It uses your coffee files to run your code giving stack traces etc.. that correspond to your coffee source.
Your app.coffee would not need to require coffee-script as you are already using coffee at that point.