Develop Reference

Dynamic require() in pkg (node.js)

IMPORTANT: If you want to help me or you have the same issue discuss it here: https://github.com/vercel/pkg/discussions/1580
I'm writing a node.js app which downloads repositories from GitHub at runtime. These contain javascript files which I require at runtime. But now I want to package my code into an executable using pkg, but I read that I can't just simply require packages at runtime using pkg. So my question is: How can I require these files at runtime? I'ts important for me to access its exported function, as I need to pass variables into it. Here is an example dynamically downloaded file:
module.exports = function PluginServerSideRenderer (app, dir, config) {
/*
*
* app is the express app
* dir is the current-working-directory of the server or mostly the root
* config is the config.json file as a javascript object
*
*/
app.get('/__plugin-template__', (req, res) => {
// Uncomment if user must be logged in to access this page
// if (req.session.loggedin !== true) return res.redirect('/login') // Login Check
res.send('Hello World from the Plugin-Template!')
})
}
Normally I would use this code to import these files:
// Plugin Server Side Renderer
const plugindirx = path.join(dir, '.tvt', 'plugins')
const files = fs.readdirSync(plugindirx)
const activeplugins = JSON.parse(fs.readFileSync(path.join(plugindirx, 'active.json'), 'utf8').toString())
for (const plugindiroutoffiles of files) {
if (plugindiroutoffiles === 'active.json') continue
if (!(activeplugins[genFileName(plugindiroutoffiles)])) continue
const filesx = fs.readdirSync(path.join(plugindirx, plugindiroutoffiles))
const ssrfile = path.join(plugindirx, plugindiroutoffiles, filesx[0], 'serverside.js')
require(ssrfile)(app, dir, config, Logger, IServConfig)
}
// End Plugin Server Side Renderer

Deploying Vue App Locally and Out of Sandbox

Is there any way to deploy a VueJS app locally without having it limited by the browser?
I want the user to be able to call the application from the command line, for example app_name --port 6000, and then be able to load up the app from http://127.0.0.1:6000 or any other port they selected. The important thing is, the Vue App must be able to import and use native NodeJS modules, such as fs and zeromq.

Vuejs code must run in a browser environment in which you don't have access to your hardware. So basically you have three options to achieve this:
1. Use Electron
Electron gives you a chromium instance in which your UI (Vuejs) code can run and a separate process in which you can have access to your hardware using nodejs.
2. Create a simple http server with node
In this case you need to create a server with nodejs. For example you can create a file named server.js like this:
const http = require('http');
const fs = require('fs')
http.createServer((req, res) => {
// Do something with fs
const data = fs.readFileSync('path/to/some/file');
// return back the data with http
http.writeHead(200, {ContentType: 'text/plain'})
http.write(data);
}).listen(8000)
And fetch the data in your Vuejs app with http like this:
export default {
data () {
return {
myData: null
}
},
methods: {
onSomeAction () {
const {data} = await axios.get('localhost:8000')
this.myData = data;
}
}
}
Now in your package.json you can put this to run both when npm serve is called:
...
"scripts": {
...
"serve": "node server.js & vue-cli-service serve"
}
3. Use Nuxt
In case you need to do something with your hardware ONCE, and BEFORE the render you can use Nuxt to access the hardware before rendering the Vuejs app and handing it over to your browser.

How to use electron-dl

I am trying to get the package electron-dl working. For that, i use the standard electron-quick-start example project.
Unfortunately, applying the example code from electron-dl to the electon-quick-start example does not work for me, meaning nothing is happening and no errors in the browser-console or the terminal.
This is how i implemented the minimal example:
main.js
const {app, BrowserWindow, ipcMain} = require('electron')
const {download} = require('electron-dl');
let mainWindow
ipcMain.on('download-item', async (event, {url}) => {
event.sender.send('download-success', url)
console.log(url)
const win = BrowserWindow.getFocusedWindow();
console.log(await download(win, url));
});
// ...
renderer.js
const { ipcRenderer } = require('electron')
$('#btn-dl').click(() => {
const newURL = "http://ipv4.download.thinkbroadband.com/5MB.zip"
ipcRenderer.send('download-item', {url: newURL})
})
ipcRenderer.on('download-success', (event, arg) => {
console.log(arg)
})
index.html
<h1>Hello World!</h1>
<button id="btn-dl">Download</button>
<script>
// jQuery
$ = require('jquery')
// You can also require other files to run in this process
require('./renderer.js')
</script>
In this implementation i am simply trying to download the file 5MB.zip when pressing the Download button.
What am i doing wrong?
Could someone please provide a simple working example of the implenentation of electron-dl using the electron-quick-start example?
Thanks for your help!

Having recreated exactly what you described, it works flawlessly for me.
The following is output on the console:
http://ipv4.download.thinkbroadband.com/5MB.zip
DownloadItem {
_events: { updated: [Function], done: [Function] },
_eventsCount: 2 }
The package determines the location to store the file itself, if you don't specify it. You can output the path it chooses by default using app.getPath('downloads'). For me, this is my home directory (Linux).
If you want to set the download directory yourself:
download(win, url, {
directory: "/path/to/my/directory/"
})
The package will create directories as needed.

I would just like to add, I had spent a couple of days messing around with this issue. I encountered the exact same thing on my project. By disabling my antivirus shields, it stopped interrupting the downloads. Make sure you make an exception for your project / any visual studio code processes. I just chose to disable the shield as a whole.
Edit: do so at your own risk.

Environment variable is undefined in electron even it has been set inside webpack.DefinePlugin

I have a requirement where we need to set dll path based upon whether it is executing in production or in development environment. So I decided to place that value in environment variable and tried to achieve that using webpack.DefinePlugin({}).
Method 1:
webpack.config.json
plugins: [
new webpack.DefinePlugin({
'process.env.NODE_ENV' : JSON.stringify('production')
})
And then I tried to get that value in electron's main process, In my case elec.js
elec.js
const Electron = require("electron");
const app = require("electron");
var dllPath = "";
function createWindow() {
let win = new BrowserWindow({
width: 800,
height: 600,
title: "Test",
icon: "Test.ico"
});
win.setMenu(null);
win.loadURL(
url.format({
pathname: path.join(__dirname, "../renderer/index.html"),
protocol: "file:",
slashes: true
})
);
if (process.env.NODE_ENV ==='production') {
dllPath = path.join(
__dirname,
"./../../dll/test.dll"
);
} else {
dllPath = path.join(
__dirname,
"./../../../dll/test.dll"
);
}
}
app.on("ready", createWindow);
But problem is that when I try to access that value in createWindow() function it is undefined so flow always goes to else block.
Is there anything I am missing?
Method 2:
I tried to achieve the same using cross-env node package, but no luck. Please find below code block which I tried using cross-env.
package.json
"scripts": {
"build": "cross-env process.env.NODE_ENV=production && rimraf ./dist/ && webpack --progress && node-sass
./src/renderer/scss/ -o ./dist/renderer/ && rimraf ./dist/renderer/includes/"
}

The problem is multi-faceted.
First, your elec.js is executed by Electron before the app is loaded. Electron runs elec.js, which creates the Browser window (let win = new BrowserWindow(...)) and loads HTML file (win.loadURL(...)) into it inside the browser process, the HTML then loads your webpack'ed js. So none of the webpacked js code is available in the elec.js. The webpack'ed code is also running in another process than the elec.js.
Another thing to note is that webpack plugin does not create any assignment to the variable it points too. It is done by simple text search and replace, in your example, all instances of process.env.NODE_ENV will be replaced with "production" string in the source code that is webpack'ed. That is not too obvious, but messes up the expected results.
One last thing - webpack plugin does not change any code in elec.js file, as that file is not webpack'ed.
So all that makes process.env.NODE_ENV from the build/webpack time not available in the elec.js code.
Once the mechanisms are clear, there are few ways to solve the problem, I will give general ideas, as there are plenty of discussions on each, and depending on circumstances and desired use case, some are better than others:
Generate a js file with necessary assignments based on environment variable during build (e.g. copy one of env-prod.js / env-dev.js -> env.js), copy it next to the elec.js, and reference it (require(env.js)) in elec.js.
Pass environment variable from command line (e.g. NODE_ENV=1 electron .) - it will get to elec.js.
Include a file into webpack based on environment variable (e.g. copy one of env-prod.js / env-dev.js -> env.js) and peek into webpacked' files from elec.js, e.g. using asar commands.
Use different version in package.json depending on build (e.g. version: "1.0.0-DEBUG" for debug), and read & parse it by calling app.getVersion() in elec.js. It is tricky as package.json should be a single file, but OS commands could be used (e.g. in "scripts") to copy one of prepared package.json files before invoking npm.
Here are some links that could help too:
Electron issue #7714 - discussion on relevant features in Electron
electron-is-dev - module checking if it is in dev
Electron boilerplate - example boilerplate that uses config/env-prod/dev files

The insight provided by iva2k is what allowed me to come to a solution for this same problem.
Using dotenv to create a .env file for my config got me halfway to where I wanted to be (setting up a few environment variables for use in a production setting). The problem then became that Electron wasn't passing those from the Main process down to the Renderer process by default.
The work-around is simple: use Electron's own ipcMain and ipcRenderer modules to pass the dotenv object between the two.
In your main file (e.g. your elec.js file), place an ipcMain event listener after requiring the module:
const config = require('dotenv').config();
const electron = require('electron');
const { app, BrowserWindow, ipcMain } = electron;
...
ipcMain.on('get-env', (event) => {
event.sender.send('get-env-reply', config);
});
Elsewhere, in your application's rendering-side, place this anywhere necessary:
async function getConfig()
{
const { ipcRenderer } = window.require('electron');
let config = null;
ipcRenderer.on('get-env-reply', (event, arg) => {
// The dotenv config object should return an object with
// another object inside caled "parsed". Change this if need be.
config = arg.parsed;
});
ipcRenderer.send('get-env');
return config;
}
This basically allowed me to declare one event in the Main process file, and then re-use it in any process-side file I wanted, thus allowing me to obfuscate config variables in a file that goes with the build, but isn't accessible to end-users without opening up the dev-tools.

Maybe late but can use simple hack in elec.js
const isProduction = process.env.NODE_ENV === 'production' || (!process || !process.env || !process.env.NODE_ENV);

In your console
For Windows
set MY_VARIABLE=true
For linux
$ export MY_VARIABLE=true
window.process.env.MY_VARIABLE

Setting environment variables in Gatsby

I used this tutorial: https://github.com/gatsbyjs/gatsby/blob/master/docs/docs/environment-variables.md
Steps I followed:
1) install dotenv#4.0.0
2) Create two files in root folder: ".env.development" and ".env.production"
3) "follow their setup instructions" (example on dotenv npm docs)
In gatsby-config.js:
const fs = require('fs');
const dotenv = require('dotenv');
const envConfig =
dotenv.parse(fs.readFileSync(`.env.${process.env.NODE_ENV}`));
for (var k in envConfig) {
process.env[k] = envConfig[k];
}
Unfortunately, when i run gatsby develop, NODE_ENV isn't set yet:
error Could not load gatsby-config
Error: ENOENT: no such file or directory, open 'E:\Front-End Projects\Gatsby\sebhewelt.com\.env.undefined'
It works when I set it manually:
dotenv.parse(fs.readFileSync(`.env.development`));
I need environment variables in gatsby-config because I put sensitive data in this file:
{
resolve: `gatsby-source-contentful`,
options: {
spaceId: envConfig.CONTENTFUL_SPACE_ID,
accessToken: envConfig.CONTENTFUL_ACCESS_TOKEN
}
}
How to make it work?
PS: Additional question - As this made me think, I know I shouldn't put passwords and tokens on github, but as netlify builds from github, is there other safe way?

I had a similar issue, I created 2 files in the root ".env.development" and ".env.production" but was still not able to access the env file variables - it was returning undefined in my gatsby-config.js file.
Got it working by npm installing dotenv and doing this:
1) When running gatsby develop process.env.NODE_ENV was returning undefined, but when running gatsby build it was returning 'production' so I define it here:
let env = process.env.NODE_ENV || 'development';
2) Then I used dotenv but specify the filepath based on the process.env.NODE_ENV
require('dotenv').config({path: `./.env.${env}`});
3) Then you can access your variables for your config:
module.exports = {
siteMetadata: {
title: `Gatsby Default Starter`,
},
plugins: [
`gatsby-plugin-react-helmet`,
{
resolve: `gatsby-source-contentful`,
options: {
spaceId: `${process.env.CONTENTFUL_ID}`,
accessToken: `${process.env.CONTENTFUL_TOKEN}`,
},
},
],
}

You should only use env files when you're comfortable checking those into git. For passwords/tokens/etc. add them to Netlify or whatever build tool you use through their dashboard.
These you can access in gatsby-config.js & gatsby-node.js via process.env.ENV_VARIABLE.
You can't access environment variables added this way in the browser however. For this you'll need to use .env.development & .env.production.

I really dislike the .env.production file pattern, our build system sets up and uses env variables and having extra build steps to write those into a file is weird. But Gatsby only whitelists GATSBY_ of the env vars, with no obvious way of adding your own.
But doing that isn't so hard, you can do it by adding something like this in the gatsby-node.js file:
exports.onCreateWebpackConfig = ({ actions, getConfig }) => {
const config = getConfig();
// Allow process.env.MY_WHITELIST_PREFIX_* environment variables
const definePlugin = config.plugins.find(p => p.definitions);
for (const [k, v] of Object.entries(process.env)) {
if (k.startsWith("MY_WHITELIST_PREFIX_")) {
definePlugin.definitions[`process.env.${k}`] = JSON.stringify(v);
}
}
actions.replaceWebpackConfig(config);
};

After doing a few searches, I found that we can set environment variables through netlify website, here are the steps:
Under your own netlify console platform, please go to settings
Choose build & deploy tab (can be found on sidebar)
Choose environment sub-tab option
Click edit variables and add/put your credentials in
Done!