Before I give some background, I'd like to clarify that I'm not looking for how to simply spawn a new script under the Electron runtime as a renderer process, I'm trying to use a plain Node runtime.
So I'm aware Electron has some different flavor of a JS runtime under its hood similar to NW.js and I'm trying to get consistent performant results for my report.
Unfortunately, this seems to be much more difficult than I had imagined. I'm specifically testing the speed of the mailparser module although that's not necessarily important here.
I first ran it on the Electron app we're working with, which uses Electron Forge. I called the test script through IPC as that's what we intend to use, so it was called within the callback for ipcMain.handle. Here, the performance was really bad and it was taking 30-50 seconds for our test to complete.
I then ran a test script that just opens a blank HTML file in the same Electron Forge app, and runs the test script in the background. This was much better at 8-12 seconds.
Next, I set up a new directory with a test set, a plain Electron installation, and a mailparser installation. I did not electron-rebuild here, but mailparser does rely on node-iconv and so has native bindings.
I ran a test script with Electron just calling the same line of code. This did not use Electron Forge. The performance here was slightly better at 5-9 seconds.
I then ran another test script, this time just with plain old node, and the performance here was excellent at 1-3s.
So I have two questions here:
Why is the performance varying so much in the Electron tests? Although I used IPCMain, I used the new .handle which should be asynchronous and run in the context of Electron's node runtime, so it should have the same performance as running outside the callback. Moreover, the Electron Forge and plain Electron tests also differ by a couple of seconds which made no sense to me as I assumed Electron Forge would just wrap the Electron binary under the hood.
Seeking optimal and consistent results, I'm wondering how I can spin up a child process with the node runtime inside Electron. Doing this normally just starts a new renderer process which is running Electron's (slower) JS runtime. I'd like to avoid leaving Electron Forge but the only solution I can think of is to bundle precompiled binaries with the process running under the Node runtime built for each platform.
For question 1 it’s hard to know what the problem is without being able to replicate it in code. You might try posting an issue on the Github site for the Electron team re this. They are more likely to know the answer, but they ask for code as well.
Having said that, it isn’t that hard to spin up a child process that is just running node and frees you from Electron/Electron Forge overhead. The easiest is to use node’s fork command, but to tell it to use the main node executable rather than electron.exe. You can just hand it a script to run, so you don’t need to worry about precompiled binaries.
The code below (and here) running on the main process will run a script called server.js in the same folder:
const serverPath = path.join(__dirname, 'server.js');
const { fork } = require('child_process');
child = fork(serverPath, [],
{
execPath: "node",
stdio: ['pipe', 'pipe', 'pipe', 'ipc']
});
If you do that you get a child node.exe process with IPC enabled, rather than an electron.exe process. It can be made to communicate over IPC with the main Electron process. It can also use any npm modules installed from your usual package.json, because the script can be in the same folder as your other scripts. So if it works this is quite a clean solution.
I entertained myself this afternoon by writing some code that does this. I’ve put this on Github. It uses mailparser to parse a simple mail in the node.exe process in server.js.
This also packages correctly with Electron Forge as far as I can see, in terms of including the correct scripts. It won't package node itself, so that either needs to be installed on the target machine or I think you'll need to distribute node.exe.
By the way it isn’t entirely clear whether you’ve actually tried this approach and it’s one of those scenarios where it runs slowly. There is still some overhead, because we’re setting up IPC. If it does still run slowly in your use case then I think it’s possible to use node’s spawn command to create a completely standalone process.
Related
Some things to know:
I understand how to make a HTML / CSS / JS website.
I understand how to make a Node JS app and host it on Heroku
I encountered a problem that was very confusing and I still working it out. I am making a firebase project using their latest tree-shaking V9 SDK
import { initializeApp } from 'firebase/app';
I understand Node JS is meant for backend (it can't be run in a browser) so how am I supposed to "build" this into something that I can reference in a script tag? I've heard of webpack but that requires you to run npm run build so how is that practical in any way? That would mean every change I make I would have to build it?
Developer Testing:
How would one live preview this Node JS app on a localhost if Node JS can't be run in a browser? I would live to be able to preview this Node JS app and quickly make changes if that's possible.
I've heard of webpack but that requires you to run npm run build so how is that practical in any way? That would mean every change I make I would have to build it?
Yes, that's the general idea. You make changes to script files on your local machine (or network), and then in order to see the changes take effect, you rebuild the app so that a new bundle is generated. This bundle can then be hosted on your web server or development server, and once it's there, you can refresh the page to see the differences.
But almost all of this can be automated, resulting in it not really being much of a chore at all.
Nodemon is a popular tool that watches a directory for changes and can run Node scripts when a change is detected. With it, you could, for example, make a change to a file, and then it'll automatically rebuild your app.
For Webpack in particular, there's webpack-dev-server, which can automatically refresh a page when the app gets rebuilt.
So, during development, the workflow can be as simple as - make a change, then alt-tab to your browser (hosting the app on localhost) to look at the changes.
Bundles built for the purpose of eventually hosting them on the front-end can easily incorporate Node modules - the build process can take the code from the module, and the bundle produced can include that code. This sort of thing is extraordinarily common.
I am trying to write a Node app that, once deployed, will log to Windows Event Log.
I looked through available packages and was able to successfully use node-windows to write logs to the Windows Event Viewer when I ran the app using the command line. However, when I used pkg to turn the app into an .exe file and tried to run the .exe file, it was no longer logging to Windows Event Log.
As an example, I tried writing a basic app as follows:
const EventLogger = require('node-windows').EventLogger;
const log = new EventLogger('TestApp');
log.info('Test test test!', 1000)
If I run this app using the command line (e.g. node index.js), it logs to Windows Event Viewer. However, when I run pkg to convert the project into an executable file and try to run the executable file (both as an instance and using Windows Task Scheduler), it no longer logs out.
I have already checked to ensure that the .exe is running as administrator, so I don't think it is an issue with permissions. Anyone have any thoughts on why the .exe may not be logging out? Are there any other NPM packages/libraries that provide the ability to log to Windows Event Log?
Thank you in advance!
pkg packages js source files and other assets into an EXE, and exposes them within a node process using a virtual file system. This works fine for most things, but breaks if any of those assets need to be used outside of your process.
For most of its functionality, the node-windows module distributes a few executables of its own or uses Windows builtins. It exec()s those EXEs on your behalf.
For the helpers distributed by the module, the EXE must exist on the real filesystem in order to exec() it. It is highly likely that pkg breaks this; you'd need to distribute those EXEs and place them on disk yourself.
Event logging uses the builtin eventcreate, which should work, but maybe pkg is doing something weird with the virtual filesystem that breaks how child_process works.
At any rate, I strongly recommend against using node-windows for logging because it exec()s a new child process for every log call. This is highly inefficient on Windows: it has to spawn a shell (cmd.exe) and then run the helper (eventcreate.exe), and process creation on Windows is slow. Painfully slow.
If you start logging enough things, you'll soon bring your entire system to its knees with process bookkeeping.
Instead, use a native module that calls the ReportEvent API directly. windows-eventlog fits this bill.
You'll also want to consider pkg's notes regarding native modules.
Native addons (.node files) use is supported, but packaging .node files inside the executable is not resolved yet. You have to deploy native addons used by your project to the same directory as the executable.
I want to deploy my node application as single executable file, is it possible by using systemd, containers. I dont have enough knowledge on systemd and containers. Please help me if anybody knows about it.
Where i work we use pm2 to run NodeJS applications.
It allows you to run multiple instances on one server, monitors them and restarts if needed (on failure or you can provide a memory limit).
If you insist going with systemd, you will have to create a unit - a file that describes the execution path of you application for systemd.
It would usually be in /usr/lib/systemd/system
You would have to create a file ending with ".service"
[Unit]
Description=My NodeJS App
[Service]
ExecStart=/usr/bin/node /path/to/my/app/index.js
I am currently getting my feet wet using Express. To start out, I used express-generator to scaffold a simple app.
While examining the project, I noticed that the npm start command is mapped to a binary (bin/www). Upon further inspection I noticed that this file actually contains code to be executed in Node, hence the #!/usr/bin/env node pragma. For anyone having a deeper understanding of Express/Node the answer may be obvious, but still I am wondering: Why didn't they simply use a .js file to bootstrap the framework. That file could then be run using node www.js, I imagine.
There are probably a few reasons why the script was made an executable
npm scripts can be mapped to execute local JS files in the project or executables on the system.
By mapping npm start to bin/www it is effectively the same as running ./bin/www on the command line with the important distinction that by running it via a npm start, it will also work cross platform (e.g. on systems that ignore the hashbang statement, like Windows), otherwise you would need to run it as node bin/www on those systems.
There's a binary ready to add to startup scripts.
I'd like to replicate what sencha app watch is doing in a custom executable to get more control over the whole process... Watching the file system seems pretty straight forward, but what is it with the rebuild that app watch is allegedly doing?
In the documentation it just says (I quote):
This command watches the current application’s source code for changes
and and rebuild the necessary outputs to support “dev mode”.
So what exactly does "rebuild" mean? When I manually run sencha app build it takes like forever to finish, while sencha app watch is doing it quite a bit faster... So I doubt that this is what it actually does.
A rebuild command per se doesn't exist either...
It's doing the equivalent of sencha app build development. The reason it goes faster is that it keeps the JVM running and it doesn't re-run the initialisation tasks continually.
If you want to take more control of this yourself, the relevant Ant tasks are in the Sencha CMD distribution - most (but not all) Sencha CMD commands are delegated down to the Ant tasks.
It's a little smarter than doing a simple build - because it knows which files have been changed, it knows what steps it needs to do. As such, it won't run redundant steps (another speed win).
One key difference is with CSS - using sencha app watch will create the CSS once, and then subsequent edits are processed using Fashion instead.