I have a music player built with Electron.
I am having some performance / process blocking that I didn't expect. I am trying to do some background processes for heavy IO operations. (determining songs duration and album covers)
I am doing this by calling methods through the electron.remote module.
I have noticed that the app didn't do these things asynchronously somehow.
I have been running the performance tool to check and saw the click handler taking a huge time.
Digging deeper I found that ipcRenderer.sendSync is called.
There is a warning about sendSync blocking nature in Electron Docs. But, my own code does not call it. So I suspect the remote module or something else in my code causing sendSync to be called.
The entire app code is on Github but here is an example of electron.remote usage.
The gist is something like this:
import {remote} from 'electron'
const fs = remote.require('fs')
const mm = remote.require('musicmetadata')
// read song file, IO
function readMetadata (filePath) {
return new Promise(function (resolve, reject) {
const stream = fs.createReadStream(filePath)
mm(stream, {duration: true}, function (err, metadata) {
// ...
})
})
}
// get metadata for an array of songs
async function refreshSongsDuration (songs) {
const songsMetadata = await Promise.all(songs.map((song) => readMetadata(song.filePath)))
return songs.map((song, index) => {
song.duration = songsMetadata[index].duration
return song
})
}
Then in a click handler I'll have something like this:
playArtist (artistID) {
const songs = this.library.getArtistSongs(artistID)
this.playlist.setSongs(songs)
musicPlayer.play()
const shouldGetDuration = songs.some((song) => song.duration === 0)
// This is expected to be asynchronous and non blocking.
if (shouldGetDuration) {
mediaLibrary.refreshSongsDuration(songs)
.then((updatedSongs) => {
this.playlist.set('songs', updatedSongs)
})
}
}
So, I guess the simple question here is, what am I doing wrong causing these blocking processes?
https://github.com/electron/electron/blob/master/docs/api/remote.md#remote-objects
Each object (including functions) returned by the remote module represents an object in the main process (we call it a remote object or remote function). When you invoke methods of a remote object, call a remote function, or create a new object with the remote constructor (function), you are actually sending synchronous inter-process messages.
every remote module is sync in nature.
Related
This answer to a similar question does a great job at explaining how fastify-plugin works and what it does. After reading the explanation, I still have a question remaining; how is this different from a normal function call instead of using the .register() method?
To clarify with an example, how are the two approaches below different from each other:
const app = fastify();
// Register a fastify-plugin that decorates app
const myPlugin = fp((app: FastifyInstance) => {
app.decorate('example', 10);
});
app.register(myPlugin);
// Just decorate the app directly
const decorateApp = (app: FastifyInstance) => {
app.decorate('example', 10);
};
decorateApp(app);
By writing a decorateApp function you are creating your own "API" to load your application.
That said, the first burden you will face soon is sync or async:
decorateApp is a sync function
decorateAppAsync within an async function
For example, you need to preload something from the database before you can start your application.
const decorateApp = (app) => {
app.register(require('#fastify/mongodb'))
};
const businessLogic = async (app) => {
const data = await app.mongo.db.collection('data').find({}).toArray()
}
decorateApp(app)
businessLogic(app) // whoops: it is async
In this example you need to change a lot of code:
the decorateApp function must be async
the mongodb registration must be awaited
the main code that loads the application must be async
Instead, by using the fastify's approach, you need to update only the plugin that loads the database:
const applicationConfigPlugin = fp(
+ async function (fastify) {
- function (fastify, opts, next) {
- app.register(require('#fastify/mongodb'))
- next()
+ await app.register(require('#fastify/mongodb'))
}
)
PS: note that fastify-plugin example code misses the next callback since it is a sync function.
The next bad pattern will be high hidden coupling between functions.
Every application needs a config. Usually, the fastify instance is decorated with it.
So, you will have something like:
decorateAppWithConfig(app);
decorateAppWithSomethingElse(app);
Now, decorateAppWithSomethingElse will need to know that it is loaded after decorateAppWithConfig.
Instead, by using the fastify-plugin, you can write:
const applicationConfigPlugin = fp(
async function (fastify) {
fastify.decorate('config', 42);
},
{
name: 'my-app-config',
}
)
const applicationBusinessLogic = fp(
async function (fastify) {
// ...
},
{
name: 'my-app-business-logic',
dependencies: ['my-app-config']
}
)
// note that the WRONG order of the plugins
app.register(applicationBusinessLogic);
app.register(applicationConfigPlugin);
Now, you will get a nice error, instead of a Cannot read properties of undefined when the config decorator is missing:
AssertionError [ERR_ASSERTION]: The dependency 'my-app-config' of plugin 'my-app-business-logic' is not registered
So, basically writing a series of functions that use/decorate the fastify instance is doable but it adds
a new convention to your code that will have to manage the loading of the plugins.
This job is already implemented by fastify and the fastify-plugin adds many validation checks to it.
So, by considering the question's example: there is no difference, but using that approach to a bigger application
will lead to a more complex code:
sync/async loading functions
poor error messages
hidden dependencies instead of explicit ones
I'm trying to use the SDK within my CDK application. My stack is creating a directory, some networking stuff, then some instances which it joins to the domain using a PowerShell script which requires the AD DNS IPs, I'm currently using it like so:
const ds = new DirectoryService();
const result = new Promise(function (resolve: (value: string) => any, reject) {
ds.describeDirectories({}, function (err, data) {
if (err){
reject(data)
}else{
try{
if (data.DirectoryDescriptions){
if (data.DirectoryDescriptions[0].DnsIpAddrs){
resolve(data.DirectoryDescriptions[0].DnsIpAddrs.toString())
}
}
}catch (e) {
reject("Directory doesn't exist yet.")
}
}
})
});
result.then(value => {
const subs = {
"#{DNS_ADDRESSES}": value,
"#{SECRET_ID}": directory.directorySecret.secretArn
};
Object.entries(subs).forEach(
([key, value]) => {
domainJoinScript = domainJoinScript.replace(key, String(value));
}
);
new Stack(app, 'instances', networking.vpc, domainJoinScript);
}).catch(error => {
print(error)
});
Now this works, but it's far from a clean solution. My new Stack has multiple resources within it which means I have to pass the result of the SDK call through several levels, instead of just directly where it's needed, and if I had to make multiple SDK calls this would get even messier.
The core problem is the AWS SDK being purely asynchronous in JS, which means I have to use the fairly verbose pattern above to wrap it.
Does anyone have a better way to do this?
You could implement the AWS SDK calls using a AwsCustomResource
This allows calling AWS SDK functions on different Resource actions (oncreate, ondelete, onupdate)
The result data can be consumed by calling getData. This does further allow the cdk created CloudFormation Template to work without a client to run it.
One Example where I used this is the following:
const userPoolDomainDescription = new customResources.AwsCustomResource(this, 'user-pool-domain-description', {
onCreate: {
physicalResourceId: 'user-pool-domain-description',
service: 'CognitoIdentityServiceProvider',
action: 'describeUserPoolDomain',
parameters: {
Domain: userPoolDomain.domain
}
}
});
const dnsName = userPoolDomainDescription.getData('DomainDescription.CloudFrontDistribution').toString();
I'm new to Electron and trying to figure out the best way to handle a shared object.
Essentially, I want to initialize something once in the Main process, then use it in multiple renderer processes, like so:
// main.js
const node = rosnodejs.initNode(); // returns a promise
// renderer1.js
node.then((nh) => {
nh.subscribe("topic1");
})
// renderer2.js
node.then((nh) => {
nh.subscribe("topic2");
})
I was able to share node using remote, but then nh.subscribe becomes an anonymous function in my renderers and fails. Here's what I'm doing:
// main.js
global.node = rosnodejs.initNode(); // returns a promise
global.node.then((nh) => {
nh.subscribe("topic1"); // WORKS PERFECTLY!
})
// renderer1.js
const remote = require('electron').remote;
const node = remote.getGlobal('node');
node.then((nh) => {
nh.subscribe("topic2"); // FAILS MISERABLY.
})
Failure message is Error: Could not call remote function 'subscribe'. Check that the function signature is correct. Underlying error: type.datatype is not a function.
Is there a decent way to handle this? Should I be using ipcMain/ipcRenderer instead?
I have a solution that has both a Windows Runtime Component (C#) and a Universal App (JS).
One of my classes in the WRC has the following static function:
public static IAsyncOperation<Project> Import()
{
return System.Threading.Tasks.Task.Run<Project>(async () =>
{
try
{
FileOpenPicker picker = new FileOpenPicker();
picker.ViewMode = PickerViewMode.List;
picker.SuggestedStartLocation = PickerLocationId.DocumentsLibrary;
picker.FileTypeFilter.Add(".xml");
StorageFile source = await picker.PickSingleFileAsync();
if (source != null)
{
StorageFile destination = await ApplicationData.Current.RoamingFolder.CreateFileAsync(source.Name, CreationCollisionOption.ReplaceExisting);
await source.MoveAndReplaceAsync(destination);
return await Project.Open(source.DisplayName);
}
else
{
return null;
}
}
catch (Exception)
{
return null;
}
}).AsAsyncOperation<Project>();
}
I am trying to call this function from JS using:
SignalOne.Data.Project.import().done(function () {
new Windows.UI.Popups.MessageBox("Done").showAsync();
}
However, while the "Done" message appears, the file open dialog does not. If I put a message box as the first line inside the try of the C#, it doesn't display, either.
I know I have an upper-case Import in C# and a lower-case import in JS, but that is how it comes up with Intellisense, and if I change it to upper-case in JS it crashes.
I'm sure I'm missing something small/stupid, but I can't put my finger on it.
Thanks.
As you known, if we want to use the async method in Windows Runtime Components, we should be able to use the WindowsRuntimeSystemExtensions.AsAsyncAction or AsAsyncOperation extension method to wrap the task in the appropriate interface.
You can use .NET Framework tasks (the Task class and generic Task class) to implement your asynchronous method. You must return a task that represents an ongoing operation, such as a task that is returned from an asynchronous method written in C# or Visual Basic, or a task that is returned from the Task.Run method.
For more info, see Asynchronous operations.
Also the FileOpenPicker.PickSingleFileAsync method should be run in UI thread.
In this example, the event is being fired on the UI thread. If you fire the event from a background thread, for example in an async call, you will need to do some extra work in order for JavaScript to handle the event. For more information, see Raising Events in Windows Runtime Components.
So we should be able to use CoreWindow.GetForCurrentThread method get the UI thread before the async Task is run that the async Task is not run on the UI thread.
For example:
var window = Windows.UI.Core.CoreWindow.GetForCurrentThread();
var m_dispatcher = window.Dispatcher;
Then we should be able to use the FileOpenPicker.PickSingleFileAsync method in the CoreDispatcher.RunAsync method.
For example:
await m_dispatcher.RunAsync(CoreDispatcherPriority.Normal, new DispatchedHandler(async () =>
{
var source = await picker.PickSingleFileAsync();
}));
With RequireJS on the front-end, we can listen to see when modules get loaded into the runtime module cache using:
requirejs.onResourceLoad = function (context, map, depArray) {
console.log('onResourceLoad>>>', 'map.id:', map.id, 'context:', context);
};
Can we do this with Node.js somehow? Will be useful for debugging. Especially when servers are loading different files (or in different order) based on configuration.
I assume this might be documented in
https://nodejs.org/api/modules.html
but I am not seeing anything
If you look at the source code for require(), you will find this:
Module._load = function(request, parent, isMain) {
if (parent) {
debug('Module._load REQUEST %s parent: %s', request, parent.id);
}
This shows that you can leverage the debug() call to get the information you need. In order to do this, you will notice that module is setup using util.debuglog('module'). This means that you need to run your application with with the NODE_DEBUG variable set to module. You can do it the following way from the console:
NODE_DEBUG=module node main.js
This will log what you are looking for.
I'm not aware of a documented callback API for the purpose of module load callbacks (although a logging mechanism for module loading appears to exist).
Here's a quick workaround to the apparent lack of such a callback, by monkeypatching Module._load:
const Module = require('module');
const originalModuleLoad = Module._load;
Module._load = function() {
originalModuleLoad.apply(this, arguments);
console.log("Loaded with arguments " + JSON.stringify(arguments));
}
I executed the above code in a REPL and then did require('assert'). Lo and behold:
> require('assert')
Loading with arguments {"0":"assert","1":{"id":"<repl>","exports":{},"filename":null,"loaded":false,"children":[],"paths":["/Users/mz2/Projects/manuscripts-endnote-promo/repl/node_modules","/Users/mz2/Projects/manuscripts-endnote-promo/node_modules","/Users/mz2/Projects/node_modules","/Users/mz2/node_modules","/Users/node_modules","/Users/mz2/.nvm-fish/v6.1.0/lib/node_modules","/Users/mz2/.node_modules","/Users/mz2/.node_libraries","/Users/mz2/.nvm-fish/v6.1.0/lib/node"]},"2":false}
Please don't think about using code like above for anything but debug only purposes.
Because node.js modules are imported (required) synchronously, simply having the require statement means the module is imported.
While RequireJS can import modules asynchronously, the even listening is an important feature, but native require in Node.js leaves this necessity out. This way, as you probably know:
const module = require('module')
// You can use the module here, async or sync.
To add to that, not only require is sync, but also in order to use a module it has to be explicitly required in the same file where it's used. This can be bypassed in several ways, but best practice is to require in every module where you use a module.
For specific modules which require async initialization, either the module should provide an event, or you can wrap the init function using a promise or a callback. For example, using a promise:
const module = require('module')
// Create a promise to initialize the module inside it:
const initialized = new Promise((resolve, reject) => {
// Init module inside the promise:
module.init((error) => {
if(error){
return reject(error)
}
// Resolve will indicate successful init:
resolve()
})
})
// Now with wrapped init, proceed when done:
initialized
.then(() => {
// Module is initialized, do what you need.
})
.catch(error => {
// Handle init error.
})