Background
In XULRunner version belowe 12.0 it's works, but when i'm trying port it to version 12.0 or higher it crash application.
Main reason is that in sdk v12 or newer developers remove proxy objects to xpcom components and recommend replace it
by wrapping objects with nsRunnable/nsIRunnable and route invocation to main thread by function NS_DispatchToMainThread (click here)
What i'm developing?
I created db connector which is async and comunicate with main thread by callbacks.
Using: XULRunner v6, porting to XULRunner v17 or above
//nsIDBCallback.idl
[scriptable, function, uuid(XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX)]
interface nsIDBCallback : nsISupports {
void onInfo(in long phase, in long status, in string info);
}
//nsDBService.h, it is XPCOM component
class nsDBService : public nsIDBService, nsIRunnable
{
public:
NS_DECL_ISUPPORTS
NS_DECL_NSIRUNNABLE
NS_DECL_NSIDBSERVICE
private:
std::vector<nsIThread*> threads;
std::vector<nsIDBCallback*> callbacks;
std::vector<const char*> sqls;
nsIThread* makeNewThread();
void runOperationIfNotBussy();
public:
NS_IMETHODIMP Query(const char *sql, nsIDBCallback *callback);
}
//nsDBService.cpp
// adding query and other data to buffers,
// it's thread safe, there are used mutex's
NS_IMETHODIMP nsDBService::Query(const char *sql, nsIDBCallback *callback)
{
callbacks.push_back(callback);
sqls .push_back(sql);
threads .push_back( makeNewThread() );
//run added operation if db driver is free,
//if driver is bussy then invocation is in buffer and need to wait
runOperationIfNotBussy();
return NS_OK;
}
void nsDBService::runOperationIfNotBussy()
{
//some conditions, test's etc.
//run first operation on list
// RUNNING A THREAD, still ok
if(...) threads.front()->Dispatch(this, nsIEventTarget::DISPATCH_NORMAL);
}
//if this method is used by another thread+db query,
//then other operations can't run and need to wait
//operations are stored and supported like fifo
NS_IMETHODIMP nsDBService::Run(void)
{
//some other operations
//real db operations in background
int32_t phase = 3; //endphase
int32_t code = 0; //ok
const char *msg = "OK";
nsIDBCallback *callback = callbacks.pop();
//wrapping callback function with runnable interface
nsIRunnable *runCallback = new nsResultCallback(callback,
phase,
code,
msg);
//routing event to main thread
NS_DispatchToMainThread(runCallback, NS_DISPATCH_NORMAL);
runOperationIfNotBussy();
}
//nsResultCallback.h
class nsResultCallback: public nsRunnable
{
public:
NS_DECL_ISUPPORTS
public:
NS_DECL_NSIRUNNABLE
private:
nsIDBCallback* callback;
int32_t resPhase;
int32_t resStatus;
const char* resMessage;
public:
nsResultCallback(nsIDBCallback* callback,
int32_t phase,
int32_t status,
const std::string &message)
: callback(callback),
resPhase(phase),
resStatus(status),
resMessage(c_str_clone(message.c_str())) {};
~nsResultCallback();
};
//nsResultCallback.cpp
NS_IMETHODIMP nsResultCallback::Run(void)
{
nsresult rv = NS_ERROR_FAILURE;
try
{
// APP HANDS AND CRUSH !
if(this->callback) this->callback->OnInfo(resPhase, resStatus, resMessage);
}
catch(...)
{
rv = NS_ERROR_UNEXPECTED;
ERRF("nsBackpack::Run call method OnInfo from callback failed");
}
return rv;
}
INVOCATION
// *.js
nsDBService.query("SELECT * FROM t", function(phase, code, mes) {
//some UI actions or others db queries
});
Problem:
Application freeze and crash when code execution look like this:
nsDBService::Query //main thread ok
nsDBService::runOperationIfNotBussy //main thread
nsDBService::threads.front()->Dispatch //run bg thread
nsDBService:Run //bg thread
NS_DispatchToMainThread //main thread
nsResultCallback::Run //main thread
nsIDBCallback::OnInfo //main thread, crash
If code execution look like this, everything is ok:
nsDBService::Query //main thread ok
NS_DispatchToMainThread //main thread
nsResultCallback::Run //main thread
nsIDBCallback::OnInfo //main thread ok
Question:
When nsIDBCallback is invoked from NS_DispatchToMainThread and NS_DispatchToMainThread is invoked from other thread then main app thread, then execution fails, what i'm missing, don't understand? Or what is another approach for background tasks?
Cannot reproduce, as you didn't provide a self-contained, complete example, so some remarks instead:
The first thing I noticed is the cross-thread access of std::vector. You wrote something about mutexes in the comments, so this might be OK.
What is certainly wrong, is storing raw pointers to nsIDBCallback. XPCOM objects are ref-counted. So as soon your Query method returns, the underlying object might be deleted if there are no other references to it, leaving behind a dangling pointer in your vector. I think this is what is happening here!
You need to keep the object alive until the thread is done with it, preferably by putting it into a nsCOMPtr<nsIDBCallback> somewhere, e.g. in an nsCOMPArray<nsIDBCallback>.
PS: Turns out this is a somewhat old question, which I missed... So sorry for the delay answering it :p
Related
We are building an Electron app that allows users to supply their own 'modules' to run. We are looking for a way to require the modules but then delete or kill the modules if need be.
We have looked a few tutorials that seem to discuss this topic but we can't seem to get the modules to fully terminate. We explored this by using timers inside the modules and can observe the timers still running even after the module reference is deleted.
https://repl.it/repls/QuerulousSorrowfulQuery
index.js
// Load module
let Mod = require('./mod.js');
// Call the module function (which starts a setInterval)
Mod();
// Delete the module after 3 seconds
setTimeout(function () {
Mod = null;
delete Mod;
console.log('Deleted!')
}, 3000);
./mod.js
function Mod() {
setInterval(function () {
console.log('Mod log');
}, 1000);
}
module.exports = Mod;
Expected output
Mod log
Mod log
Deleted!
Actual output
Mod log
Mod log
Deleted!
Mod log
...
(continues to log 'Mod log' indefinitely)
Maybe we are overthinking it and maybe the modules won't be memory hogs, but the modules we load will have very intensive workloads and having the ability to stop them at will seems important.
Edit with real use-case
This is how we are currently using this technique. The two issues are loading the module in the proper fashion and unloading the module after it is done.
renderer.js (runs in a browser context with access to document, etc)
const webview = document.getElementById('webview'); // A webview object essentially gives us control over a webpage similar to how one can control an iframe in a regular browser.
const url = 'https://ourserver.com/module.js';
let mod;
request({
method: 'get',
url: url,
}, function (err, httpResponse, body) {
if (!err) {
mod = requireFromString(body, url); // Module is loaded
mod(webview); // Module is run
// ...
// Some time later, the module needs to be 'unloaded'.
// We are currently 'unloading' it by dereferencing the 'mod' variable, but as mentioned above, this doesn't really work. So we would like to have a way to wipe the module and timers and etc and free up any memory or resources it was using!
mod = null;
delete mod;
}
})
function requireFromString(src, filename) {
var Module = module.constructor;
var m = new Module();
m._compile(src, filename);
return m.exports;
}
https://ourserver.com/module.js
// This code module will only have access to node modules that are packaged with our app but that is OK for now!
let _ = require('lodash');
let obj = {
key: 'value'
}
async function main(webview) {
console.log(_.get(obj, 'key')) // prints 'value'
webview.loadURL('https://google.com') // loads Google in the web browser
}
module.exports = main;
Just in case anyone reading is not familiar with Electron, the renderer.js has access to 'webview' elements which are almost identical to iframes. This is why passing it to the 'module.js' will allow the module to access manipulate the webpage such as change URL, click buttons on that webpage, etc.
There is no way to kill a module and stop or close any resources that it is using. That's just not a feature of node.js. Such a module could have timers, open files, open sockets, running servers, etc... In addition node.js does not provide a means of "unloading" code that was once loaded.
You can remove a module from the module cache, but that doesn't affect the existing, already loaded code or its resources.
The only foolproof way I know of would be to load the user's module in a separate node.js app loaded as a child process and then you can exit that process or kill that process and then the OS will reclaim any resources it was using and unload everything from memory. This child process scheme also has the advantage that the user's code is more isolated from your main server code. You could even further isolate it by running this other process in a VM if you wanted to.
What is the difference between hasPendingMacrotasks or hasPendingMicrotasks within NgZone? The documentation seems to be lacking information. All I know is that they return a boolean. But what exactly are they checking for? What is considered a micro task? And what is considered a macro task?
class NgZone {
static isInAngularZone() : boolean
static assertInAngularZone() : void
static assertNotInAngularZone() : void
constructor({enableLongStackTrace = false}: any)
run(fn: () => any) : any
runGuarded(fn: () => any) : any
runOutsideAngular(fn: () => any) : any
onUnstable : EventEmitter<any>
onMicrotaskEmpty : EventEmitter<any>
onStable : EventEmitter<any>
onError : EventEmitter<any>
isStable : boolean
hasPendingMicrotasks : boolean
hasPendingMacrotasks : boolean
}
My best guess is that micro refers to tasks from within a specific class whereas macro probably refers to a task in regards to the whole application. Can anyone verify or confirm this assumption? Or shed some light on the specifics?
NgZone Docs:
https://angular.io/docs/ts/latest/api/core/index/NgZone-class.html#!#hasPendingMicrotasks-anchor
There are three kinds of tasks
1) MicroTask:
A microtask is work which will execute as soon as possible on empty stack frame. A microtask is guaranteed to run before host environment performs rendering or I/O operations. A microtask queue must be empty before another MacroTask or EventTask runs.
i.e. Promise.then() executes in microtask
2) MacroTask
MacroTasks are interleaved with rendering and I/O operations of the host environment. They are guaranteed to run at least once or canceled (some can run repeatedly such as setInterval). Macro tasks have an implied execution order.
i.e. setTimeout, setInterval, setImmediate
3) EventTask
EventTasks are similar to macro tasks, but unlike macro tasks they may never run. When an EventTask is run, it pre-empts whatever the next task is the macro task queue. Event tasks do not create a queue.
i.e. user click, mousemove, XHR state change.
Why is it useful to know if any of the tasks are currently being performed?
Knowing when a task has executed and a microtask queue is empty allows frameworks to know when it is time to render the UI.
Tracking when all scheduled tasks are executed allows a test framework to know when an asynchronous test has completed.
ng_zone.ts
private checkStable() {
if (this._nesting == 0 && !this._hasPendingMicrotasks && !this._isStable) {
try {
this._nesting++;
this._onMicrotaskEmpty.emit(null);
} finally {
this._nesting--;
if (!this._hasPendingMicrotasks) {
try {
this.runOutsideAngular(() => this._onStable.emit(null));
} finally {
this._isStable = true;
}
}
}
}
}
See also
what is the use of Zone.js in Angular 2
I have a solution that has both a Windows Runtime Component (C#) and a Universal App (JS).
One of my classes in the WRC has the following static function:
public static IAsyncOperation<Project> Import()
{
return System.Threading.Tasks.Task.Run<Project>(async () =>
{
try
{
FileOpenPicker picker = new FileOpenPicker();
picker.ViewMode = PickerViewMode.List;
picker.SuggestedStartLocation = PickerLocationId.DocumentsLibrary;
picker.FileTypeFilter.Add(".xml");
StorageFile source = await picker.PickSingleFileAsync();
if (source != null)
{
StorageFile destination = await ApplicationData.Current.RoamingFolder.CreateFileAsync(source.Name, CreationCollisionOption.ReplaceExisting);
await source.MoveAndReplaceAsync(destination);
return await Project.Open(source.DisplayName);
}
else
{
return null;
}
}
catch (Exception)
{
return null;
}
}).AsAsyncOperation<Project>();
}
I am trying to call this function from JS using:
SignalOne.Data.Project.import().done(function () {
new Windows.UI.Popups.MessageBox("Done").showAsync();
}
However, while the "Done" message appears, the file open dialog does not. If I put a message box as the first line inside the try of the C#, it doesn't display, either.
I know I have an upper-case Import in C# and a lower-case import in JS, but that is how it comes up with Intellisense, and if I change it to upper-case in JS it crashes.
I'm sure I'm missing something small/stupid, but I can't put my finger on it.
Thanks.
As you known, if we want to use the async method in Windows Runtime Components, we should be able to use the WindowsRuntimeSystemExtensions.AsAsyncAction or AsAsyncOperation extension method to wrap the task in the appropriate interface.
You can use .NET Framework tasks (the Task class and generic Task class) to implement your asynchronous method. You must return a task that represents an ongoing operation, such as a task that is returned from an asynchronous method written in C# or Visual Basic, or a task that is returned from the Task.Run method.
For more info, see Asynchronous operations.
Also the FileOpenPicker.PickSingleFileAsync method should be run in UI thread.
In this example, the event is being fired on the UI thread. If you fire the event from a background thread, for example in an async call, you will need to do some extra work in order for JavaScript to handle the event. For more information, see Raising Events in Windows Runtime Components.
So we should be able to use CoreWindow.GetForCurrentThread method get the UI thread before the async Task is run that the async Task is not run on the UI thread.
For example:
var window = Windows.UI.Core.CoreWindow.GetForCurrentThread();
var m_dispatcher = window.Dispatcher;
Then we should be able to use the FileOpenPicker.PickSingleFileAsync method in the CoreDispatcher.RunAsync method.
For example:
await m_dispatcher.RunAsync(CoreDispatcherPriority.Normal, new DispatchedHandler(async () =>
{
var source = await picker.PickSingleFileAsync();
}));
I'm getting frustrated with part of a Yeoman Generator I'm building. As it's my first, I have no doubt I'm missing something obvious, but here it goes.
Simply put, I'm trying to log a message, Do Things™ and then log another message only when those things have been done.
Here's the method:
repos: function () {
var self = this;
this.log(highlightColour('Pulling down the repositories'));
// Skeleton
this.remote('user', 'skeleton', 'master', function(err, remote) {
if (!err) {
remote.bulkDirectory('.', self.destinationRoot());
} else {
self.log('\n');
self.log(alertColour('Failed to pull down Skeleton'));
repoErr = true;
}
});
//
// Three more near identical remote() tasks
//
if (!repoErr) {
self.log(successColour('Success!'));
self.log('\n');
} else {
self.log(alertColour('One or more repositories failed to download!'));
}
},
Each of the individual remote() tasks are working fine, but I get both the first and last self.log() messages before the file copying happens. It seems trivial, but I simply want the success message to come after everything has been completed.
For example, in the terminal I see:
Pulling down the repositories
Success!
file copying results
It should be:
Pulling down the repositories
file copying results
Success!
I thought it could be something to do with using this.async() with done() at the end of each remote() task, and I tried that, but whenever I do, none of the code fires at all.
I've even tried breaking everything (including the messages) into separate methods, but still no luck.
Such a simple goal, but I'm out of ideas! I'd be grateful for your help!
EDIT: In case you're wondering, I know the messages are coming first because any alerts regarding file conflicts are coming after the messages :)
This is not an issue related to Yeoman. You have asynchronous code, but you're handling it as if it was synchronous.
In the example you posted here, just do the logging as part of this.remote callback:
repos: function () {
var self = this;
this.log(highlightColour('Pulling down the repositories'));
// Skeleton
this.remote('user', 'skeleton', 'master', function(err, remote) {
if (!err) {
remote.bulkDirectory('.', self.destinationRoot());
self.log(successColour('Success!'));
self.log('\n');
} else {
self.log('\n');
self.log(alertColour('Failed to pull down Skeleton'));
self.log(alertColour('One or more repositories failed to download!'));
}
});
},
Maybe your actual use case is more complex; in this case you can use a module like async (or any other alternative) to handle more complex async flow. Either way, Yeoman doesn't provide helpers to handle asynchronous code as this is the bread and butter of Node.js.
I do not have much knowledge about javascript. I have written in C++ a shared library that does certain things in a daemon thread. I needed this to be invoked from javascript. By using SWIG I've successfully able to generate a wrapper and compile my code along with it into .node module using node-gyp (wrote binding.gyp for it too). Now i can drop to node prompt and do something like:
> var a = require("./module_name")
> a.SomeCppFunction("SomeString")
and wonderfully invoke the cpp functions, start a detached thread there and return the control back to javascript. However I want to notify the javascript from the detached cpp thread about stuffs. I tried registering javascript functions by collecting function() {} signature types in void(*fp)() etc., to call them back later from c++, but that didn't work. Is there anyway to be able to achieve this ie., register javascript functions (or something else) as callback in the cpp code ?
You can use a combination of SWIG and Napi. An existing repo which does this is available here, with a blog here. But I'll sum up the process here.
Create your class to use in SWIG, which has a thread running in the threadMain method :
#include <Thread.H>
class Test : public ThreadedMethod {
void *threadMain(void);
public:
Test();
void setFnPointer(const char* s);
};
Now in the Napi code, you will generate your thread safe function like so :
Napi::ThreadSafeFunction tsfn; ///< The node api's threadsafe function
Napi::Value Start( const Napi::CallbackInfo& info ){
Napi::Env env = info.Env();
// Create a ThreadSafeFunction
tsfn = Napi::ThreadSafeFunction::New(env,
info[0].As<Napi::Function>(), // JavaScript function to call
"Resource Name", 0,1);
// return the tsfn as a pointer in a string
char addr[24];
sprintf(addr,"%p",&tsfn);
return Napi::String::New(env, addr);
}
// some small code to call NODE_API_MODULE here, check the file NapiCode.C in the repo
You compile the SWIG code to one module and the Napi code down to a different module and you pass the thread safe funciton pointer from one to the other like so :
var libNapiNodejs = require('../swig/.libs/libNapiNodejs');
let fp = libNapiNodejs.start(function () {
console.log("JavaScript callback called with arguments", Array.from(arguments));
}, 5);
// SWIG get our C++ and thread running
var libSwigCNodejs = require('../swig/.libs/libSwigCNodejs');
let test = new libSwigCNodejs.Test;
test.setFnPointer(fp); // tell swig the callback function pointer to execute
test.run(); // run the C++ thread in the SWIG module
You will see that the C++ thread calls the javascript function. This is what the C++ thread looks like in SWIG :
Napi::ThreadSafeFunction *tsfn; ///< The node api's threadsafe function
void *Test::threadMain(void){
printf("C++ Thread enter %s\n",__func__);
auto callback = []( Napi::Env env, Napi::Function jsCallback, int* value ) {
jsCallback.Call( {Napi::Number::New( env, *value )} );
};
for (int i=0; i<10; i++){
sleep(1);
if (*tsfn) {
printf("calling tsfn->BlockingCall\n");
napi_status status = tsfn->BlockingCall( &i, callback );
if ( status != napi_ok ) // Handle error
break;
}
}
tsfn->Release();
printf("C++ Thread exit %s\n",__func__);
return NULL;
}