Develop Reference

How to initialize a child process with passed in functions in Node.js

Context
I'm building a general purpose game playing A.I. framework/library that uses the Monte Carlo Tree Search algorithm. The idea is quite simple, the framework provides the skeleton of the algorithm, the four main steps: Selection, Expansion, Simulation and Backpropagation. All the user needs to do is plug in four simple(ish) game related functions of his making:
a function that takes in a game state and returns all possible legal moves to be played
a function that takes in a game state and an action and returns a new game state after applying the action
a function that takes in a game state and determines if the game is over and returns a boolean and
a function that takes in a state and a player ID and returns a value based on wether the player has won, lost or the game is a draw. With that, the algorithm has all it needs to run and select a move to make.
What I'd like to do
I would love to make use of parallel programming to increase the strength of the algorithm and reduce the time it needs to run each game turn. The problem I'm running into is that, when using Child Processes in NodeJS, you can't pass functions to the child process and my framework is entirely built on using functions passed by the user.
Possible solution
I have looked at this answer but I am not sure this would be the correct implementation for my needs. I don't need to be continually passing functions through messages to the child process, I just need to initialize it with functions that are passed in by my framework's user, when it initializes the framework.
I thought about one way to do it, but it seems so inelegant, on top of probably not being the most secure, that I find myself searching for other solutions. I could, when the user initializes the framework and passes his four functions to it, get a script to write those functions to a new js file (let's call it my-funcs.js) that would look something like:
const func1 = {... function implementation...}
const func2 = {... function implementation...}
const func3 = {... function implementation...}
const func4 = {... function implementation...}
module.exports = {func1, func2, func3, func4}
Then, in the child process worker file, I guess I would have to find a way to lazy load require my-funcs.js. Or maybe I wouldn't, I guess it depends how and when Node.js loads the worker file into memory. This all seems very convoluted.
Can you describe other ways to get the result I want?

child_process is less about running a user's function and more about starting a new thread to exec a file or process.
Node is inherently a single-threaded system, so for I/O-bound things, the Node Event Loop is really good at switching between requests, getting each one a little farther. See https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/
What it looks like you're doing is trying to get JavaScript to run multiple threads simultaniously. Short answer: can't ... or rather it's really hard. See is it possible to achieve multithreading in nodejs?
So how would we do it anyway? You're on the right track: child_process.fork(). But it needs a hard-coded function to run. So how do we get user-generated code into place?
I envision a datastore where you can take userFn.ToString() and save it to a queue. Then fork the process, and let it pick up the next unhandled thing in the queue, marking that it did so. Then write to another queue the results, and this "GUI" thread then polls against that queue, returning the calculated results back to the user. At this point, you've got multi-threading ... and race conditions.
Another idea: create a REST service that accepts the userFn.ToString() content and execs it. Then in this module, you call out to the other "thread" (service), await the results, and return them.
Security: Yeah, we just flung this out the window. Whether you're executing the user's function directly, calling child_process#fork to do it, or shimming it through a service, you're trusting untrusted code. Sadly, there's really no way around this.

Assuming that security isn't an issue you could do something like this.
// Client side
<input class="func1"> // For example user inputs '(gamestate)=>{return 1}'
<input class="func2">
<input class="func3">
<input class="func4">
<script>
socket.on('syntax_error',function(err){alert(err)});
submit_funcs_strs(){
// Get function strings from user input and then put into array
socket.emit('functions',[document.getElementById('func1').value,document.getElementById('func2').value,...
}
</script>
// Server side
// Socket listener is async
socket.on('functions',(funcs_strs)=>{
let funcs = []
for (let i = 0; i < funcs_str.length;i++){
try {
funcs.push(eval(funcs_strs));
} catch (e) {
if (e instanceof SyntaxError) {
socket.emit('syntax_error',e.message);
return;
}
}
}
// Run algorithm here
}

How can I attach an event handler to the process's exit in a native Node.js module?

I'm working on implementing correct memory management for a native Node.js module. I've ran into the problem described in this question:
node.js native addon - destructor of wrapped class doesn't run
The suggested solution is to bind the destructors of native objects to process.on('exit'), however the answer does not contain how to do that in a native module.
I've taken a brief look at the libuv docs as well, but they didn't contain anything useful in this regard, either.
NOTE: I'm not particularly interested in getting the process object, but I tried it that way:
auto globalObj = NanGetCurrentContext()->Global();
auto processObj = ::v8::Handle<::v8::Object>::Cast(globalObj->Get(NanNew<String>("process")));
auto processOnFunc = ::v8::Handle<::v8::Function>::Cast(processObj->Get(NanNew<String>("on")));
Handle<Value> processOnExitArgv[2] = { NanNew<String>("exit"), NanNew<FunctionTemplate>(onProcessExit)->GetFunction() };
processOnFunc->Call(processObj, 2, processOnExitArgv);
The problem then is that I get this message when trying to delete my object:
Assertion `persistent().IsNearDeath()' failed.
I also tried to use std::atexit and got the same assertion error.
So far, the best I could do is collecting stray ObjectWrap instances in an std::set and cleaning up the wrapped objects, but because of the above error, I was unable to clean up the wrappers themselves.
So, how can I do this properly?

I was also getting the "Assertion persistent().IsNearDeath()' failed" message.
There is a node::AtExit() function that runs just before Node.js shuts down - the equivalent of process.on('exit').
Pass a callback function to node::AtExit from within your add-on's init function (or where ever is appropriate).
The function is documented here:
https://nodejs.org/api/addons.html#addons_atexit_hooks
For example:
NAN_MODULE_INIT(my_exports)
{
// other exported stuff here
node::AtExit(my_cleanup);
}
NODE_MODULE(my_module, my_exports) //add-on exports
//call C++ dtors:
void my_cleanup()
{
delete my_object_ptr; //call object dtor, or other stuff that needs to be cleaned up here
}

Is it possible to fork child processes and wait for them to return in Node/JS?

I have to do a certain calculation many many times. I want to split it over multiple cores, so I want to use child_process.fork(), and then pass each child a chunk of the work.
This works fine, except that my main process (the one that does the fork()) just keeps going after fork()ing and has already terminated by the time the children complete their work.
What I really want to do is wait for the children to finish before continuing execution.
I can't find any good way to do this in Node. Any advice?

If you spawn a new V8 process via .fork(), it returns a new child object which implements a communication layer. For instance
var cp = require( 'child_process' ),
proc = cp.fork( '/myfile.js' );
proc.on('message', function( msg ) {
// continue whatever you want here
});
and within /myfile.js you just dispatch an event when you're done with the work
process.send({ custom: 'message' });
Be aware of the fact that this method indeed spawns a new V8 instance, which eats a good chunk of memory. So you should use that very thoughtfully. Maybe you don't even need to do it that way, maybe there is a more "node like" solution (using process.nextTick to calculate heavy data).

Performance heavy algorithms on Node.js

I'm creating some algorithms that are very performance heavy, e.g. evolutionary and artificial intelligence. What matters to me is that my update function gets called often (precision), and I just can't get setInterval to update faster than once per millisecond.
Initially I wanted to just use a while loop, but I'm not sure that those kinds of blocking loops are a viable solution in the Node.js environment. Will Socket.io's socket.on("id", cb) work if I run into an "infinite" loop? Does my code somehow need to return to Node.js to let it check for all the events, or is that done automatically?
And last (but not least), if while loops will indeed block my code, what is another solution to getting really low delta-times between my update functions? I think threads could help, but I doubt that they're possible, my Socket.io server and other classes need to somehow communicate, and by "other classes" I mean the main World class, which has an update method that needs to get called and does the heavy lifting, and a getInfo method that is used by my server. I feel like most of the time the program is just sitting there, waiting for the interval to fire, wasting time instead of doing calculations...
Also, I'd like to know if Node.js is even suited for these sorts of tasks.

You can execute havy algorithms in separate thread using child_process.fork and wait results in main thread via child.on('message', function (message) { });
app.js
var child_process = require('child_process');
var child = child_process.fork('./heavy.js', [ 'some', 'argv', 'params' ]);
child.on('message', function(message) {
// heavy results here
});
heavy.js
while (true) {
if (Math.random() < 0.001) {
process.send({ result: 'wow!' });
}
}

What are the inner workings of the Selenium waitFor mechanism?

I am trying to customize the behavior of Selenium's click command, (via user-extentions.js), by intercepting calls to doClick(locator). Basically I need to delay click actions whenever our application's "busy indicator" is being displayed.
(Now the standard answer for this kind of thing is to insert a waitFor into the script for those situations. Indeed, we currently have zillions of them throughout our scripts. I'm trying to eliminate those.)
Detecting the page element is the trivial part. The tricky part is getting the script to actually wait. My promising looking, but failed attempt looks like this:
var nativeClick = Selenium.prototype.doClick;
Selenium.prototype.doClick = function(locator) {
this.doWaitForCondition("!selenium.browserbot.findElementOrNull('busy-indicator')", 5000);
return nativeClick.call(this, locator);
}
The doWaitForCondition gets called before every click, but it does not wait when the condition evaluates to false. nativeClick always gets called immediately, and so no delay is introduced. I suspect that the doWaitForCondition function doesn't actually do any waiting per se, but rather establishes the conditions for it within the command execution loop. And in this case the click command is already in play, and I'm trying to run a command within a command.
Can somebody shed some light on how Selenium command execution and waitFor works, or offer suggestions on how this might be done?

I have finally solved this. And with an approach that is much better than trying to intercept click processing in its various forms. My refined goal is: to delay execution of script command completion when our application is "busy".
How Selenium command processing works:
Upon completion, each selenium command returns an ActionResult object, (see ActionHandler.prototype.execute). The terminationCondition attribute on this object is a function that determines when it is okay for selenium to proceed to the next command, (TestLoop.prototype.continueTestWhenConditionIsTrue). Basically, selenium repeatedly executes the condition function until it yields true. The result object it quite trivial:
function ActionResult(terminationCondition) {
this.terminationCondition = terminationCondition;
}
Customizing it:
I want to delay execution any time myAppIsBusy() returns true. Of course all of the standard delays need to remain in place as well, like waiting for page loads, and explicit waitFor conditions as scripted. The solution is to redefine the selenium result object in my user-extensions.js, as follows:
function ActionResult(terminationCondition) {
this.terminationCondition = function() {
// a null terminationCondition means okay to continue
return (!terminationCondition || terminationCondition()) && !myAppIsBusy();
}
}
The great thing is that this is at a low enough level that it works for the IDE, as well as for RC.
Note that this does not affect Accessor or Assert command types, which return different result objects. But that should be fine, because those commands don't effect the state of the application.

Well, a look at the java drivers com.thoughtworks.selenium.Wait class reveals this:
public void wait(String message, long timeoutInMilliseconds, long intervalInMilliseconds) {
long start = System.currentTimeMillis();
long end = start + timeoutInMilliseconds;
while (System.currentTimeMillis() < end) {
if (until()) return;
try {
Thread.sleep(intervalInMilliseconds);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
throw new WaitTimedOutException(message);
}
I am not to deep into selenium but I excpect that every waitXXX Method points to this.
So, Selenium is working with Thread.sleep(). While this might not look like an ideal solution it shows at least that you cant make it worse by using Thread.sleep() on your own if neccessary. ;-)