Develop Reference

How to initialize a child process with passed in functions in Node.js

Context
I'm building a general purpose game playing A.I. framework/library that uses the Monte Carlo Tree Search algorithm. The idea is quite simple, the framework provides the skeleton of the algorithm, the four main steps: Selection, Expansion, Simulation and Backpropagation. All the user needs to do is plug in four simple(ish) game related functions of his making:
a function that takes in a game state and returns all possible legal moves to be played
a function that takes in a game state and an action and returns a new game state after applying the action
a function that takes in a game state and determines if the game is over and returns a boolean and
a function that takes in a state and a player ID and returns a value based on wether the player has won, lost or the game is a draw. With that, the algorithm has all it needs to run and select a move to make.
What I'd like to do
I would love to make use of parallel programming to increase the strength of the algorithm and reduce the time it needs to run each game turn. The problem I'm running into is that, when using Child Processes in NodeJS, you can't pass functions to the child process and my framework is entirely built on using functions passed by the user.
Possible solution
I have looked at this answer but I am not sure this would be the correct implementation for my needs. I don't need to be continually passing functions through messages to the child process, I just need to initialize it with functions that are passed in by my framework's user, when it initializes the framework.
I thought about one way to do it, but it seems so inelegant, on top of probably not being the most secure, that I find myself searching for other solutions. I could, when the user initializes the framework and passes his four functions to it, get a script to write those functions to a new js file (let's call it my-funcs.js) that would look something like:
const func1 = {... function implementation...}
const func2 = {... function implementation...}
const func3 = {... function implementation...}
const func4 = {... function implementation...}
module.exports = {func1, func2, func3, func4}
Then, in the child process worker file, I guess I would have to find a way to lazy load require my-funcs.js. Or maybe I wouldn't, I guess it depends how and when Node.js loads the worker file into memory. This all seems very convoluted.
Can you describe other ways to get the result I want?

child_process is less about running a user's function and more about starting a new thread to exec a file or process.
Node is inherently a single-threaded system, so for I/O-bound things, the Node Event Loop is really good at switching between requests, getting each one a little farther. See https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/
What it looks like you're doing is trying to get JavaScript to run multiple threads simultaniously. Short answer: can't ... or rather it's really hard. See is it possible to achieve multithreading in nodejs?
So how would we do it anyway? You're on the right track: child_process.fork(). But it needs a hard-coded function to run. So how do we get user-generated code into place?
I envision a datastore where you can take userFn.ToString() and save it to a queue. Then fork the process, and let it pick up the next unhandled thing in the queue, marking that it did so. Then write to another queue the results, and this "GUI" thread then polls against that queue, returning the calculated results back to the user. At this point, you've got multi-threading ... and race conditions.
Another idea: create a REST service that accepts the userFn.ToString() content and execs it. Then in this module, you call out to the other "thread" (service), await the results, and return them.
Security: Yeah, we just flung this out the window. Whether you're executing the user's function directly, calling child_process#fork to do it, or shimming it through a service, you're trusting untrusted code. Sadly, there's really no way around this.

Assuming that security isn't an issue you could do something like this.
// Client side
<input class="func1"> // For example user inputs '(gamestate)=>{return 1}'
<input class="func2">
<input class="func3">
<input class="func4">
<script>
socket.on('syntax_error',function(err){alert(err)});
submit_funcs_strs(){
// Get function strings from user input and then put into array
socket.emit('functions',[document.getElementById('func1').value,document.getElementById('func2').value,...
}
</script>
// Server side
// Socket listener is async
socket.on('functions',(funcs_strs)=>{
let funcs = []
for (let i = 0; i < funcs_str.length;i++){
try {
funcs.push(eval(funcs_strs));
} catch (e) {
if (e instanceof SyntaxError) {
socket.emit('syntax_error',e.message);
return;
}
}
}
// Run algorithm here
}

What is the correct way to chain async calls in javascript?

I'm trying to find the best way to create async calls when each call depends on the prior call to have completed. At the moment I'm chaining the methods by recursively calling a defined process function as illustrated below.
This is what I'm currently doing.
var syncProduct = (function() {
var done, log;
var IN_CAT = 1, IN_TITLES = 2, IN_BINS = 3;
var state = IN_CAT;
var processNext = function(data) {
switch(state) {
case IN_CAT:
SVC.sendJsonRequest(url("/api/lineplan/categories"), processNext);
state = IN_TITLES;
break;
case IN_TITLES:
log((data ? data.length : "No") + " categories retrieved!");
SVC.sendJsonRequest(url("/api/lineplan/titles"), processNext);
state = IN_BINS;
break;
case IN_BINS:
log((data ? data.length : "No") + " titles retrieved!");
SVC.sendJsonRequest(url("/api/lineplan/bins"), processNext);
state = IN_MAJOR;
break;
default:
log((data ? data.length : "No") + " bins retrieved!");
done();
break;
}
}
return {
start: function(doneCB, logCB) {
done = doneCB; log = logCB; state = IN_CAT;
processNext();
}
}
})();
I would then call this as follows
var log = function(message) {
// Impl removed.
}
syncProduct.start(function() {
log("Product Sync Complete!");
}, log);
While this works perfectly fine for me I can't help but think there has to be a better (simpler) way. What happens later when my recursive calls get too deep?
NOTE: I am not using javascript in the browser but natively within the Titanium framework, this is akin to Javascript for Node.js.

There are lots of libraries and tools that do async chaining and control-flow for you and they mostly come in two main flavours:
Control-flow libraries
For example, see async, seq and step (callback based) or Q and futures (promise based). The main advantage of these is that they are just plains JS libraries that ease the pain of async programming.
In my personal experience, promise-based libraries tend to lead to code that looks more like usual synchronous code, since you return values using "return" and since promise values can be passed and stored around, similarly to real values.
On the other hand, continuation-based code is more low level since it manipulates code paths explicitely. This can possibly allow for more flexible control flow and better integration with existing libraries, but it might also lead to more boilerplaty and less intuitive code.
Javascript CPS compilers
Extending the language to add native support for coroutines/generators lets you write asynchronous code in a very straightforward manner and plays nice with the rest of the language meaning you can use Javascript if statements, loops etc instead of needing to replicate them with functions. This also means that its very easy to convert previously sync code into an async version. However, there is the obvious disadvantage that not every browser will run your Javascript extension so you will need to add a compilation step in your build proccess to convert your code to regular JS with callbacks in continuation-passing-style. Anyway, one promising alternative is the generators in the Ecmascript 6 spec - while only firefox supports them natively as of now, there are projects such as regenerator and Traceur to compile them back to callbacks. There are also other projects that create their own async syntax (since es6 generators hadn't come up back then). In this category, you will find things such as tamejs and Iced Coffeescript. Finally, if you use Node.js there you could also take a look at Fibers.
My recomendation:
If you just want something simple that won't complicate your build proccess, I would recomend going with whatever control-flow library best fits your personal style and the libraries you already use.
However, if you expect to write lots of complicated and deeply-integrated asynchronous code, I would strongly recommend at least looking into a compiler-based alternative.

How to manage dependencies in JavaScript?

I have scripts that needs to wait for certain conditions to be met before they run - for example wait for another script to be loaded, or wait for a data object to be created.
How can I manage such dependencies? The only way I can think of is to use setTimeout to loop in short intervals and check the existence of functions or objects. Is there a better way?
And if setTimeout is the only choice, what is a reasonable time interval to poll my page? 50 ms, 100 ms?
[Edit] some of my scripts collect data, either from the page itself or from Web services, sometimes from a combination of multiple sources. The data can be ready anytime, either before or after the page has loaded. Other scripts render the data (for example to build charts).
[update] thanks for the useful answers. I agree that I shouldn't reinvent the wheel, but if I use a library, at least I'd like to understand the logic behind (is it just a fancy timeout?) to try and anticipate the performance impact on my page.

You could have a function call like loaded(xyz); at the end of the scripts that are being loaded. This function would be defined elsewhere and set up to call registered callbacks based on the value of xyz. xyzcan be anything, a simple string to identify the script, or a complex object or function or whatever.
Or just use jQuery.getScript(url [, success(data, textStatus)] ).

For scripts that have dependencies on each other, use a module system like RequireJS.
For loading data remotely, use a callback, e.g.
$.get("/some/data", "json").then(function (data) {
// now i've got my data; no polling needed.
});
Here's an example of these two in combination:
// renderer.js
define(function (require, exports, module) {
exports.render = function (data, element) {
// obviously more sophisticated in the real world.
element.innerText = JSON.stringify(data);
};
});
// main.js
define(function (require, exports, module) {
var renderer = require("./renderer");
$(function () {
var elToRenderInto = document.getElementById("#render-here");
$("#fetch-and-render-button").on("click", function () {
$.get("/some/data", "json").then(function (data) {
renderer.render(data, elToRenderTo);
});
});
});
});

There are many frameworks for this kind of thing.
I'm using Backbone at the moment http://documentcloud.github.com/backbone/
Friends have also recommended knockout.js http://knockoutjs.com/
Both of these use an MVC pattern to update views once data has been loaded by a model
[update] I think at their most basic level these libraries are using callback functions and event listeners to update the various parts of the page.
e.g.
model1.loadData = function(){
$.get('http://example.com/model1', function(response){
this.save(response);
this.emit('change');
});
}
model1.bind('change',view1.update);
model1.bind('change',view2.update);

I've used pxLoader, a JavaScript Preloader, which works pretty well. It uses 100ms polling by default.
I wouldn't bother reinventing the wheel here unless you need something really custom, so give that (or any JavaScript preloader library really) a look.

sequential functions javascript

I'm having major trouble getting my app to behave while loading. I don't think it is very fair to users to allow them access to the app until it's finished loading and is ready / responsive.
Please note: every thing else works fine in my app I just can't get functions to run in order.
function LOADINGapp(){
//app loads but allows user to enter before loading is finished
$.when(getToday()).then(reorderdivs()).then(getSomething()).then(setupSomethingElse()).then(loadAnotherSomething()).then(INITapp());
//app stops dead at getToday but (Crome javascript console) no errors
getToday(function(){reorderdivs(function(){getSomething(function(){setupSomethingElse(function(){loadAnotherSomething(function(){INITapp();});});});});});
//app loads but allows user to enter before loading is finished
getToday(reorderdivs(),getSomething(),setupSomethingElse(),loadAnotherSomething(),INITapp());
//getToday();
//reorderdivs();
//getSomething();
//setupSomethingElse();
//loadAnotherSomething();
//INITapp();
}
function INITapp(){
$('#SPLASH').hide();
}
Can someone please assist me, I don't understand. Done and doing a tone of research to get this to behave.
Thanks

I haven't used when/then from jquery before, but it looks like you need to pass the pointers to the methods. Right now you are actually executing them all at once. Try:
$.when(getToday())
.then(reorderdivs)
.then(getSomething)
.then(setupSomethingElse)
.then(loadAnotherSomething)
.then(INITapp);
(Notice the missing parentheses)
Also, be aware the only true asynchronous object in JS is an AJAX call. setTimeout is NOT asynchronous, it is a queueing method. Read John Resig's post:
http://ejohn.org/blog/how-javascript-timers-work/

I solved it by jumping (only allowing my LOADINGapp function to do one thing at a time) so it works in stages. once a stage (a function) is complete it jumps back into the LOADINGapp function.
var L=0;
function LOADINGapp(){
if(L==0){getToday();}
else if(L==1){reorderdivs();}
else if(L==2){getSomething();}
else if(L==3){setupSomethingElse();}
else if(L==4){loadAnotherSomething();}
else if(L==5){INITapp();};
}
function INITapp(){
$('#SPLASH').hide();
}
example:
function getSomething(){
//app suff
//app suff
//app suff
L = 3;
LOADINGapp();
}
the only thing I will improve is just do L++; at the end of LOADINGapp as it saves writing it all over the place. PS:thanks for the efforts guys

It is probably something to do with asynchronous functions. So you could try making those functions with a callback, as mentioned here: http://pietschsoft.com/post/2008/02/JavaScript-Function-Tips-and-Tricks.aspx.

Threads (or something like) in javascript

I need to let a piece of code always run independently of other code. Is there a way of creating a thread in javascript to run this function?
--why setTimeout doesn't worked for me
I tried it, but it runs just a single time. And if I call the function recursively it throws the error "too much recursion" after some time. I need it running every 100 milis (it's a communication with a embedded system).
--as you ask, here goes some code
function update(v2) {
// I removed the use of v2 here for simplicity
dump("update\n"); // this will just print the string
setTimeout(new function() { update(v2); }, 100); // this try doesn't work
}
update(this.v);
It throws "too much recursion".

I am assuming you are asking about executing a function on a different thread. However, Javascript does not support multithreading.
See: Why doesn't JavaScript support multithreading?
The Javascript engine in all current browsers execute on a single thread. As stated in the post above, running functions on a different thread would lead to concurrency issues. For example, two functions modifying a single HTML element simultaneously.

As pointed out by others here, perhaps multi-threading is not what you actually need for your situation. setInterval might be adequate.
However, if you truly need multi-threading, JavaScript does support it through the web workers functionality. Basically, the main JavaScript thread can interact with the other threads (workers) only through events and message passing (strings, essentially). Workers do not have access to the DOM. This avoids any of the concurrency issues.
Here is the web workers spec: http://www.whatwg.org/specs/web-workers/current-work/
A more tutorial treatment: http://ejohn.org/blog/web-workers/

Get rid of the new keyword for the function you're passing to setTimeout(), and it should work.
function update(v2) {
try {
dump("update\n");
} catch(err) {
dump("Fail to update " + err + "\n");
}
setTimeout(function() { update(v2); }, 100);
}
update(this.v);
Or just use setInterval().
function update(v2) {
try {
dump("update\n");
} catch(err) {
dump("Fail to update " + err + "\n");
}
}
var this_v = this.v;
setInterval(function() {update(this_v);}, 100);
EDIT: Referenced this.v in a variable since I don't know what the value of this is in your application.

window.setTimeout() is what you need.

maybe you should to view about the javascirpt Workers (dedicated Web Workers provide a simple means for web content to run scripts in background threads), here a nice article, which explain how this works and how can we to use it.
HTML5 web mobile tutororial

U can try a loop instead of recursivity