In the browser, we can create workers from a javascript string as follows:
var blob = new Blob([sourceString]);
var url = window.URL.createObjectURL(blob);
var worker = new Worker(url);
Is there any way to do this with node's child process? I have a single JavaScript file I and want to create workers that are coded dynamically.
The source string is a created dynamically at run time.
The closest answer I found was this one, but it requires a seperate file.
If I understood you right, I created a module which does just that yesterday.
It was not intended to create workers from strings but from actual functions even, because the actual function code must be passed thought message, they are stringified to be rebuilt (thought eval()) inside the worker.
And this is done thought the code:
var source = fn.toString();
...so, having that string prototype also has a .toString() method, passing the function as string must work too (and in fact works. I just tested it).
It may not be what you want: If you need to pass in and out messages from and to the worker, this module is not for you. But you can see the code and modify it to fit your needings.
On the other hand, if you only want to execute some function in background and get the result it is much simpler than dealing with worker plumbings because you can pass-in parameters to the function and get the result just as a simple function call.
Example:
// Reauires funwork (`npm install --save funwork`)
var funwork = require("funwork");
var workerfn = funwork(function_src_string); // or actual function.
It has the drawback that the function must be evaluated though eval() but, in your case, (having a string source) I think this is anyway a must.
EDIT: Here is a modified version of funwork to approach what you want as we discussed in comments:
var Worker = require('webworker-threads').Worker;
var Deasync = require('deasync');
function strWorker(fn){
var source = fn.toString();
return function() {
var done = false;
var args = Array.prototype.slice.call(arguments);
var error;
// Create worker://{{{
var worker = new Worker(function(){
var fn;
var me = this;
// Wait for function source and arguments:
me.onmessage = function(event) {
switch (event.data.oper) {
case "src":
// "Compile" function thougt source evaluation.
try {
eval ("fn = " + event.data.msg + ";");
postMessage(['ready']);
} catch (e) {
postMessage(['error', "Error trying to evaluate function source"]);
};
break;
case "args":
// Call the function with given arguments and reset the rest of worker stuff.
try {
// Reset worker (inside) event handler:
delete me.onmessage;
// Notify that worker is ready:
postMessage(["ok"]);
// Start function execution:
fn.apply(me, event.data.msg);
} catch (e) {
postMessage(['error', e]);
};
break;
};
};
});//}}}
// Event handling://{{{
worker.onmessage = function(event) {
switch (event.data[0]) {
case 'error':
worker.postMessage({oper: "end"});
done = true;
error = event.data[1];
break;
case 'ready':
worker.postMessage({oper: "args", msg: args});
break;
case 'ok':
done = true;
break;
};
};//}}}
// Send function source to worker:
worker.postMessage({oper: "src", msg: source});
// Wait (without blocking) until worker executed passed function:
Deasync.loopWhile(function(){return !done;});
if (error) throw error;
// Reset worker (outside) event handler:
delete worker.onmessage;
return worker;
};
};
module.exports = strWorker;
I kept the ability of passing arguments to the function because it is already implemented and you can simply don't use it if you doesn't need to pass anything.
The usage is the same with the only difference that the generated function returns a running worker instead of a function return value.
Used event handlers (inside and outside the worker ) are deleted prior to function (passed in as string) execution and worker returning, respectively, to avoid any side effect and the execution context ("this") of the passed-in function is also set to the actual worker "parent" function. .
If you want a single file js to spin up different processes creating a cluster might be a solution. Here is a pretty good tutorial: Tutorial
Basically node come with native cluster module
var cluster = require('cluster');
You can tell if the process is a master or a worker by cluster.isMaster. If the process is the master process you can spin up workers by doing cluster.fork()
if (cluster.isMaster) {
for (var i = 0; i < numCPUs; i++) {
cluster.fork();
}
} else {
http.createServer(function(req, res) {
res.writeHead(200);
res.end('process ' + process.pid + ' says hello!');
}).listen(8000);
}
Hope this helps.
As for child process, you can child_process.fork(modulePath[, args][, options]) to run other modules and pass in arguments. The module can do different things according to the argument so it is dynamic... Seems you just want dynamic behavior based on the input and child_process can do it if you can make put the code in a different file. If you can only have one, try the cluster solution.
Related
How can I define a Meteor method which is also callable in a template helper?
I have these two files:
file: lib/test.js
Meteor.methods({
viewTest : function (str) {
return str;
}
});
file: client/myView.js
Template.helloWorld.helpers({
txt : function () {
var str = Meteor.call('viewTest', 'Hello World.');
return str;
}
});
When I give "str" a normal string everything works fine. But in this case my template does not get any value. I defined - for the test - in the same file where the method is a normal function and tried to call the function. The error I got was that the function does not exist. So I think that Meteor tries to render the template before it knows anything about the methods I defined for it. But I think that this is a bit unusual - isn't it?
There is now a new way to do this (Meteor 0.9.3.1) which doesn't pollute the Session namespace
Template.helloWorld.helpers({
txt: function () {
return Template.instance().myAsyncValue.get();
}
});
Template.helloWorld.created = function (){
var self = this;
self.myAsyncValue = new ReactiveVar("Waiting for response from server...");
Meteor.call('getAsyncValue', function (err, asyncValue) {
if (err)
console.log(err);
else
self.myAsyncValue.set(asyncValue);
});
}
In the 'created' callback, you create a new instance of a ReactiveVariable (see docs) and attach it to the template instance.
You then call your method and when the callback fires, you attach the returned value to the reactive variable.
You can then set up your helper to return the value of the reactive variable (which is attached to the template instance now), and it will rerun when the method returns.
But note you'll have to add the reactive-var package for it to work
$ meteor add reactive-var
Sashko added a neat little package called meteor-reactive-method to solve this problem.
$ meteor add simple:reactive-method
Template.helloWorld.helpers({
txt: function() {
return ReactiveMethod.call('viewTest', 'Hello World.');
}
});
As I point out in common mistakes, helpers should be side-effect free, so I'd use this technique with caution. However, it's a really handy shortcut for cases where:
The helper should fire only once (it doesn't depend on reactive state).
The invoked method doesn't mutate the database.
You need to interface your return value with a Session variable as the request is asynchronous:
Template.helloWorld.helpers({
txt : function () {
return Session.get("txt") || "Loading";
}
});
Template.helloWorld.created = function() {
Meteor.call('viewTest', 'Hello World.', function(err, result) {
Session.set("txt", result);
});
}
So .rendered should be called once when your template loads (at least it should with the newer version of Meteor.)
The value would be called and displayed. Otherwise it would say "Loading".
Methods on the client side are asynchronous, and their return value is always undefined. To get the actual value returned by the method, you need to provide a callback:
Meteor.call('method', 'argument', function(error, result) {
....
});
Now, there's no easy way to use the result in your helper. However, you can store it in your template as a data object and then return it in the helper:
Template.template.created = function() {
var self = this;
self.data.elephantDep = new Deps.Dependency();
self.data.elephant = '';
Meteor.call('getElephant', 'greenOne', function(error, result) {
self.data.elephant = result;
self.data.elephantDep.changed();
});
};
Template.template.showElephant = function() {
this.elephantDep.depend();
return this.elephant;
};
This is expected behavior. You are not using methods as they are intended.
Your code defines a server method viewTest and a corresponding method stub on the client with the same name.
Meteor.call('viewTest', 'Hello World.'); remotely calls viewTest on the server and in parallel runs the stub on the client.
Regarding the return value of the stub please see the documentation here, in particular:
On the client, the return value of a stub is ignored. Stubs are run
for their side-effects: they are intended to simulate the result of
what the server's method will do, but without waiting for the round
trip delay.
Regarding the return value of the server method please see the documentation here, in particular:
On the client, if you do not pass a callback and you are not inside a
stub, call will return undefined, and you will have no way to get the
return value of the method. That is because the client doesn't have
fibers, so there is not actually any way it can block on the remote
execution of a method.
There is a fine little package for this by #msavin:
https://atmospherejs.com/msavin/fetcher
I'm trying to use the NodeJS module "pcsc-lite" to communicate with a card reader. If you want to take a look at the module : https://github.com/santigimeno/node-pcsclite.
I'm looking for a way to send a sequence of data to my reader using my own method. Because, the module is event-based. So I have to declare two listeners (one in the other) to be able to call the send method.
For example :
module.on("reader", function(reader){
//...
reader.on("status", function(status){
//...
reader.connect({ share_mode : this.SCARD_SHARE_SHARED },function(err, protocol) {
//This is the method I want to be able to call "when I need it"
reader.transmit(...);
});
});
});
I would like to call the transmit method like this for example :
function send(...){
reader.transmit(...);
}
I think there is a way to do it, but I seem to be a little bit hooked to my C/Java programming habits.
Thanks in advance.
If your reader will be a singleton, you can declare it outside the callback, and then assign the variable when you're ready. Without knowing more, here's a simple example:
let reader; // we prepare a variable that's outside of scope of it all.
// your `send` function
function send(params) {
let stuff = doStuffWithParams(params);
reader.transmit(stuff, callback);
}
// we take out init stuff too
function initialize() {
// we know reader variable is already initialized.
reader.on('status', function() {
reader.connect({
share_mode : this.SCARD_SHARE_SHARED
},function(err, protocol) {
// send.
send();
// or even better, emit some event or call some callback from here, to let somebody outside this module know you're ready, then they can call your `send` method.
});
});
}
// now your module init section
let pcsc = require('pcsclite')();
pcsc.on('reader', function(r) {
// assign it to our global reader
reader = r;
initialize();
});
Note: don't call your variables module, it's refering to the file being currently executed and you can get unexpected behavior.
normally we might create a simple singleton object like so with Node.js:
var foo = {};
module.exports = foo;
or
function Foo(){}
module.exports = new Foo();
however
what is the best way to make a clean singleton module that needs an external variable for initialization? I end up making something like this:
var value = null;
function init(val){
if(value === null){
value = val;
}
return value;
}
module.exports = init;
this way someone using the module can pass in an initializing value for a certain variable. Another way to do it would be like so:
function Baz(value){
this.value = value;
}
var instance = null;
module.exports = function init(value){
if(instance === null){
instance = new Baz(value);
}
return instance;
}
there's two problems I encounter with this:
(1) this is minor, but the semantics is wrong. We can rename init to getInstance, but we can't make the same function literal mean "initialize and get" since they are different meanings. So we have to have a function that does two different things. Create an instance and retrieve and instance. I don't like this especially since in some cases we need to make sure the argument to initialize the instance is not null. With multiple developers using a module it's not clear if a module has been initialized yet, and if they pass in undefined into the module that hasn't been initialized, that could become a problem or just confusing at the least.
(2) this is more important - in some cases initializing Baz is asynchronous. For example, making a Redis connection or reading from a file to initialize a singleton, or making a socket.io connection. This is what really trips me up.
e.g. here is a module that I have that I consider really ugly that stores a socket.io connection:
var io = null;
var init = function ($io) {
if (io === null) {
io = $io;
io.on('connection', function (socket) {
socket.on('disconnect', function () {
});
});
}
return io;
};
module.exports = {
getSocketIOConn: init
};
the above module is initialized like so:
var server = http.createServer(app);
var io = socketio.listen(server);
require('../controllers/socketio.js').getSocketIOConn(io);
So I am looking for a design pattern that allows us to create a singleton module where the initialization process is asynchronous. Ideally we won't have the same function both initializing the instance as well as retrieving it. Does such a thing exist?
I don't think there is necessarily a way to create a pattern that solves this problem but perhaps I am making the mistake of structuring my code in a way that is creating a problem that doesn't need to exist- the problem of initializing a module with a value only once, but using one function to both init the instance and retrieve the instance.
It sounds like you're trying to create a module that gets initialized in one place and then uses some shared resource from that initialization for other users of that module. That is a semi-common need in the real world.
First off, it's ideal if a module can load or create the things that it depends on because that makes it more modular and useful on its own and puts less of a burden on someone using it. So, in your case, if your module could just create/load the thing that it needs when the module is first created and just store that resource in it's own module variable, then that would be the ideal case. But, that is not always possible because the shared resource may be someone else's responsibility to set up and initialize and this module just needs to be made aware of that.
So, the common way to do that is to just use a constructor function for the module. In Javascript, you can allow the constructor to take an optional argument that provides the initialization info. The code responsible for setting up the module would call the constructor with the desired setup parameter. Other users of the module that weren't responsible for setting up the module could just either not call the constructor or if they want a return value or there are other constructor parameters that they should pass, they could pass null for that setup parameter.
For example, you could do this:
var io;
module.exports = function(setup_io) {
if (setup_io) {
io = setup_io;
}
return module.exports;
};
module.exports.method1 = function() {
if (!io) {
throw new Error("Can't use method1 until io is properly initalized");
}
// code here for method1
};
// other methods here
Then, users of the module could either do this:
// load myModule and initialize it with a shared variable
var myModule = require('myModule')(io);
or this:
// load myModule without initializing it
// (assume some other module will initialize it properly)
var myModule = require('myModule');
Note: For developer sanity, it would be useful to have individual methods that require appropriate setup (before they can be used properly) to check to see if the module has been setup when any method is called that needs that setup in order to properly inform a developer that they have called a method before setting up the module properly. Otherwise, errors can happen much further downstream and likely won't have useful error messages.
If you now want the initialization process to be async, that can be done too, but it certainly complicates other uses of the module because they won't necessarily know when/if the module has been initialized.
var moduleData;
var readyList = new EventEmitter();
module.exports = function(arg, callback) {
// do some async operation here involving arg
// when that operation completes, you stored the result
// in local module data and call the callback
readyList.on("ready", callback);
someAsyncOperation(arg, function() {
// set moduleData here
// notify everyone else that the module is now ready
readyList.emit("ready");
// remove all listeners since this is a one-shot event
readyList.removeAllListeners("ready");
});
return module.exports;
};
If you have other users of this module that wish to be notified when it has finished initializing, you can allow them to register a callback themselves to be notified when the module is ready.
// pass a callback to this method that will be called
// async when the module is ready
module.exports.ready = function(fn) {
// if module already ready, then schedule the callback immediately
if (moduleData) {
setImmediate(fn);
} else {
readyList.on("ready", fn);
}
};
If, for reasons I don't quite understand, you want to use the same constructor for both initialization and ready detection, that can be done, though I don't think it's near as clear as just using a separate method for ready detection:
var moduleData;
var readyList = new EventEmitter();
module.exports = function(arg, callback) {
// if both arguments passed, assume this is a request for module
// initialization
if (arguments.length === 2) {
// do some async operation here involving arg
// when that operation completes, you stored the result
// in local module data and call the callback
readyList.on("ready", callback);
someAsyncOperation(arg, function() {
// set moduleData here
// notify everyone else that the module is now ready
readyList.emit("ready");
// remove all listeners since this is a one-shot event
readyList.removeAllListeners("ready");
});
} else {
// constructor called just for a ready request
// arg is the callback
if (moduleData) {
// if module already ready, then schedule the callback immediately
setImmediate(arg);
} else {
// otherwise, save the callback
readyList.on("ready", arg);
}
}
return module.exports;
};
Usage for async initializing the module:
// async initialization form
var myModule = require("myModule")(someArg, function() {
// can use myModule here
});
Usage for loading the module and getting notified when someone else has initialized it:
var myModule = require("myModule")(function() {
// can use myModule here
});
When I start a new instance of GameServer, it sets up the socket and listeners as follows.
var GameServer = function() {
this.player = new Player();
var that = this;
// Setup sockets and listeners
var socket = io.listen(8000);
socket.sockets.on('connection', function(client) {
client.on('message', that.onSocketMessage);
client.on('disconnect', that.onSocketDisconnect);
});
}
I then have two prototypes GameServer.prototype.onSocketMessage & onSocketDisconnect.
I have two problems with the current code:
Using that = this and the closure? function. Looks ugly.
When onSocketMessage is called, the idea is it works out what the message is then calls another function within GameServer. Only this isn't possible as now this belongs to the socket.io system. See below:
...
function onSocketMessage() {
this.player.move();
}
this.player is no longer available as this. is no longer part of GameServer.
Should my socket setup and message passing be handled outside of GameServer function and prototypes?
Or how else could I resolve this?
Cheers
EDIT
Ok so I have tried this and it works but looks pretty ugly I think:
var socket = io.listen(8000);
socket.sockets.on('connection', function(client) {
client.on('message', function (data) {
that.onSocketMessage(this, that, data);
});
client.on('disconnect', function() {
that.onSocketDisconnect(this, that);
});
});
Can it be improved upon?
Two things that may help. Thing the first:
You can modify a function's vision of this using the bind method.
socket.sockets.on('connection', function(client) {
client.on('message', this.onSocketMessage);
client.on('disconnect', this.onSocketDisconnect);
}.bind(this));
Notice the call to bind(this) at the end of the function; this instructs JavaScript to create a closure for you, making whatever this is outside the function, this inside the function. (If you wanted to make this inside the function, say, MySomething, you could just as easily call bind(MySomething), though bind(this) is the most common use).
Thing the second:
You can store data in a Socket.IO socket. So, for example, if one socket is always associated with a player, you can do
socket.set('player', player, function() {
// callback is called when variable is successfully stored
console.log('player has been stored');
});
// and later
socket.get('player', function(err, player) {
// callback is called either with an error set or the value you requested
player.move();
});
The get and set methods take callbacks because the Socket.IO data store can be set to something other than an in-memory store; for example, Redis.
I am trying to create an actions array. Basically these are a list of functions that know the function to be fired next. I've tried lots of ways but can't get them to work reliably. I am doing gui and have pop up frames that callback when they have completed their work (Select file, edit properties etc.)
Example flow:
Task 1: open FileBrowser (iframe browsing control), when selected, execute Task 2
Task 2: get value from FileBrowser, open iframe FileProperties. when complete execute Task 3
Task 3: get FileProperties id, add id to in page control
Each task has to wait for a callback from the previous task. I am open to all ideas how to do this! The only thing is the step are generated server side so I need to avoid hard coding! Below is my attempt so far!
All ideas, gratefully received.
FruitBat
/* action array */
var actions = new Array();
var data = {};
var op = function(fn, next, data) {
this.op = this;
this.fn = fn;
this.next = next;
this.data = data;
};
function one(myop) {
var op1 = myop;
var url = 'poptarget.aspx?cid=21&data=' + escape('[Id]=[2],[cid]=[3]');
var frame = $().popFrame('UniqueId', { caption: 'Page Item', url: url, width: 'auto', modal: true });
frame.bindEvent("cancelled saved", function(e, d) {
console.log('other');
callback();
});
frame.bindEvent("close", function(e, d) {
console.log('close');
op1.next.fn.call(op1.next);
});
};
function two(myop) {
console.log('woohoo');
};
actions.push(new op(one, two, null));
actions.push(new op(two, null, null));
var start = actions[0];
start.fn.call(start);
console.log(data);`
I think you are using .call() incorrectly. See the MDN doco on call. As I understand it:
start.fn.call(start); // sets 'this' to start
will call the function fn and set its this to reference start, which would've happened by default given that you call fn using dot notation from start. Within your function one() you are not using this at all, you are using a declared parameter that you seem to expect to be start. Try changing that line to this:
start.fn(start); // passes start as a parameter
Also within your one() function you reference callback() which doesn't seem to be defined anywhere.
Have you looked at any async/flow control libraries? For example, async provides a pretty comprehensive number of ways to chain functions together.