Develop Reference

eventemitter.emit method doesn't return in node js

I have a problem using eventemitter.emit method.
Basically this is what I want to do. I have a long running process (CPU bounded) that generates output objects, and since this is CPU bounded process i run it as a separate process using fork().
class Producer extends EventEmitter {
constructor() {
this.on('MyEvent', this.produce);
}
produce(input) {
var output = longRunningProcess();
this.emit('MyEvent, output);
process.send(output);
}
}
var producer = new Producer();
producer.emit('MyEvent', 0); // To kick off the execution
And once each output is generated, I want to send it to the parent process. And also use it to emit an event to produce another object and so on.
Now, the problem is that the process.send(output) doesn't seem to be executed. I can see the outputs being printed in the console one after one. But the parent doesn't seem to be receiving anything from the child process. In my understanding, nodejs event loop shouldn't pick up a new task until it finishes the current one and the stack is empty, but this is not the case here.
So can you guys help me with this?
Edit: Parent process code
this.producer = ChildProcess.fork('.path/to/produer.js'silent: true });
this.producer.on('message', (data) => {
this.miningProcess.send({ type: "StopMining", body: 0 });
});

It looks to me like you may be starving the event loop (never giving it any cycles to processing incoming events) which can wreck the ability to process networking, even outbound networking. I'd suggest that you start the next iteration only after the process.send() has completed.
class Producer extends EventEmitter {
constructor() {
this.on('MyEvent', this.produce.bind(this));
}
produce(input) {
let output = longRunningProcess();
process.send(output, () => {
// When the send finishes, start the next iteration
// This should allow the node.js event queue to process things
this.emit('MyEvent, output);
});
}
}
var producer = new Producer();
producer.emit('MyEvent', 0); // To kick off the execution
Other comments of note:
You need this.produce.bind(this) on your event handler instead of just this.produce to make sure the right this value is set when that function is called.
Keep in mind that eventEmitter.emit() is synchronous. It does not allow the event queue to process events and eventEmitter events do not go through the event queue.
This code assumes that the process.send() callback is called asynchronously and gives the event loop enough chances to process any events that are waiting. It also makes sure the interprocess message is completely sent before you start the next CPU intensive iteration which will temporarily block the event queue processing again. This way, you are sure the whole communication is done before blocking the event queue again.
You probably could have made things work with an appropriately places setTimeout() to kick off the next iteration, but I think it's more reliable to make sure the interprocess messaging is done before kicking off the next iteration.
FYI, if you're not using the EventEmitter you derive from for anything other than is shown here, then it isn't really needed. You could just call methods on your object directly rather than using EventEmitter events.

NodeJS function getting interrupted by socketio event

I'm seeing some strange behavior in my nodejs game server in which there appears to be concurrency. This is strange because Nodejs is supposed to run in one thread as it doesn't use any concurrency. The problem is that I have an update function that's repeatedly called using setImmediate(). In this function I am using an array in two places. However, this same array is also modified when the "disconnect" event fires (which is when the client disconnects from the server). So it so happens that when the timing aligns so that the disconnect event fires AFTER the first place in which the array is accessed in the update function but BEFORE the second place, the array is modified and so the server crashes when the array is attempted to be accessed in the second place.
Here's some code that might make this picture clear:
function update(){
for(var i = 0; i < gameWorlds.length; i++){
gameWorlds[i].update();
console.log("GAMEWORLDS LENGTH BEFORE: " + gameWorlds.length);
NetworkManager.sendToClient(gameWorlds[i].id, "gameupdate", gameWorlds[i].getState());
console.log("GAMEWORLDS LENGTH AFTER: " + gameWorlds.length);
gameWorlds[i].clearGameState();
}
}
setImmediate(update);
//in the NetworkManager module, the disconnect event handler:
socket.on("disconnect", function(){
for(var a = 0; a < sockets.length; a++){
if(sockets[a].id === socket.id){
sockets.splice(a, 1);
}
}
listenerFunction("disconnect", socket.id);
console.log("Client " + socket.id + " DISCONNECTED!");
});
//also in the NetworkManager module, the sendToClient function:
function sendToClient(clientId, messageName, data){
for(var i = 0; i < sockets.length; i++){
if(sockets[i].id === clientId){
sockets[i].emit(messageName, data);
}
}
}
//in the main module (the same one as the update function), the listener
//function that's called in the disconnect event handler:
function networkEventsListener(eventType, eventObject){
if(eventType === "disconnect"){
for(var i = 0; i < gameWorlds.length; i++){
if(gameWorlds[i].id === eventObject){
gameWorlds.splice(i, 1);
console.log("GAME WORLD DELETED");
}
}
}
}
Now, I have a socketio event listener set up for when the client disconnects in which an element in the array is deleted. When this event occurs RIGHT in between the first and second places the array is accessed (as shown above), my server crashes. Either threads are being used or my function is stopped to let the event handler execute and then my function is resumed. Either way, I don't want this to be happening. Thank you!
EDIT 1: I edited the code to incorporate the console logs I have in my code. The reason why I am saying my loop is getting interrupted is because of the fact that the second console log outputs a length of 0 while the first console log outputs it greater than 0. Also, there is another console log in the disconnect event handler which FIRES in between the two console logs in my update function. This means that my function is getting interrupted.
EDIT 2: Thank you for all your replies I really appreciate it. I think there's been some confusion regarding:
1. The fact that no one has acknowledged how the console logs are appearing. In my previous edit, I changed the code to reflect how I am logging to see the problem. The issue is that in the disconnect event handler, I have a console log which is happening in between the two console logs in the loop. I.e. the disconnect event handler executes BEFORE the second console log is reached in the loop. Unless I am confused about the implementation of the console log function, the logs should be happening in the correct order (that is that the two console logs in the loop should always occur before any other console log in the rest of the program due to the ASYNC nature as most of you have stated.) But this is not the case, which leads me to believe something strange is happening.
2. None of the code inside the loop is changing the array. In a lot of your replies, you assume that there is code which actually modifies the array INSIDE the loop, which is not the case. The only code that modifies the array is code OUTISDE of the loop, which is why it's very strange that the first part of the loop in which the array is accessed doesn't crash but the second part does, even though the code in between DOESN'T change the array.
EDIT 3: Ok so a lot of the replies have been asking for the COMPLETE code. I have update the code with all the relevant REAL code.

Javascript in node.js is single threaded. A given thread of execution in Javascript will NOT be interrupted by a socket.io disconnect event. That physically can't happen. node.js is event driven. When the disconnect event happens, an event will be put into the Javascript event queue and ONLY when your current thread of execution is done will Javascript grab the next event out of the event queue and call the callback associated with it.
You don't show enough of your real code to know for sure, but what could be happening is if you have asynchronous operations, then when you start an async operation and register a callback for its completion, then you are finishing that Javascript thread of execution and it is merely a race to see which async event happens next (the completion of this specific async operation or the disconnect event from the socket.io disconnect). That is indeterminate and those events can happen in any order. So, if you have async code in the code in question, then the disconnect event can get processed while that code is waiting for a completion of an async event.
That is the type of race conditions that you have to be aware of in node.js programming. Anytime your logic goes asynchronous, then other things can get processed in node.js while your code is waiting for the asynchronous callback that signals the operation is complete.
What exactly to do about this depends entirely upon the exact situation and we would need to see and understand your real code (not pseudo code) to know which option to best recommend to you. FYI, this is one of the reasons we can always help you better if you show us your real code, not just pseudo code.
Here are some of the techniques that can be used when you are operating with async operations on a shared data structure that could be changed by other async code:
Make a copy of the data you want to process so no other code has access to your copy so it can't be modified by any other code. This might be making a copy of an array or it might be just using a closure to capture an index locally so the index can't be impacted by other code.
Use a flag to protect a data structure that is in the middle of being modified and train all other code to respect that flag. How exactly to do this depends upon the specific data. I have code in a Raspberry Pi node.js app that regularly saves data to disk and is subject to a race condition where other event driven code may want to update that data while I'm in the middle of using async I/O to write it to disk. Because the data is potentially large and the memory of the system not so large, I can't make a copy of the data as suggested in the first point. So, I used a flag to indicate that I'm in the middle of writing the data to disk and any code that wishes to modify the data while this flag is set, adds its operations to a queue rather than directly modifies the data. Then, when I'm done writing the data to disk, the code checks the queue to see if any pending operations need to be carried out to modify the data. And, since the data is represented by an object and all operations on the data are carried out by methods on the object, this is all made transparent to the code using the data or trying to modify the data.
Put the data in an actual database that has concurrency features and controls built into it so that it can make atomic changes to the data or data can be locked for brief periods of time or data can be fetched or updated in a safe way. Databases have lots of possible strategies for dealing with this since it happens with them a lot.
Make all accesses to the data be asynchronous so if some other async operation is in the middle of modifying the data, then other unsafe attempts to access the data can "block" until the original operation is done. This is one technique that databases use. You do, of course, have to watch out for deadlocks or for error paths where the flags or locks aren't cleared.
Some new comments based on your posting of more code:
This code is just wrong:
//in the main module (the same one as the update function), the listener
//function that's called in the disconnect event handler:
function networkEventsListener(eventType, eventObject){
if(eventType === "disconnect"){
for(var i = 0; i < gameWorlds.length; i++){
if(gameWorlds[i].id === eventObject){
gameWorlds.splice(i, 1);
console.log("GAME WORLD DELETED");
}
}
}
}
When you call .splice() in the middle of a for loop on the array you are iterating, it causes you to miss an item in the array you are iterating. I don't know if this has anything to do with your issue, but it is wrong. One simple way to avoid this issue it to iterate the array backwards. Then calling .splice() will not influence the position of any of the array elements that you have not yet iterated and you won't miss anything in the array.
Same issue in the for loop in your disconnect handler. If you only ever expect one array element to match in your iteration, then you can break right after the splice() and this will avoid this issue and you won't have to iterate backwards.

Two things I think you should change to fix the problem.
1) don't modify the length of the array when disconnect occurs but instead make a value that is falsey. A boolean or a one and zero scenario
2) add logic in the form of an if statement to check if the value is falsey for player two. That way you'll know they disconnected and don't deserve to have anything because they're lame and couldn't watch the loser screen.
That should fix the issue and you can. Decide what to do if they're to lazy to stay and watch the winning losing ceremony of your game.
var gameWorld = [ ];
function update(){ // some code } is async and is pushed to the event loop.
function disconnect(){ // some code } is also async and gets pushed to the event loop.
Even though update() is running on the call stack it's waiting for the event loop and it doesn't mean that it'll complete it's execution before the next tick occurs. gameWorld is outside both scopes it can be modified in the middle of update(). So when update() tries to access the array again it's different then when it started.
disconnect() is called before update() finishes and modifies the array on the event loop nexttick() thus by the time the code for update() gets to second player bam the array is messed up.

Even if you have an event listener, execution should not just stop mid function. When the event occurs, node will push the event callback on to the stack. Then when node finishes executing the current function it will start processing the other requests on the stack. You can't be sure of the order things will execute, but you can be sure that things will not get interrupted mid execution.
If your doWhatever function is async then the problem may be occurring because when node finally gets around to servicing the requests on the stack the loop has already finished, therefore everytime doWhatever is called it is being called with the same index (whatever its last value was.)
If you want to call async functions from a loop then you should wrap them in a function to preserve the arguments.
e.g.
function doWhateverWrapper(index){
theArray[index].doWhatever();
}
function update(){
for(var i = 0; i < theArray.length; i++){
//first place the array is accessed
doWhateverWrapper(i);
....more code.....
//second place the array is accessed
doWhateverWrapper(i);
}
}
setImmediate(update);

Why isn't this code executing synchronously?

I was under the impression that all DOM manipulations were synchronous.
However, this code is not running as I expect it to.
RecordManager.prototype._instantiateNewRecord = function(node) {
this.beginLoad();
var new_record = new Record(node.data.fields, this);
this.endLoad();
};
RecordManager.prototype.beginLoad = function() {
$(this.loader).removeClass('hidden');
};
RecordManager.prototype.endLoad = function() {
$(this.loader).addClass('hidden');
};
The Record constructor function is very large and it involves instantiating a whole bunch of Field objects, each of which instantiates some other objects of their own.
This results in a 1-2 second delay and I want to have a loading icon during this delay, so it doesn't just look like the page froze.
I expect the flow of events to be:
show loading icon
perform record instantiation operation
hide loading icon
Except the flow ends up being:
perform record instantiation operation
show loading icon
hide loading icon
So, you never even see the loading icon at all, I only know its loading briefly because the updates in the chrome development tools DOM viewer lag behind a little bit.
Should I be expecting this behavior from my code? If so, why?

Yes, this is to be expected. Although the DOM may have updated, until the browser has a chance to repaint, you won't see it. The repaint will get queued the same way as all other things get queued in the browser (ie it won't happen until the current block of JavaScript has finished executing), though pausing in a debugger will generally allow it to happen.
In your case, you can fix it using setTimeout with an immediate timeout:
RecordManager.prototype._instantiateNewRecord = function(node) {
this.beginLoad();
setTimeout(function() {
var new_record = new Record(node.data.fields, this);
this.endLoad();
}, 0);
};
This will allow the repaint to happen before executing the next part of your code.

JavaScript is always synchronous. It mimics multi-threaded behavior when it comes to ajax calls and timers, but when the callback gets returned, it will be blocking as usual.
That said, you most likely have a setTimeout in that constructor somewhere (or a method you're using does). Even if it's setTimeout(fnc, 0).

Is it possible to show an element just before entering a long running sync process?

This is a very simple use case. Show an element (a loader), run some heavy calculations that eat up the thread and hide the loader when done. I am unable to get the loader to actually show up prior to starting the long running process. It ends up showing and hiding after the long running process. Is adding css classes an async process?
See my jsbin here:
http://jsbin.com/voreximapewo/12/edit?html,css,js,output

To explain what a few others have pointed out: This is due to how the browser queues the things that it needs to do (i.e. run JS, respond to UI events, update/repaint how the page looks etc.). When a JS function runs, it prevents all those other things from happening until the function returns.
Take for example:
function work() {
var arr = [];
for (var i = 0; i < 10000; i++) {
arr.push(i);
arr.join(',');
}
document.getElementsByTagName('div')[0].innerHTML = "done";
}
document.getElementsByTagName('button')[0].onclick = function() {
document.getElementsByTagName('div')[0].innerHTML = "thinking...";
work();
};
(http://jsfiddle.net/7bpzuLmp/)
Clicking the button here will change the innerHTML of the div, and then call work, which should take a second or two. And although the div's innerHTML has changed, the browser doesn't have chance to update how the actual page looks until the event handler has returned, which means waiting for work to finish. But by that time, the div's innerHTML has changed again, so that when the browser does get chance to repaint the page, it simply displays 'done' without displaying 'thinking...' at all.
We can, however, do this:
document.getElementsByTagName('button')[0].onclick = function() {
document.getElementsByTagName('div')[0].innerHTML = "thinking...";
setTimeout(work, 1);
};
(http://jsfiddle.net/7bpzuLmp/1/)
setTimeout works by putting a call to a given function at the back of the browser's queue after the given time has elapsed. The fact that it's placed at the back of the queue means that it'll be called after the browser has repainted the page (since the previous HTML changing statement would've queued up a repaint before setTimeout added work to the queue), and therefore the browser has had chance to display 'thinking...' before starting the time consuming work.
So, basically, use setTimeout.

let the current frame render and start the process after setTimeout(1).
alternatively you could query a property and force a repaint like this: element.clientWidth.

More as a what is possible answer you can make your calculations on a new thread using HTML5 Web Workers
This will not only make your loading icon appear but also keep it loading.
More info about web workers : http://www.html5rocks.com/en/tutorials/workers/basics/

javascript, while loop

i'm trying to get my script to wait for user input (click of a button) before continuing, this is v feasible in other languages, but seems impossible in js. basically, i want the user to select an option within a given time frame, if the user selects the wrong option, they're told..script then conts...otherwise, if after a certain amount of time theres no response...script just continues again sowing them the correct ans, but there seems to be nothing in js to make the script wait for that user input! ive tried a while loop, but that is just a big no no in js, ive used settimeout but has no real effect because the script just continues like normal then performs an action after x amount of time, ive tried setting variables and letting the script cont only if it is of a particular value, which is set only if the user clicks...eg var proceed=false, this is only set to true if the user clicks a button, but it still doesn't work... ive tried sooo many other solutions but nothing actually seems to be working. i like the idea of a while loop, because it doeas exactly what i want it to so, but if completly freezes my browser, is there a more effecient type of loop that will will peroform in the same manner with crashing my browser?
heres my code below that compltely freezes my computer. this method is called within a for loop which calls another method after it.
function getUserResp(){
$("#countdown").countdown({seconds: 15});
setTimeout("proceed=true", 16000);
$("#ans1").click(function(){
ansStr=$(this).text();
checkAns(ansStr);
});
$("#ans2").click(function(){
ansStr=$(this).text();
checkAns(ansStr);
});
$("#ans3").click(function(){
ansStr=$(this).text();
checkAns(ansStr);
});
would like something like this.....or just some sort of loop to make the script wait before going ahead so at least it gives the user some time to respond rather than running straight though!
do{
$(".ans").mouseover(function(){
$(this).addClass("hilite").fadeIn(800);
});
$(".ans").mouseout(function(){
$(this).removeClass("hilite");
});
}while(proceed==false);
}

You're doing it wrong.
JavaScript in the browser uses an event-driven model. There's no main function, just callbacks that are called when an event happens (such as document ready or anchor clicked). If you want something to happen after a user clicks something, then put a listener on that thing.
What you've done just keeps adding an event listener every time round the loop.
If you want to wait for user input then just don't do anything - the browser waits for user input (it's got an internal event loop). The worst thing you can do is try to reimplement your own event loop on top of the browser's.
You need to learn JavaScript. Trying to write JavaScript like you would another language only leads to pain and suffering. Seriously.
Douglas Crockford said it best:
JavaScript is a language that most people don’t bother to learn before they use. You can’t do that with any other language, and you shouldn’t want to, and you shouldn’t do that with this language either. Programming is a serious business, and you should have good knowledge about what you’re doing, but most people feel that they ought to be able to program in this language without any knowledge at all, and it still works. It’s because the language has enormous expressive power, and that’s not by accident.

You can't block the Javascript from running in the same way that you can in some other imperative languages. There's only one thread for Javascript in the browser, so if you hang it in a loop, nothing else can happen.
You must use asynchronous, event-driven programming. Setting a click handler (or whatever) combined with a timeout is the right way to start. Start a 15 second setTimeout. Inside the click handler for the answers, cancel the timeout. This way the timeout's handler only happens if the user doesn't click an answer.
For example:
var mytimeout = setTimeout(15000, function() {
// This is an anonymous function that will be called when the timer goes off.
alert("You didn't answer in time.");
// Remove the answer so the user can't click it anymore, etc...
$('#ans').hide();
});
$('#ans').click(function() {
// Clear the timeout, so it will never fire the function above.
clearTimeout(mytimeout);
alert("You picked an answer!");
});
See how the code must be structured such that it's event-driven. There's no way to structure it to say "do this thing, and wait here for an answer."

You're looking at client-side javascript as if it wasn't already in an event-driven loop. All you need to do is wait for the appropriate event to happen, and if it hasn't happened yet, continue to wait, or else perform some default action.
You don't need to:
create main loop: // All
wait for user input // Of
timer = start_timer() // This
// Is done for you
if [user has input data]:
process_data()
else if [timer > allowed_time]:
process_no_data()
else:
wait() // By the Browser
You only need the middle part. All you need to do is (Actual javascript follows, not pseudo-code):
// First, store all of the answer sections,
// so you're not grabbing them every time
// you need to check them.
var answers = {};
answers.ans1 = $("#ans1");
answers.ans2 = $("#ans2");
answers.ans3 = $("#ans3");
// This is a flag. We'll use it to check whether we:
// A. Have waited for 16 seconds
// B. Have correct user input
var clear_to_proceed = false;
var timer_id;
// Now we need to set up a function to check the answers.
function check_answers() {
if ( ! clear_to_proceed ) {
clear_to_proceed = checkAns(answers.ans1.text());
clear_to_proceed = checkAns(answers.ans2.text());
clear_to_proceed = checkAns(answers.ans3.text());
// I assume checkAns returns
// true if the answer is correct
// and false if it is wrong
}
if ( clear_to_proceed ) {
clearTimeout(timer_id);
return true; // Or do whatever needs be done,
// as the client has answered correctly
} else {
// If we haven't set a timer yet, set one
if ( typeof timer_id === 'undefined' ) {
timer_id = setTimeout(function(){
// After 16 seconds have passed we'll check their
// answers one more time and then force the default.
check_answers();
clear_to_proceed = true;
check_answers();
}, 16000);
}
return false; // We're just waiting for now.
}
}
// Finally, we check the answers any time the user interact
// with the answer elements.
$("#ans1,#ans2,#ans3").bind("focus blur", function() {
check_answers();
});