Develop Reference

setTimeout() Irregular Execution Behaviour [duplicate]

The following example is given in a Node.js book:
var open = false;
setTimeout(function() {
open = true
}, 1000)
while (!open) {
console.log('wait');
}
console.log('open sesame');
Explaining why the while loop blocks execution, the author says:
Node will never execute the timeout callback because the event loop is
stuck on this while loop started on line 7, never giving it a chance
to process the timeout event!
However, the author doesn't explain why this happens in the context of the event loop or what is really going on under the hood.
Can someone elaborate on this? Why does node get stuck? And how would one change the above code, whilst retaining the while control structure so that the event loop is not blocked and the code will behave as one might reasonably expect; wait
will be logged for only 1 second before the setTimeout fires and the process then exits after logging 'open sesame'.
Generic explanations such as the answers to this question about IO and event loops and callbacks do not really help me rationalise this. I'm hoping an answer which directly references the above code will help.

It's fairly simple really. Internally, node.js consists of this type of loop:
Get something from the event queue
Run whatever task is indicated and run it until it returns
When the above task is done, get the next item from the event queue
Run whatever task is indicated and run it until it returns
Rinse, lather, repeat - over and over
If at some point, there is nothing in the event queue, then go to sleep until something is placed in the event queue or until it's time for a timer to fire.
So, if a piece of Javascript is sitting in a while() loop, then that task is not finishing and per the above sequence, nothing new will be picked out of the event queue until that prior task is completely done. So, a very long or forever running while() loop just gums up the works. Because Javascript only runs one task at a time (single threaded for JS execution), if that one task is spinning in a while loop, then nothing else can ever execute.
Here's a simple example that might help explain it:
var done = false;
// set a timer for 1 second from now to set done to true
setTimeout(function() {
done = true;
}, 1000);
// spin wait for the done value to change
while (!done) { /* do nothing */}
console.log("finally, the done value changed!");
Some might logically think that the while loop will spin until the timer fires and then the timer will change the value of done to true and then the while loop will finish and the console.log() at the end will execute. That is NOT what will happen. This will actually be an infinite loop and the console.log() statement will never be executed.
The issue is that once you go into the spin wait in the while() loop, NO other Javascript can execute. So, the timer that wants to change the value of the done variable cannot execute. Thus, the while loop condition can never change and thus it is an infinite loop.
Here's what happens internally inside the JS engine:
done variable initialized to false
setTimeout() schedules a timer event for 1 second from now
The while loop starts spinning
1 second into the while loop spinning, the timer is ready to fire, but it won't be able to actually do anything until the interpreter gets back to the event loop
The while loop keeps spinning because the done variable never changes. Because it continues to spin, the JS engine never finishes this thread of execution and never gets to pull the next item from the event queue or run the pending timer.
node.js is an event driven environment. To solve this problem in a real world application, the done flag would get changed on some future event. So, rather than a spinning while loop, you would register an event handler for some relevant event in the future and do your work there. In the absolute worst case, you could set a recurring timer and "poll" to check the flag ever so often, but in nearly every single case, you can register an event handler for the actual event that will cause the done flag to change and do your work in that. Properly designed code that knows other code wants to know when something has changed may even offer its own event listener and its own notification events that one can register an interest in or even just a simple callback.

This is a great question but I found a fix!
var sleep = require('system-sleep')
var done = false
setTimeout(function() {
done = true
}, 1000)
while (!done) {
sleep(100)
console.log('sleeping')
}
console.log('finally, the done value changed!')
I think it works because system-sleep is not a spin wait.

There is another solution. You can get access to event loop almost every cycle.
let done = false;
setTimeout(() => {
done = true
}, 5);
const eventLoopQueue = () => {
return new Promise(resolve =>
setImmediate(() => {
console.log('event loop');
resolve();
})
);
}
const run = async () => {
while (!done) {
console.log('loop');
await eventLoopQueue();
}
}
run().then(() => console.log('Done'));

Node is a single serial task. There is no parallelism, and its concurrency is IO bound. Think of it like this: Everything is running on a single thread, when you make an IO call that is blocking/synchronous your process halts until the data is returned; however say we have a single thread that instead of waiting on IO(reading disk, grabbing a url, etc) your task continues on to the next task, and after that task is complete it checks that IO. This is basically what node does, its an "event-loop" its polling IO for completion(or progress) on a loop. So when a task does not complete(your loop) the event loop does not progress. To put it simply.

because timer needs to comeback and is waiting loop to finish to add to the queue, so although the timeout is in a separate thread, and may indeed finsihed the timer, but the "task" to set done = true is waiting on that infinite loop to finish

var open = false;
const EventEmitter = require("events");
const eventEmitter = new EventEmitter();
setTimeout(function () {
open = true;
eventEmitter.emit("open_var_changed");
}, 1000);
let wait_interval = setInterval(() => {
console.log("waiting");
}, 100);
eventEmitter.on("open_var_changed", () => {
clearInterval(wait_interval);
console.log("open var changed to ", open);
});
this exemple works and you can do setInterval and check if the open value changed inside it and it will work

Promise is not resolving in Node with infinite while loop? [duplicate]

The following example is given in a Node.js book:
var open = false;
setTimeout(function() {
open = true
}, 1000)
while (!open) {
console.log('wait');
}
console.log('open sesame');
Explaining why the while loop blocks execution, the author says:
Node will never execute the timeout callback because the event loop is
stuck on this while loop started on line 7, never giving it a chance
to process the timeout event!
However, the author doesn't explain why this happens in the context of the event loop or what is really going on under the hood.
Can someone elaborate on this? Why does node get stuck? And how would one change the above code, whilst retaining the while control structure so that the event loop is not blocked and the code will behave as one might reasonably expect; wait
will be logged for only 1 second before the setTimeout fires and the process then exits after logging 'open sesame'.
Generic explanations such as the answers to this question about IO and event loops and callbacks do not really help me rationalise this. I'm hoping an answer which directly references the above code will help.

It's fairly simple really. Internally, node.js consists of this type of loop:
Get something from the event queue
Run whatever task is indicated and run it until it returns
When the above task is done, get the next item from the event queue
Run whatever task is indicated and run it until it returns
Rinse, lather, repeat - over and over
If at some point, there is nothing in the event queue, then go to sleep until something is placed in the event queue or until it's time for a timer to fire.
So, if a piece of Javascript is sitting in a while() loop, then that task is not finishing and per the above sequence, nothing new will be picked out of the event queue until that prior task is completely done. So, a very long or forever running while() loop just gums up the works. Because Javascript only runs one task at a time (single threaded for JS execution), if that one task is spinning in a while loop, then nothing else can ever execute.
Here's a simple example that might help explain it:
var done = false;
// set a timer for 1 second from now to set done to true
setTimeout(function() {
done = true;
}, 1000);
// spin wait for the done value to change
while (!done) { /* do nothing */}
console.log("finally, the done value changed!");
Some might logically think that the while loop will spin until the timer fires and then the timer will change the value of done to true and then the while loop will finish and the console.log() at the end will execute. That is NOT what will happen. This will actually be an infinite loop and the console.log() statement will never be executed.
The issue is that once you go into the spin wait in the while() loop, NO other Javascript can execute. So, the timer that wants to change the value of the done variable cannot execute. Thus, the while loop condition can never change and thus it is an infinite loop.
Here's what happens internally inside the JS engine:
done variable initialized to false
setTimeout() schedules a timer event for 1 second from now
The while loop starts spinning
1 second into the while loop spinning, the timer is ready to fire, but it won't be able to actually do anything until the interpreter gets back to the event loop
The while loop keeps spinning because the done variable never changes. Because it continues to spin, the JS engine never finishes this thread of execution and never gets to pull the next item from the event queue or run the pending timer.
node.js is an event driven environment. To solve this problem in a real world application, the done flag would get changed on some future event. So, rather than a spinning while loop, you would register an event handler for some relevant event in the future and do your work there. In the absolute worst case, you could set a recurring timer and "poll" to check the flag ever so often, but in nearly every single case, you can register an event handler for the actual event that will cause the done flag to change and do your work in that. Properly designed code that knows other code wants to know when something has changed may even offer its own event listener and its own notification events that one can register an interest in or even just a simple callback.

This is a great question but I found a fix!
var sleep = require('system-sleep')
var done = false
setTimeout(function() {
done = true
}, 1000)
while (!done) {
sleep(100)
console.log('sleeping')
}
console.log('finally, the done value changed!')
I think it works because system-sleep is not a spin wait.

There is another solution. You can get access to event loop almost every cycle.
let done = false;
setTimeout(() => {
done = true
}, 5);
const eventLoopQueue = () => {
return new Promise(resolve =>
setImmediate(() => {
console.log('event loop');
resolve();
})
);
}
const run = async () => {
while (!done) {
console.log('loop');
await eventLoopQueue();
}
}
run().then(() => console.log('Done'));

Node is a single serial task. There is no parallelism, and its concurrency is IO bound. Think of it like this: Everything is running on a single thread, when you make an IO call that is blocking/synchronous your process halts until the data is returned; however say we have a single thread that instead of waiting on IO(reading disk, grabbing a url, etc) your task continues on to the next task, and after that task is complete it checks that IO. This is basically what node does, its an "event-loop" its polling IO for completion(or progress) on a loop. So when a task does not complete(your loop) the event loop does not progress. To put it simply.

because timer needs to comeback and is waiting loop to finish to add to the queue, so although the timeout is in a separate thread, and may indeed finsihed the timer, but the "task" to set done = true is waiting on that infinite loop to finish

var open = false;
const EventEmitter = require("events");
const eventEmitter = new EventEmitter();
setTimeout(function () {
open = true;
eventEmitter.emit("open_var_changed");
}, 1000);
let wait_interval = setInterval(() => {
console.log("waiting");
}, 100);
eventEmitter.on("open_var_changed", () => {
clearInterval(wait_interval);
console.log("open var changed to ", open);
});
this exemple works and you can do setInterval and check if the open value changed inside it and it will work

Creating a cron job using a while loop

There are several libraries (especially for NodeJS and Javascript) that allow you to implement cron jobs and subsequently host them on a server.
In essence, cron jobs seem to me nothing more than repetitive tasks that are executed at a specific time/date on a day.
I was wondering therefore what the difference is between these libraries and just let's say a custom while loop. For instance in Javascript we could write:
var keepRunning = true
while (keepRunning) {
setTimeout(function () {
// call function to be executed when time constraint satisfied
}, 5000);
}
My questions are therefore:
why do we use cron job libraries? What are the benefits above a custom function like above?

This would not work as you might expect:
var keepRunning = true
while (keepRunning) {
setTimeout(function () {
// call function to be executed when time constraint satisfied
}, 5000);
}
That code will schedule new setTimeout callbacks as fast as it can while keepRuning is true, never unwinding the call stack and letting the event loop run any of those callbacks. It will likely consume all of your memory without running the scheduled code even once.
What you can do is something like this:
var keepRunning = true;
function run() {
if (keepRunning) {
// call function to be executed when time constraint satisfied
setTimeout(run, 5000);
}
}
setTimeout(run, 5000);
If you want to schedule all the callbacks at once, then you might do something like this:
for (let i = 1; i <= 100; i++) {
setTimeout(function () {
// call function to be executed when time constraint satisfied
}, 5000 * i);
}
but in this example you need to multiply the timeout by the iteration variable to make sure that they are not scheduled to run at the same time - i.e. they are still scheduled all at once but they are later run at different times.
Remember that JavaScript runs to completion and callbacks are executed later when the call stack unwinds. It's also important that for and while loops block the event loop from executing and no event can be handled while the loop is running.

Cron handles very time specific events far better than this. If you wanted something to happen at 9am each day you would absolutely have to use Cron over some method like this.
Cron measures time from epoch and is the most accurate way of scheduling tasks. I would also imagine it would result in better performance than what you are suggesting.
Why would you NOT use Cron?

A library is a collection of useful code. Those collections tend to group up a significant amount of functions, objects, etc.
Your example was just one situation that had very little versatility. Libraries would provide much more options for your loop, and would go beyond just addressing the rate, but also other factors (depending on what specific library you are referring to).

Why does a while loop block the event loop?

The following example is given in a Node.js book:
var open = false;
setTimeout(function() {
open = true
}, 1000)
while (!open) {
console.log('wait');
}
console.log('open sesame');
Explaining why the while loop blocks execution, the author says:
Node will never execute the timeout callback because the event loop is
stuck on this while loop started on line 7, never giving it a chance
to process the timeout event!
However, the author doesn't explain why this happens in the context of the event loop or what is really going on under the hood.
Can someone elaborate on this? Why does node get stuck? And how would one change the above code, whilst retaining the while control structure so that the event loop is not blocked and the code will behave as one might reasonably expect; wait
will be logged for only 1 second before the setTimeout fires and the process then exits after logging 'open sesame'.
Generic explanations such as the answers to this question about IO and event loops and callbacks do not really help me rationalise this. I'm hoping an answer which directly references the above code will help.

It's fairly simple really. Internally, node.js consists of this type of loop:
Get something from the event queue
Run whatever task is indicated and run it until it returns
When the above task is done, get the next item from the event queue
Run whatever task is indicated and run it until it returns
Rinse, lather, repeat - over and over
If at some point, there is nothing in the event queue, then go to sleep until something is placed in the event queue or until it's time for a timer to fire.
So, if a piece of Javascript is sitting in a while() loop, then that task is not finishing and per the above sequence, nothing new will be picked out of the event queue until that prior task is completely done. So, a very long or forever running while() loop just gums up the works. Because Javascript only runs one task at a time (single threaded for JS execution), if that one task is spinning in a while loop, then nothing else can ever execute.
Here's a simple example that might help explain it:
var done = false;
// set a timer for 1 second from now to set done to true
setTimeout(function() {
done = true;
}, 1000);
// spin wait for the done value to change
while (!done) { /* do nothing */}
console.log("finally, the done value changed!");
Some might logically think that the while loop will spin until the timer fires and then the timer will change the value of done to true and then the while loop will finish and the console.log() at the end will execute. That is NOT what will happen. This will actually be an infinite loop and the console.log() statement will never be executed.
The issue is that once you go into the spin wait in the while() loop, NO other Javascript can execute. So, the timer that wants to change the value of the done variable cannot execute. Thus, the while loop condition can never change and thus it is an infinite loop.
Here's what happens internally inside the JS engine:
done variable initialized to false
setTimeout() schedules a timer event for 1 second from now
The while loop starts spinning
1 second into the while loop spinning, the timer is ready to fire, but it won't be able to actually do anything until the interpreter gets back to the event loop
The while loop keeps spinning because the done variable never changes. Because it continues to spin, the JS engine never finishes this thread of execution and never gets to pull the next item from the event queue or run the pending timer.
node.js is an event driven environment. To solve this problem in a real world application, the done flag would get changed on some future event. So, rather than a spinning while loop, you would register an event handler for some relevant event in the future and do your work there. In the absolute worst case, you could set a recurring timer and "poll" to check the flag ever so often, but in nearly every single case, you can register an event handler for the actual event that will cause the done flag to change and do your work in that. Properly designed code that knows other code wants to know when something has changed may even offer its own event listener and its own notification events that one can register an interest in or even just a simple callback.

This is a great question but I found a fix!
var sleep = require('system-sleep')
var done = false
setTimeout(function() {
done = true
}, 1000)
while (!done) {
sleep(100)
console.log('sleeping')
}
console.log('finally, the done value changed!')
I think it works because system-sleep is not a spin wait.

There is another solution. You can get access to event loop almost every cycle.
let done = false;
setTimeout(() => {
done = true
}, 5);
const eventLoopQueue = () => {
return new Promise(resolve =>
setImmediate(() => {
console.log('event loop');
resolve();
})
);
}
const run = async () => {
while (!done) {
console.log('loop');
await eventLoopQueue();
}
}
run().then(() => console.log('Done'));

Node is a single serial task. There is no parallelism, and its concurrency is IO bound. Think of it like this: Everything is running on a single thread, when you make an IO call that is blocking/synchronous your process halts until the data is returned; however say we have a single thread that instead of waiting on IO(reading disk, grabbing a url, etc) your task continues on to the next task, and after that task is complete it checks that IO. This is basically what node does, its an "event-loop" its polling IO for completion(or progress) on a loop. So when a task does not complete(your loop) the event loop does not progress. To put it simply.

because timer needs to comeback and is waiting loop to finish to add to the queue, so although the timeout is in a separate thread, and may indeed finsihed the timer, but the "task" to set done = true is waiting on that infinite loop to finish

var open = false;
const EventEmitter = require("events");
const eventEmitter = new EventEmitter();
setTimeout(function () {
open = true;
eventEmitter.emit("open_var_changed");
}, 1000);
let wait_interval = setInterval(() => {
console.log("waiting");
}, 100);
eventEmitter.on("open_var_changed", () => {
clearInterval(wait_interval);
console.log("open var changed to ", open);
});
this exemple works and you can do setInterval and check if the open value changed inside it and it will work

Checking if a JavaScript setTimeout has fired

I'd like to be able to dispatch a bunch of work via JavaScript to be done in the browser in such a way that the browser stays responsive throughout.
The approach I'm trying to take is to chunk up the work, passing each chunk to a function that is then queued with a setTimeout(func, 0) call.
I need to know when all the work is done, so I'm storing the returned timer ID in a map (id -> true|false). This mapping is set to false in the next block of code after I have the timer ID, and the queued function sets the mapping to true when it completes... except, of course, the queued function doesn't know its timer ID.
Maybe there's a better/easier way... or some advice on how I can manipulate my map as I need to?

I would queue the work in an array, use one timeout to process the queue and call a callback once the queue is empty. Something like:
var work = [...];
var run = function(work, callback) {
setTimeout(function() {
if(work.length > 0) {
process(work.shift());
setTimeout(arguments.callee, 25);
}
else {
callback();
}
}, 25);
};
run(work, function() {
alert('Work is done!');
});
As JavaScript in browsers is single threaded there is no real advantage to run multiple timeouts (at least I think this is what you are doing). It may even slow down the browser.

I'd like to add that although javascript is single threaded you can still have multiple ajax calls going at once. I recently had a site that needed to do potentially hundreds of ajax calls and the browser just couldn't handle it. I created a queue that used setTimeOut to run 5 calls at once. When one of the ajax calls returned it fired a callback (which is handled by a single thread) and then made the next call on the stack.
Imagine you're a manager that can only talk to one person at a time, you give 5 employees assignments, then wait for their responses, which may come in any order. Once the first employee comes back and gives you the information, you give them a new assignment and wait for the next employee (or perhaps even the same employee) to come back. So although you're "single threaded" 5 things are going on at once.

There is an example right in the HTML Standard, how it is best to handle it:
To run tasks of several milliseconds back to back without any delay,
while still yielding back to the browser to avoid starving the user
interface (and to avoid the browser killing the script for hogging the
CPU), simply queue the next timer before performing work:
function doExpensiveWork() {
var done = false;
// ...
// this part of the function takes up to five milliseconds
// set done to true if we're done
// ...
return done;
}
function rescheduleWork() {
var handle = setTimeout(rescheduleWork, 0); // preschedule next iteration
if (doExpensiveWork())
clearTimeout(handle); // clear the timeout if we don't need it
}
function scheduleWork() {
setTimeout(rescheduleWork, 0);
}
scheduleWork(); // queues a task to do lots of work
The moment of finishing the work is pretty clear, when clearTimeout is called.