Develop Reference

Running an operation periodically when the length of operation is not known

I need to execute an operation that needs to be executed relatively fast (let's say 10 times per second. It should be fast enough, but I can sacrifice speed if there are issues.) This is an ajax request, so potentially I do not know how much time it takes - it could even take seconds if network is bad.
The usual:
setInterval(() => operation(), 100);
Will not work here, because if the network is bad and my operation takes more than 100 ms, It might be scheduled one after another, occupying JS engine time (please correct me if I'm wrong)
The other possible solution is to recursively run it:
function execute() {
operation();
setTimeout(execute, 100);
}
This means that there will be 100 ms between the calls to operation(), which is OK for me. The problem with this is that I'm afraid that it will fail at some point because of stack overflow. Consider this code:
i = 0;
function test() { if (i % 1000 == 0) console.log(i); i++; test(); }
If I run it my console, this fails in around 12000 calls. if I add setTimeout in the end, this would mean 12000 / 10 / 60 = 20 minutes, potentially ruining the user experience.
Are there any simple ways how to do this and be sure it can run for days?

There's no "recursion" in asynchronous JavaScript. The synchronous code (the test function) fails because each call occupies some space in the call stack, and when it reaches the maximum size, further function calls throw an error.
However, asynchrony goes beyond the stack: when you call setTimeout, for example, it queues its callback in the event loop and returns immediately. Then, the code, that called it can return as well, and so on until the call stack is empty. setTimeout fires only after that.
The code queued by setTimeout then repeats the process, so no calls accumulate in the call stack.
Therefore, "recursive" setTimeout is a good solution to your problem.
Check this example (I recommend you to open it in fullscreen mode or watch it in the browser console):
Synchronous example:
function synchronousRecursion(i){
if(i % 5000 === 0) console.log('synchronous', i)
synchronousRecursion(i+1);
//The function cannot continue or return, waiting for the recursive call
//Further code won't be executed because of the error
console.log('This will never be evaluated')
}
try{
synchronousRecursion(1)
}catch(e){
console.error('Note that the stack accumuates (contains the function many times)', e.stack)
}
/* Just to make console fill the available space */
.as-console-wrapper{max-height: 100% !important;}
Asynchronous example:
function asynchronousRecursion(i){
console.log('asynchronous',i)
console.log('Note that the stack does not accumuate (always contains a single item)', new Error('Stack trace:').stack)
setTimeout(asynchronousRecursion, 100, i+1);
//setTimeout returns immediately, so code may continue
console.log('This will be evaluated before the timeout fires')
//<-- asynchronusRecursion can return here
}
asynchronousRecursion(1)
/* Just to make console fill the available space */
.as-console-wrapper{max-height: 100% !important;}

The two alternatives you showed here actually share the flaw you're concerned about, which is that the callbacks might bunch up and run together. Using setTimeout like this is (for your purposes) identical to calling setInterval (except for some small subtleties that don't apply with a light call like making an AJAX request.)
It sounds like you might want to guarantee that the callbacks run in order, or potentially that if multiple callbacks come in at once, that only the most recent one is run.
To build a service that runs the most recent callback, consider a setup like this:
let lastCallbackOriginTime = 0;
setInterval(()=>{
const now = new Date().getTime();
fetch(url).then(x=>x.json()).then(res=>{
if ( now > lastCallbackOriginTime ) {
// do interesting stuff
lastCallbackOriginTime = now;
}
else console.log('Already ran a more recent callback');
});
}, 100);
Or let's make it run the callbacks in order. To do this, just make each callback depend on a promise returned by the previous one.
let promises = [Promise.resolve(true)];
setInterval(()=>{
const promise = new Promise((resolve, reject)=> {
fetch(url).then(x=>x.json()).then(serviceResponse=>{
const lastPromise = promises[promises.length - 1];
lastPromise.then(()=>resolve(serviceResponse));
}).then((serviceResponse)=>{
// Your actual callback code
});
promises.push(promise)
});
}, 100);

Recursion in js setTimeout how it work? and why it work?

function printNumbers(from, to) {
let current = from;
function go() {
alert(current);
if (current < to) {
setTimeOut(go, 1000); //recursion to function go()
}
current++; // because of recursion execution should not reach this point
}, 1000);
}
printNumbers(5, 10);
Please explain to me why ^current++^ works immediately? but because of recursion it should not work. isnt it? please explain me who understand how it work and why

setTimeout is not blocking. It puts a function on a queue to be called when some time has passed. The rest of the function still executes without pause.
There is no return statement or anything else that would stop the JS engine from reaching the current++; statement.

To supplement Quentin's answer, this setTimeout(go, 1000) line (FYI, setTimeout is the proper spelling, not setTimeOut) isn't actually doing any recursion. It's passing off a function go to be called after 1000 ms in the event loop after the stack empties (the same stack that can do recursion and overflow). It's incidental that this all happens within the same function that's passed as a parameter to the timeout which makes it visually look recursive.
What happens is the setTimeout line runs and adds the function go to the event loop and guarantees a delay of at least 1000 ms. Then, the rest of the synchronous code runs, including the rest of go and current++;. Later on, when the stack is empty, tasks on the loop are executed in order, including go (assuming the 1000 ms timer has elapsed).
This explains why code like:
(function run() {
requestAnimationFrame(run);
// do stuff
})();
never overflows the stack: it's not actually recursion and each call frame is destroyed before the next call happens.
As an aside, it may be surprising that
(function run() {
requestAnimationFrame(run);
// do stuff
})();
and
(function run() {
// do stuff
requestAnimationFrame(run);
})();
behave pretty much the same. The reason is that the next run callback is guaranteed to execute once all synchronous code (the current call to run and anything else on the call stack) completes execution, so it's not like the location causes a block in synchronous execution and the child call gets to do work as would be the case with recursion.

Call Stack while using setTimeout()

I am a bit confused about setTimeout.I want to confirm whether the output for the following code will always be:
inside abc
inside sample
The code:
function abc() {
xyz();
// interactions and modifications in DOM
$("#id1").append("something");
$("#id2").val("set something");
$("#id3").after("create some dynamic element");
// 10 to 20 interaction more...
console.log('inside abc');
}
function xyz() {
setTimeout(function() {
sample();
},0);
}
function sample() {
console.log('inside sample')
}
It would be great,if somebody could explain the whole flow with the call stack.

Yes, it will always have that output.
The callback inside a setTimeout will not be called until the execution context is clear - i.e. the currently executing sequence of code has finished.
This means that even if you do
setTimeout(function () { console.log("1 second!"); }, 1000);
var start = +new Date();
while ((+new Date()) - start < 5000) {}
1 second! will not be logged any sooner than 5 seconds have passed.

setTimeout() will run asynchronously after finishing current execution block. So output should be same always:
inside abc
inside sample
Javascript internally manage an event queues internally for all async tasks. Every time it checks its async queue after finishing current execution block.

Yes, the console output will always be the same. setTimeout's callback function is executed asynchronously after the context that called it is cleared. When setTimeout is called, it places its callback function on the stack and returns to the current execution context. When that is done (in your example, when abc is fully executed), the callback function is executed, which in your example basically calls sample immediately. So your code output will never be different.
The fact that setTimeout's callbacks are themselves executed asynchronously can be seen if you placed a longer setTimeout function somewhere inside abc before calling xyz:
function abc() {
setTimeout(function(){
console.log('wait')
},1000);
xyz();
console.log('inside abc');
}
function xyz() {
setTimeout(function(){
sample();
} ,0);
}
function sample() {
console.log('inside sample');
}
abc();
...your console will log:
inside abc
inside sample
wait
To hold sample's execution until the longer timeout is complete you would need to place the setTimeout calling sample inside the longer setTimeout.
If setTimeout's callback is ever behaving differently, it's most likely due to being accidentally passed a function call instead of a function pointer, like this:
setTimeout(sample(),0);
instead of
setTimeout(sample,0)
Also, just in case you didn't know (or for others), you can add
debugger;
at any point(s) in your Javascript code and then run the code to view the callstack at that point using dev tools (in Chrome it is in the right panel under 'Sources').

Is setTimeout a valid way to avoid stackoverflow when using callbacks?

Say I want to create some callback hell, such as :
function handleUserInput(callback) {
...
}
function get(uri, callback) {
...
}
function processResponseInWorker(callback) {
....
}
function updateIndexedDB(callback) {
...
}
function updateUI() {
...
}
handleUserInput(get(hot_uri,processResponseInWorker(updateIndexedDB(updateUI()))));
This is mostly academic since the stack only gets 5 high.
In fact, is there even a stack at all? Since these calls will return immediately, and the callback will only be invoked, outside of these contexts, by whatever asynchronous tasks these functions perform.
Okay, just say there IS a call stack here, if each callback was forced to execute inside a setTimeout(func,0) then the calling function would return immediately, the whole chain would return immediately and the functions would be executed off the setTimout queue.
Is that correct?

setTimeout will not call the provided functions unless there is no other code running.
As is demonstrated in this code the timeout, although it is a 0 millisecond delay, will not execute until no other code is being executed. That is the nature of classic javascript (synchronized).
console.log(1);
setTimeout(function() {
console.log(2);
}, 0);
console.log(3);
for(var i = 4; i < 100; i++) {
console.log(i);
}
Initially lets assume the depth is 0. Let's assume that no functions other than the callbacks and the functions in the original code are called
updateUI()
Call the function updateUI our depth becomes 1. We return a function from this function and our depth becomes 0 again and we have something that essentially looks like this.
updateIndexedDB(function(){})
So we call updateIndexedDB and our depth becomes 1 which calls the provided callback function and our depth becomes 2. The callback returns and depth becomes 1 and updateIndexedDB returns a functions so that we have something looks like this.
processResponseInWorker(function() {})
Similar process occurs until we have this
get(hot_uri, function() {})
Again the same thing until we have this
handleUserInput(function() {})
Without using timeout the max depth i observe is 2 but using timeouts on your callbacks (which i don't know if you can do because i don't know if your callbacks give you future callbacks) your max is 1 since the callbacks will individually be executed after all code has been executed (on their own stack).
I feel like you meant to write your code this way
handleUserInput(function() {
get(hot_uri , function() {
processResponseInWorker(function() {
updateIndexedDB(function() {
updateUI();
});
});
});
});
which would result in a depth of 6 again unless you use timeouts which would result in a stack size of 1.

Mechanics behind Javascript callbacks being asynchronous

I'm having a bit of trouble understanding the mechanics of JS callbacks. I have a fair idea of how callbacks can be used in JS, but I do not understand how a callback is asynchronous.
For e.g., if my understanding is correct, a callback is of the nature:
db.query(param1, param2 , callback_fn1(){..} );
And the implementation of db.query() is along the lines of :
db.prototype.query = function(p1 , p2 , callback ){
//some code
callback();
}
How does the above implementation make db.query an asynchronous function? Does this not mean that a function called callback is passed to query and that function is called inside query? It looks like query is just another synchronous function. Could someone help me understand what I'm overlooking here? Thanks!

The code sample you've shown is actually still synchronous because is instructed to run immediately. An asynchronous callback is a callback that doesn't immediately need to be executed, so it doesn't block the event loop until you instruct it to run.
The most common way in Node.js to do this is with process.nextTick() which runs a specified function when the event loop call stack is empty. Here's an example:
var async = function(args, callback) {
// do some work
process.nextTick(function() {
callback(val);
});
};
Then we call the function like this:
async(args, function(val) {
console.log(val);
});
console.log('end');
In this example, the function async() and console.log('end') are added to the call stack. The call stack empties once both of those functions are run, and once it's empty, console.log(val) then runs.
If you're still confused, think of process.nextTick() as an optimized version of this code:
var fn = function() {};
setTimeout(fn, 0);
It basically means "run this function as soon as possible when you are not busy".

Edit: I just now realized the question is tagged with node.js. My answer is more about Javascript in the browser, #hexacyanide's answer is more about node.js. I guess knowing both doesn't hurt, though!
The way you posted it the code will indeed be blocking. For asynchronous behavior there are a few things you can utilize, such as
setTimeout and setInterval
Built-in, asynchronous methods such as from the FileReader API
Ajax requests
Web workers (see #html5rocks)
Your example code could be written as follows (fiddle):
function doStuff(callback) {
setTimeout(function () {
for (var i = 0; i < 1000; i++) {
// do some busy work
var x = Math.sqrt(i);
}
callback();
}, 0);
}
console.log('start');
doStuff(function () {
console.log('callback called');
});
console.log('after doStuff()');
The setTimeout call will allow the Javascript interpreter/compiler (however exactly they work these days) to run the function in a non-blocking matter which is why (most likely), you will see the output
start
after doStuff()
callback called
Note that asynchronousity is different from multi-threading. Javascript is still single-threaded (with the exception of web workers!).
A more in-depth explanation can be found for example here