When debugging Node.js code I often encounter call stacks that do not include my program's code in them, only node_modules/non-user code, despite the current line of execution being at a location in my code. This defeats the purpose of following the call stack to see the execution path through my application code.
Why are my source files not showing in the call stack?
It appears that you're looking at an asynchronous stack trace where your code is not in the stack except for your callback because your code unwound/finished and THEN the async callback was called.
All .then() handlers for all promises are called asynchronously with a clean stack. That's per the promise specification. So, promises always let the current thread of execution finish and unwind and then they fire .then() handlers with no user code on the stack when the callback is called. What you are describing is how synchronous code would work, not asynchronous code. We could talk a lot more specifically instead of theoretically if you showed actual code and described where you're looking at the call stack.
Async progress often has to be tracked with logging because you can't easily step through it and you can't just break and look at stack traces either.
As a little simpler example to look at:
function foo() {
setTimeout(() => {
console.log("timer"); // set breakpoint here
}, 100);
}
foo();
The function foo() has finished executing and returned before the callback is called and thus the stack trace will not have any of your code (other than just the callback) on it.
While .then() handlers use a slightly different scheduler than setTimeout(), the principle is the same.
Related
Apologies if the title of this question is slightly confusing, I was struggling to find a way to word this.
Basically, when an asynchronous function is called, how much further down the code will be executed if the asynchronous function is called from other functions, objects, or files. For example:
function func1() {
// async function 1
// async function 2
}
function func2() {
// more code
}
func1();
func2();
Let's say we execute func1() and both the async functions inside of it take a very long time to run. Will we continue on to func2() while the content of func1() is still being run, or will we block until func1() is finished before continuing to func2()?
JavaScript is single threaded, non-blocking and asynchronous language. JavaScript has call stack, event loop and a callback queue. Words are taken directly from this video. Javascript works on v8 engine (chrome), spider monkey (firefox) which provide JavaScript call stack and heap. v8 or spider monkey provide call stack to Javascript so whenever a function is called, it is stored in the call stack of the runtime (browser in our case, local if you have node installed). Browser also provide the web apis to the javascript like setTimeOut, XMLHttpRequest and DOM. The pictorial illustration is something like this. (Source is the same video which I've tagged.)
JavaScript is single threaded and it means it can execute one function at a time as it has only one call stack. So whenever an asynchronous code (inside func1)is executing, it is executed through webAPIs provided by the browser. The role of callback queues come here. Whenever the result from the Async code is executed, it is stored in the callback queue and it waits for the stack to get empty (event driven programming). Whenever it sees that the stack is empty, the function from the callback queues will start executing.
In your case, you're calling func1, it kicks in some async code but currently func1 is in stack. If the async code has completed and if it sees the stack as empty, it will executed the async code first and then func2 but if the async code hasn't completed, it will start executing the func2 and the callback queue will wait for the stack to get empty. In this case, the flow will be func1 --> func2 --> async code.
So, it is matter of timing. For async code, if the code has returned and waiting in callback queue, as soon as it sees the stack empty, it will kick in and start executing the callbacks (.then or callback from setTimeOut etc.) I'll recommend you to watch that video for better insights and what an event loop in JS is. If I'm missing out something, anyone please feel free to edit this answer or provide suggestions for edit.
JavaScript is single threaded, it literally means nothing can be executed in the same time. If your code has asynchronous nature it means that it is executed in uknown future. I said uknown because it depends from the code. For example asynchronous code for ajax call will be executed when the browser gets the response from the server.
That said, the async code is not executed in the declaration moment but in uknown next event loop. Furthermore it not blocks the current execution. In javascript your execution code can not be disturbed. It just runs through all lines.
Going back to the example, if func1 has no sync blocking code then func2 will be executed right away without any delay.
Asynchronous callback are top level events (similar to button click events) and can only be started when there is no other scripts executing.
Execution will continue till call stack is empty (including "global user code" which what your sample shows) and script engine itself gets control back - so both func1() and func2() will finish before callbacks are executed.
Note: you can't rely on such behavior for normal code - if "asynchronous" function has both synchronous and asynchronous code branches callback from that function can be executed synchronously without even returning to calling function (this can happen with plain callbacks or any other form including promises).
I want to invoke a function after an event gets fired then in the same call back call the function again. This is to create a sort of event listener when the function completes.
You'll know what i'm trying to do when you see the code:
"use strict";
var page = require('webpage').create();
var system = require('system');
function onStdReadLine(callback) {
system.stdin.readLineAsync(function(err, line) {
callback(line);
onStdReadLine(callback);
});
}
onStdReadLine(function(line) {
// do something when the line comes in
system.stdout.writeLine(line);
});
Question:
Could this potentially cause a stack overflow? Is there a way to refactor this code to not be recursive?
Thanks!
Could this potentially cause a stack overflow?
No, it will not cause a stack overflow. An async callback is called when the stack is completely unwound so there is no stack buildup.
Remember how an async operation works. The current thread of Javascript starts the async operation. That async operation is then managed by some other native code. The current thread of Javascript then runs to its completion and finishes (thus clearing the stack).
Some time later, the native code that is running the async operation sees that the operation has completed and posts an event to the Javascript event queue to call the callback for that async operation. When the JS engine is not running anything else, it will then pull that event out of the queue and process it (calling the callback). At the time it calls the callback, it will have an empty stack frame.
There will be an active scope object for the callback function, but in Javascript scopes are completely separate from the stack frame.
The only time this could be a problem is if your callback was ever called synchronously. Then, there would obviously be a stack build-up and the potential for stack overflow. But, as long as the callback is always called asynchronously (which it should be), then there is no issue.
Is there a way to refactor this code to not be recursive?
It is a very useful design pattern in Javascript asynchronous programming to run the next iteration of something from the completion of the previous iteration and calling a function from within it is the usual way to do so. There is no reason to refactor it.
No, this won't cause a stack overflow. When you call an asynchronous function, the original function returns, so it's stack frame is freed. The callback function is called from the event handler, not from the original function.
I've read that callbacks make JavaScript asynchronously. But I'm not sure if I understood the explanation. This is what I get
Callback functions allow us to do things asynchronously, since they ensure
that the lines prior to the callback are completely finished before loading
the next line.
Is that true? Thanks
First off, it isn't the callback that enables anything. A given operation in node.js or even browser-based javascript either is or isn't asynchronous. It really has nothing to do with the callback, though a callback is typically used to communicate results of an asynchronous operation.
For example, Javascript's array.forEach() uses a callback, but it is not asynchronous. So, async operations are async because their underlying implementation is non-blocking. You start the operation by making a function call, the operation proceeds in the background and the rest of your code continues to run. Meanwhile, when the asynchronous operation completes, it usually then needs to tell your code that it is done and perhaps communicate some results. A callback function is the chosen mechanism for communicating the completion of the async operation.
I've read that callbacks make JavaScript asynchronously.
No, that is not really true. Callbacks can be used with synchronous operations too. Just because one uses a callback does not make anything asynchronous. The underlying native code implementation of an operation must be asynchronous (such as an Ajax call or other networking operation). Callbacks are used to communicate the results of asynchronous operations. They are also have many other non-asynchronous uses too. So, a callback is just one tool used in an asynchronous operation and a callback is a tool with many other uses too. You cannot say callback === asynchronous.
Callback functions allow us to do things asynchronously, since they
ensure that the lines prior to the callback are completely finished
before loading the next line.
It is hard to tell exactly what you mean by this, but it sounds wrong to me. When using asynchronous operations, code is typically not executed in the order laid out in the file. For example, if you did this:
console.log("async start");
callSomeAsyncOperation(function(result) {
console.log("async done");
});
console.log("I'm here now");
You would see this in the log:
async start
I'm here now
async done
Callbacks and Event Queues Explained
It may also be useful to understand how an asynchronous operation works. Javascript works off an event queue. A given sequence of Javascript code runs to its completion. When that completes, the engine looks in the event queue to see if there are any more events to process. If so, the first event in the queue is pulled out and a callback that was registered for that event is called. This starts a new sequence of Javascript code running. That code continues to run until it finishes. When it finishes, the engine checks for another event. If there is one, it is then processed by calling a callback associated with that event. When there are no more events to process, the engine goes to sleep waiting for the next event. When an event occurs (outside the main Javascript thead), it is then added to the queue and that process of adding it to the queue causes the JS engine to wake up and service that event.
When writing your Javascript, you will often register and event handler for an event. The way this works in Javascript is that you say what event you're interested in (which may also include specifying some other information like what object you're looking for events on) and then you pass it is a function reference. You are, in essence, telling the Javascript engine that you want it to call your function reference when this event occurs. This type of function reference is referred to as a "callback". It's just a normal function, but the context in which it is being used is called a "callback" because some other code will "call you back" at some time in the future by executing your function. You can then place appropriate code in that function reference (inside that callback function) that can respond to that event. Depending upon the type of event, you may only get called once or it may call you every time that event occurs.
You can read more about how this event queue and callbacks work in these references:
Run Arbitrary Code While Waiting For Callback in Node?
blocking code in non-blocking http server
Hidden threads in Javascript/Node that never execute user code: is it possible, and if so could it lead to an arcane possibility for a race condition?
How does JavaScript handle AJAX responses in the background? (written about the browser, but concept is the same)
First of all, let's understand what callback is (by definition):
In computer programming, a callback is a piece of executable code that
is passed as an argument to other code, which is expected to call back
(execute) the argument at some convenient time. The invocation may be
immediate as in a synchronous callback, or it might happen at later
time as in an asynchronous callback. In all cases, the intention is to
specify a function or subroutine as an entity that is, depending on
the language, more or less similar to a variable.
Now, talking about Javascript, not always a callback is asynchronous. For example:
function syncOperation(callback) {
callback();
}
syncOperation(function() {
console.log('a');
console.log('b');
});
console.log('c');
That callback is synchronous, since it doesn't make any asynchronous operation. If you open your console and run the code above, you'll see that it will log a, then b, then c.
Now, let's see an asynchronous example:
function asyncOperation(callback) {
setTimeout(callback, 0);
}
asyncOperation(function() {
console.log('a');
console.log('b');
});
console.log('c');
You'll see firstly c, then a, and b. That's because the setTimeout function runs asynchronously.
Some built-in Javascript functions are asynchronous, and they will run, and at some time, they will call the function you passed as parameter back. That's why if you do:
var a = 'text';
setTimeout(function() { a = 'new text'; }, 0);
console.log(a);
You'll see in your console text, because it will run before the variable has been changed.
I'm getting slightly annoyed now after reading many different articles on how to write proper code in node js.
I would just like some clarification on whether I'm correct or not with each of these statements:
Code is executed synchronously
A for loop or while loop etc. will be executed asynchronously
By doing the code below isn't proper asynchronous:
L
function doSomething(callback) {
// doing some code
// much much code
callback();
}
The reason that people are saying that this won't work properly is that code is executed asynchronously, so the callback won't be fired off at the end of the code it will all be executed at once.
So for example if you were filling some object by doing some stuff and you wanted to post that full object back through the callback it doesn't work because it will all be executed at the same time.
No, for and while loops are synchronous in Node.js.
Whether your doSomething function example will work or not all depends on whether you're making any calls to asynchronous functions before calling callback(). If you're making asynchronous calls, then you'd need to postpone calling callback() until those asynchronous calls have completed. If you're only making synchronous calls, then you don't need to use a callback and your function can synchronously return the result.
Code in node.js isn't standardly asynchronous. Calls that you place one after another will still be executed one after another. A for or while loop will still block the main code in this way.
However, you can pass around functions which will be executed later, through process.nextTick(callback). This method will add a function to the event queue, which node.js takes care of internally. This will only be executed after any other blocking code has been extecuted. An example:
function doWork(callback) {
//Do some work here, for instance a for loop that might take a while
callback(workReslt);
}
function workDone(workResult) {
//Process the result here
}
function startWork() {
process.nextTick(doWork(workDone));
}
This will execute doWork() when the server passes through the event loop again. This way, the startWork() function will not block the main code.
However, most modules in node.js already implement this. Database and file access is usually non-blocking, it will simply fire a callback when it is done with it's work.
If you really have to perform heavy async calculations (or perhaps some IO that doesn't already have a module for it), you can look into the async module, which has a nice tutorial here: http://justinklemm.com/node-js-async-tutorial/
As for coding conventions, think of it this way. If you know the action is going to take time, you should make it asynchronous (either through the method I mentioned first or the async module). Actions like these are usually related to I/O, which usually also means there is already an asynchronous module for it. However, most cases I have come across up to now, have never required this.
Sources:
http://howtonode.org/understanding-process-next-tick
I am trying to grasp on Javascript Asynchronous functions and callbacks.
I got stuck on the concept of callback functions, where I am reading on some places: they are use to have sequential execution of code (mostly in context of jquery e.g animate)and some places specially in the context of Nodejs; they are use to have a parallel execution Asynchronous and avoid blocking of code.
So can some expert in this topic please shed light on this and clear this fuzz in my mind (examples??).
so I could make my mind for the usage of callback function
or that is solely depends on the place of where you are calling/placing a callback function in your code? .
Thanks,
P.S: I am scared that this question would be close as subjective but still I could expect concrete answer for this (perhaps some examples)
Edit: actually this is the example from internet which makes me ambigous:
function do_a(){
// simulate a time consuming function
setTimeout( function(){
console.log( '`do_a`: this takes longer than `do_b`' );
}, 1000 );
}
function do_b(){
console.log( '`do_b`: this is supposed to come out after `do_a` but it comes out before `do_a`' );
}
do_a();
do_b();
Result
`do_b`: this is supposed to come out after `do_a` but it comes out before `do_a`
`do_a`: this takes longer than `do_b`
when JS is sequential then do_b should always come after do_a according to my understanding.
The core of JavaScript is largely synchronous, in that functions complete their task fully, before completing. Prior to the advent of AJAX, it was really only setTimeout and setInterval that provided asynchronous behavior.
However, it's easy to forget that event handlers are, effectively async code. Attaching a handler does not invoke the handler code and that code isn't executed until some unknowable time in the future.
Then came AJAX, with its calls to the server. These calls could be configured to be synchronous, but developers generally preferred async calls and used callback methods to implement them.
Then, we saw the proliferation of JS libraries and toolkits. These strove to homogenize different browsers' implementations of things and built on the callback approach to async code. You also started to see a lot more synchronous callbacks for things like array iteration or CSS query result handling.
Now, we are seeing Deferreds and Promises in the mix. These are objects that represent the value of a long running operation and provide an API for handling that value when it arrives.
NodeJS leans towards an async approach to many things; that much is true. However this is more a design decision on their part, rather than any inherent async nature of JS.
Javascript is always a synchronous(blocking) single thread language but we can make Javascript act Asynchronous through programming.
Synchronous code:
console.log('a');
console.log('b');
Asynchronous code:
console.log('a');
setTimeout(function() {
console.log('b');
}, 1000);
setTimeout(function() {
console.log('c');
}, 1000);
setTimeout(function() {
console.log('d');
}, 1000);
console.log('e');
This outputs: a e b c d
In node long running processes use process.nextTick() to queue up the functions/callbacks. This is usually done in the API of node and unless your programming(outside the api) with something that is blocking or code that is long running then it doesn't really effect you much. The link below should explain it better then I can.
howtonode process.nextTick()
jQuery AJAX also takes callbacks and such as it its coded not to wait for server responses before moving on to the next block of code. It just rememebers the function to run when the server responds. This is based on XMLHTTPRequest object that the browsers expose. The XHR object will remember the function to call back when the response returns.
setTimeout(fn, 0) of javascript will run a function once the call stack is empty (next available free tick) which can be used to create async like features.setTimeout(fn, 0) question on stackoverflow
To summerise the async abilities of javascript is as much to do with the environments they are programmed in as javascript itself. You do not gain any magic by just using lots of function calls and callbacks unless your using some API/script.
Jquery Deferred Object Is another good link for async capabilities of jQuery. Googling might find you information on how jQuery Deferred works also for more insight.
In JavaScript the term "asynchronous" typically refers to code that gets executed when the call stack is empty and the engine picks a job from one of its job queues for execution.
Once code is being executed, it represents a synchronous sequence of execution, which continues until the call stack is empty again. This sequence of execution will not be interrupted by events in order to execute some other JavaScript code (when we discard Web Workers). In other words, a single JavaScript environment has no preemptive concurrency.
While synchronous execution is ongoing, events might be registered as jobs in some job queues, but the engine will not process those before first properly execution what is on the call stack. Only when the call stack is empty will the engine take notice of the job queues, pick one according to priority, and execute it (and that is called asynchronous).
Callbacks
Callbacks can be synchronous or asynchronous -- this really depends on how they are called.
For instance, here the callback is executed synchronously:
new Promise(function (resolve) { /* .... */ });
And here the callback is executed asynchronously:
setTimeout(function () { /* ... */ });
It really depends on the function that takes the callback as argument; how it deals with eventually calling that callback.
Ways to get code to execute asynchronously
The core ECMAScript language does not offer a lot of ways to do this. The well-known ones are offered via other APIs, such as the Web API, which are not part of the core language (setTimeout, setInterval, requestAnimationFrame, fetch, queueMicrotask, addEventListener, ...).
Core ECMAScript offers Promise.prototype.then and (depending on that) await. The callback passed to then is guaranteed to execute asynchronously. await will ensure that the next statements in the same function will be executed asynchronously: await makes the current function return, and this function's execution context will be restored and resumed by a job.
It also offers listening to when the garbage collector is about to garbage collect an object, with FinalizationRegistry.
Web Workers
Web Workers will execute in a separate execution environment, with their own call stack. Here preemptive concurrency is possible. When the term asynchronous is used in the JavaScript world, it typically does not refer to this kind of parallelism, although the communication with a Web Worker happens via asynchronous callback functions.
Javascript by default is "synchronous", it's the web APIs that handle "asynchronous" behaviour.
As for the setTimeout example,
console.log(...), in the global scope works straight away, while those inside functions wrapped inside setTimeout, wait inside the callback queue only to be pushed back on the call stack once ready. Thus they take time. Also, the time specified is not exact but the minimum time after which that piece of code can run anytime.
Thanks !