I'm curious as to whether all javascript callbacks are asynchronous, or whether that is the case only in certain situations. Also, I'm sure what makes javascript code asynchronous (or ways to use asynchronous javascript) differ between the browser and nodejs, so I'd like to know in each situation what constitutes real asynchronous javascript.
I'm under the impression that in the following scenarion, I'm not actually writing asynchronous code.
function addOne(value){
value = value + 1;
return value;
}
function simpleMap(values, callback){
for(i = 0; i < values.length; i++){
val = values[i];
val = callback(val);
values[i] = val;
}
return values;
}
newValues = simpleMap([1,2,3], addOne);
However, for example, I know that jQuery's AJAX functions are truly asynchronous (not taking in to account the promises which are now available). What is it that make jQuery's AJAX asynchronous? Is it as simple that it involves XHR requests, and in the browser, all XHR requests are asynchronous?
I have the same question for the nodejs environment. Can something in node only be asynchronous if it involves something like file i/o, process.nextTick, setTimeout, or setInterval? Why when I do something like a database call with mongodb/mongoose, is that asynchronous? What's going on behind the scenes that's making it so?
Are asynchronous "situations" predetermined by the environment? Or is there some way to make one's own function truly asynchronous without leveraging very specific functions of the environment (such as xhr, file io in node, process.nexttick, etc)?
I'm curious as to whether all javascript callbacks are asynchronous
No. For instance, the callback used by Array#sort is not asynchronous, nor is the one used by String#replace.
The only way you know whether a callback is asynchronous is from its documentation. Typically, ones involving requests for external resources (ajax calls, for instance) are asynchronous, and others may or may not be.
However, for example, I know that jQuery's AJAX functions are truly asynchronous...
Not necessarily, as currently jQuery still has the async flag which you can set false to force a synchronous request. (It's not a good idea, and they're going to remove that, but you can. jQuery passes the flag to the underlying browser object which provides the synchronous/asynchronous behavior.)
What is it that make jQuery's AJAX asynchronous?
The browser. jQuery's ajax calls use the XMLHttpRequest object (or in certain situations, a script element), which defaults to asynchronous operation provided by the browser.
Or is there some way to make one's own function truly asynchronous without leveraging very specific functions of the environment...
Until recently, no. Up through the 5th edition specification, JavaScript the language was basically silent on the entire concept of threads and asynchronicity; it was only when you got into environments that it came up. The only way to make something asynchronous was to use a host-provided function, such as nextTick (or any of the various operations that completes asynchronously) on NodeJS or setTimeout on browsers.
In the ECMAScript 6th edition specification in June 2015, they introduced promises into the language. The callbacks hooked up to an ES6 promise via then and such are always invoked asynchronously (even if the promise is already settled when the callback is attached), and so JavaScript has asynchronicity at a language level now. So if you implement your function so that it returns a promise rather than accepting a callback, you'll know that the then callbacks hooked up to it will be triggered asynchronously.
Callbacks that you call yourself are regular function calls, which are always synchronous.
Certain native APIs (eg, AJAX, geolocation, Node.js disk or network APIs) are asynchronous and will execute their callbacks later in the event loop.
If you call a callback synchronously from within an async callback, it will end up being async too.
To create your own asynchronous functions you have to make use of other asynchronous functions which may be provided by the interpreter.
This code for example defines a function "addKeyHandler" which is asynchronous. But that only works because document.onKey is called asynchronously by the JS engine. The JavaScript engine is able to provide asynchronous functionality because the operating system provides such functionality which is then used by JS. The OS in turn can only provide async functionality because the hardware provides it (called hardware interrupts).
However if the OS and hardware didn't provide any async functions it would still be possible to write a JS interpreter. But it would have to use an infinite loop and check in each iteration if any events occured and then invoke the appropriate callbacks. That would mean the CPU would always be under full load.
var keyCallbacks = [];
var addKeyHandler = function(f) {
keyCallbacks.push(f);
};
document.onkeypress = function(e) {
keyCallbacks.forEach(function(f) {
f(e);
});
};
addKeyHandler(function(e) {
console.log(String.fromCharCode(e.charCode));
});
Simply taking a callback doesn't make a function asynchronous. There are many examples of functions that take a function argument but are not asynchronous, for example, Array's forEach.
For a function to be asynchronous it needs to perform an asynchronous operation. Ways of introducing asynchronicity can be
timer functions setTimeout, setInterval
special functions nextTick, setImmediate
performing I/O (listening to network, querying a database, reading or writing from a resource)
subscribing to an event
According to article "Does taking a callback make a function asynchronous?"
Going for a simple answer:
Unless you're dealing with promises, JS callbacks are only asynchronous if they rely on an API external to JS (such as provided by the browser).
setTimeout, fetch, and so on are asynchronous because they rely on external api's. (setTimeout is part of the windowOrGlobalWorker web api, for example, and fetch is a web api in itself.)
If a callback does not rely on an external API, it's synchronous.
Promises are the only native async functionality in JS.
A great way to get your head around all this is read at least the first few articles in the MDN primer on async: https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous
Related
since javascript is synchronous then why do we need callback function,
in the below picture both code can same thing
As #Meno-101 rightly said, you're looking at the trivial function invocation utility of callbacks, they're rather used for asynchronous calls. In asynchronous calls we wait for some data to arrive and then invoke a function, you won't see a difference when you're being presented with the result in an instant. setTimeout is one easy way to replicate such a situation;
Callbacks are generally used when the function needs to perform events before the callback is executed, or when the function does not (or cannot) have meaningful return values to act on, as is the case for Asynchronous JavaScript (based on timers) or XMLHttpRequest requests. Checkout the deets here
But if we're are bent on using it synchronously , I could only come up with this code:
This made the code reusable which isn't possible the other way, we'd have to write different definitions of same function for disp() and disp1() to function.
You’re just confusing callback’s purpose with a function invocation purpose.
Callbacks are used in the case of web requests, reading/writing to files, or maybe even just waiting for some reason the programmer only know why, take this example:
setTimeout(()=> console.log(“hello”), 3000)
There must be a reason for you to have this in your code, what ever it’s you don’t want all the page to hang up on the user.
And, I do recommend you look for function call stack if you’re just starting off.
I am writing a node.js addon to perform some cryptographic computation, which may take about 1 μs – 20 μs. Now I have a choice: implement this as a synchronous or as an asynchronous method (which does the computation on a background worker)?
It is obvious that network and I/O, which sometimes takes longer than a millisecond should be done asynchronously. Parsing JSON input is fast and should be done synchronously.
In my situation keeping the latency low is important, but optimizing away microseconds feels a lot like premature optimization. So with this context in mind I would be interested to get your view on the question:
When using node.js, how long does a (synchronous) call have to block until you decide to run it asynchronously on a background thread?
It is obvious that network and I/O, which sometimes takes longer than a millisecond should be done asynchronously. Parsing JSON input is fast and should be done synchronously.
This is not so obvious. There are asynchronous JSON parsers for Node. See:
https://www.npmjs.com/package/async-json-parse
https://www.npmjs.com/package/json-parse-async
But it's true that at some point for a CPU intensive operation you need to use asynchronous operations. I would say that any CPU-intensive logic shouldn't be done in the main thread blocking the event loop and should be done in external processes or worker, or in a thread spawned from C++ to make it maximally transparent to the user.
See how it is done in bcrypt and bcrypt-nodejs:
https://www.npmjs.com/package/bcrypt
https://www.npmjs.com/package/bcrypt-nodejs
If you can make your function work asynchronously (not only in a sense of using a callback but by actually not blocking the event loop) then I would recommend making at least two kinds of APIs - a function taking a callback and a function returning a promise (which can be one function in practice).
Currently with async/await you can use any function that returns a promise almost as if it were synchronous:
let x = await f();
let y = await g(x);
// ...
But there are some cases where you need a truly synchronous function, like if you want to have something that you can directly export from a module:
module.exports = f();
Here when the f() function is blocking there is no harm because the require() itself is blocking as well, and you should only use it once during startup. But if the function is asynchronous - by being declared with an async keyword and thus implicitly returning a promise, by explicitly returning a promise or by taking a callback, then you will not be able to export the value from a module and use it in certain ways.
So if you think that it makes sense that the return value of your function might be exported from modules then you may also need to provide a blocking, synchronous version.
Why not implement this as a synchronous AND as an asynchronous method with 2 function like cryptAsync() and cryptSync() ? I think it's better and not difficult for you to do.
Which part of syntax provides the information that this function should run in other thread and be non-blocking?
Let's consider simple asynchronous I/O in node.js
var fs = require('fs');
var path = process.argv[2];
fs.readFile(path, 'utf8', function(err,data) {
var lines = data.split('\n');
console.log(lines.length-1);
});
What exactly makes the trick that it happens in background? Could anyone explain it precisely or paste a link to some good resource? Everywhere I looked there is plenty of info about what callback is, but nobody explains why it actually works like that.
This is not the specific question about node.js, it's about general concept of callback in each programming language.
EDIT:
Probably the example I provided is not best here. So let's do not consider this node.js code snippet. I'm asking generally - what makes the trick that program keeps executing when encounter callback function. What is in syntax
that makes callback concept a non-blocking one?
Thanks in advance!
There is nothing in the syntax that tells you your callback is executed asynchronously. Callbacks can be asynchronous, such as:
setTimeout(function(){
console.log("this is async");
}, 100);
or it can be synchronous, such as:
an_array.forEach(function(x){
console.log("this is sync");
});
So, how can you know if a function will invoke the callback synchronously or asynchronously? The only reliable way is to read the documentation.
You can also write a test to find out if documentation is not available:
var t = "this is async";
some_function(function(){
t = "this is sync";
});
console.log(t);
How asynchronous code work
Javascript, per se, doesn't have any feature to make functions asynchronous. If you want to write an asynchronous function you have two options:
Use another asynchronous function such as setTimeout or web workers to execute your logic.
Write it in C.
As for how the C coded functions (such as setTimeout) implement asynchronous execution? It all has to do with the event loop (or mostly).
The Event Loop
Inside the web browser there is this piece of code that is used for networking. Originally, the networking code could only download one thing: the HTML page itself. When Mosaic invented the <img> tag the networking code evolved to download multiple resources. Then Netscape implemented progressive rendering of images, they had to make the networking code asynchronous so that they can draw the page before all images are loaded and update each image progressively and individually. This is the origin of the event loop.
In the heart of the browser there is an event loop that evolved from asynchronous networking code. So it's not surprising that it uses an I/O primitive as its core: select() (or something similar such as poll, epoll etc. depending on OS).
The select() function in C allows you to wait for multiple I/O operations in a single thread without needing to spawn additional threads. select() looks something like:
select (max, readlist, writelist, errlist, timeout)
To have it wait for an I/O (from a socket or disk) you'd add the file descriptor to the readlist and it will return when there is data available on any of your I/O channels. Once it returns you can continue processing the data.
The javascript interpreter saves your callback and then calls the select() function. When select() returns the interpreter figures out which callback is associated with which I/O channel and then calls it.
Conveniently, select() also allows you to specify a timeout value. By carefully managing the timeout passed to select() you can cause callbacks to be called at some time in the future. This is how setTimeout and setInterval are implemented. The interpreter keeps a list of all timeouts and calculates what it needs to pass as timeout to select(). Then when select() returns in addition to finding out if there are any callbacks that needs to be called due to an I/O operation the interpreter also checks for any expired timeouts that needs to be called.
So select() alone covers almost all the functionality necessary to implement asynchronous functions. But modern browsers also have web workers. In the case of web workers the browser spawns threads to execute javascript code asynchronously. To communicate back to the main thread the workers must still interact with the event loop (the select() function).
Node.js also spawns threads when dealing with file/disk I/O. When the I/O operation completes it communicates back with the main event loop to cause the appropriate callbacks to execute.
Hopefully this answers your question. I've always wanted to write this answer but was to busy to do so previously. If you want to know more about non-blocking I/O programming in C I suggest you take a read this: http://www.gnu.org/software/libc/manual/html_node/Waiting-for-I_002fO.html
For more information see also:
Is nodejs representing Reactor or Proactor design pattern?
Performance of NodeJS with large amount of callbacks
First of all, if something is not Async, it means it's blocking. So the javascript runner stops on that line until that function is over (that's what a readFileSync would do).
As we all know, fs is a IO library, so that kind of things take time (tell the hardware to read some files is not something done right away), so it makes a lot of sense that anything that does not require only the CPU, it's async, because it takes time, and does not need to freeze the rest of the code for waiting another piece of hardware (while the CPU is idle).
I hope this solves your doubts.
A callback is not necessarily asynchronous. Execution depends entirely on how fs.readFile decides to treat the function parameter.
In JavaScript, you can execute a function asynchronously using for example setTimeout.
Discussion and resources:
How does node.js implement non-blocking I/O?
Concurrency model and Event Loop
Wikipedia:
There are two types of callbacks, differing in how they control data flow at runtime: blocking callbacks (also known as synchronous callbacks or just callbacks) and deferred callbacks (also known as asynchronous callbacks).
I have been doing extensive reading on asynchronous programming for the web and use of callbacks in JavaScript and jQuery. I have understood the fundamentals of AJAX. What I cannot figure out is the use of callback functions when not used in asynchronous programming.
From my understanding, simply adding a callback to a function does not make it non-blocking/asynchronous. Asynchronous capability is actually provided by the environment (browser APIs). So adding a callback to a function that I have written will not lead to any asynchronous execution.
For example:
var X;
function Test(A, B, Callback) {
X=A+B*A*B;
Callback(X);
}
Test(99999,999999,function(Data) {
alert(Data);
});
alert("This is not printed first, as it would be in Async.");
In the above I'm taking two numbers and performing algebraic operations on them. Even though I'm using a callback function, the code execution will be blocked while the operations are performed. Then the callback will be executed, displaying an alert with the result of the computation. Then the next alert follows. Had I made an XMLHttpRequest instead of the algebraic operation, I would have got the second alert first because of asynchronous implementation. The code flow would not be blocked during the request as is happening during the mathematical operation.
Thus, what is the use of a callback in non-async calls when code execution is blocked even with a callback?
Some very common examples of synchroneous callbacks are the methods on Array.prototype: forEach, map, filter, etc.
The role of the callback is to provide a partial implementation that can be easily swapped in a larger algorithm. Some design patterns like template method and strategy come to mind.
You're right.
There is usually no reason to have a sync callback.
Exceptions include callbacks that might sometimes be async, or callbacks that can be raised more than once (eg, [].map()).
I use callbacks in the way you describe whenever I am working with complicated access methods which I don't want to rewrite, and I use them when I know the component will be run in a context the component doesn't know about, but am not particularly interested in developing or using an event emitter.
I do not know if other people find them as easy to work with as I do. So on a team, I try to get a sense of whether it is productive to use callbacks in the way you describe.
I am trying to grasp on Javascript Asynchronous functions and callbacks.
I got stuck on the concept of callback functions, where I am reading on some places: they are use to have sequential execution of code (mostly in context of jquery e.g animate)and some places specially in the context of Nodejs; they are use to have a parallel execution Asynchronous and avoid blocking of code.
So can some expert in this topic please shed light on this and clear this fuzz in my mind (examples??).
so I could make my mind for the usage of callback function
or that is solely depends on the place of where you are calling/placing a callback function in your code? .
Thanks,
P.S: I am scared that this question would be close as subjective but still I could expect concrete answer for this (perhaps some examples)
Edit: actually this is the example from internet which makes me ambigous:
function do_a(){
// simulate a time consuming function
setTimeout( function(){
console.log( '`do_a`: this takes longer than `do_b`' );
}, 1000 );
}
function do_b(){
console.log( '`do_b`: this is supposed to come out after `do_a` but it comes out before `do_a`' );
}
do_a();
do_b();
Result
`do_b`: this is supposed to come out after `do_a` but it comes out before `do_a`
`do_a`: this takes longer than `do_b`
when JS is sequential then do_b should always come after do_a according to my understanding.
The core of JavaScript is largely synchronous, in that functions complete their task fully, before completing. Prior to the advent of AJAX, it was really only setTimeout and setInterval that provided asynchronous behavior.
However, it's easy to forget that event handlers are, effectively async code. Attaching a handler does not invoke the handler code and that code isn't executed until some unknowable time in the future.
Then came AJAX, with its calls to the server. These calls could be configured to be synchronous, but developers generally preferred async calls and used callback methods to implement them.
Then, we saw the proliferation of JS libraries and toolkits. These strove to homogenize different browsers' implementations of things and built on the callback approach to async code. You also started to see a lot more synchronous callbacks for things like array iteration or CSS query result handling.
Now, we are seeing Deferreds and Promises in the mix. These are objects that represent the value of a long running operation and provide an API for handling that value when it arrives.
NodeJS leans towards an async approach to many things; that much is true. However this is more a design decision on their part, rather than any inherent async nature of JS.
Javascript is always a synchronous(blocking) single thread language but we can make Javascript act Asynchronous through programming.
Synchronous code:
console.log('a');
console.log('b');
Asynchronous code:
console.log('a');
setTimeout(function() {
console.log('b');
}, 1000);
setTimeout(function() {
console.log('c');
}, 1000);
setTimeout(function() {
console.log('d');
}, 1000);
console.log('e');
This outputs: a e b c d
In node long running processes use process.nextTick() to queue up the functions/callbacks. This is usually done in the API of node and unless your programming(outside the api) with something that is blocking or code that is long running then it doesn't really effect you much. The link below should explain it better then I can.
howtonode process.nextTick()
jQuery AJAX also takes callbacks and such as it its coded not to wait for server responses before moving on to the next block of code. It just rememebers the function to run when the server responds. This is based on XMLHTTPRequest object that the browsers expose. The XHR object will remember the function to call back when the response returns.
setTimeout(fn, 0) of javascript will run a function once the call stack is empty (next available free tick) which can be used to create async like features.setTimeout(fn, 0) question on stackoverflow
To summerise the async abilities of javascript is as much to do with the environments they are programmed in as javascript itself. You do not gain any magic by just using lots of function calls and callbacks unless your using some API/script.
Jquery Deferred Object Is another good link for async capabilities of jQuery. Googling might find you information on how jQuery Deferred works also for more insight.
In JavaScript the term "asynchronous" typically refers to code that gets executed when the call stack is empty and the engine picks a job from one of its job queues for execution.
Once code is being executed, it represents a synchronous sequence of execution, which continues until the call stack is empty again. This sequence of execution will not be interrupted by events in order to execute some other JavaScript code (when we discard Web Workers). In other words, a single JavaScript environment has no preemptive concurrency.
While synchronous execution is ongoing, events might be registered as jobs in some job queues, but the engine will not process those before first properly execution what is on the call stack. Only when the call stack is empty will the engine take notice of the job queues, pick one according to priority, and execute it (and that is called asynchronous).
Callbacks
Callbacks can be synchronous or asynchronous -- this really depends on how they are called.
For instance, here the callback is executed synchronously:
new Promise(function (resolve) { /* .... */ });
And here the callback is executed asynchronously:
setTimeout(function () { /* ... */ });
It really depends on the function that takes the callback as argument; how it deals with eventually calling that callback.
Ways to get code to execute asynchronously
The core ECMAScript language does not offer a lot of ways to do this. The well-known ones are offered via other APIs, such as the Web API, which are not part of the core language (setTimeout, setInterval, requestAnimationFrame, fetch, queueMicrotask, addEventListener, ...).
Core ECMAScript offers Promise.prototype.then and (depending on that) await. The callback passed to then is guaranteed to execute asynchronously. await will ensure that the next statements in the same function will be executed asynchronously: await makes the current function return, and this function's execution context will be restored and resumed by a job.
It also offers listening to when the garbage collector is about to garbage collect an object, with FinalizationRegistry.
Web Workers
Web Workers will execute in a separate execution environment, with their own call stack. Here preemptive concurrency is possible. When the term asynchronous is used in the JavaScript world, it typically does not refer to this kind of parallelism, although the communication with a Web Worker happens via asynchronous callback functions.
Javascript by default is "synchronous", it's the web APIs that handle "asynchronous" behaviour.
As for the setTimeout example,
console.log(...), in the global scope works straight away, while those inside functions wrapped inside setTimeout, wait inside the callback queue only to be pushed back on the call stack once ready. Thus they take time. Also, the time specified is not exact but the minimum time after which that piece of code can run anytime.
Thanks !