I'm having a bit of trouble understanding the mechanics of JS callbacks. I have a fair idea of how callbacks can be used in JS, but I do not understand how a callback is asynchronous.
For e.g., if my understanding is correct, a callback is of the nature:
db.query(param1, param2 , callback_fn1(){..} );
And the implementation of db.query() is along the lines of :
db.prototype.query = function(p1 , p2 , callback ){
//some code
callback();
}
How does the above implementation make db.query an asynchronous function? Does this not mean that a function called callback is passed to query and that function is called inside query? It looks like query is just another synchronous function. Could someone help me understand what I'm overlooking here? Thanks!
The code sample you've shown is actually still synchronous because is instructed to run immediately. An asynchronous callback is a callback that doesn't immediately need to be executed, so it doesn't block the event loop until you instruct it to run.
The most common way in Node.js to do this is with process.nextTick() which runs a specified function when the event loop call stack is empty. Here's an example:
var async = function(args, callback) {
// do some work
process.nextTick(function() {
callback(val);
});
};
Then we call the function like this:
async(args, function(val) {
console.log(val);
});
console.log('end');
In this example, the function async() and console.log('end') are added to the call stack. The call stack empties once both of those functions are run, and once it's empty, console.log(val) then runs.
If you're still confused, think of process.nextTick() as an optimized version of this code:
var fn = function() {};
setTimeout(fn, 0);
It basically means "run this function as soon as possible when you are not busy".
Edit: I just now realized the question is tagged with node.js. My answer is more about Javascript in the browser, #hexacyanide's answer is more about node.js. I guess knowing both doesn't hurt, though!
The way you posted it the code will indeed be blocking. For asynchronous behavior there are a few things you can utilize, such as
setTimeout and setInterval
Built-in, asynchronous methods such as from the FileReader API
Ajax requests
Web workers (see #html5rocks)
Your example code could be written as follows (fiddle):
function doStuff(callback) {
setTimeout(function () {
for (var i = 0; i < 1000; i++) {
// do some busy work
var x = Math.sqrt(i);
}
callback();
}, 0);
}
console.log('start');
doStuff(function () {
console.log('callback called');
});
console.log('after doStuff()');
The setTimeout call will allow the Javascript interpreter/compiler (however exactly they work these days) to run the function in a non-blocking matter which is why (most likely), you will see the output
start
after doStuff()
callback called
Note that asynchronousity is different from multi-threading. Javascript is still single-threaded (with the exception of web workers!).
A more in-depth explanation can be found for example here
Related
I want to preface by saying I've viewed a lot of stackoverflow questions regarding this topic, but I haven't found any 'duplicates' per se since none of them contain solutions that would solve this specific case.
I've mainly looked at How do I return the response from an asynchronous call?, and the section on 'Promises with async/await' would work inside an asynchronous function, but the function I'm working on is not async and is part of an existing codebase that I can't change easily. I wouldn't be able to change the function to async.
The section on callbacks wouldn't work either, which is explained further below. Anyway, I'll jump into my question:
I'm editing a function (standard function, not async) in JavaScript by adding an asynchronous function call. I want to wait until the async call finishes before I return from the function (since the result of the async call needs to be included in the return value). How would I do so?
I looked into using callbacks, which would allow me to write code which is guaranteed to run only after the async call completes. However, this wouldn't interrupt the flow of the program in the original function, and the original function could still return before the callback is run. A callback would allow me to execute something sequentially after the async function, but it wouldn't allow me to wait for asynchronous call to complete at the highest level.
Example code, which wouldn't return the desired result:
function getPlayers() {
... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall()
... other code ...
allPlayers.add(outfieldPlayers)
return allPlayers // the returned value may or may not include outfield players
}
The actual problem I'm facing is a little more complicated - I'm calling the async function in each iteration of a for loop, and need to wait until all calls have completed before returning. But, I think if I can solve this simpler problem, I can solve the problem with a for loop.
Sadly, it is pretty much impossible to wait for async code in a synchronous way. This is because there is no threading in JS (most JS runtimes, but some are). So code is either synchronous or asynchronous.
Asynchronous code is possible because of the event loop. The event loop is part of the javascript runtime. It works by keeping a stack of callback functions that run when events trigger them - usually either timeout events (which you can set with setTimeout()) or IO events (which happen when you make disk or HTTP requests, or on user interaction). However, these callbacks only run when no other code is running, so only when the program is idle and all functions have returned.
This means that techniques like "spin loops" (where you just run a loop until a condition is changed by another thread) that work in threaded environments don't work because the async code won't run until the spin loop finishes.
More Info: https://medium.com/front-end-weekly/javascript-event-loop-explained-4cd26af121d4
If you are using NodeJS, this is possible through execSync.
This requires you to place your asynchronous code in a separate file, spawn a separate process using execSync, which will wait until it exits.
For example, consider the following async function which prints an array.
// task.js
(async function() {
await new Promise((resolve) => setTimeout(() => {
console.log(JSON.stringify([3,4,5]));
resolve();
}, 1000));
})();
Now, you can invoke this from your main process:
function asyncGetOutfieldPlayersCall() {
const execSync = require('child_process').execSync;
return JSON.parse(execSync("node task.js"));
}
function getPlayers() {
let allPlayers = [1,2];
// ... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall();
// ... other code ...
allPlayers = allPlayers.concat(outfieldPlayers)
return allPlayers;
}
i have a little problem in understand of callbacks. I have read a lot in the last 2 days and what i have understand is the following (correct me, if i'm wrong):
JavaScript is a single thread language and you can program synchronous and asynchronous.
Synchronous means, each statement waits for the previous statement to finish before executing. That can lead into trouble, because if for instance a connection to a database needs a lot of time, the statements after the previous has to wait.
Finally that's very bad and that's why it's better to program asynchronous in Javascript, because Asynchronous code doesn't have to wait, the code can continue to run and the user don't have to wait.
To program asynchronous the Callbacks (functions of higher order) are needed.
Now i have try to program a little example by a lot of tutorials, etc.
function testCallback(a,callback){
console.log('1.function and given parameter: '+a);
callback(10);
}
testCallback(5 , function(x){
console.log("2.function and given parameter of 1. function: "+x);
});
Is that right? the output is:
1.function and given parameter: 5
2.function and given parameter of 1. function: 10
I do not understand, what the advantage is of this code, because i think that can still lead into trouble? If "console.log('1.function and....') has problems, the callback(10) function would even stop or not?
Thanks for any help!
JavaScript is a single thread language...
No, it isn't. The language says nothing about threading. Most environments give you a single thread per global environment, though (and on browsers you can create more, which interoperate through messaging). NodeJS provides just the one thread. Some environments (such as Rhino or Nashorn on the JDK) provide true multi-threading (and all the advantages and hassles that can involve).
Using a callback doesn't make code asynchronous. Your example, for instance, is not asynchronous. Consider:
function testCallback(a,callback){
console.log('1.function and given parameter: '+a);
callback(10);
}
console.log("Before the call");
testCallback(5 , function(x){
console.log("2.function and given parameter of 1. function: "+x);
});
console.log("After the call");
Note how we don't see After the call until after 2.function and given parameter of 1. function: 10. If the callback were asynchronous, we'd see it before:
function testCallback(a,callback){
console.log('1.function and given parameter: '+a);
setTimeout(function() { // Using setTimeout
callback(10); // to make this call
}, 0); // asynchronous
}
console.log("Before the call");
testCallback(5 , function(x){
console.log("2.function and given parameter of 1. function: "+x);
});
console.log("After the call");
Whether a callback is called synchronously or asynchronously depends entirely on what the function you're passing it to does. For instance, the callback used by Array#sort is called synchronously, but the callback used by setTimeout is called asynchronously.
For code to be asynchronous, it has to start an operation and then have that operation complete later, triggering a callback. setTimeout does that, as does ajax when used correctly, as do a wide range of other things.
Note that callbacks are currently how you handle asynchronous behavior (simple callbacks like the above, or promise callbacks), but the next specification (ES2017) will define built-in language semantics for dealing with asynchronousity without callbacks in the form of async and await. You can use that syntax today if you transpile with a tool like Babel.
In javascript callbacks can be synchronous or asynchronous. Synchronous callbacks can have a lot of benefits, but they don't do anything to stop your code blocking.
I think the best way to understand what asynchronous code is, and why it's beneficial, is to learn how Javascript actually evaluates your code. I recommend this video, which explains the process very clearly https://www.youtube.com/watch?v=8aGhZQkoFbQ
A JavaScript Callback Function is a function that is passed as a parameter to another JavaScript function. Callbacks can be synchronous or asynchronous. Simply passing a function as parameter will not change its current behavior.
It's behavior can be changed by the method which will execute it by calling it inside an event listener or setTimeout function etc. Basically event listener or setTimeout etc are handled by webapi in async fashion. If callback functions are inside these functions then these are moved to a queue by webapi when they are activated like button click(event listener) or time declared in setTimeout passed. They will move from queue to stack(if stack is empty) and run on stack finally.
The main advantage of using callback function can be seen from below code :-
var add = function(x,y) {
return x+y;
}
var multiply = function(x,y){
return x*y;
}
var calculate = function(x,y,callback){
return callback(x,y);
}
console.log(calculate(4,9,add));
I've written a couple of express apps in Node, and I have a shameful gap in my knowledge. Please, go easy on me.
Is it ever appropriate to use this traditional format for functions in Node.js?
function isGreaterThanZero ( val ){
return val > 0;
}
or should you always use a callback?
function isGreaterThanZero (val, cb){
if (val > 0) {cb(true);}
else {cb(false);}
}
The async nature of Node.js has my brain turned inside-out, and I'm questioning everything.
Of course it's appropriate
This is for synchronous code
function isGreaterThanZero(val){
return val > 0;
}
In lay terms, you'll use synchronous functions when there's nothing to wait for. You'll use synchronous functions outside of the exceptional cases noted below
For in-memory operations, things like basic calculations, string manipulations, array append, etc, you can use synchronous functions.
However, some synchronous functions do use callbacks but they're more for mapping or filtering purposes. In the comments, I see you asked about arr.filter. Other functions like arr.forEach, arr.map, or arr.reduce take callbacks but they're not implicitly asynchronous. In this case, the callback function is just used to augment the behavior of the called function.
This is for asynchronous code
function isGreaterThanZeroAsync(val, cb){
// some long process...
cb(val > 0);
}
You'll use asynchronous functions when you have something to wait for. Generally that's network, file system, or database access. An async function will allow you function to return (exit) immediately (as sync functions do) but the callback will get the value as soon as it's ready.
Here's a little async demo
function demoAsync(cb) {
console.log("demo start");
setTimeout(function() {
cb("hello world");
}, 1000);
}
console.log("init");
demoAsync(function(val) {
console.log(val);
console.log("demo done");
});
console.log("init done");
Output
init
demo start
init done
// 1000 ms later...
hello world
demo done
Why is this important?
You get "init", "demo start", and "init done" immediately. This demonstrates your program can continue while we're waiting for the demo to finish.
After 1000 ms we see "hello world" and "demo done".
Hooray! Everything happens in order.
I am reading The Node Beginner Book. In the chapter Event-driven asynchronous callbacks, the author gives an example to illustrate the idea of asynchronous callbacks. The code example is like:
var result = database.query("SELECT * FROM hugetable");
console.log("Hello World");
After adding a callback function to database.query, the code becomes asynchronous:
database.query("SELECT * FROM hugetable", function(rows) {
var result = rows;
});
console.log("Hello World");
My question is why the database.query() function becomes asynchronous simply after adding a callback function.
I have no experience with Javascript and JQuery before, that might be the reason I cannot understand.
There are many functions in node.js that have both an asynchronous flavor and a synchronous flavor. For example, there are two ways to read the contents of a file (docs):
//asynchronous
fs.readFile("filename.txt", function(err, data) {
});
//synchronous
var data = fs.readFileSync("filename.txt");
The example the author provides does in fact look somewhat confusing, but its possible that the database.query makes an asynchronous call depending on whether a callback is passed in as the second argument.
For example, it could be implemented something like this:
function query(queryString, callback) {
if(callback !== undefined) {
queryInternal(queryString, callback);
return;
}
else {
return queryInternalSync(queryString);
}
}
In general, I think the convention is that a function is either asynchronous or synchronous (not both) so your intuition is right.
Note that in the synchronous case, console.log will be executed after result has the queried contents whereas in the asynchronous case, console.log will be executed as soon as query function returns and before callback is executed.
Asynchronously means it don't waits for the response and go to the next statements
to be executed
In your second example the callback function handles your response while executing this,
it doesn't waits and console.log("Hello World"); shows the output in console.
Read this:
http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/
http://nodemanual.org/latest/nodejs_dev_guide/writing_asynchronous_code.html
http://www.sebastianseilund.com/nodejs-async-in-practice
I have a node application that is not a web application - it completes a series of asynchronous tasks before returning 1. Immediately before returning, the results of the program are printed to the console.
How do I make sure all the asynchronous work is completed before returning? I was able to achieve something similar to this in a web application by making sure all tasks we completed before calling res.end(), but I haven't any equivalent for a final 'event' to call before letting a script return.
See below for my (broken) function currently, attempting to wait until callStack is empty. I just discovered that this is a kind of nonsensical approach because node waits for processHub to complete before entering any of the asynchronous functions called in processObjWithRef.
function processHub(hubFileContents){
var callStack = [];
var myNewObj = {};
processObjWithRef(samplePayload, myNewObj, callStack);
while(callStack.length>0){
//do nothing
}
return 1
}
Note: I have tried many times previously to achieve this kind of behavior with libraries like async (see my related question at How can I make this call to request in nodejs synchronous?) so please take the answer and comments there into account before suggesting any answers based on 'just use asynch'.
You cannot wait for an asynchronous event before returning--that's the definition of asynchronous! Trying to force Node into this programming style will only cause you pain. A naive example would be to check periodically to see if callstack is empty.
var callstack = [...];
function processHub(contents) {
doSomethingAsync(..., callstack);
}
// check every second to see if callstack is empty
var interval = setInterval(function() {
if (callstack.length == 0) {
clearInterval(interval);
doSomething()
}
}, 1000);
Instead, the usual way to do async stuff in Node is to implement a callback to your function.
function processHub(hubFileContents, callback){
var callStack = [];
var myNewObj = {};
processObjWithRef(samplePayload, myNewObj, callStack, function() {
if (callStack.length == 0) {
callback(some_results);
}
});
}
If you really want to return something, check out promises; they are guaranteed to emit an event either immediately or at some point in the future when they are resolved.
function processHub(hubFileContents){
var callStack = [];
var myNewObj = {};
var promise = new Promise();
// assuming processObjWithRef takes a callback
processObjWithRef(samplePayload, myNewObj, callStack, function() {
if (callStack.length == 0) {
promise.resolve(some_results);
}
});
return promise;
}
processHubPromise = processHub(...);
processHubPromise.then(function(result) {
// do something with 'result' when complete
});
The problem is with your design of the function. You want to return a synchronous result from a list of tasks that are executed asynchronously.
You should implement your function with an extra parameter that will be the callback where you would put the result (in this case, 1) for some consumer to do something with it.
Also you need to have a callback parameter in your inner function, otherwise you won't know when it ends. If this last thing is not possible, then you should do some kind of polling (using setInterval perhaps) to test when the callStack array is populated.
Remember, in Javascript you should never ever do a busy wait. That will lock your program entirely as it runs on a single process.
deasync is desinged to address your problem exactly. Just replace
while(callStack.length>0){
//do nothing
}
with
require('deasync').loopWhile(function(){return callStack.length>0;});
The problem is that node.js is single-threaded, which means that if one function runs, nothing else runs (event-loop) until that function has returned. So you can not block a function to make it return after async stuff is done.
You could, for example, set up a counter variable that counts started async tasks and decrement that counter using a callback function (that gets called after the task has finished) from your async code.
Node.js runs on A SINGLE threaded event loop and leverages asynchronous calls for doing various things, like I/O operations.
if you need to wait for a number of asynchronous operations to finish before executing additional code
you can try using Async -
Node.js Async Tutorial
You'll need to start designing and thinking asynchronously, which can take a little while to get used to at first. This is a simple example of how you would tackle something like "returning" after a function call.
function doStuff(param, cb) {
//do something
var newData = param;
//"return"
cb(newData);
}
doStuff({some:data}, function(myNewData) {
//you're done with doStuff in here
});
There's also a lot of helpful utility functions in the async library available on npm.