Develop Reference

Javascript - wait for async call to finish before returning from function, without the use of callbacks

I want to preface by saying I've viewed a lot of stackoverflow questions regarding this topic, but I haven't found any 'duplicates' per se since none of them contain solutions that would solve this specific case.
I've mainly looked at How do I return the response from an asynchronous call?, and the section on 'Promises with async/await' would work inside an asynchronous function, but the function I'm working on is not async and is part of an existing codebase that I can't change easily. I wouldn't be able to change the function to async.
The section on callbacks wouldn't work either, which is explained further below. Anyway, I'll jump into my question:
I'm editing a function (standard function, not async) in JavaScript by adding an asynchronous function call. I want to wait until the async call finishes before I return from the function (since the result of the async call needs to be included in the return value). How would I do so?
I looked into using callbacks, which would allow me to write code which is guaranteed to run only after the async call completes. However, this wouldn't interrupt the flow of the program in the original function, and the original function could still return before the callback is run. A callback would allow me to execute something sequentially after the async function, but it wouldn't allow me to wait for asynchronous call to complete at the highest level.
Example code, which wouldn't return the desired result:
function getPlayers() {
... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall()
... other code ...
allPlayers.add(outfieldPlayers)
return allPlayers // the returned value may or may not include outfield players
}
The actual problem I'm facing is a little more complicated - I'm calling the async function in each iteration of a for loop, and need to wait until all calls have completed before returning. But, I think if I can solve this simpler problem, I can solve the problem with a for loop.

Sadly, it is pretty much impossible to wait for async code in a synchronous way. This is because there is no threading in JS (most JS runtimes, but some are). So code is either synchronous or asynchronous.
Asynchronous code is possible because of the event loop. The event loop is part of the javascript runtime. It works by keeping a stack of callback functions that run when events trigger them - usually either timeout events (which you can set with setTimeout()) or IO events (which happen when you make disk or HTTP requests, or on user interaction). However, these callbacks only run when no other code is running, so only when the program is idle and all functions have returned.
This means that techniques like "spin loops" (where you just run a loop until a condition is changed by another thread) that work in threaded environments don't work because the async code won't run until the spin loop finishes.
More Info: https://medium.com/front-end-weekly/javascript-event-loop-explained-4cd26af121d4

If you are using NodeJS, this is possible through execSync.
This requires you to place your asynchronous code in a separate file, spawn a separate process using execSync, which will wait until it exits.
For example, consider the following async function which prints an array.
// task.js
(async function() {
await new Promise((resolve) => setTimeout(() => {
console.log(JSON.stringify([3,4,5]));
resolve();
}, 1000));
})();
Now, you can invoke this from your main process:
function asyncGetOutfieldPlayersCall() {
const execSync = require('child_process').execSync;
return JSON.parse(execSync("node task.js"));
}
function getPlayers() {
let allPlayers = [1,2];
// ... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall();
// ... other code ...
allPlayers = allPlayers.concat(outfieldPlayers)
return allPlayers;
}

Async callback for EventEmitter

Is it possible to do provide an async callback to an EventEmitter in TypeScript or JavaScript?
someEmitter.on("anEvent", async () => console.log("hello"));
Will this cause the function to be run asynchronously? If so, why would one ever not use an async function on an EventEmitter?

Is it possible to do provide an async callback to an EventEmitter in TypeScript or JavaScript?
Yes, you can provide an async function to an eventEmitter and that will allow you to use await inside the callback, but it does not magically make your function run asynchronously.
It also will not change anything outside of your callback function. The eventEmitter will call your callback when the event occurs. Your callback will end up returning a promise that the eventEmitter object will not do anything with.
So, you can do it in order to use await inside your callback function (for your own internal reasons), but it doesn't change anything else outside the callback. It doesn't change how the eventEmitter does the rest of its business in any way.
Will this cause the function to be run asynchronously?
No. It doesn't change how your function is called. If your function consists entirely of synchronous code, it will still be called and run synchronously.
FYI, this is true for any function declared async. That does not change how the function is called in any way. It changes a few things that you can do inside the function and it forces the function to return a promise, but it doesn't change how the function is called. Synchronous code is still synchronous code whether its in an async function or a regular function. I'd really suggest you read a lot more about what an async function actually is.

How to properly use Javascript Async eachLimit

I'm having problems with using async.eachLimit. It works properly for the first 10 elements, but it doesn't continue past that; it simply ends. So, if there are 100 elements, it only does the first 10. This is clearly an issue of me misunderstanding callbacks. What is the proper way of using eachLimit with an external function that does not contain a callback? Or is it required for such a function to have one?
async.eachLimit(items, 10, function(item, callback) {
outsideFunction(item.attrOne, item.attrTwo};
//callback(); ---> leads to all running in parallel.
},
function(err) {
console.log(err);
}
);

Your issue here is you're using an async library for a function that isn't asynchronous (or isn't acting like it's asynchronous). What async.eachLimit does is go over each item in the array, only executing limit amount at a time, and waits for callback() to be called saying the current iteration is finished and can add another one to be executed.
In your code example, the callback (when uncommented) gets called immediately after it tries calling outsideFunction because the function call is non-blocking. It doesn't wait because async says "I've been told it's done, I'll go onto the next one" so all 100 will try and be executed at the same time. If outsideFunction is an asynchronous function, it needs a callback (or have it use promises) to say it has finished executing, and inside that callback you call the callback for async.eachLimit and then it will only do 10 at a time in the way you want. Here's an example:
async.eachLimit(items, 10, function(item, callback)
{
outsideFunction(item.attrOne, item.attrTwo, function(someResult)
{
// Your outside function calls back saying it's finished ...
callback(); // ... so we tell async we're done
});
},
function(err)
{
console.log(err);
});
If outsideFunction isn't your function, and the function is actually asynchronous, then it's either using promises or you need to find a library that writes asynchronous functions properly. If the function isn't asynchronous, then async.eachLimit won't work.
If it's your function, you should make it send back a callback to say it's done (or use promises).

How do you create custom asynchronous functions in node.js?

I was unsure how node.js was able to realize what functions where async and which were not and how to create a custom async function.
Say I wanted to create a custom asynchronous function. I would be surprised if just because I called my last argument to the async function callback or cb that it would just know its an async function:
function f(arg1, callback){
//do stuff with arg1
arg1.doStuff()
//call callback
callback(null, arg1.result());
}
I tried something like that and it did not work async. How do you tell node.js that f is actually async?

NOTE: this answer was written in 2014, before the existence of async function, and before Promises gaining popularity. While the same principles apply as well, I would recommend reading on Promises before trying to get your head around how they relate to "traditional" callback-driven async functions.
To create a function that calls its callback asynchronously, you have to use some platform-provided async primitive (typically IO-related) on it - timers, reading from the filesystem, making a request etc.
For example, this function takes a callback argument, and calls it 100ms after:
function asyncFn(callback) {
setTimeout(() => {
callback();
}, 100);
}
A possible reason for making a function async when it doesn't need to be, is for API consistency. For example, suppose you have a function that makes a network request, and caches the result for later calls:
var cache = null;
function makeRequest(callback) {
if (!cache) {
makeAjax(result => {
cache = result;
callback(result);
});
} else {
callback(cache);
}
}
The problem is, this function is inconsistent: sometimes it is asynchronous, sometimes it isn't. Suppose you have a consumer like this:
makeRequest(result => doSomethingWithResult(result));
doSomethingElse();
The doSomethingElse function may run before or after the doSomethingWithResult function, depending on whether the result was cached or not. Now, if you use an async primitive on the makeRequest function, such as process.nextTick:
var cache = null;
function makeRequest(callback) {
if(!cache) {
makeAjax(result => {
cache = result;
callback(result);
});
} else {
process.nextTick(() => callback(cache));
}
}
The call is always async, and doSomethingElse always runs before doSomethingWithResult.

Only native functions (with access to the event loop) are asynchronous. You would need to call one of them to get asynchronity for your callback. See What is a simple example of an asynchronous javascript function?.
If you aren't using any, there's hardly a reason to make your function asynchronous.

AJAX workflow: How do I order the execution of these functions?

I'm trying to figure the best way to get my functions executing in the correct order.
I have 3 functions
function 1 - squirts OPTIONs into a SELECT via JSON and marks them as selected
function 2 - squirts OPTIONS into a 2nd SELECT and marks them as selected
function 3 - gets the values from the above SELECTs along with some additional INPUT values, does an AJAX GET resulting in JSON data, which is read and populates a table.
With JQuery Onload, I execute:
function1();
function2();
function3();
I'm finding function3 is executing before the SELECTs have been populated with OPTIONS and hence the table has no results, because the values sent in the GET were blank.
I know this is probably a very simple problem and that there are probably a dozen ways to accomplish this, but basically I need the best way to code this so that function3 only runs if function1 and 2 are complete.
I've come into Javascript via the back door having learnt the basics of JQuery first!
Thanks for your assistance.

Javascript executes synchronously, which means that function3 must wait for function2 to complete, which must wait for function1 to complete before executing.
The exception is when you run code that is asynchronous, like a setTimeout, setInterval or an asynchronous AJAX request.
Any subsequent code that relies on the completion of such asynchronous code needs to be called in such a manner that it doesn't execute until the asynchronous code has completed.
In the case of the setTimeout, you could just place the next function call at the end of the function you're passing to the setTimeout.
In the case of an AJAX call, you can place the next function call in a callback that fires upon a completed request.
If you don't want the execution of the subsequent function to occur every time, you can modify your functions to accept a function argument that gets called at the end of the asynchronous code.
Something like:
function function1( fn ) {
setTimeout(function() {
// your code
// Call the function parameter if it exists
if( fn ) {
fn();
}
}, 200);
}
function function2() {
// some code that must wait for function1
}
onload:
// Call function1 and pass function2 as an argument
function1( function2 );
// ...or call function1 without the argument
function1();
// ...or call function2 independently of function1
function2();

I recommend you use a Promises library. You can hack simple solutions like other answers suggest, but as your application grows, you'll find you are doing more and more of these hacks. Promises are intended to solve these kinds of problems when dealing with asynchronous calls.
The CommonJS project has several Promises proposals which you should check out. Here is a question I asked on SO about Promises a while back with links to other solutions. Learn more about Promises in this Douglas Crockford video. The whole video is good, but skip to just past half way for promises.
I'm using the FuturesJS library currently as it suits my needs. But there are advantages to other implementations as well. It allows you to do sequences very easily:
// Initialize Application
Futures.sequence(function (next) {
// First load the UI description document
loadUI(next); // next() is called inside loadUI
})
.then(function(next) {
// Then load all templates specified in the description
loadTemplates(next); // next() is called inside loadTemplates
})
.then(function(next) {
// Then initialize all templates specified in the description
initTemplates();
});
Even more powerful is when you need to join async events together and do another action when all of the other async events have completed. Here's an example (untested) that will load a bunch of HTML files and then perform an action only once ALL of them have completed loading:
var path = "/templates/",
templates = ["one.html","two.html","three.html"],
promises = [];
$.each(templates, function(i,name) {
promises[i] = Futures.promise();
var $container = $("<div>");
$container.load(path+name, function(response,status,xhr) {
promises[i].fullfill();
}
});
Futures.join(promises, {timeout: 10000}) // Fail if promises not completed in 10 seconds
.when(function(p_arr) {
console.log("All templates loaded");
})
.fail(function(p_arr) {
console.log("Error loading templates");
});
This might be overkill for your application. But if the application is growing in complexity, using promises will help you in the long run.
I hope this helps!

invoke function2 inside of function1 and function3 inside of function2.

It's not clear why f1 and f2 are executing before f3.
Also, are you using the preferred $(document).ready() or some variation of onload?
It might be helpful if you provide a reproducible test case.

fun3() will only run after both are ready. It might run twice. You can fix this with a lock inside fun3() you would need a Singleton to guarantee it works correctly.
var select1ready = false, select2ready = false;
fun1()
{
// do stuff
select1ready = true;
fun3();
}
fun2()
{
// do stuff
select2ready = true;
fun3();
}
fun3()
{
if (select1ready && select2ready)
{
}
}
fun1();
fun2();