Develop Reference

Running multiple functions asynchronously in relation to each other but synchronously in relation to program as a whole

Essentially, I have a try block in an async function that basically looks like the following:
async function functionName() {
try {
await function1() // each of function 1, 2, and 3 are async functions
await function2()
await function3()
//some more awaits below this point
} catch(err) {
console.log(err)
}
}
Essentially, I'd like to run function1, function2, and function3 asynchronously in relation to each other (AKA have the three of them run at the same time) but I'd like to wait for all of them to finish before continuing the program (so have the set of 3 run synchronously in relation to the rest of the function). I can't just put the code from function2 and function3 inside of function1 for a variety of reasons, so unfortunately that's not an option. Any help much appreciated!

Assuming all your functions return promises that resolve/reject when the asynchronous operations are done, then you can use Promise.all() to let them all be in-flight at the same time and be notified when they are all done:
async function functionName() {
try {
let results = await Promise.all([function1(), function2(), function3()]); //some more awaits below this point
} catch(err) {
console.log(err)
}
}
Or, if you don't want the short-circuit if there's an error, you can use Promise.allSettled() instead and it will notify you only when all requests are done, regardless of whether any of them had an error.
Keep in mind that only truly non-blocking, asynchronous operations in your functions will actually run in parallel. Any blocking, synchronous code in your functions will still only run one at a time. And, keep in mind that just because a function is tagged as async does not make anything run asynchronously. All the async keyword does is allow you to use await inside the function and make the function automatically return a promise. It does not make anything in the function run asynchronously.
If you showed your real code rather than pseudo-code, we could comment more specifically about exactly what you're doing and the best way to code it. Please resist the temptation to post questions with only pseudo-code. We can always help better when we see your real code.
Also, none of this runs synchronously with regard to the program as a whole. It will appear to run synchronously within the body of the function (as the function execution will be suspended by the await). But, as soon as the function hits the await, it immediately returns an unresolved promise and execution after this function call continues. The only way to run someting synchronously with regard to the program as a whole is to use synchronous coding, not non-blocking, asynchronous coding.

Javascript - wait for async call to finish before returning from function, without the use of callbacks

I want to preface by saying I've viewed a lot of stackoverflow questions regarding this topic, but I haven't found any 'duplicates' per se since none of them contain solutions that would solve this specific case.
I've mainly looked at How do I return the response from an asynchronous call?, and the section on 'Promises with async/await' would work inside an asynchronous function, but the function I'm working on is not async and is part of an existing codebase that I can't change easily. I wouldn't be able to change the function to async.
The section on callbacks wouldn't work either, which is explained further below. Anyway, I'll jump into my question:
I'm editing a function (standard function, not async) in JavaScript by adding an asynchronous function call. I want to wait until the async call finishes before I return from the function (since the result of the async call needs to be included in the return value). How would I do so?
I looked into using callbacks, which would allow me to write code which is guaranteed to run only after the async call completes. However, this wouldn't interrupt the flow of the program in the original function, and the original function could still return before the callback is run. A callback would allow me to execute something sequentially after the async function, but it wouldn't allow me to wait for asynchronous call to complete at the highest level.
Example code, which wouldn't return the desired result:
function getPlayers() {
... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall()
... other code ...
allPlayers.add(outfieldPlayers)
return allPlayers // the returned value may or may not include outfield players
}
The actual problem I'm facing is a little more complicated - I'm calling the async function in each iteration of a for loop, and need to wait until all calls have completed before returning. But, I think if I can solve this simpler problem, I can solve the problem with a for loop.

Sadly, it is pretty much impossible to wait for async code in a synchronous way. This is because there is no threading in JS (most JS runtimes, but some are). So code is either synchronous or asynchronous.
Asynchronous code is possible because of the event loop. The event loop is part of the javascript runtime. It works by keeping a stack of callback functions that run when events trigger them - usually either timeout events (which you can set with setTimeout()) or IO events (which happen when you make disk or HTTP requests, or on user interaction). However, these callbacks only run when no other code is running, so only when the program is idle and all functions have returned.
This means that techniques like "spin loops" (where you just run a loop until a condition is changed by another thread) that work in threaded environments don't work because the async code won't run until the spin loop finishes.
More Info: https://medium.com/front-end-weekly/javascript-event-loop-explained-4cd26af121d4

If you are using NodeJS, this is possible through execSync.
This requires you to place your asynchronous code in a separate file, spawn a separate process using execSync, which will wait until it exits.
For example, consider the following async function which prints an array.
// task.js
(async function() {
await new Promise((resolve) => setTimeout(() => {
console.log(JSON.stringify([3,4,5]));
resolve();
}, 1000));
})();
Now, you can invoke this from your main process:
function asyncGetOutfieldPlayersCall() {
const execSync = require('child_process').execSync;
return JSON.parse(execSync("node task.js"));
}
function getPlayers() {
let allPlayers = [1,2];
// ... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall();
// ... other code ...
allPlayers = allPlayers.concat(outfieldPlayers)
return allPlayers;
}

Is there a practical advantage to returning a pending promise? [duplicate]

This question already has answers here:
Difference between `return await promise` and `return promise`
(7 answers)
Closed 4 years ago.
For the most part, I think I understand how async functions work in JavaScript / Node.js, and I am familiar with the syntax of async/await. But this question is about a very specific case, which I can’t seem to find anywhere, and I think it involves the inner workings of await, which is beyond my understanding.
First I’ll start off with a simple example, and then I’ll extend it to real working code. (Note: I’m actually using Typescript, so you’ll see void instead of undefined, but that’s not important to the problem.)
async function subtask() {
// doSomeStuff();
return; // returns a `Promise<void>`
}
// async function run() {
// see options below
// }
function app() {
run().catch((e) => console.error(e));
}
Option 1
async function run() {
await subtask(); // resolves to `void`
return; // returns a new `Promise<void>`
}
Option 2
async function run() {
return subtask(); // returns a new Promise, resolving to the `Promise<void>` of `subtask()`
}
In the simple example above, I have an async function run that calls a smaller async function subtask. Both functions must return a Promise<void>. I have two options: (1) to await the smaller function and return a new Promise<void>, or (2) to return a wrapped promise, given by the smaller function, which will later resolve to void.
My lack of understanding is about how this works. In Option 1, is execution is paused before subtask() returns? What does that actually mean? Does that mean that the async subtask executes synchronously? Does that mean that app, which calls run(), will also pause execution? What if app was async, would that make a difference?
Is it “better” (more performant) to let the promise bubble up and resolve later, or to resolve it directly within the run function?
The reason this is important is that in my real code I have a bunch of smaller subtasks, which all return void, and then the big function must also return void—it cannot return an array. (Note that the subtasks need not run in any particular order.)
Option 1
async function run() {
await Promise.all([
subtask0(),
subtask1(),
subtask2(),
]);
return;
}
Option 2
async function run() {
return Promise.all([
subtask0(),
subtask1(),
subtask2(),
]).then((_all) => {});
}
function app() {
// do some stuff
run(); // if `run` contains `await`, does execution pause here?
// what if `app` was async?
// do some more stuff
}

It is always better to let the Promise bubble up. This avoids the creation of an extra Promise object that will also be awaited on (whether this is optimized away behind the scenes in whichever JavaScript engine you're using is up to debate, though).
async function run() {
await subtask();
return;
}
This creates an extra Promise (and subsequently an extra callback in the chain that will be executed).
async function run() {
return subtask();
}
This isn't actually doing what you think it is. This is also creating an extra Promise (since you're using the async keyword) and is almost functionally the same as the previous example. By using the async keyword, you're creating and returning a new Promise that will resolve/reject to the same value as the Promise created from the call to subtask(). If you remove the async keyword, then this will avoid the creation of the extra unnecessary Promise .
Now with your Promise.all() examples, I think they're both optimal (assuming you remove the unnecessary async keyword from the 2nd one, like noted above). The first would create 2 Promise s (the Promise.all() one, and the one created from returning from an async function), and the 2nd would as well (the Promise.all() one, and the one created from calling then()). Whichever you'd want to use is basically up to personal choice. Personally, I like the 2nd example since it's not mixing the use of async functions and Promises, which I think makes it a little easier to understand.
And for the last part of your question, execution will be paused wherever the await keyword exists (so within the run() call).
You can also essentially think of the await keyword transforming your code from this:
// Do some stuff
let result = await run();
// Do some other stuff
to this:
// Do some stuff
run().then(result => {
// Do some other stuff
};
(the whole point of await is to reduce confusion/complexity caused by nesting)

Using "await" inside non-async function

I have an async function that runs by a setInterval somewhere in my code. This function updates some cache in regular intervals.
I also have a different, synchronous function which needs to retrieve values - preferably from the cache, yet if it's a cache-miss, then from the data origins
(I realize making IO operations in a synchronous manner is ill-advised, but lets assume this is required in this case).
My problem is I'd like the synchronous function to be able to wait for a value from the async one, but it's not possible to use the await keyword inside a non-async function:
function syncFunc(key) {
if (!(key in cache)) {
await updateCacheForKey([key]);
}
}
async function updateCacheForKey(keys) {
// updates cache for given keys
...
}
Now, this can be easily circumvented by extracting the logic inside updateCacheForKey into a new synchronous function, and calling this new function from both existing functions.
My question is why absolutely prevent this use case in the first place? My only guess is that it has to do with "idiot-proofing", since in most cases, waiting on an async function from a synchronous one is wrong. But am I wrong to think it has its valid use cases at times?
(I think this is possible in C# as well by using Task.Wait, though I might be confusing things here).

My problem is I'd like the synchronous function to be able to wait for a value from the async one...
They can't, because:
JavaScript works on the basis of a "job queue" processed by a thread, where jobs have run-to-completion semantics, and
JavaScript doesn't really have asynchronous functions — even async functions are, under the covers, synchronous functions that return promises (details below)
The job queue (event loop) is conceptually quite simple: When something needs to be done (the initial execution of a script, an event handler callback, etc.), that work is put in the job queue. The thread servicing that job queue picks up the next pending job, runs it to completion, and then goes back for the next one. (It's more complicated than that, of course, but that's sufficient for our purposes.) So when a function gets called, it's called as part of the processing of a job, and jobs are always processed to completion before the next job can run.
Running to completion means that if the job called a function, that function has to return before the job is done. Jobs don't get suspended in the middle while the thread runs off to do something else. This makes code dramatically simpler to write correctly and reason about than if jobs could get suspended in the middle while something else happens. (Again it's more complicated than that, but again that's sufficient for our purposes here.)
So far so good. What's this about not really having asynchronous functions?!
Although we talk about "synchronous" vs. "asynchronous" functions, and even have an async keyword we can apply to functions, a function call is always synchronous in JavaScript. An async function is a function that synchronously returns a promise that the function's logic fulfills or rejects later, queuing callbacks the environment will call later.
Let's assume updateCacheForKey looks something like this:
async function updateCacheForKey(key) {
const value = await fetch(/*...*/);
cache[key] = value;
return value;
}
What that's really doing, under the covers, is (very roughly, not literally) this:
function updateCacheForKey(key) {
return fetch(/*...*/).then(result => {
const value = result;
cache[key] = value;
return value;
});
}
(I go into more detail on this in Chapter 9 of my recent book, JavaScript: The New Toys.)
It asks the browser to start the process of fetching the data, and registers a callback with it (via then) for the browser to call when the data comes back, and then it exits, returning the promise from then. The data isn't fetched yet, but updateCacheForKey is done. It has returned. It did its work synchronously.
Later, when the fetch completes, the browser queues a job to call that promise callback; when that job is picked up from the queue, the callback gets called, and its return value is used to resolve the promise then returned.
My question is why absolutely prevent this use case in the first place?
Let's see what that would look like:
The thread picks up a job and that job involves calling syncFunc, which calls updateCacheForKey. updateCacheForKey asks the browser to fetch the resource and returns its promise. Through the magic of this non-async await, we synchronously wait for that promise to be resolved, holding up the job.
At some point, the browser's network code finishes retrieving the resource and queues a job to call the promise callback we registered in updateCacheForKey.
Nothing happens, ever again. :-)
...because jobs have run-to-completion semantics, and the thread isn't allowed to pick up the next job until it completes the previous one. The thread isn't allowed to suspend the job that called syncFunc in the middle so it can go process the job that would resolve the promise.
That seems arbitrary, but again, the reason for it is that it makes it dramatically easier to write correct code and reason about what the code is doing.
But it does mean that a "synchronous" function can't wait for an "asynchronous" function to complete.
There's a lot of hand-waving of details and such above. If you want to get into the nitty-gritty of it, you can delve into the spec. Pack lots of provisions and warm clothes, you'll be some time. :-)
Jobs and Job Queues
Execution Contexts
Realms and Agents

You can call an async function from within a non-async function via an Immediately Invoked Function Expression (IIFE):
(async () => await updateCacheForKey([key]))();
And as applied to your example:
function syncFunc(key) {
if (!(key in cache)) {
(async () => await updateCacheForKey([key]))();
}
}
async function updateCacheForKey(keys) {
// updates cache for given keys
...
}

This shows how a function can be both sync and async, and how the Immediately Invoked Function Expression idiom is only immediate if the path through the function being called does synchronous things.
function test() {
console.log('Test before');
(async () => await print(0.3))();
console.log('Test between');
(async () => await print(0.7))();
console.log('Test after');
}
async function print(v) {
if(v<0.5)await sleep(5000);
else console.log('No sleep')
console.log(`Printing ${v}`);
}
function sleep(ms : number) {
return new Promise(resolve => setTimeout(resolve, ms));
}
test();
(Based off of Ayyappa's code in a comment to another answer.)
The console.log looks like this:
16:53:00.804 Test before
16:53:00.804 Test between
16:53:00.804 No sleep
16:53:00.805 Printing 0.7
16:53:00.805 Test after
16:53:05.805 Printing 0.3
If you change the 0.7 to 0.4 everything runs async:
17:05:14.185 Test before
17:05:14.186 Test between
17:05:14.186 Test after
17:05:19.186 Printing 0.3
17:05:19.187 Printing 0.4
And if you change both numbers to be over 0.5, everything runs sync, and no promises get created at all:
17:06:56.504 Test before
17:06:56.504 No sleep
17:06:56.505 Printing 0.6
17:06:56.505 Test between
17:06:56.505 No sleep
17:06:56.505 Printing 0.7
17:06:56.505 Test after
This does suggest an answer to the original question, though. You could have a function like this (disclaimer: untested nodeJS code):
const cache = {}
async getData(key, forceSync){
if(cache.hasOwnProperty(key))return cache[key] //Runs sync
if(forceSync){ //Runs sync
const value = fs.readFileSync(`${key}.txt`)
cache[key] = value
return value
}
//If we reach here, the code will run async
const value = await fsPromises.readFile(`${key}.txt`)
cache[key] = value
return value
}

Now, this can be easily circumvented by extracting the logic inside updateCacheForKey into a new synchronous function, and calling this new function from both existing functions.
T.J. Crowder explains the semantics of async functions in JavaScript perfectly. But in my opinion the paragraph above deserves more discussion. Depending on what updateCacheForKey does, it may not be possible to extract its logic into a synchronous function because, in JavaScript, some things can only be done asynchronously. For example there is no way to perform a network request and wait for its response synchronously. If updateCacheForKey relies on a server response, it can't be turned into a synchronous function.
It was true even before the advent of asynchronous functions and promises: XMLHttpRequest, for instance, gets a callback and calls it when the response is ready. There's no way of obtaining a response synchronously. Promises are just an abstraction layer on callbacks and asynchronous functions are just an abstraction layer on promises.
Now this could have been done differently. And it is in some environments:
In PHP, pretty much everything is synchronous. You send a request with curl and your script blocks until it gets a response.
Node.js has synchronous versions of its file system calls (readFileSync, writeFileSync etc.) which block until the operation completes.
Even plain old browser JavaScript has alert and friends (confirm, prompt) which block until the user dismisses the modal dialog.
This demonstrates that the designers of the JavaScript language could have opted for synchronous versions of XMLHttpRequest, fetch etc. Why didn't they?
[W]hy absolutely prevent this use case in the first place?
This is a design decision.
alert, for instance, prevents the user from interacting with the rest of the page because JavaScript is single threaded and the one and only thread of execution is blocked until the alert call completes. Therefore there's no way to execute event handlers, which means no way to become interactive. If there was a syncFetch function, it would block the user from doing anything until the network request completes, which can potentially take minutes, even hours or days.
This is clearly against the nature of the interactive environment we call the "web". alert was a mistake in retrospect and it should not be used except under very few circumstances.
The only alternative would be to allow multithreading in JavaScript which is notoriously difficult to write correct programs with. Are you having trouble wrapping your head around asynchronous functions? Try semaphores!

It is possible to add a good old .then() to the async function and it will work.
Should consider though instead of doing that, changing your current regular function to async one, and all the way up the call stack until returned promise is not needed, i.e. there's no work to be done with the value returned from async function. In which case it actually CAN be called from a synchronous one.

ES2017 - Async vs. Yield

I am confused about the current discussion of adding async functions and the keyword await to the next EcmaScript.
I do not understand why it is necessary to have the async keyword before the function keyword.
From my point of view the await keyword to wait for a result of a generator or promise done, a function's return should be enough.
await should simple be usable within normal functions and generator functions with no additional async marker.
And if I need to create a function what should be usable as an result for an await, I simply use a promise.
My reason for asking is this good explanation, where the following example comes from:
async function setupNewUser(name) {
var invitations,
newUser = await createUser(name),
friends = await getFacebookFriends(name);
if (friends) {
invitations = await inviteFacebookFriends(friends);
}
// some more logic
}
It also could be done as normal function, if the execution of a function will wait for finishing the hole function until all awaits are fulfilled.
function setupNewUser(name) {
var invitations,
newUser = await createUser(name),
friends = await getFacebookFriends(name);
if (friends) {
invitations = await inviteFacebookFriends(friends);
}
// return because createUser() and getFacebookFriends() and maybe inviteFacebookFriends() finished their awaited result.
}
In my opinion the whole function execution is holding until the next tick (await fulfillment) is done. The difference to Generator-Function is that the next() is triggering and changing the object's value and done field. A function instead will simple give back the result when it is done and the trigger is a function internal trigger like a while-loop.

I do not understand why it is necessary to have the async keyword before the function keyword.
For the same reason that we have the * symbol before generator functions: They mark the function as extraordinary. They are quite similar in that regard - they add a visual marker that the body of this function does not run to completion by itself, but can be interleaved arbitrarily with other code.
The * denotes a generator function, which will always return a generator that can be advanced (and stopped) from outside by consuming it similar to an iterator.
The async denotes an asynchronous function, which will always return a promise that depends on other promises and whose execution is concurrent to other asynchronous operations (and might be cancelled from outside).
It's true that the keyword is not strictly necessary and the kind of the function could be determined by whether the respective keywords (yield(*)/await) appear in its body, but that would lead to less maintainable code:
less comprehensible, because you need to scan the whole body to determine the kind
more errorprone, because it's easy to break a function by adding/removing those keywords without getting a syntax error
a normal function, whose execution will wait for finishing the hole body until all awaits are fulfilled
That sounds like you want a blocking function, which is a very bad idea in a concurrent setting.

By marking a function as async, you're telling JS to always return a Promise.
Because it will always return a Promise, it can also await on promises inside of its own block. Imagine it like one giant Promise chain - what happens internally to the function gets effectively gets bolted on to its internal .then() block, and what's returned is the final .then() in the chain.
For example, this function...
async function test() {
return 'hello world';
}
... returns a Promise. So you can execute it like one, .then() and all.
test().then(message => {
// message = 'hello world'
});
So...
async function test() {
const user = await getUser();
const report = await user.getReport();
report.read = true
return report;
}
Is roughly analogous to...
function test() {
return getUser().then(function (user) {
return user.getReport().then(function (report) {
report.read = true;
return report;
});
});
}
In both cases, the callback passed to test().then() will receive report as its first parameter.
Generators (i.e. marking a function * and using the yield keyword) are a different concept altogether. They don't use Promises. They effectively allow you to 'jump' between different portions of your code, yielding a result from inside of a function and then jumping back to that point and resuming for the next yield block.
Although they feel somewhat similar (i.e. 'halting' execution until something happens somewhere else), async/await only gives you that illusion because it messes with the internal ordering of Promise execution. It's not actually waiting - it's just shuffling when the callbacks happen.
Generators, by contrast, are implemented differently so that the generator can maintain state and be iterated over. Again, nothing to do with Promises.
The line is further blurred because at the current time of writing, support for async/await is scare; Chakracore supports it natively, and V8 has it coming soon. In the meantime, transpilers like Babel allow you to write async/await and convert the code to generators. It's a mistake to conclude that generators and async/await are therefore the same; they're not... it just so happens that you can bastardize how yield works alongside Promises to get a similar result.
Update: November 2017
Node LTS now has native async/await support, so you ought never to need to use generators to simulate Promises.

These answers all give valid arguments for why the async keyword is a good thing, but none of them actually mentions the real reason why it had to be added to the spec.
The reason is that this was a valid JS pre-ES7
function await(x) {
return 'awaiting ' + x
}
function foo() {
return(await(42))
}
According to your logic, would foo() return Promise{42} or "awaiting 42"? (returning a Promise would break backward compatibility)
So the answer is: await is a regular identifier and it's only treated as a keyword inside async functions, so they have to be marked in some way.

The reason for the async keyword in front is simple so you know that return value will be transformed into a promise. If no keyword how would the Interpreter know to do this.
I think this was first introduce in C# and EcmaScript is taking a loot of stuff from TypeScript. TypeScript and C# are conceived by Anders Hejlsberg and are similar.
lets say you have a function (this one is just to have some asynchronous work)
function timeoutPromise() {
return (new Promise(function(resolve, reject) {
var random = Math.random()*1000;
setTimeout(
function() {
resolve(random);
}, random);
}));
}
this function will make us wait for a random time and return a Promise (if you use jQuery Promise is similar to Deferred) object. To use this function today you would write something like this
function test(){
timeoutPromise().then(function(waited){
console.log('I waited' + waited);
});
}
And this is fine. Now lets try to return the log message
function test(){
return timeoutPromise().then(function(waited){
var message = 'I waited' + waited;
console.log(message);
return message; //this is where jQuery Deferred is different then a Promise and better in my opinion
});
}
Ok this is not bad but there are two return statements and a function in the code.
Now with async this will look like this
async function test(){
var message = 'I waited' + (await timeoutPromise());
console.log(message);
return message;
}
The code is short and inline. If you written a lot of .then() or . done() you know how unreadable the code can get.
Now why the async keyword in front of function. Well this is to indicate that your return value is not what is returned. In theory you could write this(This can be done in c# I don't know if js will allow since it's not done).
async function test(wait){
if(wait == true){
return await timeoutPromise();
}
return 5;
}
you see, you return a number but the actual return will be a Promise you don't have to use
return new Promise(function(resolve, reject) { resolve(5);};
Since you can't await a number only a Promise await test(false) will throw an exception and await test(true) will not if you don't indicate async in front.