Develop Reference

Unawaited promises still executed

While debugging a set-up to control promise concurrency for some CPU intensive asynchronous tasks, I came across the following behavior that I can not understand.
My route to control the flow was to first create an array of Promises, and after that control the execution of these Promises by explicitly awaiting them. I produced the following snippet that shows the unexpected behavior:
import fs from 'node:fs';
import { setTimeout } from 'node:timers/promises';
async function innerAsync(n: number) {
return fs.promises.readdir('.').then(value => {
console.log(`Executed ${n}:`, value);
return value;
});
}
async function controllerAsync() {
const deferredPromises = new Array<Promise<string[]>>();
for (const n of [1, 2, 3]) {
const defer = innerAsync(n);
deferredPromises.push(defer);
}
const result = deferredPromises[0];
return result;
}
const result = await controllerAsync();
console.log('Resolved:', result);
await setTimeout(1000);
This snippet produces the following result (assuming 2 files are in .):
Executed 1: [ 'test.txt', 'test2.txt' ]
Resolved: [ 'test.txt', 'test2.txt' ]
Executed 2: [ 'test.txt', 'test2.txt' ]
Executed 3: [ 'test.txt', 'test2.txt' ]
Observed
The promises defined at line 15 are all being executed, regardless if they are returned/awaited (2 and 3).
Question
What causes the other, seemingly unused, promises to be executed? How can I make sure in this snippet only the first Promise is executed, where the others stay pending?
Node v19.2
Typescript v4.9.4
StackBlitz

What causes the other, seemingly unused, promises to be executed?
A promise is a tool used to watch something and provide an API to react to the something being complete.
Your innerAsync function calls fs.promises.readdir. Calling fs.promises.readdir makes something happen (reading a directory).
The then callback is called as a reaction when reading that directory is complete.
(It isn't clear why you marked innerAsync as async and then didn't use await inside it.)
If you don't use then or await, then that doesn't change the fact you have called fs.promises.readdir!
How can I make sure in this snippet only the first Promise is executed, where the others stay pending?
Focus on the function which does the thing, not the code which handles it being complete.
Or an analogy:
If you tell Alice to go to the shops and get milk, but don't tell Bob to put the milk in the fridge when Alice gets back, then you shouldn't be surprised that Alice has gone to the shops and come back with milk.

Promises aren't "executed" at all. A promise is a way to observe an asynchronous process that is already in progress. By the time you have a promise, the process it's reporting the result of is already underway.¹ It's the call to fs.readdir that starts the process, not calling then on a promise. (And even if it were, you are calling then on the promise from fs.readdir right away, in your innerAsync function.)
If you want to wait to start the operations, wait to call the methods starting the operations and giving you the promises. I'd show an edited version of your code, but it's not clear to me what it's supposed to do, particularly since controllerAsync only looks at the first element of the array and doesn't wait for anything to complete before returning.
A couple of other notes:
It's almost never useful to combine async/await with explicit calls to .then as in innerAsync. That function would be more clearly written as:
async function innerAsync(n: number) {
const value = await fs.promises.readdir(".");
console.log(`Executed ${n}:`, value);
return value;
}
From an order-of-operations perspective, that does exactly the same thing as the version with the explicit then.
There's no purpose served by declaring controllerAsync as an async function, since it never uses await. The only reason for making a function async is so you can use await within it.
¹ This is true in the normal case (the vast majority of cases), including the case of the promises you get from Node.js' fs/promises methods. There are (or were), unfortunately, a couple of libraries that had functions that returned promises but didn't start the actual work until the promise's then method was called. But those were niche outliers and arguably violate the semantics of promises.

trying to understand async / await / sync in node

i know this probably has been asked before, but coming from single-threaded language for the past 20 years, i am really struggling to grasp the true nature of node. believe me, i have read a bunch of SO posts, github discussions, and articles about this.
i think i understand that each function has it's own thread type of deal. however, there are some cases where i want my code to be fully synchronous (fire one function after the other).
for example, i made 3 functions which seem to show me how node's async i/o works:
function sleepA() {
setTimeout(() => {
console.log('sleep A')
}, 3000)
}
function sleepB() {
setTimeout(() => {
console.log('sleep B')
}, 2000)
}
function sleepC() {
setTimeout(() => {
console.log('sleep C')
}, 1000)
}
sleepA()
sleepB()
sleepC()
this outputs the following:
sleep C
sleep B
sleep A
ok so that makes sense. node is firing all of the functions at the same time, and whichever ones complete first get logged to the console. kind of like watching horses race. node fires a gun and the functions take off, and the fastest one wins.
so now how would i go about making these synchronous so that the order is always A, B, C, regardless of the setTimeout number?
i've dabbled with things like bluebird and the util node library and am still kind of confused. i think this async logic is insanely powerful despite my inability to truly grasp it. php has destroyed my mind.

how would i go about making these synchronous so that the order is always A, B, C
You've confirmed that what you mean by that is that you don't want to start the timeout for B until A has finished (etc.).
Here in 2021, the easiest way to do that is to use a promise-enabled wrapper for setTimeout, like this one:
const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms));
Then, make sleepA and such async functions:
async function sleepA() {
await delay(3000);
console.log("sleep A");
}
async function sleepB() {
await delay(2000);
console.log("sleep B");
}
async function sleepC() {
await delay(1000);
console.log("sleep C");
}
Then your code can await the promise each of those functions returns, which prevents the code from continuing until the promise is settled:
await sleepA();
await sleepB();
await sleepC();
You can do that inside an async function or (in modern versions of Node.js) at the top level of a module. You can't use await in a synchronous function, since await is asynchronous by nature.
i think i understand that each function has it's own thread type of deal
No, JavaScript functions are just like those of other synchronous languages (despite its reputation, JavaScript itself is overwhelmingly synchronous; it's just used in highly-asynchronous environments and recently got some features to make that easier). More on this below.
ok so that makes sense. node is firing all of the functions at the same time, and whichever ones complete first get logged to the console. kind of like watching horses race. node fires a gun and the functions take off, and the fastest one wins.
Close. The functions run in order, but all that each of them does is set up a timer and return. They don't wait for the timer to fire. Later, when it does fire, it calls the callback you passed into setTimeout.
JavaScript has a main thread and everything runs on that main thread unless you explicitly make it run on a worker thread or in a child process. Node.js is highly asynchronous (as are web browsers, the other place JavaScript is mostly used), but your JavaScript code all runs on one thread by default. There's a loop that processes "jobs." Jobs get queued by events (such as a timer firing), and then processed in order by the event loop. I go into a bit more on it here, and the Node.js documentation covers their loop here (I think it's slightly out of date, but the principal hasn't changed).

Can JavaScript execute two asynchronous functions or callbacks at the same time? [duplicate]

I am new to Javascript and Ascynchronous programming, and a few things confuse me, please point out the misfacts in my learning.
Callbacks of an Asynchronous function are put in a message queue and
executed via event loop.
Asynchronous execution is non-blocking, done
via event loop.
Functions like setTimeout are asynchronous.
Asynchronous functions are blocking, only their callbacks are
non-blocking.
If any of the above are wrong, please elaborate it.

Callbacks of an Asynchronous function are put in a message queue and executed via event loop.
No.
There are generally two kinds of asynchronous functions - ones that take some sort of callback to run when they are done and ones that produce a Promise as a result.
Callback-based functions
This is an example of a callback-based function:
setTimeout(() => console.log("foo"), 0);
console.log("bar");
/* output:
bar
foo
*/
setTimeout will delay a callback to be executed later. Even with a timeout of zero milliseconds, it still schedules the callback until after the current execution is complete, so the order is:
setTimeout schedules the callback.
console.log("bar") runs.
The callback with console.log("foo") runs.
There isn't a message queue, there is a queue of tasks to execute. As a brief overview, the event loop takes one task from the queueand executes it to completion, then takes the next one and executes it to completion, etc. A simple pseudo-code representation would be:
while(true) {
if (eventQueue.hasNext()) {
task = eventQueue.takeNext();
task();
}
}
See here for more information on the event loop.
With all that said, the event loop has more bearing on setTimeout than many other callback-based functions. Other examples of callback based functions are jQuery's $.ajax() or the Node.js fs module for file system access (the non-promise API for it). They, and others, also take a callback that will be executed later but the bounding factor is not the event loop but whatever underlying operation the function makes. In the case of $.ajax() when the callback would be called depends on network speed, latency, and the server which responds to the request. So, while the callback will be executed by the event loop, so is everything else. So, it's hardly special. Only setTimeout has a more special interaction here, since the timings might be imprecise based on what tasks are available - if you schedule something to run in 10ms and some task takes 12ms to finish, the callback scheduled by setTimeout will not run on time.
Promise-based functions
Here is an example:
async function fn(print) {
await "magic for now";
console.log(print);
}
fn("foo")
.then(() => console.log("bar"));
/* output:
foo
bar
*/
(I'm omitting a lot of details for now for illustration.)
Promises are technically an abstraction over callbacks tailored towards handling asynchronous operations. The .then() method of a promise takes a callback and will execute it after the promise gets resolved, which also happens after the asynchronous operation finishes. Thus, we can sequence together execution in the right order:
async function fn(print) {
await "magic for now";
console.log(print);
}
fn("foo");
console.log("bar");
/* output:
bar
foo
*/
In a way, promises are sort of callbacks because they are here to replace them and do it in broadly the same fashion. You still pass callbacks to be executed when something succeeds or fails. But they aren't just callbacks.
At any rate, a callback given to a promise is still delayed:
Promise.resolve()
.then(() => console.log("foo"));
console.log("bar");
/* output:
bar
foo
*/
But not via the same event queue as the one setTimeout uses. There are two queues:
macrotasks queue - setTimeout places things in it, and all UI interactions are also added here.
microtasks queue - promises schedule their things here.
When the event loop runs, the microtasks queue (so, promises) has the highest priority, then comes the macrotask queue. This leads to:
setTimeout(() => console.log("foo")); //3 - macrotask queue
Promise.resolve()
.then(() => console.log("bar")); //2 - microtask queue
console.log("baz"); //1 - current execution
/* output:
baz
bar
foo
*/
At any rate, I don't think I'm comfortable saying that promise-based functions work via callbacks. Sort of yes but in a reductionist sense.
Asynchronous execution is non-blocking, done via event loop.
No.
What is "blocking"?
First of all, let's make this clear - blocking behaviour is when the environment is busy executing something. Usually no other code can run during that execution. Hence, further code is blocked from running.
Let's take this code as example:
setTimeout(taskForLater, 5000);
while (somethingIsNotFinished()) {
tryToFinish();
}
Here taskForLater will be scheduled to run in 5 seconds. However, the while loop will block the execution. Since no other code will run, taskForLater might run in 10 seconds time, if that's how lot it takes for the loop to finish.
Running an asynchronous function doesn't always mean it runs in parallel with the current code. The environment executes one thing at time in most cases. There is no multi-threading by default in JavaScript, parallel execution is an opt-in behaviour, for example by using workers.
What is "asynchronous execution"?
This can mean a couple of things and it's not clear which one you reference:
Running through the body of an asynchronous function
Waiting until the underlying asynchronous operation is finished
In both cases the quoted statement is wrong but for different reasons.
The body of an async function
Async functions are syntactic sugar over promises. They use the same mechanism but just present the code differently. However, an async function is blocking. As a side note, so are promise executors (the callback given to a promise constructor). In both cases, the function will run and block until something causes it to pause. The easiest way to demonstrate it is with an async function - using await will pause the execution of the function and schedule the rest of it to be finished later.
Whole body is blocking:
async function fn() {
console.log("foo");
console.log("bar");
}
fn();
console.log("baz");
/* output:
foo
bar
baz
*/
Pausing in the middle:
async function fn() {
console.log("foo");
await "you can await any value";
console.log("bar");
}
fn();
console.log("baz");
/* output:
foo
baz
bar
*/
As a point of clarification, any value can be awaited, not just promises. Awaiting a non-promise will still pause and resume the execution but since there is nothing to wait for, this will cause it to be among the first things on the microtask queue.
At any rate, executing the body of an async function might block. Depends on what the operation is.
The underlying asynchronous operation
When talking about "underlying operation", most of the times we mean that we hand off the control to something else, for example the browser or a library. In this case, the operation is not fulfilled by the current JavaScript environment we call something which will perform an operation and only notify the current JavaScript environment when it finishes. For example, in a browser calling fetch will fire off a network request but it's the browser handling it, not our code. So it's non-blocking but not by outside the execution environment.
fetch("https://official-joke-api.appspot.com/random_joke")
.then(res => res.json())
.then(joke => console.log(joke.setup + "\n" + joke.punchline));
console.log("foo");
With that said, we cannot even generalise what a given asynchronous operation will do. It might actually block the execution, at least for a while, or it might be entirely carried out in a separate process to the current JavaScript environment.
Functions like setTimeout are asynchronous.
Yes.
Although, it's probably considering what is like setTimeout. Is it setInterval? Is it any callback-based asynchronous function? The problem is that the definition starts to become circular "asynchronous callback-based functions like setTimeout are asynchronous".
Not every function that takes a callback is asynchronous. Those that are might be considered similar to setTimeout.
Asynchronous functions are blocking, only their callbacks are non-blocking.
No.
As discussed above, asynchronous functions might block. Depends on what exactly they do. For example $.ajax will initiate a network request using the browser, similar to fetch. In the case of $.ajax the function blocks while preparing the request but not after it's sent.
The second problem with the statement is that it's incorrect to call the callbacks non-blocking - executing them will certainly block the execution again. The callback is a normal JavaScript code that will be executed via the event loop when its time comes. While the task is running to completion, execution is still blocked.
If you mean that they are non-blocking in the sense that the callback will be put on the event queue, that's still not guaranteed. Consider the following illustrative code:
function myAsyncFunction(callback) {
const asyncResult = someAsynchronousNonBlockingOperation();
doStuff(result);
callback(result);
doMoreStuff(result);
}
Once someAsynchronousNonBlockingOperation() produces a value executing callback will not be scheduled for later but will be part of the synchronous sequence of code that processes that result. So, callback will be executed later but it will not be a task by itself but part of an overall task that encompasses doStuff and doMoreStuff.

Callbacks of an Asynchronous function are put in a message queue and executed via event loop.
Just in a queue (no messages). By asynchronous function I suppose you mean Promises. Remember async and await are syntactic sugar for Promises. They are execute by the event loop.
Asynchronous execution is non-blocking, done via event loop.
Not really. Javascript is single threaded: whenever it starts executing your async code, it stays there as long as needed, unless you use Workers.
Functions like setTimeout are asynchronous.
Like Promise, setTimeout and similars put the callback in a queue, which will be executed somewhere in the event loop (which may behave different for each kind of async functions).
Asynchronous functions are blocking, only their callbacks are non-blocking.
Right!

Javascript - wait for async call to finish before returning from function, without the use of callbacks

I want to preface by saying I've viewed a lot of stackoverflow questions regarding this topic, but I haven't found any 'duplicates' per se since none of them contain solutions that would solve this specific case.
I've mainly looked at How do I return the response from an asynchronous call?, and the section on 'Promises with async/await' would work inside an asynchronous function, but the function I'm working on is not async and is part of an existing codebase that I can't change easily. I wouldn't be able to change the function to async.
The section on callbacks wouldn't work either, which is explained further below. Anyway, I'll jump into my question:
I'm editing a function (standard function, not async) in JavaScript by adding an asynchronous function call. I want to wait until the async call finishes before I return from the function (since the result of the async call needs to be included in the return value). How would I do so?
I looked into using callbacks, which would allow me to write code which is guaranteed to run only after the async call completes. However, this wouldn't interrupt the flow of the program in the original function, and the original function could still return before the callback is run. A callback would allow me to execute something sequentially after the async function, but it wouldn't allow me to wait for asynchronous call to complete at the highest level.
Example code, which wouldn't return the desired result:
function getPlayers() {
... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall()
... other code ...
allPlayers.add(outfieldPlayers)
return allPlayers // the returned value may or may not include outfield players
}
The actual problem I'm facing is a little more complicated - I'm calling the async function in each iteration of a for loop, and need to wait until all calls have completed before returning. But, I think if I can solve this simpler problem, I can solve the problem with a for loop.

Sadly, it is pretty much impossible to wait for async code in a synchronous way. This is because there is no threading in JS (most JS runtimes, but some are). So code is either synchronous or asynchronous.
Asynchronous code is possible because of the event loop. The event loop is part of the javascript runtime. It works by keeping a stack of callback functions that run when events trigger them - usually either timeout events (which you can set with setTimeout()) or IO events (which happen when you make disk or HTTP requests, or on user interaction). However, these callbacks only run when no other code is running, so only when the program is idle and all functions have returned.
This means that techniques like "spin loops" (where you just run a loop until a condition is changed by another thread) that work in threaded environments don't work because the async code won't run until the spin loop finishes.
More Info: https://medium.com/front-end-weekly/javascript-event-loop-explained-4cd26af121d4

If you are using NodeJS, this is possible through execSync.
This requires you to place your asynchronous code in a separate file, spawn a separate process using execSync, which will wait until it exits.
For example, consider the following async function which prints an array.
// task.js
(async function() {
await new Promise((resolve) => setTimeout(() => {
console.log(JSON.stringify([3,4,5]));
resolve();
}, 1000));
})();
Now, you can invoke this from your main process:
function asyncGetOutfieldPlayersCall() {
const execSync = require('child_process').execSync;
return JSON.parse(execSync("node task.js"));
}
function getPlayers() {
let allPlayers = [1,2];
// ... other code ...
let outfieldPlayers = asyncGetOutfieldPlayersCall();
// ... other code ...
allPlayers = allPlayers.concat(outfieldPlayers)
return allPlayers;
}

Isn't it better to be async with Node

Here is my code
var x = 0
data.results[0].taxonomies.some(function(e){
if(taxo.indexOf(e.code)!=-1){ //if at least one code from the data base is in my list of codes
callback(validLic(data.results[0].taxonomies)) //return true after some other validations
return true
}else{
x++
return false
}
})
if(x==data.results[0].taxonomies.length){callback(false)}//if the entire array was processed, and I didn't find anything I was looking for, return false
I'd like someone to confirm that due the async nature of node, the last if statement is at some point, bound to fire off before I'm done processing the array.
How can I better manage this situation without the help of some sync or parallel library?
The reason I ask it that way is because I'm under the impression that if I can't write something to be completely async, then I'm not writing it efficiently, right?
EDIT:
Based on Luc Hendirks' logic, I have changed my code to this:
var flag = data.results[0].taxonomies.some(function(e){
if(taxo.indexOf(e.code)!=-1){ //if at least one code from the data base is in my list of codes
return true
}else{
return false
}
})
if(flag==true){
callback(validLic(data.results[0].taxonomies))
}else{
callback(false)
}
Because this follows the sync traits outlined below, I shouldn't have an issue with flag being undefined before the callback is called now right?

Javascript (and Node) are single threaded, meaning it has only 1 CPU available. If the functions you call only require CPU time, making it async is useless. If you need to call a function and the CPU has to wait (do nothing), then making it async is very useful. Because while it is waiting until the function is finished it can do something else.
A function that checks if a url is valid with a regular expression can be synchronous, as the CPU needs to do some calculations and you get the result. If the function actually does a GET request and checks the response code, the CPU has to wait until the response is received. In the meantime it could do something else, so this function should be made asynchronous.
The difference of a synchronous and asynchronous function is that a synchronous function returns a value:
function(a) { return a; }
and an asynchronous function returns the result using a callback function (this is an actual function that you put in as a function argument):
function(callback){
// Do something that takes time but not CPU, like an API call...
callback('Some result');
}
A synchronous function is called like this:
var a = something();
Asynchronous like this:
something(function(result){
console.log(result);
});
So to answer your question, if some() is an asynchronous function, then the last if statement can be executed before the some function is finished. Because: the CPU does not want to wait. It can do other stuff while waiting. That's the power of asynchronous programming. t
It is also important to know that "parallel" does not exist in Javascript/Node. There is only 'doing stuff instead of waiting', like executing multiple API calls at the same time. That is not parallel computing as in using multiple threads.
Here is some more info: What is the difference between synchronous and asynchronous programming (in node.js)