Develop Reference

Unawaited promises still executed

While debugging a set-up to control promise concurrency for some CPU intensive asynchronous tasks, I came across the following behavior that I can not understand.
My route to control the flow was to first create an array of Promises, and after that control the execution of these Promises by explicitly awaiting them. I produced the following snippet that shows the unexpected behavior:
import fs from 'node:fs';
import { setTimeout } from 'node:timers/promises';
async function innerAsync(n: number) {
return fs.promises.readdir('.').then(value => {
console.log(`Executed ${n}:`, value);
return value;
});
}
async function controllerAsync() {
const deferredPromises = new Array<Promise<string[]>>();
for (const n of [1, 2, 3]) {
const defer = innerAsync(n);
deferredPromises.push(defer);
}
const result = deferredPromises[0];
return result;
}
const result = await controllerAsync();
console.log('Resolved:', result);
await setTimeout(1000);
This snippet produces the following result (assuming 2 files are in .):
Executed 1: [ 'test.txt', 'test2.txt' ]
Resolved: [ 'test.txt', 'test2.txt' ]
Executed 2: [ 'test.txt', 'test2.txt' ]
Executed 3: [ 'test.txt', 'test2.txt' ]
Observed
The promises defined at line 15 are all being executed, regardless if they are returned/awaited (2 and 3).
Question
What causes the other, seemingly unused, promises to be executed? How can I make sure in this snippet only the first Promise is executed, where the others stay pending?
Node v19.2
Typescript v4.9.4
StackBlitz

What causes the other, seemingly unused, promises to be executed?
A promise is a tool used to watch something and provide an API to react to the something being complete.
Your innerAsync function calls fs.promises.readdir. Calling fs.promises.readdir makes something happen (reading a directory).
The then callback is called as a reaction when reading that directory is complete.
(It isn't clear why you marked innerAsync as async and then didn't use await inside it.)
If you don't use then or await, then that doesn't change the fact you have called fs.promises.readdir!
How can I make sure in this snippet only the first Promise is executed, where the others stay pending?
Focus on the function which does the thing, not the code which handles it being complete.
Or an analogy:
If you tell Alice to go to the shops and get milk, but don't tell Bob to put the milk in the fridge when Alice gets back, then you shouldn't be surprised that Alice has gone to the shops and come back with milk.

Promises aren't "executed" at all. A promise is a way to observe an asynchronous process that is already in progress. By the time you have a promise, the process it's reporting the result of is already underway.¹ It's the call to fs.readdir that starts the process, not calling then on a promise. (And even if it were, you are calling then on the promise from fs.readdir right away, in your innerAsync function.)
If you want to wait to start the operations, wait to call the methods starting the operations and giving you the promises. I'd show an edited version of your code, but it's not clear to me what it's supposed to do, particularly since controllerAsync only looks at the first element of the array and doesn't wait for anything to complete before returning.
A couple of other notes:
It's almost never useful to combine async/await with explicit calls to .then as in innerAsync. That function would be more clearly written as:
async function innerAsync(n: number) {
const value = await fs.promises.readdir(".");
console.log(`Executed ${n}:`, value);
return value;
}
From an order-of-operations perspective, that does exactly the same thing as the version with the explicit then.
There's no purpose served by declaring controllerAsync as an async function, since it never uses await. The only reason for making a function async is so you can use await within it.
¹ This is true in the normal case (the vast majority of cases), including the case of the promises you get from Node.js' fs/promises methods. There are (or were), unfortunately, a couple of libraries that had functions that returned promises but didn't start the actual work until the promise's then method was called. But those were niche outliers and arguably violate the semantics of promises.

Benefits of using async await in simple functions

I have a simple function that aims to log a user in, and some guard clauses in it to rule out errors
async signIn(email: string, passwort: string): Promise<void> {
const user: IPerson = await this.fetchUser(email);
if (user === undefined) {
return Promise.reject('Wrong email');
}
if (user.passwort !== passwort) {
return Promise.reject('Wrong password');
}
if (!this.logInUser(user)) {
return Promise.reject('Login Failed');
}
return Promise.resolve();
}
I used async await to wait for the promise resolve of fetchUser giving me the user, but this made me think, what benefits does await have here?
As summarized in some blog this is how async works:
"Let’s emphasize: await literally suspends the function execution until the promise settles, and then resumes it with the promise result. That doesn’t cost any CPU resources, because the JavaScript engine can do other jobs in the meantime: execute other scripts, handle events, etc."
But in my case, there is no other other jobs in the meantime, the function can only resolve if the fetchUser provides something. Doesnt this mean JS Engine automatically waits and behaves just the same like using async? That would make it redundant.

what benefits does await have here?
It lets you write syntax that is easier to follow than using then with a callback.
That's all.
But in my case, there is no other other jobs in the meantime, the function can only resolve if the fetchUser provides something.
fetchUser is asynchronous. It isn't made asynchronous by the use of await. It provides a promise.
If you don't await that promise then you are just changing where the code execution is suspended.
Instead of signIn being suspended and the code that (you say) does nothing outside it continuing, you'll instead only suspend fetchUser and signIn will continue processing with the promise returned by fetchUser instead of the User object that that promise eventually resolves with.

Doesn't this mean JS Engine automatically waits and behaves just the same like using async? That would make it redundant.
No, the JS engine doesn't automatically "wait". Without await it would immediately continue executing the if statements, which obviously is not desired, as then you would not have the actual user object to work with, but "just" a promise object.
If the only thing that you wanted to do was to call fetch then it would be redundant to await it, but since you need the response from that request, you need to get informed when that response is available, and that is what await is offering you. You could alternatively chain a then call, whose callback would be called when the response becomes available.

How to blocking read a queue in JavaScript?

I'm new to js and async programming and I'm trying to pass the keyboard event to a running wasm instance.
I'm using wasm-clang to build and run simple c programs in browser, but it's in-memory file system don't support reading from stdin, as described in this issue. I'm trying to write a patch of the memfs.c, by importing a JavaScript function and call it when reading from stdin.
In short, how to make a message queue which can blocking read, and export the read function to WebAssembly?

I'm not sure about the specifics of your message queue but:
You cannot use await unless you are within an async function, and async/await are based around promises so you need to think more about how they work, than as await being a 'block here' keyword
assuming someQueue.pop() returns a promise (async) rather than a synchronous value:
promises:
someQueue.pop()
.then(val => {
doSomething(val)
})
.catch(err => handleErr(err))
await/async:
async function read() {
let val = await someQueue.pop()
doSomething(val)
}
either way, you must put the 'doSomething' logic inside the promise or async function because javascript cannot pause execution to wait for asynchronous logic to complete
I suggest learning to use promises, then how that relates to async/await. It's a little confusing at first but really easy to understand and use once it clicks

Using "await" inside non-async function

I have an async function that runs by a setInterval somewhere in my code. This function updates some cache in regular intervals.
I also have a different, synchronous function which needs to retrieve values - preferably from the cache, yet if it's a cache-miss, then from the data origins
(I realize making IO operations in a synchronous manner is ill-advised, but lets assume this is required in this case).
My problem is I'd like the synchronous function to be able to wait for a value from the async one, but it's not possible to use the await keyword inside a non-async function:
function syncFunc(key) {
if (!(key in cache)) {
await updateCacheForKey([key]);
}
}
async function updateCacheForKey(keys) {
// updates cache for given keys
...
}
Now, this can be easily circumvented by extracting the logic inside updateCacheForKey into a new synchronous function, and calling this new function from both existing functions.
My question is why absolutely prevent this use case in the first place? My only guess is that it has to do with "idiot-proofing", since in most cases, waiting on an async function from a synchronous one is wrong. But am I wrong to think it has its valid use cases at times?
(I think this is possible in C# as well by using Task.Wait, though I might be confusing things here).

My problem is I'd like the synchronous function to be able to wait for a value from the async one...
They can't, because:
JavaScript works on the basis of a "job queue" processed by a thread, where jobs have run-to-completion semantics, and
JavaScript doesn't really have asynchronous functions — even async functions are, under the covers, synchronous functions that return promises (details below)
The job queue (event loop) is conceptually quite simple: When something needs to be done (the initial execution of a script, an event handler callback, etc.), that work is put in the job queue. The thread servicing that job queue picks up the next pending job, runs it to completion, and then goes back for the next one. (It's more complicated than that, of course, but that's sufficient for our purposes.) So when a function gets called, it's called as part of the processing of a job, and jobs are always processed to completion before the next job can run.
Running to completion means that if the job called a function, that function has to return before the job is done. Jobs don't get suspended in the middle while the thread runs off to do something else. This makes code dramatically simpler to write correctly and reason about than if jobs could get suspended in the middle while something else happens. (Again it's more complicated than that, but again that's sufficient for our purposes here.)
So far so good. What's this about not really having asynchronous functions?!
Although we talk about "synchronous" vs. "asynchronous" functions, and even have an async keyword we can apply to functions, a function call is always synchronous in JavaScript. An async function is a function that synchronously returns a promise that the function's logic fulfills or rejects later, queuing callbacks the environment will call later.
Let's assume updateCacheForKey looks something like this:
async function updateCacheForKey(key) {
const value = await fetch(/*...*/);
cache[key] = value;
return value;
}
What that's really doing, under the covers, is (very roughly, not literally) this:
function updateCacheForKey(key) {
return fetch(/*...*/).then(result => {
const value = result;
cache[key] = value;
return value;
});
}
(I go into more detail on this in Chapter 9 of my recent book, JavaScript: The New Toys.)
It asks the browser to start the process of fetching the data, and registers a callback with it (via then) for the browser to call when the data comes back, and then it exits, returning the promise from then. The data isn't fetched yet, but updateCacheForKey is done. It has returned. It did its work synchronously.
Later, when the fetch completes, the browser queues a job to call that promise callback; when that job is picked up from the queue, the callback gets called, and its return value is used to resolve the promise then returned.
My question is why absolutely prevent this use case in the first place?
Let's see what that would look like:
The thread picks up a job and that job involves calling syncFunc, which calls updateCacheForKey. updateCacheForKey asks the browser to fetch the resource and returns its promise. Through the magic of this non-async await, we synchronously wait for that promise to be resolved, holding up the job.
At some point, the browser's network code finishes retrieving the resource and queues a job to call the promise callback we registered in updateCacheForKey.
Nothing happens, ever again. :-)
...because jobs have run-to-completion semantics, and the thread isn't allowed to pick up the next job until it completes the previous one. The thread isn't allowed to suspend the job that called syncFunc in the middle so it can go process the job that would resolve the promise.
That seems arbitrary, but again, the reason for it is that it makes it dramatically easier to write correct code and reason about what the code is doing.
But it does mean that a "synchronous" function can't wait for an "asynchronous" function to complete.
There's a lot of hand-waving of details and such above. If you want to get into the nitty-gritty of it, you can delve into the spec. Pack lots of provisions and warm clothes, you'll be some time. :-)
Jobs and Job Queues
Execution Contexts
Realms and Agents

You can call an async function from within a non-async function via an Immediately Invoked Function Expression (IIFE):
(async () => await updateCacheForKey([key]))();
And as applied to your example:
function syncFunc(key) {
if (!(key in cache)) {
(async () => await updateCacheForKey([key]))();
}
}
async function updateCacheForKey(keys) {
// updates cache for given keys
...
}

This shows how a function can be both sync and async, and how the Immediately Invoked Function Expression idiom is only immediate if the path through the function being called does synchronous things.
function test() {
console.log('Test before');
(async () => await print(0.3))();
console.log('Test between');
(async () => await print(0.7))();
console.log('Test after');
}
async function print(v) {
if(v<0.5)await sleep(5000);
else console.log('No sleep')
console.log(`Printing ${v}`);
}
function sleep(ms : number) {
return new Promise(resolve => setTimeout(resolve, ms));
}
test();
(Based off of Ayyappa's code in a comment to another answer.)
The console.log looks like this:
16:53:00.804 Test before
16:53:00.804 Test between
16:53:00.804 No sleep
16:53:00.805 Printing 0.7
16:53:00.805 Test after
16:53:05.805 Printing 0.3
If you change the 0.7 to 0.4 everything runs async:
17:05:14.185 Test before
17:05:14.186 Test between
17:05:14.186 Test after
17:05:19.186 Printing 0.3
17:05:19.187 Printing 0.4
And if you change both numbers to be over 0.5, everything runs sync, and no promises get created at all:
17:06:56.504 Test before
17:06:56.504 No sleep
17:06:56.505 Printing 0.6
17:06:56.505 Test between
17:06:56.505 No sleep
17:06:56.505 Printing 0.7
17:06:56.505 Test after
This does suggest an answer to the original question, though. You could have a function like this (disclaimer: untested nodeJS code):
const cache = {}
async getData(key, forceSync){
if(cache.hasOwnProperty(key))return cache[key] //Runs sync
if(forceSync){ //Runs sync
const value = fs.readFileSync(`${key}.txt`)
cache[key] = value
return value
}
//If we reach here, the code will run async
const value = await fsPromises.readFile(`${key}.txt`)
cache[key] = value
return value
}

Now, this can be easily circumvented by extracting the logic inside updateCacheForKey into a new synchronous function, and calling this new function from both existing functions.
T.J. Crowder explains the semantics of async functions in JavaScript perfectly. But in my opinion the paragraph above deserves more discussion. Depending on what updateCacheForKey does, it may not be possible to extract its logic into a synchronous function because, in JavaScript, some things can only be done asynchronously. For example there is no way to perform a network request and wait for its response synchronously. If updateCacheForKey relies on a server response, it can't be turned into a synchronous function.
It was true even before the advent of asynchronous functions and promises: XMLHttpRequest, for instance, gets a callback and calls it when the response is ready. There's no way of obtaining a response synchronously. Promises are just an abstraction layer on callbacks and asynchronous functions are just an abstraction layer on promises.
Now this could have been done differently. And it is in some environments:
In PHP, pretty much everything is synchronous. You send a request with curl and your script blocks until it gets a response.
Node.js has synchronous versions of its file system calls (readFileSync, writeFileSync etc.) which block until the operation completes.
Even plain old browser JavaScript has alert and friends (confirm, prompt) which block until the user dismisses the modal dialog.
This demonstrates that the designers of the JavaScript language could have opted for synchronous versions of XMLHttpRequest, fetch etc. Why didn't they?
[W]hy absolutely prevent this use case in the first place?
This is a design decision.
alert, for instance, prevents the user from interacting with the rest of the page because JavaScript is single threaded and the one and only thread of execution is blocked until the alert call completes. Therefore there's no way to execute event handlers, which means no way to become interactive. If there was a syncFetch function, it would block the user from doing anything until the network request completes, which can potentially take minutes, even hours or days.
This is clearly against the nature of the interactive environment we call the "web". alert was a mistake in retrospect and it should not be used except under very few circumstances.
The only alternative would be to allow multithreading in JavaScript which is notoriously difficult to write correct programs with. Are you having trouble wrapping your head around asynchronous functions? Try semaphores!

It is possible to add a good old .then() to the async function and it will work.
Should consider though instead of doing that, changing your current regular function to async one, and all the way up the call stack until returned promise is not needed, i.e. there's no work to be done with the value returned from async function. In which case it actually CAN be called from a synchronous one.

Async functions vs await

I'm coming from a PHP background and I'm trying to learn NodeJS.
I know that everything in Node is async but I've found that i've been using the async / await combo quite a lot in my code and I wanted to make sure I wasn't doing something stupid.
The reason for it is that I have a lot of situations where I need the result of something before continuing (for example for a small ajax request). This small ajax request has no other purpose other than to do the set of things that I want it to and in the order that I specify so even if I do things the "async way" and write it using a callback i'm still having to wait for things to finish in the right order.
Right now whenever I find myself in this situation I just use await to wait for the result:
ie:
var result = await this.doSomething(data);
Opposed to using a callback
this.doSomething(data, function(callback) {
// code
callback();
});
To me the first example looks cleaner than the second one which is why I've been opting for that. But I'm worried that I might be missing something fundamental here. But in a situation where there is nothing else to process below the async call and the only way for things to progress is for it to follow a syncronous style, is there anything wrong with using the first style over the second?

But I'm worried that I might be missing something fundamental here.
Nope, you're not, that's exactly what you want to do, assuming this.doSomething(data) is asynchronous (and if it's an ajax call, one hopes it is async) and that it returns a promise (which all functions defined with the async keyword do implicitly). (If it's not asynchronous, you don't need the await, although it's allowed.)
You're probably being a bit confused (understandably) by the fact that things are going through a transition. Until recently, the overwhelming convention in Node APIs (the built-in ones and ones provided by third-party modules) was to use the "Node callback" pattern, which is that a function that will do asynchronous work expects its last argument to be a callback, which it will call with a first argument indicating success/failure (null = success, anything else is an error object) with subsequent arguments providing the result. (Your second example assumes doSomething is one of these, instead of being a function that returns a promise.)
Example: fs.readFile, which you use like this:
fs.readFile("/some/file", "utf-8", function(err, data) {
if (err) {
// ...handle the fact an error occurred..
return;
}
// ...use the data...
});
This style quickly leads to callback hell, though, which is one of the reasons promises (aka "futures") were invented.
But a lot of Node APIs still use the old pattern.
If you need to use "Node callback"-pattern functions, you can use util.promisify to create promise-enabled versions of them. For instance, say you need to use fs.readFile, which uses the Node callback pattern. You can get a promise version like this:
const readFilePromise = util.promisify(fs.readFile);
...and then use it with async/await syntax (or use the promise directly via then):
const data = await readFilePromise("/some/file", "utf-8");
There's also an npm module called promisify that can provide a promise-ified version of an entire API. (There's probably more than one.)
Just because promises and async/await replace the old Node callback style in most cases doesn't mean callbacks don't still have a place: A Promise can only be settled once. They're for one-off things. So callbacks still have a place, such as with the on method of EventEmitters, like a readable stream:
fs.createReadStream("/some/file", "utf-8")
.on("data", chunk => {
// ...do something with the chunk of data...
})
.on("end", () => {
// ...do something with the fact the end of the stream was reached...
});
Since data will fire multiple times, it makes sense to use a callback for it; a promise wouldn't apply.
Also note that you can only use await in an async function. Consequently, you may find yourself getting into the habit of a "main" module structure that looks something like this:
// ...Setup (`require` calls, `import` once it's supported, etc.)...
(async () => {
// Code that can use `await `here...
})().catch(err => {
// Handle the fact an error/promise rejection occurred in the top level of your code
});
You can leave the catch off if you want your script to terminate on an unhandled error/rejection. (Node doesn't do that yet, but it will once unhandled rejection detection matures.)
Alternately, if you want to use a promise-enabled function in a non-async function, just use then and catch:
this.doSomething()
.then(result => {
// Use result
})
.catch(err => {
// Handle error
});
Note: async/await is directly supported in Node 7.x and above. Be sure your target production environment supports Node 7.x or above if your'e going to use async/await. If not, you could transpile your code using something like Babel and then use the transpiled result on the older version of Node.