Develop Reference

trying to understand async / await / sync in node

i know this probably has been asked before, but coming from single-threaded language for the past 20 years, i am really struggling to grasp the true nature of node. believe me, i have read a bunch of SO posts, github discussions, and articles about this.
i think i understand that each function has it's own thread type of deal. however, there are some cases where i want my code to be fully synchronous (fire one function after the other).
for example, i made 3 functions which seem to show me how node's async i/o works:
function sleepA() {
setTimeout(() => {
console.log('sleep A')
}, 3000)
}
function sleepB() {
setTimeout(() => {
console.log('sleep B')
}, 2000)
}
function sleepC() {
setTimeout(() => {
console.log('sleep C')
}, 1000)
}
sleepA()
sleepB()
sleepC()
this outputs the following:
sleep C
sleep B
sleep A
ok so that makes sense. node is firing all of the functions at the same time, and whichever ones complete first get logged to the console. kind of like watching horses race. node fires a gun and the functions take off, and the fastest one wins.
so now how would i go about making these synchronous so that the order is always A, B, C, regardless of the setTimeout number?
i've dabbled with things like bluebird and the util node library and am still kind of confused. i think this async logic is insanely powerful despite my inability to truly grasp it. php has destroyed my mind.

how would i go about making these synchronous so that the order is always A, B, C
You've confirmed that what you mean by that is that you don't want to start the timeout for B until A has finished (etc.).
Here in 2021, the easiest way to do that is to use a promise-enabled wrapper for setTimeout, like this one:
const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms));
Then, make sleepA and such async functions:
async function sleepA() {
await delay(3000);
console.log("sleep A");
}
async function sleepB() {
await delay(2000);
console.log("sleep B");
}
async function sleepC() {
await delay(1000);
console.log("sleep C");
}
Then your code can await the promise each of those functions returns, which prevents the code from continuing until the promise is settled:
await sleepA();
await sleepB();
await sleepC();
You can do that inside an async function or (in modern versions of Node.js) at the top level of a module. You can't use await in a synchronous function, since await is asynchronous by nature.
i think i understand that each function has it's own thread type of deal
No, JavaScript functions are just like those of other synchronous languages (despite its reputation, JavaScript itself is overwhelmingly synchronous; it's just used in highly-asynchronous environments and recently got some features to make that easier). More on this below.
ok so that makes sense. node is firing all of the functions at the same time, and whichever ones complete first get logged to the console. kind of like watching horses race. node fires a gun and the functions take off, and the fastest one wins.
Close. The functions run in order, but all that each of them does is set up a timer and return. They don't wait for the timer to fire. Later, when it does fire, it calls the callback you passed into setTimeout.
JavaScript has a main thread and everything runs on that main thread unless you explicitly make it run on a worker thread or in a child process. Node.js is highly asynchronous (as are web browsers, the other place JavaScript is mostly used), but your JavaScript code all runs on one thread by default. There's a loop that processes "jobs." Jobs get queued by events (such as a timer firing), and then processed in order by the event loop. I go into a bit more on it here, and the Node.js documentation covers their loop here (I think it's slightly out of date, but the principal hasn't changed).

Why does the browser not freeze when awaiting these promises? [duplicate]

When using Javascript promises, does the event loop get blocked?
My understanding is that using await & async, makes the stack stop until the operation has completed. Does it do this by blocking the stack or does it act similar to a callback and pass of the process to an API of sorts?

When using Javascript promises, does the event loop get blocked?
No. Promises are only an event notification system. They aren't an operation themselves. They simply respond to being resolved or rejected by calling the appropriate .then() or .catch() handlers and if chained to other promises, they can delay calling those handlers until the promises they are chained to also resolve/reject. As such a single promise doesn't block anything and certainly does not block the event loop.
My understanding is that using await & async, makes the stack stop
until the operation has completed. Does it do this by blocking the
stack or does it act similar to a callback and pass of the process to
an API of sorts?
await is simply syntactic sugar that replaces a .then() handler with a bit simpler syntax. But, under the covers the operation is the same. The code that comes after the await is basically put inside an invisible .then() handler and there is no blocking of the event loop, just like there is no blocking with a .then() handler.
Note to address one of the comments below:
Now, if you were to construct code that overwhelms the event loop with continually resolving promises (in some sort of infinite loop as proposed in some comments here), then the event loop will just over and over process those continually resolved promises from the microtask queue and will never get a chance to process macrotasks waiting in the event loop (other types of events). The event loop is still running and is still processing microtasks, but if you are stuffing new microtasks (resolved promises) into it continually, then it may never get to the macrotasks. There seems to be some debate about whether one would call this "blocking the event loop" or not. That's just a terminology question - what's more important is what is actually happening. In this example of an infinite loop continually resolving a new promise over and over, the event loop will continue processing those resolved promises and the other events in the event queue will not get processed because they never get to the front of the line to get their turn. This is more often referred to as "starvation" than it is "blocking", but the point is that macrotasks may not get serviced if you are continually and infinitely putting new microtasks in the queue.
This notion of an infinite loop continually resolving a new promise should be avoided in Javascript. It can starve other events from getting a chance to be serviced.

Do Javascript promises block the stack
No, not the stack. The current job will run until completion before the Promise's callback starts executing.
When using Javascript promises, does the event loop get blocked?
Yes it does.
Different environments have different event-loop processing models, so I'll be talking about the one in browsers, but even though nodejs's model is a bit simpler, they actually expose the same behavior.
In a browser, Promises' callbacks (PromiseReactionJob in ES terms), are actually executed in what is called a microtask.
A microtask is a special task that gets queued in the special microtask-queue.
This microtask-queue is visited various times during a single event-loop iteration in what is called a microtask-checkpoint, and every time the JS call stack is empty, for instance after the main task is done, after rendering events like resize are executed, after every animation-frame callback, etc.
These microtask-checkpoints are part of the event-loop, and will block it the time they run just like any other task.
What is more about these however is that a microtask scheduled from a microtask-checkpoint will get executed by that same microtask-checkpoint.
This means that the simple fact of using a Promise doesn't make your code let the event-loop breath, like a setTimeout() scheduled task could do, and even though the js stack has been emptied and the previous task has been executed entirely before the callback is called, you can still very well lock completely the event-loop, never allowing it to process any other task or even update the rendering:
const log = document.getElementById( "log" );
let now = performance.now();
let i = 0;
const promLoop = () => {
// only the final result will get painted
// because the event-loop can never reach the "update the rendering steps"
log.textContent = i++;
if( performance.now() - now < 5000 ) {
// this doesn't let the event-loop loop
return Promise.resolve().then( promLoop );
}
else { i = 0; }
};
const taskLoop = () => {
log.textContent = i++;
if( performance.now() - now < 5000 ) {
// this does let the event-loop loop
postTask( taskLoop );
}
else { i = 0; }
};
document.getElementById( "prom-btn" ).onclick = start( promLoop );
document.getElementById( "task-btn" ).onclick = start( taskLoop );
function start( fn ) {
return (evt) => {
i = 0;
now = performance.now();
fn();
};
}
// Posts a "macro-task".
// We could use setTimeout, but this method gets throttled
// to 4ms after 5 recursive calls.
// So instead we use either the incoming postTask API
// or the MesageChannel API which are not affected
// by this limitation
function postTask( task ) {
// Available in Chrome 86+ under the 'Experimental Web Platforms' flag
if( window.scheduler ) {
return scheduler.postTask( task, { priority: "user-blocking" } );
}
else {
const channel = postTask.channel ||= new MessageChannel();
channel.port1
.addEventListener( "message", () => task(), { once: true } );
channel.port2.postMessage( "" );
channel.port1.start();
}
}
<button id="prom-btn">use promises</button>
<button id="task-btn">use postTask</button>
<pre id="log"></pre>
So beware, using a Promise doesn't help at all with letting the event-loop actually loop.
Too often we see code using a batching pattern to not block the UI that fails completely its goal because it is assuming Promises will let the event-loop loop. For this, keep using setTimeout() as a mean to schedule a task, or use the postTask API if you are in a near future.
My understanding is that using await & async, makes the stack stop until the operation has completed.
Kind of... when awaiting a value it will add the remaining of the function execution to the callbacks attached to the awaited Promise (which can be a new Promise resolving the non-Promise value).
So the stack is indeed cleared at this time, but the event loop is not blocked at all here, on the contrary it's been freed to execute anything else until the Promise resolves.
This means that you can very well await for a never resolving promise and still let your browser live correctly.
async function fn() {
console.log( "will wait a bit" );
const prom = await new Promise( (res, rej) => {} );
console.log( "done waiting" );
}
fn();
onmousemove = () => console.log( "still alive" );
move your mouse to check if the page is locked

An await blocks only the current async function, the event loop continues to run normally. When the promise settles, the execution of the function body is resumed where it stopped.
Every async/await can be transformed in an equivalent .then(…)-callback program, and works just like that from the concurrency perspective. So while a promise is being awaited, other events may fire and arbitrary other code may run.

As other mentioned above... Promises are just like an event notification system and async/await is the same as then(). However, be very careful, You can "block" the event loop by executing a blocking operation. Take a look to the following code:
function blocking_operation_inside_promise(){
return new Promise ( (res, rej) => {
while( true ) console.log(' loop inside promise ')
res();
})
}
async function init(){
let await_forever = await blocking_operation_inside_promise()
}
init()
console.log('END')
The END log will never be printed. JS is single threaded and that thread is busy right now. You could say that whole thing is "blocked" by the blocking operation. In this particular case the event loop is not blocked per se, but it wont deliver events to your application because the main thread is busy.
JS/Node can be a very useful programming language, very efficient when using non-blocking operations (like network operations). But do not use it to execute very intense CPU algorithms. If you are at the browser consider to use Web Workers, if you are at the server side use Worker Threads, Child Processes or a Microservice Architecture.

Can I use setTimeout() on JavaScript as a parallel processed function

I know JS is single threaded. But I have a function which takes time for the calculation. I would like it to work paralleled, so this function would not freeze next statement. Calculation inside of function will take approximately 1-2 seconds.
I used to create it using promise, but it still freeze the next statement.
console.log("start");
new Promise((res, rej) => {
/* calculations */
}).then((res) => console.log(res));
console.log("end");
Then I used setTimeout function with time interval 0. LoL
console.log("start");
setTimeout(() => {
/* calculations */
console.log(res);
}, 0);
console.log("end");
Outputs:
start
end
"calculation result"
Both cases shows similar result, but using promise prevented to show console.log("end") before calculation finishes. Using setTimeout works as I wanted, and shows console.log("end") before calculation, so it was not freeze till calculation done.
I hope it was clear enough. Now for me using setTimeout is the best solution, but I would be happy to hear your ideas or any other method calculating concurrently without setTimeout.

The code you write under new Promise(() => {..code here..}) is not asynchronous. The is a very common misconception that everything under the Promise block would run asynchronously.
Instead, this JS API just let's get us a hook of some deferred task to be done once the promise is resolved. MDN
Promises are a comparatively new feature of the JavaScript language that allow you to defer further actions until after a previous action
has completed, or respond to its failure. This is useful for setting
up a sequence of async operations to work correctly.
new Promise(() => {
// whatever I write here is synchromous
// like console.log, function call, setTimeout()/fetch()/async web apis
// if there are some async tasks like fetch, setTimeout.
// they can be async by themselves but their invocation is still sync
//
})
setTimeout is not the correct option either. Code under setTimeout would run when the event stack is empty and once it enters, it would block the main thread again.
The right approach to this would be to use Web Workers.

Is Javascript async and await the equivalent to mulithreading?

Is using async and await the crude person's threads?
Many moons ago I learned how to do multithreaded Java code on Android. I recall I had to create threads, start threads, etc.
Now I'm learning Javascript, and I just learned about async and await.
For example:
async function isThisLikeTwoThreads() {
const a = slowFunction();
const b = fastFunction();
console.log(await a, await b);
}
This looks way simpler than what I used to do and is a lot more intuitive.
slowFunction() would start first, then fastFunction() would start, and console.log() would wait until both functions resolved before logging - and slowFunction() and fastFunction() are potentially running at the same time. I expect it's ultimatly on the browser whether or not thesea are seperate threads. But it looks like it walks and talks like crude multithreading. Is it?

Is using async and await the crude person's threads?
No, not at all. It's just syntax sugar (really, really useful sugar) over using promises, which in turn is just a (really, really useful) formalized way to use callbacks. It's useful because you can wait asynchronously (without blocking the JavaScript main thread) for things that are, by nature, asynchronous (like HTTP requests).
If you need to use threads, use web workers, Node.js worker threads, or whatever multi-threading your environment provides. Per specification (nowadays), only a single thread at a time is allowed to work within a given JavaScript "realm" (very loosely: the global environment your code is running in and its associated objects, etc.) and so only a single thread at a time has access to the variables and such within that realm, but threads can cooperate via messaging (including transferring objects between them without making copies) and shared memory.
For example:
async function isThisLikeTwoThreads() {
const a = slowFunction();
const b = fastFunction();
console.log(await a, await b);
}
Here's what that code does when isThisLikeTwoThreads is called:
slowFunction is called synchronously and its return value is assigned to a.
fastFunction is called synchronously and its return value is assigned to b.
When isThisLikeTwoThreads reaches await a, it wraps a in a promise (as though you did Promise.resolve(a)) and returns a new promise (not that same one). Let's call the promise wrapped around a "aPromise" and the promise returned by the function "functionPromise".
Later, when aPromise settles, if it was rejected functionPromise is rejected with the same rejection reason and the following steps are skipped; if it was fulfilled, the next step is done
The code in isThisLikeTwoThreads continues by wrapping b in a promise (bPromise) and waiting for that to settle
When bPromise settles, if it was rejected functionPromise is rejected with the same rejection reason; if it was fulfilled, the code in isThisLikeTwoThreads continues by logging the fulfillment values of aPromise and bPromise and then fulfilling functionPromise with the value undefined
All of the work above was done on the JavaScript thread where the call to isThisLikeTwoThreads was done, but it was spread out across multiple "jobs" (JavaScript terminology; the HTML spec calls them "tasks" and specifies a fair bit of detail for how they're handled on browsers). If slowFunction or fastFunction started an asynchronous process and returned a promise for that, that asynchronous process (for instance, an HTTP call the browser does) may have continued in parallel with the JavaScript thread while the JavaScript thread was doing other stuff or (if it was also JavaScript code on the main thread) may have competed for other work on the JavaScript thread (competed by adding jobs to the job queue and the thread processing them in a loop).
But using promises doesn't add threading. :-)

I suggest you read this to understand that the answer is no: https://developer.mozilla.org/en-US/docs/Web/JavaScript/EventLoop
To summarize, the runtime uses multiple threads for internal operations (network, disk... for a Node.js environment, rendering, indexedDB, network... for a browser environment) but the JavaScript code you wrote and the one you imported from different libraries will always execute in a single thread. Asynchronous operations will trigger callbacks which will be queued and executed one by one.
Basically what happens when executing this function:
async function isThisLikeTwoThreads() {
const a = slowFunction();
const b = fastFunction();
console.log(await a, await b);
}
Execute slowFunction.
Execute fastFunction.
Enqueue the rest of the code (console.log(await a, await b)) when a promise and b promise have resolved. The console.log runs in the same thread after isThisLikeTwoThreads has returned, and after the possible enqueued callbacks have returned. Assuming slowFunction and fastFunction both return promises, this is an equivalent of:
function isThisLikeTwoThreads() {
const a = slowFunction();
const b = fastFunction();
a.then(aResult => b.then(bResult => console.log(aResult, bResult)));
}

Similar but not the same. Javascript will only ever be doing 'one' thing at a time. It is fundamentally single-threaded. That said, it can appear odd as results may appear to be arriving in different orders - exacerbated by functions such as async and await.
For example, even if you're rendering a background process at 70fps there are small gaps in rendering logic available to complete promises or receive event notifications -- it's during these moments promises are completed giving the illusion of multi-threading.
If you REALLY want to lock up a browser try this:
let a = 1;
while (a == 1){
console.log("doing a thing");
}
You will never get javascript to work again and chrome or whatever will murder your script. The reason is when it enters that loop - nothing will touch it again, no events will be rendered and no promises will be delivered - hence, single threading.
If this was a true multithreaded environment you could break that loop from the outside by changing the variable to a new value.

Why doesn't await block lazily?

I'm interested to understand why using await immediately blocks, instead of lazily blocking only once the awaited value is referenced.
I hope this is an illuminating/important question, but I'm worried it may be considered out of scope for this site - if it is I'll apologize, have a little cry, and take the question elsewhere.
E.g. consider the following:
let delayedValue = (value, ms=1000) => {
return new Promise(resolve => {
setTimeout(() => resolve(val), ms);
});
};
(async () => {
let value = await delayedValue('val');
console.log('After await');
})();
In the anonymous async function which immediately runs, we'll only see the console say After await after the delay. Why is this necessary? Considering that we don't need value to be resolved, why haven't the language designers decided to execute the console.log statement immediately in a case like this?
For example, it's unlike the following example where delaying console.log is obviously unavoidable (because the awaited value is referenced):
(async () => {
let value = await delayedValue('val');
console.log('After await: ' + value);
});
I see tons of advantages to lazy await blocking - it could lead to automatic parallelization of unrelated operations. E.g. if I want to read two files and then work with both of them, and I'm not being dilligent, I'll write the following:
(async() => {
let file1 = await readFile('file1dir');
let file2 = await readFile('file2dir');
// ... do things with `file1` and `file2` ...
});
This will wait to have read the 1st file before beginning to read the 2nd. But really they could be read in parallel, and javascript ought to be able to detect this because file1 isn't referenced until later. When I was first learning async/await my initial expectation was that code like the above would result in parallel operations, and I was a bit disappointed when that turned out to be false.
Getting the two files to read in parallel is STILL a bit messy, even in this beautiful world of ES7 because await blocks immediately instead of lazily. You need to do something like the following (which is certainly messier than the above):
(async() => {
let [ read1, read2] = [ readFile('file1dir'), readFile('file2dir') ];
let [ file1, file2 ] = [ await read1, await read2];
// ... do things with `file1` and `file2` ...
});
Why have the language designers chosen to have await block immediately instead of lazily?
E.g. Could it lead to debugging difficulties? Would it be too difficult to integrate lazy awaits into javascript? Are there situations I haven't thought of where lazy awaits lead to messier code, poorer performance, or other negative consequences?

Reason 1: JavaScript is not lazily evaluated. There is no infrastructure to detect "when a value is actually needed".
Reason 2: Implicit parallelism would be very hard to control. What if I wanted my files to be read sequentially, how would I write that? Changing little things in the syntax should not lead to very different evaluation.
Getting the two files to read in parallel is STILL a bit messy
Not at all:
const [file1, file2] = await Promise.all([readFile('file1dir'), readFile('file2dir')]);
You need to do something like the following
No, you absolutely shouldn't write it like that.