Take the following loop:
for(var i=0; i<100; ++i){
let result = await some_slow_async_function();
do_something_with_result();
}
Does await block the loop? Or does the i continue to be incremented while awaiting?
Is the order of do_something_with_result() guaranteed sequential with regard to i? Or does it depend on how fast the awaited function is for each i?
Does await block the loop? Or does the i continue to be incremented while awaiting?
"Block" is not the right word, but yes, i does not continue to be incremented while awaiting. Instead the execution jumps back to where the async function was called, providing a promise as return value, continuing the rest of the code that follows after the function call, until the code stack has been emptied. Then when the awaiting is over, the state of the function is restored, and execution continues within that function. Whenever that function returns (completes), the corresponding promise -- that was returned earlier on -- is resolved.
Is the order of do_something_with_result() guaranteed sequential with regard to i? Or does it depend on how fast the awaited function is for each i?
The order is guaranteed. The code following the await is also guaranteed to execute only after the call stack has been emptied, i.e. at least on or after the next microtask can execute.
See how the output is in this snippet. Note especially where it says "after calling test":
async function test() {
for (let i = 0; i < 2; i++) {
console.log('Before await for ', i);
let result = await Promise.resolve(i);
console.log('After await. Value is ', result);
}
}
test().then(_ => console.log('After test() resolved'));
console.log('After calling test');
As #realbart says, it does block the loop, which then will make the calls sequential.
If you want to trigger a ton of awaitable operations and then handle them all together, you could do something like this:
const promisesToAwait = [];
for (let i = 0; i < 100; i++) {
promisesToAwait.push(fetchDataForId(i));
}
const responses = await Promise.all(promisesToAwait);
You can test async/await inside a "FOR LOOP" like this:
(async () => {
for (let i = 0; i < 100; i++) {
await delay();
console.log(i);
}
})();
function delay() {
return new Promise((resolve, reject) => {
setTimeout(resolve, 100);
});
}
async functions return a Promise, which is an object that will eventually "resolve" to a value, or "reject" with an error. The await keyword means to wait until this value (or error) has been finalized.
So from the perspective of the running function, it blocks waiting for the result of the slow async function. The javascript engine, on the other hand, sees that this function is blocked waiting for the result, so it will go check the event loop (ie. new mouse clicks, or connection requests, etc.) to see if there are any other things it can work on until the results are returned.
Note however, that if the slow async function is slow because it is computing lots of stuff in your javascript code, the javascript engine won't have lots of resources to do other stuff (and by doing other stuff would likely make the slow async function even slower). Where the benefit of async functions really shine is for I/O intensive operations like querying a database or transmitting a large file where the javascript engine is well and truly waiting on something else (ie. database, filesystem, etc.).
The following two bits of code are functionally equivalent:
let result = await some_slow_async_function();
and
let promise = some_slow_async_function(); // start the slow async function
// you could do other stuff here while the slow async function is running
let result = await promise; // wait for the final value from the slow async function
In the second example above the slow async function is called without the await keyword, so it will start execution of the function and return a promise. Then you can do other things (if you have other things to do). Then the await keyword is used to block until the promise actually "resolves". So from the perspective of the for loop it will run synchronous.
So:
yes, the await keyword has the effect of blocking the running function until the async function either "resolves" with a value or "rejects" with an error, but it does not block the javascript engine, which can still do other things if it has other things to do while awaiting
yes, the execution of the loop will be sequential
There is an awesome tutorial about all this at http://javascript.info/async.
No Event loop isn't blocked, see example below
function sayHelloAfterSomeTime (ms) {
return new Promise((resolve, reject) => {
if (typeof ms !== 'number') return reject('ms must be a number')
setTimeout(() => {
console.log('Hello after '+ ms / 1000 + ' second(s)')
resolve()
}, ms)
})
}
async function awaitGo (ms) {
await sayHelloAfterSomeTime(ms).catch(e => console.log(e))
console.log('after awaiting for saying Hello, i can do another things ...')
}
function notAwaitGo (ms) {
sayHelloAfterSomeTime(ms).catch(e => console.log(e))
console.log('i dont wait for saying Hello ...')
}
awaitGo(1000)
notAwaitGo(1000)
console.log('coucou i am event loop and i am not blocked ...')
Here is my test solution about this interesting question:
import crypto from "crypto";
function diyCrypto() {
return new Promise((resolve, reject) => {
crypto.pbkdf2('secret', 'salt', 2000000, 64, 'sha512', (err, res) => {
if (err) {
reject(err)
return
}
resolve(res.toString("base64"))
})
})
}
setTimeout(async () => {
console.log("before await...")
const a = await diyCrypto();
console.log("after await...", a)
}, 0);
setInterval(() => {
console.log("test....")
}, 200);
Inside the setTimeout's callback the await blocks the execution. But the setInterval is keep runnning, so the Event Loop is running as usual.
Let me clarify a bit because some answers here have some wrong information about how Promise execution works, specifically when related to the event loop.
In the case of the example, await will block the loop. do_something_with_result() will not be called until await finishes it's scheduled job.
https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous/Async_await#handling_asyncawait_slowdown
As for the other points, Promise "jobs" run before the next event loop cycle, as microtasks. When you call Promise.then() or the resolve() function inside new Promise((resolve) => {}), you creating a Job. Both await and async are wrapper, of sorts, for Promise, that will both create a Job. Microtasks are meant to run before the next event loop cycle. That means adding a Promise Job means more work before it can move on to the next event loop cycle.
Here's an example how you can lock up your event loop because your promises (Jobs) take too long.
let tick = 0;
let time = performance.now();
setTimeout(() => console.log('Hi from timeout'), 0);
const tock = () => console.log(tick++);
const longTask = async () => {
console.log('begin task');
for(let i = 0; i < 1_000_000_000; i++) {
Math.sqrt(i);
}
console.log('done task');
}
requestAnimationFrame(()=> console.log('next frame after', performance.now() - time, 'ms'));
async function run() {
await tock();
await tock();
await longTask(); // Will stall your UI
await tock(); // Will execute even though it's already dropped frames
await tock(); // This will execute too
}
run();
// Promise.resolve().then(tock).then(tock).then(longTask).then(tock).then(tock);
In this sample, 5 total promises are created. 2 calls for tock, 1 for longTask and then 2 calls for tock. All 5 will run before the next event loop.
The execution would be:
Start JS execution
Execute normal script
Run 5 scheduled Promise jobs
End JS execution
Event Loop Cycle Start
Request Animation Frame fire
Timeout fire
The last line commented line is scheduling without async/await and results in the same.
Basically, you will stall the next event loop cycle unless you tell your JS execution where it can suspend. Your Promise jobs will continue to run in the current event loop run until it finishes its call stack. When you call something external, (like fetch), then it's likely using letting the call stack end and has a callback that will resolve the pending Promise. Like this:
function waitForClick() {
return new Promise((resolve) => {
// Use an event as a callback;
button.onclick = () => resolve();
// Let the call stack finish by implicitly not returning anything, or explicitly returning `undefined` (same thing).
// return undefined;
})
}
If you have a long job job that want to complete, either use a Web Worker to run it without pausing, or insert some pauses with something like setTimeout() or setImmediate().
Reshaping the longTask function, you can do something like this:
const longTask = async () => {
console.log('begin task');
for(let i = 0; i < 1_000_000_000; i++)
if (i && i % (10_000_000) === 0) {
await new Promise((r) => setTimeout(r,0));
}
Math.sqrt(i);
console.log('done task');
}
Basically, instead of doing 1 billion records in one shot, you only do 10 million and then wait until the next event (setTimeout) to run the next one. The bad here is it's slower because of how much you hand back to the event loop. Instead, you can use requestIdleCallback() which is better, but still not as good as multi-threading via Web Workers.
But be aware that just slapping on await or Promise.resolve().then() around a function won't help with the event loop. Both will wait until the function returns with either a Promise or a value before letting up for the event loop. You can mostly test by checking to see if the function you're calling returns an unresolved Promise immediately.
Does await block the loop? Or does the i continue to be incremented while awaiting?
No, await won't block the looping. Yes, i continues to be incremented while looping.
Is the order of do_something_with_result() guaranteed sequential with regard to i? Or does it depend on how fast the awaited function is for each i?
Order of do_something_with_result() is guaranteed sequentially but not with regards to i. It depends on how fast the awaited function runs.
All calls to some_slow_async_function() are batched, i.e., if do_something_with_result() was a console then we will see it printed the number of times the loop runs. And then sequentially, after this, all the await calls will be executed.
To better understand you can run below code snippet:
async function someFunction(){
for (let i=0;i<5;i++){
await callAPI();
console.log('After', i, 'th API call');
}
console.log("All API got executed");
}
function callAPI(){
setTimeout(()=>{
console.log("I was called at: "+new Date().getTime())}, 1000);
}
someFunction();
One can clearly see how line console.log('After', i, 'th API call'); gets printed first for entire stretch of the for loop and then at the end when all code is executed we get results from callAPI().
So if lines after await were dependent on result obtained from await calls then they will not work as expected.
To conclude, await in for-loop does not ensure successful operation on result obtained from await calls which might take some time to finish.
In node, if one uses neo-async library with waterfall, one can achieve this.
Related
I have an asynchronous function that runs in a loop. It should run indefinitely unless one of the inner async functions rejects or the application is terminated. It should not impact the rest of the application.
import { setTimeout } from 'timers/promises';
let executing = true;
const example = async () => {
while (executing) {
await setTimeout(1000);
await someOtherFunc();
await yetAnotherFunc();
}
}
In my code I execute it without awaiting (since that blocks further execution). Later, I stop the loop by changing the value.
const main = async () => {
// don't await
example();
// ... do stuff 'in parallel' to the asynchronous loop
// possibly weeks later, elsewhere in the code
executing = false;
}
main().then();
(Emphasis that the examples are contrived for Stack Overflow.)
It works, but it feels like a bit of a hack. Testing the execution/stop logic is tricky because the loop logic executes without the remaining test logic waiting. I need to mock the contents of the loop (the various async functions) for testing but because the test continues to execute without awaiting, the test later resets the mocks and the inner loop logic then tries to make unmocked calls.
What is the idiomatic way of achieving this in javascript / nodejs? I feel like a callback would allow me to keep using promises but I can't quite figure out how to achieve it.
// don't await
There's the problem. Your main function definitely should wait until the loop has completed before returning (fulfilling its promise) to its caller.
You can achieve that by
async function main() {
const promise = example();
// ... do stuff
executing = false;
await promise;
}
or better (with proper error handling)
async function stuff() {
// ... do stuff
executing = false;
}
async function main() {
await Promise.all([example(), stuff()];
}
Btw you probably shouldn't have a global executing variable. Better have one per call to example(), so that you can stop each running loop individually. Pass a mutable token object as an argument for that.
I'm attempting to get a better grasp on async functions and promises in JS. To do this, I wrote a sample program that has the goal of calling a function that does busy work (purposefully not using async setTimeout as I want to mimic a long-running process) but returns immediately. However, I can't seem to figure out why this isn't working.
test();
async function intense(){
var start = new Date().getTime();
for (var i = 0; i < 1e6; i++) {
if ((new Date().getTime() - start) > 2000){
break;
}
}
console.log("Done with async work");
}
async function test(){
console.log("Print 1");
intense(); // does some busy work for a few seconds
console.log("Print 2"); // want this to print immediately after print 1
}
When I run it, I get:
Print 1
Done with async work
Print 2
And I would like it to be:
Print 1
Print 2
Done with async work
I thought that it would print the latter sequence because I declared the function intense() to be async, so it would return a promise immediately and continue work asynchronously.
I even tried to refactor the intense function to be a promise that resolves immediately, but to no avail.
async function intense(){
return new Promise((resolve)=> {
resolve();
var start = new Date().getTime();
for (var i = 0; i < 1e6; i++) {
if ((new Date().getTime() - start) > 2000){
break;
}
}
console.log("Done with async work");
}, null)
}
What am I missing?
There a are a couple of reasons for what you're seeing:
An async function is synchronous up until its first await or return, so the entire function runs before returning in your case.
Busy-waiting isn't asynchronous.
test needs to use await if it's going to wait for intense to complete before continuing.
Moving something into a promise doesn't take it off the thread. The only way to do that in most JavaScript environments (including browsers) is to use a Worker thread (MDN, Node.js docs — Node.js has had Worker since ~v10.5, and while it's still marked "experimental" the main parts of the API should be fairly stable as they're drawn from the web worker standard API).
It's important to remember that promises don't make anything asynchronous¹, they provide a means of observing the result of something that's already asynchronous.
Here's an example using setTimeout for the asynchronous part that help you understand these better:
function delay(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
async function intense(value) {
console.log("intense(" + value + ") - This is synchronous");
await delay(100);
console.log("intense(" + value + ") - This is asynchronous, because it's after `await`");
}
async function test(){
console.log("Print 1");
intense(1); // <== WITHOUT await
console.log("Print 2");
await intense(2); // <== With await
console.log("Print 3");
}
test();
.as-console-wrapper {
max-height: 100% !important;
}
¹ There's one small caveat to that: The handler you pass to then, catch, or finally will always be called asynchronously, even if the promise you're calling them on is already settled. That's literally the only thing that promises actually make asynchronous.
so it would return a promise immediately and continue work asynchronously.
No it would not. The callback passed to the Promise constructor is called immeadiately. What is async is the process of calling resolve or reject later on, and how .then chains get called back somewhen.
However it isnt async in the sense that the code runs on another thread or gets deferred, that won't happen as JS itself is executed in a single thread*.
console.log(1);
const promise = new Promise((resolve, reject) => {
console.log(2); // gets executed immeadiately
});
promise.then(() => console.log(4)); // < Promise resolve asynchronously
console.log(3);
*If you plan to do really "intense" work, it might be benefitial to do that in another thread (see WebWorker in browsers and child_process.spawn for NodeJS).
This is a misunderstanding that is pretty easy to make with javascript. Your function intense() blocks the thread. Putting something in an async function does not change the fact that you only have one thread in javascript. As soon as the interpreted starts running that for loop it's going to use the one thread to run it until it's over. Nothing else will happen until then.
Async functions don't return immediately, they run the body of the code until the hit an await and return a promise. In your example, the entire function will run before it returns.
You can't use this kind of long running process without blocking. That's why node goes out of its way to offload things like i/o access and timers to another thread.
See here for more details: https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/
If you want to run some long-running code like this asynchronously you will need to spawn a child process: https://nodejs.org/api/child_process.html
I would like to know, if awaiting a resolved promise, will lead to a synchronous code execution or may lead to asynchronous one.
I made this little snippet to check on browsers :
const promise = new Promise((resolve) => {
console.log('exec promise');
setTimeout(() => {
console.log('executed promise');
resolve();
}, 1000);
});
(async () => {
console.log('start');
for (let i = 0; i < 1e8; i += 1) {
await promise;
}
console.log('end');
})();
And it looks like browsers make it synchronous (considering the screen freeze).
BUT ... is it due to browser specific implementation ? Or by design ?
Here's an alternative demo that doesn't freeze the browser (which I think is getting the way of what we're trying to show), and which shows the opposite behaviour to your initial conclusion:
const p = new Promise((r) => r());
p.then(() => {
(async() => {
console.log('a');
await p;
console.log('p');
})()
console.log('b');
});
In this case, we get hold of p, a resolved promise. We then kick off an async function that awaits it. The output is:
a
b
p
IE the async function still returned control back (and b was logged) when it hit the await on the already resolved p. Only after b was logged and the event loop was free to return execution back to after the await was p subsequently logged.
Is this standard behaviour? Yes.
The specification effectively turns the awaited expression to the then part of a promise, and in steps 9 and 10 of a subsequent step makes the decision on how to proceed.
If promise.[[PromiseState]] is pending, then
a. Append fulfillReaction as the last element of the List that is promise.[[PromiseFulfillReactions]].
b. Append rejectReaction as the last element of the List that is promise.[PromiseRejectReactions]].
Else if promise.[[PromiseState]] is fulfilled,
a. then Let value be promise.[[PromiseResult]].
b. Let fulfillJob be NewPromiseReactionJob(fulfillReaction, value)
c. Perform HostEnqueuePromiseJob(fulfillJob.[[Job]], fulfillJob.[[Realm]]).
Step 9 says if it's pending, to add the generated then to the list of things to be called when the promise is fulfilled.
Step 10 gets to the heart of your question - it says if it's fulfilled already, to enqueue the job - ie to place it on the queue of things to be called back.
The spec effectively says that an await should never return synchronously.
This is expected behavior. You freeze the event loop there, because the promise has already been resolved, the next await operator will wait&resolve for the next promise descriptor on the same iteration of the event loop. But this behavior can be platform-specific, depending on which promise implementation you're actually using - native or shim.
const promise= Promise.resolve();
(async()=>{
for(let i=0; i< 10000000; i++) {
await promise;
}
console.log('end');
})()
setTimeout(()=> console.log('nextEventLoopTick'), 0);
Will output the follwing:
end
nextEventLoopTick
As you can see, the async fn will be fully resolved before setTimeout. So, this indicates that the async function is resolved on the same eventloop tick.
As far as I understand, an async function allows to execute code asynchronously but it doesn't seem to work in my code :
async function longAsyncWork() {
await foo(); // takes about 10 seconds to complete
}
console.log('start');
longAsyncWork();
console.log('end');
The 'end' is written in the console after the longAsyncWork has completed but I thought that the goal of async functions was to allow to execute code without blocking the main execution.
Maybe it's a particularity of the console.log call that prevent me from seeing the truth? Can I assume that the code after the longAsyncWork call will be executed before the completion of the longAsyncWork function?
Asnyc functions work asynchronously but javascript is single-threaded, which means the browser can do only one task at a time. What you do inside the function must be asynchronous by nature to actually make it work, however you probably did some blocking operation there that is synchronous by nature. Let me explain in more details by giving example:
async function fooThatCanBeAsync(){
var result = await fetch("https://stackoverflow.com");
console.log("after async task 1");
return result;
}
async function fooThatCannotBeAsync(){
var x = 0;
for(i=0; i<10000000; i++){
x += i;
}
console.log("after async task 2");
}
fooThatCanBeAsync();
fooThatCannotBeAsync();
console.log("end...");
The above silly example has two async functions. However, only one of them can run async!
The first function tries to fetch something by sending http request and it will take some time. This means, cpu is now free until it starts receiving the result. By making this function async, you can now use this time in between to do other tasks. This is how concurrency works in javascript.
The second function does a for loop, making some calculations. Those calculations cannot be async, because there is no way to interrupt it, no time to wait in between, and no i/o operation. Just because you add async keyword, you can't change the fact that only one cpu thread is executing that code.
Simply making a function async does not mean that expensive processing inside that function will not block the main thread - rather, it'll mean that the function automatically returns a Promise, and that await can be used inside of it (among other things). Any synchronous work that the function does (including any synchronous work done by await calls inside the function) will still block the main thread.
const foo = () => new Promise((resolve) => {
for (let i = 0; i < 1e9; i++) {
}
console.log('foo expensive sync work done');
resolve();
});
async function longAsyncWork() {
await foo(); // takes about 10 seconds to complete
}
console.log('start');
longAsyncWork();
console.log('end, still on main thread');
The final Promise that the call of longAsyncWork resolves to (or any awaits inside) will not resolve until after the main thread is finished, but synchronous processing before any awaits have resolved will still block.
If you want to ensure that expensive processing doesn't block the main thread, you can use a web worker instead:
const workerFn = () => {
self.onmessage = () => {
for (let i = 0; i < 1e9; i++) {
}
self.postMessage('done');
}
};
const workerFnStr = `(${workerFn})();`;
const blob = new Blob([workerFnStr], { type: 'text/javascript' });
const worker = new Worker(window.URL.createObjectURL(blob));
worker.onmessage = (result) => {
console.log('result:', result.data);
};
console.log('about to post message');
worker.postMessage('arg');
console.log('main thread continuing here');
You can also call Promise.resolve() before running longAsyncWork, but unlike the worker version, this will still block the thread of the parent window (though, it will be after the main thread has finished). This is often undesirable because it can prevent the user from interacting with the page - a web worker is often a better choice.
const foo = () => new Promise((resolve) => {
for (let i = 0; i < 1e10; i++) {
}
console.log('foo expensive sync work done');
resolve();
});
async function longAsyncWork() {
await foo(); // takes about 10 seconds to complete
}
console.log('start');
Promise.resolve()
.then(longAsyncWork);
console.log('end, still on main thread');
<p>Page won't display for a while, or if page is already displayed, scrolling won't work, because main thread is blocked</p>
A working version of your code.
const sleep = ms => new Promise(r => setTimeout(r, ms))
async function longAsyncWork() {
// await resolves the sleep promise, so it lock longAsyncWork.
await sleep(1000)
console.log('async')
}
console.log('start');
// async longAsyncWork is a promise, so if you doesn't lock it with await, it will run async.
longAsyncWork();
console.log('end');
press Run code snippet to see it working.
I am new to node js and trying to understand how to make async calls.
I am trying to wrap a function into a promise in order to make it async. For simplicity, the function is just a loop that takes time :
var p = new Promise(resolve => {
for (let i=0;i<999999999;i++){}
resolve('foo')
});
p.then(function (value) { console.log(value);});
console.log('bar');
I am expecting to see :
bar // printed immediately
foo // printed after the loop finishes
Instead, they are both printed after the loop completion.
How to make the loop block run asynchronously?
Ps. Apologies for the indentation/formatting, I had to write this from a phone.
Thanks
You seem to assume that "asynchronous code" involves concurrency, in the sense that code will run at the same time in another thread. That is not the case. Unless you start another thread by purpose, JS itself will run in a single thread. Therefore no matter what you do with promises: Either the loop runs first and then the logs run or the logs run first and then the loop runs.
You could also achieve concurrent execution through multitasking: If you stop the loop inbetween, other code can run in the meantime:
(async function() {
while(true) {
console.log("loop");
await Promise.resolve();
}
})();
console.log("in the meantime");
But there's nothing asynchronous about your promise. Creating a promise starts the execution of the function, and JS always runs to completion.
Typically, a promise is used to initiate something asynchronous, like an API call, or even a plain setTimeout, that runs outside JS's thread. Your code, however, will iterate through the empty loop bajillion times, and only after that any following lines will be run.
Replace the empty line with a timeout, and it will become async:
var p = new Promise(resolve => {
setTimeout(() => resolve("foo"), 2000);
});
p.then(function(value) {
console.log(value);
});
console.log('bar');