I have an asynchronous function that runs in a loop. It should run indefinitely unless one of the inner async functions rejects or the application is terminated. It should not impact the rest of the application.
import { setTimeout } from 'timers/promises';
let executing = true;
const example = async () => {
while (executing) {
await setTimeout(1000);
await someOtherFunc();
await yetAnotherFunc();
}
}
In my code I execute it without awaiting (since that blocks further execution). Later, I stop the loop by changing the value.
const main = async () => {
// don't await
example();
// ... do stuff 'in parallel' to the asynchronous loop
// possibly weeks later, elsewhere in the code
executing = false;
}
main().then();
(Emphasis that the examples are contrived for Stack Overflow.)
It works, but it feels like a bit of a hack. Testing the execution/stop logic is tricky because the loop logic executes without the remaining test logic waiting. I need to mock the contents of the loop (the various async functions) for testing but because the test continues to execute without awaiting, the test later resets the mocks and the inner loop logic then tries to make unmocked calls.
What is the idiomatic way of achieving this in javascript / nodejs? I feel like a callback would allow me to keep using promises but I can't quite figure out how to achieve it.
// don't await
There's the problem. Your main function definitely should wait until the loop has completed before returning (fulfilling its promise) to its caller.
You can achieve that by
async function main() {
const promise = example();
// ... do stuff
executing = false;
await promise;
}
or better (with proper error handling)
async function stuff() {
// ... do stuff
executing = false;
}
async function main() {
await Promise.all([example(), stuff()];
}
Btw you probably shouldn't have a global executing variable. Better have one per call to example(), so that you can stop each running loop individually. Pass a mutable token object as an argument for that.
I'm attempting to get a better grasp on async functions and promises in JS. To do this, I wrote a sample program that has the goal of calling a function that does busy work (purposefully not using async setTimeout as I want to mimic a long-running process) but returns immediately. However, I can't seem to figure out why this isn't working.
test();
async function intense(){
var start = new Date().getTime();
for (var i = 0; i < 1e6; i++) {
if ((new Date().getTime() - start) > 2000){
break;
}
}
console.log("Done with async work");
}
async function test(){
console.log("Print 1");
intense(); // does some busy work for a few seconds
console.log("Print 2"); // want this to print immediately after print 1
}
When I run it, I get:
Print 1
Done with async work
Print 2
And I would like it to be:
Print 1
Print 2
Done with async work
I thought that it would print the latter sequence because I declared the function intense() to be async, so it would return a promise immediately and continue work asynchronously.
I even tried to refactor the intense function to be a promise that resolves immediately, but to no avail.
async function intense(){
return new Promise((resolve)=> {
resolve();
var start = new Date().getTime();
for (var i = 0; i < 1e6; i++) {
if ((new Date().getTime() - start) > 2000){
break;
}
}
console.log("Done with async work");
}, null)
}
What am I missing?
There a are a couple of reasons for what you're seeing:
An async function is synchronous up until its first await or return, so the entire function runs before returning in your case.
Busy-waiting isn't asynchronous.
test needs to use await if it's going to wait for intense to complete before continuing.
Moving something into a promise doesn't take it off the thread. The only way to do that in most JavaScript environments (including browsers) is to use a Worker thread (MDN, Node.js docs — Node.js has had Worker since ~v10.5, and while it's still marked "experimental" the main parts of the API should be fairly stable as they're drawn from the web worker standard API).
It's important to remember that promises don't make anything asynchronous¹, they provide a means of observing the result of something that's already asynchronous.
Here's an example using setTimeout for the asynchronous part that help you understand these better:
function delay(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
async function intense(value) {
console.log("intense(" + value + ") - This is synchronous");
await delay(100);
console.log("intense(" + value + ") - This is asynchronous, because it's after `await`");
}
async function test(){
console.log("Print 1");
intense(1); // <== WITHOUT await
console.log("Print 2");
await intense(2); // <== With await
console.log("Print 3");
}
test();
.as-console-wrapper {
max-height: 100% !important;
}
¹ There's one small caveat to that: The handler you pass to then, catch, or finally will always be called asynchronously, even if the promise you're calling them on is already settled. That's literally the only thing that promises actually make asynchronous.
so it would return a promise immediately and continue work asynchronously.
No it would not. The callback passed to the Promise constructor is called immeadiately. What is async is the process of calling resolve or reject later on, and how .then chains get called back somewhen.
However it isnt async in the sense that the code runs on another thread or gets deferred, that won't happen as JS itself is executed in a single thread*.
console.log(1);
const promise = new Promise((resolve, reject) => {
console.log(2); // gets executed immeadiately
});
promise.then(() => console.log(4)); // < Promise resolve asynchronously
console.log(3);
*If you plan to do really "intense" work, it might be benefitial to do that in another thread (see WebWorker in browsers and child_process.spawn for NodeJS).
This is a misunderstanding that is pretty easy to make with javascript. Your function intense() blocks the thread. Putting something in an async function does not change the fact that you only have one thread in javascript. As soon as the interpreted starts running that for loop it's going to use the one thread to run it until it's over. Nothing else will happen until then.
Async functions don't return immediately, they run the body of the code until the hit an await and return a promise. In your example, the entire function will run before it returns.
You can't use this kind of long running process without blocking. That's why node goes out of its way to offload things like i/o access and timers to another thread.
See here for more details: https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/
If you want to run some long-running code like this asynchronously you will need to spawn a child process: https://nodejs.org/api/child_process.html
I have some code here...
async function test() {
for(let i = 0; i < 10000; i++){
console.log("test");
}
}
console.log("a");
test();
console.log("b");
Now my understanding is that test() here should run asynchronously and if I would want to run a routine after it's completion I would use .then() after it.
I'm not doing that here though so surely it should just do it's thing in the background while everything else contiunes running? With that understanding I would expect some output like this:
a
b
test
test
..etc
However what I actually get is this...
a
test
test
...etc
b
Why does this happen? It seems that console.log("b") waits until test() completes but that's synchronous behaviour.
Apologies if something similar has been asked before but I couldn't find anything on this myself.
Setting something as async will wrap the return object into a Promise. Promises are used to deal with asynchronous behavior, but there's nothing inherently asynchronous about the content of the test function. If instead you were to wrap the contents into a setTimeout function, that would cause a delay which would mimic asynchronous behavior and match the outcome you were expecting.
async function test() {
setTimeout(() => {
for(let i = 0; i < 2; i++){
console.log("test");
}
})
}
console.log("a");
test();
console.log("b")
If you're looking to set up background tasks that can lock up the browser's main thread, you should check out WebWorkers. They are a way to offload performance impacting tasks onto a separate thread.
Synchronous code inside an async function runs synchronously. The interpreter will only move on to the next line after the call of the async function (here, the console.log("b");) after all synchronous code has finished - for example, if it runs into an await.
Here, since there's no await or anything asynchronous at all going on in test, test runs to the end before the next line (that logs b) runs.
It's pretty similar to how the Promise constructor runs.
From MDN:
An async function can contain an await expression, that pauses the execution of the async function and waits for the passed Promise's resolution, and then resumes the async function's execution and returns the resolved value.
If you do not have an await expression you are never 'pausing' the coroutine and it will execute synchronously without letting other functions run.
an async function means that what is inside the function is expected to have some asynchronous calls. like if you had an Http call inside your test function.
and since nothing is asynchronous inside your function (you only have a normal loop) then your function will execute normally and sequentially until it finishes executing.
async keyword don't mean that function wil run asynchronously. To run it asynchronously use setTimeout or Promise etc.
function test() {
setTimeout(() => {
for(let i = 0; i < 3; i++){
console.log("test");
}
});
}
console.log("a");
test();
console.log("b");
Async keyword usually is used with Promises and await keyword to write async code in synchronous way (to use await keyword we need put code inside async function)
function test() {
return new Promise( res => {
for(let i = 0; i < 3; i++){
console.log("test");
}
res();
});
}
async function start() {
console.log("a");
await test();
console.log("b");
}
start();
I am new to node js and trying to understand how to make async calls.
I am trying to wrap a function into a promise in order to make it async. For simplicity, the function is just a loop that takes time :
var p = new Promise(resolve => {
for (let i=0;i<999999999;i++){}
resolve('foo')
});
p.then(function (value) { console.log(value);});
console.log('bar');
I am expecting to see :
bar // printed immediately
foo // printed after the loop finishes
Instead, they are both printed after the loop completion.
How to make the loop block run asynchronously?
Ps. Apologies for the indentation/formatting, I had to write this from a phone.
Thanks
You seem to assume that "asynchronous code" involves concurrency, in the sense that code will run at the same time in another thread. That is not the case. Unless you start another thread by purpose, JS itself will run in a single thread. Therefore no matter what you do with promises: Either the loop runs first and then the logs run or the logs run first and then the loop runs.
You could also achieve concurrent execution through multitasking: If you stop the loop inbetween, other code can run in the meantime:
(async function() {
while(true) {
console.log("loop");
await Promise.resolve();
}
})();
console.log("in the meantime");
But there's nothing asynchronous about your promise. Creating a promise starts the execution of the function, and JS always runs to completion.
Typically, a promise is used to initiate something asynchronous, like an API call, or even a plain setTimeout, that runs outside JS's thread. Your code, however, will iterate through the empty loop bajillion times, and only after that any following lines will be run.
Replace the empty line with a timeout, and it will become async:
var p = new Promise(resolve => {
setTimeout(() => resolve("foo"), 2000);
});
p.then(function(value) {
console.log(value);
});
console.log('bar');
Take the following loop:
for(var i=0; i<100; ++i){
let result = await some_slow_async_function();
do_something_with_result();
}
Does await block the loop? Or does the i continue to be incremented while awaiting?
Is the order of do_something_with_result() guaranteed sequential with regard to i? Or does it depend on how fast the awaited function is for each i?
Does await block the loop? Or does the i continue to be incremented while awaiting?
"Block" is not the right word, but yes, i does not continue to be incremented while awaiting. Instead the execution jumps back to where the async function was called, providing a promise as return value, continuing the rest of the code that follows after the function call, until the code stack has been emptied. Then when the awaiting is over, the state of the function is restored, and execution continues within that function. Whenever that function returns (completes), the corresponding promise -- that was returned earlier on -- is resolved.
Is the order of do_something_with_result() guaranteed sequential with regard to i? Or does it depend on how fast the awaited function is for each i?
The order is guaranteed. The code following the await is also guaranteed to execute only after the call stack has been emptied, i.e. at least on or after the next microtask can execute.
See how the output is in this snippet. Note especially where it says "after calling test":
async function test() {
for (let i = 0; i < 2; i++) {
console.log('Before await for ', i);
let result = await Promise.resolve(i);
console.log('After await. Value is ', result);
}
}
test().then(_ => console.log('After test() resolved'));
console.log('After calling test');
As #realbart says, it does block the loop, which then will make the calls sequential.
If you want to trigger a ton of awaitable operations and then handle them all together, you could do something like this:
const promisesToAwait = [];
for (let i = 0; i < 100; i++) {
promisesToAwait.push(fetchDataForId(i));
}
const responses = await Promise.all(promisesToAwait);
You can test async/await inside a "FOR LOOP" like this:
(async () => {
for (let i = 0; i < 100; i++) {
await delay();
console.log(i);
}
})();
function delay() {
return new Promise((resolve, reject) => {
setTimeout(resolve, 100);
});
}
async functions return a Promise, which is an object that will eventually "resolve" to a value, or "reject" with an error. The await keyword means to wait until this value (or error) has been finalized.
So from the perspective of the running function, it blocks waiting for the result of the slow async function. The javascript engine, on the other hand, sees that this function is blocked waiting for the result, so it will go check the event loop (ie. new mouse clicks, or connection requests, etc.) to see if there are any other things it can work on until the results are returned.
Note however, that if the slow async function is slow because it is computing lots of stuff in your javascript code, the javascript engine won't have lots of resources to do other stuff (and by doing other stuff would likely make the slow async function even slower). Where the benefit of async functions really shine is for I/O intensive operations like querying a database or transmitting a large file where the javascript engine is well and truly waiting on something else (ie. database, filesystem, etc.).
The following two bits of code are functionally equivalent:
let result = await some_slow_async_function();
and
let promise = some_slow_async_function(); // start the slow async function
// you could do other stuff here while the slow async function is running
let result = await promise; // wait for the final value from the slow async function
In the second example above the slow async function is called without the await keyword, so it will start execution of the function and return a promise. Then you can do other things (if you have other things to do). Then the await keyword is used to block until the promise actually "resolves". So from the perspective of the for loop it will run synchronous.
So:
yes, the await keyword has the effect of blocking the running function until the async function either "resolves" with a value or "rejects" with an error, but it does not block the javascript engine, which can still do other things if it has other things to do while awaiting
yes, the execution of the loop will be sequential
There is an awesome tutorial about all this at http://javascript.info/async.
No Event loop isn't blocked, see example below
function sayHelloAfterSomeTime (ms) {
return new Promise((resolve, reject) => {
if (typeof ms !== 'number') return reject('ms must be a number')
setTimeout(() => {
console.log('Hello after '+ ms / 1000 + ' second(s)')
resolve()
}, ms)
})
}
async function awaitGo (ms) {
await sayHelloAfterSomeTime(ms).catch(e => console.log(e))
console.log('after awaiting for saying Hello, i can do another things ...')
}
function notAwaitGo (ms) {
sayHelloAfterSomeTime(ms).catch(e => console.log(e))
console.log('i dont wait for saying Hello ...')
}
awaitGo(1000)
notAwaitGo(1000)
console.log('coucou i am event loop and i am not blocked ...')
Here is my test solution about this interesting question:
import crypto from "crypto";
function diyCrypto() {
return new Promise((resolve, reject) => {
crypto.pbkdf2('secret', 'salt', 2000000, 64, 'sha512', (err, res) => {
if (err) {
reject(err)
return
}
resolve(res.toString("base64"))
})
})
}
setTimeout(async () => {
console.log("before await...")
const a = await diyCrypto();
console.log("after await...", a)
}, 0);
setInterval(() => {
console.log("test....")
}, 200);
Inside the setTimeout's callback the await blocks the execution. But the setInterval is keep runnning, so the Event Loop is running as usual.
Let me clarify a bit because some answers here have some wrong information about how Promise execution works, specifically when related to the event loop.
In the case of the example, await will block the loop. do_something_with_result() will not be called until await finishes it's scheduled job.
https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous/Async_await#handling_asyncawait_slowdown
As for the other points, Promise "jobs" run before the next event loop cycle, as microtasks. When you call Promise.then() or the resolve() function inside new Promise((resolve) => {}), you creating a Job. Both await and async are wrapper, of sorts, for Promise, that will both create a Job. Microtasks are meant to run before the next event loop cycle. That means adding a Promise Job means more work before it can move on to the next event loop cycle.
Here's an example how you can lock up your event loop because your promises (Jobs) take too long.
let tick = 0;
let time = performance.now();
setTimeout(() => console.log('Hi from timeout'), 0);
const tock = () => console.log(tick++);
const longTask = async () => {
console.log('begin task');
for(let i = 0; i < 1_000_000_000; i++) {
Math.sqrt(i);
}
console.log('done task');
}
requestAnimationFrame(()=> console.log('next frame after', performance.now() - time, 'ms'));
async function run() {
await tock();
await tock();
await longTask(); // Will stall your UI
await tock(); // Will execute even though it's already dropped frames
await tock(); // This will execute too
}
run();
// Promise.resolve().then(tock).then(tock).then(longTask).then(tock).then(tock);
In this sample, 5 total promises are created. 2 calls for tock, 1 for longTask and then 2 calls for tock. All 5 will run before the next event loop.
The execution would be:
Start JS execution
Execute normal script
Run 5 scheduled Promise jobs
End JS execution
Event Loop Cycle Start
Request Animation Frame fire
Timeout fire
The last line commented line is scheduling without async/await and results in the same.
Basically, you will stall the next event loop cycle unless you tell your JS execution where it can suspend. Your Promise jobs will continue to run in the current event loop run until it finishes its call stack. When you call something external, (like fetch), then it's likely using letting the call stack end and has a callback that will resolve the pending Promise. Like this:
function waitForClick() {
return new Promise((resolve) => {
// Use an event as a callback;
button.onclick = () => resolve();
// Let the call stack finish by implicitly not returning anything, or explicitly returning `undefined` (same thing).
// return undefined;
})
}
If you have a long job job that want to complete, either use a Web Worker to run it without pausing, or insert some pauses with something like setTimeout() or setImmediate().
Reshaping the longTask function, you can do something like this:
const longTask = async () => {
console.log('begin task');
for(let i = 0; i < 1_000_000_000; i++)
if (i && i % (10_000_000) === 0) {
await new Promise((r) => setTimeout(r,0));
}
Math.sqrt(i);
console.log('done task');
}
Basically, instead of doing 1 billion records in one shot, you only do 10 million and then wait until the next event (setTimeout) to run the next one. The bad here is it's slower because of how much you hand back to the event loop. Instead, you can use requestIdleCallback() which is better, but still not as good as multi-threading via Web Workers.
But be aware that just slapping on await or Promise.resolve().then() around a function won't help with the event loop. Both will wait until the function returns with either a Promise or a value before letting up for the event loop. You can mostly test by checking to see if the function you're calling returns an unresolved Promise immediately.
Does await block the loop? Or does the i continue to be incremented while awaiting?
No, await won't block the looping. Yes, i continues to be incremented while looping.
Is the order of do_something_with_result() guaranteed sequential with regard to i? Or does it depend on how fast the awaited function is for each i?
Order of do_something_with_result() is guaranteed sequentially but not with regards to i. It depends on how fast the awaited function runs.
All calls to some_slow_async_function() are batched, i.e., if do_something_with_result() was a console then we will see it printed the number of times the loop runs. And then sequentially, after this, all the await calls will be executed.
To better understand you can run below code snippet:
async function someFunction(){
for (let i=0;i<5;i++){
await callAPI();
console.log('After', i, 'th API call');
}
console.log("All API got executed");
}
function callAPI(){
setTimeout(()=>{
console.log("I was called at: "+new Date().getTime())}, 1000);
}
someFunction();
One can clearly see how line console.log('After', i, 'th API call'); gets printed first for entire stretch of the for loop and then at the end when all code is executed we get results from callAPI().
So if lines after await were dependent on result obtained from await calls then they will not work as expected.
To conclude, await in for-loop does not ensure successful operation on result obtained from await calls which might take some time to finish.
In node, if one uses neo-async library with waterfall, one can achieve this.