I have an asynchronous function that runs in a loop. It should run indefinitely unless one of the inner async functions rejects or the application is terminated. It should not impact the rest of the application.
import { setTimeout } from 'timers/promises';
let executing = true;
const example = async () => {
while (executing) {
await setTimeout(1000);
await someOtherFunc();
await yetAnotherFunc();
}
}
In my code I execute it without awaiting (since that blocks further execution). Later, I stop the loop by changing the value.
const main = async () => {
// don't await
example();
// ... do stuff 'in parallel' to the asynchronous loop
// possibly weeks later, elsewhere in the code
executing = false;
}
main().then();
(Emphasis that the examples are contrived for Stack Overflow.)
It works, but it feels like a bit of a hack. Testing the execution/stop logic is tricky because the loop logic executes without the remaining test logic waiting. I need to mock the contents of the loop (the various async functions) for testing but because the test continues to execute without awaiting, the test later resets the mocks and the inner loop logic then tries to make unmocked calls.
What is the idiomatic way of achieving this in javascript / nodejs? I feel like a callback would allow me to keep using promises but I can't quite figure out how to achieve it.
// don't await
There's the problem. Your main function definitely should wait until the loop has completed before returning (fulfilling its promise) to its caller.
You can achieve that by
async function main() {
const promise = example();
// ... do stuff
executing = false;
await promise;
}
or better (with proper error handling)
async function stuff() {
// ... do stuff
executing = false;
}
async function main() {
await Promise.all([example(), stuff()];
}
Btw you probably shouldn't have a global executing variable. Better have one per call to example(), so that you can stop each running loop individually. Pass a mutable token object as an argument for that.
Related
I would like to run this code with babel:
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
inside an async function without await the first line. is this OK?
how else can I run something that I don't care?
Can I just fire the non-promisified function del('key',null) without a callback?
Yes, you can do that, and it will run the two asynchronous functions in parallel. You've just created a promise and thrown it away.
However, this means that when the promise is rejected you won't notice. You'll just get an unhandledRejection eventually which will crash your process if not handled.
Is this OK? How can I run something that I don't care?
Probably it's not OK. If you truly wouldn't care, you hadn't run it in the first place. So you should be clear and explicit what you care about (and what not):
do you want to wait? (for side effects)
do you need the result?
do you want to catch exceptions?
If you only want to wait and don't care for the result value, you can easily throw away the result:
void (await someAsyncFunction()); // or omit the void keyword,
// doesn't make a difference in an expression statement
If you don't care about exceptions, you can ignore them using
… someAsyncFunction().catch(function ignore() {}) …
You can throw that away, await it, do anything with it.
If you want the result, you have to await it. If you care about exceptions, but don't really want to wait, you may want to execute it in parallel with the following functions:
var [_, res] = await Promise.all([
someAsyncFunction(), // result is ignored, exceptions aren't
someOtherAsyncFunction()
]);
return res;
inside an async function without await the first line. is this OK?
Yes, there are cases where you'd want to do this which are perfectly reasonable. Especially where you don't care about the result - one example is an analytics tracking operation that should not interfere with business critical code.
how else can I run something that I don't care?
In many ways, however simply calling the promise function works. Your del without a callback would probably work in this case but some functions don't guard against not passing callbacks, so you can pass an empty function instead (.del('key', () => {})).
You do want to however make sure that you know about it failing, even if you don't want to disrupt the operation of code - so please consider adding a process.on("unhandledRejection', event handler to explicitly ignore these particular exceptions or suppress them via:
redisClient.delAsync('key').catch(()=>{});
Or preferably, something like:
redisClient.delAsync('key').catch(logErr);
From all the research I've made so far, I think it's fine to do it, as long as you guarantee that the function you are not awaiting for guarantees a way to handle its own errors in case that happens. For example, a try-catch wrapping the whole function body, like you see in the following snippet for the asyncFunction.
It doesn't matter if the function throws synchronously or asynchronously. It guarantees the your mainFunction will complete no matter what. That's the key point here.
If you don't guarantee that, you have to risks:
If it throws synchronously, your main function will not complete.
If it throws asynchronously, you'll get an unhandled excepction
// THIS IS SOME API CALL YOU DON'T WANT TO WAIT FOR
const mockAPI = () => {
console.log("From mockAPI");
return new Promise((resolve,reject) => {
setTimeout(() => reject("LATE THROW: API ERROR"), 500);
});
};
// THIS IS THE SOME ASYNC FUNCTION YOU CALL BUT NOT AWAIT FOR
const asyncFunction = async (syncThrow) => {
try {
console.log("Async function START");
if (syncThrow) throw new Error("EARLY THROW");
await mockAPI();
console.log("Async function DONE");
}
catch(err) {
console.log("From async function catch");
console.log(err.message || err);
return;
}
};
// THIS IS YOUR MAIN FUNCTION
const mainFunction = async (syncThrow) => {
try {
console.clear();
console.log("Main function START");
asyncFunction(syncThrow);
console.log("Main function DONE <<< THAT'S THE IMPORTANT PART");
}
catch(err) {
console.log("THIS WILL NEVER HAPPEN");
console.log(err);
}
};
<div>
<button onClick="mainFunction(true)">Sync throw</button>
<button onClick="mainFunction(false)">Async throw</button>
</div>
Not in Node.js.
Node does not wait for ever-pending Promises. If other tasks are already completed and there is nothing left in the event loop, the Node process will be terminated even though there exists pending promise.
For the following script, if someOtherAsyncFunction() get resolved in 5 seconds, but redisClientAsync.delAsync('key') takes 10 seconds to execute, the Node process will be terminated after 5 seconds in theory, before the first line is resolved.
async function doSomething() {
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
}
await doSomething();
What if we do not wait for an asynchronous javascript function?
As far as I know some languages like C # should not run an asynchronous function unmanaged!
I wanted to know if this is also true for the JavaScript language?
var asynchronousFunction = async function() {
//...
}
function main() {
var result = true;
//...
asynchronousFunction(); // The result of this function has no effect on our output (result)
//...
return result;
}
It's run just the same. (In fact, you never await a function, you await for the the Promise it returns.)
The asynchronous function is run synchronously until the first await or return within it, at which point a Promise is returned to the caller and the rest of the function is arranged to run later.
It's up to the caller to do something (or nothing) to the Promise. After all, you might wish to store the promise in an array and await for the lot of them (Promise.all) or do something more esoteric about it, so JavaScript itself doesn't care.
Some smart enough IDEs and linters are able to raise a warning about unhandled promises, though, especially if you have enough type information to do so (e.g. by using TypeScript).
It's true for javascript as well.
You don't want to just create a promise and leave it totally hanging, if there are errors then they become unhandled errors and if it exits unexpectedly then you have no way of knowing that.
What I recommend is using the Promise.race at the top level and then it will run all of your async functions in parallel but will exit if any one of them exits unexpectedly.
async function worker() {
while (true) {
// do background work in a loop
}
}
async function server() {
await init()
await listen()
}
function main() {
const p0 = worker()
const p1 = server()
try {
await Promise.race([p0, p1])
console.log('done')
return true
} catch (err) {
console.log('The server had an error unexpectedly', err)
return false
}
}
If you expect the promises to eventually exit gracefully then use Promise.all instead, which will wait until all promises exit successfully before resolving.
I would like to run this code with babel:
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
inside an async function without await the first line. is this OK?
how else can I run something that I don't care?
Can I just fire the non-promisified function del('key',null) without a callback?
Yes, you can do that, and it will run the two asynchronous functions in parallel. You've just created a promise and thrown it away.
However, this means that when the promise is rejected you won't notice. You'll just get an unhandledRejection eventually which will crash your process if not handled.
Is this OK? How can I run something that I don't care?
Probably it's not OK. If you truly wouldn't care, you hadn't run it in the first place. So you should be clear and explicit what you care about (and what not):
do you want to wait? (for side effects)
do you need the result?
do you want to catch exceptions?
If you only want to wait and don't care for the result value, you can easily throw away the result:
void (await someAsyncFunction()); // or omit the void keyword,
// doesn't make a difference in an expression statement
If you don't care about exceptions, you can ignore them using
… someAsyncFunction().catch(function ignore() {}) …
You can throw that away, await it, do anything with it.
If you want the result, you have to await it. If you care about exceptions, but don't really want to wait, you may want to execute it in parallel with the following functions:
var [_, res] = await Promise.all([
someAsyncFunction(), // result is ignored, exceptions aren't
someOtherAsyncFunction()
]);
return res;
inside an async function without await the first line. is this OK?
Yes, there are cases where you'd want to do this which are perfectly reasonable. Especially where you don't care about the result - one example is an analytics tracking operation that should not interfere with business critical code.
how else can I run something that I don't care?
In many ways, however simply calling the promise function works. Your del without a callback would probably work in this case but some functions don't guard against not passing callbacks, so you can pass an empty function instead (.del('key', () => {})).
You do want to however make sure that you know about it failing, even if you don't want to disrupt the operation of code - so please consider adding a process.on("unhandledRejection', event handler to explicitly ignore these particular exceptions or suppress them via:
redisClient.delAsync('key').catch(()=>{});
Or preferably, something like:
redisClient.delAsync('key').catch(logErr);
From all the research I've made so far, I think it's fine to do it, as long as you guarantee that the function you are not awaiting for guarantees a way to handle its own errors in case that happens. For example, a try-catch wrapping the whole function body, like you see in the following snippet for the asyncFunction.
It doesn't matter if the function throws synchronously or asynchronously. It guarantees the your mainFunction will complete no matter what. That's the key point here.
If you don't guarantee that, you have to risks:
If it throws synchronously, your main function will not complete.
If it throws asynchronously, you'll get an unhandled excepction
// THIS IS SOME API CALL YOU DON'T WANT TO WAIT FOR
const mockAPI = () => {
console.log("From mockAPI");
return new Promise((resolve,reject) => {
setTimeout(() => reject("LATE THROW: API ERROR"), 500);
});
};
// THIS IS THE SOME ASYNC FUNCTION YOU CALL BUT NOT AWAIT FOR
const asyncFunction = async (syncThrow) => {
try {
console.log("Async function START");
if (syncThrow) throw new Error("EARLY THROW");
await mockAPI();
console.log("Async function DONE");
}
catch(err) {
console.log("From async function catch");
console.log(err.message || err);
return;
}
};
// THIS IS YOUR MAIN FUNCTION
const mainFunction = async (syncThrow) => {
try {
console.clear();
console.log("Main function START");
asyncFunction(syncThrow);
console.log("Main function DONE <<< THAT'S THE IMPORTANT PART");
}
catch(err) {
console.log("THIS WILL NEVER HAPPEN");
console.log(err);
}
};
<div>
<button onClick="mainFunction(true)">Sync throw</button>
<button onClick="mainFunction(false)">Async throw</button>
</div>
Not in Node.js.
Node does not wait for ever-pending Promises. If other tasks are already completed and there is nothing left in the event loop, the Node process will be terminated even though there exists pending promise.
For the following script, if someOtherAsyncFunction() get resolved in 5 seconds, but redisClientAsync.delAsync('key') takes 10 seconds to execute, the Node process will be terminated after 5 seconds in theory, before the first line is resolved.
async function doSomething() {
redisClientAsync.delAsync('key');
return await someOtherAsyncFunction();
}
await doSomething();
I am new to node js and trying to understand how to make async calls.
I am trying to wrap a function into a promise in order to make it async. For simplicity, the function is just a loop that takes time :
var p = new Promise(resolve => {
for (let i=0;i<999999999;i++){}
resolve('foo')
});
p.then(function (value) { console.log(value);});
console.log('bar');
I am expecting to see :
bar // printed immediately
foo // printed after the loop finishes
Instead, they are both printed after the loop completion.
How to make the loop block run asynchronously?
Ps. Apologies for the indentation/formatting, I had to write this from a phone.
Thanks
You seem to assume that "asynchronous code" involves concurrency, in the sense that code will run at the same time in another thread. That is not the case. Unless you start another thread by purpose, JS itself will run in a single thread. Therefore no matter what you do with promises: Either the loop runs first and then the logs run or the logs run first and then the loop runs.
You could also achieve concurrent execution through multitasking: If you stop the loop inbetween, other code can run in the meantime:
(async function() {
while(true) {
console.log("loop");
await Promise.resolve();
}
})();
console.log("in the meantime");
But there's nothing asynchronous about your promise. Creating a promise starts the execution of the function, and JS always runs to completion.
Typically, a promise is used to initiate something asynchronous, like an API call, or even a plain setTimeout, that runs outside JS's thread. Your code, however, will iterate through the empty loop bajillion times, and only after that any following lines will be run.
Replace the empty line with a timeout, and it will become async:
var p = new Promise(resolve => {
setTimeout(() => resolve("foo"), 2000);
});
p.then(function(value) {
console.log(value);
});
console.log('bar');
Take the following loop:
for(var i=0; i<100; ++i){
let result = await some_slow_async_function();
do_something_with_result();
}
Does await block the loop? Or does the i continue to be incremented while awaiting?
Is the order of do_something_with_result() guaranteed sequential with regard to i? Or does it depend on how fast the awaited function is for each i?
Does await block the loop? Or does the i continue to be incremented while awaiting?
"Block" is not the right word, but yes, i does not continue to be incremented while awaiting. Instead the execution jumps back to where the async function was called, providing a promise as return value, continuing the rest of the code that follows after the function call, until the code stack has been emptied. Then when the awaiting is over, the state of the function is restored, and execution continues within that function. Whenever that function returns (completes), the corresponding promise -- that was returned earlier on -- is resolved.
Is the order of do_something_with_result() guaranteed sequential with regard to i? Or does it depend on how fast the awaited function is for each i?
The order is guaranteed. The code following the await is also guaranteed to execute only after the call stack has been emptied, i.e. at least on or after the next microtask can execute.
See how the output is in this snippet. Note especially where it says "after calling test":
async function test() {
for (let i = 0; i < 2; i++) {
console.log('Before await for ', i);
let result = await Promise.resolve(i);
console.log('After await. Value is ', result);
}
}
test().then(_ => console.log('After test() resolved'));
console.log('After calling test');
As #realbart says, it does block the loop, which then will make the calls sequential.
If you want to trigger a ton of awaitable operations and then handle them all together, you could do something like this:
const promisesToAwait = [];
for (let i = 0; i < 100; i++) {
promisesToAwait.push(fetchDataForId(i));
}
const responses = await Promise.all(promisesToAwait);
You can test async/await inside a "FOR LOOP" like this:
(async () => {
for (let i = 0; i < 100; i++) {
await delay();
console.log(i);
}
})();
function delay() {
return new Promise((resolve, reject) => {
setTimeout(resolve, 100);
});
}
async functions return a Promise, which is an object that will eventually "resolve" to a value, or "reject" with an error. The await keyword means to wait until this value (or error) has been finalized.
So from the perspective of the running function, it blocks waiting for the result of the slow async function. The javascript engine, on the other hand, sees that this function is blocked waiting for the result, so it will go check the event loop (ie. new mouse clicks, or connection requests, etc.) to see if there are any other things it can work on until the results are returned.
Note however, that if the slow async function is slow because it is computing lots of stuff in your javascript code, the javascript engine won't have lots of resources to do other stuff (and by doing other stuff would likely make the slow async function even slower). Where the benefit of async functions really shine is for I/O intensive operations like querying a database or transmitting a large file where the javascript engine is well and truly waiting on something else (ie. database, filesystem, etc.).
The following two bits of code are functionally equivalent:
let result = await some_slow_async_function();
and
let promise = some_slow_async_function(); // start the slow async function
// you could do other stuff here while the slow async function is running
let result = await promise; // wait for the final value from the slow async function
In the second example above the slow async function is called without the await keyword, so it will start execution of the function and return a promise. Then you can do other things (if you have other things to do). Then the await keyword is used to block until the promise actually "resolves". So from the perspective of the for loop it will run synchronous.
So:
yes, the await keyword has the effect of blocking the running function until the async function either "resolves" with a value or "rejects" with an error, but it does not block the javascript engine, which can still do other things if it has other things to do while awaiting
yes, the execution of the loop will be sequential
There is an awesome tutorial about all this at http://javascript.info/async.
No Event loop isn't blocked, see example below
function sayHelloAfterSomeTime (ms) {
return new Promise((resolve, reject) => {
if (typeof ms !== 'number') return reject('ms must be a number')
setTimeout(() => {
console.log('Hello after '+ ms / 1000 + ' second(s)')
resolve()
}, ms)
})
}
async function awaitGo (ms) {
await sayHelloAfterSomeTime(ms).catch(e => console.log(e))
console.log('after awaiting for saying Hello, i can do another things ...')
}
function notAwaitGo (ms) {
sayHelloAfterSomeTime(ms).catch(e => console.log(e))
console.log('i dont wait for saying Hello ...')
}
awaitGo(1000)
notAwaitGo(1000)
console.log('coucou i am event loop and i am not blocked ...')
Here is my test solution about this interesting question:
import crypto from "crypto";
function diyCrypto() {
return new Promise((resolve, reject) => {
crypto.pbkdf2('secret', 'salt', 2000000, 64, 'sha512', (err, res) => {
if (err) {
reject(err)
return
}
resolve(res.toString("base64"))
})
})
}
setTimeout(async () => {
console.log("before await...")
const a = await diyCrypto();
console.log("after await...", a)
}, 0);
setInterval(() => {
console.log("test....")
}, 200);
Inside the setTimeout's callback the await blocks the execution. But the setInterval is keep runnning, so the Event Loop is running as usual.
Let me clarify a bit because some answers here have some wrong information about how Promise execution works, specifically when related to the event loop.
In the case of the example, await will block the loop. do_something_with_result() will not be called until await finishes it's scheduled job.
https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous/Async_await#handling_asyncawait_slowdown
As for the other points, Promise "jobs" run before the next event loop cycle, as microtasks. When you call Promise.then() or the resolve() function inside new Promise((resolve) => {}), you creating a Job. Both await and async are wrapper, of sorts, for Promise, that will both create a Job. Microtasks are meant to run before the next event loop cycle. That means adding a Promise Job means more work before it can move on to the next event loop cycle.
Here's an example how you can lock up your event loop because your promises (Jobs) take too long.
let tick = 0;
let time = performance.now();
setTimeout(() => console.log('Hi from timeout'), 0);
const tock = () => console.log(tick++);
const longTask = async () => {
console.log('begin task');
for(let i = 0; i < 1_000_000_000; i++) {
Math.sqrt(i);
}
console.log('done task');
}
requestAnimationFrame(()=> console.log('next frame after', performance.now() - time, 'ms'));
async function run() {
await tock();
await tock();
await longTask(); // Will stall your UI
await tock(); // Will execute even though it's already dropped frames
await tock(); // This will execute too
}
run();
// Promise.resolve().then(tock).then(tock).then(longTask).then(tock).then(tock);
In this sample, 5 total promises are created. 2 calls for tock, 1 for longTask and then 2 calls for tock. All 5 will run before the next event loop.
The execution would be:
Start JS execution
Execute normal script
Run 5 scheduled Promise jobs
End JS execution
Event Loop Cycle Start
Request Animation Frame fire
Timeout fire
The last line commented line is scheduling without async/await and results in the same.
Basically, you will stall the next event loop cycle unless you tell your JS execution where it can suspend. Your Promise jobs will continue to run in the current event loop run until it finishes its call stack. When you call something external, (like fetch), then it's likely using letting the call stack end and has a callback that will resolve the pending Promise. Like this:
function waitForClick() {
return new Promise((resolve) => {
// Use an event as a callback;
button.onclick = () => resolve();
// Let the call stack finish by implicitly not returning anything, or explicitly returning `undefined` (same thing).
// return undefined;
})
}
If you have a long job job that want to complete, either use a Web Worker to run it without pausing, or insert some pauses with something like setTimeout() or setImmediate().
Reshaping the longTask function, you can do something like this:
const longTask = async () => {
console.log('begin task');
for(let i = 0; i < 1_000_000_000; i++)
if (i && i % (10_000_000) === 0) {
await new Promise((r) => setTimeout(r,0));
}
Math.sqrt(i);
console.log('done task');
}
Basically, instead of doing 1 billion records in one shot, you only do 10 million and then wait until the next event (setTimeout) to run the next one. The bad here is it's slower because of how much you hand back to the event loop. Instead, you can use requestIdleCallback() which is better, but still not as good as multi-threading via Web Workers.
But be aware that just slapping on await or Promise.resolve().then() around a function won't help with the event loop. Both will wait until the function returns with either a Promise or a value before letting up for the event loop. You can mostly test by checking to see if the function you're calling returns an unresolved Promise immediately.
Does await block the loop? Or does the i continue to be incremented while awaiting?
No, await won't block the looping. Yes, i continues to be incremented while looping.
Is the order of do_something_with_result() guaranteed sequential with regard to i? Or does it depend on how fast the awaited function is for each i?
Order of do_something_with_result() is guaranteed sequentially but not with regards to i. It depends on how fast the awaited function runs.
All calls to some_slow_async_function() are batched, i.e., if do_something_with_result() was a console then we will see it printed the number of times the loop runs. And then sequentially, after this, all the await calls will be executed.
To better understand you can run below code snippet:
async function someFunction(){
for (let i=0;i<5;i++){
await callAPI();
console.log('After', i, 'th API call');
}
console.log("All API got executed");
}
function callAPI(){
setTimeout(()=>{
console.log("I was called at: "+new Date().getTime())}, 1000);
}
someFunction();
One can clearly see how line console.log('After', i, 'th API call'); gets printed first for entire stretch of the for loop and then at the end when all code is executed we get results from callAPI().
So if lines after await were dependent on result obtained from await calls then they will not work as expected.
To conclude, await in for-loop does not ensure successful operation on result obtained from await calls which might take some time to finish.
In node, if one uses neo-async library with waterfall, one can achieve this.