I'm running into a few different problems when trying to move nested Promise.all's into await/async.
I think I'm just not getting how I should be using Promise.all and await.
I want to do something like below - loop through an array, perform an action on it, and the results being saved to a varibale - ready to be used on the next Promise.all.
doThing() and doAThing() are both async functions.
const foo = await Promise.all(arr.map(p => doAThing(p)));
const bar = await Promise.all(foo.map(p => doAnotherThing(p)));
I'm getting lots of undefined, and it's definitely not waiting for the results to fill before continuing.
I'm really just trying to avoid nested promise alls.
Thanks,
Ollie
If doAThing() returns a promise that resolves to a value when all the asynchronous operations in it are done, then foo will be an array of those resolved values.
If you get undefined values in the foo array, then you either aren't returning anything from doAThing() or you're returning a promise that resolves to undefined.
If the await isn't waiting for all the async operations to complete, then you either aren't returning a promise at all from doAThing() or that promise is getting resolved before all the async operations are done. For all this to work properly, doAThing() has to return a promise that is ONLY resolved when all the async operations you wanted to wait for are actually done. Promises provide no magic. They have to be wired up properly to resolve when their corresponding asynchronous operations are done. If you have multiple async operations in doAThing(), then those have to be all monitored together so the function returns a single promise when they are all done (either chained or using Promise.all() themselves).
For us to help you more specifically, you'd have to show us the code for doAThing(). Then, we could show you exactly where your code needs fixing.
Related
I've researched many many posts and articles, and documentation. I'm seeking deeper understanding.
From https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/await
The await expression causes async function execution to pause until a Promise is settled, [...]
let fetchLocation = () => {
//fetches information from satellite system and
//returns a Vector2 of lat long on earth
return new Promise(resolve => {
setTimeout(() => {
resolve({
lat: 73,
long: -24
});
}, 2000);
});
}
//async functions automatically return a promise.
let main = async() => {
//awaits the promises resolution
let result = await fetchLocation();
console.log("I'm the awaited promise result", result)
//immediately returns a promise as promise pending
return result;
}
console.log(main().then(res => console.log("I'm the original promise, just passed along by the async function, it didn 't a-wait for me", res)))
Before going down this rabbit hole, I knew async/await was syntactical sugar to make async code look synchronous.
I therefore fully expected to see main return the lat long coordinates. Instead it returns a promise.
A couple questions.
Is the MDN documentation wrong? Does it not actually pause execution? It seems that the await keyword does not actually "pause" function execution... invoking main immediately returns a Promise<pending> object. However, after it returns we receive the value of the fulfilled promise in the awaited variable result. I suppose this came from the event loop? (And an async return statement is still synchronous.)
So it seems that await unwraps a promise, and async wraps a promise, but you need to use await within async. I get that await might actually be promise.then() underneath the hood but if they want to make it fully synchronous in appearance, why not have the async function return the resolved promise value when used with await? Otherwise async await seems a bit redundant.
Is the MDN documentation wrong?
No
Does it not actually pause execution?
It pauses the execution of the async function, hands a Promise back to the calling function — because it hasn't reached the point in the async function where it knows what to return — then the execution of the calling function (and the rest of the code) continues.
However, after it returns we receive the value of the fulfilled promise in the awaited variable result. I suppose this came from the event loop?
When the event loop stops being busy (running the code that called the async function in the first place and anything else that it picks up in the meantime) and the promise that was being awaited resolves, then the async function is woken up and given the resolved value of the function to continue processing.
why not have the async function return the resolved promise value when used with await?
Because that would block the entire event loop, which is why asynchronous logic (starting with callback style APIs) was introduced into JS in the first place.
Otherwise async await seems a bit redundant.
Consider a situation where you want to request a number of URLs from an API serially (so that you don't shove too many simultaneous requests at the API):
const data = [];
for (let i = 0; i < urls.length; i++) {
const response = await fetch(urls[i]);
const response_data = await response.json();
data.push(response_data);
}
If you were to rewrite that without await then you'd probably end up with something recursive to count your way through the loop. It would be significantly more complex.
async/await makes dealing with complex combinations of promises simple, and simple promises trivial.
knew async/await was syntactical sugar to make async code look synchronous.
And it does that inside the async function and only inside it.
It doesn't really pause the function, it just allows you to write the rest of the function code as if it were. But it's really just wrapping the rest of the code of the function into a .then().
E.g.
await foo = someFunc();
console.log(foo);
is executed like
someFunc().then((foo => {
console.log(foo);
});
Since it's really still asynchronous, you can't return the resolved value, because the original calling function returns immediately. But if the calling function is also declared async, it returns a Promise, and the caller can either use await again (so it appears to be synchronous) or use .then().
This question already has answers here:
When should I use a return statement in ES6 arrow functions
(6 answers)
Closed 27 days ago.
I just wonder if there is different between those 2 calls by await Promise.all([/* ... /*])
In my performance test, it took the same time.
This is my method to be invoked by the Promise.all:
const makeFetch1 = async () => {
axios.get("https://jsonplaceholder.typicode.com/todos/1");
};
const makeFetch2 = async () => {
return axios.get("https://jsonplaceholder.typicode.com/todos/1");
};
1) await Promise.all([makeFetch1(), makeFetch1(), ...])
2) await Promise.all([makeFetch2(), makeFetch2(), ...])
Both have the same execution time result, what i should choose?
The first one will fire of a request and not return it. so makeFetch1() will result in a Promise that resolves immediately. This is almost certainly never what you want. An async function should await or return inside somewhere, otherwise it may as well be a standard function (that returns nothing).
Even if you don't care about what the request resolves to, you should still use the result of the request (by returning or awaiting, just so you can see if the request succeeds or not - and communicate to the user that there was a problem if it fails)
The second one will fire of a request and return it. so makeFetch1() will result in a Promise that resolves once the request resolves - which is likely to take a number of milliseconds.
There are several major differences and multiple coding mistakes.
First off this code:
const makeFetch1 = async () => {
axios.get("https://jsonplaceholder.typicode.com/todos/1");
};
is an async function that calls axios.get() and then immediately returns a promise that is already resolved and has an undefined resolved value. Nothing here waits for the axios.get() call to finish.
Your second code block:
const makeFetch2 = async () => {
return axios.get("https://jsonplaceholder.typicode.com/todos/1");
};
calls axios.get() and returns the promise that it creates. So, the promise this function returns will be tied to when the axios.get() call completes and the resolved value will be the resolved value of the axios call.
Differences:
You could use the makeFetch2() function to get the result of the axios.get() call (resolved value or reject error). You could not use makeFetch1() to get the result of the axios.get() call because that result is not communicated back to the caller in any way.
Similarly, makeFetch2() will allow the caller to know when the axios operation has completed. makeFetch1() will not.
If the axios.get() operation rejected, then makeFetch1() would result in an unhandled rejection error because nothing is listening for the rejection.
Then, these two:
await Promise.all([makeFetch1, makeFetch1, ...])
await Promise.all([makeFetch2, makeFetch2, ...])
Are both pretty useless because you're not calling any of the functions so nothing is executing. These will both just resolve immediately with a resolved value of an array of function references. None of the axios calls are executed because nothing actually calls makeFetch1() or makeFetch2().
If you meant to actually execute the functions with these two:
await Promise.all([makeFetch1(), makeFetch1(), ...])
await Promise.all([makeFetch2(), makeFetch2(), ...])
Then, the first one just resolves immediately with an array of undefined values as the resolved value because remember the makeFetch1() return value isn't connected in any way to the axios.get() call.
The second one will properly wait for all the axios.get() calls in makeFetch2() to complete and will resolve with an array of values from the axios.get() calls.
Both have the same execution time result, what i should choose?
That's because as you've implemented them, you aren't calling the makeFetchN() functions in either case. So, they aren't doing anything and thus have the same execution time.
If you change the implementation to actually call the functions as in:
await Promise.all([makeFetch1(), makeFetch1(), ...])
await Promise.all([makeFetch2(), makeFetch2(), ...])
There will be a significant timing difference because the one based on makeFetch1() does not wait for any of the axios.get() calls to finish and does not communicate back results or completion and is subject to uncaught rejections.
The one based on makeFetch1() has no practical use as you could just do:
makeFetch1();
makeFetch1();
...
with the exact same result because makeFetch1() just returns an immediately resolved promise so passing them all to Promise.all() isn't doing anything useful.
makeFetch1() is probably just not useful since it doesn't inform the caller of completion, error or resolved value.
I want to push promise to an array. Then I want to resolve it using Promise.all(). But I'm not sure, is promise work when i push to array?
For example:
const productIds= [1,2,3,4,5,6] // example
const promises = [];
productIds.map(productId => promises.push(ProductDataAccess.updateProductStock(store,productId,incQuery)));
If I don't Promise.all(), will db process occur? Or does the db process occur when I make Promise.all().
const productIds= [1,2,3,4,5,6] // example
const promises = [];
productIds.map(productId => promises.push(ProductDataAccess.updateProductStock(store,productId,incQuery)));
Pormise.all(promises);
If I don't Promise.all(), will db process occur?
It depends. The general answer is yes. But be aware that this is not always true.
Normally, functions that return a Promise schedules an asynchronous process when you call them. So the asynchronous process will happen regardless of you waiting for them.
However, there are some functions that don't really return a promise. They return a promise-like object. And sometimes (not always) they don't start the asynchronous process unless you call .then(). And database libraries that use fluent/chainable functions do this. This is a common design pattern in database libraries:
// knex example:
let x = knex('my_table').select(['id','some_info']); // will not trigger db query
console.log(x); // knex object - not a Promise!
x = x.where('id', 0); // still no db query
console.log(x); // still not a Promise!
x = x.then(result => console.log(result)); // TRIGGERS DB QUERY!
console.log(x); // Yay! A Promise!
Lots of database libraries do this in order to implement a fluent/chaining style API that is transparent to the user. They detect the end of query construction by the .then() function being called.
Now, I don't know what database library you are using but if you are affected by this then to trigger the database query process you will need to call then either directly or indirectly:
call .then() yourself
pass the object to Promise.all() which internally calls .then()
await the result in an async function which internally calls .then()
Yes, promises run when they are created, not when they are awaited upon (be it with Promise.all() or otherwise).
You will start the execution of the promises from the moment each promise is created (regardless of the usage of Promise.all), but not wait for them.
If this block of code is in an async function, you can await the promises in this way (and also, you can simply use map, without push, to build the array of promises you need):
const productIds = [1,2,3,4,5,6];
const promises = productIds.map((productId) => ProductDataAccess.updateProductStock(store,productId,incQuery));
await Promise.all(promises);
I now understand that async functions return promises of values not values directly. But what I don't understand is what the point of doing that in the first place. As far as I understand, async functions are used to synchronize code inside them, now after we got the value from a promise using the await statement inside the async function, why should we return another promise? Why don't we just return the result directly or return void?
async functions are used to synchronize code inside them
They aren't. async functions provide syntactic sugar for promises, thus eliminating the need to use callbacks. async function acts exactly like regular function that returns a promise.
There's no way how the result can be returned synchronously from asynchronous function.
If the result should be returned from async function, it should be called and awaited inside another async function, and so on - possibly up to application entry point.
Because you can't know when the asynchronous call will be done. So it just returns a promise to let you make your rest logic with asynchronous call by making then-s chain.
Yes, async function are used to sequentialise the code inside them. They do not - cannot - stop the code execution outside of them. As you probably remember, blocking is bad. You cannot get the result from the future of course, but you don't want to stop the world just to wait for that function to finish either. And that's why when you call such a function, you get back a promise that you either await, or can schedule a callback on and do other things while it waits.
I am trying to create an array of Promises, then resolve them with Promise.all(). I am using got, which returns a promise.
My code works, but I don't fully understand how. Here it is:
const got = require('got');
const url = 'myUrl';
const params = ['param1', 'param2', 'param3'];
let promiseArray = [];
for (param of params) {
promiseArray.push(got(url + param));
}
// Inspect the promises
for (promise of promiseArray) {
console.log(JSON.stringify(promise));
// Output: promise: {"_pending":true,"_canceled":false,"_promise":{}}
}
Promise.all(promiseArray).then((results) => {
// Operate on results - works just fine
}).catch((e) => {
// Error handling logic
});
What throws me off is that the Promises are marked as "pending" when I add them into the array, which means they have already started.
I would think that they should lie inactive in promiseArray, and Promise.all(promiseArray) would both start them and resolve them.
Does this mean I am starting them twice?
You're not starting them twice. Promises start running as soon as they're created - or as soon as the JS engine finds enough resources to start them. You have no control on when they actually start.
All Promise.all() does is wait for all of them to settle (resolve or reject). Promise.all() does not interfere with nor influence the order/timing of execution of the promise itself.
Promises don't run at all. They are simply a notification system for communicating when asynchronous operations are complete.
So, as soon as you ran this:
promiseArray.push(got(url + param));
Your asynchronous operation inside of got() is already started and when it finishes, it will communicate that back through the promise.
All Promise.all() does is monitor all the promises and tell you when the first one rejects or when all of them have completed successfully. It does not "control" the async operations in any way. Instead, you start the async operations and they communicate back through the promises. You control when you started the async operations and the async operations then run themselves from then on.
If you break down your code a bit into pieces, here's what happens in each piece:
let promiseArray = [];
for (param of params) {
promiseArray.push(got(url + param));
}
This calls got() a bunch of times starting whatever async operation is in that function. got() presumably returns a promise object which is then put into your promiseArray. So, at this point, the async operations are all started already and running on their own.
// Inspect the promises
for (promise of promiseArray) {
console.log(JSON.stringify(promise));
// Output: promise: {"_pending":true,"_canceled":false,"_promise":{}}
}
This loop, just looks at all the promises to see if any of them might already be resolved, though one would not expect them to be because their underlying async operations were just started in the prior loop.
Promise.all(promiseArray).then((results) => {
// Operate on results - works just fine
}).catch((e) => {
// Error handling logic
});
Then, with Promise.all(), you're just asking to monitor the array of promises so it will tell you when either there's a rejected promise or when all of them complete successfully.
Promises "start" when they are created, i.e. the function that gives you the promise, has already launched the (often asynchronous) operations that will eventually lead into an asynchronous result. For instance, if a function returns a promise for a result of an HTTP request, it has already launched that HTTP request when returning you the promise object.
No matter what you do or not do with that promise object, that function (got) has already created a callback function which it passed on to an asynchronous API, such as a HTTP Request/Response API. In that callback function (which you do not see unless you inspect the source of got) the promise will be resolved as soon as it gets called back by that API. In the HTTP request example, the API calls that particular callback with the HTTP response, and the said callback function then resolve the promise.
Given all this, it is a bit strange to think of promises as things that "start" or "run". They are merely created in a pending state. The remaining thing is a pending callback from some API that will hopefully occur and then will change the state of the promise object, triggering then callbacks.
Please note that fetching an array of urls with Promise.all has got some possible problems:
If any of the urls fail to fetch your resolve is never called (so
one fails and your resolve function is never called.
If your array is very large you will clobber the site and your network with requests, you may want to throttle the maximum open requests and or requests made in a certain timeframe.
The first problem is easily solved, you process the failed requests and add them to the results. In the resolve handler you can decide what to do with the failed requests:
const got = require('got');
const url = 'myUrl';
const params = ['param1', 'param2', 'param3'];
const Fail = function(details){this.details = details;};
Promise.all(
params.map(
param =>
got(url + param)
.then(
x=>x,//if resolved just pass along the value
reject=>new Fail([reject,url+param])
)
)
).then((results) => {
const successes = results.filter(result=>(result && result.constructor)!==Fail),
const failedItems = results.filter(result=>(result && result.constructor)===Fail);
}).catch((e) => {
// Error handling logic
});
Point 2 is a bit more complicated, throttling can be done with this helper function and would look something like this:
... other code
const max5 = throttle(5);
Promise.all(
params.map(
param =>
max5(got)(url + param)
.then(
x=>x,//if resulved just pass along the value
reject=>new Fail([reject,url+param])
)
)
)