I am using map() over a long array and using fetch for each value. It will take many seconds to complete. I would like to know when the final push is complete and use the data in the array.
I tried Promise, Promise.all, Async/Await, but may have missed something obvious. I created this sample code to more simply illustrate the problem.
const arr = new Array(100).fill("todos")
const arrPush = []
const result = arr.map(data => {
fetch(`https://jsonplaceholder.typicode.com/${data}`)
.then(res => res.json())
.then(res => arrPush.push(res))
})
Promise.all(result).then(() => console.log(arrPush))
When the final value is added to array, do something. In this case console.log the complete array.
The function that you're passing to map doesn't have a return statement, so result is an array of undefineds. As a result, there's nothing for Promise.all to wait for.
Additionally, there's no need to manually push into an array. Once we add in the return statement, you'll have an array of promises, and Promise.all will resolve to an array with all the stuff you're currently trying to push.
So try this:
const arr = new Array(100).fill("todos")
const promises = arr.map(data => {
return fetch(`https://jsonplaceholder.typicode.com/${data}`)
.then(res => res.json());
});
Promise.all(promises)
.then(arrayOfResults => console.log(arrayOfResults));
Related
I am using Node JS and Express JS, here is my controller code:
const UserComment = require("../model/UserComment");
router.post("/get/comments", async (request, response) =>{
try{
let currentUserID = request.body.userID;
let myUserComment = await UserComment.find({userID: currentUserID});
let friendsCommentsArray = [ ...myUserComment];
let friendsComments = await axios.post(`http://localhost:5000/router/accounts/account/following/list`, {userID: currentUserID})
.then((resp) => {
resp.data.message.map((parentArrayOfArray) =>{
parentArrayOfArray.map((friendID) =>{
let friendsCommentsToLookUp = UserComment.find({userID: friendID})
friendsCommentsToLookUp.then((commentsArray) =>{
commentsArray.map((comment) =>{
if(String(comment.userID) === friendID){
friendsCommentsArray.push(comment);
}else{
console.log("no")
}
})
});
});
});
}).catch((err) =>{
console.log("err: ", err);
throw err;
});
return response.status(200).json({message: friendsPostsArray});
}catch(err){
return response.status(400).json({message: `${err}`});
}
});
The friendsCommentsArray, when I console.log it I can see the data, but when I return it, it’s empty. What is the problem, why is it empty, even though i'm pushing every comment iterated over to the friendsCommentsArray.
However, the returned friendsCommentsArray is empty. how to solve this issue ?
Thanks.
To make await Promise.all() work you need to return the promise
return axios.get(`http://localhost:5000/comments/by/post/${post._id}`)
Generally when you use await, you don't need to use .then(). Your problem is that your inner .map() is using friendsCommentsToLookUp.then(), but nothing is waiting for these promises to resolve before you move on in your code. One might think that you can await the friendsCommentsToLookUp promise, but this won't work, as the calls to the map callback are not awaited.
Removing the .then()'s makes this easier to work with:
const resp = await axios.post(`http://localhost:5000/router/accounts/account/following/list`, {userID: currentUserID});
const message = resp.data.message;
for(const parentArrayOfArray of message) {
for(const friendID of parentArrayOfArray) {
const commentsArray = await UserComment.find({userID: friendID});
for(const comment of commentsArray) {
if(String(comment.userID) === friendID){
friendsCommentsArray.push(comment);
}
}
}
}
Above the for..of allows us to pause moving to the next iteration of the for loop until the Promises within the current iteration of the for loop have resolved. ie: it's sequential (note: if you tried to do this with .forEach() or .map(), your code would proceed directly to the portion after the loop before your Promises have resolved). Although, what you're after doesn't need to be sequential. We can create an array of Promises that we pass to Promise.all() which we can wait to resolve in parallel. Below I've shown a different approach of using .flatMap() to create an array of Promises that we can await in parallel with Promise.all():
const resp = await axios.post(`http://localhost:5000/router/accounts/account/following/list`, {userID: currentUserID});
const message = resp.data.message;
const promises = message.flatMap(parentArr => parentArr.map(async friendID => {
const commentsArray = await UserComment.find({userID: friendID});
return commentsArray.filter(comment => String(comment.userID) === friendID);
}));
const nestedComments = await Promise.all(promises);
const friendsCommentsArray = [...myUserComment, ...nestedComments.flat()];
instead of push try concatenation array and let me know if its work.
friendsCommentsArray = [...friendsCommentsArray , {...comment}];
// insted of
friendsCommentsArray.push(comment);
also try to use forEach instead of map() while you don't want to return a new array from your map statement.
The map method is very similar to the forEach method—it allows you to execute a function for each element of an array. But the difference is that the map method creates a new array using the return values of this function. map creates a new array by applying the callback function on each element of the source array. Since map doesn't change the source array, we can say that it’s an immutable method.
So I have an asynchronous function that I need to call a number of times. Naturally, I would like to use Promise.all
const promises = [];
ids.forEach((id) => {
stockoutPromises.push(
asyncFunc(id),
});
});
results[] = Promise.all(promises);
Okay no problem there but how do I know which id goes with which result? Sure I could iterate through both arrays again, but is there another way to do this? Is there a way that I can create a promise that when resolved gives me an object that contains both id and result?
Thanks!
Chain a .then onto the call of asyncFunc so that the resulting item is not just of the asyncFunc result, but of it and the ID in an object:
ids.forEach((id) => {
stockoutPromises.push(
asyncFunc(id).then(result => ({ id, result }))
);
});
But it'd be better to use .map instead of .push in a loop:
const results = await Promise.all(
ids.map(id =>
asyncFunc(id).then(result => ({ id, result }))
)
);
const firstItem = results[0];
console.log(firstItem.result, firstItem.id);
The result of Promise.all will be in the order that they were in originally (in the array of promises). So you can easily reassociate just using the index:
// zip takes an array of keys, and an array of values, and creates an object:
const zip = (a, b) => Object.fromEntries(a.map((k, i) => [k, b[i]]));
// first id will resolve last, second will resolve next, etc.
const ids = [0, 1, 2, 3, 4];
const promises = ids.map((i) => new Promise((resolve) => {
setTimeout(() => resolve(i), 700 - (100 * i));
}));
// but we still get the correct item in each spot, because of how
// Promise.all works:
(async () => {
console.log(zip(ids, await Promise.all(promises)));
})()
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all:
Returned values will be in order of the Promises passed, regardless of completion order.
Instead of using forEach, use map:
const promises = ids.map((id) => {
return asyncFunc(id)
});
const results = await Promise.all(promises);
map will return a new array of objects based on what the supplied function returns. In your case, you are calling an asyncFunc, which, I assume, returns a promise itself. So you can just return the results of map directlty without pushing to a new promises array.
Also, make sure to "await" the call to Promise.all;
You can check it out here in this jsfiddle that simply has the asyncFunc return a promise that doubles the id in the ids array.
Here I have a function that takes an array of string that contains the user names of github accounts. And this function is going to return an array of user data after resolving. There should be one fetch request per user. and requests shouldn’t wait for each other. So that the data arrives as soon as possible. If there’s no such user, the function should return null in the resulting array.
An example for the input would be ["iliakan", "remy", "no.such.users"], and the expected returned promise after resolving would give us [null, Object, Object], Object being the data that contained info about a user.
Here is my attempt to solve this question.
function getUsers(names) {
return new Promise(resolve => {
const array = [];
const url = "https://api.github.com/users/";
const requests = names.map(name => {
const endpoint = `${url}${name}`;
return fetch(endpoint);
});
Promise.all(requests).then(reponses => {
reponses.forEach(response => {
if (response.status === 200) {
response.json().then(data => {
array.push(data);
});
} else {
array.push(null);
}
});
resolve(array);
});
});
}
It does work, i.e. returning an array [null, Object, Object]. And I thought it fulfilled the requirements I stated above. However, after looking at it closely, I felt like I couldn't fully make sense of it.
My question is, look at where we resolve this array, it resolved immediately after the forEach loop. One thing I don't understand is, why does it contain all three items when some of the items are pushed into it asynchronously after the json() is finished. what I mean is, in the case where response.status === 200, the array is pushed with the data resolved from json(), and I would assume this json() operation should take some time. Since we didn't resolve the array after json() operation is finished, how come we still ended up with all data resolved from json()?
Promise.all(requests).then(reponses => {
reponses.forEach(response => {
if (response.status === 200) {
response.json().then(data => {
array.push(data); <--- this should take some time
});
} else {
array.push(null);
}
});
resolve(array); <--- resolve the array immediately after the `forEach` loop
});
});
It looks to me like the array we get should only have one null in it since at the time it is revolved, the .json() should not be finished
You're right, the result is pushed later into the array.
Try to execute this:
const test = await getUsers(['Guerric-P']);
console.log(test.length);
You'll notice it displays 0. Before the result is pushed into the array, its length is 0. You probably think it works because you click on the array in the console, after the result has arrived.
You should do something like this:
function getUsers(names) {
const array = [];
const url = "https://api.github.com/users/";
const requests = names.map(name => {
const endpoint = `${url}${name}`;
return fetch(endpoint);
});
return Promise.all(requests).then(responses => Promise.all(responses.map(x => x.status === 200 ? x.json() : null)));
};
You should avoid using the Promise constructor directly. Here, we don't need to use it at all.
const url = "https://api.github.com/users/";
const getUsers = names =>
Promise.all(names.map(name =>
fetch(url + name).then(response =>
response.status === 200 ? response.json() : null)));
getUsers(["iliakan", "remy", "no.such.users"]).then(console.log);
The Promise constructor should only be used when you're creating new kinds of asynchronous tasks. In this case, you don't need to use the Promise constructor because fetch already returns a promise.
You also don't need to maintain an array and push to it because Promise.all resolves to an array. Finally, you don't need to map over the result of Promise.all. You can transform the promises returned by fetch.
The thing is that because json() operation is really quick, especially if response data is small in size it just has the time to execute. Second of all as objects in JavaScript passed by reference and not by value and Array is a object in JavaScript, independently of execution time it'll still push that data to the array even after it was resolved.
I have a code to:
Read last three data from Firebase
Iterate each retrieved data
Push a Promise-returning function expression to an array of Promise to be processed sequentially later
Process said array sequentially
Code:
firebase.database().ref('someRef').limitToLast(3).on('value', snapshot => {
let promiseArray = [];
snapshot.forEach(e => {
promiseArray.push(() => {
firebase.storage().ref(e.key).getDownloadURL().then(url => {
//Do something with URL
//In this case, I print out the url to see the order of URL retrieved
//Unfortunately, the order was incorrect
return 'Resolved, please continue'; //Return something to resolve my Promise
});
});
});
let result = Promise.resolve([]);
promiseArray.forEach(promise => {
result = result.then(promise);
});
});
I think that it should already be correct. However, the result I want to get is wrong. What did I miss?
EDIT
I seem to have missed a point. In my Promise array, I want the first function to resolve the Promise it returns first before continuing to the second function.
You should use reduce. A very good example you will find here: https://decembersoft.com/posts/promises-in-serial-with-array-reduce/
forEach is a synchronous method. You can use map to create the array of promises and then use promise.all.
firebase.database().ref('someRef').limitToLast(3).on('value', snapshot => {
let promiseArray = [];
const promiseArray = snapshot.map(e => firebase.storage().ref(e.key).getDownloadURL());
Promise.all(promiseArray).then((resultArr) => {
// Do anything with your result array
});
}
For sequential execution of promises you can use async await.
firebase.database().ref('someRef').limitToLast(3).on('value', async (snapshot) => {
let promiseArray = [];
const promiseArray = snapshot.map(e => firebase.storage().ref(e.key).getDownloadURL());
let result;
for(let i = 0; i < promiseArray.length; i++) {
result = await promiseArray[i];
}
});
I figured it out: apparently, I forgot to my function is not returning a Promise. Because of that, when I'm chaining the thens, it's not waiting for my Promise to resolve first as it wasn't even returned in the first place. I'm basically returning a void, thus the functions continue without waiting for the previous Promise to resolve. Simply adding return fixes the problem:
firebase.database().ref('someRef').limitToLast(3).on('value', snapshot => {
let promiseArray = [];
snapshot.forEach(e => {
promiseArray.push(() => {
return firebase.storage().ref(e.key).getDownloadURL().then(url => { //Add return here
//Do something with URL
//In this case, I print out the url to see the order of URL retrieved
//Unfortunately, the order was incorrect
return 'Resolved, please continue'; //Return something to resolve my Promise
});
});
});
let result = Promise.resolve([]);
promiseArray.forEach(promise => {
result = result.then(promise);
});
});
getCategory().then((res) => {
let itemList = res;
return new Promise((resolve, reject) => {
for(let i=0; i < res.length; i++ ) {
getItems(res[i].url).then((itempage) => {
const $ = itempage;
const pagination = $('.pagination .page').toArray().length;
itemList[i].total_page = pagination;
});
}
resolve(itemList);
});
}).then((res) => {
console.log(res);
});
Above is my code. First of all, I assign res to a new variable called itemList which is an object array.
Then I do some functions with promise and try to add new data to itemList.
However, when the console.log(res) still prints out original itemList.
I know this may caused by promise function but how to fix it?
Try this instead
getCategory().then(itemList => {
return Promise.all([itemList, Promise.all(itemList.map(x => getItems(x.url)))]);
}).then(([itemList, items]) => {
items.forEach(($, i) => {
itemList[i].total_page = $('.pagination .page').toArray().length;
});
return itemList;
}).then(console.log);
I think that will get you there (i.e. with the side-effected itemList). Note that this will fail if any of your getItems calls fails.
This is actually a common problem with Promises: you want to use the results asynchronously (as if you had a choice) but combine that with the original results. So here I've used Promise.all to resolve an array of Promises of the getItems calls which turns that into a single Promise that awaits all of the results. Then I put that in an array with the itemList and then call Promise.all on that and return that Promise. Then next .then in the chain gets an array with two arrays inside: the original itemList and the array of the now-resolved calls to getItems. At this point the forEach executes the side effects, and we just return the now updated itemList.