This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Closed last year.
In node.js I need to use a function procesMultipleCandidates () which contains Array.foreach which process insert every element into db. but the entire function should return response after completing all insertion operation
JavaScript Code
async function procesMultipleCandidates (data) {
let generatedResponse = []
await data.forEach(async (elem) => {
try {
// here candidate data is inserted into
let insertResponse = await insertionInCandidate(elem)
//and response need to be added into final response array
generatedResponse.push(insertResponse)
} catch (error) {
console.log('error'+ error);
}
})
console.log('complete all') // gets loged first
return generatedResponse // return without waiting for process of
}
And as described above last return statement not waiting for the foreach execution to complete first.
Use Array.prototype.map and Promise.all:
async function procesMultipleCandidates (data) {
let generatedResponse = []
await Promise.all(data.map(async (elem) => {
try {
// here candidate data is inserted into
let insertResponse = await insertionInCandidate(elem)
// and response need to be added into final response array
generatedResponse.push(insertResponse)
} catch (error) {
console.log('error'+ error);
}
}))
console.log('complete all') // gets loged first
return generatedResponse // return without waiting for process of
}
Or use a for/of loop if you don't want the loop run concurrently:
async function procesMultipleCandidates (data) {
let generatedResponse = []
for(let elem of data) {
try {
// here candidate data is inserted into
let insertResponse = await insertionInCandidate(elem)
// and response need to be added into final response array
generatedResponse.push(insertResponse)
} catch (error) {
console.log('error'+ error);
}
}
console.log('complete all') // gets loged first
return generatedResponse // return without waiting for process of
}
Array.prototype.forEach() tries to execute sequentially and you may not always get the expected result.
Array.prototype.map() is the widely used for creating Promises using data and then resolved using await Promise.all([])
using .map() function, all the promises are resolved in parallel and if you happen to call an API you may end up getting error 429: Too many Requests
To execute all the promises sequentially, you can use for ...of.
const array1 = ['a', 'b', 'c'];
for (const element of array1) {
console.log(element);
}
MDN Reference
As far as I'm concern, for of works better with asynchrony than forEach. Anyway, in this case, if you only care for the aggregated result, I think this could be a solution.
async function procesMultipleCandidates(data) {
const promises = data.map(insertionInCandidate);
return await Promise.all(promises);
}
Use Promise.all instead, which will resolve once all Promises in the array passed to it have resolved. The syntax will probably be simplified if you use and return a plain Promise chain rather than async/await:
const procesMultipleCandidates = data => Promise.all(
data.map(insertionInCandidate)
)
.catch(err => {
console.log('err: ' + err);
});
Note that if there's an error, procesMultipleCandidates will currently return a resolved Promise rather than rejecting - it might be a better idea to have the consumer of procesMultipleCandidates handle errors, so that it can handle them appropriately (rather than simply console.logging it).
Related
I am using Node JS and Express JS, here is my controller code:
const UserComment = require("../model/UserComment");
router.post("/get/comments", async (request, response) =>{
try{
let currentUserID = request.body.userID;
let myUserComment = await UserComment.find({userID: currentUserID});
let friendsCommentsArray = [ ...myUserComment];
let friendsComments = await axios.post(`http://localhost:5000/router/accounts/account/following/list`, {userID: currentUserID})
.then((resp) => {
resp.data.message.map((parentArrayOfArray) =>{
parentArrayOfArray.map((friendID) =>{
let friendsCommentsToLookUp = UserComment.find({userID: friendID})
friendsCommentsToLookUp.then((commentsArray) =>{
commentsArray.map((comment) =>{
if(String(comment.userID) === friendID){
friendsCommentsArray.push(comment);
}else{
console.log("no")
}
})
});
});
});
}).catch((err) =>{
console.log("err: ", err);
throw err;
});
return response.status(200).json({message: friendsPostsArray});
}catch(err){
return response.status(400).json({message: `${err}`});
}
});
The friendsCommentsArray, when I console.log it I can see the data, but when I return it, it’s empty. What is the problem, why is it empty, even though i'm pushing every comment iterated over to the friendsCommentsArray.
However, the returned friendsCommentsArray is empty. how to solve this issue ?
Thanks.
To make await Promise.all() work you need to return the promise
return axios.get(`http://localhost:5000/comments/by/post/${post._id}`)
Generally when you use await, you don't need to use .then(). Your problem is that your inner .map() is using friendsCommentsToLookUp.then(), but nothing is waiting for these promises to resolve before you move on in your code. One might think that you can await the friendsCommentsToLookUp promise, but this won't work, as the calls to the map callback are not awaited.
Removing the .then()'s makes this easier to work with:
const resp = await axios.post(`http://localhost:5000/router/accounts/account/following/list`, {userID: currentUserID});
const message = resp.data.message;
for(const parentArrayOfArray of message) {
for(const friendID of parentArrayOfArray) {
const commentsArray = await UserComment.find({userID: friendID});
for(const comment of commentsArray) {
if(String(comment.userID) === friendID){
friendsCommentsArray.push(comment);
}
}
}
}
Above the for..of allows us to pause moving to the next iteration of the for loop until the Promises within the current iteration of the for loop have resolved. ie: it's sequential (note: if you tried to do this with .forEach() or .map(), your code would proceed directly to the portion after the loop before your Promises have resolved). Although, what you're after doesn't need to be sequential. We can create an array of Promises that we pass to Promise.all() which we can wait to resolve in parallel. Below I've shown a different approach of using .flatMap() to create an array of Promises that we can await in parallel with Promise.all():
const resp = await axios.post(`http://localhost:5000/router/accounts/account/following/list`, {userID: currentUserID});
const message = resp.data.message;
const promises = message.flatMap(parentArr => parentArr.map(async friendID => {
const commentsArray = await UserComment.find({userID: friendID});
return commentsArray.filter(comment => String(comment.userID) === friendID);
}));
const nestedComments = await Promise.all(promises);
const friendsCommentsArray = [...myUserComment, ...nestedComments.flat()];
instead of push try concatenation array and let me know if its work.
friendsCommentsArray = [...friendsCommentsArray , {...comment}];
// insted of
friendsCommentsArray.push(comment);
also try to use forEach instead of map() while you don't want to return a new array from your map statement.
The map method is very similar to the forEach method—it allows you to execute a function for each element of an array. But the difference is that the map method creates a new array using the return values of this function. map creates a new array by applying the callback function on each element of the source array. Since map doesn't change the source array, we can say that it’s an immutable method.
Hi I need to execute promises one after the other how do I achieve this using promise.all any help would be awesome. Below is the sample of my code I am currently using but it executes parallel so the search will not work properly
public testData: any = (req, res) => {
// This method is called first via API and then promise is triggerd
var body = req.body;
// set up data eg 2 is repeated twice so insert 2, 5 only once into DB
// Assuming we cant control the data and also maybe 3 maybe inside the DB
let arrayOfData = [1,2,3,2,4,5,5];
const promises = arrayOfData.map(this.searchAndInsert.bind(this));
Promise.all(promises)
.then((results) => {
// we only get here if ALL promises fulfill
console.log('Success', results);
res.status(200).json({ "status": 1, "message": "Success data" });
})
.catch((err) => {
// Will catch failure of first failed promise
console.log('Failed:', err);
res.status(200).json({ "status": 0, "message": "Failed data" });
});
}
public searchAndInsert: any = (data) => {
// There are database operations happening here like searching for other
// entries in the JSON and inserting to DB
console.log('Searching and updating', data);
return new Promise((resolve, reject) => {
// This is not an other function its just written her to make code readable
if(dataExistsInDB(data) == true){
resolve(data);
} else {
// This is not an other function its just written her to make code readable
insertIntoDB(data).then() => resolve(data);
}
});
}
I looked up in google and saw the reduce will help I would appreciate any help on how to convert this to reduce or any method you suggest (Concurrency in .map did not work)
the Promises unfortunatelly does not allow any control of their flow. It means -> once you create new Promise, it will be doing its asynchronous parts as they like.
The Promise.all does not change it, its only purpose is that it checks all promises that you put into it and it is resolved once all of them are finished (or one of them fail).
To be able to create and control asynchronous flow, the easiest way is to wrap the creation of Promise into function and create some kind of factory method. Then instead of creating all promises upfront, you just create only one promise when you need it, wait until it is resolved and after it continue in same behaviour.
async function doAllSequentually(fnPromiseArr) {
for (let i=0; i < fnPromiseArr.length; i++) {
const val = await fnPromiseArr[i]();
console.log(val);
}
}
function createFnPromise(val) {
return () => new Promise(resolve => resolve(val));
}
const arr = [];
for (let j=0; j < 10; j++) {
arr.push(createFnPromise(Math.random()));
}
doAllSequentually(arr).then(() => console.log('finished'));
PS: It is also possible without async/await using standard promise-chains, but it requires to be implemented with recursion.
If anyone else cares about ESLint complaining about the use of "for" and the "no await in loop" here is a typescript ESLint friendly version of the above answer:
async function runPromisesSequentially<T>(promises: Array<Promise<T>>):Promise<Array<T>> {
if (promises.length === 0) return [];
const [firstElement, ...rest] = promises;
return [await firstElement, ...(await runPromisesSequentially(rest))];
}
You can then just replace Promise.all by runPromisesSequentially.
#lmX2015's answer is close but it's taking in promises that have already started executing.
A slight modification fixes it
export async function runPromisesSequentially<T>(functions: (() => Promise<T>)[]): Promise<T[]> {
if (functions.length === 0) {
return [];
}
const [first, ...rest] = functions;
return [await first(), ...(await runPromisesSequentially(rest))];
}
This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Closed 4 years ago.
when I create my api in nodejs and trying to pushing mongoose return count to new created array, it's not wait for the forEach and execute json.res() and giving null response. when I use
setTimeout() then it's giving proper result.
let newcategories = [];
let service = 0;
const categories = await Category.find({}, '_id name');
categories.forEach(async (category) => {
service = await Service.count({category: category});
newcategories.push({ count:service });
console.log('newcategories is -- ', newcategories);
}); /* while executing this forEach it's not wait and execute res.json..*/
console.log('result --- ',result);
console.log('out newcategories is -- ', newcategories);
res.json({status: 200, data: newcategories});
You need to use map instead of forEach, to collect the awaits and wait for them to complete. Edit: Or you can use for..of which is pretty neat (thanks other ppl)!
const categories = ['a', 'b', 'c'];
function getNextCategory(oldCategory) {
return new Promise((resolve) => {
setTimeout(() => {
resolve(String.fromCharCode(oldCategory.charCodeAt(0)+1));
}, 1000);
});
}
async function blah() {
const categoryPromises = categories.map(getNextCategory);
const nextCategories = await Promise.all(categoryPromises);
console.log(nextCategories);
}
blah();
async function blah2() {
const nextCategories = [];
for (const category of categories) {
nextCategories.push(await getNextCategory(category));
};
console.log(nextCategories);
}
blah2();
So the problem you have is that async marked functions will return a promise per default, but that the Array.prototype.forEach method doesn't care about the result type of your callback function, it is just executing an action.
Inside your async function, it will properly await your responses and fill up your new categories, but the forEach loop on categories will be long gone.
You could either choose to convert your statements into a for .. of loop, or you could use map and then await Promise.all( mapped )
The for..of loop would be like this
for (let category of categories) {
service = await Service.count({category: category});
newcategories.push({ count:service });
console.log('newcategories is -- ', newcategories);
}
the map version would look like this
await Promise.all( categories.map(async (category) => {
service = await Service.count({category: category});
newcategories.push({ count:service });
console.log('newcategories is -- ', newcategories);
}));
The second version simply works because Promise.all will only resolve once all promises have completed, and the map will return a potentially unresolved promise for each category
I am using Firebase Cloud Firestore, however, I think this may be more of a JavaScript asynchronous vs synchronous promise return issue.
I am doing a query to get IDs from one collection, then I am looping over the results of that query to lookup individual records from another collection based on that ID.
Then I want to store each found record into an array and then return the entire array.
results.length is always 0 because return results fires before the forEach completes. If I print results.length from inside the forEach it has data.
How can I wait until the forEach is done before returning from the outer promise and the outer function itself?
getFacultyFavoritesFirebase() {
var dbRef = db.collection("users").doc(global.user_id).collection("favorites");
var dbQuery = dbRef.where("type", "==", "faculty");
var dbPromise = dbQuery.get();
var results = [];
return dbPromise.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
var docRef = db.collection("faculty").doc(doc.id);
docRef.get().then(function(doc) {
if (doc.exists) {
results.push(doc);
}
})
});
console.log(results.length);
return results;
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
}
The trick here is to populate results with promises rather than the result. You can then call Promise.all() on that array of promises and get the results you want. Of course, you can't check if doc.exists before pushing the promise so you will need to deal with that once Promise.all() resolves. For example:
function getFacultyFavoritesFirebase() {
var dbRef = db.collection("users").doc(global.user_id).collection("favorites");
var dbQuery = dbRef.where("type", "==", "faculty");
var dbPromise = dbQuery.get();
// return the main promise
return dbPromise.then(function(querySnapshot) {
var results = [];
querySnapshot.forEach(function(doc) {
var docRef = db.collection("faculty").doc(doc.id);
// push promise from get into results
results.push(docRef.get())
});
// dbPromise.then() resolves to a single promise that resolves
// once all results have resolved
return Promise.all(results)
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
}
getFacultyFavoritesFirebase
.then(results => {
// use results array here and check for .exists
}
If you have multiple items of work to perform at the same time that come from a loop, you can collect all the promises from all the items of work, and wait for them all to finish with Promise.all(). The general form of a possible solution looks like this:
const promises = [] // collect all promises here
items.forEach(item => {
const promise = item.doWork()
promises.push(promise)
})
Promise.all(promises).then(results => {
// continue processing here
// results[0] is the result of the first promise in the promises array
})
You can adapt this to something that suits your own specific form.
Use for of instead of forEach. Like this:
for (const item of array) {
//do something
}
console.log("finished");
"finished" will be logged after finishing the loop.
Well I know, the thread is old, but the problem is still the same. And because I did run into the same issue and non of the answers did work for me, I want share my solution.
I think it will help someone out there. And maybe it will help me, if I run into the same problem again. ;-)
So the solution is super easy. firebase implements "map" but not direct on snaposhot, but on snapshot.docs.map.
In combination with Promieses.all it works just fine.
const promise = snapshot.docs.map(async (tenant) => {
return CheckTenant(tenant.id).catch(error =>
reject(error),
);
});
Promise.all(promise).then(result => {
// do somothing with the result});
I am trying to use the new async/await in conjunction with mapping over an array of values.
However I am a bit confused, which should have the await keyword infront of it, in this specific scenario.
There isnt much information regarding this topic yet.
I am trying to simply map() over and array of values, which are used in an Axios get() request, which then returns a then() value, which gets added to the array returned by the map() function.
This is my code right now:
async function get_prices (arrayOfDates, crypto, FIAT) {
// I tried this way (return value can be found below)
let arry_of_prices = arrayOfDates.map(async function(date){
let a = await fetch_data(date, crypto, FIAT)
return a
});
// And this way (return value can be found below)
let arry_of_prices = await arrayOfDates.map(date => fetch_data(date, crypto, FIAT));
}
const fetch_data = (timestamp, crypto, FIAT) => {
axios.get(`https://min-api.cryptocompare.com/data/pricehistorical?fsym=${crypto}&tsyms=${FIAT}&ts=${timestamp}`)
.then(function (response) {
return Object.values(response.data).map(key => Object.values(key))[0][0]
})
.catch(function (error) {
console.log(error);
});
}
let arr = [1468965600, 1469052000, 1469138400,1469484000]
get_prices(arr, "BTC", "USD").then(arr => console.log(arr))
The first try returned 4 resolved promises, but the values were undefined.
Whereas the second return an array with 4 undefined.
Does anyone have an idea of how this could be structured, so that the map() waits until the function is resolved and then adds the value to the array?
map doesn't know anything about promises, so when you use an async callback with map, you'll get an array of promises, not values. If you then want a promise that will resolve when all of those promises have resolved, you can use Promise.all.
However, in your case, you don't need an async function, since you're using fetch_data in the callback. You need to modify fetch_data so it returns the promise from axios.get (and remove the catch handler):
const fetch_data = (timestamp, crypto, FIAT) => {
return axios.get(`https://min-api.cryptocompare.com/data/pricehistorical?fsym=${crypto}&tsyms=${FIAT}&ts=${timestamp}`)
.then(function (response) {
return Object.values(response.data).map(key => Object.values(key))[0][0]
});
}
(This is one of the rules of promises: If you're returning the promise, you don't handle rejections from it [unless you want to convert them into different rejections, or if you can usefully convert them to resolutions instead]. If you don't return the promise, you do handle rejections.)
Now you already have a promise, so you use that in the map (no need for async):
async function get_prices (arrayOfDates, crypto, FIAT) {
let arr_of_price_promises = arrayOfDates.map(function(date){
let a = fetch_data(date, crypto, FIAT)
return a
});
return Promise.all(arr_of_price_promises);
}
Notice I removed the await from in front of fetch_data. We return fetch_data's promise directly, then wait for them all.
Or with an arrow function:
async function get_prices (arrayOfDates, crypto, FIAT) {
let arr_of_price_promises = arrayOfDates.map(date => fetch_data(date, crypto, FIAT));
return Promise.all(arr_of_price_promises);
}