I am trying to loop and get different documents from firestore. The 'document ids' are provided by an array named 'cart' as you can see in the code below.
The programming logic which I have tried goes like this the while loop in every iteration gets document from firestore and in first 'then' section it saves the data which it just have got and in second 'then' it increments the 'i' and does the next cycle of loop.
The problem is while loop doesn't wait for that get request to finish. It just keeps looping and crashes.
The thing is even if I somehow manage to do the loop part correct. How would I manage the overall execution flow of program so that only after completing the loop part further code gets executed since the code below uses the cart array which loop part updates.
let i = 0
while (i < cart.length) {
let element = cart[i]
db.collection(`products`).doc(element.productID).get().then((doc1) => {
element.mrp = doc1.data().mrp
element.ourPrice = doc1.data().ourPrice
return console.log('added price details')
}).then(() => {
i++;
return console.log(i)
}).catch((error) => {
// Re-throwing the error as an HttpsError so that the client gets the error details.
throw new functions.https.HttpsError('unknown', error.message, error);
});
}
return db.collection(`Users`).doc(`${uid}`).update({
orderHistory: admin.firestore.FieldValue.arrayUnion({
cart,
status: 'Placed',
orderPlacedTimestamp: timestamp,
outForDeliveryTimestamp: '',
deliveredTimestamp: ''
})
}).then(() => {
console.log("Order Placed Successfully");
})
Your question is not about firebase, you're asking about looping asynchronously. You can see some promises examples here, and async/await here
You can use reduce on the promises.
Note that all the promises are being created at the same time, but the call to the server is done one after the other.
cart.reduce(
(promise, element) =>
promise.then(() => {
return db.collection(`products`)
.doc(element.productID)
.get()
.then(doc1 => {
element.mrp = doc1.data().mrp;
element.ourPrice = doc1.data().ourPrice;
});
}),
Promise.resolve()
);
If you can, use async/await instead. Here all the promises are being created one after the other.
async function fetchCart() {
for (const element of cart) {
const doc1 = await db.collection(`products`).doc(element.productID);
element.mrp = doc1.data().mrp;
element.ourPrice = doc1.data().ourPrice;
console.log('added price details');
}
}
Each call to Cloud Firestore happens asynchronously. So your while loop fires off multiple such requests, but it doesn't wait for them to complete.
If you have code that needs all the results, you will need to uses Promises to ensure the flow. You're already using the promise in the while loop to get doc1.data().mrp. If cart is an array, you can do the following to gather all promises of when the data is loaded:
var promises = cart.map(function(element) {
return db.collection(`products`).doc(element.productID).get().then((doc1) => {
return doc1.data();
});
});
Now you can wait for all data with:
Promise.all(promises).then(function(datas) {
datas.forEach(function(data) {
console.log(data.mrp, data.ourPrice);
});
});
If you're on a modern environment, you can use async/await to abstract away the then:
datas = await Promise.all(promises);
datas.forEach(function(data) {
console.log(data.mrp, data.ourPrice);
});
Related
I have this piece of code that works. However, I am not sure why and I feel like it might behave inconsistently.
await Listing.find({}, (err, listings) => {
if (err) {
console.log(err);
}
listings.forEach(async (listing) => {
//console.log(listing);
let championsUpdate = {};
for (let key in listing["champions"]) {
championsUpdate[key] = rankingDB[key];
}
await Listing.updateOne(
{ _id: listing._id },
{ $set: { champions: championsUpdate } }
);
});
});
Pretty much I am finding all the listings that I need to update, and then for each listing I am updating one of the properties based off the data I retrieved earlier.
So far it's been behaving appropriately but I remember being told to avoid using async await in a forEach loop because it does not behave as we expect. But I can't figure out why this is working and if I should avoid the forEach and use a forOf. I am also worried about having nested async awaits.
Does anyone know if something like this is ok? For more context on my application
Because the callback in the forEach loop is async, things that follow the call to forEach may execute before forEach finishes, and it will not wait for each iteration to finish before continuing.
For example the await in front of the updateOne call is actually pointless, since the outer async function isn't awaited on, which shows that it is probably not behaving the way you intend it to.
The reason it is recommended that you not use async inside a forEach is that it is almost never behaving the way you intend it to, or you don't actually need it, or you may forget that you called it that way later and unintentionally cause a race condition later (ie: it makes the execution order hard to reason about and virtually unpredictable). It has valid use if you actually do not care about the results of the call, when they are called, and when they resolve.
Below is a demo showing how it can cause unexpected results. sleep with random time to make each .updateOne call resolve at random times. Notice that call to console.log(`Will execute before...`) executes before the forEach iterations.
async function runit() {
await Listing.find({}, (err, listings) => {
if (err) {
console.log(err);
}
listings.forEach(async (listing) => {
//console.log(listing);
let championsUpdate = {};
for (let key in listing["champions"]) {
championsUpdate[key] = rankingDB[key];
}
await Listing.updateOne(
{ _id: listing._id },
{ $set: { champions: championsUpdate } }
);
});
});
console.log(`Will execute before the updateOne calls in forEach resolve`)
}
runit()
<script>
// mock data with sequential _id from 0...9
listings = Array(10).fill().map((_,_id)=>({_id, champions: {1:1,2:2}}))
rankingDB = Array(10).fill({1:1.1,2:2.2})
// Promise that resolves in ms milliseconds
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
// mock Listing
Listing = {
find(x, fn) {
console.log('find')
return new Promise(res=>
res(fn(undefined,listings)))
},
// updateOne resolves after random time < 1000 ms
async updateOne({_id}) {
await sleep(Math.random()*1000)
console.log('updateOne with listing _id=',_id)
}
}
</script>
I am trying to seed the following data to my MongoDB server:
const userRole = {
role: 'user',
permissions: ['readPost', 'commentPost', 'votePost']
}
const authorRole = {
role: 'author',
permissions: ['readPost', 'createPost', 'editPostSelf', 'commentPost',
'votePost']
}
const adminRole = {
role: 'admin',
permissions: ['readPost', 'createPost', 'editPost', 'commentPost',
'votePost', 'approvePost', 'approveAccount']
}
const data = [
{
model: 'roles',
documents: [
userRole, authorRole, adminRole
]
}
]
When I try to iterate through this object / array, and to insert this data into the database, I end up with three copies of 'adminRole', instead of the three individual roles. I feel very foolish for being unable to figure out why this is happening.
My code to actually iterate through the object and seed it is the following, and I know it's actually getting every value, since I've done the console.log testing and can get all the data properly:
for (i in data) {
m = data[i]
const Model = mongoose.model(m.model)
for (j in m.documents) {
var obj = m.documents[j]
Model.findOne({'role':obj.role}, (error, result) => {
if (error) console.error('An error occurred.')
else if (!result) {
Model.create(obj, (error) => {
if (error) console.error('Error seeding. ' + error)
console.log('Data has been seeded: ' + obj)
})
}
})
}
}
Update:
Here is the solution I came up with after reading everyone's responses. Two private functions generate Promise objects for both checking if the data exists, and inserting the data, and then all Promises are fulfilled with Promise.all.
// Stores all promises to be resolved
var deletionPromises = []
var insertionPromises = []
// Fetch the model via its name string from mongoose
const Model = mongoose.model(data.model)
// For each object in the 'documents' field of the main object
data.documents.forEach((item) => {
deletionPromises.push(promiseDeletion(Model, item))
insertionPromises.push(promiseInsertion(Model, item))
})
console.log('Promises have been pushed.')
// We need to fulfil the deletion promises before the insertion promises.
Promise.all(deletionPromises).then(()=> {
return Promise.all(insertionPromises).catch(()=>{})
}).catch(()=>{})
I won't include both promiseDeletion and promiseInsertion as they're functionally the same.
const promiseDeletion = function (model, item) {
console.log('Promise Deletion ' + item.role)
return new Promise((resolve, reject) => {
model.findOneAndDelete(item, (error) => {
if (error) reject()
else resolve()
})
})
}
Update 2: You should ignore my most recent update. I've modified the result I posted a bit, but even then, half of the time the roles are deleted and not inserted. It's very random as to when it will actually insert the roles into the server. I'm very confused and frustrated at this point.
You ran into a very common problem when using Javascript: You shouldn't define (async) functions in a regular for (-in) loop. What happens, is that while you loop through the three values the first async find is being called. Since your code is async, nodejs does not wait for it to finish, before it continues to the next loop iteration and counts up to the third value, here the admin rule.
Now, since you defined your functions in the loop, when the first async call is over, the for-loop already looped to the last value, which is why admin is being inserted three times.
To avoid this, you can just move the async functions out of the loop to force a call by value rather than reference. Still, this can bring up a lot of other problems, so I'd recommend you to rather have a look at promises and how to chain them (e.g. Put all mongoose promises in an array and the await them using Promise.all) or use the more modern async/await syntax together with the for-of loop that allows for both easy readability as well as sequential async command instructions.
Check this very similar question: Calling an asynchronous function within a for loop in JavaScript
Note: for-of is being discussed as to performance heavy, so check if this applies to your use-case or not.
When using async functions in loops could cause some problems.
You should change the way you work with findOne to make it synchronous function
First you need to set your function to async, and then use the findOne like so:
async function myFucntion() {
let res = await Model.findOne({'role':obj.role}).exec();//Exec will fire the function and give back a promise which the await can handle.
//do what you need to do here with the result..
}
I'm using firebase cloud functions in which I define a variable commentIdSpecific. When I log it inside the function: -- console.log(comm id ${commentIdSpecific}); -- it prints its value. When I try to print it here: -- console.log(test of variables inisde of post: ${usernameWhoOwnsThePost}, uwotpi: ${commentIdSpecific}) -- it returns undefined. I've looked at three websites talking about global vars and it doesn't seem any different from what I have here.
How do I go about getting the value in the second print statement to be in the first? Thanks in advance.
var commentIdSpecific;
db.ref(`/users/${usernameWhoOwnsThePost}/posts/${usernameWhoOwnsThePostID}/comments`).once('value').then(snap => {
commentIdSpecific = snap.val();
let ids = [];
for (var id in snap.val()) {
ids.push(id);
}
let lastValueId = ids[ids.length - 1]
console.log(`last id value ${lastValueId}. UserPost: ${usernameWhoOwnsThePost}. user owner post id: ${usernameWhoOwnsThePostID}...`);
commentIdSpecific = lastValueId;
console.log(`comm id ${commentIdSpecific}`);
return commentIdSpecific;
}).catch(err => {
console.log(err);
});
var commentPoster;
db.ref(`/users/${usernameWhoOwnsThePost}/posts/${usernameWhoOwnsThePostID}/comments/${commentIdSpecific}/comment`).once('value').then(snap => {
commentPoster = snap.val();
console.log(`commentPoster: ${snap.val()}`);
console.log(`test of variables inisde of post: ${usernameWhoOwnsThePost}, uwotpi: ${commentIdSpecific}`)
return commentPoster
}).catch(err => {
console.log(err);
});
once() is asynchronous and returns immediately with a promise that indicates when the async work is complete. Likewise, then() returns immediately with a promise. The callback you pass to then() is executed some unknown amount of time later, whenever the results of the query are finished. Until that happens, your code keeps executing at the next line, which means commentIdSpecific will be undefined when it's first accessed.
You need to use a promise chain to make sure the work that depends on the results of async work is only accessed after it becomes available.
You may want to watch the videos on JavaScript promises on this page in order to better learn how they're used in Cloud Functions. It's absolutely critical to understand how they work to write effective code.
https://firebase.google.com/docs/functions/video-series/
You should make the second db.ref call in promise chain once the first promise resolved like this:
db.ref(`/users/${usernameWhoOwnsThePost}/posts/${usernameWhoOwnsThePostID}/comments`).once('value')
.then(snap => {
commentIdSpecific = snap.val();
let ids = [];
for (var id in snap.val()) {
ids.push(id);
}
let lastValueId = ids[ids.length - 1]
console.log(`last id value ${lastValueId}. UserPost: ${usernameWhoOwnsThePost}. user owner post id: ${usernameWhoOwnsThePostID}...`);
commentIdSpecific = lastValueId;
console.log(`comm id ${commentIdSpecific}`);
return commentIdSpecific;
})
.then(commentIdSpecific => {
db.ref(`/users/${usernameWhoOwnsThePost}/posts/${usernameWhoOwnsThePostID}/comments/${commentIdSpecific}/comment`).once('value').then(snap => {
commentPoster = snap.val();
console.log(`commentPoster: ${snap.val()}`);
console.log(`test of variables inisde of post: ${usernameWhoOwnsThePost}, uwotpi: ${commentIdSpecific}`)
return commentPoster
}).catch(err => {
console.log(err);
});
})
.catch(err => {
console.log(err);
});
once() is an async operation so it might possible that console.log(test of variables inside of post: ${usernameWhoOwnsThePost}, uwotpi: ${commentIdSpecific}) executed before commentIdSpecific = snap.val(); and commentIdSpecific = lastValueId;
So what you need to do is first let db.ref(/users/${usernameWhoOwnsThePost}/posts/${usernameWhoOwnsThePostID}/comments) be completed and then make call to db.ref(/users/${usernameWhoOwnsThePost}/posts/${usernameWhoOwnsThePostID}/comments/${commentIdSpecific}/comment) in next .then() in the chain.
As you are returning commentIdSpecific form first .then() So it will be available as param in second .then().
https://javascript.info/promise-chaining will help you to dig .then chaining more into deep.
I got a little bit of a problem. Here is the code:
Situation A:
var foundRiders = [];
riders.forEach(function(rider){
Rider.findOne({_id: rider}, function(err, foundRider){
if(err){
console.log("program tried to look up rider for the forEach loop finalizing the results, but could not find");
} else {
foundRiders.push(foundRider);
console.log(foundRiders);
}
});
});
Situation B
var foundRiders = [];
riders.forEach(function(rider){
Rider.findOne({_id: rider}, function(err, foundRider){
if(err){
console.log("program tried to look up rider for the forEach loop finalizing the results, but could not find");
} else {
foundRiders.push(foundRider);
}
});
});
console.log(foundRiders);
So in Situation A when I console log I get that foundRiders is an array filled with objects. In situation B when I put the console.log outside the loop, my roundRiders array is completely empty...
How come?
As others have said, your database code is asynchronous. That means that the callbacks inside your loop are called sometime later, long after your loop has already finishes. There are a variety of ways to program for an async loop. In your case, it's probably best to move to the promise interface for your database and then start using promises to coordinate your multiple database calls. You can do that like this:
Promise.all(riders.map(rider => {
return Rider.findOne({_id: rider}).exec();
})).then(foundRiders => {
// all found riders here
}).catch(err => {
// error here
});
This uses the .exec() interface to the mongoose database to run your query and return a promise. Then, riders.map() builds and returns an array of these promises. Then,Promise.all()monitors all the promises in the array and calls.then()when they are all done or.catch()` when there's an error.
If you want to ignore any riders that aren't found in the database, rather than abort with an error, then you can do this:
Promise.all(riders.map(rider => {
return Rider.findOne({_id: rider}).exec().catch(err => {
// convert error to null result in resolved array
return null;
});
})).then(foundRiders => {
foundRiders = foundRiders.filter(rider => rider !== null);
console.log(founderRiders);
}).catch(err => {
// handle error here
});
To help illustrate what's going on here, this is a more old fashioned way of monitoring when all the database callbacks are done (with a manual counter):
riders.forEach(function(rider){
let cntr = 0;
Rider.findOne({_id: rider}, function(err, foundRider){
++cntr;
if(err){
console.log("program tried to look up rider for the forEach loop finalizing the results, but could not find");
} else {
foundRiders.push(foundRider);
}
// if all DB requests are done here
if (cntr === riders.length) {
// put code here that wants to process the finished foundRiders
console.log(foundRiders);
}
});
});
The business of maintaining a counter to track multiple async requests is what Promise.all() has built in.
The code above assumes that you want to parallelize your code and to run the queries together to save time. If you want to serialize your queries, then you could use await in ES6 with a for loop to make the loop "wait" for each result (this will probably slow things down). Here's how you would do that:
async function lookForRiders(riders) {
let foundRiders = [];
for (let rider of riders) {
try {
let found = await Rider.findOne({_id: rider}).exec();
foundRiders.push(found);
} catch(e) {
console.log(`did not find rider ${rider} in database`);
}
}
console.log(foundRiders);
return foundRiders;
}
lookForRiders(riders).then(foundRiders => {
// process results here
}).catch(err => {
// process error here
});
Note, that while this looks like it's more synchronous code like you may be used to in other languages, it's still using asynchronous concepts and the lookForRiders() function is still returning a promise who's result you access with .then(). This is a newer feature in Javascript which makes some types of async code easier to write.
I am using Firebase Cloud Firestore, however, I think this may be more of a JavaScript asynchronous vs synchronous promise return issue.
I am doing a query to get IDs from one collection, then I am looping over the results of that query to lookup individual records from another collection based on that ID.
Then I want to store each found record into an array and then return the entire array.
results.length is always 0 because return results fires before the forEach completes. If I print results.length from inside the forEach it has data.
How can I wait until the forEach is done before returning from the outer promise and the outer function itself?
getFacultyFavoritesFirebase() {
var dbRef = db.collection("users").doc(global.user_id).collection("favorites");
var dbQuery = dbRef.where("type", "==", "faculty");
var dbPromise = dbQuery.get();
var results = [];
return dbPromise.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
var docRef = db.collection("faculty").doc(doc.id);
docRef.get().then(function(doc) {
if (doc.exists) {
results.push(doc);
}
})
});
console.log(results.length);
return results;
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
}
The trick here is to populate results with promises rather than the result. You can then call Promise.all() on that array of promises and get the results you want. Of course, you can't check if doc.exists before pushing the promise so you will need to deal with that once Promise.all() resolves. For example:
function getFacultyFavoritesFirebase() {
var dbRef = db.collection("users").doc(global.user_id).collection("favorites");
var dbQuery = dbRef.where("type", "==", "faculty");
var dbPromise = dbQuery.get();
// return the main promise
return dbPromise.then(function(querySnapshot) {
var results = [];
querySnapshot.forEach(function(doc) {
var docRef = db.collection("faculty").doc(doc.id);
// push promise from get into results
results.push(docRef.get())
});
// dbPromise.then() resolves to a single promise that resolves
// once all results have resolved
return Promise.all(results)
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
}
getFacultyFavoritesFirebase
.then(results => {
// use results array here and check for .exists
}
If you have multiple items of work to perform at the same time that come from a loop, you can collect all the promises from all the items of work, and wait for them all to finish with Promise.all(). The general form of a possible solution looks like this:
const promises = [] // collect all promises here
items.forEach(item => {
const promise = item.doWork()
promises.push(promise)
})
Promise.all(promises).then(results => {
// continue processing here
// results[0] is the result of the first promise in the promises array
})
You can adapt this to something that suits your own specific form.
Use for of instead of forEach. Like this:
for (const item of array) {
//do something
}
console.log("finished");
"finished" will be logged after finishing the loop.
Well I know, the thread is old, but the problem is still the same. And because I did run into the same issue and non of the answers did work for me, I want share my solution.
I think it will help someone out there. And maybe it will help me, if I run into the same problem again. ;-)
So the solution is super easy. firebase implements "map" but not direct on snaposhot, but on snapshot.docs.map.
In combination with Promieses.all it works just fine.
const promise = snapshot.docs.map(async (tenant) => {
return CheckTenant(tenant.id).catch(error =>
reject(error),
);
});
Promise.all(promise).then(result => {
// do somothing with the result});