Firestore slow queries are causing the entire logic to crash - javascript

I am currently designing a todo app with react and firebase without any node js server code. Everything was fine when inserting data and authenticating, but when I tried to query all the tasks where uid equals a certain value(that means all the tasks that belongs to a particular user), somehow the query gets skipped and returns null. But then, after a second or so, the array got retrieved and printed out.
The code:
function retrieveTasksOfUser(uid) {
// get all tasks where the uid equals the given value
return db.collection("tasks").where("uid", "==", uid).get();
}
function retrieveTasks() {
let res = [];
retrieveTasksOfUser(currentUser["uid"])
.then((snapshot) => {
snapshot.forEach((doc) => {
// include the doc id in each returned object
let buffer = doc.data();
buffer["id"] = doc.id;
res.push(buffer);
});
console.log(res);
return res;
})
.catch((err) => {
console.error("Error retrieving tasks: ", err);
return res;
});
}
let tasks = retrieveTasks();
console.log({ tasks });
EDIT:
Inspired by Frank's answer, I modified my code and got:
async function retrieveTasks() {
let res = [];
const snapshot = await retrieveTasksOfUser(currentUser["uid"]);
snapshot.forEach((doc) => {
let buffer = doc.data();
buffer["id"] = doc.id;
res.push(buffer);
});
return res;
}
let tasks = [];
let promise = retrieveTasks();
promise.then((res) => {
console.log("Tasks are");
tasks = res;
console.log(res);
});
console.log(tasks);
But the result turns out to be an empty array
Thanks a bunch in advance.

You're not returning anything from the top-level code in retrieveTasks, so that means that tasks will always be undefined.
The simplest fix is to return the result of the query, which then means that your return res will be bubbled up:
return retrieveTasksOfUser(currentUser["uid"])
...
But this means you're returning a promise, as the data is loaded asynchronously. So in the calling code, you have to wait for that promise to resolve with then:
retrieveTasks().then((tasks) => {
console.log({ tasks });
})
You can make all of this a lot more readable (although it'll function exactly the same) by using the async and await keywords.
With those, the code would become:
async function retrieveTasks() {
let res = [];
const snapshot = await retrieveTasksOfUser(currentUser["uid"]);
snapshot.forEach((doc) => {
let buffer = doc.data();
buffer["id"] = doc.id;
res.push(buffer);
});
return res;
}
let tasks = await retrieveTasks();
console.log({ tasks });
Or with a bit more modern JavaScript magic, we can make it even shorter:
async function retrieveTasks() {
const snapshot = await retrieveTasksOfUser(currentUser["uid"]);
return snapshot.docs.map((doc) => { ...doc.data, id: doc.id });
}
let tasks = await retrieveTasks();
console.log({ tasks });

Related

Use fetched API data outside of the function

To make my code cleaner I want to use fetched API data in a few different functions, instead of one big. Even though I 've did manage to reffer to that data in other functions, the problem is the API im a fetching throws different, randomized results every time it is called. And so the output from userData() does not equal that from userData2(), even though my intention is different and I'd like the result variable contents to be the same between functions.
const getData = () =>
fetch("https://opentdb.com/api.php?amount=10").then((response) =>
response.json()
);
const useData = async () => {
const result = await getData();
console.log(result);
};
const useData2 = async () => {
const result = await getData();
console.log(result);
};
Your getData() function returns a promise. One fun fact about promises is that while they can only resolve once, that resolved value can be accessed and used as many times as you want.
const dataPromise = getData();
const useData = async () => {
const result = await dataPromise;
console.log(result);
};
const useData2 = async () => {
const result = await dataPromise;
console.log(result);
};
Using await resolves the promise value, the equivalent of...
dataPromise.then((result) => {
console.log(result);
});
// or `dataPromise.then(console.log)` if you like brevity
I like to point this out about the fetch-api... you should always check the Response.ok property
const getData = async () => {
const res = await fetch("https://opentdb.com/api.php?amount=10");
if (!res.ok) {
throw new Error(`${res.status}: ${await res.text()}`);
}
return res.json();
};

Calling async / await function and receiving a value back

I have got a function in an API library that calls firestore and gets data back. This part works fine:
export const getUserByReferrerId = async id => {
let doc = await firestore
.collection(FIRESTORE_COLLECTIONS.USERS)
.where('grsfId', '==', id)
.get()
.then(querySnapshot => {
if (!querySnapshot.empty) {
console.log ("we found a doc");
// use only the first document, but there could be more
const snapshot = querySnapshot.docs[0];
console.log ("snapshot", snapshot.id);
return snapshot.id // uid of the user
}
});
}
I am calling this library from a component. I have another function that runs on a button click. I cannot figure out how to get the value from the async api call.
I have tried this - a promise appears when I console.log the return:
testCreditFunction = (id) => {
let uid = getUserByReferrerId(id).then(doc => {
console.log("uid1", doc);
});
}
I have also tried this - log shows null for uid.
testCreditFunction = (id) => {
let uid = '';
(async () => {
uid = await getUserByReferrerId(id);
console.log ("uid in the function", uid);
})();
}
I have seen this question asked a few times and I have tried several of the answers and none are working for me. The odd thing is that I have done this same thing in other areas and I cannot figure out what the difference is.
Change your funtion to this.
export const getUserByReferrerId = async id => {
return await firestore
.collection(FIRESTORE_COLLECTIONS.USERS)
.where('grsfId', '==', id)
.get();
}
Try getting data this way.
testCreditFunction = (id) => {
let querySnapshot = '';
(async () => {
querySnapshot = await getUserByReferrerId(id);
const snapshot = querySnapshot.docs[0];
console.log ("snapshot", snapshot.id);
})();
One thing I notice right off the bat is that you're mixing async/await & .then/.error. While it's not strictly forbidden it definitely makes things more difficult to follow.
As others have mentioned in the comments, you need to make a return for the promise to resolve (complete). Here's how you might write getUserByReferrerId (using async/await).
const getUserByReferrerId = async id => {
const usersRef = firestore.collection('users');
const query = usersRef.where('grsfId', '==', id);
const snapshot = await query.get();
if (snapshot.empty) {
console.log('no doc found');
return;
}
console.log('doc found');
const doc = snapshot.docs[0]; // use first result only (there might be more though)
return doc.id
}
You'll notice how I split up the query building steps from the snapshot request. This was intentional as the only promised function in this is the get request.
Using this getUserByReferrerID can be done with a simple await. Just be sure to check that the result isn't undefined.
const userID = await getUserByReferrerId('someid')
if (!userID) {
console.log('user not found');
}
console.log(`User ID is ${userID}`);
p.s - I would recommend renaming the function to getUserIdByReferrerId to more accurately reflect the fact that you're returning an ID and not a user doc. Alternatively, you can return the user object and leave the name as is.

Need help in fetching document with reference id in firestore using reactjs

I'm a newbie with firestore and reactjs. I'm trying to write a simple react code to fetch documents from 2 collections
user-habits {email, habitid}
habits {habitid, content +few more fields}
each user can have one or more habits
I'm trying to execute the following react code to
export async function getHabit(habitid) {
let result = [];
return db.collection('habits')
.where('habitid','==',`${habitid}`)
.get()
.then(snapshot => {
snapshot.forEach(doc => {
result.push({...doc.data()})
})
return result;
})
}
export async function getUserHabit(email) {
let result = [];
return db.collection('user-habits')
.where('email','==',`${email}`)
.get()
.then(snapshot => {
snapshot.forEach(doc => {
let habitid = doc.data().habitid
let habit = getHabit(habitid)
result.push({...doc.data(), ...habit, uid:doc.id})
})
return result;
})
}
For some reason, I always get blank data on getHabit call. console log statement, shows that a Promise is returned by getHabit, which resolves after the code is completely executed. Thanks in advance
You can use async and await inside the getUserHabit() function to wait for the promise returned by getHabit() to be resolved
export async function getUserHabit(email) {
let result = [];
return db.collection('user-habits')
.where('email','==',`${email}`)
.get()
.then(snapshot => {
snapshot.forEach(async doc => {
let habitid = doc.data().habitid
let habit = await getHabit(habitid)
console.log(habit) // will log the habit value
result.push({...doc.data(), ...habit, uid:doc.id})
})
return result;
})
}
You don't need to return the db.collection in this case. Since you're calling from the front end, you can just reference the collection path and wait for Firestore to return what you're looking for. Also, you don't need to use the .then() approach with async/await. If you're expecting an array in return, try something like this:
export async function getHabit(habitid) {
let result = [];
let get = await db.collection('habits').where('habitid','==',`${habitid}`).get()
get.forEach(doc => {
result.push({...doc.data()})
})
return result;
})
}
export async function getUserHabit(email) {
let result = [];
let get = await db.collection('user-habits').where('email','==',`${email}`).get()
get.forEach(doc => {
let habitid = doc.data().habitid
let habit = getHabit(habitid)
result.push({...doc.data(), ...habit, uid:doc.id})
})
return result;
})
}

Javascript promise chain hell

I'm having a problem with promise chains that I don't know how to resolve. Summary of my code: I do a mongoose query for a specific user, fetch his CarIds and then query each car for his details and return these details via JSON response.
let carsDetails = [];
User.findById(userId)
.then(user => {
const carIds = user.carsDetails;
carIds.forEach((carId) => {
Car.findById(carId)
.then(car => {
console.log(car);
carsDetails.push(car);
})
.catch(err => { throw err; });
});
return res.status(200).json({ data: carsDetails });
})
.catch(error => {
throw error;
});
The problem is that no cars get actually pushed onto carsDetails array on the carsDetails.push(car); line, because it jumps to return statement before it manages to fill up the array. Is there a workaround that could do those queries and return a result in a form of an array, object...? I tried writing everything in async await form too with self-made async forEach statement, but it doesn't help me. Any suggestions? I already tried with Promise.all(), but didn't manage to fix the issue. Thanks!
You'll need to collect the promises of your Car.findById(carId) calls and use Promise.all() to wait for all of them before responding. You can use array.map() to map each ID to a promise from Car.findById().
User.findById(userId)
.then(user => {
const carIds = user.carsDetails;
const carPromises = carIds.map(carId => Car.findById(carId))
return Promise.all(carPromises)
})
.then(cars => {
res.status(200).json({ data: cars })
})
.catch(error => {
throw error
})
If you can have a lot of cars to find, you may want to do your query in a single request, no need to stack multiple promises :
User.findById(userId)
.then(user => {
const carIds = user.carsDetails;
// if carsDetails is not an array of objectIds do this instead :
// const carIds = user.carsDetails.map(id => mongoose.Types.ObjectId(id));
return Car.find({ _id: { $in: carIds });
})
.then(userCars => {
res.status(200).json({ data: userCars })
})
await/async is the way to go, with await/async you use regular for ... of loops instead of forEach.
async function getCarDetails() {
let carsDetails = [];
let user = await User.findById(userId);
const carIds = user.carsDetails;
for (let carID of carIds) {
let car = await Car.findById(carId)
console.log(car);
carsDetails.push(car);
}
return res.status(200).json({
data: carsDetails
});
}
Or you use Promise.all and map instead of for ... of
async function getCarDetails() {
let user = await User.findById(userId);
const carIds = user.carsDetails;
let carsDetails = await Promise.all(carIds.map(carID => Car.findById(carID)));
return res.status(200).json({
data: carsDetails
});
}
Those two solutions are slightly different. The second version with the map will send all requests to the DB at once, and then waits until they are all resolved. The first one will send the request one after another. Which one is better depends on the use-case, the second one could lead to request peeks, and might be easier be abused for DDoS.
Try to use async/await to solve this problem. It will be more readable and clean
async function getCarDetail() {
let carsDetails = []
try {
const user = await User.findById(userId)
const carIds = user.carsDetails
for (let i = 0; i < carIds.length; i++) {
const carId = carIds[i]
const car = await Car.findById(carId)
carsDetails.push(car)
}
return res.status(200).json({ data: carsDetails })
} catch(error) {
console.log(error)
}
}

Getting all documents from one collection in Firestore

Hi I'm starting with javascript and react-native and I'm trying to figure out this problem for hours now. Can someone explain me how to get all the documents from firestore collection ?
I have been trying this:
async getMarkers() {
const events = await firebase.firestore().collection('events').get()
.then(querySnapshot => {
querySnapshot.docs.map(doc => {
console.log('LOG 1', doc.data());
return doc.data();
});
});
console.log('LOG 2', events);
return events;
}
Log 1 prints all the objects(one by one) but log 2 is undefined, why ?
The example in the other answer is unnecessarily complex. This would be more straightforward, if all you want to do is return the raw data objects for each document in a query or collection:
async getMarker() {
const snapshot = await firebase.firestore().collection('events').get()
return snapshot.docs.map(doc => doc.data());
}
if you want include Id
async getMarkers() {
const events = await firebase.firestore().collection('events')
events.get().then((querySnapshot) => {
const tempDoc = querySnapshot.docs.map((doc) => {
return { id: doc.id, ...doc.data() }
})
console.log(tempDoc)
})
}
Same way with array
async getMarkers() {
const events = await firebase.firestore().collection('events')
events.get().then((querySnapshot) => {
const tempDoc = []
querySnapshot.forEach((doc) => {
tempDoc.push({ id: doc.id, ...doc.data() })
})
console.log(tempDoc)
})
}
I made it work this way:
async getMarkers() {
const markers = [];
await firebase.firestore().collection('events').get()
.then(querySnapshot => {
querySnapshot.docs.forEach(doc => {
markers.push(doc.data());
});
});
return markers;
}
if you need to include the key of the document in the response, another alternative is:
async getMarker() {
const snapshot = await firebase.firestore().collection('events').get()
const documents = [];
snapshot.forEach(doc => {
const document = { [doc.id]: doc.data() };
documents.push(document);
}
return documents;
}
The docs state:
import { collection, getDocs } from "firebase/firestore";
const querySnapshot = await getDocs(collection(db, "cities"));
querySnapshot.forEach((doc) => {
// doc.data() is never undefined for query doc snapshots
console.log(doc.id, " => ", doc.data());
});
However I am using the following (excuse the TypeScript):
import { collection, Firestore, getDocs, Query, QueryDocumentSnapshot, QuerySnapshot } from 'firebase/firestore'
const q: Query<any> = collection(db, 'videos')
const querySnapshot: QuerySnapshot<IVideoProcessed> = await getDocs(q)
const docs: QueryDocumentSnapshot<IVideoProcessed>[] = querySnapshot.docs
const videos: IVideoProcessed[] = docs.map((doc: QueryDocumentSnapshot<IVideoProcessed>) => doc.data())
where db has the type Firestore
You could get the whole collection as an object, rather than array like this:
async function getMarker() {
const snapshot = await firebase.firestore().collection('events').get()
const collection = {};
snapshot.forEach(doc => {
collection[doc.id] = doc.data();
});
return collection;
}
That would give you a better representation of what's in firestore. Nothing wrong with an array, just another option.
Two years late but I just began reading the Firestore documentation recently cover to cover for fun and found withConverter which I saw wasn't posted in any of the above answers. Thus:
If you want to include ids and also use withConverter (Firestore's version of ORMs, like ActiveRecord for Ruby on Rails, Entity Framework for .NET, etc), then this might be useful for you:
Somewhere in your project, you probably have your Event model properly defined. For example, something like:
Your model (in TypeScript):
./models/Event.js
export class Event {
constructor (
public id: string,
public title: string,
public datetime: Date
)
}
export const eventConverter = {
toFirestore: function (event: Event) {
return {
// id: event.id, // Note! Not in ".data()" of the model!
title: event.title,
datetime: event.datetime
}
},
fromFirestore: function (snapshot: any, options: any) {
const data = snapshot.data(options)
const id = snapshot.id
return new Event(id, data.title, data.datetime)
}
}
And then your client-side TypeScript code:
import { eventConverter } from './models/Event.js'
...
async function loadEvents () {
const qs = await firebase.firestore().collection('events')
.orderBy('datetime').limit(3) // Remember to limit return sizes!
.withConverter(eventConverter).get()
const events = qs.docs.map((doc: any) => doc.data())
...
}
Two interesting quirks of Firestore to notice (or at least, I thought were interesting):
Your event.id is actually stored "one-level-up" in snapshot.id and not snapshot.data().
If you're using TypeScript, the TS linter (or whatever it's called) sadly isn't smart enough to understand:
const events = qs.docs.map((doc: Event) => doc.data())
even though right above it you explicitly stated:
.withConverter(eventConverter)
Which is why it needs to be doc: any.
(But! You will actually get Array<Event> back! (Not Array<Map> back.) That's the entire point of withConverter... That way if you have any object methods (not shown here in this example), you can immediately use them.)
It makes sense to me but I guess I've gotten so greedy/spoiled that I just kinda expect my VS Code, ESLint, and the TS Watcher to literally do everything for me. 😇 Oh well.
Formal docs (about withConverter and more) here: https://firebase.google.com/docs/firestore/query-data/get-data#custom_objects
I prefer to hide all code complexity in my services... so, I generally use something like this:
In my events.service.ts
async getEvents() {
const snapchot = await this.db.collection('events').ref.get();
return new Promise <Event[]> (resolve => {
const v = snapchot.docs.map(x => {
const obj = x.data();
obj.id = x.id;
return obj as Event;
});
resolve(v);
});
}
In my sth.page.ts
myList: Event[];
construct(private service: EventsService){}
async ngOnInit() {
this.myList = await this.service.getEvents();
}
Enjoy :)
Here's a simple version of the top answer, but going into an object with the document ids:
async getMarker() {
const snapshot = await firebase.firestore().collection('events').get()
return snapshot.docs.reduce(function (acc, doc, i) {
acc[doc.id] = doc.data();
return acc;
}, {});
}
General example to get products from Cloud Firestore:
Future<void> getAllProducts() async {
CollectionReference productsRef =
FirebaseFirestore.instance.collection('products');
final snapshot = await productsRef.get();
List<Map<String, dynamic>> map =
snapshot.docs.map((doc) => doc.data() as Map<String, dynamic>).toList();
}
In version 9 sdk of firebase you can get all the documents from a collection using following query:
const querySnapshot = await getDocs(collection(db, "cities"));
querySnapshot.docs.forEach((doc) => {
console.log(doc.id, " => ", doc.data());
});
See Get multiple documents from a collection
I understand your query, This is because how Javascript handles promises and variables. So basically events variable is hoisted with the value undefined and printed on the LOG 2 console log, while the Event Loop responsible for the promise call resulted in an array of objects as the value of the events variable and then the console log (LOG 1) was printed with the resolved promise response
All answers are true, but when you have heavy data you will face memory and bandwidth problems, so you have to write a [cursor] function to read data part by part.
also, you may face to Bandwidth Exhausted error, please have a look at this solution I have implemented on a gist
https://gist.github.com/navidshad/973e9c594a63838d1ebb8f2c2495cf87
Otherwise, you can use this cursor I written to read a collection doc by doc:
async function runCursor({
collection,
orderBy,
limit = 1000,
onDoc,
onDone,
}) {
let lastDoc;
let allowGoAhead = true;
const getDocs = () => {
let query = admin.firestore().collection(collection).orderBy(orderBy).limit(limit)
// Start from last part
if (lastDoc) query = query.startAfter(lastDoc)
return query.get().then(sp => {
if (sp.docs.length > 0) {
for (let i = 0; i < sp.docs.length; i++) {
const doc = sp.docs[i];
if (onDoc) onDoc(doc);
}
// define end of this part
lastDoc = sp.docs[sp.docs.length - 1]
// continue the cursor
allowGoAhead = true
} else {
// stop cursor if there is not more docs
allowGoAhead = false;
}
}).catch(error => {
console.log(error);
})
}
// Read part by part
while (allowGoAhead) {
await getDocs();
}
onDone();
}

Categories

Resources