Firebase Firestore - Create dynamic compound index for queries - javascript

I have created a chat system where, for sending a message, I previosly check that the sender and receiver is in the room, using a query. In a first instance, I thought to use a field "members" which was an array containing two strings (the users ids). But, the problem I had was that I would have to use two array-contains in that query
chatsRef
.where("members", "array-contains", fromId)
.where("members", "array-contains", toId)
...
Something which is not valid, as the docs says:
you can include at most one array-contains or array-contains-any
clause in a compound query.
So, instead of using an array, I have used a map where the key is the user's id, and the value a true boolean.
members = {
"832384798238912789": true,
"90392p2010219021ud": true,
}
Then I query like:
chatsRef
.where(`members.${fromId}`, "==", true)
.where(`members${toId}`, "==", true)
...
Now, the problem I am having is that I need to get all the chats of an specific user, in order of the last message date.
For this, I am querying like this:
const query = firebase
.collection("chats")
.where(`members.${firebase.getCurrentUser().uid}`, "==", true)
.orderBy("lastMessage.date")
.startAfter(startAfter);
Something which throws me a link to create a compound index which looks like this:
members.832384798238912789 Asc lastMessage.date Asc
How can I generalize this compound index for all users? I mean, something like:
members.key Asc lastMessage.date Asc
Is this possible? If not, any workaround to solve this problem?
What I have thought to do (looking for something better, if possible):
Add an extra field (apart from the "members" map), "membersArray", which will allow me to query like
.where("membersArray", "array-contains", userId)
.orderBy("lastMessage.date")
Solves the problem but messes up the database modeling
Add the chat id to a subcollection for each user (unnecesary writes), so I only have to query that collection with a simple index, ignoring the
.where(`members.${firebase.getCurrentUser().uid}`, "==", true)

Related

How to implement pagination in a merged set of queries when implementing a logical OR

In the Firestore documentation, it states clearly the limitations of support for query filters with logical OR.
For example:
const userPostsQuery = query(postsRef, where("author", "==", uid);
const publicPostsQuery = query(postsRef, where("public", "==", true);
If as in the above example, we need to get a list of both, user posts and public posts all sorted together by date, ie: Both queries need to be OR-ed together, such a feature is not available in Firestore and we will have to run both queries separately, and then merge and sort the results on the client-side.
I'm fine with such a sad workaround. but what if the total number of posts can be huge? thus we need to implement a pagination system where each page shows 50 posts max. How can this be done with such a sad workaround?
Firestore has very limited operators and aggregation options. However, it has limited OR support with an Array type.
A solution that could simplify your use case is to introduce a new field of type array in your post document. Let's say this field is named a. When you create your document, a is equal to [authorId, 'public'] if the post is public, [authorId] otherwise.
Then, you can query your need using the array-contains-any operator:
const q = query(postRef, where('a', 'array-contains-any', [authorId, 'public']));
You can easily add pagination with limit, orderBy, startAt, and startAfter functions.

How do I implement an index for where and orderBy timestamp?

I have this field for the collection in users with either 2 status of condition1 and condition2. And I also wanted to order this in descending order. I tried it with this but it will display that I first need to create an index.
The status is in string. And the createdAt is in timestamp
const userRef = collection(db, "users");
const q = query(
ordersRef,
where("status", "==", "Condition1"),
orderBy("createdAt", "desc")
);
const querySnapshot = await getDocs(q);
Also, is it necessary to filter this with Firebase or I could just filter using Javascript? However, I would still like to know how I can use indexing here.
If you attempt the query above without first creating the required index, Cloud Firestore returns an error message containing a link you can follow to create the missing index.
This happens any time you attempt a query not supported by an index. You can also define and manage composite indexes which store a sorted mapping of all the documents in a collection, based on an ordered list of fields to index manually by using the console or by using the Firebase CLI.
Cloud Firestore does not automatically create composite indexes like it does for single-field indexes because of the large number of possible field combinations. Instead, Cloud Firestore helps you identify and create required composite indexes as you build your app. Follow the generated link to the Firebase console, review the automatically populated info, and click Create. Indexes can take a few minutes to build, depending on the size of the query.
After you create them, you can see your indexes and their status in the Composite Indexes section. If they're still building, the Firebase console includes a building status bar.

Firestore query after adding composite index still return nothing

I have already created react app and I'm using firebase firestore.
So at the start of using I didn't have any problems, until I found that I need to create a query like that:
db.collection('schedule')
.where('working_day', '>=', new Date())
.where('master', '==', masterId)
.orderBy('working_day', 'desc')
.get();
I have a collection called schedule and this schedule collection has list of documents. Each document has structure:
working_day: timestamp;
start_hour: number;
end_hour: number;
master: string;
When I called this query - I didn't see any result (request is gone, but no results at all). Someone suggest me to add composite indexing and It must solve this problem
I did it using firestore indexing tab and now I have this indexing:
Collection ID Fields indexed Query scope Status
schedule working_day Descending master Descending Collection Enabled
It has been successfully applied - but still my request without any result.
Also I know that firestore could notice me that I need to apply this indexing and provide me the link - but I don't see this link. Where I should find this? Or maybe I do something wrong?
Break my head for the 2 days and didn't find why I have this issue
It looks like working.day is stored in your firestore database as a timestamp but your query condition is comparing against a javascript date object which could be causing an issue. Try replacing the date object with a firebase timestamp like the code below.
db.collection('schedule')
.where('working_day', '>=', new firebase.firestore.Timestamp.now())
.where('master', '==', masterId)
.orderBy('working_day', 'desc')
.get();

Using Where and Order by different fields in Firestore query

I have a Firestore collection named channels, and I'd like to get the list of channels based on an array of IDs and order it by the createdAt field, this is my function :
const getChannels = () => {
const q = query(
collection(db, "channels"),
where(documentId(), "in", [
"F0mnR5rNdhwSLPZ57pTP",
"G8p6TWSopLN4dNHJLH8d",
"wMWMlJwa3m3lYINNjCLT",
]),
orderBy("createdAt")
);
const unsubscribe = onSnapshot(q, (snapshot) => {
snapshot.docs.map((doc) => {
console.log(doc.data());
});
});
return unsubscribe;
};
But I'm getting this error
FirebaseError: inequality filter property and first sort order must be the same: __name__ and createdAt.
It only works if I orderBy documentId().
I'm aware there is a limitation in the docs about this, but I'm wondering if there is a workaround for this type of situation.
Also the answer for this question isn't working anymore I guess.
The title of your question indicates that you are trying to use where and orderBy for different fields. But note that you are using documentId() in the where condition to filter, which is not a field in the Firestore document.
So if you filter is based on documentId(), you can use only documentId() in orderBy() clause, that also in ascending order because currently Firestore does not support sorting in descending order of documentId() which is mentioned in this answer.
Let’s take a look at the following examples -
const data=await db.collection("users").where(admin.firestore.FieldPath.documentId(),"in",["104","102","101"]).orderBy(admin.firestore.FieldPath.documentId()).get();
The above will work and sort the documents based on documentId() after filtering based on documentId().
But it is not relevant to apply an orderBy() clause based on the documentId(), because without applying the orderBy() clause also yields the same result as, by default, Firestore query gives documents in ascending order of documentId(). That means the following also yields the same result -
const data=await db.collection("users").where(admin.firestore.FieldPath.documentId(),"in",["104","102","101"]).get();
Now Firestore doesn’t support to sort in descending order of documentId() which means the following will not work -
const data=await db.collection("users").where(admin.firestore.FieldPath.documentId(),"in",["104","102","101"]).orderBy(admin.firestore.FieldPath.documentId(),"desc").get();
This will ask to create an index -
The query requires an index. You can create it here:
But if you go there to create an index it will say -
__name__ only indexes are not supported.
Now let's come to your query. What you are trying to do is to filter based on documentId() and then orderBy() based on createdAt field which is not possible and it will give the following error-
inequality filter property and first sort order must be the same.
You may think to use two orderBy() clauses, something like this -
const data=await db.collection("users").where(admin.firestore.FieldPath.documentId(),"in",["104","102","101"]).orderBy(admin.firestore.FieldPath.documentId()).orderBy(“createdAt”
).get();
Which will not work and give the following error
order by clause cannot contain more fields after the key
I am not sure of your use case but it’s not a great idea to filter based on documentId(). If it is required to filter based on documentId(), I would suggest creating a field in the Firestore document which will contain the documentIds and filter based on that.
Now considering the title of the question, yes it is possible to use where() and orderBy() clauses for different fields in Firestore. There are some limitations and you need to stick to that -
If you include a filter with a range comparison (<, <=, >, >=), your first ordering must be on the same field.
const data=await db.collection("users").where(“number”,">=", “101”).orderBy(“createdAt”).get();
The above query doesn't work.
const data=await db.collection("users").where(“number”,">=", “101”).orderBy(“number”).get();
The above query works and you can still use further orderBy() on different fields, something like following -
const data=await db.collection("users").where(“number”,">=", “101”).orderBy(“number”).orderBy(“createdAt”).get();
You cannot order your query by any field included in an equality (=) or in clause.
const data=await db.collection("users").where(“number”,"in",["104","102","101"]).orderBy(“number”).get();
const data=await db.collection("users").where(“number”,"==", “101”).orderBy(“number”).get();
The above two don’t work.
Firestore's speed and efficiency comes almost ENTIRELY from it's use of indexes. Inequalities (INCLUDING in and not-in) are accomplished by sorting by the index, and using the value as a "cut-off" - thus REQUIRING (whether you want it or not) the orderby() to be on the same field as the inequality.
The "answer not working anymore" was never really working in the first place, as the above shows. If you aren't trying to paginate, do the obvious and "filter" by the document ID's and sort on the client.
BUT...
...more importantly, it is ALMOST NEVER useful nor performant to use documentId's to select from the database, unless you both copy it to a field, AND are looking for a SPECIFIC id. In almost all cases, it would be FAR better to use a query on another field (however you got the list of documentId's in the first place), then orderBy. Yes, the inequality/orderBy is a limitation, but it's there for a reason.
Going forward, an important design decision is to understand what questions you want your data to answer, and design your entire database schema to support those queries - this is the fundamental nature of NoSQL.
Problem:The other link that you have shared before perfectly works and the only solutions available is to create an index. However the reason you are not able to do a where and order with the above example is because you cannot create an index with the document id and createdAt.
Solution: To do so add the document id as one of the field say docID in the document then create an index with the fields docID and createdAt. This should be working for you.
Note: I have not physically tested this. Will update once I have checked it

Firestore get docs based off value existing in array

I am facing a little bit of a mental block in terms of how to do some relational queries with firestore while adhering to the best practices. I am creating a feed feature where you can see a feed of posts from your friends. Essentially my data structure is as follows:
Friends (collection)
-friend_doc
...data
friends_uid: [uid1, uid2]
Posts (collection)
-post_doc
...data
posted_by: uid2
Basically I am making a query to get all of the friends where the friends_uid contains my uid (uid1 in this case). And then once I mapped all of the friends uid's to an array, I want to make a firestore query to get posts where the posted_by field is equal to any of the uid's in that array of friends uid's. I haven't been able to make something that does anything like that yet.
I know that it seems most convenient to loop through the string array of friends uid's and make a query for each one like:
listOfUids.forEach(async (item) => {
const postQuerySnapshot = await firestore()
.collection('posts')
.where('uid', '==', item)
.get();
results.push(postQuerySnapshot.docs);
});
but this is extremely problematic for paging and limiting data as I could possibly receive tons of posts. I may just be too deep into this code and missing an obvious solution or maybe my data structure is somewhat flawed. Any insight would be greatly appreciated.
TLDR - how can I make a firestore query that gets all docs that have a value that exists in an array of strings?
You can use an "in" query for this:
firestore()
.collection('posts')
.where('uid', 'in', [uid1, uid2, ...])
But you are limited to 10 elements in that array. So you are probably going to have to stick to what you have now. You will not be able to use Firestore's pagination API.
Your only real alternatives for this case is to create a new collection that contains all of the data you want to query in one place, as there are no real join operations. Duplicating data like this is common for nosql type databases.

Categories

Resources