Develop Reference

Querying for object key in Firestore

I currently have a few issues with my Firestore querying technique. As per this stackoverflow post I made recently, Querying with two array with firestore security rules
The answer proposed to add the the "ids" into a object, with the key as the id, and the value simply being "true". I have completed this, and now my structure looks like so:
This leaves me with this query:
db.collection('Depots')
.where(`products.${productId}`, '==', true)
.where(`users.${userId}`, '==', true)
.where('created', '>', 1585998560500)
.orderBy('created', 'asc')
.get();
This query leaves me with throwing an error, asking to create an index:
The query requires an index. You can create it here: ...
However, this tries to index the specific object key, i.e. QXooVYGBIFWKo6C so products.QXooVYGBIFWKo6C. Which is certianly not what I want, as this query changes, and can have an infinite number of possibilities, which means I would have to create another index for each key entry in order to query it.
Is there any way to solve this issue? I am assuming it needs to index this query due to the different operators used in the query, so I was wondering if there were any workarounds to this issue.
Thank you very much in advance.

What you have here is a map field, for which indexes should usually be created automatically.
That indeed means that you'll have as many indexes as you have products, which means:
You are limited in how many products you can have, as there is a maximum of 40,000 index entries per document.
You pay more per document, as you pay for the storage of each index.
If these are not what you want, you'll have to switch back to your original model, with the query limitations you had there. There doesn't seem to be a solution that fits both of your requirements.

After our discussion in chat, this is the starting point I would suggest. Who knows what the end architecture would look like, but I think this or very close to this. You say that a user can exist in multiple depots at the same time and multiple depots can contain the same products, also at the same time. You also said that a depot can never have more than 40 users at a given time, so an array of 40 users would certainly not encroach on Firestore's document limit of 1,048,576 bytes.
[collection]
<documentId>
- field: value
[depots]
<UUID>
- depotId: string "depot456"
- productCount: num 5,000
<UUID>
- depotId: string "depot789"
- productCount: num 4,500
[products]
<UUID>
- productId: string "lotion123"
- depotId: string "depot456"
- users: [string] ["user10", "user27", "user33"]
<UUID>
- productId: string "lotion123"
- depotId: string "depot789"
- users: [string] ["user10", "user17", "user50"]
[users]
<userId>
- depots: [string] ["depot456", "depot999"]
<userId>
- depots: [string] ["depot333", "depot999"]
In NoSQL, storage is cheap and computation isn't so denormalize your data as much as you need to make your queries possible and efficient (fast and cheap).
To find all depots in a single query where user10 and lotion123 are both true, query the products collection where productId equals x and users array-contains y and collect the depotId values from those results. If you want to preserve the array-contains operation for something else, you'd have to denormalize your data further (replace the array for a single user). Or you could split this query into two separate queries.
With this model, when a user leaves a depot, get all products where users array-contains that user and remove that userId from the array. And when a user joins a depot, get all products where depotId equals x and append that userId to the array.
Watch this video, and others by Rick, to get a solid handle on NoSQL: https://www.youtube.com/watch?v=HaEPXoXVf2k

#danwillm If you are not sure about the number of users and products then your DB structure seems unfit for this situation because there are size and length limitations of the firestore document.
You should rather create a separate collection for products and users i.e normalize your data and have a reference for the user in the product collection.
User :
{
userId: documentId,
name: John,
...otherInfo
}
Product :
{
productId: documentId,
createdBy: userId,
createdOn:date,
productName:"exa",
...otherInfo
}
This way you there will be the size of the document would be limited, i.e try avoiding using maps/arrays in firestore if you are not sure about there size.
Also, in this case, the number of queries would be increased but you don't need many indexes in this case.

How to do an 'AND' statement in Firebase or equivalent?

I need to do a query where I can show only specific data using an 'AND' statement or equivalent to it. I have taken the example which is displayed in the Firebase Documentation.
// Find all dinosaurs whose height is exactly 25 meters.
var ref = firebase.database().ref("dinosaurs");
ref.orderByChild("height").equalTo(25).on("child_added", function(snapshot) {
console.log(snapshot.key);
});
I understand this line is going to retrieve all the dinosaurs whose height is exactly 25, BUT, I need to show all dinosaurs whose height is '25' AND name is 'Dino'. Is there any way to retrieve this information?
Thanks in advance.

Actually firebase only supports filtering/ordering with one propery, but if you want to filter with more than one property like you said I want to filter with age and name, you have to use composite keys.
There is a third party library called querybase which gives you some capabilities of multy property filtering. See https://github.com/davideast/Querybase

You cannot query by multiple keys.
If you need to sort by two properties your options are:
Create a hybrid key. In reference to your example, if you wanted to get all 'Dino' and height '25' then you would create a hybrid name_age key which could look something like Dino_25. This will allow you to query and search for items with exactly the same value but you lose the ability for ordering (i.e. age less than x).
Perform one query on Firebase and the other client side. You can query by name on Firebase and then iterate through the results and keep the results that match age 25.
Without knowing much about your schema I would advise you to make sure you're flattening your data sufficiently. Often I have found that many multi-level queries can be solved by looking at how I'm storing the data. This is not always the case and sometimes you may just have to take one of the routes I have mentioned above.

storing data as object vs array in MongoDb for write performance

Should I store objects in an Array or inside an Object with top importance given Write Speed?
I'm trying to decide whether data should be stored as an array of objects, or using nested objects inside a mongodb document.
In this particular case, I'm keeping track of a set of continually updating files that I add and update and the file name acts as a key and the number of lines processed within the file.
the document looks something like this
{
t_id:1220,
some-other-info: {}, // there's other info here not updated frequently
files: {
log1-txt: {filename:"log1.txt",numlines:233,filesize:19928},
log2-txt: {filename:"log2.txt",numlines:2,filesize:843}
}
}
or this
{
t_id:1220,
some-other-info: {},
files:[
{filename:"log1.txt",numlines:233,filesize:19928},
{filename:"log2.txt",numlines:2,filesize:843}
]
}
I am making an assumption that handling a document, especially when it comes to updates, it is easier to deal with objects, because the location of the object can be determined by the name; unlike an array, where I have to look through each object's value until I find the match.
Because the object key will have periods, I will need to convert (or drop) the periods to create a valid key (fi.le.log to filelog or fi-le-log).
I'm not worried about the files' possible duplicate names emerging (such as fi.le.log and fi-le.log) so I would prefer to use Objects, because the number of files is relatively small, but the updates are frequent.
Or would it be better to handle this data in a separate collection for best write performance...
{
"_id": ObjectId('56d9f1202d777d9806000003'),"t_id": "1220","filename": "log1.txt","filesize": 1843,"numlines": 554
},
{
"_id": ObjectId('56d9f1392d777d9806000004'),"t_id": "1220","filename": "log2.txt","filesize": 5231,"numlines": 3027
}

From what I understand you are talking about write speed, without any read consideration. So we have to think about how you will insert/update your document.
We have to compare (assuming you know the _id you are replacing, replace {key} by the key name, in your example log1-txt or log2-txt):
db.Col.update({ _id: '' }, { $set: { 'files.{key}': object }})
vs
db.Col.update({ _id: '', 'files.filename': '{key}'}, { $set: { 'files.$': object }})
The second one means that MongoDB have to browse the array, find the matching index and update it. The first one means MongoDB just update the specified field.
The worst:
The second command will not work if the matching filename is not present in the array! So you have to execute it, check if nMatched is 0, and create it if it is so. That's really bad write speed (see here MongoDB: upsert sub-document).
If you will never/almost never use read queries / aggregation framework on this collection: go for the first one, that will be faster. If you want to aggregate, unwind, do some analytics on the files you parsed to have statistics about file size and line numbers, you may consider using the second one, you will avoid some headache.
Pure write speed will be better with the first solution.

Parse - How do I query a Class and include another that points to it?

I have two classes - _User and Car. A _User will have a low/limited number of Cars that they own. Each Car has only ONE owner and thus an "owner" column that is a to the _User. When I got to the user's page, I want to see their _User info and all of their Cars. I would like to make one call, in Cloud Code if necessary.
Here is where I get confused. There are 3 ways I could do this -
In _User have a relationship column called "cars" that points to each individual Car. If so, how come I can't use the "include(cars)" function on a relation to include the Cars' data in my query?!!
_User.cars = relationship, Car.owner = _User(pointer)
Query the _User, and then query all Cars with (owner == _User.objectId) separately. This is two queries though.
_User.cars = null, Car.owner = _User(pointer)
In _User have a array of pointers column called "cars". Manually inject pointers to cars upon car creation. When querying the user I would use "include(cars)".
_User.cars = [Car(pointer)], Car.owner = _User(pointer)
What is your recommended way to do this and why? Which one is the fastest? The documentation just leaves me further confused.

I recommend you the 3rd option, and yes, you can ask to include an array. You even don't need to "manually inject" the pointers, you just need to add the objects into the array and they'll automatically be converted into pointers.

You've got the right ideas. Just to clarify them a bit:
A relation. User can have a relation column called cars. To get from user to car, there's a user query and then second query like user.relation("cars").query, on which you would .find().
What you might call a belongs_to pointer in Car. To get from user to car you'd have a query to get your user and you create a carQuery like carQuery.equalTo("user", user)
An array of pointers. For small-sized collections, this is superior to the relation, because you can aggressively load cars when querying user by saying include("cars") on a user query. Not sure if there's a second query under the covers - probably not if parse (mongo) is storing these as embedded.
But I wouldn't get too tied up over one or two queries. Using the promise forms of find() will keep your code nice and tidy. There probably is a small speed advantage to the array technique, which is good while the collection size is small (<100 is my rule of thumb).
It's easy to google (or I'll add here if you have a specific question) code examples for maintaining the relations and for getting from user->car or from car->user for each approach.

Equal Precedence View Collation CouchDB?

According to the view collation documentation for CouchDB(
http://wiki.apache.org/couchdb/View_collation), member order does matter for collation. I was wondering if there is a way to disable this attribute such that collation order does not matter? I want to be able to "search" my views such that the documents that are emitted satisfy all the key ranges for the field.
here is some more on view collation for your reference: CouchDB sorting and filtering in the same view
Likewise, if it is possible to set CouchDB such that order does not matter for view collation, the following parameters used for the GET request should only emit docs where doc.phone_number == "ZZZZZZZ" , whereas right now it emits the documents that fall within the range of the first 3 keys and completely ignores the last key. This occurs because the last key has the least precedence in the current collation scheme.
startkey: [null,null,null,"ZZZZZZZ"],
endkey: ["\ufff0","\ufff0","\ufff0","ZZZZZZZZ"],
Sample Mapping Function
var map = function(doc) {
/*
//Keys emitted
1. name
2. address
3. age
3. phone_number
*/
emit([doc.name,doc.address,doc.num_age,doc.phone_number],doc._id)
}
Is this possible, or do I have to create multiple views to perform this? The use of multiple views seems very inefficent.
I've read that CouchDB-Lucene:( How to realize complex search filters in couchdb? Should I avoid temporary views? )would be helpful for complex searching, but that doesn't seem applicable in this case.

Use of multiple views is not inefficient, quite to the contrary : having four views (name, address, age and phone number) will not use significantly more time or memory than having a single view emit everything. It is the simple, straightforward, efficient way of performing "WHERE field = value" queries in CouchDB.
If you are in fact looking for "WHERE field = value AND field2 = value2" queries, then CouchDB will not help you, and you will need to use Lucene.
You need to understand that the collation merely describes how keys are ordered. Even if you could specify any arbitrary collation, you will still have to deal with the fact that CouchDB need you to define an order for the keys, and only lets you query contiguous ranges of keys. This is not compatible with multi-dimensional range queries.