I thought I read that you can query subcollections with the new Firebase Firestore, but I don't see any examples. For example I have my Firestore setup in the following way:
Dances [collection]
danceName
Songs [collection]
songName
How would I be able to query "Find all dances where songName == 'X'"
Update 2019-05-07
Today we released collection group queries, and these allow you to query across subcollections.
So, for example in the web SDK:
db.collectionGroup('Songs')
.where('songName', '==', 'X')
.get()
This would match documents in any collection where the last part of the collection path is 'Songs'.
Your original question was about finding dances where songName == 'X', and this still isn't possible directly, however, for each Song that matched you can load its parent.
Original answer
This is a feature which does not yet exist. It's called a "collection group query" and would allow you query all songs regardless of which dance contained them. This is something we intend to support but don't have a concrete timeline on when it's coming.
The alternative structure at this point is to make songs a top-level collection and make which dance the song is a part of a property of the song.
UPDATE
Now Firestore supports array-contains
Having these documents
{danceName: 'Danca name 1', songName: ['Title1','Title2']}
{danceName: 'Danca name 2', songName: ['Title3']}
do it this way
collection("Dances")
.where("songName", "array-contains", "Title1")
.get()...
#Nelson.b.austin Since firestore does not have that yet, I suggest you to have a flat structure, meaning:
Dances = {
danceName: 'Dance name 1',
songName_Title1: true,
songName_Title2: true,
songName_Title3: false
}
Having it in that way, you can get it done:
var songTitle = 'Title1';
var dances = db.collection("Dances");
var query = dances.where("songName_"+songTitle, "==", true);
I hope this helps.
UPDATE 2019
Firestore have released Collection Group Queries. See Gil's answer above or the official Collection Group Query Documentation
Previous Answer
As stated by Gil Gilbert, it seems as if collection group queries is currently in the works. In the mean time it is probably better to use root level collections and just link between these collection using the document UID's.
For those who don't already know, Jeff Delaney has some incredible guides and resources for anyone working with Firebase (and Angular) on AngularFirebase.
Firestore NoSQL Relational Data Modeling - Here he breaks down the basics of NoSQL and Firestore DB structuring
Advanced Data Modeling With Firestore by Example - These are more advanced techniques to keep in the back of your mind. A great read for those wanting to take their Firestore skills to the next level
What if you store songs as an object instead of as a collection? Each dance as, with songs as a field: type Object (not a collection)
{
danceName: "My Dance",
songs: {
"aNameOfASong": true,
"aNameOfAnotherSong": true,
}
}
then you could query for all dances with aNameOfASong:
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
NEW UPDATE July 8, 2019:
db.collectionGroup('Songs')
.where('songName', isEqualTo:'X')
.get()
I have found a solution.
Please check this.
var museums = Firestore.instance.collectionGroup('Songs').where('songName', isEqualTo: "X");
museums.getDocuments().then((querySnapshot) {
setState(() {
songCounts= querySnapshot.documents.length.toString();
});
});
And then you can see Data, Rules, Indexes, Usage tabs in your cloud firestore from console.firebase.google.com.
Finally, you should set indexes in the indexes tab.
Fill in collection ID and some field value here.
Then Select the collection group option.
Enjoy it. Thanks
You can always search like this:-
this.key$ = new BehaviorSubject(null);
return this.key$.switchMap(key =>
this.angFirestore
.collection("dances").doc("danceName").collections("songs", ref =>
ref
.where("songName", "==", X)
)
.snapshotChanges()
.map(actions => {
if (actions.toString()) {
return actions.map(a => {
const data = a.payload.doc.data() as Dance;
const id = a.payload.doc.id;
return { id, ...data };
});
} else {
return false;
}
})
);
Query limitations
Cloud Firestore does not support the following types of queries:
Queries with range filters on different fields.
Single queries across multiple collections or subcollections. Each query runs against a single collection of documents. For more
information about how your data structure affects your queries, see
Choose a Data Structure.
Logical OR queries. In this case, you should create a separate query for each OR condition and merge the query results in your app.
Queries with a != clause. In this case, you should split the query into a greater-than query and a less-than query. For example, although
the query clause where("age", "!=", "30") is not supported, you can
get the same result set by combining two queries, one with the clause
where("age", "<", "30") and one with the clause where("age", ">", 30).
I'm working with Observables here and the AngularFire wrapper but here's how I managed to do that.
It's kind of crazy, I'm still learning about observables and I possibly overdid it. But it was a nice exercise.
Some explanation (not an RxJS expert):
songId$ is an observable that will emit ids
dance$ is an observable that reads that id and then gets only the first value.
it then queries the collectionGroup of all songs to find all instances of it.
Based on the instances it traverses to the parent Dances and get their ids.
Now that we have all the Dance ids we need to query them to get their data. But I wanted it to perform well so instead of querying one by one I batch them in buckets of 10 (the maximum angular will take for an in query.
We end up with N buckets and need to do N queries on firestore to get their values.
once we do the queries on firestore we still need to actually parse the data from that.
and finally we can merge all the query results to get a single array with all the Dances in it.
type Song = {id: string, name: string};
type Dance = {id: string, name: string, songs: Song[]};
const songId$: Observable<Song> = new Observable();
const dance$ = songId$.pipe(
take(1), // Only take 1 song name
switchMap( v =>
// Query across collectionGroup to get all instances.
this.db.collectionGroup('songs', ref =>
ref.where('id', '==', v.id)).get()
),
switchMap( v => {
// map the Song to the parent Dance, return the Dance ids
const obs: string[] = [];
v.docs.forEach(docRef => {
// We invoke parent twice to go from doc->collection->doc
obs.push(docRef.ref.parent.parent.id);
});
// Because we return an array here this one emit becomes N
return obs;
}),
// Firebase IN support up to 10 values so we partition the data to query the Dances
bufferCount(10),
mergeMap( v => { // query every partition in parallel
return this.db.collection('dances', ref => {
return ref.where( firebase.firestore.FieldPath.documentId(), 'in', v);
}).get();
}),
switchMap( v => {
// Almost there now just need to extract the data from the QuerySnapshots
const obs: Dance[] = [];
v.docs.forEach(docRef => {
obs.push({
...docRef.data(),
id: docRef.id
} as Dance);
});
return of(obs);
}),
// And finally we reduce the docs fetched into a single array.
reduce((acc, value) => acc.concat(value), []),
);
const parentDances = await dance$.toPromise();
I copy pasted my code and changed the variable names to yours, not sure if there are any errors, but it worked fine for me. Let me know if you find any errors or can suggest a better way to test it with maybe some mock firestore.
var songs = []
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
var songLength = querySnapshot.size
var i=0;
querySnapshot.forEach(function(doc) {
songs.push(doc.data())
i ++;
if(songLength===i){
console.log(songs
}
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
It could be better to use a flat data structure.
The docs specify the pros and cons of different data structures on this page.
Specifically about the limitations of structures with sub-collections:
You can't easily delete subcollections, or perform compound queries across subcollections.
Contrasted with the purported advantages of a flat data structure:
Root-level collections offer the most flexibility and scalability, along with powerful querying within each collection.
I have a doc in couchDB:
{
"id":"avc",
"type":"Property",
"username":"user1",
"password":"password1",
"server":"localhost"
}
I want to write a view that returns a map of all these fields.
The map should look like this: [{"username","user1"},{"password","password1"},{"server","localhost"}]
Here's pseudocode of what I want -
HashMap<String,String> getProperties()
{
HashMap<String,String> propMap;
if (doc.type == 'Property')
{
//read all fields in doc one by one
//get value and add field/value to the map
}
return propMap;
}
I am not sure how to do the portion that I have commented above. Please help.
Note: right now, I want to add username, password and server fields and their values in the map. However, I might keep adding more later on. I want to make sure what I do is extensible.
I considered writing a separate view function for each field. Ex: emit("username",doc.username).
But this may not be the best way to do this. Also needs updates every time I add a new field.
First of all, you have to know:
In CouchDB, you'll index documents inside a view with a key-value pair. So if you index the property username and server, you'll have the following view:
[
{"key": "user1", "value": null},
{"key": "localhost", "value": null}
]
Whenever you edit a view, it invalidates the index so Couch has to rebuild the index. If you were to add new fields to that view, that's something you have to take into account.
If you want to query multiple fields in the same query, all those fields must be in the same view. If it's not a requirement, then you could easily build an index for every field you want.
If you want to index multiple fields in the same view, you could do something like this:
// We define a map function as a function which take a single parameter: The document to index.
(doc) => {
// We iterate over a list of fields to index
["username", "password", "server"].forEach((key, value) => {
// If the document has the field to index, we index it.
if (doc.hasOwnProperty(key)) {
// map(key,value) is the function you call to index your document.
// You don't need to pass a value as you'll be able to get the macthing document by using include_docs=true
map(doc[key], null);
}
});
};
Also, note that Apache Lucene allows to make full-text search and might fit better your needs.
I am new to Firestore, need some help on firestore update.
I have following structure and wants to update "employee name" property. Not sure how to select and update.
Department:[
Name: Accounts
Employee:[
{Name :David,
Age :25},
{Name:Paul,
Age:27}
]
]
Here is what I was trying to do:
let depempCollectionRef = admin.firestore().collection('DepEmployee').doc('depempid')
depempCollectionRef.Department.Employee
.update({ name: 'Scott' },{merge:true})
.then(function() { console.log("Document successfully updated!"); })
Employee is just an embedded data structure in your Firestore document – so you can't address it through the reference directly. As far as Firestore is concerned, Employee is just an attribute on the Department document as Name is.
Before I propose a solution, let me point out two things:
If using update, you don't need {merge: true}. You use {merge: true} together with set to get an update-like behavior, if the document already exists.
I wouldn't use an Array of employees. It might make more sense to store the employees in their own collection in Firestore and then just list their reference IDs (= foreign keys) here. As a general rule of thumb: try to keep your data structure flat. Also use Arrays only, if you need to maintain a certain order of items.
A) If you have a separate collection for employees, updating the name is as easy as:
employeeCollection.doc('101').update({name: 'Scott'})
B) If you want to store employee data within your department document, I would still store them as a map with IDs (instead of an Array) and then access them with dot notation:
Department:[
Name: Accounts
Employees:{
101: {
Name :David,
Age :25
},
102: {
Name:Paul,
Age:27
}
}
]
depempCollectionRef.Department
.set({ ['101.name']: 'Scott' }, {merge:true})
C) And if you really want to store the data embedded in an Array, I believe you have to read and update the whole Array (not sure, if there is a better solution):
const employeesSnap = await depempCollectionRef.Department.get()
const updatedEmployees = changeNameOfScottInArray()
depempCollectionRef.Department
.update({ 'Employees': updatedEmployees })
I didn't test this code, but I hope you get the gist of it!
I'd recommend you flatten your data structure by creating a separate Employee collection and then just referencing them by their foreign key in your department (solution A).
I've looking for an answer for like 5 five hours straight, hope somebody can help. I have a MongoDb collection results (I'm using mLab) which looks like this:
{
"user":"5818be9c74aaec1824c28626"
"results":[{
"game_id":14578,
"level1":-1,
"level2":-1,
"level3":-1
},
{ ....
}],
{ "user":....
}
}
"user" is a MongoID I save in a previous part of the code, "results" is a record of scores. When an user does a new score, I have to update the score of the corresponding level (I'm using NodeJS).
This is one of the things I've tried so far.
app.get('/levelCompleted/:id/:time', function (request, response) {
var id = request.params.id;
var time = parseInt(request.params.time);
var u= game.getUserById(id);
var k = "results.$.level"+(u.level);
//I build the key to update dinamycally
dbM.collection("results").update(
{user:id,
"results.game_id":u.game_id
//u has its own game_id
},
{$set: {k:time}}
);
...
response.send(...);
});
I've checked the content of every variable and parameter, tried also using $elemMatch and dot notation, set upsert and multi, with no results. I've used an identical command on mongo shell and it has work on the first try.
Update with Mongo Shell
If someone could tell me what I'm doing wrong or point me in the right direction, it would be great.
Thanks
When you use a MongoId as a field in a MongoDB, you can't just pass a string with the id to do the query, you have to identify that string as an ObjectId (Id type in Mongo). Just add a new require in your node.js file.
var ObjectID = require("mongodb").ObjectID;
And use the imported constructor in your update request.
dbM.collection("results").update(
{user:ObjectID(id),...
...
}
userLink_titles = Entries.find({ _id:"bxSbMgszYxbCqDonF"})
returns:
docs: Object
CBqHrJvTE8xz7u2Rz: Object
_id: "CBqHrJvTE8xz7u2Rz"
author: "AHSwfYgeGmur9oHzu"
Q8m7PMbQr62E3A73f: Object
_id: "Q8m7PMbQr62E3A73f"
author: "AHSwfYgeGmur9oHzu"
bxSbMgszYxbCqDonF: Object
_id: "bxSbMgszYxbCqDonF"
author: "AHSwfYgeGmur9oHzu"
As you can see it returns the correct document but it also returns incorrect documents.
findOne: userLink_titles = Entries.findOne({ _id:"bxSbMgszYxbCqDonF"}) works as expected and only returns the correct document.
I would use the findOne except that the end intention is to make the _id selector an array such that find would return the documents for all documents that match the _id selectors in the array.
Bonus Points:
Let's say I wanted to retrieve the titles of a set of articles, where the references to those articles (the _ids of the Articles in their Articles collection) have been saved in a user collection.
So effectively I would retrieve the Article references from the user collection, and use those references to retrieve the titles of Articles from the Articles collection.
What would the code/pseudo code look like for that? Something like the following (I am assuming the below is a complete bastardization of some best practices for querying/retrieving records)
user_profile = Users.findOne({username : "Frank"});
user_saved_articles_ids = user_profile.findOne({saved_articles_ids});
userLinks = Articles.find({ _id:user_saved_articles_ids});
userLinksTitles = Articles.find({titles});
For those interested in the bonus question, you want MongoDB's $in operator.
If this will be used in a publish method on the server then you may want to check out publish-with-relations.
Something like the function below would be an efficient way to retrieve the information. It makes two calls to the database. Using the fields parameter prevents unnecessary data from being sent between the db and the webserver.
/**
* Titles of a User's saved articles
*
* #method getSavedTitles
* #param {String} username
* #return {Array} a list of article titles
*/
function getSavedTitles (username) {
var user,
ids,
linkedArticles,
titles;
user = Users.findOne({username: username},
{fields:{ _id:0, profile:1 }});
if (!user || !user.profile) {
throw new Meteor.Error(500, 'Missing user profile');
}
ids = user.profile.savedArticleIds;
if (!ids || !_.isArray(ids)) {
throw new Meteor.Error(500, 'Missing saved article ids');
}
linkedArticles = Articles.find({_id: {$in: ids}},
{fields:{ _id:0, title:1 }});
titles = _.pluck(linkedArticles.fetch(), "title");
return titles;
} // end getSavedTitles
If you want to inspect the results of a query in the console, use Entries.find(...).fetch()
Entries.find() returns a cursor, which is a construct used to render lists efficiently, so that only the entries that change need to be re-rendered. Returning cursors from helpers on which you use {{#each}} will lead to more responsive apps.