I'm trying to get the lastest records in a database. I have a field with their category. I'm trying to get 20 records, 5 of the latest post in each category. What I mean is that it should return 20 latest records but 5 of each category since the latest 20 does not necessarily mean a balanced 5 of each category. Basically what I have below, but I feel there's a better way and wasn't able to see it from reading the Sequelize docs. Thanks guys, I really appreciate it! Pardon the pseudocode. I have 4 categories. and im actually also sorting by createdAt timestamp field with limits and attributes and other checks/ error handling that I have not included for the sake of readability.
Posts.findAll({ where: { category: "Tech"}})
.then(techPosts => {
Posts.findAll({where:{category: "Science"},})
.then(sciencePosts => {
//actually 2 more nested findAlls before sending
const posts = [...techPosts, ...sciencePosts]
res.status(200).json(posts).end();
})
})
I would suggest fetching 5 different categories at the same time using "Promise.all".
The code would look like this:
const fetchTechPosts = () => {
return Posts.findAll({ where: { category: "Tech"}})
}
const fetchSciencePosts = () => {
return Posts.findAll({ where: { category: "Science"}})
}
const promises = Promise.all([fetchTech, fetchScience])
promises.then(posts => {
res.status(200).json(posts).end()
})
I'm currently trying Firestore, and I'm stuck at something very simple: "updating an array (aka a subdocument)".
My DB structure is super simple. For example:
proprietary: "John Doe",
sharedWith:
[
{who: "first#test.com", when:timestamp},
{who: "another#test.com", when:timestamp},
],
I'm trying (without success) to push new records into shareWith array of objects.
I've tried:
// With SET
firebase.firestore()
.collection('proprietary')
.doc(docID)
.set(
{ sharedWith: [{ who: "third#test.com", when: new Date() }] },
{ merge: true }
)
// With UPDATE
firebase.firestore()
.collection('proprietary')
.doc(docID)
.update({ sharedWith: [{ who: "third#test.com", when: new Date() }] })
None works. These queries overwrite my array.
The answer might be simple, but I could'nt find it...
Firestore now has two functions that allow you to update an array without re-writing the entire thing.
Link: https://firebase.google.com/docs/firestore/manage-data/add-data, specifically https://firebase.google.com/docs/firestore/manage-data/add-data#update_elements_in_an_array
Update elements in an array
If your document contains an array field, you can use arrayUnion() and
arrayRemove() to add and remove elements. arrayUnion() adds elements
to an array but only elements not already present. arrayRemove()
removes all instances of each given element.
Edit 08/13/2018: There is now support for native array operations in Cloud Firestore. See Doug's answer below.
There is currently no way to update a single array element (or add/remove a single element) in Cloud Firestore.
This code here:
firebase.firestore()
.collection('proprietary')
.doc(docID)
.set(
{ sharedWith: [{ who: "third#test.com", when: new Date() }] },
{ merge: true }
)
This says to set the document at proprietary/docID such that sharedWith = [{ who: "third#test.com", when: new Date() } but to not affect any existing document properties. It's very similar to the update() call you provided however the set() call with create the document if it does not exist while the update() call will fail.
So you have two options to achieve what you want.
Option 1 - Set the whole array
Call set() with the entire contents of the array, which will require reading the current data from the DB first. If you're concerned about concurrent updates you can do all of this in a transaction.
Option 2 - Use a subcollection
You could make sharedWith a subcollection of the main document. Then
adding a single item would look like this:
firebase.firestore()
.collection('proprietary')
.doc(docID)
.collection('sharedWith')
.add({ who: "third#test.com", when: new Date() })
Of course this comes with new limitations. You would not be able to query
documents based on who they are shared with, nor would you be able to
get the doc and all of the sharedWith data in a single operation.
Here is the latest example from the Firestore documentation:
firebase.firestore.FieldValue.ArrayUnion
var washingtonRef = db.collection("cities").doc("DC");
// Atomically add a new region to the "regions" array field.
washingtonRef.update({
regions: firebase.firestore.FieldValue.arrayUnion("greater_virginia")
});
// Atomically remove a region from the "regions" array field.
washingtonRef.update({
regions: firebase.firestore.FieldValue.arrayRemove("east_coast")
});
You can use a transaction (https://firebase.google.com/docs/firestore/manage-data/transactions) to get the array, push onto it and then update the document:
const booking = { some: "data" };
const userRef = this.db.collection("users").doc(userId);
this.db.runTransaction(transaction => {
// This code may get re-run multiple times if there are conflicts.
return transaction.get(userRef).then(doc => {
if (!doc.data().bookings) {
transaction.set({
bookings: [booking]
});
} else {
const bookings = doc.data().bookings;
bookings.push(booking);
transaction.update(userRef, { bookings: bookings });
}
});
}).then(function () {
console.log("Transaction successfully committed!");
}).catch(function (error) {
console.log("Transaction failed: ", error);
});
Sorry Late to party but Firestore solved it way back in aug 2018 so If you still looking for that here it is all issues solved with regards to arrays.
https://firebase.googleblog.com/2018/08/better-arrays-in-cloud-firestore.htmlOfficial blog post
array-contains, arrayRemove, arrayUnion for checking, removing and updating arrays. Hope it helps.
To build on Sam Stern's answer, there is also a 3rd option which made things easier for me and that is using what Google call a Map, which is essentially a dictionary.
I think a dictionary is far better for the use case you're describing. I usually use arrays for stuff that isn't really updated too much, so they are more or less static. But for stuff that gets written a lot, specifically values that need to be updated for fields that are linked to something else in the database, dictionaries prove to be much easier to maintain and work with.
So for your specific case, the DB structure would look like this:
proprietary: "John Doe"
sharedWith:{
whoEmail1: {when: timestamp},
whoEmail2: {when: timestamp}
}
This will allow you to do the following:
var whoEmail = 'first#test.com';
var sharedObject = {};
sharedObject['sharedWith.' + whoEmail + '.when'] = new Date();
sharedObject['merge'] = true;
firebase.firestore()
.collection('proprietary')
.doc(docID)
.update(sharedObject);
The reason for defining the object as a variable is that using 'sharedWith.' + whoEmail + '.when' directly in the set method will result in an error, at least when using it in a Node.js cloud function.
#Edit (add explanation :) )
say you have an array you want to update your existing firestore document field with. You can use set(yourData, {merge: true} ) passing setOptions(second param in set function) with {merge: true} is must in order to merge the changes instead of overwriting. here is what the official documentation says about it
An options object that configures the behavior of set() calls in DocumentReference, WriteBatch, and Transaction. These calls can be configured to perform granular merges instead of overwriting the target documents in their entirety by providing a SetOptions with merge: true.
you can use this
const yourNewArray = [{who: "first#test.com", when:timestamp}
{who: "another#test.com", when:timestamp}]
collectionRef.doc(docId).set(
{
proprietary: "jhon",
sharedWith: firebase.firestore.FieldValue.arrayUnion(...yourNewArray),
},
{ merge: true },
);
hope this helps :)
addToCart(docId: string, prodId: string): Promise<void> {
return this.baseAngularFirestore.collection('carts').doc(docId).update({
products:
firestore.FieldValue.arrayUnion({
productId: prodId,
qty: 1
}),
});
}
i know this is really old, but to help people newbies with the issue
firebase V9 provides a solution using the arrayUnion and arrayRemove
await updateDoc(documentRef, {
proprietary: arrayUnion( { sharedWith: [{ who: "third#test.com", when: new Date() }] }
});
check this out for more explanation
Other than the answers mentioned above. This will do it.
Using Angular 5 and AngularFire2. or use firebase.firestore() instead of this.afs
// say you have have the following object and
// database structure as you mentioned in your post
data = { who: "third#test.com", when: new Date() };
...othercode
addSharedWith(data) {
const postDocRef = this.afs.collection('posts').doc('docID');
postDocRef.subscribe( post => {
// Grab the existing sharedWith Array
// If post.sharedWith doesn`t exsit initiated with empty array
const foo = { 'sharedWith' : post.sharedWith || []};
// Grab the existing sharedWith Array
foo['sharedWith'].push(data);
// pass updated to fireStore
postsDocRef.update(foo);
// using .set() will overwrite everything
// .update will only update existing values,
// so we initiated sharedWith with empty array
});
}
We can use arrayUnion({}) method to achive this.
Try this:
collectionRef.doc(ID).update({
sharedWith: admin.firestore.FieldValue.arrayUnion({
who: "first#test.com",
when: new Date()
})
});
Documentation can find here: https://firebase.google.com/docs/firestore/manage-data/add-data#update_elements_in_an_array
Consider John Doe a document rather than a collection
Give it a collection of things and thingsSharedWithOthers
Then you can map and query John Doe's shared things in that parallel thingsSharedWithOthers collection.
proprietary: "John Doe"(a document)
things(collection of John's things documents)
thingsSharedWithOthers(collection of John's things being shared with others):
[thingId]:
{who: "first#test.com", when:timestamp}
{who: "another#test.com", when:timestamp}
then set thingsSharedWithOthers
firebase.firestore()
.collection('thingsSharedWithOthers')
.set(
{ [thingId]:{ who: "third#test.com", when: new Date() } },
{ merge: true }
)
If You want to Update an array in a firebase document.
You can do this.
var documentRef = db.collection("Your collection name").doc("Your doc name")
documentRef.update({
yourArrayName: firebase.firestore.FieldValue.arrayUnion("The Value you want to enter")});
Although firebase.firestore.FieldValue.arrayUnion() provides the solution for array update in firestore, at the same time it is required to use {merge:true}. If you do not use {merge:true} it will delete all other fields in the document while updating with the new value. Here is the working code for updating array without loosing data in the reference document with .set() method:
const docRef = firebase.firestore().collection("your_collection_name").doc("your_doc_id");
docRef.set({yourArrayField: firebase.firestore.FieldValue.arrayUnion("value_to_add")}, {merge:true});
If anybody is looking for Java firestore sdk solution to add items in array field:
List<String> list = java.util.Arrays.asList("A", "B");
Object[] fieldsToUpdate = list.toArray();
DocumentReference docRef = getCollection().document("docId");
docRef.update(fieldName, FieldValue.arrayUnion(fieldsToUpdate));
To delete items from array user: FieldValue.arrayRemove()
If the document contains a nested object in the form of an array, .dot notation can be used to reference and update nested fields.
Node.js example:
const users = {
name: 'Tom',
surname: 'Smith',
favorites: {
sport: 'tennis',
color: 'red',
subject: 'math'
}
};
const update = await db.collection('users').doc('Tom').update({
'favorites.sport': 'snowboard'
});
or Android sdk example:
db.collection("users").document("Tom")
.update(
'favorites.sport': 'snowboard'
);
There is a simple hack in firestore:
use path with "." as property name:
propertyname.arraysubname.${id}:
db.collection("collection")
.doc("docId")
.update({arrayOfObj: fieldValue.arrayUnion({...item})})
I'm creating a StencilJS app (no framework) with a Google Firestore backend, and I want to use the RxFire and RxJS libraries as much as possible to simplify data access code. How can I combine into a single observable stream data coming from two different collections that use a reference ID?
There are several examples online that I've read through and tried, each one using a different combination of operators with a different level of nested complexity. https://www.learnrxjs.io/ seems like a good resource, but it does not provide line-of-business examples that make sense to me. This question is very similar, and maybe the only difference is some translation into using RxFire? Still looking at that. Just for comparison, in SQL this would be a SELECT statement with an INNER JOIN on the reference ID.
Specifically, I have a collection for Games:
{ id: "abc000001", name: "Billiards" },
{ id: "abc000002", name: "Croquet" },
...
and a collection for Game Sessions:
{ id: "xyz000001", userId: "usr000001", gameId: "abc000001", duration: 30 },
{ id: "xyz000002", userId: "usr000001", gameId: "abc000001", duration: 45 },
{ id: "xyz000003", userId: "usr000001", gameId: "abc000002", duration: 55 },
...
And I want to observe a merged collection of Game Sessions where gameId is essentially replace with Game.name.
I current have a game-sessions-service.ts with a function to get sessions for a particular user:
import { collectionData } from 'rxfire/firestore';
import { Observable } from 'rxjs';
import { GameSession } from '../interfaces';
observeUserGameSesssions(userId: string): Observable<GameSession[]> {
let collectionRef = this.db.collection('game-sessions');
let query = collectionRef.where('userId', '==', userId);
return collectionData(query, 'id);
}
And I've tried variations of things with pipe and mergeMap, but I don't understand how to make them all fit together properly. I would like to establish an interface GameSessionView to represent the merged data:
export interface GameSessionView {
id: string,
userId: string,
gameName: string,
duration: number
}
observeUserGameSessionViews(userId: string): Observable<GameSessionView> {
this.observeUserGameSessions(userId)
.pipe(
mergeMap(sessions => {
// What do I do here? Iterate over sessions
// and embed other observables for each document?
}
)
}
Possibly, I'm just stuck in a normalized way of thinking, so I'm open to suggestions on better ways to manage the data. I just don't want too much duplication to keep synchronized.
You can use the following code (also available as Stackblitz):
const games: Game[] = [...];
const gameSessions: GameSession[] = [...];
combineLatest(
of(games),
of(gameSessions)
).pipe(
switchMap(results => {
const [gamesRes, gameSessionsRes] = results;
const gameSessionViews: GameSessionView[] = gameSessionsRes.map(gameSession => ({
id: gameSession.id,
userId: gameSession.userId,
gameName: gamesRes.find(game => game.id === gameSession.gameId).name,
duration: gameSession.duration
}));
return of(gameSessionViews);
})
).subscribe(mergedData => console.log(mergedData));
Explanation:
With combineLatest you can combine the latest values from a number of Obervables. It can be used if you have "multiple (..) observables that rely on eachother for some calculation or determination".
So assuming you lists of Games and GameSessions are Observables, you can combine the values of each list.
Within the switchMap you create new objects of type GameSessionView by iterating over your GameSessions, use the attributes id, userId and duration and find the value for gameName within the second list of Games by gameId. Mind that there is no error handling in this example.
As switchMap expects that you return another Observable, the merged list will be returned with of(gameSessionViews).
Finally, you can subscribe to this process and see the expected result.
For sure this is not the only way you can do it, but I find it the simplest one.
If a collection have a list of dogs, and there is duplicate entries on some races. How do i remove all, but a single specific/non specific one, from just one query?
I guess it would be possible to get all from a Model.find(), loop through every index except the first one and call Model.remove(), but I would rather have the database handle the logic through the query. How would this be possible?
pseudocode example of what i want:
Model.remove({race:"pitbull"}).where(notFirstOne);
To remove all but one, you need a way to get all the filtered documents, group them by the identifier, create a list of ids for the group and remove a single id from
this list. Armed with this info, you can then run another operation to remove the documents with those ids. Essentially you will be running two queries.
The first query is an aggregate operation that aims to get the list of ids with the potentially nuking documents:
(async () => {
// Get the duplicate entries minus 1
const [doc, ...rest] = await Module.aggregate([
{ '$match': { 'race': 'pitbull'} },
{ '$group': {
'_id': '$race',
'ids': { '$push': '$_id' },
'id': { '$first': '$_id' }
} },
{ '$project': { 'idsToRemove': { '$setDifference': [ ['$id'], '$ids' ] } } }
]);
const { idsToRemove } = doc;
// Remove the duplicate documents
Module.remove({ '_id': { '$in': idsToRemove } })
})();
if purpose is to keep only one, in case of concurrent writes, may as well just write
Module.findOne({race:'pitbull'}).select('_id')
//bla
Module.remove({race:'pitbull', _id:{$ne:idReturned}})
If it is to keep the very first one, mongodb does not guarantee results will be sorted by increasing _id (natural order refers to disk)
see Does default find() implicitly sort by _id?
so instead
Module.find({race:'pitbull'}).sort({_id:1}).limit(1)
I have two nodes in my database - one which is allUsers and one which is usersChildren.
For example:
allUsers: { user1: {...}, user2: {...}}
usersChildren: { user1: {...} }
In this case user1 has children data and user2 does not.
I want to retrieve a list of all user objects, and inside each user's object I wish to add the children data from the usersChildren node(if there is one).
However, I am not really familiar with how I can do that. I have tried the following but this results in obtaining only the children information and not
a combined object with both the children information and the user meta data.
this.af.getObservable(`allUsers`).pipe(map(allUsers =>
allUsers.map(user => this.af.getObservable(`usersChildren/${user.id}`)))
.subscribe(allUsersData => this.userList = allUsersData);
What is the best way to achieve what I desire?
Try this
this.af.getObservable(`allUsers`)
.pipe(
mergeMap(users => forkJoin(
users.map(user => this.af.getObservable(`usersChildren/${user.id}`))
))
);
You make an HTTP call to get all the users, then you use forkJoin to make 1 call per user. The call is made thank to Array.map, which transforms your user into an HTTP call.
Now you can subscribe to it like this
this.myService.getAllUsers().subscribe(users => { console.log(users); });
I would use concat operator in your case.
Concat will combine two observables into a combined sequence, but the second observable will not start emitting until the first one has completed.
Example:
let first = Observable.timer(10,500).map(r => {
return {source:1,value:r};
}).take(4);
let second = Observable.timer(10,500).map(r => {
return {source:2,value:r};
}).take(4);
first.concat(second).subscribe(res => this.concatStream.push(res));