Develop Reference

MongoDB/Mongoose: AddToSet an array into an array field, but also addToSet a nested array field value if the parent object already exists

I am building an education application and I am trying to add/update a field which is an array of objects with addToSet from a javascript array, and if the object already exists (matched with objectId) I want to update the already existing object's array (addToSet) and change another field of that same object.
My model looks like this (simplified):
const course = new Schema(
{
events: [
{
type: Schema.Types.ObjectId,
ref: 'event'
}
],
students: [
{
user: {
type: Schema.Types.ObjectId,
ref: 'user'
},
status: {
type: String,
enum: ['notBooked', 'booked', 'attended', 'completed']
},
events: [
{
type: Schema.Types.ObjectId,
ref: 'event'
}
]
}
],
});
And ideally I would like to use an updateOne query to both addToSet to the course's list of events, while also updating the students list.
Currently I am using this code to accomplish my updates by first finding the course and then using javascript to update it, which works:
const studentsToAdd = this.attendees.map((attendee) => ({
user: attendee._id,
status: 'booked',
events: [this._id]
}));
const studentIds = studentsToAdd.map((student) => student.user);
const course = await courseModel.findById(this.course);
console.log(studentIds);
course.events.push(this._id);
course.students.forEach((student) => {
if (studentIds.some((s) => s.equals(student.user))) {
student.events.push(this._id);
student.status = 'booked';
studentsToAdd.splice(studentsToAdd.indexOf(studentsToAdd.find((s) => s.user.equals(student.user))), 1);
}
});
course.students.push(...studentsToAdd);
course.save();
However I am curious if it is possible to solve this using a single updateOne on the courseModel schema.
This is part of a mongoose middleware function, and "this" references an event object, which has its own _id, attendees of the event, and the course ID (named course).
The goal is to add the student object part of studentsToAdd to the students array of the course IF the student does not exist (signified by user being a reference by objectId), and if the student already exists then I want to add the eventId (this._id) to the events array for that particular student and set status for that student to "booked".
Is this possible? I have tried many iterations using $cond, $elemmatch, "students.$[]" and so forth but I am quite new to mongodb and am unsure how to go about this.

How can I update the value of an item in an array in firebase? [duplicate]

I'm currently trying Firestore, and I'm stuck at something very simple: "updating an array (aka a subdocument)".
My DB structure is super simple. For example:
proprietary: "John Doe",
sharedWith:
[
{who: "first#test.com", when:timestamp},
{who: "another#test.com", when:timestamp},
],
I'm trying (without success) to push new records into shareWith array of objects.
I've tried:
// With SET
firebase.firestore()
.collection('proprietary')
.doc(docID)
.set(
{ sharedWith: [{ who: "third#test.com", when: new Date() }] },
{ merge: true }
)
// With UPDATE
firebase.firestore()
.collection('proprietary')
.doc(docID)
.update({ sharedWith: [{ who: "third#test.com", when: new Date() }] })
None works. These queries overwrite my array.
The answer might be simple, but I could'nt find it...

Firestore now has two functions that allow you to update an array without re-writing the entire thing.
Link: https://firebase.google.com/docs/firestore/manage-data/add-data, specifically https://firebase.google.com/docs/firestore/manage-data/add-data#update_elements_in_an_array
Update elements in an array
If your document contains an array field, you can use arrayUnion() and
arrayRemove() to add and remove elements. arrayUnion() adds elements
to an array but only elements not already present. arrayRemove()
removes all instances of each given element.

Edit 08/13/2018: There is now support for native array operations in Cloud Firestore. See Doug's answer below.
There is currently no way to update a single array element (or add/remove a single element) in Cloud Firestore.
This code here:
firebase.firestore()
.collection('proprietary')
.doc(docID)
.set(
{ sharedWith: [{ who: "third#test.com", when: new Date() }] },
{ merge: true }
)
This says to set the document at proprietary/docID such that sharedWith = [{ who: "third#test.com", when: new Date() } but to not affect any existing document properties. It's very similar to the update() call you provided however the set() call with create the document if it does not exist while the update() call will fail.
So you have two options to achieve what you want.
Option 1 - Set the whole array
Call set() with the entire contents of the array, which will require reading the current data from the DB first. If you're concerned about concurrent updates you can do all of this in a transaction.
Option 2 - Use a subcollection
You could make sharedWith a subcollection of the main document. Then
adding a single item would look like this:
firebase.firestore()
.collection('proprietary')
.doc(docID)
.collection('sharedWith')
.add({ who: "third#test.com", when: new Date() })
Of course this comes with new limitations. You would not be able to query
documents based on who they are shared with, nor would you be able to
get the doc and all of the sharedWith data in a single operation.

Here is the latest example from the Firestore documentation:
firebase.firestore.FieldValue.ArrayUnion
var washingtonRef = db.collection("cities").doc("DC");
// Atomically add a new region to the "regions" array field.
washingtonRef.update({
regions: firebase.firestore.FieldValue.arrayUnion("greater_virginia")
});
// Atomically remove a region from the "regions" array field.
washingtonRef.update({
regions: firebase.firestore.FieldValue.arrayRemove("east_coast")
});

You can use a transaction (https://firebase.google.com/docs/firestore/manage-data/transactions) to get the array, push onto it and then update the document:
const booking = { some: "data" };
const userRef = this.db.collection("users").doc(userId);
this.db.runTransaction(transaction => {
// This code may get re-run multiple times if there are conflicts.
return transaction.get(userRef).then(doc => {
if (!doc.data().bookings) {
transaction.set({
bookings: [booking]
});
} else {
const bookings = doc.data().bookings;
bookings.push(booking);
transaction.update(userRef, { bookings: bookings });
}
});
}).then(function () {
console.log("Transaction successfully committed!");
}).catch(function (error) {
console.log("Transaction failed: ", error);
});

Sorry Late to party but Firestore solved it way back in aug 2018 so If you still looking for that here it is all issues solved with regards to arrays.
https://firebase.googleblog.com/2018/08/better-arrays-in-cloud-firestore.htmlOfficial blog post
array-contains, arrayRemove, arrayUnion for checking, removing and updating arrays. Hope it helps.

To build on Sam Stern's answer, there is also a 3rd option which made things easier for me and that is using what Google call a Map, which is essentially a dictionary.
I think a dictionary is far better for the use case you're describing. I usually use arrays for stuff that isn't really updated too much, so they are more or less static. But for stuff that gets written a lot, specifically values that need to be updated for fields that are linked to something else in the database, dictionaries prove to be much easier to maintain and work with.
So for your specific case, the DB structure would look like this:
proprietary: "John Doe"
sharedWith:{
whoEmail1: {when: timestamp},
whoEmail2: {when: timestamp}
}
This will allow you to do the following:
var whoEmail = 'first#test.com';
var sharedObject = {};
sharedObject['sharedWith.' + whoEmail + '.when'] = new Date();
sharedObject['merge'] = true;
firebase.firestore()
.collection('proprietary')
.doc(docID)
.update(sharedObject);
The reason for defining the object as a variable is that using 'sharedWith.' + whoEmail + '.when' directly in the set method will result in an error, at least when using it in a Node.js cloud function.

#Edit (add explanation :) )
say you have an array you want to update your existing firestore document field with. You can use set(yourData, {merge: true} ) passing setOptions(second param in set function) with {merge: true} is must in order to merge the changes instead of overwriting. here is what the official documentation says about it
An options object that configures the behavior of set() calls in DocumentReference, WriteBatch, and Transaction. These calls can be configured to perform granular merges instead of overwriting the target documents in their entirety by providing a SetOptions with merge: true.
you can use this
const yourNewArray = [{who: "first#test.com", when:timestamp}
{who: "another#test.com", when:timestamp}]
collectionRef.doc(docId).set(
{
proprietary: "jhon",
sharedWith: firebase.firestore.FieldValue.arrayUnion(...yourNewArray),
},
{ merge: true },
);
hope this helps :)

addToCart(docId: string, prodId: string): Promise<void> {
return this.baseAngularFirestore.collection('carts').doc(docId).update({
products:
firestore.FieldValue.arrayUnion({
productId: prodId,
qty: 1
}),
});
}

i know this is really old, but to help people newbies with the issue
firebase V9 provides a solution using the arrayUnion and arrayRemove
await updateDoc(documentRef, {
proprietary: arrayUnion( { sharedWith: [{ who: "third#test.com", when: new Date() }] }
});
check this out for more explanation

Other than the answers mentioned above. This will do it.
Using Angular 5 and AngularFire2. or use firebase.firestore() instead of this.afs
// say you have have the following object and
// database structure as you mentioned in your post
data = { who: "third#test.com", when: new Date() };
...othercode
addSharedWith(data) {
const postDocRef = this.afs.collection('posts').doc('docID');
postDocRef.subscribe( post => {
// Grab the existing sharedWith Array
// If post.sharedWith doesn`t exsit initiated with empty array
const foo = { 'sharedWith' : post.sharedWith || []};
// Grab the existing sharedWith Array
foo['sharedWith'].push(data);
// pass updated to fireStore
postsDocRef.update(foo);
// using .set() will overwrite everything
// .update will only update existing values,
// so we initiated sharedWith with empty array
});
}

We can use arrayUnion({}) method to achive this.
Try this:
collectionRef.doc(ID).update({
sharedWith: admin.firestore.FieldValue.arrayUnion({
who: "first#test.com",
when: new Date()
})
});
Documentation can find here: https://firebase.google.com/docs/firestore/manage-data/add-data#update_elements_in_an_array

Consider John Doe a document rather than a collection
Give it a collection of things and thingsSharedWithOthers
Then you can map and query John Doe's shared things in that parallel thingsSharedWithOthers collection.
proprietary: "John Doe"(a document)
things(collection of John's things documents)
thingsSharedWithOthers(collection of John's things being shared with others):
[thingId]:
{who: "first#test.com", when:timestamp}
{who: "another#test.com", when:timestamp}
then set thingsSharedWithOthers
firebase.firestore()
.collection('thingsSharedWithOthers')
.set(
{ [thingId]:{ who: "third#test.com", when: new Date() } },
{ merge: true }
)

If You want to Update an array in a firebase document.
You can do this.
var documentRef = db.collection("Your collection name").doc("Your doc name")
documentRef.update({
yourArrayName: firebase.firestore.FieldValue.arrayUnion("The Value you want to enter")});

Although firebase.firestore.FieldValue.arrayUnion() provides the solution for array update in firestore, at the same time it is required to use {merge:true}. If you do not use {merge:true} it will delete all other fields in the document while updating with the new value. Here is the working code for updating array without loosing data in the reference document with .set() method:
const docRef = firebase.firestore().collection("your_collection_name").doc("your_doc_id");
docRef.set({yourArrayField: firebase.firestore.FieldValue.arrayUnion("value_to_add")}, {merge:true});

If anybody is looking for Java firestore sdk solution to add items in array field:
List<String> list = java.util.Arrays.asList("A", "B");
Object[] fieldsToUpdate = list.toArray();
DocumentReference docRef = getCollection().document("docId");
docRef.update(fieldName, FieldValue.arrayUnion(fieldsToUpdate));
To delete items from array user: FieldValue.arrayRemove()

If the document contains a nested object in the form of an array, .dot notation can be used to reference and update nested fields.
Node.js example:
const users = {
name: 'Tom',
surname: 'Smith',
favorites: {
sport: 'tennis',
color: 'red',
subject: 'math'
}
};
const update = await db.collection('users').doc('Tom').update({
'favorites.sport': 'snowboard'
});
or Android sdk example:
db.collection("users").document("Tom")
.update(
'favorites.sport': 'snowboard'
);

There is a simple hack in firestore:
use path with "." as property name:
propertyname.arraysubname.${id}:

db.collection("collection")
.doc("docId")
.update({arrayOfObj: fieldValue.arrayUnion({...item})})

How to update a document based on two filter conditions?

I have a document structure that looks like this:
type Document = {
_id: string
title: string
variants: VariantType[]
}
type VariantType = {
timestamp: Int
active: Boolean
content: any[]
}
I'm trying to filter a document based on two filter conditions in one query. First I want to match the _id and then find a specific variant based on a timestamp.
My previous version of the query was filtering based on the active key.
const updatedDocument = await allDocuments
.findOneAndUpdate({ _id: mongoId, 'variants.active': false }, .... };
Changing it to
const updatedDocument = await allDocuments
.findOneAndUpdate({ _id: mongoId, 'variants.timestamp': timestamp }, .... };
returns null.
Can mongo even match a document like this. I saw that there is an $eq query selector but I can't seem to get it working either.

Turns out I the problem was with the timestamp I was sending to the server. I was getting it from the wrong place and it wasn't matching any of the variants.

How to make Mongoose not insert empty array or object fields into a document

Let's say we have a Mongoose schema in our Node.js project:
let coolSchema = new mongoose.Schema({
field_1 : Number,
field_2 : String,
field_3 : [ String ],
});
And let's we have an according object:
var data = {
field_1 : 123,
field_2 : 'blah',
field_3 : ['aa', 'bb'],
};
Now to save this data into MongoDB we can use this code:
let Model = require('mongoose').model('CoolModel', coolSchema);
(new Model(data)).save();
Ok, while it's all cool.
But if data does not contain field_3 (array field, and the same will be for an object field) Mongoose will anyway add this field into the being created document with empty value.
Can we somehow tell Mongoose not to create this field if it's not contained in the data object?

you can do it easily skip the array field and array of object field.. This will let you skip saving empty array in new documents.but you have to use pre hook for this .
var brandSchema = new Schema({
name : {type:String},
email:String,
check:[]
})
brandSchema.pre('save', function (next) {
if (this.isNew && 0 === this.check.length) {
this.check = undefined;
}
next();
})
when new document is inserted in your schema you have to use this middlware.this works fine so try this.
this is the response when we want to insert any document
"data": {
"__v": 0,
"name": "testing",
"email": "testing#gmail.com",
"_id": "5915b018e292833edda8837f"
}
so i have send only email and name but check(array) field is skipped(Not send any value).

The accepted answer is good. But if you wouldn't want to use pre-hook, then you can add default: undefined to the array fields. For example:
var schema = new Schema({
myArr: { type: [String], default: undefined }
});
Refer to this comment for more explanation.

Not particularly an answer to the question itself but some thought on the matter.
It's not clear exactly what you're trying to achieve here. You defined a schema that is supposed to contain a list of string. Mongoose correctly does so that the data saved in your schema is consistent with the definition of the schema.
In this case, the list is more of a structural part of the schema. If you have different behaviour, you'd have to handle special case in your code in case the list isn't present. Now, you can safely assume that you schema is always returning a list so fetching some data will always allow you to do:
coolData.field_3.forEach(function(x) {
do_cool_things(x)
})
What you're asking is to make the schema allow inconsistent data being returned from mongodb... In other words, you'd have to do this in order to prevent accessing attributes on undefined:
if (coolData.field_3) {
coolData.field_3.forEach(function(x) {
do_cool_things(x)
})
}
Also, I you were trying to optimize the size of you objects/database, you could fill a bug report so mongoose doesn't define empty values while saving the objects and autofill them with defaults when the field is missing from mongodb. (I could be wrong but did you actually check if the data in mongodb was containing empty values or you were just looking at data coming from mongoose?)

It's because you're not marking the fields as required in your schema definition.
Do this:
let coolSchema = new mongoose.Schema({
field_1 : { type: Number, required: true },
field_2 : { type: String, required: true },
field_3 : { type: [ String ], required: true },
});

Mongoose find/update subdocument

I have the following schemas for the document Folder:
var permissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
var folderSchema = new Schema({
name: { type: string },
permissions: [ permissionSchema ]
});
So, for each Page I can have many permissions. In my CMS there's a panel where I list all the folders and their permissions. The admin can edit a single permission and save it.
I could easily save the whole Folder document with its permissions array, where only one permission was modified. But I don't want to save all the document (the real schema has much more fields) so I did this:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }, function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
but the problem is that perm is always undefined! I tried to "statically" fetch the permission in this way:
var perm = data.permissions[0];
and it works great, so the problem is that Underscore library is not able to query the permissions array. So I guess that there's a better (and workgin) way to get the subdocument of a fetched document.
Any idea?
P.S.: I solved checking each item in the data.permission array using a "for" loop and checking data.permissions[i]._id == permission._id but I'd like a smarter solution, I know there's one!

So as you note, the default in mongoose is that when you "embed" data in an array like this you get an _id value for each array entry as part of it's own sub-document properties. You can actually use this value in order to determine the index of the item which you intend to update. The MongoDB way of doing this is the positional $ operator variable, which holds the "matched" position in the array:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$": permission
}
},
function(err,doc) {
}
);
That .findOneAndUpdate() method will return the modified document or otherwise you can just use .update() as a method if you don't need the document returned. The main parts are "matching" the element of the array to update and "identifying" that match with the positional $ as mentioned earlier.
Then of course you are using the $set operator so that only the elements you specify are actually sent "over the wire" to the server. You can take this further with "dot notation" and just specify the elements you actually want to update. As in:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$.role": permission.role
}
},
function(err,doc) {
}
);
So this is the flexibility that MongoDB provides, where you can be very "targeted" in how you actually update a document.
What this does do however is "bypass" any logic you might have built into your "mongoose" schema, such as "validation" or other "pre-save hooks". That is because the "optimal" way is a MongoDB "feature" and how it is designed. Mongoose itself tries to be a "convenience" wrapper over this logic. But if you are prepared to take some control yourself, then the updates can be made in the most optimal way.
So where possible to do so, keep your data "embedded" and don't use referenced models. It allows the atomic update of both "parent" and "child" items in simple updates where you don't need to worry about concurrency. Probably is one of the reasons you should have selected MongoDB in the first place.

In order to validate subdocuments when updating in Mongoose, you have to 'load' it as a Schema object, and then Mongoose will automatically trigger validation and hooks.
const userSchema = new mongoose.Schema({
// ...
addresses: [addressSchema],
});
If you have an array of subdocuments, you can fetch the desired one with the id() method provided by Mongoose. Then you can update its fields individually, or if you want to update multiple fields at once then use the set() method.
User.findById(userId)
.then((user) => {
const address = user.addresses.id(addressId); // returns a matching subdocument
address.set(req.body); // updates the address while keeping its schema
// address.zipCode = req.body.zipCode; // individual fields can be set directly
return user.save(); // saves document with subdocuments and triggers validation
})
.then((user) => {
res.send({ user });
})
.catch(e => res.status(400).send(e));
Note that you don't really need the userId to find the User document, you can get it by searching for the one that has an address subdocument that matches addressId as follows:
User.findOne({
'addresses._id': addressId,
})
// .then() ... the same as the example above
Remember that in MongoDB the subdocument is saved only when the parent document is saved.
Read more on the topic on the official documentation.

If you don't want separate collection, just embed the permissionSchema into the folderSchema.
var folderSchema = new Schema({
name: { type: string },
permissions: [ {
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
} ]
});
If you need separate collections, this is the best approach:
You could have a Permission model:
var mongoose = require('mongoose');
var PermissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
module.exports = mongoose.model('Permission', PermissionSchema);
And a Folder model with a reference to the permission document.
You can reference another schema like this:
var mongoose = require('mongoose');
var FolderSchema = new Schema({
name: { type: string },
permissions: [ { type: mongoose.Schema.Types.ObjectId, ref: 'Permission' } ]
});
module.exports = mongoose.model('Folder', FolderSchema);
And then call Folder.findOne().populate('permissions') to ask mongoose to populate the field permissions.
Now, the following:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }).populate('permissions').exec(function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
The perm field will not be undefined (if the permission._id is actually in the permissions array), since it's been populated by Mongoose.

just try
let doc = await Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{ "permissions.$": permission},
);