I have the following schemas for the document Folder:
var permissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
var folderSchema = new Schema({
name: { type: string },
permissions: [ permissionSchema ]
});
So, for each Page I can have many permissions. In my CMS there's a panel where I list all the folders and their permissions. The admin can edit a single permission and save it.
I could easily save the whole Folder document with its permissions array, where only one permission was modified. But I don't want to save all the document (the real schema has much more fields) so I did this:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }, function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
but the problem is that perm is always undefined! I tried to "statically" fetch the permission in this way:
var perm = data.permissions[0];
and it works great, so the problem is that Underscore library is not able to query the permissions array. So I guess that there's a better (and workgin) way to get the subdocument of a fetched document.
Any idea?
P.S.: I solved checking each item in the data.permission array using a "for" loop and checking data.permissions[i]._id == permission._id but I'd like a smarter solution, I know there's one!
So as you note, the default in mongoose is that when you "embed" data in an array like this you get an _id value for each array entry as part of it's own sub-document properties. You can actually use this value in order to determine the index of the item which you intend to update. The MongoDB way of doing this is the positional $ operator variable, which holds the "matched" position in the array:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$": permission
}
},
function(err,doc) {
}
);
That .findOneAndUpdate() method will return the modified document or otherwise you can just use .update() as a method if you don't need the document returned. The main parts are "matching" the element of the array to update and "identifying" that match with the positional $ as mentioned earlier.
Then of course you are using the $set operator so that only the elements you specify are actually sent "over the wire" to the server. You can take this further with "dot notation" and just specify the elements you actually want to update. As in:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$.role": permission.role
}
},
function(err,doc) {
}
);
So this is the flexibility that MongoDB provides, where you can be very "targeted" in how you actually update a document.
What this does do however is "bypass" any logic you might have built into your "mongoose" schema, such as "validation" or other "pre-save hooks". That is because the "optimal" way is a MongoDB "feature" and how it is designed. Mongoose itself tries to be a "convenience" wrapper over this logic. But if you are prepared to take some control yourself, then the updates can be made in the most optimal way.
So where possible to do so, keep your data "embedded" and don't use referenced models. It allows the atomic update of both "parent" and "child" items in simple updates where you don't need to worry about concurrency. Probably is one of the reasons you should have selected MongoDB in the first place.
In order to validate subdocuments when updating in Mongoose, you have to 'load' it as a Schema object, and then Mongoose will automatically trigger validation and hooks.
const userSchema = new mongoose.Schema({
// ...
addresses: [addressSchema],
});
If you have an array of subdocuments, you can fetch the desired one with the id() method provided by Mongoose. Then you can update its fields individually, or if you want to update multiple fields at once then use the set() method.
User.findById(userId)
.then((user) => {
const address = user.addresses.id(addressId); // returns a matching subdocument
address.set(req.body); // updates the address while keeping its schema
// address.zipCode = req.body.zipCode; // individual fields can be set directly
return user.save(); // saves document with subdocuments and triggers validation
})
.then((user) => {
res.send({ user });
})
.catch(e => res.status(400).send(e));
Note that you don't really need the userId to find the User document, you can get it by searching for the one that has an address subdocument that matches addressId as follows:
User.findOne({
'addresses._id': addressId,
})
// .then() ... the same as the example above
Remember that in MongoDB the subdocument is saved only when the parent document is saved.
Read more on the topic on the official documentation.
If you don't want separate collection, just embed the permissionSchema into the folderSchema.
var folderSchema = new Schema({
name: { type: string },
permissions: [ {
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
} ]
});
If you need separate collections, this is the best approach:
You could have a Permission model:
var mongoose = require('mongoose');
var PermissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
module.exports = mongoose.model('Permission', PermissionSchema);
And a Folder model with a reference to the permission document.
You can reference another schema like this:
var mongoose = require('mongoose');
var FolderSchema = new Schema({
name: { type: string },
permissions: [ { type: mongoose.Schema.Types.ObjectId, ref: 'Permission' } ]
});
module.exports = mongoose.model('Folder', FolderSchema);
And then call Folder.findOne().populate('permissions') to ask mongoose to populate the field permissions.
Now, the following:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }).populate('permissions').exec(function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
The perm field will not be undefined (if the permission._id is actually in the permissions array), since it's been populated by Mongoose.
just try
let doc = await Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{ "permissions.$": permission},
);
Related
I'm trying to delete a mongodb object and then once deleted, I want to delete everything associated with that mongodb object. Including nested mongodb objects from my mongo database.
var parentObjectSchema = new mongoose.Schema({
name: String,
split: Number,
parts: [
{
type: mongoose.Schema.Types.ObjectId,
ref: "ChildObjectSchema"
}
],
});
var childObjectSchema = new mongoose.Schema({
name: String,
number: Number,
things: [
{
type: mongoose.Schema.Types.ObjectId,
ref: "Things"
}
],
});
So I am trying to delete the parentObject, and childObjects that come along with it. Not sure how I would go about doing that. I am successful in deleting the parentObject but that childObject is still in the mongodb, taking up space. Any ideas?
MongoDB doesn't provide the notion of foreign keys like other databases do. Mongoose has convenience methods in the client library that populates your documents with other documents using multiple queries and joining the results:
https://mongoosejs.com/docs/populate.html
If you want to do a cascading deletion then you'll need to grab the object ids of the children in the parent documents you want to delete, and then execute a delete against those children documents.
Here's a simplified example:
const deleteThing = (thingId) => {
thingObjectSchema.remove({ _id: thingId });
};
const deleteChild = (childId) => {
childObjectSchema.findOne({ _id: childId }).select('things').lean().exec((err, child) => {
for (const thingId of child.things) {
deleteThing(thingId);
}
childObjectSchema.remove({ _id: childId });
})
};
const deleteParent = (parentId) => {
parentObjectSchema.findOne({ _id: parentId }).select('parts').lean().exec((err, parent) => {
for (const childId of parent.parts) {
deleteChild(childId);
}
parentObjectSchema.remove({ _id: parentId });
})
};
// note: not actually tested
I'm newbie using Vertx, I'm building a basic Api Rest with Vertx + Mongo using Javascript.
I'm looking for some way to automaticly validate the incoming documents before insert or update (for example, something like schemas in Mongoose).
I got the following:
POST entry point Inserting a new cat
var BodyHandler = require("vertx-web-js/body_handler");
var Router = require("vertx-web-js/router");
var router = Router.router(vertx);
router.post("/cat")
.produces("application/json")
.handler(BodyHandler.create().handle)
.handler(controller.createCat);
Controller's function Inserting a new cat
createCat: function (ctx) {
var response = ctx.response();
var body = ctx.getBodyAsJson() || {};
console.log('inserting cat')
connection.mongoClient.insert("cats", { name: body.name }, function (res, res_err) {
if (res_err == null) {
var id = res;
console.log("Inserted cat with id " + id);
response.putHeader("content-type", "application/json");
response.end(JSON.stringify(id));
} else {
console.log('err')
res_err.printStackTrace();
}
});
}
The problem is that I can insert empty documents resulting documents with only one field: the mongo ID.
TL;DR - I want to know if there is some way to tell Mongo that name field is ALWAYS required.
Solution I used:
mongoClient.runCommand("collMod", { collMod: "cats", validator: { $and: [ {"name": {$type: "string", $exists: true}} ] }, validationLevel: "strict", validationAction: "error" }, function(res, res_err) {
if (res_err) res_err.printStackTrace()
else console.log('The schema of cats collection has been updated')
});
In mongodb there is a concept of validation with query filters for collection: https://docs.mongodb.com/manual/core/schema-validation/#query-expressions
If the "name" field is always required, you can create your collection like this:
db.createCollection("cats", {
validator: {
$and: [ {"name": {$type: "string", $exists: true}} ]
})
you can create your own schema in a separate json file, then create your own validation method when the context is provided by ur router.
I assume different endpoint has different validation.
you can also create a middleware using gateleen
you have plenty of options, mongoose its benefits but the whole point of using mongodb is not creating a schema. Therefore, u can assume that the request should have specific mandatory fields to validate.
Does anyone know why the following one-to-many relationship between "users" and "posts" (users can have many posts) is not working? It appears I have setup my mongoose associations correctly, but when a new post is created, not only is it not assigned a user, but the users themselves are also not associated with any posts. I'm not sure what I might be doing wrong here.
If you see the JSON object below, it should have a user value, denoting the user whom created the post. You'll see in the Post Model below, that a user value should be created, but does not.
What am I doing wrong?
Here's the JSON object after creating a new post
{
__v: 0
_id: "587ee8f5a99b1709b012ce8f"
createdAt: "2017-01-18T04:03:01.446Z"
message: "This is my first test post!"
updatedAt: "2017-01-18T04:03:01.446Z"
}
Question: Why is the user field missing from the JSON above despite being created in the Post Model below?
Here's my Post Model:
// Setup dependencies:
var mongoose = require('mongoose');
// Setup a schema:
var PostSchema = new mongoose.Schema (
{
message: {
type: String,
minlength: 2,
maxlength: 2000,
required: true,
trim: true,
}, // end message field
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
},
{
timestamps: true,
}
);
// Instantiate our model and export it:
module.exports = mongoose.model('Post', PostSchema)
Here's my User Model:
// Setup dependencies:
var mongoose = require('mongoose');
// Setup a schema:
var UserSchema = new mongoose.Schema (
{
username: {
type: String,
minlength: 2,
maxlength: 20,
required: true,
trim: true,
unique: true, // username must be unique
dropDups: true,
lowercase: true,
validate: {
validator: function(username) {
var regex = /^[a-z0-9_]+$/i;
return regex.test(username);
},
message: 'Username may contain only letters, numbers or underscores.',
},
}, // end username field
posts: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'Post'
}],
},
{
timestamps: true,
});
// Instantiate our model and export it:
module.exports = mongoose.model('User', UserSchema)
Here's the Controller that queries the DB:
Note: This is the method that runs when the post form is submitted.
// Grab our Mongoose Models:
var User = require('mongoose').model('User');
var Post = require('mongoose').model('Post');
module.exports = {
// Creates a new post for logged in user:
newPost: function(req, res) {
Post.create(req.body)
.then(function(newPost) {
return res.json(newPost);
})
.catch(function(err) {
return res.json(err);
})
}
};
Does anyone know if my associations are improperly setup and this is why I'm not getting any actual posts or users to show up in their respective fields?
It seems that my server-side controller is firing properly, as the post is actually created. But the associations themselves are not linking up and I'm not sure what I'm doing wrong.
I'm adding just a simple answer below to follow up with the example above. Essentially, #cdbajorin was correct, I was absently thinking there was some automation going on and was not appropriately following through the proper mongoose commands to achieve my desired results.
The solution to my question is as follows:
In the User Model, update the UserSchema posts attribute to be an empty array, instead of a mongoose.Schema.Types.ObjectID, since an object ID is not stored here anyhow and I misunderstood how this works.
The code:
posts: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'Post'
}],
Instead, should be written simply:
posts: [],
The newPost method, in the server Controller, should be modified as follows (see comments inline for clarification):
newPost: function(req, res) {
// creates new post:
Post.create(req.body)
.then(function(newPost) {
// look up current user based on session ID:
// note: session setup not shown in this example.
User.findById(req.session.userID)
.then(function(user) {
// push new post into users.posts array from model setup**:
user.posts.push(newPost);
user.save();
return res.json(newPost);
})
})
.catch(function(err) {
return res.json(err);
})
This does solve the issue of the new post being generated, and then pushed into a user's posts array (from the UsersSchema).
Though the issue from the initial post is solved, one may question if this is the best use of database management. Storing posts inside of a user, as this example does, can take up a lot of space as users and posts start to add up.
This post ends up being duplicated in the database twice: first, as a document itself in the posts collection, and secondly, as an object in the posts array within the UserSchema.
A better solution is to keep the post as a unique document in the posts collection, but add the userID from the session information to it. Then, if all of user's posts are needed for any reason, a query to the Posts collection, based on the userID, would return all posts with that userID assigned to it. Then, only one copy of the post exists in the DB instead of two.
** Additional Note: Another way to modify the existing document would be to use an instance method, where an actual method would be inserted into the User Model (Schema) file, and called when needed:
For example, inserting the following code before the module.exports line in the UserSchema Model above, allows for convenient access this function when needed:
UserSchema.methods.addPost = function(post) {
this.posts.push(post);
this.save();
return true;
};
To call this instance method from our server Controller, we could re-write our Controller as follows:
User.findById(req.session.userID)
.then(function(user) {
// call our instance method above:
user.addPost(newPost);
return res.json(newPost);
});
The post will be pushed and saved by the instance method, which has been built into the instance object itself.
I'm looking for an easy way of updating an embedded document using mongoose without having to set each specific field manually. Looking at the accepted answer to this question, once you find the embedded document that you want to update you have to actually set each respective property and then save the parent. What I would prefer to do is pass in an update object and let MongoDB set the updates.
e.g. if I was updating a regular (non embedded) document I would do this:
models.User.findOneAndUpdate({_id: req.params.userId}, req.body.user, function(err, user) {
err ? resp.status(500).send(err) : user ? resp.send(user) : resp.status(404).send();
});
Here I don't actually have to go through each property in req.body.user and set the changes. I can't find a way of doing this kind of thing with sub documents as well ?
My Schema is as follows:
var UserSchema = BaseUserSchema.extend({
isActivated: { type: Boolean, required: true },
files: [FileSchema]
});
var FileSchema = new mongoose.Schema(
name: { type: String, required: true },
size: { type: Number, required: true },
type: { type: String, required: true },
});
And I'm trying to update a file based on user and file id.
Do I need to create a helper function to set the values, or is there a MongoDB way of doing this ?
Many thanks.
Well presuming that you have something that has you "filedata" in a variable, and of course the user _id that you are updating, then you wan't the $set operator:
var user = { /* The user information, at least the _id */
var filedata = { /* From somewhere with _id, name, size, type */ };
models.User.findOneAndUpdate(
{ "_id": user._id, "files._id": filedata._id },
{
"$set": {
"name": filedata.name,
"size": filedata.size,
"type": filedata.type
}
},
function(err,user) {
// Whatever in here such a message, but the update is already done.
}
);
Or really, just only $set the fields that you actually mean to "update" as long as you know which ones you mean. So if you only need to change the "size" then just set that for example.
I have models called "Activities" that I am querying for (using Mongoose). Their schema looks like this:
var activitySchema = new mongoose.Schema({
actor: {
type: mongoose.Schema.ObjectId,
ref: 'User',
required: true
},
recipient: {
type: mongoose.Schema.ObjectId,
ref: 'User'
},
timestamp: {
type: Date,
default: Date.now
},
activity: {
type: String,
required: true
},
event: {
type: mongoose.Schema.ObjectId,
ref: 'Event'
},
comment: {
type: mongoose.Schema.ObjectId,
ref: 'Comment'
}
});
When I query for them, I am populating the actor, recipient, event, and comment fields (all the references). After that, I also deep-populate the event field to get event.creator. Here is my code for the query:
var activityPopulateObj = [
{ path: 'event' },
{ path: 'event.creator' },
{ path: 'comment' },
{ path: 'actor' },
{ path: 'recipient' },
{ path: 'event.creator' }
],
eventPopulateObj = {
path: 'event.creator',
model: User
};
Activity.find({ $or: [{recipient: user._id}, {actor: {$in: user.subscriptions}}, {event: {$in: user.attending}}], actor: { $ne: user._id} })
.sort({ _id: -1 })
.populate(activityPopulateObj)
.exec(function(err, retrievedActivities) {
if(err || !retrievedActivities) {
deferred.reject(new Error("No events found."));
}
else {
User.populate(retrievedActivities, eventPopulateObj, function(err, data){
if(err) {
deferred.reject(err.message);
}
else {
deferred.resolve(retrievedActivities);
}
});
}
});
This is already a relatively complex query, but I need to do even more. If it hits the part of the $or statement that says {actor: {$in: user.subscriptions}}, I also need to make sure that the event's privacy field is equal to the string public. I tried using $elemMatch, but since the event has to be populated first, I couldn't query any of its fields. I need to achieve this same goal in multiple other queries, as well.
Is there any way for me to achieve this further filtering like I have described?
The answer is to change your schema.
You've fallen into the trap that many devs have before you when coming into document database development from a history of using relational databases: MongoDB is not a relational database and should not be treated like one.
You need to stop thinking about foreign keys and perfectly normalized data and instead, keep each document as self-contained as possible, thinking about how to best embed relevant associated data within your documents.
This doesn't mean you can't maintain associations as well. It might mean a structure like this, where you embed only necessary details, and query for the full record when needed:
var activitySchema = new mongoose.Schema({
event: {
_id: { type: ObjectId, ref: "Event" },
name: String,
private: String
},
// ... other fields
});
Rethinking your embed strategy will greatly simplify your queries and keep the query count to a minimum. populate will blow your count up quickly, and as your dataset grows this will very likely become a problem.
You can try below aggregation. Look at this answer: https://stackoverflow.com/a/49329687/12729769
And then, you can use fields from $addFields in your query. Like
{score: {$gte: 5}}
but since the event has to be populated first, I couldn't query any of its fields.
No can do. Mongodb cannot do joins. When you make a query, you can work with exactly one collection at a time. And FYI all those mongoose populates are additional, distinct database queries to load those records.
I don't have time to dive into the details of your schema and application, but most likely you will need to denormalize your data and store a copy of whatever event fields you need to join on in the primary collection.