I would query only a part of my document but without precising any value to retrieve, just to avoid to query the whole document. I have seen the different queries, even the where query, but seems all require a value to match with the document's properties,
for example:
here my Model.Schema:
const ScheduleSchema=new Schema({
month:{type: Number, required:true},
day:{type: Number, required:true},
hour:{
morning: {
time: [{
timePlot:String,
prospect:[{
type: documentType,
ref:"Prospect"
}]
}]
},
afternoon:{
time: [{
timePlot:String,
prospect:[{
type: documentType,
ref:"Prospect"
}]
}]
}
}
})
How would only fetch the afternoon section for example, however, the queries I have seen so far ask to me to provide some value to match with my path, so my question is how query for a subpath providing only the path instead of path:value. For example as following
Model.find({path: afternoon}, () => {...}
any hint would be great,
thanks
Specifying which parts of the document to return is done by providing a projection argument to find as the second parameter:
Model.find({path: afternoon}, 'path', () => {...}
Related
I have opened a related issue on GitHub, but maybe someone here will be able to help quicker.
Summary:
ValidationException: Query key condition not supported
I need to find records in last (amount) seconds on a given location.
Pretty simple, but already related to other issues:
One and another one
WORKS:
Activity.query('locationId').eq(locationId).exec();
DOES NOT WORK:
Activity.query('locationId').eq(locationId).where('createdAt').ge(date).exec();
Code sample:
Schema
const Activity = (dynamoose: typeof dynamooseType) => dynamoose.model<ActivityType, {}>('Activity',
new Schema({
id: {
type: String,
default: () => {
return uuid();
},
hashKey: true,
},
userId: {
type: String,
},
locationId: {
type: String,
rangeKey: true,
index: {
global: true,
},
},
createdAt: { type: Number, rangeKey: true, required: true, default: Date.now },
action: {
type: Number,
},
}, {
expires: 60 * 60 * 24 * 30 * 3, // activity logs to expire after 3 months
}));
Code which executes the function
Funny part is that I found this as workaround proposed to be used until they merge PR giving ability to specify timestamps as keys, but unfortunately it does not work.
async getActivitiesInLastSeconds(locationId: string, timeoutInSeconds: number) {
const Activity = schema.Activity(this.dynamoose);
const date = moment().subtract(timeoutInSeconds, 'seconds').valueOf();
return await Activity.query('locationId').eq(locationId)
.where('createdAt').ge(date).exec();
}
I suspect createdAt is not a range key of your table / index. You need to either do .filter('createdAt').ge(date) or modify your table / index schema.
I'm pretty sure the problem is that when you specifying rangeKey: true on the createdAt property you are telling that to be used on the global index (I don't think that is the correct term). That range key will be linked to the id property.
I believe the easiest solution would be to change your locationId index to be something like the following:
index: {
global: true,
rangeKey: 'createdAt',
},
That way you are being very explicit about which index you want to set createdAt as the rangeKey for.
After making that change please remember to sync your changes with either your local DynamoDB server or the actual DynamoDB service, so that the changes get populated both in your code and on the database system.
Hopefully this helps! If it doesn't fix your problem please feel free to comment and I'll help you further.
I am using mongodb as database with mongoose as ORM. I have a field booking_id in my schema which is unique , so I cannot have it null. Thus I have designed my code something like this.
var bookingSchema = new Schema({
booking_id_customer: {
type: Number,
default : Math.floor(Math.random()*900000000300000000000) + 1000000000000000,
index: { unique: true }
},
It works perfectly for the first time, but from 2nd time onwards I get this duplicacy error.
{ [MongoError: E11000 duplicate key error index: xx.bookings.$booking_id_customer_1 dup key: { : 4.439605615108491e+20 }]
name: 'MongoError',
message: 'E11000 duplicate key error index:
I expect it to generate random numbers but I have no clue about whats going wrong in 2nd time.
You are setting the default just once, at schema creation.
If you want it to be called for each new document, you need to turn it into a function that Mongoose will call:
default : function() {
return Math.floor(Math.random()*900000000300000000000) + 1000000000000000
}
However, there is another issue with your code: the values you're using (900000000300000000000 and 1000000000000000) exceed Number.MAX_SAFE_INTEGER, which can lead to problems.
I would suggest using mongoose.Types.ObjectId as id generator, which is also what Mongoose and MongoDB use to create (unique) document id's:
booking_id_customer : {
type : mongoose.Schema.Types.ObjectId,
default : mongoose.Types.ObjectId,
index : { unique: true }
}
Or re-use the _id property of the document, which is also unique.
edited after #enRaiser's answer.
I have a sandbox mongoDB database with a single collection called "hotels", the document-schema of which looks like this:
var roomSchema = new mongoose.Schema({
type: String,
number: Number,
description: String,
photos: [ String ],
price: Number
});
var hotelSchema = new mongoose.Schema({
name: String,
stars: Number,
description: String,
photos: [ String ],
currency: String,
location: {
address: String,
coordinates: [ Number ] /* enforce later validation to have max of two numbers in the array */
},
rooms: [roomSchema],
reviews: [{
name: String,
id: String,
review: String,
rating: Number
}],
services: [ String ]
});
Now, I'd like to have two versions of schema for Hotel, one for a 'deep' data model and the other for a min model.
var hotelMinSchema = new mongoose.Schema({
name: String,
stars: Number,
location: {
address: String,
coordinates: [ Number ]
},
currency: String
})
module.exports = {
full: mongoose.model('hotel', hotelSchema),
min: mongoose.model('hotel', hotelMinSchema)
}
Aparently I'm not supposed to have two models for a collection.. not so sure. I get this error thrown.
OverwriteModelError: Cannot overwrite hotel model once compiled.
I think there should be a work-around for this. Any help or suggestion would be appreciated.
This is totally wrong way of developing any Database. even in MYSQL, I would not have think of this way of designing DB.
Firstly there are duplicate data. You alwayse have to take care of syncing them.
and 2nd, even in your full model there is duplication of comment. the comment info is present in both User ( i.e the commenter and the blogger)
Irrespective of the DB. when ever you think of solution you have to identify the real entities. here in your use case there are only two entity User and comment. So just make two model. not more.(in case of MYSQL, I would say just make two tables User table and comment table.)
Then set up a relation between them. for that in mongoose learn the how to make relation and how to populate that data based on relation. its just like setting up foreign key in MYSQL.
Sorry, I just found this out.
var hotelListPromise = Hotel.find({})
.select('name stars location currency')
.exec((err, hotelData) => {
// my callback stuff here
});
I have the following schemas for the document Folder:
var permissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
var folderSchema = new Schema({
name: { type: string },
permissions: [ permissionSchema ]
});
So, for each Page I can have many permissions. In my CMS there's a panel where I list all the folders and their permissions. The admin can edit a single permission and save it.
I could easily save the whole Folder document with its permissions array, where only one permission was modified. But I don't want to save all the document (the real schema has much more fields) so I did this:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }, function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
but the problem is that perm is always undefined! I tried to "statically" fetch the permission in this way:
var perm = data.permissions[0];
and it works great, so the problem is that Underscore library is not able to query the permissions array. So I guess that there's a better (and workgin) way to get the subdocument of a fetched document.
Any idea?
P.S.: I solved checking each item in the data.permission array using a "for" loop and checking data.permissions[i]._id == permission._id but I'd like a smarter solution, I know there's one!
So as you note, the default in mongoose is that when you "embed" data in an array like this you get an _id value for each array entry as part of it's own sub-document properties. You can actually use this value in order to determine the index of the item which you intend to update. The MongoDB way of doing this is the positional $ operator variable, which holds the "matched" position in the array:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$": permission
}
},
function(err,doc) {
}
);
That .findOneAndUpdate() method will return the modified document or otherwise you can just use .update() as a method if you don't need the document returned. The main parts are "matching" the element of the array to update and "identifying" that match with the positional $ as mentioned earlier.
Then of course you are using the $set operator so that only the elements you specify are actually sent "over the wire" to the server. You can take this further with "dot notation" and just specify the elements you actually want to update. As in:
Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{
"$set": {
"permissions.$.role": permission.role
}
},
function(err,doc) {
}
);
So this is the flexibility that MongoDB provides, where you can be very "targeted" in how you actually update a document.
What this does do however is "bypass" any logic you might have built into your "mongoose" schema, such as "validation" or other "pre-save hooks". That is because the "optimal" way is a MongoDB "feature" and how it is designed. Mongoose itself tries to be a "convenience" wrapper over this logic. But if you are prepared to take some control yourself, then the updates can be made in the most optimal way.
So where possible to do so, keep your data "embedded" and don't use referenced models. It allows the atomic update of both "parent" and "child" items in simple updates where you don't need to worry about concurrency. Probably is one of the reasons you should have selected MongoDB in the first place.
In order to validate subdocuments when updating in Mongoose, you have to 'load' it as a Schema object, and then Mongoose will automatically trigger validation and hooks.
const userSchema = new mongoose.Schema({
// ...
addresses: [addressSchema],
});
If you have an array of subdocuments, you can fetch the desired one with the id() method provided by Mongoose. Then you can update its fields individually, or if you want to update multiple fields at once then use the set() method.
User.findById(userId)
.then((user) => {
const address = user.addresses.id(addressId); // returns a matching subdocument
address.set(req.body); // updates the address while keeping its schema
// address.zipCode = req.body.zipCode; // individual fields can be set directly
return user.save(); // saves document with subdocuments and triggers validation
})
.then((user) => {
res.send({ user });
})
.catch(e => res.status(400).send(e));
Note that you don't really need the userId to find the User document, you can get it by searching for the one that has an address subdocument that matches addressId as follows:
User.findOne({
'addresses._id': addressId,
})
// .then() ... the same as the example above
Remember that in MongoDB the subdocument is saved only when the parent document is saved.
Read more on the topic on the official documentation.
If you don't want separate collection, just embed the permissionSchema into the folderSchema.
var folderSchema = new Schema({
name: { type: string },
permissions: [ {
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
} ]
});
If you need separate collections, this is the best approach:
You could have a Permission model:
var mongoose = require('mongoose');
var PermissionSchema = new Schema({
role: { type: String },
create_folders: { type: Boolean },
create_contents: { type: Boolean }
});
module.exports = mongoose.model('Permission', PermissionSchema);
And a Folder model with a reference to the permission document.
You can reference another schema like this:
var mongoose = require('mongoose');
var FolderSchema = new Schema({
name: { type: string },
permissions: [ { type: mongoose.Schema.Types.ObjectId, ref: 'Permission' } ]
});
module.exports = mongoose.model('Folder', FolderSchema);
And then call Folder.findOne().populate('permissions') to ask mongoose to populate the field permissions.
Now, the following:
savePermission: function (folderId, permission, callback) {
Folder.findOne({ _id: folderId }).populate('permissions').exec(function (err, data) {
var perm = _.findWhere(data.permissions, { _id: permission._id });
_.extend(perm, permission);
data.markModified("permissions");
data.save(callback);
});
}
The perm field will not be undefined (if the permission._id is actually in the permissions array), since it's been populated by Mongoose.
just try
let doc = await Folder.findOneAndUpdate(
{ "_id": folderId, "permissions._id": permission._id },
{ "permissions.$": permission},
);
I have models called "Activities" that I am querying for (using Mongoose). Their schema looks like this:
var activitySchema = new mongoose.Schema({
actor: {
type: mongoose.Schema.ObjectId,
ref: 'User',
required: true
},
recipient: {
type: mongoose.Schema.ObjectId,
ref: 'User'
},
timestamp: {
type: Date,
default: Date.now
},
activity: {
type: String,
required: true
},
event: {
type: mongoose.Schema.ObjectId,
ref: 'Event'
},
comment: {
type: mongoose.Schema.ObjectId,
ref: 'Comment'
}
});
When I query for them, I am populating the actor, recipient, event, and comment fields (all the references). After that, I also deep-populate the event field to get event.creator. Here is my code for the query:
var activityPopulateObj = [
{ path: 'event' },
{ path: 'event.creator' },
{ path: 'comment' },
{ path: 'actor' },
{ path: 'recipient' },
{ path: 'event.creator' }
],
eventPopulateObj = {
path: 'event.creator',
model: User
};
Activity.find({ $or: [{recipient: user._id}, {actor: {$in: user.subscriptions}}, {event: {$in: user.attending}}], actor: { $ne: user._id} })
.sort({ _id: -1 })
.populate(activityPopulateObj)
.exec(function(err, retrievedActivities) {
if(err || !retrievedActivities) {
deferred.reject(new Error("No events found."));
}
else {
User.populate(retrievedActivities, eventPopulateObj, function(err, data){
if(err) {
deferred.reject(err.message);
}
else {
deferred.resolve(retrievedActivities);
}
});
}
});
This is already a relatively complex query, but I need to do even more. If it hits the part of the $or statement that says {actor: {$in: user.subscriptions}}, I also need to make sure that the event's privacy field is equal to the string public. I tried using $elemMatch, but since the event has to be populated first, I couldn't query any of its fields. I need to achieve this same goal in multiple other queries, as well.
Is there any way for me to achieve this further filtering like I have described?
The answer is to change your schema.
You've fallen into the trap that many devs have before you when coming into document database development from a history of using relational databases: MongoDB is not a relational database and should not be treated like one.
You need to stop thinking about foreign keys and perfectly normalized data and instead, keep each document as self-contained as possible, thinking about how to best embed relevant associated data within your documents.
This doesn't mean you can't maintain associations as well. It might mean a structure like this, where you embed only necessary details, and query for the full record when needed:
var activitySchema = new mongoose.Schema({
event: {
_id: { type: ObjectId, ref: "Event" },
name: String,
private: String
},
// ... other fields
});
Rethinking your embed strategy will greatly simplify your queries and keep the query count to a minimum. populate will blow your count up quickly, and as your dataset grows this will very likely become a problem.
You can try below aggregation. Look at this answer: https://stackoverflow.com/a/49329687/12729769
And then, you can use fields from $addFields in your query. Like
{score: {$gte: 5}}
but since the event has to be populated first, I couldn't query any of its fields.
No can do. Mongodb cannot do joins. When you make a query, you can work with exactly one collection at a time. And FYI all those mongoose populates are additional, distinct database queries to load those records.
I don't have time to dive into the details of your schema and application, but most likely you will need to denormalize your data and store a copy of whatever event fields you need to join on in the primary collection.