Develop Reference

Mongoose - Cannot populate with sort on path notifications.from because it is a subproperty of a document array

I have a very simple mongo scheme I'm accessing with mongoose
I can map the username and firstname to each notification's from field by using populate, the issue is I can't seem to get any sorting to work on the date field
With this code I get an error of
MongooseError: Cannot populate with sort on path notifications.from
because it is a subproperty of a document array
Is it possible to do this a different way, or newer way (deep populate, virtuals)? I'm on Mongoose 5.
I'd rather not use vanilla javascript to sort the object afterwards or create a separate schema
var UserSchema = new Schema({
username: String,
firstname: String,
notifications: [
{
from: { type: Schema.Types.ObjectId, ref: 'User'},
date: Date,
desc: String
}
]
});
app.get('/notifications', function(req, res) {
User.findOne({ _id: req._id }, 'notifications')
.populate({
path: 'notifications.from',
populate: {
path: 'from',
model: 'User',
options: { sort: { 'notifications.date': -1 } }
}
})
.exec(function(err, user) {
if (err) console.log(err)
})
});
That possible duplicate is almost 2 years old about Mongo. I'm asking if there are newer or different ways of doing this in Mongoose as it has changed a bit since 2016 with newer features.

From Mongoose V5.0.12 FAQ : http://mongoosejs.com/docs/faq.html#populate_sort_order
Q. I'm populating a nested property under an array like the below
code:
new Schema({
arr: [{
child: { ref: 'OtherModel', type: Schema.Types.ObjectId }
}] });
.populate({ path: 'arr.child', options: { sort: 'name' } }) won't sort by arr.child.name?
A. See this GitHub issue. It's a known issue but one that's
exceptionally difficult to fix.
So unfortunately, for now, it's not possible,
One way to achieve this is to simply use javascript's native sort to sort the notifications after fetching.
.exec(function(err, user) {
if (err) console.log(err)
user.notifications.sort(function(a, b){
return new Date(b.date) - new Date(a.date);
});
})

It can be achievable using nesting populate like this -
eg - schema - {donationHistory: {campaignRequestId: [ref ids]}}
await user.populate({
path: 'donationHistory.campaignRequestId',
populate: [{
path: 'campaignRequestId',
model: 'CampaignRequest',
options: { sort: { 'createdAt': -1 } },
}],
...deepUserPopulation,
}).execPopulate();

Mongoose - Need two schemas for full and min versions of a model for a document-collection

edited after #enRaiser's answer.
I have a sandbox mongoDB database with a single collection called "hotels", the document-schema of which looks like this:
var roomSchema = new mongoose.Schema({
type: String,
number: Number,
description: String,
photos: [ String ],
price: Number
});
var hotelSchema = new mongoose.Schema({
name: String,
stars: Number,
description: String,
photos: [ String ],
currency: String,
location: {
address: String,
coordinates: [ Number ] /* enforce later validation to have max of two numbers in the array */
},
rooms: [roomSchema],
reviews: [{
name: String,
id: String,
review: String,
rating: Number
}],
services: [ String ]
});
Now, I'd like to have two versions of schema for Hotel, one for a 'deep' data model and the other for a min model.
var hotelMinSchema = new mongoose.Schema({
name: String,
stars: Number,
location: {
address: String,
coordinates: [ Number ]
},
currency: String
})
module.exports = {
full: mongoose.model('hotel', hotelSchema),
min: mongoose.model('hotel', hotelMinSchema)
}
Aparently I'm not supposed to have two models for a collection.. not so sure. I get this error thrown.
OverwriteModelError: Cannot overwrite hotel model once compiled.
I think there should be a work-around for this. Any help or suggestion would be appreciated.

This is totally wrong way of developing any Database. even in MYSQL, I would not have think of this way of designing DB.
Firstly there are duplicate data. You alwayse have to take care of syncing them.
and 2nd, even in your full model there is duplication of comment. the comment info is present in both User ( i.e the commenter and the blogger)
Irrespective of the DB. when ever you think of solution you have to identify the real entities. here in your use case there are only two entity User and comment. So just make two model. not more.(in case of MYSQL, I would say just make two tables User table and comment table.)
Then set up a relation between them. for that in mongoose learn the how to make relation and how to populate that data based on relation. its just like setting up foreign key in MYSQL.

Sorry, I just found this out.
var hotelListPromise = Hotel.find({})
.select('name stars location currency')
.exec((err, hotelData) => {
// my callback stuff here
});

Querying a collection from ObjectId's with $ne and $or condition

I need to serve an unanswered (by user) question to the user where user model holds the questions asked in past with an array, say it user.asked.
I want to clarify that I promisified mongoose library, hence I'm using those functions, nothing fancy.
User model:
var UserSchema = new Schema({
:
asked: [{ type: Schema.Types.ObjectId, ref: 'Question' }]
})
Here's what I tried so far:
Question.findAsync({
_id: {
$ne: {
$or: req.user.asked
}
}
})
which results: Cast to ObjectId failed for value \"[object Object]\" at path \"_id\".
I also tried to make an aggregation (example given):
{
$match: {
_id: {
$ne: {
$or: [ ObjectId("56ccfb048f896e0c2d06d08f"), ObjectId("56ccfb048f896e0c2d06d98f") ]
}
}
}
}
however this returns all of the documents in the collection (even the referred ones).
Do you have any suggestions?

You can use the $nin operator to find a doc with a field that matches none of the values in an array:
Question.findAsync({
_id: {
$nin: [ObjectId("56ccfb048f896e0c2d06d08f"), ObjectId("56ccfb048f896e0c2d06d98f")]
}
})

Grab object from MongoDB but limit number of items pulled from array

My API currently has a route for Getting an event from my MongoDB database based on event_id. This works fine. However, I have a 'photos' array within this event object that is growing (currently over 3,000 objects within this array).
I want to pass a limit parameter to limit the number of results pulled from this array, but cannot figure out how. Below is my current node route and mongoDB schema:
route:
// get event by _id
app.get('/api/events/:event_id', function(req, res) {
// use mongoose to get event
Event.findOne({object_id: req.params.event_id}, function(err, event) {
// if there is an error retrieving, send the error. nothing after res.send(err) will execute
if (err)
res.send(err)
if (req.params.limit >= 0) {
// res.jsonp(event) with photos array limited to req.params.limit
}
res.jsonp(event); // return event in JSON format
});
});
schema:
var eventSchema = new Schema({
event: String,
city: String,
state: String,
date: String,
start: String,
end: String,
dateState: String,
radius: String,
team_1: String,
team_2: String,
object_id: String,
longitude: String,
latitude: String,
cover: {
img: String,
username: String
},
photos: []
})

Don't have a constantly growing array field. It's not good for performance because MongoDB (well, if <= 2.6/using mmap) will be moving the document around when it grows outside of the space allocated for it by the storage engine, causing performance problems. You should change your schema to avoid an array like this, but I can't really say more about how you should do it because I don't know much about your use case.
There is a way to limit the number of array elements returned in a find query though, using $slice projection.
> db.test.drop()
> db.test.insert({ "_id" : 0, "x" : [0, 1, 2, 3, 4] })
> db.test.find({ "_id" : 0 }, { "x" : { "$slice" : 2 } })
{ "_id" : 0, "x" : [0, 1] }

Event.findOne({object_id: req.params.event_id})
.limit(10)
.exec(function(e,doc){
...
});
Edit
Or if you have ref on the photo... you can populate the doc array of referenced id with limit option. Hope it helps :) All abount population
.find(...)
.populate({
path: 'photos',
options: { limit: 5 }
})
.exec(...)
Schema
var eventSchema = new Schema({
event: String,
city: String,
state: String,
...
photos: [{ type:String, ref:'pictureSchema' }]
}
var pictureSchema = new Schema({
name : {type:String},
url : {type:String},
...
}
In photos array than you just put id of the pictures doc, when you populate the photos array it will put pictureSceham doc insted of _id.

MongoDB query on populated fields

I have models called "Activities" that I am querying for (using Mongoose). Their schema looks like this:
var activitySchema = new mongoose.Schema({
actor: {
type: mongoose.Schema.ObjectId,
ref: 'User',
required: true
},
recipient: {
type: mongoose.Schema.ObjectId,
ref: 'User'
},
timestamp: {
type: Date,
default: Date.now
},
activity: {
type: String,
required: true
},
event: {
type: mongoose.Schema.ObjectId,
ref: 'Event'
},
comment: {
type: mongoose.Schema.ObjectId,
ref: 'Comment'
}
});
When I query for them, I am populating the actor, recipient, event, and comment fields (all the references). After that, I also deep-populate the event field to get event.creator. Here is my code for the query:
var activityPopulateObj = [
{ path: 'event' },
{ path: 'event.creator' },
{ path: 'comment' },
{ path: 'actor' },
{ path: 'recipient' },
{ path: 'event.creator' }
],
eventPopulateObj = {
path: 'event.creator',
model: User
};
Activity.find({ $or: [{recipient: user._id}, {actor: {$in: user.subscriptions}}, {event: {$in: user.attending}}], actor: { $ne: user._id} })
.sort({ _id: -1 })
.populate(activityPopulateObj)
.exec(function(err, retrievedActivities) {
if(err || !retrievedActivities) {
deferred.reject(new Error("No events found."));
}
else {
User.populate(retrievedActivities, eventPopulateObj, function(err, data){
if(err) {
deferred.reject(err.message);
}
else {
deferred.resolve(retrievedActivities);
}
});
}
});
This is already a relatively complex query, but I need to do even more. If it hits the part of the $or statement that says {actor: {$in: user.subscriptions}}, I also need to make sure that the event's privacy field is equal to the string public. I tried using $elemMatch, but since the event has to be populated first, I couldn't query any of its fields. I need to achieve this same goal in multiple other queries, as well.
Is there any way for me to achieve this further filtering like I have described?

The answer is to change your schema.
You've fallen into the trap that many devs have before you when coming into document database development from a history of using relational databases: MongoDB is not a relational database and should not be treated like one.
You need to stop thinking about foreign keys and perfectly normalized data and instead, keep each document as self-contained as possible, thinking about how to best embed relevant associated data within your documents.
This doesn't mean you can't maintain associations as well. It might mean a structure like this, where you embed only necessary details, and query for the full record when needed:
var activitySchema = new mongoose.Schema({
event: {
_id: { type: ObjectId, ref: "Event" },
name: String,
private: String
},
// ... other fields
});
Rethinking your embed strategy will greatly simplify your queries and keep the query count to a minimum. populate will blow your count up quickly, and as your dataset grows this will very likely become a problem.

You can try below aggregation. Look at this answer: https://stackoverflow.com/a/49329687/12729769
And then, you can use fields from $addFields in your query. Like
{score: {$gte: 5}}

but since the event has to be populated first, I couldn't query any of its fields.
No can do. Mongodb cannot do joins. When you make a query, you can work with exactly one collection at a time. And FYI all those mongoose populates are additional, distinct database queries to load those records.
I don't have time to dive into the details of your schema and application, but most likely you will need to denormalize your data and store a copy of whatever event fields you need to join on in the primary collection.