Develop Reference

single chained query mongodb / mongoose to fetch all comments

I am trying to develop a personal project, a website that functions in a similair way to Stack Exchange, a user can ask a question which can receive multiple answers. Each question and answer can have multiple comments.
I am using nodeJS for my backend.
How can I fetch all comments for all the answers on a particular question in a single mongoDB / mongoose query?
It would be even more helpful if you could tell me how to fetch all comments for all the answers on a particular question as well as all the comments for the question in a single mongoDB / mongoose query?
Mongoose Schemas:
const questionSchema = new mongoose.Schema({
title: String,
content: String
})
const answerSchema = new mongoose.Schema({
questionId: String,
content: String,
})
const commentSchema = new mongoose.Schema({
idQuestion: String, // nullable
idAnswer: String, // nullable
content: String
})
Currently, I am performing a mongoose query to find all the answers for a particular questions. Then, using forEach, performing a mongoose query on each answer to find all the comments for each answer. I believe this is very taxing, performance wise and is not an ideal way to do what I would like to achieve.

You can try below aggregation. Match on question id followed by join to lookup all the answers ids with question id followed by lookup to pull in all comments.
db.questions.aggregate([
{"$match":{"_id":input_question_id}},
{"$lookup":{
"from":"answers",
"localField":"_id",
"foreignField":"questionId",
"as":"answers"
}},
{"$lookup":{
"from":"comments",
"let":{"ids":{"answers_id":"$answers._id","question_id":"$_id"}},
"pipeline":[
{"$match":{"$expr":{
"$or":[
{"$eq":["$idQuestion","$$ids.question_id"]},
{"$in":["$idAnswer","$$ids.answers_id"]}
]
}}}
],
"as":"comments"
}},
{"$project":{"comments":"$comments.content"}}
])
Working example here - https://mongoplayground.net/p/qBlKqk-JsxA

You can try,
$match your conditions questionId
$lookup join with comments
db.answers.aggregate([
{ $match: { questionId: 1 } },
{
$lookup: {
from: "comments",
localField: "_id",
foreignField: "idAnswer",
as: "comments"
}
}
])
Playground
Second approach, if you want to select questions with all answers and comment then try,
$match your conditions
$lookup with pipeline join with answers collection
pipeline field can allow to add all stages of pipeline that we are using in root level
$match questionId and get answers
$lookup join with comments collection
db.questions.aggregate([
{ $match: { _id: 1 } }, // this is optional if you want to select all questions then remove this
{
$lookup: {
from: "answers",
let: { questionId: "$_id" },
pipeline: [
{ $match: { $expr: { $eq: ["$$questionId", "$questionId"] } } },
{
$lookup: {
from: "comments",
localField: "_id",
foreignField: "idAnswer",
as: "comments"
}
}
],
as: "answers"
}
}
])
Playground
Show or hide extra fields, you can use $project operator at the end of above query,
You can show fields as per your needs
{
$project: {
_id: 1,
content: 1,
"comments._id": 1,
"comments.content": 1
}
}
Playground
Suggestions:
I am not sure, you have already done or not, but try to define object id type in reference field instead of string type, like I have updated your schema, this will add a default index in object id and this will increase a speed of fetching data,
const answerSchema = new mongoose.Schema({
questionId: mongoose.Types.ObjectId,
content: String,
})
const commentSchema = new mongoose.Schema({
idQuestion: mongoose.Types.ObjectId, // nullable
idAnswer: mongoose.Types.ObjectId, // nullable
content: String
})

Mongoose: find document by values inside an array field

I've got a chat schema that looks like that:
var chatSchema = new mongoose.Schema({
users: [{
type: mongoose.Schema.Types.ObjectId,
required: true
}]
});
It contains array of user IDs.
Now I want to find one chat document that contains an array of two user IDs.
At the beginning I tried to do this:
Chat.findOne({ users: { $in: [req.user_id, receiver._id] }})
.then(chat => { })
But it seems that every time it gives me the chat that contains at least one of the IDs I mentioned in the query.
So I've tried to change it to this but with no luck:
Chat.findOne()
.where({ users: { $in: [req.user_id] }})
.where({ users: { $in: [receiver._id] }})
.then(chat => { })
I need to find the chat that contains both of the user ID's inside the users array otherwise I expect for a null value.
How can I achieve this goal?
Thanks!

This is the way $in works - returns the document when at least one value matches. You should use $all instead:
Chat.findOne({ users: { $all: [req.user_id, receiver._id] }})

Mongoose virtuals are not added to the result

I'm building a quiz editor where rounds contain questions and questions can be in multiple rounds. Therefor I have the following Schemes:
var roundSchema = Schema({
name: String
});
var questionSchema = Schema({
question: String,
parentRounds: [{
roundId: { type: Schema.Types.ObjectId, ref: 'Round'},
isOwner: Boolean
}]
});
What I want is to query a round, but also list all questions related to that round.
Therefor I created the following virtual on roundSchema:
roundSchema.virtual('questions', {
ref : 'Question',
localField : '_id',
foreignField : 'parentRounds.roundId'
});
Further instantiating the Round and Question model and querying a Round results in an object without questions:
var Round = mongoose.model('Round', roundSchema, 'rounds');
var Question = mongoose.model('Question', questionSchema, 'questions');
Round.findById('5ba117e887f66908ae87aa56').populate('questions').exec((err, rounds) => {
if(err) return console.log(err);
console.log(rounds);
process.exit();
});
Result:
Mongoose: rounds.findOne({ _id: ObjectId("5ba117e887f66908ae87aa56") }, { projection: {} })
Mongoose: questions.find({ 'parentRounds.roundId': { '$in': [ ObjectId("5ba117e887f66908ae87aa56") ] } }, { projection: {} })
{ _id: 5ba117e887f66908ae87aa56, __v: 0, name: 'Test Roundname' }
As you can see, I have debugging turned on, which shows me the mongo queries. It seems like the second one is the one used to fill up the virtual field.
Executing the same query using Mongohub DOES result in a question:
So why doesn't Mongoose show that questions array I'm expecting?
I've also tried the same example with just one parentRound and no sub-objects, but that also doesn't work.

Found the answer myself...
Apparently, I have to use
console.log(rounds.toJSON({virtuals: true}));
instead of
console.log(rounds);
Why would Mongoose do such a devil thing? :(

Mongoose - Cannot populate with sort on path notifications.from because it is a subproperty of a document array

I have a very simple mongo scheme I'm accessing with mongoose
I can map the username and firstname to each notification's from field by using populate, the issue is I can't seem to get any sorting to work on the date field
With this code I get an error of
MongooseError: Cannot populate with sort on path notifications.from
because it is a subproperty of a document array
Is it possible to do this a different way, or newer way (deep populate, virtuals)? I'm on Mongoose 5.
I'd rather not use vanilla javascript to sort the object afterwards or create a separate schema
var UserSchema = new Schema({
username: String,
firstname: String,
notifications: [
{
from: { type: Schema.Types.ObjectId, ref: 'User'},
date: Date,
desc: String
}
]
});
app.get('/notifications', function(req, res) {
User.findOne({ _id: req._id }, 'notifications')
.populate({
path: 'notifications.from',
populate: {
path: 'from',
model: 'User',
options: { sort: { 'notifications.date': -1 } }
}
})
.exec(function(err, user) {
if (err) console.log(err)
})
});
That possible duplicate is almost 2 years old about Mongo. I'm asking if there are newer or different ways of doing this in Mongoose as it has changed a bit since 2016 with newer features.

From Mongoose V5.0.12 FAQ : http://mongoosejs.com/docs/faq.html#populate_sort_order
Q. I'm populating a nested property under an array like the below
code:
new Schema({
arr: [{
child: { ref: 'OtherModel', type: Schema.Types.ObjectId }
}] });
.populate({ path: 'arr.child', options: { sort: 'name' } }) won't sort by arr.child.name?
A. See this GitHub issue. It's a known issue but one that's
exceptionally difficult to fix.
So unfortunately, for now, it's not possible,
One way to achieve this is to simply use javascript's native sort to sort the notifications after fetching.
.exec(function(err, user) {
if (err) console.log(err)
user.notifications.sort(function(a, b){
return new Date(b.date) - new Date(a.date);
});
})

It can be achievable using nesting populate like this -
eg - schema - {donationHistory: {campaignRequestId: [ref ids]}}
await user.populate({
path: 'donationHistory.campaignRequestId',
populate: [{
path: 'campaignRequestId',
model: 'CampaignRequest',
options: { sort: { 'createdAt': -1 } },
}],
...deepUserPopulation,
}).execPopulate();

MongoDB query on populated fields

I have models called "Activities" that I am querying for (using Mongoose). Their schema looks like this:
var activitySchema = new mongoose.Schema({
actor: {
type: mongoose.Schema.ObjectId,
ref: 'User',
required: true
},
recipient: {
type: mongoose.Schema.ObjectId,
ref: 'User'
},
timestamp: {
type: Date,
default: Date.now
},
activity: {
type: String,
required: true
},
event: {
type: mongoose.Schema.ObjectId,
ref: 'Event'
},
comment: {
type: mongoose.Schema.ObjectId,
ref: 'Comment'
}
});
When I query for them, I am populating the actor, recipient, event, and comment fields (all the references). After that, I also deep-populate the event field to get event.creator. Here is my code for the query:
var activityPopulateObj = [
{ path: 'event' },
{ path: 'event.creator' },
{ path: 'comment' },
{ path: 'actor' },
{ path: 'recipient' },
{ path: 'event.creator' }
],
eventPopulateObj = {
path: 'event.creator',
model: User
};
Activity.find({ $or: [{recipient: user._id}, {actor: {$in: user.subscriptions}}, {event: {$in: user.attending}}], actor: { $ne: user._id} })
.sort({ _id: -1 })
.populate(activityPopulateObj)
.exec(function(err, retrievedActivities) {
if(err || !retrievedActivities) {
deferred.reject(new Error("No events found."));
}
else {
User.populate(retrievedActivities, eventPopulateObj, function(err, data){
if(err) {
deferred.reject(err.message);
}
else {
deferred.resolve(retrievedActivities);
}
});
}
});
This is already a relatively complex query, but I need to do even more. If it hits the part of the $or statement that says {actor: {$in: user.subscriptions}}, I also need to make sure that the event's privacy field is equal to the string public. I tried using $elemMatch, but since the event has to be populated first, I couldn't query any of its fields. I need to achieve this same goal in multiple other queries, as well.
Is there any way for me to achieve this further filtering like I have described?

The answer is to change your schema.
You've fallen into the trap that many devs have before you when coming into document database development from a history of using relational databases: MongoDB is not a relational database and should not be treated like one.
You need to stop thinking about foreign keys and perfectly normalized data and instead, keep each document as self-contained as possible, thinking about how to best embed relevant associated data within your documents.
This doesn't mean you can't maintain associations as well. It might mean a structure like this, where you embed only necessary details, and query for the full record when needed:
var activitySchema = new mongoose.Schema({
event: {
_id: { type: ObjectId, ref: "Event" },
name: String,
private: String
},
// ... other fields
});
Rethinking your embed strategy will greatly simplify your queries and keep the query count to a minimum. populate will blow your count up quickly, and as your dataset grows this will very likely become a problem.

You can try below aggregation. Look at this answer: https://stackoverflow.com/a/49329687/12729769
And then, you can use fields from $addFields in your query. Like
{score: {$gte: 5}}

but since the event has to be populated first, I couldn't query any of its fields.
No can do. Mongodb cannot do joins. When you make a query, you can work with exactly one collection at a time. And FYI all those mongoose populates are additional, distinct database queries to load those records.
I don't have time to dive into the details of your schema and application, but most likely you will need to denormalize your data and store a copy of whatever event fields you need to join on in the primary collection.