Develop Reference

MongoDB Aggregate is not matching specific field

I'm new to Aggregation in MongoDB and I'm trying to understand the concepts of it by making examples.
I'm trying to paginate my subdocuments using aggregation but the returned document is always the overall values of all document's specific field.
I want to paginate my following field which contains an array of Object IDs.
I have this User Schema:
const UserSchema = new mongoose.Schema({
username: {
type: String,
unique: true,
required: true
},
firstname: String,
lastname: String,
following: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}],
...
}, { timestamps: true, toJSON: { virtuals: true }, toObject: { getters: true, virtuals: true } });
Without aggregation, I am able to paginate following,
I have this route which gets the user's post by their username
router.get(
'/v1/:username/following',
isAuthenticated,
async (req, res, next) => {
try {
const { username } = req.params;
const { offset: off } = req.query;
let offset = 0;
if (typeof off !== undefined && !isNaN(off)) offset = parseInt(off);
const limit = 2;
const skip = offset * limit;
const user = await User
.findOne({ username })
.populate({
path: 'following',
select: 'profilePicture username fullname',
options: {
skip,
limit,
}
})
res.status(200).send(user.following);
} catch (e) {
console.log(e);
res.status(500).send(e)
}
}
);
And my pagination version using aggregate:
const following = await User.aggregate([
{
$match: { username }
},
{
$lookup: {
'from': User.collection.name,
'let': { 'following': '$following' },
'pipeline': [
{
$project: {
'fullname': 1,
'username': 1,
'profilePicture': 1
}
}
],
'as': 'following'
},
}, {
$project: {
'_id': 0,
'following': {
$slice: ['$following', skip, limit]
}
}
}
]);
Suppose I have this documents:
[
{
_id: '5fdgffdgfdgdsfsdfsf',
username: 'gagi',
following: []
},
{
_id: '5fgjhkljvlkdsjfsldkf',
username: 'kuku',
following: []
},
{
_id: '76jghkdfhasjhfsdkf',
username: 'john',
following: ['5fdgffdgfdgdsfsdfsf', '5fgjhkljvlkdsjfsldkf']
},
]
And when I test my route for user john: /john/following, everything is fine but when I test for different user which doesn't have any following: /gagi/following, the returned result is the same as john's following which aggregate doesn't seem to match user by username.
/john/following | following: 2
/kuku/following | following: 0
Aggregate result:
[
{
_id: '5fdgffdgfdgdsfsdfsf',
username: 'kuku',
...
},
{
_id: '5fgjhkljvlkdsjfsldkf',
username: 'gagi',
...
}
]
I expect /kuku/following to return an empty array [] but the result is same as john's. Actually, all username I test return the same result.
I'm thinking that there must be wrong with my implementation since I've only started exploring aggregation.

Mongoose uses a DBRef to be able to populate the field after it has been retrieved.
DBRefs are only handled on the client side, MongoDB aggregation does not have any operators for handling those.
The reason that aggregation pipeline is returning all of the users is the lookup's pipeline does not have a match stage, so all of the documents in the collection are selected and included in the lookup.
The sample document there is showing an array of strings instead of DBRefs, which wouldn't work with populate.
Essentially, you must decide whether you want to use aggregation or populate to handle the join.
For populate, use the ref as shown in that sample schema.
For aggregate, store an array of ObjectId so you can use lookup to link with the _id field.

Mongoose updateMany documents containing specific Array element

I've got a Mongoose schema set up as follows:
const mongoose = require('mongoose');
const TodoSchema = mongoose.Schema({
id: {
type: String,
required: true,
},
todos: {
type: Array,
required: true,
},
date: {
type: Date,
default: Date.now(),
},
});
module.exports = mongoose.model('todo', TodoSchema, 'todos');
Each of the elements in the todos Array is an Object and has the following format (example):
{
id: 1,
todo: "Do the dishes."
category: "Kitchen"
}
There are multiple documents in my Todo collection and they all contain the same list of Todos. If I wanted to update a specific Todo across ALL documents, I figure I need to use updateMany. I'm using the following in my Todo Update route to update all instances of a Todo:
const { todo } = req.body; // todo.todo contains "Clean the dishes." as an update
Todo.updateMany(
{
todos: { $elemMatch: { id: todo.id } },
},
{ $set: { todo: todo } }
);
I'm assigning the result of the above route code to a variable and console logging the result which comes back with:
{ ok: 0, n: 0, nModified: 0 }
What am I doing wrong? The passed todo id matches the id of a Todo in each of the Todos arrays.

First of all, for your object array, is recommendable create a schema too:
const subSchema = new mongoose.Schema({
id: Number,
todo: String,
category: String
})
const MongooseModel = new mongoose.Schema({
id: String,
todos: [subSchema],
date: Date
})
So now, your array object is defined.
And, the query question is something like that:
db.collection.update({
"todos.id": todo.id
},
{
"$set": {
"todos.$": {newTodo}
}
},
{
"multi": true
})
First, you look for all elements that match your criteria; that is: todos.id = todo.id, then you use $ operator to set all element that match the criteria with your object.
The last line multi is to updated all element that match.
Example playground here
Using moongoose, multi attribute is not neccessary because is set true by default using updateMany().
So moongose query should be something like that.
var update = await model.updateMany(
{
"todos.id": 1
},
{
"$set": {
"todos.$": {
"id": 20,
"todo": "newTodo",
"category": "newCategory"
}
}
})
And for this example data the result is
{ n: 3, nModified: 3, ok: 1 }

Is there a way to find documents in nested array in mongoDB

const userSchema = new Schema({
username: {
type: String,
required: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
},
posts: [{
type: Schema.Types.ObjectId,
ref: 'Post'
}],
friends: [{
type: Schema.Types.ObjectId,
ref: 'User'
}],
});
// Exporting the schema so it can be accessed by requiring it.
module.exports = mongoose.model('User', userSchema);
As you can see I got this user schema that has a friends array and a posts array.
User.findById(userId).then(result => {
Post.find(query).then(posts => {
res.status(200).json(posts)
}).catch(err => {
if (!err.statusCode) {
err.statusCode = 500;
}
next(err);
})
});
Is there any query that can fit in the find() above in order to get all the posts of the user's friends?

If in the post model you have a link to the user model, that is, some field that identifies who wrote the post, you could use a for loop to search for posts made by the user's friends.
I don't know if this is the best solution but I hope it helps.
As a tip, you should use asynchronous syntax instead of promises, this helps when correcting errors.
async function getFriendsPosts(req,res){
/*in this array we will store the
posts of the user's friends */
let posts = [];
try{
//we check if the user exists
let user = User.findById(req.params.id);
//if it doesn't exist we will send a message
if(!user) res.status(404).send("User not Found");
else{
/* here we compare the id of the friends with the id of
the friends with the "creator" field in the post model*/
for await(let friend of user.friends){
for await(let creator of Post.find()){
/* if there is a match we send
it to the post array*/
if(friend._id.equals(creator._id)){
posts.push(creator);
}
}
}
/*finally we send the array with the posts*/
res.send(posts);
}
}catch(err){
res.status(500).send("Internal Server Error");
}
}

If I suppose that the Post Schema is something like that
{
title: String,
content: String,
owner: { type: Schema.Types.ObjectId, ref: 'User'}
}
then we can use aggregate pipeline to get the friends posts of some user
something like that
db.users.aggregate([
{
$match: {
_id: "userId1" // this should be of type ObjectId, you need to convert req.params.id to ObjectId (something like: mongoose.Types.ObjectId(req.params.id) instead of 'userId1')
}
},
{
$lookup: {
from: "posts",
let: {
friendsIDs: "$friends"
},
pipeline: [
{
$match: {
$expr: {
$in: ["$owner", "$$friendsIDs"]
}
}
}
],
as: "friendsPosts"
}
}
])
you can test it here Mongo Playground
feel free to replace these 'userId1', 'userId2', ..., 'postId1, 'postId2', .. in this link with your real users and posts Ids
by this way, you got the friends posts of some user in one query rather than two queries
then the function will be something like that
User.aggregate([
{
$match: {
_id: mongoose.Types.ObjectId(req.params.id)
}
},
{
$lookup: {
from: "posts", // this should be the posts collection name, It may be 'Post' not 'posts', check it
let: {
friendsIDs: "$friends"
},
pipeline: [
{
$match: {
$expr: {
$in: ["$owner", "$$friendsIDs"]
}
}
}
],
as: "friendsPosts"
}
}
]).then(result => {
// the aggregate pipeline is returning an array
// but we are sure it will be an array of only one element as we are searching for only one user, so we can use result[0]
result = result || []; // double check the result array
result[0] = result[0] || {}; // double check the user object
var posts = result[0].friendsPosts; // here is the friends posts array
// return the posts array
res.json(posts);
})
hope it helps
Update
If we need to sort the firendsPosts, and then limit them
we can use the following
db.users.aggregate([
{
$match: {
_id: "userId1"
}
},
{
$lookup: {
from: "posts",
let: {
friendsIDs: "$friends"
},
pipeline: [
{
$match: {
$expr: {
$in: [
"$owner",
"$$friendsIDs"
]
}
}
}
],
as: "friendsPosts"
}
},
{
$unwind: "$friendsPosts" // unwind the array to get a stream of documents
},
{
$sort: {
"friendsPosts.createdAt": 1 // then sort the posts by the createdAt Date in ascending order
}
},
{
$group: { // then group the posts again after sorting
_id: "$_id",
friendsPosts: {
$push: "$friendsPosts"
}
}
},
{
$project: {
friendsPosts: {
$slice: ["$friendsPosts", 2] // this is to limit the posts
}
}
}
])
you can test it here Mongo Playground 2

Error when using Mongoose ArrayFilters to update nested subdocument

I have a Mongoose collection of which I want to update a nested subdocument.
The basic setup is this:
a parent entry (Map)
which contains an array of children (Phases)
each child consists of one or more grandchildren (Steps).
I want to be able to find a grandchild and update it. My approach is to find the parent (Map) that contains the grandchild and then update it.
These are my simplified schemas:
const phaseSchema = new mongoose.Schema(
{
name: String,
color: String,
steps: [
new mongoose.Schema(
{
name: String,
body: String,
entry: {
type: mongoose.SchemaTypes.ObjectId,
ref: 'entry',
default: null
}
},
{ timestamps: false }
)
]
},
{ timestamps: false, _id: true }
)
const mapSchema = new mongoose.Schema(
{
name: String,
phases: [phaseSchema]
},
{ timestamps: false }
)
export const Map = mongoose.model('map', mapSchema)
I'm trying to use ArrayFilters to find and update the subdocument, but to no luck:
req.body = {
map_id: 'some_mongoose_id',
step_id: 'some_other_mongoose_id'
}
const newEntryId = 'a_new_mongoose_id'
// Find the parent (Map) and update it
const UpdatedMap = await Map.update(
{ _id: req.body.map_id },
{
$set: {
'phases.$[i].steps.$[j].entry': mongoose.Types.ObjectId(newEntryId)
}
},
{
arrayFilters: [
{
'i.steps._id': mongoose.Types.ObjectId(req.body.step_id)
},
{
'j._id:': mongoose.Types.ObjectId(req.body.step_id)
}
]
}
)
This throws the following error:
Error: Could not find path "phases.0.steps.0._id:" in schema
Coincidentally, the phase and step I'm trying to update are both at index 0.
If I swap out the arrayFilters and hardcode the indexes, like so $set: {'phases.0.steps.0.entry': mongoose.Types.ObjectId(newEntryId)'}, it works.
What am I doing wrong?

The i is what's causing the error and it's also redundant. the nested arrayFilter will work as required without it.
const UpdatedMap = await Map.update(
{ _id: req.body.map_id },
{
$set: {
'phases.steps.$[j].entry': mongoose.Types.ObjectId(newEntryId)
}
},
{
arrayFilters: [
{
'j._id:': mongoose.Types.ObjectId(req.body.step_id)
}
]
}
)

Mongoose update multiple subdocuments

I'm trying to update one subdocument addresses (works) and then update many subdocuments except the previous one. Basically every time an address change is_preferred to true, it must update the previous address that is_preferred was true to false (i'm trying to update everyone except the address that changed to true).
User document
_id: ObjectId("5b996f0fd5fbf511709f668f");
addresses: [
{
_id: ObjectId("5ba33e0991cd7a3bb85dab7e");
is_preferred:true
},
{
_id: ObjectId("5ba3e9337310c637207b44cb");
is_preferred:false
}
]
Here is my solution:
// model
User = mongoose.model('user', new Schema({
_id: { type: mongoose.Schema.Types.ObjectId, required: true },
addresses: [
{
_id: { type: mongoose.Schema.Types.ObjectId, required: true },
is_preferred: { type: Boolean, required: true }
}
],
}, { collection: 'user' }););
// route
router.put('/updateAddress/:addressId', auth, user.updateAddress);
// user.js
exports.updateAddress = wrap(async(req, res, next) => {
// update one object address `is_preferred` to true and return an array 'addresses' containing it
const user = await User.findOneAndUpdate(
{ addresses: { $elemMatch: { _id: mongoose.Types.ObjectId(req.params.addressId) } } }, { 'addresses.$': req.body },
{ projection: {
addresses: {
$elemMatch: { _id: mongoose.Types.ObjectId(req.params.addressId) }
}
}, new: true }).lean();
if (user) {
// updated object `is_preferred` changed to true, so another objects must be false
if (user.addresses[0].is_preferred) {
// doesnt work
await User.update({ _id: { $ne: mongoose.Types.ObjectId(req.params.addressId) }, is_preferred: true },
{ $set: { addresses: { is_preferred: false } } }, { multi: true });
}
res.status(200).json({success: true, message: 'Saved.', new_object: user.addresses[0]});
} else {
res.status(400).json({success: false, message: 'Error.'});
}
});
I'm able to update the user subdocument addresses is_preferred to true. However updating another addresses is_preferred to false isn't working. What Am I doing wrong?

I would recommend for a scenario like yours to utilize the mongoose middleware pre or post schema hooks. The idea is that instead of dealing with this in your controller you would take care of it in your schema via that middleware.
The only inconvenience is that the pre and post hooks do not fire on findOneAndUpdate and you would need to do first find then update.
So you would do something like this for the post hook:
User.post('save', doc => {
// You can update all the rest of the addresses here.
});
Also for your update to work you need to do something like this:
User.update(
{ "addresses._id": { $ne: mongoose.Types.ObjectId(req.params.addressId) }},
{ $set: { 'addresses.0.is_preferred': false }},
{ multi: true }
)