I have this aggregation operation, and it's giving me the correct output, but with an inconsistent order. When I reload, the nested output array (posteriorThread) changes the order of the documents, and there seems to be no rhyme or reason!
I'm confused why the order keeps changing, and I would like to know why it's happening, but I figured I would just sort it, which I did, but I'm having trouble grouping it back together.
I'll show you both of my broken solutions below, but essentially I want output 1 but with the correct order. I'm using mongoose, but that shouldn't make a difference.
Thanks.
1: Inconsistent order solution
const posteriorThread = await Comment.aggregate([
{
$match: {
_id: post.threadDescendant,
},
},
{
$graphLookup: {
from: 'comments',
startWith:'$threadDescendant',
connectFromField: 'threadDescendant',
connectToField: '_id',
as: 'posteriorThread',
},
},
]);
OUTPUT: 1
posteriorThread [
{
"_id": "000",
"name": "foo bar",
"text": "testing one",
"threadDescendant": "123",
"posteriorThread": [
{
"_id": "234",
"name": "foo bar",
"text": "testing four",
"threadDescendant": "345"
},
{
"_id": "345",
"name": "foo bar",
"text": "testing three",
},
{
"_id": "123",
"name": "foo bar",
"text": "testing two",
"threadDescendant": "234"
},
]
}
]
2: Correct older but lose root document
const posteriorThread = await Comment.aggregate([
{
$match: {
_id: post.threadDescendant,
},
},
{
$graphLookup: {
from: 'comments',
startWith: '$threadDescendant',
connectFromField: 'threadDescendant',
connectToField: '_id',
as: 'posteriorThread',
},
},
{
$unwind: '$posteriorThread',
},
{
$sort: { 'posteriorThread.depth': 1 },
},
{
$group: { _id: '$_id', posteriorThread: { $push: '$posteriorThread' } },
},
]);
OUTPUT 2:
posteriorThread [
{
"_id": "000",
"posteriorThread": [
{
"_id": "123",
"name": "foo bar",
"text": "testing two",
"threadDescendant": "234"
},
{
"_id": "234",
"name": "foo bar",
"text": "testing four",
"threadDescendant": "345"
},
{
"_id": "345",
"name": "foo bar",
"text": "testing three",
},
]
}
]
The $graphLookup pipeline stage doesn't offer any built-in sorting capability, thus your second approach is correct. You just need to use $first in order to preserve root object's fields. You can use $replaceRoot and special $$ROOT variable to avoid specifying each field explicitly:
{
$group: {
_id: "$_id",
posteriorThread: { $push: "$posteriorThread" },
root: { $first: "$$ROOT" }
}
},
{
$project: {
"root.posteriorThread": 0
}
},
{
$replaceRoot: {
newRoot: {
$mergeObjects: [
{ posteriorThread: "$posteriorThread" },
"$root"
]
}
}
}
Mongo Playground
Related
I have this data stored in database.
{
"_id": "62fa5aa25778ec97bc6ee231",
"user": "62f0eb5ebebd0f236abcaf9d",
"name": "Marketing Plan",
"columns": [
{
"name": "todo",
"_id": "62fa5aa25778ec97bc6ee233",
"tasks": [
{
"title": "Task Four testing 2",
"description": "This is task four",
"subtasks": [
{
"name": "wash dshes test",
"completed": false,
"_id": "62ff74bfe80b11ade2d34456"
},
{
"name": "do homework",
"completed": false,
"_id": "62ff74bfe80b11ade2d34457"
}
],
"_id": "62ff74bfe80b11ade2d34455"
}
]
},
{
"name": "doing",
"_id": "62fa5aa25778ec97bc6ee234",
"tasks": []
},
{
"name": "done",
"_id": "62fa5aa25778ec97bc6ee235",
"tasks": []
}
],
"__v":0
}
I want to be able to return a single object with the id equal to the req.params.id, in this case that would be 62ff74bfe80b11ade2d34455.
{
"title": "Task Four testing 2",
"description": "This is task four",
"subtasks": [
{
"name": "wash dshes test",
"completed": false,
"_id": "62ff74bfe80b11ade2d34456"
},
{
"name": "do homework",
"completed": false,
"_id": "62ff74bfe80b11ade2d34457"
}
],
"_id": "62ff74bfe80b11ade2d34455"
}
I researched stackoverflow and came across this potential solution: Mongoose retrieve one document from nested array which implemented the aggregate framework. But when I test this in postman, the request isn't made.
const getTask = asyncHandler(async (req, res) => {
const task = await Board.aggregate([
{
$match: {
"columns.tasks._id": req.params.id,
},
},
{
$project: {
columns: {
$first: {
$filter: {
input: "$columns.tasks",
cond: {
$eq: ["$$this._id", req.params.id],
},
},
},
},
},
},
{
$replaceRoot: {
newRoot: "$columns",
},
},
]);
});
Having an array inside an array complicates the query a bit, but here's one way to retrieve the data you want.
db.Board.aggregate([
{
$match: {
"columns.tasks._id": req.params.id
}
},
{"$unwind": "$columns"},
{
$match: {
"columns.tasks._id": req.params.id
}
},
{
"$project": {
"task": {
"$first": {
"$filter": {
"input": "$columns.tasks",
"cond": {"$eq": ["$$this._id", req.params.id]}
}
}
}
}
},
{"$replaceWith": "$task"}
])
Try it on mongoplayground.net. [The mongoplayground.net example uses "62ff74bfe80b11ade2d34455" rather than req.params.id.]
I'm making a kanban task management app and I'm trying to remove a task with the _id: req.params.id which has the value of 62fa5ae05778ec97bc6ee23a. I tried the following:
const task = await Board.findOneAndUpdate(
{
"columns.tasks._id": req.params.id,
},
{ $pull: { "columns.$.tasks.$._id": req.params.id } },
{ new: true }
);
But I get the error Too many positional (i.e. '$') elements found in path'columns.$.tasks.$._id'
I searched for a while and came across arrayFilters from the docs but I'm struggling a lot to understand how to implement it for this particular need.
{
"_id": "62fa5aa25778ec97bc6ee231",
"user": "62f0eb5ebebd0f236abcaf9d",
"name": "Marketing Plan",
"columns": [
{
"name": "todo",
"_id": "62fa5aa25778ec97bc6ee233",
"tasks": [
{
"title": "Task Four",
"description": "This is task four",
"subtasks": [
{
"name": "wash dshes",
"completed": false,
"_id": "62fa5ae05778ec97bc6ee23b"
},
{
"name": "do homework",
"completed": false,
"_id": "62fa5ae05778ec97bc6ee23c"
}
],
"_id": "62fa5ae05778ec97bc6ee23a"
}
]
},
{
"name": "doing",
"_id": "62fa5aa25778ec97bc6ee234",
"tasks": []
},
{
"name": "done",
"_id": "62fa5aa25778ec97bc6ee235",
"tasks": []
}
],
"__v": 0
}
You need to use $[] positional operator in order to pull from the nested array. Try running this query:
db.Board.updateOne({
"_id" : "62fa5aa25778ec97bc6ee231",
}, {
$pull: { 'columns.$[].tasks': { '_id': '62fa5ae05778ec97bc6ee23a' } }
});
I have classroom document which contain 'modules', the whole document looks like this:
{
"_id": "628a7ea21e2a666d7872efbf",
"name": "Test Class",
"owner": "60763491b98b9e186ef33137",
"schoolId": "607dff27c712219af1e65d83",
"description": "This is a test class.",
"roster": [],
"modules": [
{
"name": "Test Module 1",
"id": "62a7082d0bf84c43fdfe95ff",
"isPublished": false
},
{
"name": "Test Module 2",
"id": "62a72d6378ce044dca32e1a2",
"isPublished": false
}
]
}
I also have assignment documents as such:
{
"classroomId": "628a7ea21e2a666d7872efbf",
"moduleId": "62a7082d0bf84c43fdfe95ff",
"name": "Assignment 1",
"description": "Test description",
"created": 1655120822055,
"reading": null,
"questions": [],
"isPublished": true,
"_id": "62a723b6683ffc4b11940c7b"
}
My question is how could do aggregation such that if I want to do a lookup on the assignments for the classroom, I am able to group the assignment documents by moduleId and then add them as a field to the modules array. The final document would like this:
{
"_id": "628a7ea21e2a666d7872efbf",
"name": "Test Class",
"owner": "60763491b98b9e186ef33137",
"schoolId": "607dff27c712219af1e65d83",
"description": "This is a test class.",
"roster": [],
"modules": [
{
"name": "Test Module 1",
"id": "62a7082d0bf84c43fdfe95ff",
"isPublished": false,
"assignments": [
{
"_id": "62a708ab0bf84c43fdfe9600",
"classroomId": "628a7ea21e2a666d7872efbf",
"moduleId": "62a7082d0bf84c43fdfe95ff",
"name": "Assignment 1",
"description": "Test description",
"created": 1655113899629,
"due": null,
"settings": null,
"reading": null,
"questions": [],
"isPublished": true
},
{
"_id": "62a723b6683ffc4b11940c7b",
"classroomId": "628a7ea21e2a666d7872efbf",
"moduleId": "62a7082d0bf84c43fdfe95ff",
"name": "Assignment 1",
"description": "Test description",
"created": 1655120822055,
"due": null,
"settings": null,
"reading": null,
"questions": [],
"isPublished": true
}
]
},
{
"name": "Test Module 2",
"id": "62a72d6378ce044dca32e1a2",
"isPublished": false,
"assignments": [
]
}
]
}
Right now I just have the base lookup and obviously this just gets me a separate assignments field, without groups.
lookup = [
{
$lookup: {
from: "assignments",
localField: "modules.id",
foreignField: "moduleId",
as: "assignments"
}
}
];
$lookup
$set - Set modules field.
2.1. $map - Iterate the modules array and returns a new array.
2.1.1. $mergeObject - Merge current iterate module document with the document with assignments array from the result 2.1.1.1.
2.1.1.1. $filter - Filter the assignments array by matching moduleId.
$unset - Remove assignments field.
db.classroom.aggregate([
{
$lookup: {
from: "assignments",
localField: "modules.id",
foreignField: "moduleId",
as: "assignments"
}
},
{
$set: {
modules: {
$map: {
input: "$modules",
as: "module",
in: {
$mergeObjects: [
"$$module",
{
"assignments": {
$filter: {
input: "$assignments",
as: "asgn",
cond: {
$eq: [
"$$module.id",
"$$asgn.moduleId"
]
}
}
}
}
]
}
}
}
}
},
{
$unset: "assignments"
}
])
Sample Mongo Playground
I need to retrieve the entire single object hierarchy from the database as a JSON. Actually the proposal about any other solution to achive this result would be highly appriciated. I decided to use MongoDB with its $lookup support.
So I have four collections:
Users
{ "_id" : "2", "name" : "john" }
{ "_id" : "1", "name" : "Doe" }
Posts
{"_id": "2","body": "hello", likes: []},
{"_id": "1","name": "hello 4", likes: [1,]},
Comments
{"_id": "c2","body": "hello 3",postId: "1",likes: [1,2]},
{"_id": "c1","body": "hello 2",postId: "1",likes: [1,2]},
Replies
{"_id": "r1","name": "hello 4",commentId: "c1",likes: [1]},
{"_id": "r3","name": "hello five",commentId: "c2",likes: [1,2]}
I basically need to retrieve all posts with all corresponding comments and comments.replies as part of my result . My aggregation:
const posts = await PostModel.aggregate([
{
$lookup: {
from: "comments",
localField: "_id",
foreignField: "postId",
as: "comments",
},
},
{
$unwind: "$comments",
},
{
$lookup: {
from: "replies",
localField: "comments._id",
foreignField: "commentId",
as: "comments.replies",
},
},
]).sort({
createdAt: -1,
});
MongoDb PlayGround
The result is pretty weird. Some records are ok. But comments return an object . There are also some duplications on the post with _id="1".
[
{
"_id": "1",
"comments": {
"_id": "c2",
"body": "hello 3",
"likes": [
1,
2
],
"postId": "1",
"replies": [
{
"_id": "r3",
"commentId": "c2",
"likes": [
1,
2
],
"name": "hello five"
}
]
},
"likes": [
1
],
"name": "hello 4"
},
{
"_id": "1",
"comments": {
"_id": "c1",
"body": "hello 2",
"likes": [
1,
2
],
"postId": "1",
"replies": [
{
"_id": "r1",
"commentId": "c1",
"likes": [
1
],
"name": "hello 4"
}
]
},
"likes": [
1
],
"name": "hello 4"
}
]
Brief, This is my expected result.
I want to get all posts, with all comments and replies associated with them .
I want to append a count of likes likesCount:{$size:["likes"]} and since I have the user auth id(uid) ready I want to check if the user liked the post , comment or reply based on if isLiked: {$in:[ID(uid),"likes"]}
Since each post have multiple comments, after unwinding comments you need to group it together to form an array
Update
I have updated the fetch approach a lil bit like the below.
db.posts.aggregate([
{
$lookup: {
from: "comments",
localField: "_id",
foreignField: "postId",
as: "comments",
},
},
{
$unwind: "$comments",
},
{
$lookup: {
from: "replies",
localField: "comments._id",
foreignField: "commentId",
as: "replies",
},
},
{
$unwind: "$replies",
},
{
"$addFields": {
"replies.countOflikes": {
$size: {
$ifNull: [
"$replies.likes",
[]
]
}
},
"replies.isLiked": {
$cond: {
if: {
$eq: [
{
$size: {
$filter: {
input: "$replies.likes",
as: "item",
cond: {
$eq: [
"$$item",
1//id of the user whom you wanna check if liked the reply
]
}
}
}
},
0
]
},
then: false,
else: true
}
}
}
},
{
$group: {
_id: "$comments._id",
postId: {
$first: "$_id"
},
body: {
$first: "$body"
},
"comments": {
$first: "$comments"
},
replies: {
$push: "$replies"
}
}
},
{
$addFields: {
"comments.replies": "$replies"
}
},
{
$group: {
_id: "$postId",
body: {
$first: "$body"
},
comments: {
$push: "$comments"
}
}
}
])
Summary of the change
Unwinded both comments and it's replies
Added new fields for displaying isLiked and countOfLikes using addFields stage
grouped twice to reform original structure of the data(first grouped by comments then posts)
https://mongoplayground.net/p/lymCfeIIy9j
I am trying to retrieve some lookup data for an embedded array in a document.
Here is a sample of the data:
{
"_id": "58a4fa0e24180825b05e14e9",
"fullname": "Test User",
"username": "testuser"
"teamInfo": {
"challenges": [
{
"levelId": "5e14e958a4fa0",
"title": "test challenge 1.1"
},
{
"levelId": "5e14e958a4fa0",
"title": "test challenge 1.2"
},
{
"levelId": "5e14e958a4fa1",
"title": "test challenge 2.1"
}
]
}
}
As you see, teamInfo.challenges is an array, containing levelId fields. These are pointing to the _id field in another collection called levels.
But how can I do to getting json response like this?
{
"_id": "58a4fa0e24180825b05e14e9",
"fullname": "Test User",
"username": "testuser"
"teamInfo": {
"challenges": [
{
"levelInfo": {
"name": "Level 1"
},
"title": "test challenge 1.1"
},
{
"levelInfo": {
"name": "Level 1"
},
"title": "test challenge 1.2"
},
{
"levelInfo": {
"name": "Level 2"
},
"title": "test challenge 2.1"
}
]
}
}
Im trying using unwind, project, and group. But im so confused.
const user = await User.aggregate([
{
$match: {_id: new mongoose.Types.ObjectId(req.user.userId)}
},
{
$lookup: {
from: 'levels',
localField: 'teamInfo.challenges.levelId',
foreignField: '_id',
as: 'challLevelInfo'
}
},
{
$group: {
_id: "$_id",
........IM CONFUSED HERE........
}
}
]);
You can use lookup pipeline to handle nested lookup
const pipeline = [
{
$match: {_id: new mongoose.Types.ObjectId(req.user.userId)}
},
{
$lookup: {
from: 'levels',
let: { level_id: "$teamInfo.challenges.levelId" },
pipeline: [
{
$match: {
$expr: {
$eq: ["$_id", "$$level_id"]
}
}
},
{
$lookup: {
from: '<level collection>',
localField: "levelId",
foreignField: "_id",
as: "levelInfo"
}
},
{
$project: {
levelInfo: {
name: "$levelInfo.name"
}
title: 1
}
}
],
as: "challenges"
},
},
{ $project: {
_id: 1,
fullname: 1,
username: 1,
teamInfo: {
challenges: "$challenges"
}
}}
]
const result = await User.Aggregate(pipeline)
hope this help !