Develop Reference

How to get expected output from MongoDB?

I'm new to MongoDB aggregation. I am not getting desired output
The output I'm getting from aggregation:-
[
{tweet:{key:value}},
{tweet:{key:value}},
{tweet:{key:value}},
{tweet:{key:value}},
]
but I want the following output from the pipeline:-
[
{key:value},
{key:value},
{key:value},
]
and lastly, pipeline I'm running:-
const pipeline = [[
{
$match: {
$expr: {
$in: [
Mongoose.Types.ObjectId(userid), '$likedBy.user'
]
}
}
}, {
$lookup: {
from: 'tweets',
localField: 'tweet',
foreignField: '_id',
as: 'tweet'
}
}, {
$unwind: {
path: '$tweet'
}
}, {
$lookup: {
from: 'users',
localField: 'tweet.user',
foreignField: '_id',
as: 'user'
}
}, {
$unwind: {
path: '$user'
}
}, {
$addFields: {
'tweet.user': '$user'
}
},
{
$addFields: {
'tweet.isLiked': true,
}
},{
$project:{
tweet:1,
}
},
]
];
const likedTweets = await TweetLike.aggregate(pipeline)
I know I can do this with javascript but I want to do it with the pipeline

You can replace your last project stage with the following to achieve what you need:
{$project:{key:"$tweet.key"}}

Answering my own question
i wanted to return sub-document so i found this https://stackoverflow.com/a/43411988/12332711
all i had to do is use
{
$replaceRoot: {newRoot: "$tweet"}
}
it worked for me

Group by Date part only (without the time) in MongoDB + NodeJS

Suppose we have the query :
EightWeekGamePlan.aggregate(
[
{
$group: {
_id: {
LeadId: "$LeadId",
Week: "$Week",
InsertDate: "$InsertDate" , // I want to group by the date part
Status: "$Status"
},
count: { $count: 1 }
}
},
{
$lookup: {
from: "leads",
localField: "_id",
foreignField: "LeadId",
as: "Joined"
}
},
{ $unwind: "$Joined" },
{ $replaceRoot: { newRoot: { $mergeObjects: ["$Joined", "$$ROOT"] } } },
{ $sort: { total: -1 } }
],
function(err, results) {
if (err) {
console.log(err);
}
// ... do some manipulations ...
console.log(_filtered);
return res.json(_filtered);
}
);
I grouping by multiple fields and I want to take only the date part of InsertDate and disregard the time.
How can we do that ?

I believe your question is addressed in mongodb documentations under Group by Day of the Year:
https://docs.mongodb.com/manual/reference/operator/aggregation/group/
You have to convert the date into date-formatted string using $dateToString and add it to $group _id
_id : {$dateToString: { format: "%Y-%m-%d", date: "$InserDate" }}
I hope this helps!

How to use aggregate to also include the documents that aren't being returned from that method?

Currently I have this function which retrieves all Tag documents used in all Page documents and counts the number of occurrences and attached them to each Tag document returned:
exports.getAll = (req, res, next) => {
const config = utils.prepareOptions(req.query);
async.waterfall([
(done) => {
Tag.count({}).exec((err, total) => {
return done(err, total);
});
}
], (err, total) => {
Page.aggregate([
// {
// $unwind: "$tags"
// },
{
$group: {
_id: "$tags",
occurrences: {
$sum: 1
}
}
},
{
$lookup: {
from: "tags",
localField: "_id", // this is supposely wrong but I can't prove it
foreignField: "_id",
as: "tagsData"
}
},
{
$unwind: "$tagsData"
},
{
$match: {
"tagsData.label": new RegExp(config.query, 'i')
}
},
{
$project: {
occurrences: "$occurrences",
tagData: '$tagsData'
}
},
{
$addFields: {
"tagData._id": "$_id",
"tagData.occurrences": "$occurrences"
}
},
{
$replaceRoot: {
newRoot: "$tagData"
}
},
{$sort: {[config.sortBy]: config.order}},
{$limit: config.limit},
{$skip: config.offset}
], (err, tags) => {
console.log(tags);
console.log(err);
console.log(total);
if (err) {
return next(err);
}
res.json({
error: null,
data: tags,
total: total,
results: tags.length
});
});
});
};
The problem with this is that a Tag doesn't necessarily have to be used with a Page at any given time. This results in a problem when calling this function because the Tags that aren't used are not returned.
So what I need to do is to also include every Tag that isn't used and set an occurrences: 0 on them.
So if I have 3 Tag documents:
[
{_id: 1203f8sad9sf, label: 'Tag 1'},
{_id: 1203asdf89sf, label: 'Tag 2'},
{_id: 1203asqw89sf, label: 'Tag 3'}
]
And a Page document:
{
_id: 90182312,
tags: [ObjectId(1203f8sad9sf), Object(1203asdf89sf)]
}
And Tag 1 and Tag 2 are part of the Page's tags array, how do I also return Tag 3 so that it is included in the tags that is returned from the aggregate method?

Given I understand your intention correctly and also assuming that you've got some code in place to make sure that there are no duplicate tag IDs in a page's tags field this query can be rewritten in a substantially more efficient and concise way:
db.getCollection('tags').aggregate([{
$lookup: {
from: "page",
localField: "_id",
foreignField: "tags",
as: "pageData"
}
}, {
$addFields: {
"occurrences": {
$size: "$pageData"
}
}
}])

using max query with mongoose

I am new with mongoose and still trying to understand how make correct queries
I have 2 simple Models
User :
const UserSchema = new Schema({
name: String,
age: Number,
movies:[{
type: Schema.Types.ObjectId,
ref: 'movie'
}]
}, { collection: 'USER_COLLEC' });
Movie :
const MovieSchema = new Schema({
title:String ,
duration: Number
}, { collection: 'MOVIE_COLLEC' });
What I want is the user with le longest movie ( highest duration )
For now I got that :
db.getCollection('USER_COLLEC') .
aggregate([
{ "$unwind": "$movies" } ,
{ $lookup:
{from: "MOVIE_COLLEC",
localField: "movies",
foreignField: "_id",
as: "movieContent"},
} ,
{ $unwind: "$movieContent" },
{ $group:
{ maxDuration: { $max: "$movieContent.duration" },
}
}
])
But it will only find the max duration with no user attached to it...
And indeed I only ask for the max duration on my query, but after the lookup I lose my user :(
How can I can keep it, or retrieve my user data ?
If you have any idea, I am completely stuck...
Thanks guys !

you can use $push to get the movie object as well.
db.getCollection('USER_COLLEC') .
aggregate([
{ "$unwind": "$movies" } ,
{ $lookup:
{from: "MOVIE_COLLEC",
localField: "movies",
foreignField: "_id",
as: "movieContent"},
} ,
{ $unwind: "$movieContent" },
{ $group:
{ _id: { $max: "$movieContent.duration" },
"movie": {
"$push": "movieContent"
}
}
}
])
After this, just get search for the Movie's _id in the user's movies array
UserSchema.find({movies:{$in:movieContent[0]._id}});
OR, instead of $push you can also use $first
{ $first: "$movieContent" }
Then you won't get it in an array.
Update:
Instead of {$push: $movieContent} or{$first: $movieContent}, you could just push $$ROOT:
{$push: $$ROOT} or {$first: $$ROOT}
and then you'll get the entire object. You don't need to fire another query to get the user.

I finally managed to find the solution, the $group was not the solution
db.getCollection('USER_COLLEC') .
aggregate([
{ "$unwind": "$movies" } ,
{ $lookup:
{from: "MOVIE_COLLEC",
localField: "movies",
foreignField: "_id",
as: "movieContent"},
} ,
{ $unwind: "$movieContent" },
{$sort: {"movieContent.duration":-1}},
{ $project: { "user":"$name","duration" : "$movieContent.duration"} } ,
{ $limit : 1 }
Which gives me something like :
{
"_id" : ObjectId("59d2f64dded1c008192f7e73"),
"user" : "Michael",
"duration" : 96
}

aggregation project mongodb output not expected

I'm coding a complex query in javascript.
and I'm using aggregation.
I'm using two tables. invoice, travel
this is my code.
Invoice.aggregate([
// filter the documents from invoice of only 2016
{
$match: {
executed: {
$gte: startDate,
$lte: endDate
},
"modelHolder.name": 'Travel'
}
},
// $lookup is working alone, it is not taking the input from function 1 of aggregate
{
$lookup: {
from: "travels",
localField: "modelHolder.id",
foreignField: "_id",
as: "dataFromTravels"
}
},
// filter by date of reservation and date of arrival
$match: { $or:
[
{
'dataFromTravels.from.date': {
$gte: departDate, $lte: endDate
}
},
{
'dataFromTravels.to.date': {
$lte: arrivalDate
},
}
]
}
},
{
$limit: 2
}
// 2nd function to work on the first function output as the input
], function (err, result) {
if (err) {
return console.log(('ERROR', err));
//next(err);
} else {
console.log('Result', result);
// res.json(result);
return;
}
});
// console.log('invoice !');
// console.log(invoice._id);
self.resume();
})
.on('error', function (err) {
console.error('Error occurred while streaming invoices', err);
})
.on('end', function () {
console.log('Successfully displayed invoices');
cb();
});
I want to have the list of cars and count how many times we used each of them in the period restricted in the match functions.
I added this.
{
$group: { "_id": "$dataFromTravels.car.plateNumber", "count": { $sum: 1 } }
},
the car is under travel table.
and I just got an array of 3 cars. or I have hundreds of cars in the same period.
How could I manage this?
thanks for you suggestions.

When you make a join query using mongodb. The target collection is stored as an array. so when you make use of lookup
{
$lookup: {
from: "travels",
localField: "modelHolder.id",
foreignField: "_id",
as: "dataFromTravels"
}
}
The output of the query will be like:-
{
"_id": ObjectId("554dc5e937c1482c491d9d36"),
// invoice data
"dataFromTravels": [
// travel data
]
}
And with mongodb aggregation you cannot use $match over nested documents.
So, in the query to match
db.invoice.aggregate([
{
$match: {
executed: {
$gte: ISODate(startDate),
$lte: ISODate(endDate)
},
"modelHolder.name": 'Travel'
}
},
{
$lookup: {
from: "travels",
localField: "modelHolder.id",
foreignField: "_id",
as: "dataFromTravels"
}
}, {
$unwind: "$dataFromTravels"
},{
$match: {
$or: [{
'dataFromTravels.from.date': {
$gte: ISODate(startDate),
$lte: ISODate(endDate)
}
}, {
'dataFromTravels.to.date': {
$lte: ISODate(startDate)
},
}]
}
}, {
$limit: 2
}
]);
Now, the dataFromTravels will be available as an object & you can use $match over the from & to properties