Develop Reference

how to use mongoose aggregation to get sum of two matching documents depending on field

I have two collections "Employee", "Office"
I am trying to find how many employees are in each area which contains office code. But there might be more than one office in the same area.
This is how my Office documents might look like
[
{
_id: "5b7d0f77e231b6b530b0ee5a",
code: "OB123456",
city: "Canmore"
// some other fields
},
{
_id: "5b7d0f77e531b6b530b0ee5b",
code: "OB858758",
city: "Vancouver"
},
{
_id: "5b7d0f77e531b6b530b0ee5d",
code: "EE858758",
city: "Vancouver"
},
]
this is how my Employee documents might look like
[
{
_id: "5b7d0f77e531b6b530b0edda",
name: 'Charlie',
office: {
code: 'OB123456'
// some other fields
}
},
{
_id: "5b7d0f73e531b6b530b0ee5b",
name: 'Bill',
office: {
code: 'EE858758'
}
},
{
_id: "5b7d0f77e531b6b530b0ee5n",
name: 'Echo',
office: {
code: 'OB123456'
}
},
];
I am looking into mongoose aggregate, and only tried
await Employee.aggregate([
{
$lookup: {
from: 'offices',
localField: 'office.code',
foreignField: 'code',
as: 'officeCode'
},
$group: {
_id: 'officeCode.city',
count: { $sum: 1 }
}
}
]);
which for sure does not work, I tried reading some of the aggregation documention but cannot come up with a good idea how to get this done
Thanks in advance for any suggestions or advices.
Sample output of what I am looking for
{
"Vancouver": 1,
"Canmore": 2
}

You have to start from office instead of employee, so you can create a list of code for each area (city), then lookup to map with your employees.
db.office.aggregate([
{
$group: {
_id: "$city",
codes: {
$addToSet: "$code"
}
}
},
{
$lookup: {
from: "employee",
localField: "codes",
foreignField: "office.code",
as: "employees"
},
},
{
$group: {
_id: null,
data: {
$push: {
k: "$_id",
v: {
$size: "$employees"
}
}
}
}
},
{
$replaceRoot: {
newRoot: {
"$arrayToObject": "$data"
}
}
}
])
The two last stages are here only to format your result as described in your expected output.
You can test it here

Mongoose custom sort with date

How can I retrieve data with a custom sort in Mongoose?
There is a job starting date that needs to be sorted by the month and year, but currently this script is only sorting from December to January.
router.get('/', (req, res) => {
Job.find()
.sort({ from: -1 })
.then(jobs => res.json(jobs))
.catch(err => res.status(404).json(err));
});
The problem is in the sort; values for from is like 12.2018, 06.2019, 03.2020, 11.2009 and so on.
I want to sort these results first from the year (which is after the dot) and then sort from the months. I cannot currently change how the data is set and it's stored as a String in the model Schema.

You have to use aggregation framework to first transform your string to a valid date by
$spliting it,
$convert parts from string to int
and using $dateFromParts,
then you sort and finally remove created field.
Here's the query :
db.collection.aggregate([
{
$addFields: {
date: {
$dateFromParts: {
year: {
$convert: {
input: {
$arrayElemAt: [
{
$split: [
"$from",
"."
]
},
1
]
},
to: "int"
}
},
month: {
$convert: {
input: {
$arrayElemAt: [
{
$split: [
"$from",
"."
]
},
0
]
},
to: "int"
}
},
}
}
}
},
{
$sort: {
date: -1
}
},
{
$project: {
date: 0
}
}
])
You can test it here

get the latest document after $group by

I'm using the official MongoDB driver for Node.js.
And this is how my message data is structured. As you can see, every post has a timestamp, an userId and the id of the topic.
[
{
"_id" : ObjectId("5b0abb48b20c1b4b92365145"),
"topicId" : "XN7iqmCFD4jpgJZ6f",
"timestamp" : 1527429960,
"user" : "5b0869636f4e363e300d105a",
"content" : "lorem ipsum"
}
]
Now I need to check if there are topics, which newest post (=highest timestamp) doesn't match my own ID.
With that I do know which topic has a new answer (which is not my own post).
So I started with this:
db.messages.find({
$query: {
user: { $ne: "myUserId" }
},
$orderby: {
timestamp: -1
}
}).limit(1).toArray()
My problem is, that I do not know how to group my query by the topicId. And somehow there seems to be wrong syntax in my attempt.

You have to use aggregate to group by topicId and then $sort for sorting according to time and then $limit to limit the query
db.messages.aggregate([
{ $match: { user: { $ne: "myUserId" } }},
{ $group: {
_id: "$topicId",
timestamp: { $first: "$timestamp"},
user: { $first: "$user" },
content: { $first: "$content" }
}},
{ $sort: { timestamp: -1 } },
{ $limit: 1 }
])

Use aggregate pipeline to do this instead of find.
1) $match $ne :"myUserId"
2) $sort timestamp:-1
3) $group topicId $first
A sample query I have written in the past..
{ "$match": { $and: [ {'latestTimeStamp': { "$gte": new Date('3/11/2018') }}, {'latestTimeStamp': { "$lte": new Date('4/12/2018') }} ]} },
{ $unwind: '$latestSev'},
{ $sort: {'latestSev.sevRating': -1}},
{ $group:{ _id:{_id:'$_id', latestTimeStamp:'$latestTimeStamp', latestLikeli:'$latestLikeli', latestControl:'$latestControl', residualRatingArray:'$residualRatingArray'},
latestMaxImpName: {$first:'$latestSev.impName'} ,
latestMaxSevRating: {$first:'$latestSev.sevRating'}
}
}

using max query with mongoose

I am new with mongoose and still trying to understand how make correct queries
I have 2 simple Models
User :
const UserSchema = new Schema({
name: String,
age: Number,
movies:[{
type: Schema.Types.ObjectId,
ref: 'movie'
}]
}, { collection: 'USER_COLLEC' });
Movie :
const MovieSchema = new Schema({
title:String ,
duration: Number
}, { collection: 'MOVIE_COLLEC' });
What I want is the user with le longest movie ( highest duration )
For now I got that :
db.getCollection('USER_COLLEC') .
aggregate([
{ "$unwind": "$movies" } ,
{ $lookup:
{from: "MOVIE_COLLEC",
localField: "movies",
foreignField: "_id",
as: "movieContent"},
} ,
{ $unwind: "$movieContent" },
{ $group:
{ maxDuration: { $max: "$movieContent.duration" },
}
}
])
But it will only find the max duration with no user attached to it...
And indeed I only ask for the max duration on my query, but after the lookup I lose my user :(
How can I can keep it, or retrieve my user data ?
If you have any idea, I am completely stuck...
Thanks guys !

you can use $push to get the movie object as well.
db.getCollection('USER_COLLEC') .
aggregate([
{ "$unwind": "$movies" } ,
{ $lookup:
{from: "MOVIE_COLLEC",
localField: "movies",
foreignField: "_id",
as: "movieContent"},
} ,
{ $unwind: "$movieContent" },
{ $group:
{ _id: { $max: "$movieContent.duration" },
"movie": {
"$push": "movieContent"
}
}
}
])
After this, just get search for the Movie's _id in the user's movies array
UserSchema.find({movies:{$in:movieContent[0]._id}});
OR, instead of $push you can also use $first
{ $first: "$movieContent" }
Then you won't get it in an array.
Update:
Instead of {$push: $movieContent} or{$first: $movieContent}, you could just push $$ROOT:
{$push: $$ROOT} or {$first: $$ROOT}
and then you'll get the entire object. You don't need to fire another query to get the user.

I finally managed to find the solution, the $group was not the solution
db.getCollection('USER_COLLEC') .
aggregate([
{ "$unwind": "$movies" } ,
{ $lookup:
{from: "MOVIE_COLLEC",
localField: "movies",
foreignField: "_id",
as: "movieContent"},
} ,
{ $unwind: "$movieContent" },
{$sort: {"movieContent.duration":-1}},
{ $project: { "user":"$name","duration" : "$movieContent.duration"} } ,
{ $limit : 1 }
Which gives me something like :
{
"_id" : ObjectId("59d2f64dded1c008192f7e73"),
"user" : "Michael",
"duration" : 96
}

aggregation project mongodb output not expected

I'm coding a complex query in javascript.
and I'm using aggregation.
I'm using two tables. invoice, travel
this is my code.
Invoice.aggregate([
// filter the documents from invoice of only 2016
{
$match: {
executed: {
$gte: startDate,
$lte: endDate
},
"modelHolder.name": 'Travel'
}
},
// $lookup is working alone, it is not taking the input from function 1 of aggregate
{
$lookup: {
from: "travels",
localField: "modelHolder.id",
foreignField: "_id",
as: "dataFromTravels"
}
},
// filter by date of reservation and date of arrival
$match: { $or:
[
{
'dataFromTravels.from.date': {
$gte: departDate, $lte: endDate
}
},
{
'dataFromTravels.to.date': {
$lte: arrivalDate
},
}
]
}
},
{
$limit: 2
}
// 2nd function to work on the first function output as the input
], function (err, result) {
if (err) {
return console.log(('ERROR', err));
//next(err);
} else {
console.log('Result', result);
// res.json(result);
return;
}
});
// console.log('invoice !');
// console.log(invoice._id);
self.resume();
})
.on('error', function (err) {
console.error('Error occurred while streaming invoices', err);
})
.on('end', function () {
console.log('Successfully displayed invoices');
cb();
});
I want to have the list of cars and count how many times we used each of them in the period restricted in the match functions.
I added this.
{
$group: { "_id": "$dataFromTravels.car.plateNumber", "count": { $sum: 1 } }
},
the car is under travel table.
and I just got an array of 3 cars. or I have hundreds of cars in the same period.
How could I manage this?
thanks for you suggestions.

When you make a join query using mongodb. The target collection is stored as an array. so when you make use of lookup
{
$lookup: {
from: "travels",
localField: "modelHolder.id",
foreignField: "_id",
as: "dataFromTravels"
}
}
The output of the query will be like:-
{
"_id": ObjectId("554dc5e937c1482c491d9d36"),
// invoice data
"dataFromTravels": [
// travel data
]
}
And with mongodb aggregation you cannot use $match over nested documents.
So, in the query to match
db.invoice.aggregate([
{
$match: {
executed: {
$gte: ISODate(startDate),
$lte: ISODate(endDate)
},
"modelHolder.name": 'Travel'
}
},
{
$lookup: {
from: "travels",
localField: "modelHolder.id",
foreignField: "_id",
as: "dataFromTravels"
}
}, {
$unwind: "$dataFromTravels"
},{
$match: {
$or: [{
'dataFromTravels.from.date': {
$gte: ISODate(startDate),
$lte: ISODate(endDate)
}
}, {
'dataFromTravels.to.date': {
$lte: ISODate(startDate)
},
}]
}
}, {
$limit: 2
}
]);
Now, the dataFromTravels will be available as an object & you can use $match over the from & to properties