I have two fields:
date (ISODate)
value (it's a number - however can be also undefined)
I want to sort them in a specific way.
Firstly - sort by value. If it exists (not undefined), let them come first. If value is undefined, push these objects at the end.
Secondly - sort by date. So at this moment, we will have everything sorted by value. But now I want to sort it by date.
Expected result:
[
{
value: 1,
date: today, // ofc it's a date but just to represent the case
},
{
value: 2,
date: yesterday,
},
{
value: undefined,
date: today,
},
{
value: undefined,
date: yesterday,
}
]
Current solution:
.sort({
value: -1,
date: -1
});
However it fails in some situations, unfortunately Im unable to detect why it sometimes sort it in a wrong way. But it does...
Thank you in advance
To achieving this you can use aggregate with $cond.
db.getCollection('collectionName').aggregate([
{
$project :
{
"id" : 1,
"value" : 1,
"date": 1,
sortValue: {
$cond: {
if: { $eq: ['$value',undefined ]},
then: 0,
else: 1,
},
}
}
},
{
$sort :
{
"sortValue" :-1,
"value" : 1,
"date" : -1
}
}])
Tested in Mongo GUI.
Aggregate In Mongoose:
Model.aggregate([
{
$project:
{
"id": 1,
"value": 1,
"date": 1,
sortValue: {
$cond: {
if: { $eq: ['$value', undefined] },
then: 0,
else: 1,
},
}
}
},
{
$sort:
{
"sortValue": -1,
"value": 1,
"date": -1
}
}
], function (err, result) {
if (err) {
console.log(err);
return;
}
console.log(result);
});
Related
For example, I have something in my database like in customers collection.
{
Max: {
shoping_list: {
food: { Pizza: 2, Ramen: 1, Sushi: 5 }
}
},
John: {
shoping_list: {
food: { Pizza: 2, Ramen: 1, Burger: 1 }
}
}
}
In my backend, I want to get the sum of food
const request = await customers.aggregate([
{
$group: {
_id: null,
Pizza: {
$sum: '$shoping_list.food.Pizza',
},
Is there a way how to update or get the sum automatically without manually writing every food from the shopping_list?
The design of the document may lead the query looks complex but still achievable.
$replaceRoot - Replace the input document with a new document.
1.1. $reduce - Iterate the array and transform it into a new form (array).
1.2. input - Transform key-value pair of current document $$ROOT to an array of objects such as: [{ k: "", v: "" }]
1.3. initialValue - Initialize the value with an empty array. And this will result in the output in the array.
1.4. in
1.4.1. $concatArrays - Combine aggregate array result ($$value) with 1.4.2.
1.4.2. With the $cond operator to filter out the document with { k: "_id" }, and we transform the current iterate object's v shoping_list.food to the array via $objectToArray.
$unwind - Deconstruct the foods array into multiple documents.
$group - Group by foods.k and perform sum for foods.v.
db.collection.aggregate([
{
$replaceRoot: {
newRoot: {
foods: {
$reduce: {
input: {
$objectToArray: "$$ROOT"
},
initialValue: [],
in: {
$concatArrays: [
"$$value",
{
$cond: {
if: {
$ne: [
"$$this.k",
"_id"
]
},
then: {
$objectToArray: "$$this.v.shoping_list.food"
},
else: []
}
}
]
}
}
}
}
}
},
{
$unwind: "$foods"
},
{
$group: {
_id: "$foods.k",
sum: {
$sum: "$foods.v"
}
}
}
])
Demo # Mongo Playground
Background:
A customer is an object that has a name field.
A line is an object that has the following fields:
inLine - an array of customers
currentCustomer - a customer
processed - an array of customers
The collection 'line' contains documents that are line objects.
Problem:
I'm trying to implement a procedure which would do the following:
Push currentCustomer to processed
Set currentCustomer to the 1st element in inLine
Pop the 1st element of inLine
Since the new value of a field depends on the previous value of another, atomicity is important here.
What I tried so far:
Naive approach
db.collection('line').findOneAndUpdate({
_id: new ObjectId(lineId),
}, {
$set: {
currentCustomer: '$inLine.0',
},
$pop: {
inLine: -1,
},
$push: {
processed: '$currentCustomer',
},
});
However, currentCustomer is set to a string which is literally "$inLine.0" and processed has a string which is literally "$currentCustomer".
Aggregation approach
db.collection('line').findOneAndUpdate({
_id: new ObjectId(lineId),
}, [{
$set: {
currentCustomer: '$inLine.0',
},
$pop: {
inLine: -1,
},
$push: {
processed: '$currentCustomer',
},
}]);
However, I got the following error:
MongoError: A pipeline stage specification object must contain exactly one field.
Multi-stage aggregation approach
db.collection('line').findOneAndUpdate({
_id: new ObjectId(lineId),
}, [{
$set: {
currentCustomer: '$inLine.0',
},
}, {
$pop: {
inLine: -1,
},
}, {
$push: {
processed: '$currentCustomer',
},
}]);
However, $pop and $push are Unrecognized pipeline stage names.
I tried making it using only $set stages, but it ended up very ugly and I still couldn't get it to work.
Based on turivishal's answer, it was solved like so:
db.collection('line').findOneAndUpdate({
_id: new ObjectId(lineId),
}, [{
$set: {
// currentCustomer = inLine.length === 0 ? null : inLine[0]
currentCustomer: {
$cond: [
{ $eq: [{ $size: '$inLine' }, 0] },
null,
{ $first: '$inLine' },
],
},
// inLine = inLine.slice(1)
inLine: {
$cond: [
{ $eq: [{ $size: '$inLine' }, 0] },
[],
{ $slice: ['$inLine', 1, { $size: '$inLine' }] },
],
},
// if currentCustomer !== null then processed.push(currentCustomer)
processed: {
$cond: [
{
$eq: ['$currentCustomer', null],
},
'$processed',
{
$concatArrays: [
'$processed', ['$currentCustomer'],
],
}
],
},
},
}]);
I don't think its possible with simple update using $push or $pop.
As per your experiment, the aggregation can not support direct $push, $pop stage in root level, so I have corrected your query,
currentCustomer check condition if size of inLine is 0 then return null otherwise get first element from inLine array using $arrayElemAt,
inLine check condition if size of inLine is 0 then return [] otherwise remove first element from inLine array using $slice and $size
processed concat both arrays using $concatArrays, $ifNull to check if field is null then return blank array, check condition if currentCustomer null then return [] otherwise return currentCustomer
db.collection('line').findOneAndUpdate(
{ _id: new ObjectId(lineId), },
[{
$set: {
currentCustomer: {
$cond: [
{ $eq: [{ $size: "$inLine" }, 0] },
null,
{ $arrayElemAt: ["$inLine", 0] }
]
},
inLine: {
$cond: [
{ $eq: [{ $size: "$inLine" }, 0] },
[],
{ $slice: ["$inLine", 1, { $size: "$inLine" }] }
]
},
processed: {
$concatArrays: [
{ $ifNull: ["$processed", []] },
{
$cond: [
{ $eq: ["$currentCustomer", null] },
[],
["$currentCustomer"]
]
}
]
}
}
}]
);
Playground
I'm trying to find stocks in the Stock collection where the sum of all owners' shares is less than 100. Here is my schema.
const stockSchema = new mongoose.Schema({
owners: [
{
owner: {
type: Schema.Types.ObjectId,
ref: "Owner"
},
shares: {
type: Number,
min: 0,
max: 100
}
}
]
}
const Stock = mongoose.model("Stock", stockSchema);
I've tried to use aggregate but it returns a single object computed over all stocks in the collection, as opposed to multiple objects with the sum of each stock's shares.
stockSchema.statics.getUnderfundedStocks = async () => {
const result = await Stock.aggregate([
{ $unwind: "$owners" },
{ $group: { _id: null, shares: { $sum: "$owners.shares" } } },
{ $match: { shares: { $lt: 100 } } }
]);
return result;
};
So, rather than getting:
[ { _id: null, shares: 150 } ] from getUnderfundedStocks, I'm looking to get:
[ { _id: null, shares: 90 }, { _id: null, shares: 60 } ].
I've come across $expr, which looks useful, but documentation is scarce and not sure if that's the appropriate path to take.
Edit: Some document examples:
/* 1 */
{
"_id" : ObjectId("5ea699fb201db57b8e4e2e8a"),
"owners" : [
{
"owner" : ObjectId("5ea62a94ccb1b974d40a2c72"),
"shares" : 85
}
]
}
/* 2 */
{
"_id" : ObjectId("5ea699fb201db57b8e4e2e1e"),
"owners" : [
{
"owner" : ObjectId("5ea62a94ccb1b974d40a2c72"),
"shares" : 20
},
{
"owner" : ObjectId("5ea62a94ccb1b974d40a2c73"),
"shares" : 50
},
{
"owner" : ObjectId("5ea62a94ccb1b974d40a2c74"),
"shares" : 30
}
]
}
I'd like to return an array that just includes document #1.
You do not need to use $group here. Simply use $project with $sum operator.
db.collection.aggregate([
{ "$project": {
"shares": { "$sum": "$owners.shares" }
}},
{ "$match": { "shares": { "$lt": 100 } } }
])
Or even you do not need to use aggregation here
db.collection.find({
"$expr": { "$lt": [{ "$sum": "$owners.shares" }, 100] }
})
MongoPlayground
How can I retrieve data with a custom sort in Mongoose?
There is a job starting date that needs to be sorted by the month and year, but currently this script is only sorting from December to January.
router.get('/', (req, res) => {
Job.find()
.sort({ from: -1 })
.then(jobs => res.json(jobs))
.catch(err => res.status(404).json(err));
});
The problem is in the sort; values for from is like 12.2018, 06.2019, 03.2020, 11.2009 and so on.
I want to sort these results first from the year (which is after the dot) and then sort from the months. I cannot currently change how the data is set and it's stored as a String in the model Schema.
You have to use aggregation framework to first transform your string to a valid date by
$spliting it,
$convert parts from string to int
and using $dateFromParts,
then you sort and finally remove created field.
Here's the query :
db.collection.aggregate([
{
$addFields: {
date: {
$dateFromParts: {
year: {
$convert: {
input: {
$arrayElemAt: [
{
$split: [
"$from",
"."
]
},
1
]
},
to: "int"
}
},
month: {
$convert: {
input: {
$arrayElemAt: [
{
$split: [
"$from",
"."
]
},
0
]
},
to: "int"
}
},
}
}
}
},
{
$sort: {
date: -1
}
},
{
$project: {
date: 0
}
}
])
You can test it here
Good afternoon all,
I am having a really tough time working with aggregation queries in MongoDB 3.4. I have a problem that is asking me to do push the results of my aggregation query into an empty array called categories which I have been able to do successfully using this code:
var categories = [];
database.collection("item").aggregate([{
$group : {
_id : "$category",
num : {$sum : 1}
}},
{$sort:{_id:1}}]).toArray(function(err, data){
categories.push(...data);
callback(categories);
console.log(categories);
})
}
categories looks like this:
[ { _id: 'Apparel', num: 6 },
{ _id: 'Books', num: 3 },
{ _id: 'Electronics', num: 3 },
{ _id: 'Kitchen', num: 3 },
{ _id: 'Office', num: 2 },
{ _id: 'Stickers', num: 2 },
{ _id: 'Swag', num: 2 },
{ _id: 'Umbrellas', num: 2 } ]
Next I have the following task:
In addition to the categories created by your aggregation query,
include a document for category "All" in the array of categories
passed to the callback. The "All" category should contain the total
number of items across all categories as its value for "num". The
most efficient way to calculate this value is to iterate through
the array of categories produced by your aggregation query, summing
counts of items in each category.
The problem is that it seems like inside my .toArray() method the data parameter sometimes acts like an array and sometimes not. For example if I wanted to add perhaps just the value of the num key to the categories array like so: categories.push(...data["num"]) I get an error stating undefined is not iterable.
Since I cannot iterate over each data.num key I cannot extract it's value and add it to a running total of all data.num values.
What am I not understanding about what is going on here?
You don't need to use application logic to group data, mongoDB aggregation is made for this task. Add another $group to your query with a new field All that $sum your $num field and $push all documents to a new field called categories :
db.item.aggregate([{
$group: {
_id: "$category",
num: { $sum: 1 }
}
}, { $sort: { _id: 1 } }, {
$group: {
_id: 1,
All: { $sum: "$num" },
categories: {
$push: {
_id: "$_id",
num: "$num"
}
}
}
}])
It gives :
{
"_id": 1,
"All": 23,
"categories": [{
"_id": "Swag",
"num": 2
}, {
"_id": "Office",
"num": 2
}, {
"_id": "Stickers",
"num": 2
}, {
"_id": "Apparel",
"num": 6
}, {
"_id": "Umbrellas",
"num": 2
}, {
"_id": "Kitchen",
"num": 3
}, {
"_id": "Books",
"num": 3
}, {
"_id": "Electronics",
"num": 3
}]
}
For consuming the output, data is an array, to access the first element use data[0] :
var categories = [];
database.collection("item").aggregate([{
$group: {
_id: "$category",
num: { $sum: 1 }
}
}, { $sort: { _id: 1 } }, {
$group: {
_id: 1,
All: { $sum: "$num" },
categories: {
$push: {
_id: "$_id",
num: "$num"
}
}
}
}]).toArray(function(err, data) {
var totalCount = data[0]["All"];
console.log("total count is " + totalCount);
categories = data[0]["categories"];
for (var i = 0; i < categories.length; i++) {
console.log("category : " + categories[i]._id + " | count : " + categories[i].num);
}
})
What I wanted to achieve was pushing or unshifting as we'll see in a moment an object that looked like this into my categories array:
var allCategory = {
_id: "All",
num: [sum of all data.num values]
}
I ended up messing with .reduce() method and used it on the categories array. I got lucky through some console.log-ing and ended up making this:
var categories = [];
database.collection("item").aggregate([{
$group : {
_id : "$category",
num : {$sum : 1}
}},
{$sort:{_id:1}}]).toArray(function(err, data){
categories.push(...data);
var sum = categories.reduce(function(acc, val){
// console.log(acc, val["num"])
return acc + val["num"]
},0);
var allCategory = {
_id: "All",
num: sum
}
categories.unshift(allCategory)
callback(categories);
})
First I use a spread operator to push all the objects from data into categories. Then declare sum which runs .reduce() on categories returning the accumulation of val["num"] which is really data.num (console log is life). I create the allCategory document/object then use .unshift to place it at the beginning of my categories array (this placement was a requirement) then use my callback.
I think it's a hacky way of accomplishing my goal and I had to go through some trial and error as to the correct order of methods and variables in my .toArray(). Yet it worked and I learned something. Thanks for the help #Bertrand Martel .