Related
I have this pipeline :
let pipeline = [
{
$match: {
date: { $gte: new Date("2022-10-19"), $lte: new Date("2022-10-26") },
},
},
{
$group: {
_id: "$date",
tasks: { $push: "$$ROOT" },
},
},
{
$sort: { _id: -1 },
},
];
const aggregationData = await ScheduleTaskModel.aggregate(pipeline);
where i group all "tasks" between a date range by date and i get that result :
[
{
"date": "2022-10-21T00:00:00.000Z",
"tasks": [...tasks with this date]
},
{
"date": "2022-10-20T00:00:00.000Z",
"tasks": [...tasks with this date]
}
]
as you see i have "tasks" only for 2 dates in that range,what if i want all dates to appear even the ones with no tasks so it would be like this with empty arrays ?
[
{
"date": "2022-10-26T00:00:00.000Z",
"tasks": []
},
{
"date": "2022-10-25T00:00:00.000Z",
"tasks": []
},
{
"date": "2022-10-24T00:00:00.000Z",
"tasks": []
},
{
"date": "2022-10-23T00:00:00.000Z",
"tasks": []
},
{
"date": "2022-10-22T00:00:00.000Z",
"tasks": []
},
{
"date": "2022-10-21T00:00:00.000Z",
"tasks": [...tasks with this date]
},
{
"date": "2022-10-20T00:00:00.000Z",
"tasks": [...tasks with this date]
},
{
"date": "2022-10-19T00:00:00.000Z",
"tasks": []
},
]
i tried to use $densify but unfortunately it requires upgrading my mongoDb atlas cluster which is not possible..
The answer of #WernfriedDomscheitAnother has a downside of grouping together all the documents in the collection, creating one large document, while a document has a size limit. A variation on it, without this downside, can be:
$match only the relevant document, same as in your current query
Use $facet to handle the case of no relevant documents at all. This will allow you to group all the relevant documents as you did in your query, but to keep a working-document even if there are any.
Add the relevant dates inside an array (since we use $facet this will happen even if the first match is empty)
Concatenate the array of matched data with the array of empty entries, use the real-data first.
$unwind the separate the documents by date, and $group again by date to remove the duplicates.
Format the result
db.collection.aggregate([
{$match: {date: {$gte: new Date("2022-10-19"), $lte: new Date("2022-10-26")}}},
{$facet: {
data: [
{$group: {_id: "$date", tasks: {$push: "$$ROOT"}}},
{$project: {date: "$_id", tasks: 1}}
]
}},
{$addFields: {
dates: {$map: {
input: {$range: [0, 8]},
// maybe more dynamic with $dateDiff -> { $dateDiff: { startDate: new Date("2022-10-19"), endDate: new Date("2022-10-26") }, unit: "day" } }
in: {
date: {$dateAdd: {
startDate: ISODate("2022-10-19T00:00:00.000Z"),
unit: "day",
amount: "$$this"
}},
tasks: []
}
}}
}},
{$project: {data: {$concatArrays: ["$data", "$dates"]}}},
{$unwind: "$data"},
{$group: {_id: "$data.date", "tasks": {$first: "$data.tasks"}}},
{$project: { _id: 0, date: "$_id", tasks: 1 }},
{$sort: { date: -1 }},
])
See how it works on the playground example
New function $densify would be the simplest, of course. The manual way of doing it would be this one:
db.collection.aggregate([
{
$group: {
_id: null,
data: { $push: "$$ROOT" }
}
},
{
$set: {
dates: {
$map: {
input: { $range: [ 0, 8 ] }, // maybe more dynamic with $dateDiff -> { $dateDiff: { startDate: new Date("2022-10-19"), endDate: new Date("2022-10-26") }, unit: "day" } }
in: {
date: {
$dateAdd: {
startDate: ISODate("2022-10-19T00:00:00.000Z"),
unit: "day",
amount: "$$this"
}
}
}
}
}
}
},
{
$set: {
dates: {
$map: {
input: "$dates",
as: "d",
in: {
$mergeObjects: [
"$$d",
{
tasks: {
$filter: {
input: "$data",
cond: { $eq: [ "$$d.date", "$$this.date" ] }
}
}
}
]
}
}
}
}
},
{
$project: {
data: {
$map: {
input: "$dates",
in: {
$cond: {
if: { $eq: [ "$$this.tasks", [] ] },
then: "$$this",
else: { $first: "$$this.tasks" }
}
}
}
}
}
},
{ $unwind: "$data" },
{ $replaceWith: "$data" }
])
Mongo Playground
With MongoDB aggregate, how do you lookup child documents from an array on the parent document and match a field on the child to a value?
I want to lookup the ShopItems for a Shop. And I only want the ShopItems that are not kidFriendly.
Details:
Let's say I have Shops collection
Shop Schema
{
shopItems: [
{
type: mongoose.Schema.Types.ObjectId,
ref: 'ShopItem',
},
],
}
And a shop can have multiple Shop Items:
ShopItem Schema
[
itemName: { type: String },
kidFriendly: { type: Boolean, default: false },
]
I want to lookup the ShopItems for a Shop. And I only want the ShopItems that are not kidFriendly.
I tried
const result3 = await Shop.aggregate([
{
$match: {
_id: mongoose.Types.ObjectId('623ae52ba5b1af0004e1c4ec'),
},
},
{
$lookup: {
from: ShopItem.collection.collectionName,
pipeline: [
{
$match: {
kidFriendly: { $ne: true },
_id: { $in: '$shopItems' },
},
},
],
as: 'adultOnlyItems',
},
},
]);
but I get an error:
$in needs an array.
Example data
// shops
[
{
_id: ObjectId('623ae52ba5b1af0004e1c4ec'),
shopItems: [ObjectId('631e6b133b688a0004a17265'), ObjectId('62f4cc974a255f00044c01b5'), ObjectId('625ffc48eec7b20004c9294c')]
},
{
_id: ObjectId('623ae52ba5b1af0004e1c4eb'),
shopItems: [ObjectId('631e6b133b688a0004a17263')]
}
]
//shop items
[
{
_id: ObjectId('631e6b133b688a0004a17265'),
itemName: "Barbie",
kidFriendly: true
},
{
_id: ObjectId('62f4cc974a255f00044c01b5'),
itemName: "Alcohol",
kidFriendly: false
},
{
_id: ObjectId('625ffc48eec7b20004c9294c'),
itemName: "Glass Vase",
kidFriendly: false
},
{
_id: ObjectId('631e6b133b688a0004a17263'),
itemName: "Beach Ball",
kidFriendly: true
}
]
When you use $lookup with pipeline, you need to have the let to store the field value from shops into variable. And use the $expr operator as well.
db.shops.aggregate([
{
$match: {
_id: mongoose.Types.ObjectId('623ae52ba5b1af0004e1c4ec')
}
},
{
$lookup: {
from: ShopItem.collection.collectionName,
let: {
shopItems: "$shopItems"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$ne: [
"$kidFriendly",
true
]
},
{
$in: [
"$_id",
"$$shopItems"
]
}
]
}
}
}
],
as: "adultOnlyItems"
}
}
])
Demo # Mongo Playground
Reference
Join Conditions and Subqueries on a Joined Collection
I’ve defined the following query which fetches me all items with an id which is in a given list of ids, and a status of either active or retracted.
const query = {
$and : [
{
$or: [
{
status: ‘active’,
},
{
status: ‘retracted’,
},
],
},
{
id: { $in: ids },
},
],
};
Each of these items has a parent_id field, which can either be null if the item does not have a parent, or can be the id of the parent.
I want my query to fetch all items with the ids I supply, as well as their parent items, if such a parent exists.
For example, if I supply the following IDs
[1,2,3]
and item 2 has a parent with id 5, while item 1 and 3 have parent_id set to null, I want my query to return the following items:
[1,2,3,5].
To achieve this I wrote the following query:
const collection = db.collection(‘myCollection’);
const data = await collection.aggregate([
{$match : query},
{
$lookup: {
from: ‘myCollection’,
let: { parentID: ‘$parent_id’},
pipeline: [
{
$match: {
$expr: {
$eq: [‘$id’, ‘$$parentID’],
},
},
},
as: ‘parent’,
},
},
]).sort(‘created_date’, ‘desc’).toArray();
return data;
However, this always returns null.
Sample Data:
[
{
id: 1,
parent_id: 3,
data: ‘bla bla’
},
{
id: 2,
parent_id: null,
data: ‘bla bla bla’
},
{
id: 3,
parent_id: null,
data: ‘bla’
}
]
Input: [1]
Output:
[
{
id: 1,
parent_id: 3,
data: ‘bla bla’
},
{
id: 3,
parent_id: null,
data: ‘bla’
}
]
The approach with $lookup being run upon same collection should work however it gives you a nested array so you need few additional stages to flatten such array and get all elements as on result set:
db.collection.aggregate([
{
$match: { id: { $in: [1] } }
},
{
$lookup: {
from: "collection",
localField: "parent_id",
foreignField: "id",
as: "parent"
}
},
{
$project: {
all: {
$concatArrays: [
"$parent",
[ "$$ROOT" ]
]
}
}
},
{
$project: {
"all.parent": 0
}
},
{
$unwind: "$all"
},
{
$replaceRoot: {
newRoot: "$all"
}
}
])
Mongo Playground
Your aggregation was malformed and lack some "]" for example closing the pipeline fied.
If you fix that the query works fine for me. Example
You can try this. The input array is [2,3] where 2 has parent id=1 and that is not in the input array. But the output array has the entry.
Working Playground
db.collection.aggregate([
{
$match: {
_id: {
$in: [
2,
3
]
}
}
},
{
$lookup: {
from: "collection",
localField: "p",
foreignField: "_id",
as: "parent"
}
},
{
$project: {
_id: 0,
id: {
$concatArrays: [
[
"$_id"
],
"$parent._id"
]
}
}
},
{
$unwind: "$id"
},
{
$sort: {
id: 1
}
}
])
I am using MongoDB aggregate function and after $match and $group I have this array of data
[ { _id: { item: 'AAA' }, total: 66 },
{ _id: { item: 'BBB' }, total: 3 } ]
Is there any way to use $project to turn the final result into
[ {AAA: 66}, {BBB: 3} ]
And to make it more fun, let's say there is {CCC: 0} in the results as well, how can we filter and remove element CCC if it have value equal to 0?
You can add below stages after your final $group stage
db.collection.aggregate([
{ "$group": {
"_id": null,
"data": {
"$push": {
"k": "$_id.item",
"v": "$total"
}
}
}},
{ "$replaceRoot": {
"newRoot": {
"$arrayToObject": "$data"
}
}}
])
Output
[
{
"AAA": 66,
"BBB": 3
}
]
In the following function I am trying to dynamically build a mongo $or query condition. The priceRanges is received as a parameter to the function. I iterate over priceRanges as follows to build the $or statement for my projection:
let $or = [];
for(let filter of priceRanges) {
$or.push( { $gte: [ "$price", +filter.low ] }, { $lte: [ "$price", +filter.high ] })
}
The $or array now contains the following values:
console.log('$or', $or)
$or [
{ '$gte': [ '$price', 100 ] }, { '$lte': [ '$price', 200 ] },
{ '$gte': [ '$price', 200 ] }, { '$lte': [ '$price', 300 ] },
{ '$gte': [ '$price', 300 ] }, { '$lte': [ '$price', 400 ] }
]
Here I build the project statement:
let $project = {
name:1,
producttype:1,
brand:1,
model:1,
price:1,
list_price:1,
description:1,
rating:1,
sku:1,
feature:1,
image:1,
images: 1,
specifications:1,
};
I append the $or condition to the projection:
$project.priceRange = {$or: $or};
The $or statement looks like this:
{ '$or':
[ { '$gte': [Array] },{ '$lte': [Array] },
{ '$gte': [Array] }, { '$lte': [Array] },
{ '$gte': [Array] }, { '$lte': [Array] } ] }
I create an array of my projection statement:
aggregateArray.push({$project: $project});
console.log(aggregateArray) looks like this:
aggregateArray [ { '$project':
{ name: 1,
producttype: 1,
brand: 1,
model: 1,
price: 1,
list_price: 1,
description: 1,
rating: 1,
sku: 1,
feature: 1,
image: 1,
images: 1,
specifications: 1,
priceRange: [Object] } },
{ '$skip': 1 },
{ '$limit': 4 } ]
I execute the projection as follows:
let products = await Product.aggregate(aggregateArray);
When executed, the $or statement doesn't seem to have any effect. The result contains random prices and not the ranges specified.
The problem here is that javascript array's push methods takes an array of values as a parameter so $or.push( { $gte: [ "$price", +filter.low ] }, { $lte: [ "$price", +filter.high ] }) pushes two separate filtering conditions. Therefore price equal to 50 will also be included in your result since it matches second condition (lower than 200). To fix that you need to combine those pairs using $and so your final filtering condition should look like this:
var $or = [
{ $and: [ { '$gte': [ '$price', 100 ] }, { '$lte': [ '$price', 200 ] } ] },
{ $and: [ { '$gte': [ '$price', 200 ] }, { '$lte': [ '$price', 300 ] } ] },
{ $and: [ { '$gte': [ '$price', 300 ] }, { '$lte': [ '$price', 400 ] } ] }
]
Then if you need it for filtering it should be using with $expr inside of $match stage.
db.col.aggregate([
{
$match: { $expr: { $or: $or } }
},
// other aggregation stages
])