I want to create a mongo db view from two collections with a new value that is a sum of values from one of the collection according to an operation from another collection.
Below is the structure:
/* First collection */
{
"product": "test",
"labels" : [
{"code": "label1", "value": 42},
{"code": "label2", "value": 50}
]
}
/* Second collection */
{
"code": "label3",
"calculation" : [
{"label" : "label1", "operation":"+"},
{"label" : "label2", "operation":"-"}
]
}
In my aggregated collection i want a new field that would be label1 - label2.
{
"product" : "test",
"labels" : [
{"code": "label1", "value": 42},
{"code": "label2", "value": 50}
],
"vlabels" : [
{"code": "label3", "value": -8}
]
}
Although it is possible. I doubt it would be optimal, if you don't need further processing on the database, I suggest you do this at the application layer.
However, I have attempted to do this as an exercise. This approach would check only for the "-" operator and assign a negative value, other operator will use the existing value.
/* First collection: "products" */
/* Second collection: "vlabels" */
db.products.aggregate([
{
$lookup: {
from: "vlabels", // lookup calculation from vlabels
let: {
labels: "$labels"
},
pipeline: [
{
$set: {
calculation: {
$map: {
input: "$calculation", // map over calculation in vlabels
as: "calc",
in: {
operation: "$$calc.operation",
product: {
$arrayElemAt: [
{
$filter: {
input: "$$labels", // filter for matching product labels and get the first element using $arrayAlemAt to get the value
as: "label",
cond: {
$eq: ["$$calc.label", "$$label.code"]
}
}
},
0
]
}
}
}
}
}
},
{
$project: {
_id: false,
code: "$code",
value: {
$reduce: { // reducing by adding all values in calculation array, use negative value on "-" operator
input: "$calculation",
initialValue: 0,
in: {
$add: [
"$$value",
{
$cond: [
{
$eq: ["-", "$$this.operation"]
},
{
$multiply: [
-1,
{ $ifNull: ["$$this.product.value", 0] }
]
},
{ $ifNull: ["$$this.product.value", 0] }
]
}
]
}
}
}
}
}
],
as: "vlabels"
}
}
])
Mongo Playground
Related
I have the following document in MongoDB
{
"product_id": "10001"
"product_name": "Banana"
"product_date": "2022-10-20T00:00:00.000+00:00"
"product_price": 255.15
"dates": {
"2022-10-10": {
"recorded_price": 195.15
},
"2022-10-15": {
"recorded_price": 230.20
},
"2022-10-20": {
"recorded_price": 255.20
}
}
}
I would like to add a new field named "min_7day_price" which would select the minimum price from the date object in the past 7 days.
Something like this:
{
"product_id": "10001"
"product_name": "Banana"
"product_date": "2022-10-20T00:00:00.000+00:00"
"product_price": 255.15
"dates": {
"2022-10-10": {
"recorded_price": 195.15
},
"2022-10-15": {
"recorded_price": 230.20
},
"2022-10-20": {
"recorded_price": 255.20
}
},
"min_7day_price": 230.20
}
I tried using aggregation to create a new field and convert the object to an array but I can't filter the values inside.
{
"min_7day_price": {
$objectToArray: "$dates"
}
}
One option is to use update with pipeline:
Convert the dictionary to array
Use $reduce to keep only one item from it, by iterating and comparing the current item: $$this to the best item so far: $$value
Format the answer
db.collection.update({},
[
{$set: {datesArr: {$objectToArray: "$dates"}}},
{$set: {
datesArr: {
$reduce: {
input: "$datesArr",
initialValue: {
k: {$dateAdd: {startDate: "$$NOW", amount: -7, unit: "day"}},
v: {recorded_price: {$max: "$datesArr.v.recorded_price"}}
},
in: {
$cond: [
{$and: [
{$gte: [{$dateFromString: {dateString: "$$this.k"}}, "$$value.k"]},
{$lte: ["$$this.v.recorded_price", "$$value.v.recorded_price"]}
]},
{
k: {$dateFromString: {dateString: "$$this.k"}},
v: "$$this.v.recorded_price"
},
"$$value"
]
}
}
}
}
},
{$set: {min_7day_price: "$datesArr.v", datesArr: "$$REMOVE"}}
])
See how it works on the playground example
There are 2 array fields after I looked up in MongoDB aggregation pipeline.
the first one
[
{
"colorId": "60828a1b216b0972da695f2a",
"name": "Exellent",
"description": "Great work"
}
]
and the second one
[
{
"_id": "60828a1b216b0972da695f2a",
"colorName": "Green",
"hexColorCodes": "#2D9D78",
"sequence": 1,
"isActivated": true,
"created_at": "2021-04-23T08:49:31.729Z",
"updated_at": "2021-04-23T08:49:31.729Z",
"__v": 0,
"isDefault": true
}
]
the result I want is
[
{
"colorId": "60828a1b216b0972da695f2a",
"name": "Exellent",
"description": "Great work",
"colorName": "Green",
"hexColorCodes": "#2D9D78"
}
]
then I want to map colorName and hexColorCodes to the first array. Here is my aggregate pipeline
db.collection.aggregate([
{
$lookup: {
from: "color_tags",
localField: "colors.colorId",
foreignField: "_id",
as: "tempColors",
},
},
{
$addFields: {
stages3: {
$map: {
input: "$colors",
in: {
$mergeObjects: [
"$$this",
{
$arrayElemAt: [
"$tempColors",
{
$indexOfArray: [
"$tempColors._id",
"$$this.colors.colorId",
],
},
],
},
],
},
},
},
},
}
])
but the result is not what I expected. It mapped with incorrect id. Please suggest.
$map to iterate loop of first array
$filter to iterate loop of second array and match colorId with _id and return matching result
$arrayElemAt to get first matching element
$mergeObjects to merge current object with return result from second array
{
$project: {
first: {
$map: {
input: "$first",
as: "f",
in: {
$mergeObjects: [
"$$f",
{
$arrayElemAt: [
{
$filter: {
input: "$second",
cond: { $eq: ["$$this._id", "$$f.colorId"] }
}
},
0
]
}
]
}
}
}
}
}
If you want to result specific fields then add a $project stage at the end,
{
$project: {
"first.colorId": 1,
"first.name": 1,
"first.description": 1,
"first.colorName": 1,
"first.hexColorCodes": 1
}
}
Playground
I have a Devices collection in MongoDB with the following structure:
{
"group": [
"group1"
]
},
{
"group": [
"group1",
"group2"
]
},
{
"group": []
},
{
"group": [
"group3",
"group4"
]
}
How can I filter or aggregate the documents so that I only return the last element of each group array including the blank arrays?
Expected result:
["group1", "group2", "", "group4"]
You can $group and use $arrayElemAt to get the last element. Additionally you need $ifNull to specify the default value (empty string):
db.collection.aggregate([
{
$group: {
_id: null,
lastElements: { $push: { $ifNull: [ { $arrayElemAt: [ "$group", -1 ] }, "" ] } }
}
}
])
Mongo Playground
I'm struggling to find out how to add a new status field in an aggregate based on a calculation of an array of items.
Currently, I do this in Angular front-end by asking both collections and iterating each element with the _.some() method of Lodash. But I want to move the calculation to the backend and I'm stuck with the MongoDB aggregate.
Input: Each Subscription (one per user) has many Contracts (one per month), and I want to calculate the status of the subscription out of its contracts.
[
{
"_id": "5b4d9a2fde57780a2e175agh",
"user": "5a709da7c2ffc105hu47b254",
"contracts": [
{
"_id": "5b4d9a2fde57780a2e175agh",
"date": "2018-07-15T13:00:00.000Z",
"totalPrice": 200,
"totalPaid": 67,
"isCanceled": false,
"paymentFailed": false
},
{
"_id": "5b4d9a2fde57780a2e175agh",
"date": "2018-08-15T13:00:00.000Z",
"totalPrice": 200,
"totalPaid": 0,
"isCanceled": false,
"paymentFailed": false
},
{
"_id": "5b4d9a2fde57780a2e175agh",
"date": "2018-09-15T13:00:00.000Z",
"totalPrice": 200,
"totalPaid": 0,
"isCanceled": false,
"paymentFailed": false
}
]
}
]
Output: In this case, take the past contracts and check if the user has paid what totalPrice says (and if there weren't any payment errors). If not, the payment of the subscription is “pending”:
{
"_id": "5b4d9a2fde57780a2e175agh",
"user": "5a709da7c2ffc105hu47b254",
"status": "PAYMENT_PENDING" // or "PAYMENT_ERROR" or "SUCCESS"…
}
But I cannot calculate by each array item: it gives an error if I try to use "$contracts.$.totalPaid" (“'FieldPath field names may not start with '$'.'”)
This is my step of the aggregate (testing only two status conditions):
$addFields: {
"status": {
$cond: [
{ $and: [
{ $lt: [ "$contracts.totalPaid", "$contracts.totalPrice" ]},
{ $eq: [ "$contracts.paymentFailed", true ] },
{ $lte: [ "$contracts.date", ISODate("2018-08-24T18:32:50.958+0000") ]},
{ $eq: [ "$contracts.2.isCanceled", false ] }
]},
'PAYMENT_ERROR',
{ $cond: [
{ $and: [
{ $lt: [ "$contracts.paidAmount", "$contracts.checkout.totalPrice" ]},
//{ $eq: [ "$contracts.paymentFailed", false ] },
//{ $lte: [ "$contracts.subscriptionDate", ISODate("2018-08-24T18:32:50.958+0000") ]},
{ $eq: [ "$contracts.isCanceled", true ] }
]},
'PAYMENT_PENDING',
'SOMETHING_ELSE'
]}
]
}
}
I have succeeded in calculating the status out of the Subscription's fields, but not out of its array of contracts.
I would appreciate if anybody could point me in the right direction with the aggregate framework, as other examples/questions I've found $sum/calculate but do not add new fields.
Thank you very much.
I found a way: instead of calculating directly in the $addFields step, I do several more steps.
Please, feel free to suggest improvements to the aggregate, as this is my first big agrgegate :)
Step 1: $match
Conditions of Subscriptions I'm interested in. (Use your own)
Step 2: $lookup
Join each Subscription with all its contracts:
$lookup: {
// Join with subscriptioncontracts collection
"from" : "subscriptioncontracts",
"localField" : "_id",
"foreignField" : "subscription",
"as" : "contracts"
}
Step 3: $unwind
Make one document per subscription contract:
$unwind: {
// Make one document per subscription contract
path : "$contracts",
preserveNullAndEmptyArrays : false // optional
}
Step 4: $sort
(I need something special using the last/most modern contract, so I need to sort them)
$sort: {
// Assure the last contract if the most modern
"_id": 1,
"contracts.subscriptionDate": 1
}
Step 5: $group
Here is the magic: Add new fields with the calculation using all the subscription contracts (now each “contract” is in its own document, instead of in an array)
I need to add “subscription” because I'll need to project it as the response.
$group: {
// Calculate status from contracts (group by the same subscription _id)
"_id": "$_id",
"subscription": { "$first": "$$CURRENT" },
"_lastContract": { $last: "$contracts" },
"_statusPaymentPending": {
$sum: { $cond: [
{ $and: [
{ $lt: [ "$contracts.paidAmount", "$contracts.checkout.totalPrice" ] },
{ $lt: [ "$contracts.subscriptionDate", new Date() ] },
{ $eq: [ "$contracts.paymentFailed", false ] },
{ $eq: [ "$contracts.isCanceled", false ] }
]}, 1, 0
] }
},
"_statusPaymentFailed": {
$sum: { $cond: [
{ $and: [
{ $lt: [ "$contracts.paidAmount", "$contracts.checkout.totalPrice" ] },
{ $lt: [ "$contracts.subscriptionDate", new Date() ] },
{ $eq: [ "$contracts.paymentFailed", true ] },
{ $eq: [ "$contracts.isCanceled", false ] }
]}, 1, 0
] }
}
}
Step 6: $project
Here I calculate other statuses from the subscription data (not the contracts)
$project: {
// Calculate other statuses
"_id": "$_id",
"subscription": "$subscription",
"_statusCanceled": { $cond: [ "$subscription.isCanceled", true, false ] },
"_statusFutureStart": { $cond: [ { $gte: [ "$subscription.subscriptionStartDate", new Date() ] }, true, false ] },
"_statusUnsubscribed": { $cond: [ { $gte: [ "$subscription.subscriptionEndDate", new Date() ] }, true, false ] },
"_statusFinished": {
$cond: [
{ $and: [
{ $ne: [ "$subscription.subscriptionEndDate", undefined ] },
{ $lte: [ "$subscription.subscriptionEndDate", new Date() ] }
]},
true,
false
]
},
"_statusPaymentPending": "$_statusPaymentPending",
"_statusPaymentFailed": "$_statusPaymentFailed",
"_statusExtensionPending": { $cond: [ { $lte: [ "$_lastContract.expirationDate", new Date() ] }, true, false ] }
}
Step 7: $project
And finally, I merge all statuses on one “status” field:
$project: {
"subscription": 1,
// Condense all statuses into one Status field
"status": {
$cond: [
"$_statusCanceled",
'CANCELED',
{ $cond: [
"$_statusPaymentFailed",
'PAYMENT_ERROR',
{ $cond: [
"$_statusPaymentPending",
'PAYMENT_PENDING',
{ $cond: [
"$_statusUnsubscribed",
'UNSUBSCRIBED',
{ $cond: [
"$_statusExtensionPending",
'PENDING_EXTEND_CONTRACT',
{ $cond: [
"$_statusFutureStart",
'FUTURE_START',
{ $cond: [
"$_statusFinished",
'FINISHED',
'OK'
]}
]}
]}
]}
]}
]}
]
}
}
TODO
Maybe you can improve my flow:
Instead of having a subscription and status final object, is it possible to move all the data from the subscription object to the root (accompanied by the computed status field)?
Do you see other better way of calculating this final status field, instead of having a $group and two $project?
Thank you for any improvement you may suggest!
I have this problem that I want to sort the result of a query based on the field values from another collection,
Problem: I want to first get the user 123 friends and then get their posts and then sort the post with the friends strength value,
I have this :
POST COLLECTON:
{
user_id: 8976,
post_text: 'example working',
}
{
user_id: 673,
post_text: 'something',
}
USER COLLECTON:
{
user_id: 123,
friends: {
{user_id: 673,strength:4}
{user_id: 8976,strength:1}
}
}
Based on the information you have retrieved from your user you essentially want to come out to an aggregation framework query that looks like this:
db.posts.aggregate([
{ "$match": { "user_id": { "$in": [ 673, 8976 ] } } },
{ "$project": {
"user_id": 1,
"post_text": 1,
"weight": {
"$cond": [
{ "$eq": [ "$user_id", 8976 ] },
1,
{ "$cond": [
{ "$eq": [ "$user_id", 673 ] },
4,
0
]}
]
}
}},
{ "$sort": { "weight": -1 } }
])
So why aggregation when this does not aggregate? As you can see, the aggregation framework does more than just aggregate. Here it is being used to "project" a new field into the document an populate it with a "weight" to sort on. This allows you to get the results back ordered by the value you want them to be sorted on.
Of course, you need to get from your initial data to this form in a "generated" way that you do do for any data. This takes a few steps, but here I'll present the JavaScript way to do it, which should be easy to convert to most languages
Also presuming your actual "user" looks more like this, which would be valid:
{
"user_id": 123,
"friends": [
{ "user_id": 673, "strength": 4 },
{ "user_id": 8976, "strength": 1 }
]
}
From an object like this you then construct the aggregation pipeline:
// user is the structure shown above
var stack = [];
args = [];
user.friends.forEach(function(friend) {
args.push( friend.user_id );
var rec = {
"$cond": [
{ "$eq": [ "user_id", friend.user_id ] },
friend.strength
]
};
if ( stack.length == 0 ) {
rec["$cond"].push(0);
} else {
var last = stack.pop();
rec["$cond"].push( last );
}
stack.push( rec );
});
var pipeline = [
{ "$match": { "user_id": { "$in": args } } },
{ "$project": {
"user_id": 1,
"post_text": 1,
"weight": stack[0]
}},
{ "$sort": { "weight": -1 } }
];
db.posts.aggregate(pipeline);
And that is all there is to it. Now you have some code to go through the list of "friends" for a user and construct another query to get all posts from those friends weighted by the "strength" value for each.
Of course you could do much the same things with a query for all posts by just removing or changing the $match, but keeping the "weight" projection you can "float" all of the "friends" posts to the top.