pass along the field and value while aggregating - javascript

I had an output document from a pipeline something like : {upvotes : 1, downvotes : 5}. I just wanted to pass that along to the next part of the pipeline. I did something like this:
{
$group :{
_id : "$_id",
upvotes : {$max : "$upvotes"},
downvotes : {$max : "$downvotes"},
average : {$avg : "$statements.result"},
count : {$max : "$count"}
}
}
I didn't want to use max I don't want the maximum of anything. I just wanted the field and value: {upvotes : 1, downvotes : 5}. lets say they were not numbers. lets say the fields were object I want all the objects for that field to be outputted not the max.
make this question clearer: how can I get the the field and value without using max?
outputs:
[ { _id: 57aea6791016c0b023a71e9d,
upvotes: 3,
downvotes: 1,
average: 7.857142857142857,
count: 5 },
{ _id: 57aed883139c1e702beb471e,
upvotes: 0,
downvotes: 1,
average: 7,
count: 1 } ]
The output is good but I don't want to use max to get it.
For some reason I think I need an Accumulator Operator. That is why I use max.

Since you are grouping by "$_id", you're actually not grouping at all. Your results are going to correlate 1:1 to your initial data set. To accomplish what you are asking, passing along upvotes and downvotes without using an aggregate function like $max, you'd just include them in the _id for your group stage which is where you specify your group by key.
So you would do:
{
$group :{
_id : {
"_id":"$_id",
"upvotes": "$upvotes",
"downvotes": "$downvotes"
}
average : {$avg : "$statements.result"},
count : {$max : "$count"}
}
}
But still, you are going to end up with a 1:1 correlation to your original data.

Following your question thread, your pipeline in question is most probably a preceding pipeline step from an $unwind operation thence the following suggested solution is based on that assumption. Please feel free to suggest otherwise.
To include a field in the $group pipeline using the accumulators, the $first or $last seem most appropriate here since you have just flattened an array and those operators will return a value from the first/last document for each group. What is means is if the field is not part of a flattened array then $first/$last would give you its actual value before the $unwind took place.
{
"$group" :{
"_id": "$_id",
"upvotes": { "$first": "$upvotes"},
"downvotes": { "$first": "$downvotes"},
"average": { "$avg": "$statements.result"},
"count": { "$max": "$count"}
}
}

Related

How do I sort in the front end with javascript an array of items based on other fields inside the item [duplicate]

This question already has answers here:
How to sort an array of objects by multiple fields?
(38 answers)
Closed 2 years ago.
I am trying to create 2 sort types in my FE with javascript for my items that come from the backend but I don't know what the logic should look like for it. Where can I read about something like this?
The first way would be to sort them by the date (The most recent first)
The second way would be to sort them by the total number of the awards (Highest number first)
This is how my data looks like:
[
{
awards: {awardOne: 1, awardTwo: 4, awardThree: 8}
createdAt: "2020-11-13T21:12:50.742Z"
text: "Some text"
username: "username"
},
{
awards: {awardOne: 1, awardTwo: 4, awardThree: 8}
createdAt: "2020-11-13T21:12:50.742Z"
text: "Some text"
username: "username"
},
{
awards: {awardOne: 2, awardTwo: 3, awardThree: 2}
createdAt: "2020-11-13T21:12:50.742Z"
text: "Some text"
username: "username"
},
]
Here is an example sorting the data by number of awards in each object: code sandbox. I'm reducing the items in each awards object to a single value and comparing those.
To sort by date, you can use localeCompare like others have pointed out and use a similar pattern.
Update: I just added an working example of sorting by date to the same sandbox
I am thinking you can sort the items by date lexiographically.
Using String.prototype.localeCompare your code would something like
data.sort((a, b) => {
return ('' + a.createdAt).localeCompare(b.createdAt);
}
Source
To sort by number of awards you would need a smaller function, that calculates number of awards, and then write something like:
data.sort((a, b) => {
return (calcNumOfAwards(a) - calcNumOfAwards(b))
}

Sorting Null values last in MongoDB

I'm using the following query to populate items from MongoDB, in ascending order, according to a field called sortIndex.
Sometimes though items in the DB don't have the sortIndex field. With the following query, the items with a null sortIndex are showing up at the top, and I'm wondering how to get them to show up at the bottom. Would I need two queries for this or is there a way to use one query?
.populate({path: 'slides', options: { sort: { 'sortIndex': 'ascending' } } })
You can do something like this:
db.collection.aggregate([
{ $addFields:
{
hasValue : { $cond: [ { $eq: [ "$value", null ] }, 2, 1 ] },
}
},
])
.sort({hasValue : 1, value : 1});
Duplicate of: How to keep null values at the end of sorting in Mongoose?
Anyway posting the same solution ...
Am not sure about the solution am about to say. I cant test this out as I dont have a mongo db set right now, but I think that you can use <collection>.aggregate along with $project and $sort to achieve this.
Sample code:
db.inventory.aggregate(
[
{
$project: {
item: 1,
description: { $ifNull: [ "$amount", -1*(<mimimum value>)* ] }
}
},
{
$sort : {
amount : (-1 or 1 depending on the order you want)
}
}
]
)
Hope this helps !!

Mongoose: Sorting

what's the best way to sort the following documents in a collection:
{"topic":"11.Topic","text":"a.Text"}
{"topic":"2.Topic","text":"a.Text"}
{"topic":"1.Topic","text":"a.Text"}
I am using the following
find.(topic:req.body.topic).(sort({topic:1}))
but is not working (because the fields are strings and not numbers so I get):
{"topic":"1.Topic","text":"a.Text"},
{"topic":"11.Topic","text":"a.Text"},
{"topic":"2.Topic","text":"a.Text"}
but i'd like to get:
{"topic":"1.Topic","text":"a.Text"},
{"topic":"2.Topic","text":"a.Text"},
{"topic":"11.Topic","text":"a.Text"}
I read another post here that this will require complex sorting which mongoose doesn't have. So perhaps there is no real solution with this architecture?
Your help is greatly appreciated
i will suggest you make your topic filed as type : Number, and create another field topic_text.
Your Schema would look like:
var documentSchema = new mongoose.Schema({
topic : Number,
topic_text : String,
text : String
});
Normal document would look something like this:
{document1:[{"topic":11,"topic_text" : "Topic" ,"text":"a.Text"},
{"topic":2,"topic_text" : "Topic","text":"a.Text"},
{"topic":1,"topic_text" : "Topic","text":"a.Text"}]}
Thus, you will be able to use .sort({topic : 1}) ,and get the result you want.
while using topic value, append topic_text to it.
find(topic:req.body.topic).sort({topic:1}).exec(function(err,result)
{
var topic = result[0].topic + result[0].topic_text;//use index i to extract the value from result array.
})
If you do not want (or maybe do not even can) change the shape of your documents to include a numeric field for the topic number then you can achieve your desired sorting with the aggregation framework.
The following pipeline essentially splits the topic strings like '11.Topic' by the dot '.' and then prefixes the first part of the resulting array with a fixed number of leading zeros so that sorting by those strings will result in 'emulated' numeric sorting.
Note however that this pipeline uses $split and $strLenBytes operators which are pretty new so you may have to update your mongoDB instance - I used version 3.3.10.
db.getCollection('yourCollection').aggregate([
{
$project: {
topic: 1,
text: 1,
tmp: {
$let: {
vars: {
numStr: { $arrayElemAt: [{ $split: ["$topic", "."] }, 0] }
},
in: {
topicNumStr: "$$numStr",
topicNumStrLen: { $strLenBytes: "$$numStr" }
}
}
}
}
},
{
$project: {
topic: 1,
text: 1,
topicNumber: { $substr: [{ $concat: ["_0000", "$tmp.topicNumStr"] }, "$tmp.topicNumStrLen", 5] },
}
},
{
$sort: { topicNumber: 1 }
},
{
$project: {
topic: 1,
text: 1
}
}
])

Mongo $and selector

How does mongo $and selector work? I have trouble getting correct results back.
Let's say I have a collection something like this:
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 5 }
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 9 }
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 34 }
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 66 }
and I run query:
var selectorMin = 9;
var selectorMax = 42;
ScrapReport.find({ $and: [ { quantity: { $gte: selectorMin }, quantity: { $lte: selectorMax } } ] })
I would expect mongo to return me only 9 and 34. But for some reason it also returns 5 and 66.
What's wrong with my query?
Your query is returning all the documents in that sample because it is first looking for documents whose quantity >= 9 i.e. 9, 34 and 66 AND combines that query with documents whose quantity <= 42 i.e 34, 9 and 5. It's not looking for documents within a particular range but your query explicitly looks for all documents that satisify two ranges i.e.
Documents which satisfy "quantity >= 9"
+
Documents which satisfy "quantity <= 42"
not
Documents which satisfy "9 <= quantity <= 42"
Just simplify your query to
ScrapReport.find({ "quantity": { "$gte": selectorMin, "$lte": selectorMax } })
That way, you specify a range for MongoDB to filter your documents with i.e.
9 <= quantity <= 42
Specifying a comma separated list of expressions implies an implicit AND operation and use an explicit AND with the $and operator when when the same field or operator has to be specified in multiple expressions.
Using an implicit AND operation like the other answers suggested would work. But I would like to dig deeper into the specifics. Why is your query not working as you expected it to work?
Why? Why is this seemingly correct query of yours returning not so correct results? After all, whether you use implicit or explicit AND operation should be a matter of your choice and you should be able to achieve your goal irrespective of which form you use. How to make your query work with an explicit AND operation?
Let us look at the syntax of the AND operation.
{ $and: [ { <expression1> }, { <expression2> } , ... , { <expressionN> } ] }
The value of your AND operator should be an array containing expressions on which you would like to perform the AND operation.
After a first glance at your query, everything looks fine. But if you take a moment to look deeper, you would see that your query is not matching the AND syntax exactly. It is still syntactically correct. No doubt about that. But it is logically incorrect. I will explain how.
This is your $and operator value
{ $and: [ { quantity: { $gte: selectorMin }, quantity: { $lte: selectorMax } } ] }
You think you have an expression1 quantity: { $gte: selectorMin } and an expression2 quantity: { $lte: selectorMax }. An AND operation with these expressions should return the documents with quantity 9 and 34. But actually, all you have is one expression. Pay close attention to the braces. You have added both these expressions in a single {} block. Do you see it? So effectively, there is no 2nd expression for the AND operator to work with. But AND operator requires two or more expressions to function properly.
So your query is of the form
{ $and: [ { <expression1> } ] }
With an incorrect form, the results will also be incorrect. The correct query using an explicit AND operation would be
ScrapReport.find({ $and: [ { quantity: { $gte: selectorMin } }, { quantity: { $lte: selectorMax } } ] })
Do you see the difference? Try this query and you will get the results that you expected in the first place.
If you are not satisfied by just having the answer and are curious to know how Mongo interpreted your first query, read further.
Consider this query
ScrapReport.find({ quantity: 9 })
What would you expect the result to be? If you expected Mongo to return a single document whose value in the quantity field is 9, you are right. That is exactly what the result is. Now consider the same query with a small twist.
ScrapReport.find({ quantity: 9, quantity: 5 })
What would the result be now? Things are getting interesting now, huh? If you execute this query and have a look at the result, you will still see only a single document. But the value in the quantity field is 5. Now that is interesting!
ScrapReport.find({ quantity: 9, quantity: 5, quantity: 34 })
What about this? The result is still a single document with value in the quantity field being 34. You can try other combinations. What you will find out is this -
Within an expression, if you are referencing a field multiple times, the result will be determined by the last reference to that field in that expression.
Now apply this concept to your original query. It has already been pointed out that you have a single expression with two parts quantity: { $gte: selectorMin } and quantity: { $lte: selectorMax }. Since within an expression, you are referring to the same field twice, only the last one will be relevant. The selection criteria will be quantity: { $lte: selectorMax }. The result will be 3 documents with quantity values 5, 9 and 34.
If you swap the order i.e. write quantity: { $lte: selectorMax } first and then quantity: { $gte: selectorMin }, the selection criteria will now be determined by quantity: { $gte: selectorMin }. The result will be 3 documents with quantity values 9, 34 and 66.
Although it wasn't your intention, your original query is effectively
ScrapReport.find({ quantity: { $gte: selectorMin }, quantity: { $lte: selectorMax } })
When you miss braces or add them at the wrong position, it can completely change the meaning of your query.
Moral - Pay close attention to where you place your braces in complex queries.
Actually you have two problems in there:
Your query is equivalent to the following:
ScrapReport.find( { "$and": [{ "quantity": { "$lte": selectorMax } } ] } )
or even better:
ScrapReport.find( { "quantity": { "$lte": selectorMax } } )
The reason is because duplicate key are allowed in JSON document but the last value for a given key is maintained.
So this will only return all those documents where "quantity" is less than or equal selectorMax.
The second problem is already mentioned in #chridam's answer so the right query is:
ScrapReport.find({ "quantity": { "$gte": selectorMin, "$lte": selectorMax } })

How to weight documents to create sort criteria?

I'm trying to aggregate a collection in which there are documents that look like this:
[
{
"title" : 1984,
"tags" : ['dystopia', apocalypse', 'future',....]
},
....
]
And I have a criteria array of keywords, for instance:
var keywords = ['future', 'google', 'cat',....]
What I would like to achieve is to aggregate the collection in order to $group it according to a "convenience" criteria in order to sort the document by the one that contains the more of the keywords in its tags field.
This means, if one document contains in its tags: 'future', 'google', 'cat' it will be sorted before another one that has 'future', 'cat', 'apple'.
So far, I have tried something like this:
db.books.aggregate(
{ $group : { _id : {title:"$title"} , convenience: { $sum: { $cond: [ {tags: {$in: keywords}}, 1, 0 ] } } } },
{ $sort : {'convenience': -1}})
But the $in operator is not a boolean so it does not work. I've looked around and didn't find any operator that could help me with this.
As you said you need a logical operator in order to evaluate $cond. It's a bit terse, but here is an implementation using $or :
db.books.aggregate([
{$unwind: "$tags" },
{$group: {
_id: "$title",
weight: {
$sum: {$cond: [
// Test *equality* of the `tags` value against any of the list
{$or: [
{$eq: ["$tags", "future"]},
{$eq: ["$tags", "google"]},
{$eq: ["$tags", "cat"]},
]},
1, 0 ]}
}
}}
])
I'll leave the rest of the implementation up to you, but this should show the basic construction to the point of the matching you want to do.
Addition
From your comments there also seem to be a programming issue you are struggling with, related to how you perform an aggregation like this where you have an Array of items to query in the form you gave above:
var keywords = ['future', 'google', 'cat',....]
Since this structure cannot be directly employed in the pipeline condition, what you need to do is transform it into what you need. Each language has it's own approach, but in a JavaScript version:
var keywords = ['future', 'google', 'cat'];
var orCondition = [];
keywords.forEach(function(value) {
var doc = {$eq: [ "$tags", value ]};
orCondition.push(doc);
});
And then just define the aggregation query with the orCondition variable in place:
db.books.aggregate([
{$unwind: "$tags" },
{$group: {
_id: "$title",
weight: {
$sum: {$cond: [
{$or: orCondition }
1, 0 ]}
}
}}
])
Or for that matter, any of the parts you need to construct. This is generally how it is done in the real world, where we would almost never hard-code a data structure like this.

Categories

Resources