How does mongo $and selector work? I have trouble getting correct results back.
Let's say I have a collection something like this:
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 5 }
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 9 }
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 34 }
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 66 }
and I run query:
var selectorMin = 9;
var selectorMax = 42;
ScrapReport.find({ $and: [ { quantity: { $gte: selectorMin }, quantity: { $lte: selectorMax } } ] })
I would expect mongo to return me only 9 and 34. But for some reason it also returns 5 and 66.
What's wrong with my query?
Your query is returning all the documents in that sample because it is first looking for documents whose quantity >= 9 i.e. 9, 34 and 66 AND combines that query with documents whose quantity <= 42 i.e 34, 9 and 5. It's not looking for documents within a particular range but your query explicitly looks for all documents that satisify two ranges i.e.
Documents which satisfy "quantity >= 9"
+
Documents which satisfy "quantity <= 42"
not
Documents which satisfy "9 <= quantity <= 42"
Just simplify your query to
ScrapReport.find({ "quantity": { "$gte": selectorMin, "$lte": selectorMax } })
That way, you specify a range for MongoDB to filter your documents with i.e.
9 <= quantity <= 42
Specifying a comma separated list of expressions implies an implicit AND operation and use an explicit AND with the $and operator when when the same field or operator has to be specified in multiple expressions.
Using an implicit AND operation like the other answers suggested would work. But I would like to dig deeper into the specifics. Why is your query not working as you expected it to work?
Why? Why is this seemingly correct query of yours returning not so correct results? After all, whether you use implicit or explicit AND operation should be a matter of your choice and you should be able to achieve your goal irrespective of which form you use. How to make your query work with an explicit AND operation?
Let us look at the syntax of the AND operation.
{ $and: [ { <expression1> }, { <expression2> } , ... , { <expressionN> } ] }
The value of your AND operator should be an array containing expressions on which you would like to perform the AND operation.
After a first glance at your query, everything looks fine. But if you take a moment to look deeper, you would see that your query is not matching the AND syntax exactly. It is still syntactically correct. No doubt about that. But it is logically incorrect. I will explain how.
This is your $and operator value
{ $and: [ { quantity: { $gte: selectorMin }, quantity: { $lte: selectorMax } } ] }
You think you have an expression1 quantity: { $gte: selectorMin } and an expression2 quantity: { $lte: selectorMax }. An AND operation with these expressions should return the documents with quantity 9 and 34. But actually, all you have is one expression. Pay close attention to the braces. You have added both these expressions in a single {} block. Do you see it? So effectively, there is no 2nd expression for the AND operator to work with. But AND operator requires two or more expressions to function properly.
So your query is of the form
{ $and: [ { <expression1> } ] }
With an incorrect form, the results will also be incorrect. The correct query using an explicit AND operation would be
ScrapReport.find({ $and: [ { quantity: { $gte: selectorMin } }, { quantity: { $lte: selectorMax } } ] })
Do you see the difference? Try this query and you will get the results that you expected in the first place.
If you are not satisfied by just having the answer and are curious to know how Mongo interpreted your first query, read further.
Consider this query
ScrapReport.find({ quantity: 9 })
What would you expect the result to be? If you expected Mongo to return a single document whose value in the quantity field is 9, you are right. That is exactly what the result is. Now consider the same query with a small twist.
ScrapReport.find({ quantity: 9, quantity: 5 })
What would the result be now? Things are getting interesting now, huh? If you execute this query and have a look at the result, you will still see only a single document. But the value in the quantity field is 5. Now that is interesting!
ScrapReport.find({ quantity: 9, quantity: 5, quantity: 34 })
What about this? The result is still a single document with value in the quantity field being 34. You can try other combinations. What you will find out is this -
Within an expression, if you are referencing a field multiple times, the result will be determined by the last reference to that field in that expression.
Now apply this concept to your original query. It has already been pointed out that you have a single expression with two parts quantity: { $gte: selectorMin } and quantity: { $lte: selectorMax }. Since within an expression, you are referring to the same field twice, only the last one will be relevant. The selection criteria will be quantity: { $lte: selectorMax }. The result will be 3 documents with quantity values 5, 9 and 34.
If you swap the order i.e. write quantity: { $lte: selectorMax } first and then quantity: { $gte: selectorMin }, the selection criteria will now be determined by quantity: { $gte: selectorMin }. The result will be 3 documents with quantity values 9, 34 and 66.
Although it wasn't your intention, your original query is effectively
ScrapReport.find({ quantity: { $gte: selectorMin }, quantity: { $lte: selectorMax } })
When you miss braces or add them at the wrong position, it can completely change the meaning of your query.
Moral - Pay close attention to where you place your braces in complex queries.
Actually you have two problems in there:
Your query is equivalent to the following:
ScrapReport.find( { "$and": [{ "quantity": { "$lte": selectorMax } } ] } )
or even better:
ScrapReport.find( { "quantity": { "$lte": selectorMax } } )
The reason is because duplicate key are allowed in JSON document but the last value for a given key is maintained.
So this will only return all those documents where "quantity" is less than or equal selectorMax.
The second problem is already mentioned in #chridam's answer so the right query is:
ScrapReport.find({ "quantity": { "$gte": selectorMin, "$lte": selectorMax } })
Related
I have an array of numbers [111, 444, 777]
I want to look inside these 2 documents
{
_id: 1,
lookHere: {rangeLow: 333, rangeHigh: 555}
},
{
_id: 2,
lookHere: {rangeLow: 222, rangeHigh: 333}
}
I want to write a query that returns only the 1st document since my array of numbers includes 444 which is between 333 and 555.
Is there a query that can achieve this result in mongo/mongoose ?
You want an $or query. You can .map() the source array into the arguments for $or:
var inputs = [111, 444, 777];
collection.find({
"$or": inputs.map( n => ({
"lookHere.rangeLow": { "$lt": n },
"lookHere.rangeHigh": { "$gt": n }
}) )
})
That is basically looking to see if the Low value is less than and the High is greater than each of the current elements, and returns as true when any match both those conditions.
Note that all MongoDB query arguments are implicitly AND conditions unless stated otherwise.
I'm trying to query my database for prices greater than/less than a user specified number. In my database, prices are stored like so:
{price: "300.00"}
According to the docs, this should work:
db.products.find({price: {$gt:30.00}}).pretty()
But I get no results returned. I've also tried {price: {$gt:30}}.
What am I missing here?
It it because the prices are stored as a string rather than a number in the DB? Is there a way around this?
If you intend to use $gt with strings, you will have to use regex, which is not great in terms of performance. It is easier to just create a new field which holds the number value of price or change this field type to int/double. A javascript version should also work, like so:
db.products.find("this.price > 30.00")
as js will convert it to number before use. However, indexes won't work on this query.
$gt is an operator that can work on any type:
db.so.drop();
db.so.insert( { name: "Derick" } );
db.so.insert( { name: "Jayz" } );
db.so.find( { name: { $gt: "Fred" } } );
Returns:
{ "_id" : ObjectId("51ffbe6c16473d7b84172d58"), "name" : "Jayz" }
If you want to compare against a number with $gt or $lt, then the value in your document also needs to be a number. Types in MongoDB are strict and do not auto-convert like they f.e. would do in PHP. In order to solve your issue, make sure you store the prices as numbers (floats or ints):
db.so.drop();
db.so.insert( { price: 50.40 } );
db.so.insert( { price: 29.99 } );
db.so.find( { price: { $gt: 30 } } );
Returns:
{ "_id" : ObjectId("51ffbf2016473d7b84172d5b"), "price" : 50.4 }
Starting Mongo 4.0, there is a new $toDouble aggregation operator which converts from various types to double (in this case from a string):
// { price: "300.00" }
// { price: "4.2" }
db.collection.find({ $expr: { $gt: [{ $toDouble: "$price" }, 30] } })
// { price: "300.00" }
If you have newer version of mongodb then you can do this:
$expr: {
$gt: [
{ $convert: { input: '$price', to: 'decimal' } },
{ $convert: { input: '0.0', to: 'decimal' } }
]
}
$expr operator: https://docs.mongodb.com/manual/reference/operator/query/expr/
$convert opetator: https://docs.mongodb.com/manual/reference/operator/aggregation/convert/index.html
Alternatively you can convert the values to Int, as per:
http://www.quora.com/How-can-I-change-a-field-type-from-String-to-Integer-in-mongodb
var convert = function(document){
var intValue = parseInt(document.field, 10);
db.collection.update(
{_id:document._id},
{$set: {field: intValue}}
);
}
db.collection.find({field: {$type:2}},{field:1}).forEach(convert)
I had an output document from a pipeline something like : {upvotes : 1, downvotes : 5}. I just wanted to pass that along to the next part of the pipeline. I did something like this:
{
$group :{
_id : "$_id",
upvotes : {$max : "$upvotes"},
downvotes : {$max : "$downvotes"},
average : {$avg : "$statements.result"},
count : {$max : "$count"}
}
}
I didn't want to use max I don't want the maximum of anything. I just wanted the field and value: {upvotes : 1, downvotes : 5}. lets say they were not numbers. lets say the fields were object I want all the objects for that field to be outputted not the max.
make this question clearer: how can I get the the field and value without using max?
outputs:
[ { _id: 57aea6791016c0b023a71e9d,
upvotes: 3,
downvotes: 1,
average: 7.857142857142857,
count: 5 },
{ _id: 57aed883139c1e702beb471e,
upvotes: 0,
downvotes: 1,
average: 7,
count: 1 } ]
The output is good but I don't want to use max to get it.
For some reason I think I need an Accumulator Operator. That is why I use max.
Since you are grouping by "$_id", you're actually not grouping at all. Your results are going to correlate 1:1 to your initial data set. To accomplish what you are asking, passing along upvotes and downvotes without using an aggregate function like $max, you'd just include them in the _id for your group stage which is where you specify your group by key.
So you would do:
{
$group :{
_id : {
"_id":"$_id",
"upvotes": "$upvotes",
"downvotes": "$downvotes"
}
average : {$avg : "$statements.result"},
count : {$max : "$count"}
}
}
But still, you are going to end up with a 1:1 correlation to your original data.
Following your question thread, your pipeline in question is most probably a preceding pipeline step from an $unwind operation thence the following suggested solution is based on that assumption. Please feel free to suggest otherwise.
To include a field in the $group pipeline using the accumulators, the $first or $last seem most appropriate here since you have just flattened an array and those operators will return a value from the first/last document for each group. What is means is if the field is not part of a flattened array then $first/$last would give you its actual value before the $unwind took place.
{
"$group" :{
"_id": "$_id",
"upvotes": { "$first": "$upvotes"},
"downvotes": { "$first": "$downvotes"},
"average": { "$avg": "$statements.result"},
"count": { "$max": "$count"}
}
}
what's the best way to sort the following documents in a collection:
{"topic":"11.Topic","text":"a.Text"}
{"topic":"2.Topic","text":"a.Text"}
{"topic":"1.Topic","text":"a.Text"}
I am using the following
find.(topic:req.body.topic).(sort({topic:1}))
but is not working (because the fields are strings and not numbers so I get):
{"topic":"1.Topic","text":"a.Text"},
{"topic":"11.Topic","text":"a.Text"},
{"topic":"2.Topic","text":"a.Text"}
but i'd like to get:
{"topic":"1.Topic","text":"a.Text"},
{"topic":"2.Topic","text":"a.Text"},
{"topic":"11.Topic","text":"a.Text"}
I read another post here that this will require complex sorting which mongoose doesn't have. So perhaps there is no real solution with this architecture?
Your help is greatly appreciated
i will suggest you make your topic filed as type : Number, and create another field topic_text.
Your Schema would look like:
var documentSchema = new mongoose.Schema({
topic : Number,
topic_text : String,
text : String
});
Normal document would look something like this:
{document1:[{"topic":11,"topic_text" : "Topic" ,"text":"a.Text"},
{"topic":2,"topic_text" : "Topic","text":"a.Text"},
{"topic":1,"topic_text" : "Topic","text":"a.Text"}]}
Thus, you will be able to use .sort({topic : 1}) ,and get the result you want.
while using topic value, append topic_text to it.
find(topic:req.body.topic).sort({topic:1}).exec(function(err,result)
{
var topic = result[0].topic + result[0].topic_text;//use index i to extract the value from result array.
})
If you do not want (or maybe do not even can) change the shape of your documents to include a numeric field for the topic number then you can achieve your desired sorting with the aggregation framework.
The following pipeline essentially splits the topic strings like '11.Topic' by the dot '.' and then prefixes the first part of the resulting array with a fixed number of leading zeros so that sorting by those strings will result in 'emulated' numeric sorting.
Note however that this pipeline uses $split and $strLenBytes operators which are pretty new so you may have to update your mongoDB instance - I used version 3.3.10.
db.getCollection('yourCollection').aggregate([
{
$project: {
topic: 1,
text: 1,
tmp: {
$let: {
vars: {
numStr: { $arrayElemAt: [{ $split: ["$topic", "."] }, 0] }
},
in: {
topicNumStr: "$$numStr",
topicNumStrLen: { $strLenBytes: "$$numStr" }
}
}
}
}
},
{
$project: {
topic: 1,
text: 1,
topicNumber: { $substr: [{ $concat: ["_0000", "$tmp.topicNumStr"] }, "$tmp.topicNumStrLen", 5] },
}
},
{
$sort: { topicNumber: 1 }
},
{
$project: {
topic: 1,
text: 1
}
}
])
I have a query:
ownUnnamedPages = Entries.find( { author : this.userId, title : {$regex: /^unnamed-/ }}, {sort: { title: 1 }}).fetch()
That returns the following array sorted:
[ {
title: 'unnamed-1',
text: '<p>sdaasdasdasd</p>',
tags: [],
_id: 'Wkxxpapm8bbiq59ig',
author: 'AHSwfYgeGmur9oHzu',
visibility: 'public' },
{
title: 'unnamed-10',
text: '',
author: 'AHSwfYgeGmur9oHzu',
visibility: 'public',
_id: 'aDSN2XFjQPh9HPu4c' },
{
title: 'unnamed-2',
text: '<p>kkhjk</p>',
tags: [],
_id: 'iM9FMCsyzehQvYGKj',
author: 'AHSwfYgeGmur9oHzu',
visibility: 'public' },
{
title: 'unnamed-3',
text: '',
tags: [],
_id: 'zK2w9MEQGnwsm3Cqh',
author: 'AHSwfYgeGmur9oHzu',
visibility: 'public' }]
The problem is that it seems to sort on the first numeric character so it thinks the proper sequence is 1, 10, 2, 3, etc....
what I really want is for it to sort on both the whole numerical part so that 10 would be at the end.
I'd prefer not to do this by having additional numbers such as 01 or 001 for the numbers.
How would I do that?
You can use
db.collectionName.find().sort({title: 1}).collation({locale: "en_US", numericOrdering: true})
numericOrdering flag is boolean and is Optional. Flag that determines whether to compare numeric strings as numbers or as strings.
If true, compare as numbers; i.e. "10" is greater than "2".
If false, compare as strings; i.e. "10" is less than "2".
Default is false.
See mongo's collation documentation for an updated explanation of those fields.
MongoDB can't sort by numbers stored as strings. You either have to store the number as an integer in its own field, pad with leading zeroes, or sort the results after they've been returned from the database.
If you 0 pad the numbers you will be able to search as a string in the right order, so instead of 0,1,2,3,4,5,6,7,8,9,10,11...
use 01,02,03,04,05,06,07,08,09,10,11...
and a string search will return them in order.
The mongo documentation said you can use Collation for this goal
as #Eugene Kaurov said you can use
.collation({locale: "en_US", numericOrdering: true})
this is the official documentation:
mongo ref
and be aware that the accepted answer is not correct now
In mongo is not possible (sort strings in ascii) but you can sort with the below function after you get all documents from the collection
const sortString = (a, b) => {
const AA = a.title.split('-');
const BB = b.title.split('-');
if (parseInt(AA[1], 10) === parseInt(BB[1], 10)) {
return 0;
}
return (parseInt(AA[1], 10) < parseInt(BB[1], 10)) ? -1 : 1;
};
document.sort(sortString);
In my case we work with aggregations. The approach was to sort using the length of our string; only works when the text part is always the same (unnamed- in your case)
db.YourCollection.aggregate([
{
$addFields: {
"TitleSize": { $strLenCP: "$Title" }
}
},
{
$sort: {
"TitleIdSize": 1,
"Title": 1
}
}
]);
Now we sort using length, the second sort will use the content.
Example:
"unnamed-2", Titlesize: 9
"unnamed-7", Titlesize: 9
"unnamed-30", Titlesize: 10
"unnamed-1", Titlesize: 9
The first sort will put the ids in this order: 2, 7, 1, 30. Then the second sort will put the ids in the correct order: 1, 2, 7, 30.