what's the best way to sort the following documents in a collection:
{"topic":"11.Topic","text":"a.Text"}
{"topic":"2.Topic","text":"a.Text"}
{"topic":"1.Topic","text":"a.Text"}
I am using the following
find.(topic:req.body.topic).(sort({topic:1}))
but is not working (because the fields are strings and not numbers so I get):
{"topic":"1.Topic","text":"a.Text"},
{"topic":"11.Topic","text":"a.Text"},
{"topic":"2.Topic","text":"a.Text"}
but i'd like to get:
{"topic":"1.Topic","text":"a.Text"},
{"topic":"2.Topic","text":"a.Text"},
{"topic":"11.Topic","text":"a.Text"}
I read another post here that this will require complex sorting which mongoose doesn't have. So perhaps there is no real solution with this architecture?
Your help is greatly appreciated
i will suggest you make your topic filed as type : Number, and create another field topic_text.
Your Schema would look like:
var documentSchema = new mongoose.Schema({
topic : Number,
topic_text : String,
text : String
});
Normal document would look something like this:
{document1:[{"topic":11,"topic_text" : "Topic" ,"text":"a.Text"},
{"topic":2,"topic_text" : "Topic","text":"a.Text"},
{"topic":1,"topic_text" : "Topic","text":"a.Text"}]}
Thus, you will be able to use .sort({topic : 1}) ,and get the result you want.
while using topic value, append topic_text to it.
find(topic:req.body.topic).sort({topic:1}).exec(function(err,result)
{
var topic = result[0].topic + result[0].topic_text;//use index i to extract the value from result array.
})
If you do not want (or maybe do not even can) change the shape of your documents to include a numeric field for the topic number then you can achieve your desired sorting with the aggregation framework.
The following pipeline essentially splits the topic strings like '11.Topic' by the dot '.' and then prefixes the first part of the resulting array with a fixed number of leading zeros so that sorting by those strings will result in 'emulated' numeric sorting.
Note however that this pipeline uses $split and $strLenBytes operators which are pretty new so you may have to update your mongoDB instance - I used version 3.3.10.
db.getCollection('yourCollection').aggregate([
{
$project: {
topic: 1,
text: 1,
tmp: {
$let: {
vars: {
numStr: { $arrayElemAt: [{ $split: ["$topic", "."] }, 0] }
},
in: {
topicNumStr: "$$numStr",
topicNumStrLen: { $strLenBytes: "$$numStr" }
}
}
}
}
},
{
$project: {
topic: 1,
text: 1,
topicNumber: { $substr: [{ $concat: ["_0000", "$tmp.topicNumStr"] }, "$tmp.topicNumStrLen", 5] },
}
},
{
$sort: { topicNumber: 1 }
},
{
$project: {
topic: 1,
text: 1
}
}
])
Related
I'm using the following query to populate items from MongoDB, in ascending order, according to a field called sortIndex.
Sometimes though items in the DB don't have the sortIndex field. With the following query, the items with a null sortIndex are showing up at the top, and I'm wondering how to get them to show up at the bottom. Would I need two queries for this or is there a way to use one query?
.populate({path: 'slides', options: { sort: { 'sortIndex': 'ascending' } } })
You can do something like this:
db.collection.aggregate([
{ $addFields:
{
hasValue : { $cond: [ { $eq: [ "$value", null ] }, 2, 1 ] },
}
},
])
.sort({hasValue : 1, value : 1});
Duplicate of: How to keep null values at the end of sorting in Mongoose?
Anyway posting the same solution ...
Am not sure about the solution am about to say. I cant test this out as I dont have a mongo db set right now, but I think that you can use <collection>.aggregate along with $project and $sort to achieve this.
Sample code:
db.inventory.aggregate(
[
{
$project: {
item: 1,
description: { $ifNull: [ "$amount", -1*(<mimimum value>)* ] }
}
},
{
$sort : {
amount : (-1 or 1 depending on the order you want)
}
}
]
)
Hope this helps !!
I'm trying to query my database for prices greater than/less than a user specified number. In my database, prices are stored like so:
{price: "300.00"}
According to the docs, this should work:
db.products.find({price: {$gt:30.00}}).pretty()
But I get no results returned. I've also tried {price: {$gt:30}}.
What am I missing here?
It it because the prices are stored as a string rather than a number in the DB? Is there a way around this?
If you intend to use $gt with strings, you will have to use regex, which is not great in terms of performance. It is easier to just create a new field which holds the number value of price or change this field type to int/double. A javascript version should also work, like so:
db.products.find("this.price > 30.00")
as js will convert it to number before use. However, indexes won't work on this query.
$gt is an operator that can work on any type:
db.so.drop();
db.so.insert( { name: "Derick" } );
db.so.insert( { name: "Jayz" } );
db.so.find( { name: { $gt: "Fred" } } );
Returns:
{ "_id" : ObjectId("51ffbe6c16473d7b84172d58"), "name" : "Jayz" }
If you want to compare against a number with $gt or $lt, then the value in your document also needs to be a number. Types in MongoDB are strict and do not auto-convert like they f.e. would do in PHP. In order to solve your issue, make sure you store the prices as numbers (floats or ints):
db.so.drop();
db.so.insert( { price: 50.40 } );
db.so.insert( { price: 29.99 } );
db.so.find( { price: { $gt: 30 } } );
Returns:
{ "_id" : ObjectId("51ffbf2016473d7b84172d5b"), "price" : 50.4 }
Starting Mongo 4.0, there is a new $toDouble aggregation operator which converts from various types to double (in this case from a string):
// { price: "300.00" }
// { price: "4.2" }
db.collection.find({ $expr: { $gt: [{ $toDouble: "$price" }, 30] } })
// { price: "300.00" }
If you have newer version of mongodb then you can do this:
$expr: {
$gt: [
{ $convert: { input: '$price', to: 'decimal' } },
{ $convert: { input: '0.0', to: 'decimal' } }
]
}
$expr operator: https://docs.mongodb.com/manual/reference/operator/query/expr/
$convert opetator: https://docs.mongodb.com/manual/reference/operator/aggregation/convert/index.html
Alternatively you can convert the values to Int, as per:
http://www.quora.com/How-can-I-change-a-field-type-from-String-to-Integer-in-mongodb
var convert = function(document){
var intValue = parseInt(document.field, 10);
db.collection.update(
{_id:document._id},
{$set: {field: intValue}}
);
}
db.collection.find({field: {$type:2}},{field:1}).forEach(convert)
How does mongo $and selector work? I have trouble getting correct results back.
Let's say I have a collection something like this:
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 5 }
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 9 }
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 34 }
{ "_id" : "F7mdaZC2eBQDXA5wx", "quantity" : 66 }
and I run query:
var selectorMin = 9;
var selectorMax = 42;
ScrapReport.find({ $and: [ { quantity: { $gte: selectorMin }, quantity: { $lte: selectorMax } } ] })
I would expect mongo to return me only 9 and 34. But for some reason it also returns 5 and 66.
What's wrong with my query?
Your query is returning all the documents in that sample because it is first looking for documents whose quantity >= 9 i.e. 9, 34 and 66 AND combines that query with documents whose quantity <= 42 i.e 34, 9 and 5. It's not looking for documents within a particular range but your query explicitly looks for all documents that satisify two ranges i.e.
Documents which satisfy "quantity >= 9"
+
Documents which satisfy "quantity <= 42"
not
Documents which satisfy "9 <= quantity <= 42"
Just simplify your query to
ScrapReport.find({ "quantity": { "$gte": selectorMin, "$lte": selectorMax } })
That way, you specify a range for MongoDB to filter your documents with i.e.
9 <= quantity <= 42
Specifying a comma separated list of expressions implies an implicit AND operation and use an explicit AND with the $and operator when when the same field or operator has to be specified in multiple expressions.
Using an implicit AND operation like the other answers suggested would work. But I would like to dig deeper into the specifics. Why is your query not working as you expected it to work?
Why? Why is this seemingly correct query of yours returning not so correct results? After all, whether you use implicit or explicit AND operation should be a matter of your choice and you should be able to achieve your goal irrespective of which form you use. How to make your query work with an explicit AND operation?
Let us look at the syntax of the AND operation.
{ $and: [ { <expression1> }, { <expression2> } , ... , { <expressionN> } ] }
The value of your AND operator should be an array containing expressions on which you would like to perform the AND operation.
After a first glance at your query, everything looks fine. But if you take a moment to look deeper, you would see that your query is not matching the AND syntax exactly. It is still syntactically correct. No doubt about that. But it is logically incorrect. I will explain how.
This is your $and operator value
{ $and: [ { quantity: { $gte: selectorMin }, quantity: { $lte: selectorMax } } ] }
You think you have an expression1 quantity: { $gte: selectorMin } and an expression2 quantity: { $lte: selectorMax }. An AND operation with these expressions should return the documents with quantity 9 and 34. But actually, all you have is one expression. Pay close attention to the braces. You have added both these expressions in a single {} block. Do you see it? So effectively, there is no 2nd expression for the AND operator to work with. But AND operator requires two or more expressions to function properly.
So your query is of the form
{ $and: [ { <expression1> } ] }
With an incorrect form, the results will also be incorrect. The correct query using an explicit AND operation would be
ScrapReport.find({ $and: [ { quantity: { $gte: selectorMin } }, { quantity: { $lte: selectorMax } } ] })
Do you see the difference? Try this query and you will get the results that you expected in the first place.
If you are not satisfied by just having the answer and are curious to know how Mongo interpreted your first query, read further.
Consider this query
ScrapReport.find({ quantity: 9 })
What would you expect the result to be? If you expected Mongo to return a single document whose value in the quantity field is 9, you are right. That is exactly what the result is. Now consider the same query with a small twist.
ScrapReport.find({ quantity: 9, quantity: 5 })
What would the result be now? Things are getting interesting now, huh? If you execute this query and have a look at the result, you will still see only a single document. But the value in the quantity field is 5. Now that is interesting!
ScrapReport.find({ quantity: 9, quantity: 5, quantity: 34 })
What about this? The result is still a single document with value in the quantity field being 34. You can try other combinations. What you will find out is this -
Within an expression, if you are referencing a field multiple times, the result will be determined by the last reference to that field in that expression.
Now apply this concept to your original query. It has already been pointed out that you have a single expression with two parts quantity: { $gte: selectorMin } and quantity: { $lte: selectorMax }. Since within an expression, you are referring to the same field twice, only the last one will be relevant. The selection criteria will be quantity: { $lte: selectorMax }. The result will be 3 documents with quantity values 5, 9 and 34.
If you swap the order i.e. write quantity: { $lte: selectorMax } first and then quantity: { $gte: selectorMin }, the selection criteria will now be determined by quantity: { $gte: selectorMin }. The result will be 3 documents with quantity values 9, 34 and 66.
Although it wasn't your intention, your original query is effectively
ScrapReport.find({ quantity: { $gte: selectorMin }, quantity: { $lte: selectorMax } })
When you miss braces or add them at the wrong position, it can completely change the meaning of your query.
Moral - Pay close attention to where you place your braces in complex queries.
Actually you have two problems in there:
Your query is equivalent to the following:
ScrapReport.find( { "$and": [{ "quantity": { "$lte": selectorMax } } ] } )
or even better:
ScrapReport.find( { "quantity": { "$lte": selectorMax } } )
The reason is because duplicate key are allowed in JSON document but the last value for a given key is maintained.
So this will only return all those documents where "quantity" is less than or equal selectorMax.
The second problem is already mentioned in #chridam's answer so the right query is:
ScrapReport.find({ "quantity": { "$gte": selectorMin, "$lte": selectorMax } })
I have following data in MongoDB:
[{id:3132, home:'NSH', away:'BOS'}, {id:3112, home:'ANA', away:'CGY'}, {id:3232, home:'MIN', away:'NSH'}]
Is it possible to get total game count for each team with aggregate pipeline?
desired result:
[{team: 'NSH', totalGames: 2}, {team:'MIN', totalGames: 1}, ...}]
i can get each on seperately to their own arrays with two aggregate calls:
[{$group: {_id: "$home", gamesLeft: {$sum: 1}}}]
and
[{$group: {_id: "$away", gamesLeft: {$sum: 1}}}]
resulting
var homeGames = [ { _id: 'NSH', totalGames: 1 }, { _id: 'SJS', totalGames: 2 }, ...]
var awayGames = [ { _id: 'NSH', totalGames: 1 }, { _id: 'SJS', totalGames: 4 }, ...]
But i really want to get it working with just one query. If not possible what would be the best way to combine these two results in to one using javascript?
After some puzzling, I found a way to get it done using an aggregate pipeline. Here is the result:
db.games.aggregate([{
$project: {
isHome: { $literal: [true, false] },
home: true,
away: true
}
}, {
$unwind: '$isHome'
}, {
$group: {
_id: { $cond: { if: '$isHome', then: '$home', else: '$away' } },
totalGames: { $sum: 1 }
}
}
]);
As you can see it consists of three stages. The first two are meant to duplicate each document into one for the home team and one for the away team. To do this, the project stage first creates a new isHome field on each document containing a true and a false value, which the unwind stage then splits into separate documents containing either the true or the false value.
Then in the group phase, we let the isHome field decide whether to group on the home or the away field.
It would be nicer if we could create a team field in the project step, containing the array [$home, $away], but mongo only supports adding array literals here, hence the workaround.
I have a query:
ownUnnamedPages = Entries.find( { author : this.userId, title : {$regex: /^unnamed-/ }}, {sort: { title: 1 }}).fetch()
That returns the following array sorted:
[ {
title: 'unnamed-1',
text: '<p>sdaasdasdasd</p>',
tags: [],
_id: 'Wkxxpapm8bbiq59ig',
author: 'AHSwfYgeGmur9oHzu',
visibility: 'public' },
{
title: 'unnamed-10',
text: '',
author: 'AHSwfYgeGmur9oHzu',
visibility: 'public',
_id: 'aDSN2XFjQPh9HPu4c' },
{
title: 'unnamed-2',
text: '<p>kkhjk</p>',
tags: [],
_id: 'iM9FMCsyzehQvYGKj',
author: 'AHSwfYgeGmur9oHzu',
visibility: 'public' },
{
title: 'unnamed-3',
text: '',
tags: [],
_id: 'zK2w9MEQGnwsm3Cqh',
author: 'AHSwfYgeGmur9oHzu',
visibility: 'public' }]
The problem is that it seems to sort on the first numeric character so it thinks the proper sequence is 1, 10, 2, 3, etc....
what I really want is for it to sort on both the whole numerical part so that 10 would be at the end.
I'd prefer not to do this by having additional numbers such as 01 or 001 for the numbers.
How would I do that?
You can use
db.collectionName.find().sort({title: 1}).collation({locale: "en_US", numericOrdering: true})
numericOrdering flag is boolean and is Optional. Flag that determines whether to compare numeric strings as numbers or as strings.
If true, compare as numbers; i.e. "10" is greater than "2".
If false, compare as strings; i.e. "10" is less than "2".
Default is false.
See mongo's collation documentation for an updated explanation of those fields.
MongoDB can't sort by numbers stored as strings. You either have to store the number as an integer in its own field, pad with leading zeroes, or sort the results after they've been returned from the database.
If you 0 pad the numbers you will be able to search as a string in the right order, so instead of 0,1,2,3,4,5,6,7,8,9,10,11...
use 01,02,03,04,05,06,07,08,09,10,11...
and a string search will return them in order.
The mongo documentation said you can use Collation for this goal
as #Eugene Kaurov said you can use
.collation({locale: "en_US", numericOrdering: true})
this is the official documentation:
mongo ref
and be aware that the accepted answer is not correct now
In mongo is not possible (sort strings in ascii) but you can sort with the below function after you get all documents from the collection
const sortString = (a, b) => {
const AA = a.title.split('-');
const BB = b.title.split('-');
if (parseInt(AA[1], 10) === parseInt(BB[1], 10)) {
return 0;
}
return (parseInt(AA[1], 10) < parseInt(BB[1], 10)) ? -1 : 1;
};
document.sort(sortString);
In my case we work with aggregations. The approach was to sort using the length of our string; only works when the text part is always the same (unnamed- in your case)
db.YourCollection.aggregate([
{
$addFields: {
"TitleSize": { $strLenCP: "$Title" }
}
},
{
$sort: {
"TitleIdSize": 1,
"Title": 1
}
}
]);
Now we sort using length, the second sort will use the content.
Example:
"unnamed-2", Titlesize: 9
"unnamed-7", Titlesize: 9
"unnamed-30", Titlesize: 10
"unnamed-1", Titlesize: 9
The first sort will put the ids in this order: 2, 7, 1, 30. Then the second sort will put the ids in the correct order: 1, 2, 7, 30.