I need to use skip and limit for pagination, and the distinct for not return equal values.
If i use
MyModel.find().distinct('blaster', function(err, results) {
res.render('index', {
data: results
});
});
This works.
If i use
MyModel.find().sort('brand').skip((page-1)*15).limit(15).exec(function(err, results) {
res.render('index', {
data: results
});
});
This is also working, but how use both?
If i try, the error will show:
Error: skip cannot be used with distinct
You don't do that. .distinct() is a method that returns an "array", and therefore you cannot modify something that is not a "Cursor" with "cursor modifiers" like .limit() and .skip().
What you want is the .aggregate() method. Much more than just adding things up:
MyModel.aggregate(
[
{ "$group": { "_id": "$blaster" } },
{ "$skip": ( page-1 ) * 15 },
{ "$limit": 15 }
],
function(err,results) {
// results skipped and limited in here
}
);
The aggregation framework provides another way to achieve "distinct" results. But in a more flexible way. See the operators for $group, $skip and $limit.
Related
If a collection have a list of dogs, and there is duplicate entries on some races. How do i remove all, but a single specific/non specific one, from just one query?
I guess it would be possible to get all from a Model.find(), loop through every index except the first one and call Model.remove(), but I would rather have the database handle the logic through the query. How would this be possible?
pseudocode example of what i want:
Model.remove({race:"pitbull"}).where(notFirstOne);
To remove all but one, you need a way to get all the filtered documents, group them by the identifier, create a list of ids for the group and remove a single id from
this list. Armed with this info, you can then run another operation to remove the documents with those ids. Essentially you will be running two queries.
The first query is an aggregate operation that aims to get the list of ids with the potentially nuking documents:
(async () => {
// Get the duplicate entries minus 1
const [doc, ...rest] = await Module.aggregate([
{ '$match': { 'race': 'pitbull'} },
{ '$group': {
'_id': '$race',
'ids': { '$push': '$_id' },
'id': { '$first': '$_id' }
} },
{ '$project': { 'idsToRemove': { '$setDifference': [ ['$id'], '$ids' ] } } }
]);
const { idsToRemove } = doc;
// Remove the duplicate documents
Module.remove({ '_id': { '$in': idsToRemove } })
})();
if purpose is to keep only one, in case of concurrent writes, may as well just write
Module.findOne({race:'pitbull'}).select('_id')
//bla
Module.remove({race:'pitbull', _id:{$ne:idReturned}})
If it is to keep the very first one, mongodb does not guarantee results will be sorted by increasing _id (natural order refers to disk)
see Does default find() implicitly sort by _id?
so instead
Module.find({race:'pitbull'}).sort({_id:1}).limit(1)
Sorry if this is pretty basic, but I'm a mongodb newbie and haven't been able to find an answer to this:
Let's say I'm doing the following:
db.collection("bugs").updateOne({ _id: searchId }, { $set: { "fixed": true }}
How to set "fixed" to the contrary of whatever the last value of "fixed" was? Without any additional queries? Something like { $set: { "fixed": !fixed }}
It is not really possible to achieve this in MongoDB as of now in just one operation by sticking to the idea of storing boolean values (might be possible in future versions of MongoDB). But, there is a workaround to do this by storing bits (0 or 1) to represent true or false instead of boolean values and performing bitwise xor operation on those in MongoDB as follows:
db.collection("bugs").updateOne(
{
_id: searchId
},
{
$bit : {
fixed: {
xor: NumberInt(1)
}
}
}
)
Please note that you also have to store 0 as NumberInt(0) to represent false and 1 as NumberInt(1) to represent true in the fixed prop as MongoDB by default treats all numbers as floating-point values.
This is not possible in MongoDB. You have to retrieve the doc from the db and then update it:
var doc = db.collection("bugs").findOne({ _id: searchId });
db.collection("bugs").updateOne({ _id: searchId }, { $set: { "fixed": !doc.fixed } }
Yes, it's possible to do that with MongoDB, of course !! ... just use the find's forEach feature like this:
db.collection("bugs").find({ _id: searchId }).forEach(function(bugDoc) {
db.collection("bugs").updateOne({ _id: searchId }, { $set: { "fixed": !bugDoc.fixed }});
});
NOTE: bugDoc contains all the fields of the original document and you can make all the calculations and changes you want in this double operation update
I've stumbled upon some very strange behavior with MongoDB. For my test case, I have an MongoDB collection with 9 documents. All documents have the exact same structure, including the fields expired_at: Date and location: [lng, lat].
I now need to find all documents that are not expired yet and are within a bounding box; I show match documents on map. for this I set up the following queries:
var qExpiry = {"expired_at": { $gt : new Date() } };
var qLocation = { "location" : { $geoWithin : { $box : [ [ 123.8766, 8.3269 ] , [ 122.8122, 8.24974 ] ] } } };
var qFull = { $and: [ qExpiry, qLocation ] };
Since the expiry date is long in the past, and when I set the bounding box large enough, the following queries give me all 9 documents as expected:
db.docs.find(qExpiry);
db.docs.find(qLocation);
db.docs.find(qFull);
db.docs.find(qExpiry).sort({"created_at" : -1});
db.docs.find(qLocation).sort({"created_at" : -1});
Now here's the deal: The following query returns 0 documents:
db.docs.find(qFull).sort({"created_at" : -1});
Just adding sort to the AND query ruins the result (please note that I want to sort since I also have a limit in order to avoid cluttering the map on larger scales). Sorting by other fields yield the same empty result. What's going on here?
(Actually even stranger: When I zoom into my map, I sometimes get results for qFull, even with sorting. One could argue that qLocation is faulty. But when I only use qLocation, the results are always correct. And qExpiry is always true for all documents anyway)
You may want to try running the same query using the aggregation framework's $match and $sort pipelines:
db.docs.aggregate([
{ "$match": qFull },
{ "$sort": { "created_at": -1 } }
]);
or implicitly using $and by specifiying a comma-separated list of expressions as in
db.docs.aggregate([
{
"$match": {
"expired_at": { "$gt" : new Date() },
"location" : {
"$geoWithin" : {
"$box" : [
[ 123.8766, 8.3269 ],
[ 122.8122, 8.24974 ]
]
}
}
}
},
{ "$sort": { "created_at": -1 } }
]);
Not really sure why that fails with find()
chridam suggestion using the aggregation framework of MongoDB proved to be the way to go. My working query now looks like this:
db.docs.aggregate(
[
{ $match : { $and : [qExpiry, qLocation]} },
{ $sort: {"created_at": -1} }.
{ $limit: 50 }.
]
);
Nevertheless, if any can point out way my first approach did not work, that would be very useful. Simply adding sort() to a non-empty query shouldn't suddenly return 0 documents. Just to add, since I still tried for a bit, .sort({}) return all documents but was not very useful. Everything else failed including .sort({'_id': 1}).
I hit an API which follows 50 members' data in a game once a day, and use mongoose to convert the JSON into individual documents in a collection. Between days there is data which is consistent, for example each member's tag (an id for the member in game), but there is data which is different (different scores etc.). Each document has a createdAt property.
I would like to find the most recent document for each member, and thus have an array with each member's tag.
I an currently using the following query to find all documents where tags match, however they are returning all documents, not just one. How do I sort/limit the documents to the most recent one, whilst keep it as one query (or is there a more "mongodb way")?
memberTags = [1,2,3,4,5];
ClanMember.find({
'tag': {
$in: memberTags
}
}).lean().exec(function(err, members) {
res.json(members);
});
Thanks
You can query via the aggregation framework. Your query would involve a pipeline that has stages that process the input documents to give you the desired result. In your case, the pipeline would have a $match phase which acts as a query for the initial filter. $match uses standard MongoDB queries thus you can still query using $in.
The next step would be to sort those filtered documents by the createdAt field. This is done using the $sort operator.
The preceding pipeline stage involves aggregating the ordered documents to return the top document for each group. The $group operator together with the $first accumulator are the operators which make this possible.
Putting this altogether you can run the following aggregate operation to get your desired result:
memberTags = [1,2,3,4,5];
ClanMember.aggregate([
{ "$match": { "tag": { "$in": memberTags } } },
{ "$sort": { "tag": 1, "createdAt: -1 " } },
{
"$group": {
"_id": "$tag",
"createdAt": { "$first": "$createdAt" } /*,
include other necessary fields as appropriate
using the $first operator e.g.
"otherField1": { "$first": "$otherField1" },
"otherField2": { "$first": "$otherField2" },
...
*/
}
}
]).exec(function(err, members) {
res.json(members);
});
Or tweak your current query using find() so that you can sort on two fields, i.e. the tag (ascending) and createdAt (descending) attributes. You can then select the top 5 documents using limit, something like the following:
memberTags = [1,2,3,4,5];
ClanMember.find(
{ 'tag': { $in: memberTags } }, // query
{}, // projection
{ // options
sort: { 'createdAt': -1, 'tag': 1 },
limit: memberTags.length,
skip: 0
}
).lean().exec(function(err, members) {
res.json(members);
});
or
memberTags = [1,2,3,4,5];
ClanMember.find({
'tag': {
$in: memberTags
}
}).sort('-createdAt tag')
.limit(memberTags.length)
.lean()
.exec(function(err, members) {
res.json(members);
});
Ok, so, first, let's use findOne() so you get only one document out of the request
Then to sort by the newest document, you can use .sort({elementYouWantToSort: -1}) (-1 meaning you want to sort from newest to oldest, and 1 from the oldest to the newest)
I would recommend to use this function on the _id, which already includes creation date of the document
Which gives us the following request :
ClanMember.findOne({
'tag': {
$in: memberTags
}
}).sort({_id: -1}).lean().exec(function(err, members) {
res.json(members);
});
I have this Mongoose Query:
return Domain.find({domain:req.params.domain})
.where('date').equals(date)
.exec(function (err, domain) {
if (!err) {
if (!isEmpty(domain[0].visitors)) {
domain[0]['visitors'] = domain[0].visitors.slice(0,99);
}
}
I want to slice directly in the database and not after retrieving object. Mongoose cannot do this or it is not documented or I did not find documentation.
So I fall back to node.js native MongoDB Driver by using the collection keyword.
My code looks now as follow and fails:
return Domain.collection.find(
{ "domain":req.params.domain },
{ "date":date },
{ "visitors": { $slice:100 } },
function(err,domain){
if (!err) {
res.status(200).send({
domain:domain
});
}
Full code: https://gist.github.com/nottinhill/b3837d4c913b9e5dd879
I tried in MongoDB console to construct a query that will work, but cannot get this simple query to work. MongoDB documentation does not show how to query with two conditions. I want to:
Pseudo-Code of what I want:
find
giveBack wholeDomainObject
where domain == domain
where date == date
also slice visitorsArray
Projection is a single object definition. Also you "query" for things rather than ask for matches in projections other than specific fields matching criteria. $slice is a special case that does not exclude other fields in the projection by default:
Domain.collection.find(
{ "domain":req.params.domain, "date": date },
{ "visitors": { "$slice":100 } },
function(err,domain){
// process results here
}
);
Probably also to note that the $slice here ( just like JavaScript ) is a defined "number of entries" and not a n-1 reference as in an array index.