This question already has answers here:
MongoDB : aggregation framework : $match between fields
(2 answers)
Closed 6 years ago.
So I have this query,
db.collection.find({$where: "this.field1 !== this.field2"})
But now I need to create a similar query and aggregate the results in a complex query which tried and true, only can be done by using the aggregation pipeline or go "cannon for a fly" and use the mapReduce option.
Since I want to avoid mapReduce, is there a way to achieve something similar to the {$where: "this.field1 !== this.field2"} approach?
Some observations, an answer to the general approach to solve the situation above is most desirable, but in case that is not possible, in order to solve my problem, I can also use the raw value in the query with the following restrictions. I need to find what follows
db.collection.find({ $or :[
{field1: value},
{field2: value}
], $and: [
{ field1: {$not: {$eq: value}}},
{ field2: {$not: {$eq: value}} }
]})
I tried the above query but it discards results that contains the value in either field, and what I want is a way to discard the objects that have the same value in both fields at the same time, but obtain the documents that have the value in either field, I usually prefer an one query approach but after reading a lot, I think the only alternative to avoid mapReduce, is to do something like
db.collection.find({$where: "this.field1 === this.field2"}, function (err, results){
var match = { $or :[
{field1: value},
{field2: value}
], {
_id : { $not : {$in: results.map((x) => x._id)}}
}
db.collection([ {$match: match}, {$project: projectionObject}, ...., etc])
})
So I'm going to use the be solution above, which can be optimized a lot, (I just wrote it while writing the question), but I would like to avoid sending an incredibly big list of objectIds back to the database if at all possible. I will refactor it a bit to use matching operators before the where statement
that's complex case, and maybe you could use a trick with $cond inside $project
var projectDataForMatch = {
$project : {
_id : 1, //list all fields needed here
filterThisDoc : {
$cond : {
if : {
$eq : ["$field1", "$filed2"]
},
then : 1,
else : 0
} //or use compare operator $cmp
}
}
}
var match = {
$match : {
filterThisDoc : 1
}
}
db.col.aggregate([projectDataForMatch, match])
you can extend $cond to fit your needs as desired.
Related
The data type of the field is String. I would like to fetch the data where character length of field name is greater than 40.
I tried these queries but returning error.
1.
db.usercollection.find(
{$where: "(this.name.length > 40)"}
).limit(2);
output :error: {
"$err" : "TypeError: Cannot read property 'length' of undefined near '40)' ",
"code" : 16722
}
this is working in 2.4.9 But my version is 2.6.5
For MongoDB 3.6 and newer:
The $expr operator allows the use of aggregation expressions within the query language, thus you can leverage the use of $strLenCP operator to check the length of the string as follows:
db.usercollection.find({
name: { $exists: true },
$expr: { $gt: [{ $strLenCP: '$name' }, 40] }
})
For MongoDB 3.4 and newer:
You can also use the aggregation framework with the $redact pipeline operator that allows you to proccess the logical condition with the $cond operator and uses the special operations $$KEEP to "keep" the document where the logical condition is true or $$PRUNE to "remove" the document where the condition was false.
This operation is similar to having a $project pipeline that selects the fields in the collection and creates a new field that holds the result from the logical condition query and then a subsequent $match, except that $redact uses a single pipeline stage which is more efficient.
As for the logical condition, there are String Aggregation Operators that you can use $strLenCP operator to check the length of the string. If the length is $gt a specified value, then this is a true match and the document is "kept". Otherwise it is "pruned" and discarded.
Consider running the following aggregate operation which demonstrates the above concept:
db.usercollection.aggregate([
{ $match: { name: { $exists: true } } },
{ $redact: {
$cond: [
{ $gt: [ { $strLenCP: "$name" }, 40] },
"$$KEEP",
"$$PRUNE"
]
} },
{ $limit: 2 }
])
If using $where, try your query without the enclosing brackets:
db.usercollection.find({ $where: "this.name.length > 40" }).limit(2);
A better query would be to to check for the field's existence and then check the length:
db.usercollection.find({ name: { $type: 2 }, $where: "this.name.length > 40" }).limit(2);
or:
db.usercollection.find({ name: { $exists: true }, $where: "this.name.length >
40" }).limit(2);
MongoDB evaluates non-$where query operations before $where expressions and non-$where query statements may use an index. A much better performance is to store the length of the string as another field and then you can index or search on it; applying $where will be much slower compared to that. It's recommended to use JavaScript expressions and the $where operator as a last resort when you can't structure the data in any other way, or when you are dealing with a
small subset of data.
A different and faster approach that avoids the use of the $where operator is the $regex operator. Consider the following pattern which searches for
db.usercollection.find({"name": {"$type": 2, "$regex": /^.{41,}$/}}).limit(2);
Note - From the docs:
If an index exists for the field, then MongoDB matches the regular
expression against the values in the index, which can be faster than a
collection scan. Further optimization can occur if the regular
expression is a “prefix expression”, which means that all potential
matches start with the same string. This allows MongoDB to construct a
“range” from that prefix and only match against those values from the
index that fall within that range.
A regular expression is a “prefix expression” if it starts with a
caret (^) or a left anchor (\A), followed by a string of simple
symbols. For example, the regex /^abc.*/ will be optimized by
matching only against the values from the index that start with abc.
Additionally, while /^a/, /^a.*/, and /^a.*$/ match equivalent
strings, they have different performance characteristics. All of these
expressions use an index if an appropriate index exists; however,
/^a.*/, and /^a.*$/ are slower. /^a/ can stop scanning after
matching the prefix.
Queries with $where and $expr are slow if there are too many documents.
Using $regex is much faster than $where, $expr.
db.usercollection.find({
"name": /^[\s\S]{40,}$/, // name.length >= 40
})
or
db.usercollection.find({
"name": { "$regex": "^[\s\S]{40,}$" }, // name.length >= 40
})
This query is the same meaning with
db.usercollection.find({
"$where": "this.name && this.name.length >= 40",
})
or
db.usercollection.find({
"name": { "$exists": true },
"$expr": { "$gte": [ { "$strLenCP": "$name" }, 40 ] }
})
I tested each queries for my collection.
# find
$where: 10529.359ms
$expr: 5305.801ms
$regex: 2516.124ms
# count
$where: 10872.006ms
$expr: 2630.155ms
$regex: 158.066ms
Here is one of the way in mongodb you can achieve this.
db.usercollection.find({ $where: 'this.name.length < 4' })
This query will give both field value and length:
db.usercollection.aggregate([
{
$project: {
"name": 1,
"length": { $strLenCP: "$name" }
}} ])
I had a similar kind of scenario, but in my case string is not a 1st level attribute. It is inside an object. In here I couldn't find a suitable answer for it. So I thought to share my solution with you all(Hope this will help anyone with a similar kind of problem).
Parent Collection
{
"Child":
{
"name":"Random Name",
"Age:"09"
}
}
Ex: If we need to get only collections that having child's name's length is higher than 10 characters.
db.getCollection('Parent').find({$where: function() {
for (var field in this.Child.name) {
if (this.Child.name.length > 10)
return true;
}
}})
Find anything with a name with 40 or more characters:
db.usercollection.find({name: /.{40}/})
(simplified the RegEx from Fumiya Karasawa's answer)
I am trying to add dynamic conditions using mongoose library
var and_cond = { $and: [] };
var or_cond = { $or: [] };
and_cond.$and.push ({ "doc_no" : /first/i })
or_cond.$or.push ({ "doc_type" : /third/i })
TRModel.find(
and_cond,
or_cond
)
What I expect something which can give me both AND and OR condition work in a query,
The above ended in an abrupt query which is wrong obviously.
title_records.find({ '$and': [ { doc_no: /first/i } ] }, { fields: {
'$or': [ { doc_type: /third/i } ] } })
There are two problems:
You're trying to use two top-level logical operators. To my knowledge, you need to have a single top-level operator from the set of $and, $or, and $nor. You can nest as many as you want within, however.
You're passing in two separate object when what you really want to do is merge the two.
It looks like you're trying to perform your query such that you find all documents that match both the $and condition AND the $or condition. In that case, try the following:
TRModel.find({
$and: [
and_cond,
or_cond
]
});
If, however, you want to use one OR the other, you can simply change the top-level $and to $or:
TRModel.find({
$or: [
and_cond,
or_cond
]
});
You will need to first make sure you are matching the mongoose find parameters which are defined specifically as:
MyModel.find({ <conditions> }, { <projections> }, { <options> }, <callback function>);
e.g.
MyModel.find({ name: /john/i }, null, { skip: 10 }, function (err, docs) {});
So you have to make sure your end result is one object going in into the conditions part of the find.
Next thing is you have to make sure you have one operator from these types:
$and, $or, $not and $not
As your top level one and then you can nest others inside.
Like you could have top level $and with multiple $or inside.
Sorry if this is pretty basic, but I'm a mongodb newbie and haven't been able to find an answer to this:
Let's say I'm doing the following:
db.collection("bugs").updateOne({ _id: searchId }, { $set: { "fixed": true }}
How to set "fixed" to the contrary of whatever the last value of "fixed" was? Without any additional queries? Something like { $set: { "fixed": !fixed }}
It is not really possible to achieve this in MongoDB as of now in just one operation by sticking to the idea of storing boolean values (might be possible in future versions of MongoDB). But, there is a workaround to do this by storing bits (0 or 1) to represent true or false instead of boolean values and performing bitwise xor operation on those in MongoDB as follows:
db.collection("bugs").updateOne(
{
_id: searchId
},
{
$bit : {
fixed: {
xor: NumberInt(1)
}
}
}
)
Please note that you also have to store 0 as NumberInt(0) to represent false and 1 as NumberInt(1) to represent true in the fixed prop as MongoDB by default treats all numbers as floating-point values.
This is not possible in MongoDB. You have to retrieve the doc from the db and then update it:
var doc = db.collection("bugs").findOne({ _id: searchId });
db.collection("bugs").updateOne({ _id: searchId }, { $set: { "fixed": !doc.fixed } }
Yes, it's possible to do that with MongoDB, of course !! ... just use the find's forEach feature like this:
db.collection("bugs").find({ _id: searchId }).forEach(function(bugDoc) {
db.collection("bugs").updateOne({ _id: searchId }, { $set: { "fixed": !bugDoc.fixed }});
});
NOTE: bugDoc contains all the fields of the original document and you can make all the calculations and changes you want in this double operation update
I've stumbled upon some very strange behavior with MongoDB. For my test case, I have an MongoDB collection with 9 documents. All documents have the exact same structure, including the fields expired_at: Date and location: [lng, lat].
I now need to find all documents that are not expired yet and are within a bounding box; I show match documents on map. for this I set up the following queries:
var qExpiry = {"expired_at": { $gt : new Date() } };
var qLocation = { "location" : { $geoWithin : { $box : [ [ 123.8766, 8.3269 ] , [ 122.8122, 8.24974 ] ] } } };
var qFull = { $and: [ qExpiry, qLocation ] };
Since the expiry date is long in the past, and when I set the bounding box large enough, the following queries give me all 9 documents as expected:
db.docs.find(qExpiry);
db.docs.find(qLocation);
db.docs.find(qFull);
db.docs.find(qExpiry).sort({"created_at" : -1});
db.docs.find(qLocation).sort({"created_at" : -1});
Now here's the deal: The following query returns 0 documents:
db.docs.find(qFull).sort({"created_at" : -1});
Just adding sort to the AND query ruins the result (please note that I want to sort since I also have a limit in order to avoid cluttering the map on larger scales). Sorting by other fields yield the same empty result. What's going on here?
(Actually even stranger: When I zoom into my map, I sometimes get results for qFull, even with sorting. One could argue that qLocation is faulty. But when I only use qLocation, the results are always correct. And qExpiry is always true for all documents anyway)
You may want to try running the same query using the aggregation framework's $match and $sort pipelines:
db.docs.aggregate([
{ "$match": qFull },
{ "$sort": { "created_at": -1 } }
]);
or implicitly using $and by specifiying a comma-separated list of expressions as in
db.docs.aggregate([
{
"$match": {
"expired_at": { "$gt" : new Date() },
"location" : {
"$geoWithin" : {
"$box" : [
[ 123.8766, 8.3269 ],
[ 122.8122, 8.24974 ]
]
}
}
}
},
{ "$sort": { "created_at": -1 } }
]);
Not really sure why that fails with find()
chridam suggestion using the aggregation framework of MongoDB proved to be the way to go. My working query now looks like this:
db.docs.aggregate(
[
{ $match : { $and : [qExpiry, qLocation]} },
{ $sort: {"created_at": -1} }.
{ $limit: 50 }.
]
);
Nevertheless, if any can point out way my first approach did not work, that would be very useful. Simply adding sort() to a non-empty query shouldn't suddenly return 0 documents. Just to add, since I still tried for a bit, .sort({}) return all documents but was not very useful. Everything else failed including .sort({'_id': 1}).
So if I had something like this:
{
_id:'123',
parts: ['One','Two','Three']
}
So if I wanted parts[0], is it possible to do the find for just that?
So something like:
db.stories.find({_id:'123', part:{$eq:0}})
I know the above is wrong, but I'm wondering if it's more efficient to do the find properly if possibly (criteria related to object/array key), or simply do the broad find and go from there (i.e. use the criteria without object/document.
You can use $project and $arrayElemAt to get the first element in the array
db.stories.aggregate([
{
$match: {_id: '123'}
},{
$project: {part: {$arrayElemAt: ['$parts',0]}}
}
])
If you want to get 0 index element from the parts array you can write in this way
db.sample.find({ _id : "123" } , { parts : { $slice : [0 , 1] } } )