The data type of the field is String. I would like to fetch the data where character length of field name is greater than 40.
I tried these queries but returning error.
1.
db.usercollection.find(
{$where: "(this.name.length > 40)"}
).limit(2);
output :error: {
"$err" : "TypeError: Cannot read property 'length' of undefined near '40)' ",
"code" : 16722
}
this is working in 2.4.9 But my version is 2.6.5
For MongoDB 3.6 and newer:
The $expr operator allows the use of aggregation expressions within the query language, thus you can leverage the use of $strLenCP operator to check the length of the string as follows:
db.usercollection.find({
name: { $exists: true },
$expr: { $gt: [{ $strLenCP: '$name' }, 40] }
})
For MongoDB 3.4 and newer:
You can also use the aggregation framework with the $redact pipeline operator that allows you to proccess the logical condition with the $cond operator and uses the special operations $$KEEP to "keep" the document where the logical condition is true or $$PRUNE to "remove" the document where the condition was false.
This operation is similar to having a $project pipeline that selects the fields in the collection and creates a new field that holds the result from the logical condition query and then a subsequent $match, except that $redact uses a single pipeline stage which is more efficient.
As for the logical condition, there are String Aggregation Operators that you can use $strLenCP operator to check the length of the string. If the length is $gt a specified value, then this is a true match and the document is "kept". Otherwise it is "pruned" and discarded.
Consider running the following aggregate operation which demonstrates the above concept:
db.usercollection.aggregate([
{ $match: { name: { $exists: true } } },
{ $redact: {
$cond: [
{ $gt: [ { $strLenCP: "$name" }, 40] },
"$$KEEP",
"$$PRUNE"
]
} },
{ $limit: 2 }
])
If using $where, try your query without the enclosing brackets:
db.usercollection.find({ $where: "this.name.length > 40" }).limit(2);
A better query would be to to check for the field's existence and then check the length:
db.usercollection.find({ name: { $type: 2 }, $where: "this.name.length > 40" }).limit(2);
or:
db.usercollection.find({ name: { $exists: true }, $where: "this.name.length >
40" }).limit(2);
MongoDB evaluates non-$where query operations before $where expressions and non-$where query statements may use an index. A much better performance is to store the length of the string as another field and then you can index or search on it; applying $where will be much slower compared to that. It's recommended to use JavaScript expressions and the $where operator as a last resort when you can't structure the data in any other way, or when you are dealing with a
small subset of data.
A different and faster approach that avoids the use of the $where operator is the $regex operator. Consider the following pattern which searches for
db.usercollection.find({"name": {"$type": 2, "$regex": /^.{41,}$/}}).limit(2);
Note - From the docs:
If an index exists for the field, then MongoDB matches the regular
expression against the values in the index, which can be faster than a
collection scan. Further optimization can occur if the regular
expression is a “prefix expression”, which means that all potential
matches start with the same string. This allows MongoDB to construct a
“range” from that prefix and only match against those values from the
index that fall within that range.
A regular expression is a “prefix expression” if it starts with a
caret (^) or a left anchor (\A), followed by a string of simple
symbols. For example, the regex /^abc.*/ will be optimized by
matching only against the values from the index that start with abc.
Additionally, while /^a/, /^a.*/, and /^a.*$/ match equivalent
strings, they have different performance characteristics. All of these
expressions use an index if an appropriate index exists; however,
/^a.*/, and /^a.*$/ are slower. /^a/ can stop scanning after
matching the prefix.
Queries with $where and $expr are slow if there are too many documents.
Using $regex is much faster than $where, $expr.
db.usercollection.find({
"name": /^[\s\S]{40,}$/, // name.length >= 40
})
or
db.usercollection.find({
"name": { "$regex": "^[\s\S]{40,}$" }, // name.length >= 40
})
This query is the same meaning with
db.usercollection.find({
"$where": "this.name && this.name.length >= 40",
})
or
db.usercollection.find({
"name": { "$exists": true },
"$expr": { "$gte": [ { "$strLenCP": "$name" }, 40 ] }
})
I tested each queries for my collection.
# find
$where: 10529.359ms
$expr: 5305.801ms
$regex: 2516.124ms
# count
$where: 10872.006ms
$expr: 2630.155ms
$regex: 158.066ms
Here is one of the way in mongodb you can achieve this.
db.usercollection.find({ $where: 'this.name.length < 4' })
This query will give both field value and length:
db.usercollection.aggregate([
{
$project: {
"name": 1,
"length": { $strLenCP: "$name" }
}} ])
I had a similar kind of scenario, but in my case string is not a 1st level attribute. It is inside an object. In here I couldn't find a suitable answer for it. So I thought to share my solution with you all(Hope this will help anyone with a similar kind of problem).
Parent Collection
{
"Child":
{
"name":"Random Name",
"Age:"09"
}
}
Ex: If we need to get only collections that having child's name's length is higher than 10 characters.
db.getCollection('Parent').find({$where: function() {
for (var field in this.Child.name) {
if (this.Child.name.length > 10)
return true;
}
}})
Find anything with a name with 40 or more characters:
db.usercollection.find({name: /.{40}/})
(simplified the RegEx from Fumiya Karasawa's answer)
Related
I am trying to add dynamic conditions using mongoose library
var and_cond = { $and: [] };
var or_cond = { $or: [] };
and_cond.$and.push ({ "doc_no" : /first/i })
or_cond.$or.push ({ "doc_type" : /third/i })
TRModel.find(
and_cond,
or_cond
)
What I expect something which can give me both AND and OR condition work in a query,
The above ended in an abrupt query which is wrong obviously.
title_records.find({ '$and': [ { doc_no: /first/i } ] }, { fields: {
'$or': [ { doc_type: /third/i } ] } })
There are two problems:
You're trying to use two top-level logical operators. To my knowledge, you need to have a single top-level operator from the set of $and, $or, and $nor. You can nest as many as you want within, however.
You're passing in two separate object when what you really want to do is merge the two.
It looks like you're trying to perform your query such that you find all documents that match both the $and condition AND the $or condition. In that case, try the following:
TRModel.find({
$and: [
and_cond,
or_cond
]
});
If, however, you want to use one OR the other, you can simply change the top-level $and to $or:
TRModel.find({
$or: [
and_cond,
or_cond
]
});
You will need to first make sure you are matching the mongoose find parameters which are defined specifically as:
MyModel.find({ <conditions> }, { <projections> }, { <options> }, <callback function>);
e.g.
MyModel.find({ name: /john/i }, null, { skip: 10 }, function (err, docs) {});
So you have to make sure your end result is one object going in into the conditions part of the find.
Next thing is you have to make sure you have one operator from these types:
$and, $or, $not and $not
As your top level one and then you can nest others inside.
Like you could have top level $and with multiple $or inside.
I know that MongoDB supports the syntax find{array.0.field:"value"}, but I specifically want to do this for the last element in the array, which means I don't know the index. Is there some kind of operator for this, or am I out of luck?
EDIT: To clarify, I want find() to only return documents where a field in the last element of an array matches a specific value.
In 3.2 this is possible. First project so that myField contains only the last element, and then match on myField.
db.collection.aggregate([
{ $project: { id: 1, myField: { $slice: [ "$myField", -1 ] } } },
{ $match: { myField: "myValue" } }
]);
You can use $expr ( 3.6 mongo version operator ) to use aggregation functions in regular query.
Compare query operators vs aggregation comparison operators.
For scalar arrays
db.col.find({$expr: {$gt: [{$arrayElemAt: ["$array", -1]}, value]}})
For embedded arrays - Use $arrayElemAt expression with dot notation to project last element.
db.col.find({$expr: {$gt: [{"$arrayElemAt": ["$array.field", -1]}, value]}})
Spring #Query code
#Query("{$expr:{$gt:[{$arrayElemAt:[\"$array\", -1]}, ?0]}}")
ReturnType MethodName(ArgType arg);
Starting Mongo 4.4, the aggregation operator $last can be used to access the last element of an array:
For instance, within a find query:
// { "myArray": ["A", "B", "C"] }
// { "myArray": ["D"] }
db.collection.find({ $expr: { $eq: [{ $last: "$myArray" }, "C"] } })
// { "myArray": ["A", "B", "C"] }
Or within an aggregation query:
db.collection.aggregate([
{ $addFields: { last: { $last: "$myArray" } } },
{ $match: { last: "C" } }
])
use $slice.
db.collection.find( {}, { array_field: { $slice: -1 } } )
Editing:
You can make use of
{ <field>: { $elemMatch: { <query1>, <query2>, ... } } } to find a match.
But it won't give exactly what you are looking for. I don't think that is possible in mongoDB yet.
I posted on the official Mongo Google group here, and got an answer from their staff. It appears that what I'm looking for isn't possible. I'm going to just use a different schema approach.
Version 3.6 use aggregation to achieve the same.
db.getCollection('deviceTrackerHistory').aggregate([
{
$match:{clientId:"12"}
},
{
$project:
{
deviceId:1,
recent: { $arrayElemAt: [ "$history", -1 ] }
}
}
])
You could use $position: 0 whenever you $push, and then always query array.0 to get the most recently added element. Of course then, you wont be able to get the new "last" element.
Not sure about performance, but this works well for me:
db.getCollection('test').find(
{
$where: "this.someArray[this.someArray.length - 1] === 'pattern'"
}
)
You can solve this using aggregation.
model.aggregate([
{
$addFields: {
lastArrayElement: {
$slice: ["$array", -1],
},
},
},
{
$match: {
"lastArrayElement.field": value,
},
},
]);
Quick explanations. aggregate creates a pipeline of actions, executed sequentially, which is why it takes an array as parameter. First we use the $addFields pipeline stage. This is new in version 3.4, and basically means: Keep all the existing fields of the document, but also add the following. In our case we're adding lastArrayElement and defining it as the last element in the array called array. Next we perform a $match pipeline stage. The input to this is the output from the previous stage, which includes our new lastArrayElement field. Here we're saying that we only include documents where its field field has the value value.
Note that the resulting matching documents will include lastArrayElement. If for some reason you really don't want this, you could add a $project pipeline stage after $match to remove it.
For the answer use $arrayElemAt,if i want orderNumber:"12345" and the last element's value $gt than "value"? how to make the $expr? thanks！
For embedded arrays - Use $arrayElemAt expression with dot notation to project last element.
db.col.find({$expr: {$gt: [{"$arrayElemAt": ["$array.field", -1]}, value]}})
db.collection.aggregate([
{
$match: {
$and: [
{ $expr: { $eq: [{ "$arrayElemAt": ["$fieldArray.name", -1] }, "value"] } },
{ $or: [] }
]
}
}
]);
I've stumbled upon some very strange behavior with MongoDB. For my test case, I have an MongoDB collection with 9 documents. All documents have the exact same structure, including the fields expired_at: Date and location: [lng, lat].
I now need to find all documents that are not expired yet and are within a bounding box; I show match documents on map. for this I set up the following queries:
var qExpiry = {"expired_at": { $gt : new Date() } };
var qLocation = { "location" : { $geoWithin : { $box : [ [ 123.8766, 8.3269 ] , [ 122.8122, 8.24974 ] ] } } };
var qFull = { $and: [ qExpiry, qLocation ] };
Since the expiry date is long in the past, and when I set the bounding box large enough, the following queries give me all 9 documents as expected:
db.docs.find(qExpiry);
db.docs.find(qLocation);
db.docs.find(qFull);
db.docs.find(qExpiry).sort({"created_at" : -1});
db.docs.find(qLocation).sort({"created_at" : -1});
Now here's the deal: The following query returns 0 documents:
db.docs.find(qFull).sort({"created_at" : -1});
Just adding sort to the AND query ruins the result (please note that I want to sort since I also have a limit in order to avoid cluttering the map on larger scales). Sorting by other fields yield the same empty result. What's going on here?
(Actually even stranger: When I zoom into my map, I sometimes get results for qFull, even with sorting. One could argue that qLocation is faulty. But when I only use qLocation, the results are always correct. And qExpiry is always true for all documents anyway)
You may want to try running the same query using the aggregation framework's $match and $sort pipelines:
db.docs.aggregate([
{ "$match": qFull },
{ "$sort": { "created_at": -1 } }
]);
or implicitly using $and by specifiying a comma-separated list of expressions as in
db.docs.aggregate([
{
"$match": {
"expired_at": { "$gt" : new Date() },
"location" : {
"$geoWithin" : {
"$box" : [
[ 123.8766, 8.3269 ],
[ 122.8122, 8.24974 ]
]
}
}
}
},
{ "$sort": { "created_at": -1 } }
]);
Not really sure why that fails with find()
chridam suggestion using the aggregation framework of MongoDB proved to be the way to go. My working query now looks like this:
db.docs.aggregate(
[
{ $match : { $and : [qExpiry, qLocation]} },
{ $sort: {"created_at": -1} }.
{ $limit: 50 }.
]
);
Nevertheless, if any can point out way my first approach did not work, that would be very useful. Simply adding sort() to a non-empty query shouldn't suddenly return 0 documents. Just to add, since I still tried for a bit, .sort({}) return all documents but was not very useful. Everything else failed including .sort({'_id': 1}).
This question already has answers here:
MongoDB : aggregation framework : $match between fields
(2 answers)
Closed 6 years ago.
So I have this query,
db.collection.find({$where: "this.field1 !== this.field2"})
But now I need to create a similar query and aggregate the results in a complex query which tried and true, only can be done by using the aggregation pipeline or go "cannon for a fly" and use the mapReduce option.
Since I want to avoid mapReduce, is there a way to achieve something similar to the {$where: "this.field1 !== this.field2"} approach?
Some observations, an answer to the general approach to solve the situation above is most desirable, but in case that is not possible, in order to solve my problem, I can also use the raw value in the query with the following restrictions. I need to find what follows
db.collection.find({ $or :[
{field1: value},
{field2: value}
], $and: [
{ field1: {$not: {$eq: value}}},
{ field2: {$not: {$eq: value}} }
]})
I tried the above query but it discards results that contains the value in either field, and what I want is a way to discard the objects that have the same value in both fields at the same time, but obtain the documents that have the value in either field, I usually prefer an one query approach but after reading a lot, I think the only alternative to avoid mapReduce, is to do something like
db.collection.find({$where: "this.field1 === this.field2"}, function (err, results){
var match = { $or :[
{field1: value},
{field2: value}
], {
_id : { $not : {$in: results.map((x) => x._id)}}
}
db.collection([ {$match: match}, {$project: projectionObject}, ...., etc])
})
So I'm going to use the be solution above, which can be optimized a lot, (I just wrote it while writing the question), but I would like to avoid sending an incredibly big list of objectIds back to the database if at all possible. I will refactor it a bit to use matching operators before the where statement
that's complex case, and maybe you could use a trick with $cond inside $project
var projectDataForMatch = {
$project : {
_id : 1, //list all fields needed here
filterThisDoc : {
$cond : {
if : {
$eq : ["$field1", "$filed2"]
},
then : 1,
else : 0
} //or use compare operator $cmp
}
}
}
var match = {
$match : {
filterThisDoc : 1
}
}
db.col.aggregate([projectDataForMatch, match])
you can extend $cond to fit your needs as desired.
In products collection, i have an Array of recentviews which has 2 fields viewedBy & viewedDate.
In a scenario if i already have a record with viewedby, then i need to update it. For e.g if i have array like this :-
"recentviews" : [
{
"viewedby" : "abc",
"vieweddate" : ISODate("2014-05-08T04:12:47.907Z")
}
]
And user is abc, so i need to update the above & if there is no record for abc i have to $push.
I have tried $set as follows :-
db.products.update( { _id: ObjectId("536c55bf9c8fb24c21000095") },
{ $set:
{ "recentviews":
{
viewedby: 'abc',
vieweddate: ISODate("2014-05-09T04:12:47.907Z")
}
}
}
)
The above query erases all my other elements in Array.
Actually doing what it seems like you say you are doing is not a singular operation, but I'll walk through the parts required in order to do this or otherwise cover other possible situations.
What you are looking for is in part the positional $ operator. You need part of your query to also "find" the element of the array you want.
db.products.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
"$set": {
"recentviews.$.vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
)
So the $ stands for the matched position in the array so the update portion knows which item in the array to update. You can access individual fields of the document in the array or just specify the whole document to update at that position.
db.products.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
"$set": {
"recentviews.$": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
)
If the fields do not in fact change and you just want to insert a new array element if the exact same one does not exist, then you can use $addToSet
db.products.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
$addToSet:{
"recentviews": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
)
However if you are just looking for for "pushing" to an array by a singular key value if that does not exist then you need to do some more manual handling, by first seeing if the element in the array exists and then making the $push statement where it does not.
You get some help from the mongoose methods in doing this by tracking the number of documents affected by the update:
Product.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
"$set": {
"recentviews.$": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
},
function(err,numAffected) {
if (numAffected == 0) {
// Document not updated so you can push onto the array
Product.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095")
},
{
"$push": {
"recentviews": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
},
function(err,numAffected) {
}
);
}
}
);
The only word of caution here is that there is a bit of an implementation change in the writeConcern messages from MongoDB 2.6 to earlier versions. Being unsure right now as to how the mongoose API actually implements the return of the numAffected argument in the callback the difference could mean something.
In prior versions, even if the data you sent in the initial update exactly matched an existing element and there was no real change required then the "modified" amount would be returned as 1 even though nothing was actually updated.
From MongoDB 2.6 the write concern response contains two parts. One part shows the modified document and the other shows the match. So while the match would be returned by the query portion matching an existing element, the actual modified document count would return as 0 if in fact there was no change required.
So depending on how the return number is actually implemented in mongoose, it might actually be safer to use the $addToSet operator on that inner update to make sure that if the reason for the zero affected documents was not just that the exact element already existed.