Develop Reference

Count keys within array elements

Hi I want to return all the collections that have arrays count lesser than 2 inside features array in mongodb. I tried using $size but it is not possible.
I don't want to get the result and loop each of the features and count it. I want to return the productId 123 because it has a count of 1 in one of features array. Please take below document as an example:
{
"productId" : 123.0,
"features" : [
{
"a" : true
},
{
"a" : true,
"b" : true
}
]
},
{
"productId" : 456.0,
"features" : [
{
"a" : true,
"b" : true
},
{
"a" : true,
"b" : true
}
]
}

What you are actually asking for is matching on the "count of the number of keys" within the array elements. You have different approaches to this depending on the available MongoDB version.
MongoDB 3.4.4 and upwards
You can use $objectToArray to coerce each element into an "array" itself, representing the "key/value" pairs of the elements:
db.collection.aggregate([
{ "$redact": {
"$cond": {
"if": {
"$anyElementTrue": {
"$map": {
"input": "$features",
"as": "f",
"in": {
"$lt": [
{ "$size": { "$objectToArray": "$$f" } },
2
]
}
}
}
},
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
You basically feed the condition with $redact which determines that for the results of $map where the $objectToArray is applied to each element and then tested for the $size, where any of the tested array elements returned true via $anyElementTrue.
All Other Versions
Anywhere else, looking somewhat more brief but actually not as performance effective is using $where to apply a JavaScript expression to test the array elements. Same principle though using Object.keys() and Array.some():
db.collection.find({
"$where": function() {
return this.features.some(f => Object.keys(f).length < 2 )
}
})
Same deal but since the JavaScript requires interpretation and evaluation against every document, it actually runs quite a bit slower that the aggregation expression given.
Both return the same document, which is the one which has an element with "less than two keys" in the inner object, just as asked:
/* 1 */
{
"productId" : 123.0,
"features" : [
{
"a" : true
},
{
"a" : true,
"b" : true
}
]
}

Robmongo - aggregate values distinct by other value\cloumn

I'm new to robmongo and I received an assignment to write some queries.
let say I have a collection that each key has some values for example value of "userId" and value of "deviceModel".
I need to write a query that shows for each device model how many users has this device.
this is what I got so far:
db.device_data.aggregate([ {"$group" : {_id:"$data.deviceModel", count:{$sum:1}}}])
The problem is that this aggregate for each device the number of keys it appears.
{
"_id" : { "$binary" : "AN6GmE7Thi+Sd/dpLRjIilgsV/4AAAg=", "$type" : "00" },
"auditVersion" : "1.0",
"currentTime" : NumberLong(1479301118381),
"data" : {
"deviceDesign" : "bullhead",
"loginType" : "GOOGLE",
"source" : "SDKLoader",
"systemUptimeMillis" : 137652880.0,
"simCountryIso" : "il",
"networkOperatorName" : "Cellcom",
"hasPhonePermission" : true,
"deviceIdentifier" : "353627074839559",
"sdkVersion" : "0.7.939.2016-11-14.masterDev",
"brand" : "google",
"osVersion" : "7.0",
"osVersionIncremental" : "3239497",
"deviceModel" : "Nexus 5X",
"deviceSDKVersion" : 24.0,
"manufacturer" : "LGE",
"sdkShortBuildDate" : "2016-11-14",
"sdkFullBuildDate" : "Mon Nov 14 22:16:40 IST 2016",
"product" : "bullhead"
},
"timezone" : "Asia/Jerusalem",
"collectionAlias" : "DEVICE_DATA",
"shortDate" : 17121,
"userId" : "00DE86984ED3862F9277F7692D18C88A#1927cc81cfcf7a467e9d4f4ac7a1534b"}
this is an example of how one key locks like.

The below query should give you distinct count of userId for a deviceModel. I meant if a same userId present for a deviceModel multiple items, it will be counted only once.
db.collection.aggregate([ {"$group" : {_id:"$data.deviceModel", userIds:{$addToSet: "$userId"}}
},
{
$unwind:"$userIds"
},
{
$group: { _id: "$_id", userIdCount: { $sum:1} }
}])
Unwind:-
Deconstructs an array field from the input documents to output a
document for each element.
In the above solution, it deconstructs the userId array formed on the first pipeline.
addToSet:-
Returns an array of all unique values that results from applying an
expression to each document in a group of documents that share the
same group by key.
This function ensures that only unique values are added to an array. In the above case, the userId is added to an array in the first pipeline.

Matching field by omitting spaces - MongoDB

I'm trying to find the mongo document by matching the "Tel" field value,
{
"_id" : ObjectId("54f047aa5b9e5c7c13000000"),
"data" : [
{
"Id" : "1",
"Country" : "India",
"Timezone" : "Europe/Paris",
**"Tel" : "03 20 14 97 70",**
"Prenom" : "ddd",
"Email" : "ddd#gmail.com",
"City" : "Chennai",
"date" : "",
"active" : "true"
}
]
}
how to fetch the above document from mongo collection using the below find method without space in "Tel" field,
>db.test.find({"data.Tel":"0320149770"})
Please can anyone help me !!!

If this is what you really want to do on a regular basis then you are best off adding another field to the document that has the string present without any spaces.
The reason why is though there are functions you can perform to do the search, none of the methods are able to use an index to match the document, so this means scanning everything in the collection in order to find a match.
You can do this with JavaScript evaluation in a $where clause:
db.test.find(function() {
return this.data.some(function(el) {
el.Tel.replace(/ /g,"") == "0320149770"
});
});
But don't do that because it's really bad. you are better off just updating all the data instead:
db.test.find().forEach(function(doc) {
doc.data = doc.data.map(function(el) {
el.TelNum = el.Tel.replace(/ /g,"");
})
db.test.update({ "_id": doc._id },{ "$set": { "data": doc.data } });
})
Or something along those lines to have a field without spaces all ready to search on directly.

String manipulation and aggregation

I have a log collection in MongoDB that has a structure that looks like this:
{
url : "http://example.com",
query : "name=blah,;another_param=bleh",
count : 5
}
where the "query" field is the query parameters in the requested url.
I want to compute a total of count grouped by the query parameter "name". For example, for this collection:
[{
url : "http://example.com",
query : "name=blah,;another_param=bleh",
count : 3
},
{
url : "http://example.com",
query : "name=blah,;another_param=xyz",
count : 4
},
{
url : "http://example.com",
query : "name=another_name,;another_param=bleh",
count : 3
}]
I need this output:
[{
key : "blah",
count : 7
},
{
key : "another_name",
count : 3
}]
It doesnt look like I can do this string manipulation using the aggregation framework. I can do this via map-reduce, but can a map-reduce operation be part of the aggregation pipeline?

The aggregation framework does not have the string manipulation operators necessary to dissect the string content and break this up into the key/value pairs you need for this operation. The only string manipulation currently available is $substr, which is not going to help unless you are dealing with fixed length data.
So the only server side way to do this at present is with mapReduce since you can just the JavaScript functions available to do the right manipulation. Something like this:
For the mapper:
function() {
var obj = {};
this.query.split(/,;/).forEach(function(item) {
var temp = item.split(/=/);
obj[temp[0]] = temp[1];
});
if (obj.hasOwnProperty('name')
emit(obj.name,this.count);
}
And the reducer:
function(key,values) {
return Array.sum( values );
}
Which is the basic structure of the JavaScript functions required to split out the "name" parameters and use them as the "keys" for aggregation, or general counting of the "key" occurrences.
So the aggregation framework cannot execute any JavaScript itself, as it just runs native code operators over the data.
It would be a good idea though to look at changing how your data is stored, so that the elements are broken down into a an "object" representation rather than a string when the documents are inserted to MongoDB. This allows native query forms that don't rely on JavaScript execution to manipulate the data:
[{
"url": "http://example.com",
"query": {
"name": "blah",
"another_param": "bleh"
},
"count": 3
},
{
"url": "http://example.com",
"query": {
"name": "blah",
"another_param": "xyz"
},
"count": 4
},
{
"url": "http://example.com",
"query": {
"name": "another_name",
"another_param": "bleh"
},
"count": 3
}]
This makes a $group pipeline stage quite simple as the data is now organized in a form that can be natively processed:
{ "$match": { "query.name": { "$exists": true } },
{ "$group": {
"_id": "$query.name",
"count": { "$sum": "$count" }
}}
So use mapReduce for now, but ultimately consider changing your recording of the data to split the "tokens" from the query string and represent this as structured data, optionally keeping the original string in another field.
The aggregation framework will process this much faster than mapReduce can, so this would be the better ongoing option.

Query nested document with mongoose

I know this question has been asked a lot of times but I'm kinda new to mongo and mongoose as well and I couldn't figure it out !
My problem:
I have a which looks like this:
var rankingSchema = new Schema({
userId : { type : Schema.Types.ObjectId, ref:'User' },
pontos : {type: Number, default:0},
placarExato : {type: Number, default:0},
golVencedor : {type: Number, default:0},
golPerdedor : {type: Number, default:0},
diferencaVencPerd : {type: Number, default:0},
empateNaoExato : {type: Number, default:0},
timeVencedor : {type: Number, default:0},
resumo : [{
partida : { type : Schema.Types.ObjectId, ref:'Partida' },
palpite : [Number],
quesito : String
}]
});
Which would return a document like this:
{
"_id" : ObjectId("539d0756f0ccd69ac5dd61fa"),
"diferencaVencPerd" : 0,
"empateNaoExato" : 0,
"golPerdedor" : 0,
"golVencedor" : 1,
"placarExato" : 2,
"pontos" : 78,
"resumo" : [
{
"partida" : ObjectId("5387d991d69197902ae27586"),
"_id" : ObjectId("539d07eb06b1e60000c19c18"),
"palpite" : [
2,
0
]
},
{
"partida" : ObjectId("5387da7b27f54fb425502918"),
"quesito" : "golsVencedor",
"_id" : ObjectId("539d07eb06b1e60000c19c1a"),
"palpite" : [
3,
0
]
},
{
"partida" : ObjectId("5387dc012752ff402a0a7882"),
"quesito" : "timeVencedor",
"_id" : ObjectId("539d07eb06b1e60000c19c1c"),
"palpite" : [
2,
1
]
},
{
"partida" : ObjectId("5387dc112752ff402a0a7883"),
"_id" : ObjectId("539d07eb06b1e60000c19c1e"),
"palpite" : [
1,
1
]
},
{
"partida" : ObjectId("53880ea52752ff402a0a7886"),
"quesito" : "placarExato",
"_id" : ObjectId("539d07eb06b1e60000c19c20"),
"palpite" : [
1,
2
]
},
{
"partida" : ObjectId("53880eae2752ff402a0a7887"),
"quesito" : "placarExato",
"_id" : ObjectId("539d0aa82fb219000054c84f"),
"palpite" : [
2,
1
]
}
],
"timeVencedor" : 1,
"userId" : ObjectId("539b2f2930de100000d7356c")
}
My question is, first: How can I filter the resumo nested document by quesito ? Is it possible to paginate this result, since this array is going to increase. And last question, is this a nice approach to this case ?
Thank you guys !

As noted, your schema implies that you actually have embedded data even though you are storing an external reference. So it is not clear if you are doing both embedding and referencing or simply embedding by itself.
The big caveat here is the difference between matching a "document" and actually filtering the contents of an array. Since you seem to be talking about "paging" your array results, the large focus here is on doing that, but still making mention of the warnings.
Multiple "filtered" matches in an array requires the aggregation framework. You can generally "project" the single match of an array element, but this is needed where you expect more than one:
Ranking.aggregate(
[
// This match finds "documents" that "contain" the match
{ "$match": { "resumo.quesito": "value" } },
// Unwind de-normalizes arrays as documents
{ "$unwind": "$resumo" },
// This match actually filters those document matches
{ "$match": { "resumo.quesito": "value" } },
// Skip and limit for paging, which really only makes sense on single
// document matches
{ "$skip": 0 },
{ "$limit": 2 },
// Return as an array in the original document if you really want
{ "$group": {
"_id": "$_id",
"otherField": { "$first": "$otherField" },
"resumo": { "$push": "$resumo" }
}}
],
function(err,results) {
}
)
Or the MongoDB 2.6 way by "filtering" inside a $project using the $map operator. But still you need to $unwind in order to "page" array positions, but there is possibly less processing as the array is "filtered" first:
Ranking.aggregate(
[
// This match finds "documents" that "contain" the match
{ "$match": { "resumo.quesito": "value" } },
// Filter with $map
{ "$project": {
"otherField": 1,
"resumo": {
"$setDifference": [
{
"$map": {
"input": "$resumo",
"as": "el",
"in": { "$eq": ["$$el.questio", "value" ] }
}
},
[false]
]
}
}},
// Unwind de-normalizes arrays as documents
{ "$unwind": "$resumo" },
// Skip and limit for paging, which really only makes sense on single
// document matches
{ "$skip": 0 },
{ "$limit": 2 },
// Return as an array in the original document if you really want
{ "$group": {
"_id": "$_id",
"otherField": { "$first": "$otherField" },
"resumo": { "$push": "$resumo" }
}}
],
function(err,results) {
}
)
The inner usage of $skip and $limit here really only makes sense when you are processing a single document and just "filtering" and "paging" the array. It is possible to do this with multiple documents, but is very involved as there is no way to just "slice" the array. Which brings us to the next point.
Really with embedded arrays, for paging that does not require any filtering you just use the $slice operator, which was designed for this purpose:
Ranking.find({},{ "resumo": { "$slice": [0,2] } },function(err,docs) {
});
Your alternate though is to simply reference the documents in the external collection and then pass the arguments to mongoose .populate() to filter and "page" the results. The change in the schema itself would just be:
"resumo": [{ "type": "Schema.Types.ObjectId", "ref": "Partida" }]
With the external referenced collection now holding the object detail rather than embedding directly in the array. The use of .populate() with filtering and paging is:
Ranking.find().populate({
"path": "resumo",
"match": { "questio": "value" },
"options": { "skip": 0, "limit": 2 }
}).exec(function(err,docs) {
docs = docs.filter(function(doc) {
return docs.comments.length;
});
});
Of course the possible problem there is that you can no longer actually query for the documents that contain the "embedded" information as it is now in another collection. This results in pulling in all documents, though possibly by some other query condition, but then manually testing them to see if they were "populated" by the filtered query that was sent to retrieve those items.
So it really does depend on what you are doing and what your approach is. If you regularly intend to "search" on inner arrays then embedding will generally suit you better. Also if you really only interesting in "paging" then the $slice operator works well for this purpose with embedded documents. But beware growing embedded arrays too large.
Using a referenced schema with mongoose helps with some size concerns, and there is methodology in place to assist with "paging" results and filtering them as well. The drawback is that you can no longer query "inside" those elements from the parent itself. So parent selection by the inner elements is not well suited here. Also keep in mind that while not all of the data is embedded, there is still the reference to the _id value of the external document. So you can still end up with large arrays, which may not be desirable.
For anything large, consider that you will likely be doing the work yourself, and working backwards from the "child" items to then match the parent(s).

I am not sure that you can filter sub-document directly with mongoose. However you can get the parent document with Model.find({'resumo.quesito': 'THEVALUE'}) (you should also and an index on it)
Then when you have the parent you can get the child by comparing the quesito
Additionnal doc can be found here: http://mongoosejs.com/docs/subdocs.html