Develop Reference

getBypage in Nested Array in MongoDB using Aggregate

I am using mongoDB as backend server, I have nested array(one level array) like this
{
"_id" : ObjectId("60b1fc6d3c43f74e0c1dba92"),
"seriesId" : "60acebf73acb5b3a98d14331",
"name" : "Season 1",
"logoURL" : "uploads/season/1622277216401.png",
"yearOfPublish" : "2021-05-29",
"description" : "Season 1",
"createdBy" : ObjectId("609cbf49ba46cc3924859ab5"),
"createdOn" : "2021-05-29T08:33:49.480Z",
"episode" : [
{
"seasonId" : "60b1fc6d3c43f74e0c1dba92",
"name" : "Episode 1",
"id" : 0,
"logoURL" : "uploads/episode/1622278616899.png",
"dateOfTelecast" : null,
"description" : "sadfgh",
"duration" : "30",
"videoType" : "customURL",
"embedCode" : "",
"url" : "https://youtu.be/kbpsXMUr7ss",
"liveboxChannel" : "",
"createdOn" : "2021-05-29T08:56:59.230Z",
"createdBy" : ObjectId("609cbf49ba46cc3924859ab5"),
"_id" : "ZaVrpOLO5"
},
{
"seasonId" : "60b1fc6d3c43f74e0c1dba92",
"name" : "Episode 2",
"id" : 0,
"logoURL" : "uploads/episode/1622279206607.png",
"dateOfTelecast" : null,
"description" : "adfd",
"duration" : "30",
"videoType" : "customURL",
"embedCode" : "",
"url" : "https://youtu.be/kbpsXMUr7ss",
"liveboxChannel" : "",
"createdOn" : "2021-05-29T09:06:48.637Z",
"createdBy" : ObjectId("609cbf49ba46cc3924859ab5"),
"_id" : "9GKqXhxcH"
},}
I have more no of seasons. from the season collection,i have episode array under the name of Episode.
Now My frontend page required that episode array alone.
response = {episode: all the episode data} and this episode data is based on skip and limit value
I have tried something in mongodb,
db.getCollection('season_copy').aggregate([
{$project: {
episodes: {
$cond:{ if: { $isArray: "$episode" }, then: { input:"$episode" }, else: 0 }
},
},
},
])
Can anyone suggest me some idea?

Check this out:
Without aggregate:
db.getCollection('season_copy')
.find({ _id: ObjectID(id)})
.project({ episode: 1 }).toArray();
With aggregate:
MongoDB playground
db.getCollection('season_copy')
.aggregate([
{
$match: {
_id: ObjectId("60b1fc6d3c43f74e0c1dba92")
}
},
{
$project: {
episode: 1
}
}
])

Sort MongoDB documents by specific content then by timestamp

I'm using nodejs with official mongodb's package. I got many documents in mongodb containing "type" and "timestamp" field. I want to sort it by prioritizing "type" (only specific content) and then "timestamp".
As example I have following documents:
{ type: "book", timestamp: 1580825471 }
{ type: "house", timestamp: 1580825502 }
{ type: "water", timestamp: 1580825515 }
{ type: "book", timestamp: 1580825478 }
{ type: "smartphone", timestamp: 1580825522 }
{ type: "book", timestamp: 1580825424 }
My goal is to have sorted by that way to priority the type "book" first (and then sort it by timestamp)
{ type: "book", timestamp: 1580825478 }
{ type: "book", timestamp: 1580825471 }
{ type: "book", timestamp: 1580825424 }
{ type: "smartphone", timestamp: 1580825522 }
{ type: "water", timestamp: 1580825515 }
{ type: "house", timestamp: 1580825502 }
I was trying to use the db.collection.aggregate with following $sort value:
$sort: {
type: "book",
timestamp: -1
}
But that didn't worked out because the $sort field's value can only have the value of "1", "-1" or "{ $meta: "textScore" }".
Does anybody have an idea how to solve that issue?
Thanks in advance
EDIT:
This solution by using
$sort: {
type: 1,
timestamp: -1
}
is not a solution since then all types are also sorted which I don't want it. I just want to have "book" as first result then after that, types can be randomized (but timestamp is still being sorted.). Reason for that is that I want to list history entries (that's why I'm using timestamp to sort it), but I want to show type "book" at first. Even if the document are older than other documents.
So yeah, for other types expect "book", I want it to be sorted by timestamp.

You can add an extra field in a project stage that creates a sort priority, then use that to sort on.
For example:
db.data.aggregate([
{ $addFields : { sortPriority: { $eq: [ "$type", "book" ] } } },
{ $sort: { sortPriority: -1, timestamp: -1} }
])
This will output the following:
{ "_id" : ObjectId("5e39892e0f18de54afe4d874"), "type" : "book", "timestamp" : 1580825478, "sortPriority" : true }
{ "_id" : ObjectId("5e39892e0f18de54afe4d871"), "type" : "book", "timestamp" : 1580825471, "sortPriority" : true }
{ "_id" : ObjectId("5e39892e0f18de54afe4d876"), "type" : "book", "timestamp" : 1580825424, "sortPriority" : true }
{ "_id" : ObjectId("5e39892e0f18de54afe4d875"), "type" : "smartphone", "timestamp" : 1580825522, "sortPriority" : false }
{ "_id" : ObjectId("5e39892e0f18de54afe4d873"), "type" : "water", "timestamp" : 1580825515, "sortPriority" : false }
{ "_id" : ObjectId("5e39892e0f18de54afe4d872"), "type" : "house", "timestamp" : 1580825502, "sortPriority" : false }
If you want to ommit the extra field add $unset stage:
db.data.aggregate([
{ $addFields : { sortPriority: { $eq: [ "$type", "book" ] } } },
{ $sort: { sortPriority: -1, timestamp: -1} },
{ $unset: "sortPriority" }
])
This will then output:
{ "_id" : ObjectId("5e39892e0f18de54afe4d874"), "type" : "book", "timestamp" : 1580825478 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d871"), "type" : "book", "timestamp" : 1580825471 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d876"), "type" : "book", "timestamp" : 1580825424 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d875"), "type" : "smartphone", "timestamp" : 1580825522 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d873"), "type" : "water", "timestamp" : 1580825515 }
{ "_id" : ObjectId("5e39892e0f18de54afe4d872"), "type" : "house", "timestamp" : 1580825502 }

You can create a sorting key by your own:
db.col.aggregate([
{
$addFields: {
sortBy: {
$cond: {
if: { $eq: ["$type", "book"] }, then: 0, else: 1
}
}
}
},
{ $sort: { sortBy: 1, timestamp: 1 } },
{ $unset: "sortBy" }
])
Output:
{ "_id" : ObjectId("5e398952227b6d209de231bb"), "type" : "book", "timestamp" : 1580825424, "sortPriority" : true }
{ "_id" : ObjectId("5e398952227b6d209de231b6"), "type" : "book", "timestamp" : 1580825471, "sortPriority" : true }
{ "_id" : ObjectId("5e398952227b6d209de231b9"), "type" : "book", "timestamp" : 1580825478, "sortPriority" : true }
{ "_id" : ObjectId("5e398952227b6d209de231b7"), "type" : "house", "timestamp" : 1580825502, "sortPriority" : false }
{ "_id" : ObjectId("5e398952227b6d209de231b8"), "type" : "water", "timestamp" : 1580825515, "sortPriority" : false }
{ "_id" : ObjectId("5e398952227b6d209de231ba"), "type" : "smartphone", "timestamp" : 1580825522, "sortPriority" : false }

MongoDB: Getting document and only one object of an array

I got multiple documents like this:
{
"_id" : "sgG6G9XTvvjj7uxwQ",
"title" : "A title",
"notes" : [
{
"id" : "ePce6fBAHx9KeKjuM",
"timestamp" : 1453731403807,
"message" : "some text",
"name" : "Tom"
},
{
"id" : "BAHx9KeKjuMePce6f",
"timestamp" : 1453731403808,
"message" : "some text",
"name" : "Bob"
},
{
"id" : "ePce6fBAHx9KeKjuM",
"timestamp" : 1453731403809,
"message" : "some text",
"name" : "Tom"
}
]
}
I get this document by using this find-query:
Collection.find({}, { sort: { title: 1 }});
But I don't need the complete notes field. I'm only interested in the id of the last note-object. That means I have to order all notes object by timestamp and just take the first (=newest) one.
I'm not quite sure, if I can do that by the find-query or if I have to do that after getting the complete data.
So the best result for me would be:
{
"_id" : "sgG6G9XTvvjj7uxwQ",
"title" : "A title",
"notes" : [
{
"id" : "ePce6fBAHx9KeKjuM"
}
]
}

You can use the $slice projection to limit the number of array elements returned. For example:
db.collection.find({}, {title: 1, "notes.id": 1, notes: {$slice: 1}}).sort({title: 1});
Will return:
{
"_id" : "sgG6G9XTvvjj7uxwQ",
"title" : "A title",
"notes" : [
{
"id" : "ePce6fBAHx9KeKjuM"
}
]
}

Map Reduce for MongoDB

I have a collection in which each object contains details of the user along with the comments user has given on specific products which is given below
{
"_id" : ObjectId("51efcbc8786df13540e46887"),
"value": {
"UserDetails" : [
[
{
"country" : "CA",
"gender" : "M",
"age" : "18",
"userIdtemp" : ObjectId("51efcbc8786df13540e46887")
}
]
],
"comments" : [
{
"commentId" : ObjectId("51efcc41786df13540e46891"),
"comment" : "Hey, what's up?",
"created" : ISODate("2013-07-24T12:44:49.400Z"),
"productId" : ObjectId("51efcbd4786df13540e4688c"),
"userId" : ObjectId("51efcbc8786df13540e46887")
},
{
"commentId" : ObjectId("51efcc43786df13540e46893"),
"comment" : "Cool",
"created" : ISODate("2013-07-24T12:44:51.004Z"),
"productId" : ObjectId("51efcbd2786df13540e4688b"),
"userId" : ObjectId("51efcbc8786df13540e46887")
}
]
}
}
{
"_id" : ObjectId("51efcbc8786df13540e46888"),
"value" : {
"UserDetails" : [
[
{
"country" : "US",
"gender" : "M",
"age" : "25",
"userIdtemp" : ObjectId("51efcbc8786df13540e46888")
}
]
],
"comments" : [
{
"commentId" : ObjectId("51efcc41786df13540e46892"),
"comment" : "Not much",
"created" : ISODate("2013-07-24T12:44:49.475Z"),
"productId" : ObjectId("51efcbd4786df13540e4688c"),
"userId" : ObjectId("51efcbc8786df13540e46888")
}
]
}
}
{
"_id" : ObjectId("51efcbc8786df13540e46889"),
"value" : {
"UserDetails" : [
{
"country" : "US",
"gender" : "F",
"age" : "13",
"userIdtemp" : ObjectId("51efcbc8786df13540e46889")
}
]
}
}
I have to extract comments separately along with there userDetails with key as productId so i have written map something like following
mapCommentsFrom = function(){
if("comments" in this.value)
{
for(var idx = 0;idx<this.value.comments.length;idx++){
var key = this.value.comments[idx].productId;
var value = [{
commentId: this.value.comments[idx].commentId,
comment:this.value.comments[idx].comment,
created:this.value.comments[idx].created,
productId:this.value.comments[idx].productId,
userId:this.value.comments[idx].userId,
country:this.value.UserDetails[0][0].country,
gender:this.value.UserDetails[0][0].gender,
age : this.value.UserDetails[0][0].age
}]
}
}
emit(key,value);
}
reduceFrom = function(k,values){
return values;
}
but where ever the number of comments are more than one i am getting only the last comment along with user details and other's key as well as value is coming null. Something like this
{ "_id" : null, "value" : null }
{
"_id" : ObjectId("51efcbd2786df13540e4688b"),
"value" : [
{
"length" : 2,
"commentId" : ObjectId("51efcc43786df13540e46893"),
"comment" : "Cool",
"created" : ISODate("2013-07-24T12:44:51.004Z"),
"productId" : ObjectId("51efcbd2786df13540e4688b"),
"userId" : ObjectId("51efcbc8786df13540e46887"),
"country" : "CA",
"gender" : "M",
"age" : "18"
}
]
}
{
"_id" : ObjectId("51efcbd4786df13540e4688c"),
"value" : [
{
"length" : 1,
"commentId" : ObjectId("51efcc41786df13540e46892"),
"comment" : "Not much",
"created" : ISODate("2013-07-24T12:44:49.475Z"),
"productId" : ObjectId("51efcbd4786df13540e4688c"),
"userId" : ObjectId("51efcbc8786df13540e46888"),
"country" : "US",
"gender" : "M",
"age" : "25"
}
]
}
Can somebody please help me as to what i am missing?
Thanks for help in advance

I cannot add comments due to reputation. But had you considered using the aggregation framework.
The $unwind operator will return you an array of sub documents quite easily and it's faster than using map/reduce.
I'm not sure it will exactly do what you're looking for but may help.
Take a look, http://docs.mongodb.org/manual/reference/aggregation/unwind/

Its because you are not emitting them in the map function.
Move the emit function inside the for loop.
mapCommentsFrom = function(){
if("comments" in this.value){
for(var idx = 0;idx<this.value.comments.length;idx++){
var key = this.value.comments[idx].productId;
var value = {
commentId: this.value.comments[idx].commentId,
comment:this.value.comments[idx].comment,
created:this.value.comments[idx].created,
productId:this.value.comments[idx].productId,
userId:this.value.comments[idx].userId,
country:this.value.UserDetails[0][0].country,
gender:this.value.UserDetails[0][0].gender,
age : this.value.UserDetails[0][0].age
}
emit(key,value);
}
}
}
Then you may also need to rewrite your reduce function to something like this
reduceFrom = function(k,valueArray){
var returnData = { values : [] } ;
for(var i=0;i<valueArray.length;i++)
returnData.values.push(valueArray[i]);
return returnData;
}

By far the easiest is to just use the aggregation framework for this. The aggregation framework allows you to execute operators on data, there is $match for doing queries (like find()) and various others. See for more information: http://docs.mongodb.org/manual/core/aggregation/
The aggregation framework also has an $unwind function that does exactly what you want. You use it like:
db.collection.aggregate( [
{ $unwind: '$value.comments' },
{ $project: {
_id: '$value.comments.productId',
value: 1
} }
] );
On your sample documents, this returns:
{
"result" : [
{
"_id" : ObjectId("51efcbd4786df13540e4688c"),
"value" : {
"UserDetails" : [
[
{
"country" : "CA",
"gender" : "M",
"age" : "18",
"userIdtemp" : ObjectId("51efcbc8786df13540e46887")
}
]
],
"comments" : {
"commentId" : ObjectId("51efcc41786df13540e46891"),
"comment" : "Hey, what's up?",
"created" : ISODate("2013-07-24T12:44:49.400Z"),
"productId" : ObjectId("51efcbd4786df13540e4688c"),
"userId" : ObjectId("51efcbc8786df13540e46887")
}
}
},
{
"_id" : ObjectId("51efcbd2786df13540e4688b"),
"value" : {
"UserDetails" : [
[
{
"country" : "CA",
"gender" : "M",
"age" : "18",
"userIdtemp" : ObjectId("51efcbc8786df13540e46887")
}
]
],
"comments" : {
"commentId" : ObjectId("51efcc43786df13540e46893"),
"comment" : "Cool",
"created" : ISODate("2013-07-24T12:44:51.004Z"),
"productId" : ObjectId("51efcbd2786df13540e4688b"),
"userId" : ObjectId("51efcbc8786df13540e46887")
}
}
},
{
"_id" : ObjectId("51efcbd4786df13540e4688c"),
"value" : {
"UserDetails" : [
[
{
"country" : "US",
"gender" : "M",
"age" : "25",
"userIdtemp" : ObjectId("51efcbc8786df13540e46888")
}
]
],
"comments" : {
"commentId" : ObjectId("51efcc41786df13540e46892"),
"comment" : "Not much",
"created" : ISODate("2013-07-24T12:44:49.475Z"),
"productId" : ObjectId("51efcbd4786df13540e4688c"),
"userId" : ObjectId("51efcbc8786df13540e46888")
}
}
}
],
"ok" : 1
}

How do I query referenced objects in MongoDB?

I've got two collections in my Mongo database, and the Foos contain references to one or more Bars:
Foo: {
prop1: true,
prop2: true,
bars: [
{
"$ref": "Bar",
"$id": ObjectId("blahblahblah")
}
]
}
Bar: {
testprop: true
}
What I want is to find all of the Foos that have at least one Bar that has its testprop set to true. I've tried this command, but it doesn't return any results:
db.Foo.find({ "bars.testprop" : { "$in": [ true ] } })
Any ideas?

You can now do it in Mongo 3.2 using $lookup
$lookup takes four arguments
from: Specifies the collection in the same database to perform the join with. The from collection cannot be sharded.
localField: Specifies the field from the documents input to the $lookup stage. $lookup performs an equality match on the localField to the foreignField from the documents of the from collection.
foreignField: Specifies the field from the documents in the from collection.
as: Specifies the name of the new array field to add to the input documents. The new array field contains the matching documents from the from collection.
db.Foo.aggregate(
{$unwind: "$bars"},
{$lookup: {
from:"bar",
localField: "bars",
foreignField: "_id",
as: "bar"
}},
{$match: {
"bar.testprop": true
}}
)

You can't. See http://www.mongodb.org/display/DOCS/Database+References
You have to do it in the client.

We have had a similar issue as we use MongoDB (3.4.4, actually 3.5.5 for testing) in combination with Morphia where we use #Referenece on a couple of entities. We are though not that happy with this solution and are considering removing these declarations and instead do the reference lookups manually.
I.e. we have a company collection and a user collection. The user entity in Morphia contains a #Refrence declaration on a company entity. The respective company collections contains entries like:
/* 1 */
{
"_id" : ObjectId("59a92501df01110fbb6a5dee"),
"name" : "Test",
"gln" : "1234567890123",
"uuid" : "f1f86961-e8d5-40bb-9d3f-fdbcf549066e",
"creationDate" : ISODate("2017-09-01T09:14:41.551Z"),
"lastChange" : ISODate("2017-09-01T09:14:41.551Z"),
"version" : NumberLong(1),
"disabled" : false
}
/* 2 */
{
"_id" : ObjectId("59a92501df01110fbb6a5def"),
"name" : "Sample",
"gln" : "3210987654321",
"uuid" : "fee69ee4-b29c-483b-b40d-e702b50b0451",
"creationDate" : ISODate("2017-09-01T09:14:41.562Z"),
"lastChange" : ISODate("2017-09-01T09:14:41.562Z"),
"version" : NumberLong(1),
"disabled" : false
}
while the user collections contains the following entries:
/* 1 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df0"),
"userId" : "admin",
"userKeyEncrypted" : {
"salt" : "78e0528db239fd86",
"encryptedAttribute" : "e4543ddac7cca9757721379e4e70567bb13956694f473b73f7723ac2e2fc5245"
},
"passwordHash" : "$2a$10$STRNORu9rcbq4qYUMld4G.HJk8QQQQBmAswSNC/4PBn2bih0BvjM6",
"roles" : [
"ADMIN"
],
"company" : {
"$ref" : "company",
"$id" : ObjectId("59a92501df01110fbb6a5dee")
},
"uuid" : "b8aafdcf-d5c4-4040-a96d-8ab1a8608af8",
"creationDate" : ISODate("2017-09-01T09:14:41.673Z"),
"lastChange" : ISODate("2017-09-01T09:14:41.765Z"),
"version" : NumberLong(1),
"disabled" : false
}
/* 2 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df1"),
"userId" : "sample",
"userKeyEncrypted" : {
"salt" : "e3ac48695dea5f51",
"encryptedAttribute" : "e804758b0fd13c219c3fc383eaa9267b70f7b8a1ed74f05575add713ce11804a"
},
"passwordHash" : "$2a$10$Gt2dq1vy4J9MeqDnXjokAOtvFcvbhe/g9wAENXFPaPxLAw1L4EULG",
"roles" : [
"USER"
],
"company" : {
"$ref" : "company",
"$id" : ObjectId("59a92501df01110fbb6a5def")
},
"uuid" : "55b62d4c-e5ee-408d-80c0-b79e02085b02",
"creationDate" : ISODate("2017-09-01T09:14:41.873Z"),
"lastChange" : ISODate("2017-09-01T09:14:41.878Z"),
"version" : NumberLong(1),
"disabled" : false
}
/* 3 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df2"),
"userId" : "user",
"userKeyEncrypted" : {
"salt" : "ab9df671340a7d8b",
"encryptedAttribute" : "7d8ad4ca6ad88686d810c70498407032f1df830596f72d931880483874d9cce3"
},
"passwordHash" : "$2a$10$0FLFw3ixW79JIBrD82Ly6ebOwnEDliS.e7GmrNkFp2nkWDA9OE/RC",
"uuid" : "d02aef94-fc3c-4539-a22e-e43b8cd78aaf",
"creationDate" : ISODate("2017-09-01T09:14:41.991Z"),
"lastChange" : ISODate("2017-09-01T09:14:41.995Z"),
"version" : NumberLong(1),
"disabled" : false
}
In order to create a special company user view we also wanted to dereference the company in the user and only include selected fields. Based on a comment within a bug report we learned that MongoDB provides a $objectToArray: "$$ROOT.element" operation which basically splits fields of the given elements into key and value pairs. Note that $objectToArray operation was added in MongoDB version 3.4.4!
An aggregation on the company element contained in the user collection using the $objectToArray operation may look like below:
dp.user.aggregate([{
$project: {
"userId": 1,
"userKeyEncrypted": 1,
"uuid":1,
"roles": 1,
"passwordHash": 1,
"disabled": 1,
company: { $objectToArray: "$$ROOT.company" }
}
}])
The result of above aggregation looks like this:
/* 1 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df0"),
"userId" : "admin",
"userKeyEncrypted" : {
"salt" : "78e0528db239fd86",
"encryptedAttribute" : "e4543ddac7cca9757721379e4e70567bb13956694f473b73f7723ac2e2fc5245"
},
"passwordHash" : "$2a$10$STRNORu9rcbq4qYUMld4G.HJk8QQQQBmAswSNC/4PBn2bih0BvjM6",
"roles" : [
"ADMIN"
],
"uuid" : "b8aafdcf-d5c4-4040-a96d-8ab1a8608af8",
"disabled" : false,
"company" : [
{
"k" : "$ref",
"v" : "company"
},
{
"k" : "$id",
"v" : ObjectId("59a92501df01110fbb6a5dee")
}
]
}
/* 2 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df1"),
"userId" : "sample",
"userKeyEncrypted" : {
"salt" : "e3ac48695dea5f51",
"encryptedAttribute" : "e804758b0fd13c219c3fc383eaa9267b70f7b8a1ed74f05575add713ce11804a"
},
"passwordHash" : "$2a$10$Gt2dq1vy4J9MeqDnXjokAOtvFcvbhe/g9wAENXFPaPxLAw1L4EULG",
"roles" : [
"USER"
],
"uuid" : "55b62d4c-e5ee-408d-80c0-b79e02085b02",
"disabled" : false,
"company" : [
{
"k" : "$ref",
"v" : "company"
},
{
"k" : "$id",
"v" : ObjectId("59a92501df01110fbb6a5def")
}
]
}
/* 3 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df2"),
"userId" : "user",
"userKeyEncrypted" : {
"salt" : "ab9df671340a7d8b",
"encryptedAttribute" : "7d8ad4ca6ad88686d810c70498407032f1df830596f72d931880483874d9cce3"
},
"passwordHash" : "$2a$10$0FLFw3ixW79JIBrD82Ly6ebOwnEDliS.e7GmrNkFp2nkWDA9OE/RC",
"uuid" : "d02aef94-fc3c-4539-a22e-e43b8cd78aaf",
"disabled" : false,
"company" : null
}
Now it's simply a matter of filtering unwanted stuff (i.e. users that have no company assigned and selecting the right array entries) in order to feed the $lookup operation #sidgate has already explained and copy the value of the dereferenced company into the user response.
I.e. an aggregation like the one below will perform an join and add the data of the company to users that have a company assigned as the as value defined in the lookup:
db.user.aggregate([
{ $project: { "userId": 1, "userKeyEncrypted": 1, "uuid":1, "roles": 1, "passwordHash": 1, "disabled": 1, company: { $objectToArray: "$$ROOT.company" }} },
{ $unwind: "$company" },
{ $match: { "company.k": "$id"} },
{ $lookup: { from: "company", localField: "company.v", foreignField: "_id", as: "company_data" } }
])
The result to the above aggregation can be seen below:
/* 1 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df0"),
"userId" : "admin",
"userKeyEncrypted" : {
"salt" : "78e0528db239fd86",
"encryptedAttribute" : "e4543ddac7cca9757721379e4e70567bb13956694f473b73f7723ac2e2fc5245"
},
"passwordHash" : "$2a$10$STRNORu9rcbq4qYUMld4G.HJk8QQQQBmAswSNC/4PBn2bih0BvjM6",
"roles" : [
"ADMIN"
],
"uuid" : "b8aafdcf-d5c4-4040-a96d-8ab1a8608af8",
"disabled" : false,
"company" : {
"k" : "$id",
"v" : ObjectId("59a92501df01110fbb6a5dee")
},
"company_data" : [
{
"_id" : ObjectId("59a92501df01110fbb6a5dee"),
"name" : "Test",
"gln" : "1234567890123",
"uuid" : "f1f86961-e8d5-40bb-9d3f-fdbcf549066e",
"creationDate" : ISODate("2017-09-01T09:14:41.551Z"),
"lastChange" : ISODate("2017-09-01T09:14:41.551Z"),
"version" : NumberLong(1),
"disabled" : false
}
]
}
/* 2 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df1"),
"userId" : "sample",
"userKeyEncrypted" : {
"salt" : "e3ac48695dea5f51",
"encryptedAttribute" : "e804758b0fd13c219c3fc383eaa9267b70f7b8a1ed74f05575add713ce11804a"
},
"passwordHash" : "$2a$10$Gt2dq1vy4J9MeqDnXjokAOtvFcvbhe/g9wAENXFPaPxLAw1L4EULG",
"roles" : [
"USER"
],
"uuid" : "55b62d4c-e5ee-408d-80c0-b79e02085b02",
"disabled" : false,
"company" : {
"k" : "$id",
"v" : ObjectId("59a92501df01110fbb6a5def")
},
"company_data" : [
{
"_id" : ObjectId("59a92501df01110fbb6a5def"),
"name" : "Sample",
"gln" : "3210987654321",
"uuid" : "fee69ee4-b29c-483b-b40d-e702b50b0451",
"creationDate" : ISODate("2017-09-01T09:14:41.562Z"),
"lastChange" : ISODate("2017-09-01T09:14:41.562Z"),
"version" : NumberLong(1),
"disabled" : false
}
]
}
As can hopefully be seen we only have the two users that contained a company reference and the two users now have also the complete company data in the response. Now additional filtering can be applied to get rid of the key/value helper and also to hide unwanted data.
The final query we came up with looks like this:
db.user.aggregate([
{ $project: { "userId": 1, "userKeyEncrypted": 1, "uuid":1, "roles": 1, "passwordHash": 1, "disabled": 1, company: { $objectToArray: "$$ROOT.company" }} },
{ $unwind: "$company" },
{ $match: { "company.k": "$id"} },
{ $lookup: { from: "company", localField: "company.v", foreignField: "_id", as: "company_data" } },
{ $project: { "userId": 1, "userKeyEncrypted": 1, "uuid":1, "roles": 1, "passwordHash": 1, "disabled": 1, "companyUuid": { $arrayElemAt: [ "$company_data.uuid", 0 ] } } }
])
Which finally returns our desired representation:
/* 1 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df0"),
"userId" : "admin",
"userKeyEncrypted" : {
"salt" : "78e0528db239fd86",
"encryptedAttribute" : "e4543ddac7cca9757721379e4e70567bb13956694f473b73f7723ac2e2fc5245"
},
"passwordHash" : "$2a$10$STRNORu9rcbq4qYUMld4G.HJk8QQQQBmAswSNC/4PBn2bih0BvjM6",
"roles" : [
"ADMIN"
],
"uuid" : "b8aafdcf-d5c4-4040-a96d-8ab1a8608af8",
"disabled" : false,
"companyUuid" : "f1f86961-e8d5-40bb-9d3f-fdbcf549066e"
}
/* 2 */
{
"_id" : ObjectId("59a92501df01110fbb6a5df1"),
"userId" : "sample",
"userKeyEncrypted" : {
"salt" : "e3ac48695dea5f51",
"encryptedAttribute" : "e804758b0fd13c219c3fc383eaa9267b70f7b8a1ed74f05575add713ce11804a"
},
"passwordHash" : "$2a$10$Gt2dq1vy4J9MeqDnXjokAOtvFcvbhe/g9wAENXFPaPxLAw1L4EULG",
"roles" : [
"USER"
],
"uuid" : "55b62d4c-e5ee-408d-80c0-b79e02085b02",
"disabled" : false,
"companyUuid" : "fee69ee4-b29c-483b-b40d-e702b50b0451"
}
Some final note to this approach: This aggregation isn't very fast, sadly, but at least it gets the job done. I haven't tested it with an array of references as originally asked though this may require some additional unwindings probably.
Update: A further way of aggregating the data, which is more in line with the comments in the above mentioned bug report, can be seen below:
db.user.aggregate([
{ $project: { "userId": 1, "userKeyEncrypted": 1, "uuid":1, "roles": 1, "passwordHash": 1, "disabled": 1, companyRefs: { $let: { vars: { refParts: { $objectToArray: "$$ROOT.company" }}, in: "$$refParts.v" } } } },
{ $match: { "companyRefs": { $exists: true } } },
{ $project: { "userId": 1, "userKeyEncrypted": 1, "uuid":1, "roles": 1, "passwordHash": 1, "disabled": 1, "companyRef": { $arrayElemAt: [ "$companyRefs", 1 ] } } },
{ $lookup: { from: "company", localField: "companyRef", foreignField: "_id", as: "company_data" } },
{ $project: { "userId": 1, "userKeyEncrypted": 1, "uuid":1, "roles": 1, "passwordHash": 1, "disabled": 1, "companyUuid": { $arrayElemAt: [ "$company_data.uuid", 0 ] } } }
])
Here the $let: { vars: ..., in: ... } operation copies the key and value of the reference into an own object and thus allows later on to lookup the reference via the corresponding operation.
Which of these aggregations performs better has yet to be profiled.

Well.. you could query the Bar Model for the _id of all documents with testprop: true, then do a find $in and populate bars on the Foo Model with an array of those _id's you got from the first query.. :P
Maybe that counts as "In the Client" :P just a thought.

It wasn't possible before, but improvements from Mongo v3.4 we can get very close to it.
You can do it with mongo-join-query. Your code would look like this:
const mongoose = require("mongoose");
const joinQuery = require("mongo-join-query");
joinQuery(
mongoose.models.Foo,
{
find: { "bars.testprop": { $in: [true] } },
populate: ["bars"]
},
(err, res) => (err ? console.log("Error:", err) : console.log("Success:", res.results))
);
How does it work?
Behind the scenes mongo-join-query will use your Mongoose schema to determine which models to join and will create an aggregation pipeline that will perform the join and the query.
Disclosure: I wrote this library to tackle precisely this use case.