I have the collections books, with the following data:
books: {
title: "Mickey",
subtitle: "The Mouse",
authors: [
{
name: "Walt Disney",
type: "Author"
},
{
name: "Donald Duffy",
type: "Co-Author"
},
categories: [
"kids, education"
]
]
}
What I am trying to do is after $unwind my authors array and then $match with $regex for coincidences if the field autor.name has whatever string I am sending, for example
$match: { "author.name": {$regex: ".*walt.*", $options: "si"}, title: "Mickey"}
then after $group and $push my array, I end up with
books: {
title: "Mickey",
subtitle: "The Mouse",
authors: [
{
name: "Walt Disney",
type: "Author"
}, categories: [
"kids, education"
]
]
}
Is there a way or operator in mongodb to keep all my subdocuments after a matching a field, the reason why I want this, is because on the front-end of my app every author and category would be a link to all the books that have that name or category and I want to display all of them.
You mean you want to include “Donald Duffy” as in your initial document? If so you can use simple find by making use of the dot notation so you won’t have to use aggregation. db.collection.find({"authors.name": {$regex: ".*walt.*", $options: "si"}, "title": "Mickey"})
The same with aggregation framework:
db.collection.aggregate([
{
$match: {
"authors.name": {$regex: ".*walt.*", $options: "si"},
"title": "Mickey"
}
}
])
Related
I have two simple models defined in Mongoose, composed of two schema Client and City, I have the property city defined in Client as a ObjectId, ref: 'City', so far so good.
If I query for a client and want also to filter by the 'province' property of City, I do it like this:
const client = await Client
.find({ name: "Gérard" })
.populate([{
path: 'city',
model: City,
match: { province: 'BA' }
}]);
And the output is just fine:
{
"id": "627264e3ec261a883d42ead9",
"name": "Gérard",
"email": "gerard#depardieu.fr",
"date": "1948-12-27",
"active": true,
"city": {
"id": "627264e3ec261a883d42ead1",
"name": "Buenos Aires",
"province": "BA"
}
}
Howerver, if I input a province code of a nonexistent city:
const client = await Client
.find({ name: "Gérard" })
.populate([{
path: 'city',
model: City,
match: { province: 'CA' }
}]);
It returns me this:
{
"id": "627264e3ec261a883d42ead9",
"name": "Gérard",
"email": "gerard#depardieu.fr",
"date": "1948-12-27",
"active": true,
"city": null
}
I don't want in this particular scenario, any instance of Client to be returned, and I don't know how to avoid this behavior with Mongoose, a behavior I never had to worry about with Spring Data for instance.
Somebody here told me to do something like this in my code:
const client = await Client
.find({ name: "Gérard" })
.populate([{
path: 'city',
model: City,
match: { province: 'CA' }
}]);
if(client && client.city){
return client; //or pass it to the response... whatever...
}
However, lets suppose this query is done over a collection with millions of clients, the code above is such a waste of resources that I won't even start the on what's wrong with it(ok, you can imagine how many clients not in province 'CA' the database server will send over to my app server just to be turned down....).
What I want is for no client to be returned at all, if that city/province doesn't match.
An example would be the analog SQL:
SELECT * FROM CLIENT JOIN CITY ON CITY.ID=CLIENT.CITY_ID WHERE CITY.PROVINCE = 'CA'
No client returned here, once the only client in database has its province = 'BA'.
So, how can it be done?
Thanks in advance.
I solved it myself, I had to go a little lower level with Mongoose and use aggregates and lookups.
const client = await Client.aggregate([
{
$match: { name: "Gérard" }
},
{
$lookup: {
from: City.collection.name,
pipeline: [
{
$match: {
province: 'BA'
}
}
], as: "city"
}
},
{
$unwind: "$city"
},
{
$match: {
city: { $ne: [] }
}
}
]);
Expected result:
{
"id": "627264e3ec261a883d42ead9",
"name": "Gérard",
"email": "gerard#depardieu.fr",
"date": "1948-12-27",
"active": true,
"city": {
"id": "627264e3ec261a883d42ead1",
"name": "Buenos Aires",
"province": "BA"
}
}
Witch is ok, client name "Gérard" lives in "Buenos Aires", situated in province "BA".
On the other hand:
const client = await Client.aggregate([
{
$match: { name: "Gérard" }
},
{
$lookup: {
from: City.collection.name,
pipeline: [
{
$match: {
province: 'CA'
}
}
], as: "city"
}
},
{
$unwind: "$city"
},
{
$match: {
city: { $ne: [] }
}
}
]);
Returns nothing, once the city of "Buenos Aires" is not located in province "CA".
Notice here the last parameter (as an object) the array passed to Client.aggregate() receives:
{
$match: {
city: { $ne: [] }
}
}
This tells MongoDB that in order to return data city must be not equal to an empty array.
And with that the issue is solved.
I have two mongoDB queries one is aggregate and another one is a find.
They are coded in a way that if "aggregate" query gives result then "find" query doesn't run, otherwise, if "aggregate" gives no result then find query runs.
In the following way:-
var pipeline1 = [{
$match: { "user_id": "123" } //dynamic value based on request
}, {
$lookup: {
from: "config_rules",
localField: "group_id",
foreignField: "rule_type_value", //this field has group id mapped || or can be null
as: "rule"
}
},
{
$unwind: "$rule"
},{
$match:{ "rule.configtype": "profile" } //dynamic value based on request
}];
db.getCollection("user_group_mapping").aggregate(pipeline);
If the above aggregate gives a result then, the same is returned. or else we run the following find query to get config rule for the general user, and return it
var query = {
$and: [
{ rule_type_value: null }, //null for general user rules
{ configtype: "profile" }
]
}
db.getCollection("config_rules").find(query)
In simple words for a request, we check if the requester is in a group if yes, then we return config rule based on this group,
If the requester is not in any group then we return general config rule.
So my query is as seen above these are two different query running on different collection, and requires two separate mongo calls. Can I somehow combine these queries into 1 query?,
Like- If for a given user he is in a group return group-specific config or return general config rule.
I want to combine these so that in my code I will need to make only one DB call(this db call itself has both query consolidated in one) instead of two.
Sample document in user_group_mapping collection
{ "user_id": "123",
"group_id": "beta_users"
},
{ "user_id": "213",
"group_id": "alpha_testers";
}
Sample data in config_rules :
{ "rule_type_value":"beta_users",
"configType": "help",
"configVersion": "1.1"
},
{ "rule_type_value":null,
"configType": "help",
"configVersion": "1.0"
},
{ "rule_type_value":"alpha_testers",
"configType": "help",
"configVersion": "1.3"
}
Sample Input:
Req 1 user_id: "123"
configType: "help"
Req 2 user_id : "678"
configType: "help"
Sample output: (I have only written rule content for simplicity)
Req 1 config v1.1 will be returned
{ "rule_type_value":"beta_users",
"configType": "help",
"configVersion": "1.1"
}
Req 2 v1.0 will be returned
{ "rule_type_value":null,
"configType": "help",
"configVersion": "1.0"
}
try:
https://mongoplayground.net/p/m3HxBQIuqpS
please set configType at line 23 and set user_id at line 31
db.config_rules.aggregate([
{
$lookup: {
from: "user_group_mapping",
localField: "rule_type_value",
foreignField: "group_id",
as: "rule"
}
},
{
$addFields: {
"ruleCount": {
$size: "$rule",
},
"user_id": {
$first: "$rule.user_id"
}
}
},
{
$match: {
"configType": "help"
}
},
{
$match: {
$or: [
{
user_id: {
$eq: "678"//123 or 678
}
},
{
user_id: {
$exists: false
}
}
]
}
},
{
$sort: {
"ruleCount": -1
}
},
{
$limit: 1
},
{
$project: {
"_id": 0,
"rule_type_value": 1,
"configType": 1,
"configVersion": 1
}
}
])
MongoDB aggregation does not have flow control, and it will not execute subsequent stages if there are no documents output from a stage.
If you want to retrieve 1 of 2 possible values from the linked collection, change the $lookup stage so that all potential documents are selected, and filter the returned list afterward. Perhaps something similar to:
[
{$match: { "user_id": "123" }},
{$lookup: {
from: "config_rules",
let: {targetgroup: "$group_id"},
pipeline: [{$match:{
configtype: "profile",
$or:[
{$expr:{$eq:["$rule_type_value","$$targetgroup"]}},
{ rule_type_value: null, }
]
}}],
as: "rule"
}},
{$set: {
rule: {$cond: {
if: {$in: ["$group_id", "$rule.rule_type_value"]},
then: {$filter: {
input: "$rule",
cond: {$eq: ["$group_id", "$$this.rule_type_value"]}
}},
else: "$rule"
}},
hey I am quite new to mongoose and can't get my head around search.
models
User->resumes[]->employments[]
UserSchema
{
resumes: [ResumeSchema],
...
}
ResumeSchema
{
employments: [EmploymentSchema],
...
}
EmploymentSchema
{
jobTitle: {
type: String,
required: [true, "Job title is required."]
},
...
}
Background
User has to enter job title and needs suggestions from the existing data of the already present resumes and their employment's job title
I have tried the following code.
let q = req.query.q; // Software
User.find({ "resumes.employments.jobTitle": new RegExp(req.query.q, 'ig') }, {
"resumes.employments.$": 1
}, (err, docs) => {
res.json(docs);
})
Output
[
{
_id: '...',
resumes:[
{
employments: [
{
jobTitle: 'Software Developer',
...
},
...
]
},
...
]
},
...
]
Expected OutPut
["Software Developer", "Software Engineer", "Software Manager"]
Problem
1:) The Data returned is too much as I only need jobTitle
2:) All employments are being returned whereas the query matched one of them
3:) Is there any better way to do it ? via index or via $search ? I did not find much of information in mongoose documentation to create search index (and I also don't really know how to create a compound index to make it work)
I know there might be a lot of answers but none of them helped or I was not able to make them work ... I am really new to mongodb I have been working with relational databases via SQL or through ORM so my mongodb concepts and knowledge is limited.
So please let me know if there is a better solution to do it. or something to make the current one working.
You can use one of the aggregation query below to get this result:
[
{
"jobTitle": [
"Software Engineer",
"Software Manager",
"Software Developer"
]
}
]
Query is:
First using $unwind twice to deconstructs the arrays and get the values.
Then $match to filter by values you want using $regex.
Then $group to get all values together (using _id: null and $addToSet to no add duplicates).
And finally $project to shown only the field you want.
User.aggregate({
"$unwind": "$resumes"
},
{
"$unwind": "$resumes.employments"
},
{
"$match": {
"resumes.employments.jobTitle": {
"$regex": "software",
"$options": "i"
}
}
},
{
"$group": {
"_id": null,
"jobTitle": {
"$addToSet": "$resumes.employments.jobTitle"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here
Also another option is using $filter into $project stage:
Is similar as before but using $filter instead of $unwind twice.
User.aggregate({
"$unwind": "$resumes"
},
{
"$project": {
"jobs": {
"$filter": {
"input": "$resumes.employments",
"as": "e",
"cond": {
"$regexMatch": {
"input": "$$e.jobTitle",
"regex": "Software",
"options": "i"
}
}
}
}
}
},
{
"$unwind": "$jobs"
},
{
"$group": {
"_id": null,
"jobTitle": {
"$addToSet": "$jobs.jobTitle"
}
}
},
{
"$project": {
"_id": 0
}
})
Example here
I'm using mongoose with node.js.
Let's say I have 'Posts' DB where each document in it is a post.
Each post has a 'ReadBy' array which holds names of users that had read this post.
When I'm searching for documents in this DB, I want to "change" the 'ReadBy' value to show by Boolean value if the user that is searching for it is in this array or not.
For example, let's say these are 2 documents that are in this DB:
{ "PostName": "Post Number 1", "ReadBy": ["Tom", "John", "Adam"] }
{ "PostName": "Post Number 2", "ReadBy": ["John", "Adam"] }
If I'm user 'Tom', I want to get the results like this:
[
{
"PostName": "Post Number 1",
"ReadBy": true,
},
{
"PostName": "Post Number 2",
"ReadBy": false,
}
]
Now, I know that I can get the documents and go over each one of them with forEach function, and then use forEach again on the "ReadBy" array and change this field.
I'm asking if there is more efficient way to do it in the mongoDB query itself, or some other way in the code.
If there is another way with mongoose - even better.
Using mongoDb $setIntersection in aggregation you get the result like this :
db.collectionName.aggregate({
"$project": {
"ReadBy": {
"$cond": {
"if": {
"$eq": [{
"$setIntersection": ["$ReadBy", ["Tom"]]
},
["Tom"]
]
},
"then": true,
"else": false
}
},
"PostName": 1
}
})
So above working first like this
{ $setIntersection: [ [ "Tom", "John", "Adam"], [ "Tom"] ] }, return [ "Tom"]
{ $setIntersection: [ [ "John", "Adam"], [ "Tom"] ] }, return [ ]
and $eq to check whether setIntersection results matched with ["Tom"] if yes then return true else false
You can try something similar to
var unwind = {"$unwind": "$ReadBy"}
var eq = {$eq: ["$ReadBy", "Bob"]}
var project = {$project: {PostName: 1, seen: eq}}
db.posts.aggregate([unwind, project])
Just notice that you solution is highly inefficient. Both for storing the data ( growing array) and for searching.
Does anyone know if it's possible to populate a list of IDs for another model using waterline associations? I was trying to get the many-to-many association working but I don't think it applies here since one side of the relationship doesn't know about the other. Meaning, a user can be a part of many groups but groups don't know which users belong to them. For example, I'm currently working with a model with data in mongodb that looks like:
// Group
{
_id: group01,
var: 'somedata',
},
{
_id: group02,
var: 'somedata',
},
{
_id: group03,
var: 'somedata',
}
// User
{
_id: 1234,
name: 'Jim',
groups: ['group01', 'group03']
}
And I'm trying to figure out if it's possible to setup the models with an association in such a way that the following is returned when querying the user:
// Req: /api/users/1234
// Desired result
{
id: 1234,
name: 'Jim',
groups: [
{
_id: group01,
var: 'somedata',
},
{
_id: group03,
var: 'somedata',
}
]
}
Yes, associations are supported in sails 0.10.x onwards. Here is how you can setup the models
Here is how your user model will look like:
// User.js
module.exports = {
tableName: "users",
attributes: {
name: {
type: "string",
required: true
},
groups: {
collection: "group",
via: "id"
}
}
};
Here is how your group model will look like:
// Group.js
module.exports = {
tableName: "groups",
attributes: {
name: {
type: "string",
required: "true"
}
}
};
Setting up models like this will create three tables in your DB:
users,
groups and
group_id__user_group
The last table is created by waterline to save the associations. Now go on and create groups. Once groups are created, go ahead and create user.
Here is a sample POST request for creation a new user
{
"name": "user1",
"groups": ["547d84f691bff6663ad08147", "547d850c91bff6663ad08148"]
}
This will insert data into the group_id__user_group in the following manner
{
"_id" : ObjectId("547d854591bff6663ad0814a"),
"group_id" : ObjectId("547d84f691bff6663ad08147"),
"user_groups" : ObjectId("547d854591bff6663ad08149")
}
/* 1 */
{
"_id" : ObjectId("547d854591bff6663ad0814b"),
"group_id" : ObjectId("547d850c91bff6663ad08148"),
"user_groups" : ObjectId("547d854591bff6663ad08149")
}
The column user_groups is the user id. And group_id is the group id. Now if you fetch the user using GET request, your response will look like this:
{
"groups": [
{
"name": "group1",
"createdAt": "2014-12-02T09:23:02.510Z",
"updatedAt": "2014-12-02T09:23:02.510Z",
"id": "547d84f691bff6663ad08147"
},
{
"name": "group2",
"createdAt": "2014-12-02T09:23:24.851Z",
"updatedAt": "2014-12-02T09:23:24.851Z",
"id": "547d850c91bff6663ad08148"
}
],
"name": "user1",
"createdAt": "2014-12-02T09:24:21.182Z",
"updatedAt": "2014-12-02T09:24:21.188Z",
"id": "547d854591bff6663ad08149"
}
Please note that groups are not embedded in the user collection. Waterline does the fetch from groups, users and group_id__user_group to show this result to you.
Also, if you want to do this in your controller, you will need to execute like this
User.findOne({'id': "547d854591bff6663ad08149"})
.populate('groups')
.exec(function (err, user){
// handle error and results in this callback
});
Without populate('groups'), you won't get the groups array. Hope this serves your purpose