Develop Reference

Consolidate two mongodb find and aggregate into one

I have two mongoDB queries one is aggregate and another one is a find.
They are coded in a way that if "aggregate" query gives result then "find" query doesn't run, otherwise, if "aggregate" gives no result then find query runs.
In the following way:-
var pipeline1 = [{
$match: { "user_id": "123" } //dynamic value based on request
}, {
$lookup: {
from: "config_rules",
localField: "group_id",
foreignField: "rule_type_value", //this field has group id mapped || or can be null
as: "rule"
}
},
{
$unwind: "$rule"
},{
$match:{ "rule.configtype": "profile" } //dynamic value based on request
}];
db.getCollection("user_group_mapping").aggregate(pipeline);
If the above aggregate gives a result then, the same is returned. or else we run the following find query to get config rule for the general user, and return it
var query = {
$and: [
{ rule_type_value: null }, //null for general user rules
{ configtype: "profile" }
]
}
db.getCollection("config_rules").find(query)
In simple words for a request, we check if the requester is in a group if yes, then we return config rule based on this group,
If the requester is not in any group then we return general config rule.
So my query is as seen above these are two different query running on different collection, and requires two separate mongo calls. Can I somehow combine these queries into 1 query?,
Like- If for a given user he is in a group return group-specific config or return general config rule.
I want to combine these so that in my code I will need to make only one DB call(this db call itself has both query consolidated in one) instead of two.
Sample document in user_group_mapping collection
{ "user_id": "123",
"group_id": "beta_users"
},
{ "user_id": "213",
"group_id": "alpha_testers";
}
Sample data in config_rules :
{ "rule_type_value":"beta_users",
"configType": "help",
"configVersion": "1.1"
},
{ "rule_type_value":null,
"configType": "help",
"configVersion": "1.0"
},
{ "rule_type_value":"alpha_testers",
"configType": "help",
"configVersion": "1.3"
}
Sample Input:
Req 1 user_id: "123"
configType: "help"
Req 2 user_id : "678"
configType: "help"
Sample output: (I have only written rule content for simplicity)
Req 1 config v1.1 will be returned
{ "rule_type_value":"beta_users",
"configType": "help",
"configVersion": "1.1"
}
Req 2 v1.0 will be returned
{ "rule_type_value":null,
"configType": "help",
"configVersion": "1.0"
}

try:
https://mongoplayground.net/p/m3HxBQIuqpS
please set configType at line 23 and set user_id at line 31
db.config_rules.aggregate([
{
$lookup: {
from: "user_group_mapping",
localField: "rule_type_value",
foreignField: "group_id",
as: "rule"
}
},
{
$addFields: {
"ruleCount": {
$size: "$rule",
},
"user_id": {
$first: "$rule.user_id"
}
}
},
{
$match: {
"configType": "help"
}
},
{
$match: {
$or: [
{
user_id: {
$eq: "678"//123 or 678
}
},
{
user_id: {
$exists: false
}
}
]
}
},
{
$sort: {
"ruleCount": -1
}
},
{
$limit: 1
},
{
$project: {
"_id": 0,
"rule_type_value": 1,
"configType": 1,
"configVersion": 1
}
}
])

MongoDB aggregation does not have flow control, and it will not execute subsequent stages if there are no documents output from a stage.
If you want to retrieve 1 of 2 possible values from the linked collection, change the $lookup stage so that all potential documents are selected, and filter the returned list afterward. Perhaps something similar to:
[
{$match: { "user_id": "123" }},
{$lookup: {
from: "config_rules",
let: {targetgroup: "$group_id"},
pipeline: [{$match:{
configtype: "profile",
$or:[
{$expr:{$eq:["$rule_type_value","$$targetgroup"]}},
{ rule_type_value: null, }
]
}}],
as: "rule"
}},
{$set: {
rule: {$cond: {
if: {$in: ["$group_id", "$rule.rule_type_value"]},
then: {$filter: {
input: "$rule",
cond: {$eq: ["$group_id", "$$this.rule_type_value"]}
}},
else: "$rule"
}},

Moongoose update subdocument array object

I want to update one object of an subdoc array with findByIdAndUpdate by parent id and subdoc object id. When executing this code, I got this error:
The positional operator did not find the match needed from the query.
When I use updateOne with filter parameter, it works. But I would like to get the updated document to return as json for rest api.
Is there any way to get the updated document?
My Code:
Subject.findByIdAndUpdate(
{ _id: req.params.subjectId, "bookmarks._id": req.params.bookmarkId },
{
$set: {
"bookmarks.$.uri": req.body.uri
}
},
{ new: true }
)
Schema:
{
"_id": "5e7fbfc05ff6be1446b51af7",
"user_id": "5e7e68c3fd5e9404ce6a14a3",
"title": "Hello World",
"date": "2020-03-28T21:21:04.434Z",
"bookmarks": [
{
"date": "2020-03-28T21:21:20.806Z",
"_id": "5e7fbfd05ff6be1446b51afa",
"uri": "lorem ipsum"
},
{
"date": "2020-03-28T21:21:21.433Z",
"_id": "5e7fbfd15ff6be1446b51afb",
"uri": "lorem ipsum"
}
]
}

You should be using .findOneAndUpdate() :
As your req.params.subjectId and req.params.bookmarkId are strings & respective fields in your DB will be of type ObjectId() - So convert strings to ObjectId() using below code :
const mongoose = require('mongoose');
const _id = mongoose.Types.ObjectId(req.params.subjectId);
const bookmarkId = mongoose.Types.ObjectId(req.params.bookmarkId);
Subject.findOneAndUpdate(
{ _id: _id, "bookmarks._id": bookmarkId },
{
$set: {
"bookmarks.$.uri": 'new new'
}
},
{ new: true }
)
Your issue should be mongoose's .findByIdAndUpdate() does takes in just one string value & internally converts it into {_id : ObjectId(req.params.subjectId)} to use .findOneAndUpdate(), it's just kind of wrapper.

Node js / Mongoose .find()

Im having trouble using the .find() function within mongoose on a node js server I've been trying to use this but I cannot get the key information out of my database.
user.find({key: 1} , function(err, data){
if(err){
console.log(err);
};
console.log("should be the key VVV");
console.log(data.key);
});
I'm mainly just having trouble wrapping my head around how this function takes queries and gives you back the response from your DB. If someone can break it down id be very thankful the mongoose docs weren't much help.
Also this is my user schema if it helps
var userSchema = new mongoose.Schema({
username: {type: String, unique: true},
password: {type: String},
key: {type: String},
keySecret: {type: String}
}, {collection: 'user'});
var User = mongoose.model('user',userSchema);
module.exports = User;

If you imagine your DB looking like this:
[
{
"name": "Jess",
"location": "Auckland"
},
{
"name": "Dave",
"location": "Sydney"
},
{
"name": "Pete",
"location": "Brisbane"
},
{
"name": "Justin",
"location": "Auckland"
},
]
executing the following query;
myDB.find({location: 'Brisbane'})
will return:
[
{
"name": "Pete",
"location": "Brisbane"
}
]
While myDB.find({location: 'Auckland'}) will give you
[
{
"name": "Jess",
"location": "Auckland"
},
{
"name": "Justin",
"location": "Auckland"
},
]
As you can see, you're looking through the array for a key that matches the one you're looking to find and gives you back all of the documents that match that key search in the form of an array.
The Mongoose interface gives this data to you in the form of a callback, and you just need to look for the item inside of the array it returns
user.find({location: "Auckland"}, function(err, data){
if(err){
console.log(err);
return
}
if(data.length == 0) {
console.log("No record found")
return
}
console.log(data[0].name);
})

Maybe you should use
Model.findOne({key: '1'}, function(err, data) {
console.log(data.key);
});
find() will get a doc array, and findOne() can get just one doc.
Your field key is String type, so your query obj shoule be {key: '1'}, not {key: 1}.
Read the mongoose docs carefully may help you.

hi my friend I use this method to retrieve data from db I hope that can help
user.find({key: 1})
.then((userOne)=>{
//if is an API
res.status(200).json({data : userOne});
//OR
// if you had a view named profile.ejs for example
res.render('/profile',{data : userOne, title : 'User profile' });
console.log(userOne); // just for check in console
})
.catch((error)=>{
//if an api
res.status(400).json({messageError : error});
// OR if not an api
res.render('/profile',{data : 'No data of this profile',errorToDisplay : error})
});
});

Create ID with data of other fields

I'm new to mongodb, and I'm using mongoose to validate and order the data (I'm open to change it to MySQL if this doesn't work).
The app will be an e-shop, to buy merchandising related to movies, games, ext.
My schema is as follows:
var productSchema = {
id: {
type: String,
required: true
},
name: {
type: String,
required: true
},
img: {
type: String,
required: true
},
price: {
type: Number,
required: true
},
stock: {
type: Number,
required: true
},
category: {
object: {
type: String,
required: true
},
group: {
type: String,
required: true
},
name: {
type: String,
required: true
}
}
};
This is what I would like to do:
If I have the following data in category:
category.object = "ring"
category.group = "movies"
category.name= "lord of the rings"
I want the id to be made of the first letters of every field in category and a number (the number of the last item added plus 1). In this case, It would be RMLOTR1.
What I'm doing right now
I'm adding a lot of data at the same time, so every time I do it, I made a function that iterates through all the items added and does what I want but...
My question is
Is there a built-in way to do this with mongodb or mongoose, adding the data and creating the id at the same time? I know I can do a virtual, but I want the data to be stored.
Extras
If it's not posible to do this with mongodb, is there a way to do this with MySQL?
Is doing this kind of thing considered a correct/wrong approach?

You are basically looking for a "pre" middleware hook on the "save" event fired by creating new documents in the collection. This will inspect the current document content and extract the "strings" from values in order to create your "prefix" value for _id.
There is also another part, where the "prefix" needs the addition of the numeric counter when there is already a value present for that particular "prefix" to make it distinct. There is a common technique in MongoDB used to "Generate an auto-incrementing sequence field", which basically involves keeping a "counters" collection and incrementing the value each time you access it.
As a complete and self contained demonstration, you combine the techniques as follows:
var async = require('async'),
mongoose = require('mongoose'),
Schema = mongoose.Schema;
mongoose.connect('mongodb://localhost/warehouse');
var counterSchema = new Schema({
"type": { "type": String, "required": true },
"prefix": { "type": String, "required": true },
"counter": Number
});
counterSchema.index({ "type": 1, "prefix": 1 },{ "unique": true });
counterSchema.virtual('nextId').get(function() {
return this.prefix + this.counter;
});
var productSchema = new Schema({
"_id": "String",
"category": {
"object": { "type": String, "required": true },
"group": { "type": String, "required": true },
"name": { "type": String, "required": true }
}
},{ "_id": false });
productSchema.pre('save', function(next) {
var self = this;
if ( !self.hasOwnProperty("_id") ) {
var prefix = self.category.object.substr(0,1).toUpperCase()
+ self.category.group.substr(0,1).toUpperCase()
+ self.category.name.split(" ").map(function(word) {
return word.substr(0,1).toUpperCase();
}).join("");
mongoose.model('Counter').findOneAndUpdate(
{ "type": "product", "prefix": prefix },
{ "$inc": { "counter": 1 } },
{ "new": true, "upsert": true },
function(err,counter) {
self._id = counter.nextId;
next(err);
}
);
} else {
next(); // Just skip when _id is already there
}
});
var Product = mongoose.model('Product',productSchema),
Counter = mongoose.model('Counter', counterSchema);
async.series(
[
// Clean data
function(callback) {
async.each([Product,Counter],function(model,callback) {
model.remove({},callback);
},callback);
},
function(callback) {
async.each(
[
{
"category": {
"object": "ring",
"group": "movies",
"name": "lord of the rings"
}
},
{
"category": {
"object": "ring",
"group": "movies",
"name": "four weddings and a funeral"
}
},
{
"category": {
"object": "ring",
"group": "movies",
"name": "lord of the rings"
}
}
],
function(data,callback) {
Product.create(data,callback)
},
callback
)
},
function(callback) {
Product.find().exec(function(err,products) {
console.log(products);
callback(err);
});
},
function(callback) {
Counter.find().exec(function(err,counters) {
console.log(counters);
callback(err);
});
}
],
function(err) {
if (err) throw err;
mongoose.disconnect();
}
)
This gives you output like:
[ { category: { name: 'lord of the rings', group: 'movies', object: 'ring' },
__v: 0,
_id: 'RMLOTR1' },
{ category:
{ name: 'four weddings and a funeral',
group: 'movies',
object: 'ring' },
__v: 0,
_id: 'RMFWAAF1' },
{ category: { name: 'lord of the rings', group: 'movies', object: 'ring' },
__v: 0,
_id: 'RMLOTR2' } ]
[ { __v: 0,
counter: 2,
type: 'product',
prefix: 'RMLOTR',
_id: 57104cdaa774fcc73c1df0e8 },
{ __v: 0,
counter: 1,
type: 'product',
prefix: 'RMFWAAF',
_id: 57104cdaa774fcc73c1df0e9 } ]
To first understand the Counter schema and model, you are basically defining something where you are going to look up a "unique" key and also attach a numeric field to "increment" on match. For convenience this just has a two fields making up the unique combination and a compound index defined. This could just also be a compound _id if so wanted.
The other convenience is the virtual method of nextId, which just does a concatenation of the "prefix" and "counter" values. It's also best practice here to include something like "type" here since your Counter model can be used to service "counters" for use in more than one collection source. So here we are using "product" whenever accessing in the context of the Product model to differentiate it from other models where you might also keep a similar sequence counter. Just a design point that is worthwhile following.
For the actual Product model itself, we want to attach "pre save" middleware hook in order to fill the _id content. So after determining the character portion of the "prefix", the operation then goes off and looks for that "prefix" with the "product" type data in combination in the Counter model collection.
The function of .findOneAndUpdate() is to look for a document matching the criteria in the "counters" collection and then where a document is found already it will "increment" the current counter value by use of the $inc update operator. If the document was not found, then the "upsert" option means that a new document will be created, and at any rate the same "increment" will happen in the new document as well.
The "new" option here means that we want the "modified" document to be returned ( either new or changed ) rather than what the document looked like before the $inc was applied. The result is that "counter" value will always increase on every access.
Once that is complete and a document for Counter is either incremented or created for it's matching keys, then you now have something you can use to assign to the _id in the Product model. As mentioned earlier you can use the virtual here for convenience to get the prefix with the appended counter value.
So as long as your documents are always created by either the .create() method from the model or by using new Product() and then the .save() method, then the methods attached to your "model" in your code are always executed.
Note here that since you want this in _id, then as a primary key this is "immutable" and cannot change. So even if the content in the fields referenced was later altered, the value in _id cannot be changed, and therefore why the code here makes no attempt when an _id value is already set.

Change MongoDB find results dynamically

I'm using mongoose with node.js.
Let's say I have 'Posts' DB where each document in it is a post.
Each post has a 'ReadBy' array which holds names of users that had read this post.
When I'm searching for documents in this DB, I want to "change" the 'ReadBy' value to show by Boolean value if the user that is searching for it is in this array or not.
For example, let's say these are 2 documents that are in this DB:
{ "PostName": "Post Number 1", "ReadBy": ["Tom", "John", "Adam"] }
{ "PostName": "Post Number 2", "ReadBy": ["John", "Adam"] }
If I'm user 'Tom', I want to get the results like this:
[
{
"PostName": "Post Number 1",
"ReadBy": true,
},
{
"PostName": "Post Number 2",
"ReadBy": false,
}
]
Now, I know that I can get the documents and go over each one of them with forEach function, and then use forEach again on the "ReadBy" array and change this field.
I'm asking if there is more efficient way to do it in the mongoDB query itself, or some other way in the code.
If there is another way with mongoose - even better.

Using mongoDb $setIntersection in aggregation you get the result like this :
db.collectionName.aggregate({
"$project": {
"ReadBy": {
"$cond": {
"if": {
"$eq": [{
"$setIntersection": ["$ReadBy", ["Tom"]]
},
["Tom"]
]
},
"then": true,
"else": false
}
},
"PostName": 1
}
})
So above working first like this
{ $setIntersection: [ [ "Tom", "John", "Adam"], [ "Tom"] ] }, return [ "Tom"]
{ $setIntersection: [ [ "John", "Adam"], [ "Tom"] ] }, return [ ]
and $eq to check whether setIntersection results matched with ["Tom"] if yes then return true else false

You can try something similar to
var unwind = {"$unwind": "$ReadBy"}
var eq = {$eq: ["$ReadBy", "Bob"]}
var project = {$project: {PostName: 1, seen: eq}}
db.posts.aggregate([unwind, project])
Just notice that you solution is highly inefficient. Both for storing the data ( growing array) and for searching.