Develop Reference

How do I add a new Subdocument if it's not exist already, then push another subdocument under it

I have a Schema named user on mongoose, and that schema has lastExams property as below:
lastExams: [{
lecture: {
type: String,
required: true
},
exams: [{
examID: {type: [mongoose.Schema.Types.ObjectId], ref: Exam, required: true},
resultID: {type: [mongoose.Schema.Types.ObjectId], ref: Result, required: true},
date: {type: Date, required: true},
result: {}
}]}]
With this, I want to keep the last 10 exams user have taken for each lecture they have. So after each exam, I want to check if the corresponding 'lastExams.lecture' subdocument already exists, if so, push the result that lastExams.$.exams array. Else, upsert that subdocument with first element of the exams array.
for example, thats an user document without any exams on it;
user: {
name: { firstName: '***', lastName: '***' },
email: '****#****.***',
photo: 'https://****.jpg',
role: 0,
status: true,
program: {
_id: 6017b829c878b5bf117dfb92,
dID: '***',
eID: '****',
pID: '****',
programName: '****',
__v: 0
},
lectures: [
{
some data
}
],
currentExams: [
some data
],
lastExams: []
}}
If user sends an exam data for math-1 lecture, since there is no exam with that lecture name, I need to upsert that document to get user document to become as below;
user: {
name: {
firstName: '***',
lastName: '***'
},
email: '****#****.***',
photo: 'https://****.jpg',
role: 0,
status: true,
program: {
_id: 6017 b829c878b5bf117dfb92,
dID: '***',
eID: '****',
pID: '****',
programName: '****',
__v: 0
},
lectures: [{
some data
}],
currentExams: [
some data
],
lastExams: [{
lecture: 'math-1',
exams: [
examID: 601 ba71e62c3d45a4f10f080,
resultID: '602c09b2148214693694b16c',
date: 2021 - 02 - 16 T18: 06: 42.559 Z,
result: {
corrects: 11,
wrongs: 9,
empties: 0,
net: 8.75,
score: 43.75,
qLite: [
'some question objects'
]
}
]
}]
}
}
I can do that like this;
User.findOneAndUpdate({email: result.user}, {$addToSet: {'lastExams': {
lecture: result.lecture,
exams: [{
examID: doc.examID, // btw, idk why, these id's saving to database as arrays
resultID: doc.id,
date: doc.createdAt,
result: doc.results
}]
}}})
But since this adds new subdoc with same lecture value each time. I am having to check if there is a subdoc with that lecture value first manually. if not so, run the above code, else, to push just exam data to that lectures subdoc, I am using below code;
User.findOneAndUpdate({email: result.user, 'lastExams.lecture': result.lecture }, {$addToSet: {'lastExams.$.exams': {
examID: doc.examID,
resultID: doc.id,
date: doc.createdAt,
result: doc.results
}}})
So, I am having to make User.find() query first to see if that lecture is already there, and pop an item if it's exams.lengt is 10. then deciding to what kind of User.findOneAndUpdate() to use.
Do you think there is any way to make this proccess in a single query? Without going to database 2-3 times for each exam save?
I know it's too long, but i couldn't put it straight with my poor english. Sorry.

Two methods:
Like you already did, multiple queries.
In the first query, you have to check if the subdocument exists, if it does, update it in the second query, else create subdocument with first item in the second query.
Using $out in an aggregation pipeline
If multiple round trips is only the issue, checkout $out aggregation pipeline method, which allows to write aggregation output to a collection. In there, you can first match you document, check if the subdocument exists, using $filter followed by $cond. Once you have the data ready, use $out to write it back to the collection.
NB: Aggregation pipeline is expensive operation than findOneAndUpdate IMO, so make sure you test the average latency for both 2 query method and single aggregation method and decide which is faster in your case.
PS: Sorry for not providing an example, I simply don't know it very well to create a working example for you. You can refer to the mongoDB docs for details.
https://docs.mongodb.com/manual/reference/operator/aggregation/out/
Also, there is a Jira ticket discussion going on for this specific use case in MongoDB. Hoping some simple solution will be implemented in MongoDB in the upcoming versions

Cannot populate nested array

I know that this has been asked a million times, but it doesn't seem to be working for me. I cannot find a way to populate the references in an object when getting it from the database. No matter what I try it either returns an empty list or just the list of id's. What am I doing wrong here?
displayInventory: (req, res)=>{
Merchant.find({otherId: merchantId})
.populate({
path: "ingredients",
populate: {
path: "ingredient",
model: "Ingredient"
}
})
.then((merchant)=>{
console.log(merchant);
if(merchant){
return res.render("./inventory/inventory", {merchant: merchant});
}else{
return res.redirect("/merchant/new");
}
})
.catch((err)=>{
console.log(err);
return res.render("error");
});
}
const MerchantSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
otherId: {
type: String,
required: true
},
lastUpdatedTime: {
type: Date,
default: Date.now
},
ingredients: [{
ingredient: {
type: mongoose.Schema.Types.ObjectId,
ref: "Ingredient"
},
quantity: {
type: Number,
min: [0, "Quantity cannot be less than 0"]
}
}],
recipes: [{
type: mongoose.Schema.Types.ObjectId,
ref: "Recipe"
}]
});

Refer to your schema, look like you just need to do:
Merchant.find({cloverId: merchantId}).populate("ingredients.ingredient")...

I will answer my own question in case others have a similar problem. My problem was that I renamed some variables but forgot to do it when saving data to the database. Stupid mistake, but just a reminder to be careful when refactoring. I used Coung Le Ngoc's answer, it works perfectly.

You can try the below code.
Merchant.find({otherId: merchantId})
.populate({ path: 'ingredients' })
.exec(function(err, docs) {
var options = {
path: 'ingredients.components',
model: 'Ingredient'
};
if (err) return res.json(500);
Merchant.populate(docs, options, function (err, merchant) {
res.json(merchant);
});
});

How to count number of ObjectId occurrences in an array of ObjectIds' and attach the result to each document that is being returned?

I have a Tag schema:
const TagSchema = new Schema({
label: {type: String, unique: true, minlength: 2},
updatedAt: {type: Date, default: null},
createdAt: {type: Date, default: new Date(), required: true},
createdBy: {type: mongoose.Schema.Types.ObjectId, ref: 'Account', default: null}
});
const Tag = mongoose.model('Tag', TagSchema);
Then I have a Page schema:
const PageSchema = new Schema({
tags: {type: [{type: mongoose.Schema.Types.ObjectId, ref: 'Tag'}], default: [], maxlength: 5}
});
const Page = mongoose.model('Page', PageSchema);
As you can see it contains a Tag reference in its tags array.
Now what I need to do is when I fetch all tags via /tags, I also need to count the number of times each tag is used in all Pages.
So if a Tag is used 2times across all Pages, it should set an .occurrences property on the returned tag. For example this would be a response from /tags:
[
{_id: '192398dsf7asdsdds8f7d', label: 'tag 1', updatedAt: '20170102', createdAt: '20170101', createdBy: '01238198dsad8s7d78ad7', occurrences: 2},
{_id: '19239asdasd8dsf7ds8f7d', label: 'tag 2', updatedAt: '20170102', createdAt: '20170101', createdBy: '01238198dsad8s7d78ad7', occurrences: 1},
{_id: '192398dsf7zxccxads8f7d', label: 'tag 1', updatedAt: '20170102', createdAt: '20170101', createdBy: '01238198dsad8s7d78ad7', occurrences: 5},
]
I would have imagined that I could achieve this pretty easily in a mongoose pre('find') hook like this:
TagSchema.pre('find', function() {
Page.count({tags: this._id}, (err, total) => {
this.occurrences = total;
});
});
However, there are two problems here:
Page.count throws an error saying it's not a function, which I don't understand why because I use it somewhere else perfectly fine. And Page has been imported properly. Suggesting that you can't use count in a hook or something similar.
this is not the Tag document.
So I guess that the way I am going about this is completely wrong.
Since I am a novice in MongoDB maybe someone can provide me with a better, working, solution to my problem?

db.Page.aggregate([
{
$unwind:"$tags"
},
{
$group:{
_id:"$tags",
occurrences:{$sum:1}
}
},
{
$lookup:{ //get data for each tag
from:"tags",
localField:"_id",
foreignField:"_id",
as:"tagsData"
}
},
{
$project:{
tagData:{$arrayElemAt: [ "$tagsData", 0 ]},
occurrences:1
}
}
{
$addFields:{
"tagData._id":"$_id",
"tagData.occurrences":"$occurrences"
}
},
{
$replaceRoot: { newRoot: "$tagData" }
}
])

Ok first, you need getQuery() this allows you to access the properties that you are finding the tag with. so if you find the tag by _id, you will have access to it in the pre hook by this.getQuery()._id
Second, you are going to need to use the "Parallel" middleware, can be found in the docs. This allows the find method to wait until the done function is called, so it is gonna wait until the tag is updated with the new occurrences number.
As the object can no be accessed inn the pre hook as it is yet to be found, you are going to have to use findOneAndUpdate method in the pre hook.
So the code should look like :
The find method :
Tag.find({ _id: "foo"}, function (err, foundTag) {
console.log(foundTag)
})
The pre hook :
TagSchema.pre('find', true, function (next, done) {
mongoose.models['Page'].count({ tags: this.getQuery()._id }, (err, total) => {
mongoose.models['Tag'].findOneAndUpdate({ _id: this.getQuery()._id }, { occurrences: total }, function () {
next()
done()
})
});
});

MongoDB index search works in console, but not from browser

All of this was working yesterday so I'm really not sure whats wrong. I'v searched on SO and been through the docs and steps a number of times but I havnt got it to work. Any help much appreciated as always!
If I type db.movies.find({ '$text': { '$search': 'elephant' } }) into the shell, it returns a list of titles with 'elephant' in the title. But if I run the same search from my browser it crashes the node server and I get MongoError: text index required for $text query.
The error comes from the error response from the find():
Movie.find(
{ '$text': { '$search': 'elephants' } },
(err, movies) => {
if (err) throw err; // MongoError: text index required for $text query
if (!movies) {
res.json({
data: null,
success: false,
message: 'No movies data'
});
} else {
res.json({
success: true,
data: movies
});
}
})
The index on my DB looks like this:
{
"v" : 2,
"key" : {
"_fts" : "text",
"_ftsx" : 1
},
"name" : "title_text",
"ns" : "db.movies",
"weights" : {
"title" : 1
},
"default_language" : "english",
"language_override" : "language",
"textIndexVersion" : 3
}
I also have mongoose schemas defined and I'm using an auto increment plugin to increment a custom id field on each movie save. (Saving movies works, so I am connecting to the DB ok)
mongoose.model('Job', new Schema({
id: Number,
title: String,
desc: String,
genre: String
})
.index({ title: 'text' })
.plugin(autoIncrement.mongoosePlugin));

As pointed out in the comments by #Chris Satchell
Adding the index to the mongoose schema solves this.
instead of:
mongoose.model('Job', new Schema({
id: Number,
title: String,
desc: String,
genre: String
})
do this:
mongoose.model('Job', new Schema({
id: Number,
title: { type: String, text: true },
desc: String,
genre: String
})
even though I still had .index({ title: 'text' }) on the Schema, you still need to define the { type: String, text: true } on the property.

Pushing object into array using MongoDB syntax and SimpleSchema validation

The idea is to push an object that looks like this into a field called likes, which is an array:
{
movieId: "VgtyvjVUAjf8ya",
information: {
genre: "Action",
length: "160",
language: "English"
}
}
I thought this would do it:
Meteor.users.update({_id: Meteor.userId()}, {$push: {likes: {movieId: movieId, information: informationObj}}})
But either it is wrong or the validation by SimpleSchema has some issues (it doesn't complain, though) because all I get is an empty object in an array! And no, there's nothing wrong with the values themselves, I have checked.
The SimpleSchema for the field in question looks like this:
likes: {
type: [Object],
optional: true
}
I've tried reading through the documentation but I don't really understand what's wrong. Anyone knows?

If you don't care to validate the objects that get pushed into the likes property, you can set blackbox to true in your schema, like so:
likes: {
type: [Object],
optional: true,
blackbox: true
}
This will allow you to put whatever you want into a "like" object.
If you do want to validate the "like" objects, then you'll need to create some additional schemas, like so:
var likeInfoSchema = new SimpleSchema({
genre: {
type: String
},
length: {
type: String
},
language: {
type: String
}
});
var likeSchema = new SimpleSchema({
movieId: {
type: String
},
information: {
type: likeInfoSchema
}
});
Meteor.users.attachSchema(new SimpleSchema({
// ...
likes: {
type: [likeSchema]
}
}));