Develop Reference

Populate field then find - mongoose

I have a Cheques and a Payees collection, every cheque has its corresponding Payee ID.
What I'm trying to do is to write some queries on cheques, but I need to preform the searching after populating the payee (to get the name)
const search = req.query.search || "";
const cheques = await Cheque
.find({
isCancelled: false,
dueDate: { $gte: sinceDate, $lte: tillDate }
})
.select("_id serial dueDate value payee")
.skip(page * limit)
.limit(limit)
.sort({ dueDate: -1, serial: 1 })
.populate({
path: "payee",
select: "name"
})
I guess what I'm trying do is fit this somewhere in my code,
match: {
name: { $regex: search, $options: "i" }
},
I have tried to put the match within the populate, but then it will still find all cheques even if they don't satisfy the population match but populate as null.

I hate this answer and I hope someone is going to post a better one, but I've surfed the web for that with no luck.
The only method I was able to find is to use the $lookup method in aggregation.
So you'll have to change your code from calling .find() to .aggregate().
It's not the sad news, it's great, stable and no problems at all.
but I hated that because it's going to change some patterns you might be following in your code.
const search = req.query.search || "";
const cheques = await Cheque
.aggregate([
{
$lookup: { // similar to .populate() in mongoose
from: 'payees', // the other collection name
localField: 'payee', // the field referencing the other collection in the curent collection
foreignField: '_id', // the name of the column where the cell in the current collection can be found in the other collection
as: 'payee' // the field you want to place the db response in. this will overwrite payee id with the actual document in the response (it only writes to the response, not on the database, no worries)
},
{ // this is where you'll place your filter object you used to place inside .find()
$match: {
isCancelled: false,
dueDate: { $gte: sinceDate, $lte: tillDate }
'payee.branch': 'YOUR_FILTER', // this is how you access the actual object from the other collection after population, using the dot notation but inside a string.
}
},
{ // this is similar to .select()
$project: {_id: 1, serial: 1, dueDate: 1, value: 1, payee: 1}
},
{
$unwind: '$payee' // this picks the only object in the field payee: [ { payeeDoc } ] --> { payeeDoc }
}
])
.skip(page * limit)
.limit(limit)
.sort({ dueDate: -1, serial: 1 })
Notice how you can no longer chain .select() and .populate() on the model query the way you used to do it on .find(), because now you're using .aggregate() which returns a different class instance in mongoose.
you can call .projcet() instead of doing it inside the aggregation array if you want to, but as far as I know, you can't use the .select() method.
My opinion-based solution to this problem is to include the payee information you need for filtering in the Cheque collection.
In my senario, this happen when I was filtering for the sake of my users-roles and permissions, so someone can not see what another one is seeing.
It's up to you, but this makes it easier later when you want to generate the reports (I assume you're working on a payment service).

The populate() feature provided by mongoose first fetches all the Cheques with given conditions and then makes another query to payee with _ids to populate the fields you wanted.
https://mongoosejs.com/docs/api.html#query_Query-populate
by putting match in populate you're filtering which cheques need to be populated but not the cheques themselves.
A simple solution for this is to filter the cheques which are populated as null and return them for your use.
If you see more queries of this sort and/or the collection is huge, it's better you add the payee name in the Cheques collection itself if that fits your purpose.

MongoDB: How can populate reference, and delete element in array after based on the ID of the reference

So I have a situation where I need to delete elements in an array of reference / ObjectIds, but the delete condition will be based on a field in the reference.
For example, I have schemas like the following:
const UserSchema = new mongoose.Schema({
firstName: String,
lastName: String,
homeFeeds:[{type: Schema.Types.ObjectId, requried: true, ref: "Activity"}];
}); // User , is the referenece name
const ActivitySchema = new mongoose.Schema({
requester: {type: Schema.Types.ObjectId, requried: true, ref: "User"},
message: String,
recipient: {type: Schema.Types.ObjectId, requried: true, ref: "User"},
}) // Activity, is the reference name
Now I need to delete some of the homeFeeds for a user, and the ones that should be deleted need to be by certain requester. That'll require the homeFeeds (array of 'Activity's) field to be populated first, and then update it with the $pull operator, with a condition that the Activity requester matches a certain user.
I do not want to read the data first and do the filtering in Nodejs/backend code, since the array can be very long.
Ideally I need something like:
await User.find({_id: ID})
.populate("homeFeeds", "requester")
.updateMany({
$pull: {
homeFeeds.requester: ID
}
});
But it does not work, Id really appreciate if anyone can help me out with this?
Thanks

MongoDB doesn't support $lookup in update as of version v6.0.1.
MongoServerError: $lookup is not allowed to be used within an update.
Though, this doesn't have to do with Mongoose's populate as populate doesn't depend on $lookup and fires additional queries to get the results. Have a look at here. Therefore, even if, you could achieve what you intend, that is avoiding fetching a large array on nodejs/backend, using mongoose will do the same thing for you behind the scenes which defeats your purpose.
However you should raise an issue at Mongoose's official github page and expect a response.

TypeORM findby Child Properties of a MongoDB Document

I have a MongoDB document in the following format. I can verify that it exists in MongoDB using Compass. I'm using TypeORM to make the query, not MondoDB.
{
_id: 'some id'
user: {
details: {
email: "test#test.ch",
username: "testname"
},
status: 'active'
}
}
Is it possible to use TypeORM to find by, say, the email?
I've tried
const emailExists = await this.userRepo.findOneBy({
user: {
details: {
email: "test#test.ch"
}
}
});
but emailExists always returns null even though I can validate that it exists in MongoDB. I've tried other ways to find by email using find, findOne, and more.
How do you find a matching value of a child property, like email? Is there a better approach?

MongoDB: Query on Nested Field
To specify a query condition on fields in an embedded/nested document, use dot notation.
Example: 'field.nestedField'
When querying using dot notation:
The field and nested field must be inside quotation marks.
Applying in your code:
const emailExists = await this.userRepo.findOneBy({'user.details.email': 'test#test.ch'});
Reference:
MongoDB Official Documentation: Query on Embedded/Nested Documents
Update: Looks TypeORM not work well with MongoDB, but you can try use $match.
Example:
$match : { 'field.nestedField': nestedField }
Applying in your code:
this.userRepo.findOneBy({$match: { 'user.details.email': 'test#test.ch' }});
If not work maybe try to change TypeORM to Mongoose.
Reference:
https://github.com/typeorm/typeorm/issues/2483

How to filter an array of subdocuments by two fields in each subdocument

I am attempting to add a help request system which allows the requestor to make only one request for help on each topic from an expert. If the expert lists multiple topics which they can help, I want to limit each requestor to one help request per topic per expert.
I am using node.js and mongoose.js with a self-hosted mongodb instance
I have tried using the $and operator to find the ._id of the expert as long as they don't already have an existing request from the same requestor on the same topic. It works for one update but after the experts document has a subdocument inserted with either the topic_id or the requestor_id the filter is applied and no expert is returned.
// Schema
ExpertSchema = new mongoose.Schema({
expert_id: String,
helpRequests: [
requestor_id: String,
topic_id: String
]
});
//query
const query = {
$and:[
{expert_id: req.body.expert_id},
{'helpRequests.requestor_id': {$ne: req.body.requestor_id}},
{'helpRequests.topic_id': {$ne: req.body.topic_id}}
]
};
// desired update
const update = {
$push: {
helpRequests: {
requestor_id: req.body.requestor_id,
topic_id: req.body.topic_id
}
}
Expert.findOneAndUpdate(query, update, {new: true}, (err, expert) =>{
// handle return or error...
});

The reason you are not getting any expert is condition inside your query.
Results always returned based on the condition of your query if your condition inside query get satisfied you will get your result as simple as that.
Your query
{'helpRequests.requestor_id': {$ne: req.body.requestor_id}},
{'helpRequests.topic_id': {$ne: req.body.topic_id}}
you will get your expert only if requestor_id and topic_id is not exists inside helpRequests array. thats you are querying for.
Solution
As per you schema if helpRequests contains only requestor_id and topic_id then you can achieve what you desire by below query.
Expert.findOneAndUpdate(
{
expert_id: req.body.expert_id,
}, {
$addToSet: {
helpRequests: {
requestor_id: req.body.requestor_id,
topic_id: req.body.topic_id
}
}
}, { returnNewDocument: true });

Mongoose: Adding an element to array

I'm using Drywall to create a website.
I'm trying to add a dashboard element to the accounts section of the admin site. The dashboard element is to store an array of dashboards (strings) that the user has access to.
I've managed to successfully add the "dashboards" into the schema and store data in it.
Here's the problem:
I need to be able to add elements to the array. The way the code stands currently replaces the contents of dashboards in the database.
I know I can use $addToSet, but I'm not sure how I'd do that since the fieldsToSet variable is sent to the findByIdAndUpdate() method as a single object.
Here's the snippet of my code:
workflow.on('patchAccount', function() {
var fieldsToSet = {
name: {
first: req.body.first,
middle: req.body.middle,
last: req.body.last,
full: req.body.first +' '+ req.body.last
},
company: req.body.company,
phone: req.body.phone,
zip: req.body.zip,
search: [
req.body.dashboards,
req.body.first,
req.body.middle,
req.body.last,
req.body.company,
req.body.phone,
req.body.zip,
]
};
req.app.db.models.Account.findByIdAndUpdate(req.params.id, fieldsToSet, function(err, account) {
if (err) {
return workflow.emit('exception', err);
}
workflow.outcome.account = account;
return workflow.emit('response');
});
});
Here's a link to the original file: (lines 184-203)
Thanks!

fieldsToSet is a bad name (at least misleading in this case), the parameter is actually update which can take $actions like $addToSet
I don't think you want to set (only) the search field with dashboards. I'm guessing that field is used to index users for a search. So you'll probably wind up doing something like this:
fieldsToSet = {
....all the regular stuff,
$addToSet: {dashboard: req.body.dashboardToAdd}
//I'm not sure that you can add multiple values at once
}
Since this is setting all of the values each time I'm not sure you actually will want to add single dashboard items. Instead you might want to get the full set of dashboards the user has and set the whole array again anyway (what if they removed one?)
fieldsToSet = {
....all the regular stuff,
dashboards: req.body.dashboards
//In this case you'd want to make sure dashboards is an appropriate array
}