Develop Reference

MongoDB conditional update array elements

I'm using Mongoose in a Node.js backend and I need to update a subset of elements of an array within a document based on a condition. I used to perform the operations using save(), like this:
const channel = await Channel.findById(id);
channel.messages.forEach((i) =>
i._id.toString() === messageId && i.views < channel.counter
? i.views++
: null
);
await channel.save();
I'd like to change this code by using findByIdAndUpdate since it is only an increment and for my use case, there isn't the need of retrieving the document. Any suggestion on how I can perform the operation?
Of course, channel.messages is the array under discussion. views and counter are both of type Number.
EDIT - Example document:
{
"_id": {
"$oid": "61546b9c86a9fc19ac643924"
},
"counter": 0,
"name": "#TEST",
"messages": [{
"views": 0,
"_id": {
"$oid": "61546bc386a9fc19ac64392e"
},
"body": "test",
"sentDate": {
"$date": "2021-09-29T13:36:03.092Z"
}
}, {
"views": 0,
"_id": {
"$oid": "61546dc086a9fc19ac643934"
},
"body": "test",
"sentDate": {
"$date": "2021-09-29T13:44:32.382Z"
}
}],
"date": {
"$date": "2021-09-29T13:35:33.011Z"
},
"__v": 2
}

You can try updateOne method if you don't want to retrieve document in result,
match both fields id and messageId conditions
check expression condition, $filter to iterate loop of messages array and check if messageId and views is less than counter then it will return result and $ne condition will check the result should not empty
$inc to increment the views by 1 if query matches using $ positional operator
messageId = mongoose.Types.ObjectId(messageId);
await Channel.updateOne(
{
_id: id,
"messages._id": messageId,
$expr: {
$ne: [
{
$filter: {
input: "$messages",
cond: {
$and: [
{ $eq: ["$$this._id", messageId] },
{ $lt: ["$$this.views", "$counter"] }
]
}
}
},
[]
]
}
},
{ $inc: { "messages.$.views": 1 } }
)
Playground

How to group two collections in one query with MongoDB or Mongoose

I have page like this. And the data is from my MongoDB server.
At first, I create User collection and put those user data. But I add the Counter collection because, I want to show their View Count.
Each user has multiple pages, and the View Count data is the sum of the Counter data.
User collection's are look like this and I only need login, id(unique) data.
Users Collection
{
"_id": {
"$oid": "5f7e92b88dc8f64cb4e6c2c0"
},
"login": "mojombo",
"id": 1,
"avatar_url": "https://avatars0.githubusercontent.com/u/1?v=4",
"url": "https://api.github.com/users/mojombo",
"html_url": "https://github.com/mojombo",
"type": "User",
"site_admin": "false",
"name": "Tom Preston-Werner",
"company": null,
"blog": "http://tom.preston-werner.com",
"location": "San Francisco",
"email": "tom#mojombo.com",
"hireable": null,
"bio": null,
"created_at": {
"$date": "2007-10-20T05:24:19.000Z"
},
"updated_at": {
"$date": "2020-09-22T15:50:44.000Z"
},
"registerDate": {
"$date": "2020-10-08T04:16:56.459Z"
},
"__v": 0
}
Counters collection
{
"_id": {
"$oid": "5f8e5bde9054ba2477dc2c57"
},
"repoName": "ale",
"repoNumber": 171780764,
"userName": "technicalpickles",
"userNumber": 159,
"viewDate": "2020-10-20",
"count": 1
}
In Counters collection I only need userName, count field to calculate counter data.
So I try for loop and aggregate method but the problem is User data is too much so I have to send too many queries to server. (If I have 10000 users data I have to send 10000 requests) So, I must not to use for loop method.
router.get(`/sitemap/0`, async (req, res, next) => {
let name;
let dataArray = [];
let pageNumber = (Number(req.params.page) * 10000); // Current Page Number
let nextPage = (Number(req.params.page) + 1) * 10000; // Next Page Number
let pageResult; // Page Result
try {
let users = await User.find({}, 'login id').limit(1000)
for (let i = 0; i < 1000; i++) {
name = users[i].login
let counters = await Counter.aggregate([{
$match: {
id: users.id,
userName: users[i].login
}
},
{
$group: {
_id: `${users[i].login}`,
count: {
$sum: "$count"
}
}
}
])
dataArray.push(counters)
console.log(counters)
}
console.log(dataArray)
} catch (e) {
// throw an error
throw e;
}
res.render("sitemap", {
dataArray
})
});
Codes result
So I want to send single query and I think it should be use 'aggregate' method. But without 'for loop'.
I want to join Users collection and Counters collection but I heard that in MongoDB & Mongoose there are no join function.
I just want to make the result like this. Is there anyway to make it like this?
{
"_id": {
"$oid": "5f7e92b88dc8f64cb4e6c2c0"
},
"login": "mojombo", --------------Match login with counter collection 'userName' value
"id": 1, ------------------------Match id with counter collection 'id' value
"count": [Sum of counters count data and it should be Number] ------- Only this field is added
"avatar_url": "https://avatars0.githubusercontent.com/u/1?v=4",
"url": "https://api.github.com/users/mojombo",
"html_url": "https://github.com/mojombo",
"type": "User",
"site_admin": "false",
"name": "Tom Preston-Werner",
"company": null,
"blog": "http://tom.preston-werner.com",
"location": "San Francisco",
"email": "tom#mojombo.com",
"hireable": null,
"bio": null,
"created_at": {
"$date": "2007-10-20T05:24:19.000Z"
},
"updated_at": {
"$date": "2020-09-22T15:50:44.000Z"
},
"registerDate": {
"$date": "2020-10-08T04:16:56.459Z"
},
"__v": 0
}

$lookup with counters collection using pipeline stage, let to pass 2 fields id, login to match with counters collection
$match both fields conditions
$addFields to count sum of count field from counters array, using $reduce to iterate loop and $add to sum values of count field
$skip for pagination
$limit pass your limit of documents
let page = 1; // start pagination from first
let limit = 1000;
let skip = (page - 1) * limit;
let users = await User.aggregate([
{
$lookup: {
from: "counters",
let: {
id: "$id",
login: "$login"
},
pipeline: [
{
$match: {
$expr: {
$eq: ["$userName", "$$login"],
$eq: ["$userNumber", "$$id"]
}
}
}
],
as: "count"
}
},
{
$addFields: {
count: {
$reduce: {
input: "$count",
initialValue: 0,
in: { $add: ["$$value", "$$this.count"] }
}
}
}
},
{ $skip: skip },
{ $limit: limit }
])
Playground

getting error unknown group operator '$group' in mongoDB

I am trying to count distinct(not unique) or Emp No in same department.but getting error
query failed: unknown group operator '$group'
here is my code
https://mongoplayground.net/p/UvYF9NB7vZx
db.collection.aggregate([
{
$group: {
_id: "$Department",
total: {
"$group": {
_id: "$Emp No"
}
}
}
}
])
Expected output
[
{
"_id": "HUAWEI”,
“total”:1
},
{
"_id": "THBS”,
“total”:2
}
]
THBShave two different Emp No A10088P2C and A20088P2C
HUAWEI have only one Emp No A1016OBW

so, $group is Pipeline stage, you can only use it in upper level.
But for your required output there is lots of ways i believe,
we can do something like this as well:
db.collection.aggregate([
{
$group: {
_id: {
dept: "$Department",
emp: "$Emp No"
},
total: {
"$sum": 1
}
}
},
{
$group: {
_id: "$_id.dept",
total: {
"$sum": 1
}
}
}
])
Here, in first stage we are grouping with Department and its Emp No , and also we are having count of how many Emp No is in each dept.
[this count you can remove though as we are not using it.]
result of this stage will be:
[
{
"_id": {
"dept": "THBS",
"emp": "A10088P2C"
},
"total": 2
},
{
"_id": {
"dept": "THBS",
"emp": "A20088P2C"
},
"total": 1
},
{
"_id": {
"dept": "HUAWEI",
"emp": "A1016OBW"
},
"total": 3
}
]
next on top of this part data, i'm grouping again, with the dept. which comes in $_id.dept, and making count in the same way, which gives the result in your required format.
[
{
"_id": "HUAWEI",
"total": 1
},
{
"_id": "THBS",
"total": 2
}
]
Demo

$match in mongoose won't query aggregated fields

I'm trying to query from a field from another collection:
const Order = require("../../models/orders");
const User = require("../../models/user");
module.exports = {
async list(req, res) {
const { page = 1, limit = 10 } = req.query;
let filters = req.query;
filters = {
...filters,
page: undefined,
limit: undefined
};
const myAggregate = Order.aggregate([
{
$lookup: {
from: "users",
localField: "user",
foreignField: "_id",
as: "user"
}
},
{
$unwind: "$user"
},
{
$match: { ...filters }
},
{
$project: {
coupon: 1,
status: 1,
"user._id": 1,
"user.firstName": 1,
"user.lastName": 1,
currency: 1,
rate: 1,
amount: 1,
subtotal: 1,
total: 1,
recipient: 1,
createdAt: 1
}
}
]);
const orders = await Order.aggregatePaginate(myAggregate, { page, limit });
return res.send(orders);
}
};
I'm using a plugin called mongoose-aggregate-paginate-v2, but it just add pagination to the mongoose array, so the problem is with the $match.
The collection Orders stores this object:
{
"_id" : ObjectId("5d447d49a910461fd3b5e155"),
"coupon" : [],
"status" : "Processing",
"user" : ObjectId("5d35f67f1cf34e0018e36d69"),
"currency" : "BRL",
"rate" : 10640,
"amount" : 30,
"subtotal" : 30,
"total" : 30,
"recipient" : ObjectId("5d404b39f005f70419d89ae3"),
"createdAt" : ISODate("2019-07-30T13:59:48.532Z")
}
Which, after the aggregation, turns this:
{
"docs": [
{
"_id": "5d447d49a910461fd3b5e155",
"coupon": [],
"status": "Processing",
"user": {
"_id": "5d35f67f1cf34e0018e36d69",
"firstName": "Boo",
"lastName": "Foo"
},
"currency": "BRL",
"rate": 10640,
"amount": 30,
"subtotal": 30,
"total": 30,
"recipient": "5d404b39f005f70419d89ae3",
"createdAt": "2019-07-30T13:59:48.532Z"
}
],
"totalDocs": 1,
"limit": 10,
"page": 1,
"totalPages": 1,
"pagingCounter": 1,
"hasPrevPage": false,
"hasNextPage": false,
"prevPage": null,
"nextPage": null
}
Inside the docs is the array returned by mongoose. If I make a query of any of the Orders fields, the query returns successful, only with data that corresponds to the query. However, if I try to query the firstName, no matter how I try to format the $match, the query is an empty array.
How should I query aggregated fields?

mongodb to return object from facet

Is it possible to have facet to return as an object instead of an array? It seems a bit counter intuitive to need to access result[0].total instead of just result.total
code (using mongoose):
Model
.aggregate()
.match({
"name": { "$regex": name },
"user_id": ObjectId(req.session.user.id),
"_id": { "$nin": except }
})
.facet({
"results": [
{ "$skip": start },
{ "$limit": finish },
{
"$project": {
"map_levels": 0,
"template": 0
}
}
],
"total": [
{ "$count": "total" },
]
})
.exec()

Each field you get using $facet represents separate aggregation pipeline and that's why you always get an array. You can use $addFields to overwrite existing total with single element. To get that first item you can use $arrayElemAt
Model
.aggregate()
.match({
"name": { "$regex": name },
"user_id": ObjectId(req.session.user.id),
"_id": { "$nin": except }
})
.facet({
"results": [
{ "$skip": start },
{ "$limit": finish },
{
"$project": {
"map_levels": 0,
"template": 0
}
}
],
"total": [
{ "$count": "total" },
]
})
.addFields({
"total": {
$arrayElemAt: [ "$total", 0 ]
}
})
.exec()

You can try this as well
Model
.aggregate()
.match({
"name": { "$regex": name },
"user_id": ObjectId(req.session.user.id),
"_id": { "$nin": except }
})
.facet({
"results": [
{ "$skip": start },
{ "$limit": finish },
{
"$project": {
"map_levels": 0,
"template": 0
}
}
],
"total": [
{ "$count": "total" },
]
})
.addFields({
"total": {
"$ifNull": [{ "$arrayElemAt": [ "$total.total", 0 ] }, 0]
}
})
.exec()

imagine that you want to pass the result of $facet to the next stage, let's say $match. well $match accepts an array of documents as input and return an array of documents that matched an expression, if the output of $facet was just an element we can't pass its output to $match because the type of output of $facet is not the same as the type of input of $match ($match is just an example). In my opinion it's better to keep the output of $facet as array to avoid handling those types of situations.
PS : nothing official in what i said