Develop Reference

How to populate a nested path using aggregate?

I have been trying to find the averageSum and averageRating, but I cannot get it done because I do not know how to populate using aggregate or if there is a work around. I have heard of $lookup, but I am not sure how to do it, also it tells me something about atlas tier does not do it. Is there a another way around to this? Can I populate then aggregate or can I find the averageSum and averageRating at the end using another method? Please help me
here is how my schema looks:
const favoriteSchema = new mongoose.Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: "User",
unique: true,
},
favoriteSellers: [
//create array of object id, make sure they are unique for user not to add multiple sellers
{
type: mongoose.Schema.Types.ObjectId,
ref: "Seller",
unique: true,
},
],
});
and here is my Seller schema:
const sellerSchema = new mongoose.Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: "User",
unique: true,
},
business: businessSchema,
sellerType: [String],
reviews: [
{
by: {
type: mongoose.Schema.Types.ObjectId,
ref: "User",
unique: true,
},
title: {
type: String,
},
message: {
type: String,
},
rating: Number,
imagesUri: [String],
timestamp: {
type: Date,
default: Date.now,
},
},
],
...
});
So I have an array of favorite sellers, I want to populate the sellers, then populate the reviews.by and user paths, and then do the calculation for the average sum and do the average rating. If possible please help me. What are my options here? Just do it outside on the expressjs route logic?
Here is my aggregate:
aggregatePipeline.push({
$match: { user: req.user._id },
});
//****** Here is where I want to populate before start the rest **********
then continue to following code because the fields(paths) are not populated so it averageSum will be 0 at all times.
aggregatePipeline.push({
$addFields: {
ratingSum: {
$reduce: {
initialValue: 0,
input: "$favoriteSellers.reviews",
in: { $sum: ["$$value", "$$this.rating"] },
},
},
},
});
//get average of rating ex. seller1 has a 4.5 averageRating field
aggregatePipeline.push({
$addFields: {
averageRating: {
$cond: [
{ $eq: [{ $size: "favoriteSellers.reviews" }, 0] }, //if it does not have any reviews, then we will just send 0
0, //set it to 0
{
$divide: ["$ratingSum", { $size: "$reviews" }], //else we can divide to get average Rating
},
],
},
},
});
let favList = await Favorite.aggregate(aggregatePipeline).exec();
When I retrieve my code, the array looks like:
[
{
_id: new ObjectId("62a7ce9550094eafc7a61233"),
user: new ObjectId("6287e4e61df773752aadc286"),
favoriteSellers: [ new ObjectId("6293210asdce81d9f2ae1685") ],
}
]
Here is a sample on how I want it to look:
(so each seller should have a field of average rating like and averageSum)
_id: 'favorite_id.....'
user: 'my id',
favoriteSellers:[
{
_id: 'kjskjhajkhsjk',
averageRating: 4.6
reviews:[.....],
...
},
{
_id: 'id______hsjk',
averageRating: 2.6
reviews:[.....],
...
},
{
_id: 'kjid______khsjk....',
averageRating: 3.6
reviews:[.....],
...
}
]

Wrong implementation of $lookup from MongoDB in NodeJs

I have an Entity model and a Review model, they are related by entityId field which is part Review model.
I am trying to find all the reviews from a specific entity and then calculate the average of all the rating of all reviews. (rating is another field of Review model, given below)
This is how Entity model looks:
const entitySchema = new Schema({
name: {
type: String,
required: true,
trim: true,
unique: true,
}
});
and this is Review model:
const reviewSchema = new Schema({
rating: {
type: Number,
min: 0,
max: 5,
required: true,
},
comment: {
type: String,
trim: true,
},
public: {
type: Boolean,
required: true,
default: false,
},
entityId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Entity',
required: true,
}
}, {
timestamps: true,
});
I want to $lookup function here and this is what I have tried till now:
router.get('/entities/reviews/average', async (req, res) => {
try {
const entity = await Entity.find();
const entityId = [];
Object.keys(entity).forEach((key) => {
entityId.push(entity[key]._id);
});
Object.keys(entityId).forEach((key) => {
const reviews = Review.aggregate([
{ $match: { entityId: ObjectId(entityId[key]) } },
{
$lookup: {
from: 'entity',
localField: '_id',
foriegnField: 'entityId',
as: 'rating',
},
},
{
$group: {
_id: null,
avg: { $avg: '$rating' },
},
},
]);
res.send(reviews);
});
} catch (e) {
res.status(500).send();
}
});
But this doesn't work it gives this response back
{
"_pipeline": [
{
"$match": {
"entityId": "5eb658d7"
}
},
{
"$lookup": {
"from": "entity",
"localField": "_id",
"foriegnField": "entityId",
"as": "rating"
}
},
{
"$group": {
"_id": null,
"avg": {
"$avg": "$rating"
}
}
}
],
"options": {}
}
How to do this? What am I doing wrong?

I am not getting the reason behind that you are getting same query in return,
If i am not wrong then you are doing average of rating for entity, my suggestion is you can combine query and do it in single query,
$lookup to join rating collection
$addFields to do average, make array of rating using $map and then do average using $avg
router.get('/entities/reviews/average', async (req, res) => {
try {
let reviews = await Entity.aggregate([
{
$lookup: {
from: "Review",
localField: "_id",
foreignField: "entityId",
as: "avgRating"
}
},
{
$addFields: {
avgRating: {
$avg: {
$map: {
input: "$avgRating",
in: "$$this.rating"
}
}
}
}
}
])
res.send(reviews);
} catch (e) {
res.status(500).send();
}
});
Playground

https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/
Lookup is doing a sql type join so the two fields you want to join on would have to match. I couldn't get you query working in mongo shell but I did get the following to work.
Reviews.aggregate([
{
$group: {
_id: { entityId: "5f56460d567f27054739c3bb" },
averageRating: { $avg: "$rating" },
},
},
])
It's run in mongo shell as well.

Update value inside mongodb array object

I'm trying to update a value inside my array of objects.
Looking at the above mongoDB schema what I want is:
Find an expense with the ID match with the _id and need to update the fields with new ones from the req.body.
Just need to update the: expensesType, description, price and status.
The following code is what I tried to do.
First I need to match the right expense and it works fine but when I try to house.save() show me a message 'house.save is not a function'. So I think maybe I need to use a mongoDB function to get the result.
router.put("/editExpense/:id", ensureAuthenticated, (req, res) => {
var id = mongoose.Types.ObjectId(req.params.id);
House.find(
{ "expensesHouse._id": id },
{
members: 1,
name: 1,
description: 1,
address: 1,
type: 1,
user: 1,
userID: 1,
userType: 1,
expensesHouse: { $elemMatch: { _id: id } },
date: 1
}
).then(house => {
console.log(house);
expenseType = req.body.expenseType;
description = req.body.description;
price = req.body.price;
status = req.body.status;
house.save().then(() => {
req.flash("success_msg", "Expenses Updated");
res.redirect("/houses/dashboard");
});
});
});
****** UPDATED ******
After a search I found this updateOne and after adjusts, this is my final result but this way I delete every record..
router.put("/editExpense/:id", ensureAuthenticated, (req, res) => {
var id = mongoose.Types.ObjectId(req.params.id);
House.updateOne(
{ "expensesHouse._id": id },
{
members: 1,
name: 1,
description: 1,
address: 1,
type: 1,
user: 1,
userID: 1,
userType: 1,
expensesHouse: { $elemMatch: { _id: id } },
date: 1
},
{ $set: { "expensesHouse.expenseType": req.body.expenseType } }
).then(house => {
req.flash("success_msg", "Expenses Updated");
res.redirect("/houses/dashboard");
});
});
*********** RESOLUTION ***********
I just fixed the problem the way I show below.
House.updateOne(
{ "expensesHouse._id": id },
{
$set: {
expensesHouse: {
expenseType: req.body.expenseType,
description: req.body.description,
price: req.body.price,
status: req.body.status
}
}
}

You are really close to the answer the problem right now that you are having is syntax difference between find and UpdateOne
This is what Find expects, Check MongoDB docs
db.collection.find(query, projection)
This is what updateOne expects, Check Mongo docs
db.collection.updateOne(
<filter>,
<update>,
{
upsert: <boolean>,
writeConcern: <document>,
collation: <document>,
arrayFilters: [ <filterdocument1>, ... ],
hint: <document|string> // Available starting in MongoDB 4.2.1
}
)
See the Difference? Second parameter should be update not projection because Update one
returns
matchedCount containing the number of matched documents
modifiedCount containing the number of modified documents
upsertedId containing the _id for the upserted document.
A boolean acknowledged as true if the operation ran with write concern or false if write concern was disabled.
So Your code should be
House.updateOne(
{ "expensesHouse._id": id },
{ $set: { "expensesHouse.expenseType": req.body.expenseType } }
).then(house => {
req.flash("success_msg", "Expenses Updated");
res.redirect("/houses/dashboard");
});
});

House.findOneAndUpdate({userId : req.params.userId},
{ $set: { "expensesHouse.$[element].status": req.body.status } },
{ multi:true, arrayFilters: [{ "element.userID" : req.params.subUserId }], new:true })
Your Api reuquest consist of both the IDs (outer as well as inner) like /api/update/:userId/:subUserId

append to list if exist or add list in dynamoDB

I have a product table in DynamoDB which has some items. Now I need to add list of buyers to the product which can grow i.e. append to list. It works for if I have an empty list or a list with some items in the table item but for the first addition it throws an error. Is there any way to check if list exists then append else add a list. here is my code
let params = {
TableName: "product",
ExpressionAttributeNames: {
"#Y": "buyer"
},
ExpressionAttributeValues: {
":y": ["PersonXYZ"]
},
Key: {
id: 'Hy2H4Z-lf'
},
UpdateExpression: "SET #Y = list_append(#Y,:y)"
};
updateItemInDDB(params).then((data) => {
res.status(200).send(data);
}, err => {
console.log(err);
res.sendStatus(500);
});
UpdateItemInDDB is just a function which takes a params and run dnamodb code on it. I am using javascript sdk for DynamoDB with Document Client.

var params = {
TableName: "user",
Key: {
"user_id": {
S: user_id
}
},
UpdateExpression: "SET #ri = list_append(if_not_exists(#ri, :empty_list), :vals)",
ExpressionAttributeNames: {
"#ri": "images"
},
ExpressionAttributeValues: {
":vals": {"L": [{
"S": "dummy data"
}]},
":empty_list": {"L": []}
},
ReturnValues: 'ALL_NEW'
};
":empty_list": {"L": []}
this is important if you want to set empty list, otherwise it will give below exception.
"errorMessage": "ExpressionAttributeValues contains invalid value: Supplied AttributeValue is empty, must contain exactly one of the supported datatypes for key :empty_list",
"errorType": "ValidationException"

EDIT: Nest the conditional expressions
You could run SET append_list with a ConditionalExpression that the attribute does exist, then if that fails run SET with a ConditinalExpression that the attribute does not exist.
let params1 = {
TableName: "product",
ExpressionAttributeNames: {
"#Y": "buyer"
},
ExpressionAttributeValues: {
":y": ["PersonXYZ"]
},
Key: {
id: 'Hy2H4Z-lf'
},
ConditionExpression: "attribute_exists(buyer)",
UpdateExpression: "SET #Y = list_append(#Y,:y)"
};
updateItemInDDB(params1).then((data) => {
res.status(200).send(data);
}, err => {
console.log(err);
let params2 = {
TableName: "product",
ExpressionAttributeNames: {
"#Y": "buyer"
},
ExpressionAttributeValues: {
":y": ["PersonXYZ"]
},
Key: {
id: 'Hy2H4Z-lf'
},
ConditionExpression: "attribute_not_exists(buyer)",
UpdateExpression: "SET #Y = (#Y,:y)"
};
updateItemInDDB(params2).then((data) => {
res.status(200).send(data);
}, err => {
console.log(err);
res.sendStatus(500);
});
});

Sails Js populate don't retrieve all attributes

I have a problem with populate. I made a query to get User, Project and Topic information (Those are my 3 models). I need to show multiples dates in profile view. This is my code:
Project.js:
module.exports = {
attributes: {
name: {
type: "string"
},
topics: {
collection: "topic",
via: "projects"
},
members: {
collection: "user",
via: "projects"
},
content: {
collection: "content",
via: "projectData"
}
}
};
Topic.js:
module.exports = {
attributes: {
name: {
type: "string"
},
projects: {
collection: "project",
via: "topics"
}
}
};
in User.js:
show: function(req, res, next) {
User.findOne({id: req.session.User.id}).populateAll().exec(function prjFound(err, user){
if (err) return next(err);
if (!user) return next();
console.log(user);
res.view({
user: user
});
});
},
Console print this:
{ projects:
[ { name: 'Fran',
createdAt: '2017-06-19T21:33:17.152Z',
updatedAt: '2017-06-19T21:33:17.190Z',
id: 97 },
{ name: 'River Plate',
createdAt: '2017-06-19T21:36:38.757Z',
updatedAt: '2017-06-19T21:36:38.798Z',
id: 98 },
{ name: 'Guido',
createdAt: '2017-06-20T01:33:53.843Z',
updatedAt: '2017-06-20T01:33:53.926Z',
id: 99 } ],
group: [],
mat: 222222,
name: 'Francisco',
lastname: 'P',
email: 'fran#1.com.ar',
encryptedPassword: '$2a$10$nKp/eAOCDPw4BS.PvQCThe42wa2/8ZABw4JzA0no9GPVT4VjFl3ZO',
createdAt: '2017-06-19T21:32:10.535Z',
updatedAt: '2017-06-19T21:32:10.535Z',
id: '594842da6aeecd880ebab4e6'
}
I want to get all atributes of project model (Content, topic, and members), not only the name and id.
Anyone can explain Why my code is wrong?

Sails/Waterline populate/populateAll do 1 level of population. For 2 or deeper level you need to write code for it.
E.g. Gather ids of user's project and do populateAll on Project.find

Sailsjs doesn't currently support population within a populated field. Write a query in the returned response and append it to the field that you want to populate, send the response with your desired results.

Check this.
let result = await model.find(filter).populate("fieldName", {select:['attribute1','attribute1']})