Develop Reference

Mongoose - Best way to get followers posts

I am building a social media and one of the features I have is the ability to follow other users and have their posts show up on a home page, much like an Instagram feed. I have an endpoint that loops through all the users you are following and gets the post that are rated for your age, sorts them and puts them in an array.
router.get("/following", auth, async (req, res) => {
try {
const user = await User.findOne({ firebaseUID: req.authId });
const dob = dayjs(user.dob.toISOString().split("T")[0]);
const currentDate = dayjs(new Date().toISOString().split("T")[0]);
const age1 = currentDate.diff(dob, "year");
let posts = [];
for (let i = 0; i < user.following.length; i++) {
posts.push(
await Post.find({
userID: user.following[i].user,
age: { $lte: age1 },
})
);
}
posts = [].concat.apply([], posts);
posts = _.sortBy(posts, "date").reverse();
res.json(posts);
} catch (err) {
console.error(err.message);
res.status(500).json({ msg: "Server Error" });
}
});
The user schema looks like this
const UserSchema = new Schema({
firebaseUID: {
type: String,
required: true,
},
name: {
type: String,
required: true,
},
username: {
type: String,
required: true,
unique: true,
},
avatar: {
type: String,
},
dob: {
type: Date,
default: Date.now,
required: true,
},
slug: {
type: String,
},
following: [
//array of user IDs
{
user: {
type: Schema.Types.ObjectId,
ref: "users", //here so we know which lieks came from which users
},
},
],
followers: [
//array of user IDs
{
user: {
type: Schema.Types.ObjectId,
ref: "user", //here so we know which lieks came from which users
},
},
],
});
The problem is that I am trying to add infinite scroll pagination to the front end that will start by getting the first 15 posts and then get the next 15 and so on from endpoints that provide this.
What would the best way to do this be? For other routes, I have used .skip() and .limit(). I could split the array of posts up right before I send it as a response, but is there a way to do without getting all the posts as this would be taxing if you're following 100s of people with 1000s of posts.
I would need to loop through all the users you are following and get all their posts in line with your age rating but only get the first 15 sorted in date order from newest to oldest. Is there a mongoose function that I could use or would the best way be to split the array of posts?
Thank you.

MongoDB Aggregate is not matching specific field

I'm new to Aggregation in MongoDB and I'm trying to understand the concepts of it by making examples.
I'm trying to paginate my subdocuments using aggregation but the returned document is always the overall values of all document's specific field.
I want to paginate my following field which contains an array of Object IDs.
I have this User Schema:
const UserSchema = new mongoose.Schema({
username: {
type: String,
unique: true,
required: true
},
firstname: String,
lastname: String,
following: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}],
...
}, { timestamps: true, toJSON: { virtuals: true }, toObject: { getters: true, virtuals: true } });
Without aggregation, I am able to paginate following,
I have this route which gets the user's post by their username
router.get(
'/v1/:username/following',
isAuthenticated,
async (req, res, next) => {
try {
const { username } = req.params;
const { offset: off } = req.query;
let offset = 0;
if (typeof off !== undefined && !isNaN(off)) offset = parseInt(off);
const limit = 2;
const skip = offset * limit;
const user = await User
.findOne({ username })
.populate({
path: 'following',
select: 'profilePicture username fullname',
options: {
skip,
limit,
}
})
res.status(200).send(user.following);
} catch (e) {
console.log(e);
res.status(500).send(e)
}
}
);
And my pagination version using aggregate:
const following = await User.aggregate([
{
$match: { username }
},
{
$lookup: {
'from': User.collection.name,
'let': { 'following': '$following' },
'pipeline': [
{
$project: {
'fullname': 1,
'username': 1,
'profilePicture': 1
}
}
],
'as': 'following'
},
}, {
$project: {
'_id': 0,
'following': {
$slice: ['$following', skip, limit]
}
}
}
]);
Suppose I have this documents:
[
{
_id: '5fdgffdgfdgdsfsdfsf',
username: 'gagi',
following: []
},
{
_id: '5fgjhkljvlkdsjfsldkf',
username: 'kuku',
following: []
},
{
_id: '76jghkdfhasjhfsdkf',
username: 'john',
following: ['5fdgffdgfdgdsfsdfsf', '5fgjhkljvlkdsjfsldkf']
},
]
And when I test my route for user john: /john/following, everything is fine but when I test for different user which doesn't have any following: /gagi/following, the returned result is the same as john's following which aggregate doesn't seem to match user by username.
/john/following | following: 2
/kuku/following | following: 0
Aggregate result:
[
{
_id: '5fdgffdgfdgdsfsdfsf',
username: 'kuku',
...
},
{
_id: '5fgjhkljvlkdsjfsldkf',
username: 'gagi',
...
}
]
I expect /kuku/following to return an empty array [] but the result is same as john's. Actually, all username I test return the same result.
I'm thinking that there must be wrong with my implementation since I've only started exploring aggregation.

Mongoose uses a DBRef to be able to populate the field after it has been retrieved.
DBRefs are only handled on the client side, MongoDB aggregation does not have any operators for handling those.
The reason that aggregation pipeline is returning all of the users is the lookup's pipeline does not have a match stage, so all of the documents in the collection are selected and included in the lookup.
The sample document there is showing an array of strings instead of DBRefs, which wouldn't work with populate.
Essentially, you must decide whether you want to use aggregation or populate to handle the join.
For populate, use the ref as shown in that sample schema.
For aggregate, store an array of ObjectId so you can use lookup to link with the _id field.

How can I use MongoDB to find documents that match with custom field?

I am trying to find all documents that match with custom field in node.js.
node.js code:
req.app.db.models.Property.find({
user: {
id: req.params.id
}
}).exec(function(err, user) {
if (err) {
return next(err);
}
console.log("id:" + req.params.id);
console.log("user:" + user);
res.status(200).json(user);
});
But, console shows like this
id:5941cfc42df14b2fe811d531
user:
And the schema is like below
user: {
id: { type: mongoose.Schema.Types.ObjectId, ref: 'User' },
name: { type: String, default: '' },
email: { type: String, defaul: ''}
},
propertyType: { type: String, default: '' },
..........
}
It seems that it can't find documents.
What's wrong with this? Several documents exist on Property Collection.

The problem is in your query, its finding a user having an object id only.
So what you are actually equating is "user" with "{ id: req.params.id }" which return only id parameter. So for your solution do following:
var query = {'user.id':req.params.id}
Property.find(query).exec(function(err, user) {
if (err) {
return next(err);
}
if you want to specify the type of object returned, if found.
You can mention it as follows:
Property.find(query,'user.id user.name').exec(...)
this will return an object with user id and email only and not the name.

How to write a mongoose query to filter subdocuments

I have a Mongoose schema called "Users" which has a "Roles" subdocument as one of its variables like so:
var UserSchema = new mongoose.Schema({
email: { type: String, required: true, unique: true },
password: { type: String, required: true },
roles: [ { type: Number, ref: 'Role' } ]
});
var RoleSchema = new mongoose.Schema({
_id: Number,
name: { type: String, required: true, unique: true },
description: { type: String, required: true }
});
I want to create a Mongoose query that will find all users with roles.names of "admin" or "owner". I've tried using this query, which I thought would work, but I don't get any users when I use the where...in part.
var roles = ["owner", "admin"];
User.find()
.populate('roles')
.where('roles.name').in(roles)
.sort({'_id': 1})
.exec(function (err, users) {
res.send(users);
});
Can someone tell me where my logic is wrong?

It's not possible to query more than one collection with a single query. If Role was an embedded subdocument in User then you could do a query on roles.name but presently that's not supported because it's a separate collection that is referenced in User.
However, a workaround is to add another filter step after the query returns that manually filters out documents which don't have any roles that matched the populate criteria but first you need to use a match query in populate instead of where method:
var roles = ["owner", "admin"];
User.find()
.populate('roles', null, { name: { $in: roles } } )
.sort({'_id': 1})
.exec(function (err, users) {
users = users.filter(function(user){
return user.roles.length;
});
res.send(users);
});

This is built into Mongoose populate() here . You can simply structure a query like this:
var roles = ["owner", "admin"];
User.find()
.populate({
path: 'roles',
match: { name: { $in: roles }},
select: 'name'
})
.sort({'_id': 1})
.exec(function (err, users) {
res.send(users);
});

using populate to display username instead of id

I'm running Expressjs with mongoosejs I made the connection between the collections CustomerId as below:
.
.
/**
* Customer Schema
*/
var CustomerSchema = new Schema({
id : Number,
name: String,
joined: { type: Date, default: Date.now },
city: String
});
mongoose.model('Customer', CustomerSchema);
.
.
/**
* Order Schema
*/
var OrderSchema = new Schema({
id : Number,
products: [Schema.Types.Mixed],
total: Number,
comments: String,
customerId: {type: Number, ref: 'Customer'}
});
mongoose.model('Order', OrderSchema);
.
.
exports.customerOrders = function (req, res) {
return Order.find({customerId: req.params.customerId}, function (err, orders) {
Order.populate(orders, {path: 'customerId', model: 'Order'}, function (err, orders) {
if (!err) {
return res.json(orders);
} else {
return res.send(err);
}
});
});
};
the above code generate the following error:
{
message: "Cast to ObjectId failed for value "1" at path "_id"",
name: "CastError",
type: "ObjectId",
value: 1,
path: "_id"
}
the relation between objects is id no _id
Please help me to use the populate method in the right way.
Thanx,

Mongoose's populate functionality only supports using the _id field to find the related doc in the referenced collection.
So you can't use another field like id and you'd need to change customerId to be:
customerId: {type: ObjectId, ref: 'Customer'}
in OrderSchema and then populate it with the _id value of the customer instead.