Develop Reference

Map the nested data from other table using promise and async-await

I need the expert advice for this code. I need to know Is there any better way to solve this.
I am using the mongoose for db. I have a dataset like this:
Below is matchTable:
{
_id: 617bc0113176d717f4ddd6ce,
car: [],
status: true
},
{
_id: 617bc0113176d717f4ddd6cg,
car: [
{
aid: '5c1b4ffd18e2d84b7d6febcg',
}
],
status: true
}
And I have a Car table in which car name is there on behalf of id
like this
{ _id: ObjectId('5c1b4ffd18e2d84b7d6febce'), name: 'ford' },
{ _id: ObjectId('5c1b4ffd18e2d84b7d6febcg'), name: 'mitsubishi' },
So I want to make join the data from car table, so that response get name on behalf of aid.
Desired result will be like
{
_id: 617bc0113176d717f4ddd6ce,
car: [],
status: true
},
{
_id: 617bc0113176d717f4ddd6cg,
car: [
{
aid: '5c1b4ffd18e2d84b7d6febcg',
name: 'mitsubishi'
}
],
status: true
}
For that I have to merge the car table on matchTable. I have done this but I want to give some suggestion that is there any better way to do or is it fine. I need expert advice.
const getData = await matchTable.find(
{ status: true }
).lean().exec();
let dataHolder = [];
await Promise.all (
getData.map(async x => {
await Promise.all(
x.car.map(async y => {
let data = await Car.findOne(
{ _id: ObjectId(y.aid) },
{ name: 1 }
).lean().exec();
y.name = '';
if (data) {
y.name = data.name;
}
})
)
// If I return { ...x }, then on response it will return {}, {} on car column
dataHolder.push(x) //So I have chosen this approach
})
);
Please guide me if any better and efficient solution is there. Thanks in advance

You can make use of aggregation here.
const pipeline = [
{
$match : { status : true }
},
{
$unwind: '$matchtable',
},
{
$lookup: {
from: "cars",
localField: "car.aid",
foreignField: "_id",
as: "matchcars"
}
},
{
$addFields: {
"car.carName": { $arrayElemAt: ["$matchcars.name", 0] }
}
},
{
$group: {
_id: "$_id",
cars: { $push: "$matchcars" }
}
}
]
const result = await matchTable.aggregate(pipeline).exec();
Please make sure, aid field inside car array (in matchTable collection) is an ObjectId because its being matched to _id (which is an ObjectId) inside cars collection.

How to check if item exists in MonoDB array?

I have a MongoDB model:
const userSchema = new Schema = ({
name: { type: String },
company: [
companyId: {
type: String,
},
movies: [
{
genre: {
type: String,
enum: [
'horror',
'comedy',
'action',
'romance',
],
},
ratings: { type: String }
},
]
],
})
In my query, I have an endpoint that pushes a genre to the movies array but I want to check if there is an existing genre with the name already, if it exists, I want to show a message that says it already exists, otherwise, push the new items to the movies array
const result = await UserProfile.updateOne(
{
_id: id,
'company.companyId{ $eq: req.params.companyId},
'company.movies.$.genre': {
$eq: { genre: req.body.genre},
},
}
},
{
$push: {
'company.$.movies': {
...model,
},
},
},
{ new: true, runValidators: true }
).catch((err) => handleErrorThrow(err));
if (result.nModified === 0)
throw new CustomError(409, 'Movie exists already');
And if I want to remove the array based on another endpoint, I tried the same thing it doesn't work
const result = await UserProfile.updateOne(
{
_id: id
}
},
{
$pull: {
company: {
$elemMatch: {
companyId: req.params.companyId,
"movies.genre": {
$ne: req.body.genre
}
}
}
},
{ new: true, runValidators: true }
)
.catch((err) => handleErrorThrow(err));
if (result.nModified === 0)
throw new CustomError(409, 'Not exist');
It returned Not exist'

use $elemMatch for nested array condition, and $ne for genre should not exists before push into movies,
const result = await UserProfile.updateOne(
{
_id: id,
company: {
$elemMatch: {
companyId: req.params.companyId,
"movies.genre": {
$ne: req.body.genre
}
}
}
},
{
$push: {
"company.$.movies": model
}
},
{ runValidators: true }
)
.catch((err) => handleErrorThrow(err));
if (result.nModified === 0) {
throw new CustomError(409, 'Movie exists already');
}
And if I want to remove the array based on another endpoint
const result = await UserProfile.updateOne(
{
_id: id,
company: {
$elemMatch: {
companyId: req.params.companyId,
"movies.genre": req.body.genre
}
}
},
{
$pull: {
"company.$.movies": {
genre: req.body.genre
}
}
},
{ runValidators: true }
).catch((err) => handleErrorThrow(err));
if (result.nModified === 0) {
throw new CustomError(409, 'Not exist');
}

Add new field to report - Script - NodeJS and MongoDB

I have to run a report from a script.
The script works fine
Need to add a new field that indicates if the user is admin or not.
I am not sure how to add that with a true or false condition.
Right now it is returning undefined or false.
The field in mongo database is this one:
mongo-database
var orderTotalsByUserId = {};
db.orders.aggregate([{ $match: { completed: { $exists: true }, completed: { $gt : new Date('2020-08-01') } } }, { $group: { _id: { userId : "$userId" }, total : { $sum: "$total"} } }]).forEach( d => orderTotalsByUserId[d._id.userId] = d.total )
var userIdsByOrders = db.orders.aggregate([{ $match: { completed: { $exists: true }, completed: { $gt : new Date('2020-08-01') } } }, { $group: { _id: { userId : "$userId" }, total : { $sum: "$total"} } }]).map( d => ObjectId(d._id.userId) )
var customers = {};
db.organizations.find().forEach(c => { customers[c._id.valueOf()] = c });
db.users.find({ _id: { $in: userIdsByOrders } }).forEach(d => print(`${d.firstName}, ${d.lastName}, ${d.profile.email}, ${customers[d.customerId].name},${d.sensitive.active.globalAdmin}, ${(orderTotalsByUserId[d._id.valueOf()] * .01).toFixed(2)}`) )

After some time, I've been able to solve it.
Hope it can be helpful for someone else.
First filtering the featurePermissions field in the pipeline.
Then, created an if statement, indicating that if the user has a Customer, it should print the following values.
The solution was:
var orderTotalsByUserId = {};
db.orders.aggregate([{ $match: { completed: { $exists: true }, completed: { $gt : new Date('2020-08-01') } } }, { $group: { _id: { userId : "$userId" }, total : { $sum: "$total"} } }]).forEach( d => orderTotalsByUserId[d._id.userId] = d.total );
var customers = {};
db.organizations.find().forEach(c => { customers[c._id.valueOf()] = c });
db.users.find({ featurePermissions: { $exists: true } }, { _id: 1, firstName: 1, lastName: 1, profile: 1, customerId: 1, featurePermissions: 1 }).forEach(d => { if (customers[d.customerId]) { print(`${d.firstName}, ${d.lastName}, ${d.profile.email}, ${customers[d.customerId].name}, ${orderTotalsByUserId[d._id.valueOf()] ? (orderTotalsByUserId[d._id.valueOf()] * .01).toFixed(2) : 0 }, ${d.featurePermissions.user_management ? 'true' : 'false' }`)}});

Implement feed with retweets in MongoDB

I want to implement retweet feature in my app. I use Mongoose and have User and Message models, and I store retweets as array of objects of type {userId, createdAt} where createdAt is time when retweet occurred. Message model has it's own createdAt field.
I need to create feed of original and retweeted messages merged together based on createdAt fields. I am stuck with merging, whether to do it in a single query or separate and do the merge in JavaScript. Can I do it all in Mongoose with a single query? If not how to find merge insertion points and index of the last message?
So far I just have fetching of original messages.
My Message model:
const messageSchema = new mongoose.Schema(
{
fileId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'File',
required: true,
},
userId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
required: true,
},
likesIds: [{ type: mongoose.Schema.Types.ObjectId, ref: 'User' }],
reposts: [
{
reposterId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
createdAt: { type: Date, default: Date.now },
},
],
},
{
timestamps: true,
},
);
Edit: Now I have this but pagination is broken. I am trying to use newCreatedAt field for cursor, that doesn't seem to work. It returns empty array in second call when newCreatedAt is passed from the frontend.
messages: async (
parent,
{ cursor, limit = 100, username },
{ models },
) => {
const user = username
? await models.User.findOne({
username,
})
: null;
const options = {
...(cursor && {
newCreatedAt: {
$lt: new Date(fromCursorHash(cursor)),
},
}),
...(username && {
userId: mongoose.Types.ObjectId(user.id),
}),
};
console.log(options);
const aMessages = await models.Message.aggregate([
{
$addFields: {
newReposts: {
$concatArrays: [
[{ createdAt: '$createdAt', original: true }],
'$reposts',
],
},
},
},
{
$unwind: '$newReposts',
},
{
$addFields: {
newCreatedAt: '$newReposts.createdAt',
original: '$newReposts.original',
},
},
{ $match: options },
{
$sort: {
newCreatedAt: -1,
},
},
{
$limit: limit + 1,
},
]);
const messages = aMessages.map(m => {
m.id = m._id.toString();
return m;
});
//console.log(messages);
const hasNextPage = messages.length > limit;
const edges = hasNextPage ? messages.slice(0, -1) : messages;
return {
edges,
pageInfo: {
hasNextPage,
endCursor: toCursorHash(
edges[edges.length - 1].newCreatedAt.toString(),
),
},
};
},
Here are the queries. The working one:
Mongoose: messages.aggregate([{
'$match': {
createdAt: {
'$lt': 2020 - 02 - 02 T19: 48: 54.000 Z
}
}
}, {
'$sort': {
createdAt: -1
}
}, {
'$limit': 3
}], {})
And the non working one:
Mongoose: messages.aggregate([{
'$match': {
newCreatedAt: {
'$lt': 2020 - 02 - 02 T19: 51: 39.000 Z
}
}
}, {
'$addFields': {
newReposts: {
'$concatArrays': [
[{
createdAt: '$createdAt',
original: true
}], '$reposts'
]
}
}
}, {
'$unwind': '$newReposts'
}, {
'$addFields': {
newCreatedAt: '$newReposts.createdAt',
original: '$newReposts.original'
}
}, {
'$sort': {
newCreatedAt: -1
}
}, {
'$limit': 3
}], {})

This can be done in one query, although its a little hack-ish:
db.collection.aggregate([
{
$addFields: {
reposts: {
$concatArrays: [[{createdAt: "$createdAt", original: true}],"$reports"]
}
}
},
{
$unwind: "$reposts"
},
{
$addFields: {
createdAt: "$reposts.createdAt",
original: "$reposts.original"
}
},
{
$sort: {
createdAt: -1
}
}
]);
You can add any other logic you want to the query using the original field, documents with original: true are the original posts while the others are retweets.

Mongoose, sort query by populated field

As far as I know, it's possible to sort populated docs with Mongoose (source).
I'm searching for a way to sort a query by one or more populated fields.
Consider this two Mongoose schemas :
var Wizard = new Schema({
name : { type: String }
, spells : { [{ type: Schema.ObjectId, ref: 'Spell' }] }
});
var Spell = new Schema({
name : { type: String }
, damages : { type: Number }
});
Sample JSON:
[{
name: 'Gandalf',
spells: [{
name: 'Fireball',
damages: 20
}]
}, {
name: 'Saruman',
spells: [{
name: 'Frozenball',
damages: 10
}]
}, {
name: 'Radagast',
spells: [{
name: 'Lightball',
damages: 15
}]
}]
I would like to sort those wizards by their spell damages, using something like :
WizardModel
.find({})
.populate('spells', myfields, myconditions, { sort: [['damages', 'asc']] })
// Should return in the right order: Saruman, Radagast, Gandalf
I'm actually doing those sorts by hands after querying and would like to optimize that.

You can explicitly specify only required parameters of populate method:
WizardModel
.find({})
.populate({path: 'spells', options: { sort: [['damages', 'asc']] }})
Have a look at http://mongoosejs.com/docs/api.html#document_Document-populate
Here is an example from a link above.
doc
.populate('company')
.populate({
path: 'notes',
match: /airline/,
select: 'text',
model: 'modelName'
options: opts
}, function (err, user) {
assert(doc._id == user._id) // the document itself is passed
})

Even though this is rather an old post, I'd like to share a solution through the MongoDB aggregation lookup pipeline
The important part is this:
{
$lookup: {
from: 'spells',
localField: 'spells',
foreignField:'_id',
as: 'spells'
}
},
{
$project: {
_id: 1,
name: 1,
// project the values from damages in the spells array in a new array called damages
damages: '$spells.damages',
spells: {
name: 1,
damages: 1
}
}
},
// take the maximum damage from the damages array
{
$project: {
_id: 1,
spells: 1,
name: 1,
maxDamage: {$max: '$damages'}
}
},
// do the sorting
{
$sort: {'maxDamage' : -1}
}
Find below a complete example
'use strict';
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
mongoose.connect('mongodb://localhost/lotr');
const db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', () => {
let SpellSchema = new Schema({
name : { type: String },
damages : { type: Number }
});
let Spell = mongoose.model('Spell', SpellSchema);
let WizardSchema = new Schema({
name: { type: String },
spells: [{ type: Schema.Types.ObjectId, ref: 'Spell' }]
});
let Wizard = mongoose.model('Wizard', WizardSchema);
let fireball = new Spell({
name: 'Fireball',
damages: 20
});
let frozenball = new Spell({
name: 'Frozenball',
damages: 10
});
let lightball = new Spell({
name: 'Lightball',
damages: 15
});
let spells = [fireball, frozenball, lightball];
let wizards = [{
name: 'Gandalf',
spells:[fireball]
}, {
name: 'Saruman',
spells:[frozenball]
}, {
name: 'Radagast',
spells:[lightball]
}];
let aggregation = [
{
$match: {}
},
// find all spells in the spells collection related to wizards and fill populate into wizards.spells
{
$lookup: {
from: 'spells',
localField: 'spells',
foreignField:'_id',
as: 'spells'
}
},
{
$project: {
_id: 1,
name: 1,
// project the values from damages in the spells array in a new array called damages
damages: '$spells.damages',
spells: {
name: 1,
damages: 1
}
}
},
// take the maximum damage from the damages array
{
$project: {
_id: 1,
spells: 1,
name: 1,
maxDamage: {$max: '$damages'}
}
},
// do the sorting
{
$sort: {'maxDamage' : -1}
}
];
Spell.create(spells, (err, spells) => {
if (err) throw(err);
else {
Wizard.create(wizards, (err, wizards) =>{
if (err) throw(err);
else {
Wizard.aggregate(aggregation)
.exec((err, models) => {
if (err) throw(err);
else {
console.log(models[0]); // eslint-disable-line
console.log(models[1]); // eslint-disable-line
console.log(models[2]); // eslint-disable-line
Wizard.remove().exec(() => {
Spell.remove().exec(() => {
process.exit(0);
});
});
}
});
}
});
}
});
});

here's the sample of mongoose doc.
var PersonSchema = new Schema({
name: String,
band: String
});
var BandSchema = new Schema({
name: String
});
BandSchema.virtual('members', {
ref: 'Person', // The model to use
localField: 'name', // Find people where `localField`
foreignField: 'band', // is equal to `foreignField`
// If `justOne` is true, 'members' will be a single doc as opposed to
// an array. `justOne` is false by default.
justOne: false,
options: { sort: { name: -1 }, limit: 5 }
});