My model is the following :
const scheduleTaskSchema = new Schema({
activity: { type: Object, required: true },
date: { type: Date, required: true },
crew: Object,
vehicle: Object,
pickups: Array,
details: String,
});
pickups is an array of objects with the following structure :
{
time : //iso date,
...rest //irrelevant
}
i want to return my data grouped by date,and also every group within has to be sorted by pickup[0].time so it will finally return something like this example :
{
"2022-08-28T00:00:00.000+00:00": [
{
date: "2022-08-28T00:00:00.000+00:00",
pickups: [
{
time: "2022-08-28T07:30:00.000Z",
...rest //irrelevant
}
],
...rest //irrelevant
},
{
date: "2022-08-28T00:00:00.000+00:00",
pickups: [
{
time: "2022-08-28T09:30:00.000Z",
...rest //irrelevant
}
],
...rest //irrelevant
}
]
,
"2022-08-29T00:00:00.000+00:00": [
{
date: "2022-08-29T00:00:00.000+00:00",
pickups: [
{
time: "2022-08-29T10:00:00.000Z",
...rest //irrelevant
}
],
...rest //irrelevant
},
{
date: "2022-08-29T00:00:00.000+00:00",
pickups: [
{
time: "2022-08-29T11:30:00.000Z",
...rest //irrelevant
}
],
...rest //irrelevant
}
]
}
You need to use the aggregation framework for this.
Assuming the dates are exactly the same (down to the millisecond), your code would look something like this.
const scheduledTasksGroups = await ScheduledTask.aggregate([{
$group: {
_id: '$date',
scheduledTasks: { $push: '$$ROOT' }
}
}]);
The output will be something like:
[
{ _id: "2022-08-29T10:00:00.000Z", scheduledTasks: [...] },
{ _id: "2022-08-28T10:00:00.000Z", scheduledTasks: [...] }
]
If you want to group by day instead of by millisecond, your pipeline would look like this:
const scheduledTasksGroups = await ScheduledTask.aggregate([{
$group: {
// format date first to `YYYY-MM-DD`, then group by the new format
_id: { $dateToString: { format: '%Y-%m-%d', date: '$date' } },
scheduledTasks: { $push: '$$ROOT' }
}
}]);
For what it's worth, this is a MongoDB feature, the grouping happens on the MongoDB server side; mongoose doesn't do anything special here; it just sends the command to the server. Then the server is responsible for grouping the data and returning them back.
Also, keep in mind that mongoose does not cast aggregation pipelines by default, but this plugin makes mongoose cast automatically whenever possible.
Related
So I tried several ways, but I can't, I can modify several objects with the same key but I can't modify any with different keys, if anyone can help me is quite a complex problem
{
id: 123,
"infos": [
{ name: 'Joe', value: 'Disabled', id: 0 },
{ name: 'Adam', value: 'Enabled', id: 0 }
]
};
In my database I have a collection with an array and several objects inside which gives this.
I want to modify these objects, filter by their name and modify the value.
To give you a better example, my site returns me an object with the new data, and I want to modify the database object with the new object, without clearing the array, the name key never changes.
const object = [
{ name: 'Joe', value: 'Hey', id: 1 },
{ name: 'Adam', value: 'None', id: 1 }
];
for(const obj in object) {
Schema.findOneAndUpdate({ id: 123 }, {
$set: {
[`infos.${obj}.value`]: "Test"
}
})
}
This code works but it is not optimized, it makes several requests, I would like to do everything in one request, and also it doesn't update the id, only the value.
If anyone can help me that would be great, I've looked everywhere and can't find anything
My schema structure
new Schema({
id: { "type": String, "required": true, "unique": true },
infos: []
})
I use the $addToSet method to insert objects into the infos array
Try This :
db.collection.update({
id: 123,
},
{
$set: {
"infos.$[x].value": "Value",
"infos.$[x].name": "User"
}
},
{
arrayFilters: [
{
"x.id": {
$in: [
1
]
}
},
],
multi: true
})
The all positional $[] operator acts as a placeholder for all elements in the array field.
In $in you can use dynamic array of id.
Ex :
const ids = [1,2,..n]
db.collection.update(
//Same code as it is...
{
arrayFilters: [
{
"x.id": {
$in: ids
}
},
],
multi: true
})
MongoPlayGround Link : https://mongoplayground.net/p/Tuz831lkPqk
Maybe you look for something like this:
db.collection.update({},
{
$set: {
"infos.$[x].value": "test1",
"infos.$[x].id": 10,
"infos.$[y].value": "test2",
"infos.$[y].id": 20
}
},
{
arrayFilters: [
{
"x.name": "Adam"
},
{
"y.name": "Joe"
}
],
multi: true
})
Explained:
You define arrayFilters for all names in objects you have and update the values & id in all documents ...
playground
I have a large MongoDB dataset of around 34gb and I am using Fastify and Mongoose for the API. I want to retrieve all list of unique userUuid from the date range. I tried the distinct method from Mongoose:
These are my filters:
let filters = {
applicationUuid: opts.applicationUuid,
impressions: {
$gte: opts.impressions
},
date: {
$gte: moment(opts.startDate).tz('America/Chicago').format(),
$lt: moment(opts.endDate).tz('America/Chicago').format()
}
}
This is my distinct Mongoose function:
return await Model.distinct("userUuid", filters)
This method will return an array with unique userUuid based from the filters.
This works fine for small dataset, but it has a memory cap of 16MB when it comes to huge dataset.
Therefore, I tried the aggregate method to achieve similar results, having read that it is better optimized. Nevertheless, the same filters object above does not work inside the match pipeline because aggregate does not accept string date that comes as the result of moment; but only JavaScript Date is accepted. However, JavaScript date dissregards all the timezones since it is unix based.
This is my aggregate function to get distinct values based on filters.
return await Model.aggregate(
[
{
$match: filters
},
{
$group: {
_id: {userUuid: "$userUuid" }
}
}
]
).allowDiskUse(true);
As I said, $match does not work with moment, but only with new Date(opts.startDate), however, JavaScript's new Date disregards moment's timezone. Nor it has a proper native timezone. Any thought on how to achieve this array of unique ids based on filters with Mongoose?
This is the solution I came up with and it works pretty well regarding the performance. Use this solution for large dataset:
let filters = {
applicationUuid: opts.applicationUuid,
impressions: { $gte: opts.impressions },
$expr: {
$and: [
{
$gte: [
'$date',
{
$dateFromString: {
dateString: opts.startDate,
timezone: 'America/Chicago',
},
},
],
},
{
$lt: [
'$date',
{
$dateFromString: {
dateString: opts.endDate,
timezone: 'America/Chicago',
},
},
],
},
],
},
}
return Model.aggregate([
{ $match: filters },
{
$group: {
_id: '$userUuid',
},
},
{
$project: {
_id: 0,
userUuid: '$_id',
},
},
])
.allowDiskUse(true)
Which will return a list of unique ids i.e.
[
{ userUuid: "someId" },
{ userUuid: "someId" }
]
Use the following method on small dataset which is more convenient:
let filters = {
applicationUuid: opts.applicationUuid,
impressions: {
$gte: opts.impressions
},
date: {
$gte: opts.startDate,
$lte: opts.endDate
}
}
return Model.distinct("userUuid", filters)
Which will return the following result:
[ "someId", "someOtherId" ]
I need the expert advice for this code. I need to know Is there any better way to solve this.
I am using the mongoose for db. I have a dataset like this:
Below is matchTable:
{
_id: 617bc0113176d717f4ddd6ce,
car: [],
status: true
},
{
_id: 617bc0113176d717f4ddd6cg,
car: [
{
aid: '5c1b4ffd18e2d84b7d6febcg',
}
],
status: true
}
And I have a Car table in which car name is there on behalf of id
like this
{ _id: ObjectId('5c1b4ffd18e2d84b7d6febce'), name: 'ford' },
{ _id: ObjectId('5c1b4ffd18e2d84b7d6febcg'), name: 'mitsubishi' },
So I want to make join the data from car table, so that response get name on behalf of aid.
Desired result will be like
{
_id: 617bc0113176d717f4ddd6ce,
car: [],
status: true
},
{
_id: 617bc0113176d717f4ddd6cg,
car: [
{
aid: '5c1b4ffd18e2d84b7d6febcg',
name: 'mitsubishi'
}
],
status: true
}
For that I have to merge the car table on matchTable. I have done this but I want to give some suggestion that is there any better way to do or is it fine. I need expert advice.
const getData = await matchTable.find(
{ status: true }
).lean().exec();
let dataHolder = [];
await Promise.all (
getData.map(async x => {
await Promise.all(
x.car.map(async y => {
let data = await Car.findOne(
{ _id: ObjectId(y.aid) },
{ name: 1 }
).lean().exec();
y.name = '';
if (data) {
y.name = data.name;
}
})
)
// If I return { ...x }, then on response it will return {}, {} on car column
dataHolder.push(x) //So I have chosen this approach
})
);
Please guide me if any better and efficient solution is there. Thanks in advance
You can make use of aggregation here.
const pipeline = [
{
$match : { status : true }
},
{
$unwind: '$matchtable',
},
{
$lookup: {
from: "cars",
localField: "car.aid",
foreignField: "_id",
as: "matchcars"
}
},
{
$addFields: {
"car.carName": { $arrayElemAt: ["$matchcars.name", 0] }
}
},
{
$group: {
_id: "$_id",
cars: { $push: "$matchcars" }
}
}
]
const result = await matchTable.aggregate(pipeline).exec();
Please make sure, aid field inside car array (in matchTable collection) is an ObjectId because its being matched to _id (which is an ObjectId) inside cars collection.
How can I retrieve data with a custom sort in Mongoose?
There is a job starting date that needs to be sorted by the month and year, but currently this script is only sorting from December to January.
router.get('/', (req, res) => {
Job.find()
.sort({ from: -1 })
.then(jobs => res.json(jobs))
.catch(err => res.status(404).json(err));
});
The problem is in the sort; values for from is like 12.2018, 06.2019, 03.2020, 11.2009 and so on.
I want to sort these results first from the year (which is after the dot) and then sort from the months. I cannot currently change how the data is set and it's stored as a String in the model Schema.
You have to use aggregation framework to first transform your string to a valid date by
$spliting it,
$convert parts from string to int
and using $dateFromParts,
then you sort and finally remove created field.
Here's the query :
db.collection.aggregate([
{
$addFields: {
date: {
$dateFromParts: {
year: {
$convert: {
input: {
$arrayElemAt: [
{
$split: [
"$from",
"."
]
},
1
]
},
to: "int"
}
},
month: {
$convert: {
input: {
$arrayElemAt: [
{
$split: [
"$from",
"."
]
},
0
]
},
to: "int"
}
},
}
}
}
},
{
$sort: {
date: -1
}
},
{
$project: {
date: 0
}
}
])
You can test it here
I want to implement retweet feature in my app. I use Mongoose and have User and Message models, and I store retweets as array of objects of type {userId, createdAt} where createdAt is time when retweet occurred. Message model has it's own createdAt field.
I need to create feed of original and retweeted messages merged together based on createdAt fields. I am stuck with merging, whether to do it in a single query or separate and do the merge in JavaScript. Can I do it all in Mongoose with a single query? If not how to find merge insertion points and index of the last message?
So far I just have fetching of original messages.
My Message model:
const messageSchema = new mongoose.Schema(
{
fileId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'File',
required: true,
},
userId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
required: true,
},
likesIds: [{ type: mongoose.Schema.Types.ObjectId, ref: 'User' }],
reposts: [
{
reposterId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
},
createdAt: { type: Date, default: Date.now },
},
],
},
{
timestamps: true,
},
);
Edit: Now I have this but pagination is broken. I am trying to use newCreatedAt field for cursor, that doesn't seem to work. It returns empty array in second call when newCreatedAt is passed from the frontend.
messages: async (
parent,
{ cursor, limit = 100, username },
{ models },
) => {
const user = username
? await models.User.findOne({
username,
})
: null;
const options = {
...(cursor && {
newCreatedAt: {
$lt: new Date(fromCursorHash(cursor)),
},
}),
...(username && {
userId: mongoose.Types.ObjectId(user.id),
}),
};
console.log(options);
const aMessages = await models.Message.aggregate([
{
$addFields: {
newReposts: {
$concatArrays: [
[{ createdAt: '$createdAt', original: true }],
'$reposts',
],
},
},
},
{
$unwind: '$newReposts',
},
{
$addFields: {
newCreatedAt: '$newReposts.createdAt',
original: '$newReposts.original',
},
},
{ $match: options },
{
$sort: {
newCreatedAt: -1,
},
},
{
$limit: limit + 1,
},
]);
const messages = aMessages.map(m => {
m.id = m._id.toString();
return m;
});
//console.log(messages);
const hasNextPage = messages.length > limit;
const edges = hasNextPage ? messages.slice(0, -1) : messages;
return {
edges,
pageInfo: {
hasNextPage,
endCursor: toCursorHash(
edges[edges.length - 1].newCreatedAt.toString(),
),
},
};
},
Here are the queries. The working one:
Mongoose: messages.aggregate([{
'$match': {
createdAt: {
'$lt': 2020 - 02 - 02 T19: 48: 54.000 Z
}
}
}, {
'$sort': {
createdAt: -1
}
}, {
'$limit': 3
}], {})
And the non working one:
Mongoose: messages.aggregate([{
'$match': {
newCreatedAt: {
'$lt': 2020 - 02 - 02 T19: 51: 39.000 Z
}
}
}, {
'$addFields': {
newReposts: {
'$concatArrays': [
[{
createdAt: '$createdAt',
original: true
}], '$reposts'
]
}
}
}, {
'$unwind': '$newReposts'
}, {
'$addFields': {
newCreatedAt: '$newReposts.createdAt',
original: '$newReposts.original'
}
}, {
'$sort': {
newCreatedAt: -1
}
}, {
'$limit': 3
}], {})
This can be done in one query, although its a little hack-ish:
db.collection.aggregate([
{
$addFields: {
reposts: {
$concatArrays: [[{createdAt: "$createdAt", original: true}],"$reports"]
}
}
},
{
$unwind: "$reposts"
},
{
$addFields: {
createdAt: "$reposts.createdAt",
original: "$reposts.original"
}
},
{
$sort: {
createdAt: -1
}
}
]);
You can add any other logic you want to the query using the original field, documents with original: true are the original posts while the others are retweets.