Develop Reference

Mongoose how to count unique days in a month?

i am working on a problem where in my database there supposed to be multiple entries from day to day.
each entry includes a timestamp. the problem is im supposed to find how many days in each month a data has been entered.
ex: if a user sends a message every 2 days in may. my answer would be: "the user used the message function for 15 days in may".
now if the user sends 5 messages every 2 days in may. the answer would be still 15. im only counting the days the user has been using the app.
using this query:
model.find({
date: {
$gte: new Date(startOfMonth),
$lt: new Date(endOfMonth)
}
})
i was able to find all data entries on a specific month.
the data may look like something like this:
Date: dd/mm/yy Message:
1/5/2022 "Hello"
1/5/2022 "World"
1/5/2022 "!"
5/5/2022 "Hello World!"
8/5/2022 "Hello World!"
the desired answer would be 3 unique days in may.
How do i achieve this using mongodb? the simplest answer that come to mind is to use additional queries to group by unique days, but i have no idea how to do that using mongo provided i need to access the date.

This might solves your problem. it will return distinct messages.
Model.aggregate(
[
{ "$match": {
"date": {
"$gte": new Date(startOfMonth), "$lt": new Date(endOfMonth)
}
}},
{ "$group": {
"_id": {
"year": { "$year": "$date" },
"month": { "$month": "$date" },
"day": { "$dayOfMonth": "$date" }
}
"count": { "$sum": 1 }
}}
],
function(err,result) {
// do something with result
}
);

You can use distinct
db.collection.distinct({date:{$gte:new Date(startOfMonth),$lt:new Date(endOfMonth)}}).length
if you are directly storing the date of it.
Note : This may not work if you're storing complete timestamp instead of date.

Subtract days from a timestamp doesn't work with MongoDB

I still can't seem to do what I want to do. My Data Base:
{lastSeen: { $date: {$numberLong: 1614618000000 }}}
I want substract days to value and selects the documents where the value of the field is greater than or equal. I test this:
{"lastSeen": { "$gte": {"$date": { "$subtract": [ "$date", 1616000000000 ] }}}}
but the response is 'no result to show' on Gamesparks.
When i test with this query:
{"lastSeen": { "$gte": {"$date": "2021-03-13T00:00:00.000Z"}}}
i have responses.
An idea ? Thank you.

Query Last Matching Date in Array

I need to find all datasets in my mongoDB with an expired date value. Expired means, that the last array element timestamp is older then the current timestamp plus a defined interval (which is defined by a category)
Every dataset has a field like this
{
"field" : [
{
"category" : 1,
"date" : ISODate("2019-03-01T12:00:00.464Z")
},
{
"category" : 1,
"date" : ISODate("2019-03-01T14:52:50.464Z")
}
]
}
The category defines a time interval. For example 'category 1' stands for 90 minutes, 'category 2' for 120 minutes.
Now I need to get every dataset with a date value which is expired, which means the last array element has a value which is older then 90 minutes before the current timestamp.
Something like
Content.find({ 'field.$.date': { $gt: new Date() } })
But with that attempt I've two problems:
How do I query for the last array element?
How to implement the category time interval in the query?

Let's break down the problem into parts.
Query the "last" ( most recent ) array element
Part 1: Logical and Fast
A quick perusal of MongoDB query operators related to arrays should tell you that you can in fact always query an array element based on the index position. This is very simple to do for the "first" array element since that position is always 0:
{ "field.0.date": { "$lt": new Date("2019-03-01T10:30:00.464Z") } }
Logically the "last" position would be -1, but you cannot actually use that value in notation of this form with MongoDB as it would be considered invalid.
However what you can do here instead is add new items to the array in a way so that rather than appending to the end of the array, you actually prepend to the beginning of the array. This means your array content is essentially "reversed" and it's then easy to access as shown above. This is what the $position modifier to $push does for you:
collection.updateOne(
{ "_id": documentId },
{
"$push": {
"field": {
"$each": [{ "category": 1, "date": new Date("2019-03-02") }],
"$position": 0
}
}
}
)
So that means newly added items go to the beginning rather than the end. That may be practical but it does mean you would need to re-order all your existing array items.
In the case where the "date" is static and basically never changes once you write the array item ( i.e you never update the date for a matched array item ) then you can actually re-order sorting on that "date" property in a single update statement, using the $sort modifier:
collection.updateMany(
{},
{ "$push": { "field": { "$each": [], "$sort": { "date": -1 } } } }
)
Whilst it might feel "odd" to use $push when you are not actually adding anything to the array, this is where the $sort modifier lives. The empty array "$each": [] argument essentially means "add nothing" yet the $sort applies to all current members of the the array.
This could optionally be done much like the earlier example with $position, in which the $sort would be applied on every write. However as long as the "date" applies to the "timestamp when added" ( as I suspect it does ) then it's probably more efficient to use the "$position": 0 approach instead of sorting every time something changes. Depends on your actual implementation and how you otherwise work with the data.
Part 2: Brute force, and slow
If however for whatever reason you really don't believe that being able to "reverse" the content of the array is a practical solution, then the only other available thing is to effectively "calculate" the "last" array element by projecting this value from a supported operator.
The only practical way to do that is typically with the Aggregation Framework and specifically the $arrayElemAt operator:
collection.aggregate([
{ "$addFields": {
"lastDate": { "$arrayElemAt": [ "$field.date", -1 ] }
}}
])
Basically that is just going to look at the supplied array content ( in this case just the "date" property values for each element ) and then extract the value at the given index position. This operator allows the -1 index notation, meaning the "last" element in the array.
Clearly this is not ideal as the extraction is decoupled from the actual expression needed to query or filter the values. That's in the next part, but you need to realize this just iterated through your whole collection before we can even look at comparing the values to see which you want to keep.
Sample Date by "Category"
Part 1: Fast query logic
Following on from the above the next criteria is based on the "category" field value, with the next main issues being
90 minutes adjust for value 1
120 minutes adjust for value 2
By the same logic just learned you should conclude that "calculating" as you process data is "bad news" for performance. So the trick to apply here is basically including the logic in the query expression to use different supplied "date" values depending on what the "category" value being matched in the document is.
The most simple application of this is with an $or expression:
var currentDateTime = new Date();
var ninetyMinsBefore = new Date(currentDateTime.valueOf() - (1000 * 60 * 90));
var oneTwentyMinsBefore = new Date(currentDateTime.valueOf() - (1000 * 60 * 120));
collection.find({
"$or": [
{
"field.0.category": 1,
"field.0.date": { "$lt": ninetyMinsBefore }
},
{
"field.0.category": 2,
"field.0.date": { "$lt": oneTwentyMinsBefore }
}
]
})
Note here that instead of calculating the "date" which is stored adjusted by the variable interval and seeing how that compares to the current date you instead calculate the differences from the current date and then conditionally apply that depending on the value of "category".
This is the fast and efficient way since you were able to re-order the array items as described above and then we can apply the conditions to see if that "first" element met them.
Part 2: Slower forced calculation
collection.aggregate([
{ "$addFields": {
"lastDate": {
"$arrayElemAt": [ "$field.date", -1 ]
},
"lastCategory": {
"$arrayElemAt": [ "$field.category", -1 ]
}
}},
{ "$match": {
"$or": [
{ "lastCategory": 1, "lastDate": { "$lt": ninetyMinsBefore } },
{ "lastCategory": 2, "lastDate": { "$lt": oneTwentyMinsBefore } }
]
}}
])
Same basic premise as even though you already needed to project values from the "last" array elements there's no real need to adjust the stored "date" values with math, which would just be complicating things further.
The original $addFields projection is the main cost, so the main disservice here is the $match on the bottom.
You could optionally use $expr with modern MongoDB releases, but it's basically the same thing:
collection.find({
"$expr": {
"$or": [
{
"$and": [
{ "$eq": [ { "$arrayElemAt": [ "$field.category", -1 ] }, 1 ] },
{ "$lt": [ { "$arrayElemAt": [ "$field.date", -1 ] }, ninetyMinsBefore ] }
]
},
{
"$and": [
{ "$eq": [ { "$arrayElemAt": [ "$field.category", -1 ] }, 2 ] },
{ "$lt": [ { "$arrayElemAt": [ "$field.date", -1 ] }, oneTwentyMinsBefore ] }
]
}
]
}
})
Worth noting the special "aggregation" forms of $or and $and since everything within $expr is an aggregation expression that needs to resolve to a Boolean value of true/false.
Either way it's all just the same problem as the initial "query only" examples are natively processed and can indeed use an index to speed up matching and results. None of these "aggregation expressions" can do that, and thus run considerably slower.
NOTE: If you are storing "date" with the purpose of meaning "expired" as the ones you want to select then it is "less than" the current date ( minus the interval ) rather than "greater than" as you presented in your question.
This means the current time, then subtract the interval ( instead of adding to the stored time ) would be the "greater" value in the selection, and therefore things "expired" before that time.
N.B Normally when you query for array elements with documents matching multiple properties you would use the $elemMatch operator in order for those multiple conditions to apply to that specific array element.
The only reason that does not apply here is because of the use of the numeric index value for the 0 position explicitly on each property. This means that rather than over the entire array ( like "field.date" ) this is specifically applying to only the 0 position.

MongoDB Get highest value for each date and put them into an array

What's the best way to go about getting the highest "score" for each "date" and storing them into an array. Let's the say there are over 50 scores for any particular date.
The database looks like this
{
"_id" : ObjectId("5c06b91b583248493294"),
"classid" : "00010109e2",
"score" : 720,
"height" : 1440,
"time" : "2018-11-27T18:05:13.297621823Z",
"__v" : 0
}
And what I'm trying to do is get the highest score for each date, from a date-range of around 2 weeks and store one highest score for each date in a simple array.
I've tried loads of things, including recursion to no avail.
Can anyone shed any light on this, or point me in the right direction?

You should look into mongodb aggregation and especially $group operator, it usually used to perform such kind of operations.
In this case your code will look like that:
Scores.aggregate([
{
$match: {
time: {
$gte: startOfSomePeriod,
$lte: endOfSomePeriod
}
}
},
{
$group: {
_id: {
year: { $year: "$time" },
month: { $month: "$time" },
day: { $dayOfMonth: "$time" }
},
score: { $max: "$score" }
}
}
]);
P.S. You can use mongooses default createdAt timestamp by simply adding an option to your schema definition.

Aggregation output to Nest Arrays

I have a dataset of records stored in mongodb and i have been trying to extract a complex set of data from the records.
Sample records are as follows :-
{
bookId : '135wfkjdbv',
type : 'a',
store : 'crossword',
shelf : 'A1'
}
{
bookId : '13erjfn',
type : 'b',
store : 'crossword',
shelf : 'A2'
}
I have been trying to extract data such that for each bookId, i get a count (of records) for each shelf per store name that holds the book identified by bookId where the type of the book is 'a'.
I understand that the aggregation query allows a pipeline that allows grouping, matching etc, but I have not been able to reach a solution.
The desired output is of the form :-
{
bookId : '135wfkjdbv',
stores : [
{
name : 'crossword'
shelves : [
{
name : 'A1',
count : 12
},
]
},
{
name : 'granth'
shelves : [
{
name : 'C2',
count : 12
},
{
name : 'C4',
count : 12
},
]
}
]
}

The process isn't really that difficult when you look at at. The aggregation "pipeline" is exactly that, where each "stage" feeds a result into the next for processing. Just like unix "pipe":
ps -ef | grep mongo | tee out.txt
So it's just adding stages, and in fact three $group stages where the first does the basic aggregation and the remaining two simply "roll up" the arrays required in the output.
db.collection.aggregate([
{ "$group": {
"_id": {
"bookId": "$bookId",
"store": "$store",
"shelf": "$shelf"
},
"count": { "$sum": 1 }
}},
{ "$group": {
"_id": {
"bookId": "$_id.bookId",
"store": "$_id.store"
},
"shelves": {
"$push": {
"name": "$_id.shelf",
"count": "$count"
}
}
}},
{ "$group": {
"_id": "$_id.bookId",
"stores": {
"$push": {
"name": "$_id.store",
"shelves": "$shelves"
}
}
}}
])
You could possibly $project at the end to change the _id to bookId, but you should already know that is what it is and get used to treating _id as a primary key. There is a cost to such operations, so it is a habit you should not get into and learn doing things correctly from the start.
So all that really happens here is all the fields that would make up the grouping detail are made the primary key of $group with the other field being produced as count, to count the shelves within that grouping. Think the SQL equivalent:
GROUP BY bookId, store, shelf
All each other stage does is transpose each grouping level into array entries, first by shelf within the store and then the store within the bookId. Each time the fields in the primary grouping key are reduced down by the content going into the produced array.
When you start thinking in terms of "pipeline" processing, then it becomes clear. As you construct one form, then take that output and move it to the next form and so on. This is basically how you fold the results within two arrays.