Supposing a collection of a large number of documents like this:
{
createdAt: 1630789008561, // timestamp
count: 5
},
{
createdAt: 1630788666511, // timestamp
count: 50
},
...
I would like to query the 50 documents with the highest count that were created after a certain timestamp.
One simple solution is to query all documents after this certain timestamp and then order them yourself. But I can't since my database is too large.
My first query attempt was:
db.collection("docs")
.where("createdAt", ">", timestamp)
.orderBy("count", "desc")
.limit(50)
.get();
This query does not work in Firestore and throws:
Error: 3 INVALID_ARGUMENT: inequality filter property and first sort order must be the same: createdAt and count`
I then tried:
db.collection("docs")
.where("createdAt", ">", timestamp)
.orderBy("createdAt", "desc")
.orderBy("count", "desc")
.limit(50)
.get()
This query will not work for me since what it does is taking 50 documents after the timestamp and then order these by count in case they have the same timestamp, which is not what I want.
Edit on why this query does not work:
If I have 150 documents:
50 created on day 1 with a count of 1
50 created on day 2 with a count of 2
50 created on day 3 with a count of 1
If I use this query with a timestamp of 0, I would get back the 50 documents last created (because I am using .orderBy("createdAt", "desc")) which have a count of 1 and are obviously not the ones with the highest count.
If you use asc instead, you get the first documents, which still have not the highest count.
I would like a query that for a timestamp of 0 would give me the 50 documents with the count of 2.
How can I achieve what I want?
EDIT on solution I am currently using (which I'd like to change):
Since my time range is a day, I add a day field in every documents like this:
{
day: '2021-09-06',
createdAt: 1630789008561, // timestamp
count: 5
},
{
day: '2021-09-05',
createdAt: 1630788666511, // timestamp
count: 50
},
...
I can then do an equality check like this:
db
.collection("docs")
.where("day", "==", '2021-09-05')
.orderBy("count", "desc")
.limit(50)
.get();
This works but this is not great since everytime my time range changes I need to update my db schema and update every documents with the new field.
Return the first 50 docs sorted by count. Then you filter by timestamp on the device.
QuerySnapshot<Map<String, dynamic>> query = await db
.collection("docs")
.orderBy("count", "desc")
.limit(50)
.get();
List docs = query.docs.map((e) => e.data()).toList();
// then filter by timestamp on the device
docs.removeWhere((element) => element['createdAt'] < timestamp);
This cannot be done by using where query because there is no way around this limitation.
If you include a filter with a range comparison (<, <=, >, >=), your first ordering must be on the same field:
this is the right query.
db.collection("docs")
.where("createdAt", ">", timestamp)
.orderBy("createdAt", "desc")
.orderBy("count", "desc")
.limit(50)
.get()
But you need to set Firestore Composite Indexes.
Related
I want every data sent last to be sorted as the last mongo document in my firebase collection. In any case, the last sent data will be stored as the last document in the collection. I used sorting by dates and current time but it is not effective for all dates. (for example date 2.6 puts first instead of 19.6)
This is my function:
Reservations(){
this.user = store.currentUser
this.empty_form=false;
if(this.current_museum!=''&&this.date!=''&&this.ticket_number!=''&&this.user!='')
{
//Date I used for sorting
const current = new Date();
const date = current.getFullYear()+'-'+(current.getMonth()+1)+'-'+current.getDate();
const time = current.getHours() + ":" + current.getMinutes() + ":" + current.getSeconds();
const dateTime = date +' '+ time;
let counter = counter + 1;
router.replace({path:'/ticket_reserved', replace: true})
try {
db.collection("reservations")
.doc(dateTime)
.set({
current_museum: this.current_museum,
date: this.date,
ticket_number: this.ticket_number,
user: this.user
});
}
catch(error) {
}
}else{
this.empty_form=true;
}},
I used sorting by dates and current time but it is not effective for
all dates. (for example date 2.6 puts first instead of 19.6)
This is a classical problem with date sorting. One common solution is to store the data in the following format: YYYYMMDD (e.g. 20220602 and 20220619). This way the lexicographic ordering is correct.
You can very well duplicate the date field in your Firestore document: one date field with the date in the desired date format (most probably of type Timestamp) and another field, e.g. dateSorting, using the YYYYMMDD format and of type String. You use the second field for building your query.
Note that it works also for date/time format, e.g. YYYYMMDD-HHMMSS.
Pretty self-explanatory.
I searched multiple mongo aggregation documentation documents, and built the query the best I could.
Worth to point out I have almost none aggregation experience, and this is onde of the most complex aggregates I have done so far.
let currentYear = new Date();
currentYear.setFullYear(currentYear.getFullYear());
currentYear.setMonth(0);
currentYear.setDate(1);
await Order.aggregate([
{ $match: {
order_ordered_on:{
$gte: currentYear
},
order_status: 'Delivered'
}},
{ $group : {
_id: {month: {$month : '$order_ordered_on'}},
total: {$sum: '$order_total'},
}},
{ $sort : {
_id : 1
}},
{$project: {
_id:1,
total: 1
}}
]).exec((err, result) =>{
if (err) throw err;
let amount = result.map(a => a.total);
res.json({
amount
});
})
To explain my thought process, First I set a date for current year, january first until current time.
Then I match it with the order_ordered_on which has the date which Im working of, and the also match to select only the orders which have the Delivered status.
I group by _id to divide it in months and sum the total amount by months aswell.
Sort it in ascending order do order the array and project the resulting data.
After that I map the result to an array, which is my goal(each position in the array corresponds to a months profit).
The problem is that the total value being returned is absurd and it doesnt match what is expected. Since I dont have much experience, I could not find where Im making the aggregation mistake.
Any help is appreciated.
To make myself clearer, Im currently using a single Order model, and the order_ordered_on, order_status and order_total fields.
The main goal of this aggregate is to split the current year profit in monthly profits, from january 1st until now, and then plot it in a graph through front-end development. The data contains Orders from 2017 until current day. The format the front-end needs to receive is that of an ordered array with the profit values.
So I have this code line here:
let advertisements = await Advertisement.find({created_at: {$lt: moment().valueOf()}});
This is what it does:
created_at: returns a timestamp when post was created at: 1551198203488.0
moment.valueOf() return me a current timestamp for example: 1551198203488.0
Now I need to write this code line that it only finds ads that are 1 hour old. Is is possible somehow ?
Also I storing and group2_date which store current timestamp BUT with 1 hour added to it like this: moment().add(1, 'h')
You can subtract one hour from the moment using .subtract function and then use $gte operator to obtain only the greater values/documents from the subtracted hours and less then from the current time.
let advertisements = await Advertisement.find({
"created_at": {
"$lte": moment().toDate(),
"$gte": moment().subtract(1, 'hours').toDate()
}
})
In my mongo database I have field timestamp which holds time of creation in timestamp ie: "timestamp": 1544029233021
I want to create TTL index on this field, but in docs example is done on "createdAt": new Date(), which is ISODate("2018-12-13T17:00:10.433Z")
Is it possible to in any way to make TTL Index work on timestamp field?
Because this doesnt work:
db.coll.createIndex( { "timestamp": 1 }, { expireAfterSeconds: 3600 } )
The documents aren't being expired because the timestamp value is an integer.
TTL indexes will only work on documents where the indexed field is a Date or holds an array of Dates:
If the indexed field in a document is not a date or an array that
holds a date value(s), the document will not expire.
(https://docs.mongodb.com/manual/core/index-ttl/#expiration-of-data)
Context
I'm building a query where a user can enter a range of dates to search for 'Player' documents. The query should return any players whose start and end dates overlap with the queried dates. I'm doing this by using an OR condition where either the player's end date is after the query's start date or the player's start date is before the query's end date. Both dates in the query are optional, allowing the user to specify only an end date or a start date, or neither to return all relevant documents.
Problem
The query works when only one of the two query dates are specified (see screenshots below). However, it returns all documents if both dates are specified (functionally ignoring both query parameters).
Code
let query = Player.find({'casino': user.casino});
let queryArray = [];
if(req.body.startDate) {
queryArray.push({playerEndDtm: {$gt: moment(req.body.startDate).toDate()}});
}
if(req.body.endDate) {
queryArray.push({playerStartDtm: {$lt: moment(req.body.endDate).endOf('day').toDate()}});
}
if(queryArray.length > 0) {
query.or(queryArray);
}
Screenshots
When both query dates are specified:
Results:
When only 1 query date is specified:
Results:
Can anyone help me determine what I'm doing wrong? Thanks!
if(req.body.endDate) {
queryArray.push({playerStartDtm: {$lt:
moment(req.body.endDate).endOf('day').toDate()}});
}
In the above query you are justing checking less then the end date. You need to check range between dates:
({ playerStartDtm : { $gt : expression, $lt : expression}})