How to create TTL Index on long timestamp in MongoDB - javascript

In my mongo database I have field timestamp which holds time of creation in timestamp ie: "timestamp": 1544029233021
I want to create TTL index on this field, but in docs example is done on "createdAt": new Date(), which is ISODate("2018-12-13T17:00:10.433Z")
Is it possible to in any way to make TTL Index work on timestamp field?
Because this doesnt work:
db.coll.createIndex( { "timestamp": 1 }, { expireAfterSeconds: 3600 } )

The documents aren't being expired because the timestamp value is an integer.
TTL indexes will only work on documents where the indexed field is a Date or holds an array of Dates:
If the indexed field in a document is not a date or an array that
holds a date value(s), the document will not expire.
(https://docs.mongodb.com/manual/core/index-ttl/#expiration-of-data)

Related

How to properly sort documents in firebase collection from first to last

I want every data sent last to be sorted as the last mongo document in my firebase collection. In any case, the last sent data will be stored as the last document in the collection. I used sorting by dates and current time but it is not effective for all dates. (for example date 2.6 puts first instead of 19.6)
This is my function:
Reservations(){
this.user = store.currentUser
this.empty_form=false;
if(this.current_museum!=''&&this.date!=''&&this.ticket_number!=''&&this.user!='')
{
//Date I used for sorting
const current = new Date();
const date = current.getFullYear()+'-'+(current.getMonth()+1)+'-'+current.getDate();
const time = current.getHours() + ":" + current.getMinutes() + ":" + current.getSeconds();
const dateTime = date +' '+ time;
let counter = counter + 1;
router.replace({path:'/ticket_reserved', replace: true})
try {
db.collection("reservations")
.doc(dateTime)
.set({
current_museum: this.current_museum,
date: this.date,
ticket_number: this.ticket_number,
user: this.user
});
}
catch(error) {
}
}else{
this.empty_form=true;
}},
I used sorting by dates and current time but it is not effective for
all dates. (for example date 2.6 puts first instead of 19.6)
This is a classical problem with date sorting. One common solution is to store the data in the following format: YYYYMMDD (e.g. 20220602 and 20220619). This way the lexicographic ordering is correct.
You can very well duplicate the date field in your Firestore document: one date field with the date in the desired date format (most probably of type Timestamp) and another field, e.g. dateSorting, using the YYYYMMDD format and of type String. You use the second field for building your query.
Note that it works also for date/time format, e.g. YYYYMMDD-HHMMSS.

Convert date object to Firestore timestamp

I'm trying to convert a date object to a Firestore timestamp.
var dateOBJ = new Date();
var timeStamp = new firebase.firestore.Timestamp(dateOBJ);
This gives me an error:
Uncaught Error: Timestamp seconds out of range: Sun Dec 09 2018 11:37:05 GMT+0100
I tried converting the date object to seconds first by using .getTime() / 1000, but it's still out of range.
The timestamp is gonna be the expiration date for an url, so I need to add some time to it.
There are two ways of setting a date field in Cloud Firestore:
You specify a Date value for the field, in which case you fully determine what date is written.
You specify the firebase.firestore.FieldValue.serverTimestamp(), in which case the server writes the current date.
There is no way in the API to combine these two options, you either use one or the other.
Since you comment that you want to store the timestamp and an offset, that is also what I'd store:
a timestamp field that you let the server populate with firebase.firestore.FieldValue.serverTimestamp().
a offset field, that you populate from the app with the offset in days/hours.
That way you can reconstitute the effective expiration timestamp by combining the two fields.
You could even add a third field that stores the expiration timestamp, but that will require an extra write operation. I'd typically do this in Cloud Functions, to ensure you keep full control over the field and clients can't spoof it. If you don't do it in Cloud Functions, consider writing security rules that validate that the value if the calculated field is indeed the result of that calculation.
You won't get a consistent server side timestamp with a JavaScript date. Instead, send the server timestamp from the SDK:
const timestamp = firebase.firestore.FieldValue.serverTimestamp()
If you still want to set the timestamp as a Date you can just pass new Date() to Firestore and it will be saved as a timestamp.
Frank is right about setting timestamps into firestore.
If you want to check that timestamp on the front end afterwards you need to use .toDate on the timestamp object returned from firestore to turn it back into a JS date.

Failing to make MongoDB documents expire using TTL indexes

In my Node.js app, I'm trying to add a TTL index to a date field in a MongoDB collecion to make it expire at a specified date.
The application gets the current date through new Date(), converts it to miliseconds through the getTime() method, adds a number of miliseconds specified by the user (expiration time), and converts the result back to Date format through setTime(). The result is saved to a field named expireAt in a JSON object that is eventually inserted into a MongoDB collection.
The result looks fine, as it accurately represents the date according to UTC timezone adjusted to the expiration time that was added to the current date. For example:
expireAt: "2017-05-14T13:59:01.998Z", which was inserted at approximately 13:00 UTC with a 1 hour expiration time.
To add the TTL index, I added the following line in my Node application:
collection.createIndex({"expireAt": 1}, {expireAfterSeconds: 0, name: "_exp"});
This, however, gave me a MongoError: Values in the index key pattern cannot be 0 error and no index was created, so I switched to:
collection.createIndex({"expireAt": 1}, {expireAfterSeconds: 1, name: "_exp"});
This time, and index was created when I ran it, as I could see using MongoDB Compass
I then proceeded to insert documents with a expireAt field, such as the one explained above with expireAt: "2017-05-14T13:59:01.998Z". However, it's been nearly an hour since the document should have expired and it has not. In addition, the image above shows that the TTL index has 0 usage, which suggests that for some reason the new documents inserted are not making use of that index despite having the expireAt field.
Moreover, MongoDB compass displays the content of the expireAt field as type string instead of the specific BSON date type. However, I'm not sure if this is only a Compass thing since it doesn't let me edit the field type to anything other than String, Object or Array.
Is there anything I could be missing, or has anyone ever ran into a similar issue and found a soultion? I tried to look for a solution in similar questions without success.
It seems that the problem is related with field type. TTL index field must be date type. https://docs.mongodb.com/manual/core/index-ttl/
If the indexed field in a document is not a date or an array that
holds a date value(s), the document will not expire.
You should consider create document with a new Date object.
"expireAt" : new Date("2017-05-14T13:59:01.998Z")

Mongoose.js find yesterday's date in database

I want to find all the users created up until yesterday. This is my code to make the query string:
var today = new Date();
var a = today.getDate();
a--;
today.setDate(a);
var yesterday = today.toDateString();
and it returns something like: Sun Jan 17 2016... which IS yesterday's date, but the db stores the date in iso format like: "date" : ISODate("2016-01-13T13:23:08.419Z") so what I get is 2016-01-13T13:23:08.419Z. My problem now is that I can't use the yesterday variable to query the db for the users I need AND even if I could, I don't know how to find every registration not including the ones that took place today.
Any help? Thank you very much!
You are generating a date on the front end, and then pushing it back a day, which is totally fine for a lot of circumstances -
For this, since you are trying to find DB entries that occured, perhaps try querying the database with a timestamp ranged pulled from the ID's of each document in your database.
Here is some documentation on how to do that from mongoDB. https://docs.mongodb.org/v3.0/reference/method/ObjectId.getTimestamp/
I've also provided some additional resources that may help you figure out exactly what to query in regard to this method:
https://steveridout.github.io/mongo-object-time/
https://gist.github.com/tebemis/0e55aa0089e928f362d9
Some psuedo code:
1. Query documents in the database
2. Get a timestamp from the ID's of the documents in the database
3. Set a range of the timestamps
4. Compare returned timestamps vs a timestamp range variable (yesterdays date in this case)
5. Have the DB return only documents that are within the range
I hope this helps!
Try this, using Moment.js:
yesterday = moment().add(-1, 'days');
db.users.find({ "date": { "$lt": yesterday }});
Create a date object that represents the start of day today, use it to query your collection for documents where the date field is less than that variable, as in the following example
var start = new Date();
start.setHours(0,0,0,0);
db.users.find({ "date": { "$lt": start }});
This will look for users created up until end of day yesterday.
momentjs is a super handy utility for doing manipulations like this. Using the library, this can be achieved with the startOf() method on the moment's current date object, passing the string 'day' as arguments:
Local GMT:
var start = moment().startOf('day'); // set to 12:00 am today
db.users.find({ "date": { "$lt": start }});
For UTC:
var start = moment.utc().startOf('day');

How to handle date stored as string in Mongodb?

So I have this field called task_time stored as string in Mongodb in 'YYYY-MM-DD' format (eg. '2012-12-21').
Now I need to query this collection to get the data whose task_time is within a given time interval.
The time interval is given as a pair of strings that represent start time and end time in 'YYYY-MM-DD hh:mm:ss' format (eg. '2015-12-21 16:00:00').
Is there any way to do this directly in Mongo query without bringing this task to my javascript code?
As I know $gte and $lt can work with strings too. The same in mongoose after this issue
items.find({
task_time: {
$gte: "2015-12-21 12:00:00",
$lt: "2015-12-21 16:00:00"
}
})

Categories

Resources