In my Node.js app, I'm trying to add a TTL index to a date field in a MongoDB collecion to make it expire at a specified date.
The application gets the current date through new Date(), converts it to miliseconds through the getTime() method, adds a number of miliseconds specified by the user (expiration time), and converts the result back to Date format through setTime(). The result is saved to a field named expireAt in a JSON object that is eventually inserted into a MongoDB collection.
The result looks fine, as it accurately represents the date according to UTC timezone adjusted to the expiration time that was added to the current date. For example:
expireAt: "2017-05-14T13:59:01.998Z", which was inserted at approximately 13:00 UTC with a 1 hour expiration time.
To add the TTL index, I added the following line in my Node application:
collection.createIndex({"expireAt": 1}, {expireAfterSeconds: 0, name: "_exp"});
This, however, gave me a MongoError: Values in the index key pattern cannot be 0 error and no index was created, so I switched to:
collection.createIndex({"expireAt": 1}, {expireAfterSeconds: 1, name: "_exp"});
This time, and index was created when I ran it, as I could see using MongoDB Compass
I then proceeded to insert documents with a expireAt field, such as the one explained above with expireAt: "2017-05-14T13:59:01.998Z". However, it's been nearly an hour since the document should have expired and it has not. In addition, the image above shows that the TTL index has 0 usage, which suggests that for some reason the new documents inserted are not making use of that index despite having the expireAt field.
Moreover, MongoDB compass displays the content of the expireAt field as type string instead of the specific BSON date type. However, I'm not sure if this is only a Compass thing since it doesn't let me edit the field type to anything other than String, Object or Array.
Is there anything I could be missing, or has anyone ever ran into a similar issue and found a soultion? I tried to look for a solution in similar questions without success.
It seems that the problem is related with field type. TTL index field must be date type. https://docs.mongodb.com/manual/core/index-ttl/
If the indexed field in a document is not a date or an array that
holds a date value(s), the document will not expire.
You should consider create document with a new Date object.
"expireAt" : new Date("2017-05-14T13:59:01.998Z")
Related
I'm trying to convert my date to the correct time zone with moment.js. However, I always get the whole thing without a time specification.
Here is my program code:
console.log(von);
console.log(bis);
var nVon = moment.tz(von, "Europe/Berlin");
var nBis = moment.tz(bis, "Europe/Berlin");
console.log(nVon.format());
console.log(nBis.format());
This is what I see in the console:
2022-10-31T00:00:00+01:00
And here the original German format, which I want to save in the correct time zone in MongoDb.:
The problem is that it is saved in MongoDB with an hour loss of time like this, without UTC etc.: 2022-10-31T19:44:39.000+00:00
Date values in MongoDB are stored as UTC times - always and only!
If you need to preserve the client input time zone, then you must store it in a separate field. Usually the client takes responsibility to display the MongoDB UTC times as local times.
NB, you should never store date values as string, it's a design flaw. Store always proper Date objects. Thus store
moment.tz(von, "Europe/Berlin").toDate()
I have a collection elements with documents that contain start date (timestamp data type) and end date (timestamp data type).
If I want to get all elements which end date is greater than today, I do:
firebase.firestore().collection('elements').where('end', '>', new Date()).get();
This works perfectly. Now I want to add a rule with the same check on firestore, to prevent that a malicious user could perform this request with new Date() containing a past date and return expired documents.
I added the following rule to firestore:
match /elements/{document=**} {
allow read: resource.data.end > request.time
}
Now all the queries return FirebaseError: Missing or insufficient permissions and I can't understand why. The query is requesting for all elements that end date is greater than today's date, and the rule is verifying that document's end date is greater than server time. In my mind this should work.
The problem is almost certainly that the client machine's clock does not match the Google server's clock. If the client clock is running even slightly faster than Google's clock, the rule will deny the query.
You could try adding some padding from the date to add a reasonable offset from the current time:
// add 10 seconds from current time so as not to undershoot the end
.where('end', '>', new Date(Date.now() + 10000))
While this might work OK for clients whose clocks are reasonably in sync with world standards, it could still fail if the client's clock is still way off.
I am pulling in some data from a Firebase database, and I need to run some processing on it and then return it in the same format. For the most part this is not too bad, however when I pull the Date of Birth field it is sent to me in a Timestamp format as shown here:
This is then being converted into a date, everything that needs to be done is done to it, and I am then running the following code on the date format to turn it back into a timestamp.
updateData['dob'] = new Date(newUserDob).getTime();
This works and gets me the exact same value as I would be getting from the timestamp, however when I create it it is in this format
I have tried using:
var tStamp = new Timestamp('seconds', 1571184000000);
However this produces a timestamp of the format
Timestamp {value: 'seconds', timestamp: 1571184000}
Which is also not what I was getting originally.
Does anyone know how to just create a timestamp of the format shown in the first image? Thank you.
Edit: The second picture is showing nanoseconds instead of seconds. That was a bug at the time when the picture was taken, but rest assured the value is 1571184000.
{seconds: new Date(newUserDob).getTime(), nanoseconds: 0}
Ok, I figured it out eventually.
It was specifically because I was using Firebase to store the timestamp.
So I was able to recreate the timestamp format with:
firestore.Timestamp.fromDate(new Date(newUserDob));
I'm trying to convert a date object to a Firestore timestamp.
var dateOBJ = new Date();
var timeStamp = new firebase.firestore.Timestamp(dateOBJ);
This gives me an error:
Uncaught Error: Timestamp seconds out of range: Sun Dec 09 2018 11:37:05 GMT+0100
I tried converting the date object to seconds first by using .getTime() / 1000, but it's still out of range.
The timestamp is gonna be the expiration date for an url, so I need to add some time to it.
There are two ways of setting a date field in Cloud Firestore:
You specify a Date value for the field, in which case you fully determine what date is written.
You specify the firebase.firestore.FieldValue.serverTimestamp(), in which case the server writes the current date.
There is no way in the API to combine these two options, you either use one or the other.
Since you comment that you want to store the timestamp and an offset, that is also what I'd store:
a timestamp field that you let the server populate with firebase.firestore.FieldValue.serverTimestamp().
a offset field, that you populate from the app with the offset in days/hours.
That way you can reconstitute the effective expiration timestamp by combining the two fields.
You could even add a third field that stores the expiration timestamp, but that will require an extra write operation. I'd typically do this in Cloud Functions, to ensure you keep full control over the field and clients can't spoof it. If you don't do it in Cloud Functions, consider writing security rules that validate that the value if the calculated field is indeed the result of that calculation.
You won't get a consistent server side timestamp with a JavaScript date. Instead, send the server timestamp from the SDK:
const timestamp = firebase.firestore.FieldValue.serverTimestamp()
If you still want to set the timestamp as a Date you can just pass new Date() to Firestore and it will be saved as a timestamp.
Frank is right about setting timestamps into firestore.
If you want to check that timestamp on the front end afterwards you need to use .toDate on the timestamp object returned from firestore to turn it back into a JS date.
I'm working on a scheduling system for music venues. The basic idea is that there's an "Create new schedule" page, on which there is a DatePicker calendar (using AngularUI Bootstrap). The user selects a Date, then adds performers into timeslots. The built object looks something like this:
{
date: 2017-6-22 00:00:00.000-5:00
venue: VenueID
performances: [
{
performer: performerID,
time: 2017-06-22 22:00:23.231-5:00
},{
perfomer: performer2ID,
time: 2017-06-22 23:00:42.523-5:00
}
]
}
There's a couple of problems here. For the original date selection, I set the time (using myDate.setHours(0,0,0,0)) to midnight because the time doesn't really matter, I only care about the actual date. Likewise for the timeslots, their date doesn't matter (since they belong to the schedule for that day), so I only care about the time. Then in another project, we have a node/mongo app that saves these schedules, and returns them to a page in the angular project that lets you select a schedule for editing/etc. It selects which ones to return by grabbing all the schedules for a specific venue, and doing "if (schedule.date >= new Date().setHours(0,0,0,0)) { add schedule to return list }"
Anyway, on to the actual problem. The angular app does all of the date calculations client side. What I mean is, I'm in CST. If I select a Date on the calendar and save a schedule for that date, then someone in EST selects the same day on the calendar and saves a schedule, they have different dates in the database. For example, when I make the schedule, the date in the DB is "2017-06-22 00:00:00.000-5:00". When the EST friend makes a schedule on the same date, it gets saved as "2017-06-22 00:00:00.000-4:00".
In the "Select a schedule to view/edit" page, I do something like this:
<select ng-model="schedule" ng-options="s.date|date:'fullDate' for s in schedules" ng-show="schedules.length>=1"></select>
Of course this doesn't work because when my EST friend looks at the list, he sees the correct date. But when I look at one that he created, the date is one day off because "2017-06-22 00:00:00.000-4:00" converted to local timezone is "2017-06-21 23:00:00.000-5:00".
I guess TL;DR is I'm not sure how to handle it since the venue and anyone creating/editing the schedules may not share the same time zone. I want all of the dates/times to show up in the timezone of the venue (which I have the address for. I guess I could geolocate to find timezone?). I'm just not sure how to go about it.
The DatePicker gives you a date object. Instead of storing the entire value string just grab the day month and year Date(value).getYear() + '-' + Date(value).getMonth() + '-' + Date(value).getDate(). As for the times do the same as the dates. Store those values in the DB and then when you get them back you will have to convert them back to a date object so that the date picker can understand them.
Ultimately with this solution your just trying to store dates without the timezones. Make sure to state in your app that the times are for those areas.
You have to distinguish between the format the date/time is transported, saved vs. how the date will be shown to the user.
For transportation and saving use UTC in a format that is easy computable (eg. ISO8601).
For visualization to the user convert this value to the timezone and desired user format by using some helper library.