javascript + postgres: timezone and timestamp usage - javascript

I'm not quite understanding the timestamp usage,
e.g.
User create article and they can choose PublishDate , the system also store CreateDate automatically.
a. Should I make PublishDate and CreateDate timestamp with time zone and set utc?
b. User post string and then I convert like below use momentjs to utc timestamp and store, when someone select this row , show them as user client time reverse use momentjs
c. I use CURRENT_TIMESTAMP to the CreateDate, the CURRENT_TIMESTAMP does that mean the server time? am I doing correct?
My thinking is I always insert utc timezone timestamp to the database, and wherever the place user/client read, convert the data to user/client timezone? am I doing correct?
a. my database(postgres) created by
CREATE TABLE IF NOT EXISTS "Article"(
"ArticleId" SERIAL NOT NULL,
"PublishDate" timestamp with time zone,
"Active" bit NOT NULL,
"CreateByUserId" integer,
"CreateDate" timestamp with time zone,
PRIMARY KEY ("ArticleId")
);
SET timezone = 'UTC';
b. user submit post to store (nodejs)
// publishDate: '{"y":2015,"m":8,"d":16,"h":15,"mi":46,"s":24}
var publishDate = JSON.parse(req.body.publishDate);
var leadingZeroAndDateFormat = function(publishDate) {
return new Promise(function (fulfill, reject){
if (publishDate.m < 10) { publishDate.m = '0'+publishDate.m; }
if (publishDate.d < 10) { publishDate.d = '0'+publishDate.d; }
if (publishDate.h < 10) { publishDate.h = '0'+publishDate.h; }
if (publishDate.mi < 10) { publishDate.mi = '0'+publishDate.mi; }
if (publishDate.s < 10) { publishDate.s = '0'+publishDate.s; }
var str = publishDate.y+'-'+publishDate.m+'-'+publishDate.d+' '+publishDate.h+':'+publishDate.mi+':'+publishDate.s;
var utc = moment(str).unix();
fulfill(utc);
});
};
c. insert to database the CreateDate use CURRENT_TIMESTAMP
var insertArticle = function(publishDate, active, createByUserId) {
return new Promise(function (fulfill, reject){
var query = 'INSERT INTO "Article" ("PublishDate","Active","CreateByUserId","CreateDate") VALUES ($1,$2,$3,CURRENT_TIMESTAMP) RETURNING "ArticleId"';
dbClient.query(query,[publishDate,active,createByUserId], function(error, result) {
if (error) {
reject(error);
} else {
fulfill(result);
}
});
});
};
Update
When I change all column without timezone then I execute insertArticle shows the error
{ [error: date/time field value out of range: "1439717298"]
name: 'error',
length: 158,
severity: 'ERROR',
code: '22008',
detail: undefined,
hint: 'Perhaps you need a different "datestyle" setting.',
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: 'datetime.c',
line: '3775',
routine: 'DateTimeParseError' }
var insertArticle = function(publishDate, active, createByUserId) {
return new Promise(function (fulfill, reject){
var query = 'INSERT INTO "Article" ("PublishDate","Active","CreateByUserId","CreateDate") VALUES ($1,$2,$3,$4) RETURNING "ArticleId"';
dbClient.query(query,[publishDate,active,createByUserId,moment.utc().unix()], function(error, result) {
if (error) {
reject(error);
} else {
fulfill(result);
}
});
});
};

The simplest way is to always store time stamps without time zone and in UTC. This way it is always easy to use them for display and calculations. A difference is just a subtraction and comparisons go directly.
If the column was a time stamp with time zone, then the input would be converted to UTC anyway, but the output would be in the currently set time zone and if it's not set properly it might show wrong values. It also makes the database less efficient.
In the presentation layer the timestamps can be shown in the proper time zone and values input can also be converted to UTC for storage.
This is the simplest, most efficient and most flexible way of handling timestamps.
Using CURRENT_TIMESTAMP will get the timestamp from the server at the time of execution.

The incoming timestamps without timezone are parsed in local time instead of UTC -- I'd call this a bug in node-postgres. Anyway, you can override it to do the right thing by adding the following code:
pg.types.setTypeParser(1114, function(stringValue) {
console.log(stringValue);
return new Date(Date.parse(stringValue + "+0000"));
})

Related

Why is a client's Date.now giving a timestamp in the future?

I have an app that displays data to users based on the date. I send an array from the server to the client, on load, which contains 3 objects with some data - effectively an object for 'yesterday', 'today' and 'tomorrow' in UTC, since at most there can be 3 different dates across the world. Each object contains some data for that date.
E.g:
[
{dateStamp: '2022-05-31', data: 'some data for 31 May'},
{dateStamp: '2022-06-01', data: 'some data for 1 Jun'},
{dateStamp: '2022-06-02', data: 'some data for 2 Jun'}
]
In JS, I then decide what timezone the client is in and pick the appropriate object to show them (UTC-1 day, UTC today, UTC+1 day). I do this based on a simple Date object. From that object I generate a timestamp string (YYYY-MM-DD) and then match it against the 3 objects' timestamp strings (generated in PHP with the same format).
But I'm noticing in a really odd error in my activity logs. (This is data I post from clients via AJAX and store on the server for monitoring/debugging). I first spotted it when some users were getting shown data for tomorrow.
Some users (not many) are reporting UNIX timestamps (from Date.now()) that are in the future (whilst appearing to report a UTC offset that is correct).
The reported UTC offsets are generated in JS at the client and sent via AJAX as such:
let nowDate = new Date();
let utcOffset = nowDate.getTimezoneOffset();
if (!isEmpty(utcOffset) && !isNaN(utcOffset)) { // isEmpty is my own function to match the PHP empty() function
utcOffset = (-1 * utcOffset) / 60;
}
else {
utcOffset = undefined;
}
Now obviously this data is coming from client, so a malicious user could simply use the console to manipulate this to anything they like as it's being sent.
But it's a small app with relatively few users and that seems extremely unlikely to me.
So, excluding a malicious user, some clients apparently have their local clock set wrong (set to the future). Why?? Or how?? How are they in the right timezone but the wrong clock time by several days? This is not an app that attracts 1000s of users - I have 100s and I pretty much know they're all accessing via fairly normal means, primarily via mobiles - I know it's an assumption but I would have expected their clocks to be (approximately) correct.
What's the best way of checking what the actual time in their timezone is? I'm thinking I send the UTC offset to the server via AJAX and my server reports back what the date is in that timezone?
For completeness, here is what/how I'm sending to the server from the client:
function getDateStamp() { // Aware this function is verbose, haven't had time to improve
let now = new Date();
let todayDay = now.getDate().toString();
let todayMonth = (now.getMonth() + 1).toString();
let todayYear = now.getFullYear().toString();
if (!isEmpty(todayDay) && todayDay.length < 2) {
todayDay = "0" + todayDay;
}
if (!isEmpty(todayMonth) && todayMonth.length < 2) {
todayMonth = "0" + todayMonth;
}
let fullDateStamp = todayYear + "-" + todayMonth + "-" + todayDay;
if (!isEmpty(todayYear) && !isEmpty(todayMonth) && !isEmpty(todayDay)) {
return fullDateStamp;
}
else {
return false;
}
}
let clientDateStamp = getDateStamp();
let clientUnixTimestamp = Math.floor(Date.now() / 1000);
let objectToSend = {dateStamp: clientDateStamp, unixTimestamp: clientUnixTimestamp};
var dataTransfer = $.ajax({ // Using JQuery for AJAX post
url: serverEndpointURL, // Where serverEndpointURL is an HTTPS url to my server script
type: "POST",
dataType: "json",
data: {json: objectToSend}
});
In PHP, on the server I decode that JSON:
if ($_SERVER['REQUEST_METHOD'] === 'POST') {
$json = $_POST["json"];
$postData = json_decode($json, true);
}
error_log($postData["unixTimestamp"] . "\r\n",3, $errorLogFile); // $errorLogFile points to a txt file
In $errorLogFile I can see timestamps in the future. I.e. $postData["unixTimestamp"] > time() would have been true at the time of logging.

Mongoose Date Filter

I have data stored in a MongoDB database and I'm using Mongoose to query the data. I'm trying to run date queries against my data to return objects from the database that fall within the specified data-range.
My webform sends an API request to an external micro-service/api that is responsible for querying the data in the Mongo database. The API receives a single value that is representative of a number of days. eg: date: "7d". I then proceed to build the mongoose query like so:
if (data.date) {
const date = new Date();
const dateRange = data.date.slice(0, -1); // strip the "d" from "7d"
date.setDate(date.getDate() - dateRange);
query.start = { $lte: date.toISOString() };
console.log(query);
}
and an example date from my database is:
start: 2016-02-10T09:09:01.000Z
The query is then executed:
Call.find(query, function (error, docs) {
if (error) {
callback(error, null);
} else {
callback(null, docs);
}
});
and finally, an example of query:
{ approved: 1, start: { '$lte': '2016-02-09T14:24:29.115Z' } }
However, no matter what date I send to my API the response is always empty...
Use the actual date object for your query, not string as you are doing presently. Because mongo stores dates wrapped with the ISODate helper and the underlying BSON (the storage data format used by mongo natively) has a dedicated date type UTC datetime which is a 64 bit (so, 8 byte) signed integer denoting milliseconds since Unix time epoch, your query doesn't return anything as it will be comparing the date fields in mongo with an ISO formatted string.
So, drop the toISOString() conversion and use the date object:
if (data.date) {
const date = new Date();
const dateRange = data.date.slice(0, -1); // strip the "d" from "7d"
date.setDate(date.getDate() - dateRange);
query.start = { $lte: date };
console.log(query);
}
Call.find(query, function (error, docs) {
if (error) callback(error, null);
callback(null, docs);
});
Better yet, you can use the momentjs plugin that has a very intuitive and easy datetime manipluation API. One method you can use is the subtract() function to get the date object n number of days ago:
if (data.date) {
const dateRange = data.date.slice(0, -1); // strip the "d" from "7d"
const date = moment().subtract(dateRange, "days");
query.start = { $lte: date };
console.log(query);
}
Call.find(query, function (error, docs) {
if (error) callback(error, null);
callback(null, docs);
});

Problems in setting a date constraint on a parse query

I'm trying to make a parse query and put a date constraint on the query via javascript API.
This is the Result table/object (on parse.com) which I'm making a query for:
Column Data type
objectId String
createdDate Date
updatedAt Date
obtainedPonts Number
relatedDriver Relation
relatedTest Relation
ACL ACL
resultCeasesToBeValid Date
didPassTheTest Boolean
Some example data from resultCeasesToBeValid:
This is my goal: I want the query to give me a set of Result where today < resultCeasesToBeValid.
My problem is that I always recieve an Result array with .length = 0 when I'm trying to put date constraints on the query.
function IsApprovedAndHasValidResults(currentDriverObjectId) {
var Driver = Parse.Object.extend('Driver');
var currentDriverObject = new Driver();
currentDriverObject.id = currentDriverObjectId;
var queryResult = new Parse.Query(Result);
//set the constraints
queryResult.equalTo('relatedDriver', currentDriverObject); //this constraint works as expected
/****************************************************************************************************
*var today = new Date(); // "today" is as time of writing 3 oct 2014 *
* *
* //will give me a parseResults[] of .length=0: *
*queryResult.lessThan('resultCeasesToBeValid', today); *
* *
* //will give me a parseResults[] of .length=0: *
*queryResult.lessThan('resultCeasesToBeValid', { "__type": "Date", "iso": today.toISOString() }); *
* *
*****************************************************************************************************/
queryResult.find({
success: function (parseResults) {
// results is an array of Parse.Object.
/*when the code gets here parseResults array.lenght equals 0*/
},
error: function (error) {
// error is an instance of Parse.Error.
/*will never be here*/
}
});
}
Other developers seem to have the same problem, user Abhishek suspects a bug. According to Héctor Ramos (Parse), long time ago: "You should use a Date object, not a string, when dealing with dates in JavaScript.". Obviously this doesn't work!
Is my code wrong in some way?
Workaround:
queryResult._where.resultCeasesToBeValid = {'$lt' :{ "__type": "Date", "iso": today}}};
_where is a "private property" though and i wouldn't rely on it too much. Parse should fix this.
If you can [change the column type], i would suggest storing the date as a unix timestamp to avoid this kind of issues:
var timestamp = +new Date();
result.save({'resultCeasesToBeValid': timestamp});

Google Form on Submit get values and format the time

I am using Google Apps Script with a Google form. When the user submits the Google Form I get a value from a question. I then take that value and make it a date object, from what I saw on this post about daylight savings I use that to determine the timezone. I run the date object through Utilities.formatDate and want to get the correctly formatted date.
example: 9:00 AM
But instead I am getting a completely different time than expected.
My question is: Can someone help me understand why the below code is outputting a time that is 3 hours different?
function onSubmit(e) {
var values = e.values;
Logger.log(values);
try {
var start1 = new Date(values[3]);
var startN = new Date(start1).toString().substr(25,6)+"00";
var startT = Utilities.formatDate(start1, startN, "h:mm a");
Logger.log(startT);
} catch(error) {
Logger.log(error);
}
}
The assumption that Utilities formatDate does not support GMT... parameter is not true.
The post you mentioned in reference is used to get calendar events and is a useful way to get the right value when you get events from another daylight saving period (getting the TZ info from the calendar event itself), for example events for next month will be in "summer time" while we are still in "winter time"...
Your issue might come from different sources depending on time zone settings of your script vs timezone of the source. Could you describe the exact configuration in which you use this script ?
In the mean time, here is a small code that demonstrates how the code is working + the logger results :
function testOnSubmit() {
var eventInfo = {};
var values = {};
values['3'] = new Date();
eventInfo['values'] = values;
Logger.log('eventInfo = '+JSON.stringify(eventInfo)+'\n\n');
onSubmit(eventInfo);
}
function onSubmit(e) {
var values = e.values;
try {
var start1 = new Date(values[3]);
Logger.log('onSubmit log results : \n');
Logger.log('start1 = '+start1)
var startN = new Date(start1).toString().substr(25,6)+"00";
Logger.log('startN = '+startN);
var startT = Utilities.formatDate(start1, startN, "h:mm a");
Logger.log('result in timeZone = '+startT);
} catch(error) {
Logger.log(error);
}
}
EDIT : additionally, about the 30 and 45' offset, this can easily be solved by changing the substring length like this :
var startN = new Date(start1).toString().substr(25,8);
the result is the same, I had to use the other version a couple of years ago because Google changed the Utilities.formatDate method at some moment (issue 2204) but this has been fixed.
EDIT 2 : on the same subject, both methods actually return the same result, the GMT string has only the advantage that you don't have to know the exact timezone name... there is also the Session.getScriptTimeZone() method. Below is a demo script that shows the resulst for 2 dates in January and July along with the log results :
function testOnSubmit() {
var eventInfo = {};
var values = {};
values['3'] = new Date(2014,0,1,8,0,0,0);
eventInfo['values'] = values;
Logger.log('eventInfo = '+JSON.stringify(eventInfo)+'\n\n');
onSubmit(eventInfo);
values['3'] = new Date(2014,6,1,8,0,0,0);
eventInfo['values'] = values;
Logger.log('eventInfo = '+JSON.stringify(eventInfo)+'\n');
onSubmit(eventInfo);
}
function onSubmit(e) {
var values = e.values;
var start1 = new Date(values[3]);
Logger.log('onSubmit log results : ');
Logger.log('start1 = '+start1)
var startN = new Date(start1).toString().substr(25,8);
Logger.log('startN = '+startN);
Logger.log('result in timeZone using GMT string = '+Utilities.formatDate(start1, startN, "MMM,d h:mm a"));
Logger.log('result in timeZone using Joda.org string = '+Utilities.formatDate(start1, 'Europe/Brussels', "MMM,d h:mm a"));
Logger.log('result in timeZone using Session.getScriptTimeZone() = '+Utilities.formatDate(start1, Session.getScriptTimeZone(), "MMM,d h:mm a")+'\n');
}
Note also that the Logger has its own way to show the date object value ! it uses ISO 8601 time format which is UTC value.
Try this instead:
var timeZone = Session.getScriptTimeZone();
var startT = Utilities.formatDate(start1, timeZone, "h:mm a");
The Utilities.formatDate function expects a time zone that is a valid IANA time zone (such as America/Los_Angeles), not a GMT offset like GMT+0700.
I am making the assumption that Session.getScriptTimeZone() returns the appropriate zone. If not, then you might need to hard-code a specific zone, or use a different function to determine it.
Additionally, the +"00" in the script you had was assuming that all time zones use whole-hour offsets. In reality, there are several that have 30-minute or 45-minute offsets.

Javascript - Convert ####-##-## to Epoch time

Is there a way to take a date object from a HTML object in the format of ####-##-## and convert it to epoch time. For example, the user inputs the value of August 12, 2012 which shows as 2012-08-12 when I print out the .val() of it, and I need to get this in Epoch time.
EDIT
Code to date:
if (hvStartDate == "") {
hvStartDate = "start"
}
else {
console.log($("#hv-start-date").val()); // => 2012-08-20
hvStartDate = new Date($("#hv-start-date").val()).getTime(); // => NaN
}
if (hvEndDate == "") {
hvEndDate = "end"
}
else {
hvEndDate = new Date($("#hv-end-date").val()).getTime(); // => NaN
}
var myTmp = new Date("2012-08-20");
console.log(myTmp.getTime()); // => NaN
Javascript's Date built-in allows you to pass a date string into its constructor, giving you a Date based on that string. From there, calling getTime( ) will give you the epoch time.
new Date($('.user-value').val()).getTime(); // => epoch time
new Date('2012-08-12').getTime(); // 1344729600000
Caveat: Beware of locale strings and locale-specific date formatting (for example, the position of days and months switch depending on locale).
EDIT: Based on your code in the comment below, here's what you need to do. Notice that you have to instantiate a new Date Object before calling getTime():
if (hvStartDate == "") {
hvStartDate = "start"
}
else {
hvStartDate = new Date($("#hv-start-date").val()).getTime();
}
Simply use the getTime() function. It returns the number of milliseconds since Epoch :
var msSinceEpoch = myDate.getTime();
Complete Date reference at MDN : https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Date
EDIT : if you have to parse it too, you may :
use new Date(theString) if it has the good format
set yourself the different date fields (see reference) after having parsed it
use a date parsing library. I use this one : http://www.datejs.com/ which is very powerful for all date parsing, computing and formating.

Categories

Resources