My application heavily relies on time calculation and I read online that GCP App Engine uses UTC time zone.
So I made these calculations and on my local dev, the calcTime is totally different than App Engine, App Engine is giving me a big -negative number I'm not sure why??
const endTime = Math.floor(new Date(filteredTimeLeft).getTime() / 1000);
const convertSec = Number(coin.seconds);
const dateNow = Math.floor(Date.now() / 1000);
const calcTime = endTime - convertSec - dateNow;
Isn't my code suppose to work universally , independent where the server is located at?
Related
There is always a miscalculation when deployed on aws server.
Context:
All the dates stored in mongoDB are in UTC format. And I want to convert the dates to IST before I export them in excel.
My code works perfectly fine on my local machine, but when deployed on server, it fails.
Here is an example that will help understand the issue.
const myDate = new Date('2022-12-21T18:41:18.384+00:00');
// The value of convertedDate should be 2022/12/21
// But, it for some reason returns - 2022/12/22
const convertedDate = myDate.toLocaleDateString('en-GB');
I also tried to user external library date-fns to handle the issue. Following is the sample code:
const zonedDate = utcToZonedTime(myDate, 'Asia/Kolkata');
// this is also - 2022/12/22
const convertedDate = formatInTimeZone(zonedDate, 'Asia/Kolkata', 'yyyy-MM-dd');
);
Another variation for date-fns that I tried was:
const zonedDate = utcToZonedTime(
new Date(myDate.valueOf() + myDate.getTimezoneOffset() * 60 * 1000),
'Asia/Kolkata',
);
// this is also - 2022/12/22
const convertedDate = formatInTimeZone(zonedDate, 'Asia/Kolkata', 'yyyy-MM-dd');
Please note that locally, everything works perfectly fine.
I also tried to use external library date-fns to handle the issue.
It seems you converted back-and-forth, or even converted twice (for double the timezone offset?). You should not need to use utcToZonedTime at all. Just pass your original myDate timestamp into formatInTimeZone:
const convertedDate = formatInTimeZone(myDate, 'Asia/Kolkata', 'yyyy-MM-dd');
locally, everything works perfectly fine
Change your system timezone to be that of the server (most likely UTC) and you should be able to reproduce the issue. For node.js, start it with the TZ environment variable.
My application heavily relines on time being on pacific time
If I deploy my application on Google App Engine (US-WEST region), the time that is calculated by the cloud app engine is different than my local machine.
I've tried using Moment-Timezone to make it force the cloud server machine to use los angeles time format however its still giving me a different time than my local machine.
Is there a way to use moment so that the time values are consistent no matter where its deployed? Or did I implement the moment-time zone incorrectly?
const test = moment(new Date(filteredTimeLeft))
const endTime = test.tz('America/Los_Angeles').unix();
console.log('endTime', endTime)
const convertSec = Number(coin.seconds);
console.log('convertSec', convertSec)
const newDate = moment(new Date())
const calcTime = endTime - convertSec - newDate.tz('America/Los_Angeles').unix();
Not sure how you came with usin the plain JS new Date along with moment, nor the use of unix().
And I don't know what coin.seconds can be...
Anyway, here an example showing
the plain JS date
the UTC time
your local time
the Los Angeles time
the New York time
let js_date = new Date();
console.log("Plain JS date", js_date);
let moment_UTC = moment().utc();
console.log("UTC time", moment_UTC.format("hh:mm:ss"));
let moment_local = moment();
console.log("Your local time", moment_local.format("hh:mm:ss"));
let moment_Los_Angeles = moment().tz('America/Los_Angeles');
console.log("Los Angeles", moment_Los_Angeles.format("hh:mm:ss"));
let moment_New_York = moment().tz('America/New_York');
console.log("New York", moment_New_York.format("hh:mm:ss"));
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.1/moment.min.js"></script>
<script src="https://momentjs.com/downloads/moment-timezone-with-data.js"></script>
I am using node-redis. I have a cron job that updates my db and I have redis cache that caches the db response.
The problem I'm having is that my cron job runs everyday at 12am, however I can only set redis cache to expire in x seconds from now. Is there a way to make node-redis cache expire everyday at 12am exactly. Thanks.
Code:
const saveResult = await SET_CACHE_ASYNC('cacheData', response, 'EX', 15);
yes, you can use https://redis.io/commands/expireat command, if you use https://www.npmjs.com/package/redis package as redis driver, code will be like this
const redis = require('redis')
const client = redis.createClient();
const when = (Date.now()+24*60*60*1000) / 1000;
client.expireat('cacheData', when, function(error){....};
``
Recently I had the same problem with an application. My work around was creating a new timespan based on the time difference between my set and expiration time. Here is my code:
private TimeSpan GetTimeSpanUntilNextDay(int hour)
=> new DateTime(DateTime.Now.Date.AddDays(1).Ticks).AddHours(hour) - DateTime.Now;
Using the stack exchange lib for a 6 AM absolute expirition time the code looks like so:
public async Task<bool> SetEntranceAsync(string key, MYTYPE unserializedObject)
{
var db = _multiplexer.GetDatabase();
var jsonValue = JsonConvert.SerializeObject(unserializedObject);
return await db.StringSetAsync(key, jsonValue, GetTimeSpanUntilNextDay(6));
}
I used C# but you should be able to do the trick in any language.
I've been doing a test to compare the speeds at which the Google BigQuery Python client library downloads query results compared to the Node JS library. It would seem that, out-of-the-box, the Python libraries download data about twice as fast as the Javascript Node JS client. Why is that so?
Below I provide the two tests, one in Python and one in Javascript.
I've selected the usa_names public dataset of BigQuery as an example. The usa_1910_current table in this dataset is about 6 million rows and about 180Mb in size. I have a 200Mb fibre download link (for information about the last mile). The data, after being packed into a pandas dataframe, is about 1.1Gb (with Pandas overhead included).
Python test
from google.cloud import bigquery
import time
import pandas as pd
bq_client = bigquery.Client("mydata-1470162410749")
sql = """SELECT * FROM `bigquery-public-data.usa_names.usa_1910_current`"""
job_config = bigquery.QueryJobConfig()
start = time.time()
#---------------------------------------------------
query_job = bq_client.query(
sql,
location='US',
job_config=job_config)
#---------------------------------------------------
end = time.time()
query_time = end-start
start = time.time()
#---------------------------------------------------
rows = list(query_job.result(timeout=30))
df = pd.DataFrame(data=[list(x.values()) for x in rows], columns=list(rows[0].keys()))
#---------------------------------------------------
end = time.time()
iteration_time = end-start
dataframe_size_mb = df.memory_usage(deep=True).sum() / 1024 ** 2
print("Size of the data in Mb: " + str(dataframe_size_mb) + " Mb")
print("Shape of the dataframe: " + str(df.shape))
print("Request time:", query_time)
print("Fetch time:", iteration_time)
Node JS test
// Import the Google Cloud client library
const {BigQuery} = require('#google-cloud/bigquery');
const moment = require('moment')
async function query() {
const bigqueryClient = new BigQuery();
const query = "SELECT * FROM `bigquery-public-data.usa_names.usa_1910_current`";
const options = {
query: query,
location: 'US',
};
// Run the query as a job
const [job] = await bigqueryClient.createQueryJob(options);
console.log(`Job ${job.id} started.`);
// Wait for the query to finish
let startTime = moment.utc()
console.log('Start: ', startTime.format("YYYY-MM-DD HH:mm:ss"));
const [rows] = await job.getQueryResults();
let endTime = moment.utc()
console.log('End: ', endTime.format("YYYY-MM-DD HH:mm:ss"));
console.log('Difference (s): ', endTime.diff(startTime) / 1000)
}
query();
Python library test results with 180Mb of data:
Size of the data in Mb: 1172.0694370269775 Mb
Shape of the dataframe: (6028151, 5)
Request time: 3.58441424369812
Fetch time: 388.0966112613678 <-- This is 6.46 mins
Node JS library test results with 180Mb of data:
Start: 2019-06-03 19:11:03
End: 2019-06-03 19:24:12 <- About 13 mins
For further reference, I also ran the tests against a 2Gb table...
Python library test results with 2Gb of data:
Size of the data in Mb: 3397.0339670181274 Mb
Shape of the dataframe: (1278004, 21)
Request time: 2.4991791248321533
Fetch time: 867.7270500659943 <-- This is 14.45mins
Node JS library test results with 2Gb of data:
Start: 2019-06-03 15:30:59
End: 2019-06-03 16:02:49 <-- The difference is just below 31 mins
As I can see Node JS uses pagination to manage the datasets while Python with looks like it brings the entire datasets and start to work with it.
This is maybe affecting the performance of the Node JS client library, my recommendation is to take a look at the source code of both clients and read constantly the Google Cloud Blog, where Google publishes sometimes tips and best practices to use their products, as an example this article: Testing Cloud Pub/Sub clients to maximize streaming performance.
I am using AngularJS client-side application and working with date and time. My problem is that local system date anyone can change, to protect this I want to use server date without any get or post request. Is there any way to get server date using JavaScript?
If you are running JavaScript at the client-side, the only way to find the server time is asking the server what is the current time. There is no magic here. You need to make a request to the server.
Options:
Use AJAX or Fetch.
If the HTML page is rendered in the server, you can write the current time during the page render and send it to client.
Please, note that it is not possible to have a precise time of the server due to network delays, but you can get pretty close using the code from this answer (modified):
var offset = 0;
function calcOffset() {
var xmlhttp = new XMLHttpRequest();
xmlhttp.open("GET", "https://stackoverflow.com/", false);
xmlhttp.send();
var dateStr = xmlhttp.getResponseHeader('Date');
var serverTimeMillisGMT = Date.parse(new Date(Date.parse(dateStr)).toUTCString());
var localMillisUTC = Date.parse(new Date().toUTCString());
offset = serverTimeMillisGMT - localMillisUTC;
}
function getServerTime() {
var date = new Date();
date.setTime(date.getTime() + offset);
return date;
}