I built an HTML5 multiplayer game that depends on having a reasonably accurate time sync between server and client. For the most part, the algorithm I use is very accurate -- all it does is estimate what the client-server time delta is, i.e. the difference between the current time on the server and the current time on client. For example, if the server time is exactly 5 seconds ahead of the client time, the time delta is 5000 ms.
The client and server (node.js) are both written in Javascript. The algorithm works as follows:
Record time on the client:
var clientTime = Date.now();
Ping the server. When the server receives the message, it immediately sends a response containing just one thing: the time on the server when the message was received.
var serverTime = Date.now();
// Send serverTime to the client
When the client receives the server response, immediately record the time:
var clientTime2 = Date.now();
Now, we know that when the server received the message, the client time must have been somewhere between clientTime and clientTime2.
If the server received the message when client time was clientTime (i.e. client->server request took 0ms somehow), then the time delta is
var delta1 = (serverTime - clientTime);
If the server received the message when client time was clientTime (i.e. server->client response took 0ms somehow), then the time delta is
var delta2 = (serverTime - clientTime2).
Thus we can safely say that the time delta is somewhere between delta1 and delta2. Now, repeat this process a bunch of times, each time narrowing the range based on whatever results you got, and you can get a pretty good estimate of the time delta.
I've tested this hundreds of times on 7 different browsers and multiple machines and have never had any issue with it. It's never been inconsistent.
The issue, though, is that my server logs show that, every now and then, a few people will get wildly inconsistent time sync results. Here is an actual example of one player's time sync:
The client went through 74 cycles of the above algorithm and successfully narrow the range of possible time deltas to: [-186460, -186431] without a single inconsistency. 29ms accuracy.
On the 75th cycle, possibly a few seconds after the 74th cycle, the client calculated the range of possible time deltas to be: [-601, -596]. 5ms accuracy, except for it's extremely inconsistent with the past 74 cycles: it's 3 minutes off!
I would blame this on crazy edge cases, except it happens almost 100 times a day... how could this happen? Is there any possible error when using Date.now()?
performance.now() instead of Date.now(), because performance.now() is monotonically increasing and not subject to clock drift. See the comments, thanks to everyone for their help!
Your difficulty is that you depend on estimating round-trip times to the server, over an Internet that has variance in round-trip times. Sometimes that variance will be unexpectedly substantial, as in cases where temporary congestion and large router buffers delay a single message far longer than normal. (Cf "bufferbloat".)
Your second difficulty is that you are using self-reported client times, and that means that if a client has a clock that's weird it will look to you like a round-trip estimation gone wrong. As another poster noted, internet time protocols will sometimes slew a clock rapidly to correct from local timekeeping anomalies.
What it sounds like is that you need some filtering in your code that takes into account previous results so that when you get an anomalous result you don't immediately accept it as true.
Related
This is not a post about HOW to change channel's name (I know it).
I have an international server using several bots. And we all depend on UTC time (to coordinate through the world). So there was borned a solution to make a time-bot which will show current UTC-time in the dedicated channel nobody can visit. And yes, precision is necessary, even seconds.
I created a voice channel with permissions not to join for #everyone. Everything worked fine, it updated every 1000 ms. Then (after several months of good work) something was broke, it started updating incorrect. I've increased update interval up to 5000 ms and it have started to work fine... until yesterday.
Now it doesn't work anymore. Even if I increase interval much more. It works sometimes I don't really know what the interval is, it's huge and unpredictable.. the time-bot is broken for now and cannot be used anymore in that case.
Is there any restrictions for updating channel name? I can't find any information about this in available documentations.
Client.setInterval(() => {
const { h, m, s } = getTime();
channel.edit({ name: `${getClockEmoji({ h, m })} UTC: ${h}-${m}-${s}` }).catch((err) => console.log(err));
}, updateInterval);
Providing data is correct, 'cause I send it to console and it updates as good as I need in interval I set. But channel name not updates that often..
Does discord filter too often update requests?
discord.js version is v.12.2.0
Discord had set the rate limit for things like channelrename to 2 requests per 10 minutes.
"Normal" requests like sendmessage is limited to 10,000 per 10 minutes.
This seems to likely be an issue directly related to rate limiting:
https://discord.com/developers/docs/topics/rate-limits
IP addresses that make too many invalid HTTP requests are automatically and temporarily restricted from accessing the Discord API. Currently, this limit is 10,000 per 10 minutes. An invalid request is one that results in 401, 403, or 429 statuses.
For every API request made, we return optional HTTP response headers containing the rate limit encountered during your request.
You should probably decrease the interval by a considerable amount to reduce the risk of the IP being restricted.
I have a websocket client that receives 200-300 messages per second from a websocket stream. My JavaScript client that receives the messages is doing a few DOM manipulations with each message received. I'm afraid that after a few minutes of running the app in my browser, the processing of the messages could fall behind. Like a chat application that gets overwhelmed with incoming messages and after a while the latency between receiving the websocket message and displaying it to the user grows and grows... Another example would be a real-time stock market page, but after being open for a few minutes, the real-time prices aren't real-time anymore...
How do I determine if my browser is keeping up with the incoming websocket messages?
UPDATE
I ended up having every message update a clock on the page to see if it ever fell behind:
function onMessage(evt)
{
var dt = new Date();
$("#clock").text( dt.toLocaleTimeString() ); // "8:43:55 PM"
// other DOM manipulations related to each websocket message ...
}
If you mean how do you determine it when debugging, then you can look at your CPU utilization to see if you're swamping your CPU.
If you mean how to you determine it live from within your Javascript browser then I can think of some ideas:
Put a server time stamp in each message. Then, when you start processing messages, calculate the diff between current system time and the time stamp in the message. If that diff is going up and up, then the client is getting behind. If it's staying relatively constant, then the client is keeping up.
You can give yourself some idea about how back-logged your server is with something like this:
code:
let start = Date.now();
setTimeout(function() {
console.log(Date.now() - start);
}, 0);
If that outputs a larger number, then your event queue is backed up some number of milliseconds.
I'm trying to develop a simple LiveDateTime object that's reliable, accurate and somewhat independent of the client system date as far as the updating of the time goes.
I've tried several solutions:
Taking the server date and the time difference with the client system date to calculate the elapsed time and update the time accordingly every 150ms.
Taking the client system UTC date and increase it with a second every 1000ms with setInterval() or setTimeout(). Also tried it with a self-correcting delay loop.
Some similar variants of the above.
All of the above didn't meet the requirements, either because if the client system date was changed it would set the live time off as well, or just the delay of the loop was inaccurate and drifting off over time.
So, I decided to just go online and search for some popular websites that have what I want already implemented and working. I stumbled upon this Greenwich Mean Time example that's reliable, accurate and somewhat independent of the client system. If I change the date of my system, the date that they display is unaffected.
Upon inspecting their code I found it hard trying to understand what they are doing that I could replicate, because either their code is badly written or I'm not that advanced enough to understand it.
I do know that they use AJAX calls (request.js) to the server for something (maybe the server time). And that they're using a setInterval(func, 150) and a whole bunch of other presentational methods of which I think aren't the core of the live time's functionality.
Anyone who could help me figuring out the core of the functionality of their accurate live time? If I can figure that out, from there it would be easy for me to replicate.
The example is combined with a server-side language to get the server time. The time is directly written into the JS by PHP, in this line for example:
var ServerDSTCheck = new Date(parseFloat(1426849548940.7)).getTime();
That float 1426849548940.7 comes from the server, and that's why it's not affected by the client machine time. The AJAX call returns the same thing in intervals to maintain the server time, so it can't be messed up on the client and to avoid the client lag.
That's the base of what you need. The rest of the code is about daylight savings, timezone compensation and presentational stuff.
As I understand, you want independent clock in JS, right? It's pretty simple:
Make an ajax request to your server, while the script is loaded by the user, to get current time (UNIX).
Set interval for example 1 sec, and just increment your time, that you get in pt 1, for the interval you choose.
When user change his time on local machine it wouldn't affect the script, also refreshing script would again load time from your server.
Since js timers are extremely unreliable by default I'd say you must use two parallel intervals.
One, which will correct the frontend's time drift against server time and other which will fill in the blanks on the frontend (to not strain the server and user's connection).
var time = Date.now(); // Or initial server time
var oldTime = Date.now();
var newTime = oldTime;
var feInterval = setInterval( function() {
// Incrementing "time" by whatever frontend difference, aka Delta time
time += ( newTime = Date.now() ) - oldTime;
oldTime = newTime;
}, 40 );
var beInterval = setInterval( function() {
// Fetch server UTC milliseconds and overwrite "time".
// Above interval will just use that as a new base time.
var serverTime = Math.random() * 10000000000; //use Ajax to get server time.
time = serverTime;
}, 10000 );
setInterval( function() {
// Do something with "time"
// This interval is just for demo, you don't need it in general - use your own "time" consumer.
console.log( 'Fairly correct time (random each 10 seconds):', new Date ( time ).toString() );
}, 40 );
I guess there is someone else trying solve the same problem,
check out below page,
show server time including timezone offset
Maximum HTTP Request execution time is 120sec in Node.js , how can i increase max execution time on http get/post requests?
NOTE:I'm using expressjs framework
http://nodejs.org/api/http.html#http_server_timeout
server.timeout
Number Default = 120000 (2 minutes)
The number of milliseconds of inactivity before a socket is presumed to have timed out.
Note that the socket timeout logic is set up on connection, so changing this value only affects new connections to the server, not any existing connections.
Set to 0 to disable any kind of automatic timeout behavior on incoming connections.
You would use server.setTimeout(0); to get what you're after. It's documented here. Although I can't figure out why you would need to change this except for troubleshooting.
I have an area in my page where messages go when a database has changed. Now, some days the database will change so much that a new message is displayed every 10 minutes; other days it will change only a few times. The issue I am having is that the EventSource seems to time out after 1hr 22 minutes, and no longer will the browser receive notifications.
I am wondering if there is a way to keep EventSources persistent (i.e., for as long as the browser is displaying the page, the EventSource is alive). According to what I have found in my Google searches, EventSources should remain alive until the tab/window is closed. Unfortunately, there seems to be so very little that I find in my Google searches, and for me this doesn't seem to be the case.
You don't say where the socket closure is happening (on the browser, socket on client machine, socket on server-side, etc.) but it doesn't really matter as the fix is the same for all of them: send keep-alive messages.
The server should send a keep-alive message. Either every, say, 15 seconds; or only after 15 seconds of inactivity. (Whichever is easier to code, server-side, for you.) It can be as simple as an SSE comment: ":\n\n" (lines starting with colons are ignored). I prefer to send actual data, because:
You get to see a message, allowing client-side keep-alive checking (see below)
There is bound to be something useful you want to send, like a timestamp (for a check that client/server clocks are in sync), or metrics, etc.
On the client-side, run a timer with setTimeout() set to 20 seconds. Each time you receive any data from the server (whether genuine data, or your keep-alive), kill the timer, and start it again. Therefore the only time the time-out function will get called is if your server went more than 20 seconds without sending you anything. When that happens, kill the connection and reconnect.
The above is assuming the problem is at the socket-level. The problem might instead be the browser is crashing: perhaps it has run out of memory. The fix I'd do in that case is a once/hour timer (setTimeout() in JavaScript), to manually close and re-open the EventSource connection. Or clear out some memory buffers you might be using. A bit of profiling with FireBug or Chrome tools will tell you if you have a memory problem.
Plug: Over half of the "Making our App production quality" chapter in my coming-soon SSE book is about keep-alive and using LastId on the reconnect. Please buy when it comes out :-)
I had the same problem with Chrome reporting "net::ERR_SPDY_PROTOCOL_ERROR 200" every two minutes.
Sending SSE comments every minute solved the problem for me. See the Nodejs / Express example code below.
exports.addWebServices = function(app) {
app.get('/ws/clientEvent', function(req, res) {
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive'
})
/* Event handlers for SSE here */
let keepAliveMS = 60 * 1000;
function keepAlive() {
// SSE comment for keep alive. Chrome times out after two minutes.
res.write(':\n\n');
setTimeout(keepAlive, keepAliveMS);
}
setTimeout(keepAlive, keepAliveMS);
}
}