Develop Reference

Javascript Workers - why is the worker message treated so lately and can I do something against it?

I have a Worker that shares a SharedArrayBuffer with the "main thread". To work correctly, I have to make sure that the worker has access to the SAB before the main thread accesses to it. (EDIT: The code creating the worker has to be in a seperate function (EDIT2: which returns an array pointing to the SAB).) (Maybe, already this is not possible, you'll tell me).
The initial code looks like this:
function init() {
var code = `onmessage = function(event) {
console.log('starting');
var buffer=event.data;
var arr = new Uint32Array(buffer);// I need to have this done before accessing the buffer again from the main
//some other code, manipulating the array
}`
var buffer = new SharedArrayBuffer(BUFFER_ELEMENT_SIZE);
var blob = new Blob([code], { "type": 'application/javascript' });
var url = window.URL || window.webkitURL;
var blobUrl = url.createObjectURL(blob);
var counter = new Worker(blobUrl);
counter.postMessage(buffer);
let res = new Uint32Array(buffer);
return res;
}
function test (){
let array = init();
console.log('main');
//accessing the SAB again
};
The worker code is always executed after test(), the console shows always main, then starting.
Using timeouts does not help. Consider the following code for test:
function test (){
let array = [];
console.log('main');
setTimeout(function(){
array = initSAB();
},0);
setTimeout(function(){
console.log('main');
//accessing the SAB again
},0);
console.log('end');
};
The console shows end first, followed by main, followed by starting.
However, assigning the buffer to a global array outside the test() function does the job, even without timeouts.
My questions are the following:
why does the worker does not start directly after the message was send (= received?). AFAIK, workers have their own event queue, so they should not rely on the main stack becoming empty?
Is there a specification detailing when a worker starts working after sending a message?
Is there a way to make sure the worker has started before accessing the SAB again without using global variables? (One could use busy waiting, but I beware...) There is probably no way, but I want to be sure.
Edit
To be more precise:
In a completly parallel running scenario, the Worker would be able to
handle the message immediately after it was posted. This is obviously
not the case.
Most Browser API (and Worker is such an API) use a callback queue to handle calls to the API. But if this applied, the message would be
posted/handled before the timeout calbacks were executed.
To go even further: If I try busy waiting after postMessage by reading from the SAB until it changes one value will block the
program infinitely. For me, it means that the Browser does
not posts the message until the call stack is empty As far as
I know, this behaviour is not documentated and I cannot explain it.
To summerize: I want to know how the browser determines when to post the message and to handle it by the worker, if the call of postMessage is inside a function. I already found a workaround (global variables), so I'm more interested in how it works behind the scenes. But if someone can show me a working example, I'll take it.
EDIT 2:
The code using the global variable (the code that works fine) looks like this
function init() {
//Unchanged
}
var array = init(); //global
function test (){
console.log('main');
//accessing the SAB again
};
It prints starting, then main to the console.
What is also worth noticing : If I debug the code with the Firefox Browser (Chrome not tested) I get the result I want without the global variable (starting before main) Can someone explain?

why does the worker does not start directly after the message was sen[t] (= received?). AFAIK, workers have their own event queue, so they should not rely on the main stack becoming empty?
First, even though your Worker object is available in main thread synchronously, in the actual worker thread there are a lot of things to do before being able to handle your message:
it has to perform a network request to retrieve the script content. Even with a blobURI, it's an async operation.
it has to initialize the whole js context, so even if the network request was lightning fast, this would add up on parallel execution time.
it has to wait the event loop frame following the main script execution to handle your message. Even if the initialization was lightning fast, it will anyway wait some time.
So in normal circumstances, there is very little chances that your Worker could execute your code at the time you require the data.
Now you talked about blocking the main thread.
If I try busy waiting after postMessage by reading from the SAB until it changes one value will block the program infinitely
During the initialization of your Worker, the message are temporarily being kept on the main thread, in what is called the outside port. It's only after the fetching of the script is done that this outside port is entangled with the inside port, and that the messages actually pass to that parallel thread.
So if you do block the main thread before the ports have been entangled it won't be able to pass it to the worker's thread.
Is there a specification detailing when a worker starts working after sending a message?
Sure, and more specifically, the port message queue is enabled at the step 26, and the Event loop is actually started at the step 29.
Is there a way to make sure the worker has started before accessing the SAB again without using global variables? [...]
Sure, make your Worker post a message to the main thread when it did.
// some precautions because all browsers still haven't reenabled SharedArrayBuffers
const has_shared_array_buffer = window.SharedArrayBuffer;
function init() {
// since our worker will do only a single operation
// we can Promisify it
// if we were to use it for more than a single task,
// we could promisify each task by using a MessagePort
return new Promise((resolve, reject) => {
const code = `
onmessage = function(event) {
console.log('hi');
var buffer= event.data;
var arr = new Uint32Array(buffer);
arr.fill(255);
if(self.SharedArrayBuffer) {
postMessage("done");
}
else {
postMessage(buffer, [buffer]);
}
}`
let buffer = has_shared_array_buffer ? new SharedArrayBuffer(16) : new ArrayBuffer(16);
const blob = new Blob([code], { "type": 'application/javascript' });
const blobUrl = URL.createObjectURL(blob);
const counter = new Worker(blobUrl);
counter.onmessage = e => {
if(!has_shared_array_buffer) {
buffer = e.data;
}
const res = new Uint32Array(buffer);
resolve(res);
};
counter.onerror = reject;
if(has_shared_array_buffer) {
counter.postMessage(buffer);
}
else {
counter.postMessage(buffer, [buffer]);
}
});
};
async function test (){
let array = await init();
//accessing the SAB again
console.log(array);
};
test().catch(console.error);

According to MDN:
Data passed between the main page and workers is copied, not shared. Objects are serialized as they're handed to the worker, and subsequently, de-serialized on the other end. The page and worker do not share the same instance, so the end result is that a duplicate is created on each end. Most browsers implement this feature as structured cloning.
Read more about transferring data to and from workers
Here's a basic code that shares a buffer with a worker. It creates an array with even values (i*2) and it sends it to the worker. It uses Atomic operations to change the buffer values.
To make sure the worker has started you can just use different messages.
var code = document.querySelector('[type="javascript/worker"]').textContent;
var blob = new Blob([code], { "type": 'application/javascript' });
var blobUrl = URL.createObjectURL(blob);
var counter = new Worker(blobUrl);
var sab;
var initBuffer = function (msg) {
sab = new SharedArrayBuffer(16);
counter.postMessage({
init: true,
msg: msg,
buffer: sab
});
};
var editArray = function () {
var res = new Int32Array(sab);
for (let i = 0; i < 4; i++) {
Atomics.store(res, i, i*2);
}
console.log('Array edited', res);
};
initBuffer('Init buffer and start worker');
counter.onmessage = function(event) {
console.log(event.data.msg);
if (event.data.edit) {
editArray();
// share new buffer with worker
counter.postMessage({buffer: sab});
// end worker
counter.postMessage({end: true});
}
};
<script type="javascript/worker">
var sab;
self['onmessage'] = function(event) {
if (event.data.init) {
postMessage({msg: event.data.msg, edit: true});
}
if (event.data.buffer) {
sab = event.data.buffer;
var sharedArray = new Int32Array(sab);
postMessage({msg: 'Shared Array: '+sharedArray});
}
if (event.data.end) {
postMessage({msg: 'Time to rest'});
}
};
</script>

AngularJS view not updating after model change

I'm writing a web-based game that allows you to bid on cards and trade them with other players. For this application I'm using Node, Express, MongoDB and Angular.
The view shows the player avatars and names along with their connection status. This connection status is saved in the game model but also tracked real-time via socket.io websockets.
When a player joins the game, the server sends a message to all clients. The client then checks if the message is meant for them by comparing the game id.
socketService.on("playerConnection", function(data) {
if (data.gameId === gameId) {
handlePlayerConnection(data);
}
});
If the message is meant for our client, data is passed to function 'handlePlayerConnection'.
function handlePlayerConnection(data) {
if ($scope.game) {
for (var i = 0; i < $scope.game.players.length; i++) {
if ($scope.game.players[i].user.id === data.userId) {
$scope.game.players[i].isConnected = data.connected;
}
}
}
}
In the view, I simply bind the player avatar to 'isConnected', so it shows/hides when a player connects/disconnects.
Every now and then, the game model changes (obviously), and the server tells the clients (again, by socket) to refresh $scope.game.
Code below shows how the $scope.game is refreshed.
function getGame(gameId, callback) {
gameService.getGame(gameId, function(err, data) {
if (!err) {
$scope.game = data; // Set game
if (callback) callback(data);
}
});
}
At this point, the view is basically frozen. It completely stops responding to player connects and disconnects. Refreshing the browser or controller fixes the problem.
I've logged $scope.game.players to console, and it seems to contain the correct connection statuses. The view is simply not responding.
I have tried to implement $scope.$apply() in various ways and places but all that gives me is 'angular.js:13920 Error: [$rootScope:inprog] $apply already in progress'. Wrapping $scope.$apply() in $timeout() gets rid of aforementioned error, but does not solve the problem.
After spending hours trying to fix this, I might have a case of tunnelvision. Endless kudos to anyone who can help me.
EDIT: Just realised all socket data is handled through a $rootScope.apply(). Full code:
var socket = io.connect("http://localhost:8180/");
return {
on: function(eventName, callback) {
socket.on(eventName, function() {
var args = arguments;
$rootScope.$apply(function() {
callback.apply(socket, args);
});
});
},
emit: function(eventName, data, callback) {
socket.emit(eventName, data, function() {
var args = arguments;
$rootScope.$apply(function() {
if (callback) {
callback.apply(socket, args);
}
});
})
},
off: function(eventName) {
socket.off(eventName);
}
};
Now I'm even less sure what to do though..

if (!$scope.$$phase) {
$scope.game = data; // Set game
if (callback) callback(data);
}
This will trigger a digest if one is not currently in progress. Otherwise, your code will get updated by the running digest.

Finally fixed it...
I forgot I was using a directive as a 'player'. For lazy reasons, instead of creating an isolated scope and passing the player object, I assigned the player object to the scope in the following way:
link: function(scope, elem, attrs) {
/*
Decided not to use an isolated scope because the
player directive needs access to the $rootScope.
The solution below eliminates the need of said scope.
*/
scope.player = scope.$eval(attrs.player);
}
As you see there is even a comment explaining my idiotic decision.

How to reconnect to socket after disconnecting in html5

I'm working with HTML5 socket functions to establish a socket connection to my server. HTML5 has functions below to handle disconnecting
Socket.onclose = function()
{
...
}
Socket.onerror = function()
{
...
}
My problem is, how try for reconnect after onclose function executes? I tried to put a while loop inside of it like
ws.onclose = function()
{
While(conn==0)
{
ws = new WebSocket("ws://example.com");
}
}
and
ws.onopen = function()
{
conn=1;
...
}
But did't work.
Any idea?

Here's the script that comes with the Plezi websocket framework... It's fairly basic, but it works on the browsers I used it on (Safari, Chrome and FireFox).
The trick is to leverage the onclose method WITHOUT a loop.
The onclose method will be called even if the websocket never opened and the connection couldn't be established (without calling onopen).
Initiating a reconnect within an onclose is enough.
Writing a loop or a conditional review will not only fail, but will halt all the scripts on the page. Allow me to explain:
Javascript is single threaded. Again: it's an even/task based, single threaded, environment.
This means that your code acts like an atomic unit - nothing happens and nothing changes until your code finished running it's course.
Because connections could take a while to establish, the new WebSocket was designed (and rightfully so) as an asynchronous function.
This is how come you can define the onopen event callback AFTER the creation of the event.
The new websocket connection will be attempted only once the current task/event is finished...
...so a loop will get you stuck forever waiting for a task that can't be performed until your code stops running...
Back to the issue at hand, here's the code. If you have any ideas for improvements, please let me know:
// Your websocket URI should be an absolute path. The following sets the base URI.
// remember to update to the specific controller's path to your websocket URI.
var ws_controller_path = window.location.pathname; // change to '/controller/path'
var ws_uri = (window.location.protocol.match(/https/) ? 'wss' : 'ws') + '://' + window.document.location.host + ws_controller_path
// websocket variable.
var websocket = NaN
// count failed attempts
var websocket_fail_count = 0
// to limit failed reconnection attempts, set this to a number.
var websocket_fail_limit = NaN
// to offer more or less space between reconnection attempts, set this interval in miliseconds.
var websocket_reconnect_interval = 250
function init_websocket()
{
if(websocket && websocket.readyState == 1) return true; // console.log('no need to renew socket connection');
websocket = new WebSocket(ws_uri);
websocket.onopen = function(e) {
// reset the count.
websocket_fail_count = 0
// what do you want to do now?
};
websocket.onclose = function(e) {
// If the websocket repeatedly you probably want to reopen the websocket if it closes
if(!isNaN(websocket_fail_limit) && websocket_fail_count >= websocket_fail_limit) {
// What to do if we can't reconnect so many times?
return
};
// you probably want to reopen the websocket if it closes.
if(isNaN(websocket_fail_limit) || (websocket_fail_count <= websocket_fail_limit) ) {
// update the count
websocket_fail_count += 1;
// try to reconect
setTimeout( init_websocket, websocket_reconnect_interval);
};
};
websocket.onerror = function(e) {
// update the count.
websocket_fail_count += 1
// what do you want to do now?
};
websocket.onmessage = function(e) {
// what do you want to do now?
console.log(e.data);
// to use JSON, use:
// var msg = JSON.parse(e.data); // remember to use JSON also in your Plezi controller.
};
}
// setup the websocket connection once the page is done loading
window.addEventListener("load", init_websocket, false);

How to determine that an SSE connection was closed?

We have an SSE (Server-Sent Events) connection open in JavaScript which can time to time get closed, either because of server restarts or other causes. In that case it would be good to reestablish the connection. How to do it? Is there a way to find out on the client side that the connection was closed?
Here: https://developer.mozilla.org/en-US/docs/Web/API/EventSource I found only a way to close the connection, but no callback or a test method for determining whether the connection is still alive.
Thank you for your help.

If the connection is closed (in such a way that the browser can realize it), it will auto-connect. And it tends to do this quickly (default is 3 seconds in Chrome, 5 seconds in Firefox). readyState will be CONNECTING (0) while it is doing this. It is only ever CLOSED (2) if there was some problem connecting in the first place (e.g. due to a CORS issue). Once CLOSED, it does not retry.
I prefer to add a keep-alive mechanism on top, as the browser cannot always detect dead sockets (not to mention a remote server process that is locked up, etc.). See ch.5 of Data Push Apps with HTML5 SSE for detailed code, but basically it involves having the server send a message every 15 seconds, then a JavaScript timer that runs for 20 seconds, but is reset each time a message is received. If the timer ever does expire, we close the connection and reconnect.

EventSource API Update
EventSource API now has three event handlers:
onerror
onmessage
onopen
These should be enough to handle everything you need on the client side.
Something like this:
const ssEvent = new EventSource( eventUrl );
ssEvent.onopen = function (evt) {
// handle newly opened connection
}
ssEvent.onerror = function (evt) {
// handle dropped or failed connection
}
ssEvent.onmessage = function (evt) {
// handle new event from server
}
Ref: mozilla.org : EventSource : Event handlers
Browser support for EventSource API: onopen - caniuse.com

Check readyState property:
var es = new EventSource();
// Сheck that connection is not closed
es.readyState !== 2;
// or
es.readyState !== EventSource.CLOSED;

It is best not to try to determine if the connection was closed. I do not think there is a way to do it. Server Side Events work differently in all of the browsers, but they all close the connection during certain circumstances. Chrome, for example, closes the connection on 502 errors while a server is restarted. So, it is best to use a keep-alive as others suggest or reconnect on every error. Keep-alive only reconnects at a specified interval that must be kept long enough to avoid overwhelming the server. Reconnecting on every error has the lowest possible delay. However, it is only possible if you take an approach that keeps server load to a minimum. Below, I demonstrate an approach that reconnects at a reasonable rate.
This code uses a debounce function along with reconnect interval doubling. It works well, connecting at 1 second, 4, 8, 16...up to a maximum of 64 seconds at which it keeps retrying at the same rate.
function isFunction(functionToCheck) {
return functionToCheck && {}.toString.call(functionToCheck) === '[object Function]';
}
function debounce(func, wait) {
var timeout;
var waitFunc;
return function() {
if (isFunction(wait)) {
waitFunc = wait;
}
else {
waitFunc = function() { return wait };
}
var context = this, args = arguments;
var later = function() {
timeout = null;
func.apply(context, args);
};
clearTimeout(timeout);
timeout = setTimeout(later, waitFunc());
};
}
// reconnectFrequencySeconds doubles every retry
var reconnectFrequencySeconds = 1;
var evtSource;
var reconnectFunc = debounce(function() {
setupEventSource();
// Double every attempt to avoid overwhelming server
reconnectFrequencySeconds *= 2;
// Max out at ~1 minute as a compromise between user experience and server load
if (reconnectFrequencySeconds >= 64) {
reconnectFrequencySeconds = 64;
}
}, function() { return reconnectFrequencySeconds * 1000 });
function setupEventSource() {
evtSource = new EventSource(/* URL here */);
evtSource.onmessage = function(e) {
// Handle even here
};
evtSource.onopen = function(e) {
// Reset reconnect frequency upon successful connection
reconnectFrequencySeconds = 1;
};
evtSource.onerror = function(e) {
evtSource.close();
reconnectFunc();
};
}
setupEventSource();

How to reconnect to websocket after close connection [duplicate]

This question already has answers here:
Reconnection of Client when server reboots in WebSocket
(9 answers)
Closed 2 years ago.
I construct my websocket connection with this code (e.g.):
var socket = new WebSocket("ws://94.12.176.177:8080");
And I close the connection with this one:
socket.close();
But how do I reestablish connection?
I've done some research and tried several methods. This question could not help me: Socket.io reconnect on disconnect?
It's the only result which close to what I'm looking for.
The reason I want to do this is to allow users to stop sending data to the web temporary, and resending again after a period of time. Without reconnection, user have to refresh the page in order to resend. This may cause some data lost. Thank you.

When the server closes the connection, the client does not try to reconnect. With some JS frameworks maybe, but the question was, at the time of this answer, tagged as plain Vanilla JS.
I'm a bit frustrated because the accepted, upvoted answer is plainly wrong, and it cost me some additional time while finding the correct solution.
Which is here: Reconnection of Client when server reboots in WebSocket

I found a great solution on this page: sam-low.com
Once the original connection has been closed, you need to create a new WebSocket object with new event listeners
function startWebsocket() {
var ws = new WebSocket('ws://localhost:8080')
ws.onmessage = function(e){
console.log('websocket message event:', e)
}
ws.onclose = function(){
// connection closed, discard old websocket and create a new one in 5s
ws = null
setTimeout(startWebsocket, 5000)
}
}
startWebsocket();
Note that if there’s a problem reconnecting, the new WebSocket object will still receive another close event, meaning that onclose() will be executed even if it never technically opened. That’s why the delay of five seconds is sensible - without it you could find yourself creating and destroying thousands of websocket connections at a rate that would probably break something.

NOTE: The question is tagged socket.io so this answer is specifically regarding socket.io.
As many people have pointed out, this answer doesn't apply to vanilla websockets, which will not attempt to reconnect under any circumstances.
Websockets will not automatically try to reconnect. You'll have to recreate the socket in order to get a new connection. The only problem with that is you'll have to reattach your handlers.
But really, websockets are designed to stay open.
A better method would be to have the server close the connection. This way the websocket will fire an onclose event but will continue attempting to make the connection. When the server is listening again the connection will be automatically reestablished.

Flawless implementation:
var socket;
const socketMessageListener = (event) => {
console.log(event.data);
};
const socketOpenListener = (event) => {
console.log('Connected');
socket.send('hello');
};
const socketCloseListener = (event) => {
if (socket) {
console.error('Disconnected.');
}
socket = new WebSocket('ws://localhost:8080');
socket.addEventListener('open', socketOpenListener);
socket.addEventListener('message', socketMessageListener);
socket.addEventListener('close', socketCloseListener);
};
socketCloseListener();
To test it:
setTimeout(()=>{
socket.close();
},5000);
Edit: Take note of the Exponential Backoff implementation (at the linked thread by top comment: https://stackoverflow.com/a/37038217/8805423), not in above code BUT VERY VERY CRUCIAL.
Edit again: Check out back from primus: https://www.npmjs.com/package/back, it's a flexible sexy implementation.

function wsConnection(url){
var ws = new WebSocket(url);
var s = (l)=>console.log(l);
ws.onopen = m=>s(" CONNECTED")
ws.onmessage = m=>s(" RECEIVED: "+JSON.parse(m.data))
ws.onerror = e=>s(" ERROR")
ws.onclose = e=>{
s(" CONNECTION CLOSED");
setTimeout((function() {
var ws2 = new WebSocket(ws.url);
ws2.onopen=ws.onopen;
ws2.onmessage = ws.onmessage;
ws2.onclose = ws.onclose;
ws2.onerror = ws.onerror;
ws = ws2
}
).bind(this), 5000)
}
var f = m=>ws.send(JSON.stringify(m)) || "Sent: "+m;
f.ping = ()=>ws.send(JSON.stringify("ping"));
f.close = ()=>ws.close();
return f
}
c=new wsConnection('wss://echo.websocket.org');
setTimeout(()=>c("Hello world...orld...orld..orld...d"),5000);
setTimeout(()=>c.close(),10000);
setTimeout(()=>c("I am still alive!"),20000);
<pre>
This code will create a websocket which will
reconnect automatically after 5 seconds from disconnection.
An automatic disconnection is simulated after 10 seconds.