Can I miss web socket events if I reassign ws.onmessage? - javascript

I am receiving data from a rest api via the function callRestApi, below. However, I receive updates on the rest data received via a websocket and I want to ensure that I dont miss anything. Therefore I start buffering websocket events prior to calling the rest endpoint. Having received the rest response I dispatch the buffered events and start dispatching anything new that is subsequently received, updating my copy of the data. But have I implemented this correctly? In particular, is there a risk that I could miss an event in the function startDispatchingEvents where I assign ws.onmessage a new value. I am using a redux dispatcher.
export const startBufferingEvents = () => {
eventBuffer = [];
ws.onmessage = msg => {
eventBuffer.push(msg);
}
};
export const startDispatchingEvents = (dispatcher) => {
eventBuffer.forEach(evt => dispatcher(evt));
ws.onmessage = evt => dispatcher(evt);
};
startEventBuffering()
.then(()=>callRestApi())
.then(restResponse=>dispatch(action(restResponse)))
.then(()=>startDispatchingEvents((evt)=>eventDispatcher(dispatch, evt)))
};

It shouldn't be a problem. All the code that is executed in JavaScript (assuming you're running this in a browser) is single-threaded. That means you cannot run into race conditions or any other concurrency issues. When the callback that replaces the value of onmessage is executed, access to the variable by, say, a websocket message, will wait (a.k.a. go into the event queue) until the callback execution finishes.
This might not be particularly related to the question, but if you are interested in learning a bit more about the event handling system of JS and the internals of the engine, I found this talk quite good: https://youtu.be/8aGhZQkoFbQ

Related

Javascript Canceling Function that contains list of await tasks

I was wondering if there is any to cancel / stop execution of a javascript function that contains multiple await functions. Due to the nature of promises and their lack of proper cancellations, is there any other implementation or library to help me achieve something like this?
async function run(x,y,z) {
return new Promise(async(resolve,reject) => {
await doSomething(x)
await doSomething(y)
//cancel could be happen around here and stop the last "doSomething"
await doSomething(z)
})
}
setTimeout(() => {
run.cancel()
},500) //cancel function after 500ms
To just stop the advancement from one function call to the next, you can do something like this:
function run(x, y, z) {
let stop = false;
async function run_internal() {
await doSomething(x)
if (stop) throw new Error("cancelled");
await doSomething(y)
if (stop) throw new Error("cancelled");
await doSomething(z)
}
return {
cancel: () => {
stop = true;
},
promise: run_internal();
};
}
const retVal = run(a, b, c);
retVal.promise.then(result => {
console.log(result);
}).catch(err => {
console.log(err);
})
setTimeout(() => {
retVal.cancel()
}, 500); //cancel function after 500ms
Javascript does not have a generic way to "abort" any further execution of a function. You can set a flag via an external function and then check that flag in various points of your function and adjust what you execute based on that flag.
Keep in mind that (except when using workerThreads or webWorkers), Javascript runs your code in a single thread so when it's running, it's running and none of your other code is running. Only when it returns control back to the event loop (either by returning or by hitting an await) does any of your other code get a chance to run and do anything. So, "when it's actually running", your other code won't be running. When it's sitting at an await, your other code can run and can set a flag that can be checked later (as my example above shows).
fetch() in a browser has some experimental support for the AbortController interface. But, please understand that once the request has been sent, it's been sent and the server will receive it. You likely won't be aborting anything the server is doing. If the response still hasn't come back yet or is in the process of coming back, your abort may be able to interrupt that. Since you can't really know what is getting aborted, I figure it's better to just put a check in your own code so that you won't process the response or advance to further processing based on a flag you set.
You could wrap this flag checking into an AbortController interface if you want, but it doesn't change the fundamental problem in any way - it just affects the API you expose for calling an abort.
The way to actually use cancellation is through the AbortController which is available in the browser and on Node 15+
Node reference: https://nodejs.org/api/globals.html#class-abortcontroller
MDN reference: https://developer.mozilla.org/en-US/docs/Web/API/AbortController
Some APIs are currently using out of the box the abort signal like fetch in the browser or setTimeout timers API in Node (https://nodejs.org/api/timers.html#timerspromisessettimeoutdelay-value-options).
For custom functions/APIs you need to implement it by yourself but it's highly encouraged to follow the Abort signal methodology so you can chain both custom and oob functions and make use of a single signal that does not need translation

Potential race conditions when Promise used in subscriptions in Javascript / TypeScript

I recently dived into subscriptions of Subject/BehaviorSubject/ etc and I am looking for the goto approach when used in combinations with Promises.
Given is the example code below:
firebase.user.subscribe((user: any | null) => {
fs.readFile('path/to/file')
.then((buf: Buffer) => {
this.modifySomeData = buf;
});
});
I subscribe to a Subject that triggers whenever the user logs in or out of their service. Whenever this happens, I read a file from disk. This readFile event could potentially take longer than the next "login/logout" event. Of course, I am in JS and in an asynchronous environment. This means, my user code is not multithreaded, but still, the 2nd user event and 2nd readFile could theoretically be faster than the first readFile.
First user event fired
First readFile is executed
Second user event is fired
Second readFile is executed
Second readFile is resolved <---
First readFile is resolved <---
The order is mixed up. The silliest approach I could think of is to create a uuid before reading the file and check inside the promise if this is still the same. If it's not I discard the data.
Is there a better solution?
If i have a process where older requests can be discarded i often keep a variable in scope to track the latest request and compare, similar to your UUID idea:
let lastRead: Promise<Buffer> | null = null;
firebase.user.subscribe((user: any | null) => {
const read = lastRead = fs.readFile('path/to/file');
read.then((buf: Buffer) => {
if (read != lastRead)
return;
this.modifySomeData = buf;
});
});
In this specific case, readFile also supports an abort signal. So you might also be able to abort the last request instead; you will still need to track it though.
The first approach is to see if your event generation logic could handle waiting for event handling. For example, you can use a promise to wait for the event OR generate another event, say doneReadFile and only then send the next event. Usually, this is not the case for a generic (distributed) environment.
If event generation does not care about how long it took to handle events, you can still use the above approach but check for the intermediate event doneReadFile in the next event handler (login/logout). This can be achieved by implementing some kind of polling or busy-wait/sleep

What does it mean that all event handlers are fired synchronously?

I am confused about some terms. I am trying to find out how the event system of Node.js actually works, and in a lot of places I read that the event handlers are totally synchronous.
For me that seemed really strange, because one of the advantages of using an event-driven approach would be that the main thread would not be blocked by events. So I tried to come up with my own example, and it seems like that what did happen was what I actually expected:
const fs = require('fs')
const util = require('util')
const readFile = util.promisify(fs.readFile)
const events = require('events')
const emitter = new events.EventEmitter()
emitter.on('fire', () => {
readFile('bigFile.txt')
.then(() => console.log('Done reading bigFile.txt'))
.catch(error => console.log(error))
console.log('Sync thing in handler')
})
emitter.on('fire', () => {
console.log('Second handler')
})
console.log('First outside')
emitter.emit('fire')
console.log('Last outside')
Note that bigFile.txt is an actually large text file, processing it takes a few hundred milliseconds on my machine.
Here I first log out 'First outside' synchronously. Then I raise the event which starts the event handling process. The event handler does seem to be asynchronous, because even though we first log out the synchronous 'Sync thing in handler' text, we start using the thread pool in the background to return back with the result of reading the file later. After running the first handler, the second handler runs printing out its message, and finally we print out the last sync message, 'Last outside'.
So I started with trying to prove what some people say, which is that event handlers are by nature synchronous, and then I found them to be asynchronous. My best guess is that either people saying that the event system is synchronous mean something else, or that I have some conceptual misunderstanding. Please help me understand this issue!
The EventEmitter class is synchronous in regard to the emit function: event handlers are called synchronously from within the .emit() call, as you've demonstrated with the fire event you fired yourself.
In general, events that come from the operating system (file and network operations, timers etc) through node's event loop are fired asynchronously. You're not firing them yourself, some native API does fire them. When you listen to these events, you can be sure that they will occur not before the next tick.
The event handler does seem to be asynchronous, because even though we first log out the synchronous 'Sync thing in handler' text, we start using the thread pool in the background to return back with the result of reading the file later
Yes, you are calling the asynchronous function readFile (that will notify you later), but that doesn't make your event listener function or the .emit('fire') call asynchronous. Even "asynchronous functions" that start a background process will immediately (synchronously) return something - often nothing (undefined) or a promise.

Synchronization in Javascript & Socket.io

I have a Node.js server running the following code, which uses socket.io library :
var Common = require('./common');
var _ = require('lodash');
var store = (function() {
var userMapByUserId = {};
var tokenSet = {};
var generateToken = function(userId) {
if (_.has(userMapByUserId, userId)) {
var token = '';
do {
token = Common.randomGenerator();
} while (_.has(tokenSet, token));
if (userMapByUserId[userId].tokens === undefined) {
userMapByUserId[userId].tokens = {};
}
userMapByUserId[userId].tokens[token] = true;
}
};
var deleteToken = function(userId, tokenValue) {
if (_.has(userMapByUserId, userId)) {
if (userMapByUserId[userId].tokens !== undefined) {
userMapByUserId[userId].tokens = _.omit(userMapByUserId[userId].tokens, tokenValue);
}
if (_.has(tokenSet, tokenValue)) {
tokenSet = _.omit(tokenSet, tokenValue);
}
}
};
return {
generateToken: generateToken,
deleteToken: deleteToken
};
}());
module.exports = function(socket, io) {
socket.on('generateToken', function(ownerUser) {
store.generateToken(ownerUser.userId);
io.sockets.emit('userList', {
userMapByUserId: store.getUserList()
});
});
socket.on('deleteToken', function(token) {
store.deleteToken(token.userId, token.tokenValue);
io.sockets.emit('userList', {
userMapByUserId: store.getUserList()
});
});
};
So, basically we can have multiple clients sending requests to this server to add/remove tokens. Do I need to worry about synchronization and race conditions ?
I don't see any concurrency issues in the code you've shown.
The Javascript in node.js is single threaded and works via an event queue. One thread of execution runs until it completes and then the JS engine fetches the next event waiting to run and runs that thread of execution until it completes. As such, there is no pre-emptive concurrency in Javascript like you might have to worry about in a language or situation using threads.
There are still some places in node.js where you can get opportunities for concurrency issues. This can happen if you are using asynchronous operations in your code (like async IO to a socket or disk). In that case, your thread of execution runs to completion, but your async operation is still running and has not finished yet and has not called its callback yet. At this point, some other event can get processed and other code can run. So, if your async callback refers to state that could get changed by other event handlers, then you could have concurrency issues.
In the two socket event handlers you disclosed in your question, I don't see any asynchronous callbacks within the code running in the event handlers. If that's the case, then there are no opportunities for concurrency there. Even if there were concurrency opportunities, but the separate pieces of code weren't using/modifying the same underlying data, then you would still be OK.
To give you an example of something that can cause a concurrency issue, I had a node.js app that was running on a Raspberry Pi. Every 10 seconds it would read a couple of digital temperature sensors and would record the data in an in-memory data structure. Every few hours, it would write the data out to disk. The process of writing the data out to disk was all done with a serios of async disk IO operations. Each async disk IO operation has a callback and the code continued only within that callback (technically I was using promises to manage the callbacks, but its the same concept). This create a concurrency issue because if the timer that records new temperatures fires while I'm in the middle of one of these async disk operations, then the data structure could get changed in the middle of writing it to disk which could cause me problems.
It wasn't that the async operation would get interrupted in the middle of it's disk write (node.js is not pre-emptive in that way), but because the whole operation of writing to disk consisted of many separate async file writes, other events could get processed between the separate async file writes. If those events could muss with your data, then it could create a problem.
My solution was to set a flag when I started writing the data to disk (then clear the flag when done) and if I wanted to add something to the data structure while that flag was set, then the new data would go into a queue and would be processed later so the core data structure was not changed while I was writing it to disk. I was able to implement this entirely in a few methods that were used for modifying the data so that the calling code didn't even need to know anything different was going on.
Though this answer was written for Ajax in a browser, the event queue concept is the same in node.js so you may find it helpful. Note that node.js is called "event IO for V8 Javascript" for a reason (because it runs via an event queue).

How does a callback get executed when assigned after consuming method got called

Question:
I am reading a book about the Html5 data storage IndexedDb. The IndexedDb API uses the following code to open a database connection:
var request = indexedDB.open('some name');
request.onsuccess = function(event) {
obj.id = event.target.result
}
request.onerror = function(event) {...}
The request variable is assigned a callback which is called when the request was executed successfully. But the request gets never called after those lines. So my question is:
How does the onsuccess callback get executed? It can't be in in the indexedDB.open method because there the onsuccess callback wasn't assigned yet?
What am I missing?
Edit:
After the comment from James I found the missing link to my question:
Say it with me now: async programming does not necessarily mean multi-threaded.
Javascript is a single-threaded runtime - you simply aren't able to create new threads in JS because the language/runtime doesn't support it.
source: How does Asynchronous programming work in a single threaded programming model?
Solution:
As James points out in his answer below, async functions like indexedDB.open() are pushed into a special queue known as the “event loop”:
The event loop is a special queue for callback functions. When an async code is executed, a callback is then pushed into
the queue. The Javascript engine will only execute the event loop if the code after the async function has finished executing.
source: http://www.hiddenwebgenius.com/blog/guides/understanding-javascripts-asynchronous-code/
If indexedDB.open executes the request asynchronously then the request won't be completed until after your code relinquishes control, so request.onsuccess will have been assigned by then.
Javascript in the browser is single threaded so there is no chance of the request completing before onsuccess / onerror is assigned

Categories

Resources