Develop Reference

How do I queue incoming websocket events in javascript for slow execution?

I have an open Websocket connection and it's handing out events. All good, but once a new event arrives, I need to do a whole lot of things and sometimes events arrive so quickly one after the other that there is no time to get the stuff done properly. I need some sort of queue inside this function that tells the events to take it easy and only keep going at most one per second, and otherwise wait in some sort of queue until the second elapses to go ahead and continue.
edit: No external libraries allowed, unfortunately.
ws = new WebSocket(`wss://hallo.com/ws/`);
ws.onmessage = readMessage;
async function readMessage(event) {
print(event)
//do important things
//but not too frequently!
}
How do I do that?
I found this but it goes over my simple head:
"You can have a queue-like promise that keeps on accumulating promises to make sure they run sequentially:
let cur = Promise.resolve();
function enqueue(f) {
cur = cur.then(f); }
function someAsyncWork() {
return new Promise(resolve => {
setTimeout(() => {
resolve('async work done');
}, 5);
}); } async function msg() {
const msg = await someAsyncWork();
console.log(msg); }
const main = async() => {
web3.eth.subscribe('pendingTransactions').on("data", function(tx) {
enqueue(async function() {
console.log('1st print: ',tx);
await msg();
console.log('2nd print: ',tx);
});
}) }
main();
"

I'd honestly use something like lodash's throttle to do this. The following snippet should solve your problem.
ws = new WebSocket(`wss://hallo.com/ws/`);
ws.onmessage = _.throttle(readMessage, 1000);
async function readMessage(event) {
print(event)
//do important things
//but not too frequently!
}

For achieving queuing, you can make use of "settimeout" in simple/core javascript.
Whenever you receive a message from websocket, put the message processing function in a settimeout, this will ensure that the message is processed not immediately as its received, but with a delay, hence in a way you can achieve queuing.
The problem with this is that it does not guarantee that the processing of messages is sequential as they are received if that is needed.
By default settimeout in javascript does give the guarantee of when the function inside will be triggered after the time given is elapsed.
Also it may not reduce the load on your message processor service for a high volume situation and since individual messages are queued two/more functions can become ready to be processed from setimeout within some time frame.
An ideal way to do so would be to create a queue. On a high level code flow this can be achieved as follows
var queue = [];
function getFromQueue() {
return queue.shift();
}
function insertQueue(msg) { //called whenever a new message arrives
queue.push(msg);
console.log("Queue state", queue);
}
// can be used if one does not want to wait for previous message processing to finish
// (function executorService(){
// setTimeout(async () => {
// const data = getFromQueue();
// await processData(data);
// executorService();
// }, 1000)
// })()
(function executorService(){
return new Promise((res, rej) => {
setTimeout(async () => {
const data = getFromQueue();
console.log("Started processing", data)
const resp = await processData(data); //waiting for async processing of message to finish
res(resp);
}, 2000)
}).then((data) =>{
console.log("Successfully processed event", data)
}).catch((err) => {
console.log(err)
}).finally(() => {
executorService();
})
})()
// to simulate async processing of messages
function processData(data){
return new Promise((res, rej) => {
setTimeout(async () => {
console.log("Finished processing", data)
res(data);
}, 4000)
})
}
// to simulate message received by web socket
var i = 0;
var insertRand = setInterval(function(){
insertQueue(i); // this must be called on when web socket message received
i+=1;
}, 1000)

Websocket waiting for server response with a queue

I'm using websocket to send and receive data (up to 30 small messages per seconds). I want the client to send a websocket payload and wait for a specific message from the server.
Flow:
The client send a request
It also store the requestId (163) in waitingResponse object as a new object with sent timestamp
waitingResponse = {
163: { sent: 1583253453549 }
}
When the server response, another function validate the payload and then append the result to that request object
waitingResponse = {
163: { sent: 1583253453549, action: "none" }
}
The client checks every x ms that object for the action key
I have a function sendPayload that sends the payload and then await for a value from awaitResponse (the function below). Right now this function doesn't work. I tried making 2 separate function, one that would be the setTimeout timer, the other was the promise. I also tried having both in the same function and decide if it was the loop or the promise with the original argument you can see below. Now I'm thinking that function should always return a promise even in the loop but I cannot seem to make that work with a timer and I'm afraid of the cost of multiple promise in one another. Let's say I check for a response every 5 ms and the timeout it 2000ms. That is a lot of promises.
public async sendPayload(details) {
console.log("sendPlayload", details);
this.waitingResponse[details.requestId] = { sent: +new Date() };
if (this.socket.readyState === WebSocket.OPEN) {
this.socket.send(JSON.stringify(details));
}
const bindAwaitResponse = this.awaitResponse.bind(this);
return new Promise(async function (resolve, reject) {
const result = await bindAwaitResponse(details.requestId, true);
console.log("RES", result);
console.info("Time took", (+new Date() - result.sent) / 1000);
resolve(result);
});
}
public async awaitResponse(requestId, original) {
// console.log(requestId, "awaitResponse")
return new Promise((resolve, reject) => {
// Is it a valid queued request
if (this.waitingResponse[requestId]) {
// Do we have an answer?
if (this.waitingResponse[requestId].hasOwnProperty("action")) {
console.log(requestId, "Got a response");
const tmp = this.waitingResponse[requestId];
delete this.waitingResponse[requestId]; // Cleanup
resolve(tmp);
} else {
// No answer yet from remote server
// console.log("no answer: ", JSON.stringify(this.waitingResponse));
// Check if request took too long
if (+new Date() - this.waitingResponse[requestId].sent > 5000) { // TODO: Option for time out
console.warn(requestId, "Request timed out");
// Timed out, result took too long
// TODO: Option, default action when timed out
delete this.waitingResponse[requestId]; // Cleanup
resolve({
action: "to" // For now, just sent a timeout action, maybe the default action should be outside of the Network class?
})
} else {
// console.log(requestId, "Still waiting for results");
console.log(JSON.stringify(this.waitingResponse));
// Still waiting, after x ms, recall function
return setTimeout(async () => { resolve(await this.awaitResponse(requestId, false)); }, 200);
}
}
}
});
}
private async processMessage(msg) {
console.log("WS received Message", JSON.stringify(msg.data));
console.log("Current: ", JSON.stringify(this.waitingResponse));
let data = JSON.parse(msg.data);
// console.log("Received: ", data);
if (data.hasOwnProperty("requestId") && this.waitingResponse[data.requestId]) {
// console.log("processMessage ID found");
this.waitingResponse[data.requestId] = { ...data, ...this.waitingResponse[data.requestId] };
}
}
Note: I put the websocket tag below because I looked hard for that. Maybe I came across the solution without even realising it, but if you have better tags for this question to be found easier, please edit them :)

Yeah, you're mixing a lot of callback style functions with intermediate promises and async/await. Don't do polling when waiting for a response! Instead, when writing a queue, put the resolve function itself in the queue so that you can directly fulfill/reject the respective promise from the response handler.
In your case:
public async sendPayload(details) {
const request = this.waitingResponse[details.requestId] = { sent: +new Date() };
try {
if (this.socket.readyState === WebSocket.OPEN) {
this.socket.send(JSON.stringify(details));
}
const result = await new Promise(function(resolve) {
request.resolve = resolve;
setTimeout(() => {
reject(new Error('Timeout')); // or resolve({action: "to"}), or whatever
}, 5000);
});
console.info("Time took", (+new Date() - request.sent) / 1000);
return result; // or {...request, ...result} if you care
} finally {
delete this.waitingResponse[details.requestId];
}
}
private async processMessage(msg) {
let data = JSON.parse(msg.data);
if (data.hasOwnProperty("requestId") {
const request = this.waitingResponse[data.requestId]
if (request)
request.resolve(data)
else
console.warn("Got data but found no associated request, already timed out?", data)
} else {
console.warn("Got data without request id", data);
}
}
You might even do away with the request object altogether and only store the resolve function itself, if the processMessage function does not need any details about the request.

Node, wait and retry api calls that fail

So I fetch an array of urls from api with a rate limit, currently I handle this by adding a timeout to each call like this:
const calls = urls.map((url, i) =>
new Promise(resolve => setTimeout(resolve, 250 * i))
.then(() => fetch(url)
)
);
const data = await Promise.all(calls);
forcing a 250ms wait between each call. This ensures that the rate limit is never exceeded.
The thing is, this isn't really necessary. I've tried with 0ms wait time, and most of the cases I have to repeatedly reload the page four or five times before the api starts to return:
{ error: { status: 429, message: 'API rate limit exceeded' } }
and most of the times you only have to wait a second or so before you can safely reload the page and get all data.
A more reasonable approach would be to collect the calls that return 429 (if they do), wait for a set amount of time and then retry them (and perhaps redo this a set amount of times).
Problem, I'm a bit stumped as to how one would go about achieving this?
EDIT:
Just got home and will look through the answers but there seem to have been an assumption made which I don't believe is necessary: The calls does not have to be sequential, they can be fired (and returned) in any order.

The term for what you want is exponential backoff. You can modify your code so that it continues trying on a certain failure condition:
const max_wait = 2000;
async function wait(ms) {
return new Promise(resolve => {
setTimeout(resolve, ms);
});
}
const calls = urls.map(async (url) => {
let retry = 0, result;
do {
if (retry !== 0) { await wait(Math.pow(2, retry); }
result = await fetch(url);
retry++;
} while(result.status !== 429 || (Math.pow(2, retry) > max_wait))
return result;
}
Or you can try using a library to handle the backoff for you like https://github.com/MathieuTurcotte/node-backoff

If I understand the question right, your trying to:
a) Execute fetch() calls sequentially (with a possibly optional delay)
b) Retry failed requests with a backoff delay
As you likely found out, .map() does not really help with a) as it does not wait for any async stuff when iterating (which is why you create a greater and greater timeout with i*250).
I personally find it the easiest to keep things sequential by using a for of loop instead, as this will work nicely with async/await:
const fetchQueue = async (urls, delay = 0, retries = 0, maxRetries = 3) => {
const wait = (timeout = 0) => {
if (timeout) { console.log(`Waiting for ${timeout}`); }
return new Promise(resolve => {
setTimeout(resolve, timeout);
});
};
for (url of urls) {
try {
await wait(retries ? retries * Math.max(delay, 1000) : delay);
let response = await fetch(url);
let data = await (
response.headers.get('content-type').includes('json')
? response.json()
: response.text()
);
response = {
headers: [...response.headers].reduce((acc, header) => {
return {...acc, [header[0]]: header[1]};
}, {}),
status: response.status,
data: data,
};
// in reality, only do that for errors
// that make sense to retry
if ([404, 429].includes(response.status)) {
throw new Error(`Status Code ${response.status}`);
}
console.log(response.data);
} catch(err) {
console.log('Error:', err.message);
if (retries < maxRetries) {
console.log(`Retry #${retries+1} ${url}`);
await fetchQueue([url], delay, retries+1, maxRetries);
} else {
console.log(`Max retries reached for ${url}`);
}
}
}
};
// populate some real URLs urls to fetch
// index 0 will generate an inexistent URL to test error behaviour
const urls = new Array(101).fill(null).map((x, i) => `https://jsonplaceholder.typicode.com/todos/${i}`);
// fetch urls one after another (sequentially)
// and delay each request by 250ms
fetchQueue(urls, 250);
If a request fails (e.g. you get one of the errors specified in the array with error status codes), the above function will retry them a maximum of 3 times (by default) with a backoff delay that increases by a second on each retry.
As you wrote, the delay between requests is probably not necessary, so you could just remove the 250 in the function call. Because each request is executed one after the other, you're less likely to run into rate limit issues but if you do, it's very easy to add some custom delay.

Here is an example that allows to handle an array of promises sequencially, by setting a delay expressed in milliseconds and accepting a third callback determining whether the request should be retried.
In the below code, some sample requests are mocked to:
Test a successful response.
Test an error response. If the error response contains an error code and the error code is 403, true is returned and the call is retried in the next run (delayed by x milliseconds).
Test an error response without an error code.
There is a global counter below that give up the promise after N tries (in the below example 5), all of that is handled in this code:
const result = await resolveSequencially(promiseTests, 250, (err) => {
return ++errorCount, !!(err && err.error && err.error.status === 403 && errorCount <= 5);
});
Where the error count is first increased and it returns true if the error is defined, has an error property and its status is 403.
Of course, the example is just to test things out, but I think you're looking for something allowing you to have a cleverer control over the promise loop cycle, hence here is a solution doing just that.
I will add some comments below, you can run the test below to check what happens directly in the console.
// Nothing that relevant, this one is just for testing purposes!
let errorCount = 0;
// Declare the function.
const resolveSequencially = (promises, delay, onFailed, onFinished) => {
// store the results.
const results = [];
// Define a self invoking recursiveHandle function.
(recursiveHandle = (current, max) => { // current is the index of the currently looped promise, max is the maximum needed.
console.log('recursiveHandle invoked, current is, ', current ,'max is', max);
if (current === max) onFinished(results); // <-- if all the promises have been looped, resolve.
else {
// Define a method to handle the promise.
let handlePromise = () => {
console.log('about to handle promise');
const p = promises[current];
p.then((success) => {
console.log('success invoked!');
results.push(success);
// if it's successfull, push the result and invoke the next element.
recursiveHandle(current + 1, max);
}).catch((err) => {
console.log('An error was catched. Invoking callback to check whether I should retry! Error was: ', err);
// otherwise, invoke the onFailed callback.
const retry = onFailed(err);
// if retry is true, invoke again the recursive function with the same indexes.
console.log('retry is', retry);
if (retry) recursiveHandle(current, max);
else recursiveHandle(current + 1, max); // <-- otherwise, procede regularly.
});
};
if (current !== 0) setTimeout(() => { handlePromise() }, delay); // <-- if it's not the first element, invoke the promise after the desired delay.
else handlePromise(); // otherwise, invoke immediately.
}
})(0, promises.length); // Invoke the IIFE with a initial index 0, and a maximum index which is the length of the promise array.
}
const promiseTests = [
Promise.resolve(true),
Promise.reject({
error: {
status: 403
}
}),
Promise.resolve(true),
Promise.reject(null)
];
const test = () => {
console.log('about to invoke resolveSequencially');
resolveSequencially(promiseTests, 250, (err) => {
return ++errorCount, !!(err && err.error && err.error.status === 403 && errorCount <= 5);
}, (done) => {
console.log('finished! results are:', done);
});
};
test();

Promises inside of setInterval

I've got this routine function that runs every 60000ms on a setInterval function. Inside of this routine function, I loop over all the usernames on a parsed JSON(db) and check whether or not they're available via a promise (checkUsername), which is a network request.
However, It's pretty clear to me that this is a terrible approach because the promises may take more than 60 seconds to complete, and I've been getting ETIMEDOUT errors all over the place. But I just don't understand promises and asynchrony enough to think of a solution.
What would be a better approach to this?
Would async/await fit here?
function routine() {
db.users.forEach(userObj => {
userObj.username.forEach(username => {
checkUsername(username).then(hasUser => {
if(!hasUser) {
bot.sendMessage(userObj.chatId, `‼️ Username #${username} is now AVAILABLE ‼️`);
removeName(username, userObj.chatId);
}
}).catch(err => {
console.log(err);
})
})
});
}
setInterval(routine, 120000);

var interval = 60000;
function routine() {
return Promise.all(db.users.map(userObj => {
return Promise.all(userObj.username.map(username => {
return checkUsername(username).then(hasUser => {
if(!hasUser){
return removeName(username, userObj.chatId).then(function(){
return bot.sendMessage(userObj.chatId, `‼️ Username #${username} is now AVAILABLE ‼️`)
})
}
}))
})).then(function(){
setTimeout(routine, interval);
}, function(error){
console.error(error);
setTimeout(routine, interval);
})
}
routine();
This will run every 60 seconds plus the time it takes to resolve all the requests. If it fails, it will run again in 60 seconds. You could be running into timeout issues if you have too many users. Also, if "removeName" fails, "bot.sendMessage" won't run.
You can end promises many different ways. It depends on what you need to do with the promise.
.then(function(){
//do stuff
setTimeout(routine, interval);
}, function(error){
console.error(error);
setTimeout(routine, interval);
})
or
.then(function(){
//do stuff
setTimeout(routine, interval);
}).catch(function(error){
console.error(error);
setTimeout(routine, interval);
})
or
.catch(function(error){
console.error(error);
}).finally(function(){
setTimeout(routine, interval);
})
or
.finally(function(){
setTimeout(routine, interval);
})

I made a code snippet that you can run that makes use of Promise.all as well as async/await ES7 to make your code a bit easier to handle and understand. I'm also hitting a real API endpoint I found online just for the sake of a complete example.
I've also added a way to stop the timeout for good if you ever want that option.
// How often the timeout will run.
// Since the timeout is dependent on when all the requests finish, the timeout will run this many ms after all the requests finish.
var interval = 5000;
// This is what your db seems to resemble.
var db = {
users: [{
username: ['1']
},
{
username: ['2']
},
{
username: ['3']
},
{
username: ['4']
},
]
};
// This will hold the timeout function so you can cancel it at a later time.
var timeoutFn;
// Returns returns a single Promise that resolves when all of the promises it contains have resolved/rejected. It rejects with the first promise that rejects.
function routine() {
console.log("-- Running routine function --");
// Return an array of promises. Please see my comments at the bottom of this whole answer which questions db architecture with username being an array.
// I'm also using map instead of forEach because map automatically returns and array.
let promiseArray = db.users.map(userObj => {
return Promise.all(userObj.username.map(username => {
// This processUsername() function should do all the work related to the username. It helps to keep the routine function as clean as possible since there's already a lot happening in here.
return processUsername(username);
}));
});
// Returns an array of array of promises. This means that every single promise within each array (see above) has to resolve before the `then` runs. If any reject, `catch` will run instead.
return Promise.all(promiseArray).then(() => {
runRoutineAgain();
}).catch((err) => {
console.log('err:', err)
});
}
// This will create a timeout and run routine after interval.
function runRoutineAgain() {
timeoutFn = setTimeout(routine, interval);
}
// This async function returns a promise
async function processUsername(username) {
// Make API call to get data
console.log('Processing username for', username);
// I'm using this free API endpoint online just for the sake of making a complete example. Obviously your API will be returning very different data.
return await fetch(`https://jsonplaceholder.typicode.com/todos/${username}`)
.then(response => response.json())
.then((json) => {
console.log(json);
// This is where you can do your processing.
// if(!hasUser) {
// bot.sendMessage(userObj.chatId, `‼️ Username #${username} is now AVAILABLE ‼️`);
// removeName(username, userObj.chatId);
// }
});
}
function stopTimeout() {
clearTimeout(timeoutFn);
}
routine();
Basically, I'm using Promise.all to capture and wait on the result of individual promises which is very useful since you have many users you need to get data for.
Feel free to open up Web Console to better see the output data.
I'm also using async/await ES7 syntax just to demonstrate other (easier, some might say) ways to write Promises. I understand Promises can be daunting so here are some links that really hit the nail on the head when trying to learn them.
https://javascript.info/promise-basics - This covers Promises
https://javascript.info/async-await - This covers async/await
Also is there a reason why you are looping over each username for a user in your original code?
db.users.forEach(userObj => {
userObj.username.forEach(username => {…
If there’s only 1 username for a userObj, that second loop adds unnecessary complexity. But if your db has multiple usernames for a single userObj, then that’s completely fine!

One way is to clear the interval as soon as you start handling the interval-function. Then when your interval-function is done, you can start the interval again. This way your interval is not ticking while your interval-function is being executed. Take a look at below code.
let interval = setInterval(routine, 1000)
function routine() {
let sequence = Promise.resolve()
sequence
.then(function () {
clearInterval(interval)
return Promise.resolve()
})
// ... chain rest of your promise here.
.then(function (value) {
return new Promise(function (resolve) {
setTimeout(function () {
console.log('I take long time.');
resolve()
}, 5000)
})
})
.then(function () { // in last promise, start the timer again.
interval = setInterval(routine, 1000)
})
}
Here you interval-function is supposed to run every 1s, but the promise itself takes 5s. So the first thing the method does is stop the timer, then do its job, and finally re-enable the timer.
With timeout,
let timeout = setTimeout(routine, 1000)
function routine() {
let sequence = Promise.resolve()
sequence
.then(function () {
return Promise.resolve()
})
// ... chain rest of your promise here.
.then(function (value) {
return new Promise(function (resolve) {
setTimeout(function () {
console.log('I take long time.');
resolve()
}, 5000)
})
})
.then(function () { // in last promise, start the timer again.
timeout = setTimeout(routine, 1000)
})
}

how to allow delay in looped http request but also use Promise.all to detect when requests are done

Hi I am trying to allow 50ms before next http request is made to the server but also would like to implement some function when all the request have been made.
I made an example to illustrate what I would like to do.
So below is my example express api
app.get('/morans/test', (req, res) => {
console.log(`start test request`);
let targetUrl = `http://-domain-/solr/genea_expression/smplGeoclust?q=text:"GO:0003674"&stats.facet=geohash_3&rows=10320`;
let i = 0;
let promises = [];
console.log(`making request ${i}`);
for (let i = 0; i < 200; i++) {
setTimeout(
() => {
let testReq = new Promise((resolve, reject) => {
http.get(targetUrl, (result) => {
result.on('data', (chunk) => {
console.log(`working ${i}`);
}).on('end', () => {
console.log(`ending ${i}`);
}).on('error', (err) => {
console.log(`this is error message ${err}`);
})
})
});
promises.push(testReq);
}, 50 * i
)
}
Promise.all(promises).then(() => {
console.log(`done`);
});
});
The delay works fine; however, Promise.all(promises)... is getting a premature trigger (console.log('done') triggers as soon as first request is made). It works fine when there is no timeout so I am thinking that the setTimeout is not allowing enough time for promise to be pushed to promises array. How can I write a code so that http request is made in every 50ms but also know when all the requests are successfully returned?

You are calling Promise.all at a time your promises array is empty, since none of the time outs has expired yet.
You could move that Promise.all call inside your timeout callback, right after the push, and check the length of that array:
for (let i = 0; i < 200; i++) {
setTimeout(
() => {
// ...etc ... etc
promises.push(testReq);
if (promises.length == 200) { // <---- add this block
Promise.all(promises).then(() => {
console.log(`done`);
});
}
}, 50 * i
)
}
Also make sure you call resolve() within your new Promise callback, otherwise your promises will stay pending:
.on('end', () => {
console.log(`ending ${i}`);
resolve();
})
See the documentation on http.get to see how to actually get the response data... It would make sense to call resolve with that data as argument.