I am trying to let a Web-Worker manage its state, meanwhile serving multiple async requests.
worker.ts file
let a =0; //this is my worker's state
let worker=self as unknown as Worker;
worker.onmessage =(e)=>{
console.log("Rec msg", e.data);
if(e.data === "+1"){
setTimeout(()=>{
a=a+1;
worker.postMessage(a);
},3000);
}else if(e.data=== "+2"){
setTimeout(()=>{
a=a+2;
worker.postMessage(a);
},1000)
}
}
And this is my main file: main.ts
let w =new Worker("./worker.ts", {type: "module"})
let wf =async (op: string)=>{
w.postMessage(op);
return new Promise<any>((res,rej)=>{
w.onmessage=res;
});
}
(async()=>{
let f1 = await wf("+1");
console.log("f1",f1.data);
})();
(async()=>{
let f2 = await wf("+2");
console.log("f2",f2.data);
})()
Only f2 is returned , and f1 is lost.
I have used timeouts to simulate say some async task done by worker themselves.
How do I receive both f1 and f2?
Your problem is that you are trying to take an event based API and use it as a Promise based one, but events may fire multiple times, while Promise should resolve only once.
The communication between the Worker and the main thread works by sending and receiving messages, but there is by default no one-to-one relation between these messages. Both ends of the communication (ports) will simply stack incoming messages, and handle them sequentially, when they'll get time.
In your code, the main thread's worker.onmessage handler of f1 has been overwritten by the second call f2 synchronously (one microtask later, but that's still synchronous for our matter).
You could attach your event using the addEventListener method, at least this way it wouldn't be overwritten. But even then, when the first message event will fire on worker, both handlers will think it's there own message that did arrive, while in fact it was the one of f2. so that's not what you need...
What you need is to set up a protocol of communication which would allow both ends to identify each task. You could for instance wrap all your tasks' data with an object containing a .UIID member, be sure both ends wraps their message this way, and then from main thread check that UUID to resolve the appropriate Promise.
But that can become a bit complicated to implement and to use.
My personal favorite way is to create a new MessageChannel per task. If you don't know this API, I invite you to read this answer of mine explaining the basics.
Since we are sure the only one message that will come through this MessageChannel is the response from the Worker to the one task we sent to it, we can await it just like a Promise.
All we have to do, is to make sure that in the Worker thread we respond through the transferred port instead of the global scope.
const url = getWorkerURL();
const worker = new Worker(url)
const workerFunc = (op) => {
// we create a new MessageChannel
const channel = new MessageChannel();
// we transfer one of its ports to the Worker thread
worker.postMessage(op, [channel.port1]);
return new Promise((res,rej) => {
// we listen for a message from the remaining port of our MessageChannel
channel.port2.onmessage = (evt) => res(evt.data);
});
}
(async () => {
const f1 = await workerFunc("+1");
console.log("f1", f1);
})();
(async () => {
const f2 = await workerFunc("+2");
console.log("f2", f2);
})()
// SO only
function getWorkerURL() {
const elem = document.querySelector( '[type="worker-script"]' );
const script = elem.textContent;
const blob = new Blob( [script], { type: "text/javascript" } );
return URL.createObjectURL( blob );
}
<script type="worker-script">
let a = 0;
const worker = self;
worker.onmessage = (evt) => {
const port = evt.ports[0]; // this is where we will respond
if (evt.data === "+1") {
setTimeout(() => {
a = a + 1;
// we respond through the 'port'
port.postMessage(a);
}, 3000);
}
else if (evt.data === "+2") {
setTimeout(() => {
a = a + 2;
// we respond through the 'port'
port.postMessage(a);
}, 1000)
}
};
</script>
Related
I have an open Websocket connection and it's handing out events. All good, but once a new event arrives, I need to do a whole lot of things and sometimes events arrive so quickly one after the other that there is no time to get the stuff done properly. I need some sort of queue inside this function that tells the events to take it easy and only keep going at most one per second, and otherwise wait in some sort of queue until the second elapses to go ahead and continue.
edit: No external libraries allowed, unfortunately.
ws = new WebSocket(`wss://hallo.com/ws/`);
ws.onmessage = readMessage;
async function readMessage(event) {
print(event)
//do important things
//but not too frequently!
}
How do I do that?
I found this but it goes over my simple head:
"You can have a queue-like promise that keeps on accumulating promises to make sure they run sequentially:
let cur = Promise.resolve();
function enqueue(f) {
cur = cur.then(f); }
function someAsyncWork() {
return new Promise(resolve => {
setTimeout(() => {
resolve('async work done');
}, 5);
}); } async function msg() {
const msg = await someAsyncWork();
console.log(msg); }
const main = async() => {
web3.eth.subscribe('pendingTransactions').on("data", function(tx) {
enqueue(async function() {
console.log('1st print: ',tx);
await msg();
console.log('2nd print: ',tx);
});
}) }
main();
"
I'd honestly use something like lodash's throttle to do this. The following snippet should solve your problem.
ws = new WebSocket(`wss://hallo.com/ws/`);
ws.onmessage = _.throttle(readMessage, 1000);
async function readMessage(event) {
print(event)
//do important things
//but not too frequently!
}
For achieving queuing, you can make use of "settimeout" in simple/core javascript.
Whenever you receive a message from websocket, put the message processing function in a settimeout, this will ensure that the message is processed not immediately as its received, but with a delay, hence in a way you can achieve queuing.
The problem with this is that it does not guarantee that the processing of messages is sequential as they are received if that is needed.
By default settimeout in javascript does give the guarantee of when the function inside will be triggered after the time given is elapsed.
Also it may not reduce the load on your message processor service for a high volume situation and since individual messages are queued two/more functions can become ready to be processed from setimeout within some time frame.
An ideal way to do so would be to create a queue. On a high level code flow this can be achieved as follows
var queue = [];
function getFromQueue() {
return queue.shift();
}
function insertQueue(msg) { //called whenever a new message arrives
queue.push(msg);
console.log("Queue state", queue);
}
// can be used if one does not want to wait for previous message processing to finish
// (function executorService(){
// setTimeout(async () => {
// const data = getFromQueue();
// await processData(data);
// executorService();
// }, 1000)
// })()
(function executorService(){
return new Promise((res, rej) => {
setTimeout(async () => {
const data = getFromQueue();
console.log("Started processing", data)
const resp = await processData(data); //waiting for async processing of message to finish
res(resp);
}, 2000)
}).then((data) =>{
console.log("Successfully processed event", data)
}).catch((err) => {
console.log(err)
}).finally(() => {
executorService();
})
})()
// to simulate async processing of messages
function processData(data){
return new Promise((res, rej) => {
setTimeout(async () => {
console.log("Finished processing", data)
res(data);
}, 4000)
})
}
// to simulate message received by web socket
var i = 0;
var insertRand = setInterval(function(){
insertQueue(i); // this must be called on when web socket message received
i+=1;
}, 1000)
I tried working with stockfish.js and I have this one problem. I also used #ninjapixel/chess for easier methods than the original.
(async function() {
console.log('test1`');
console.log(chess.ascii());
await stockfish.postMessage('go depth 15');
console.log(chess.ascii()); // this should change, but it doesn't
console.log('test2');
})();
Why does the async function with await keyword didn't work?
Why did the test2 was logged first before the test3 since I'm using an async function, it should work, but it didn't, why is that?
Here's how I handled onmessage:
stockfish.onmessage = (e) => {
let info = e.data ? e.data : e;
if (info.includes('bestmove') || info.includes('ponder')) {
const bestmove = info.split(/ +/)[1];
const ponder = info.split(/ +/)[3];
chess.move(bestmove, {
promotion: 'q',
sloppy: true
});
// test2 was logged first before this
console.log('test3'); // ascii worked in this part
stockfish.postMessage(`position fen ${chess.fen()}`); // update fen of stockfish
}
};
If there's no way to solve this then should I actually depend on the onmessage respond? Because what I wanna do is after I do postMessage I want to actually respond after the postMessage NOT directly inside the onmessage response.
Like something this:
let bestMove = null;
stockfish.onmessage = (e) => {
let info = e.data ? e.data : e;
if (info.includes('bestmove') || info.includes('ponder')) {
bestMove = info.split(/ +/)[1];
// set bestmove after calculating...
}
}
(async function() {
await stockfish.postMessage('go depth 15');
// do the move after the message
chess.move(bestMove);
await stockfish.postMessage(`position fen ${chess.fen()}`);
})();
The await keyword, didn't have any effect in your code because the function stockfish.postMessage doesn't return a Promise. postMessage emits an event that will be processed in the next tick. This is why console.log('test3') was logged after console.log('test2'). After emitting the event it will return synchronously, logging console.log('test2').
stockfish.onmessage will need to be the control centre of all events. It will need to understand the different messages, and do the appropiate actions. You dont need to put the implemention of the actions in the onmessage. You can create a function and called it from stockfish.onmessage... and then inside the action function you could call to stockfish.postMessage if you need to notify stockfis of a change... and so on.... and on...
I’ve nodes program which I need to run two function in the beginning of the program
And later on access the function results, currently with await each function at a time this works,
However in order to save a time and not waiting to GetService and GetProcess as I need the data later on in the project
It takes about 4 seconds to get this data and I want to run it on the background as I don’t need the results immediately,
How I can do it in node js, If I run promise.all It would wait until the getService and getProcess and then go to rest of the program.
an example
function main() {
//I want to run this both function in background to save time
let service = await GetServices();
this.process = await GetProcess();
…..//Here additional code is running
//let say that after 30 second this code is called
Let users = GetUser(service);
Let users = GetAdress(this.process);
}
im actually running yeoman generator
https://yeoman.io/authoring/
https://yeoman.io/authoring/user-interactions.html
export default class myGenerator extends Generator {
//here I want run those function in background to save time as the prompt to the user takes some time (lets say user have many questions...)
async initializing() {
let service = await GetServices();
this.process = await GetProcess();
}
async prompting() {
const answers = await this.prompt([
{
type: "input",
name: "name",
message: "Your project name",
default: this.appname // Default to current folder name
},
{
type: "confirm",
name: "list",
choises: this.process //here I need to data from the function running in background
}
]);
}
Let's assume that getServices() may take 3 seconds and getProcess() may take 4 seconds, so if you run these both functions at the same time you will be returned in total 4 seconds with the return values from both promises.
You can execute the code while this process is running in the background there will be a callback when the promises resolved, your late functions will be called at this stage.
Check the below simple example;
let service;
let process;
function main() {
// Both functions will execute in background
Promise.all([getServices(), getProcess()]).then((val) => {
service = val[0];
process = val[1];
console.log(service, process);
// Aafter completed this code will be called
// let users = GetUser(service);
// let users = GetAdress(process);
console.log('I am called after all promises completed.')
});
// Current example.
// let service = await GetServices();
// this.process = await GetProcess();
/* Code blocks.. */
console.log('Code will execute without delay...')
}
function getServices() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve("service is returned")
}, 3000);
});
}
function getProcess() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve("process is returned")
}, 4000);
});
}
main();
You can start the asynchronous operation but not await it yet:
function suppressUnhandledRejections(p) {
p.catch(() => {});
return p;
}
async function main() {
// We have to suppress unhandled rejections on these promises. If they become
// rejected before we await them later, we'd get a warning otherwise.
const servicePromise = suppressUnhandledRejections(GetServices());
this.processPromise = suppressUnhandledRejections(GetProcess());
// Do other stuff
const service = await servicePromise;
const process = await this.processPromise;
}
Also consider using Promise.all() which returns a promise for the completion of all promises passed to it.
async function main() {
const [ services, process, somethingElse ] = await Promise.all([
GetServices(),
GetProcess(),
SomeOtherAsyncOperation(),
]);
// Use the results.
}
To do what who you need, you have to understand the event loop.
Nodejs is designed to work in a single thread unlike languages like go, however nodejs handle proccess on different threads. so you can use nextTick () to add a new event to the main thread and it will be executed at the end of the whole block.
function main() {
//I want to run this both function in background to save time
let service = await GetServices();
this.process = await GetProcess();
…..//Here additional code is running
//Let say that after 30 second this code is called
Let users = GetUser(service);
Let users = GetAdr(this.process);
}
function someFunction(){
// do something...
}
main();
process.nextTick(someFunction());// happens after all main () processes are terminated...
The final line of this code successfully calls the _read method of a custom Duplex stream in node.
const timeContext = new TimeContext(sampleRate);
const input = new InputStream(timeContext); // Stream.Readable
const throttle = new Throttle(sampleRate); // Stream.Transform
const stackSource = [];
const stack = new StackStream(stackSource); // Stream.Duplex
input.pipe(throttle).pipe(stack);
stack.read(); // This will call the _read method of StackStream
Adding setTimeout to delay the stream.read() call, setTimeout's callback does NOT get called:
const timeContext = new TimeContext(sampleRate);
const input = new InputStream(timeContext); // Stream.Readable
const throttle = new Throttle(sampleRate); // Stream.Transform
const stackSource = [];
const stack = new StackStream(stackSource); // Stack.Duplex
input.pipe(throttle).pipe(stack);
setTimeout(() => {
stack.read(); // This callback never gets called
}, 1000);
It definitely does get called but something else is erroring
setTimeout(() => {
console.log('We got here');
stack.read(); // This is what is crashing in your code
console.log('We don\'t get here');
}, 1000);
It is just not behaving as you expect because some other error is occurring. Look in the console to see what errors are raised.
Looks like, read() function is a local property of the stack object and the setTimeout is not able to see this local property of stack object. That's why it's behaving in such a way.
Refer this solution for reference,
https://stackoverflow.com/a/4536268/10371717
I have an application where i have a Client class, that needs to be executed with different configurations for each user when i receive a message on discord(It's a chat application having its own API calls for checking for new messages etc). I have initialized the obj constructor whenever i receive a new message with new configurations for that user, but the problem is when i receive multiple messages at the same time, only the latest user's configurations are used for all the client objects created. Attaching sample code from the application:
Wait for message code:
const app = require("./Client")
const app2 = require("./MyObject")
bot.on('message', async (message) => {
let msg = message.content.toUpperCase(), pg;
let client = await new app2.MyObject();
//set all such configurations
config.a = "valuea";
// initialize my
pg = await new app.Client(config, client);
let result = await pg.Main(bot, message, params).then((result) => {
// Do stuff with result here
});
});
Client class:
class Client {
constructor(config, client) {
this.config = config;
this.client = client;
}
async Main(bot, message, params) {
let result = {};
this.client.setProperty1("prop");
await this.client.doSOmething();
result = await this.doSOmethingMore(message);
this.client.doCleanUp();
return result;
}
}
I had also tried initializing the obj constructor in the Client class, but even that fails for some reason.
Any suggestions how Can i correct my code?
You don't need to use .then and await at the same time.
bot.on('message', async (message) => {
let msg = message.content.toUpperCase();
let client = await new MyObject();
//set all such configurations
config.a = "valuea";
// initialize my
pg = new app.Client(config, client);
let result = await pg.Main(bot, message, params);
// Do stuff with result here
console.log(result);
});
(You don't need to use await at constructor call because it is not an async method)
note: async-await will work on higher version of node than 7.6
If you want to use .then:
bot.on('message', (message) => {
let msg = message.content.toUpperCase();
new MyObject().then(client => {
//set all such configurations
config.a = "valuea";
// initialize my
pg = new app.Client(config, client);
pg.Main(bot, message, params).then(result => {
// Do stuff with result here
console.log(result);
});
});
});
It's difficult to be certain, since you don't include the MyObject code, but generally it's a bad practice to return a promise from a constructor.
Further, if you don't do it right, then whatever that promise resolves to might not in fact be the Client that your calling code expects.