I have some code that continuously updates a series of objects via network calls looks like this. I was wondering if this is bad practice and if there might be a better way. I cant use Set Interval as the time between MakeAsyncCall replies is variable and can cause a leak if the time to make the call is longer than the delay. I will be using this info to update a UI. Will this cause blocking? What are your thoughts? Let me know if you need more info.
let group = [item1, item2, item3];
// Start Loop
readForever(group, 100);
// Function to Delay X ms
const delay = ms => {
return new Promise((resolve, _) => {
const timeout = setTimeout(() => {
resolve();
}, ms);
});
};
// Function to continuously Make Calls
const readForever = async (group, ms) => {
while(true) {
// Make Async Call
for (let item of group) {
await MakeAsyncCall(item);
}
// Wait X ms Before Processing Continues
await delay(ms);
}
};
The given code won't cause any UI blocking. And is a valid way to update the UI continually.
Instead of a loop you could write it that way:
const readForever = async (group, ms) => {
// Make Async Call
for (let item of group) {
await MakeAsyncCall(item);
}
// Wait X ms Before Processing Continues
await delay(ms);
if (true) { // not needed, but there you could define an end condition
return readForever(group, ms);
}
};
In addition to the comment about the delay function:
You could directly pass the resolve to setTimeout, and because you do not cancel the Timeout anywhere you do not need to store the result setTimeout in a variable.
const delay = ms => {
return new Promise(resolve => setTimeout(resolve, ms));
};
Related
I have a database setup with NodeJS and want to wait until certain table is created before start to create any others. This method tableExists resolves with the status being either true/false, but I want it to wait until it's true only.
const checkTableExists = async () => {
const exists = await queryInterface.tableExists('Subjects');
return exists;
}
How can I force a wait until checkTableExists returns true?
Using setTimeout:
const CHECK_INTERVAL = 200; // every 200ms
const checkTableExists = async () => {
const exists = await queryInterface.tableExists('Subjects');
if (!exists) {
return new Promise((resolve, reject) => {
setTimeout(() => checkTableExists().then(resolve).catch(reject), CHECK_INTERVAL);
});
}
return exists;
}
The solution to something like this is not to keep on waiting. There are other issues that may cause the table not to be created. You may want to adjust the above code to stop checking after it has checked for set number of times, or a duration has passed. Use something reasonable, depending on the environment where your db is running.
Add a delay and repeat:
// Utility function
const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
const checkTableExists = async () => {
while (true) {
const exists = await queryInterface.tableExists('Subjects');
if (exists) return true;
await delay(10000); // Wait 10 seconds before trying again.
}
}
Although this resolves the promise with true, it is actually is not necessary to return a boolean, as the resolving promise is enough as a signal that the table now exists -- true is the only possible outcome when the promise resolves.
The most common implementation of a sleep function in javascript is returning a Promise after setTimeout resolves:
function sleep(ms) {
return new Promise((resolve) => {
setTimeout(resolve, ms);
});
}
I have for loop with await sleep to keep it from executing too fast, such as not requesting xhr too fast. I also have a isBreak flag elsewhere to tell me when to stop the for loop. However, the issue I have is that when I break the for loop, the previous await sleep has already executed and is holding up the for loop. Is there a better way of breaking the for loop and also terminating the await sleep instantaneously?
const items = [];
let isBreak = false; // Somewhere else in the application
for (const item of items) {
if (isBreak) break;
// Do something, like xhr request
await sleep(15000); // 15 seconds sleep
if (isBreak) break;
}
Is there a way for me to signal for early
In JS, when an await operation starts, it can no longer be interrupted; it will wait until its operand promise is settled.
So, you have to make the promise you're awaiting cancelable in some way.
Unfortunately, your code can't get notified about a variable reassignment (when you set isBreak to true), and polling it would be inefficient.
Instead of a flag, you could use an AbortSignal (which was invented for this purpose), and make your sleep accept one:
function sleep(ms, signal) {
return new Promise((resolve, reject) => {
signal.throwIfAborted();
const timeout = setTimeout(() => {
resolve();
signal.removeEventListener('abort', abort);
}, ms);
const abort = () => {
clearTimeout(timeout);
reject(signal.reason);
}
signal.addEventListener('abort', abort);
});
}
Then, you use it like this:
const items = [];
const isBreak = new AbortController(); // Somewhere else in the application, call `isBreak.abort()`
try {
for (const item of items) {
// Do something, like xhr request
await sleep(15000, isBreak.signal); // 15 seconds sleep
}
} catch (e) {
if (e.name === 'TimeoutError') {
// Handle a cancellation
console.log('Cancelled');
} else {
// Not a cancellation, rethrow it
throw e;
}
}
An AbortSignal works well with fetch as well, in case you have to cancel that too.
An answer i found in a blog in the past that i adjusted. It is similiar to FZs answer. Same usage, too. Just to give an alternative.
relevant too: How to cancel timeout inside of Javascript Promise?
function sleep(ms, abortSignal) {
return new Promise((resolve, reject) => {
signal.addEventListener("abort", abort);
if(abortSignal.aborted){
abort();
}
const timeout = setTimeout(end, ms);
function abort() {
clearTimeout(timeout);
abortSignal.removeEventListener("abort", abort);
reject(new Error("sleep aborted"));
}
function end() {
abortSignal.removeEventListener("abort", abort);
resolve();
}
});
}
Arriving with a theory question :)
I have a front that sends (axios) N requests in a Promise.all() with a map function. This works fine. Each time one of the promises is good, I have a little table that gets updated with each request's answer until I get the full table and the array of the answers at the end. ✅
The problem comes when I want to read, at the same time, the logs of the server
So my objective is to run another axios request to my express.js server that will run each 2 seconds to retrieve the logs of the last 2 seconds, this way I could show the logs of what is happening with each answer in real time.
Any ideas of how doing this two tasks in parallel?
In the front I'm using react and the promise.All has this is structure:
setIsLoading(true); // setting a flag to know this is running
const doAllTheTable = await Promise.all(
tableData.map(async (lineOfMyTable) => {
const answer = await doMyRequest(lineOfMyTable) // my axios.get request
return updateTableLine(answer) // the functions that update the good line
})
);
//all promises are good now
setIsLoading(false)
So, basically I want to have another loop that runs each 2 seconds while "isLoading" is true to update another part of my front and show the logs meanwhile. But I need both things to happen at the same time!
Thank you for your ideas :)
Rather than awaiting your Promise.all immediately, store a reference to the promise so you can start checking the logs:
const doAllTheTablePromise = Promise.all(
tableData.map(async lineOfMyTable => {
const answer = await doMyRequest(lineOfMyTable); // my axios.get request
return updateTableLine(answer); // the functions that update the good line
});
);
let cancelled = false;
(async () => {
while (!cancelled) {
// Check your logs..
await new Promise(r => setTimeout(r, 2000)); // 2 second delay
}
})();
await doAllTheTablePromise;
cancelled = true;
Once your doAllTheTablePromise has resolved, you can stop checking the logs.
Must be many ways to write this. Here's one involving a token provided by the caller of two async processes, foo() and bar(), for communication between them.
async function foo(tableData, token) {
try {
await Promise.all(tableData.map(async (lineOfMyTable) => {
const answer = await doMyRequest(lineOfMyTable);
return updateTableLine(answer);
}));
token.setIsLoading = false; // lower flag when all requests are complete
} catch(error) {
token.setIsLoading = false; // lower flag if synchronous or asynchronous error occurs
}
}
async function bar(token) {
function delay(ms) { // this can be written as inner or outer function, whichever suits.
return new Promise(resolve => {
setTimeout(resolve, ms);
});
}
if(token.setIsLoading) {
let logs = await retrieveLogs();
// process/display logs here
await delay(2000);
return bar(token); // recursive call
} else {
return "complete"; // optional
}
}
async function myCaller() {
// ... preamble
let loadingToken = { // passed to foo() and bar() as a means of communication between them.
'setIsLoading': true // raise flag before calling foo() and bar().
};
return Promise.all(foo(tableData, loadingToken), bar(loadingToken));
}
EDIT:
Maybe better written like this, with the caller looking after lowering the flag:
async function foo(tableData) {
return Promise.all(tableData.map(async (lineOfMyTable) => {
return updateTableLine(await doMyRequest(lineOfMyTable));
}));
}
async function bar(token) {
function delay(ms) { // this can be written as inner or outer function, whichever suits.
return new Promise(resolve => {
setTimeout(resolve, ms);
});
}
if(token.setIsLoading) {
let logs = await retrieveLogs();
// process/display logs here
await delay(2000);
return bar(token); // recursive call
} else {
return "complete"; // optional
}
}
async function myCaller() {
// ... preamble
let loadingToken = { // passed to bar().
'setIsLoading': true // raise flag before calling foo() and bar().
};
return Promise.all(
foo(tableData).finally(() => { loadingToken.setIsLoading = false }),
bar(loadingToken)
);
}
I have a piece of code simplified version of which looks like this:
let dataStorage1; //declare global vars for easier access later on
let dataStorage2;
let stopLight = true; //this variable is used to 'mark' an iteration as successful (= true) or
//failed (= false) and in need of a retry before continuing to the next
//iteration
let delay = 2000; //the standard time for a delay between api calls
async function tryFetch() {
try {
dataStorage1 = await api.fetch('data_type_1'); //fetch needed data trough api, which
//fills the global variable with an
//object
dataStorage2 = await api.fetch('data_type_2'); //do the same
stopLight = true; //change the value of stopLight to true, thus marking this iteration
//as successful
} catch (err) {
console.log(err);
stopLight = false;
}
}
async function fetchData() {
stopLight = true; //change the stopLight to default before execution
await tryFetch(); //fetch data and assign it to variables
//this section is needed for retrial of fetching after a 2s delay if the first attempt was
//unsuccessful, which is repeated until it's either successful or critical error occurred
while (stopLight == false) {
setTimeout(async () => await tryFetch(), delay);
}
}
(async function main() {
await fetchData(); //finally call the function
setTimeout(main, delay); //repeat the main function after 2s
})();
As you can see, self-executing, pseudo-recursive main() calls for await fetchData(), then fetchData() calls for await tryFetch() and finally tryFetch() calls for await api.fetch('~'), as it's defined in the api.
However, once I started the script and paused it after a couple of iterations, I noticed that both dataStorage1 and dataStorage2 remain undefined. If I go through the code step by step in debugger, what happens is that the execution starts at the beginning of fetchData(), moves to the await tryFetch(); line, skips it, and then goes onto the next iteration.
For the reference, if I call dataStorage1/2 = await api.fetch(`~`); in the body of main() directly without any nesting, it works perfectly (unless error occurs, since they are not handled properly).
So, my question is what have I missed?
Indeed, if in an async function you call setTimeout you cannot expect it to perform an await on anything that relates to the callback passed to setTimeout. The call to setTimeout returns immediately, and your while loop is effectively a synchronous loop. It is a so called "busy loop" -- blocking your GUI as it potentially will loop for thousands of times.
As a rule of thumb, use setTimeout only once: to define a delay function, and then never again.
Also avoid using a global variable like stopLight: this is bad practice. Let the async function return a promise that resolves when this is supposed to be true, and rejects when not.
// Utility function: the only place to use setTimeout
const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms));
async function tryFetch() {
try {
let dataStorage1 = await api.fetch('data_type_1');
let dataStorage2 = await api.fetch('data_type_2');
return { dataStorage1, dataStorage2 }; // use the resolution value to pass results
} catch (err) {
console.log(err);
// retry
throw err; // cascade the error!
}
}
async function fetchData() {
while (true) {
try {
return await tryFetch(); // fetch data and return it
} catch (err) {} // repeat loop
}
}
(async function main() {
let intervalTime = 2000; //the standard time for a delay between api calls
while (true) { // for ever
let { dataStorage1, dataStorage2 } = await fetchData();
// ... any other logic that uses dataStorage1, dataStorage2
// should continue here...
await delay(intervalTime); //repeat the main function after 2s
}
})();
I think the problem is in this line: setTimeout(async () => await tryFetch(), delay);. The await statement inside the callback makes the promise returned by that callback wait, not the whole function. So async () => await tryFetch() is a function that returns a promise, but nothing waits for that promise to complete.
Try replacing that code with something line
await new Promise((resolve) => setTimeout(resolve, delay));
await tryFetch();
I need my program to repeat itself continuously. My program starts to fetch proxies from servers and saves them to a database and then send those saved proxies to another server again. So I don't know how long it takes for my program to do this task.
I wanna know what happens if any problem happens that makes this startJob() function take more than 30 seconds.
Does setInterval call it again or waits for function to finish?
What's the best approach for my program to repeat itself after it's done without setInterval?
(for exmaple startJob() being called again after it's done.)
I was wondering if it is ok to put this function in a loop with a big number like:
for ( let i = 0 ; i < 999999999 ; i ++ ) {
await startJob()
}
Here is my code:
const startJob = async () => {
await postProxyToChannel()
grabProxies()
}
setInterval(function(){
startJob()
}, (30000))
grabProxies() takes about 10 seconds and postProxyToChannel() takes about 5 seconds on my server.
No matter what happens inside startJob, setInterval will call it every 30 seconds. This means that postProxyToChannel will be called every 30 seconds. If that function throws, you'll get an unhandled Promise rejection, but the interval will continue.
Even if postProxyToChannel takes, say, 45 seconds, that won't prevent startJob from being called again before the prior startJob has completed.
If you want to make sure that startJob is only called 30 seconds after it finishes, you could await it in your for loop, then await a Promise that resolves every 30 seconds:
(async () => {
for ( let i = 0 ; i < 999999999 ; i ++ ) {
await startJob();
await new Promise(resolve => setTimeout(resolve, 30000));
}
})()
.catch((err) => {
console.log('There was an error', err);
});
But it would probably make more sense just to have a recursive call of startJob, eg:
const startJob = async () => {
try {
await postProxyToChannel();
} catch(e) {
// handle error
}
grabProxies();
setTimeout(startJob, 30000);
};
startJob();
Yup an infinite loop sounds good, that can be compared with a timer to pause the loop:
const timer = ms => new Promise(resolve => setTimeout(resolve, ms));
(async function() {
while(true) {
await postProxyToChannel();
await grabProxies();
await timer(30000);
}
})();
Now that loop will run the task, wait 30secs, then do that again. Therefore the loop will not run every 30secs but will usually take longer. To adjust that, you could measure the time the task took, then await the rest of the time:
const start = Date.now();
await postProxyToChannel();
await grabProxies();
await timer(30000 - (Date.now() - start));