Develop Reference

How to catch an error on a async callback function on outer try/catch block

Ok,
So I am using the puppeteer framework and I have an async function that interact with a webpage.
This function clicks and selects and elements of a webpage while it waiting for the traffic of the page to be idle.
This function works most of the time, but sometimes it stalls.
I want to be able to set a timeout so that if the function is taking longer than a certain amount of time, it throws an error and I can run it again.
So far I cannot seem to get this to work because I cannot get the callback function I pass to setTimeOut() to 'interact' with the outer function.
My code looks like this:
const scrap_webtite = async page => {
/* scrap the site */
try{ // catch all
// set timeout
let timed_out_ID = setTimeout(()=> throw "timeOut", 1000);
// run the async
let el = await sometimes_stalls_function(page);
// if function ran finished correcly
clearTimeout(timed_out_ID);
// save el
save_el(el);
}
}catch(e){
console.error("Something went wrong!", e);
// this makes the function run again
// here is where I want to ideally catch the timeout error
return false
}
}
I have also tried wrapping the setTimeOut function in an Promise as per this post and the using the .then().catch() callbacks to try to catch the error to no avail.
Apologies if this is a stupid question, thank for you help.

The problem you're running into is essentially that the error thrown in setTimeout() is not related to your function flow, and thus can't be caught there. You can essentially think of the timer's callback function as a "detached" function: the variables from the parent scope will still be available, but you can't return a value to the parent directly etc.
To work around this problem you have a few options, Promise.race() is one possible solution. The idea is to first make an async version of a timeout:
const rejectAfter = (timeout) => {
return new Promise((resolve, reject) => {
setTimeout(() => reject(), timeout);
});
};
Then extract your business logic out into a separate async function a-la:
const doTheThing = async () => {
// TODO: Implement
};
And finally in your scraping function, use Promise.race() to use the result from whichever of the two finishes first:
const scrape = async (page) => {
try {
const el = await Promise.race([
rejectAfter(1000),
doTheThing()
]);
} catch(error) {
// TODO: Handle error
}
}

try turning everything in the try block into a promise
const scrap_webtite = async page => {
/* scrap the site */
try{ // catch all
return await new Promise(async(r,j)=>{
// set timeout
let timed_out_ID = setTimeout(()=>j("timeOut"),1000);
// run the async
let el = await sometimes_stalls_function(page);
// if function ran finished correcly
clearTimeout(timed_out_ID);
// save el
r(save_el(el));
})
}catch(e){
console.error("Something went wrong!", e);
// this makes the function run again
// here is where I want to ideally catch the timeout error
return false
}
}

Fetching data from server randomly stops fetching

I have something very strange happening in my program. I have consol.logged the crap out of it with timestamps to try and figure out what is happening. Basically my program randomly stops fetching data. I have ensured a new stream of data is there but I have to refresh the entire page or resave the program to restart everything when it gets hung up. On top of that, there are no errors or flags telling me why it stops. I tried to isolate the issue but it is something to do with the async function most likely. Here is the code....
function App() {
const data = async() => {
try {
console.log('try block initiated')
const apiResponse = await fetch(ipAddress)
console.log(apiResponse);
console.log("BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB")
const responseText = await apiResponse.text();
console.log(responseText)
console.log("CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC")
if (loading === true){
setLoading(false);
}
console.log("DDDDDDDDDDDDDDDDDDDDDDDDDDDDDD")
return responseText;
} catch (error) {
console.log('catch initiated')
setError(true);
}
};
console.log("AAAAAAAAAAAAAAAAAAAAAAA")
try{
console.log("EEEEEEEEEEEEEEEEEEEEEEEEEEE")
data().then(response=>setMoisture(response));
console.log("FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF")
} catch(error){
console.log('gotcha')
setError(true);
}
let value = moisture;
console.log(value);
count += 1;
console.log(count);
return(
<div className="App">
<WeatherCard moisture={value}/>
</div>
);
}
Here is what the console looks like right before it stop fetching new data...

HERE IS A POSSIBLE SOLUTION. After many headaches and researching async functions. I think this is going to work. I am going to let this run all night and see if it ever gets caught up but so far, Here is the solution...
const data = async() => {
try {
const apiResponse = await fetch(ipAddress)
const responseText = await apiResponse.text();
console.log("CCCCCCCCCCCCCCCCCCC");
console.log(responseText);
if (loading === true){
setLoading(false);
}
data().then(response=>setMoisture(response));
return responseText;
} catch (error) {
console.log("EEEEERRRRRRRRRROOOOOOOOOOORRRRRRRRRRRR")
setError(true);
}
};
console.log("AAAAAAAAAAAAAAAAAAAA");
if (moisture === "initialState"){
data().then(response=>setMoisture(response));
}else{
console.log("QQQQQQQQQQQQQQQQQQQQQQQQ")
}
Here is what I changed... I made an initial state called "initialSate and if the variable was this value, it runs the async function. I then NEVER CALL THIS ASYNC AGAIN OUTSIDE OF THE ASYNS ITSELF. Instead I call itself right before returning a value from the function. This works because my program is running while the async function is running. However, the program is sequential to itself and the async is sequential to itself. They are NOT sequential together. So, the async function now only gets called again once the async function finishes its first run. Simultaneaously the rest of my program is running. Before I think it was getting caught because my program was trying to call the async function before it had finished the first time. This is a pure guess because I have no way to proving that but it makes sense. If this overnight run fails. I will pull this from the solution. Otherwise, I think this will work. FINGERS CROSSED!

retrieve logs from server while performing a "parallel" promise.all( request to the server) (parallel challenge!)

Arriving with a theory question :)
I have a front that sends (axios) N requests in a Promise.all() with a map function. This works fine. Each time one of the promises is good, I have a little table that gets updated with each request's answer until I get the full table and the array of the answers at the end. ✅
The problem comes when I want to read, at the same time, the logs of the server
So my objective is to run another axios request to my express.js server that will run each 2 seconds to retrieve the logs of the last 2 seconds, this way I could show the logs of what is happening with each answer in real time.
Any ideas of how doing this two tasks in parallel?
In the front I'm using react and the promise.All has this is structure:
setIsLoading(true); // setting a flag to know this is running
const doAllTheTable = await Promise.all(
tableData.map(async (lineOfMyTable) => {
const answer = await doMyRequest(lineOfMyTable) // my axios.get request
return updateTableLine(answer) // the functions that update the good line
})
);
//all promises are good now
setIsLoading(false)
So, basically I want to have another loop that runs each 2 seconds while "isLoading" is true to update another part of my front and show the logs meanwhile. But I need both things to happen at the same time!
Thank you for your ideas :)

Rather than awaiting your Promise.all immediately, store a reference to the promise so you can start checking the logs:
const doAllTheTablePromise = Promise.all(
tableData.map(async lineOfMyTable => {
const answer = await doMyRequest(lineOfMyTable); // my axios.get request
return updateTableLine(answer); // the functions that update the good line
});
);
let cancelled = false;
(async () => {
while (!cancelled) {
// Check your logs..
await new Promise(r => setTimeout(r, 2000)); // 2 second delay
}
})();
await doAllTheTablePromise;
cancelled = true;
Once your doAllTheTablePromise has resolved, you can stop checking the logs.

Must be many ways to write this. Here's one involving a token provided by the caller of two async processes, foo() and bar(), for communication between them.
async function foo(tableData, token) {
try {
await Promise.all(tableData.map(async (lineOfMyTable) => {
const answer = await doMyRequest(lineOfMyTable);
return updateTableLine(answer);
}));
token.setIsLoading = false; // lower flag when all requests are complete
} catch(error) {
token.setIsLoading = false; // lower flag if synchronous or asynchronous error occurs
}
}
async function bar(token) {
function delay(ms) { // this can be written as inner or outer function, whichever suits.
return new Promise(resolve => {
setTimeout(resolve, ms);
});
}
if(token.setIsLoading) {
let logs = await retrieveLogs();
// process/display logs here
await delay(2000);
return bar(token); // recursive call
} else {
return "complete"; // optional
}
}
async function myCaller() {
// ... preamble
let loadingToken = { // passed to foo() and bar() as a means of communication between them.
'setIsLoading': true // raise flag before calling foo() and bar().
};
return Promise.all(foo(tableData, loadingToken), bar(loadingToken));
}
EDIT:
Maybe better written like this, with the caller looking after lowering the flag:
async function foo(tableData) {
return Promise.all(tableData.map(async (lineOfMyTable) => {
return updateTableLine(await doMyRequest(lineOfMyTable));
}));
}
async function bar(token) {
function delay(ms) { // this can be written as inner or outer function, whichever suits.
return new Promise(resolve => {
setTimeout(resolve, ms);
});
}
if(token.setIsLoading) {
let logs = await retrieveLogs();
// process/display logs here
await delay(2000);
return bar(token); // recursive call
} else {
return "complete"; // optional
}
}
async function myCaller() {
// ... preamble
let loadingToken = { // passed to bar().
'setIsLoading': true // raise flag before calling foo() and bar().
};
return Promise.all(
foo(tableData).finally(() => { loadingToken.setIsLoading = false }),
bar(loadingToken)
);
}

Nodejs running function on background

I’ve nodes program which I need to run two function in the beginning of the program
And later on access the function results, currently with await each function at a time this works,
However in order to save a time and not waiting to GetService and GetProcess as I need the data later on in the project
It takes about 4 seconds to get this data and I want to run it on the background as I don’t need the results immediately,
How I can do it in node js, If I run promise.all It would wait until the getService and getProcess and then go to rest of the program.
an example
function main() {
//I want to run this both function in background to save time
let service = await GetServices();
this.process = await GetProcess();
…..//Here additional code is running
//let say that after 30 second this code is called
Let users = GetUser(service);
Let users = GetAdress(this.process);
}
im actually running yeoman generator
https://yeoman.io/authoring/
https://yeoman.io/authoring/user-interactions.html
export default class myGenerator extends Generator {
//here I want run those function in background to save time as the prompt to the user takes some time (lets say user have many questions...)
async initializing() {
let service = await GetServices();
this.process = await GetProcess();
}
async prompting() {
const answers = await this.prompt([
{
type: "input",
name: "name",
message: "Your project name",
default: this.appname // Default to current folder name
},
{
type: "confirm",
name: "list",
choises: this.process //here I need to data from the function running in background
}
]);
}

Let's assume that getServices() may take 3 seconds and getProcess() may take 4 seconds, so if you run these both functions at the same time you will be returned in total 4 seconds with the return values from both promises.
You can execute the code while this process is running in the background there will be a callback when the promises resolved, your late functions will be called at this stage.
Check the below simple example;
let service;
let process;
function main() {
// Both functions will execute in background
Promise.all([getServices(), getProcess()]).then((val) => {
service = val[0];
process = val[1];
console.log(service, process);
// Aafter completed this code will be called
// let users = GetUser(service);
// let users = GetAdress(process);
console.log('I am called after all promises completed.')
});
// Current example.
// let service = await GetServices();
// this.process = await GetProcess();
/* Code blocks.. */
console.log('Code will execute without delay...')
}
function getServices() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve("service is returned")
}, 3000);
});
}
function getProcess() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve("process is returned")
}, 4000);
});
}
main();

You can start the asynchronous operation but not await it yet:
function suppressUnhandledRejections(p) {
p.catch(() => {});
return p;
}
async function main() {
// We have to suppress unhandled rejections on these promises. If they become
// rejected before we await them later, we'd get a warning otherwise.
const servicePromise = suppressUnhandledRejections(GetServices());
this.processPromise = suppressUnhandledRejections(GetProcess());
// Do other stuff
const service = await servicePromise;
const process = await this.processPromise;
}
Also consider using Promise.all() which returns a promise for the completion of all promises passed to it.
async function main() {
const [ services, process, somethingElse ] = await Promise.all([
GetServices(),
GetProcess(),
SomeOtherAsyncOperation(),
]);
// Use the results.
}

To do what who you need, you have to understand the event loop.
Nodejs is designed to work in a single thread unlike languages like go, however nodejs handle proccess on different threads. so you can use nextTick () to add a new event to the main thread and it will be executed at the end of the whole block.
function main() {
//I want to run this both function in background to save time
let service = await GetServices();
this.process = await GetProcess();
…..//Here additional code is running
//Let say that after 30 second this code is called
Let users = GetUser(service);
Let users = GetAdr(this.process);
}
function someFunction(){
// do something...
}
main();
process.nextTick(someFunction());// happens after all main () processes are terminated...

Nested multilayered async/await doesn't seem to wait

I have a piece of code simplified version of which looks like this:
let dataStorage1; //declare global vars for easier access later on
let dataStorage2;
let stopLight = true; //this variable is used to 'mark' an iteration as successful (= true) or
//failed (= false) and in need of a retry before continuing to the next
//iteration
let delay = 2000; //the standard time for a delay between api calls
async function tryFetch() {
try {
dataStorage1 = await api.fetch('data_type_1'); //fetch needed data trough api, which
//fills the global variable with an
//object
dataStorage2 = await api.fetch('data_type_2'); //do the same
stopLight = true; //change the value of stopLight to true, thus marking this iteration
//as successful
} catch (err) {
console.log(err);
stopLight = false;
}
}
async function fetchData() {
stopLight = true; //change the stopLight to default before execution
await tryFetch(); //fetch data and assign it to variables
//this section is needed for retrial of fetching after a 2s delay if the first attempt was
//unsuccessful, which is repeated until it's either successful or critical error occurred
while (stopLight == false) {
setTimeout(async () => await tryFetch(), delay);
}
}
(async function main() {
await fetchData(); //finally call the function
setTimeout(main, delay); //repeat the main function after 2s
})();
As you can see, self-executing, pseudo-recursive main() calls for await fetchData(), then fetchData() calls for await tryFetch() and finally tryFetch() calls for await api.fetch('~'), as it's defined in the api.
However, once I started the script and paused it after a couple of iterations, I noticed that both dataStorage1 and dataStorage2 remain undefined. If I go through the code step by step in debugger, what happens is that the execution starts at the beginning of fetchData(), moves to the await tryFetch(); line, skips it, and then goes onto the next iteration.
For the reference, if I call dataStorage1/2 = await api.fetch(`~`); in the body of main() directly without any nesting, it works perfectly (unless error occurs, since they are not handled properly).
So, my question is what have I missed?

Indeed, if in an async function you call setTimeout you cannot expect it to perform an await on anything that relates to the callback passed to setTimeout. The call to setTimeout returns immediately, and your while loop is effectively a synchronous loop. It is a so called "busy loop" -- blocking your GUI as it potentially will loop for thousands of times.
As a rule of thumb, use setTimeout only once: to define a delay function, and then never again.
Also avoid using a global variable like stopLight: this is bad practice. Let the async function return a promise that resolves when this is supposed to be true, and rejects when not.
// Utility function: the only place to use setTimeout
const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms));
async function tryFetch() {
try {
let dataStorage1 = await api.fetch('data_type_1');
let dataStorage2 = await api.fetch('data_type_2');
return { dataStorage1, dataStorage2 }; // use the resolution value to pass results
} catch (err) {
console.log(err);
// retry
throw err; // cascade the error!
}
}
async function fetchData() {
while (true) {
try {
return await tryFetch(); // fetch data and return it
} catch (err) {} // repeat loop
}
}
(async function main() {
let intervalTime = 2000; //the standard time for a delay between api calls
while (true) { // for ever
let { dataStorage1, dataStorage2 } = await fetchData();
// ... any other logic that uses dataStorage1, dataStorage2
// should continue here...
await delay(intervalTime); //repeat the main function after 2s
}
})();

I think the problem is in this line: setTimeout(async () => await tryFetch(), delay);. The await statement inside the callback makes the promise returned by that callback wait, not the whole function. So async () => await tryFetch() is a function that returns a promise, but nothing waits for that promise to complete.
Try replacing that code with something line
await new Promise((resolve) => setTimeout(resolve, delay));
await tryFetch();