First of all, I'm sorry if this is a basic question, but I am not being able to do this.
I have a promise that inserts data into a table (sqlite), and I have a for iterating over an array. I want to put all of that data into the table, but I want to know when it ends, to display a message at the end. I was verifying (i == array.length -1) it displayed the message but that doesn't seem correct. If I don't do this in this way the message displays before it has ended.
I have two other promises that should run along this one and the solution(it seems a bad solution) above wouldn't work in this case, because one can end after or before the iteration. How can I know when they all are done too?
Could you help me, please?
Here's my code:
for (let i = 0; i < this.data_array.length; i++) {
this.database.insertService(this.data_array[i]).then(async () => {
console.log('Inserting object number ' + this.data_array[i].id);
if( i == this.data_array.length - 1) {
console.log('done!');
}
}).catch(err => console.log('error inserting object into the table'));
}
Thank you.
It is a better practice to use Promise.all when dealing with such cases. A simple example would be the following:
let i;
let promises = [];
for (i = 0; i < 5; ++i)
promises.push(someAsyncFunc(i));
Promise.all(promises)
.then((results) => {
console.log("All done", results);
})
.catch((e) => {
console.log(e)
});
You can solve this using Promise.all
Here is an example and here some useful documentation
the example is using map instead of a for loop to iterate over the array.
I haven't tested it but that should work
Promise.all(
this.data_array.map((data) => {
console.log('Inserting object number ' + data.id);
return this.database.insertService(data);
})
)
.then((result) => console.log('done', result))
.catch((error) => console.log(error));
You could use Promise.all([]) where .all() take an array.
Promise.all(
// Map the items in data_array to the async insertService function
this.data_array.map(dataEntry => this.database.insertService(dataEntry))
).then((resultArray) => {
console.log(resultArray.length);
console.log('done!');
}).catch((e) => {
console.log(e)
});
I think that Promise.all is the best solution here
async method() {
const promises = this.data_array.map(item =>
this.database.insertService(item));
try {
const result = await Promise.all(promises);
console.log({ result });
} catch (error) {
console.log({ error });
}
}
Related
I have a function that tries to load web images and tracks the count of loaded images and the count of the failed images. I am loading the images using fetch and using Promise.allSettled to run operations after all the images are validated.
const data = ["/test1.png", "/test2.png", "/test3.png"];
let imagesValidated = 0;
let imagesFailed = 0;
const promiseArr = [];
data.forEach((item) => {
const imgPromise = fetch(item);
promiseArr.push(imgPromise);
imgPromise
.then((resp) => {
if (!resp.ok()) imagesFailed += 1;
})
.catch((error) => {
imagesFailed += 1;
})
.finally(() => {
// For the last image `test3.png`, the finally blocks runs after `allSettled`.
imagesValidated += 1;
});
});
Promise.allSettled(promiseArr).then(() => {
// some operations
});
The issue I am facing is with the finally block. For the last image the finally block is running after the allSettled callback. This causes the imagesValidated to be lesser than the actual images scanned count. I do not want to remove the finally block as in the future I will be adding more cleanup code into it.
Is this the expected behavior of the Promise resolution methods? Is there a way I can fix this code without removing the finally block?
You're pushing the fetch Promise to the array - not the chain that goes through .then and .finally. Push the whole chained Promise to the array.
data.forEach((item) => {
promiseArr.push(
fetch(item)
.then((resp) => {
if (!resp.ok()) imagesFailed += 1;
})
.catch((error) => {
imagesFailed += 1;
})
.finally(() => {
// For the last image `test3.png`, the finally blocks runs after `allSettled`.
imagesValidated += 1;
})
);
});
Promise.allSettled(promiseArr).then(() => {
// some operations
});
Or, even better, use .map on the original data instead.
Promise.allSettled(
data.map(item => fetch(item)
.then((resp) => {
if (!resp.ok()) imagesFailed += 1;
})
.catch((error) => {
imagesFailed += 1;
})
.finally(() => {
// For the last image `test3.png`, the finally blocks runs after `allSettled`.
imagesValidated += 1;
})
)
.then(() => {
// some operations
});
Though, note that using Promise.allSettled isn't helping you that much here - none of the Promises can reject due to the .catch. Consider either using Promise.all, or use Promise.allSettled with just the .fetch, so you can increment the counters after all responses have come back.
so what I have right now is
window.addEventListener('load', fetchInfo)
function fetchInfo() {
const tableRows = //an array of results
tableRows.forEach((row) => {
const rowId = //get the id of each row
fetch(...) //fetch some stuff using the id
.then(() => {
//do some stuff
return rowId;
})
.then((id) => {
//do some stuff
}
})
})
}
basically using rowId to fetch information and populate each table row, so this happens a few times, the table maxes out at 10 rows so max 10 fetches
I want to have an event listener to see when all the fetching is done, aka when the table is completely done loading. How should I go about that?
Edit: these fetches are api requests so they take a few seconds to respond. I've tried using Promise.all(tableRows.map(row) => and it returned results before the api could respond. So in the end, it still doesn't really detect when does the table actually finish loading information.
Use Promise.all:
async function fetchInfo() {
const tableRows = //an array of results
const results = await Promise.all(tableRows.map(row => fetch(...))
console.log(results)
}
From the suggestion in the comments length was indeed the way, however that would only be half of the solution. Because the key point here is that I want to know whether or not the table has been fully populated with requested information.
Tried Promise.all and map() and didn't work out because as stated in my edit, these will return when the fetch was called, when the fetch was still pending, and doesn't really care if the fetch was a 200 OK, which was what I needed.
So the solution was to use response.status
function fetchInfo() {
const tableRows = ...
let successFetch = 0;
tableRows.forEach((row) => {
fetch(...)
.then((response) => {
if (response.status == 200) {
successFetch = successFetch + 1
}
if (successFetch == tableRows.length) {
//this point here was exactly what I needed
}
return response;
})
.then((response) => {
...
})
})
}
Use Promise.all to wait for all promises to complete
Promise.all(
tableRows.map((row) => {
return fetch(...).then(() => {
return rowId;
})
})
).then((results) => {
console.log(results);
})
I have some code that does this: First scrape this array of webpages. After that, scrape another array of webpages.
The following code does what I expect:
let bays=[];
let promises=promisesN=[];
for (let y=2019;y>=2015;y--)
promises.push(new Promise(resolve=>
curl.get(`/*url*/${y}.html`,null, (error,resp,body)=>
resp.statusCode==200? resolve(parse(body)):reject(error)
)));
Promise.all(promises).then(()=>{
bays.forEach(bay=>{
if (bay.no.match(/\d+/)<=103) return;
promisesN.push(new Promise(resolve=>
curl.get(`/*url*/${bay.code}/`,null, (error,resp,body)=>
resp.statusCode==200? resolve(image(bey,body)):reject(error)
)))});
Promise.all(promisesN).then(()=>{
bays.sort((a,b)=>{return parseInt(a.no.match(/\d+/))<parseInt(b.no.match(/\d+/))? -1:1});
console.log(bays);
});
}).catch(error=>console.log(error));`
So I've read you can write a simplier nesting-free syntax:
doSomething()
.then(function(result) {
return doSomethingElse(result);
})
.then(function(newResult) {
return doThirdThing(newResult);
})
.then(function(finalResult) {
console.log('Got the final result: ' + finalResult);
})
.catch(failureCallback);
How to apply this to the code above?
correctness
let promises=promisesN=[];
This is really incorrect. It makes both variables reference the same array, and makes promisesN an implicit global. The fact that it appears to work means you aren’t in strict mode. Always use strict mode. The correct version of what you intended is:
let promises = [];
let promisesN = [];
cleanliness
new Promise(resolve=>
curl.get(`/*url*/${y}.html`,null, (error,resp,body)=>
resp.statusCode==200? resolve(parse(body)):reject(error)
))
You’re repeating this pattern, so make it into a function, or use a package that does the job for you, like request-promise[-native] or axios. (Also, please show your real code. reject isn’t defined here.)
const getAsync = url => new Promise((resolve, reject) => {
curl.get(url, null, (error, resp, body) => {
if (resp.statusCode === 200) {
resolve(body);
} else {
reject(error);
}
});
});
Notice how you’re free to make the function more readable when it isn’t repeated, and to extend it later.
let promises = [];
let promisesN = [];
for (let y = 2019; y >= 2015; y--) {
promises.push(getAsync(`/*url*/${y}.html`).then(parse));
}
Promise.all(promises).then(bays => {
bays.forEach(bay => {
if (bay.no.match(/\d+/) <= 103) return;
promisesN.push(getAsync(`/*url*/${bay.code}/`).then(body => image(bay, body)));
});
Promise.all(promisesN).then(() => {
bays.sort((a, b) => {return parseInt(a.no.match(/\d+/)) < parseInt(b.no.match(/\d+/)) ? -1 : 1;});
console.log(bays);
});
}).catch(error => console.log(error));
I had to take a few guesses at what your real code looks like again, because you’re surely doing something with the resolved value of Promise.all(promises). It doesn’t have any easily-accessible side-effects. bey also seemed likely enough to be bay.
Now you can give promisesN a more appropriate scope:
let promises = [];
for (let y = 2019; y >= 2015; y--) {
promises.push(getAsync(`/*url*/${y}.html`).then(parse));
}
Promise.all(promises).then(bays => {
let promisesN = bays
.filter(bay => bay.no.match(/\d+/) > 103)
.map(bay => getAsync(`/*url*/${bay.code}/`).then(body => image(bay, body)));
Promise.all(promisesN).then(() => {
bays.sort((a, b) => {return parseInt(a.no.match(/\d+/)) < parseInt(b.no.match(/\d+/)) ? -1 : 1;});
console.log(bays);
});
}).catch(error => console.log(error));
and use an expression-bodied arrow function where appropriate, since you’re already using them whenever they aren’t appropriate:
bays.sort((a, b) => parseInt(a.no.match(/\d+/)) < parseInt(b.no.match(/\d+/)) ? -1 : 1);
Now, if my guess about bays is right, then you can’t unnest. If it comes from somewhere else then you can. Normally I would leave a comment about that but I already wrote all this, so… please clarify that for further cleanup.
If you're looking to simplify your code, you might consider the use of async/await instead of promises.
The async/await syntax will greatly simplify the presentation and ease comprehension of the code, especially given that your logic relies on asynchronous iteration of arrays.
Consider the following code revision of your code:
/* Define local helper that wraps curl() in async function declaration */
function async doRequest(url) {
return (await new Promise(resolve=> curl.get(url, null, (error,resp,body) =>
resp.statusCode==200 ? resolve(res) : reject(error))))
}
/* Re-define simplified scrape logic using await/async */
function async doScrape() {
try {
var bays = []
/* Iterate date range asynchronously */
for (let y=2019; y>=2015; y--) {
/* Use doRequest helper function to fetch html */
const response = await doRequest(`/*url*/${y}.html`)
const bay = parse(response)
bays.push(bay)
}
/* Iterate bays array that was obtained */
for(const bay of bays) {
/* Use doRequest helper again to fetch data */
const response = await doRequest(`/*url*/${bay.code}/`)
/* Await may not be needed here */
await image(bay, response)
}
/* Perform your sort (which is non asynchronous) */
bays.sort((a,b)=> parseInt(a.no.match(/\d+/))<parseInt(b.no.match(/\d+/))? -1:1);
console.log("Result", bays);
}
catch(err) {
/* If something goes wrong we arrive here - this is
essentially equivalent to your catch() block */
console.error('Scrape failed', err);
}
}
/* Usage */
doScrape()
Hope that helps!
Not entirely sure if this is what you want, but I've separated your code out a bit because I found it easier for me to read.
let bays = [];
let promises = [];
let promisesN = [];
for (let y = 2019; y >= 2015; y--) {
const promiseOne = new Promise((resolve, reject) => {
return curl.get(`/*url*/${y}.html`, null, (error, resp, body) => {
resp.statusCode === 200 ? resolve(parse(body)) : reject(error);
});
});
promises.push(promiseOne);
}
Promise.all(promises)
.then(() => {
bays.forEach((bay) => {
if (bay.no.match(/\d+/) <= 103) {
return;
}
const promiseTwo = new Promise((resolve, reject) => {
return curl.get(`/*url*/${bay.code}/`, null, (error, resp, body) => {
resp.statusCode === 200 ? resolve(image(bay, body)) : reject(error);
});
});
promisesN.push(promiseTwo);
});
return Promise.all(promisesN);
})
.then(() => {
bays.sort((a, b) => {
return parseInt(a.no.match(/\d+/), 10) < parseInt(b.no.match(/\d+/), 10) ? -1 : 1;
});
console.log(bays);
})
.catch((error) => {
console.log(error);
});
I am wondering though, you are firing the promises instantly on each iteration of your for loop. This might be intentional, but it means if those promises resolve before the code gets to execute Promise.all you may run into issues. I personally would do something like, e.g. const promiseOne = () => somePromise, that way you can create a bunch of promises, and then once they're all created, map over that array and fire them at once. Same thing goes for the second promises.
Not sure if this is helpful, let me know if it is. Feel free to ask more questions too.
Hi I need to execute promises one after the other how do I achieve this using promise.all any help would be awesome. Below is the sample of my code I am currently using but it executes parallel so the search will not work properly
public testData: any = (req, res) => {
// This method is called first via API and then promise is triggerd
var body = req.body;
// set up data eg 2 is repeated twice so insert 2, 5 only once into DB
// Assuming we cant control the data and also maybe 3 maybe inside the DB
let arrayOfData = [1,2,3,2,4,5,5];
const promises = arrayOfData.map(this.searchAndInsert.bind(this));
Promise.all(promises)
.then((results) => {
// we only get here if ALL promises fulfill
console.log('Success', results);
res.status(200).json({ "status": 1, "message": "Success data" });
})
.catch((err) => {
// Will catch failure of first failed promise
console.log('Failed:', err);
res.status(200).json({ "status": 0, "message": "Failed data" });
});
}
public searchAndInsert: any = (data) => {
// There are database operations happening here like searching for other
// entries in the JSON and inserting to DB
console.log('Searching and updating', data);
return new Promise((resolve, reject) => {
// This is not an other function its just written her to make code readable
if(dataExistsInDB(data) == true){
resolve(data);
} else {
// This is not an other function its just written her to make code readable
insertIntoDB(data).then() => resolve(data);
}
});
}
I looked up in google and saw the reduce will help I would appreciate any help on how to convert this to reduce or any method you suggest (Concurrency in .map did not work)
the Promises unfortunatelly does not allow any control of their flow. It means -> once you create new Promise, it will be doing its asynchronous parts as they like.
The Promise.all does not change it, its only purpose is that it checks all promises that you put into it and it is resolved once all of them are finished (or one of them fail).
To be able to create and control asynchronous flow, the easiest way is to wrap the creation of Promise into function and create some kind of factory method. Then instead of creating all promises upfront, you just create only one promise when you need it, wait until it is resolved and after it continue in same behaviour.
async function doAllSequentually(fnPromiseArr) {
for (let i=0; i < fnPromiseArr.length; i++) {
const val = await fnPromiseArr[i]();
console.log(val);
}
}
function createFnPromise(val) {
return () => new Promise(resolve => resolve(val));
}
const arr = [];
for (let j=0; j < 10; j++) {
arr.push(createFnPromise(Math.random()));
}
doAllSequentually(arr).then(() => console.log('finished'));
PS: It is also possible without async/await using standard promise-chains, but it requires to be implemented with recursion.
If anyone else cares about ESLint complaining about the use of "for" and the "no await in loop" here is a typescript ESLint friendly version of the above answer:
async function runPromisesSequentially<T>(promises: Array<Promise<T>>):Promise<Array<T>> {
if (promises.length === 0) return [];
const [firstElement, ...rest] = promises;
return [await firstElement, ...(await runPromisesSequentially(rest))];
}
You can then just replace Promise.all by runPromisesSequentially.
#lmX2015's answer is close but it's taking in promises that have already started executing.
A slight modification fixes it
export async function runPromisesSequentially<T>(functions: (() => Promise<T>)[]): Promise<T[]> {
if (functions.length === 0) {
return [];
}
const [first, ...rest] = functions;
return [await first(), ...(await runPromisesSequentially(rest))];
}
I am new to Nodejs and first time working on promises so now the context is when I try to return promise it shows status Promise . How to fix it can anyone guide me through this?
Here is the code where I am calling a function that will return a promise. bold line showing where I want to return that promise and store in an object.
for(let i = 0; i<responseArray.length; i++){
let dollar = {
amount : 0
};
if(i == 1){
continue;
}
dollar.amount = **currenciesService.getCurrencyLatestInfo(responseArray[i].currency);**
dollarAmount.push(dollar);
}
console.log("$", dollarAmount);
Here is a code which is returning promise.
const getCurrencyLatestInfo = function(currency) {
return new Promise(function(resolve, reject) {
request('https://min-api.cryptocompare.com/data/price?fsym='+currency+'&tsyms='+currency+',USD', { json: true }, (err, res, body) =>
{
if (err) {
reject(err);
} else {
var result= body;
resolve(result);
console.log("RESULT: ",result.USD);
}
});
})
}
You'll need to wait for those promises to resolve before you can use the resolved values
here is a small rewrite of your loop that should work
let promises = [];
for(let i = 0; i<responseArray.length; i++){
if(i == 1){
continue;
}
let dollar = currenciesService.getCurrencyLatestInfo(responseArray[i].currency)
.then(amount => ({amount})); // do you really want this?
promises.push(dollar);
}
Promise.all(promises)
.then(dollarAmount =>console.log("$", dollarAmount))
.catch(err => console.error(err));
This should result in an array like [{amount:123},{amount:234}] as your code seems to expect
The above can also be simplified to
Promise.all(
responseArray
.filter((_, index) => index != 1)
.map(({currency}) =>
currenciesService.getCurrencyLatestInfo(currency)
.then(amount => ({amount})) // do you really want this?
)
)
.then(dollarAmount =>console.log("$", dollarAmount))
.catch(err => console.error(err));
Note: your original code suggests you want the results to be in the form {amount:12345} - which seems odd when you want to console.log("$", ....) ... because the console output would be something like
$ [ { amount: 1 }, { amount: 0.7782 } ]
given two results of course - can't see your responseArray so, am only guessing