Asynchronous processing of arrays (Promise.all or for..of) - javascript

I know there is a lot of information on this and I have read several articles, but I am still wandering.
I have an array of HTTP request urls. The order of each element in the array is very important. I want to get a response by fetching these request urls, and put the responses in the same order as the original array.
Below is the code I wrote. I thought it was right to use promise.all. We found promise.all to process arrays in parallel, but ordering is not guaranteed. (Is this correct?)
const getAllImagePaths = async urls => {
let copyPaths = [];
try {
const data = await Promise.all(urls.map(url => fetch(url)));
for (let item of data) {
const { url } = item;
copyPaths.push(url);
}
return copyPaths;
} catch (err) {
const { response } = err;
if (response) alert(MESSAGES.GET_PHOTOS_FAIL);
}
};
So I modified the code by using the for..of statement.
const getAllImagePaths = async urls => {
let copyPaths = [];
try {
for(let url of urls) {
const res = await fetch(url);
const json = await res.json();
const data = json.parse();
const { url } = data;
copyPaths.push(data);
}
return copyPaths;
} catch (err) {
const { response } = err;
if (response) alert(MESSAGES.GET_PHOTOS_FAIL);
}
};
However, when the for of statement is used, the response type of the data is 'cors'.
So to fix this, should I change the data to json and then parse the json again to use the data? I think this is very weird. This is because I only want to contain information called 'url' in the response object in the array.

We found promise.all to process arrays in parallel, but ordering is not guaranteed. (Is this correct?)
No, it isn't correct. The result array you get on fulfillment of the Promise.all promise is in the same order as the input array, regardless of the order in which the input promises settled. Here's an example:
function delayedValue(ms, value) {
return new Promise(resolve => {
setTimeout(() => {
console.log(`Resolving with ${value}`);
resolve(value);
}, ms);
});
}
async function example() {
const results = await Promise.all([
delayedValue(100, 1),
delayedValue(10, 2),
delayedValue(200, 3),
]);
console.log(`results: ${results}`);
}
example();
So you can just use the result directly:
const getAllImagePaths = async urls => {
try {
const data = await Promise.all(urls.map(url => fetch(url)));
return data;
} catch (err) {
const { response } = err;
if (response) alert(MESSAGES.GET_PHOTOS_FAIL);
}
};

Related

Performing an api call on each item in an array and appending the result to a new array

I have an array and I am trying to perform an api call on each item.
Like so -
shoppingData.items.map(item => {
getItemInformation(item.id)
.then(response => response.json())
.then(data => {
JSON.parse(data.text);
});
getItemInformation is my api call -
export async function getItemInformation(id) {
try {
const req = await fetch(`**api url**`);
return await req;
} catch (e) {
console.error(e);
return 'Error';
}
}
However, once I have parsed the data, I would like to append it to a new array. This new array will then be used to render a component later down the page like so -
{newArray?.map((item, index) => (
<ShoppingItem key={index} itemDescription={item.description} itemCatergory={item.catergory} />
))}
Im having issues doing this as I have been trying to do it in a useEffect as ideally I need it to happen when the page renders. Furthermore, I tried having newArray as a state e.g const [newArray, setNewArray] = useState([]) but because I am appending items to an array, setNewArray wouldn't allow me to do this.
use Promise.all
const allItems = await Promise.all(
shoppingData.items.map(
item => getItemInformation(item.id)
.then(response => response.json())
)
);
You could also simplify this a bit by putting all the "asyncy" stuff into your getItemInformation method
const allItems = await Promise.all(
shoppingData.items.map(
item => getItemInformation(item.id)
)
);
and
export async function getItemInformation(id) {
try {
const req = await fetch(`**api url**`);
const json = await req.json();
return json;
} catch (e) {
console.error(e);
return 'Error';
}
}
Live example usiong JSONPlaceholder demo api:
async function getItemInformation(id) {
console.log("getItemInformation",id);
try {
const req = await fetch(`https://jsonplaceholder.typicode.com/todos/${id}`);
const json = await req.json();
return json;
} catch (e) {
console.error(e);
return 'Error';
}
}
(async function testIt() {
const shoppingData = {
items: [{
id: 1
}, {
id: 2
}, {
id: 3
}]
};
const allItems = await Promise.all(
shoppingData.items.map(
item => getItemInformation(item.id)
)
);
console.log(allItems);
})()
You can easily call setNewArray(allItems) in react useEffect using this code (basically where I did consolew.log(allItems) above.
I am not sure how do you append the data to your array while you are calling async function in .map without waiting for the result.
I assume you may be trying something like this:
const parsedData = shoppingData.items.map(item => {
getItemInformation(item.id)
.then(response => response.json())
.then(data => {
JSON.parse(data.text);
});
setNewArray(parsedData);
There are two points to note. Firstly getItemInformation() is called on each item async so the order of parsedItem may not be what your expected. Secondly setNewArray() is called while parsedData's async calls may not finish.
One possible way you can try, if you really want to use .map, is using Promise.all(). Promise.all() resolves when all its promises are resolved:
const getParsedData = async() => {
return Promise.all(shoppingData.items.map(item => getItemInfo.......));
}
const data = await getParsedData();
then append data to your array.
Another simpler solution is using for-loop and await:
for (let i=0;i<shoppingData.items.length;i++){
const parsedItem = await getItemInformation(shoppingData.items[i]).........
newArray.push(parsedItem);
}

Populate an array in the loop using promise

I have developed a code to fetch the image information of an object inside a loop statement. However, when I print the output at the bottom of the loop, it is empty. Can anyone help me with this, please? The getMediaInfo function is an Axios call.
const postsWithImageURLS = [];
res.data.forEach(async (post) => {
const response = await getMediaInfo(post.featured_media);
postsWithImageURLS.push({...post, featured_url: response});
});
console.log(postsWithImageURLS);
Promise.all(res.data.map(async (post) => {
if (post.categories.includes(NEWS_CATEGORY_ID)) {
const response = await getMediaInfo(post.featured_media);
post = {...post, featured_url: response};
return post;
}
})).then(postsWithImageURLS => console.log(postsWithImageURLS));
You should access postsWithImageURLS after all async methods finish.
I don't know exact content of the getMediaInfo function. But if it doesn't return a promise you can't use await before calling it.
Check this out:
const getMediaInfo = (data) => {
return new Promise(async (resolve, reject) => {
try {
let response = await axios.get(data); // data must be a url ofc
resolve(response); // Resolve must get a data you want to get when you call getMediaInfo
} catch (error) {
reject(error);
}
});
}
const postsWithImageURLS = [];
res.data.forEach(async (post) => {
const response = await getMediaInfo(post.featured_media);
postsWithImageURLS.push({...post, featured_url: response});
});
console.log(postsWithImageURLS);

Firestore slow queries are causing the entire logic to crash

I am currently designing a todo app with react and firebase without any node js server code. Everything was fine when inserting data and authenticating, but when I tried to query all the tasks where uid equals a certain value(that means all the tasks that belongs to a particular user), somehow the query gets skipped and returns null. But then, after a second or so, the array got retrieved and printed out.
The code:
function retrieveTasksOfUser(uid) {
// get all tasks where the uid equals the given value
return db.collection("tasks").where("uid", "==", uid).get();
}
function retrieveTasks() {
let res = [];
retrieveTasksOfUser(currentUser["uid"])
.then((snapshot) => {
snapshot.forEach((doc) => {
// include the doc id in each returned object
let buffer = doc.data();
buffer["id"] = doc.id;
res.push(buffer);
});
console.log(res);
return res;
})
.catch((err) => {
console.error("Error retrieving tasks: ", err);
return res;
});
}
let tasks = retrieveTasks();
console.log({ tasks });
EDIT:
Inspired by Frank's answer, I modified my code and got:
async function retrieveTasks() {
let res = [];
const snapshot = await retrieveTasksOfUser(currentUser["uid"]);
snapshot.forEach((doc) => {
let buffer = doc.data();
buffer["id"] = doc.id;
res.push(buffer);
});
return res;
}
let tasks = [];
let promise = retrieveTasks();
promise.then((res) => {
console.log("Tasks are");
tasks = res;
console.log(res);
});
console.log(tasks);
But the result turns out to be an empty array
Thanks a bunch in advance.
You're not returning anything from the top-level code in retrieveTasks, so that means that tasks will always be undefined.
The simplest fix is to return the result of the query, which then means that your return res will be bubbled up:
return retrieveTasksOfUser(currentUser["uid"])
...
But this means you're returning a promise, as the data is loaded asynchronously. So in the calling code, you have to wait for that promise to resolve with then:
retrieveTasks().then((tasks) => {
console.log({ tasks });
})
You can make all of this a lot more readable (although it'll function exactly the same) by using the async and await keywords.
With those, the code would become:
async function retrieveTasks() {
let res = [];
const snapshot = await retrieveTasksOfUser(currentUser["uid"]);
snapshot.forEach((doc) => {
let buffer = doc.data();
buffer["id"] = doc.id;
res.push(buffer);
});
return res;
}
let tasks = await retrieveTasks();
console.log({ tasks });
Or with a bit more modern JavaScript magic, we can make it even shorter:
async function retrieveTasks() {
const snapshot = await retrieveTasksOfUser(currentUser["uid"]);
return snapshot.docs.map((doc) => { ...doc.data, id: doc.id });
}
let tasks = await retrieveTasks();
console.log({ tasks });

Why is my apolloFetch call returning an empty query when called from within a promise.all?

I'm trying to use apolloFetch inside a Promise.all in my Node.js microservice but keep getting an error that the query is empty. The reason for using apolloFetch is to call another micro service and pass it an array of queries. Can someone give me some direction? My code is as follows:
const uri = "dsc.xxx.yyyy.com/abc/def/graphql";
const apolloFetch = CreateApolloFetch({uri});
const QryAllBooks = {
type: new GraphQLList(BookType),
args: {},
resolve() {
return new Promise((resolve, reject) => {
let sql = singleLineString`
select distinct t.bookid,t.bookname,t.country
from books_tbl t
where t.ship_status = 'Not Shipped'
`;
pool.query(sql, (err, results) => {
if (err) {
reject(err);
}
resolve(results);
const str = JSON.stringify(results);
const json = JSON.parse(str);
const promises = [];
for (let p = 0; p < results.length; p++) {
const book_id = json[p].bookid;
const query = `mutation updateShipping
{updateShipping
(id: ${book_id}, input:{
status: "Shipped"
})
{ bookid
bookname }}`;
promises.push(query);
}
//Below is the Promise.all function with the
//apolloFetch that calls another graphql endpoint
//an array of queries
Promise.all(promises.map(p => apolloFetch({p}))).then((result) => {
//this is the problem code^^^^^^^^^^^^^^^^^^^^^
resolve();
console.log("success!");
}).catch((e) => {
FunctionLogError(29, "Error", e);
});
});
});
}
};
module.exports = {
QryAllBooks,
BookType
};
It looks like apolloFetch requires query - you are passing p
change
Promise.all( promises.map(p=>apolloFetch({p})) )
to
Promise.all( promises.map(query=>apolloFetch({query})) )
You also call resolve twice
To resolve all errors or success
const final_results = []
Promise.all(promises.map(query => apolloFetch({
query,
}))).then((result) => {
final_results.push(result)
}).catch((e) => {
final_results.push(e)
}).then(() => {
resolve(final_results)
});
You immediately resolve or rejects once the pool.query() callback starts:
if(err){ reject(err);}resolve(results);
So unless the query fails, you never resolve with the results from the apolloFetch calls, since the promise is already resolved with the pool.query() results. I guess you're missing an else block:
if( err ) {
reject();
}
else {
const promises = ...
}
PS: you can try using node.js' util.promisify() to turn pool.query() into a promise as well so you can just write something resembling: query(...).then(results=>results.map(apolloFetch) instead of ahving to mix callbacks and promises.

How can I store the data I get from a Promise back into an object with the same key it came from?

A bit of an ambiguous title, but I'll explain...
I am making a fetch call to 4 different URLs from The Movie Database.
Once these fetch calls retrieve the data, it will then setState and update my initial state. However, I don't want my page to load until all of the data is retrieved, so I am using Promise.all (or attempting to).
My code so far...
state = {
movies: {
trending: {},
topRated: {},
nowPlaying: {},
upcoming: {},
},
};
const allMovieURLs = {
trending: `https://api.themoviedb.org/3/trending/all/day?api_key=${API_KEY}`,
topRated: `https://api.themoviedb.org/3/movie/top_rated?api_key=${API_KEY}&language=en-US&page=1`,
nowPlaying: `https://api.themoviedb.org/3/movie/now_playing?api_key=${API_KEY}&language=en-US&page=1`,
upcoming: `https://api.themoviedb.org/3/movie/upcoming?api_key=${API_KEY}&language=en-US&page=1`
};
//initialize an empty array to Promise.all on
const promiseArr = [];
//loop through movie URLs, call fetch + json()
for (const movies in allMovieURLs) {
//create an object that directly describes the data you get back from the url with a key
const obj = {
[movies]: fetch(allMovieURLs[movies]).then(res => res.json())
};
//push that object into an array so you can use Promise.all
promiseArr.push(obj);
Now what I have here is an array (promiseArr) of objects that have the correct key with the correct Promise stored inside of them.
My next plan was going to be to call Promise.all on them, but I need an array of promises. After I get the actual data back from the URL calls, I wanted to store them back in the object, to the correct corresponding key?
I've been stuck at this part for a couple hours now...any tips would be appreciated!
You can use async/await and Object.entries() to convert a JavaScript plain object to an array of arrays of key, value pairs
(async() => {
for (const [movie, url] of Object.entries(allMovieURLs)) {
try {
allMovieURLs[movie] = await fetch(url).then(res => res.json())
.catch(e => {throw e})
} catch(e) {
console.error(e)
}
}
})()
Don't call then when adding to promise array. Instead call it in promise all
const trending = fetch(trending);
...
Promise.all([trending, topRated, nowplaying, upcoming]).then(([trending, topRated, nowplaying, upcoming]) => {
movies.trending = trending.json();
....
});
You could just also store the promises themselves in another array.
function fetch() {
var time = Math.random() * 1E3;
return new Promise(function (res, rej) {
setTimeout(function () {
res(time);
}, time);
});
}
const API_KEY = "";
const allMovieURLs = {
trending: `https://api.themoviedb.org/3/trending/all/day?api_key=${API_KEY}`,
topRated: `https://api.themoviedb.org/3/movie/top_rated?api_key=${API_KEY}&language=en-US&page=1`,
nowPlaying: `https://api.themoviedb.org/3/movie/now_playing?api_key=${API_KEY}&language=en-US&page=1`,
upcoming: `https://api.themoviedb.org/3/movie/upcoming?api_key=${API_KEY}&language=en-US&page=1`
};
const promiseArr = [];
const promises = [];
for (const movies in allMovieURLs) {
const obj = { [movies]: undefined };
promises.push(fetch(allMovieURLs[movies]).then(res => obj[movies] = res));
promiseArr.push(obj);
}
Promise.all(promises).then(function () {
console.log(promiseArr);
});
If that is not feasible, you can do something like: Promise.all(promiseArr.map(obj => Object.values(obj)[0]))

Categories

Resources