I am using fetch to retrieve a tree, one layer at a time. How can I trigger an action after all branches have been fetched? The tree has multiple branches and I don't know the depth in advance.
Pseudo-code:
until Children is empty fetchChildren()
after all children are retrieved doSomething()
I found some answers, but they only deal with chained then(), not trees.
[edit] Although not mandatory, I am hoping that the solution can run in IE11. I am already using promise and fetch polyfills.
Just first wait for the layer elements to be fetched, then recursively proceed with the childrens and await that:
async function fetchChildren(layer) {
await Promise.all(layer.map(fetch));
await Promise.all(layer.map(el => fetchChildren(el.children)));
}
(fetch(el) has to be a promising function)
Here's a way to do it with a recursive async function waiting for the fetching of an element's children to be done, before returning a promise.
That way, we can call doSomething when everything is done.
const tree = {
url: 'http://www.json-generator.com/api/json/get/cevhxOsZnS',
children: [{
url: 'http://www.json-generator.com/api/json/get/cguaPsRxAi'
}, {
url: 'http://www.json-generator.com/api/json/get/cguaPsRxAi',
children: [{
url: 'http://www.json-generator.com/api/json/get/cfDZdmxnDm'
}]
}]
};
function doSomething() {
console.log("doing something");
}
async function fetchIt(element) {
if (element.children) {
await Promise.all(element.children.map(fetchIt));
}
return new Promise((resolve) => {
console.log("fetching " + element.url);
fetch(element.url).then(res => {
console.log("done");
resolve();
});
});
}
fetchIt(tree).then(doSomething);
Related
In one of my API endpoints I fetch a json resource (1) from the web and edit it to fit my needs. In the "lowest" or "deepest" part of the tree I'm trying to fetch another resource and add it to the final json object. I'm relatively new to async/await but am trying to move away from the "old" Promises since I see the advantage (or the gain) of using async/await.
The object from (1) looks like;
const json = {
date,
time,
trips: [{
name,
legs: [{
id
},
{
id
}
]
}]
};
Here's how I "reformat" and change the json object;
{
date,
time,
trips: json.trips.map(trip => formatTrip(trip))
};
function formatTrip(trip) {
return {
name,
legs: trip.legs.map(leg => formatLeg(leg))
};
};
async function formatLeg(leg) {
const data = await fetch();
return {
id,
data
};
};
The problem with this is that after I've "reformatted/edited" the original json to look how I want it (and ran through all format... functions) the legs objects are empty {}.
I figured this might be due to the async/await promises not finishing. I've also read that if a child-function uses async/await all the higher functions has to use async/await as well.
Why? How can I rewrite my code to work and look good? Thanks!
EDIT:
I updated my code according to Randy's answer. getLegStops(leg) is still undefined/empty.
function formatLeg(leg) {
return {
other,
stops: getLegStops(leg)
};
};
function getLegStops(leg) {
Promise.all(getLegStopRequests(leg)).then(([r1, r2]) => {
/* do stuff here */
return [ /* with data */ ];
});
};
function getLegStopRequests(leg) {
return [ url1, url2 ].map(async url => await axios.request({ url }));
};
Two things lead you to want to nest these Promises:
The old way of thinking about callbacks and then Promises
Believing the software process must match the data structure
It appears you only need to deal with the Promises once if I understand correctly.
Like this:
async function getLegs(){
return trip.legs.map(async leg => await fetch(...)); // produces an array of Promises
}
const legs = Promise.all(getLegs());
function formatLegs(legs) {
// do something with array of legs
};
function formatTrip(){
//format final output
}
EDIT: per your comment below, this snippet represents what I've demonstrated and what your goal should look like. Please review your code carefully.
const arr = [1, 2, 3, ];
const arrPromises = arr.map(async v => await new Promise((res) => res(v)));
const finalPromise = Promise.all(arrPromises);
console.log(finalPromise.then(console.log));
I am reading a JSON object and looping through each item. I am first checking to see if the item already exists in the database and if so I want to log a message. If it doesn't already exist I want to add it.
This is working correctly however, I would like to add a callback or finish the process with process.exit();
Because the mongoose calls are asynchronous I can't put it at the end of the for loop because they haven't finished.
Whats the best way I should handle this?
function storeBeer(data) {
data.forEach((beer) => {
let beerModel = new Beer({
beer_id: beer.id,
name: beer.name,
image_url: beer.image_url
});
Beer.findOne({
'name': beer.name
}).then(function (result) {
if (result) {
console.log(`Duplicate ${result.name}`)
} else {
beerModel.save(function (err, result) {
console.log(`Saved: ${result.name}`)
});
}
});
});
}
Is there anything I should read up on to help solve this?
One means of managing asynchronous resources is through Promise objects, which are first-class objects in Javascript. This means that they can be organized in Arrays and operated on like any other object. So, assuming you want to store these beers in parallel like the question implies, you can do something like the following:
function storeBeer(data) {
// This creates an array of Promise objects, which can be
// executed in parallel.
const promises = data.map((beer) => {
let beerModel = new Beer({
beer_id: beer.id,
name: beer.name,
image_url: beer.image_url
});
return Beer.findOne({
'name': beer.name
}).then(function (result) {
if (result) {
console.log(`Duplicate ${result.name}`)
} else {
beerModel.save(function (err, result) {
console.log(`Saved: ${result.name}`)
});
}
});
);
});
return Promise.all(promises);
}
Don't forget that the storeBeer function is now asynchronous, and will need to be utilized either through a Promise chain or through async/await.
For example, to add process exit afterwards:
async function main() {
const beers = [ ... ];
await storeBeer(beer);
process.exit(0);
}
You can also modify the above example to invoke the storeBeer function within a try / catch block to exit with a different error code based on any thrown errors.
I am having difficulty figuring out what is happening (and not happening) in my action creator.
I need to make a call to one API endpoint, get the ids and names of all the items returned, then for each of those ids, make another call. I want to store the return of the last call and the ids/names from the first call in an object and dispatch it.
{
category: name //name of category
subcategory: [] //array of categories in the category above.
}
Right now, my reducer does end up having what I want, but when I attempt to log that particular prop in the component it is empty. (below I am using OpenSocialAPI or osapi. This is just a basic wrapper for an ajax request. Allows for me to not have to authenticate as it sees I am already authenticated.)
export function fetchCategories(id){
let categories = []
return function(dispatch){
dispatch(requestCategories(id))
return osapi.core.get({
v: "v3",
href: "/places/" + id + "/places"
}).execute(function(response) {
response.content.list.forEach(function(category) {
osapi.core.get({
v: "v3",
href: "/places/" + category.id+ "/categories"
}).execute(function(response) {
categories.push({
category: category.name,
subcategories: response.content.list.map(category => category.name)
})
})
})
console.log("Category response: ", categories)
dispatch(receiveCategories(id, categories))
})
}
}
export function receiveCategories(id,array){
return {
type: RECEIVE_CATEGORIES,
id,
categories: array,
recievedAt: new Date(Date.now()),
isFetching: false
}
}
And in my app I am dispatching the action creator in componentDidMount
componentDidMount() {
const { dispatch } = this.props
dispatch(fetchCategoriesIfNeeded(id))
}
Right now when I console log in my Category component and in the execute above, it is empty. But looking at my state in my logger, when recieveCategories is completed, I have the array of objects I want
[{category:...,
subcategories:[...]
},
{category:...,
subcategories:[...]
}]
I suspect this is because of something asynchronous but I'm unsure how to proceed.
I attempted to create my own wrapper for the call that was promise based, but I had similar issues, probably more so because I'm not sure if resolve(response) is what I want.
function httpService(path){
return new Promise(function(resolve, reject){
osapi.core.get({
v: 'v3',
href: path
}).execute(function(response, error){
if(error){
return reject(new Error("Error: ", error))
}
resolve(response)
})
})
}
export function fetchCategories(spaceId) {
let categories = []
return function(dispatch) {
dispatch(requestCategories(id))
return httpService("/places/" + id + "/places")
.then(function(response) {
response.content.list.forEach(function(category) {
fetchSubCategories("/places/" + category.id + "/categories")
.then(function(response) {
categories.push({
category: category.name,
subcategories: response
})
})
})
console.log("CATEGORIES: ", categories)
dispatch(receiveCategories(id, categories))
})
}
}
function fetchSubCategories(url){
return httpService(url)
.then(function(response){
return response.content.list.map(category => category.name)
})
}
Can you look at this and give guidance? Also, is me dispatching an array that I built based on the API responses a valid way of doing things or is there someway better? Thank you
I was only able to find 1 other question with similar use case but they are using bluebird or something similar. I'd really like to keep this without anything extra besides Redux.
It looks like you just need to dispatch your categories inside your .execute() callback, not outside of it. You're doing osapi.core.get().execute((response) => but then outside of that execute callback, you dispatch receiveCategories, which will execute long before your Promise resolves, and dispatch the empty array you initialize.
You also need to use Promise.all to get the response of all of your nested GET requests.
There's also no reason to keep a mutating array around.
I guess osapi.core.get is some kind of promise based fetch library? And .execute is called when the get succeeds?
If so, then what you're missing is that you're not waiting for all asynchronous calls to finish.
I'm going to show a solution based on generic fetch and native Promises so you can understand the solution and adopt it based on your specific libraries.
const promises = [];
response.content.list.forEach(function(category) {
const promise = fetch("/places/" + category.id+ "/categories");
promises.push(promise);
})
Promise.all(promises).then(responses => {
categories = responses.map(response => ({/* do your object mapping here */}))
// now you can dispatch with all received categories
dispatch(receiveCategories(id, categories))
});
Also, you're using the same variable in your nested functions - while this may work and the computers may understand it, it makes it super hard for any human to figure out which response belongs to which scope. So you may want to take a second look at your variable names as well.
Are there any differences between the two functions?
I recently found a problem while using Promise.all within a controller.
The view would render before the Promise.all completion and I'd get empty variables inside the view.
So if I use one m.request to hit a single api then the view waits for it to complete. If I use many m.request wrapped in Promise.all then it does not wait! Am I doing something wrong?
Is this the correct behaviour? Is m.sync would behave somehow different, it seems to have the same signature as Promise.all?
Thanks.
EDIT
The code with relevant bits
// Does two requests and wraps them in Promise.all
Table.show = (id, load) => {
var header = m.request({
method: "GET",
url: cfg.apiurl(`/tables/${id}`),
});
var body = m.request({
method: "POST",
url: cfg.apiurl(`/tables/${id}`),
data: {
data: load
}
});
return Promise.all([header, body]);
};
// The component
var ShowTable = {
controller, view
};
// Controller function
function controller() {
this.header = m.prop({});
this.records = m.prop([]);
this.pages = m.prop(0);
var load = {
filter: {},
paging: {
number: 1,
size: 10
}
};
var showTable = () => {
Table.show(m.route.param("id"), load).then((res) => {
this.header(res[0].data);
this.records(res[1].data);
this.pages(res[1].meta.pages);
}, (res) => {
popa();
});
};
showTable();
}
// View function
function view(vm) {
return m("div", [
m("p", vm.header()),
m("p", vm.records()),
m("p", vm.pages()),
]);
}
EDIT2
Yes, m.sync works. But Promise.all does not.
A quick look at the documentation landed me to this page. Redraw is done only for mithril's built-in functions, Promise.all is a native javascript function so no auto-redraw. You should either use m.sync or m.startComputation/m.endComputation in your showTable function. You can even return directly [header, body] without Promise.all/m.sync, they are both promises and mithril props so you can use/assign them to your viewmodel and when their value is populated, it should trigger a redraw in the view.
Anyway, if it still doesn't work, don't waste your time, just put m.redraw() in the then of Promise.all.
I got some problems with Sequelize and promise (mainly with promise).
Example :
I want all my Events and the master of this events. I've done something like that :
models.events.findAll().then(function(event) {
events.forEach(function(event){
events.dataValues.master = event.getUsers({
where: ['UserRelationEvent.relation = "master"'],
joinTableAttributes: []
}).then(function(user){
return user[0].dataValues;
});
});
return next(events);
}).catch(function(err) {next(err)});
But there I got an Sequelize Object.. I checked my content of user[0].dataValues It's exactly what I want.. so I think I miss something and misunderstood something with promises :/
I try many things but mainly my question is : how can I retrieve my string from my console.log
event.getUsers returns a promise - so you are assigning events.dataValues.master (which is the array, not the particular event btw) the promise, not the value.
Also, you are returning next, before getUsers is done, because they happen async
Something like this should work
models.events.findAll().then(function(events) {
sequelize.Promise.each(events, function(event){
return event.getUsers({
through: {
where: { relation: 'master' }
},
joinTableAttributes: []
}).then(function(user){
event.dataValues.master = user[0];
});
}).then(function (events) {
return next(events);
});
}).catch(function(err) {next(err);});
When you return a promise to the callback in promise.each, the then (which calls next) is not invoked, before all the returned promises are done (i.e. before all getUsers calls are done.
I've also changed the where clause to use an object instead of string :)
But wait! We can do better!
models.events.findAll({
include: [
{
model: User,
through: {
where: {
relation: 'master'
}
}
}
]
}).then(function(events) {
return next(events);
}).catch(function(err) {next(err);});
This left joins the users table where the relation is master. You may want to do a map in the then, because users will be placed under .users, not .master