Develop Reference

Chaining unknown number promises in a recursive function

I'm trying to find the best way to go about this service call where I retain all the data in a single object. The call returns an object that has a property of next_page_url. If there is a next_page_url the function should keep chaining. Since I don't know what the url is until the next call resolves I need to call these in order and resolve them in order. I'm also collecting data from each call. I haven't been able to figure out what the structure should be
what I have so far
getDataFromAllPages = (url) => {
waniKaniAxios.get(url).then(object => {
if(object.data.pages.next_url){
return this.getDataFromAllPages(object.data.pages.next_url.replace(waniKaniAxios.defaults.baseURL, ''));
}
});
}
getWanikaniData = () => {
this.getDataFromAllPages('/subjects?types=vocabulary').then(result => {
console.log(result);
});
}

Abstract away the wanikaniaxios.get in another function to make recursion clearer.
Here's my badly formatted code (don't know how SF editor works) , feel to ask any questions if you have any. Happy coding.
getWanikaniData = () => {
this.getDataFromAllPages("/subjects?types=vocabulary")
.then((result) => {
console.log(result);
})
.catch((err) => {
console.log(err); // always put a .catch when you're using prmomises.
});
};
getDataFromAllPages = async (url) => {
// using async await;
try {
let arr = []; // i am assuming you'll improve upon what data structure you might want to return. Linked list seems best to me.
const object = await waniKaniAxios.get(url);
if (object.data.pages.next_url) {
const laterData = await this.getDataFromAllPages(
object.data.pages.next_url.replace(waniKaniAxios.defaults.baseURL, "")
);
arr = [...arr, ...laterData];
} else {
arr = [...arr, object];
}
Promise.resolve(arr);
} catch (err) {
Promise.reject(new Error(`Oops new wanikani error, ${err}`));
}
};
FINAL UPDATE
Using part of the answer below I managed to get it working. Had to partially give up on the recursion aspect because I didn't how to make the promise resolve into data
Here's the final solution that I came up with
getDataFromAllPages = async (url) => {
let results = {};
try {
//getting intial data
const initialData = await waniKaniAxios.get(url);
//using the intial data and mapping out the levels then saving it into results object
results = this.mapOutLevels(initialData.data, results);
//get the next page url
let nextPageUrl = initialData.data.pages.next_url;
//while there is a next page url keep calling the service and adding it to the results object
while (nextPageUrl) {
const laterData = await waniKaniAxios.get(nextPageUrl);
nextPageUrl = laterData.data.pages.next_url;
results = this.mapOutLevels(laterData.data, results);
}
} catch (err) {
Promise.reject(new Error(`Opps new wanikani error, ${err}`));
}
return Promise.resolve(results);
};
getWanikaniData = () => {
this.getDataFromAllPages("/subjects?types=vocabulary")
.then((result) => {
console.log(result);
})
.catch((err) => {
console.log(err);
});
};

getting data after async functions executed in an old ES version

I have the following code with two functions that get executed asynchronously
products = [];
categories = [];
getProduct = function() {
productService.list().then(function(result) {
products = ["product1", "product2"]
});
}
getcategories = function() {
categoryService.list().then(function(result) {
categories = ["category1", "category2"]
});
}
i want another function that accept the two parameters, products and categories, something like that:
all = function(products, categories) {
console.log(products, categories);
}
but of course because of the async function, i'll get undefined. is there anyway to get the data after the previous functions get executed. m using old javascript so i can't use asynch await .

You can use promises chaining, call one by one,
see:
products = [];
categories = [];
productService.list()
.then(function(result) {
products = ["product1", "product2"]; // products = result;
return categoryService.list();
}).then(function(result) {
categories = ["category1", "category2"]; // categories = result;
all(products, categories);
}).catch(function(ex) {
// Error handele
}) ;
see exmple here
one more option is to use Promise.all
it can fit your use case as you don't need to read the functions one by one. in this option you should do something like:
Promise.all([productService.list(), categoryService.list()]).then(function(values) {
all(value[0],value[1]);
});

ES6 Solution with Promise.all:
EDIT For shortest solution:
Promise.all([productService.list(), categoryService.list()]).then(values => all(...values));
Previous Solution:
getProduct = function() {
return new Promise(res => {
productService.list().then(function(result) {
res(["product1", "product2"]);
});
});
}
getcategories = function() {
return new Promise(res => {
categoryService.list().then(function(result) {
res(["category1", "category2"]);
});
});
}
Promise.all([getProduct(), getcategories()]).then(aResolveValues => {
aResolveValues[0] //Has the values of the first resolve promise (products)
aResolveValues[1] //Has the values of the second resolve promise (categories)
});

You could chain if your situation suits it, or depending on your version of js (you don't mention what version you're targeting) you can either use Promise.all or fallback to a basic callback, e.g:
var products, categories,
all = function () {
if (products && categories) {
console.log(products, categories);
}
},
getProducts = function (oncomplete) {
return productService.list().then(function(result) {
products = ["product1", "product2"];
oncomplete();
});
},
getCategories = function (oncomplete) {
return categoryService.list().then(function(result) {
categories = ["category1", "category2"];
oncomplete();
});
};
getProducts(all) && getCategories(all);

properly using async and await

The function below calls several asynchronous functions in a for loop. It's parsing different CSV files to build a single JavaScript object. I'd like to return the object after the for loop is done. Its returning the empty object right away while it does the asynchronous tasks. Makes sense, however I have tried various Promise / async /await combinations hopes of running something once the for loop has completed. I am clearly not understanding what is going on. Is there a better pattern to follow for something like this or am I thinking about it incorrectly?
async function createFormConfig(files: string[]): Promise<object>
return new Promise(resolve => {
const retConfig: any = {};
for (const file of files) {
file.match(matchFilesForFormConfigMap.get('FIELD')) ?
parseCsv(file).then(parsedData => {
retConfig.fields = parsedData.data;
})
: file.match(matchFilesForFormConfigMap.get('FORM'))
? parseCsv(file).then(parsedData => retConfig.formProperties = parsedData.data[0])
: file.match(matchFilesForFormConfigMap.get('PDF'))
? parseCsv(file).then(parsedData => retConfig.jsPdfProperties = parsedData.data[0])
: file.match(matchFilesForFormConfigMap.get('META'))
? parseCsv(file).then(parsedData => {
retConfig.name = parsedData.data[0].name;
retConfig.imgType = parsedData.data[0].imgType;
// console.log(retConfig); <- THIS CONSOLE WILL OUTPUT RETCONFIG LOOKING LIKE I WANT IT
})
: file.match(matchFilesForFormConfigMap.get('PAGES'))
? parseCsv(file).then(parsedData => retConfig.pages = parsedData.data)
: console.log('there is an extra file: ' + file);
}
resolve(retConfig); // <- THIS RETURNS: {}
});
This is the code I'm using to call the function in hopes of getting my 'retConfig' filled with the CSV data.
getFilesFromDirectory(`${clOptions.directory}/**/*.csv`)
.then(async (files) => {
const config = await createFormConfig(files);
console.log(config);
})
.catch(err => console.error(err));
};

First, an async function returns a Promise, so you dont have to return one explicitely.Here is how you can simplify your code:
async function createFormConfig(files: string[]): Promise<object> {
// return new Promise(resolve => { <-- remove
const retConfig: any = {};
// ...
// The value returned by an async function is the one you get
// in the callback passed to the function `.then`
return retConfig;
// }); <-- remove
}
Then, your function createFormConfig returns the config before it has finished to compute it. Here is how you can have it computed before returning it:
async function createFormConfig(files: string[]): Promise<object> {
const retConfig: any = {};
// Return a Promise for each file that have to be parsed
const parsingCsv = files.map(async file => {
if (file.match(matchFilesForFormConfigMap.get('FIELD'))) {
const { data } = await parseCsv(file);
retConfig.fields = data;
} else if (file.match(matchFilesForFormConfigMap.get('FORM'))) {
const { data } = await parseCsv(file);
retConfig.formProperties = data[0];
} else if (file.match(matchFilesForFormConfigMap.get('PDF'))) {
const { data } = await parseCsv(file);
retConfig.jsPdfProperties = data[0];
} else if (file.match(matchFilesForFormConfigMap.get('META'))) {
const { data } = await parseCsv(file);
retConfig.name = data[0].name;
retConfig.imgType = data[0].imgType;
} else if (file.match(matchFilesForFormConfigMap.get('PAGES'))) {
const { data } = await parseCsv(file);
retConfig.pages = data;
} else {
console.log('there is an extra file: ' + file);
}
});
// Wait for the Promises to resolve
await Promise.all(parsingCsv)
return retConfig;
}

async functions already return promises, you don't need to wrap the code in a new one. Just return a value from the function and the caller will receive a promise that resolves to the returned value.
Also, you have made an async function, but you're not actually using await anywhere. So the for loop runs through the whole loop before any of your promises resolve. This is why none of the data is making it into your object.
It will really simplify your code to only use await and get rid of the then() calls. For example you can do this:
async function createFormConfig(files: string[]): Promise<object> {
const retConfig: any = {};
for (const file of files) {
if (file.match(matchFilesForFormConfigMap.get('FIELD')){
// no need for the then here
let parsedData = await parseCsv(file)
retConfig.field = parsedData.data
}
// ...etc
At the end you can just return the value:
return retConfig

Returning value from multiple promises within Meteor.method

After a bunch of looking into Futures, Promises, wrapAsync, I still have no idea how to fix this issue
I have this method, which takes an array of images, sends it to Google Cloud Vision for logo detection, and then pushes all detected images with logos into an array, where I try to return in my method.
Meteor.methods({
getLogos(images){
var logosArray = [];
images.forEach((image, index) => {
client
.logoDetection(image)
.then(results => {
const logos = results[0].logoAnnotations;
if(logos != ''){
logos.forEach(logo => logosArray.push(logo.description));
}
})
});
return logosArray;
},
});
However, when the method is called from the client:
Meteor.call('getLogos', images, function(error, response) {
console.log(response);
});
the empty array is always returned, and understandably so as the method returned logosArray before Google finished processing all of them and returning the results.
How to handle such a case?

With Meteor methods you can actually simply "return a Promise", and the Server method will internally wait for the Promise to resolve before sending the result to the Client:
Meteor.methods({
getLogos(images) {
return client
.logoDetection(images[0]) // Example with only 1 external async call
.then(results => {
const logos = results[0].logoAnnotations;
if (logos != '') {
return logos.map(logo => logo.description);
}
}); // `then` returns a Promise that resolves with the return value
// of its success callback
}
});
You can also convert the Promise syntax to async / await. By doing so, you no longer explicitly return a Promise (but under the hood it is still the case), but "wait" for the Promise to resolve within your method code.
Meteor.methods({
async getLogos(images) {
const results = await client.logoDetection(images[0]);
const logos = results[0].logoAnnotations;
if (logos != '') {
return logos.map(logo => logo.description);
}
}
});
Once you understand this concept, then you can step into handling several async operations and return the result once all of them have completed. For that, you can simply use Promise.all:
Meteor.methods({
getLogos(images) {
var promises = [];
images.forEach(image => {
// Accumulate the Promises in an array.
promises.push(client.logoDetection(image).then(results => {
const logos = results[0].logoAnnotations;
return (logos != '') ? logos.map(logo => logo.description) : [];
}));
});
return Promise.all(promises)
// If you want to merge all the resulting arrays...
.then(resultPerImage => resultPerImage.reduce((accumulator, imageLogosDescriptions) => {
return accumulator.concat(imageLogosDescriptions);
}, [])); // Initial accumulator value.
}
});
or with async/await:
Meteor.methods({
async getLogos(images) {
var promises = [];
images.forEach(image => {
// DO NOT await here for each individual Promise, or you will chain
// your execution instead of executing them in parallel
promises.push(client.logoDetection(image).then(results => {
const logos = results[0].logoAnnotations;
return (logos != '') ? logos.map(logo => logo.description) : [];
}));
});
// Now we can await for the Promise.all.
const resultPerImage = await Promise.all(promises);
return resultPerImage.reduce((accumulator, imageLogosDescriptions) => {
return accumulator.concat(imageLogosDescriptions);
}, []);
}
});

Problems with optional promise that has nested promises

I have a server GET request where I am trying to create a promise from a function but it's not going too well. I need this promise to be optional, if that's possible, so that if the result is empty it should be ignored (I've marked where it shouldn't carry on with the function with the words IGNORE FUNCTION).
The idea is that it loops through the docs inside the snapshot, and adds them to an array calls jamDocs. Then if that array isn't empty it should loop through each jam and call an async function for each of them. This then sets the jam's hostName. When this is completed, the original promise should return the jamDocs.
If the jamDocs are empty, the whole promise should be ignored. Finally, after other promises are completed (there is one called profilePromise that returns a profile object), the jams should be assigned to the profile object, and the object sent back to the client with a 200 status code.
My code:
var jamsQuery = firebase.db.collection('users')
.where("hostId", "==", id)
.orderBy("createdAt")
.limit(3)
var jamsPromise = jamsQuery.get().then(snapshot => {
var jamDocs = []
snapshot.forEach(doc => {
var jam = doc.data()
jam.id = doc.id
jamDocs.push(jam)
})
if (!snapshot.length) { // Error: Each then() should return a value or throw
return // IGNORE FUNCTION
} else {
var jamNamePromises = jamDocs.map(function(jam) {
var jamRef = firebase.db.collection('users').doc(jam.hostId)
return jamRef.get().then(doc => {
if (doc.exists) {
jam.hostName = doc.data().firstName
return jam
} else {
throw new Error("yeah")
}
})
})
Promise.all(jamNamePromises)
.then(function() {
return jamDocs
})
.catch(...)
}
})
// Later on
Promise.all(profilePromise, jamsPromise)
.then(objectArray => {
var profile = objectArray[0]
myProfile.jams = jamDocs
return res.status(200).send(myProfile);
})
.catch(
res.status(500).send("Could not retrieve profile.")
)
I get errors like: Each then() should return a value or throw and Avoid nesting promises. How do I fix this? Thanks!

Your then on this line doesn't return anything and also has nested promises in it.
var jamsPromise = jamsQuery.get().then(snapshot => {
You should refactor it, to move the nested promises outside of this then:
var jamsPromise = jamsQuery.get()
.then(snapshot => {
var jamDocs = []
snapshot.forEach(doc => {
var jam = doc.data()
jam.id = doc.id
jamDocs.push(jam)
})
if (!snapshot.length) {
return // IGNORE FUNCTION
} else {
var jamNamePromises = getJamDocsPromises();
return Promise.all(jamNamePromises);
})
.catch(...)
});
function getJamDocsPromises(jamDocs) {
return jamDocs.map(function(jam) {
var jamRef = firebase.db.collection('users').doc(jam.hostId)
return jamRef.get().then(doc => {
if (doc.exists) {
jam.hostName = doc.data().firstName
return jam
} else {
throw new Error("yeah")
}
});
}

If the result is empty it should be ignored (shouldn't carry on with the function). If that array isn't empty it should loop through each jam
Don't make it more complicated than it needs to be. Looping over an empty collection does nothing anyway, so you don't need a special case for that. Just continue normally with the empty array.
var jamsPromise = jamsQuery.get().then(snapshot => {
var jamDocs = []
snapshot.forEach(doc => {
jamDocs.push(Object.assign(doc.data(), {id: doc.id}))
})
return Promise.all(jamDocs.map(function(jam) {
return firebase.db.collection('users').doc(jam.hostId).get().then(doc => {
if (doc.exists) {
jam.hostName = doc.data().firstName
return jam
} else {
throw new Error("yeah")
}
})
}))
})
I get errors like: Each then() should return a value or throw
You had forgotten a return in front of the Promise.all.
and Avoid nesting promises. How do I fix this?
There's nothing wrong with nesting promises, but in your case the .then(() => jamDocs) was not necessary since the Promise.all already resolves with an array of all the jams that the inner promises fulfill with.