Develop Reference

javascript Promise not wait Promise.all

So getAstronautsData make request to API then return array of promises. This promises mast make request to Wiki API and parse response in object. Then exampleAsyncFunc must wait all promises and return one big object with all info about Astronauts.
But if I use Promise.all function ending and console is clear.
function getAstronautsData() {
return new Promise((resolve, reject) => {
getData('http://api.open-notify.org/astros.json', "http", (data) => {
resolve(data) // get Astronauts list from API
})
}).then((astronautsList) => {
return astronautsList.people.map((person => // return array of promises
new Promise(resolve => {
getWikiData(person.name, (data) => { // request on Wiki API
resolve({info: data.extract, img: data.thumbnail.source})
})
})
))
})
}
async function exampleAsyncFunc (){
let promisesList = await getAstronautsData()
// next code just few variant was i try
let data = await Promise.all(promisesList)// it's not working.
console.log(data)
Promise.all(promisesList).then(data => console.log(data)) //it's not working. Function display nothing
promisesList.forEach((promise) => { //it's working but not so elegant
promise.then(data => console.log(data))
})
}
exampleAsyncFunc ()
function getWikiData(searhTerm, callback) {
getData(getUrlString(searhTerm), "https", (data) => {
const regex = new RegExp(searhTerm.replaceAll(" ", ".*"));
for (let page in data.query.pages) {
if (data.query.pages[page].title === searhTerm || regex.test(data.query.pages[page].title)) {
callback(data.query.pages[page])
return
}else{
callback(null)
}
}
})
}

You appear to be using Promise.all correctly, but if any of the Promises in Promise.all rejects, then overall Promise.all promise will reject too and nothing will happen, where in your forEach version it'll simply skip those promises silently and move on to the next entries.
Likewise if any of the promises in the list stays pending: if so then the Promise.all promise will never resolve. This could be because you have a long list of return values and the whole list takes a longer-than-expected time to resolve, or because your getWikiData call encounters an error and you don't pass that out to reject that particular promise in your array.
You can debug this behavior by ensuring that each of your calls to then is followed by .catch(console.error) (or some more robust error handler).

Let me first disclose that I am a big promise partisan and frankly deplore callbacks. The implication here is that I would not have written your getData and getWikiData with callbacks.
I will also point out that I second what #t.niese said in the comments: Because it does not make sense having both let data = await Promise.all(promisesList) and promisesList.forEach((promise) => {.
Anyway, your code is unnecessarily complex and can be simplified like so:
function getAstronautsData(callback) {
getData('http://api.open-notify.org/astros.json', "http", data => {
callback(data.people.map(person =>
new Promise(resolve => {
getWikiData(person.name, data => {
resolve(data);
})
}))
)
})
}
function exampleAsyncFunc (){
getAstronautsData(promises => {
Promise.all(promises)
.then(result => {
//result will contain those resolved promises
console.log(result);
})
});
}
exampleAsyncFunc ()
Notice that I am passing a callback to getAstronautsData and call it from inside that function with the array of promises you ultimately want to resolve. No need for async here either as you can see.

Ok, problem was in API (in API one of astronauts have name "Tom Marshburn" but on Wiki his page have title "Thomas Marshburn") and function getWikiData not return any data on error. So i fixed this problem.
Thanks you all for you help!!!

Is there a way to use async/await with this algolia example code?

This code i am about to put here does not work in the respect that my graphQL server always receives an empty array back... i know this is because the call completes before the data is available.. i don't have much experience with promises so i am wondering how i can get it to work, i played with a bunch of different combinations to attempt to await the response so that i could just pass back an array of id's to the playground... the browsewr.on(end) part works fine and the proper data shows in the console? Anyone care to shed some light for me?
const browser = await lookbookIndex.browseAll();
let hits = [];
let returnArray = [];
browser.on('result', (content) => {
hits = hits.concat(content.hits);
});
browser.on('end', () => {
console.log('Finished!');
console.log('We got %d hits', hits.length);
returnArray = hits ? hits.map(a => a.objectID) : [];
});
browser.on('error', err => err);
return returnArray;

Async syntax automatically wraps everything to Promise instance.
But you also can return Promise instance manually. It accepts callback with resolve & reject arguments. And you can call resolve to resolve the promise:
async function search() {
const browser = await lookbookIndex.browseAll();
return new Promise((resolve, reject) => {
let hits = [];
browser.on('result', (content) => {
hits = hits.concat(content.hits);
});
browser.on('end', () => {
console.log('Finished!');
console.log('We got %d hits', hits.length);
resolve(hits ? hits.map(a => a.objectID) : [])
});
browser.on('error', err => reject(err));
})
}
And you can await the promise:
await new Promise(...)
Similar questions:
Return data from function

Correct way to wait for an event before returning

I'm using a csv-parser npm module to read a csv file, process it, and then create a statistical model based on the data. The issue I'm having is that the other file that uses this isn't waiting for the model to finish before moving on, so it ends up trying to use values/methods that are not yet defined. Based on other posts, this is what I have:
this.read = async function(){
return new Promise((resolve, reject) => {
console.log("in mv read");
fs.createReadStream("./assets/stats-csv.csv")
.pipe(csv({
mapValues: ({ header, index, value }) => this.toNumber(header, value)
}))
.on('data', (data) => this.process(data))
.on('error', err => {
reject(err);
})
.on('end', () => {
this.model();
console.log('finished mv model');
resolve(true);
});
})
}
And then the other file uses the method the following way:
this.train_mv = async function(){
console.log("in train mv wrapper")
const success = await this.mvReg.read();
return success;
//console.log(success);
}
I added the "success" bit just to see if returning and using a value from the promise would help, but it doesn't. The function just moves on and doesn't even go to the "return success" line. Am I missing something about async/await? Shouldn't the train_mv function pause and wait until the promise resolves? I would appreciate any help. Thanks!

Reading files one by one and resolve them

I have multiple(hundreds) of files in a folder i need to read. Currently I am using a method that reads all the files and then returns the content as an array. But i need to change it so that i read a file and i send it to the client and then i read the second one and so one until the end... to make it more efficient. Any suggestions?
Currently code:
const fse = require("fs-extra");
const path = require("path");
return (
fse
.readdir(direc)
.then((filenames: any) => {
return filenames.map((filename: any) => path.join(direc, filename));
})
.then((filepaths: any) => {
return filepaths.map((filepath: any) =>
fse.readFile(filepath).then((filecontents: any) => {
return JSON.parse(filecontents);
})
);
})
// Promise.all consumes an array of promises, and returns a
// new promise that will resolve to an array of concrete "answers"
.then((filecontents: any) => Promise.all(filecontents))
.then((realcontents: any) => {
return realcontents;
})
);

Instead of putting all the file reading promises in an array and then calling Promise.all, chain them together. I'm not very familiar with Typescript syntax, so I'll provide an example code in plain JS and I hope you can convert whatever's needed:
// ...
.then(filepaths => {
let promise = Promise.resolve();
filepaths.forEach(filepath => {
promise = promise
.then(() => fs.readFile(filePath))
.then(handleFile)
.catch(handleError);
});
return promise;
})
// ...
Then you just need to create a handleFile function that takes in the file contents of a single file and does whatever you want with it (parses it, sends data back over a response, etc.). This way each file will be handled in order. You're going to want your handleError function to return a resolved promise, too, so one failed file read doesn't break the entire chain.

Promise All retry

I know that promise.all() fails when even 1 of the promise is failed. I only want to try for failed promises and don't want to run promise.all() again.
Any recommendations on how I can achieve this in minimal way?

Promises are eager construct and model a value obtained asynchronously,
a Promise is produced using some kind of producer, like fetch for instance.
If you retain a reference to this producer then you can replay the nmechanism
that produced the Promise in the first place.
// producer function
function getData (arg) {
const result = new Promise();
return result.then(value => {
return { ok:true, value };
}, error => {
return {
ok: false,
value: error,
// retry is a function which calls the producer with the same arguments
retry: () => getData(arg)
};
})
}
Then if you have something like:
const data = [];
// Promise<{ok: boolean, value: any, retry?: function}>
// No promises will fail in this array
const asyncResults = data.map(getResults);
Promise.all(asyncResults)
.then((results) => {
const successes = results.filter(res => res.ok);
const retrys = results.filter(res => !res.ok).map(res => res.retry()); // retry all failed promises
})
Memory leaks, stack overflow: because I retain a reference to original arguments in order to retry and the algorithm is recursive there could be a memory leak. However the algorithm cannot "stack overflow":
getData calls do not get "deeper" over time (see retry definition)
the asyncrhonicity of the algorithm prevent this behaviour if a promise was never resolved
old data is properly discarded when accessing the results as const resultData = results.filter(res => res.ok).map(res => res.value);
However the algorithm could take a long time to settle if a promise keep on not getting resolved and prevent access to the rest of the values.
In an alternative I suggest you take a look at another async primitive, not yet part of the language (maybe some day) : Observables which are designed for this kind of tasks: lazy, retry-able async operations.

You may use async package and wrap all promise calls with closure with done argument.
Then simply resolve results.
const async = require('async');
const promiseAll = promises => {
return new Promise((resolve) => {
// preparing array of functions which has done method as callback
const parallelCalls = promises.map(promise => {
return done => {
promise
.then(result => done(null, result)
.catch(error => {
console.error(error.message);
done();
});
}
});
// calling array of functions in parallel
async.parallel(
parallelCalls,
(_, results) => resolve(results.filter(Boolean))
);
});
};
router.get('/something', async (req, res) => {
...
const results = await promiseAll(promises);
...
});
Or we can simply do Promise.all without using async package:
router.get('/something', async (req, res) => {
...
const results = (
await Promise.all(
promises.map(promise => {
return new Promise(resolve => {
promise.then(resolve).catch(e => resolve());
});
});
)
).filter(Boolean);
...
});