Develop Reference

Promise.all in loop, one by one. Await until function fully completed [duplicate]

This question already has answers here:
Resolve promises one after another (i.e. in sequence)?
(36 answers)
Closed 1 year ago.
I'm trying to resolve promises one by one, but i don't understand how to. I want to wait till video message load fully then move to another promise, otherwise it freeze browser. In my code example it don't wait until video hit loadedmetadata event, and go to next promise. And after all promises executed, load's all videos. How it looks: https://prnt.sc/10mg4oc
//All messages loop
for (var i = 0; i < data[0].length; i++) {
let urls = [];
$.each(file_id.split(','), function( index, value ) {
file_id = value;
urls.push("//localhost/storage/"+data[0][i].user_id+"/"+data[0][i].file_cluster_id+"/"+value)
});
//Get chunks array from file cluster
let chunkPromises = urls.map(function callback(url,index) {
return fetch(url).then((res) => {
return res.arrayBuffer();
});
});
console.log(chunkPromises);
console.log(0);
//I'm trying to understand how to do it
(async function loop() {
await Promise.all(chunkPromises).then(chunks => {
console.log(chunks);
console.log(1);
//Appending file chunks to file back
for (const value of chunks) {
console.log(2);
}
video.attr('src', URL.createObjectURL(full_video));
console.log(3);
//I want that promise complete loadedmetadata, then go to another promise iteration
video.on("loadedmetadata", function () {
console.log(`${secondsReady} seconds of video are ready to play.`);
});
});
});
}
If it possible i woold like, not touch all messages loop.

You can actually use async/await in for loops, it doesn't work well with other forloop iteration like forEach, filter etc. :
const PromiseMe = () => {
return new Promise((res, rej) => {
setTimeout(() => {
res('Integrate Me');
}, 1000)
});
}
const main = async () => {
for(let i = 0; i< 10; i++) {
let val = await PromiseMe();
console.log(val);
}
}
main();
So, you can actually implement this in you for-loop like this.
const result = [];
const mockFunction = async () => {
for (var i = 0; i < data[0].length; i++) {
//Get chunks array from file cluster
//urls is an array
let chunkPromises = await fetch(url[i]);
result.push(chunkPromises); //parse this data to specific format if needed
}
console.log(result);

Javascript await inside a loop

I am trying to work with an api where I have to send a request for each item in a list.
However, I see that the loop doesn't seem to wait for every request, i.e, the loop doesn't work as expected. Here's the code below
getInfo = async () => {
const mylist = ["item1","item2","item3","item4","item5","item6","item7"]
const responses = []
const len = mylist.length
for (let i = 0; i < len; i++) {
//console.log("inside loop")
await axios.get("some_url/"+mylist[i])
.then(res => {
responses.push(res.data)
})
}
When I run the program, all the console.log("inside loop") executes immediately without waiting for the request to be complete.
How can I modify the code so as to wait for each response to be completed before updating the for loop counter variable?

You could try re-arranging the code to something like this. But using a Promise.all with Array.prototype.map would be more idiomatic solution for the problem.
await the async call (remove unnecessary .then call) and then console.log
getInfo = async () => {
const mylist = ["item1","item2","item3","item4","item5","item6","item7"]
const responses = []
const len = mylist.length
for (let i = 0; i < len; i++) {
responses.push((await axios.get("some_url/"+mylist[i])).data)
console.log("inside loop")
}
}

Internally, await is translated into a Promise chain. Since the for loop can't be transformed into a Promise-continuation, you'll need to convert it to a Promise-based construct.
Depending on what you want to achieve there are multiple ways to go about it.
Constructing the responses array could be done with a map statement.
const promises = mylist.map(item => {
return axios.get("some_url/"+item).then(res => { return res.data; })
});
const data = await Promise.all(promises);
No manual pushing items around or fiddling with the array length.

insertMany inside setTimeout and for loop with async functions

I'm trying to write the following code and make it work synchronously, but the only problem that it works correctly with console.log, which prints me every item in array with delay in 1 second, but don't work with the following structure:
for (let i = 0; i < array.length; i++) {
setTimeout(function () {
1.http request via rp or request.get (I receive a huge data array)
2. .map results
3.insert to Mongo via mongoose
}
}
as for now I have the following code inside:
request.get({url: array[i].url}), function (error, body) {
body.map(element => {
//do stuff, it works fine
});
collection.insertMany(body, function (err, docs) {
//#justloggerthings
}
Or I have almost the same version with rp instead of request.get
By default I have mongoose.Promise = global.Promise;
Why this cause a problem? Because body.length is very huge dataset which eat a lot of RAM. (Now imagine 20+ arrays with insertMany)
So Mongo trying to insertMany all responses from request at once (when they ready, w/o 1000s delay). Actually that's why I choose request instead of rp (request-promise) but it seems look async too. So should I choose another http get module from npm and switch to it. And not to worry about it?
Or should I wrap this operations to promise || made an async function and recall it inside loop every time (1000s for example) when I it's correctly finished. In this case, the only thing which I found actual on StackOverflow is:
How to insert data to mongo synchronously (Nodejs, Express)
Bit it's a bit outdated. So any ideas?

Well, i dont have your actual code so i will write in pseudo code what you can do.
const chunkArray = (array, chunkSize) => {
let i,j,chunks = [];
for (i = 0, j = array.length; i < j; i += chunkSize) {
chunks.push(array.slice(i, i + chunkSize));
}
return chunks;
}
for (let i = 0; i < array.length; i++) {
let bigArray = await request('your data url');
// do some mapping or whatever
// then break your large array into smaller chunks
let chunks = chunkArray(bigArray, 10);
let j;
for (j = 0; j < chunks.length; j++) {
await collection.insertMany(chunks[j]);
}
}

Actual code that solve my problem is:
async function test (name, url, lastModified) {
try {
const response = await rp({uri: url, json: true});
response.map(async (element) => {
if (element.buyout > 0) {
element.price = (element.buyout / element.quantity);
}
element.lastModified = lastModified
});
return collection.insertMany(response);
} catch (err) {
console.error(err)
}
}
async function addAsync() {
const z = await test();
console.log(z);
}
addAsync();

Javascript Converting ethereum web3 json object into array

I'm working with web3 trying to pull live data from the Ethereum main net. The promise returns a JSON object, I want to get the info for the last 10 blocks and push this object into an array latest so that I can output it elsewhere using something similar to, for example, latest[0].difficulty.
Every time I push the block info into the array it seems to just bundle it in in a weird way so that when I do latest.length it comes back as 0
I could do with a primer on arrays and objects I think but any info is much appreciated.
window.addEventListener('load', function() {
// sconsole.log('No Web3 Detected... using HTTP Provider')
const web3 = new Web3(new Web3.providers.WebsocketProvider("wss://rinkeby.infura.io/ws"));
console.log('Web3 Detected! ')
window.web3 = new Web3(web3.currentProvider);
if (typeof window.web3 !== 'undefined') {
// You have a web3 browser! Continue below!
// startApp(web3);
init()
//alert("Web3");
}
})
async function getArray() {
latest = []
await web3.eth.getBlockNumber(function(error, blockNumber) {
for (var i = 0; i < 10; i++) {
web3.eth.getBlock(blockNumber - i).then(obj => {
latest.push(obj)
})
}
})
return latest
}
async function init() {
await getArray().then(v => {
console.log(v.length)
console.log(v)
})
}
The data output from the initial for loop over the blocks looks like this, basically 10 individual objects with data for those blocks
{
difficulty:"1"
extraData:"0xd683010811846765746886676f312e3130856c696e7578000000000000000000cf7828c6662bacad0822d8bdc84a93344f25a1902c54fecb714667177a997dc9569305ec918859784fa1849509da15003eb46f53e831b630991697f3debb842600"
gasLimit:7134847
gasUsed:7070346
hash:"0xb7fff64678bb329288fbec01aaa57038250403674297d754b686b50f5c5c273f"
logsBloom:"0x00002000024010080000022040108000000000000000008001002004000004000000021000000800000000200006000000000080000000000000000000000002000000000020000000000008000000000000000000001004000000000000004000000000020000002000000000000000000000000000040000000010000000000020108000000000080088000008000000000000000000000000000000000000000080004002000008000002c00000000000000000000000002010000010000002000002000000000010800000000000000000040008000000000000000000100000800200000400000000000480000022000000000480000000000000008000"
miner:"0x0000000000000000000000000000000000000000"
mixHash:"0x0000000000000000000000000000000000000000000000000000000000000000"
nonce:"0x0000000000000000"
number:3077903
parentHash:"0x4ca2eabaa9d5759170be9efb43d0a803ab11c8ab35f9649701456a5c2448d781"
receiptsRoot:"0x7467d9dabf04e6b8037786dd62b2cc1a7b9aad85cb5188ab44e8ba2b82b90b54"
sha3Uncles:"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347"
size:3166
stateRoot:"0xa81abd0e2f1017179e6ccbc8bf88b579041e65eec7efbce2c763b3b982869191"
timestamp:1538295579
totalDifficulty:"5718449"
transactions:(9) ["0x172af85792c52de0adf86841816523db6cd3bcfac8f0e01f6eb2bcac46621dff", "0xe6dc06855a3514eb41e1770ff4b731a1b1ed4672ae4afd3537e3f04594b8a8ba", "0x622f667af2d6741642b42873ce002b11e98472b86d2fc26a011c085e67fc68e5", "0x6cf3f5a6f0c22ce26bec9372b23bd9b434b1d7371592c272203492da1feb1f9e", "0xaebcdd5fc8f29fc8c265c2b0bc261665e2f5bc7d8c0acf711886731d47ad7158", "0x0ee2d28ba83cc9f3f82b2c8184e51719006d3bd38102d9d7e99dc5803c499ab7", "0x56314a0423a3683feda5e4a6eb9117e35e07e853b8ae28d91dbfc597fa04d98a", "0xaaaee387c6837a8531b3d38fbed0961d8cb34924c4a6fe5ce5e907c793c81359", "0xb5580743c4bcf879c9059d785b9821c8d6d41d68f4a751bc097c74a5ef2d5517"]
transactionsRoot:"0xc15d937720ee7807fce8606c66af67c495afd917fd733a2a6121a410b8530019"
}

There seemed to be some confusion on when to use the await operator and when to use a callback function, since you were trying to use both.
I've changed the code to solely use the promise versions, and the await operator.
getArray = async () => {
const blocknumber = await web3.eth.getBlockNumber()
for (var i = 0; i < 10; i++) {
const block = await web3.eth.getBlock(blocknumber - i)
latest.push(block)
}
return latest
}
This will properly return an array of ten blocks, counting backwards from the current block number.
When you want to use it you can call it as such:
someAsyncFunctionWhereWeAreWorking = async () => {
const blocks = await this.getArray()
blocks.map(block => {
//we can do things here to each item of the block
console.log(block.difficulty)
})
}
Note that this must be called from within an async function

Unable to receive proper data from the promise function

I am trying to scrap wikipedia page to fetch list of airlines by first scrapping first page and then going to each individual page of airline to get the website url. I have divided the code in two functions. One to scrap main page and get a new url, and second function to scrap another page from the created url to get the website name from that page. I have used request-promise module for getting the html and then cheerio to parse the data.
export async function getAirlinesWebsites(req,res) {
let response = await request(options_mainpage);
console.log(`Data`);
let $ = cheerio.load(response);
console.log('Response got');
$('tr').each((i,e)=>{
let children = '';
console.log('inside function ', i);
if($(e).children('td').children('a').attr('class') !== 'new') {
children = $(e).children('td').children('a').attr('href');
let wiki_url = 'https://en.wikipedia.org' + children;
console.log(`wiki_url = ${wiki_url}`);
let airline_url = getAirlineUrl(wiki_url);
console.log(`airline_url = ${airline_url}`);
}
})
And then the getAirlineUrl() function will parse another page based on the provided url.
async function getAirlineUrl(url){
const wiki_child_options = {
url : url,
headers : headers
}
let child_response = await request(wiki_child_options);
let $ = cheerio.load(child_response);
let answer = $('.infobox.vcard').children('tbody').children('tr').children('td').children('span.url').text();
return answer;
})
However when I console log the answer variable in the parent function, I get a [object Promise] value instead of a String. How do I resolve this issue?

Async function return promise.In case of that,you need to use then to get resolved response or use await.
This should work if other part of your code is ok.
export async function getAirlinesWebsites(req, res) {
let response = await request(options_mainpage);
console.log(`Data`);
let $ = cheerio.load(response);
console.log("Response got");
$("tr").each(async (i, e) => {
let children = "";
console.log("inside function ", i);
if ($(e).children("td").children("a").attr("class") !== "new") {
children = $(e).children("td").children("a").attr("href");
let wiki_url = "https://en.wikipedia.org" + children;
console.log(`wiki_url = ${wiki_url}`);
let airline_url = await getAirlineUrl(wiki_url);
console.log(`airline_url = ${airline_url}`);
}
});
}

Since your getAirlineUrl function returns a promise, you need to await that promise. You can't have await nested inside of the .each callback because the callback is not an async function, and if it was it wouldn't work still. The best fix is the avoid using .each and just use a loop.
export async function getAirlinesWebsites(req,res) {
let response = await request(options_mainpage);
console.log(`Data`);
let $ = cheerio.load(response);
console.log('Response got');
for (const [i, e] of Array.from($('tr')).entries()) {
let children = '';
console.log('inside function ', i);
if($(e).children('td').children('a').attr('class') !== 'new') {
children = $(e).children('td').children('a').attr('href');
let wiki_url = 'https://en.wikipedia.org' + children;
console.log(`wiki_url = ${wiki_url}`);
let airline_url = await getAirlineUrl(wiki_url);
console.log(`airline_url = ${airline_url}`);
}
}
}