I came from Ruby, so I spent a day to figure out why process.exit(1) executes before loop end. Any ideas what I need to change to make it work?
export default async () => {
const workbook = Xlsx.readFile(path.join(__dirname, './excel.xlsx'));
const data = Xlsx.utils.sheet_to_json(workbook.Sheets.Extract, {
header: getHeaderFor(workbook.Sheets.Extract),
});
await data.map(async (product, index) => {
if (index < 6) return;
await processProduct(product);
});
process.exit(1);
};
I think that If you replace:
await data.map(async (product, index) => {
if (index < 6) return;
await processProduct(product);
});
with
for (let i = 6; i < data.length; i++) {
await processProduct(data[i]);
}
your problem will be solved.
Some explanation: javascript's map function is creating new array from source array when it is calling a callback function for each element of the source array and pushing the returned value of the callback into the result array. In this case await data.map(... is invalid statement, because the result of map is array. To fix this it is possible to wrap the result in await Promise.all(data.map(...)) or to use a normal for loop. You can use the first case when you don't care about concurrency and the second case when you want to achieve sequential order.
While #codtex's answer will work, it will also result in poor performance since each iteration of the loop will wait until the previous iteration. What you really want to do is collect all of the Promises, and then wait for them to finish:
const promises = data.map((product, index) => {
if (index < 6) return;
return processProduct(product);
});
await Promise.all(promises);
Related
I have a loop that gets iterated through 1000 times where each iteration makes a request and then prints the result of that request.
Similar to below.
let start = console.log(Date.now())
for (i = 0; i < 1000; i++) {
request.then(data => {
console.log(Date.now() - start)
})
}
This however results in the first request taking much longer to be completed then if there was just 1 iteration of the loop.
with i < 1000:
5860
...
with i < 1:
220,
...
For the aim of this script however, I want to wait for each result to be received before starting the next iteration of the loop.
If you want to stick with the ES5 way of handling Promises, you could use the following approach:
const start = console.log(Date.now())
// Create a instantly resolved promise
for (let i = 0, p = Promise.resolve(); i < 1000; i++) {
// append new promise to the chain
p = p.then(() => request())
.then(() => console.log(Date.now() - start));
}
If you can use ES6 methods, you could write this using the async/await pattern like so:
const asyncLoop = async () => {
const start = console.log(Date.now())
for (let i = 0; i < 1000; i++) {
const data = await request();
console.log(Date.now() - start);
}
}
asyncLoop();
The chance that you really want to make the requests one after the other is actually pretty small though, so in case you want to make the request simultaneously, but do something after all of them resolved, you can use Promise.all(...)
const start = console.log(Date.now())
const requests = [request1, request2, ...];
Promise.all(requests)
.then(() => console.log(Date.now() - start));
You can use the async-await pattern here.
You can achieve this by changing the code
async function iteration() {
let start = console.log(Date.now())
for (let i = 0; i < 1000; i++) {
const data = await httpRequest()
console.log(Date.now() - start)
}
}
async function httpRequest() {
return new Promise((resolve, reject) => {
request.then(data => {
//Do anything you want to do with data
resolve(data)
}).catch(error => {
console.error(`Error in request`)
reject(error)
})
})
}
Explanation:
I have moved the request code in a separate function httpRequest and this function will return promise on success
I called httpRequest in for loop using await keyword, now loop will wait till the request is completed
I have also wrapped for loop code in function since you can only use await inside an async function
I am trying to work with an api where I have to send a request for each item in a list.
However, I see that the loop doesn't seem to wait for every request, i.e, the loop doesn't work as expected. Here's the code below
getInfo = async () => {
const mylist = ["item1","item2","item3","item4","item5","item6","item7"]
const responses = []
const len = mylist.length
for (let i = 0; i < len; i++) {
//console.log("inside loop")
await axios.get("some_url/"+mylist[i])
.then(res => {
responses.push(res.data)
})
}
When I run the program, all the console.log("inside loop") executes immediately without waiting for the request to be complete.
How can I modify the code so as to wait for each response to be completed before updating the for loop counter variable?
You could try re-arranging the code to something like this. But using a Promise.all with Array.prototype.map would be more idiomatic solution for the problem.
await the async call (remove unnecessary .then call) and then console.log
getInfo = async () => {
const mylist = ["item1","item2","item3","item4","item5","item6","item7"]
const responses = []
const len = mylist.length
for (let i = 0; i < len; i++) {
responses.push((await axios.get("some_url/"+mylist[i])).data)
console.log("inside loop")
}
}
Internally, await is translated into a Promise chain. Since the for loop can't be transformed into a Promise-continuation, you'll need to convert it to a Promise-based construct.
Depending on what you want to achieve there are multiple ways to go about it.
Constructing the responses array could be done with a map statement.
const promises = mylist.map(item => {
return axios.get("some_url/"+item).then(res => { return res.data; })
});
const data = await Promise.all(promises);
No manual pushing items around or fiddling with the array length.
The following code (which I've simplified for clarity) loops through and returns the cardsToInsert before each part of the loop finishes, so the array doesn't get built properly.
The loops build the array correctly, but the result got returned near the beginning, not after it was built.
How do I get it to finish all the loops before returning the array?
async function loopCards(cardsToGet) {
for (let cardToGet of cardsToGet) {
getDataFunctionWhichReturnsAPromise()
.then(async (card) => {
writeCard = await buildingCard(card)
cardsToInsert.push(writeCard)
}
return cardsToInsert
}
thanks
Full Code Added as Requested
// wixData.get() returns a promise
async function loopCards(cardsToGet) {
let writeCard
let buildCard
for (let index = 0; index < cardsToGet.length; index++) {
const cardToGet = cardsToGet[index].card
buildCard = wixData.get("Card", cardToGet)
.then(async (card) => {
writeCard = await buildingCard(card)
cardsToInsert.push(writeCard)
}
)
.catch((err) => {
let errorMsg = err;
console.log("getcard error: " + errorMsg);
return errorMsg
});
}
return cardsToInsert
}
Here is a detailed explaination
loops are synchronous.
Promises are asynchronous.
To get the data from promises you need to wait for it to finish using callback,async-await or promise.
In your code, you are putting .then to access the result of wixData but the whole wixData.get("Card", cardToGet).then(async (card) => {}) is an async function for loopCards(cardsToGet) and because of this your loop finishes and the result array is empty.
Solution -
Just wait for wixData.get("Card", cardToGet) to finish and then do the manipulations. Here is a solution using async await.
async function loopCards(cardsToGet) {
let cardsToInsert = [];
for (let index = 0; index < cardsToGet.length; index++) {
const cardToGet = cardsToGet[index].card
let card = await wixData.get("Card", cardToGet)
let writeCard = await buildingCard(card)
cardsToInsert.push(writeCard)
}
return cardsToInsert
}
In the above code I wait for the wixData.get("Card", cardToGet) to finish and store the result in get card. This should fix your problem.
but this code by performance is not optimal as you are waiting for each network call. what you can do is execute all the Promises at once using Promise.all
Here is the code for that with error handling -
async function loopCards(cardsToGet){
try {
return await Promise.all( cardsToGet.map( cardToGet => buildCard( cardToGet.card )));
}
catch (error) {
console.log(error);
}
}
async function buildCard(cardToGet){
try {
let card = await wixData.get("Card", cardToGet);
let writeCard = await buildingCard(card);
return writeCard;
}
catch (error) {
console.log("getcard error: " + error);
return error;
}
}
The above code might have some erros since I haven't tested it but I hope you get the approach.
you should wrap loop with in promise .then or also you are using async its mean a very first item of array will be print and then it will await
getDataFunctionWhichReturnsAPromise()
.then(async (card) => {
for (let cardToGet of cardsToGet) {
writeCard = await buildingCard(card)
cardsToInsert.push(writeCard)}
}
var a = ['url1', 'url2', 'url3'];
var op = [];
cb = (callback) => {
for (var i = 0; i < a.length; i++) {
gtts.savetos3(`${a[i]}.mp3`, a[i], 'ta', function(url) {
console.log(url['Location']);
op.push(url['Location']);
});
}
callback()
}
cb(() => {
console.log(op);
})
In the above code the gtts.savetos3 is an asynchronous function.It takes significance amount of time to complete the execution for every element in array.Due to asynchronous feature I cannot print the complete array of url in op array as it prints an empty array.
gtts.savetos3 function calls the given callback with correct url so that i can print the output using console.log but when comes to looping i got messed up.
My question is
How to make the callback function called only after the execution of
the all array elements get processed by the gtts.savetos3 function.
can we achieve the solution for above problem without Promise.all or without Promise with the help of using only callbacks.
Thanks in Advance ...!
You can keep a counter and increase it inside the methods's callback,
call your done callback only when the counter reaches the length of
the array.
cb = (done) => {
let counter = 0;
for (let i = 0; i < a.length; i++) {
gtts.savetos3(`${a[i]}.mp3`, a[i], 'ta', function (url) {
console.log(url['Location']);
op.push(url['Location']);
++counter;
if (counter == a.length) {
done();
}
});
}
}
cb(() => {
console.log(op);
})
This is just a way to solve the problem without Promises or any third party module but not the elegant or correct way.
If you want to stick to the callback and are ok to use third-party module have a look on
Async waterfall method.
If you are using aws-sdk's s3 put object, then sdk already provides a
promisified methods as well, you can simply append your method with
.promise to get the same.
To solve the problem with promises just change your wrapper into an
async function.
async savetos3(...parametres) {
//Some implementation
let res = await S3.putObject(....params).promise();
//Some implementation
}
cb = Promise.all(a.map(name => savetos3(`${name}.mp3`, name , 'ta')));
cb.then(() => {
console.log(op);
})
Here is my solution.
const a = ['url1', 'url2', 'url3'];
const op = [];
const saveToS3 = name => new Promise((resolve, reject) => {
gtts.savetos3(`${name}.mp3`, name, 'ta', function (url) {
console.log(url['Location']);
resolve(url)
});
})
Promise.all(a.map(item => saveToS3(item))).then(() => {
console.log(op)
})
Ok so I´m having troubles understanding how async/await, Promises, etc. in nodejs work, this is my first time programming in an asynchronous language.
What im trying to do here is basically select one random entry from the mongoose-model "SecSolution". Currently when arr is returned it´s empty, and the debug message on the bottom gets printed before the debug on the top gets printed.
I just want the function to return "arr" after it gets its value.
async function getRandomCardIds(deckIdentifier, cardCount) {
let arr;
switch (deckIdentifier) {
case 102:
await SecSolution.count().exec(async function (err, count) {
let promises = [];
var random = Math.floor(Math.random() * count);
for (let i = 0; i < 2; i++) {
promises.push((await SecSolution.findOne().skip(random).lean())._id);
}
arr = await Promise.all(promises);
debug("This gets printed second" + arr);
});
break;
}
debug("this gets printed first");
return arr;
}
Thanks in advance!
Do not use callbacks when working with async/await. (And when working with plain promises, use only then callbacks). Also you shouldn't use await on a promise that you still need as a promise object, to pass it to Promise.all. Your code should be
async function getRandomCardIds(deckIdentifier, cardCount) {
switch (deckIdentifier) {
case 102:
const count = await SecSolution.count(); // a promise(like object)
let promises = [];
var random = Math.floor(Math.random() * count);
for (let i = 0; i < 2; i++) {
promises.push(SecSolution.findOne().skip(random).lean());
}
let arr = await Promise.all(promises);
debug("This gets printed second" + arr);
return [arr[0]._id, arr[1]._id];
break;
}
debug("this gets printed first");
return undefined;
}
Instead of accessing the _ids on the objects in the result array, you could also have transformed the promises directly (similar to what you tried with the await):
promises.push(SecSolution.findOne().skip(random).lean().then(card => card._id));
well basically you have to think that it will try to run everything and all the code that needs to wait to be resolved wont stop the process of running all the code.
thus, looking at your code, we can see the following:
1) define arr as undefined then go inside the switch.
2) in the switch statement we have an await so it will wait (but it wont stop the other code to run because it is not on the same statement), it will resolve later.
3)prints the debug message
4) returns undefined because the await inside of switch is not resolved.
5) one day the statement is resolved but nothing you can do about it.
an example could be the following.
function resolveAfter2Seconds() {
return new Promise(resolve => {
setTimeout(() => {
resolve('resolved');
}, 2000);
});
}
async function asyncCall() {
console.log('calling');
var result = await resolveAfter2Seconds();
console.log(result);
// expected output: "resolved"
}
asyncCall();
so what you can do in your case is the following:
function resolveInnerFunct() {
return new Promise(resolve => {
let promises = [];
var random = Math.floor(Math.random() * count);
for (let i = 0; i < 2; i++) {
promises.push(SecSolution.findOne().skip(random).lean())._id));
}
Promise.all(promises).then(values=> {return resolve(values)};
});
}
async function asyncCall() {
console.log('calling');
let arr;
switch (deckIdentifier) {
case 102:
// this will wait until the resolveInnerFunct resolves.
arr = await resolveInnerFunct();
debug("this wont get printed before the other message")
break;
}
debug("this gets printed first");
return arr;
}
asyncCall();