Develop Reference

using async/await in a filter operation not waiting to finish after first await call

For a specific function, a list of tags (rfids) are required from a database, but various tables should be checked.
The issue is centered around the problem of a long running operation isn't awaited upon to finish before proceeding to the next.
Problem:
The code called below runs as follows (# numbers correspond to number in code below):
getAllTags() is called, starting the proc
await db.dbRfid.findAll() returns with X amount of valid results (not Promise objects)
start filtering call on valid objects & await finish (await place since some function calls inside are async calls.
call const driverTags = await tag.getDrivers();
At this point, one would expect this function to return the result and proceed to the next function i.e.
// #5
const truckTags = await tag.getTrucks();
What infact happens is for each of the allTags items, the getDrivers() gets called and the filter() exits and the code continues on by executing this next:
// #6
if (pureTags) {
return filteredTags;
}
Question:
To my understanding, I am awaiting async operations correctly, yet it seems like the filter only accepts/allows one async operation.
I assume I did something wrong, yet I am unable to the cause of the problem. Any advice would be appreciated!
Full code implementation below:
Called with e.g.
const listOfTags = await getAllTags();
Specific tag requirements (all valid, unused, enabled & unrevoked tags)
const listOfTags = await getAllTags(false, true, true, true);
Problem code:
// #1
const getAllTags = async (pureTags = false, unused = true, enabled = true, notRevoked = false) => {
// #2
let allTags = await db.dbRfid.findAll()
// #3
let filteredTags = await allTags.filter(async tag => {
// check tag used
if (unused) {
// #4
const driverTags = await tag.getDrivers();
// #5
const truckTags = await tag.getTrucks();
const userTags = await tag.getUsers();
if (driverTags.length > 0) {
return false;
}
if (truckTags.length > 0) {
return false;
}
if (userTags.length > 0) {
return false;
}
}
// check tag enabled
if (enabled && !tag.enabled) {
return false;
}
// check tag revoked or return true
return notRevoked && !tag.revoked;
});
// return tags as is
// #6
if (pureTags) {
return filteredTags;
}
return filteredTags.map(tag => {
return {
id: tag.id,
rfid: tag.rfid,
expiryDate: tag.expiryDate,
revoked: tag.revoked,
enabled: tag.enabled
}
});
}
Update
I should mention:
no 'errors' of any sort are shown to indicate of some problem.
the .getDrivers() is a getter created by sequelize which returns a promise.
Update 2
After comment by evgeni fotia
I originally had this code, however chose not to include it as it may havve complicated matters. However, this it the original code I attempted with the addition of the await Promise.all().
const getAllTags = async (pureTags = false, unused = true, enabled = true, notRevoked = false) => {
let allTags = await db.dbRfid.findAll()
let filteredTags = await Promise.all(
// await allTags.filter(async tag => { //tried the await here too - for good measure
allTags.filter(async tag => {
// check tag used
if (unused) {
const driverTags = await tag.getDrivers();
const truckTags = await tag.getTrucks();
const userTags = await tag.getUsers();
if (driverTags.length > 0) {
return false;
}
if (truckTags.length > 0) {
return false;
}
if (userTags.length > 0) {
return false;
}
}
// check tag enabled
if (enabled && !tag.enabled) {
return false;
}
// check tag revoked or return true
return notRevoked && !tag.revoked;
})
);
// return tags as is
if (pureTags) {
return filteredTags;
}
return filteredTags.map(tag => {
return {
id: tag.id,
rfid: tag.rfid,
expiryDate: tag.expiryDate,
revoked: tag.revoked,
enabled: tag.enabled
}
});
}
Please note, after running the code in this update or in the original question, I see the debugger hitting the getDrivers() method, then the (HTTP) response is sent to the client, and only after 0.5~1s I see the getDrivers() method returning and proceeding to the next method.

Your central problem is, as T.J. Chowder has commented, that you are trying to .filter() on something that is sometimes a boolean, and sometimes a promise.
You should do the mapping and the filtering in different steps. I tend to prefer the .then() syntax, so here's my approach:
const getAllTags = (pureTags = false, unused = true, enabled = true, notRevoked = false) => {
return db.dbRfid.findAll()
.then(allTags => allTags.map(tag => Promise.resolve(
enabled === tag.enabled && notRevoked === !tag.revoked && Promise.all([
tag.getDrivers(), tag.getTrucks(), tag.getUsers()
]).then(results => results.some(r => r.length))
).then(ok => ok ? tag : null)))
.then(pendingTags => Promise.all(pendingTags))
.then(resolvedTags => resolvedTags.filter(tag => tag))
.then(filteredTags => filteredTags.map(tag => pureTags ? tag : {
id: tag.id,
rfid: tag.rfid,
expiryDate: tag.expiryDate,
revoked: tag.revoked,
enabled: tag.enabled
}));
};
The logic of this code:
fetch tags from DB
produces Promise<Tag[]>, i.e. a promise for an array of tags
map each tag to a new promise that resolves either to the tag itself, or null, depending on a condition we figure out using Promise.resolve(), to account for the potentially(*) async nature of that check
produces Promise<Promise<Tag|null>[]>
wait for all those inner promises to resolve using Promise.all()
produces <Promise<(Tag|null)[]>
filter out the null values (i.e. tags we don't want)
produces <Promise<Tag[]>
map the tags to the overall result data structure we want, depending on pureTags
produces <Promise<(Tag|object)[]>
(*) The Promise.all() is only invoked if the preconditions check out, due to short circuiting. If not, it's simply a Promise.resolve(false), which resolves to false immediately. In the other case it's going to be Promise.resolve(<Promise<Boolean>), which works exactly the same way as Promise.resolve(Boolean). This way we can unify the synchronous and the asynchronous test.
In the subsequent .then() we can decide whether to return the tag - or null, to indicate that this tag failed the filter conditions. The null values are then picked out by resolvedTags.filter(tag => tag).
Note that your code serializes the three async checks (getDrivers, getTrucks, getUsers) because each is awaited before the next is started. Using Promise.all() lets them run in parallel, which is one of the reasons why I dislike using async/await for everything.
I leave rewriting this into async/await style as an exercise.

Using variable from a Async function [duplicate]

This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference
(7 answers)
Closed 2 years ago.
I have an item that I got from an Async function as follows:
items = await statusCheck(items.split('\n'))
The items is an array which I need to add to localStorage. The issue is, although it has elements inside of it, they are not accessible.
Therefore, I have to do this:
items = await statusCheck(items.split('\n'))
setTimeout(() => {
localStorage.setItem(localKey, items.join('\n'))
}, 1000)
Is there a more elegant way of doing this without having to resort to a setTimeout function?
Is there more information I can provide to help resolve this issue? The FULL code is as follows:
async function clearHistory() {
var localKey = "uploadcare_" + UPLOADCARE_PUBLIC_KEY
var items = localStorage.getItem(localKey);
items = await statusCheck(items.split('\n'))
setTimeout(() => {
localStorage.setItem(localKey, items.join("\n"));
}, 1000)
}
async function statusCheck (items) {
let newItems = []
items.forEach(async (item) => {
let url = "https://somelink" + item.split(' ')[0] + "/54x54/"
let status = await fetchResponse(url)
if (status === 200) {
newItems.push(item)
}
})
return newItems
}
async function fetchResponse(url) {
try {
let response = await fetch(url)
let status = response.status
return status
} catch (e) {
console.log(e)
}
}
clearHistory()

You shouldn't be doing a forEach with async. You should use map and get each promise out. Then do Promise.all. Something like this:
async function statusCheck (items) {
let promises = items.map(item => {
let url = "https://somelink" + item.split(' ')[0] + "/54x54/";
return fetchResponse(url);
});
await Promise.all(promises);
// figure out which ones to return.
}

Async functions need to return a Promise. Your code runs synchronously. I am not sure why the browser won't tell you. So your statusCheck() function needs to return a Promise object to be awaitable. This also applies to the the forEach loop in the statusCheck. And I think that you cant do the foreach async because the native function wont wait for those callbacks.
See this reference for async function: https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Statements/async_function

There is no reason to use setTimeout. the main problem is that you are using await inside the function you are passing to forEach, and there is no reason for javascript to wait until all of the requests resolve and then returning the array. it keeps going and returns possibly an empty array first. and when you are using the new created array it, some of the requests might resolve and they results will be pushed to that array. you can use this instead:
async function statusCheck (items) {
return Promise.all(items.map(async (item) => {
let url = "https://somelink" + item.split(' ')[0] + "/54x54/"
let status = await fetchResponse(URL)
if(status !== 200) {
return;
}
return item;
}).filter(Boolean);
}
I used Boolean just to remove items that checking them by API might result in a status other than 200. in this case the function return;s or in the other words, returns undefined. as a matter of fact, I've filtered those results by filter(Boolean)

Problems with Promise.all [duplicate]

This question already has an answer here:
Promise and Promise.all(array) executed before array fulfilled [duplicate]
(1 answer)
Closed 26 days ago.
I have to get back to this forum for help because I can't still make a 'Promise.all' work!
First, I have this function, which is supposed to return a promise:
const myFetch = (a, b) => {
var url;
// some stuff bulding 'url' using a and b
fetch(url).then(response => {
return response.json();
}
})
The idea is that the above function returns a promise, whose value, once resolved, is the json object. I have checked that the json is actually valid. If I replace the "return ..." line with the line below I actually get a valid json:
response.json().then(res=> console.log(res))
Second, I have this for loop, after which I expect to have an array of promises:
promises = [];
for (...){
// some other stuff
promises.push(myFetch(a, b))
}
Eventually I execute this code:
Promise.all(promises)
.then(responses => { // <=== Here I get all "undefined"
responses.forEach(response => {
console.log(response);// <=== Here I get all "undefined"
});
// some other stuff that I can do only after all fetches are complete
})
I expect the .then portion to be executed only once all promises are resolved, and also expect "responses" to be a list of all the json responses from the individual promises above. Still, I get a string of "undefined". The impression is that the portion of code within .then is running even though the promises are not yet resolved.
What did I do wrong? How can I be sure to have all the json objects from the individual fetches before proceeding? (note, I cannot use await/async). Thanks

You need to return the promise from the fetch call, otherwise the promise chain will be broken, once you do this, all should play nice!
Something like so should work:
const myFetch = (a, b) => {
var url;
// some stuff bulding 'url' using a and b
return fetch(url).then(response => {
return response.json();
})
};
A snippet example:
const myFetch = (url, a, b) => {
return fetch(url).then(response => {
return response.json();
})
};
function testMyFetch() {
promises = [];
for(let i = 0; i < 5; i++) {
promises.push(myFetch("https://jsonplaceholder.typicode.com/users/" + (i+1)));
}
Promise.all(promises).then(result => console.log("Promise.all result:", result));
}
testMyFetch();

Sequential execution of Promise.all

Hi I need to execute promises one after the other how do I achieve this using promise.all any help would be awesome. Below is the sample of my code I am currently using but it executes parallel so the search will not work properly
public testData: any = (req, res) => {
// This method is called first via API and then promise is triggerd
var body = req.body;
// set up data eg 2 is repeated twice so insert 2, 5 only once into DB
// Assuming we cant control the data and also maybe 3 maybe inside the DB
let arrayOfData = [1,2,3,2,4,5,5];
const promises = arrayOfData.map(this.searchAndInsert.bind(this));
Promise.all(promises)
.then((results) => {
// we only get here if ALL promises fulfill
console.log('Success', results);
res.status(200).json({ "status": 1, "message": "Success data" });
})
.catch((err) => {
// Will catch failure of first failed promise
console.log('Failed:', err);
res.status(200).json({ "status": 0, "message": "Failed data" });
});
}
public searchAndInsert: any = (data) => {
// There are database operations happening here like searching for other
// entries in the JSON and inserting to DB
console.log('Searching and updating', data);
return new Promise((resolve, reject) => {
// This is not an other function its just written her to make code readable
if(dataExistsInDB(data) == true){
resolve(data);
} else {
// This is not an other function its just written her to make code readable
insertIntoDB(data).then() => resolve(data);
}
});
}
I looked up in google and saw the reduce will help I would appreciate any help on how to convert this to reduce or any method you suggest (Concurrency in .map did not work)

the Promises unfortunatelly does not allow any control of their flow. It means -> once you create new Promise, it will be doing its asynchronous parts as they like.
The Promise.all does not change it, its only purpose is that it checks all promises that you put into it and it is resolved once all of them are finished (or one of them fail).
To be able to create and control asynchronous flow, the easiest way is to wrap the creation of Promise into function and create some kind of factory method. Then instead of creating all promises upfront, you just create only one promise when you need it, wait until it is resolved and after it continue in same behaviour.
async function doAllSequentually(fnPromiseArr) {
for (let i=0; i < fnPromiseArr.length; i++) {
const val = await fnPromiseArr[i]();
console.log(val);
}
}
function createFnPromise(val) {
return () => new Promise(resolve => resolve(val));
}
const arr = [];
for (let j=0; j < 10; j++) {
arr.push(createFnPromise(Math.random()));
}
doAllSequentually(arr).then(() => console.log('finished'));
PS: It is also possible without async/await using standard promise-chains, but it requires to be implemented with recursion.

If anyone else cares about ESLint complaining about the use of "for" and the "no await in loop" here is a typescript ESLint friendly version of the above answer:
async function runPromisesSequentially<T>(promises: Array<Promise<T>>):Promise<Array<T>> {
if (promises.length === 0) return [];
const [firstElement, ...rest] = promises;
return [await firstElement, ...(await runPromisesSequentially(rest))];
}
You can then just replace Promise.all by runPromisesSequentially.

#lmX2015's answer is close but it's taking in promises that have already started executing.
A slight modification fixes it
export async function runPromisesSequentially<T>(functions: (() => Promise<T>)[]): Promise<T[]> {
if (functions.length === 0) {
return [];
}
const [first, ...rest] = functions;
return [await first(), ...(await runPromisesSequentially(rest))];
}

Chained and Nested promises with For Loop

I am trying to get each property of my games within chained promises (Each of the property is coming from a different async calls).
Logic of my algorithm:
Check the Network and Get the smart contract address
Register the contract containing the addresses of all the Games
Get the number of Games
For each game, perform one aSync call
per property
Print all the games and details (here I am not able
to get the updated object)
Code:
var games = [];
window.addEventListener('load', function() {
// Check the Network and assign the smart contract address
web3.eth.net.getId()
.then(function(networkId) {
let contractAddressRegistry;
if (networkId == 1) {
contractAddressRegistry = "0xQWERTYUIOPQWERTYUIOPQWERTY"
} else {
contractAddressRegistry = "0x12345678901234567890123456"
}
return contractAddressRegistry;
})
.then(function(contractAddressRegistry) {
let contractRegistry = new web3.eth.Contract(contractAbiRegistry, contractAddressRegistry);
contractRegistry.methods.numberOfGames().call()
.then(function(numberOfGames) {
for (let i = 0; i < numberOfGames; i++) {
let game = {};
game.propertyA = aSyncCallGetPropertyA(i); // Promise
game.propertyB = aSyncCallGetPropertyB(i); // Promise
game.propertyC = aSyncCallGetPropertyC(i); // Promise
}
games.push(game);
})
})
.then(function() {
console.log(games) // Empty
})
})
I tried used Promises.all() but I am not able to sync it properly as some async calls are within a then().
How can I make sure to get the object Games filled with all its properties?

You should use Promise.all like this. Basically, you need to wrap all three aSyncCallGetProperty async calls in Promise.all for waiting until they really finish then assign the results to object game.
whatever
.then(function(contractAddressRegistry) {
let contractRegistry = new web3.eth.Contract(contractAbiRegistry, contractAddressRegistry);
return contractRegistry.methods.numberOfGames().call();
})
.then(function(numberOfGames) {
return Promise.all(numberOfGames.map(() => {
return Promise.all([
aSyncCallGetPropertyA(),
aSyncCallGetPropertyB(),
aSyncCallGetPropertyC()
]).then(results => {
let game = {};
game.propertyA = results[0];
game.propertyB = results[1];
game.propertyC = results[2];
return game;
});
}));
})
.then(function(games) {
console.log(JSON.stringify(games));
})

#Lewis' code seems right but I can not make sure what numberOfGames is. Assuming that it's an integer as used in your question (not an array as treated in the other answer) here is a further rephrased version without nested .then()s.
window.addEventListener('load', function() {
web3.eth.net.getId()
.then(networkId => networkId === 1 ? "0xQWERTYUIOPQWERTYUIOPQWERTY"
: "0x12345678901234567890123456")
.then(contractAddressRegistry => new web3.eth.Contract(contractAbiRegistry, contractAddressRegistry).methods.numberOfGames().call())
.then(numberOfGames => Promise.all(Array(numberOfGames).fill()
.map(_ => Promise.all([aSyncCallGetPropertyA(),
aSyncCallGetPropertyB(),
aSyncCallGetPropertyC()]))))
.then(function(games){
games = games.map(game => ({propertyA: game[0],
propertyB: game[1],
propertyC: game[2]}));
doSomethingWith(games);
});
});