Develop Reference

Why does this recursive function not run asynchronously?

I have a start(node, array) function that should perform a DFS by traversing an object tree via recursive calls to an API through callMsGraph(token, end) until image properties are found at the end of the tree, at which point they are pushed to array. The function seems like it works, but I can't get the output unless I wrap it in a 2 second setTimeout which indicates the recursion is not being waited on to complete. I would want to play around with async/await more, but it's not at the top-level.
I'm not sure if the nextNode.then is doing anything or maybe callMsGraph() needs to be awaited on differently to how I know. A solution would be much appreciated.
shelfdb.data = async (accessToken) => {
const token = accessToken;
const endpoint = 'https://graph.microsoft.com/v1.0/sites/webgroup.sharepoint.com,23e7ef7a-a529-4dde-81ba-67afb4f44401,0fa8e0f7-1c76-4ad0-9b6e-a485f9bfd63c/drive/items/01GNYB5KPQ57RHLPZCJFE2QMVKT5U3NYY3/children'
function start(node, array) {
if(node.value.length > 0) {
node.value.forEach(function(child) {
var end = 'https://graph.microsoft.com/v1.0/sites/webgroup.sharepoint.com,23e7ef7a-a529-4dde-81ba-67afb4f44401,0fa8e0f7-1c76-4ad0-9b6e-a485f9bfd63c/drive/items/' + child.id + '/children';
var nextNode = callMsGraph(token, end);
nextNode.then(function(currResult) {
if (currResult.value.length > 0) {
if ('image' in currResult.value[0]) {
currResult.value.forEach(function(imgChild) {
let img = {
'name': imgChild.name,
'job': imgChild.parentReference.path.split("/")[6],
'path': imgChild.webUrl,
'id': imgChild.id
}
array.push(img);
})
// complete storing images at tail object, go one level up after loop
return;
}
// if no 'image' or value, go into child
start(currResult, array);
}
}).catch(function(e) {
console.error(e.message);
})
})
}
return array;
}
var res = await callMsGraph(token, endpoint); // start recursion
var output = start(res, []);
console.log(output); // only displays value if wrapped in setTimeout
return output; // empty []
}
Each query to the API via callMsGraph(), returns an object like this, where subsequent queries are made with the id of each object/folder (as new endpoint) in value until an object with image property is found. The MS Graph API requires that folders are expanded at each level to access their children.
{
id: '01GNYB5KPQ57RHLPZCJFE2QMVKT5U3NYY3'
value: [
{
id: '01GNYB5KJMH5T4GXADUVFZRSITWZWNQROS',
name: 'Folder1',
},
{
id: '01GNYB5KMJKILOFDZ6PZBZYMXY4BGOI463',
name: 'Folder2',
}
]
}
This is the callMsGraph() helper:
function callMsGraph(accessToken, graphEndpoint) {
const headers = new Headers();
const bearer = `Bearer ${accessToken}`;
headers.append("Authorization", bearer);
const options = {
method: "GET",
headers: headers
};
return fetch(graphEndpoint, options)
.then(response => response.json())
.catch(error => {
console.log(error);
throw error;
});
}

The rule with promises is that once you opt into one (more likely, are forced into it by a library), all code that needs to block for a result anywhere after it also has to await. You can't "go back" to sync and if even a single piece of the promise chain between where the promise starts and where you want its result isn't awaited, the result will be unreachable*.
Taking a snippet of the code:
function start(node, array) { // not async!
// ..
node.value.forEach(function(child) { // doesn't await!
// ..
nextNode.then(function(currResult) {
// this promise is not hooked up to anything!
start(...) // recurse without await!
There's no await in front of then, start doesn't return a promise and isn't awaited recursively, and forEach has no way to await its callback's asynchronous results, so each promise in the nextNode.then chain is orphaned into the void forever*.
The solution is a structure like this:
async function start(node, array) {
// ..
for (const child of node.value) {
// ..
const currResult = await callMsGraph(token, end);
// ..
await start(...);
array.push(currResult);
}
// returns a promise implicitly
}
// ..
await start(...);
// `array` is populated here
Or Promise.all, which runs in parallel and returns an array (which could replace the parameter array):
function start(node, array) {
return Promise.all(node.value.map(async child => {
const currResult = await callMsGraph(token, end);
// ..
await start(...);
return currResult;
}));
}
I'd be happy to provide a minimal, runnable example, but the code you've provided isn't runnable, so you'll have to massage this a bit to work for you. If you make sure to await everything, you're good to go (and generally avoid mixing .then and async/await--the latter seems easier for this use case).
* (for all practical intents and purposes)

There is a few places where you are not handling promises returned in you code. nextNode.then if your forEach loop is just "called", next line of the code will not wait for it to complete, forEach loop will complete execution before then callbacks are called.
I changed you code a bit, but I have no way to check if it works correctly due to i would need to populate dummy data for callMsGraph but if you encounter any - tell me and I'll modify the answer
shelfdb.data = async (accessToken) => {
const token = accessToken;
const endpoint = 'https://graph.microsoft.com/v1.0/sites/webgroup.sharepoint.com,23e7ef7a-a529-4dde-81ba-67afb4f44401,0fa8e0f7-1c76-4ad0-9b6e-a485f9bfd63c/drive/items/01GNYB5KPQ57RHLPZCJFE2QMVKT5U3NYY3/children'
const images = [];
async function start(node, array) {
if (node.value.length <= 0) return array; // or === 0 or whatever
for (const child of node.value) {
const end = `https://graph.microsoft.com/v1.0/sites/webgroup.sharepoint.com,23e7ef7a-a529-4dde-81ba-67afb4f44401,0fa8e0f7-1c76-4ad0-9b6e-a485f9bfd63c/drive/items/${child.id}/children`;
const nextNode = await callMsGraph(token, end);
if (nextNode.value.length > 0) {
if ('image' in nextNode.value[0]) {
const mapped = nextNode.value.map(imgChild => {
return {
'name': imgChild.name,
'job': imgChild.parentReference.path.split("/")[6],
'path': imgChild.webUrl,
'id': imgChild.id
}
});
array.push(...mapped);
}
// if no 'image' or value, go into child
await start(nextNode, array);
}
}
return array;
}
var res = await callMsGraph(token, endpoint);
var output = await start(res, []);
console.log(output);
return output;
}
Also, please, feel free to add a try{} catch{} blocks in any place you need them, I skipped them

Make Synchronus Api inside for Loop

I have an array of Objects/String.
this.addtenant().subscribe(r => {
let a = ['plant1', 'plant2'];
for (let b of a) {
this.myService.saveAllData(b).subscribe(r => {
console.log("result" r);
});
}
});
and getAllData is in myservice file which returns an Observable.
saveAlldata(b) {
this.http.post(url,b);
}
The problem is since i am using subscribe the call is being asynchronus, i want to have it something like:
first "plant1" post call has to finish, and then plant2 has to be made. In simple words synchronus call.

I think you should use async/await for synchronous calls.
Here is one of the tutorial:
https://www.techiediaries.com/javascript-async-await-tutorial/
A sample code demo would be something like:
responseArr: any[] = [];
func1()
let x = [1,2,3];
func2(x);
}
async func2(arr) { // make the function async
for(let i=0; i<arr.length ; i++){
const response: any = await this.myService(arr[i]); // wait on each service call. This will give synchronous behaviour
this.handleResponse(response); // handle response in a function if there is common functionality in each response
}
reuturn responseArr; // return the final response
}
handleResponse(response) {
responseArr.push(response);
}

I have found a solution by myself. You can use an async await inside a loop, since forEach is not promise aware and cannot be used with Promise. Here is the final code:
this.addtenant().subscribe(async r => {
let a = ['plant1', 'plant2'];
for(let index=0; index=a.length; index++)
await this.myService.saveAllData(b).toPromise().then(r => {
console.log("result" r);
});
}
}
});
As await works with Promise, so toPromise will replace subscribe()

Sequential execution of Promise.all

Hi I need to execute promises one after the other how do I achieve this using promise.all any help would be awesome. Below is the sample of my code I am currently using but it executes parallel so the search will not work properly
public testData: any = (req, res) => {
// This method is called first via API and then promise is triggerd
var body = req.body;
// set up data eg 2 is repeated twice so insert 2, 5 only once into DB
// Assuming we cant control the data and also maybe 3 maybe inside the DB
let arrayOfData = [1,2,3,2,4,5,5];
const promises = arrayOfData.map(this.searchAndInsert.bind(this));
Promise.all(promises)
.then((results) => {
// we only get here if ALL promises fulfill
console.log('Success', results);
res.status(200).json({ "status": 1, "message": "Success data" });
})
.catch((err) => {
// Will catch failure of first failed promise
console.log('Failed:', err);
res.status(200).json({ "status": 0, "message": "Failed data" });
});
}
public searchAndInsert: any = (data) => {
// There are database operations happening here like searching for other
// entries in the JSON and inserting to DB
console.log('Searching and updating', data);
return new Promise((resolve, reject) => {
// This is not an other function its just written her to make code readable
if(dataExistsInDB(data) == true){
resolve(data);
} else {
// This is not an other function its just written her to make code readable
insertIntoDB(data).then() => resolve(data);
}
});
}
I looked up in google and saw the reduce will help I would appreciate any help on how to convert this to reduce or any method you suggest (Concurrency in .map did not work)

the Promises unfortunatelly does not allow any control of their flow. It means -> once you create new Promise, it will be doing its asynchronous parts as they like.
The Promise.all does not change it, its only purpose is that it checks all promises that you put into it and it is resolved once all of them are finished (or one of them fail).
To be able to create and control asynchronous flow, the easiest way is to wrap the creation of Promise into function and create some kind of factory method. Then instead of creating all promises upfront, you just create only one promise when you need it, wait until it is resolved and after it continue in same behaviour.
async function doAllSequentually(fnPromiseArr) {
for (let i=0; i < fnPromiseArr.length; i++) {
const val = await fnPromiseArr[i]();
console.log(val);
}
}
function createFnPromise(val) {
return () => new Promise(resolve => resolve(val));
}
const arr = [];
for (let j=0; j < 10; j++) {
arr.push(createFnPromise(Math.random()));
}
doAllSequentually(arr).then(() => console.log('finished'));
PS: It is also possible without async/await using standard promise-chains, but it requires to be implemented with recursion.

If anyone else cares about ESLint complaining about the use of "for" and the "no await in loop" here is a typescript ESLint friendly version of the above answer:
async function runPromisesSequentially<T>(promises: Array<Promise<T>>):Promise<Array<T>> {
if (promises.length === 0) return [];
const [firstElement, ...rest] = promises;
return [await firstElement, ...(await runPromisesSequentially(rest))];
}
You can then just replace Promise.all by runPromisesSequentially.

#lmX2015's answer is close but it's taking in promises that have already started executing.
A slight modification fixes it
export async function runPromisesSequentially<T>(functions: (() => Promise<T>)[]): Promise<T[]> {
if (functions.length === 0) {
return [];
}
const [first, ...rest] = functions;
return [await first(), ...(await runPromisesSequentially(rest))];
}

Call API untill certain response

I really struggle with async functions in Javasctipt. Here I have async function that calls api and saves the result. It is working fine but I need to make a loop now so this api is called untill certain condition is met. I understand that it needs to be done using await but I just can't figure out how exactly.
I tried setting up if statement and then doing something like "if condition is not met" setTimeout(getResults()); (repeat the call to async function).
async getResults() {
try {
const res = await axios(`https://blablabla`);
this.result = res.data.info;
} catch (error) {
alert(error);
}
}

async getResults() {
try {
let i = 100;
while(i-->=0){
const res = await axios(`https://blablabla`);
this.result = res.data.info;
if(this.result == 'some process finished')
{
return this.result;
}
//else retry
}
} catch (error) {
alert(error);
}
just use some cycle like while(true). And repeat body of cycle until your conditions are met

Async await with a forEach loop still running asynchronously [duplicate]

This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Closed 4 years ago.
First of all I did read through similar questions, and still cannot see where I'm making my mistake.
Here's my code:
async function validateWebsites(website) {
var result = url.parse(`http://${website}`);
console.log(result.hostname);
return await fetch(`http://www.${result.hostname}`)
.then(() => console.log(true))
.catch(() => console.log(false));
}
var wrongWebsites = [];
var wrongWebsites = [];
var i = 0;
websites.forEach(website => {
i++;
if (validateWebsites(website) === false
) {
wrongWebsites.push(i);
}
});
console.log(wrongWebsites);
How it works:
The user passes an array of websites, and I want to validate if they're valid websites, not to waste resources and block other errors. Now to the console:
digitlead.com
google.com
georgiancollege.ca
youtube.com
[]
true
true
true
true
So as you see, it prints out first the websites array, and then the response. So it's still async. How do I make it wait? I changed the loop from a for to forEach as suggested by many posts, I used the await and I am returning a promise. So what else do I have to do?
Edit:
I tried to do this:
async function validateWebsites(website) {
var result = url.parse(`http://${website}`); // TODO figure out if filtering all the subpages is a good idea.
console.log(result.hostname);
return await fetch(`http://www.${result.hostname}`)
.then(()=>console.log(true))
.catch(()=>console.log(false));
}
But it doesn't change anything
I found a function called readFileSync. That's more or less what I'm looking for, but with the ability to call a different website.

Here is how you can get all valid websites.
Problem with your validateWebsites function is that it returns promise wchih is resolving to undefined thanks to promise chaining and your loging
Also using forEach to filter array is unnesesery.
But if you wanted you could do something like this
websites.forEach(async website => {
i++;
if (await validateWebsites(website) === false) { // now value is Boolean instead of Promise
wrongWebsites.push(i);
}
});
Also note that if you use global i with asyncronous functions to keep track of index this can lead to many errors.
However I think this soultion should satisfy you
async function validateWebsites(website) {
var result = url.parse(`http://${website}`)
return fetch(`http://www.${result.hostname}`)
.then(() => true) // async function returns promise
.catch(() => false)
}
const websites = ['digitlead.com',
'google.com',
'georgiancollege.ca',
'youtube.com',
'111.1',
'foobarbaz']
async function filter(array, func) {
const tmp = await Promise.all( // waits for all promises to resolve
array.map(func) // evecutes async function and stores it result in new array then returns array of promises
)
return array.filter((_, i) => tmp[i]) // removes invalid websites
}
const validWebsites = filter(websites, validateWebsites)
validWebsites.then(console.log)
Get the indexes of the non-valid sites
async function filter(array, func) {
const tmp = await Promise.all(array.map(func))
return tmp
.map((x, i) => !x && i) // flip true to false and asign index when x is false
.filter(x => x !== false) // return indexes
}

destoryeris saying you should do something like this:
websites.forEach(async website => {
i++;
if (await validateWebsites(website) === false
) {
wrongWebsites.push(i);
}
});
But that alone is problematic because you have wrap your async functions in a try/catch to handle their errors. So something more like this:
websites.forEach(async website => {
i++;
try {
const validSites = await validateWebsites(website);
if (validSites === false) {
wrongWebsites.push(i);
}
} catch(e) {
// handle e
}
})