Develop Reference

How to replace multiple async/await calls with Promise.all?

I have the following try/catch block which is making 3 different api calls.
The following code is working fine but it is taking lot of time to execute when firstData has large dataset.
try {
const firstData = await myservice1.myservice1Func();
for(let i=0; i<firstData.total; i++){
const hostName = firstData.rows[i]['hostname'];
if (hostName !== null && firstData.rows[i]['myservice1Id'] !== null) {
const aRes = await myService2(hostName);
firstData.rows[i]['mylist'] =
aRes[0].dataValues;
}
if (hostName !== null && firstData.rows[i]['type'].includes('type1')) {
const oRes = await myService3(hostName);
firstData.rows[i]['ores'] = oRes.rows[0];
}
if (hostName !== null && firstData.rows[i]['type'].includes('type2')) {
const vRes = await myService4(hostName);
firstData.rows[i]['vRes'] = vRes.rows[0];
}
}
return firstData;
} catch (err) {
console.log(err);
}
Here,
const firstData =
{
"total": 2,
"rows": [
{
"hostname": "abc.com",
"ipAddress": "11.11.11.11",
"myservice1Id": "ee0f77c9-ef15",
"type": "type1"
},
{
"hostname": "cde.com",
"ipAddress": "12.12.12.12",
"type": "type2",
"myservice1Id": null
}
]
}
const aRes =
[
{
"listType": "list1",
"createdAt": "2020-12-07"
}
]
const oRes =
{
"rows": [
{
"status": "FAIL"
}
]
}
const vRes =
{
"rows": [
{
"status": "FAIL"
}
]
}
The final value of firstData returned is as following:
{
"total": 2,
"rows": [
{
"hostname": "abc.com",
"ipAddress": "11.11.11.11",
"myservice1Id": "ee0f77c9-ef15",
"type": "type1",
"oRes": {
"status": "PASS"
},
"mylist": {
"listType": "list1",
"createdAt": "2020-12-07"
}
},
{
"hostname": "cde.com",
"ipAddress": "12.12.12.12",
"type": "type2",
"myservice1Id": null,
"vRes": {
"status": "FAIL"
}
}
]
}
Here, one thing to notice is that all the 3 if blocks can be executed in parallel because they are independent of each other.
Can I use Promise.all to execute all the 3 if blocks in parallel?
If yes, how the updated code will look like using Promise.all?

Simplest tweak would be to push each Promise to an array inside the ifs:
const proms = [];
if (hostName !== null && firstData.rows[i].myservice1Id !== null) {
proms.push(
myService2(hostName)
.then(aRes => firstData.rows[i].mylist = aRes[0].dataValues)
);
}
// other ifs changed around the same way
await Promise.all(proms);
You could also make the code easier by making the hostName check only once, and it looks like you're iterating over the whole array, which can be done more easily by invoking the iterator:
try {
const firstData = await myservice1.myservice1Func();
for (const row of firstData.rows) {
const hostName = row.hostname;
if (hostName === null) continue;
const proms = [];
if (row.myservice1Id !== null) {
proms.push(
myService2(hostName)
.then(aRes => row.mylist = aRes[0].dataValues)
);
}
// etc

Hi you have bit of code alterations,
for(let i=0; i<firstData.total; i++){
const hostName = firstData.rows[i]['hostname'];
//check if condition inside the service and return a null (a promise)
Promise.all([myService2(hostName), myService3(hostName), myService4(hostName)]).then((values) => {
console.log(values);
//[resutl1,null,result3]
});
}
Now the problem here is you have to wait until the slowest iteration to complete,
You can fix that with promise pool use,
#supercharge/promise-pool
MDN Promise Medium Blog Source

API Call Before Next Iteration Starts in Loop

I would like to send a POST request to a certain app through their API. What I am trying to do is to process the input data (called data) and send a POST request on one record by one record in the loop. Then, I delete the corresponding object in data for optimization purpose. I know that because of the asynchronous feature of JavaScript, the loop finishes before the function gets called. However, even though I wrap the api function in IIFE or wrap it in an async function with await(the code is below), the compiler still gives me function calls with the same parameter which is the last object. So, when I see created records on the app, David's information was generated three times. The screenshot below is each record object after being processed. If you could tell me ways of triggering the api call before the next iteration in the loop, that would be greatly appreciated.
const obj = [];
var record = {};
var data = [
{
"userId": "123",
"name": "John",
"phoneNumber": "123-456-6789"
},
{
"userId": "345",
"name": "Summer",
"phoneNumber": "535-631-9742"
},
{
"userId" : "789",
"name": "David",
"phoneNumber": "633-753-1352"
}
]
var dataLen = data.length;
var people = data;
createKeyValue = ((key, value) => {
var temp = {};
temp["value"] = value;
obj[key] = temp;
});
apiCall = ((record) => {
clientInformation.record.addRecord.then((resp) => {
console.log(resp);
}).catch((err) => {
console.log(err);
});
});
async function asyncFunction(record) {
let promise = new Promise((resolve, reject) => {
setTimeout(() => apiCall(record), 1000)
});
let result = await promise;
console.log(result);
}
while (dataLen > 0) {
for (let [key, value] of Object.entries(data[0])) {
switch(key) {
case 'userId':
createKeyValue(key, value);
break;
case 'name':
createKeyValue(key, value);
break;
default:
}
}
record["record"] = obj;
asyncFunction(record);
data.shift();
dataLen -= 1;
}
Here is the screenshot of how each processed data looks like.

I think you haven't understand how the for loop inside the while works. The data should be incremented each time to get the next array inside data.
The data[0] => { userId: 123 ... }, data[1] => { userId: 345 ... } and so on .
At each for loop iteration checks the 3 elements of each sub array, so each time temp stores the key values for userId and name. So when the loop finishes, the temp contains as key => userId, name and the corresponding values.
var data = [
{
"userId": "123",
"name": "John",
"phoneNumber": "123-456-6789"
},
{
"userId": "345",
"name": "Summer",
"phoneNumber": "535-631-9742"
},
{
"userId" : "789",
"name": "David",
"phoneNumber": "633-753-1352"
}
]
var dataLen = data.length;
let i = 0 ;
while ( i < dataLen) {
let temp = [];
for (let [key, value] of Object.entries(data[i])) {
if(key == 'userId' || key == 'name'){
temp[key] = value;
}
}
//Just to print the values and understand
for(let k in temp){
console.log(k+" -> "+temp[k]);
}
//here you will pass the temp values to functions
console.log(" At each iteration execute the required functions ");
//asyncFunction(temp);
i += 1;
}

How to parse JSON having nested arrays in javascript or jquery

I want to parse JSON like below
{
"nodeId":3892718504,
"root":true,
"subs":[
{
"nodeId":3892717286
},
{
"nodeId":3892716092,
"subs":[
{
"nodeId":3892715856,
"subs":[
{
"nodeId":3892718592,
"subs":[
{
"nodeId":3892717580
}
]
}
]
}
]
},
{
"nodeId":3892717497
}
]
}
Each node can have subs and those subs can have nodes that can have their own subs. all I want is an array having all nodeId, how can I parse this JSON such that an array called nodes_list is populated with all nodeId.
I can use javascript or jquery.
I'm trying the following approach to get an array of nodeId
jQuery.each(response.topology, function(i,obj) {
if(i == "nodeId") {
node_list.push(obj)
}
if(i == "subs"){
jQuery.each(i, function(key,value) {
if(i == "nodeId") {
node_list.push(obj)
}
}
}
});
I just need a little hint on how it can be in an iterative manner.

This can be done with function generators.
Perhaps not the most enjoyable approach, but I'm pretty sure the other solutions will already imply using other ways, so here is a solution using generators.
PS: Beware of browser support: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/yield
const input = {
"nodeId":3892718504,
"root":true,
"subs":[
{
"nodeId":3892717286
},
{
"nodeId":3892716092,
"subs":[
{
"nodeId":3892715856,
"subs":[
{
"nodeId":3892718592,
"subs":[
{
"nodeId":3892717580
}
]
}
]
}
]
},
{
"nodeId":3892717497
}
]
};
function* nodeLookup(obj) {
if (obj.nodeId) yield obj.nodeId;
if (obj.subs) for (var i = 0; i < obj.subs.length; i++) yield *nodeLookup(obj.subs[i]);
};
const node_ids = [...nodeLookup(input)];
console.log(node_ids);

Just use recursion to iterate over subs
var nodeIds = [];
if (data.nodeId) nodeIds.push(data.nodeId);
function fetchNodeIds (subs) {
if (!subs.length) return cb([]);
var abc = [];
subs.forEach(function (sub) {
abc.push(sub.nodeId);
if (sub.subs && sub.subs.length) abc = abc.concat(fetchNodeIds(sub.subs))
});
return abc;
}
nodeIds = nodeIds.concat(fetchNodeIds(data.subs));
console.log('--All nodeIds--', nodeIds)

It's straightforward to do recursively:
const gatherIds = ({nodeId, subs}, results = []) => subs
? [...results, nodeId, ...(subs .flatMap (sub => gatherIds (sub, results) ))]
: [...results, nodeId]
const response = {"nodeId": 3892718504, "root": true, "subs": [{"nodeId": 3892717286}, {"nodeId": 3892716092, "subs": [{"nodeId": 3892715856, "subs": [{"nodeId": 3892718592, "subs": [{"nodeId": 3892717580}]}]}]}, {"nodeId": 3892717497}]}
console .log (
gatherIds (response)
)
If your target environments don't support flatmap, it's easy enough to shim.

How to modularise function in Chakram to get values

I have an API that returns data in this format -
{
"data": [
{
"id": 121,
"id_type": "some string",
"id_value": "test",
"attribute1": {
"attr_id": 140,
"attribute_client_id": null,
},
"attribute2": {
"attr2_id": 143,
"attribute2_client_id": null,
},
"status": "some string",
"person_name": "James Allen",
"friends": [
{
"friend_id": 1,
"data_id": null,
},
{
"friend_id": 2,
"data_id":null
}
],
"text_description": "Some string",
"text_format": [
"something",
"else"
],
"job_description": "new string",
"is_member": false,
"is_external": false
},
....
]
}
I want to have a function that calculates if of array with is_member is true.
I can do this in the code itself using the filter function with something like this - I am using Chakram library to hit the API end points.
describe('Check if is member is true',()=>{
it('get data',()=>{
let length_of_arr
return response.then((resp)=>{
let length_of_arr = resp.body.data;
length_of_arr.length= Object.keys(length_of_arr).length;
console.log(length_of_arr);
let new_arr = Array.from(length_of_arr);
let r = new_arr.filter(({is_member})=>is_member === true);
console.log(r.length);
expect(r.length).to.be.greater.than(0);
}) ;
});
This works perfectly fine and I am able to get the correct results. However, I need to use this same test for the same API at other places too. So I wanted to have a function which can do it.
In the root directory, I created a file custom_functions.js, which has code like
module.exports = {
get_member_details(resp,data,attr){
let length_of_arr;
let length_of_arr = resp.body.data;
length_of_arr.length= Object.keys(length_of_arr).length;
console.log(length_of_arr);
let new_arr = Array.from(length_of_arr);
let r = new_arr.filter(({attr})=>attr === true);
console.log(r.length);
}
}
However, this is not correct and it gives error that data is not defined. How can I achieve this kind of modularisation when using Javascript. I would also welcome if there are suggestions to improve how to approach this problem as well.

As i understand you want to define a function that you can call it in many tests:
//custom_functions.js
function has_member(data){
return data.filter(res => res.is_member).length > 0;
}
module.exports = {
has_member,
}
// in your test you can call this function like this :
const { has_member } require ('./custom_functions');
describe('Check if is member is true',()=>{
it('get data',() => {
return response.then((resp)=>{
const data = resp.body.data;
const has_member = has_member(data);
expect(has_member).to.be.true;
});
});

Javascript (mongojs): Return a value in a function

I'm working with mongojs and I have to retrieve a field from an object taken from mongodb.
I can not understand how to return the field:
function retrieveVertById(id){
var result = [];
db.clusters.find({id: id}, function (err, clusters){
if( err || !clusters) console.log("No cluster found");
else clusters.forEach( function (cluster) {
vert = cluster["vertices"];
result.push(vert);
console.log(result);
});
})
return result;
};
var a = retrieveVertById("001");
console.log(a);
The print inside the 'forEach' prints the correct value:
(ex. [ [ [ 8, 2, 2 ], [ 2, 2, 5 ], [ 2, 2, 2 ], [ 5, 2, 2 ] ] ] )
On the contrary the print outside the cycle shows an empty array.
what does not work with the return?
Thanks in advance for your help.

I've not used mongojs, but any db lookup will almost certainly be asynchronous. This means the function you passed to db.clusters.find will not run immediately, but rather when the asynchronous call returns from mongo. Instead of returning a value from retrieveVertById, try a callback function instead:
function retrieveVertById(id, successCallback) {
db.clusters.find({
id: id
}, function (err, clusters) {
if (err || !clusters) {
console.log("No cluster found");
} else {
var result = [];
clusters.forEach(function (cluster) {
vert = cluster["vertices"];
result.push(vert);
});
successCallback(result);
}
});
};
retrieveVertById("001", function(result) {
console.log(result);
});

oh, i see... you should remember that javascript is async language
return result;
after forEach() will not return result from inside forEach(). you should send result after last value parsed.
var i = 0;
clusters.forEach( function (cluster) {
vert = cluster["vertices"];
result.push(vert);
if (i >= clusters.length)
return result;
i++;
});