Develop Reference

Run a function in a loop until

I'm currently trying to loop running a function.
Can't figure it out and here's what I tried:
do {
queryLastCursor(lastCursor).then(lastCursorResults => {
if (lastCursorResults.hasNext = false) {
hasNextPage = false;
}
console.log(hasNextPage);
})
} while (hasNextPage);
queryLastCursor is a method with a call to an API. When it returns the data it would have a value of hasNext if it returns false then I'd like to set hasNextPage to false. The expected behavior would be that it runs the function again and again until we get the result hasNext = false. Any idea of what am I doing wrong?

If you want to do an async process in a loop, I suggest doing it recursively:
const runQuery = () => {
queryLastCursor(lastCursor)
.then(result => {
if (result.hasNext) {
// recursively call itself if hasNext is true
runQuery();
}
});
}
runQuery();
Assuming you'd want to return some data, you can do:
const runQuery = async (data) => {
return queryLastCursor(lastCursor)
.then(result => {
if (!data) {
data = [];
}
// assuming you are returning the data on result.data
data.push(result.data);
if (result.hasNext) {
// recursively call itself if hasNext is true
return runQuery(data);
}
retun data;
});
}
runQuery()
.then(data => {
// data should be an array of all the data now
});

I would do something like that:
const callApi = async () => {
let result = await someMagic();
while (result.hasNext) {
doSomethingwith(result);
result = await someMagic();
}
return result;
};
const myResult = await callApi();
Though this seems risky, what is we always get a hastNext = true ? Some security seems good, like a limit of loops in the while loop.

How to change Promise.all to send requests one by one

I have a an array of chunked data that I need to upload one chunk at time. The current implementation I used it to encapsulate the logic in an Promise.all() since I need to return the result of the promise,
The problem with this approach is that all the upload is done asynchronously resulting in a Timeout error as the server can't process all the requests at the same time, How can I modify this method so that the upload is done one chunk at time ?.
My code:
var chunks = _.chunk(variableRecords, 30);
return Promise.all(
chunks.map(chunk => this.portalService.updateDataForChart(variableId, chunk)))
.then((updateRes: boolean[]) => {
if (updateRes.every(updateStatus => updateStatus)) {
return this.executeRequest<HealthDataSource, boolean>({
path: `/variable/user/datasources/${dataSource.identifier}`,
method: 'PUT',
body: {
libelle: dataSource.datasource.libelle,
type: dataSource.datasource.type,
lastSyncDate: Math.max(maxDate, dataSource.datasource.lastSyncDate)
},
headers: this.getHeaders()
});
} else {
return false;
}
});

You need them in SEQUENCE , for of is the way to go :
async function chunksSequence(chunks) {
for(const chunk of chunks) {
await // your other code here
}
};
If you need to return something
async function chunksSequence(chunks) {
let results = []
for(const chunk of chunks) {
let result = await // your other code here
results.push(result)
}
return results
};
Because of comment needed in a promise on return
async function chunksSequence(chunks) {
return new Promise((resolve, reject)=>{
let results = []
for(const chunk of chunks) {
let result = await // your other code here
results.push(result)
}
resolve(results)
}
};

You can do this with the help of Array.reduce()
const chunks = _.chunk(variableRecords, 30);
return tasks.reduce((promiseChain, currentTask) => {
return promiseChain.then(chainResults =>
currentTask.then(currentResult =>
[ ...chainResults, currentResult ]
)
);
}, Promise.resolve([])).then(arrayOfResults => {
// Do something with all results
});
Source : https://decembersoft.com/posts/promises-in-serial-with-array-reduce/

If you don't / can't use await you could use something like this
function runSequenceItem(chunks, index) {
return new Promise(chunks[index])
.then(res => {
index ++
if (index < chunks.length) {
return runSequence(chunks[index], index + 1)
} else {
// this is not needed actually
return 'done'
}
})
}
function runInSequence(chunks) {
return runSequenceItem(chunks, 0)
}
If you also need the results then you can return an array at the end of the recursion runInSequence
function runSequenceItem(chunks, index, results) {
return new Promise(chunks[index])
.then(res => {
results.push(res)
index ++
if (index < chunks.length) {
return runSequence(chunks[index], index + 1)
} else {
return results
}
})
}
function runInSequence(chunks) {
return runSequenceItem(chunks, 0, [])
}
and then retrieve it at the end
let results = runInSequence(chunks)

function inside function is not waiting for promise in javascript

Sorry if my title is not very explicit I dont know how to explain this properly.
I am trying to use the distinct function for my app that uses loopback 3 and mongodb. It seems to work right but my endpoint wont hit the return inside my function.
This is my code
const distinctUsers = await sellerCollection.distinct('userId',{
hostId : host.id,
eventId:{
"$ne" : eventId
}
}, async function (err, userIds) {;
if(!userIds || userIds.length ==0)
return [];
const filter = {
where:{
id: {
inq: userIds
}
}
};
console.log("should be last")
return await BPUser.find(filter);
});
console.log(distinctUsers);
console.log("wtf??");
//return [];
If I uncomment the return [] it will send the return and later it will show the should be last, so even when I dont have the return it seems to finish. It is now waiting for the response. I dont like the way my code looks so any pointer of how to make this look better I will take it.

It looks like the sellerCollection.distinct takes a callback as one of it's parameters, therefore, you cannot use async/await with a callback-style function, since it's not a promise.
I would suggest turning this call into a promise if you'd like to use async/await:
function findDistinct(hostId, eventId) {
return new Promise((resolve, reject) => {
sellerCollection.distinct(
'userId',
{ hostId, eventId: { "$ne": eventId } },
function (error, userIds) {
if (error) {
reject(error);
return;
}
if (!userIds || userIds.length === 0) {
resolve([]);
return;
}
resolve(userIds);
}
)
})
}
Then, you can use this new function with async/await like such:
async function getDistinctUsers() {
try {
const hostId = ...
const eventId = ...
const distinctUsers = await findDistinct(hostId, eventId)
if (distinctUsers.length === 0) {
return
}
const filter = {
where: {
id: { inq: userIds }
}
}
const bpUsers = await BPUser.find(filter) // assuming it's a promise
console.log(bpUsers)
} catch (error) {
// handle error
}
}

Calling async function multiple times

So I have a method, which I want to call multiple times in a loop. This is the function:
function PageSpeedCall(callback) {
var pagespeedCall = `https://www.googleapis.com/pagespeedonline/v4/runPagespeed?url=https://${websites[0]}&strategy=mobile&key=${keys.pageSpeed}`;
// second call
var results = '';
https.get(pagespeedCall, resource => {
resource.setEncoding('utf8');
resource.on('data', data => {
results += data;
});
resource.on('end', () => {
callback(null, results);
});
resource.on('error', err => {
callback(err);
});
});
// callback(null, );
}
As you see this is an async function that calls the PageSpeed API. It then gets the response thanks to the callback and renders it in the view. Now how do I get this to be work in a for/while loop? For example
function PageSpeedCall(websites, i, callback) {
var pagespeedCall = `https://www.googleapis.com/pagespeedonline/v4/runPagespeed?url=https://${websites[i]}&strategy=mobile&key=${keys.pageSpeed}`;
// second call
var results = '';
https.get(pagespeedCall, resource => {
resource.setEncoding('utf8');
resource.on('data', data => {
results += data;
});
resource.on('end', () => {
callback(null, results);
});
resource.on('error', err => {
callback(err);
});
});
// callback(null, );
}
var websites = ['google.com','facebook.com','stackoverflow.com'];
for (let i = 0; i < websites.length; i++) {
PageSpeedCall(websites, i);
}
I want to get a raport for each of these sites. The length of the array will change depending on what the user does.
I am using async.parallel to call the functions like this:
let freeReportCalls = [PageSpeedCall, MozCall, AlexaCall];
async.parallel(freeReportCalls, (err, results) => {
if (err) {
console.log(err);
} else {
res.render('reports/report', {
title: 'Report',
// bw: JSON.parse(results[0]),
ps: JSON.parse(results[0]),
moz: JSON.parse(results[1]),
// pst: results[0],
// mozt: results[1],
// bw: results[1],
al: JSON.parse(results[2]),
user: req.user,
});
}
});
I tried to use promise chaining, but for some reason I cannot put it together in my head. This is my attempt.
return Promise.all([PageSpeedCall,MozCall,AlexaCall]).then(([ps,mz,al]) => {
if (awaiting != null)
var areAwaiting = true;
res.render('admin/', {
title: 'Report',
// bw: JSON.parse(results[0]),
ps: JSON.parse(results[0]),
moz: JSON.parse(results[1]),
// pst: results[0],
// mozt: results[1],
// bw: results[1],
al: JSON.parse(results[2]),
user: req.user,
});
}).catch(e => {
console.error(e)
});
I tried doing this:
return Promise.all([for(let i = 0;i < websites.length;i++){PageSpeedCall(websites, i)}, MozCall, AlexaCall]).
then(([ps, mz, al]) => {
if (awaiting != null)
var areAwaiting = true;
res.render('admin/', {
title: 'Report',
// bw: JSON.parse(results[0]),
ps: JSON.parse(results[0]),
moz: JSON.parse(results[1]),
// pst: results[0],
// mozt: results[1],
// bw: results[1],
al: JSON.parse(results[2]),
user: req.user,
});
}).catch(e => {
console.error(e)
});
But node just said it's stupid.
And this would work if I didn't want to pass the websites and the iterator into the functions. Any idea how to solve this?
To recap. So far the functions work for single websites. I'd like them to work for an array of websites.
I'm basically not sure how to call them, and how to return the responses.

It's much easier if you use fetch and async/await
const fetch = require('node-fetch');
async function PageSpeedCall(website) {
const pagespeedCall = `https://www.googleapis.com/pagespeedonline/v4/runPagespeed?url=https://${website}&strategy=mobile&key=${keys.pageSpeed}`;
const result = await fetch(pagespeeddCall);
return await result.json();
}
async function callAllSites (websites) {
const results = [];
for (const website of websites) {
results.push(await PageSpeedCall(website));
}
return results;
}
callAllSites(['google.com','facebook.com','stackoverflow.com'])
.then(results => console.log(results))
.error(error => console.error(error));
Which is better with a Promise.all
async function callAllSites (websites) {
return await Promise.all(websites.map(website => PageSpeedCall(website));
}

Starting on Node 7.5.0 you can use native async/await:
async function PageSpeedCall(website) {
var pagespeedCall = `https://www.googleapis.com/pagespeedonline/v4/runPagespeed?url=https://${website}&strategy=mobile&key=${keys.pageSpeed}`;
return await promisify(pagespeedCall);
}
async function getResults(){
const websites = ['google.com','facebook.com','stackoverflow.com'];
return websites.map(website => {
try {
return await PageSpeedCall(website);
}
catch (ex) {
// handle exception
}
})
}
Node http "callback" to promise function:
function promisify(url) {
// return new pending promise
return new Promise((resolve, reject) => {
// select http or https module, depending on reqested url
const lib = url.startsWith('https') ? require('https') : require('http');
const request = lib.get(url, (response) => {
// handle http errors
if (response.statusCode < 200 || response.statusCode > 299) {
reject(new Error('Failed to load page, status code: ' + response.statusCode));
}
// temporary data holder
const body = [];
// on every content chunk, push it to the data array
response.on('data', (chunk) => body.push(chunk));
// we are done, resolve promise with those joined chunks
response.on('end', () => resolve(body.join('')));
});
// handle connection errors of the request
request.on('error', (err) => reject(err))
})
}

Make PageSpeedCall a promise and push that promise to an array as many times as you need, e.g. myArray.push(PageSpeedCall(foo)) then myArray.push(PageSpeedCall(foo2)) and so on. Then you Promise.all the array.
If subsequent asynch calls require the result of a prior asynch call, that is what .then is for.
Promise.all()
Promise.all([promise1, promise2, promise3]).then(function(values) {
console.log(values);
});

Promises are not behaving as I expect them to

I'm using Express for routing and Sequelize for DB management.
app.get('/api/users/:username', (req, res) => {
let username = req.params.username;
findChattersPerRole()
.then(chattersPerRole => {
console.log('instakbot should\'ve been added by now...');
});
});
The function findChattersPerRole returns an object with each user's username and role as another object.
const findChattersPerRole = () => {
return fetch('https://tmi.twitch.tv/group/user/instak/chatters')
.then(parseJSON)
.then(r => {
let chatters = r.chatters;
let chattersPerRole = Object.keys(chatters).map(role => {
return chatters[role].map(username => {
console.log('findOrCreateViewer will be executed after this');
findOrCreateViewer(username, role);
return {
username: username,
role: role
};
});
});
return Promise.resolve(flattenDeep(chattersPerRole));
}).catch(err => {
console.log(`Error in fetch: ${err}`);
});
};
The problem is, in my route, I expect the console.log('instakbot should\'ve been added by now...'); to be executed AFTER my viewers got inserted into the database because in my function findChattersPerRole I already insert them with the function findOrCreateViewer. I expect this to happen because in my route I write the console.log when findChattersPerRole() is resolved...
const findOrCreateViewer = (username, role) => {
return Viewer.findOrCreate({
where: {
username
},
defaults: {
instakluiten: 5,
role
}
}).spread((unit, created) => {
console.log('unit is: ', unit.dataValues.username);
if(created){
return `created is ${created}`;
}else{
return unit;
}
});
};
However, in my terminal you can see that this is not the way it's happening... Why aren't my promises being executed at the expected time?
Screenshot of my terminal

The return {username: ...} after findOrCreateViewer(username, role); happens immediately after the function is called and before any data has been inserted. That also means that return Promise.resolve(flattenDeep(chattersPerRole)); happens before any data has been inserted, etc.
You said findOrCreateViewer returns a promise, so you need to wait until that promise is resolved (i.e. wait until after the data was inserted) before continuing with something else.
You want chattersPerRole to be an array of (arrays of) promises and only proceed after all the promises are resolved.
This is easy to do with Promise.all:
const findChattersPerRole = () => {
return fetch('https://tmi.twitch.tv/group/user/instak/chatters')
.then(parseJSON)
.then(r => {
let chatters = r.chatters;
let chattersPerRole = Object.keys(chatters).map(
role => chatters[role].map(username => {
console.log('findOrCreateViewer will be executed after this');
return findOrCreateViewer(username, role).then(
() => ({username, role})
);
});
);
return Promise.all(flattenDeep(chattersPerRole));
}).catch(err => {
console.log(`Error in fetch: ${err}`);
});
};
Now the promise returned by findChattersPerRole will be resolved after all the promises returned by findOrCreateViewer are resolved.

Promises are doing no magic. Returning a promise doesn't mean that calling the function will block, but rather that you can easily chain callbacks to do something with the result. You'll need to use
function findChattersPerRole() {
return fetch('https://tmi.twitch.tv/group/user/instak/chatters')
.then(parseJSON)
.then(r => {
let chatters = r.chatters;
let chattersPerRole = Object.keys(chatters).map(role => {
return chatters[role].map(username => {
console.log('findOrCreateViewer will be executed after this');
return findOrCreateViewer(username, role).then(() => {
// ^^^^^^ ^^^^^
return {
username: username,
role: role
};
});
});
});
return Promise.all(flattenDeep(chattersPerRole));
// ^^^ get a promise for an array of results from an array of promises
}).catch(err => {
console.log(`Error in fetch: ${err}`);
});
}