Develop Reference

Promise Never Resolves Using Axios

I am using WDIO and defining a customer reporter to integrate with the testrails api. The plan was to use axios to make these requests inside the testing hooks.
Unfortunately, I am unable to get axios to return any valid data on requests. In most cases when we await a response, the thread just stops executing entirely without any logging output. If I jimmy it enough sometimes I can get it to return an unresolved promise, but nothing I can do ultimately resolves the promise.
Also in none of my attempts have the requests been received by testrails (I've tested a few other urls as well, I'm fairly certain the issue is not at the destination).
I've made sure that network access and security are not factors. We have also attempted using both the axios post, and the straight up axios() methods, no luck there.
I'll copy the file below, I've added roughly a dozen attempts/configurations with notes on each as to what we're getting. The meat of the issue is in the addRun() method.
In most cases we never appear to resolve the promise. there is one exception, where we don't interact at all with the response, just log inside the then() statement. If we do that, we can see those logs, but the results of the axios call never take effect (the run is not created in testrails).
const WDIOReporter = require('#wdio/reporter').default
const axios = require('axios').default;
module.exports = class TestrailsReporter extends WDIOReporter{
constructor(options) {
/*
* make reporter to write to the output stream by default
*/
options = Object.assign(options, { stdout: true })
super(options)
}
// I have tried marking this as both async and not, no difference
async onRunnerEnd(suite) {
console.log("CHECKPOINT RUNNER END")
this.recordResults(caseIds[5], results[5], 'renters api tests', 5);
}
/**
* takes the results from a test suite and records them in testrails
* #param suiteId -- the suite defined in the testrails project
* #param projectId -- the project id defined in the testrails project
* #param caseIds -- a list of cases with which to create the test run
* #param results -- a list of case:result pairings
*/
async recordResults(caseIds, results, name, projectId) {
console.log(`CHECKPOINT RECORDING RESULTS ${projectId}`)
let testRun = await this.addRun(results['suiteId'], caseIds['cases'], name, projectId);
testRun.then(console.log)
await this.addResults(testRun, results['cases']);
}
async addRun(suiteId, caseIds, name = '', projectId) {
console.log("CHECKPOINT ADD RUN")
let addRunConfig = {
method: 'post',
url: `https://REDACTED.testrail.io/index.php?/api/v2/add_run/${projectId}`,
headers: {
'Content-Type': 'application/json',
Authorization: token,
Cookie: 'tr_session=041c4349-688f-440a-95a3-afc29d39320a'
},
data: JSON.stringify({
suite_id: suiteId,
include_all: false,
case_ids: caseIds,
name: name
})
};
// let x = axios.get('https://www.google.com/')
// console.log(x)
axios.defaults.timeout = 1000;
// THIS DOES NOT EXECUTE THE CODE INSIDE THE THEN STATEMENT, RETURNS PENDING PROMISE TO RESPONSE
// let response = axios(addRunConfig)
// .then(function (response) {
// console.log("WHAAAT?")
// return response.data.id;
// })
// .catch(function (error) {
// console.log("HELP!")
// console.log(error);
// });
// THIS DOES NOT EXECUTE THE CODE INSIDE THE THEN STATEMENT, NO LOGGING APPEARS AFTER
let response = await axios(addRunConfig)
.then(function (response) {
console.log("WHAAAT?")
return response.data.id;
})
.catch(function (error) {
console.log("HELP!")
console.log(error);
});
// THIS DOES NOT EXECUTE THE CODE INSIDE THE THEN STATEMENT
// await axios.post(`https://REDACTED.testrail.io/index.php?/api/v2/add_run/${projectId}`, addRunConfig)
// .then(
// function (response){
// console.log('WHAAAT?')
// console.log(response)
// console.log('NO WAY?')
// })
// THIS DOES NOT EXECUTE THE CODE INSIDE THE THEN STATEMENT, BUT RETURNS A PENDING PROMISE TO RESPONSE
// let response = axios.post(`https://REDACTED.testrail.io/index.php?/api/v2/add_run/${projectId}`, addRunConfig)
// .then(
// function (run){
// console.log('WHAAAT?')
// console.log(run)
// console.log('NO WAY?')
// })
// THIS DOES NOT EXECUTE THE CODE INSIDE THE THEN STATEMENT, BUT RETURNS A PENDING PROMISE TO RESPONSE
// let response = axios.post(`https://REDACTED.testrail.io/index.php?/api/v2/add_run/${projectId}`, addRunConfig)
// .then(
// function (run){
// console.log('WHAAAT?')
// })
// THIS DOES NOT EXECUTE THE CODE INSIDE THE THEN STATEMENT, BUT RETURNS A PENDING PROMISE TO RESPONSE
// let response = axios.post(`https://REDACTED.testrail.io/index.php?/api/v2/add_run/${projectId}`, addRunConfig)
// .then(run => {
// console.log('WHAAAT?')
// })
// THIS EXECUTES THE CONSOLE.LOG INSIDE THE THEN STATEMENT, BUT NOT AFTER
// let response = await axios.post(`https://REDACTED.testrail.io/index.php?/api/v2/add_run/${projectId}`, addRunConfig)
// .then(console.log('WHAAAT?'))
// THIS EXECUTES THE CONSOLE.LOG INSIDE THE THEN STATEMENT, AND AFTER
// let response = axios.post(`https://REDACTED.testrail.io/index.php?/api/v2/add_run/${projectId}`, addRunConfig)
// .then(console.log('WHAAAT?'))
// EXECUTES THE CONSOLE.LOG INSIDE THE THEN STATEMENT, NOTHING FROM THE CATCH, AND NOTHING AFTER
// const response = await axios(addRunConfig).then(console.log("HI")).catch(function (error) {
// console.log("HELP!")
// console.log(error);
// });
console.log("ANYTHING")
console.log(response)
return response
}```

Figured this out, it was the WDIOReporter parent class not playing nice with the asynchronous calls from axios. Solution found here:
https://github.com/webdriverio/webdriverio/issues/5701

Have you tried calling the method using
await axios.post(...) instead of defining everything in the addRunConfig object?
Not sure if it makes a difference, but it's something to try.

There is some confusion around the concepts of defining functions, calling functions and asynchronous functions here.
First of all: If you are calling an asynchronous function and do not want your calling function to return before that asynchronous function has returned, you want to await that call.
In this case, your recordResults function awaits something, and is thus async. Therefor, you probably want that onRunnerEnd awaits your call to recordResults. If you dont do that, the function will terminate prematurely and likely not wait for a result.
async onRunnerEnd(suite) {
console.log("CHECKPOINT RUNNER END")
await this.recordResults(caseIds[5], results[5], 'renters api tests', 5);
}
Secondly, if you use then and await together, the value returned by the await expression is whatever the function inside the then returns. Thus, all your attempts that have no return value inside the function called by then will never return anything but a void promise. There is no reason to combine these concepts.
Thirdly, putting a function call (rather than a declaration or reference) inside a then clause, will immediately call that function. I.e. .then(console.log('WHAAAT?')) just immediately calls console.log and also registers a non existent function as the callback for then (since console.log does not return a function reference).
Lastly, passing unbound functions is not going to work in general. Doing things like testRun.then(console.log) will not work, depending on the implementation of then and console.log. Either do testRun.then(console.log.bind(console) or testRun.then((x) => console.log(x)) to be on the safe side.
So, first of all, add that await inside onRunnerEnd, and then just use the await result without any then or catch in your addRun:
async addRun(suiteId, caseIds, name = '', projectId) {
console.log("CHECKPOINT ADD RUN")
let addRunConfig = {
method: 'post',
url: `https://REDACTED.testrail.io/index.php?/api/v2/add_run/${projectId}`,
headers: {
'Content-Type': 'application/json',
Authorization: token,
Cookie: 'tr_session=041c4349-688f-440a-95a3-afc29d39320a'
},
data: JSON.stringify({
suite_id: suiteId,
include_all: false,
case_ids: caseIds,
name: name
})
};
// let x = axios.get('https://www.google.com/')
// console.log(x)
axios.defaults.timeout = 1000;
let response = await axios(addRunConfig);
console.log(response);
console.log(response.data.id);
return response.data.id;
}

How can I return multiple function values in a single array

I am crawling 5 different sites for data using node-fetch and cheerio.
everything checks out but I need to collect the returned data from these 5 separate functions in an array.
First I store function name and url for each site in an object like so
url: 'https://sampleurl.com',
crawl: FirstLinkCrawl
}
const secondLink = {
url: 'https://sampleurl.com',
crawl: secondLinkCrawl
}
}```
Then I write the function to crawl each site like so, I ran this function with and without promise, both check out
```const secondLinkCrawl = (body) =>{
return new Promise((res, rej)=>{
"this crawl function is ran here, everything works fine, Data is an object"
const error = false
if(!error){
res(Data)
}else{
rej()
}
})
}```
This here is my fetch function that takes the url and a callback, which is the crawl function
```async function Fetch(url, callback){
const response = await fetch(url)
const html = await response.text()
callback(html)
}
Then I call the fetch and crawl using promise.all() in an attempt to have it return in an array, but in an array,
const promises = [
Fetch(firstLink.url, firstLink.crawl),
Fetch(secondLink.url, secondLink.crawl)]
Promise.all(promises)
.then(values => console.log(values))
.catch(err => console.log(err))
}
When I run this, I get [ undefined, undefined ]
But when I run it without promises and simply log the result, both of them run successfully.
My aim is to get my results in a single array. what can I do?
I also tried declaring an array a the top of the page and then pushing each result into the array, after which I log the array at the bottom of the functions call. But the array returns empty

You're not returning anything from Fetch function that's why it's undefined. You can fix it by -
async function Fetch(url, callback){
const response = await fetch(url)
const html = await response.text()
const result = await callback(html);
return result;
}
As the callback, you are passing in Fetch function returns Promise so we can await it and return the result

Error: could not load the default credentials -- creds are fine, function dies

All of my other functions are working fine and this function works with a limited data set, so it's not really a credentials problem. It appears that I am running into the issue describes in these answers:
https://stackoverflow.com/a/58828202/3662110
https://stackoverflow.com/a/58779000/3662110
I'm fetching a pretty large chunk of JSON, so I think the issue is similar in that it's timing out, but I'm unable to add return or await in the same way described in those answers (tried them both, but return does nothing and await causes another error) because I'm looping through the data and writing many documents. This function has worked in the past when I limit the number of results fetched from the API. How do I keep the function alive long enough to write all the data?
const functions = require('firebase-functions');
const admin = require('firebase-admin');
exports.fetchPlayers = functions.pubsub
.schedule('every Tuesday of sep,oct,nov,dec 6:00')
.timeZone('America/New_York')
.onRun(async context => {
const response = fetch(
`<<< the endpoint >>>`,
{<<< the headers >>>}
)
.then(res => {
<<< handle stuff >>>
})
.then(res => res.json());
const resObj = await response;
const players = [];
resObj.dfsEntries[0].dfsRows.forEach(x => {
<<< extract the data >>>
players.push({ id, fName, lName, pos, team, [week]: fp });
})
players.forEach(player =>
admin
.firestore()
.collection('players')
.doc(`${player.id}`)
.set(player, {merge: true})
.then(writeResult => {
// write is complete here
})
);
});

In order to correctly terminate a Cloud Function, you are required to return a promise from the function that resolves when all of the asynchronous work is complete. You can't ignore any promises. Calling then on a promise isn't enough, since then just returns another promise. The problem is that you're ignoring the promises generated inside the forEach loop. You will need to collect all those promises in an array, then use Promise.all() to create a new promise that you can return from the function. To put it briefly:
const promises = players.map(player => {
return admin.firestore....set(...)
})
return Promise.all(promises)
If you don't wait for all the promises to resolve, then the function will terminate and shut down early, before the async work is done.

How to order the order of returned API calls with generators?

I'm practicing some more advanced Javascript techniques, and came across generators and iterators as something I wanted to look into. I know that I'm doing this incorrectly, but I'm not really sure how to go about it.
The idea of my little program is this: I want to make API calls to the OpenWeather API for four (or more, but I'm testing with four) cities. The cities are stored in an array and one by one, the city is appended to the URL and a fetch request is sent. Each response is appended to an array and the array is sent to the client.
This was my original code:
// node/express setup here
const cities = ["London%2Cuk", "New York%2Cus", "Johannesburg%2Cza", 'Kingston%2Cjm']
const url = process.env.URL_BASE;
const headers = {
"X-RapidAPI-Host": process.env.HOST,
"X-RapidAPI-Key": process.env.API_KEY
}
const requestInit = { method: 'GET',
headers: headers
};
const fetchWeather = (ep) => {
const appendedURL = url + ep;
return fetch(appendedURL, requestInit)
.then(r => r.json());
}
app.get('/', (req, res, err) => {
const data = []
Promise.all(
cities.map( async (city) => {
await fetchWeather(city)
.then(returns => {
data.push(returns)
})
})
)
.then(() => {
res.send(data)
return data;
})
.catch(err => console.log(err))
})
Right? Solid, works ok. But now I'm stuck on how to order it. The way I would think to do this is to switch await fetchWeather(city) to yield fetchWeather(city) and have a generator manager that would continue calling next(city) until the array had completed, but I'm having an issue figuring out the pattern. I refactored the api call to a generator and am testing out a generator management function.
The paradigm I have based on my understanding is this:
First .next() starts the iteration
Second .next(args) passes the designated city to the first yield
Third .next() sends the yielded fetch request and should (ideally) return the response object that can be .then()'d.
Here is my tester generator code:
function *fetchWeather() {
for (let i = 0; i < cities.length; i++){
const appendedURL = url + (yield);
yield fetch(appendedURL, requestInit)
.then(r => {
return r.json()
});
}
}
const generatorManager = (generator) =>{
if (!generator) {
generator = fetchWeather();
}
generator.next()
generator.next(cities[i])
generator.next().value.then( e =>
console.log(e));
}
I'm getting an error:TypeError: Cannot read property 'then' of undefined And I'm not sure where I'm going wrong here with my logic. How do I refactor this to allow me to wait for specific promises if I can't individually pass known values? I know there has to be a way, but I'm missing something.
Thanks in advance.

I don't understand what benefit you hope to get from using a generator here, but the reason you're getting that error is you're doing one to many .next()'s
The first generator.next() runs fetchWeather until the first yield, which is the yield at the end of const appendedURL = url + (yield);. The return value from calling generator.next() in this case is { value: undefined, done: false }
After that, generator.next(cities[i]) resumes fetchWeather, with cities[i] being the result of the previous yield. The generator continues running, calling fetch, then calling .then on that promise, and then yielding the resulting promise. So the return value that generatorManager sees from doing generator.next(cities[i]) is { value: /* a promise object */, done: false }.
So to fix that error, you need to reduce the number of calls you're making to generator.next
generator.next()
generator.next(cities[i]).value.then(e =>
console.log(e));
As mentioned in the comments, the usual way i'd do this is map the cities to promises, and then do promise.all. For example:
Promise.all(
cities.map((city) => fetchWeather(city)) // note, this is the original fetch weather, not the generator
).then((data) => {
res.send(data);
return data;
})
.catch(err => console.log(err))

Conditional Promise Chaining

I get an array of args as an argument, and then I make lots of server calls based on the algo below.
Post to endpoint /abc with args array as data.
Iterate over args array,
a. Pull 3 at a time and send 3 Get calls to endpoint /pqr
b. Once 3 calls in step '2.a' succeeds send 3 Post calls to endpoint /def
c. Collect responses from step '2.a' server call and push it in an array.
d. Repeat step a,b,c till args length.
Code Snippet for the entire process is given below, execution starts at function execute(args).
import Promise from 'bluebird';
import request from 'superagent';
// sends a post request to server
const servercall2 = (args, response) => {
const req = request
.post(`${baseUrl}/def`)
.send(args, response)
.setAuthHeaders();
return req.endAsync();
};
// sends a post request to server
const servercall1 = (args) => {
const req = request
.post(`${baseUrl}/abc`)
.send(args)
.setAuthHeaders();
return req.endAsync()
.then((res) => resolve({res}))
.catch((err) => reject(err));
};
async function makeServerCalls(args, length) {
// convert args to two dimensional array, chunks of given length [[1,2,3], [4,5,6,], [7,8]]
const batchedArgs = args.reduce((rows, key, index) => (index % length === 0 ? rows.push([key])
: rows[rows.length - 1].push(key)) && rows, []);
const responses = [];
for (const batchArgs of batchedArgs) {
responses.push(
// wait for a chunk to complete, before firing the next chunk of calls
await Promise.all(
***// Error, expected to return a value in arrow function???***
batchArgs.map((args) => {
const req = request
.get(`${baseUrl}/pqr`)
.query(args)
// I want to collect response from above req at the end of all calls.
return req.endAsync()
.then((response) =>servercall2(args,response));
})
)
);
}
// wait for all calls to finish
return Promise.all(responses);
}
export function execute(args) {
return (dispatch) => {
servercall1(args)
.then(makeServerCalls(args, 3))
.then((responses) => {
const serverresponses = [].concat(...responses);
console.log(serverresponses);
});
};
}
I am facing couple of issues
2.c seems not to be working fine "Collect responses from step '2.a' server call and push it in an array.". Error: expected to return a value in arrow function. What am I doing wrong here? Please note that at the end I care about the response from step 2.a only.
Is this a right chaining or it can be optimized, based on the requirements mentioned above?
Is there any other failure handling I have to do?

You have a brick wall of text so its becoming a little hard to decipher what you're actually trying to achieve, but I will give my two cents on the code given.
//Both server calls can be simplified.. no need to
//wrap in another promise if one is being returned
const servercall2 = (args, response) => {
const req = request
.post(`${baseUrl}/def`)
.send(args, response)
.setAuthHeaders();
return req.endAsync();
};
//Here... you return no value in the function passed to map, thus an
//error is being thrown. You need to return a Promise from here so that
//it can be passed into Promise.all
const allFinished = await Promise.all(
batchArgs.map((args) => {
const req = request
.get(`${baseUrl}/pqr`)
.query(args)
// I want to collect response from above req at the end of all calls.
return req.endAsync()
})
);
allFinished.then(function(results){
});

It might be this -- each item in batchArgs.map should be a Promise I think? Then Promise.all will wait for each to finish:
batchArgs.map((args) => {
const req = request
.get(`${baseUrl}/pqr`)
.query(args)
// Return promise here
return req.endAsync()
.then((response) =>servercall2(args,response))
.then((res) => res);
})