I am trying add retry to my api call using axios-retry module. In order to test I am using mockoon macosx client. I have setup the endpoint in mockoon to return 502 response all the time. So that I can test the retry.
import axios from "axios";
import axiosRetry from 'axios-retry';
async function sendRequest(method): Promise<any> {
try {
// return 502 after 100ms
let url = `http://localhost:3000/answer`
axiosRetry(axios, {
retries: 3
});
const response = await axios[method](url);
console.log('api call completed');
return response;
} catch (error) {
console.log('api call error: ', error);
throw error;
}
}
(async () => {
const response = await sendRequest('get')
})()
The issue here is, the axios.get not completing the execution. Therefore it does not log either api call error or api call completed messages. Any help would be highly appreciated.
axiosRetry does not work with axios 0.19.0 (the current axios release) : https://github.com/softonic/axios-retry#note
Alternative
Use a general purpose async retry function e.g.
async function retry<T>(fn: () => Promise<T>, n: number): Promise<T> {
let lastError: any;
for (let index = 0; index < n; index++) {
try {
return await fn();
}
catch (e) {
lastError = e;
}
}
throw lastError;
}
// use
const response = await retry(() => axios[method](url), 3);
More
Source of the retry function.
Related
I want to wait on the HTTP POST request to complete and then return response to the caller function. I am getting Undefined when I print the received results.
I have defined post method as below:
// httpFile.js
const axios = require('axios');
module.exports = {
getPostResult: async function(params) {
console.log("getPostResult async called...");
var result = await axios.post("https://post.some.url", params)
.then ((response) => {
console.log("getPostResult async success");
return {response.data.param};
})
.catch ((error) => {
console.log("getPostResult async failed");
return {error.response.data.param};
});
}
}
And I am calling it in this way:
// someFile.js
const httpFile = require('./httpFile');
// Called on some ext. event
async function getPostResult() {
var params = {var: 1};
var result = await httpFile.getPostResult(params);
// Getting Undefined
console.log("Done Result: " + JSON.stringify(result));
}
I don't want to handle the .then and .catch in the calling function as I want to return the different values based on the POST result.
How should I wait for the response and get the return results.
In the above code I am getting the log statements as expected and "Done Result" get printed at the very end after the 'getPostResult' returns.
you are using both await & .then thats why it returns undefined.
this is how it should look
// httpFile.js
const axios = require('axios')
module.exports = {
getPostResult: async function (params) {
try {
const res = await axios.post('https://post.some.url', params)
return res.data
} catch (error) {
// I wouldn't recommend catching error,
// since there is no way to distinguish between response & error
return error.response.data
}
},
}
if you want to catch error outside of this function then this is way to go.
getPostResult: async function (params) {
const res = await axios.post('https://post.some.url', params)
return res.data
},
I am running this asynchronous function in my React app -
const getMetaData = async (hashes: any) => {
console.log({ hashes });
try {
const data = hashes.map(async (hash: any) => {
const url = `http://localhost:3003/user/pinata/getmetadata/${hash}`;
const metadata = await axios.get(url);
return metadata.data.response;
});
console.log("data1", data);
const metadata = await Promise.all(data);
console.log('data2', metadata);
} catch (error) {
console.log('getMetaData Error', error);
}
};
console.log("data1", data) gives me -
data1 (12) [Promise, Promise, Promise, Promise, Promise, Promise, Promise, Promise, Promise, Promise, Promise, Promise]
The problem here is after I do a await Promise.all(data) I don't get data2 anywhere in the console. Maybe because the Promises are not even getting resolved?
Any idea what might be wrong?
Thanks in advance.
It seems that your code works fine when using SWAPI API so it can be that the API you use does not deliver data appropriately. I run the below code to test. Here's a link to codebox to play around with it if you want.
import axios from "axios";
const data = ["people", "planets", "starships"];
const getMetaData = async (hashes) => {
console.log({ hashes });
try {
const data = hashes.map(async (hash) => {
const url = `https://swapi.dev/api/${hash}`;
const metadata = await axios.get(url);
return metadata.data.results;
});
console.log("data1", data);
const metadata = await Promise.all(data);
console.log("data2", metadata);
} catch (error) {
console.log("getMetaData Error", error);
}
};
getMetaData(data);
With this code, it appears the most likely situation is that one of the promises in the loop is not resolving or rejecting. To confirm that, you can log every possible path with more local error handling so you can see exactly what happens to each request. I also added a timeout to the request so you can definitely find out if it's just not giving a response, but you can also see that by just looking at the logging for begin and end of each request in the loop:
function delay(msg, t) {
return new Promise((resolve, reject)) => {
setTimeout(() => {
reject(new Error(msg));
}), t);
});
}
const getMetaData = async (hashes: any) => {
console.log({ hashes });
try {
const data = hashes.map(async (hash: any, index: number) => {
try {
console.log(`Request: ${index}, hash: ${hash}`);
const url = `http://localhost:3003/user/pinata/getmetadata/${hash}`;
const metadata = await axios.get(url);
console.log(`Request: ${index}, result: ${metadata.data.response}`);
return metadata.data.response;
} catch (e) {
console.log(`Request: ${index} error: `, e);
throw e;
}
});
console.log("data1", data);
const metadata = await Promise.all(data.map((p: any, index: number) => {
return Promise.race(p, delay(`Timeout on request #${index}`, 5000));
});
console.log('data2', metadata);
} catch (error) {
console.log('getMetaData Error', error);
}
};
FYI, I don't really know Typescript syntax so if I've made any Typescript mistakes here, you can hopefully see the general idea and fix the syntax.
I am calling the following cloud function from my Angular application, but the value returned is always null even though the cloud function is correctly logging the result. Not sure what I am doing incorrectly.
My Angular code is as follows:
const data = {test: testToProcess};
const process = this.fns.httpsCallable("gprocess"); // fns is Angular Fire Functions
process(data)
.toPromise() // since Angular Fire function returns an observable, Iam converting to a promise
.then(
(result) => {
console.log("function called: " + JSON.stringify(result));
},
(err) => {
console.log("Function call error " + JSON.stringify(err));
}
);
My cloud functions code is as follows:
import * as functions from "firebase-functions";
const fs = require("fs");
const { google } = require("googleapis");
const script = google.script("v1");
const scriptId = "MY_SCRIPT_ID";
export const gprocess = functions.https.onCall(async (data: any, context: any) => {
const test = data.test;
return fs.readFile("gapi_credentials.json", (err: any, content: string) => {
if (err) {return err;}
const credentials = JSON.parse(content); // load the credentials
const { client_secret, client_id, redirect_uris } = credentials.web;
const googleAuth = require("google-auth-library");
const functionsOauth2Client = new googleAuth.OAuth2Client(client_id, client_secret, redirect_uris); // Constuct an auth client
functionsOauth2Client.setCredentials({refresh_token: credentials.refresh_token,}); // Authorize a client with credentials
return runScript(functionsOauth2Client,scriptId,JSON.stringify(test))
.then((scriptData: any) => {
console.log("Script data in main function is" + JSON.stringify(scriptData));
return scriptData;
})
.catch((error) => {return error;});
});
});
function runScript(auth: any, scriptid: string, test: any) {
return new Promise(function (resolve, reject) {
script.scripts.run({
auth: auth,
scriptId: scriptid,
resource: {function: "doGet",devMode: true,parameters: test}
})
.then((err: any, respons: any) => {
if (err) {
console.log("API returned an error: " + JSON.stringify(err));
resolve(err);
} else if (respons) {
console.log("Script is run and response is " + JSON.stringify(respons));
resolve(respons);
}
});
});
}
The angular function is returning this result before the processing on the cloud function can be completed. IT is not waiting for the results to be returned from the cloud function.
detailed.component.ts:691 function called: null
After some time the results are logged on the cloud function console but this is not returned back to the angular client. The log on the cloud function is as follows and as shown below the correct result is logged:
5:53:32.633 PM gpublish Function execution started
5:53:32.694 PM gpublish Function execution took 61 ms, finished with status code: 204
5:53:33.185 PM gpublish Function execution started
5:53:33.804 PM gpublish Function execution took 620 ms, finished with status code: 200
5:54:31.494 PM gpublish Script is run and response is : {"config":... some result}
5:54:31.593 PM gpublish Script data in main function is{"config":... some result}
Please help!
Your function is not correctly returning a promise that resolves to the data to serialize and send to the client. The problem is that fs.readFile doesn't actually return a promise. It's asychronous and returns nothing, and that's what the client will receive. The callback is being invoked, but nothing inside that callback will affect what the client sees.
You will need to find another way of doing your file I/O that is either synchronous (such as fs.readFileSync), or actually works with promises instead of just an async callback.
I changed my code to use readFileSync as mentioned above (Thank you Doug!)
In addition, I stringified the data being sent back from the cloud function to the Angular client.
My cloud function now looks like this:
import * as functions from "firebase-functions";
const fs = require("fs");
const { google } = require("googleapis");
const script = google.script("v1");
const scriptId = "MY_SCRIPT_ID";
export const gprocess = functions.https.onCall(
async (data: any, context: any) => {
try {
const test = data.test;
const content = fs.readFileSync("credentials.json"); // CHANGED TO READFILESYNC
const credentials = JSON.parse(content);
const { client_secret, client_id, redirect_uris } = credentials.web;
const googleAuth = require("google-auth-library");
const functionsOauth2Client = ... some code to construct an auth client and authorise it
return runScript(functionsOauth2Client,scriptId,JSON.stringify(test)).then((scriptData: any) => {
return JSON.stringify(scriptData); // STRINGIFIED THE DATA
});
} catch (err) {
return JSON.stringify(err);
}
});
function runScript(auth: any, scriptid: string, test: any) {
return new Promise(function (resolve, reject) {
script.scripts.run({auth: auth,scriptId: scriptid,resource: {function: "doGet",parameters: test}})
.then((respons: any) => {resolve(respons.data);})
.catch((error: any) => {reject(error);});
});
}
I'm getting a "deadline-exceeded" error on the frontend when calling a firebase callable cloud function (onCall).
I know that I have to return a Promise so the function knows when to clean itself, but it is still not working.
After 60 seconds, "deadline-exceeded" is throw to the frontend but the function keeps running on the server and finish with success. All batch operations are written to the firestore.
10:37:14.782 AM
syncExchangeOperations
Function execution took 319445 ms, finished with status code: 200
10:36:57.323 AM
syncExchangeOperations
Function execution started
10:36:57.124 AM
syncExchangeOperations
Function execution took 170 ms, finished with status code: 204
10:36:56.955 AM
syncExchangeOperations
Function execution started
async function syncBinanceOperations(
userId,
userExchange,
userExchangeLastOperations,
systemExchange
) {
try {
const client = Binance({
apiKey: userExchange.apiKey,
apiSecret: userExchange.privateKey
});
const batch = admin.firestore().batch();
const lastOperations = userExchangeLastOperations
? userExchangeLastOperations
: false;
const promises = [];
promises.push(
syncBinanceTrades(client, lastOperations, userId, systemExchange, batch)
);
promises.push(
syncBinanceDeposits(client, lastOperations, userId, systemExchange, batch)
);
promises.push(
syncBinanceWhitdraws(
client,
lastOperations,
userId,
systemExchange,
batch
)
);
promises.push(
updateUserExchange(userId, userExchange.id, {
lastSync: moment().format('x')
})
);
await Promise.all(promises);
return batch.commit();
} catch (error) {
return handleErrors(error);
}
}
exports.syncExchangeOperations = functions.https.onCall(
async (data, context) => {
try {
userAuthenthication(data.userId, context.auth);
let user = await getUser(data.userId);
if (!user.plan.benefits.syncExchanges) {
throw 'Operação não autorizada para o plano contratado';
}
let userExchange = await getUserExchange(data.userId, data.exchangeId);
let response = await Promise.all([
getUserLastOperations(data.userId, userExchange.exchangeId),
getSystemExchange(userExchange.exchangeId)
]);
let userExchangeLastOperations = response[0];
let systemExchange = response[1];
switch (systemExchange.id) {
case 'binance':
return syncBinanceOperations(
user.id,
userExchange,
userExchangeLastOperations,
systemExchange
);
}
} catch (error) {
return handleErrors(error);
}
}
);
It works fine if I change this function to a HTTP request. It waits the function to finish and returns.
exports.syncExchangeOperations = functions
.runWith(runtimeOpts)
.https.onRequest((req, res) => {
return cors(req, res, async () => {
try {
let auth = await admin.auth().verifyIdToken(req.get('Authorization').split('Bearer ')[1]);
let userExchange = await getUserExchange(
auth.uid,
req.query.exchangeId
);
let response = await Promise.all([
getUserLastOperations(auth.uid, userExchange.exchangeId),
getSystemExchange(userExchange.exchangeId)
]);
let userExchangeLastOperations = response[0];
let systemExchange = response[1];
switch (systemExchange.id) {
case 'binance':
await syncBinanceOperations(
auth.uid,
userExchange,
userExchangeLastOperations,
systemExchange
);
}
res.status(200).send();
} catch (error) {
res.status(401).send(handleErrors(error));
}
});
});
The "deadline-exeeded" that you encountered is an error thrown by the Firebase Javascript library on the client (not the function itself). The Firebase docs are lacking documentation o how to use functions.runWithOptions() on a callable function. For some reason the functions().httpsCallable() has a built in timeout on the client side.
So if you use this on your Node.js function:
exports.testFunction = functions.runWith({ timeoutSeconds: 180 }).https.onCall(async (data, ctx) => {
// Your Function Code that takes more than 60second to run
});
You need to override the buit in Javascript Library timeout on the client like this:
let testFunction = firebase.functions().httpsCallable("testFunction", {timeout: 180000});
I don't know what is the purpose of the built in timeout on the client, for me it has no purpose since it doesn't even stop the execution of the function on the server. But it must be there for some internal reasons.
Notice the Node.js timeoutSeconds is in seconds and the timeout option on the client library is in milliseconds.
"Deadline exceeded" means that the function invocation timed out from the perspective of the client. The default is 60 seconds.
Try increasing the timeout on both the client and function so that it has time to complete before the client timeout is reached. You can do this by specifying it in an HttpsCallableOptions object.
Also try returning something other than batch.commit(). Whatever that function return will be serialized and sent to the client, which could cause problems. Instead, just await batch.commit() then return something predictable, like a plain JavaScript object.
See the API documentation for information on setting the timeout:
https://firebase.google.com/docs/reference/js/firebase.functions.Functions#https-callable
https://firebase.google.com/docs/reference/js/firebase.functions.HttpsCallableOptions.html#timeout
I have an application that uses axios for it's ajax requests. When a user experiences a network issue (for example, their wifi goes out and they no longer have an internet connection while on my application), I want to make sure that only the first axios request is made, and if I detect there is a network issue, to not attempt any more requests, but instead to retry the same request until successful.
My application performs many requests, including a request every 2.5 seconds (in this example, getData). It also establishes a Twilio connection when the application initializes (it executes twilio() on initialization).
When a connection is lost, the following happens:
getData fails, resulting in a console message of this is a network error.
TwilioDevice.offline is executed. This results in two error messages: first a this is a network error. message (error message #2) when TwilioDevice.offline tries fetchToken(), and then a received an error. message (error message #3) after the fetchToken() fails.
Given #'s 1 and 2, how can I make sure that:
If I experience a network error, I only receive one error message instead of 3 saying that "there was a network error"
My app detects that there is a network error, then tries to re-establish a connection, then, if successful, resumes fetching data, Twilio tokens, etc.
Thanks! Code is below.
example code:
const getData = async () => {
try {
const response = await axios.get('api/data');
return response.data;
} catch (error) {
handleError(error);
}
};
const fetchToken = async () => {
try {
const data = await axios.get('api/twilio-token');
return data.token;
} catch (error) {
return handleError(error);
}
};
const handleError = error => {
if (!error.response) {
console.log("this is a network error.");
} else {
console.log("received an error.");
}
};
twilio.js:
import { Device as TwilioDevice } from 'twilio-client';
const registerEvents = () => {
TwilioDevice.ready(() => {
console.log('Twilio.Device is now ready for connections');
});
TwilioDevice.connect((conn) => {
console.log(`Connecting call with id ${conn.parameters.CallSid}`);
// code to start call
conn.disconnect((connection) => {
console.log(`Disconnecting call with id ${connection.parameters.CallSid}`);
// code to end call
});
});
TwilioDevice.error((error) => {
console.log("Twilio Error:");
console.log(error);
});
TwilioDevice.offline(async () => {
try {
const newTwilioToken = await fetchToken(); // error message #2
return TwilioDevice.setup(newTwilioToken);
} catch (error) {
return handleError(error); // error message #3
}
});
};
export const twilio = async () => {
try {
registerEvents();
const twilioToken = await fetchToken();
TwilioDevice.setup(twilioToken);
} catch (error) {
return handleError(error);
}
};
I would recommend making your fetchToken and getData methods to throw errors rather than handling it themselves so that they can be handled by their outer functions.
Something like,
const getData = async () => {
try {
const response = await axios.get('api/data');
return response.data;
} catch (error) {
throw (error);
}
};
const fetchToken = async () => {
try {
const data = await axios.get('api/twilio-token');
return data.token;
} catch (error) {
throw (error);
}
};
So that when you call twilio() that function can handle the error like retrying etc.