I see a lot of answers and none of them work for me. I am implementing retry code in the browser, where if the API hasn't responded in 4000ms it retries.
The problem is I want to do this for POST requests that are not idempotent, and the response state in chrome dev tools (whether it succeeds or fails) does NOT match axios or my implemented logic of when a timeout occurs.
This results in POST requests calling twice successfully on the server even though the connection throws an error within my axios code. It's a race condition somewhere, I'm assuming the time between axios connects and when it is able to set the result of the response.
I've tried default axios timeout which doesn't work, as that is a response timeout.
I've also tried to implement a connection timeout and I still am encountering the same issue.
The issue starts occuring if I set the connTimeout to be right in the ballpark of how long it takes the server to response on average, +/- a few ms. I feel like when a request is cancelled, somehow it's not checking if the connection actually succeeded or not before attempting to cancel.
I'd do it myself (before calling source.cancel(), but I'm not sure what I can read to get the state. The only thing I see there is an unresolved promise)
const makeRequest = async (args, connTimeout, responseTimeout) => {
const source = axios.CancelToken.source();
const argsWithToken = {
...args,
cancelToken: source.token,
};
const api = buildAxios(responseTimeout);
const timeout = setTimeout(source.cancel, connTimeout);
return api(argsWithToken).then(result => {
clearTimeout(timeout);
return result;
});
};
const handleRetries = (args, maxRetries) => (
new Promise(async (resolve, reject) => { /* eslint-disable-line */
let retries = 0;
let success = false;
while (!success && retries < maxRetries) {
try {
const result = await makeRequest(args, 300, 30000); /* eslint-disable-line */
success = true;
resolve(result);
} catch (err) {
retries += 1;
// console.log(`Error making ${args.method} request to ${args.url}, retrying... #${retries}`);
}
}
// line below is included to prevent process leaks
if (!success) reject(new Error(`Retried ${retries} times and still failed: ${args.url}`));
})
);
handleRetries({url: '/settings', method: 'get'}, 3)
Related
I'm in need of implementing following features for network/http client using fetch API.
Timeout
Also abort a previous request when user make multiple requests.
I was able to implement both using the abortController. But on the case of "Timeout" (no 1), I want to catch the abort and show a proper error message with "retry" option.
But when I wrap my network request inside try catch, I can't distinguish between above 1 and 2 cases. Cause both abort are thrown with same exception name/message.
The web implementation does support passing a "reason" into the abort() call. But looks like reactNative doesn't have that implemented ( Using react-native 0.63.3 )
async function request(url, abortController) {
// Manually timing out, as I did not find any support for timeout on react-native
const timeoutRef = setTimeout(() => abortController.abort(), 90000); // CASE 1 : Timeout abort
return await fetch(url,
{
signal: controller.signal
})
}
var abortController = null;
var requestPending = false;
async function searchWebsite(searchQuery) {
// If there is already pending requesting - we cancel that previous
// pending request.
if ( abortController && !controller.signal.aborted && requestPending) {
abortController.abort(); // CASE 2 : abort previous request
}
// Create a new request
try {
abortController = new AbortController();
requestPending = true;
let apiRequest = await request("http://someurl.com", abortController);
// Do whatever with `apiRequest`
requestPending = false;
} catch(e) {
requestPending = false;
if (e.name == 'AbortError') {
// HERE I'M STRUGGLING WITH
// figure out how to distinguish between "timeout" and "previous request" abort
}
}
}
How can I distinguish between different type of abortController abort on react-native?
I was in this situation before and decided to move the cancellation logic to the async function itself instead of the fetch, and create another abort controller instance, then you can throw a different message based on which abort controller you used to abort the request, based on that message you'll know which abort controller caused the error (the cancellation)
Here is an example of what i did (in a react hook), but you should be able to apply the same logic and throw your own errors in whichever way you like
return new Promise((resolve, reject) => {
if (abortControllerCancel.current.signal.aborted) {
reject({
message: 'canceled',
reason: 'canceled',
});
}
if (abortControllerDuplicate.current.signal.aborted) {
reject({
message: 'canceled',
reason: 'duplicate',
});
}
// the rest of the async function and resolving the promise
abortControllerCancel.current.signal.addEventListener(
'abort',
() => {
reject({
message: 'canceled',
reason: 'canceled',
});
}
);
abortControllerDuplicate.current.signal.addEventListener(
'abort',
() => {
reject({
message: 'canceled',
reason: 'duplicate',
});
}
);
}
I have an inputfield, onChange it sends my value of the inputfield to an API. So, the api will start fetching all the data. but when I continue typing again, I want that previous request to be canceled.
I'm using axios for making my request and tried looking at the documentation but I can't seem to figure out how it really works, can someone explain how to do this?
Here is my function that gets called by every new input:
const onChange = (value) => {
setTimeout(async() => {
let result = []
if (y === "keyword") result = await AutoSuggestionKeyword({
value: value
});
else {
const CancelToken = axios.CancelToken;
const source = CancelToken.source();
await axios.get(`https://${api_uri}/${value.toLowerCase()}`)
.catch(function(thrown) {
if (axios.isCancel(thrown)) {
console.log('Request canceled', thrown.message);
} else {
// handle error
}
}).then(resp => {
console.log(resp)
});
source.cancel();
}
}, 500)
}
You need to provide a cancelToken in your request,
axios.get(`https://${api_uri}/${value.toLowerCase()}`, {
cancelToken: source.token
}).catch(function(thrown) {
if (axios.isCancel(thrown)) {
console.log('Request canceled', thrown.message);
} else {
// handle error
}
});
I don't think you can cancel an HTTP request, but what you can do is wrap it in debounce function, debounce function wait for a certain time before calling the callback or function you wrap or pass on it.
you can simply google debounce and it will give you articles or npm packages that you can use.
I think this article also has the same issue you are trying to resolve
Happy coding
Edit 1: yeah so you can cancel the http request see comment below
I'm getting a "deadline-exceeded" error on the frontend when calling a firebase callable cloud function (onCall).
I know that I have to return a Promise so the function knows when to clean itself, but it is still not working.
After 60 seconds, "deadline-exceeded" is throw to the frontend but the function keeps running on the server and finish with success. All batch operations are written to the firestore.
10:37:14.782 AM
syncExchangeOperations
Function execution took 319445 ms, finished with status code: 200
10:36:57.323 AM
syncExchangeOperations
Function execution started
10:36:57.124 AM
syncExchangeOperations
Function execution took 170 ms, finished with status code: 204
10:36:56.955 AM
syncExchangeOperations
Function execution started
async function syncBinanceOperations(
userId,
userExchange,
userExchangeLastOperations,
systemExchange
) {
try {
const client = Binance({
apiKey: userExchange.apiKey,
apiSecret: userExchange.privateKey
});
const batch = admin.firestore().batch();
const lastOperations = userExchangeLastOperations
? userExchangeLastOperations
: false;
const promises = [];
promises.push(
syncBinanceTrades(client, lastOperations, userId, systemExchange, batch)
);
promises.push(
syncBinanceDeposits(client, lastOperations, userId, systemExchange, batch)
);
promises.push(
syncBinanceWhitdraws(
client,
lastOperations,
userId,
systemExchange,
batch
)
);
promises.push(
updateUserExchange(userId, userExchange.id, {
lastSync: moment().format('x')
})
);
await Promise.all(promises);
return batch.commit();
} catch (error) {
return handleErrors(error);
}
}
exports.syncExchangeOperations = functions.https.onCall(
async (data, context) => {
try {
userAuthenthication(data.userId, context.auth);
let user = await getUser(data.userId);
if (!user.plan.benefits.syncExchanges) {
throw 'Operação não autorizada para o plano contratado';
}
let userExchange = await getUserExchange(data.userId, data.exchangeId);
let response = await Promise.all([
getUserLastOperations(data.userId, userExchange.exchangeId),
getSystemExchange(userExchange.exchangeId)
]);
let userExchangeLastOperations = response[0];
let systemExchange = response[1];
switch (systemExchange.id) {
case 'binance':
return syncBinanceOperations(
user.id,
userExchange,
userExchangeLastOperations,
systemExchange
);
}
} catch (error) {
return handleErrors(error);
}
}
);
It works fine if I change this function to a HTTP request. It waits the function to finish and returns.
exports.syncExchangeOperations = functions
.runWith(runtimeOpts)
.https.onRequest((req, res) => {
return cors(req, res, async () => {
try {
let auth = await admin.auth().verifyIdToken(req.get('Authorization').split('Bearer ')[1]);
let userExchange = await getUserExchange(
auth.uid,
req.query.exchangeId
);
let response = await Promise.all([
getUserLastOperations(auth.uid, userExchange.exchangeId),
getSystemExchange(userExchange.exchangeId)
]);
let userExchangeLastOperations = response[0];
let systemExchange = response[1];
switch (systemExchange.id) {
case 'binance':
await syncBinanceOperations(
auth.uid,
userExchange,
userExchangeLastOperations,
systemExchange
);
}
res.status(200).send();
} catch (error) {
res.status(401).send(handleErrors(error));
}
});
});
The "deadline-exeeded" that you encountered is an error thrown by the Firebase Javascript library on the client (not the function itself). The Firebase docs are lacking documentation o how to use functions.runWithOptions() on a callable function. For some reason the functions().httpsCallable() has a built in timeout on the client side.
So if you use this on your Node.js function:
exports.testFunction = functions.runWith({ timeoutSeconds: 180 }).https.onCall(async (data, ctx) => {
// Your Function Code that takes more than 60second to run
});
You need to override the buit in Javascript Library timeout on the client like this:
let testFunction = firebase.functions().httpsCallable("testFunction", {timeout: 180000});
I don't know what is the purpose of the built in timeout on the client, for me it has no purpose since it doesn't even stop the execution of the function on the server. But it must be there for some internal reasons.
Notice the Node.js timeoutSeconds is in seconds and the timeout option on the client library is in milliseconds.
"Deadline exceeded" means that the function invocation timed out from the perspective of the client. The default is 60 seconds.
Try increasing the timeout on both the client and function so that it has time to complete before the client timeout is reached. You can do this by specifying it in an HttpsCallableOptions object.
Also try returning something other than batch.commit(). Whatever that function return will be serialized and sent to the client, which could cause problems. Instead, just await batch.commit() then return something predictable, like a plain JavaScript object.
See the API documentation for information on setting the timeout:
https://firebase.google.com/docs/reference/js/firebase.functions.Functions#https-callable
https://firebase.google.com/docs/reference/js/firebase.functions.HttpsCallableOptions.html#timeout
I need to implement a cancel-able client-side HTTP request in Node.js, without using external libraries. I'm giving a Promise object - cancellationPromise - which gets rejected when the cancellation is externally requested. This is how I know I may need to call request.abort().
The question is, should I be calling request.abort() only if https.request is still pending and response object is not yet available?
Or, should I be calling it even if I already got the response object and am processing the response data, like in the code below? In which case, will that stop any more response.on('data') events from coming?
async simpleHttpRequest(url, oauthToken, cancellationPromise) {
let cancelled = null;
let oncancel = null;
cancellationPromise.catch(error => {
cancelled = error; oncancel && oncancel(error) });
try {
const response = await new Promise((resolve, reject) => {
const request = https.request(
url.toString(),
{
method: 'GET',
headers: { 'Authorization': `Bearer ${oauthToken}` }
},
resolve);
oncancel = error => request.abort();
request.on('error', reject);
request.end();
});
if (cancelled) throw cancelled;
// do I need "oncancel = null" here?
// or should I still allow to call request.abort() while fetching the response's data?
// oncancel = null;
try {
// read the response
const chunks = await new Promise((resolve, reject) => {
response.on('error', reject);
const chunks = [];
response.on('data', data => chunks.push(data));
response.on('end', () => resolve(chunks));
});
if (cancelled) throw cancelled;
const data = JSON.parse(chunks.join(''));
return data;
}
finally {
response.resume();
}
}
finally {
oncancel = null;
}
}
It depends what you want to achieve by aborting a request.
Just a bit of background. HTTP 1 is not able to "cancel" a request it sends it and then waits for the response. You cannot "roll back" the request you did. You need a distributed transaction to do so. (Further reading.) As the MDN developer document states:
The XMLHttpRequest.abort() method aborts the request if it has already been sent. When a request is aborted, its readyState is changed to XMLHttpRequest.UNSENT (0) and the request's status code is set to 0.
Basically you stop the response from being processed by your application. The other application will probably (if you called abort() after it was sent to it) finish its processing anyways.
From the perspective of the question:
The question is, should I be calling request.abort() only if https.request is still pending and response object is not yet available?
TL.DR.: It only matters from the point of view of your application. As I glance at your code, I think it will work fine.
in ruby I can:
require 'timeout'
Timeout.timeout 10 do
# do smth > 10 seconds
end
it will raise timeout error to avoid code lock, how to do same thing in nodejs, nodejs #setTimeout doesn't fit my need
one case is, when i http.get timeout(for ex, netowrk is unstable), I should set timeout and handle the failed get request, I hope impl #timeout, how should i do?
try {
timeout(10, function () {
http.get("example.com/prpr")
})
} catch (e) {
if (e.message == "timeout") {
// do smth
} else {
throw e
}
}
You could look into a Promise-based approach here.
Using promises you can pass a function to be executed, and then the standard catch is called if that function raises an exception.
There is a helpful promise-based timeout library on NPM (npm install promise-timeout request-promise), and you could use it in Node something along the lines of...
'use strict';
var promiseTimeout = require('promise-timeout');
var requestPromise = require('request-promise');
promiseTimeout.timeout(requestPromise("http://example.com/prpr"), 10000)
.then(function (result) {
console.log({result});
}).catch(function (err) {
if (err instanceof pt.TimeoutError) {
console.error('HTTP get timed out');
}
});
I had a similar situation with nestJS based on node.js.
When calling an external API, it was a problem that even my service slowed down if it took too long. (If the external api is delayed, my service also had a problem of waiting forever.)
I figured out 2 ways.
First way:
const result = await axios({
timeout: 10000, // error: [AxiosError: timeout of 10000ms exceeded] { code: 'ECONNABORTED', ...
...
});
Second way: Promise.race()
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/race
// first function
const callAPI = axios({
method: "GET",
url: "http://yourapi",
headers: {
...
}
});
// second function
const timeoutCheck = (s) => {
return new Promise(resolve => setTimeout(resolve, s));
}
// check delay (first function VS second function)
const result = await Promise.race([
callAPI,
timeoutCheck(10000).then(() => {
throw new Error("api not responding for more than 10 seconds");
}),
]);
const { data: { resultCode, resultData } } = result;
You can try this out in your case:
var request = http.get(options, function (res) {
// other code goes here
});
request.setTimeout( 10000, function( ) {
// handle timeout here
});