Develop Reference

Fetch with retry, abort, etc

I've been trying to find a wrapper that does fetch with retries, timeouts, aborts, etc. I came across https://pastebin.com/54Ct4xEh a little bit ago, and after fixing a couple typos (missing options. and =>), it works, except... well, maybe it works, but I don't know how to use it. How do I abort a fetch with this particular wrapper? I have a fiddle, https://jsfiddle.net/1fdwb2o6/2/. With this code, how can I, say, click a button and have it abort this fetch loop? For my use case, I' using boopstrap, and I have a modal that, when shown, attempts to load dynamic content. If the user clicks Cancel while it's loading, I want the fetch process to stop. From what I can tell, I should be able to do it with the code below... but I'm not sure how to perform the abort. Perhaps this isn't possible, as structured, with a Promise... but I don't know enough (anything) about promises to know better, one way or the other.
const fetchWithRetry = (userOptions) => {
let abort = false;
const options = {
url: '',
options: {},
cancel: {},
retries: 5,
retryDelay: 1000,
...userOptions
};
// Add an abort to the cancel object.
options.cancel.abort = () => {
abort = true;
};
// Abort or proceed?
return abort ? Promise.reject('aborted') : fetch(options.url).then(response => {
// Reject because of abort
return abort ? Promise.reject('aborted')
// Response is good
: response.ok ? Promise.resolve(response.text())
// Retries exceeded
: !options.retries ? Promise.reject('retries exceeded')
// Retry with one less retry
: new Promise((resolve, reject) => {
setTimeout(() => {
// We use the returned promise's resolve and reject as
// callback so that the nested call propagates backwards.
fetchWithRetry({
...options,
retries: options.retries - 1
}).then(resolve, reject);
}, options.retryDelay);
});
});
}
var xxx;
console.clear();
xxx = fetchWithRetry({
url: "some_file_that_doesnt_exist.php"
})
.then((response) => {
alert(response);
}).catch(function(err) {
// Error: response error, request timeout or runtime error
alert("Error! Cannot load folder list! Please try again!");
});
setTimeout(function() {
// somehow, abort the fetch...
// xxx.abort(); <-- no worky...
}, 1234);

As I said in my comments, the code you have in your question does not provide a cancel() function that the caller can use. It has a cancel() function internally, but that's not something the caller can use. As written that function just returns a promise so the caller has nothing they can call to cancel the retries.
So, I decided to write my own version of fetchWithRetry() that would work for your use case. This has a number of capabilities that the one in your question does not:
It returns both the promise and a cancel function so the caller can cancel the retries.
It allows you to pass the init options for fetch() so you can pass any of the various arguments that fetch() supports and are often needed such as withCredentials.
It has an option to check the response.ok boolean so it will detect and retry more things that you would if you required the promise to be rejected before a retry (note: fetch() doesn't reject on a 404, for example).
If There was a fetch() rejection and it was either cancelled or it ran out of retries, then it will use the newest Error class feature where it will set the cause to the actual fetch() error so the caller can see what the original error was.
Note that this version of fetchWithRetry() returns an object containing both a promise and a cancel function. The caller uses the promise the same way they would any promise from fetch() and they can use the cancel() function to cancel any further retries.
Here's the code:
const Deferred = function() {
if (!(this instanceof Deferred)) {
return new Deferred();
}
const p = this.promise = new Promise((resolve, reject) => {
this.resolve = resolve;
this.reject = reject;
});
this.then = p.then.bind(p);
this.catch = p.catch.bind(p);
if (p.finally) {
this.finally = p.finally.bind(p);
}
}
function fetchWithRetry(url, userOptions = {}, init = {}) {
const options = {
// default options values, can be overridden by userOptions
retries: 3,
retryDelay: 1000,
checkResponseOk: true,
...userOptions
};
let cancelled = false;
let timerDeferred;
let timer;
function run() {
return fetch(url, init).then(response => {
// force retry on non 2xx responses too
if (options.checkResponseOk && !response.ok) {
throw new Error(`fetch failed with status ${response.status}`);
}
return response;
}).catch(err => {
// got error, set up retry
console.log(err);
if (cancelled) {
throw new Error("fetch cancelled", { cause: err });
}
--options.retries;
if (options.retries < 0) {
throw new Error("fetch max retries exceeded", { cause: err });
}
// create new Deferred object for use with our timer
// so it can be resolved by the timer or rejected
// by the cancel callback
timerDeferred = new Deferred();
timer = setTimeout(() => {
timerDeferred.resolve();
timer = null;
}, options.retryDelay);
return timerDeferred.then(() => {
if (cancelled) {
throw new Error("fetch cancelled", { cause: err });
}
return run();
});
});
}
return {
promise: run(),
cancel: () => {
cancelled = true;
// if currently in a timer waiting, reject immediately
if (timer) {
clearTimeout(timer);
timer = null;
}
if (timerDeferred) {
timerDeferred.reject(new Error("fetch cancelled"));
}
}
}
};
Sample usage:
const result = fetchWithRetry(someUrl);
result.promise.then(resp => {
return resp.text().then(data => {
// got final result here
console.log(data.slice(0, 100));
});
}).catch(err => {
console.log(err);
});
// simulate user cancel after 1.5 seconds
setTimeout(() => {
result.cancel();
}, 1500);

Axios connection timeout

I see a lot of answers and none of them work for me. I am implementing retry code in the browser, where if the API hasn't responded in 4000ms it retries.
The problem is I want to do this for POST requests that are not idempotent, and the response state in chrome dev tools (whether it succeeds or fails) does NOT match axios or my implemented logic of when a timeout occurs.
This results in POST requests calling twice successfully on the server even though the connection throws an error within my axios code. It's a race condition somewhere, I'm assuming the time between axios connects and when it is able to set the result of the response.
I've tried default axios timeout which doesn't work, as that is a response timeout.
I've also tried to implement a connection timeout and I still am encountering the same issue.
The issue starts occuring if I set the connTimeout to be right in the ballpark of how long it takes the server to response on average, +/- a few ms. I feel like when a request is cancelled, somehow it's not checking if the connection actually succeeded or not before attempting to cancel.
I'd do it myself (before calling source.cancel(), but I'm not sure what I can read to get the state. The only thing I see there is an unresolved promise)
const makeRequest = async (args, connTimeout, responseTimeout) => {
const source = axios.CancelToken.source();
const argsWithToken = {
...args,
cancelToken: source.token,
};
const api = buildAxios(responseTimeout);
const timeout = setTimeout(source.cancel, connTimeout);
return api(argsWithToken).then(result => {
clearTimeout(timeout);
return result;
});
};
const handleRetries = (args, maxRetries) => (
new Promise(async (resolve, reject) => { /* eslint-disable-line */
let retries = 0;
let success = false;
while (!success && retries < maxRetries) {
try {
const result = await makeRequest(args, 300, 30000); /* eslint-disable-line */
success = true;
resolve(result);
} catch (err) {
retries += 1;
// console.log(`Error making ${args.method} request to ${args.url}, retrying... #${retries}`);
}
}
// line below is included to prevent process leaks
if (!success) reject(new Error(`Retried ${retries} times and still failed: ${args.url}`));
})
);
handleRetries({url: '/settings', method: 'get'}, 3)

Manage error in custom step for autologin in CodeceptJS

I'm using autoLogin plugin in CodeceptJS project with Puppeteer as testing lib.
This is my first project with CodeceptJS, the autoLogin workflow is working fine, but I'm not using the application pages (UI) to login or check the current token, I'm using directly REST API calls to make all the process a bit faster.
My autoLogin config is something like:
autoLogin: {
enabled: true,
saveToFile: true,
inject: 'login',
users: {
admin: {
login: async (I) => {
I.amOnPage('/login');
await I.loginWithCredentials(adminUser.username, adminUser.password);
},
check: (I) => {
const token = codeceptjs.store['admin_session'];
I.validateToken(token);
},
fetch: async (I) => {
const cookie = await I.grabCookie('token');
return cookie.value;
},
restore: (I, sessionToken) => {
I.amOnPage('/login');
I.saveTokenData(sessionToken);
}
},
}
}
The steps saveTokenData(), validateToken() and loginWithCredentials() are custom steps defined with actor(), for instance:
module.exports = function() {
return actor({
async validateToken(token) {
let response = await this.sendGetRequest(`/api/session/validate?token=${token}`);
if (response.status === 200) {
if (response.data === true) {
return true;
}
}
throw new Error('Invalid token !!');
}
});
The line throw new Error('Invalid token !!'); It's generating a "unexpected" error in the workflow, showing this line in the log:
(node:5986) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
I've tried several approaches like using the recorder.add() (throwing the error inside the recorder), I also tried to use chained promises (without async keyword) and the assert lib with assert.fail() , but always the same problem.
How can I "mark" my custom step as a failure without odd log messages?, I just want to implement the check step in the autoLogin without the exiting API to validate the user interface (I.see(), I.seeElement(), I.dontSee()...)
I'm using an ugly workaround until I get a better solution, the custom step returns a boolean, without throwing any error, and in the check code I've got this:
check: async (I) => {
console.log('admin.check');
const token = codeceptjs.store['admin_session'];
const isValid = await I.validateToken(token);
if (!isValid) {
I.see("NOTHING TO SEE HERE, so It'll fail");
}
},
I opened an issue in CodeceptJS GitHub project #2600

How about use the assert.fail method? It will fail your entire Scenario and you can customize the message!
const assert = require("assert");
module.exports = function() {
return actor({
async validateToken(token) {
let response = await this.sendGetRequest("/api/session/validate?token=${token}");
if (response.status !== 200 || response.data !== true) {
assert.fail("Invalid token!");
}
}
});
Check method:
check: async (I) => {
console.log('admin.check');
const token = codeceptjs.store['admin_session'];
await I.validateToken(token);
}

How to cancel token using a custom Axios instance?

I have an custom Axios instance using axios.create(). I would like to use the cancellation feature of Axios but the request fired from custom instance never gets cancelled. It does't get detected in the .isCancel() method. But it works fine when used with the global Axios object.
const axiosAuth = axios.create();
const cancelToken = axios.CancelToken.source();
//request
const getProducts = async () => {
try {
const response = await axiosAuth.get('api', {
cancelToken: cancelToken.token
});
if (response.status === 200) {
return response.data;
}
} catch (err) {
if (axios.isCancel(err)) {
console.log('Error: ', err.message);
return true;
} else {
throw new Error(err);
}
}
};
// I'm cancelling the request on button click using `cancelToken.cancel()`
I don't understand why cancellation doesn't work with a custom Axios instance.

Figured it out there was an issue in the one the Interceptors. Just make sure you check if its cancellation error there as well using Axios.isCancel() before you do anything with the error object.

Javascript : Continue execution after ajax error

I'm trying to accomplish the following:
On load of the page, the code should do a $.getJSON request (which
basically is an ajax get request) (on success yayy just continue execution otherwise go down this list).
when this fails with code 400 I would like to wait 1 second and retry this request (if this succeeds than yayy continue code execution otherwise go down this list)
after that just skip the ajax get request and continue executing the other code.
But Currently, I can't manage to continue to execute the code because of the thrown error. For this I tried :
try{
$.getJSON('/services/getData').success(function(data) {
configurationObject = data["configuration"];
})
} catch(err) {
console.log("error");
}
and
$.getJSON('/services/getData').success(function(data) {
configurationObject = data["configuration"];
})
.error(function(){
console.log("keep running please")
}
but both just stops the execution of the complete javascript code. Is there any way I can keep running after an error occurred when using ajax calls?

Try getting used to promises for doing asynchronous tasks. this is how i would have done it if i where using jQuery
function getData() {
return $.getJSON('/services/getData').then(function(data){
configurationObject = data["configuration"];
return configurationObject
}, function(err) {
var dfd = $.Deferred();
if (err.status === 400) {
? setTimeout(function () {
dfd.resolve(getData());
}, 1000)
} else {
dfd.reject(err)
}
return dfd.promise();
})
}
getData().then(successFn, errorFn)
but if i could get lose on using es7 and async/await and just vanilla javascript then this is how i would have done it instead
const sleep = delay => new Promise(rs => setTimeout(rs, delay))
async function getData() {
const res = await fetch('/services/getData')
if (res.status === 400) {
await sleep(1000)
return getData()
} else if (!res.ok) {
throw new Error('Could not get data')
}
const json = await res.json()
configurationObject = json['configuration']
return configurationObject
}
getData().then(successFn, errorFn)

Please see http://api.jquery.com/jQuery.getJSON/
.error handler has been removed. You can use .fail instead, whose handler receives a jqXhr argument. From that, you can read status to check for 400 etc.