Develop Reference

How to replay a Puppeteer HTTPRequest?

Using Puppeteer, I am able to intercept HTTPResponses and their HTTPRequests:
page.on("response", async response => {
let request = response.request(); // Getting the response request
let responseHeaders = response.headers(); // Check response headers
response.buffer().then(buffer => {
// Play with response content
});
})
Depending on the response content, I need to send the request again like a fresh one and get its response buffer. Instantiating an identical and new request is a valid option.
I know I could use node-fetch as a last resort, but Puppeteer seems to have everything embedded to do it without adding packages.
Do you know how to achieve this?
Puppeteer HTTPRequest doc
Puppeteer HTTPRequest class

You can use page.evaluate to send a post request using fetch API
await page.evaluate(() => {
return fetch('url', {method: 'POST', body: 'test' }).then(res => res.json())
})
So then you can make a request after the requestfinished event fired.
page.setRequestInterception(true)
page.on('requestfinished', async (request: Request) => {
let response = request.response() // Getting the response request
let responseHeaders = response.headers() // Check response headers
let responseBuffer = await response.buffer() // Get the buffer required
let responseJSON = await response.json() // Get parsed JSON body
await page.evaluate(([headers, buffer, json]) => { // Replay request with buffer received
let someData1 = buffer.toString() // Change buffer to string type
let someData2 = headers['Content-Type'] // or maybe use some headers data
let someData3 = json.properties.value // or use response data object properties
// This fetch API below will do the rest
return fetch('url', { method: 'POST', body: 'test' }).then(res => res.json())
}, [responseHeaders, responseBuffer, responseJSON])
})

Interceptor for fetch and fetch retry? (Javascript)

I am trying to create an interceptor for fetch in javascript (React to be more specific). It should get the result from every fetch that gets called, and if it is an 401 error it should initiate a new fetch call to another route to get a cookie (a refresh token). Then, the original fetch call should be tried again (because now the user is logged in).
I have managed to trigger the new fetch call and send back the cookie for each, but I got these two problems below:
I do not now how to retry the fetch call after the refresh token has been recieved. Is that possible? I found the fetch-retry npm (https://www.npmjs.com/package/fetch-retry) but not sure how and if I can implement that on an interceptor when it should be done for the original fetch call.
I seem to be doing something wrong with async await (I think), because the intercept is not waiting for the fetch call before returning the data (the statuscode on the original fetch seems to be 401 and not 200 which it should be after we get the cookie. I also tried to return the response of the fetch inside the interceptor but that returned undefined).
Any idea about how to solve this? Anyone who have done something similar?
Below is my code:
(function () {
const originalFetch = fetch;
fetch = function() {
return originalFetch.apply(this, arguments).then(function(data) {
if(data.status === 401) {
console.log('not authorized, trying to get refresh cookie..')
const fetchIt = async () => {
let response = await fetch(`/api/token`, {
method: 'POST',
credentials: 'include',
headers: {
'Content-Type': 'application/json'
},
});
}
fetchIt();
}
return data
});
};
})();
EDIT: To make it more clear what I am after. I need an interceptor like I described above to work so I don't have to do something like this after every fetch call:
getData() {
const getDataAsync = async () => {
let response = await fetch(`/api/loadData`, { method: 'POST' });
if(response.status === 401) {
let responseT = await fetch(`/api/token`, {
method: 'POST',
credentials: 'include',
headers: {
'Content-Type': 'application/json'
},
});
if(responseT.status === 401) {
return responseT.status
}
if(responseT.status === 200) {
response = await fetch(`/api/loadData`, { method: 'POST' });
}
}
let data = await response.json();
//Do things with data
};
getDataAsync();
};
So basically the interceptor should:
Check if there is a 401, if so then:
fetch api/token
If api/token returns 401, it should just return that.
If api/token returns 200, it should run original fetch again

You can simple use originalFetch for token and await for response if response is 401 then you simply return empty response to first fetch call else you updated token and then let it go to next condition which will rerun old request.
let TEMP_API = {
'401': {
url: 'https://run.mocky.io/v3/7a98985c-1e59-4bfb-87dd-117307b6196c',
args: {}
},
'200': {
url: 'https://jsonplaceholder.typicode.com/todos/2',
args: {}
},
'404': {
url: 'https://jsonplaceholder.typicode.com/todos/1',
args: {
method: "POST",
credentials: "include"
}
}
}
const originalFetch = fetch;
fetch = function() {
let self = this;
let args = arguments;
return originalFetch.apply(self, args).then(async function(data) {
if (data.status === 200) console.log("---------Status 200----------");
if (data.status === 401) {
// request for token with original fetch if status is 401
console.log('failed');
let response = await originalFetch(TEMP_API['200'].url, TEMP_API['200'].args);
// if status is 401 from token api return empty response to close recursion
console.log("==========401 UnAuthorize.=============");
console.log(response);
if (response.status === 401) {
return {};
}
// else set token
// recall old fetch
// here i used 200 because 401 or 404 old response will cause it to rerun
// return fetch(...args); <- change to this for real scenarios
// return fetch(args[0], args[1]); <- or to this for real sceaerios
return fetch(TEMP_API['200'].url, TEMP_API['200'].args);
}
// condition will be tested again after 401 condition and will be ran with old args
if (data.status === 404) {
console.log("==========404 Not Found.=============");
// here i used 200 because 401 or 404 old response will cause it to rerun
// return fetch(...args); <- change to this for real scenarios
// return fetch(args[0], args[1]); <- or to this for real scenarios
return fetch(TEMP_API['200'].url, TEMP_API['200'].args);
sceaerios
} else {
return data;
}
});
};
(async function() {
console.log("==========Example1=============");
let example1 = await fetch(TEMP_API['404'].url, TEMP_API['404'].args);
console.log(example1);
console.log("==========Example2=============");
let example2 = await fetch(TEMP_API['200'].url, TEMP_API['200'].args);
console.log(example2);
console.log("==========Example3=============");
let example3 = await fetch(TEMP_API['401'].url, TEMP_API['401'].args);
console.log(example3);
})();
Example1 request made to api for 404 status which will cause the 404 condition to run which will then call 200 api after which response will be returned
Example2 request made to 200 api which will return 200 status code which will cause 200 condition to pass and run and return response
Example3 request made to api for 401 status which will cause 401 condition to pass which will then call 200 api and print response after which it will fall out of condition where you can set token which will then be used in another fetch request

Try retuning the fetch promise instead of awaiting that.
(function () {
const originalFetch = fetch;
fetch = function () {
return originalFetch.apply(this, arguments).then(function (data) {
if (data.status === 200) console.log("---------Status 200----------");
if (data.status === 404) {
console.log("==========404 Not Found.=============");
return fetch(`https://jsonplaceholder.typicode.com/todos/2`);
} else {
return data;
}
});
};
})();
function test(id) {
//will trigger 404 status
return fetch(`https://jsonplaceholder.typicode.com/todos/` + id, {
method: "POST",
credentials: "include",
});
}
test(1).then((i) => console.log(i));

Interceptor library for the native fetch command. It patches the global fetch method and allows you the usage in Browser, Node and Webworker environments.
fetch-retry It wraps any Fetch API package (eg: isomorphic-fetch, cross-fetch, isomorphic-unfetch and etc.) and retries requests that fail due to network issues. It can also be configured to retry requests on specific HTTP status codes.

Get only HTML in single fetch request in service worker

I'm using Cloudflare service workers and I want on every request to:
request only the HTML (therefore count as only 1 request)
search the response for a string
Purge that page's cache if the message exists
I've solved points #2 and #3. Can't figure out if #1 is feasible or possible at all.
I need it as only one request because there is a limit per day on the number of free requests. Otherwise I have about 50-60 requests per page.
My current attempt for #1, which doesn't work right:
async function handleRequest(request) {
const init = {
headers: {
'content-type': 'text/html;charset=UTF-8',
},
};
const response = await fetch(request);
await fetch(request.url, init).then(function(response) {
response.text().then(function(text) {
console.log(text);
})
}).catch(function(err) {
// There was an error
console.warn('Something went wrong.', err);
});
return response;
}
addEventListener('fetch', event => {
return event.respondWith(handleRequest(event.request))
});

You can't request "only the html", the worker will act on any request that matches the route that it is deployed at. If you only care about the html, you will need to set up your worker path to filter to only the endpoints that you want to run the worker on.
Alternatively you can use the worker on every request and only do your logic if the response Content-Type is one that you care about. This would be something along these lines:
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request));
})
async function handleRequest(request) {
let response = await fetch(request);
let type = response.headers.get("Content-Type") || "";
if (type.startsWith("text/")) {
//this is where your custom logic goes
}
return response;
}

Use fetch to get HTTP results before the end of the request

I'm trying to consume the /api/stream endpoint of a Drone 1.0 server. This endpoint keeps the HTTP connection opened and just streams new upcoming events to notify the consumer of events.
I tried with that piece of code using the javascript Fetch API
await fetch("https://drone.company.com/api/stream/", {
headers
})
.then(function(response) {
return response.text();
})
.then(function(data) {
console.log(data);
});
This code works well, but the then callback is always called after the request has ended.
Is there a way I can make fetch a stream of the body received while the request is processing ?

you can read from response.getReader() like
let response = await fetch("https://drone.company.com/api/stream/", {
headers
});
const reader = response.body.getReader();
while(true) {
const {done, value} = await reader.read();
if (done) {
break;
}
const text = new TextDecoder("utf-8").decode(value);
console.log(`Received ${text}`)
}
we need to use TextDecoder since value contains Uint8Array data not text. Is not supported by IE. fetch is not supported either btw.
based on https://javascript.info/fetch-progress

React Native Http Interceptor

Like most applications, I'm writing an application that requires a lot of similar logic in the http response/requests handlers. For instance, I have to always check for refresh tokens and save them to the AsyncStorage, or always set the headers to my AuthService headers, or even check for 404 to route to the same 404 error page.
I'm a big fan of the http interceptor in Angular; where you can define and register an http interceptor to (lack of a better term) intercept all http traffic and then run the combined, common logic.
I have 2 main questions:
Since in React Native, we define these independent components, should we not be extracting common http logic in the first place in order to preserve the re-usability of the component?
If we don't want to duplicate code, is there a way in React Native (first) or Objective-C/Swift (second) to intercept http traffic and provide handlers for the requests?

Have you considered axios if you are trying to intercept only xhr?
I am using axios interceptors - https://www.npmjs.com/package/axios
and so far it seems to work.
Here is the sample code
import axios from 'axios';
import promise from 'promise';
// Add a request interceptor
var axiosInstance = axios.create();
axiosInstance.interceptors.request.use(function (config) {
// Do something before request is sent
//If the header does not contain the token and the url not public, redirect to login
var accessToken = getAccessTokenFromCookies();
//if token is found add it to the header
if (accessToken) {
if (config.method !== 'OPTIONS') {
config.headers.authorization = accessToken;
}
}
return config;
}, function (error) {
// Do something with request error
return promise.reject(error);
});
export default axiosInstance;
and then import this axiosInstance where ever you want to make xhr calls

I'm not sure if I'm understanding this question correctly, or if your looking for more magic, but it sounds like you just want a wrapper to the XMLHttpRequest (or fetch API). Wrap it in a class or a function and you can do whatever you want, whenever you want. Here's an example of an xhr wrapped in a promise:
function request(url, method = "GET") {
const xhr = new XMLHttpRequest();
// Do whatever you want to the xhr... add headers etc
return new Promise((res, rej) => {
xhr.open(method, url);
xhr.onload = () => {
// Do whatever you want on load...
if (xhr.status !== 200) {
return rej("Upload failed. Response code:" + xhr.status);
}
return res(xhr.responseText);
};
xhr.send();
});
}
Then you can just use that whenever you want to do HTTP calls...
request("http://blah.com")
.then(data => console.log(`got data: ${data}`))
.catch(e => console.error(`error: ${e}`));

you can use react-native-easy-app that is easier to send http request and formulate interception request
import {XHttpConfig} from 'react-native-easy-app';
XHttpConfig.initHttpLogOn(true) // Print the Http request log or not
.initBaseUrl(ApiCredit.baseUrl) // BaseUrl
.initContentType(RFApiConst.CONTENT_TYPE_URLENCODED)
.initHeaderSetFunc((headers, request) => {
// Set the public header parameter here
})
.initParamSetFunc((params, request) => {
// Set the public params parameter here
})
.initParseDataFunc((result, request, callback) => {
let {success, json, response, message, status} = result;
// Specifies the specific data parsing method for the current app
});
* Synchronous request
const response = await XHttp().url(url).execute('GET');
const {success, json, message, status} = response;
* Asynchronous requests
XHttp().url(url).get((success, json, message, status)=>{
if (success){
this.setState({content: JSON.stringify(json)});
} else {
showToast(msg);
}
});