I'm trying to use a Cloudflare Worker to proxy a POST request to another server.
It is throwing a JS exception – by wrapping in a try/catch blog I've established that the error is:
TypeError: A request with a one-time-use body (it was initialized from a stream, not a buffer) encountered a redirect requiring the body to be retransmitted. To avoid this error in the future, construct this request from a buffer-like body initializer.
I would have thought this could be solved by simply copying the Response so that it's unused, like so:
return new Response(response.body, { headers: response.headers })
That's not working. What am I missing about streaming vs buffering here?
addEventListener('fetch', event => {
var url = new URL(event.request.url);
if (url.pathname.startsWith('/blog') || url.pathname === '/blog') {
if (reqType === 'POST') {
event.respondWith(handleBlogPost(event, url));
} else {
handleBlog(event, url);
}
} else {
event.respondWith(fetch(event.request));
}
})
async function handleBlog(event, url) {
var newBlog = "https://foo.com";
var originUrl = url.toString().replace(
'https://www.bar.com/blog', newBlog);
event.respondWith(fetch(originUrl));
}
async function handleBlogPost(event, url) {
try {
var newBlog = "https://foo.com";
var srcUrl = "https://www.bar.com/blog";
const init = {
method: 'POST',
headers: event.request.headers,
body: event.request.body
};
var originUrl = url.toString().replace( srcUrl, newBlog );
const response = await fetch(originUrl, init)
return new Response(response.body, { headers: response.headers })
} catch (err) {
// Display the error stack.
return new Response(err.stack || err)
}
}
A few issues here.
First, the error message is about the request body, not the response body.
By default, Request and Response objects received from the network have streaming bodies -- request.body and response.body both have type ReadableStream. When you forward them on, the body streams through -- chunks are received from the sender and forwarded to the eventual recipient without keeping a copy locally. Because no copies are kept, the stream can only be sent once.
The problem in your case, though, is that after streaming the request body to the origin server, the origin responded with a 301, 302, 307, or 308 redirect. These redirects require that the client re-transmit the exact same request to the new URL (unlike a 303 redirect, which directs the client to send a GET request to the new URL). But, Cloudflare Workers didn't keep a copy of the request body, so it can't send it again!
You'll notice this problem doesn't happen when you do fetch(event.request), even if the request is a POST. The reason is that event.request's redirect property is set to "manual", meaning that fetch() will not attempt to follow redirects automatically. Instead, fetch() in this case returns the 3xx redirect response itself and lets the application deal with it. If you return that response on to the client browser, the browser will take care of actually following the redirect.
However, in your worker, it appears fetch() is trying to follow the redirect automatically, and producing an error. The reason is that you didn't set the redirect property when you constructed your Request object:
const init = {
method: 'POST',
headers: event.request.headers,
body: event.request.body
};
// ...
await fetch(originUrl, init)
Since init.redirect wasn't set, fetch() uses the default behavior, which is the same as redirect = "automatic", i.e. fetch() tries to follow redirects. If you want fetch() to use manual redirect behavior, you could add redirect: "manual" to init. However, it looks like what you're really trying to do here is copy the whole request. In that case, you should just pass event.request in place of the init structure:
// Copy all properties from event.request *except* URL.
await fetch(originUrl, event.request);
This works because a Request has all of the fields that fetch()'s second parameter wants.
What if you want automatic redirects?
If you really do want fetch() to follow the redirect automatically, then you need to make sure that the request body is buffered rather than streamed, so that it can be sent twice. To do this, you will need to read the whole body into a string or ArrayBuffer, then use that, like:
const init = {
method: 'POST',
headers: event.request.headers,
// Buffer whole body so that it can be redirected later.
body: await event.request.arrayBuffer()
};
// ...
await fetch(originUrl, init)
A note on responses
I would have thought this could be solved by simply copying the Response so that it's unused, like so:
return new Response(response.body, { headers: response.headers })
As described above, the error you're seeing is not related to this code, but I wanted to comment on two issues here anyway to help out.
First, this line of code does not copy all properties of the response. For example, you're missing status and statusText. There are also some more-obscure properties that show up in certain situations (e.g. webSocket, a Cloudflare-specific extension to the spec).
Rather than try to list every property, I again recommend simply passing the old Response object itself as the options structure:
new Response(response.body, response)
The second issue is with your comment about copying. This code copies the Response's metadata, but does not copy the body. That is because response.body is a ReadableStream. This code initializes the new Respnose object to contain a reference to the same ReadableStream. Once anything reads from that stream, the stream is consumed for both Response objects.
Usually, this is fine, because usually, you only need one copy of the response. Typically you are just going to send it to the client. However, there are a few unusual cases where you might want to send the response to two different places. One example is when using the Cache API to cache a copy of the response. You could accomplish this by reading the whole Response into memory, like we did with requests above. However, for responses of non-trivial size, that could waste memory and add latency (you would have to wait for the entire response before any of it gets sent to the client).
Instead, what you really want to do in these unusual cases is "tee" the stream so that each chunk that comes in from the network is actually written to two different outputs (like the Unix tee command, which comes from the idea of a T junction in a pipe).
// ONLY use this when there are TWO destinations for the
// response body!
new Response(response.body.tee(), response)
Or, as a shortcut (when you don't need to modify any headers), you can write:
// ONLY use this when there are TWO destinations for the
// response body!
response.clone()
Confusingly, response.clone() does something completely different from new Response(response.body, response). response.clone() tees the response body, but keeps the Headers immutable (if they were immutable on the original). new Response(response.body, response) shares a reference to the same body stream, but clones the headers and makes them mutable. I personally find this pretty confusing, but it's what the Fetch API standard specifies.
I want to make an ajax request from IBM Connections XCC:
let api = 'https://my-server2/api.xml'
var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange = () => {
if (xmlhttp.readyState == XMLHttpRequest.DONE)
if (xmlhttp.status == 200) {
console.log(xmlhttp.responseText)
}else {
console.log(`Error: ${xmlhttp.readyState}`)
}
}
Result in the network tab is a request to https://connections-host/communities/ajaxProxy/https/my-server2/api.xml so the request is proxied over the connections server. Because of this I get an empty API result since I need an authorized user session. My idea was: The user is logged in in his browser on my-server2 application. So when making an ajax request to my-server2, I can get the API information in his user context.
So my question is: How can I bypass those proxy?
Since I don't set it, I assume that connections manipulate the XMLHttpRequest class in a way like this: https://gist.github.com/dewdad/8830348
I want to view it's code to see the manipulation with this code in the console, but it only shows native code
window.XMLHttpRequest.prototype.open.toString()
"function open() {
[native code]
}"
Connections uses an AJAX proxy to control what's sent out to non-Connections sites/apps. You can configure it for your site to allow specific methods, headers and cookies to be sent to the non-Connections site. I'd take a look at this document on Connections 6.0 https://www.ibm.com/support/knowledgecenter/en/SSYGQH_6.0.0/admin/secure/t_admin_config_ajax_proxy_feature.html
I think that should help you get what you want.
I have a JSON script loaded from an external website. In its simplest form, the code has been like this (and working):
jQuery.getJSON("http://adressesok.posten.no/api/v1/postal_codes.json?postal_code=" + document.querySelector("input").value + "&callback=?",
function(data){
document.querySelector("output").textContent = data.postal_codes[0].city;
});
However, the website owner don't want jQuery if it's not crucial, so I recoded .getJSON to the request = new XMLHttpRequest(); model:
request = new XMLHttpRequest();
request.open("GET", "http://adressesok.posten.no/api/v1/postal_codes.json?postal_code=" + document.querySelector("input").value + "&callback=?", true);
request.onload = function() {
var data = JSON.parse(request.responseText);
document.querySelector("output").textContent = data.postal_codes[0].city;
};
request.onerror = function() { /* this gets called every time */ };
I've modified my code many times, read documentations over and over again, yet the .onerror function is the only one always displaying. This is the console:
Which in Norwegian says that this script requested CORS, that it can't find the origin in the head of Access-Control-Allow-Origin, and that the XMLHttpRequest had a network error, and says "no access".
There could be several reasons as to why this occurs:
1: There's something wrong with the new code
2: There's something in the .getJSON jQuery function (a hack?) that prevents the error from happening
3: There's something crucial in the new code that I have forgot adding
4: There's something with my browser (IE 11 at the moment)
5: Something else?
It would be lovely with some help on this.
DEMO: http://jsbin.com/muxigulegi/1/
That isn't a network error. It's a cross origin error. The request is successful but the browser is denying access to the response to your JavaScript.
Since you have callback=? in the URL, jQuery will generate a JSONP request instead of an XMLHttpRequest request. This executes the response as a script instead of reading the raw data.
You are manually creating an XMLHttpRequest, so it fails due to the Same Origin Policy.
Create a JSONP request instead.
From http://api.jquery.com/jquery.getjson/:
JSONP
If the URL includes the string "callback=?" (or similar, as defined by the server-side API), the request is treated as JSONP instead. See the discussion of the jsonp data type in $.ajax() for more details.
You do have a callback. Which means that the JQuery function can request data from another domain, unlike your XHR call.
Getting Access Denied Error on https in IE. while executing ajax call (to my server1) response which has another ajax call While my Ajax call url is on http .By implementing cors I able to execute on http.Tried JSONP also but then it execute my response till it find another ajax call .How to solve ?
below is on myserver1
alert('aaa');
var xhr2 = new XMLHttpRequest();
xhr2.open('GET', 'http://myserver2.com', true);
xhr2.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
xhr2.send();
xhr2.onload = function () {
alert('fff');
var appCreds = xhr2.responseText;
alert(appCreds);
};
alert('xxx');
In addition to the trusted site requirement I found that the problem was not fixed until I used the same protocol for the request as my origin, e.g. my test site was hosted on a https but failed with any destination using http (without the s).
This only applies to IE, Chrome just politely logs a warning in the debug console and doesn't fail.
i try to make a Firefox Addon which runs a XMLHttp Request in Javascript. I want to get the data from this request and send it to *.body.innerhtml.
That's my code so far...
var xhr = new XMLHttpRequest();
xhr.open("GET", "http://xx.xxxxx.com", true);
xhr.send();
setTimeout(function() { set_body(xhr.responseHtml); }, 6000);
Instead of receiving the data, I get "undefined". If I change xhr.responseHtml to responseText I get nothing. I don't know why I'm getting nothing. I'm working on Ubuntu 12.04 LTS with Firefox 12.0.
If you need any more details on the script please ask!
Update:
set_body Function
document.body.innerHTML = '';
document.body.innerHTML = body;
document.close();
Update SOLVED:
I had to determine the RequestHeaders (right after xhr.open):
xhr.setRequestHeader("Host", "xxx");
For following Items: Host, Origin and Referer. So it seems there was really a problem with the same origin policy.
But now it works! Thanks to all!
when you set the last param of open to true you are asking for an async event. So you need to add a callback to xhr like so:
xhr.onReadyStateChange = function(){
// define what you want to happen when server returns
}
that is invoked when the server responds. To test this without async set the third param to false. Then send() will block and wait there until the response comes back. Setting an arbitrary timeout of 6 seconds is not the right way to handle this.
This code should work:
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function () {
if (xhr.readyState == 4) {
set_body(xhr.responseText);
}
};
xhr.open("GET", "http://xx.xxxxx.com", true);
xhr.send();
Make sure that you are getting a correct response from URL http://xx.xxxxx.com. You may have a problem with cross-domain calls. If you have a page at domain http://first.com and you try to do XMLHttpRequest from domain http://second.com, Firefox will fail silently (there will be no error message, no response, nothing). This is a security measure to prevent XSS (Cross-site scripting).
Anyway, if you do XMLHttpRequest from a chrome:// protocol, it is considered secure and it will work. So make sure you use this code and make the requests from your addon, not from your localhost or something like that.