I've stumbled across some weird behaviour while using axios to fetch files from an
external endpoint.
The endpoints url looks something like this: https://example.com/path/to/file/filename.pdf
When accessing the url using the browser, a download dialog would pop up and request a confirmation in order to download the file.
While this simple GET request seemed to work fine, doing the same using axios did not.
const res = await axios.get("https://example.com/path/to/file/filename.pdf", { responseType: "blob" }); // throws a 400 error
After doing some further digging I decided to try out requesting the url from the same machine using python - this worked without an issue.
import requests
url = 'https://example.com/path/to/file/filename.pdf'
r = requests.get(url)
open('test.pdf', 'wb').write(r.content)
The solution
After some (more) frustration and searching the web for issues without any success I've finally discovered the issue and its solution.
The issue only occured when the filename in the specified url included german characters such as ä, ö or ü. Passing the URL to encodeURI and then calling axios with the returned value did the trick.
const res = await axios.get(encodeURI("https://example.com/path/to/file/filenäme.pdf"), { responseType: "blob" }); // works even with ä, ö, ü in the url
The remaining question
The question that remains is why? Checking the outgoing network requests reveals that the requests seem to go to the same url, meaning the URL in chromes network tab always looked the same. The filename "Lösung" was encoded to be "L%C3%B6sung", no matter if the request was run using the browser, a webpage (axios in the browser), a Node app (axios /w node) or Python. Still, axios seems to handle the encoding different to the other methods.
Perhaps Chromes Network tools parse the displayed url too, meaning the actual request was sent to /Lösung.pdf but was displayed as /L%C3%B6sung.pdf while viewing the outgoing network requests? Is there a setting for URL parsing in the axios configuration I might have missed?
Related
I have a simple flask server that returns a JSON (flask automatically do that when you return a python dict) when it receives a GET request to / endpoint.
It's running on my 5000 port:
I know it's running and reachable as I managed to request to it, and receive a valid response, using curl, twice:
Both requests are logged into the server logs as well, on second print.
I'm trying to fetch to my server from a html/js script as:
const URL = "http://127.0.0.1:5000/"
fetch( URL )
.then(response=>response.json())
.then(json=>console.log(json))
or
const URL = "http://localhost:5000/"
fetch( URL )
.then(response=>response.json())
.then(json=>console.log(json))
But I get the same error twice:
1: GET http://localhost:5000/ net::ERR_CONNECTION_REFUSED
2: Uncaught (in promise) TypeError: Failed to fetch
I know the code itself works because I managed to fetch Githubs API:
const URL = "https://api.github.com/users/nluizsoliveira"
fetch( URL )
.then(response=>response.json())
.then(json=>console.log(json))
I'm not sure why I can't fetch to my localhost. I think it has something to do with how cloud9 deals with ports. When removing http://localhost from the url:
const URL = ":5000"
fetch( URL )
.then(response=>response.json())
.then(json=>console.log(json))
the snippet also fails, but it seems that the request is somehow appending the url to my C9 url.
Have someone faced that situation before?
Thanks a lot in advance!
EDIT:
Just to clarify, i'm not running that js/html (directly) on my browser tab. I'm running it on the C9's built in browser, which is available through "preview running application":
With AWS Cloud 9 Preview, AWS gives you a private link like https://12a34567b8cd9012345ef67abcd890e1.vfs.cloud9.us-east-2.amazonaws.com which gives you access to your application. To get that link click preview application and copy it from the browser tab.
Use that link in your code instead of localhost. AWS Documentation:Preview a running application.
I seem to be missing a key piece here. Any ideas?
componentDidMount(){this.retrieveConfig();}
retrieveConfig = () => {const { locnNbr } = this.props.data;
axios.get(`/some/uri/stuff`).then(res =>
{console.log('retrieveConfig res', res);
this.setState({config: res. data});
});
};
componentDidMount = () => {
this.retrieveConfig();
};
retrieveConfig = () => {
Axios.get("https://jsonplaceholder.typicode.com/todos/1")
.then(res => console.log(res.data))
.catch(err => console.log(err));
};
I hope this fixes your problem.
Html response you got only for mobile and it will work from postman because the url you are trying to hit from server mobile will not able to hit. So you add the cer to your mobile.
public certificate of server where as postman allows it but mobile does not allow to hit all server urls
Consider double checking your URLs.
I'm assuming you're trying to reach a REST API endpoint but you're getting a web page as your response, which for me would suggest you're hitting the wrong port on the right domain. I believe Axios defaults to using port 8080 (web default), and it's unlikely your API is running on that port. So make sure you're specifying your URL like:
Axios.get("http://sub.domain.com:123/endpoint").then...
If you could share which URL you're using in browser and in Postman that might give more information on what's going on differently there.
The 200/304 thing makes sense from that perspective as well. It's a 200 since the request went fine but Chrome is caching the request for that web page and it's reporting that it hasn't changed (that's what 304 means).
Sometimes I come across this issue in Firefox Developer Edition and I have not figured out why this happens (I believe it has to do with the response headers).
I manage to fix it every time by clearing the browser's cache.
I have a generated React site I am hosting in an S3 bucket. One of my components attempts to fetch something when loaded:
require('isomorphic-fetch')
...
componentDidMount() {
fetch(`${url}`)
.then(res => {
console.log(res);
this.setState({
users: res
})
})
.catch(e => {
// do nothing
})
}
The url I am fetching is an AWS API Gateway. I have enabled CORS there, via the dropdown, with no changes to the default configuration.
In my console, for both the remote site and locally during development, I see:
"Failed to load url: No 'Access-Control-Allow-Origin' header is present on the requested resource." etc
However, in the Chrome Network tab, I can see the request and the response, with status 200, etc. In the console, my console.log and this.setState are never called, however.
I understand that CORS is a common pain point, and that many questions have touched on CORS. My question: Why does the response show no error in the Network tab, while simultaneously erroring in the console?
The fetch(`${url}`) call returns a promise that resolves with a Response object, and that Response object provides methods that resolve with text, JSON data, or a Blob.
So to get the data you want, you need to do something like this:
componentDidMount() {
fetch(`${url}`)
.then(res => res.text())
.then(text => {
console.log(text);
this.setState({
users: text
})
.catch(e => {
// do nothing
})
}
Failed to load url: No 'Access-Control-Allow-Origin' header is present on the requested resource." etc
That means the browser isn’t allowing your frontend code to access the response from the server, because the response doesn’t include the Access-Control-Allow-Origin header.
So in order for the above code to work, you’ll need to fix the server configuration on that server so that it sends the necessary Access-Control-Allow-Origin response header.
However, in the Chrome Network tab, I can see the request and the response, with status 200, etc. In the console, my console.log and this.setState are never called, however.
That’s expected in the case where the server doesn’t send the Access-Control-Allow-Origin response header. In that case, the browser still gets the response — and that’s why you can see it in the devtools Network tab — but just because the browser gets the response doesn’t mean it will expose the response to your frontend JavaScript code.
The browser will only let your code access the response if it includes the Access-Control-Allow-Origin response header; if the response doesn’t include that header, then the browser blocks your code from accessing it.
My question: Why does the response show no error in the Network tab, while simultaneously erroring in the console?
For the reason outlined above. The browser itself runs into no error in getting the response. But your code hits an error because it’s trying to access an res object that’s not there; the browser hasn’t created that res object, because the browser isn’t exposing the response to your code.
You may be seeing the status 200 for the OPTIONS not the GET. There is a setting for CORS to handle legacy, so it won't confuse your client. I had to do that last time in a React app. Your error is that your CORS isn't configured properly (sorry, obviously). Chrome won't let your client tlak to the backend if it doesn't get the headers properly. Other browsers probably also, probably React also. It may be some kind of HTTP protocol if only one side has CORS enabled. Someone can correct me there. It's a similar security consideration as sending a request to HTTP from HTTPS. Chrome blocks it.
It looks to me like it's your backend. CORS isn't active or it would put that header on, and after that, you would see errors about origin mismatch in the frontend client.
In my experience, it's a 2-3 step combo, make sure OPTIONS don't send confusing signals to your client (look for settings to do with 200). This is a config setting in your backend. Then, make sure the backend is configured to use CORS. You very specifically need to enter the origin hostname and port that the backend is to expect traffic from.
I could probably give better input if I see what languages and/or frameworks you are using besides React.
This is what you would do in Express JS and node for your Backend:
const cors = require('cors')
// note http or https
app.use(cors({
origin: 'http://example.com:1337',
//origin: '*',
methods: 'GET,HEAD,PUT,PATCH,POST,DELETE',
optionsSuccessStatus: 200
// some legacy browsers (IE11, various SmartTVs) choke on 204
}))
My last React app was detonating without optionsSuccessStatus by throwing success when it was fail.
To give you a little bit of imagery to work with, CORS is simple but finicky. It's a simple matter of alignment. Once your backend is configured to a) use CORS and b) know who to accept traffic from, it's done. Once your frontend is configured to handle this traffic, it's done. It's like aligning a square peg in a round hole until you get the config settings aligned.
Try using Postman to send some GET requests to the Backend. You can observe the headers from there.
I think this question will be easy for someone and will be a face-palm situation for me.
I have a Laravel 5.3 site, and various pages have ajax requests. Because I use the csrf_field() feature, they work fine.
But there is one page where the ajax produces this error:
Mixed Content: The page at 'https://example.com/fb/reports' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://example.com/fb/json?levelType=&id=&aggLevel=ad&start=&end='. This request has been blocked; the content must be served over HTTPS.
My javascript looks like this:
var relUrl = '/fb/json/';
var payload = {
levelType: levelType,
id: id,
aggLevel: aggLevel,
start: start,
end: end
};
console.log(relUrl);
return $.ajax({
type: 'GET',
dataType: 'json',
data: payload,
url: relUrl,
headers: {
'X-CSRF-TOKEN': $('meta[name="csrf-token"]').attr('content')
}
});
I've read tons of articles about this error. I've tried tons of suggested solutions, including changing the relative URL to the full https URL, or starting it with 2 slashes.
I've even tried changing the way my Laravel routes work and am now using just querystring parameters.
I've studied all of the articles below (and more).
Also, since this one ajax query is in a password-protect part of the site (and the ajax queries that work are in a public/open part of the site), I figured maybe that was related to the problem. But then I used SSH to log into the production server and vim to temporarily remove the line that required any authentication, and the https error still happens.
What steps can I take to debug further from here? What logs can I 'tail' on my Cloudways server?
Is there anything that Cloudflare might be interfering with (which I doubt, since other ajax queries work, all on https)?
Thanks!
jQuery AJAX Request to HTTPS getting served to HTTP with Laravel and Select2
This request has been blocked; the content must be served over HTTPS
Mixed content issue - insecure XMLHttpRequest endpoint
XHR response blocked by Chrome, because of mixed content issue (http/https)
Forcing AJAX call to be HTTPS from HTTPS Page
MixedContent when I'm loading https page through ajax, but browser still thinks it's http
jQuery ajax won't make HTTPS requests
Laravel 5.1 ajax url parameter is url
Summary:
I needed to replace var relUrl = '/fb/json/'; with var relUrl = '/fb/json'; (remove the trailing slash) because that's what my Laravel web.php routes file expected.
In Chrome console, I noticed that the https XHR request was being "canceled" and replaced with an http request.
So then I used ssh to log into the remote production server and vim to temporarily disable the requirement of authentication.
Then in the Chrome console, I defined and ran a new ajax command using an absolute https URL with querystring params on the end. That worked (no mixed content error). Then I tried a relative URL like that, and it worked too.
Even a relative URL with no payload or querystring params or trailing slash worked.
Then I added the trailing slash again, and it didn't work.
I still wish there had been an easier way to trace or debug the redirect paths or whatever was happening. I still feel like I stumbled onto the answer clumsily (after many hours) instead of knowing how to dissect this problem reliably.
When changing from HTTP to HTTPS, it's possible to get the problem Mixed content issue - Content must be served as HTTPS.
So, first, modify APP_URL in the .env file, if we use the assets helper, this shouldn't give any problem with the URL.
APP_URL=https://url.net
Finally, add the following to the beginning of **api.php** or **web.php**:
if (App::environment('production')) {
URL::forceScheme('https');
}
I want the user to be able to enter their website URL into an input box that is part of a Chrome Extension and the Chrome extension will use an AJAX request or something similar to detect and tell the user if the server behind the URL supports sending responses via HTTP2. Is this possible?
Maybe the WebRequest has a way of picking up this information? Or the new Fetch API? Could your request tell the server somehow that only HTTP2 replies are understood? I can't see an obvious way.
I know you can use window.chrome.loadTimes().connectionInfo to get the protocol of the current page but this requires loading the whole page which I don't want to do.
Example URLS:
Delivered over HTTP2: https://cdn.sstatic.net/
Delivered over HTTP 1.1: https://stackoverflow.com/
HTTP/2 responses require a "status" response header - https://http2.github.io/http2-spec/#HttpResponse, so to check whether the response is using HTTP/2, you can use the chrome.webRequest.onHeadersReceived event with "responseHeaders" in extraInfoSpec. For example, with your test cases:
chrome.webRequest.onHeadersReceived.addListener(function(details) {
var isHttp2 = details.responseHeaders.some(function(header) {
return header.name === 'status';
});
console.log('Request to ' + details.url + ', http2 = ' + isHttp2);
}, {
urls: ['https://cdn.sstatic.net/*', 'http://stackoverflow.com/*'],
types: ['xmlhttprequest']
}, ['responseHeaders']);
// Tests:
fetch('http://stackoverflow.com');
fetch('https://cdn.sstatic.net');
EDIT: Apparently you can do this with the iframe and webRequest trick! I found a reference gist (but I haven't tested it myself though):
https://gist.github.com/dergachev/e216b25d9a144914eae2
OLD ANSWER
You probably won't able able to do this without an external API. Here's why
1) Using ajax only requires that the server of the url to be tested sends CORS headers back to the user, otherwise the browser will not accept it.
2) You could create an iframe on the fly and use chrome.loadTimes().connectionInfo in the iframe contentWindow but if the server sends X-Frame-Options: Deny header the browser won't let you load the url in the iframe either.
3) Stripping the X-frame headers via webRequest API as mentioned here
Getting around X-Frame-Options DENY in a Chrome extension?
will likely not work, afaik Chrome extension are not allowed to modify the response body.
Possible solutions
1) The problems above could be solved using a simple proxy that adds the appropriate headers. Here's a reference on how to do it using Nginx
http://balaji-damodaran.com/programming/2015/07/30/nginx-headers.html
2) Just create a custom API that does the request for you server-side and parses the result to check for http2 support. If your extension gets popular it would still be fairly easy to scale it up e.g via caching and horizontal scaling.
Hope this helps!