I'm trying to scrape OG data using open-graph-scraper and Vue.js, but it gets blocked because of CORB in Chrome. It is working fine for scripts I run using Node, but is there a work around for this problem or another to get OG data from an input URL?
Cross-Origin Read Blocking (CORB) blocked cross-origin response SOME-INPUT-URL with MIME type text/html. See https://www.chromestatus.com/feature/5629709824032768 for more details.
CORB is a standard that is based around the idea
"what I asked for is what I received"
if you are troubleshooting CORB errors in chrome it means that your request method is being responded to with either no content type or a content type that does not correspond to your method of request.
in a 3rd party API what I would recommend is using a tool like postman (unaffiliated) to inspect the incoming content type in the response headers. Once you know the content-type header of the call then tweak your request method in an attempt to match the expected output. Many API's use poorly implemented frameworks so just because the response is well formed JSON doesn't mean that the header will say JSON. I often see json outputs where the headers are 'text/plain'.
If this doesn't work, for example some api's fail to respond with content headers at all
- another method would be to use an intermediary call to make the request. For example using a combination of the AWS API gateway and a AWS Lambda function you can create a route that will make the request using node but where you then gain the total control over both the response headers and body. You can then add the content header you desire and pass it back to your client.
ref:
https://www.chromium.org/Home/chromium-security/corb-for-developers
Related
Okay so im trying to get access token im following the main guide provided by facebook app developers which is: https://developers.facebook.com/docs/instagram-basic-display-api/getting-started. Im at step: 5, where i try to exchange my code for access_token. I do post request to https://api.instagram.com/oauth/access_token however i get cors blocked with this error:
Access to XMLHttpRequest at
'https://api.instagram.com/oauth/access_token' from origin
'https://localhost' has been blocked by CORS policy: Request header
field content-type is not allowed by Access-Control-Allow-Headers in
preflight response.
I literlly tried everything. Adding headers to axios, using ngrok to get an actual secure https: and not self made cert (for dev mode ofc). I also changed the app ouath urls. What could the problem be?
Yeap i allowed urls i did everything. I tried https://cors-anywhere.herokuapp.com/https://api.instagram.com/oauth/access_token it didn't work, it says bad request. However i managed to solve it with an extra library called axios-oauth-client: https://www.npmjs.com/package/axios-oauth-client. After implementing this library and instead of doing axios.post(url, data) but using this library and passing the data in there everything works fine and i get the access token. Something with the headers might have been the problem
This question already has answers here:
XMLHttpRequest cannot load XXX No 'Access-Control-Allow-Origin' header
(11 answers)
Closed 3 years ago.
I'm unable to retrieve data from the Rescue Time API. I'm making a request in a JavaScript file using the jQuery get() method. Here is a look at the JavaScript related to the API GET request:
$.get('https://www.rescuetime.com/anapi/data?key=########################&format=json&restrict_kind=overview', function(data) {
// callback function code...
});
The "key=########################" is the paramater that includes my API key.
When running the script (either locally or on my personal domain), I receive a cross origin error:
XMLHttpRequest cannot load https://www.rescuetime.com/anapi/data?key=########################&format=json&restrict_kind=overview. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access.
I understand that this is happening because i'm requesting content that is on a different domain than the one that is making the AJAX request. That being said, how do I get around this? I've read the CORS MDN documentation, but could not decode what actionable steps I need to follow in order to resolve this issue.
I need some actionable steps.
Set up a CORS proxy using the code from https://github.com/Rob--W/cors-anywhere/ or similar.
https://cors-anywhere.herokuapp.com/ is a public instance running that code, and the way you could use it is by changing your existing code to this:
$.get('https://cors-anywhere.herokuapp.com/https://www.rescuetime.com/anapi/data?key=########################&format=json&restrict_kind=overview', function(data) {
// callback function code...
});
Be aware though that if you do that, your key would potentially be exposed to the operator of that https://cors-anywhere.herokuapp.com/ instance. So if that’s a concern then don’t try it, and instead set up your own proxy at https://some.url.for.your.proxy and change your code to:
$.get('https://some.url.for.your.proxy/https://www.rescuetime.com/anapi/data?key=########################&format=json&restrict_kind=overview', function(data) {
// callback function code...
});
Either way the result will be that your request gets sent through the specified CORS proxy, which forwards the request to the https://www.rescuetime.com/anapi/data… endpoint and then receives the response. The proxy backend then adds the Access-Control-Allow-Origin header to the response and finally passes that back to your requesting frontend code.
Your browser then allows your frontend code to access the response, because that response with the Access-Control-Allow-Origin response header is what the browser sees. Otherwise, if the response lacks Access-Control-Allow-Origin, browsers won’t let your code access it.
A CORS proxy like that is the only option if you want to make the request from frontend JavaScript code running in a browser, and want to consume the response from that frontend code. Otherwise, without the use of such a proxy, browsers will block your code from accessing the response—because the https://www.rescuetime.com/anapi/data… API endpoint doesn’t itself send the necessary Access-Control-Allow-Origin response header.
Your only other option otherwise is to not make the request from your frontend code but instead make the request from whatever backend server-side code you’re running. In that case there’s no browser in the middle enforcing cross-origin restrictions on the request.
The $http.jsonp method described in the official documentation seems to always perform get requests: http://docs.angularjs.org/api/ng.$http#methods_jsonp.
I have tried setting the config option to 'POST' but it still sends a GET:
$http.jsonp('/api/new?callback=JSON_CALLBACK', {method: 'POST'});
I have also tried setting a data argument in the hope that angular would switch to a POST:
$http.jsonp('/api/new?callback=JSON_CALLBACK', {data: {stuff: true}});
But it still doesn't :)
As for making a post like this:
$http.post('/api/new?callback=JSON_CALLBACK')
It does make a POST obviously but doesn't do the angular magic thingy with the JSON_CALLBACK and produces the following JS error:
No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://xxx.yyy.zzz' is therefore not allowed access.
(The API is not on the same server as the app, that's the point of JSONP).
Google was most unhelpful on this issue and reading through angular's sources is not the easiest task. So how can I make a JSONP POST request with angular?
You cannot make a POST request using JSON-P (with or without Angular)
A JSON-P request works by generating a <script> element with a src attribute. This will always trigger a GET request.
If you want to make a cross-domain POST request with JavaScript then you must use either XMLHttpRequest (and have the server supply suitable access control headers as per the CORS specification) or proxy the request through the server hosting the page.
I tried so many tutorials online but everything I try is really old and fails - I can't even create an XMLHTTPRequest object!
I just want to get the header from google.com - how can I do that?
You would use xhr.getResponseHeader() to get a single header, or xhr.getAllResponseHeaders() to read all of the headers from an XMLHttpRequest response.
The reasons this won't work for you:
XMLHttpRequest is case sensitive. If you are using HTTP in all caps, it will fail.
Unless you are a google employee adding code to google.com, your request falls victim to the same origin policy. You'll have to use your server as a proxy to get headers from a google request.
You cant simply do this by JS. You'll have to use AJAX and do a server request to PHP,ASP, Java or whatever. The XMLHTTPRequest should do it - if you really want to do it manually. But it will really not work with foreign domains, so you are forced to do the XMLHTTTPRequest to a page on your server which will deliver the header.
I have written a web application that posts a file via http to a restful web service. The web service can reply with a 400 or 403 response if the service finds any problems with the request. The response also contains xml describing the reason(s) for replying with a http error code.
My web application posts the file to a hidden iFrame and uses the iFrame's onload event handler to execute a function that parses the server response presented in the iFrame and let's the user know how the file upload went.
My solution works great with firefox and chrome but not in internet explorer 7.
My problem is that if the server responds with an error code e.g. 400 or 403 internet explorer 7 loads its own static error page. This means that my script can't parse the error message sent in the response since the static error page is not from the same domain as the script itself and violates the same origin policy (and since it's a static error page the web service's detailed error message won't be there anyway).
I see only two workarounds to this problem and I would prefer to avoid them both if possible:
A) Have the web service return 200, when the user-agent indicates internet explorer, even though an error has occured but include a xml response that indicates an error.
B) Have the web application post to an "intermediary" that forwards the request to the web service, reads the response and then translates it to a 200 or anything else that works (so it's basically option A but more flexible and at least this keeps the restful web service "clean").
Is there another way to solve my problem?
Assuming you have control of the server, you may find the best solution is to use the iframe only for sending the file (i.e. one way....client to server). Then use an ajax polling solution to determine whether or not the post was successful. It can be a bit messy, but should be much more reliable, and you can also get information back before the post is complete.
I managed to solve this since a colleague remembered that if the response body is not of a certain length when sending a 4xx response, Internet Explorer will load its static error page.
The workaround is to send back a longer response body with your 4xx response, e.g. a xml-comment containing white space.