I'm developing a web app in Angular 10 that works as follows:
I'm dealing with CORS issue. I do not have permission to add code to the server I'm fetching.
I want to be able to:
Fetch the website
Parse the result, and put it in my database
I'm aiming to deploy the solution on an Apache server.
Here is the CORS error I'm dealing with:
Blocking a Cross-Origin Request: The "Same Origin" policy does not
allow viewing the remote resource located at
https://wwwfrance1.CENSORED.eu.com/api/?apikey=CENSORED.
Reason: "Access-Control-Allow-Origin" CORS header is missing. Status
code: 200.
Here is what i've tried:
Using MOSIF mozilla extension (works, but not sustainable for deployment, and for some reason, when I'm ignoring the CORS security, I cannot post on my DB any more)
Adding a header in my fetching request, such as:
/******API SEACH****/
/***Global Update***/
private updateClients() {
let xmlRequestPromise = fetch('https://wwwfrance1.CENSORED.eu.com/api/?apikey=CENSORED&service=list_clients', {
method: 'GET',
headers: {
'Access-Control-Allow-Origin': '*',
}
})
.then(async response => this.clients = this.regexSearchClient(await response.text()))
return xmlRequestPromise
}
But that doesn't work either. I've verified that the header appears in the request.
How to proceed?
What is CORS ?
Cross-origin resource sharing is a mechanism that allows restricted resources on a web page to be requested from another domain outside the domain from which the first resource was served. From wiki
In simple terms only an internal webserver can send Requests which are potentially dangerous to it's web server, and requests from other server's are simply blocked.
But few HTTP requests are allowed ,Few of the allowed methods are GET, HEAD, POST.
How do I resolve the issue ?
Apparently in this circumstance you cannot send a fetch request to a web server having CORS header. Instead you can do a GET request to the web server as a web server having CORS allows HTTP GET requests.
Note - If you send a GET request to a web server using angular in your browser it wouldn't work as browser's convert GET requests into fetch requests and fetch requests aren't allowed from a web server with CORS. Instead send a GET request from a webserver/local machine rather than a browser.
Create your own server and make a route which fetches that API. From your Angular application fetch that route on your server.
You have to use a package as a middleware. If you are using nodejs-framework expressjs.At first, you have to run npm install cors -force.Then add the code that is given bellow:-
const cors=require('cors')
app.use(cors({origin:true}))
I am building a web API. I found whenever I use Chrome to POST, GET to my API, there is always an OPTIONS request sent before the real request, which is quite annoying. Currently, I get the server to ignore any OPTIONS requests. Now my question is what's good to send an OPTIONS request to double the server's load? Is there any way to completely stop the browser from sending OPTIONS requests?
edit 2018-09-13: added some precisions about this pre-flight request and how to avoid it at the end of this reponse.
OPTIONS requests are what we call pre-flight requests in Cross-origin resource sharing (CORS).
They are necessary when you're making requests across different origins in specific situations.
This pre-flight request is made by some browsers as a safety measure to ensure that the request being done is trusted by the server.
Meaning the server understands that the method, origin and headers being sent on the request are safe to act upon.
Your server should not ignore but handle these requests whenever you're attempting to do cross origin requests.
A good resource can be found here http://enable-cors.org/
A way to handle these to get comfortable is to ensure that for any path with OPTIONS method the server sends a response with this header
Access-Control-Allow-Origin: *
This will tell the browser that the server is willing to answer requests from any origin.
For more information on how to add CORS support to your server see the following flowchart
http://www.html5rocks.com/static/images/cors_server_flowchart.png
edit 2018-09-13
CORS OPTIONS request is triggered only in somes cases, as explained in MDN docs:
Some requests don’t trigger a CORS preflight. Those are called “simple requests” in this article, though the Fetch spec (which defines CORS) doesn’t use that term. A request that doesn’t trigger a CORS preflight—a so-called “simple request”—is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (for example, Connection, User-Agent, or any of the other headers with names defined in the Fetch spec as a “forbidden header name”), the only headers which are allowed to be manually set are those which the Fetch spec defines as being a “CORS-safelisted request-header”, which are:
Accept
Accept-Language
Content-Language
Content-Type (but note the additional requirements below)
DPR
Downlink
Save-Data
Viewport-Width
Width
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
No event listeners are registered on any XMLHttpRequestUpload object used in the request; these are accessed using the XMLHttpRequest.upload property.
No ReadableStream object is used in the request.
Have gone through this issue, below is my conclusion to this issue and my solution.
According to the CORS strategy (highly recommend you read about it) You can't just force the browser to stop sending OPTIONS request if it thinks it needs to.
There are two ways you can work around it:
Make sure your request is a "simple request"
Set Access-Control-Max-Age for the OPTIONS request
Simple request
A simple cross-site request is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (e.g. Connection, User-Agent, etc.), the only headers which are allowed to be manually set are:
Accept
Accept-Language
Content-Language
Content-Type
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
A simple request will not cause a pre-flight OPTIONS request.
Set a cache for the OPTIONS check
You can set a Access-Control-Max-Age for the OPTIONS request, so that it will not check the permission again until it is expired.
Access-Control-Max-Age gives the value in seconds for how long the response to the preflight request can be cached for without sending another preflight request.
Limitation Noted
For Chrome, the maximum seconds for Access-Control-Max-Age is 600 which is 10 minutes, according to chrome source code
Access-Control-Max-Age only works for one resource every time, for example, GET requests with same URL path but different queries will be treated as different resources. So the request to the second resource will still trigger a preflight request.
Please refer this answer on the actual need for pre-flighted OPTIONS request: CORS - What is the motivation behind introducing preflight requests?
To disable the OPTIONS request, below conditions must be satisfied for ajax request:
Request does not set custom HTTP headers like 'application/xml' or 'application/json' etc
The request method has to be one of GET, HEAD or POST. If POST, content type should be one of application/x-www-form-urlencoded, multipart/form-data, or text/plain
Reference:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
When you have the debug console open and the Disable Cache option turned on, preflight requests will always be sent (i.e. before each and every request). if you don't disable the cache, a pre-flight request will be sent only once (per server)
Yes it's possible to avoid options request. Options request is a preflight request when you send (post) any data to another domain. It's a browser security issue. But we can use another technology: iframe transport layer. I strongly recommend you forget about any CORS configuration and use readymade solution and it will work anywhere.
Take a look here:
https://github.com/jpillora/xdomain
And working example:
http://jpillora.com/xdomain/
For a developer who understands the reason it exists but needs to access an API that doesn't handle OPTIONS calls without auth, I need a temporary answer so I can develop locally until the API owner adds proper SPA CORS support or I get a proxy API up and running.
I found you can disable CORS in Safari and Chrome on a Mac.
Disable same origin policy in Chrome
Chrome: Quit Chrome, open an terminal and paste this command: open /Applications/Google\ Chrome.app --args --disable-web-security --user-data-dir
Safari: Disabling same-origin policy in Safari
If you want to disable the same-origin policy on Safari (I have 9.1.1), then you only need to enable the developer menu, and select "Disable Cross-Origin Restrictions" from the develop menu.
As mentioned in previous posts already, OPTIONS requests are there for a reason. If you have an issue with large response times from your server (e.g. overseas connection) you can also have your browser cache the preflight requests.
Have your server reply with the Access-Control-Max-Age header and for requests that go to the same endpoint the preflight request will have been cached and not occur anymore.
I have solved this problem like.
if($_SERVER['REQUEST_METHOD'] == 'OPTIONS' && ENV == 'devel') {
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Headers: X-Requested-With');
header("HTTP/1.1 200 OK");
die();
}
It is only for development. With this I am waiting 9ms and 500ms and not 8s and 500ms. I can do that because production JS app will be on the same machine as production so there will be no OPTIONS but development is my local.
You can't but you could avoid CORS using JSONP.
After spending a whole day and a half trying to work through a similar problem I found it had to do with IIS.
My Web API project was set up as follows:
// WebApiConfig.cs
public static void Register(HttpConfiguration config)
{
var cors = new EnableCorsAttribute("*", "*", "*");
config.EnableCors(cors);
//...
}
I did not have CORS specific config options in the web.config > system.webServer node like I have seen in so many posts
No CORS specific code in the global.asax or in the controller as a decorator
The problem was the app pool settings.
The managed pipeline mode was set to classic (changed it to integrated) and the Identity was set to Network Service (changed it to ApplicationPoolIdentity)
Changing those settings (and refreshing the app pool) fixed it for me.
OPTIONS request is a feature of web browsers, so it's not easy to disable it. But I found a way to redirect it away with proxy. It's useful in case that the service endpoint just cannot handle CORS/OPTIONS yet, maybe still under development, or mal-configured.
Steps:
Setup a reverse proxy for such requests with tools of choice (nginx, YARP, ...)
Create an endpoint just to handle the OPTIONS request. It might be easier to create a normal empty endpoint, and make sure it handles CORS well.
Configure two sets of rules for the proxy. One is to route all OPTIONS requests to the dummy endpoint above. Another to route all other requests to actual endpoint in question.
Update the web site to use proxy instead.
Basically this approach is to cheat browser that OPTIONS request works. Considering CORS is not to enhance security, but to relax the same-origin policy, I hope this trick could work for a while. :)
you can also use a API Manager (like Open Sources Gravitee.io) to prevent CORS issues between frontend app and backend services by manipulating headers in preflight.
Header used in response to a preflight request to indicate which HTTP headers can be used when making the actual request :
content-type
access-control-allow-header
authorization
x-requested-with
and specify the "allow-origin" = localhost:4200 for example
One solution I have used in the past - lets say your site is on mydomain.com, and you need to make an ajax request to foreigndomain.com
Configure an IIS rewrite from your domain to the foreign domain - e.g.
<rewrite>
<rules>
<rule name="ForeignRewrite" stopProcessing="true">
<match url="^api/v1/(.*)$" />
<action type="Rewrite" url="https://foreigndomain.com/{R:1}" />
</rule>
</rules>
</rewrite>
on your mydomain.com site - you can then make a same origin request, and there's no need for any options request :)
It can be solved in case of use of a proxy that intercept the request and write the appropriate headers.
In the particular case of Varnish these would be the rules:
if (req.http.host == "CUSTOM_URL" ) {
set resp.http.Access-Control-Allow-Origin = "*";
if (req.method == "OPTIONS") {
set resp.http.Access-Control-Max-Age = "1728000";
set resp.http.Access-Control-Allow-Methods = "GET, POST, PUT, DELETE, PATCH, OPTIONS";
set resp.http.Access-Control-Allow-Headers = "Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since";
set resp.http.Content-Length = "0";
set resp.http.Content-Type = "text/plain charset=UTF-8";
set resp.status = 204;
}
}
What worked for me was to import "github.com/gorilla/handlers" and then use it this way:
router := mux.NewRouter()
router.HandleFunc("/config", getConfig).Methods("GET")
router.HandleFunc("/config/emcServer", createEmcServers).Methods("POST")
headersOk := handlers.AllowedHeaders([]string{"X-Requested-With", "Content-Type"})
originsOk := handlers.AllowedOrigins([]string{"*"})
methodsOk := handlers.AllowedMethods([]string{"GET", "HEAD", "POST", "PUT", "OPTIONS"})
log.Fatal(http.ListenAndServe(":" + webServicePort, handlers.CORS(originsOk, headersOk, methodsOk)(router)))
As soon as I executed an Ajax POST request and attaching JSON data to it, Chrome would always add the Content-Type header which was not in my previous AllowedHeaders config.
Would really appreciate anyone's help. I am relatively new to developing in React, using Mac OSX and Chrome as my browser. I have a small application that attempts to make an async GET request from Yelp Fusion's API using 'isomorphic-fetch', but receive the following error:
Fetch API cannot load https://api.yelp.com/v3/businesses/search?[remaining URL] Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access. The response had HTTP status code 500. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
I have done a lot of searching to see what responses to identical issues already exist, but I am left even more confused in how to tackle my problem with my relatively new knowledge to this kind of development environment. (Answers that seem particularly helpful are: Response to preflight request doesn't pass access control check and API Request with HTTP Authorization Header inside of componentDidMount, but I do not really understand how to actually implement those solutions with my environment. Any attempts I make seem incorrect and do not result in a change.).
As a side note: I have installed the Allow-Control-Allow-Origin: * extension on my Chrome browser, but I receive the same error - just a shortened, less elaborate description of it:
Fetch API cannot load https://api.yelp.com/v3/businesses/search?[remaining URL]. Response for preflight has invalid HTTP status code 500
The following is how I call the fetch in my code:
var options = (
method: 'get',
headers: new Headers({
'Access-Control-Allow-Origin': '*',
'Authorization': [my token]
'Content-Type': 'application/json'
})
}
return fetch(url, options);
Is this an issue due to the syntax of my header with Yelp Fusion's OAUTH2 token requirements, do I need to do something proxy-related, or is the reason because of something else? If proxy-related, currently I am running a fully client-driven application and do not use server-side code at all. Would this still be possible given my environment? Any guidance as to which direction I should go and clarification of my misconceptions would be greatly appreciated.
Again, thank you for your help for a growing developer.
This cause of the problem is that https://api.yelp.com/ doesn’t support CORS.
And there’s nothing you can in your own application code to fix that—no matter what you try, you can’t change the fact that https://api.yelp.com/ doesn’t support CORS.
Apparently the Yelp API does support JSONP though; see for example Yelp API Origin http://localhost:8888 is not allowed by Access-Control-Allow-Origin.
So using https://api.jquery.com/jquery.getjson/ or similar in your frontend code would allow you make requests to the Yelp API cross-origin from your frontend code.
A related issue in the GitHub issue tracker for the Yelp API examples repo confirms no CORS:
TL;DR: No CORS is not supported by api.yelp.com
And another related issue:
As I answered in #99 , we do not provide the CORS headers necessary to use clientside js to directly make requests to the api.
Both of the comments cited above are from a Yelp engineer.
So what the means is, there’s no way your frontend JavaScript code can make requests directly to Yelp API endpoints and get normal responses (as opposed to JSONP responses).
Specifically, because responses from the https://api.yelp.com/v3/businesses/search API endpoint don’t include the Access-Control-Allow-Origin response header, browsers will not allow your frontend JavaScript code to access those responses.
Also, because your request includes the Authorization and a Content-Type header with the value application/json, your browser does a CORS preflight options request before ever attempting the actual GET request you’re trying to send.
And that preflight is what’s specifically failing in this case. But any other request you make from the frontend code to that API endpoint would also fail—even if it didn’t trigger a preflight.
I am trying to build a quick demo site that I do not have control over the server I am trying to connect to. Here is the code that I am using to build it with AngularJS. I am running the file through a simple Python HTTP Server and viewing it at localhost:8000.
var retrieveAppliances = function () {
console.log('Attempting to retrieve appliance list.');
var requestUrl = '****';
$http({
method: 'GET',
url: requestUrl,
})
.then(function (response) {
console.log(response);
});
};
retrieveAppliances();
I have read multiple places to try switching the method to JSONP but doing so resulted in a parsing error.
While I have considered trying to build a server.js file and running NodeJS with it, I am unsuccessful in learning the basics of making an AJAX request and proxying that to my app.js.
I will greatly appreciate any help that someone may be able to give me, with clear and easy to follow steps.
If you're running an Ajax call to a different origin (e.g. different host, port or protocol) and the server at that origin does not have support for cross origin requests, then you cannot fix that from your client. There is nothing you can do from the client.
If the server supported JSONP, you could use that, but that also requires specific server support.
The only solutions from a browser web page are:
CORS support on the target server.
JSONP (also requires support on the target server).
Set up your own server that you do have access to (either on your existing page domain or with CORS) and then have that server get the file/data for you and proxy it back to you. You can either write your own proxy or deploy a pre-built proxy.
Find some existing third party proxy service that you can use.
If you're interested in making your own node.js proxy, you can see a simple example here: How to create a simple http proxy in node.js?.
How do you get data from a REST API with JavaScript. I have several basic API's that I would like to get data from that don't require any authentication. All of the API's return the data I want back in JSON. For example https://www.codewars.com/api/v1/users/MrAutoIt. I thought this would be a very simple process using xmlhttprequest but it appears the same-origin policy is giving me problems.
I have tried following several tutorials but they don’t seem to work on cross domains or I don’t understand them. I tried to post links to the tutorials but I don't have a high enough reputation on here yet.
If you are trying to access a web service that is not on the same host:port as the webpage that is issuing the request, you will bump into the same origin policy. There are several things you can do, but all of them require the owner of the service to do things for you.
1) Since same origin policy does not impact scripts, allow the service to respond by JSONP instead of JSON; or
2) Send Access-Control-Allow-Origin header in the web service response that grants your webpage access
If you cannot get the service owner to grant you access, you can make a request serverside (e.g. from Node.js or PHP or Rails code) from a server that is under your control, then forward the data to your web page. However, depending on terms of service of the web service, you may be in breach, and you risk them banning your server.
In fact, it depends on what your server REST API supports regarding JSONP or CORS. You also need to understand how CORS works because there are two different cases:
Simple requests. We are in this case if we use HTTP methods GET, HEAD and POST. In the case of POST method, only content types with following values are supported: text/plain, application/x-www-form-urlencoded, multipart/form-data.
Preflighted requests. When you aren't in the case of simple requests, a first request (with HTTP method OPTIONS) is done to check what can be done in the context of cross-domain requests.
That said, you need to add something into your AJAX requests to enable CORS support on the server side. I think about headers like Origin, Access-Control-Request-Headers and Access-Control-Request-Method.
Most of JS libraries / frameworks like Angular support such approach.
With jQuery (see http://api.jquery.com/jquery.ajax/). There are some possible configurations at this level through crossDomain and xhrFields > withCredentials.
With Angular (see How to enable CORS in AngularJs):
angular
.module('mapManagerApp', [ (...) ]
.config(['$httpProvider', function($httpProvider) {
delete $httpProvider.defaults.headers.common['X-Requested-With'];
});
If you want to use low-level JS API for AJAX, you need to consider several things:
use XMLHttpRequest in Firefox 3.5+, Safari 4+ & Chrome and XDomainRequest object in IE8+
use xhr.withCredentials to true, if you want to use credentials with AJAX and CORS.
Here are some links that could help you:
Understanding and using CORS: https://templth.wordpress.com/2014/11/12/understanding-and-using-cors/
4 jQuery Cross-Domain AJAX Request methods: http://jquery-howto.blogspot.fr/2013/09/jquery-cross-domain-ajax-request.html#cors (see "1. CORS (Cross-Origin Resource Sharing)")
Unleash your AJAX requests with CORS: http://dev.housetrip.com/2014/04/17/unleash-your-ajax-requests-with-cors/
Using CORS for Cross Domain AJAX requests: http://techblog.constantcontact.com/software-development/using-cors-for-cross-domain-ajax-requests/
Cross origin resource sharing cors AJAX requests between jQuery and Node.js: http://www.bennadel.com/blog/2327-cross-origin-resource-sharing-cors-ajax-requests-between-jquery-and-node-js.htm
Hop it helps you,
Thierry
Here is how you get data.
var request = new XMLHttpRequest();
request.open('GET', 'https://www.codewars.com/api/v1/users/MrAutoIt', true);
request.onload = function() {
if (this.status >= 200 && this.status < 400) {
var resp = this.response; // Success! this is your data.
} else {
// We reached our target server, but it returned an error
}
};
request.onerror = function() {
// There was a connection error of some sort
};
request.send();
As far as running into same origin policy... You should be requesting from an origin you control, or you can try disabling Chrome's web security, or installing an extension such as Allow-Control-Allow-Origin * to force headers.
For a get method you could have something like this:
#section scripts{
<script type="text/javascript">
$(function()
{
$.getJSON('/api/contact', function(contactsJsonPayload)
{
$(contactsJsonPayload).each(function(i, item)
{
$('#contacts').append('<li>' + item.Name + '</li>');
});
});
});
</script>
}
In this tutorial check the topic: Exercise 3: Consume the Web API from an HTML Client