I'm using the Peloton API to generate statistics based on workout data grabbed from their API. Certain API requests require the user to be logged in, which can be done by sending your username and password to /auth/login. It returns a session ID that is needed to get those locked requests, such as workout history. The session ID is seemingly sent as a cookie (using credentials: 'include').
When I set credentials to include, it says "Access to fetch at (url) from origin (my site) has been blocked by CORS policy: The value of the 'Access-Control-Allow-Origin' header in the response must not be the wildcard '\*' when the request's credentials mode is 'include'. How do I fix this? My code is below.
async function loginAndGetData(username, password) {
const info = { 'username_or_email': username, 'password': password };
const response = await fetch("https://pelotoncors.herokuapp.com/https://api.onepeloton.com/auth/login",
{
method: 'POST',
body: JSON.stringify(info)
});
const loginInfo = await response.json();
const workoutInfo = await fetch('https://pelotoncors.herokuapp.com/https://api.onepeloton.com/api/user/' + loginInfo.user_id + '/workouts?limit=1234567890',
{
method: 'GET',
credentials: 'include'
});
}
So I identified multiple problems:
Paleton doesn't have an official API or even an API doc (that doesn't make it impossible to use their API, but it can be difficult, and I would be cautious with that)
CORS
cors-anywhere
Logically, to prevent the use of the API by anybody, Paleton, like anybody else who builds a serious API, decided to put CORS headers on their responses, which is obviously not only from a security standpoint reasonable, but can be quite annoying because it makes it basically impossible to use the API from the browsers because the browser enforces the CORS-headers that are being set on responses by the servers.
So your first idea with cors-anywhere seems rational, but doesn't work because you use credentials: 'include' which requires the 'Access-Control-Allow-Origin' header to NOT be set to wildcard *.
If you are not familiar of what all that above means, let me run you through it:
In your above code, which I assume because of the above error message you run on a browser, you try to make a fetch call to cors-anywhere with the URL of the API and userId. Cors-anywhere makes the call to the API for you and ignores the CORS-Headers (it can do that because it is just a policy created for browsers to increase security), send you the data it gets from the API back BUT with the 'Access-Control-Allow-Origin' (by default) set to wildcard *. Because you use credentials: 'include' though, which requires the 'Access-Control-Allow-Origin' header do NOT be set on wildcard, your browser detects a CORS-policy violation, and you get the above error.
So what can you do now ?
Either you offload the API calls to your own backend (which is BTW always a better option), ignore the CORS by yourself and set the 'Access-Control-Allow-Origin' header appropriately or
Maybe there is a query you can add to the URL for cors-anywhere to change the 'Access-Control-Allow-Origin'? (But I don't know that) or
You look for an alternative
I guess the problem is your backend on Heroku, you need to set header "Access-Control-Allow-Origin": true on your backend.
Related
Working with Apache Airflow REST API, and having issues with CORS.
When calling the endpoint using the fetch API in JavaScript I get the following error:
Access to fetch at 'my_url/api/v1/dags/example_bash_operator/tasks' from origin 'my_url' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
This is how I am calling it:
let url = "my_url/api/v1/dags/example_bash_operator/tasks";
let username = 'my_username';
let password = 'my_password';
let headers = new Headers();
headers.set('Authorization', 'Basic ' + btoa(username + ":" + password));
fetch(url, {
headers: headers,
method: 'GET',
})
.then(response => response.json())
.then(data => {
console.log('Success:', data);
})
.catch((error) => {
console.error('Error:', error);
});
I also tried adding mode: 'no-cors' but just get the "unexpected end of input" error.
For some background, the following works fine:
starting the airflow webserver and scheduler
accessing the airflow UI
accessing the SwaggerUI authenticating Swagger and calling the REST endpoints with this tool
calling my_url in the address bar of a new browser tab (returns the expected JSON)
I have set the auth_backend in airflow.cfg:
auth_backend = airflow.api.auth.backend.default
Although with the latest REST API version I don't think this makes a difference since everything is set to deny.
I have also set the access control headers in airflow.cfg as described in the docs:
access_control_allow_headers = origin, content-type, accept
access_control_allow_methods = POST, GET, OPTIONS, DELETE
access_control_allow_origin = my_url
...and also tried with wildcard for the access_control_allow_origin:
access_control_allow_origin = *
So the REST calls work fine through Swagger and through the browser address bar, but I cannot call it with fetch using JS. Note that the JS is in an index.html file on the same server (and same root directory) as the airflow files.
The described behavior makes sense, since CORS is used by the browser to prevent attacks from scripts of different resources.
You are still able to fetch via Swagger, Postman or other tools, even through the browser via address bar. But if the policy does not allow to fetch from a different origin, then the browser prevents fetching from your script, which is probably served on a different port. Origin contains host and port.
Your main issue, I cannot help with at the moment.
I've faced the issue of not being able to set the origin policy within the Airflow 2.0 server/API through the (docker-compose) environment variable AIRFLOW__API__ACCESS_CONTROL_ALLOW_ORIGIN.
Maybe it's related to your issue, since I can see from the url of your question (containing the v1), that you're are also using Airflow 2.x.
By the way, the message from chrome is CORS error: Preflight Missing Allow Origin Header, referring to the question in the comments of the original question.
I'm trying to access to introspect endpoint in my Keycloak server /openid-connect/token/introspect from my front app, but I get next error:
Access to fetch at 'http://localhost:8180/auth/realms/backoffice/protocol/openid-connect/token/introspect' from origin 'http://localhost:8080' has been blocked by CORS policy:
Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.
If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Using Postman, curl or Node app this request works fine, but from my front-app using fetch method thows this error. I'm not sure it's possible query for introspect endpoint from front-app in the browser or if it's only possible from server app.
Other endpoints like:
openid-connect/token:
openid-connect/userinfo:
Works fine using the Postman JS code.
Keycloak config
My client in Keycloak has set up Web Origins * and Access Type confidential.
Client Code
My front app is simply the Postman code JS, and I deploy it using node http-server.
var myHeaders = new Headers();
myHeaders.append("Content-Type", "application/x-www-form-urlencoded");
var urlencoded = new URLSearchParams();
urlencoded.append("client_id", "my-client");
urlencoded.append("client_secret", "my-secret");
urlencoded.append("token", "eyJ...oCA");
var requestOptions = {
method: 'POST',
headers: myHeaders,
body: urlencoded,
redirect: 'follow'
};
fetch("http://localhost:8180/auth/realms/backoffice/protocol/openid-connect/token/introspect", requestOptions)
.then(response => response.text())
.then(result => console.log(result))
.catch(error => console.log('error', error));
Header Response
The header response in userinfo endpoint comes with Access-Control-Allow-Origin and Access-Control-Allow-Credentials but is not present in introspect endpoint.
From the looks of it, the Keycloak server prevents the CORS headers to be set for the introspection endpoint. This could be a bug or by design. I tried it and I get the same error.
If you really want to access the introspect endpoint from the web app, you could set up a NGINX reverse-proxy in front of your Keycloak server and use it to add the missing headers.
That being said, according to oauth.com you should not leave the introspection endpoint available to the public, which is what you are currently doing since anyone can retrieve the client id and secret from your web app.
If the introspection endpoint is left open and un-throttled, it presents a means for an attacker to poll the endpoint fishing for a valid token. To prevent this, the server must either require authentication of the clients using the endpoint, or only make the endpoint available to internal servers through other means such as a firewall.
This could explain the decision not to allow CORS.
Another thing, it looks like you forgot to set the token_type_hint check out this stackoverflow post for more information.
I'm trying to hit an API that needs a cookie to return data.
If i hit the url directly in the browser I get the data i want. The protocol is https.
However, whenever I try to fetch the data using window.fetch I run into CORS errors. I think this is happening because I cant get the cookie in the client request, which is causing the server to redirect to an auth server that is not sending back a CORS header.
I have tried using { credentials: 'include' } to no avail.
I was assuming that because the cookie exists in the browser it will be part of the request.
Any fundamental knowledge I'm missing here?
hmm this is a bit weird, by default fetch uses { credentials: 'same-origin' } which will allow cookies to be sent { credentials: 'include' } also will send cookies even if its not the same origin and that's the only difference.
what i am thinking here is that either somewhere in your code the cookie gets deleted somehow or your request is firing before the cookie is set, or the server doesn't allow CORS or it doesn't allow the OPTIONS method if its a different origin.
Basically, I'm trying to get a username by id from Sequelize. The problem is that I am either stuck with a CORS problem or 500 Internal Server error depending on the response(status)
cors and 500
controller code
async getUserFromUserId (req, res) {
try {
// const user = await User.findByPk(req.body.id)
const id = req.body.id
const user = await User.findByPk(id)
res.send(user.username)
} catch (err) {
// or res.status(some random number).send() for CORS problem to appear
res.status(500).send({
error: 'an error has occured trying to fetch the users id'
})
}
},
client code
this.notifiedUser = (await UserService.getUserFromUserId({id: UserId})).data
I get a Status: 200 OK from postman though.
Postman Solution
Edit:
I have seen how the other Solution for the cors thingy, but the solutions does not specify as to why I get "undefined" results after resolving the cors problem.
So, CORS is actually really obnoxious in this regard, but there's a fairly straightforward way to fix this. It's a super useful security feature, though it is frustrating at best sometimes.
Your browser does what is called a Preflight Request, which is of the http verb OPTIONS. Your browser calls whatever route you want, but instead of what you asked it to do, it calls using OPTIONS first. Your server should accept all routes that the client can ask for with the OPTIONS method, and your server should respond with the following headers to be an externally available, cross-origin API.
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, ...
(note, you should not put the ... in, but you can put any HTTP verb in this list)
If you require your own headers (for auth purposes), you want to add this header for Client -> Server.
Access-Control-Allow-Headers: YourHeader, YourHeader2, YourHeader3
You want to add this one for Server -> Client
Access-Control-Expose-Headers: YourHeader,YourHeader3
Note that the OPTIONS call is an entirely separate call that you should handle as well as the GET method.
You've now told the browser what it is allowed to ask for, and what it can expect to get back from your API. If you don't respond to the OPTIONS request, the browser terminates the request, resulting in a CORS error.
I'm going to take a gander at a guess and assume you're likely using Express, which this answer describes how to set the headers on.
What do the headers mean, in English?
Access-Control-Allow-Origin
From where are clients allowed to access this resource (endpoint)? This can match partial domains with wildcards, or just a * to allow anywhere.
Access-Control-Allow-Methods
What HTTP methods are permissible on this route?
Access-Control-Expose-Headers
When I get a response from the server, what should I (the browser) expose to the client-side?
Access-Control-Allow-Headers
What am I as the client side allowed to send as headers?
Okay, so I figured out the problem.
In a way, I did not have to deal with any of the cors stuff because I believe that was not the main source of the problem.
So, instead of accessing my database data through "GET" and getting the data by doing this:
this.data = (Service.function(bodyValue)).data
I did "POST" to get the data, and accessed the data by simply doing this
const response = Service.function({
id: bodyValue
})
this.data = response.data
This accesses the data without having to get "secured" information from the database, but by accessing the data from the database by getting Observer object info from the database.
The Observer object looks as follows, which treats the user data as an object instead of pure data.
Compared to a data object, where each data {...} has user information.
I am not sure if I am using the correct words, but these are to the extent of my current understanding.
If your origin is from localhost, Chrome usually blocks any CORS request originating from this origin.
You can install this extension:
https://chrome.google.com/webstore/detail/allow-cors-access-control/lhobafahddgcelffkeicbaginigeejlf?hl=en
Or you can disable the security when running chrome (add the flag):
--disable-web-security
I can't execute the 'GET' request with the getTasks() function.
$(document).ready(function(){
getTasks();
});
const apiKey = 'xxxxxxx';
function getTasks(){
$.ajax({
type: 'GET',
url: 'https://api.mlab.com/api/1/databases/taskmanager/collections/tasks?apiKey='+apiKey,
contentType: 'application/json',
xhrFields: {
withCredentials: true
},
success: function(data){
console.log(data);
},
error: function(){
console.log('FAIL')
}
})
}
The error that I get is:
api.mlab.com/api/1/databases/taskmanager/collections/tasks?apiKey=xxxxxxx
Failed to load resource: the server responded with a status of 400
(Bad Request)
Response to preflight request doesn't pass access control check: No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'null' is therefore not allowed access. The response
had HTTP status code 400.
I understand that Google-Chrome on Windows is CORS enabled, and will not (by default) allow communication with a different domain. I'm not sure what a preflight request is. Regardless, I tried to implement what I saw from Using CORS - HTML5 Rocks (from the CORS from jQuery section), but to no avail.
At a guess, the remote API simply does not respond to pre-flight requests for GET calls (because it shouldn't have to).
Your code is triggering a pre-flight request because it is non-simple. This is due to your adding a Content-type: application/json header. A request Content-type header is used to indicate the request payload format. As it is a GET, there is no payload.
Try this instead...
$.getJSON('https://api.mlab.com/api/1/databases/taskmanager/collections/tasks', {
apiKey: apiKey
}).done(function(data) {
console.log(data)
}).fail(function() {
console.log('FAIL')
})
CORS is there to protect you. If you want some more info on it, wikipedia has a good entry on it.
It appears the issue here is that you're trying to access your mongodb hosted by mlab directly from your web app. As you can see in your code, you're providing credentials/api keys to make that request.
My guess is that mlab's intent of not allowing CORS is to prevent you from doing this. You should never put your private API keys in html to be hosted on a web page, as it's easily accessible by reading source code. Then someone would have direct access to your mongodb.
Instead, you should create a server-side application (node, or... ** Whatever **) that exposes an api you control on the same domain (or a domain you give permission to via CORS).
As far as the "preflight" request, if you look in your chrome debugging tools, you should see an additional request go out with the "OPTIONS" method. This is the request that chrome (and most other http clients) send out first to a server hosted on a different domain. it's looking for the Access-Control-Allow-Origin header to find out whether it's allowed to make the request. Pretty interesting stuff if you ever have some time to dig into it.