I implemented a REST service, which I'm calling from a javascript application on a different domain.
I'm attempting to do a GET request, and setting the Authorization: header, with a custom authentication scheme.
Because I'm setting a custom header, Firefox will start with a preflight OPTIONS request. This request looks like this (simplified):
OPTIONS /v1/articles HTTP/1.1
Host: example.org
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13 FirePHP/0.5
Origin: http://example.com
Access-Control-Request-Method: GET
Access-Control-Request-Headers: authorization
My response is as follows:
HTTP/1.1 200 OK
Date: Wed, 15 Dec 2010 16:36:47 GMT
Server: Apache/2.2.9 (Debian) PHP/5.3.3-0.dotdeb.1 with Suhosin-Patch mod_vhost_hash_alias/1.0
X-Powered-By: PHP/5.3.3-0.dotdeb.1
Access-Control-Allow-Origin: http://example.com
Access-Control-Request-Method: GET,POST,PUT,DELETE,HEAD,OPTIONS
Access-Control-Request-Headers: Authorization, X-Authorization
Content-Length: 2
Content-Type: application/json
After this, the actual GET request is simply not performed. I'm worried I made a mistake in my response, but I can't seem to spot it.
The other problem is that I've found no way to get a detailed error message. As you can see, I also tried X-Authorization instead of Authorization.
My questions:
Is there something wrong with my response? How can I find more details about the problem?
Thanks!
I needed a fresh look at this. The next morning I realized the correct headers are:
Access-Control-Allow-Methods: GET,POST,PUT,DELETE,HEAD,OPTIONS
Access-Control-Allow-Headers: Authorization, X-Authorization
Related
I have an application (React SPA) that calls a bunch of servers on different subdomains of the application domain, i.e.:
the web app sits at foo.bar.com,
and talks to api.foo.bar.com and media.foo.bar.com.
When accessing api.foo.bar.com, I get an error from the browser (be it Edge, Chrome, or Firefox) telling me that the origin (foo.bar.com) is different from the value of the Access-Control-Allow-Origin response header. However, by inspection of the response, they are the same:
(I unfortunately have to obfuscate the address.)
Those apps are hosted on Kubernetes; the ingress is NGINX, and it's is not providing CORS (cors-enabled annotation is false). Both applications (api and media) are Express apps, and both have the same CORS configuration allowing the specific origin.
I'm wondering if this has something to do with the redirect - the call to the media... endpoint returns a redirect (302) whose Location is a api... address.
Other than that, I have no clue what could be wrong. Something is, for sure, because all browsers agree that my request should be blocked (on account of the origin).
In all cases, I've checked the address multiple times for typos, ending forward-slashes, etc. I've called OPTIONS on those endpoints with cURL and Postman, using all headers or just a few. They always answer the correct address.
Additional information, as requested:
Preflight request:
OPTIONS /media/1.0.0/rtsp/hls?feedUrl=https%3A%2F%2Flive.monuv.com.br%2Fa1%2F14298.stream%2Fstr27%2Fchunklist.m3u8%3Fm_hash%3DkhV_hCnKG3nhaNCFaYZxBnoMz-99idQVHiQh80ADW78%253D HTTP/2
Host: media.aiXXXXXXXXXXXXXX.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Access-Control-Request-Method: GET
Access-Control-Request-Headers: feedurl
Referer: https://aiXXXXXXXXXXXXXXXX.com/
Origin: https://aiXXXXXXXXXXXXXXXX.com
DNT: 1
Connection: keep-alive
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
Pragma: no-cache
Cache-Control: no-cache
TE: trailers
Preflight response:
HTTP/2 204 No Content
date: Fri, 08 Oct 2021 13:33:10 GMT
x-powered-by: Express
access-control-allow-origin: https://aiXXXXXXXXXXXXXXXXXX.com
vary: Origin
access-control-allow-credentials: true
access-control-allow-methods: GET,HEAD,PUT,PATCH,POST,DELETE
access-control-allow-headers: Content-Type, feedUrl
strict-transport-security: max-age=15724800; includeSubDomains
X-Firefox-Spdy: h2
Request
The preflight passes, and the browsers starts a "flight" request:
GET /media/1.0.0/rtsp/hls?feedUrl=https%3A%2F%2Flive.monuv.com.br%2Fa1%2F14298.stream%2Fstr27%2Fchunklist.m3u8%3Fm_hash%3DkhV_hCnKG3nhaNCFaYZxBnoMz-99idQVHiQh80ADW78%253D HTTP/2
Host: media.aiXXXXXXXXXXXXXXXXXXXXX.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
feedUrl: https://live.monuv.com.br/a1/14298.stream/str27/chunklist.m3u8?m_hash=khV_hCnKG3nhaNCFaYZxBnoMz-99idQVHiQh80ADW78%3D
Origin: https://aiXXXXXXXXXXXXXXXX.com
DNT: 1
Connection: keep-alive
Referer: https://aiXXXXXXXXXXXXXXXXX.com/
Cookie: ory_kratos_session=MTYzMzYzODY1OHxEdi1CQkFFQ180SUFBUkFCRUFBQVJfLUNBQUVHYzNSeWFXNW5EQThBRFhObGMzTnBiXXXXXXXXXXXXYVc1bkRDSUFJSHBtUWxsaWFsVlJhWGRTVGxSMmIzZHRkbTFqYm5CUlRWVkdkelpPWkRoWnXXXTyqwgK-0Pe0qtZHjNhfU-YoASjg3istMZi672swQ==
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
Pragma: no-cache
Cache-Control: no-cache
TE: trailers
Response
HTTP/2 302 Found
date: Fri, 08 Oct 2021 13:33:10 GMT
content-type: text/plain; charset=utf-8
content-length: 129
location: https://api.aiXXXXXXXXXXXXXXXXXX.com/media/1.0.0/hls/streams/19dd149d-f551-4093-b2aa-e5558388d545/hls.m3u8
x-powered-by: Express
access-control-allow-origin: https://aiXXXXXXXXXXXXXXXX.com
vary: Origin, Accept
access-control-allow-credentials: true
strict-transport-security: max-age=15724800; includeSubDomains
X-Firefox-Spdy: h2
At this response, the browser fails saying that the origin don't match the access-control-allow-origin.
(the first image was from Edge, since the log was more clear; this log is from Firefox)
Problem
The error message—I'm using dummy URLs and origins below—from the browser can be a bit confusing:
Access to XMLHttpRequest at 'https://api.example.com/' (redirected from 'https://media.example.com/') from origin 'https://example.com' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: The 'Access-Control-Allow-Origin' header has a value 'https://example.com' that is not equal to the supplied origin.
The key here is that, as sideshowbarker hinted at in his comment, because your first preflighted request to https://media.example.com/ responds with a cross-origin redirect to https://api.example.com/, the browser performs another whole CORS access-control check for that resource. However, because the redirect resulting from the first preflighted request happens to be cross-origin, the browser sets the origin of the second preflight request (which the error message refers to as the "supplied origin"), not as https://example.com, but as the null origin!
Here's a rundown of what is likely happening:
Because https://api.example.com likely doesn't (and shouldn't!) allow the null, the second access-control check fails and you get that annoying CORS error.
Solution
Resist the temptation to allow the null origin on https://api.example.com/, as doing so has serious security ramifications: it amount to voiding the protection that the Same-Origin Policy provides.
Instead, you should get rid of that redirect from https://media.example.com/ to https://api.example.com/ and make your frontend request the https://api.example.com/ resource directly.
Alternatively, if you cannot completely get rid of the redirect but you can change its destination, make it a same-origin redirect (from somewhere https://media.example.org to elsewhere on https://media.example.org).
So, I have two sites http://localhost/ and http://3rdPartyLocallyHostedAPI/ (Not the real names) - both are local intranet sites, and due to the nature of 3rdPartyLocallyHostedAPI being it's namesake, localhost is having to make CORS requests to it.
These requests are working fine, data is returned or can be posted to 3rdPartyLocallyHostedAPI as would be expected, however this warning is being shown:
A cookie associated with a cross-site resource at http://3rdPartyLocallyHostedAPI/ was set without the SameSite attribute. A future release of Chrome will only deliver cookies with cross-site requests if they are set with SameSite=None and Secure. You can review cookies in developer tools under Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032.
Now, I've looked at multiple answers such as this one, this one and this one which state that the SameSite attribute needs to be set on the server, which doesn't make any sense as the two cookies it's taking issue with (ss-pid and ss-id) are set in the request, not returned in the response? This has confused me, as I can't figure out how or where to make a change to ensure the SameSite policy on these cookies is set to none or secure.
I think it's the jQuery that is performing the AJAX request that is at fault though:
// trimType and queryValue are determined elsewhere by some jQuery selections, their values are not important to the question being asked.
$.ajax({
url: 'http://3rdPartyLocallyHostedAPI?q=' + trimType + '?q=' + queryValue + '&resultsOnly=true',
data: {
properties: (trimType === 'Record') ? 'Title,Number,RecordRecordType' : 'NameString'
},
xhrFields: {
withCredentials: true
},
dataType: 'json'
}).done(function (data) {
if (data.Results.length > 0) {
$resultsPane.html('');
for (var i = 0; i < data.Results.length; i++) {
// Not relevant to the question being asked so removed, only some jQuery in here to display results on page.
}
} else {
$resultsPane.html('<p class="py-1 pl-1 list-group-item text-muted">No Results found.</p>');
}
}).fail(function () {
$resultsPane.html('<p class="py-1 pl-1 list-group-item text-muted">No Results found.</p>');
});
When it doesn't have the withCredentials = true property set, and therefore is authenticating anonymously against the API (which only gives limited access, hence the need to pass windows credentials), the SameSite warning does not appear. Here is the request header:
GET /CMServiceAPIAuth/Location?q=%22SDC%20*%22&resultsOnly=true&properties=NameString HTTP/1.1
Host: serverName
Connection: keep-alive
Accept: application/json, text/javascript, */*; q=0.01
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36
Origin: http://localhost:64505
Referer: http://localhost:64505/Home/DisplayRecord
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: ss-pid=OQtDrnmok62FvLlZPnZV; ss-id=cIaIcS3j0jmoouAaHHGT
The two cookies that chrome is having issues with are ss-pid and ss-id, there are no cookies being passed back by the response headers:
HTTP/1.1 200 OK
Cache-Control: private,no-cache
Content-Type: application/json; charset=utf-8
Vary: Accept
Server: Microsoft-IIS/8.5
X-Content-Type-Options: nosniff
X-Powered-By: ServiceStack/4.512 NET45 Win32NT/.NET
X-AspNet-Version: 4.0.30319
Access-Control-Allow-Origin: http://localhost:64505
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: POST,GET,OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization
Persistent-Auth: true
X-Powered-By: ASP.NET
Date: Mon, 27 Jul 2020 07:02:06 GMT
Content-Length: 1597
So, with all this in mind, can someone explain where I'm going wrong? Do I need to make changes to the jQuery AJAX to prevent this warning (and therefore prevent a future issue when the change the warning is alerting me about happens) - or, do I actually need to set an additional header on the server, I'm wondering if in the pre-flight OPTIONS request if it tries to figure out the SameSite setting for the request or something like that?
Via an IIS Module, I do have access to add additional headers to the response the server is sending, so if that is what is needed, I can do it - I just don't quite understand on which end the warning is being caused by and would appreciate any explanation people can provide.
Ok then, I think I've done enough research to figure out the issue I'm facing, so I'll answer my own question.
So, one page that really helped me to actually understand what SameSite is about was this one, so to anyone else having issues with SameSite, take a read so that you understand the reason behind it and how it works.
Having done some reading and seeing this answer to another post helped me to connect the dots. I deployed the web site I'm working on to an actual web server, and found that the following was the response header:
HTTP/1.1 200 OK
Cache-Control: private,no-cache
Content-Type: application/json; charset=utf-8
Vary: Accept
Server: Microsoft-IIS/10.0
X-Content-Type-Options: nosniff
X-Powered-By: ServiceStack/4.512 NET45 Win32NT/.NET
X-AspNet-Version: 4.0.30319
Set-Cookie: ss-pid=0QyVIKf4edkAKd2h4be5; expires=Fri, 27-Jul-2040 09:58:39 GMT; path=/; HttpOnly
Set-Cookie: ss-id=fmM1WQsDxXGfR8q9GL6e; path=/; HttpOnly
Access-Control-Allow-Origin: http://server
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: POST,GET,OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization
Persistent-Auth: true
WWW-Authenticate: Negotiate oYG2MIGzoAMKAQChCwYJKoZIgvcSAQICooGeBIGbYIGYBgkqhkiG9xIBAgICAG+BiDCBhaADAgEFoQMCAQ+ieTB3oAMCARKicARusah2q1K2ACHwoq1n6DCNq5rx/HFdbK5sU9EohnxrRSpzmelskTTa9xmW8wgeUdwRNQCqMsD/dZ/pUjhdl2CVWjmFZZAfnKl6JEker+s79E9uFXThZZKnqfpqEgSvvqSYpp1KMkaYBYd1uf5mRyE=
Date: Mon, 27 Jul 2020 09:58:40 GMT
Content-Length: 1597
There are two Set-Cookie headers being issued by the server to store values for ss-id and ss-pid. These cookies apparently stand for permanant session ID and session ID, and are issued by ASP.NET for tracking sessions. The browser does not accept and set these two cookies as they lack the SameSite=none setting and the Secure setting - these are the two cookies mentioned in the post above that I was talking about.
So to fix this issue, first off I need to switch to using https for the API (and potentially the site itself) - which I have already done, and somehow figure out how to get the 3rd party API to set SameSite attributes in it's session related cookies.
So for others who hopefully have complete control of your API's etc, you should just be able to set these attributes whenever you are creating/sending cookies in responses, and therefore send cookies from the site to other domains by setting SameSite=None and Secure.
I'm new to web programming so bear with me. I've created a simple REST API in python flask, and am hosting it with Apache 2.4. I've tested it via cURL and it works. Now I'm trying to access it via a web interface with jQuery.
The REST api is at http://api.localhost and the website that accesses it is at http://localhost.
The code I'm using to try and do a POST looks like this:
$.ajax({
type: 'POST',
url: 'http://api.localhost/auth',
data: '{"username":"user1", "password":"abcxyz"}',
success: function(data) { console.log(data); alert('data: ' + data); },
contentType: "application/json",
dataType: 'json'
});
However, it seems the success function doesn't run. Looking in dev console (f12) I can see that instead of a POST to that URL, and OPTIONS HTTP request is made. My understanding is this is a Cross-Origin Resource Sharing (CORS) preflight check, to make sure it's OK for localhost to access api.localhost.
I've added the following lines to my apache config for api.localhost:
Header always set Access-Control-Allow-Origin "*"
Header always set Access-Control-Allow-Methods "POST, GET, OPTIONS"
It seems to be working to because the OPTIONS request returns a 200 (and no other data). However, there is no follow up. My understanding is since the server is saying it's OK for anyone to POST to api.localhost, that it should go ahead and do the POST next, but it doesn't.
Here are the preflight check request headers:
Host: api.localhost
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
Firefox/52.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Access-Control-Request-Method: POST
Access-Control-Request-Headers: content-type
Origin: http://localhost
Connection: keep-alive
Cache-Control: max-age=0
Here are the response headers for that same request (remember, status 200):
Access-Control-Allow-Methods: POST, GET, OPTIONS
Access-Control-Allow-Origin: *
Allow: POST, OPTIONS
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 20
Content-Type: text/html; charset=utf-8
Date: Thu, 07 Dec 2017 00:17:08 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.29 (Debian)
Vary: Accept-Encoding
You can see that the server is saying any domain is fine (*) and that POST is OK. However, there is no follow up POST. What am I missing?
Thanks.
http://api.localhost/auth must also send Access-Control-Allow-Headers: content-type.
So to your Apache config, you also need to add this:
Header always set Access-Control-Allow-Headers "content-type"
That’s necessary because the contentType: "application/json" part of your frontend code adds a Content-Type: application/json header to the request, and any values for the Content-Type request header other than text/plain, application/x-www-form-urlencoded, or multipart/form-data will trigger browsers to send a CORS preflight OPTIONS request.
So if you have your http://api.localhost/auth server send back the Access-Control-Allow-Headers: content-type response header, that should cause the preflight to succeed, and so cause the browser to move on to making the POST request from your frontend code.
I have some working JavaScript (running inside Firefox (v41)) which I need to modify to support cross-domain XMLHttpRequests (my POST requests retrieve JSON encoded data). I have control over the server in question, so I capture OPTIONS requests and reply with:
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With
Access-Control-Max-Age: 86400
The browser then correctly sends the POST request, my server responds with the data and that data arrives back at my machine; I can see it in Wireshark and it is well formed JSON.
HOWEVER, the data doesn't get to my JavaScript. I can see in the Firefox window that the response to the POST request does arrive, with all the expected headers indicating (for example) 1120 bytes of content but, when I click on the "Response" tab, there is nothing in it: SyntaxError: JSON.parse: unexpected end of data at line 1 column 1 of the JSON data. My JavaScript code ends up in the XMLHttpRequest's onerror function.
What do I need to do to get my data correctly? Any advice welcomed.
Here is a sample of one complete HTTP exchange, as seen by Wireshark on the browser machine:
OPTIONS /getAllData HTTP/1.1
Host: blah:blah
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Origin: null
Access-Control-Request-Method: POST
Access-Control-Request-Headers: content-type
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
HTTP/1.1 200 OK
Access-Control-Allow-Headers: Content-Type, X-Requested-With
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 86400
Date: Fri, 26 Aug 2016 09:22:14 GMT
Content-Length: 0
Content-Type: text/plain; charset=utf-8
test
POST /getAllData HTTP/1.1
Host: blah:blah
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/string; charset=UTF-8
Content-Length: 4
Origin: null
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Date: Fri, 26 Aug 2016 09:22:15 GMT
Content-Length: 1121
{"wellformed":"data 1121 bytes long"}
I have toyed with Access-Control-Allow-Origin and the header needs to be implemented in each and every response that is sent to the client.
So, whenever you make that POST, the answer MUST include the ACAO header otherwise the browser will filter out the content for security purposes. I do not see the header from the capture you made, which might explain the issue.
You can take a look at the examples from Mozilla, you will see that the response to the POST do include the ACAO header.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
Also, it seems that your body content is not separated from the header by a conventional empty line (\r\n in HTTP protocol). The body seems to be part of the header in your pastes, but it might just be a glitch from your copy-paste. If it's not then it's also a potential explanation: no body = no content.
Finally, I recommend that you debug your trafic through a tool such as BurpSuite which implements a nice Proxy allowing you not only to real-time view and edit your requests, but also to replay them and toy around. Initially a security tool, it is still great for debugging web apps.
https://portswigger.net/burp/
I have no idea what's going on. I worked so hard to get the signature and header perfect. Everything is perfect. I compared it with the oauth tool here:
https://dev.twitter.com/oauth/tools/signature-generator/
However I keep getting:
status: 401
statusText: Unauthorized
response: {"errors":[{"code":32,"message":"Could not authenticate you."}]}
I read somewhere that it might be a multipart post so I need to not include the POST/query data in the signature so I tried that but I get 403:
status: 403
statusText: Forbidden
response: {"errors":[{"code":170,"message":"Missing required parameter: status."}]}
Does anyone know what can possibly be going on? I don't know what code to share, because everything is matching perfectly. I upgrade a request token to an access token with this same algorithm ( but of course in this call the oauth_token_secret is not used). I also use the same exact method to upload an gif for attachment to tweet and it works prefectly, of course though the difference here is the upload is a multipart so I have to make it not use the post data or query parameters in the signature base string.
I amm of course generating new header per request. And my token is acccess token not request token.
Here are the headers for the tweet:
https://api.twitter.com/1.1/statuses/update.json
POST /1.1/statuses/update.json HTTP/1.1
Host: api.twitter.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Authorization: OAuth oauth_consumer_key="......", oauth_nonce="Cdo4tSOnRkOhxOAxMJWwjrnpB2qUYyjQXfnv5kes", oauth_signature="LyRDKV44MGYxCk3TNEm8lrCPUeg%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1466330342", oauth_token="15......97-ivs............rPqXxBk", oauth_version="1.0"
Content-Length: 17
Content-Type: text/plain;charset=UTF-8
Cookie: ...........
Connection: keep-alive
status=Hellohiiii
HTTP/2.0 401 Unauthorized
Content-Encoding: gzip
Content-Length: 89
Content-Type: application/json; charset=utf-8
Date: Sun, 19 Jun 2016 09:58:52 GMT
Server: tsa_a
Strict-Transport-Security: max-age=631138519
x-connection-hash: 1db624e689db4dc937d06c68e7318aa9
x-response-time: 6
x-tsa-request-body-time: 1
X-Firefox-Spdy: h2
Does anyone have any ideas?
Edit:
Because the multipart formdata submits fine, I am suspecting it is the post data I am sending. When I post the data I tried with encodeURIComponent and without it. Neither had affect though, still would get 401. In the signature making I used of course without any encoding.
Wow, I had to set Content-Type: application/x-www-form-urlencoded in the headers... This should be made clear