Develop Reference

Fetching data from Javascript

How can I get data from a server with no 'Access-Control-Allow-Origin' ?
I can get an Opaque answer only.
Problem is that I need to call a public API on a server with no Access-Control-Allow-Origin - I can try to convince site owner to add it, but how do I do it in the meantime ? I am trying to not run a proxy on my own domain. The data is personal data (electric power consumption), which I want the browser to download and process, and I do not want my server to see the users private API key, which is stored in localStorage only. At some point I want to it to become a true web app.
Fetch with no-cors mode gives me Opaque data, and with cors enabled I get:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://api.eloverblik.dk/. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing)
Any other way to call an API / Get data from a 3rd party site on the web ?
Here is example code:
fetch( 'https://www.eloverblik.dk', { method: 'GET', redirect: 'follow'})
.then( (response) => { console.log('x'); console.log('Resp: ' + response.type) })
.then((result) => { console.log('Res: ' + result )})
.catch( (error) => { console.error('Error:', error); })

Thanks for the help here.
Basically, the answer is, that it is not possible.
The workaround is to use a reverse proxy like nginx and add the header.
I have contacted the site owner and asked them to add the header. It is a public available API, that requires login with an API key you can generate on a website.
They do run another api server, giving anonymous access to public data. That one is run on cloudapp.net. The one where they "forgot" the header has personal electricity consumption measurements - Down to 15m intervals. It is hosted under t-msedge.net. Hope they will do the change, so I can do away with the proxy.

Enable Cross-Origin Requests (CORS) in POCO C++ Libraries

I'm building a C++ backend with heavy calculations that are meant to work as an JSON API for connecting clients. To accomplish this, I've used HTTPServer in Poco::Net from POCO C++ Libraries.
Unfortunately when building two different clients it turned out that a regular webpage (HTML+JS) can't use Fetch to communicate with the backend due to CORS error. My understanding is that they need to use the same localhost: and that's not the case when manually opening the HTML document on the computer that's also running the backend.
All I can come up with when searching is the generic advice that servers need to enable CORS and whitelist relevant domains. Unfortunately I can't find documentation on how to accomplish this. The only relevant result was an answer on a related question where he recommended the following:
response.set("Access-Control-Allow-Origin", "*");
Naturally whitelisting everything isn't recommended from a security point of view but the main goal here is to just get it running locally to continue the development. Unfortunately it seems to make no difference and the browser console still says:
Access to fetch at 'http://localhost:6363/' from origin 'null' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Hovering the error in the Chrome Network tab I get the following:
Cross-Origin Resource Sharing error: PreflightMissingAllowOriginHeader
My current JavaScript call:
const data = { test: 'test' }
fetch('http://localhost:6363', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(data),
})
.then(response => response.text())
.then(message => {
console.log('Data retrieved:', message);
})
.catch((error) => {
console.error('Error:', error);
});
Any suggestions on how to proceed?

fetch() works in Extension but fails in Chrome Console/Snippet

My Chrome Extension performs a get request which works fine. Because testing is faster with snippets, I want to do the exact same thing in the Chrome Console or in the Chrome Snippets. Minimal example:
fetch(url, {
method: "GET"
}).then(response => response.text())
.then(html => console.log(html))
.catch(error => console.log(error))
Unfortunately, there I only get
TypeError: Failed to fetch for the error and
Failed to load resource: net::ERR_FAILED in Chrome's inline error marker
In my Chrome Extension I ran into a CORS issue so what I did in my AWS Lambda function was to set the headers to
const headers = {
'Content-Type': 'application/json',
"Access-Control-Allow-Headers" : "Content-Type",
"Access-Control-Allow-Origin" : "*",
"Access-Control-Allow-Credentials" : true
};
so I suppose CORS isn't the problem here. But I can't seem to figure out as to what differences it could make to have the requests run in the Extension vs. in the console/snippets. What could I be missing here?
I also do not see the request in AWS CloudWatch so I suppose it doesn't even leave my machine. I am testing on a Chrome User that has 0 extensions installed, same happens in incognito
To circle out any issues with my server I have also inserted the examples from https://lockevn.medium.com/snippet-to-fetch-remote-rest-api-and-display-in-console-of-chrome-devtool-6896a7cd6041
async function testGet() {
const response = await fetch('https://jsonplaceholder.typicode.com/posts')
console.log(await response.json())
}
async function testPost() {
let r = await fetch('https://jsonplaceholder.typicode.com/posts', {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
lockevn: 1
}),
})
console.log(await r.json())
}
testGet()
testPost()
Chrome's Network tab shows the request as stalled
The linked 'explanation' gives
Queueing: The browser queues requests when:
There are higher priority requests.
There are already six TCP connections open for this origin, which is
the limit. Applies to HTTP/1.0 and HTTP/1.1 only.
The browser is briefly allocating space in the disk cache
Stalled: The request could be stalled for any of the reasons described in Queueing.
Higher priority seems odd, 6 connections can't be the issue either since I have restarted my browser before testing, and the disk cache issue doesn't sound like the problem either. I'm not macOS with no anti virus

I managed to find the issue. In order to avoid potentially privileging my requests by opening the Chrome Developer Console in my AWS dashboard tab, I have created a new tab (chrome://new-tab-page/) and performed the requests in the console. This returned the errors described.
When I have updated my question with the example code I wanted to confirm if it was running before asking someone to try it if it works on their machine. For quick runtime-validation I opened the Console in the Stackoverflow tab and it worked. I only wanted to check if the code can be interpreted but it turned out to actually return a result. The same is valid for my AWS instance, if I run it on a https website it works fine. No idea why this is not documented but "disk cache" is mentioned as a potential error.
tldr don't open Chrome Console in new tab for requests in the console, use any website. This may have to do with CORS headers only working if the request doesn't have empty headers to begin with maybe (?)
I specifically avoided using a website console instance for testing because I wanted to prevent potential cookies on the AWS page from doing something that someone else couldn't do on their machine. Good thinking bad result haha
Thank you so much for your comments suggesting the help, much appreciated.

Why fetch() request fails in chrome console when URL is empty ? . But Works fine in Firefox

When I try to do this on chrome browser:
fetch('https://jsonplaceholder.typicode.com/users')
.then(response=> response.json())
.then(user=> console.log(user))
it seems net::ERR_FAILED

As #Sergiu Paraschiv noted in the comments, the core issue is CORS, and I highly recommend reading the link he provided.
However, just to elaborate slightly, for your specific case, of trying to make an AJAX request on the "start" or "no URL" page of a browser ...
... different browsers handle such "home page AJAX requests" differently, because such a page doesn't have a true URL.
Chrome (evidently) treats it as a URL of "", and so any URL would by definition violate CORS. In contrast, Firefox seems to ignore the CORS URL restrictions on requests made from the "home page".
More generally, if you're going to be testing your API without a client, I'd highly recommend using an independent request-making tool like Postman, as such tools will completely bypass CORS restrictions (and also let you make POST, DELETE, and other non-GET requests).

Set-Cookie is not working in Chrome - with two websites

I have a problem with Set-Cookie not working in Chrome (I didn't check other browsers). It worked in the past but it stopped working recently. I have two websites with two domain names, and I need to set the cookie in both websites. I'm calling a URL in each of the domain names to set the cookie. But it doesn't set the cookie on the other website (the website I'm not browsing now).
The users login or logout or sign up to one website, and I want them to login or logout from the other website too, automatically. Currently if they login or logout to one website, it doesn't affect the other website.
The Django view code is:
#csrf_exempt
def set_session(request):
"""
Cross-domain authentication.
"""
response = HttpResponse('')
origin = request.META.get('HTTP_ORIGIN')
if isinstance(origin, bytes):
origin = origin.decode()
netloc = urlparse(origin).netloc
if isinstance(netloc, bytes):
netloc = netloc.decode()
valid_origin = any(netloc.endswith('.' + site.domain) for site in Site.objects.all().order_by("pk"))
if (not (valid_origin)):
return response
if (request.method == 'POST'):
session_key = request.POST.get('key')
SessionStore = import_module(django_settings.SESSION_ENGINE).SessionStore
if ((session_key) and (SessionStore().exists(session_key))):
# Set session cookie
request.session = SessionStore(session_key)
request.session.modified = True
else:
# Delete session cookie
request.session.flush()
response['Access-Control-Allow-Origin'] = origin
response['Access-Control-Allow-Credentials'] = 'true'
return response
And the JavaScript code is:
window.speedy = {};
window.speedy.setSession = function (domain, key) {
$.ajax({
url: '//' + domain + '/set-session/',
method: 'post',
data: {
key: key
},
xhrFields: {
withCredentials: true
}
});
};
Then there is a JavaScript code that calls this function twice:
speedy.setSession('speedy.net', 'session_key');
speedy.setSession('speedymatch.com', 'session_key');
Where 'session_key' is replaced by the session key of the user.
Is there any solution to this problem? I think this is due to recent changes in Chrome.
Update: We have a staging server where both the websites domains are subdomains of the same registered domain name. And there, Set-Cookie works fine. But in the production websites, I think the other site's cookies are blocked by Chrome because the other site's domain is different from the domain the user is currently browsing.
I checked and the cookies from the other website also don't work with Firefox and Dolphin. It might be related to the upgrade to Django 2.1 which we upgraded recently.

The same origin policy for cookies being triggered here; from a domain you can set cookies for:
own domain
parent domain (unless the parent domain is a (g)TLD)
So as the two domains in question do not share the parent-child relationship and the only common parent of them could be the TLD (assuming same TLD), you can't do this.
From MDN doc:
Cookies use a separate definition of origins. A page can set a cookie for its own domain or any parent domain, as long as the parent domain is not a public suffix. Firefox and Chrome use the Public Suffix List to determine if a domain is a public suffix. Internet Explorer uses its own internal method to determine if a domain is a public suffix. The browser will make a cookie available to the given domain including any sub-domains, no matter which protocol (HTTP/HTTPS) or port is used. When you set a cookie, you can limit its availability using the Domain, Path, Secure and Http-Only flags. When you read a cookie, you cannot see from where it was set. Even if you use only secure https connections, any cookie you see may have been set using an insecure connection.

Thanks to #aaron I found out the problem. This problem started only recently, after I upgraded Django to 2.1. Django 2.1 introduced the SESSION_COOKIE_SAMESITE setting, which must be set to None for our websites to work properly with session cookies. On the other hand, CSRF_COOKIE_SAMESITE for our websites can be set to 'Strict', since we use separate CSRF cookies for each website. Therefore, I added the following lines to our base settings:
SESSION_COOKIE_SECURE = True
SESSION_COOKIE_SAMESITE = None
CSRF_COOKIE_SECURE = True
CSRF_COOKIE_SAMESITE = 'Strict'
From those lines, only SESSION_COOKIE_SAMESITE = None is necessary to fix the problem I mentioned in this question. I relied on the default setting of Django 2.1 to the value of SESSION_COOKIE_SAMESITE, which was not working for us in this case.
Currently the login and logout works in Chrome on my desktop and in one mobile phone. But I checked another mobile phone I have, and there it doesn't work - the problem persists as it was before. I'm not sure if this is due to a personal settings in this mobile phone or in the Chrome app? But login and logout to both websites simultaneously doesn't work there. If I login to one website, I'm still logged out from the other website, and vice versa.
Currently the login and logout works in Chrome. The problem was cookies settings - check your settings at chrome://settings/cookies (desktop) or settings > site settings > cookies (mobile).
(August 2020) Update: It is now required to use the following settings for Chrome. Please see this question and answer.
SESSION_COOKIE_SECURE = True
SESSION_COOKIE_SAMESITE = 'None'