Develop Reference

Using http request headers to enforce caching of response

I'm working with a public API that is poorly configured, i.e. the server doesn't set the Cache-Control header when it should.
Can I use HTTP request headers or alter response headers, so that a response is cached at the browser level for some period of time?
In the context of this API the response will be fresh until a specific data time calculated previously.
I looked quite a bit, but cannot seem to find anything related to setting request headers.
Note, I'm working in javascript and using fetch() to make requests.
Here is the state of the current headers:
Request Headers
:authority: fantasy.premierleague.com
:method: GET
:path: /api/entry/1157414/event/3/picks/
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-GB,en-US;q=0.9,en;q=0.8
sec-ch-ua: "Google Chrome";v="87", " Not;A Brand";v="99", "Chromium";v="87"
sec-ch-ua-mobile: ?0
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-origin
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_1_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
Response Headers
accept-ranges: bytes
content-length: 571
content-type: text/html
date: Fri, 25 Dec 2020 16:57:32 GMT
server: nginx/1.18.0
via: 1.1 google, 1.1 varnish, 1.1 varnish
x-cache: MISS, MISS
x-cache-hits: 0, 0
x-served-by: cache-lhr7366-LHR, cache-fra19130-FRA
x-timer: S1608915452.118518,VS0,VE22

304 and no json response on deployed app on Heroku

I'm working on the React + Express + mongo project.
Everything works locally, but when I deployed project on heroku.com, a problem appeared with GET requests.
It returns 304 and instead of body JSON, i'm getting this error:
To run this application, you need to enable JavaScript.
Instead of json response, I'm getting HTML response with this information about JS. In body I should get normal data JSON that I display on the page.
I will write again, everything WORKS locally ( i get 200 and json response), I don't know what's going on with this heroku.
I thought that problem is 304 code, so I used middleware that disables cache (no-cache), the result is 200, but there is still no body.
REQUEST ON CHROME NETWORK
GENERAL:
Request URL: MYURL.com/...
Request Method: GET
Status Code: 200 OK
Remote Address: 34.255.19.16:443
Referrer Policy: no-referrer-when-downgrade
RESPONSE HEADERS::
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Cache-Control: no-store, no-cache, must-revalidate, proxy-revalidate
Connection: keep-alive
Content-Length: 2366
Content-Type: text/html; charset=UTF-8
Date: Fri, 01 Nov 2019 13:54:09 GMT
Etag: W/"93e-16e271de930"
Expires: 0
Last-Modified: Fri, 01 Nov 2019 13:18:22 GMT
Pragma: no-cache
Server: Cowboy
Surrogate-Control: no-store
Via: 1.1 vegur
X-Powered-By: Express
REQUEST HEADERS:
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: pl-PL,pl;q=0.9,en-US;q=0.8,en;q=0.7
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkYTZkNjZhYTExYWI5MTM3MDU1MTY2OCIsIm5hbWUiOiJGaWxpcCIsImlhdCI6MTU3MjYxNjQ0MiwiZXhwIjoxNTcyNjIwMDQyfQ.c4uPda0njISa3VWNX0kK5cPRVW2X6A3wBNnt5hc5N-k
Connection: keep-alive
Content-Type: application/json
Cookie: AUTHORIZATION_JWT=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkYTZkNjZhYTExYWI5MTM3MDU1MTY2OCIsIm5hbWUiOiJGaWxpcCIsImlhdCI6MTU3MjYxNjQ0MiwiZXhwIjoxNTcyNjIwMDQyfQ.c4uPda0njISa3VWNX0kK5cPRVW2X6A3wBNnt5hc5N-k
Host: dashboard-pwa.herokuapp.com
Referer: https://dashboard-pwa.herokuapp.com/tickets/show/5dbb2310f67d543c84053a79
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
I want to get 200 with JSON body response.

adam tropp thank you! It helped me a lot. It turns out that i had redirect to client index.html in wrong place, it should be placed after routes declaration, not before (just like I had before).
file server.js:
app.use('/api/user', require('./routes/user'));
// For any request that doesn't
// match one above, send back React's index.html file.
app.get('*', (req, res) => {
res.sendFile(path.join(__dirname+'/client/build/index.html'));
});
Thanks, once again.

Setting a cookie with a webpack-dev-server client and hapi API

I have a ReactJS client running webpack-dev-server on localhost:3000. It connects to a Hapi API server on localhost:8080 and I'm trying to provide a basic cookie using hapi-auth-jwt2 (I've also tried hapi-auth-cookie with equal results).
I can see the response header provides a valid set-cookie header and everything looks okay, but all my browser tests ignore it and the cookie is never set (verified by checking document.cookie and using the browser tools like Chrome's Application tab). When I connect directly to the API server with Postman, it picks up the set-cookie header correctly and stores it so I think it's just some kind of domain/port/host configuration issue.
As a simple test, I tried deploying to our ec2 environment but that didn't help. The ec2 environment is similar, with one instance serving the client and another instance serving the API. I've also tried modifying my local hosts file to redirect a domain like 127.0.0.1 example.com and providing the domain=.example.com field in the cookie, but that also didn't help.
I think I'm just missing something basic but I don't know what it is. See below for response/request headers on login.
Request Headers
POST /login HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 47
Accept: application/json, text/plain, */*
Origin: http://localhost:3000
Authorization: undefined
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36
Content-Type: application/json
Referer: http://localhost:3000/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Response Headers
HTTP/1.1 200 OK
authorization: <jwt token>
vary: origin,accept-encoding
access-control-allow-origin: http://localhost:3000
access-control-allow-credentials: true
access-control-expose-headers: WWW-Authenticate,Server-Authorization
content-type: application/json; charset=utf-8
set-cookie: cookie=token; Max-Age=604800; Expires=Wed, 16 May 2018 21:11:23 GMT; SameSite=Lax; Path=/
cache-control: no-cache
content-encoding: gzip
Date: Wed, 09 May 2018 21:11:23 GMT
Connection: keep-alive
Transfer-Encoding: chunked

http-proxy-middleware, which webpack-dev-server uses, has options for cookie domain/path rewrites.
You should see if those satisfy your needs. Otherwise you can also manually parse and reset cookies in the onProxyRes callback. Here is an example.

Chrome not persisting CORS cookie

The setup uses a Backbone Model, Nginx server. The users enter their username and password which is then passed via a post. The server authenticates and returns a session cookie.
When the backend and front-end are on the same server (e.g. connect via localhost) the cookie is stored. However when the connection is remote, it is not stored in Chrome; however, it is stored in Safari and FireFox.
Ajax is setup via
$.ajaxSetup({
crossDomain: true,
xhrFields: {
withCredentials: true
}
});
The request headers are
POST /login HTTP/1.1
Host: localhost:8102
Connection: keep-alive
Content-Length: 59
Accept: application/json, text/javascript, */*; q=0.01
Origin: http://127.0.0.1:9102
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36
Content-Type: application/json
DNT: 1
Referer: http://127.0.0.1:9102/somefolder
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
The response headers are
HTTP/1.1 200 OK
access-control-allow-origin: http://127.0.0.1:9102
access-control-allow-credentials: true
vary: origin,accept-encoding
access-control-expose-headers: WWW-Authenticate,Server-Authorization
content-type: text/html; charset=utf-8
set-cookie: na-auth-token=encrypted-string; Max-Age=86400; Expires=Thu, 27 Apr 2017 13:32:44 GMT; HttpOnly; SameSite=Strict; Path=/
cache-control: no-cache
content-encoding: gzip
Date: Wed, 26 Apr 2017 13:32:44 GMT
Connection: keep-alive
Transfer-Encoding: chunked
In FireFox and Safari the cookie is stored just fine, but in Chrome it gets the response and tosses the cookie without any notification.
Update
The cookie is actually being saved under the localhost domain, however when you navigate back to the page (e.g. via a window.location.reload) the cookie disappears.

So the answer to my question lied in a library we were using
hapi-auth-cookie
The package updated and a flag was introduced isSameSite. Changed this value to false to allow the cors cookie to persist between pages loads.

Connecting to IIS 8 from Safari 5.1.7 by WebSocket

My IE, Chrome, Chrome androud, Opera, Firefox browsers successfully connected to my IIS by WebSocket. See an example:
The handshake from the client looks as follows:
GET /SimpleChatApplication/ChatHandler.ashx HTTP/1.1
Host: pc2014:80
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http//pc2014
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: ru,en;q=0.8,en-US;q=0.6
Sec-WebSocket-Key: rrKYVVaPKmMnINtsUh9BNg==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
The handshake from the server looks as follows:
HTTP/1.1 101 Switching Protocols
Cache-Control: private
Upgrade: Websocket
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
Sec-WebSocket-Accept: IQ/qcgyu36N/d1p0JEU7nZ3W8Vo=
Connection: Upgrade
X-Powered-By: ASP.NET
Date: Wed, 16 Sep 2015 23:54:37 GMT
But if I use Safary bowser then:
The handshake from the client looks as follows:
GET /SimpleChatApplication/ChatHandler.ashx HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: pc2014:80
Origin: http://pc2014
Sec-WebSocket-Key1: N9 Q32K\275z 5z < 1= 2 c
Sec-WebSocket-Key2: -1 z 6#HA g)1Aue0483 420d
The handshake from the server looks as follows:
HTTP/1.1 200 OK
Cache-Control: private
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 16 Sep 2015 23:56:19 GMT
Content-Length: 0
HTTP/1.1 400 Bad Request
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Wed, 16 Sep 2015 23:56:19 GMT
Connection: close
Content-Length: 326
Bad Request
Bad Request - Invalid Verb
HTTP Error 400. The request verb is invalid.
Please explain me where is error? Or Safary sends incorrect request, or IIS recognizes incorrectly.
I have used www.websocket.org site for testing. I have replaced the server address from "ws://echo.websocket.org" to my IIS address before testing

It seems that Safari implements an old version of the WebSocket standard, named hybi.
Apparently ASP.NET only supports the definitive one, RFC6455. That is the version that major browsers use as well.

Use the SignalR library. ASP.NET SignalR is a new library for ASP.NET developers that makes it incredibly simple to add real-time web functionality to your applications. What is "real-time web" functionality? It's the ability to have your server-side code push content to the connected clients as it happens, in real-time.