I am trying to understand how http cache and fingerprinting
works. I've setup my express server to cache assets forever like this:
router.use('/public',
express.static(path.join(__dirname, '..', 'public'),
{ maxAge: 864000000 }));
I am expecting this to cache assets forever, even if i change the
content of the files, thus i will need to fingerprint the filenames to
bust the cache. However;
This is Google Chrome Headers output for static asset common.js after a reload
Remote Address:192.168.56.101:3000
Request URL:http://192.168.56.101:3000/public/assets2/scripts/app/common.js
Request Method:GET
Status Code:304 Not Modified
Request Headers
GET /public/assets2/scripts/app/common.js HTTP/1.1
Host: 192.168.56.101:3000
Connection: keep-alive
Cache-Control: max-age=0
Accept: */ *
If-None-Match: W/"ogrxaeWybJBlXMTTr2leWA=="
If-Modified-Since: Fri, 11 Jul 2014 13:46:01 GMT
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36
Referer: http://192.168.56.101:3000/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4
Response Headers
HTTP/1.1 304 Not Modified
X-Powered-By: Express
Accept-Ranges: bytes
Date: Fri, 11 Jul 2014 13:48:34 GMT
Cache-Control: public, max-age=864000
Last-Modified: Fri, 11 Jul 2014 13:46:01 GMT
ETag: W/"ogrxaeWybJBlXMTTr2leWA=="
Connection: keep-alive
Nice, i get a 304.
Now i change the contents of common.js and do a reload again, this is the output:
Remote Address:192.168.56.101:3000
Request URL:http://192.168.56.101:3000/public/assets2/scripts/app/common.js
Request Method:GET
Status Code:200 OK
Request Headers
GET /public/assets2/scripts/app/common.js HTTP/1.1
Host: 192.168.56.101:3000
Connection: keep-alive
Cache-Control: max-age=0
Accept: */ *
If-None-Match: W/"ogrxaeWybJBlXMTTr2leWA=="
If-Modified-Since: Fri, 11 Jul 2014 13:46:01 GMT
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36
Referer: http://192.168.56.101:3000/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4
Response Headers
HTTP/1.1 200 OK
X-Powered-By: Express
Accept-Ranges: bytes
Date: Fri, 11 Jul 2014 13:50:35 GMT
Cache-Control: public, max-age=864000
Last-Modified: Fri, 11 Jul 2014 13:50:33 GMT
ETag: W/"o65+0J5C8swpsmHMxNPH+w=="
Content-Type: application/javascript
Content-Length: 1908322
Connection: keep-alive
At this point, i was expecting to get a 304 but
appearently server detected the changes and sent a 200.
So i didn't have to use fingerprinting. Where did i go wrong?
Express is permanently caching it on the server side by keeping it in memory. My guess is that the express framework maintains cache consistency by checking whether or not cached resource has been modified.
Sending a request with a if-none-match and/or a if-modified-since header is the correct behavior for a user-agent. IE is attempting optimization by skipping a network round-trip, which may lead to incorrectly loaded pages.
What you need to do is either use fingerprinting - which assigns a new generic name to each modified resource - or have more low-level control about how your server serves resources, i.e. parsing the url yourself and defining rules about how responses are formed, a 304 response in your case.
I think the issue is with the Google Chrome, apparently when i hit reload, or press enter on url bar, Chrome still sends a If-None-Match request to server, and gets a 200. I tried with Internet Explorer, and it successfully server from cache without hitting the server. I am still wondering what is wrong with Chrome though, and how do i make it serve from cache without hitting the server.
Related
I'm working on the React + Express + mongo project.
Everything works locally, but when I deployed project on heroku.com, a problem appeared with GET requests.
It returns 304 and instead of body JSON, i'm getting this error:
To run this application, you need to enable JavaScript.
Instead of json response, I'm getting HTML response with this information about JS. In body I should get normal data JSON that I display on the page.
I will write again, everything WORKS locally ( i get 200 and json response), I don't know what's going on with this heroku.
I thought that problem is 304 code, so I used middleware that disables cache (no-cache), the result is 200, but there is still no body.
REQUEST ON CHROME NETWORK
GENERAL:
Request URL: MYURL.com/...
Request Method: GET
Status Code: 200 OK
Remote Address: 34.255.19.16:443
Referrer Policy: no-referrer-when-downgrade
RESPONSE HEADERS::
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Cache-Control: no-store, no-cache, must-revalidate, proxy-revalidate
Connection: keep-alive
Content-Length: 2366
Content-Type: text/html; charset=UTF-8
Date: Fri, 01 Nov 2019 13:54:09 GMT
Etag: W/"93e-16e271de930"
Expires: 0
Last-Modified: Fri, 01 Nov 2019 13:18:22 GMT
Pragma: no-cache
Server: Cowboy
Surrogate-Control: no-store
Via: 1.1 vegur
X-Powered-By: Express
REQUEST HEADERS:
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: pl-PL,pl;q=0.9,en-US;q=0.8,en;q=0.7
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkYTZkNjZhYTExYWI5MTM3MDU1MTY2OCIsIm5hbWUiOiJGaWxpcCIsImlhdCI6MTU3MjYxNjQ0MiwiZXhwIjoxNTcyNjIwMDQyfQ.c4uPda0njISa3VWNX0kK5cPRVW2X6A3wBNnt5hc5N-k
Connection: keep-alive
Content-Type: application/json
Cookie: AUTHORIZATION_JWT=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkYTZkNjZhYTExYWI5MTM3MDU1MTY2OCIsIm5hbWUiOiJGaWxpcCIsImlhdCI6MTU3MjYxNjQ0MiwiZXhwIjoxNTcyNjIwMDQyfQ.c4uPda0njISa3VWNX0kK5cPRVW2X6A3wBNnt5hc5N-k
Host: dashboard-pwa.herokuapp.com
Referer: https://dashboard-pwa.herokuapp.com/tickets/show/5dbb2310f67d543c84053a79
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36
I want to get 200 with JSON body response.
adam tropp thank you! It helped me a lot. It turns out that i had redirect to client index.html in wrong place, it should be placed after routes declaration, not before (just like I had before).
file server.js:
app.use('/api/user', require('./routes/user'));
// For any request that doesn't
// match one above, send back React's index.html file.
app.get('*', (req, res) => {
res.sendFile(path.join(__dirname+'/client/build/index.html'));
});
Thanks, once again.
I have a ReactJS client running webpack-dev-server on localhost:3000. It connects to a Hapi API server on localhost:8080 and I'm trying to provide a basic cookie using hapi-auth-jwt2 (I've also tried hapi-auth-cookie with equal results).
I can see the response header provides a valid set-cookie header and everything looks okay, but all my browser tests ignore it and the cookie is never set (verified by checking document.cookie and using the browser tools like Chrome's Application tab). When I connect directly to the API server with Postman, it picks up the set-cookie header correctly and stores it so I think it's just some kind of domain/port/host configuration issue.
As a simple test, I tried deploying to our ec2 environment but that didn't help. The ec2 environment is similar, with one instance serving the client and another instance serving the API. I've also tried modifying my local hosts file to redirect a domain like 127.0.0.1 example.com and providing the domain=.example.com field in the cookie, but that also didn't help.
I think I'm just missing something basic but I don't know what it is. See below for response/request headers on login.
Request Headers
POST /login HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 47
Accept: application/json, text/plain, */*
Origin: http://localhost:3000
Authorization: undefined
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36
Content-Type: application/json
Referer: http://localhost:3000/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Response Headers
HTTP/1.1 200 OK
authorization: <jwt token>
vary: origin,accept-encoding
access-control-allow-origin: http://localhost:3000
access-control-allow-credentials: true
access-control-expose-headers: WWW-Authenticate,Server-Authorization
content-type: application/json; charset=utf-8
set-cookie: cookie=token; Max-Age=604800; Expires=Wed, 16 May 2018 21:11:23 GMT; SameSite=Lax; Path=/
cache-control: no-cache
content-encoding: gzip
Date: Wed, 09 May 2018 21:11:23 GMT
Connection: keep-alive
Transfer-Encoding: chunked
http-proxy-middleware, which webpack-dev-server uses, has options for cookie domain/path rewrites.
You should see if those satisfy your needs. Otherwise you can also manually parse and reset cookies in the onProxyRes callback. Here is an example.
The setup uses a Backbone Model, Nginx server. The users enter their username and password which is then passed via a post. The server authenticates and returns a session cookie.
When the backend and front-end are on the same server (e.g. connect via localhost) the cookie is stored. However when the connection is remote, it is not stored in Chrome; however, it is stored in Safari and FireFox.
Ajax is setup via
$.ajaxSetup({
crossDomain: true,
xhrFields: {
withCredentials: true
}
});
The request headers are
POST /login HTTP/1.1
Host: localhost:8102
Connection: keep-alive
Content-Length: 59
Accept: application/json, text/javascript, */*; q=0.01
Origin: http://127.0.0.1:9102
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36
Content-Type: application/json
DNT: 1
Referer: http://127.0.0.1:9102/somefolder
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
The response headers are
HTTP/1.1 200 OK
access-control-allow-origin: http://127.0.0.1:9102
access-control-allow-credentials: true
vary: origin,accept-encoding
access-control-expose-headers: WWW-Authenticate,Server-Authorization
content-type: text/html; charset=utf-8
set-cookie: na-auth-token=encrypted-string; Max-Age=86400; Expires=Thu, 27 Apr 2017 13:32:44 GMT; HttpOnly; SameSite=Strict; Path=/
cache-control: no-cache
content-encoding: gzip
Date: Wed, 26 Apr 2017 13:32:44 GMT
Connection: keep-alive
Transfer-Encoding: chunked
In FireFox and Safari the cookie is stored just fine, but in Chrome it gets the response and tosses the cookie without any notification.
Update
The cookie is actually being saved under the localhost domain, however when you navigate back to the page (e.g. via a window.location.reload) the cookie disappears.
So the answer to my question lied in a library we were using
hapi-auth-cookie
The package updated and a flag was introduced isSameSite. Changed this value to false to allow the cors cookie to persist between pages loads.
I added ASP.NET javascript bundling and minfication to our website, and have discovered that now if I refresh the website, the bundled javascript file is downloaded as gzip but never uncompressed by the browser, so the site of course doesn't work since the javascript is still compressed. If I refresh again, the site is fine. Refresh again, corrupt. Back and forth. This is ONLY happening in Chrome and Safari, but not in IE.
Watching in Fiddler, IE gets the javascript the first time, and subsequent refreshes return 304 not modified which is correct. Chrome / Safari refreshes continually return 200 for each refresh.
If I run my website locally hosting it in IIS Express, this problem does NOT occur. It's only in our other QA, Staging and Production environments where the website is hosted by IIS 7.5.
I did a Fiddler compare of the good Chrome request and the bad Chrome request to see what's different. They are identical except the bad request has the "If-Modified-Since" header.
The response from the server when the "If-Modified-Since" header is there has a Content-Type of "application/javascript", and this is what I think is causing the problem, the browser doesn't know it should uncompress the response body.
Here is the GOOD request and response (body omitted for brevity):
GET https://wwwq.website.com/Scripts/main.js?v=z3QOtK2bjf3mvSQyLf2KY82lAidw4JR0ePo01WHF93U1 HTTP/1.1
Host: www.website.com
Connection: keep-alive
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36
DNT: 1
Referer: https://www.website.com/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: < cookies omitted >
HTTP/1.1 200 OK
Date: Fri, 05 Feb 2016 13:46:30 GMT
Server: Microsoft-IIS/7.5
Cache-Control: public
Content-Type: text/javascript; charset=utf-8
Expires: Sat, 04 Feb 2017 13:46:31 GMT
Last-Modified: Fri, 05 Feb 2016 13:46:31 GMT
Vary: User-Agent,Accept-Encoding
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Via: 1.1 wwwq.website.com (Access Gateway-ag-2772908347919084-467794)
Keep-Alive: timeout=300, max=99
Connection: Keep-Alive
Content-Length: 62702
Here is the BAD request and response (body omitted for brevity):
GET https://www.website.com/Scripts/main.js?v=z3QOtK2bjf3mvSQyLf2KY82lAidw4JR0ePo01WHF93U1 HTTP/1.1
Host: www.website.com
Connection: keep-alive
Cache-Control: max-age=0
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36
DNT: 1
Referer: https://www.website.com/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: < cookies omitted >
If-Modified-Since: Fri, 05 Feb 2016 13:46:31 GMT
HTTP/1.1 200 OK
Date: Fri, 05 Feb 2016 13:46:43 GMT
Server: Microsoft-IIS/7.5
Cache-Control: private
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Vary: Accept-Encoding
Via: 1.1 www.website.com (Access Gateway-ag-2772908347919084-467849)
Content-Length: 16042
Keep-Alive: timeout=300, max=94
Connection: Keep-Alive
Content-Type: application/javascript
If anyone can help me out I would be incredibly grateful I am running out of ideas.
My IE, Chrome, Chrome androud, Opera, Firefox browsers successfully connected to my IIS by WebSocket. See an example:
The handshake from the client looks as follows:
GET /SimpleChatApplication/ChatHandler.ashx HTTP/1.1
Host: pc2014:80
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http//pc2014
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: ru,en;q=0.8,en-US;q=0.6
Sec-WebSocket-Key: rrKYVVaPKmMnINtsUh9BNg==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
The handshake from the server looks as follows:
HTTP/1.1 101 Switching Protocols
Cache-Control: private
Upgrade: Websocket
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
Sec-WebSocket-Accept: IQ/qcgyu36N/d1p0JEU7nZ3W8Vo=
Connection: Upgrade
X-Powered-By: ASP.NET
Date: Wed, 16 Sep 2015 23:54:37 GMT
But if I use Safary bowser then:
The handshake from the client looks as follows:
GET /SimpleChatApplication/ChatHandler.ashx HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: pc2014:80
Origin: http://pc2014
Sec-WebSocket-Key1: N9 Q32K\275z 5z < 1= 2 c
Sec-WebSocket-Key2: -1 z 6#HA g)1Aue0483 420d
The handshake from the server looks as follows:
HTTP/1.1 200 OK
Cache-Control: private
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 16 Sep 2015 23:56:19 GMT
Content-Length: 0
HTTP/1.1 400 Bad Request
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Wed, 16 Sep 2015 23:56:19 GMT
Connection: close
Content-Length: 326
Bad Request
Bad Request - Invalid Verb
HTTP Error 400. The request verb is invalid.
Please explain me where is error? Or Safary sends incorrect request, or IIS recognizes incorrectly.
I have used www.websocket.org site for testing. I have replaced the server address from "ws://echo.websocket.org" to my IIS address before testing
It seems that Safari implements an old version of the WebSocket standard, named hybi.
Apparently ASP.NET only supports the definitive one, RFC6455. That is the version that major browsers use as well.
Use the SignalR library. ASP.NET SignalR is a new library for ASP.NET developers that makes it incredibly simple to add real-time web functionality to your applications. What is "real-time web" functionality? It's the ability to have your server-side code push content to the connected clients as it happens, in real-time.