I have a web application that lets the browser cache AJAX requests result for a long time. I have found out how to make a request that bypasses the cache entirely, when probable modifications are detected. But I would want to let the user trigger a data refresh.
In this scenario, I'd like the browser to check with the server if the cache is stalled but use it if it is not (that is, if the server responds with a 304 code). The goal is to spare the loading time because the data is huge.
The server includes the following headers in all responses:
Cache-Control: private, max-age=604800
Last-Modified: ... # actual last modification date
I managed to burst the cached object entirely in Chrome (not tested other browsers yet) by using the following HTTP headers in the request:
Cache-Control: max-age=0
If-Last-Modified: Tue, 01 Jan 1970 01:00:00 +0100
The If-Last-Modified line is the one that really has an effect. Chrome seems to ignore the Cache-Control header in the request.
I have also found that using Cache-Control: must-revalidate in the server response forces the browser to validate its cache with the server for each request.
But is there any way to revalidate for just one precise request, decided on the client-side?
Note that I'm not specially attached to doing this with HTTP headers, so any other method that I would not be aware of is welcome!
you can add a url parameter which value base on time to clean cache for just one precise request.
$.ajax({
url:"/questions?nocache="+Date.now(),
"success":function(data){
console.log(data);
}
});
Related
I am really sorry about if I am missing something very basic here, but here goes...
BRIEF: My question is the same as the one found here: How to set headers while requesting a page in nodejs?, and Mr Khan's answer there is just falling short of explaining how to set the headers from backend (Node.js). I would have commented there, but I don't have enough Karma for that :(
This is what I've done so far:
const newTokens = await jwt.generateJWT(user); // generateJWT is a custom function that returns two tokens
res.setHeader("Authorization", `Bearer ${newTokens.accessToken}`);
res.setHeader("refresh-token", newTokens.refreshToken);
return res.redirect("/member/dashboard");
The above code is able to send the HTTP headers to the browser, but is not able to set them on the browser for the domain.
The response headers as in Firefox are:
HTTP/1.1 302 Found
X-Powered-By: Express
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoiNjA3MDgyNDlmNjBjNjE1YWU4NTdjMmU4IiwidXNlcl9yb2xlIjoibWVtYmVyIiwidXNlcl9uYW1lIjoiQWxleCIsImlhdCI6MTYxNzk5OTM5NywiZXhwIjoxNjE3OTk5OTk3fQ.Odb6TrWBnf9dq00T_ddxD9hqVjhFQYdqA5pP2u6y-2k
refresh-token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoiNjA3MDgyNDlmNjBjNjE1YWU4NTdjMmU4IiwiaWF0IjoxNjE3OTk5Mzk3LCJleHAiOjE2MTc5OTk5OTd9.kY9DZWprHxZFMI3btX-yzZxiUrqZY3kdmxzyc3apAyw
Location: /member/dashboard
Vary: Accept
Content-Type: text/html; charset=utf-8
Content-Length: 78
Date: Fri, 09 Apr 2021 20:16:37 GMT
Connection: keep-alive
Note: The "Authorization" and "refresh-token" headers have been sent, and the redirect "location"
has also been set causing the 302 status code.
Unfortunately, the headers don't seem to be returning on all subsequent requests from the client as the headers are not being set.
Please let me know if I am doing something obviously wrong.
EDIT: The reason I am trying to do this from the backend directly is that I don't want to depend on the frontend to handle this job, as I do not intend on implementing a framework-specific frontend, i.e., it should work across all frameworks.
PS: Forgive me if my English is bad, it isn't my native language.
When you do res.redirect(), the browser will NOT apply the headers you set on that response to the redirected request. Those headers are part of the response back to the requesting client and that's all. They will NOT be sent with the redirected request.
Headers on the redirected request cannot be controlled by the server. Browsers just don't work that way so you can't design things that way if you're relying on a standard browser to be the client.
If you're using redirection and you want something sent back with the redirection, then your best option is typically to put stuff into a cookie or into the query string of the redirect URL. That cookie or query string will be sent with the redirected request and the server can get it from there.
You could also establish a server-side session and put data into that session. This will set a session cookie which will be present on future client requests and the server can then access data from the server-side session object on future requests from that client.
We noticed Chrome caches files locally and doesn't even send a request to our server to check if there's a newer version of the javascript file.
Example of HTTP response headers for a js file that Google cached:
Accept-Ranges:bytes
Access-Control-Allow-Headers:Content-Type
Access-Control-Allow-Methods:GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Origin:*
Content-Encoding:gzip
Content-Length:5479
Content-Type:application/javascript
Date:Tue, 12 Jan 2016 22:46:07 GMT
ETag:"7d68e1ceb647d11:0"
Last-Modified:Tue, 05 Jan 2016 12:44:25 GMT
Server:Microsoft-IIS/8.5
Vary:Accept-Encoding
x-robots-tag:noindex
Is it valid that Chrome cached the file? There's no Cache-control header or something that declares that the file can be cached locally, it only has ETag and Last-Modified.
BTW
Is there a way (maybe a header) to instruct Chrome to check if the cached file has changed without appending version to the file name? Setting no-cache is not an option since I do want it to be cached, but I want to use the ETag and Last-Modified headers as it should.
Unless specifically constrained by a cache-control (section 14.9)
directive, a caching system MAY always store a successful response
(see section 13.8) as a cache entry, MAY return it without validation
if it is fresh, and MAY return it after successful validation.
You can always use the must-revalidate directive.
When the must-revalidate directive is present in a response received
by a cache, that cache MUST NOT use the entry after it becomes stale
to respond to a subsequent request without first revalidating it with
the origin server.
Source
I wish to implement ETag based cache-control, and I am wondering how long ETags are stored in a browser and if they can be kept for longer than the current session.
I have the server setting and sending ETags for responses correctly, and I am sending requests from the browser with jQuery.ajax.
The system works correctly, responding with the appropriate 304 response if the Not-Modified header (previous ETag) matches the generated ETag on the server. This all works fine for the current session, however when I close the browser and reopen it, the Not-Modified header is not being sent by jQuery any more.
I am wondering if there are any methods I could use to trigger the default browser behaviour, and send the Not-Modified header for responses (if applicable), without manually setting the header in the ajax request or storing the ETag in localStorage/indexedDb.
An Etag must be accompanied by other tags that tell the browser how long to hold the content
https://developers.google.com/speed/docs/best-practices/caching?csw=1#LeverageBrowserCaching
It is important to specify one of Expires or Cache-Control max-age, and one of Last-Modified or ETag, for all cacheable resources. It is redundant to specify both Expires and Cache-Control: max-age, or to specify both Last-Modified and ETag.
So it all depends on the duration you specify with the Expires or max-age tag on how long the browser will hold it
The only purpose of the etag is so that the browser can issue a "if modified" request after the initial expiration period to see if the content has actually changed or gets back a 304 not modified.
Badly managed 304's can cause other problems, like unnecessary and redundant 304 requests: http://calendar.perfplanet.com/2010/easy-cache-headers/
How do I send an HTTP response that any client would cache forever (or until its cache is cleared), such that when the browser needs that resource, it makes no HTTP request and instead retrieves the HTTP response from the local file system?
Usage note: this is for versioned client code in an Ajax application. Everything is accessed through the uncacheable example.com/front.htm, which merely contains a script tag linking to example.com/currentversion/bootstrap.js which is cached indefinitely. Because the uncacheable HTML file determines the current version, there is no need for the client to update scripts.
According to the RFC, "to mark a response as 'never expires,' an origin server sends an Expires date approximately one year from the time the response is sent. HTTP/1.1 servers SHOULD NOT send Expires dates more than one year in the future."
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.21.
I don't know what the reasoning behind the one-year limitation is, so take it for what it is.
Cache until 2038. You can't go any farther than that (reliably) because of the 32-bit Unix Epoch bug. Use this header:
Expires: Sun, 17 Jan 2038 19:14:07 GMT
I am working on an ajax long polling type application, and I would like to minimize the amount of bandwidth I am using. One of the big costs right now are the client side HTTP headers. Once I have a connection established and a session id stored on the client, I don't really want to squander any more bandwidth transferring redundant http information (such as browser type, accept encodings, etc.). Over the course of many connections, this quickly adds up to a lot of data!
I would really like to just take my XMLHttpRequest and nuke all of the headers so that only the absolute minimum gets transmitted to the server. Is it possible to do this?
You have very little control over request headers, but you can still do a few things -
Reduce the size of the cookie. In general, you only want the session id, everything else can be eliminated and stored server side.
Minimize http referrer by keeping a short URL. The longer your page url, the more data will have to be sent via the http referrer. One trick is to store data in the fragment identifier (the portion of the url after the #). The fragment identifier is never sent to the server, so you save a few bytes over there.
Some request headers are only sent if you had previous set corresponding response headers. For example, you can indirectly control the ETag and if-modified-since request headers.
You may want to consider Web Sockets. Support is pretty good (IE10+).
You may be able to override some of the standard headers using setRequestHeader() before sending the request, but it is possible the browser may not allow overriding of some and it seems there is no way to get a list of headers (besides asking the server to echo them back to you) to know which to try to override.
I think it's possible to remove all headers at least in some browsers.
Take a look at the communication between gmail/calendar apps and the backend from google in chrome (it's not the same in firefox)
it's possible google has some hidden api for the XMLHttpRequest object,
you'll see something like the below output (notice there is no request headers section):
Request URL:https://mail.google.com/mail/u/0/channel/bind?XXXXXXXXXXXXXX
Request Method:POST
Status Code:200 OK
Query String Parameters
OSID:XXXXXXXXXXXXX
OAID:XXXXXXXXX
VER:8
at:XXXXXXXXXXXXXX
it:30
SID:XXXXXXXXXXXX
RID:XXXXXXXXX
AID:XXXXXXXXXX
zx:XXXXXXXXXXXX
t:1
Request Payload
count=1&ofs=211&req0_type=cf&req0_focused=1&req0__sc=c
Response Headers
cache-control:no-cache, no-store, max-age=0, must-revalidate
content-encoding:gzip
content-type:text/plain; charset=utf-8
date:Tue, 09 Oct 2012 08:52:46 GMT
expires:Fri, 01 Jan 1990 00:00:00 GMT
pragma:no-cache
server:GSE
status:200 OK
version:HTTP/1.1
x-content-type-options:nosniff
x-xss-protection:1; mode=block