We would like to know from where the initial html document itself was served, either directly from the browser cache, our cdn cache or from origin. The later are easy (since our cdn adds server-timing to indicate a cache hit/miss), but figuring out if the document was served directly from the browser turned out more difficult.
I know about developer tools/network tab and how Chrome for example can show this information, but this question is specifically about any JavaScript API that can be used to detect and report it to us.
So far I've tried three things:
var isCached = performance.getEntriesByType("navigation")[0].transferSize === 0; from this answer, but today this seems to report the previous transferSize. When I try the same with the latest Chrome, I never get transferSize === 0, even when the devTools show me that it was cached. Possibly it only works for other resources, but not the html document itself.
var isCached = window.performance.navigation.type === 2 according to this answer gets the navigation type (in this case backward/forward), which is not always a true indicator of the document being cached. E.g. clicking a link is of type navigation and can also be cached.
Storing the timestamp in the document itself as suggested here on the server and comparing it does not work either, especially since we are using a cdn that has its own cache. We wouldn't be able to differentiate between a cached version from cdn or the browser.
From the MDN documentation and the PerformanceNavigationTiming interface described here, we can see that :
the requestStart attribute returns the time immediately before the user agent starts requesting the current document from the server, or from the HTTP cache or from local resources
We also have at our disposal the responseStart attribute :
responseStart returns the time after the user agent receives the first byte of the response from the server, or from the HTTP cache or from local resources
So, you should be able to detect when your initial HTML Document is served from the browser cache when the timing between the requestStart and responseStart is identical or almost identical.
It will be something like the following:
const timingDelta = 10; // to be conservative, but should be less than 5ms
const performanceEntry = performance.getEntriesByName("<URL of your document>")[0]; // ex: performance.getEntriesByName("https://www.google.com/")[0];
if (performanceEntry.responseStart - performanceEntry.requestStart < delta) {
console.log('document was served from cache');
}
// that info is also available on the performance.timing object but it's deprecated
// for your information, that would look like this:
if (performance.timing.responseStart - performance.timing.resquestStart < delta) { ... }
Why not checking http response code ?
Especially the 3xx codes.
304 Not Modified
Indicates that the resource has not been modified since the version specified by the request headers If-Modified-Since or If-None-Match. In such case, there is no need to retransmit the resource since the client still has a previously-downloaded copy.
See https://en.wikipedia.org/wiki/List_of_HTTP_status_codes
Our site has an asynchronously loaded application.js:
<script async="async" src="//...application-123456.js"></script>
Additionally, we have a lot of third party scripts that (1) are asynchronously loaded, and (2) create in turn an async <script> tag where a bigger script is called.
Just to give an example, one of these third party scripts is Google's gpt.js (you can have a quick look to understand how it works).
Our problem is that, while all the third party scripts load asynchronously as expected, the application.js one gets stack in "queuing" status for more than 4 seconds.
I tried to change the script and make it load like the third party ones: create a <script> element, set the "src" attribute and load it:
<script async>
(function() {
var elem = document.createElement('script');
elem.src = 'http://...application-123456.js';
elem.async = true;
elem.type = 'text/javascript';
var scpt = document.getElementsByTagName('script')[0];
scpt.parentNode.insertBefore(elem, scpt);
})();
</script>
but nothing changed.
Then I studied the network cascade in a page of our site that almost doesn't contain images, and I saw that the queuing time was almost zero. I tried the same experiment in pages with different amounts of images, and saw that the queuing time proportionally increases in pages with more images.
I read this in Chrome's network cascade documentation:
QUEUING TIME: The request was postponed by the rendering engine because it's considered lower priority than critical resources (such as scripts/styles). This often happens with images.
Is it possible that for some reason the browser is marking our application.js as "lower priority"? I looked on the web and it seems that nobody has experienced problems with the queuing time. Anybody has an idea?
Thank you very much.
Browsers use a pre-loader to improve network utilisation. This article explains the concept.
In the Chrome Documentation you linked to above, it says the following about queuing:
If a request is queued it indicated that:
The request was postponed by: the rendering engine because it's considered lower priority than critical resources (such as scripts/styles). This often happens with images.
The request was put on hold to wait for an unavailable TCP
socket that's about to free up. The request was put on hold because the browser only allows six TCP connections per origin on HTTP 1.
Time spent making disk cache entries (typically very quick.)
The pre-loader would have retrieved the lightweight resources quickly, such as the styles and scripts, and then queued up the images because, as the criteria above suggests, only 6 TCP connections are permitted per origin. Therefore, this would explain the delay in the total response time.
I noticed something odd regarding ajax and image loading. Suppose you have an image on the page, and ajax requests the same image - one would guess that ajax requests would hit the browser cache, or it should at least only make one request, the resulting image going to the page and the script that wants to read/process the image.
Surprisingly, I found that even when the javascript waits for the entire page to load, the image request still makes a new request! Is this a known bug in Firefox and Chrome, or something bad jQuery ajax is doing?
Here you can see the problem, open Fiddler or Wireshark and set it to record before you click "run":
<script src="http://code.jquery.com/jquery-1.11.1.min.js"></script>
<div id="something" style="background-image:url(http://jsfiddle.net/img/logo-white.png);">Hello</div>
<script>
jQuery(function($) {
$(window).load(function() {
$.get('http://jsfiddle.net/img/logo-white.png');
})
});
</script>
Note that in Firefox it makes two requests, both resulting in 200-OK, and sending the entire image back to the browser twice. In Chromium, it at least correctly gets a 304 on second request instead of downloading the entire contents twice.
Oddly enough, IE11 downloads the entire image twice, while it seems IE9 aggressively caches it and downloads it once.
Ideally I would hope the ajax wouldn't make a second request at all, since it is requesting exactly the same url. Is there a reason css and ajax in this case usually have different caches, as though the browser is using different cache storage for css vs ajax requests?
I use the newest Google Chrome and it makes one request. But in your JSFIDDLE example you are loading jQuery twice. First with CSS over style attribute and second in your code over script tag. Improved: JSFIDDLE
<div id="something" style="background-image:url('http://jsfiddle.net/img/logo-white.png');">Hello</div>
<script>
jQuery(window).load(function() {
jQuery.get('http://jsfiddle.net/img/logo-white.png');
});
// or
jQuery(function($) {
jQuery.get('http://jsfiddle.net/img/logo-white.png');
});
</script>
jQuery(function($) {...} is called when DOM is ready and jQuery(window).load(...); if DOM is ready and every image and other resources are loaded. To put both together nested makes no sense, see also here: window.onload vs $(document).ready()
Sure, the image is loaded two times in Network tab of the web inspector. First through your CSS and second through your JavaScript. The second request is probably cached.
UPDATE: But every request if cached or not is shown in this tab. See following example: http://jsfiddle.net/95mnf9rm/4/
There are 5 request with cached AJAX calls and 5 without caching. And 10 request are shown in 'Network' tab.
When you use your image twice in CSS then it's only requested once. But if you explicitly make a AJAX call then the browser makes an AJAX call. As you want. And then maybe it's cached or not, but it's explicitly requested, isn't it?
This "problem" could a be a CORS pre-flight test.
I had noticed this in my applications awhile back, that the call to retrieve information from a single page application made the call twice. This only happens when you're accessing URLs on a different domain. In my case we have APIs we've built and use on a different server (a different domain) than that of the applications we build. I noticed that when I use a GET or POST in my application to these RESTFUL APIs the call appears to be made twice.
What is happening is something called pre-flight (https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS), an initial request is made to the server to see if the ensuing call is allowed.
Excerpt from MDN:
Unlike simple requests, "preflighted" requests first send an HTTP request by the OPTIONS method to the resource on the other domain, in order to determine whether the actual request is safe to send. Cross-site requests are preflighted like this since they may have implications to user data. In particular, a request is preflighted if:
It uses methods other than GET, HEAD or POST. Also, if POST is used to send request data with a Content-Type other than application/x-www-form-urlencoded, multipart/form-data, or text/plain, e.g. if the POST request sends an XML payload to the server using application/xml or text/xml, then the request is preflighted.
It sets custom headers in the request (e.g. the request uses a header such as X-PINGOTHER)
Your fiddle tries to load a resource from another domain via ajax:
I think I created a better example. Here is the code:
<img src="smiley.png" alt="smiley" />
<div id="respText"></div>
<script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<script>
$(window).load(function(){
$.get("smiley.png", function(){
$("#respText").text("ajax request succeeded");
});
});
</script>
You can test the page here.
According to Firebug and the chrome network panel the image is returned with the status code 200 and the image for the ajax request is coming from the cache:
Firefox:
Chrome:
So I cannot find any unexpected behavior.
Cache control on Ajax requests have always been a blurred and buggy subject (example).
The problem gets even worse with cross-domain references.
The fiddle link you provided is from jsfiddle.net which is an alias for fiddle.jshell.net. Every code runs inside the fiddle.jshell.net domain, but your code is referencing an image from the alias and browsers will consider it a cross-domain access.
To fix it, you could change both urls to http://fiddle.jshell.net/img/logo-white.png or just /img/logo-white.png.
The helpful folks at Mozilla gave some details as to why this happens. Apparently Firefox assumes an "anonymous" request could be different than normal, and for this reason it makes a second request and doesn't consider the cached value with different headers to be the same request.
https://bugzilla.mozilla.org/show_bug.cgi?id=1075297
This may be a shot in the dark, but here's what I think is happening.
According to,
http://api.jquery.com/jQuery.get/
dataType
Type: String
The type of data expected from the server.
Default: Intelligent Guess (xml, json, script, or html).
Gives you 4 possible return types. There is no datatype of image/gif being returned. Thus, the browser doesn't test it's cache for the src document as it is being delivered a a different mime type.
The server decides what can be cached and for how long. However, it again depends on the browser, whether or not to follow it. Most web browsers like Chrome, Firefox, Safari, Opera and IE follow it, though.
The point that I want to make here, is that your web sever might be configured to not allow your browser to cache the content, thus, when you request the image through CSS and JS, the browser follows your server's orders and doesn't cache it and thus it requests the image twice...
I want JS-accessible image
Have you tried to CSS using jQuery? It is pretty fun - you have full CRUD (Create, read, update, delete) CSS elements. For example do image resize on server side:
$('#container').css('background', 'url(somepage.php?src=image_source.jpg'
+ '&w=' + $("#container").width()
+ '&h=' + $("#container").height() + '&zc=1');
Surprisingly, I found that even when the javascript waits for the entire page to load, the image request still makes a new request! Is this a known bug in Firefox and Chrome, or something bad jQuery ajax is doing?
It is blatantly obvious that this is not a browser bug.
The computer is deterministic and does what exactly you tell it to (not want you want it to do). If you want to cache images it is done in server side. Based on who handles caching it can be handled as:
Server (like IIS or Apache) cache - typically caches things that are reused often (ex: 2ce in 5 seconds)
Server side application cache - typically it reuses server custom cache or you create sprite images or ...
Browser cache - Server side adds cache headers to images and browsers maintain cache
If it is not clear then I would like to make it clear : You don't cache images with javascript.
Ideally I would hope the ajax wouldn't make a second request at all, since it is requesting exactly the same url.
What you try to do is to preload images.
Once an image has been loaded in any way into the browser, it will be
in the browser cache and will load much faster the next time it is
used whether that use is in the current page or in any other page as
long as the image is used before it expires from the browser cache.
So, to precache images, all you have to do is load them into the
browser. If you want to precache a bunch of images, it's probably best
to do it with javascript as it generally won't hold up the page load
when done from javascript. You can do that like this:
function preloadImages(array) {
if (!preloadImages.list) {
preloadImages.list = [];
}
for (var i = 0; i < array.length; i++) {
var img = new Image();
img.onload = function() {
var index = preloadImages.list.indexOf(this);
if (index !== -1) {
// remove this one from the array once it's loaded
// for memory consumption reasons
preloadImages.splice(index, 1);
}
}
preloadImages.list.push(img);
img.src = array[i];
}
}
preloadImages(["url1.jpg", "url2.jpg", "url3.jpg"]);
Then, once they've been preloaded like this via javascript, the browser will have them in its cache and you can just refer to the normal URLs in other places (in your web pages) and the browser will fetch that URL from its cache rather than over the network.
Source : How do you cache an image in Javascript
Is there a reason css and ajax in this case usually have different caches, as though the browser is using different cache storage for css vs ajax requests?
Even in absence of information do not jump to conclusions!
One big reason to use image preloading is if you want to use an image
for the background-image of an element on a mouseOver or :hover event.
If you only apply that background-image in the CSS for the :hover
state, that image will not load until the first :hover event and thus
there will be a short annoying delay between the mouse going over that
area and the image actually showing up.
Technique #1 Load the image on the element's regular state, only shift it away with background position. Then move the background
position to display it on hover.
#grass { background: url(images/grass.png) no-repeat -9999px -9999px; }
#grass:hover { background-position: bottom left; }
Technique #2 If the element in question already has a background-image applied and you need to change that image, the above
won't work. Typically you would go for a sprite here (a combined
background image) and just shift the background position. But if that
isn't possible, try this. Apply the background image to another page
element that is already in use, but doesn't have a background image.
#random-unsuspecting-element {
background: url(images/grass.png) no-repeat -9999px -9999px; }
#grass:hover { background: url(images/grass.png) no-repeat; }
The idea create new page elements to use for this preloading technique
may pop into your head, like #preload-001, #preload-002, but that's
rather against the spirit of web standards. Hence the using of page
elements that already exist on your page.
The browser will make the 2 requests on the page, cause an image called from the css uses a get request (not ajax) too before rendering the entire page.
The window load is similar to de attribute, and is loading before the rest of the page, then, the image from the Ajax will be requested first than the image on the div, processed during the page load.
If u would like to load a image after the entire page is loaded, u should use the document.ready() instead
I am trying to learning the browser's (Chrome/Firefox) cache mechanism.
I set up a simple HTML:
<HTML><BODY>
Hellow World
<script>
function loadJS(){
var s = document.createElement('script');
s.setAttribute('src','/myscript');
document.body.appendChild(s);
}
loadJS()
</script>
<BODY></HTML>
I output "Cache-Control: max-age:30" for "/myscript"
Everytime I press F5, browser will re-validate /myscript with my server to get a 304 back.
But if I use
setTimeout(loadJS, 1);
Everytime I press F5, it looks like browser will check expire time, and if not expired, browser will use the cache directly instead of going to server for revalidation.
My question is:
Why? is there a detail explanation for this?
Does it mean if I want browser to use cache and reduce network request as much as possible, I need to wait the page loaded, and then request resources by js later?
I've done a fair amount of experimentation with browser cache control, and I am surprised that no one has posted an answer.
Many people do not pay attention to this. As a results websites--for no reason at all--make browsers perform useless roundtrips for a 304-not modified on images, js or css files which are unlikely changed in 5 years--like who is going to change jquery.v-whatever?
So anyway, I have found that when you hard refresh the browser using F5 or ctrl-r, Chrome will revalidate just about everything on the page--as it should. This is very helpful and is why you want keep the etags in the response header.
When testing your max-age and expires headers, browse the site as a user naturally would by clicking the links on the page. Watch the web server's logfile (I use http://www.apacheviewer.com) and you'll get a good idea of how the browsers are caching.
Setting the headers works. I posted this a while back: Apache: set max-age or expires in .htaccess for directory
The easiest way for me to manage the web server is to create a /cache directory and instruct apache to set a 1 year max-age and expires header for everything in every subdir.
It works wonders. My pages make 1 round trip to the server, where as they used to make 3-5 trips with each request, just to get a 304.
Write your html as you normally would. The browsers will obey the cache settings in the headers.
Just know that hard refreshing the browser causes the browser to ignore max-age and relies on etags.
I work on an internal corporate system that has a web front-end using Tomcat.
How can I monitor the rendering time of specific pages in a browser (IE6)?
I would like to be able to record the results in a log file (separate log file or the Tomcat access log).
EDIT: Ideally, I need to monitor the rendering on the clients accessing the pages.
The Navigation Timing API is available in modern browsers (IE9+) except Safari:
function onLoad() {
var now = new Date().getTime();
var page_load_time = now - performance.timing.navigationStart;
console.log("User-perceived page loading time: " + page_load_time);
}
In case a browser has JavaScript enabled one of the things you could do is to write an inline script and send it first thing in your HTML. The script would do two things:
Record current system time in a JS variable (if you're lucky the time could roughly correspond to the page rendering start time).
Attach JS function to the page onLoad event. This function will then query the current system time once again, subtract the start time from step 1 and send it to the server along with the page location (or some unique ID you could insert into the inline script dynamically on your server).
<script language="JavaScript">
var renderStart = new Date().getTime();
window.onload=function() {
var elapsed = new Date().getTime()-renderStart;
// send the info to the server
alert('Rendered in ' + elapsed + 'ms');
}
</script>
... usual HTML starts here ...
You'd need to make sure that the page doesn’t override onload later in the code, but adds to the event handlers list instead.
As far as non-invasive techniques are concerned, Hammerhead measures complete load time (including JavaScript execution), albeit in Firefox only.
I've seen usable results when a JavaScript snippet could be added globally to measure the start and end of each page load operation.
Have a look at Selenium - they offer a remote control that can automatically start different browsers (e.g. IE6), load pages, test for specific content on the page. At the end reports are generated that also show the rendering times.
Since others are posting answers that use other browsers, I guess I will too. Chrome has a very detailed profiling system that breaks down the rendering time of the page and shows the time it took for each step along the way.
As for IE, you might want to consider writing a plugin. There seems to be few tools like this on the market. Maybe you could sell it.
On Firefox you can use Firebug to monitor load time. With the YSlow plugin you can even get recommendations how to improve the performance.