I'm using the NavigationTiming object to monitor performance of my website.
According to the W3C document on the redirectStart property of the NavigationTiming object:
If there are HTTP redirects or equivalent when navigating and if all the redirects or equivalent are from the same origin, this attribute must return the starting time of the fetch that initiates the redirect.
My site currently has a login page, which submits POST and receives a 302 redirect to a welcome page. I expected the NavigationTiming object on the welcome page to include the redirectStart and redirectEnd properties to be populated, but they are 0.
When should they be populated, if not then?
Are the login page and welcome pages different origins, for example, different schemes (HTTP/HTTPS) or servers (foo.com vs www.foo.com)?
NavigationTiming will zero-out the redirectStart and redirectEnd attributes if the origin differs between the two addresses in any way.
(Note the POST method is not part of the origin calculation and thus should not cause the property to zero-out just because it's a POST 302 to a GET)
If you care to share your site I could investigate further.
You can also verify your browser properly supports the redirect case by running the W3C test cases here.
Related
I have the following script at http://localhost/test.html:
<script>
alert(document.referrer);
</script>
If I access it directly the result is an empty alert, which isn't surprising.
If I link from a different document at http://example.com/different.html, the alert will be that URL, again, not surprising.
What is suprising to me is that, if I intercept the HTTP request and change the Referer Header:
GET /test.html HTTP/1.1
Host: localhost
Referer: test
Then the alert will still alert the original URL, not test.
So where does document.referrer come from if not from the referer HTTP Header? Is it not influenced by the HTTP request at all? Is there a standard for this, or do different browsers handle it differently? And is there a way to influence it, without creating a new file linking to the code myself?
Referrer header your have intercepted is the request done by the client to the server. The client already know which is the referring page, you cannot fool it.
Per MDN documentation:
document.referrer:
Returns the URI of the page that linked to this page.
Further notes on why it displays empty to you:
The value is an empty string if the user navigated to the page
directly (not through a link, but, for example, via a bookmark). Since
this property returns only a string, it does not give you DOM access
to the referring page.
More info can be found at: MDN
Now looking at the developer tools from both Chrome, Firefox and IE I can see the header is being set to: Referer:https://www.google.com/ whenever I hit a search result from google and this value is being set automatically by the browser. How it's set depends on browser implementor but this is the corresponding document describing the header value RFC 7231
The "Referer" [sic] header field allows the user agent to specify a
URI reference for the resource from which the target URI was obtained
(i.e., the "referrer", though the field name is misspelled).
The value is set by the browser, I mean the browser is setting the value "test" when you are doing the http request.
I want to capture the HTTP request header fields, primarily the Referer and User-Agent, within my client-side JavaScript. How may I access them?
Google Analytics manages to get the data via JavaScript that they have you embed in you pages, so it is definitely possible.
Related:
Accessing the web page's HTTP Headers in JavaScript
If you want to access referrer and user-agent, those are available to client-side Javascript, but not by accessing the headers directly.
To retrieve the referrer, use document.referrer.
To access the user-agent, use navigator.userAgent.
As others have indicated, the HTTP headers are not available, but you specifically asked about the referer and user-agent, which are available via Javascript.
Almost by definition, the client-side JavaScript is not at the receiving end of a http request, so it has no headers to read. Most commonly, your JavaScript is the result of an http response. If you are trying to get the values of the http request that generated your response, you'll have to write server side code to embed those values in the JavaScript you produce.
It gets a little tricky to have server-side code generate client side code, so be sure that is what you need. For instance, if you want the User-agent information, you might find it sufficient to get the various values that JavaScript provides for browser detection. Start with navigator.appName and navigator.appVersion.
This can be accessed through Javascript because it's a property of the loaded document, not of its parent.
Here's a quick example:
<script type="text/javascript">
document.write(document.referrer);
</script>
The same thing in PHP would be:
<?php echo $_SERVER["HTTP_REFERER"]; ?>
Referer and user-agent are request header, not response header.
That means they are sent by browser, or your ajax call (which you can modify the value), and they are decided before you get HTTP response.
So basically you are not asking for a HTTP header, but a browser setting.
The value you get from document.referer and navigator.userAgent may not be the actual header, but a setting of browser.
One way to obtain the headers from JavaScript is using the WebRequest API, which allows us to access the different events that originate from http or websockets, the life cycle that follows is this:
WebRequest Lifecycle
So in order to access the headers of a page it would be like this:
browser.webRequest.onHeadersReceived.addListener(
(headersDetails)=> {
console.log("Request: " + headersDetails);
},
{urls: ["*://hostName/*"]}
);`
The issue is that in order to use this API, it must be executed from the browser, that is, the browser object refers to the browser itself (tabs, icons, configuration), and the browser does have access to all the Request and Reponse of any page , so you will have to ask the user for permissions to be able to do this (The permissions will have to be declared in the manifest for the browser to execute them)
And also being part of the browser you lose control over the pages, that is, you can no longer manipulate the DOM, (not directly) so to control the DOM again it would be done as follows:
browser.webRequest.onHeadersReceived.addListener(
browser.tabs.executeScript({
code: 'console.log("Headers success")',
});
});
or if you want to run a lot of code
browser.webRequest.onHeadersReceived.addListener(
browser.tabs.executeScript({
file: './headersReveiced.js',
});
});
Also by having control over the browser we can inject CSS and images
Documentation: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onHeadersReceived
I would imagine Google grabs some data server-side - remember, when a page loads into your browser that has Google Analytics code within it, your browser makes a request to Google's servers; Google can obtain data in that way as well as through the JavaScript embedded in the page.
var ref = Request.ServerVariables("HTTP_REFERER");
Type within the quotes any other server variable name you want.
I want to dynamically fetch a Wikipedia webpage in the browser in order to further process the XHTML with XSLTProcessor.
Unfortunately, this does not work because I can't get Wikipedia to send the "Access-Control-Allow-Origin" header in the HTTP response.
I tried to include the "origin" parameter as it is stated on https://www.mediawiki.org/wiki/Manual:CORS, but without success.
It is important to me to obtain the complete web page HTML as it is obtained by the browser when navigating to that page, so the MediaWiki API is out of the question for me.
This is what I have tried:
var url = "https://en.wikipedia.org/wiki/Star_Trek?origin=https://my-own-page.com";
fetch(url).then(function(response){
console.log(response);
});
Unfortunately, this does not work because I can't get Wikipedia to send the "Access-Control-Allow-Origin" header in the HTTP response.
No, you can't. It is up to Wikipedia to decide if they want to explicitly grant permission to JavaScript running on other sites access to their pages.
Since this would allow users' personal information to leak (e.g. logged in Wikipedia pages display the user's username, which could be used to enhance a phishing attack), this is clearly something undesirable.
var url = "https://en.wikipedia.org/wiki/Star_Trek?origin=https://my-own-page.com";
origin is an HTTP request header, not a query string parameter, and is automatically included in cross origin XMLHttpRequest/fetch requests without you needing to do anything special.
I want to capture the HTTP request header fields, primarily the Referer and User-Agent, within my client-side JavaScript. How may I access them?
Google Analytics manages to get the data via JavaScript that they have you embed in you pages, so it is definitely possible.
Related:
Accessing the web page's HTTP Headers in JavaScript
If you want to access referrer and user-agent, those are available to client-side Javascript, but not by accessing the headers directly.
To retrieve the referrer, use document.referrer.
To access the user-agent, use navigator.userAgent.
As others have indicated, the HTTP headers are not available, but you specifically asked about the referer and user-agent, which are available via Javascript.
Almost by definition, the client-side JavaScript is not at the receiving end of a http request, so it has no headers to read. Most commonly, your JavaScript is the result of an http response. If you are trying to get the values of the http request that generated your response, you'll have to write server side code to embed those values in the JavaScript you produce.
It gets a little tricky to have server-side code generate client side code, so be sure that is what you need. For instance, if you want the User-agent information, you might find it sufficient to get the various values that JavaScript provides for browser detection. Start with navigator.appName and navigator.appVersion.
This can be accessed through Javascript because it's a property of the loaded document, not of its parent.
Here's a quick example:
<script type="text/javascript">
document.write(document.referrer);
</script>
The same thing in PHP would be:
<?php echo $_SERVER["HTTP_REFERER"]; ?>
Referer and user-agent are request header, not response header.
That means they are sent by browser, or your ajax call (which you can modify the value), and they are decided before you get HTTP response.
So basically you are not asking for a HTTP header, but a browser setting.
The value you get from document.referer and navigator.userAgent may not be the actual header, but a setting of browser.
One way to obtain the headers from JavaScript is using the WebRequest API, which allows us to access the different events that originate from http or websockets, the life cycle that follows is this:
WebRequest Lifecycle
So in order to access the headers of a page it would be like this:
browser.webRequest.onHeadersReceived.addListener(
(headersDetails)=> {
console.log("Request: " + headersDetails);
},
{urls: ["*://hostName/*"]}
);`
The issue is that in order to use this API, it must be executed from the browser, that is, the browser object refers to the browser itself (tabs, icons, configuration), and the browser does have access to all the Request and Reponse of any page , so you will have to ask the user for permissions to be able to do this (The permissions will have to be declared in the manifest for the browser to execute them)
And also being part of the browser you lose control over the pages, that is, you can no longer manipulate the DOM, (not directly) so to control the DOM again it would be done as follows:
browser.webRequest.onHeadersReceived.addListener(
browser.tabs.executeScript({
code: 'console.log("Headers success")',
});
});
or if you want to run a lot of code
browser.webRequest.onHeadersReceived.addListener(
browser.tabs.executeScript({
file: './headersReveiced.js',
});
});
Also by having control over the browser we can inject CSS and images
Documentation: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onHeadersReceived
I would imagine Google grabs some data server-side - remember, when a page loads into your browser that has Google Analytics code within it, your browser makes a request to Google's servers; Google can obtain data in that way as well as through the JavaScript embedded in the page.
var ref = Request.ServerVariables("HTTP_REFERER");
Type within the quotes any other server variable name you want.
I'd like to offer a way to my users to promote my website, blog etc. on their website.
I can make a banner, logo whatever that they can embed to their site, but I'd like to offer dynamic content, like "the 5 newest entry's title from my blog".
The problem is the same origin policy. I know there is a solution (and I use it): they embed a simple div and a JavaScript file. The JS makes an XmlHttpRequest to my server and gets the data as JSONP, parses the data and inserts into the div.
But is it the only way? Isn't there a better way I could do this?
On the Internet there are tons of widget (or whatever, I don't know how they call...) that gain the data from another domain. How they do that?
A common theme of many of the solutions, instead, is getting JavaScript to call a proxy program (either on the client or the server) which, in turn, calls the web service for you.
The output can be written to the response stream and then is available, via the normal channels, such as the responseText and responseXML properties of XMLHttpRequest.
you can find more solution here :
http://developer.yahoo.com/javascript/howto-proxy.html
or here :
http://www.simple-talk.com/dotnet/asp.net/calling-cross-domain-web-services-in-ajax/
CORS is a different way than JSONP.
Plain AJAX. All your server has to do is to set a specific header: Access-Control-Allow-Origin
More here: http://hacks.mozilla.org/2009/07/cross-site-xmlhttprequest-with-cors/
If you go the JSONP route, you will implicitly ask your users to trust you, as they will give you full access to the resources of their page (content, cookies,...). If they know that they main complain.
While if you go the iframe route there is no problems.One famous example today of embeddable content by iframe is the Like button of facebook.
And making that server side with a proxy or other methods would be much more complex, as there are plenty of environments out there. I don't know other ways.
You can also set the HTTP Access-Control headers in the server side. This way you're basically controlling from the server side on whether the client who has fired the XMLHttpRequest is allowed to process the response. Any recent (and decent) webbrowser will take action accordingly.
Here's a PHP-targeted example how to set the headers accordingly.
header('Access-Control-Allow-Origin: *'); // Everone may process the response.
header('Access-Control-Max-Age: 604800'); // Client may cache this for one week.
header('Access-Control-Allow-Methods: GET, POST'); // Allowed request methods.
The key is Access-Control-Allow-Origin: *. This informs the client that requests originating from * (in fact, everywhere) is allowed to process the response. If you set it to for example Access-Control-Allow-Origin: http://example.com, then the webbrowser may only process the response when the initial page is been served from the mentioned domain.
See also:
MDC - HTTP Access Control