I'm not looking for code/how to. Just knowledge.
A client has just come to us with a question: Can we access the user's history from within a banner advert to give them some targeted advertising based on their history.
Obviously, this presents a privacy issue, but I need to give a good case for why it is technically not a viable option.
So I have a few questions...
Which browsers still, if any, support accessing a user's history, using window.history.
If some do and some don't. When did those who don't allow it stop allowing it?
If all browsers allow it (I have yet to find a script that works), why is it not commonly used?
Finally,
Having been on Amazon.co.uk, I then go to Macrumors.com and the adverts give me adverts based on products I have bought/looked at. I'm guessing this is just based on cookies/a system that amazon has implemented?
Just to make things clear:
I know it is a privacy issue. I am not looking for code/a way to do it (as I mentioned above)
There are ways to "sniff" for visited links within a page.
There used to be a way using the JavaScript history object, to list all the objects within your history (from the current site). history.length still works now. I seem to remember some browsers only returning undefined for each item, some returning them as an unreadable object.
No!
There's no browser (that I know of) that legitimately give you access to a user's browsing history.
There has been incidents where it was possible to do so by exploiting certain behaviors of the browser. Recently, in Firefox 16 there's a vulnerability that, if exploited properly, allows you to peek into the user's browsing history.
In the case you're describing (Amazon), yes, cookies are used. To be more accurate, Third-Party Cookies are used.
Update:
I was very interested in your last edit (about history being completely open in the past), so I tried to go back a little.
Related
I am about to begin work on an information and support services website for victims of domestic and sexual abuse. Naturally, the user's visit to this website is sensitive and the repercussions of their abuser finding out that they have could be devastating. Therefore, I am seeking a way to keep the user's visit as discreet as possible.
I cannot assume the technical knowledge of each user and there are likely to be users of a wide variety of languages. Also, there is the possibility they will need to exit the site quickly (I have the solution for this) and perhaps may not be able to return to the computer before being discovered. So, while the most obvious solution is to have a page educating users on how to clear their browsing history - this may not be the most foolproof method in practice. Because of all the variables in play, a blanket solution would be the best solution.
So far, I can think of two solutions to this but am hitting a wall with both:
Firstly, simply not have the website recorded in the browsers' search history. From what I have read this is going to be problematic between browsers if not impossible to implement.
The second would be to have a landing page at an innocuous domain name that wouldn't draw suspicion and then have a button that automatically loaded the website through a Private or Incognito browser (I could simply write instructions, 'Right Click on the Button and Select 'Open in an Incognito Browser' - but I am searching for a more foolproof solution if possible).
While some incarnation of the second solution seems more plausible - I need to consider that abusers searching through browser history is a possibility and therefore, the first solution is the most desirable.
Any ideas on either of these two methods or anything more ideas you have would be most welcomed.
You cannot modify the browser history using JavaScript.
You can alter the stack of recently visited pages, but it only affect the previous/next page buttons.
Best thing I can think of : advise people visiting the website to use private browsing mode and clean the history themselves.
This is probably not a technical question but more related to legislation:
In these years European countries are due adjust with eu cookies law (synthetically every web site must inform/asks agreement/and so on to their users about every kind of cookies used by the sites).
Recently we spent some time to analyze possible implementations about informer banner/pop up. Our doubt concerns javascript implementation, for example as suggested here https://www.cookiechoices.org/ with Google collaboration.
These implementations - pure Javascript - should work only if a browser enables javascript, so what happens if a browser disables javascript? Would this web site outlaw?
These implementations - pure Javascript - should work only if a browser enables javascript, so what happens if a browser disables javascript? Would this web site outlaw?
The mechanism used to show agreement is quite simple, explained on the first page and can be implemented by serverside code just as well as in Javascript:
With each tool, you can customise a notice or statement directed to your visitors and invite your visitors to close the message (using wording of your choice such as "close", "accept" or "I agree"). Users also have the option of clicking a link to another page, such as a page that explains how your site uses cookies (again, you choose the wording for this link text). If users close the message, a cookie will be set to your domain. It is called 'displayCookieConsent' and has the value 'y'.
Simply put an iframe in the div to communicate cookie preferences serverside.
Although your question specifically relates to the tools available from the cookiechoice website, you are no doubt aware that article 5(3) of the ePrivacy directive has much wider scope and conditional context that may negate the need for this kind of consent, or indeed, depending on your application, require the capability to allow and deal with the scenario where the user explicitly explicitly refuses cookies. Hence whether your site would be illegal (which also brings the question of the implementations of the directive in different jurisdictions) is highly dependent on what your site does and how it uses the cookies. (e.g. see this discussion of exemptions)
Is there any way to consistently detect PhantomJS/CasperJS? I've been dealing with a spat of malicious spambots built with it and have been able to mostly block them based on certain behaviours, but I'm curious if there's a rock-solid way to know if CasperJS is in use, as dealing with constant adaptations gets slightly annoying.
I don't believe in using Captchas. They are a negative user experience and ReCaptcha has never worked to block spam on my MediaWiki installations. As our site has no user registrations (anonymous discussion board), we'd need to have a Captcha entry for every post. We get several thousand legitimate posts a day and a Captcha would see that number divebomb.
I very much share your take on CAPTCHA. I'll list what I have been able to detect so far, for my own detection script, with similar goals. It's only partial, as they are many more headless browsers.
Fairly safe to use exposed window properties to detect/assume those particular headless browser:
window._phantom (or window.callPhantom) //phantomjs
window.__phantomas //PhantomJS-based web perf metrics + monitoring tool
window.Buffer //nodejs
window.emit //couchjs
window.spawn //rhino
The above is gathered from jslint doc and testing with phantom js.
Browser automation drivers (used by BrowserStack or other web capture services for snapshot):
window.webdriver //selenium
window.domAutomation (or window.domAutomationController) //chromium based automation driver
The properties are not always exposed and I am looking into other more robust ways to detect such bots, which I'll probably release as full blown script when done. But that mainly answers your question.
Here is another fairly sound method to detect JS capable headless browsers more broadly:
if (window.outerWidth === 0 && window.outerHeight === 0){ //headless browser }
This should work well because the properties are 0 by default even if a virtual viewport size is set by headless browsers, and by default it can't report a size of a browser window that doesn't exist. In particular, Phantom JS doesn't support outerWith or outerHeight.
ADDENDUM: There is however a Chrome/Blink bug with outer/innerDimensions. Chromium does not report those dimensions when a page loads in a hidden tab, such as when restored from previous session. Safari doesn't seem to have that issue..
Update: Turns out iOS Safari 8+ has a bug with outerWidth & outerHeight at 0, and a Sailfish webview can too. So while it's a signal, it can't be used alone without being mindful of these bugs. Hence, warning: Please don't use this raw snippet unless you really know what you are doing.
PS: If you know of other headless browser properties not listed here, please share in comments.
There is no rock-solid way: PhantomJS, and Selenium, are just software being used to control browser software, instead of a user controlling it.
With PhantomJS 1.x, in particular, I believe there is some JavaScript you can use to crash the browser that exploits a bug in the version of WebKit being used (it is equivalent to Chrome 13, so very few genuine users should be affected). (I remember this being mentioned on the Phantom mailing list a few months back, but I don't know if the exact JS to use was described.) More generally you could use a combination of user-agent matching up with feature detection. E.g. if a browser claims to be "Chrome 23" but does not have a feature that Chrome 23 has (and that Chrome 13 did not have), then get suspicious.
As a user, I hate CAPTCHAs too. But they are quite effective in that they increase the cost for the spammer: he has to write more software or hire humans to read them. (That is why I think easy CAPTCHAs are good enough: the ones that annoy users are those where you have no idea what it says and have to keep pressing reload to get something you recognize.)
One approach (which I believe Google uses) is to show the CAPTCHA conditionally. E.g. users who are logged-in never get shown it. Users who have already done one post this session are not shown it again. Users from IP addresses in a whitelist (which could be built from previous legitimate posts) are not shown them. Or conversely just show them to users from a blacklist of IP ranges.
I know none of those approaches are perfect, sorry.
You could detect phantom on the client-side by checking window.callPhantom property. The minimal script is on the client side is:
var isPhantom = !!window.callPhantom;
Here is a gist with proof of concept that this works.
A spammer could try to delete this property with page.evaluate and then it depends on who is faster. After you tried the detection you do a reload with the post form and a CAPTCHA or not depending on your detection result.
The problem is that you incur a redirect that might annoy your users. This will be necessary with every detection technique on the client. Which can be subverted and changed with onResourceRequested.
Generally, I don't think that this is possible, because you can only detect on the client and send the result to the server. Adding the CAPTCHA combined with the detection step with only one page load does not really add anything as it could be removed just as easily with phantomjs/casperjs. Defense based on user agent also doesn't make sense since it can be easily changed in phantomjs/casperjs.
For security reasons:
How to prevent a webpage link being stored on the history?
Additionally, how to prevent most of caching from the given link?
While I don't have a specific answer for you, I think you might find this article quite helpful: http://css-tricks.com/website-escape/ The use case described there is a site for helping victims of domestic abuse. Looking in the comments, you'll see many tips that others have found for hiding traces of visits to the site, etc. In terms of preventing a site from entering browser history, I don't know of a good way to do that, but perhaps the best choice is user education about various privacy modes that the browsers themselves offer (i.e. Chrome's private browsing or the similar feature in Safari).
For a password protected site, is there a way to allow users to log in to the site multiple times as long as it's on the same computer (even in different browsers)?
One way I have thought to do it is always send MAC address on login, but that's not available in browsers. Is there some other way of doing this?
As I understand it, you want to keep your users logged in to your application even if they close the browser and switch to another one. Briefly looking into this, a possible solution might be to use "flash cookies" or local shared objects, which are managed browser independent. However, there are some security, privacy and legal concerns plus the drawbacks from using the proprietary Flash technology. I would go with Alex's advice of rethinking your approach.
Use a flash (or java, or ...) applet to get the needed unique information.