Is there any way to consistently detect PhantomJS/CasperJS? I've been dealing with a spat of malicious spambots built with it and have been able to mostly block them based on certain behaviours, but I'm curious if there's a rock-solid way to know if CasperJS is in use, as dealing with constant adaptations gets slightly annoying.
I don't believe in using Captchas. They are a negative user experience and ReCaptcha has never worked to block spam on my MediaWiki installations. As our site has no user registrations (anonymous discussion board), we'd need to have a Captcha entry for every post. We get several thousand legitimate posts a day and a Captcha would see that number divebomb.
I very much share your take on CAPTCHA. I'll list what I have been able to detect so far, for my own detection script, with similar goals. It's only partial, as they are many more headless browsers.
Fairly safe to use exposed window properties to detect/assume those particular headless browser:
window._phantom (or window.callPhantom) //phantomjs
window.__phantomas //PhantomJS-based web perf metrics + monitoring tool
window.Buffer //nodejs
window.emit //couchjs
window.spawn //rhino
The above is gathered from jslint doc and testing with phantom js.
Browser automation drivers (used by BrowserStack or other web capture services for snapshot):
window.webdriver //selenium
window.domAutomation (or window.domAutomationController) //chromium based automation driver
The properties are not always exposed and I am looking into other more robust ways to detect such bots, which I'll probably release as full blown script when done. But that mainly answers your question.
Here is another fairly sound method to detect JS capable headless browsers more broadly:
if (window.outerWidth === 0 && window.outerHeight === 0){ //headless browser }
This should work well because the properties are 0 by default even if a virtual viewport size is set by headless browsers, and by default it can't report a size of a browser window that doesn't exist. In particular, Phantom JS doesn't support outerWith or outerHeight.
ADDENDUM: There is however a Chrome/Blink bug with outer/innerDimensions. Chromium does not report those dimensions when a page loads in a hidden tab, such as when restored from previous session. Safari doesn't seem to have that issue..
Update: Turns out iOS Safari 8+ has a bug with outerWidth & outerHeight at 0, and a Sailfish webview can too. So while it's a signal, it can't be used alone without being mindful of these bugs. Hence, warning: Please don't use this raw snippet unless you really know what you are doing.
PS: If you know of other headless browser properties not listed here, please share in comments.
There is no rock-solid way: PhantomJS, and Selenium, are just software being used to control browser software, instead of a user controlling it.
With PhantomJS 1.x, in particular, I believe there is some JavaScript you can use to crash the browser that exploits a bug in the version of WebKit being used (it is equivalent to Chrome 13, so very few genuine users should be affected). (I remember this being mentioned on the Phantom mailing list a few months back, but I don't know if the exact JS to use was described.) More generally you could use a combination of user-agent matching up with feature detection. E.g. if a browser claims to be "Chrome 23" but does not have a feature that Chrome 23 has (and that Chrome 13 did not have), then get suspicious.
As a user, I hate CAPTCHAs too. But they are quite effective in that they increase the cost for the spammer: he has to write more software or hire humans to read them. (That is why I think easy CAPTCHAs are good enough: the ones that annoy users are those where you have no idea what it says and have to keep pressing reload to get something you recognize.)
One approach (which I believe Google uses) is to show the CAPTCHA conditionally. E.g. users who are logged-in never get shown it. Users who have already done one post this session are not shown it again. Users from IP addresses in a whitelist (which could be built from previous legitimate posts) are not shown them. Or conversely just show them to users from a blacklist of IP ranges.
I know none of those approaches are perfect, sorry.
You could detect phantom on the client-side by checking window.callPhantom property. The minimal script is on the client side is:
var isPhantom = !!window.callPhantom;
Here is a gist with proof of concept that this works.
A spammer could try to delete this property with page.evaluate and then it depends on who is faster. After you tried the detection you do a reload with the post form and a CAPTCHA or not depending on your detection result.
The problem is that you incur a redirect that might annoy your users. This will be necessary with every detection technique on the client. Which can be subverted and changed with onResourceRequested.
Generally, I don't think that this is possible, because you can only detect on the client and send the result to the server. Adding the CAPTCHA combined with the detection step with only one page load does not really add anything as it could be removed just as easily with phantomjs/casperjs. Defense based on user agent also doesn't make sense since it can be easily changed in phantomjs/casperjs.
I have a few HTML pages that use javascript to run sql queries etc. these html files are accessed from a share drive on the network. this works perfectly in IE6. My workplace is updating to IE8 and I now get the following warning:
"This Web site uses a data provider that may be unsafe. If you trust the Web site, click OK, otherwise click Cancel."
This is very annoying as every sql interaction results in this warning.
I have spent a lot of time researching and the solution seems to be that I need to add the site to my trusted sites list. However, I am unable to do so due to group policy disabling access to IE settings, and my IT department says that they will not alter the trusted sites list.
Are there any other options? If I could somehow get a trusted certificate would that help? I have also researched other ways to try and run sql without javascript and without a web server but I didnt have much luck.
you're getting the cross-domain alert because your accessing data on a different server. you can put them on the same server and that should fix your problem.'
regardless, and i mean this with all sincerity, you need to get up with your it department and have a little heart-to-heart; they're going to be wary of you and most likely dissmissive...do not let them dissuade you. be polite but be just as stubborn. and if you really want to be a dick about it, when they're up on their high horse telling you nay, you should question their expertise...particularly with user-agents. if they're converting from ie6 in 2012....there's no situation they can describe that makes that acceptable. i'm assuming your intranet was built specifically for ie6 and that'll be their main excuse.....dude ie7 came out in 2007. their excuse is five years old. furthermore....it's still weak sauce.
you and all of your coworkers can enjoy chrome if you add chrome frame to your installation. look it up. wow your it department.
lastly....you're a developer that is not allowed to choose browser, and the browser they grant you access to isn't even fully functional? a)Quit b)Quit c)Quit
Albert already talked you through it (and how right he is). Otherwise a solution could be to put a pass through server (somewhere in your network) between the client and database, e.g. using node.js with an odbc library like node-tds or tedious.
I have a client that hosts a touch screen kiosk in their lobby. It's essentially an internal html website. They want customers to be able to click a link and have a pdf print without seeing a print dialog or having to back track... completely unnoticeable printing.
I have read articles and tried snippets of code for almost a year without finding a reputable solution. There are those out there that rightly warn of security breaches or that it cannot be done at all.
The client's old touch screen structure was made entirely in Flash which I am avoiding. BUT, they were able to pull this off. I am sure this question has been asked a thousand times, but is it really impossible?
The current CPU running the touch screen is Windows XP. They do have an IT guy that works close, but I am not sure what to ask. He never offered any other solutions.
Thank you
Not 100% sure what you're trying to accomplish but maybe you could trigger a server-side program to run instead of relying on the kiosk itself to handle the printing. You could configure the server to send these PDF requests straight to the printer.
Maybe you can bake a custom browser for them (using webkit for example) that prints without asking.
Chrome running in kisok mode, started with the --kiosk AND --kiosk-printing switches and with a default printer set, can print silently using javascript print().
This is Windows only AFAIK.
I want to redirect to Internet Explorer from other browsers by JavaScript. How can I do that?
You can't.
In a standard security context, browsers provide no way of launching other programs (and a goodly number of systems don't have Internet Explorer anyway). In non-standard security contexts, most browsers still don't provide a way of launching other programs.
JavaScript does not allow you to execute programs (like Internet Explorer) from another web browser. This isn't possible.
I would recommend to either:
Show an error/warning that the site doesn't work or might not work correctly when not using IE
Fix the site to work cross-browser (recommended)
Short answer is that you can't.
The real reason is security, which will (hopefully) always prevent you from running applications on the user's computer, but even if there weren't for that reason... I'm sorry, I just have to nitpick a bit here. What you're trying to do should be absolutely unacceptable in no uncertain terms to every single user and developer. I think I can safely speak for the overwhelming majority of experienced web developers, that techniques designed to dictate to the user which browser to use are always, always, always wrong, not just technically but morally. Yes, I used the M-word. It's immoral to make those sorts of decisions for your users, no matter how dumb they are. Warn them if your site is designed specifically for a particular browser, that's fine, but don't make the actual decision for them.
Sorry, I know this ain't the place for arguing, but I do believe this needs to be clear because it's a real problem in web development that really does waste time and really does cost money and really does harm the web as a whole.
So as far as I'm concerned, I'm glad these security measures are there, because this is an extraordinarily bad idea in the first place, even without the security risks.
No disrespect intended. It just had to be said.
You can't and you shouldn't!
However, this simple javascript, which only works in IE6, can runs firefox.exe from C:\Program Files\Mozilla Firefox ( YES I KNOW IT CAN BE SOMEWHERE IN E: OR D: .... but as I know the default path of Firefox is C:\Program Files\Mozilla Firefox ... you can complete this script and find the correct path of firefox.exe)
This HELL script shows that IE6 is really insecure. ( of course it shows a warning message )
You should never 'FORCE' visitors using your favorite browser. Never ever use such script, this is just for demonstration purpose.
<html>
<head>
<SCRIPT Language="JScript">
function runFirefox() {
File="c:\\PROGRA~1\\MOZILL~1\\FIREFOX.EXE http://stackoverflow.com";
WSH=new ActiveXObject("WScript.Shell");
WSH.run(File);
}
</SCRIPT>
</head>
<body onLoad="javascript:runFirefox();">
<b>For the best experience we FORCE you using Firefox.</b>
</body>
</html>
You should just develop websites that are viewable in all common browsers ;).
No work-arounds for people using different browser types / version.
When developing websites just make sure you meet the standards and all websites have sheets for the common browsers.
You can't.
As far as I'm concerned, I'd love the other way around: redirect any user with IE6 to Firefox/Chrome/safari :-)
You can. We do it because IE has a problem with how many cookies (and how large cookie storage can be) per domain. Because we have so much in our domain we hit a wall with IE. Also, PeopleSoft does not support browsers besides IE so PS users are forced to use IE while everyone else can get locked out of other apps when they hit the IE cookie limit. Is it ugly? Yes. Horribly. It shows what happens when enough planning isn't put into a process and what happens when one group is allowed to make decisions in a vacuum without talking to other groups about potential issues. But this is a fact of life in any organization and especially smaller (size and thinking) ones where there are big egos and weak leadership. Nonetheless...
It is possible to push out to the desktop something that handles a custom mime type. Therefore, when a response goes back to the desktop of type blah/blahblah this triggers an EXE that gets the response (a target location) and fires up IE. Or milks a cow. Whatever you want. You could even use this to tell a PC to take a picture...lots of possibilities.
you can do the following:
function openInChrome(url) {
var ws = new ActiveXObject("WScript.Shell");
ws.Exec("c:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe " + url);
}
I'm currently building a project and I would like to make use of some simple javascript - I know some people have it disabled to prevent XSS and other things. Should I...
a) Use the simple javascript, those users with it disabled are missing out
b) Don't use the simple javascript, users with it enabled have to click a little more
c) Code both javascript-enabled and javascript-disabled functionality
I'm not really sure as the web is always changing, what do you recommend?
Degrade gracefully - make sure the site works without JavaScript, then add bells and whistles for those with JavaScript enabled.
Everyone else has committed good comments, but there are a few other considerations to make.
Sometimes the javascript will be hosted on a different domain, and be prone to timeout.
Sometimes that domain may become inacessible, while your site remains accessible. Its not good to have your site completely stack itself in this scenario.
For this reason, "blocking" scripts ( ie: document write inline ) like that present in google's tracker, should be avoided, or at very least, should go as late in the page as possible so the page renders whether or not the domain is timing out requests or not.
If you happen to be serving JS from a broken/malicious server, by intent or by accident, one can halt page rendering simply by having a script that serves that javascript which just calls "sleep(forever)" once its sent all the headers.
Some People Use NoScript
Like the above problem, sometimes the clients environment may block certain script sources, be it the users choosing, or other reasons ( ie: browser security satisfactions, odd antivirus/anti-malware apps ). The most popular and controllable instance of this is NoScript, and I myself paranoidly block some of the popular tracking/advertising services with it ( some proxy servers will do this too ).
However, if a site is not well designed, the failing of one script to load still executes code that was dependant on that script being present, which yeilds errors and stops everything working.
My recommendation is :
Use Firebug
Use NoScript and block out everything --> See Site still works
Enable core site scripts that you cant' do without for anything --> See site still works and firebug doesn't whine.
Enable 3rd party stuff --> See site still works and firebug doesn't whine.
There are a lot of other complications that can crop up, but satisfying the above 2 should solve most of them. Just assume that, for whatever reason, one or more resources that comprise a page are viable to spontaneously disappear ( they do, all the time ), and you want the page to "survive" this problem as amicably as possible. For the problems that may persist for < 10 seconds, its not so bad, refresh the page and its fixed, but if its a problem that can occur, and severley hamper usability for an hour or more at a time.
In essence, instead of thinking "oh, theres the edge case users that don't have javascript", try thinking more a long the lines of "its really easy to have something go wrong, and have ALL of our users with broken javascript. Ouch! Lets try make it so we dont' really hose ourself when that does happen"
( I've seen IE updates get rolled out and hose javascript for that entire browser until the people whom wrote the scripts find a workaround. Losing all your IE customers is not a good thing )
:set sarcasm
:set ignoreSpelling
:set iq=76
Don't worry, its only a 5% Niché Market
Nobody cares about targeting Niché markets right? All those funny propeller heads running lynx in their geeky stupid linoox cpus, spending all their time on the intarwebs surfing because they have nothing better to do with their life or money? the crazy security paranoid nerds disabling javascript left and right because they don't like it?
Nobody wants them as your primary customer now do they?
Niché markets. Pfft. Who cares!
:set nosarcasm
Consider your audience
"Degrade gracefully" is generally the best answer. But lots of sites now depend on JS - especially AJAX.
Consider your audience. If your site is aimed at extremely tech-savvy people, the chances of them not having javascript are small, and you can notify them to turn it on if necessary.
If your audience may access your site with mobile devices, don't assume they have JavaScript, and don't even assume they support CSS properly. Aim to degrade gracefully all the way down to bare HTML.
I've learned a lot from my question: What's With Those Do-Not-Use Javascript People
Go with Ajax and Web 2.0. It's the way the web is going and it's wonderful. Isn't Stackoverflow great to be on? It's not quite as nice with your Javascript turned off.
Once you have your site ready, but before you let it go live, test it with Javascript off, and just add whatever you feel you need to make your site appear and function to them. You only need to add what you feel is essential.
Remember, except for visually impared people using screen readers, the others have chosen to turn javascript off. They can also choose to trust your site and turn javascript on for your site if they want to use all the functionality you have. It really is their choice.
As other have said, it should "degrade gracefully".
In other works, it must work without Javascript (period). It doesn't have to work well. The folks who've disabled Javascript know the limitations that causes and have accepted them. But if you are trying to sell them something, it's important that they can still buy it.
On the site I'm designing, there's a javascript-based fly-out menu. With Javascript off, all the flyouts are always open. It doesn't look as cool as it would with JS, but it can still be used to navigate the site.
It depends on how much time you have to develop and maintain both solutions, and how much the non-javascript users are worth to you.
My e-commerce site relies heavily on javascript, and in over a year and a half, I've not received a single complaint.
In fact, I don't think I've seen a single visitor with javascript disabled in any of logs since I started.
That doesn't mean they're not out there. It just means that either (a) they're a tiny percentage, (b) they're not interested in what I'm selling, or (c) both of the above.
Code your web site with support for the bare minimum kind of browser. Then more people can use your site without frustration even if they don't have all the bells and whistles--like Flash, Javascript, and Java--enabled. It may not be practical to continue support for ancient browsers, say Netscape Navigator 4, because a user can be reasonably expected to keep their computer up-to-date. However, features like Javascript, Flash, and Java can be security holes in old or modern browsers, as well as being an annoyance.
Neither of my parents keep Javascript or Flash enabled because they've had too many experiences with them slowing down their already slow connection, crashing their browsers, or being more of an annoyance on sites that use it stupidly (which is a lot of them...) than a useful feature. It's just bad design if, for example, your form requires an AJAX call be made and you can't actually hit a submit button to send the form when Javascript is disabled.
My mother was recently quite frustrated to discover that she is now unable to click through eBay results pages because each one requires Javascript. The only way she can see the next page of results is to turn on Javascript or to show more results per page. Now what reason would there be for page links to require Javascript while the 'results per page' links are just plain links? They should all be plain old HTML links. Maybe Javascript could be used to add some whiz-bang to the navigation, but a user should not be punished with a bad interface for having Javascript disabled. It's stupid on eBay's part, and it causes undue hassle for their users.
I am one of those that uses 'No-Script.' And I can tell you that sites that use javascript and don't work without it enabled is extremely annoying, stackOverflow... No we don't expect it to be very fancy, if I upvote load a new page that says "Thank you."
We expect to be able to use the site with reasonable limitations, don't ever display a page that says JS must be enabled, though, even if the site is crap without it. And yes if your site convinces us to stay we will enable. A function that isn't in common use on the site can also require javascript.
Please note that your site should also look good with no JS or CSS, if nothing else it is good for Bots.
As others have pointed out some phones don't have JS, this is changing but another good reason to have reasonable non-JS. I suggest code with non-JS and add JS after the former works, there are good ways where JS can work with the non-JS layout.
It helps me in my implementations to think about it as "progressive enhancement" rather than graceful degradation. Degradation often leads you to figure out how to make it work w/o js after it is implemented, instead of making a baseline and enhancing with js.
It is essential to at least test your website is functional when JavaScript is turned off.
As orip says, degrading gracefully is very important. It should be vital that your page both looks nice and functions when JavaScript is disabled.
For a standard web site that is primarily intended for conveying information, degrade gracefully always.
For web applications:
When building a web application for a standard internet audience, I would keep the three following facts in mind:
95%-97% of potential users will have JavaScript enabled.
At times established users will need to access functionality when JavaScript is not available.
3%-5% of potential users will have JavaScript intentionally disabled.
Given fact one, if you believe that building a JavaScript reliant web application will deliver a superior user experience, then by all means do it. Doing so may help you accumulate users.
However, given fact two, you should always provide a means by which your users can access core functionality without JavaScript. Do you need to offer every single feature? Probably not. But a user should be able to get his or her work done. This will keep your users happy when they find themselves temporarily without JavaScript.
Given fact three, I would also provide an in depth tour as an attempt to entice these users to enable JavaScript.
As an aside, one of my most favorite web applications, Remember The Milk follows this approach. Also, Google's Calendar application is unusable without JavaScript. So JavaScript reliant web apps are on the rise and that trend is probably unstoppable. In my opinion this is a good thing.
(Do keep in mind that JavaScript make Accessbility a bigger problem than it is already. Please do make an effort to make your apps usable by those with disabilities.)
As said before, it depends on your target audience.
If I'm part of it, you want to make sure that your site works (if not ideally) on my phone, and that it gives me reason to turn Javascript on when I surf there with it off. Nobody expects full functionality with Javascript disabled, and anybody who uses their phone to access websites expects some issues, but you need to at least provide teasers. For a web store, make sure customers can see at least some merchandise anyway, even if they can't buy without Javascript.