Is there an API that allows accessing Google Chrome sync data, such as bookmarks, history and opened tabs, by a third-party web application, ideally using only client-side javascript?
Purpose
I am trying to write a self-contained HTML app, that can synchronize reading progress of webcomics across mobile and desktop devices. Ideally, such functionality would be redundant, as bookmarks in theory can fill that purpose. In practice however, especially mobile versions of Chrome lack the means to conveniently update bookmarks, and sorting mechanisms (e.g. sorting by last-modified date). Note that these issues only become apparent when following upwards of 50 webcomics, most of which at that point will only be caught up on very rarely.
Since this is meant as a hobby project, the cost of hosting a webservice or developing for iOS are prohibitive, thus the attempt to reuse existing cloud accounts of the user for synchronization.
Possible discontinued API
Until roughly one or two years ago, the Dolphin and Mercury browsers on iOS allowed synchronizing with Google Chrome directly, by authenticating the app for access to one's Google account. At that time both browsers required that bookmarks were synchronized without an encryption passphrase.
Then suddenly they both dropped this feature, and instead started providing their own synchronization extensions and servers (again, without end-to-end encryption).
So I fear, that Google might have once provided a limited public synchronization API, which was dropped within the last two years; I was however unable to verify even this.
As far as I know they dropped support
We locked down access to chromesync scoped refresh tokens to address a security vulnerability. When we did so, we knew that this may break some 3P browsers which made use of chromesync scoped refresh tokens to leverage Chrome Sync for their users.
Chrome Sync has never officially been supported for 3P browsers. We do not intend to create a solution by which 3P browsers can whitelist themselves or their users so that they can get chromesync scoped refresh tokens. Note that Chromium for Android is technically considered a 3P browser.
Marking as WontFix, accordingly.
https://bugs.chromium.org/p/chromium/issues/detail?id=677887
Related
I've tried on getting browser info from javascript - navigator.userAgent, but cannot find any related string with "Vivaldi".
Please help to suggest if there are any other programming method or javascript can be used to detect Vivaldi browser?
The correct syntax in JavaScript is navigator.userAgent.
You can check window size, but it may be incorrect.
Vivaldi has its own sidebar and bottom bar that the others not.
But remember, user can change the window size to prevent detection.
According to Vivaldi, the userAgent will not have Vivaldi in it for a better user experience and mimic Chrome in all sites except a few known ones.
For the next release of Vivaldi, we have decided to try something different. The problem with our current approach is that with the web being almost infinite, we can’t possibly discover all the websites who have blocks set against us. Thus maintaining a list of sites where we present a non-Vivaldi User Agent is difficult. Instead, we will try doing the reverse. For a handful of sites where we know the label Vivaldi (and our version number) is responsibly used, we will present our full User Agent. Those sites being our own and a handful of interesting alternative search engines: duckduckgo.com, ecosia.org, qwant.com, and startpage.com. Every other site will get a User Agent that appears to be identical to Chrome.
src: https://vivaldi.com/blog/user-agent-changes/
Since you haven't mentioned your use case, assuming you are building sites, the best solution would be to do what everyone does. I recommend building websites agnostic to browsers and devices, but responsive to resolutions and feature support. Almost all of the browsers are running on Chromium these days, except Safari and Firefox (to my knowledge). So they are all going to work consistently across the board. For any given feature, one could always refer to w3c or mdn docs for compatibility and build around that.
Browser detection is also not quite reliable, as there are extensions to spoof the information. Like this extension that mimics a TV agent so one can run youtube console version on it.
On the other hand, once again I don't know your use case, one might benefit from platform detection over browser detection, like catering to mobile and TV users. Even then, if you have a site responsive enough the platform becomes non-essential information.
Is there any way to consistently detect PhantomJS/CasperJS? I've been dealing with a spat of malicious spambots built with it and have been able to mostly block them based on certain behaviours, but I'm curious if there's a rock-solid way to know if CasperJS is in use, as dealing with constant adaptations gets slightly annoying.
I don't believe in using Captchas. They are a negative user experience and ReCaptcha has never worked to block spam on my MediaWiki installations. As our site has no user registrations (anonymous discussion board), we'd need to have a Captcha entry for every post. We get several thousand legitimate posts a day and a Captcha would see that number divebomb.
I very much share your take on CAPTCHA. I'll list what I have been able to detect so far, for my own detection script, with similar goals. It's only partial, as they are many more headless browsers.
Fairly safe to use exposed window properties to detect/assume those particular headless browser:
window._phantom (or window.callPhantom) //phantomjs
window.__phantomas //PhantomJS-based web perf metrics + monitoring tool
window.Buffer //nodejs
window.emit //couchjs
window.spawn //rhino
The above is gathered from jslint doc and testing with phantom js.
Browser automation drivers (used by BrowserStack or other web capture services for snapshot):
window.webdriver //selenium
window.domAutomation (or window.domAutomationController) //chromium based automation driver
The properties are not always exposed and I am looking into other more robust ways to detect such bots, which I'll probably release as full blown script when done. But that mainly answers your question.
Here is another fairly sound method to detect JS capable headless browsers more broadly:
if (window.outerWidth === 0 && window.outerHeight === 0){ //headless browser }
This should work well because the properties are 0 by default even if a virtual viewport size is set by headless browsers, and by default it can't report a size of a browser window that doesn't exist. In particular, Phantom JS doesn't support outerWith or outerHeight.
ADDENDUM: There is however a Chrome/Blink bug with outer/innerDimensions. Chromium does not report those dimensions when a page loads in a hidden tab, such as when restored from previous session. Safari doesn't seem to have that issue..
Update: Turns out iOS Safari 8+ has a bug with outerWidth & outerHeight at 0, and a Sailfish webview can too. So while it's a signal, it can't be used alone without being mindful of these bugs. Hence, warning: Please don't use this raw snippet unless you really know what you are doing.
PS: If you know of other headless browser properties not listed here, please share in comments.
There is no rock-solid way: PhantomJS, and Selenium, are just software being used to control browser software, instead of a user controlling it.
With PhantomJS 1.x, in particular, I believe there is some JavaScript you can use to crash the browser that exploits a bug in the version of WebKit being used (it is equivalent to Chrome 13, so very few genuine users should be affected). (I remember this being mentioned on the Phantom mailing list a few months back, but I don't know if the exact JS to use was described.) More generally you could use a combination of user-agent matching up with feature detection. E.g. if a browser claims to be "Chrome 23" but does not have a feature that Chrome 23 has (and that Chrome 13 did not have), then get suspicious.
As a user, I hate CAPTCHAs too. But they are quite effective in that they increase the cost for the spammer: he has to write more software or hire humans to read them. (That is why I think easy CAPTCHAs are good enough: the ones that annoy users are those where you have no idea what it says and have to keep pressing reload to get something you recognize.)
One approach (which I believe Google uses) is to show the CAPTCHA conditionally. E.g. users who are logged-in never get shown it. Users who have already done one post this session are not shown it again. Users from IP addresses in a whitelist (which could be built from previous legitimate posts) are not shown them. Or conversely just show them to users from a blacklist of IP ranges.
I know none of those approaches are perfect, sorry.
You could detect phantom on the client-side by checking window.callPhantom property. The minimal script is on the client side is:
var isPhantom = !!window.callPhantom;
Here is a gist with proof of concept that this works.
A spammer could try to delete this property with page.evaluate and then it depends on who is faster. After you tried the detection you do a reload with the post form and a CAPTCHA or not depending on your detection result.
The problem is that you incur a redirect that might annoy your users. This will be necessary with every detection technique on the client. Which can be subverted and changed with onResourceRequested.
Generally, I don't think that this is possible, because you can only detect on the client and send the result to the server. Adding the CAPTCHA combined with the detection step with only one page load does not really add anything as it could be removed just as easily with phantomjs/casperjs. Defense based on user agent also doesn't make sense since it can be easily changed in phantomjs/casperjs.
I am currently developing a browser extension/plugin that one would install and would then report information such as page load times, number of objects on each page, etc. to a server so that the data could be analyzed.
I was curious if something like Yahoo's Boomerang JavaScript code (http://hacks.bluesmoon.info/boomerang/doc/) would be able to be able to do this. From what I have read, it seems like Boomerang was developed for developers to implement into their own website in order to gather data, but would I be able to gather the same kind of data by putting this code in a browser extension in order to gather the data from each website that is visited?
The link you're using for boomerang is very outdated (it was my first experimental page). Use http://lognormal.github.com/boomerang/doc/
Boomerang does already use these APIs, and much more, but as a browser extension, you could do much more in terms of removing code that supports other browsers, and also maintaining offline storage more efficiently than boomerang's cookies.
FWIW, yslow already does most of what you want, so maybe just use that (it was built by the same team)
I don't see why not from a technical perspective, at least in Firefox and Chrome. However, user privacy issues and policies of the browser extension stores might prevent you from tracking users in certain ways and/or without consent. So better check that first to avoid surprises later.
You'd need a way to gather information. Judging from your question text, the regular DOM APIs and the PerformanceTiming API might be sufficient. And that's probably what Boomerang uses already.
You'd just attach your code (or Boomerang) with e.g. Firefox Add-on SDK PageMod or Chrome extension Content Scripts.
You'll also need to transmit the data somewhere. Both Firefox (XUL1, Add-on SDK) and Chrome extensions allow cross-origin XHR.
So there you are. ;)
1 XUL overlay scripts are privileged, and not restricted by the same-origin policy.
I'm trying to detect the presense of certain addons that restrict functionality on our sites (such as Ghostery, or DoNotTrackMe), using plain JavaScript. If Firefox had a standard API to query for the existence of addons, it would short circuit my work. Probably something like:
AddonManager.getAddonByID('?')
Detection is possible for plugins such as Java and Flash (navigator.plugins), but not for addons, yet.
Detection is also possible if you are the developer of the addon.
No, there is no such API and there won't be. If the browser allows websites to detect what extensions are installed then it is a bug and a privacy issue - Firefox and Chrome fixed such issues in the past, currently no such issues are supposed to exist. Why webpages shouldn't know this:
The add-on list allows fairly precise fingerprinting of users and might allow tracking users even if cookies and similar mechanisms are disabled.
Attacking vulnerabilities in extensions gets a lot simpler if a website can check in advance whether a vulnerable extension/extension version is installed. Same goes for social engineering attacks that rely on the presence of particular extensions.
There is always a chance that websites decide to exclude website visitors that have unpopular software installed (ad blockers of all kinds, vulnerability scanners, download helpers etc).
The list of installed extensions tells a lot about the user, his browsing habits and preferences - providing this information to websites would be a huge privacy issue.
navigator.plugins functionality shares many of the same issues of course - it was introduced in the dark ages of the internet back when nobody considered such issues. This is the reason why this functionality is being actively discussed and might get limited in future.
Now of course websites can still try to detect extensions by their effects. However, this isn't a very reliable approach, e.g. the Adblock detector you linked to says "no ad blocking detected" to me and is clearly wrong.
Bonus info: Google Chrome does provide a list of installed extensions but only to a single website: Chrome Web Store. The Web Store uses it to indicate which extensions you already installed. Whether it does anything beyond it with that data is something where your guess is as good as mine.
You can detect AdBlock in FF, IE, Chrome
http://adblockdetector.com/
You can detect FlashBlock in FF, IE, Chrome and others
https://gist.github.com/artlung/1420308
https://github.com/browserstack/flashblock-detector/blob/master/flashblock-detector.js
You can detect Ghostery in Chrome
http://blog.securitee.org/?p=277
Please note these scripts use inference to detect the presense of these addons, there's no API. I haven't found detectors for other addons but they possibly exist.... (ongoing)
I've been thinking about the access web-based applications have to an OS.
I'm curious:
What is the best way of determining
this as it currently stands?
Is the trend leaning toward more, or
less access?
What functionalities should be
open/closed?
A simple example would be.. say your g-mail alerted you in the task-bar when an incoming e-mail is received.
In the past, browsers have shown a strong tendency to prevent all manner of access to the operating system (and also the browser chrome). The fundamental risk is about trust and deception; if all you have done is visited a website, do you want it popping up dialog boxes or reading files on your hard drive? Even worse, you might not have visited the website on purpose; you might have been subject to a phishing attack where a spammy email linked you to a URL that looks like a popular site, but has a "1" instead of an "I".
It's nuanced to say which way the trend is going. In one sense, you can now do more than before, because you have the browsers enabling GPU support and, as part of HTML5, offline storage. All of this is being done with careful consideration of the security issues though, ensuring sandboxing takes place. On the other hand, you have browsers locking down on things like cookies on the file:// URI.
Many apps are increasingly web apps, but it's not as simple as just pointing to the web app in your browser. It might be an app you installed as a mobile web widget, or purchased in an app store like the Palm Pre's, where most apps are basically just web apps. The point is, the trust scenario is different in each case; I'd feel more confident giving certain OS access to an app from a reputable app store, where the code has been inspected, the producer has signed it, and the app's actions and privileges can be tracked ... than a random website I happened upon.
Any access that is given is going to be abused by malware. Guaranteed.
I think the trend is toward less access; consider Google's Chrome OS, in which you do all of your work within web applications, which have no native access to your system at all.