I use Abot for crawling,
I want to crawl a website that appears to block any request that doesn't have javascript enabled
it's a php page, and i get a "Please activate javascript to view this site." instead of the real site
How does a page know if javascript is enabled or not? (and in your opinion - do you think i can overcome that?)
Thanks
Unless you crawl the site with a javascript-enabled client, you are out of luck.
Related
I was wondering if there's any way to attach a js lib to an external webpage after the page has loaded?
To provide a simple example, could I load www.google.com into IE and somehow display the webpage with a green scroll bar?
I would like this process to happen automatically on each page load instead of having to manually execute this process on each page load.
I am assuming that you are talking from a web developer's point of view.
I don't think it is possible without any hacks.
This would also be a huge security risk, because loading javascript code on an external website means that the code can potentially do anything on behalf of the user. It can capture keystrokes, take screenshots, note down passwords and do a lot of illegal stuff.
So instead of this, you can create a browser extension (add-on) which will have to be installed by user's permission (and his knowledge), and can run any code on any page (if the user allows it)
Would it be possible to disable javascript in a website that you are trying to frame? If that website is trying to bust your framing attempts, could you essentially disarm all of their javascript completely, even if this means that the user will have to interact with the site without javascript?
Nope. If you are loading it in a frame or iframe there is no way to specify "load this website but don't enable javascript in its frame".
You might be able to proxy it and strip the js out: bad choice but probably your best option.
That should not be possible, for several reasons. You can't enable/disable JavaScript at all from inside a page, other than to simply not use it, and you can't affect what's going on in pages loaded from different domains.
"Frame busting" is something sites do to protect themselves from sites like the one you're describing :-)
You could do this server side by fetching the site and removing all of their script tags, then sending that to the user instead of a direct connection. But that could get very complicated if you needed to use cookies or something like that.
There is no way to do it client side because browsers prevent xss. http://en.wikipedia.org/wiki/Cross-site_scripting
Is there any way to programatically disable cookies being created by a page displayed inside of a frame/iframe? Or to generalise further, is it possible to programatically disable javascript running on such a page?
Thanks,
DLiKS
with iframe sandbox attribute (html5) it will be possible (implemented in chrome)
http://dev.w3.org/html5/spec/Overview.html#attr-iframe-sandbox
NullUserException already answered what you can do today without browser support
The only way you could change that for an external website you have no control over is to retrieve the pages using a server-side script, filter the input and display it to the user (ie: act as a proxy).
You just can't modify sites out of your domain (or subdomain - it might depend on the browser) using J/S for security reasons.
If you mean that you want to change the settings of the browser by code in a web page, then no, this is not possible (and if it were possible, it's a huge security breach and all alarms would go off).
You may surpass this, however, by writing a plugin, but then each user must first download the plugin. You can also request higher priviledges, but it'll depend on the browser whether you can change any user settings.
If you mean that you want to write a script on every pc in your company to disable JS + cookies for certain pages, you can write plugins and install them everywhere, or use a proxy (as has been suggested by others) and filter the pages. If it is just for debugging a self-made page, use the Developer Toolbar for the various browsers, that can turn JS/Cookies on and off.
Why would you want to do such a thing? If you want to disable cookies, you disable it on your own page by simply not using cookies. The same goes for javascript: don't add it to your page and you've disabled it on your page.
Note: if any page would change anything of the user settings of the browser, your page will probably be blacklisted by Google, most virus scanners and fishing prevention tools.
I apologize if this has been asked before. I searched but did not find anything. It is a well-known limitation of AJAX requests (such as jQuery $.get) that they have to be within the same domain for security reasons. And it is a well-known workaround for this problem to use iframes to pull down some arbitrary HTML from another website and then you can inspect the contents of this HTML using javascript which communicates between the iframe and the parent page.
However, this doesn't work on the iPhone. In some tests I have found that iframes in the Safari iPhone browser only show content if it is content from the same site. Otherwise, they show a blank content area.
Is there any way around this? Are there other alternatives to using iframes that would allow me to pull the HTML from a different domain's page into javascript on my page?
Edit:
One answer mentioned JSONP. This doesn't help me because from what I understand JSONP requires support on the server I'm requesting data from, which isn't the case.
That same answer mentioned creating a proxy script on my server and loading data through there. Unfortunately this also doesn't work in my case. The site I'm trying to request data from requires user login. And I don't want my server to have to know the user's credentials. I was hoping to use something client-side so that my app wouldn't have to know the user's credentials at the other site.
I'm prepared to accept that there is no way to accomplish what I want to do on the iPhone. I just wanted to confirm it.
You generally can NOT inspect the contents of an iframe from another domain via JavaScript. The most common answers are to use JSONP or have your original server host a proxy script to retrieve the inner contents for you.
Given your revisions, without modification or support from the secondary site, you are definitely not going to be able to do what you want via the iPhone's browser.
"In some tests I have found that iframes in the Safari iPhone browser only show content if it is content from the same site"
I found the same thing. Is this documented somewhere? Is there a workaround? This sounds like broken web standards to me, and I am wondering if there is a solution.
Is there any way I can detect when my page has been set as the user's homepage in their browser?
I'm most interested in something in javascript, but I'd be happy to hear about other approaches as well.
Edit: I'm not looking for anything sneaky. I'm wondering if there is anything that is explicitly allowed through the browsers to find out this information.
There isn't likely to be a foolproof method, as that's an intrusion into the privacy of the user.
One thing that comes to mind is checking for a referrer. If the user arrived at your page without following a link, they a) typed the url, b) followed a bookmark, or c) have your page set as their homepage. But that's about the best I can do.
Nope. You can tell if they got to your page by following a link or not. Check the referrer. However, the browser going to your page as the home page will not appear any different than the user typing in your page or using a bookmark.
Mozilla/Firefox has a window.home() method which loads the user's home page. This method could be used (in an iframe, maybe) combined with server access logging, to see if the site's home page is instantly requested loaded by the current user.
However, other browsers don't seem to support this javascript method.
Simple solution, you shouldn't be checking if you or anyone else's site is set as the user's homepage. If they want it as their homepage, they'll make it so.
However, if you've got some Javascript that will check their email and see if they've sent links to your site to their friends or colleagues, I'd be very interested in that functionality ;-)