Apologies if this is a roundabout way of asking this question, but I am a little confused about how the web and javascript work.
What I want to do: execute javascript on all pages of a list of urls I have found. (Specifically use jquery to pull info from them)
Problem I can't execute Javascript on these pages because they aren't mine and don't have the Access-Control-Allow-Origin header. So I can't load them (with AJAX) in order to use JQuery on them.
BUT Google Chrome can both load pages and execute javascript on them (with their developer's console). So if I wanted too, I could go to each page, open the developers console, and pull the information from there. If there's nothing stopping Chrome from accessing these, then why am I stopped? And, is there a way around this?
Thank you, and I hope my description makes sense. I've been researching this for a while but have found nothing that explains how seemingly inconsistent CORS is.
I could go to each page, open the developers console, and pull the information from there. If there's nothing stopping Chrome from accessing these, then why am I stopped?
You're not stopped. You, the human at the keyboard, can do exactly as you say, by visiting each page as a top-level page.
What is stopped -- happily -- is any and all scripts on the Web you happen to run having the same level of visibility that you do. Based on your cookies and your network topology, you have a unique view into the Web. You can see your home router's control interface (on 192.168.1.1 or similar). You can see any local web server you're running on 127.0.0.1. No one else can see these. If the same-origin policy were not in place, then any script that you loaded on the Web could inspect these.
And, is there a way around this?
If you have some scripts that you trust absolutely (hopefully a significant subset of "all scripts that exist on the Web") that you want to be able to bypass the same-origin policy and see your full, cross-domain view of the Web, you could load them as an extension, which can act with elevated permissions beyond the abilities of normal web pages. (See How does Same Origin Policy apply to browser extensions?)
I'm going to assume that you are looking to grab data from these pages that aren't yours and store it somewhere. I have done this before with curl using php. If you are looking to display these sites for users to interact in a different way, but starting from a page that is yours, you may be able to render these pages by grabbing the source html using curl and rendering it as a sort of proxy.
I've used this tutorial for something similar https://www.youtube.com/watch?v=_kQN-3aNCeI . Hopefully this gives you a start. I think you should be a little more detailed in your question though to get more help.
Related
How can I copy the source code from a website (with javascript)? I want to copy the text that is showing the temperature from this website: http://www.accuweather.com/
I want to copy only the number that is displaying the temperature. Is there a way of copying that exact line from source code on the website? I heard about html scraping. if not javascript, what would be simplest way of doing it? Just copying the temeprature, and displaying it on my webpage.
Well the way you could do something like that in a simple way by loading the site into a hidden HTML element via AJAX and then search DOM for the element you want.
There is also a jQuery command that allows that directly. It would be something like:
<div id='temp'></div>
<script>
$('div#temp').load('https://www.accuweather.com/ #popular-locations-ul .large-temp', { limit: 1 });
</script>
#popular-locations-ul .large-temp is a css locator for the specific elements that contain the temperature.
However for some time web has a security feature called CORS. To be able to load something from one site via AJAX, the target site has to allow CORS headers explicitly. In the case of this particular site, CORS headers aren't present in the site configuration, so that means that any connection that tries to load something via AJAX won't be allowed.
You can only use a command like the above mentioned in a site you control and that you specify to allow CORS headers or in a site who already has this specification.
But as people have told you that's not a good thing from the start due to web sites impermanent nature. Things change a lot. So even if you could get a value in the way I mentioned from some other site, sometime later, the site would change and your code would be broken.
The reason I answered is because you are just learning and need guidance and not trying to do 'serious work'. Serious work would be using an API as people told you.
An web api is a special url you access (something like https://www.accuweather.com:1234/api/temperature/somecity) normally with some kind of security and that responds with the result you need for the function you want. For this kind service CORS is allowed because you are accessing in a secure and 'official' way.
Hope I clarified a bit.
It looks like AJAX is indeed unable (at least for all practical purposes) to write foreign HTML to the current page. But what if your CDN website had, say, a JS that would simply document.write() everything? Then your HTML document would have nothing but a remote script.
<html>
<script src="https://pastebin.com/raw.php?i=0wm5v7i6">
</script>
</html>
I tried this. Funny thing is, sometimes it works and other times it does a kind of security error:
Why doesn't this work? What if, on your own website, you simply put everything on an easy host like Google Drive?
What if, on your own website, you simply put everything on an easy host like Google Drive?
That is possible, unless
You want control over your website and don't want to depend on the security and availibility of another site, or that somebody reports your pastebin as abuse and it gets deleted.
You want to make proper use of security features like content security policy and don't want to allow everything from pastebin.com.
You want search engines to find you. Although at least google does limited interpretation of JavaScript I doubt that they will handle this content the way you like.
From the looks of it, PasteBin doesn't supply content over SSL (https). You've put https in the URL to your script, but PasteBin just redirects this request to http, and the net effect is that you are trying to access a script over http when the page is accessed over https, and Chrome prevents that.
Just try going to https://pastebin.com/raw.php?i=0wm5v7i6: your browser will be redirected to http://pastebin.com/raw.php?i=0wm5v7i6.
On the website http://imaginaryman-test.blogspot.com/ the typewriter is inside of an IFAME . Everything works correctly on all browsers when you go to the site directly http://castedspell.com/mark/ but when viewing the version embeded in an IFRAME it does not work on IE and throws errors in Chrome.
Unsafe JavaScript attempt to access frame with URL http://imaginaryman-test.blogspot.com/ from frame with URL http://castedspell.com/mark/. Domains, protocols and ports must match.
This is the source code for the embedded IFRAME
https://github.com/totheleftpanda/typeWrite/tree/master/mark
I understand that this is a security problem but I don`t know how to fix it and can not find any material that would help me solve the issue.
The easiest method is to set a PHP (or any server language) proxy that just gets the content of the page from the other domain and outputs it. The only real drawback is that the cookies of the client for the remote domain aren't sent.
Take a look at http://benalman.com/projects/jquery-postmessage-plugin/. This is a jquery plugin that sends message between the two frames. The two frames do not need to be on the same domain. But you do need to access both pages to be able modify them. I also wrote a post here that answers communication between iframes. How to capture clicks from iframe on another domain?
Your only chance is something like easyXDM. (or do it manually using the hash, but would prefer easyXDM)
See the SO answer: Cross-domain hash change communication
eg. if you wanna call a method:
http://easyxdm.net/wp/2010/03/17/remote-procedure-calls-rpc/
EDIT:
If I try your demo in firefox I don't get the "Unsafe JavaScript attempt to access" error at all. But in Chrome it's thrown many times.
You have so much other code in your example that I'm not even sure that your code causes the problem. You should do a very limited/basic test to see if your flash-communication works, without all those other javascripts.
I have had similar issues with this before. Basically if you have an iframe that contains a page from a domain that differs from the main page's domain, javascript will not be able to cross the boundaries between them. Javascript within the iframe will be able to talk within the iframe, javascript in the main page will be able to talk within the main page, but they will not be able to talk to each other.
This is a security issue that aims to stop cross-site scripting attacks. There are a number of hacks that you can put in place to get around this problem but they are all (or at least the ones I know of) rather hairy.
Here are some questions that you should answer before trying to go further:
1) What exactly are you trying to do between the pages using javascript?
2) Do you have access to the source of both pages?
It may be waaay simpler than the above answers. It looks like this function:
function playSound(){
swf.playSound();
}
Is written in the DOM timeline before swf is actually assigned to the swfObject in the function below it.
I would recommend moving that function down further and then retest.
Certain websites like Twitter, Flickr, etc avoid being stuck within an iframe. Is there any way to stop this from happening? I just need to see the public data so I am open to disabling Javascript, etc. How do I disable Javascript running on the iframe? Is this possible?
You can't disable JavaScript on iFrames or any other resources AFAIK.
The only way to reliably do this is to load the sites through a proxy PHP or other server-side script, filter out any JavaScript (which will break many sites), and fix all relative references to images and other media - a task that would take an insane amount of time to complete if you want the sites to actually work.
If you just need some data from the sites, proxying might work. Seeing as the Same Origin Policy would prevent you getting anything from an IFRAME from a different domain anyway, it is also the only way to access content on those sites using JavaScript.
In IE only, there is the <iframe security="restricted"> attribute. This disables JavaScript in the targeted document, which would break a JS frame-escape script — along with all other interaction that's script-dependent.
However, apart from the browser compatibility issue, it's very rude to frame a site that doesn't want to be framed, and it will work less and less anyway as more sites deploy X-Frame-Options.
I'm not sure what you mean by “need to see the public data”... as Pekka said, you won't be able to ‘see’ into an iframe's DOM from outside it, as that would be a security problem.
I apologize if this has been asked before. I searched but did not find anything. It is a well-known limitation of AJAX requests (such as jQuery $.get) that they have to be within the same domain for security reasons. And it is a well-known workaround for this problem to use iframes to pull down some arbitrary HTML from another website and then you can inspect the contents of this HTML using javascript which communicates between the iframe and the parent page.
However, this doesn't work on the iPhone. In some tests I have found that iframes in the Safari iPhone browser only show content if it is content from the same site. Otherwise, they show a blank content area.
Is there any way around this? Are there other alternatives to using iframes that would allow me to pull the HTML from a different domain's page into javascript on my page?
Edit:
One answer mentioned JSONP. This doesn't help me because from what I understand JSONP requires support on the server I'm requesting data from, which isn't the case.
That same answer mentioned creating a proxy script on my server and loading data through there. Unfortunately this also doesn't work in my case. The site I'm trying to request data from requires user login. And I don't want my server to have to know the user's credentials. I was hoping to use something client-side so that my app wouldn't have to know the user's credentials at the other site.
I'm prepared to accept that there is no way to accomplish what I want to do on the iPhone. I just wanted to confirm it.
You generally can NOT inspect the contents of an iframe from another domain via JavaScript. The most common answers are to use JSONP or have your original server host a proxy script to retrieve the inner contents for you.
Given your revisions, without modification or support from the secondary site, you are definitely not going to be able to do what you want via the iPhone's browser.
"In some tests I have found that iframes in the Safari iPhone browser only show content if it is content from the same site"
I found the same thing. Is this documented somewhere? Is there a workaround? This sounds like broken web standards to me, and I am wondering if there is a solution.