Requesting data from any location on my domain with .load() (or any jQuery ajax functions) works just fine.
Trying to access a URL in a different domain doesn't work though. How do you do it? The other domain also happens to be mine.
I read about a trick you can do with PHP and making a proxy that gets the content, then you use jQuery's ajax functions, on that php location on your server, but that's still using jQuery ajax on your own server so that doesn't count.
Is there a good plugin?
EDIT: I found a very nice plugin for jQuery that allows you to request content from other pages using any of the jQuery function in just the same way you would a normal ajax request in your own domain.
The post: http://james.padolsey.com/javascript/cross-domain-requests-with-jquery/
The plugin: https://github.com/jamespadolsey/jQuery-Plugins/tree/master/cross-domain-ajax/
This is because of the cross-domain policy, which, in sort, means that using a client-side script (a.k.a. javascript...) you cannot request data from another domain. Lucky for us, this restriction does not exist in most server-side scripts.
So...
Javascript:
$("#google-html").load("google-html.php");
PHP in "google-html.php":
echo file_get_contents("http://www.google.com/");
would work.
Different domains = different servers as far as your browser is concerned. Either use JSONP to do the request or use PHP to proxy. You can use jQuery.ajax() to do a cross-domain JSONP request.
One really easy workaround is to use Yahoo's YQL service, which can retrieve content from any external site.
I've successfully done this on a few sites following this example which uses just JavaScript and YQL.
http://icant.co.uk/articles/crossdomain-ajax-with-jquery/using-yql.html
This example is a part of a blog post which outlines a few other solutions as well.
http://www.wait-till-i.com/2010/01/10/loading-external-content-with-ajax-using-jquery-and-yql/
I know of another solution which works.
It does not require that you alter JQuery. It does require that you can stand up an ASP page in your domain. I have used this method myself.
1) Create a proxy.asp page like the one on this page http://www.itbsllc.com/zip/proxyscripts.html
2) You can then do a JQuery load function and feed it proxy.asp?url=.......
there is an example on that link of how exactly to format it.
Anyway, you feed the foreign page URL and your desired mime type as get variables to your local proxy.asp page. The two mime types I have used are text/html and image/jpg.
Note, if your target page has images with relative source links those probably won't load.
I hope this helps.
Related
I have a page on my site (let's say on domain A) and I would like to pull in some more content into it from another page, say, on domain B. As a default, this functionality is blocked by the browsers for security reasons.
As far as I've found, there are a few ways to do this.
CORS: As I understand, this method requires contributions from both the server and the client. The server needs to add a header to its response (i.e. Access-Control-Allow-Origin: [DOMAINS], as of http://enable-cors.org/server.html). On the other hand, the client needs to adjust their requests (e.g. http://www.html5rocks.com/en/tutorials/cors/).
If using jQuery, there is this small plug-in which uses the YahooAPI (i.e. http://james.padolsey.com/snippets/cross-domain-requests-with-jquery/). The advantage of this is that the client can use it on its own to get pages from other domains. The catch is that Yahoo limits the number of requests per hour per IP, and for commercial use Yahoo's permission is needed.
I've also read about JSONP but I haven't done much digging.
My question is: are there other possibly better options that I might be overlooking?
For the record, the site I'm working with is a huge commercial site with millions of users every day.
You can do JSONP, permit CORS and use plain JSON, use a DIY JSONP wrapper, or use a JSONP Proxy service. Here are the solutions in detail: JSONP with remote URL does not work
The easiest option in your situation is to roll your own JSONP proxy service. Here's a demo barebones PHP wrapper to get past CORS if you fetch a JSON string. No catch, no limits unlike Yahoo's YQL.
<?php
$callback = isset($_GET["callback"]) ? $_GET["callback"] : "?";
$json = file_get_contents('http://somedomain.com/someurl/results.json');
header('Access-Control-Allow-Origin: *');
header("Content-type: application/json");
echo $callback . "(" . $json . ");";
?>
Are you trying to get content, or code? If you're trying to get content, is it possible to just use an iframe?
If you want code, I think the options you outlined are pretty much what you have available. JSONP might be your best bet due to browser support. For example, IE only supported it as of version 10. If you're on a site with millions of users per day, my guess is there are some folks on older versions of IE (unfortunately).
Edit: Depending on the content, another option is to introduce your own local proxy. For example, I've done things where I need to call WebServiceX on some other provider. I call the WebServiceX in server side code and implement my own web service that my JavaScript accesses. This means I'm not going cross domain because the cross domain access happens server-side, not client-side. It also allowed me to introduce caching and other things (depending on the type of data) that improved performance.
Approach for cross domain data passing - create JavaScript object and assign source from another domain. Here is quick and dirty example:
File test.html:
<html>
<body>
Test done
</body>
<script>
var s = document.createElement("script");
s.type='text/javascript';
s.src='test.js';
document.body.appendChild(s);
</script>
</html>
and test.js
abc={a:'A',b:'B',c:'C'};
alert(abc.a);
test.js could be in any domain and function alert() could be any function.
I have more elegant ways to attach or run such approach but this one is sufficient enough to undersatnd the idea.
As the title suggests, I'm looking for a hopefully straightforward way of scraping all of the HTML from a webpage. Storing it in a string perhaps, and then navigating through that string to pull out the desired element.
Specifically, I want to scrape my twitter page and display my profile picture inside a new div. I know there are several tools for doing just this, but I would anyone have some code examples or suggestions for how I might do this myself?
Thanks a lot
UPDATE
After a very helpful response from T.J. Crowder I did some more searching online and found this resource.
In theory, this is easy. You just do an ajax call to get the text of the page, then use jQuery to turn that into a disconnected DOM, and then use all the usual jQuery tools to find and extract what you need.
$.ajax({
url: "http://example.com/some/path",
success: function(html) {
var tree = $(html);
var imgsrc = tree.find("img.some-class").attr("src");
if (imgsrc) {
// ...add the image to your page
}
}
});
But (and it's a big one) it's not likely to work, because of the Same Origin Policy, which prevents cross-origin ajax calls. Certain individual sites may have an open CORS policy, but most won't, and of course supporting CORS on IE8 and IE9 requires an extra jQuery plug-in.
So to do this with sites that don't allow your origin via CORS, there must be a server involved. It can be your server and you can grab the text of the page you want using server-side code and then send it to your page via ajax (or just build the bits you want into your page when you first render it). All of the usual server-side stacks (PHP, Node, ASP.Net, JVM, ...) have the ability to grab web pages. Or, in some cases, you may be able to use YQL as a cross-domain proxy, using their server rather than your own.
Context: I'm trying to code a javascript function to like a certain post on tumblr, based on this link . I tried using an ajax call instead of changing the source of an iframe, but it doesn't work. Of course, changing the source of an iframe works.
So, what can be the difference that make this not work?
$baseUrl = 'http://tumblr.com/like/';
function LikePost( $postID, $reblogUrl )
{
/*
http://www.tumblr.com/<command>/<oauthId>?id=<postId>
<command>: like or unlike
<oauthId>: last eight characters of {ReblogURL}
<postId>: {PostID}
Exemple of Url
http://www.tumblr.com/like/fGKvAJgQ?id=16664837215
*/
$oauthId = $reblogUrl.substring( $reblogUrl.length - 8, $reblogUrl.length);
$likeUrl = $baseUrl + $oauthId + '?id=' + $postID;
$.ajax({
url: $likeUrl,
type:'POST'
});
}
AJAX requests are bound by same domain policy, with some exceptions that aren't worth listing since they don't work unless you control both domains.
In this case, you're calling a tumblr domain from your website, which you can't do through AJAX. However, iframes, script elements, and img elements can point to any domain, so if the like url isn't returning any content to you, you can use any of those means to record the like.
If you didn't want to use an iframe, the other method you could use would be to make a request to your server via AJAX, then proxy the request to tumblr. Your server can go to any url it wants.
However, the iframe approach is easiest. I suggest going that route since you already got it working. ;)
They are intended for different purposes. As jmort253 noted above, AJAX calls work only for the same domain, whereas Iframes may span different domains. But if you are interested in loading data from the same domain, AJAX may be a better option. Many times, while using IFrame, you will see a loading sign on the tab-bar of the page, showing that something inside it is loading (it's the IFrame page which is loading, not the entire page), which you don't want the user to see, because that is the point of AJAX, loading data seamlessly, giving the user the illusion that the data is coming almost simultaneously. With AJAX, you won't have these problems.
And even if you want to load data from different domains, while Javascript itself is not upto the task, you can use PHP to do the loading part, then use Javascript to fetch the data from there.
I have a URL of a remote page from a different domain which I have to download, parse, and update DOM of the current page. I've found examples of doing this using new ActiveXObject("Msxml2.XMLHTTP"), but that's limited to IE, I guess, and using new java.net.URL, but I don't want to use Java. Are there any alternatives?
Same domain policy is going to get you.
1) Proxy through your server. browser->your server->their server->your server->browser.
2) Use flash or silverlight. The 3rd party has to give you access. The bridge between javascript and flash isn't great for large amounts of data and there are bugs. Silverlight isn't ubiquitous like flash...
3) use a tag. This really isn't safe... Only works if 3rd party content is valid javascript.
Whats about load an PHP Script via AJAX which does file_get_contents() ? This should work for different domain. If i understand correct.
Writing a server-side script that will retrieve the page's content for you is the way to go. You can use the XMLHttpRequest object to make an AJAX call to that script, which will just put through all html (?) for you.
Still, I advise against it. I don't know exactly how much you trust the other site, but the same origin policy exists for a reason. What is it exactly you are trying to do? Usually, there is a workaround.
I dont think you can do this according to the constraints of same origin policy. Two communicate between two domains using Iframes also we can use JS code but both domains need to have communicating code in them. The Child frame can contact the grandparent frame (window) but not here.
Since you are referring to some other url all togeather.
The only way is to do it using your server side code to access the content on the other domain.
Just use PHP:
<?php
$url = "http://www.domaintoretrieve.com";
ob_start();
include_once( $url );
$html = ob_get_contents();
ob_end_clean();
?>
$html contains the entire page to manipulate as needed.
The XMLHTTPRequest object is common to most modern browsers and is what powers AJAX web applications.
I've been at this for several days and searches including here haven't give me any solutions yet.
I am creating a Bookmarklet which is to interact with a POST API. I've gotten most of it working except the most important part; the sending of data from the iframe (I know horrible! If anyone knows a better solution please let me know) to the javascript on my domain (same domain as API so the communication with the API is no problem).
From the page the user clicks on the bookmarklet I need to get the following data to the javascript that is included in the iFrame.
var title = pageData[0].title;
var address = pageData[0].address;
var lastmodified = pageData[0].lastmodified;
var referralurl = pageData[0].referralurl;
I first fixed it with parsing this data as JSON and sending it through the name="" attribute of the iFrame but realized on about 20% of webpages this breaks. I get an access denied; also it's not a very pretty method.
Does anyone have anyidea on how I can solve this. I am not looking to use POSTS that redirect I want it all to be AJAX and as unobtrusive as possible. It's also worth noting I use the jQuery library.
Thank you very much,
Ice
You should look into easyXDM, it's very easy to use. Check out one of the examples on http://consumer.easyxdm.net/current/example/methods.html
After a lot of work I was able to find a solution using JSONP which is enables Cross Domain Javascript. It's very tricky with the Codeigniter Framework because passing data allong the URLs requires a lot of encoding and making sure you dont have illegal characters. Also I'm still looking to see how secure it really is.
If I understand your question correctly, you might have some success by looking into using a Script Tag proxy. This is the standard way to do cross domain AJAX in javascript frameworks like jquery and extjs.
See Jquery AJAX Documentation
If you need to pass data to the iframe, and the iframe is actually including another page, but that other page is on the same domain (a lot of assumptions, I know).
Then the man page code can do this:
DATA_FOR_IFRAME = ({'whatever': 'stuff'});
Then the code on the page included by the iframe can do this:
window.parent.DATA_FOR_IFRAME;
to get at the data :)