Can JavaScript load an RSS XML feed from Yahoo?
Is client-side JS allowed to access 3rd-party domains?
You can use the technique outlined in my blog post Unwritten guide to Yahoo Query Langauge
You would query the XML data table with a yql statment like this:
select * from xml
where url="http://path/to/xml
Then you would add a script tag to your html (can be done with document.createElement('script')) with a src http://query.yahooapis.com/v1/public/yql?q={your yql here}&format=json&callback={your function here} where {your yql here} is replace with a URI Encoded version of you yql statment.
An easy way to do this is to proxy the request through the server that your page resides on. Steps are:
Write a server side script performs an http request on the rss feed, when that script itself is request (i.e. via get or post)
Use ajax to request the server side script, or just call it from the main script for that page.
The server side script then returns the feed source in some displayable form.
Profit!
On IE 8 and FF 3.1(not certain), it is possible to make these requests through specialized cross site calls, but the last generation of browsers will still cause problems. See:
http://dannythorpe.com/2009/01/15/ie8-cross-domain-request-support-demo/
http://ejohn.org/blog/cross-site-xmlhttprequest/ Feature is restricted in FF 3.0, unclear if it will be back in 3.1
However, the steps above are guaranteed not to run afoul of any browser CSS security, at the expense of some lag and extra hw load on your server.
You can use the Google Feed API to load RSS in JavaScript from any domain / server. More than just a proxy, it actually serves the RSS content from the Google cache instead of hitting the original server. This could be a lifesaver for small servers that can't handle Slashdot traffic surges.
I used the Feed API for a cross-site RSS access in an article on Silverlight several years ago.
I'm not sure about JS but I know that you can use one of google's APIs and they have an RSS reader. I know this probably isn't what you want, but if you read through the documentation you may be able to get your answer on how it works.
Not directly. You can use Dana's suggestion of proxing the request, or look into a method called JSONP, which essentially wraps the returned JSON object in a custom callback function, requested by a script tag you inject into your DOM. Most API providers support this (including Yahoo's APIs).
Related
it is possible to crawl a website within an Angular-App? I am speaking about to call a website from Angular, not crawling an Angular-App. If that so, then I am wondering which IP will be shown on the crawled website. Since JavaScript is client-side, I would suggest, its the IP of the client, not of the server (like probably at nodejs). But all I know, its mostly browser-implemented stuff what we can use in JS, so it is even possible to crawl websites with methods from JavaScript (or Angular)?
Best Regards
Buzz
In theory, you can create an AJAX request to fetch the data with reponse type text/html. That would give you the remote document as a string. The browser wouldn't try to load the JavaScript and CSS in that document, though. That might not be a problem but CORS is. For security reasons, most browsers prevent you from loading data from somewhere else (otherwise, it would be too easy for criminals to put JavaScript into any web page). See here for details: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
If you have control over the second domain, you can configure the server there to send Access-Control-Allow-Origin headers to the browser to allow access from the Angular App.
Note: You could use an iframe to load the other website but when the domains of the current document and the one in the iframe don't match, then you can't access the contents of the iframe from JavaScript.
One way to work around this is to install a proxy on your server. The browser can then ask your server for the pages in question. In this case, the remote web site will get the IP of your server.
Here's my situation -- I have an account on a site that allows API access. So, theoretically, I could write a program to query the site via its API. I would like to build a local html page using javascript, that shows some results returned from an API call to this site.
My first impulse was to use XMLHttpRequest, which won't return anything from the site -- a known problem due to the single origin policy.
I have no control over what the API returns -- it's XML or nothing.
I would very much like to keep my solution simple -- just HTML and javascript, no php, asp, c# or any of the rest of the alphabet soup of potential technologies out there. I'm also not running my own web server.
Is this even possible? Is there some simple solution I've overlooked?
(I should note here that I'm not trying to hack a website -- i've already got a legitimate account there, and they give me access to the data on the site via their API. I'm just trying to show their data in a more interesting way on my local machine.)
If the Web Service API you are trying to hit does not except a JSONP request or does not implement Cors headers then the only option is to create a Web Service of your own that is either on the same origin of your website or implements JSONP or Cors headers which will be used to hit the the Web Service with the desired data you are looking for. This is a very common problem when interacting with web services these days.
If API doesn't provide jsonp, or is not CORS enabled, there is one other javascript option and that's Yahoo's YQL service. It uses server proxy to get data in multiple formats including scraping html using Xpath selectors, grabbing xml or json or csv and returning data in either xml, json or jsonp format.
This means with javascript you get get data from virtually any API
YQL console link
I fear I may be trying to do something that certain security policies are specifically designed to forbid.
So there's a certain site with a certain AJAX-based chat application. It periodically polls the server and receives HTML fragments in return. I am looking to write an alternate mobile frontend that directly queries the existing backend using JS (i.e. does not use my server as a reflector).
Two main issues here that make this different from most such questions:
The server owner wouldn't mind me doing this but he's not going to go out of his way to help me, and so the format for talking with the server is not something I can change. That is, the server doesn't talk JSON let alone JSONP. It's HTML fragments but for my purposes that's essentially text.
I need to have the return value available to parse manually. It should not be automatically parsed/inserted/what-have-you through inclusion in the DOM or some other such mechanism.
If anyone has some advice on this matter, I would really appreciate it.
You could use a server side script to proxy it through your server.
You could use YQL as the middle man and use JSONP or CORS.
Tell the person on the other server to set up CORS for your server (tell them to add a header for each request, e.g. Access-Control-Allow-Origin: example.com).
could you create a php proxy,
ajax send url to fetch to local php(or other serverside script)
php uses curl to fetch that page and returns result.
Is it possible to open sockets and get data from web in JavaScript.
My aim is: to work with web data using JS.
I have looked for XmlHttp/AJAX solution, but I have found one note, that AJAX can be used only for calling localhost programs, which will be used as proxy and then only returns data from web.
It's because of sandbox model in browsers, which don't allow to work with sockets/other sites from JavaScript, and it works only in localhost.
Are there any solutions with JS to work with other world?
Your issue is due to cross domain request security, where you can't asynchronously get data from a domain which doesn't match the current host (this includes subdomains). You can however use jsonp, but this relies on the service that you're quering to supply the data in a jsonp format (a function call with the json data as a response).
If you have no control over the services you're requesting (which i assume you don't), you can use a javascript library, such as YUI or jQuery to perform the cross domain request for you (which typically uses Flash as the data proxy). However this will only work if you the site in question allows cross domain requests from your domain (defined in crossdomain.xml).
I have seen server side proxy workarounds for retrieving rss (xmls) from cross-domains. In fact this very question addressess my same problem but gives out a different solution.
I have a constraint of do not use a proxy to retrieve rss feeds. And hence the Google AJAX Feed API solution also goes out of picture. Is there a client-only workaround for this problem.
JSONP is the solution for requests that respond with JSON output. But here, I have RSS feeds which can respond with pure xml .
How do I solve the problem.
Use something like Yahoo! Pipes to serve as your proxy and translate the RSS XML into a JSON response.
Here is an article with instructions and code samples that explains how to do it: Yahoo Pipes--RSS without Server Side Scripts.
If you have control over both domains, you can try a cross-domain scripting library like EasyXDM, which wraps cross-browser quirks and provides an easy-to-use API for communicating in client script between different domains using the best available mechanism for that browser (e.g. postMessage if available, other mechanisms if not).
Caveat: you need to have control over both domains in order to make it work (where "control" means you can place static files on both of them). But you don't need any server-side code changes.
Another Caveat: there are security implications here-- make sure you trust the other domain's script!
Right now there really isn't a cross-platform solution for cross-site scripting. Do you have control or access to the RSS feeds? If so, why not simply respond with JSON and use JSONP?
There are other things coming down the pike with HTML5, like cross-site messaging (referred to as Cross-Document Messaging) that may be capable of delivering a payload of XML, but last time I checked, they hadn't even fully decided on a size limit for the messaging.
You can see the spec here: http://dev.w3.org/html5/spec/Overview.html#crossDocumentMessages
A solution for cross-domain calls without a server-side proxy is to use a SWF component.
You can script yourself one or use the readily available FLSend
The component uses ActionScript's URLRequest to call remote domains and ExternalInterface to communicate with the JavaScript methods that render your content.
The only way I can think of would be to embed a signed java applet on the webpage to retrive the xml and use javascript to interface with that. I'm not even 100% certain what the java security model is for that at present though but I think it would work.