Using Google Mini for a website that needs output from the Google Mini in a JSON/JSONP format for front-end querying purposes. Google Mini does publish an XML feed that could potentially be used by a middle process to convert to JSON/JSONP.
Can Google Search Appliance / Mini output to JSON/JSONP using a plug-in, modification to an XSLT template, or other unknown method?
Solutions discovered
Seems like digging through code.google.com reveals that GSA/Mini does not have a method to output in the JSON/JSONP format without using XSLT to create the feed. It is a matter of creating this XSLT to do the conversion.
Other option is to create a server-side script using available technology (PHP?) to retrieve the XML from GSA/Mini and convert the feed to JSONP upon request from the Front-end (AJAX). Downside is the increased overhead in this call.
Update 10/8/2010
Created a Google Mini frontend that generates a JSONP response.
I had the same problem trying to get JSON and JSONP. The solution worked in general on all other browsers, but IE. I could not change the mime-type to something other than text/html since that was global in GSA. Created an XSLT to solve that problem for browsers like IE which will not allow content to be returned due to a security feature ( IE – SEC7112 )
https://github.com/kjonnala/gsa_template
Hope that helps.
I'm working on approach #2 above. We have version 6.4 of the appliance and it provides JSON for the suggestion service (probably new since Riley's question). For example:
http://code.google.com/apis/searchappliance/documentation/64/xml_reference.html#RichOutputFormat
What I ran into was that cross-domain limitations prevented using the appliance JSON service on non-appliance website search forms, including those in subdomains of our shared high level domain.
I'm testing a simple Java servlet now that makes the suggest call. It takes incoming requests, gets the response from the appliance, outputs in JSONP and sends back to the calling page.
There are a couple references I've found related to proxying the suggest service:
http://groups.google.com/group/Google-Search-Appliance-Help/browse_thread/thread/72406a271a6d9917/
http://www.mcplusa.com/blog/2009/07/adding-google-search-appliance-suggest-search-to-your-existing-page/
http://sites.google.com/site/lightbends/gsa_qs_6_2
and some related to JSONP with jQuery:
http://devlog.info/2010/03/10/cross-domain-ajax/
http://www.ibm.com/developerworks/library/wa-aj-jsonp1/
JSON to JSONP in the servlet is quick and probably dirty in the approach I'm taking :)
It's not in production yet, so I can't speak to what we'll find in terms of performance and other challenges. I like that the service is exposed from a secondary server rather than directly from the appliance (allowing for throttling of incoming requests if needed, etc.) if the performance can keep up.
October 1, 2010 Update
Sorry - my post above applies only to the suggest service and not to the general search results. You could still take the XML response, process and wrap that in JSON/JSONP, but that would take longer I suspect. I was using the JSONP in the context of suggesting search queries with jQuery autocomplete, so having the quick response (as they type) is important.
Related
Here's my situation -- I have an account on a site that allows API access. So, theoretically, I could write a program to query the site via its API. I would like to build a local html page using javascript, that shows some results returned from an API call to this site.
My first impulse was to use XMLHttpRequest, which won't return anything from the site -- a known problem due to the single origin policy.
I have no control over what the API returns -- it's XML or nothing.
I would very much like to keep my solution simple -- just HTML and javascript, no php, asp, c# or any of the rest of the alphabet soup of potential technologies out there. I'm also not running my own web server.
Is this even possible? Is there some simple solution I've overlooked?
(I should note here that I'm not trying to hack a website -- i've already got a legitimate account there, and they give me access to the data on the site via their API. I'm just trying to show their data in a more interesting way on my local machine.)
If the Web Service API you are trying to hit does not except a JSONP request or does not implement Cors headers then the only option is to create a Web Service of your own that is either on the same origin of your website or implements JSONP or Cors headers which will be used to hit the the Web Service with the desired data you are looking for. This is a very common problem when interacting with web services these days.
If API doesn't provide jsonp, or is not CORS enabled, there is one other javascript option and that's Yahoo's YQL service. It uses server proxy to get data in multiple formats including scraping html using Xpath selectors, grabbing xml or json or csv and returning data in either xml, json or jsonp format.
This means with javascript you get get data from virtually any API
YQL console link
Is it possible to open sockets and get data from web in JavaScript.
My aim is: to work with web data using JS.
I have looked for XmlHttp/AJAX solution, but I have found one note, that AJAX can be used only for calling localhost programs, which will be used as proxy and then only returns data from web.
It's because of sandbox model in browsers, which don't allow to work with sockets/other sites from JavaScript, and it works only in localhost.
Are there any solutions with JS to work with other world?
Your issue is due to cross domain request security, where you can't asynchronously get data from a domain which doesn't match the current host (this includes subdomains). You can however use jsonp, but this relies on the service that you're quering to supply the data in a jsonp format (a function call with the json data as a response).
If you have no control over the services you're requesting (which i assume you don't), you can use a javascript library, such as YUI or jQuery to perform the cross domain request for you (which typically uses Flash as the data proxy). However this will only work if you the site in question allows cross domain requests from your domain (defined in crossdomain.xml).
I had a discussion with my colleague about Flash security. We're in the phase of planning some things for our web project that is using Flash plugin to display content. We need to dynamically pull settings for the Flash application from the server, using JSON.
Proposal that I offered was that we should save an extra HTTP request to pull the data file after the plugin is loaded and embed the JSON directly in the page containing the Flash plugin. Flash would fire a Javascript function that'd return the deserialized JSON data to it.
My colleague opposed this proposal with significant "security concerns".
I believe that there's literally zero difference between these two approaches besides the fact that his approach requires additional HTTP request. All of this is client/server and client should never be trusted. If I want to change the data that is in the JSON query, I can do that in both cases. File pull is little more difficult to hack though, but possible with custom HTTP proxy.
What are your thoughts?
There is no difference. Both can be fabricated.
if you really care that much about delivering original settings to the .swf:
don't use http - httpFox is a brilliant plugin - use a server that supports RTMP/RTMPE and NetConnection.call() to retrieve the data.
create an algorithm for validating original json so that your app won't work if the config doesn't pass the test.
after the config is loaded your swf might check the values with the server (not all at a time) and throw an error if something goes wrong
I have seen server side proxy workarounds for retrieving rss (xmls) from cross-domains. In fact this very question addressess my same problem but gives out a different solution.
I have a constraint of do not use a proxy to retrieve rss feeds. And hence the Google AJAX Feed API solution also goes out of picture. Is there a client-only workaround for this problem.
JSONP is the solution for requests that respond with JSON output. But here, I have RSS feeds which can respond with pure xml .
How do I solve the problem.
Use something like Yahoo! Pipes to serve as your proxy and translate the RSS XML into a JSON response.
Here is an article with instructions and code samples that explains how to do it: Yahoo Pipes--RSS without Server Side Scripts.
If you have control over both domains, you can try a cross-domain scripting library like EasyXDM, which wraps cross-browser quirks and provides an easy-to-use API for communicating in client script between different domains using the best available mechanism for that browser (e.g. postMessage if available, other mechanisms if not).
Caveat: you need to have control over both domains in order to make it work (where "control" means you can place static files on both of them). But you don't need any server-side code changes.
Another Caveat: there are security implications here-- make sure you trust the other domain's script!
Right now there really isn't a cross-platform solution for cross-site scripting. Do you have control or access to the RSS feeds? If so, why not simply respond with JSON and use JSONP?
There are other things coming down the pike with HTML5, like cross-site messaging (referred to as Cross-Document Messaging) that may be capable of delivering a payload of XML, but last time I checked, they hadn't even fully decided on a size limit for the messaging.
You can see the spec here: http://dev.w3.org/html5/spec/Overview.html#crossDocumentMessages
A solution for cross-domain calls without a server-side proxy is to use a SWF component.
You can script yourself one or use the readily available FLSend
The component uses ActionScript's URLRequest to call remote domains and ExternalInterface to communicate with the JavaScript methods that render your content.
The only way I can think of would be to embed a signed java applet on the webpage to retrive the xml and use javascript to interface with that. I'm not even 100% certain what the java security model is for that at present though but I think it would work.
I'm building an ASP.NET AJAX application that uses JavaScript to call web services to get its data, and also uses Silverlights Isolated Storage to cache the data on the client machine. Ultimately once the data is downloaded it is passed to JavaScript which displays in on the page using the HTML DOM.
What I'm trying to figure out is, does it make sense for me to make these Web Service calls in Silverlight then pass the data to JavaScript once it's loaded? Also, Silverlight will be saving the data to disk using Isolated Storage whether I call the Web Services with JavaScript or Silverlight. If I call the Web Services with JavaScript, the data will be passed to Silverlight to cache.
I've done some prototyping both ways, and I'm finding the performance to pretty much be the same either way. Also, one of the kickers that is pointing me towards using Silverlight for the whole client-side data access layer, is I need to have timers periodically check for updated data and download it to the cache so the JavaScript can loading when it needs to.
Has anyone done anything similar to this? If so, what are your experiences relating to performance with either the JavaScript or Silverlight method described?
Since Silverlight can handle JSON and XML based services, the format of the response is totally irrelevant. What you must consider, however, is the following:
1) Silverlight is approximately 1000 times faster than JavaScript
2) If your web service is natively SOAP based, Visual Studio can generate a proxy for you, so that you don't need to parse the SOAP message.
3) Silverlight has LINQ to XML and LINQ to JSON, which makes parsing both POX and JSON a breeze.
In a perfect world, I would go with Silverlight for the "engine", and fall back to JavaScript in case Silverlight is not available.
Greetings,
Laurent
Another thing to consider - getting your data in JSON format will be faster than XML and Web Services. JSON becomes a JavaScript object pretty quickly and doesn't have to be parsed like XML does. Personally, I'd go with JavaScript.
article: Speeding Up AJAX with JSON
Since JavaScript isn't multithreaded, I'm finding that using Silverlight to access/cache the data then pass it to JavaScript for display produces much better performance, while refraining from locking/freezing the browser so the user can keep doing stuff while the data loads.
Passing JSON-formatted data is in part faster because unlike an XML SOAP message, it doesn't require a SOAP header or any other miscellaneous info - it's just pure data. Thus, making the total size of the message smaller.