I am trying to parse a list of JSON files iteratively. Using PHP I have managed to compile the list of JSON files in a directly into a a single JSON object. Now I would like to parse each of these objects and output a property from each of them in HTML. I am sure that the JSON I am initially passing works. Any ideas? here is the error and the function.
$(document).ready(function(){
console.log("something")
for(var i = 2; i < Object.keys(jsTrips).length; i++){
var data_file = "http://localhos:8080/trips/" + jsTrips[i];
var currTrip = JSON.parse(data_file)
document.getElementsByClassName(i).innerHTML = currJSON.start_time;
}
console.log("finished")
});
ignore the missing t in localhost. The problem persists even when the typo is fixed
Thanks in advance!!
UPDATE:
The Javascript object jsTrips is formatted like this:
{2: name.json, 3:name.json, 4:name.json}
The the JSON files named in jsTrips are formatted like this:
{start_time: some start time, coords: [{lat: ##, long: ##}, {lat: ##, long: ##}...], end_time: some end time}
To address the error you see:
SyntaxError: JSON.parse: Unexpected character at Line 1 Column 1
The JSON.parse() method parses a JSON string, constructing the JavaScript value or object described by the string. You are feeding it a URL when it is expecting [ or { as the first character. So h causes a Syntax Error.
Assuming you define the Object jsTrips someplace in your code, and this is a more basic object, I would suggest the following:
$(function(){
console.log("Start jsTrips");
var i = 0;
$.each(jsTrips, function(k, v){
if(i >= 2){
$.getJSON("http://localhos:8080/trips/" + v, function(data_file){
$("." + i).html(data_file.start_time);
});
}
i++;
}
console.log("Finished");
});
This code also assumes there are HTML Elements with attributes like class="2". It would be better to update your Post with an example of the Objects and an example of the JSON being returned.
Now, if the Index of the Object is the class name, then it might look more like:
$.getJSON("http://localhos:8080/trips/" + v, function(data_file){
$("." + k).html(data_file.start_time);
}
Again, need to know what you're sending and what you expect to get back.
jQuery.getJSON() Load JSON-encoded data from the server using a GET HTTP request.
See More: https://api.jquery.com/jquery.getjson/
Update
Now using JSONP method via $.getJSON(), this will help address CORS:
$(function() {
var jsTrips = {
2: "file-1.json",
3: "file-2.json",
4: "file-3.json"
};
console.log("Start jsTrips");
$.each(jsTrips, function(k, v) {
var url = "http://localhos:8080/trips/" + v;
console.log("GET " + url);
$.getJSON(url + "?callback=?", function(json) {
$("." + k).html(json.start_time);
});
});
console.log("Finished");
});
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
As you can see, this builds the new URL from your jsTrips as expected. You can get the start_time from the JSON directly. You don't need to parse it when JSON is expected.
In regards to the new CORS issue, this is more complicated. Basically, you're not calling the same URI so the browser is protecting itself from outside code.
Cross-Origin Resource Sharing (CORS) is a mechanism that uses additional HTTP headers to tell a browser to let a web application running at one origin (domain) have permission to access selected resources from a server at a different origin. A web application makes a cross-origin HTTP request when it requests a resource that has a different origin (domain, protocol, and port) than its own origin.
An example of a cross-origin request: The frontend JavaScript code for a web application served from http://domain-a.com uses XMLHttpRequest to make a request for http://api.domain-b.com/data.json.
For security reasons, browsers restrict cross-origin HTTP requests initiated from within scripts. For example, XMLHttpRequest and the Fetch API follow the same-origin policy. This means that a web application using those APIs can only request HTTP resources from the same origin the application was loaded from, unless the response from the other origin includes the right CORS headers.
See more: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS & https://www.sitepoint.com/jsonp-examples/
If you are unable to change the port being used for the target JSON files, which I suspect is creating the CORS issue, you can consider using JSONP method versus GET method. Comment and let me know if this is the case, and I can update the answer. Example included in update.
Hope that helps.
This will not probably be a complete answer, because I don't really understand the question. But maybe it will help.
You told us you compiled in PHP one file from several JSON files, in general, if you have object in JSON file, it will look like { ...some key-values here... }, if you have array there, it will be [ ...some key-values here... ].
So when you compile several files with objects into one, you'll get {...some key-values here...} {...some key-values here...} and JSON does not know how to parse those, it will throw error:
console.log(JSON.parse('{"key": "value"}{"key": "value"}'))
This will work just fine, only one object there:
console.log(JSON.parse('{"key": "value"}'))
So, if for some reason you really need to compile several JSON files into one, there is a solution - to make such resulting file with new lines as separators. Than in JS you can split your file by new line, and parse each line without issues.
Like so:
const arrayOfJSONs = Array(10).fill(null).map((_,i) => JSON.stringify({key: i, keyXTen: i * 10}))
// then you join them in one big file with \\n new lines as separators
const oneBigFile = arrayOfJSONs.join("\n");
console.log("oneBigFile:\n", oneBigFile)
// on the client you get your one big file, and then parse each line like so
const parsedJSONs = oneBigFile.split("\n").map(JSON.parse)
console.log("parsedJSONs\n", parsedJSONs)
JSON.parse take string input
Javacript fetch
$(document).ready(function(){
console.log("something")
for(var i = 2; i < Object.keys(jsTrips).length; i++){
var data_file = "http://localhos:8080/trips/" + jsTrips[i];
fetch(data_file).then((res) => res.json())
.then((currJSON) => {
// document.getElementsByClassName(i).innerHTML = currJSON.start_time;
// update your DOM here
})
}
console.log("finished")
});
JQuery $.getJSON
$(document).ready(function(){
console.log("something")
for(var i = 2; i < Object.keys(jsTrips).length; i++){
var data_file = "http://localhos:8080/trips/" + jsTrips[i];
$.getJSON(data_file, (currJSON) => {
// document.getElementsByClassName(i).innerHTML = currJSON.start_time;
// Update DOM here
});
}
console.log("finished")
});
Related
I'm doing a little bit of reverse engineering on the Rapportive API in Gmail.
I make this request
import requests
url ='https://api.linkedin.com/uas/js/xdrpc.html'
r = requests.get(url)
print r.text
The response is an empty HTML file that has a lot of Javascript in it. On line 3661, it sets the RequestHeader for the subsequent call to Rapportive:
ak.setRequestHeader("oauth_token", ae);
Is there a way I can request that page and then return ae?
I think you can try:
Get the page as you already does;
Remove all non-javascript elements from the response page;
Prepend a javascript (described below) in the page's javascript to override some code;
Execute it with eval('<code>');
Check if the token has been set correctly;
I'm proposing the following code to override the XMLHttpRequest.setRequestHeader functionality to be able to get the token:
// this will keep the token
var headerToken;
// create a backup method
XMLHttpRequest.prototype.setRequestHeaderBkp =
XMLHttpRequest.prototype.setRequestHeader;
// override the "setRequestHeader" method
XMLHttpRequest.prototype.setRequestHeader = function(key, val)
{
if ('oauth_token' === key)
headerToken = val;
this.setRequestHeaderBkp(key, val);
}
If you are just interested in retrieving the token can't you just do a regex match:
var str = '<script>var a = 1;...ak.setRequestHeader("oauth_token", ae);...</script>';
var token = str.match(/setRequestHeader\("oauth_token",\s*([^)]+)/)[1];
Although this assumes ae is the actual string value. If it's a variable this approach wouldn't work as easily.
Edit: If it's a variable you could do something like:
str.replace(/\w+\.setRequestHeader\([^,]+,\s*([^)]+)\s*\);/, 'oauthToken = \1';
Before running the JavaScript returned from the page, then the global oauthToken (notice the missing 'var') will contain the value of the token, assuming the the evaluation of the code is run in the same scope as the caller.
Im having issues using trying to pull the first 15 words out of the file from the API. I have tried both as an XML and JSON and still seem to be getting this error:
XMLHttpRequest cannot load
No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access.
Im using the We feel fine API.
Here is my code:
<script type="text/javascript">
(function() {
var WeFeelAPI = "http://api.wefeelfine.org:8080/ShowFeelings?display=json&returnfields=feeling,conditions&limit=15";
$.getJSON( WeFeelAPI,function (json){
var feel = json.results[15];
console.log('Our feelings : ', feel);
});
})();
</script>
Any help would be appreciated i'm very new to all this, thanks
Reading up on the We Feel Fine APIs, it doesn't seem like they support JSONP, or even JSON from what I can see.
The issue preventing you from calling it is known as the Same Origin Policy. It prevents a domain from making an illegal request to another domain because of the security concerns it poses. You can read on it here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Same_origin_policy_for_JavaScript
JSONP (JSON with Padding) is a way for sites to work around it by loading the response a an external script that then triggers a callback function to validate the response content. This actual provides good info on SOP and JSONP: http://www.codeproject.com/Articles/42641/JSON-to-JSONP-Bypass-Same-Origin-Policy.
Unfortunately, the API you're using doesn't look to support JSONP so it would require the proxy approach. There is a clever/creative/maybe hackish(opinion) approach using something called Yahoo Query Language (YQL). YQL allows you to perform a x-domain request by using Yahoo's query service as the "proxy." You pass a request with a SQL-like query to it and Yahoo handles the JSONP approach. You can read about that here: http://developer.yahoo.com/yql/ (sorry for all the reading.)
And now for some code to demonstrate this. Note the QUERY being used to retrieve your XML and the fact that it must be encoded for URI use:
(function () {
var url = 'http://api.wefeelfine.org:8080/ShowFeelings?display=xml&returnfields=feeling,conditions&limit=15'
// using yahoo query
var query = 'http://query.yahooapis.com/v1/public/yql?q=' + encodeURIComponent('select * from xml where url="' + url + '"') +
'&format=json&callback=?';
// make request via YQL and show data
$.getJSON( query, function(data) {
console.log(data);
// yql returns "results" in "query" from data
console.log(data.query.results);
});
})();
Play with the fiddle: http://jsfiddle.net/Ty3y2/
This same approach can actually be used to load HTML, and in fact is probably used for that more. The key is "select * from xml where..." which tells it to select everything inside the XML element found at the requested URL. Remember that XML data has a XML element at the root. Most times you will see this as "select * from html where..." because a typical web request returns HTML which is a HTML element at the root.
I have used this approach for a couple projects, though most of mine use a proxy via PHP or C#. However, I have had good success with this and it's useful when you don't want/need to put together a proxy for it.
Here's a simple PHP proxy you can run along-side your page with the JavaScript
<?php
// Saved as ShowFeelings-proxy.php
$options = array_merge($_GET, ['display' => 'xml']);
// if you don't have PHP 5.4+, you need to use the legacy array literal syntax, eg
// array('display' => 'xml')
$uri = 'http://api.wefeelfine.org:8080/ShowFeelings?' . http_build_query($options);
$xml = simplexml_load_file($uri);
// assuming you'd rather work with JSON (I know I would)
$data = [];
foreach ($xml->feeling as $feeling) {
$entry = [];
foreach ($feeling->attributes() as $attr => $val) {
$entry[$attr] = (string) $val;
}
$data[] = (object) $entry;
}
header('Content-type: application/json');
echo json_encode($data);
exit;
Then in your JavaScript...
+function($) {
var url = 'ShowFeelings-proxy.php',
options = {
'returnfields': 'feeling,conditions',
'limit': 15
};
$.getJSON(url, options, function(data) {
var feeling = data[14]; // array is zero-based
console.log(feeling);
});
}(jQuery);
I am trying to parse a sequence of html pages using python, I am having trouble grabbing the pages in iterative fashion. The link to the web page.
Milano Library
After peeking through the source, I found a function that responds to the click event on button element for the next page.
function SaltaAPagina() {
var CalcPag = VAIAPAGINA.value;
if (CalcPag > 0) {
CalcPag=CalcPag;
}
else {
CalcPag="1";
}
document.location = "/OPACMI01/cat/SDW?W=CODICE_BIBLIO+%3D+%27LO1+01%27+AND+EDITORE+PH+WORDS+%27sonzogno%27+AND+DATA_PUBBLICAZIONE+%3C+1943+ORDER+BY+ORDINAMENTO/Ascend&M=" + CalcPag + "&R=Y";
}
I know that I can encode parameters using pythons urllib2 module using the urlencode method. But I am not sure what I should be including as a parameter
lomba_link='http://www.biblioteche.regione.lombardia.it/OPACMI01/cat/SDW?W%3DCODICE_BIBLIO+%3D+%27LO1+01%27+AND+EDITORE+PH+WORDS+%27sonzogno%27+AND+DATA_PUBBLICAZIONE+%3C+1943+ORDER+BY+ORDINAMENTO/Ascend%26M%3D1%26R%3DY'
params = urllib.urlencode([('CalcPag',4)])
# this has not worked.
req = urllib2.Request(lomba_link)
print req
response = urllib2.urlopen(req,params)
html_doc = response.read()
What am I missing here?
Thanks
The javascript function you posted is passing several parameters to the target page:
document.location = "/OPACMI01/cat/SDW" + // This is the path of the page
"?W=CODICE_BIBLIO+%3D+%27LO1+01%27+AND+EDITORE+PH+WORDS+%27sonzogno%27+AND+DATA_PUBBLICAZIONE+%3C+1943+ORDER+BY+ORDINAMENTO/Ascend" + // The first parameter
"&M=" + CalcPag + // The second parameter
"&R=Y"; // The third parameter
In your code, you've encoded all of the & and = symbols in the URL, so you're passing a single, long parameter with no value - changing those symbols back to what they were in the javascript function should do the trick.
lomba_link='http://www.biblioteche.regione.lombardia.it/OPACMI01/cat/SDW'
params = urllib.urlencode([
('W', 'CODICE_BIBLIO+%3D+%27LO1+01%27+AND+EDITORE+PH+WORDS+%27sonzogno%27+AND+DATA_PUBBLICAZIONE+%3C+1943+ORDER+BY+ORDINAMENTO/Ascend'),
('M', 4),
('R', 'Y')
])
It's much easier to work with the brilliant requests library, rather than the urllib2 library...
In regards to urllib2.urlopen the params is for POST requests. Unfortunately you need to append the query string to the url to make a GET request.
eg:
req = urllib2.urlopen(req + '?' + params)
With requests, this would be much simpler:
page = requests.get(some_url, params={'CalcPag': '4'})
I am trying to read the finance info from the google page into a json object.
Code is below:
try {
$.getJSON("http://finance.google.com/finance/info?client=ig&q=NSE:GOLDBEES&jsoncallback=?",function(data){
alert(data);//var jsondata = data;
//jsonobj = $.parseJSON(jsondata);
//alert(jsonobj[0].id);
});
} catch(e) {
alert(e.toString());
}
However I keep getting this error all the time on firebug
invalid label
"id": "4052464"
Is there any way this info can be read. My ultimate goal is to create a windows 7 gadget that doesnt use server side scripting and can be used from any Windows 7 system.
Appreciate all the help.
John
Response isn't valid JSON (response is prefixed with //), so jQuery won't be able to parse it correctly anyway.
To solve change &jsoncallback=? to &callback=?
so
$.getJSON("http://finance.google.com/finance/info?client=ig&q=NSE:GOLDBEES&callback=?", function(data) {
alert(data)
});
The response from Google has two leading /'s, making the response invalid JSON... for some reason.
Because of this, you cannot use jQuery.getJSON, as it expects a JSON response. Instead, you should use jQuery.get, and parse the JSON yourself after removing the two leading slashes.
jQuery.get('http://finance.google.com/finance/info?client=ig&q=NSE:GOLDBEES&jsoncallback=?', function (string) {
var validJson = string.slice(2);
var obj = jQuery.parseJSON(validJSON);
// use obj
});
Two additional points:
No JSONP is being used, so you don't need the jsoncallback=? in your request URL
The Windows Sidebar has been retired, so you cannot publish you finished gadget to the official gallery.
Question:
I'm trying to use JSON accross domains, but all i find is JSON parsers, which I don't need...
I've read that it's possible to do cross-domain requests with JSON,
but so far, all I see is implementations that use XMLHttpRequest...
- which means you can't use cross-domain requests, at least not outside IE 8...
I've been on http://www.json.org/, but all I find is either parsers or useless.
The best I've found with google so far is
http://devpro.it/JSON/files/JSONRequest-js.html
but this is rather a mess, doesn't work cross domain, and intra-domain neither - or rather not at all...
var the_object = {};
var http_request = new XMLHttpRequest();
http_request.open( "GET", url, true );
http_request.onreadystatechange = function () {
if ( http_request.readyState == 4 && http_request.status == 200 ) {
the_object = JSON.parse( http_request.responseText );
}
};
http_request.send(null);
What you can do cross-domain is inject a script include:
var s = document.createElement('script');
s.src = 'http://someotherdomain/getMeMyJs.aspx?parameter=value';
s.onload = someOptionalCallback;
s.type = 'text/javascript';
if(document.getElementsByTagName('head').length > 0)
document.getElementsByTagName('head')[0].appendChild(s);
Now, the code returned by that request will be executed immediately. If you want for that to interact with your code, you can make sure that it's being returned with all data wrapped in a function call:
jsonCallback({ object: json, whatever: value });
You can use that to build APIs, where you pass the name of a callback function as a request querystring parameter. Here's an example of such an API
JSON is just a serialization method. There is no relation whatsoever between the method of the serialization and the question of whether or not the browser will try to stop you from accessing data across domains. (This explains why you are only finding parsers - there is nothing to JSON, except encoding and decoding it).
XMLHTTPRequest is just named XML HTTPRequest. It doesn't really have anything to do with XML. It can be used to send text data, data encoded in JSON, or any others serialization method.
There are several methods to access data cross domain. one described in David Hedlund's answer. Others can be found in answers to similar questions (see here and here).