I have been struggling with a site I am scrapping using scrappy.
This site, returns a series of Javascript variables (array) with the products data.
Example:
datos[0] = ["12345","3M YELLOW CAT5E CABLE","6.81","1","A","N","N","N","N","N",0,0,0,0,0,"0","0","0","0","0","P","001-0030","12","40K8957","28396","250","Due: 30-12-1899",0.0000,1,"",\'\'];
datos[1] = ["12346","3M GREEN CAT5E CABLE","7.81","1","A","N","N","N","N","N",0,0,0,0,0,"0","0","0","0","0","P","001-0030","12","40K8957","28396","250","Due: 30-12-1899",0.0000,1,"",\'\'];
...
So on...
Fetching the array into a string with scrapy was easy, since the site response prints the variables.
The problem is I want to transform it into Json so I can process it and store it in a database table.
Normally I would use Javascript's function Json.stringify to convert it to Json and post it in PHP.
However when using Python's json.loads and even StringIO I am unable to load the array into json.
Probably is a format error, but I am unable to identify it, since I am not expert in Json nor Python.
EDIT:
I just realize since scrapy is unable to execute Javascript probably the main issue is that the data is just a string. I should format it into a Json format.
Any help is more than welcome.
Thank you.
If you wanted to take an array and create a json object, you could do something like this.
values = ["12345","3M YELLOW CAT5E CABLE","6.81","1","A","N","N","N","N","N",0,0,0,0,0,"0","0","0","0","0","P","001-0030","12","40K8957","28396","250","Due: 30-12-1899",0.0000,1]
keys = [x for x in range(len(values))]
d = dict(zip(keys, values))
x = json.dumps(d)
There is a section in the scrapy doc to find various ways to parse the JavaScript code. For your case, if you just need to have it in an array, you can use the regex to get the data.
Since the website you are scraping is not present in the question, I am assuming this would be a more straightforward way to get it, but you could use whichever way seems suitable.
i want to parse any json in javascript (react more exactly) and put it in a big object (to show it on UI and modify before put it again in json)
But i don't know how to exactly do that !
I tried with
var data = require("./test.json");
var jsonString = JSON.stringify(data);
But after that i don't know how to parse correctly my file,i can't name programaticaly my properties in my array so impossible to parse manually..
I precise i don't know json content so i can't do pre-build object.
Thank you for you answers ! :)
I'm working with Google's civic info API and I'm trying to figure out how to convert a JSON repsonse into a variable javascript object (we'll call that object civicInfoObject). Then I want to display to contents of that object onto the page with HTML. I'm new to javascript and haven't been able to successfully come up with a solution or find an answer.
It's a relative big project for me so I am working in bite-size pieces. I copied and pasted the JSON response to my query and created a new HTML document with it. So basically my goal right now is to store this static JSON string in a variable (civicInfoResponse), convert it into an object and store that in a variable (civicInfoObject), and then display the contents of that object onto a page with HTML.
Here is my strategy thus far:
var civicInfoResponse = "copy-and-pasted string
that spans
multiple lines"
var civicInfoObject = JSON.parse(civicInfoResponse);
document.getElementById("Info").innerHTML = civicInfoObject;
Here is the fiddle I created so you can see the JSON response in its entirety: https://jsfiddle.net/km38o815/2/
The code I wrote might be completely wrong, I'm not sure. I'm having a hard time with my limited knowledge about javascript. Can anyone help me out or at least point me in the right direction? It would be very much appreciated. Thanks!
I updated your fiddle. If you have the JSON you should need quotes around it? is this what you are looking for?
document.getElementById("Info").innerHTML = JSON.stringify(civicInfoResponse);
https://jsfiddle.net/km38o815/7/ oops this should work now.
This question is related to: JavaScript string with new line - but not using \n
In other words, if you need the JSON to span multiple lines, use \ before starting a new line:
var civicInfoResponse = "copy-and-pasted string \
that spans \
multiple lines"
civicInfoResponseshould contain:
copy-and-pasted string that spans multiple lines
Ok. So the title sounds confusing but that's probably the best way to describe it.
I'm using the TextExt jQuery plugin to create a tag list in a form. However the plugin creates an array in a string when submitted. eg. "["this","that","other"]".
How can I convert this to an actual array? ["this","that","other"]
Cue really simple answer I completely overlooked.
Since you are using jQuery you can just pass it through jQuery.parseJSON like this:
var array = jQuery.parseJSON('["this","that","other"]')
Looks like JSON. So then use JSON.parse(), or better jQuery.parseJSON() when you already have jQuery inlcuded.
I need a simple example that has a json data and I have to parse it and render in html. Can someone suggest me a simple way to do this?
You can create a string from an object in JavaScript like this...
var jsonString = JSON.stringify(myJsonObject);
Then you can use that string to apply to a html element. For example...
document.getElementById('myDivID').innerText = jsonString;
With JQuery you can update a DIV with the following...
$("#MyDiv").html(jsonString);
I'm not entirely sure what you are asking. You do not have to use jQuery specifically to parse the object. All you need is standard JavaScript.
Given a JSON string, you can parse it into a JavaScript object using the JSON library
var myJSONObject = JSON.parse(myJSONString);
Or into a string from an object:
var myJSONString= JSON.stringify(myJSONObject);
If you are looking for the individual items of a JSON structure, then you can use a for loop:
for (var key in myJSONObject){
alert(myJSONObject[key]);
}
I've alerted myJSONObject[key] in the above, however, you can do what you want with it.
You would use jQuery to select out the container into which you wanted the info to be displayed, as suggested in usefan's answer.