Parsing malformed JSON with Javascript - javascript

I want to parse this content using Javascript. The data looks like this:
{"ss":[["Thu","7:00","Final",,"BAL","19","ATL","20",,,"56808",,"PRE4","2015"],["Thu","7:00","Final",,"NO","10","GB","38",,,"56809",,"PRE4","2015"]]}
Every single tutorial online teaches you how to parse JSON using Twitter, but I am not quite sure how parsing with JSON works.
I would like to set this up on a website to view the NFL team scores for a fun project and a good learning experience about parsing JSON, as I could care less about Twitter stuff.
Is this possible? Any good tutorials to start with? Even some starting code?

Generally speaking, you can use JSON.parse to do this. However, that snippet that you have does not appear to be strictly valid JSON (as seen here: http://jsfiddle.net/yK3Gf/ and also by validating the source JSON here: http://jsonlint.com/).
So you will either need to parse it by hand, or get nfl.com to fix up their JSON.
As an alternative, their JSON does parse successfully when using eval(), so you could parse it with something like:
var parsedData = eval('(' + jsonData + ')');
...as shown here: http://jsfiddle.net/yK3Gf/1/
Though be aware that parsing JSON in this way is generally frowned upon (particularly when the data being parsed is being delivered by a third-party source), as it leaves you open to XSS attacks should the data happen to include any executable code inside of it.

I am in a similar position - non javascript expert working on a fun project to familiarize myself with javascript, ajax, and json.
I took three different steps to handle the problem. I welcome any feedback on improving the solution.
The first step is to query the nfl site to pull down the scores. Because the source of the json, the nfl site, is different from your site, you will have to work around the javascript security constraints against cross domain querying. I found this stackoverflow link to be a good reference. I used JSONP for the workaround. I used http://whateverorigin.org/ as the indirection site.
$.getJSON('http://whateverorigin.org/get?url=' + encodeURIComponent('http://www.nfl.com/liveupdate/scorestrip/scorestrip.json') + '&callback=?', handleQueryForScoresResult);
As others have pointed out, the nfl site returns invalid json data. The following sample line illustrates the problem:
["Sun","4:25","Final",,"TEN","7","MIN","30",,,"55571",,"REG5","2012"],
Notice the empty array element values (the repeated commas with no data in between). So in my json callback function, I corrected the data by adding empty strings (two double quotes) to repeated commas before calling jquery to parse the json data:
function handleQueryForScoresResult(data) {
var jsonStr = data.contents;
jsonStr = jsonStr.replace(/,,/g, ',"",');
jsonStr = jsonStr.replace(/,,/g, ',"",');
var scoresData = jQuery.parseJSON(jsonStr).ss;
.
.
.
}
Lastly, I created GameScores object to encapsulate the json data.
function GameScore(scoreData) {
this.scoreData = scoreData;
scoreData[2] = scoreData[2].toLowerCase();
scoreData[5] = parseInt(scoreData[5]);
scoreData[7] = parseInt(scoreData[7]);
}
function GameScore_getAwayTeam() { return this.scoreData[4]; }
function GameScore_getHomeTeam() { return this.scoreData[6]; }
function GameScore_isFinal() { return this.scoreData[2]=="final"; }
function GameScore_getHomeTeamScore() { return this.scoreData[7]; }
function GameScore_getAwayTeamScore() { return this.scoreData[5]; }
function GameScore_doesHomeTeamLead() { return this.scoreData[7]> this.scoreData[5]; }
function GameScore_doesAwayTeamLead() { return this.scoreData[5]> this.scoreData[7]; }
function GameScore_getWeekId() { return this.scoreData[12]; }
GameScore.prototype.getHomeTeam = GameScore_getHomeTeam;
GameScore.prototype.getAwayTeam = GameScore_getAwayTeam;
GameScore.prototype.isFinal = GameScore_isFinal;
GameScore.prototype.getHomeTeamScore = GameScore_getHomeTeamScore;
GameScore.prototype.getAwayTeamScore = GameScore_getAwayTeamScore;
GameScore.prototype.doesHomeTeamLead = GameScore_doesHomeTeamLead;
GameScore.prototype.doesAwayTeamLead = GameScore_doesAwayTeamLead;
GameScore.prototype.getWeekId = GameScore_getWeekId;
I only added a few accessors as I did not need most of the data. Your needs may vary.

We are using mootools for stuff like that, but you can do it it plain JavaScript as well: http://www.json.org/js.html.

Let's assume you already have a valid JSON String (jsonString) to parse. (If you don't know how to retrieve a String to parse using XMLHttpRequest from the given url you will have to look into that first.)
With plain JavaScript you will have to add Douglas Crockford's JSON library (or something similar) in order to provide a parsing Function if there is no native implementation:
var json = json_parse(jsonString) ;
link
With a JavaScript library like jQuery this would be
var json = $.parseJSON(jsonString) ;
Now, traversing the resultant JSON Object is a whole other issue, because you will have to know its structure before you can retrieve specific data.
In this particular case -- if it was indeed well formed -- you would have to do the following:
var data = json.ss ;
for(var i = 0 ; i < data.length ; i++) {
var entry = data[i] ;
var day = entry[0] ; //!! the Arrays seem to have a format where the first entry always contains the data and so forth...
/* ... */
// then do something with the data bits
}

Your main problem is that fact that the JSON your pulling in is malformed or not valid according to RFC 4627.
What you can do is grab the copy the JSON data and format it using this tool http://www.freeformatter.com/json-formatter.html
After you have the formatted version then you can use the jQuery ajax call
$.ajax({
url: "your-formatted.json",
dataType: 'json',
success: function (data) {
for (var i = 0; i < data.ss.length; i++) {
document.write("Day: " + data.ss[i][0]);
document.write("<br/>");
document.write("Time: " + data.ss[i][1]);
document.write("<br/><br/>");
}
}
});
You shouldn't actually use document.write in your application. This is only for example purpose of displaying the data.

For this specific issue (the empty indexes within the arrays from the JSON response) I did a regex replacement with a lookahead assertion. Considering that request contains the XMLHttpRequest:
request.responseText.replace(/,(?=,)/gm, ",\"\"")
This will turn ,, into ,"", and will also work in case there are more commas in sequence, so ,,, becomes ,"","",. You can use JSON.parse() afterwards.

This malformed JSON can be parsed by the dirty-json NPM package (I am the author).
You can test a demo of the parser here: https://rmarcus.info/dirty-json
The parser interprets the JSON in your original question as equivalent to the following valid JSON:
{
"ss": [
[
"Thu",
"7:00",
"Final",
"BAL",
"19",
"ATL",
"20",
"56808",
"PRE4",
"2015"
],
[
"Thu",
"7:00",
"Final",
"NO",
"10",
"GB",
"38",
"56809",
"PRE4",
"2015"
]
]
}

Related

Generating fully valid JSON with client-side JavaScript

So I am trying to create a JSON explorer / editor. I am able to parse the initial JSON into the div and format it how I like.
this is the function i use to loop through the initial JSON
_iterate(_tab, raw_json){
var tab = _tab;
tab++;
for(var key1 in raw_json){
var data_type = typeof raw_json[key1];
var d = String(raw_json[key1])
if(d == String){
d = "String";
}
if(d == Number){
d= "Number"
}
if(data_type == "object" || data_type == "array"){
this.input.append(`<json-tab tab-width="${tab}"></json-tab><div class="json-editor-input-container-2 -je-${data_type}">'<span class="-je-key">${key1}</span>' :{</div></br>`)
this._iterate(tab, raw_json[key1])
}else{
this.input.append(`<div class="json-editor-row"><json-tab tab-width="${tab}"></json-tab><div class="json-editor-input-container-2">'<span class="-je-key">${key1}<span>' : '<div class="json-editor-input -je-${data_type}" contenteditable="true" for="{key: '${key1}', data: '${d}'}"></div>', </div></br></div>`)
}
}
this.input.append(`<json-tab tab-width="${tab -1}"></json-tab>},</br>`)
}
in order to save the JSON I was going to retrieve the JSON from the text of the div using
getJSON(){
var json_text = this.input.text().slice(0, -1)
return JSON.parse(`"${json_text}"`)
}
right now this is able to be parse by JSON.parse(); but when i want to console.log(getJSON()[0]) this returns {
am i not formating the JSON correctly. a live example of this can be found here
First, your console.log result doesn't make sense. A parsed JSON object is now usable in JavaScript and, if has (only) properties x and y, would result in undefined when requesting property 0 as you have. It looks like your call to console.log was to a different (earlier?) version of the getJSON() function, where it returned the raw string, and in that case it makes sense that you're just retrieving the first character of the JSON text: "{".
But then, assuming the version of getJSON() as written, it would actually throw a parse exception:
VM1511:1 Uncaught SyntaxError: Unexpected token ' in JSON at position 1
Looking at your site, I was able to do, in the console:
jsonString = $('json-editor').text()
// value: "{'partName' : '', 'partRevision' : '', ..."
That is illegal JSON. JSON specifies (only) the quotation mark " for strings (Unicode/ASCII 0x22) on page 7 of its specification.
The fact that 'partName' is legal as a JavaScript string literal is irrelevant but perhaps confusing.
As a minor style point, simplify JSON.parse(`"${json_text}"`) to JSON.parse(json_text).
#BaseZen's answer was very helpful for me to understand what was going wrong with my code. My JSON was incorrectly formatted even though online linters say its correct. Along with what BaseZen pointed out, JSON.parse() will not work with trailing commas. To fix this:
_remove_trailing_commas(json_string){
var regex = /\,(?!\s*?[\{\[\"\'\w])/g;
return json_string.replace(regex, '');
}
I found this information at SO post JSON Remove trailiing comma from last object
SO user Dima Parzhitsky's answer was what helped me also figure out this question.

How do you parse Ajax/Jquery response data using regex?

my apologies if I used incorrect terms in my title. I am not sure how else to put this.
I am using Ajax to make a GET call and it responds with data in the form of html/txt/json etc. all in one file. I am trying to extract only whats in between [{ }].
I have successfully used the regex below to achieve this using https://regexr.com:
(?=\[\{).*(\}\])
On this response data returned by my GET call:
Lots of random text, random html, etc.
[{There is text in here hello world}]
Lots of random text, random html, etc.
As you can see this regex will properly extract this:
[{There is text in here hello world}]
This works great! But I just can't seem to figure out how to automatically parse the data after I get the response. I am currently trying:
$.ajax(settings).done(function (response) {
console.log(response.replace(/(?=\[\{).*(\}\])/));
});
But this isn't working. Am I going at this completely wrong? It's only outputting the full GET response and not the regex data. Any help is greatly appreciated.
If your response is a string, then you could use string.match() like this:
$.ajax(settings).done(function (response) {
console.log(response.match(/(?=\[\{).*(\}\])/));
});
You could try using the .indexOf() to find the indices of { and }:
var first_index = indexOf("{");
var second_index = indexOf("}"):
Once you find the indices, define a substring based on these indices:
var parsed_string = response.substring(first_index + 1, second_index);
You will need to add 1 to the first_index variable so that you do not get the "{" in the parsed_string variable.
jsfiddle
The more robust method would be to request the data in JSON, parse it accordingly then do javascript stuff with it:
var obj = JSON.parse(response);
// do js stuff with obj

Creating object from JSON and parsing JSON - different results

I am trying to use JQuery to parse some JSON being sent back from an AJAX call. It appears to be failing to parse, and JSLint also says it's invalid JSON.
However, if I create the object directly, it works and I am able to loop through it - please see below:
var json = {layers:[{layer1:[17,16,15,14,12]}]}
alert(json)// <- This works and output object Object
var somestring = "{layers:[{layer1:[17,16,15,14,12]}]}"
var parsing = JSON.parse(somestring)
alert(parsing) // <- this doesn't and breaks on parse
// The below code will work provided the parsing is commented out
json.layers.forEach(function (outerObj)
{
Object.keys(outerObj).forEach(function (key)
{
outerObj[key].forEach(function (item)
{
alert(item)
});
});
});
I'm struggling to wrap my head around why it won't parse, but appears to work.
Edit
I realise by wrapping quotes around layers and layer1 fixes it, just not sure why it works one way - but not the other.
there is a difference between javascript object and JSON object, all keys of JSON object must be quoted.
var somestring = "{layers:[{layer1:[17,16,15,14,12]}]}"// not a valid json to parse, it is a normal string, you can use JSON.stringify() to make it a valid json identifiable string.
so the correct JSON string will look like
var somestring = '{"layers":[{"layer1":[17,16,15,14,12]}]}';
var parsedJson = JSON.parse(somestring)
If you change sometring to some of the following examples, it will works.
var somestring = '{"layers":[{"layer1":[17,16,15,14,12]}]}'
var somestring = "{\"layers\":[{\"layer1\":[17,16,15,14,12]}]}"
The reason for this is, basically, that's how JSON was specified.
For further examples, take a look at w3schools
Best practice is to use JSON.stringify(Object) on one side, and JSON.parse(String) on the other. This will save you many hours of scratching your head over some niggling detail.
In your example, you could resolve the problem by
var somestring = JSON.stringify(json)
For future reference, however, JSON keys must be quoted, so your somestring should be written as:
var somestring = '{"layers":[{"layer1":[17,16,15,14,12]}]}'
Good luck!

How to parse JSON Array in java script

I am returning SQL Query result as a JSONArray to a JSP page. Now i want to show the data. I have written a code but it is working fine only for 23 objects in the JSONArray if JSONArray contains more the 23 object eval or JSON.parse function doesn't work. Please let me know how to solve this problem.
Below is the JS code i have written to iterate over this JSONArray.
var data = '<%=(JSONArray) request.getAttribute("resultArray")%>';
data = eval("(" + data + ")");
$(document).ready(function() {
var table = $('<table/>').appendTo($('#column'));
var rows = $('<tr/>').appendTo(table);
$.each(data, function(rowid, row) {
var rows = $('<tr/>').appendTo(table);
$.each(row, function(column, data) {
($('<td/>').text(data)).appendTo(rows);
})});
});
Just don't let JSP print it as a JS string syntax within quotes (which obviously needs to be parsed in order to get a JS object). Get rid of those quotes. JSON is already in proper JS object syntax. That's also all what "JSON" stands for.
var data = <%=request.getAttribute("resultArray")%>;
$(document).ready(function() {
// ...
});
By the way, using scriptlets in JSP is a poor practice. If you're on JSP 2.0 already (which is out for almost a decade already), just use EL.
var data = ${resultArray};
$(document).ready(function() {
// ...
});
Note, also here, just don't quote it. It becomes otherwise a JS string instead of a JS object.
Unrelated to the concrete problem, is it absolutely necessary to introduce the extra JSON/jQuery step here? Why don't you just use for example JSTL to let JSP generate the desired HTML in the server side instead of JS/jQuery in the client side?

Replace all strings "<" and ">" in a variable with "<" and ">"

I am currently trying to code an input form where you can type and format a text for later use as XML entries. In order to make the HTML code XML-readable, I have to replace the code brackets with the corresponding symbol codes, i.e. < with < and > with >.
The formatted text gets transferred as HTML code with the variable inputtext, so we have for example the text
The <b>Genji</b> and the <b>Heike</b> waged a long and bloody war.
which needs to get converted into
The <b>Genji</b> and the <b>Heike</b> waged a long and bloody war.
I tried it with the .replace() function:
inputxml = inputxml.replace("<", "<");
inputxml = inputxml.replace(">", ">");
But this would just replace the first occurrence of the brackets. I'm pretty sure I need some sort of loop for this; I also tried using the each() function from jQuery (a friend recommended I looked at the jQuery package), but I'm still new to coding in general and I have troubles getting this to work.
How would you code a loop which would replace the code brackets within a variable as described above?
Additional information
You are, of course, right in the assumption that this is part of something larger. I am a graduate student in Japanese studies and currently, I am trying to visualize information about Japenese history in a more accessible way. For this, I am using the Simile Timeline API developed by MIT grad students. You can see a working test of a timeline on my homepage.
The Simile Timeline uses an API based on AJAX and Javascript. If you don't want to install the AJAX engine on your own server, you can implement the timeline API from the MIT. The data for the timeline is usually provided either by one or several XML files or JSON files. In my case, I use XML files; you can have a look at the XML structure in this example.
Within the timeline, there are so-called "events" on which you can click in order to reveal additional information within an info bubble popup. The text within those info bubbles originates from the XML source file. Now, if you want to do some HTML formatting within the info bubbles, you cannot use code bracket because those will just be displayed as plain text. It works if you use the symbol codes instead of the plain brackets, however.
The content for the timeline will be written by people absolutely and totally not accustomed to codified markup, i.e. historians, art historians, sociologists, among them several persons of age 50 and older. I have tried to explain to them how they have to format the XML file if they want to create a timeline, but they occasionally slip up and get frustrated when the timeline doesn't load because they forgot to close a bracket or to include an apostrophe.
In order to make it easier, I have tried making an easy-to-use input form where you can enter all the information and format the text WYSIWYG style and then have it converted into XML code which you just have to copy and paste into the XML source file. Most of it works, though I am still struggling with the conversion of the text markup in the main text field.
The conversion of the code brackets into symbol code is the last thing I needed to get working in order to have a working input form.
look here:
http://www.bradino.com/javascript/string-replace/
just use this regex to replace all:
str = str.replace(/\</g,"<") //for <
str = str.replace(/\>/g,">") //for >
To store an arbitrary string in XML, use the native XML capabilities of the browser. It will be a hell of a lot simpler that way, plus you will never have to think about the edge cases again (for example attribute values that contain quotes or pointy brackets).
A tip to think of when working with XML: Do never ever ever build XML from strings by concatenation if there is any way to avoid it. You will get yourself into trouble that way. There are APIs to handle XML, use them.
Going from your code, I would suggest the following:
$(function() {
$("#addbutton").click(function() {
var eventXml = XmlCreate("<event/>");
var $event = $(eventXml);
$event.attr("title", $("#titlefield").val());
$event.attr("start", [$("#bmonth").val(), $("#bday").val(), $("#byear").val()].join(" "));
if (parseInt($("#eyear").val()) > 0) {
$event.attr("end", [$("#emonth").val(), $("#eday").val(), $("#eyear").val()].join(" "));
$event.attr("isDuration", "true");
} else {
$event.attr("isDuration", "false");
}
$event.text( tinyMCE.activeEditor.getContent() );
$("#outputtext").val( XmlSerialize(eventXml) );
});
});
// helper function to create an XML DOM Document
function XmlCreate(xmlString) {
var x;
if (typeof DOMParser === "function") {
var p = new DOMParser();
x = p.parseFromString(xmlString,"text/xml");
} else {
x = new ActiveXObject("Microsoft.XMLDOM");
x.async = false;
x.loadXML(xmlString);
}
return x.documentElement;
}
// helper function to turn an XML DOM Document into a string
function XmlSerialize(xml) {
var s;
if (typeof XMLSerializer === "function") {
var x = new XMLSerializer();
s = x.serializeToString(xml);
} else {
s = xml.xml;
}
return s
}
https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/String/replace
You might use a regular expression with the "g" (global match) flag.
var entities = {'<': '<', '>': '>'};
'<inputtext><anotherinputext>'.replace(
/[<>]/g, function (s) {
return entities[s];
}
);
You could also surround your XML entries with the following:
<![CDATA[...]]>
See example:
<xml>
<tag><![CDATA[The <b>Genji</b> and the <b>Heike</b> waged a long and bloody war.]]></tag>
</xml>
Wikipedia Article:
http://en.wikipedia.org/wiki/CDATA
What you really need, as mentioned in comments, is to XML-encode the string. If you absolutely want to do this is Javascript, have a look at the PHP.js function htmlentities.
I created a simple JS function to replace Greater Than and Less Than characters
Here is an example dirty string: < noreply#email.com >
Here is an example cleaned string: [ noreply#email.com ]
function RemoveGLthanChar(notes) {
var regex = /<[^>](.*?)>/g;
var strBlocks = notes.match(regex);
strBlocks.forEach(function (dirtyBlock) {
let cleanBlock = dirtyBlock.replace("<", "[").replace(">", "]");
notes = notes.replace(dirtyBlock, cleanBlock);
});
return notes;
}
Call it using
$('#form1').submit(function (e) {
e.preventDefault();
var dirtyBlock = $("#comments").val();
var cleanedBlock = RemoveGLthanChar(dirtyBlock);
$("#comments").val(cleanedBlock);
this.submit();
});

Categories

Resources