"Fixing" JSON coming out of MySQL - javascript

I'm fetching JSON code stored in MySQL and it has extra slashes, which I have to remove in order to parse it in JavaScript, after I print it on the page. Right now I'm doing the following:
$save = str_replace("\n", "<br>", $save); // Replace new line characters with <br>
$save = str_replace('\\"', '"', $save); // top-level JSON
$save = str_replace('\\\\"', '\"', $save); // HTML inside top level JSON
$save = str_replace('\\\\\\\\\\"', '\\\\\"', $save); // HTML inside second level JSON
Here is an example JSON code, as it comes out from MySQL:
{\"id\":2335,\"editor\":{\"selected_shape\":\"spot-7488\"},\"general\":{\"name\":\"HTML Test\",\"shortcode\":\"html-test\",\"width\":1280,\"height\":776},\"spots\":[{\"id\":\"spot-7488\",\"x\":9.9,\"y\":22.6,\"default_style\":{\"use_icon\":1},\"tooltip_content\":{\"content_type\":\"content-builder\",\"plain_text\":\"<p class=\\\"test\\\">Test</p>\",\"squares_json\":\"{\\\"containers\\\":[{\\\"id\\\":\\\"sq-container-293021\\\",\\\"settings\\\":{\\\"elements\\\":[{\\\"settings\\\":{\\\"name\\\":\\\"Paragraph\\\",\\\"iconClass\\\":\\\"fa fa-paragraph\\\"},\\\"options\\\":{\\\"text\\\":{\\\"text\\\":\\\"<p class=\\\\\\\"test\\\\\\\">Test</p>\\\"}}}]}}]}\"}}]}
And here is how it's supposed to look in order to get parsed correctly (using jsonlint.com to test):
{"id":2335,"editor":{"selected_shape":"spot-7488"},"general":{"name":"HTML Test","shortcode":"html-test","width":1280,"height":776},"spots":[{"id":"spot-7488","x":9.9,"y":22.6,"default_style":{"use_icon":1},"tooltip_content":{"content_type":"content-builder","plain_text":"<p class=\"test\">Test</p>","squares_json":"{\"containers\":[{\"id\":\"sq-container-293021\",\"settings\":{\"elements\":[{\"settings\":{\"name\":\"Paragraph\",\"iconClass\":\"fa fa-paragraph\"},\"options\":{\"text\":{\"text\":\"<p class=\\\"test\\\">Test</p>\"}}}]}}]}"}}]}
Please note that I have HTML code inside JSON, which is inside another JSON and this is where it gets a bit messy.
My question - is there a function or library for PHP (for JS will work too) which covers all those corner cases, because I'm sure someone will find a way to break the script.
Thanks!

The short answer, which is woefully inadequate, is for you to use stripslashes. The reason this answer is not adequate is that your JSON string might have been escaped or had addslashes called on it multiple times and you would have to call stripslashes precisely once for each time this had happened.
The proper solution is to find out where the slashes are being added and either a) avoid adding the slashes or b) understand why the slashes are there and respond accordingly. I strongly believe that the process that creates that broken JSON is where the problem lies.
Slashes are typically added in PHP in a few cases:
magic_quotes are turned on. This is an old PHP feature which has been removed. The basic idea is that PHP used to auto-escape quotes in incoming requests to let you just cram incoming strings into a db. Guess what? NOT SAFE.
add_slashes has been called. Why call this? Some folks use it as an incorrect means of escaping data before sticking stuff in a db. Others use it to keep HTML from breaking when echoing variables out (htmlspecialchars should probably be used instead). It can also come in handy in a variety of other meta situations when you are defining code in a string.
When escaping data input. The most common escaping function is mysqli_real_escape_string. It's very important to escape values before inserting them in a db to prevent sql injection and other exploits but you should never escape things twice.
So there's a possibility that your code is double-escaping things or that addslashes is getting called or something like magic_quotes is causing the problem, but I suspect it is another problem: some JS code might be supplying this JSON not as a proper JSON string, but one that has been escaped so to define a string within javascript.
If you take your example JSON string above, and slap some quotes around it:
var myJSON = "<put your string here>";
then SURPRISE your javascript is not broken and the var myJSON contains a string that is actually valid JSON and can be parsed into an a valid JSON object:
var myJSON = "{\"id\":2335,\"editor\":{\"selected_shape\":\"spot-7488\"},\"general\":{\"name\":\"HTML Test\",\"shortcode\":\"html-test\",\"width\":1280,\"height\":776},\"spots\":[{\"id\":\"spot-7488\",\"x\":9.9,\"y\":22.6,\"default_style\":{\"use_icon\":1},\"tooltip_content\":{\"content_type\":\"content-builder\",\"plain_text\":\"<p class=\\\"test\\\">Test</p>\",\"squares_json\":\"{\\\"containers\\\":[{\\\"id\\\":\\\"sq-container-293021\\\",\\\"settings\\\":{\\\"elements\\\":[{\\\"settings\\\":{\\\"name\\\":\\\"Paragraph\\\",\\\"iconClass\\\":\\\"fa fa-paragraph\\\"},\\\"options\\\":{\\\"text\\\":{\\\"text\\\":\\\"<p class=\\\\\\\"test\\\\\\\">Test</p>\\\"}}}]}}]}\"}}]}";
console.log(JSON.parse(myJSON)); // this is an actual object
The key here is to examine the point of entry where this JSON arrives in your system. I suspect some AJAX request has created some object and rather than sending valid JSON Of that object, it is sending instead an escaped string of a JSON object.
EDIT:
Here's a simple example of what happens when you have too many encodings. Try running this JS in your browser and observe the console output:
var myObj = {"key":"here is my value"};
console.log(myObj);
var myJSON = JSON.stringify(myObj);
console.log(myJSON);
var doubleEncoded = JSON.stringify(myJSON);
console.log(doubleEncoded);

Related

Javascript crashes on special characters from query string

To use this value in my TypeScript I am getting it from my query string like this:
var UserName = #Request.QueryString["UserName"];
But I get a Unexpeted Identifier error on it because if in DevTool if I go to where it breaks that query string has a value like this:
var UserName = ANT -- ANT 37690 / THIRD PARTY
So is there a way to do some kind of sanitation on it so it wouldn't crash? I guess there are illegal characters in that value for JS?
The error has nothing to do with "special" characters, but with the fact that the right side of the assignment - unwrapped in quotes - contains what js engine views as unknown identifier[s].
One way to properly format data that becomes part of javascript code is to use JavaScriptSerializer class from System.Web.Script.Serialization namespace.
var UserName = #new System.Web.Script.Serialization.JavaScriptSerializer().Seria‌​lize(Request.Query‌​St‌​ring["UserName"]);
The shorter version of this for a string is:
var UserName = "#System.Web.HttpUtility.JavaScriptStringEncode(Request.Query‌​St‌​ring["UserName"])";
or overloaded version that wraps the result in double quotes:
var UserName = #System.Web.HttpUtility.JavaScriptStringEncode(Request.Query‌​St‌​ring["UserName"], true);
You need to include quotes for the value.
var UserName = "#(Request.QueryString["UserName"])";
Otherwise the name will come through verbatim in your code and cause the problems you are seeing.
There is no need to protect against an attack vector here as the user can alter the page as they see fit at any time with a user script, and the QueryString is entered by them and only seen as a result by them in this scenario.
If there was a need to scrub the user input, it should be done prior to it actually reaching the view on server side. However, if still concerned about scrubbing output into a view in this type of scenario in general, it would be prudent to include an encode from razor's library.
var sanitizedJsVariable = "#System.Web.HttpUtility.JavaScriptStringEncode(model.VariableFromServer)";

UTF-8 symbol is converted when inserted to dom

I have a following problem, i am building app that uses data stream from ajax calls, the data that is coming is therefore escaped inside json string.
example: 1°Set
When i insert that data to DOM it is being converted like this: 1°Set
I dont use any libraries like jQuery, pure Javascript.
I tried to store converted name also in another place but i cannot seem to convert it manually, i tried following functions:
var test = function(str) {
console.log(unescape(encodeURIComponent(str)) );
console.log(decodeURIComponent(escape(str)) );
};
test('1°Set');
It stays the same, does anyone have an idea how to convert it to a DOM like version?
I have a following problem, i am building app that uses data stream from ajax calls, the data that is coming is therefore escaped inside json string.
example: 1°Set
Sounds like you're having a problem because your backend serves a JSON that looks like:
{
"something": "1°Set"
}
Instead of a string "1°Set", you're serving HTML source code that amounts to "1°Set". This looks very unnecessary. I cannot see a good reason of using HTML escaping inside JSON, unless you actually want your JSON to actually contain a part of HTML source (with formatting and everything), rather than just a string.
My suggestion: Let's keep it simple and instead serve something like:
{
"something": "1°Set"
}
or equivalently escape it properly using JSON syntax:
{
"something": "1\u00b0Set"
}
Now you'll JavaScript will receive a plain string that can be easily displayed, for example inside element.textContent or element.value or anywhere else. You won't even need any conversions.

Difference between angular.fromJson and $scope.$eval when applied to JSON string

In my angularjs apps, I usually parse a JSON string by using angular.fromJson, like so:
var myObject=angular.fromJSON(jsonString);
However, it seems that I would obtain the same result by using $scope.$eval:
var myObject=$scope.$eval(jsonString);
See this fiddle
Or by using vanilla javaScript, like so:
var myObject=JSON.parse(jsonString);
Is there any particular reason to use angular.fromJSON rather than JSON.parse?
Is there any possible issue when using $scope.$eval to parse a JSON string?
Check out the source code:
function fromJson(json) {
return isString(json)
? JSON.parse(json)
: json;
}
They're just passing through to JSON.parse.
As for $eval it shells out to $parse:
// $scope.$eval source:
$eval: function(expr, locals) {
return $parse(expr)(this, locals);
},
$parse source is too long to post, but it is essentially capable of converting inline (stringified) objects to real Objects and so it makes sense that in this case, it will actually convert your JSON as well.
(I did not know this until reading through the $parse source just now.)
Is there any particular reason to use angular.fromJSON rather than JSON.parse?
Nope, not really. Although they do check to you to ensure that you don't double-parse a JSON string, like so:
var jsonString = '{"foo":"bar"}';
var json = JSON.parse(jsonString); // Parsing once is good :)
JSON.parse(json); // Parsing twice is bad :(
Is there any possible issue when using $scope.$eval to parse a JSON string?
I don't think so off the top of my head, other than that you're doing more work than is necessary. So if you know you have JSON, there's no reason to use the heavier $parse function.
The above answer is almost correct. However, there is a potential issue with using $scope.$eval() to parse a JSON string, which does not exist with either JSON.parse() or angular.fromJson(): security. Angular allows an expression to contain complex JavaScript including function calls, conditionals with ?:, variable assignments, and so on. All of these are recognised and processed if you use $scope.$eval(), even if they were added by a malicious end-user.
JSON does not support any of those more complex JavaScript features, nor anything else potentially "dangerous". If you use a true JSON parser like JSON.parse() or angular.fromJson(), there is no chance of malicious code being injected and executed.
Since Angular expressions are isolated and evaluate only in the current $scope, the risk of code injection is somewhat mitigated - $scope.$eval() is far less dangerous than JavaScript's native eval() for parsing JSON. However there is still no reason to use either function for this purpose, since there is a potential security risk and using a proper JSON parser is likely to be faster.

remove double quotes from Json return data using Jquery

I use JQuery to get Json data, but the data it display has double quotes. It there a function to remove it?
$('div#ListingData').text(JSON.stringify(data.data.items[0].links[1].caption))
it returns:
"House"
How can I remove the double quote? Cheers.
Use replace:
var test = "\"House\"";
console.log(test);
console.log(test.replace(/\"/g, ""));
// "House"
// House
Note the g on the end means "global" (replace all).
For niche needs when you know your data like your example ... this works :
JSON.parse(this_is_double_quoted);
JSON.parse("House"); // for example
The stringfy method is not for parsing JSON, it's for turning an object into a JSON string.
The JSON is parsed by jQuery when you load it, you don't need to parse the data to use it. Just use the string in the data:
$('div#ListingData').text(data.data.items[0].links[1].caption);
Someone here suggested using eval() to remove the quotes from a string. Don't do that, that's just begging for code injection.
Another way to do this that I don't see listed here is using:
let message = JSON.stringify(your_json_here); // "Hello World"
console.log(JSON.parse(message)) // Hello World
I also had this question, but in my case I didn't want to use a regex, because my JSON value may contain quotation marks. Hopefully my answer will help others in the future.
I solved this issue by using a standard string slice to remove the first and last characters. This works for me, because I used JSON.stringify() on the textarea that produced it and as a result, I know that I'm always going to have the "s at each end of the string.
In this generalized example, response is the JSON object my AJAX returns, and key is the name of my JSON key.
response.key.slice(1, response.key.length-1)
I used it like this with a regex replace to preserve the line breaks and write the content of that key to a paragraph block in my HTML:
$('#description').html(studyData.description.slice(1, studyData.description.length-1).replace(/\\n/g, '<br/>'));
In this case, $('#description') is the paragraph tag I'm writing to. studyData is my JSON object, and description is my key with a multi-line value.
You can simple try String(); to remove the quotes.
Refer the first example here: https://www.w3schools.com/jsref/jsref_string.asp
Thank me later.
PS: TO MODs: don't mistaken me for digging the dead old question. I faced this issue today and I came across this post while searching for the answer and I'm just posting the answer.
What you are doing is making a JSON string in your example. Either don't use the JSON.stringify() or if you ever do have JSON data coming back and you don't want quotations, Simply use JSON.parse() to remove quotations around JSON responses! Don't use regex, there's no need to.
I dont think there is a need to replace any quotes, this is a perfectly formed JSON string, you just need to convert JSON string into object.This article perfectly explains the situation : Link
Example :
success: function (data) {
// assuming that everything is correct and there is no exception being thrown
// output string {"d":"{"username":"hi","email":"hi#gmail.com","password":"123"}"}
// now we need to remove the double quotes (as it will create problem and
// if double quotes aren't removed then this JSON string is useless)
// The output string : {"d":"{"username":"hi","email":"hi#gmail.com","password":"123"}"}
// The required string : {"d":{username:"hi",email:"hi#gmail.com",password:"123"}"}
// For security reasons the d is added (indicating the return "data")
// so actually we need to convert data.d into series of objects
// Inbuilt function "JSON.Parse" will return streams of objects
// JSON String : "{"username":"hi","email":"hi#gmail.com","password":"123"}"
console.log(data); // output : Object {d="{"username":"hi","email":"hi#gmail.com","password":"123"}"}
console.log(data.d); // output : {"username":"hi","email":"hi#gmail.com","password":"123"} (accessing what's stored in "d")
console.log(data.d[0]); // output : { (just accessing the first element of array of "strings")
var content = JSON.parse(data.d); // output : Object {username:"hi",email:"hi#gmail.com",password:"123"}" (correct)
console.log(content.username); // output : hi
var _name = content.username;
alert(_name); // hi
}
I had similar situation in a Promise just solved doing a cast as (String)
export async function getUserIdByRole(userRole: string): Promise<string> {
const getRole = userData.users.find((element) => element.role === userRole);
return String (getRole?.id);
}

Parsing JSON with Javascript

I have this JSon string received from an AJAX call:
{status:OK,addresses:[0,1,2,3,4,5]}
To convert it to a JSon object I have this line:
var jsonObj = eval(jsonString);
But an exception is thrown! This one has no message in the exception variable.
I also tried using
{"status":"OK","addresses":[0,1,2,3,4,5]}
And, yet again, an exception is thrown but saying that an unexpected character '&' was found.
I'm using Struts2 and the JSon is received from an action.
Any help will be appreciated.
Thank you
{status:OK,addresses:[0,1,2,3,4,5]}
is not valid JSON because the quotes around status and addresses are missing, and is neither valid JSON nor valid JavaScript since the quotes around OK are missing.
Also, don't use eval to parse JSON - it allows an attacker to execute arbitrary JavaScript in the context of your page. Instead, use the safe alternatives JSON.parse(built-in in modern browsers and other EcmaScript 5 implementations) or JSON2.
Don't use eval: use a proper JSON parser such as JSON2.
You probably have extra content in the response: check that you are not printing anything else out.
This is working for me:
JSON.parse('{ "status" : "OK", "addresses" : [0,1,2,3,4,5]}');
If you want to use eval, then you need to use the second example you posted ({"status":"OK","addresses":[0,1,2,3,4,5]}) and you need to surround the string with parenthesis as such:
var jsonObj = eval( '('+jsonString+')' );
This makes jsonString a valid javascript statement.
With that being said, I encourage you use JSON.parse, as many others have posted. It is far more secure.
You don't have a JSON string. You do have an object literal. You need the names to have quotes.
{"status":OK, "addresses":[0,1,2,3,4,5]}
Based on this comment:
So I verified that when JSon is received from the request, all the " are replaced by " ... could this be the problem?
Yes. A JSON parser expects to receive JSON as input, not HTML encoded JSON.
Two issues to fix:
Add quotes around the "OK" to make it a legal javascript string.
Add parens around the string before sending to eval like this eval("(" + jsonString + ")")';
This:
{status:OK,addresses:[0,1,2,3,4,5]}
would have to be changed to this:
{status:"OK",addresses:[0,1,2,3,4,5]}
to be valid Javascript (note the quotes around "OK").
It should be this to be valid JSON (quotes around the keys too):
{"status":"OK", "addresses":[0,1,2,3,4,5]}
OK all by itself is not a known piece of Javascript without the quotes around it to make it into a Javascript string. In the future, you can test yourself in a small test bed and see what the error is in your favorite javascript debugger:
http://jsfiddle.net/jfriend00/FcSKR/
var jsonString = '{"status":"OK","addresses":[0,1,2,3,4,5]}';
var jsonObj = eval("(" + jsonString + ")");
alert("success");
If you still get an error with {"status":"OK","addresses":[0,1,2,3,4,5]} and the adding of parens before sending to eval, then your data isn't what you think it is and you need to do some debugging to see exactly what is in the response (look at the value in the debugger, put the value into an alert, etc...). There is probably some other stuff in the response that you didn't know would be there.
Note: there are some situations where a JSON parser like JSON.parse() and a legal JSON string is safer than eval() with Javascript that can be anything.

Categories

Resources