Develop Reference

Weird charaters in a stringified buffer in javascript

I have a particular context in which one data are transformed a lot to get transferred across network. At the end, when I try to get this data back, I have unwanted characters at the beginning of the string.
First, I get the data from a db and it returns it to me as bytes (<Array.<byte>>), fully readable with .toString(). The result is:
{\"company\":\"xxx\",\"email\":\"xxx\",\"firstName\":\"xxx\",\"lastName\":\"xxx\",\"providerId\":\"xxx\",\"role\":\"xxx\",\"status\":\"xxx\"}
These data are passed to another "environment" with a function (not developed by me and that I cannot change) that returns the data in a "I don't really know what format it is".
I can decode it with the following piece of code:
jsonIdentity = JSON.stringify(bufferIdentity);
Buffer.from(JSON.parse(jsonIdentity).payload.buffer.data).toString('utf-8')
However, at the beginning of the string, I have the following:
"\u0008\u0006\u001a�\u0001\u0008�\u0001\u001a{{\"company\":\"xxx\",\"email\":\"xxx\",\"firstName\":\"xxx\",\"lastName\":\"xxx\",\"providerId\":\"xxx\",\"role\":\"xxx\",\"status\":\"xxx\"}
Also represented like that in my logs:
��{{"company":"xxx","email":"xxx","firstName":"xxx","lastName":"xxx","providerId":"xxx","role":"xxx","status":"xxx"
How can I remove it/prevent it to get in my result? It prevents me from using the JSON.
Update: here is the buffer I get:
{"status":200,"message":"","payload":{"buffer":{"type":"Buffer","data":[8,6,26,128,1,8,200,1,26,123,123,34,99,111,109,112,97,110,121,34,58,34,105,98,109,34,44,34,101,109,97,105,108,34,58,34,102,64,105,98,109,46,99,111,109,34,44,34,102,105,114,115,116,78,97,109,101,34,58,34,102,108,111,114,105,97,110,34,44,34,108,97,115,116,78,97,109,101,34,58,34,99,97,115,116,34,44,34,112,114,111,118,105,100,101,114,73,100,34,58,34,102,99,34,44,34,114,111,108,101,34,58,34,117,115,101,114,34,44,34,115,116,97,116,117,115,34,58,34,111,107,34,125,34,64,98,54,57,51,50,51,53,100,49,52,97,49,98,102,57,57,56,100,50,99,97,102,53,53,52,52,100,97,49,50,50,51,55,101,97,55,99,50,56,55,50,49,56,97,101,55,51,100,55,97,50,53,101,52,55,48,48,51,56,52,100,54,53,54,58,14,100,101,102,97,117,108,116,99,104,97,110,110,101,108]},"offset":10,"markedOffset":-1,"limit":133,"littleEndian":true,"noAssert":false}}

Can you try this out:
const yourString = JSON.parse(jsonIdentity).payload.buffer.data;
console.log(Buffer.from(yourString, 'base64').toString('utf-8'))
An ugly solution would be just trim or replace the characters from your result

Problem fixed. Solution is available on Jira here: https://jira.hyperledger.org/browse/FAB-14785?focusedCommentId=58680&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-58680

JSP attribute JSON object is dropping escape characters in Javascript

I have a server-side controller which is adding an attribute which is a Java object converted to JSON using Jackson:
ObjectMapper mapper = new ObjectMapper();
model.addAttribute("commentObj", mapper.writeValueAsString(commentObject);
My commentObject has a field "comments" which contains quotation marks (") that JSON needs to escape. When I do a simple print-out from the JSP of the commentObject, everything looks good:
[{"commentId":123,"comments": "this \"test\" is here"}]
As you see, the quotes get escaped properly (\").
My problem comes with the javascript on this page that also needs to access this object. So when I run the following:
<script>
var test = ${commentObj};
</script>
If I take a look at the 'test' var, the comments field now reads as follows:
"this "test" is here"
vs.
"this \"test\" is here"
This causes a problem because JSON.parse() will throw an error on 'test' because it is not properly formed JSON anymore.
So my question is, how do I manage to get my commentObj into a javascript object while keeping the escape characters so that I can JSON.parse it properly?

Sorry, written at the end of a long day while sick and only just realized now that the 'test' var ends up being a fully formed JS object. So I am able to use that.
That said, I still have the question of how that is happening? It seems to me that it should be coming in as a JSON string that I need to call JSON.parse on. Anyone have an explanation?

string.slice() not working on Server Side Javascript in ASPx

I am writing server side javascript code in aspx to pull out the 7 columns from CSV files which may or may not use quotes to enclose the data, depending on the file.
Currently, I am working on the code to remove the commas using javascript's string.sllice() for the first column, but is is returning an empty string.
Here is a sample line from the file. This is public info.
12/6/2017,8:30 AM,2013FA000060,In RE: the Support of: D.D.K. and A.P.C.,MH,Motion hearing,"Grill, Leo",State of Wisconsin,"Vesely, Tori A; Lawton, Mark David";
In working with my Regular expression based code for the first column, I have the correct expected string data for strTemp prior to the offending code:
12/6/2017,
Here is the code to remove the comma at the end:
cleanData = strTemp.slice(0, -1); // remove last 1 character
I have verified that strTemp is correct right before this statement is executed to make sure the string var assignments are not the problem (as seen above).
The expected result should be strTemp data without the comma at the end:
12/6/2017
I receive no errors. Just an empty string.

OK, so it looks like the data coming from the regular expression execute is not a string. Once I cast it to string it worked
cleanData = strTemp.toString().slice(0, -1);
I really do NOT like working with weakly typed languages!!

Parsing malformed JSON in JavaScript

Thanks for looking!
BACKGROUND
I am writing some front-end code that consumes a JSON service which is returning malformed JSON. Specifically, the keys are not surrounded with quotes:
{foo: "bar"}
I have NO CONTROL over the service, so I am correcting this like so:
var scrubbedJson = dirtyJson.replace(/(['"])?([a-zA-Z0-9_]+)(['"])?:/g, '"$2": ');
This gives me well formed JSON:
{"foo": "bar"}
Problem
However, when I call JSON.parse(scrubbedJson), I still get an error. I suspect it may be because the entire JSON string is surrounded in double quotes but I am not sure.
UPDATE
This has been solved--the above code works fine. I had a rogue single quote in the body of the JSON that was returned. I got that out of there and everything now parses. Thanks.
Any help would be appreciated.

You can avoid using a regexp altogether and still output a JavaScript object from a malformed JSON string (keys without quotes, single quotes, etc), using this simple trick:
var jsonify = (function(div){
return function(json){
div.setAttribute('onclick', 'this.__json__ = ' + json);
div.click();
return div.__json__;
}
})(document.createElement('div'));
// Let's say you had a string like '{ one: 1 }' (malformed, a key without quotes)
// jsonify('{ one: 1 }') will output a good ol' JS object ;)
Here's a demo: http://codepen.io/csuwldcat/pen/dfzsu (open your console)

something like this may help to repair the json ..
$str='{foo:"bar"}';
echo preg_replace('/({)([a-zA-Z0-9]+)(:)/','$1"$2"${3}',$str);
Output:
{"foo":"bar"}
EDIT:
var str='{foo:"bar"}';
str.replace(/({)([a-zA-Z0-9]+)(:)/,'$1"$2"$3')

There is a project that takes care of all kinds of invalid cases in JSON https://github.com/freethenation/durable-json-lint

I was trying to solve the same problem using a regEx in Javascript. I have an app written for Node.js to parse incoming JSON, but wanted a "relaxed" version of the parser (see following comments), since it is inconvenient to put quotes around every key (name). Here is my solution:
var objKeysRegex = /({|,)(?:\s*)(?:')?([A-Za-z_$\.][A-Za-z0-9_ \-\.$]*)(?:')?(?:\s*):/g;// look for object names
var newQuotedKeysString = originalString.replace(objKeysRegex, "$1\"$2\":");// all object names should be double quoted
var newObject = JSON.parse(newQuotedKeysString);
Here's a breakdown of the regEx:
({|,) looks for the beginning of the object, a { for flat objects or , for embedded objects.
(?:\s*) finds but does not remember white space
(?:')? finds but does not remember a single quote (to be replaced by a double quote later). There will be either zero or one of these.
([A-Za-z_$\.][A-Za-z0-9_ \-\.$]*) is the name (or key). Starts with any letter, underscore, $, or dot, followed by zero or more alpha-numeric characters or underscores or dashes or dots or $.
the last character : is what delimits the name of the object from the value.
Now we can use replace() with some dressing to get our newly quoted keys:
originalString.replace(objKeysRegex, "$1\"$2\":")
where the $1 is either { or , depending on whether the object was embedded in another object. \" adds a double quote. $2 is the name. \" another double quote. and finally : finishes it off.
Test it out with
{keyOne: "value1", $keyTwo: "value 2", key-3:{key4:18.34}}
output:
{"keyOne": "value1","$keyTwo": "value 2","key-3":{"key4":18.34}}
Some comments:
I have not tested this method for speed, but from what I gather by reading some of these entries is that using a regex is faster than eval()
For my application, I'm limiting the characters that names are allowed to have with ([A-Za-z_$\.][A-Za-z0-9_ \-\.$]*) for my 'relaxed' version JSON parser. If you wanted to allow more characters in names (you can do that and still be valid), you could instead use ([^'":]+) to mean anything other than double or single quotes or a colon. You can have all sorts of stuff in here with this expression, so be careful.
One shortcoming is that this method actually changes the original incoming data (but I think that's what you wanted?). You could program around that to mitigate this issue - depends on your needs and resources available.
Hope this helps.
-John L.

How about?
function fixJson(json) {
var tempString, tempJson, output;
tempString = JSON.stringify(json);
tempJson = JSON.parse(tempString);
output = JSON.stringify(tempJson);
return output;
}

How to process a javascript function call that returns a string with quotes in it

I'm having an issue with a javascript call to an api function in NetSuite that returns a string with quotes in it. An error is thrown each time the call is made.
var selling_point_1 = "<%=getCurrentAttribute('item','custitemsellingpoint1')%>";
when looking in the debugger, this evaluates to:
var selling_point_1 = "Product Dimensions: H:14" W:24"";
Any string function (like .length or charAt(0) ) on this also throws an error. I have no control over what the function call returns, so i need to know how to handle embedded quotes.
Any help would be greatly appreciated, John

Although not the most robust method you could use:
var selling_point_1 = escape("<%=getCurrentAttribute('item','custitemsellingpoint1')%>");
This is actually for URI escaping, but will get rid of the pesky double quotes, plus you can use unescape to get the original format back. As suggested
var selling_point_1 = '<%=getCurrentAttribute(\'item\',\'custitemsellingpoint1\')%>';
Should also work in your case.

See this thread for someone dealing with roughly the same issue. The short answer is that you need to run some kind of escape function in the server-side code (i.e., within the <%=...%> block) so that only escaped values get inserted into the client-side code. All of the solutions below can handle an unlimited number of single and double quotes.
My first suggestion is to try:
var selling_point_1 = decodeURI("<%=Server.URLEncode(getCurrentAttribute('item','custitemsellingpoint1'))%>");
This will produce server-side JS that looks like:
var selling_point_1 = decodeURI("Product Dimensions: H:14%22 W:24%22");
The decodeURI JavaScript function will convert the %22 back into quotes and the correct string will be stored in selling_point_1.
If that fails, you might also try something like:
var selling_point_1 = unescape("<%=HttpServerUtility.HtmlEncode(getCurrentAttribute('item','custitemsellingpoint1'))%>");
which operates similarly, but tuuns your quotes into \" sequences, which will be converted back into ordinary quotes by JavaScript's unescape.