Develop Reference

UINT8 Array to String without escape characters

I'm parsing a Uint8 array that is an HTML document. It contains a script tag which in turn contains JSON data that I would like to parse.
I first converted the array to text:
data = Buffer.from(str).toString('utf8')
I then searched for the script tag, and extracted the string containing the JSON:
... {\"phrase\":\"Go to \"California\"\",\"color\":\"red\",\"html\":\"<div class=\"myclass\">Ok</div>\"} ...
I then did a replace to clean it up.
data = data.replace(/\\"/g, "\"").replace(/\\/g, "").
{"phrase":"Go to "California"","color":"red","html":"<div class="myclass">Ok</div>"}
I tried to parse using JSON.parse() and got an error because the attributes contain quotes. Is there a way to process this further using a regex ? Or perhaps a library? I am working with Cheerio, so can use that if helpful.

The escape characters are necessary if you want to parse the JSON. The embedded quotes would need to be double escaped, so the extracted text isn't even valid JSON.
"{\"phrase\":\"Go to \\\"California\\\"\",\"color\":\"red\",\"html\":\"<div class=\\\"myclass\\\">Ok</div>\"}"
or, using single quotes:
'{"phrase":"Go to \\"California\\"","color":"red","html":"<div class=\\"myclass\\">Ok</div>"}'

Thanks.
After some more tinkering around, I realized that I should have encoded the data to Uint8 at the source (a Lambda function) before transmitting it for further processing. So now, I have:
Text
Encoded text to Uint8
Return from Lambda function.
Decode from Uint8 to text
Process readily as no escape characters.
Before, I was skipping step 2. And so Lambda was encoded the text however it does by default.

Decoding Base64 String in Java

I'm using Java and I have a Base64 encoded string that I wish to decode and then do some operations to transform.
The correct decoded value is obtained in JavaScript through function atob(), but in java, using Base64.decodeBase64() I cannot get an equal value.
Example:
For:
String str = "AAAAAAAAAAAAAAAAAAAAAMaR+ySCU0Yzq+AV9pNCCOI="
With JavaScript atob(str) I get ->
"Æ‘û$‚SF3«àö“Bâ"
With Java new String(Base64.decodeBase64(str)) I get ->
"Æ?û$?SF3«à§ö?â"
Another way I could fixed the issue is to run JavaScript in Java with a Nashorn engine, but I'm getting an error near the "$" symbol.
Current Code:
ScriptEngine engine = new ScriptEngineManager().getEngineByName("JavaScript");
String script2 = "function decoMemo(memoStr){ print(atob(memoStr).split('')" +
".map((aChar) => `0${aChar.charCodeAt(0).toString(16)}`" +
".slice(-2)).join('').toUpperCase());}";
try {
engine.eval(script2);
Invocable inv = (Invocable) engine;
String returnValue = (String)inv.invokeFunction("decoMemo", memoTest );
System.out.print("\n result: " + returnValue);
} catch (ScriptException | NoSuchMethodException e1) {
e1.printStackTrace();
Any help would be appreciated. I search a lot of places but can't find the correct answer.

btoa is broken and shouldn't be used.
The problem is, bytes aren't characters. Base64 encoding does only one thing. It converts bytes to a stream of characters that survive just about any text-based transport mechanism. And Base64 decoding does that one thing in reverse, it converts such characters into bytes.
And the confusion is, you're printing those bytes as if they are characters. They are not.
You end up with the exact same bytes, but javascript and java disagree on how you're supposed to turn that into an ersatz string because you're trying to print it to a console. That's a mistake - bytes aren't characters. Thus, some sort of charset encoding is being used, and you don't want any of this, because these characters clearly aren't intended to be printed like that.
Javascript sort of half-equates characters and bytes and will freely convert one to the other, picking some random encoding. Oof. Javascript sucks in this regard, it is what it is. The MDN docs on btoa explains why you shouldn't use it. You're running into that problem.
Not entirely sure how you fix it in javascript - but perhaps you don't need it. Java is decoding the bytes perfectly well, as is javascript, but javascript then turns those bytes into characters into some silly fashion and that's causing the problem.

What you have there is not a text string at all. The giveaway is the AA's at the beginning. Those map to a number of zero bytes. That doesn't translate to meaningful text in any standard character set.
So what you have there is most likely binary data. Converting it to a string is not going to give you meaningful text.
Now to explain the difference you are seeing between Java and Javascript. It looks to me as if both Java and Javascript are making a "best effort" attempt to convert the binary data as if is was encoded in ISO-8859-1 (aka ISO LATIN-1).
The problem is some of the bytes codes are mapping to unassigned codes.
In the Java case those unassigned codes are being mapped to ?, either when the string is created or when it is being output.
In the Javascript case, either the unassigned codes are not included in the string, or them are being removed when you attempt to display them.
For the record, this is how an online base64 decoder the above for me:
����������������Æû$SF3«àöBâ
The unassigned codes are 0x91 0x82 and 0x93. 0x15 and 0x0B are non-printing control codes.
But the bottom line is that you should not be converting this data into a string in either Java or in Javascript. It should be treated as binary; i.e. an array of byte values.

byte[] data = Base64.getDecoder().decode(str);

Decode a JSON String in Javascript

I have a encoded JSON string returning the following sting which I am displaying on my webpage from a database call.
{"Value 1":"1234","Value 2":"123456"}
How do I decode this string and also format the data to be displayed in a table?
I am at a lost as how to do this. Thanks

Here is how to turn a string containing a JSON into an object:
JSON.parse('{"Value 1":"1234","Value 2":"123456"}');

Display unicode characters (emojies) in Javascript

I have a database with tweets such as "\U0001f374 Lunch. Had loads of meat..." -- that is, with emojies represented as unicode (\U0001f374 is the knife&fork emoji). In my Web app I fetch tweets using Ajax requests and want to display them.
No big deal, and I have it so far up and running that I can display the "raw" tweet strings with the unicode. However, I like to render the emojies. How can I do this in Javascript?

Since a notation like \U0001f374 is undefined in JavaScript, you need to construct the character from it with your own code (or suitable library code). You could parse the Unicode number from the string and convert it to a pair of surrogate code points.
But if you are using JavaScript in the HTML (or XML) context, you could let the HTML (or XML) parser do the job. Just change the string (assumed to have 8 hex digits) to an HTML or XML character reference and make sure the result is parsed as markup:
var sample = document.getElementById('in').value;
sample = sample.replace(/\\U([0-9a-f]{8})/gi, "&#x$1;");
document.getElementById('demo').innerHTML = sample;
<input id=in size=40
value="\U0001f374 Lunch. Had loads of meat...">
<div id=demo>To be replaced</div>

How to convert formatted string to regularly javascript dictionary?

I get formatted json string with all \ before " and \n for newlines.How to convert this string to regularly javascript dictionary ?
I thought to replace all \n with '' and \" with " but it is kinda bruteforce solution. Is there moreelegant way ?

It sounds like you're receiving JSON encoded data. To convert the raw data into an object, use the JSON.parse function:
var test = "{\"foo\":\"bar\"}";
var data = JSON.parse(test);
console.log(data);

I am not sure I understand what you mean by 'JavaScript dictionary' exactly but in my experience the easiest way to convert a JSON string to any kind of usable JavaScript object is to use JSON.parse, see Parse JSON in JavaScript? for some good information on this.
Also in future a small sample of what you are trying to do, your source data etc. would be helpful!

It's a escaped string, you should unescape it and using eval will return the object represented by the json string. A JSON string is simply a javascript serialized object, so you may eval'd with javascript and will return the "map" or object that represents.
Newlines are valid in json so you don't require to remove them.
var o = eval("o = {name:\"test\"}");
alert(o.name);

You're probably thinking of a dictionary implementation as you'd find in other languages such as Objective C or C# - JavaScript does not have a dictionary implementation. So is your question how to parse JSON so you can get some values into key value pairs? If so then it sounds like JSON.parse is going to work for you.
If your question is about how to implement something like a dictionary in JavaScript, with data populated from JSON - then you'll want to parse the JSON and set up some simple JavaScript objects to act like a dictionary:
var dictionary = {"key1":"hello", "key2":"hello2", "key3":"hello3"};
console.log(dictionary["key3"]); // gives the value "hello3"