Develop Reference

Passing Unicode string from Python dict to JSON

I have a dict that contains some Unicode strings (among other objects). I'd like to save this dict as a JSON file, and then display the content of this via AJAX.
If final_res is the dict, I usually do this:
json.dumps(final_res, ensure_ascii=True)
In the result, I see strings like:
"l\\u00a0m\\u00fcdale"
I imagine these are unicode encoded characters. But when I try to display them in Javascript, these get printed with the slashes, instead of the encoded unicode letter.
Is there something I am doing wrong in Javascript for displaying these properly? Or should I do decode these into ASCII in Python, before outputting to JSON?
UPDATE:
Based on the discussion in the comments below with #spectra, I realized that json.dumps should not be outputting double slashes. When I parse this in the browser, this prints it as a literal single slash.
I am trying to figure out a way to fix this with the json module, not sure why it's happening.

The solution for me was to save the result of json.dumps to the database with the "single-slashed" version. I did it by calling print on the result of json.dumps and then copy-pasting that to the database.

You can encode the json file in UTF8 instead of escaped charaters:
json.dumps(final_res,ensure_ascii=False).encode('utf8')
For example
print json.dumps({'name':u'l\u00a0m\u00fcdale'},ensure_ascii=False).encode('utf8')
# {"name": "l müdale"}
Then in your client side AJAX code, set the encoding to 'utf8' :
How to set encoding in .getJSON JQuery
$.ajax({ contentType: "application/json; charset=utf-8", ... })

Error Parsing JSON with escaped quotes

I am getting the following json object when I call the URL from Browser which I expect no data in it.
"{\"data\":[], \"SkipToken\":\"\", \"top\":\"\"}"
However, when I tried to call it in javascript it gives me error Parsing Json message
dspservice.callService(URL, "GET", "", function (data) {
var dataList = JSON.parse(data);
)};
This code was working before I have no idea why all of a sudden stopped working and throwing me error.

You say the server is returning the JSON (omitting the enclosing quotes):
{\"data\":[], \"SkipToken\":\"\", \"top\":\"\"}
This is invalid JSON. The quote marks in JSON surrounding strings and property names should not be preceded by a backslash. The backslash in JSON is strictly for inserting double quote marks inside a string. (It can also be used to escape other characters inside strings, but that is not relevant here.)
Correct JSON would be:
{"data":[], "SkipToken":"", "top":""}
If your server returned this, it would parse correctly.
The confusion here, and the reports by other posters that it seems like your string should work, lies in the fact that in a simple-minded test, where I type this string into the console:
var x = "{\"data\":[], \"SkipToken\":\"\", \"top\":\"\"}";
the JavaScript string literal escaping mechanism, which is entirely distinct from the use of escapes in JSON, results in a string with the value
{"data":[], "SkipToken":"", "top":""}
which of course JSON.parse can handle just fine. But Javascript string escaping applies to string literals in source code, not to things coming down from the server.
To fix the server's incorrectly-escaped JSON, you have two possibilities. One is to tell the server guys they don't need to (and must not) put backslashes before quote marks (except for quote marks inside strings). Then everything will work.
The other approach is to undo the escaping yourself before handing it off to JSON.parse. A first cut at this would be a simple regexp such as
data.replace(/\\"/g, '"')
as in
var dataList = JSON.parse(data.replace(/\\"/g, '"')
It might need additional tweaking depending on how the server guys are escaping quotes inside strings; are they sending \"\\"\", or possibly \"\\\"\"?
I cannot explain why this code that was working suddenly stopped working. My best guess is a change on the server side that started escaping the double quotes.

Since there is nothing wrong with the JSON string you gave us, the only other explanation is that the data being passed to your function is something other than what you listed.
To test this hypothesis, run the following code:
dspservice.callService(URL, "GET", "", handler(data));
function handler(data) {
var goodData = "{\"data\":[], \"SkipToken\":\"\", \"top\":\"\"}";
alert(goodData); // display the correct JSON string
var goodDataList = JSON.parse(goodData); // parse good string (should work)
alert(data); // display string in question
var dataList = JSON.parse(data); // try to parse it (should fail)
}
If the goodData JSON string can be parsed with no issues, and data appears to be incorrectly-formatted, then you have the answer to your question.
Place a breakpoint on the first line of the handler function, where goodData is defined. Then step through the code. From what you told me in your comments, it is still crashing during a JSON parse, but I'm willing to wager that it is failing on the second parse and not the first.

Did you mean that your JSON is like this?
"{\"data\":[], \"SkipToken\":\"\", \"top\":\"\"}"
Then data in your callback would be like this:
'"{\"data\":[], \"SkipToken\":\"\", \"top\":\"\"}"'
Because data is the fetched text content string.
You don't have to add extra quotes in your JSON:
{"data":[], "SkipToken":"", "top":""}

Sending JSON with AJAX to Flask Syntax Error

url: '/upload',
type: 'POST',
data: JSON.stringify(pdata),
contentType: "application/json; charset=utf-8",
dataType: "text",
is what I'm using to POST the JSON and it is sending it with the double quotes needed
but in flask
content = request.get_json()
gives me content with the double quotes swapped with single quotes
for example {"S":[],"R":[]} goes to {'S':[],'R':[]}
it is something I'm doing wrong or do I need to somehow convert this back to two quotes if I want to send this to a javascript program?

request.get_json() returns a python dictionary with values of double-quoted items represented as strings. When you print a python string it's printed with single quotes by default.
When the dictionary is converted back into JSON format you will get doublequotes.

How to prepare encoded POST data on javascript?

I need to send data by POST method.
For example, I have the string "bla&bla&bla". I tried using encodeURI and got "bla&bla&bla" as the result. I need to replace "&" with something correct for this example.
What kind of method should I call to prepare correct POST data?
UPDATED:
I need to convert only charachters which may broke POST request. Only them.

>>> encodeURI("bla&bla&bla")
"bla&bla&bla"
>>> encodeURIComponent("bla&bla&bla")
"bla%26bla%26bla"

You can also use escape() function.The escape() function encodes a string.
This function makes a string portable, so it can be transmitted across any network to any computer that supports ASCII characters.This function encodes special characters, with the exception of: * # - _ + . /
var queryStr = "bla&bla&bla";
alert(queryStr); //bla&bla&bla
alert(escape(queryStr)); //bla%26bla%26bla
Use unescape() to decode a string.
var newQueryStr=escape(queryStr);
alert(unescape(newQueryStr)); //bla&bla&bla
Note:
escape() will not encode: #*/+
encodeURI() will not encode: ~!##$&*()=:/,;?+'
encodeURIComponent() will not encode: ~!*()'
After some search on internet, I got the following:
escape()
Don't use it.
encodeURI()
Use encodeURI when you want a working URL. Make this call:
encodeURI("http://www.google.com/a file with spaces.html")
to get:
http://www.google.com/a%20file%20with%20spaces.html
Don't call encodeURIComponent since it would destroy the URL and return
http%3A%2F%2Fwww.google.com%2Fa%20file%20with%20spaces.html
encodeURIComponent()
Use encodeURIComponent when you want to encode a URL parameter.
param1 = encodeURIComponent("http://xyz.com/?a=12&b=55")
Then you may create the URL you need:
url = "http://domain.com/?param1=" + param1 + "&param2=99";
And you will get this complete URL:
http://www.domain.com/?param1=http%3A%2F%2Fxyz.com%2F%Ffa%3D12%26b%3D55¶m2=99
Note that encodeURIComponent does not escape the ' character. A common bug is to use it to create html attributes such as href='MyUrl', which could suffer an injection bug. If you are constructing html from strings, either use " instead of ' for attribute quotes, or add an extra layer of encoding (' can be encoded as %27).
REF:When are you supposed to use escape instead of encodeURI / encodeURIComponent?
Also, as you are using JQuery, take a look at this built-in function.

Use encodeURIComponent() as encodeURI() will not encode: ~!##$&*()=:/,;?+'
This has been explained quite well at the following link:
http://xkr.us/articles/javascript/encode-compare/

More recent DOM APIs for URLSearchParams (and via URL, possibly others too) handle encoding in some cases. For example, create or use an existing URL object (like from an anchor tag) I map entries of an object as key value pairs for URL encoded params (to use for GET/POST/etc in application/x-www-form-urlencoded mimetype). Note how the emoji, ampersand and double quotes are encoded without any special handling (copied from the Chrome devtools console):
var url = new URL(location.pathname, location.origin);
Object.entries({a:1,b:"🍻",c:'"stuff&things"'}).forEach(url.searchParams.set, url.searchParams);
url.search;
"?a=1&b=%F0%9F%8D%BB&c=%22stuff%26things%22"
fetch(url.pathname, {
method: 'POST',
headers: new Headers({
"Content-type": "application/x-www-form-urlencoded"
}),
// same format as GET url search params a&b&c
body: url.searchParams
}).then((res)=>{ console.log(res); return res }).catch((res)=>{ console.warn(res); return res; });

I want POST the javascript-created hidden form.
So the question is if encodeURIComponent() should be used on each POST variable.
I haven't found the answer for Dmitry's (and my) question in this thread.
But I have found the answer in this thread.
In case of form/POST where you have upload field(s) you must use <form enctype="multipart/form-data">, if no upload field is used, you should choose yourself as described here.
Submitting the form should do the job completly, so there is no need to use encodeURIComponent() explicitly.
If you create a Http POST without using a form or without some library which creates a Http POST from your data, then you need choose an enctype= and join data yourselves.
This will be easy for application/x-www-form-urlencoded, where you will use encodeURIComponent() on each value and join them exactly as for GET request.
If you decide use multipart/form-data then ....? You should google more how to encode and join them in such case.

Single quotes in string with jQuery ajax

I have run into a problem where the user enters data and if there are single quotes at all, the script errors out.
What's the best way to handle single quotes that users enter so it doesn't interfere with the jquery/javascript?
UPDATE:
I'm sending it through ajax to a database. here is the data parameter for a json ajax call.
data: "{str_" + sectionName + " :'" + UpdateText + "',EntityID: '" + EntityID + "' }",
with update text being the string that can contain the quotes.

You need to escape the quotes with a \ or depending on how you plan to use the string you can use the javascript escape and unescape functions.
alert(escape("mike's"));
alert(unescape(escape("mike's")));
Also check this out for ways to escape strings with jQuery

For escaping values in AJAX request, Do not write your own implementation of escape or use escape() method. (escape() is deprecated). Instead create a JSON object and use JSON.stringify method.
For your case it should be like (ignoring dynamic property for now):
//Create Javascript object
var obj = { SectionName: UpdateText, EntityID: EntityID };
Later in your ajax request you can do :
data: JSON.stringify(obj),
If you want to use dynamic properties with your JSON object then for your particular case you can create the object in two steps like:
var obj = { EntityID: EntityID };
obj["str_" + sectionName] = UpdateText;
This practice will save you from manually escaping single/double quotes and other invalid characters. JSON.stringify will take care of that.
(I came here looking for a somewhat similar issue, but couldn't find a suitable working solution, so ended up posting one here)

You could find one of the many String.replaceAll implementations or write your own, and just replace any single or double quotes with an escaped version like \" or \'.

Since you mentioned AJAX, there is a possibility that the strings involving single quotes are getting rejected at the server side.
Make sure you use escape string function provided , for example by php, before inserting strings, to the database.
$user_name = $_REQUEST['username'];
$user_name = mysqli_real_escape_string($conn,$user_name);
$query = "INSERT into chat(username,message) VALUES('".$user_name."')";
This helps in escaping any single or double quotes that may appear in the '$user_name' string.
Also it prevents against any kind of SQL injection atacks!

You should really sanitize your input inside your server-side script for a variety of reasons. If you're just displaying everything the user enters then your application can likely be used to launch a cross-site scripting attack.

Javascript has a built in method just for this that covers more than just single quotes. Its called encodeURIComponent, from Javascript Kit:
Used to encode the parameter portion of a URI for characters that have special meaning, to separate them from reserved characters such as "&" that act as key/value separators. More inclusive than encodeURI(), it encodes all characters with special meaning in a URL string, including "=" and "&". Use this method only on the parameter portion of a URI; otherwise, the URI may no longer be valid if it contains one of the characters that are part of a valid URI (ie: "+") yet should be escaped if part of the URI parameter.
So your code should become:
data: "{str_" + encodeURIComponent(sectionName) + " :'" + encodeURIComponent(UpdateText) + "',EntityID: '" + encodeURIComponent(EntityID) + "' }",
I encode everything I send in a query string to be safe, but encoding the EntityID could arguably be skipped because it doesn't come from the user(I'm assuming), so you know it won't have special characters.

To escape a single quote in Javascript use
UpdateText.replace('\'', '\\\'')
To escape all single quotes use
UpdateText.replace(/'/g, '\\\'')

Thanks to mbrevoort,
I elaborate more on his answer
When You are sending a single quote in a query
empid = " T'via"
empid = escape(empid)
When You get the value including a single quote
var xxx = request.QueryString("empid")
xxx = unscape(xxx)
If you want to search/ insert the value which includes a single quote in a query
xxx = Replace(empid, "'", "''")

The accepted answer should not be the solution to use.
In order to send this through AJAX to DB where request data has single quote ' in the string, do below:
Organize your request data as an object.
var data = {
"sectionName" : sectionName,
"UpdateText" : updateText,
"EntityID" : entityID
}
Stringfy your data to JSON and send with AJAX
data = JSON.stringify(data);
$.ajax({
url: "",
type: "POST",
data: data,
contentType: "application/json; charset=utf-8",
dataType: "json"
}).done(function (res) {
alert(res);
});
Depends on the Database, for SQL Server, replace your single quote ' to double quote '' to escape the single quote .
string data = date.Replace("'", "''")