Develop Reference

How to % encode single quotes when making jQuery AJAX call?

I am using $.ajax to send a SQL query to my web server. The query string contains several single quotes ('). I encountered a very messy problem on encoding single quotes.
My code like this, please note the single quotes in query string:
var query = "select SID, age from Students where Name=\'Jason\'" + String.fromCharCode(10) +
"order by age asc";
$.ajax({
url: "http://mywebserver/query",
data: {
env: "dbserver1",
endTime: "getUTCDate()",
startTime: "dateadd(hour, -336, getUTCDate())",
text: query
},
type: "GET",
dataType: "json"
}).done(function (datum) {
});
If I don't explicitly call encodeURIComponent before make AJAX call, jQuery will encode it for me, however, the single quotes are not encoded to %27 by default, so query doesn't work;
If I pass an encoded query string to jQuery, it will encode again, that messes up the query string.
The only solution i can imagine is, i have to overwrite the behavior how jQuery encodes URL, and replace all (') with %27. But I don't know if jQuery supports that.
Does anyone have a solution for this?

As you're using GET, you can "override" this behaviour of jQuery by creating your own query string...
url: "http://mywebserver/query"+
"?env=" + encodeURIComponent(dbserver1)+
"&endTime="+encodeURIComponent(getUTCDate())+
"&startTime="+encodeURIComponent(dateadd(hour, -336, getUTCDate()))+
"&text="+encodeURIComponent(query)
This is equivalent to what you're doing

How to prepare encoded POST data on javascript?

I need to send data by POST method.
For example, I have the string "bla&bla&bla". I tried using encodeURI and got "bla&bla&bla" as the result. I need to replace "&" with something correct for this example.
What kind of method should I call to prepare correct POST data?
UPDATED:
I need to convert only charachters which may broke POST request. Only them.

>>> encodeURI("bla&bla&bla")
"bla&bla&bla"
>>> encodeURIComponent("bla&bla&bla")
"bla%26bla%26bla"

You can also use escape() function.The escape() function encodes a string.
This function makes a string portable, so it can be transmitted across any network to any computer that supports ASCII characters.This function encodes special characters, with the exception of: * # - _ + . /
var queryStr = "bla&bla&bla";
alert(queryStr); //bla&bla&bla
alert(escape(queryStr)); //bla%26bla%26bla
Use unescape() to decode a string.
var newQueryStr=escape(queryStr);
alert(unescape(newQueryStr)); //bla&bla&bla
Note:
escape() will not encode: #*/+
encodeURI() will not encode: ~!##$&*()=:/,;?+'
encodeURIComponent() will not encode: ~!*()'
After some search on internet, I got the following:
escape()
Don't use it.
encodeURI()
Use encodeURI when you want a working URL. Make this call:
encodeURI("http://www.google.com/a file with spaces.html")
to get:
http://www.google.com/a%20file%20with%20spaces.html
Don't call encodeURIComponent since it would destroy the URL and return
http%3A%2F%2Fwww.google.com%2Fa%20file%20with%20spaces.html
encodeURIComponent()
Use encodeURIComponent when you want to encode a URL parameter.
param1 = encodeURIComponent("http://xyz.com/?a=12&b=55")
Then you may create the URL you need:
url = "http://domain.com/?param1=" + param1 + "&param2=99";
And you will get this complete URL:
http://www.domain.com/?param1=http%3A%2F%2Fxyz.com%2F%Ffa%3D12%26b%3D55¶m2=99
Note that encodeURIComponent does not escape the ' character. A common bug is to use it to create html attributes such as href='MyUrl', which could suffer an injection bug. If you are constructing html from strings, either use " instead of ' for attribute quotes, or add an extra layer of encoding (' can be encoded as %27).
REF:When are you supposed to use escape instead of encodeURI / encodeURIComponent?
Also, as you are using JQuery, take a look at this built-in function.

Use encodeURIComponent() as encodeURI() will not encode: ~!##$&*()=:/,;?+'
This has been explained quite well at the following link:
http://xkr.us/articles/javascript/encode-compare/

More recent DOM APIs for URLSearchParams (and via URL, possibly others too) handle encoding in some cases. For example, create or use an existing URL object (like from an anchor tag) I map entries of an object as key value pairs for URL encoded params (to use for GET/POST/etc in application/x-www-form-urlencoded mimetype). Note how the emoji, ampersand and double quotes are encoded without any special handling (copied from the Chrome devtools console):
var url = new URL(location.pathname, location.origin);
Object.entries({a:1,b:"🍻",c:'"stuff&things"'}).forEach(url.searchParams.set, url.searchParams);
url.search;
"?a=1&b=%F0%9F%8D%BB&c=%22stuff%26things%22"
fetch(url.pathname, {
method: 'POST',
headers: new Headers({
"Content-type": "application/x-www-form-urlencoded"
}),
// same format as GET url search params a&b&c
body: url.searchParams
}).then((res)=>{ console.log(res); return res }).catch((res)=>{ console.warn(res); return res; });

I want POST the javascript-created hidden form.
So the question is if encodeURIComponent() should be used on each POST variable.
I haven't found the answer for Dmitry's (and my) question in this thread.
But I have found the answer in this thread.
In case of form/POST where you have upload field(s) you must use <form enctype="multipart/form-data">, if no upload field is used, you should choose yourself as described here.
Submitting the form should do the job completly, so there is no need to use encodeURIComponent() explicitly.
If you create a Http POST without using a form or without some library which creates a Http POST from your data, then you need choose an enctype= and join data yourselves.
This will be easy for application/x-www-form-urlencoded, where you will use encodeURIComponent() on each value and join them exactly as for GET request.
If you decide use multipart/form-data then ....? You should google more how to encode and join them in such case.

Escape Characters in JavaScript Function for Double quote

I have a web application where I am dynamically creating a url. The url has a parameter and I must pass a double quote. I have tried this all different ways but it is still not working. Anybody have any ideas?
To create the URL:
searchSurveyDetail.setSurveyFormURL(surveyDetail.getSurveyFormURL()+"#search="+ "\"" + searchValue + "\"");
on the Page:
onClick="window.open('${surveyDetail.surveyInstructionsURL}')"
The result:
onClick="window.open('http://www.mytest.com/survey1.pdf#search="company"')"

The short answer is you need to double escape the double quotes. So you need:
searchSurveyDetail.setSurveyFormURL(surveyDetail.getSurveyFormURL()+"#search="+ "\\\"" + searchValue + "\\\"");
which produces:
onClick="window.open('http://www.mytest.com/survey1.pdf#search=\"company\"')"
which will escape the quotes properly.
Couple of things to keep in mind:
This doesn't take care of double quotes in the search term itself. Make sure you escape that.
I'm not sure why you want to wrap the search term in double quotes. For a typical search url, you'll want a query string like: search=term not search="term" because you'll just end up stripping the quotes later. But maybe you need that for some reason.

I gather you're using PHP on the server side? In that case you should run the URL through htmlspecialchars() before concatenating it to the HTML.

POST data issues

I have an issue with submitting post data. I have a form which have a couple of text fields in, and when a button is pressed to submit the data, it is run through a custom from validation (JS), then I construct a query string like
title=test&content=some content
which is then submitted to the server. The problem I had is when I have '&' (eg &nbsp) entered into one of the inputs which then breaks up the query string. Eg:
title=test&content=some content &nbsp
How do I get around this?
Thanks in advance,
Harry.

Run encodeURIComponent over each key and value.
var title = "test";
var content = "some content &nbsp ";
var data = encodeURIComponent('title') + /* You don't actually need to encode this as it is a string that only contains safe characters, but you would if you weren't sure about the data */
'=' + encodeURIComponent(title) +
'&' + encodeURIComponent('content') +
'=' + encodeURIComponent(content);

Encode the string..when you want to encode a query string with special characters you need to use encoding. ampersand is encoded like this
title=test&content=some content %26
basically any character in a query string can be replaced by its ASCII Hex equivalent with a % as the prefix
Space = %20
A = %41
B = %42
C = %43
...

You need to encode your query to make it URL-safe. You can refer to the following links on how to do that in JS:
http://xkr.us/articles/javascript/encode-compare/
http://www.webtoolkit.info/javascript-url-decode-encode.html

You said:
...and when a button is pressed to submit the data, it is run through a custom from validation (JS), then I construct a query string...
In the section where you are building the query string you should also run the value of each input through encodeURIComponent() as David Dorward suggested.
As you do - be careful that you only assign the new value to your processed query string and NOT the form element value, otherwise your users will think their input was somehow corrupted and potentially freak out.
[EDIT]
I just re-read your question and realized something important: you're encoding an &nbsp ;character. This is probably a more complicated issue than other posters here have read into. If you want that character, and other &code; type characters to transfer over you'll need to realize that they are codes. Those characters &, n, b, s, p and ; are not themselves the same as " " which is a space character that does not break.
You'll have to add another step of encoding/decoding. You can place this step either before of after the data is sent (or "POSTed").
Before:
(Using this question's answers)
var data = formElement.value;
data = rhtmlspecialchars(data, 0);
Which is intended to replace your "special" characters like with " " so that they are then properly encoded by encodeURIComponent(data)
Or after:
(using standard PHP functions)
<?PHP
$your_field_name = htmlspecialchars_decode(urldecode($_POST['your_field_name']));
?>
This assumes that you escaped the & in your POST with %26
If you replaced it with some function other than encodeURIComponent() you'll have to find a different way to decode it in PHP.

This should solve your problem:
encodeURIComponent(name)+'='+encodeURIComponent(value)+'&'+encodeURIComponent(name2)+'='+encodeURIComponent(value2)
You need to escape each value (and name if you want to be on the safe side) before concatenating them when you're building your query.
The JavaScript global function encodeURIComponent() does the escaping.
The global function escape() (DOM) does this for you in a browser. Although people are saying it is not doing the escaping well for unicode chars. Anyway if you're only concerned about '&' then this would solve your problem.

jQuery.parseJSON throws “Invalid JSON” error due to escaped single quote in JSON

I’m making requests to my server using jQuery.post() and my server is returning JSON objects (like { "var": "value", ... }). However, if any of the values contains a single quote (properly escaped like \'), jQuery fails to parse an otherwise valid JSON string. Here’s an example of what I mean (done in Chrome’s console):
data = "{ \"status\": \"success\", \"newHtml\": \"Hello \\\'x\" }";
eval("x = " + data); // { newHtml: "Hello 'x", status: "success" }
$.parseJSON(data); // Invalid JSON: { "status": "success", "newHtml": "Hello \'x" }
Is this normal? Is there no way to properly pass a single quote via JSON?

According to the state machine diagram on the JSON website, only escaped double-quote characters are allowed, not single-quotes. Single quote characters do not need to be escaped:
Update - More information for those that are interested:
Douglas Crockford does not specifically say why the JSON specification does not allow escaped single quotes within strings. However, during his discussion of JSON in Appendix E of JavaScript: The Good Parts, he writes:
JSON's design goals were to be minimal, portable, textual, and a subset of JavaScript. The less we need to agree on in order to interoperate, the more easily we can interoperate.
So perhaps he decided to only allow strings to be defined using double-quotes since this is one less rule that all JSON implementations must agree on. As a result, it is impossible for a single quote character within a string to accidentally terminate the string, because by definition a string can only be terminated by a double-quote character. Hence there is no need to allow escaping of a single quote character in the formal specification.
Digging a little bit deeper, Crockford's org.json implementation of JSON for Java is more permissible and does allow single quote characters:
The texts produced by the toString methods strictly conform to the JSON syntax rules. The constructors are more forgiving in the texts they will accept:
...
Strings may be quoted with ' (single quote).
This is confirmed by the JSONTokener source code. The nextString method accepts escaped single quote characters and treats them just like double-quote characters:
public String nextString(char quote) throws JSONException {
char c;
StringBuffer sb = new StringBuffer();
for (;;) {
c = next();
switch (c) {
...
case '\\':
c = this.next();
switch (c) {
...
case '"':
case '\'':
case '\\':
case '/':
sb.append(c);
break;
...
At the top of the method is an informative comment:
The formal JSON format does not allow strings in single quotes, but an implementation is allowed to accept them.
So some implementations will accept single quotes - but you should not rely on this. Many popular implementations are quite restrictive in this regard and will reject JSON that contains single quoted strings and/or escaped single quotes.
Finally to tie this back to the original question, jQuery.parseJSON first attempts to use the browser's native JSON parser or a loaded library such as json2.js where applicable (which on a side note is the library the jQuery logic is based on if JSON is not defined). Thus jQuery can only be as permissive as that underlying implementation:
parseJSON: function( data ) {
...
// Attempt to parse using the native JSON parser first
if ( window.JSON && window.JSON.parse ) {
return window.JSON.parse( data );
}
...
jQuery.error( "Invalid JSON: " + data );
},
As far as I know these implementations only adhere to the official JSON specification and do not accept single quotes, hence neither does jQuery.

If you need a single quote inside of a string, since \' is undefined by the spec, use \u0027 see http://www.utf8-chartable.de/ for all of them
edit: please excuse my misuse of the word backticks in the comments. I meant backslash. My point here is that in the event you have nested strings inside other strings, I think it can be more useful and readable to use unicode instead of lots of backslashes to escape a single quote. If you are not nested however it truly is easier to just put a plain old quote in there.

I understand where the problem lies and when I look at the specs its clear that unescaped single quotes should be parsed correctly.
I am using jquery`s jQuery.parseJSON function to parse the JSON string but still getting the parse error when there is a single quote in the data that is prepared with json_encode.
Could it be a mistake in my implementation that looks like this (PHP - server side):
$data = array();
$elem = array();
$elem['name'] = 'Erik';
$elem['position'] = 'PHP Programmer';
$data[] = json_encode($elem);
$elem = array();
$elem['name'] = 'Carl';
$elem['position'] = 'C Programmer';
$data[] = json_encode($elem);
$jsonString = "[" . implode(", ", $data) . "]";
The final step is that I store the JSON encoded string into an JS variable:
<script type="text/javascript">
employees = jQuery.parseJSON('<?=$marker; ?>');
</script>
If I use "" instead of '' it still throws an error.
SOLUTION:
The only thing that worked for me was to use bitmask JSON_HEX_APOS to convert the single quotes like this:
json_encode($tmp, JSON_HEX_APOS);
Is there another way of tackle this issue? Is my code wrong or poorly written?
Thanks

When You are sending a single quote in a query
empid = " T'via"
empid =escape(empid)
When You get the value including a single quote
var xxx = request.QueryString("empid")
xxx= unscape(xxx)
If you want to search/ insert the value which includes a single quote in a query
xxx=Replace(empid,"'","''")

Striking a similar issue using CakePHP to output a JavaScript script-block using PHP's native json_encode. $contractorCompanies contains values that have single quotation marks and as explained above and expected json_encode($contractorCompanies) doesn't escape them because its valid JSON.
<?php $this->Html->scriptBlock("var contractorCompanies = jQuery.parseJSON( '".(json_encode($contractorCompanies)."' );"); ?>
By adding addslashes() around the JSON encoded string you then escape the quotation marks allowing Cake / PHP to echo the correct javascript to the browser. JS errors disappear.
<?php $this->Html->scriptBlock("var contractorCompanies = jQuery.parseJSON( '".addslashes(json_encode($contractorCompanies))."' );"); ?>

I was trying to save a JSON object from a XHR request into a HTML5 data-* attribute. I tried many of above solutions with no success.
What I finally end up doing was replacing the single quote ' with it code ' using a regex after the stringify() method call the following way:
var productToString = JSON.stringify(productObject);
var quoteReplaced = productToString.replace(/'/g, "'");
var anchor = '<a data-product=\'' + quoteReplaced + '\' href=\'#\'>' + productObject.name + '</a>';
// Here you can use the "anchor" variable to update your DOM element.

Interesting. How are you generating your JSON on the server end? Are you using a library function (such as json_encode in PHP), or are you building the JSON string by hand?
The only thing that grabs my attention is the escape apostrophe (\'). Seeing as you're using double quotes, as you indeed should, there is no need to escape single quotes. I can't check if that is indeed the cause for your jQuery error, as I haven't updated to version 1.4.1 myself yet.