I have a string returned from a javascript API call they are like these:
"#|net\brown_gf"
"#|net\nugent_he"
I then need to use these strings as they are to pass them to another api call on a different system. The problem is \b or \n of the string are being treated as escaped characters and nothing I have tried allows me to add an extra backslash to escape the slash.
The string I recieve is not able to be changed as it is from an api I have no access to modify and no choice but to use.
I am guessing my only option is to look for every escaped character combination and convert them back to what I want. Any simpler solution is requested, or a way to tell javascript that the string is a literal string with 0 escaped characters in it
Related
I am using xlsx-populate to write to excel files. Whenever the text contains special characters such as smart quotes or bullets points, it gets printed as weird character like ’. However if replace them with their Unicode value (Right Single Quotation mark with \u2019), it renders correctly in excel sheet.
I tried js libraries like slugify to convert the special characters into ascii version but all of them would individually convert every constituent byte of special char and not as whole.
Currently, I replace the special character with their Unicode value using regex but there will be always some character that I will miss. Is there a better way to handle this problem?
I used iconv-lite library to first encode the string into 'win1252' and then decoded it with 'utf-8'. The resulting string renders correctly when written to excel.
iconv.decode(iconv.encode(string, 'win1252'), 'utf-8')
Thanks #JosefZ for suggesting this path.
In HTML and JavaScript sometimes you have to replace certain characters with special escaped versions.
For instance, JavaScript & would be changed to HTML &, and JavaScript < would be changed to HTML <.
What is this process called, and are there native or non-native JavaScript routines that can perform this task bi-directionally?
Thanks.
This special characters called Entity , and its text that begins with an ampersand (&) and ends with a semicolon (;), Some special characters are reserved for use in HTML, meaning that your browser will parse them as HTML code. For example, if you use the less-than (<) sign, the browser interprets any text that follows as a tag. so if you want to display that special charterer you need to write its entity , and its the browser job to unescape or mapping
the chars to their corresponding values
so its escape and unescape or mapping
and its something that the browser did himself when you write the entity
you can see a list of html entities
Character entity reference
You can use escape() and unescape() js functions alternatively.
I need to modify the value using javascript, to make it ready to be put as part of a SQL insert query.
Currently I have the following code to handle the single quote ' character.
value = value.replace(/'/g, "\\'");
This code works without an issue. Now I noticed that stand-alone backslashes are causing errors.
How can I remove those stand alone backslashes?
Now I noticed that stand-alone backslashes are causing errors.
Backslashes in the string you're operating on won't have any effect on replacing ' characters whatsoever. If your goal is to replace backslash characters, use this:
value = value.replace(/\\/g, "whatever");
...which will replace all backslashes in the string with "whatever". Note that I've had to write two backslashes rather than just one. That's because in a regular expression literal, the backslash is used to introduce various special characters and character classes, and is also used as an escape — two backslashes together in a regular expression literal (as in a string) represent a single actual backslash in the string.
To change a single backslash into two backslashes, use:
value = value.replace(/\\/g, "\\\\");
Note that, again, to get a literal backslash in the replacement string, we have to escape each of the two — resulting in four in total in the replacement string.
I need to modify the value using javascript, to make it ready to be put as part of a SQL insert query.
You don't want to do this by hand. Any technology that allows you to make database queries and such (JDBC, ODBC, etc.) will provide some form of prepared or parameterized statement (link), which deals with these sorts of escaping issues for you. Doing it yourself is virtually guaranteed to leave security holes in your software which could be exploited. You want to use the work of a team that's had to think this through, and which updates the resulting code periodically as issues come to light, rather than flying alone. Further, if your JavaScript is running on the client (as most is, but by no means all — I use JavaScript server-side all the time), then nothing you do to escape the string can make it safe, because client requests to the server can be spoofed, completely bypassing your client-side code.
You should use a escape function provided by some kind of database library, rolling your own will only cause trouble.
why does encodeURI and encodeURIComponent encode spaces as hex values, but then I see other encodings using the plus sign? there's something i'm obviously missing.
thanks!
IIRC + is a form encoded , while %20 is a standard URI encoding.
They are interchangeable, so don't worry about which one you use.
"+" is allowed as a substitute for spaces, however there are lots of other special characters that need escaping as hex values (in the form %nn). Presumably the authors of encodeURI and encodeURIComponent decided to use hex values for everything including space, since it made their code simpler and they didn't think the extra two characters for each space in a uri was really that important.
Look here for a discussion of the differences between escape, encodeURI, and encodeURIComponent, with interactive examples for all three of them:
http://xkr.us/articles/javascript/encode-compare/
To summarize:
The escape() method does not encode
the + character which is interpreted
as a space on the server side as well
as generated by forms with spaces in
their fields. Due to this shortcoming
and the fact that this function fails
to handle non-ASCII characters
correctly, you should avoid use of
escape() whenever possible. The
best alternative is usually
encodeURIComponent().
escape() will not encode: #*/+
Use of the encodeURI() method is a
bit more specialized than escape()
in that it encodes for
URIs
as opposed to the querystring, which
is part of a URL. Use this method when
you need to encode a string to be used
for any resource that uses URIs and
needs certain characters to remain
un-encoded. Note that this method does
not encode the ' character, as it is a
valid character within URIs.
encodeURI() will not encode:
~!##$&*()=:/,;?+'
Lastly, the encodeURIComponent()
method should be used in most cases
when encoding a single component of a
URI. This method will encode certain
chars that would normally be
recognized as special chars for URIs
so that many components may be
included. Note that this method does
not encode the ' character, as it is a
valid character within URIs.
encodeURIComponent() will not
encode: ~!*()'
If I run this in a JavaScript console in Chrome or Firebug, it works fine.
JSON.parse('"\u0027"') // Escaped single-quote
But if I run either of these 2 lines in a Javascript console, it throws an error.
JSON.parse('"\u0022"') // Escaped double-quote
JSON.parse('"\u005C"') // Escaped backslash
RFC 4627 section 2.5 seems to imply that \ and " are allowed characters as long as they're properly escaped. The 2 browsers I've tried this in don't seem to allow it, however. Is there something I'm doing wrong here or are they really not allowed in strings? I've also tried using \" and \\ in place of \u0022 and \u005C respectively.
I feel like I'm just doing something very wrong, because I find it hard to believe that JSON would not allow these characters in strings, especially since the specification doesn't seem to mention anything that I could find saying they're not allowed.
You need to quote the backslash!
that which we call a rose
By any other name would smell as sweet
A double quote is a double quote, no matter how you express it in the string constant. If you put a backslash before your \u expression within the constant, then the effect is that of a backslash-quoted double-quote, which is in fact what you've got.
The most interesting thing about your question is that it helps me realize that every JavaScript string parser I've ever written is wrong.
JavaScript is interpreting the Unicode escape sequences before they get to the JSON parser. This poses a problem:
'"\u0027"' unquoted the first time (by JavaScript): "'" The second time (by the JSON parser) as valid JSON representing the string: '
'"\u0022"' unquoted the first time (by JavaScript): """ The JSON parser sees this unquoted version """ as invalid JSON (does not expect the last quotation mark).
'"\u005C"' unquoted the first time (by JavaScript): "\" The JSON parser also sees this unquoted version "\" as invalid JSON (second quotation mark is backslash-escaped and so does not terminate the string).
The fix for this is to escape the escape sequence, as \\u.... In reality, however, you would probably just not use the JSON parser. Used in the correct context (ensured by wrapping it within parentheses, all JSON is valid JavaScript.
You are not escaping the backslash and the double-quote, as \u00xx is being interpreted by the javascript parser.
JSON.parse('"\\\u0022"')
JSON.parse('"\\\""')
and
JSON.parse('"\\\u005C"')
JSON.parse('"\\\\"')
work as expected.