Develop Reference

Writing special characters to excel file using xlsx-populate

I am using xlsx-populate to write to excel files. Whenever the text contains special characters such as smart quotes or bullets points, it gets printed as weird character like ’. However if replace them with their Unicode value (Right Single Quotation mark with \u2019), it renders correctly in excel sheet.
I tried js libraries like slugify to convert the special characters into ascii version but all of them would individually convert every constituent byte of special char and not as whole.
Currently, I replace the special character with their Unicode value using regex but there will be always some character that I will miss. Is there a better way to handle this problem?

I used iconv-lite library to first encode the string into 'win1252' and then decoded it with 'utf-8'. The resulting string renders correctly when written to excel.
iconv.decode(iconv.encode(string, 'win1252'), 'utf-8')
Thanks #JosefZ for suggesting this path.

Backslash in string keeps being treated as making an escaped character

I have a string returned from a javascript API call they are like these:
"#|net\brown_gf"
"#|net\nugent_he"
I then need to use these strings as they are to pass them to another api call on a different system. The problem is \b or \n of the string are being treated as escaped characters and nothing I have tried allows me to add an extra backslash to escape the slash.
The string I recieve is not able to be changed as it is from an api I have no access to modify and no choice but to use.
I am guessing my only option is to look for every escaped character combination and convert them back to what I want. Any simpler solution is requested, or a way to tell javascript that the string is a literal string with 0 escaped characters in it

How can I replace a backslash with a double backslash using RegExp?

I need to modify the value using javascript, to make it ready to be put as part of a SQL insert query.
Currently I have the following code to handle the single quote ' character.
value = value.replace(/'/g, "\\'");
This code works without an issue. Now I noticed that stand-alone backslashes are causing errors.
How can I remove those stand alone backslashes?

Now I noticed that stand-alone backslashes are causing errors.
Backslashes in the string you're operating on won't have any effect on replacing ' characters whatsoever. If your goal is to replace backslash characters, use this:
value = value.replace(/\\/g, "whatever");
...which will replace all backslashes in the string with "whatever". Note that I've had to write two backslashes rather than just one. That's because in a regular expression literal, the backslash is used to introduce various special characters and character classes, and is also used as an escape — two backslashes together in a regular expression literal (as in a string) represent a single actual backslash in the string.
To change a single backslash into two backslashes, use:
value = value.replace(/\\/g, "\\\\");
Note that, again, to get a literal backslash in the replacement string, we have to escape each of the two — resulting in four in total in the replacement string.
I need to modify the value using javascript, to make it ready to be put as part of a SQL insert query.
You don't want to do this by hand. Any technology that allows you to make database queries and such (JDBC, ODBC, etc.) will provide some form of prepared or parameterized statement (link), which deals with these sorts of escaping issues for you. Doing it yourself is virtually guaranteed to leave security holes in your software which could be exploited. You want to use the work of a team that's had to think this through, and which updates the resulting code periodically as issues come to light, rather than flying alone. Further, if your JavaScript is running on the client (as most is, but by no means all — I use JavaScript server-side all the time), then nothing you do to escape the string can make it safe, because client requests to the server can be spoofed, completely bypassing your client-side code.

You should use a escape function provided by some kind of database library, rolling your own will only cause trouble.

Escape dynamic strings in JavaScript

I'm writing a script for a signature function in a forum program and any time someone puts a quote or some other JavaScript parse-able character into it, it breaks my program.
Is there a way either to force JavaScript to recognize it as a string without parsing it as script or, failing that, a function that escapes all scripting within a string that will be dynamic?
I did a search and all I could find were endless webpages on how to escape individual characters with a slash - perhaps my search skills need work.

Are you putting the contents of the signature using a server-side language, dynamically, in a JavaScript string literal? That probably isn't the best way to go; you may want to reconsider the way you are doing it.
For example, a better way to do it could be that you could just have an element on the page for the signature (which doesn't have to be visually distinct) and then get the contents of that for use in the script during JavaScript runtime.
If you still wanted to take the route you are going, you could replace ' with \' (or " with \" if you are using double-quoted strings in your script) and replace \n with \\n, which replaces real newlines with newline escapes.

Why aren't double quotes and backslashes allowed in strings in the JSON standard?

If I run this in a JavaScript console in Chrome or Firebug, it works fine.
JSON.parse('"\u0027"') // Escaped single-quote
But if I run either of these 2 lines in a Javascript console, it throws an error.
JSON.parse('"\u0022"') // Escaped double-quote
JSON.parse('"\u005C"') // Escaped backslash
RFC 4627 section 2.5 seems to imply that \ and " are allowed characters as long as they're properly escaped. The 2 browsers I've tried this in don't seem to allow it, however. Is there something I'm doing wrong here or are they really not allowed in strings? I've also tried using \" and \\ in place of \u0022 and \u005C respectively.
I feel like I'm just doing something very wrong, because I find it hard to believe that JSON would not allow these characters in strings, especially since the specification doesn't seem to mention anything that I could find saying they're not allowed.

You need to quote the backslash!
that which we call a rose
By any other name would smell as sweet
A double quote is a double quote, no matter how you express it in the string constant. If you put a backslash before your \u expression within the constant, then the effect is that of a backslash-quoted double-quote, which is in fact what you've got.
The most interesting thing about your question is that it helps me realize that every JavaScript string parser I've ever written is wrong.

JavaScript is interpreting the Unicode escape sequences before they get to the JSON parser. This poses a problem:
'"\u0027"' unquoted the first time (by JavaScript): "'" The second time (by the JSON parser) as valid JSON representing the string: '
'"\u0022"' unquoted the first time (by JavaScript): """ The JSON parser sees this unquoted version """ as invalid JSON (does not expect the last quotation mark).
'"\u005C"' unquoted the first time (by JavaScript): "\" The JSON parser also sees this unquoted version "\" as invalid JSON (second quotation mark is backslash-escaped and so does not terminate the string).
The fix for this is to escape the escape sequence, as \\u.... In reality, however, you would probably just not use the JSON parser. Used in the correct context (ensured by wrapping it within parentheses, all JSON is valid JavaScript.

You are not escaping the backslash and the double-quote, as \u00xx is being interpreted by the javascript parser.
JSON.parse('"\\\u0022"')
JSON.parse('"\\\""')
and
JSON.parse('"\\\u005C"')
JSON.parse('"\\\\"')
work as expected.