Develop Reference

Is jQuery / Backbone mangling my UTF-8 characters?

I'm migrating the front-end of a site from an old YUI2 framework to jQuery/BackBone. The PHP/mySQL back-end hasn't changed. All is well, except UTF-8 characters sent via Backbone save (via $.ajax) are getting mangled and I can't figure out why.
Here's what I do know:
The backend handles UTF-8 fine. It hasn't changed as part of this rebuild. I know that's true, because when I change the config to load the old YUI2 front-end, UTF-8 characters work fine. They're escaped in Javascript using escape(string), passed via YAHOO.util.Connect.asyncRequest as JSON in an XMLHttpRequest, unescaped and saved in the database as UTF-8, fully readable and nice.
In the new front-end, I've added <meta charset="UTF-8"> and <meta http-equiv="content-type" content="text/html; charset=UTF-8"> to all page headers. The old front-end didn't have these settings. I only mention that because it's a difference.
In the new front-end, UTF-8 characters work fine when I save them as a <form> submit.
I the new front-end, the request Content-Type looks fine in the console. Content-Type:application/x-www-form-urlencoded; charset=UTF-8
How am I passing data in the new front-end?
Sometimes via a regular Backbone model.save(), other times passing data in options like this:
var text = $('#input-' + targetId).val();
var atts = {};
atts['target_id'] = targetId;
atts['user_id'] = userId;
atts['text'] = text;
var comment = new Comment(atts);
comment.save(
{},
{
type: 'POST',
url: '/api/comment?',
data: atts,
processData: true,
success: function(comment, response){
//success handling
},
error: function(model, response){
//error handling
},
},
);
So, what do these mangled special characters look like?
As entered in the input: テクス テクサン テクス テクサン
When I pass completely unescaped, they look fine in the request in the console in the Form Data section: text: テクス テクサン テクス テクサン, but mangled in the database as ãã¯ã¹ ãã¯ãµã³ ãã¯ã¹ ãã¯ãµã³. Perhaps this is a clue, I don't know. I've always escaped user-entered text when passing via AJAX.
When I escape(text), I get text:%u30C6%u30AF%u30B9%20%u30C6%u30AF%u30B5%u30F3%20%u30C6%u30AF%u30B9%20%u30C6%u30AF%u30B5%u30F3 in the console, and テクス%20テクサン%20テクス%20テクサン in the database.
That's better, but it's different from the old front end, which uses escape(text), passes %u30C6%u30AF%u30B9%20%u30C6%u30AF%u30B5%u30F3%20%u30C6%u30AF%u30B9%20%u30C6%u30AF%u30B5%u30F3, shows in the console as text: (unable to decode value) and saves in the database unescaped as テクス テクサン テクス テクサン
Of course, it's 2016 now and we all know escape() should not be used. We should use encodeURIComponent() instead. So, when I encodeURIComponent(text), here's what I get in the console: text: %E3%83%86%E3%82%AF%E3%82%B9%20%E3%83%86%E3%82%AF%E3%82%B5%E3%83%B3%20%E3%83%86%E3%82%AF%E3%82%B9%20%E3%83%86%E3%82%AF%E3%82%B5%E3%83%B3 which is saved in the database as %E3%83%86%E3%82%AF%E3%82%B9%20%E3%83%86%E3%82%AF%E3%82%B5%E3%83%B3%20%E3%83%86%E3%82%AF%E3%82%B9%20%E3%83%86%E3%82%AF%E3%82%B5%E3%83%B3 That technically works, and I can always decodeURIComponent when displaying this text, but that's a real pain and it's just masking the issue.
I've also tried unescape(encodeURIComponent(text)) with the following result: text:ãã¯ã¹ ãã¯ãµã³ ãã¯ã¹ ãã¯ãµã³ in the console, ãÂÂã¯ã¹ ãÂÂã¯ãµã³ ãÂÂã¯ã¹ ãÂÂã¯ãµã³ in the database.
It seems that there's some sort of double-encoding going on, or perhaps the back-end was built to handle the specific format that's passed via the YUI2 Async request. I don't know.
Any ideas for what I should try next? What are the best practices?

Now that I've had a night to sleep on it, I've realized a few things and I think I've found a solution.
It's clear now that the old front-end wasn't passing data correctly...that's evidenced by the text: (unable to decode value) in the console when sending the request. Somehow, the PHP back-end was able to handle the passed text even though there was no decoding in the api or db storage classes. That's a mystery for another day.
Here's what I did to fix the problem:
Pass text from the front-end as encodeURIComponent(text)
Decode the text in the PHP back-end api using $comment->set_text(urldecode(Request::get('text')));
The text is stored in the DB unescaped as readable UTF-8 characters and I don't need to do anything special on read/display. I will need to add the urldecode to all of my api endpoints on the back-end, but that feels like a solid approach, so I think it's resolved.
I'd be interested to hear thoughts on the use of encodeURIComponent on the front-end and urldecode on the back-end. Is this the best way to solve the problem?

Update XML on server without PHP

I have a problem. I have a JavaScript function which should update an XML file (btw please check if its correct):
function changename(node2){
var nodenumber = node2;
var newname = "tescik";
$.ajax({
type: "GET",
url: "config2.xml",
dataType: "xml",
success: function(xml) {
$(xml).find('device').each(function () {
var node = $(this).find('node');
if (node.text() == nodenumber) {
var name = $(this).find('name').text(newname);
alert(name.text());
$.post(
"config2.xml",
{
name: "Bravo"
},
function(dane){
alert("Dane otrzymane: " + dane);
}
);
}
});
}
});
}
The problem now is: How to update that file on the web server? I can not install PHP server on this. There can be only a web server. Nothing else.
Please send my any instruction or materials which can help me.

You can't save anything to a server without any server side technology. Either you use PHP, ASP.NET or any similar server side technology or leave it.
There must be a service on the other side dictating what to do with the file it receives. You can't just copy the file over there (if you want to, use FTP or something like that)
Just realize what it will mean if this is possible:
What if I could just change a file on your webserver. Would you like that? No, of course not! At least there must be some authorization, authentication, and some software that tells where to place the file.

That is not possible. If the server does not provide some server-side way to update a file, then you can’t do it.
Imagine if a simple JavaScript file that is executed on the client’s machine could modify files on the server. That would be highly insecure. Instead, you will need at least something that processes this change on the server itself. And it’s really recommended to add some validation there too, so that one cannot just store anything (for example malicious code).

Javascript - Sanitize Malicious code from file (string)

I have a data javascript file, which is being dynamically added to website via some custom code.
This file comes from a third party vendor, who could potentially add malicious code in the file
Before this file is added to the website, I would like to parse through it, and look for malicious code, such as redirects or alerts, that inherently get executed upon a files inclusion in the project/website.
For example, my js file could look like this :
alert ('i am malicious');
var IAmGoodData =
[
{ Name :'test', Type:'Test2 },
{ Name :'test1', Type:'Test21' },
{ Name :'test2', Type:'Test22' }
]
I load this file into a object via a XMLHttpRequest call, and when this call returns, I can use the variable (which is my file text) and search it for words:
var client = new XMLHttpRequest();
client.open('GET', 'folder/fileName.js');
client.onreadystatechange = function()
{
ScanText(client.responseText);
}
client.send();
function ScanText(text)
{
alert(text);
var index = text.search('alert'); //Here i can search for keywords
}
The last line would return index of 0, as the word alert is found at index 0 in the file.
Questions:
Is there a more efficient way to search for keywords in the file?
What specific keywords should i be searching for to prevent malicious code being run? ie redirects, popups, sounds etc.....

Instead of having them include var IAmGoodData =, make them simply provide JSON (which is basically what the rest of the file is, or seems to be). Then you parse it as JSON, using JSON.parse(). If it fails, they either didn't follow the JSON format well, or have external code, and in either case you would ignore the response.
For example, you'd expect data from the external file like:
[
{ Name :'test', Type:'Test2' },
{ Name :'test1', Type:'Test21' },
{ Name :'test2', Type:'Test22' }
]
which needs to be properly serialized as JSON (double quotes instead of single quotes, and double quotes around the keys). In your code, you'd use:
var json;
try {
json = JSON.parse(client.responseText);
catch (ex) {
// Invalid JSON
}
if (json) {
// Do something with the response
}
Then you could loop over json and access the Name and Type properties of each.
Random Note:
In your client.onreadystatechange callback, make sure you check client.readyState === 4 && client.status === 200, to know that the request was successful and is done.

This is extremely difficult to do. There are no intrinsically malicious keywords or functions in JavaScript, there are malicious applications. You could be getting false positives for "malicious" activity and prevent a legitimate code with a real purpose from being executed. And at the same time, anyone with a little bit of imagination could bypass any "preventive" method you may implement.
I'd suggest you look for a different approach. This is one of those problems (like CAPTCHA) in which it's trivial for a human to solve while for a machine is practically impossible to do so. You could try having a moderator or some human evaluator to interpret the code and accept it.

You should have them provide valid JSON rather than arbitrary Javascript.
You can then call JSON.parse() to read their data without any risk of code execution.
In short, data is not code, and should not be able to contain code.

You shouldn't. The user should be allowed to type whatever they want, and it's your job to display it.
It all depends on where it is being put, of course:
Database: mysql_real_escape_string or equivalent for whatever engine you're using.
HTML: htmlspecialchars in PHP, createTextNode or .replace(/</g,"<") in JavaScript
JavaScript: json_encode in PHP, JSON.stringify in JavaScript.
At the end of the day, just don't be Yahoo

Error while writing a Google CCS server in node using node-xmpp

This is python code in Google documentation of writing a CCS server:
https://developer.android.com/google/gcm/gs.html#server
I figured out most of it, and how to code it in Javascript using https://github.com/astro/node-xmpp
But I'm failing to understand how to send data using templates, precisely this part of the code:
def send(json_dict):
template = ("<message><gcm xmlns='google:mobile:data'>{1}</gcm></message>")
client.send(xmpp.protocol.Message(
node=template.format(client.Bind.bound[0], json.dumps(json_dict))))
where in node-xmpp, a send is done by this:
var cl = new xmpp.Client({ jid: username,
password: password });
cl.addListener('online',
function() {
argv.slice(5).forEach(
function(to) {
cl.send(new xmpp.Element('message',
{ to: to,
type: 'chat'}).
c('body').
t(argv[4]));
});
I understand the JSON being sent, But I'm not able to do the binding of the template that they are managing in Python. Any help?

The important part is to send the message in the required format :
<message id="">
<gcm xmlns="google:mobile:data">
{
"to":"REGISTRATION_ID", // "to" replaces "registration_ids"
"message_id":"m-1366082849205" // new required field
"data":
{
"hello":"world",
}
"time_to_live":"600",
"delay_while_idle": true/false
}
</gcm>
</message>
It doesn't matter if you use a template or not. I don't know python nor javascript, but the purpose of the template in the python example seems to be simply to avoid the need to write the xml tags that wrap the JSON every time you send a message. You can append them to the JSON when you send the message.

Encode string javascript so that it can be transmitted to a server

I'm trying to send json string in the get request to the server, here is how it looks before encoding :
filters={"groupOp":"AND","rules":[{"field":"countrycode","op":"eq","data":"ARG"}]}
Naturally I end up with null pointer when trying to get this json string, then I googled this encodeURIComponent and it partially encodes this string like this :
filters={"groupOp"%3A"AND"%2C"rules"%3A[{"field"%3A"countrycode"%2C"op"%3A"eq"%2C"data"%3A"ARG"}]}
But this is how it supposed to be in order to work :
filters=%7B%22groupOp%22%3A%22AND%22%2C%22rules%22%3A%5B%7B%22field%22%3A%22countrycode%22%2C%22op%22%3A%22eq%22%2C%22data%22%3A%22ARG%22%7D%5D%7D
How do I get this, entirely encoded string so I can read it at server side properly ?
Reason why I used get instead of post
I'm sending this filter(json) content to the server side, web service gets data from the database and returns pdf document.
Using post, I'm able to send correct data and the response is successfully displayed in my firebug console. But I need to return pdf doc to override the current page or open new window/tab and return in that one.

I think you're overworking this problem. Or encoding too many times. Or something.
You've got a JSON string, and you are trying to JSON encode it. That seems...unhelpful.
A better approach might be to produce a Javascript object, then JSON.Stringify that, and then transmit it as a parameter.
var thing = {
groupOp : "AND",
rules : [
{ field : "countrycode", op : "eq", data : "ARG" },
...
],
...
};
var stringrep = JSON.stringify(thing);
// post via jQuery
$.ajax({
type: 'POST',
url: url,
data: stringrep,
dataType: 'json'
success: function() { ... },
});
Normally for a JSON stringified message to or from the server, you'd want to use HTTP POST. HTTP GET puts all "parameters" in the URL; there is no message body. In contrast, HTTP POST allows you to attach a message body to the HTTP message, which can be "anything". With that approach, you don't need to url-encode the quotes and spaces; the JSON message just gets transmitted as the message body of the HTTP message.
HTTP POST is the way applications upload images, or transmit XML documents, and so on. Anything complex gets transmitted via POST.

var filtersParameter = 'filters=' + encodeURI(JSON.stringify(filters));

var filtersParameter = 'filters=' + atob(JSON.stringify(filters));
Note: atob() method uses base64 algorithm to encode the data. This encoded data can be easily passed to a server where it can be decoded using corresponding decoding methods (in python base64.b64decode(encoded_string) is used).