Develop Reference

in node.js how to sanitize api requests that failed input validation for future security review

I need to sanitize api requests that failed input validation for future security review.
I have it in json format and I need to keep them in json format.
this is for example valid json but I can't log it to our system like that:
{
"<script>alert('test!!!!');</script>": "<script>alert('xss12!!!!');</script>"
}
I need a lib or some way in node.js to sanitize it and keep it json and then log it, for example convert it to utf:
{
"#utf3c#script#utf3c#alert('test#utf3c##utf3c##utf3c##utf3c#')#utf3c##utf3c##utf3c#script#utf3c#": "#utf3c#script#utf3c#alert('xss12#utf3c##utf3c##utf3c#')#utf3c##utf3c##utf3c#script#utf3c#"
}
I had the idea to convert it to string with JSON.stringify convert to codePointAt with this replacer with string replace and then convert back to json with JSON.parse but the string it generate can't be converted back to json easily, with complex nested json it breaks.
function replacer(match, index, wholeString) {
const result = "codePoint#" + wholeString.codePointAt(index);
return result;
}
const dataToEncodeAsString = JSON.stringify(dataToEncode);
const test = dataToEncodeAsString.replace(/[^a-zA-Z1-9:'" ]/g, replacer);
console.log(test);

I found this package flat
and used it to flat the structure of the deeply nested JSON and then clean all chars from its keys and values and then used flat again with unflatten function to bring the JSON to its original structure.
hope it will help others that need this kind of solution!

How to convert a string to JSON format?

I was getting list in array form. So at first place i converted array list to string-
var myJsonString = JSON.stringify(result);
myJsonString="[{"productId":"PI_NAME",
"firstName":null,
"lastName":null,
"customer":null
},
{"productId":"PI_NAME",
"firstName":null,
"lastName":null,
"customer":null
}]"
But again i need to convert myJsonString to Json format, What i need to do? I mean i need to replace 1st" and last ", I guess

You need to call parse now.
JSON.parse(myJsonString)

First, if you ever find yourself building a JSON string by concatenating strings, know that this is probably the wrong approach.
I don't really understand how the first line of your code relates to the second, in that you are not doing anything with JSON-encoded string output from result, but instead just overwriting this on the following line.
So, I am going to limit my answer to show how you could better form JSON from an object/array definition like you have. That might look like this:
// build data structure first
// in this example we are using javascript array and object literal notation.
var objArray = [
{
"productId":"PI_NAME",
"firstName":null,
"lastName":null,
"customer":null
},{
"productId":"PI_NAME",
"firstName":null,
"lastName":null,
"customer":null
}
];
// now that your data structure is built, encoded it to JSON
var JsonString = JSON.stringify(objArray);
Now if you want to work with JSON-encoded data, You just do the opposite:
var newObjArray = JSON.parse(JsonString);
These are really the only two commands you should ever use in javascript when encoding/decoding JSON. You should not try to manually build or modify JSON strings, unless you have a very specific reason to do so.

Handle Data returned by POST request in Javascript

I have a PHP script - it returns a file like this after POSTing to it with the help of $.post() using jQuery.
13.0519 77.5416
13.028 77.5409
12.9787 77.5724
12.9317 77.6227
How do I go through the lines of data returned in the $.post().done(function(data){}) in Javascript ?

You can split returned text by newline character and then loop over resulting array of lines, probably splitting one more time by space to get array of numbers:
$.post(/*...*/).done(function(data) {
var lines = data.split(/\n/);
lines.forEach(function(line) {
console.log(line.split(' '));
});
});

The best way is to have PHP return JSON (with an array of these).
If not possible, however, then you can get the string and split it on new lines, to achieve a similar effect. Something along the lines of
var arrayOfResults = postData.split("\n");
Then just iterate and do whatever is needed.

How to convert the json results to array using javascript?

I need to convert the Json response to array using JavaScript .
I have the json response which is :
["simmakkal madurai","goripalayam madurai"].
now I want to change this results to array format .
So, How to I convert the json result to array.

Try this :-
var obj = $.parseJSON('["simmakkal madurai","goripalayam madurai"]');
If you alert(obj[0]), you will get answer as simmakkal madurai

var arr = JSON.parse('["simmakkal madurai","goripalayam madurai"]');

Yeah,
if your response is
var res = '["simmakkal madurai","goripalayam madurai"]';
Then you just need to do
var arr = JSON.parse(res);
Alternativly you could do an eval but evals are never realy recommended, but since you are going from server side to client side (php -> javascript) it wouldnt the the worst thing.
Anyways JSON.parse should work

How to pass an array of integers as a parameter from javascript to python?

I have a javascript code that obtains the value of several checked boxes and inserts their value (integers) into an array:
var my_list = $('.item:checked').map(function(){return $(this).attr('name');}).get();
I want to pass this javascript array to a python function as a paramter. This is how I am doing it right now, with ajax and jQuery:
$.ajax({
url : "{{tg.url('/foo/bar')}}",
data : {
my_list: JSON.stringify(my_list),
...
},
On the python side, I have this code, to get the parameters:
item_list = get_paramw(kw, 'my_list', unicode)
This works, but I am not receiving the array as an array of integers, but as a single string containing the "[", "]", "," and """ symbols, which I would have to parse and isn't the most elegant way of dealing with this whole situation I think.
How should it be done to receive a javascript array of integers as an array of integers in python (not as a string)?

The easiest way to send simple arbitrarily-structured data from Javascript to Python is JSON. You've already got the Javascript side of it, in that JSON.stringify(my_list). All you need is the Python side. And it's one line of code:
item_list_json = get_paramw(kw, 'my_list', unicode)
item_list = json.loads(item_list_json)
If item_list in your JS code were the Javascript array of numbers [1, 2, 3, 4], item_list in your Python code would be the Python list of ints [1, 2, 3, 4].
(Well, you also have to import json at the top of your script, so I guess it's two lines.)

The proper way to send multiple values for one key is
data: {
my_list: my_list,
},
traditional: true
...
Suppose my_list = [ 1, 2, 3 ], this results a query string / POST data
my_list=1&my_list=2&my_list=3
Without the traditional flag, the result would be
my_list[]=1&my_list[]=2&my_list[]=3
Which of course works, but in TurboGears you need to access it as my_list[].
On the other hand, I am not sure if the TG keyword argument passing work with multiple values; if you get a single value in kw, you can use request.getall('foo') to return all the foo keys as a list (possibly an empty list).