I'm going mad with string encoding between js/php/sql
I created a web site, with a public and an administrative part. I use javascript in the front end (I'm not using jquery). Php and mysqli in backend.
In the public part, I have some text in differents parts : as full visible text ans in tag like title="", alt=""
In the administrative part, the owner of the site can add text from input[text] element. He can write any text he want, so I need to encode everything correctly. Then the text is send to a php page by a xmlHttpRequest/JSON
[removed text, I edit the question below]
If I have a quote or " ; or ", or & or & ; or #xx;, the text in the title tag or in the display text will be different.
Is there a clear/simple way to send/retreive/display data from js to php and from php to js ?
EDIT
I didn't send code because I try lots of possibilities, and never ever found a solution.
I try to explain better
admin :
the user can add photos, with title and text to each. I use an input for title and text.
When he save the data
for(i=0;i<listimage.length;i++)
{
listimage[i]['title']=encode(document.getElementById("inputtitle"+i).value);
listimage[i]['texte']=encode(document.getElementById("imputtext"+i).value);
}
gdata={big:masonrybig,larg,listimage:listgalerieimage};
let postdata="key="+keypage+"&data="+JSON.stringify(gdata);
In the encode function, I try lots and lots of thing
function encode(str)
{
return encodeURIcomponent(str);
return encodeURIcomponent(he.encode(str));
return he.encode(encodeURIcomponent(str));
return myOwnEncodingFunction(str); //like htmlspecialchar, htmlentities,..
}
in php :
$data=json_decode($_POST["data"]); //always, the data received are JSON string
//and one or more of the following functions
$data=trim($data);
$data=stripslashes($data);
$data=strip_tags($data);
$data=htmlspecialchars($data);
$data=mysqli_real_escape_string($data);
$data=htmlentities($data);
$data=str_replace('\','\',$data);
$data=str_replace('"','\"',$data);
$data=str_replace("'","\'",$data);
$data=addslashes($data);
$data=htmlspecialchars($data,ENT_NOQUOTES | ENT_SUBSTITUTE | ENT_HTML401, "UTF-8");
And when I want to display the data
reading from the database
while ($row = $result->fetch_object())
{
$image=array();
$image['title']=$row->title;
$image['text']=$row->text;
array_push($listimage,$image);
}
echo($listimage)
displaying the data ($datapage['content']['listimage'] = echo($listimage)
for($i=0;$i<count($datapage['content']['listimage']);$i++)
{
$ititle=$datapage['content']['listimage'][$i]['title'];
$itexte=$datapage['content']['listimage'][$i]['text'];
$tititle=htmlspecialchars($ititle);
$titext=htmlspecialchars($itext);
echo('<a href="someurl" title="'.$tititle.$titext.'>');
echo('<img id="img'.$i.' src="someurl" alt="'.$tititle.'">');
echo('<div>'.$ititle.'</div>');
echo('</a>');
}
Here I use htmlspecialchars, but I tried almost every (combination of) functions I can imagine
When the input text contains quote, &, slash, ', ... Some part of the display is ok, some are not, depends of the encoding/decoding functions I use, and the characters.
For exemple with & : with no coding, the json_decode in php send an error, as the & is considered as a url paramater (https:www.xxx.com?a=1 & b=2). If I encodeURIcomponent the &, the json_decode is ok, but when I want to display it, the title tag in the img tag will show & ; and the html text will show & (of course it depends of the functions I use to decode the string). Sometimes it works well everywhere, except for one character at one point of the process. And when I find a solution for this particular case, it creates a problem somewhere else for another character.
The problem occurs mainly with the characters & " ' \ < >. For example, I try that string : < " '& \ \ \ ' <<> "'"' (here another problem : \ ' is two characters written by the user, and must no be interpreted as an escaping ' )
I never find the good combinations of functions to send, store and retreive the text, there is always a problem somewhere
Related
Good day,
I am using Angular 8 as my frontend, and java api service as my backend.
I need to retrieve some String from backend, and the String will having \n in between, for example:
"Instructions:\n1. Key in 122<16 digit \nreload pin># to reload.\n2. Press SEND/CALL from
In my .ts file, I am setting this String value as follow:
this.str = this.apiRes.responseMsg1;
console.log("this.str : " + this.str);
This will give me Instructions:\n1. Key in 122<16 digit \nreload pin># to reload.\n2. Press SEND/CALL, thus when I use it to display in html, it will just display as 1 line.
If I hard code this String to a String variable, for example:
this.str = "Instructions:\n1. Key in 122<16 digit \nreload pin># to reload.\n2. Press SEND/CALL from";
console.log("this.str : " + this.str);
It will give me :
Instructions:
1. Key in 122<16 digit
reload pin># to reload.
2. Press SEND/CALL from
Which is what I want.
I am not really familiar with Angular, I am trying to find this answer in google, but cant get any related result.
May I know why is this happen? And any way I can display the api message accordingly?
HTML ignores \n line breaks. You basically have two options to fix this:
Replace the line breaks with <br> elements.
Add a line of css white-space: pre-wrap; to your HTML element which is displaying your string.
I'm building and inventory system and I'm trying to insert some data into MySql and noticed that I get errors when trying to insert things such as 1" Monitor Cable, or anything with quotation marks for that matter. Things such as part description and the part name could all potentially contain quotation marks or apostrophes.
I know that I can just put a "\" before the quotation marks or apostrophe's by using javascript's replace method, but I'm not sure what that would look like. If anyone could help me out, that would be great!
I assume that you process some sql in node.js and its parts are not directly copied from user input (this will be danger/unsafe) but first you filter input to prevent sql injection - if yes then try
let str = '1" Monitor Cable'
console.log( str.replace(/"/g,'\\"') );
Hello friends facing a problem with AJAX and PHP ???
Let me tell you what i did ...!
I created 2 pages
one page (AJAX_view.php) with a drop down with all the blood group in short code like (b+, b-) and i placed my ajax code on the same page. to asynchronously search for the blood groups available in database (MySql).
Now the another page (search.php) has all the php code and the query to retrieve the data from database..
Now the real problem is from the 1st page i passed the string in url to the another page using the code
xmlhttp.open("POST","search.php?q="+str,true);
xmlhttp.send();
and the str here is 'B+' without quotes
and it doesnt reads the "+" , what it reads is only B... this only happens in the case of "+ " while if i pass the string str as "A-" its give the result without any problem or error and returns result from database
but there are certain blood group like B+ ,AB+ , O+ and there is no result show for them.
$q=$_GET['q'];
this is how i retrieve the string on search page..
$select_query="SELECT * FROM blood_donate WHERE blood_group='$q'";
$res=mysql_query($select_query);
this is how i select data from database??
please help
You need to encode +, otherwise it'll be treated as space (). Use encodeURIComponent on str for this.
xmlhttp.open("POST","search.php?q="+encodeURIComponent(str),true);
if str is A+ without encodeURIComponent it'd generate url search.php?q=A+. This makes $_GET['q']='A ' at the PHP end. (note the space)
With encodeURIComponent it'd be search.php?q=A%2B which yields $_GET['q']='A+'.
This question seems to be asked often before, but I found no working solution for data, that is very long and contains special charcaters like "<" or "{" or...and so on.
I am submitting some huge XML data to PHP on a server, which looks like this:
<root><id>1</id><text>Here is a very long text with
line breaks, white-spaces and many very unsual charchaters, e.g. < % & }
the text can be more then 5000 characters long
</text></root>
On the server side, I am trying to get the "raw data" between the the text tag. The raw data inside the "text tags", can include anything you can image: white-spaces, line-breaks, weird characters. What I submit is source code and text, formatted by CKEditor and a code syntaxhighlighter.
I read me through this and basically everybody says "use a XML parser", like domDocument don't use RegEx.
First of all I tried several RegEx statements, for example. This is not the only one I tried. When the data conatins brackets and the data is too long, it will fail:
//#<text[^>]*>[\s\S]*?</text>#
$regex = "#<".$element_name."[^>]*>[\s\S]*?</".$element_name.">#";
$found = preg_match($regex, $xml, $matches);
if ($found != false)
{
$result = $matches[0];
return $result;
}
Second I tried with this, which works if the data inside the tags is not too weird. I think the parser doesn't like brackets "<" and thinks the xml is not valid.
function getTextBetweenTags($tag, $html, $strict=0)
{
/*** a new dom object ***/
$dom = new domDocument;
/*** load the html into the object ***/
if($strict==0)
{
$dom->loadXML($html);
}
else
{
$dom->loadHTML($html);
}
/*** discard white space ***/
$dom->preserveWhiteSpace = false;
/*** the tag by its tag name ***/
$content = $dom->getElementsByTagname($tag);
/*** the array to return ***/
//$out = array();
foreach ($content as $item)
{
/*** add node value to the out array ***/
//$out[] = $item->nodeValue;
/*** return only the first found element value ***/
return $item->nodeValue;
}
/*** return empty string if nothing found ***/
return "";
}
So my question is:
If I exactly know, there is only one beginning and ending "text"-tag inside the data, what is the best way, to read the raw data with PHP?
If anyone gave me a working regex or a code-snippet, it would be great.
Sorry for my medium english.
===RESPONSE TO ANSWERS===RESPONSE TO ANSWERS===RESPONSE TO ANSWERS===
Ok, both answers from BogdanM and Steven do work, but my favourite answer is from BogdanM.
What I did. to get this working:
I create my own XML on the client site and I now use CDATA to tell the parser where the data begins and ends
On server side I use SimpleXML to parse the data. Using CDATA it has no more problems to parse it. No more matter how "weird" the data is.
I eliminated a common "Rookie error" to send big data with HTTP-GET. I now just use HTTP-POST to have not limits
Thank you again for the help.
Are you generating the XMLs as well?
Because if you are, you should put your text data between CDATA. Then load your xml with simplexml or some parser of your choice and get the text tag contents.
Make sure you don't have UTF-8 characters orm some such which are not allowed in XML at all: http://www.w3.org/TR/2000/REC-xml-20001006#NT-Char
Otherwise you could do this:
preg_match('#<text>(.+?)</text>#is', $xml, $matches);
echo $matches[1]; // your data between <text> and </text>
Firstly, your original regex pattern, is okay and should work fine:
#<".$item_name."[^>]*>([\s\S]*?)</".$item_name.">#
However, you could alter it to make it more readable/functional etc...
Possibilities
Regex 1
#<text>(.*)</text>#is
Simply capture everything between the text tags. Use modifiers i to allow TEXT and text tags and s to make . match new lines.
Regex 2
#<text.*?>(.*)</text>#is
Your original regex implies that you expect to receive extra characters in your opening text tag. The .*? within the opening tag allows for this - the ? makes it stop at the first >.
Regex 3
#<(text).*?>(.*)</\1>#is
As the opening and closing tag name are the same (i.e. text) you can place parenthesis around the opening tag to make it a capture group and simply reference in the closing tag with \1 - as it's the first capture group.
This means that there's one less chance to make a spelling mistake!
Regex 4
#<('.$item_name.').*?>(.*)</\1>#is
Making it more dynamic. You can replace the word text with a variable (as per your original). Mixing this with the capture groups and referencing as in Regex 3 you only have to insert the variable once leading to cleaner more readable code.
Comparison v Original
#<('.$item_name.').*?>(.*)</\1>#is
#<".$item_name."[^>]*>([\s\S]*?)</".$item_name.">#
Working Example
Using Regex 4 above
$string = "
<root><id>1</id><text>Here is a very long text with
line breaks, white-spaces and many very unsual charchaters, e.g. < % & }
the text can be more then 5000 characters long
</text></root>";
preg_match('#<('.$item_name.').*?>(.*)</\1>#is', $string, $matches);
var_dump($matches);
/**
Output:
array(3) {
[0]=>
string(167) "<text>Here is a very long text with
line breaks, white-spaces and many very unsual charchaters, e.g. < % & }
the text can be more then 5000 characters long
</text>"
[1]=>
string(4) "text"
[2]=>
string(154) "Here is a very long text with
line breaks, white-spaces and many very unsual charchaters, e.g. < % & }
the text can be more then 5000 characters long
"
}
*/
NOTE: If you are unable to get the above working example to... work... then could you perhaps provide (either by editing into your question or linking too) an example case where it would not work?
I have an issue with submitting post data. I have a form which have a couple of text fields in, and when a button is pressed to submit the data, it is run through a custom from validation (JS), then I construct a query string like
title=test&content=some content
which is then submitted to the server. The problem I had is when I have '&' (eg  ) entered into one of the inputs which then breaks up the query string. Eg:
title=test&content=some content  
How do I get around this?
Thanks in advance,
Harry.
Run encodeURIComponent over each key and value.
var title = "test";
var content = "some content   ";
var data = encodeURIComponent('title') + /* You don't actually need to encode this as it is a string that only contains safe characters, but you would if you weren't sure about the data */
'=' + encodeURIComponent(title) +
'&' + encodeURIComponent('content') +
'=' + encodeURIComponent(content);
Encode the string..when you want to encode a query string with special characters you need to use encoding. ampersand is encoded like this
title=test&content=some content %26
basically any character in a query string can be replaced by its ASCII Hex equivalent with a % as the prefix
Space = %20
A = %41
B = %42
C = %43
...
You need to encode your query to make it URL-safe. You can refer to the following links on how to do that in JS:
http://xkr.us/articles/javascript/encode-compare/
http://www.webtoolkit.info/javascript-url-decode-encode.html
You said:
...and when a button is pressed to submit the data, it is run through a custom from validation (JS), then I construct a query string...
In the section where you are building the query string you should also run the value of each input through encodeURIComponent() as David Dorward suggested.
As you do - be careful that you only assign the new value to your processed query string and NOT the form element value, otherwise your users will think their input was somehow corrupted and potentially freak out.
[EDIT]
I just re-read your question and realized something important: you're encoding an   ;character. This is probably a more complicated issue than other posters here have read into. If you want that character, and other &code; type characters to transfer over you'll need to realize that they are codes. Those characters &, n, b, s, p and ; are not themselves the same as " " which is a space character that does not break.
You'll have to add another step of encoding/decoding. You can place this step either before of after the data is sent (or "POSTed").
Before:
(Using this question's answers)
var data = formElement.value;
data = rhtmlspecialchars(data, 0);
Which is intended to replace your "special" characters like with " " so that they are then properly encoded by encodeURIComponent(data)
Or after:
(using standard PHP functions)
<?PHP
$your_field_name = htmlspecialchars_decode(urldecode($_POST['your_field_name']));
?>
This assumes that you escaped the & in your POST with %26
If you replaced it with some function other than encodeURIComponent() you'll have to find a different way to decode it in PHP.
This should solve your problem:
encodeURIComponent(name)+'='+encodeURIComponent(value)+'&'+encodeURIComponent(name2)+'='+encodeURIComponent(value2)
You need to escape each value (and name if you want to be on the safe side) before concatenating them when you're building your query.
The JavaScript global function encodeURIComponent() does the escaping.
The global function escape() (DOM) does this for you in a browser. Although people are saying it is not doing the escaping well for unicode chars. Anyway if you're only concerned about '&' then this would solve your problem.