I have a basic chat system set up that uses an SQL database and a PHP script -- when the user inputs a message, its sent to the database and then is retrieved and displayed. New messages are displayed every 5 seconds regardless.
All that being said, its fairly easy to just spam messages causing the website to stop responding at which point clicking any links will result in an error page, and no further messages will be input.
Is this a common scenario? How should I improve the chat's performance? Note: I'm really new PHP and JS/Jquery.
Here is the main script that is frequently called to update the html chatbox with new messages for the logged-in user:
Two auto-incremented values are compared to determine "new messages", the value of the last displayed message, and the value of the last message in the database.
<?php
session_start();
if (isset($_SESSION['logged_in']) && $_SESSION['logged_in'] == true) {
$alias = $_SESSION['username'];
$host = 'localhost';
$user = 'root';
$pass = '';
$database = 'vethergen_db_accounts';
$table = 'table_messages';
$user_table = 'table_user_info';
$last_id_table = 'table_chat_sync';
$connection = mysqli_connect($host, $user, $pass) or die ("Unable to connect!");
mysqli_select_db($connection,$database) or die ("Unable to select database!");
if ($redis->exists("/lastId/$alias"))
{
$last_id = $redis->get("/lastId/$alias"); //Gets the last id from cache...
}
else
{
$last_id_query = "SELECT last_id FROM $last_id_table WHERE alias = '$alias'";
$last_id_result = mysqli_query($connection,$last_id_query);
$last_id_rows = mysqli_fetch_array($last_id_result);
$last_id = $last_id_rows['last_id'];
// Now that you just read it, create a last_id cache entry for this user
$redis->set("/lastId/$alias", $last_id);
}
$query = "SELECT * FROM $table WHERE text_id > '$last_id'"; //SELECT NEW MESSAGES
$result = mysqli_query($connection,$query);
if ($result && mysqli_num_rows($result) > 0)
{
while($row = mysqli_fetch_array($result))
{
$color_alias = $row['alias'];
$text_color_query = "SELECT color FROM $user_table WHERE alias = '$color_alias'";
$text_color_result = mysqli_query($connection,$text_color_query);
$text_color_rows = mysqli_fetch_array($text_color_result);
$text_color = $text_color_rows['color'];
if ($row['alias'] === "Vether")
{
echo '<p id = "chat_text" style="color:'.$text_color.'">'.'<b>'.$row['alias'].': '.'</b>'.$row['text']."</p>";
echo '<p id = "time_stamp">'.$row['time'].'</p>';
echo '<p id = "chat_number">'.$row['text_id'].'</p>';
}
else
{
echo '<p id = "chat_text" style="color:'.$text_color.'">'.'<b class = "bold_green">'.$row['alias'].': '.'</b>'.$row['text']."</p>";
echo '<p id = "time_stamp">'.$row['time'].'</p>';
echo '<p id = "chat_number">'.$row['text_id'].'</p>';
}
echo '<hr class = "chat_line"></hr>';
$last_row_id = $row['text_id'];
}
//UPDATE LAST SYNC ID
$update_query = "UPDATE $last_id_table SET last_id = '$last_row_id' WHERE alias = '$alias'";
$redis->delete("/lastId/$alias");
mysqli_query($connection,$update_query);
}
else {echo '';}
?>
There is no specific right answer because your question is very general, but there are a few things that are obvious here. You have built a botteneck in your database where the more users you have, the more updates you are doing on the table_chat_sync.
As an aside, I have no idea why you are putting a constant (the table name) into PHP variables for your queries. At very least these should be php constants but that makes the syntax pretty painful. Your code is simpler and better just using the table names in the SQL.
InnoDB
Are you using InnoDB tables? You should be, given that you are updating a row and with InnoDB you have row level locking.
You also want to make sure that you have enough innodb buffer pool cache allocated to insure that the db is in memory. This will buffer your select activity a lot and buy you some head room.
MySQL EXPLAIN
You also need to do explain plans on your select query and insure that it is properly indexed so that the queries are being returned using indexes and you are not table scanning or having temporary tables created.
SQL queries against mysql are quite slow compared to getting data from cache, and the reality is that the full set of chat messages doesn't change much, and yet your system is repeatedly going to be querying the chat or a subset of it over and over again. For this reason, most sophisticated systems are using some sort of caching system or queuing. There is overlap between these two technologies and they tend to offer better scalability as well as support for concepts like publish/subscribe that fit chat very well.
Redis & Other backends
Redis as an example, could be the back end for chat and completely supplant the actual storage and retrieval of messages. The document database MongoDB is also an alternative option that has in-memory characteristics when the dataset can be controlled.
Using Redis with MySQL
Redis is often combined with an RDBMS and in your code there are a few places where it could be a great help. For example, you do this query repeatedly:
$last_id_query = "SELECT last_id FROM $last_id_table WHERE alias = '$alias'";
With redis you could do something like this:
if ($redis->exists("/lastId/$alias")) {
$last_id = $redis->get("/lastId/$alias");
} else {
$last_id_query = "SELECT last_id FROM $last_id_table WHERE alias = '$alias'";
$last_id_result = mysqli_query($connection,$last_id_query);
$last_id_rows = mysqli_fetch_array($last_id_result);
$last_id = $last_id_rows['last_id'];
// Now that you just read it, create a last_id cache entry for this user
$redis->set("/lastId/$alias", $last_id);
}
The only other detail is that when you update the last Id you would want to delete the redis key:
$redis->delete("/lastId/$alias");
Hopefully you can see that this would lower the load on mysql quite a bit, because no query will occur without a new message being added. This will buffer mysql quite a bit, and the same concept can be used to cache the other queries you are doing, such that you never require mysql queries unless you have a new user actively using Redis. I didn't go into this but you can set the expiration of a key to some period of time, so it will clean up old keys from non-active users.
Load Testing to understand your bottlenecks and capacity
Your choice of reliance on MySQL is something you will have to accept as limiting, although again you may be able to tune it so that within your use case and load, it runs acceptably, but that is impossible to predict without detailed configuration analysis and load testing. There are many load testing and stress testing tools that are FOSS, with Apache JMeter being one of the oldest ones, so I'll advise you to start with that.
Websockets
Last but not least, polling is inherently wasteful and most chat systems these days are built using websockets which is just a better fit for the task of having a sustained client-server connection. Websocket is client & Server code, and being that you are a PHP dev, there are a few projects that can help you out here, Ratchet being one that has been around for a while. There's a PHP client lib Pawl that shows you how to make a simple robust websocket connection.
You can add a Limit to the end of SELECT * FROM $table WHERE text_id > '$last_id' and that will keep some of the spam messages from slowing down the thread. Also you can prohibit duplicates on the INSERT statement.
Related
This is a challenge I am facing in Reactjs, though I don't believe that it is necessarily attributed to it. I am trying to make an API call in React. And while it works, the code also reveals my API key, which below is indicated by my javascript variable sting. When I preview the code in my browser, sting quite clearly shows my API key.
render: function() {
if (this.state.trial) {
return this.iftroo();
}
}
});
var Troo = React.createClass({
render: function() {
var sting = "<?php
$con = mysqli_connect('localhost', 'root', '', 'worldly') or die("Trying");
$query = "select * from testi";
$result = mysqli_query($con, $query);
while($row = mysqli_fetch_array($result)){
echo $row["userName"];}
?>";
var weather = new XMLHttpRequest();
weather.open("GET", "http://api.openweathermap.org/data/2.5/weather?q=London,uk&units=imperial&appid="+sting, false);
weather.send(null);
var r = JSON.parse(weather.response);
var tempurature = r.main.temp;
return (
<p>
{tempurature}
</p>
I understand that in order to get this to work, I will likely have to embed my javascript code inside my PHP. However, doing so leads to errors, such as PHP not recognizing javascript var characters.
What measures can I take to hide my API keys from the browser?
If you want to hide the API key from the browser then you must simply never give it to the browser.
You can't use the API key from client side JavaScript and keep it hidden from the client.
Make any HTTP request that needs to use it from your server (e.g. using PHP's cURL library).
You could generate one-time jwt api keys, for a special user, with expiration time, and what ever information assigned it.
edit
OK, now I see, that the api key is for an external service. Don't know how the policy for the weather service is, but.. I think this is not the right way to go, you should make this request on the server.
I am here to ask you about a PHP file I've been coding. I am building a live auction website that lets the users take part to auctions, each one shown in a site's page. In order to let users know when someone else has made a bid, I have used a SSE system. I use AJAX for the user's bid action's registration on the database (when the user bids, the price of the item in the database gets higher to exactly 1 dollar more. The price is stored as current price + 100 in the database because I am only storing integers (the number of cents) for the prices, for security reasons).
My AJAX works well, so the bid actually happens.
My SSE connection gets opened as expected, at the moment of the page load. The javascript EventSource object successfully opens the stream to my PHP file, but I can't understand why, when I make a test bid in the auction's page using my AJAX, the PHP SSE worker doesn't send a message to the auction's page (that has opened the stream to the worker).
Here is the PHP code that is supposed to check whether there's a new bid and send a message to the auction's page containing the new bid's data.
<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
$mysqli = new mysqli(* data *);
$listings = $mysqli->query("SELECT listing_id, price, buyer_id FROM listings WHERE schedule_requested = '1' AND finished = '0'");
$to_check = array();
while ($li = $listings->fetch_array()) {
$to_check[] = array($li["listing_id"], $li["price"], $li["buyer_id"]);
}
while (true) {
$change = 0;
$listings_2 = $mysqli->query("SELECT listing_id, price, buyer_id FROM listings WHERE schedule_requested = '1' AND finished = '0'");
$different = array();
while ($li_2 = $listings_2->fetch_array()) {
foreach ($to_check as &$li_check) {
if ($li_check[0] === $li_2["listing_id"]) {
if (intval($li_2["price"]) !== intval($li_check[1])) {
$li_check = array($li_2["listing_id"], $li_2["price"], $li_2["buyer_id"]);
$different[] = $li_check;
$change = 1;
}
break;
}
}
}
if ($change === 1) {
echo "data: " . json_encode($different) . PHP_EOL;
echo PHP_EOL;
flush();
}
sleep(1);
}
?>
I hope you can follow how I reasoned to write this code: when the connection is opened, all the listings (that are the lots the auction is composed of) that are eligibile for taking part in the auction will be checked and stored in the array called $to_check. Every time the outer while loop iterates, the code sends a query to the database again to check if some price fields have been changed. In case it has happened, the code sends a message to the auction's page, in order to inform the user.
The problem is that, when I try to bid from the auction's page, the bid is successful but I do not receive any message from the PHP worker (that is supposed to tell me that the x listing price has changed). What is the problem with that PHP code?
Thanks for your help.
P.S. For clarity reasons, I will also write the Javascript code that handles the SSE system.
<script type = "text/javascript">
var evtSource = new EventSource("bidchecker.php");
evtSource.onmessage = function(e) {
console.log("onmessage");
// etc.
}
</script>
I have a small MySQL database with a few hundred rows (all in text, no images). I am requesting all the rows using iQuery and do all filtering at client end. iQuery code is the following:
$(document).ready( function () {
$.get("alldata.php", function(data){
$('#result').text(data);
});
});
On the server side, the "alldata.php" has the following code and pass the data in JSON back to iQuery:
$sql = "SELECT title FROM mydatabase";
$result = mysqli_query($conn, $sql);
$arr = array();
while($row = mysqli_fetch_assoc($result)){
$row_array['Title'] =$row['title'];
array_push($arr,$row_array);
}
mysqli_close($conn);
echo json_encode($arr);
It seems to me there will not be any risk of injection since there is NO user input submitted to the database. Am I right or wrong? Thanks a lot for your input!
You are right. Your SQL statement includes no parameters outside of itself, so there is no vector for injection. While injection attacks ARE possible on SELECT statements, in your case the query is not created dynamically so cannot be tampered with.
You are safe since there are no user input. A malicious user needs user input to inject query. So never trust user input.
Im working on a web based, mobile application for my technicians to print out invoices in the field.
The application is programmed mostly in php/html/css and javascript.
The printing is done with a canon pixma i110 connected via wifi to a 4g tablet. When the technician tries to print, he is presented with a blank screen if the variables aren't defined locally. I've echoed the session variables in question and they have the expected values.
$var1 = $_SESSION['var1'];
//$var1 = "value1";
$var2 = $_SESSION['var2'];
//$var2 = "value2"
mysql_connect("host","user","pass")or die("cannot connect");
mysql_select_db("table")or die("cannot select DB");
$result = mysql_query("SELECT `field1`,`field2` FROM mytable WHERE `fieldx` = '$var1' AND `fieldz` = '$var2' ORDER BY `id` ASC") or die(mysql_error());
while ($row = mysql_fetch_array($result)){
$info1=$row["invoice_name"];
$info2=$row["print_copy"];
}
echo $info2;
When the mysql query is performed as is above, the print page yields nothing. When I use the local defined variables, the print page gives me the expected result.
I've been at this for a while now, i've done plenty of searches on this site as well as others, but have yet to find a solution. I apologize in advance if this is a duplicate question.
Thanks for your help.
Edit: It appears to me that the session variables are not available to the browser during conversion to print. How can I get around this?
Solution: Include favicon in headers. I feel like a moron.
I need to write a application that checks database from external server every 10 seconds to see if there is new data. Currently I have a javascript that checks if data has changed server by comparing two JSON (the old JSON and the new fetched from server) and if it has alerts user. But that is not what I need in this application. User should be alerted only when data is new, not when it has changed.
I was thinking that maybe I could do this with a PHP code that queries MYSQL and if query num_results is 0 loop until num_results is more than 0 when user gets notified. In this application it doesn't matter whether the new data is available for user in 0,1 second or 10 seconds, just as long as user gets it. This is how I tried to do the MYSQL check, but it isn't working:
<?php
include 'config.php';
if(isset($_GET['ID'])) {
$maxLoop = 20;
while($maxLoop--) {
$dbh = new PDO("mysql:host=$dbhost;dbname=$dbname", $dbuser, $dbpass);
$dbh->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
try {
$sth = $dbh->prepare('select * from testit where id = :id');
$sth->bindParam(':id',$_GET['ID'],PDO::PARAM_INT);
$sth->execute();
if($sth->rowCount()>0) {
$results = $sth->fetchAll(PDO::FETCH_OBJ);
echo '{"key":'. json_encode($results) .'}';
exit; // Found new data, end loop and script
}
} catch(PDOException $e) {
break;
}
sleep(3);
} // end while
} // end if
So how can I alter this code to make it work, or should I just try to write some javascript that would do this? And if so, how can I check whether data is new or not, instead of just checking whether it has changed or not?
How do you record 'new' data? is there a timestamp? an auto_increment primary key field?
The easiest method for polling is to simply keep track of the last known id/timestamp and see if there's anything with a higher id/newer timestamp.
As is, your method is quite inefficient. You select ALL records in the table, forcing mysql/pdo to start fetching that data from disk, etc... then simply throw it away after getting the row count. A more efficient method is do to a select count(*) ... and checking that value. This keeps mysql from having to start fetching actual row data from disk. And on some table types (myisam in particular), a count(*) operation is VERY quick.
If you want your application to check for changes every 10 seconds, you will have to use AJAX to make asyncronous requests to a php file on the server. In the php file, you only have to select * from testit where condition=false and you will get your "new data". Here's a link from where you can learn the basics of AJAX :
http://www.w3schools.com/ajax/default.asp