Improve Page Performance, save PHP array on server? - javascript

is it possible to store a PHP-array to my server, right now it always gets created when someone reloads the page from a CSV file but that is unnecessary since the file only chances after each hour.
ATM, the page takes like 9 seconds to load, which is quite long. The CSV file has 10k+ rows with 9 elements per row, so it would be really good for performance if the server didn't have to process 100k elements for each user.
I already have a cronjob for downloading the csv file so it would be good if the parse command would be executed after the download finished, only once per hour.
cronjob:
<?php
function download_remote_file($file_url, $save_to) {
$content = file_get_contents($file_url);
file_put_contents($save_to, $content);
}
download_remote_file(<url here>, realpath(".") . '/dump.csv');
?>
and this happens with every reload of the page:
1st: Parse data to array
$url = 'dump.csv';
$csvData = file_get_contents($url);
$lines = explode(PHP_EOL, $csvData);
$array = array();
foreach ($lines as $line) {
$line = str_replace("\\", "\", $line);
$line = str_replace("#", "#", $line);
$array[] = str_getcsv($line);
2nd: pass array to Javascript
var array = <?php echo json_encode( $array ) ?>;
3rd: create HTML table
//some code
4th: initialise data table plugin
$(document).ready( function () {
createtable();
$('#scoreboard').DataTable( {
"iDisplayLength": 50,
language: {
decimal: ".",
},
"lengthMenu": false,
"bLengthChange": false
} );
} );
Is there something that could be done faster?
Like, as mentioned, save the php array server-side or maybe saving the JS array with the HTML table somehow?
-Innerwolf

After you parse your CSV, do this:
$file = fopen('/tmp/output.js', 'w');
fwrite($file, '<script type="text/javascript">');
fwrite($file, 'var array =');
fwrite($file, json_encode( $array ));
fwrite($file, ';');
fwrite($file, '</script>');
fclose($file);
copy('/path/to/script.js', '/path/to/script.js.bak');
move('/tmp/output.js', '/path/to/script.js');
Then, later on when you are outputting the HTML, you just need to stick in a:
<script type="text/javascript" src="/scripts/script.js">
in the header. People's browsers should cache it properly too. Note the copy and move -- you don't strictly need to make a backup copy, but you MUST use a move() to replace the 'live' script -- move() is atomic, more or less, and won't result in anyone getting a half-file.
Also, note that you'll need write permissions to where the script is -- there are ways to keep this pretty secure (not letting your PHP script write all over the hard drive), but that's out of scope here.

Since you mention getting the data on an hourly basis I suggest the following:
grab the CSV file with cron and store the data in a database on an hourly basis
configure your data tables component to use server side data
This way you won't force every user to download the entire array at once on every first page load.
The server side script only fetches the number of records that need to be displayed on that particular page in the table.

Related

Export PHP table with pagination to EXCEL, PDF, PRINTABLE

How can I export my php table with pagination into excel and pdf format? I've tried many plugins and tutorials on youtube still no luck but they only get the data displayed not the entire table itself. I want to export my table to at least printable version better if I can convert my table into PDF or Excel Format. On my table I have a maxium of 100 rows paginated with 10 rows per page. How can I do this? I need your expertise.
PHP Export to branch:
<?php
include "connect.php";
require('lib/js/fpdf.php');
$result = mysqli_query($conn,"SELECT * FROM tblSales");
$header = mysqli_query($conn,"SELECT UCASE('date','sales')
FROM 'INFORMATION_SCHEMA'.'COLUMNS'
WHERE 'TABLE_SCHEMA'='DB_NAME'
AND 'TABLE_NAME'='tblSales'
and 'COLUMN_NAME' in ('date','sales')");
$pdf = new FPDF();
$pdf->AddPage();
$pdf->SetFont('Arial','B',16);
foreach($header as $heading) {
foreach($heading as $column_heading)
$pdf->Cell(95,12,$column_heading,1);
}
foreach($result as $row) {
$pdf->Ln();
foreach($row as $column)
$pdf->Cell(95,12,$column,1);
}
$pdf->Output();
?>
AJAX Code:
$(document).on('click', '.export-branch-excel', function () {
var branch = $("#branch-hidden-data").val();
$.ajax ({
url:"export-branch-excel.php",
data: "id="+branch,
method: "POST",
dataType: "text",
success: function(data){
window.location = "export-branch-excel.php";
}
});
});
#lawrence agulto I changed this code to procedural type as you said. Try this code.
Download In excel:
<?php
include_once "connect.php";
$query = mysqli_query($conn,"SELECT * FROM tblSales ORDER BY date DESC ");
$columnHeader = "Column Name"."\t"."Column Name1"."\t"."Column Name2"."\t"."Column Name3"."\t";
$setData='';
if (mysqli_num_rows($query) > 0) {
while ($rec = mysqli_fetch_assoc($query)) {
$rowData = '';
foreach ($rec as $value) {
$value = '"'.$value.'"'."\t";
$rowData.=$value;
}
$setData.=trim($rowData)."\n";
}
}
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=Nobelz_Sushank.xls");
header("Pragme: no-cache");
header("Expires: 0");
echo "\t\tSales Data\n";
echo ucwords($columnHeader)."\n".$setData."\n";
?>
Download in pdf :
Download the FPDF library from here.
And change the query, table name, and TABLE_COLUMN_NAME according to your need.
<?php
$result = mysqli_query($conn," Your QUery");
$header = mysqli_query($conn,"SELECT UCASE(`COLUMN_NAME`)
FROM `INFORMATION_SCHEMA`.`COLUMNS`
WHERE `TABLE_SCHEMA`='DB_NAME'
AND `TABLE_NAME`='TABLE_NAME'
and `COLUMN_NAME` in ('TABLE_COLUMN_NAME','TABLE_COLUMN_NAME1', 'TABLE_COLUMN_NAME2', 'TABLE_COLUMN_NAME3')");
require('fpdf181/fpdf.php');
$pdf = new FPDF();
$pdf->AddPage();
$pdf->SetFont('Arial','B',16);
foreach($header as $heading) {
foreach($heading as $column_heading)
$pdf->Cell(95,12,$column_heading,1);
}
foreach($result as $row) {
$pdf->Ln();
foreach($row as $column)
$pdf->Cell(95,12,$column,1);
}
$pdf->Output();
?>
The subject is way too broad. What you need to do is:
supply a download link on the page (which means that any parameters needed for the export need be saved in a session; OR you can do a more difficult "download after AJAX POST")
the link will run the same query you use to paginate, with the necessary parameters (see above), but without the pagination. It will then retrieve a number of rows. Possibly a large number of rows. You are already 95% done on this.
you use a PHP library (the new PHPExcel for flexibility and features, Spout for speed) or a templating library (e.g. TBS) to pull these rows into an Excel file. This is the part you need to flesh out.
send along the Excel file as binary download.
The PDF version is the same (if a bit more complicated, since you normally need to compose the PDF yourself - there are ways to do otherwise but require access to the server, e.g. TBS + unoconv), except that you use a PDF library instead of an Excel one. E.g. FPDF or TCPDF.
For a quick result and if you don't care about a formatted Excel file (backgrounds, logos, borders...) you can replace the Excel part with a "fake" Excel - create a HTML table, a CSV file, or even worse a simple unescaped tab-separated blob of data, give it a XLS(X) extension and go. From my experience, this gives you the worst result possible: something that works 95-97% of the time 1. Great expectations, periodically dashed, with huge maintenance backlog2. If it's a pet project, go for it; if you need it in a professional setting, take the time to do it right.
(1) the remaining 3-5% being comprised of UTF-8 characters, quotes, numbers and dates written in any format but that which Excel understands for import, etc.
(2) add date formatting. Improve quoting. Detect UTF8. Detect UTF8 which actually was ISO-8859-15. Wait, unescaped. And so on and so forth. Oh, and every time everything must be done yesterday. And why it still does not work?

Can't unserialize full string

Brief & Code
I have the content of a Chat log stored in a .txt file. The content is as follows:
a:3:{s:6:"author";s:5:"e297f";s:7:"message";s:4:"test";s:4:"time";s:8:"14:15:54";}a:3:{s:6:"author";s:5:"e297f";s:7:"message";s:6:"test 2";s:4:"time";s:8:"14:16:55";}a:3:{s:6:"author";s:5:"e297f";s:7:"message";s:6:"test 3";s:4:"time";s:8:"14:17:59";}
A jQuery function calls out to a page chatretrieve.php to collect the content of this file. The PHP file looks like this:
<?php
session_start();
$data = unserialize(file_get_contents('../sessions/chats/log_'.$_SESSION['chatCode'].'.txt'));
#exit(print_r($data));
$content = '';
for($i = 0; $i < count( $data ); $i++){
$content.='<div class="msgln">';
$content.='<div class="meta">';
$content.='<span class="name">'.$data[$i]['author'].'</span>';
$content.='<span class="time">'.$data[$i]['time'].'</span>';
$content.='</div>';
$content.='<div class="msg">'.stripslashes(htmlspecialchars($data[$i]['message'])).'</div>';
$content.='</div>';
}
return $content;
The relevant part of the jQuery function is as follows:
$.post('inc/chatretrieve.php').done(function(data) {
console.log(data);
});
The problem
When I comment out the exit(print_r($data)) part of the PHP page, the console returns only the first of the array variables in the .txt file:
Array
(
[author] => e297f
[message] => test
[time] => 14:15:54
)
1
As there are three messages in the .txt file (and retrieved with the file_get_contents() function), why can I only see the first line when I use the unserialize() function?
The issue is that your serialised data isn't valid:
'a:3:{s:6:"author";s:5:"e297f";s:7:"message";s:4:"test";s:4:"time";s:8:"14:15:54";}a:3:{s:6:"author";s:5:"e297f";s:7:"message";s:6:"test 2";s:4:"time";s:8:"14:16:55";}a:3:{s:6:"author";s:5:"e297f";s:7:"message";s:6:"test 3";s:4:"time";s:8:"14:17:59";}
Because this is three arrays it's unclear how to unserialise this (I'm surprised that PHP doesn't fail outright but instead returns the first object).
You will need to either store this as a serialised array or, alternatively find a way to split the file into sections for each message - this could probably be done on newlines or something similar.
e.g. Something like
<?php
//Note the added newlines.
$sez = 'a:3:{s:6:"author";s:5:"e297f";s:7:"message";s:4:"test";s:4:"time";s:8:"14:15:54";}
a:3:{s:6:"author";s:5:"e297f";s:7:"message";s:6:"test 2";s:4:"time";s:8:"14:16:55";}
a:3:{s:6:"author";s:5:"e297f";s:7:"message";s:6:"test 3";s:4:"time";s:8:"14:17:59";}';
foreach(explode("\n",$sez) as $line){
$data = unserialize($line);
print_r($data);
}
Example here http://codepad.org/sq1SbhIz

Sending data to PHP page to make a spreadsheet out of it

For some reason, when I try to send JSON data to a PHP page (where it gets downloaded as a spreadsheet), it runs without error, but doesn't bring up the prompt to download the spreadsheet. The JSON has generated without problem (I have made the PHP page create the file on the server, before trying to make it download without creating it).
Here is the JavaScript code that sends the JSON data to the server:
function writeToSpreadsheet()
{
// get the json for #theTable
var tableJSON = tableToJSON("tr:not(#titleRow)");
//alert(tableJSON);
alert("Sending table data to be written to the spreadsheet...");
$.post('/ResearchProject/tableContent/exportTable.php', {'table': tableJSON}).done(
function(response) { alert(((response == '') ? response : (tableJSON.title + ' written to file!')));})
.fail(function (xhr, ajaxOptions, thrownError) { alert("ERROR:" + xhr.responseText+" - "+thrownError); });
}
and here is exportTable.php
<?php
function cleanData(&$str)
{
$str = preg_replace("/\t/", "\\t", $str); // escaping all of the tabs
$str = preg_replace("/\r?\n/", "\\n", $str); // escaping any and all cases of carriage return
// if there is a single double-quote in the string, we wrap the string in quotes, replace every single double-quote with double double-quotes, and
// end with a double-quote
if(strstr($str, '"')) $str = '"' . str_replace('"', '""', $str) . '"';
}
// the data is coming from a JSON object that is being sent here
if (isset($_POST['table']))
{
$tableJSON = $_POST['table']; # somehow, this is already a PHP array (exactly the one we need)!!
// get the name of the table from the $tableJSON
$tableName = $tableJSON['title'];
// get the title row from $tableJSON
$titleRow = $tableJSON['titleRow'];
// fix the titleRow
foreach ($titleRow as $heading)
{
$heading = trim(preg_replace('/\s+/', ' ', $heading));
}
// get the rows from $tableJSON
$rows = $tableJSON['rows'];
// form the filename from the tableName
$fileName = $tableName . '.xls';
// here, we download the file without even creating it
header("Content-Disposition: attachment; filename=\"$fileName\"");
header("Content-Type: application/vnd.ms-excel");
// we echo the titleRow first
array_walk($titleRow, 'cleanData');
echo implode(chr(9), $titleRow) . "\r\n";
?>
<script>console.log('Title row written to file.');</script>
<?php
// now we echo the data
foreach($rows as $row)
{
array_walk($row, 'cleanData');
echo implode(chr(9), $row) . "\r\n";
?>
<script>console.log('Data row written to file.');</script>
<?php
}
}
else
{
echo 'You sent me no data :(\n';
}
?>
OK, MikeWarren, how do I test this??
You can test it by selecting a table from the dropdown menu and clicking the "Export table to spreadsheet" button here: http://dinotator.biokdd.org/ResearchProject/tableViewer.php
I am trying to have it where the table that is on the HTML page gets converted into an JSON object, and then downloaded. Thus, I would need to POST the data to the PHP page, right? (Query strings don't work.)
Query strings won't work because you are using jQuery's $.post call which means that your data is sent in the body of the request, as opposed to a query string which is what a GET uses. For JSON you do indeed want to use a POST.
As for what's going wrong, you need to decode your JSON into a PHP array using json_decode. Unfortunately it can't simply handle JSON how it is.
So most likely you'll want to do:
// now a poorly named variable
$tableJSON = json_decode($_POST['table']);
Also, looking at your Ajax, $.post does accept a .fail() listener, but it doesn't pass any error data as part of the callback. So if you want to be able to handle incoming response errors you'll need to use $.ajax:
$.ajax({
type: "POST",
url: "/your/url.php",
dataType: "json",
error: errorCallback
});
Finally, looking at how your code is structured, if you're actually trying to save to file, you're going to need some more logic. Right now, you're just rendering that table, and then returning it as a response which will show up in your done function. You're going to add some more logic in order to make it actually download. This question entails your exact problem.
Good luck!
I have found so much bad advice on the internet about how to solve this problem. One of the answers here also didn't work. :(
I have decided to get advice from a friend of mine, and me and him have decided on this approach:
Have my exportData.php simply write the data to $_SESSION, echo a JSON-encoded "success", and then exit
On exit, on the client-side of things, if "success" has been received, have the JavaScript open up a new tab to a file that I have created called downloadFile.php which actually does the downloading.
Why didn't sending the data between files work?
Downloading data entails setting the right headers and printing the data. When you send data to the file to do this (via AJAX), the buffer that the data is printed to is the one for response. You can see this by saying something like
success: function(response)
{
alert(response);
} and see the data that you "downloaded" not get downloaded, but get printed on-screen.
However, if you go to the file instead of simply passing data to it, your data will download, provided that it has access to the data that you are trying to download. You can see examples of this here: www.the-art-of-web.com/php/dataexport/ . In those examples, the data was "static" (that is, only existing in the scope of that PHP file, until download happened).
We then see that we should let another file handle the downloading. Here is what its contents should look like:
<?php
if (!isset($_SESSION))
session_start();
function cleanData(&$str)
{
$str = preg_replace("/\t/", "\\t", $str); // escaping all of the tabs
$str = preg_replace("/\r?\n/", "\\n", $str); // escaping any and all cases of carriage return
// if there is a single double-quote in the string, we wrap the string in quotes, replace every single double-quote with double double-quotes, and
// end with a double-quote
if(strstr($str, '"')) $str = '"' . str_replace('"', '""', $str) . '"';
}
// get the data from $_SESSION
if (isset($_SESSION))
{
$fileName = $_SESSION['fileName'];
$titleRow = $_SESSION['titleRow'];
$rows = $_SESSION['rows'];
// set the excel headers
header("Content-Type: application/vnd.ms-excel");
//header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=\"$fileName\"");
header("Pragma: no-cache");
header("Expires: 0");
// attempt download
array_walk($titleRow, 'cleanData');
echo implode(chr(9), $titleRow) . "\r\n";
// now we echo the data
foreach($rows as $row)
{
array_walk($row, 'cleanData');
echo implode(chr(9), $row) . "\r\n";
}
}
else
{
die('Problem with session variable. Data could not be sent for download.');
}
exit;
?>
Of course, before doing this, make sure that you have 'fileName', 'titleRow', and 'rows' already written to $_SESSION.
This should help anyone having problem downloading HTML table to Excel spreadsheet via PHP, and the best part is that you don't have to bloat your server by downloading an entire library, for potentially the functionality of one button!!

How to transfer a value on DB to JAVASCRIPT through PHP

<?php
// Connect to database server
mysql_connect("192.168.1.101/phpmyadmin/", "root", "praticas") or die (mysql_error ());
// Select database
mysql_select_db("logs_history") or die(mysql_error());
//---> at this point, I want yo pull a VALUE from a column called potencia_ativa and insert in
// the code JS below
mysql_query($strSQL);
// Close the database connection
mysql_close();
?>
//JS code-------------------------------------------------------------------------------------------
<div id="graphHolder" style="height:75%;width:100%"></div>
<script type="text/javascript">
setInterval(
function() {
var d1 = [];
for (var i = 0; i < parseInt(document.getElementById('number').value,10); i += 0.5) {
d1.push([ //----------> pull the value from the php/DB as a FLOAT
]);
}
$.plot("#graphHolder", [{
label: "Label",
data: d1,
lines: { show: true, fill: true }
}
]);
}, 1000);
</script>
Remember, PHP executes on the server and GENERATES the page that runs the Javascript on the client. Putting a value from PHP into the JS code is as simple as:
<script>
var foo = <?php echo json_encode('bar'); ?>;
</script>
If you need to send the PHP data over AFTER the page was generated and already send to the client, then you need to have the client execute an AJAX request to fetch that data from the server. Once PHP has shoved the page out the door, it's basically done and can't "reach out" to the client to do updates on its own.
Also: Note the use of json_encode(). You should NEVER dump output from PHP directly into a Javascript context without it. Anything else puts you at risk of generating JS syntax errors, which will kill the entire JS code block.
Why are you using Interval? Are you trying to update the displaying page data/value automatically after some time passed? That won't work, because if anything changes in database, It won't change in the displaying page, unless you use AJAX to call your php script and return some values, and then, change in the displaying page.
To get a value from PHP script to JAVASCRIPT, you just need to use the php clausule;
var test = <?php echo $variable; ?>
// echo for strings, ints, floats.
// print_r for arrays.
Of course, if your PHP script already injected the data into the page/the data is in the same page you're using JAVASCRIPT, assuming that you have already fetched the data, handled it and etc.

What is the most efficient and correct way of handling PHP array variables within JavaScript and being able it obtain those values using indexing

THE QUESTION
What is the most efficient and correct way of handling PHP array variables within JavaScript and being able it obtain those values using indexing.
I have a MYSQL database and have a PHP script that creates an indexed row array of the database information.
Now that this information is within the array i am comfortable about echoing this data on screen from within PHP.
i.e.
echo $lastplayed[1]['artist'];
My next step is to take the array into JavaScript so that i can use the variable information to display data on screen, make calculations and create an Ajax timer that looks for a value from a variable and refreshes the page..
Its basically a internet radio station that will display what is and has been played and when a counter reaches zero will refresh the page. (the counter being time left of a song)
I could echo each variable into a separate PHP script and then use JavaScript to call each of those PHP scripts that contain the different variables (This seems long-winded) AND puts unnecessary request strain on the MYSQL server
**I really feel that there must be a better way of transferring and handling the data, surely there must be some type of bridge between PHP and JavaScript, should i be looking into JSON ?
So my end result is to be able to take an indexed array from PHP, transfer this array into JavaScript and be able to call on different variables from within the array using indexing (i.e call the variable that resides in result 3 column 3)
And while this is happening i will be using separate PHP and JavaScript files...
Here is my code for the PHP part.
<?php
date_default_timezone_set('Europe/London');
require_once("DbConnect.php");
$sql = "SELECT `artist`, `title`, `label`, `albumyear`, `date_played`, `duration`,
`picture` FROM historylist ORDER BY `date_played` DESC LIMIT 5 ";
$result = $db->query($sql);
$lastplayed = array();
$i = 1;
while ($row=$result->fetch_object()) {
$lastplayed[$i]['artist'] = $row->artist;
$lastplayed[$i]['title'] = $row->title;
$lastplayed[$i]['label'] = $row->label;
$lastplayed[$i]['albumyear'] = $row->albumyear;
$lastplayed[$i]['date_played'] = $row->date_played;
$lastplayed[$i]['duration'] = $row->duration;
$lastplayed[$i]['picture'] = $row->picture;
$i++;
}
$starttime = strtotime($lastplayed[1]['date_played']);
$curtime = time();
$timeleft = $starttime+round($lastplayed[1]['duration']/1000)-$curtime;
$secsremain = (round($lastplayed[1]['duration'] / 1000)-($curtime-$starttime))
?>
Any thoughts on this would be greatly appreciated and thanks so much for your time.
Justin.
PROGRESS:
Thanks for the comments, i really need to take a JavaScript course at this point...
Now i have created a new output.PHP file that does the following
<?php
require_once("dblastplayedarray.php");
echo json_encode($lastplayed);
?>
So this file now echo's out the data i need in a JSON format from my array $lastplayed.
#VCNinc you say that i now can use the following code to take the data into JavaScript
<script>
var array = <?=json_encode($lastplayed)?>;
</script>
Please could you detail where i put the path information in this code so that the program knows where to look for the .PHP file output.php
Am i doing this right.. should i be printing the data into another .PHP file and then use your code to take the array into JavaScript..
Thanks
Justin.
JSON is the bridge!
You can "export" the variable to a json string and print on the output:
echo json_encode($lastplayed);
TIP: if the php file is used to show a html GUI AND you still want output a JSON too, you can create a GET variable like "&json=1" and, before output your HTML GUI, you do a IF. This way tou can use the same php file to output a GUI and the JSON. WHen you do the request via ajax, you call using the "&json=1".
if(isset($_GET['json']) && $_GET['json']==1){
echo json_encode($lastplayed);
exit;
}
Then, use AJAX to download this JSON string by calling your php script.
$.getJSON(url, function (json) {
//here the 'json' variable will be the array
//so you can interact on it if you want
$.each( json, function( key, value ) {
alert( key + ": " + value ); //here you can do anything you want
});
});
If you have a PHP array $array, you can easily export it into JavaScript like this:
<script>
var array = <?=json_encode($array)?>;
</script>
(and from that point you can manipulate it as JSON...)

Categories

Resources