Develop Reference

PHP script to download text from site, convert and store it locally

How can I store text from a site to a local file?
So basically the script needs to do the following:
go to this site (fake site)
http://website/webtv/secure?url=http://streamserver.net/channel/channel.m3u8**&TIMESTAMP**
where TIMESTAMP can be a timestamp to make it unique.
the site will respond with:
{
"url":"http://streamserver.net/channel/channel.m3u8?st=8frnWMzvuN209i-JaQ1iXA\u0026e=1451001462",
"alternateUrl":"",
"Ip":"IPADRESS"
}
Grab the url and convert the text as follows:
http://streamserver.net/channel/channel.m3u8?st=8frnWMzvuN209i-JaQ1iXA\u0026e=1451001462
must be:
http://streamserver.net/channel/channel.m3u8?st=8frnWMzvuN209i-JaQ1iXA&e=1451001462
so \u0026e is replaced by &
and store this text in a local m3u8 file.
I am looking for a script either php or any other code is welcome which can perform this. Any help is appreciated.
I tried a small script just to show the contents but then I get the error:
Failed to open stream: HTTP request Failed!
It seems that php tries to open it as a stream instead of a website. It should see it as a site because only then the response is sent.
<?php
$url = 'http://website/webtv/secure?url=http://streamserver.net/channel/channel.m3u8&1';
$output = file_get_contents($url);
echo $output;
?>

This is not a tutorial website, so I am not going to provide you more details. You can try the following code:
<?php
$json_url = "http://linktoyour.site"; //change the url to your needs
$data = file_get_contents($json_url); //Get the content from url
$json = json_decode($data, true); //Decodes string to JSON Object
$data_to_save=$json["url"]; //Change url to whatever key you want value of
$file = 'm3u8.txt'; //Change File name to your desire
file_put_contents($file, $data_to_save); //Writes to File
?>

I think there is issue with your PHP configuration.
It like as allow_url_fopen is denied.
See more http://php.net/manual/en/filesystem.configuration.php#ini.allow-url-fopen

Improve Page Performance, save PHP array on server?

is it possible to store a PHP-array to my server, right now it always gets created when someone reloads the page from a CSV file but that is unnecessary since the file only chances after each hour.
ATM, the page takes like 9 seconds to load, which is quite long. The CSV file has 10k+ rows with 9 elements per row, so it would be really good for performance if the server didn't have to process 100k elements for each user.
I already have a cronjob for downloading the csv file so it would be good if the parse command would be executed after the download finished, only once per hour.
cronjob:
<?php
function download_remote_file($file_url, $save_to) {
$content = file_get_contents($file_url);
file_put_contents($save_to, $content);
}
download_remote_file(<url here>, realpath(".") . '/dump.csv');
?>
and this happens with every reload of the page:
1st: Parse data to array
$url = 'dump.csv';
$csvData = file_get_contents($url);
$lines = explode(PHP_EOL, $csvData);
$array = array();
foreach ($lines as $line) {
$line = str_replace("\\", "\", $line);
$line = str_replace("#", "#", $line);
$array[] = str_getcsv($line);
2nd: pass array to Javascript
var array = <?php echo json_encode( $array ) ?>;
3rd: create HTML table
//some code
4th: initialise data table plugin
$(document).ready( function () {
createtable();
$('#scoreboard').DataTable( {
"iDisplayLength": 50,
language: {
decimal: ".",
},
"lengthMenu": false,
"bLengthChange": false
} );
} );
Is there something that could be done faster?
Like, as mentioned, save the php array server-side or maybe saving the JS array with the HTML table somehow?
-Innerwolf

After you parse your CSV, do this:
$file = fopen('/tmp/output.js', 'w');
fwrite($file, '<script type="text/javascript">');
fwrite($file, 'var array =');
fwrite($file, json_encode( $array ));
fwrite($file, ';');
fwrite($file, '</script>');
fclose($file);
copy('/path/to/script.js', '/path/to/script.js.bak');
move('/tmp/output.js', '/path/to/script.js');
Then, later on when you are outputting the HTML, you just need to stick in a:
<script type="text/javascript" src="/scripts/script.js">
in the header. People's browsers should cache it properly too. Note the copy and move -- you don't strictly need to make a backup copy, but you MUST use a move() to replace the 'live' script -- move() is atomic, more or less, and won't result in anyone getting a half-file.
Also, note that you'll need write permissions to where the script is -- there are ways to keep this pretty secure (not letting your PHP script write all over the hard drive), but that's out of scope here.

Since you mention getting the data on an hourly basis I suggest the following:
grab the CSV file with cron and store the data in a database on an hourly basis
configure your data tables component to use server side data
This way you won't force every user to download the entire array at once on every first page load.
The server side script only fetches the number of records that need to be displayed on that particular page in the table.

check script status in PHP using ajax

I have a file upload page in my application. I need to show "Uploading" while file is uploading then show "Processing" while file is processing. Then after completion of script my page got redirected to some url.
I have tried to use PHP SESSIONS in the script. As in code below:
$_SESSION['uploaded']=0;
if (!empty($_FILES)) {
$tempFile = $_FILES['file']['tmp_name'];
$targetPath = dirname( __FILE__ ) . $ds. $storeFolder . $ds;
$_FILES['file']['name']=date('Ymdhis').$_FILES['file']['name'];
$targetFile = $targetPath. $_FILES['file']['name'];
if(move_uploaded_file($tempFile,$targetFile)){
$_SESSION['uploaded']=1;
//some processing here which takes some 4-5second to complete
}
}
After file upload complete I update session. I am checking session every second by calling following function in javascript:
function countdown(seconds){
console.log(<?php echo $_SESSION['uploaded']; ?>);
if(<?php echo $_SESSION['uploaded']; ?>==0){
setTimeout(function() {
//uploading
seconds--;
countdown(seconds);
}, 1000);
}
else{
//processing
}
}
After searching from google for long time I came to know that in a single script SESSION is locked till script execution completed. Then I used session_write_close(); But it also not works. I am always getting 0 value of SESSION.
Please help me figuring out solution in simplest way. Thanks.
UPDATE
Unable to make it work with Ajax request also. So further tried using the MySQL table.
What I do is create table when upload script is called. Then insert value of status=0 in it using following code:
$session=session_id();
$stmt=$conn->prepare("DROP TABLE IF EXISTS $session");
$stmt->execute();
$stmt=$conn->prepare("CREATE TABLE $session (id INT(11), status INT(11))");
$stmt->execute();
$stmt=$conn->prepare("INSERT INTO $session VALUES(1,0)");
$stmt->execute();
Then after upload completion I update the status to 1 and do the processing on file.
Then after successful completion of script I redirect to result page and drop table using session_id().
But My Ajax script which is checking status every second doesn't respond till the upload.php script ends. I have tried closing connection after every query but in vain. Code on getstatus.php
<?php
session_start();
$session=session_id();
require_once('connect.php');
$stmt=$conn->prepare("SELECT * FROM $session WHERE id=1");
$stmt->execute();
$res=$stmt->fetch(PDO::FETCH_ASSOC);
echo $res['status'];
?>
Unable to find solution for it till now. Help is greatly appreciated. Thanks.

Instead of invoking a PHP process on the server side every second, you could use a static file to check the upload state.
When generating the upload form for the client:
Create a tempnam for a directory that is accessible for the
client.
Write 'uploading' to the temporary file
Store the filename in the session. (Be aware: The user might open multiple upload forms. Store the filenames in an array)
Send the filename to the client as a hidden field.
On the server side after user submitted the form:
Check if filename sent from the client matches a filename stored in the session.
Write 'processing' to the state file
At the end of your upload script write 'finished' to the state file
On the client side after user submits the form, check the upload state by doing ajax requests on the state file.
Remarks
Disable caching for the state file with .htaccess. If this is no option you can achieve the same behavior with a php state script and the upload state saved to a session variable instead of a state file.
To make sure all generated files are deleted register a destroy handler that deletes files generated in the session: http://php.net/manual/en/function.session-set-save-handler.php

<?php echo $_SESSION['uploaded']; ?> is preprocessed by PHP only once, just before this javascript is sent to client. That said, the javascript on client looks like:
function countdown(seconds){
console.log(0);
if(0==0){
setTimeout(function() {
//uploading
seconds--;
countdown(seconds);
}, 1000);
}
else{
//processing
}
}
You should find other way (ajax?) to update information on the client side.

This became too long for a comment.
I'm unsure how you'd respond with progress information with PHP. I tried once and failed.
Socket.io is awesome in Node.js and there is a PHP server emitter. I would potentially give that a go. It should offer near instantaneous communication without waiting for scripts to complete.
Alternatively I would check out Jquery upload, it has a PHP server script. Supports progress bars Jquery Upload. Either implement it directly or check out the source code for how display progress info. I tried having a quick look but couldn't identify how they do it easily.

Why use DATABASE if you can do it on server ?
To save your bandwidth and database traffic you can seperate your process into 2 file
Create upload.php to serve uploading process
$_SESSION['uploaded']=0;
if (!empty($_FILES)) {
$tempFile = $_FILES['file']['tmp_name'];
$targetPath = dirname( __FILE__ ) . $ds. $storeFolder . $ds;
$_FILES['file']['name']=date('Ymdhis').$_FILES['file']['name'];
$targetFile = $targetPath. $_FILES['file']['name'];
if(move_uploaded_file($tempFile,$targetFile)){
// Save path to session var
$_SESSION['uploaded']=$targetFile;
//You can tell client if the uploading process were done and show 'Processing ...'
// Place some code
exit;
}
}
Next, create a file called progress.php
// check
if(!empty($_SESSION['uploaded'])){
// Do your processing code here
// Remove session
unset($_SESSION['uploaded']);
// Then send response to client after your processing were done
echo 'Done';
exit;
}
You can redirect client using jquery as you tagged it. Good luck

PHP inside Javascript in a registration form (for validation)

I'm developing a registration form for my site. Actually when a visitor choose an username, a php query to my MySQL DB is used to control if it's already used and if so, a javascript windowd appear.
Can i use a PHP query inside Javascript for displaing a real-time notice near the form (using HTML5)?
<script>
var username = document.getElementById('username');
var userdb = <? php control_username($username); ?>
var checkUsername = function () {
if (userdb.value == true) {
username.setCustomValidity('Username already used');
} else {
username.setCustomValidity('');
}
};
username.addEventListener('change', checkUsername, false);
</script>
and here there's the php function:
<?php function control_username($username){
$db=connessione_db();
$query = "SELECT username FROM utente WHERE username = '$username';";
$result = mysql_query($query);
$row = mysql_fetch_row($result);
mysql_close();
if($row[0]==$username){
return TRUE;
}
else{
return FALSE;
}
$query=NULL;
}
how can i do?

You can use AJAX or jQuery AJAX to send a request to a php page, Check if the username exists, return the result, and display it using Javascript again.
Here is the jQuery sample:
<script>
$.ajax({
type: 'POST',
url : 'checkUsername.php',
data: {'username' : $('#username').html()},
cache : false,
success: function(data){
if(data == 'exists')
//username exists
alert('username already exists!');
},
error: function(request , status , error){
alert(request.resposeText);
}
});
</script>
and this should be your checkUsername.php file:
<?php
$db=connessione_db();
$query = "SELECT count(*) as count FROM utente WHERE username = '$username'";
$result = mysql_query($query);
$row = mysql_fetch_row($result);
mysql_close();
if($row[count] > 0)
echo 'exists';
else
echo '';

PHP is run on the server, Javascript is run on the client's machine. Once your PHP script has generated a page and sent it to the user, it doesn't run any longer. Your client has no knowledge of a PHP script. It can't directly run, call, or read anything about your PHP script because it resides solely on the server (never on the client's machine). Your client's machine does, however, know about your Javscript since it has been sent along with the page. If you want to access PHP functionality from your page, you can either send a GET/POST call to the server and reload the page, or use AJAX to make the call in the background. Check out Jquery's implementation of AJAX calls, it makes using it pretty simple.

No you can't do it like that. PHP is serverside, Javascript clientside. The moment Javascript is executed is the code working clientside. All PHP code is fixed.
Compare it to the combination of HTML and PHP in an HTML page. It is fixed there as well. Same applies to PHP in Javascript.
Here are some answers from related questions on stackoverflow:
How to put php inside javascript?
How to embed php in javascript?
Here is an example from ajaxref, showing the basics:
http://ajaxref.com/ch3/asyncsend.html
This is another tutorial showing how an ajax call is handled:
http://code.tutsplus.com/articles/how-to-make-ajax-requests-with-raw-javascript--net-4855
I advice you to first understand this process and later on start using a framework like jQuery.

How do I render javascript from another site, inside a PHP application?

What I'm trying to do is read a specific line from a webpage from inside of my PHP application. This is my experimental setup thus far:
<?php
$url = "http://www.some-web-site.com";
$file_contents = file_get_contents($url);
$findme = 'text to be found';
$pos = strpos($file_contents, $findme);
if ($pos == false) {
echo "The string '$findme' was not found in the string";
} else {
echo "The string '$findme' was found in the string";
echo " and exists at position $pos";
}
?>
The "if" statements contain echo operators for now, this will change to database operators later on, the current setup is to test functionality.
Basically the problem is, with using this method any java on the page is returned as script. What I need is the text that the script is supposed to render inside the browser. Is there any way to do this within PHP?
What I'm ultimately trying to achieve is updating stock from within an ecommerce site via reading the stock level from the site's supplier. The supplier does not use RSS feeds for this.

cURL does not have a javascript parser. as such, if the content you are trying to read is placed in the page via Javascript after initial page render, then it will not be accesible via cURL.

The result of the script is supposed executed and return back to your script.
PHP doesn't support any feature about web browser itself.
I suggest you try to learn about "web crawler" and "webbrowsers" which are included in .NET framework ( not PHP )
so that you can use the exec() command in php to call it.
try to find out the example code of web crawler and web browsers on codeproject.com
hope it works.

You can get the entire web page as a file like this:
function get_data($url)
{
$ch = curl_init();
$timeout = 5;
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$returned_content = get_data('http://example.com/page.htm');
$my_file = 'file.htm';
$handle = fopen($my_file, 'w') or die('Cannot open file: '.$my_file);
fwrite($handle, $returned_content);
Then I suppose you can use a class such as explained in this link below as a guide to separate the javascript from the html (its in the head tags usually). for linked(imported) .js files you would have to repeat the function for those urls, and also for linked/imported css. You can also grab images if you need to save them as files.
http://www.digeratimarketing.co.uk/2008/12/16/curl-page-scraping-script/