Develop Reference

Cross-Domain Rss Feed Request?

Ok, so for about a week now I've been doing tons of research on making xmlhttprequests to servers and have learned a lot about CORS, ajax/jquery request, google feed api, and I am still completely lost.
The Goal:
There are 2 sites in the picture, both I have access to, the first one is a wordpress site which has the rss feed and the other is my localhost site running off of xampp (soon to be a published site when I'm done). I am trying to get the rss feed from the wordpress site and display it on my localhost site.
The Issue:
I run into the infamous Access-Control-Allow-Origin error in the console and I know that I can fix that by setting it in the .htaccess file of the website but there are online aggregators that are able to just read and display it when I give them the link. So I don't really know what those sites are doing that I'm not, and what is the best way to achieve this without posing any easy security threats to both sites.
I highly prefer not to have to use any third party plugins to do this, I would like to aggregate the feed through my own code as I have done for an rss feed on the localhost site, but if I have to I will.
UPDATE:
I've made HUGE progress with learning php and have finally got a working bit of code that will allow me to download the feed files from their various sources, as well as being able to store them in cache files on the server. What I have done is set an AJAX request behind some buttons on my site which switches between the rss feeds. The AJAX request POSTs a JSON encoded array containing some data to my php file, which then downloads the requested feed via cURL (http_get_contents copied from a Github dev as I don't know how to use cURL yet) link and stores it in a md5 encoded cache file, then it filters what I need from the data and sends it back to the front end. However, I have two more questions... (Its funny how that works, getting one answer and ending up with two more questions).
Question #1: Where should I store both the cache files and the php files on the server? I heard that you are supposed to store them below the root but I am not sure how to access them that way.
Question #2: When I look at the source of the site through the browser as I click the buttons which send an ajax request to the php file, the php file is visibly downloaded to the list of source files but also it downloads more and more copies of the php file as you click the buttons, is there a way to prevent this? I may have to implement another method to get this working.
Here is my working php:
//cURL http_get_contents declaration
<?php
function http_get_contents($url, $opts = array()) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
curl_setopt($ch, CURLOPT_USERAGENT, "{$_SERVER['SERVER_NAME']}");
curl_setopt($ch, CURLOPT_URL, $url);
if (is_array($opts) && $opts) {
foreach ($opts as $key => $val) {
curl_setopt($ch, $key, $val);
}
}
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
if (false === ($retval = curl_exec($ch))) {
die(curl_error($ch));
} else {
return $retval;
}
}
//receive and decode $_POSTed array
$post = json_decode($_POST['jsonString'], true);
$url = $post[0];
$xmn = $post[1]; //starting item index number (i.e. to return 3 items from the feed, starting with the 5th one)
$xmx = $xmn + 3; //max number (so three in total to be returned)
$cache = '/tmp/' . md5($url) . '.html';
$cacheint = 0; //this is how I set if the feed will be downloaded from the site it is from, or if it will be read from the cache file, I will implement a way to check if there is a newer version of the file on the other site in the future
//if the cache file doesn't exist, download feed and write contents to cache file
if(!file_exists($cache) || ((time() - filemtime($cache)) > 3600 * $cacheint)) {
$feed_content = http_get_contents($url);
if($feed_content = http_get_contents($url)) {
$fp = fopen($cache, 'w');
fwrite($fp, $feed_content);
fclose($fp);
}
}
//parse and echo results
$content = file_get_contents($cache);
$x = new SimpleXmlElement($content);
$item = $x->channel->item;
echo '<tr>';
for($i = $xmn; $i < $xmx; $i++) {
echo '<td class="item"><p class="title clear">' .
$item[$i]->title .
'</p><p class="desc">' .
$desc=substr($item[$i]->description, 0, 250) .
'... <a href="' .
$item[$i]->link .
'" target="_blank">more</a></p><p class="date">' .
$item[$i]->pubDate .
'</p></td>';
}
echo '</tr>';
?>

PHP And AJAX Download of a few MB file freezes website

Hello ive searched everywhere to find the answer however none of the solutions ive tried helped
What i am building is a site which connects to Youtube to allow users to search and download videos as MP3 files. I have built the site with the search etc however i am having a problem with the download part (ive worked out how to get the youtube audio file). The format for the audio is originally audio/mp4 so i need to convert it to mp3 however first i need to get the file on the server
So on the download page ive made a script that sends an ajax request to the server to start downloading the file. It then sends a request to a different page every few seconds to find out the progress and update it on the page the user is viewing.
However the problem is while the video is downloading the whole website freezes (all the pages dont load until the file is fully downloaded) and so when the script tries to find out the progress it cant until its fully done.
The file which downloads:
<?php
session_start();
if (isset($_GET['yt_vid']) && isset($_GET['yrt'])) {
set_time_limit(0); // to prevent the script from stopping execution
include "assets/functions.php";
define('CHUNK', (1024 * 8 * 1024));
if ($_GET['yrt'] == "gphj") {
$vid = $_GET['yt_vid'];
$mdvid = md5($vid);
if (!file_exists("assets/videos/" . $mdvid . ".mp4")) { // check if the file already exists, if not proceed to downloading it
$url = urlScraper($vid); // urlScraper function is a function to get the audio file, it sends a simple curl request and takes less than a second to complete
if (!isset($_SESSION[$mdvid])) {
$_SESSION[$mdvid] = array(time(), 0, retrieve_remote_file_size($url));
}
$file = fopen($url, "rb");
$localfile_name = "assets/videos/" . $mdvid . ".mp4"; // The file is stored on the server so it doesnt have to be downloaded every time
$localfile = fopen($localfile_name, "w");
$time = time();
while (!feof($file)) {
$_SESSION[$mdvid][1] = (int)$_SESSION[$mdvid][1] + 1;
file_put_contents($localfile_name, fread($file, CHUNK), FILE_APPEND);
}
echo "Execution time: " . (time() - $time);
fclose($file);
fclose($localfile);
$result = curl_result($url, "body");
} else {
echo "Failed.";
}
}
}
?>

I also had that problem in the past, the reason that it does not work is because the session can only be once open for writing.
What you need to do is modify your download script and use session_write_close() each time directly after writing to the session.
like:
session_start();
if (!isset($_SESSION[$mdvid])) {
$_SESSION[$mdvid] = array(time(), 0, retrieve_remote_file_size($url));
}
session_write_close();
and also in the while
while (!feof($file)) {
session_start();
$_SESSION[$mdvid][1] = (int)$_SESSION[$mdvid][1] + 1;
session_write_close();
file_put_contents($localfile_name, fread($file, CHUNK), FILE_APPEND);
}

jQuery load PHP that display flash content

What my php side does is load data use 'id' from database, and then show the raw data of swf files.
The code of php like this:
<?php
if(isset($_GET['id']))
{
include ('database.php');
$record = mysql_query("SELECT `Flash`, `FlashBlob` FROM `information` WHERE `id` = '". $_GET['id'] ."'; ", $link);
$swf = mysql_fetch_array($record);
mysql_close($link);
header("Content-type: " . $swf['Flash']);
echo $swf['FlashBlob'];
exit;
}
So if I just load the php in the web link, it goes well( the whole php page will show the flash I stored in database).
In my main page, I want to load the page into my div($("#gameArea")), I have tried:
var xmlhttp;
if (window.XMLHttpRequest) {
xmlhttp = new XMLHttpRequest();
} else {
xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
}
xmlhttp.open("GET", 'api/swfInfo.php?id=' + id,true);
xmlhttp.send();
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
document.getElementById('gameArea').innerHTML=xmlhttp.responseText;
}
}
but the result is it load only raw data( binary data) not the flash(swf file to be shown), what should I do?
I even not sure whether I should find a flash plugin for my flash or not
because I put my flash(swf file) in the database for some reason......
Please help me for the direction, thanks in advance!

Your overcomplicating the problem. If in-fact your swfInfo.php is outputting the swf file's bytes with a content type of application/x-shockwave-flash all you need to do is set the data attribute of your <object> tag to that URL.
<object type="application/x-shockwave-flash" data="api/swfInfo.php?id=7">
<param /> <!-- parameters -->
</object>
I would also recommend a content-length declaration to ensure your connection closes properly when loading files this way.
header('Content-length: ' . mb_strlen($swf['FlashBlob']));
header('Connection: close');

Try the following:
header('Content-Type: application/x-shockwave-flash');
header("Content-Disposition:attachment; filename="test.swf");
My questions are:
What is $swf['Flash'] ? Maybe there is the error?
Have you tried your script with a file_read_content() just for debugging reasons?

Read range chunks chunked with php

I have an input field where I paste a download url.
After that, the I use an AJAX request to get the fileinfos such as headerinfo, content-length, mime type & in case I use curl accept-ranges.
I then start a consecutive loop of xhr2 requests with ranges to my php file.
http://www.example.com/chunks.php?url=http://url.com/someFile.ext&range=0-1024
http://www.example.com/chunks.php?url=http://url.com/someFile.ext&range=1024-2048
....
I can also change it to
http://www.example.com/chunks.php?url=http://url.com/someFile.ext&range=0-1024
http://www.example.com/chunks.php?url=http://url.com/someFile.ext&range=1025-2049
....
depending where my script starts to read the file.
My first approach was using cUrl & setting the ranges
<?php
$ch=curl_init();
curl_setopt($ch,CURLOPT_URL,$_GET['url']);
curl_setopt($ch,CURLOPT_RANGE,$_GET['range']);
curl_setopt($ch,CURLOPT_BINARYTRANSFER,1);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
$result=curl_exec($ch);
curl_close($ch);
echo $result;
?>
works great but if the range chunks are bigger than 1mb there is no animation on the client side onprogress event using ajax.
i prolly could use a custom CURLOPT_READFUNCTION... but i don't know how that works... so i changed approach and used the simple fopen
<?php
$r=explode('-',$_GET['range']);//get (from to) ranges
$cc=($r[1]-$r[0]); //Calculate Client Chunk length
$sc=128; //Set the Server chunk length
$b=""; //Buffer
$bytes=0; //bytes read
$h=fopen($_GET['url'],"rb"); // open the url
fseek($h,$r[0]); // jump to the from pointer retrieved from links
while($bytes<$cc){ //while bytes read is smaller than my client chunk
$sc=(($bytes+$sc)>$cc?($cc-$bytes):$sc); //prolley an error here
//if the server chunk + bytes read is bigger than the client chunk
//then the server chunk is clinet chunk - bytes read
$b=fread($h,$sc); // read the buffer
$bytes+=strlen($b); //add the buffer length to bytes read
echo $b;// echo the buffer
ob_flush(); // flush
flush(); // flush
}
fclose($h); //close
?>
now this works ... I get the right animation on the client and also the final size is correct the pointers should be ok (0-1024,1024-2048) as I use fseek && fread.
but the file is corrupt.
Now after some tests ... this is very slow.
A better approach would be cUrl with CURLOPT_READFUNCTION or fsoket open...
so I guess:
<?php
function $READ(){
//here i need small chuncks of the response flushed.
}
$ch=curl_init();
curl_setopt($ch,CURLOPT_URL,$_GET['url']);
curl_setopt($ch,CURLOPT_RANGE,$_GET['range']);
curl_setopt($ch,CURLOPT_BINARYTRANSFER,1);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_READFUNCTION,$READ);
$result=curl_exec($ch);
curl_close($ch);
echo $result;
?>
If you have a better solution I'm open to everything that uses javascript and php.
The point of this is to create a download manager with resume that stores the file into the window.webkitRequestFileSystem without filling the memory of the browser.
Let's say the client has chunks of 8mb and the server chunks are 256kb ..
then every 8mb of the chunk is appended to a file previously created with window.webkitRequestFileSystem
and every 256kb I have an update of the average download speed and this way I can create a nice animation.
The php on the server uses only 256kb ram and the client browser can empty the garbage collection every 8mb (theoretically).
EDIT2
For this code I found a solution:
the code allows you to get ranges for example:0-100
and get the output of this 100bytes chunked !!
this allows you to have a AJAX script that has a continuos flawless PROGRESSBAR
<?php
function w($ch,$chunk){
echo $chunk;
ob_flush();
flush();
return strlen($chunk);
};
$ch=curl_init();
curl_setopt($ch,CURLOPT_URL,$_GET['url']);
curl_setopt($ch,CURLOPT_RANGE,$_GET['range']);
curl_setopt($ch,CURLOPT_BINARYTRANSFER,1);
curl_setopt($ch,CURLOPT_WRITEFUNCTION,w);
curl_exec($ch);
curl_close($ch);
?>
But I hope you guys have a better solution at all!! thanks

I could get it to work with PHP curl's CURLOPT_WRITEFUNCTION callback setting. The following example callback function curl_write_flush intended for that curl option writes every chunk received and flushes the output to the browser.
<?php
/**
* CURLOPT_WRITEFUNCTION which flushes the output buffer and the SAPI buffer.
*
* #param resource $curl_handle
* #param string $chunk
*/
function curl_write_flush($curl_handle, $chunk)
{
echo $chunk;
ob_flush(); // flush output buffer (Output Control configuration specific)
flush(); // flush output body (SAPI specific)
return strlen($chunk); // tell Curl there was output (if any).
};
$curl_handle = curl_init($_GET['url']);
curl_setopt($curl_handle, CURLOPT_RANGE, $_GET['range']);
curl_setopt($curl_handle, CURLOPT_BINARYTRANSFER, 1);
curl_setopt($curl_handle, CURLOPT_WRITEFUNCTION, 'curl_write_flush');
curl_exec($curl_handle);
curl_close($curl_handle);
I tried with small files and big files and it works great but you can't set custom chunk size.
Download stream is the same speed as I can get with my ISP.
If you have anything better i'm open for any answer.

Request a file with a custom header

I have an unusual requirement. Essentially I need a way so that, when the user clicks on a link or button, they will receive a PDF. The tricky part here is that the server won't process the request at all unless a custom header is sent with it (otherwise it deems the person logged out and sends them to the login screen).
At the moment the way the header works cannot be changed so please don't dwell on it; it will get changed in the future and is an internal application that I have no control over.
The options I have explored:
Using an iframe or simply opening a new window with some sort of path that will return the PDF. This can't work because I cannot specify the required header for the PDF and would be redirected before reaching the PDF itself.
Using a form and submitting the request can't work because I can't
add any custom headers to forms (only XHR and plugins can, AFAIK).
Using XHR can't work because, while it can add the header and
retrieve the file, there is no way to save it on the client side.
It would appear my only options at this point are essentially:
Use some sort of plugin such as Flash or Silverlight to request the file.
Force the change of the requirement much earlier than expected so that a header is no longer required.
Is there anything I am missing here? I'm hoping someone can either verify my findings or point me to something I missed because, as far as I can tell, there isn't really anything I can do here.
EDIT: This seems apt and confirms what I was thinking: XMLHttpRequest to open PDF in browser

Tested to work in chrome:
function toBinaryString(data) {
var ret = [];
var len = data.length;
var byte;
for (var i = 0; i < len; i++) {
byte=( data.charCodeAt(i) & 0xFF )>>> 0;
ret.push( String.fromCharCode(byte) );
}
return ret.join('');
}
var xhr = new XMLHttpRequest;
xhr.open( "GET", "/test.pdf" ); //I had test.pdf this on my local server
xhr.addEventListener( "load", function(){
var data = toBinaryString(this.responseText);
data = "data:application/pdf;base64,"+btoa(data);
document.location = data;
}, false);
xhr.setRequestHeader("magic", "header" );
xhr.overrideMimeType( "application/octet-stream; charset=x-user-defined;" );
xhr.send(null);
You can change application/pdf to application/octet-stream to have download prompt. But it's pretty easy to download from the chrome's reader as well.
In firefox nothing happens I guess it's because I don't have a plugin to deal with application/pdf installed. Changing to application/octet-stream will prompt a dl.
With IE I suppose you need some kind of VBScript/ActiveX hackery
If the file is huge, using data uri might crash the browser, in that case you can use BlobBuilder and Object URLs.

Instead of linking to the .PDF file, instead do something like
Download my eBook
which outputs a custom header, opens the PDF (binary safe) and prints the data to the user's browser, then they can choose to save the PDF despite their browser settings. The pdf_server.php should look like this:
header("Content-Type: application/octet-stream");
$file = $_GET["file"] .".pdf";
header("Content-Disposition: attachment; filename=" . urlencode($file));
header("Content-Type: application/force-download");
header("Content-Type: application/octet-stream");
header("Content-Type: application/download");
header("Content-Description: File Transfer");
header("Content-Length: " . filesize($file));
flush(); // this doesn't really matter.
$fp = fopen($file, "r");
while (!feof($fp))
{
echo fread($fp, 65536);
flush(); // this is essential for large downloads
}
fclose($fp);
EDIT: The only way to add headers to a request from inside a browser (client-side) is use the XmlHttpRequest setRequestHeader method.
xhr.setRequestHeader('custom-header', 'value');