Develop Reference

Login to site with PHP cURL

I'm trying login to a remote site, by having curl to the login form.
I want to redirect to another subdomain and get content.
The code I have doesn't seem to work and only tries to show the main page of the site.
<?php
$username = 'user';
$password = 'pass';
$loginUrl = 'https://site_url';
//init curl
$ch = curl_init();
//Set the URL to work with
curl_setopt($ch, CURLOPT_URL, $loginUrl);
// ENABLE HTTP POST
curl_setopt($ch, CURLOPT_POST, 1);
//Set the post parameters
curl_setopt($ch, CURLOPT_POSTFIELDS, 'user='.$username.'&password='.$password);
//Handle cookies for the login
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt ($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
//Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
//not to print out the results of its query.
//Instead, it will return the results as a string return value
//from curl_exec() instead of the usual true/false.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//execute the request (the login)
$store = curl_exec($ch);
//the login is now done and you can continue to get the
//protected content.
//set the URL to the protected file
curl_setopt($ch, CURLOPT_URL, 'https://site_url/statistics');
//execute the request
$content = curl_exec($ch);
curl_close($ch);
//save the data to disk
file_put_contents('~/file.txt', $content);
?>

Does it have to be via cURL?
Have you considered leveraging a library like Guzzle?
They have good documentation that covers scenarios as you have described.
https://docs.guzzlephp.org/en/stable/quickstart.html#post-form-requests
Something like the below but could be laid out better
use GuzzleHttp\Client;
use GuzzleHttp\Exception\BadResponseException;
$client = new Client(['base_uri' => 'https://url_site/', 'cookies' => true]);
// Login
try {
$client->request(
'POST',
'login/',
[
'form_params' => [
'username' => $username,
'password' => $password,
],
]
);
} catch (BadResponseException $e) {
echo "Error Logging On for User {$username}";
exit;
}
// Navigate
$response = $client->request('GET', '/statistics');
$content = $response->getBody()->getContents();
file_put_contents('~/file.txt', $content);

PHP cURL request to JavaScript

I need to convert this PHP cURL request to JavaScript. I'm not exactly sure how to go about doing it.
I basically need to make this request be client sided instead of server sided.
function createSite($template_id) {
//create array with data
$data = array("template_id"=>$template_id);
//turn data into json to pass via cURL
$data = json_encode($data);
print $data;
//Set cURL parameters
$ch = curl_init();
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch,
CURLOPT_URL,'https://api.duda.co/api/sites/multiscreen/create');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, API_USER.':'.API_PASS);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type:
application/json'));
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
//execute cURL call and get template_idte data
$output = curl_exec($ch);
//check for errors in cURL
if(curl_errno($ch)) {
die('Curl error: ' . curl_error($ch));
}
//decode result
$output = json_decode($output);
//return unique site_name
return $output->site_name;

XmlHttpRequest - post JSON - json string too many quotes (i think) [duplicate]

I looked around a lot before posting this question so my apologies if it is on another post and this is only my second quesiton on here so apologies if I don't format this question correctly.
I have a really simple web service that I have created that needs to take post values and return a JSON encoded array. That all worked fine until I was told I would need to post the form data with a content-type of application/json. Since then I cannot return any values from the web service and it is definitely something to do with how I am filtering their post values.
Basically in my local setup I have created a test page that does the following -
$curl = curl_init();
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json',
'Content-Length: ' . strlen($data))
);
curl_setopt($curl, CURLOPT_URL, 'http://webservice.local/'); // Set the url path we want to call
$result = curl_exec($curl);
//see the results
$json=json_decode($result,true);
curl_close($curl);
print_r($json);
On the webservice I have this (I have stripped out some of the functions) -
<?php
header('Content-type: application/json');
/* connect to the db */
$link = mysql_connect('localhost','root','root') or die('Cannot connect to the DB');
mysql_select_db('webservice',$link) or die('Cannot select the DB');
if(isset($_POST['action']) && $_POST['action'] == 'login') {
$statusCode = array('statusCode'=>1, 'statusDescription'=>'Login Process - Fail');
$posts[] = array('status'=>$statusCode);
header('Content-type: application/json');
echo json_encode($posts);
/* disconnect from the db */
}
#mysql_close($link);
?>
Basically I know that it is due to the $_POST values not being set but I can't find what I need to put instead of the $_POST. I tried
json_decode($_POST), file_get_contents("php://input") and a number of other ways but I was shooting in the dark a bit.
Any help would be greatly appreciated.
Thanks, Steve
Thanks Michael for the help, that was a definite step forward I now have at least got a repsonse when I echo the post....even if it is null
updated CURL -
$curl = curl_init();
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($curl, CURLOPT_URL, 'http://webservice.local/');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, json_encode($data));
updated php on the page that the data is posted to -
$inputJSON = file_get_contents('php://input');
$input= json_decode( $inputJSON, TRUE ); //convert JSON into array
print_r(json_encode($input));
As I say at least I see a response now wheras prior it was returning a blank page

You have empty $_POST. If your web-server wants see data in json-format you need to read the raw input and then parse it with JSON decode.
You need something like that:
$json = file_get_contents('php://input');
$obj = json_decode($json);
Also you have wrong code for testing JSON-communication...
CURLOPT_POSTFIELDS tells curl to encode your parameters as application/x-www-form-urlencoded. You need JSON-string here.
UPDATE
Your php code for test page should be like that:
$data_string = json_encode($data);
$ch = curl_init('http://webservice.local/');
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json',
'Content-Length: ' . strlen($data_string))
);
$result = curl_exec($ch);
$result = json_decode($result);
var_dump($result);
Also on your web-service page you should remove one of the lines header('Content-type: application/json');. It must be called only once.

Hello this is a snippet from an old project of mine that uses curl to get ip information from some free ip databases services which reply in json format. I think it might help you.
$ip_srv = array("http://freegeoip.net/json/$this->ip","http://smart-ip.net/geoip-json/$this->ip");
getUserLocation($ip_srv);
Function:
function getUserLocation($services) {
$ctx = stream_context_create(array('http' => array('timeout' => 15))); // 15 seconds timeout
for ($i = 0; $i < count($services); $i++) {
// Configuring curl options
$options = array (
CURLOPT_RETURNTRANSFER => true, // return web page
//CURLOPT_HEADER => false, // don't return headers
CURLOPT_HTTPHEADER => array('Content-type: application/json'),
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_ENCODING => "", // handle compressed
CURLOPT_USERAGENT => "test", // who am i
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 5, // timeout on connect
CURLOPT_TIMEOUT => 5, // timeout on response
CURLOPT_MAXREDIRS => 10 // stop after 10 redirects
);
// Initializing curl
$ch = curl_init($services[$i]);
curl_setopt_array ( $ch, $options );
$content = curl_exec ( $ch );
$err = curl_errno ( $ch );
$errmsg = curl_error ( $ch );
$header = curl_getinfo ( $ch );
$httpCode = curl_getinfo ( $ch, CURLINFO_HTTP_CODE );
curl_close ( $ch );
//echo 'service: ' . $services[$i] . '</br>';
//echo 'err: '.$err.'</br>';
//echo 'errmsg: '.$errmsg.'</br>';
//echo 'httpCode: '.$httpCode.'</br>';
//print_r($header);
//print_r(json_decode($content, true));
if ($err == 0 && $httpCode == 200 && $header['download_content_length'] > 0) {
return json_decode($content, true);
}
}
}

you can put your json in a parameter and send it instead of put only your json in header:
$post_string= 'json_param=' . json_encode($data);
//open connection
$ch = curl_init();
//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_POST, 1);
curl_setopt($ch,CURLOPT_POSTFIELDS, $post_string);
curl_setopt($curl, CURLOPT_URL, 'http://webservice.local/'); // Set the url path we want to call
//execute post
$result = curl_exec($curl);
//see the results
$json=json_decode($result,true);
curl_close($curl);
print_r($json);
on the service side you can get your json string as a parameter:
$json_string = $_POST['json_param'];
$obj = json_decode($json_string);
then you can use your converted data as object.

Scraping data in dynamic sites

I'm trying to scrape data from our local government. What I want is address from kids adoption offices. Here, in Brazil, all adoptions go through the government. So I have the URL of one office, there are 2 or 3 thousands more. But if I can manage to get one, the others will be easy.
I made many attempts, bellow I show three.
The problem could be related to a Javascript (Ajax maybe) that refresh the page.
Note: I am not a PHP developer.
First attempt
echo '<html><head></head><body>';
echo '<h1>Scraper PHP GET 1</h1>';
echo ini_get("allow_url_fopen");
echo ini_get("allow_url_fopen");
// I used this url for test
//$url = 'http://www.portaldaadocao.com.br';
//This is the URL that I really want
$url = 'http://www.cnj.jus.br/cna/Controle/ConsultaPublicaBuscaControle.php?transacao=CONSULTA&vara=2673';
$html = file_get_contents($url);
var_dump($html);
echo '</body></html>';
// Output
// 11
// Warning:
file_get_contents(http://www.cnj.jus.br/cna/Controle/ConsultaPublicaBuscaControle.php?
transacao=CONSULTA&vara=2673) [function.file-get-contents]: failed to open stream: HTTP
request failed! HTTP/1.1 404 Not Found in /home/rsl/www/sc01_get.php on line 14
// bool(false)
Second attempt
echo '<html><head></head><body>';
echo '<h1>Scraper PHP CURL 3</h1>';
// I used this url for test
//$url = 'http://www.portaldaadocao.com.br';
//This is the URL that I really want
$url = 'http://www.cnj.jus.br/cna/Controle/ConsultaPublicaBuscaControle.php?transacao=CONSULTA&vara=2673';
$curl = curl_init($url);
#curl_setopt($curl, CURLOPT_POSTFIELDS, "foo");
#curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
#curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST");;
$html=#curl_exec($curl);
if (!$html) {
echo "<br />cURL error number:" .curl_errno($curl);
echo "<br />cURL error:" . curl_error($curl);
exit;
}
else{
echo '<br>begin HTML[';
echo $html;
echo '<br>]end html ';
}
echo '</body></html>';
// Output
// 1
third attempt
function curl($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.6 (KHTML, like Gecko) Chrome/16.0.897.0 Safari/535.6');
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_REFERER, "http://www.windowsphone.com");
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
echo '<html><head></head><body>';
echo '<h1>Scraper PHP CURL 5</h1>';
// I used this url for test
//$url = 'http://www.portaldaadocao.com.br';
//This is the URL that I really want
$url = 'http://www.cnj.jus.br/cna/Controle/ConsultaPublicaBuscaControle.php?transacao=CONSULTA&vara=2673';
$curl = curl_init($url);
#curl_setopt($curl, CURLOPT_POSTFIELDS, "foo");
#curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
#curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST");;
$html=#curl($curl);
if (!$html) {
echo "<br />cURL error number:" .curl_errno($curl);
echo "<br />cURL error:" . curl_error($curl);
exit;
}
else{
echo '<br>begin HTML[';
echo $html;
echo '<br>]end html ';
}
echo '</body></html>';
// Output
// cURL error number:0
// cURL error:

Javascript to make Post or Get call to a URL and return the text that's on the page

I've been trying to do this all morning I need to make either a POST or a GET call to this
URL http://cabbagetexter.com/send.php
I need it to return the text that's on the page, I know it can't be that difficult but I'm completely code blocked on this one,
I've tried using JQuerys .post and .get functions but I cant seem to return just the text on the page
any help would be appriciated
EDIT:
Ah ok so there's a technical reason for not being able to do it. balls, Thanks for the help guys

(function ($) {
$.ajax({
url: 'http://cabbagetexter.com/send.php',
type: 'text',
success: function (response) {
//do something with the text from the site
}
});
}(jQuery));
Obviously you need to host this script on the URL you are loading because of the same origin policy

You are running into the cross domain limitation. You can only to a request to a page in the same domain.

There is another possibility if you need to post calls to a page on another domain. Let's say your Javascript is being run from index.php. You might create a file called ctexter.php.
ctexter.php would use curl to make the post request to http://cabbagetexter.com/send.php, and would then output the response (the output from) send.php. So, if index.php makes an ajax call to ctexter.php, and ctexter.php is outputting the response from send.php, you have in effect achieved your goal.
You could make the curl post requests with this function:
function post_request($url, $data) {
$output = array();
foreach ($data as $key => $value) {
if(is_object($value) || is_array($value)){
$data[$key] = serialize($value);
}
}
$output = array();
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
$result = curl_exec($ch);
if ($result) {
$output['status'] = "ok";
$output['content'] = $result;
} else {
$output['status'] = "failure";
$output['error'] = curl_error($ch);
}
curl_close($ch);
return $output;
}
where $url is (obviously) the url to post to, and $data is an associative array containing the data you want to submit.
Then on ctexter.php you could do something like:
// Since we already built the post array in the
// ajax call, we'll just pass it right through
$response = post_request("http://cabbagetexter.com/send.php", $_POST);
if($response['status'] == "ok"){
echo $response['content'];
}
else{
echo "It didn't work";
}
Finally, hit ctexter.php using JQuery .post():
$.post("ctexter.php",
{
firstparamname: "firstparamvalue",
somethingelse: "llama"
},
function(data) {
alert("It worked! Data Loaded: " + data);
});

If you're on the same domain, you'd use some code like this:
var ajax = new XMLHttpRequest();
ajax.onreadystatechange=function()
{
if (ajax.readyState==4 && ajax.status==200)
{
document.getElementById("targetElementID").textContent = ajax.responseText;
}
}
ajax.open("GET","http://cabbagetexter.com/send.php",true);
ajax.send();
Learn how to use AJAX
If not, then, sorry, you're out of luck because you'll run into the same origin policy error.

There is a way to make a request to that URL and get around the same origin policy. Put something like a PHP script on your own domain that makes a request to http://cabbagetexter.com/send.php and then call your own script from the javascript.
If your host supports PHP and cURL a script like this would do the job:
<?php
$url="http://cabbagetexter.com/send.php";
$post="";
if(strstr($url,"?")){
$split=explode("?",$url);
$url=$split[0];
$post=$split[1];
}
$ch = curl_init ($url);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HEADER, 0);
if($post!=""){
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
}else{
curl_setopt($ch, CURLOPT_POST, 0);
}
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$result = curl_exec($ch);
curl_close($ch);
print $result;
?>