Some website I need to make a request to, using cURL in PHP, contains a captcha, which can be deactivated via setting a cookie "downloadcaptcha=1". Now how do I pass that cookie to the website on a cURL request? Before, you mark as duplicate, I've already made research and came across to using cookie jar files. I have never done something like that and couldn't find any newbie-friendly documentation.
The JS-equivalent code of setting the cookie, taken from the website's function that disables captchas:
var exdate=new Date();
var exdays = 1;
var c_name = "downloadcaptcha";
exdate.setDate(exdate.getDate() + exdays);
var c_value=escape(1) + ((exdays==null) ? "" : "; expires="+exdate.toUTCString());
document.cookie=c_name + "=" + c_value+";path=/";
window.location.reload();
I don't want to necessarily make it expire in 1 day.
Help is appreciated.
Here's the output. Not a single mention of the cookie
Request Header is missing the Cookie:
Host: www.emuparadise.me
Accept: */*
Accept-Encoding: deflate, gzip
Referer: https://m.emuparadise.me/
Change the $request to:
$cookie = '[downloadcaptcha=1'
$request = array();
$request[] = "Host: www.emuparadise.me";
$request[] = "Accept: */*";
$request[] = "Accept-Encoding: deflate, gzip";
$request[] = "Referer: https://m.emuparadise.me/";
$request[] = "Cookie: $cookie";
I have tried that already but the website still asks for a captcha.
Take a look at the JS code I posted. It's taken from the website's
function to disable captchas
I could help you much better if I had a URL to test with.
If there is a redirect you must use curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
You may have to make the HTTP Request Header look identical to the JS HTTP header.
You need to look at the cURL HTTP request and response and compare it to the JS HTTP request and response.
To see the JS HTTP request and response:
right click the page
Select Inspect (Chrome) or Insect Element (FireFox)
Select "Network" tab
Go to the page with the JS request or if already there Refresh.
Trouble shooting PHP code:
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, $request);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_TIMEOUT,10);
curl_setopt($ch, CURLOPT_FAILONERROR,true);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_HEADER, true);
$data = curl_exec($ch);
if (curl_errno($ch)){
$data .= 'Retreive Base Page Error: ' . curl_error($ch);
}
else {
$info = rawurldecode(var_export(curl_getinfo($ch),true));
$skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE));
$responseHeader= substr($data,0,$skip);
$response = substr($data,$skip);
var_export($info);
echo "----------------------------------\n$responseHeader\n--------------------------\n$response \n------------------------------------\n";
}
Original Post
curl_setopt($ch, CURLOPT_COOKIE, $cookie );
Single Cookie
$cookie = 'CookieName=CookieValue'
Multiple Cookies separated by semi-colon. There is no semi-colon after the last cookie.
$cookie = 'CookieName=CookieValue; CookieName=CookieValue'
curl_setopt($ch, CURLOPT_COOKIE, $cookie );
Cookies can also be set using CURLOPT_HTTPHEADER
See last line of the following example.
curl_setopt($ch, CURLOPT_HTTPHEADER, $request);
$request = array();
$request[] = "Host: www.example.com";
$request[] = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$request[] = "User-Agent: MOT-V9mm/00.62 UP.Browser/6.2.3.4.c.1.123 (GUI) MMP/2.0";
$request[] = "Accept-Language: en-US,en;q=0.5";
$request[] = "Connection: keep-alive";
$request[] = "Cache-Control: no-cache";
$request[] = "Pragma: no-cache";
$request[] = "Cookie: $cookie";
Related
We need to call an SAP web service from our website and submit an XML file to the SAP url. SAP API has credentials.
The purpose is to execute a function module in SAP which creates an SAP user ID and then returns the SAP user ID name back to our website via a return xml.
Below is the PHP code which i have tried.
<?php
$credentials = "Username:Password";
// Read the XML to send to the Web Service
$request_file = "m2bsubscr.XML";
$fh = fopen($request_file, 'r');
$xml_data = fread($fh, filesize($request_file));
fclose($fh);
$url = "SAP API HERE";
$headers = array(
"Content-type: text/xml;charset=\"utf-8\"",
"Accept: text/xml",
"Cache-Control: no-cache",
"Pragma: no-cache",
"SOAPAction: \"run\"",
"Content-length: ".strlen($xml_data),
"Authorization: Basic " . base64_encode($credentials)
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_USERAGENT, $defined_vars['HTTP_USER_AGENT']);
// Apply the XML to our curl call
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml_data);
$data = curl_exec($ch);
if (curl_errno($ch)) {
print "Error: " . curl_error($ch);
} else {
// Show me the result
echo "Success!<br />\n";
}
curl_close($ch);
// Handle the response from a successful request
$xmlobj = simplexml_load_string($data);
var_dump($xmlobj);
?>
My XML File:
<n0:_-majul_-m2bSubscriptionCreate xmlns:n0="urn:sap-com:document:sap:soap:functions:mc-style">
<IsSubscription>
<Bupar></Bupar>
<Sstat></Sstat>
<Categ></Categ>
<Fname>Wolfgang</Fname>
<Lname>Haerle</Lname>
<Email>whaerle#hotmail.com</Email>
<Srmrk></Srmrk>
<Cczip></Cczip>
<Cccty></Cccty>
<Ccadr></Ccadr>
<Cntry>US</Cntry>
<Product>1</Product>
<Begda>2020-07-08</Begda>
<Endda>2020-08-08</Endda>
<Price>765</Price>
<Curr>USD</Curr>
<Quantity>1</Quantity>
<Unit>MON</Unit>
<Userid></Userid>
</IsSubscription>
</n0:_-majul_-m2bSubscriptionCreate>
but it returns Success with bool(false) as a result. I tried to display error but it has no error.
Please help me to find what is the issue.
I wanted to do this in JavaScript. Kindly suggest me the way to do.
I'm using adal.js to generate an access token via Microsoft OAuth, however whenever I try to use the access token to call the https://graph.microsoft.com/v1.0/me endpoint (or graph.windows.net), I receive the following error: Authentication_MissingOrMalformed: Access Token missing or malformed.
Any ideas on how I can remedy this? Here's my config in JS:
<script src="https://secure.aadcdn.microsoftonline-p.com/lib/1.0.12/js/adal.min.js"></script>
<script>
var authContext = new AuthenticationContext({
instance: 'https://login.microsoft.com/',
tenant: 'xxxxxx-xxxxxxx-xxxxxx-xxxxxx', //COMMON OR YOUR TENANT ID
clientId: 'xxxxxx-xxxxxxx-xxxxxx-xxxxxx', //REPLACE WITH YOUR CLIENT ID
redirectUri: '/login.php', //REPLACE WITH YOUR REDIRECT URL
callback: getUser,
popUp: true,
cacheLocation: 'localStorage'
});
...
authContext.login();
// SET COOKIE
var newToken = authContext.getCachedToken('tenantid-xxxxxxx-xxxxxx-xxxxxx');
var now = new Date();
now.setTime(now.getTime() + 1 * 3600 * 1000);
document.cookie = "token="+newToken+"; expires=" + now.toUTCString() + "; path=/";
</script>
And here's how I'm attempting to pull/use the token in my PHP script:
<?php
// Get the token
$token = $_COOKIE['token'];
// Set headers
$headers = array(
"Authorization: Bearer " . $token,
'Content-Type: application/json'
);
// Make request to Graph API
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://graph.windows.net/mywebsite.org/me?api-version=1.6");
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
$response = json_decode($response);
curl_close($ch);
echo "<pre>";
var_dump($response);
echo "</pre>";
?>
All it does is return this error: Authentication_MissingOrMalformed: Access Token missing or malformed.
How can I fix this?? Is it a problem with not specifying the correct resource?
To call the Azure AD Graph REST successfully, we need to acquire the token for the Azure AD Graph.
To check whether the token is correct for the Azure AD Graph, you can print the token and parse it from here.
The aud claim in the token should be https://graph.windows.net. If it does't match, you need to acquire the token using the acquireToken instead of getting the token from cache. And the resource parameter should be https://graph.windows.net.
I have tried file_get_content and curl but both don't seem to work on the website. I have used both on previous projects.
Website: https://colruyt.collectandgo.be/cogo/nl/zoeken?z=5030
Anyone has a working solution. Been looking and testing for hours now :).
Curl also does not seem to work.
HTTP/1.1 200 OK Content-Length: 5395 Pragma: no-cache Cache-Control: no-cache Content-Type: text/html
Redirects to my own main domain name
I used this code:
<?php
function geturl($url){
(function_exists('curl_init')) ? '' : die('cURL Must be installed for geturl function to work. Ask your host to enable it or uncomment extension=php_curl.dll in php.ini');
$cookie = tempnam ("/tmp", "CURLCOOKIE");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (compatible; CrawlBot/1.0.0)');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT , 5);
curl_setopt($ch, CURLOPT_TIMEOUT, 5);
//curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_ENCODING, "");
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); # required for https urls
curl_setopt($ch, CURLOPT_MAXREDIRS, 15);
$html = curl_exec($ch);
$status = curl_getinfo($ch);
curl_close($ch);
if($status['http_code']!=200){
if($status['http_code'] == 301 || $status['http_code'] == 302) {
list($header) = explode("\r\n\r\n", $html, 2);
$matches = array();
preg_match("/(Location:|URI:)[^(\n)]*/", $header, $matches);
$url = trim(str_replace($matches[1],"",$matches[0]));
$url_parsed = parse_url($url);
return (isset($url_parsed))? geturl($url):'';
}
}
return $html;
}
echo geturl("https://colruyt.collectandgo.be/cogo/nl/zoeken?z=5030");
?>
Take a look at request
var request = require('request');
request('https://colruyt.collectandgo.be/cogo/nl/zoeken?z=5030', function (error, response, body) {
console.log(body)
})
This prints the body of the response.
I am trying to use free Google Translate API which is extracted from Firefox's S3 Google Translator addon, ie.
https://translate.google.com/translate_a/single?client=t&sl=auto&
tl=en&hl=en&dt=bd&dt=ex&dt=ld&dt=md&dt=qca&dt=rw&dt=rm&dt=ss&dt=t
&dt=at&ie=UTF-8&oe=UTF-8&otf=2&srcrom=1&ssel=0&tsel=0&q=Hello
in PHP cURL ie.
$isPOST=isset($_POST) && !empty($_POST);
$q=$isPOST ? $_POST['q'] : $_GET['q'];
$url='https://translate.google.com/translate_a/single';
$data='client=t&sl=auto&tl=en&hl=en&dt=bd&dt=ex&dt=ld&dt=md&dt=qca&dt=rw&dt=rm&dt=ss&dt=t&dt=at&ie=UTF-8&oe=UTF-8&otf=2&srcrom=1&ssel=0&tsel=0&q='.$q;
$ch=curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_NOBODY, 0);
curl_setopt($ch, CURLOPT_URL, !$isPOST ? $url.'?'.$data : $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
if($isPOST){
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
}
$return=curl_exec($ch);
curl_close($ch);
I am calling this page using ajax..
$.ajax({
type: text.length>750 ? 'post' : 'get',
url: 'translate.php',
data: 'q='+text,
success: function(d){ alert(d); }
});
but doing this all, I get this response from Google Translate, ie.
Error: 400. That’s an error.
Your client has issued a malformed or illegal request. That’s all we know.
Please, help me solve this error and get the translated text..
I checked your URL in your browser it shows 400 Error. it means illegal request.
try http://www.sitepoint.com/using-google-translate-api-php/ this URL.
<?php
$apiKey = '<paste your API key here>';
$text = 'Hello world!';
$url = 'https://www.googleapis.com/language/translate/v2?key=' . $apiKey . '&q=' . rawurlencode($text) . '&source=en&target=fr';
$handle = curl_init($url);
curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($handle);
$responseDecoded = json_decode($response, true);
curl_close($handle);
echo 'Source: ' . $text . '<br>';
echo 'Translation: ' . $responseDecoded['data']['translations'][0]['translatedText'];
?>
Sorry, I tried to POST with same code and it worked..
Thank all.
I have the same issue in a Visual Basic 6 Project and Thamaraiselvam's comment guided me to the correct direction..
I was building the URL correctly and if I try it in the browser it works, but in the http component of vb6 it wasn't working. the solution was to simply url_encode the data sent. (putting it in the browser was automatically doing it)
hope this helps someone else.
I am trying to monitor cricket scores on scorespro/cricket by making browser AJAX requests. Analysing the network traffic in Google Chrome, I can see my browser making requests of the form:
http://www.scorespro.com/cricket/ajax.php?g_sort=league&date=2014-10-02&mut=1412265716&sut=0&(some_random_number)
When I click on the response IN Google Chrome, I can see the data that has been received. However when I try to request the request URL myself, no data is received. Why is that happening (is it to do with the random string) and how can I get around it?
Is doing this from javascript a requirement? Have you considered abstracting the requests by calling a script on a server you control?
For example on your server you could have a PHP script called, for example, "grabber.php"
<?php
$r = '0.' . rand(1000000000000000, 9000000000000000);
$url = 'http://www.scorespro.com/cricket/ajax.php?g_sort=league&date=2014-10-03&mut=1412328280&sut=0&' . $r;
$useragent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:32.0) Gecko/20100101 Firefox/32.0';
$referer = 'http://www.scorespro.com/cricket/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_REFERER, $referer);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, '/tmp/cookie.txt');
$response = curl_exec($ch);
curl_close($ch);
$data = array('payload' => $response);
echo json_encode($data);
exit();
?>
You could then call that page via a simple ajax request :
<script src="//code.jquery.com/jquery-1.11.0.min.js"></script>
<script type="text/javascript">
$.ajax({
url: 'http://yourserver.com/grabber.php',
dataType: 'json',
type: 'GET',
success: function(data, textStatus, jqXHR){
if (data['payload']){
alert(data['payload']);
} else {
alert ('oops');
}
}
});
Of course if you went with this approach you'd have to decide how to get the URL's you need to request from the cricket site to the grabber script (i.e. pass them from javascript or get them directly from within the PHP script depending on your requirements)