So im trying to login to anghami.com using post request. I was able to login if I pass all of the values needed.
My issue is im not sure were to find the 2 missing values "re_token" ( I guess its recapthca from google) and "fingerprint"
check my code for working test:
headers = {
'authority': '',
'accept': 'application/json, text/plain, */*',
'accept-language': 'en-US,en;q=0.9',
'content-type': 'application/x-www-form-urlencoded; charset=UTF-8',
'origin': 'https://play.anghami.com',
'referer': 'https://play.anghami.com/login',
'sec-ch-ua': '".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"Windows"',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-site',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
}
link = 'https://api.anghami.com/gateway.php'
s = requests.Session()
proxies = {
"http": "http://",
}
data = {
'm': 'an',
'u': 'email',
'p': 'password',
'devicename': 'Chrome 104',
're_token':'03ANYolqvEDNfBhyM-qJ77j92_vkw8yt1VtKKc9e6jZFl9mG4sysFOvVZ0LlQWsMecFRWRCMFGG8KAgdWw1S0kUPB-1yW5kfJ8B2XGLnlaW7XAReGvyYpB2WgZeGXPdxlTi0PINbN2Ga9wI2ecF9jltpf7gcUj9MLucb9KDaUYENySmFq2ts5qh9g_2nr6AXx_igsD53xvWPGrGi_n7evy224P7A0NitmjcXKlAKL_rlkkXqbwOd4qbzF_IkTKX6iSNLfb2FFso8S75OKa0dlbfLO_7eY2zU7VzVKa23XWet3RXDED7q8Rx8RKFaO9n_lvbG-PORGCpmajnbWtWoIhEZpY06mt41vx4AoW0JnCtV9Z3v5AsAoM_SIZNawTLVBKyI3iVk9AbsGskh5DZ0DzIQ2Hp_2325fuyhjp2gjW_yUud7DuGVZ9Zn7WjteVnE0Yv4ZQoWx5Z2Hz-s7Qy7G2Acm6WLbuIvS_5JsJsfLYh_hiB_DY79UyKHNpeQtulqS1wMGwHqDFbmfv',
'ngsw-bypass': 'true',
'type': 'authenticate',
'language': 'en',
'lang': 'en',
'web2': 'true',
'fingerprint': 'eyJmcCI6ImE2MzcxYTRjLTU1ODEtNDE2My1iMWRkLTA3NjBkMmI5OWZlYSIsImgiOiI0OTIxZDM1OCJ9',
'angh_type': 'authenticate',
}
auth = HTTPProxyAuth(proxyacc)
s.proxies = proxies
s.auth = auth
ext_ip = s.post(link, data=data, headers=headers)
print (ext_ip.text)
print (ext_ip.url)
How can I create 'fingerprint':
how to get 're_token':
So,
re_token -> recaptcha token(answer)
fingerprint -> base64 encoded string with fingerprint uid and
unknown 'h' param.
For more accurate information needed debug vendor.67cc4b67b66a6114.js this code.
As for the recaptcha, you need to understand what type of captcha is used on the site and, accordingly, find the key and additional information if needed.
To solve it, you can use any ready-made service like anti-captcha.
By the way, it's a bad idea to specify the authority header, since this header is specified automatically and works only with http2.0, otherwise, it betrays you even more.
Related
I'm trying to use UrlfetchApp to send a request to a page. Request is good. But returning
<p>Your browser is currently set to block cookies. Please enable cookies in your browser preferences and try again.</p>
in the html body.
Here is the code:
const res1 = UrlFetchApp.fetch('https://url.com', {
method: 'POST',
headers: {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'Accept-Language': 'en-US,en;q=0.9',
'Cache-Control': 'max-age=0',
'Connection': 'keep-alive',
'Content-Type': 'application/x-www-form-urlencoded',
'Cookie': 'JSESSIONID=Dfaefaefaesfsaefgr',
'Origin': 'https://url.com',
'Referer': 'https://url.com',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'same-origin',
'Sec-Fetch-User': '?1',
'Upgrade-Insecure-Requests': '1',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36',
'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="100", "Google Chrome";v="100"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"Windows"'
},
payload: 'payloaddata'});
I don't know how to turn on the cookie when using UrlFetch function. Does anyone has the same issue?
Thanks for any suggestions.
What I have tried:
Seperate Cookie property from headers into a json object.
cookies = {
'JSESSIONID': '778C494754356A23F080849C10F2A851',
'TS01ee6e39': '018c1954d58470c0adb8d6d0df850dba9363aa54c16fc4653b2178278f62a5f0dde96d5780b3338857c8ea3ff92c6e99b9bd3a5867cef327bd847ac05ab9242a7414fa0832',
'X-HR-ClientSessionId': '10_107.162.4.39_1651626597548',
'locale': 'en',
'TS0189a565': '018c1954d5ae31eb4b4d18a85d43414cdcd9158bcc6fc4653b2178278f62a5f0dde96d578016166d716abd84dc80b66117923674fd68b4dd222e33d15361b818320e9dbd03333b4682ebc552fb370cd462eb3e5d2b5cc5273e789e87f2bd772fa800fa9e77744d459169cf3d8594422a3d7ae7968ba33103e373fbcf2c83f38da92d9643e5f9ef6a925938338bef881e38e827bf0a5635126e7297731cc06d71fd1a883702',}
But this does not work, request immediately return session expired. Please try again.
Edit on 05/07/2022:
I'm trying to get the JSESSIONID from fetch request, I can find this object on the browser, But could not find it on the urlfetch return response.
Endpoint: https://quizlet.com/webapi/3.2/images/search?query=hello&perPage=2
You guys can try to access this page as Incognito, from my side It works. So I think I can fetch data from that site.
I try to copy the request and run in Javascirpt, Python. However, It doesn't work. I got 403 error.
I also try to use Burp Suite. I can't access this site through Burp's browser.
Moreover, As I tried using incognito so I don't think it is relevant to cookies.
Code sample (JS):
import fetch from "node-fetch";
const response = await fetch(
"https://quizlet.com/webapi/3.2/images/search?query=hello&perPage=2",
{
headers: {
accept:
"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-language": "en",
"cache-control": "no-cache",
pragma: "no-cache",
"sec-ch-ua":
'"Google Chrome";v="93", " Not;A Brand";v="99", "Chromium";v="93"',
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": '"Linux"',
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "none",
"sec-fetch-user": "?1",
"upgrade-insecure-requests": "1",
},
referrerPolicy: "strict-origin-when-cross-origin",
body: null,
method: "GET",
mode: "cors",
credentials: "include",
}
);
const data = await response.status;
console.log(data);
Code Python
import requests
headers = {
'authority': 'quizlet.com',
'pragma': 'no-cache',
'cache-control': 'no-cache',
'sec-ch-ua': '"Google Chrome";v="93", " Not;A Brand";v="99", "Chromium";v="93"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"Linux"',
'upgrade-insecure-requests': '1',
'user-agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.82 Safari/537.36',
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'sec-fetch-site': 'none',
'sec-fetch-mode': 'navigate',
'sec-fetch-user': '?1',
'sec-fetch-dest': 'document',
'accept-language': 'en',
'cookie': 'qi5=i2x3g7y1z9a6%3At3vMoQQig2yLcpN.HKWn; qtkn=7gT4DE7pN9URJ2AFDYeaVe; fs=qzkse0; app_session_id=9781a407-4f37-4c09-8e97-8156f182bb45; search_session=%7B%22search_session_id%22%3A%22-2379864199063990974614477b859794%22%2C%22query%22%3A%22overrated%22%2C%22version%22%3A%221.1.1%22%2C%22platform%22%3A%22WEB%22%2C%22depth%22%3Anull%2C%22target_object_type%22%3A%22QImage%22%7D; __cf_bm=cB7hRf6JbcOFZ2kvQ3W12V4bxXiIgn_kF3n87RcI0h0-1631877048-0-Ac+Hi0pATLgW5N3JjqYa7uc5W4ZfDLOumvmCQixWJIKdcVj7stciFh8cYFVTOpr+q5pM2Q7LrXC/LsffOB6Mh2E=; __cfruid=81f16a673e6117331dd4270b3f4f29111590d7d8-1631877048',
}
params = (
('query', 'hello'),
('perPage', '2'),
)
response = requests.get(
'https://quizlet.com/webapi/3.2/images/search', headers=headers, params=params)
# NB. Original query string below. It seems impossible to parse and
# reproduce query strings 100% accurately so the one below is given
# in case the reproduced version is not "correct".
# response = requests.get('https://quizlet.com/webapi/3.2/images/search?query=hello&perPage=2', headers=headers)
print(response.status_code)
Please help me out. I don't even know how can be that? (browser works, while code doesn't). Thank anyway.
From the python side. I had a look out of interest, as I'm currently developing a REST API and was curious how they where securing it.
Using Wireshark it appears that the "requests" module in python does not handle http requests in the same manor as Chrome/Firefox, which I suspect they are using as a tell to give a captcha.
Anyway switching requests for the httpx module;
pip install httpx
https://www.python-httpx.org/
And changing the headers to replicate Firefox in full;
import httpx
headers = [
('Accept','text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'),
('Accept-Encoding','gzip, deflate, br'),
('Accept-Language','en-GB,en;q=0.5'),
('Cache-Control','max-age=0'),
('Connection','keep-alive'),
('Host','quizlet.com'),
('Sec-Fetch-Dest','document'),
('Sec-Fetch-Mode','navigate'),
('Sec-Fetch-Site','none'),
('Sec-Fetch-User','?1'),
('TE','trailers'),
('Upgrade-Insecure-Requests','1'),
('User-Agent','Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0'),
]
params = (
('query', 'hello'),
('perPage', '2'),
)
response = httpx.get('https://quizlet.com/webapi/3.2/images/search', headers=headers, params=params,)
print(response.content)
Gives the following as appose to the captcha page for me;
{
"responses": [{
"models": {
"image": [{
"id": 18957872,
"personId": 16641862,
"timestamp": 1416579222,
"lastModified": 1416579222,
"code": "Gfg5XS88MRmYq8RS",
"license": 1,
"width": 480,
"height": 360,
"flickrId": null,
"flickrOwner": null,
"_legacyUrl": "http://o.quizlet.com/cZDE.6rHW7IrGptXSGm8FA.gif",
"_legacyUrlSquare": "http://o.quizlet.com/cZDE.6rHW7IrGptXSGm8FA_s.gif",
"_legacyUrlSmall": "http://o.quizlet.com/cZDE.6rHW7IrGptXSGm8FA_m.gif",
"_secureLegacyUrl": "https://o.quizlet.com/cZDE.6rHW7IrGptXSGm8FA.gif",
"_secureLegacyUrlLarge": "https://o.quizlet.com/cZDE.6rHW7IrGptXSGm8FA_b.gif",
"_secureLegacyUrlSquare": "https://o.quizlet.com/cZDE.6rHW7IrGptXSGm8FA_s.gif",
"_secureLegacyUrlSmall": "https://o.quizlet.com/cZDE.6rHW7IrGptXSGm8FA_m.gif"
}, {
"id": 9228314,
"personId": 513525,
"timestamp": 1406222781,
"lastModified": 1406222781,
"code": "bPHbzaV7KsGWfuXJ",
"license": 1,
"width": 298,
"height": 232,
"flickrId": null,
"flickrOwner": null,
"_legacyUrl": "http://o.quizlet.com/ptqCa7LsKjiVSBVPI3OfTA.jpg",
"_legacyUrlSquare": "http://o.quizlet.com/ptqCa7LsKjiVSBVPI3OfTA_s.jpg",
"_legacyUrlSmall": "http://o.quizlet.com/ptqCa7LsKjiVSBVPI3OfTA_m.jpg",
"_secureLegacyUrl": "https://o.quizlet.com/ptqCa7LsKjiVSBVPI3OfTA.jpg",
"_secureLegacyUrlLarge": "https://o.quizlet.com/ptqCa7LsKjiVSBVPI3OfTA_b.jpg",
"_secureLegacyUrlSquare": "https://o.quizlet.com/ptqCa7LsKjiVSBVPI3OfTA_s.jpg",
"_secureLegacyUrlSmall": "https://o.quizlet.com/ptqCa7LsKjiVSBVPI3OfTA_m.jpg"
}]
},
"paging": {
"total": 50,
"page": 1,
"perPage": 2,
"token": "UuKKKAkmxv.r4YtwFDuRevZVGAHr"
}
}]
}
I wrote something in Python and am trying to figure out why the hell the seemingly equivalent code in JS isn't working.
Working Python -
Headers used:
self.session = requests.Session()
#Set headers
self.headers = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'Connection': 'keep-alive',
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'en-US,en;q=0.8',
'Upgrade-Insecure-Requests': '1',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36',
}
Code:
link = 'https://www.kith.com/cart'
data = [
('updates'+'['+'888074764295'+']', '1'),
('updates'+'['+'888463982599'+']', '0'),
]
click = self.session.post(link, headers= self.headers, data=data, verify = False)
Not working JS -
const secondaryVar = `updates[888463982599]`;
const desiredVariant = `updates[888074764295]`;
const checkoutForm = {};
checkoutForm[desiredVariant] = '1';
checkoutForm[secondaryVar] = '0';
//Post request to cart to update it with desired product
request({
url: 'https://www.kith.com/cart',
followAllRedirects: true,
method: 'post',
formData: checkoutForm,
headers : {
'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'Accept-Encoding':'gzip, deflate, br',
'Accept-Language':'en-US,en;q=0.9',
'Cache-Control':'max-age=0',
'Connection':'keep-alive',
'Upgrade-Insecure-Requests':'1',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36',
},
},
function(err, res, body) {
I've narrowed it down to this bit of code, but as far as I can tell there is no significant difference between the code in Python and the JS code. My guess is it has something to do with the session or headers...but again I don't know.
Thanks for any responses
I think the Python might not respect cors, which would explain the difference. I don't know what JavaScript framework you are using, but using jQuery, the following works when executing this code from the kith.com website.
To avoid any issues with CORS, I removed the headers that are automatically set by the browser, and I change the url from www.kith.com to kith.com.
jQuery.ajax("https://kith.com/cart", settings={method:"post", headers : {
'Accept':'application/json',
'Accept-Language':'en-US,en;q=0.9',
'Cache-Control':'max-age=0',
'Upgrade-Insecure-Requests':'1',
}, data:{"desiredVariant":1,"secondaryVar":0}}).error(function(err){console.log("error"+ err)}).success(function(res){console.log(res)})
I've been trying to make a simple script work for the past couple of days and have not been successful so far.
Here is the problem :
The authentification on the website (like on many) consists of a first 'Post' request, wich redirects you with a 302 response, and then a 'Get' request, to go on the home page logged in.
So, i'm trying to log in with a POST request, get the cookie included in the response, then add this cookie to my GET request.
Here is my code :
var headers = {
'Upgrade-Insecure-Request' : '1',
'User-Agent' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36',
'Content-Type' : 'application/x-www-form-urlencoded',
'Accept' : 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'Accept-Encoding' : 'gzip, deflate, br',
'Connection' : 'keep-alive',
'Accept-Language' : 'fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4,it;q=0.2,es;q=0.2'
};
var payload = {
'ptl' : 'edt',
'codensa' : 'pblv',
'taiga_user' : 'user',
'taiga_mdp' : 'mdp',
'submit' : 'connexion'
};
var options = {
'method' : 'POST',
'headers': headers,
'payload' : payload,
'followRedirects' : false
};
var login = UrlFetchApp.fetch('https://etudiant.archi.fr/taiga/etd/pages/login.php', options);
var login_cookie = login.getAllHeaders()['Set-Cookie'].split(';')[0];
return login_cookie;
And then, my GET request :
var headers2 = {
'Upgrade-Insecure-Request' : '1',
'User-Agent' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36',
'Accept' : 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'Accept-Encoding' : 'gzip, deflate, br',
'Accept-Language' : 'fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4,it;q=0.2,es;q=0.2',
'Connection' : 'keep-alive',
'Cookie' : login_cookie,
};
var options2 = {
'method': 'GET',
'headers': headers2,
'followRedirects' : false
};
var index = UrlFetchApp.fetch('https://etudiant.archi.fr/taiga/etd/pages/index.php?im', options2);
And this doesnt work...
However, i suppose my script is kind of correct, since in my headers2, if i remplace 'login_cookie' with an actual cookie that i get manually with my brower, it works !!
Can anyone help me ? I've tried everything... haha
i'm trying to make a post request using this sync-request library. And they don't support body options, so i need set manually in the headers, but i don't know exactly how.
Currently i'm trying like this:
let req = request('POST', LINK, {
'headers': {
'Accept' : 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'Accept-Encoding' : 'gzip, deflate',
'Accept-Language' : 'en-US,en;q=0.8,pt;q=0.6',
'body' : 'ESTCS=1; ESTID=00726861000151',
'Cache-Control' : 'max-age=0',
'Connection' : 'keep-alive',
'Content-Length' : '167',
'Content-Type' : 'application/x-www-form-urlencoded',
'Referer' :
'Upgrade-Insecure-Requests' : '1',
'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2584.0 Safari/537.36',
'X-FirePHP-Version' : '0.0.6'
}
});
But nothing happens.
I need send two things in my body:
let body = {
'ESTCS': 1,
'ESTID': '00726861000151'
};
i'm trying like this:
'body': 'ESTCS=1; ESTID=00726861000151',
Thanks.
Body should be & separated. Try it this way:
'body': 'ESTCS=1&ESTID=00726861000151',