Develop Reference

Outlook login script in python3: Javascript not enabled - cannot login

I am currently trying to log into an outlook account using requests only in Python. I did the same thing with selenium before, but because of better performance and the ability to use proxies with it more easily I would like to use requests now. The problem is that whenever I send a post request to the outlook post URL it returns a page saying that it will not function without javascript.
I read on here that I would have to do the requests that javascript would do with requests, so I used the network analysis tool in Firefox and made requests to all the URLs that the browser made requests to. It still returns the same page, saying that js is not enabled and it will not function without js.
s = requests.Session()
s.headers.update ({
"Accept": "application/json",
"Accept-Language": "en-US",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Host": "www.login.live.com",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36",
})
url = "https://login.live.com/"
r = s.get(url, allow_redirects=True)
#Simulate JS
sim0 = s.get("https://logincdn.msauth.net/16.000.28299.9
#...requests 1-8 here
sim9 = s.get("https://logincdn.msauth.net/16.000.28299.9/images/Backgrounds/0.jpg?x=a5dbd4393ff6a725c7e62b61df7e72f0", allow_redirects=True)
post_url = "https://login.live.com/ppsecure/post.srf?contextid=EFE1326315AF30F4&bk=1567009199&uaid=880f460b700f4da9b692953f54786e1c&pid=0"
payload = {"username": username}
sleep(4)
s.cookies.update({"logonLatency": "LGN01=637025992217600244"})
s.cookies.update({"MSCC": "1567001943"})
s.cookies.update({"CkTst": "G1567004031095"})
print(s.cookies.get_dict())
print ("---------------------------")
print(json.dumps(payload))
print ("---------------------------")
rp = s.post(post_url, data=json.dumps(payload), allow_redirects=True)
rg = s.get("https://logincdn.msauth.net/16.000.28299.9/images/arrow_left.svg?x=a9cc2824ef3517b6c4160dcf8ff7d410")
print (rp.text)
print (rp.status_code)
If anyone could hint me in the right direction on how to fix this I would highly appreciate it! Thanks in advance

Error x-amzn-errortype: InvalidSignatureException on DELETE request to API gateway

I am attempting to build a javascript application.
I am unable to get a DELETE request authenticated via IAM. 
The API gateway delete route has been configured to use AWS_IAM authentication. Cognito user group has a role attached which grants access to invoke the delete route.
I've signed the request using AWS.Signers.V4
var httpRequest = new AWS.HttpRequest("https://bo5o2odxxx.execute-api.ap-south-1.amazonaws.com/wrhprod/"+uri, "ap-south-1");
httpRequest.headers.host = "https://bo5o2odxxx.execute-api.ap-south-1.amazonaws.com";
httpRequest.headers['Content-Type'] = "application/json";
httpRequest.method = method; 
var v4signer = new AWS.Signers.V4(httpRequest, "execute-api", true);
v4signer.addAuthorization(awsCredentials, AWS.util.date.getDate());
delete httpRequest.headers['host']
delete httpRequest.headers['X-Amz-User-Agent']
I am using axios to send the delete request:
axios.delete('/events/2017/1523502329582',
{ headers: httpRequest.headers})
.then(response => {
//success
})
.catch(error => {
//fail
});
This is my request Header:
:authority: bo5o2odxxx.execute-api.ap-south-1.amazonaws.com
:method: DELETE
:path: /wrhprod/events/2017/1523502329582
:scheme: https
accept: application/json, text/plain, /
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
authorization: AWS4-HMAC-SHA256 Credential=ASIAIGIXC2II7KWVVUVA/20180413/ap-south-1/execute-api/aws4_request, SignedHeaders=host;x-amz-date;x-amz-security-token;x-amz-user-agent, Signature=f1b12cf3c1a3693f259e2f0079231974d6de0e4225bca0af3754a32eecf95277
origin: http://localhost:3000
referer: http://localhost:3000/events
user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36
x-amz-date: 20180413T071741Z
x-amz-security-token: AgoGb3JpZ2luEAwaCmFwLXNvdXRoLTEigAIBwvwgU/vlYPZI1+XYbJUieH7lQB1XI32w08GaX0MjobxyfhFK7J3md5GaKlPdO3I+fqjYpBWOlrqNNllgjKTLZwNOx7SxrVq5T57qk27nq6kp1Y5nSDTy+wj6pHVbSmzRgrfqvOgsgWrU6NsB41taHLaN3sjQaneSC7/phQ3+UvdjQrEI7egI7TikiUs/DW6oQWMupW/75c9GEVnoA2RRdYmVqVh//OmbWdG6DMk09ritP6eLpUYvWHGrMNniGnT2Rj+3kbOgSst0JrKLwssOsO3lIm+iDxSk3NGoKRUX4iB/Lnv7XQnUaFoK2PC0DNyj9XnJVn0RXoXCPOOwPf9wKqwFCEAQABoMMTQ0MTUzMjMxNzA0Igwr7GTDa3MwS/6L7RIqiQU4WK45J//oxnc98LgRZOApKGYPyRfiDgd+LHG006GHDJTodDTysv1YIkLbOq6Da1d6ENpwdkW7h8OgtR4r+BoOPeP7h+vV55LJHexPGAGC8GKimK8iKFLjvyy1ArGWGb9FEbYtsJkjjxFZw2uZJ3ex3YWQk5UiBDkuJd8paU8rqSctcF/0SLFZyK77AndiLs7Ir3AwBvlQvOh+J3Q8MCfAuRLGOAT1VTYOMis/ZL7cczi1YRWK+7jlRdeoeQu/aP9Y07SbxRxzIh7YyhFBilLXUslRiyeKiehYK6ufI/czC2bDipegBmCAFiEff/PKvBinSUJRTkSJODmh6E25OVerbbSBgI1aCGj+b3bRiBiMwu+kIF77czK18jBfE5OeexKfh+r1w2puyYDwPgQMz9ki5BdQtHvzm0/p8W88+tPGNB0EO7E4TyWtfJ0HyLU6wgVpf4LB9YPDeUxoR8JdB+QLCXNDPdFEuWPQRgm4MgngAyHwVn+NnFsBk1U/GNHabWeX0k+PsxNn5LJ6u4RQCDP0YDsGrQmIO6DIrUWidBZ00XMeaJXWrsgHb/6qINijvg3sk1y9P7kxEZyfarotDEjqLRjpyUBufRfiCVccQ5bF0MeOxxMN4ZFC7mX7xtaSQKcmv2sG1uJus9d7yyd3phQ4FluiI77UE5CVx3bkdRlEObRXB/DyCv+gfk3VfouIchF8xNpMU1jmaHN5+rB0/lbTMxVmEX8qYc6UdZXIIWRm2spB2YwnacBhC0Q36CnCwU1hcRo7YhcminiCPQy6+7OrOTLLXKAenRjNIuPYGAaZQl1MgPMBwFpPGAdbrJG/iWleNf5fN+dettOD18VT8jrmdQ0jRM5N9OjEMJqvwdYF
This is the AWS response header:
content-length: 192
content-type: application/json
date: Fri, 13 Apr 2018 07:17:41 GMT
status: 403
x-amz-apigw-id: FRLvXEDeBcwFuLQ=
x-amzn-errortype: InvalidSignatureException
x-amzn-requestid: c0f040c3-3eea-11e8-bd7a-dd445896cd03
I've spent hours on this & have not found a solution, please let me know if I can supply any other information to help troubleshoot.
This same API was working using cognito as an API gateway authorizer, I switches to IAM so that I may have more fine grained control on the cognito users.

I was not able to get AWS.Signers.V4 to work. From what I read, this API is not public yet.
I was able to get my APP working using a custom client to sign the request. It is available for download Here

OPTIONS (failed) only on Chrome and Firefox

I make a POST request and the request just sits, pending until it eventually fails. I've monitored the nginx logs and the node server logs and the request doesn't even register. This works for anyone else that I've had test it except one other colleague. If I use the edge browser or a different computer it works fine.
I have attempted to make POST requests to other (custom) servers and it hangs on options there as well. I have also made the POST request with jQuery and it fails the same way.
It's maybe worth noting that I am using the withCredentials flag.
Headers:
Provisional headers are shown
Access-Control-Request-Headers:content-type
Access-Control-Request-Method:GET
Origin:http://localhost:8080
Referer:http://localhost:8080/<path>
User-Agent:Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36
The request:
public login(user) {
const endpoint = `http://<url>`;
let headers = new Headers();
headers.append('Content-type', 'application/json');
return this.http
.post(endpoint, JSON.stringify(user), {
headers: headers,
});
}
I subscribe to the call in my component:
this._accountService.login(this.user)
.subscribe(res => {
console.log("logged in!");
if (res.json().status === "success") {
window.location.href = `/home/${this.org}/${this.product}`;
}
else {
// What other options are there?
console.log("Do something else maybe?");
}
},
err => {
this.invalidLogin = true;
console.log("Ye shall not pass!");
});
Successful user's headers
Accept:*/*
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8
Access-Control-Request-Headers:content-type
Access-Control-Request-Method:POST
Connection:keep-alive
Host:<url>
Origin:<url>
Referer:<url>
User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.33 Safari/537.36
From chrome://net-internals/#events
t=61869793 [st= 0] +REQUEST_ALIVE [dt=60162]
--> has_upload = false
--> is_pending = true
--> load_flags = 34624 (DO_NOT_SAVE_COOKIES | DO_NOT_SEND_AUTH_DATA | DO_NOT_SEND_COOKIES | MAYBE_USER_GESTURE | VERIFY_EV_CERT)
--> load_state = 14 (WAITING_FOR_RESPONSE)
--> method = "OPTIONS"
--> net_error = -1 (ERR_IO_PENDING)
--> status = "IO_PENDING"
--> url = "<url>"
t=61929955 [st=60162] -HTTP_STREAM_PARSER_READ_HEADERS
--> net_error = -324 (ERR_EMPTY_RESPONSE)
t=61929955 [st=60162] -HTTP_TRANSACTION_READ_HEADERS
--> net_error = -324 (ERR_EMPTY_RESPONSE)
t=61929955 [st=60162] -URL_REQUEST_START_JOB
--> net_error = -324 (ERR_EMPTY_RESPONSE)
t=61929955 [st=60162] URL_REQUEST_DELEGATE [dt=0]
t=61929955 [st=60162] -REQUEST_ALIVE
--> net_error = -324 (ERR_EMPTY_RESPONSE)
I'm really guessing this is related to something that is cached in my browser(s) but I really cannot find what. I've cleared all cookies and anything that could be stored. Where else can I check to clear things? This is clearly something local to my computer/browser (and one other unfortunate person).

Please try to subscribe() to the observable.
return this.http
.post(endpoint, JSON.stringify(user), {
headers: headers,
}).subscribe(() => console.log("POST done!"));

Have you tried setting the 'Cache-Control' in your headers? I think in jQuery you can simply set
$.ajax({
cache: false
});
or adding a header with a regular ajax request
request.setRequestHeader("Cache-Control", "no-cache");

Why don't you just prevent getting into OPTIONS request loop . It really drives you crazy at times . Other browsers do not trigger OPTIONS request but chrome and firefox does to ensure CORS . I have successfully used this library named as xdomain from github , and it really works !! Their github introduction page introduce xdomain as a CORS alternative . And most importantly i used it in JQuery , but it also does support Angular's http service . Have a look at it . It may help you for good :) . Here's the link to library Xdomain CORS Alternative

There are issues with CORS and using localhost as the domain (which you have listed in the ORIGIN headers). Typically CORS / OPTIONS requests don't work properly when localhost is involved for certain security reasons, but hanging isn't normally what happens so this might not be the correct answer but its worth a shot!
Try adding a new host to your local machine and removing localhost from the equation. Just throwing this idea out there and hope that it might help you out!
As per comment below
Your server appears to allow the connection, but it does not appear to send a response. Are you able to post the headers from a successful OPTIONS request to prove that the server is actually able to handle these requests.

What is wrong with this HTTP-Get Basic authentication code?

I am using node.js restify as backend to run a REST API server and angularjs as front-end to call the HTTP GET. The REST server uses HTTP Basic Authentication. The username is foo and password is bar.
I have tested that the backend code works by using a restify client. Here is the working client code;
var client = restify.createJsonClient({
url: 'http://127.0.0.1'
});
client.basicAuth('foo', 'bar');
client.get('/alert?list=alertList', function(err, req, res, obj) {
console.log(obj);
});
I have trouble getting my angularjs http-get code to work. Here is the relevant code;
.controller('ViewCtrl', ['$scope', '$http', '$base64',
function ($scope, $http, $cookies, $base64) {
var url = '127.0.0.1/alert?list=alertList';
var auth = $base64.encode('foo:bar');
$http.defaults.headers.common['Authorization'] = 'Basic ' + auth;
$http.get(url).then(function (response) {
tableData = response.data;
//handle data
});
}
I cannot figure out what is wrong with the angularjs code. I am using restify authorizationParser. Are there any extra header requirements to get HTTP basic authentication working with restify authorizationParser?
The error message on the browser looks like this;
{
code: "NotAuthorized",
message: ""
}
In the chrome debugger, this is what I see;
Request Method:GET
Status Code:403 Forbidden
Remote Address:127.0.0.1:80
Request Headers
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4,zh-TW;q=0.2,ja;q=0.2
Cache-Control:max-age=0
Connection:keep-alive
Host:127.0.0.1
If-Modified-Since:Wed, 23 Dec 2015 02:22:04 GMT
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
I am using this base64 angular module.
https://github.com/ninjatronic/angular-base64
EDIT: I discovered there is nothing with the angular code. The problem lies with the restify server. The restify server supports static web server and when http basic authentication was enabled, this static web server stopped working.

Inside the controller, you can pass the authentication header like this:
var url = '127.0.0.1/alert?list=alertList';
var auth = $base64.encode('foo:bar');
var headers = {"Authorization": "Basic " + auth};
$http.get(url, {headers: headers}).then(function (response)
tableData = response.data;
//handle data
});

Node.js Login to blog site and Create new post

I want to login to blogfa.com (a persian blog service) and create a new post by node.js
To do that i use request.js to post login site and go this url "/Desktop/Post.aspx?action=newpost" and post a new content
here is code i got so far :
var request = require('request');
var cheerio = require('cheerio');
var j = request.jar();
request = request.defaults({ jar : j }); //it will make the session default for every request
//...
var headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36 OPR/28.0.1750.48',
'Content-Type': 'application/x-www-form-urlencoded'
};
request({
url:"http://blogfa.com/Desktop/login.aspx",
method:"POST",
form:{uid:"demoblog1",pwd:"test"},
headers: headers,
followRedirect:true
},
function(error,response,body){
console.log(body);
});
Problem i have its, i cant login in to the site .
i dont why not working ! i used jar for cookie , posted user and password , also set headers.
here is runnable code you can test :
http://code.runnable.com/Vc7EnmyVlgRa1Hx-/blog-login-for-node-js
Update
Request Cookies
.ASPXBLOG BC045A0CD184FEA10D91561EB67A302F1E036D88E50CE4264E4ABD003
__utma 36873331.1996897518.1435135939.1437159460.1439563539.7
__utmz 36873331.1437159460.6.6.utmcsr=chat.delgarm.com|utmccn=(referral)
pubset ar=1&z=12600&ds=0&cmt=0&cats=0&tag=0&nu=1&bt=dGVzdCB
ten 67145377

By default request does not follow redirects for non-GET requests, which is what is happening in this case. So set followAllRedirects: true in your request() options and it should work fine.