I'm trying to use python / mechanize to login to this webpage:
http://www.solaradata.com/cgi-bin/mainProgram.cgi
The login form uses a Javascript function that produces an MD5 hash from several field values before submitting the results for authentication. Since mechanize can't do javascript, I tried to replicate the same functionality inside of python and then submit the resulting values. However, I'm still getting "invalid user / password" errors.
Here's my current code, can anybody point me towards where I went wrong? Thanks!
url_login = 'http://www.solaradata.com/cgi-bin/mainProgram.cgi'
import mechanize
import md5
username = 'superfly' #not my real user/pass
password = 'stickyguy' #not my real user/pass
br = mechanize.Browser()
br.open(url_login)
br.select_form(nr=0)
br.set_all_readonly(False)
session = br['session']
br['user'] = username
br['password'] = password
m1 = md5.new()
m1.update(password + username)
br['password'] = m1.digest()
m2 = md5.new()
m2.update(password + session)
br['hash'] = m2.digest()
for form in br.forms():
#print form
request2 = form.click() # mechanize.Request object
try:
response2 = mechanize.urlopen(request2)
except mechanize.HTTPError, response2:
pass
print response2.geturl()
# headers
for name, value in response2.info().items():
if name != "date":
print "%s: %s" % (name.title(), value)
print response2.read() # body
response2.close()
Use m1.hexdigest() instead of m1.digest()
I'm not familiar with python, but it appears the they are returning the hex value of the MD5 hash in the javascript version of the algorithm. Does the python MD5 do the same?
You should be able to test this without going through the submission process and testing for success. Instead, using a JavaScript developer tool such as Firebug or Chrome developer tools, calculate the result you get in-page. Then, using the same inputs, see what you get from your program. They should match, character for character.
It way be overkill but if you really need to script access a javascript heavy site you can look at selenium-rc or source labs.
These tools allow you to script an actual browser the same as a user.
Selenium
Related
I am attempting to save javascript source as text to a mysql table. I am using MEDIUM TEXT for the data type in the database. When testing I noticed I am unable to successfully save the text if it includes alert(); anywhere within the textarea being saved to the database. I understand the XSS vulnerability and thought maybe it's a built-in precaution by the server provider so I even attempted to save the string in base64 encoding. Still I receive a 403 forbidden error when trying to save the source text.
The description on the 403 Forbidden error is: You don't have permission to access / on this server.
$string1 = 'console.log("test");';
//successfully saves when attempting to insert into database
$string2 = 'alert("boo");';
//error 403 Forbidden when attempting to insert into database
$string3 = 'alert("boo");';
$encstr = base64_encode($string3);
//error 403 Forbidden when attempting to insert into database
Any ideas how to fix or circumvent? This is a new service provider I am using (namecheap), I don't get this error when running the same code on my digitalocean VPS.
UPDATE TO SHOW SOURCE:
<?php
require("conn.php");
if(isset($_POST["savecode"])){
$getuniqueid = uniqid();
$gettime = time();
$getscriptcode = $_POST["scriptcode"];
$sanitizescriptcode = mysqli_real_escape_string($conn, $getscriptcode);
//$sanitizescriptcode = htmlspecialchars($sanitizescriptcode);
$sanitizescriptcode = base64_encode($sanitizescriptcode);
$insertscript = "INSERT INTO scriptcode (`postid`,`content`,`ttime`) VALUES('$getuniqueid','$sanitizescriptcode','$gettime')";
mysqli_query($conn, $insertscript)or die("Fatal error attempting to save JavaScript");
header("Location: ?id=".$getuniqueid);
exit();
}
?>
<!DOCTYPE html>
<html>
<body>
<div id="hbanner"><form method="post"><button type="submit" name="savecode" id="savecode" class="btn btn-info">SAVE</button></div>
<textarea id="scriptcode" name="scriptcode" class="codeinput"></textarea></form>
</body>
</html>
NameCheap uses ModSecurity, which is a Web Application Firewall (or WAF). A number of the filters are related to preventing cross-site-scripting. (Note: Their support can either disable it at an account level, or whitelist certain rules for you, but you have to contact them directly. They can also pull logs to tell you which filters are being hit.)
It's possible that the base64-encoded version is actually hitting a different rule, since this is commonly used by malware to obfuscate their malicious code. Another possibility is that it's the pressense of the script in the POST request, so you could try base64-encoding that before it's submitted using JavaScript (using window.btoa(data)) and then decoding it on the PHP side with base64_decode.
You can test whether it's at the MySQL level by just commenting out all of that code and seeing if you still get a 403 when submitting the form.
If it's at the SQL level, you could try to pass a chunked string with some known delimiter that wouldn't appear in your strings, then replace it out in SQL. For example, something like this:
$insertscript = "INSERT INTO scriptcode (`postid`,`content`,`ttime`) VALUES('$getuniqueid',REPLACE('" . mysqli_real_escape_string($conn,chunk_split($_POST['scriptcode'],4,"†")) . "','†',""),'$gettime')";
In this example, chunk_split is inserting a special character every 4th position, and then the MySQL REPLACE function is stripping them out after the query is already sent to the database server.
Keep in mind that this is super hacky and somewhat inefficient—I'm just trying to offer some potential workarounds that don't involve support.
New at coding. So part of my assignment require me to validate login credentials through JSON.
user.json file would look something like this.. first is the email address then their password.
{
"mary#mary.com":"12345678",
"joseph#gmail.com":"293sfvdet"
}
my website will ask for the login and will go through JSON to validate the information. I am only allowed to use JSON, JS and HTML.I am definitely not familiar with JSON.
i would like to know how I could access my JSON file through JS and how i should go about using JSON for validation.
You may link your local json file in the page:
<script type="text/javascript" src="user.json"></script>
Then you can use it like:
var userdata = JSON.parse(user);
First of all (although you are maybe aware of this) this is not a secure way to implement a login at all but I assume, since this is for your assignment, this isn't really relevant to you.
Instead of writing your user data into a seperate JSON-file, you could just declare a constant inside your JS code like so:
const users = {
"mary#mary.com":"12345678",
"joseph#gmail.com":"293sfvdet"
}
After that you could just validate the entered credentials against this.
You may want to read this to learn more about javascript objects.
You're going to want to use Javascript to read the contents of the JSON file. (jQuery can help with this, if you're allowed to use a library, otherwise, look into fetch).
Then, you'll take the returned object that will look something like: response = { "mary#mary.com":"12345678", "joseph#gmail.com":"293sfvdet" } and you'll check to see if the email/password entered matches any record in the json.
var enteredEmail = 'mary#mary.com';
var enteredPassword = '12345678';
if(response[enteredEmail] == enteredPassword){
// login validated.
}
As other users have pointed out, storing user info in a JSON file like this is not a secure practice - nor is it practical. However, if it's just an assignment, this should work.
I'm working on making a real-time application for my website in NodeJS, allowing my users to log in with their accounts, etc.
However, I am having some issues with the logging in part.
When I register/login the users on the main site, I hashed their password using PHP's hash() function like so:
$passwordSalt = mcrypt_create_iv(100);
$hashed = hash("sha256", $password.$passwordSalt.$serverSalt);
and it works great on my site
However I need to be able to grab the user's salt from the database in NodeJS and be able to hash the user's inputted password, check it against the database's password, and make sure they match to log the user in.
I do this by doing something like this:
//Check if Username exists, then grab the password salt and password
//Hash the inputted password with the salt in the database for that user
//and the salt I used for $serverSalt in PHP when creating passwords
//check if hashed result in NodeJS is equal to the database password
function checkPass(dbPassword, password, dbSalt){
var serverSalt = "mysupersecureserversalt";
var hashed = crypto.createHash("sha256").update(password+dbSalt+serverSalt).digest('hex');
if(hashed === dbPassword)
return true;
return false;
}
However, when I console.log() the hashed variable and the dbPassword variable, they're not equal - so it always return false/responds with incorrect password.
So, my question:
Is there any way I can accurately hash a sha256 string in NodeJS the same way I do in PHP?
PS:
For now I am using Ajax/jQuery to login via PHP Script but I want to be able to completely move from Apache/PHP hosting to the site just being hosted with NodeJS (SocketIO, Express, MySQL).
I just started working with NodeJS recently and I have made functionality on my Apache site to work with NodeJS, but I heard that hosting the whole site itself using NodeJS would be a lot better/more efficient.
Edit:
So, I decided to make a quick test.js without using the database/socketio/express.
var crypto = require("crypto");
var serverSalt = "";
var passwordSalt = ""; //The salt directly copied from database
var checkPassword = "password123"+passwordSalt+serverSalt; //All added together
var password = ""; //The hashed password from database
var hashed = crypto.createHash("sha256").update(checkPassword).digest('hex');
console.log(password);
console.log(hashed); //This doesn't match the hash in the database
if(password == hashed){
console.log("Logged in!");
} else {
console.log("Error logging in!");
}
As for how I'm connecting to the database, I'm doing so:
connection.query("SELECT password,passwordSalt FROM users WHERE username = "+connection.escape(data.username), function(err,res){
if(err){console.log(err.stack);socket.emit("message", {msg:"There was an error logging you in!", mType:"error"});}else{
if(res.length != 0){
var dbSalt = res[0]['passwordSalt'];
var serverSalt = ""; //My server salt
var dbPassword = res[0]['password'];
var checkPassword = data.password+dbSalt+serverSalt;
console.log("CheckPass: "+checkPassword);
var hashed = crypto.createHash("sha256").update(checkPassword).digest('hex');
console.log("Hashed: "+hashed);
if(hashed === dbPassword){
console.log("Worked!");
socket.emit("message", {msg: "Logged in!", type:"success"});
} else {
console.log("Error logging in!");
socket.emit("message", {msg: "Your password is incorrect!", type:"error"});
}
} else {
socket.emit("message", {msg: "That user ("+data.username+") doesn't exist!", mType:"error"});
}
}
});
MySQL version: 5.5.44-0+deb7u1 (Debian)
The column the password salt is stored in is type of text, and has a collation of utf8_unicode_ci
Note: When I change
var hashed = crypto.createHash("sha256").update(checkPassword).digest('hex');
To:
var hashed = crypto.createHash("sha256").update(checkPassword, "utf8").digest('hex');
The hashes are different, however, the hashed variable doesn't match the database password still.
TL;DR
2 possibilities:
The ideal solution: change the database field for the salt from TEXT to BLOB.
The compromise: cast the TEXT to binary latin1 using:
BINARY(CONVERT(passwordSalt USING latin1)) as passwordSalt
Then in both cases, use Buffer values everywhere:
var hashed = crypto.createHash("sha256").update(
Buffer.concat([
new Buffer(password),
dbSalt, // already a buffer
new Buffer(serverSalt)
])
).digest('hex');
It's tested and working.
The longer version
And of course the culprit is character encoding, what a surprise. Also a terrible choice on your part to store raw binary to a TEXT field.
Well, that was annoying to debug. So, I set up a MySQL table with a TEXT field and a BLOB field and stored the output of mcrypt_create_iv(100) in both. Then I made the same queries from both PHP and NodeJS.
PHP presents both values as identical.
JavaScript presents 2 different values.
In both cases, the BLOB was accurate and I even managed to get the proper hash under JavaScript by using Buffer values for all 3 components of the input.
But this did not explain why PHP and JavaScript were seeing 2 differents values for the TEXT field.
The TEXT value had a length of 143 octets.
The BLOB had a length of 100 octets.
Clearly the BLOB was correct and the TEXT wasn't, yet PHP didn't seem bothered by the difference.
If we look at the MySQL connection status under PHP:
$mysqli->get_charset();
Partial output:
[charset] => latin1
[collation] => latin1_swedish_ci
Unsurprising really, it is notorious that PHP operates under ISO-8859-1 by default (or latin1 in MySQL), which is why both values where the same there.
For some reason, it seems that setting the charset in the MySQL module for NodeJS doesn't work, at least for me. The solution was to convert at the field level and preserve the data by casting to BINARY:
BINARY(CONVERT(passwordSalt USING latin1)) as passwordSalt
This returned exactly the same output as the BLOB.
But this is not enough yet. We have a mixture of strings and binary to feed to the hashing function, we need to consolidate that. We cast the password and the server salt to Buffer and concatenate:
var hashed = crypto.createHash("sha256").update(
Buffer.concat([
new Buffer(password),
dbSalt, // already a buffer
new Buffer(serverSalt)
])
).digest('hex');
This returns the same output as PHP.
While this works, the best solution is still to use BLOB in the database. In both cases, casting to Buffer is necessary.
I want to automate form filling on a website with information of certain parameters that will return products based on the parameters I enter. I tried using mechanize in python but it does not support javascript and it seems like in order to navigate the entire process of filling in parameters requires pressing some buttons that seem like javascript objects. For instance the Guided Selection button:
<a onclick="_gaq.push(['_trackEvent', 'Navigation Menu', 'Click', 'Guided Selection Link']);" id="ctl00_NavigationMenu_ConfigureLink" href="javascript:__doPostBack('ctl00$NavigationMenu$ConfigureLink','')">Guided Selection</a></li>
I also tried using selenium but I do not want to create a new instance of a browser. Any python based suggestions? Perhaps jython? Thanks a lot!
My recommendation would be to simply capture the form information using a tool like firebug. And then "replay" the request (whether it's GET or POST) through Python. Here is a snippet for a POST request.
import urllib
import urllib2
url = 'http://www.someserver.com/cgi-bin/register.cgi'
values = {'name' : 'Michael Foord',
'location' : 'Northampton',
'language' : 'Python' }
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()
In fire bug you can get the url and values from under the console tab.
This probably won't work if you are doing automated form discovery.
I am making an AJAX chat room with the guidance of an AJAX book teaching me to use JSON and eval() function.
This chat room has normal chat function and a whiteboard feature.
When a normal text message comes from the php server in JSON format, the javascript in browser does this:
Without Whiteboard Command -------------------------------------------
function importServerNewMessagesSince(msgid) {
//loadText() is going to return me a JSON object from the server
//it is an array of {id, author, message}
var latest = loadText("get_messages_since.php?message=" + msgid);
var msgs = eval(latest);
for (var i = 0; i < msgs.length; i++) {
var msg = msgs[i];
displayMessage(escape(msg.id), escape(msg.author), escape(msg.contents));
} ...
The whiteboard drawing commands are sent by server in JSON format with special user name called "SVR_CMD", now the javascript is changed slightly:
With Whiteboard Command --------------------------------------------------
function importServerNewMessagesSince(msgid) {
//loadText() is going to return me a JSON object from the server
//it is an array of {id, author, message}
var latest = loadText("get_messages_since.php?message=" + msgid);
var msgs = eval(latest);
for (var i = 0; i < msgs.length; i++) {
var msg = msgs[i];
if (msg.author == "SVR_CMD") {
eval(msg.contents); // <-- Problem here ...
//I have a javascript drawLine() function to handle the whiteboard drawing
//server command sends JSON function call like this:
//"drawLine(200,345,222,333)" eval() is going to parse execute it
//It is a hacker invitation to use eval() as someone in chat room can
//insert a piece of javascript code and send it using the name SVR_CMD?
else {
displayMessage(escape(msg.id), escape(msg.author), escape(msg.contents));
}
} ...
Now, if the hacker changes his username to SVR_CMD in the script, then in the message input start typing javascript code, insdead of drawLine(200,345,222,333), he is injecting redirectToMyVirusSite(). eval() will just run it for him in everyone's browser in the chat room.
So, as you can see, to let the eval to execute a command from an other client in the chat room is obviously a hacker invitation. I understand the book I followed is only meant to be an introduction to the functions. How do we do it properly with JSON in a real situation?
e.g. is there a server side php or .net function to javascriptencode/escape to make sure no hacker can send a valid piece of javascript code to other client's browser to be eval() ? Or is it safe to use JSON eval() at all, it seems to be a powerful but evil function?
Thank you,
Tom
What is this book? eval is evil, there is not a single reason to use it, ever.
To transform a JSON string into a javascript object, you can do the following:
var obj = JSON.parse(latest)
Which means you can then use:
[].forEach.call(obj, function( o ) {
// You can use o.message, o.author, etc.
} )
To do the opposite (javascript object -> JSON string), the following works:
var json = JSON.stringify(obj)
It only is unsafe if the executed code is generated by other clients and not by the server. Of course you would need to prevent anybody to use that name, though I don't understand why you would use the "author" field? Just send an object {"whiteboard":"drawLine(x,y,z)"} instead of {"author":"SVR_CMD","contents":"drawLine(x,y,z)"}.
But it is right, eval() is still an invitation for hackers. One can always send invalid data and try to influence the output more or less directly. The only way for escaping is a proper serialisation of the data you want to receive and send - the drawings data. How do you receive the whiteboard commands? There is no serverside "escape" function to make javascript code "clean" - it would always be a security hole.
I would expect a serialisation like
message = {
"author": "...", // carry the information /who/ draws
"whiteboard": {
"drawline": [200, 345, 222, 333]
}
}
so you can sanitize the commands (here: "drawline") easiliy.
The use of eval() might be OK if you have very complex commands and want to reduce the transferred data by building them serverside. Still, you need to parse and escape the received commands from other clients properly. But I'd recommend to find a solution without eval.
Setting eval issue aside, do not use field that can be filled by user - .author in your code - for authentication purposes. Add another field to your JSON message, say .is_server_command that when present, would signify special treating of message. This field is will be not depended on user input and thus wouldn't be hijacked by "hacker".