I'm using python to implement a simple websocket server.
The handshake I'm using comes from http://en.wikipedia.org/w/index.php?title=WebSockets&oldid=372387414.
The handshake itself seems to work, but when I hit send, I get a javascript error:
Uncaught Error: INVALID_STATE_ERR: DOM Exception 11
Here's the html:
<!doctype html>
<html>
<head>
<title>ws_json</title>
</head>
<body onload="handleLoad();" onunload="handleUnload();">
<input type="text" id='input' />
<input type="button" value="submit" onclick="handleSubmit()" />
<div id="display"></div>
<script type="text/javascript">
function showmsg(str){
display = document.getElementById("display");
display.innerHTML += "<p>" + str + "</p>";
}
function send(str){
ws.send(str.length);
ws.send(str);
}
function handleSubmit(){
input = document.getElementById('input');
send(input.value);
input.focus();
input.value = '';
}
function handleLoad(){
ws = new WebSocket("ws://localhost:8888/");
ws.onopen = function(){
showmsg("websocket opened.");
}
ws.onclose = function(){
showmsg("websocket closed.");
}
}
function handleUnload(){
ws.close();
}
</script>
</body>
</html>
And here's the python code:
import socket
import threading
import json
PORT = 8888
LOCATION = "localhost:8888"
def handler(s):
print " in handler "
ip, _ = s.getpeername()
print "New connection from %s" % ip
request = s.recv(1024)
print "\n%s\n" % request
print s.getpeername()
# send response
response = "HTTP/1.1 101 Web Socket Protocol Handshake\r\n"
response += "Upgrade: WebSocket\r\n"
response += "Connection: Upgrade\r\n"
try:
peername = s.getpeername()
response += "Sec-WebSocket-Origin: http://%s\r\n" % peername[0] # % request[request.index("Origin: ")+8:-4]
except ValueError:
print "Bad Request"
raise socket.error
response += "Sec-WebSocket-Location: ws://%s\r\n" % LOCATION
response += "Sec-WebSocket-Protocol: sample"
response = response.strip() + "\r\n\r\n"
print response
s.send(response)
while True:
length = s.recv(1)
print length
if not length:
break
length = int(length)
print "Length: %i" % length
data = s.recv(length)
print "Received: %s" % data
print ""
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('localhost', PORT))
s.listen(5)
print "server is running..."
while True:
sock, addr = s.accept()
threading.Thread(target=handler, args=(sock, )).start()
Does anyone know what I'm doing wrong here?
I tested your code on Firefox 4 and got the same error upon hitting send, however before that I got
Firefox can't establish a connection
to the server at ws://localhost:8888/.
which is probably why the WebSocket object was destroyed. I suspect your handshake response is missing something, so Firefox is closing the socket.
From the Wikipedia article on Websockets:
The Sec-WebSocket-Key1 and
Sec-WebSocket-Key2 fields and the
eight bytes after the fields are
random tokens which the server uses to
construct a 16 byte token at the end
of its handshake to prove that it has
read the client's handshake.
Your server's response does not have this special number at the bottom, So I think we need to figure out how to generate it, and include it.
EDIT: How to generate that number
Lets start with key1, key2, and the 8 bytes at the end of the handshake
key1 = "18x 6]8vM;54 *(5: { U1]8 z [ 8"
key2 = "1_ tx7X d < nw 334J702) 7]o}` 0"
end8 = "Tm[K T2u"
We make a number for each key by ignoring every character that is not a digit 0-9. In Python:
def numFromKey(key):
return int(filter(lambda c: c in map(str,range(10)),key))
next we divide that number by the number of spaces in the original key string, so here is a is a function that counts the spaces in a string.
def spacesIn(key):
return len(filter(lambda c: c==' ',key))
The two numbers resulting from the keys are:
pkey1 = numFromKey(key1)/spacesIn(key1)
pkey2 = numFromKey(key2)/spacesIn(key2)
Now we need to concatenate the bytes of pkey1, pkey2, and end8. The processed keys need to be represented as 32 bit Big-Endian numbers.
from struct import pack
catstring = pack('>L',pkey1) + pack('>L',pkey2) + end8
Then we take the md5 hash of those bytes to get the magic number that we tack on the end of the handshake
import md5
magic = md5.new(catstring).digest()
Thats how I think it works at least
As of Version 8, this protocol is deprecated please refer to:
http://tools.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-12.txt
for the new version of the protocol.
Related
I have setup a http server to send data in intervals of 20 seconds. The data starts at 101 and this number is incremented every time. So the sequence of numbers will be 101,102,103, etc
I also append to the data, after a ; delimiter, the timestamp that the server sends the data.
I think I have some bug in my javascript code, because I am observing this behaviour:
http server sends data "105" at 12:28:52.654
in my web page, I see data item "105" at time 12:29:12:690, ie 20 seconds later. 20 seconds is the interval that I send the data. So it seems like the EventSource onmessage function is being called but is processing the previous data item, "104" in this case.
The web page code:
<!DOCTYPE HTML>
<html>
<head>
<script type="text/javascript">
let source = new EventSource('/startmonitoring');
function startCallMonitoring(){
source.onmessage = function(event) {
console.log(event.data);
addCall(event.data);
};
source.addEventListener('error', function(e) {
if (e.readyState == EventSource.CLOSED) {
console.log("closed");
}
}, false);
}
function stopCallMonitoring() {
source.close();
}
function gettime() {
var currentDate = new Date();
var hour = currentDate.getHours();
var minute = currentDate.getMinutes();
var second = currentDate.getSeconds();
var millisecond = currentDate.getMilliseconds();
return pad(hour) + ":" + pad(minute) + ":" + pad(second) + "." + millisecond;
}
function getdate() {
var currentDate = new Date();
var date = currentDate.getDate();
var month = currentDate.getMonth(); //Be careful! January is 0 not 1
var year = currentDate.getFullYear();
return pad(date) + "/" + pad(month + 1) + "/" + pad(year);
}
function pad(n) {
return n<10 ? '0'+n : n;
}
function addCall(callerid) {
// insert new row.
var tableref = document.getElementById('CallsTable').getElementsByTagName('tbody')[0];
var newrow = tableref.insertRow(0);
var datecell = newrow.insertCell(0);
var timecell = newrow.insertCell(1);
var calleridcell = newrow.insertCell(2);
var customerlinkcell = newrow.insertCell(3);
datecell.innerHTML = getdate();
timecell.innerHTML = gettime();
calleridcell.innerHTML = callerid;
customerlinkcell.innerHTML = "customerlink";
console.log("added " + callerid + " at " + gettime());
}
</script>
</head>
<body>">
<button onclick="startCallMonitoring()">Start Call Monitoring</button>
<button onclick="stopCallMonitoring()">Stop Call Monitoring</button>
<table id="CallsTable">
<thead>
<tr>
<th>Date</th>
<th>Time added to table</th>
<th>CallerID</th>
<th>link</th>
</tr>
</thead>
<tbody>
<tr>
</tr>
</tbody>
</table>
</body>
</html>
Screenshot of event stream in Chrome developer tools.
Why this behaviour? How can I fix it?
Additional information regarding the server side.
I wrote the http server myself so that could be a cause. Without sending the whole code for the server, which is quite large, here is the code using some helper functions to create an HTTP response message.
This timerfunc is called every 20 seconds.
Basically, when I see in the server console the output:
timerfunc() - sending: HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 29
Cache-Control: no-cache
Content-Type: text/event-stream
Access-Control-Allow-Origin: *
id: 7
data: 106;12:29:12.689
to 192
Then in web browser, data item 105 is populated.
void http_server::timerfunc() {
http_response rs;
rs.status = 200;
rs.set_version(1, 1);
rs.add_header("Connection", "keep-alive");
rs.add_header("Content-Type", "text/event-stream"); // this is REQUIRED
//header('Cache-Control: no-cache');
rs.add_header("Cache-Control", "no-cache"); // not sure if required, investigate what it does
rs.add_header("Access-Control-Allow-Origin", "*"); // think because for node.js demo was on different network - don't think need this
//rs.add_header("Transfer-Encoding", "chunked"); // doesn't work if you don't do chunking - investigate - but don't need
static unsigned number = 100;
std::string callerid = std::to_string(number);
char timebuf[50] = {};
get_timestamp(timebuf);
rs.set_body("id: 7\ndata: " + callerid + ";" + timebuf + "\n");
rs.set_content_length_to_body_length();
unsigned retcode = 0;
const size_t len = rs.get_content_length();
for (auto client : clients) {
std::string s = codec.make_http_response_message(rs);
retcode = send(client, s.c_str(), s.length());
std::cout << "timerfunc() - sending: " << s << " to " << client << std::endl;
}
number++;
if (number == 999)
number = 100;
}
I thought it would be useful to publish how I eventually fixed the problem.
It was nothing to do with the client. I had to make the following changes to the web server sending the event to make it fully work:
Change header in requests with event payload to Transfer-Type: chunked.
Payload of request must use the chunked format of \r\n\r\n0\r\n
For Server Sent Events the data stream to send must be prepended with "data: "
So if you wanted to send "My lovely streamed data part 1" as a stream message the http message would look like this:
HTTP/1.1 200 OK\r\n
Connection: Keep-Alive\r\n
Date: Wed, 15 Jan 2020 18:40:24 GMT\r\n
Content-Type: text/event-stream\r\n
Transfer-Encoding: chunked\r\n
Cache-Control: no-cache\r\n
\r\n
24\r\n
data: My lovely streamed data part 1\r\n
0\r\n
Not sure if the Cache-Control header is required.
The web server can send multiple segments of text prepended with length then \r\n then string then \r\n.
Read up on Transfer-Encoding here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding
I'm in the process of converting a PHP function to JS for use with Node.
The PHP function:
takes in a partially formed packet as an arg
finalizes the packet
creates a socket
sends the packet over the socket
Reads the response code from the server
Looks up the error code in an array
Sends the error code back to the caller
I have most of the function converted except for the bit that reads the server response.
Original PHP Function
(Comments are my understanding of what the code does. May be incorrect)
function serverInteractive($buf) { // $buf = partially formed packet
$fp = fsockopen($ip, $port , $errno, $errstr, 5);
$rs = '';
if (!$fp) return $this -> fsockerror;
$packet = pack("s", (strlen($buf)+2)).$buf; // Finalizes the packet
fwrite($fp, $packet); // Sends the packet to the server.
// ----- Read Server Response START -----//
$len = unpack("v", fread($fp, 2));
$rid = unpack("c", fread($fp, 1));
for ($i = 0; $i < (($len[1] - 4) / 4); $i++) {
$read = unpack("i", fread($fp, 4));
$rs .= $read[1];
}
// ----- Read Server Response FINISH -----//
fclose($fp); // Closes the socket
$result = $this -> socketerrors[$rs];
// $socketerrors is an array of error messages.
return($result);
}
My JavaScript Version
var net = require('net');
var submitPacket = function(packet) {
// Generate Final Packet
var p = pack('s', packet.length + 2) + packet;
// Create socket to Server
var serverSocket = net.createConnection(config.serverConfig.port,
config.serverConfig.host,
function() {
// Executes of connection is OK.
console.log("Connected to Server");
if (serverSocket.write(p, function() {
console.log("Buffer Flushed!");
})) {
console.log("Packet sent ok!");
} else {
console.log("There was a problem sending the packet!")
}
});
serverSocket.on('error', function(error) {
if (error) {
console.log(error);
}
});
serverSocket.on('data', function(data) {
if (data) {
console.log("Response: ", data);
// Need to put the error code generation
// here and fire a callback.
serverSocket.end();
}
});
}
The response i get from the server looks something like this when everything is ok:
<Buffer 07 00 06 01 00 00 00>
When not ok, looks something like this:
<Buffer 0b 00 06 00 00 00 00 03 00 00 00>
Any advice would be greatly appreciated.
UPDATE 1: This is what i've come up with so far however it the resulting code is undefined.
serverdSocket.on('data', function(data) {
if (data) {
var response = toArrayBuffer(data);
var len = response.slice(0,2);
var rid = response.slice(0,1);
var rs = '';
for (var i = 0; i < ((len[1]-4) / 4); i++) {
var read = response.slice(0,4);
rs += read[1];
}
console.log("Code: ",rs);
}
UPDATE 2: The PHP unpack function does indeed convert a buffer of binary data into an array. It looks like i can do the same thing by JSON.stringify then JSON.parse() to get it into an array. I now have an array object of the correct data, but the rest of the function doesnt seem to replicate the original.
I'll give it a try, although you haven't actually said what you want the "Code:" output to look like for the two input strings. We'll start with the differences between the PHP code and the JavaScript code.
Let's talk about these lines from the PHP code:
$len = unpack("v", fread($fp, 2));
$rid = unpack("c", fread($fp, 1));
Now those little fread() function calls are actually reading data from an input stream/file/whatever. The bottom line is that $len gets its value from the first two bytes of the stream and $rid gets its value from the third byte.
Now compare with the JavaScript code:
var len = response.slice(0,2);
var rid = response.slice(0,1);
I have a few observations about this code. First, the calls to slice() are both operating on the same array-like object, starting from the same location. So the value of rid will be wrong. Second, the value for rid is never used in the sample code, so if you never really need it you can eliminate that line entirely (which you could not do in the PHP code). The third observation is that calling slice() seems like overkill. Why not just use the square brackets on response?
One final puzzle about the original PHP code. Looking at the code that builds up $rs:
$rs .= $read[1];
It looks like this is a string concatenation of the successive integer data values. It's been a few years since I worked in PHP, so maybe I'm missing some subtlety, but this seems like a kind of odd thing to do. If you could tell me what the expected output codes are supposed to be it would help.
That brings me to the data itself. Just guessing from the data examples in the PHP code, it looks like the first two bytes are a little-endian encoded 16-bit buffer length. The next byte is the rid value, which is 6 in both examples. Then it looks like the rest of the buffer is composed of 32-bit little-endian values. Of course this is a WAG, but I'm just working with what has been provided.
So here's some code that processes the data values from the two sample buffers. For the first buffer it prints Code: 1 and for the second Code: 03. If these are not the correct values, then let me know what the expected output is and I'll see if I can make it work.
function toArrayBuffer(buffer) {
var ab = new ArrayBuffer(buffer.length);
var view = new Uint8Array(ab);
for (var i = 0; i < buffer.length; ++i) {
view[i] = buffer[i];
}
return ab;
}
function serverSocketOnData(data)
{
if (data) {
var response = toArrayBuffer(data);
var len = response[0] + 256*response[1];
var rid = response[2]; // get rid of this line?
var rs = '';
for (var i = 3; i < len; i+=4) {
rs += response[i];
// if the 32-bit value could ever exceed 255, then use this instead:
// rs += response[i] + 256*response[i+1] +
// 256*256*response[i+2] + 256*256*256*response[i+3];
}
console.log("Code: ",rs);
}
}
function testArray(sourceArray)
{
var sourceBuffer = new Buffer(sourceArray);
console.log("source buffer");
console.log(sourceBuffer);
serverSocketOnData(sourceBuffer);
}
function main()
{
var sourceArray1 = [0x07,0x00,0x06,0x01,0x00,0x00,0x00];
var sourceArray2 = [0x0b,0x00,0x06,0x00,0x00,0x00,0x00,0x03,0x00,0x00,0x00];
testArray(sourceArray1);
testArray(sourceArray2);
}
main();
Are you using slice appropriately?
When you call slice on a string, it returns a string.
var hello = "hello"
var hel = hello.slice(0, 3)
var h = hel[1]
var ello = hello.slice(1)
Basically, I have a cgi script that prints out valid json, I have checked and I have a similar functions that work the same way but this one doesn't some reason and I can't find it.
Javascript:
function updateChat(){
$.ajax({
type: "get",
url: "cgi-bin/main.py",
data: {'ajax':'1', 'chat':'1'},
datatype:"html",
async: false,
success: function(response) {
alert(response); //Returns an empty string
},
error:function(xhr,err)
{
alert("Error connecting to server, please contact system administator.");
}
});
Here is the JSON that python prints out:
[
"jon: Hi.",
"bob: Hello."
]
I used json.dumps to create the JSON it worked in previous functions that have pretty much the same JSON layout only different content.
There is a whole bunch more of server code, I tried to copy out the relevant parts. Basically I'm just trying to filter an ugly chat log for learning purposes. I filter it with regex and then create a json out of it.
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
print "Content-type: text/html\n\n"
print
import cgi, sys, cgitb, datetime, re, time, random, json
cgitb.enable()
formdata = cgi.FieldStorage()
def tail( f, window=20 ):
BUFSIZ = 1024
f.seek(0, 2)
bytes = f.tell()
size = window
block = -1
data = []
while size > 0 and bytes > 0:
if (bytes - BUFSIZ > 0):
# Seek back one whole BUFSIZ
f.seek(block*BUFSIZ, 2)
# read BUFFER
data.append(f.read(BUFSIZ))
else:
# file too small, start from begining
f.seek(0,0)
# only read what was not read
data.append(f.read(bytes))
linesFound = data[-1].count('\n')
size -= linesFound
bytes -= BUFSIZ
block -= 1
return '\n'.join(''.join(data).splitlines()[-window:])
def updateChatBox():
try:
f = open('test.txt', 'r')
lines = tail(f, window = 20)
chat_array = lines.split("\n")
f.close()
except:
print "Failed to access data"
sys.exit(4)
i = 0
while i < len(chat_array):
#remove timer
time = re.search("(\[).*(\])", chat_array[i])
result_time = time.group()
chat_array[i] = chat_array[i].replace(result_time, "")
#Removes braces around user
user = re.search("(\\().*?(_)", chat_array[i])
result_user = user.group()
chat_array[i] = chat_array[i].replace("(", "")
chat_array[i] = chat_array[i].replace(")", "")
#Removes underscore and message end marker
message = re.search("(_).*?(\|)", chat_array[i])
result_message = message.group()
chat_array[i] = chat_array[i].replace("_", ":")
chat_array[i] = chat_array[i].replace("|", "")
data += chat_array[i] + "\n"
i = i + 1
data_array = data.split("\n")
json_string = json.dumps(data_array)
print json_string
if formdata.has_key("ajax"):
ajax = formdata["ajax"].value
if ajax == "1": #ajax happens
if formdata.has_key("chat"):
chat = formdata["chat"].value
if chat == 1:
updateChatBox()
else:
print "ERROR"
elif formdata.has_key("get_all_stats"):
get_all_stats = formdata["get_all_stats"].value
if get_all_stats == "1":
getTopScores()
else:
print "ERROR"
Here is also a function that works perfectly and is in the same python file
def getTopScores():
try:
f = open('test_stats.txt', 'r')
stats = f.read()
stats_list = stats.split("\n")
f.close()
except:
print "Failed reading file"
sys.exit(4)
json_string = json.dumps(stats_list)
print json_string
The only difference is using the tail function and regex, the end result JSON actually looks identical.
Are you certain that updateChatBox is even getting called? Note that you compare ajax to the string "1" but you compare chat to the integer 1. I bet one of those doesn't match (in particular the chat one). If that doesn't match, your script will fall through without ever returning a value.
Also, though it isn't the root cause, you should clean up your content types for correctness. Your Javascript AJAX call is declared as expecting html in response, and your cgi script is also set to return content-type:text/html. These should be changed to json and content-type:application/json, respectively.
I wonder if anyone can help me. I'm new to nodejs and I've been trying to send a message to a server using nodejs as the client. The server is written in C and looking at a PHP installation it makes use of the pack('N',len) to send the length of the string to the server. I have tried to implement a similar thing in javascript but am hitting some problems. I wonder if you can point out where I am going wrong (credit to phpjs from where I copied the packing string code).
My client nodejs javascript code is:
var net = require('net');
var strs = Array("<test1>1</test1>",`
"<test_2>A STRING</test_2>", "<test_3>0</test_3>",
"<test_4></test_4>", "<test_5></test_5>",
"<test_6></test_6>", "<test_7_></test_7>",
"<test_8></test_8>", "<test_9>10</test_9>",
"<test_10></test_10>", "<test_11></test11>",
"<test_12></test_12>", "<test_13></test_13>",
"<test_14></test_14>");
hmsg = strs[0] + strs[1] + strs[2] + strs[3] + strs[4] + strs[5] + strs[6];
console.log(hmsg.length);
msg = hmsg + "<test_20></test_20>"`
msglen = hmsg.length;
astr = '';
astr += String.fromCharCode((msglen >>> 24) && 0xFF);
astr += String.fromCharCode((msglen >>> 16) && 0xFF);
astr += String.fromCharCode((msglen >>> 8) & 0xFF);
astr += String.fromCharCode((msglen >>> 0) & 0xFF);
var pmsg = astr + msg;
console.log(pmsg);
var client = net.createConnection({host: 'localhost', port: 1250});
console.log("client connected");
client.write(pmsg);
client.end();
Running 'node testApp' prints out the correct length of the header string. If I look at what the server is receiving I can see that as long as the header string is < 110 chars it decodes the correct length, but if the header string is > 110 (by adding strs[6] or more to the hmsg) the decoded length is incorrect. Including strs[6] I get a string of length 128
on the client side and 194 on the server side.
I am clearly doing something wrong in packing the integer, but I'm not familiar with packing bits and am not sure where I am going wrong. Can anyone point out where my error is?
Many thanks!
Update
Thanks to Fedor Indutny on the nodejs mailing list the following worked for me:
console.log(hmsg.length, msg.length);
var msglen = hmsg.length;
var buf = new Buffer(msg.length+4);
mslen = buf.writeUInt32BE(msglen, 0);
mslen = buf.write(msg, 4);
var client = net.createConnection({host: 'localhost', port: 8190});
console.log("client connected");
client.write(buf);
client.end();
I.e. using the Buffer's writeUInt32 was all that was needed for the length of the header message. I'm posting here in the hope it may help someone else.
I see that you managed to get it working on your own, but for a much easier fix you could just use the binary module.
https://github.com/substack/node-binary
Install with npm install binary
I'm trying to decrypt a ciphertext created by CryptoJS using PyCrypto. I am using AES-256-CTR, with a 12-byte random prefix and 4-byte counter. So far, I've had limited success. Please read this previous post where I made a first attempt.
This works in Javascript:
Install the CryptoCat extension
Run CryptoCat
Fire up the developer console (F12 in Chrome/Firefox)
Run these lines of code
key = 'b1df40bc2e4a1d4e31c50574735e1c909aa3c8fda58eca09bf2681ce4d117e11';
msg = 'LwFUZbKzuarvPR6pmXM2AiYVD2iL0/Ww2gs/9OpcMy+MWasvvzA2UEmRM8dq4loB\ndfPaYOe65JqGQMWoLOTWo1TreBd9vmPUZt72nFs=';
iv = 'gpG388l8rT02vBH4';
opts = {mode: CryptoJS.mode.CTR, iv: CryptoJS.enc.Base64.parse(iv), padding: CryptoJS.pad.NoPadding};
CryptoJS.AES.decrypt(msg, CryptoJS.enc.Hex.parse(key), opts).toString(CryptoJS.enc.Utf8);
Expected output: "Hello, world!ImiAq7aVLlmZDM9RfhDQgPp0CrAyZE0lyzJ6HDq4VoUmIiKUg7i2xpTSPs28USU8".
Here is a script I wrote in Python that partially(!) decrypts the ciphertext:
import struct
import base64
import Crypto.Cipher.AES
import Crypto.Util.Counter
def bytestring_to_int(s):
r = 0
for b in s:
r = r * 256 + ord(b)
return r
class IVCounter(object):
def __init__(self, prefix="", start_val=0):
self.prefix = prefix
self.first = True
self.current_val = start_val
def __call__(self):
if self.first:
self.first = False
else:
self.current_val += 1
postfix = struct.pack(">I", self.current_val)
n = base64.b64decode(self.prefix) + postfix
return n
def decrypt_msg(key, msg, iv):
k = base64.b16decode(key.upper())
ctr = IVCounter(prefix=iv)
#ctr = Crypto.Util.Counter.new(32, prefix=base64.b64decode(iv), initial_value=0, little_endian=False)
aes = Crypto.Cipher.AES.new(k, mode=Crypto.Cipher.AES.MODE_CTR, counter=ctr)
plaintext = aes.decrypt(base64.b64decode(msg))
return plaintext
if __name__ == "__main__":
#original:
key = 'b1df40bc2e4a1d4e31c50574735e1c909aa3c8fda58eca09bf2681ce4d117e11'
msg = 'LwFUZbKzuarvPR6pmXM2AiYVD2iL0/Ww2gs/9OpcMy+MWasvvzA2UEmRM8dq4loB\ndfPaYOe65JqGQMWoLOTWo1TreBd9vmPUZt72nFs='
iv = 'gpG388l8rT02vBH4'
decrypted = decrypt_msg(key, msg, iv)
print "Decrypted message:", repr(decrypt_msg(key, msg, iv))
print decrypted
The output is:
'Hello, world!Imi\xfb+\xf47\x04\xa0\xb1\xa1\xea\xc0I\x03\xec\xc7\x13d\xcf\xe25>l\xdc\xbd\x9f\xa2\x98\x9f$\x13a\xbb\xcb\x13\xd2#\xc9T\xf4|\xd8\xcbaO)\x94\x9aq<\xa7\x7f\x14\x11\xb5\xb0\xb6\xb5GQ\x92'
The problem is, only the first 16 bytes of the output match the first 16 bytes of the expected output!
Hello, world!ImiAq7aVLlmZDM9RfhDQgPp0CrAyZE0lyzJ6HDq4VoUmIiKUg7i2xpTSPs28USU8
When I modify the script to do this:
def __init__(self, prefix="", start_val=1):
and
self.current_val += 0 #do not increment
which makes the counter output the same value (\x00\x00\x00\x01) every time it is called, the plaintext is:
\xf2?\xaf:=\xc0\xfd\xbb\xdf\xf6h^\x9f\xe8\x16I\xfb+\xf47\x04\xa0\xb1\xa1\xea\xc0I\x03\xec\xc7\x13dQgPp0CrAyZE0lyzJ\xa8\xcd!?h\xc9\xa0\x8b\xb6\x8b\xb3_*\x7f\xf6\xe8\x89\xd5\x83H\xf2\xcd'\xc5V\x15\x80k]
where the 2nd block of 16 bytes (dQgPp0CrAyZE0lyzJ) matches the expected output.
When I set the counter to emit \x00\x00\x00\x02 and \x00\x00\x00\x03, I get similar results- subsequent 16-byte blocks are revealed. The only exception is that with 0s, the first 32 bytes are revealed.
All 0s: reveals first 32 bytes.
'Hello, world!ImiAq7aVLlmZDM9RfhD\xeb=\x93&b\xaf\xaf\x8d\xc9\xdeA\n\xd2\xd8\x01j\x12\x97\xe2i:%}G\x06\x0f\xb7e\x94\xde\x8d\xc83\x8f#\x1e\xa0!\xfa\t\xe6\x91\x84Q\xe3'
All 1s: reveals next 16 bytes.
"\xf2?\xaf:=\xc0\xfd\xbb\xdf\xf6h^\x9f\xe8\x16I\xfb+\xf47\x04\xa0\xb1\xa1\xea\xc0I\x03\xec\xc7\x13dQgPp0CrAyZE0lyzJ\xa8\xcd!?h\xc9\xa0\x8b\xb6\x8b\xb3_*\x7f\xf6\xe8\x89\xd5\x83H\xf2\xcd'\xc5V\x15\x80k]"
All 2s: reveals next 16 bytes.
'l\xba\xcata_2e\x044\xb2J\xe0\xf0\xd7\xc8e\xae\x91yX?~\x7f1\x02\x93\x17\x93\xdf\xd2\xe5\xcf\xe25>l\xdc\xbd\x9f\xa2\x98\x9f$\x13a\xbb\xcb6HDq4VoUmIiKUg7i\x17P\xe6\x06\xaeR\xe8\x1b\x8d\xd7Z\x7f"'
All 3s: reveals next 13 bytes.
'I\x92\\&\x9c]\xa9L\xb1\xb6\xbb`\xfa\xbet;#\x86\x07+\xa5=\xe5V\x84\x80\x9a=\x89\x91q\x16\xea\xca\xa3l\x91\xde&\xb6\x17\x1a\x96\x0e\t/\x188\x13`\xd2#\xc9T\xf4|\xd8\xcb`aO)\x94\x9a2xpTSPs28USU8'
If you concat the "correct" blocks, you'll get the expected plaintext:
Hello, world!ImiAq7aVLlmZDM9RfhDQgPp0CrAyZE0lyzJ6HDq4VoUmIiKUg7i2xpTSPs28USU8
This is really strange. I am definitely doing something wrong on the Python end as things can be decrypted, but not all in one go. If anyone can help, I would be really grateful. Thank you.
There are a couple problems here. First, the message is not a multiple of the block size, and you're not using padding. And second -- and the most crucial to this issue -- is that the IV is also not the correct size. It should be 16 bytes, but you only have 12. Probably both implementations should fail with an exception, and in the next major revision of CryptoJS, this will be the case.
Here's what happens due to this mistake: When the counter increments for the first time, it tries to increment the undefined value, because the last byte of the IV is missing. Undefined + 1 is NaN, and NaN | 0 is 0. That's how you end up getting 0 twice.
When using crypto mode CryptoJS.mode.CTR (where CTR stands for counter) the Initailization vector together with a counter is encrypted and then applied to the data to encrypt. This is done for each block of data you encrypt.
You explain that different parts of the message are decrypted correctly, when you apply different values to start_val, so I suspect that the counter is simply not correctly increased with each decrypt of a block.
Take a look at Block Cipher Mode: CTR at wikipedia
Caution: Please note that when using the CTR mode, the combination of the initialization vector + the counter should never be repeated.
Fixed. I simply made the counter start with 0 twice. Does anyone know if this is a vulnerability?
import struct
import base64
import Crypto.Cipher.AES
import Crypto.Util.Counter
import pdb
def bytestring_to_int(s):
r = 0
for b in s:
r = r * 256 + ord(b)
return r
class IVCounter(object):
def __init__(self, prefix="", start_val=0):
self.prefix = prefix
self.zeroth = True
self.first = False
self.current_val = start_val
def __call__(self):
if self.zeroth:
self.zeroth = False
self.first = True
elif self.first:
self.first = False
else:
self.current_val += 1
postfix = struct.pack(">I", self.current_val)
n = base64.b64decode(self.prefix) + postfix
return n
def decrypt_msg(key, msg, iv):
k = base64.b16decode(key.upper())
ctr = IVCounter(prefix=iv)
#ctr = Crypto.Util.Counter.new(32, prefix=base64.b64decode(iv), initial_value=0, little_endian=False)
aes = Crypto.Cipher.AES.new(k, mode=Crypto.Cipher.AES.MODE_CTR, counter=ctr)
plaintext = aes.decrypt(base64.b64decode(msg))
return plaintext
if __name__ == "__main__":
#original:
key = 'b1df40bc2e4a1d4e31c50574735e1c909aa3c8fda58eca09bf2681ce4d117e11'
msg = 'LwFUZbKzuarvPR6pmXM2AiYVD2iL0/Ww2gs/9OpcMy+MWasvvzA2UEmRM8dq4loB\ndfPaYOe65JqGQMWoLOTWo1TreBd9vmPUZt72nFs='
iv = 'gpG388l8rT02vBH4'
decrypted = decrypt_msg(key, msg, iv)
print "Decrypted message:", repr(decrypt_msg(key, msg, iv))
print decrypted