I am trying to setup a communication between a python script (that will do a lot of computation on data that cannot be done in javascript and send send that data as a json) and a javascript client.
I have the following code for my python server:
import socket
import sys
from thread import *
HOST = '' # Symbolic name meaning all available interfaces
PORT = 9888 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
#Bind socket to local host and port
try:
s.bind((HOST, PORT))
except socket.error as msg:
print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
sys.exit()
print 'Socket bind complete'
#Start listening on socket
s.listen(10)
print 'Socket now listening'
#Function for handling connections. This will be used to create threads
def clientthread(conn):
#Sending message to connected client
conn.send('Welcome to the server. Type something and hit enter\n') #send only takes string
#infinite loop so that function do not terminate and thread do not end.
while True:
#Receiving from client
data = conn.recv(1024)
reply = 'OK...' + data
if not data:
break
conn.sendall(reply)
#came out of loop
conn.close()
#now keep talking with the client
while 1:
#wait to accept a connection - blocking call
conn, addr = s.accept()
print 'Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(clientthread ,(conn,))
s.close()
And the following code for my javascript client:
var connection = new WebSocket('ws://127.0.0.1:8999');
connection.onopen = function () {
connection.send('Hello'); // Send the message to the server
};
I get the following error from my javascript client:
Error during WebSocket handshake: net::ERR_INVALID_HTTP_RESPONSE
And the following output from my python server
Socket created
Socket bind complete
Socket now listening
Connected with 127.0.0.1:53956
Unhandled exception in thread started by <function clientthread at 0x10abac578>
Traceback (most recent call last):
File "server.py", line 71, in clientthread
data = conn.recv(1024)
socket.error: [Errno 54] Connection reset by peer
Would anyone know what is wrong?
Edit: forgot to mention that I have seen this SO Post before, but my problem is not the same, or rather should I say that the error encountered by OP is not the same as mine.
A WebSocket is a not the same as a plain TCP socket you create. WebSocket is a protocol on top of TCP instead which starts with a HTTP handshake and then continues with a framing based protocol. If you want to implement a WebSocket server in Python you need to implement this protocol as specified in RFC 6455 or use existing WebSocket libraries.
An example server-side python code using WebSocket is:
import asyncio
import websockets
async def handle_message(message):
print(message)
async def consumer_handler(websocket, path):
while True:
message = await websocket.recv()
await handle_message(message)
start_server = websockets.serve(consumer_handler, 'localhost', 8765)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
Related
Here is a small websockets client and server POC.
It sends a single hard-coded message string from the (Python) server to the Javascript client page.
The question is, how to send further, ad-hoc messages? From the server to the client.
Tiny HTML client page with embedded Javascript:
<!DOCTYPE html>
<html lang="en">
<body> See console for messages </body>
<script>
# Create websocket
const socket = new WebSocket('ws://localhost:8000');
# Add listener to receive server messages
socket.addEventListener('open', function (event) {
socket.send('Connection Established');
});
# Add message to browser console
socket.addEventListener('message', function (event) {
console.log(event.data);
});
</script>
</html>
Here is the Python server code:
import asyncio
import websockets
import time
# Create handler for each connection
async def handler(websocket, path):
await websocket.send("message from websockets server")
# Start websocket server
start_server = websockets.serve(handler, "localhost", 8000)
# Start async code
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
This successfully sends a hard-coded message from server to client.
You can see the message in the browser console.
At this point the websocket is open.
The main application (not shown) now needs to send messages.
These will be dynamic messages, not hard-coded.
How can we send later, dynamic messages from the server?
After the code here runs?
I would like to put the socket into a global variable and call a send method but this is not possible because the server runs a continuous loop.
You can insert further messages into the Python server code like this:
import asyncio
import datetime
from typing import Iterator
import websockets
import random
websocket_connections = set()
sock_port = 8000
sock_url = 'localhost'
global_socket = lambda: None
async def register(websocket):
print('register event received')
websocket_connections.add(websocket) # Add this client's socket
global_socket = websocket
async def poll_log():
await asyncio.sleep(0.3) # Settle
while True:
await asyncio.sleep(0.3) # Slow things down
# Send a dynamic message to the client after random delay
r = random.randint(1, 10)
if (r == 5): # Only send 10% of the time
a_msg = "srv -> cli: " + str(random.randint(1,10000))
print("sending msg: " + a_msg)
websockets.broadcast(websocket_connections, a_msg) # Send to all connected clients
async def main():
sock_server = websockets.serve(register, sock_url, sock_port)
await asyncio.sleep(0.3) # Start up time
async with sock_server: await poll_log()
if __name__ == "__main__":
print("Websockets server starting up ...")
asyncio.run(main())
There is a very helpful example of a complete, full-duplex Websockets application here.
That example is part of the Websockets 10.4 documentation.
It's very helpful as a reference and to understand how websockets are used.
Bottom line up front: The Paho MQTT client sucessfully connects to my Mosquitto broker, but immediately disconnects when I try to publish a message or when it receives a message from a topic it's subscribed to. I've tried changing Mosquitto's listening port and authentication settings, and using two different versions of Paho MQTT, and I still have the same problem.
Now let's get into detail.
Intro: I'm making a dashboard for some facial recognition devices that communicate through MQTT. I set up a Mosquitto broker and I've had no problems connecting to it and communicating with the devices using the Paho MQTT client for Python (I made a kind of server to sync the device's info to a database). Now I'm making the web interface, so I added a WebSockets listener to my mosquitto.conf and wrote a script using the Paho MQTT library for Javascript to connect to it, subscribe to topic sgdrf/out, send a simple JSON message to topic sgdrf/in to get the list of online devices, and process the response of the Python server once it arrives.
Problem and attempted solutions: I ran the Django server, loaded the web page and opened the JS console to find that the MQTT client successfully connected to the broker but immediately disconnected when it tried to publish the message to topic sgdrf/in. Here's each line of console output with their explanations:
The message produced by the onSuccess function, which indicates that the client successfully connected to the Mosquitto broker:
Conexión exitosa al broker MQTT.
In the onConnected function, I added console.log(uri) to see the URI used by the client to connect to the broker. I got:
ws://localhost:61613/
After printing uri to console, I made the client subscribe to sgdrf/out and then print 'subscribed' to console:
subscribed
Then I call get_online_devices(mqtt_client), a function which creates a simple JSON string and publishes it to the topic sgdrf/in. But first, it prints the strign to the console so that I can check it (just in case):
{"operator":"GetOnlineDevices","messageId":96792535859850080000,"info":{}}
Then, when the publish method is actually executed, is when I get this error (captured by the onConnectionLost function):
Pérdida de conexión con el broker MQTT: AMQJS0005E Internal error. Error Message: message is not defined, Stack trace: No Error Stack Available (código: 5)
I checked the Mosquitto log file and it only says when a new client was connected and then when it was disconnected because of a socket error (each time for every page reload). Tail of /var/log/mosquitto/mosquitto.log:
1614796149: New connection from 127.0.0.1 on port 61612.
1614796149: New client connected from 127.0.0.1 as mqttx_53195902 (p2, c1, k60, u'admin').
1614796182: Socket error on client sgdrf_dashboard_8499, disconnecting.
1614796325: New client connected from ::ffff:127.0.0.1 as sgdrf_dashboard_1597 (p2, c1, k60, u'admin').
1614796325: Socket error on client sgdrf_dashboard_1597, disconnecting.
1614796336: New client connected from ::ffff:127.0.0.1 as sgdrf_dashboard_6565 (p2, c1, k60, u'admin').
1614796336: Socket error on client sgdrf_dashboard_6565, disconnecting.
1614796931: New client connected from ::ffff:127.0.0.1 as sgdrf_dashboard_9773 (p2, c1, k60, u'admin').
1614796931: Socket error on client sgdrf_dashboard_9773, disconnecting.
1614797168: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
I tried changing the listening port in mosquitto.conf, and enabling and disabling authentication, but it changes nothing. And obviously I've had to restart Mosquito every time I changed the config file. I don't think the problem is Mosquitto.
I have the same problem whether I use Paho MQTT version 1.1.0 or 1.0.3.
As an experiment, I commented out the call to get_online_devices in my Javascript so that it doesn't try to publish anything, reloaded the page and there was no error, as expected. Then, I used MQTTX to send a JSON message to the sgdrf/out topic to which the MQTT JS client is subscribed to, and it immediately disconnected with the same error message.
Code: At the bottom of the page (index.html) I have the following code (the original code has Django template tags to fill in some values, so this is the actual code received by the browser):
<!-- Paho MQTT -->
<script src="/static/js/paho-mqtt-min.js"></script>
<!-- Scripts -->
<script src="/static/js/dashboard.js"></script>
<script>
function get_online_devices(mqtt_client) {
cmd = {
operator: "GetOnlineDevices",
messageId: generate_random_number_n_exp(20),
info: {}
};
payload_string = JSON.stringify(cmd);
console.log(payload_string)
mqtt_client.publish("sgdrf/in", payload_string);
}
function add_device_to_list(device) {
// Omitted for brevity. It's not being used yet.
}
let mqtt_client = make_mqtt_client("localhost", 61613);
let connection_options = make_connection_options(
"admin",
"CENSORED_PASSWORD"
);
mqtt_client.onConnected = function(reconnect, uri) {
console.log(uri)
mqtt_client.subscribe("sgdrf/out");
console.log('subscribed');
get_online_devices(mqtt_client);
};
mqtt_client.onConnectionLost = mqtt_client_on_connection_lost;
mqtt_client.onMessageDelivered = mqtt_client_on_message_delivered;
mqtt_client.onMessageArrived = function (msg) {
// Omitted for brevity. Checks if the payload is a
// JSON object with the right data and calls
// add_device_to_list for each item of a list in it.
};
$(document).ready(function() {
mqtt_client.connect(connection_options);
$("#reload-device-list-btn").click(function() {
get_online_devices(mqtt_client);
});
});
</script>
The dashboard.js files mentioned above just has some functions that I think will be useful for other pages, so I separated them to a file:
// dashboard.js
function generate_random_number_n_exp(n) {
return parseInt(Math.random() * Math.pow(10, n), 10)
}
function make_mqtt_client(host, port) {
let client_id = "sgdrf_dashboard_" + generate_random_number_n_exp(4);
return new Paho.Client(host, port, '/', client_id);
}
function make_connection_options(user, password) {
let connection_options = {
userName: user,
password: password,
onSuccess: mqtt_client_on_success,
onFailure: mqtt_client_on_failure,
};
return connection_options;
}
function mqtt_client_on_success() {
console.log('Conexión exitosa al broker MQTT.');
}
function mqtt_client_on_failure(error) {
console.log(
'Fallo de conexión con el broker MQTT: ' + error.errorMessage
+ ' (código: ' + error.errorCode + ')'
);
}
function mqtt_client_on_connection_lost (error) {
console.log('Pérdida de conexión con el broker MQTT: ' + error.errorMessage
+ ' (código: ' + error.errorCode + ')'
);
}
function mqtt_client_on_message_delivered(msg) {
let topic = message.destinationName;
let payload = message.payloadString;
console.log("Mensaje enviado a " + topic + ": " + payload);
}
function mqtt_client_on_message_arrived(msg) {
let topic = message.destinationName;
let payload = message.payloadString;
console.log("Mensaje recibido de " + topic + ": " + payload);
}
Here are the contents of my mosquitto.conf file:
per_listener_settings true
listener 61612
allow_anonymous false
password_file /home/s8a/Projects/sgdrf/config/pwdfile.txt
listener 61613
protocol websockets
allow_anonymous false
password_file /home/s8a/Projects/sgdrf/config/pwdfile.txt
It just sets up a TCP listener and a WebSockets listener, both disallow anonymous connections, and authenticate using a pwdfile. As I said before, I have enabled and disabled anonymous connections, and changed the port number to 9001 and back to 61613, and I still have the same error.
Conclusion: I don't know what to do and this project's deadline is next week.
I feel kinda stupid, because it was really a trivial typing mistake. The problem is that the onMessageDelivered and onMessageArrived functions have msg as argument, but I wrote messagein the function body for some reason. That's what the "message is not defined" error meant, message is literally not defined. Anyway I fixed that and now it sends and receives messages without problems.
...
More detailed story: What was not trivial is how I figured it out.
I decided to get my hands dirty and opened the non-minified version of paho-mqtt.js. I looked for "Invalid error" and found where the error constant is defined, and two places where it's used in a catch block. In both catch blocks I noticed that there was a ternary operator checking if (error.hasOwnProperty("stack") == "undefined") but the true and false clauses were inverted, which is why I was getting "No Error Stack Available".
So I inverted the clauses, and indeed I got a stack trace in the console (maybe I should file a bug report to the Paho dev team when I can). The stack trace had my mqtt_client_on_message_delivered function right at the top, so I read it again and suddenly everything made sense. Then I felt stupid for wasting an afternoon on this.
I am trying to make a connection from my javascript client in a browser to my python server and to send data from the server to the client, but when trying to connect I get the error: failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
I tried using wss instead of ws.
I also tried using socket.io instead of websocket, which wasn't helping either. I don't remember all the things I tried but none of them helped.
Javascript client:
var conn = new WebSocket('ws://localhost:2001');
conn.onmessage = function (event) {
console.log(event.data);
Python server:
# Echo server program
import socket
HOST = 'localhost'
PORT = 2001
def sendSensorData(data):
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(HOST, PORT)
s.listen(1)
conn, addr = s.accept()
with conn:
print('Connected by', addr)
while True:
print(data)
message = str(data).encode()
try:
# if not data: break
conn.sendall(message)
except BaseException as e:
print (e)
Edit: This is the error from python:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "/Users/joep/Documents/GitHub/Walnut-installation-web-app/testAPI/workers/sensorStream.py", line 26, in sensorStream
sendSensorData.sendSensorData(data)
File "/Users/joep/Documents/GitHub/Walnut-installation-web-app/testAPI/functions/sendSensorData.py", line 19, in sendSensorData
conn.send(message)
BrokenPipeError: [Errno 32] Broken pipe
Hopefully it is a simple trick to fix it.
I would like to fiddle with websockets a bit. I installed a Ruby gem called "websocket-ruby" (https://github.com/imanel/websocket-ruby) I started a pry / IRB session and typed:
require "websocket"
#handshake = WebSocket::Handshake::Server.new(:host => "localhost", :port => 8080,:secure=>true)
This starts a websocket server as far as I know. Then I opened in my browser the Javascript HTML page which attempt to connect to the server:
<!doctype html>
<html lang="en">
<head>
<title>Websocket Client</title>
</head>
<body>
<script>
var exampleSocket = new WebSocket("wss://localhost:8080");
exampleSocket.onopen = function (event) {
exampleSocket.send("Can you hear me?");
};
exampleSocket.onmessage = function (event) {
console.log(event.data);
}
</script>
</body>
</html>
But it says in the console log:
failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
I tried different ports both in server and in the client respectively: 8081, 12345, but I always get this error message.
I have some idea about websocket and javascript, but not websocket-ruby.
I hope it will helpful you.
In nodejs.. server.js file, write below code
var WebSocketServer = require("ws").Server;
var wss = new WebSocketServer({port:8100});
console.log("websocket Server is Running...");
wss.on('connection', function connection(ws) {
// Store the remote systems IP address as "remoteIp".
var remoteIp = ws.upgradeReq.connection.remoteAddress;
// Print a log with the IP of the client that connected.
console.log('Connection received: ', remoteIp);
// Add a listener which listens for the "message" event.
// When a "message" event is received, take the contents
// of the message and pass it to the broadcast() function.
ws.on('message', wss.broadcast);
});
wss.broadcast = function(msg) {
wss.clients.forEach(function each(client) {
client.send(msg);
})
};
In javascript...
var SERVER_URL = 'ws://localhost:8100';
//instead of localhost you can also use IP address of your system
var ws;
function connect() {
alert('connect');
ws = new WebSocket(SERVER_URL, []);
// Set the function to be called when a message is received.
ws.onmessage = handleMessageReceived;
// Set the function to be called when we have connected to the server.
ws.onopen = handleConnected;
// Set the function to be called when an error occurs.
ws.onerror = handleError;
}
function handleMessageReceived(data) {
// Simply call logMessage(), passing the received data.
logMessage(data.data);
}
function handleConnected(data) {
// Create a log message which explains what has happened and includes
// the url we have connected too.
var logMsg = 'Connected to server: ' + data.target.url;
// Add the message to the log.
logMessage(logMsg)
}
function handleError(err) {
// Print the error to the console so we can debug it.
console.log("Error: ", err);
}
function logMessage(msg) {
// with the new message.
console.log(msg);
}
/** This is the scope function that is called when a users hits send. */
function sendMessage{
ws.send(msg);
};
connect();
in html use one button to send message to websocket server
<button onclick="sendMessage('Hi Websocket')">send message</button>
To the best of my knowledge, the Ruby code you presented does not start a Websocket server... what it does is initiate a server-side parser.
To start a server you need to use an actual websocket server.
ActionCable (with Rails) uses the websocket-ruby library to parse websocket events and it uses nio4r to operate the actual server.
Faye have a similar solution and em-websockets use the websocket-ruby gem with EventMachine.
Other Ruby Websocket servers include Iodine, which uses the C library facil.io. Iodine is used by the framework plezi as well as independently.
Since you were trying to run an echo server, here's a quick example using the Plezi framework (you can use it as middleware in Sinatra or Rails)...
...place the following in a config.ru file:
require 'plezi'
class WebsocketSample
# HTTP index
def index
'Hello World!'
end
# called when Websocket data is recieved
#
# data is a string that contains binary or UTF8 (message dependent) data.
def on_message(data)
puts "Websocket got: #{data}"
write data
end
end
Plezi.route '/', WebsocketSample
run Plezi.app
To run the server, call (ignore the $ sign, it marks this code as terminal code):
$ iodine
notice: Iodine requires a BSD / Unix / Linux machine, such as macOS, Ubuntu, etc'. It won't work on windows.
Using the following code I can create a Socket Server in my Raspberry Pi which works great if accessed via Socket clients like Android apps.
However, I would like to integrate websocket functionality to my website so I started trying to send a simple message via HTML textbox that the python script will receive and answer back.
The problem is I cant get the HTML code to open, send and keep open the socket for communication. I do acknowledge the html client connecting to python but cant get data as it seems the connection closes.
Python code
#!/usr/bin/env python
#python3
# https://pythonprogramming.net/client-server-python-sockets/
import socket # Websocket
import sys #
from _thread import * # Used for multi-threading The thread module has been renamed to _thread in Python 3.
import time # Used to create delays
# ******* WEBSOCKET VARIABLES *******
numberClients = 0
host = ''
PORT = 2223
# ******* WEBSOCKET VARIABLES *******
# ************************** FUNCTIONS **************************
def threaded_client(conn,address): # receive as parameters, the connection object (conn), and the address object that contains the ip and port
global numberClients
conn.send(str.encode('Welcome, type your info\n')) # data should be bytes
numberClients = numberClients + 1
# CHECK USER USING PASSWORD OR SOMETHING
if ("192.168" in str(address[0])):
print (" VALID CLIENT!!")
while True:
data = conn.recv(2048)
if (data):
reply = "" + 'Server output: '+ data.decode('utf-8').rstrip() + "\n"
print(str(address[0]) + " - Clients(" + str(numberClients) + ") -> Data received: >" + data.decode('utf-8').rstrip() + "<")
if not data:
#print("no data")
#break
foo = 2
try:
conn.sendall(str.encode(reply)) # data should be bytes
except Exception as e:
foo = 1
print("Thread connection closed by client: " + address[0])
conn.close()
numberClients = numberClients - 1
else:
print (" INVALID CLIENT -> Thread connection closed by USER VALIDATION: " + address[0])
conn.close()
numberClients = numberClients - 1
# ************************** FUNCTIONS **************************
# ************************** SETUP **************************
print ("\n----------- Starting Websocket Python Program -----------\n")
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # "s" here is being returned a "socket descriptor" by socket.socket.
print(s)
# we are simply attempeting to bind a socket locally, on PORT 5555.
try:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) # reuse the port number (in case we just got an error and port was not freed)
s.bind((host, PORT)) # server side - take IN connections
print ("Server started on port " + str(PORT))
except socket.error as e:
print(str(e))
print('Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1])
#sys.exit()
print('Socket bind complete')
s.listen(5) # the "5" stands for how many incoming connections we're willing to queue before denying any more.
print('Waiting for a connection.')
# ************************** SETUP **************************
# ************************** MAIN LOOP **************************
while True:
conn, addr = s.accept() # code will stop here whilst waiting for a new connection. Old connections will be running in the threads
print('Connected to: '+addr[0]+':'+str(addr[1]))
start_new_thread(threaded_client,(conn,addr))
# ************************** MAIN LOOP **************************
One of the many HTML codes I have tried:
<script type="text/javascript">
function WebSocketTest()
{
if ("WebSocket" in window)
{
alert("WebSocket is supported by your Browser!");
// Let us open a web socket
var ws = new WebSocket("ws://192.168.1.20:5252/echo");
ws.onopen = function()
{
// Web Socket is connected, send data using send()
ws.send("133:L1");
alert("Message is sent...");
};
ws.onmessage = function (evt)
{
var received_msg = evt.data;
alert("Message is received...");
};
ws.onclose = function()
{
// websocket is closed.
alert("Connection is closed...");
};
}
else
{
// The browser doesn't support WebSocket
alert("WebSocket NOT supported by your Browser!");
}
}
</script>
</head> <body>
<div id="sse">
Run WebSocket
</div>
</body> </html>
As you can see things which are working:
- More than one client can connect
- Clients connecting from android app can send and receive messages and keep connection open
- html client is accepted but no message is send
That's not a Websocket application you create with Python but a Socket application. Websocket is a protocol on top of HTTP which sits on top of TCP which are the actual sockets you've used in your Python application. To create a Websockets server with python you might try the websockets library.
For more details about the difference see Difference between socket and websocket? or Differences between TCP sockets and web sockets, one more time for the difference. For server code see https://stackoverflow.com/questions/5839054/websocket-server-in-python.