Develop Reference

How can I emit Flask-SocketIO requests with callbacks that still work after a user rejoins and their sid changes?

Summarize the Problem
I am using Flask-SocketIO for a project and am basically trying to make it so that users can rejoin a room and "pick up where they left off." To be more specific:
The server emits a request to the client, with a callback to process the response and a timeout of 1 second. This is done in a loop so that the request is resent if a user rejoins the room.
A user "rejoining" a room is defined as a user joining a room with the same name as a user who has previously been disconnected from that room. The user is given their new SID in this case and the request to the client is sent to the new SID.
What I am seeing is this:
If the user joins the room and does everything normally, the callback is processed correctly on the server.
It a user rejoins the room while the server is sending requests and then submits a response, everything on the JavaScript side works fine, the server receives an ack but does not actually run the callback that it is supposed to:
uV7BTVtBXwQ6oopnAAAE: Received packet MESSAGE data 313["#000000"]
received ack from Ac8wmpy2lK-kTQL7AAAF [/]
This question is similar to mine but the solution for them was to update Flask-SocketIO and I am running a version newer than theirs: python flask-socketio server receives message but doesn't trigger event
Show Some Code
I have created a repository with a "minimal" example here: https://github.com/eshapiro42/socketio-example.
In case something happens to that link in the future, here are the relevant bits:
# app.py
from gevent import monkey
monkey.patch_all()
import flask_socketio
from collections import defaultdict
from flask import Flask, request, send_from_directory
from user import User
app = Flask(__name__)
socketio = flask_socketio.SocketIO(app, async_mode="gevent", logger=True, engineio_logger=True)
#app.route("/")
def base():
return send_from_directory("static", "index.html")
#app.route("/<path:path>")
def home(path):
return send_from_directory("static", path)
# Global dictionary of users, indexed by room
connected_users = defaultdict(list)
# Global dictionary of disconnected users, indexed by room
disconnected_users = defaultdict(list)
#socketio.on("join room")
def join_room(data):
sid = request.sid
username = data["username"]
room = data["room"]
flask_socketio.join_room(room)
# If the user is rejoining, change their sid
for room, users in disconnected_users.items():
for user in users:
if user.name == username:
socketio.send(f"{username} has rejoined the room.", room=room)
user.sid = sid
# Add the user back to the connected users list
connected_users[room].append(user)
# Remove the user from the disconnected list
disconnected_users[room].remove(user)
return True
# If the user is new, create a new user
socketio.send(f"{username} has joined the room.", room=room)
user = User(username, socketio, room, sid)
connected_users[room].append(user)
return True
#socketio.on("disconnect")
def disconnect():
sid = request.sid
# Find the room and user with this sid
user_found = False
for room, users in connected_users.items():
for user in users:
if user.sid == sid:
user_found = True
break
if user_found:
break
# If a matching user was not found, do nothing
if not user_found:
return
room = user.room
socketio.send(f"{user.name} has left the room.", room=room)
# Remove the user from the room
connected_users[room].remove(user)
# Add the user to the disconnected list
disconnected_users[room].append(user)
flask_socketio.leave_room(room)
#socketio.on("collect colors")
def collect_colors(data):
room = data["room"]
for user in connected_users[room]:
color = user.call("send color", data)
print(f"{user.name}'s color is {color}.")
if __name__ == "__main__":
socketio.run(app, debug=True)
# user.py
from threading import Event # Monkey patched
class User:
def __init__(self, name, socketio, room, sid):
self.name = name
self.socketio = socketio
self.room = room
self._sid = sid
#property
def sid(self):
return self._sid
#sid.setter
def sid(self, new_sid):
self._sid = new_sid
def call(self, event_name, data):
"""
Send a request to the player and wait for a response.
"""
event = Event()
response = None
# Create callback to run when a response is received
def ack(response_data):
print("WHY DOES THIS NOT RUN AFTER A REJOIN?")
nonlocal event
nonlocal response
response = response_data
event.set()
# Try in a loop with a one second timeout in case an event gets missed or a network error occurs
tries = 0
while True:
# Send request
self.socketio.emit(
event_name,
data,
to=self.sid,
callback=ack,
)
# Wait for response
if event.wait(1):
# Response was received
break
tries += 1
if tries % 10 == 0:
print(f"Still waiting for input after {tries} seconds")
return response
// static/client.js
var socket = io.connect();
var username = null;
var room = null;
var joined = false;
var colorCallback = null;
function joinedRoom(success) {
if (success) {
joined = true;
$("#joinForm").hide();
$("#collectColorsButton").show();
$("#gameRoom").text(`Room: ${room}`);
}
}
socket.on("connect", () => {
console.log("You are connected to the server.");
});
socket.on("connect_error", (data) => {
console.log(`Unable to connect to the server: ${data}.`);
});
socket.on("disconnect", () => {
console.log("You have been disconnected from the server.");
});
socket.on("message", (data) => {
console.log(data);
});
socket.on("send color", (data, callback) => {
$("#collectColorsButton").hide();
$("#colorForm").show();
console.log(`Callback set to ${callback}`);
colorCallback = callback;
});
$("#joinForm").on("submit", (event) => {
event.preventDefault();
username = $("#usernameInput").val();
room = $("#roomInput").val()
socket.emit("join room", {username: username, room: room}, joinedRoom);
});
$("#colorForm").on("submit", (event) => {
event.preventDefault();
var color = $("#colorInput").val();
$("#colorForm").hide();
colorCallback(color);
});
$("#collectColorsButton").on("click", () => {
socket.emit("collect colors", {username: username, room: room});
});
<!-- static/index.html -->
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Socket.IO Example</title>
</head>
<body>
<p id="gameRoom"></p>
<form id="joinForm">
<input id="usernameInput" type="text" placeholder="Your Name" autocomplete="off" required>
<input id="roomInput" type="text" placeholder="Room ID" autocomplete="off" required>
<button id="joinGameSubmitButton" type="submit" btn btn-dark">Join Room</button>
</form>
<button id="collectColorsButton" style="display: none;">Collect Colors</button>
<form id="colorForm" style="display: none;">
<p>Please select a color.</p>
<input id="colorInput" type="color" required>
<button id="colorSubmitButton" type="submit">Send Color</button>
</form>
<script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8=" crossorigin="anonymous"></script>
<script src="https://cdn.socket.io/4.4.1/socket.io.min.js" integrity="sha384-fKnu0iswBIqkjxrhQCTZ7qlLHOFEgNkRmK2vaO/LbTZSXdJfAu6ewRBdwHPhBo/H" crossorigin="anonymous"></script>
<script src="client.js"></script>
</body>
</html>
Edit
Steps to Reproduce
Start the server python app.py and visit localhost:5000 in your browser.
Enter any username and Room ID and click "Join Room."
Click "Collect Colors."
Select a color and click "Send." The selector should disappear and the server should print out a confirmation.
Reload everything.
Repeat steps 2 and 3 and copy the Room ID.
Exit the page and then navigate back to it.
Enter the same username and Room ID as you did in step 6 and click "Join Room."
Select a color and click "Send." The selector disappears briefly but then comes back, since the server did not correctly process the response and keeps sending requests instead.
Edit 2
I managed to work around (not solve) the problem by adding more state variables on the server side and implementing a few more events to avoid using callbacks entirely. I would still love to know what was going wrong with the callback-based approach though since using that seems cleaner to me.

The reason why those callbacks do not work is that you are making the emits from a context that is based on the old and disconnected socket.
The callback is associated with the socket identified by request.sid. Associating the callback with a socket allows Flask-SocketIO to install the correct app and request contexts when the callback is invoked.
The way that you coded your color prompt is not great, because you have a long running event handler that continues to run after the client goes aways and reconnects on a different socket. A better design would be for the client to send the selected color in its own event instead of as a callback response to the server.

How can I send object by Node.js to browser in Real Time

I am working on a project in Node.js and Asterisk AMI
Asterisk sends events to Node.js server and Node.js sends events in Obj to browser
My problem is the node server does not send a new object when updating data from AMI
var AsteriskAmi = require('asterisk-ami');
var ami = new AsteriskAmi( { host: '127.0.0.1', username: 'admin', password: '123456' } );
var Send;
ami.on('ami_data', function(data){
// this obj update when ami send event
Send = data;
});
ami.connect(function(){
});
var http = require('http');
var app = http.createServer(function(req,res) {
res.setHeader('Content-Type': 'text/plain');
//my problem here , object was sent but not update when ami send other object
res.end(Send);
});
app.listen(3000);

1. res.end(Send)
This is your first issue. When somebody is visiting your host, you are sending hat was collected as last message and you are sending it to browser. And closing connection. For real time event transportation to browser you should use socket based mechanisms like socket.io.
To connect to this server you will have to use an html page.
Full working version in my environment - using asterisk-manager
In public_html/test/index.html:
<html>
<body>
<pre id="console">
</pre>
<script src="http://code.jquery.com/jquery.min.js"></script>
<script src="https://cdn.socket.io/socket.io-1.2.1.js"></script>
<script>
var __url = 'http://localhost';
var __port = 3001;
__socketurl = __url + ':' + __port;
socket = io.connect(__socketurl);
socket.on('notification', function (data) {
document.getElementById('console').innerHTML = JSON.stringify(data) + "\r\n";
});
socket.on('ami_event', function (data) {
document.getElementById('console').innerHTML += JSON.stringify(data) + "\r\n";
});
</script>
</body>
</html>
Above code tries to connect to server at localhost:3001 and prints out every event coming from node app.
In public_html/test/node/app.js:
var app = require('http').createServer().listen(3001);
var io = require('socket.io').listen(app);
var AsteriskAmi = require('asterisk-manager');
var ami = new require('asterisk-manager')('25038','192.168.0.2','amilogin','amipassword', true);
io.sockets.on('connection', function(socket) {
socket.emit('notification', {message: "connected"});
});
ami.on('managerevent', function(data) {
console.log(data);
io.sockets.emit('ami_event', data);
});
ami.connect(function(){
});
Above code opens a socket at port 3001 and keeps it until user diconnects. Please refer to manual how to create IO servers here. Demo tutorias are available there also. Script does also connect to AMI itself and passes every event from AMI to every socket connected via socket.io.
Packages in public_html/test/node/packages.json:
{
"name": "Server",
"version": "0.0.1",
"private": true,
"dependencies": {
"socket.io": "latest",
"asterisk-manager": "latest"
}
}
Filled proper data in app.js to log into AMI, npm install'ed, node app.js and got it working flawlessly.

receive data from flash and display on angularJS client side

I am trying to get data (a user's score), from an extremely simple flash game I made, to be displayed on a simple leader board which is displayed through AngularJS. You can get a copy of all of the code here (you might need to run npm install to get it to work). I am using NodeJS/Express/Socket.io to transfer the data from the game.
Here is the code from app.js (server side):
var express = require('express'),
app = express(),
server = require('http').createServer(app),
io = require('socket.io').listen(server);
app.configure(function() {
app.use(express.static(__dirname + '/public'));
app.set('views', __dirname + '/views');
});
io.configure(function() {
io.set('transports', ['websocket','xhr-polling']);
io.set('flash policy port', 10843);
});
var contestants = [];
io.sockets.on('connection', function(socket) {
socket.on('data', function (data) {
socket.broadcast.emit(data);
});
socket.on('listContestants', function(data) {
socket.emit('onContestantsListed', contestants);
});
socket.on('createContestant', function(data) {
contestants.push(data);
socket.broadcast.emit('onContestantCreated', data);
});
socket.on('updateContestant', function(data){
contestants.forEach(function(person){
if (person.id === data.id) {
person.display_name = data.display_name;
person.score = data.score;
}
});
socket.broadcast.emit('onContestantUpdated', data);
});
socket.on('deleteContestant', function(data){
contestants = contestants.filter(function(person) {
return person.id !== data.id;
});
socket.broadcast.emit('onContestantDeleted', data);
});
});
server.listen(8000);
The key lines from above are:
socket.on('data', function (data) {
socket.broadcast.emit(data);
});
That is where I am trying to send the data from the server side to the client side. On the client side - from within my main controller, I have this.
leader-board.js (main client side javascript file):
socket.on('data', function(data) {
$scope.score.push(data);
})
// Outgoing
$scope.createContestant = function() {
$scope.$digest;
console.log($scope.score[0]);
var contestant = {
id: new Date().getTime(),
display_name: "Bob",
score: Number($scope.score[0])
};
$scope.contestants.push(contestant);
socket.emit('createContestant', contestant);
_resetFormValidation();
};
As you can see - I am trying to get the emitted data, and push it to an array where I will keep the scores. The createContestant function gets called when the user clicks a submit button from within the main index.html file.
index.html
<body>
...
<button ng-click="createContestant()" class="btn btn-success"
ng-disabled="
ldrbd.contestantName.$error.required ||
ldrbd.contestantScore.$error.required
"
>
Submit Score
</button>
...
</body>
The line console.log($scope.score[0]);, from within the createContestant function, is always undefined. I am not sure if I am emitting the data correctly from the server side with socket.io - and I am not sure if I am receiving it correctly either. I use $scope.$digest to refresh the scope because the socket.io stuff is outside of AngularJS (or so I have read). Any help would be greatly appreciated. Again - I am trying to store data emitted from a flash game into an array, however - before the data is stored, it needs to be fetched correctly - and my fetch always turns up undefined, when it should be retrieving a number which is being emitted from the game (I know that I am emitting the number from the game because I have tested it with log messages). Thanks!
UPDATE
Changed server side code to this:
socket.on('message', function (data) {
console.log(data)
score = data;
socket.emit('score', score);
})
...and client side to this:
socket.on('score', function(data) {
console.log(data);
$scope.score = data;
});
Still no luck - but I added the console.log message to the server side to confirm that the data was getting sent and received (at least by node) and it is - the output of that message is a number which is the score. The thing I am realizing is...the score is supposed to be input on the client side when the button is clicked. But the data gets emitted from the server side when the game is over...so when the button is clicked...is the data available to the client side in that moment? Is this the discrepancy?

Here is the working socket code (took me a while but I got it)!
Server side (Node/Express):
socket.on('message', function (data) {
console.log(data);
score = data;
console.log("Transfered:" + " " + score);
//
})
socket.on('score', function() {
socket.emit('sendscore', score);
})
Client side (AngularJS)
socket.on('sendscore', function(data) {
console.log(data);
$scope.score = data;
});
// Outgoing
$scope.createContestant = function() {
socket.emit('score')
//$scope.$digest;
//console.log($scope.score[0]);
var contestant = {
id: new Date().getTime(),
display_name: "Bob",
score: $scope.score
};
$scope.contestants.push(contestant);
socket.emit('createContestant', contestant);
_resetFormValidation();
};
The link in the question still works for the code if you want to try it yourself!

include socket.io server into my html

I have run my nodeServer and Client sucessfully. But I want to connect its server to a proxy.
Here is the node client:
<html>
<head>
<script src="http://localhost:8000/socket.io/socket.io.js"></script>
<script src="http://code.jquery.com/jquery-1.6.2.min.js"></script>
<script>
var name = '';
var socket = io.connect('http://localhost:8000');
// at document read (runs only ones).
$(document).ready(function(){
// on click of the button (jquery thing)
// the things inside this clause happen only when
// the button is clicked.
$("button").click(function(){
// just some simple logging
$("p#log").html('sent message: ' + $("input#msg").val());
// send message on inputbox to server
socket.emit('chat', $("input#msg").val() );
// the server will recieve the message,
// then maybe do some processing, then it will
// broadcast it again. however, it will not
// send it to the original sender. the sender
// will be the browser that sends the msg.
// other browsers listening to the server will
// recieve the emitted message. therefore we will
// need to manually print this msg for the sender.
$("p#data_recieved").append("<br />\r\n" + name + ': ' + $("input#msg").val());
// then we empty the text on the input box.
$("input#msg").val('');
});
// ask for the name of the user, ask again if no name.
while (name == '') {
name = prompt("What's your name?","");
}
// send the name to the server, and the server's
// register wait will recieve this.
socket.emit('register', name );
});
// listen for chat event and recieve data
socket.on('chat', function (data) {
// print data (jquery thing)
$("p#data_recieved").append("<br />\r\n" + data.msgr + ': ' + data.msg);
// we log this event for fun :D
$("p#log").html('got message: ' + data.msg);
});
</script>
</head>
<body>
<input type="text" id="msg"></input><button>Click me</button>
<p id="log"></p>
<p id="data_recieved"></p>
</body>
</html>
Here is Server:
var check="check";
var io = require('socket.io').listen(8000);
// open the socket connection
io.sockets.on('connection', function (socket) {
// listen for the chat even. and will recieve
// data from the sender.
console.log('hello nahid');
socket.on('chat', function (data) {
// default value of the name of the sender.
var sender = 'unregistered';
check=sender;
// get the name of the sender
socket.get('nickname', function (err, name) {
console.log('Chat message by ', name);
console.log('error ', err);
sender = name;
});
// broadcast data recieved from the sender
// to others who are connected, but not
// from the original sender.
socket.broadcast.emit('chat', {
msg : data,
msgr : sender
});
});
// listen for user registrations
// then set the socket nickname to
socket.on('register', function (name) {
// make a nickname paramater for this socket
// and then set its value to the name recieved
// from the register even above. and then run
// the function that follows inside it.
socket.set('nickname', name, function () {
// this kind of emit will send to all! :D
io.sockets.emit('chat', {
msg : "naay nag apil2! si " + name + '!',
msgr : "mr. server"
});
});
});
});
They work well together. Now I want to have included server file into another html client like this:
<script type="text/javascript" src="nodeServer.js"></script>
In order to access variables in nodeServer.js. But The error "Uncaught ReferenceError: require is not defined "
What can I do to solve this and also I have this line code :
such that it knows what socket.io syntax is. But the error exists.
What should I do to make my html file know socket.io.js.
EDIT:
The last comment on the checked answer is what solved my issue. Thanks to #Houseman.

I'm pretty sure you can't just throw a Node.js file into a html DOM and expect it to work. Node.js files are run by Node.js, not by client-side html javascript.
This line <script src="http://localhost:8000/socket.io/socket.io.js"></script> is sufficient for loading the io object into your DOM, and make it accessible through javascript.
Remove this line:
<script type="text/javascript" src="nodeServer.js"></script>
, and you should be fine, if your server is running on port 8000. Your server code, however, doesn't seem to have the necessary components to actually run.
EDIT:
If you want to pass variables from your client to your server, or the other way around, use sockets, just like you're already doing.

WebSocket javascript client and python server. Retrieving garbage in output

I am trying to send string/text data from browser client to python server and simply print it out. I have followed several examples on the internet, and all are the same: by using javascript
web_socket.send("text to be sent")
and (python)
data = web_socket.recv(1024)
print data
they receive what they want, what is clear and nice printout "text to be sent" on server site.
You can find my .html and .py below:
<!DOCTYPE html>
<html lang="en">
<head>
<title>Test</title>
<script src="jquery.js"></script>
<script type="application/javascript">
var ws;
function init() {
var servermsg = document.getElementById("servermsg");
ws = new WebSocket("ws://127.0.0.1:9877/");
ws.onopen = function(){
servermsg.innerHTML = servermsg.innerHTML + "<br>Server connected";
};
ws.onmessage = function(e){
servermsg.innerHTML = servermsg.innerHTML + "<br><< Recieved data: " + e.data;
};
ws.onclose = function(){
servermsg.innerHTML = servermsg.innerHTML + "<br>Server disconnected";
};
}
function postmsg(){
var text = document.getElementById("message").value;
ws.send(text);
servermsg.innerHTML = servermsg.innerHTML + "<br>>> Data sent: " + text;
}
//$(function(){
// var text = document.getElementById("message").value;
// ws.send(text);
// servermsg.innerHTML = servermsg.innerHTML + "<br>Sent: " + text;
//});
</script>
</head>
<body onload="init();">
<form action="" onSubmit="postmsg();return false;">
<input type="text" name="message" value="" id="message">
<input type="submit" name="submit" value="" id="submit">
</form>
<div id="servermsg"><h1>Message log:</h1></div>
</body>
</html>
Server:
#!/usr/bin/env python
import socket
import threading
import struct
import hashlib
import base64
PORT = 9877
_address = ""
def create_handshake_resp(handshake):
final_line = ""
lines = handshake.splitlines()
for line in lines:
parts = line.partition(": ")
if parts[0] == "Sec-WebSocket-Key":
key = parts[2]
magic = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11'
accept_key = base64.b64encode(hashlib.sha1(key+magic).digest())
return (
"HTTP/1.1 101 Switching Protocols\r\n"
"Upgrade: WebSocket\r\n"
"Connection: Upgrade\r\n"
"Sec-WebSocket-Accept: " + accept_key + "\r\n\r\n")
def handle(s, addr):
data = s.recv(1024)
response = create_handshake_resp(data)
s.sendto(response, addr)
lock = threading.Lock()
while 1:
print "Waiting for data from", addr
data = s.recv(1024)
print "Done"
if not data:
print "No data"
break
print 'Data from', addr, ':', data
print 'Client closed:', addr
lock.acquire()
clients.remove(s)
lock.release()
s.close()
def start_server():
print 'STARTING SERVER...'
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', PORT))
s.listen(1)
print 'SERVER STARTED'
while 1:
conn, addr = s.accept()
print 'NEW CONNECTION ['+str(len(clients))+'], connected by ', addr
clients.append(conn)
threading.Thread(target = handle, args = (conn, addr)).start()
clients = []
start_server()
And server printout (when input was like "AA", or "ABC"):
STARTING SERVER...
SERVER STARTED
NEW CONNECTION [0], connected by ('127.0.0.1', 43877)
Waiting for data from ('127.0.0.1', 43877)
Done
Data from ('127.0.0.1', 43877) : ����w�q
Waiting for data from ('127.0.0.1', 43877)
Done
Data from ('127.0.0.1', 43877) : ��)B�h
Waiting for data from ('127.0.0.1', 43877)

I'm working on something similar myself. The Websocket protocol mandates that the client sends all its data using a mask. This is why you see 'garbage' - it's the masked text.
https://www.rfc-editor.org/rfc/rfc6455#section-5
"a client MUST mask all frames that it
sends to the server"
Read section 5 of the protocol and all will become clear. The browser (ie the client) is just implementing the protocol as it should (when you call ws.send). You need to do your bit.
Note also that when the sever sends data to the client it must NOT mask. But it still has to supply other info before the actual data (type, length etc).

To send a message from server side to websocket client you need to do as follows:
message = bytearray([0b10000001, len(original_msg)])
for byte in bytearray(original_msg):
message.append(byte)
See a stable server to client unidirectional socket library at https://github.com/westial/SingleSocket

The problem with the junk data was the javascript code sends the masked data and you must unmask it on the server side and the server sendes the unmasked data to client side.To solve this problem see my git-hub page
[][1]https://github.com/mohanbe/web-chat