I am looking into using Pubnub's service to set up WebRTC connections between peers for video.
With this I am hoping to avoid using socket io which is what I am currently using, although I just cannot find any good examples that demonstrate how to do this.
Right now socket io is handling the events emitted from the client and the server. From what I understand, the current node js server would no longer need to handle any of the emitted events since socket io would not be used but this is what I am having problems with. I am not sure how to set up the clients to signal each other the information that they require (who to connect to, etc)
Are there any simple examples or implementations of pubnub being used instead of socket io for a project or perhaps someone can shed some light on something I may not be seeing, thanks!
edit: Also with anyone experienced in Pubnub, is what I am trying to do even possible haha
WebRTC Signaling Exchanging ICE Candidates via PubNub
The goal is to exchange ICE candidate packets between two peers. ICE candidate packets are structured payloads which contain possible path recommendations between two peers. You can use a lib which will take care of the nitty gritty such as http://www.sinch.com/ and below is the general direction you want to take:
Signaling Example Code Follows
<script src="http://cdn.pubnub.com/pubnub-3.6.3.min.js"></script>
<script>(function(){
// INIT P2P Packet Exchanger
var pubnub = PUBNUB({
publish_key : 'demo',
subscribe_key : 'demo'
})
// You need to specify the exchange channel for the peers to
// exchange ICE Candidates.
var exchange_channel = "p2p-exchange";
// LISTEN FOR ICE CANDIDATES
pubnub.subscribe({
channel : exchange_channel,
message : receive_ice_candidates
})
// ICE CANDIDATES RECEIVER PROCESSOR FUNCTION
function receive_ice_candidates(ice_candidate) {
// Attempt peer connection or upgrade route if better route...
console.log(ice_candidate);
// ... RTC Peer Connection upgrade/attempt ...
}
// SEND ICE CANDIDATE
function send_ice_candidate(ice) {
pubnub.publish({
channel : exchange_channel,
message : ice
})
}
Generate ICE Candidates Example Code Follows:
// CREATE ICE CANDIDATES
var pc = new RTCPeerConnection();
navigator.getUserMedia( {video: true}, function(stream) {
pc.onaddstream({stream:stream});
pc.addStream(stream);
pc.createOffer( function(offer) {
pc.setLocalDescription(
new RTCSessionDescription(offer),
send_ice_candidate, // - SEND ICE CANDIDATE via PUBNUB
error
);
}, error );
} );
// ERROR CALLBACK
function error(e) {
console.log(e);
}
})();</script>
More fun details await - https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection
Related
I create 60 client connections to socket.io server in google chrome browser.
Server at specific time send screenshot to the clients. And some websocket connections, that are subprotocol of socket.io are broken, so connection at about 1-4 chrome tabs are closed. I tried to increase pingTimeout, it helped to overcome tcp transport close problem only (this problem I have as well), but this solution doesn't help to fix sending screenshot problem.
In my opinion google chrome can't support about 50-60 tabs at one time, because CPU and RAM are increased to the max values because of sending screenshots to 60 clients (each client has 2 websocket connection: the first for simple messages, the second for graphics (to send screenshots)), so chrome closes some websocket connections.
Part of code for the server socket io here:
// server
this.http = this._createHttpServer(sslCert, sslKey);
this.io = socketIo(this.http, {
'pingTimeout': 180000,
'pingInterval': 60000
});
const jwtAuth = socketioJwt.authorize({
secret: jwtSecret,
timeout: 15000
});
this.io.on('connection', (socket) => {
socket.once('authenticate', (data) => {
socket.rawAuthData = data;
});
jwtAuth(socket);
});
// client
var connOptions = {
"reconnectionAttempts": 2
};
var socket = io(options.url, connOptions);
socket.on('connect', function() {
if (options.token) {
socket.emit('authenticate', {token: options['token'], tag : tag});
socket.on('authenticated', function() {
ctx.printLog('Authorized. Waiting for handshake');
socket.once('tunnel-handshake', function() {
ctx.printLog('handshake received! connection is ready');
processConnected();
});
}).on('unauthorized', function(msg) {
ctx.printLog("Authorization failed: " + JSON.stringify(msg.data));
eventHandlers.onerror({ code: ctx.ERROR_CODE.INVALID_TOKEN});
});
} else {
processConnected();
}
});
socket.on('reconnect_failed', eventHandlers.onerror.bind(this, {code: 1, reason: "Reconnection failed"}));
socket.on('disconnect', eventHandlers.onclose);
socket.on('error', eventHandlers.onerror);
Does exist any ideas, what the cause could be? Does exist any solution of this problem?
Is it google chrome problem or socket.io options problem?
Thanks
Changing socket.io to 3.0 version can't resolve the issue. socket.io v3.0 has engine.io v4.0. Next information in release notes of engine.io v4.0 describes the problem ("Heartbeat mechanism reversal" title):
We have received a lot of reports from users that experience random disconnects due to ping timeout, even though their Internet connection is up and the remote server is reachable. It should be noted that in that case the client reconnects right away, but still it was an annoying issue.
After analysis, it seems to be caused by delayed timers on the
client-side. Those timers are used in the ping-pong mechanism which
helps to ensure the connection between the server and the client is
still healthy. A delay on the client-side meant the client sent the
ping packet too late, and the server considered that the connection
was closed.
That’s why the ping packets will now be sent by the server, and the
client will respond with a pong packet.
But increasing pingTimeout and pingInterval to 1073741823 value resolves the issue.
websocketServer.on('connection', function(socket, req) {
socket.on('message', onMessage);
sub.subscribe('chat'); // sub: Redis subscription connection
sub.on('message', onSubMessage);
});
function onMessage(message) {
pub.publish('chat', message); // pub: Redis publishing connection
}
function onSubMessage(channel, message) {
// how to access 'socket' from here?
if (channel === 'chat') socket.send(message);
}
I'm trying to get away with as few state & bindings as possible, to make WS server efficient & to have the ability to just add more machines if I need to scale - state would only make this harder. Im still not understanding everything about Node memory management & garbage collection.
What would be the recommended solution in this case? Move onSubMessage into connection callback to access socket? But function would be then initialized on every connection?
What other choices do I have?
Little background about this:
The user opens a WebSocket connection with the server. If the user sends a message, it gets sent to Redis channel (might also know it as Pub/Sub topic) which broadcasts it to every subscribed client (onSubMessage). Redis Pub/Sub acts as a centralized broadcaster: I don't have to worry about different servers or state, Redis sends a message to everybody who is interested. Carefree scaling.
You can use bind() to pre-define an extra argument to the callback function:
...
sub.on('message', onSubMessage.bind(sub, socket));
...
function onSubMessage(socket, channel, message) {
if (channel === 'chat') socket.send(message);
}
This does create a new function instance for every new connection, so there is probably not a real upside to using a wrapping function in terms of memory usage.
I can't figure out how to debug WebRTC. I keep getting 'ICE Failed' errors, but I doubt that's the issue. Here's my code: https://github.com/wamoyo/webrtc-cafe/tree/master/2.1%20Establishing%20a%20Connection%20%28within%20a%20Local%20Area%20Network%29
I'm using node.js/express/socket.io for setting up rooms and connecting peers, and then some default public servers for signalling.
The strange thing is, it appears the I have the remoteStream on the client.
Here's the two errors I'm getting (By they way, for now, I'm just trying to connect form my phone to laptop or two browser tabs, all within a LAN):
HTTP "Content-Type" of "text/html" is not supported. Load of media resource http://192.168.1.2:3000/%5Bobject%20MediaStream%5D failed.
ICE failed, see about:webrtc for more details
Any help would rock!
I've made a few comments already, but I think it's also worthwhile to write an answer.
There are 3 big things I see after my first quick read of your code. I haven't tried to actually run or debug your code beyond a superficial reading.
First, you should set the remoteVideo.src URL parameter in the same way as you do the local video stream:
pc.onaddstream = function(media) { // This function runs when the remote stream is added.
console.log(media);
remoteVideo.src = window.URL.createObjectURL(media.stream);
}
Second, you should pass a constraints object to the createOffer() and createAnswer() methods of RTCPeerConnection. The constraints should/could look like this:
var constraints = {
mandatory: {
OfferToReceiveAudio: true,
OfferToReceiveVideo: true
}
};
And you pass this after the success and error callback arguments:
pc.createOffer(..., ..., constraints);
and:
pc.createAnswer(..., ..., constraints);
Lastly, you are not exchanging ICE candidates between your peers. ICE candidates can be part of the offer/answer SDP, but not always. To ensure that you send all of them, you should implement an onicecandidate handler on the RTCPeerConnection:
pc.onicecandidate = function (event) {
if (event.candidate) {
socket.emit("ice candidate", event.candidate);
}
}
You will have to implement "ice candidate" message relaying between clients in your server.js
Hope this helps, and good luck!
I'm trying to learn how to create an RTCPeerConnection so that I can use the DataChannel API. Here's what I have tried from what I understood:
var client = new mozRTCPeerConnection;
var server = new mozRTCPeerConnection;
client.createOffer(function (description) {
client.setLocalDescription(description);
server.setRemoteDescription(description);
server.createAnswer(function (description) {
server.setLocalDescription(description);
client.setRemoteDescription(description);
var clientChannel = client.createDataChannel("chat");
var serverChannel = server.createDataChannel("chat");
clientChannel.onmessage = serverChannel.onmessage = onmessage;
clientChannel.send("Hello Server!");
serverChannel.send("Hello Client!");
function onmessage(event) {
alert(event.data);
}
});
});
I'm not sure what's going wrong, but I'm assuming that the connection is never established because no messages are being displayed.
Where do I learn more about this? I've already read the Getting Started with WebRTC - HTML5 Rocks tutorial.
I finally got it to work after sifting through a lot of articles: http://jsfiddle.net/LcQzV/
First we create the peer connections:
var media = {};
media.fake = media.audio = true;
var client = new mozRTCPeerConnection;
var server = new mozRTCPeerConnection;
When the client connects to the server it must open a data channel:
client.onconnection = function () {
var channel = client.createDataChannel("chat", {});
channel.onmessage = function (event) {
alert("Server: " + event.data);
};
channel.onopen = function () {
channel.send("Hello Server!");
};
};
When the client creates a data channel the server may respond:
server.ondatachannel = function (channel) {
channel.onmessage = function (event) {
alert("Client: " + event.data);
};
channel.onopen = function () {
channel.send("Hello Client!");
};
};
We need to add a fake audio stream to the client and the server to establish a connection:
navigator.mozGetUserMedia(media, callback, errback);
function callback(fakeAudio) {
server.addStream(fakeAudio);
client.addStream(fakeAudio);
client.createOffer(offer);
}
function errback(error) {
alert(error);
}
The client creates an offer:
function offer(description) {
client.setLocalDescription(description, function () {
server.setRemoteDescription(description, function () {
server.createAnswer(answer);
});
});
}
The server accepts the offer and establishes a connection:
function answer(description) {
server.setLocalDescription(description, function () {
client.setRemoteDescription(description, function () {
var port1 = Date.now();
var port2 = port1 + 1;
client.connectDataConnection(port1, port2);
server.connectDataConnection(port2, port1);
});
});
}
Phew. That took a while to understand.
I've posted a gist that shows setting up a data connection, compatible with both Chrome and Firefox.
The main difference is that where in FF you have to wait until the connection is set up, in Chrome it's just the opposite: it seems you need to create the data connection before any offers are sent back/forth:
var pc1 = new RTCPeerConnection(cfg, con);
if (!pc1.connectDataConnection) setupDC1(); // Chrome...Firefox defers per other answer
The other difference is that Chrome passes an event object to .ondatachannel whereas FF passes just a raw channel:
pc2.ondatachannel = function (e) {
var datachannel = e.channel || e;
Note that you currently need Chrome Nightly started with --enable-data-channels for it to work as well.
Here is a sequence of events I have working today (Feb 2014) in Chrome. This is for a simplified case where peer 1 will stream video to peer 2.
Set up some way for the peers to exchange messages. (The variance in how people accomplish this is what makes different WebRTC code samples so incommensurable, sadly. But mentally, and in your code organization, try to separate this logic out from the rest.)
On each side, set up message handlers for the important signalling messages. You can set them up and leave them up. There are 3 core messages to handle & send:
an ice candidate sent from the other side ==> call addIceCandidate with it
an offer message ==> SetRemoteDescription with it, then make an answer & send it
an answer message ===> SetRemoteDescription with it
On each side, create a new peerconnection object and attach event handlers to it for important events: onicecandidate, onremovestream, onaddstream, etc.
ice candidate ===> send it to other side
stream added ===> attach it to a video element so you can see it
When both peers are present and all the handlers are in place, peer 1 gets a trigger message of some kind to start video capture (using the getUserMedia call)
Once getUserMedia succeeds, we have a stream. Call addStream on the peer 1's peer connection object.
Then -- and only then -- peer 1 makes an offer
Due to the handlers we set up in step 2, peer 2 gets this and sends an answer
Concurrently with this (and somewhat obscurely), the peer connection object starts producing ice candidates. They get sent back and forth between the two peers and handled (steps 2 & 3 above)
Streaming starts by itself, opaquely, as a result of 2 conditions:
offer/answer exchange
ice candidates received, exchanged, and added
I haven't found a way to add video after step 9. When I want to change something, I go back to step 3.
I'm assuming this isn't possible, but wanted to ask in case it is. If I want to provide a status information web page, I want to use WebSockets to push the data from the server to the browser. But my concerns are the effect a large number of browsers will have on the server. Can I broadcast to all clients rather than send discrete messages to each client?
WebSockets uses TCP, which is point to point, and provides no broadcast support.
Not sure how is your client/server setup, but you can always just keep in the server a collection of all connected clients - and then iterate over each one and send the message.
A simple example using Node's Websocket library:
Server code
var WebSocketServer = require('websocket').server;
var clients = [];
var socket = new WebSocketServer({
httpServer: server,
autoAcceptConnections: false
});
socket.on('request', function(request) {
var connection = request.accept('any-protocol', request.origin);
clients.push(connection);
connection.on('message', function(message) {
//broadcast the message to all the clients
clients.forEach(function(client) {
client.send(message.utf8Data);
});
});
});
As noted in other answers, WebSockets don't support multicast, but it looks like the 'ws' module maintains a list of connected clients for you, so it's pretty easy to iterate through them. From the docs:
const WebSocketServer = require('ws').Server;
const wss = new WebSocketServer({ port: 8080 });
wss.broadcast = function(data) {
wss.clients.forEach(client => client.send(data));
};
Yes, it is possible to broadcast messages to multiple clients.
In Java,
#OnMessage
public void onMessage(String m, Session s) throws IOException {
for (Session session : s.getOpenSessions()) {
session.getBasicRemote().sendText(m);
}
}
and here it is explained.
https://blogs.oracle.com/PavelBucek/entry/optimized_websocket_broadcast.
It depends on the server-side really. Here's an example of how it's done using Tomcat7:
Tomcat 7 Chat Websockets Servlet Example
and an explanation of the how it's constructed here.
Yes you can and there are many socket servers out there written in various scripting languages that are doing it.
The Microsoft.Web.WebSockets namespace has a WebSocketCollection with Broadcast capability. Look for the assembly in Nuget. The name is Microsoft.WebSockets.