Audio/Voice over webRTC - javascript

I am trying to implement a one directional voice transmission with webRTC and php as a server side.
Looking at the samples, I fail to understand the webRTC mechanism.
The way I see it, the flow should look like this:
Caller and recipient register on server
Recipient listens for incoming calls
Caller asks server for recipient's IP
Server sends IP to caller
Caller connects to recipient directly
However the sample code, (that runs on a local machine)
function call() {
trace('Starting call');
var servers = null;
var pcConstraints = {
'optional': []
};
pc1 = new RTCPeerConnection(servers, pcConstraints);
trace('Created local peer connection object pc1');
pc1.onicecandidate = iceCallback1;
pc2 = new RTCPeerConnection(servers, pcConstraints);
trace('Created remote peer connection object pc2');
pc2.onicecandidate = iceCallback2;
pc2.onaddstream = gotRemoteStream;
trace('Requesting local stream');
navigator.mediaDevices.getUserMedia({
audio: true,
video: false
})
.then(gotStream)
.catch(function(e) {
alert('getUserMedia() error: ' + e.name);
});
}
Does not use ip address or any kind of identification token that could be translated into IP.
How could that be ?

To get started you will need to provide some kind of signaling channel between your clients. Most people are using some protocol (typically JSON based or SIP) over WebSockets to something like Node.js, SIP server or message broker on the backend which can then bridge between the two clients. You can pretty much send blobs of data that are pulled directly from the WebRTC PeerConnection. This includes SDPs, ICE candidates, etc. The PeerConnection can directly consume the data generated from the other peer so you don't have to deal with formatting the data. Just pack it into a JSON object and send it to Node over a WebSockets and let Node send it out to the other side. Its up to you to design how registered endpoints find each other and create this bridge.
We used MQTT over WebSockets to do this. In our model each client subscribes on their own MQTT topic and each side can publish messages to the other client on those topics. Clients register these topics with a back end service that is subscribed on a topic as well or you can use retained messages to let the MQTT broker manage the topics. You can read more about our approach here: http://www.wasdev.net/webrtc. We open sourced the signaling protocol and you can use any open MQTT broker for this. We also created SDKs including an AngularJS module you can play with here: http://angular-rtcomm.wasdev.developer.ibm.com/

Related

Web sockets, socket.io or other alternative

I'm kinda new on sockets related subject so I'm sorry for any dumb question.
I would like to do something like this… I have am hybrid app and a website, and I wanted that when I click in a button on the app, it shows me alert/notificaion on the website. I read about Socket io and it does the job on localhost, but I want na alternative that not uses a server behind, since I'm not being able to run it using CPANEL (What I have access to)
Is it possible to have like a "direct" connection from the app to the site when I click the button?
You can consider using firebase for this:
In your javascript:
// execute the following script on click
importScripts('https://www.gstatic.com/firebasejs/4.8.1/firebase-app.js');
importScripts('https://www.gstatic.com/firebasejs/4.8.1/firebase-messaging.js');
// Initialize the Firebase app in the service worker by passing in the
// messagingSenderId.
firebase.initializeApp({
'messagingSenderId': 'YOUR-SENDER-ID'
});
// Retrieve an instance of Firebase Messaging so that it can handle background
// messages.
const messaging = firebase.messaging();
messaging.send({data: "your data if you want to send"}).then((response) => {
// Response is a message ID string.
console.log('Successfully sent message:', response);
})
.catch((error) => {
console.log('Error sending message:', error);
});
// similarly, on your browser:
messaging.onMessage(function(payload) {
console.log('Message received. ', payload);
// ...
});
link: https://firebase.google.com/docs/
Hope it helps
Let's break the problem down into a few parts, starting with the transport to the browser, as that's what you're asking about.
Web Sockets are a way to establish a bi-directional connection between a server and a client. It's a standard implemented by most any modern browser. Socket.IO is a web-socket-like abstraction that can use Web Sockets or other transports under the hood. It was originally built as sort of a polyfill, allowing messages to be sent via Web Sockets, or even long-polling. Using Socket.IO doesn't give you any additional capability than you have with just the browser, but it does provide some nice abstractions for "rooms" and such.
If you're sending data only from the server to the client, Web Sockets aren't the ideal choice. For streaming of data in general, the Fetch API and ReadableStream are more appropriate. Then, you can just make a normal HTTP connection. However, what you're looking for is event-style data, and for that there are Server-Sent Events (SSE). https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events Basically, you instantiate an EventSource object on the client, pointed at a URL on the server. The client automatically maintains a connection, reconnecting if necessary. It's also capable of synchronizing to a point in the stream, providing the server with the last message received so that the client can be caught up to present time.
Now, how does your server endpoint know when to send this data, and what to send? Ideally, you'll use some sort of pub/sub system. These capabilities are built into Redis, which is commonly used for this. (There are others as well, if you don't like Redis for some reason.) Basically, when your server receives something from the app, the app is "publishing" a message to a particular channel where all "subscribers" will receive it. Your server will be that EventSource and can simply relay data (verifying it and authenticating of course, along the way).
You can write a PHP script that has a POST/GET endpoint. Your app will communicate to this endpoint. The endpoint needs to handle the message and write it to a database. Your website can then poll to see if there are any new entries, and show something if there are
Alright, let's do it in PHP. This is just the most basic example. Just put it somewhere and link to the script from your app.
<?php
function requestVars($type = 'REQUEST'){
if($type == 'REQUEST')
$r = $_REQUEST;
elseif($type == 'POST')
$r = $_POST;
elseif($type == 'GET')
$r = $_GET;
$ret = array();
foreach($r as $r1 => $r2)
$ret[$r1] = $r2;
return $ret;
}
$vars = requestVars(); //get variables from request
echo $vars['var1']; // var1 is what comes in from the client
?>
I haven't tested this, so if something is wrong let me know.

Try to connect to a server with Google Assistance App

I need to send data out from my google assistance app to a database. In order to do this, I've created a server that takes the data, packages it, and then sends it out. I have the hostname and port and it works in a normal javascript/node.js program but when I use it in my google assistant app nothing happens. I tried figuring out the problem and it looks like the code just isn't connecting. The code I'm using to send data to the server is as follows:
function sendData(app){
var net = require('net');
var message = {"test": 200};
var thisMessage = JSON.stringify(message);
var client = new net.Socket();
client.connect(<port>, '<hostname>', function() {
app.tell(JSON.stringify(client.address()));
console.log('Connected');
client.write(thisMessage);
});
client.on('data', function(data) {
console.log('Received: ' + data);
client.destroy();
});
client.on('close', function() {
console.log('Connection closed');
});
return 0;
}
(NOTE: Port and hostname left out for privacy purposes)
This completely skips over the app.tell, leading me to believe the connection is never made. I know it works asynchronously with the server, however, I don't understand why it isn't connecting whatsoever.
I have tried it both in simulation and on my smartphone with sandbox on and off. Is there a better way to connect? Note that the server I'm connecting to is python-based.
The problem is likely that you're running it on Cloud Functions for Firebase which has a limit on outbound connections under their free "Spark" plan. With this plan, you can only connect to other Google services. This is usually a good way to start understanding how to handle Action requests, but has limitations. To access endpoints outside of Google, you need to upgrade to either their "Flame" fixed price plan or "Blaze" pay-as-you-go plan.
You do not, however, need to run on Google's servers or need to use node.js. All you need is a public HTTPS server with a valid SSL cert. If you are familiar with JSON, you can use any programming language to handle the request and response. If you are familiar with node.js, you just need a node.js server that can create Express request and response objects.

MQTT PUBACK web sockets

I'm working on HiveMQ Websocket Client and I'm facing some issues with the message delivery.
so, I've come across the word PUBACK
let me explain you about my understanding and then I will ask my question.
whenever we send a message with QOS1, the hivemq server will acknowledge the sender with a PUBACK callback.
Now, I'm planning to subscibe to onPubackReceived event in my websockets, but the event is not firing after sending the message.
My Code:
var clientId = ClientIdentifier;
mqtt = new Messaging.Client(
host,
port,
clientId);
var options = {
timeout: 3,
keepAliveInterval: 60,
useSSL: useTLS,
cleanSession: cleansession,
onSuccess: onConnect,
onFailure: function (message) {
connected = false;
setTimeout(MQTTconnect, reconnectTimeout);
}
};
mqtt.onConnectionLost = onConnectionLost;
mqtt.onMessageArrived = onMessageArrived;
mqtt.onPubackReceived = OnPubackReceived;
Both the onConnectionLost and onMessageArrived are firing properly when a connection lost and message arrived, but the onPubackReceived is not firing.
please let me know, if I have understood it correctly or if I'm doing some mistake?
This not a HiveMQ issue.
My assumption is, that you used the HiveMQ Websocket Client as a starting point for your implementation.
In any case a Paho MQTT Client does not have a onPubackReceived field.
If you provide more details about your use case or what's your issue with message delivery, I might be able to point you into the right direction.
EDIT:
What you are describing is called Quality of Service 1 in MQTT. It is a guarantee, that a message is received at least once.
It is the client implementation's job to keep this guarantees and therefor resend a message, should a PUBACK not be received. Manually interfering with this behaviour in your application would result in inconsistency regarding the client's persistence.
For clarification:
Simply setting duplicate=truewill not result in a message being recognised as a duplicate. It will also have to have the the same messageID as the original.
I was not able to actually find any documentation about paho.jskeeping the Quality of Service = 1.
However, MQTT.js does.
QoS 1 : received at least once : The packet is sent and stored as long as the client has not received a confirmation from the server. MQTT ensures that it will be received, but there can be duplicates.
To sum things up:
Resending of messages, no PUBACK was received on, is the client Object's job. This is part of the MQTT specification.
Using the MQTT.js works over Websockets and ensures to keep QoS levels
Hope this helps.

SignalR - connect to websocket service from javascript without SignalR library

I have a small SignalR project that I've started that right now all it does is receives a string and echo it back to all connected users.
What I'm wondering is, since SignalR open websockets on my server - how can I connect to the service using regular websockets javascript code? (I have a reason for wanting to do it that way without the SignalR library).
I've watched using chrome developer tools and I found that the address the browser is connecting to is:
ws://localhost:53675/signalr/connect?transport=webSockets&clientProtocol=1.4&connectionToken=YKgNxA6dhmtHya1srzPPIv6KFIYEjLTFOogXqcjPErFcKCmRdjvS2X6A2KmraW%2BrLnRUNf68gYPdOkOOgJ8yRcq4iCDm%2BbUyLejsr2NySNZBvLloWuMIAvgI6oC%2Fyt%2Ba&connectionData=%5B%7B%22name%22%3A%22ophirhubtest%22%7D%5D&tid=7
How do I generate the token?
Then, it seems that the messages going between the client and server are just regular json formatted text (which will be easy to mimic):
{"C":"d-9E7D682A-A,1|E,0|F,1|G,0","S":1,"M":[]}
{"H":"ophirhubtest","M":"Echo","A":["test"],"I":0}
{"C":"d-9E7D682A-A,2|E,0|F,1|G,0","M":[{"H":"ophirHubTest","M":"printEcho","A":["You said: test"]}]}
If I just try to connect than it connects but the connection is quickly closed. If I remove the token it closes immediately.
Is it possible to connect to the WS "manually" ?
Before you can connect to the server there is connection negotiation happening. This is when the server sends all the data needed to send and receive messages. Without connection negotiation you won't be able to connect to the server. Once you implement connection negotiation you will be probably half into implementing a SignalR client. I wrote a blog post describing SignalR protocol which should help you understand how things works underneath and why connecting to the server with your own websocket is not straightforward (or simply impossible if you don't follow the SignalR protocol).
EDIT
The ASP.NET Core version of SignalR now allows connecting to the server with bare webSocket.
I just want to add a that it is possible to connect to ASP.NET Core version of SignalR with websocket but you have to add the magic char 30 at the end of every call you make
const endChar = String.fromCharCode(30);
socket.send(`{"arguments":["arg1"],"streamIds":[],"target":"TestMethod","type":1}${endChar}`);
Great answers by Frédéric Thibault but there is one important thing missing. You need to send the protocol and the version directly after connecting. Otherwise you will get the error:
An unexpected error occurred during connection handshake.
Here is a full working example on how to use signalR with plain JavaScript and WebSockets:
let socket = new WebSocket("wss://your-url");
socket.onopen = function(e) {
console.log("[open] Connection established");
const endChar = String.fromCharCode(30);
// send the protocol & version
socket.send(`{"protocol":"json","version":1}${endChar}`);
};
socket.onmessage = function(event) {
console.log(`[message] Data received from server: ${event.data}`);
// parse server data
const serverData = event.data.substring(0, event.data.length - 1);
// after sending the protocol & version subscribe to your method(s)
if (serverData === "{}") {
const endChar = String.fromCharCode(30);
socket.send(`{"arguments":[],"invocationId":"0","target":"Your-Method","type":1}${endChar}`);
return;
}
// handle server messages
};
socket.onclose = function(event) {
if (event.wasClean) {
console.log(`[close] Connection closed cleanly, code=${event.code} reason=${event.reason}`);
} else {
console.log('[close] Connection died');
}
};
socket.onerror = function(error) {
console.log(`[error] ${error.message}`);
};

Node.js Remote Start and Communication between Servers

I am new to Node.js and also pretty new to server communication.
I have tried to find previous answers, but they are often concerned about communication between server and client.
I have a different case, so need your considerate helps.
Let's assume a scenario that we have three systems, localhost (i.e., laptop) and two cloud servers. I want to code an js app in the localhost that will slice an array of data into two blocks and send them to the cloud servers (block #1 to the server #1 and block #2 to the server #2). Receiving them, two remote servers start to work at the same time. Then, they do the same computation and send their calculation results to each other if they have updated values.
In this scenario, I want to tackle bolded sentences. I believe using the module "socket.io" will be a proper approach to handle this (especially, remote start and communication) but do not have any clear idea in designing codes. In addition, understanding "socket.io" itself is a bit tricky. If you need further specification on the scenario, please comment.
Along with socket.io, check out a module named Faye (http://faye.jcoglan.com/node.html). I have been using it for a couple of years and really like it. Faye is a publish subscribe communication scheme which would allow you to extend your scenario to as many clients as you need. To install faye on your system, run the following command:
npm install -g faye
Now, here is your server code:
var faye = require('faye');
var Server = new faye.NodeAdapter({mount: ('/FayeServer'), timeout: 120});
//now fire the server up on the port 5555
Server.listen(5555);
//subscribe to channel DataChannel
var Subscription = Server.getClient().subscribe("DataChannel",
function(dataObject){ console.log(dataObject) },
function(status) {
console.log('Subscription Status: ' + status);
//send message with two numbers to any client listening to DataChannel
Server.getClient().publish('/DataChannel', {A:5,B:12});
});
Now, here is the client code:
var faye = require('faye');
//open client to server
var Client = new faye.Client('http://127.0.0.1:5555/FayeServer');
//now subscribe to the channel DataChannel
Client.subscribe('/DataChannel', function(dataObject)
{
Client.publish('/DataChannel', {C:(dataObject.A * dataObject.B)};
});
There is a lot more that can be done, but with this basic framework you can stand up server to N client programs that respond to messages from the server.
You will need to replace 127.0.0.1 with your specific URL and use port numbers and channel names more applicable to your specific application.

Categories

Resources