Develop Reference

Try to connect to a server with Google Assistance App

I need to send data out from my google assistance app to a database. In order to do this, I've created a server that takes the data, packages it, and then sends it out. I have the hostname and port and it works in a normal javascript/node.js program but when I use it in my google assistant app nothing happens. I tried figuring out the problem and it looks like the code just isn't connecting. The code I'm using to send data to the server is as follows:
function sendData(app){
var net = require('net');
var message = {"test": 200};
var thisMessage = JSON.stringify(message);
var client = new net.Socket();
client.connect(<port>, '<hostname>', function() {
app.tell(JSON.stringify(client.address()));
console.log('Connected');
client.write(thisMessage);
});
client.on('data', function(data) {
console.log('Received: ' + data);
client.destroy();
});
client.on('close', function() {
console.log('Connection closed');
});
return 0;
}
(NOTE: Port and hostname left out for privacy purposes)
This completely skips over the app.tell, leading me to believe the connection is never made. I know it works asynchronously with the server, however, I don't understand why it isn't connecting whatsoever.
I have tried it both in simulation and on my smartphone with sandbox on and off. Is there a better way to connect? Note that the server I'm connecting to is python-based.

The problem is likely that you're running it on Cloud Functions for Firebase which has a limit on outbound connections under their free "Spark" plan. With this plan, you can only connect to other Google services. This is usually a good way to start understanding how to handle Action requests, but has limitations. To access endpoints outside of Google, you need to upgrade to either their "Flame" fixed price plan or "Blaze" pay-as-you-go plan.
You do not, however, need to run on Google's servers or need to use node.js. All you need is a public HTTPS server with a valid SSL cert. If you are familiar with JSON, you can use any programming language to handle the request and response. If you are familiar with node.js, you just need a node.js server that can create Express request and response objects.

Understanding how to use Redis with Node.js and Server Sent Events

My Project is built with Nodejs as proxy server to communicate with an external API.
The API send product updates via Redis (pub/sub); The Proxy server handle the message and send it to the client via SSE (Server Sent Events).
It is the first time for me using Redis and SSE and looking online for tutorials seems to be easy to implement and I did it.
On the Client side I just created an EventSource and as soon as I receive an update I do something with it:
// Client Side
var source = new EventSource('/redis'); // /redis is path to proxy server
source.addEventListener('items', handleItemsCallback, false);
source.addEventListener('users', handleUsersCallback, false);
source.addEventListener('customers', handleCustomersCallback, false);
// Function sample...
function handleItemsCallback (msg) {
// Do something with msg...
}
In the Proxy server I created a controller with routing to /redis to handle Redis messages:
exports.redisUpdates = function (req, res) {
// Redis Authentication
var redisURL = url.parse(process.env.REDISCLOUD_URL);
var client = redis.createClient(redisURL.port, redisURL.hostname, {ignore_subscribe_messages: false});
client.auth(redisURL.auth.split(":")[1]);
// let request last as long as possible
req.socket.setTimeout(0);
// Subscribe to channels
client.subscribe('items', 'users', 'customers');
// Handle messages
client.on('message', function (channel, message) {
res.write('retry: 5000\n');
res.write('event: ' + channel + '\n');
res.write('data: ' + message + '\n\n');
res.flush(); // If I do not add this it doesn't push updates to the client (?)
});
//send headers for event-stream connection
res.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive'
});
res.write('\n');
};
Using it locally in a development environment it works fine but using it in Production generate several different errors, the App is hosted on Heroku and the Heroku Metrics show several H18, H12, H27 Errors;
Sometimes the /redis call return status 503;
What I wish to understand is if I'm using those services correctly, why all tutorials do not mention res.flush() and I discovered it by myself to let it work the first time...

In all fairness, this question is not really answerable for a few reasons. I don't know which tutorials you are talking about since you didn't reference any in the question. I cannot speak on behalf of those who wrote the unreferenced tutorials. They could just be wrong, or maybe the architecture of what you are trying to accomplish differs in some small way. I also don't know what framework or optional middleware you are using in your project.
Now, with all of that said there are a few things I can share that may help you out.
Most tutorials you find out there are probably not going to open a connection and read from the stream indefinitely. When the process ends, the http response is closed with .end() or something similar. Since an HTTP response is a write stream, it follows the same rules as any other stream. You can find a lot of good info about streams here:
https://github.com/substack/stream-handbook
Something important to understand is that a stream can have a buffer and most http frameworks enable compression which causes buffers to be used. The code sample in the next link is a good example of what a framework would do for you behind the scenes (a minimal implementation of course)
https://nodejs.org/en/docs/guides/anatomy-of-an-http-transaction/#what-we-ve-got-so-far
Since you want the output to continue being updated, you either have to wait until the output buffer size is reached or you have to call .flush().
If you ARE using express, check out this next Stack Overflow post related to compression middleware. I believe you'll have to have it disabled for your /redis route.
Node Express Content-Length
I hope that helped a little. Like I said, its kind of hard to answer this question. ;)

Correct way to handle Websocket

I've a client to server Websocket connection which should be there for 40 seconds or so. Ideally it should be forever open.
The client continually sends data to server and vice-versa.
Right now I'm using this sequence:
var socket;
function senddata(data)
{
if (!socket)
{
socket = new WebSocket(url);
socket.onopen = function (evt) {
socket.send(data);
socket.onmessage = function (evt) {
var obj = JSON.parse(evt.data);
port.postMessage(obj);
}
socket.oneerror = function (evt) {
socket.close();
socket = null;
}
socket.onclose = function(evt){
socket = null;
}
}
}
else
{
socket.send(data);
}
}
Clearly as per current logic, in case of error, the current request data may not be sent at all.
To be frank it sometimes gives error that websocket is still in connecting state. This connection breaks often due to networking issues. In short it does not work perfectly well.
I've read a better design : How to wait for a WebSocket's readyState to change but does not cover all cases I need to handle.
Also I've Googled about this but could not get the correct procedure for this.
So what is the right way to send regular data through Websockets which handles well these issues like connection break etc?

An event you don't seem to cover is onclose. Which should work really well, since it's called whenever the connection terminates. This is more reliable than onerror, because not all connection disruptions result in an error.

I personally use Socket.IO, it enables real-time bidirectional event-based communication between client and server.
It is event driven. Events such as
on connection :: socket.on('conection',callback);
and
on disconnect :: socket.on('disconnect',callback);
are built in with socket.io so it can help you with your connection concerns. Pretty much very easy to use, check out their site if you are interested.

I use two-layer scheme on client: abstract-wrapper + websocket-client:
The responsibilities of the websocket-client are interacting with a server, recovering the connection and providing interfaces (event-emitter and some methods) to abstract-wrapper.
The abstract-wrapper is a high-level layer, which interacts with websocket-client, subscribes to its events and aggregating data, when the connection is temporary failed. The abstract-wrapper can provide to application layer any interface such as Promise, EventEmitter and so on.
On application layer, I just work with abstract-wrapper and don't worry about connection or data losing. Undoubtedly, it's a good idea to have here information about the status of connection and data sending confirmation, because it's useful.
If it is necessary, I can provide some code for example

This apparently is a server issue not a problem in the client.
I don't know how the server looks like here. But this was a huge problem for me in the past when I was working on a websocket based project. The connection would continuously break.
So I created a websocket server in java, and that resolved my problem.
websockets depend on lots of settings, like if you're using servlets then servlet container's settings matter, if you're using some php etc, apache and php settings matter, for example if you create a websocket server in php and php has default time-out of 30 seconds, it will break after 30 seconds. If keep-alive is not set, the connection wont stay alive etc.
What you can do as quick solution is
keep sending pings to a server after a certain amount of time (like 2 or 3 seconds, so that if a websocket is disconnected it is known to the client so it could invoke onclose or ondisconnect, I hope you know that there is no way to find if a connection is broken other than failing to send something.
check server's keep-alive header
If you have access to server, then it's timeouts etc.
I think that would help

SignalR - connect to websocket service from javascript without SignalR library

I have a small SignalR project that I've started that right now all it does is receives a string and echo it back to all connected users.
What I'm wondering is, since SignalR open websockets on my server - how can I connect to the service using regular websockets javascript code? (I have a reason for wanting to do it that way without the SignalR library).
I've watched using chrome developer tools and I found that the address the browser is connecting to is:
ws://localhost:53675/signalr/connect?transport=webSockets&clientProtocol=1.4&connectionToken=YKgNxA6dhmtHya1srzPPIv6KFIYEjLTFOogXqcjPErFcKCmRdjvS2X6A2KmraW%2BrLnRUNf68gYPdOkOOgJ8yRcq4iCDm%2BbUyLejsr2NySNZBvLloWuMIAvgI6oC%2Fyt%2Ba&connectionData=%5B%7B%22name%22%3A%22ophirhubtest%22%7D%5D&tid=7
How do I generate the token?
Then, it seems that the messages going between the client and server are just regular json formatted text (which will be easy to mimic):
{"C":"d-9E7D682A-A,1|E,0|F,1|G,0","S":1,"M":[]}
{"H":"ophirhubtest","M":"Echo","A":["test"],"I":0}
{"C":"d-9E7D682A-A,2|E,0|F,1|G,0","M":[{"H":"ophirHubTest","M":"printEcho","A":["You said: test"]}]}
If I just try to connect than it connects but the connection is quickly closed. If I remove the token it closes immediately.
Is it possible to connect to the WS "manually" ?

Before you can connect to the server there is connection negotiation happening. This is when the server sends all the data needed to send and receive messages. Without connection negotiation you won't be able to connect to the server. Once you implement connection negotiation you will be probably half into implementing a SignalR client. I wrote a blog post describing SignalR protocol which should help you understand how things works underneath and why connecting to the server with your own websocket is not straightforward (or simply impossible if you don't follow the SignalR protocol).
EDIT
The ASP.NET Core version of SignalR now allows connecting to the server with bare webSocket.

I just want to add a that it is possible to connect to ASP.NET Core version of SignalR with websocket but you have to add the magic char 30 at the end of every call you make
const endChar = String.fromCharCode(30);
socket.send(`{"arguments":["arg1"],"streamIds":[],"target":"TestMethod","type":1}${endChar}`);

Great answers by Frédéric Thibault but there is one important thing missing. You need to send the protocol and the version directly after connecting. Otherwise you will get the error:
An unexpected error occurred during connection handshake.
Here is a full working example on how to use signalR with plain JavaScript and WebSockets:
let socket = new WebSocket("wss://your-url");
socket.onopen = function(e) {
console.log("[open] Connection established");
const endChar = String.fromCharCode(30);
// send the protocol & version
socket.send(`{"protocol":"json","version":1}${endChar}`);
};
socket.onmessage = function(event) {
console.log(`[message] Data received from server: ${event.data}`);
// parse server data
const serverData = event.data.substring(0, event.data.length - 1);
// after sending the protocol & version subscribe to your method(s)
if (serverData === "{}") {
const endChar = String.fromCharCode(30);
socket.send(`{"arguments":[],"invocationId":"0","target":"Your-Method","type":1}${endChar}`);
return;
}
// handle server messages
};
socket.onclose = function(event) {
if (event.wasClean) {
console.log(`[close] Connection closed cleanly, code=${event.code} reason=${event.reason}`);
} else {
console.log('[close] Connection died');
}
};
socket.onerror = function(error) {
console.log(`[error] ${error.message}`);
};

Error while getting messges from sockets in javascript

Hi i am having trouble with creating a socket communication from java script code.
I am always getting error while sending a message or closing the socket from server.
My Socket server code.
// Start listening for connections.
while (true)
{
Console.WriteLine("Waiting for a connection...");
// Program is suspended while waiting for an incoming connection.
Socket handler = listener.Accept();
data = null;
// An incoming connection needs to be processed.
while (true)
{
int bytesRec = handler.Receive(bytes);
data += Encoding.ASCII.GetString(bytes, 0, bytesRec);
break;
}
// Show the data on the console.
Console.WriteLine("Text received : {0}", data);
// Echo the data back to the client.
byte[] msg = Encoding.ASCII.GetBytes(data);
handler.Send(msg);
handler.Shutdown(SocketShutdown.Both);
handler.Close();
}
JavaScript code:
var connection = new WebSocket('ws://Myip:11000', ['soap', 'xmpp']);
// When the connection is open, send some data to the server
connection.onopen = function () {
connection.send('Ping'); // Send the message 'Ping' to the server
connection.send('your message');
};
// Log errors
connection.onerror = function (error) {
console.log('WebSocket Error ' + error);
};
connection.onclose = function (msg) {
console.log('WebSocket Error ' + msg);
};
It gets connected to server socket, but always gets error while closing or sending a message from server.

If this is really your actual code:
handler.Send(msg);
handler.Shutdown(SocketShutdown.Both);
handler.Close();
…then it's pretty broken. First, you can't assume that Socket.Send() actually sends all the bytes you asked it to. You have to check the return value, and keep sending until you've actually sent all the data.
Second, the initiation of a graceful closure should use SocketShutdown.Send, not SocketShutdown.Both. Specifying "Both" means (among other things) that you're not going to wait for the other end to negotiate the graceful closure. That you're just done and won't even receive any more data, in addition to being done sending.
And of course, the code is calling Close() before the other end has in fact acknowledged the graceful closure (by itself sending any remaining data it wanted to send and then shutting down with "Both").
Is all this the reason for your problem? I can't say for sure, since I have no way to test your actual code. But it's certainly a reasonable guess. If you tear down the connection without waiting after you try to send something, there's not any guarantee that the data will ever leave your machine, and in any case the other end could easily see the connection reset before it gets a chance to process any data that was sent to it.
There aren't a huge number of rules when it comes to socket programming, but what rules exist are there for a reason and are generally really important to follow. You should make sure your code is following all the rules.
(The rest of the code is also different from what I'd consider the right way to do things, but the problems aren't entirely fatal, the way that the over-eager connection destruction is).

I am afraid WebSocket does not work that way.
When the Javascript code connects to the server, it will send a HTTP request as ASCII text. That request will include a HTTP header Sec-WebSocket-Protocol: soap, xmpp, as you are requiring those protocols in your WebSocket creation.
Since your server code does not reply with an appropiate HTTP response accepting the websocket connection, the connection will fail. When you try to send data back, the client will not recognize it as a HTTP response and a error will be thrown.
A websocket is not a regular socket connection, it won't work that way. It requires HTTP negotiation and there is a schema for data framing. I recommend you to go through this article that explains very well how it works: http://chimera.labs.oreilly.com/books/1230000000545/ch17.html
If you are interested in learning how to develop a server, take a look to this tutorial in MDN: https://developer.mozilla.org/en-US/docs/WebSockets/Writing_WebSocket_server I also have an open source WebSocket server in C# you can take a look if you like.