Develop Reference

sending a websocket request through a proxy server in nodeJS [duplicate]

Generalizing that would be the question... how to make websockets to go through a proxy in node.js?
In my particular case I'm using pusher.com with the node.js client library they recommend. Looking inside the code I would like to know some hints on what I should change in order to make this library to work with a proxy... you can take a look in the code here
Maybe I should somehow replace or modified the websockets module that is being used by the library?
EDIT
Thanks for your answers/comments! A couple of things to take into consideration (excuse me if I'm wrong with some/all of them, just learning):
I don't want to create a proxy server. I just want to use an existent proxy server within my company in order to proxified my websockets requests (particularly pusher.com)
Just to let you know, if I use a proxifier like the one for windows Proxifier and I set up the rule to inspect for all connections to port 443 to go through the proxy server proxy-my.coporate.com:1080 (type SOCKS5) it works like a charm.
But I don't want to go this way. I want to programatically configuring this proxy server within my node js code (even if that involved to modified the pusher library I mentioned)
I know how to do this for HTTP using Request module (look for the section that mentions how to use a proxy).
I want a similarly thing for websockets.

From
https://www.npmjs.com/package/https-proxy-agent
var url = require('url');
var WebSocket = require('ws');
var HttpsProxyAgent = require('https-proxy-agent');
// HTTP/HTTPS proxy to connect to
var proxy = process.env.http_proxy || 'http://168.63.76.32:3128';
console.log('using proxy server %j', proxy);
// WebSocket endpoint for the proxy to connect to
var endpoint = process.argv[2] || 'ws://echo.websocket.org';
var parsed = url.parse(endpoint);
console.log('attempting to connect to WebSocket %j', endpoint);
// create an instance of the `HttpsProxyAgent` class with the proxy server information
var options = url.parse(proxy);
var agent = new HttpsProxyAgent(options);
// finally, initiate the WebSocket connection
var socket = new WebSocket(endpoint, { agent: agent });
socket.on('open', function () {
console.log('"open" event!');
socket.send('hello world');
});
socket.on('message', function (data, flags) {
console.log('"message" event! %j %j', data, flags);
socket.close();
});

Using a proxy for websockets should work roughly the same as for https connections; you should use the CONNECT method. At least that's what both the HTTP and HTML5 specs say. So if your proxy implements CONNECT, you're good to go.

Try node-http-proxy
It allows you to send http or websocket requests through a proxy.
var http = require('http'),
httpProxy = require('http-proxy');
//
// Create a basic proxy server in one line of code...
//
// This listens on port 8000 for incoming HTTP requests
// and proxies them to port 9000
httpProxy.createServer(9000, 'localhost').listen(8000);
//
// ...and a simple http server to show us our request back.
//
http.createServer(function (req, res) {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.write('request successfully proxied!' + '\n' + JSON.stringify(req.headers, true, 2));
res.end();
}).listen(9000);
Source: link

Most web proxies don't support websockets yet. The best workaround is to use encryption by specifying wss:// (websocket secure protocol):
wss://ws.pusherapp.com:[port]/app/[key]

How to connect to a TCP server and pass a Javascript to it

I'm definitely a newbie with JS and node. I have telescope management software called SkyX Pro, and it has the ability to run a TCP Server on port 3040. I can connect to it using Netcat and hand it a Javascript starting with //* Javascript *// this works and allows me to startup cameras and other equipment and send commands for taking pictures etc. The issue is it needs to be run from a batch file which makes getting any information back to an HTML page tough (Like Camera, focuser and filter wheel status and temperatures).
The NC call looks like "NC localhost 3040 < Javascript-file.js
To get around the browser to local machine security issues I want to run this from node.js with maybe socket.io-client if possible, but I don't know the proper syntax for it.
I have seen plenty of client syntax sending hello's etc. but nothing send javascript and allowing for two-way connectivity that I can understand.
I have tried using:
var socket = io.connect('http://localhost');`enter code here`
socket.on('httpServer', function (data) {
console.log(data);
document.write(data + "\r\n");
socket.emit('tcp', "For TCP");
});
const net = require('net');
const client = new net.Socket();
client.connect({ port: 3040, host: process.argv[2] });
client.on('data', (data) => {
console.log(data.toString('utf-8'));
But I do not understand it well enough to troubleshoot why it's not working.
Any help would be wonderful, and please treat me like a baby that needs its step by step.
Cheer
Peter

Reading [1], We can assume socket-io isn't the perfect fit for you, because that Server you have sound like a typical tcp-socket server, not a socket.io server ( which requires special headers ) or a web-socket server.
So you only needs "net" library to do the job.
const net = require('net');
// module to send a message to TCP-socket server and wait for the response from socket-server
const sendAndReceive = async (client, message) => {
client.write(message);
let response = null
await ( new Promise( (resolve, reject) => {
client.on('data', function(data) {
response = data;
resolve()
});
}))
return response;
}
// send a single message to the socket-server and print the response
const sendJSCode = (message) => {
// create socket-client
const client = new net.Socket();
client.connect(3040, 'localhost', async function() {
console.log('Connected');
// send message and receive response
const response = await sendAndReceive(client, message)
// parse and print repsonse string
const stringifiedResponse = Buffer.from(response).toString()
console.log('from server: ', stringifiedResponse)
// clean up connection
client.destroy()
});
}
sendJSCode('var Out; \n Out="TheSky Build=" + Application.build \n\r')
This script will:
Initiate a socket client
on connection successfully, client sends a message
client receives back response from that message
client prints response to terminal
Note that TheSkyX has a limitation of 4096 bytes for each message[2], any more than that and we will need to chunk the message. So you may want to keep the js-code short and precise.
that snippet I gave is minimal, it doesn't handle errors from server. If you want, you can add client.on("error", .. ) to handle it.
Your point of connecting to the socket server directly from browser is very intriguing, unfortunately it is not allowed by modern browsers natively due to security concerns 3
[1] https://socket.io/docs/#What-Socket-IO-is-not:~:text=That%20is%20why%20a%20WebSocket%20client,to%20a%20plain%20WebSocket%20server%20either.
[2] https://www.bisque.com/wp-content/scripttheskyx/scriptOverSocket.html#MSearchField:~:text=set%20to%204096%20bytes

When tunnelling a TLS connection, how to pass additional information?

I have created a bare-bones HTTP proxy that performs HTTP tunnelling using HTTP CONNECT method.
const http = require('http');
const https = require('https');
const pem = require('pem');
const net = require('net');
const util = require('util');
const createHttpsServer = (callback) => {
pem.createCertificate({
days: 365,
selfSigned: true
}, (error, {serviceKey, certificate, csr}) => {
const httpsOptions = {
ca: csr,
cert: certificate,
key: serviceKey
};
const server = https.createServer(httpsOptions, (req, res) => {
// How do I know I here whats the target server port?
res.writeHead(200);
res.end('OK');
});
server.listen((error) => {
if (error) {
console.error(error);
} else {
callback(null, server.address().port);
}
});
});
};
const createProxy = (httpsServerPort) => {
const proxy = http.createServer();
proxy.on('connect', (request, requestSocket, head) => {
// Here I know whats the target server PORT.
const targetServerPort = Number(request.url.split(':')[1]);
console.log('target server port', targetServerPort);
const serverSocket = net.connect(httpsServerPort, 'localhost', () => {
requestSocket.write(
'HTTP/1.1 200 Connection established\r\n\r\n'
);
serverSocket.write(head);
serverSocket.pipe(requestSocket);
requestSocket.pipe(serverSocket);
});
});
proxy.listen(9000);
};
const main = () => {
createHttpsServer((error, httpsServerPort) => {
if (error) {
console.error(error);
} else {
createProxy(httpsServerPort);
}
});
};
main();
The server accepts a HTTPS connection and responds with "OK" message without forwarding the request further.
As you can see in the code (see // Here I know whats the target server PORT.), I can obtain the target server's port within the HTTP CONNECT event handler. However, I am unable to figure out how to pass this information to the createHttpsServer HTTP server router (see // How do I know I here whats the target server port?).
When tunnelling a TLS connection, how to pass additional information?
The above code can be tested by running:
$ node proxy.js &
$ curl --proxy http://localhost:9000 https://localhost:59194/foo.html -k
The objective is to respond with "OK localhost:59194".

You can't add anything to a TLS stream (thankfully), short of tunneling it inside another protocol⁠—which is what the Connect method already does. But, since you have the HTTP proxy and the HTTPS server in the same codebase, you don't need to fling the TLS stream over the network another time. Instead, you want to parse the TLS stream, and then you can pass any variables to the code that handles it.
However, after parsing TLS you'll still have a raw HTTP stream, and you'll need an HTTP server to turn it into requests and to handle responses.
The quick and rather dirty way to go about it is to use Node's HTTPS server to both decode TLS and parse HTTP. But the server's API doesn't provide for dealing with sockets that are already connected, and server's code isn't cleanly separated from connection code. So you need to hijack the server's internal connection-handling logic—this is of course susceptible to breakage in case of future changes:
const http = require('http');
const https = require('https');
const pem = require('pem');
const createProxy = (httpsOptions) => {
const proxy = http.createServer();
proxy.on('connect', (request, requestSocket, head) => {
const server = https.createServer(httpsOptions, (req, res) => {
res.writeHead(200);
res.end('OK');
});
server.emit('connection', requestSocket);
requestSocket.write('HTTP/1.1 200 Connection established\r\n\r\n');
});
proxy.listen(9000);
};
const main = () => {
pem.createCertificate({
days: 365,
selfSigned: true
}, (error, {serviceKey, certificate, csr}) => {
createProxy({
ca: csr,
cert: certificate,
key: serviceKey
});
});
};
main();
To avoid creating an HTTPS server instance on every request, you can move the instance out and tack you data onto the socket object instead:
const server = https.createServer(httpsOptions, (req, res) => {
res.writeHead(200);
// here we reach to the net.Socket instance saved on the tls.TLSSocket object,
// for extra dirtiness
res.end('OK ' + req.socket._parent.marker + '\n');
});
proxy.on('connect', (request, requestSocket, head) => {
requestSocket.marker = Math.random();
server.emit('connection', requestSocket);
requestSocket.write('HTTP/1.1 200 Connection established\r\n\r\n');
});
With the above code, if you do several successive requests:
curl --proxy http://localhost:9000 https://localhost:59194/foo.html \
https://localhost:59194/foo.html https://localhost:59194/foo.html \
https://localhost:59194/foo.html https://localhost:59194/foo.html -k
then you'll also notice that they're processed on a single connection, which is nice:
OK 0.6113572936982015
OK 0.6113572936982015
OK 0.6113572936982015
OK 0.6113572936982015
OK 0.6113572936982015
I can't quite vouch that nothing will be broken by handing the socket to the HTTPS server while the proxy server already manages it. [The server has the presence of mind to not overwrite another instance on the socket object](https://github.com/nodejs/node/blob/v10.9.0/lib/_http_server.js#L331), but otherwise seems to be rather closely involved with the socket. You'll want to test it with longer-running connections.
As for the `head` argument, [which can indeed contain initial data](https://www.rfc-editor.org/rfc/rfc2817#section-5.2):
you might be able to put it back on the stream with requestSocket.unshift(head), but I'm not sure that it won't be immediately consumed by the proxy server.
Or, you might be able to chuck it over to the HTTPS server with requestSocket.emit('data', head) since the HTTP server seems to use the stream events, however TLS socket source calls read() for whatever reason, and that's mutually exclusive with the events, so I'm not sure how they even work with each other.
One solution would be to make your own wrapper for stream.Duplex that will forward all calls and events, except for read() in the case when this initial buffer exists—and then use this wrapper in place of requestSocket. But you'll then need to replicate the 'data' event also, in accordance with the logic of Node's readable streams.
Finally, you can try creating a new duplex stream, write head and pipe the socket to it, like you did initially, and use the stream in place of the socket for the HTTPS server—not sure that it will be compatible with HTTP server's rather overbearing management of the socket.
An cleaner approach is to decode the TLS stream and use a standalone parser for the resultant HTTP stream. Thankfully, Node has a tls module that is nicely isolated and turns TLS sockets into regular sockets:
proxy.on('connect', (request, requestSocket, head) => {
const httpSocket = new tls.TLSSocket(requestSocket, {
isServer: true,
// this var can be reused for all requests,
// as it's normally saved on an HTTPS server instance
secureContext: tls.createSecureContext(httpsOptions)
});
...
});
See caveats on tls.createSecureContext regarding replicating the behavior of the HTTPS server.
Alas, Node's HTTP parser isn't so usable: it's a C library, which necessitates quite a bit of legwork between the socket and the parser calls. And the API can (and does) change between versions, without warnings, with a larger surface for incompatibilities compared to the HTTP server internals used above.
There are NPM modules for parsing HTTP: e.g. one, two, but none seem too mature and maintained.
I also have doubts about the feasibility of a custom HTTP server because network sockets tend to require plenty of nurture over time due to edge cases, with hard-to-debug timeout issues and such—which should all be already accounted for in the Node's HTTP server.
P.S. One possible area of investigation is how the Cluster module handles connections: afaik the parent process in a cluster hands connection sockets over to the children, but it doesn't fork on every request—which suggests that the child processes somehow deal with connected sockets, in code that's outside of an HTTP server instance. However, since the Cluster module is now in the core, it may exploit non-public APIs.

ldapjs authentification (user login setup)

So I'm currently running node.js, which has ldapjs installed. My aim is to have a system that uses ldapjs to allow users to login with a username and password.
I have been reading over the http://ldapjs.org documentation for awhile now but am struggling to understand the whole idea of ldap and ldapjs's implementation of it.
I currently have this from the documentation
var ldap = require('ldapjs');
var server = ldap.createServer();
server.bind('cn=root', function(req, res, next) {
if (req.dn.toString() !== 'cn=root' || req.credentials !== 'secret')
return next(new ldap.InvalidCredentialsError());
res.end();
return next();
});
server.listen(1389, function() {
console.log('LDAP server up at: %s', server.url);
});
Which allows me to run the below and successfully bind to the server.
ldapsearch -H ldap://localhost:1389 -x -D cn=root -w secret -LLL -b "o=myhost" objectclass=*
But I'm really unsure of where to go from here or even if this is the correct approach...
The ideal setup would be to have a range of users and passwords, and on a successful ldap connection confirm the details are correct and respond with a true, or false if the username/pass was incorrect.
Does anyone know of any good resources for finding out more about this, or better yet can suggest some basic client/server side code to give me an idea of where to go next!
Any replies would be really appreciated.
Many Thanks

I never used ldapjs, but based on what I just quickly read in its seemingly incomplete document, it can be used to implement an LDAP server or an LDAP client, which seems to be what you're trying to do (i.e., I'm assuming you want to authenticate users in your application against an existing LDAP server). Most of the examples in its document focus on creating an LDAP server that listens on a certain port and interacts with a back-end database. If you're not trying to put an LDAP-based interface between your back-end database or store of users and passwords, then you probably don't need the server API. If you already have an LDAP server running, then you will need to use its client API to do something like this:
1.Bind anonymously to the LDAP server that provides the directory services including the authentication services. It looks like you can just do this with:
var ldap = require('ldapjs');
var client = ldap.createClient({
url: 'ldap://my.ldap.server'
});
2.Search by the username (e.g., e-mail address) for the corresponding entry's DN
var opts = {
filter: '(mail=USERNAME)',
scope: 'sub'
};
client.search('ou=users,o=acme.com', opts, function(err, res) {
assert.ifError(err);
res.on('searchEntry', function(entry) {
console.log('entry: ' + JSON.stringify(entry.object));
});
res.on('searchReference', function(referral) {
console.log('referral: ' + referral.uris.join());
});
res.on('error', function(err) {
console.error('error: ' + err.message);
});
res.on('end', function(result) {
console.log('status: ' + result.status);
});
});
3.Grab the DN of the returned entry ( entry.object ). The documentation of this library doesn't talk much about how these objects can be used (e.g., what their methods, properties, etc. are). So, you will have to figure out how to actually get the DN or string representation of the DN of the entry you just retrieved from the directory server. [See the comment(s) below this answer]
4.Rebind to the server using that DN:
client.bind(DN_RETRIEVED, PASSWORD_USER_ENTERED, function(err) {
assert.ifError(err);
});
5.The result of the bind above is what you will need to use to determine whether or not the authentication was successful.
If you are trying to implement an LDAP server in front of your user/password data store for LDAP-based authentication, then you will need to follow their server examples. I personally think this is an overkill and could be problematic in terms of security.

Can I broadcast to all WebSocket clients

I'm assuming this isn't possible, but wanted to ask in case it is. If I want to provide a status information web page, I want to use WebSockets to push the data from the server to the browser. But my concerns are the effect a large number of browsers will have on the server. Can I broadcast to all clients rather than send discrete messages to each client?

WebSockets uses TCP, which is point to point, and provides no broadcast support.

Not sure how is your client/server setup, but you can always just keep in the server a collection of all connected clients - and then iterate over each one and send the message.
A simple example using Node's Websocket library:
Server code
var WebSocketServer = require('websocket').server;
var clients = [];
var socket = new WebSocketServer({
httpServer: server,
autoAcceptConnections: false
});
socket.on('request', function(request) {
var connection = request.accept('any-protocol', request.origin);
clients.push(connection);
connection.on('message', function(message) {
//broadcast the message to all the clients
clients.forEach(function(client) {
client.send(message.utf8Data);
});
});
});

As noted in other answers, WebSockets don't support multicast, but it looks like the 'ws' module maintains a list of connected clients for you, so it's pretty easy to iterate through them. From the docs:
const WebSocketServer = require('ws').Server;
const wss = new WebSocketServer({ port: 8080 });
wss.broadcast = function(data) {
wss.clients.forEach(client => client.send(data));
};

Yes, it is possible to broadcast messages to multiple clients.
In Java,
#OnMessage
public void onMessage(String m, Session s) throws IOException {
for (Session session : s.getOpenSessions()) {
session.getBasicRemote().sendText(m);
}
}
and here it is explained.
https://blogs.oracle.com/PavelBucek/entry/optimized_websocket_broadcast.

It depends on the server-side really. Here's an example of how it's done using Tomcat7:
Tomcat 7 Chat Websockets Servlet Example
and an explanation of the how it's constructed here.

Yes you can and there are many socket servers out there written in various scripting languages that are doing it.

The Microsoft.Web.WebSockets namespace has a WebSocketCollection with Broadcast capability. Look for the assembly in Nuget. The name is Microsoft.WebSockets.