Generalizing that would be the question... how to make websockets to go through a proxy in node.js?
In my particular case I'm using pusher.com with the node.js client library they recommend. Looking inside the code I would like to know some hints on what I should change in order to make this library to work with a proxy... you can take a look in the code here
Maybe I should somehow replace or modified the websockets module that is being used by the library?
EDIT
Thanks for your answers/comments! A couple of things to take into consideration (excuse me if I'm wrong with some/all of them, just learning):
I don't want to create a proxy server. I just want to use an existent proxy server within my company in order to proxified my websockets requests (particularly pusher.com)
Just to let you know, if I use a proxifier like the one for windows Proxifier and I set up the rule to inspect for all connections to port 443 to go through the proxy server proxy-my.coporate.com:1080 (type SOCKS5) it works like a charm.
But I don't want to go this way. I want to programatically configuring this proxy server within my node js code (even if that involved to modified the pusher library I mentioned)
I know how to do this for HTTP using Request module (look for the section that mentions how to use a proxy).
I want a similarly thing for websockets.
From
https://www.npmjs.com/package/https-proxy-agent
var url = require('url');
var WebSocket = require('ws');
var HttpsProxyAgent = require('https-proxy-agent');
// HTTP/HTTPS proxy to connect to
var proxy = process.env.http_proxy || 'http://168.63.76.32:3128';
console.log('using proxy server %j', proxy);
// WebSocket endpoint for the proxy to connect to
var endpoint = process.argv[2] || 'ws://echo.websocket.org';
var parsed = url.parse(endpoint);
console.log('attempting to connect to WebSocket %j', endpoint);
// create an instance of the `HttpsProxyAgent` class with the proxy server information
var options = url.parse(proxy);
var agent = new HttpsProxyAgent(options);
// finally, initiate the WebSocket connection
var socket = new WebSocket(endpoint, { agent: agent });
socket.on('open', function () {
console.log('"open" event!');
socket.send('hello world');
});
socket.on('message', function (data, flags) {
console.log('"message" event! %j %j', data, flags);
socket.close();
});
Using a proxy for websockets should work roughly the same as for https connections; you should use the CONNECT method. At least that's what both the HTTP and HTML5 specs say. So if your proxy implements CONNECT, you're good to go.
Try node-http-proxy
It allows you to send http or websocket requests through a proxy.
var http = require('http'),
httpProxy = require('http-proxy');
//
// Create a basic proxy server in one line of code...
//
// This listens on port 8000 for incoming HTTP requests
// and proxies them to port 9000
httpProxy.createServer(9000, 'localhost').listen(8000);
//
// ...and a simple http server to show us our request back.
//
http.createServer(function (req, res) {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.write('request successfully proxied!' + '\n' + JSON.stringify(req.headers, true, 2));
res.end();
}).listen(9000);
Source: link
Most web proxies don't support websockets yet. The best workaround is to use encryption by specifying wss:// (websocket secure protocol):
wss://ws.pusherapp.com:[port]/app/[key]
Related
I have created a bare-bones HTTP proxy that performs HTTP tunnelling using HTTP CONNECT method.
const http = require('http');
const https = require('https');
const pem = require('pem');
const net = require('net');
const util = require('util');
const createHttpsServer = (callback) => {
pem.createCertificate({
days: 365,
selfSigned: true
}, (error, {serviceKey, certificate, csr}) => {
const httpsOptions = {
ca: csr,
cert: certificate,
key: serviceKey
};
const server = https.createServer(httpsOptions, (req, res) => {
// How do I know I here whats the target server port?
res.writeHead(200);
res.end('OK');
});
server.listen((error) => {
if (error) {
console.error(error);
} else {
callback(null, server.address().port);
}
});
});
};
const createProxy = (httpsServerPort) => {
const proxy = http.createServer();
proxy.on('connect', (request, requestSocket, head) => {
// Here I know whats the target server PORT.
const targetServerPort = Number(request.url.split(':')[1]);
console.log('target server port', targetServerPort);
const serverSocket = net.connect(httpsServerPort, 'localhost', () => {
requestSocket.write(
'HTTP/1.1 200 Connection established\r\n\r\n'
);
serverSocket.write(head);
serverSocket.pipe(requestSocket);
requestSocket.pipe(serverSocket);
});
});
proxy.listen(9000);
};
const main = () => {
createHttpsServer((error, httpsServerPort) => {
if (error) {
console.error(error);
} else {
createProxy(httpsServerPort);
}
});
};
main();
The server accepts a HTTPS connection and responds with "OK" message without forwarding the request further.
As you can see in the code (see // Here I know whats the target server PORT.), I can obtain the target server's port within the HTTP CONNECT event handler. However, I am unable to figure out how to pass this information to the createHttpsServer HTTP server router (see // How do I know I here whats the target server port?).
When tunnelling a TLS connection, how to pass additional information?
The above code can be tested by running:
$ node proxy.js &
$ curl --proxy http://localhost:9000 https://localhost:59194/foo.html -k
The objective is to respond with "OK localhost:59194".
You can't add anything to a TLS stream (thankfully), short of tunneling it inside another protocol—which is what the Connect method already does. But, since you have the HTTP proxy and the HTTPS server in the same codebase, you don't need to fling the TLS stream over the network another time. Instead, you want to parse the TLS stream, and then you can pass any variables to the code that handles it.
However, after parsing TLS you'll still have a raw HTTP stream, and you'll need an HTTP server to turn it into requests and to handle responses.
The quick and rather dirty way to go about it is to use Node's HTTPS server to both decode TLS and parse HTTP. But the server's API doesn't provide for dealing with sockets that are already connected, and server's code isn't cleanly separated from connection code. So you need to hijack the server's internal connection-handling logic—this is of course susceptible to breakage in case of future changes:
const http = require('http');
const https = require('https');
const pem = require('pem');
const createProxy = (httpsOptions) => {
const proxy = http.createServer();
proxy.on('connect', (request, requestSocket, head) => {
const server = https.createServer(httpsOptions, (req, res) => {
res.writeHead(200);
res.end('OK');
});
server.emit('connection', requestSocket);
requestSocket.write('HTTP/1.1 200 Connection established\r\n\r\n');
});
proxy.listen(9000);
};
const main = () => {
pem.createCertificate({
days: 365,
selfSigned: true
}, (error, {serviceKey, certificate, csr}) => {
createProxy({
ca: csr,
cert: certificate,
key: serviceKey
});
});
};
main();
To avoid creating an HTTPS server instance on every request, you can move the instance out and tack you data onto the socket object instead:
const server = https.createServer(httpsOptions, (req, res) => {
res.writeHead(200);
// here we reach to the net.Socket instance saved on the tls.TLSSocket object,
// for extra dirtiness
res.end('OK ' + req.socket._parent.marker + '\n');
});
proxy.on('connect', (request, requestSocket, head) => {
requestSocket.marker = Math.random();
server.emit('connection', requestSocket);
requestSocket.write('HTTP/1.1 200 Connection established\r\n\r\n');
});
With the above code, if you do several successive requests:
curl --proxy http://localhost:9000 https://localhost:59194/foo.html \
https://localhost:59194/foo.html https://localhost:59194/foo.html \
https://localhost:59194/foo.html https://localhost:59194/foo.html -k
then you'll also notice that they're processed on a single connection, which is nice:
OK 0.6113572936982015
OK 0.6113572936982015
OK 0.6113572936982015
OK 0.6113572936982015
OK 0.6113572936982015
I can't quite vouch that nothing will be broken by handing the socket to the HTTPS server while the proxy server already manages it. [The server has the presence of mind to not overwrite another instance on the socket object](https://github.com/nodejs/node/blob/v10.9.0/lib/_http_server.js#L331), but otherwise seems to be rather closely involved with the socket. You'll want to test it with longer-running connections.
As for the `head` argument, [which can indeed contain initial data](https://www.rfc-editor.org/rfc/rfc2817#section-5.2):
you might be able to put it back on the stream with requestSocket.unshift(head), but I'm not sure that it won't be immediately consumed by the proxy server.
Or, you might be able to chuck it over to the HTTPS server with requestSocket.emit('data', head) since the HTTP server seems to use the stream events, however TLS socket source calls read() for whatever reason, and that's mutually exclusive with the events, so I'm not sure how they even work with each other.
One solution would be to make your own wrapper for stream.Duplex that will forward all calls and events, except for read() in the case when this initial buffer exists—and then use this wrapper in place of requestSocket. But you'll then need to replicate the 'data' event also, in accordance with the logic of Node's readable streams.
Finally, you can try creating a new duplex stream, write head and pipe the socket to it, like you did initially, and use the stream in place of the socket for the HTTPS server—not sure that it will be compatible with HTTP server's rather overbearing management of the socket.
An cleaner approach is to decode the TLS stream and use a standalone parser for the resultant HTTP stream. Thankfully, Node has a tls module that is nicely isolated and turns TLS sockets into regular sockets:
proxy.on('connect', (request, requestSocket, head) => {
const httpSocket = new tls.TLSSocket(requestSocket, {
isServer: true,
// this var can be reused for all requests,
// as it's normally saved on an HTTPS server instance
secureContext: tls.createSecureContext(httpsOptions)
});
...
});
See caveats on tls.createSecureContext regarding replicating the behavior of the HTTPS server.
Alas, Node's HTTP parser isn't so usable: it's a C library, which necessitates quite a bit of legwork between the socket and the parser calls. And the API can (and does) change between versions, without warnings, with a larger surface for incompatibilities compared to the HTTP server internals used above.
There are NPM modules for parsing HTTP: e.g. one, two, but none seem too mature and maintained.
I also have doubts about the feasibility of a custom HTTP server because network sockets tend to require plenty of nurture over time due to edge cases, with hard-to-debug timeout issues and such—which should all be already accounted for in the Node's HTTP server.
P.S. One possible area of investigation is how the Cluster module handles connections: afaik the parent process in a cluster hands connection sockets over to the children, but it doesn't fork on every request—which suggests that the child processes somehow deal with connected sockets, in code that's outside of an HTTP server instance. However, since the Cluster module is now in the core, it may exploit non-public APIs.
My problem is that the current solution I have for sending a specific socket using the library "ws" with node.js is not good enough.
The reason is because if I connect with multiple tabs to the websocket server with the same userid which is defined on the client-side, it will only refer to the latest connection with the userid specified.
This is my code:
// Server libraries and configuration
var server = require("ws").Server;
var s = new server({ port: 5001});
// An array which I keep all websockets clients
var search = {};
s.on("connection", function(ws, req) {
ws.on("message", function(message){
// Here the server process the user information given from the client
message = JSON.parse(message);
if(message.type == "userinfo"){
ws.personName = message.data;
ws.id = message.id;
// Defining variable pointing to the unique socket
search[ws.id] = ws;
return;
}
})
})
As you can see, each time a socket with same id connects, it will refer to the latest one.
Example If you did not understand:
Client connect to server with ID: 1337
search[1337] defined as --> websocket 1
A new connection with same ID: 1337
search[1337] becomes instead a variable refering to websocket 2 instead
Websockets provide a means to create a low-latency network "socket" between a browser and a server.
Note that the client here is the browser, not a tab on a browser.
If you need to manage multiple user sessions between the browser and server, you'll need to write code to do it yourself.
Does anyone know if it is possible to send basic http authentication credentials with EventSource?
I'm looking for a solution to the same problem. This post here says this:
Another caveat is that as far as we know, you cannot change the HTTP
headers when using EventSource, which means you have to submit an
authorization query string param with the value that you would have
inserted using HTTP Basic Auth: a base64 encoded concatenation of your
login and a token.
Here is the code from the post:
// First, we create the event source object, using the right URL.
var url = "https://stream.superfeedr.com/?";
url += "&hub.mode=retrieve";
url += "&hub.topic=http%3A%2F%2Fpush-pub.appspot.com%2Ffeed";
url += "&authorization=anVsaWVuOjJkNTVjNDhjMDY5MmIzZWFkMjA4NDFiMGViZDVlYzM5";
var source = new EventSource(url);
// When the socket has been open, let's cleanup the UI.
source.onopen = function () {
var node = document.getElementById('sse-feed');
while (node.hasChildNodes()) {
node.removeChild(node.lastChild);
}
};
// Superfeedr will trigger 'notification' events, which corresponds
// exactly to the data sent to your subscription endpoint
// (webhook or XMPP JID), with a JSON payload by default.
source.addEventListener("notification", function(e) {
var notification = JSON.parse(e.data);
notification.items.sort(function(x, y) {
return x.published - y.published;
});
notification.items.forEach(function(i) {
var node = document.getElementById('sse-feed');
var item = document.createElement("li");
var t = document.createTextNode([new Date(i.published * 1000), i.title, i.content].join(' '));
item.appendChild(t);
node.insertBefore(item, node.firstChild);
// We add the element to the UI.
});
});
If your talk about cookies (not http auth):
EventSource uses http, so cookies are sent with the EventSource connection request.
Http auth should be supported as any other http url, although from the spec CORS+http auth is not supported.
Nowadays there is a NPM package to change the HTTP Header
https://www.npmjs.com/package/eventsource
This library is a pure JavaScript implementation of the EventSource
client. The API aims to be W3C compatible.
You can use it with Node.js or as a browser polyfill for browsers that
don't have native EventSource support.
You can use event-source-polyfill to add headers like this
import { EventSourcePolyfill } from 'event-source-polyfill';
new EventSourcePolyfill(`/api/liveUpdate`, {
headers: {
Authorization: `Bearer 12345`,
'x-csrf-token': `xxx-xxx-xxx`,
},
});
EventSource is about the server sending events to the client. I think you need bidirectional communication for authentication. How would you otherwise send the actual credentials?
WebSockets, however, can achieve that. Is that what you are looking for?
Update:
You can achieve what you want by utilizing cookies, as pointed out by 4esn0k. Cookies are sent along with the initial request that the browser makes to establish the connection. So, just make sure you set the session identifier for the cookie before launching any EventSource connections.
Here's the deal: I'm using node.js to connect to a local CouchDB instance.
The problem: the "response" event isn't firing. So I'm assuming it's not getting a connection.
var http = require('http'),
sys = require('sys'),
url = require('url');
var server = http.createServer(function (request, response) {
var client = http.createClient(5984, '127.0.0.1'); // couchdb
var request_db = client.request('GET', '/_all_dbs');
console.log('Created a request for _all_dbs');
request_db.addListener('response', function(response_db) {
console.log('Some response from CouchDB');
});
}).listen(8124);
console.log('Server running at http://127.0.0.1:8124/');
The only only output is:
Server running at http://127.0.0.1:8124/
Created a request for _all_dbs
I'm also guessing this might be a proxy problem. In Ubuntu 10.10, I have a system-wide proxy setting, but I tell it to ignore 127.0.0.1. Doing a curl on 127.0.0.1:5984/_all_dbs gives an error on "Error Code: 502 Proxy Error.".
First, try accessing 127.0.0.1:8124/_utils as #PartlyCloudy suggests.
Second, try using a couchdb lib such as cradle to make your life easier (and minimize mistakes)
npm install cradle
Remember to stop into #node.js and ask questions! Make sure to report back with your findings.
If you are using request to make the http request set proxy as empty string.. It will take preference over the system settings (you can do it only when the target ip is current machine ip)
var options = {
agent: ...,
url: ...,
method: ...,
proxy: ''
};
var req= request(options);
.
.
.
I'm assuming this isn't possible, but wanted to ask in case it is. If I want to provide a status information web page, I want to use WebSockets to push the data from the server to the browser. But my concerns are the effect a large number of browsers will have on the server. Can I broadcast to all clients rather than send discrete messages to each client?
WebSockets uses TCP, which is point to point, and provides no broadcast support.
Not sure how is your client/server setup, but you can always just keep in the server a collection of all connected clients - and then iterate over each one and send the message.
A simple example using Node's Websocket library:
Server code
var WebSocketServer = require('websocket').server;
var clients = [];
var socket = new WebSocketServer({
httpServer: server,
autoAcceptConnections: false
});
socket.on('request', function(request) {
var connection = request.accept('any-protocol', request.origin);
clients.push(connection);
connection.on('message', function(message) {
//broadcast the message to all the clients
clients.forEach(function(client) {
client.send(message.utf8Data);
});
});
});
As noted in other answers, WebSockets don't support multicast, but it looks like the 'ws' module maintains a list of connected clients for you, so it's pretty easy to iterate through them. From the docs:
const WebSocketServer = require('ws').Server;
const wss = new WebSocketServer({ port: 8080 });
wss.broadcast = function(data) {
wss.clients.forEach(client => client.send(data));
};
Yes, it is possible to broadcast messages to multiple clients.
In Java,
#OnMessage
public void onMessage(String m, Session s) throws IOException {
for (Session session : s.getOpenSessions()) {
session.getBasicRemote().sendText(m);
}
}
and here it is explained.
https://blogs.oracle.com/PavelBucek/entry/optimized_websocket_broadcast.
It depends on the server-side really. Here's an example of how it's done using Tomcat7:
Tomcat 7 Chat Websockets Servlet Example
and an explanation of the how it's constructed here.
Yes you can and there are many socket servers out there written in various scripting languages that are doing it.
The Microsoft.Web.WebSockets namespace has a WebSocketCollection with Broadcast capability. Look for the assembly in Nuget. The name is Microsoft.WebSockets.