I'm running a Go webserver using Gorilla/websocket for a real-time chess/chat application. As far as I know, I define the address and port in two locations, the Go http.ListenAndServer() method and the Javascript new Websocket(); function. This works great accross browsers on the same host machine, but it gets tricky when trying to extend the functionality over my internal network. So I use, for instance:
Go:
PORT := "0.0.0.0:8080"
// ...
http.HandleFunc("/ws", func(w http.ResponseWriter, r *http.Request) {
h.serveWs(hub, w, r) // h is a chessboard Handler wrapper
})
//...
http.ListenAndServe(PORT, nil)
Javascript:
conn = new WebSocket("ws://0.0.0.0:8080/ws");
In my effort to get this working on different devices over the network I've tried using, instead of the 0.0.0.0 address, localhost or my hostname and my host's internal ip (e.g.) 10.232.44.20; With the latter, the internal ip hardcoded in, it works, but the other attempts don't. I thought that using 0.0.0.0 would accept all connections from within the network, which I gathered from this answer:
What is the difference between 0.0.0.0, 127.0.0.1 and localhost?
How should I program ListenAndServe as well as new Websocket so that I can access the server within the internal network, without having to hardcode my internal ip address? Should I access that number programmatically from Js and Go? In the end, I want to be able to run this server on a variety of networks, without configuration, and have it just work. What am I missing?
If the web page and websocket endpoints are served by the same host, then create the websocket using the host used to fetch the web page:
conn = new WebSocket("ws://" + window.location.host + "/ws");
This will just work in all network configurations.
As far as the Go code is concerned, the easiest approach is to listen on all networks:
if err := http.ListenAndServe(":8080", nil); err != nil {
// handle error
}
Related
I want to make server (for multiplayer game), but I can't connect to my pc through my public IP address.
My server is programmed in Java like this:
int port = 60000;
int client_num = 0;
ClientHandler clientHandler;
try (ServerSocket serverSocket = new ServerSocket(port)) {
System.out.println("Server is listening on port " + port);
while (true) {
Socket client = serverSocket.accept();
System.out.println("New client connected: "+client);
}
} catch (IOException ex) {
System.out.println("Server exception: " + ex.getMessage());
ex.printStackTrace();
}
Client is programmed in javascript like this(I am using websockets):
var websocket = new WebSocket(
'ws://127.0.0.1:60000');
websocket.onopen = function () {
$('h1').css('color', 'green');
websocket.send("Hello");
$('h1').css('color', 'purple');
};
You can see, that I use the same port (60 000) in both-server and client. In client where I write IP address (ws://127.0.0.1) I can write localhost(127.0.0.1) or my private ip (192.168.0.100) - AND IT WORKS! But when I write there my public ip (something like 91...***), it dont work and server dont write message "New client connected:" (like in case of local host do)... So I thought that problem is in port forwarding. But I tried to port forward, as you can see in image:
But it still don't work. I also tried to use DMZ in my router. Same results. I also tried to turn off firewall and web shield of my anitivirus. Again same result...Even if i try to check if my port 60 000 is open with some online test (https://www.yougetsignal.com/tools/open-ports/) it say that my port is closed...So why it don't work?
I have found out, that even if I turn on Apache which listen on ports 80 and 443, tools for checking open ports also show me this ports (80 and 443) as closed (even when I turn DMZ on my router)...WHY?
Note: I am sure, that i use right local IP for setting up port forwarding and DMZ, I use IP which give me ipconfig(and also as I already wrote - client-server communication works when I use this local address (192.168.0.100) of my pc where server runs...)
Please help me! :)
Your issue is about networking and network routing. Try to connect to your server from the outside of your local network. Somewhere from internet. Buy a VPS or ask your friend. First of all you can check your port is opened using nmap. If it were opened you are able to connect to your server. Otherwise it might be your D-LINK problem. I see you have 22 port forwarding already. When I used D-LINK DIR-600 router I have similar issue - port was adjusted correctly but I was not able to connect via opened port. And I could fix this issue only by using DD-WRT - unofficial firmware.
I need to send data out from my google assistance app to a database. In order to do this, I've created a server that takes the data, packages it, and then sends it out. I have the hostname and port and it works in a normal javascript/node.js program but when I use it in my google assistant app nothing happens. I tried figuring out the problem and it looks like the code just isn't connecting. The code I'm using to send data to the server is as follows:
function sendData(app){
var net = require('net');
var message = {"test": 200};
var thisMessage = JSON.stringify(message);
var client = new net.Socket();
client.connect(<port>, '<hostname>', function() {
app.tell(JSON.stringify(client.address()));
console.log('Connected');
client.write(thisMessage);
});
client.on('data', function(data) {
console.log('Received: ' + data);
client.destroy();
});
client.on('close', function() {
console.log('Connection closed');
});
return 0;
}
(NOTE: Port and hostname left out for privacy purposes)
This completely skips over the app.tell, leading me to believe the connection is never made. I know it works asynchronously with the server, however, I don't understand why it isn't connecting whatsoever.
I have tried it both in simulation and on my smartphone with sandbox on and off. Is there a better way to connect? Note that the server I'm connecting to is python-based.
The problem is likely that you're running it on Cloud Functions for Firebase which has a limit on outbound connections under their free "Spark" plan. With this plan, you can only connect to other Google services. This is usually a good way to start understanding how to handle Action requests, but has limitations. To access endpoints outside of Google, you need to upgrade to either their "Flame" fixed price plan or "Blaze" pay-as-you-go plan.
You do not, however, need to run on Google's servers or need to use node.js. All you need is a public HTTPS server with a valid SSL cert. If you are familiar with JSON, you can use any programming language to handle the request and response. If you are familiar with node.js, you just need a node.js server that can create Express request and response objects.
I am trying to implement a basic ldap bind with the following node.js file. Unfortunately, I keep getting a bind error with code 128. I looked online and found no references of code 128. The LDAP server I am trying to search is an eDirectory. Does anyone have any experience with this or have you had similar problems? My node version is v0.10.22 and my ldapjs version is v0.7.1
var ldap = require('ldapjs');
var creds = {
url: "ldaps://ldap.url.com:636",
bindDN: "cn=ldap,o=com"
};
var opts = {
filter: "(cn=username)",
scope: "sub"
};
function authDN(client, dn, password, cb) {
client.bind(dn, password, function (err) {
client.unbind();
cb(err === null, err);
});
}
function output(res, err) {
if (res) {
console.log('success');
} else {
console.log(['Error',err.code, err.dn, err.message ]);
}
}
var client = ldap.createClient(creds);
authDN(client, '(cn=username)', 'password', output);
This authenticates when i added the following to the top of my file:
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
I haven't researched enough to know why this works but I found this answer here: https://github.com/mikeal/request/issues/418
In general when debugging an eDirectory issue, ask for access to iMonitor, so you can look at DStrace with the +LDAP option. That would show you what the LDAP server is sending back, making troubleshooting easier.
To augment Kaiser's answer, an explanation on why adding process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0"; to the code may work is found at the top of this link: https://github.com/visionmedia/superagent/issues/205.
Potential fixes:
Add process.env.NODE_TLS_REJECT_UNAUTHORIZED = 0; to the top of your script for node v0.10.x (and above)
Setup a trusted CA certificate on the server instead of a self-signed certificate (must have server admin rights and pay for a valid cert)
Use the LDAP server IP or load balancer IP instead of dns for the url parameter.
Because you are using the secure protocol (ldaps:// instead of ldap://), and I'm assuming you are trying to connect to a server with a self-signed certificate, you will get a failure if using node v0.10.x (and probably all later versions as well) and the code/module you are using doesn't specifically set the process.env.NODE_TLS_REJECT_UNAUTHORIZED to false.
NODE_TLS_REJECT_UNAUTHORIZED was changed to true by default for a reason. If you choose to set NODE_TLS_REJECT_UNAUTHORIZED to false, you are opening up more security risks, and I would advise only doing this on private networks at best, and never in production environments. Without going down a security discussion rabbit hole, it's always best to use a cert signed by a CA. More info on the differences on certs can be found here. This can also cause problems if your application is robust enough to make multiple connections to various secured servers where only some use self signed certs, again mentioned in this link.
If the cert wasn't self-signed, you most likely shouldn't be getting this error, so another potential fix is to setup and use a trusted CA Certificate on the LDAP server instead.
On the other hand, if you are using a normal, non-secure ldap connection (not through TLS), and/or you get this error only occasionally while other times it goes through, you should try setting the ldap url to the LDAP server IP or load balancer IP (and use port 3268 to allow searching in all domains). In larger network setups this will avoid potential round robin dns queries that sometimes point you to a slow server or one you can't route to.
Okay, I've spent over a week trying to figure this out to no avail, so if anyone has a clue, you are a hero. This isn't going to be an easy question to answer, unless I am being a dunce.
I am using node-http-proxy to proxy sticky sessions to 16 node.js workers running on different ports.
I use Socket.IO's Web Sockets to handle a bunch of different types of requests, and use traditional requests as well.
When I switched my server over to proxying via node-http-proxy, a new problem crept up in that sometimes, my Socket.IO session cannot establish a connection.
I literally can't stably reproduce it for the life of me, with the only way to turn it on being to throw a lot of traffic from multiple clients to the server.
If I reload the user's browser, it can then sometimes re-connect, and sometimes not.
Sticky Sessions
I have to proxy sticky sessions as my app authenticates on a per-worker basis, and so it routes a request based on its Connect.SID cookie (I am using connect/express).
Okay, some code
This is my proxy.js file that runs in node and routes to each of the workers:
var http = require('http');
var httpProxy = require('http-proxy');
// What ports the proxy is routing to.
var data = {
proxyPort: 8888,
currentPort: 8850,
portStart: 8850,
portEnd: 8865,
};
// Just gives the next port number.
nextPort = function() {
var next = data.currentPort++;
next = (next > data.portEnd) ? data.portStart : next;
data.currentPort = next;
return data.currentPort;
};
// A hash of Connect.SIDs for sticky sessions.
data.routes = {}
var svr = httpProxy.createServer(function (req, res, proxy) {
var port = false;
// parseCookies is just a little function
// that... parses cookies.
var cookies = parseCookies(req);
// If there is an SID passed from the browser.
if (cookies['connect.sid'] !== undefined) {
var ip = req.connection.remoteAddress;
if (data.routes[cookies['connect.sid']] !== undefined) {
// If there is already a route assigned to this SID,
// make that route's port the assigned port.
port = data.routes[cookies['connect.sid']].port;
} else {
// If there isn't a route for this SID,
// create the route object and log its
// assigned port.
port = data.currentPort;
data.routes[cookies['connect.sid']] = {
port: port,
}
nextPort();
}
} else {
// Otherwise assign a random port, it will/
// pick up a connect SID on the next go.
// This doesn't really happen.
port = nextPort();
}
// Now that we have the chosen port,
// proxy the request.
proxy.proxyRequest(req, res, {
host: '127.0.0.1',
port: port
});
}).listen(data.proxyPort);
// Now we handle WebSocket requests.
// Basically, I feed off of the above route
// logic and try to route my WebSocket to the
// same server regular requests are going to.
svr.on('upgrade', function (req, socket, head) {
var cookies = parseCookies(req);
var port = false;
// Make sure there is a Connect.SID,
if (cookies['connect.sid'] != undefined) {
// Make sure there is a route...
if (data.routes[cookies['connect.sid']] !== undefined) {
// Assign the appropriate port.
port = data.routes[cookies['connect.sid']].port;
} else {
// this has never, ever happened, i've been logging it.
}
} else {
// this has never, ever happened, i've been logging it.
};
if (port === false) {
// this has never happened...
};
// So now route the WebSocket to the same port
// as the regular requests are getting.
svr.proxy.proxyWebSocketRequest(req, socket, head, {
host: 'localhost',
port: port
});
});
Client Side / The Phenomena
Socket connects like so:
var socket = io.connect('http://whatever:8888');
After about 10 seconds on logging on, I get this error back on this listener, which doesn't help much.
socket.on('error', function (data) {
// this is what gets triggered. ->
// Firefox can't establish a connection to the server at ws://whatever:8888/socket.io/1/websocket/Nnx08nYaZkLY2N479KX0.
});
The Socket.IO GET request that the browser sends never comes back - it just hangs in pending, even after the error comes back, so it looks like a timeout error. The server never responds.
Server Side - A Worker
This is how a worker receives a socket request. Pretty simple. All workers have the same code, so you think one of them would get the request and acknowledge it...
app.sio.socketio.sockets.on('connection', function (socket) {
// works... some of the time! all of my workers run this
// exact same process.
});
Summary
That's a lot of data, and I doubt anyone is willing to confront it, but i'm totally stumped, don't know where to check next, log next, whatever, to solve it. I've tried everything I know to see what the problem is, to no avail.
UPDATE
Okay, I am fairly certain that the problem is in this statement on the node-http-proxy github homepage:
node-http-proxy is <= 0.8.x compatible, if you're looking for a >=
0.10 compatible version please check caronte
I am running Node.js v0.10.13, and the phenomena is exactly as some have commented in github issues on this subject: it just drops websocket connections randomly.
I've tried to implement caronte, the 'newer' fork, but it is not at all documented and I have tried my hardest to piece together their docs in a workable solution, but I can't get it forwarding websockets, my Socket.IO downgrades to polling.
Are there any other ideas on how to get this implemented and working? node-http-proxy has 8200 downloads yesterday! Sure someone is using a Node build from this year and proxying websockets....
What I am look for exactly
I want to accomplish a proxy server (preferrably Node) that proxies to multiple node.js workers, and which routes the requests via sticky sessions based on a browser cookie. This proxy would need to stably support traditional requests as well as web sockets.
Or...
I don't mind accomplishing the above via clustered node workers, if that works. My only real requirement is maintaining sticky sessions based on a cookie in the request header.
If there is a better way to accomplish the above than what I am trying, I am all for it.
In general I don't think node is not the most used option as a proxy server, I, for one use nginx as a frontend server for node and it's a really great combination. Here are some instructions to install and use the nginx sticky sessions module.
It's a lightweight frontend server with json like configuration, solid and very well tested.
nginx is also a lot faster if you want to serve static pages, css. It's ideal to configure your caching headers, redirect traffic to multiple servers depending on domain, sticky sessions, compress css and javascript, etc.
You could also consider a pure load balancing open source solution like HAProxy. In any case I don't believe node is the best tool for this, it's better to use it to implement your backend only and put something like nginx in front of it to handle the usual frontend server tasks.
I agree with hexacyanide. To me it would make the most sense to queue workers through a service like redis or some kind of Message Query system. Workers would be queued through Redis Pub/Sub functionality by web nodes(which are proxied). Workers would callback upon error, finish, or stream data in realtime with a 'data' event. Maybe check out the library kue. You could also roll your own similar library. RabbitMQ is another system for similar purpose.
I get using socket.io if you're already using that technology, but you need to use tools for their intended purpose. Redis or a MQ system would make the most sense, and pair great with websockets(socket.io) to create realtime, insightful applications.
Session Affinity(sticky sessions) is supported through Elastic LoadBalancer for aws, this supports webSockets. A PaaS provider(Modulus) does this exactly. Theres also satalite which provides sticky sessions for node-http-proxy, however I have no idea if it supports webSockets.
I've been looking into something very similar to this myself, with the intent of generating (and destroying) Node.js cluster nodes on the fly.
Disclaimer: I'd still not recommend doing this with Node; nginx is more stable for the sort of design architecture that you're looking for, or even more so, HAProxy (very mature, and easily supports sticky-session proxying). As #tsturzl indicates, there is satellite, but given the low volume of downloads, I'd tread carefully (at least in a production environment).
That said, since you appear to have everything already set up with Node, rebuilding and re-architecting may be more work than it's worth. Therefore, to install the caronte branch with NPM:
Remove your previous http-node-proxy Master installation with npm uninstall node-proxy and/or sudo npm -d uninstall node-proxy
Download the caronte branch .zip and extract it.
Run npm -g install /path/to/node-http-proxy-caronte
In my case, the install linkage was broken, so I had to run sudo npm link http-proxy
I've got it up and running using their basic proxy example -- whether or not this resolves your dropped sessions issue or not, only you will know.
I am making an app which makes lots of api calls to some site. The trouble I've run into is that the site has a limit on the number of api calls that can be made per minute. To get around this I was hoping to use Tor in conjunction with node-http-proxy to create a proxy table which uses anonymous ip addresses taken from the tor api.
So my question is, how possible is this, and what tools would you recommend for getting it done. My app is written in javascript, so solutions involving things like node-tor are preferable.
I've found a reasonable solution using tor and curl command line tools via Node.js.
Download the tor command-line tool and set it in your $PATH.
Now, we can send requests through this local tor proxy which will establish a "circuit" through the TOR network. Let's see our IP address using http://ifconfig.me. You can copy paste all of these things into your Node REPL:
var cp = require('child_process'),
exec = cp.exec,
spawn = cp.spawn,
tor = spawn('tor'),
puts = function(err,stdo,stde){ console.log(stdo) },
child;
After this, you may want to build in a delay while the tor proxy is spawned & sets itself up.
Next, let's go through the TOR network and ask http://ifconfig.me what IP address is accessing it.
function sayIP(){
child = exec('curl --proxy socks5h://localhost:9050 http://ifconfig.me',puts);
}
sayIP();
If you want a new IP address, restarting tor by turning it off and then on seems to be the most reliable method:
function restartTor(){
tor.kill('SIGINT');
tor = spawn('tor');
}
restartTor();
Note: There is another way I've seen people describe getting a new IP address (setting up a new "circuit") on the fly, but it only seems to work about 10% of the time in my tests. If you want to try it:
Find & copy torrc.sample to torrc, then change torrc as follows:
Uncomment ControlPort 9051 (9050 is the local proxy, opening 9051 lets us control it)
Uncomment & set CookieAuthentication 0.
Uncomment HashedControlPassword and set to result of:
$ tor --hash-password "your_password"
Then you could use a function like this to send a NEWNYM signal to your local tor proxy to try getting a new IP address without restarting.
function newIP(){
var signal = 'echo -e "AUTHENTICATE \"your_password\"\r\nsignal NEWNYM\r\nQUIT" | nc -v 127.0.0.1 9051';
child = exec(signal,puts);
}
newIP();