I'm hosting a website on git pages that opens a video stream on certain clicks over a secure websocket (wss) using this library: github.com/websockets/ws. When connecting video from an http page over a simple websocket (ws) everything was fine, but I realized the github page will be hosted over https and firefox does not allow unsecured websocket connections over https, wss is required.
To do that I followed the steps on the websockets ws page and the websocket server is sort of "hiding" behind an https server that does the certificate handshake. The problem is this: If I use my web page normally and open the link that starts a wss connection to the video it doesn't work, the server doesn't know it exists. If I first visit the server directly via IP and port using an https://IP:999 link it retrieves the certs and from then on the website works indefinitely.
Is there a way to do this naturally that doesn't require me trying to visit a headless server to do the cert handshake? I just want to open a wss connection but the overhead of having to visit a server directly for certs seems a bit bizarre.
The server setup looks like this:
const server = https.createServer({
cert: fs.readFileSync('./cert/cert.pem'),
key: fs.readFileSync('./cert/key.pem'),
}, function (req, res) {
console.log(new Date() + ' ' +
req.connection.remoteAddress + ' ' +
req.method + ' ' + req.url);
res.writeHead(200);
res.end("hello foobarbackend\n");
});
this.wsServer = new ws.Server({
server
})
this.wsServer.on("connection", (socket, request) => {
return this.onSocketConnect(socket, request)
})
server.listen(9999, '0.0.0.0');
Once the certs are retrieved via https://:9999 I can then play videos no problem on that browser at wss://:9999, I must be missing something.
The question here is answered entirely by the world of SSL/TLS. The issue here was that secure browsers will pretty much silently reject WSS:// connections to servers with self signed certificates. My certificates were self signed.
This is why the user would have to navigate to the server IP directly over HTTPs first and then accept the warnings. From there it's business as usual.
What needed to be done was register a domain name for the IP the server was located on (Droplet). Then certbot was used to generate real certificates (key, cert) for the domain. I replaced the cert.pem and key.pem above with the true generated ones. Domain name can be anything like mywebsitewhatever.app.
Now on the client side you can open up a connection to wss://mywebsitewhatever.app:9999 and the browser will accept it automatically and things work. No warnings or navigation to a warning page to accept.
Related
I am making a website that accesses the devices sensors and sends them via socket.io to my local machine.
I can't access the device sensors without HTTPS, so I have to use HTTPS for the website, which is why I uploaded my site to Heroku. The problem is the localhost server I open on my computer is HTTP, and my HTTPS website can't send data from HTTPS (heroku site) to HTTP (local machine: localhost). Is there any way I can share the data between them?
This is the code used to connect to localhost written on the heroku client side site:
const socket = io("https://192.168.1.15:16", { transports: ['websocket', 'polling', 'flashsocket']});
While this is what I use in my local machine:
const httpServer = require("http").createServer(app);
const io = require("socket.io")(httpsServer,{
});
as per socket.io documentation
I get this error:
Mixed Content: The page at '**The website**' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint 'ws://192.168.1.35:16/socket.io/?EIO=4&transport=websocket'. This request has been blocked; this endpoint must be available over WSS.
Like #R3FL3CT says, it's most likely a CORS issue - check your console to be sure.
It seems that the initial request that it makes is the one getting blocked. For example,
const socket = io('wss://echo.websocket.org/');
socket.on("connection", (socket) => {
console.log(`Connected!`)
});
Would get blocked with the error
Access to XMLHttpRequest at 'https://echo.websocket.org/socket.io/?EIO=4&transport=polling&t=Nb17pKo' from origin 'http://127.0.0.1:5501' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Whereas just using a pure WebSocket
const websocket = new WebSocket('wss://echo.websocket.org/');
websocket.onopen = e => {
console.log(`Connected!`);
}
Connected okay.
So your solution is to either roll back to an earlier version of socket.io that doesn't force cors (before v3), or just use your own WebSocket - example: https://www.websocket.org/echo.html
Here's one way you could do something to try and communicate. If you host your own version of CORS Anywhere, you can communicate with http:// websites. I have a working link you can use, if you don't want to have to host one, but here the Github is. The way it works, is that you append the URL to the end of your URL for CORS Anywhere.
e.g https://cors.anywhere.com/google.com
Issue
So i created a simple http web server with node.js and express (mostly its just the skeleton from the express application generator). Then i uploaded the server to an AWS Beanstalk web environment.
The issue is that i can't load the ressources (CSS and javascript) from the server if i connect to it.
i get a
net::ERR_CONNECTION_TIMED_OUT
for loading all ressources if i open the site in my browser.
I assume the issue is that the get request on beanstalk uses a "https" url,
Request URL: https://...elasticbeanstalk.com/javascripts/GameLogic.js
Because it works on my localhost but there it uses a "http" url.
Request URL: http://localhost:3000/javascripts/GameLogic.js
Also the html site itself loads (after the timeout of the ressources) but this also uses a "http" request
Request URL: http://....elasticbeanstalk.com/
can you change the header request url (for CSS, JS) in AWS Beanstalk Web-Environment to use http instead of https? Or change it in HTML or on Node.js?
info
The server uses the node.js helmet module.
Then i just send my html page on incoming requests:
app.get('/', function(req, res, next) {
res.sendFile(path.join(__dirname + "/public/main.html")); //Um bei / als pfad die main.html zu geben
});
In the html page i request the ressources:
<link rel="stylesheet" type="text/css" href="\stylesheets\style.css">
<script src="/javascripts/jquery.min.js"></script>
<script src="/javascripts/GameLogic.js"></script>
solution attempts
I have tried not using helmet, but that version behaves exactly the same and doesn't load ressource if it is on beanstalk (on localhost server it always worked).
I also tried chaning some security group rules to allow https on port 443 from all sources to the loadbalancer and https on port 443 from the loadbalancer to the ec2 instance. Situation didn't change.
then i tried redirecting https requests to http
app.get('/', function(req, res, next){
console.log("redirect to http?");
res.redirect('http://' + req.headers.host + req.url);
});
But then the site doesn't even load the html because of "to many redirects".
So currently im out of ideas on how to make the https request work or how to change it to an http request.
note
I am also using a student account so i have no rights to use the AWS Certificate Manager or redirect ELBs to HTTPS, if that has anything to do with it.
OK it was the helmet module for node.js at fault. I dont undertand it good enough to say what exactly changed the headers to https, hsts seems to be only part of it.
But after completly removed it, i and other people could access the web app and load the ressources.
The reason why i didn`t catch it earlier in my tests:
I never deleted or regenerated my package-lock after removing helmet. So it was still in there. Now i uninstalled it and made a new package-lock before uploading to AWS Beanstalk.
Let's say there is a website called https://example.com. A DNS lookup on the domain returns the following A records:
1.1.1.1 A example.com
1.1.1.2 A example.com
1.1.1.3 A example.com
1.1.1.4 A example.com
Each one of these addresses has the SSL certificate for example.com and is running a websocket server on port 443.
A client can connect to one of the servers using something like
var wss = new WebSocket("wss://example.com");
This will connect the client to a server based on DNS Round Robin: the client has no control over what server she connects to.
My question is the following:
How can a client connect to a specific address without having certificate issues?
For example, if the client knows that she wishes to connect to 1.1.1.4 and tries to connect with
var wss = new WebSocket("wss://1.1.1.4");
she will encounter an ssl error since the server cannot have the individual address as an alternative name in its certificate.
Is it possible to specify the hostname along with an individual address when making the connection; something like
var wss = new WebSocket("wss://example.com?ip=1.1.1.4");
I believe you can do this using CURL with something like curl https:// DOMAIN.TLD --resolve 'DOMAIN.TLD:443:IP_ADDRESS'
To avoid the certificate issue, request a domain name instead of an IP. I suggest this :
1.1.1.1 A example.com
1.1.1.2 A example.com
1.1.1.3 A example.com
1.1.1.4 A example.com
1.1.1.1 A a.example.com
1.1.1.2 A b.example.com
1.1.1.3 A c.example.com
1.1.1.4 A d.example.com
and one single SAN certificate for all of the 5 domains (example.com + [a-d].example.com)
You can test any of [a-d].example.com individually, or example.com, all will respond with the same certificate and there's no issue I think.
I am working on a gps tracking system and have built a server on node js.
This is how the file looks like for reference.
const net = require('net');
const lora_packet = require('lora-packet');
const dataParser = require('./dataParser');
const clients = [];
net.createServer(function(socket) {
socket.name = socket.remoteAddress + ":" + socket.remotePort;
clients.push(socket);
socket.on('data', function(data) {
console.log("Buffer sent by terminal : ");
console.log(data);
const packet = lora_packet.fromWire(data, 'hex');
const str = packet._packet.PHYPayload.toString('hex');
dataParser.parse_data(str, socket);
});
socket.on('end', function() {
clients.splice(clients.indexOf(socket), 1);
//broadcast(socket.name + "has left the cartel.\n");
});
function broadcast(message, sender) {
clients.forEach(function(client) {
if (client === sender) client.write(message + "\nsent\n");
return;
client.write(message);
});
process.stdout.write(message);
}
}).listen(8080);
console.log("cartel is running on the port 8080\n");
This server file handles only requests from the hardware and processes raw tcp/ip requests.
I want the server to handle http requests also and want to incorporate routing feature in the server too for client side applicarions for browser.
1) Is there any way that http requests can also be handled by the same server or should I open another port and deploy an express node js app on that?
2) If I use the same 8080 port for http, how can the routing be achieved?
3) If I use different ports for http and raw tcp/ip, what would be the best way for communication between the two server. The communication between tcp/ip server and http server should happen via socket(sending data dynamically).
From http server using socket, data has to be sent dynamically to browser to update live location
So is the flow right?
Hardware (<---->)TCP/IP server(<--->)Http server(<--->)Browser
If more information is needed to solve the query, I'll provide with that!
Thank you
It's very complicated to try to speak multiple protocols on the same port. It requires some sort of scheme at the beginning of each connection to sample the incoming data and identify which protocol it is and then shunt that connection off to the right code to handle that protocol. I wouldn't suggest it.
It is way, way easier to just open a second server on a different port for an Express server to field your http requests. Very simple. You can do it right in the same app. Because both servers can be in the same app, you can just directly read from one connection and write to the other. There's no need for interprocess communication.
Is there any way that http requests can also be handled by the same server or should I open another port and deploy an express node js app on that?
Open another port. No need to write another app unless you have a specific reason to use two processes. You can put both the plain TCP server and the Express server in the same node.js app.
If I use the same 8080 port for http, how can the routing be achieved?
It's not easy. Not suggest to use the same port for multiple protocols.
If I use different ports for http and raw tcp/ip, what would be the best way for communication between the two server. The communication between tcp/ip server and http server should happen via socket(sending data dynamically).
You can put both servers in the same node.js app and then you can just read/write directly from one to the other with the same code. No need for interprocess communication.
From http server using socket, data has to be sent dynamically to browser to update live location
Sending data dynamically to a browser usually means you want the browser to hold something like a webSocket or socket.io connection to your server so you can then send data to the browser at any time over the existing connection. Otherwise, you would have to "wait" for the browser to request data and then respond with the data when it asks.
So I'm using localtunnel to expose my ports over the internet, but I only want to let devices on the same network as the server access the server.
I'm using express-ip-filter to filter away anything that's on a different network. I tried a few things: first I tried using 192.168.1.0/24 as the only ips that could access the website, but that didn't work, as it didn't let anything in. I then tried using the ip I got from WhatsMyIp, but that wouldn't let any device in. I then found out that express-ip-filter spits out a message saying that a certain ip was not allowed and, on every device, independently on the network it was connected to, the address was 127.0.0.1. I tried confirming by only allowing 127.0.0.1, and then every device could access the server. Why would ip-filter only get 127.0.0.1 as ip? Here's my code as a reference:
// Init dependencies
var express = require('express'),
ipfilter = require('express-ipfilter').IpFilter
app = express()
// Blacklist the following IPs
var ips = ['192.168.1.0/24']
// Create the server
app.use(ipfilter(ips, { mode: "allow" }))
app.get('/', function (req, res) {
res.send('Hi')
})
app.listen(8080, () => console.log('Up'))
From my limited understanding of localtunnel it seems like it proxies users requests to you via the localtunnel software which causes all users to have the same IP. In laymans terms:
User connects to your site through localtunnel
localtunnel copies the users request and sends it to your computer
Your application receives the request but it looks like all traffic is coming from localtunnel because it's incredibly difficult if not impossible for localtunnel to imitate someone else's IP.
Why use localtunnel at all if you only want devices on the same network to connect, you don't need to do any port forwarding or DNS setup if you just want to access another machine on the same local network.
If you really do need to tunnel connections then there is a solution, not with localtunnel(Which as far as i can tell does not use forwading headers, although if someone knows if they do ill change my answer) but using https://ngrok.com instead which does exactly the same thing but also sends a little extra bit of data in every request which tells the application what the clients actual IP is.
Install ngrok
Run ngrok http -subdomain=(the subdomain you want) 80
Edit your application code to find the real client IP
var findProxyIP = function(req) {
var realIP = req.header('x-forwarded-for');
return realIP;
}
app.use(ipfilter(ips, {
mode: "allow",
detectIP: findProxyIP
}));
ngrok is much more complex and has a lot more features compared to localtunnel, however, it is freemium software and its free plan is quite limiting.