How to handle partial connection? - javascript

I try to create an reverse proxy that work bidirectional. its different from apache that stateless reverse proxy. its means, device can host webserver without public ip. so far i can reverse static content without any problem, css, javascript. but, something like media that return 206 code. it broken in middle. it stop reverse proxy, and try to keep run the server again and again. and i got the video load full. what the problem? i think, it okay already. cause, i jsut forward everything as binary. not specific coding.
reverse proxy port 80 forward to express server in port 8080
import net from 'net';
const server = new net.Server();
server.on('connection', socket => {
socket.on('data', data => {
const client = new net.Socket();
client.connect({
host: '127.0.0.1',
port: 8080
});
client.write(data);
client.on('data', data => {
socket.write(data)
})
})
})
server.listen(80, () => {
console.log('server is running')
});

Related

NodeJS HTTPS server serving over HTTP instead of HTTPS when running inside Docker container

I'm creating a website using NextJS and Docker so that I can easily deploy it. I used npx-create-next-app to initialize it and used this Dockerfile (slightly modified) to containerize it. Since I wanted to use SSL with my server without going through the hassle of setting up a proxy, I followed this article, and setup the custom server.
This worked fine when I ran it outside of a docker container, and performed as expected, serving over HTTPS. However when I containerized it, and tried to open the webpage over HTTPS, I came up with SSL_ERROR_RX_RECORD_TOO_LONG, but I could open the page using just HTTP (which I could not do when running outside of a container). Some googling led me to this question, from which I concluded that when running outside of a docker container, the custom server runs the server over HTTPS, as expected, however when I containerize it, it starts running HTTP, even though no code has been changed.
I'd expect the behavior to be the same when running locally or containerized.
At first I assumed this was due to invalid key and cert values in httpsOptions however I wasn't able to find anything that would make them invalid, and I don't see how that would cause this strange behavior. I tried changing the Docker run environment from node:alpine-16 to just node:latest to see if it had something to do with the parent image, but that was fruitless.
One other minor issue I had is that console.log does not seem to output to the container's log for some reason, I tried googling this but didn't find much of anything pertaining to it. This has made debugging much harder as I can't really output any debug data. The only log I get when running inside of a container is Listening on port 3000 url: http://localhost:3000, which I assume is output by some library/package as it isn't anywhere in my code.
Here is my custom server code in case it would be helpful:
const https = require('https');
const fs = require('fs');
const { parse } = require('url');
const next = require('next');
const dev = process.env.NODE_ENV !== 'production';
const hostname = "127.0.0.1";
const port = process.env.PORT || 3000
const app = next({ dev, hostname, port })
const handle = app.getRequestHandler()
const httpsOptions = {
key: fs.readFileSync('./cert/privkey.pem'),
cert: fs.readFileSync('./cert/fullchain.pem')
};
app.prepare().then(() => {
https.createServer(httpsOptions, async (req, res) => { // When running on docker this creates an HTTP server instead of HTTPS
const parsedUrl = parse(req.url, true)
const { pathname, query } = parsedUrl
await handle(req, res, parsedUrl)
}).listen(port, (err) => {
if(err) throw err
console.log(`Ready on https://localhost:${port}`)
})
})
Link to a reproducible example here.
The thing is, based on your sample repo, that your server.js file that is in the root of your repo gets overwritten in the image because of this line in the Dockerfile:
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
So the actual server.js that is running in the container is the server.js that is created by the yarn build command and it looks like this (you can exec into the container and see it for yourself):
const NextServer = require('next/dist/server/next-server').default
const http = require('http')
const path = require('path')
process.env.NODE_ENV = 'production'
process.chdir(__dirname)
// Make sure commands gracefully respect termination signals (e.g. from Docker)
// Allow the graceful termination to be manually configurable
if (!process.env.NEXT_MANUAL_SIG_HANDLE) {
process.on('SIGTERM', () => process.exit(0))
process.on('SIGINT', () => process.exit(0))
}
let handler
const server = http.createServer(async (req, res) => {
try {
await handler(req, res)
} catch (err) {
console.error(err);
res.statusCode = 500
res.end('internal server error')
}
})
const currentPort = parseInt(process.env.PORT, 10) || 3000
server.listen(currentPort, (err) => {
if (err) {
console.error("Failed to start server", err)
process.exit(1)
}
const nextServer = new NextServer({
hostname: 'localhost',
port: currentPort,
dir: path.join(__dirname),
dev: false,
customServer: false,
conf: {"env":{},"webpack":null,"webpackDevMiddleware":null,"eslint":{"ignoreDuringBuilds":false},"typescript":{"ignoreBuildErrors":false,"tsconfigPath":"tsconfig.json"},"distDir":"./.next","cleanDistDir":true,"assetPrefix":"","configOrigin":"next.config.js","useFileSystemPublicRoutes":true,"generateEtags":true,"pageExtensions":["tsx","ts","jsx","js"],"target":"server","poweredByHeader":true,"compress":true,"analyticsId":"","images":{"deviceSizes":[640,750,828,1080,1200,1920,2048,3840],"imageSizes":[16,32,48,64,96,128,256,384],"path":"/_next/image","loader":"default","loaderFile":"","domains":[],"disableStaticImages":false,"minimumCacheTTL":60,"formats":["image/webp"],"dangerouslyAllowSVG":false,"contentSecurityPolicy":"script-src 'none'; frame-src 'none'; sandbox;","remotePatterns":[],"unoptimized":false},"devIndicators":{"buildActivity":true,"buildActivityPosition":"bottom-right"},"onDemandEntries":{"maxInactiveAge":15000,"pagesBufferLength":2},"amp":{"canonicalBase":""},"basePath":"","sassOptions":{},"trailingSlash":false,"i18n":{"locales":["en"],"defaultLocale":"en"},"productionBrowserSourceMaps":false,"optimizeFonts":true,"excludeDefaultMomentLocales":true,"serverRuntimeConfig":{},"publicRuntimeConfig":{},"reactStrictMode":true,"httpAgentOptions":{"keepAlive":true},"outputFileTracing":true,"staticPageGenerationTimeout":60,"swcMinify":true,"output":"standalone","experimental":{"middlewarePrefetch":"flexible","optimisticClientCache":true,"manualClientBasePath":false,"legacyBrowsers":false,"newNextLinkBehavior":true,"cpus":7,"sharedPool":true,"profiling":false,"isrFlushToDisk":true,"workerThreads":false,"pageEnv":false,"optimizeCss":false,"nextScriptWorkers":false,"scrollRestoration":false,"externalDir":false,"disableOptimizedLoading":false,"gzipSize":true,"swcFileReading":true,"craCompat":false,"esmExternals":true,"appDir":false,"isrMemoryCacheSize":52428800,"fullySpecified":false,"outputFileTracingRoot":"","swcTraceProfiling":false,"forceSwcTransforms":false,"largePageDataBytes":128000,"enableUndici":false,"adjustFontFallbacks":false,"adjustFontFallbacksWithSizeAdjust":false,"trustHostHeader":false},"configFileName":"next.config.js"},
})
handler = nextServer.getRequestHandler()
console.log(
'Listening on port',
currentPort,
'url: http://localhost:' + currentPort
)
})
And as you see it starts a http server not a https. Also this is why the console.log("lksdfjls"); in your own server.js will not get executed.
What I would suggest is to leave node as it is, running on http://localhost:3000 and set up a reverse proxy that would forward incoming requests to this node backend that is accessible only from the reverse proxy. And of course reverse proxy would handle TLS termination. A docker compose setup would be more convenient for this so you could put the reverse proxy container (nginx for example) in the compose project too and map a directory from the docker host where your cert files are stored into the reverse proxy container at runtime - DO NOT BAKE CERTS OR ANY OTHER SECRETS INTO ANY IMAGE, not even if it is an internally used image only because it could leak out accidentally any time.
Also you could just manually run the two container with docker run but compose would make life easier it has a lot of capabilities for example you could scale compose services up and down so your backend service would run not in one but many containers. But if this would be a high load and/or business critical production stuff then you are better off with a better (real) container orchestrator like kubernetes, docker swarm, nomad etc but today as I see it the de facto container orchestrator is kubernetes.

Javascript: Can I open websocket server connection at random port

I want to create webserver socket connection at random port. And I want to return server port to calling application or just print it in terminal.
The typical code to create a server connection is as below.
const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 0 });
wss.on('connection', function connection(ws) {
ws.on('message', function incoming(message) {
console.log('received: %s', message);
});
ws.send('something');
});
So I am trying to create server at port 0. I assume it will create server at random port. How do I get that random port?
I want to know the port number, as soon as server socket is created.
Now I am able to create the websocket server at random port and able to get the port number as well. Not sure if it is the right way, but it works.
const http = require('http');
const WebSocket = require('ws');
const url = require('url');
const server = http.createServer();
const wss = new WebSocket.Server({ noServer: true });
wss.on('connection', function connection(ws) {
console.log(wss);
});
server.on('upgrade', function upgrade(request, socket, head) {
const pathname = url.parse(request.url).pathname;
wss.handleUpgrade(request, socket, head, function done(ws) {
wss.emit('connection', ws, request);
});
});
server.listen(0, '127.0.0.1', function incoming() {console.log (server.address().port);});
Websocket works with http/s on port 80 or 443. The server may listen on any port it chooses, but if it chooses any port other than 80 or 443, it may have problems with firewalls and/or proxies. Browsers generally require a secure connection for WebSockets, although they may offer an exception for local devices.

Simple example on how to use Websockets between Client and Server

I am new to websockets and just trying to get a handle of how to listen to a message from a client browser from the server and vice-versa.
I'm using a Node.js/Express setup and just want to be able to firstly listen for any messages from the client.
I've been looking at this https://github.com/websockets/ws library and have tried the examples but am not able to get this working within my localhost environment.
I'm also not clear what I need to look out for, when I'm listening for a message.
What code do I use on the client, i.e. url + port and what code do I use on the server?
const WebSocket = require('ws');
const ws = new WebSocket('ws://localhost/path', {
perMessageDeflate: false
});
Using websockets directly might be troublesome, it's advised you use a framework to abstract this layer, so they can easily fallback to other methods when not supported in the client. For example, this is a direct implementation using Express js and Websockets directly. This example also allows you to use the same server for HTTP calls.
const express = require('express');
const http = require('http');
const WebSocket = require('ws');
const app = express();
//initialize a simple http server
const server = http.createServer(app);
//initialize the WebSocket server instance
const wss = new WebSocket.Server({ server });
wss.on('connection', (ws) => {
//connection is up, let's add a simple simple event
ws.on('message', (message) => {
//log the received message and send it back to the client
console.log('received: %s', message);
ws.send(`Hello, you sent -> ${message}`);
});
//send immediatly a feedback to the incoming connection
ws.send('Hi there, I am a WebSocket server');
});
//start our server
server.listen(3000, () => {
console.log(`Server started on port ${server.address().port} :)`);
});
For the client, you can do something like this:
const ws = new WebSocket('ws://localhost:3000')
ws.onopen = () => {
console.log('ws opened on browser')
ws.send('hello world')
}
ws.onmessage = (message) => {
console.log(`message received`, message.data)
}
Like i have mentioned above, it is advised that you use a mature framework for websockets. Should your app be minimal and not need scaling, you can use any open source library, with socket.io being the most popular.
However, if you are talking about implementing this to be used at production level, you should know that the open source solutions do not allow for scalability, failover, message ordering etc. In that case, you’ll have to implement a realtime platform as a service tool.
Just a note, socket.io is a backend/frontend library that uses websocket but also has a number of fallbacks if the client browser does not support websocket. The example below works with ws backend.
Server
const WS = require('ws')
const PORT = process.env.PORT || 8080
const wss = new WS.Server({
port: PORT
}, () => console.log(`ws server live on ${PORT}`))
const errHandle = (err) => {
if(err) throw err
}
wss.on('connection', (socket) => {
console.log('something connected')
socket.send('you are connected', errHandle)
socket.on('message', (data) => {
console.log(`socket sent ${data}`)
socket.send('message received', errHandle)
})
})
client (browser)
(() => {
const ws = new WebSocket('ws://localhost:8080')
ws.onopen = () => {
console.log('ws opened on browser')
ws.send('hello world')
}
ws.onmessage = (message) => {
console.log(`message received ${message}`)
}
})()
edit: oh, and ws and http are different protocols. you will need a different server to serve your http files

How to check if WebSocket.server connection is open or closed?

Im using node js and I cant figure out how to check if my WebSocket.Server connection is Open or Closed, is there any function like socket.readyState?
Im asking because I have a problem, when me + someone else reloads the 192....../xxx in the same moment I get an error Error: listen EADDRINUSE :::3001 and I cant figure out where it blows up..
Also Id like to mention that I DO close the connection but only in one spot, here is my code example;
router.get('/', function(req, res, next) {
wss.on('connection', function connection(ws) {
ws.on('message', function incoming(message) {
console.log('Client: Im on... /zzz');
})
});
wss.on('error', function error(err) {
console.log(wss.clients);
next(err);
});
//??
//wss.clients.clear();
db.all('SELECT rowid, * FROM ZZZZZZZZ', (err, rows) => {
//idk if its ok
if (err) next(err);
if (rows.length == 0) {
res.render('xxxxx/index', {
AAA: 'empty'
});
} else {
wss.close(function(err) {
if (err) next(err);
console.log('closing websocket at /zzz');
server.close();
setTimeout(function() {
wss = new WebSocket.Server({
port: 3001
});
}, 100);
});
res.render('xxx/index', {
AAA: rows
});
}
});
});
And heres what I have above my router.get
let express = require('express');
let router = express.Router();
let sqlite3 = require('sqlite3').verbose();
let db = new sqlite3.Database('./xx/xx/xx/xxx/xx/xxx.db');
let WebSocket = require('ws');
let wss = new WebSocket.Server({ port: 3001 });
let server = wss._server;
Looks to me like you have the websocket server depend on a specific endpoint. The error you're getting is trying to tie multiple websocket servers to the same port. You only really need one websocket server to handle multiple client connections. I suggest that you would make them separate and have the websocket server run alongside of the express app.
Having endpoints to perform certain actions like closing the websocket is cool, but I would suggest authenticating them if you're doing something like that.
The error you are having is 100% server side, port 3001 is in use. It's trying to start up a websocket server on port 3001 and there is already one running. You can either use different ports, or just use the original websocket server to serve multiple clients.
To actually answer the question you asked, I believe the proper way for doing so would be to implement a heartbeat for your websocket server. That should send after n amount of minutes/seconds/milliseconds (I'd probably go with minutes). It's a good idea to have it bidirectional so your server and clients know when a connection has been dropped.

What does socket remote port represent?

http-client.js:
const http = require('http');
http.get
(
{
port : 9001,
host : 'localhost'
},
(res) =>
{
//...
}
);
tcp-server.js:
const net = require('net');
let server = new net.Server();
server.listen(9001, 'localhost', (err) =>
{
console.log('Started listening', server.address());
});
server.on('connection', (sock) =>
{
console.log(`Connected ${sock.remoteAddress}:${sock.remotePort}`);
});
I run node tc-server.js and then when I run node http-client.js I see output like:
Started listening { address: '127.0.0.1', family: 'IPv4', port: 9001 }
Connected 127.0.0.1:59506
I close http-client.js and run node http-client.js again. I see: Connected 127.0.0.1:59508
I close server and run again, and run the client again, I see Connected 127.0.0.1:59510
So the socket.remotePort is increasing all the time. What I don't understand is why those numbers for ports, I was expecting to see 9001 for port number since that's where the http request was being sent and successfully reached the listening tcp server.
Both sides of a TCP conversation have to have an address and a port. E.g., clients use ports too. What your console.log was telling you was that the client connected to your port 9001 using its port 59506. When your server sends packets to the client, it addresses them with the client's address and that port number, so the TCP layer of the network stack on the client knows what process to send the packet to. (More in the Wikipedia article on TCP.) You see the number increasing just as a byproduct of how your client system assigns available ports to connections.
You don't normally need to care about the client's port.

Categories

Resources