I am trying to connect to a node http server socket (expressJs) with net.connect in roder to pass that socket to my repl to be able to basically connect to my http server and launch commands.
when trying this I got the error EPIPE the second I started the repl.
here is the code for the repl:
const args = process.argv.slice(2);
if (args.length < 1) {
process.exit(1);
}
const url = args[0];
const [host, port] = url.split(':');
//this will get the url to connect to
const socket = net.connect(parseInt(port), host);
process.stdin.pipe(socket);
socket.pipe(process.stdout);
Console.start({
expose: { container, Metric:metricsObject},
socket:socket
});
The start function :
start(options = {}) {
const { expose, socket } = options;
const repl = REPL.start({
eval: promisableEval,
terminal:true,
input: socket,
output: socket,
});
Object.assign(repl.context, expose);
}
The http server running :
const http = this.express
.listen(this.config.web.port, () => {
const { port } = http.address();
this.logger.info(`[p ${process.pid}] Listening at port ${port}`);
resolve();
});
this.express is just an instance of express : this.express = express();
It looks like you're trying to connect to an http (or https?) server at a URL like http://mine.example.com:3000/path/item by saying
net.connect(parseInt('3000/path/item'), 'http://mine.example.com');
It won't work for a number of reasons.
Unless you're a pretty good programmer expert at the http protocol, you should not use net.connect to talk to http servers. Try using http.clientRequest instead.
hostnames passed to net.connect should be raw machine names like 'mine.example.com' and not preceded by a protocol specifier.
ports, similar.
Sorry, I don't get what you're trying to do with stdin and stdout. But your socket would not be ready for use until its connect operation completes and you get an event announcing that.
You can use the old telnet program to connect to an http server. It lets you type stuff to the server, and then displays what you get back. In your case you'd do
telnet localhost 3000 # from shell your command line
The server then connects and sits there waiting. You type
GET / HTTPS/1.0
and then two <Enter>s. The server then sends back whatever it would send if you put http://localhost:3000 into a browser location line. It then closes the connection.
I'm not sure how that http protocol operation fits into your REPL.
The example you mention at https://medium.com/trabe/mastering-the-node-js-repl-part-3-c0374be0d1bf doesn't connect to an http server, it connects to a tcp server. Http servers (including all node/express servers) are a subspecies of tcp server, but they layer the http protocol on the tcp protocol. The http protocol isn't suitable for the back-and-forth conversational style of REPLs.
Related
I'm using Express (v4.17.3), Socket.io, and Node.Js's http module. I'm adding a middleware for express to capture all incoming requests but that's failing.
I'll first show the code I'm using and the output then explain my understanding/expectation of the output (I'm new to Node and all the mentioned libraries so perhaps I'm missing something)
First of all, below is the code I'm referring to. Using Express's middleware I'm trying to capture all the incoming requests and log them, and doing the same for the http on("request"). However, requests going to socket.io aren't captured by the middleware.
// Express
const express = require("express");
const app = express()
// Socket
const server = require('http').createServer(app);
const {Server} = require("socket.io");
const io = new Server(server)
// Want to listen to all incoming requests using the middleware (this doesn't work)
app.use((req,res,next)=>{
console.log(`Express request = ${req.url}`)
next()
})
// Listening to all incoming requests (this works)
server.on("request", (req, res)=>{
console.log(`Http request = ${req.url}`)
})
server.listen(8080, () => {
console.log(`Listening on port 8080`)
})
output when I GET /
Express request = /
Http request = /
Http request = /socket.io/socket.io.js
Http request = /socket.io/?EIO=4&transport=polling&t=O0va...
Http request = /socket.io/?EIO=4&transport=polling&t=O0va24A&sid=c...
Http request = /socket.io/?EIO=4&transport=polling&t=O0va24F&sid=c...
Http request = /socket.io/?EIO=4&transport=polling&t=O0va27x&sid=c...
My expected output is to have equal logs for the middleware app.use() and on("request") ("Express request = " & "Http request = ")
My understanding:
1- When I add a middleware for express as in the code below, any incoming requests should be captured here first before going anywhere else. (correct?)
app.use((req,res,next)=>{...})
2- When I'm passing the express app as an argument to http'screateServer, that the express app will be treated as a listener and any request events will be passed to it. (correct?)
const server = require('http').createServer(app);
So if my understanding is correct, why aren't all the requests captured by the request event passed to the middleware as well?
This is normal. Socket.io puts itself in front of express (or any other listener for incoming requests on the http server) so that it takes the request before Express sees it. Thus, Express (or it's middleware) never see any socket.io connection requests.
Socket.io has its own middleware layer that you can use to participate in the initialization of socket.io requests.
Or, you can register for incoming socket.io connections (to be called after they are already connected) with the io.on('connection', ...) event handler.
When I add a middleware for express as in the code below, any incoming requests should be captured here first before going anywhere else. (correct?)
That is true except for code that registers directly request handlers right on the http server and inserts itself before Express in the listener chain, thus preventing Express from seeing any requests that are destined for socket.io.
When I'm passing the express app as an argument to http'screateServer, that the express app will be treated as a listener and any request events will be passed to it. (correct?)
That is true. But socket.io jumps in front of Express and takes/hides any requests it wants so that Express never sees them.
If you're curious, here's the socket.io code that jumps in line to the front of all listeners for the http server thus bypassing the express listener:
attachServe(srv) {
debug("attaching client serving req handler");
const evs = srv.listeners("request").slice(0);
srv.removeAllListeners("request");
srv.on("request", (req, res) => {
if (this.clientPathRegex.test(req.url)) {
this.serve(req, res);
}
else {
for (let i = 0; i < evs.length; i++) {
evs[i].call(srv, req, res);
}
}
});
}
It grabs all the existing listeners into an array. Then, it removes them all. Then, it registers for the request event itself and, if it is a socket.io request, then it does not call the prior listeners. If it is not a socket.io prefix, then it manually calls the prior listeners.
I believe you need to log the messages sent on socket.
io.on('connection', (socket) => { socket.on('chat message', (msg) => { console.log('message: ' + msg); }); });
How to get the link of the created server in the http.createServer() function?
I am trying to use the http package from node.js and use the createServer() function, then I want to print the server's link with the port, like this:
const createServer = http.createServer();
const server = createServer.listen(port);
And then get the link from the createServer variable like http://localhost or from a repl.it/replit.com link
.
this is my code in JS:
const port = 1234;
const createServer = http.createServer();
const server = createServer.listen(port);
const websocket = new web({ httpServer: server });
console.log(createServer); // This doesnt print the server link
You can get the information about the server by calling server.address():
console.dir(server.address());
//{ address: '::', family: 'IPv6', port: 1234 }
Since you didn't specify a host when creating the server, the default one will be used which according to the documentation:
If host is omitted, the server will accept connections on the unspecified IPv6 address (::) when IPv6 is available, or the unspecified IPv4 address (0.0.0.0) otherwise.
I am new to NodeJS. My code is
const webSocket= require('ws');
const express =
require('express');
const app=express();
Var url = "wss://stream.binance.com:9443/BTCUSDT#trade`"
const ws = new webSocket(url);
app.get('/',(req,res)=>{
ws.on('message',data=>{
res.send(data)
})
})
app.listen(3000)
When i use console.log instead of res.send() it works successfully and consoled the data.
How can i display these dynamic data to a browser. Thanks In Advance.
Websocket is event-driven
The syntax for a event emmiter on the server which emit the event will be like the following:
ws.emit('eventName',data);
The syntax of a event listener which receives a message will be like the following:
//For the client to receive the message, you should put something like this at the front end
ws.on('eventName',data=>{
const content = data;
document.getElementById('chat').innerHTML += content
});
Websocket is a different protocol than HTTP. It uses HTTP established the connection. The connection will then be upgraded to websocket. By the time, you shouldn't HTTP to send data anymore like res.send() but instead use ws.emit('eventName', data) for example.
Websocket Protocol
http-client.js:
const http = require('http');
http.get
(
{
port : 9001,
host : 'localhost'
},
(res) =>
{
//...
}
);
tcp-server.js:
const net = require('net');
let server = new net.Server();
server.listen(9001, 'localhost', (err) =>
{
console.log('Started listening', server.address());
});
server.on('connection', (sock) =>
{
console.log(`Connected ${sock.remoteAddress}:${sock.remotePort}`);
});
I run node tc-server.js and then when I run node http-client.js I see output like:
Started listening { address: '127.0.0.1', family: 'IPv4', port: 9001 }
Connected 127.0.0.1:59506
I close http-client.js and run node http-client.js again. I see: Connected 127.0.0.1:59508
I close server and run again, and run the client again, I see Connected 127.0.0.1:59510
So the socket.remotePort is increasing all the time. What I don't understand is why those numbers for ports, I was expecting to see 9001 for port number since that's where the http request was being sent and successfully reached the listening tcp server.
Both sides of a TCP conversation have to have an address and a port. E.g., clients use ports too. What your console.log was telling you was that the client connected to your port 9001 using its port 59506. When your server sends packets to the client, it addresses them with the client's address and that port number, so the TCP layer of the network stack on the client knows what process to send the packet to. (More in the Wikipedia article on TCP.) You see the number increasing just as a byproduct of how your client system assigns available ports to connections.
You don't normally need to care about the client's port.
My node.js server uses cluster module in order to work on multiple processes.
If the server receives requests from clients with Socket.IO, it conveys the data to another server with redis publish. And it receive refined data with redis subscribe, and then it just toss this data to clients.
I use one node process to receive data with redis sub, and other processes to send data to clients with socket.io.
And the client connect socket.io when page loaded.
Here, this is my problem.
The connect event occured repeatedly not even the page loaded.
When the client connect, I get the socket.id from that socket, and I use it later when I want to send data to that client socket. But this connect occur repeatedly, I think socket that client use changed. So, the first socket.id that I remembered will be useless. I can't send data from that socket.id. I stored auth information in the socket object, so the changed client socket is no help.
index.pug
$(document).ready(function(){
var socket = io.connect();
(...)
app.js
var cluster = require('cluster');
var socketio = require('socket.io');
var NRP = require('node-redis-pubsub');
var nrpForChat = new NRP(config.chatRedisConfig);
var nrpForCluster = new NRP(config.clusterRedisConfig);
var startExpressServer = function(){
var app = express();
var server = require('http').createServer(app);
var io = socketio.listen(server);
var redis = require('socket.io-redis');
io.adapter(redis({ host: 'localhost', port: 6380 }));
io.sockets.on('connection', function(socket){
socketController.onConnect(io, socket, nrpForChat);
});
server.listen(config.port, function(){
console.log('Server app listening on port '+config.port);
});
nrpForCluster.on('to:others:proc', function(data){
var socket = io.sockets.connected[data.target.sockid];
if (socket) {
if (data.event == '_net_auth') {
if (data.data.res){
socket.enterId = data.data.data.enterId;
socket.memberKey = data.data.data.memberKey;
socket.sid = data.data.data.sid;
socket.emit(data.event, data.data);
}else{
console.log('auth failed.');
}
}
} else {
socket.emit(data.event, data.data);
}
});
module.exports = app;
}
var numCpus = require('os').cpus().length;
if (cluster.isMaster) {
for (var i = 0; i < numCpus; i++) {
cluster.fork();
}
}
else {
if (cluster.worker.id == numCpus) {
nrpForChat.on('chat:to:relay', function(data){
nrpForCluster.emit('to:others:proc', data);
});
if (numCpus == 1) {
startExpressServer();
}
}
else {
startExpressServer();
}
}
By default, socket.io connects with several consecutive http requests. It essentially starts in HTTP polling mode and then after some initial data exchange, it switches to a webSocket transport.
Because of this, a cluster that does not have any sort of sticky load balancing will not work. Each of the initial consecutive http requests that are all supposed to go to the same server process will probably be sent to different server processes in the cluster and the initial connection will not work.
There are two solutions that I know of:
Implement some sort of sticky load balancing (in the clustering module) so that each client repeatedly goes to the same server process and thus all the consecutive http requests at the beginning of a connection will go to the same server process.
Switch your client configurations to immediately switch to the webSocket transport and never use the HTTP polling. The connection will still start with an http request (since that's how all webSocket connections start), but that exact same connection will be upgraded to webSocket so there will only ever be one connection.
FYI, you will also need to make sure that the reconnect logic in socket.io is properly reconnecting to the original server process that is was connected to.
socket.io has node.js clustering support in combination with redis. While the socket.io documentation site has been down for multiple days now, you can find some info here and Scaling Socket.IO to multiple Node.js processes using cluster and here's a previously cached version of the socket.io doc for clustering.