Develop Reference

Can I print the current readline value after printing other content to the command line?

I have a simple http server that returns a 200 - success response, and a readline function that prints whatever is typed into the terminal.
Question: Is there a way to collect what has been typed and reprint it below so the user can continue typing while the server continues serving requests? Or is there a smarter way to combine stdin and stdout in a nodeJS application?
Explanation:
The server code:
const http = require("http");
const readline = require("readline");
const host = "localhost";
const port = 8080;
const server = http.createServer(async (request, response) => {
console.log("request received");
response.writeHead(200);
response.end("success");
});
server.listen(port, host, () => {
console.log(`Listening on ${port}`);
});
const lineReader = readline.createInterface({
input: process.stdin
});
lineReader.on("line", (line) => {
console.log(`Received ${line}`);
});
If I type something into the command line, it works:
[~/workspace/tests/async-prompt] node index.js
Listening on 8080
Something
Received Something
And if I visit the server, it works:
[~/workspace/tests/async-prompt] node index.js
Listening on 8080
request received
But if, while I'm typing into the terminal, a request comes in, it gets muddled up:
[~/workspace/tests/async-prompt] node index.js
Listening on 8080
typingtypingtypingrequest received
typing
Received typingtypingtypingtyping
Is there a way to print what I was in typing when interrupted by something else printing to the output? I would be satisfied with something like this:
[~/workspace/tests/async-prompt] node index.js
Listening on 8080
typingtypingtypingrequest received
typingtypingtyping_

I discovered the answer in this article: Real Time Chat with NodeJS/Readline/SocketIO. I have to use a helper function to get the current input from the terminal, then clear the line, print the incoming output, then fill the input again:
const lineReader = readline.createInterface({
input: process.stdin,
output: process.stdout,
prompt: "",
});
const safeLog = (message, ...optionalParams) => {
const currentlyTyped = lineReader.line;
process.stdout.clearLine();
process.stdout.cursorTo(0);
if (optionalParams.length > 0){
console.log(message, optionalParams);
} else {
console.log(message);
}
process.stdout.write(currentlyTyped);
}
const interval = setInterval(() => {
safeLog("Logging on timeout")
}, 2000);
lineReader.on("SIGINT", () => process.exit());
lineReader.on("line", (line) => {
safeLog(`Received ${line}`);
});
Note that when using readline like this, ctrl-c (or SIGINT) will kill the line reader, but not necessarily the rest of the app, so I added a handler for that.

Can't connect Redis server to nodejs, Docker compose

I'm struggling to connect a redis deployment to my nodejs app. Of course locally without the use of docker, it works well, so I'm at odds as to whether this is an issue to do with my code, or the way I've set up my docker compose file
Dockerfile:
FROM node:8
WORKDIR /app
COPY package.json /app
COPY . /app
RUN npm install
CMD ["npm", "start"]
EXPOSE 3000
docker-compose.yml
version: "3"
services:
web:
container_name: web-container
restart: always
depends_on:
- redis
build: .
ports:
- "3000:3000"
links:
- redis
redis:
container_name: redis-container
image: "redis:latest"
ports:
- "6379:6379"
volumes:
- ./data:/data
Redis Connection File (RedisService.js)
const redis = require("redis");
const client = redis.createClient();
const DbUtils = require("../../db_utils");
const {promisify} = require("util");
const getAsync = promisify(client.get).bind(client);
const existsAsync = promisify(client.exists).bind(client);
class RedisCache {
constructor () {
var connected;
// * Initiliase the connection to redis server
client.on("connect", () => {console.log("📒 Redis cache is ready"); connected = true;})
client.on("error", (e) => {console.log("Redis cache error:\n" + e); connected = false;});
}
async setData (id, data) {
// * Stringify data if it's an object
data = data instanceof Object ? JSON.stringify(data) : data;
client.set(id, data);
return true;
}
async getData (key) {
return getAsync(key).then(data => {
data = JSON.parse(data) instanceof Object ? JSON.parse(data) : data;
return data;
})
}
async exists (key) {
return existsAsync(key).then(bool => {
return bool;
})
}
// Returns status of redis cache
async getStatus () {
return this.connected;
}
}
module.exports = new RedisCache();
ERROR
Error: Redis connection to 127.0.0.11:6379 failed - connect ECONNREFUSED 127.0.0.11:6379

When you run your containers via docker-compose they are all connected to a common network. Service name is a DNS name of given container so to access redis container from web you should create the client like :
const client = redis.createClient({
port : 6379,
host : 'redis'
});
You have not configured the host so it uses the default one - 127.0.0.1. But from the point of view of your web container the redis is not running on the localhost. Instead it runs in it's own container which DNS name is redis.

The beginning (docker part) of this tutorial worked for me :
https://medium.com/geekculture/using-redis-with-docker-and-nodejs-express-71dccd495fd3
docker run -d --name <CONTAINER_NAME> -p 127.0.0.1:6379:6379 redis
then in the node server (like in official redis website example) :
const redis = require('redis');
async function start() {
const client = redis.createClient(6379,'127.0.0.1');
await client.connect();
await client.set('mykey', 'Hello from node redis');
const myKeyValue = await client.get('mykey');
console.log(myKeyValue);
}
start();

ECONNRESET in Express.js (Node.js) with multiple requests

Given a standard Express.js setup
const express = require('express');
const app = express();
const router = express.Router();
router.get('/test/:id', (req, res) => {
return res.status(200).json({ hello: 'world' });
});
app.use('/api', router);
app.listen(3000, () => console.info('Up on port 3000));
I am making 1000 requests agains the endpoint, one after the other:
const fetch = require('node-fetch');
for (let i = 0; i < 1000; i++) {
let id = Math.floor(Math.random() * 12) + 1;
fetch(`http://localhost:3000/api/test/${id}`).then(res => res.json()).then(data => console.log(data)).catch(error => console.error(error));
}
I do see the data returned however, every now and then I see an ECONNRESET error. The amount of ECONNRESET error messages also vary: sometimes I get a few, sometimes a lot more. I do understand the message but I can't get my head around solving the issue behind it.
Here's a sample error:
{ FetchError: request to http://localhost:3000/api/test/8 failed, reason: connect ECONNRESET 127.0.0.1:3000
at ClientRequest.<anonymous> (node_modules/node-fetch/lib/index.js:1345:11)
at ClientRequest.emit (events.js:182:13)
at Socket.socketErrorListener (_http_client.js:399:9)
at Socket.emit (events.js:182:13)
at emitErrorNT (internal/streams/destroy.js:82:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
at process.internalTickCallback (internal/process/next_tick.js:72:19)
message:
'request to http://localhost:3000/api/departments/8 failed, reason: connect ECONNRESET 127.0.0.1:3000',
type: 'system',
errno: 'ECONNRESET',
code: 'ECONNRESET' }
Note that I have tried to make the request using axios, the built-in HTTP module all to avail. I'm sure the issue is with my Express app handling the request but not sure how to fix it exactly.
Update 1:
As per the suggestion in the comment, here's the async version:
async function f() {
const array = Array.from(Array(1000).keys());
for (const el of array) {
try {
let id = Math.floor(Math.random() * 12) + 1;
const result = await fetch(`http://localhost:3000/api/test/${id}`).then(res => res.json());
console.log(result);
return result;
} catch(e) {
console.log(e);
}
}
}
f();
Now I am receiving occasional ECONNREFUSED messages.
Update 2:
Based on Mazki516's answer here's the solution that works:
// previous require statements
const cluster = require('cluster');
const os = require('os');
if (cluster.isMaster) {
const cpuCount = os.cpus().length
for (let i = 0; i < cpuCount; i++) {
cluster.fork()
}
} else {
const app = express();
// rest of the route definitions
// also app.listen() etc...
}
cluster.on('exit', worker => {
console.log(`${worker.id} removed`);
cluster.fork();
});

One of the reasons you see this is because you make the calls in "parallel" .
You do start the calls one after the other , but the loops will end probably before the first results returned from the server.
The loop continues until the end , making the call stack filled with 1000 async requests to the server .
Your'e are hitting hardware/software limits and nothing is wrong with the code.
if you did want to build a server which can handle 1k (and much more) requests concurrently I would take a look into the "cluster" module of node .
Please notice that when doing network job between server , it's acceptable to use a concurrency limit . (for example: up to 4 requests concurrently)
but you can always scale your server beyond one machine and handle much more traffic .

Node.js Server crashed when Refresh Browser

I tried to build a chat box server by node.js. When the browser requestes the page, it workes well at first. But when I refresh the page, the Server crashes.
Below is the error message:
events.js:183
throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at _errnoException (util.js:1022:11)
at TCP.onread (net.js:615:25)
I used the node --inspect index.js, but could not find the point.
Below is the code of index.js:
const http = require('http');
const fs = require('fs');
const extract = require('./extract');
const wss = require('./websockets-server');
var handleError = function (err,res) {
res.writeHead(404);
res.end();
}
var server = http.createServer(function (req, res) {
console.log("Responding to a request.");
var filePath = extract(req.url);
console.log("filePath:"+filePath);
fs.readFile(filePath,function (err,data) {
if(err){
handleError(err,res);
return;
}else {
res.end(data);
}
})
})
server.listen(3000);
When I comment the 4th line, the import of websockets-server. Server works well when I refresh the page. Maybe it's about the websocket while it works without websocket.
Below is code of websockets-server.js:
const WebSocket = require('ws');
var WebSocketServer = WebSocket.Server;
var port = 3001;
var ws = new WebSocketServer({
port:port
});
var message = [];
console.log('websockets server started');
ws.on('connection', function (socket) {
console.log('client connection established');
message.forEach(function (msg) {
socket.send(msg);
})
socket.on('message', function (data) {
console.log('message received: ' + data);
message.push(data);
ws.clients.forEach(function (clientSocket) {
clientSocket.send(data);
});
});
});
Does the problem is about the websocket? Whether should I do process when the client shutdown the connection with the server while refreshing the page.
extract.js below:
const path = require('path');
var extractFilePath = function (url) {
var filePath;
var fileName = 'index.html';
if(url.length > 1){
fileName = url.substring(1);
}
console.log('The fileName is: ' + fileName);
filePath = path.resolve(__dirname, 'app', fileName);
return filePath;
}
module.exports = extractFilePath;

I guess that you maybe execute var ws = new WebSocket("ws://localhost:3001"); in html file. I haven't figured out exact reason about your error as I'm not proficient in WebSocket. But there is a solution:
window.onbeforeunload = function () {
ws.close();
}
close connection before reload, then the error will not reappear.

You need to add an error listener on the socket. Error listener only on the websocket instance does not help in this case.
socket.on('error', function(e){
console.log(e);
});

The ECONNRESET error means that the other side (browser) closed the connection abruptly. On browser refresh, browser simple killed the connection with the websocket server.
To solve this, you have to listen for the error event on the websocket server instance.
// listen for "error" event so that the whole app doesn't crash
wss.on("error", function(error){
console.log(error);
}

I was having the same problem, but it resolved after this command:
npm install #ionic/app-scripts#nightly --save-dev

How to properly close Node.js Express server?

I need to close server after getting callback from /auth/github/callback
url. With usual HTTP API closing
server is currently supporting with server.close([callback])
API function, but with node-express server i’m getting TypeError: Object function app(req, res){ app.handle(req, res); } has no method 'close'
error. And I don't know how to find information to solve this problem.
How should I close express server?
NodeJS configuration notes:
$ node --version
v0.8.17
$ npm --version
1.2.0
$ npm view express version
3.0.6
Actual application code:
var app = express();
// configure Express
app.configure(function() {
// … configuration
});
app.get(
'/auth/github/callback',
passport.authenticate('github', { failureRedirect: '/login' }),
function(req, res) {
res.redirect('/');
setTimeout(function () {
app.close();
// TypeError: Object function app(req, res){ app.handle(req, res); } has no method 'close'
}, 3000)
}
);
app.listen('http://localhost:5000/');
Also, I have found ‘nodejs express close…’ but I don't sure if I can use it with code I have: var app = express();.

app.listen() returns http.Server. You should invoke close() on that instance and not on app instance.
Ex.
app.get(
'/auth/github/callback',
passport.authenticate('github', { failureRedirect: '/login' }),
function(req, res) {
res.redirect('/');
setTimeout(function () {
server.close();
// ^^^^^^^^^^^
}, 3000)
}
);
var server = app.listen('http://localhost:5000/');
// ^^^^^^^^^^
You can inspect sources: /node_modules/express/lib/application.js

In express v3 they removed this function.
You can still achieve the same by assigning the result of app.listen() function and apply close on it:
var server = app.listen(3000);
server.close((err) => {
console.log('server closed')
process.exit(err ? 1 : 0)
})
https://github.com/visionmedia/express/issues/1366

If any error occurs in your express app then you must have to close the server and you can do that like below-
var app = express();
var server = app.listen(process.env.PORT || 5000)
If any error occurs then our application will get a signal named SIGTERM. You can read more SIGTERM here - https://www.gnu.org/software/libc/manual/html_node/Termination-Signals.html
process.on('SIGTERM', () => {
console.info('SIGTERM signal received.');
console.log('Closing http server.');
server.close((err) => {
console.log('Http server closed.');
process.exit(err ? 1 : 0);
});
});

I have answered a variation of "how to terminate a HTTP server" many times on different node.js support channels. Unfortunately, I couldn't recommend any of the existing libraries because they are lacking in one or another way. I have since put together a package that (I believe) is handling all the cases expected of graceful express.js HTTP(S) server termination.
https://github.com/gajus/http-terminator
The main benefit of http-terminator is that:
it does not monkey-patch Node.js API
it immediately destroys all sockets without an attached HTTP request
it allows graceful timeout to sockets with ongoing HTTP requests
it properly handles HTTPS connections
it informs connections using keep-alive that server is shutting down by setting a connection: close header
it does not terminate the Node.js process

calling server.close does the job
server.close((err) => {
console.log('server closed')
process.exit(err ? 1 : 0)
})
also it is good to listen for system(user) signals and shutdown gracefully on them too, for that you should listen on both SIGTERM and SIGINT
const port = process.env.PORT || 5000;
const server = app.listen(port);
console.log(`listening on port:${port}`);
for (let signal of ["SIGTERM", "SIGINT"])
process.on(signal, () => {
console.info(`${signal} signal received.`);
console.log("Closing http server.");
server.close((err) => {
console.log("Http server closed.");
process.exit(err ? 1 : 0);
});
});

Old question but now Node v18.2.0 introduced server.closeAllConnections(). It should be noted that server.close never runs its callback when the browser sends the request Connection: keep-alive, because server.close only stops the server from accepting new connections, it does not close old connections.
Before Node v18.2.0 I tackled this problem by waiting 5 seconds for the server to shutdown, after which it would force exit.
This code encompasses both situations
process.on('SIGINT', gracefulShutdown)
process.on('SIGTERM', gracefulShutdown)
function gracefulShutdown (signal) {
if (signal) console.log(`\nReceived signal ${signal}`)
console.log('Gracefully closing http server')
// closeAllConnections() is only available from Node v18.02
if (server.closeAllConnections) server.closeAllConnections()
else setTimeout(() => process.exit(0), 5000)
try {
server.close(function (err) {
if (err) {
console.error('There was an error', err)
process.exit(1)
} else {
console.log('http server closed successfully. Exiting!')
process.exit(0)
}
})
} catch (err) {
console.error('There was an error', err)
setTimeout(() => process.exit(1), 500)
}
}

Most answers call process.exit(), I don't think this is a good idea. You probably need to perform some teardown, also it's simply not needed.
const server = app.listen(port);
server.on('close', () => {
// Perform some teardown, for example with Knex.js: knex.destroy()
});
// FYI Docker "stop" sends SIGTERM
// If SIGTERM is not catched, Docker is forced to kill the container after ~10s
// and this provokes an exit code 137 instead of 0 (success)
process.on('SIGTERM', () => server.close());
Check Express.js 4.x documentation: https://expressjs.com/en/advanced/healthcheck-graceful-shutdown.html#graceful-shutdown