Develop Reference

handle websocket connections on multiple instances

I need to understand the concept of handling websockets on multiple instances so that it could be shared across all the instances. For e.g I have three nodes running which is being connected by the load balancer. On receiving the data which needs to be emit on the specific socket. My initial idea was to create a hashmap or json objects to holds the websocket connections. But I realize that this couldn't be done in this way as it will be only specific to its particular instance. If I will receive the data on any of the instances then most of the data will not be going to emit on that websocket connection as it doesn't know on which instance the websocket is created. Is there a good way to handle websocket connections so that it could be shared on all instances. My ideas was to use redis or postgres sql database because they are shared among all the instances.
Also I have tried the postgres solution to store the websocket connection but when I save the connection it says
TypeError: Converting circular structure to JSON
Is there some good solution to handle websocket connections that could be shared among all instances. If database is a good solution how can I resolve
TypeError: Converting circular structure to JSON

I guess the thing you are looking for are Adapters, and it's being documented relatively well in the official socket.io documentation (/v4/adapter/).
When scaling to multiple Socket.IO servers, you will need to replace the default in-memory adapter by another implementation, so the events are properly routed to all clients.
You have several official adapters you can choose from :
Redis adapter
MongoDB adapter
Postgres adapter
Cluster adapter
Here is an example using the MongoDB Adapter :
npm install #socket.io/mongo-adapter mongodb
const { Server } = require("socket.io");
const { createAdapter } = require("#socket.io/mongo-adapter");
const { MongoClient } = require("mongodb");
// DATABASE CONNECTION
const DB = "mydb";
const COLLECTION = "socket.io-adapter-events";
const mongoClient = new MongoClient("mongodb://localhost:27017/?replicaSet=rs0", {
useUnifiedTopology: true,
});
const io = new Server();
const main = async () => {
await mongoClient.connect();
try {
await mongoClient.db(DB).createCollection(COLLECTION, {
capped: true,
size: 1e6
});
} catch (e) {
// collection already exists
}
const mongoCollection = mongoClient.db(DB).collection(COLLECTION);
// HERE COMES THE IMPORTANT PART :)
// CREATE MONGO DB ADAPTER AND USE IT
io.adapter(createAdapter(mongoCollection));
io.listen(3000);
}
main();
The adapter then will take care of the rest. Every packet that is sent to multiple clients (e.g. io.to("room1").emit() or socket.broadcast.emit()) will be
sent to all matching clients connected to the current server
inserted in a MongoDB capped collection, and received by the other Socket.IO servers of the cluster

Is there any purpose of splitting up express and app?

I see these two lines in ( every ?) Express app.
const express = require('express');
const app = express();
I have to wonder if any parameters can be passed to express()?
Checked here and did not see any
https://expressjs.com/en/4x/api.html
Why are some methods on express() and some on app()?
It seems like they should be combined and have all methods on one object.

express does not take any parameters, no. The purpose of calling it is to create the app object, so the fact you have to call it does make sense, even without parameters.
Another way you often see that written is:
const app = require("express")();
It'll need to be separate again when using ESM, though.
import express from "express"; // Or similar
const app = express();
In a comment you've said:
For example, is there anything useful I can do with out creating app? Is there anything I can do only with express?
As far as I know, you have to have create at least one Application object to do anything useful with Express. Note that you don't have to create just one app. That's the typical use case, but there's no reason you can't create multiple apps running on different ports.
You may be wondering why express can't just give you an Application object directly from require. The reason is that modules are only loaded once and cached, so what you get from require is shared. So Express exports the express function, which you use to create the Application object (or objects, plural, if you want more than one).

Is there anything I can do only with express?
The reason for doing two separate lines like this:
const express = require('express');
const app = express();
instead of this:
const app = require('express')();
is that the express module has other methods on it that are sometimes needed such as:
express.static(...) // middleware for serving static files
express.json(...) // middleware for parsing json requests
express.urlencoded(...) // middleware for parsing urlencoded requests
express.Router(...) // constructor for a new Router object

What is the right way to manage connections to mongoDB, using node?

I'm using node.js and mongoDB. Right now, for my test app, the connection to the db is in the main node file, but I guess this is a wrong practice.
What I want/need: a secure way (i.e. not storing password on files users can access) to connect to the db just when needed.
For example: I want several admin pages (users, groups, etc..). Each page should connect to the db, find some data, and display it. It also have a form for adding a document to the db and a delete option.
I thought maybe to create some kind of a connection function - send it what you want to do (add, update, find, delete), to where (collection name) and whatever it needs. But I can't just include this function, because then it'll reveal the password to the db. So what can I do?
Thanks!

I'm going to answer your question bit by bit.
Right now, for my test app, the connection to the db is in the main node file
This is fine, though you might want to put it in a separate file for easier reuse. NodeJS is a continuesly running process, so in theory you could serve all of your HTTP responses using the same connection to the database. In practice you'd want to create a connection pool, but the Mongodb driver for NodeJS already does this automatically.
Each page should connect to the db, find some data, and display it.
When you issue a query on the MongoDB driver, it will automatically use a connection from its internal connection pool, as long as you gave it the credentials when your application was starting up.
What I want/need: a secure way (i.e. not storing password on files users can access) to connect to the db just when needed.
I would advice to keep your application configuration (any variables that depend on the environment in which the app is running) in a separate file which you don't commit to your VCS. A module like node-config can help a great deal with that.
The code you will end up with, using node-config, is something like:
config/default.json:
{
"mongo": null
}
This is the default configuration file which you commit.
config/local.json:
{
"mongo": "mongo://user:pass#host:port/db"
}
The local.json should be ignored by your VCS. It contains secret sauce.
connection.js:
var config = require('config');
var MongoClient = require('mongodb').MongoClient;
var cache;
module.exports = function(callback){
if(cache){
return callback(cache);
}
MongoClient.connect(config.get('mongo'), function(err, db){
if(err){
console.error(err.stack);
process.exit(1);
}
cache = db;
callback(db);
});
}
An incomplete example of how you might handle reusing the database connection. Note how the configuration is gotten using config.get(*). An actual implementation should have more robust error handling and prevent multiple connections from being made. Using Promises would make all that a lot easier.
index.js:
var connect = require('./connection');
connect(function(db){
db.find({whatever: true})
});
Now you can just require your database file anywhere you want, and reuse the same database connection, which handles pooling for you and you don't have your passwords hard-coded anywhere.

Two connections with mongoose in the same application. The second connection is to connect the first dependent

I have a problem. I want to have two connections to the database. In the first database, I have the login information and the database name where are the other information. It is possible to do this using Node.js, mongoose and mongoDB?

You could do it as follows:
var mongoose1 = require('mongoose'),
mongoose2 = require('mongoose');
mongoose1.connect('mongodb://FirstIPAdrress:27017/firstDatabase');
//get values for the second database via find
mongoose2.connect('mongodb://SecondIPAdrress:27017/secondDatabase');

Why is there separate mongo.Server and mongo.Db in mongodb-native driver?

I am just learning mongodb-native driver for nodejs.
I connect like this.
var mongo=require("mongodb")
var serv=mongo.Server("localhost", 27017)
var dbase=mongo.Db("MyDatabase", serv)
And that works. But if I try to create a new database connection using the same server I get an error.
var dbase2=mongo.Db("MyDatabase2", serv)
"Error: A Server or ReplSet instance cannot be shared across multiple Db instances"
But it works if a make a new server connection first.
var serv2=mongo.Server("localhost", 27017)
var dbase2=mongo.Db("MyDatabase2", serv2)
So my question is why there are 2 connection functions, one for Server and one for Db, when it seems like they must always be used together?
Why doesn't it go like this.
var dbase=mongo.Db("localhost", 27017, "MyDatabase")
I want to make my own function that does this, but I wonder if there is some other reason they are separate.
Thanks.

Here is a link to the solution on the mongo docs, for reference. (seems like the same solution the other poster mentioned)
http://mongodb.github.com/node-mongodb-native/markdown-docs/database.html#sharing-the-connections-over-multiple-dbs
The point of separating the connection to the mongo server, and then the DB is for cases like when you want to connect to a ReplSet server, or other custom params. This way, you have a separate process connecting to a mongodb server.
The database connection call is separate simply because of the case you have here: you dont simply want to connect to a mongo server and a single db, but multiple dbs. This separation of connecting to db and server allows this flexibility.
Another Solution: Use node-mongoskin
Mongoskin does what you want to... it allows connecting to server and db all in one command. Not a solution for mongo-native, but worth considering as an alternative library for your future projects.
var mongo = require('mongoskin');
var db = mongo.db('localhost:27017/testDB');

For what it's worth, you can do what you want to do by using Db#db(), which doesn't seem to appear in the official documentation but is listed in the source code of db.js as being a public API:
/**
* Create a new Db instance sharing the current socket connections.
*
* #param {String} dbName the name of the database we want to use.
* #return {Db} a db instance using the new database.
* #api public
*/
so you could do
var serv=mongo.Server("localhost", 27017);
var dbase=mongo.Db("MyDatabase", serv);
var dbase2=dbase.db("MyDatabase2");

Because these are two separate and distinct actions - you have to connect (or already have a connection) to DB server (computer) in order to query any of the databases on that particular server. You can create distinct database query connections for each of the databases that you will want to use, but at the same time you will be using the same connection to the server.
Most of the time you will NOT want to create a separate server connection for each of the databases (if there are many) because the server usually limits the number of connections.