Develop Reference

How to close a MongoDB connection in nodejs

I have the following method to connect to MongoDB:
import { Db, MongoClient } from 'mongodb';
let cachedConnection: { client: MongoClient; db: Db } | null = null;
export async function connectToDatabase(mongoUri?: string, database?: string) {
if (!mongoUri) {
throw new Error(
'Please define the MONGO_URI environment variable inside .env.local'
);
}
if (!database) {
throw new Error(
'Please define the DATABASE environment variable inside .env.local'
);
}
if (cachedConnection) return cachedConnection;
cachedConnection = await MongoClient.connect(mongoUri, {
useNewUrlParser: true,
useUnifiedTopology: true,
}).then((client) => ({
client,
db: client.db(database),
}));
return cachedConnection!;
}
And I use it in the following way:
const { db, client } = await connectToDatabase(
config.URI,
config.USERS_DATABASE
);
const user = await db
.collection(config.USERS_COLLECTION)
.findOne({ _id: new ObjectId(userId) });
It seems to be ok, but it is not. The problem of this method is that it doesn't close the connections. For example I have a cluster on Atlas, and the connections keep growing till 500. after that it doesn't serve anymore, goes in timeout and then my backend crashes.
To solve this I tried with client.close() just before returning the response to frontend.
It throws me one error saying MongoError: Topology is closed, please connect. I believe that it closes the connection before it finishes? is it right? Even if I put it after the DB responded.
this is the screenshot of the error:
Do you think there is a way to solve this or I just have to do the entire procedure in each file I need to connect to mongo? Do you also think I did something wrong?

Empty object returned from gRPC Server

Basically, the JSON object that's returned from a callback in my gRPC server is empty no matter what I do.
For the most part I'm following this tutorial, except I'm using a SQLite3 server instead of knex, and I've worked to the listProducts method. I haven't tried working on the other product methods yet.
In server.js I get some data from a SQLite3 database, and try to return it in a callback (at the bottom of the method). I also print out the data from the DB to confirm I'm actually getting valid data.
gRPC server.js
function listProducts(call, callback) {
console.log("******** Listed the products *********");
var data = "";
let db = new sqlite3.Database('../data/testDB.db', sqlite3.OPEN_READONLY, (err) => {
if(err){
console.error(err.message);
}
console.log("connected to DB");
});
db.serialize(() => {
db.get('SELECT NAME as name FROM PEEPS', (err, row) => {
if(err){
console.error(err.message);
}
console.log(row.name);
data.name = row.name;
});
});
db.close((err) => {
if(err) {
console.error(err.message);
}
console.log('closed db');
});
callback(null, { products: data.name });
}
Out put from gRPC server.js
******** Listed the products *********
connected to DB
Jeff // Correct data from DB.
closed db
The callback returns to client.js, where it was called. However, the object is always empty.
If I uncomment res.json({ name: "jessie" }); and comment res.json(result);, the code works as expected; name: jessie is sent to the browser as a JSON object.
So that tells me that from the client to the browser the data is being handled correctly. Therefore the problem is when the data is passed from the server.js to client.js.
gRPC client.js
// requirements
const path = require('path');
const protoLoader = require('#grpc/proto-loader');
const grpc = require('grpc');
// gRPC client
const productProtoPath = path.join(__dirname, '..', '..', 'protos', 'product.proto');
const productProtoDefinition = protoLoader.loadSync(productProtoPath);
const productPackageDefinition = grpc.loadPackageDefinition(productProtoDefinition).product;
const client = new productPackageDefinition.ProductService('localhost:50051', grpc.credentials.createInsecure());
// handlers
const listProducts = (req, res) => {
client.listProducts({}, (err, result) => {
console.log(result);
console.log(typeof result);
// console.log(res.json(result));
res.json(result);
// res.json({ name: "jessie" });
console.log("*******************");
});
};
Output from gRPC client.js
Server listing on port 3000
{} //Oh no! An empty JSON object!
object
*******************
Edit
Here is a link to my repository: https://github.com/burke212/grpc-node

The main problem here is that in your server code, your db methods are asynchronous but you are trying to access the result synchronously. You need to call the main callback for listProducts in the callback for db.get to ensure that you have the result of that database request before trying to use it. After making this change your listProducts method implementation should look more like this:
function listProducts(call, callback) {
let db = new sqlite3.Database('../data/testDB.db', sqlite3.OPEN_READONLY);
db.serialize(() => {
db.get('SELECT NAME as name FROM PEEPS', (err, row) => {
if(err){
console.error(err.message);
}
// Call the callback here to use the result of db.get
callback(null, { products: row.name });
});
});
db.close();
}
For simplicity I omitted the logging. Also, the sqlite3.Database constructor and db.close do not have callbacks in the example in the sqlite3 README. I suggest checking again whether those functions actually take callbacks.
In addition to that, now that you have shared the product.proto file that defines your service, there is another problem. The listProducts method in the ProductService service is declared as returning a ProductList object. In that message type, the products field must be an array of Product objects. All of the code in your method implementation is directed towards returning a string in that field, and that does not result in a compatible object.

How to connect to MongoDB with Async/Await in Nodejs?

I have a Connection class with static methods that allows me to create a singleton type object for MongoDB connections. Using Async along with Await, I never get the connection to 'fire' before the rest of my code executes.
Using the traditional Promise / .then this Connection class works. Using Latest Nodejs version and MongoDB version.
static connectDb() {
//If MongoDB is already connected, return db object
if (this.dbClient) {
//const currDbClient = Promise.resolve(this.dbClient);
console.log(`MongoDB already connected!`);
return this.dbClient;
}
//Otherwise connect
else {
async () => {
try {
const newDbClient = await MongoClient.connect(this.url, this.options);
console.log(`DB is connected? ${newDbClient.isConnected()}`);
ConnectMeCore.dbClient = newDbClient;
return newDbClient;
} catch (error) {
console.error(`MongoDB connection failed with > ${error}`);
}
};
}
}
I expect the await to 'wait' for the DB to connect, or at least resolve the promise.

Thanks to #JaromandaX for helping find the answer!
The calling code can use a Promise.then to execute code once the DB connection happens.
DbConnection.connectDb().then(() => {
console.log("Is it connected? " + DbConnection.isConnected());
//Do CRUD
DbConnection.closeDb();
});
You can import this method(as part of 'Connection' class) into any class that needs to have a DB connect. A singleton for on DB connection. The working method fragment is as follows.
static async connectDb() {
//If MongoDB is already connected, return db object
if (this.dbClient) {
const currDbClient = Promise.resolve(this.dbClient);
console.log(`MongoDB already connected!`);
return currDbClient;
}
//Otherwise connect using 'await', the whole methos is async
else {
try {
const newDbClient = await MongoClient.connect(this.url, this.options);
console.log(`DB is connected? ${newDbClient.isConnected()}`);
this.dbClient = newDbClient;
return newDbClient;
} catch (error) {
console.error(`MongoDB connection failed with > ${error}`);
throw error;
}
}
}

How to wait until mysql query is done and then continue to run the script?

I've made a class in which the constructor is being declared mostly by a mySQL query that looks like that:
constructor(username) {
this.mysql = require('mysql');
// create a connection variable with the required details
this.con = this.mysql.createConnection({
host: "localhost", // ip address of server running mysql
user: "root", // user name to your mysql database
password: "", // corresponding password
database: "db" // use the specified database
});
this.username = username;
this._password = "";
this.con.query("SELECT * FROM users WHERE username = ?", this.username, function (err, result, fields) {
if (err) throw err;
this._password = result[0].password;
});
}
get password() {
return this._password;
}
The issue is , when I'm declaring the class like that:
const user= require("./user.js");
let bot = new user("user1");
console.log(user.password();
The code first accessing the get and getting undefined , and only after the the query is done.
How can I fix it?

You can't make a constructor asynchronous, that's not supported by Javascript.
What you can do is create an asynchronous createUser function that returns a user, and modify the constructor of your User class to take the data returned from the database.
Something like:
class User {
// Synchronous constructor
constructor(mysqlData) {
this._password = mysqlData.password;
}
}
// Asynchronous create user function
function createUser(name, callback) {
con.query('your query', [], function (err, result) {
if (err) return callback(err); // Traditional error-first callback
const user = new User(result[0]);
callback(null, user);
}
}
// Create the user and then do something once the user is created
createUser('user1', function (err, user) {
if (err) throw err;
console.log(user.password());
});
Unrelated tip: look into Promises and async/await, they are so much nicer to use than callbacks. Then your createUser function would look something like this (notice the await keyword instead of a callback for the query):
async function createUser(name) {
const result = await con.query('your query', []);
const user = new User(result[0]);
return user;
}
And then you can do const user = await createUser('user1'); from another async function (you can only use await inside of an async function).
Keep in mind that for this to work, you need:
A runtime that supports async/await (Node 7.6+ or a modern browser), or Babel to transpile the code so it can run in older runtimes & browsers.
A mysql client that supports promises (there are wrappers on npm for most packages).

POST response error after collection.insert with NodeJS Mongo module

I've got this error when trying to POST
> process.nextTick(function() { throw err; });
> ^
>
> TypeError: first argument must be a string or Buffer
> at ServerResponse.OutgoingMessage.end (_http_outgoing.js:524:11)
Errors shows that something's wrong with utils and cursor both from mongodb module, but what are they?
Everything works nice on GET but brakes on POST (postman and passing as text {"name":"Computer","price":2500}) - i cannot trace which module or instance is braking the code.
This is my conn with db:
// Our primary interface for the MongoDB instance
var MongoClient = require('mongodb').MongoClient;
// Used in order verify correct return values
var assert = require('assert');
var connect = function (databaseName, callBack) {
var url = 'mongodb://localhost:27017/' + databaseName;
MongoClient.connect(url,
function (error, database) {
assert.equal(null, error);
console.log("Succesfully connected to MongoDB instance!");
callBack(database);
});
};
exports.find = function (databaseName, collectionName, query, callback) {
connect(databaseName, function (database) {
var collection = database.collection(collectionName);
collection.find(query).toArray(
// Callback method
function (err, documents) {
// Make sure nothing went wrong
assert.equal(err, null);
// Print all the documents which we found, if any
console.log("MongoDB returned the following documents:");
console.dir(documents)
callback(err, documents);
// Close the database connection to free resources
database.close();
})
})
};
exports.insert = function (databaseName, collectionName, object, callback) {
connect(databaseName, function (database) {
var collection = database.collection(collectionName);
collection.insert(document, {w: 1}, function (err, documents) {
console.log("Added a new document");
console.log(documents[0]);
callback(err, documents[0]);
});
})
};
exports.remove = function (databaseName, collectionName, object, callback) {
connect(databaseName, function (database) {
var collection = database.collection(collectionName);
collection.remove(object, function (err, result) {
callback(err, result);
database.close();
});
})
};

The issue is actually pretty straightforward, so I'm surprised that you're not getting a better error message.
In your code:
collection.insert(document, {w: 1}, function (err, documents) {
console.log("Added a new document");
console.log(documents[0]); // I expect this to log undefined
callback(err, documents[0]);
});
The second argument passed into the collection.insert callback is actually a results object, not the documents that were inserted. So, documents[0] ends up being undefined because it's not an array of documents. Thus, when you trying to send undefined as a response, it's failing.
If you intention is to pass the newly created documents, you're going to have to use the result object to get the _id and attach it to the document you inserted.
As a side note, I would consider keeping a connection open to your database rather than creating a new connection every time you want to talk with Mongo.