I am creating a RESTful API that uses Node.js and Express. My application uses an Oracle database so I installed the node-oracledb module from npm. I've looked through the documentation and viewed some of the samples presented in the module's github page; however, I don't see any examples where a connection pool is used. Correct me if I'm wrong, but for applications that will make many calls to the database the recommendation is to use connection pools as opposed to using standalone connections. Below is a sample of the code I wrote:
createPool = function(poolAttrs, fetchPool){
oracledb.createPool(poolAttrs, function(error, pool){
if(error){
console.error(`Could not create pool using specified attributes: `, error.message);
}
else{
console.log(`Pool created successfully using poolAlias: ${poolAttrs.poolAlias}`);
fetchPool(pool);
}
});
};
createConnection = function(poolAlias, connection){
oracledb.getConnection(poolAlias, function(error, conn){
if(error){
console.error(`Could not get connection: `, error.message);
} else {
console.log(`New connection obtained: ${conn}`);
connection(conn);
}
});
};
executeQuery = function(queryString, poolAlias){
console.log(queryString);
var conn = createConnection(poolAlias, function connection(conn){
conn.execute(queryString, function(error, result){
if(error){
console.error(`Could not execute query: `, error.message);
} else {
console.log(result);
}
conn.close(function(error){
if(error){
console.error(`Could not close connection.`);
} else {
console.log(`Connection successfully closed.`);
}
})
});
});
}
closePool = function(pool){
pool.close(60, function(error){
if(error){
console.error(`Could not close connection pool ${pool.poolAlias}`, error.message);
} else {
console.log(`Pool closed successfully.`);
}
});
};
createPool(attrs, function fetchPool(pool){
var poolAlias = pool.poolAlias;
console.log(poolAlias);
});
executeQuery(queryString, attrs.poolAlias);
When I run this code I get the following error:
Could not get connection: NJS-047: poolAlias "amDBpool" not found in the connection pool cache
I know why the error is happening. As my understanding of callbacks go, I know that calling the asynchronous createPool() function with the fetchPool(pool) callback registers this callback (fetchPool) in the callback stack. All synchronous code will execute before it. So when I call executeQuery and the function reaches the line of execution where it calls createConnection(poolAlias...) the poolAlias variable is null since the createPool function is still waiting in the callback stack to execute. Hence, no pool with the alias "poolAlias" exists and the call fails. I know that I could stick the call to createPool inside the executeQuery method, but wouldn't that try to create a new pool every time I execute a query? My question is, is there a way to test inside the executeQuery method that the pool exists, and if it does then not try to recreate it. Other than this, the only other way to do this would be with an Promises or Async/Await right?
The node-oracledb examples using a connection pool are the connectionpool.js and examples/webap.js files. Yes, a connection pool would be a good idea in a web service. Also I recommend using the async/await style of programming.
You can see the examples creates a connection pool before doing anything else:
await oracledb.createPool({
user: dbConfig.user,
password: dbConfig.password,
connectString: dbConfig.connectString
});
The returned pool from the oracledb.createPool() call is ignored because the pool is later accessed via the pool cache since no credentials are passed to getConnection():
let connection = await oracledb.getConnection();
This uses the default pool alias (which happens to be the string "default"), so no alias is specified when the pool is created or used.
Documentation and tips on using node-oracledb connection pools is in Connection Pooling.
You may be interested in Creating a REST API with Node.js and Oracle Database,
A node-oracledb Web Service in Docker and Demo: GraphQL with Oracle Database and node-oracledb
Related
I have a C# server running my hub class, which contains only 1 method in there, which is as follows,
public class HothHub : Hub
{
public async Task AddSingleUserGroup(string name)
{
await Groups.AddToGroupAsync(Context.ConnectionId, name);
}
}
I also have a JavaScript client, which connects to the hub via the following code,
var connection;
async function signalRStart() {
connection = new signalR.HubConnectionBuilder()
.withUrl("https://somesignalrurl.com/hothhub", { withCredentials: false })
.withAutomaticReconnect()
.configureLogging(signalR.LogLevel.Information)
.build();
connection.on("hothHubToHothUpdate", () => {
console.log("Called by the server!");
});
connection.onreconnecting(error => {
console.log("Connection lost due to error " + error + ". Reconnecting.");
});
// Start the connection.
await start();
}
async function start() {
try {
await connection.start();
connection.invoke("addSingleUserGroup", "someUniqueUserName");
} catch (err) {
console.log(err);
setTimeout(start, 5000);
}
};
Now when the client initiates the connections and run start() on itself, this part seems to run fine. A connection to the signalR hub is made successfully. The problem I'm having is when connection.invoke("addSingleUserGroup", "someUniqueUserName"); is run although the error does not happen all the time. On first run, the method at the server end is hit successfully however, it looks like subsequent calls to it fail and this is the error returned in the client,
Uncaught (in promise) Error: Failed to invoke 'addSingleUserGroup' due to an error on the server. HubException: Method does not exist.
at _callbacks.<computed> (signalr.js:1252:36)
at HubConnection._processIncomingData (signalr.js:1364:33)
at HubConnection.connection.onreceive (signalr.js:985:52)
at webSocket.onmessage (signalr.js:2236:30)
I've read a few articles on here but most seemed to be related to the client calling the wrong method name having used a capital letter at the start of the method name when invoking it and some having mentioned issues with the method expecting 1 type parameter and receiving another type although in my instance here its hard to think how the server would not treat the incoming parameter as a string, which is what is being passed in. Has anyone got any ideas on what could be wrong here or what I could try?
Thanks!
Unfortunately I dont have an actual answer for this but after deploying the solution to my Azure App Service, the release version does not produce the error. It seems the error only persisted when in debug mode but like I said I'am not sure why.
I have been following online instructions to set up a mongodb database using mongoose on node.js. I have managed to get the mongodb running and listening on port 27017, however when I run my connection code in node.js I get a successful response even when the mongo.db isn't running and I don't see any updates on the mongo.db to say it has received any data.
I have tried quite a few different examples from the internet but can't get any of them to work.
So I have my MongoDB saying:
2019-03-26T12:00:46.039+0000 I NETWORK [initandlisten] waiting for connections on port 27017
This is the basic code I am trying to get to work as my original API wasn't working:
//sample.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/test', function(err){
if(!err) console.log('Successfully Connected');
console.log(mongoose.connection.host);
console.log(mongoose.connection.port);
});
I receive this response when I run
node sample.js
Successfully Connected
localhost
27017
But I receive this response even when my mongo.db has been shut down, so I think there is some error, I'm not sure how to check if they are connecting properly.
this works for me. try to set some debugging to find out what is happening.
mongoose.set('debug', true);
mongoose.connect("mongodb://localhost:27017/test", {useNewUrlParser: true})
.then(() => {
console.log('connected to the db');
})
.catch(err => {
console.log('connection failed ' + err);
});
I think you have to check the API to connect.
https://mongoosejs.com/docs/connections.html
Second parameter is option and third is callback - but u seem to have passed callback as second param - also, check what are you getting as response
mongoose.connect(uri, options, function(error) {
// Check error in initial connection. There is no 2nd param to the callback.
});
// Or using promises
mongoose.connect(uri, options).then(
() => { /** ready to use. The `mongoose.connect()` promise resolves to undefined. */ },
err => { /** handle initial connection error */ }
);
I am using node js for web service I use pool.getConnection() in my code , after use pool.getConnection() for close this connection i am using connection.release(); but in any query i am not use any function for close connection .please tell me this code is valid or not?.
pool.getConnection(function(err,connection){
connection.release();
if(!err){
email=req.user.email
connection.query('select * from users where email=?',[email],
function(err,user_detail){
if(!err)
{
connection.query('insert into message(??,time,type,user_id) values(?,?,?,?)',[messagecolumns,dbValues,time,type,user_id],
function(err,message_inserted){
if(!err)
{
console.log('Message send',message_inserted.insertId);
connection.query('select u.firstname,u.lastname,u.img_path,m.* from users u, message m where u.id=m.user_id and m.id=?',[message_inserted.insertId],
function(err,message_detail){
if(!err)
{
connection.query('select count(id) as message_count from message where appointment_request_id=?',[appointment_request_id],
function(err,total_message){
if(!err)
{
connection.query('update appointment_request set message_count=? where appointment_request_id=?',[total_message[0].message_count,appointment_request_id],
function(err,update_message_count){
if(!err)
{
}
});
}
else {
console.log('connection Error',err);
}
I feel this is not valid method for connection because from some web site I realize that after SQL query I must close connection but in this code I never use any close connection and my code are some time give me 500 server error and some time it work successfully so I have to fix this error and I want to permanently solution if anyone can help me please give me solution for this. if you have any confutation for understanding you may ask me.
A pool has a limited number of connections (Default is 10).
When you getConnection then one of those connections is removed from pool and is passed to you so that you can use it. You will call connection.release() after you are finished working with the connection. As soon as you called connection.release() the connection is passed back to the pool and is free to be used by other parts of your code. As of that it is not valid to work with this connection after you have called release on it, because it might be closed or used by another part of your code that requested a connection with getConnection (in most of the cases it might still work, but it could unexpectedly fail).
In your code you call connection.release(); at the beginning of you callback so the usage of connection after that release is not valid.
You you must not call connection.release() multiple times on the same connection.
Background Information
I'm attempting my first node.js API/application. As a learning exercise, I'm trying to create some test cases initially delete all records in a table, insert 3 specific records, and then query for those 3 records.
Code
Here's the code I have cobbled together:
http://pastebin.com/duQQu3fm
Problem
As you can see from the code, I'm trying to put the database connection logic in a dbSession.js file and pass it around.
I am able to start up the http server by doing the following:
dev#devbox:~/nimble_node$ sudo nodejs src/backend/index.js
Server started and listening on port: 8080
Database connection successful
However, when I try to run my jasmine tests, it fails with the following error:
F
Failures:
1) The API should respond to a GET request at /api/widgets/
Message:
TypeError: Object #<MongoClient> has no method 'collection'
Stacktrace:
TypeError: Object #<MongoClient> has no method 'collection'
at resetDatabase (/home/dev/nimble_node/spec/resetDatabase.js:6:29)
at /home/dev/nimble_node/spec/e2e/apiSpec.js:23:25
at /home/dev/nimble_node/node_modules/async/lib/async.js:683:13
at iterate (/home/dev/nimble_node/node_modules/async/lib/async.js:260:13)
at async.forEachOfSeries.async.eachOfSeries (/home/dev/nimble_node/node_modules/async/lib/async.js:279:9)
at _parallel (/home/dev/nimble_node/node_modules/async/lib/async.js:682:9)
at Object.async.series (/home/dev/nimble_node/node_modules/async/lib/async.js:704:9)
at null.<anonymous> (/home/dev/nimble_node/spec/e2e/apiSpec.js:19:9)
at null.<anonymous> (/home/dev/nimble_node/node_modules/jasmine-node/lib/jasmine-node/async-callback.js:45:37)
Finished in 0.01 seconds
1 test, 1 assertion, 1 failure, 0 skipped
Database connection successful
Line 6 of resetDatabase is:
var collection = dbSession.collection('widgets');
Given that after the error appears, I get the "Database connection successful" message, I think what's happening is that when the tests request the dbSession library, the database hasn't finished running the code to connect. And therefore, I can't get the collection object.
I'm currently reading through the mongodb online manual to see if I can find some hints as to how to do something like this.
Any suggestions or pointers would be appreciated.
EDIT 1
To prove that there is a collection method on the MongoClient object, I changed the dbSession.js code to look like this:
'use strict';
var DBWrapper = require('mongodb').MongoClient;
var dbWrapper = new DBWrapper;
dbWrapper.connect("mongodb://localhost:27017/test", function(err, db) {
if (!err) {
console.log("Database connection successful");
dbWrapper = db;
var collection = dbWrapper.collection('widgets');
console.log('just created a collection...');
}
});
module.exports = dbWrapper;
And now, when I start up the http server (index.js), notice the messages:
dev#devbox:~/nimble_node$ sudo nodejs src/backend/index.js
Server started and listening on port: 8080
Database connection successful
just created a collection...
It could be an async issue.
Your code in dbSessionjs
dbWrapper.connect("mongodb://localhost:27017/test", function(err, db) {
if (!err) {
console.log("Database connection successful");
dbWrapper = db;
}
});
module.exports = dbWrapper;
Starts the connection at dbWrapper asynchronously, but exports dbWrapper right away, which is then imported in resetDatabase. Thus yes, the connect function may have not yet returned from the async function when you call it in resetDatabase (and is what the log suggests,as the error appears before the success log).
You could add a callback after dbWrapper.connect() returns, in order to actually only be able to use dbWrapper when the connection finished.
(With sqlite, this may not happen as it accesses the DB faster on the commandline).
This may not be your problem but looks like a candidate.
EDIT: Here's a possible example for a callback, but please take note it depends on what you need to do so there are a lot of different solutions. The key is to call a callback function when you are done initializing.
Another solution could be to simply wait, and/or poll (e.g. chcke a variable 'initialized').
'use strict';
var DBWrapper = require('mongodb').MongoClient;
var dbWrapper = new DBWrapper;
function doConnect(callback) {
console.log("Initializing DB connection...");
dbWrapper.connect("mongodb://localhost:27017/test", function(err, db) {
if (!err) {
console.log("Database connection successful");
dbWrapper = db;
var collection = dbWrapper.collection('widgets');
console.log('just created a collection...');
console.log('calling callback...');
callback(dbWrapper);
} else {
console.log("Error connectingi: " + err);
}
});
};
doConnect(function(correctDbWrapper) {
//Now you can use the wrapper
console.log("Inside callback, now consuming the dbWrapper");
dbWrapper = correctDbWrapper;
var collection = dbWrapper.collection('widgets');
});
It's interesting though I never ran into this issue, although I have generally used similar code like yours. I guess because normally I have this DB initialization right at the top, and then have to do lots of initializations on the node app, which gives the app time enough to return from the connect call....
I'm aware of the best practice of MongoDB connection pooling in NodeJS of the singleton DB connection type like this
var db = null;
var connection = function getDBConnection(callback) {
if(db) { callback(null, db) } else { MongoClient.connect( .... ) }
}
module.exports = getDBConnection;
However, what I cannot get my head around at the moment is how to handle this in a one-shot script that, say, does some pre-initialization on the documents of a certain db collection:
getDBConnection(function (err, database) {
var collection = database.collection("objects");
var allObjectsArray = collection.find( /* ... */
).toArray(function (err, objects) {
if(err != null) console.log(err);
assert.equal(null, err);
_.each(objects, function (item) {
collection.update(
{ id: item.id},
{ $set: { /* ... */ }},
function (err, result) {
if(err != null) console.log(err);
assert.equal(null, err);
}
);
});
// database.close(); <-- this fails with "MongoError: Connection Closed By Application", thrown by the update callback
});
// database.close(); <-- this fails too, thrown by the toArray callback
});
If I call the script like that, it never terminates, due to the still open connection. If I close the connection at the bottom, it fails because of, well, a closed connection.
Considering that opening a new connection for every update is not really an option, what am I missing? Keeping the connection open may be fine for webapps, but for a one-shot script called from a shell script this really doesn't work out, does it?
Sorry if this question has arisen before, I've given it some research but have not quite been able to come up with a working answer for me...
Thanks!
Julian
As a "pooled connection" there is code running to keep the connection alive and establish more connections in the pool if required under the driver connection. So much like various "server code" methods, event loop handlers have been invoked and the process does not exit at the end of your code until these are de-registered.
Therefore your two choices to call after all your code has executed are either:
Call db.close() or in your code context specifically database.close() once all is done.
Call process.exit() which is a generic call in node.js applications which will shut the whole process down and therefore stop any other current event loop code. This actually gives you an option to throw an error on exit if you want your code to be "shell integrated" somewhere and look for the exit status.
Or call both. db.close() will allow execution to the next line of code and whatever you put there will also run.
But you have to wait until everything is called, so you can cannot use synchronous loops with asynchronous code in the middle:
async.each(objects,function(item,callback) {
collection.update(
{ "_id": item._id },
{
// updates
},
callback
);
},function(err) {
if (err) throw err;
database.close();
});