I'm using node.js and mongoDB. Right now, for my test app, the connection to the db is in the main node file, but I guess this is a wrong practice.
What I want/need: a secure way (i.e. not storing password on files users can access) to connect to the db just when needed.
For example: I want several admin pages (users, groups, etc..). Each page should connect to the db, find some data, and display it. It also have a form for adding a document to the db and a delete option.
I thought maybe to create some kind of a connection function - send it what you want to do (add, update, find, delete), to where (collection name) and whatever it needs. But I can't just include this function, because then it'll reveal the password to the db. So what can I do?
Thanks!
I'm going to answer your question bit by bit.
Right now, for my test app, the connection to the db is in the main node file
This is fine, though you might want to put it in a separate file for easier reuse. NodeJS is a continuesly running process, so in theory you could serve all of your HTTP responses using the same connection to the database. In practice you'd want to create a connection pool, but the Mongodb driver for NodeJS already does this automatically.
Each page should connect to the db, find some data, and display it.
When you issue a query on the MongoDB driver, it will automatically use a connection from its internal connection pool, as long as you gave it the credentials when your application was starting up.
What I want/need: a secure way (i.e. not storing password on files users can access) to connect to the db just when needed.
I would advice to keep your application configuration (any variables that depend on the environment in which the app is running) in a separate file which you don't commit to your VCS. A module like node-config can help a great deal with that.
The code you will end up with, using node-config, is something like:
config/default.json:
{
"mongo": null
}
This is the default configuration file which you commit.
config/local.json:
{
"mongo": "mongo://user:pass#host:port/db"
}
The local.json should be ignored by your VCS. It contains secret sauce.
connection.js:
var config = require('config');
var MongoClient = require('mongodb').MongoClient;
var cache;
module.exports = function(callback){
if(cache){
return callback(cache);
}
MongoClient.connect(config.get('mongo'), function(err, db){
if(err){
console.error(err.stack);
process.exit(1);
}
cache = db;
callback(db);
});
}
An incomplete example of how you might handle reusing the database connection. Note how the configuration is gotten using config.get(*). An actual implementation should have more robust error handling and prevent multiple connections from being made. Using Promises would make all that a lot easier.
index.js:
var connect = require('./connection');
connect(function(db){
db.find({whatever: true})
});
Now you can just require your database file anywhere you want, and reuse the same database connection, which handles pooling for you and you don't have your passwords hard-coded anywhere.
Related
After researching a lot I can't find anything similar to help me solve this problem.
I have a Node server, with several environments (dev, test, demo, prod). This server is deployed in production on a Linux server, via a service.
I need to be able to have several production environments for several different customers.
Example:
I have 2 urls: https://customer1.com and https://customer2.com.
The code of these two clients are identical, only the url changes.
For the server, it must be able to recognize which client is sending it a request, because the data to be sent back to the client is not the same.
The customer1 will have its database on a different url than that of customer2. The server will therefore have to make the distinction in order to return only the data concerning the client making the request.
My question: how to achieve this?
I would like to avoid deploying 1 server per client, which I know would be simpler, but less maintainable.
Currently, I am using Express-Session to define environments.
In fact, I have a middleware which will look in mysql for the environment variables of each client:
con.connect(function(err) {
if (err) throw err;
con.query(`SELECT * FROM environments WHERE CLIENT_URL = '${req.headers.origin}'`, function(err, result) {
if (err) throw err;
delete result[0].ID;
for (var prop in result[0]) {
req.session[prop] = result[0][prop];
}
next();
});
con.end();
});
It seems to work but it doesn't seem very stable or very reliable to me, am I wrong?
What better can I use to separate the data so that there is no customer1 that can receive the data from customer2?
Thank you for your help!
Following all comments under your original post, you need to do something like this:
SELECT * FROM environments WHERE CLIENT_URL = '${req.headers.origin}' AND CUSTOMER_NAME
LIKE yourUserCustomerFromSession
Before, any user could query data for any customer as long as they use the URL for that customer, now this is no longer possible.
Even better way of doing it, if you don't want to hold the Client name in the session, you can do 2 queries - the first one to get the Client name for the logged in User and the second one similar to the code above:
SELECT * FROM environments WHERE CLIENT_URL = '${req.headers.origin}' AND
CUSTOMER_NAME LIKE theClientNameYouJustGotForTheLoggedInUser
I am working on a backend application with node using typescript / javascript. My backend is using a sqlite database. I structured my project so that there is one file that contains all the database logic.
I could either write a module that will be required by all modules that or I could write a class which connects to the database in the constructor.
For me it seems a little bit weird to pack all database logic in a module that is not an object. What is the best practice in this case and why? (I know this might be a stupid question but I'm just a hobbyist)
Thanks in advance
In my case I make the connection to the database before running the server. In pseudocode:
connectorDB.connect( path, config, () => {
server.listen( port )
})
It will depend on how important the database connection is in the operation of your app.
So I figured the best way to learn is to try and fail over and over. I am building a webapp, at least trying to. I am curious how to go about using node to query my db. I am able to make a connection to the db with my single app.js file.
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'xxxxxxxx-us-east-1.rds.amazonaws.com',
port : '3306',
user : 'user',
password : 'password',
database : 'app'
});
connection.connect(function(err){
if(!err) {
console.log("Database is connected ... ");
} else {
console.log("Error connecting database ... ");
}
});
My problem, or lack of understand begins when I try to integrate this into my client-side js code. For instance say I wanted to trigger the db connection when a user uploads a photo.
var upload = s3.putObject({
Bucket: albumBucketName,
Key: photoKey,
Body: file,
ACL: "public-read",
});
var promise = upload.promise();
Can I include the app.js node file?
Sorry if this is a dumb question. I feel like I am missing some fundamental understanding of how to integrate the functionality of node with my current client side JS. Any help or further reading is appreciated--I am even curious about PHP solutions.
X
Server and client side code are separate. However you can create a Node module that harnesses the AWS and returns an appropriate response to the client after completed.
To do this, you need to create an endpoint that you post your data to from the client, then process with the same AWS modules only for Node. You also need to be able to access the connection instance from a different NodeJS module. This can be accomplished several ways. First, if the library that instantiates the connection tracks all of the connections, you should be able to require the library in a different module, then use the library's API to access one of the connections. Second, if you create only one instance of the connection and allow it to export, then you can import the module with that connection. Third, you can use something like a request/response pattern between the two modules, with the pattern instance declared globally.
I am using memcached as a backing store for an ExpressJS session - chosen via an app configuration setting. I would like to fall back from memcached to memory if the memcached host cannot be contacted. (This isn't necessarily a production strategy as memcached is very reliable - it's more for when I forget to boot the Docker instance in dev, but could still be a production fail-safe.)
I first thought I could "app.use" a new session instance, and try to remove the first one, but I have read that it's difficult (if possible) to "un-use" Express middleware i.e. to swap the middleware in-place in the chain, or generally tinker with the chain once it's been setup.
The problem there is after the connection timeout period, the app middleware has been setup with many further services installed after the session middleware.
My second thought was can I re-configure the Express Session instance itself and change the store after it's been created? I could not see any way in the documentation.
My third idea was to wrap the express-session in a new "swappable store" class, but I'm wary of the scope of wrapping the entire interface.
For example in my app setup:
app.use(services.session.middleware());
// then a lot of other middleware...
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended: false}));
app.use(cookieParser());
app.use(express.static(path.join(rootPath, 'public')));
...etc
And in the session service, which selects and configures the session instance:
function middleware() {
// default / Memory Store options
const opts = {
resave: false,
saveUninitialized: false,
...etc
}
};
// Install any configured backing store
if(config.session.storage === "memcached"){
const MemcachedStore = require('connect-memcached')(session);
opts.proxy = 'true';
opts.store = new MemcachedStore({
hosts: ...,
secret: ...
});
const errorHandler = (type, details)=> {
/* HERE I WOULD LIKE TO RE-MAKE THE SESSION USING MEMORY
* AND DISCARD THE MEMCACHED ONE
* (THIS HAPPENS AFTER APP BOOT HAS FINISHED)
*/
console.error( String.format("Memcached {3} with host:{0} details:{1}.", details.server, details.messages.join( '' ), type));
}
opts.store.client.on('failure', details => errorHandler('failure', details));
opts.store.client.on('issue', details => errorHandler('issue', details));
}
return session(opts);
}
I think there are several approaches you can try for this. The first is to configure a storage system based on which configuration you have supplied on startup (probably via a module like config or nconf). The second is to run a quick check when booting up the app to make sure it can access the memcache service, and if it can't then fallback to memory with an error.
I would be fairly weary of doing either of these, since you're using docker and it should be easy to boot memcache. This is because you'll be introducing code which might trigger in production should there be some connection issue, and then you might find yourself accidentally serving sessions out of memory rather than something like memcache potentially without realising.
I'll expand on both strategies here and provide a third possibly better option.
1. Choose the cache system based on a config
This should be fairly straight forward, simply extract your configuration into some sort of config manager / environment variables (checkout config or nconf). When starting the application and connecting your session middleware, you can pull out all the possibly configurations, see which exist and attach one based on that. This is similar to how your if (config.session.storage === 'memcache") looks at the moment. Just use a fallback of not configuring one and the express-session middleware will fall back to memory. This way you can leave out the configuration completely and just always use memory for development.
2. Run a test before connecting to the desired service
In combination with the above, if memcache details are provided you could run a quick test by attempting to store something in memcache on startup. Perhaps new Date(); to signal when the application booted up? If this throws an error, then just don't attach the MemcachedStore to the express-session options and you can safely destroy the MemcachedStore.
3. Throw an error if you cannot connect to Memcached
This is in further combination to #2. If you identify that memcache configurations are provided, then I would personally do a check to see if you can contact the serivce and if not then throw an error and stop the application. This would mean that in development you immedietely know the issue, and in production you would as well and can trigger automatic alerts for yourself based on the fact that the application failed to start.
This is probably the best and most robust solution, generally doing a silent fallback is not a great idea when talking about connected services as things can go wrong and you have no idea. I appreciate that this is for development purposes and you need to be pragmatic, but if it saves you accidentally serving all sessions from your servers memory then this would be super beneficial.
I am building a Node.js application, which reads from a MongoDB cluster. The application uses Mongoose to communicate with Mongo.
I would like to build a functionality, which is able to tell me, what does the mongoose know about the MongoDB replica set real-time (like calling rs.status()), but so far I was not able to find any kind of informations around the internet.
The purpose of this would be to be able to monitor, if something was changed in the replica set, and report it back if needed.
The problem is, I've found so far nothing on the internet in this subject. Does anyone have any idea on how to start it? It would be nice, if I could use the current mongoose connections for this purpose.
You can do this, but you need to be connected to the "admin" database and you will probably want a different connection for this other than what the rest of your application uses. Something like:
var mongoose = require("mongoose");
mongoose.connect(
"mongodb://localhost:27017,localhost:27018,localhost:27019/test");
var conn = mongoose.createConnection(
"mongodb://localhost:27017,localhost:27018,localhost:27019/admin");
conn.on("open",function() {
conn.db.command({"replSetGetStatus":1 },function(err,result) {
console.log( result );
});
});
It is important to wait for the connection to be established as well, hence the "event" callback. Mongoose will internally "queue" it's own methods operations until a connection is made, but this does not apply when grabbing a handle to the native db object and executing methods from that.