I am working on a backend application with node using typescript / javascript. My backend is using a sqlite database. I structured my project so that there is one file that contains all the database logic.
I could either write a module that will be required by all modules that or I could write a class which connects to the database in the constructor.
For me it seems a little bit weird to pack all database logic in a module that is not an object. What is the best practice in this case and why? (I know this might be a stupid question but I'm just a hobbyist)
Thanks in advance
In my case I make the connection to the database before running the server. In pseudocode:
connectorDB.connect( path, config, () => {
server.listen( port )
})
It will depend on how important the database connection is in the operation of your app.
Related
I am trying to create a webpage that uses data from a MariaDB. My current idea (which has been giving me a lot of trouble) is to just connect to the database from the app.js file, which is the main script for my index.html.
const dotenv = require("dotenv");
dotenv.config();
const mariadb = require("mariadb");
const pool = mariadb.createPool({
database: process.env.DATABASE,
host: process.env.HOST,
user: process.env.USER_TOKEN,
password: process.env.PASSWORD,
});
// the rest of the code involves selecting from the db, and parsing the data
However, I have been running into many issues. I'm not too knowledgeable on all this, but I found that I need to webpack the file if I want to be able to use the "require" keyword. But I could not figure that out as I kept running into weird issues when using Browserify; I think there may be an incompatibility with MariaDB. I also looked into using JS modules, but I am not sure if that is possible with MariaDB.
I am trying to come up with another solution, potentially using some sort of API to a back end, which would make the GET request to the database, but I feel like it should not have to be that complicated for my sake (I also wouldn't really know where to start with this). All I basically want to do, is make a GET request to a MariaDB when the page loads on the client's browser and display that data on the webpage. Is there a simple way to do this?
I suggest you use nodejs to connect and query database as it will greatly resolve a lot of overhead for you..
The easiest way i can think of is using a prisma starter template here
https://github.com/prisma/prisma-examples/tree/latest/javascript/script
It also gives the added advantage of the ORM function...
Hope it helps.
So I figured the best way to learn is to try and fail over and over. I am building a webapp, at least trying to. I am curious how to go about using node to query my db. I am able to make a connection to the db with my single app.js file.
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'xxxxxxxx-us-east-1.rds.amazonaws.com',
port : '3306',
user : 'user',
password : 'password',
database : 'app'
});
connection.connect(function(err){
if(!err) {
console.log("Database is connected ... ");
} else {
console.log("Error connecting database ... ");
}
});
My problem, or lack of understand begins when I try to integrate this into my client-side js code. For instance say I wanted to trigger the db connection when a user uploads a photo.
var upload = s3.putObject({
Bucket: albumBucketName,
Key: photoKey,
Body: file,
ACL: "public-read",
});
var promise = upload.promise();
Can I include the app.js node file?
Sorry if this is a dumb question. I feel like I am missing some fundamental understanding of how to integrate the functionality of node with my current client side JS. Any help or further reading is appreciated--I am even curious about PHP solutions.
X
Server and client side code are separate. However you can create a Node module that harnesses the AWS and returns an appropriate response to the client after completed.
To do this, you need to create an endpoint that you post your data to from the client, then process with the same AWS modules only for Node. You also need to be able to access the connection instance from a different NodeJS module. This can be accomplished several ways. First, if the library that instantiates the connection tracks all of the connections, you should be able to require the library in a different module, then use the library's API to access one of the connections. Second, if you create only one instance of the connection and allow it to export, then you can import the module with that connection. Third, you can use something like a request/response pattern between the two modules, with the pattern instance declared globally.
I'm using node.js and mongoDB. Right now, for my test app, the connection to the db is in the main node file, but I guess this is a wrong practice.
What I want/need: a secure way (i.e. not storing password on files users can access) to connect to the db just when needed.
For example: I want several admin pages (users, groups, etc..). Each page should connect to the db, find some data, and display it. It also have a form for adding a document to the db and a delete option.
I thought maybe to create some kind of a connection function - send it what you want to do (add, update, find, delete), to where (collection name) and whatever it needs. But I can't just include this function, because then it'll reveal the password to the db. So what can I do?
Thanks!
I'm going to answer your question bit by bit.
Right now, for my test app, the connection to the db is in the main node file
This is fine, though you might want to put it in a separate file for easier reuse. NodeJS is a continuesly running process, so in theory you could serve all of your HTTP responses using the same connection to the database. In practice you'd want to create a connection pool, but the Mongodb driver for NodeJS already does this automatically.
Each page should connect to the db, find some data, and display it.
When you issue a query on the MongoDB driver, it will automatically use a connection from its internal connection pool, as long as you gave it the credentials when your application was starting up.
What I want/need: a secure way (i.e. not storing password on files users can access) to connect to the db just when needed.
I would advice to keep your application configuration (any variables that depend on the environment in which the app is running) in a separate file which you don't commit to your VCS. A module like node-config can help a great deal with that.
The code you will end up with, using node-config, is something like:
config/default.json:
{
"mongo": null
}
This is the default configuration file which you commit.
config/local.json:
{
"mongo": "mongo://user:pass#host:port/db"
}
The local.json should be ignored by your VCS. It contains secret sauce.
connection.js:
var config = require('config');
var MongoClient = require('mongodb').MongoClient;
var cache;
module.exports = function(callback){
if(cache){
return callback(cache);
}
MongoClient.connect(config.get('mongo'), function(err, db){
if(err){
console.error(err.stack);
process.exit(1);
}
cache = db;
callback(db);
});
}
An incomplete example of how you might handle reusing the database connection. Note how the configuration is gotten using config.get(*). An actual implementation should have more robust error handling and prevent multiple connections from being made. Using Promises would make all that a lot easier.
index.js:
var connect = require('./connection');
connect(function(db){
db.find({whatever: true})
});
Now you can just require your database file anywhere you want, and reuse the same database connection, which handles pooling for you and you don't have your passwords hard-coded anywhere.
I very much like Meteor's pub/sub. I wonder if there is a way to get a similar workflow, using sails.js or just a socket library in general.
In particular, what I would like to be able to do is something along the lines of:
// Server-side:
App.publish('myCollection', -> collection.find({}))
// Client-side:
let myCollection = App.subscribe('myCollection')
let bob = myCollection.find({name: 'Bob'})
myCollection.insert({name: 'Amelie'}, callback)
All interaction with the server should happen in the background.
I very much like Meteor's pub/sub. I wonder if there is a way to get a similar workflow, using sails.js or just a socket library in general
Basically yes, at least about realtime sync between backend and frontend. Let's review what meteor's have and answer point by point.
Pub/sub
The Pub / Sub concept, as stated by Sabbir, is also supported by sails.js. Though the basics are slightly different :
In meteor, the client can subscribes to everything he wants, and the server control what it receives by only publishing to who he wants;
whereas in sails.js, the server both does subscribe some clients sockets and publish to all binded sockets
Note that, by default:
meteor contains the autopublish package that just notify every client without any kind of filtering. To acheive some filtering, you have to meteor remove autopublish then you can handle what will your client receive by adding a mongo request to it, like explained here.
sails by default, on its automatic "select" blueprints actions, auto-subscribes the calling socket to the events on the objects returned by the "select".
As a server-side conclusion:
Subscribe: just call findor findOne blueprint default action, through a socket (attaching some where filters or not) and your socket will automatically be subscribed to every events concerning returned objects => you don't have to code anything on the server, in most cases, for the Subscribe logic.
Publish: every blueprint default actions (create, update, destroy, add, remove) auto-publish to subscribed sockets => you don't have to code anything on the server, in most cases, for the Publish logic.
(Though, if you find yourself implementing some manual controller actions, sails API helps you publishing and subscribing easily)
Client handling
Therefore, with both meteor and sails, clients only receive what they're supposed to receive. Time for front-end now.
Philosophy
meteor in one hand, with it's isomorphic dimension, does provide a front-end connector by nature, exposing it's data-bound collections.
sails on the other hand, is front-end agnostic, and can be attacked by any http REST connector (JS or not), such as $http, $resource, or more advanced ones like Restangular.
Though, being aware of the complexity using raw sockets on their API (when it comes to session, CORS, CSRF and stuff), they developped a javascript socket.io wrapper called sails.io.js designed to be REST-like-over-socket, and just works like a charm.
Basically, The main difference is that meteor is one step higher-level than sails, because it provides the logic of syncing collections and objects.
All interaction with the server should happen in the background.
sails.io.js, the official front-end component, is just not that high-level. When it comes to Angular.js.
Though, you can find some community connectors that aim to, kinda, provide the same feature as mongo data-bound collections and objects. There is sails-resource, spinnaker or angular resource sails. I tried both of them, and I should say that I was disapointed. The abstraction level is so high that it just becomes annoying, IMHO. For example, with not-very-RESTful-friendly custom actions, like a login, it becomes very hard to adapt it for your needs.
==> I would advice to use a low-level connector, such as angularSails or (my prefered) https://github.com/janpantel/angular-sails, or even raw sails.io.js if you're not using Angular.
Edit: just foun a backbone version, by the sails' creator
It just works great, and believe me, the "keep my collection in sync with that socket" code is so ridiculous, that finding a module for this is just not worth it.
Some code please, stop talking
In particular, what I would like to be able to do is something along the lines of:
Server
Meteor
# Server-side:
App.publish('myCollection', -> collection.find({}))
Sails
//Nothing to do, just sails generate api myCollection
Client
Meteor
# Client-side:
myCollection = App.subscribe('myCollection')
Sails, with sails.io.js
(Here using lodash for convenience)
var myCollection;
sails.io.get('/myCollection').then(
function(res) {
myCollection = res.data;
},
function(err) {
//Handle error
}
);
sails.io.on('myCollection').function(msg) {
switch(msg.verb) {
case 'created':
myCollection.push(msg.data);
break;
case 'updated':
_.extend(_.find(myCollection, 'id', msg.id), msg.data);
break;
case 'destroyed':
_.remove(myCollection, 'id', msg.id);
break;
};
});
(I leave the find where and create to your imagination with [the doc])
All interaction with the server should happen in the background.
Well, Sails, only for angular, with sails ressources
I'm not pretty used to that process, so I leave you reading here or here, but once again I'd choose manual .on()method.
Since I asked this question, I've learned a few things and some new projects have popped up. I decided against sails.io, because when developing with React.js, most of the community's weight is behind webpack, but sails.io uses gulp. I realize these can be used together and there is even an npm package for this, but I wasn't too keen on making my stack bigger than it had to be, so I went with a simple express.js server that I could tailor to my needs.
In order to sync my data, I'm using rethinkdb which allows me to asynchronously watch the database for changes and then publish the changes to the clients through websockets.
I've set up a simple script where I keep an instance of a baobab tree on both the client and the server.
When the tree gets modified on the server, it sends transaction data to the appropriate clients through the websocket
The client merges the transaction with the tree.
This method does not make use of local storage and keeps the data in memory in the node.js process. The data in the transaction is also quite redundant.
The future plan has always been to set something up using redis and local storage ...
... until yesterday when I found deepstream.io!
This is a tool that does exactly what I want and need! Nothing more, nothing less.
Another project worth mention is meatier: "like meteor, but meatier". It is composed of many other well supported open source projects, so you could even pick and choose.
I am new to MEAN stack, I am trying to create a basic one page application at the moment.
I am trying to connect to the mongodb and then list the values in a certain collection in a controller.
However, when I looked for the answer, I came across this answer
Using AngularJs and MongoDB/Mongoose
Which then confuses me as what is the point of having the code below if you can't use it between angular and mongo ? Or are there other interim steps that use it?
var mongoose = require('mongoose');
var db = mongoose.createConnection('mongodb://localhost:3000/database');
var orderSchema = new mongoose.Schema({
routeFrom : String,
routeTo : String,
leaving: String
});
var Order = db.model('Order', orderSchema);
module.exports = Order;
Edit: The situation i am trying to use it in is such:
Geek.html
<div class="jumbotron text-center">
<h1>Geek City</h1>
<p>{{tagline}}</p>
<ul>
<li ng-repeat="value in dataValues">
{{value.name}}
</li>
</ul>
</div>
GeekController
angular.module('GeekCtrl', []).controller('GeekController', function($scope) {
$scope.tagline = 'The square root of life is pi!';
$scope.dataValues = function(){
var mongo = require('../config/db.js');
var collectionValues = mongo.myCollection.find();
return collectionValues;
};
});
You cannot require db.js config file in Angular because it's not set to be used on the client side. What you describe is so called 'Isomorphic' approach.
What I mean by that: mongo is a database system (roughly speaking). To get data from the database, we usually don't trust the client. So we have some server-side code (PHP, Ruby, Node.js, Java, what have you) which authorizes the client, processes and filters the data and returns it to the client (in this case Angular.js). So your Mongoose models are set to be used by the server-side javascript and that part of the app. That server side should also serve data to Angular so you'd connect to Node.js from Angular, not directly to Mongo. So the same server that (usually) serves your angular files, will also serve the data it reads from mongo.
If you want server-less data with Angular, you can take a look at Firebase.js. It's angular-ready and it could help you not mess around with Mongo, mongoose and the server-side code.
You could try a hybrid approach with something like meteor.js or backbone.js set to work both on client and server, or take a look at this article for more info.
Or for what it's worth, if you want to run your own Mongo, you could start mongo with --rest, then you'd be able to connect to Angular directly to Mongo, at http://somehost:28017/mydatabase or something similar, depending on your setup.
Mongoose is a node module, and as far as I know it doesn't have a front end component, so you won't be using it directly in your frontend js code. It's only going to help you on the server side of your app. If you're relatively new to Node then this stuff can get pretty confusing, since it's all end-to-end javascript and sometimes it's not clear what modules work on the server or frontend, especially since some can do both.
Node, MongoDB, Express, and Mongoose all live on the server.
Angular lives in the browser, and can't use any of the server-side components directly.
Using the MEAN stack, You will be building a node app that uses mongoose to talk to mongodb and express to expose an api to your front end. Then in in your html/js code you'll be using angular and its $http service to talk to the server to get and set data.
There is a great tutorial that walks you through the entire process on scotch.io:
http://scotch.io/bar-talk/setting-up-a-mean-stack-single-page-application