I have a database on MongoLab. It has several collections. All but one work. One collection is called "selectopts". It has two documents. I can clearly see these two documents.
In my express code I have...
var db = require('mongojs');
db.connect('mongodb://xxxx:xxxx#ds053xxx.mongolab.com:53xxx/rednecks',['selectopts']);
exports.selects = function (req, res) {
db.selectopts.find(function (err, s) {
if (err) return;
res.json(s);
});
};
It always errors at db.selectopts.find..., TypeError: Cannot call method 'find' of undefined. This exact same stupid simple code works fine for four other collections. Why is just this one collection not coming back from MongoLab?
I'm so completely stumped.
EDIT...
Tried db.collection('selectopts').find(... and got this error...
EDIT again...
Here are the two docs in the selectopts collection on MongoLab. Do you see some problem with the docs?...
EDIT x 3...
This is the correct/working mongo connection setup code...
var mongojs = require('mongojs');
var db = mongojs.connect(
'xxx:xxx#ds053xxx.mongolab.com:53xxx/rednecks',
);
See the main difference? (SMFH) :-/
Sometimes when I post at SO I don't receive the exact actual answer from any one reply, but somehow you guys always steer me toward the answer. In this case, it was programmer stupidity. My code at the top of the route js file is wrong. You can see it wrong at the top of this post. I edited and added the correct syntax, which magically got everything working.
To eliminate this type of repetition/syntax/non-DRY bungling, I moved the mongo connection lines into a separate database.js file and require it at the top of the route files. Genius huh?!? :-D
"Thank you" x 100 for all your replies! You always get me back on track :-)
Connect using the following style:
var MongoClient = require('mongodb').MongoClient;
var db;
MongoClient.connect(
'mongodb://xxxx:xxxx#ds053xxx.mongolab.com:53xxx/rednecks',
{auto_reconnect: true},
function(e, database)
{
db = database;
});
exports.selects = function (req, res) {
db.selectopts.find(function (err, s) {
if (err) return;
res.json(s);
});
};
Related
I'm new to javascript, node.js (or backend at all). I am trying to create a controller for the login page requests and I am confused about getting data from the MYSQL table and User Authentication and working with JWT package !
In my Controller, I first check if the user input is available in the user table (with a simple stored procedure), then I compare the database password and the user input, after this I want to create a token and with limited time. (I have watched some tutorial videos about JWT and there is no problem with it), my main problem is to figure out how to write a proper controller with this functions?
I have 2 other questions:
1.Is it the right and secure way to get data from MySQL table inside the route? Or should I create a JS class for my controller? (I'm a bit confused and doubtful here)
2.Assuming that comparePassword() returns true, how can I continue coding outside of the db.query callback function scope? Because I have to execute comparePasssword() inside db.query callback
loginController.js :
const { validationResult } = require('express-validator');
const bcrypt = require('bcrypt');
const db = require('../../sqlConnection')
let comparePassword = (dbPass, inputPass) => {
bcrypt.compare(inputPass, dbPass, function(err, result) {
console.log(result)
});
}
// for get request
exports.getController = (req, res) => {
res.send('login')
}
// for post request
exports.postController = (req, res) => {
let errors = validationResult(req)
if(!errors.isEmpty()) {
res.status(422).json({ errors: errors.array() })
}
// find data from MYSQL table
let sql = `CALL findUser(?)`
db.query(sql, [req.body.username], (err, res) => {
if(err) console.log(err)
//console.log(Object.values(JSON.parse(JSON.stringify(res[0]))))
var data = JSON.stringify(res[0])
data = JSON.parse(data).find(x => x)
data ? comparePassword(data.password, req.body.password) : res.status(400).send('cannot find
user')
})
res.send('post login')
}
login.js :
const express = require('express')
const router = express.Router()
const { check } = require('express-validator');
const loginCont = require('../api/controllers/loginController')
router.route('/')
.get(
loginCont.getController
)
.post(
[
check('username').isLength({min: 3}).notEmpty(),
check('password').isLength({min: 4}).notEmpty()
],
loginCont.postController
)
module.exports = router
In my point of view, looks like there is no easy answer for your question so I will try to give you some directions so you can figure out which are the gaps in your code.
First question: MySQL and business logic on controller
In a design pattern like MVC or ADR (please take a look in the links for the flow details) The Controllers(MVC) Or Actions(ADR) are the entry point for the call, and a good practice is to use these entry points to basically:
Instantiate a service/class/domain-class that supports the request;
Call the necessary method/function to resolve what you want;
Send out the response;
This sample project can help you on how to structure your project following a design pattern: https://riptutorial.com/node-js/example/30554/a-simple-nodejs-application-with-mvc-and-api
Second question: db and continue the process
For authentication, I strongly suggest you to take a look on the OAuth or OAuth2 authentication flow. The OAuth(2) has a process where you generate a token and with that token you can always check in your Controllers, making the service a lot easier.
Also consider that you may need to create some external resources/services to solve if the token is right and valid, but it would facilitate your job.
This sample project should give you an example about how to scope your functions in files: https://github.com/cbroberg/node-mvc-api
Summary
You may have to think in splitting your functions into scoped domains so you can work with them in separate instead of having all the logic inside the controllers, then you will get closer to classes/services like: authenticantion, user, product, etc, that could be used and reused amount your controllers.
I hope that this answer could guide you closer to your achievements.
Hi fellow developers!
I've got this error showing up in my console when I try to save two identical documents in a collection in MongoDB that has nothing to do with the index shown in the error.
Here's the error: E11000 duplicate key error collection: Bohemian.orders index: user.email_1 dup key: { user.email: null }
Now this makes no sense, because I'm trying to save an Order document in a separate collection, which has nothing to do with the user router I had set up previously.
Here is the schema and model code:
const mongoose = require('mongoose');
const orderSchema = new mongoose.Schema({
amountToPay: Number,
});
const Order = mongoose.model("Order", orderSchema);
module.exports.Order = Order;
As shown here, I am only trying to save the amount to be payed into the database in a separate collection.
Here is the router file:
const express = require('express');
const router = express.Router();
const { Order } = require('../models/Order');
router.get("/", async (req, res, next) => {
try {
const orders = await Order.find();
if (orders.length === 0) return res.status(404).send("There are currently no orders");
res.send(orders);
} catch (ex) {
console.error(ex);
next();
}
});
router.post("/", async (req, res, next) => {
try {
const order = new Order({
amountToPay: req.body.amountToPay
});
await order.save();
res.send(order);
} catch (ex) {
console.error(ex);
next();
}
});
module.exports = router;
As you can see there is nothing relative to the error that I'm getting and I have no clue why I'm getting a duplicate user.email = null key , when I haven't made any reference to the User model or router.
Here is the POST call I'm making from POSTMAN to test:
Pretty straight forward, nothing extreme, nothing tangled, right? Well the first ever POST call saves the document in the Database, but from then on I keep getting the same error. The only thing I can take from that is that when I save the first document, Mongo looks for the user.email property when I'm creating the new instance of Order and when it doesnt find it, it creates it with a value of null and then the next document would naturally be a duplicate, hence the error. But I'm confused, because this model and router should not absolutely nothing to do with the user ones.
Here is the error:
So please if anyone can help me understand why MongoDB is screwing with me or where I'm making a mistake, I would really appreciate it.
I found out what the problems was, thanks to Molda:
At some point I had created these indexes, which I'm still unsure when and how, but I did. Which essentially lead to this error, when I tried to save the second document.
A simple quick console log of the indexes in the collection showed that I had that index in there.
const indexes = await Order.collection.getIndexes();
console.log(indexes);
Then I removed them using this method:
await Order.collection.dropIndexes("user.email_1");
And everything worked flawlessly from there.
I hope this helps anyone in this situation in the future and thanks Molda! :)
I have a Node.js server which queries MySQL database. It serves as an api end point where it returns JSON and also backend server for my Express application where it returns the retrieved list as an object to the view.
I am looking into implementing flat-cache for increasing the response time. Below is the code snippet.
const flatCache = require('flat-cache');
var cache = flatCache.load('productsCache');
//get all products for the given customer id
router.get('/all/:customer_id', flatCacheMiddleware, function(req, res){
var customerId = req.params.customer_id;
//implemented custom handler for querying
queryHandler.queryRecordsWithParam('select * from products where idCustomers = ? order by CreatedDateTime DESC', customerId, function(err, rows){
if(err) {
res.status(500).send(err.message);
return;
}
res.status(200).send(rows);
});
});
//caching middleware
function flatCacheMiddleware(req, res, next) {
var key = '__express__' + req.originalUrl || req.url;
var cacheContent = cache.getKey(key);
if(cacheContent){
res.send(cacheContent);
} else{
res.sendResponse = res.send;
res.send = (body) => {
cache.setKey(key,body);
cache.save();
res.sendResponse(body)
}
next();
}
}
I ran the node.js server locally and the caching has indeed greatly reduced the response time.
However there are two issues I am facing that I need your help with.
Before putting that flatCacheMiddleware middleware, I received the response in JSON, now when I test, it sends me an HTML. I am not too well versed with JS strict mode (planning to learn it soon), but I am sure the answer lies in the flatCacheMiddleware function.
So what do I modify in the flatCacheMiddleware function so it would send me JSON?
I manually added a new row to the products table for that customer and when I called the end point, it still showed me the old rows. So at what point do I clear the cache?
In a web app it would ideally be when the user logs out, but if I am using this as an api endpoint (or even on webapp there is no guarantee that the user will log out the traditional way), how do I determine if new records have been added and the cache needs to be cleared.
Appreciate the help. If there are any other node.js caching related suggestions you all can give, it would be truly helpful.
I found a solution to the issue by parsing the content to JSON format.
Change line:
res.send(cacheContent);
To:
res.send(JSON.parse(cacheContent));
I created cache 'brute force' invalidation method. Calling clear method will clear both cache file and data stored in memory. You have to call it after db change. You can also try delete specified key using cache.removeKey('key');.
function clear(req, res, next) {
try {
cache.destroy()
} catch (err) {
logger.error(`cache invalidation error ${JSON.stringify(err)}`);
res.status(500).json({
'message' : 'cache invalidation error',
'error' : JSON.stringify(err)
});
} finally {
res.status(200).json({'message' : 'cache invalidated'})
}
}
Notice, that calling the cache.save() function will remove other cached API function. Change it into cache.save(true) will 'prevent the removal of non visited keys' (like mentioned in comment in the flat-cache documentation.
On my localhost/list page, my GET is showing
[{"_id":"5756f1aa64fa4d3104f98f89","mcr":"relationship","info":{"test":"test"}}]
but I would like to only return
[{"info":{"test":"test"}}]
My find function works just fine on the mongo command line and it returns what is expected.:
db.usercollection.find({},{"_id":0,"mcr":0})
However, when called in my JS file, it doesn't filter out the query. I'm using express and monk along with MongoDB. This is my router.get:
router.get('/list', function(req, res) {
var db = req.db;
var collection = db.get('usercollection');
collection.find({},{"_id":0,"mcr":0},function(e,docs){
res.json(docs);
});
});
No errors are thrown and the status code is 200, what could the problem be? I've tried a ton of variations within the find function, and haven't had any luck.
I found the issue, apparently mongo hadn't updated their docs completely. I needed to use {fields: {_id:0}}.
Final code:
router.get('/list', function(req, res) {
req.db.get('usercollection').find({},{fields: {"_id":0,"mcr":0}},function(e,docs){
res.json(docs);
//console.log(docs);
});
});
I am new with NodeJs and Express frameworks. I have understood that Node works with only one thread on the server side. So, I have noticed this causes me some problems in order to develop correctly my application.
In my routes folder, I have a file index.js.
This file manage the navigation asked by the user from app.js.
So I decided to create a route function "test".
In this function, I had just that code
exports.test = function(req, res){
res.render('test', {});
};
So simple, so easy. That's rend the template test.jade in my views folder. Greats !
But I wanna complexify the process. In this test route function, I want load some content from my MYSQL database.
For that, I have created a folder Models in the folders node_modules
Inside, I have only 2 file, the first mysqlConnection.js which exports the variable DB in order to make queries.
var mysql = require('mysql');
var DB = mysql.createConnection(
{
host : 'localhost',
user : 'root',
password : '',
database : 'test',
}
);
DB.connect();
module.exports = DB;
In the second file, articles_class.js, I just have
var DB = require('models/mysqlConnection');
var Article = function() {
this.getArticles = function()
{
DB.query('SELECT * FROM articles;', function(err, rows, fields)
{
if (err)
throw err;
else
{
console.log(rows);
return (rows);
}
});
}
}
module.exports = Article;
Go back in my route test function :
I just want to load from table "test" all the articles. Very basic. But not easy.
Why ?
Because before the query is finished, NodeJs respond to the client with the template render, but, unfornlty, without the rows loaded. Asynchronous problem ... Mysql doesn't block the Nodejs javascript Instruction.
The code of the function :
exports.test = function(req, res){
var Article = require('models/articles_class');
a = new Article();
articles = a.getArticles();
console.log(articles); // undefined
res.render('test', {});
};
I Found others subjects in stackoverflow which speak about this problem. Make sync queries, work with callbacks ect ..
But for, here, if I try to manage this problem with callbacks, That's cannot work ... Because I need to send to the client the template with articles but I can't block the process with a sync method.
I am very lost ... I don't understand how I have to build my application. I am not able to create a good proceed in order to manage the sql queries. There is a pattern or a specific method ?
Or perhaps I have to make only ajax requests from the client. I load the template "test". And in a javascript file in the public folder, I ask to the server to load me the articles content and wait success callback function ? it's not very clean ...
Thx for your answers. The others answers I have found didn't help me to understand how manage that with NodeJs.
Pass a callback to getArticles:
exports.test = function(req, res){
var Article = require('models/articles_class');
a = new Article();
a.getArticles( function( articles ) {
console.log(articles); // undefined
res.render('test', { articles: articles });
});
};
Changes to your get articles function:
var DB = require('models/mysqlConnection');
var Article = function() {
this.getArticles = function( callback )
{
DB.query('SELECT * FROM articles;', function(err, rows, fields)
{
if (err)
throw err;
else
{
console.log(rows);
callback && callback(rows);
}
});
}
}
module.exports = Article;
Express will only return the template through the open http connection once res.render() is called. So it's just a matter of passing it as a callback through your call stack, so it should only be called after you have your database rows.
As we are working with callbacks, they don't block your application.