I'm using Node.js and Express on Heroku, with the MongoDB addon.
My database connection works fine and I can successfully push some data in, but not other.
Here is the database connection:
mongodb.MongoClient.connect(mongoURI, function (err, database) {
if (err) {
console.log(err);
process.exit(1);
}
// Save database object from the callback for reuse.
db = database;
console.log("Database connection ready");
// Initialize the app.
var server = app.listen(process.env.PORT || dbport, function () {
var port = server.address().port;
console.log("App now running on port", port);
});
});
I can successfully push my Twitter API response into the database like this:
db.collection(TWEETS_COLLECTION).insert(data);
('data' is just a JSON variable)
But when I try to push another JSON variable into the database in the same method, I get an error. Code:
var jsonHash = '{"hashtag":"","popularity":1}';
var objHash = JSON.parse(jsonHash);
objHash.hashtag = req.body.hashtag;
JSON.stringify(objHash);
collection(HASHTAG_COLLECTION).insert(jsonHash);
And the error:
TypeError: Cannot create property '_id' on string '{"hashtag":"myhash","popularity":1}'
at Collection.insertMany...
...
Any ideas what I'm doing wrong?
I don't know where you are getting the jsonHash variable from but I think you are doing unecessary JSON-handling here. You are also inserting the wrong variable, you want to insert objHash which is a valid object to insert, now you are inserting jsonHash which is just a string. JSON.stringify(objHash); is not doing anything as you are not saving the JSON returned from the function. I think you want something like this?
var objHash = {
hashtag: "",
popularity:1
};
objHash.hashtag = req.body.hashtag;
collection(HASHTAG_COLLECTION).insert(objHash);
jsonHash is still a string. May be you want to save objHash instead without JSON.stringify ?
Related
I have a Node.js server which queries MySQL database. It serves as an api end point where it returns JSON and also backend server for my Express application where it returns the retrieved list as an object to the view.
I am looking into implementing flat-cache for increasing the response time. Below is the code snippet.
const flatCache = require('flat-cache');
var cache = flatCache.load('productsCache');
//get all products for the given customer id
router.get('/all/:customer_id', flatCacheMiddleware, function(req, res){
var customerId = req.params.customer_id;
//implemented custom handler for querying
queryHandler.queryRecordsWithParam('select * from products where idCustomers = ? order by CreatedDateTime DESC', customerId, function(err, rows){
if(err) {
res.status(500).send(err.message);
return;
}
res.status(200).send(rows);
});
});
//caching middleware
function flatCacheMiddleware(req, res, next) {
var key = '__express__' + req.originalUrl || req.url;
var cacheContent = cache.getKey(key);
if(cacheContent){
res.send(cacheContent);
} else{
res.sendResponse = res.send;
res.send = (body) => {
cache.setKey(key,body);
cache.save();
res.sendResponse(body)
}
next();
}
}
I ran the node.js server locally and the caching has indeed greatly reduced the response time.
However there are two issues I am facing that I need your help with.
Before putting that flatCacheMiddleware middleware, I received the response in JSON, now when I test, it sends me an HTML. I am not too well versed with JS strict mode (planning to learn it soon), but I am sure the answer lies in the flatCacheMiddleware function.
So what do I modify in the flatCacheMiddleware function so it would send me JSON?
I manually added a new row to the products table for that customer and when I called the end point, it still showed me the old rows. So at what point do I clear the cache?
In a web app it would ideally be when the user logs out, but if I am using this as an api endpoint (or even on webapp there is no guarantee that the user will log out the traditional way), how do I determine if new records have been added and the cache needs to be cleared.
Appreciate the help. If there are any other node.js caching related suggestions you all can give, it would be truly helpful.
I found a solution to the issue by parsing the content to JSON format.
Change line:
res.send(cacheContent);
To:
res.send(JSON.parse(cacheContent));
I created cache 'brute force' invalidation method. Calling clear method will clear both cache file and data stored in memory. You have to call it after db change. You can also try delete specified key using cache.removeKey('key');.
function clear(req, res, next) {
try {
cache.destroy()
} catch (err) {
logger.error(`cache invalidation error ${JSON.stringify(err)}`);
res.status(500).json({
'message' : 'cache invalidation error',
'error' : JSON.stringify(err)
});
} finally {
res.status(200).json({'message' : 'cache invalidated'})
}
}
Notice, that calling the cache.save() function will remove other cached API function. Change it into cache.save(true) will 'prevent the removal of non visited keys' (like mentioned in comment in the flat-cache documentation.
I'm using the combo Express (Node.js) and Mongoose to make a REST API. I'm trying to make the login using a JWT token but I've got a problem. When I execute the following code
const mongoose = require('mongoose');
const User = mongoose.model('User');
// other code
_api.post('/login', function (req, res) {
const data = req.body;
// some data control
User.findOne({ username: data.username}, function(err, doc) {
if (hash(password) == doc.password) { // password check
myToken = generateToken(); // generating the token
doc.jwtToken = myToken; // including the generated token to the response
res.status(200).json(doc); // return the final JSON to client
}
}
}
the final JSON returned by the API doesn't have the field "jwtToken":"mygeneratedtoken" and this is strange. I included other times new fields inside a JSON with the same syntax and it worked. I tried to use a tmp variable to which I assigned the doc content (that is a javascript object) and then I added the jwtToken filed and return the tmp variable. But nothing.
Can someone explain me if there is something wrong with my code or if there is something that I need to know?
Documents returned by mongoose are immutable, and thus assignment to doc.jwtToken does not modify the object. You can either use the lean method to modify the query, or toObject to convert the document to a regular javascript object. Try:
var docObject = doc.toObject();
docObject.jwtToken = myToken;
res.status(200).json(docObject);
Background Information
I'm attempting my first node.js API/application. As a learning exercise, I'm trying to create some test cases initially delete all records in a table, insert 3 specific records, and then query for those 3 records.
Code
Here's the code I have cobbled together:
http://pastebin.com/duQQu3fm
Problem
As you can see from the code, I'm trying to put the database connection logic in a dbSession.js file and pass it around.
I am able to start up the http server by doing the following:
dev#devbox:~/nimble_node$ sudo nodejs src/backend/index.js
Server started and listening on port: 8080
Database connection successful
However, when I try to run my jasmine tests, it fails with the following error:
F
Failures:
1) The API should respond to a GET request at /api/widgets/
Message:
TypeError: Object #<MongoClient> has no method 'collection'
Stacktrace:
TypeError: Object #<MongoClient> has no method 'collection'
at resetDatabase (/home/dev/nimble_node/spec/resetDatabase.js:6:29)
at /home/dev/nimble_node/spec/e2e/apiSpec.js:23:25
at /home/dev/nimble_node/node_modules/async/lib/async.js:683:13
at iterate (/home/dev/nimble_node/node_modules/async/lib/async.js:260:13)
at async.forEachOfSeries.async.eachOfSeries (/home/dev/nimble_node/node_modules/async/lib/async.js:279:9)
at _parallel (/home/dev/nimble_node/node_modules/async/lib/async.js:682:9)
at Object.async.series (/home/dev/nimble_node/node_modules/async/lib/async.js:704:9)
at null.<anonymous> (/home/dev/nimble_node/spec/e2e/apiSpec.js:19:9)
at null.<anonymous> (/home/dev/nimble_node/node_modules/jasmine-node/lib/jasmine-node/async-callback.js:45:37)
Finished in 0.01 seconds
1 test, 1 assertion, 1 failure, 0 skipped
Database connection successful
Line 6 of resetDatabase is:
var collection = dbSession.collection('widgets');
Given that after the error appears, I get the "Database connection successful" message, I think what's happening is that when the tests request the dbSession library, the database hasn't finished running the code to connect. And therefore, I can't get the collection object.
I'm currently reading through the mongodb online manual to see if I can find some hints as to how to do something like this.
Any suggestions or pointers would be appreciated.
EDIT 1
To prove that there is a collection method on the MongoClient object, I changed the dbSession.js code to look like this:
'use strict';
var DBWrapper = require('mongodb').MongoClient;
var dbWrapper = new DBWrapper;
dbWrapper.connect("mongodb://localhost:27017/test", function(err, db) {
if (!err) {
console.log("Database connection successful");
dbWrapper = db;
var collection = dbWrapper.collection('widgets');
console.log('just created a collection...');
}
});
module.exports = dbWrapper;
And now, when I start up the http server (index.js), notice the messages:
dev#devbox:~/nimble_node$ sudo nodejs src/backend/index.js
Server started and listening on port: 8080
Database connection successful
just created a collection...
It could be an async issue.
Your code in dbSessionjs
dbWrapper.connect("mongodb://localhost:27017/test", function(err, db) {
if (!err) {
console.log("Database connection successful");
dbWrapper = db;
}
});
module.exports = dbWrapper;
Starts the connection at dbWrapper asynchronously, but exports dbWrapper right away, which is then imported in resetDatabase. Thus yes, the connect function may have not yet returned from the async function when you call it in resetDatabase (and is what the log suggests,as the error appears before the success log).
You could add a callback after dbWrapper.connect() returns, in order to actually only be able to use dbWrapper when the connection finished.
(With sqlite, this may not happen as it accesses the DB faster on the commandline).
This may not be your problem but looks like a candidate.
EDIT: Here's a possible example for a callback, but please take note it depends on what you need to do so there are a lot of different solutions. The key is to call a callback function when you are done initializing.
Another solution could be to simply wait, and/or poll (e.g. chcke a variable 'initialized').
'use strict';
var DBWrapper = require('mongodb').MongoClient;
var dbWrapper = new DBWrapper;
function doConnect(callback) {
console.log("Initializing DB connection...");
dbWrapper.connect("mongodb://localhost:27017/test", function(err, db) {
if (!err) {
console.log("Database connection successful");
dbWrapper = db;
var collection = dbWrapper.collection('widgets');
console.log('just created a collection...');
console.log('calling callback...');
callback(dbWrapper);
} else {
console.log("Error connectingi: " + err);
}
});
};
doConnect(function(correctDbWrapper) {
//Now you can use the wrapper
console.log("Inside callback, now consuming the dbWrapper");
dbWrapper = correctDbWrapper;
var collection = dbWrapper.collection('widgets');
});
It's interesting though I never ran into this issue, although I have generally used similar code like yours. I guess because normally I have this DB initialization right at the top, and then have to do lots of initializations on the node app, which gives the app time enough to return from the connect call....
I was trying to use findOne method. But it didn't show anything.It looks like it didn't execute. Would you like to help me solve this problem?
var mongojs = require('mongojs');
var databaseUrl = "mongodb:local:27017/mydb";
var db = mongojs(databaseUrl, ["profiles"]);
var password;
db.profiles.findOne({"userId": "liu1234"}, function(err, doc) {
if (err) throw err;
else console.log(doc);
});
The format of databaseUrl is incorrect. The mongodb driver is unable to find your database.
Try:
var databaseUrl = "mongodb://localhost:27017/mydb";
The first part, mongodb://, refers to the protocol that mongodb uses to interact with the database. The next part, localhost , is a hostname that points to your machine. :27017 refers to the default port that mongodb communicates over. And, obviously, /mydb refers to your database.
If you're using a default configuration, you don't even need to specify the protocol, the host, or the port. Mongojs assumes the defaults if you don't enter them, so you can use this instead:
var databaseUrl = "mydb";
For more information check out: https://github.com/mafintosh/mongojs
Short here's the purpose of my application.
I have a PHP file reading data through an ODBC connection and writing them to a local database.
This PHP file needs to be run LOCAL ON THE SERVER each time a loop is processed in my node.js, socket.io server.
My setup is Apache, PHP 5.5.12 and node.js.
I was pretty convinced there was a simple way to this but I haven't had any luck getting Ajax or similar to work inside node.js by following other guides.
The code where the file should be processed inside looks like this.
var pollingLoop = function () {
// Update the local database
// HERE I WANT THE PHP FILE TO EXECUTE
// Make the database query
var query = connection.query('SOME LONG SQL QUERY IN HERE'),
status = []; // this array will contain the result of our db query
// set up the query listeners
query
.on('error', function(err) {
// Handle error, and 'end' event will be emitted after this as well
console.log( err );
updateSockets( err );
})
.on('result', function( runningstatus ) {
// it fills our array looping on each runningstatus row inside the db
status.push( runningstatus );
})
.on('end',function(){
// loop on itself only if there are sockets still connected
if(connectionsArray.length) {
pollingTimer = setTimeout(pollingLoop, POLLING_INTERVAL);
updateSockets({status:status});
}
});
};
Am I totally of track by trying to do that?
Try invoking php through the shell interface:
var exec = require("child_process").exec;
app.get('/', function(req, res){exec("php index.php", function (error, stdout, stderr) {res.send(stdout);});});
OR this link