Develop Reference

Pushing String into database utilizing nodeJS and mongoDB

I am new to JS and I am utilizing the MEAN stack to create a place where students can add classes to their user profile. I already have a session store in my database, so "req.user" will return the currently logged in user information and specifically "req.user.id" will return the currently logged in user's id. Also, I have figured out how to search a course in my database from my application. Ultimately, my goal is that when the user makes the post request to search in the database, I also want those "values" to be pushed into the classes "key". I have provided two options, both of which do not add the respective strings to the database. Thank you for any help!
Portion of Search.JS Option #1
router.post('/', ensureAuthenticated, function (req,res,next) {
var query = {course: req.body.coursename};
db.collection('courses').find(query).toArray()
.then(db.collection('DefaultUser').update({_id: req.user.id}, {$push: {classes: req.body.coursename}}));
res.render('search', {title: 'search'})
});
Portion of Search.JS Option #2
router.post('/', ensureAuthenticated, function(req,res,next) {
var query = {course: req.body.coursename};
db.collection('courses').find(query).toArray((err, results) => {
if (err) {
throw err;
} else {
db.collection('DefaultUser').updateOne({_id: '5c17161d3347e79410ff29ba'}, {
$push: {
classes: req.body.coursename
}
})
console.log(results)
res.render('search', {courses: results, title: 'search'})
}
})
});

Some tips may help:
req.body will hold nothing if you forget to use app.use(express.urlencoded()) to parse the posted data.
You may use ObjectId('<string>') (in option #2) for finding and updating queries, not just the string, because mongodb stores _id as ObjectId type by default.
You may use $addToSet instead of $push modifier to add a unique element to an array, or you may get one student subscribed with two more same class, if he repeatedly submit the form.
In your code, you find in the courses collection first and then update, but since you did nothing with the find result, it is not necessary (empty result does not throw any error). Checking the data is valid is good practice, if you would like to do so, in both options, you need to use findOne instead of find to make sure the course is existed, and .then(course => {/* check if the course is null and then throws an error */} ).
I don't have the full code of your project so I can only guess there may be the problems listed above, wish you good luck!

How to add new fields to existing users

I'm having a big deal - the meteor app I've been developing the last weeks is finally online. But, for an update, I need to add a field to my users profile.
I thought that walling a methods with the following code would work :
updateUsrs_ResetHelps: function(){
if(Meteor.users.update({}, {
$set: {
'profile.helps': []
}
}))
console.log("All users profile updated : helps reset");
else
throw new Meteor.Error(500, 'Error 500: updateUsrs_ResetHelps',
'the update couldn\'t be performed');
}
The problem is that my users have the classic Meteor.accounts document, whith emails, _id, services, profile, etc... but, in the profile, they don't have a .helps fields. I need to create it.
For the future users, I've modified the accounts creation function to add this fields when they sign up, but for the 200 users I already got signed up, I do really need a solution.
EDIT : Might it be because of the selector in the update ? Is a simple {} selector valid to update all the users / documents of the collection at once ?

From the Mongo documentation (http://docs.mongodb.org/manual/reference/method/db.collection.update/):
By default, the update() method updates a single document. Set the
Multi Parameter to update all documents that match the query criteria.
If you've already taken care of adding the field for new users and you just need to fix the old ones, why not just do it one time directly in the database?
Run meteor to start your application, then meteor mongo to connect to the database. Then run an update on records where the field doesn't already exist. Something like:
db.users.update({"profile.helps": {"$exists": false}}, {"$set": {"profile.helps": []}}, {multi:true})
The Mongo documentation specifies the multi parameter as:
Optional. If set to true, updates multiple documents that meet the
query criteria. If set to false, updates one document. The default
value is false.

Insert an array of documents into a model

Here's the relevant code:
var Results = mongoose.model('Results', resultsSchema);
var results_array = [];
_.each(matches, function(match) {
var results = new Results({
id: match.match_id,
... // more attributes
});
results_array.push(results);
});
callback(results_array);
});
}
], function(results_array) {
results_array.insert(function(err) {
// error handling
Naturally, I get a No method found for the results_array. However I'm not sure what else to call the method on.
In other functions I'm passing through the equivalent of the results variable here, which is a mongoose object and has the insert method available.
How can I insert an array of documents here?
** Edit **
function(results_array) {
async.eachLimit(results_array, 20, function(result, callback) {
result.save(function(err) {
callback(err);
});
}, function(err) {
if (err) {
if (err.code == 11000) {
return res.status(409);
}
return next(err);
}
res.status(200).end();
});
});
So what's happening:
When I clear the collection, this works fine.
However when I resend this request I never get a response.
This is happening because I have my schema to not allow duplicates that are coming in from the JSON response. So when I resend the request, it gets the same data as the first request, and thus responds with an error. This is what I believe status code 409 deals with.
Is there a typo somewhere in my implementation?
Edit 2
Error code coming out:
{ [MongoError: insertDocument :: caused by :: 11000 E11000 duplicate key error index:
test.results.$_id_ dup key: { : 1931559 }]
name: 'MongoError',
code: 11000,
err: 'insertDocument :: caused by :: 11000 E11000 duplicate key error index:
test.results.$_id_ dup key: { : 1931559 }' }
So this is as expected.
Mongo is responding with a 11000 error, complaining that this is a duplicate key.
Edit 3
if (err.code == 11000) {
return res.status(409).end();
}
This seems to have fixed the problem. Is this a band-aid fix though?

You seem to be trying to insert various documents at once here. So you actually have a few options.
Firstly, there is no .insert() method in mongoose as this is replaced with other wrappers such as .save() and .create(). The most basic process here is to just call "save" on each document you have just created. Also employing the async library here to implement some flow control so everything just doesn't queue up:
async.eachLimit(results_array,20,function(result,callback) {
result.save(function(err) {
callback(err)
});
},function(err) {
// process when complete or on error
});
Another thing here is that .create() can just take a list of objects as it's arguments and simply inserts each one as the document is created:
Results.create(results_array,function(err) {
});
That would actually be with "raw" objects though as they are essentially all cast as a mongooose document first. You can ask for the documents back as additional arguments in the callback signature, but constructing that is likely overkill.
Either way those shake, the "async" form will process those in parallel and the "create" form will be in sequence, but they are both effectively issuing one "insert" to the database for each document that is created.
For true Bulk functionality you presently need to address the underlying driver methods, and the best place is with the Bulk Operations API:
mongoose.connection.on("open",function(err,conn) {
var bulk = Results.collection.initializeUnorderedBulkOp();
var count = 0;
async.eachSeries(results_array,function(result,callback) {
bulk.insert(result);
count++;
if ( count % 1000 == 0 ) {
bulk.execute(function(err,response) {
// maybe check response
bulk = Results.collection.initializeUnorderedBulkOp();
callback(err);
});
} else {
callback();
}
},function(err) {
// called when done
// Check if there are still writes queued
if ( count % 1000 != 0 )
bulk.execute(function(err,response) {
// maybe check response
});
});
});
Again the array here is raw objects rather than those cast as a mongoose document. There is no validation or other mongoose schema logic implemented here as this is just a basic driver method and does not know about such things.
While the array is processed in series, the above shows that a write operation will only actually be sent to the server once every 1000 entries processed or when the end is reached. So this truly does send everything to the server at once.
Unordered operations means that the err would normally not be set but rather the "response" document would contain any errors that might have occurred. If you want this to fail on the first error then it would be .initializeOrderedBulkOp() instead.
The care to take here is that you must be sure a connection is open before accessing these methods in this way. Mongoose looks after the connection with it's own methods so where a method such as .save() is reached in your code before the actual connection is made to the database it is "queued" in a sense awaiting this event.
So either make sure that some other "mongoose" operation has completed first or otherwise ensure that your application logic works within such a case where the connection is sure to be made. Simulated in this example by placing within the "connection open" event.
It depends on what you really want to do. Each case has it's uses, with of course the last being the fastest possible way to do this as there are limited "write" and "return result" conversations going back and forth with the server.

Add a new field to a document mongodb

I am very new to mongodb and have a basic question that I am having trouble with. How do I get the ID field of a document that has already been created? I need the ID so i can update/add a new field to the document.
//newProfile is an object, one string it holds is called school
if(Schools.find({name: newProfile.school}).fetch().length != 1){
var school = {
name: newProfile.school
}
Meteor.call('newSchool', school);
//Method 1 (doesn't work)
var schoolDoc = Schools.findOne({name: newProfile.school});
Schools.update({_id: schoolDoc._id}, {$set: {enrolledStudents: Meteor.user()}});
//Method 2?
//Schools.update(_id: <what goes here?>, {$push: {enrolledStudents: Meteor.user()}});
}
else {
//Schools.update... <add users to an existing school>
}
I create a new school document if the listed school does not already exist. Schools need to hold an array/list of students (this is where i am having trouble). How do I add students to a NEW field (called enrolledStudents)?
Thanks!

I'm having some trouble understanding exactly what you're trying to do. Here's my analysis and understanding so far with a couple pointers thrown in:
if(Schools.find({name: newProfile.school}).fetch().length != 1){
this would be more efficient
if(Schools.find({name: new Profile.school}).count() != 1) {
Meteor.call('newSchool', school);
Not sure what you're doing here, unless you this will run asynchronously, meaning by the time the rest of this block of code has executed, chances are this Meteor.call() function has not completed on the server side.
//Method 1 (doesn't work)
var schoolDoc = Schools.findOne({name: newProfile.school});
Schools.update({_id: schoolDoc._id}, {$set: {enrolledStudents: Meteor.user()}});
Judging by the if statement at the top of your code, there is more than one school with this name in the database. So I'm unsure if the schoolDoc variable is the record you're after.

I believe you are having trouble because of the asynchronous nature of Meteor.call on the client.
Try doing something like this:
// include on both server and client
Meteor.methods({
newSchool: function (school) {
var newSchoolId,
currentUser = Meteor.user();
if (!currentUser) throw new Meteor.Error(403, 'Access denied');
// add some check here using the Meteor check/match function to ensure 'school'
// contains proper data
try {
school.enrolledStudents = [currentUser._id];
newSchoolId = Schools.insert(school);
return newSchoolId;
} catch (ex) {
// handle appropriately
}
}
});
// on client
var schoolExists = false;
if (Schools.findOne({name: newProfile.school})) {
schoolExists = true;
}
if (schoolExists) {
var school = {
name: newProfile.school
};
Meteor.call('newSchool', school, function (err, result) {
if (err) {
alert('An error occurred...');
} else {
// result is now the _id of the newly inserted record
}
})
} else {
}
Including the method on both the client and the server allows Meteor to do latency compensation and 'simulate' the insert immediately on the client without waiting for the server round-trip. But you could also just keep the method on the server-side.
You should do the enrolledStudents part on the server to prevent malicious users from messing with your data. Also, you probably don't want to actually be storing the entire user object in the enrolledStudents array, just the user _id.

For what you're trying to do, there is no need to get the _id. When you use update, just switch out the {_id: schoolDoc._id} with your query. Looks like using {name: newProfile.school} will work, assuming that the rest of your code does what you want it to do.
While that would work with the normal Mongo driver, I see that Meteor does not allow your update query to be anything but _id: Meteor throws throwIfSelectorIsNotId exception
First, make sure that you're pulling the right document, and you can try something like this:
var school_id = Schools.findOne({name: newProfile.school})._id;
Schools.update({_id: school_id}, { $push: { enrolledStudents: Meteor.user()}});
If that doesn't work, you'll have to do a little debugging to see what in particular about it isn't working.

CouchDB, Node.js, Cradle - How to get data based on returned data

I am working on a messaging system using node.js + cradle and couchdb.
When a user pulls a list of their messages, I need to pull the online status of the user that sent them the message. The online status is stored in the user document for each registered user, and the message info is stored in a separate document.
Here is the only way I can manage to do what I need, but its hugely inefficient
privatemessages/all key = username of the message recipient
db.view('privatemessages/all', {"key":username}, function (err, res) {
res.forEach(function (rowA) {
db.view('users/all', {"key":rowA.username}, function (err, res) {
res.forEach(function (row) {
result.push({onlinestatus:row.onlinestatus, messagedata: rowA});
});
});
});
response.end(JSON.stringify(result));
});
Can someone tell me the correct way of doing this?
Thank you

Your code could return empty result because you are calling response at the time when user statuses may not yet be fetched from DB. Other problem is that if I received multiple messages from the same user, then call for his status may be duplicit. Below is a function which first fetch messages from DB avoiding duplicity of users and then get their statuses.
function getMessages(username, callback) {
// this would be "buffer" for senders of the messages
var users = {};
// variable for a number of total users I have - it would be used to determine
// the callback call because this function is doing async jobs
var usersCount = 0;
// helpers vars
var i = 0, user, item;
// get all the messages which recipient is "username"
db.view('privatemessages/all', {"key":username}, function (errA, resA) {
// for each of the message
resA.forEach(function (rowA) {
user = users[rowA.username];
// if user doesn't exists - add him to users list with current message
// else - add current message to existing user
if(!user) {
users[rowA.username] = {
// I guess this is the name of the sender
name: rowA.username,
// here will come his current status later
status: "",
// in this case I may only need content, so there is probably
// no need to insert whole message to array
messages: [rowA]
};
usersCount++;
} else {
user.messages.push(rowA);
}
});
// I should have all the senders with their messages
// and now I need to get their statuses
for(item in users) {
// assuming that user documents have keys based on their names
db.get(item, function(err, doc) {
i++;
// assign user status
users[item].status = doc.onlineStatus;
// when I finally fetched status of the last user, it's time to
// execute callback and rerutn my results
if(i === usersCount) {
callback(users);
}
});
}
});
}
...
getMessages(username, function(result) {
response.end(JSON.stringify(result));
});
Although CouchDB is a great document database you should be careful with frequent updates of existing documents because it creates entirely new document version after each update (this is because of it's MVCC model which is used to achieve high availability and data durability). Consequence of this behavior is higher disk space consumption (more data/updates, more disk space needed - example), so you should watch it and run database consumption accordingly.

I think your system could use an in memory hashmap like memcached. Each user status entry would expire after a time limit.
Mapping would be
[user -> lasttimeseen]
If the hashmap contains the user, then the user is online.
On some certain actions, refresh the lasttimeseen.
Then instead of pinging the whole world each time, just query the map itself and return the result.

I'm reminded of this presentation:
Databases Suck for Messaging
And its quote from Tim O'Reilly:
"On monday friendfeed polled flickr nearly 3 million times for 45000 users, only 6K of whom were logged in. Architectural mismatch."
As pointed out in the other answers, updates in CouchDB are expensive and should be avoided if possible, and there's probably no need for this data to be persistent. A cache or messaging system may solve your problem more elegantly and more efficiently.