I have two models defined like this:
var OrganizationSchema = new mongoose.Schema({
users: [mongoose.Schema.Types.ObjectId]
});
and
var UserSchema = new mongoose.Schema({
organizations: [mongoose.Schema.types.ObjectId]
});
When a user want to join an organization I need to update both the organization collection and the user collection. I want to know what is the best way to achieve this ? Is it worth considering the case when one update request fail ? I'm currently doing something like this (where organization is a collection instance of the model Organization):
User.findByIdAndUpdate(req.userSession.userId, { $push: { organizations: organization.id } }, function (err){
if (err)
{
// server error
console.log(err);
}
else
{
organization.users.push(req.userSession.userId);
organization.save(function (err){
if (err)
{
// server error we need to cancel
User.findByIdAndUpdate(req.userSession.userId, { $pull: { organizations: organization.id } }, function (err){
if (err)
{
// we got a problem one collection updated and not the other one !!
console.log(err);
}
});
}
else
{
// success
}
});
}
});
The problem is: if my second update method fail I will end up with one collection updated and not the other ? Is there a way to make sure they are both updated ?
Well firstly, I would stay clear of that design. I would either reference or embed user in organisations and the other way around, not both of them at same time, so I wouldn't have problems like this(which happens every-time you duplicate data).
MongoDB doesn't have support for simultaneous updates, or transactions. So you are left to manage this in your code.
So yes, if the second update fails, then as you wrote your code you have to rollback, and if the rollback fails, you have to retry till it succeeds(though with exponential backoff probably). Keep in mind that might intefer with other requests(another user tries to save the same thing simultaneously). To handle that you have to give a unique to each entry in the array.
Related
I have a problem I can't find an answer to in Loopback's docs.
Say I have a model Company and a modelEmployee. There is an 1Xn relation between the Company and its Employees. When /api/Employees is called, server returns all the employees.
I only want to return the list of employees who are in the same company with the user requesting the list.
For this, I created a remote hook
Employee.beforeRemote('find', function(context, modelInstance, next) {
var reject = function() {
process.nextTick(function() {
next(null, false);
});
};
// do not allow anonymous users
var userId = context.req.accessToken.userId;
if (!userId) {
return reject();
}
//I get the details of the user who sent the request
//to learn which company does he belong to
Employee.findById(userId, function(err, user) {
if(!context.req.query.filter) context.req.query.filter={};
context.req.query.filter.where = {brandId:user.companyId};
console.log(context.req.query);
next();
});
});
I thought this should work every time, but appearantly it only works when find already has some query filters like include - although the console.log prints a correct context.req.query object.
What am I missing? Any help would be greatly appreciated!
context.args.filter seems to work for this purpose.
As a side note, instead of replacing where, you might want to merge it with something provided by client. For implementation idea you can refer to: https://github.com/strongloop/loopback-datasource-juggler/blob/master/lib/utils.js#L56-L122
I have Parse table with entities.
User - default class
Commets - class with pointer to _User entity.
I need to delete user from entity User together with all its comments, located in Comments entity:
Right now I have JS Cloud code:
Parse.Cloud.define("deleteUser", function(request, response) {
var User = Parse.Object.extend("User");
var query = new Parse.Query(User);
var userID = request.params.userID;
query.get(userID,{
success: function(User) {
var message = 'success';
User.destroy({
useMasterKey: true ,
success:function() {
response.success(message);
return;
},
error:function(error) {
response.error('Could not delete object '+ User.id);
return;
}
});
},
error: function(object, error) {
var message = 'User could not found';
response.error(message);
}
});
});
It's removing user only.
How I can combine to remove also records from other entity by user?
Thanks for advance
As eth3lbert pointed out in the comments, you should use an afterDelete hook that gets called after the User object has been deleted. You can kick off any other delete operations you want.
However, there is a little gotcha, the before* and after* methods get killed after 3 seconds of wall clock time which might lead to unwanted results depending on the amount of data that you need to delete.
The best solution for you is to setup a background job (they can run for up to 15 minutes), schedule it to run, lets say once every day, and do any cleanup work in that job.
You could create a simple table for that, that contains the objectIds of deleted users, whenever your afterDelete method gets called, you add the deleted users id into that table, your background job then queries that table on run and deletes the content that was associated with it.
You can delete user easily from user request as below.
Parse.Cloud.define('deleteUser', async (req) => {
const user = req.user;
return user.destroy({ useMasterKey: true});
})
and add additional deleting logic related to User.
Here's the relevant code:
var Results = mongoose.model('Results', resultsSchema);
var results_array = [];
_.each(matches, function(match) {
var results = new Results({
id: match.match_id,
... // more attributes
});
results_array.push(results);
});
callback(results_array);
});
}
], function(results_array) {
results_array.insert(function(err) {
// error handling
Naturally, I get a No method found for the results_array. However I'm not sure what else to call the method on.
In other functions I'm passing through the equivalent of the results variable here, which is a mongoose object and has the insert method available.
How can I insert an array of documents here?
** Edit **
function(results_array) {
async.eachLimit(results_array, 20, function(result, callback) {
result.save(function(err) {
callback(err);
});
}, function(err) {
if (err) {
if (err.code == 11000) {
return res.status(409);
}
return next(err);
}
res.status(200).end();
});
});
So what's happening:
When I clear the collection, this works fine.
However when I resend this request I never get a response.
This is happening because I have my schema to not allow duplicates that are coming in from the JSON response. So when I resend the request, it gets the same data as the first request, and thus responds with an error. This is what I believe status code 409 deals with.
Is there a typo somewhere in my implementation?
Edit 2
Error code coming out:
{ [MongoError: insertDocument :: caused by :: 11000 E11000 duplicate key error index:
test.results.$_id_ dup key: { : 1931559 }]
name: 'MongoError',
code: 11000,
err: 'insertDocument :: caused by :: 11000 E11000 duplicate key error index:
test.results.$_id_ dup key: { : 1931559 }' }
So this is as expected.
Mongo is responding with a 11000 error, complaining that this is a duplicate key.
Edit 3
if (err.code == 11000) {
return res.status(409).end();
}
This seems to have fixed the problem. Is this a band-aid fix though?
You seem to be trying to insert various documents at once here. So you actually have a few options.
Firstly, there is no .insert() method in mongoose as this is replaced with other wrappers such as .save() and .create(). The most basic process here is to just call "save" on each document you have just created. Also employing the async library here to implement some flow control so everything just doesn't queue up:
async.eachLimit(results_array,20,function(result,callback) {
result.save(function(err) {
callback(err)
});
},function(err) {
// process when complete or on error
});
Another thing here is that .create() can just take a list of objects as it's arguments and simply inserts each one as the document is created:
Results.create(results_array,function(err) {
});
That would actually be with "raw" objects though as they are essentially all cast as a mongooose document first. You can ask for the documents back as additional arguments in the callback signature, but constructing that is likely overkill.
Either way those shake, the "async" form will process those in parallel and the "create" form will be in sequence, but they are both effectively issuing one "insert" to the database for each document that is created.
For true Bulk functionality you presently need to address the underlying driver methods, and the best place is with the Bulk Operations API:
mongoose.connection.on("open",function(err,conn) {
var bulk = Results.collection.initializeUnorderedBulkOp();
var count = 0;
async.eachSeries(results_array,function(result,callback) {
bulk.insert(result);
count++;
if ( count % 1000 == 0 ) {
bulk.execute(function(err,response) {
// maybe check response
bulk = Results.collection.initializeUnorderedBulkOp();
callback(err);
});
} else {
callback();
}
},function(err) {
// called when done
// Check if there are still writes queued
if ( count % 1000 != 0 )
bulk.execute(function(err,response) {
// maybe check response
});
});
});
Again the array here is raw objects rather than those cast as a mongoose document. There is no validation or other mongoose schema logic implemented here as this is just a basic driver method and does not know about such things.
While the array is processed in series, the above shows that a write operation will only actually be sent to the server once every 1000 entries processed or when the end is reached. So this truly does send everything to the server at once.
Unordered operations means that the err would normally not be set but rather the "response" document would contain any errors that might have occurred. If you want this to fail on the first error then it would be .initializeOrderedBulkOp() instead.
The care to take here is that you must be sure a connection is open before accessing these methods in this way. Mongoose looks after the connection with it's own methods so where a method such as .save() is reached in your code before the actual connection is made to the database it is "queued" in a sense awaiting this event.
So either make sure that some other "mongoose" operation has completed first or otherwise ensure that your application logic works within such a case where the connection is sure to be made. Simulated in this example by placing within the "connection open" event.
It depends on what you really want to do. Each case has it's uses, with of course the last being the fastest possible way to do this as there are limited "write" and "return result" conversations going back and forth with the server.
I am very new to mongodb and have a basic question that I am having trouble with. How do I get the ID field of a document that has already been created? I need the ID so i can update/add a new field to the document.
//newProfile is an object, one string it holds is called school
if(Schools.find({name: newProfile.school}).fetch().length != 1){
var school = {
name: newProfile.school
}
Meteor.call('newSchool', school);
//Method 1 (doesn't work)
var schoolDoc = Schools.findOne({name: newProfile.school});
Schools.update({_id: schoolDoc._id}, {$set: {enrolledStudents: Meteor.user()}});
//Method 2?
//Schools.update(_id: <what goes here?>, {$push: {enrolledStudents: Meteor.user()}});
}
else {
//Schools.update... <add users to an existing school>
}
I create a new school document if the listed school does not already exist. Schools need to hold an array/list of students (this is where i am having trouble). How do I add students to a NEW field (called enrolledStudents)?
Thanks!
I'm having some trouble understanding exactly what you're trying to do. Here's my analysis and understanding so far with a couple pointers thrown in:
if(Schools.find({name: newProfile.school}).fetch().length != 1){
this would be more efficient
if(Schools.find({name: new Profile.school}).count() != 1) {
Meteor.call('newSchool', school);
Not sure what you're doing here, unless you this will run asynchronously, meaning by the time the rest of this block of code has executed, chances are this Meteor.call() function has not completed on the server side.
//Method 1 (doesn't work)
var schoolDoc = Schools.findOne({name: newProfile.school});
Schools.update({_id: schoolDoc._id}, {$set: {enrolledStudents: Meteor.user()}});
Judging by the if statement at the top of your code, there is more than one school with this name in the database. So I'm unsure if the schoolDoc variable is the record you're after.
I believe you are having trouble because of the asynchronous nature of Meteor.call on the client.
Try doing something like this:
// include on both server and client
Meteor.methods({
newSchool: function (school) {
var newSchoolId,
currentUser = Meteor.user();
if (!currentUser) throw new Meteor.Error(403, 'Access denied');
// add some check here using the Meteor check/match function to ensure 'school'
// contains proper data
try {
school.enrolledStudents = [currentUser._id];
newSchoolId = Schools.insert(school);
return newSchoolId;
} catch (ex) {
// handle appropriately
}
}
});
// on client
var schoolExists = false;
if (Schools.findOne({name: newProfile.school})) {
schoolExists = true;
}
if (schoolExists) {
var school = {
name: newProfile.school
};
Meteor.call('newSchool', school, function (err, result) {
if (err) {
alert('An error occurred...');
} else {
// result is now the _id of the newly inserted record
}
})
} else {
}
Including the method on both the client and the server allows Meteor to do latency compensation and 'simulate' the insert immediately on the client without waiting for the server round-trip. But you could also just keep the method on the server-side.
You should do the enrolledStudents part on the server to prevent malicious users from messing with your data. Also, you probably don't want to actually be storing the entire user object in the enrolledStudents array, just the user _id.
For what you're trying to do, there is no need to get the _id. When you use update, just switch out the {_id: schoolDoc._id} with your query. Looks like using {name: newProfile.school} will work, assuming that the rest of your code does what you want it to do.
While that would work with the normal Mongo driver, I see that Meteor does not allow your update query to be anything but _id: Meteor throws throwIfSelectorIsNotId exception
First, make sure that you're pulling the right document, and you can try something like this:
var school_id = Schools.findOne({name: newProfile.school})._id;
Schools.update({_id: school_id}, { $push: { enrolledStudents: Meteor.user()}});
If that doesn't work, you'll have to do a little debugging to see what in particular about it isn't working.
I have this code
App.Model('users').find(function (err, users) {
users.forEach(function(user) {
console.log(user.username);
});
});
//make usernames availible here.
This console logs the usernames.
Instead of just logging to the console I want make use of the data.
But How can do this?
Thanks
They will never be available where you want them. This isn't how async/node.js programming works.
Possible:
App.Model('users').find(function (err, users) {
users.forEach(function(user) {
myCallBack(user);
});
});
var myCallBack = function(user) {
//make usernames availible here.
//this will be called with every user object
console.log(user);
}
Other possibilites: EventEmitter, flow controll libraries (e.g. async).
If you have written code in other languages before you need to be open to take a complete new approach how data will be handled.
With node js you don't do:
//make usernames availible here.
You do:
App.Model('users').find(function (err, users) {
//usernames are available here. Pass them somewhere. Notify some subscribers that you have data.
});
//The code here has executed a long time ago