I am building a web application, and I am spending so long time to take care of updates between related documents.
For example, I have 'Task' document and 'User' document. When task is made, multiple users will be assigned to it. Thus,
taskA.assigned = ["1321231fsdfsdf"(userA's _id), "12312313asdasdasd"(userB's _id)]
userA.tasks = [..., "1231321"(taskA's _id),...]
userB.tasks = [..., "12313211"(taskB's _id),...]
I could handle it well when it comes to just creating tasks. However, it becomes too tricky when I am going to edit tasks. If user B is deleted from taskA, I have to delete userB's id and go to the userB's tasks property and delete taskA's id too.
Is there any shortcut and automatic way to deal with it? Thank you for your time to read it. Let me know if I was too vague, I will add more detail.
In a relational database like MySQL using foreign keys and cascade updates could be done automatically, but in MongoDB that's not possible.
But I see in the tags you are using Moongose, so using a post save hook could do the trick. You can set a hook that updates automatically the user collection each time a task is updated, or viceversa.
Other option would be changing your data estructure, but this depends on your case, there are some facts to take into account. I think we don't have enough information to judge, but there are many resources speaking about data normalization in MongoDB, you can check for example the official MongoDB manual.
I found a way to do this easily and with less lines of codes. Around 100 lines reduced to around 30 lines by doing this.
Long story short, I used 'update()' method and various mongo operators, such as $in, $push, or $pull.
Here is my final codes that are optimized with use of update method.
var edit = req.body;
edit.assignedTo = edit.assignedTo.split(',');
var old = req.task;
var idQueries = edit.assignedTo.map(function (x) {
return mongoose.Types.ObjectId(x);
});
User.update({tasks: old._id}, {$pull: {tasks: old._id}}, {multi: true}, function () {
// Update to remove original task's id from users assigned to it.
User.update({_id: {$in: idQueries}}, {$push: {tasks: old._id}}, {multi: true}, function () {
// Update to add edited tasks'id to new users assigned to it.
old.lastAction = 'edited';
old.edited = true;
old.editedAt = Date.now();
old.titke = edit.title;
old.desc = edit.desc;
old.dueBy = edit.dueBy;
old.assignedTo = edit.assignedTo;
old.save(function (err, task) {
if (err) return next(err);
User.populate(task, 'assignedTo', function (err, task) {
res.json(task);
});
});
});
});
Wish this help some people!
Related
I have a collection
/userFeed
Where I create/delete docs (representing users) when the current user starts following/unfollowing them.
...
/userFeed (C)
/some-followed-user (D)
-date <timestamp>
-interactions <number>
When the user likes a post, the interactions field will be updated. But... what if the user doesn't follow the post owner? Then, I will just need to skip the document update, without necessity of producing failures/errors.
const currentUserFeedRef = firestore
.collection("feeds")
.doc(currentUserId)
.collection("userFeed")
.doc(otherUserId);
const data = {
totalInteractions: admin.firestore.FieldValue.increment(value),
};
const precondition = {
exists: false, // I am trying weird things
};
if (batchOrTransaction) {
return batchOrTransaction.update(
currentUserFeedRef,
data,
precondition
);
}
Is it possible to just "skip the update if the doc doesn't exist"?
Is it possible to just "skip the update if the doc doesn't exist"?
No, not in the way that you're explaining it. Firestore updates don't silently fail.
If you need to know if a document exists before updating it, you should simply read it first and check that it exists. You can do this very easily in a transaction, and you can be sure that the update won't fail due to the document being missing if you check it this way first using the transaction object.
In fact, what you are trying to do is illustrated as the very first example in the documentation.
I am new to JS and I am utilizing the MEAN stack to create a place where students can add classes to their user profile. I already have a session store in my database, so "req.user" will return the currently logged in user information and specifically "req.user.id" will return the currently logged in user's id. Also, I have figured out how to search a course in my database from my application. Ultimately, my goal is that when the user makes the post request to search in the database, I also want those "values" to be pushed into the classes "key". I have provided two options, both of which do not add the respective strings to the database. Thank you for any help!
Portion of Search.JS Option #1
router.post('/', ensureAuthenticated, function (req,res,next) {
var query = {course: req.body.coursename};
db.collection('courses').find(query).toArray()
.then(db.collection('DefaultUser').update({_id: req.user.id}, {$push: {classes: req.body.coursename}}));
res.render('search', {title: 'search'})
});
Portion of Search.JS Option #2
router.post('/', ensureAuthenticated, function(req,res,next) {
var query = {course: req.body.coursename};
db.collection('courses').find(query).toArray((err, results) => {
if (err) {
throw err;
} else {
db.collection('DefaultUser').updateOne({_id: '5c17161d3347e79410ff29ba'}, {
$push: {
classes: req.body.coursename
}
})
console.log(results)
res.render('search', {courses: results, title: 'search'})
}
})
});
Some tips may help:
req.body will hold nothing if you forget to use app.use(express.urlencoded()) to parse the posted data.
You may use ObjectId('<string>') (in option #2) for finding and updating queries, not just the string, because mongodb stores _id as ObjectId type by default.
You may use $addToSet instead of $push modifier to add a unique element to an array, or you may get one student subscribed with two more same class, if he repeatedly submit the form.
In your code, you find in the courses collection first and then update, but since you did nothing with the find result, it is not necessary (empty result does not throw any error). Checking the data is valid is good practice, if you would like to do so, in both options, you need to use findOne instead of find to make sure the course is existed, and .then(course => {/* check if the course is null and then throws an error */} ).
I don't have the full code of your project so I can only guess there may be the problems listed above, wish you good luck!
I have an app that uses firebase, the whole stack pretty much, functions, database, storage, auth, messaging, the whole 9. I want to keep the client end very lightweight. So if a user comments on a post and "tags" another user, let's say using the typical "#username" style tagging, I moved all of the heavy lifting to the firebase functions. That way the client doesn't have to figure out the user ID based on the username, and do everything else. It is setup using triggers, so when the above scenario happens I write to a "table" called "create_notifications" with some data like
{
type: "comment",
post_id: postID,
from: user.getUid(),
comment_id: newCommentKey,
to: taggedUser
}
Where the taggedUser is the username, the postID is the active post, the newCommentKey is retrieved from .push() on the comments db reference, and the user.getUid() is from the firebase auth class.
Now in my firebase functions I have a "onWrite" trigger for that specific table that gets all of the relevant information and sends out a notification to the poster of the post with all the relevant details. All of that is complete, what I am trying to figure out is... how do I delete the incoming event, that way I don't need any sort of cron jobs to clear out this table. I can just grab the event, do my needed calculations and data gathering, send the message, then delete the incoming event so it never even really exists in the database except for the small amount of time it took to gather the data.
A simplified sample of the firebase functions trigger is...
exports.createNotification = functions.database.ref("/create_notifications/{notification_id}").onWrite(event => {
const from = event.data.val().from;
const toName = event.data.val().to;
const notificationType = event.data.val().type;
const post_id = event.data.val().post_id;
var comment_id, commentReference;
if(notificationType == "comment") {
comment_id = event.data.val().comment_id;
}
const toUser = admin.database().ref(`users`).orderByChild("username").equalTo(toName).once('value');
const fromUser = admin.database().ref(`/users/${from}`).once('value');
const referencePost = admin.database().ref(`posts/${post_id}`).once('value');
return Promise.all([toUser, fromUser, referencePost]).then(results => {
const toUserRef = results[0];
const fromUserRef = results[1];
const postRef = results[2];
var newNotification = {
type: notificationType,
post_id: post_id,
from: from,
sent: false,
create_on: Date.now()
}
if(notificationType == "comment") {
newNotification.comment_id = comment_id;
}
return admin.database().ref(`/user_notifications/${toUserRef.key}`).push().set(newNotification).then(() => {
//NEED TO DELETE THE INCOMING "event" HERE TO KEEP DB CLEAN
});
})
}
So in that function in the final "return" of it, after it writes the finalized data to the "/user_notifications" table, I need to delete the event that started the whole thing. Does anyone know how to do that? Thank you.
First off, use .onCreate instead of .onWrite. You only need to read each child when they are first written, so this will avoid undesirable side effects. See the documentation here for more information on the available triggers.
event.data.ref() holds the reference where the event occurred. You can call remove() on the reference to delete it:
return event.data.ref().remove()
The simplest way to achieve this is through calling the remove() function offered by the admin sdk,
you could get the reference to the notification_id through the event, i.e event.params.notification_id then remove it when need be with admin.database().ref('pass in the path').remove(); and you are good to go.
For newer versions of Firebase, use:
return change.after.ref.remove()
I have to connect to the external database and get access to its collections. It works fine, when I use it, but the problem is when I need collection hooks, e.g. Collection.after.insert(function(userId, doc)). The hook is not being fired. I have following code:
// TestCollection.js
let database = new MongoInternals.RemoteCollectionDriver("mongodb://127.0.0.1:3001/meteor",
{
oplogUrl: 'mongodb://127.0.0.1:3001/local'
});
let TestCollection = new Mongo.Collection("testCollection", { _driver: database });
module.exports.TestCollection = TestCollection;
console.log(TestCollection.findOne({name: 'testItem'})); // writes out the item correctly
// FileUsingCollection.js
import { TestCollection } from '../collections/TestCollection.js';
console.log(TestCollection.findOne({name: 'testItem'})); // writes out the item correctly second time
TestCollection.after.update(function (userId, doc) {
console.log('after update');
}); // this is NOT being fired when I change the content of remote collection (in external app, which database I am connected)
How to make this work?
EDIT:
I have read many hours about it and I think it might be connected with things like:
- oplog
- replicaSet
But I am newbie to Meteor and can’t find out what are those things about. I have set MONGO_OPLOG_URL and I added oplog parameter to database driver as I read here: https://medium.com/#lionkeng/2-ways-to-share-data-between-2-different-meteor-apps-7b27f18b5de9
but nothing changed. And I don’t know how to use this replicaSet, how to add it to the url. Anybody can help?
You can also try something like below code,
var observer = YourCollections.find({}).observeChanges({
added: function (id, fields) {
}
});
You can also have 'addedBefore(id, fields, before)', 'changed(id, fields)', 'movedBefore(id, before)', 'removed(id)'
For more features goto link.
I'm trying to display a forum/category. I need to get the latest posts. The problem is that I also need data on the user for each post as well as the latest reply.
db.post.find({
"inForum": forumID,
},
{
'sort': [['date', -1]]
},
function(err, cursor) {
cursor.count(function(err, count) {
cursor.skip(skip).limit(20).toArray(function(err, posts) {
var complete = _.after(nodes.length, function () {
res.send(posts)
});
// for every post get its author info and the latest post info
posts.forEach(function (post) {
var users = _.pluck(posts, 'user');
user.load(users, function (profiles) {
_.each(posts,
function(post, k) {
if (profiles[post.user]) post.fieldAvatar = profiles[post.user].fieldAvatar;
});
if (post.latestReply) {
post.load(post.latestReply.id, function (latestReply) {
if (latestReply) post.latestReply = latestReply
complete()
})
}
else {
complete()
}
})
})
});
})
})
This is what I'm doing and it seems really slow / really inelegant to me. Am I doing this correctly and is there any advice for speeding this up?
Thanks.
The best thing you should do here is to embed some information for the author of the posts (username & email or avatar) into those posts so that you don't make multiple queries to the database, one should suffice (sure you have some duplicate data, but the performance is optimal).
If you don't want to / can't do that you can also modify your second query to find all authors in [array_of_ids_of_the_posts]. That would reduce your [number_of_posts] queries into only one.
You could use some caching. For example you could save the users in an dictionary during the loop so you only have to fetch it on the first occurrence from mongodb.
Maybe you could create some kind of thread model where you save basic information about the containing posts, so you only have to go through the threads.
You could save the result of the function and delete it when a new post is added .. so won't go through all posts on every call.
You should not use a document storage like a sql database. Maybe it is better to generate the forum page directly when a post is created/edited and save the whole data in a document, so you only have to make one read call to mongo to show it.