Develop Reference

MongoDB complex faceted search not yielding expected result

I am trying to build in one aggregation query a match text search followed by another match query to avoid sending multiple queries to the database, but I am not receiving any result.
Here is what my query looks like:
query = await Product.aggregate([
{ $match: { $text: { $search: userQuery } } },
{ $addFields: { score: { $meta: "textScore" } } },
{
$facet: {
Forfait: [
{
$match: { Forfait: "midnight" }
}
]
}
}
]).exec();
Here is what I am trying to accomplish:
1. Perform a text search on user input on the collection
2. and also perform a match on the collection on the forfait field
The query returns nothing, it seems that it is performing a sort of logical AND query and short-circuits but I don't know how I can perform the two queries in one aggregation.
I have tried following a similar issue here MongoDB Facet Error pipeline requires text score metadata, but there is no text score available which reflects my query.
Any directions or help will be greatly appreciated.
Below is what the document model looks just a simple doc
{
id: String
title: String
description: String
forfait: String
}

Mongoose/MongoDB: $in and .sort()

I hit an API which follows 50 members' data in a game once a day, and use mongoose to convert the JSON into individual documents in a collection. Between days there is data which is consistent, for example each member's tag (an id for the member in game), but there is data which is different (different scores etc.). Each document has a createdAt property.
I would like to find the most recent document for each member, and thus have an array with each member's tag.
I an currently using the following query to find all documents where tags match, however they are returning all documents, not just one. How do I sort/limit the documents to the most recent one, whilst keep it as one query (or is there a more "mongodb way")?
memberTags = [1,2,3,4,5];
ClanMember.find({
'tag': {
$in: memberTags
}
}).lean().exec(function(err, members) {
res.json(members);
});
Thanks

You can query via the aggregation framework. Your query would involve a pipeline that has stages that process the input documents to give you the desired result. In your case, the pipeline would have a $match phase which acts as a query for the initial filter. $match uses standard MongoDB queries thus you can still query using $in.
The next step would be to sort those filtered documents by the createdAt field. This is done using the $sort operator.
The preceding pipeline stage involves aggregating the ordered documents to return the top document for each group. The $group operator together with the $first accumulator are the operators which make this possible.
Putting this altogether you can run the following aggregate operation to get your desired result:
memberTags = [1,2,3,4,5];
ClanMember.aggregate([
{ "$match": { "tag": { "$in": memberTags } } },
{ "$sort": { "tag": 1, "createdAt: -1 " } },
{
"$group": {
"_id": "$tag",
"createdAt": { "$first": "$createdAt" } /*,
include other necessary fields as appropriate
using the $first operator e.g.
"otherField1": { "$first": "$otherField1" },
"otherField2": { "$first": "$otherField2" },
...
*/
}
}
]).exec(function(err, members) {
res.json(members);
});
Or tweak your current query using find() so that you can sort on two fields, i.e. the tag (ascending) and createdAt (descending) attributes. You can then select the top 5 documents using limit, something like the following:
memberTags = [1,2,3,4,5];
ClanMember.find(
{ 'tag': { $in: memberTags } }, // query
{}, // projection
{ // options
sort: { 'createdAt': -1, 'tag': 1 },
limit: memberTags.length,
skip: 0
}
).lean().exec(function(err, members) {
res.json(members);
});
or
memberTags = [1,2,3,4,5];
ClanMember.find({
'tag': {
$in: memberTags
}
}).sort('-createdAt tag')
.limit(memberTags.length)
.lean()
.exec(function(err, members) {
res.json(members);
});

Ok, so, first, let's use findOne() so you get only one document out of the request
Then to sort by the newest document, you can use .sort({elementYouWantToSort: -1}) (-1 meaning you want to sort from newest to oldest, and 1 from the oldest to the newest)
I would recommend to use this function on the _id, which already includes creation date of the document
Which gives us the following request :
ClanMember.findOne({
'tag': {
$in: memberTags
}
}).sort({_id: -1}).lean().exec(function(err, members) {
res.json(members);
});

MongoDB updated object with item remove not saving

I'm using Angular Fullstack for an web app.
I'm posting my data by $http.post() my object:
{ title: "Some title", tags: ["tag1", "tag2", "tag3"] }
When I edit my object and try to $http.put() for example:
{ title: "Some title", tags: ["tag1"] }
In console I get HTTP PUT 200 but when I refresh the page I still recive the object with all 3 tags.
This is how I save in the MongoDB:
exports.update = function(req, res) {
if (req.body._id) {
delete req.body._id;
}
Question.findByIdAsync(req.params.id)
.then(handleEntityNotFound(res))
.then(saveUpdates(req.body))
.then(responseWithResult(res))
.catch(handleError(res));
};
function saveUpdates(updates) {
return function(entity) {
var data = _.merge(entity.toJSON(), updates);
var updated = _.extend(entity, data);
return updated.saveAsync()
.spread(function(updated) {
return updated;
});
};
}
Can someone explain how to save the object with removed items?
What I'm doing wrong?

This is pretty bad practice to use things like _.merge or _.extend in client ( meaning your nodejs client to database and not browser ) code after retrieving from the database. Also notably _.merge is the problem here as it is not going to "take away" things, but rather "augment" what is already there with the information you have provided. Not what you want here, but there is also a better way.
You should simply using "atomic operators" like $set to do this instead:
Question.findByIdAndUpdateAsync(
req.params.id,
{ "$set": { "tags": req.body.tags } },
{ "new": true }
)
.then(function(result) {
// deal with returned result
});
You also really should be targeting your endpoints and not having a "generic" object write. So the obove would be specically targeted at "PUT" for related "tags" only and not touch other fields in the object.
If you really must throw a whole object at it and expect an update from all the content, then use a helper to fix the update statement correctly:
function dotNotate(obj,target,prefix) {
target = target || {},
prefix = prefix || "";
Object.keys(obj).forEach(function(key) {
if ( typeof(obj[key]) === "object" ) {
dotNotate(obj[key],target,prefix + key + ".");
} else {
return target[prefix + key] = obj[key];
}
});
return target;
}
var update = { "$set": dotNotate(req.body) };
Question.findByIdAndUpdateAsync(
req.params.id,
update,
{ "new": true }
)
.then(function(result) {
// deal with returned result
});
Which will correctly structure not matter what the object you throw at it.
Though in this case then probably just directly is good enough:
Question.findByIdAndUpdateAsync(
req.params.id,
{ "$set": req.body },
{ "new": true }
)
.then(function(result) {
// deal with returned result
});
There are other approaches with atomic operators that you could also fit into your logic for handling. But it is best considered that you do these per element, being at least root document properties and things like arrays treated separately as a child.
All the atomic operations interact with the document "in the database" and "as is at modification". Pulling data from the database, modifiying it, then saving back offers no such guarnatees that the data has not already been changed and that you just may be overwriting other changes already comitted.
I truth your "browser client" should have been aware that the "tags" array had the other two entries and then your "modify request" should simply be to $pull the entries to be removed from the array, like so:
Question.findByIdAndUpdateAsync(
req.params.id,
{ "$pull": { "tags": { "$in": ["tag2", "tag3"] } } },
{ "new": true }
)
.then(function(result) {
// deal with returned result
});
And then, "regardless" of the current state of the document on the server when modified, those changes would be the only ones made. So if something else modified at added "tag4", and the client had yet to get the noficiation of such a change before the modification was sent, then the return response would include that as well and everything would be in sync.
Learn the update modifiers of MongoDB, as they will serve you well.

Mongo query doesn't include my filter clause

I am building a mongo query in node as such:
query = {"status":{$ne:"Complete"}}, {"processId":1, "_id":0};
then I execute the query:
return this.engine.dbcollectionName.find(query).toArray(function(err, result){...
If i evaluate "conditions", the result is 'status=$ne:"Complete"'...however, my projection is missing.
Am I missing something about how to represent json in javascript variable? It is acting like I didn't include anything after the comma.
Another way to say it, in mongo CLI I want:
db.collection.find( {"status":{$ne:"Complete"}}, {"processId":1, "_id":0})
whereas within node, i'm getting equivalent of (notice no projection):
db.collection.find( {"status":{$ne:"Complete"}})
Any help appreciated! Thanks

Your query and projection must be separate object parameters to find:
var query = {"status":{$ne:"Complete"}};
var projection = {"processId":1, "_id":0};
this.engine.dbcollectionName.find(query, projection).toArray(function(err, result){...

New to JavaScript I see. If you want to construct an array of arguments then you need an actual array, and with the way method signatures work here you will also need the JavaScript .apply() method:
var async = require('async'),
mongodb = require('mongodb'),
MongoClient = mongodb.MongoClient;
MongoClient.connect('mongodb://localhost/test',function(err,db) {
var query = [{ "status": { "$ne": "Complete" } }, { "processId": 1, "_id": 0 } ];
db.collection('collectionname',function(err,coll) {
coll.find.apply(coll,query).toArray(function(err,docs) {
if (err) throw err;
console.log( JSON.stringify( docs, undefined, 2 ) );
});
});
});
Otherwise you are separating arguments like
coll.find.(
{ "status": { "$ne": "Complete" } },
{ "processId": 1, "_id": 0 }
).toArray(function(err,docs) {
So that each item is passed to the function individually.
.apply() lets you pass in list variables or arrays of arguments to your function. It often requires a context as the first argument, which in this case is the "collection" object that the .find() method belongs to.

MongoDB - $set to update or push Array element

In products collection, i have an Array of recentviews which has 2 fields viewedBy & viewedDate.
In a scenario if i already have a record with viewedby, then i need to update it. For e.g if i have array like this :-
"recentviews" : [
{
"viewedby" : "abc",
"vieweddate" : ISODate("2014-05-08T04:12:47.907Z")
}
]
And user is abc, so i need to update the above & if there is no record for abc i have to $push.
I have tried $set as follows :-
db.products.update( { _id: ObjectId("536c55bf9c8fb24c21000095") },
{ $set:
{ "recentviews":
{
viewedby: 'abc',
vieweddate: ISODate("2014-05-09T04:12:47.907Z")
}
}
}
)
The above query erases all my other elements in Array.

Actually doing what it seems like you say you are doing is not a singular operation, but I'll walk through the parts required in order to do this or otherwise cover other possible situations.
What you are looking for is in part the positional $ operator. You need part of your query to also "find" the element of the array you want.
db.products.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
"$set": {
"recentviews.$.vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
)
So the $ stands for the matched position in the array so the update portion knows which item in the array to update. You can access individual fields of the document in the array or just specify the whole document to update at that position.
db.products.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
"$set": {
"recentviews.$": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
)
If the fields do not in fact change and you just want to insert a new array element if the exact same one does not exist, then you can use $addToSet
db.products.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
$addToSet:{
"recentviews": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
)
However if you are just looking for for "pushing" to an array by a singular key value if that does not exist then you need to do some more manual handling, by first seeing if the element in the array exists and then making the $push statement where it does not.
You get some help from the mongoose methods in doing this by tracking the number of documents affected by the update:
Product.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
"$set": {
"recentviews.$": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
},
function(err,numAffected) {
if (numAffected == 0) {
// Document not updated so you can push onto the array
Product.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095")
},
{
"$push": {
"recentviews": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
},
function(err,numAffected) {
}
);
}
}
);
The only word of caution here is that there is a bit of an implementation change in the writeConcern messages from MongoDB 2.6 to earlier versions. Being unsure right now as to how the mongoose API actually implements the return of the numAffected argument in the callback the difference could mean something.
In prior versions, even if the data you sent in the initial update exactly matched an existing element and there was no real change required then the "modified" amount would be returned as 1 even though nothing was actually updated.
From MongoDB 2.6 the write concern response contains two parts. One part shows the modified document and the other shows the match. So while the match would be returned by the query portion matching an existing element, the actual modified document count would return as 0 if in fact there was no change required.
So depending on how the return number is actually implemented in mongoose, it might actually be safer to use the $addToSet operator on that inner update to make sure that if the reason for the zero affected documents was not just that the exact element already existed.