When using update with upsert does the driver combine the query and $set objects.
Here is some example code
db.collection('objects').update({_key: 'test1'}, {$set: {a:1}}, {upsert: true, w: 1}, callback);
When I do this, if the document test1 doesn't exist mongo seems to insert the following document.
{
_id: ,
_key: 'test1',
a: 1
}
So it takes the key from the query and uses it as a field along with a:1
Is this any different than :
var data = {
_key : 'test1',
a: 1
};
db.collection('objects').update({_key: 'test1'}, {$set: data}, {upsert: true, w: 1}, callback);
Is the second one going to be slower due to overwriting the _key. There is an index on _key.
Basically no, the values specified in the query do not overwrite where the document matching that query exists.
How an "upsert" basically works is that the conditions set in the query portion first look for a document matching those conditions. Where the document exists, then any arguments supplied in the "update document" portion of the statement are used to write to the document(s) that match. In this case, there are just fields specified with the $set operator, so these are the only fields that are touched as opposed to sending a whole object which would overwrite the existing document.
Should the document have not matched, then first, whatever values are specified in the query condition are written to the new document. This makes sense since you were asking for a document that met those conditions but was not there.
Then (but really all at once) any values in the "update document" portion of the statement are also applied to the new document. You can also specify fields you want to create on "insert" which are not present in the query portion using the $setOnInsert operator, so those values are only used on creation.
In fact a good way to demonstrate this is to do two "upsert" operations as follows:
db.test.update({ "a": "test" },{ "$set": { "b": "data" } },{ "upsert": true});
db.test.update({ "a": "test" },{ "b": "data" },{ "upsert": true});
Without any of the update operators here you are just sending a whole document the second iteration. This means when the "insert" occurs on the first you have a document like this:
{ "a": "test", "b": "data" }
But of course when you perform the second update which matches the document with field "a" equal to "test" then the resulting document is this:
{ "b": "data" }
Which is a clear demonstration that the query portion of the statement is not used to write on an actual update. So this only happens on insertion and only when used with an update operator present in the update document portion of the statement
Related
So I have a collection looking like this:
[
{"url":"website.com/test", "links":[ {"url": "www.something.fr/page.html","scoreDiff": 0.44} ], "justUpdated": true, "score": 0.91},
{"url":"domain.com/", "links":[], "justUpdated": true, "score": 0.81},
{"url":"www.something.fr/page.html", "links":[], "justUpdated": false, "score": 0.42},
]
The goal here is to get the third document, because in one of the documents where "justUpdated" equals true (the first one here), there is its url as a value in one of the "links" array elements.
To achieve that, I tried:
To find all the documents with "justUpdated" equals to true, then in NodeJS concatenate all the urls in their "links" arrays (let's call this array urlsOfInterest). And finally do another query to find all the documents where the url is in urlsOfInterest.
The problem is that it takes some time to do the first query then process the result and do the second query.
So I thought maybe I could do it all at once in an aggregate query. I use $group (with $cond to check if justUpdated equals true) to get all the arrays of "links" in one new variable named urlsOfInterest. For now this is an array of arrays of object so I then use $project with $reduce to have all these {url: "...", score: X} objects as one big array. Finally I use $project and $map to only have the url as the score value doesn't interest me here.
So I get an output looking like this:
_id:"urlsOfInterest",
urlsOfInterest: ["www.something.fr/page.html"]
Which is pretty great but I am stuck because now I just need to get the documents where url is in this variable named urlsOfInterest except I can't because all my documents have "disappeared" after the $group stage.
Please help me to find a way to perform this final query :) Or if this isn't the right way to do this, please point me in the right direction !
PS: the real goal here would be to update for all the documents where justUpdated equals true, every scoreDiff values in their links array. For our exemple, we do abs(0.91 - 0.42) = 0.49 so we replace our scoreDiff value of 0.44 by 0.49 (0.91 being the score of the document where justUpdated equals true and 0.42 the score of the document where url equals www.something.fr/page.html, explaining why I need to fetch this last document.) I don't believe there could be a way of doing all of this at once but if there is, please tell me !
You can use $lookup to get all matching documents in an array:
db.collection.aggregate([
{
"$match": {
"justUpdated": true
}
},
{
"$lookup": {
"from": "collection",
"localField": "links.url",
"foreignField": "url",
"as": "result"
}
},
{
"$match": {
"result": {
$gt: []
}
}
}
])
Then either $unwind and $replaceRoot the results array to get the documents as a cursor and do the math on the application level or do the calculations in the same pipeline, e.g. with $reduce
The "PS: the real goal" is not quite clear as it is based on a particular example but if you play a little bit with it in the playground I am sure you can calculate the numbers as per your requirements.
I have a database like this :
Db.collection.object
{
"item1": [{
"quantity":"",
"price":""
}],
"item2": [{
"quantity":"",
"price":""
}],
}
I want to update "item2.quantity" so I'm using this (it's working):
db.object.updateOne({"item2":{$exists:true}},{$set:{"item2.$.quantity":"55"}})
But sometimes, I need to create "item2" so I want to do "upsert:true" so my "update function" will create the document.
db.object.updateOne({"item2":{$exists:true}},{$set:{"item2.$.quantity":"55"}}, {upsert:true})
But it does nothing because the query function is false ?
[thread1] WriteError: The positional operator did not find the match needed from the query.
To use the $, You must include the array field as part of the query document.
If the query matches the array using a negation operator, such as $ne, $not, or $nin, then you cannot use the positional operator to update values from this array.
Your $exists:true probably falls under negation
Official Document Here
I have a problem when querying mongoDB with nested objects notation:
db.messages.find( { headers : { From: "reservations#marriott.com" } } ).count()
0
db.messages.find( { 'headers.From': "reservations#marriott.com" } ).count()
5
I can't see what I am doing wrong. I am expecting nested object notation to return the same result as the dot notation query. Where am I wrong?
db.messages.find( { headers : { From: "reservations#marriott.com" } } )
This queries for documents where headers equals { From: ... }, i.e. contains no other fields.
db.messages.find( { 'headers.From': "reservations#marriott.com" } )
This only looks at the headers.From field, not affected by other fields contained in, or missing from, headers.
Dot-notation docs
Since there is a lot of confusion about queries MongoDB collection with sub-documents, I thought its worth to explain the above answers with examples:
First I have inserted only two objects in the collection namely: message as:
> db.messages.find().pretty()
{
"_id" : ObjectId("5cce8e417d2e7b3fe9c93c32"),
"headers" : {
"From" : "reservations#marriott.com"
}
}
{
"_id" : ObjectId("5cce8eb97d2e7b3fe9c93c33"),
"headers" : {
"From" : "reservations#marriott.com",
"To" : "kprasad.iitd#gmail.com"
}
}
>
So what is the result of query: db.messages.find({headers: {From: "reservations#marriott.com"} }).count()
It should be one because these queries for documents where headers equal to the object {From: "reservations#marriott.com"}, only i.e. contains no other fields or we should specify the entire sub-document as the value of a field.
So as per the answer from #Edmondo1984
Equality matches within sub-documents select documents if the subdocument matches exactly the specified sub-document, including the field order.
From the above statements, what is the below query result should be?
> db.messages.find({headers: {To: "kprasad.iitd#gmail.com", From: "reservations#marriott.com"} }).count()
0
And what if we will change the order of From and To i.e same as sub-documents of second documents?
> db.messages.find({headers: {From: "reservations#marriott.com", To: "kprasad.iitd#gmail.com"} }).count()
1
so, it matches exactly the specified sub-document, including the field order.
For using dot operator, I think it is very clear for every one. Let's see the result of below query:
> db.messages.find( { 'headers.From': "reservations#marriott.com" } ).count()
2
I hope these explanations with the above example will make someone more clarity on find query with sub-documents.
The two query mechanism work in different ways, as suggested in the docs at the section Subdocuments:
When the field holds an embedded document (i.e, subdocument), you can either specify the entire subdocument as the value of a field, or “reach into” the subdocument using dot notation, to specify values for individual fields in the subdocument:
Equality matches within subdocuments select documents if the subdocument matches exactly the specified subdocument, including the field order.
In the following example, the query matches all documents where the value of the field producer is a subdocument that contains only the field company with the value 'ABC123' and the field address with the value '123 Street', in the exact order:
db.inventory.find( {
producer: {
company: 'ABC123',
address: '123 Street'
}
});
I'm trying to find all documents that do not contain at least one document with a specific field value. For example here is a sample collection:
{ _id : 1,
docs : [
{ foo : 1,
bar : 2},
{ foo : 3,
bar : 3}
]
},
{ _id : 2,
docs : [
{ foo : 2,
bar : 2},
{ foo : 3,
bar : 3}
]
}
I want to find every record where there is not a document in the docs block that does not contain at least one record with foo = 1. In the example above, only the second document should be returned.
I have tried the following, but it only tells me if there are any that don't match (which returns document 1.
db.collection.find({"docs": { $not: {$elemMatch: {foo: 1 } } } })
UPDATE: The query above actually does work. As many times happens, my data was wrong, not my code.
I have also looked at the $nin operator but the examples only show when the array contains a list of primitive values, not an additional document. When I've tried to do this with something like the following, it looks for the EXACT document rather than just the foo field I want.
db.collection.find({"docs": { $nin: {'foo':1 } } })
Is there anyway to accomplish this with the basic operators?
Using $nin will work, but you have the syntax wrong. It should be:
db.collection.find({'docs.foo': {$nin: [1]}})
Use the $ne operator:
db.collection.find({'docs.foo': {$ne: 1}})
Update: I'd advise against using $nin in this case.
{'docs.foo': {$ne: 1}} takes all elements of docs, and for each of them it checks whether the foo field equals 1 or not. If it finds a match, it discards the document from the result list.
{'docs.foo': {$nin: [1]}} takes all elements of docs, and for each element it checks whether its foo field matches any of the members of the array [1]. This is a Cartesian product, you compare an array to another array, each element to each element. Although MongoDB might be smart and optimize this query, I assume you only use $nin because "it has do to something with arrays". But if you understand what you do here, you'll realize $nin is superfluous, and has possibly subpar performance.
I am trying to update a value in the nested array but can't get it to work.
My object is like this
{
"_id": {
"$oid": "1"
},
"array1": [
{
"_id": "12",
"array2": [
{
"_id": "123",
"answeredBy": [], // need to push "success"
},
{
"_id": "124",
"answeredBy": [],
}
],
}
]
}
I need to push a value to "answeredBy" array.
In the below example, I tried pushing "success" string to the "answeredBy" array of the "123 _id" object but it does not work.
callback = function(err,value){
if(err){
res.send(err);
}else{
res.send(value);
}
};
conditions = {
"_id": 1,
"array1._id": 12,
"array2._id": 123
};
updates = {
$push: {
"array2.$.answeredBy": "success"
}
};
options = {
upsert: true
};
Model.update(conditions, updates, options, callback);
I found this link, but its answer only says I should use object like structure instead of array's. This cannot be applied in my situation. I really need my object to be nested in arrays
It would be great if you can help me out here. I've been spending hours to figure this out.
Thank you in advance!
General Scope and Explanation
There are a few things wrong with what you are doing here. Firstly your query conditions. You are referring to several _id values where you should not need to, and at least one of which is not on the top level.
In order to get into a "nested" value and also presuming that _id value is unique and would not appear in any other document, you query form should be like this:
Model.update(
{ "array1.array2._id": "123" },
{ "$push": { "array1.0.array2.$.answeredBy": "success" } },
function(err,numAffected) {
// something with the result in here
}
);
Now that would actually work, but really it is only a fluke that it does as there are very good reasons why it should not work for you.
The important reading is in the official documentation for the positional $ operator under the subject of "Nested Arrays". What this says is:
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
Specifically what that means is the element that will be matched and returned in the positional placeholder is the value of the index from the first matching array. This means in your case the matching index on the "top" level array.
So if you look at the query notation as shown, we have "hardcoded" the first ( or 0 index ) position in the top level array, and it just so happens that the matching element within "array2" is also the zero index entry.
To demonstrate this you can change the matching _id value to "124" and the result will $push an new entry onto the element with _id "123" as they are both in the zero index entry of "array1" and that is the value returned to the placeholder.
So that is the general problem with nesting arrays. You could remove one of the levels and you would still be able to $push to the correct element in your "top" array, but there would still be multiple levels.
Try to avoid nesting arrays as you will run into update problems as is shown.
The general case is to "flatten" the things you "think" are "levels" and actually make theses "attributes" on the final detail items. For example, the "flattened" form of the structure in the question should be something like:
{
"answers": [
{ "by": "success", "type2": "123", "type1": "12" }
]
}
Or even when accepting the inner array is $push only, and never updated:
{
"array": [
{ "type1": "12", "type2": "123", "answeredBy": ["success"] },
{ "type1": "12", "type2": "124", "answeredBy": [] }
]
}
Which both lend themselves to atomic updates within the scope of the positional $ operator
MongoDB 3.6 and Above
From MongoDB 3.6 there are new features available to work with nested arrays. This uses the positional filtered $[<identifier>] syntax in order to match the specific elements and apply different conditions through arrayFilters in the update statement:
Model.update(
{
"_id": 1,
"array1": {
"$elemMatch": {
"_id": "12","array2._id": "123"
}
}
},
{
"$push": { "array1.$[outer].array2.$[inner].answeredBy": "success" }
},
{
"arrayFilters": [{ "outer._id": "12" },{ "inner._id": "123" }]
}
)
The "arrayFilters" as passed to the options for .update() or even
.updateOne(), .updateMany(), .findOneAndUpdate() or .bulkWrite() method specifies the conditions to match on the identifier given in the update statement. Any elements that match the condition given will be updated.
Because the structure is "nested", we actually use "multiple filters" as is specified with an "array" of filter definitions as shown. The marked "identifier" is used in matching against the positional filtered $[<identifier>] syntax actually used in the update block of the statement. In this case inner and outer are the identifiers used for each condition as specified with the nested chain.
This new expansion makes the update of nested array content possible, but it does not really help with the practicality of "querying" such data, so the same caveats apply as explained earlier.
You typically really "mean" to express as "attributes", even if your brain initially thinks "nesting", it's just usually a reaction to how you believe the "previous relational parts" come together. In reality you really need more denormalization.
Also see How to Update Multiple Array Elements in mongodb, since these new update operators actually match and update "multiple array elements" rather than just the first, which has been the previous action of positional updates.
NOTE Somewhat ironically, since this is specified in the "options" argument for .update() and like methods, the syntax is generally compatible with all recent release driver versions.
However this is not true of the mongo shell, since the way the method is implemented there ( "ironically for backward compatibility" ) the arrayFilters argument is not recognized and removed by an internal method that parses the options in order to deliver "backward compatibility" with prior MongoDB server versions and a "legacy" .update() API call syntax.
So if you want to use the command in the mongo shell or other "shell based" products ( notably Robo 3T ) you need a latest version from either the development branch or production release as of 3.6 or greater.
See also positional all $[] which also updates "multiple array elements" but without applying to specified conditions and applies to all elements in the array where that is the desired action.
I know this is a very old question, but I just struggled with this problem myself, and found, what I believe to be, a better answer.
A way to solve this problem is to use Sub-Documents. This is done by nesting schemas within your schemas
MainSchema = new mongoose.Schema({
array1: [Array1Schema]
})
Array1Schema = new mongoose.Schema({
array2: [Array2Schema]
})
Array2Schema = new mongoose.Schema({
answeredBy": [...]
})
This way the object will look like the one you show, but now each array are filled with sub-documents. This makes it possible to dot your way into the sub-document you want. Instead of using a .update you then use a .find or .findOne to get the document you want to update.
Main.findOne((
{
_id: 1
}
)
.exec(
function(err, result){
result.array1.id(12).array2.id(123).answeredBy.push('success')
result.save(function(err){
console.log(result)
});
}
)
Haven't used the .push() function this way myself, so the syntax might not be right, but I have used both .set() and .remove(), and both works perfectly fine.