I've created a Collection containing 1 million documents, and I'm trying to select 50000 of these records based on the ObjectID, and update one of the values (i'm working in Mongo shell, running on Ubuntu).
Is it possible to define a 50000 document range? It doesn't matter which documents are included in the 50000 range, I simply want to ringfence a definite number of records and run an update operation using the primary id so that I can measure the performance time.
The code I've tried to run doesn't work:
use Assignment
var _start = new Date()
db.FlightsDate.update({$set:{Airtime: 8888}}).limit(50000).hint({_id:1});
var _end = new Date();
print("Time to Bulk Update AirTime key for 50000 documents… " + ((_end _start)/1000));
...i'm gathering that MongoDB needs me to include a query in the command to specify which docs are to be updated (I now understand from reading other posts that .limit won't constrain the number of records than an .update writes to).
Please can anyone advise a method that'll enable me to define the number of records to be updated?
Grateful for advice.
R,
Jon
If you are simply looking for a "range" that covers 50,000 of the documents in the collection then your best approach is to query and find the "starting" and "ending" documents of your range first. Then apply that "range" specification to your update.
var start_id = db.FlightsDate.find({}).limit(1).toArray()[0]._id;
var end_id = db.FlightsDate.find({}).skip(49999).limit(1).toArray()[0]._id;
var _start = new Date();
db.FlightsDate.update(
{ "_id": { "$gte": start_id, "$lte": end_id } },
{ "$set"; { "Airtime": 8888 } },
{ "multi": true }
);
var _end = new Date();
( _end - _start )/1000;
If you then wanted the next 50,000 in an additional range then :
var start_id = db.FlightsDate.find(
{ "_id": { "$gt": end_id } }
).limit(1).toArray()[0]._id;
var end_id = db.FlightsDate.find(
{ "_id": { "$gt": end_id } }
).skip(49999).limit(1).toArray()[0]._id;
And do it all again.
The point is you need to know where to "start" and when to "end" within a range to limit your update to just 50,000 documents without any other criteria to do so.
Also note the usage of "multi" in the update method there. By default, .update() does not "update" any more than one document, essentially being the first match. So what you mean to do is update "all documents in the range" and that is why you need to apply "multi" here.
Related
I am trying to perform 2 operations in one findOneAndUpdate():
Update date in one field lastUpdatedTimestamp, set it to current date (this one works fine in my statement),
Update date in other field expiryTimestamp, by adding 1 day to $currentDate (I couldn't find a way to achieve it so I'm trying to $add 1 day to the the value read from the above field lastUpdatedTimestamp) - (I can't make this one work).
findOneAndUpdate(
{"_id":123},
{ $currentDate: {"lastUpdatedTimestamp":true}, $set: {"expiryTimestamp": {$add: ["$lastUpdatedTimestamp", 24*60*60000]}}}
)
Here's the error I'm receiving:
{ "ok" : 0.0, "errmsg" : "The dollar ($) prefixed field '$add' in 'expiryTimestamp.$add' is not valid for storage.", "code" : 52 }
Is it even possible? I'd appreciate your help.
You can use the setDate() method to set the "expiryTimestamp" value.
db.collection.updateOne(
{ "_id": 123 },
{ "$set": {
"lastUpdatedTimestamp": new Date(),
"expiryTimestamp": new Date().setDate(new Date().getDate() + 1)
}}
)
You don't need to use findOneAndUpdate unless you want to return the new or old document.
The marked answer is wrong in the sense that using new Date() will not use database timestamp which is important if your server and database hosted on different region and also count network time for sending data. To correctly do this, use $currentDate like this:
db.collection.findOneAndUpdate({_id: 123 }, {
$set: { /** update doc here **/ },
$currentDate: { lastUpdatedTimestamp: true}
});
Similarly using updateOne
db.collection.updateOne({_id: 123 }, {
$set: { /** update doc here **/ },
$currentDate: { lastUpdatedTimestamp: true}
});
I'm new to firebase, actually I'm trying to load some data based on timestamp and retrieve between two timestamp using startat and endat. Data is pushed through Python.
The data is as shown in the sample screenshot
sample data
The data path popping-fire-7751.firebaseio.com/device488
activevolt: 396
avgPowerFactor: 0.95
avgThdi: 7.5
cubetemp: 37
datetime: "2016-02-08 15:16:32"
timestamp: 1454924792
totalPowerGen: 34
I'm pushing data through python setting the priority as timestamp.
When I try to filter as shown here it returns null. But without startAt and endAt it shows all the values.
http://jsfiddle.net/casperkamal/EZUtP/64/
var startTime = 1454924792;
var endTime = 1454924798;
new Firebase("https://popping-fire-7751.firebaseio.com/")
.startAt(startTime)
.endAt(endTime)
.once('value', show);
can anybody help me on what I'm doing wrong ?
You're trying to skip a level in your JSON tree:
var startTime = 1454924792;
var endTime = 1454924798;
new Firebase("https://popping-fire-7751.firebaseio.com/")
.child('device488')
.startAt(startTime)
.endAt(endTime)
.once('value', show);
I highly recommend that you don't depend on the implicit priority anymore and instead just filter using orderByChild()
new Firebase("https://popping-fire-7751.firebaseio.com/")
.child('device488')
.orderByChild('timestamp')
.startAt(startTime)
.endAt(endTime)
.once('value', show);
You'll need to add an index to your Security Rules:
{
"rules": {
"device488": {
".indexOn": ["timestamp"]
}
}
}
But the more explicit behavior is well worth the effort of adding this.
In MongoDB collection I have 3 objects. I need to update one variable (date type) in each object. The main task is to increment the date of the objects. For example: all objects have the same variable:
"Time1" : ISODate("2016-01-12T21:37:46.738Z")
My problem is to update the first object with the current date, manually I do it in this way:
$db.getCollection('my.data')({'_id':ObjectId("52e637fca92cf1ec6a73c1e8")}, {$currentDate: {Time1: true}})
The next is to increase the date of the second object by 1 day, I mean to update it with tomorrow date. I couldn't do it through the shell, because $inc doesn't work with Date type.
So, I am lost with javascript
I found how to get it with java script but I don't know how to collect all if this in one script.
var tomorrow = new Date();
tomorrow.setDate(today.getDate()+1);
Thank you for your help.
You could use the $set operator for the other fields, together the the $currentDate operator all within the update object:
var tomorrow = new Date();
tomorrow.setDate(tomorrow.getDate()+1);
var dayAfterTomorrow = new Date();
dayAfterTomorrow.setDate(dayAfterTomorrow.getDate()+2);
db.getCollection("my.data").update(
{ "_id": ObjectId("52e637fca92cf1ec6a73c1e8") },
{
"$currentDate": { "Time1": true },
"$set": {
"Time2": tomorrow,
"Time3": dayAfterTomorrow
}
}
)
Is there a quicker way to do this? From what I understand it cannot be done in a projection using the aggregation pipeline. Do I have to pre-calculate this? I basically want to emit part of the date (i.e. an hour) in a map function (for a map-reduce). I appreciate you taking the time to help me :-)
db.events.find().snapshot().forEach(
function (e) {
e.StartTime = new Date(e.start_time);
db.events.save(e);
}
)
The Bulk Operations API is the fastest "safe" way to do this:
var bulk = db.events.initializeOrderedBulkOp();
var count = 0;
db.events.find({ },{ "start_time": 1 }).snapshot().forEach(function(e) {
bulk.find({ "_id": e._id }).updateOne({
"$set": { "StartTime": new Date(e.start_time) }
});
count++;
if ( count % 1000 == 0 ) {
bulk.execute();
bulk = db.events.initializeOrderedBulkOp();
}
});
if ( count % 1000 != 0 )
bulk.execute();
That will only send and return from the server once per every 1000 documents read. So the decreased traffic there saves a lot of time, as does working with only the required fields.
If this is absolutely a "one off" operation that does not need to continue to happen in production, and if you are able to do so then you can always use db.eval(). But please read the documentation and warnings there as it is not a very good idea:
db.eval(function() {
db.events.find({ },{ "start_time": 1 }).snapshot().forEach(function(e) {
db.events.update(
{ "_id": e._id },
{ "$set": { "StartTime": new Date(e.start_time) } }
});
]);
But if you are looking for any other way to "convert" a field, there is presently no way for an update operation to refer to an existing value of a field and use it to update another, or even itself. There are exceptions such as $inc or $bit, but these have specific purposes.
I have collection named inventory where I have multiple documents that has values for each doc
{ "apples": 2 ,"oranges": 3, "carrots": 5 }
{ "apples": 4, "oranges": 6, "carrots": 9 }
How do I update push all fruits in to a single array on multiple documents like so:
{ "fruits": { "apples":2 ,"oranges":3 }, "carrots": 5 }
First thing to note here is that the example you give is not an array but just a sub-document for "fruits" that has different keys. An "array" in MongoDB would look like this:
{ "fruits": [{ "apples":2 } , { "orange":3 }], "carrot": 5 }
Also, aside from the term "fruits" being subjective, as with no other identifier you would have to specify a "list" of things that qualify as fruits, the other thing to consider is that there is no actual way in MongoDB at present to refer to the existing value of a field when processing an update.
What that means is you need to .find() each document to retrieve the data in order to be able to work with the sort of "re-structure" that you want. This essentially means looping the results an performing an .update() operation for each document.
The Bulk API for MongoDB 2.6 and greater can be of some help here, where at least the "write" operations to the database can be sent in batches, rather than one at a time:
var bulk = db.collection.initializeOrderedBulkOp();
var count = 0;
var fruits = ["apples","oranges"];
var unset = {};
fruits.forEach(function(fruit) {
unset[fruit] = 1;
});
db.collection.find({}).forEach(function(doc) {
var fields = [];
fruits.forEach(function(fruit) {
if ( doc.hasOwnProperty(fruit) ) {
var subDoc = {};
subDoc[fruit] = doc[fruit];
fields.push(subDoc);
}
});
bulk.find({ "_id": doc._id }).updateOne({
"$unset": unset, "$push": { "fruits": { "$each": fields } }
});
count++;
if ( count % 1000 == 0 ) {
bulk.execute();
var bulk = db.collection.initializeOrderedBulkOp();
}
});
if ( count % 1000 != 0 )
bulk.execute();
That also uses the $each modifier for $push in order to add multiple array entries at once. The $unset operator can be safely called for fields that don't exist in the document so there is no need to check for their presence in the document as is otherwise required when constructing the array of elements to $push.
Of course if you actually want a document like what you gave an example of that is not actually an array, then you construct differently with the $set operator:
var fields = {};
fruits.forEach(function(fruit) {
if ( doc.hasOwnProperty(fruit) )
fields[fruit] = doc[fruit];
});
bulk.find({ "_id": doc._id }).updateOne({
"$unset": unset, "$set": { "fruits": fields }
});
Whatever the case is you need to loop the existing collection. There is no operation that allows you to "take" an existing value in a document and "use it" in order to set a new value from a server side perspective.