I try to update several items in mongo collection with a single request:
// [1, 2, 3] - numbers array.
const days = req.body.days;
const updated = await Item.update(
{shift: shiftId, day: {$in: days}},
{multi : true},
{update: {
name: 'one value for all objects witch corresponding condition',
},
function(err, docs) {
console.log(docs);
}
);
This Item schema:
const itemSchema = new Schema({
shift: {
ref: 'shift',
type: Schema.Types.ObjectId,
required: true
},
day: {
type: Number,
required: true
},
name: {
type: String
}
});
But then I call this code updating only one objects.
I had many Items with have the same shift. But every Item had a unique day and I need update all Items which contains in days array.
For example, if we have shiftId = 'abc' and days = [1, 2], I need update all Items which have shiftId = 'abc' and have day = 1 OR day = 2 both should be updated.
Why my solution have unexpected behavior and updating only one object meanwhile I set {multi : true}? How to fix it? Thank You.
multi: true is the last parameter in an update query and you have used it second parameter.
So you have to use multi: true in last parameter and need to use $set to update a field.
const updated = await Item.update(
{ "shift": shiftId, "day": { "$in": days }},
{ "$set": { "name": "one value for all objects witch corresponding condition" }},
{ "multi": true }
)
Related
I have a Schema that holds an array of objects for comments and I would like to update the boolean value of the flagged comments accordingly, I have tried updateOne and aggregate but it isn't working out at this point, I have also tried to use $elemMatch but it isn't working.
The comment _id is being pulled from the front end element that has an ID that is the same as the id that needs to be pulled from MongoDB.
Comments Array within the question Schema:
comments: [
{
user: {
type: Object,
},
commentDate: {
type: Date,
default: Date.now()
},
flagged: {
type: Boolean,
default: false
},
flaggedDate:{type: Date},
comment: String,
}
],
function I tried to run last.
const id = req.params.id
const updateFlag = Question.updateOne(
{
comments: [
{
_id: id
}
]
},
{
$set: {
comments: [
{
flagged: req.body.flagged
}
]
}
}
)
Any help would be appreciated!
You can do it with positional operator - $:
db.collection.update({
"comments._id": "3"
},
{
"$set": {
"comments.$.flagged": true
}
})
Working example
I have one collection who include some value coming from sensor. My collection look like this.
const MainSchema: Schema = new Schema(
{
deviceId: {
type: mongoose.Types.ObjectId,
required: true,
ref: 'Device',
},
sensorId: {
type: mongoose.Types.ObjectId,
default: null,
ref: 'Sensor',
},
value: {
type: Number,
},
date: {
type: Date,
},
},
{
versionKey: false,
}
);
I want to get data from this collection with my endpoint. This collection should has more 300.000 documents. I want to get data from this collection with sensor data. (like name and desc. to "Sensor")
My Sensor Collection:
const Sensor: Schema = new Schema(
{
name: {
type: String,
required: true,
min: 3,
},
description: {
type: String,
default: null,
},
type: {
type: String,
},
},
{
timestamps: true,
versionKey: false,
}
);
I use 2 method for get data from MainSchema. First approach is look like this (Include aggregate):
startDate, endDate and _sensorId are passed by parameter for this functions.
const data= await MainSchema.aggregate([
{
$lookup: {
from: 'Sensor',
localField: 'sensorId',
foreignField: '_id',
as: 'sensorDetail',
},
},
{
$unwind: '$sensorDetail',
},
{
$match: {
$and: [
{ sensorId: new Types.ObjectId(_sensorId) },
{
date: {
$gte: new Date(startDate),
$lt: new Date(endDate),
},
},
],
},
},
{
$project: {
sensorDetail: {
name: 1,
description: 1,
},
value: 1,
date: 1,
},
},
{
$sort: {
_id: 1,
},
},
]);
Second approach look like this (Include find and populate):
const data= await MainSchema.find({
sensorId: _sensorId,
date: {
$gte: new Date(startDate),
$lte: new Date(endDate),
},
})
.lean()
.sort({ date: 1 })
.populate('sensorId', { name: 1, description: 1});
Execution time for same data set:
First approach: 25 - 30 second
Second approach: 11 - 15 second
So how can i get this data more faster. Which one is best practise?
And how can i do extras for improve the query speed?
Overall #NeNaD's answer touches on a lot of the important points. What I'm going to say in this one should be considered in addition to that other information.
Index
Just to clarify, the ideal index here would be a compound index of { sensorId: 1, date: 1 }. This index follows the ESR Guidance for index key ordering and will provide the most efficient retrieval of the data according to the query predicates specified in the $match stage.
If the index: true annotation in Mongoose creates two separate single field indexes, then you should go manually create this index in the collection. MongoDB will only use one of those indexes to execute this query which will not be as efficient as using the compound index described above.
Also regarding the existing approach, what is the purpose of the trailing $sort?
If the application (a chart in this situation) does not need sorted results then you should remove this stage entirely. If the client does need sorted results then you should:
Move the $sort stage earlier in the pipeline (behind the $match), and
Test if including the sort field in the index improves performance.
As written, the $sort is currently a blocking operation which is going to prevent any results from being returned to the client until they are all processed. If you move the $sort stage up and can change it to sort on date (which probably makes sense for sensor data) the it should automatically use the compound index that we mentioned earlier to provide the sort in a non-blocking manner.
Stage Ordering
Ordering of aggregation stages is important, both for semantic purposes as well as for performance reasons. The database itself will attempt to do various things (such as reordering stages) to improve performance so long as it does not logically change the result set in any way. Some of these optimizations are described here. As these are version specific anyway, you can always take a look at the explain plan to get a better indication of what specific changes the database has applied. The fact that performance did not improve when you manually moved the $match to the beginning (which is generally a best practice) could suggest that the database was able to automatically do that on your behalf.
Schema
I'm a little curious about the schema itself. Is there any reason that there are two separate collections here?
My guess is that this is mostly a play at 'normalization' to help reduce data duplication. That is mostly fine, unless you find yourself constantly performing $lookups like this for most of your read operations. You could certainly consider testing what performance (and storage) looks like if you combine them.
Also for this particular operation, would it make sense to just issue two separate queries, one to get the measurements and one to get the sensor data (a single time)? The aggregation matches on sensorId and the value of that field is what is then used to match against the _id field from the other collection. Unless I'm doing the logic wrong, this should be the same data for each of the source documents.
Time Series Collections
Somewhat related to schema, have you looked into using Time Series Collections? I don't know what your specific goals or pain points are, but it seems that you may be working with IoT data. Time Series collections are purpose-built to help handle use cases like that. Might be worth looking into as they may help you achieve your goals with less hassle or overhead.
Frist step
Create index for sensorId and date properties in the collection. You can do it by specifying index: true in your model:
const MainSchema: Schema = new Schema(
{
deviceId: { type: mongoose.Types.ObjectId, required: true, ref: 'Device' },
sensorId: { type: mongoose.Types.ObjectId, default: null, ref: 'Sensor', index: true },
value: { type: Number },
date: { type: Date, index: true },
},
{
versionKey: false,
}
);
Second step
Aggregation queries can take leverage of indexes only if your $match stage is the first stage in the pipeline, so you should change the order of the items in your aggregation query:
const data= await MainSchema.aggregate([
{
$match: {
{ sensorId: new Types.ObjectId(_sensorId) },
{
date: {
$gte: new Date(startDate),
$lt: new Date(endDate),
},
},
},
},
{
$lookup: {
from: 'Sensor',
localField: 'sensorId',
foreignField: '_id',
as: 'sensorDetail',
},
},
{
$unwind: '$sensorDetail',
},
{
$project: {
sensorDetail: {
name: 1,
description: 1,
},
value: 1,
date: 1,
},
},
{
$sort: {
_id: 1,
},
},
]);
I need to take array from my model, delete from it some days and push to it some other days. It looks something like deleteAndUpdate.
To sum up:
I need to take Car from database. Take reserved property (it's a array), then delete from reserved days from given list, and then add to reserved days from other given list.
My model look:
const CarSchema = mongoose.Schema({
mark:{
type: String,
required: true,},
model:{
type: String,
required: true,},
price:{
type: Number,
required: true,},
available: {
type: Boolean,
required: true,},
reserved:{
type:[Date],
},
pic_1:{
type:String,
required:true,
},
pic_2:{
type:String,
required:true,
},
},
{ collection: 'cars' }
)
I take car by: var car= await Car.findById(carID); and then i need to do sth like that:
car['reserved'].deleted(old_days);
car['reserved'].push(new_days;
car.save();
Could someone help me?
Update can't allow multiple operation at a time in same field, It will throw multiple write error and would create a conflict at your field,
Regular update:
If you want to do it by regular update query you have to do separate do 2 queries,
Delete days: If you want to delete multiple days use $pullAll, and for single you can use $pull
var old_days = [new Date("2021-04-24"), new Date("2021-04-25")];
await Car.updateOne({ _id: carID }, { $pullAll: { reserved: old_days } });
Add days: if you want to add multiple days you can use $push with $each, and for single you can use just $push,
var new_days = [new Date("2021-04-26"), new Date("2021-04-27")];
await Car.updateOne({ _id: carID }, { $push: { reserved: { $each: new_days } } });
Update with aggregation pipeline:
If you are looking for single query you can use update with aggregation pipeline starting from MongoDB 4.2,
$filter to iterate loop of reserved array and remove old days
$concatArrays to concat reserved array with new days
var old_days = [new Date("2021-04-24"), new Date("2021-04-25")];
var new_days = [new Date("2021-04-26"), new Date("2021-04-27")];
await Car.updateOne(
{ _id: carID },
[{
$set: {
reserved: {
$filter: {
input: "$reserved",
cond: { $not: { $in: ["$$this", old_days] } }
}
}
}
},
{
$set: {
reserved: {
$concatArrays: ["$reserved", new_days]
}
}
}]
);
Playground
To remove old item from array you can use $pull
car.update(
{ _id: carID },
{ $pull: { 'reserved': old_days } }
);
You can use $unset to unset the value in the array (set it to null), but not to remove it completely.
To add the item new_days in array, You can either use $push or $addToSet
car.update(
{ _id: carID },
{ $push: { 'reserved': new_days } }
);
I have the following sub-documents :
{
id: 1,
date:2019-04-01 00:21:19.000
},
{
id: 2,
date:2019-03-31 00:21:19.000
} ...
Document schema is :
const barEventSchema = new Schema({
id: {
type: Number,
unique: true,
required: true
},
raw: { type: String },
date: { type: Date },
type: { type: String },
})
const FooSchema = new Schema({
bar: [barEventSchema]
})
I want to do a query based on a date range picked from html input, which has values like 2019-04-01, 2019-03-31.
So on serverside, I want to do something like:
//#star_date = 2019-04-01, #end_date = 2019-04-01
Foo.findOne('bar.date' : {$lte : start_date, $gte: end_date})
However, this returns all the documents.
All documents having any subdocument with date between start and end date range can be retrieved using:
const conditions = {
'bar': {
$elemMatch: {
'date': {
$gte: new Date(start_date),
$lte: new Date(end_date)
}
}
}
}
Foo.find(conditions)
This will return all the documents where there is at least a subdocument having its date between the range specified in condition.
The $elemMatch operator is used to effect this condition on the date field of the bar subdocument.
In Mongoose, how can I update an Array property with item-wise max, and have default null upon instantiation?
I have a Mongodb collection of timeseries where values are stored in a fixed-length array (1 item per minute, 1 document per day).
{
'series': '#1',
'values': [null, null, 1, 2, 3, -4, ... ] //24h*60min=1440 items
}
I am doing computations on ~x000 timeseries on a rather high frequency (100ms), and I want to store the maximum that each of these series met during every minute. For some reason, when I update the documents, Mongoose transforms the .values property into an object, which is more space-consuming and less efficient for my use.
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost:27017/test_db');
const TestSchema = new mongoose.Schema({
series: String,
values: {type: Array, default: Array(5).fill(null), required: true},
});
const Test = mongoose.model('Test', TestSchema);
async function update({series, values}){ //called every minute or so
let updated = { };
for (let {i, v} of values) {
if (updated[`values.${i}`] && updated[`values.${i}`]<v) updated[`values.${i}`]= v;
if (!updated[`values.${i}`]) updated[`values.${i}`]=v;
};
return mongoose.connection.models.Test.updateOne(
{'series':series},
{ '$max': updated },
{ upsert: true, strict: true, new: true}
);
}
async function test_update(){
//get rid of previous tentatives
await mongoose.connection.models.Test.deleteMany({});
let s = new Test({series: '#1'});
await update({series:'#1', values:[{i:3, v:-3},{i:4, v:6},{i:2, v:4}, {i:2, v:7}]});
}
test_update();
I would want my script to return
{
"_id" : ObjectId("5cb351d9d615cd456bd6a4ed"),
"series" : "#1",
"__v" : 0,
"values" : [null, null, 7, -3, 6]
}
and not the current result below:
{
"_id" : ObjectId("5cb351d9d615cd456bd6a4ed"),
"series" : "#1",
"__v" : 0,
"values" : { //values is an Object, not an Array
"2" : 7,
"3" : -3,
"4" : 6
}
}
Thanks!
`I THINK it may be your schema
Instead of:
const TestSchema = new mongoose.Schema({
series: String,
values: {type: Array, default: Array(5).fill(null), required: true},
});
You should make values an array of numbers like this
const TestSchema = new mongoose.Schema({
series: String,
values: {type: [Number], default: Array(5).fill(null), required: true},
});