I get a list of objects from an API:
let sold = [
{ objId: 3240747,
soldDate: '2018-09-27',
soldPrice: 4610000,
apartmentNumber: '1202',
soldPriceSource: 'bid',
},
{ objId: 3234263,
soldDate: '2018-09-24',
soldPrice: 2580000,
soldPriceSource: 'bid',
}
...
]
I store these in a collection:
soldCollection.insertMany(sold)
Some of the objects have been retrieved before, and I only want to store the once that are not already in the database.
dbo.collection("sold").createIndex({ "objId": 1 }, { unique: true })
What would be an efficient way of doing this? Should I ask for each object before storing it or is there method for dealing with this?
By default insertMany will stop inserting when first error occurs (E11000 duplicate key error in this case). You can change that behavior by specifying ordered parameter set to false. In that case you'll get a list of errors from failed inserts however all valid documents will be inserted succesfully:
db.sold.insertMany(sold, { ordered: false })
docs example here
Related
Suppose I have an object in mongoDB:
collection_name: book_info
{_id:"121as", "book_num":"12a", "book_name":"lotr", "info":[ {"borrowerAddress":["NY","Delhi"] },
{"borrowerAddress":["SI","Ghana"] } ] }
I want to delete an element "NY" from info.
For this I can do is fetch the object from book_info collection and apply loop and delete when element found inside array and save the data into db.
But I want to do is delete and update at same time without fetching data and looping and updating again.
Also, if direct db manipulation is possible, can anyone suggest me whether fetching data, applying loop and deleting data is efficient or deleting data from db directly is more faster, based on solutions provided.
If anyone needs any further information, please let me know.
Additional Question:
doc1: {_id:"121as", "book_num":"12a", "book_name":"lotr", "info":[ {"borrowerAddress":["NY","Delhi"] },
{"borrowerAddress":["SI","Ghana"] } ] }
doc2: {_id:"213s", "book_num":"1c", "book_name":"hp", "info":[ {"borrowerAddress":["NY","Delhi"] },
{"borrowerAddress":["SI","Ghana"] } ] }
Assume I have multiple documents inside a book_info collection with different book names and details and I want to delete document for "book_name":"lotr" only, I delete address "NY" from document.
You can try update query with $pull operator to remove matching element from array, $[] positional for all elements in info array,
db.collection.update(
{ "info.borrowerAddress": "NY" },
{ $pull: { "info.$[].borrowerAddress": "NY" },
{ multi: true }
})
Playground
Assume I have multiple documents inside a book_info collection with different book names and details and I want to delete document for "book_name":"lotr" only, I delete address "NY" from document.
db.collection.update(
{ "book_name":"lotr", "info.borrowerAddress": "NY" },
{ $pull: { "info.$[].borrowerAddress": "NY" },
{ multi: true }
})
Playground
// my db structure now
rcv : {
visible: 'all',
ids: [
[0] : userId,
[1] : user2Id ]
}
this is how i query to get the data it works.
//service.ts
getAlbumByUserId(userId) {
return this.afs.collection('albums', ref => ref.where('rcv.visible', '==', 'all').where('rcv.ids', 'array-contains', userId)).valueChanges();
}
//component.ts
this.service.getAlbumByUserId(this.userId);
but i want to set the structure like this but i don't know how to query nested objects in firebase
// database structure
rcv : {
visible: 'all',
ids: {
userId: {
id: userId
}
user2Id: {
id: user2Id
}
}
}
You're looking for the array-contains operator, which can check if a field that is an array contains a certain value.
You're already using the correct array-contains operator, but not with the correct syntax. The array-contains operator checks whether any element of your array is exactly the same as the value you pass in. So you need to pass in the complete value that exists in the array:
ref.where('rcv.visible', '==', 'all').where('rcv.ids', 'array-contains', { id: userId })
As you add more data to the array, it may become unfeasible to reproduce the entire array element for the query. In that case, the common approach is to add an additional field where you keep just the IDs.
So you'd end up with one field (say rcv.users) where you keep all details about the receiving users, and one field (say rcv.ids) where you just keep their IDs, and that you use for querying.
In products collection, i have an Array of recentviews which has 2 fields viewedBy & viewedDate.
In a scenario if i already have a record with viewedby, then i need to update it. For e.g if i have array like this :-
"recentviews" : [
{
"viewedby" : "abc",
"vieweddate" : ISODate("2014-05-08T04:12:47.907Z")
}
]
And user is abc, so i need to update the above & if there is no record for abc i have to $push.
I have tried $set as follows :-
db.products.update( { _id: ObjectId("536c55bf9c8fb24c21000095") },
{ $set:
{ "recentviews":
{
viewedby: 'abc',
vieweddate: ISODate("2014-05-09T04:12:47.907Z")
}
}
}
)
The above query erases all my other elements in Array.
Actually doing what it seems like you say you are doing is not a singular operation, but I'll walk through the parts required in order to do this or otherwise cover other possible situations.
What you are looking for is in part the positional $ operator. You need part of your query to also "find" the element of the array you want.
db.products.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
"$set": {
"recentviews.$.vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
)
So the $ stands for the matched position in the array so the update portion knows which item in the array to update. You can access individual fields of the document in the array or just specify the whole document to update at that position.
db.products.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
"$set": {
"recentviews.$": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
)
If the fields do not in fact change and you just want to insert a new array element if the exact same one does not exist, then you can use $addToSet
db.products.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
$addToSet:{
"recentviews": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
)
However if you are just looking for for "pushing" to an array by a singular key value if that does not exist then you need to do some more manual handling, by first seeing if the element in the array exists and then making the $push statement where it does not.
You get some help from the mongoose methods in doing this by tracking the number of documents affected by the update:
Product.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095"),
"recentviews.viewedby": "abc"
},
{
"$set": {
"recentviews.$": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
},
function(err,numAffected) {
if (numAffected == 0) {
// Document not updated so you can push onto the array
Product.update(
{
"_id": ObjectId("536c55bf9c8fb24c21000095")
},
{
"$push": {
"recentviews": {
"viewedby": "abc",
"vieweddate": ISODate("2014-05-09T04:12:47.907Z")
}
}
},
function(err,numAffected) {
}
);
}
}
);
The only word of caution here is that there is a bit of an implementation change in the writeConcern messages from MongoDB 2.6 to earlier versions. Being unsure right now as to how the mongoose API actually implements the return of the numAffected argument in the callback the difference could mean something.
In prior versions, even if the data you sent in the initial update exactly matched an existing element and there was no real change required then the "modified" amount would be returned as 1 even though nothing was actually updated.
From MongoDB 2.6 the write concern response contains two parts. One part shows the modified document and the other shows the match. So while the match would be returned by the query portion matching an existing element, the actual modified document count would return as 0 if in fact there was no change required.
So depending on how the return number is actually implemented in mongoose, it might actually be safer to use the $addToSet operator on that inner update to make sure that if the reason for the zero affected documents was not just that the exact element already existed.
I have the following data structure for my users in my mongodb:
{
_id: "someId",
profile: {
username: "oliv",
friendRequests: [
{ fromUserId: "anId", accepted: false, created: "someDate"},
{ fromUserId: "otherId", accepted: true, created: "otherDate"}
]
}
I'd like to retrieve the user objects that are referenced in my logged user's friendsRequested.
So I tried something like this:
Meteor.users.find({_id: {$in: Meteor.user().profile.friendRequests.fromUserId}});
// FYI: Meteor.users is the collection and Meteor.user() retrieves the current user
But it's not working. I'm assuming it's because of the nested array.
Is there any way of telling mongo to iterate through the fromUserId or something?
Thanks
Change your query to:
Meteor.users.find({_id: {$in: _.pluck(Meteor.user().profile.friendRequests, 'fromUserId') }});
Reasoning:
friendRequests is an array of objects, and $in wants an array of strings(the ids), using _.pluck you're able to pass an array of objects, and tell _ to only return the field fromUserId for each object in the array.
Question in comment ("what if I only want to get the ids from friend requests where "accepted" is false?"):
_.pluck(
_.filter(Meteor.user().profile.friendRequests, function (req) {
return !req.accepted;
}),
'fromUserid'
);
Underscore filter docs
I've got an object in my $scope that contains a bunch of details about, say, an election. This object includes a voters array of objects, each with an _id:
$scope.election = {
voters: [
{ _id: '123' },
{ _id: '456' },
{ _id: '789' }
]
}
Also in my scope I have details about the currently logged in user:
$scope.user = { _id: '456' }
How can I bind ng-disabled to the presence of $scope.user._id in the array of objects $scope.voters?
What I've Tried
I have success simply displaying the presence of $scope.user._id in $scope.election.voters like this (Jade syntax):
pre(ng-bind="election.voters | filter:{user._id} | json")
When the current user is among the voters, they get displayed. When they're not among the voters, I get an empty array. That seems quite close to what I want.
But using the same filter (sans | json) with ng-disabled, I get the Angular Infinite $digest loop error.
Is this situation too complicated? Should I move it to a $filter? If so, how would I go about making it generic enough to be useful in a number of situations (if that's even feasible)?
Can run a simple filter right in controller, or using app.filter('filterName', func...) create a custom filter you can use in markup
$scope.userIsVoter = function() {
return $scope.election.voters.filter(function(el) {
return el._id == $scope.user._id;
}).length
}
<button ng-disabled="userIsVoter()">Do Something</button>