Well, I am stuck since a long time on how to treat properly an array in Node js in order to persist it in my mongobd database.
I tried a lot of stuff, i am feeling close to success, but still stuck with it with the var filmDataSchema and the var film syntax.
I really need a little help, please:
Are you using mongoose? it's an elegant mongodb object modeling for node.js and has a really nice interface to handle arrays so let's say you have a student model with grades array and you want to add a new grade, you can use the '$push' (if) or $addToSet operators like so
StudentModel.update(
{ _id: student._id },
{ $addToSet: { grades: grade } }
);
StudentModel.update(
{ _id: student._id },
{ $push: { grades: grade } }
);
use $addToSet if you want only unique items to be pushed into array. use $push if you just want to add the object to array whether or not the object is already there
more about mongoose can be found here https://mongoosejs.com/
Related
This is a document from my profiles collection:
{
_id: ObjectId("5f2ba3a43feccd0004b8698c")
userID: "238906554584137728",
serverID: "533691583845892100",
username: "W.M.K",
money: 15775,
__v: 4,
items: [...],
serverName: "W-15i: Overworld",
gladium: 7959.33,
stocks: [{
_id: {...},
stockID: "605b26d309a48348e05d88c9",
name: "GOOGL",
amount: 1,
desc: "Alphabet Inc's (Google) Stock."
}]
}
I'm using const profile = await Profile.findOne({userID: userID, 'stocks.stockName': stock.name})
EDIT: For one stock, it's as easy as profile.stocks[0].amount -=1. But when I have multiple stocks, how do I get the amount and manipulate it like amount -= 1?
there are few ways you can go through an array of objects in mongoose... you can map through the array, you can also use dot notation in an update. Either way one thing you'll need to do is mark the document modified because mongo doesn't detect changes in arrays of objects. To do this, make the changes and then use the markModified() function on the document. It takes a parameter of the array field name that was modified. in this case.... profile.markModified('stocks')
This is done after changes and before the save.
You can use dot notation with mongoose and you can also iterate through the array via map or forEach
The benefit of using map is that you can double it up with .filter() to filter the array for certain stocks
As already stated by Jeremy, it is basically a matter of array manipulations.
Here is one example:
for(let i = 0; i < profile.stocks.length; i++) {
profile.stocks[i].amount -= 1;
}
If you just want to update the profile on your database, you could also look into some very advanced mongodb querys (I think it's called "aggregation"), but that is probably overkill.
Is MongoDB capable of funding number of random documents without making multiple queries?
e.g. I implemented on the JS side after loading all the document in the collection, which is wasteful - hence just wanted to check if this can be done better with one db query?
The path I took on the JS side:
get all data
make an array of the IDs
shuffle array of IDs (random order)
splice the array to the number of document required
create a list of document by selecting them by ID which we have left after two previous operations, one by one from the whole collection
Two major drawback are that I am loading all data - or I make multiple queries.
Any suggestion much appreciated
This was answered long time ago and, since then, MongoDB has greatly evolved.
As posted in another answer, MongoDB now supports sampling within the Aggregation Framework since version 3.2:
The way you could do this is:
db.products.aggregate([{$sample: {size: 5}}]); // You want to get 5 docs
Or:
db.products.aggregate([
{$match: {category:"Electronic Devices"}}, // filter the results
{$sample: {size: 5}} // You want to get 5 docs
]);
However, there are some warnings about the $sample operator:
(as of Nov, 6h 2017, where latest version is 3.4) => If any of this is not met:
$sample is the first stage of the pipeline
N is less than 5% of the total documents in the collection
The collection contains more than 100 documents
If any of the above conditions are NOT met, $sample performs a
collection scan followed by a random sort to select N documents.
Like in the last example with the $match
OLD ANSWER
You could always run:
db.products.find({category:"Electronic Devices"}).skip(Math.random()*YOUR_COLLECTION_SIZE)
But the order won't be random and you will need two queries (one count to get YOUR_COLLECTION_SIZE) or estimate how big it is (it is about 100 records, about 1000, about 10000...)
You could also add a field to all documents with a random number and query by that number. The drawback here would be that you will get the same results every time you run the same query. To fix that you can always play with limit and skip or even with sort. you could as well update those random numbers every time you fetch a record (implies more queries).
--I don't know if you are using Mongoose, Mondoid or directly the Mongo Driver for any specific language, so I'll write all about mongo shell.
Thus your, let's say, product record would look like this:
{
_id: ObjectId("..."),
name: "Awesome Product",
category: "Electronic Devices",
}
and I would suggest to use:
{
_id: ObjectId("..."),
name: "Awesome Product",
category: "Electronic Devices",
_random_sample: Math.random()
}
Then you could do:
db.products.find({category:"Electronic Devices",_random_sample:{$gte:Math.random()}})
then, you could run periodically so you update the document's _random_sample field periodically:
var your_query = {} //it would impact in your performance if there are a lot of records
your_query = {category: "Electronic Devices"} //Update
//upsert = false, multi = true
db.products.update(your_query,{$set:{_random_sample::Math.random()}},false,true)
or just whenever you retrieve some records you could update all of them or just a few (depending on how many records you've retrieved)
for(var i = 0; i < records.length; i++){
var query = {_id: records[i]._id};
//upsert = false, multi = false
db.products.update(query,{$set:{_random_sample::Math.random()}},false,false);
}
EDIT
Be aware that
db.products.update(your_query,{$set:{_random_sample::Math.random()}},false,true)
won't work very well as it will update every products that matches your query with the same random number. The last approach works better (updating some documents as you retrieve them)
Since 3.2 there is an easier way to to get a random sample of documents from a collection:
$sample
New in version 3.2.
Randomly selects the specified number of documents from its input.
The $sample stage has the following syntax:
{ $sample: { size: <positive integer> } }
Source: MongoDB Docs
In this case:
db.products.aggregate([{$sample: {size: 10}}]);
Here is what I came up in the end:
var numberOfItems = 10;
// GET LIST OF ALL ID's
SchemaNameHere.find({}, { '_id': 1 }, function(err, data) {
if (err) res.send(err);
// shuffle array, as per here https://github.com/coolaj86/knuth-shuffle
var arr = shuffle(data.slice(0));
// get only the first numberOfItems of the shuffled array
arr.splice(numberOfItems, arr.length - numberOfItems);
// new array to store all items
var return_arr = [];
// use async each, as per here http://justinklemm.com/node-js-async-tutorial/
async.each(arr, function(item, callback) {
// get items 1 by 1 and add to the return_arr
SchemaNameHere.findById(item._id, function(err, data) {
if (err) res.send(err);
return_arr.push(data);
// go to the next one item, or to the next function if done
callback();
});
}, function(err) {
// run this when looped through all items in arr
res.json(return_arr);
});
});
skip didn't work out for me. Here is what I wound up with:
var randomDoc = db.getCollection("collectionName").aggregate([ {
$match : {
// criteria to filter matches
}
}, {
$sample : {
size : 1
}
} ]).result[0];
gets a single random result, matching the criteria.
Sample may not be best as you wouldn't get virtual like that.
Instead, create a function in your back end that shuffles the results.
Then return the shuffled array instead of the mongodb result
I am trying to update a value in the nested array but can't get it to work.
My object is like this
{
"_id": {
"$oid": "1"
},
"array1": [
{
"_id": "12",
"array2": [
{
"_id": "123",
"answeredBy": [], // need to push "success"
},
{
"_id": "124",
"answeredBy": [],
}
],
}
]
}
I need to push a value to "answeredBy" array.
In the below example, I tried pushing "success" string to the "answeredBy" array of the "123 _id" object but it does not work.
callback = function(err,value){
if(err){
res.send(err);
}else{
res.send(value);
}
};
conditions = {
"_id": 1,
"array1._id": 12,
"array2._id": 123
};
updates = {
$push: {
"array2.$.answeredBy": "success"
}
};
options = {
upsert: true
};
Model.update(conditions, updates, options, callback);
I found this link, but its answer only says I should use object like structure instead of array's. This cannot be applied in my situation. I really need my object to be nested in arrays
It would be great if you can help me out here. I've been spending hours to figure this out.
Thank you in advance!
General Scope and Explanation
There are a few things wrong with what you are doing here. Firstly your query conditions. You are referring to several _id values where you should not need to, and at least one of which is not on the top level.
In order to get into a "nested" value and also presuming that _id value is unique and would not appear in any other document, you query form should be like this:
Model.update(
{ "array1.array2._id": "123" },
{ "$push": { "array1.0.array2.$.answeredBy": "success" } },
function(err,numAffected) {
// something with the result in here
}
);
Now that would actually work, but really it is only a fluke that it does as there are very good reasons why it should not work for you.
The important reading is in the official documentation for the positional $ operator under the subject of "Nested Arrays". What this says is:
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
Specifically what that means is the element that will be matched and returned in the positional placeholder is the value of the index from the first matching array. This means in your case the matching index on the "top" level array.
So if you look at the query notation as shown, we have "hardcoded" the first ( or 0 index ) position in the top level array, and it just so happens that the matching element within "array2" is also the zero index entry.
To demonstrate this you can change the matching _id value to "124" and the result will $push an new entry onto the element with _id "123" as they are both in the zero index entry of "array1" and that is the value returned to the placeholder.
So that is the general problem with nesting arrays. You could remove one of the levels and you would still be able to $push to the correct element in your "top" array, but there would still be multiple levels.
Try to avoid nesting arrays as you will run into update problems as is shown.
The general case is to "flatten" the things you "think" are "levels" and actually make theses "attributes" on the final detail items. For example, the "flattened" form of the structure in the question should be something like:
{
"answers": [
{ "by": "success", "type2": "123", "type1": "12" }
]
}
Or even when accepting the inner array is $push only, and never updated:
{
"array": [
{ "type1": "12", "type2": "123", "answeredBy": ["success"] },
{ "type1": "12", "type2": "124", "answeredBy": [] }
]
}
Which both lend themselves to atomic updates within the scope of the positional $ operator
MongoDB 3.6 and Above
From MongoDB 3.6 there are new features available to work with nested arrays. This uses the positional filtered $[<identifier>] syntax in order to match the specific elements and apply different conditions through arrayFilters in the update statement:
Model.update(
{
"_id": 1,
"array1": {
"$elemMatch": {
"_id": "12","array2._id": "123"
}
}
},
{
"$push": { "array1.$[outer].array2.$[inner].answeredBy": "success" }
},
{
"arrayFilters": [{ "outer._id": "12" },{ "inner._id": "123" }]
}
)
The "arrayFilters" as passed to the options for .update() or even
.updateOne(), .updateMany(), .findOneAndUpdate() or .bulkWrite() method specifies the conditions to match on the identifier given in the update statement. Any elements that match the condition given will be updated.
Because the structure is "nested", we actually use "multiple filters" as is specified with an "array" of filter definitions as shown. The marked "identifier" is used in matching against the positional filtered $[<identifier>] syntax actually used in the update block of the statement. In this case inner and outer are the identifiers used for each condition as specified with the nested chain.
This new expansion makes the update of nested array content possible, but it does not really help with the practicality of "querying" such data, so the same caveats apply as explained earlier.
You typically really "mean" to express as "attributes", even if your brain initially thinks "nesting", it's just usually a reaction to how you believe the "previous relational parts" come together. In reality you really need more denormalization.
Also see How to Update Multiple Array Elements in mongodb, since these new update operators actually match and update "multiple array elements" rather than just the first, which has been the previous action of positional updates.
NOTE Somewhat ironically, since this is specified in the "options" argument for .update() and like methods, the syntax is generally compatible with all recent release driver versions.
However this is not true of the mongo shell, since the way the method is implemented there ( "ironically for backward compatibility" ) the arrayFilters argument is not recognized and removed by an internal method that parses the options in order to deliver "backward compatibility" with prior MongoDB server versions and a "legacy" .update() API call syntax.
So if you want to use the command in the mongo shell or other "shell based" products ( notably Robo 3T ) you need a latest version from either the development branch or production release as of 3.6 or greater.
See also positional all $[] which also updates "multiple array elements" but without applying to specified conditions and applies to all elements in the array where that is the desired action.
I know this is a very old question, but I just struggled with this problem myself, and found, what I believe to be, a better answer.
A way to solve this problem is to use Sub-Documents. This is done by nesting schemas within your schemas
MainSchema = new mongoose.Schema({
array1: [Array1Schema]
})
Array1Schema = new mongoose.Schema({
array2: [Array2Schema]
})
Array2Schema = new mongoose.Schema({
answeredBy": [...]
})
This way the object will look like the one you show, but now each array are filled with sub-documents. This makes it possible to dot your way into the sub-document you want. Instead of using a .update you then use a .find or .findOne to get the document you want to update.
Main.findOne((
{
_id: 1
}
)
.exec(
function(err, result){
result.array1.id(12).array2.id(123).answeredBy.push('success')
result.save(function(err){
console.log(result)
});
}
)
Haven't used the .push() function this way myself, so the syntax might not be right, but I have used both .set() and .remove(), and both works perfectly fine.
I am working in Node.js and am attempting to push or pull the contents of an array to my mongodb collection. Currently my [working] code to pull objects from the array in FieldArray looks something like this:
for (var i=0; i < MyList.length; i++) {
collection.update(
{field:"MyValue"},
{$pull: {FieldArray: MyList[i]}},
function(err, item){...}
);
}
I'm aware of the ability to use $push/$each, $addToSet/$each and $pullall but they don't seem to accept values dynamically from an array (or I haven't found any indication that it can). Basically, I'd like to be able to use this function with an array of one item or one hundred, using the appropriate batch calls.
Is there any way to make such a call without having to loop through a separate call on the database for each iteration?
You want $pullAll. It does exactly what you are trying to iterate over
collection.update(
{ "field": "MyValue" },
{ "$pullAll": { "FieldArray": MyList } }
)
If that doesn't work then then your array elements are not matching the structure used in your document. Make them that way.
i have this structure in my mongodb
{
category:['A','B'],
info:.....,
}
{
category:['A','F','T'],
info:.....,
}
{
category:['A','C'],
info:.....,
}
{
category:['D','B'],
info:.....,
}
i have to query all categories,
var db = mongo.db(read+"#127.0.0.1:27017/database",{safe:false});
db.collection('comercio').find({},{_id:0,'category.$':1},function(err, result_array)
first question, there is any way to get all categories?? an other aproach instead of mine??
second question....
i have to make an array that contains all categories but not repeat any category... in this example i have to make an array that contains this...
all_categories=['A','B','C','D','F','T'];
thank you all again...
You must use Aggregation framework for this query, like this:
db.comercio.aggregate(
{ $unwind : "$category" }
);
After unwind you can use other aggregations (e.g. group) to get what you need.
You don't need aggregation framework to get back an array of distinct categories.
Just use distinct:
> db.comercio.distinct("category");
["A","B","C","D","F","T"]