Ive looked related posts and couldn't quite find what I was looking for.
So I am build a backend rest api and I have certain tests I am collecting data on. The tests have their own models, and these models are associated with collections (obviously).
So I have a separate controller for each model. Now I have a "job" controller which queries data from each separate test. Now I have a separate script where I store these model objects in an JSON object. I am wondering how I can access these models properly (I am close but cant quite assign properly). Here is the block:
const testMappings = {
'aprobe':aprobe,
'status':status,
//'rxserial':rxserial,
}
Now when I try assignment as follows, where testMappings is the imported script variable:
const testMappings = activeTests.testMappings;
console.log(testMappings['aprobe']);
I get the following output:
Model {aprobe}
I would like to access the actual aprobe object. Also if anyone knows a better way of dynamically assigning these (instead of having bunch of if statements ie if(name == 'aprobe').... do something), it would be much appreciated.
You are probably looking for something like below :
const name = 'aprobe';
Object.keys(testMappings).indexOf(name) > -1 ? testMappings[name] : null
the above should give you: Model {aprobe}
So basically if the key exists in your object then you'd like to fetch the value of that key which would give you your model dynamically.
My Meteor client receives data from the server and stores it in minimongo. This data is guaranteed not to change during their session, so I don't need Meteor's reactivity. The static data just happens to arrive by that route; let's just take that as a given.
The data looks like this:
{_id: 'abc...', val: {...}}
On the client, is it more efficient for me to look up values using:
val_I_need = Collection.findOne({id})
or to create a JavaScript object:
data = {}
Collection.find().fetch().map((x) => {data[x._id] = x.val})
and use it for look ups:
val_I_need = data[id]
Is there a tipping point, either in terms of the size of the data or the number of look ups, where the more efficient method changes, or outweighs the initial cost of building the object?
FindOne may be more efficient on larger datasets because it looks up using cursors where _id is an indexed key while your find().fetch() approach requires to get all docs and then iterate manually by mapping.
Note, that findOne could also be replaced by .find({_id:desiredId}).fetch()[0](assuming it returns the desired doc).
More on this in the mongo documentation on query performance.
However, if it concerns only one object that is afterwards not reactively tracked, I would rather load it via a "findOne"-returning method from the server:
export const getOne = new ValidatedMethod({
name: "getOne",
validate(query) {
// validate query schema
// ...
},
run(query) {
// CHECK PERMISSIONS
// ...
return MyCollection.findOne(query);
});
This avoids using publications / subscriptions and thus minimongo for this collection on the current client template. Think about that pub/sub has already some reactivity initialized to observe the collection and thus eats up some computation somewhere.
My gut feeling is that you'll never hit a point where the performance gain of putting it in an object makes a noticeable difference.
It's more likely that your bottleneck will be in the pub/sub mechanism, as it can take a while to send all documents to the client.
You'll see a much more noticeable difference for a large dataset by retrieving the data using a Meteor method.
At which point you've got it in a plain old javascript object anyway and so end up with the small performance gain of native object lookups as well.
What is the best, or just a good, practice for loading and saving relational data in Polymer elements' published properties?
I've used https://github.com/PaulUithol/Backbone-relational to load and save relational data. It depends on Backbone. But now with Polymer's use of Object.observe(), I mostly don't need Backbone's complex model objects (at least I don't need their get() and set() methods), but I can't figure out how I can best get rid of Backbone's complex model objects and just use plain JavaScript objects AND load and save relational data to my data store.
Is there a Polymer-compatible library/web component out there which already implements this? Or a native way to do it?
Here are a couple ways I could do it myself without a library, but I expect I'm missing lots of edge cases.
Load relational data:
// Load from server
model = store.get('model-id');
// model == {'id':'abc', 'name':'Parent', child_ids:['child-id1', 'child-id2']}
for (child_id in model.child_ids){
model.children[child_id] = store.get(child_id);
}
// Use model in Polymer element's published property here
Save relational data:
// Get model from Polymer published property here
model.child_ids = [];
for (child in model.children){
model.child_ids.push(child.id);
}
delete model.children;
store.set(JSON.stringify(model));
// or just store.set(model);
I'am using MongoDB with a nodejs REST service which exposes my data stored inside. I have a question about how to interrogate my data which uses $ref.
Here is a sample of an Object which contains a reference to another object (detail) in anther collection :
{
"_id" : ObjectId("5962c7b53b6a02100a000085"),
"Title" : "test",
"detail" : {
"$ref" : "ObjDetail",
"$id" : ObjectId("5270c7b11f6a02100a000001")
},
"foo" : bar
}
Actually, using Node.js and mongodb module, I do the following :
db.collection("Obj").findOne({"_id" : new ObjectID("5962c7b53b6a02100a000085"},
function(err, item) {
db.collection(item.$ref).findOne({"_id" : item.$id}, function(err,subItem){
...
});
});
In fact I make 2 queries, and get 2 objects. It's a kind of "lazy loading" (not exactly but almost)
My question is simple : is it possible to retrieve the whole object graph in one query ?
Thank you
No, you can't.
To resolve DBRefs, your application must perform additional queries to return the referenced documents. Many drivers have helper methods that form the query for the DBRef automatically. The drivers do not automatically resolve DBRefs into documents.
From the MongoDB docs http://docs.mongodb.org/manual/reference/database-references/.
Is it possible to fetch parent object along with it's $ref using single MongoDB query?
No, it's not possible.
Mongo have no inner support for refs, so it up to your application to populate them (see Brett's answer).
But is it possible to fetch parent object with all its ref's with a single node.js command?
Yes, it's possible. You can do it with Mongoose. It has build-in ref's population support. You'll need to change your data model a little bit to make it work, but it's pretty much what you're looking for. Of course, to do so Mongoose will make the same two MongoDB queries that you did.
Answer of Vladimir is not still valid as the db.dereference method was deleted from MongoDB Nodejs API:
https://www.mongodb.com/blog/post/introducing-nodejs-mongodb-20-driver
The db instance object has been simplified. We've removed the following methods:
db.dereference due to db references being deprecated in the server
No, very few drivers for MongoDb include special support for a DBRef. There are two reasons:
MongoDb doesn't have any special commands to make retrieval of referenced documents possible. So, drivers that do add support are artificially populating the resulting objects.
The more, "bare metal" the API, the less it makes sense. In fact, as. MongoDb collections are schema-less, if the NodeJs driver brought back the primary document with all references realized, if the code then saved the document without breaking the references, it would result in an embedded subdocument. Of course, that would be a mess.
Unless your field values vary, I wouldn't bother with a DBRef type and would instead just store the ObjectId directly. As you can see, a DBRef really offers no benefit except to require lots of duplicate disk space for each reference, as a richer object must stored along with its type information. Either way, you should consider the potentially unnecessary overhead of storing a string containing the referenced collection's documents.
Many developers and MongoDb, Inc. have added an object document mapping layer on top of the existing base drivers. One popular option for MongoDb and Nodejs is Mongoose. As the MongoDb server has no real awareness of referenced documents, the responsibility of the references moves to the client. As it's more common to consistently reference a particular collection from a given document, Mongoose makes it possible to define the reference as a Schema. Mongoose is not schema-less.
If you accept having and using a Schema is useful, then Mongoose is definitely worth looking at. It can efficiently fetch a batch of related documents (from a single collection) from a set of documents. It always is using the native driver, but it generally does operations extremely efficiently and takes some of the drudgery out of more complex application architectures.
I would strongly suggest you have a look at the populate method (here) to see what it's capable of doing.
Demo /* Demo would be a Mongoose Model that you've defined */
.findById(theObjectId)
.populate('detail')
.exec(function (err, doc) {
if (err) return handleError(err);
// do something with the single doc that was returned
})
If instead of findById, which always returns a single document, find were used, with populate, all returned documents' details property will be populated automatically. It's smart too that it would request the same referenced documents multiple times.
If you don't use Mongoose, I'd suggest you consider a caching layer to avoid doing client side reference joins when possible and use the $in query operator to batch as much as possible.
I reach the desired result with next example:
collection.find({}, function (err, cursor) {
cursor.toArray(function (err, docs) {
var count = docs.length - 1;
for (i in docs) {
(function (docs, i) {
db.dereference(docs[i].ref, function(err, doc) {
docs[i].ref = doc;
if (i == count) {
(function (docs) {
console.log(docs);
})(docs);
}
});
})(docs, i)
}
});
});
Not sure that it solution is best of the best, but It is simplest solution that i found.
In this stackoverflow post i read about filtering backbone collections and using subsets.
One answer (by sled) recommends using backbone.subset.js (usage example).
I could not find any further resources on backbone.subset.js and I failed implementing it into my project.
It seems like backbone.subset.js is the perfect solution for what i'm trying to achieve.
(Having one "parent" collection that holds all models at all times, and depending on user input filtering the relevant models from the parent collection into a backbone.subset collection.)
My "parent" collection, holding all tasks:
var TasksAll = Backbone.Collection.extend({
url: '/tasks', // the REST url to retrieve collection data
model: Task // the models of which the collection consists of
});
var allTasks = new TasksAll();
Now i want to create a subset collection for e.g. tasks where task.status = 0:
var TasksTrash = new Backbone.Subset({
superset: allTasks,
filter: function(Task) {
return Task.isTrash();
}
});
var trashTasks = new TasksTrash();
Whereas inside the Task model, the method "isTrash" returns true if:
this.get('status') == 0
a) Are there any more resources on backbone.subset.js?
b) How do I implement above scenario?
c) Can I pass 'superset' and 'filter' options as params to the Backbone.Subset init function?
d) I looked into the backbone.subset.js code, when I 'reset' my parent Collection my subset Collections should be updated straight away, right?
PS: I'm fairly new to Backbone. Thanks for your help.
Looking at the source for backbone-subset, it looks as though there is a pre-initialization hook which you could utilize in order to make the 'sieve' or filter available as an option or argument:
https://github.com/masylum/Backbone.Subset/blob/master/backbone.subset.js#L50
As for providing parent as an argument, there is an outstanding patch to add that exact functionality:
https://github.com/masylum/Backbone.Subset/pull/5
With it, you can pass in parent as an option, if it is not an option the library will fall back to looking for it on the object Prototype