Develop Reference

How can I follow an _id in Mongoose.js to find an item and process it?

I am new to databases and these MongoDB/Mongoose.js async functions have annoyed the hell out of me over the last few hours. I have written and rewritten this bit so many times:
router.get('/districts', function(req, res) {
districtNames = [];
// I'm using mongoose-simpledb, so db.District refers to the districts collection
db.District.find(function(err, found) {
found.forEach(function(element) {
findParentProv(element, districtNames);
});
res.render('districts', {title: "Districts page", district_list: districtNames});
})
});
function findParentProv(element, namesArray) {
db.Province.findById(element.parent, function(err, found) {
console.log(found.name);
namesArray.push(element.name + " " + found.name);
});
}
I want to get all items in the districts collection, follow their parent field (which contains an ObjectID), find that item from the provinces collection and push both their names as a string into districtNames.
How should I do this?

Well, you do seem to be on the right track.
The one major issue I recognize in your solution is that after kicking off all the async queries for parents, you immediately return the (most likely empty) districtNames array, without waiting for the queries to finish.
This is indeed very annoying, and not surprisingly so. MongoDB is a non-relational DB, and so join operations like what you're trying to do aren't easy to get right.
The solution that would probably require the least fundamental changes to what you're doing would be to wait on all the queries before calling res.render. The most basic way to do this would be to check the length of namesArray/districtNames after pushing each element, and once you see it's gotten to the desired size, only then calling render. There are, however, more standardized ways of doing this, and I'd suggest looking into something like Async (specifically async.parallel) or a Promise framework such as Bluebird.
Now, another approach to solving this problem is de-normalizing the data. For someone with a relational background this probably sound appalling, but in Mongo it might actually be a valid solution to just include the province names along with their IDs in the districts collection, in which case your one initial query should be sufficient.
Another approach, which might be suitable if you're dealing with relatively small collections, would be to run 2 queries, 1 for all the districts and 1 for all the provinces, and do the correlation in-app. Obviously, this isn't a very efficient solution, and should definitely be avoided if there's any chance the collections contain, or will in the future contain, more than a handful of objects.

Best bet moving forward is to use ES6 Promise patterns to help with your callback patterns..
suggested modules:
lodash [optional] has a lot of useful methods, not needed here, but you may need, for example _.flatten, or _.assign
i-promise will give you a native Promise (node 0.11.3+) or a scripted implementation
es6-promise is the fallback for i-promise to use
promisify-patch is an inline promisify for specific methods.
Install the modules required for your use (in this example).
npm install --save es6-promise i-promise promisify-patch
Use Promise pattern with your example.
require('promisify-patch').patch();
var Promise = require('i-promise')
;
//returns a promise to resolve to your list for display
function getDistricts() {
//gets all of the db.District
return db.District.find.bind(db.District).promise()
//after districts retrieved
.then(function(districts){
//resolve an array of promises, will return an array of results
return Promise.all(districts.map(getDistrictProv)); //map each district via getDistrictProv
});
}
//returns a promise to resolve a specific district/province name
function getDistrictProv(district){
return db.Provice.findById.bind(db.Province).promise(element.parent)
.then(function(province){
return district.name + ' ' + province.name;
});
}
...
//express handler
router.get('/districts', function(req, res, next) {
//get the district names
getDistricts()
//then process the rendering with the names
.then(function(names){
res.render('districts', {title: "Districts page", district_list: names});
})
//if there was an error in the promise chain
// pass it along, so it can be handled by another express plugin
.catch(next)
});
Disclosure: I made i-promise and promisify-patch to make situations like this easier to convert node-style callbacks into promise chains.
NOTE: If you are creating general purpose libraries for Node or the Browser that are not flow-control related, you should at least implement the node-style callback implementation.
Further, you may wish to look into co, koa for using generators as well.
The question seemed to be how to control the flow of data, in which promises are likely the best answer. If your issue is trying to fit non-relational data into a relational box or vice-versa, may want to re-evaluate your data structure as well...
http://blog.mongodb.org/post/88473035333/6-rules-of-thumb-for-mongodb-schema-design-part-3
You should probably have some key data for parents/children replicated to those affected documents in other collections. There are configuration options via Mongoose to support this, but that doesn't mean you should avoid the consideration.
If you do many/large join operations like this it will negatively affect your performance. This isn't meant to be a religious comment only that MongoDB/SQL or other SQL vs. NoSQL considerations should be made depending on your actual needs.
The data in question seems to be highly cacheable data that may well be better with a relational/sql database.

the best pattern to keep references to global collections in Backbone

I'm developing an application in Marionette, but the topic refers to raw Backbone as well.
In my app, I've got many collections. Among them, there are 3 important ones, that are used all over the application (e.g. user data, such as names, which is displayed in most of views). The main question is: what is the best way (a good pattern to follow) to keep references to collections in Backbone/Marionette? I came up with few ideas below:
should I attach them as attributes of the Application object? If so, I'd have to pass the reference to Application object to all views, then views to their subviews, subviews to subsubviews, etc. But this seems a lame and ugly design.
instead, I could pass each collection separately, but this seems even worse solution, because I can't predict which (sub)view will ever need a collection. Keeping those nested references in order would be much more difficult than passing the Application object which I can pass once, always.
the is also a possibility to import the Application as a singleton. I'm using requireJS, most of the modules return constructors now (constructors of views, models and collections). Now the app module returns Application constructor, but instead, it could return the Application object. Then if a view requires to display some data from the collections, it could just require the app module and that's all.
finally, basing on the previous point, I thought I could create a plain map of collections and make it a singleton just as before. This is only to disable all views to have access to Application object, which still seems a bad pattern.
Please, suggest what you think is the best (commentson points above are welcome as well). I just need a good design pattern here. As far as I know, Marionette docs doesn't suggest anything here.

I follow the suggestions made in David Sulc's book Backbone.Marionette.js: A Gentle Introduction. (also the next book on goes into how to then sturcutre the same project with require https://leanpub.com/structuring-backbone-with-requirejs-and-marionette)The code examples he uses are available on github so you could look at the final example which to get an idea of what he does if you didn;t want to buy the book but i really recommend it as it really helped me with how to structure my projects.
To start I have setup an Entities module. The structure of the files also follows this I have an Entities folder which has separate entities.
Each Entity file concerns all actions and stucture of that particular entity. I like this approach as I when i want to edit an entities strcuture or method of getting data from the server i only need to go to one place to make this change.
Interactions with entity are handled through marionettes req/res system. In this way you can expose a handler to the rest of your app but they don;t need to be concerned with how that req gets handled as long as it brings back the required response.
Here is an example of one of my entities to show what i mean - My app needs a collection called positions at various stages so this is something that is loaded early in the app then is available through-out it's life-cycle.
//i am using require so i define my app to use
define(["app", ], function(MyApp) {
//All of this will be added to the Entities module so if i want direct access
//i can go Myapp.Entities.whateverEntityIhaveDeclared
MyApp.module("Entities", function(Entities, MyApp, Backbone, Marionette, $, _) {
//model (nothing new here)
Entities.Position = Backbone.Model.extend({
urlRoot: "api/positions",
defaults: {
name: "",
}
});
//collection again nothing new here
Entities.PositionCollection = Backbone.Collection.extend({
url: "api/positions",
model: Entities.Position,
comparator: "name"
});
//an init function to attach a position collection onto Entities module so it can be available throughout my app
var initializePositions = function() {
if (Entities.positions === undefined) {
Entities.positions = new Entities.PositionCollection();
}
};
//
var API = {
//returns a jquery deferred promise so that this request can easily be handled async
loadPositionEntitiesRemote: function() {
//init positions make's sure i have the collectoin avaliable if it
//has not yet been defined
initializePositions();
//setup defer object
var defer = $.Deferred();
//I actually use a custom sever object here when dealing
//with ajax requests
//but because my app always go through this API i can
//easily swap out how the collection is retrieved.
// Here is an example using backbones fetch
Entities.positions.fetch({
success: function() {
defer.resolve();
},
error: function(data) {
defer.reject(data);
}
});
//setup promise to return
var promise = defer.promise();
return promise;
},
//get the positions collection from here i could
//directly access the attributes or add to the collection
refrencePositionEntities: function() {
initializePositions();
return Entities.positions;
},
//get a position from the collection based on id
//
getPositionEntity: function(positionId) {
initializePositions();
return Entities.positions.get(positionId);
}
};
//setup handlers for the app to use
MyApp.reqres.setHandler("position:entities:remote", function() {
return API.loadPositionEntitiesRemote();
});
MyApp.reqres.setHandler("position:entities:refrence", function() {
return API.refrencePositionEntities();
});
MyApp.reqres.setHandler("position:entity", function(id) {
return API.getPositionEntity(id);
});
MyApp.reqres.setHandler("position:entity:new", function(position) {
return new Entities.Position(position);
});
});
return;
});
now to use this in my app here is an example of how it can now be used
someFunction: function(){
//so early in the app i will get the positions
var positionPromise = MyApp.request("position:entities:remote");
$.when(positionPromise).done(function() {
//do what ever as data has been loaded
}).fail(function(data){
//something failed so handle here might through up an error page but up to you
}).always(function(){
//something to always do no matter if fail or sucess
});
}
anotherFunction: function(){
//later in the app in another controller i might to get the collection
// I could also get it through MyApp.Entities.positions but i rather use the
// API set up so if i ever decided i want to add so checks or something in
// when retrieving the collection its super easy
var positionsCollection = MyApp.request("position:entities:refrence");
}
Not sure if i've done a great job explaining this but if you are looking for a ideas on good design in marionette check out the book as it explains this a lot better than i have just done

Database inheritance model with Node.js

Using Node.js and a SQL-like database, what are my options when it comes to implementing database inheritance model? I looked around and it seems that due to popularity of noSQL database in Node.js, database inheritance is overlooked.
I'm trying to accomplish a very simple thing: extend a Users table based on a few business rules. For simplicity, I'd want to add Reader and Moderator tables and classes whose properties would differ. This is something that would be relatively easy to accomplish in .NET using NHibernate. Can I achieve the same result without using sticking everything in the same table? If not, what are the best alternatives?
Btw, I'm using Node.js with PostreSQL via Sequelize.

I have not seen an ORM for SQL databases that supports inheritance yet. Even then, I believe there are other opportunities thanks to the nature of JS that you might enjoy (I sure do):
Using jugglingdb (the one I would recommend), coupled with underscore.js (something most node.js programmers should pay attention to) you can do something like the following:
var user_common_properties = {
name: String,
bio: Schema.Text,
approved: Boolean,
joinedAt: Date,
age: Number
};
var User = schema.define('User', user_common_properties);
var Moderator = schema.define('Moderator', _.extend(user_common_properties, {
power : String,
level : Number
}));
var Reader = schema.define('Moderator', _.extend(user_common_properties, {
lurker : Boolean
}));
While I agree it's not the same, there are no reasons to consider this bad :) I think it is very JavaScript-like.
Also, if you want to re-use methods/validations, you can! Just assign it to them (re-use the functions) or if you want to re-use a bunch of functions, add them to an object and then extend the Reader and Moderator objects. For example...
var repeated_methods = {
first : function () { },
second: function () {}
};
//I want to reuse this for both User and Moderator but not for Reader...
User = _.extend(User, repeated_methods);
Moderator = _.extend(Moderator, repeated_methods);
This is very flexible and allow for has-a relationships (behaviors), which is great for horizontal reuse.

How to integrate backbone.sync with jstorage?

I'm a newbie javascript developer (first post here!) and been recently trying to play with backbone.sync. I've been reading through the todo example and notice that it uses backbone-localstorage. My feeling is that backbone-localstorage was just a quick implementation that the author used for demo purposes. I also noticed that the keys are randomly generated, whereas I would like something that would allow me to name my own key values.
I've been looking at jstorage (www.jstorage.info), and would really appreciate some pointers (preferably with code samples) on how to integrate it with backbone.js. I imagine backbone.sync would need to be overriden somehow as is done in backbone-localstorage.
Alternatively, what localStorage plugin/wrapper would you recommend, and how would it integrate with backbone?
Thanks for help in advance.

My feeling is that backbone-localstorage was just a quick implementation that the author used for demo purposes.
Exactly right. I'd say that the implementation is fine for most things, but what support you can get for it is probably minimal.
I also noticed that the keys are randomly generated, whereas I would like something that would allow me to name my own key values.
This isn't really correct. I assume you are referring to this:
// Add a model, giving it a (hopefully)-unique GUID, if it doesn't already
// have an id of it's own.
create: function(model) {
if (!model.id) model.id = model.attributes.id = guid();
this.data[model.id] = model;
this.save();
return model;
}
Here, what's happening is that when a create is called on a model, it tries to determine if an id has been set on the model, and if no such id is provided, then the guid function is used to build one. This isn't setting a key randomly, it's fulfilling the requirement that every saved model should have an id (by assigining a random one).
Modifying the backbone-localstorage.js code should be fairly trivial. Let's look at the store constructor as an example
Before:
var Store = function(name) {
this.name = name;
var store = localStorage.getItem(this.name);
this.data = (store && JSON.parse(store)) || {};
};
The only thing we need to update here is the call to localStorage:
After:
var Store = function(name) {
this.name = name;
//Notice here, that jStorage gives us the option to define a default.
var store = $.jStorage.get(this.name, {});
this.data = store;
};
Similarly simple is the save function:
Before:
save: function() {
localStorage.setItem(this.name, JSON.stringify(this.data));
}
again, simply update the call to localStorage:
After:
save: function() {
$.jStorage.set(this.name, this.data);
}
Update: jStorage handles all the JSON.stringify, and JSON.parse for you, so I've updated the above code to reflect that.
Easy peasy, right!
Anyways, this is a great exercise, and I wish you the best of luck. I hope that I've been able to help you out.
P.s. One thing that bothers me about the localStorage plugin is that it uses model.attributes.xxx, where it's better to use model.get("xxx"). This way, if they ever change the internals of model change, your code won't break.
P.p.s. as far as how to generate guids, and weather a random guid is appropriate for you, depends upon your domain. With the TODO example, there is no backing database to sync with, so any old guid would do the trick, but in many other cases, you may be using html5 storage to create an offline version of your app, so id's need to be compatable with the db. In these cases, using a random guid is probably not that great an idea, but the one provided by the backbone-localstorage plugin is designed so that it isn't likely to collide with your db's actual id's, so it's not _that bad. There isn't really a right answer here, so do what makes sense for your domain.

You should probably look first at the local storage plugin that backbone.js provides. There are plenty of code samples that should give you an idea of how you would swap in jstorage as the underlying storage mechanism. See it here:
backbone-localstorage.js

Model-View-Controller in JavaScript

tl;dr: How does one implement MVC in JavaScript in a clean way?
I'm trying to implement MVC in JavaScript. I have googled and reorganized with my code countless times but have not found a suitable solution. (The code just doesn't "feel right".)
Here's how I'm going about it right now. It's incredibly complicated and is a pain to work with (but still better than the pile of code I had before). It has ugly workarounds that sort of defeat the purpose of MVC.
And behold, the mess, if you're really brave:
// Create a "main model"
var main = Model0();
function Model0() {
// Create an associated view and store its methods in "view"
var view = View0();
// Create a submodel and pass it a function
// that will "subviewify" the submodel's view
var model1 = Model1(function (subview) {
view.subviewify(subview);
});
// Return model methods that can be used by
// the controller (the onchange handlers)
return {
'updateModel1': function (newValue) {
model1.update(newValue);
}
};
}
function Model1(makeSubView) {
var info = '';
// Make an associated view and attach the view
// to the parent view using the passed function
var view = View1();
makeSubView(view.__view); // Dirty dirty
// Return model methods that can be used by
// the parent model (and so the controller)
return {
'update': function (newValue) {
info = newValue;
// Notify the view of the new information
view.events.value(info);
}
};
}
function View0() {
var thing = document.getElementById('theDiv');
var input = document.getElementById('theInput');
// This is the "controller", bear with me
input.onchange = function () {
// Ugly, uses a global to contact the model
main.updateModel1(this.value);
};
return {
'events': {},
// Adds a subview to this view.
'subviewify': function (subview) {
thing.appendChild(subview);
}
};
}
// This is a subview.
function View1() {
var element = document.createElement('div');
return {
'events': {
// When the value changes this is
// called so the view can be updated
'value': function (newValue) {
element.innerHTML = newValue;
}
},
// ..Expose the DOM representation of the subview
// so it can be attached to a parent view
'__view': element
};
}
How does one implement MVC in JavaScript in a cleaner way? How can I improve this system? Or is this the completely wrong way to go, should I follow another pattern?

There are at least a couple of established and usable MVC frameworks for JavaScript JavaScriptMVC and pureMVC. There are probably more. I've used JavaScriptMVC for browser based and Air apps and keep coming back to it - it has its problems but I've found it to be quite useful.
There are other solutions too, have a look at Sammy, a new thing I've heard good things about. I haven't used myself but intend to try soon. I don't know enough about it to describe it properly, but to me it seems like a front controller which works on routes, a templating system and ReSTful data stores. I'm not sure if it is MVC but has similar ingredients.
I have to disagree with mway's answer. MVC may be a bit diferent to implement in JavaScript but its benefits are very important to organising this mess. The design patterns usually associated with OO languages don't go out the window just because js isn't class based.
I would say that MVC is more suitable for JavaScript apps than for request based (server side) applications. Those objects can hang around for a while in a one page JavaScript app - minutes if not hours - and having a well organised way of organising their interaction will make your code much more robust and easy to deal with. There are books on the subject.
A couple of other points regarding the code you posted.
The view objects have responsibility for applying event listeners to DOM elements. This is the controller's job. The view just renders the HTML - the controller listens for the events and acts accordingly.
Your models seem to know your views. The model layer should have minimal knowledge of the view layer (perhaps being registered as observers). Keep your model clean and to the point, I mean the business point - business logic. In js apps you may just be proxying for a sever side model layer but it is important for your sanity to keep your model to the business logic and nothing else. Application logic is the controllers job

To be honest, MVC isn't well-suited for Javascript. It can support the basic fundamentals of the design, sure - you can create pseudoclasses to act as controllers or models, support basic inheritance, and you can have it manipulate or create any number of DOM elements, but there's a price that you pay for that - in overhead, accessibility, and usability.
In my opinion, I consider Javascript more of an augmentation - the KISS mentality exists for a good reason. If you're interested in better ways to organize your code, there's always the option of packaging related functionality into modules (sic) and abstracting out portions as appropriate. For example, creating a factory to do more complex AJAX request management, or a pseudoclass to handle processing of similar types of data. Using a standard base function for controllers, another for models, etc, as prototypes for new instances of those objects can accomplish similar functionality... but again, it's sort of going against the grain of Javascript.
However, if you're stuck on the MVC idea just for the sake of structure, consider something like the following:
;(function(window, $) {
/**
* Event Object
* A quick description goes here.
**/
var Events = window.Events = {
'bindTrackables': function() {
$('a.trackable').live('click', function() {
if(!_gaq)
_gaq = [];
_gaq.push(['_trackPageview', '/ajax/foobar']);
});
},
'bindSomeEvent': function() {
// etc
}
};
/**
* Data Cache
* I'll need to remember stuff later, so I store it here
**/
var Cache = window.Cache = {
'data': {},
'store': function(key, value) {
Cache.data[key] = value;
},
'fetch': function(key) {
return Cache.data[key];
}
};
/**
* Request Object
* Stores native AJAX requests for later use
**/
var Request = window.Request = {
'current_requests': [],
'send': function(url, type, data, callback) {
Request.current_requests.push($.ajax({
'url': url,
'type': type,
'data': data,
'callback': callback
}));
},
}
// add some private logic here
})(window, jQuery);
It's extremely basic, but you get the idea. Modular code is key... in JS, this is more important than forcing your application (or the language) to fit a certain style.