I have just started experimenting with building a website using node.js, and I am encountering an issue when organizing the models of my project.
All the real world examples I have found on the Internet are using Mongoose. This library allows you to define your models in a static way. So you can write this:
// models/foo.js
module.exports = require('mongoose').model('Foo', ...);
// app.js
mongoose.connect(...);
// some_controller_1.js
var Foo = require('./models/foo');
Foo.find(...);
// some_controller_2.js
var Foo = require('./models/foo');
Foo.find(...);
But since I don't want to use MongoDB, I need another ORM. And all the other ORMs I have found don't allow this. You first need to create an instance, and then only you can register your models. Also they don't seem to allow access to the list of registered models.
So I tried doing this:
// models/user.js
var registrations = [];
module.exports = function(sequelize) {
var result = null;
registrations.forEach(function(elem) {
if (elem.db == sequelize)
result = elem.value;
});
if (result) return result;
// data definition
var user = sequelize.define("User", ...);
registrations.push({ db: sequelize, value: user });
return user;
};
Which I can use this like:
// some_controller_1.js
var Foo = require('./models/foo')(app.get('database'));
Foo.find(...); // using Foo
But these small header and footer that I have to write on every single model file are a bit annoying and directly violate the "don't repeat youself" principle. Also, while not a huge issue, this is kind of a memory leak since the "sequelize" object will never be freed.
Is there a better way to do, which I didn't think about?
You can find an article about handling models with sequelize here: http://sequelizejs.com/articles/express#the-application
You basically just create a models/index.js as described here: http://sequelizejs.com/articles/express#block-3-line-0
Afterwards you just put your model definitions within files in the models folder as pointed out here: http://sequelizejs.com/articles/express#block-4-line-0
Related
I'm building a fairly large node.js client library and I'd like to be able to 'namespace' portions of it to keep it more organized. For example:
var client = new Client(config);
client.activities.get(activityId, function(activity) {
...
});
...
client.user.get(userId, function(user) {
...
});
I'm currently trying to do something like this in the module:
function Client(config) {
this.config = config;
}
Client.prototype.activities = require('./activities');
Client.prototype.user = require('./user');
module.exports = Client;
but when 'get' is called in the activities 'submodule', the "this" is for the module, of course, and not the outer Client function. Basically, both of the submodules need access to the outer configuration information ('config' in this example). What is the best practice around doing this in node.js?
I think a modeling issue with the plan you have is that new X() copies each property reference of X.prototype to the new item; for instance, X.myFunction is the exact same reference as X.prototype.myFunction; but does NOT create a new copy of myFunction.
It sounds like if activities is a part of a Client (the same way StreetAddress might be), you actually want activities to be a Class, not a module, and for Client to create a new instance of it when it's constructed. That's not really something that prototype is useful for. If I'm wrong, maybe you could show an example of a basic operation activities would be used for.
I have an object called NetFlowStorage that contains methods to access a specific elasticsearch index. My constructor looks like:
function NetFlowStorage() {
this.host = 'localhost:9200';
this.shards = '4';
this.replicas = '0';
this.index_name = 'flow_track2';
this.client = null;
}
Inside of the object I have a method called connect which, when called, will make the connection and store the elasticsearch client object in the this.client property (if there isn't one already there). This way all of the object methods can get access to the elasticsearch client by using this.client
First question, is this an appropriate pattern? If not, what is preferable?
Second question (and the one that drove me here), how would I mock calls to things like this.client.index({}) I'm just starting to mess around with unit testing and mocks under node/js so I don't really have a preference in terms of framework (currently using mocha/chai/sinon)
Full code is here if you want to see in more detail.
For something like this I would use dependency injection.
You want to decouple the NetFlowStorage class from the actual elasticsearch client:
function NetFlowStorage(esClient) {
this.host = 'localhost:9200';
this.shards = '4';
this.replicas = '0';
this.index_name = 'flow_track2';
// if you don't wanna share connections across several instances
// you can instantiate the client here otherwise you can pass the
// client instance
this.client = esClient; // or new esClient({ host: this.host })
}
This way you won't even need the elasticsearch as part of the node module and even share a connection across more than one instance (or not?)
This decoupling will also make it easier to mock the esClient as you would inject the mock elasticsearch client in the test itself.
I think that you should pass a config object and a connection object to the method.
So if you would use Jasmine for testing for example you could pass a spy
var client = {index:function(){}}
spyOn(client, 'index');
....
expect(client.index)toHaveBeenCalled();
and to pass it at some point with injection or a singleton to the SUT
I'm constructing a CRM application with a SPA structure, using BreezeJS and AngularJS, and I'm utilizing a dynamically-generated tabbed environment to display what I'll refer to here on out as modules. When a user clicks a side menu item, a new tab is created, and an HTML template (aka module) is loaded into the new content area. While some of the module types are to be opened only once at a time (a grid of Accounts), others, such as an Account Editor module, can be opened many times. That way, the user can modify many accounts at any given time throughout the day, but always be doing so from a single Accounts grid instance.
So far, I have a working system for the Account Editor module, in that I create a master Breeze EntityManager with appropriate configuration parameters (service name, save options, etc.) and then make a copy of it any time a new Account Editor is created, using masterManager.createEmptyCopy() (as per the code outlined at http://www.breezejs.com/documentation/multiple-managers#CreateMultipleManagers):
var serviceName = "/breeze/accounts";
var ds = new breeze.DataService({
serviceName: serviceName,
hasServerMetadata: true
});
var masterManager = new breeze.EntityManager({
dataService: ds,
saveOptions: new breeze.SaveOptions({ allowConcurrentSaves: false })
});
function createManagerCopy() {
// same configuration as master; no entities in cache.
var sandboxManager = masterManager.createEmptyCopy();
return sandboxManager;
}
Then, I call an EntityQuery, by passing in the copied EntityManager and entity Id (key), in order to get the appropriate Account and populate each open editor with apparently all the benefits of Breeze.
function queryAccountByKey(mgr, key) {
var keyPredicate = breeze.Predicate.create('id', 'eq', key);
var query = new breeze.EntityQuery('AccountsBase')
.expand('ContactsBase')
.where(keyPredicate);
var promise = mgr.executeQuery(query)
.catch(queryFailed);
return promise;
function queryFailed(error) {
console.log('Query Account by Id Failed', error);
return $q.reject(error); // so downstream promise users know it failed
}
}
There does not appear to be any conflict with the other open editors and their respective entity managers, so long I maintain the copied EntityManager in the module's Angular scope. Editing and saving this way is, well…a breeze! :P (sorry)
The problem I’m having is when I switch to another Angular route, such as a login screen, and then come back to the home screen. Due to the complexities of each tabbed module, any Account Editor module I had opened before the routing took place must be reloaded from stored settings (even if the layout is stored in the cache). However, then there are two Breeze entity managers handling that account. The effect is that saving changes for a single Account Editor now commits twice (or however many times you’ve gone away and back from the home layout).
What I’m trying to figure out is how to get access to a specific EntityManager instance, from the client, once I navigate back to the home layout and the original instance reference (in the scope, in my case) is lost. In other words, is there an EntityManager collection that I can query to reuse or delete an instance? If I can grab a manager instance by name or other Id, I can simply reassign it to my scope without removing and recreating it.
In the event that I do need to remove the EntityManager, I can’t find anything anywhere to describe implementing something like a destroy() method…just a clear() method, which only clears the entities and does not remove the EntityManager from the client. Of course, if I'm going about this all wrong, please advise as to a better approach. Hopefully I've explained things clearly enough that someone can lend a possible solution.
SOLUTION
So, thanks to PW Kad's answer, I was able to reuse the entity managers instead of deleting and re-creating them, by adding them to an empty object collection on the $rootScope when they're initialized (still using the createEmptyCopy() method outlined above). This allows for access throughout the Angular app, without polluting the global namespace. I had already implemented a unique ID to be associated with each tab - and thus the modules in the content areas - so I appended that ID to create a name, such as 'EM_' + instanceId, for storage in the $rootScope.entityManagers object. Later, I can retrieve the EntityManager instance in the $rootScope, using this ID, which is found in each Account Editor's Angular Controller.
Here's the new code:
// NEW: Add an 'entityManagers' object to the $rootScope of my main app module
angular.module('app', [])
.run(['$rootScope', function($rootScope) {
$rootScope.entityManagers = {};
$rootScope.entityManagers.count = 0;
}]);
// In the dataServices factory for the main app module
var serviceName = "/breeze/accounts";
var ds = new breeze.DataService({
serviceName: serviceName,
hasServerMetadata: true
});
var masterManager = new breeze.EntityManager({
dataService: ds,
saveOptions: new breeze.SaveOptions({ allowConcurrentSaves: false })
});
function createManager(instanceId) {
// make a copy of the above EntityManager (with no cached entities)
var sandboxManager = masterManager.createEmptyCopy();
// NEW: Save the EntityManager instance to the $rootScope
$rootScope.entityManagers['EM_' + instanceId] = sandboxManager;
$rootScope.entityManagers.count++;
return sandboxManager;
}
// In the event that you want to delete the EntityManager from the $rootScope
function deleteManager(instanceId) {
var manager = $rootScope.entityManagers['EM_' + instanceId];
manager.clear();
delete $rootScope.entityManagers['EM_' + instanceId];
$rootScope.entityManagers.count--;
}
// And lastly, inside any Angular controller
$scope.instanceId = '1234'; // Dynamically-assigned at runtime
$scope.init = function() {
var manager = $rootScope.entityManagers['EM_' + $scope.instanceId];
if (manager === undefined) {
$scope.entityManager =
entityManagerService.createManager($scope.instanceId);
} else {
$scope.entityManager = manager;
}
}
$scope.getEntityById = function(id){
//use $scope.entityManager here to query database via Breeze
}
While I would still like to know where BreezeJS keeps its EntityManager collection, I have a valid solution. I hope it helps someone!
It seems like a poor design choice to reinstantiate a new entity manager every time you change pages as in effect you are losing the caching mechanism and sharing the entities across pages but if you must do so you can always do so fairly trivially with something like -
var manager = {};
manager = manager ? manager.clear() : new breeze.EntityManager();
or
manager = manager ? (function () { delete manager; return new breeze.EntityManager(); })() : new breeze.EntityManager();
or many other ways really.
I would recommend not doing that though and just doing something like this -
var manager = {};
// Some route activation logic
if (!manager) {
manager = new breeze.EntityManager();
}
Edit
Well the short answer is I am not 100% sure how the breeze object in the global namespace is referencing the entity manager. I don't think there is a collection of entity managers that the breeze object keeps, but I may be wrong. I don't see why calling delete on the entity manager isn't working but this should do what you are trying to do -
Somewhere in either one of your closures or in the global namespace create an object called entityManagers. Example -
window.entityManagers = {};
window.entityManagers.count = 0;
Then tack the manager while it is created onto that namespace. There is probably a more dynamic way to do this just giving some pseudo-code -
window.entityManagers.createNewManager = function (name) {
window.entityManagers[name] = new breeze.EntityManager();
}
window.entityManagers.createNewManager('ManagerNumber1');
And then when you want to dispose of a specific instance just clear it then delete it. Get the instance you want either by a variable reference or if you can't do that for some crazy reason just grab it off the affected entity -
window.entityManagers.deleteManager = function (name) {
window.entityManagers[name].clear();
delete window.entityManagers[name];
}
window.entityManagers.deleteManager('ManagerNumber1');
For all intents and purposes, as long as there no other references to that instance of the manager in other modules / controllers / whatever than this should delete the entityManager from the world. I still don't fully understand the use case so take it with a grain of salt.
I defined mysql connection with all parameters necessary to app.js, how can make visible to other scripts in routes/ by default, without requiring or redefining mysql parameters, just using client.query(..)?
A pattern I use is to set up my db object in a module once and export it: (let's call it utils/mySQL.js)
//I haven't used real mysql in node so excuse the pseudo-syntax:
var db = require('mysql-driver-thingy');
db.connect('localhost', 'sqlport', options...);
db.otherSetupFunctions();
console.log("Finished db setup. You should only see this message once! Cool.");
module.exports = db;
And then I can require the db object everywhere I need it. Since requires are cached, this does't actually call the setup methods multiple times.
In app.js:
var db = require('./utils/mySQL.js');
...
In models/user.js:
var db = require('../utils/mySQL.js');
...
A final option, which isn't recommended, is to pollute the global namespace. This seems to be the answer you're really after:
//set up your db
...
// and now make it available everywhere:
global.client = db.client
You can now magically use the client object in all your modules, without even requiring it.
There are many reasons globals are bad, though:
If your code and other code define globals, they could conflict and overwrite each other.
It's hard to find where you defined the db/client object, etc.
You can inject mysql connection into other scripts like this:
app.js
var mysqlConnection = new Conection(params);
require('controller/main.js)(mysqlConnection);
main.js
module.exports = function(mysqlConnection) {
// You can access your mysql connection here
};
UPDATE:
You can inject several variables same way. Also you still can export methods from module if you need this:
app.js
var mysqlConnection = new Conection(params);
var news = require('model/news.js)(app, mysqlConnection);
news.list(function(err, news) {
// Do something
});
news.js
module.exports = function(app, mysqlConnection) {
var methods = {};
// mysql connection and app available from here
methods.list = function(cb) {
mysqlConnection.list(function(err, data) {
cb(err, data);
});
};
return methods;
};
my problem is not about "memory leakage", but about "memory purge" of node.js (expressjs) app.
My app should maintain some objects in memory for the fast look-up's during the service. For the time being (one or two days) after starting the app, everthing seemed fine, until suddenly my web client failed to look-up the object bacause it has been purged (undefined). I suspect Javascript GC (garbage collection). However, as you can see in the psedu-code, I assigned the objects to the node.js "global" variable properties to prevent GC from purging them. Please give me some clue what caused this problem.
Thanks much in advance for your kind advices~
My node.js environments are node.js 0.6.12, expressjs 2.5.8, and VMWare cloudfoundry node hosting.
Here is my app.js pseudo-code :
var express = require("express");
var app = module.exports = express.createServer();
// myMethods holds a set of methods to be used for handling raw data.
var myMethods = require("myMethods");
// creates node.js global properties referencing objects to prevent GC from purging them
global.myMethods = myMethods();
global.myObjects = {};
// omited the express configurations
// creates objects (data1, data2) inside the global.myObjects for the user by id.
app.post("/createData/:id", function(req, res) {
// creates an empty object for the user.
var myObject = global.myObjects[req.prams.id] = {};
// gets json data.
var data1 = JSON.parse(req.body.data1);
var data2 = JSON.parse(req.body.data2);
// buildData1 & buildData2 functions transform data1 & data2 into the usable objects.
// these functions return the references to the transformed objects.
myObject.data1 = global.myMethods.buildData1(data1);
myObject.data2 = global.myMethods.buildData2(data2);
res.send("Created new data", 200);
res.redirect("/");
});
// returns the data1 of the user.
// Problem occurs here : myObject becomes "undefined" after one or two days running the service.
app.get("/getData1/:id", function(req, res) {
var myObject = global.myObjects[req.params.id];
if (myObject !== undefined) {
res.json(myObject.data1);
} else {
res.send(500);
}
});
// omited other service callback functions.
// VMWare cloudfoundry node.js hosting.
app.listen(process.env.VCAP_APP_PORT || 3000);
Any kind of cache system (whether is roll-your-own or a third party product) should account for this scenario. You should not rely on the data always being available on an in-memory cache. There are way too many things that can cause in-memory data to be gone (machine restart, process restart, et cetera.)
In your case, you might need to update your code to see if the data is in cache. If it is not in cache then fetch it from a persistent storage (a database, a file), cache it, and continue.
Exactly like Haesung I wanted to keep my program simple, without database. And like Haesung my first experience with Node.js (and express) was to observe this weird purging. Although I was confused, I really didn't accept that I needed a storage solution to manage a json file with a couple of hundred lines. The light bulb moment for me was when I read this
If you want to have a module execute code multiple times, then export a function, and call that function.
which is taken from http://nodejs.org/api/modules.html#modules_caching. So my code inside the required file changed from this
var foo = [{"some":"stuff"}];
export.foo;
to that
export.foo = function (bar) {
var foo = [{"some":"stuff"}];
return foo.bar;
}
And then it worked fine :-)
Then I suggest to use file system, I think 4KB overhead is not a big deal for your goals and hardware. If you familiar with front-end javascript, this could be helpful https://github.com/coolaj86/node-localStorage