tl;dr:
Is it possible to reinforce / guarantee (using a library or something else) that a .js file will only have access to a specific subset of modules?
For instance: If the .js file is on the folder /controllers it's only allowed to have access to specific global modules (ex: express) and import from specific folders (ex: ../repositories). Anything outside this list is not allowed
Rationale behind:
In a large project with many developers it is almost inevitable that someone will invade layers. For example: within an action of an MVC controller, access the database via db driver / SQL without going through a repository or a more sofisticated abstraction.
const express = require('express');
const router = express.Router();
const pg = require('pg');
const path = require('path');
const connectionString = process.env.DATABASE_URL;
router.get('/api/v1/todos', (req, res, next) => {
const results = [];
pg.connect(connectionString, (err, client, done) => {
const query = client.query('SELECT * FROM items ORDER BY id ASC;');
query.on('row', (row) => { results.push(row); });
query.on('end', () => {
done();
return res.json(results);
});
});
});
This very basic example, just to illustrate the problem. In fact, in a multi layered / multi modules system, things are actually a bit more complicated and it's quite common to see invalid dependency graphs appearing as the project grows.
I know that part of the problem is solved with team training, but I wonder if there is any way to systematically reinforce this. Maybe something close to no-restricted-imports or no-restricted-modules on ESLint.
edit:
ESLint references.
Related
There are two 2 files with routes.
user.route.js
router.get('/user/:id');
user-transaction.js
router.get('/user/transaction');
My architecture looks like this:
src/
user/user.route.js
user-transaction/user-transaction.route.js
I connect all my routes in the project in the following way:
const dirs = await recursive(dir);
dirs.forEach((res) => {
if (res.includes('route')) {
console.log(res);
const path = `#modules/${res}}`;
router.use(require(path));
}
});
The problem is that Express doesn't see the /user/transaction route because it connects later than /user/:id and thinks /user/transaction refers to /user/:id.
It seems logical to just move the route higher, but I connect the routes dynamically. That is, it simply takes the files in order and connects the routes in the same order. Is there a solution to this problem or will I have to manually connect all the routes?
I am trying to utilize an MVC pattern for express. I am modularizing routes and trying to declare the express router in only the server entry file and nowhere else.
My current issue is when exporting my appRoute function in my main routes file (see below), the first route (user route), returns users. I have another route called game that is exported in the same function but it still returns users instead of games.
Both routes have a controller function called getAll that gets different data from different tables.
If I try and visit the route: http://localhost:8000/user/getAll, it returns all users just fine.
If I try and visit the route: http://localhost:8000/game/getAll, it still returns all users even when they're different routes...
If I were to flip the order of users and games in the main routes file where game is first and user is second, users starts to return games. It's like the second route mimics the first route.
This may be something simple, but any help I will appreciate.
My code is as shown below
server entry point (index.js)
const app = express();
const router = express.Router();
const bootstrap = require('./src/bootstrap');
bootstrap(app, router);
I am passing on the app and router instance to my bootstrap file where all routes will be exported to.
bootstrap.js (this file gets all exported routes and uses them within the app)
const { appRoute } = require('./routes');
module.exports = (app, router) => {
return app.use('/', appRoute(router));
};
I am passing on the router instance to my main routes file where all routes are exported from.
Main routes file (index.js): this file requires all routes and uses them within the router instance. I think this might be where my issue is but I am a little stuck on how I might fix it.
const { userRoute } = require('./userRoute');
const { gameRoute } = require('./gameRoute');
exports.appRoute = (router) => {
router.use('/user', userRoute(router));
router.use('/game', gameRoute(router));
return router;
};
Game route files (index.js): returns all users instead of games
const { gameController } = require('../../controllers');
exports.gameRoute = (router) => {
return router.get('/getAll', gameController.getAll);
};
Any help is greatly appreciated. If there is any clarification needed please let me know.
I think you need to create a separate router for game and user.
See the express.Routing section and birds.js example here
I am working on installing multiple databases in one server. To start this process, I am assigning one database in the index.js export object such that I will be able to add another DB as I build up the system. Below is the description of the issue.
When I run the server.js file, the .sync function for the database db1 syncs Table1 and Table2 and then begins listening on its port. Its clear the server.js file is receiving the module.exports = db through the var db = require('./models'). However in the file models/Table1.js the server throws an error TypeError: Cannot read property 'Table1' of undefined. I did check the console output for db in the models/Table1.js and it's empty, so its clear module.exports = db is not being accessed in this model.
Can someone provide a solution to correct this issue?
The partial code for the Table1.js model and other files listed above is below:
models/Table1.js
var db = require('./.')
[...]
var new_instance = db.db1.Table1.build({...})
server.js
var db = `require('./models')`
[...]
db.db1.sync(function(err) {});
models/index.js
var sq = new Sequelize(dbname, user, password, config);
db['db1'] = {
Sequelize: Sequelize,
sequelize: sq,
Table1: sq.import(__dirname + '/...'),
Table2: sq.import(__dirname + '/...')
}
module.exports = db;
The module.exports = db is being accessed, but you have created a cyclic dependency between models/index.js and models/Table1.js due to the fact that in the Table1.js model you require index.js file and in the index.js file you perform sequelize.import() call which performs require(path) to include the model definition, so in your case it calls require('./Table1.js') - there comes the cyclic dependency.
You should consider putting the var new_instance = db.db1.Table1.build({...}) somewhere else in order to avoid this situation. Files with model definitions should be only used to create the Model definitions, try to avoid performing additional operations.
Take a look at this question - How to deal with cyclic dependencies in Node.js to find out how you can deal with this kind of situations.
EDIT
According to your comment let's consider simple example with below structure
- script.js
- models
- index.js
- Table1.js
In script.js
var db = require('./models');
var new_instance = db.db1.Table1.build({...});
It can be done if you would not require the script.js inside any of the files placed inside models directory.
In a node application it's common that you start loading some modules like this:
var express = require('express')
, app = express()
, mongodb = require('mongodb')
, http = require('http');
Now, let's say you have a routes.js file which looks like this:
module.exports = function(app) {
app.get('/', function(req, res) {
res.render('homepage');
});
// other routes
};
So when you call that routes.js file, you pass the app var:
require('./routes')(app);
My question is: what is happening there? Is there resource consumption when passing the app variable to the routes module? I know that node.js caches modules, but I would like to know how that affects variables passed between them and I wonder if the following approach is efficient:
Let's start loading some modules, but let's do in a different way:
var _ = {};
_.express = require('express');
_.app = _.express();
_.mongodb = require('mongodb');
_.http = require('http');
Routes.js:
module.exports = function(_) {
_.app.get('/', function(req, res) {
res.render('homepage');
});
// other routes
};
Call to routes.js:
require('./routes')(_);
Obviously the _ variable will be a large one, but it will include anything you may need in any module. So I wonder if the size of the passed variable affects performance, in which case it would be just stupid to pass more data than needed.
I seek for achieving the best achievable performance in my applications while I try to simplify things when writing code, so any advice that may help with this, or any explanation about how this works behind the scenes in node.js will be appreciated.
Thanks in advance.
See here.
tl;dr: objects are passed by reference, not by value. So you are not expanding your memory consumption when you pass the same object multiple times to multiple modules.
I'm in early stages of a node.js project, and I'm looking to improve the overall app organization. In the past I worked with Symfony2 (PHP) and now I code a lot in Angular, both of which relly heavily on DI. So, I really like the idea of applying the same principles in my node.js project.
I know the existence of packages like Rewire, but for now I'll like to try the DI approach. The issue is, how to achieve an equilibrium to keep the lightweight feeling that gives working with node with the solidity of a well tested dependency injected app (I know that well tested is what gives the solidity ;-)).
Node modules
One of the issues, would be how to manage the external modules, what to do if some object needs fs module? As Vojta Jina (from AngularJS) states in this article:
So the best way that works for me right now is something like this: Modules are stateless. They only contain definitions of classes/functions/constants.
So, I suppose that I would have to inject everything:
function Foo(fs) {
this.fs = fs;
}
Foo.prototype.doSomething: function () {
// this.fs...
};
module.exports = Foo;
Somewhere:
var fs = require('fs');
var Foo = require('./Foo');
var foo = new Foo(fs);
foo.doSomething();
Express
Since Express uses apply() to call the handlers the context is lost and we can't use this. So we're left with these:
// foo.js
function Foo(fs) {
this.fs = fs;
}
Foo.prototype.index = function () {
var self = this;
return function (req, res, next) {
// self.fs...
};
};
module.exports = Foo;
// bar.js
module.exports.index = function (fs) {
return function (req, res, next) {
// fs...
};
};
// app.js
var express = require('express');
var fs = require('fs');
var app = express();
var Foo = require('./foo');
var foo = new Foo(fs);
var bar = require('./bar');
app.get('/foo', foo.index());
app.get('/bar', bar.index(fs));
So...
Has someone taken this approach? What about the use of DI frameworks? (like di.js) And how to keep the experience lean? All ideas are welcome. Thanks!
You have some good thoughts to which I'd like to add:
Having stateless modules will help you to scale your app horizontally. If all state is in a database it will be easy to run multiple node.js instances in parallel.
I also prefer to inject everything. Otherwise the time will come when I would like to write a unit test and it gets hard because I have a hardcoded (not injected) dependencies I can't mock.
To keep this lightweight feeling when working with node you need an approach for dependency injection that doesn't add too much complexity. Your express example above reminds me of a talk by Vojta Jina in which he makes an important point about the wiring part of dependency injection. (Watch minute 3:35 to 8:05) I can't explain it any better than Vojtja does in his talk but basically he says that we need a di framework that takes care of the wiring (what is injected into what). Otherwise the code we manually write to set up the wiring won't be maintainable. Also each unit test would need such wiring code. And that IMO is where manual dependency injection is not an option anymore.
When you use a di framework (or a di container as many people say) the basic idea is that each individual module states which dependencies it requires and through which id it can be required by other modules. Then the di framework can be invoked to initialize the module that serves as an entry point (e.g. app.js) and the framework will look up all dependencies and takes over the hard work of injecting the appropriate module instances.
There are many di frameworks for node.js to which I'd like to add my own: "Fire Up!" If you would use it your example would look like this:
foo.js
// Fire me up!
function Foo(fs) {
this.fs = fs;
}
Foo.prototype.index = function () {
var self = this;
return function (req, res, next) {
// self.fs...
};
};
module.exports = {
implements: 'foo',
inject: ['require(fs)'],
_constructor: Foo
};
bar.js
// Fire me up!
module.exports = {
implements: 'bar',
inject: ['require(fs)'],
factory: function (fs) {
return {
index: function (req, res, next) {
// fs...
}
};
}
};
app.js
// Fire me up!
module.exports = {
implements: 'app',
inject: ['require(express)', 'foo', 'bar']
};
module.exports.factory = function (express, foo, bar) {
var app = express();
app.get('/foo', foo.index());
app.get('/bar', bar.index);
};
index.js
var fireUpLib = require('fire-up');
var fireUp = fireUpLib.newInjector({
basePath: __dirname,
modules: ['./lib/**/*.js'] // foo.js, bar.js, app.js are on this path
});
fireUp('app'); // This is where the injection is kicked off.
When running node index.js you get the following output:
fireUp# INFO Requested: app, implemented in: lib/app.js
fireUp# INFO |-- Requested: require(express)
fireUp# INFO |-- Requested: foo, implemented in: lib/foo.js
fireUp# INFO |-- Requested: require(fs)
fireUp# INFO |-- Requested: bar, implemented in: lib/bar.js
fireUp# INFO |-- Requested: require(fs)
If this looks worth trying out you might be interested in the Getting Started section which shows an example based on express.
Hope that helps!
You can check https://www.npmjs.com/package/plus.container
This is close to DIC in Symfony