Develop Reference

Difference between require('module')() and const mod = require('module') mod() in node/express

I have two files: server.js and db.js
server.js looks as such:
...
const app = express();
app.use('/db', db());
app.listen(3000, () => {
console.log('Server started on port 3000')
});
...
and db.js as such:
...
function init() {
const db = require('express-pouchdb')(PouchDB, {
mode: 'minimumForPouchDB'
});
return db;
}
...
This works just fine, and I am able to reach the pouchdb http-api from my frontend. But before, I had const PouchDBExpress = require('pouchdb-express'); in the top of db.js, and the first line in init() looked like this; const db = PouchDBExpress(PouchDB, {. This gave an error in one of the internal files in pouchdb saying cannot set property query on req which only has getters (paraphrasing).
So this made me copy the exaples from pouchdb-servers GitHub examples which requires and invokes pouched-express directly, and everthing worked fine. Is there an explanation for this? I'm glad it works now, but I'm sort of confused as to what could cause this.

The only difference between:
require('module')()
and
const mod = require('module');
mod();
is that in the second case, you retain a reference to the module exports object (perhaps for other uses) whereas in the first one you do not.
Both cases load the module and then call the exported object as a function. But, if the module export has other properties or other methods that you need access to then, obviously, you need to retain a reference to it as in the second option.
For us to comment in more detail about the code scenario that you said did not work, you will have to show us that exact code scenario. Describing what is different in words rather than showing the actual code makes it too hard to follow and impossible to spot anything else you may have inadvertently done wrong to cause your problem.

In require('module')(), you don't retain a reference of the module imported.
While in const mod = require('module'); mod(), you retain a reference and can use the same reference later in your code.
This problem might be due to some other reason like -
Are you using a some another global instance of the db, and your code works in the given case as you are making a local instance
Some other code dependent scenario.
Please provide more details for the same

How to mock an external dependency method callback params in Nodejs?

Let's say that I have the following:
lib/modules/module1.js
var m2 = require('module2');
module.exports = function(){
return {
// ...
get: function(cb){
m2.someMethod(params, function(error, data){
if(error){
cb(error)
}
cb(null, data)
})
},
// ...
}
}
Now let's say that I have a set of tests in another dir, e.g. tests/testModule1.js. From this file, I create an instance of module1 to peform some tests.
I would like to mock the objects passed by the m2.someMethod to it's callback function (not the cb function), from the file testModule1.js.
I've looked into Sinon.js, but I couldn't figure a way to do this. Actually, I that even possible?
Thanks.

You could use something like proxyquire, but I'm not a fan of modifying the built in require.
Personally, I would suggest rewriting your code to use dependency injection:
module.exports = function(m2){
return {
// ...
get: function(cb){
m2.someMethod(params, function(error, data){
if(error){
cb(error)
}
cb(null, data)
})
},
// ...
}
}
Note that I moved m2 to be a parameter in your exported function. Then somewhere else (app, or main, or whatever), you could do this:
app.js
var module1Creator = require('module1');
var module2 = require('module2');
var module1 = module1Creator(module2);
Then when you need to test it...
testModule1.js
var module1Creator = require('module1');
// inject the "fake" version containing test data, spies, etc
var module2Mocked = require('module2mock');
var module1 = module1Creator(module2mocked);

I would normaly agree that about changing the design and, as suggested by #dvlsg, DI would be my choice as well.
However, the project I'm working on is already on the move, and has a considerable size. By doing this change would imply in a huge manpower cost that, in this case specifically, might not be worth doing it.
As a solution, I've realized that once you do a require('someModule'), the someModule is loaded and it's stored as a singleton, in some sort of global cache (I don't fully understand this mechanism, but I'll look into it), and doesn't matter if you require('someModule') from another file, you will receive the cached version.
So, if in lib/modules/module1.js I do require('module2'), module2 is loaded and stored in this cache and, I can require('module2') and mock it in tests/testModule1.js. This will reflect when the get() from lib/modules/module1.js is called.
For that, I used Sinon.js to create the mocks in the test files.
The procedure above actually solved my problem, in a way that I didn't have to change the whole design, and I was able to do the tests. Thats why I'm posting this as an answer. However, I'll not set this as the correct answer here because, as I said before, I don't fully understand this mechanism and change required modules is not a good practice.
I would like to see what other devs have to say about this and, if the discussion leads to acceptance, I'll ultimately set this as correct.

Odd behavior of fs.readdirSync

I use nodejs 5.5.0, and pm2 1.0.1. I run the script with:
npm start
And in package.json, it is defined as:
"scripts": {
"start": "pm2 start bin/www --watch"
},
I have this simple script that returns a list of videos and it works:
var express = require('express');
var router = express.Router();
router.get('/', function(req, res, next) {
const fs = require('fs');
shows = fs.readdirSync(`public/videos`);
res.render('index', shows);
});
module.exports = router;
But I observed something interesting... If I make the below modification, it no longer works, and returns an empty list:
//shows = fs.readdirSync(`public/videos`);
res.render('index', fs.readdirSync(`public/videos`));
To make it more interesting, the following works:
shows = fs.readdirSync(`public/videos`);
res.render('index', fs.readdirSync(`public/videos`));
It almost seems to me, if there is some sort of delay or caching, fs.readdirSync works fine, but not if it is called straight on the same line.
Did I miss something simple?

Somebody correct me if I'm wrong..
But I believe it is because res.render has a third argument that you don't see which is callback noted here
So when you create the variable in your first example, you block until you get the variable, and then your render the template with the variables available at run time.
But in your second example, you're passing in that option, but res.render's callback is calling before readdirSync can finish populating.

I figured it out and it was actually something really simple that I missed. The name of the local variable is significant.
In case 3, although shows is not explicitly passed in, it is actually visible to the page being rendered using swig.
If I do not want to introduce the variable shows, I could have done the following and it works:
res.render('index', {shows: fs.readdirSync('public/videos')});

JavaScript Autoloader: How To Recover From Errors

I have a proof-of-concept working for a JavaScript autoloader, but it currently suffers from a major flaw: it requires the code to be re-executed in its entirety rather than simply trying again from the line that failed.
Here is the prototype:
<!doctype html>
<html>
<head>
</head>
<body>
<script type="text/javascript">
var app = function(){
console.log('Initialize App');
var test = new Test();
var foo = new Foo();
var bar = new Bar();
};
var autoload = function(app){
var recover = function(error){
var name = error.message.split(' ')[0];
console.log('Loading '+name);
//A file could be synchronously loaded here instead
this[name] = function(){
console.log(name+' has been dynamically created');
};
load(app);
};
var load = function(app){
try {
app();
} catch (error){
if (error.name == "ReferenceError"){
console.log(error.message);
recover(error, app);
}
}
};
load(app);
};
autoload(app);
</script>
</body>
</html>
How It's Supposed To Work
The idea is that all of your application code would get executed within the app function. Eventually, if I can get it working properly, you could also pass in a dependency map to autoloader with the app function to synchronously load dependencies when a function is not defined. The dependency map would simply be an object mapping function names to file names.
How It Currently Works
If you don't feel like trying it out, the above code outputs the following to the console:
Initialize App
Test is not defined
Loading Test
Initialize App
Test has been dynamically created
Foo is not defined
Loading Foo
Initialize App
Test has been dynamically created
Foo has been dynamically created
Bar is not defined
Loading Bar
Initialize App
Test has been dynamically created
Foo has been dynamically created
Bar has been dynamically created
The complete app function is re-executed each time the autoloader catches an error. Obviously, this is less than ideal for a number of reasons.
Recovering From The Error
To move to the next step for making this work, I need to find a way to recover from the error without re-executing the entire app function. The error object from the catch block does provide both the line number and file name where the error occurred, but so far, I haven't been able to find a way to take advantage of that information. There are three general approaches that I can think of:
Restart script execution at the given line
Restart script execution at the beginning, but skip all lines until the given line
Grab the file as a string, split it into an array by line number, and eval the remaining lines.
Unfortunately, I wasn't able to find information on either of the first two approaches. Of the three, #1 seems like it would be more ideal, but I would certainly be open to other creative suggestions as well. As far as I can tell, JavaScript doesn't provide a way to start script execution at an arbitrary line number. #3 might work, but I'm not sure it would be very performant. The only way I can think of doing it would be to require an extra request each time to load the file text into a string.
The Questions
This is admittedly pushing the boundaries of how dependencies could be loaded in JavaScript. I'm not even sure if it is possible because I don't know if JavaScript allows for this type of error recovery. That said, I'm interested in exploring it further until I find out it's absolutely impossible.
In the interests of getting this working:
Is there a way to start script execution at an arbitrary line of JavaScript?
Are there other approaches to this that might be more fruitful? Feel free to be creative!
Taking a step back to look at the bigger picture (assuming I can get this to work):
Is a JavaScript autoloader something that people would even want?
What are the advantages/disadvantages to an autoloader like this vs. an approach like AMD?
What kind of performance issues would be encountered with this approach? Would the performance hit be too much to make it worth it?

It's kind of complicated to make, throwing and catching is kind of expensive as well. You could use typeof window["Test"]!=="function" and then create it instead of using the try catch like this.
But for a general recover and continue approach the following code would do the trick.
var app = (function(i){
var objects=new Array(3),
fnNames=["Test","Foo","Bar"];
return function(){
var len=fnNames.length
,test,foo,bar;
//check if closure vars have been reset, if so
// this has ran successfully once so don't do anything?
if(objects===false){
console.log("nothing to do, initialized already");
return;
}
while(i<len){
try{
objects[i] = new window[fnNames[i]]();
i++;
}catch(e){
if (e.name == "TypeError"){//different syntax different error
throw {"fnName":fnNames[i]};
}
}
}
//store instances in the variables
test = objects[0];
foo = objects[1];
bar = objects[2];
//reset closure vars assuming you only call app once
// when it's successful
i=null;objects=false;
console.log("Got all the instances",test,foo,bar);
};
}(0));
var autoload = function(app){
var recover = function(name){
console.log('Loading '+name);
//A file could be synchronously loaded here instead
this[name] = function(){
this.name=name;
console.log(this.name+' has been dynamically created');
};
load(app);
};
var load = function(app){
try {
app();
} catch (error){
//if statement here no longer needed
// object thrown has fnName (function name)
recover(error.fnName, app);
}
};
load(app);
};
autoload(app);
autoload(app);

URL routing in Node.js

Homework done:
The Node Beginner Book
How do I get started with Node.js [closed]
Structuring handlers in Node
Backstory: I wanted to try and write my own framework but I'm running into some troubles, most likely due to not understanding it fully.
What I want to achieve is a syntax that looks like this:
var app = require('./app'); //this part is understood and it works in my current code.
app.get('/someUrl', function(){ //do stuff here });
app.post('/someOtherUrl', function(){ //do stuff here });
I know of the Express-framework that has this same syntax but reading their source code still eludes me.
This might be a trivial task to achieve but I simply can't produce it, yet.
Trying to require('./app'); in a file deeper in the application produces a undefined object, so I'm guessing that a server is a singleton object.
So what have I tried?
My current code looks like this, and somehow I feel like this is the way to go, but I can't apparently do it like this.
I'm omitting all the require(); statements to keep it more readable.
server.js:
var app = module.exports = {
preProcess: function onRequest(request, response){
processor.preRequest(request); //this object adds methods on the request object
var path = urllib.parse(request.url).pathname;
router.route(urls, path, request, response);
},
createServer: function() {
console.log("Server start up done.");
return this.server = http.createServer(this.preProcess);
}
};
exports.app = app;
At the time of writing I'm experimenting with extending this object with a get() method.
index.js:
var app = require('./server');
app.createServer().listen('1337');
the router.route() bit basically sends the request onward into the application and inside the router.js-file I do some magic and route the request onward to a function that maps (so far) to the /urlThatWasRequested
This is the behavior I'd like to leave behind.
I know this might be a pretty tall order but all my code is easily discardable and I'm not afraid of rewriting the entire codebase as this is my own project.
I hope this is sufficient in explaining my question otherwise, please say what I should add to make this a bit more clear.
Thanks in advance!

I'm not exactly sure what your question is, but here's some thoughts:
1) You are creating a circular reference here:
var app = module.exports = {
// some other code
}
exports.app = app;
You add app as a property of app. There's no need for the last line.
2) You need to hold handlers in app. You can try something like this:
var app = module.exports = {
handlers : [],
route : function(url, fn) {
this.handlers.push({ url: url, fn: fn });
},
preProcess: function onRequest(request, response){
processor.preRequest(request);
var path = urllib.parse(request.url).pathname;
var l = this.handlers.length, handler;
for (var i = 0; i < l; i++) {
handler = this.handlers[i];
if (handler.url == path)
return handler.fn(request, response);
}
throw new Error('404 - route does not exist!');
},
// some other code
}
Note that you may alter this line: if (handler.url == path) in such a way that handler.url is a regular expression and you test path against it. Of course you may implement .get and .post variants, but from my experience it is easier to check whether a request is GET or POST inside the handler. Now you can use the code above like this:
app.route('/someUrl', function(req, res){ //do stuff here });
The other thing is that the code I've shown you only fires the first handler for a given URL it matches. You would probably want to make more complex routes involving many handlers (i.e. middlewares). The architecture would be a bit different in that case, for example:
preProcess: function onRequest(request, response){
var self = this;
processor.preRequest(request);
var path = urllib.parse(request.url).pathname;
var l = self.handlers.length,
index = 0,
handler;
var call_handler = function() {
var matched_handler;
while (index < l) {
handler = self.handlers[index];
if (handler.url == path) {
matched_handler = handler;
break;
}
index++;
}
if (matched_handler)
matched_handler.fn(request, response, function() {
// Use process.nextTick to make it a bit more scalable.
process.nextTick(call_handler);
});
else
throw new Error('404: no route matching URL!');
};
call_handler();
},
Now inside your route you can use
app.route('/someUrl', function(req, res, next){
//do stuff here
next(); // <--- this will take you to the next handler
});
which will take you to another handler (or throw exception if there are no more handlers).
3) About these words:
Trying to require('./app'); in a file deeper in the application produces a undefined
object, so I'm guessing that a server is a singleton object.
It isn't true. require always returns the reference to the module object. If you see undefined, then you've messed up something else (altered the module itself?).
Final note: I hope it helps a bit. Writing your own framework can be a difficult job, especially since there already are excelent frameworks like Express. Good luck though!
EDIT
Implementing .get and .set methods is actually not difficult. You just need to alter route function like this:
route : function(url, fn, type) {
this.handlers.push({ url: url, fn: fn, type: type });
},
get : function(url, fn) {
this.route(url, fn, 'GET');
},
post : function(url, fn) {
this.route(url, fn, 'POST');
},
and then in routing algorithm you check whether type property is defined. If it is not then use that route (undefined type means: always route). Otherwise additionally check if a request's method matches type. And you're done!