Develop Reference

How to add plugin and using some external module/file on RT

I'm having node.js application/module which is working OK with plug-in concept ,i.e.
My module is acting like proxy with additional capabilities such as adding new functionality to the out-of-the-box functionality(methods).
To do this you need to do the following:
clone my application
create new folder which is called extenders(inside my app)
In this folder you should provide two files
extend.js with your logic as functions/methods
extend.json which define your API (to know which file to invoke)
Note: the JS & JSON file name must be identical
for example lets assume that this is your extend.json file
{
"extenders": [
{
"path": "run",
"fn": "runFn"
},
}
In this case when the user put in the browser the following link
localhost:3000/run
Im invoking the runFn function (which exist in the extend.js file)with its logic and this is working as expected (under the hood I read the json & js files and invoke the function like extender[fnName](req, res));
Now I want to support the use case of adding external extender via code
for example that the user will do something like
var myModule = require('myModule');
myModule.extend('./pathTo/newExternalPathforExtendersFolder');
so when my module will run it search weather there is new external extenders exist with all configuration and if so refer to it in RT (to the js&json files).
My questions are:
I need to find when my module is starting who is register to my module and then do my logic on this module , how it can be done in node?
2.if there is other solution in node please let me know.

You could implement initialization function in your api to give freedom to module users. For example.
var yourModule = require('yourModule').init({
extenders: [
{
"path": "run",
"fn": "runFn"
}
]
});
yourModule.listen(3000);
Or as MattW wrote you can implement it like an express middleware, so module users could use it with their own server. For example:
var yourModule = require('yourModule').init({
extenders: [
{
"path": "run",
"fn": "runFn"
}
]
});
app = require('express')();
app.use(yourModule.getMiddleware());
Look at webpack-dev-server, webpack-dev-middleware like another example. Hope there's some similarity with your task. Webpack also deals with fs and configs. They've just splitted middleware and standalone server to separate modules. And there's no "working code", because we need your module's code, to talk about wrapper, which would depend on yourModule implementation. Just some thoughts.

If i'm not wrong understanding your problem maybe this approach can help you.
I think you could list your extenders in an ACL like JSON which not only include the path or the fnName, but the file_to_js path or any other property you need, like if it's active or security parameters.
extenders: [
{
"path": "run",
"fn": "runFn",
"file": "file_path"
"api-key": true,
"active": true
}
]
Then you can preload your modules reading ACL json and let them cached ready for extend.
var Router = {
extenders: {},
init: function () {
this.extenders = {};
this.loadExtenderFiles();
},
loadExtenderFiles: function () {
var key, extender;
// Iterate extender files
for (key in ACL_JSON) {
// extender load logic
extender = ACL_JSON[key];
if (extender.active) {
this.extenders[extender.fn] = require(extender.file);
}
}
},
// this fn should allow you to Router.extend() anywhere during the request process
extend: function (fn, request, response) {
// Parse request/response to match your extender module pattern
// extender process logic
this.extenders[fn](request, response);
}
};
module.exports = Router;
So Router.init() should do the cache work on server init;
Router.extend() should resolve your api request or extend one being processed.
Hope it helps you!

I believe a simple router should satisfy your requirements:
var userModule = require('userModule');
router.use('/run', function (req, res, next) {
return next(userModule(req));
}).all(yourReverseProxy);

Webpack: How can I create a loader for "webpack" which takes an array of dependencies?

For example, I use AMD definition in my project, and use "webpack" for project building. It's possible to create some loader which will take a dependencies in array format?
define(
[
'mySuperLoader![./path/dependency-1, ./path/dependency-2, ...]'
],
function() {
// ... some logic here
}
)
Project example: gitHub

If you want to port the load-plugin's behavior to webpack, you need to do this:
1. Create a custom resolver
This is because mySuperLoader![./path/dependency-1, ./path/dependency-2, ...] does not point to a single file. When webpack tries to load a file, it first:
resolves the file path
loads the file content
matches and resolves all loaders
passes the file content to the loader chain
Since [./path/dependency-1, ./path/dependency-2, ...] is not a proper file path, there is some work to do. It is even not a proper JSON.
So, our first goal is to turn this into mySuperLoader!some/random/file?["./path/dependency-1", "./path/dependency-2", ...]. This is usually done by creating a custom resolver:
// webpack.config.js
var customResolverPlugin = {
apply: function (resolver) {
resolver.plugin("resolve", function (context, request) {
const matchLoadRequest = /^\[(.+)]$/.exec(request.path);
if (matchLoadRequest) {
request.query = '?' + JSON.stringify(
matchLoadRequest[1]
.split(", ")
);
request.path = __filename;
}
});
}
};
module.exports = {
...
plugins: [
{
apply: function (compiler) {
compiler.resolvers.normal.apply(customResolverPlugin);
}
}
]
};
Notice request.path = __filename;? We just need to give webpack an existing file so that it does not throw an error. We will generate all the content anyway. Probably not the most elegant solution, but it works.
2. Create our own load-loader (yeah!)
// loadLoader.js
const path = require("path");
function loadLoader() {
return JSON.parse(this.request.match(/\?(.+?)$/)[1])
.map(module =>
`exports['${path.basename(module, '.js')}'] = require('${module}');`
)
.join('\n');
}
module.exports = loadLoader;
This loader parses the request's query we have re-written with our custom resolver and creates a CommonJS module that looks like this
exports['dependency-1'] = require('path/to/dependency-1');
exports['dependency-2'] = require('path/to/dependency-2');
3. Alias our own load-loader
// webpack.config.js
...
resolveLoader: {
alias: {
load: require.resolve('./loadLoader.js')
}
},
4. Configure root
Since /path/to/dependency-1 is root-relative, we need to add the root to the webpack config
// webpack.config.js
resolve: {
root: '/absolute/path/to/root' // usually just __dirname
},
This is neither a beautiful nor an ideal solution, but should work as a makeshift until you've ported your modules.

I don't think that you should use a loader for that. Why don't you just write:
require("./path/dependency-1");
require("./path/dependency-2");
require("./path/dependency-3");
It accomplishes the same thing, is much more expressive and requires no extra code/loader/hack/configuration.
If you're still not satisfied, you might be interested in webpack contexts which allow you to require a bulk of files that match a given filter. So, if you write
require("./template/" + name + ".jade");
webpack includes all modules that could be accessed by this expression without accessing parent directories. It's basically the same like writing
require("./table.jade");
require("./table-row.jade");
require("./directory/folder.jade")
You can also create contexts manually like this
var myRequire = require.context(
"./template", // search inside this directory
false, // false excludes sub-directories
/\.jade$/ // use this regex to filter files
);
var table = myRequire("./table.jade");

Require.js looking in the wrong path for the nested dependencies

I use WebJars to manage my JavaScript dependencies which means that most of the libraries I use are outside the Require.js base path. The config is created automatically and it works for most libraries. I only have problems with those libraries that call require() to load some inner dependencies (usually just external files) - for example when.js.
This is the require.js config generated for when library:
requirejs.config({"paths":{"when":["/webjars/when-node/3.5.2/when","when"]}});
This file loads properly. But the problem is that it tries to load further files:
...
var timed = require('./lib/decorators/timed');
var array = require('./lib/decorators/array');
var flow = require('./lib/decorators/flow');
var fold = require('./lib/decorators/fold');
...
I would expect require.js to use the location of when.js to determine the correct locations of the other required files, i. e.:
/webjars/when-node/3.5.2/lib/decorators/timed.js
But unfortunately require.js uses instead the location of the main.js file as the base path which obviously results in a lot of 404 errors and the application crashes.
How can I tell require.js to look in the correct subdirectory?

So according to the RequireJS documentation this seems to be the correct behavior. Unlike CommonJS, every require() call is resolved using the base path (it is relative to the base path, not to the location of the file it is called in).
The only way to get around this (as far as I know) is to configure the dependency as a package. The package location is then used for path resolution instead of the general base path. For the mentioned when package, the configuration should look something like this:
requirejs.config({
packages: [
{ name: 'when', location: '/path/to/when', main: 'when' }
]
});
To fix this problem automatically for all my current or future dependencies, I've replaced the Scala Play Framework loading script with my own solution which loads all WebJars as separate packages.
var require = {
callback : function () {
var packages = [];
var shim = {};
[
#for(webJarJson <- org.webjars.RequireJS.getSetupJson(routes.WebJarAssets.at("").url).values()) {
#Html(webJarJson.toString),
}
].forEach(function (webjar) {
if (webjar.paths) {
for (var name in webjar.paths) {
if (webjar.paths.hasOwnProperty(name)) {
packages.push({
name: name,
location: webjar.paths[name][0].replace(/\/[^\/]+$/, ""),
main: webjar.paths[name][1]
});
}
}
}
if (webjar.shim) {
for (var name in webjar.shim) {
if (webjar.shim.hasOwnProperty(name)) {
shim[name] = webjar.shim[name];
}
}
}
});
requirejs.config({
packages: packages,
shim: shim
});
}
}

Is there a Java classpath-like feature for server-side Javascript?

When using JUnit and Maven in Java, one can have separate property files for src/main and src/test. This allows different configuration for code and tests, having Maven to manage the resources by using Java classpath.
Is there a similar way in Javascript code run by Node.js? I use Mocha for unit-testing and Grunt for task management.
Code example for script.js:
var config = require('./config/dev/app.js');
exports.getFileName = function() {
return config.fileName; // returns 'code.txt'
}
What I need is to make the script.js use different config file when being required in a test.js unit test like this:
var assert = require('assert');
var s = require('./script.js');
describe('Test', function () {
it('should use different config file', function() {
assert.equal('test.txt', s.getFileName());
});
});
Is there a way to use different configuration ./config/test/app.js in the script.js without having to alter the code of script.js? What I really try to avoid is to adjust the code to support unit tests. Instead, I want to achieve similar functionality such as mentioned Java classpath.

Please try this code.
Script.js
var config;
if(process.env.dev===true){
config = require('./config/dev/config.js');
}
if(process.env.prod===true){
config = require('./config/prod/config.js');
}
exports.getFileName = function() {
return config.fileName; // returns 'code.txt'
}
test.js
//set the environment here
process.env.dev = true;
var assert = require('assert');
var s = require('./script.js');
describe('Test', function () {
it('should use different config file', function() {
assert.equal('test.txt', s.getFileName());
});
});

I have not found any elegant solution out there on the web so I have implemented and published my own.
Check it out here: https://npmjs.org/package/app-config
Using the app-config plugin, only the script.js needs to get changed this way:
var config = require('app-config').app;
exports.getFileName = function() {
return config.fileName; // returns 'code.txt'
}
The app needs to be run this way for example
NODE_ENV=dev node script.js
NODE_ENV=unitTest mocha test.js
Depending on the NODE_ENV environmental variable, the right set of configuration files will be loaded by the app-config plugin.

Why do I see "define not defined" when running a Mocha test with RequireJS?

I am trying to understand how to develop stand-alone Javascript code. I want to write Javscript code with tests and modules, running from the command line. So I have installed node.js and npm along with the libraries requirejs, underscore, and mocha.
My directory structure looks like this:
> tree .
.
├── node_modules
├── src
│   └── utils.js
└── test
└── utils.js
where src/utils.js is a little module that I am writing, with the following code:
> cat src/utils.js
define(['underscore'], function () {
"use strict";
if ('function' !== typeof Object.beget) {
Object.beget = function (o) {
var f = function () {
};
f.prototype = o;
return new f();
};
}
});
and test/utils.js is the test:
> cat test/utils.js
var requirejs = require('requirejs');
requirejs.config({nodeRequire: require});
requirejs(['../src/utils'], function(utils) {
suite('utils', function() {
test('should always work', function() {
assert.equal(1, 1);
})
})
});
which I then try to run from the top level directory (so mocha sees the test directory):
> mocha
node.js:201
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Calling node's require("../src/utils") failed with error: ReferenceError: define is not defined
at /.../node_modules/requirejs/bin/r.js:2276:27
at Function.execCb (/.../node_modules/requirejs/bin/r.js:1872:25)
at execManager (/.../node_modules/requirejs/bin/r.js:541:31)
...
So my questions are:
Is this the correct way to structure code?
Why is my test not running?
What is the best way to learn this kind of thing? I am having a hard time finding good examples with Google.
Thanks...
[sorry - momentarily posted results from wrong code; fixed now]
PS I am using requirejs because I also want to run this code (or some of it) from a browser, later.
Update / Solution
Something that is not in the answers below is that I needed to use mocha -u tdd for the test style above. Here is the final test (which also requires assert) and its use:
> cat test/utils.js
var requirejs = require('requirejs');
requirejs.config({nodeRequire: require});
requirejs(['../src/utils', 'assert'], function(utils, assert) {
suite('utils', function() {
test('should always work', function() {
assert.equal(1, 1);
})
})
});
> mocha -u tdd
.
✔ 1 tests complete (1ms)

The reason your test isn't running is because src/utils.js is not a valid Node.js library.
According to the RequireJS documentation, in order to co-exist with Node.js and the CommonJS require standard, you need to add a bit of boilerplate to the top of your src/utils.js file so RequireJS's define function is loaded.
However, since RequireJS was designed to be able to require "classic" web browser-oriented source code, I tend to use the following pattern with my Node.js libraries that I also want running in the browser:
if(typeof require != 'undefined') {
// Require server-side-specific modules
}
// Insert code here
if(typeof module != 'undefined') {
module.exports = whateverImExporting;
}
This has the advantage of not requiring an extra library for other Node.js users and generally works well with RequireJS on the client.
Once you get your code running in Node.js, you can start testing. I personally still prefer expresso over mocha, even though its the successor test framework.

The Mocha documentation is lacking on how to set this stuff up, and it's perplexing to figure out because of all the magic tricks it does under the hood.
I found the keys to getting browser files using require.js to work in Mocha under Node: Mocha has to have the files added to its suites with addFile:
mocha.addFile('lib/tests/Main_spec_node');
And second, use beforeEach with the optional callback to load your modules asynchronously:
describe('Testing "Other"', function(done){
var Other;
beforeEach(function(done){
requirejs(['lib/Other'], function(_File){
Other = _File;
done(); // #1 Other Suite will run after this is called
});
});
describe('#1 Other Suite:', function(){
it('Other.test', function(){
chai.expect(Other.test).to.equal(true);
});
});
});
I created a bootstrap for how to get this all working: https://github.com/clubajax/mocha-bootstrap

You are trying to run JS modules designed for browsers (AMD), but in the backend it might not work (as modules are loaded the commonjs way). Because of this, you will face two issues:
define is not defined
0 tests run
In the browserdefine will be defined. It will be set when you require something with requirejs. But nodejs loads modules the commonjs way. define in this case is not defined. But it will be defined when we require with requirejs!
This means that now we are requiring code asynchronously, and it brings the second problem, a problem with async execution.
https://github.com/mochajs/mocha/issues/362
Here is a full working example.
Look that I had to configure requirejs (amd) to load the modules, we are not using require (node/commonjs) to load our modules.
> cat $PROJECT_HOME/test/test.js
var requirejs = require('requirejs');
var path = require('path')
var project_directory = path.resolve(__dirname, '..')
requirejs.config({
nodeRequire: require,
paths: {
'widget': project_directory + '/src/js/some/widget'
}
});
describe("Mocha needs one test in order to wait on requirejs tests", function() {
it('should wait for other tests', function(){
require('assert').ok(true);
});
});
requirejs(['widget/viewModel', 'assert'], function(model, assert){
describe('MyViewModel', function() {
it("should be 4 when 2", function () {
assert.equal(model.square(2),4)
})
});
})
And for the module that you want to test:
> cat $PROJECT_HOME/src/js/some/widget/viewModel.js
define(["knockout"], function (ko) {
function VideModel() {
var self = this;
self.square = function(n){
return n*n;
}
}
return new VideModel();
})

Just in case David's answer was not clear enough, I just needed to add this:
if (typeof define !== 'function') {
var define = require('amdefine')(module);
}
To the top of the js file where I use define, as described in RequireJS docs ("Building node modules with AMD or RequireJS") and in the same folder add the amdefine package:
npm install amdefine
This creates the node_modules folder with the amdefine module inside.

I don't use requirejs so I'm not sure what that syntax looks like, but this is what I do to run code both within node and the browser:
For imports, determine if we are running in node or the browser:
var root = typeof exports !== "undefined" && exports !== null ? exports : window;
Then we can grab any dependencies correctly (they will either be available already if in the browser or we use require):
var foo = root.foo;
if (!foo && (typeof require !== 'undefined')) {
foo = require('./foo');
}
var Bar = function() {
// do something with foo
}
And then any functionality that needs to be used by other files, we export it to root:
root.bar = Bar;
As for examples, GitHub is a great source. Just go and check out the code for your favorite library to see how they did it :) I used mocha to test a javascript library that can be used in both the browser and node. The code is available at https://github.com/bunkat/later.