I'd like to get a built in module (for example Math or path or fs), whether from the global object or require, I thought about doing something like this:
function getModuleByName(name) {
return global[name] || require(name);
}
Is there a way to check that it is indeed a module and not something else? Would this make a security problem?
Is there a way to check that it is indeed a module and not something else?
Other methods but here's an example:
function getModuleByName(name)
{
let module = null;
try {
module = require(name);
} catch (e) {
// Recommend Logging e Somewhere
}
return module;
}
This will graciously fail as null where the module does not exist, or return it.
Would this make a security problem?
Quite possibly, it depends on how it's used. I'd argue it is more of a general design issue however and would blanket say avoid doing it (without any context, you may have a very good reason).
You, like anyone, obviously have a finite amount of modules you could be loading. These modules have all been selected by yourself for your application for specific reasons, or are bundled into your node version natively and are expected parts of your environment.
What you are doing by introducing this functionality is adding the addition of unexpected elements in your environment. If you are using getModuleByName to access a third party library- you should know outright that library is available and as such there's no reason why you can't just require it directly.
--
If you do think your use case warrants this, please let me know what it is as I may never have encountered it before. I have used dynamic imports like the following:
https://javascript.info/modules-dynamic-imports
But that hasn't been for global packages/libraries, but for dynamic reference to modules built internally to the application (i.e. routing to different views, invokation of internal scripts).
These I have protected by ensuring filepaths can't be altered by whitelisting the target directories, making sure each script follows a strict interface per use case and graciously failing where a module doesn't exist (error output "this script does not exist" for the script usage and a 404 view for the routing example).
Related
I'm working on a complicated map-reduce process for a mongodb database. I've split some of the more complex code off into modules, which I then make available to my map/reduce/finalize functions by including it in my scopeObj like so:
const scopeObj = {
userCalculations: require('../lib/userCalculations')
}
function myMapFn() {
let userScore = userCalculations.overallScoreForUser(this)
emit({
'Key': this.userGroup
}, {
'UserCount': 1,
'Score': userScore
})
}
function myReduceFn(key, objArr) { /*...*/ }
db.collection('userdocs').mapReduce(
myMapFn,
myReduceFn,
{
scope: scopeObj,
query: {},
out: {
merge: 'userstats'
}
},
function (err, stats) {
return cb(err, stats);
}
)
...This all works fine. I had until recently thought it wasn't possible to include module code into a map-reduce scopeObj, but it turns out that was just because the modules I was trying to include all had dependencies on other modules. Completely standalone modules appear to work just fine.
Which brings me (finally) to my question. How can I -- or, for that matter, should I -- incorporate more complex modules, including things I've pulled from npm, into my map-reduce code? One thought I had was using Browserify or something similar to pull all my dependencies into a single file, then include it somehow... but I'm not sure what the right way to do that would be. And I'm also not sure of the extent to which I'm risking severely bloating my map-reduce code, which (for obvious reasons) has got to be efficient.
Does anyone have experience doing something like this? How did it work out, if at all? Am I going down a bad path here?
UPDATE: A clarification of what the issue is I'm trying to overcome:
In the above code, require('../lib/userCalculations') is executed by Node -- it reads in the file ../lib/userCalculations.js and assigns the contents of that file's module.exports object to scopeObj.userCalculations. But let's say there's a call to require(...) somewhere within userCalculations.js. That call isn't actually executed yet. So, when I try to call userCalculations.overallScoreForUser() within the Map function, MongoDB attempts to execute the require function. And require isn't defined on mongo.
Browserify, for example, deals with this by compiling all the code from all the required modules into a single javascript file with no require calls, so it can be run in the browser. But that doesn't exactly work here, because I need to be the resulting code to itself be a module that I can use like I use userCalculations in the code sample. Maybe there's a weird way to run browserify that I'm not aware of? Or some other tool that just "flattens" a whole hierarchy of modules into a single module?
Hopefully that clarifies a bit.
As a generic response, the answer to your question: How can I -- or, for that matter, should I -- incorporate more complex modules, including things I've pulled from npm, into my map-reduce code? - is no, you can not safely include complex modules in node code you plan to send to MongoDB for mapReduce jobs.
You mentioned the problem yourself - nested require statements. Now, require is sync, but if you have nested functions inside, these require calls would not be executed until call time, and MongoDB VM would throw at this point.
Consider the following example of three files: data.json, dep.js and main.js.
// data.json - just something we require "lazily"
false
// dep.js -- equivalent of your userCalculations
module.exports = {
isValueTrue() {
// The problem: nested require
return require('./data.json');
}
}
// main.js - from here you send your mapReduce to MongoDB.
// require dependency instantly
const calc = require('./dep.js');
// require is synchronous, the effectis the same if you do:
// const calc = (function () {return require('./dep.js')})();
console.log('Calc is loaded.');
// Let's mess with unwary devs
require('fs').writeFileSync('./data.json', 'false');
// Is calc.isValueTrue() true or false here?
console.log(calc.isValueTrue());
As a general solution, this is not feasible. While vast majority of modules will likely not have nested require statements, HTTP calls, or even internal, service calls, global variables and similar, there are those who do. You cannot guarantee that this would work.
Now, as a your local implementation: e.g. you require exactly specific versions of NPM modules that you have tested well with this technique and you know it will work, or you published them yourself, it is somewhat feasible.
However, even if this case, if this is a team effort, there's bound to be a developer down the line who will not know where your dependency is used or how, use globals (not on purpose, but by ommission, e.g they wrongly calculate this) or simply not know the implications of whatever they are doing. If you have strong integration testing suite, you could guard against this, but the thing is, it's unpredictable. Personally I think that when you can choose between unpredictable and predictable, almost always you should use predictable.
Now, if you have an explicitly stated purpose for a certain library to be used in MongoDB mapReduce, this would work. You would have to guard well against ommisions and problems, and have strong testing infra, but I would make certain the purpose is explicit before feeling safe enough to do this. But of course, if you're using something that is so complex that you need several npm packages to do, maybe you can have those functions directly on MongoDB server, maybe you can do your mapReducing in something better suited for the purpose, or similar.
To conclude: As a purposefuly built library with explicit mission statement that it is to be used with node and MongoDB mapReduce, I would make sure my tests cover all my mission-critical and important functionality, and then import such npm package. Otherwise I would not use nor recommend this approach.
I tried something like:
var path = '../right/here';
var module = require(path);
but it can't find the module anymore this way, while:
var module = require('../right/here');
works like a charm. Would like to load modules with a generated list of strings, but I can't wrap my head around this problem atm. Any ideas?
you can use template to get file dynamically.
var myModule = 'Module1';
var Modules = require(`../path/${myModule}`)
This is due to how Browserify does its bundling, it can only do static string analysis for requirement rebinding. So, if you want to do browserify bundling, you'll need to hardcode your requirements.
For code that has to go into production deployment (as opposed to quick prototypes, which you rarely ever bother to add bundling for) it's always advisable to stick with static requirements, in part because of the bundling but also because using dynamic strings to give you your requirements means you're writing code that isn't predictable, and can thus potentially be full of bugs you rarely run into and are extremely hard to debug.
If you need different requirements based on different runs (say, dev vs. stage testing vs. production) then it's usually a good idea to use process.env or a config object so that when it comes time to decide which library to require for a specific purposes, you can use something like
var knox = config.offline ? require("./util/mocks3") : require("knox");
That way your code also stays immediately traversable for others who need to track down where something's going wrong, in case a bug does get found.
require('#/path/'.concat(fileName))
You can use .require() to add the files that you want to access calculating its path instead of being static at build time, this way this modules will be included and when calling require() later they will be found.
I am working on a JavaScript project where we use requirejs to manage our code. The product we create is to be used on 3rd party web sites, and therefore we cannot assume that an AMD compatible library is present. To solve this, we include almond in our target file and expose our main module on window. The file created by our build looks like this:
(function() {
//Almond lib
//Our code
define('window-exposer', ['main-module'], function(mainModule) {
window.mainModule = mainModule;
});
require('window-exposer');
}());
When building a site that want to use mainModule an error is thrown because when the site specific code tries to access window.mainModule it has not been set yet. There are also cases where the module has indeed been initialized and the code works.
Is there any way to guarantee that the window-exposer is run before other JavascriptCode is?
I solved it by using the solution provided here https://github.com/jrburke/almond#exporting-a-public-api
I have come across JSHint but was hoping it would validate node.js syntax but it doesn't, for example if I do:
var server = http.createServerfunctionThatDoesNotExists(function(request, response) {....
The test passes even though there isn't a function called createServerfunctionThatDoesNotExists
What am I missing with JSHint and the node.js option?
I think it doesn't dig into CommonJS modules as they can contain virtually everything.
So even if you are require'ing http it could be your own module with any interface.
'Assume Node' means it tolerates global variables like module or exports
JSHint is basically JSLint and is more for syntax/style/JS-nonos rather than a deep-scan product that executes your JS. Fully vetting JS code (or any dynamic language, really, depending on your definition of dynamic) pretty much requires running it, since things like methods can be defined more or less anywhere, including at runtime.
JSHint does go a bit further and allow the definition of high-level constructs used by some JS libraries.
I'm working on a project with Node.js and the server-side code is becoming large enough that I would like to split it off into multiple files. It appears this has been done client-side for ages, development is done by inserting a script tag for each file and only for distribution is something like "Make" used to put everything together. I realize there's no point in concatting all the server-side code so I'm not asking how to do that. The closest thing I can find to use is require(), however it doesn't behave quite like script does in the browser in that require'd files do not share a common namespace.
Looking at some older Node.js projects, like Shooter, it appears this was once not the case, that or I'm missing something really simple in my code. My require'd files cannot access the global calling namespace at compile time nor run time. Is there any simple way around this or are we forced to make all our require'd JS files completely autonomous from the calling scope?
You do not want a common namespace because globals are evil. In node we define modules
// someThings.js
(function() {
var someThings = ...;
...
module.exports.getSomeThings = function() {
return someThings();
}
}());
// main.js
var things = require("someThings");
...
doSomething(things.getSomeThings());
You define a module and then expose a public API for your module by writing to exports.
The best way to handle this is dependency injection. Your module exposes an init function and you pass an object hash of dependencies into your module.
If you really insist on accessing global scope then you can access that through global. Every file can write and read to the global object. Again you do not want to use globals.
re #Raynos answer, if the module file is next to the file that includes it, it should be
var things = require("./someThings");
If the module is published on, and installed through, npm, or explicitly put into the ./node_modules/ folder, then the
var things = require("someThings");
is correct.