I need to load untrusted modules, written by third parties. I'm using vm for the sandbox and I was thinking to use threads (from npm: here) in order to load the module asynchronously and avoid blocking code.
I have stripped down the code to the minimum, because I'm stuck and I dont' understand if what I'm trying to achieve is impossible or it's just me messing with scopes.
Here is a dummy module:
exports.dummy = function () {
console.log('Dummy');
};
exports.val = 5;
And here is a module where I try to load this dummy module using threads:
var spawn = require('threads').spawn;
var mod;
var other;
var t = spawn(function (input, done) {
var m = require(input.dir + '/dummyMod');
done({m: m, other: 'hey'})
});
t.send({dir: __dirname}).on('message', function (result) {
mod = result.m;
other = result.other;
console.log(mod);
console.log(other);
t.kill();
});
The logged output is:
{ val: 5 }
hey
As you can see, the function in the dummy module has been skipped. If I try to load the module in the main process and log it, then the function is, of course, part of the object.
You need to properly serialize and deserialize the function. JSON.stringify ignores functions, probably because json is a format for storing data, not scripts.
Serialize the function by calling toString() on it. Then you can send it along as a string.
done({m: m.toString(), other: 'hey'})
Converting m to a string will give you something like this:
"function m(){console.log(\'called m()\')}"
On the receiving end, you will need to deserialize the function.
var m = new Function("return " + result.m)()
Related
I know that converting a buffer to a json object is trivial with:
JSON.parse(buffer)
But what about converting a buffer that contains a javascript object with functions to a real javascript object ??
I have files that contact objects with functions and I need to read these files in and save them to another master object.
While the task of reading the files is not a problem, I cannot find any way of converting the buffer back into a real javascript object again.
This is a very simple example of a file 'test.js' that I am trying to load
{
get:function(){return 'hello'},
somevar: "xxx",
put: function(){return 'world'}
}
Reading this data in it is a buffer, I can't convert using JSON as this contains functions and I cannot read using utf8 encoding as it will become a string !
var funcs = {}
fs.readFile('test.js',function(err,buff){
funcs['test'] = buff;
})
Is it possible to read a file and convert it into a real javascript object ?
Edit
OK, I have found a solution but it is using the dreaded eval(), however this is backend code and as far as I can tell there's no way for anything to be injected by a user from the frontend, I would prefer not to use it but unless there's anything that will work without modifying the format of the files, I will use it:
var funcs = {}
fs.readFile('test.js','utf8',function(err,buff){
eval('var_='+buff);
funcs['test'] = _;
})
For what it's worth, you could use the Function constructor. It's slightly safer than eval because it doesn't access the local scope. But it can still access global variables.
var script = buffer.toString('utf8'); // assuming the file is in UTF-8
var returnObject = new Function('return ' + script);
var myObject = returnObject();
Depending on the complexity of the code you're dealing with, this approach may or may not suit your needs:
You can create a temporary file, say "test-module.js", which includes the object from "test.js" but with a module export prependend, for example:
module.exports = {
get: function() { return 'hello' },
somevar: 'xxx',
put: function() { return 'world' }
}
Then, from your main file, you can retrieve the file content already available as a Javascript object:
var funcs = {}
var buff = require('./test-module');
funcs['test'] = [buff.get, buff.put];
Would this solution work for you?
I am trying to use a library found on the web, called himalaya, which is an html parser.
https://github.com/andrejewski/himalaya
I followed their guide on importing the library, so I do
var himalaya = require('himalaya');
However when I call one of its member functions, I get an error
TypeError: himalaya.parse is not a function
I tried executing himalaya.parse() on the web browser console directly, it works. I tried commenting out the require statement in the js file, the function no longer works on web browser.
I guess this implies the require statement works? But for some reasons I cannot use it in my javascript file, only on the browser console.
Perhaps something with file scopes? Here is part of my code.
var himalaya = require('himalaya');
Template.main.onCreated(function () {
var http = new HttpGet("www.someurl.com/", "/somedirectories/", function (response) {
console.log(himalaya.parse(response.content));
});
http.sendRequest();
});
I am certain that response.content does contain a valid html string.
When you call the himalaya.parse inside the main.onCreated function it seems like the library is not completed loaded at that time. That's why it only runs in your browser console. Check if the himalaya library has a onReady function to let you know exactly when you can use it. If not, you can:
a) Call the parse function inside the main.onRendered or
b) Keep the parse call inside the main.onCreated and set a timeOut to call it after a half second like this:
var himalaya = require('himalaya');
Template.main.onCreated(function () {
var http = new HttpGet("www.someurl.com/", "/somedirectories/", function (response) {
setTimeout(function(){himalaya.parse(response.content)},500);
});
http.sendRequest();
});
If you have an issue with the setTimeout check this answer:
Meteor.setTimeout function doesn't work
I am new to both node.js and promise style function call. By looking at an denodeify example at http://runnable.com/Ulatc0QnzUgUAAAK/adapting-node-js-with-q-for-promises, I am trying to denodeify the methods of the node.js node-ftp module as following:
var ftp = require('ftp');
var q = require('q');
var ftpClient = new ftp();
ftpClient.on('ready', function() {
var ftpList = q.denodeify(ftpClient.list);
ftpList().then(function(list) {
console.log(list);
}.then(null, function(err) {
console.log(err);
}).done(function() {
ftpClient.end();
});
});
ftpClient.connect();
However, when running that code with node, it shows the error "list error: TypeError: Object # has no method '_pasv'"
I am not sure what's wrong with that piece of code. Does anyone know what's wrong with that? Can you point me some way to debug/troubleshoot the cause of that error message?
Thanks.
When you pass
ftpClient.list
to Q.denodefiy, you are getting the function object, list from the ftpClient object. It will be just a function and the relationship with the parent is lost. This is important because, the bound function list might be dependent on the ftpClient object. So, you must make sure that link is not broken.
Quoting from the Q.denodeify docs,
Note that if you have a method that uses the Node.js callback pattern,
as opposed to just a function, you will need to bind its this value
before passing it to denodeify, like so:
var Kitty = mongoose.model("Kitty");
var findKitties = Q.denodeify(Kitty.find.bind(Kitty));
The better strategy for methods would be to use Q.nbind, as shown below.
So, you can fix your code in two ways,
Using Q.denodeify and Function.prototype.bind, like this
var ftpList = q.denodeify(ftpClient.list.bind(ftpClient));
Using Q.nbind, like this
var ftpList = q.nbind(ftpClient.list, ftpClient);
you need to use q.nbind
q.nbind(ftpClient.list, ftpClient);
For instance, I have app.js to have a Module object:M by loading(require) a file:_core.js, then properties of M are to add by loading(require) files under a directory: ./Project_Functions/
var Project_FunctionsDIR = './Project_Functions/';
var Project_coreFile = '_core.js';
var M = require(Project_FunctionsDIR + Project_coreFile);
require("fs")
.readdir(Project_FunctionsDIR,
function(err, files)
{
files.forEach(function(file)
{
if (file !== Project_coreFile)
{
var name = file.split('.js')[0];
var filepath = Project_FunctionsDIR + file;
M[name] = require(filepath);
}
});
});
console.log('module.exports');
module.exports = M;
Here, the constructed moduleObject:M is exported by module.exports.
Please note the fs.readdir is async, so module.exports = M; is evaluated before M properties are constructed in app.js.
Now, I want to test this object M in test.js,
var expect = require('chai').expect;
var M = require('./app.js');
//........
On var M = require('./app.js'); line, the app.js is evaluated, which indicates that the M object is exported before the whole properties are constructed.
In fact, I inserted setTimeout at the fs.readdir callback, the test fails obviously the M object is not ready yet.
Firstly, I thought module.exports = M; should be put within the fs.readdir callback, but if I do that, now var M = require(Project_FunctionsDIR + Project_coreFile); part fails, probably because module.exports = M; is not exposed to read properly.
Yes, this can be resolved simply by using fs.readdirSync and actually, I have been doing so when I encounter this sort of problem, however, I always feel uncomfortable to use node Sync methods and rather try to seek a smarter work-around.
So, here's my question; is it possible by not using Sync method for this issue? or impossible?
Furthermore, I'd like to ask more in general.
Can we do anything without using Sync methods, or in some cases, we must use Sync methods in node.js?
In functional programming scheme, Sync methods should be avoided if possible since the Sync concept depends on not function(callback) but time.
Thanks for your thought.
EDIT:
I've just found a related topic here:
Asynchronous initialization of Node.js module
The conclusion is we must use Sync method to correspond a Sync method. In this case, require is a sync method.
A common pattern is for the module to expose an asynchronous initialize method that does the asynchronous work. For your module, it would look like this:
var Project_FunctionsDIR = './Project_Functions/';
var Project_coreFile = '_core.js';
var M = require(Project_FunctionsDIR + Project_coreFile);
M.initialize = function(next) {
require("fs")
.readdir(Project_FunctionsDIR,
function(err, files)
{
files.forEach(function(file)
{
if (file !== Project_coreFile)
{
var name = file.split('.js')[0];
var filepath = Project_FunctionsDIR + file;
M[name] = require(filepath);
}
});
next();
});
}
module.exports = M;
Then you can test it with:
var M = require('./app.js');
M.initialize(function() {
//........
}
This is a useful pattern because many asynchronous methods don't have synchronous versions, and it allows you to run many of these asynchronous initializes in parallel, drastically reducing startup time for larger apps.
node.js uses CommonJS module pattern, which by definition export their exposed objects synchronously on load. What you're trying to do is not possible asynchronously, although you can do it just by exposing the other files into your 'main' export. If you want to dynamically expose all files in a directory from one object, just use the synchronous readdir, it'll only be executed on start up.
I am running Node.JS Sandbox module to create a Child Process and I need to be able to have my String Based Javascript access certain functions and Modules of Node. Right now, the Sandbox module is blocking all access to Node API's and functions outside of the Sandbox.
Example
function GetAccessTo() {
return "I want to call this function"
}
var str = "GetAccessTo();"
var s = new Sandbox();
s.run(str, function (output) {
output.result;
});
To add methods to the context , go to Shovel.js and add methods to var contextand you will be able to call from your Javascript String
Like this:
s.run("hi('something',function(result){return result})", function (output) {
logic = output.result;
});
var context = Script.createContext();
context.hi = function(a,b) {
c="hi"
return b(c)};