I am trying to use a library found on the web, called himalaya, which is an html parser.
https://github.com/andrejewski/himalaya
I followed their guide on importing the library, so I do
var himalaya = require('himalaya');
However when I call one of its member functions, I get an error
TypeError: himalaya.parse is not a function
I tried executing himalaya.parse() on the web browser console directly, it works. I tried commenting out the require statement in the js file, the function no longer works on web browser.
I guess this implies the require statement works? But for some reasons I cannot use it in my javascript file, only on the browser console.
Perhaps something with file scopes? Here is part of my code.
var himalaya = require('himalaya');
Template.main.onCreated(function () {
var http = new HttpGet("www.someurl.com/", "/somedirectories/", function (response) {
console.log(himalaya.parse(response.content));
});
http.sendRequest();
});
I am certain that response.content does contain a valid html string.
When you call the himalaya.parse inside the main.onCreated function it seems like the library is not completed loaded at that time. That's why it only runs in your browser console. Check if the himalaya library has a onReady function to let you know exactly when you can use it. If not, you can:
a) Call the parse function inside the main.onRendered or
b) Keep the parse call inside the main.onCreated and set a timeOut to call it after a half second like this:
var himalaya = require('himalaya');
Template.main.onCreated(function () {
var http = new HttpGet("www.someurl.com/", "/somedirectories/", function (response) {
setTimeout(function(){himalaya.parse(response.content)},500);
});
http.sendRequest();
});
If you have an issue with the setTimeout check this answer:
Meteor.setTimeout function doesn't work
I have a proof-of-concept working for a JavaScript autoloader, but it currently suffers from a major flaw: it requires the code to be re-executed in its entirety rather than simply trying again from the line that failed.
Here is the prototype:
<!doctype html>
<html>
<head>
</head>
<body>
<script type="text/javascript">
var app = function(){
console.log('Initialize App');
var test = new Test();
var foo = new Foo();
var bar = new Bar();
};
var autoload = function(app){
var recover = function(error){
var name = error.message.split(' ')[0];
console.log('Loading '+name);
//A file could be synchronously loaded here instead
this[name] = function(){
console.log(name+' has been dynamically created');
};
load(app);
};
var load = function(app){
try {
app();
} catch (error){
if (error.name == "ReferenceError"){
console.log(error.message);
recover(error, app);
}
}
};
load(app);
};
autoload(app);
</script>
</body>
</html>
How It's Supposed To Work
The idea is that all of your application code would get executed within the app function. Eventually, if I can get it working properly, you could also pass in a dependency map to autoloader with the app function to synchronously load dependencies when a function is not defined. The dependency map would simply be an object mapping function names to file names.
How It Currently Works
If you don't feel like trying it out, the above code outputs the following to the console:
Initialize App
Test is not defined
Loading Test
Initialize App
Test has been dynamically created
Foo is not defined
Loading Foo
Initialize App
Test has been dynamically created
Foo has been dynamically created
Bar is not defined
Loading Bar
Initialize App
Test has been dynamically created
Foo has been dynamically created
Bar has been dynamically created
The complete app function is re-executed each time the autoloader catches an error. Obviously, this is less than ideal for a number of reasons.
Recovering From The Error
To move to the next step for making this work, I need to find a way to recover from the error without re-executing the entire app function. The error object from the catch block does provide both the line number and file name where the error occurred, but so far, I haven't been able to find a way to take advantage of that information. There are three general approaches that I can think of:
Restart script execution at the given line
Restart script execution at the beginning, but skip all lines until the given line
Grab the file as a string, split it into an array by line number, and eval the remaining lines.
Unfortunately, I wasn't able to find information on either of the first two approaches. Of the three, #1 seems like it would be more ideal, but I would certainly be open to other creative suggestions as well. As far as I can tell, JavaScript doesn't provide a way to start script execution at an arbitrary line number. #3 might work, but I'm not sure it would be very performant. The only way I can think of doing it would be to require an extra request each time to load the file text into a string.
The Questions
This is admittedly pushing the boundaries of how dependencies could be loaded in JavaScript. I'm not even sure if it is possible because I don't know if JavaScript allows for this type of error recovery. That said, I'm interested in exploring it further until I find out it's absolutely impossible.
In the interests of getting this working:
Is there a way to start script execution at an arbitrary line of JavaScript?
Are there other approaches to this that might be more fruitful? Feel free to be creative!
Taking a step back to look at the bigger picture (assuming I can get this to work):
Is a JavaScript autoloader something that people would even want?
What are the advantages/disadvantages to an autoloader like this vs. an approach like AMD?
What kind of performance issues would be encountered with this approach? Would the performance hit be too much to make it worth it?
It's kind of complicated to make, throwing and catching is kind of expensive as well. You could use typeof window["Test"]!=="function" and then create it instead of using the try catch like this.
But for a general recover and continue approach the following code would do the trick.
var app = (function(i){
var objects=new Array(3),
fnNames=["Test","Foo","Bar"];
return function(){
var len=fnNames.length
,test,foo,bar;
//check if closure vars have been reset, if so
// this has ran successfully once so don't do anything?
if(objects===false){
console.log("nothing to do, initialized already");
return;
}
while(i<len){
try{
objects[i] = new window[fnNames[i]]();
i++;
}catch(e){
if (e.name == "TypeError"){//different syntax different error
throw {"fnName":fnNames[i]};
}
}
}
//store instances in the variables
test = objects[0];
foo = objects[1];
bar = objects[2];
//reset closure vars assuming you only call app once
// when it's successful
i=null;objects=false;
console.log("Got all the instances",test,foo,bar);
};
}(0));
var autoload = function(app){
var recover = function(name){
console.log('Loading '+name);
//A file could be synchronously loaded here instead
this[name] = function(){
this.name=name;
console.log(this.name+' has been dynamically created');
};
load(app);
};
var load = function(app){
try {
app();
} catch (error){
//if statement here no longer needed
// object thrown has fnName (function name)
recover(error.fnName, app);
}
};
load(app);
};
autoload(app);
autoload(app);
I need to provide with error logging in my Windows 8 Metro application developed in Html/WinJS
so that user can get to know what went wrong from a log file located in the app's local folder.
I have checked WinJS.log(message, tags, type); which will write to the console but not able to find anything via which i can get it on a local file.
What is the best way to do the same and if there are any 3rd party libraries/js available for error logging in metro applications developed in WinJS ?
Thanks in advance.
WinJS.log is just a placeholder. Without proper initialization it does nothing (in fact, it's not set at all). If you just call WinJS.Utilities.startLog() at your application startup, it defaults to wiring up a logger for the console.
If you want something more complete, you'll need to build it. I've built a small sample below.
function startFileLog() {
// choose where the file will be stored:
var fileDestination = Windows.Storage.ApplicationData.current.localFolder;
var logger = new WinJS.Promise(function (complete) {
var logfilename = new Date().toISOString().replace(/[:-]/g, "");
logfilename = "log-" + logfilename + ".log";
fileDestination.createFileAsync(logfilename,
Windows.Storage.CreationCollisionOption.generateUniqueName)
.done(function (file) {
complete(file);
});
});
var actionFn = function (message, tag, type) {
logger.then(function (file) {
var m = WinJS.Utilities.formatLog(message, tag, type);
Windows.Storage.FileIO.appendTextAsync(file, m).done();
});
};
WinJS.Utilities.startLog({ action: actionFn });
}
By calling the startFileLog function above, it creates a new log file (by using the current Date/time as part of the file name) within a promise. Then, a function called actionFn is passed to the startLog function. By passing an optional property of the options named action, the default "write to console" behavior is overwritten (if you didn't want it overwritten, you could call startLog without the action, then copy the function reference from WinJS.log and replace it with your own function, and call it as well). When the log function is called, it now calls actionFn which uses the promise created earlier to verify that the log file is in fact available for writing before continuing. If it's not ready yet, it will be queued. So, this means that even though the file may not be ready immediately, the log will, in the end, contain the results you'd expect. There would be a short period of time where, due to async nature of WinJS, if the application crashed before the file completely opened, that logged items will be missed. You could delay the application startup if you wanted until the file was opened by returning the logger promise:
function startFileLog() {
/// ... etc..
return logger;
}
startFileLog().then(function() {
// the application can now be assured that the log file is ready to accept
// writes ... (but again, it's all async, so a write may be missed in
// extreme cases)
});
You'd likely want to create a function at the end of your application to clean/close the log file.
I'm wondering if it's possible to sandbox JavaScript running in the browser to prevent access to features that are normally available to JavaScript code running in an HTML page.
For example, let's say I want to provide a JavaScript API for end users to let them define event handlers to be run when "interesting events" happen, but I don't want those users to access the properties and functions of the window object. Am I able to do this?
In the simplest case, let's say I want to prevent users calling alert. A couple of approaches I can think of are:
Redefine window.alert globally. I don't think this would be a valid approach because other code running in the page (i.e., stuff not authored by users in their event handlers) might want to use alert.
Send the event handler code to the server to process. I'm not sure that sending the code to the server to process is the right approach, because the event handlers need to run in the context of the page.
Perhaps a solution where the server processes the user defined function and then generates a callback to be executed on the client would work? Even if that approach works, are there better ways to solve this problem?
Google Caja is a source-to-source translator that "allows you to put untrusted third-party HTML and JavaScript inline in your page and still be secure."
Have a look at Douglas Crockford's ADsafe:
ADsafe makes it safe to put guest code (such as third party scripted advertising or widgets) on any web page. ADsafe defines a subset of JavaScript that is powerful enough to allow guest code to perform valuable interactions, while at the same time preventing malicious or accidental damage or intrusion. The ADsafe subset can be verified mechanically by tools like JSLint so that no human inspection is necessary to review guest code for safety. The ADsafe subset also enforces good coding practices, increasing the likelihood that guest code will run correctly.
You can see an example of how to use ADsafe by looking at the template.html and template.js files in the project's GitHub repository.
I created a sandboxing library called jsandbox that uses web workers to sandbox evaluated code. It also has an input method for explicitly giving sandboxed code data it wouldn't otherwise be able to get.
The following is an example of the API:
jsandbox
.eval({
code : "x=1;Math.round(Math.pow(input, ++x))",
input : 36.565010597564445,
callback: function(n) {
console.log("number: ", n); // number: 1337
}
}).eval({
code : "][];.]\\ (*# ($(! ~",
onerror: function(ex) {
console.log("syntax error: ", ex); // syntax error: [error object]
}
}).eval({
code : '"foo"+input',
input : "bar",
callback: function(str) {
console.log("string: ", str); // string: foobar
}
}).eval({
code : "({q:1, w:2})",
callback: function(obj) {
console.log("object: ", obj); // object: object q=1 w=2
}
}).eval({
code : "[1, 2, 3].concat(input)",
input : [4, 5, 6],
callback: function(arr) {
console.log("array: ", arr); // array: [1, 2, 3, 4, 5, 6]
}
}).eval({
code : "function x(z){this.y=z;};new x(input)",
input : 4,
callback: function(x) {
console.log("new x: ", x); // new x: object y=4
}
});
An improved version of RyanOHara's web workers sandbox code, in a single file (no extra eval.js file is necessary).
function safeEval(untrustedCode)
{
return new Promise(function (resolve, reject)
{
var blobURL = URL.createObjectURL(new Blob([
"(",
function ()
{
var _postMessage = postMessage;
var _addEventListener = addEventListener;
(function (obj)
{
"use strict";
var current = obj;
var keepProperties =
[
// Required
'Object', 'Function', 'Infinity', 'NaN', 'undefined', 'caches', 'TEMPORARY', 'PERSISTENT',
// Optional, but trivial to get back
'Array', 'Boolean', 'Number', 'String', 'Symbol',
// Optional
'Map', 'Math', 'Set',
];
do
{
Object.getOwnPropertyNames(current).forEach(function (name)
{
if (keepProperties.indexOf(name) === -1)
{
delete current[name];
}
});
current = Object.getPrototypeOf(current);
}
while (current !== Object.prototype)
;
})(this);
_addEventListener("message", function (e)
{
var f = new Function("", "return (" + e.data + "\n);");
_postMessage(f());
});
}.toString(),
")()"],
{type: "application/javascript"}));
var worker = new Worker(blobURL);
URL.revokeObjectURL(blobURL);
worker.onmessage = function (evt)
{
worker.terminate();
resolve(evt.data);
};
worker.onerror = function (evt)
{
reject(new Error(evt.message));
};
worker.postMessage(untrustedCode);
setTimeout(function ()
{
worker.terminate();
reject(new Error('The worker timed out.'));
}, 1000);
});
}
Test it:
https://jsfiddle.net/kp0cq6yw/
var promise = safeEval("1+2+3");
promise.then(function (result) {
alert(result);
});
It should output 6 (tested in Chrome and Firefox).
As mentioned in other responces, it's enough to jail the code in a sandboxed iframe (without sending it to the server-side) and communicate with messages.
I would suggest to take a look at a small library I created mostly because of the need to providing some API to the untrusted code, just like as described in the question: there's an opportunity to export the particular set of functions right into the sandbox where the untrusted code runs. And there's also a demo which executes the code submitted by a user in a sandbox:
http://asvd.github.io/jailed/demos/web/console/
I think that js.js is worth mentioning here. It's a JavaScript interpreter written in JavaScript.
It's about 200 times slower than native JavaScript, but its nature makes it a perfect sandbox environment. Another drawback is its size – almost 600 KB, which may be acceptable for desktops in some cases, but not for mobile devices.
All the browser vendors and the HTML5 specification are working towards an actual sandbox property to allow sandboxed iframes -- but it's still limited to iframe granularity.
In general, no degree of regular expressions, etc. can safely sanitise arbitrary user provided JavaScript as it degenerates to the halting problem :-/
An ugly way, but maybe this works for you:
I took all the globals and redefined them in the sandbox scope, as well I added the strict mode so they can't get the global object using an anonymous function.
function construct(constructor, args) {
function F() {
return constructor.apply(this, args);
}
F.prototype = constructor.prototype;
return new F();
}
// Sanboxer
function sandboxcode(string, inject) {
"use strict";
var globals = [];
for (var i in window) {
// <--REMOVE THIS CONDITION
if (i != "console")
// REMOVE THIS CONDITION -->
globals.push(i);
}
globals.push('"use strict";\n'+string);
return construct(Function, globals).apply(inject ? inject : {});
}
sandboxcode('console.log( this, window, top , self, parent, this["jQuery"], (function(){return this;}()));');
// => Object {} undefined undefined undefined undefined undefined undefined
console.log("return of this", sandboxcode('return this;', {window:"sanboxed code"}));
// => Object {window: "sanboxed code"}
https://gist.github.com/alejandrolechuga/9381781
An independent JavaScript interpreter is more likely to yield a robust sandbox than a caged version of the built-in browser implementation.
Ryan has already mentioned js.js, but a more up-to-date project is JS-Interpreter. The documentation covers how to expose various functions to the interpreter, but its scope is otherwise very limited.
As of 2019, vm2 looks like the most popular and most regularly-updated solution to running JavaScript in Node.js. I'm not aware of a front-end solution.
With NISP you'll be able to do sandboxed evaluation.
Though the expression you write is not exactly JavaScript code, instead you'll write S-expressions. It is ideal for simple DSLs that doesn't demand extensive programming.
Suppose you have code to execute:
var sCode = "alert(document)";
Now, suppose you want to execute it in a sandbox:
new Function("window", "with(window){" + sCode + "}")({});
These two lines when executed will fail, because "alert" function is not available from the "sandbox"
And now you want to expose a member of window object with your functionality:
new Function("window", "with(window){" + sCode + "}")({
'alert':function(sString){document.title = sString}
});
Indeed you can add quotes escaping and make other polishing, but I guess the idea is clear.
Where is this user JavaScript code coming from?
There is not much you can do about a user embedding code into your page and then calling it from their browser (see Greasemonkey). It's just something browsers do.
However, if you store the script in a database, then retrieve it and eval() it, then you can clean up the script before it is run.
Examples of code that removes all window. and document. references:
eval(
unsafeUserScript
.replace(/\/\/.+\n|\/\*.*\*\/, '') // Clear all comments
.replace(/\s(window|document)\s*[\;\)\.]/, '') // Removes window. Or window; or window)
)
This tries to prevent the following from being executed (not tested):
window.location = 'http://example.com';
var w = window;
There are a lot of limitations you would have to apply to the unsafe user script. Unfortunately, there isn't any 'sandbox container' available for JavaScript.
I've been working on a simplistic JavaScript sandbox for letting users build applets for my site. Although I still face some challenges with allowing DOM access (parentNode just won't let me keep things secure =/), my approach was just to redefine the window object with some of its useful/harmless members, and then eval() the user code with this redefined window as the default scope.
My "core" code goes like this... (I'm not showing it entirely ;)
function Sandbox(parent){
this.scope = {
window: {
alert: function(str){
alert("Overriden Alert: " + str);
},
prompt: function(message, defaultValue){
return prompt("Overriden Prompt:" + message, defaultValue);
},
document: null,
.
.
.
.
}
};
this.execute = function(codestring){
// Here some code sanitizing, please
with (this.scope) {
with (window) {
eval(codestring);
}
}
};
}
So, I can instantiate a Sandbox and use its execute() function to get code running. Also, all new declared variables within eval'd code will ultimately bound to the execute() scope, so there will not be clashing names or messing with existing code.
Although global objects will still be accessible, those which should remain unknown to the sandboxed code must be defined as proxies in the Sandbox::scope object.
You can wrap the user's code in a function that redefines forbidden objects as parameters -- these would then be undefined when called:
(function (alert) {
alert ("uh oh!"); // User code
}) ();
Of course, clever attackers can get around this by inspecting the JavaScript DOM and finding a non-overridden object that contains a reference to the window.
Another idea is scanning the user's code using a tool like JSLint. Make sure it's set to have no preset variables (or: only variables you want), and then if any globals are set or accessed do not let the user's script be used. Again, it might be vulnerable to walking the DOM -- objects that the user can construct using literals might have implicit references to the window object that could be accessed to escape the sandbox.