Sweet.js macro that calls a method - javascript

I'm trying to write a sweet.js macro which needs to generate method call syntax, obj.method(), but the method is passed in to the macro as a literal expression. For example:
mcall(obj, toString().length);
// becomes
obj.toString().length;
I've got something that's close:
macro mcall {
rule { ($o, $m:expr) } => { $o.$m }
}
mcall(obj, toString().length);
However, this apparently expands to this:
obj . ( toString ( ) . length );
Where are these extra parentheses coming from, and how do I get rid of them? Should I be using case rules and #{}? I tried permutations of that but still couldn't succeed at generating a method call without extra parentheses.

So currently in sweet.js tokens bound to an :expr pattern variable get wrapped in parens to help with getting precedence to work correctly. This is kind of a hack and will get fixed soon (some more discussion here: https://github.com/mozilla/sweet.js/issues/314).
The simple solution for your example is to just not use :expr since you don't really need it:
macro mcall {
rule { ($o, $m ...) } => { $o.$m ... }
}
Side note: using :expr is technically wrong here since the RHS of a . is not allowed to be an unrestricted expression (eg 2+4 matches $m:expr but obj.2+4 is a syntax error).

Related

What is the default “tag” function for template literals?

What is the name of the native function that handles template literals?
That is, I know that when you write tag`Foo ${'bar'}.`;, that’s just syntactic sugar for tag(['Foo ', '.'], 'bar');.¹
But what about just ​`Foo ${'bar'}.`;? I can’t just “call” (['Foo ', '.'], 'bar');. If I already have arguments in that form, what function should I pass them to?
I am only interested in the native function that implements the template literal functionality. I am quite capable of rolling my own, but the purpose of this question is to avoid that and do it “properly”—even if my implementation is a perfect match of current native functionality, the native functionality can change and I want my usage to still match. So answers to this question should take on one of the following forms:
The name of the native function to use, ideally with links to and/or quotes from documentation of it.
Links to and/or quotes from the spec that defines precisely what the implementation of this function is, so that if I roll my own at least I can be sure it’s up to the (current) specifications.
A backed-up statement that the native implementation is unavailable and unspecified. Ideally this is backed up by, again, links to and/or quotes from documentation, but if that’s unavailable, I’ll accept other sources or argumentation that backs this claim up.
Actually, the first argument needs a raw property, since it’s a TemplateStringsArray rather than a regular array, but I’m skipping that here for the sake of making the example more readable.
Motivation
I am trying to create a tag function (tag, say) that, internally, performs the default template literal concatenation on the input. That is, I am taking the TemplateStringsArray and the remaining arguments, and turning them into a single string that has already had its templating sorted out. (This is for passing the result into another tag function, otherTag perhaps, where I want the second function to treat everything as a single string literal rather than a broken up template.)
For example, tag`Something ${'cooked'}.`; would be equivalent to otherTag`Something cooked.`;.
My current approach
The definition of tag would look something like this:
function tag(textParts, ...expressions) {
const cooked = // an array with a single string value
const raw = // an array with a single string value
return otherTag({ ...cooked, raw });
}
Defining the value of raw is fairly straightforward: I know that String.raw is the tag function I need to call here, so const raw = [String.raw(textParts.raw, ...expressions)];.
But I cannot find anywhere on the internet what function I would call for the cooked part of it. What I want is, if I have tag`Something ${'cooked'}.`;, I want const cooked = `Something ${cooked}.`; in my function. But I can’t find the name of whatever function accomplishes that.
The closest I’ve found was a claim that it could be implemented as
const cooked = [expressions.map((exp, i) => textParts[i] + exp).join('')];
This is wrong—textParts may be longer than expressions, since tag`Something ${'cooked'}.`; gets ['Something ', '.'] and ['cooked'] as its arguments.
Improving this expression to handle that isn’t a problem:
const cooked = [
textParts
.map((text, i) => (i > 0 ? expressions[i-1] : '') + text)
.join(''),
];
But that’s not the point—I don’t want to roll my own here and risk it being inconsistent with the native implementation, particularly if that changes.
The name of the native function to use, ideally with links to and/or quotes from documentation of it.
There isn't one. It is syntax, not a function.
Links to and/or quotes from the spec that defines precisely what the implementation of this function is, so that if I roll my own at least I can be sure it’s up to the (current) specifications.
Section 13.2.8 Template Literals of the specification explains how to process the syntax.

Dialogflow system.entity location: location.admin-area is not defined in online editor

I use system entity #sys.location in an intent in a Dialogflow agent. In the fulfillment section, I have this function in online code editor:
function testLocation(agent) {
//check object location
console.log(' location is ' + JSON.stringify(agent.parameters.location));
if(agent.parameters.location.city) {
//do smthing
}
else if (agent.parameters.location.admin-area){
agent.add(`this is not recognized ` +agent.parameters.location.admin-area);
}else{
//....
}
}
Point is that I receive a warning sign in the editor saying 'area is not defined', but I can see its values from the Firebase Console :
{"country":"","city":"","admin-area":"Piemonte","business-name":"","street-address":"","zip-code":"","shortcut":"","island":"","subadmin-area":""}
Any clues?
Thanks in advance
You're tripping over a JavaScript syntax issue.
The expression
agent.parameters.location.admin-area
is being evaluated as
agent.parameters.location.admin - area
that is to say agent.parameters.location.admin minus area, which is causing an error because, as the error says, the "area" attribute of "location" isn't defined.
In this, agent.parameters.location is an Object, and JavaScript provides two ways to access attributes of an Object
You can use bracket notation [expression] where the expression inside the brackets should evaluate to the name of an attribute of the object. Usually this needs to be a string.
In some cases, you can use dot notation .name where the name is the name of the property. But this assumes the name doesn't have characters used by other JavaScript syntax.
Note the difference between "expression" and "name". The first lets you use a variable with a string in it, or something else you've computed. The second requires you to hardcode it.
In your case, you can use bracket notation to get the value you want. So something like
agent.parameters.location["admin-area"]
should work.

ES2015 template strings security issue

Here's a quote from MDN:
Template strings MUST NOT be constructed by untrusted users, because they have access to variables and functions.
And an example:
`${console.warn("this is",this)}`; // "this is" Window
let a = 10;
console.warn(`${a+=20}`); // "30"
console.warn(a); // 30
The example here doesn't show any vulnerabilities I can see.
Can anyone give an example of an exploit that takes advantage of this?
This makes no sense. A template string doesn't have access to anything, it is not executed either. A template string is a syntactical element of the language.
Dynamically constructing a template string is no problem therefore - it's like building an expression (in whatever format, be it a code string or an AST). The problem MDN hints at is with evaluating such an expression (e.g. using eval, serialising it into a script that is served to the user, etc.) - it may contain arbitrary code, in contrast to a string literal! But of course you wouldn't do that anyway, would you?
This warning is like saying "Concatenations using the + operator must not be constructed by untrusted users, because they have access to variables and functions." and giving the example "" + console.warn("this is",this) + "" for it. Well, this is true for any expression of the language, so it's not particularly interesting.
While we are talking about crappy coding, there is of course a scenario where using template strings (hey, they're multiline and whatnot) instead of string literals can lead to problems:
function escapeString(str) {
return JSON.stringify(str).slice(1, -1)
.replace(/\u2028/g, "\\u2028").replace(/\u2029/g, "\\u2029");
}
// This is (kinda) fine!
var statement = 'var x = "Hello,\\n'+escapeString(userInput)+'";';
eval(statement); // some kind of evaluation
// But this is not:
var statement = 'var x = `Hello,\n'+escapeString(userInput)+'`;';
// ^ ^
Now imagine userInput contains a ${…} - which we did not escape…
I think #Bergi is correct - the danger here involves using eval or similar methods to allow a user to construct the actual template string, not the substitutions.
Example exploit: A lazy developer wants to allow users to perform some string substitutions in their comments, e.g. to reference other users or questions, on a site like SO. Instead of developing tokens for this and then doing appropriate parsing and substitution, he decides he'll accept syntax like this:
"I think ${firstPoster} is an idiot! See ${question(1234)} for details!"
and run it through a function like this:
var firstPoster = {...};
function question() {...}
processInput(input) {
return eval('`' + input + '`');
}
If this code is eval'd on the client and shown to other users, a malicious user could inject an XSS attack. If it's eval'd on the server, the attacker could take control of the machine.
The example no longer seems to be in the MDN docs. As Bergi's answer points out, the given example doesn't seem to highlight anything special about template strings.
However, when it comes to building strings from objects, there is a particular security concern you should definitely be aware of:
The toString() method will be called implicitly on a non-string object if it is passed to a string interpolation / concatenation expression.
There are probably also other cases where toString() is called implicitly. But string interpolation seems to me to be one of the most common ones; indeed, it's one I experience regularly. For example, suppose you somehow receive an object from somewhere external e.g. via an iframe through postMessage. In that case, you might want to do something like log the message received to the console - and you might be tempted to just pass the object right into an interpolated string.
But the sender (possibly an attacker) has full control over the definition of toString() and can insert whatever code they like in there. So as soon as you pass that object to the interpolated or concatenated string, you are vulnerable as long as someone else controls the definition of said object.
Here is a simple example (added to codesandbox), showing that even if objects return seemingly harmless strings in toString(), they can indeed do dangerous stuff like read you local storage:
import "./styles.css";
localStorage.setItem("secret1", "sssh! One");
localStorage.setItem("secret2", "sssh! Two");
const evilObject1 = {
toString() {
alert("I stole a secret: " + localStorage.getItem("secret1"))
return "I'm innocent";
}
};
const evilObject2 = {
toString() {
alert("I stole a secret: " + localStorage.getItem("secret2"))
return "I'm innocent";
}
};
const strInter = `Seemingly innocent object, interpolated: ${evilObject1}`;
const strConcat = "Seemingly innocent object, concatenated: " + evilObject2;
let p = document.createElement("p");
p.innerHTML = strInter
let p2 = document.createElement("p");
p2.innerHTML = strConcat
document.body.appendChild(p);
document.body.appendChild(p2);
With Javascript, being duck-typed, this is quite a real vulnerability, because you might think that the object you receive (e.g. via postMessage) is a string, and indeed, it may behave like a string (because it has a cleverly designed toString() method), but unless you dynamically check the types you don't know if you actually got a string.
If you're expecting a string, you can fix the above vulnerability as follows:
const sanitized1 = typeof evilObject1 === 'string' ? evilObject1 : "BAD OBJECT1"
const sanitized2 = typeof evilObject2 === 'string' ? evilObject2 : "BAD OBJECT2"
const strInter = `Seemingly innocent object, interpolated: ${sanitized1}`;
const strConcat = "Seemingly innocent object, concatenated: " + sanitized2;
With this fix, you avoid implicitly calling toString() on an unsafe object. Either the objects you got were strings, and those will be used, or else they weren't, and you'll get the "BAD OBJECT" text instead.

JSLint "eval is evil." alternatives

I am have some JavaScript functions that run on both the client (browser) and the server (within a Java Rhino context). These are small functions - basically little validators that are well defined and don't rely upon globals or closures - self-contained and portable.
Here's an example:
function validPhoneFormat(fullObject, value, params, property) {
var phonePattern = /^\+?([0-9\- \(\)])*$/;
if (value && value.length && !phonePattern.test(value))
return [ {"policyRequirement": "VALID_PHONE_FORMAT"}];
else
return [];
}
To keep things DRY, my server code gets a handle on each of these functions and calls toString() on them, returning them to the browser as part of a JSON object. Something like this:
{ "name" : "phoneNumber",
"policies" : [
{ "policyFunction" : "\nfunction validPhoneFormat(fullObject, value, params, property) {\n var phonePattern = /^\\+?([0-9\\- \\(\\)])*$/;\n if (value && value.length && !phonePattern.test(value)) {\n return [{\"policyRequirement\":\"VALID_PHONE_FORMAT\"}];\n } else {\n return [];\n }\n}\n"
}
]
}
My browser JS code then takes this response and creates an instance of this function in that context, like so:
eval("var policyFunction = " + this.policies[j].policyFunction);
policyFailures = policyFunction.call(this, form2js(this.input.closest("form")[0]), this.input.val(), params, this.property.name));
This all works very well. However, I then run this code through JSLint, and I get back this message:
[ERROR] ValidatorsManager.js:142:37:eval is evil.
I appreciate that often, eval can be dangerous. However, I have no idea how else I could implement such a mechanism without using it. Is there any way I can do this and also pass through the JSLint validator?
I wouldn't worry about it since you are only passing these function strings from the server to the client, and are thus in control of what will be evaluated.
On the other hand, if you were going the other direction and doing the evals of client-passed code on the server, that would be an entirely different story...
Update:
As disabling the validation option in your comment may cause you to miss future errors, I would instead suggest passing the function name rather than the entire function and have the function library mirrored on the server and client. Thus, to call the function, you'd use the following code:
var policyFunction = YourLibraryName[this.policies[j].policyFunctionName];
var policyArguments = this.policies[j].policyArguments;
policyFunction.apply(this, policyArguments);
Update 2:
I was able to validate the following code with JSLint successfully, which essentially allows you to "turn off" validation for the vast minority of cases where eval is appropriate. At the same time, JSLint still validates normal eval calls, and all uses of this method should throw up flags for future developers to avoid using it/refactor it out where possible/as time allows.
var EVAL_IS_BAD__AVOID_THIS = eval;
EVAL_IS_BAD__AVOID_THIS(<yourString>);
Dont encode a function as a string in JSON. JSON is for content, which you are confounding with behavior.
Instead, I suppose you could return JS files instead, which allow real functions:
{ name : "phoneNumber",
policies : [
{ policyFunction : function() {
whateverYouNeed('here');
}
}
]
}
But while that solves the technical issue, it's still not a great idea.
The real solution here is to move your logic out of your content entirely. Import a JS file full of little validation functions and call them as needed based on a dataType property in your JSON or something. If this functions are as small and portable as you say, this should be trivial to accomplish.
Getting your data all tangled up with your code usually leads to pain. You should statically include your JS, then dynamically request/import/query for your JSON data to run through your statically included code.
I would avoid using eval in all situations. There's no reason you can't code around it. Instead of sending code to the client, just keep it hosted on the server in one contained script file.
If that's not doable, you can also have a dynamically generated javascript file then pass in the necessary parameters via the response, and then dynamically load the script on the client side. There's really no reason to use eval.
Hope that helps.
You can use
setInterval("code to be evaluated", 0);
Internally, if you pass setInterval a string it performs a function similar to eval().
However, I wouldn't worry about it. If you KNOW eval() is evil, and take appropriate precautions, it's not really a problem. Eval is similar to GoTo; you just have to be careful and aware of what you're doing to use them properly.
With very little parsing you could have had it like so:
var body = this.policies[j].policyFunction.substr;
body = body.substr(body.indexOf("(") + 1);
var arglist = body.substr(1, body.indexOf(")"));
body = body.substr(arglist.length + 1);
var policyFunction = new Function(arglist, body);
Which would provide a bit of validation, avoid the literal use of eval and work synchronously with the code. But it is surely eval in disguise, and it is prone to XSS attack. If the malevolent person can get their code loaded and evaluated this way - it will not save you. So, really, just don't do it. Add a <script> tag with the proper URL and that would be certainly safer. Well, you know, better safe then sorry.
PS. My apologises if the code above doesn't work, it only shows the intent, I've not tested it, and if I made a mistake at counting parenthesis or some such - well, you should get the idea, I'm not advertising it by any means.
DRY is definitely something I agree with, however there is a point where copy+pasting is more efficient and easy to maintain than referencing the same piece of code.
The code you're saving yourself from writing seems to be equivalent to a clean interface, and simple boiler plate. If the same code is being used on both the server and the client, you could simply pass around the common pieces of the function, rather than the whole function.
Payload:
{
"name": "phoneNumber",
"type": "regexCheck",
"checkData": "/^\\+?([0-9\\- \\(\\)])*$/"
}
if(payload.type === "regexCheck"){
const result = validPhoneFormat(fullObject, value, payload.checkData)
}
function validPhoneFormat(fullObject, value, regexPattern) {
if (value && value.length && !regexPattern.test(value))
return [ {"policyRequirement": "VALID_PHONE_FORMAT"}];
else
return [];
}
This would give you the ability to update the regex from a single location. If the interface changes it does need to be updated in 2 places, but I wouldn't consider that a bad thing. If the client is running code, why hide the structure?
If you really, really want to keep both the object structure and the patterns in one place - extract it to a single API. Have a "ValidatePhoneViaRegex" api endpoint which is called by all places you'd be passing this serialized function to.
If all of this seems like too much effort, set jslint to ignore your piece of code:
"In JSHint 1.0.0 and above you have the ability to ignore any warning with a special option syntax. The identifier of this warning is W061. This means you can tell JSHint to not issue this warning with the /*jshint -W061 */ directive.
In ESLint the rule that generates this warning is named no-eval. You can disable it by setting it to 0, or enable it by setting it to 1."
https://github.com/jamesallardice/jslint-error-explanations/blob/master/message-articles/eval.md
I would prefer to see copy+pasted code, a common api, or receiving parameters and copy+pasted boiler plate than magical functions passed in from the server to be executed.
What happens if you get a cross-browser compatibility error with one of these shared functions?
Well, the first thing to bear in mind is that jsLint does make the point that "it will hurt your feelings". It's designed to point out where you're not following best practices -- but code that isn't perfect can still work just fine; there's no compulsion upon you to follow jsLint's advice.
Having said that, eval is evil, and in virtually all cases there is always a way around using it.
In this case, you could use a library such as require.js, yepnope.js or some other library that is designed to load a script separately. This would allow you to include the javascript functions you need dynamically but without having to eval() them.
There are probably several other solutions as well, but that was the first one that came to my mind.
Hope that helps.

I need a Javascript literal syntax converter/deobfuscation tools

I have searched Google for a converter but I did not find anything. Is there any tools available or I must make one to decode my obfuscated JavaScript code ?
I presume there is such a tool but I'm not searching Google with the right keywords.
The code is 3 pages long, this is why I need a tools.
Here is an exemple of the code :
<script>([][(![]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+([![]]+[][[]])[+!+[]+[+[]]]+(![]+[])[!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+!+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]+(!![]+[])[+[]]][([][(![]+[])[+[]]+([![]]+[][[]])[+!+[]+[+[]]]+(![]+[])[!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+!+[]]]+[])[!+[]+!+[]+!+[]]+(![]+[])[+!+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[!+[]+!+[]]]()[(!![]+[])[!+[]+!+[]+!+[]]+(+(+[])+[][(![]+[])[+[]]+([![]]+[][[]])[+!+[]+[+[]]]+(![]+[])[!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+!+[]]])[!+[]+!+[]+!+[]+[+[]]]+(![]+[])[+!+[]]+(![]+[])[!+[]+!+[]]])(([]+[])[([][(![]+[])[+[]]+([![]]+[][[]])[+!+[]+[+[]]]+(![]+[])[!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+!+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+
Thank you
This code is fascinating because it seems to use only nine characters ("[]()!+,;" and empty space U+0020) yet has some sophisticated functionality. It appears to use JavaScript's implicit type conversion to coerce arrays into various primitive types and their string representations and then use the characters from those strings to compose other strings which type out the names of functions which are then called.
Consider the following snippet which evaluates to the array filter function:
([][
(![]+[])[+[]] // => "f"
+ ([![]]+[][[]])[+!+[]+[+[]]] // => "i"
+ (![]+[])[!+[]+!+[]] // => "l"
+ (!![]+[])[+[]] // => "t"
+ (!![]+[])[!+[]+!+[]+!+[]] // => "e"
+ (!![]+[])[+!+[]] // => "r"
]) // => function filter() { /* native code */ }
Reconstructing the code as such is time consuming and error prone, so an automated solution is obviously desirable. However, the behavior of this code is so tightly bound to the JavaScript runtime that de-obsfucating it seems to require a JS interpreter to evaluate the code.
I haven't been able to find any tools that will work generally with this sort of encoding. It seems as though you'll have to study the code further and determine any patterns of usage (e.g. reliance on array methods) and figure out how to capture their usage (e.g. by wrapping high-level functions [such as Function.prototype.call]) to trace the code execution for you.
This question has already an accepted answer, but I will still post to clear some things up.
When this idea come up, some guy made a generator to encode JavaScript in this way. It is based on doing []["sort"]["call"]()["eval"](/* big blob of code here */). Therefore, you can decode the results of this encoder easily by removing the sort-call-eval part (i.e. the first 1628 bytes). In this case it produces:
if (document.cookie=="6ffe613e2919f074e477a0a80f95d6a1"){ alert("bravo"); }
else{ document.location="http://www.youtube.com/watch?v=oHg5SJYRHA0"; }
(Funny enough the creator of this code was not even able to compress it properly and save a kilobyte)
There is also an explanation of why this code doesn't work in newer browser anymore: They changed Array.prototype.sort so it does not return a reference to window. As far as I remember, this was the only way to get a reference to window, so this code is kind of broken now.

Categories

Resources