SpiderMonkey AST javascript manipulation - javascript

I have SpiderMonkey AST, this is javascript object. I use "Esprima", "Acorn" or other libs to generate AST from javascript file. My AST is specified by Mozilla parser API.
I want to manipulate this AST from javascript - remove/add/edit nodes in it. How to do it safely?
For example i want to replace all boolean Literal with !1 or !0.
Now i use walker from "Acorn" lib:
acornWalk.simple(tree, {
Literal: function(node, scope) {
if(typeof node.value === 'boolean'){
var expressionTrue = {
"type": "UnaryExpression",
"operator": "!",
"prefix": true,
"argument": {
"type": "Literal",
"value": 0,
"raw": "0"
}
},
expressionFalse = {
"type": "UnaryExpression",
"operator": "!",
"prefix": true,
"argument": {
"type": "Literal",
"value": 1,
"raw": "1"
}
},
replace = function(node, newNode){
for(var prop in newNode){
node[prop] = newNode[prop];
}
};
replace(node, node.value ? expressionTrue : expressionFalse);
}
}
});
But it seems to be not good practice because the "start", "end" properties in node is not changed and it can break code generated from that AST. Am i right?

Related

How to convert a grammar into a parsing algorithm using a table?

Say I have a grammar like this:
match start
match first
match rule a
match rule b
match a
match string, "a"
match b
match string, "b("
match optional
match rule start
match optional
match many
match string, ","
match rule start
match string, ")"
It compiles into JSON like this:
{
"start": {
"type": "rule",
"name": "start",
"children": [
{
"type": "match-first",
"children": [
{
"type": "match-rule",
"rule": "a"
},
{
"type": "match-rule",
"rule": "b"
}
]
}
]
},
"a": {
"type": "rule",
"name": "a",
"children": [
{
"type": "match-string",
"value": {
"type": "string",
"value": "a"
}
}
]
},
"b": {
"type": "rule",
"name": "b",
"children": [
{
"type": "match-string",
"value": {
"type": "string",
"value": "b("
}
},
{
"type": "match-optional",
"children": [
{
"type": "match-rule",
"rule": "start"
}
]
},
{
"type": "match-optional",
"children": [
{
"type": "match-many",
"children": [
{
"type": "match-string",
"value": {
"type": "string",
"value": ","
}
},
{
"type": "match-rule",
"rule": "start"
}
]
}
]
},
{
"type": "match-string",
"value": {
"type": "string",
"value": ")"
}
}
]
}
}
It can match any of these strings theoretically:
a
b()
b(a)
b(a,a)
b(b(b()))
b(a,b(a,a,b(),a,a))
How do I generate an optimized parser (that simply returns true/false on whether or not the input string matches) out of this grammar JSON? My initial inkling is to try and do this like my recursive descent parser that just takes a grammar and a string and parses using the "uncompiled" grammar. But it was pointed out that this would be inefficient and is why "parser-generators" generate parsers specific to each grammar. How can that be done using this grammar provided in JSON form? Could it involve "parse tables" somehow? I am not really sure how this could be done. Ideally it would be done without recursion, somehow just looping and shifting the parser state appropriately.
I am not really sure how to begin because I don't see what the desired structure is supposed to be from the output of a generateParser(grammar) function.
function generateParser(grammar) {
const table = []
let x = 0
for (let key in grammar) {
let rule = grammar[key]
for (let i = 0, n = rule.children.length; i < n; i++) {
let pattern = rule.children[i]
switch (pattern.type) {
case 'match-first':
addMatchFirst(pattern)
break
}
}
}
return function parse(string) {
// iterate through table and transition somehow?
}
function addMatchFirst(p) {
table.push((state) => {
state.stack.push(/* new state? */)
})
for (let i = 0, n = p.children.length; i < n; i++) {
let pattern = p.children[i]
switch (pattern.type) {
case 'match-rule':
addMatchRule(pattern)
break
}
}
}
function addMatchRule(p) {
table.push((state) => {
state.stack.push(/* new state? */)
})
}
}
Looking at a parse table implementation in JavaScript seems to primitive and I don't see how to apply it.
If it's not possible to make this using a table, how else would you generate the parsing algorithm? All I am looking for is a true/false answer from the generated parser on whether or not the input string matches the grammar.
How you can parse depends on the semantics of your grammar notation. It has various constructs where there are alternatives:
For a "match first" line, the alternatives are its children.
For a "match optional" line, the alternatives are its child and the empty string.
For a "match many" line, the alternatives are the number of repetitions of its child.
If, at every point where there are alternatives, the semantics are that each alternative is "valid", then your notation basically defines a Context Free Grammar, so a large set of parsing options are available to you.
However, the wording of the "match first" line suggests that it means "try the 'children' in the order given and use the first that succeeds". If that's the case, then it looks like your notation is defining a Parsing Expression Grammar, so the parsing options are more limited. Packrat parsers seem popular.
It's unclear if you could use a table-based parsing algorithm such as you link to -- the Wikipedia article says "It is also possible to build LL parsers and LR parsers from parsing expression grammars", which implies that you could use an LL or LR parsing table. However, there's no citation for that statement, and I'm doubtful that it's true.
Did someone give you the generateParser code as a start at a parser implementation? It has a variable named table, but note that it isn't a table in the LL or LR sense, though there might be some similarities. At any rate, it appears to have some useful chunks of code.

Append multiple lines after a certain pattern to several JSON files

I have several JSON files that look like this
{
"$schema": "someURL",
"id": "someURL",
"type": "object",
"properties": {
"copyright": {
"id": "someURL",
"type": "object",
"description": "Setup for copyright link",
"properties": {
"translation": {
"id": "someURL",
"type": "string",
"description": "someString"
},
"url": {
"id": "someURL",
"type": "string",
"description": "someString"
}
}...
what I need to do is add a removable and appendable attribute to each item inside every instance of properties and set them to true. so the output should look like this:
{
"$schema": "someURL",
"id": "someURL",
"type": "object",
"properties": {
"copyright": {
"removable": true,
"appendable": true,
"id": "someURL",
"type": "object",
"description": "Setup for copyright link",
"properties": {
"translation": {
"removable": true,
"appendable": true,
"id": "someURL",
"type": "string",
"description": "someString"
},
"url": {
"removable": true,
"appendable": true,
"id": "someURL",
"type": "string",
"description": "someString"
}
}...
Is there a way to automate this? as in write a script that automatically adds these fields right below each item in properties?
using NodeJS I would write something like this:
const fs = require('fs');
let args = process.argv.map(val=>val));
let contents = fs.readFileSync(args[0]);
let obj = JSON.parse(contents);
const proc = (obj, append)=>{
for (prop in obj)
{
let p = obj[prop];
if (p.constructor!=Object)
continue;
if (append)
{
p.removable = true;
p.appendable = true;
}
proc(p, prop=='properties');
}
};
console.log(JSON.stringify(proc(obj), null, 2));
And then in bash do:
find -name '*\.json' -exec 'node proc.js {} > {}.new'
I didn't test anything, everything is from my head, but for starting point should be good enough.
Given that you want to apply the update to all objects that have a key named "properties", wherever they occur, I'd be inclined to use walk/1. In any case, to make things clearer and maybe easier, it will be helpful to define a helper function that will apply the update if the input is of the right type:
def update(obj):
if type == "object" and has("properties")
then .properties |= with_entries( .value += obj )
else .
end;
Using walk/1, the solution is now trivial:
walk( update({removable: true, appendable: true}) )
Robustification
It might be prudent to change the "then" line above to:
then .properties |=
with_entries( if .value | type == "object"
then .value += obj
else . end)

Apply regex expression to a json response data

Scenario client make a GET request the response is in a JSON format like this one
var data = {
"enabled": true,
"state": "schedule",
"schedules": [
{
"rule": {
"start": "2014-06-29T12:36:26.000",
"end": "2014-06-29T12:36:56.000",
"recurrence": [
"RRULE:FREQ=MINUTELY"
]
},
"wifi_state_during_rule": "disabled",
"end_state": "enabled"
}
],
"calculated_wifi_state_now": "disabled",
"time_of_next_state_change": [
"2014-07-08T18:56:56.000Z",
"2014-07-08T18:57:56.000Z"
]
};
For the purpose of this example I stored the result in a variable called "data".
My regex Expressions is:
checkPattern = /"\w+\"(?=:)/ //all keys "keyname": ...
The basic ideia here its just to get the keynames besides being inside of and object or array...since the definition of keyname's in JSON is "keyname": that's why I'm trying to use the above regex expression.
I even thought about doing this with a recursive function but is not working.
You never should parse non-regular structures with regular expressions.
Just collect what you want from parsed json object.
Just run data = JSON.parse(json_string) for parse it
function getKeysRecursive(obj) {
var result = [];
for (var key in obj) {
result.push(key);
if (typeof obj[key] == 'object') {
result = result.concat(getKeysRecursive(obj[key]));
}
}
return result;
}
getKeysRecursive(({
"enabled": true,
"state": "schedule",
"schedules": [
{
"rule": {
"start": "2014-06-29T12:36:26.000",
"end": "2014-06-29T12:36:56.000",
"recurrence": [
"RRULE:FREQ=MINUTELY"
]
},
"wifi_state_during_rule": "disabled",
"end_state": "enabled"
}
],
"calculated_wifi_state_now": "disabled",
"time_of_next_state_change": [
"2014-07-08T18:56:56.000Z",
"2014-07-08T18:57:56.000Z"
]
}))
// ["enabled", "state", "schedules", "0", "rule", "start", "end", "recurrence", "0", "wifi_state_during_rule", "end_state", "calculated_wifi_state_now", "time_of_next_state_change", "0", "1"]
You can filter them, sort, exclude numeric keys... All what you need.
You wont need a regular expression for this . Javascript has a built in function to extract Object key names .
Example :
Use Object.keys();
var data = {
"enabled": true,
"state": "schedule",
"schedules": [
{
"rule": {
"start": "2014-06-29T12:36:26.000",
"end": "2014-06-29T12:36:56.000",
"recurrence": [
"RRULE:FREQ=MINUTELY"
]
},
"wifi_state_during_rule": "disabled",
"end_state": "enabled"
}
],
"calculated_wifi_state_now": "disabled",
"time_of_next_state_change": [
"2014-07-08T18:56:56.000Z",
"2014-07-08T18:57:56.000Z"
]
};
Then
console.log(Object.keys(data));
should print
["enabled","state","schedules","calculated_wifi_state_now","time_of_next_state_change"]
Proof: http://codepen.io/theConstructor/pen/zBpWak
Now all of your object keys are stored in an array ..
Hope this helps
This will give you all of the keys as the matches:
\"(\w+)(?:\"\:)
https://regex101.com/r/iM9wB3/2
Edited to work with multiple keys on one line.

Get environment specific configuration from a JSON object using Lodash

Given that I have the following JSON object,
dbConfig = {
"db": "default",
"default": {
"defaultDB": "sqlite",
"init": "init",
"migrations": {
"directory": "migrations",
"tableName": "migrations"
},
"pool": {
"min": "2",
"max": "10"
},
"sqlite": {
"client": "sqlite3",
"connection": {
"filename": "data/default/sqlitedb/test.db"
}
},
"oracle": {
"client": "oracledb",
"config": {
"development": {
"user": "test",
"pass": "test",
"db": "test"
},
"production": {
"user": "test",
"pass": "test",
"db": "test"
},
"test": {
"user": "test",
"pass": "test",
"db": "test"
}
}
}
}
};
Using Node & Lodash, is there any possibility of getting either connection or config. depending on what dbConfig.default[dbConfig.default.defaultDB] is set to.
So for instance if i set dbConfig.default.defaultDB=oracledb and process.env.NODE_ENV=development I want to be able to get dbConfig.default[dbConfig.default.defaultDB].config.development
Or if I set dbConfig.default.defaultDB=sqlite just to get dbConfig.default[dbConfig.default.defaultDB].connection
In other words, if the database has environment specific configuration then this will be in "config": {} and if not in "connection": {}
It doesn't have to be Lodash. It can also be plain javascript.
Solution without lodash
var defaultDbName = dbConfig.default[dbConfig.default.defaultDB];
var db;
if (defaultDb === 'sqllite') {
db = dbConfig.default[defaultDb].connection;
} else {
var env = process.env.NODE_ENV;
db = dbConfig.default[defaultDb].config[env];
}
Solution with lodash
Here I'm using lodash get function to get object field value or null if it doesn't exist. Also I'm using template string syntax: ${val} to format field path.
var defaultDbName = dbConfig.default[dbConfig.default.defaultDB];
var defaultDbConf = dbConfig.default[defaultDb];
var env = process.env.NODE_ENV;
var db = defaultDbConf.connection || _.get(defaultDbConf, `config.${env}`);
Btw, your configuration json is too complex, much better to have configuration per environment.
Solution without [dependencies] (originally answered here, but not AngularJS-specific)
Your JSON is complex, yes, but it could also be smaller and more readable without all the duplication, where each environment has the same set of attributes, which may or may not vary, and would be needlessly duplicated.
With a simple algorithm (jsFiddle) you can dynamically parse your JSON configuration for specific property-name suffixes (property#suffix) and have a catalogue of environment-varying properties alongside non-varying properties, without artificially structuring your configuration and without repetition, including deeply-nested configuration objects.
You can also mix-and-match suffixes and combine any number of environmental or other arbitrary factors to groom your configuration object.
Example, snippet of pre-processed JSON config:
var config = {
'help': {
'BLURB': 'This pre-production environment is not supported. Contact Development Team with questions.',
'PHONE': '808-867-5309',
'EMAIL': 'coder.jen#lostnumber.com'
},
'help#www.productionwebsite.com': {
'BLURB': 'Please contact Customer Service Center',
'BLURB#fr': 'S\'il vous plaît communiquer avec notre Centre de service à la clientèle',
'BLURB#de': 'Bitte kontaktieren Sie unseren Kundendienst!!1!',
'PHONE': '1-800-CUS-TOMR',
'EMAIL': 'customer.service#productionwebsite.com'
},
}
... and post-processed (given location.hostname='www.productionwebsite.com' and navigator.language of 'de'):
prefer(config,['www.productionwebsite.com','de']); // prefer(obj,string|Array<string>)
JSON.stringify(config); // {
'help': {
'BLURB': 'Bitte kontaktieren Sie unseren Kundendienst!!1!',
'PHONE': '1-800-CUS-TOMR',
'EMAIL': 'customer.service#productionwebsite.com'
}
}
Obviously you can pull those values at render-time with location.hostname and window.navigator.language. The algorithm to process the JSON itself isn't terribly complex (but you may still feel more comfortable with an entire framework for some reason, instead of a single function):
function prefer(obj,suf) {
function pr(o,s) {
for (var p in o) {
if (!o.hasOwnProperty(p) || !p.split('#')[1] || p.split('##')[1] ) continue; // ignore: proto-prop OR not-suffixed OR temp prop score
var b = p.split('#')[0]; // base prop name
if(!!!o['##'+b]) o['##'+b] = 0; // +score placeholder
var ps = p.split('#')[1].split('&'); // array of property suffixes
var sc = 0; var v = 0; // reset (running)score and value
while(ps.length) {
// suffix value: index(of found suffix in prefs)^10
v = Math.floor(Math.pow(10,s.indexOf(ps.pop())));
if(!v) { sc = 0; break; } // found suf NOT in prefs, zero score (delete later)
sc += v;
}
if(sc > o['##'+b]) { o['##'+b] = sc; o[b] = o[p]; } // hi-score! promote to base prop
delete o[p];
}
for (var p in o) if(p.split('##')[1]) delete o[p]; // remove scores
for (var p in o) if(typeof o[p] === 'object') pr(o[p],s); // recurse surviving objs
}
if( typeof obj !== 'object' ) return; // validate
suf = ( (suf || suf === 0 ) && ( suf.length || suf === parseFloat(suf) ) ? suf.toString().split(',') : []); // array|string|number|comma-separated-string -> array-of-strings
pr(obj,suf.reverse());
}
The property name suffix can have any number of suffixes after the '#', delimited by '&' (ampersand) and, where there are two properties with different but preferred suffixes, will be preferred in the order in which they are passed to the function. Suffixes that contain BOTH preferred strings will be preferred above all others. Suffixes found in the JSON that are not specified as preferred will be discarded.
Preference/discrimination will be applied top-down on your object tree, and if higher-level objects survive, they will be subsequently inspected for preferred suffixes.
With this approach, your JSON (I'm making some assumptions about which attributes vary between your environments and which do not) might be simplified as follows:
dbConfig = {
"pool": {
"min": "2",
"max": "10"
},
"init": "init",
"migrations": {
"directory": "migrations",
"tableName": "migrations"
},
"db":
"client": "sqlite",
"filename": "data/default/sqlitedb/development.db"
"filename#tst": "data/default/sqlitedb/test.db"
"filename#prd": "data/default/sqlitedb/production.db"
},
"db#oracle": {
"client": "oracle",
"user": "devuser",
"user#tst": "testdbuser",
"user#prd": "testdbuser",
"pass": "devpass",
"pass#tst": "testdbpass",
"pass#prd": "testdbpass",
"db": "devdb",
"db#tst": "testdbschema",
"db#prd": "testdbschema"
}
};
So that you could feed this into the prefer() function with these args+results:
for sqlite, test env:
prefer(dbConfig,'tst');
JSON.stringify(dbConfig); // dbConfig: {
"pool": {
"min": "2",
"max": "10"
},
"init": "init",
"migrations": {
"directory": "migrations",
"tableName": "migrations"
},
"db": {
"client": "sqlite",
"filename": "data/default/sqlitedb/test.db"
}
};
for oracle, default/development environment:
prefer(dbConfig,'oracle'); // oracle, dev(default) env
JSON.stringify(dbConfig); // dbConfig: {
"pool": {
"min": "2",
"max": "10"
},
"init": "init",
"migrations": {
"directory": "migrations",
"tableName": "migrations"
},
"db": {
"client": "oracle",
"user": "devdbuser",
"pass": "devdbpass",
"db": "devdbschema"
}
};
prefer(dbConfig,'oracle,prd'); // oracle, production env
JSON.stringify(dbConfig); // dbConfig: {
"pool": {
"min": "2",
"max": "10"
},
"init": "init",
"migrations": {
"directory": "migrations",
"tableName": "migrations"
},
"db": {
"client": "oracle",
"user": "prddbuser",
"pass": "prddbpass",
"db": "prddbschema"
}
};
Abstract usage and examples:
var o = { 'a':'apple', 'a#dev':'apple-dev', 'a#fr':'pomme',
'b':'banana', 'b#fr':'banane', 'b#dev&fr':'banane-dev',
'c':{ 'o':'c-dot-oh', 'o#fr':'c-point-oh' }, 'c#dev': { 'o':'c-dot-oh-dev', 'o#fr':'c-point-oh-dev' } };
/*1*/ prefer(o,'dev'); // { a:'apple-dev', b:'banana', c:{o:'c-dot-oh-dev'} }
/*2*/ prefer(o,'fr'); // { a:'pomme', b:'banane', c:{o:'c-point-oh'} }
/*3*/ prefer(o,'dev,fr'); // { a:'apple-dev', b:'banane-dev', c:{o:'c-point-oh-dev'} }
/*4*/ prefer(o,['fr','dev']); // { a:'pomme', b:'banane-dev', c:{o:'c-point-oh-dev'} }
/*5*/ prefer(o); // { a:'apple', b:'banana', c:{o:'c-dot-oh'} }
Caveats
Usage of the # in property name is NOT standard and is invalid in dot-notation, but so far has not broken any browsers we've tested this in. The UPSIDE of this is that it prevents developers from expecting they can refer to your pre-processed, suffixed attributes. A developer would have to be aware of, and a bit unconventional and refer to your attribute as a string (obj['key#suf']) to do that, which, by the way, is the reason this function is possible.
If future JavaScript engines reject it, substitute for any other tolerable convention, just be consistent.
This algorithm has not been profiled for performance, or rigorously tested for other potential problems.
In its current form, used one-time on startup/load, we have yet to run into problems with.
As always, YMMV.

Delete object from json tree by reference in JavaScript

I need to remove an object from an JSON tree. I know a reference to that object. Is there a nice way to do it via JavaScript or jQuery besides traversing the whole tree?
Example:
party = {
"uuid": "4D326531-3C67-4CD2-95F4-D1708CE6C7A8",
"link": {
"rel": "self",
"href": "http://localhost:8080/cim/party/4D326531-3C67-4CD2-95F4-D1708CE6C7A8"
},
"type": "PERSON",
"name": "John Doe",
"properties": {
"CONTACT": [
{
"category": "CONTACT",
"type": "EMAIL",
"key": "email",
"value": "john.doe#doe.at",
"id": "27DDFF6E-5235-46BF-A349-67BEC92D6DAD"
},
{
"category": "CONTACT",
"type": "PHONE",
"key": "mobile",
"value": "+43 999 999990 3999",
"id": "6FDAA4C6-9340-4F11-9118-F0BC514B0D77"
}
],
"CLIENT_DATA": [
{
"category": "CLIENT_DATA",
"type": "TYPE",
"key": "client_type",
"value": "private",
"id": "65697515-43A0-4D80-AE90-F13F347A6E68"
}
]
},
"links": []
}
And i have a reference: contact = party.properties.contact[1]. And I want to do something like delete contact.
You may delete it this way. I just tested it.
var party = {
// ...
}
alert(party.properties.CONTACT[0]) // object Object
delete party.properties.CONTACT[0] // true
alert(party.properties.CONTACT[0]) // undefined
Fiddle
UPDATE
In the case above party is a direct property of window object
window.hasOwnProperty('party'); // true
and that's why you can't delete a property by reference. Anyhow, behavior of delete operator with host objects is unpredictable. Though, you may create a scope around the party object and then you'll be allowed to delete it.
var _scope = {};
var _scope.party = {
// ...
};
var r = _scope.party.properties.CONTACT[0];
window.hasOwnProperty('party'); // false
alert(r) // object Object
delete r // true
alert(r) // undefined
It only works one way: a variable holds a reference, but there is no way given a particular reference to infer what variables hold it (without iterating over them and comparing).

Categories

Resources