I'm trying to create a custom template engine in javascript but I'm having trouble getting started as I cannot extract tokens using regex.
Here are the requirements:
Variables are defined like this: $(variable)
Functions: $(name arg1 "this is arg2 but it contains whitespaces.")
Function arguments can contain other variables $(name $(variable) arg2)
Both variables and functions will be rendered async. For example: Get the value for $(variable) from db then replace it.
This is not for rendering an html page but to simply replace a string entered by a user on the backend.
Edit
More information:
Suppose a user enters the following string:
$(id $(lowercase John))
On the backend application must do:
Convert "John" to lowercase.
Get the id for "john" from db.
This is only a simple example to demonstrate how this should work.
Are there any libraries that can help me achieve this? If not, any idea how to implement this?
EDIT 2:
I tried using Mustache and I changed the delimiters to $(), however the function (section) tags do no satisfy the requirements. In Mustache, for functions I must do this: $(#name) $(variable) "this is arg2 but it contains whitespaces."$(/name) also it does not support async rendering.
If not, any idea how to implement this?
You should use an Abstract Syntax Tree, and write a compatible parser. While regex (as Pedro Lima stated) is good for simple templating, if you ever want to extend the parser, you'll need something a bit more robust.
As an example of an Abstract Syntax Tree parser, $(test1 $(test2) test3) lorem ipsum $(test4) would be turned into the following:
(Thanks to Mile Shang's Syntree for the tree generator.)
As for specifically how to write a parser, I think you can figure it out. Just iterate over the string and check for the template delimiter. Reading the source code for a templating library like Handlebars might help.
Here. This regex will identify the templates that can be replaced. Note that it only selects the innermost templates in nested templates.
/\$\((?<FirstTerm>\S+?)(?<OtherTerms>(?:\s+(?:\w+|".*?"))+)?\)/g
So just use a regex replace function with your templating logic recursively until there are no more matches. The inner templates will be replaced and you'll be left with the string with templates replaced.
Other answers on this post are correct, however, I want to share exactly how I managed to implement this:
Create a recursive match function. I used Steven Leviathan's article to implement this.
Create a render function and inside the function call the recursive match function to find and replace variable/function names with appropriate values.
Keep calling the render function recursively until all arguments inside a function have been replaced.
Related
What is the name of the native function that handles template literals?
That is, I know that when you write tag`Foo ${'bar'}.`;, that’s just syntactic sugar for tag(['Foo ', '.'], 'bar');.¹
But what about just `Foo ${'bar'}.`;? I can’t just “call” (['Foo ', '.'], 'bar');. If I already have arguments in that form, what function should I pass them to?
I am only interested in the native function that implements the template literal functionality. I am quite capable of rolling my own, but the purpose of this question is to avoid that and do it “properly”—even if my implementation is a perfect match of current native functionality, the native functionality can change and I want my usage to still match. So answers to this question should take on one of the following forms:
The name of the native function to use, ideally with links to and/or quotes from documentation of it.
Links to and/or quotes from the spec that defines precisely what the implementation of this function is, so that if I roll my own at least I can be sure it’s up to the (current) specifications.
A backed-up statement that the native implementation is unavailable and unspecified. Ideally this is backed up by, again, links to and/or quotes from documentation, but if that’s unavailable, I’ll accept other sources or argumentation that backs this claim up.
Actually, the first argument needs a raw property, since it’s a TemplateStringsArray rather than a regular array, but I’m skipping that here for the sake of making the example more readable.
Motivation
I am trying to create a tag function (tag, say) that, internally, performs the default template literal concatenation on the input. That is, I am taking the TemplateStringsArray and the remaining arguments, and turning them into a single string that has already had its templating sorted out. (This is for passing the result into another tag function, otherTag perhaps, where I want the second function to treat everything as a single string literal rather than a broken up template.)
For example, tag`Something ${'cooked'}.`; would be equivalent to otherTag`Something cooked.`;.
My current approach
The definition of tag would look something like this:
function tag(textParts, ...expressions) {
const cooked = // an array with a single string value
const raw = // an array with a single string value
return otherTag({ ...cooked, raw });
}
Defining the value of raw is fairly straightforward: I know that String.raw is the tag function I need to call here, so const raw = [String.raw(textParts.raw, ...expressions)];.
But I cannot find anywhere on the internet what function I would call for the cooked part of it. What I want is, if I have tag`Something ${'cooked'}.`;, I want const cooked = `Something ${cooked}.`; in my function. But I can’t find the name of whatever function accomplishes that.
The closest I’ve found was a claim that it could be implemented as
const cooked = [expressions.map((exp, i) => textParts[i] + exp).join('')];
This is wrong—textParts may be longer than expressions, since tag`Something ${'cooked'}.`; gets ['Something ', '.'] and ['cooked'] as its arguments.
Improving this expression to handle that isn’t a problem:
const cooked = [
textParts
.map((text, i) => (i > 0 ? expressions[i-1] : '') + text)
.join(''),
];
But that’s not the point—I don’t want to roll my own here and risk it being inconsistent with the native implementation, particularly if that changes.
The name of the native function to use, ideally with links to and/or quotes from documentation of it.
There isn't one. It is syntax, not a function.
Links to and/or quotes from the spec that defines precisely what the implementation of this function is, so that if I roll my own at least I can be sure it’s up to the (current) specifications.
Section 13.2.8 Template Literals of the specification explains how to process the syntax.
I have an amp-list which loads bunch of data and I show them in their respective placeholders just nice and easy. What I intend to do is get a value and run a simple script on it. Let's think I have
<div>{{title}}</div>
where title is: 'This-is-my-title'
now I would like to replace the '-' in title, I know I could do it with javascript using title.replace(/-/g,' '), how can I do that in place?
I tried
<div>{{title.replace(/-/g,' ')}}</div>
but no luck :(
In plain javascript the following:
title = 'This-is-my-title'; title.replace(/-/g, ' ');
gives you "This is my title".
I am guessing you are using angular, in that case the text within {{ }} is not evaluated as a pure javascript expression. You could write an angular filter to apply to the expresssion (as described in Angular filter to replace all underscores to spaces ). It would probably be easier to handle this in the controller behind the template. Something like:
$scope.title = $scope.title.replace(/-/g,' ');
Looks like you are using amp-mustache. I don't think there is a way for you to use custom JavaScript in Mustache.js here, and restrictions from AMP prevent you to create some kind of function that you can call in {{}}. I would suggest processing in the backend before sending. (Also unfortunately, there are no other templates other than mustache available at this point)
There is a workaround on math using amp-bind here: AMP Mustache and Math
So probably after loading the JSON with amp-state, something like
myItems.map(entry => ({
myString: entry.myString.split('').map(c => c == '-' ? ' ' : c).join('')),
})
might work (I have not tested myself but worth a try, check whitelisted functions here: https://www.ampproject.org/es/docs/reference/components/amp-bind#white-listed-functions) but might still be a pain performance-wise (amp-bind has quite a bit overhead)
Edit: this actually looks quite promising, just found out actually amp-list with amp-bind do accept object for [src], as described in the doc (learning something new myself): https://www.ampproject.org/docs/reference/components/amp-bind
(checked amp-list source code and should work)
I have run into a problem with a current project using Handlebars.js. I am writing a complex template which must be output only as text (it is fed directly as a string to a text printer) and that text can be a maximum of 40 chars wide.
The problem I have is that I need to use various helper functions to get and organise my data, but I then need to take that text and put it though another function to format it into the 40 char width.
So... I have helper functions that look a bit like this:
Handlebars.registerHelper('getLit' , function (litName) {
// some logic to retrieve a string lit in correct language
});
Handlebars.registerHelper('getArrayValue', function(array, key) {
return array[key];
});
Handlebars.registerHelper('textFormat', function(string, max_width, align) {
// logic to format the text
});
Now if my htm looks like this:
{{textFormat "This is a really long string that needs formating into the correcct length blah blah blah blah" 40 'left'}}
I have no problems.
However I need to be able to use helper function to build the string e.g.
{{textFormat {{getArrayValue address 0}} 40 'right'}}
I hope my explanation is not too convoluted, obviously the data I am dealing with is very large and very complex so simply preparing all the data to feed into the template (e.g. instead of using getLit making 6 forms of each string in their various languages) is just not practical.
Is there anyway to make this work or do I need to use a totally different approach?
You cannot use nested helpers. If you need include new helper, use construct
{{#textformat 10 'right'}}
{{getArrayValue address 0}}
{{/textformat}}
You can find out more about constructing block helpers here: http://handlebarsjs.com/block_helpers.html
I have a good knowledge of real time graphics programming and web development, and I've started a project that requires me to take a user-created conditional string and actually use those conditions in code. This is an entirely new kind of programming problem for me.
I've tried a few experiments using loops and slicing up the conditional string...but I feel like I am missing some kind of technique that would make this more efficient and straightforward. I have a feeling regular expressions may be useful here, but perhaps not.
Here is an example string:
"IF#VAR#>=2AND$VAR2$==1OR#VAR3#<=3"
The values for those actual variables will come from an array of objects. Also, the different marker symbols around the variables denote different object arrays where the actual value can be found (variable name is an index).
I have complete control over how the conditional string is formatted (adding symbols around IF/ELSE/ELSEIF AND/OR
well as special symbols around the different operands) so my options are fairly open. How would you approach such a programming problem?
The problem you're facing is called parsing and there are numerous solutions to it. First, you can write your own "interpreter" for your mini-language, including lexer (which splits the string into tokens), parser (which builds a tree structure from a stream of tokens) and executor, which walks the tree and computes the final value. Or you can use a parser generator like PEG and have the whole thing built for you automatically - you just provide the rules of your language. Finally, you can utilize javascript built-in parser/evaluator eval. This is by far the simplest option, but eval only understands javascript syntax - so you'll have to translate your language to javascript before eval'ing it. And since eval can run arbitrary code, it's not for use in untrusted environments.
Here's an example on how to use eval with your sample input:
expr = "#VAR#>=2AND$VAR2$==1OR#VAR3#<=3"
vars = {
"#": {"VAR":5},
"$": {"VAR2":1},
"#": {"VAR3":7}
}
expr = expr.replace(/([##$])(\w+)(\1)/g, function($0, $1, $2) {
return "vars['" + $1 + "']." + $2;
}).replace(/OR/g, "||").replace(/AND/g, "&&")
result = eval(expr) // returns true
using javascript, I generate HTML code, for example adding an function which starts by clicking a link, like:
$('#myDiv').append('click');
So start() should be called if somebody hits the link (click).
TERM could contain a single word, like world or moody's, the generated HTML code would look like:
click
OR
click
As you can see, the 2nd example will not work. So i decided to "escape" the TERM, like so:
$('#myDiv').append('click');
Looking at the HTML source-code using firebug, is see, that the following code was generated:
click
Thats works fine, until I really click the link - so the browser (here firefox) seams to interpret the %27 and tries to fire start('moody's');
Is there a way to escape the term persistent without interpreting the %27 until the term is handled in JS? Is there an other solution instead of using regular expressions to change ' to \'?
Don't try to generate inline JavaScript. That way lies too much pain and maintenance hell. (If you were to go down that route, then you would escape characters in JavaScript strings with \).
Use standard event binding routines instead.
Assuming that $ is jQuery, and not one of the many other libraries that use that unhelpful variable name:
$('#myDiv').append(
$('<a>').append("click").attr('href', 'A sensible fallback').click(function (e) {
alert(TERM); // Because I don't have the function you were calling
e.preventDefault();
})
);
See also http://jsfiddle.net/TudEw/
escape() is used for url-encoding stuff, not for making it possible to put in a string literal. Your code is seriously flawed for several reasons.
If you want an onclick event, use an onclick event. Do not try to "inject" javascript code with your markup. If you have the "string" in a variable, you should never need to substitute anything in it unless you are generating urls or other restricted terms.
var element = $('<span>click</span>');
element.bind('click', function () { start(TERM); });
$('#myDiv').append(element);
If you don't know what this does, then go back to basic and learn what events and function references in javascript means.
That escape() function is for escaping url's for passing over a network, not strings. I don't know that there's a built-in function to escape strings for JavaScript, but you can try this one I found online: http://www.willstrohl.com/Blog/EntryId/67/HOW-TO-Escape-Single-Quotes-for-JavaScript-Strings.
Usage: EscapeSingleQuotes(strString)
Edit: Just noticed your note about regular expressions. This solution does use regular expressions, but I think there's nothing wrong with that :-)