Develop Reference

Javascript custom template engine

I'm trying to create a custom template engine in javascript but I'm having trouble getting started as I cannot extract tokens using regex.
Here are the requirements:
Variables are defined like this: $(variable)
Functions: $(name arg1 "this is arg2 but it contains whitespaces.")
Function arguments can contain other variables $(name $(variable) arg2)
Both variables and functions will be rendered async. For example: Get the value for $(variable) from db then replace it.
This is not for rendering an html page but to simply replace a string entered by a user on the backend.
Edit
More information:
Suppose a user enters the following string:
$(id $(lowercase John))
On the backend application must do:
Convert "John" to lowercase.
Get the id for "john" from db.
This is only a simple example to demonstrate how this should work.
Are there any libraries that can help me achieve this? If not, any idea how to implement this?
EDIT 2:
I tried using Mustache and I changed the delimiters to $(), however the function (section) tags do no satisfy the requirements. In Mustache, for functions I must do this: $(#name) $(variable) "this is arg2 but it contains whitespaces."$(/name) also it does not support async rendering.

If not, any idea how to implement this?
You should use an Abstract Syntax Tree, and write a compatible parser. While regex (as Pedro Lima stated) is good for simple templating, if you ever want to extend the parser, you'll need something a bit more robust.
As an example of an Abstract Syntax Tree parser, $(test1 $(test2) test3) lorem ipsum $(test4) would be turned into the following:
(Thanks to Mile Shang's Syntree for the tree generator.)
As for specifically how to write a parser, I think you can figure it out. Just iterate over the string and check for the template delimiter. Reading the source code for a templating library like Handlebars might help.

Here. This regex will identify the templates that can be replaced. Note that it only selects the innermost templates in nested templates.
/\$\((?<FirstTerm>\S+?)(?<OtherTerms>(?:\s+(?:\w+|".*?"))+)?\)/g
So just use a regex replace function with your templating logic recursively until there are no more matches. The inner templates will be replaced and you'll be left with the string with templates replaced.

Other answers on this post are correct, however, I want to share exactly how I managed to implement this:
Create a recursive match function. I used Steven Leviathan's article to implement this.
Create a render function and inside the function call the recursive match function to find and replace variable/function names with appropriate values.
Keep calling the render function recursively until all arguments inside a function have been replaced.

Format integer thousands separator throughout the document

After months of web-development, I find myself completely helpless trying to find a good solution for a simple problem of formatting all the numbers throughout the DOM as I wish. Specifically, I have a js function my_int_formatter(), that I want to apply to all integers after the doc has been loaded. Best descriped by example - I want to do something like
<td>my_int_formatter({{django_variable}})</td>
I know the code above won't work, because I have to include 'script' tag, but first, I don't like the messy code, and second, javascript won't recognize python variable
I tried the following way:
HTML
<td class = 'my_integer'>{{django_variable}}</td>
JS
$(document).ready(function(){
// ....
content = $('.my_integer').html();
$('.my_integer').html(my_int_formatter(content));
...but as expected, I got wrong results because the js code applied the same html() content of the first .my_integer element in the DOM chain to all the others. Any ideas how to do this the short and correct way ?

If I understand correctly, your problem isn't with the formatting but actualy applying the formatting to each of your dom elements.
Try using jquerys .each() function and using $(this).html() to actualy grab the content.
$('.my_integer').each(function(){
content = $(this).html();
$(this).html(content+"formatted");
});
here's a quick fiddle :
https://jsfiddle.net/57rdq2a0/2/

If I understand you correctly, you want to use builtin django.contrib.humanize application: https://docs.djangoproject.com/en/1.9/ref/contrib/humanize/
You can format integers using some predefined filters, for example intcomma:
4500 becomes 4,500.
45000 becomes 45,000.
450000 becomes 450,000.
4500000 becomes 4,500,000.
Usage in your case would be like
{% load humanize %}
<td>{{django_variable|intcomma}}</td>
Also don't forget to include the app in INSTALLED_APPS
Also this question might be useful
If you want to apply filter to all variables of some kind, I suggest you to use Middleware to fiddle with response before rendering.

Is there a better way of creating elements and appending text to/around them?

I currently have a bunch of lines that look like:
txt = "Can't print the value for <span class='keyword'>"+arguments[1]+"</span> before it's set";
I'm then doing
$('#mydiv').append($('<div/>').html(txt));
This looks terrible and I need to escape any html inside arguments[1]
The only alternative I can think of is to make sure all the text is inside its own element:
var spans = [];
spans[0] = $('<span/>').text("Can't print the value for ");
spans[1] = $('<span/>').text(arguments[1]).className('keyword');
spans[2] = $('<span/>').text(" before it's set");
$('#mydiv').append($('<div/>').append(spans[0],spans[1],spans[2]));
This is quite a lot for just a simple line of text. Is there anything else I can do?
edit: This isn't something that should be handled by a templating engine. It's html generated by a javascript logging function.

If It's a consistent format, I'd add it as a normal string and then do a search for the keyword part.
$('<div/>')
.appendTo('#mydiv')
.html("Can't print the value for <span class='keyword'></span> before it's set")
.find('.keyword').text(arguments[1]);

If you will be continuing to create lots of HTML using JS, I would suggest working with a templating library. I am a recent convert, it took me a long time to understand the point. But seeing many successful sites (twitter,github,etc.) and the great John Resig promote and/or make heavy use of templating, i'm glad I stuck with trying to understand. Now I get it. It's for separation of concerns, keeping logic out of the view.
I'm using this very bare bones templating library: http://blueimp.github.com/JavaScript-Templates/ though the templating provided by underscore.js and mustache.js are more popular.
The advantage of the library i'm using is its really small, <1kb and is basically like writing php/asp code if you are familiar with those.
you can write HTML inside <script> tags without having to escape:
using your variable, txt, the syntax looks like this:
<script>
var data={txt : "Can't print the value for <span class='keyword'>"+arguments[1]+"</span> before it's set"};
<div>{%=o.txt%}</div>
</script>

escape() doesn't seem to work consistently?

using javascript, I generate HTML code, for example adding an function which starts by clicking a link, like:
$('#myDiv').append('click');
So start() should be called if somebody hits the link (click).
TERM could contain a single word, like world or moody's, the generated HTML code would look like:
click
OR
click
As you can see, the 2nd example will not work. So i decided to "escape" the TERM, like so:
$('#myDiv').append('click');
Looking at the HTML source-code using firebug, is see, that the following code was generated:
click
Thats works fine, until I really click the link - so the browser (here firefox) seams to interpret the %27 and tries to fire start('moody's');
Is there a way to escape the term persistent without interpreting the %27 until the term is handled in JS? Is there an other solution instead of using regular expressions to change ' to \'?

Don't try to generate inline JavaScript. That way lies too much pain and maintenance hell. (If you were to go down that route, then you would escape characters in JavaScript strings with \).
Use standard event binding routines instead.
Assuming that $ is jQuery, and not one of the many other libraries that use that unhelpful variable name:
$('#myDiv').append(
$('<a>').append("click").attr('href', 'A sensible fallback').click(function (e) {
alert(TERM); // Because I don't have the function you were calling
e.preventDefault();
})
);
See also http://jsfiddle.net/TudEw/

escape() is used for url-encoding stuff, not for making it possible to put in a string literal. Your code is seriously flawed for several reasons.
If you want an onclick event, use an onclick event. Do not try to "inject" javascript code with your markup. If you have the "string" in a variable, you should never need to substitute anything in it unless you are generating urls or other restricted terms.
var element = $('<span>click</span>');
element.bind('click', function () { start(TERM); });
$('#myDiv').append(element);
If you don't know what this does, then go back to basic and learn what events and function references in javascript means.

That escape() function is for escaping url's for passing over a network, not strings. I don't know that there's a built-in function to escape strings for JavaScript, but you can try this one I found online: http://www.willstrohl.com/Blog/EntryId/67/HOW-TO-Escape-Single-Quotes-for-JavaScript-Strings.
Usage: EscapeSingleQuotes(strString)
Edit: Just noticed your note about regular expressions. This solution does use regular expressions, but I think there's nothing wrong with that :-)

How to generate RDF/XML using a template in javascript?

I have some working Javascript code that generates an RDF/XML document using variables picked up from HTML fields:
...
peopleDoap += " <foaf:name>" + person_name + "</foaf:name>\n";
if (person_url != "") {
peopleDoap += " <foaf:homepage rdf:resource=\"" + person_url + "/\"/>\n";
}
if (person_pic != "") {
peopleDoap += " <foaf:depiction rdf:resource=\"" + person_pic + "/\"/>\n";
}
...
It's hard, looking at this code, to get any sense of what the output will look like (especially as this code is scattered amongst sub functions etc).
I'm wondering if there is an easy way that would enable me to have something like this:
...
<foaf:name>%person_name%</foaf_name>
<foaf:homepage rdf:resource="%person_url%"/>
<foaf:depiction rdf:resource="%person_pic%"/>
...
And then some substitution code. One slight complication is if fields are left blank, I will want to not generate the whole element. Ie, if person_url='', the above should generate as:
...
<foaf:name>%person_name%</foaf_name>
<foaf:depiction rdf:resource="%person_pic%"/>
...
I guess I could do this fairly naively by defining the template as a huge string, then performing a bunch of replaces on it, but is there anything more elegant? Mild preference for native Javascript rather than libraries, but happy to be convinced...
(Btw, yes, since this is RDF/XML, maybe there is a smarter way using some kind of RDF library. If you want to address that of the question instead, that's ok with me.)
Code is here.
Also, this is a widget running on a Jetty server. I don't think server-side code is an option.

I recommend using:
jQuery Templates
Mustache
John Resig's Micro-Templating
jQuery Templates are very powerful and nicely integrated with jQuery. That means that you can do things like this:
$.tmpl("Hello ${n}", {n: "World"}).appendTo('h1');
for the most simple stuff, or define templates in your HTML inside special script tags with custom MIME types, compile them, populate them with JSON data from AJAX calls, etc.

To add a bit of follow-up, I did implement John Resig's Micro-Templating (actually the refined version I posted earlier). However, I then backpedalled a bit. I found implementing control structures in the template is less readable than outside:
...
'<description xml:lang="en">#description</description>';
if (homepage) t +=
'<homepage rdf:resource="#homepage"/>';
...
rather than:
...
'<description xml:lang="en"><#= description #></description>' +
'<# if (homepage) { #>' +
'<homepage rdf:resource="<#= homepage =>"/>' +
'<# } #>';
...
I also ditched the microtemplating code for a simple substitution of variables, using #var rather than <# var #>.
Readability of templates like this is really critical, so I've done everything I could think of. In particular, keeping the javascript outside of the template lets syntax highlighting work, which is valuable to me.
That John Resig post also suggested burying the template in your HTML, in a but I preferred to keep it in my javascript, which is a separate .js.