Develop Reference

PHP template system vs javascript AJAX template

My PHP template looks like this:
$html=file_get_contents("/path/to/file.html");
$replace=array(
"{title}"=>"Title of my webpage",
"{other}"=>"Other information",
...
);
foreach(replace AS $search=>$replace){
$html=str_replace($search,$replace,$html);
}
echo $html;
I am considering switching to a javascript/ajax template system. The AJAX will fetch the $replace array in JSON format and then I'll use javascript to replace the HTML.
The page would then be a plain .html file and a loading screen would be shown until the ajax was complete.
Is there any real advantages to this or is the transition a waste of time?
A few of the reasons I think this will be beneficial:
Page will still load even if the Mysql or PHP services are down. If the ajax fails I can handle it with an error message.
Bot traffic (and anything else that doesnt run JS) will cause very little load to my server since the ajax will never be sent.
Please let me know what your thoughts are.

My 2cents is it is better to do the logic on the template side (javascript). If you have a high traffic site you can off load some of the processing to each computer calling the site. Maybe less servers.
With Javascript frameworks like AngularJs the template stuff is pretty simple and efficient. And the framework will do caching for you.
Yes, SEO can be an issue with certain sites. There at proxy tools you can put in place that will render the site and return the static html to the bot. Plus I think some bots render javascript these days.
Lastly, I like to template on the front-end because I like the backend to be a generic data provider (RESTful API). This way I can build a generic backend that drives web / mobile and other platforms in a generic way. The UI logic can be its separate thing in javascript.
But it comes down to the design needs of your application. I build lots of Software as a service applications so a single page application works well for me.

I've worked with similar design pattern in other projects. There are several ways to do this and the task would involve managing multiple project or application modules. I am assume you are working with a team of developers and not using either PHP or JavaScript MVC framework.
PHP Template
For many reasons, I'm against using “search and replace” method especially using server-side scripting language to parse HTML documents as a templating kit.
Why?
As you maintain business rules and project becomes larger, you will
find yourself reading through a long list of regular expressions,
parse HTML into DOM, and/or complicated algorithms for searching
nodes to replace with correct text(s).
If you had a placeholder, such as {title}, that would help the
script to have fewer search and replace expressions but the design
pattern could lead to messy sharing with multiple developers.
It is ok to parse one or two HTML files to manage the output but not
the entire template. The network response could be slower with
multiple and repetitive trips to server and that's just only for
template. There could be other scripts that is also making trips to
the server for different reason unrelated to template.
AJAX/JavaScript
Initially, AJAX with JavaScript might sound like a neat idea but I'm still not convinced.
Why?
You can't assume web browser is JavaScript-enabled in every mobile
or desktop. You might need to structure the HTML template in few
ways to manage the output for non-JavaScript browsers. You might
need to include <noscript> and/or <iframe> tags on every page. And,
managing alternative template for non-JavaScript browser can be
tedious.
Every web browser interpret JavaScript differently. Most developers
should know few differences between IE, FireFox, Chrome, Safari, and
to name few. You might need to create multiple JavaScript files to
detect then load JavaScript for that specific web browser. You
update one feature, you have to update script for all web browsers.
JavaScript is visible in page source. I wouldn't want to display
confidential JavaScript functions that might include credentials,
leak sensitive data about web services, and/or SQL queries. The idea
is to secure your page as much as possible.
I'm not saying both are impossible. You could still do either PHP or JavaScript for templating.
However, my “ideal” web structure should consist of a reliable MVC like Zend, Spring, or Magnolia. Those MVC framework include many useful features such as web services, data mapping, and templating kits. Granted, it's difficult for beginners with configuration requirements to integrate MVC into your project. But in the end, you could delegate tasks in configurations, MVC concepts, custom SQL queries, and test cases to developers. That's my two cents.

I think the most important aspects you forgot are:
SEO : What about search engine bots ? They wont be able to index your content if it is set by javascript only.
Execution and Network Latency : When your service is working, the browser will wait until the page is loaded (let's say 800ms) before making the extra Ajax calls to get your values. This might add an extra 500ms to get it (depending on network speed and geographic location...). If you have sent all the generated data by your server, you would have spent only ~1ms more to prepare the complete response. You would have a lot of waiting on a blank page.
Caching : You could cache the generated pages on your web app. That way your load will be minimized as well. And also, if you still want to deliver content while your backend services (MySQL/PHP..) are down you could even use Apache or Nginx caching.
But I guess it really depends on what you want to do.
For fast and simple pages, which seems to be your case, stick with backend enhancements.
For a dynamic/interactive app which can afford loading times, and doesn't care about SEO, you can delegate most things to the front-end. But then use an advanced framework like Angular, to handle templating, caching, etc...

JQTouch examples show applications as monolithic one html page creations. Is there a better paradigm for organizing the code?

I'm not a big fan of the way code is organized in the jqtouch examples I can find. So far, all I've seen are monolithic "index.html" files, which contain all the separate views for the iPhone app as separate divs.
Are there any examples out there of better organized jqtouch code?
I'm not looking for generic advice - I'd like to see specific examples of differently organized code.

What you're seeing is usually thought of as a feature of JQTouch, not as a negative "monolithic" style. -- Mobile networks tend to have a large time overhead per http request, so the general idea is to use the one request to download multiple small "pages" (as divs) all at once.
Of course, this paradigm may not fit your use case...
Added Re: alternatives: There are lots of mobile frameworks, see a list or Google. For JQTouch, you can return a response that includes only a single page if you wish to. The reason you're not seeing such examples is because the whole idea of the framework is to make it easy for the developer to return multiple "pages" as a single web server response.
For your server's responses which are a set of mobile pages, the multiple pages-at-a-time trick is the usual approach. For responses which include an infinite scroll page, or which have a lot of dynamic content, you can do Ajax updating of the mobile page, esp if you limit yourself to iPhone and Android browsers.
Overall, the per-request overhead is the big issue for good mobile web app performance. Anytime you can (or probably can) avoid a browser/server round-trip, you should aggressively do so.

Accessibility and all these JavaScript frameworks

I've been investigating a few of the JavaScript frameworks such as Backbone.js and Batman.js for a while and whilst I really like them, I have one niggling thing that I keep coming back to. That issue is accessibility.
As a web developer I've always tried to make my websites and applications with accessibility in mind, especially using the idea of progressive enhancement.
Clearly out of the box these new JS frameworks don't gracefully degrade, so I was wondering what are other developers thoughts on this issue and what are you doing about it. After all the accessibility of a website / app isn't really an optional thing as it's part of the law in many countries.
Maybe I'm just being overly zealous on this subject, and not appreciating how far things have come in terms of accessibility.

I use a js-framework (spine.js in my case) in my latest site. Still I make sure that non-js browsers (certainly not over zealous: think SEO) can navigate my site and digest the contents.
As an example I'm going with a search-page with products being shown. Products can be paged, filtered, sorted. Of course this is an example of the generalized idea.
PREREQ: use a template-engine that can both render server-side and client-side. (I use Mustache) . This makes sure you can render models without js- through server-side templating, and render models with js through client-side templating.
Initially: render the products using server-side mustache-template. Also include a 'bootstrapJSON'-object which contains the same products in JSON-format.
Initially: all links (product-detail page, paging, sorting, filtering) are real server-side urls (no hashbang urls)
The end-result is a page which can be navigated 100% with paging, sorting, filtering without the use of JS.
all paging,sorting, filtering urls result in a request to the server, which in turn results in a new set of products being rendered. Nothing special here.
JS-enabled - on domload:
fetch the bootstrapJSON and make product-models from it (use your js-framework features to do this) .
Afterwards rerender the products using the same mustache-template but now doing it client-side. (Again using your js-framework).
Visually nothing should change (after all server-side and client-side rendering was done on same models, with same template), but at least now there's a binding between the client-side model and the view.
transform urls to hashbang-urls. (e.g: /products/#sort-price-asc ) and use your js-framework features to wire the events.
now every (filtering, paging, sorting ) url should result in a client-side state-change, which would probably result in your js-framework doing an ajax-request to the server to return new products (in JSON-format) . Rerendering this again on the client should result in your updated view.
The logic part of the code to handle the ajax-request in 6. on the server-side is 100% identical to the code used in 4. Differentiate between an ajax-call and an ordinary request and spit out the products in JSON or html (using mustache server-side) respectively.
EDIT: UPDTATE JAN 2013
Since this question/answer is getting some reasonable traction I thought I'd share some closely-related aha-moments of the last year:
Spitting out JSON and rendering it client-side with your client-side mvc of choice (steps 6. and 7. above) can be pretty costly cpu-wise. This, of course, is especially apparent on mobile-devices.
I've done some testing to return html-snippets on ajax (using server-side mustache-template rendering) instead of doing the same on the client-side as suggested in my answer above. Depending on your client-device it can be up to 10 times faster (1000ms -> 100ms) , of course your mileage may vary. (practically no code changes needed, since step 7. could already do both)
Of course, when no JSON is returned there's no way for a client-side MVC to build models, manage events, etc. So why keep a clientside MVC at all? To be honest, with even very complex searchpages in hindsight I don't have much use for client-side mvc's at all. The only real benefit to me is that they help to clearly separate out logic on the client, but you should already be doing that on your own imho. Consequently, stripping out client-side MVC is on the todo.
Oh yeah, I traded in Mustache with Hogan (same syntax, a bit more functionality, but most of all extremely performant!) Was able to do so because I switched the backend from java to Node.js (which rocks imho)

Since I'm a visually-impaired user and web developer, I'll chime in here.
These frameworks, in my experience, haven't been a problem provided the appropriate steps are taken with regard to accessibility.
Many screen readers understand JavaScript, and we as developers can improve the experience using things like HTML5's aria-live attribute to alert screen readers that things are changing, and we can use the role attribute to provide additional hints to the screenreaders.
However, the basic principle of web development with JavaScript is that we should develop the underlying site first, without JavaScript, and then use that solid, working, and tested foundation to provide better features. The use of JS should not be required to purchase a product, receive services, or get information. And some users disable JavaScript because it interferes with the way their screenreaders work.
Doing a complete Backbone.js or Knockout site from the ground up without regard for accessibility will result in something akin to "new Twitter" which fails extremely hard with many screenreaders. But Twitter has a solid foundation and so we can use other means to access the platform. Grafting Backbone onto an existing site that has a well-crafted API is quite doable, and an awful lot of fun, too.
So basically, these frameworks themselves are no more of an accessibility issue than jQUery itself - the developer needs to craft a user experience that works for everyone.

Any webpage that requires javascript in order to get the content out of it will likely be met with accessibility-related challenges. The accessibility of JavaScript frameworks is definitely an issue of contention, though really, any web application suffers drawbacks when content is provided dynamically, regardless of the framework used.
There's no silver bullet to ensure your site will be accessible, and I certainly can't account for every JavaScript framework. Here's a few thoughts about how you can prevent your site from being totally inaccessible when using JavaScript:
Follow the guidelines from WCAG 2.0 on client-side scripting, and WCAG 2.0 in general.
Avoid frameworks that require you generate the page's UI, controls and/or content entirely through javascript such as Uki.js, or ones that use their own proprietary markup, like Jo. The closer you can stick with static(-ish), semantic HTML content, the better off you'll be.
Consider using ARIA roles such as role="application" and the aria-live attribute to indicate the areas of your page which are dynamic. More and more aria roles are being supported by assistive devices as time goes by, so using these aria attributes makes sense when you can add them to your app appropriately.
In terms of JS libraries, check their source and see if they output any aria roles. They might not be perfectly accessible, but it would demonstrate they're considering assistive devices.
Wherever possible, treat JavaScript as an enhancement rather than a necessity. Try to provide alternative methods or workflows to accessing the important information that don't require dynamic page updates.
Test and validate your app with your users! Do some user testing sessions with people who use assistive devices or have other difficulties using web software. Nothing will help you prove your site is accessible more than watching real people use it.
The last point is the most important, though many try to escape it. Regardless of the technology, the fact remains that you're developing an application that people will use. No machine or theory will ever be able to perfectly validate your application as being usable, but you're not building it for machines anyway. Right? :)

Chris Blouch (AOL) and Hans Hillen (TPG) had a good presentation on this regarding jQuery, including the work they do in reviewing for accessibility. Making Rich Internet Applications Accessible Through JQuery That and another related presentation on Accessibility of HTML5 and Rich Internet Applications (http://www.paciellogroup.com/training/CSUN2012/) should be of interest to you.
My money is on choosing the most accessible framework: jQuery provides a great deal of graceful degradation or progressive enhancement fallback as well as an overall pretty good focus on accessibility. Also, indirectly I help test and review several systems that leverage jQuery (Drupal public and Intranet websites) such that defects found for accessibility are found and routed back to the project for fixes.

Why do web applications send HTML over the wire?

This question pertains to web applications. I have very little web app development experience, so might be missing some very obvious points/issues. Please point them out.
As I understand, in most web applications, a web server sends HTML over the wire to a client (browser). This happens every time a HTTP request is made. I feel this is very wasteful of bandwidth.
1) Since browsers can run JavaScript, why don't we just send a JavaScript program which can generate the webpage's HTML content (which the browser then renders).
2) Further a browser might cache the JavaScript program and next time the server only need send the data. The protocol might involve the browser sending the "program version" it has.
Consider an example of a relatively simple website Hacker News [http://news.ycombinator.com]. Let us separate the data (30 posts + their metadata) from its presentation. Assuming 1) above, the server can just send the data (say in JSON) + a JavaScript program to generate HTML. This gist shows the idea. The data for the 30 posts is in JSON [http://www.json.org/js.html] format. For this particular example the data transferred is cut in 1/2 (size of data+JavaScript / size of HTML). Further if browsers can do 2) above, it reduces the data transferred on each visit to 1/4 (size of data / size of HTML). [Note: this analysis is without considering compression; gzip,deflate is very successful in reducing the size of HTML. But isn't prevention better than cure?]
I see atleast the following advantages of this :-
* For most web pages, it will reduce the size of data transferred over the wire.
* Forces web apps to separate data from its presentation.
Disadvantages might include - more complex browsers, time to run the JavaScript program to generate HTML (this might get offset by the reduction in data size).
Now my question is - why are web applications not developed this way, or, why do web applications send HTML over the wire? Surely the web server (sending out HTML) doesn't care about HTML at all, so why should it, first, generate it, and then send it over the wire?

There are a few reasons, some of them historical this is by no means a complete list but just some of my experiences:
HTML predates JS, and a lot of scripts and libraries predate JS
Older browsers (think IE<=6) had rubbish, inconsistent JS engines, their rendering engines were much more consistent in how they treat HTML. So many more libraries and scripts predate consistent JS
It is a nightmare to debug applications written as you suggest if they are not constructed right (we have one at my work, it takes 30 minutes to find where a piece of html is actually generated)
It is a lot more work to do it right - why not use templates or static docs or something much simpler
Its not really a problem - HTML compresses really well
What you suggest is done - its called AJAX (OK, so ajax is more general than this but you all know what i mean)
It simply doesn't work for most plain-text user agents including those used by most search engines. If this page is serving most of your content, its generally a good idea to make it easy for Google to parse

Well the obvious reason on why this is the case is that JavaScript wasn't around when we started sending HTML around, and HTML was an improvement to sending around plaintext documents.
The reason we don't do this now: we eschew complex solutions to problems that aren't really problems.
Average internet connections download nearly 1M bytes per second, and web browsers are quite adept at parsing and starting to render this HTML before it's even all ready to be. They're also great at parallelizing the downloading of resources on the page. If we want to save a few bytes at the cost of some compute cycles, we gzip content before sending it. Problem solved.
And for the record, we do this with AJAX in complex webpages (checkout Github's source browsing for a great example of how awesome this can be).

What you suggest can, and is, done. Remember, web pages used to be static documents. Full blown web-based applications are a relatively recent idea.
I might also suggest that it isn't necessarily more efficient, especially when your pages are sent gzipped.

What you suggest is basically what a JavaScript full stack framework like ExtJS does. You can create rich, data intensive applications without writing any HTML -- well, only enough to reference the necessary .js libraries. The complex DOM needed for layouts, grids, forms etc is all created by the framework.

The simple answer is that HTML is older. Why is C99 not fully implemented with a lot of compilers? They figure 1989 is new enough for them. Also, JavaScript exercises a lot more control over people's browsers than they seem to want. Conditional statements and encoded data pose a security concern, and some people want to keep that can of worms closed to begin with. True, HTML is a very inefficient markup, but the size is insignificant compared to the images you download from the internet. That favicon takes up as much data as the page itself, and it's only 16 pixels across.

A good reason that the server-side code of a web application might do lots of HTML template work on the server side is that in many server environments it's not made easy to bundle up server-side data structures (object graphs) for easy delivery to the client. There may be information kept in server-side data structures that really shouldn't be delivered out to the client. Thus in order to send out a "pure" data-only response, the server would have to trim off sensitive data before delivering out the JSON. That's not an unsolvable problem, but I don't know of many server frameworks that facilitate a solution.
The server has direct, unfettered access to the database and to everything else that makes an application work: user preferences, history, account details, system settings, etc. To build an application that's client-centric for rendering purposes would mean concocting ways of keeping all that information intact and up-to-date on the client. For a lot of applications, that might not be terribly easy.
Finally, it's only relatively recently that it would make sense to trust a browser to provide a stable enough platform for building a long-lived "application environment" as a continually-updating web page. By building a web app such that pages are sometimes completely reloaded, there are lots of little "reboots". That's a cheap and dumb way of keeping a lid on at least some kinds of memory leaks.

Most implementations of sites with heavy Javascript use won't start executing until the DOM has fully loaded; then you'll get every page with 'loading screens' when the page wrapper has downloaded, but none of the content has.
Also, do remember that not all users have Javascript enabled, and not all browsers support high-level Javascript (think mobiles).

I would send HTML in a response if I wanted my application to work without Javascript. I would write HTML rendering code in my server-side language (most of the time not Javascript), which could then be used for two purposes: serving whole HTML pages, and serving bits of HTML in response to XHRs.
If the Javascript code is restricted to things like reporting UI events and replacing innerHTML with server-generated code, I don't have to duplicate any of my application logic across languages/frameworks. This duplication problem is one of the reasons why server-side Javascript is getting people excited.

One page only javascript applications

Have you experimented with single page web application, i.e. where the browser only 'GETs' one page form the server, the rest being handled by client side javascript code (one good example of such an 'application page' is Gmail)?
What are some pro's and con's of going with this approach for simpler applications (such as blogs and CMSs)?
How do you go about designing such an application?
Edit: As mentioned in the response a difficuly is to handle the back button, the refresh button, bookmarking/copying url. The latter can be solved using location.hash, any clue about the remaining two issues?

I call these single page apps "long lived" apps.
For "simpler applications" as you put it it's terrible. Things that work OOTB for browsers all of a sudden need special care and attention:
the back button
the refresh button
bookmarking/copying url
Note I'm not saying you can't do these things with single-page apps, I'm saying you need to make the effort to build them into the app code. If you simply had different resources at different urls, these work with no additional developer effort.
Now, for complex apps like gmail, google maps, the benefits there are:
user-perceived responsiveness of the application can increase
the usability of the application may go up (eg scrollbars don't jump to the top on the new page when clicking on what the user thought was a small action)
no white screen flicker during the HTTP request->response
One concern with long-lived apps is memory leaks. Traditional sites that requests a new page for each user action have the added benefit that the browser discards the DOM and any unused objects to the degree that memory can be reclaimed. Newer browsers have different mechanisms for this, but lets take IE as an example. IE will require special care to clean up memory periodically during the lifetime of the long-lived app. This is made somewhat easier by libraries these days, but by no means is a triviality.
As with a lot of things, a hybrid approach is great. It allows you to leverage JavaScript for lazy-loading specific content while separating parts of the app by page/url.

One pro is that you get the full presentation power of JavaScript as opposed to non-JavaScript web sites where the browser may flicker between pages and similar minor nuisances. You may notice lower bandwidth use as well as a result of only handling with the immediately important parts that need to be refreshed instead of getting a full web page back from the server.
The major con behind this is the accessibility concern. Users without JavaScript (or those who choose to disable it) can't use your web site unless you do some serious server-side coding to determine what to respond with depending on whether the request was made using AJAX or not. Depending on what (server-side) web framework you use, this can be either easy or extremely tedious.
It is not considered a good idea in general to have a web site which relies completely on the user having JavaScript.

One major con, and a major complaint of websites that have taken AJAX perhaps a bit too far, is that you lose the ability to bookmark pages that are "deep" into the content of the site. When a user bookmarks the page they will always get the "front" page of the site, regardless of what content they were looking at when they made the bookmark.

Maybe you should check SproutCore (Apple Used it for MobileMe) or Cappuccino, these are Javascript frameworks to make exactly that, designing desktop-like interfaces that only fetch responses from the server via JSON or XML.
Using either for a blog won't be a good idea, but a well designed desktop-like blog admin area may be a joy to use.

The main reason to avoid it is that taken alone it's extremely search-unfriendly. That's fine for webapps like GMail that don't need to be publically searchable, but for your blogs and CMS-driven sites it would be a disaster.
You could of course create the simple HTML version and then progressive-enhance it, but making it work nicely in both versions at once could be a bunch of work.

I was creating exactly these kind of pages as webapps for the iPhone. My method was to really put everything in one huge index.html file and to hide or show certain content. This showing and hiding i.e. the navigation of the page, I control in a special javascript file where the necessary functions for handling the display of the parts in the page are.
Pro: Everything is loaded in the beginning and you don't need to request anything from the server anymore, e.g. "switching" content and performing actions is very fast.
Con: First, everything has to load... that can take its time, if you have a lot of content that has to be shown immediately.
Another issue is that in case when the connection goes down, the user will not really notice until he actually needs the server side. You can notice that in Gmail as well. (It sometimes can be a positive thing though).
Hope it helps! greets

Usually, you will take a framework like GWT, Echo2 or similar.
The advantage of this approach is that the application feels much more like a desktop app. When the server is fast enough, users won't notice the many little data packets that go back and forth. Also, loading a page from scratch is an expensive operation. If you just modify parts of it, the browser can keep a lot of the existing model in memory and just change the parts that changed.
Another advantage of these frameworks is that you can develop your application in pure Java. This means you can debug it in your IDE just like any other Java app, you can write unit tests and run them automatically, etc.

I'll add that on slower machines, a con is that a large amount of JavaScript will bring the browser to a screeching halt. Since all the rendering is done client-side, if the user doesn't have a higher-end computer, it will ruin the experience. My work computer is a P4 3.0GHZ with 2 GB of ram and JavaScript heavy sites cause it to chug along slower than molasses, which really kills the user experience for me.