Develop Reference

How can I load external html files?

I need to make changes to an existing project that uses iFrames to dynamically load external html files. However, the html files are part of the same project, not external sites. If I'm not mistaken, iFrames are considered a terrible way of loading html content unless they are used to actually display external sites.
I have looked into web components but apparently, browser support is still spotty and unfortunately, I need to support IE9.
I know that the JQuery load() method can accomplish this but in my online research, that doesn't often come up as a proper way of loading external html in general and a proper replacement for iFrames in particular.
Is there a reason why JQuery shouldn't be used here and are there better and established ways of doing this? For example, I once saw a framework that dynamically built the interface out of separate "partials" but I don't remember which framework that was.

It depends on the HTML -
If it's built like a full page - then iFrames are actually a decent solution - Also, iframes with the same origin let you have full control over the content from the parent, while still protecting CSS and JS variables which is pretty convenient.
If not - jQuery.load() will do the trick, you can also do it manually ofc, but if you already have jQuery in your project, just use it.

The load() function is almost always the best way to go, if you are encountering a specific issue using that function maybe you can share it?

Converting a web app into an embeddable <script> tag

I just did a proof of concept/demo for a web app idea I had but that idea needs to be embedded on pages to work properly.
I'm now done with the development of the demo but now I have to tweak it so it works within a tag on any websites.
The question here is:
How do I achieve this without breaking up the main website's stylesheets and javascript?
It's a node.js/socket.io/angularjs/bootstrap based app for your information.
I basically have a small HTML file, a few css and js files and that's all. Any idea or suggestions?

If all you have is a script tag, and you want to inject UI/HTML/etc. into the host page, that means that an iframe approach may not be what you want (although you could possibly do a hybrid approach). So, there are a number of things that you'd need to do.
For one, I'd suggest you look into the general concept of a bookmarklet. While it's not exactly what you want, it's very similar. The problems of creating a bookmarklet will be very similar:
You'll need to isolate your JavaScript dependencies. For example, you can't load a version of a library that breaks the host page. jQuery for example, can be loaded without it taking over the $ symbol globally. But, not all libraries support that.
Any styles you use would also need to be carefully managed so as to not cause issues on the host page. You can load styles dynamically, but loading something like Bootstrap is likely going to cause problems on most pages that aren't using the exact same version you need.
You'll want your core Javascript file to load quickly and do as much async work as possible as to not affect the overall page load time (unless your functionality is necessary). You'll want to review content like this from Steve Souders.
You could load your UI via a web service or you could construct it locally.
If you don't want to use JSONP style requests, you'll need to investigate enabling CORS.
You could use an iframe and PostMessage to show some UI without needing to do complex wrapping/remapping of the various application dependencies that you have. PostMessage would allow you to send messages to tell the listening iFrame "what to do" at any given point, while the code that is running in the host page could move/manipulate the iframe into position. A number of popular embedded APIs have used this technique over the years. I think DropBox was using it for example.

Very large single page application design problems

I am currently writing whats going to be a very, very large single page web/javascript application.
Technologies I am using are ASP.NET MVC4, jquery, knockout.js and amplify.js.
The problem I am having is that most if not all of the single page application examples are for smaller applications where having all of the script templates (be it jquery, handlbars, etc...) are all in the same file along with the rest of the html code. This is fine for smaller apps but the application I am building is an entire maintenance logistics application with many, many, many screens.
The approach I took so far is I have an outer shell (my main index.cshtml file) and I am using jquery's load() method to load or rather inject the particular file I need when a user makes a certain selection.
example:
function handleLoginClick(){
App.mainContentContainer.delegate('#btnLogin', 'click', function() {
App.loadModule('Home/ProductionControlMenu', App.MainMenuView.render());
});
}
here's the code for the App.loadModule function:
App.loadModule = function(path, callback){
App.mainContentContainer.load(App.siteRoot + path, callback);
};
All was going well until I needed to actually start interacting with the various form elements on the newly loaded screen. jquery can't seem to get a handle on them directly. I can't use .live() or .delegate() because they aren't events, they are textboxes and buttons and sometimes I need to change their css classes.
They only way I found is to get a handle on an element from the outer shell (something that wasn't loaded via .load() ) and use jquery's .find() method like so:
App.mainContentContainer.find('#btnLogin').addClass('disabled');
clearly I don't want to have to do something like this everytime I need to interact with or even retrieve values from a form element.
Does anybody have any ideas as to how I can create a maintainable very large single page application with potentially hundreds of .html files without having to have all that html code located in a single file and still get around this .load() issue I am having?
Any thoughts would be greatly appreciated. :-)
V/R
Chris
UPDATE
I thought I'd post an update and as to how things went and what worked. After much research I decided to go with Google's AngularJS Javascript framework. It simplified the ordeal exponentially and I would definitely, definitely advise all who are looking into making a large SPA to give it a look.
Links:
Main Site
http://angularjs.org/
Awesome free short videos on Angular:
http://www.egghead.io/

This is actually a very complicated question as it really gets down to the design of your architecture.
For large-scale single-page applications, it's best to use some sort of front-end MV* style framework such as backbone.js, which ties in to jQuery quite usefully. You should also think about using some sort of dependency management framework such as require.js in order to load your scripts and dependencies asynchronously, and even better -- use the AMD pattern in your application design to make your architecture modular and easier to manage.
As far as how this relates to your MVC4 project, you have some options:
Do you want to use MVC as a "service layer" of sorts, that simply returns JSON objects, allowing your front-end to do the markup/template creation (think handlebars.js), or
Do you want your MVC project to return partial views (HTML) as a response, where you leverage the Razor templating system and simply use the front end to display what comes back from the server?
Either way, you will have to devise a way to handle front-end events and navigation (backbone provides both of these things when coupled with jQuery). Even more complicated is the pattern you choose to notify one view of another view's activities (there are many ways to do this) -- a publish/subscribe pattern for example.
I hope I have helped a bit, I know I'm not fully answering the question, but the answer could get really long!

Lots of things are wrong with your approach. What I'd recommend is to watch some presentations on how people build Single Page Applications and what tooling is mostly used.
This seems like something reasonable: http://singlepageappbook.com/
You will at least want
some kind of modules system (I recommend AMD – http://requirejs.org)
an MV* framework (Backbone, Ember.js etc.)
DOM/AJAX Framework (jQuery, Mootools etc.). Some frameworks offer this and all of the above (Dojo, YUI, Sencha)
build solution (to have different environment in development / production)
Couple of good links:
http://nerds.airbnb.com/slides-and-video-from-spike-brehms-tech-talk
http://video.copenhagenjs.dk/video/3413395/simon-hjberg-swipely-building
http://backstage.soundcloud.com/2012/06/building-the-next-soundcloud/
http://www.youtube.com/watch?v=vXjVFPosQHw

If you don't need a complicated truly SOFEA Single Page App then I recommend you go the PJAX route.
Then you just write your app as a normal web 1.0 app with the performance benefits of a single page load. I urge you to consider this an option as it allows you to do most of your validation work server side.
The idea is very simple on every response your sending the whole page back minus the header and footer (which contains the javascript and css includes). DOM rendering time is incredible fast these days... whats not is a full page reload, So don't worry about the size of the HTML your returning back.
Also the "PJAX way" is much easier to cache, Google SEO friendly and is in fact what the new Basecamp does.

Note: Wanted this to be a comment not an answer but don't have enough exp to post comments ;(
[any corrections by community members welcome!]
Important points to consider for single page apps:
Lazy loading is vital as you don't want hundreds of js files loaded straight away as user first loads the page (very slow load time).
Good File organisation,- helps making changes easier, reduces complexity a bit and promotes reusable components. Makes testing of components easier.
Testing,- Since single page apps have a lot of javascript going on under the hood you would need a test framework for automatically testing components. This testing is on top of the tests you would use to confirm if certain user controls are rendered etc as you wouldn't want a viewless component making an ajax call to server when it shouldn't etc.
+1 for gryzzly's point about using a framework.
Sencha have a nice MVC like framework for their ExtJs product. They have data stores, ajax, lazy loading, class hierarchies and a lot more all bundled into the package. They have a good api page also to lookup object properties and methods (handy since there doesn't seem to be any intellisense for javascript :/ ). Their api page is; as far as I know, an example of a single page app. I know much of the stuff ExtJs does you can find an open source alternative but I like that it's just the one library and I don't have to download a couple of different frameworks to do various operations. [note: I have no affiliation to Sencha except for being a customer of theirs and like their stuff.]
Conclusion:
I'd say it would be quite difficult to manage a large single page app without using some client side framework; whether open source or not, and without using some architectural pattern like a client side MVC.
Single page apps I think are more complicated so your team would need to be quite handy at understanding the concept of a single page app and how to implement it. If you pull it off the site will be amazing in terms of user experience.

I would recommend to use Sammy.js and split various viewmodels in knockout to a separate url. And if you're using asp.net mvc 4, use partial views (user controls) so that you put all code in one file. And name all and split all js-code in a meaningful way, filename and namespace in js. That will help a lot in maintainability and your own sanity in the long run.
And use common sense!

The way I did it was to include javascript code related to the template along with the template itself. And then load the whole template+script thing using ajax. If you want to try this be warned that most browsers don't execute <script> tags injected into the page. Especially if done using innerHTML. As such you should eval the script tags yourself (alternatively you can use document.createElement to inject the script but this does not offer any additional advantages compared to eval since the browser will blindly execute the script anyway).
In my case, to make it easier to grab the html and script portions of the template I store my templates in XML files instead of plain old HTML. That way I can simply use .responseXML of the ajax request to parse the template. My template has the following basic structure:
<template options...>
<title>Optional</title>
<html><![CDATA[
Template body
]]></html>
<init><![CDATA[
// code that only needs to execute once
]]></init>
<script><![CDATA[
// code that needs to run every time
// the template loads
]]></script>
</template>
You also need to remember to configure your server to reply with the correct content type for you templates. Otherwise resultXML won't work. This is not the only way to implement this system of course. You could simply save your templates as HTML and then parse that HTML to extract scripts to execute.
The main bulk of your code, the functions, constructors, objects etc. can be included in a js file. The template script only need to call those functions to tell the rest of the page how to work with the template.
If you further separate your data from your template and only populate the template with data on the page or make a separate ajax request for JSON data then you can configure your server to make the browser cache your templates. This is especially useful for often used templates (such as templates for dialog boxes). This allows the browser to download the template only once and use the cached version the next time you call the template.
Anyway, that's how I did it last time. It scaled well enough to serve facebook users (the web app was a facebook app). Just sharing my experience. Hope it helps.

I've written a few huge single page application using Dojo Toolkit. I'm pretty sure whatever JavaScript framework you choose will probably work for you. I use Dojo because it provides me with features that makes huge single page application development easier to manage.
You can use Dojo's widget system to help you define all your screens and forms as widgets and then when you need them you can just instantiate and insert them wherever you need it. When ever you want to get rid of it, you can simply call destroy or destroyRecursive on that particular widget and its gone. Dojo's widget system also help you separate your HTML from your JavaScript, but still keep it together so that they are not located all over the place.
I've include a simple widget definition for a Login form.
This is the HTML template.
/* mine/forms/Login.html */
<div>
${form_name}
<label>Username</label>
<input data-dojo-type="dijit.form.TextBox"
data-dojo-attach-point="_usrfld" />
<br />
<label>Password</label>
<input data-dojo-type="dijit.form.TextBox"
data-dojo-props="type: 'password'"
data-dojo-attach-point="_pwdfld" />
<br />
<input data-dojo-type="dijit.form.Button"
data-dojo-props="label: 'Login'"
data-dojo-attach-event="onClick:_handleLogin" />
</div>
This is the JavaScript portion for the widget.
/* mine/forms/Login.js */
define([
"dojo/_base/declare",
"dijit/_Widget",
"dijit/_Templated",
"dijit/_WidgetsInTemplateMixin",
"dijit/form/Button",
"dijit/form/TextBox",
"dojo/text!./Login.html"
], function(
declare,
_Widget,
_Templated,
_WidgetsInTemplateMixin,
Button,
TextBox,
template
) {
return declare("mine.forms.Login", [_Widget, _Templated, _WidgetsInTemplateMixin], {
// assign the template
templateString: template,
// signal that we will have widgets within our template and the parser should
// locate them and instantiate them
widgetsInTemplate: true,
form_name: "My Login Form",
// place holders that will be referencing the corresponding widgets
// that I have placed a data-dojo-attach-point on
_usrfld: null,
_pwdfld: null,
_lgbtn: null,
// a call back function that will be trigger when the Login button is clicked
_handleLogin: function() {
var usr = this._usrfld.get('value');
var pwd = this._pwdfld.get('value');
// now you have the username & password
// you can use it to login
}
});
});
There are a few benefits that the widget system provides:
The HTML template will be loaded for you when needed
You can have widgets within widgets. Dojo takes care of instantiating and even destroying those widgets for you
You can use dojo's simple template language to help with inserting strings. The example above uses ${form_name}. If you want to use a more sophisticate facility dojo also supports a Django similar syntax templating language. This will allow you to use most of the tags available with Django like for, if-then-else, etc.
data-dojo-attach-point is very helpful. If you use this, you will never need to assign id to a DOM element anymore. You will not need to locate the element in your HTML DOM tree. The variable you name in data-dojo-attach-point will be auto assign to reference the widget or DOM element you have defined in your template. The example above uses the _usrfld, _pwdfld, and _lgbtn.
data-dojo-attach-event is also very helpful. If you use this, you will not have to manully add an event hook to the button, Dojo's widget system will hook it for you and will also clear the hook when your widget is destroyed.
If you use Dojo's build system, which is a system that takes all your JavaScript codes and compresses them, Dojo will replace your template with the actual HTML so that when you load your widget in production mode, Dojo doesn't have to make another AJAX request to grab your template.
These are just some of the features I use daily when I develop my projects. Hopefully it will provide you with some insight that you can use to make your decision when choosing the proper JavaScript framework for your project. As a side note, I'm not advocating Dojo or anything, just wanted to share what works for me.

To handle large size single page application it's recommended to break the individual module into different apps.
example : apps like search page app and result page app.

Javascript and website loading time optimization

I know that best practice for including javascript is having all code in a separate .js file and allowing browsers to cache that file.
But when we begin to use many jquery plugins which have their own .js, and our functions depend on them, wouldn't it be better to load dynamically only the js function and the required .js for the current page?
Wouldn't that be faster, in a page, if I only need one function to load dynamically embedding it in html with the script tag instead of loading the whole js with the js plugins?
In other words, aren't there any cases in which there are better practices than keeping our whole javascript code in a separate .js?

It would seem at first glance that this would be a good idea, but in fact it would actually make matters worse. For example, if one page needs plugins 1, 2 and 3, then a file would be build server side with those plugins in it. Now, the browser goes to another page that needs plugins 2 and 4. This would cause another file to be built, this new file would be different from the first one, but it would also contain the code for plugin 2 so the same code ends up getting downloaded twice, bypassing the version that the browser already has.
You are best off leaving the caching to the browser, rather than trying to second-guess it. However, there are options to improve things.
Top of the list is using a CDN. If the plugins you are using are fairly popular ones, then the chances are that they are being hosted with a CDN. If you link to the CDN-hosted plugins, then any visitors who are hitting your site for the first time and who have also happened to have hit another site that's also using the same plugins from the same CDN, the plugins will already be cached.
There are, of course, other things you can to to speed your javascript up. Best practice includes placing all your script include tags as close to the bottom of the document as possible, so as to not hold up page rendering. You should also look into lazy initialization. This involves, for any stuff that needs significant setup to work, attaching a minimalist event handler that when triggered removes itself and sets up the real event handler.

One problem with having separate js files is that will cause more HTTP requests.
Yahoo have a good best practices guide on speeding up your site: http://developer.yahoo.com/performance/rules.html
I believe Google's closure library has something for combining javascript files and dependencies, but I havn't looked to much into it yet. So don't quote me on it: http://code.google.com/closure/library/docs/calcdeps.html
Also there is a tool called jingo http://code.google.com/p/jingo/ but again, I havn't used it yet.

I keep separate files for each plug-in and page during development, but during production I merge-and-minify all my JavaScript files into a single JS file loaded uniformly throughout the site. My main layout file in my web framework (Sinatra) uses the deployment mode to automatically either generate script tags for all JS files (in order, based on a manifest file) or perform the minification and include a single querystring-timestamped script inclusion.
Every page is given a body tag with a unique id, e.g. <body id="contact">.
For those scripts that need to be specific to a particular page, I either modify the selectors to be prefixed by the body:
$('body#contact form#contact').submit(...);
or (more typically) I have the onload handlers for that page bail early:
jQuery(function($){
if (!$('body#contact').length) return;
// Do things specific to the contact page here.
});
Yes, including code (or even a plug-in) that may only be needed by one page of the site is inefficient if the user never visits that page. On the other hand, after the initial load the entire site's JS is ready to roll from the cache.

The network latency is the main problem.You can get a very responsive page if you reduce the http calls to one.
It means all the JS, CSS are bundled into the HTML page.And if your can forget IE6/7 you can put the images as data:image/png;base64
When we release a new version of our web app, a shell script minify and bundle everything into a single html page.
Then there is a second call for the data, and we render all the HTML client-side using a JS template library: PURE
Ensure the page is cached and gzipped. There is probably a limit in size to consider.We try to stay under 400kb unzipped, and load secondary resources later when needed.

You can also try a service like http://www.blaze.io. It automatically peforms most front end optimization tactics and also couples in a CDN.
There currently in private beta but its worth submitting your website to.

I would recommend you join common bits of functionality into individual javascript module files and load them only in the pages they are being used using RequireJS / head.js or a similar dependency management tool.
An example where you are using lighbox popups, contact forms, tracking, and image sliders in different parts of the website would be to separate these into 4 modules and load them only where needed. That way you optimize caching and make sure your site has no unnecessary flab.
As a general rule its always best to have less files than more, its also important to work on the timing of each JS file, as some are needed BEFORE the page completes loading and some AFTER (ie, when user clicks something)
See a lot more tips in the article: 25 Techniques for Javascript Performance Optimization.
Including a section on managing Javascript file dependencies.
Cheers, hope this is useful.

Lazy loading and dependency resolution

some time ago, I was reading an article(a library built by some guy) about how his library can do
lazy loading of JS
resolve dependencies between JS
(typically encountered when trying
to "include" one js from another)
include files only once. thought
specified multiple times regardless
of how they are called (either
directly specifying it as file or
specifying it as one of the
dependencies)
I forgot to bookmark it, what a mistake. Can some one point me to something which can do the above. I know DOJO and YUI library have something like this, but I am looking for something which I can use with jQuery
I am probably looking for one more feature as well.
My site has asp.net user controls
(reusable server side code snippets)
which have some JS. Some of them get
fired right away, when the page is
loading which gives a bad user
experience. Yahoo performance
guidelines specify that JS should
be at the bottom of the page, but
this is not possible in my case as
this would require me to separate the
JS and the corresponding server side
control into different files and
maintenance would be difficult. I
definitely can put a jQuery
document.ready() in my user control
JS to make sure that it fires only
after the DOM has loaded, but I am
looking for a simpler solution.
Is there anyway that I could say "begin executing any JS only after DOM has loaded" in a global way than just writing "document.ready" within every user control ?

Microsoft Research proposed a new tool called DOLOTO. It can take care of rewriting & function splitting and enable the on-demand js loading possible.
From the site..
Doloto is a system that analyzes
application workloads and
automatically performs code splitting
of existing large Web 2.0
applications. After being processed by
Doloto, an application will initially
transfer only the portion of code
necessary for application
initialization. The rest of the
application's code is replaced by
short stubs -- their actual function
code is transferred lazily in the
background or, at the latest,
on-demand on first execution.

OK I guess I found the link
[>10 years ago; now they are all broken]
http://ajaxian.com/archives/usingjs-manage-javascript-dependencies
http://www.jondavis.net/techblog/post/2008/04/Javascript-Introducing-Using-%28js%29.aspx
I also found one more, for folks who are interested in lazy loading/dynamic js dependency resolution
http://jsload.net/

About the lazy-loading scripts thingy, most libraries just adds a <script> element inside the HTML pointing to the JS file to be "included" (assynchronously), while others like DOJO, fetches it's dependencies using a XMLHttpRequest and then eval's its contents, making it work synchronously.
I've used the YUI-Loader that is pretty much simple to use and you don't need the whole library to get it working. There are other libraries that gives you just this specific funcionality, but I think YUI's is the safe choice.
About your last question, I don't think there's something like that. You would have to do it yourself, but it would be similar to using document.ready.

i did in my framework a similar thing:
i created a include_js(file); that include the js file only if it isn't included reading and executing it with a synchronous ajax call.
Simply put that code in top of the page that needs dependencies and you're done!