I want to render some static files (*.js in particularly) using Django template variables. I believe this is a common use-case, especially when doing anything AJAX-y; I don't want to hardcode AJAX urls in my .js files any more than I want to hardcode them in my .html files. Buuuut of course I don't want those static files to have to run through the template engine at every client request because this will be slow. I am referring to things like URLs (which do not change after compile/deploy) or static (non-db) model attributes. (I suppose there are use cases where these things might be changed at run-time - this is Python, after all- but I think they are uncommon). For some possible template variables (e.g. model fields), of course the file must be rendered at the time of the client request, but this is not what I'm talking about.
So wouldn't it make sense to render some of my static files through the template engine, for a subset of possible template variables, perhaps at the same time as collectstatic?
As far as I can tell this is not currently the case. To be clear, what I am looking for is a solution to render static files through the template engine at compile/deploy-time so that at "client-request-time" they are in fact plain old static files.
Such an approach would avoid these hacks:
DRY URLs in Django Javascript
Using the Django URL Tag in an AJAX Call
Disclaimers:
Yes I know there are template engines out there for javascript (mustache, handlebars, prototype, etc). Why should I add another template engine to the stack when Django already has one? Plus the syntax collides! That seems silly.
This looks like it takes a crack at it, but it's complicated and not fully implemented.
So:
Is there a solution out there that I am missing?
If not, is there a way to hook into collectstatic (like a pre-collectstatic hook) that would allow one to render certain static files through the template engine before "collecting" them?
EDIT:
No responses yet...is this a really dumb question, and I'm missing something obvious? If so...go ahead and let me know...
There are several frameworks for Django for same purpose: django-pipeline, django-assets, and etc. which integrates different static files processing strategies, with varying degrees of difficulty configuring.
I use an external tool - Grunt (it requires node.js) - for asset post-processing after collectstatic. It is easier and has a lots of plugins for any purpose (source validation, css/js/images minification, merging, testing and etc.).
It is possible to hook in collectstatic by a custom static files storage with overrided post_process method.
example/settings.py
STATIC_ROOT = 'assets'
STATICFILES_STORAGE = 'example.storage.MyStaticFilesStorage'
example/storage.py
import os
from django.contrib.staticfiles.storage import StaticFilesStorage
from django.core.files.base import ContentFile
from django.template import Template, Context
class MyStaticFilesStorage(StaticFilesStorage):
def post_process(self, paths, dry_run=False, **options):
# don't even dare to process the files if we're in dry run mode
if dry_run:
return
js_template_data = {'foo': 'bar'} # template variables
js_template_extension = '.jst'
js_extension = '.js'
for original_name, (storage, path) in paths.items():
processed = False
saved_name = original_name
original_path, original_extension = os.path.splitext(original_name)
if original_extension == js_template_extension:
with storage.open(path) as original_file:
saved_name = original_path + js_extension
if hasattr(original_file, 'seek'):
original_file.seek(0)
original_source = original_file.read()
c = Context(js_template_data)
saved_source = Template(original_source).render(c)
self.delete(saved_name)
self.delete(original_name)
self._save(saved_name, ContentFile(saved_source))
processed = True
yield original_name, saved_name, processed
A completely different way to approach the problem would be to ask if you really need to get those URLs in javascript--instead, can the Javascript get the URLs from things like data attributes in your HTML?
In other words, you might have wanted:
homepage.html:
<div id="pop-up-modal">pop me up</div>
homepage.js:
$("#pop-up-modal").click(function {
$.ajax("{% url 'some-class-name %}")
...
});
When it can often be more straightforward to do something like:
homagepage.html:
<div id="pop-up-modal" data-popurl="{% url 'some-class-name' %}">pop me up</div>
homepage.js:
$("#pop-up-modal").click(function {
$.ajax($(this).data('popurl'))
...
});
I think that django-medusa would suit your needs.
By setting up a renderer and using the disk based backend, generating the static files would be as easy as:
django-admin.py staticsitegen
You aren't crazy. I was frustrated by this as well and found myself hacking something together for each new Django project I tackled. I think the reason for the lack of direct solutions is that this is mega-drought bone DRY. Its super easy to just hard code these things and call it day. This and the two most common use cases for this involve generating code in one language from code in another which tends to be viewed as suspect.
I've recently published a new Django package django-render-static that solves this problem generically. It piggy-backs on Django's existing template engine infrastructure. A management command called render_static should be run before collectstatic. This command will look for templates registered in settings (or passed as arguments) and render them to your static file location on disk. They're then available for whatever normal static file packaging pipeline you have configured.
I'm sure there are more use cases, but the two most common I've found are providing a reverse utility in static JavaScript thats equivalent to Django's and auto-translating define-like python structures (i.e. choice fields) into JavaScript. The app provides template tags that do both.
The JavaScript url reversal code is guaranteed to be functionally equivalent to Django's reverse function. I won't bother plopping example code down here because it's been well documented.
Related
I'm working on a JavaScript app and have so far entered all my strings as plain text.
This is starting to feel really hacky (I'm used to gettext) so I'd prefer to wrap them all in something like {{translatable_string}} and have a gulp task just search/replace them all during the build step.
So, my question is; is there a generic (no framework-specific like angular-gettext or something like that) gettext replacer out there?
Obviously it doesn't even have to be connected to JavaScript in any way, you should be able to run it on any file type and have {{translatable_string}}:s be translated.
You may want to look into using gulp-replace. As they explained in this answer, you should be able to use it to find and replace any string that you want in the stream.
I suggest a database of strings for your translations if dynamic generation of page content is possible for your app. Starting with English or whichever is normal but the need to localize content is a tough issue without a robust system. A simple MongoDB table can be used to store the content, and when the app needs an interface it can be loaded with the right localized strings. As a for instance:
if(err) alert("Please turn off caps lock");
could become:
if(err) alert(Please_turn_off_caps_lock.English);
If you are needing to build static pages with gulp, a database in conjunction with gulp-replace sounds interesting. Using gulp-data to call up and package the strings, you can then feed it to gulp-replace and alter the files. The extensible nature of databases or document stores enable you to expand your localization without hacking on individual files or trees all the time.
Try gulp-gettext-parser.
var gettext = require("gulp-gettext-parser");
var rename = require("gulp-rename");
gulp.task("gettext", function() {
return gulp.src("src/**/*.js")
.pipe(gettext())
.pipe(rename("bundle.po"))
.pipe(gulp.dest("dist/"));
});
Perhaps what you need is mustache.js, take a look: https://github.com/janl/mustache.js/
I'm not used to work with mustache, but I had to do some updates in a project done with it, and I was surprised the capabilities it have.
If you're familiar with jade (now renamed to pug), you'll find is something similar but at the end, you're not forced to generate only html files, you cand generate any kind of text file.
This blog could be helpful to understand the differences between some other templating languages over Nodejs: https://strongloop.com/strongblog/compare-javascript-templates-jade-mustache-dust/
I am using a ASP.NET route (to intercept the call to the .js) and controller to generate some JS I want to use on my client. The reason I'm doing this is so as to not have to duplicate id's or constants on the client. Here's the output of my JS:
app.serviceRootURL = 'http://localhost:65211/'; // set in my web.config
app.ajaxResponseStatuses = [
{ "status":"Success", "id":0 }, // set in my C# DTO
{ "status":"Failure", "id":1 },
];
First of all, I am not sure if this is the best approach, so other suggestions to doing this would be beneficial.
More importantly though, I'm wondering how I can bundle and minify this. As I understand it, even if I could minify the JS at compile or run-time, minification will change the names of my variables. So in the above JS, app.ajaxResponseStatuses could get changed to a.bc, and then in the actual JS files where I'm trying to access that variable, they could be looking for x.yz.
Can I minify this code and get it to the server?
Will I still be able to use the above properties in other minified files?
(bonus points) Is this a good aproach to pass server-side-only values to be used on the client?
Part 1
If you are generating the js at runtime, bundling isn't possible (at least not efficiently). You would have to create a new bundle for every request which isn't terribly quick. Plus, you wouldn't be able to cache the regular, constant script bundle.
EDIT: While bundling server-generated js isn't practical, rendering the values into a script tag in the page can achieve the same benefit of bundling, fewer HTTP calls. See the edit in Part 3 for more.
Minifying the server generated js however, is totally possible. This question should have the answer you're looking for. However, I'd recommend you cache this on the server if possible, as the minification process itself could take longer than simply sending down the extra bits.
Part 2
In most minifiers, global variables (those accessible on the window object) are skipped during the name mangling. With the same respect, variables that are accessed in other files that are not defined within that file are not renamed.
For example, if you have the following file...
// outside of a closure, so globally accessible
var foo = 1;
function bar() {
// within a closure, and defined with `var`, not globally accessible
var bar;
// reference to variable declared in another file
baz = null;
}
it would be minified as follows (with whitespace included for readability
var foo = 1;
function bar() {
var b;
baz = null;
}
This is one reason it is important to always declare your variables using the var keyword, otherwise they are assumed to be references to global variables and will not be minified.
Also, JSON (not Javascript object literals!!!) will never be distorted by minifiers, because it consists of string literals for all keys, and all values that aren't of another literal type.
Part 3
Not a bad way, and at my job we do use this approach. For small files though, or simple config values, we have transitioned to rendering server values in a script tag using ASP.NET in the actual view. i.e.
Default.aspx
<script> window.globals = <%= JsonConvert.SerializeObject(new AppGlobals(currentUser)) %>; </script>
We rip this out into a code behind, but the premise is the same.
EDIT:
Server-Generated JS (at it's own uri)
Pros
Cacheable by browser (if fresh values aren't needed on every request)
Cons
Extra round trip
Use when:
Your generated files are large, but rarely change or are the same for multiple users. These scripts can be treated the same as other static assets. To give an example, we serve a js file containing all the text in our app for localization purposes. We can serve a different js file based on the language set in the user's settings, but these values only change once at most with every release, so we can set aggressive cache headers and use a hash in the uri, along with a query string for the locale, to leverage browser caching and download each language file only once per client. Plus, if this file is going to be the same for every user accessing the same uri, you can cache it at the web server (IIS, Apache, etc.).
Ex: /api/language.v1-0-0.js?locale=en
Your js is independent from the rest of your app and not having it won't delay rendering. In this case, you can add the async attribute to your script tag, and this file will be downloaded asynchronously and executed when it is received without preventing the execution of other javascript.
Server-Rendered JS (within the page in a script tag)
Pros
No extra HTTP calls
Cons
Can add extra weight to your HTML, which may not be cacheable or minified depending on your circumstances
Use when:
Your values change often. The weight added to the page should be negligible unless you have a huge number of values (in that case, you might consider splitting them up and adding API endpoints for these values, and only getting them when you need them). With this, you can cut out the extra HTTP call as the js is injected into a script tag on a page the user would already have to retrieve.
But...
Don't waste too much time worrying about it. The differences in these two approaches is almost always negligible. If it becomes a problem, try both and use the better option for your case.
I am working on a mobile single page site that uses breeze js, angular js, web API, entity framework, etc.
To optimize the site, I included the breeze metadata in a bundled JavaScript that contains all the other JavaScript the site needs. Ideally, all I would like the browser to request is index.html, which should contain everything the app needs to run including bundled and minified inline styles and JavaScript.
However, just as the breeze metadata is very important for the site to function and is therefore embedded in the bundled JavaScript, there is also a required complex entity (with some deep navigation properties to some other entities) that must also be present for the site to be fully functional. I would like to embed this entity and all the entities it references in the bundled JavaScript as well. How can I do this?
I can always create a JSON string that represents this entity and all the entities it references. Then embed that JSON string in the bundled JavaScript along with the rest. However, how can I easily import this complex entity into the breeze entity system using the entity JSON string I have embedded in JavaScript? Or is there a better solution to preload the breeze entity system with a complex entity without having to make a request for that entity from the server?
I would also like to avoid writing server code to spit out JavaScript that creates the entity on the client.
Simplest approach would be to use the "initializer" argument to the EntityManager.createEntity call.
See
http://www.breezejs.com/documentation/creating-entities and http://www.breezejs.com/sites/all/apidocs/classes/EntityManager.html#method_createEntity
This call looks this.
myEntityManager.createEntity("Employee", { lastName: Smith", firstName: "John" });
So in your case you could try:
var initialValues = JSON.parse(json);
myEntityManager.createEntity("Employee", initialValues);
Depending on your use case you may want to also set the 'entityState' of this newly created entity as well.
Here's a technique I often use to create entity data for automated tests:
Preparation
prime an EntityManager with the entities (and entity graphs) that you want available at launch.
export as a string with var exported = manager.exportEntities();. The exported string has the metadata embedded in it so you won't have to bring that down separately. Two-for-one!
capture the contents of exported to a JavaScript file that you load as script in index.html. My "capture" process is usually just to display in the console and scrape it.
Usage
Now when you need it:
load that JavaScript metadata+data file.
create a new EntityManager (remember to target the same dataservice endpoint).
import the entities you captured in your script: manager.importEntities(launchData);.
And you are good to go.
Read up on the EntityManager exportEntities and importEntities methods.
Example
One place you can see a variation on this technique is in the test directory of the "Zza-Node-Mongo".
I personally do not combine the data with the metadata so I export using the "no metadata" option. I put the metadata in one script and the data-to-load-on-launch (lookups usually) in a separate script and load both in index.html.
Caution
You say
to optimize the site, I included the breeze metadata in a bundled JavaScript that contains all the other JavaScript the site needs. Ideally, all I would like the browser to request is index.html, which should contain everything the app needs to run including bundled and minified inline styles and JavaScript.
Beware of premature optimization
I rather doubt that you will measurably improve the launch time of the app by embedding metadata and launch date in script files. Maybe some of the time if the browser caches these scripts. But that comes with its own risks and isn't a reliable strategy.
The data you want has to come over the wire to the client one way or another. It isn't self-evident that loading a script file - even a minimized script file - is any faster than pulling the metadata and launch data (both gzipped) down from the server via a web api AJAX call.
The techniques I described do speed up testing because I have to recreate metadata and the launch data before each test. I can measure the performance gain from avoiding repeated trips to the server. I gain nothing for the first trip ... which is the equivalent of your application launch.
Be mentally prepared to discover that your hard earned optimization efforts did not improve launch times ... and might even make them worse for some users.
I want to push my Django project with some JavaScript/jQuery. To make it right from the beginning on I'd like to know, which way of organizing the .js-files ist the optimal one.
For loading one big file includes less overhead than loading many small ones and also because it looks cleaner in the code I considered to make one global .js-file and include that with the base.html (from which every template inherites). However, the result would be, that JavaScript would try to assign all the event-binings, even if the elements which the events should be bind to aren't in the current document. With all the jQuery-selectors which then would have to do their work that can't be too efficient. From earlier web-development experience I know that one can do something like if(location.href == '/some/url/') { (JavaScript code) ... }. That seems not practicable for me in this case, for with changing URLs, I'd have to change the URLconf and the .js-file (while using reverse() and {% url %} to prevent that elsewhere). I guess there is no possibility to make use of the named URLs here?
Has anyone an idea how to organize the JavaScript without having a file for every single template on the one hand and without killing performance unnecessarily?
I don't know that this question is specific to Django - similar issues come up managing Javascript in all sorts of systems.
That said, I usually try to tier my Javascript files, so that truly global scripts and libraries are included in one file, scripts specific to a section of the site are included in a set of section-specific files, and scripts specific to a single page are included in yet another site of page-specific files (or in inline code, depending on the context).
Django has good support for this approach, because you can tier your templates as well. Include the global script in your base.html template, then create a mysection-base.html template that inherits from base.html and just adds the Javascript (and CSS) files specific to that section. Then subpages within that section can inherit from mysection-base.html instead of base.html, and they'll all have access to the section-specific scripts.
I find django-compressor invaluable as it automatically compresses and minifies your JavaScript and CSS pre-deployment. It even automatically handles SASS, LESS and CoffeeScript if they float your boat.
Apps from http://djangopackages.com/grids/g/asset-managers/ may help.
You use modular javascript.
Choose your packager of choice (mine is browserify) that packages all your modules into one package that you minify and gzip. You send this file to the client and it is cached.
This means you have all your code cached, minimize HTTP requests and stay lean and efficient.
And since you have modular code you just load your code as you would normally.
Personally I would use some form feature detection to load modules. You can choose to feature detect on almost any feature (some css selector, routes, url segments).
Feature detection would look like this :
var Features = {
"class": "name",
"class2": "name2",
"dynamic-scroll": "dynamic-scroll",
"tabstrip": "tabstrip",
...
}
for (var key in Features) {
require(Features[key]);
}
Where as routing with davis would look like
Davis(function() {
this.get("blog", function(req) {
require("blog")(req);
});
this.get("blog/:post", function(req) {
require("blog-post")(req);
});
this.get("shop", function(req) {
require("shop")(req);
});
...
});
Alternatively you can try an event driven architecture. This means each module binds to events
// some-module
mediator.on("blog-loaded", function() {
// load in some libraries
// construct some widgets
mediator.emit("blog-ui-build", widgets);
});
And you would need some bootstrapping in place to kick off the event loop. Feel free to look at an EDA demo
I'd like to build my own, but I'm not sure about the best way to do it. A partial is a template that is only a part of another bigger template and which can be inserted into multiple other templates at will.
Templating itself is fairly basic, just string exctraction and concatenation, but clientside partials have me a little stumped.
Here are a few methods I thought about:
1,
I could write a javascript helper function that loads partials through ajax into some form of local storage I suppose, and all subsequent templates that require that particular partial would first look inside local storage. I think this method isn't very safe because local storage isn't always guaranteed. And if I can't save them into local storage, partials would result in too many ajax calls.
2, I could put them all into script tags inside my main html file. This would work reasonably well, especially with head.js (to enable parallel loading of script tags), but still - I think each script tag is a separate call to the server right? That doesn't exactly improve the situation.
3, I could put all templates into a single script tag (or html I guess) and manually filter through some kind of delimiter...like: "#template1(blabla template1 string) #template2(blablabla template2 string) and put those strings into globals.
This would result only into a single call to the server, all the rest is done on the client.
Suggestions? I have looked at existing templating engines, but I can't really determine how they do it. The code is pretty complicated
The approach I took to spec out template calls and on-demand loads for the spec/rewrite of jQuery templates is to pipeline it.
See section 9 of the (early) draft spec, and see the conformance suite tests at the bottom for an example of custom on-demand template loading (Testcase "Main calls and Loaded just in time!" is the relevant one).
The basic gist is that plugin loaders (written in JS) get to hook in between-parsing and compiling to inspect the parse tree. A plugin pass gets an object mapping template names to parse trees. If they see any partial template selectors (to use your parlance) they can try and load any unresolved templates using AJAX calls or file I/O on Node.js, and add the partials to the input object to cause the compiler to compile the just loaded partials along with the public templates.
Efficiency-wise, see the benchmarks. I'm in the process of migrating the code to github : https://github.com/mikesamuel/jquery-jquery-tmpl-proposal in case you want to collaborate.