What I'm trying to do is offer search functionality on a webpage where all data is displayed when loading the page. The dataset is not very large (think <500 items) so it would be very user-friendly to offer some search functionality that does not require a page refresh (or any requests for that matter). I've looked at some interesting Javascript libraries (summed up in this answer), and especially Fuse.js and Lunr.js look very suitable for my scenario.
The problem is that I don't know how to apply these libraries -- which seem to search through JSON exclusively -- to the context passed by Django. I have only scratched the surface of the functionality of Django, because I've only started working on it a few days ago, so pardon me if I'm missing something obvious. Thank you!
You just need to output the JSON in your template. You can either pass it directly from your view via json.dumps(), or manually iterate through the data in your template to create the JSON. The first would be preferable.
So, using the Fuse example, your view would do:
def my_view(request):
data = ... get data from wherever ...
return render(request, 'template.html', {"data": json.dumps(data)})
and the template:
<script type="text/javascript">
var data = {{ json_data }}
var options = ...
var f = new Fuse(data, options)
</script>
While Daniel (in the comments) is certainly correct for telling me how to add JSON to the context - I probably should have phrased the question differently. But I digress.
After some more work I am now generating a correct JSON from the object hierarchy. This JSON is passed through the context, and is then used together with Mustache.js to generate HTML. I could not figure out a way to pass data as JSON and also use Django's templating language to iterate through the JSON. When approaching this from the other direction, I could not figure out a way to convert the regular Django context (e.g. a context with var.objects.all()) to JSON on the client-side (because of some silly nonsense with single-quotes..)
Nevertheless generating HTML with Mustache.js was not so bad after all, because this allowed me to easily manipulate the page with the search results generated by the search algorithm (in my case Fuse.js). Hopefully the performance will be al right in production, but it certainly looks very promising!
Related
We use DNN and often need to pass a few context specific values (like page id or module-on-page-id) into an AngularJS app. We've developed our own conventions how to do this, but would like to hear from others how they approach this to find a best practice.
So basically the situation is that the server-page has information needed by the JS. Using WebAPI is not an option, as these values are known in the page, but not in a separate request. Things I've seen so far have been:
Use in-view-razor like href="#Tab.TabId/{{...}}" (I don't like this)
Place the values in the ng-init like ng-init="config = { prop1: '#Tab.TabId' }"
Create a separate <script> tag where we generate a module on the fly containing these values so angular.module("config", []).constant('prop1', '#Tab.TabId')
Create a json with razor in the page somewhere and inject all of it as a module into the app using a generic code which does the same as #3, just with cleaner code re-use.
I've seen all these and have also used all. Currently we avoid #1 as we believe it's not good to mix templating languages and because it doesn't allow externalizing parts of the view. So basically we use #2 as for quick-and-simple (and a bit dirty) + #3/#4 for larger projects.
Do you have a better way, or which way would you prefer?
We are using variant #4.
This has the advantage that the JSON defines the exact interface for the config needed by the JS module. And Razor is great to generate URLs using #Url.Action.
we use NewtonSoft and do JSONConvert.SerializeObject(ObjectName) and then pass it over as a Session from the controller and then use #Html.Raw(ObjectName) and its a JSON Object that can be utilized easily in javascript...
I am now totally confused with the usage of data structures in bottle...
Now I am using a Jquery tool ztree to build a tree in my web page.
index.py:
data = [{'name':'1'},
{'name':'2'}]
return template('index', data)
But, when I try to get data in my JavaScript code as {{data}}, and then pass it to ztree to build my tree, it shows nothing.
On the other hand, if I pass [{'name':1'},{'name':2}] directly to ztree, a tree is built as expected. So what's the difference between the two?
You are passing in a Python structure, which a Javascript framework won't know anything about. The string representation interpolated into a template may look a lot like a JavaScript structure but there are (many) differences.
Translate it to a structure Javascript can read by using the json module:
import json
# ...
data = json.dumps(data)
then interpolate that into your Javascript code as a (unescaped) variable:
data = {{!data}};
A psuedo controller method
#RequestMapping("/foo")
public String getFoo(Model model) {
model.add("foo", repo.findFoo());
model.add("bar", repo.findBar());
model.add("barOptions", repo.findBarOptions(bar));
return "fooView";
}
Let's say the client uses Expression language to render foo and bar; but we use JavaScript to render barOptions.
<html>
<script>
var options = <mytag:toJSON object="${barOptions}"/>;
$("#options").renderOptions( options );
</script>
<body>
<mytag:renderFoo foo="${foo}"/>
<mytag:renderBar foo="${bar}"/>
<ul id="options"></ul>
</body>
</html>
Common conventions tells me this is bad. But the essence of MVC, where the controller sends data and the view determines how to use it, tells me this is good. Is there a better way to do the same thing? Is there any reason why this isn't commonly done? I could request the JSON using a separate call, but then I have to make more requests for the page to load, and there may be logic to determine barOptions in the controller method getFoo() based on other input at the time of the page load.
At first glance I can not say that I see anything blatantly wrong with approach. The only aspect that initially took me off guard was your need to convert data in the model object to json.
For me, JSON usually implies that there is some sort of server side object that needs to be converted so that a client side javascript can access or manipulate its structure in a javascript way. I guess without knowing more of the purpose of the options list, I can't see a reason why json serialization is required here.
By using a Tag to convert a model object to JSON, we avoid an
additional request made by the client
But if we assume that JSON is a requirement (perhaps for some third party jquery plugin), then I absolutely do not see anything wrong with approach.
What is special or different about the barOptions unordered list, why does it have be rendered with json? Why not just use a for loop to build the list items? Or you can have a custom tag that builds out the ul entirely.
Aside from that, I missing the point as how one may perceive this as being bad code.
Normally JSON is requested by JavaScript with an Ajax call, but if in your case it's available at page rendering time, I see nothing wrong with your solution. It's clean, compact code and easy to read. Looks perfectly OK to me, the alternative would be to loop on the options array with a forEach, but this approach looks better.
I'm using Django and Apache to serve webpages. My JavaScript code currently includes a data object with values to be displayed in various HTML widgets based on the user's selection from a menu of choices. I want to derive these data from a Python dictionary. I think I know how to embed the JavaScript code in the HTML, but how do I embed the data object in that script (on the fly) so the script's functions can use it?
Put another way, I want to create a JavaScript object or array from a Python dictionary, then insert that object into the JavaScript code, and then insert that JavaScript code into the HTML.
I suppose this structure (e.g., data embedded in variables in the JavaScript code) is suboptimal, but as a newbie I don't know the alternatives. I've seen write-ups of Django serialization functions, but these don't help me until I can get the data into my JavaScript code in the first place.
I'm not (yet) using a JavaScript library like jQuery.
n.b. see 2018 update at the bottom
I recommend against putting much JavaScript in your Django templates - it tends to be hard to write and debug, particularly as your project expands. Instead, try writing all of your JavaScript in a separate script file which your template loads and simply including just a JSON data object in the template. This allows you to do things like run your entire JavaScript app through something like JSLint, minify it, etc. and you can test it with a static HTML file without any dependencies on your Django app. Using a library like simplejson also saves you the time spent writing tedious serialization code.
If you aren't assuming that you're building an AJAX app this might simply be done like this:
In the view:
from django.utils import simplejson
def view(request, …):
js_data = simplejson.dumps(my_dict)
…
render_template_to_response("my_template.html", {"my_data": js_data, …})
In the template:
<script type="text/javascript">
data_from_django = {{ my_data }};
widget.init(data_from_django);
</script>
Note that the type of data matters: if my_data is a simple number or a string from a controlled source which doesn't contain HTML, such as a formatted date, no special handling is required. If it's possible to have untrusted data provided by a user you will need to sanitize it using something like the escape or escapejs filters and ensure that your JavaScript handles the data safely to avoid cross-site scripting attacks.
As far as dates go, you might also want to think about how you pass dates around. I've almost always found it easiest to pass them as Unix timestamps:
In Django:
time_t = time.mktime(my_date.timetuple())
In JavaScript, assuming you've done something like time_t = {{ time_t }} with the results of the snippet above:
my_date = new Date();
my_date.setTime(time_t*1000);
Finally, pay attention to UTC - you'll want to have the Python and Django date functions exchange data in UTC to avoid embarrassing shifts from the user's local time.
EDIT : Note that the setTime in javascript is in millisecond whereas the output of time.mktime is seconds. That's why we need to multiply by 1000
2018 Update: I still like JSON for complex values but in the intervening decade the HTML5 data API has attained near universal browser support and it's very convenient for passing simple (non-list/dict) values around, especially if you might want to have CSS rules apply based on those values and you don't care about unsupported versions of Internet Explorer.
<div id="my-widget" data-view-mode="tabular">…</div>
let myWidget = document.getElementById("my-widget");
console.log(myWidget.dataset.viewMode); // Prints tabular
somethingElse.addEventListener('click', evt => {
myWidget.dataset.viewMode = "list";
});
This is a neat way to expose data to CSS if you want to set the initial view state in your Django template and have it automatically update when JavaScript updates the data- attribute. I use this for things like hiding a progress widget until the user selects something to process or to conditionally show/hide errors based on fetch outcomes or even something like displaying an active record count using CSS like #some-element::after { content: attr(data-active-transfers); }.
For anyone who might be having a problems with this, be sure you are rendering your json object under safe mode in the template. You can manually set this like this
<script type="text/javascript">
data_from_django = {{ my_data|safe }};
widget.init(data_from_django);
</script>
As of mid-2018 the simplest approach is to use Python's JSON module, simplejson is now deprecated. Beware, that as #wilblack mentions you need to prevent Django's autoescaping either using safe filter or autoescape tag with an off option. In both cases in the view you add the contents of the dictionary to the context
viewset.py
import json
def get_context_data(self, **kwargs):
context['my_dictionary'] = json.dumps(self.object.mydict)
and then in the template you add as #wilblack suggested:
template.html
<script>
my_data = {{ my_dictionary|safe }};
</script>
Security warning:
json.dumps does not escape forward slashes: an attack is {'</script><script>alert(123);</script>': ''}. Same issue as in other answers. Added another answer hopefully fixing it.
You can include <script> tags inside your .html templates, and then build your data structures however is convenient for you. The template language isn't only for HTML, it can also do Javascript object literals.
And Paul is right: it might be best to use a json module to create a JSON string, then insert that string into the template. That will handle the quoting issues best, and deal with deep structures with ease.
It is suboptimal. Have you considered passing your data as JSON using django's built in serializer for that?
See the related response to this question. One option is to use jsonpickle to serialize between Python objects and JSON/Javascript objects. It wraps simplejson and handles things that are typically not accepted by simplejson.
Putting Java Script embedded into Django template is rather always bad idea.
Rather, because there are some exceptions from this rule.
Everything depends on the your Java Script code site and functionality.
It is better to have seperately static files, like JS, but the problem is that every seperate file needs another connect/GET/request/response mechanism. Sometimes for small one, two liners code os JS to put this into template, bun then use django templatetags mechanism - you can use is in other templates ;)
About objects - the same. If your site has AJAX construction/web2.0 like favour - you can achieve very good effect putting some count/math operation onto client side. If objects are small - embedded into template, if large - response them in another connection to avoid hangind page for user.
Fixing the security hole in the answers by #willblack and #Daniel_Kislyuk.
If the data is untrusted, you cannot just do
viewset.py
def get_context_data(self, **kwargs):
context['my_dictionary'] = json.dumps(self.object.mydict)
template.html
<script>
my_data = {{ my_dictionary|safe }};
</script>
because the data could be something like
{"</script><script>alert(123);</script>":""}
and forward slashes aren't escaped by default. Clearly the escaping by json.dumps may not 100% match the escaping in Javascript, which is where the problems come from.
Fixed solution
As far as I can tell, the following fixes the problem:
<script>
my_data = JSON.parse("{{ my_dictionary|escapejs }}");
</script>
If there are still issues, please post in the comments.
I'm working on an AJAXy project (Dojo and Rails, if the particulars matter). There are several places where the user should be able to sort, group, and filter results. There are also places where a user fills out a short form and the resulting item gets added to a list on the same page.
The non-AJAXy implementation works fine -- the view layer server-side already knows how to render this stuff, so it can just do it again in a different order or with an extra element. This, however, adds lots of burden to the server.
So we switched to sending JSON from the server and doing lots of (re-)rendering client-side. The downside is that now we have duplicate code for rendering every page: once in Rails, which was built for this, and once in Dojo, which was not. The latter is basically just string concatenation.
So question part one: is there a good Javascript MVC framework we could use to make the rendering on the client-side more maintainable?
And question part two: is there a way to generate the client-side views in Javascript and the server-side views in ERB from the same template? I think that's what the Pragmatic Programmers would do.
Alternatively, question part three: am I completely missing another angle? Perhaps send JSON from the server but also include the HTML snippet as an attribute so the Javascript can do the filtering, sorting, etc. and then just insert the given fragment?
Well, every time you generate HTML snippets on the client and on the server you may end up with duplicated code. There is no good way around it generally. But you can do two things:
Generate everything on the server. Use AHAH when you need to generate HTML snippets dynamically. Basically you ask server to generate an HTML fragment, receive it asynchronously, and plug it in place using innerHTML or any other similar mechanism.
Generate everything on the client (AKA the thick client paradigm). In this case even for the initial rendering you pass data instead of pre-rendered HTML, and process the data client-side using JavaScript to make HTML. Depending on the situation you can use the data island technique, or request data asynchronously. Variant: include it as <script> using JSONP so the browser will make a request for you while loading the page.
Both approaches are very simple and have different set of pros and cons. Sometimes it is possible to combine both techniques within one web application for different parts of data.
Of course you can go for exotic solutions, like using some JavaScript-based server-side framework. In this case you can share the code between the server and the client.
I don't have a complete answer for you; I too have struggled with this on a recent project. But, here is a thought:
Ajax call to Rails
Rails composes the entire grid again, with the new row.
Rails serializes HTML, which is returned to the browser.
Javascript replaces the entire grid element with the new HTML.
Note: I'm not a Rails person, so I'm not sure if those bits fit.
Has anyone tried something like the following? There's redundant data now, but all the rendering is done server-side and all the interaction is done client-side.
Server Side:
render_table_initially:
if nojs:
render big_html_table
else:
render empty_table_with_callback_to_load_table
load_table:
render '{ rows: [
{ foo: "a", bar: "b", renderedHTML: "<tr><td>...</td></tr>" },
{ foo: "c", bar: "d", renderedHTML: "<tr><td>...</td></tr>" },
...
]}'
Client side:
dojo.xhrGet({
url: '/load_table',
handleAs: 'json',
load: function(response) {
dojo.global.all_available_data = response;
dojo.each(response.rows, function(row) {
insert_row(row.renderedHTML);
}
}
});
Storing the all_available_data lets you do sorting, filtering, grouping, etc. without hitting the server.
I'm only cautious because I've never heard of it being done. It seems like there's an anti-pattern lurking in there...
"Perhaps send JSON from the server but also include the HTML snippet as an attribute so the Javascript can do the filtering, sorting, etc. and then just insert the given fragment?"
I recommend keeping the presentation layer on the client and simply downloading data as needed.
For a rudimentary templating engine, you can extend Prototype's Template construct:
http://www.prototypejs.org/api/template
As your client scales and you need a rich and flexible MVC, try PureMVC.
http://puremvc.org/content/view/102/181/
As in regular server side programming you should strive to encapsulate your entities with controls, in your case client side controls that has data properties and also render methods + events.
So for example lets say you have on the page an are that shows tree of employees, effectively you should encapsulate it behavior in a client side class that can get JSON object of the employee list / by default connect to the service and a render method to render the output, events and so on.
In ExtJS its is relatively easy to create these kind of controls - look at this article.
Maybe I'm not fully understanding your problem but this is how I would solve your requirements:
Sort, Filter
It can be done all in JavaScript, without hitting the server. It's a problem of manipulating the table rows, move rows for sorting, hide rows for filtering, there is no need for re-rendering. You will need to mark columns with the data type with custom attributes or extra class names in order to be able to parse numbers or dates.
If your reports are paginated I think it's easier and better to refresh the whole table or page.
Group
I can't help here as I don't understand how will you enable grouping. Are you switching columns to show accumulates? How do you show the data that doesn't support accumulates like text or dates?
New item
This time I would use AJAX with a JSON response to confirm that the new item was correctly saved and possibly with calculated data.
In order to add the new item to the results table I would clone a row and alter the values. Again we don't need to render anything in client-side. The last time I needed something like this I was adding an empty hidden row at the end of the results, this way you will always have a row to clone no matter if the results are empty.
Number 5 in my list of five AJAX styles tends to work pretty well.