Recently I've been working with leaflets, and I'm currently looking at several plugins.
Several of the plugins I've seen (including Leaflet.markercluster) use json to plot points, but instead of using the actual stream, or a json file, the programmer includes a javascript .js file where a variable is set with the json stream. So the js file with the json data looks like this:
var data = [
{"loc":[41.575330,13.102411], "title":"aquamarine"},
{"loc":[41.575730,13.002411], "title":"black"},
{"loc":[41.807149,13.162994], "title":"blue"},
{"loc":[41.507149,13.172994], "title":"chocolate"}
]
I've been working with other type of javascript charts, and most of them read and process a json stream.
It seems these plugins will not work if I create a service that returns json.
Why not use json instead of including a js file that sets a variable with a json stream?
I'm not a javascript expert, but I find it easier to generate json than a javascript file with json in it.
You are wrong about concepts.
1st. JavaScript as a language has its own syntax, so, if you have a function that receive a JSON object as a parameter and you pass it a Number or a String, it'll will throw an Error when you try to access some property. For Ex.
function myjson (obj) {
console.log(obj.prop)
}
myjson(34); //wrong
myjson("{prop: 123}") //wrong
myjson({prop: 123}) //Good, will print 123
Now, imagine that you have some scripts, many .js files that you have indexed in your HTML file, like
<script src="/mycode.js"> </script>
<script src="/myapp.js"> </script>
And you want to add some data, like the one you show for the plot points; then you have to include it in two ways, putting that in a .js file or getting it from a service with an AJAX call.
If you add that in a .js file, you'll have access to them directly from your code, like this
var data = [
{"loc":[41.575330,13.102411], "title":"aquamarine"},
{"loc":[41.575730,13.002411], "title":"black"},
{"loc":[41.807149,13.162994], "title":"blue"},
{"loc":[41.507149,13.172994], "title":"chocolate"}
]
console.log(data)
and if you put that in a .json file file this
/mydata.json
[
{"loc":[41.575330,13.102411], "title":"aquamarine"},
{"loc":[41.575730,13.002411], "title":"black"},
{"loc":[41.807149,13.162994], "title":"blue"},
{"loc":[41.507149,13.172994], "title":"chocolate"}
]
you'll have to fetch and parse the data yourself
fetch("/mydata.json").then(async data => {
var myjson = await data.text();
myjson = JSON.parse(myjson);
console.log(myjson) //A Javascript object
console.log(myjson[1]) //The first element
})
I like #FernandoCarvajal's answer, but I would add more context to it:
JSON is more recent than JS (you could see JSON as a "spin-off" of JS, even though it is now used in combination with much more languages than just JS).
Before JSON was widespread, the main and easiest way to load external data in Browsers was the technique you saw in the plugins demo: assign data into a global variable, which you can use in the next script. Browsers happily execute JS even from cross domain (unless you explicitly specify a Content Security Policy). The only drawback is that you have to agree on a global variable name beforehand. But for static sites (like GitHub pages in the case of the plugins demo you mention), it is easy for the developer(s) to agree on such a convention.
At this stage, you should understand that using this simple technique already fits the job for the sake of the plugins static demo. It also avoids browsers compatibility issues, aligning with Leaflet wide browsers compatibility.
With the advent of richer clients / Front-End, typically with AJAX, we can get rid of the global variable name agreement issue, but now we may face cross domain difficulty, as pointed out by #Barmar's comment. We also start hitting browsers compatibility hell.
Now that we can load arbitrary data without having to agree on a static name beforehand, we can leverage Back-End served dynamic content to a bigger scale.
To workaround the cross domain issue, and before CORS was widespread, we started using JSONP: the Front-End specifies the agreed (callback) name in its request. But in fact we just fallback to a similar technique as in point 2, mainly adding asynchronicity.
Now that we have CORS, we can more simply use AJAX / fetch techniques, and avoid security issues inherent to JSONP. We also replace the old school XML by the more JS-like JSON.
There is nothing preventing you from replacing the old school technique in point 2 by a more modern JSON consumption. If you want to ensure wide browsers compatibility, make sure to use libraries that take care of it (like jQuery, etc.). And if you need cross domain, make sure to enable CORS.
Related
I'm writing a WebGL application. Until now I put my shaders in <script>-tags right into my standard html file (let's call it index.html). So it was easy to just "parse" the shader-code by using .text on the script-tags taken with getElementbyId('foo').
But now I think that's untidy. I want my shaders to be stored in external files with extension .vert /.frag and parse them from there. My first approach was the following:
<script id="shader-vs" src="./shaders/basic_vertexShader.vert" type="x-shader/x-vertex"></script>
<script id="shader-fs" src="./shaders/basic_fragmentShader.frag" type="x-shader/x-fragment"></script>
var shaderScript = document.getElementById(id);
if (!shaderScript) {
return null;
}
var str = shaderScript.text;
The result is an empyt string (""). I double checked if all file-paths are correct. It seems, that .text only works for lokal tags.
So, what could I do, to receive the contents of that files? AJAX isn't a possibility, since the application should work without internet-connection, too, when it's saved on clientside's storage.
I'm thankful for every suggestion.
Important: As I mentioned I'm NOT searching for that usual AJAX-stuff. I'm searching for a possibility to get the code of external linked script-files. Since they are statically linke in the HTML-file, I thought this should be no problem without asynchrone requests. But it seems to be.
I recommend using server-side templating to store the code in the files you mentioned but then inject it into the html. This method presents a separation of concerns: your shader files are separate the way you want, the template gets compiled then shipped out when the HTTP request comes in. A mustachey example using pseudo-code (assumes hash-literal syntax for your server-side language):
html = system.readFile('index.html');
vert = system.readFile('myVertFile.vert');
Mustache.render(html, {data: vert});
where your HTML looks like this:
<script>{{data}}</script>
This method can also be used to do any sort of conditional module loading based on the contents of the incoming http request.
You can still use Ajax if you make it an offline (see comments!) html5 web app. (It depends of course on what browsers you need to support at the moment.)
My script adds some annotations to each page on a site, and it needs a few MBs of static JSON data to know what kind of annotations to put where.
Right now I'm including it with just var data = { ... } as part of the script but that's really awkward to maintain and edit.
Are there any better ways to do it?
I can only think of two choices:
Keep it embedded in your script, but to keep maintainable(few megabytes means your editor might not like it much), you put it in another file. And add a compilation step to your workflow to concatenate it. Since you are adding a compilation you can also uglify your script so it might be slightly faster to download for the first time.
Get it dynamically using jsonp. Put it on your webserver, amazon s3 or even better, a CDN. Make sure it will be server cachable and gzipped so it won't slow down the client network by getting downloaded on every page! This solution will work better if you want to update your data regularly, but not your script(I think tampermonkey doesn't support auto updates).
My bet would would definetly be to use special storage functions provided by tampermonkey: GM_getValue, GM_setValue, GM_deleteValue. You can store your objects there as long as needed.
Just download the data from your server once at the first run. If its just for your own use - you can even simply insert all the data directly to a variable from console or use temporary textarea, and have script save that value by GM_setValue.
This way you can even optimize the speed of your script by having unrelated objects stored in different GM variables.
I'm a bit new to javascript. Is there a way to do what I am describing in the title completely client-side and without any external libraries? Or is using jQuery the best/only way to go?
You can import a json file from a server via AJAX and them simply eval it. You don't need a library for that but using one makes it a lot easier. Of course just evaling a json string is not very secure as it can contain arbitrary text so all libraries parse it to see if it's well formed etc.
EDIT:
If you want to learn about AJAX you can start with this tutorial from w3schools. Ajax stands for Asynchronous Javascript And XML and it allows you to send a request to the server without reloading the whole page. In your case you will not be using Xml but JSON. Anyway, the tutorial explains the whole idea.
Yes there is. You can use the "document.write" to add scripts to the DOM at runtime:
in your case:
document.write('<script ...></script>');
Basically you are adding the script tag to the dom that will request the new file.
However there is something else to consider, although the script will be downloaded, you will need to have a variable assignment in it in order to use it in your page:
var x = { //json object };
I'm using Django and Apache to serve webpages. My JavaScript code currently includes a data object with values to be displayed in various HTML widgets based on the user's selection from a menu of choices. I want to derive these data from a Python dictionary. I think I know how to embed the JavaScript code in the HTML, but how do I embed the data object in that script (on the fly) so the script's functions can use it?
Put another way, I want to create a JavaScript object or array from a Python dictionary, then insert that object into the JavaScript code, and then insert that JavaScript code into the HTML.
I suppose this structure (e.g., data embedded in variables in the JavaScript code) is suboptimal, but as a newbie I don't know the alternatives. I've seen write-ups of Django serialization functions, but these don't help me until I can get the data into my JavaScript code in the first place.
I'm not (yet) using a JavaScript library like jQuery.
n.b. see 2018 update at the bottom
I recommend against putting much JavaScript in your Django templates - it tends to be hard to write and debug, particularly as your project expands. Instead, try writing all of your JavaScript in a separate script file which your template loads and simply including just a JSON data object in the template. This allows you to do things like run your entire JavaScript app through something like JSLint, minify it, etc. and you can test it with a static HTML file without any dependencies on your Django app. Using a library like simplejson also saves you the time spent writing tedious serialization code.
If you aren't assuming that you're building an AJAX app this might simply be done like this:
In the view:
from django.utils import simplejson
def view(request, …):
js_data = simplejson.dumps(my_dict)
…
render_template_to_response("my_template.html", {"my_data": js_data, …})
In the template:
<script type="text/javascript">
data_from_django = {{ my_data }};
widget.init(data_from_django);
</script>
Note that the type of data matters: if my_data is a simple number or a string from a controlled source which doesn't contain HTML, such as a formatted date, no special handling is required. If it's possible to have untrusted data provided by a user you will need to sanitize it using something like the escape or escapejs filters and ensure that your JavaScript handles the data safely to avoid cross-site scripting attacks.
As far as dates go, you might also want to think about how you pass dates around. I've almost always found it easiest to pass them as Unix timestamps:
In Django:
time_t = time.mktime(my_date.timetuple())
In JavaScript, assuming you've done something like time_t = {{ time_t }} with the results of the snippet above:
my_date = new Date();
my_date.setTime(time_t*1000);
Finally, pay attention to UTC - you'll want to have the Python and Django date functions exchange data in UTC to avoid embarrassing shifts from the user's local time.
EDIT : Note that the setTime in javascript is in millisecond whereas the output of time.mktime is seconds. That's why we need to multiply by 1000
2018 Update: I still like JSON for complex values but in the intervening decade the HTML5 data API has attained near universal browser support and it's very convenient for passing simple (non-list/dict) values around, especially if you might want to have CSS rules apply based on those values and you don't care about unsupported versions of Internet Explorer.
<div id="my-widget" data-view-mode="tabular">…</div>
let myWidget = document.getElementById("my-widget");
console.log(myWidget.dataset.viewMode); // Prints tabular
somethingElse.addEventListener('click', evt => {
myWidget.dataset.viewMode = "list";
});
This is a neat way to expose data to CSS if you want to set the initial view state in your Django template and have it automatically update when JavaScript updates the data- attribute. I use this for things like hiding a progress widget until the user selects something to process or to conditionally show/hide errors based on fetch outcomes or even something like displaying an active record count using CSS like #some-element::after { content: attr(data-active-transfers); }.
For anyone who might be having a problems with this, be sure you are rendering your json object under safe mode in the template. You can manually set this like this
<script type="text/javascript">
data_from_django = {{ my_data|safe }};
widget.init(data_from_django);
</script>
As of mid-2018 the simplest approach is to use Python's JSON module, simplejson is now deprecated. Beware, that as #wilblack mentions you need to prevent Django's autoescaping either using safe filter or autoescape tag with an off option. In both cases in the view you add the contents of the dictionary to the context
viewset.py
import json
def get_context_data(self, **kwargs):
context['my_dictionary'] = json.dumps(self.object.mydict)
and then in the template you add as #wilblack suggested:
template.html
<script>
my_data = {{ my_dictionary|safe }};
</script>
Security warning:
json.dumps does not escape forward slashes: an attack is {'</script><script>alert(123);</script>': ''}. Same issue as in other answers. Added another answer hopefully fixing it.
You can include <script> tags inside your .html templates, and then build your data structures however is convenient for you. The template language isn't only for HTML, it can also do Javascript object literals.
And Paul is right: it might be best to use a json module to create a JSON string, then insert that string into the template. That will handle the quoting issues best, and deal with deep structures with ease.
It is suboptimal. Have you considered passing your data as JSON using django's built in serializer for that?
See the related response to this question. One option is to use jsonpickle to serialize between Python objects and JSON/Javascript objects. It wraps simplejson and handles things that are typically not accepted by simplejson.
Putting Java Script embedded into Django template is rather always bad idea.
Rather, because there are some exceptions from this rule.
Everything depends on the your Java Script code site and functionality.
It is better to have seperately static files, like JS, but the problem is that every seperate file needs another connect/GET/request/response mechanism. Sometimes for small one, two liners code os JS to put this into template, bun then use django templatetags mechanism - you can use is in other templates ;)
About objects - the same. If your site has AJAX construction/web2.0 like favour - you can achieve very good effect putting some count/math operation onto client side. If objects are small - embedded into template, if large - response them in another connection to avoid hangind page for user.
Fixing the security hole in the answers by #willblack and #Daniel_Kislyuk.
If the data is untrusted, you cannot just do
viewset.py
def get_context_data(self, **kwargs):
context['my_dictionary'] = json.dumps(self.object.mydict)
template.html
<script>
my_data = {{ my_dictionary|safe }};
</script>
because the data could be something like
{"</script><script>alert(123);</script>":""}
and forward slashes aren't escaped by default. Clearly the escaping by json.dumps may not 100% match the escaping in Javascript, which is where the problems come from.
Fixed solution
As far as I can tell, the following fixes the problem:
<script>
my_data = JSON.parse("{{ my_dictionary|escapejs }}");
</script>
If there are still issues, please post in the comments.
I observed chunks like below sometimes on web pages. So i am curious to know what this really does? or why its written in such a way?
<script src="somefile.js?param1=one¶m2=two" />
i can only make out following few intentions behind it
Its not page URL (i mean .aspx/.php/.jsp etc.) so its not hacking kind of code where user can add code like this to pass data without getting users attention as its tag which does not render on UI OR implementing old type of AJAX alternative
This kind of URL param are useful if user do not wish the JS file (any other resource like image) to get cached. This can be quick way to manage caching
But i am unable to figure out following
Looks like page URL parameters but are these parameters anyway readable in JavaScript file and have some additional utility?
Do these parameters have any extra role to play here ?
What are the other possible practical scenarios where code like this can be/is used?
So please provide some inputs related with the same
Thanks,
Running Non-JS Code within .js Extensions
In cases like this, that source .js file might (given proper server-configurations) actually have PHP/.NET code within it, which can read those appended values.
As you said, Avoiding Cache...
Additionally, people will at times append a random string at the end of their referenced elements to avoid loading cached data.
The URL having '.js' means nothing. It could still be handled by a server-side script like an ASP or PHP.
Either the javascript file is not static (it is generated by the server based on the parameters in its querystring)
OR
In the JavaScript file itself, you can have it check its own querystring parameters (not just that of the page, but that of the javascript source url).
OR
(This doesn't exactly match your scenario, but) you can also add parameters at the end of image and script urls as a way of versioning. The version with the url="somescript.js?V=3" will be cached by the user until the page then changes and the url is not="somescript.js?V=4". The file will be replaced by the version on the server no matter what the browser setting may be.
My guess (without looking at this specific case) is that the javascript file is reading its own querystring. I have done this, and its very helpful.
Looks like page URL parameters but are these parameters anyway readable in JavaScript file and have some additional utility?
Yes you can read them in JavaScript, Scriptaculous uses that approach for loading modules, eg:
<script type="text/javascript" src="scriptaculous.js?load=effects,dragdrop">
</script>
Do these parameters have any extra role to play here ?
What are the other possible practical scenarios where code like this can be/is used?
That can be also used for server-side script joining and minifying, of course using some url rewriting technique to have the .js extension, and as you say, it's a common technique to add timestamp parameters to break the browser cache.
It can be used for three different reasons:
1) To generate the JavaScript file in the server depending on the parameters;
2) To avoid caching;
3) To pass parameters to JavaScript itself
An example of this in practice would be a server side handler for somefile.js that uses the parameters (names of other scripts) to determine which scripts are actually required and combine/minify them, returning them as a single somefile.js script file.