LSL HttpServer - Serving large Notecards from Prim Inventory on Secondlife - javascript

I am writing a Media-HUD that runs totally on local notecard files stored within the prim's inventory, with notecards named like index.html, style.css, icon.svg etc.
My hope is to use the LSL HttpServer functions, and the script's URL to create a totally self-contained media based HUD that is easy to edit like editing any web page.
This is completely possible on its own, however there is a limitation in that, the pages must fit into the memory allocated to the LSL script. Under mono this is only 64kb.
I want to remove this limitation, by somehow, perhaps from javascript, reading in each 'file' from a notecard line by line in the users browser itself (thusly, getting around the memory limit by only bringing one notecard line into memory at a time).
Is there a way to do this? generate a entire file in javascript procedurally by loading in the strings making it up line by line, and then serve it as though it were a whole file? I'm not sure how feasible this is.
Any idea's/guidance greatly appreciated!

You could do this through Javascript using XMLHttpRequest. jQuery's wrapper for this is called Ajax. You could request each line individually, which would be slightly slower, or read in a number of lines at a time, at the script's leisure. http_request is not throttled so either works. Note that the loader has to be sent in a single response, because the LSL server has no way of pushing data "piecemeal" like an actual server does.
Notes:
llGetNotecardLine only returns the first 255 bytes per line.
llHTTPResponse must be called within ~20 seconds of the request, so you can't feasibly read more than 20 lines from a notecard at a time.
I'm not sure how this would work for non-DOM filetypes. All files would need to be embed-able in the HTML using the Javascript DOM. To my knowledge, Javascript can't arbitrarily create an external file and serve it to itself. Obviously it would not work for non-text filetypes, but you could certainly load in the rest of the HTML, CSS, and HTML5 SVG. Basically, if it can go in a single HTML file, you could load it in through Javascript.
I have no experience with React but it gives a good example of what is possible on the UI side with loading things in purely through Javascript.

So less than 64 thousand characters in memory at most per script. I will give you some advise that might make your task feasable:
External resources
Minimize the amount of code you have to have in your notecards by sourcing popular libraries from the web Bootstrap, React and such.
You will have to rely on their mirror's availability thought. But it will greatly reduce the amount of memory needed to provide pretty and functional pages.
Minify your code
Use tools like Uglify or Closure Compiler to make your javascript lighter. Though you must be careful, as these tools will fit all your code a single long line by the default, and you can't read lines longer than 255 characters with LSL. Luckily you can customize your options in these tools to limit the amount of characters per line.
Divide and conquer
Since a single script can't handle much memory, make dedicated scripts. One could serve resources (act as a file server, providing your html and js) while the other would receive API request calls, to handle the application's logic.

Related

Duplicate an HTML file (and its content) with a different name in Javascript

I have an HTML file with some Javascript and css applied on.
I would like to duplicate that file, make like file1.html, file2.html, file3.html,...
All of that using Javascript, Jquery or something like that !
The idea is to create a different page (from that kind of template) that will be printed afterwards with different data in it (from a XML file).
I hope it is possible !
Feel free to ask more precision if you want !
Thank you all by advance
Note: I do not want to copy the content only but the entire file.
Edit: I Know I should use server-side language, I just don't have the option ):
There are a couple ways you could go about implementing something similar to what you are describing. Which implementation you should use would depend on exactly what your goals are.
First of all, I would recommend some sort of template system such as VueJS, AngularJS or React. However, given that you say you don't have the option of using a server side language, I suspect you won't have the option to implement one of these systems.
My next suggestion, would be to build your own 'templating system'. A simple implementation that may suit your needs could be something mirroring the following:
In your primary file (root file) which you want to route or copy the other files through to, you could use JS to include the correct HTML files. For example, you could have JS conditionally load a file depending on certain circumstances by putting something like the following after a conditional statement:
Note that while doing this could optimize your server's data usage (as it would only serve required files and not everything all the time), it would also probably increase loading times. Your site would need to wait for the additional HTTP request to come through and for whatever requested content to load & render on the client. While this sounds very slow it has the potential of not being that bad if you don't have too many discrete requests, and none of your code is unusually large or computationally expensive.
If using vanilla JS, the following snippet will illustrate the above:
In a script that comes loaded with your routing file:
function read(text) {
var xhr=new XMLHttpRequest;
xhr.open('GET',text);
xhr.onload=show;
xhr.send();
}
function show() {
var text = this.response;
document.body.innerHTML = text;//you can replace document.body with whatever element you want to wrap your imported HTML
}
read(path/to/file/on/server);
Note a couple of things about the above code. If you are testing on your computer (ie opening your html file on a browser, with a path like file://__) without a local server, you will get some sort of cross origin request error when trying to make an XML request. To bypass this error, either test your code on an actual server (not ideal constantly pushing code, I know) or, preferably, set up a local testing server. If this is something you would want to explore, its not that difficult to do, let me know and I'd be happy to walk you through the process.
Alternately, you could implement the above loading system with jQuery and the .load() function. http://api.jquery.com/load/
If none of the above solutions work for you, let me know more specifically what it is that you need, and I'll be happy to give a more useful/ relevant answer!

Javascript inlined between script tags vs src DATA URI UTF-8 with percent-encoding

I build mobile first and I use tiny frameworks (under 10kB) which I inline in index.html to save on HTTP request.
I looked for days now and it seems like everyone else who inlines javascript does it like this:
<script>UGLIFIED JAVASCRIPT</script>
I do it like this:
<script src="data:application/javascript;utf8, UGLIFIED PERCENT-ENCODED JAVASCRIPT"></script>
You may say percent encoding will make a file much larger but it actually doesn't because the way gzip works- it replaces the repetition and it doesn't matter if the repeated phrase is <div> or %3Cdiv%3E.
My question is- are there any potential advantages of my approach?
PS. One of my ideas was browser caching file-like DATA-URI elements but I don't know if this makes sense since then I would have to also find the way of controlling how to prevent the load of parts of index.html. Unless I could use the cached elements elsewhere - that would have it's use cases too. Thoughts?
First, if your site isn't an SPA, inlining your shared scripts (regardless of method) means you're loading them on every page, negating the value of the browser cache.
Second, the trip across the wire may be similar for encoded vs. not script, but the more important metric is the time it takes for the Javascript to be parsed and compiled. URL decoding isn't free, but while I don't think it's going to matter much in the grand scheme of things, I see no reason why it would actually be faster to load than just script within the tag.

URL to address in page data?

We have a JavaScript widget which loads data from an URL.
To reduce round-trips I would like to avoid a second HTTP request and put the data into the HTML page.
It would be great if I could leave the JavaScript widget unchanged.
Is there a URL scheme to read data from the current HTML page?
Example: Instead of https://.... this dom://....
No, but you can use data URIs, if that's a feasible approach for you. It's not the best choice for large amounts of data though.
I'm not sure to have completely caught your needs, zeroflagL answer could be a correct answer; possibly read also
http://blog.teamtreehouse.com/using-data-uris-speed-website before discarding the option.
Otherwise, although it might take a little adaptation to your javascript, consider that HTML5 has a feature called data blocks
read about it in https://developer.mozilla.org/en/docs/Using_XML_Data_Islands_in_Mozilla:
Leveraging this feature you can reduce round-trips and put one or more dataset into the HTML page, in the case into namespaces script blocks like this:
<script id="purchase-order" type="application/xml">
<purchaseOrder xmlns="http://entities.your.own.domain/PurchaseOrderML">
or this
<script id="another-set-of-data" type="application/xml">
<dataSet xmlns="http://entities.your.own.domain/DataSetML">
therefore, your javascript can access data reading them from the current HTML page; ....example:
<script>
function runDemo() {
var orderSource = document.getElementById("purchase-order").textContent;
var parser = new DOMParser();
var doc = parser.parseFromString(orderSource, "application/xml");
var lineItems = doc.getElementsByTagNameNS("http://entities.your.own.domain/PurchaseOrderML", "lineItem");
var firstPrice = lineItems[0].getElementsByTagNameNS("http://entities.your.own.domain/PurchaseOrderML", "price")[0].textContent;
document.body.textContent = "The purchase order contains " + lineItems.length + " line items. The price of the first line item is " + firstPrice + ".";
}
</script>
I am also an advocate for dataURIs as they are the most transparent (client code-wise) way to implement embedding of data in webpages.
They were, however, first used to embed small images and other resources that would hamper performance due to the connection overhead and also the parallel download limitations of HTTP/1. The tradeoff is delicate since encoding data as dataURIs can cause a (ballpark estimation) of 30% increase in data size, however the critical point where dataURIs stop being helpful is around the size of small images, which are usually orders of magnitude above serialized data.
The critical point here for a single page application scenario is that there's more than the single data-fetch roundtrip to consider.
Embedding the data for use by page's scripts on otherwise static HTML has the following implications:
The HTML itself can't be cached (only with a cached copy for every different set of embedded data and every version of the page.)
The (multiple versions of the) entire page must be generated on a server that also has knowledge of how to get the data.
The inlined data might block page rendering up to a user-perceivable time (this might be worked around by embedding the data at the end of the HTML, but client script execution would probably have to wait completely, thus also making stuff like displaying a loading indicator harder to implement.)
On the other hand, keeping the data on a separate round trip, despite the round trip itself, would:
Probably keep your already working implementation as it is
Allow for clients to use the cached HTML and scripts which would only need a refresh on an actual version change (there was a failed specification called AppCache for this purpose, now superseded by the experimental Service Workers)
Allow for that HTML and scripts to be fully static assets that can be served from 'dumb' CDNs that are faster and closer to the client browser and don't need to query the database or run any server-side code
All those are big wins in my view, so I recommend you to seriously consider the need for embedding data, because it could be an early optimization that can lead to a lot of pain and an actual decrease in performance! Specially because SPDY and now HTTP/2 are already coming in to address these round-trip and connection-number issues.
You could put the data, whatever it is, in the global window object, and use it later on.
But that requires you to change the code.

How to attach large amounts of data with tampermonkey script?

My script adds some annotations to each page on a site, and it needs a few MBs of static JSON data to know what kind of annotations to put where.
Right now I'm including it with just var data = { ... } as part of the script but that's really awkward to maintain and edit.
Are there any better ways to do it?
I can only think of two choices:
Keep it embedded in your script, but to keep maintainable(few megabytes means your editor might not like it much), you put it in another file. And add a compilation step to your workflow to concatenate it. Since you are adding a compilation you can also uglify your script so it might be slightly faster to download for the first time.
Get it dynamically using jsonp. Put it on your webserver, amazon s3 or even better, a CDN. Make sure it will be server cachable and gzipped so it won't slow down the client network by getting downloaded on every page! This solution will work better if you want to update your data regularly, but not your script(I think tampermonkey doesn't support auto updates).
My bet would would definetly be to use special storage functions provided by tampermonkey: GM_getValue, GM_setValue, GM_deleteValue. You can store your objects there as long as needed.
Just download the data from your server once at the first run. If its just for your own use - you can even simply insert all the data directly to a variable from console or use temporary textarea, and have script save that value by GM_setValue.
This way you can even optimize the speed of your script by having unrelated objects stored in different GM variables.

Browser performance impact of lots of js includes

I'm working on a website for work that uses one master layout for the whole site which includes lots (over 40) js files. This website is really slow to render. How much overhead is there for the browser to parse and (for a lack of better technical term) "deal with" all these includes? I know that they are cached, so they are not being downloaded on each page view. However, does each include get parsed and executed anew on every page refresh?
At any rate, I imagine there is some overhead in dealing with all these includes, but I'm not sure if it's big or small.
The best way to understand is to measure. Try merging those 40 js files into a single one and see if it makes a big difference. Also compressing them could reduce bandwidth costs.
There will be an overhead of having multiple includes but as you say those pages are cached and the overhead should be only on the first request. I think that if we ignore this initial overhead the performance difference won't be enormous compared to the time spent in those scripts manipulating the DOM, etc...
it depends on what they do - to test you could do this before they are all loaded:
<script>
var test_start_time = (new Date()).getTime();
</script>
and this after:
<script>
alert("took: " + (((new Date()).getTime()-test_start_time)/1000) + " seconds");
</script>
Definitely compare and contrast - that will be the most reliable judge.
Nevertheless, I do my best to only load one or two JS files in the head section, then I use jquery to test for certain elements which might require additional scripts or css once the DOM is loaded. For example, I use the source highlight js library to stylize pre tags:
if($('pre').length > 0) {
$.getScript(svx_cdns+'pkgs/shjs-0.6/sh_main.min.js', function() {
$('<link>', {
'rel': 'stylesheet',
'type': 'text/css',
'href': svx_cdns+'pkgs/shjs-0.6/css/sh_vim.min.css'
}).appendTo('head');
sh_highlightDocument('/s/pkgs/shjs-0.6/lang/', '.min.js');
});
}
That way the page loads very quickly, and then "adjusts" afterwards.
You could try to put all of the .js files into one file and then compress it.
This will lower the amount of requests made by the browser by 39 as well :).
Hope this helped.
The impact may be important. Take into account that script downloading blocks page rendering
A couple of things you may try:
Combine as many scripts as you can so you download less files
Minimize and compress combined js files
Try to put as many references as you can at the bottom of the page so they don't block the rendering (this is not easy and must be done carefully, you might end up allowing interaction with some controls before the necessary javascript is downloaded).
Implement paralell download for js files (by default they are downloaded sequentially). Here you have some examples about that
Even if the files are cached, there's still a request to check if the file has been modified. You can change your caching strategy and set your files never to expire. That way the browser will not even ask if it's been modified. That will mean you'll need to add a cache buster to all your urls. Look at firebug's net tab to be sure. I get a 304 Not modified with all my css/js/imgs.
The files are going to have to be parsed everytime, but that's probably not the bottleneck.
Try copying all your js into one file. One of our screen was including over 100 js files. We created a unified minimized file and our screen load time went from 10 seconds to less then 3.

Categories

Resources