I have a large 40,000 words array loading from a database into a JavaScript/HTML array on every page of our web application... What would be the best way/technology to optimize it? In order to avoid this unnecessary downloads.
Somehow keep the array in a cookie and read from there?
Use ajax to load the array dynamically only parts that are needed?
What is the common practice?
On modern browsers you can use sessionStorage to have it persist during the current session, or localStorage to have it hang around between sessions.
NB: both only permit storage of strings - you'll have to serialise the array (e.g. into JSON) and deserialise it on retrieval.
If you want to actually use the word list as a local database with efficient lookup you might also want to investigate indexedDB
you can place the data in session and retrieve it, the same can be used in every page with out fetching the same every time.
Thanks & best regards.
If you need all the 40k words in all pages then you can use localStorage or sessionStorage. Just keep in mind sessionStorage will delete saved data when the tab/window is closed so the whole array will be downloaded again when the website is opened in new windows/tabs.
If you need only specific parts of the array in different pages I would tidy the array's elements into taxonomy/categories (if you are able to), so that you can download only the needed for a specific section of your application.
This depends on the composition of your array, if it is formed only by words or complex objects. This will help to avoid slow load of your website when it's visited the first time.
If the array is always the same (there is no need to update it), I'd create a js file and then I'd add it to every html page. The browser's cache would do the rest to avoid unnecessary re-loading. Something like:
big-array.js file:
var myBigArray=[...]
In each html file
<html>
... whatever you need
<script src="/my-path/big-array.js"></script>
...my other scripts here
</html>
It's a bit difficult to answer this question properly as to do so would require more information about your hosting environment and what you have access to. If you have a server side language available, such as PHP, you could look at caching which is generally the most efficient way to handle data that is used repeatedly across pages. Perhaps you could post more info about what technologies you have available to you?
Related
My scenario is this: I have a very small array in my js file. When the page loads up, I have a function that loops through the array, and generates an li element for each item in the array, displaying it's name and price in the li. The array is constructed like this:
var gameList = [
{ name: "", value: 0.00},
]
Secondly, I have a simple form on the page that allows me to add new items to the array, and using localStorage, it's possible for me to keep a dynamically updated array. I push new items into the array (gameList), then at the end of the session I set it using localStorage.
localStorage.setItem("updatedGameList", JSON.stringify(gameList));
I have a couple of lines at the start of my code that sets my original array 'gameList' to be equal to the locally stored, updated game list.
var retrievedData = localStorage.getItem("updatedGameList");
gameList = JSON.parse(retrievedData);
So this is fine for now, but the growing array - which I want to keep and maintain - is only available in this browser, on this machine.
So, my question is, can I send this locally stored data somewhere? Maybe my personal domain? (Which is where I will host the app when it's finished) That way I could then reference it properly in my js file so that the data is always available? Maybe the array could have it's own js file?
I realise that this may not be the best way to be handling what is essentially a database. But I'm only part way through an online course and I'm using the tools that I have to make this work.
And lastly, in terms of maintenance of the array, is there any way to send it back to sublime in the form a .js file? I know this could be a crazy question. The updated array will become pretty big, maybe 200 items eventually, and it would be much easier to maintain from within sublime.
Thanks for your time, and apologies if part of this request is ridiculous!! :)
I have just been reading about AJAX, and thought maybe there's a way to send the updated array as a json file to somewhere(!) on my website, and then request that same file at the start of each new session, so I'm always working with, and saving, the latest updated array.
Thanks for reading, and hopefully you have some answers! :)
Although not quite what I was looking for - essentially some way of automatically getting the new array, sending it somewhere more secure than local storage, then referencing the new array to give me the most up to date starting point each time (and all with just javascript) - the 'dirty' way suggested below turned out to be sufficient for now until I start using databases.
From Kirupa, over at the forums:
Not a ridiculous question at all! You can send your own data anywhere you want, but it will require some level of server-related code. The easiest way to send data back and forth is through JSON, and you can convert your array into a JSON format easily using something like the following:
var jsonData = JSON.stringify(myArray);
From here, you can send this data to a database, to another web site, or to your e-mail server. If you want something really quick and dirty, you can literally just copy the contents of your JSON-ized array using the Chrome Dev Tools, save it on disk as a .js file, and reference it again in your app. That is a manual way of doing something that you don't really care about automating.
The best solution is to store this in a database. They've gotten easier to deal with as well. Firebase is my go-to for things like this, and this video might give you some ideas: https://www.youtube.com/watch?v=xAsvwy1-oxE
I've got a web app which gets a couple dozen items at boot. All these items are JSON and are smaller then 1kb.
Now there are a number of storage options as seen in the Question.
I was thinking of just storing these objects inside a variable in the browser JS. I don't really see why I would want to use any of these browser storages?
So what would be reasons to use any of the browser-based storage instead of a variable inside JS.
Could be that from a certain data size it is preferable to use browser storage, e.g. from 100kb onwards it's better to not use a JS variable.
var myModel = {}
NOTE
Every time the user enters the app he will get fresh content from the server. The content is too realtime for caching.
`
localStorage , globalStorage and sessionStorage:
These features are ready in browsers that have implemented the "Web Storage", they all refer to a kind of HashMap, a map between string keys and string values. but the life is different. once the active page is closed sessionStorage would be cleaned but the localStorage is permanent.(MDN DOM Storage guide)
There is a point about globalStorage, which is its being obsolete since Gecko 1.9.1 (Firefox 3.5) and unsupported since Gecko 13 (Firefox 13), since then we should use localStorage. the difference between these 2 was just the HTML5 scope support(scheme + hostname + non-standard port).
These could be useful for you to:
-Share your objects between your different pages, in your site.
-Offline programming.
-Caching large object
-Or whenever you need to a local persistent storage.
IndexedDB:
IndexedDB is useful for applications that store a large amount of data (for example, a catalog of DVDs in a lending library) and applications that don't need persistent internet connectivity to work (for example, mail clients, to-do lists, and notepads)
based on this quote from MDN you can easily find your answer out, regarding using IndexedDB, if you don't know whether IndexedDB is useful for you or not, just answer these questions:
Do you store a large amount of data on client? if yes, so consider using it.
Does your app need to be offline enabled? if yes, so consider using IndexedDB.
Does your app need to a persistent internet connectivity? If yes, it stays still an option, based on the other factors.
So other than working offline as far as you don't need it, I guess, because as you said:
The content is too realtime for caching.
These have some features like sharing objects, and managing large amount of data, which you should be the one to decide.
localStorage and sessionStorage are solving a caching problem; think of them as cookies. You've said you don't want caching, so you can ignore them.
JavaScript objects behave basically like O(1) lookup tables (see How is a JavaScript hash map implemented?, and make sure you read both the top two answers, as both have something useful to say), and there is no maximum memory limit that I am aware of, or a point where another solution becomes a better choice
The only reason I can think of that you should bother with the extra step of inserting the data in an IndexedDB is if you need O(1) lookups on a field that is not the object key you are using.
I have javascript that is being generated at design time that needs to be executed to find a value at run time. The code is stored as a string in an object and I would like to be able to execute it and retrieve the value and then scrap the code. Is there a way to do this? Do I have to use eval()?
You can use eval(String)
Or use new Function (String)
Or use document.createElement
[edited]
Depend on how it was done your code
1 -
if those strings are saved in shared across different pages (with cookies or database), then SERVER-SIDE you can generate a tag <script> with the values saved in a JSON for quick access.
2 -
If the strings are saved only at runtime (ie in pagination are not recoverable values) you may not need to save these values in Strings, talves you can create a global Json in Window Object (eg. window.MyObjectGlobal), making the values accessible at any time on the page (since there is no paging) - is idea can also be reused in case of using the SERVER-SIDE, combined with Ajax (ajax only to save the data in your database), or document.cookie (but will have to use document.createElement("script") or eval)
Good luck
Yes, you can do that using eval.
However, remember evalis evil and it could potentially introduce security risks.
Anyway, if you know what you're doing, that's the way to go
I have a list of all articles in NY Times from its beginning and want have an instant access to all of them without connecting to external database, so my solution is holding it in one variable. But isn't that a bad practice in terms of efficiency?
I doubt that the memory usage is any problem - modern browser games for example probably use an order of magnitude more memory.
I would worry more about the data structure and the operations you intend to run on it.
What kind of application are you writing?
It would properbly be a better idea to create a local database and store the articles there and then create a sync task that pulls only the articles that are of a later date than the previous sync task and save them to the local db.
It all depends on what kind of application you are writing of course
I think you need to look into using HTML5 Local Storage to store this variable or it's contents. This way you're keeping things client-side as you wish whilst also storing the data properly.
LocalStorage doesn't work here. I am looking for more of a theory type answer and not as much code. I already know how to set and delete cookies, that is now what this question is about; here is the question:
When I submit an order, I want to place
Meal
Ingredients
Name
Phone
inside cookies to be later outputted on a div to the right of the page. This I think I can do quite easily. I might put each value into an object of orders...
But that isn't the real question, how can I have multiple orders that are unique? I want to have many different orders and have the user delete the order they desire. I was thinking of separating each order with a | character and than playing some string games. But I don't know how I would delete one.
My other idea was have a order id and auto-increment it. Any help? website: philipimperato.com/mobileOrder
P.S. Only Javascript and I know how to setCookie and deleteCookie :D
Cookies don't seem the place do to this anymore. Cookies are limited and are sent with each HTTP request, including all of your images and static files unless they are on a different domain. I recommend using localStorage instead. Since this is intended for smartphones like the iPhone and Android you are ok to use localStorage. Webkit browsers have supported it for a long time. If you use localStorage you can use any kind of key value storage mechanism you like. I recommend the redis way of field:id:property for keys.
var order_id = 10203;
var key = 'order:' + order_id + ':drink';
localStorage[key] = 'Pepsi';
By using the order_id in your key field you can easily manage unique orders.
You could serialize an order object array in json and parse it back as you load
(This could present security issues, and maybe you should use a framework to parse json back to life. Many frameworks do some lint on json before evaluating it, some even parse it all by themselves)