Can browsers enforce any sort of limit on the amount of data that can be stored in JavaScript objects? If so, is there any way to detect that limit?
It appears that by default, Firefox does not:
var data;
$("document").ready(function() {
data = [];
for(var i = 0; i < 100000000000; i++) {
data.push(Math.random());
}
});
That continues to consume more and more memory until my system runs out.
Since we can't detect available memory, is there any other way to tell we are getting close to that limit?
Update
The application I'm developing relies on very fast response times to be usable (it's the core selling point). Unfortunately, it also has a very large data set (more than will fit into memory on weaker client machines). Performance can be greatly improved by preemptively loading data strategically (guessing what will be clicked). The fallback to loading the data from the server works when the guesses are incorrect, but the server round trip isn't ideal. Making use of every bit of memory I can makes the application as performant as possible.
Right now, it works to allow the user to "configure" their performance settings (max data settings), but users don't want to manage that. Also, since it's a web application, I have to handle users setting that per computer (since a powerful desktop has a lot more memory than an old iPhone). It's better if it just uses optimal settings for what is available on the systems. But guessing too high can cause problems on the client computer too.
While it might be possible on some browsers, the right approach should be to decide what limit is acceptable for the typical customer and optionally provide a UI to define their limit.
Most heavy web apps get away with about 10MB JavaScript heap size. There does not seem to be a guideline. But I would imagine consuming more than 100MB on desktop and 20MB on mobile is not really nice. For everything after that look into local storage, e.g. FileSystem API (and you can totally make it PERSISTENT)
UPDATE
The reasoning behind this answer is the following. It is next to never user runs only one application. More so with counting on the browser having only one tab open. Ultimately, consuming all available memory is never a good option. Hence determining the upper boundary is not necessary.
Reasonable amount of memory user would like to allocate to the web app is a guess work. E.g. highly interactive data analytics tool is quite possible in JS and might need millions of data points. One option is to default to less resolution (say, daily instead of each second measurements) or smaller window (one day vs. a decade of seconds). But as user keeps exploring the data set, more and more data will be needed, potentially crippling the underlying OS on the agent side.
Good solution is to go with some reasonable initial assumption. Let's open some popular web applications and go to dev tools - profiles - heap snapshots to take a look:
FB: 18.2 MB
GMail: 33 MB
Google+: 53.4 MB
YouTube: 54 MB
Bing Maps: 55 MB
Note: these numbers include DOM nodes and JS Objects on the heap.
It seems to be then, people come to accept 50MB of RAM for a useful web site. (Update 2022: nowadays averaging closer to 100MB.) Once you build your DOM Tree, fill your data structures with test data and see how much is OK to keep in RAM.
Using similar measurements while turning device emulation in Chrome, one can see the consumption of the same sites on tablets and phones, BTW.
This is how I arrived at 100 MB on desktop and 20 MB on mobile numbers. Seemed to be reasonable too. Of course, for occasional heavy user it would be nice to have an option to bump max heap up to 2 GB.
Now, what do you do if pumping all this data from the server every time is too costly?
One thing is to use Application Cache. It does create mild version management headaches but allows you to store around 5 MB of data. Rather than storing data though, it is more useful to keep app code and resources in it.
Beyond that we have three choices:
SQLite - support was limited and it seems to be abandoned
IndexDB - better option but support is not universal yet (can I use it?)
FileSystem API
Of them, FileSystem is most supported and can use sizable chunk of storage.
In Chrome the answer is Sure!
Go to the console and type:
performance.memory.jsHeapSizeLimit; // will give you the JS heap size
performance.memory.usedJSHeapSize; // how much you're currently using
arr = []; for(var i = 0; i < 100000; i++) arr.push(i);
performance.memory.usedJSHeapSize; // likely a larger number now
I think you'll want something like the following:
const memory = navigator.deviceMemory
console.log (`This device has at least ${memory}GiB of RAM.`)
You can check out the following: https://developer.mozilla.org/en-US/docs/Web/API/Navigator/deviceMemory
Note: This feature is not supported across all browsers.
Since a web app can't have access to any system-related information (like the available amount of memory), and since you would prefer not having to ask users to manually set their performance settings, you must rely on a solution that allows you to get such information about the user's system (available memory) without asking them. Seems impossible ? Well, almost...
But I suggest you do the following : make a Java applet that will automatically get the available memory size (e.g. using Runtime.exec(...) with an appropriate command), provided your applet is signed, and return that information to the server or directly to the web page (with JSObject, see http://docs.oracle.com/javafx/2/api/netscape/javascript/JSObject.html).
However, that would assume your users can all run a Java applet within their browsers, which is not always the case. Therefore, you could ask them to install a small piece of software on their machines that will measure how much memory your app should use without crashing the browser, and will send that information to your server. Of course, you would have to re-write that little program for every OS and architecture (Windows, Mac, Linux, iPhone, Android...), but it's simpler that having to re-write the whole application in order to gain some performance. It's a sort of in-between solution.
I don't think there is an easy solution. There will be some drawbacks, whatever you choose to do. Remember that web applications don't have the reputation of being fast, so if performance is critical, you should consider writing a traditional desktop application.
Related
I'd like to be able to find out about a browser's hardware resources from a web page, or at least a rough estimation.
Even when you detect the presence of modern technology (such as csstransforms3d, csstransitions, requestAnimationFrame) in a browser via a tool like Modernizr, you cannot be sure whether to activate some performance-consuming option (such as fancy 3D animation) or to avoid it.
I'm asking because I have (a lot of) experience with situations where the browser is modern (latest Chrome or Firefox supporting all cool technologies) but OS's CPU, GPU, and available memory are just catastrophic (32bit Windows XP with integrated GPU) and thus a decision based purely on detected browser caps is no good.
While Nickolay gave a very good and extensive explanation, I'd like to suggest one very simple, but possibly effective solution - you could try measuring how long it took for the page to load and decide whether to go with the resource-hungry features or not (Gmail does something similar - if the loading goes on for too long, a suggestion to switch to the "basic HTML" version will show up).
The idea is that, for slow computers, loading any page, regardless of content, should be, on average, much slower than on modern computers. Getting the amount of time it took to load your page should be simple, but there are a couple of things to note:
You need to experiment a bit to determine where to put the "too slow" threshold.
You need to keep in mind that slow connections can cause the page to load slower, but this will probably make a difference in a very small number of cases (using DOM ready instead of the load event can also help here).
In addition, the first time a user loads your site will probably be much slower, due to caching. One simple solution for this is to keep your result in a cookie or local storage and only take loading time into account when the user visits for the first time.
Don't forget to always, no matter what detection mechanism you used and how accurate it is, allow the user to choose between the regular, resource-hungry and the faster, "uglier" version - some people prefer better looking effects even if it means the website will be slower, while other value speed and snappiness more.
In general, the available (to web pages) information about the user's system is very limited.
I remember a discussion of adding one such API to the web platform (navigator.hardwareConcurrency - the number of available cores), where the opponents of the feature explained the reasons against it, in particular:
The number of cores available to your app depends on other workload, not just on the available hardware. It's not constant, and the user might not be willing to let your app use all (or whatever fixed portion you choose) of the available hardware resources;
Helps "fingerprinting" the client.
Too oriented on the specifics of today. The web is designed to work on many devices, some of which do not even exist today.
These arguments work as well for other APIs for querying the specific hardware resources. What specifically would you like to check to see if the user's system can afford running a "fancy 3D animation"?
As a user I'd rather you didn't use additional resources (such as fancy 3D animation) if it's not necessary for the core function of your site/app. It's sad really that I have to buy a new laptop every few years just to be able to continue with my current workflow without running very slowly due to lack of HW resources.
That said, here's what you can do:
Provide a fallback link for the users who are having trouble with the "full" version of the site.
If this is important enough to you, you could first run short benchmarks to check the performance and fall back to the less resource-hungry version of the site if you suspect that a system is short on resources.
You could target the specific high-end platforms by checking the OS, screen size, etc.
This article mentions this method on mobile: http://blog.scottlogic.com/2014/12/12/html5-android-optimisation.html
WebGL provides some information about the renderer via webgl.getParameter(). See this page for example: http://analyticalgraphicsinc.github.io/webglreport/
JavaScript in Chrome (or any other browser for that matter, but I rather limit the discussion to Chrome to make it simpler) does not provide an API which can be used to observe memory related information (e.g. how much memory is being used by the current tab where the JS is running).
I am looking for a creative solution for getting an estimation of how much bytes I can cache in a JavaScript object that my web page is running. The problem definition is that I would like to cache as much as possible.
Can anyone think of a decent way of estimating how much memory can a tab handle before it will crash / become unusable on a machine? I guess a statistical approach could work out fine for some cases, but I'm looking for something more dynamic.
I'm considering writing a game in JavaScript using WebGL and associated technologies. I would like to make the game as intelligent as possible, so I'm looking into monitoring CPU/memory usage.
For example:
For high CPU usage, scale back the graphics a bit or offload computations to the server
For high memory usage, offload data to the server for storage (and later retrieval)
I would like to get the data that Chrome offers in it's Task Manager. I know how to track FPS, and that can lead to some flexibility, but I would like to be have as much information as possible. The main use case is for a 'low power' mode where the CPU is utilized as little as possible (for laptops) or an idle mode when the user is browsing forums, etc.
I know how to use profilers, but I would like access to these tools from JavaScript.
Is this possible? If not, do you know if it has been proposed for standardization?
I would be willing to live with an extension, as long as it could be queried from JavaScript, but I'd like to avoid it if a native feature exists. I'm trying to target recent versions of Firefox and Chrome, but I could restrict myself to a single browser if one supports this.
Well there is no direct javascript call to get such information (which would have been a serious security problem). But there's a solution for your problem, you can use worker pools which are litterally threads for javascript to emulate a function that will run some calculations in the background to compute the CPU usage.
But since you're building a 3D application I will not advise to do this because it will unnecessarily cost you a lot of CPU usage to know the current level of CPU usage, which would be like killing a fly with a submachine gun.
What I advise you to do however is to only focus on frame per seconds because they are related to your application and are the exact benchmarking indication you need. Don't care about cpu load, your application doesn't directly depend on that, especially if you got a dual-core or quad-core processor. You should perhaps also look at GPU usage for your application and how you can fully take benefit of the GPU on a compatible browser (latest Chromes uses GPU acceleration).
Good luck with your game !
We can't retrieve CPU usage or RAM from client-side Javascript, but what matters is the refresh rate, the actual number of frames refreshed by seconds.
If the FPS is over 24 and steady, we simply feels no lags. Even safer over 30FPS, to keep a margin. This leave about 40ms for a frame refresh.
Simply, the following code calculate a frame time refresh, using requestAnimationFrame, convert it to an amount by second, and POST it to the server in JSON at the endpoint /usermetrics, using navigator.sendBeacon()
let t = Date.now();
requestAnimationFrame( () => {
let fps = Math.round(1000 / (Date.now() - t));
console.log(fps + "FPS");
navigator.sendBeacon('/usermetrics', JSON.stringify(fps))
})
From the console we can observe the POST beacon.
You might have to use it strategically, depending the context of your app, basically reduce the load if the FPS goes under 30.
The Performance API
Another example, looping FPS counter
I'm toying with the idea of progressively enabling/disabling JavaScript (and CSS) effects on a page - depending on how fast/slow the browser seems to be.
I'm specifically thinking about low-powered mobile devices and old desktop computers -- not just IE6 :-)
Are there any examples of this sort of thing being done?
What would be the best ways to measure this - accounting for things, like temporary slowdowns on busy CPUs?
Notes:
I'm not interested in browser/OS detection.
At the moment, I'm not interested in bandwidth measurements - only browser/cpu performance.
Things that might be interesting to measure:
Base JavaScript
DOM manipulation
DOM/CSS rendering
I'd like to do this in a way that affects the page's render-speed as little as possible.
BTW: In order to not confuse/irritate users with inconsistent behavior - this would, of course, require on-screen notifications to allow users to opt in/out of this whole performance-tuning process.
[Update: there's a related question that I missed: Disable JavaScript function based on user's computer's performance. Thanks Andrioid!]
Not to be a killjoy here, but this is not a feat that is currently possible in any meaningful way in my opinion.
There are several reasons for this, the main ones being:
Whatever measurement you do, if it is to have any meaning, will have to test the maximum potential of the browser/cpu, which you cannot do and maintain any kind of reasonable user experience
Even if you could, it would be a meaningless snapshot since you have no idea what kind of load the cpu is under from other applications than the browser while your test is running, and weather or not that situation will continue while the user is visiting your website.
Even if you could do that, every browser has their own strengths and weaknesses, which means, you'd have to test every dom manipulation function to know how fast the browser would complete it, there is no "general" or "average" that makes sense here in my experience, and even if there was, the speed with which dom manipulation commands execute, is based on the context of what is currently in the dom, which changes when you manipulate it.
The best you can do is to either
Let your users decide what they want, and enable them to easily change that decision if they regret it
or better yet
Choose to give them something that you can be reasonably sure that the greater part of your target audience will be able to enjoy.
Slightly off topic, but following this train of thought: if your users are not techleaders in their social circles (like most users in here are, but most people in the world are not) don't give them too much choice, ie. any choice that is not absolutely nescessary - they don't want it and they don't understand the technical consequences of their decision before it is too late.
A different approach, that does not need explicit benchmark, would be to progressively enable features.
You could apply features in prioritized order, and after each one, drop the rest if a certain amount of time has passed.
Ensuring that the most expensive features come last, you would present the user with a somewhat appropriate selection of features based on how speedy the browser is.
You could try timing some basic operations - have a look at Steve Souder's Episodes and Yahoo's boomerang for good ways of timing stuff browserside. However its going to be rather complicated to work out how the metrics relate to an acceptable level of performance / a rewarding user experience.
If you're going to provide a UI to let users opt in / opt out, why not just let the user choose the level of eye candy in the app vs the rendering speed?
Take a look at some of Google's (copyrighted!) benchmarks for V8:
http://v8.googlecode.com/svn/data/benchmarks/v4/regexp.js
http://v8.googlecode.com/svn/data/benchmarks/v4/splay.js
I chose a couple of the simpler ones to give an idea of similar benchmarks you could create yourself to test feature sets. As long as you keep the run-time of your tests between time logged at start to time logged at completion to less than 100 ms on the slowest systems (which these Google tests are vastly greater than) you should get the information you need without being detrimental to user experience. While the Google benchmarks care at a granularity between the faster systems, you don't. All that you need to know is which systems take longer than XX ms to complete.
Things you could test are regular expression operations (similar to the above), string concatenation, page scrolling, anything that causes a browser repaint or reflow, etc.
You could run all the benchmarks you want using Web Workers, then, according to results, store your detection about the performance of the machine in a cookie.
This is only for HTML5 Supported browsers, of-course
Some Ideas:
Putting a time-limit on the tests seems like an obvious choice.
Storing test results in a cookie also seems obvious.
Poor test performance on a test could pause further scripts
and trigger display of a non-blocking prompt UI (like the save password prompts common in modern web browsers)
that asks the user if they want to opt into further scripting effects - and store the answer in a cookie.
while the user hasn't answered the prompt, then periodically repeat the tests and auto-accept the scripting prompt if consecutive tests finish faster than the first one.
.
On a sidenote - Slow network speeds could also probably be tested
by timing the download of external resources (like the pages own CSS or JavaScript files)
and comparing that result with the JavaScript benchmark results.
this may be useful on sites relying on loads of XHR effects and/or heavy use of <img/>s.
.
It seems that DOM rendering/manipulation benchmarks are difficult to perform before the page has started to render - and are thus likely to cause quite noticable delays for all users.
I came with a similar question and I solved it this way, in fact it helped me taking some decisions.
After rendering the page I do:
let now, finishTime, i = 0;
now = Date.now();//Returns the number of miliseconds after Jan 01 1970
finishTime = now + 200; //We add 200ms (1/5 of a second)
while(now < finishTime){
i++;
now = Date.now();
}
console.log("I looped " + i + " times!!!");
After doing that I tested it on several browser with different OS and the i value gave me the following results:
Windows 10 - 8GB RAM:
1,500,000 aprox on Chrome
301,327 aprox on Internet Explorer
141,280 (on Firefox on a VirtualMachine running Lubuntu 2GB given)
MacOS 8GB RAM:
3,000,000 aprox on Safari
1,500,000 aprox on Firefox
70,000 (on Firefox 41 on a Virtual Machine running Windows XP 2GB given)
Windows 10 - 4GB RAM (This is an Old computer I have)
500,000 aprox on Google Chrome
I load a lot of divs in a form of list, the are loaded dinamically accordeing to user's input, this helped me to limit the number of elements I create according to the performance the have given, BUT
But the JS is not all!, because even tough the Lubuntu OS running on a virtual machine gave poor results, it loaded 20,000 div elements in less than 2 seconds and you could scroll through the list with no problem while I took more than 12 seconds for IE and the performance sucked!
So a Good way could be that, but When it comes to rendering, thats another story, but this definitely could help to take some decisions.
Good luck, everyone!
We all know how web browsers (such as Firefox) are certain to fill up memory consumption because we continuously execute JavaScript code (from websites) that is prone to memory leakage.
I am debating in developing a Desktop app, and given my experience with Javascript/Css/HTML, I thought I would give AIR a try, this way I don't have to use Java (for example) and deal with learning all its GUI swing stuff.
The problem is that I worry about memory leakage in AIR, since AIR is simply a web browser with an API layer to interact with the Operating System.
Is it plausible to worry about memory leakage in AIR? What should I do about it?
My name is Rob Christensen and I am product manager on Adobe AIR. First, let me say that it is quite easy to build a desktop application, regardless of underlying technology, that consumes a large amount of memory and/or does not free up memory.
In the next release of AIR, we are looking at providing some additional capabilities to the AIR runtime to make it easier to identify memory leaks for JavaScript-based applications. Developers that are building Flash or Flex based applications can already take advantage of the memory profiler included in Flex Builder to track this down. We are hoping to do something similar for JavaScript developers as well.
In my experience talking to developers, memory leaks often occur when objects in memory are never cleaned up. For example, imagine a Twitter client that lists tweets from users based around a search keyword. Overtime, more results show and the list becomes longer. If there is not a limit on the maximum number of Tweets visible, memory will, of course, go up over time. Instead, the application should impose a reasonable limit on the number of items that appear in that list.
There are some talks available that describe best practices around handling memory in AIR. Though the examples in this article are mostly written in ActionScript, the same concepts apply to JavaScript as well.
Performance-Tuning AIR applications
http://www.adobe.com/devnet/air/articles/air_performance.html
If there are memory leaks in the runtime, we jump on these as quickly as we can. We encourage developers to know about such issues by sending them back to our team using the following feedback form (www.adobe.com/go/wish).
If you are using an Ajax framework, you may want to look into whether there are known issues with memory leaks for that particular framework.
So, to summarize, yes, you should always worry about memory when building a desktop application -- whether with AIR or C++. As you are developing your application, you should monitor the memory usage of your application so that you can identify any issues sooner than later. One way to do this is to run longevity tests -- keep your application open over night to see if memory is creeping up.
In general, the tools available for browsers are very limited as well. I expect this will change soon as browser vendors also start providing more hooks into their browsers for identifying memory usage. Hope this helps.
Thank you!
-Rob
Product Manager, Adobe AIR
Sure. I've seen AIR apps on Linux swallow gigabytes of memory over time. It's a real blocker for me and stops me using them.
That said, other people on other platforms have no issue with it. Ultimately you need to decide what most of your market will be using and how affected they'll be by any issues in AIR (or other).
If it's not that important (but it's still an issue) submit bug reports and hope Adobe fix things.