To preface I am pretty new to programming Javascript, but I have been working with various libraries for a while now. I've been tasked to get performance metrics for various charting libraries to find the fastest and most flexible based on some of the libraries available (e.g. AmCharts, HighCharts, SyncFusion, etc.). I've tried JSPerf and it seems like I am getting performance metrics for the code execution and not the actual rendered chart which is the metrics we want (aka what the user experience will be). I've tried using the performance.now() within the Javascript code in the header and also wrapped around the tags where the charts are displayed, but neither method is working.
What is the best way to get these performance metrics based on rendering?
Short Answer :
Either :
Start your timing right before the chart code executes and setup a MutationObserver to watch the DOM and end the time when all mutation ends.
Find out if the charting library has a done() event. (But be cautious as this can be inaccurate depending on implementation/library. "done()" could mean visually done, but background work is still being performed. This could cause interactivity to be jumpy until the chart is completely ready).
Long Answer :
I'm assuming your test data is quite large since most libraries can handle a couple thousand points without any negligible degradation. Measuring performance for client-side charting libraries is actually a two sided issue: rendering times and usability.
Rendering times can be measured by the duration when a library interprets the dataset, to the visual representation of the chart. Depending on each library's interpretation algorithm, your mileage will vary depending on the data size. Let's say library X uses an aggressive sampling
algorithm and only has to draw a small percentage of the dataset. Performance will be extremely fast, but it may or may not be an accurate representation of your data set. Even more so, interactivity at a finer grain detail could be limited.
Which leads me to the usability and interactivity aspect of performance. We're using a computer and not a chart on a piece of paper; it should be as interactive as possible.
As the amount of interactivity goes up though, your browser could be susceptible to slowdown depending on the library's implementation. What if each of your million data points was to be an interactive dom node? 1 Million data points would surely crash the browser.
Most of the charting libraries out there deal with the tradeoff between performance, accuracy, and usability differently. As for what is It all depends on the implementation.
Plug/Source : I am a developer at ZingChart and we deal with our customers with large datasets all the time. We also built this which is pretty relevant to your tests : http://www.zingchart.com/demos/zingchart-vs/
My method is really basic. I create a var with current time then call a console.log() with the time I got to the end of my code block and the difference.
var start = +new Date();
//do lots of cool stuff
console.log('Rendered in ' + (new Date() - start) + ' ms');
Very generic and does what it says on the tin. If you want to measure each section of code you would have to make new time slots. Yes, the calculation takes time. But it is miniscule compared to what the code that I want to measure is doing. Example in action at the jsFiddle.
Related
TLDR: Is there data on variation of JS's Date accuracy?
I'm looking into doing some research online, gathering reaction data for experiments.
As a contrived example, let's say a user clicks a button and a new image is displayed on the screen. For the purposes of the question imagine that this takes somewhere between 50 and 100ms
I need to measure the delay between an interaction (e.g. a button click) and the displaying of the new DOM state, ideally to millisecond accuracy.
I've looked into it (including through SO with questions like this) and so far it doesn't really seem like using JS's built-in Dates will really cut it, since a delay in the execution thread can push the time out of sync. This seems a bit odd to me as dates are measured to ms precision, and yet accuracy seems to be much larger.
I'm also aware that there are other latencies associated, such as screen refresh rate. This question is purely about the execution inaccuracies.
My question is this: Is there any data on the error rates/variations etc. of the Date object across browsers/operating systems? Although it would be good to get an idea of the overall variation across systems what I'm really after is the repeat trial variation (doing the same thing on the same system over and over).
I'm looking for a solution that can be delivered entirely using a client-side browser, so no extensions or other executables that a user would need to download.
I want to add objects into some table in IndexedDB in one transaction:
_that.bulkSet = function(data, key) {
var transaction = _db.transaction([_tblName], "readwrite"),
store = transaction.objectStore(_tblName),
ii = 0;
_bulkKWVals.push(data);
_bulkKWKeys.push(key);
if (_bulkKWVals.length == 3000) {
insertNext();
}
function insertNext() {
if (ii < _bulkKWVals.length) {
store.add(_bulkKWVals[ii], _bulkKWKeys[ii]).onsuccess = insertNext;
++ii;
} else {
console.log(_bulkKWVals.length);
}
}
};
Looks like that it works fine, but it is not very optimized way of doing that especially if the number of objects is very high (~50.000-500.000). How could I possibly optimize it? Ideally I want to add first 3000, then remove it from the array, then add another 3000, namely in chunks. Any ideas?
Inserting that many rows consecutively, is not possible to get good performance.
I'm an IndexedDB dev and have real-world experience with IndexedDB at the scale you're talking about (writing hundreds of thousands of rows consecutively). It ain't too pretty.
In my opinion, IDB is not suitable for use when a large amount of data has to be written consecutively. If I were to architect an IndexedDB app that needed lots of data, I would figure out a way to seed it slowly over time.
The issue is writes, and the problem as I see it is that the slowness of writes, combined with their i/o intensive nature, makes gets worse over time. (Reads are always lightening fast in IDB, for what it's worth.)
To start, you'll get savings from re-using transactions. Because of that your first instinct might be to try to cram everything into the same transaction. But from what I've found in Chrome, for example, is that the browser doesn't seem to like long-running writes, perhaps because of some mechanism meant to throttle misbehaving tabs.
I'm not sure what kind of performance you're seeing, but average numbers might fool you depending on the size of your test. The limiting faster is throughput, but if you're trying to insert large amounts of data consecutively pay attention to writes over time specifically.
I happen to be working on a demo with several hundred thousand rows at my disposal, and have stats. With my visualization disabled, running pure dash on IDB, here's what I see right now in Chrome 32 on a single object store with a single non-unique index with an auto-incrementing primary key.
A much, much smaller 27k row dataset, I saw 60-70 entries/second:
* ~30 seconds: 921 entries/second on average (there's always a great burst of inserts at the start), 62/second at the moment I sampled
* ~60 seconds: 389/second average (sustained decreases starting to outweigh effect initial burst) 71/second at moment
* ~1:30: 258/second, 67/second at moment
* ~2:00 (~1/3 done): 188/second on average, 66/second at moment
Some examples with a much smaller dataset show far better performance, but similar characteristics. Ditto much larger datasets - the effects are greatly exaggerated and I've seen as little as <1 entries per second when leaving for multiple hours.
IndexedDB is actually designed to optimize for bulk operations. The problem is that the spec and certain docs does not advertice the way it works. If paying certain attention to the parts in the IndexedDB specification that defines how all the mutating operations in IDBObjectStore works (add(), put(), delete()), you'll find out that it allow callers to call them synchronously and omit listening to the success events but the last one. By omitting doing that (but still listen to onerror), you will get enormous performance gains.
This example using Dexie.js shows the possible bulk speed as it inserts 10,000 rows in 680 ms on my macbook pro (using Opera/Chromium).
Accomplished by the Table.bulkPut() method in the Dexie.js library:
db.objects.bulkPut(arrayOfObjects)
I have some browser-intensive CSS and animation in my webpage and I'd like to determine if the user has a fast PC or not so i can scale things accordingly to provide the best experience.
I am using http://detectmobilebrowser.com's script to detect all mobile devices, and I am going to include the clause /android|ipad|ipod|playbook|silk/i.test(a) to include all tablet devices as well.
However this doesn't and cannot really address the actual hardware. It doesn't go very far at all to paint a picture of what I'm looking for.
An iPhone 4S, for example, will be quite a lot more capable than many of the devices matched by the mobile user agent detector, and this provides no way for it to set itself apart.
Somebody might run Google Chrome on a Pentium II machine (somehow) and want to view my page. (This person probably does not have an iPhone 4S)
Obviously to actually get an idea for this I'll have to do some actual performance testing, and as with performance testing with any kind of application, it makes sense to only test the performance of the type of tasks that the application actually performs.
Even with this in mind I feel like it would be difficult to obtain any reasonably accurate numbers before the performance testing routine will have taken too long and the user will have became impatient. So this probably means go ahead with it unless I want the first initial impression to be perfect. Well, this actually happens to be the case. So I can't get away with measuring performance "after the first run" and adjusting the parameters later.
So what I've got left is to basically try to perform a similar task on initial page load, in a way that is dependent on browser rendering and processing speed, while not presenting anything to the user (so that to the user they still think the page is loading), and then preferably within a second or two obtain accurate enough numbers to set parameters for the actual page to animate and present in a pleasing manner that doesn't resemble a slideshow.
Maybe I could place a full-page white <div> over my test case so that I can prevent the user from seeing what's going on and hope that the browser will not be smart by avoiding doing all the work.
Has anybody ever done this?
I know people are going to say, "you probably don't need to do this", or "there's gotta be a better way" or "reduce the amount of effects".
The reason for doing any of the things I'm doing on the page are so that it looks good. That's the entire point of it. If I didn't care about that as much this question wouldn't exist. The goal is to give the javascript the ability to determine enough parameters to provide an awesome experience on a powerful computer, and also a passable experience on a less capable computer. When more power is available, it should be harnessed. So hopefully that can explain why such suggestions are not valid answers to the question.
I think this is a great question because it puts the user's experience first and foremost.
Several ideas come to mind:
Microsoft has published many tests demonstrating the performance of IE 9 and 10. Many of these tests focus on graphic performance, such as this one, which appears to use this JavaScript file to measure performance. There may be some code/concepts you can use.
A media-intensive page probably takes a few seconds to load anyway, so you have a little breathing room if you begin your tests while the rest of the content loads. For example, initiate AJAX/image requests, run your tests, and then handle the responses.
To test graphic performance, what about using a loading graphic as the performance test? I'm not usually a fan of "loading" screens, but if the site may take a few seconds to load, and the end result is better UX, then it isn't a bad idea.
The white screen idea may work if you draw a bunch of white shapes on it (not sure if any engines are smart enough to optimize this away because it is the same color).
Ultimately, I would err on the side of better performance and lower fidelity, and a less accurate (but fast) test versus making the user wait for too long.
Rather than measuring the user's CPU performance once and determining how many fancy visual effects to use from that, I would measure the amount of time taken by the CPU-intensive bits every time they execute (using new Date()), compare that to expected minimum and maximum values (which you will have to determine), and dynamically adjust the "effect level" up and down as appropriate.
Say if the user starts up a program in the background which eats a lot of CPU time. If you use this idea, your page will automatically tone down the visual effects to save CPU cycles. When the background program finishes, the fancy effects will come back. I don't know if your users will like this effect (but I am sure they will like the fact that their browser stays responsive when the CPU is overloaded).
This is a poor solution but it worked at the time: I used to generate two random matrices of about 100x100, multiply them (the school boy way) 100 times and time it. It took less than 1 second on regular machines and a bit more than 2 seconds in the slowest machine I could find (EeePC 1000H). After that I could say "well, this CPU can do X floating point operations per second", which is very inaccurate and probably wrong but the results were very stable and with very low standard deviations, so I guess you can call it a poor measure of javascript mathematical performance, which can tell you something about the CPU of that computer.
You can also check if it's WebGL enabled, and it will leave out all windows operating systems older than Vista. Since those don't have the hardware to run Vista or greater, those are slower PCs. You can check it with this:
function hasWebGL () {
if (typeof window.WebGLRenderingContext !== 'undefined') {
var canvas = document.createElement('canvas');
var gl = canvas.getContext('webgl') || canvas.getContext('experimental-webgl') || canvas.getContext('webkit-3d') || canvas.getContext('moz-webgl');
if(gl) {
return(true);
} else {
return(false);
}
}
}
I'm currently writing an application that displays a lot, and I mean, a lot of 2D paths (made of hundreds, thousands of tiny segments) on an HTML5 canvas. Typically, a few million points. These points are downloaded from a server into a binary ArrayBuffer.
I probably won't be using that many points in the real world, but I'm kinda interested in how I could improve the performance. You can call it curiosity if you want ;)
Anyway, I've tested the following solutions :
Using gl.LINES or gl.LINE_STRIP with WebGL, and compute everything in shaders on the GPU. Currently the fastest, can display up to 10M segments without flinching on my Macbook Air. But there are very strict constraints for the binary format if you want to avoid processing things in JavaScript, which is slow.
Using Canvas2D, draw a huge path with all the segments in one stroke() call. When I'm getting past 100k points, the page freezes for a few seconds before the canvas is updated. So, not working here.
Using Canvas2D, but draw each path with its own stroke() call. Despite what others have been saying on the internet, this is much faster than drawing everything in one call, but still a lot slower than WebGL. Things start to get bad when I reach about 500k segments.
The two Canvas2D solutions require looping through all the points of all the paths in JavaScript, so this is quite slow. Do you know of any method(s) that could improve JavaScript's iteration speed in an ArrayBuffer, or processing speed in general?
But, what's strange is, the screen isn't updated immediately after all the canvas draw calls have finished. When I start getting to the performance limit, there is a noticeable delay between the end of the draw calls and the update of the canvas. Do you have any idea where that comes from, and is there a way to reduce it?
First, WebGL was a nice and hype idea, but the amount of processing required to decode and display the binary data simply doesn't work in shaders, so I ruled it out.
Here are the main bottlenecks I've encountered. Some of them are quite common in general programming, but it's a good idea to remember them :
It's best to use multiple, small for loops
Create variables and closures at the highest level possible, don't create them inside the for loops
Render your data in chunks, and use setTimeout to schedule the next chunk after a few milliseconds : that way, the user will still be able to use the UI
JavaScript objects and arrays are fast and cheap, use them. It's best to read/write them in sequential order, from the beginning to the end.
If you don't write data sequentially in an array, use objects (because non-sequential read-writes are cheap for objects) and push the indexes into an index array. I used a SortedList implementation to keep the indexes sorted, which I found here. Overhead was minimal (about 10-20% of the rendering time), and in the end it was well worth it.
That's about everything I remember. If I do find something else, I'll update this answer!
I found some interesting post about memory usage and CPU usage here on Stack Overflow, however none of them had a direct approach to the apparently simple question:
As a generic strategy in a JavaScript app, is it better in terms of performances to use memory (storing data) or CPU (recalculating data each time)?
I refer to javascript usage in common browsers environment (FF, Chrome, IE>8)
Does anybody have a more direct and documented answer to this?
--- EDIT ---
Ok, I understand the question is very generic. I try to reduce the "scope".
Reading your answer I realized that the real question is: "how to undestand the memory limit under which my javascript code still has good performances?".
Environment: common browsers environment (FF, Chrome, IE>8)
Functions I use are not very complex math functions, but can produce quite a huge amount of data (300-400kb) and I wanted to understand if it was better to recalculate them every time or just store results in variables.
Vaguely related - JS in browsers is extremely memory hungry when you start using large objects / arrays. If you think about binary data produced by canvas elements, or other rich media APIs, then clearly you do not want to be storing this data in traditional ways - disregarding performance issues, which are also important.
From the MDN article talking about JS Typed Arrays:
As web applications become more and more powerful, adding features such as audio and video manipulation, access to raw data using WebSockets, and so forth, it has become clear that there are times when it would be helpful for JavaScript code to be able to quickly and easily manipulate raw binary data.
Here's a JS Perf comparison of arrays, and another looking at canvas in particular, so you can get some direct examples on how they work. Hope this is useful.
It's just another variation on the size/performance tradeoff. Storing values increases size, recalculating decreases performance.
In some cases, the calculation is complex, and the memory usage is small. This is particularly true for maths functions.
In other cases, the memory needed would be huge, and calculations are simple. This is particularly true when the output would be a large data structure, and you can calculate an element in the structure easily.
Other factors you will need to take into account is what resources are available. If you have very limited memory then you may have no choice, and if it is a background process then perhaps using lots of memory is not desirable. If the calculation needs to be done very often then you are more likely to store the value than if it's done once a month...
There are a lot of factors in the tradeoff, and so there is no "generic" answer, only a set of guidelines you can follow as each case arises.