Preface
This issue seems to only affect Chrome/V8, and may not be reproducible in Firefox or other browsers. In summary, the execution time of a function callback increases by an order of magnitude or more if the function is called with a new callback anywhere else.
Simplified Proof-of-Concept
Calling test(callback) arbitrarily many times works as expected, but once you call test(differentCallback), the execution time of the test function increases dramatically no matter what callback is provided (i.e., another call to test(callback) would suffer as well).
This example was updated to use arguments so as to not be optimized to an empty loop. Callback arguments a and b are summed and added to total, which is logged.
function test(callback) {
let start = performance.now(),
total = 0;
// add callback result to total
for (let i = 0; i < 1e6; i++)
total += callback(i, i + 1);
console.log(`took ${(performance.now() - start).toFixed(2)}ms | total: ${total}`);
}
let callback1 = (a, b) => a + b,
callback2 = (a, b) => a + b;
console.log('FIRST CALLBACK: FASTER');
for (let i = 1; i < 10; i++)
test(callback1);
console.log('\nNEW CALLBACK: SLOWER');
for (let i = 1; i < 10; i++)
test(callback2);
Original post
I am developing a StateMachine class (source) for a library I'm writing and the logic works as expected, but in profiling it, I've run into an issue. I noticed that when I ran the profiling snippet (in global scope), it would only take about 8ms to finish, but if I ran it a second time, it would take up to 50ms and eventually balloon as high as 400ms. Typically, running the same named function over and over will cause its execution time to drop as the V8 engine optimizes it, but the opposite seems to be happening here.
I've been able to get rid of the problem by wrapping it in a closure, but then I noticed another weird side effect: Calling a different function that relies on the StateMachine class would break the performance for all code depending on the class.
The class is pretty simple - you give it an initial state in the constructor or init, and you can update the state with the update method, which you pass a callback that accepts this.state as an argument (and usually modifies it). transition is a method that is used to update the state until the transitionCondition is no longer met.
Two test functions are provided: red and blue, which are identical, and each will generate a StateMachine with an initial state of { test: 0 } and use the transition method to update the state while state.test < 1e6. The end state is { test: 1000000 }.
You can trigger the profile by clicking the red or blue button, which will run StateMachine.transition 50 times and log the average time the call took to complete. If you click the red or blue button repeatedly, you will see that it clocks in at less than 10ms without issue - but, once you click the other button and call the other version of the same function, everything breaks, and the execution time for both functions will increase by about an order of magnitude.
// two identical functions, red() and blue()
function red() {
let start = performance.now(),
stateMachine = new StateMachine({
test: 0
});
stateMachine.transition(
state => state.test++,
state => state.test < 1e6
);
if (stateMachine.state.test !== 1e6) throw 'ASSERT ERROR!';
else return performance.now() - start;
}
function blue() {
let start = performance.now(),
stateMachine = new StateMachine({
test: 0
});
stateMachine.transition(
state => state.test++,
state => state.test < 1e6
);
if (stateMachine.state.test !== 1e6) throw 'ASSERT ERROR!';
else return performance.now() - start;
}
// display execution time
const display = (time) => document.getElementById('results').textContent = `Avg: ${time.toFixed(2)}ms`;
// handy dandy Array.avg()
Array.prototype.avg = function() {
return this.reduce((a,b) => a+b) / this.length;
}
// bindings
document.getElementById('red').addEventListener('click', () => {
const times = [];
for (var i = 0; i < 50; i++)
times.push(red());
display(times.avg());
}),
document.getElementById('blue').addEventListener('click', () => {
const times = [];
for (var i = 0; i < 50; i++)
times.push(blue());
display(times.avg());
});
<script src="https://cdn.jsdelivr.net/gh/TeleworkInc/state-machine#bd486a339dca1b3ad3157df20e832ec23c6eb00b/StateMachine.js"></script>
<h2 id="results">Waiting...</h2>
<button id="red">Red Pill</button>
<button id="blue">Blue Pill</button>
<style>
body{box-sizing:border-box;padding:0 4rem;text-align:center}button,h2,p{width:100%;margin:auto;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol"}button{font-size:1rem;padding:.5rem;width:180px;margin:1rem 0;border-radius:20px;outline:none;}#red{background:rgba(255,0,0,.24)}#blue{background:rgba(0,0,255,.24)}
</style>
Updates
Bug Report "Feature Request" filed (awaiting update) - See #jmrk's answers below for more details.
Ultimately, this behavior is unexpected and, IMO, qualifies as a nontrivial bug. The impact for me is significant - on Intel i7-4770 (8) # 3.900GHz, my execution times in the example above go from an average of 2ms to 45ms (a 20x increase).
As for nontriviality, consider that any subsequent calls to StateMachine.transition after the first one will be unnecessarily slow, regardless of scope or location in the code. The fact that SpiderMonkey does not slow down subsequent calls to transition signals to me that there is room for improvement for this specific optimization logic in V8.
See below, where subsequent calls to StateMachine.transition are slowed:
// same source, several times
// 1
(function() {
let start = performance.now(),
stateMachine = new StateMachine({
test: 0
});
stateMachine.transition(state => state.test++, state => state.test < 1e6);
if (stateMachine.state.test !== 1e6) throw 'ASSERT ERROR!';
console.log(`took ${performance.now() - start}ms`);
})();
// 2
(function() {
let start = performance.now(),
stateMachine = new StateMachine({
test: 0
});
stateMachine.transition(state => state.test++, state => state.test < 1e6);
if (stateMachine.state.test !== 1e6) throw 'ASSERT ERROR!';
console.log(`took ${performance.now() - start}ms`);
})();
// 3
(function() {
let start = performance.now(),
stateMachine = new StateMachine({
test: 0
});
stateMachine.transition(state => state.test++, state => state.test < 1e6);
if (stateMachine.state.test !== 1e6) throw 'ASSERT ERROR!';
console.log(`took ${performance.now() - start}ms`);
})();
<script src="https://cdn.jsdelivr.net/gh/TeleworkInc/state-machine#bd486a339dca1b3ad3157df20e832ec23c6eb00b/StateMachine.js"></script>
This performance decrease can be avoided by wrapping the code in a named closure, where presumably the optimizer knows the callbacks will not change:
var test = (function() {
let start = performance.now(),
stateMachine = new StateMachine({
test: 0
});
stateMachine.transition(state => state.test++, state => state.test < 1e6);
if (stateMachine.state.test !== 1e6) throw 'ASSERT ERROR!';
console.log(`took ${performance.now() - start}ms`);
});
test();
test();
test();
<script src="https://cdn.jsdelivr.net/gh/TeleworkInc/state-machine#bd486a339dca1b3ad3157df20e832ec23c6eb00b/StateMachine.js"></script>
Platform Information
$ uname -a
Linux workspaces 5.4.0-39-generic #43-Ubuntu SMP Fri Jun 19 10:28:31 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ google-chrome --version
Google Chrome 83.0.4103.116
V8 developer here. It's not a bug, it's just an optimization that V8 doesn't do. It's interesting to see that Firefox seems to do it...
FWIW, I don't see "ballooning to 400ms"; instead (similar to Jon Trent's comment) I see about 2.5ms at first, and then around 11ms.
Here's the explanation:
When you click only one button, then transition only ever sees one callback. (Strictly speaking it's a new instance of the arrow function every time, but since they all stem from the same function in the source, they're "deduped" for type feedback tracking purposes. Also, strictly speaking it's one callback each for stateTransition and transitionCondition, but that just duplicates the situation; either one alone would reproduce it.) When transition gets optimized, the optimizing compiler decides to inline the called function, because having seen only one function there in the past, it can make a high-confidence guess that it's also always going to be that one function in the future. Since the function does extremely little work, avoiding the overhead of calling it provides a huge performance boost.
Once the second button is clicked, transition sees a second function. It must get deoptimized the first time this happens; since it's still hot it'll get reoptimized soon after, but this time the optimizer decides not to inline, because it's seen more than one function before, and inlining can be very expensive. The result is that from this point onwards, you'll see the time it takes to actually perform these calls. (The fact that both functions have identical source doesn't matter; checking that wouldn't be worth it because outside of toy examples that would almost never be the case.)
There's a workaround, but it's something of a hack, and I don't recommend putting hacks into user code to account for engine behavior. V8 does support "polymorphic inlining", but (currently) only if it can deduce the call target from some object's type. So if you construct "config" objects that have the right functions installed as methods on their prototype, you can get V8 to inline them. Like so:
class StateMachine {
...
transition(config, maxCalls = Infinity) {
let i = 0;
while (
config.condition &&
config.condition(this.state) &&
i++ < maxCalls
) config.transition(this.state);
return this;
}
...
}
class RedConfig {
transition(state) { return state.test++ }
condition(state) { return state.test < 1e6 }
}
class BlueConfig {
transition(state) { return state.test++ }
condition(state) { return state.test < 1e6 }
}
function red() {
...
stateMachine.transition(new RedConfig());
...
}
function blue() {
...
stateMachine.transition(new BlueConfig());
...
}
It might be worth filing a bug (crbug.com/v8/new) to ask if the compiler team thinks that this is worth improving. Theoretically it should be possible to inline several functions that are called directly, and branch between the inlined paths based on the value of the function variable that's being called. However I'm not sure there are many cases where the impact is as pronounced as in this simple benchmark, and I know that recently the trend has been towards inlining less rather than more, because on average that tends to be the better tradeoff (there are drawbacks to inlining, and whether it's worth it is necessarily always a guess, because the engine would have to predict the future in order to be sure).
In conclusion, coding with many callbacks is a very flexible and often elegant technique, but it tends to come at an efficiency cost. (There are other varieties of inefficiency: e.g. a call with an inline arrow function like transition(state => state.something) allocates a new function object each time it's executed; that just so happens not to matter much in the example at hand.) Sometimes engines might be able to optimize away the overhead, and sometimes not.
Since this is getting so much interest (and updates to the question), I thought I'd provide some additional detail.
The new simplified test case is great: it's very simple, and very clearly shows a problem.
function test(callback) {
let start = performance.now();
for (let i = 0; i < 1e6; i++) callback();
console.log(`${callback.name} took ${(performance.now() - start).toFixed(2)}ms`);
}
var exampleA = (a,b) => 10**10;
var exampleB = (a,b) => 10**10;
// one callback -> fast
for (let i = 0; i < 10; i++) test(exampleA);
// introduce a second callback -> much slower forever
for (let i = 0; i < 10; i++) test(exampleB);
for (let i = 0; i < 10; i++) test(exampleA);
On my machine, I'm seeing times go as low as 0.23 ms for exampleA alone, and then they go up to 7.3ms when exampleB comes along, and stay there. Wow, a 30x slowdown! Clearly that's a bug in V8? Why wouldn't the team jump on fixing this?
Well, the situation is more complicated than it seems at first.
Firstly, the "slow" case is the normal situation. That's what you should expect to see in most code. It's still pretty fast! You can do a million function calls (plus a million exponentiations, plus a million loop iterations) in just 7 milliseconds! That's only 7 nanoseconds per iteration+call+exponentiation+return!
Actually, that analysis was a bit simplified. In reality, an operation on two constants like 10**10 will be constant-folded at compile time, so once exampleA and exampleB get optimized, the optimized code for them will return 1e10 immediately, without doing any multiplications.
On the flip side, the code here contains a small oversight that causes the engine to have to do more work: exampleA and exampleB take two parameters (a, b), but they're called without any arguments simply as callback(). Bridging this difference between expected and actual number of parameters is fast, but on a test like this that doesn't do much else, it amounts to about 40% of the total time spent. So a more accurate statement would be: it takes about 4 nanoseconds to do a loop iteration plus a function call plus a materialization of a number constant plus a function return, or 7 ns if the engine additionally has to adapt the arguments count of the call.
So what about the initial results for just exampleA, how can that case be so much faster? Well, that's the lucky situation that hits various optimizations in V8 and can take several shortcuts -- in fact it can take so many shortcuts that it ends up being a misleading microbenchmark: the results it produces don't reflect real situations, and can easily cause an observer to draw incorrect conclusions. The general effect that "always the same callback" is (typically) faster than "several different callbacks" is certainly real, but this test significantly distorts the magnitude of the difference.
At first, V8 sees that it's always the same function that's getting called, so the optimizing compiler decides to inline the function instead of calling it. That avoids the adaptation of arguments right off the bat. After inlining, the compiler can also see that the result of the exponentiation is never used, so it drops that entirely. The end result is that this test tests an empty loop! See for yourself:
function test_empty(no_callback) {
let start = performance.now();
for (let i = 0; i < 1e6; i++) {}
console.log(`empty loop took ${(performance.now() - start).toFixed(2)}ms`);
}
That gives me the same 0.23ms as calling exampleA. So contrary to what we thought, we didn't measure the time it takes to call and execute exampleA, in reality we measured no calls at all, and no 10**10 exponentiations either. (If you like more direct proof, you can run the original test in d8 or node with --print-opt-code and see the disassembly of the optimized code that V8 generates internally.)
All that lets us conclude a few things:
(1) This is not a case of "OMG there's this horrible slowdown that you must be aware of and avoid in your code". The default performance you get when you don't worry about this is great. Sometimes when the stars align you might see even more impressive optimizations, but… to put it lightly: just because you only get presents on a few occasions per year, doesn't mean that all the other non-gift-bearing days are some horrible bug that must be avoided.
(2) The smaller your test case, the bigger the observed difference between default speed and lucky fast case. If your callbacks are doing actual work that the compiler can't just eliminate, then the difference will be smaller than seen here. If your callbacks are doing more work than a single operation, then the fraction of overall time that's spent on the call itself will be smaller, so replacing the call with inlining will make less of a difference than it does here. If your functions are called with the parameters they need, that will avoid the needless penalization seen here. So while this microbenchmark manages to create the misleading impression that there's a shockingly large 30x difference, in most real applications it will be between maybe 4x in extreme cases and "not even measurable at all" for many other cases.
(3) Function calls do have a cost. It's great that (for many languages, including JavaScript) we have optimizing compilers that can sometimes avoid them via inlining. If you have a case where you really, really care about every last bit of performance, and your compiler happens to not inline what you think it should be inlining (for whatever reason: because it can't, or because it has internal heuristics that decide not to), then it can give significant benefits to redesign your code a bit -- e.g. you could inline by hand, or otherwise restructure your control flow to avoid millions of calls to tiny functions in your hottest loops. (Don't blindly overdo it though: having too few too big functions isn't great for optimization either. Usually it's best to not worry about this. Organize your code into chunks that make sense, let the engine take care of the rest. I'm only saying that sometimes, when you observe specific problems, you can help the engine do its job better.)
If you do need to rely on performance-sensitive function calls, then an easy tuning you can do is to make sure that you're calling your functions with exactly as many arguments as they expect -- which is probably often what you would do anyway. Of course optional arguments have their uses as well; like in so many other cases the extra flexibility comes with a (small) performance cost, which is often negligible, but can be taken into consideration when you feel that you have to.
(4) Observing such performance differences can understandably be surprising and sometimes even frustrating. Unfortunately, the nature of optimizations is such that they can't always be applied: they rely on making simplifying assumptions and not covering every case, otherwise they wouldn't be fast any more. We work very hard to give you reliable, predictable performance, with as many fast cases and as few slow cases as possible, and no steep cliffs between them. But we cannot escape the reality that we can't possibly "just make everything fast". (Which of course isn't to say that there's nothing left to do: every additional year of engineering work brings additional performance gains.) If we wanted to avoid all cases where more-or-less similar code exhibits noticeably different performance, then the only way to accomplish that would be to not do any optimizations at all, and instead leave everything at baseline ("slow") implementations -- and I don't think that would make anyone happy.
EDIT to add:
It seems there are major differences between different CPUs here, which probably explains why previous commenters have reported so wildly differing results. On hardware I can get my hands on, I'm seeing:
i7 6600U: 3.3 ms for inlined case, 28 ms for calling
i7 3635QM: 2.8 ms for inlined case, 10 ms for calling
i7 3635QM, up-to-date microcode: 2.8 ms for inlined case, 26 ms for calling
Ryzen 3900X: 2.5 ms for inlined case, 5 ms for calling
This is all with Chrome 83/84 on Linux; it's very much possible that running on Windows or Mac would yield different results (because CPU/microcode/kernel/sandbox are closely interacting with each other).
If you find these hardware differences shocking, read up on "spectre".
Related
I'm optimizing the compiler of a language to JavaScript, and found a very interesting, if not frustrating, case:
function add(n,m) {
return n === 0 ? m : add(n - 1, m) + 1;
};
var s = 0;
for (var i = 0; i < 100000; ++i) {
s += add(4000, 4000);
}
console.log(s);
It takes 2.3s to complete on my machine[1]. But if I make a very small change:
function add(n,m) {
return (() => n === 0 ? m : add(n - 1, m) + 1)();
};
var s = 0;
for (var i = 0; i < 100000; ++i) {
s += add(4000, 4000);
}
console.log(s);
It completes in 1.1s. Notice the only difference is the addition of an immediately invoked lambda, (() => ...)(), around the return of add. Why does this added call make my program two times faster?
[1] MacBook Pro 13" 2020, 2.3 GHz Quad-Core Intel Core i7, Node.js v15.3.0
Interesting! From looking at the code, it seems fairly obvious that the IIFE-wrapped version should be slower, not faster: in every loop iteration, it creates a new function object and calls it (which the optimizing compiler will eventually avoid, but that doesn't kick in right away), so generally just does more work, which should be taking more time.
The explanation in this case is inlining.
A bit of background: inlining one function into another (instead of calling it) is one of the standard tricks that optimizing compilers perform in order to achieve better performance. It's a double-edged sword though: on the plus side, it avoids calling overhead, and can often enable further optimizations, such as constant propagation, or elimination of duplicate computation (see below for an example). On the negative side, it causes compilation to take longer (because the compiler does more work), and it causes more code to be generated and stored in memory (because inlining a function effectively duplicates it), and in a dynamic language like JavaScript where optimized code typically relies on guarded assumptions, it increases the risk of one of these assumptions turning out to be wrong and a large amount of optimized code having to be thrown away as a result.
Generally speaking, making perfect inlining decisions (not too much, not too little) requires predicting the future: knowing in advance how often and with which parameters the code will be executed. That is, of course, impossible, so optimizing compilers use various rules/"heuristics" to make guesses about what might be a reasonably good decision.
One rule that V8 currently has is: don't inline recursive calls.
That's why in the simpler version of your code, add will not get inlined into itself. The IIFE version essentially has two functions calling each other, which is called "mutual recursion" -- and as it turns out, this simple trick is enough to fool V8's optimizing compiler and make it sidestep its "don't inline recursive calls" rule. Instead, it happily inlines the unnamed lambda into add, and add into the unnamed lambda, and so on, until its inlining budget runs out after ~30 rounds. (Side note: "how much gets inlined" is one of the somewhat-complex heuristics and in particular takes function size into account, so whatever specific behavior we see here is indeed specific to this situation.)
In this particular scenario, where the involved functions are very small, inlining helps quite a bit because it avoids call overhead. So in this case, inlining gives better performance, even though it is a (disguised) case of recursive inlining, which in general often is bad for performance. And it does come at a cost: in the simple version, the optimizing compiler spends only 3 milliseconds compiling add, producing 562 bytes of optimized code for it. In the IIFE version, the compiler spends 30 milliseconds and produces 4318 bytes of optimized code for add. That's one reason why it's not as simple as concluding "V8 should always inline more": time and battery consumption for compiling matters, and memory consumption matters too, and what might be acceptable cost (and improve performance significantly) in a simple 10-line demo may well have unacceptable cost (and potentially even cost overall performance) in a 100,000-line app.
Now, having understood what's going on, we can get back to the "IIFEs have overhead" intuition, and craft an even faster version:
function add(n,m) {
return add_inner(n, m);
};
function add_inner(n, m) {
return n === 0 ? m : add(n - 1, m) + 1;
}
On my machine, I'm seeing:
simple version: 1650 ms
IIFE version: 720 ms
add_inner version: 460 ms
Of course, if you implement add(n, m) simply as return n + m, then it terminates in 2 ms -- algorithmic optimization beats anything an optimizing compiler could possibly accomplish :-)
Appendix: Example for benefits of optimization. Consider these two functions:
function Process(x) {
return (x ** 2) + InternalDetail(x, 0, 2);
}
function InternalDetail(x, offset, power) {
return (x + offset) ** power;
}
(Obviously, this is silly code; but let's assume it's a simplified version of something that makes sense in practice.)
When executed naively, the following steps happen:
evaluate temp1 = (x ** 2)
call InternalDetail with parameters x, 0, 2
evaluate temp2 = (x + 0)
evaluate temp3 = temp2 ** 2
return temp3 to the caller
evaluate temp4 = temp1 + temp3
return temp4.
If an optimizing compiler performs inlining, then as a first step it will get:
function Process_after_inlining(x) {
return (x ** 2) + ( (x + 0) ** 2 );
}
which allows two simplifications: x + 0 can be folded to just x, and then the x ** 2 computation occurs twice, so the second occurrence can be replaced by reusing the result from the first:
function Process_with_optimizations(x) {
let temp1 = x ** 2;
return temp1 + temp1;
}
So comparing with the naive execution, we're down to 3 steps from 7:
evaluate temp1 = (x ** 2)
evaluate temp2 = temp1 + temp1
return temp2
I'm not predicting that real-world performance will go from 7 time units to 3 time units; this is just meant to give an intuitive idea of why inlining can help reduce computational load by some amount.
Footnote: to illustrate how tricky all this stuff is, consider that replacing x + 0 with just x is not always possible in JavaScript, even when the compiler knows that x is always a number: if x happens to be -0, then adding 0 to it changes it to +0, which may well be observable program behavior ;-)
CPU Cycles, Memory Usage, Execution Time, etc.?
Added: Is there a quantitative way of testing performance in JavaScript besides just perception of how fast the code runs?
Profilers are definitely a good way to get numbers, but in my experience, perceived performance is all that matters to the user/client. For example, we had a project with an Ext accordion that expanded to show some data and then a few nested Ext grids. Everything was actually rendering pretty fast, no single operation took a long time, there was just a lot of information being rendered all at once, so it felt slow to the user.
We 'fixed' this, not by switching to a faster component, or optimizing some method, but by rendering the data first, then rendering the grids with a setTimeout. So, the information appeared first, then the grids would pop into place a second later. Overall, it took slightly more processing time to do it that way, but to the user, the perceived performance was improved.
These days, the Chrome profiler and other tools are universally available and easy to use, as are
console.time() (mozilla-docs, chrome-docs)
console.profile() (mozilla-docs, chrome-docs)
performance.now() (mozilla-docs)
Chrome also gives you a timeline view which can show you what is killing your frame rate, where the user might be waiting, etc.
Finding documentation for all these tools is really easy, you don't need an SO answer for that. 7 years later, I'll still repeat the advice of my original answer and point out that you can have slow code run forever where a user won't notice it, and pretty fast code running where they do, and they will complain about the pretty fast code not being fast enough. Or that your request to your server API took 220ms. Or something else like that. The point remains that if you take a profiler out and go looking for work to do, you will find it, but it may not be the work your users need.
I do agree that perceived performance is really all that matters. But sometimes I just want to find out which method of doing something is faster. Sometimes the difference is HUGE and worth knowing.
You could just use javascript timers. But I typically get much more consistent results using the native Chrome (now also in Firefox and Safari) devTool methods console.time() & console.timeEnd()
Example of how I use it:
var iterations = 1000000;
console.time('Function #1');
for(var i = 0; i < iterations; i++ ){
functionOne();
};
console.timeEnd('Function #1')
console.time('Function #2');
for(var i = 0; i < iterations; i++ ){
functionTwo();
};
console.timeEnd('Function #2')
Update (4/4/2016):
Chrome canary recently added Line Level Profiling the dev tools sources tab which let's you see exactly how long each line took to execute!
We can always measure time taken by any function by simple date object.
var start = +new Date(); // log start timestamp
function1();
var end = +new Date(); // log end timestamp
var diff = end - start;
Try jsPerf. It's an online javascript performance tool for benchmarking and comparing snippets of code. I use it all the time.
Most browsers are now implementing high resolution timing in performance.now(). It's superior to new Date() for performance testing because it operates independently from the system clock.
Usage
var start = performance.now();
// code being timed...
var duration = performance.now() - start;
References
https://developer.mozilla.org/en-US/docs/Web/API/Performance.now()
http://www.w3.org/TR/hr-time/#dom-performance-now
JSLitmus is a lightweight tool for creating ad-hoc JavaScript benchmark tests
Let examine the performance between function expression and function constructor:
<script src="JSLitmus.js"></script>
<script>
JSLitmus.test("new Function ... ", function() {
return new Function("for(var i=0; i<100; i++) {}");
});
JSLitmus.test("function() ...", function() {
return (function() { for(var i=0; i<100; i++) {} });
});
</script>
What I did above is create a function expression and function constructor performing same operation. The result is as follows:
FireFox Performance Result
IE Performance Result
Some people are suggesting specific plug-ins and/or browsers. I would not because they're only really useful for that one platform; a test run on Firefox will not translate accurately to IE7. Considering 99.999999% of sites have more than one browser visit them, you need to check performance on all the popular platforms.
My suggestion would be to keep this in the JS. Create a benchmarking page with all your JS test on and time the execution. You could even have it AJAX-post the results back to you to keep it fully automated.
Then just rinse and repeat over different platforms.
Here is a simple function that displays the execution time of a passed in function:
var perf = function(testName, fn) {
var startTime = new Date().getTime();
fn();
var endTime = new Date().getTime();
console.log(testName + ": " + (endTime - startTime) + "ms");
}
I have a small tool where I can quickly run small test-cases in the browser and immediately get the results:
JavaScript Speed Test
You can play with code and find out which technique is better in the tested browser.
I think JavaScript performance (time) testing is quite enough. I found a very handy article about JavaScript performance testing here.
You could use this: http://getfirebug.com/js.html. It has a profiler for JavaScript.
I was looking something similar but found this.
https://jsbench.me/
It allows a side to side comparison and you can then also share the results.
performance.mark (Chrome 87 ^)
performance.mark('initSelect - start');
initSelect();
performance.mark('initSelect - end');
Quick answer
On jQuery (more specifically on Sizzle), we use this (checkout master and open speed/index.html on your browser), which in turn uses benchmark.js. This is used to performance test the library.
Long answer
If the reader doesn't know the difference between benchmark, workload and profilers, first read some performance testing foundations on the "readme 1st" section of spec.org. This is for system testing, but understanding this foundations will help JS perf testing as well. Some highlights:
What is a benchmark?
A benchmark is "a standard of measurement or evaluation" (Webster’s II Dictionary). A computer benchmark is typically a computer program that performs a strictly defined set of operations - a workload - and returns some form of result - a metric - describing how the tested computer performed. Computer benchmark metrics usually measure speed: how fast was the workload completed; or throughput: how many workload units per unit time were completed. Running the same computer benchmark on multiple computers allows a comparison to be made.
Should I benchmark my own application?
Ideally, the best comparison test for systems would be your own application with your own workload. Unfortunately, it is often impractical to get a wide base of reliable, repeatable and comparable measurements for different systems using your own application with your own workload. Problems might include generation of a good test case, confidentiality concerns, difficulty ensuring comparable conditions, time, money, or other constraints.
If not my own application, then what?
You may wish to consider using standardized benchmarks as a reference point. Ideally, a standardized benchmark will be portable, and may already have been run on the platforms that you are interested in. However, before you consider the results you need to be sure that you understand the correlation between your application/computing needs and what the benchmark is measuring. Are the benchmarks similar to the kinds of applications you run? Do the workloads have similar characteristics? Based on your answers to these questions, you can begin to see how the benchmark may approximate your reality.
Note: A standardized benchmark can serve as reference point. Nevertheless, when you are doing vendor or product selection, SPEC does not claim that any standardized benchmark can replace benchmarking your own actual application.
Performance testing JS
Ideally, the best perf test would be using your own application with your own workload switching what you need to test: different libraries, machines, etc.
If this is not feasible (and usually it is not). The first important step: define your workload. It should reflect your application's workload. In this talk, Vyacheslav Egorov talks about shitty workloads you should avoid.
Then, you could use tools like benchmark.js to assist you collect metrics, usually speed or throughput. On Sizzle, we're interested in comparing how fixes or changes affect the systemic performance of the library.
If something is performing really bad, your next step is to look for bottlenecks.
How do I find bottlenecks? Profilers
What is the best way to profile javascript execution?
I find execution time to be the best measure.
You could use console.profile in firebug
I usually just test javascript performance, how long script runs. jQuery Lover gave a good article link for testing javascript code performance, but the article only shows how to test how long your javascript code runs. I would also recommend reading article called "5 tips on improving your jQuery code while working with huge data sets".
Here is a reusable class for time performance. Example is included in code:
/*
Help track time lapse - tells you the time difference between each "check()" and since the "start()"
*/
var TimeCapture = function () {
var start = new Date().getTime();
var last = start;
var now = start;
this.start = function () {
start = new Date().getTime();
};
this.check = function (message) {
now = (new Date().getTime());
console.log(message, 'START:', now - start, 'LAST:', now - last);
last = now;
};
};
//Example:
var time = new TimeCapture();
//begin tracking time
time.start();
//...do stuff
time.check('say something here')//look at your console for output
//..do more stuff
time.check('say something else')//look at your console for output
//..do more stuff
time.check('say something else one more time')//look at your console for output
UX Profiler approaches this problem from user perspective. It groups all the browser events, network activity etc caused by some user action (click) and takes into consideration all the aspects like latency, timeouts etc.
Performance testing became something of a buzzword as of late but that’s not to say that performance testing is not an important process in QA or even after the product has shipped. And while I develop the app I use many different tools, some of them mentioned above like the chrome Profiler I usually look at a SaaS or something opensource that I can get going and forget about it until I get that alert saying that something went belly up.
There are lots of awesome tools that will help you keep an eye on performance without having you jump through hoops just to get some basics alerts set up. Here are a few that I think are worth checking out for yourself.
Sematext.com
Datadog.com
Uptime.com
Smartbear.com
Solarwinds.com
To try and paint a clearer picture, here is a little tutorial on how to set up monitoring for a react application.
You could use https://github.com/anywhichway/benchtest which wraps existing Mocha unit tests with performance tests.
The golden rule is to NOT under ANY circumstances lock your users browser. After that, I usually look at execution time, followed by memory usage (unless you're doing something crazy, in which case it could be a higher priority).
This is a very old question but I think we can contribute with a simple solution based on es6 for fast testing your code.
This is a basic bench for execution time. We use performance.now() to improve the accuracy:
/**
* Figure out how long it takes for a method to execute.
*
* #param {Function} method to test
* #param {number} iterations number of executions.
* #param {Array} list of set of args to pass in.
* #param {T} context the context to call the method in.
* #return {number} the time it took, in milliseconds to execute.
*/
const bench = (method, list, iterations, context) => {
let start = 0
const timer = action => {
const time = performance.now()
switch (action) {
case 'start':
start = time
return 0
case 'stop':
const elapsed = time - start
start = 0
return elapsed
default:
return time - start
}
};
const result = []
timer('start')
list = [...list]
for (let i = 0; i < iterations; i++) {
for (const args of list) {
result.push(method.apply(context, args))
}
}
const elapsed = timer('stop')
console.log(`Called method [${method.name}]
Mean: ${elapsed / iterations}
Exec. time: ${elapsed}`)
return elapsed
}
const fnc = () => {}
const isFunction = (f) => f && f instanceof Function
const isFunctionFaster = (f) => f && 'function' === typeof f
class A {}
function basicFnc(){}
async function asyncFnc(){}
const arrowFnc = ()=> {}
const arrowRFnc = ()=> 1
// Not functions
const obj = {}
const arr = []
const str = 'function'
const bol = true
const num = 1
const a = new A()
const list = [
[isFunction],
[basicFnc],
[arrowFnc],
[arrowRFnc],
[asyncFnc],
[Array],
[Date],
[Object],
[Number],
[String],
[Symbol],
[A],
[obj],
[arr],
[str],
[bol],
[num],
[a],
[null],
[undefined],
]
const e1 = bench(isFunction, list, 10000)
const e2 = bench(isFunctionFaster, list, 10000)
const rate = e2/e1
const percent = Math.abs(1 - rate)*100
console.log(`[isFunctionFaster] is ${(percent).toFixed(2)}% ${rate < 1 ? 'faster' : 'slower'} than [isFunction]`)
This is a good way of collecting performance information for the specific operation.
start = new Date().getTime();
for (var n = 0; n < maxCount; n++) {
/* perform the operation to be measured *//
}
elapsed = new Date().getTime() - start;
assert(true,"Measured time: " + elapsed);
As you can see in code below, when I increase the size of the string it leads to a 0 milliseconds difference. And moreover there is an inconsistency as the string count goes on increasing.
Am I doing something wrong here?
let stringIn = document.getElementById('str');
let button = document.querySelector('button');
button.addEventListener('click', () => {
let t1 = performance.now();
functionToTest(stringIn.value);
let t2 = performance.now();
console.log(`time taken is ${t2 - t1}`);
});
function functionToTest(str) {
let total = 0;
for(i of str) {
total ++;
}
return total;
}
<input id="str">
<button type="button">Test string</button>
I tried using await too, but result is the same (see code snippet below). The function enclosing the code below is async:
let stringArr = this.inputString.split(' ');
let longest = '';
const t1 = performance.now();
let length = await new Promise(resolve => {
stringArr.map((item, i) => {
longest = longest.length < item.length ? longest : item;
i === stringArr.length - 1 ? resolve(longest) : '';
});
});
const diff = performance.now() - t1;
console.log(diff);
this.result = `The time taken in mili seconds is ${diff}`;
I've also tried this answer as, but it is also inconsistent.
As a workaround I tried using console.time feature, but It doesn't allow the time to be rendered and isn't accurate as well.
Update: I want to build an interface like jsPerf, which will be quite similar to it but for a different purpose. Mostly I would like to compare different functions which will depend on user inputs.
There are 3 things which may help you understand what happening:
Browsers are reducing performance.now() precision to prevent Meltdown and Spectre attacks, so Chrome gives max 0.1 ms precision, FF 1ms, etc. This makes impossible to measure small timeframes. If function is extremely quick - 0 ms is understandable result. (Thanks #kaiido) Source: paper, additional links here
Any code(including JS) in multithreaded environment will not execute with constant performance (at least due to OS threads switching). So getting consistent values for several single runs is unreachable goal. To get some precise number - function should be executed multiple times, and average value is taken. This will even work with low precision performance.now(). (Boring explanation: if function is much faster than 0.1 ms, and browser often gives 0ms result, but time from time some function run will win a lottery and browser will return 0.1ms... longer functions will win this lottery more often)
There is "optimizing compiler" in most JS engines. It optimizes often used functions. Optimization is expensive, so JS engines optimize only often used functions. This explains performance increase after several runs. At first function is executed in slowest way. After several executions, it is optimized, and performance increases. (should add warmup runs?)
I was able to get non-zero numbers in your code snipet - by copy-pasting 70kb file into input. After 3rd run function was optimized, but even after this - performance is not constant
time taken is 11.49999990593642
time taken is 5.100000067614019
time taken is 2.3999999975785613
time taken is 2.199999988079071
time taken is 2.199999988079071
time taken is 2.099999925121665
time taken is 2.3999999975785613
time taken is 1.7999999690800905
time taken is 1.3000000035390258
time taken is 2.099999925121665
time taken is 1.9000000320374966
time taken is 2.300000051036477
Explanation of 1st point
Lets say two events happened, and the goal is to find time between them.
1st event happened at time A and second event happened at time B. Browser is rounding precise values A and B and returns them.
Several cases to look at:
A B A-B floor(A) floor(B) Ar-Br
12.001 12.003 0.002 12 12 0
11.999 12.001 0.002 11 12 1
Browsers are smarter than we think, there are lot's of improvements and caching techniques take in place for memory allocation, repeatable code execution, on-demand CPU allocation and so on. For instance V8, the JavaScript engine that powers Chrome and Node.js caches code execution cycles and results. Furthermore, your results may get affected by the resources that your browser utilizes upon code execution, so your results even with multiple executions cycles may vary.
As you have mentioned that you are trying to create a jsPerf clone take a look at Benchmark.js, this library is used by the jsPerf development team.
Running performance analysis tests is really hard and I would suggest running them on a Node.js environment with predefined and preallocated resources in order to obtain your results.
You might what to take a look at https://github.com/anywhichway/benchtest which just reuses Mocha unit tests. Note, performance testing at the unit level should only be one part of your performance testing, you should also use simulators that emulate real world conditions and test your code at the application level to assess
network impacts, module interactions, etc.
CPU Cycles, Memory Usage, Execution Time, etc.?
Added: Is there a quantitative way of testing performance in JavaScript besides just perception of how fast the code runs?
Profilers are definitely a good way to get numbers, but in my experience, perceived performance is all that matters to the user/client. For example, we had a project with an Ext accordion that expanded to show some data and then a few nested Ext grids. Everything was actually rendering pretty fast, no single operation took a long time, there was just a lot of information being rendered all at once, so it felt slow to the user.
We 'fixed' this, not by switching to a faster component, or optimizing some method, but by rendering the data first, then rendering the grids with a setTimeout. So, the information appeared first, then the grids would pop into place a second later. Overall, it took slightly more processing time to do it that way, but to the user, the perceived performance was improved.
These days, the Chrome profiler and other tools are universally available and easy to use, as are
console.time() (mozilla-docs, chrome-docs)
console.profile() (mozilla-docs, chrome-docs)
performance.now() (mozilla-docs)
Chrome also gives you a timeline view which can show you what is killing your frame rate, where the user might be waiting, etc.
Finding documentation for all these tools is really easy, you don't need an SO answer for that. 7 years later, I'll still repeat the advice of my original answer and point out that you can have slow code run forever where a user won't notice it, and pretty fast code running where they do, and they will complain about the pretty fast code not being fast enough. Or that your request to your server API took 220ms. Or something else like that. The point remains that if you take a profiler out and go looking for work to do, you will find it, but it may not be the work your users need.
I do agree that perceived performance is really all that matters. But sometimes I just want to find out which method of doing something is faster. Sometimes the difference is HUGE and worth knowing.
You could just use javascript timers. But I typically get much more consistent results using the native Chrome (now also in Firefox and Safari) devTool methods console.time() & console.timeEnd()
Example of how I use it:
var iterations = 1000000;
console.time('Function #1');
for(var i = 0; i < iterations; i++ ){
functionOne();
};
console.timeEnd('Function #1')
console.time('Function #2');
for(var i = 0; i < iterations; i++ ){
functionTwo();
};
console.timeEnd('Function #2')
Update (4/4/2016):
Chrome canary recently added Line Level Profiling the dev tools sources tab which let's you see exactly how long each line took to execute!
We can always measure time taken by any function by simple date object.
var start = +new Date(); // log start timestamp
function1();
var end = +new Date(); // log end timestamp
var diff = end - start;
Try jsPerf. It's an online javascript performance tool for benchmarking and comparing snippets of code. I use it all the time.
Most browsers are now implementing high resolution timing in performance.now(). It's superior to new Date() for performance testing because it operates independently from the system clock.
Usage
var start = performance.now();
// code being timed...
var duration = performance.now() - start;
References
https://developer.mozilla.org/en-US/docs/Web/API/Performance.now()
http://www.w3.org/TR/hr-time/#dom-performance-now
JSLitmus is a lightweight tool for creating ad-hoc JavaScript benchmark tests
Let examine the performance between function expression and function constructor:
<script src="JSLitmus.js"></script>
<script>
JSLitmus.test("new Function ... ", function() {
return new Function("for(var i=0; i<100; i++) {}");
});
JSLitmus.test("function() ...", function() {
return (function() { for(var i=0; i<100; i++) {} });
});
</script>
What I did above is create a function expression and function constructor performing same operation. The result is as follows:
FireFox Performance Result
IE Performance Result
Some people are suggesting specific plug-ins and/or browsers. I would not because they're only really useful for that one platform; a test run on Firefox will not translate accurately to IE7. Considering 99.999999% of sites have more than one browser visit them, you need to check performance on all the popular platforms.
My suggestion would be to keep this in the JS. Create a benchmarking page with all your JS test on and time the execution. You could even have it AJAX-post the results back to you to keep it fully automated.
Then just rinse and repeat over different platforms.
Here is a simple function that displays the execution time of a passed in function:
var perf = function(testName, fn) {
var startTime = new Date().getTime();
fn();
var endTime = new Date().getTime();
console.log(testName + ": " + (endTime - startTime) + "ms");
}
I have a small tool where I can quickly run small test-cases in the browser and immediately get the results:
JavaScript Speed Test
You can play with code and find out which technique is better in the tested browser.
I think JavaScript performance (time) testing is quite enough. I found a very handy article about JavaScript performance testing here.
You could use this: http://getfirebug.com/js.html. It has a profiler for JavaScript.
I was looking something similar but found this.
https://jsbench.me/
It allows a side to side comparison and you can then also share the results.
performance.mark (Chrome 87 ^)
performance.mark('initSelect - start');
initSelect();
performance.mark('initSelect - end');
Quick answer
On jQuery (more specifically on Sizzle), we use this (checkout master and open speed/index.html on your browser), which in turn uses benchmark.js. This is used to performance test the library.
Long answer
If the reader doesn't know the difference between benchmark, workload and profilers, first read some performance testing foundations on the "readme 1st" section of spec.org. This is for system testing, but understanding this foundations will help JS perf testing as well. Some highlights:
What is a benchmark?
A benchmark is "a standard of measurement or evaluation" (Webster’s II Dictionary). A computer benchmark is typically a computer program that performs a strictly defined set of operations - a workload - and returns some form of result - a metric - describing how the tested computer performed. Computer benchmark metrics usually measure speed: how fast was the workload completed; or throughput: how many workload units per unit time were completed. Running the same computer benchmark on multiple computers allows a comparison to be made.
Should I benchmark my own application?
Ideally, the best comparison test for systems would be your own application with your own workload. Unfortunately, it is often impractical to get a wide base of reliable, repeatable and comparable measurements for different systems using your own application with your own workload. Problems might include generation of a good test case, confidentiality concerns, difficulty ensuring comparable conditions, time, money, or other constraints.
If not my own application, then what?
You may wish to consider using standardized benchmarks as a reference point. Ideally, a standardized benchmark will be portable, and may already have been run on the platforms that you are interested in. However, before you consider the results you need to be sure that you understand the correlation between your application/computing needs and what the benchmark is measuring. Are the benchmarks similar to the kinds of applications you run? Do the workloads have similar characteristics? Based on your answers to these questions, you can begin to see how the benchmark may approximate your reality.
Note: A standardized benchmark can serve as reference point. Nevertheless, when you are doing vendor or product selection, SPEC does not claim that any standardized benchmark can replace benchmarking your own actual application.
Performance testing JS
Ideally, the best perf test would be using your own application with your own workload switching what you need to test: different libraries, machines, etc.
If this is not feasible (and usually it is not). The first important step: define your workload. It should reflect your application's workload. In this talk, Vyacheslav Egorov talks about shitty workloads you should avoid.
Then, you could use tools like benchmark.js to assist you collect metrics, usually speed or throughput. On Sizzle, we're interested in comparing how fixes or changes affect the systemic performance of the library.
If something is performing really bad, your next step is to look for bottlenecks.
How do I find bottlenecks? Profilers
What is the best way to profile javascript execution?
I find execution time to be the best measure.
You could use console.profile in firebug
I usually just test javascript performance, how long script runs. jQuery Lover gave a good article link for testing javascript code performance, but the article only shows how to test how long your javascript code runs. I would also recommend reading article called "5 tips on improving your jQuery code while working with huge data sets".
Here is a reusable class for time performance. Example is included in code:
/*
Help track time lapse - tells you the time difference between each "check()" and since the "start()"
*/
var TimeCapture = function () {
var start = new Date().getTime();
var last = start;
var now = start;
this.start = function () {
start = new Date().getTime();
};
this.check = function (message) {
now = (new Date().getTime());
console.log(message, 'START:', now - start, 'LAST:', now - last);
last = now;
};
};
//Example:
var time = new TimeCapture();
//begin tracking time
time.start();
//...do stuff
time.check('say something here')//look at your console for output
//..do more stuff
time.check('say something else')//look at your console for output
//..do more stuff
time.check('say something else one more time')//look at your console for output
UX Profiler approaches this problem from user perspective. It groups all the browser events, network activity etc caused by some user action (click) and takes into consideration all the aspects like latency, timeouts etc.
Performance testing became something of a buzzword as of late but that’s not to say that performance testing is not an important process in QA or even after the product has shipped. And while I develop the app I use many different tools, some of them mentioned above like the chrome Profiler I usually look at a SaaS or something opensource that I can get going and forget about it until I get that alert saying that something went belly up.
There are lots of awesome tools that will help you keep an eye on performance without having you jump through hoops just to get some basics alerts set up. Here are a few that I think are worth checking out for yourself.
Sematext.com
Datadog.com
Uptime.com
Smartbear.com
Solarwinds.com
To try and paint a clearer picture, here is a little tutorial on how to set up monitoring for a react application.
You could use https://github.com/anywhichway/benchtest which wraps existing Mocha unit tests with performance tests.
The golden rule is to NOT under ANY circumstances lock your users browser. After that, I usually look at execution time, followed by memory usage (unless you're doing something crazy, in which case it could be a higher priority).
This is a very old question but I think we can contribute with a simple solution based on es6 for fast testing your code.
This is a basic bench for execution time. We use performance.now() to improve the accuracy:
/**
* Figure out how long it takes for a method to execute.
*
* #param {Function} method to test
* #param {number} iterations number of executions.
* #param {Array} list of set of args to pass in.
* #param {T} context the context to call the method in.
* #return {number} the time it took, in milliseconds to execute.
*/
const bench = (method, list, iterations, context) => {
let start = 0
const timer = action => {
const time = performance.now()
switch (action) {
case 'start':
start = time
return 0
case 'stop':
const elapsed = time - start
start = 0
return elapsed
default:
return time - start
}
};
const result = []
timer('start')
list = [...list]
for (let i = 0; i < iterations; i++) {
for (const args of list) {
result.push(method.apply(context, args))
}
}
const elapsed = timer('stop')
console.log(`Called method [${method.name}]
Mean: ${elapsed / iterations}
Exec. time: ${elapsed}`)
return elapsed
}
const fnc = () => {}
const isFunction = (f) => f && f instanceof Function
const isFunctionFaster = (f) => f && 'function' === typeof f
class A {}
function basicFnc(){}
async function asyncFnc(){}
const arrowFnc = ()=> {}
const arrowRFnc = ()=> 1
// Not functions
const obj = {}
const arr = []
const str = 'function'
const bol = true
const num = 1
const a = new A()
const list = [
[isFunction],
[basicFnc],
[arrowFnc],
[arrowRFnc],
[asyncFnc],
[Array],
[Date],
[Object],
[Number],
[String],
[Symbol],
[A],
[obj],
[arr],
[str],
[bol],
[num],
[a],
[null],
[undefined],
]
const e1 = bench(isFunction, list, 10000)
const e2 = bench(isFunctionFaster, list, 10000)
const rate = e2/e1
const percent = Math.abs(1 - rate)*100
console.log(`[isFunctionFaster] is ${(percent).toFixed(2)}% ${rate < 1 ? 'faster' : 'slower'} than [isFunction]`)
This is a good way of collecting performance information for the specific operation.
start = new Date().getTime();
for (var n = 0; n < maxCount; n++) {
/* perform the operation to be measured *//
}
elapsed = new Date().getTime() - start;
assert(true,"Measured time: " + elapsed);
CPU Cycles, Memory Usage, Execution Time, etc.?
Added: Is there a quantitative way of testing performance in JavaScript besides just perception of how fast the code runs?
Profilers are definitely a good way to get numbers, but in my experience, perceived performance is all that matters to the user/client. For example, we had a project with an Ext accordion that expanded to show some data and then a few nested Ext grids. Everything was actually rendering pretty fast, no single operation took a long time, there was just a lot of information being rendered all at once, so it felt slow to the user.
We 'fixed' this, not by switching to a faster component, or optimizing some method, but by rendering the data first, then rendering the grids with a setTimeout. So, the information appeared first, then the grids would pop into place a second later. Overall, it took slightly more processing time to do it that way, but to the user, the perceived performance was improved.
These days, the Chrome profiler and other tools are universally available and easy to use, as are
console.time() (mozilla-docs, chrome-docs)
console.profile() (mozilla-docs, chrome-docs)
performance.now() (mozilla-docs)
Chrome also gives you a timeline view which can show you what is killing your frame rate, where the user might be waiting, etc.
Finding documentation for all these tools is really easy, you don't need an SO answer for that. 7 years later, I'll still repeat the advice of my original answer and point out that you can have slow code run forever where a user won't notice it, and pretty fast code running where they do, and they will complain about the pretty fast code not being fast enough. Or that your request to your server API took 220ms. Or something else like that. The point remains that if you take a profiler out and go looking for work to do, you will find it, but it may not be the work your users need.
I do agree that perceived performance is really all that matters. But sometimes I just want to find out which method of doing something is faster. Sometimes the difference is HUGE and worth knowing.
You could just use javascript timers. But I typically get much more consistent results using the native Chrome (now also in Firefox and Safari) devTool methods console.time() & console.timeEnd()
Example of how I use it:
var iterations = 1000000;
console.time('Function #1');
for(var i = 0; i < iterations; i++ ){
functionOne();
};
console.timeEnd('Function #1')
console.time('Function #2');
for(var i = 0; i < iterations; i++ ){
functionTwo();
};
console.timeEnd('Function #2')
Update (4/4/2016):
Chrome canary recently added Line Level Profiling the dev tools sources tab which let's you see exactly how long each line took to execute!
We can always measure time taken by any function by simple date object.
var start = +new Date(); // log start timestamp
function1();
var end = +new Date(); // log end timestamp
var diff = end - start;
Try jsPerf. It's an online javascript performance tool for benchmarking and comparing snippets of code. I use it all the time.
Most browsers are now implementing high resolution timing in performance.now(). It's superior to new Date() for performance testing because it operates independently from the system clock.
Usage
var start = performance.now();
// code being timed...
var duration = performance.now() - start;
References
https://developer.mozilla.org/en-US/docs/Web/API/Performance.now()
http://www.w3.org/TR/hr-time/#dom-performance-now
JSLitmus is a lightweight tool for creating ad-hoc JavaScript benchmark tests
Let examine the performance between function expression and function constructor:
<script src="JSLitmus.js"></script>
<script>
JSLitmus.test("new Function ... ", function() {
return new Function("for(var i=0; i<100; i++) {}");
});
JSLitmus.test("function() ...", function() {
return (function() { for(var i=0; i<100; i++) {} });
});
</script>
What I did above is create a function expression and function constructor performing same operation. The result is as follows:
FireFox Performance Result
IE Performance Result
Some people are suggesting specific plug-ins and/or browsers. I would not because they're only really useful for that one platform; a test run on Firefox will not translate accurately to IE7. Considering 99.999999% of sites have more than one browser visit them, you need to check performance on all the popular platforms.
My suggestion would be to keep this in the JS. Create a benchmarking page with all your JS test on and time the execution. You could even have it AJAX-post the results back to you to keep it fully automated.
Then just rinse and repeat over different platforms.
Here is a simple function that displays the execution time of a passed in function:
var perf = function(testName, fn) {
var startTime = new Date().getTime();
fn();
var endTime = new Date().getTime();
console.log(testName + ": " + (endTime - startTime) + "ms");
}
I have a small tool where I can quickly run small test-cases in the browser and immediately get the results:
JavaScript Speed Test
You can play with code and find out which technique is better in the tested browser.
I think JavaScript performance (time) testing is quite enough. I found a very handy article about JavaScript performance testing here.
You could use this: http://getfirebug.com/js.html. It has a profiler for JavaScript.
I was looking something similar but found this.
https://jsbench.me/
It allows a side to side comparison and you can then also share the results.
performance.mark (Chrome 87 ^)
performance.mark('initSelect - start');
initSelect();
performance.mark('initSelect - end');
Quick answer
On jQuery (more specifically on Sizzle), we use this (checkout master and open speed/index.html on your browser), which in turn uses benchmark.js. This is used to performance test the library.
Long answer
If the reader doesn't know the difference between benchmark, workload and profilers, first read some performance testing foundations on the "readme 1st" section of spec.org. This is for system testing, but understanding this foundations will help JS perf testing as well. Some highlights:
What is a benchmark?
A benchmark is "a standard of measurement or evaluation" (Webster’s II Dictionary). A computer benchmark is typically a computer program that performs a strictly defined set of operations - a workload - and returns some form of result - a metric - describing how the tested computer performed. Computer benchmark metrics usually measure speed: how fast was the workload completed; or throughput: how many workload units per unit time were completed. Running the same computer benchmark on multiple computers allows a comparison to be made.
Should I benchmark my own application?
Ideally, the best comparison test for systems would be your own application with your own workload. Unfortunately, it is often impractical to get a wide base of reliable, repeatable and comparable measurements for different systems using your own application with your own workload. Problems might include generation of a good test case, confidentiality concerns, difficulty ensuring comparable conditions, time, money, or other constraints.
If not my own application, then what?
You may wish to consider using standardized benchmarks as a reference point. Ideally, a standardized benchmark will be portable, and may already have been run on the platforms that you are interested in. However, before you consider the results you need to be sure that you understand the correlation between your application/computing needs and what the benchmark is measuring. Are the benchmarks similar to the kinds of applications you run? Do the workloads have similar characteristics? Based on your answers to these questions, you can begin to see how the benchmark may approximate your reality.
Note: A standardized benchmark can serve as reference point. Nevertheless, when you are doing vendor or product selection, SPEC does not claim that any standardized benchmark can replace benchmarking your own actual application.
Performance testing JS
Ideally, the best perf test would be using your own application with your own workload switching what you need to test: different libraries, machines, etc.
If this is not feasible (and usually it is not). The first important step: define your workload. It should reflect your application's workload. In this talk, Vyacheslav Egorov talks about shitty workloads you should avoid.
Then, you could use tools like benchmark.js to assist you collect metrics, usually speed or throughput. On Sizzle, we're interested in comparing how fixes or changes affect the systemic performance of the library.
If something is performing really bad, your next step is to look for bottlenecks.
How do I find bottlenecks? Profilers
What is the best way to profile javascript execution?
I find execution time to be the best measure.
You could use console.profile in firebug
I usually just test javascript performance, how long script runs. jQuery Lover gave a good article link for testing javascript code performance, but the article only shows how to test how long your javascript code runs. I would also recommend reading article called "5 tips on improving your jQuery code while working with huge data sets".
Here is a reusable class for time performance. Example is included in code:
/*
Help track time lapse - tells you the time difference between each "check()" and since the "start()"
*/
var TimeCapture = function () {
var start = new Date().getTime();
var last = start;
var now = start;
this.start = function () {
start = new Date().getTime();
};
this.check = function (message) {
now = (new Date().getTime());
console.log(message, 'START:', now - start, 'LAST:', now - last);
last = now;
};
};
//Example:
var time = new TimeCapture();
//begin tracking time
time.start();
//...do stuff
time.check('say something here')//look at your console for output
//..do more stuff
time.check('say something else')//look at your console for output
//..do more stuff
time.check('say something else one more time')//look at your console for output
UX Profiler approaches this problem from user perspective. It groups all the browser events, network activity etc caused by some user action (click) and takes into consideration all the aspects like latency, timeouts etc.
Performance testing became something of a buzzword as of late but that’s not to say that performance testing is not an important process in QA or even after the product has shipped. And while I develop the app I use many different tools, some of them mentioned above like the chrome Profiler I usually look at a SaaS or something opensource that I can get going and forget about it until I get that alert saying that something went belly up.
There are lots of awesome tools that will help you keep an eye on performance without having you jump through hoops just to get some basics alerts set up. Here are a few that I think are worth checking out for yourself.
Sematext.com
Datadog.com
Uptime.com
Smartbear.com
Solarwinds.com
To try and paint a clearer picture, here is a little tutorial on how to set up monitoring for a react application.
You could use https://github.com/anywhichway/benchtest which wraps existing Mocha unit tests with performance tests.
The golden rule is to NOT under ANY circumstances lock your users browser. After that, I usually look at execution time, followed by memory usage (unless you're doing something crazy, in which case it could be a higher priority).
This is a very old question but I think we can contribute with a simple solution based on es6 for fast testing your code.
This is a basic bench for execution time. We use performance.now() to improve the accuracy:
/**
* Figure out how long it takes for a method to execute.
*
* #param {Function} method to test
* #param {number} iterations number of executions.
* #param {Array} list of set of args to pass in.
* #param {T} context the context to call the method in.
* #return {number} the time it took, in milliseconds to execute.
*/
const bench = (method, list, iterations, context) => {
let start = 0
const timer = action => {
const time = performance.now()
switch (action) {
case 'start':
start = time
return 0
case 'stop':
const elapsed = time - start
start = 0
return elapsed
default:
return time - start
}
};
const result = []
timer('start')
list = [...list]
for (let i = 0; i < iterations; i++) {
for (const args of list) {
result.push(method.apply(context, args))
}
}
const elapsed = timer('stop')
console.log(`Called method [${method.name}]
Mean: ${elapsed / iterations}
Exec. time: ${elapsed}`)
return elapsed
}
const fnc = () => {}
const isFunction = (f) => f && f instanceof Function
const isFunctionFaster = (f) => f && 'function' === typeof f
class A {}
function basicFnc(){}
async function asyncFnc(){}
const arrowFnc = ()=> {}
const arrowRFnc = ()=> 1
// Not functions
const obj = {}
const arr = []
const str = 'function'
const bol = true
const num = 1
const a = new A()
const list = [
[isFunction],
[basicFnc],
[arrowFnc],
[arrowRFnc],
[asyncFnc],
[Array],
[Date],
[Object],
[Number],
[String],
[Symbol],
[A],
[obj],
[arr],
[str],
[bol],
[num],
[a],
[null],
[undefined],
]
const e1 = bench(isFunction, list, 10000)
const e2 = bench(isFunctionFaster, list, 10000)
const rate = e2/e1
const percent = Math.abs(1 - rate)*100
console.log(`[isFunctionFaster] is ${(percent).toFixed(2)}% ${rate < 1 ? 'faster' : 'slower'} than [isFunction]`)
This is a good way of collecting performance information for the specific operation.
start = new Date().getTime();
for (var n = 0; n < maxCount; n++) {
/* perform the operation to be measured *//
}
elapsed = new Date().getTime() - start;
assert(true,"Measured time: " + elapsed);