Develop Reference

How V8 optimise code using hidden classes and inline caching

Recently I came across the concept of hidden classes and inline caching used by V8 to optimise js code. Cool.
I understand that objects are represented as hidden classes internally. And two objects may have same properties but different hidden classes (depending upon the order in which properties are assigned).
Also V8 uses inline caching concept to directly check offset to access properties of object rather than using object's hidden class to determine offsets.
Code -
function Point(x, y) {
this.x = x;
this.y = y;
}
function processPoint(point) {
// console.log(point.x, point.y, point.a, point.b);
// let x = point;
}
function main() {
let p1 = new Point(1, 1);
let p2 = new Point(1, 1);
let p3 = new Point(1, 1);
const N = 300000000;
p1.a = 1;
p1.b = 1;
p2.b = 1;
p2.a = 1;
p3.a = 1;
p3.b = 1;
let start_1 = new Date();
for(let i = 0; i< N; i++ ) {
if (i%4 != 0) {
processPoint(p1);
} else {
processPoint(p2)
}
}
let end_1 = new Date();
let t1 = (end_1 - start_1);
let start_2 = new Date();
for(let i = 0; i< N; i++ ) {
if (i%4 != 0) {
processPoint(p1);
} else {
processPoint(p1)
}
}
let end_2 = new Date();
let t2 = (end_2 - start_2);
let start_3 = new Date();
for(let i = 0; i< N; i++ ) {
if (i%4 != 0) {
processPoint(p1);
} else {
processPoint(p3)
}
}
let end_3 = new Date();
let t3 = (end_3 - start_3);
console.log(t1, t2, t3);
}
(function(){
main();
})();
I was expecting results to be like t1 > (t2 = t3) because :
first loop : V8 will try to optimise after running twice but it will soon encounter different hidden class so it will de optimise.
second loop : same object is called all the time so inline caching can be used.
third loop : same as second loop because hidden classes are same.
But results are not satisfying. I got (and similar results running again and again) -
3553 4805 4556
Questions :
Why results were not as expected? Where did my assumptions go wrong?
How can I change this code to demonstrate hidden classes and inline caching performance improvements?
Did I get it all wrong from the starting?
Are hidden classes present just for memory efficiency by letting objects share them?
Any other sites with some simple examples of performance improvements?
I am using node 8.9.4 for testing. Thanks in advance.
Sources :
https://blog.sessionstack.com/how-javascript-works-inside-the-v8-engine-5-tips-on-how-to-write-optimized-code-ac089e62b12e
https://draft.li/blog/2016/12/22/javascript-engines-hidden-classes/
https://richardartoul.github.io/jekyll/update/2015/04/26/hidden-classes.html
and many more..

V8 developer here. The summary is: Microbenchmarking is hard, don't do it.
First off, with your code as posted, I'm seeing 380 380 380 as the output, which is expected, because function processPoint is empty, so all loops do the same work (i.e., no work) no matter which point object you select.
Measuring the performance difference between monomorphic and 2-way polymorphic inline caches is difficult, because it is not large, so you have to be very careful about what else your benchmark is doing. console.log, for example, is so slow that it'll shadow everything else.
You'll also have to be careful about the effects of inlining. When your benchmark has many iterations, the code will get optimized (after running waaaay more than twice), and the optimizing compiler will (to some extent) inline functions, which can allow subsequent optimizations (specifically: eliminating various things) and thereby can significantly change what you're measuring. Writing meaningful microbenchmarks is hard; you won't get around inspecting generated assembly and/or knowing quite a bit about the implementation details of the JavaScript engine you're investigating.
Another thing to keep in mind is where inline caches are, and what state they'll have over time. Disregarding inlining, a function like processPoint doesn't know or care where it's called from. Once its inline caches are polymorphic, they'll remain polymorphic, even if later on in your benchmark (in this case, in the second and third loop) the types stabilize.
Yet another thing to keep in mind when trying to isolate effects is that long-running functions will get compiled in the background while they run, and will then at some point be replaced on the stack ("OSR"), which adds all sorts of noise to your measurements. When you invoke them with different loop lengths for warmup, they'll still get compiled in the background however, and there's no way to reliably wait for that background job. You could resort to command-line flags intended for development, but then you wouldn't be measuring regular behavior any more.
Anyhow, the following is an attempt to craft a test similar to yours that produces plausible results (about 100 180 280 on my machine):
function Point() {}
// These three functions are identical, but they will be called with different
// inputs and hence collect different type feedback:
function processPointMonomorphic(N, point) {
let sum = 0;
for (let i = 0; i < N; i++) {
sum += point.a;
}
return sum;
}
function processPointPolymorphic(N, point) {
let sum = 0;
for (let i = 0; i < N; i++) {
sum += point.a;
}
return sum;
}
function processPointGeneric(N, point) {
let sum = 0;
for (let i = 0; i < N; i++) {
sum += point.a;
}
return sum;
}
let p1 = new Point();
let p2 = new Point();
let p3 = new Point();
let p4 = new Point();
const warmup = 12000;
const N = 100000000;
let sum = 0;
p1.a = 1;
p2.b = 1;
p2.a = 1;
p3.c = 1;
p3.b = 1;
p3.a = 1;
p4.d = 1;
p4.c = 1;
p4.b = 1;
p4.a = 1;
processPointMonomorphic(warmup, p1);
processPointMonomorphic(1, p1);
let start_1 = Date.now();
sum += processPointMonomorphic(N, p1);
let t1 = Date.now() - start_1;
processPointPolymorphic(2, p1);
processPointPolymorphic(2, p2);
processPointPolymorphic(2, p3);
processPointPolymorphic(warmup, p4);
processPointPolymorphic(1, p4);
let start_2 = Date.now();
sum += processPointPolymorphic(N, p1);
let t2 = Date.now() - start_2;
processPointGeneric(warmup, 1);
processPointGeneric(1, 1);
let start_3 = Date.now();
sum += processPointGeneric(N, p1);
let t3 = Date.now() - start_3;
console.log(t1, t2, t3);

Why isn't my for loop in Javascript changing the variable amount?

I made a Javascript code that should take two numbers in an array, arrange them from lowest to highest, and then return the sum of every number between and including them. So sumAll(1, 4) // returns the sum of 1 + 2 + 3 + 4 which is 10. However, when I do it, my variable total does not change from 0 with my for loop.
var sumAll = function(arr) {
var lower = Math.min(arr[0], arr[1]);
var upper = Math.max(arr[0], arr[1]);
var total = 0;
for (var i = lower; i <= upper; i++) {
total += i;
}
return total;
}
This code should return 10 when using 1 and 4, but it gives 0 because var total = 0. When I manually change var total to equal 10, then it turns to ten, which means my for loop is not working the way it should, and I have no idea why. Any help is greatly appreciated, I am super new to coding and have been working on this for a while.
Also, this is a TDD project so it is attached to a spec js file. So the only other part of this code is not actually me calling the array but
module.exports = sumAll
The spec file asks for this:
var sumAll = require('./sumAll')
describe('sumAll', function() {
it('sums numbers within the range', function() {
expect(sumAll(1, 4)).toEqual(10);
});

The code in your question works well, however, if you still have issues you could try this:
function sumAll(n1, n2) {
var numbers = [];
numbers.length = Math.abs(n2-n1)+1; numbers.fill(0);
numbers = numbers.map((i, j) => (Math.min(n1,n2)+j))
return numbers.reduce((acc, i) => (acc+=i))
}
sumAll(1,4); //10

idiom for async iteration over large data in JavaScript ES6

Is there an idiom for iterating over large datasets in ES6 to avoid browser timeout?
Let's say I need to do something like generate 16 million cubes or something and that a straight forward loop times out the browser.
function generateCubes(num) {
var cubes = [];
for (var ii = 0; ii < num; ++ii) {
cubes.push(generateCube());
}
return cubes;
}
var cubes = generateCubes(16000000);
So I can turn that into a async callback like this
function generateCubes(num, callback) {
var maxPerIteration = 100000;
var cubes = [];
function makeMore() {
var count = Math.min(num, maxPerIteration);
for (var ii = 0; ii < count; ++ii) {
cubes.push(generateCube());
}
num -= count;
if (count) {
setTimeout(makeMore, 0);
} else {
callback(cubes);
}
}
makeMore();
}
but sadly I suddenly have to restructure all my code
generateCubes(16000000, function(cubes) {
...
// all the code that used to be after cubes = generateCubes
});
I can turn that into something promise based but that only adds to the amount of boilerplate.
In either case I suppose I could write a generic version
function generateThings(factory, num, callback) {
var maxPerIteration = 100000;
var things = [];
function makeMore() {
var count = Math.min(num, maxPerIteration);
for (var ii = 0; ii < count; ++ii) {
things.push(factory());
}
num -= count;
if (num) {
setTimeout(makeMore, 0);
} else {
callback(things);
}
}
makeMore();
}
In this particular case I'm generating 16 million things which is a kind of iteration. Maybe next I want to iterate over those things.
function forEachAllThThings(things, op, callback) {
var maxPerIteration = 100000;
var num = things.length;
function doMore() {
var count = Math.min(num, maxPerIteration);
for (var ii = 0; ii < count; ++ii) {
op(things[ii]);
}
num -= count;
if (num) {
setTimeout(makeMore, 0);
} else {
callback();
}
}
doMore();
}
Is there some more ES6 way of doing this that is more concise or more generic?
NOTE: Please don't get hung up on generating cubes. That's not the question. Also it's not just about the timeout issue, it can also be a jank issue. For example I once worked in a project that needed to deserialize a scene graph. A moderately complicated graph might take 5-10 seconds to deserialize (turn into objects). During those 5-10 seconds the browser was frozen.
The solution was similar to forEachAllTheThings above in that we only read through N objects per tick so as not to lock up the browser. It was all custom code. I'm just wondering if some of the new ES6 features provide any kind of simplification of solving the issue of doing lots of work over multiple ticks the same way they seem to simplify async code (as this is in a sense a form of async code)
Update
Based on #Bergi's suggestion of promisifying setTimeout I think this is what was being suggested.
// returns a Promise that resolves in `time` millisecond
function sleep(time) {
return new Promise(function(resolve, reject) {
setTimeout(resolve, time);
});
}
// returns a promise that resolves to an array of things
function generateThings(factory, num) {
var maxPerIteration = 100000;
var things = [];
function makeMore() {
var count = Math.min(num, maxPerIteration);
for (var ii = 0; ii < count; ++ii) {
things.push(factory());
}
num -= count;
return num ? sleep(0).then(makeMore) : things;
}
// we need to start off with one promise
// incase num <= maxPerIteration
return Promise.resolve(makeMore());
}
function generateCube() {
return Math.random(); // could be anything
}
generateThings(generateCube, 300000)
.then(function(things) {
console.log(things.length);
});
I suppose that is slightly ES6ified and a couple of lines smaller assuming you already have sleep in your code (which seems like a reasonable assumption).

I'd probably offload the generation of the cubes to a web worker, which won't have the timeout problem, assuming that the cubes consist only of JavaScript basic types and so could be posted to the main UI thread when ready. Ideally, the cubes would be transferrable objects so you wouldn't have to clone them, but rather transfer them, from the worker thread to the main UI thread.

Javascript pattern to break up long-running recursive function

I have a long-running function that does a huge calculation: all the possible permutations of x n-sided dice and the probability of those outcomes. For small x and n, the calculation is quick. For large values (n = 100, x > 3), the calculation takes tens of seconds if not longer; meanwhile, the browser stalls.
Here's a snippet of my code:
let dist = [];
// min and max are the minimum and maximum possible values
// (if dice are all 1's or all their maximum values)
for (let i = min; i <= max; i++) {
// initialize possible values of our distribution to 0
dist.push([ i, 0 ]);
}
// total is the total outcome value so far
// dIndex is the index into the dice-array (diceList) for the die we're
// currently applying to our total--the die we're "rolling"
function applyDie(total, dIndex) {
if (dIndex === diceList.length) {
dist[total - min][1]++;
permutationsCount++;
return;
}
// diceList is an array of integers representing the number of sides
// for each die (one array element = one die of element-value sides)
for (let i = 1; i <= diceList[dIndex]; i++) {
applyDie(total + i, dIndex + 1);
}
}
// kick off recursive call
applyDie(0, 0);
I want to add two functionalities:
Cancellation
Progress reporting
Cancellation will be easy (I can do it myself) once I have the async pattern in place, so I really only need help with progress reporting, or, more accurately, simply breaking the recursive pattern into chunks based on the permutationsCount variable. i.e.
/* ... */
permutationsCount++;
if (permutationsCount % chunkSize === 0)
/* end this chunk and start a new one */
I would prefer to use Javasciprt Promises, but I'm open to other suggestions.
Ideas?

Here's a function I wrote to do something similar. It's for a calculation done entirely in javascript... I couldn't tell from your question whether you were working entirely on the client side or what.
// Break the list up into equal-sized chunks, applying f() to each item of
// the list, writing a %-complete value to the progress span after each
// chunk. Then execute a callback with the resulting data.
var chunkedLoop = function (list, f, progressSpan, callback) {
var numChunks = 10,
chunkSize = Math.round(list.length / numChunks),
chunkedList = [], // will be a list of lists
// Concatenated results of all chunks, passed to the callback.
resultList = [],
x, // just a loop variable
chunkIndex = 0; // tracks the current chunk across timeouts
progressSpan.html(0);
// Splice of chunks of equal size, but allow the last one to be of an
// arbitrary size, in case numChunks doesn't divide the length of the
// list evenly.
for (x = 0; x < numChunks - 1; x += 1) {
chunkedList.push(list.splice(0, chunkSize));
}
chunkedList.push(list);
// Schedule a series of timeouts, one for each chunk. The browser
// stops blocking for a moment between each chunk, allowing the screen
// to update. This is the only way to have progress values reported to
// the view mid-loop. If it was done in a single loop, execution would
// block all the way to the end, and the screen would only update once
// at 100%.
var chunkFunction = function () {
setTimeout(function () {
// Run f on the chunk.
var chunk = chunkedList[chunkIndex];
var r = forEach(chunk, f);
resultList = resultList.concat(r);
chunkIndex += 1;
// Update progress on the screen.
progressSpan.html(Math.round(chunkIndex / numChunks * 100));
// Schedule the next run, if this isn't the last chunk. If it
// is the last chunk, execute the callback with the results.
if (chunkIndex < chunkedList.length) {
chunkFunction();
} else if (callback instanceof Function) {
callback.call(undefined, resultList);
}
// There's no reason to delay more than the minimum one
// millisecond, since the point is just to break up javascript's
// single-threaded blocking.
}, 1);
};
chunkFunction();
};

For reporting status you can pass callback function into your recursuve function and do there whatever you like (increase counter, push status into page etc).
Also think about rewriting recursive into iterative algorithm because it will have lesser memory overhead and it will be much easier to put there some other logic (like cancellations you mentioned)

You could use setTimeout to let JavaScript do other things and unstuck the event loop. This way even infinite loop would nonblocking. Here is a quick dirty example.
http://jsfiddle.net/xx5adut6/
function isPrime(n) {
// If n is less than 2 or not an integer then by definition cannot be prime.
if (n < 2) {
return false
}
if (n != Math.round(n)) {
return false
}
var isPrime = true;
for (var i = 2; i <= Math.sqrt(n); i++) {
if (n % i == 0) {
isPrime = false
}
}
// Finally return whether n is prime or not.
return isPrime;
}
var cancel = false;
var i = 0;
var primesFound = 0;
var status = $('.status');
var timeout;
function loop4Primes() {
if (cancel) {
clearTimeout(timeout);
return;
}
if (isPrime(i++)) primesFound++;
timeout = setTimeout(loop4Primes, 1);
}
function updateText() {
status.text(primesFound);
}
var interval = setInterval(updateText, 1000);
$('#start').click(function () {
loop4Primes();
$(this).off('click');
});
$('#stop').click(function () {
clearInterval(interval);
updateText();
cancel = true;
});

Same code takes longer if executed more often?

I've got the following code inside a <script> tag on a webpage with nothing else on it. I'm afraid I do not presently have it online. As you can see, it adds up all primes under two million, in two different ways, and calculates how long it took on average. The variable howOften is used to do this a number of times so you can average it out. What puzzles me is, for howOften == 1, method 2 is faster, but for howOften == 10, method 1 is. The difference is significant and holds even if you hit F5 a couple of times.
My question is simply: how come?
(This post has been edited to incorporate alf's suggestion. But that made no difference! I'm very much puzzled now.)
(Edited again: with howOften at or over 1000, the times seem stable. Alf's answer seems correct.)
function methodOne(maxN) {
var sum, primes_inv, i, j;
sum = 0;
primes_inv = [];
for ( var i = 2; i < maxN; ++i ) {
if ( primes_inv[i] == undefined ) {
sum += i;
for ( var j = i; j < maxN; j += i ) {
primes_inv[j] = true;
}
}
}
return sum;
}
function methodTwo(maxN) {
var i, j, p, sum, ps, n;
n = ((maxN - 2) / 2);
sum = n * (n + 2);
ps = [];
for(i = 1; i <= n; i++) {
for(j = i; j <= n; j++) {
p = i + j + 2 * i * j;
if(p <= n) {
if(ps[p] == undefined) {
sum -= p * 2 + 1;
ps[p] = true;
}
}
else {
break;
}
}
}
return sum + 2;
}
// ---------- parameters
var howOften = 10;
var maxN = 10000;
console.log('iterations: ', howOften);
console.log('maxN: ', maxN);
// ---------- dry runs for warm-up
for( i = 0; i < 1000; i++ ) {
sum = methodOne(maxN);
sum = methodTwo(maxN);
}
// ---------- method one
var start = (new Date).getTime();
for( i = 0; i < howOften; i++ )
sum = methodOne(maxN);
var stop = (new Date).getTime();
console.log('methodOne: ', (stop - start) / howOften);
// ---------- method two
for( i = 0; i < howOften; i++ )
sum = methodTwo(maxN);
var stop2 = (new Date).getTime();
console.log('methodTwo: ', (stop2 - stop) / howOften);

Well, JS runtime is an optimized JIT compiler. Which means that for a while, your code is interpreted (tint), after that, it gets compiled (tjit), and finally you run a compiled code (trun).
Now what you calculate is most probably (tint+tjit+trun)/N. Given that the only part depending almost-linearly on N is trun, this comparison soes not make much sense, unfortunately.
So the answer is, I don't know. To have a proper answer,
Extract the code you are trying to profile into functions
Run warm-up cycles on these functions, and do not use timing from the warm-up cycles
Run much more than 1..10 times, both for warm-up and measurement
Try swapping the order in which you measure time for algorithms
Get into JS interpretator internals if you can and make sure you understand what happens: do you really measure what you think you do? Is JIT run during the warm-up cycles and not while you measure? Etc., etc.
Update: note also that for 1 cycle, you get run time less than the resolution of the system timer, which means the mistake is probably bigger than the actual values you compare.

methodTwo simply requires that the processor execute fewer calculations. In methodOne your initial for loop is executing maxN times. In methodTwo your initial for loop is executing (maxN -2)/2 times. So in the second method the processor is doing less than half the number of calculations that the first method is doing. This is compounded by the fact that each method contains a nested for loop. So big-O of methodOne is maxN^2. Whereas big-O of methodTwo is ((maxN -2)/2)^2.