Develop Reference

Efficient way to compute the median of an array of canvas in JavaScript

I have an array of N HTMLCanvasElements that come from N frames of a video, and I want to compute the "median canvas" in the sense that every component (r, g, b, opacity) of every pixel is the median of the corresponding component in all the canvases.
The video frames are 1280x720, so that the pixels data for every canvas (obtained with canvas.getContext('2d').getImageData(0, 0, canvas.width, canvas.height).data) is a Uint8ClampedArray of length 3.686.400.
The naive way to compute the median is to:
prepare a result Uint8ClampedArray of length 3.686.400
prepare a temporary Uint8ClampedArray of length N
loop from 0 to 3.686.399
a) loop over the N canvases to fill the array
b) compute the median of the array
c) store the median to the result array
But it's very slow, even for 4 canvases.
Is there an efficient way (or existing code) to do that? My question is very similar to Find median of list of images, but I need to to this in JavaScript, not Python.
Note: for b), I use d3.median() which doesn't work on typed arrays, as far as I understand, so that it implies converting to numbers, then converting back to Uint8Clamped.
Note 2: I don't know much of GLSL shaders, but maybe using the GPU would be a way to get faster results. It would require to pass data from the CPU to the GPU though, which takes time if done repeatedly.
Note 3: the naive solution is there: https://observablehq.com/#severo/compute-the-approximate-median-image-of-a-video

You wrote
I use d3.median() which doesn't work on typed arrays…
Although that is not exactly true it leads into the right direction. Internally d3.median() uses the d3.quantile() method which starts off like this:
export default function quantile(values, p, valueof) {
values = Float64Array.from(numbers(values, valueof));
As you can see, this in fact does make use of typed arrays, it is just not your Uint8ClampedArray but a Float64Array instead. Because floating-point arithmetic is much more computation-intensive than its integer counterpart (including the conversion itself) this has a dramatic effect on the performance of your code. Doing this some 3 million times in a tight loop kills the efficiency of your solution.
Since you are retrieving all your pixel values from a Uint8ClampedArray you can be sure that you are always dealing with integers, though. That said, it is fairly easy to build a custom function median(values) derived from d3.median() and d3.quantile():
function median(values) {
// No conversion to floating point values needed.
if (!(n = values.length)) return;
if (n < 2) return d3.min(values);
var n,
i = (n - 1) * 0.5,
i0 = Math.floor(i),
value0 = d3.max(d3.quickselect(values, i0).subarray(0, i0 + 1)),
value1 = d3.min(values.subarray(i0 + 1));
return value0 + (value1 - value0) * (i - i0);
}
On top of getting rid of the problematic conversion on the first line this implementation additionally applies some more micro-optimizations because in your case you are always looking for the 2-quantile (i.e. the median). That might not seem much at first, but doing this multiple million times in a loop it does make a difference.
With minimal changes to your own code you can call it like this:
// medianImageData.data[i] = d3.median(arr); Instead of this use line below.
medianImageData.data[i] = median(arr);
Have a look at my working fork of your Observable notebook.

3D Grid for multiple shapes

A few months ago I made a small terrain generator, like Minecraft, for a school project.
The way I did this was by using multiple chunks. Each chunk contained a 3-dimensional array that stored the blocks.
Every position in this array corresponded with the position of the block it contained.
blocks[x, y, z] = new Block();
Now I would like to add different sizes if blocks. However, I can't do that with the way I am storing the blocks right now, because bigger blocks would have to be spread over multiple positions in the 3-dimensional array.
An example of a game with different sizes of blocks (and different shapes) is LEGO Worlds. How does a game like this store all these little blocks?
I hope someone can help me with this.
The language I am using is Javascript in combination with WebGL.
Thanks in advance!

In my experience there are a few different ways of tackling an issue like this, but the one I'd recommend would depend on the amount of time you have to work on this and the scope (how big) you wanted to make this game.
Your Current Approach
At the moment I think your using what most people would consider the most straightforward approach by storing the voxels in a 3D grid
[Source].
But two problems you seem to be having is that there isn't an obvious way to create blocks that are bigger then 1x1 and that a 3D grid for a world space is fairly inefficient in terms of memory usage (As for an array you have to have memory allocated for every cell, including empty space. JavaScript is no different).
An Alternative Approach
An alternative to using a 3D array would be to instead use a different data structure, the full name being a sparse voxel octree.
This to put it simply is a tree data structure that works by subdividing an area of space until everything has been stored.
The 2D form of this where a square sub divides into four smaller quadrants is called a quad tree and likewise a 3D equivalent divides into eight quadrants, called an octree. This approach is generally preferable when possible as its much more efficient because the trees only occupy more memory when its absolutely essential and they can also be packed into a 1D array (Technically a 3D array can be too).
A common tactic used with quad/octrees in some block based games is to take a region of the same kind of voxel that fit into one larger quadrant of the tree is to simply stop sub division there, as there's no reason to go deeper if all the data is the same.
The other optimization they can make is called sparse where regions of empty space (air) are simply deleted since empty space doesn't do anything special and its location can be inferred.
[SVO Source]
[Z Order Curve Source]
Recommended Approach
Unless you have a few months to complete your game and you're at university I seriously wouldn't recommend an SVO (Though reading up about could impress any teachers you have). Instead I'd recommend taking the same approach that Minecraft appears to visibly has. E.G. A door is 1X2 but blocks can only be 1x1, then just make it two blocks.
In the example of a door you would have four unique blocks in total, two for the upper and lower half, and two variations of each being opened or closed.
E.G.
var cubeProgram; // shader program
var cubeVBO; // vertex buffer (I recommend combining vertex & UV coords)
var gl; // rendering context
// Preset list of block ID's
var BLOCK_TYPES = {
DOOR_LOWER_OPEN: 0,
DOOR_UPPER_OPEN: 1,
DOOR_LOWER_CLOSED: 2,
DOOR_UPPER_CLOSED: 3,
}
var BLOCK_MESHES = {
GENERIC_VBO: null,
DOOR_UPPER_VBO: null
DOOR_LOWER_VBO: null
}
// Declare a Door class using ES6 syntax
class Door {
// Assume X & Y are the lower half of the door
constructor(x,y,map) {
if (y - 1 > -1) {
console.error("Error: Top half of the door goes outside the map");
return;
}
this.x = x;
this.y = y;
map[x][y ] = BLOCK_TYPES.DOOR_LOWER_OPEN;
map[x][y-1] = BLOCK_TYPES.DOOR_UPPER_OPEN;
}
}

JS Canvas get pixel value very frequently

I am creating a video game based on Node.js/WebGL/Canvas/PIXI.js.
In this game, blocks have a generic size: they can be circles, polygons, or everything. So, my physical engine needs to know where exactly the things are, what pixels are walls and what pixels are not. Since I think PIXI don't allow this, I create an invisible canvas where I put all the wall's images of the map. Then, I use the function getImageData to create a function "isWall" at (x, y):
function isWall(x, y):
return canvas.getImageData(x, y, 1, 1).data[3] != 0;
However, this is very slow (it takes up to 70% of the CPU time of the game, according to Chrome profiling). Also, since I introduced this function, I sometimes got the error "Oops, WebGL crashed" without any additional advice.
Is there a better method to access the value of the pixel? I thought about storing everything in a static bit array (walls have a fixed size), with 1 corresponding to a wall and 0 to a non-wall. Is it reasonable to have a 10-million-cells array in memory?

Some thoughts:
For first check: Use collision regions for all of your objects. The regions can even be defined for each side depending on shape (ie. complex shapes). Only check for collisions inside intersecting regions.
Use half resolution for hit-test bitmaps (or even 25% if your scenario allow). Our brains are not capable of detecting pixel-accurate collisions when things are moving so this can be taken advantage of.
For complex shapes, pre-store the whole bitmap for it (based on its region(s)) but transform it to a single value typed array like Uint8Array with high and low values (re-use this instead of getting one and one pixels via the context). Subtract object's position and use the result as a delta for your shape region, then hit-testing the "bitmap". If the shape rotates, transform incoming check points accordingly (there is probably a sweet-spot here where updating bitmap becomes faster than transforming a bunch of points etc. You need to test for your scenario).
For close-to-square shaped objects do a compromise and use a simple rectangle check
For circles and ellipses use un-squared values to check distances for radius.
In some cases you can perhaps use collision predictions which you calculate before the games starts and when knowing all objects positions, directions and velocities (calculate the complete motion path, find intersections for those paths, calculate time/distance to those intersections). If your objects change direction etc. due to other events during their path, this will of course not work so well (or try and see if re-calculating is beneficial or not).
I'm sure why you would need 10m stored in memory, it's doable though - but you will need to use something like a quad-tree and split the array up, so it becomes efficient to look up a pixel state. IMO you will only need to store "bits" for the complex shapes, and you can limit it further by defining multiple regions per shape. For simpler shapes just use vectors (rectangles, radius/distance). Do performance tests often to find the right balance.
In any case - these sort of things has to be hand-optimized for the very scenario, so this is just a general take on it. Other factors will affect the approach such as high velocities, rotation, reflection etc. and it will quickly become very broad. Hope this gives some input though.

I use bit arrays to store 0 || 1 info and it works very well.
The information is stored compactly and gets/sets are very fast.
Here is the bit library I use:
https://github.com/drslump/Bits-js/blob/master/lib/Bits.js
I've not tried with 10m bits so you'll have to try it on your own dataset.
The solution you propose is very "flat", meaning each pixel must have a corresponding bit. This results in a large amount of memory being required--even if information is stored as bits.
An alternative testing data ranges instead of testing each pixel:
If the number of wall pixels is small versus the total number of pixels you might try storing each wall as a series of "runs". For example, a wall run might be stored in an object like this (warning: untested code!):
// an object containing all horizontal wall runs
var xRuns={}
// an object containing all vertical wall runs
var yRuns={}
// define a wall that runs on y=50 from x=100 to x=185
// and then runs on x=185 from y=50 to y=225
var y=50;
var x=185;
if(!xRuns[y]){ xRuns[y]=[]; }
xRuns[y].push({start:100,end:185});
if(!yRuns[x]){ yRuns[x]=[]; }
yRuns[x].push({start:50,end:225});
Then you can quickly test an [x,y] against the wall runs like this (warning untested code!):
function isWall(x,y){
if(xRuns[y]){
var a=xRuns[y];
var i=a.length;
do while(i--){
var run=a[i];
if(x>=run.start && x<=run.end){return(true);}
}
}
if(yRuns[x]){
var a=yRuns[x];
var i=a.length;
do while(i--){
var run=a[i];
if(y>=run.start && y<=run.end){return(true);}
}
}
return(false);
}
This should require very few tests because the x & y exactly specify which array of xRuns and yRuns need to be tested.
It may (or may not) be faster than testing the "flat" model because there is overhead getting to the specified element of the flat model. You'd have to perf test using both methods.
The wall-run method would likely require much less memory.
Hope this helps...Keep in mind the wall-run alternative is just off the top of my head and probably requires tweaking ;-)

multiple DIV collision detection in Javascript/JQuery

Working on a little "zombies" or "tag you're it" or "ew! you got cooties"-styled game where each AI object (a person, basically) runs around randomly. There is an initial object that is "it" or "infected" and as it moves about the screen and touches/overlaps/collides with another object it should change the touched object to the same color as the object that touched it. Newly infected objects can continue to infect other objects they randomly collide with, until - in principle - the whole population is the same color as the first infected object. (I'll worry about fancier AI where infected actively hunt nearby objects or healthy objects can avoid infected objects, later).
But after looking at various similar questions in StackOverflow that generally deal with 2 DIVs colliding, or use some sort of jQuery draggable detection trick, I'm still at a bit of a loss as to how to build upon those ideas to scale up a simple "if I am touching/overlapping/colliding with another object it should get infected too" that can be applied to a large number of elements on the page, say... less than 100 so as not to drag the browser down.
I basically get as far as determining position and widths/heights of the objects so that I know how much space they take, but then the brain goes 'bzzzzt' when trying to develop a function that checks over all the population for collisions.
Got the population moving around randomly without trouble - see JSFiddle https://jsfiddle.net/digitalmouse/5tvyjhjL/1/ for the related code. Affected function should be in the 'animateDiv()', seen below to make the stackoverflow question asking editor happy that I included some code in my question. :)
function animateDiv($target) {
var newq = makeNewPosition($target.parent());
var oldq = $target.offset();
var speed = calcSpeed([oldq.top, oldq.left], newq);
// I believe collision should be dealt with here,
// just before moving an object
$target.animate({
top: newq[0],
left: newq[1]
}, speed, function () {
animateDiv($target);
});
}
Any hints, tricks, adaptations, or code snippets that push me in the right direction are appreciated.

a quick, down and dirty solution (there are more complex algorithms) would be to use:
document.elementFromPoint(x, y);
It gets the element at the position specified. The full spec can be found here.
Assuming your 'zombies' are rectangular, you could call this for each corner, and if you get a hit, that isn't the background or the element you're checking, you've got a collision...
EDIT:
An alternate method, even 'downer and dirtier' than above, but stupidly quick, would be to get the centre points of the two objects to check, then find their absolute displacements in X and Y. If the differences are less than the sum of half their widths and heights then they are overlapping. It's by no means pix perfect, but it should be able to handle a large number objects really quickly.
EDIT 2:
First off, we need to get the centres of each object (to check)
// Values for main object
// pop these in vars as we'll need them again in a sec...
hw = object.style.width >> 1; // half width of object
hh = object.style.height >> 1; // (bit shift is faster than / 2)
cx = object.style.left + hw; // centre point in x
cy = object.style.top + hh; // and in y
// repeat for secondary object
If you don't know / store the width and height you can use:
object.getBoundingClientRect();
which returns a 'rect' object with the fields left, top, right and bottom.
Now we check proximity...
xDif = Math.abs(cx - cx1); // where cx1 is centre of object to check against
if(xDif > hw + hw1) return false; // there is no possibility of a collision!
// if we get here, there's a possible collision, so...
yDif = Math.abs(cy - cy1);
if(yDif > hh + hh1) return false; // no collision - bug out.
else {
// handle collision here...
}
Danny

How can I stop elements overlapping using JavaScript and the Raphael JavaScript library

I’m generating multiple, random sized, circular elements using the Raphael JavaScript library but because it’s random a lot of the circular elements being generate overlap or cover each other. What I wanted to know, is there any way with JavaScript to tell if one element is in already in particular position so to avoid the overlapping? Essentially, I want to create random elements on a canvas, of a random size that don’t overlap or cover each other.
There's a couple of test files I created here to give you an idea of what I'm doing. The first one generates random objects and the second link sets them to a grid to stop the overlapping.
http://files.nicklowman.co.uk/movies/raphael_test_01/
http://files.nicklowman.co.uk/movies/raphael_test_03/

The easiest way is to create an object and give it a repulsive force that degrades towards zero at it's edge. As you drop these objects onto the canvas the objects will push away from each other until they reach a point of equilibrium.

Your examples aren't working for me, so I cannot visualize your exact scenario.
Before you "drop" an element on the canvas, you could query the positions of your other elements and do some calculations to check if the new element will overlap.
A very simple example of this concept using circle elements might look like this:
function overlap(circ1, circ2) {
var attrs = ["cx", "cy", "r"];
var c1 = circ1.attr(attrs);
var c2 = circ2.attr(attrs);
var dist = Math.sqrt(Math.pow(c1.cx - c2.cx ,2) + Math.pow(c1.cy - c2.cy, 2));
return (dist < (c1.r + c2.r));
}
var next_drop = paper.circle(x, y, r);
for (var i in circles) {
if (overlap(next_drop, circles[i])) {
// do something
}
}
Of course calculating just where you're going to place a circle after you've determined it overlaps with others is a little more complicated.