Develop Reference

'Point-along-path' d3 visualization performance issue

I have gone through the 'point-along-path' d3 visualization through the code: https://bl.ocks.org/mbostock/1705868. I have noticed that while the point is moving along its path it consumes 7 to 11% of CPU Usage.
Current scenario, I have around 100 paths and on each path, I will have to move points(circles) from sources to destinations. So it consumes more than 90% of the CPU memory as more number of points are moving at the same time.
I have tried as:
function translateAlong(path) {
var l = path.getTotalLength();
return function(d, i, a) {
return function(t) {
var p = path.getPointAtLength(t * l);
return "translate(" + p.x + "," + p.y + ")";
};
};
}
// On each path(100 paths), we are moving circles from source to destination.
var movingCircle = mapNode.append('circle')
.attr('r', 2)
.attr('fill', 'white')
movingCircle.transition()
.duration(10000)
.ease("linear")
.attrTween("transform", translateAlong(path.node()))
.each("end", function() {
this.remove();
});
So what should be the better way to reduce the CPU usage?
Thanks.

There are a few approaches to this, which vary greatly in potential efficacy.
Ultimately, you are conducting expensive operations every animation frame to calculate each point's new location and to re render it. So, every effort should be made to reduce the cost of those operations.
If frame rate is dropping below 60, it probably means we're nearing CPU capacity. I've used frame rate below to help indicate CPU capacity as it is more easily measured than CPU usage (and probably less invasive).
I had all sorts of charts and theory for this approach, but once typed it seemed like it should be intuitive and I didn't want to dwell on it.
Essentially the goal is to maximize how many transitions I can show at 60 frames per second - this way I can scale back the number of transitions and gain CPU capacity.
Ok, let's get some transitions running with more than 100 nodes along more than 100 paths at 60 frames per second.
D3v4
First, d3v4 likely offers some benefits here. v4 synchronized transitions, which appears to have had the effect of slightly improved times. d3.transition is very effective and low cost in any event, so this isn't the most useful - but upgrading isn't a bad idea.
There are also minor browser specific gains to be had by using different shaped nodes, positioning by transform or by cx,cy etc. I didn't implement any of those because the gains are relatively trivial.
Canvas
Second, SVG just can't move fast enough. Manipulating the DOM takes time, additional elements slows down operations and takes up more memory. I realize canvas can be less convenient from a coding perspective but canvas is faster than SVG for this sort of task. Use detached circle elements to represent each node (the same as with the paths), and transition these.
Save more time by drawing two canvases: one to draw once and to hold the paths (if needed) and another to be redrawn each frame showing the points. Save further time by setting the datum of each circle to the length of the path it is on: no need to call path.getTotalLength() each time.
Maybe something like this
Canvas Simplified Lines
Third, we still have a detached node that has SVG paths so we can use path.getPointAtLength() - and this is actually pretty effective. A major point slowing this down though is the use of curved lines. If you can do it, draw straight lines (multiple segments are fine) - the difference is substantial.
As a further bonus, use context.fillRect() instead of context.arc()
Pure JS and Canvas
Lastly, D3 and the detached nodes for each path (so we can use path.getTotalLength()) can start to get in the way. If need be leave them behind using typed arrays, context.imageData, and your own formula for positioning nodes on paths. Here's a quick bare bones example (100 000 nodes, 500 000 nodes, 1 000 000 nodes (Chrome is best for this, possible browser limitations. Since the paths now essentially color the entire canvas a solid color I don't show them but the nodes follow them still). These can transition 700 000 nodes at 10 frames per second on my slow system. Compare those 7 million transition positioning calculations and renderings/second against about 7 thousand transition positioning calculations and renderings/second I got with d3v3 and SVG (three orders of magnitude difference):
canvas A is with curved lines (cardinal) and circle markers (link above), canvas B is with straight (multi-segment) lines and square markers.
As you might imagine, a machine and script that can render 1000 transitioning nodes at 60 frames per second will have a fair bit of extra capacity if only rendering 100 nodes.
If the transition position and rendering calculations are the primary activity and CPU usage is at 100%, then half the nodes should free up roughly half the CPU capacity. In the slowest canvas example above, my machine logged 200 nodes transitioning along cardinal curves at 60 frames per second (it then started to drop off, indicating that CPU capacity was limiting frame rate and consequently usage should be near 100%), with 100 nodes we have a pleasant ~50% CPU usage:
Horizontal centerline is 50% CPU usage, transition repeated 6 times
But the key savings are to be found from dropping complex cardinal curves - if possible use straight lines. The other key savings are from customizing your scripts to be purpose built.
Compare the above with straight lines (multi segment still) and square nodes:
Again, horizontal centerline is 50% CPU usage, transition repeated 6 times
The above is 1000 transitioning nodes on 1000 3 segment paths - more than an order of magnitude better than with curved lines and circular markers.
Other Options
These can be combined with methods above.
Don't animate every point each tick
If you can't position all nodes each transition tick before the next animation frame you'll be using close to all of your CPU capacity. One option is don't position each node each tick - you don't have to. This is a complex solution - but position one third of circles each tick - each circle still can be positioned 20 frames per second (pretty smooth), but the amount of calculations per frame are 1/3 of what they would be otherwise. For canvas you still have to render each node - but you could skip calculating the position for two thirds of the nodes. For SVG this is a bit easier as you could modify d3-transition to include an every() method that sets how many ticks pass before transition values are re-calculated (so that one third are transitioned each tick).
Caching
Depending on circumstance, caching is also not a bad idea - but the front-ending of all calculations (or loading of data) may lead to unnecessary delays in the commencement of animation - or slowness on first run. This approach did lead to positive outcomes for me, but is discussed in another answer so I won't go into it here.

Post edit:
Here is the default. (peak CPU around %99 for 100 points at 2.7Ghz i7)
Here is my version. (peak CPU around 20% for 100 points at 2.7Ghz i7)
On average I am 5 times faster.
I presume the bottleneck here is the call to getPointAtLength method at every 17ms. I would also avoid lengthy string concatenations if I have to, but in your case its not much long so I think the best way:
cache points beforehand and only calculate once with a given resolution (I divided here in 1000 parts)
reduce calls to DOM methods within the requestAnimationFrame (that function you see receiving the normalized t parameter)
In the default case there is 2 calls, 1 when you call getPointAtLength, and then another one when you are setting the translate(under the hood).
You can replace the translateAlong with this below:
function translateAlong(path){
var points = path.__points || collectPoints(path);
return function (d,i){
var transformObj = this.transform.baseVal[0];
return function(t){
var index = t * 1000 | 0,
point = points[index];
transformObj.setTranslate(point[0],point[1]);
}
}
}
function collectPoints(path) {
var l = path.getTotalLength(),
step = l*1e-3,
points = [];
for(var i = 0,p;i<=1000;++i){
p = path.getPointAtLength(i*step);
points.push([p.x,p.y]);
}
return path.__points = points;
}
And a small modification to that tweening line:
.tween("transform", translateAlong(path.node()))
setting attr is not necessary, calling it is enough. Here is the result:
http://jsfiddle.net/ibowankenobi/8kx04y29/
Tell me if it did improve, because I'm not 100% sure.

Another way of achieving this might be to use an svg:animateMotion which you can use to move an element along a given path. Here is an example from the docs. In essence you want:
<svg>
<path id="path1" d="...">
<circle cx="" cy="" r="10" fill="red">
<animateMotion dur="10s" repeatCount="0">
<mpath xlink:href="#path1" />
</animateMotion>
</circle>
</path>
</svg>
I've not profiled it, but I think you're struggle to get much better performance than using something built into SVG itself.
Browser Support
Note that after a comment from #ibrahimtanyalcin I started checking browser compatibility. Turns out that this isn't supported in IE or Microsoft Edge.

On my computer:
#mbostock uses 8% CPU.
#ibrahimtanyalcin uses 11% CPU.
#Ian uses 10% CPU.
For #mbostock and #ibrahimtanyalcin this means the transition and the transform update use the CPU.
If I put 100 of these animations in 1 SVG I get
#mbostock uses 50% CPU. (1 core full)
#Ian uses 40% CPU.
All animations look smooth.
One possibility is to add a sleep in the transform update function using https://stackoverflow.com/a/39914235/9938317
Edit
In the nice Andrew Reid answer i have found a few optimizations.
I have written a version of the Canvas+JS 100 000 test that only does the calculation part and counts how many iterations can be do in 4000ms. Half the time order=false as t controls it.
I have written my own random generator to be sure that I get the same random numbers each time I run the modified code.
The blocks version of the code: 200 iterations
According to the docs for parseInt
parseInt should not be used as a substitute for Math.floor()
Converting a number to a string and then parsing the string up to the . sounds not very efficient.
Replacing parseInt() with Math.floor(): 218 iterations
I also found a line that has no function and looks not important
let p = new Int16Array(2);
It is inside the inner while loop.
Replacing this line with
let p;
gives 300 iterations.
Using these modifications the code can handle more points with a frame rate of 60 Hz.
I tried a few other things but they turn out to be slower.
I was surprised that if I pre-calculate the lengths of the segments and simplify the calculation of segment to a mere array lookup it is slower than doing the 4 array lookups and a couple of Math.pow and additions and a Math.sqrt.

JS Canvas get pixel value very frequently

I am creating a video game based on Node.js/WebGL/Canvas/PIXI.js.
In this game, blocks have a generic size: they can be circles, polygons, or everything. So, my physical engine needs to know where exactly the things are, what pixels are walls and what pixels are not. Since I think PIXI don't allow this, I create an invisible canvas where I put all the wall's images of the map. Then, I use the function getImageData to create a function "isWall" at (x, y):
function isWall(x, y):
return canvas.getImageData(x, y, 1, 1).data[3] != 0;
However, this is very slow (it takes up to 70% of the CPU time of the game, according to Chrome profiling). Also, since I introduced this function, I sometimes got the error "Oops, WebGL crashed" without any additional advice.
Is there a better method to access the value of the pixel? I thought about storing everything in a static bit array (walls have a fixed size), with 1 corresponding to a wall and 0 to a non-wall. Is it reasonable to have a 10-million-cells array in memory?

Some thoughts:
For first check: Use collision regions for all of your objects. The regions can even be defined for each side depending on shape (ie. complex shapes). Only check for collisions inside intersecting regions.
Use half resolution for hit-test bitmaps (or even 25% if your scenario allow). Our brains are not capable of detecting pixel-accurate collisions when things are moving so this can be taken advantage of.
For complex shapes, pre-store the whole bitmap for it (based on its region(s)) but transform it to a single value typed array like Uint8Array with high and low values (re-use this instead of getting one and one pixels via the context). Subtract object's position and use the result as a delta for your shape region, then hit-testing the "bitmap". If the shape rotates, transform incoming check points accordingly (there is probably a sweet-spot here where updating bitmap becomes faster than transforming a bunch of points etc. You need to test for your scenario).
For close-to-square shaped objects do a compromise and use a simple rectangle check
For circles and ellipses use un-squared values to check distances for radius.
In some cases you can perhaps use collision predictions which you calculate before the games starts and when knowing all objects positions, directions and velocities (calculate the complete motion path, find intersections for those paths, calculate time/distance to those intersections). If your objects change direction etc. due to other events during their path, this will of course not work so well (or try and see if re-calculating is beneficial or not).
I'm sure why you would need 10m stored in memory, it's doable though - but you will need to use something like a quad-tree and split the array up, so it becomes efficient to look up a pixel state. IMO you will only need to store "bits" for the complex shapes, and you can limit it further by defining multiple regions per shape. For simpler shapes just use vectors (rectangles, radius/distance). Do performance tests often to find the right balance.
In any case - these sort of things has to be hand-optimized for the very scenario, so this is just a general take on it. Other factors will affect the approach such as high velocities, rotation, reflection etc. and it will quickly become very broad. Hope this gives some input though.

I use bit arrays to store 0 || 1 info and it works very well.
The information is stored compactly and gets/sets are very fast.
Here is the bit library I use:
https://github.com/drslump/Bits-js/blob/master/lib/Bits.js
I've not tried with 10m bits so you'll have to try it on your own dataset.
The solution you propose is very "flat", meaning each pixel must have a corresponding bit. This results in a large amount of memory being required--even if information is stored as bits.
An alternative testing data ranges instead of testing each pixel:
If the number of wall pixels is small versus the total number of pixels you might try storing each wall as a series of "runs". For example, a wall run might be stored in an object like this (warning: untested code!):
// an object containing all horizontal wall runs
var xRuns={}
// an object containing all vertical wall runs
var yRuns={}
// define a wall that runs on y=50 from x=100 to x=185
// and then runs on x=185 from y=50 to y=225
var y=50;
var x=185;
if(!xRuns[y]){ xRuns[y]=[]; }
xRuns[y].push({start:100,end:185});
if(!yRuns[x]){ yRuns[x]=[]; }
yRuns[x].push({start:50,end:225});
Then you can quickly test an [x,y] against the wall runs like this (warning untested code!):
function isWall(x,y){
if(xRuns[y]){
var a=xRuns[y];
var i=a.length;
do while(i--){
var run=a[i];
if(x>=run.start && x<=run.end){return(true);}
}
}
if(yRuns[x]){
var a=yRuns[x];
var i=a.length;
do while(i--){
var run=a[i];
if(y>=run.start && y<=run.end){return(true);}
}
}
return(false);
}
This should require very few tests because the x & y exactly specify which array of xRuns and yRuns need to be tested.
It may (or may not) be faster than testing the "flat" model because there is overhead getting to the specified element of the flat model. You'd have to perf test using both methods.
The wall-run method would likely require much less memory.
Hope this helps...Keep in mind the wall-run alternative is just off the top of my head and probably requires tweaking ;-)

Dynamically Animating Hue Shift on Canvas Image

I'm still relatively new to working with the canvas tag. What I've done so far is draw an image to the canvas. My goal is to have a fake night/day animation that cycles repeatedly.
I've exhausted quite a few different avenues (SVG, CSS3 filters, etc) and think that canvas pixel manipulation is the best route in my case. I'm trying to:
Loop through all pixels in the image
Select a certain color range
Adjust to new color
Update the canvas
Here's the code I have so far:
function gameLoop(){
requestAnimationFrame(gameLoop);
////////////////////////////////////////////////////////////////
// LOOP PIXEL DATA - PIXEL'S RGBA IS STORED IN SEQUENTIAL ARRAYS
////////////////////////////////////////////////////////////////
for(var i=0; i<data.length; i+=4){
red=data[i+0];
green=data[i+1];
blue=data[i+2];
alpha=data[i+3];
// GET HUE BY CONVERTING TO HSL
var hsl=rgbToHsl(red, green, blue);
var hue=hsl.h*360;
// CHANGE SET COLORRANGE TO NEW COLORSHIFT
if(hue>colorRangeStart && hue<colorRangeEnd){
var newRgb=hslToRgb(hsl.h+colorShift, hsl.s, hsl.l);
data[i+0]=newRgb.r;
data[i+1]=newRgb.g;
data[i+2]=newRgb.b;
data[i+3]=255;
};
};
// UPDATE CANVAS
ctx.putImageData(imgData, 0, 0);
};
The code works and selects a hue ranges and shifts it once, but is incredibly laggy. The canvas dimensions are roughly 500x1024.
My questions:
Is it possible to improve performance?
Is there a better way to perform a defined hue shift animation?
Thanks!

It's hard to do this real-time using high quality HSL conversion. Been there done that, so I came up with a quantized approach which allow you to do this in real-time.
You can find the solution here (GPL3.0 licensed):
https://github.com/epistemex/FastHSL2RGB
Example of usage can be found here (MIT license) incl. demo:
https://github.com/epistemex/HueWheel
Apologies for referencing my own solutions here, but the inner workings (the how to's) is too extensive to present in a simple form here and both of these are free to use for anything..
The key points are in any case:
Quantize the range you want to use (don't use full 360 degrees and not floating points for lightness etc.)
Cache the values in a 3D array (initial setup using web workers or use rough values)
Quantize the input values so they fit in the range of the inner 3D array
Process the bitmap using these values
It is not accurate but good enough for animations (or previews which is what I wrote it for).
There are other techniques such as pre-caching the complete processed bitmap for key positions, then interpolate the colors between those instead. This, of course, requires much more memory but is a fast way.
Hope this helps!

Clearing full canvas faster than just parts?

Noticed something I didn't expect when playing with animation in Canvas. I have a fairly easy animation of five images in different sizes moving from bottom to top of screen in a loop.
I have all my image data in an array and draw them onto the canvas via a loop, then use window.requestAnimationFrame to do a new draw on next frame.
Here is the interesting part, at first I just cleared the canvas for each frame rendered using context.clearRect(0, 0, canvas.width, canvas.height);. Then I though that this must be a waste of computing, to clear the entire canvas for each render even thou only parts of the screen actually changed.
So I rewrote the part clearing the canvas to only clear the trace of the old image draw, using something like this:
for (var key in _images) {
context.clearRect(_images[key].x-1, _images[key].y+_images[key].height, _images[key].width+2, 5);
}
But too my surprise this seems to be slower... First I had a frame rate at 49-60, and after 47-57. Any idea to why? And is there any other way to optimize this?

0) i would rather write :
for (var i=0, len=_images.length; i<len; i++) {
var thisImage = _images[i];
context.clearRect(thisImage.x, thisImage.y, thisImage.width, thisImage.height);
}
1) when requestAnimationFrame triggers, draw before updating, so you are right in sync with the screen.
2) round the coordinates to the nearest 64 bits, i.e. use &(~3) on each x coordinates can speed things up.
3) You might group some clearRect together (= all the bullets / a tiled horizontal floor, ...)
4) i think the main question here is the overhead of those methods.
I did a jsperf to know more :
http://jsperf.com/overhead-of-fillrect-and-clearrect/2
!! be sure to have the canvas in sight during the tests !!
Results :
Firefox, clearRect has very small overhead, and time seems
almost proportionnal with pixel count.
fillRect is like 60% slower and seems quite proportionnal also
with area covered.
64 bit Unalignment seems of no effet on clearRect, it makes performances
fall badly on fillRect.
some times from jsPerf (precision is so-so) :
208ns for fullscreen clearRect
55ns for halfscreen clearRect -> 4 times makes 220ns vs 208
13ns for quarter screen clearRect -> 16 times makes 208 (!!!)
588ns fullscreen fillRect
1290ns halfscreen fillRect --> 4 times makes 1160
41ns quarterscreen fillrect --> 16 times makes 656.
Safari, everything the results are quite the same, except there is
more overhead : it is only 3 times faster to draw 4 times less points.
I know the the precision is so-so but it seems that on firefox
and Safari, the conclusion is :
the less part of your screen you clear, the faster you erase.
And prefer clearRect to fillRect -
Chrome : (date 05 / 14) fillRect is much faster than clearRect.
IE11 : fillRect and clearRect are on par.
((If anyone can see on other Browser, i'll edit -when i have some time-))
( provided your images are 4 pixels aligned, the code with
64 align is :
for (var i=0, len=_images.length; i<len; i++) {
var thisImage = _images[i];
var x = thisImage.x ;
context.clearRect(x & (~3), thisImage.y, thisImage.width + ((x&3)&&(16-(x&3))), thisImage.height);
}
)

Since people answered in comments and I do feel they covered my questions. Here's a summery:
Each clearRect() takes up roughly the same amount of time, so it is better to have few than many.
To optimize one should calculate the area that needs to be cleared, if not 100%, and clear that.

Dominant color in video for each frame

I want to analyze a video (mp4) on flashing of a LED.
The background is usually grey but the color of the LED may vary.
The LED is in the video close enough so the LED-light is the biggest part of the frame.
I found Color-Thief but it's just for images and not for video.
Because the frequency of the flashing may vary too, I need to check each frame of the video for the dominant color.
If anyone has any ideas, I'd greatly appreciate some assistance. Thanks in advance for your time.
EDIT:
Two screenshots of the video (first red LED off; second red LED on) (had to remove the link for color-thief for the two screenshot links)
This is the function which gets called when the video starts playing and should analyze the video. I tried to do it via the average color but that's useless because that's usually some sort of grey/brown.
function processFrame(e) {
var video = document.getElementById("video");
if (video.ended){
//dauer();
console.log("Ende");
//alert("Ende");
}
if (video.paused || video.ended) {
return;
}
var bufferCanvas = document.getElementById("buffer");
var displayCanvas = document.getElementById("display");
var buffer = bufferCanvas.getContext("2d");
var display = displayCanvas.getContext("2d");
buffer.drawImage(video, 0,0, bufferCanvas.width, displayCanvas.height);
var frame = buffer.getImageData(sx, sy, sw, sh);
var length = frame.data.length / 4;
for (var i = 0; i < length; i++) {
var r = frame.data[i * 4 + 0];
var g = frame.data[i * 4 + 1];
var b = frame.data[i * 4 + 2];
average= average +((r+g+b)/3)
}
averagecolor[i]=average
display.putImageData(frame, 0, 0);
setTimeout(processFrame, 0);
}

It's a bit broad what you're asking as you need to do this in several steps - but it's fully possible to do. To shorten down I'll just present the steps you need to take in order to be able to analyze the LEDs.
You can do this fully automatic or semi-automatic be predefining the areas in the video frame.
The steps for automatic detection is as follows and is used once per session in case camera is moved or zoom is changed. Lighting condition can affect the detection as well but here I will assume the lighting conditions are the same and as in the images you are showing. It's also specific for these images and may not work if you change setup of the routers/camera.
Initial step to auto-detect areas:
Scan horizontal lines half way and start registering regions when all RGB values = 255 (that's the white center of the leds). Set a flag that you are in a region and count number of pixels when you start to register based on a threshold (for example 7-10 pixels that needs to be white in a row). This is to eliminate single pixels that way be white.
When you cannot find any white pixels on a line (based on threshold) end region and prepare for new registration.
When at bottom of first half (i would set a max height shorted than actual height), start over scanning from middle to the right edge.
After this you should have six regions registered (use a "calibration" frame where all LEDs are lit - this can be ran manually before running live).
For next step which will be the one running constantly you will use these regions as anchor regions and to sample points only on one side and only a few pixels in height.
Use the region as anchor and define a sample region based on that with an offset x + width of region + 2 pixels as start X (this is just suggestion for an initial offset - you may have to fine-tune this). Offset Y + 10 and set width and height to 10 pixels. This should give you a sample region of 100 points right next to the led.
Run statistics on this small region by using getImageData. You need to check all three components as you need to filter away light-grey. Find a threshold (for instance if r > 230 && b < 200 && g < 200 just to take some values) when kicks in you accumulate your red value (and another for green). This process require a calibration phase to find that threshold value you need to use. Run some tests. If camera changes f-stop you will probably need to re-calibrate.
Accumulate the green and red values to different variables. You do not not need to average them but you will need to find a threshold value for the total sum where you know the led is lit for the LED with that color (use the initial test frame from region detection for this as well). Add some tolerance to the values by comparing the regions when the LEDs are off.
This will give you in total only 600 points to scan maybe less if you can fine-tune it.
For the manual method you skip the region setting as described in first step but use a frame and set the regions manually by measuring pixels of the white top. If you change the camera position you will need to do it over.
As you understand there is a bit code that need to be written for this and there are many factors that will affect the final code making it very specific for this usage. And I believe it's in part outside the scope for SO but I hope this gives you some leads to get you going.