Develop Reference

Modifying in memory canvases before drawing to screen drastically reduces Javascript performance

I've noticed if I have a large number of canvases in memory, modifying each canvas before drawing them to the screen drastically reduces performance on my machine. This occurs even when the canvases are small and the modifications are minor.
Here is the most contrived example I could come up with:
var { canvas, ctx } = generateCanvas();
ctx.strokeStyle = "#000";
var images = [];
for (var i = 0; i < 500; i++) {
images.push(generateCanvas(50, "red"));
}
var fps = 0,
lastFps = new Date().getTime();
requestAnimationFrame(draw);
function draw() {
requestAnimationFrame(draw);
var modRects = document.getElementById("mod-rects").checked;
var drawRects = document.getElementById("draw-rects").checked;
ctx.clearRect(0, 0, 500, 500);
ctx.strokeRect(0, 0, 500, 500);
fps++;
if (new Date().getTime() - lastFps > 1000) {
console.clear();
console.log(fps);
fps = 0;
lastFps = new Date().getTime();
}
images.forEach(img => {
img.ctx.fillStyle = "yellow";
if (modRects) img.ctx.fillRect(20, 20, 10, 10);
if (drawRects) ctx.drawImage(img.canvas, 225, 225);
});
}
function generateCanvas(size = 500, color = "black") {
var canvas = document.createElement("canvas");
canvas.width = canvas.height = size;
var ctx = canvas.getContext("2d");
ctx.fillStyle = color;
ctx.fillRect(0, 0, size, size);
return {
canvas,
ctx
};
}
function generateCheckbox(name) {
var div = document.createElement("div");
var check = document.createElement("input");
check.type = "checkbox";
check.id = name;
var label = document.createElement("label");
label.for = name;
label.innerHTML = name;
div.appendChild(check);
div.appendChild(label);
return div;
}
document.body.appendChild(canvas);
document.body.appendChild(generateCheckbox("mod-rects"));
document.body.appendChild(generateCheckbox("draw-rects"));
canvas+div+div { margin-bottom: 20px; }
In this example we create 500 canvases of size 50x50. There are two checkboxes underneath the larger onscreen canvas. The first causes a small yellow square to be drawn on each of those 500 canvases. The 2nd causes the canvases to be drawn to the larger canvas. FPS is posted to the console once per second. I see no performance issues when one or the other checkbox is checked, but when both are checked, performance drops drastically.
My first thought is that it has something to do with sending in-memory canvas to the gfx card every frame when they are modified.
Here's the actual effect I'm trying to create.
Video: https://youtu.be/Vr6v2oF3G-8
Code: https://github.com/awhipple/base-command-dev/blob/e2c38946cdaf573abff5ded5399c90687ffa76a5/engine/gfx/shapes/Particle.js
My ultimate goal is to be able to smoothly transition the colors of the canvas. I'm using globalCompositeOperation = "source-in" and fillRect() to do this in the code link above.

As has been stated before, this is an issue with the overhead of sending hundreds of canvases to the GPU every single frame. When a canvas is modified in CPU it gets marked as "dirty" and is re sent to the GPU next time it's used.
The workaround I found was to create a large canvas containing a grid of my particle images. Every particle object makes its modification to its assigned section of the grid. Then once all modifications are made, we begin making draw image calls, cutting up the larger canvas as needed
I also needed to switch to globalCompositeOperation = "source-atop" to prevent all other particles from getting trashed each time I tried to change one.
Code: https://github.com/awhipple/base-command-dev/blob/2514327c6c30cb9914962d2c8d604f04bfbdbed5/engine/gfx/shapes/Particle.js
Examples: http://avocado.whipple.life/
You can see here, when this.newRender === true in draw, it queues up to be drawn later.
Then static drawQueuedParticles is called once every particle has had a chance to queue itself up.
The end result is that this larger canvas is only sent to the GPU once per frame. I saw a performance increase from 15 FPS to 60 FPS on my Razorblade Pro running a 2700 RTX GPU with 1500 on screen particles.

I expect browsers are optimized to display 1, or at most a few canvases at a time. I'm betting each canvas is uploaded to the GPU individually, which would have way more overhead than a single canvas. The GPU has a limited number of resources, and using a lot of canvases could cause a lot of churn if textures and buffers are repeatedly cleared for each canvas. This answer WebGL VS Canvas 2D hardware acceleration also claims that Chrome didn't hardware accelerate canvases under 256px.
Since you're trying to do a particle effect with sprites, you'd be better off using a webgl library that's built for this kind of thing. I've had a good experience with https://www.pixijs.com/. If you're doing 3d, https://threejs.org/ is also popular. It is possible to build your own webgl engine, but it's very complicated and a lot of work. You have to worry about things like vector math, vertex buffers, supporting mobile GPU's, batching draw calls, etc. You'd be better off using an existing library unless you really have a strong need for something unique.

What is the best way to track time/frames in a requestAnimationFrame game loop?

I've coded along with very simple game tutorials that use a simple requestAnimationFrame game loop. They have no need of tracking time elapsed or frame rate:
var canvas = document.querySelector("#canvas");
var ctx = canvas.getContext("2d");
function gameLoop() {
ctx.clearRect(0,0,canvas.width,canvas.height);
//Drawing, etc.
var myRAF = requestAnimationFrame(gameLoop);
}
gameLoop();
Now I want to learn how to animate things like walk cycles from a spritesheet, rather than only animating movement of a static object. I believe that requires first learning how to track how much time a frame took to render, or what frame you are on. I was under the impression that this wasn't necessary if you use RAF instead of setInterval or setTimeout (oops!). I've seen people use the Date Object, the requestAnimationFrame Timestamp, and performance.now, though I don't understand the code yet. Which is the best choice for game development with requestAnimationFrame if my goals are to animate from spritesheets, and to make sure movement in the game is the same speed no matter how many fps any particular player is achieving? I've read that you have to multiply all speeds in the game by a time factor, but don't know how. Otherwise a slow computer that's only getting 30fps is walking through the game and shooting bullets at half speed compared to fast machines gettine around 60fps, right?
Please show me how I would implement the time/frame tracking code in a game loop, in addition to pros and cons of different methods of accomplishing this.
At risk of sounding like I want a tutorial, please ignore the following part from the original question
It would also be nice to see how you would use that code to animate something like walking or flapping wings from a spritesheet, and how to multiply movements speeds by a time factor so everyone gets the same game experience.

requestAnimationFrame takes a callback. That callback is passed the time in milliseconds since the page started. So you can use that time subtract it from the time if the previous requestAnimationFrame callback to figure out your frame rate and to use for a "deltaTime" to mutiply other values by like velocity.
Typically you move things based on deltaTime. If your deltaTime is in seconds then it's easy to make anything based on delta-per-second. For example to move 10 units per second each frame you do something like
const unitsPerSecond = 10;
x = x + unitsPerSecond * deltaTimeInSeconds
As for frame count you just keep your own counter
const ctx = document.querySelector('canvas').getContext('2d');
let x = 0;
let y = 0;
const speed = 120; // 120 units per second
let frameNumber = 0;
let previousTime = 0;
function render(currentTime) {
// keep track of frames
++frameNumber;
// convert time to seconds
currentTime *= 0.001;
// compute how much time passed since the last frame
const deltaTime = currentTime - previousTime;
// remember the current time for next frame
previousTime = currentTime;
// move some object frame rate independently
x += speed * deltaTime;
y += speed * deltaTime;
// keep x and y on screen
x = x % ctx.canvas.width;
y = y % ctx.canvas.height;
// clear the canvas
ctx.clearRect(0, 0, ctx.canvas.width, ctx.canvas.height);
// draw something
ctx.fillStyle = (frameNumber & 0x1) ? 'red' : 'blue'; // change color based on frame
ctx.fillRect(x - 5, y - 5, 11, 11);
// draw something else based on time
ctx.fillStyle = 'green';
ctx.fillRect(
145 + Math.cos(currentTime) * 50,
75 + Math.sin(currentTime) * 50,
10, 10);
requestAnimationFrame(render);
}
requestAnimationFrame(render);
canvas { border: 1px solid black; }
<canvas></canvas>

Subtract opacity instead of multiplying

I'm trying to make it appear as though movement on my <canvas> creates motion trails. In order to do this, instead of clearing the canvas between frames I reduce the opacity of the existing content by replacing a clearRect call with something like this:
// Redraw the canvas's contents at lower opacity. The 'copy' blend
// mode keeps only the new content, discarding what was previously
// there. That way we don't have to use a second canvas when copying
// data
ctx.globalCompositeOperation = 'copy';
ctx.globalAlpha = 0.98;
ctx.drawImage(canvas, 0, 0);
ctx.globalAlpha = 1;
ctx.globalCompositeOperation = 'source-over';
However, since setting globalAlpha multiplies alpha values, the alpha values of the trail can approach zero but will never actually reach it. This means that graphics never quite fade, leaving traces like these on the canvas that do not fade even after thousands of frames have passed over several minutes:
To combat this, I've been subtracting alpha values pixel-by-pixel instead of using globalAlpha. Subtraction guarantees that the pixel opacity will reach zero.
// Reduce opacity of each pixel in canvas
const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
const data = imageData.data;
// Iterates, hitting only the alpha values of each pixel.
for (let i = 3; i < data.length; i += 4) {
// Use 0 if the result of subtraction would be less than zero.
data[i] = Math.max(data[i] - (0.02 * 255), 0);
}
ctx.putImageData(imageData, 0, 0);
This fixes the problem, but it's extremely slow since I'm manually changing each pixel value and then using the expensive putImageData() method.
Is there a more performant way to subtract, rather than multiplying, the opacity of pixels being drawn on the canvas?

Unfortunately there is nothing we can do about it except from manually iterating over the pixels to clear low-value alpha pixels like you do already.
The problem is related to integer math and rounding (more details at this link, from the answer).
There are blending modes such as "luminosity" (and to a certain degree "multiply") which can be used to subtract luma, the problem is it works on the entire surface contrary to composite modes which only works on alpha - there is no equivalent in composite operations. So this won't help here.
There is also a new luma mask via CSS but the problem is that the image source (which in theory could've been manipulated using for example contrast) has to be updated every frame and basically, the performance would be very bad.
Workaround
One workaround is to use "particles". That is, instead of using a feedback-loop instead log and store the path points, then redraw all logged points every frame. Using a max value and reusing that to set alpha can work fine in many cases.
This simple example is just a proof-of-concept and can be implemented in various ways in regards to perhaps pre-populated arrays, order of drawing, alpha value calculations and so forth. But I think you'll get the idea.
var ctx = c.getContext("2d");
var cx = c.width>>1, cy = c.height>>1, r = c.width>>2, o=c.width>>3;
var particles = [], max = 50;
ctx.fillStyle = "#fff";
(function anim(t) {
var d = t * 0.002, x = cx + r * Math.cos(d), y = cy + r * Math.sin(d);
// store point and trim array when reached max
particles.push({x: x, y: y});
if (particles.length > max) particles.shift();
// clear frame as usual
ctx.clearRect(0,0,c.width,c.height);
// redraw all particles at a log. alpha, except last which is drawn full
for(var i = 0, p, a; p = particles[i++];) {
a = i / max * 0.6;
ctx.globalAlpha = i === max ? 1 : a*a*a;
ctx.fillRect(p.x-o, p.y-o, r, r); // or image etc.
}
requestAnimationFrame(anim);
})();
body {background:#037}
<canvas id=c width=400 height=400></canvas>

ctx.clearRect canvas sprite

I am wondering how I could alter my Javascript to only clear the falling sprites, and not the entire canvas (as it does currently).
I hope to place multiple other (animated) sprites on the canvas, which do not appear with the way my function animate is structured.
Is there a way so that if there was another image/sprite was on the canvas, it would not be affected by the function animate.
I'm thinking that this line needs to change:
ctx.clearRect(0, 0, canvas.width, canvas.height);
Though I have no idea what parameters I would need to place inside.
The falling sprites draw at a size of 60x60, but as they fall downwards this is where I am a bit stuck with clearing the only the sprite path.
Any help would be appreciated :)
var canvas = document.getElementById("canvas"),
ctx = canvas.getContext("2d");
canvas.width = 1408;
canvas.height = 640;
canvasWidth = canvas.width;
canvasHeight = canvas.height;
var orangeEnemy = new Image();
orangeEnemy.src = "http://www.catholicsun.org/wp-content/uploads/2016/09/cropped-sun-favicon-512x512-270x270.png";
var yellowEnemy = new Image();
yellowEnemy.src = "http://www.clker.com/cliparts/o/S/R/S/h/9/transparent-red-circle-hi.png";
var srcX;
var srcY;
var enemySpeed = 2.75;
var images = [orangeEnemy, yellowEnemy];
var spawnLineY=-50;
var spawnRate=2500;
var spawnRateOfDescent=1.50;
var lastSpawn=-1;
var objects=[];
var startTime=Date.now();
animate();
function spawnRandomObject() {
var object = {
x: Math.random() * (canvas.width - 15),
y: spawnLineY,
image: images[Math.floor(Math.random() * images.length)]
}
objects.push(object);
}
function animate(){
var time=Date.now();
if(time>(lastSpawn+spawnRate)){
lastSpawn=time;
spawnRandomObject();
}
requestAnimationFrame(animate);
ctx.clearRect(0, 0, canvas.width, canvas.height);
// move each object down the canvas
for(var i=0;i<objects.length;i++){
var object=objects[i];
object.y += enemySpeed;
ctx.drawImage(object.image, object.x, object.y, 60, 60);
}
}
<html>
<canvas id="canvas" style="border:3px solid"></canvas>
</html>

The easiest and quickest way would be to overlay another canvas, specifically for your sprites, atop your current canvas (requires a bit of CSS). Put all your sprites in one, everything else in the other. The clearRect() in your animate() function will then only apply to your sprite canvas, and not the other.
Otherwise, you will have to keep track of the positions of the sprites, and clear each programatically with 60x60 rectangles using clearRect(offsetX, offsetY, 60, 60).
P.S. excuse the non-formatted answer... still figuring SO out

Clear once for performance.
You are much better off clearing the whole canvas and redrawing the sprites. Using the previous position, checking for overlap and then clearing each sprite in turn, making sure you don't clear an existing sprite will take many more CPU cycles than clearing the screen once.
The clear screen function is very fast and is done in hardware, the following is the results of a performance test on Firefox (currently the quickest renderer) of clearing 65K pixels using just one call for whole area then 4 calls each a quarter, then 16 calls each clearing a 16th. (µs is 1/1,000,000th second)
Each test clears 256*256 pixels Each sample is 100 tests
'Clear 1/1' Mean time: 213µs ±4µs 1396 samples
'Clear 4/4' Mean time: 1235µs ±14µs 1390 samples
'Clear 16/16' Mean time: 4507µs ±42µs 1405 samples
As you can see clearing 65K pixels is best done in one call with the actual javascript call adding about 100µs to do.
On Firefox the number of pixels to clear does not affect the execution time of the call to clearRect with a call clearRect(0,0,256,256) and clearRect(0,0,16,16) both taking ~2µs
Apart from the speed, trying to clear overlapping animated sprites per sprite becomes extremely complicated and can result in 100s of clear calls with only a dozen sprites. This is due to the overlapping.

HTML5 Canvas FPS drops when rendering another canvas as an image using drawImage() at a specific width & height

So I have run into a very strange problem while using the HTML5 Canvas API. I'm attempting to create a game and the problem occurs because I'm drawing different canvases (using drawImage()) onto the main canvas. The other canvases has had graphics drawn on them and then I am simply drawing those canvases onto the main one. The problem is that at very specific widths and heights (coming close to the width and height of the main canvas) the fps suddenly drops by about 20-30. And this happens when only drawing one of those big canvases onto the main one. I thought the performance drop might've been attributed to drawing such a big canvas with graphics on them, so I emptied the canvases and was basically drawing an "empty" canvas onto the main canvas. Even so, the performance dropped. What's even more stange is that when I subtract just ONE pixel from either the width or height of the big canvas the fps goes back to 60! The widths and heights that this has occurred (there are probably more sets) are:
W: 1797
H: 891
W: 2026
H: 790
So for example for the first set, if you were to draw a canvas with those measurements (EVEN AN EMPTY ONE) you would get 30-40 fps. Yet if you were to draw a canvas with those one of the measurements subtracted by one (i.e. W: 1796, H: 891) then it would go back to 60 fps.
What I find even more strange is that this happens on only Chrome. I have tried it on Internet Explorer and Safari and I can draw significantly bigger canvases (i.e. far bigger than even the main canvas) and still get 60 fps. I'm sorry for not being able to list the code because it would require me to post a significantly large piece of code (the code is intertwined in multiple files). Could somebody elaborate on why this is happening? Thank you!
EDIT: This also happens if I draw only a clipped version of the big canvas using the other version of drawImage(). So it doesn't even have to do with rendering the actual number of pixels...which I find extremely strange.
EDIT 2: So I have run some tests and turns out it has nothing to do with the rendering but rather the memory usage because of the fact that I have an array that's holding 25 of these big canvases. I created a fiddle to benchmark:
https://jsfiddle.net/eu3zoc4f/3/
HTML:
<body>
<canvas id="canvas" style="display: block;"></canvas>
</body>
JS:
var timer = {
startedAt: null,
stoppedAt: null,
start: function() {
this.stoppedAt = null;
this.startedAt = new Date();
},
stop: function() {
this.stoppedAt = new Date();
},
getTime: function() {
if (!this.stoppedAt) this.stop();
return this.stoppedAt.getTime() - this.startedAt.getTime();
}
};
var body = document.getElementsByTagName("body");
body[0].style.width = screen.availWidth + "px";
document.getElementById("canvas").width = window.innerWidth;
document.getElementById("canvas").height = window.innerHeight;
var context = document.getElementById("canvas").getContext("2d");
var TESTING_1 = [];
for (var c = 0; c < 25; c++) {
TESTING_1[c] = document.createElement('canvas');
TESTING_1[c].width = 2000;
TESTING_1[c].height = 1200;
TESTING_1[c].getContext('2d').fillStyle = 'rgb(255, 0, 0)';
TESTING_1[c].getContext('2d').fillRect(0, 0, 1900, 897);
}
function main() {
timer.start();
context.drawImage(TESTING_1[0], 0, 0);
context.fillStyle = "blue";
context.font = "20px Arial";
context.fillText(timer.getTime(), 100, 100);
}
main();
If you change the "25" number that's in the for loop to a lower number you will get much faster results. So the problem now is why is this happening only in Chrome and is there a way to fix it?