Develop Reference

Modifying in memory canvases before drawing to screen drastically reduces Javascript performance

I've noticed if I have a large number of canvases in memory, modifying each canvas before drawing them to the screen drastically reduces performance on my machine. This occurs even when the canvases are small and the modifications are minor.
Here is the most contrived example I could come up with:
var { canvas, ctx } = generateCanvas();
ctx.strokeStyle = "#000";
var images = [];
for (var i = 0; i < 500; i++) {
images.push(generateCanvas(50, "red"));
}
var fps = 0,
lastFps = new Date().getTime();
requestAnimationFrame(draw);
function draw() {
requestAnimationFrame(draw);
var modRects = document.getElementById("mod-rects").checked;
var drawRects = document.getElementById("draw-rects").checked;
ctx.clearRect(0, 0, 500, 500);
ctx.strokeRect(0, 0, 500, 500);
fps++;
if (new Date().getTime() - lastFps > 1000) {
console.clear();
console.log(fps);
fps = 0;
lastFps = new Date().getTime();
}
images.forEach(img => {
img.ctx.fillStyle = "yellow";
if (modRects) img.ctx.fillRect(20, 20, 10, 10);
if (drawRects) ctx.drawImage(img.canvas, 225, 225);
});
}
function generateCanvas(size = 500, color = "black") {
var canvas = document.createElement("canvas");
canvas.width = canvas.height = size;
var ctx = canvas.getContext("2d");
ctx.fillStyle = color;
ctx.fillRect(0, 0, size, size);
return {
canvas,
ctx
};
}
function generateCheckbox(name) {
var div = document.createElement("div");
var check = document.createElement("input");
check.type = "checkbox";
check.id = name;
var label = document.createElement("label");
label.for = name;
label.innerHTML = name;
div.appendChild(check);
div.appendChild(label);
return div;
}
document.body.appendChild(canvas);
document.body.appendChild(generateCheckbox("mod-rects"));
document.body.appendChild(generateCheckbox("draw-rects"));
canvas+div+div { margin-bottom: 20px; }
In this example we create 500 canvases of size 50x50. There are two checkboxes underneath the larger onscreen canvas. The first causes a small yellow square to be drawn on each of those 500 canvases. The 2nd causes the canvases to be drawn to the larger canvas. FPS is posted to the console once per second. I see no performance issues when one or the other checkbox is checked, but when both are checked, performance drops drastically.
My first thought is that it has something to do with sending in-memory canvas to the gfx card every frame when they are modified.
Here's the actual effect I'm trying to create.
Video: https://youtu.be/Vr6v2oF3G-8
Code: https://github.com/awhipple/base-command-dev/blob/e2c38946cdaf573abff5ded5399c90687ffa76a5/engine/gfx/shapes/Particle.js
My ultimate goal is to be able to smoothly transition the colors of the canvas. I'm using globalCompositeOperation = "source-in" and fillRect() to do this in the code link above.

As has been stated before, this is an issue with the overhead of sending hundreds of canvases to the GPU every single frame. When a canvas is modified in CPU it gets marked as "dirty" and is re sent to the GPU next time it's used.
The workaround I found was to create a large canvas containing a grid of my particle images. Every particle object makes its modification to its assigned section of the grid. Then once all modifications are made, we begin making draw image calls, cutting up the larger canvas as needed
I also needed to switch to globalCompositeOperation = "source-atop" to prevent all other particles from getting trashed each time I tried to change one.
Code: https://github.com/awhipple/base-command-dev/blob/2514327c6c30cb9914962d2c8d604f04bfbdbed5/engine/gfx/shapes/Particle.js
Examples: http://avocado.whipple.life/
You can see here, when this.newRender === true in draw, it queues up to be drawn later.
Then static drawQueuedParticles is called once every particle has had a chance to queue itself up.
The end result is that this larger canvas is only sent to the GPU once per frame. I saw a performance increase from 15 FPS to 60 FPS on my Razorblade Pro running a 2700 RTX GPU with 1500 on screen particles.

I expect browsers are optimized to display 1, or at most a few canvases at a time. I'm betting each canvas is uploaded to the GPU individually, which would have way more overhead than a single canvas. The GPU has a limited number of resources, and using a lot of canvases could cause a lot of churn if textures and buffers are repeatedly cleared for each canvas. This answer WebGL VS Canvas 2D hardware acceleration also claims that Chrome didn't hardware accelerate canvases under 256px.
Since you're trying to do a particle effect with sprites, you'd be better off using a webgl library that's built for this kind of thing. I've had a good experience with https://www.pixijs.com/. If you're doing 3d, https://threejs.org/ is also popular. It is possible to build your own webgl engine, but it's very complicated and a lot of work. You have to worry about things like vector math, vertex buffers, supporting mobile GPU's, batching draw calls, etc. You'd be better off using an existing library unless you really have a strong need for something unique.

HTML Canvas draw image from database [duplicate]

This question already has answers here:
Load image from url and draw to HTML5 Canvas
(3 answers)
Closed 4 years ago.
I want to get an image from my database, and show it in a canvas and draw shapes on it. Is there a way to draw image on canvas using image as byte array etc..

A <canvas> is like a regular <img> image with the difference that the Javascript code of the page can actually control the values of the pixels.
You can decide what is the content of the canvas by drawing primitives like lines, polygons, arcs or even other images. You can also set or alter the content by processing the actual red, green, blue and transparency values of every single pixel (using getImageData and putImageData). You can resize the canvas as you wish and drawing on a canvas is quite fast... fast enough to be able to do complex animations with just plain Javascript code.
For example the code to load an image and draw it in a canvas after changing it to black and white and adding a red grid could be:
function canvasTest(img) {
let canvas = document.createElement("canvas");
canvas.width = img.width;
canvas.height = img.height;
let ctx = canvas.getContext("2d");
// Copy image pixels to the canvas
ctx.drawImage(img, 0, 0);
// Now let's set R and B channels equal to G
let idata = ctx.getImageData(0, 0, canvas.width, canvas.height);
for (let i=0,n=img.width*img.height*4; i<n; i+=4) {
idata.data[i] = idata.data[i+2] = idata.data[i+1];
}
ctx.putImageData(idata, 0, 0);
// Add a red grid
ctx.beginPath();
for (let y=0; y<img.height; y+=50) {
ctx.moveTo(0, y+0.5); ctx.lineTo(img.width, y+0.5);
}
for (let x=0; x<img.width; x+=50) {
ctx.moveTo(x+0.5, 0); ctx.lineTo(x+0.5, img.height);
}
ctx.strokeStyle = "#F00";
ctx.lineWidth = 1;
ctx.stroke();
// add the final result to page
document.body.appendChild(canvas);
}
The only annoyance is that if you draw an image on a canvas and the image is coming from a different origin from where the page comes from, then the canvas becomes in the general case "tainted" and you cannot access the pixel values any more (getImageData is forbidden). The reason for this limit is security (and the sad fact that security in web application is on the client). In your case since the image database is yours there should be no problem.

ctx.clearRect canvas sprite

I am wondering how I could alter my Javascript to only clear the falling sprites, and not the entire canvas (as it does currently).
I hope to place multiple other (animated) sprites on the canvas, which do not appear with the way my function animate is structured.
Is there a way so that if there was another image/sprite was on the canvas, it would not be affected by the function animate.
I'm thinking that this line needs to change:
ctx.clearRect(0, 0, canvas.width, canvas.height);
Though I have no idea what parameters I would need to place inside.
The falling sprites draw at a size of 60x60, but as they fall downwards this is where I am a bit stuck with clearing the only the sprite path.
Any help would be appreciated :)
var canvas = document.getElementById("canvas"),
ctx = canvas.getContext("2d");
canvas.width = 1408;
canvas.height = 640;
canvasWidth = canvas.width;
canvasHeight = canvas.height;
var orangeEnemy = new Image();
orangeEnemy.src = "http://www.catholicsun.org/wp-content/uploads/2016/09/cropped-sun-favicon-512x512-270x270.png";
var yellowEnemy = new Image();
yellowEnemy.src = "http://www.clker.com/cliparts/o/S/R/S/h/9/transparent-red-circle-hi.png";
var srcX;
var srcY;
var enemySpeed = 2.75;
var images = [orangeEnemy, yellowEnemy];
var spawnLineY=-50;
var spawnRate=2500;
var spawnRateOfDescent=1.50;
var lastSpawn=-1;
var objects=[];
var startTime=Date.now();
animate();
function spawnRandomObject() {
var object = {
x: Math.random() * (canvas.width - 15),
y: spawnLineY,
image: images[Math.floor(Math.random() * images.length)]
}
objects.push(object);
}
function animate(){
var time=Date.now();
if(time>(lastSpawn+spawnRate)){
lastSpawn=time;
spawnRandomObject();
}
requestAnimationFrame(animate);
ctx.clearRect(0, 0, canvas.width, canvas.height);
// move each object down the canvas
for(var i=0;i<objects.length;i++){
var object=objects[i];
object.y += enemySpeed;
ctx.drawImage(object.image, object.x, object.y, 60, 60);
}
}
<html>
<canvas id="canvas" style="border:3px solid"></canvas>
</html>

The easiest and quickest way would be to overlay another canvas, specifically for your sprites, atop your current canvas (requires a bit of CSS). Put all your sprites in one, everything else in the other. The clearRect() in your animate() function will then only apply to your sprite canvas, and not the other.
Otherwise, you will have to keep track of the positions of the sprites, and clear each programatically with 60x60 rectangles using clearRect(offsetX, offsetY, 60, 60).
P.S. excuse the non-formatted answer... still figuring SO out

Clear once for performance.
You are much better off clearing the whole canvas and redrawing the sprites. Using the previous position, checking for overlap and then clearing each sprite in turn, making sure you don't clear an existing sprite will take many more CPU cycles than clearing the screen once.
The clear screen function is very fast and is done in hardware, the following is the results of a performance test on Firefox (currently the quickest renderer) of clearing 65K pixels using just one call for whole area then 4 calls each a quarter, then 16 calls each clearing a 16th. (µs is 1/1,000,000th second)
Each test clears 256*256 pixels Each sample is 100 tests
'Clear 1/1' Mean time: 213µs ±4µs 1396 samples
'Clear 4/4' Mean time: 1235µs ±14µs 1390 samples
'Clear 16/16' Mean time: 4507µs ±42µs 1405 samples
As you can see clearing 65K pixels is best done in one call with the actual javascript call adding about 100µs to do.
On Firefox the number of pixels to clear does not affect the execution time of the call to clearRect with a call clearRect(0,0,256,256) and clearRect(0,0,16,16) both taking ~2µs
Apart from the speed, trying to clear overlapping animated sprites per sprite becomes extremely complicated and can result in 100s of clear calls with only a dozen sprites. This is due to the overlapping.

drawImage() implementation

I have two canvases that are different sizes. My goal is copy the user's drawing from the main canvas to a second canvas as a scaled down version. So far the drawImage() and scale appear to be working, but the second canvas keeps the old version of the main drawing along with the new copy. I tried clearing it each time before calling drawImage(), but that doesn't appear to do anything. How can I copy just the current image to my secondary canvas each time the function runs?
$('#hand').dblclick(function(){
var canvas = document.getElementById('myCanvas');
var context = canvas.getContext('2d');
//var imageData = context.getImageData(0, 0, 100, 100);
var newCanvas = document.getElementById('scaledCanvas');
var destCtx = newCanvas.getContext('2d');
destCtx.clearRect(0, 0, newCanvas.width, newCanvas.height);
destCtx.scale(.5,.5);
destCtx.drawImage(canvas, 0, 0);
});
I can include more code if necessary. I also just realized that scale keeps getting called; this explains why the new copied image would get smaller each time as well, so that might be another problem.

It's quite simple actually, you're using what's called a transform (translate, rotate, or scale).
In order to use them "freshly" each time you must save and restore the canvas state each time.
$('#hand').dblclick(function(){
var canvas = document.getElementById('myCanvas');
var context = canvas.getContext('2d');
//var imageData = context.getImageData(0, 0, 100, 100);
var newCanvas = document.getElementById('scaledCanvas');
var destCtx = newCanvas.getContext('2d');
destCtx.clearRect(0, 0, newCanvas.width, newCanvas.height);
//save the current state of this canvas' drawing mode
destCtx.save();
destCtx.scale(.5,.5);
destCtx.drawImage(canvas, 0, 0);
//restore destCtx to a 1,1 scale (and also 0,0 origin and 0 rotation)
destCtx.restore();
});
It's also important to note you can push several times before calling restore, in order to perform many cool geometric tricks using recursive functions etc...
Take a look at this explanation of states and transformations:
https://developer.mozilla.org/en-US/docs/HTML/Canvas/Tutorial/Transformations
Hope this helps you understand canvas transforms a bit better.

HTML Canvas: draw image without anti-aliasing

I want to do pixel-true rendering of some images on my Canvas. Right now, I obtain the images through Javascript, so my images are HTMLImageElement instances. I can draw these on the Canvas' rendering context with drawImage. However, this performs anti-aliasing on the image, which I don't want.
There appears to be a lower-level image manipulation method named putImageData, operating on ImageData objects. Does this method perform any anti-aliasing? If not, it's a fine candidate for what I'm looking for, but I haven't found out how I can convert or blit an HTMLImageElement to an ImageData instance.
Any advice would be welcome!
Edit: my original problem was solved, I accidentally had a coordinate that was fractional, which forces anti-aliasing. The conversion-to-image-data question still stands though.

The only way to convert an image into an ImageData object is to draw it to a canvas first, so you'll need to create a temporary canvas, draw the image on it, and get the image data from there.
function imageToImageData(image) {
var canvas = document.createElement("canvas");
canvas.width = image.width;
canvas.height = image.height;
var ctx = canvas.getContext("2d");
ctx.drawImage(image, 0, 0);
return ctx.getImageData(0, 0, canvas.width, canvas.height);
}
Note though, that the same-origin policy prevents you from calling getImageData if you draw an image from a different domain to the canvas, so this will only work on images from the same domain as the document. If you need to draw images from other domains, your only option is to call drawImage on the context for you main canvas directly, making sure there are no transformations that will affect the accuracy.