So I have been working on a game in HTML5 canvas and noticed that the games lags and performs much slower when hardware acceleration is turned on in Google Chrome then when it is turned off. You can try for yourself here
From doing some profiling I see that the problem lies in drawImage.
More specifically drawing one canvas onto another. I do a lot of this.
Hardware Acceleration on.
Hardware Acceleration off.
Is there something fundamental I am missing with one canvas to another? Why would the difference be that profound?
If I remember correctly, in-DOM canvases are loaded into GPU memory in Chrome, and off-DOM canvases may not be. So each drawImage from an off-screen canvas to an on-screen canvas results in loading the content of the canvas onto the GPU as a texture, followed by a copy of that memory on-GPU onto the on-screen canvas. The cost of sending a new texture through to the GPU can be quite high. Loading textures is high-throughput, but also high-latency, on most GPUs.
Someone from the Chrome team will have to chime in with a definitive answer, but that fits my experience with canvases in Chrome.
Related
I created a sketch which draws 100s of thousands of lines. As expected, it is quite slow. So, I tried the same sketch with WEBGL mode. But this turned out to be slower than the default mode.
My understanding was that WEBGL leverages GPU for fast rendering. Is it not true?
Thank you.
createCanvas(windowWidth, windowHeight, WEBGL)
Side note: I coded this sketch first in Processing (java), where the WEBGL mode was 100 times faster than the default mode. So, I expected the same with P5js.
Drawing numerous stroked shapes in p5.js with WEBGL is notoriously slow. See: Sketch runs slow in P5.js WEBGL on the processing.org Discourse forum. If you specifically want to draw lines and/or curves and you don't need 3d perspective then a 2d canvas will actually perform better (and in most browsers it will still utilize the GPU). If you are actually utilizing 3d perspective and other WEBGL rendering capabilities then the key thing is to reduce the number of drawing instructions, and if possible avoid relying on p5.js to draw strokes. In order to give you more specific advice on how to do that you are going to have to post a minimal reproducible example of what you are trying to do.
I develop web application for mobile browsers which processes video stream from back camera in real time. Based on some calculated feature we may store current frame for followed operations.
The detailed workflow is next:
Input video stream in browser has 4K resolution. We use 3 canvases: two canvases have 4K resolution and the last one has significant lower resolution ( approximately 400x600 ). We get frame from video stream and draw it on 4K canvas. Next we draw this 4K canvas onto smallest one. Then we get array representation of image from this small canvas and perform some calculation with it. Based on that calculation we decide if this frame should be stored or not ( we say that we found "Best frame" ). This best frame we store in original resolution 4K in the second 4K canvas for final processing and continue to process next frames in hope to fine little bit better.
In my work I faced with problem that Google Chrome shows less performance than FireFox on the same device and dramatically worse performance than Safari on the devices with same class.
In order to demonstrate the problem I created test html example. In it I use all of the operations which I consider as critical. There are: drawing frame from video stream onto first 4K canvas, scale this canvas and draw it onto smallest canvas, obtaining of array buffer from smallest canvas for calculation. These three operations are called for each frame therefore their performance is most critical for me.
Repository with example
Deployed example
Measured timers:
Execution time originalCanvas.drawFrame, ms - drawing of videoframe onto 4K canvas
Execution time scaledCanvas.drawFrame, ms - drawing 4K canvas onto small canvas
Execution time scaledCanvas.getBmpData, ms - obtaining byte array from small canvas
All big canvases in testing have resolution 3840x2160, all small ones have resolution 711x400.
Now let's move on to the most important thing... Why we have such big inequality in performance in different browsers and on different devices which have the same class? Unfortunately I can't test example in Chrome and FireFox on iPhones because of prohibition of access to camera there. I consider that manipulation with canvases is a simple operation which shouldn't be so long. And why Safari has extremely amazing performance compare with Chrome or even FireFox?
I hope my topic wasn't very boring. I would be glad to hear everything about my workflow in application or about my conclusions. Thanks a lot!
These days, I need to draw many images on a canvas. The canvas size is 800x600px, and I have many images of 256x256px(some is smaller) to draw on it, these small images will compose a complete image on the canvas. I have two ways to implement this.
First, if I use canvas 2D context, that is context = canvas.getContext('2d'), then I can just use context.drawimage() method to put every image on the proper location of the canvas.
Another way, I use WebGL to draw these images on the canvas. On this way, for every small image, I need to draw a rectangle. The size of the rectangle is the same as this small image. Besides, the rectangle is on the proper location of the canvas. Then I use the image as texture to fill it.
Then, I compare the performance of these two methods. Both of their fps will reach 60, and the animation(When I click or move by the mouse, canvas will redraw many times) looks very smooth. So I compare their CPU usage. I expect that when I use WebGL, the CPU will use less because GPU will assure many work of drawing. But the result is, the CPU usage looks almost the same. I try to optimize my WebGL code, and I think it's good enough. By google, I found that browser such as Chrome, Firefox will enable Hardware acceleration by default. So I try to close the hardware acceleration. Then the CPU usage of the first method becomes much higher. So, my question is, since canvas 2D use GPU to accelerate, is it necessary for me to use WebGL just for 2D rendering? What is different between canvas 2D GPU acceleration and WebGL? They both use GPU. Maybe there is any other method to lower the CPU usage of the second method? Any answer will be appreciate!
Canvas 2D is still supported more places than WebGL so if you don't need any other functionality then going with Canvas 2D would mean your page will work on those browsers that have canvas but not WebGL (like old android devices). Of course it will be slow on those devices and might fail for other reasons like running out of memory if you have a lot of images.
Theoretically WebGL can be faster because the default for canvas 2d is that the drawingbuffer is preserved whereas for WebGL it's not. That means if you turn anti-aliasing off on WebGL the browser has the option to double buffer. Something it can't do with canvas2d. Another optimization is in WebGL you can turn off alpha which means the browser has the option to turn off blending when compositing your WebGL with the page, again something it doesn't have the option to do with canvas 2d. (there are plans to be able to turn off alpha for canvas 2d but as of 2017/6 it's not widely supported)
But, by option I mean just that. It's up to the browser to decide whether or not to make those optimizations.
Otherwise if you don't pick those optimizations it's possible the 2 will be the same speed. I haven't personally found that to be the case. I've tried to do some drawImage only things with canvas 2d and didn't get a smooth framerate were as I did with WebGL. It made no sense to be but I assumed there was something going on inside the browser I was un-aware off.
I guess that brings up the final difference. WebGL is low-level and well known. There's not much the browser can do to mess it up. Or to put it another way you're 100% in control.
With Canvas2D on the other hand it's up to the browser what to do and which optimizations to make. They might changes on each release. I know for Chrome at one point any canvas under 256x256 was NOT hardware accelerated. Another example would be what the canvas does when drawing an image. In WebGL you make the texture. You decide how complicated your shader is. In Canvas you have no idea what it's doing. Maybe it's using a complicated shader that supports all the various canvas globalCompositeOperation, masking, and other features. Maybe for memory management it splits images into chucks and renders them in pieces. For each browser as well as each version of the same browser what it decides to do is up to that team, where as with WebGL it's pretty much 100% up to you. There's not much they can do in the middle to mess up WebGL.
FYI: Here's an article detailing how to write a WebGL version of the canvas2d drawImage function and it's followed by an article on how to implement the canvas2d matrix stack.
I intend to have a FabricJS canvas that will display potentially dozens of different graphics, which could either be served up as individual image files or a sprite. For each graphic, it's possible that particular graphic would be displayed on the canvas 0 to dozens of times. I know that FabricJS has the clipping ability, which would make the use of sprites possible, and I know that generally speaking, sprites on the web are highly preferred over individual images, such as for CSS.
However, I've also heard of browser behaviors/quirks surrounding canvas where some optimizations that you might expect to take place, don't -- I can't find the link at the moment, but one that I recall was how large canvas-content elements out of view in Chrome would still take a considerable amount of time to be calculated/"rendered", despite not being relevant to on-screen appearance.
So, are there any known expectations on whether n individual non-clipped images would be preferable/unpreferable to a sprite that's clipped n times for a FabricJS canvas? In case it matters, this would not be a heavily animated FabricJS canvas; things may move when dragged, for example, but constant animation is not what's involved in this case.
Just do a test yourself to find out what slows down your PC. It depends on the PC and browser and graphics card.
I don't know anything about Fabric.js, but computer CPUs and graphics cards can handle clipping/graphic manipulation easily. You're not displaying millions of polygons like a 3D game, so you should be fine.
Here's a website that explains the CSS sprites vs individual images network performance increases:
https://medium.com/parlay-engineering/emoji-at-scale-render-performance-of-css-sprites-vs-individual-images-f0a0a2dd8039
I was wondering if there were any performance differences in using drawImage() on a dynamically created canvas (i.e. document.createElement("canvas")) versus a canvas already created in the DOM (i.e. tags on a HTML page).
My thinking is that the "slow" part of drawImage() occurs when it actually has to display visuals to the user, though I could be wrong (can't find too much information on this).
Would it be substantially more expensive to draw a group of objects to a canvas in memory followed by a final draw to the "main" canvas than just drawing straight to the latter? I feel like it'd be better to have multiple canvases, at least for organizational purposes.
Related: Does the size of the canvas impact performance if you're only drawing to a subsection of it?
Talking about Chrome and Firefox I could not find any difference between static and dynamic canvas elements. Mainly the amount of pixels drawImage() handles makes it slow + the current globalCompositeOperation (copy, source-over are fastest). However, the browser has to render the full page, so it is a bad idea to place a stretched (background) image below the canvas.
There is a difference between the canvas attributes width/height and its style width/height attributes. You may have a 300*200 pixels canvas with a style size set to 100%. Then internal drawing speed is same what ever the browsers window size is. Of course display quality is a concern.
You may want to separate drawing (lines, boxes, arcs etc) from blitting (drawImage) and find out what consumes more time in your application. As long there is no need for multiple canvas (image processing, blending videos, etc.) try to avoid drawImage(). Your code - not elements - should help you to deal with 'organizational purposes'.
A fullscreen drawImage() on a 1 GHZ Netbook with 1024x600 pixels takes about 10msec. Doing it twice means there is no way to achieve a display rate of 50Hz. Things getting worse if you target iPhone or Android smartphones.
There is no need to do good-old-double-buffering with canvas, it's already implemented. You are free to update only relevant (dirty) subparts of your canvas element whenever you want and gain essential msecs.
Instead of using multiple canvas there is an option to do all invisible operations on a huge one in different sections - drawImage() with target and source the same. It is easier then to see what is happening while debugging.