I'm coding a JavaScript game. This game obviously needs to be constantly rendering a screen, which, using canvas, must be an Uint8Array(width * height * 4) (as used by Canvas's ImageData). In order to test the expected FPS, I've tried filling that array with white. Much to my surprise, the performance was mediocre. I could barely fill a 1000x1000 bitmap with white in a high-end computer at 150 fps. Considering this is the best performance, without any game logic running, the end result will probably be much lower. My question is: why is the performance so low and what can I do to improve it?
Figuring out how many times you can fill a 1000x1000 canvas using putImageData will not give you any kind of realistic results. The reason is the graphics are pipelined. In a normal app you'd only call putImageData once a frame. If you call it more than once a frame at some point you'll fill the pipeline and stall it. In a real app though you'd be manipulating your data most the frame and only uploading it once not stalling the pipeline.
If you want to see how much work you can do, make a 1000x1000 canvas, call getImageData on it to get an ImageData, manipulate that image data a certain amount, call putImageData, then call requestAnimationFrame and do it again. Slowly increase the amount of manipulation you do until it starts running slower than 60fps. That will tell you how much work you can do realistically.
Here's a fiddle that tries this out
http://jsfiddle.net/greggman/TVA34/
Also, using putImageData has a few issues. One is that Canvas requires pre-multiplied alpha but putImageData supplies un-premultiplied alpha which means some process has to convert your data to pre-multiplied alpha. Then, now-a-days, most Canvas implementations are GPU accelerated. That's great for nearly every feature of the Canvas API but it sucks for putImageData since that data has to be transferred from the CPU to the GPU. It's even worse for getImageData since copying data from the GPU back to the CPU generally stalls the GPU. The stories is even more bad on an HD-DPI machine. getImageData and putImageData have to convert from CSS pixels to the resolution that's actually being used.
If you use WebGL you can at least skip the pre-multiplied alpha conversion step.
Here's a WebGL version
http://jsfiddle.net/greggman/XLgs6/
Interestingly on my 2012 Macbook Pro Retina I find the canvas version is faster on Chrome and Safari. I'm curious why since I would not expect that having worked on them.
/*
Canvas WebGL
Chrome 32 : 710k 650k numbers are in 'operations per frame`
Firefox 26 : 80k 190k
Safari 7.0.1 : 150k 120k
*/
My test also might not be valid. Manipulating only 710k pixels (of 1000k pixels) seems pretty slow. Maybe one of the functions like Math.random or Math.floor is particularly slow, especially given they're using doubles.
See these 2 tests
Only doing loop and assign to the same address
http://jsperf.com/variable-assign
Good old array expansion (and also another popular for loop trick that suppose to be faster)
http://jsperf.com/fill-an-type-array-expand/3
First test shows the address look up is taking about 3/4 of the time.
Second test shows the for loop is taking more than 30% of the time.
I think what typed array is really in lack of is some native code that does the block copy that would really be used in a game development (rather than set pixel by pixel in the tests). WebGL might worth considering.
Related
I am trying to create a water ripple effect in a video embedded in HTML5 default web player.
I am doing it fine with using images and and a overlay canvas on top of it, but what I am trying to do now is to get single frames from a video and output it to a canvas every 1-5ms using this tutorial.
And I am stuck at this point, I can output frame into another canvas using
canvas.toDataURL() function.
I have seen advanced web-based video players that allow for applying Processing.js sketches on top of videos, would that be a good solution?
My question is: what would be the best and most reliable solution for applying visual effects (water ripples in this case) using JavaScript to a video playing in HTML5 media player.
My question is: what would be the best and most reliable solution for applying visual effects (water ripples in this case) using JavaScript to a video playing in HTML5 media player.
In my opinion the best approach would be to use WebGL to create the effects, using the video as input texture and a simple flat geometry that is manipulated using a animated bump-map - or - directly manipulating the vertices - or - a perhaps a shader program, and then output the result to a canvas.
I would not place the video in DOM at all but create control buttons and display those together with the output [webgl-]canvas in DOM.
The obvious drawback would be besides from slightly more complex code, when used on computers which doesn't have a GPU (but that would be a drawback in any case and more so if you used a regular 2D canvas and pixel manipulation).
This is of course very broad in terms of code example. But I assume you get the general idea.
A couple of notes:
[...] what I am trying to do now is to get single frames from a video and output it to a canvas every 1-5ms [...]
This is sort of pointless in regards to the time-budget, as the typical monitor can only refresh the image every ~16.7ms which means you're wasting at least 3-4 frames that are never displayed.
Also, a typical video source never runs faster than 30 FPS (in the US, 25 FPS in Europe) which means there is only a new frame every 33.3ms (there are of course special case videos such as VR/AR, stereoscopic, games and "Mac" recorded video etc. that may use higher fps - but for the most part anything > 30 fps is usually wasted cycles) which allows for a higher time-budget for processing per frame.
I can output frame into another canvas using canvas.toDataURL() function [...]
Outch! :) This comes which a huge overhead. The browser would have to use line filtering (in case of PNG - well, some browsers skip this and use filter 0), compression, base-64 encoding, then apply the source to an image element, decode the base-64, decompress, defilter...
You can instead simply use the source canvas for drawImage() directly and only have to deal with a in-memory bitmap (super fast!).
All that being said: if simplicity is important code-wise, you can of course do all this using a 2D canvas. You can use drawImage() to displace pixels/blocks and therefor work on the GPU (but not necessarily faster than working on the bitmap directly depending on how you apply the actual displacement).
But there are still many caveats such as video source and destination resolution which has a exponential impact on performance, limited use of the GPU as you still would have to do multiple serial operations versus parallel operations with webgl/gpu and so forth. In essence, the performance will suffer compared to a webgl solution.
If you want to get high performance you can use WebGL. Following is a github reop for Water ripple project.
https://github.com/sirxemic/jquery.ripples/
Following is a ruining example for jQuery WebGL Ripples
http://sirxemic.github.io/jquery.ripples/
Think this might help
You can do it in several ways. You can write core javascript code with canvas.
But I think it is best to use Jquery plugins. There are several plugins available for water ripple.
You may check the following links:
https://github.com/virtyaluk/paper-ripple
https://github.com/sirxemic/jquery.ripples
https://github.com/andyvr/water-ripple
There seems to be some differing opinions about this going from blog to blog, video to video, and forum to forum. This is specifically for the 2D context for the canvas tag, not WebGL. I know that WebGL would give me better performance but my goal is to understand how the canvas tag works in its 2D context.
I hear that pre-rendering your objects in a "virtual" offscreen canvas is probably the best for performance which makes sense seeing the browser wouldn't literally be drawing it. People then say to grab the data from that canvas using "getImageData" which from my understanding returns a base64 code that you can apply to a canvas that is added to the DOM using "putImageData". Wouldn't this be a huge performance hit?
Should I render the entire seen for the game on this virtual canvas and then put it to the visible one using this method inside of the loop?
Yes you can. I recommend using requestAnimationFrame() and inside the draw() part of your game loop, draw all separate objects to an offscreen canavs, and then do one get/put image to the visible canvas. While this may seem to be a performance hit, but when you have many objects being drawn, this overhead of get/put image is actually negligible.
The Canvas is just an image, and from my experience there is no difference between using a canvas or an image in terms of performance.
Offscreen or on the canvas is no different, they will both take advantage of the GPU wherever possible.
Using context.getImageData(), context.createImageData(), and context.putImageData() should be avoided for realtime rendering, It does not take advantage of the GPU and any processing you do to it will be done in main memory by Javascript. Though the data is stored in a typed array Uint8ClampedArray and can be converted to any type of typed array, such as a Uint32Array allowing you to handle a single pixel with one variable, rather than 4. There are also many native functions for typed arrays that provide much quicker array manipulation than the standard Javascript array.
The limiting factor for images (including canvas as image) is the amount of GPU RAM available, when you exceed the amount of RAM available the browser will start swapping images into GPU RAM as they are needed, when it does so this blocks the GPU's ability to render, and the transfer from Main RAM to the GPU RAM is slow in comparison to normal RAM access. When this happens you will instantly see a loss in frame rate. As there are a huge variety of platforms that the browsers can run on and no way to know the machines capabilities you should be careful when you publish realtime applications.
If you have written a game with high resolution images for a high end desktop machine, it will not perform very well on tablets and low end laptops. To mitigate this problem downsample the images to match the screen resolution. Using a hires background image on a device that is 1/8th the resolution is putting undue strain on the hardware. Devices are made to handle the resolution of their screens, going over this resolution will have a major unnecessary performance hit. This is where you can use an offscreen canvas to render the image at the native resolution of the device and then dump the original hires image. There will be no loss of quality, but a huge gain in performance, turning something that is unplayable into playable. This applies to all graphic resources. Never store and use images at a resolution higher then the device you are using can display.
Because there is such a variety of things you can do with the canvas the best way to find out what runs best is to experiment. Monitor the frame rate and try different approaches to the problem at hand. If the frame rate improves you have found a better way of doing thing, if the frame rate drops then you have used the wrong method.
I know that one of the most expensive operations in HTML5 gamedev is drawing on the canvas. But, what about drawing images outside of it? How expensive is that? What exactly happens when the canvas is 100 by 100 pixels and I try to draw an image at (1000, 1000)? Would checking sprite coordinates to make sure it is inside the canvas make rendering more efficient?
In these tests I used Google Chrome version 21.0.1180.57.
I've made a small fiddle that tests this situation... You can check it out here: http://jsfiddle.net/Yannbane/Tnahv/.
I've ran the tests 1000000 times, and this is the data I got:
Rendering the image inside the canvas lasted 2399 milliseconds.
Rendering the image outside the canvas lasted 888 milliseconds.
This means that drawing outside the canvas does take some time, roughly, 37% of time it would take to render it inside.
Conclusion: It's better to check if the image is inside the canvas before rendering it.
But, of course, I wanted to know how much better... So, I did another test. This time, I, of course, implemented boundary checking, and got that it only took 3 milliseconds to "render" the image outside the canvas 1000000 times. That's 29600% better than simply rendering it outside.
You can see those tests here: http://jsfiddle.net/Yannbane/PVZnz/3/.
You need to perform this check yourself and skip drawing if a figure is out of the screen.
That being said, some browsers do optimize this in some conditions. I found out while writing an article on the IE9 performance profiler a while back that IE9 will optimize away drawing an image if it is out of bounds. The transformation matrix may have to be identity for this optimization to work, and either way you shouldn't rely on browsers doing it.
Always always check.
edit: You can run this simple test to see: http://jsperf.com/on-screen-vs-off
It looks like Chrome and Safari certainly optimize it, at least in simple cases, and firefox doesn't really
I seem to be experiencing varying performance using an HTML5 canvas based on the memory size of the page... perhaps the number of images (off-screen canvases) that are loaded. How to I best locate the source of the performance problem? Or does anyone know if in fact there is a performance issue when there's a lot of data loaded, even if it isn't all being used at once?
Here's an example of good performance. I have a relatively simple map. It's between 700 and 800 KB. You can drag to scroll around this map relatively smoothly.
There's another file (which you may not want to look at due to its large size).
It's about 16 MB and contains dozens, maybe on the order of a hundred images/canvases. It draws a smaller view so it should go faster. But it doesn't. Many of the maps lag quite severely compared to the simpler demo.
I could try to instrument the code to start timing things, but I have not done this in JavaScript before, and could use some help. If there are easier ways to locate the source of performance problems, I'd be interested.
In Google Chrome and Chromium, you can open the developer tools (tools->developer tools) and then click on "Profiles". Press the circle at the bottom, let the canvas redraw and then click on the circle again. This gives you a profile that shows how much time was spent where.
I've been working on some complex canvas stuff where rendering performance mattered to me.
I wrote some test cases in jsperf and came to the conclusion that a rule of thumb is that a source offscreen canvas should never be more than 65536 pixels.
I haven't yet come to a conclusion about why this is, but likely a data structure or data type has to be changed when dealing with large source canvases.
putImageData showed similar results.
destination canvas size didn't seem to matter.
Here are some tests I wrote that explore this performance limitation:
http://jsperf.com/magic-canvas/2
http://jsperf.com/pixel-count-matters/2
I'm interested in making a game using html and javascript. I was wondering if it really is that much faster drawing in html5 and javascript than it is with images and div's in html and javascript.
Example of a game using html and javascript that works nicely:
http://scrabb.ly/
Example of a game using html5 and javascript that works nicely:
http://htmlchess.sourceforge.net/demo/example.html
I've run a bunch of numbers on HTML-made drawing versus Canvas-made drawing. I could make a huge post about the benefits of each, but I will give some of the relevant results of my tests to consider for your specific application:
I made Canvas and HTML test pages, both had movable "nodes." Canvas nodes were objects I created and kept track of in Javascript. HTML nodes were <div>s, though they could be <image> or <video> too.
I added 100,000 nodes to each of my two tests. They performed quite differently:
The HTML test tab took forever to load (timed at slightly under 5 minutes, chrome asked to kill the page the first time). Chrome's task manager says that tab is taking up 168MB. It takes up 12-13% CPU time when I am looking at it, 0% when I am not looking.
The Canvas tab loaded in one second and takes up 30MB. It also takes up 13% of CPU time all of the time, regardless of whether or not one is looking at it.
Dragging on the HTML page is smoother, which I suppose is expected, since the current setup is to redraw EVERYTHING every 30 milliseconds in the Canvas test. There are plenty of optimizations to be had for Canvas for this. (canvas invalidation being the easiest, also clipping regions, selective redrawing, etc.. just depends on how much you feel like implementing)
Video on the HTML page, while I am not moving objects, is actually perfectly smooth.
On canvas the video is always slow, since I am redrawing constantly because I turned off my drawing canvas invalidation. There is of course plenty of room for improvement.
Drawing/loading alone is far faster in Canvas and has far more room for optimizations, too (ie, excluding things that are off-screen is very easy).
Fast as in faster rendering or faster development? I would say the answer to both is HTML5 canvas. Although it is a fairly new technology, and not even supported by all mainstream browsers yet, it already has much more functionality than you would have using DIVs with normal HTML. I've done drawing with divs before and it was incredibly frustrating just getting something to work. With canvas you already have a framework in place to do most basic drawing. Furthermore, html5 is new. Even if it is relatively slower than drawing with divs right now (which it probably isn't), that performance will increase as development and adoption increases. I can't say the same for drawing with divs.
Pros to using HTML5 Canvas:
Similar to other drawing frameworks (OpenGL, DirectX)
Will continue to increase in performance and functionality
May become hardware accelerated in the future
Possible 3D framework in the future
Neither of those games requires HTML 5. scrabb.ly does everything with rectangular objects, which divs handle just fine, and the chess game doesn't even use animation. If that's the kind of game you're thinking of building, then what you use should be decided on the grounds of familiarity and compatibility rather than performance.