I've built a canvas library for managing scenes of shapes for some work projects. Each shape is an object with a drawing method associated with it. During a refresh of the canvas, each shape on the stack is drawn. A shape may have typical mouse events bound which are all wrapped around the canvas' own DOM mouse events.
I found some techniques in the wild for detecting mouseover on individual shapes, each of which works but with some pretty serious caveats.
A cleared ghost canvas is used to draw an individual shape by itself. I then store a copy of the ghost canvas with getImageData(). As you can imagine, this takes up a LOT of memory when there are many points with mouse events bound (100 clickable shapes on a 960x800 canvas is ~300MB in memory).
To sidestep the memory issue, I began looping over the pixel data and storing only addresses to pixels with non-zero alpha. This worked well for reducing memory, but dramatically increased the CPU load. I only iterate on every 4th index (RGBA), and any pixel address with a non-zero alpha is stored as a hash key for fast lookups during mouse moves. It still overloads mobile browsers and Firefox on Linux for 10+ seconds.
I read about a technique where all shapes would be drawn to one ghost canvas using color to differentiate which shape owned each pixel. I was really happy with this idea, because it should theoretically be able to differentiatate between millions of shapes.
Unfortunately, this is broken by anti-aliasing, which cannot be disabled on most canvas implementations. Each fuzzy edge creates dozens of colors which might be safely ignored except that /they can blend/ with overlapping shape edges. The last thing I want to happen when someone crosses the mouse over a shape boundary is to fire semi-random mouseover events for unrelated shapes associated with colors that have emerged from the blending due to AA.
I know that this not a new problem for video game developers and there must be fast algorithms for this kind of thing. If anyone is aware of an algorithm that can resolve (realistically) hundreds of shapes without occupying the CPU for more than a few seconds or blowing up RAM consumption dramatically, I would be very grateful.
There are two other Stack Overflow topics on mouseover detection, both of which discuss this topic, but they go no further than the 3 methods I describe.
Detect mouseover of certain points within an HTML canvas? and
mouseover circle HTML5 canvas.
EDIT: 2011/10/21
I tested another method which is more dynamic and doesn't require storing anything, but it's crippled by a performance problem in Firefox. The method is basically to loop over the shapes and: 1) clear 1x1 pixel under mouse, 2) draw shape, 3) get 1x1 pixel under mouse. Surprisingly this works very well in Chrome and IE, but miserably under Firefox.
Apparently Chrome and IE are able to optimize if you only want a small pixel area, but Firefox doesn't appear to be optimizing at all based on the desired pixel area. Maybe internally it gets the entire canvas, then returns your pixel area.
Code and raw output here: http://pastebin.com/aW3xr2eB.
If I understand the question correctly, you want to detect when the mouse enters/leaves a shape on the canvas, correct?
If so, then you can use simple geometric calculations, which are MUCH simpler and faster than looping over pixel data. Your rendering algorithm already has a list of all visible shapes, so you know the position, dimension and type of each shape.
Assuming you have some kind of list of shapes, similar to what #Benjammmin' is describing, you can loop over the visible shapes and do point-inside-polygon checks:
// Track which shape is currently under the mouse cursor, and raise
// mouse enter/leave events
function trackHoverShape(mousePos) {
var shape;
for (var i = 0, len = visibleShapes.length; i < len; i++) {
shape = visibleShapes[i];
switch (shape.type ) {
case 'arc':
if (pointInCircle(mousePos, shape) &&
_currentHoverShape !== shape) {
raiseEvent(_currentHoverShape, 'mouseleave');
_currentHoverShape = shape;
raiseEvent(_currentHoverShape, 'mouseenter');
return;
}
break;
case 'rect':
if (pointInRect(mousePos, shape) &&
_currentHoverShape !== shape) {
raiseEvent(_currentHoverShape, 'mouseleave');
_currentHoverShape = shape;
raiseEvent(_currentHoverShape, 'mouseenter');
}
break;
}
}
}
function raiseEvent(shape, eventName) {
var handler = shape.events[eventName];
if (handler)
handler();
}
// Check if the distance between the point and the shape's
// center is greater than the circle's radius. (Pythagorean theroem)
function pointInCircle(point, shape) {
var distX = Math.abs(point.x - shape.center.x),
distY = Math.abs(point.y - shape.center.y),
dist = Math.sqrt(distX * distX + distY * distY);
return dist < shape.radius;
}
So, just call the trackHoverShape inside your canvas mousemove event and it will keep track of the shape currently under the mouse.
I hope this helps.
From comment:
Personally I would just switch to using SVG. It's more what it was
made for. However it may be worth looking at EaselJS
source. There's a method Stage.getObjectUnderPoint(), and their demo's
of this seem to work perfectly fine.
I ended up looking at the source, and the library utilises your first approach - separate hidden canvas for each object.
One idea that came to mind was attempting to create some kind of a content-aware algorithm to detect anti-aliased pixels and with what shapes they belong. I quickly dismissed this idea.
I do have one more theory, however. There doesn't seem to be a way around using ghost canvases, but maybe there is a way to generate them only when they're needed.
Please note the following idea is all theoretical and untested. It is possible I may have overlooked something that would mean this method would not work.
Along with drawing an object, store the method in which you drew that object. Then, using the method of drawing an object you can calculate a rough bounding box for that object. When clicking on the canvas, run a loop through all the objects you have on the canvas and extract ones which bounding boxes intercept with the point. For each of these extracted objects, draw them separately onto a ghost canvas using the method reference for that object. Determine if the mouse is positioned over a non-white pixel, clear the canvas, and repeat.
As an example, consider I have drawn two objects. I will store the methods for drawing the rectangle and circle in a readable manner.
circ = ['beginPath', ['arc', 75, 75, 10], 'closePath', 'fill']
rect = ['beginPath', ['rect', 150, 5, 30, 40], 'closePath', 'fill']
(You may want to minify the data saved, or use another syntax, such as the SVG syntax)
As I am drawing these circles for the first time, I will also keep note of the dimensional values and use them to determine a bounding box (Note: you will need to compensate for stroke widths).
circ = {left: 65, top: 65, right: 85, bottom: 85}
rect = {left: 150, top: 5, right: 180, bottom: 45}
A click event has occurred on the canvas. The mouse point is {x: 70, y: 80}
Looping through the two objects, we find that the mouse coordinates fall within the circle bounds. So we mark the circle object as a possible candidate for collision.
Analysing the circles drawing method, we can recreate it on a ghost canvas and then test if the mouse coordinates fall on a non-white pixel.
After determining if it does or does not, we can clear the ghost canvas to prepare for any more objects to be drawn on it.
As you can see this removes the need to store 960 x 800 x 100 pixels and only 960 x 800 x2 at most.
This idea would best be implemented as some kind of API for automatically handling the data storage (such as the method of drawing, dimensions...).
Related
I have a situation that I need help with. I have one big plane that is separated into 300k+ square tiles (and sometimes bezier curves). The whole plane is visible in the screen.
If I do a canvas draw only once, the rendering takes a bit of time (1-2s in a modern laptop) but it is doable. However, I need a way to select certain tiles by mousing over them, and selecting them (multiple with drag and drop). I have the logic for mouse position in place and I can find which tile is in which position by redrawing the scene and doing something like:
if (!found && context.isPointInPath(mouseX, mouseY)) {
context.fillStyle = 'rgb(255, 0, 0);'
found = true; // for most scenarios, this stops the expensive operations that is isPointInPath.
} else {
context.fillStyle = 'rgb(240, 240, 240);'
}
The problem arises when I try to find the hovered tile in 300k+ tile space because the time it takes to find the mouse is very slow (3-4 seconds) and the lag gives a bad user experience.
Currently, I am toying around with creating a tree structure where non-leaf nodes of the tree are bigger size nodes (e.g first node splits the plane into four shapes, the next one splits the smaller ones into four shapes; kind of like a Quadtree structure). However, I have been confused about how to split this tree into this many pieces when I have bezier curves. I can't just split the tree into four squares because some of the tiles will overlap. My main question is how to handle those edge cases where tiles overlap in between two parent nodes.
Or a bit more general question. Is there a better way to manage this problem than going with tree route?
I'm writing a game using HTML5/WinJS on Windows 8. I'm trying to produce the effect of a bullet or missile firing at something; however, I can't seem to get the object to go through another image in the background without trailing a border. My working theory is that the border I'm seeing is caused by using clearRect. Here's my code:
var moveBullet = function(missile) {
if (missile.Image == undefined || missile.Image == null) {
var image = new Image();
image.src = "/images/missileImg.png";
image.onload = function () {
canvasContext.clearRect(missile.PointX - (image.width / 2), missile.PointY, image.width, image.height);
missile.PointY += BULLET_SPEED;
canvasContext.drawImage(image, missile.PointX - (image.width / 2), missile.PointY);
};
} else {
canvasContext.clearRect(missile.PointX - (missile.Image.width / 2), missile.PointY, missile.Image.width, missile.Image.height);
missile.PointY += BULLET_SPEED;
canvasContext.drawImage(missile.Image, missile.PointX - (missile.Image.width / 2), missile.PointY);
}
}
Is there a way to achieve this effect without using clearRect, or a more efficient way of restoring the background as it moves past?
Make your clearRect area a few pixels larger than the missile image. Drawing on a canvas in general has some built-in anti-aliasing. This means that if you draw a line with one color, then draw the same line with the background color, you'll not remove the original line. Something similar might be happening here, in which case a few extra pixels should help.
That said, there's a caveat to be aware of here.
First, I assume the background is separate element from the canvas? It looks like it as you're not redrawing that part on the canvas itself.
The reason I ask is that making repeated calls to clearRect on the same canvas will eventually show performance problems. What happens is that every call to clearRect accumulates into a complex region within the canvas--essentially its transparency mask. So every time the canvas has to be rendered, which happens any time you change it, it has to process that transparent area. Gradually, as you leave more and more small clearRect trails across the canvas, this region will become more and more complex and performance will drop.
I did this experiment with the Blizzard demo on the IE Test Drive site once, where I wondered why the demo was clearing the entire canvas with every animation frame. So I tried just clearing the trail behind each snowflake (and made each one a little bigger as I suggest above, because I had trails). Seemed like the right thing to do, but the performance plummeted by several orders of magnitude. Asking around within the IE team, they confirmed the region behavior I describe.
So the best thing to do, actually, is to do a clearRect on the entire canvas with every frame, then redraw the missile and any other bits that you're animating. This may seem counter intuitive, but ends up working best and avoids all these glitches with pixel trails.
I'm working on a 2d canvas-based app using EaselJS where the user can move indefinitely on the xy-plane by dragging the background. Think google maps, except the background is a repeating tile.
The code for the movement is very simple and looks something like this:
// container - the createjs.Container being panned
// background - a createjs.Shape child of container, on which the
// background is drawn
background.onPress = function(evt) {
var x = evt.stageX, y = evt.stageY;
evt.onMouseMove = function(evt) {
// the canvas is positioned in the center of the window, so the apparent
// movement works by changing the registration point of the container in
// the opposite direction.
container.regX -= evt.stageX - x;
container.regY -= evt.stageY - y;
x = evt.stageX;
y = evt.stageY;
stage.update();
};
evt.onMouseUp = function(evt) {
// Here the background would be redrawn based on the new container coords.
// However the issue occurs even before the mouse is released.
background.redraw();
stage.update();
};
};
All works as expected until reaching 32678px (2^15) on either axis. What occurs is different in different browsers, but the point where it first happens is the same.
In Firefox, it will suddenly shift a large chunk of pixels (~100) rather than 1. It will then happen again at 65538 (2^16+2), perhaps more after that, but I haven't witnessed it. After the trouble points, the drag will continue smoothly, as expected, but remaining shifted.
In Chrome, the effect is more dramatic. The drawing breaks and results in repeated ~100px wide "stripes" of the background across the page at 32768, and does not correct itself on redraw.
Oddly, the numbers in EaselJS don't reflect the issue. The only change in the container's transform matrix is the tx or ty incrementing by 1. No other matrices change. It seems as though EaselJS is getting all the numbers right.
Can anyone shed any light this issue?
Edit:
I worked around this problem by redrawing parts of the container using a calculated regX/regY, rather than attempting to translate very large regX/regY coords on the canvas.
This question may be related
What is the maximum size for an HTML canvas?
From what I gather browsers use short int to store canvas sizes with 32,767 being the maximum possible value.
Links possibly related to your issue,
Mozilla - HTML5 Canvas slow/out of memory
https://stackoverflow.com/a/12535564/149636
I'm making a top-down shooter game that relies on the avatar always being rotated pointing to the mouse cursor. I achieve rotation like this:
//Rendering.
context.save(); //Save the context state, we're about to change it a lot.
context.translate(position[0] + picture.width/2, position[1] + picture.height/2); //Translate the context to the center of the image.
context.rotate(phi); //Rotate the context by the object's phi.
context.drawImage(picture.image, -picture.width/2, -picture.height/2); //Draw the image at the appropriate position (center of the image = [0, 0]).
context.restore(); //Get the state back.
When the phi is zero, the image is rendered in its normal quality, with sharp edges and detectable pixels. But, when I set the phi to a nonzero value (actually, when it's not 0, Pi/2, Pi, Pi+Pi/2 or 2Pi), the image looses it's sharpness and the individual pixels can't be seen anymore, because they are blurred out.
Here's a screenshot (sorry about the general bad quality of the screenshot, but I think that the difference is more than noticeable):
This is, well, a bit unacceptable. I can't have the images always blurred out! Why is this happening and can I solve it?
You could try
context.imageSmoothingEnabled = false;
See docs:
context.imageSmoothingEnabled [ = value ]
Returns whether pattern fills and the drawImage() method will attempt to smooth images if they have to rescale them (as opposed to just rendering the images with "big pixels").
Can be set, to change whether images are smoothed (true) or not (false).
If you want a true pixel-art retro style effect, you'd need to manually create rotated sprite images for several angles, look up the appropriate sprite for the current value of phi, and draw it without rotation. This obviously requires a fair amount of art work!
IF you are rotating images around their center point, make sure the image itself has an even number of pixels. Once you end up on odd coordinates the image data needs to be interpolated for the target canvas. Apple has some nice documentation on translating and rotating the canvas.
So for any image, as suggested above use rounding to snap to full pixels.
context.translate(Math.floor(img.width/2), Math.floor(img.height/2));
This way every source pixel of your image will always be drawn exactly into a pixel inside the canvas and blurring does not occur. This however is only true for multiples of 90 degrees.
It seems that all browsers do, to some extend, antialiasing in image drawing so you will probably have to provide rotated images as sprites.
According to this Chromium bug report you might be lucky there if they haven't fixed it yet. Read through and you'll learn that Ian Hickson likely opposed making antialiased image drawing optional.
(picture.width/2, picture.height/2) point won't always work.
(Math.floor(picture.width/2) + 0.5, Math.floor(picture.height/2) + 0.5) should help.
Well, actually it is something you cannot get around
If you rotate an image by a multiple of 90 degrees, your library should smart enough so that no interpolation is applied.
But as soon as you rotate an image by an angle different from a multiple of 90 degrees, you need to interpolate. As a consequence, you get that smoothing. If you are interested in the theory, you may look for a book on computer graphics or image processing.
For the concrete case of image rotation you may have a look at this paper,
http://bigwww.epfl.ch/publications/unser9502.html
I've built an analytical data visualization engine for Canvas and have been requested to add tooltip-like hover over data elements to display detailed metrics for the data point under the cursor.
For simple bar & Gaant charts, tree graphs and node maps with simple square areas or specific points of interest, I was able to implement this by overlaying absolutely-positioned DIVs with :hover attributes, but there are some more complicated visualizations such as pie charts and a traffic flow rendering which has hundreds of separate areas defined by bezeir curves.
Is is possible to somehow attach an overlay, or trigger an event when the user mouses over a specific closed path?
Each area for which hover needs to be specified is defined as follows:
context.beginPath();
context.moveTo(segmentRight, prevTop);
context.bezierCurveTo(segmentRight, prevTop, segmentLeft, thisTop, segmentLeft, thisTop);
context.lineTo(segmentLeft, thisBottom);
context.bezierCurveTo(segmentLeft, thisBottom, segmentRight, prevBottom, segmentRight, prevBottom);
/*
* ...define additional segments...
*/
// <dream> Ideally I would like to attach to events on each path:
context.setMouseover(function(){/*Show hover content*/});
// </dream>
context.closePath();
Binding to an object like this is almost trivial to implement in Flash or Silverlight, since but the current Canvas implementation has the advantage of directly using our existing Javascript API and integrating with other Ajax elements, we are hoping to avoid putting Flash into the mix.
Any ideas?
You could handle the mousemove event and get the x,y coordinates from the event. Then you'll probably have to iterate over all your paths to test if the point is over the path. I had a similar problem that might have some code you could use.
Looping over things in this way can be slow, especially on IE. One way you could potentially speed it up - and this is a hack, but it would be quite effective - would be to change the color that each path is drawn with so that it is not noticeable by humans but so that each path is drawn in a different color. Have a table to look up colors to paths and just look up the color of the pixel under the mouse.
Shadow Canvas
The best method I have seen elsewhere for mouseover detection is to repeat the part of your drawing that you want to detect onto a hidden, cleared canvas. Then store the ImageData object. You can then check the ImageData array for the pixel of interest and return true if the alpha value is greater than 0.
// slow part
ctx.clearRect(0,0,canvas.width,canvas.height);
ctx.fillRect(100,100,canvas.width-100,canvas.height-100);
var pixels = ctx.getImageData(0,0,canvas.width,canvas.height).data;
// fast part
var idx = 4 * (mouse_x + mouse_y * canvas.width) + 3;
if (pixels[idx]) { // alpha > 0
...
}
Advantages
You can detect anything you want since you're just repeating the context methods. This works with PNG alpha, crazy compound shapes, text, etc.
If your image is fairly static, then you only need to do this one time per area of interest.
The "mask" is slow, but looking up the pixel is dirt cheap. So the "fast part" is great for mouseover detection.
Disadvantages
This is a memory hog. Each mask is W*H*4 values. If you have a small canvas area or few areas to mask, it's not that bad. Use chrome's task manager to monitor memory usage.
There is currently a known issue with getImageData in Chrome and Firefox. The results are not garbage collected right away if you nullify the variable, so if you do this too frequently, you will see memory rise rapidly. It does eventually get garbage collected and it shouldn't crash the browser, but it can be taxing on machines with small amounts of RAM.
A Hack to Save Memory
Rather than storing the whole ImageData array, we can just remember which pixels have alpha values. It saves a great deal of memory, but adds a loop to the mask process.
var mask = {};
var len = pixels.length;
for (var i=3;i<len;i+=4) if ( pixels[i] ) mask[i] = 1;
// this works the same way as the other method
var idx = 4 * (mouse_x + mouse_y * canvas.width) + 3;
if (mask[idx]) {
...
}
This could be done using the method ctx.isPointInPath, but it is not implemented in ExCanvas for IE.
But another solution would be to use HTML maps, like I did for this little library : http://phenxdesign.net/projects/phenx-web/graphics/example.htm you can get inspiration from it, but it is still a little buggy.
I needed to do detect mouse clicks for a grid of squares (like cells of an excel spreadsheet). To speed it up, I divided the grid into regions recursively halving until a small number of cells remained, for example for a 100x100 grid, the first 4 regions could be the 50x50 grids comprising the four quadrants.
Then these could be divided into another 4 each (hence giving 16 regions of 25x25 each).
This requires a small number of comparisons and finally the 25x25 grid could be tested for each cell (625 comparisons in this example).
There is a book by Eric Rowell named "HTML5 CANVAS COOKBOOK". In that book there is a chapter named "Interacting with the Canvas: Attaching Event Listeners to Shapes and Regions". mousedown, mouseup, mouseover, mouseout, mousemove, touchstart, touchend and touchmove events can be implemented. I highly suggest you read that.
This can't be done (well, at least not that easily), because objects you draw on the canvas (paths) are not represented as the same objects in the canvas. What I mean is that it is just a simple 2D context and once you drawn something on it, it completely forgets how it was drawn. It is just a set of pixels for it.
In order to watch mouseover and the likes for it, you need some kind of vector graphics canvas, that is SVG or implement your own on top of existing (which is what Sam Hasler suggested)
I would suggest overlaying an image map with proper coordinates set on the areas to match your canvas-drawn items. This way, you get tooltips AND a whole lot of other DOM/Browser functionality for free.