Develop Reference

How to trace / track user input on touch screen to match image on screen

So I want to have an image, let's say a square and would want the user to 'trace' the image with their finger, I need to track this and know / understand if the user has done it correctly.
The use case is educational software where users trace shapes to learn how to draw them.
My thinking was an SVG object and then mouse hold events but because SVG has beginning and end points I am not sure if they could then be tracked all the way over the image etc.
Also how? If I have interaction on the SVG is it a matter of if statements and some kind of variance on the line and if user gets to far away from the original line then stop / break?
Sorry if this is explained badly also I couldn't find almost anything so I'm also sorry if this is a duplicate.
Found this article: https://www.smashingmagazine.com/2018/05/svg-interaction-pointer-events-property/
And: https://gist.github.com/elidupuis/11325438 / http://bl.ocks.org/elidupuis/11325438
So could maybe cobble something together but yes any direction would be appreciated.

var xmlns = "http://www.w3.org/2000/svg";
function setMouseCoordinates(event) {
// contains the size of element having id 'image svg' and its position relative to the viewport, svg width/height equal to image height and width
var boundary = document.getElementById('<id_of_image_svg>').getBoundingClientRect();
// sets the x position of the mouse co-ordinate
auth_mouseX = event.clientX - boundary.left;
// sets the y position of the mouse co-ordinate
auth_mouseY = event.clientY - boundary.top;
return [auth_mouseX, auth_mouseY];
}
// add this in mousedown or respective touch event
function drawPath(event) {
var {
auth_mouseX,
auth_mouseY
} = setMouseCoordinates(event);
// Creates an element with the specified namespace URI and qualified name.
scribble = document.createElementNS(xmlns, 'path');
// sets the stroke width and color of the drawing drawn by scribble drawing tool
scribble.style.stroke = 'red';
// sets the stroke width of the scribble drawing
scribble.style.strokeWidth = '2';
scribble.style.fill = 'none';
scribble.setAttributeNS(null, 'd', 'M' + auth_mouseX + ' ' + auth_mouseY);
}
Now you can add this function in the touchdown event and create logic accordingly. You have to check that the user has start or stopped the drawing by a varible and changing its value accordingly.
For evaluation of task whether it is in a given area or not you have to use some shape detection api or AI (You can search on google there are many of them like AWS rekognition and many others I don't remember the name)
Hope it helps anyhow. :)

How do I translate mouse movement distances to SVG coordinate space?

I have an SVG visualization of the distribution of CSS4 color keywords in HSL space here: https://meyerweb.com/eric/css/colors/hsl-dist.html
I recently added zooming via the mouse wheel, and panning via mouse clack-and-drag. I’m able to convert a point from screen space to SVG coordinate space using matrixTransform, .getScreenCTM(), and .inverse() thanks to example code I found online, but how do I convert mouse movements during dragging? Right now I’m just shifting the viewBox coordinates by the X and Y values from event, which means the image drag is faster than the mouse movement when zoomed in.
As an example, suppose I’m zoomed in on the image and am dragging to pan, and I jerk the mouse leftwards and slightly downwards. event.movementX returns -37 and event.movementY returns 6. How do I determine how far that equates to in SVG coordinates, so that the viewBox coordinates are shifted properly?
(Note: I’m aware that there are libraries for this sort of thing, but I’m intentionally writing vanilla JS code in order to learn more about both SVG and JS. So please, don’t post “lol just use library X” and leave it at that. Thanks!)
Edited to add: I was asked to post code. Posting the entire JS seems overlong, but this is the function that fires on mousemove events:
function dragger(event) {
var target = document.getElementById('color-wheel');
var coords = parseViewBox(target);
coords.x -= event.movementX;
coords.y -= event.movementY;
changeViewBox(target,coords);
}
If more is needed, then view source on the linked page; all the JS is at the top of the page. Nothing is external except for a file that just contains all the HSL values and color names for the visualization.

My recommendation:
Don't worry about the movementX/Y properties on the event.
Just worry about where the mouse started and where it is now.
(This has the additional benefit that you get the same result even if you miss some events: maybe because the mouse moved out of the window, or maybe because you want to group events so you only run the code once per animation frame.)
For where the mouse started, you measure that on the mousedown event.
Convert it to a position in the SVG coordinates, using the method you were using,
with .getScreenCTM().inverse() and .matrixTransform().
After this conversion, you don't care where on the screen this point is. You only care about where it is in the picture. That's the point in the picture that you're always going to move to be underneath the mouse.
On the mousemove events, you use that same conversion method to find out where the mouse currently is within the current SVG coordinate system. Then you figure out how far that is from the point (again, in SVG coordinates) that you want underneath the mouse. That's the amount that you use to transform the graphic. I've followed your example and am doing the transform by shifting the x and y parts of the viewBox:
function move(e) {
var targetPoint = svgCoords(event, svg);
shiftViewBox(anchorPoint.x - targetPoint.x,
anchorPoint.y - targetPoint.y);
}
You can also shift the graphic around with a transform on a group (<g> element) within the SVG; just be sure to use that same group element for the getScreenCTM() call that converts from the clientX/Y event coordinates.
Full demo for the drag to pan. I've skipped all your drawing code and the zooming effect.
But the zoom should still work, because the only position you're saving in global values is already converted into SVG coordinates.
var svg = document.querySelector("svg");
var anchorPoint;
function shiftViewBox(deltaX, deltaY) {
svg.viewBox.baseVal.x += deltaX;
svg.viewBox.baseVal.y += deltaY;
}
function svgCoords(event,elem) {
var ctm = elem.getScreenCTM();
var pt = svg.createSVGPoint();
// Note: rest of method could work with another element,
// if you don't want to listen to drags on the entire svg.
// But createSVGPoint only exists on <svg> elements.
pt.x = event.clientX;
pt.y = event.clientY;
return pt.matrixTransform(ctm.inverse());
}
svg.addEventListener("mousedown", function(e) {
anchorPoint = svgCoords(event, svg);
window.addEventListener("mousemove", move);
window.addEventListener("mouseup", cancelMove);
});
function cancelMove(e) {
window.removeEventListener("mousemove", move);
window.removeEventListener("mouseup", cancelMove);
anchorPoint = undefined;
}
function move(e) {
var targetPoint = svgCoords(event, svg);
shiftViewBox(anchorPoint.x - targetPoint.x,
anchorPoint.y - targetPoint.y);
}
body {
display: grid;
margin: 0;
min-height: 100vh;
}
svg {
margin: auto;
width: 70vmin;
height: 70vmin;
border: thin solid gray;
cursor: move;
}
<svg viewBox="-40 -40 80 80">
<polygon fill="skyBlue"
points="0 -40, 40 0, 0 40 -40 0" />
</svg>

So the script needs something so that the vectors moved by the SVG are coordinated against the vectors moved by the mouse on screen. Despite the event being on your target, your SVG, the MouseEvent properties relate to your screen alone.
The movementX read-only property of the MouseEvent interface provides the difference in the X coordinate of the mouse pointer between the given event and the previous mousemove event. In other words, the value of the property is computed like this: currentEvent.movementX = currentEvent.screenX - previousEvent.screenX.
From https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent/movementX
The screenX read-only property of the MouseEvent interface provides the horizontal coordinate (offset) of the mouse pointer in global (screen) coordinates.
So what you're measuring, and to the best of my knowledge the only thing you can measure direcly without additional libraries or complication, is the movement of the pointer in pixel terms across the screen. The only way to make this work in terms of vector for movement of your SVG is to translate the on screen movement to the dimensions that are relevant to your scaled SVG.
My initial thinking was that you would be able to work out the scaling of the SVG object, using some combination of its viewbox and its actual width on the screen. Naturally what would initially appear sensible is not. This approach won't work, if it appears to it would be purely by chance.
But it turns out that the solution is essentially to use the same type of code you've used in your scaling when you approach your mouse movements. The .getScreenCTM() and .inverse() functions are exactly what you'll need again. But instead of trying to find a single point on the SVG to work from, you need to find out what the on-screen distance translates to in the SVG by comparing two points on the SVG instead.
What I provide here isn't necessarily the most optimal solution but hopefully helps explain and gives you something to work further from...
function dragger(event) {
var target = document.getElementById('color-wheel');
var coords = parseViewBox(target);
//Get an initial point in the SVG to start measuring from
var start_pt = target.createSVGPoint();
start_pt.x = 0;
start_pt.y = 0;
var svgcoord = start_pt.matrixTransform(target.getScreenCTM().inverse());
//Create a point within the same SVG that is equivalent to
//the px movement by the pointer
var comparison_pt = target.createSVGPoint();
comparison_pt.x = event.movementX;
comparison_pt.y = event.movementY;
var svgcoord_plus_movement = comparison_pt.matrixTransform(target.getScreenCTM().inverse());
//Use the two SVG points created from screen position values to determine
//the in-SVG distance to change coordinates
coords.x -= (svgcoord_plus_movement.x - svgcoord.x);
//Repeat the above, but for the Y axis
coords.y -= (svgcoord_plus_movement.y - svgcoord.y);
//Deliver the changes to the SVG to update the view
changeViewBox(target,coords);
}
Sorry for the long winded answer, but hopefully it explains it from the beginning enough that anyone else looking to find an answer can get the whole picture if they've not come as far as you have in this script.

From MouseEvent, we have clientX and movememntX. Taken together, we can deduce our last location. We can then take the transform of our current location and subtract it from the transform of our last location:
element.onpointermove = e => {
const { clientX, clientY, movementX, movementY } = e;
const DOM_pt = svg.createSVGPoint();
DOM_pt.x = clientX;
DOM_pt.y = clientY;
const { x, y } = DOM_pt.matrixTransform(svgs[i].getScreenCTM().inverse());
DOM_pt.x += movementX;
DOM_pt.y += movementY;
const { x: last_x, y: last_y } = DOM_pt.matrixTransform(svgs[i].getScreenCTM().inverse());
const dx = last_x - x;
const dy = last_y - y;
// TODO: use dx & dy
}

How to make something happen ONLY when clicking in an exact coordinate

I just wanted to know how I could make something on a canvas change clicking ONLY on a specific area of the canvas.
Let's say my canvas is of width 400px and height 400px, and I am drawing a face.
If I click on an eye, I want it to change. Let's say the eye is in the coordinates (drawn with .arc()) (135,70,15,0,Math.PI*2,true).
How can I make it so that when I click anywhere inside said eye, and nowhere else, something happens?
Thanks a lot!!

You need to manually compute if the mouse is in the eye. The best, simple and most precise solution would be to use ctx.isPointInPath. This API will tell you if a point is in the current shape you are drawing. Example :
ctx.arc(135,70,15,0,Math.PI*2,true)
if(ctx.isPointInPath(your_mouse.x, your_mouse.y)) {
doSomething()
}
Else you can do some hit-testing, but that's way more complicated depending on the shape of your object. Example for a rectangle :
if(
mouse.x > rect.x && mouse.y > rect.y &&
mouse.x < (rect.x + rect.width) && mouse.y < (rect.y + rect.height)
) {
doSomething()
}

You need to implement hit testing.
Keep track of your drawn objects in a Scene Graph - a kind of DOM that contains objects and their positions.
When you click somewhere you need to compare the position of the click with the the positions of your objects.
If the click point lies within the area of a drawn object, you consider that a hit.
That or just ahead and use a canvas library that keeps such a Scene Graph for you, such as Paper.js or Fabric.js - both libraries allow for hit-testing as well.

Is there a way of clicking an specific point/element in a canvas (with javascript, jquery or selenium)

How can I apply a click at a specific position in a canvas? (using coordinates seems the most logical to me, but can't find any way to do it, any other idea is welcomed). Note that I do not have access to the code that creates the canvas.
EDIT: Some clarification, the canvas has multiple elements being drawn that i can't select with jquery but need to click them. I could find their coordinates manually and do some calculations, but every time i try to pass a click or mouse event into that position is not being taken as a real mouse click (the button that should be clicked, is not).

How about trying
canvas.on('click', function(e) {
// calculate x y coordinates on canvas
var x = e.pageX - $(this).offset().left,
y = e.pageY - $(this).offset().top;
// implement collision detection via the coordinates the element you want to click (assuming you know where it is)
if (y > <elementY> && y < <elementY> + <elementHeight>
&& x > <elementX> && x < <elementX> + <elementWidth>) {
alert('found it');
}
});
I have no time testing that snippet. If you know the x and y coordinates of your elements within your canvas, that code might just do the trick.
What I tried to do with that snippet is to simulate a bounding box via elmentY and elementY + elementHeight as well as elementX and elementX + elementWidth. If you know all those values, you can simply check whether the x and y values of your click event fall in that said box.
Edit
If you want to trigger a click event at a certain position, define the clickEvent beforehand and set the pageX and pageY attributes.
var e = new jQuery.Event("click");
e.pageX = 10;
e.pageY = 10;
$(<canvas>).trigger(e);
You might have to include the offset of your canvas in the calculations as well.

Html5 canvas hittest arbitrary shape

I'm trying to develop program which can render images and text in canvas. I tried to handle click on image in canvas, but it work for rectable images.
My question: Did you know solution or frameworks for handle click on visible part of image (not transparent part) in canvas ?
I'm looking for javascript implementation of ActionScript hitTestObject function, if someone familiar with it.
Here is simple algorithm with rectable hit text: http://jsfiddle.net/V92Gn/298/
var hitted = e.clientX >= position.x &&
e.clientX <= position.x + logoImg.width &&
e.clientY >= position.y &&
e.clientY <= position.y + logoImg.height;

Solution using pure JavaScript + canvas
For cases where the hit target is mixed with the background you can do two things:
Create a separate canvas which is stacked on top of the main canvas, draw the target object on that and do the testing of that canvas.
Create an off-screen canvas which contains the target object isolated and test on that
In this example I will use an off-screen canvas.
What we do is to basically replicate the image by creating an off-screen canvas the size of the image and draw in the image when loaded. This will protect the background and keep it transparent no matter what is on the main canvas:
Modified fiddle here
/// create an off-screen canvas to hold image
var ocanvas = document.createElement('canvas');
var octx = ocanvas.getContext('2d');
logoImg.onload=function(){
/// set off-screen canvas to same size as image
ocanvas.width = this.width;
ocanvas.height = this.height;
octx.drawImage(this, 0, 0);
... rest of code
Then using your existing code but with an adjustment for mouse position we can use the hitted variable you use to first check if we are inside the target object.
$(canvas).on('mousemove', function(e) {
/// correct mouse position so it becomes relative to canvas
var r = canvas.getBoundingClientRect(),
x = e.clientX - r.left,
y = e.clientY - r.top;
var hitted = x >= position.x && x <= position.x + logoImg.width &&
y >= position.y && y <= position.y + logoImg.height;
Now that we know we are inside the rectangle we can extract a pixel from the off-screen canvas by compensating for the object position:
if (hitted === true) {
/// extract a single pixel at the adjusted position
var pixel = octx.getImageData(x - position.x, y - position.y, 1, 1).data;
/// set hitted again based on alpha value is 0 or not
hitted = pixel[3] > 0;
}
...
As you can see in the modified fiddle we only change the class you use to red when we are on an actual pixel in the target no matter what the background of it is (when drawn separately).
Finally a couple of words on CORS: In this case, as you use DropBox, you can request CORS usage of the image simply by activating the crossOrigin property on the image object before you set the source:
logoImg.crossOrigin = ''; /// this is the same as setting anonymous
logoImg.src="http://dl.dropbox.com/...";
As DropBox (and image sharing sites such as ImgUr.com) support CORS usage they will allow the request and we can therefor extract pixels from the image.
If the servers didn't allow it however we wouldn't be able to do this. To be sure CORS is ok you should host the image in the same domain as the page when you release it.

I recommend using EaselJS which has a hitTest method similar to how it works in Actionscript:
myDisplayObject.hitTest(localX, localY);
Here you can find some demos that show the technique:
http://shallaazm.com/createjs/easeljs/tutorials/HitTest/