Develop Reference

Three.js manipulate 3D mouse coordinates by new direction

i have a problem at setting a new mouse position. I have an Object which is moving and rotating free in a Space and following the position of the pointerlocked cursor. Targetposition is a Vector3 positioned behind the object. Now i want to make an Animation which moves the object in the opposite direction by rotating it smoothly. So I disable my mouseListener and rotate the object with the given Three.js functions. But after I enable my listener the targetPosition turns back to the mouse position and makes the move useless. So now i want to calculate the mouse Position for my new Targetposition after the animation. How can i do that ?
The function for the 3D mouse coordinates looks something like this:
enter code here
mouseY = Math.max(-85, Math.min(70, -mouseY));
phi = THREE.Math.degToRad(90 + mouseY);
theta = THREE.Math.degToRad(mouseX);
targetPosition.x = object.position.x + 100 * Math.sin(phi) * Math.cos(theta);
targetPosition.y = object.position.y + 100 * Math.cos(phi);
targetPosition.z = object.position.z + 100 * Math.sin(phi) * Math.sin(theta);
object.lookAt(targetPosition);

HTML5 canvas get coordinates after zoom and translate

BACKGROUND: I have an HTML5 canvas and I have an image drawn on it. Now when the image is first loaded, it is loaded at a scale of 100%. The image is 5000 x 5000. And the canvas size is 600 x 600. So onload, I only see the first 600 x-pixels and 600 y-pixels. I have the option of scaling and translating the image on the canvas.
MY ISSUE: I am trying to figure out an algorithm that return the pixel coordinates of a mouse click relative to the image, not the canvas while taking into account scaling and translating.
I know there are a lot of topics already on this, but nothing I've seen has worked. My issue is when I have multiple translations and scaling. I can zoom once and get the correct coordinates, and I can then scale and get the right coordinates again, but once I zoom or scale more than once, the coordinates are off.
Here is what I have so far.
//get pixel coordinates from canvas mousePos.x, mousePos.y
(mousePos.x - x_translation)/scale //same for mousePos.y
annotationCanvas.addEventListener('mouseup',function(evt){
dragStart = null;
if (!dragged) {
var mousePos = getMousePos(canvas, evt);
var message1 = " mouse x: " + (mousePos.x) + ' ' + "mouse y: " + (mousePos.y);
var message = " x: " + ((mousePos.x + accX)/currentZoom*currentZoom) + ' ' + "y: " + ((mousePos.y + accY)/currentZoom);
console.log(message);
console.log(message1);
console.log("zoomAcc = " + zoomAcc);
console.log("currentZoom = " + currentZoom);
ctx.fillStyle="#FF0000";
ctx.fillRect((mousePos.x + accX)/currentZoom, (mousePos.y + accY)/currentZoom, -5, -5);
}
},true);
//accX and accY are the cumulative shift for x and y respectively, and xShift and xShift yShift are the incremental shifts of x and y respectively
where current zoom is the accumulative zoom. and zoomAcc is the single iteration of zoom at that point. So in this case, when I zoom in, zoomAcc is always 1.1, and currentZoom = currentZoom*zoomAcc.
Why is this wrong? if someone can please show me how to track these transformations and then apply them to mousePos.x and mousePos.y I would be grateful.
thanks
UPDATE:
In the image, the green dot is where I clicked, the red dot is where my calculation of that point is calculated, using markE's method. The m values are the matrix values in your markE's method.

When you command the context to translate and scale, these are known as canvas transformations.
Canvas transformations are based on a matrix that can be represented by 6 array elements:
// an array representing the canvas affine transformation matrix
var matrix=[1,0,0,1,0,0];
If you do context.translate or context.scale and also simultaneously update the matrix, then you can use the matrix to convert untransformed X/Y coordinates (like mouse events) into transformed image coordinates.
context.translate:
You can simultaneously do context.translate(x,y) and track that translation in the matrix like this:
// do the translate
// but also save the translate in the matrix
function translate(x,y){
matrix[4] += matrix[0] * x + matrix[2] * y;
matrix[5] += matrix[1] * x + matrix[3] * y;
ctx.translate(x,y);
}
context.scale:
You can simultaneously do context.scale(x,y) and track that scaling the matrix like this:
// do the scale
// but also save the scale in the matrix
function scale(x,y){
matrix[0] *= x;
matrix[1] *= x;
matrix[2] *= y;
matrix[3] *= y;
ctx.scale(x,y);
}
Converting mouse coordinates to transformed image coordinates
The problem is the browser is unaware that you have transformed your canvas coordinate system and the browser will return mouse coordinates relative to the browser window--not relative to the transformed canvas.
Fortunately the transformation matrix has been tracking all your accumulated translations and scalings.
You can convert the browser’s window coordinates to transformed coordinates like this:
// convert mouseX/mouseY coordinates
// into transformed coordinates
function getXY(mouseX,mouseY){
newX = mouseX * matrix[0] + mouseY * matrix[2] + matrix[4];
newY = mouseX * matrix[1] + mouseY * matrix[3] + matrix[5];
return({x:newX,y:newY});
}

There's a DOMMatrix object that will apply transformations to coordinates. I calculated coordinates for translated and rotated shapes as follows by putting my x and y coordinates into a DOMPoint and using a method of the DOMMatrix returned by CanvasRenderingContext2D.getTransform. This allowed a click handler to figure out which shape on the canvas was being clicked. This code apparently performs the calculation in markE's answer:
const oldX = 1, oldY = 1; // your values here
const transform = context.getTransform();
// Destructure to get the x and y values out of the transformed DOMPoint.
const { x, y } = transform.transformPoint(new DOMPoint(oldX, oldY));
DOMMatrix also has methods for translating and scaling and other operations, so you don't need to manually write those out anymore. MDN doesn't fully document them but does link to a page with the specification of non-mutating and mutating methods.

Three.js Click and Drag - Object Lagging Behind Pointer

I've created a near exact implementation of the Three.js Draggable Cubes example, except that my implementation ignores the z axis for movement in 2D. The example can be found here: http://threejs.org/examples/#webgl_interactive_draggablecubes
While I am able to click and move around my object, if I move the mouse too fast, the object will lag behind; the mouse pointer will move to it's location and then the object will follow the mouses path to the location. This issue is also noticeable in the Three.js example. Try dragging one of the cubes at anything beyond a moderate speed.
I've tried a few things to get the object to be directly under the mouse pointer but none have worked. The closest I think I might have come to a solution was by changing the mouse position update in the MouseMove event. However, it appears that the Three.js implementation of the code returns values between 1 and -1 rather than the screen X and Y coordinates, which leads me to wonder if the original implementation is causing the lag.
//Original Implementation - Works With Lag
mouse.x = (event.clientX / window.innerWidth) * 2 - 1;
mouse.y = -(event.clientY / window.innerHeight) * 2 + 1;
console.log(mouse.x + "," + mouse.y); //Output: -0.9729166666666667,0.8596858638743455
//My Implementation - Does Not Work
mouse.x = event.clientX - (window.innerWidth / 2);
mouse.y = event.clientY - (window.innerHeight / 2);
console.log(mouse.x + "," + mouse.y); //Output: -934,-410.5
//Update Function
void Update()
{
var vector = new THREE.Vector3(mouse.x, mouse.y, 1);
projector.unprojectVector(vector, camera);
var raycaster = new THREE.Raycaster(camera.position, vector.sub(camera.position).normalize());
var intersects = raycaster.intersectObject(plane);
meshToMove.position.x = intersects[0].point.x;
meshToMove.position.y = intersects[0].point.y;
}
//Main Loop
function Loop()
{
Update();
Render();
requestAnimationFrame(Loop);
}
Q: How can I update the three.js Draggable Cubes example so that objects don't lag behind the mouse during a click and drag?

ThreeJS: Screen position to camera position

I've seen a lot of posts people want to have the camera position to screen position. My question is how to do the contrary.
What I currently want to achieve is set the "door" position to a % of the screen, this calculation is ready, and I do have the final screen X, Y (px) position. The current Z offset = 250 of the camera.
I've found this code to convert camera position to screen position:
var vector = projector.projectVector(door.position.clone(), camera);
vector.x = (vector.x + 1) / 2 * window.innerWidth;
vector.y = -(vector.y - 1) / 2 * window.innerHeight;
To do the reversie I tried this, but does not give the result I expect:
var mouseX = ((perc.x) / window.innerWidth) * 2 - 1,
mouseY = -((perc.y) / window.innerHeight) * 2 + 1;
var vector = new THREE.Vector3(mouseX, mouseY, 1);
projector.unprojectVector(vector, camera);
return vector.sub(camera.position).normalize()
I have tried several ways and tried to Google, but didn't found an answer.
Q: How to convert screen position to camera position?

Well, as posted by Gero3, what you want to do is to create a plane that will cover all of your visible camera area (something like new THREE.PlaneGeometry(5000,5000); and then use raycaster to locate screen coordinated on that plane.
Here is an example from another similar question:
http://jsfiddle.net/PwWbT/
Hope that helps.

Use a large invisble plane that defines in which direction the door can expand and then use a raycaster on the plane to search for the position to where the door object should extend.

Adjusting camera for visible Three.js shape

I have a CubeGeometry which the camera is looking at, and I want the camera to zoom so that the cube is entirely visible, but no larger.
My initial attempt was to convert the cube verticies to the camera coordinate system,
function toScreenXY(position, camera) {
var pos = position.clone();
var projScreenMat = new THREE.Matrix4();
projScreenMat.multiply(camera.projectionMatrix, camera.matrixWorldInverse);
projScreenMat.multiplyVector3( pos );
return pos;
}
function ScaleInView() {
camera.fov = 0.0;
for (var i=0; i<8; i++) {
proj2d = toScreenXY(cube.geometry.vertices[i],camera);
angle = 57.296 * Math.max(Math.atan(proj2d.x/proj2d.z), Math.atan(proj2d.y/proj2d.z));
camera.fov = Math.max(camera.fov,angle);
}
camera.updateProjectionMatrix();
}
I thought this would work, but sometimes it's too small, and other times too large (depending on the position of the camera).
I also need to do this for Orthographic Camera.
Edit:
I know how to do this when the cube is facing the camera, I'm looking for a way to do it when the camera is moved to some arbitrary (r, theta, phi) position (spherical polar coordinates; r is actually constant for my purposes).

Perspective Camera. If the camera is centered and viewing the cube head-on, define
dist = distance from the camera to the front face ( important ! ) of the cube
and
height = height of the cube.
If you set the camera field-of-view as follows
fov = 2 * Math.atan( height / ( 2 * dist ) ) * ( 180 / Math.PI );
then the cube height will match the visible height.
Orthographic Camera. If the camera is centered and viewing the cube head-on, define
aspect = the aspect ratio of your window (i.e., width / height)
and
height = height of the cube.
Then construct your camera this way:
camera = new THREE.OrthographicCamera( -aspect * height/2, aspect * height/2, height/2, -height/2, near, far );
The cube height will match the visible height.
In either case, if the camera is not centered, or is otherwise viewing the cube at an angle, then the problem is more complicated.
Also, if the window is narrower than it is high, then the width is the constraining factor, and the problem is more complicated.

Multiplying by camera.matrixWorldInverse gives a vector in the camera's coordinates, but importantly does not apply perspective.
function toCameraCoords(position) {
return camera.matrixWorldInverse.multiplyVector3(position.clone());
}
We can then find the smallest angle that will fit all the box corners in the scene. arctan(D.x / D.z) gives the angle BCD where B is what the camera is looking at, C is the camera's position, and D the position of an object that you want to be visible in the camera coordinates.
In my case, the following ensures that the the cube boundbox is fully visible.
function ScaleInView() {
var tmp_fov = 0.0;
for (var i=0; i<8; i++) {
proj2d = toCameraCoords(boundbox.geometry.vertices[i]);
angle = 114.59 * Math.max( // 2 * (Pi / 180)
Math.abs(Math.atan(proj2d.x/proj2d.z) / camera.aspect),
Math.abs(Math.atan(proj2d.y/proj2d.z))
);
tmp_fov = Math.max(tmp_fov, angle);
}
camera.fov = tmp_fov + 5; // An extra 5 degrees keeps all lines visible
camera.updateProjectionMatrix();
}