Develop Reference

Three.js Using Raycaster to detect line and cone children of ArrowHelper

I have a functioning Raycaster for a simple painting app. I use it for a "bucket tool" in which the user can click on an object and change its color. It works for geometry objects such as BoxGeometry and CircleGeometry, but I'm struggling to apply it to the children of an ArrowHelper object. Because ArrowHelper isn't a shape and does not possess a geometry attribute, Raycaster does not detect collision with its position when checking scene.children for intersections. However, the children of ArrowHelper objects are always two things: a line and a cone, both of which have geometry, material, and position attributes.
I HAVE TRIED:
Toggling the recursive boolean of the function .intersectObjects(objects: Array, recursive: Boolean, optionalTarget: Array ) to true, so that it includes the children of the objects in the array.
Circumventing the ArrowHelper parent by iterating through scene.children for ArrowHelper objects and adding their lines and cones into a separate array of objects. From there I attempted to check for intersections with only the list of lines and cones, but no intersections were detected.
Raycaster setup:
const runRaycaster = (mouseEvent) => {
... // sets mouse and canvas bounds here
const raycaster = new THREE.Raycaster();
raycaster.setFromCamera(mouse, camera);
const intersects = raycaster.intersectObjects(scene.children, true);
if (intersects.length > 0) {
for (let i = 0; i < intersects.length; i++) {
// works for GEOMETRY ONLY
// needs modifications for checking ArrowHelpers
intersects[i].object.material.color.set(currentColor);
}
}
};
Here's my attempt to check the lines and cones individually, without the ArrowHelper parent:
let arrowObjectsList = [];
for (let i = 0; i < scene.children.length; i++) {
if (scene.children[i].type === 'ArrowHelper') {
arrowObjectsList.push(scene.children[i].line);
arrowObjectsList.push(scene.children[i].cone);
} else {
console.log(scene.children[i].type);
}
}
console.log(arrowObjectsList); // returns 2 objects per arrow on the canvas
// intersectsArrows always returns empty
const intersectsArrows = raycaster.intersectObjects(arrowObjectsList, true);
SOME NOTES:
Every ArrowHelper, its line, and its cone have uniquely identifiable names so they can be recolored/repositioned/deleted later.
The Raycaster runs with every onMouseDown and onMouseMove event.
Notably, the line and cone children of ArrowHelpers are BufferGeometry and CylinderBufferGeometry, respectively, rather than variations of Geometry. I'm wondering if this has anything to do with it. According to this example from the Three.JS documentation website, BufferGeometry can be detected by Raycaster in a similar fashion.

Setting recursion = true worked for me. Run the simple code below, and click on the arrow head. You will see the intersection information printed to the console. (three.js r125)
let W = window.innerWidth;
let H = window.innerHeight;
const renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true
});
document.body.appendChild(renderer.domElement);
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(28, 1, 1, 1000);
camera.position.set(5, 5, 5);
camera.lookAt(scene.position);
scene.add(camera);
const light = new THREE.DirectionalLight(0xffffff, 1);
light.position.set(0, 0, -1);
camera.add(light);
const mesh = new THREE.ArrowHelper(
new THREE.Vector3(0, 0, 1),
new THREE.Vector3(0, 0, 0),
2,
0xff0000,
1,
1
);
scene.add(mesh);
function render() {
renderer.render(scene, camera);
}
function resize() {
W = window.innerWidth;
H = window.innerHeight;
renderer.setSize(W, H);
camera.aspect = W / H;
camera.updateProjectionMatrix();
render();
}
window.addEventListener("resize", resize);
resize();
render();
// RAYCASTER STUFF
const raycaster = new THREE.Raycaster();
const mouse = new THREE.Vector2();
renderer.domElement.addEventListener('mousedown', function(e) {
mouse.set(
(event.clientX / window.innerWidth) * 2 - 1, -(event.clientY / window.innerHeight) * 2 + 1
);
raycaster.setFromCamera(mouse, camera);
const intersects = raycaster.intersectObjects(scene.children, true);
console.log(intersects);
});
html,
body {
width: 100%;
height: 100%;
padding: 0;
margin: 0;
overflow: hidden;
background: skyblue;
}
<script src="https://threejs.org/build/three.min.js"></script>

After a closer inspection, it was a matter of the set position, not necessarily the arrow. The position of the arrow varied based on user mouse click to specify the start point. However, it still presented several problems: It was very difficult to select the line because the lineWidth value of LineBasicMaterial cannot have any other value besides 1, despite being editable. This is due to a limitation in the OpenGL Core Profile, as addressed in the docs and in this question. Similarly, the cone would not respond to setLength. This limits the customization of the ArrowHelper tool pretty badly.
Because of this, I decided to entirely replace ArrowHelper with two objects coupled together: tubeGeometry and coneGeometry, both assigned a MeshBasicMaterial, in a way which can be accessed by Raycasters out of the box.
... // the pos Float32Array is set according to user mouse coordinates.
const v1 = new THREE.Vector3(pos[0], pos[1], pos[2]);
const v2 = new THREE.Vector3(pos[3], pos[4], pos[5]);
const material = new THREE.MeshBasicMaterial({
color: color,
side: THREE.DoubleSide,
});
// Because there are only two vectors, no actual curve occurs.
// Therefore, it's our straight line.
const tubeGeometry = new THREE.TubeBufferGeometry(
new THREE.CatmullRomCurve3([v1, v2]), 1, 3, 3, false);
const coneGeometry = new THREE.ConeGeometry(10, 10, 3, 1, false);
arrowLine = new THREE.Mesh(tubeGeometry, material);
arrowTip = new THREE.Mesh(coneGeometry, material);
// needs names to be updated later.
arrowLine.name = 'arrowLineName';
arrowTip.name = 'arrowTipName';
When placing the arrow, the user will click and drag to specify the start and end point of the arrow, so the arrow and its tip have to be updated with onMouseMove. We have to use Math.atan2 to get the angle in degrees between v1 and v2, with v1 as the center. Subtracting 90 orients the rotation to the default position.
... // on the onMouseMove event, pos is updated with new coords.
const setDirection = () => {
const v1 = new THREE.Vector3(pos[0], pos[1], pos[2]);
const v2 = new THREE.Vector3(pos[3], pos[4], pos[5]);
// copying the v2 pos ensures that the arrow tip is always at the end.
arrowTip.position.copy(v2);
// rotating the arrow tip according to the angle between start and end
// points, v1 and v2.
let angleDegrees = 180 - (Math.atan2(pos[1] - pos[4], pos[3] - pos[0]) * 180 / Math.PI - 90);
const angleRadians = angleDegrees * Math.PI / 180;
arrowTip.rotation.set(0, 0, angleRadians);
// NOT VERY EFFICIENT, but it does the job to "update" the curve.
arrowLine.geometry.copy( new THREE.TubeBufferGeometry(new THREE.CatmullRomCurve3([v1, v2]),1,3,3,false));
scene.add(arrowLine);
scene.add(arrowTip);
};
Out of the box, this "arrow" allows me to select and edit it with Raycaster without a problem. No worrying about line positioning, line thickness, or line length.

What is required to convert threejs perspective camera to orthographic

I have some code that converts a perspective camera to an orthographic camera. The problem is that when I make the conversion, the model becomes very tiny and hard to see.
I have calculated the zoom factor for the orthographic camera, based on the distance and the FOV. Are there any other properties that I need to set on the orthographic camera (e.g. clipping plane, etc..)?
I believe the position remains the same. I'm not sure what else I need to calculate.
fieldOfView = viewInfo.fov;
var getCameraPosition = function() {
return viewer._viewport._implementation.getCamera()._nativeCamera.position;
};
// Calculate the delta position between the camera and the object
var getPositionDelta = function(position1, position2) {
return {
x: position1.x - position2.x,
y: position1.y - position2.y,
z: position1.z - position2.z
}
};
var getDistance = function(positionDelta, cameraDirection) {
return dot(positionDelta, cameraDirection);
};
distance = getDistance(positionDelta, cameraDirection),
var depth = distance;
var viewportWidth = view.getDomRef().getBoundingClientRect().width;
var viewportHeight = view.getDomRef().getBoundingClientRect().height;
var aspect = viewportWidth / viewportHeight;
var height_ortho = depth * 2 * Math.atan( fieldOfView * (Math.PI/180) / 2 )
var width_ortho = height_ortho * aspect;
var near = viewInfo.near, far = viewInfo.far;
var newCamera = new THREE.OrthographicCamera(
width_ortho / -2, width_ortho / 2,
height_ortho / 2, height_ortho / -2,
near, far );
newCamera.position.copy( viewInfo.position );
var sCamera = new vk.threejs.OrthographicCamera(); //framework creatio of threejs cam
sCamera.setZoomFactor(orthoZoomFactor);
sCamera.setCameraRef(newCamera);
view.getViewport().setCamera(sCamera);
I also tried setting the same camera properties (e.g. clipping planes etc) of the perspective for the orthographic and I still had the same problem.
I guess I am missing some property or calculation required to put the object in the same position as when it was in perspective camera view.

Let's assume you have a perspective view with a given vertical field of view angle fov_y (in degrees) and you know the size of the viewport width and height. Furthermore, you have the near and far plane. These are the values which you use to setup the THREE.PerspectiveCamera:
perspCamera = new THREE.PerspectiveCamera( fov_y, width / height, near, far );
Also, you know the position of the object and the position of the camera. An object doesn't have only a single position, but you have to choose a representative position for its depth.
First you have to calculate the depth of the object.
var v3_object = .... // THREE.Vector3 : positon of the object
var v3_camera = perspCamera.position;
var line_of_sight = new THREE.Vector3();
perspCamera.getWorldDirection( line_of_sight );
var v3_distance = v3_object.clone().sub( v3_camera );
depth = v3_distance.dot( line_of_sight );
Then you have to calculate the "size" of the rectangle which is projected to the viewport at the depth:
aspect = width / height;
height_ortho = depth * 2 * Math.atan( fov_y*(Math.PI/180) / 2 )
width_ortho = height_ortho * aspect;
With these values the THREE.OrthographicCamera can be setup like this:
var orthoCamera = new THREE.OrthographicCamera(
width_ortho / -2, width_ortho / 2,
height_ortho / 2, height_ortho / -2,
near, far );
orthoCamera.position.copy( perspCamera.position );
The positon and direction of the perspective camera can be committed to the orthographic camera like this:
orthoCamera.position.copy( perspCamera.position );
orthoCamera.quaternion.copy( perspCamera.quaternion );
See also stackoverflow question Three.js - Find the current LookAt of a camera?

Three.js - Getting the X, Y, and Z coordinates of mouse click

I'm using version 68 of three.js.
I would like to click somewhere and get the X, Y, and Z coordinates. I followed the steps here, but they give me a Z value of 0: Mouse / Canvas X, Y to Three.js World X, Y, Z
Basically, if I have a mesh in the scene, and I click in the middle of it, I'm hoping to be able to calculate the same values as the position of that mesh. This is just an example. I know I could use raycasting and see if I collided with a mesh and then just check its position. However, I want this to work even if I didn't click a mesh.
Is this possible? Here is a jsfiddle: http://jsfiddle.net/j9ydgyL3/
In that jsfiddle, if I could manage to click in the center of that square, I'm hoping to calculate 10, 10, 10 for the X, Y, and Z values respectively because those are the coordinates of the square's position. Here are the two functions of concern:
function getMousePosition(clientX, clientY) {
var mouse2D = new THREE.Vector3();
var mouse3D = new THREE.Vector3();
mouse2D.x = (clientX / window.innerWidth) * 2 - 1;
mouse2D.y = -(clientY / window.innerHeight) * 2 + 1;
mouse2D.z = 0.5;
mouse3D = projector.unprojectVector(mouse2D.clone(), camera);
return mouse3D;
//var vector = new THREE.Vector3(
//( clientX / window.innerWidth ) * 2 - 1,
//- ( clientY / window.innerHeight ) * 2 + 1,
//0.5 );
//projector.unprojectVector( vector, camera );
//var dir = vector.sub( camera.position ).normalize();
//var distance = - camera.position.z / dir.z;
//var pos = camera.position.clone().add( dir.multiplyScalar( distance ) );
//return pos;
}
function onDocumentMouseUp(event) {
event.preventDefault();
var mouse3D = getMousePosition(event.clientX, event.clientY);
console.log(mouse3D.x + ' ' + mouse3D.y + ' ' + mouse3D.z);
}
I left some of the other code I tried commented out. Please note that this commented-out code didn't work in the jsfiddle website, maybe because they're still using version 54 of three.js. It works fine on my machine with version 68.
Edit: To clarify, I would like to be able to get the coordinates no matter where the mouse is. I just used a mesh in this example because it's easy to verify if it works by seeing if the calculated coordinates are the same as the mesh's. What I would really like is for it to work without using raycasting on a mesh. For example, we could have it always printing the calculated coordinates to the console every time the mouse moves, no matter what is in the scene.

You should use a THREE.Raycaster for this. When you set a list of intersectObjects you will be able to get an array of objects that intersected with the ray. So you can get the position from the 'clicked' object from returned list. Check the updated fiddle here.
I also changed your Three.js to version R68
For more advanced use of THREE.RayCaster check the examples at Threejs.org/examples like this example with interactive cubes.
Relevant code from the updated fiddle:
function getMousePosition(clientX, clientY) {
var mouse2D = new THREE.Vector3();
var mouse3D = new THREE.Vector3();
mouse2D.x = (clientX / window.innerWidth) * 2 - 1;
mouse2D.y = -(clientY / window.innerHeight) * 2 + 1;
mouse2D.z = 0.5;
mouse3D = projector.unprojectVector(mouse2D.clone(), camera);
return mouse3D;
var vector = new THREE.Vector3(
(clientX / window.innerWidth) * 2 - 1, -(clientY / window.innerHeight) * 2 + 1,
0.5);
projector.unprojectVector(vector, camera);
var dir = vector.sub(camera.position).normalize();
var distance = -camera.position.z / dir.z;
var pos = camera.position.clone().add(dir.multiplyScalar(distance));
return pos;
}
function onDocumentMouseUp(event) {
event.preventDefault();
var mouse3D = getMousePosition(event.clientX, event.clientY);
console.log(mouse3D.x + ' ' + mouse3D.y + ' ' + mouse3D.z);
var vector = new THREE.Vector3( mouse3D.x, mouse3D.y, 1 );
raycaster.set( camera.position, vector.sub( camera.position ).normalize() );
var intersects = raycaster.intersectObjects(scene.children );
if(intersects.length > 0){
console.log(intersects[0].object.position);
}
}
function animate() {
requestAnimationFrame(animate);
render();
}
function render() {
renderer.render(scene, camera);
}

three.js: transforming coordinates from camera space to scene space

I'm trying to place a cube relative to the camera, rather than relative to the scene. The thing is, to place it in the scene (which I have to do make it show), I have to know the scene coordinates that correspond to the cubes camera space coordinates. I found this function "projectionMatrixInverse" in THREE.Camera. It has a nice function called "multiplyVector3" which I hoped would enable me to transform a vector (1,1,1) back to scene space like this:
var camera, myvec, multvec; // (and others)
camera = new THREE.OrthographicCamera( window.innerWidth / - 2, window.innerWidth / 2, window.innerHeight / 2, window.innerHeight / - 2, - 2000, 1000 );
camera.position.x = 200;
camera.position.y = 100;
camera.position.z = 200;
myvec = new THREE.Vector3(1,1,1);
console.log("myvec: ", myvec);
multvec = camera.projectionMatrixInverse.multiplyVector3(THREE.Vector3(1,1,1));
console.log("multvec: ", multvec);
the thing is, on the console i get:
myvec: Object { x=1, y=1, z=1}
TypeError: v is undefined
var vx = v.x, vy = v.y, vz = v.z;
multiplyVector3 simply doesn't accept my myvec, or says it's undefined, even though the console says it's an object. I don't get it.

The camera is located at the origin of it's coordinate system, and looks down it's negative-Z axis. A point directly in front of the camera has camera coordinates of the form ( 0, 0, z ), where z is a negative number.
You convert a point p
p = new THREE.Vector3(); // create once and reuse if you can
p.set( x, y, z );
from camera coordinates to world coordinates like so:
p.applyMatrix4( camera.matrixWorld );
camera.matrixWorld is by default updated every frame, but if need be, you can update it yourself by calling camera.updateMatrixWorld();
three.js r.95

This may also be what you're after:
scene.add( camera );
brick.position.set( 0, 0, -1 );
camera.add( brick );

Detect clicked object in THREE.js

I have a THREE.js scene where a lot of elements appear, and I need to detect what object the user is clicking on.
What I have done so far is the following. The camera does not move to much - it only changes the vertical position by a limited amount, always looking towards the same point. My approximate method is the following:
I take the coordinates if the click relative to the canvas
I translate them into horizontal and vertical coordinates in the webGL scene by means of a simple rescaling, and add a Z coordinate which is sufficiently far away.
I take a horizontal ray starting from the point above, constructed by THREE.Ray()
I use ray.intersectObjects() to find the first element along the ray.
This method approximately works, but it is sometimes a few pixels away from the actual point.
Is there a more reliable technique to find out the object where a user has clicked?

Depends on what kind of camera are you using.
1) PerspectiveCamera: is ok link that Mr.doob provides.
2) OrthographicCamera: is quite different:
var init = function() {
camera = new THREE.OrthographicCamera( SCREEN_WIDTH / - 2, SCREEN_WIDTH / 2, SCREEN_HEIGHT / 2, SCREEN_HEIGHT / - 2, NEAR, FAR);
document.addEventListener( 'mousedown', onDocumentMouseDown, false );
}
function onDocumentMouseDown( e ) {
e.preventDefault();
var mouseVector = new THREE.Vector3();
mouseVector.x = 2 * (e.clientX / SCREEN_WIDTH) - 1;
mouseVector.y = 1 - 2 * ( e.clientY / SCREEN_HEIGHT );
var raycaster = projector.pickingRay( mouseVector.clone(), camera );
var intersects = raycaster.intersectObject( TARGET );
for( var i = 0; i < intersects.length; i++ ) {
var intersection = intersects[ i ],
obj = intersection.object;
console.log("Intersected object", obj);
}
}

Check out this one:
var camera = new THREE.PerspectiveCamera(45, window.innerWidth / window.innerHeight, 1, 5000);
var object; //your object
document.addEventListener('mousedown', onMouseDown, false);
function onMouseDown(e) {
var vectorMouse = new THREE.Vector3( //vector from camera to mouse
-(window.innerWidth/2-e.clientX)*2/window.innerWidth,
(window.innerHeight/2-e.clientY)*2/window.innerHeight,
-1/Math.tan(22.5*Math.PI/180)); //22.5 is half of camera frustum angle 45 degree
vectorMouse.applyQuaternion(camera.quaternion);
vectorMouse.normalize();
var vectorObject = new THREE.Vector3(); //vector from camera to object
vectorObject.set(object.x - camera.position.x,
object.y - camera.position.y,
object.z - camera.position.z);
vectorObject.normalize();
if (vectorMouse.angleTo(vectorObject)*180/Math.PI < 1) {
//mouse's position is near object's position
}
}

Checks for intersection of the mouse and any of the Cubes in 3d space and alters it's color. Maybe this help you.

I ran into problems trying to implement this for a canvas which does not take up the entire width and height of the screen. Here is the solution I found works quite well.
Initialize everything on an existing canvas:
var init = function() {
var canvas_model = document.getElementById('model')
var viewSize = 50 // Depending on object size, canvas size etc.
var camera = new THREE.OrthographicCamera(-canvas_model.clientWidth/viewSize, canvas_model.clientWidth/viewSize, canvas_model.clientHeight/viewSize, -canvas_model.clientHeight/viewSize, 0.01, 2000),
}
Add an event listener to the canvas:
canvas_model.addEventListener('click', function(event){
var bounds = canvas_model.getBoundingClientRect()
mouse.x = ( (event.clientX - bounds.left) / canvas_model.clientWidth ) * 2 - 1;
mouse.y = - ( (event.clientY - bounds.top) / canvas_model.clientHeight ) * 2 + 1;
raycaster.setFromCamera( mouse, camera );
var intersects = raycaster.intersectObjects(scene.children, true);
if (intersects.length > 0) {
// Do stuff
}
}, false)
Or for a 'touchstart' event, change the lines calculating the mouse.x and mouse.y into:
mouse.x = ( (event.touches[0].clientX - bounds.left) / canvas_model.clientWidth ) * 2 - 1;
mouse.y = - ( (event.touches[0].clientY - bounds.top) / canvas_model.clientHeight ) * 2 + 1;