Develop Reference

three.js: How can I target object's position to another (grouped) object, while allowing rotation to follow AR camera?

I'm using an augmented reality library that does some fancy image tracking stuff. After learning a whole lot about this project, I'm now beyond my current ability and could use some help. For our purposes, the library creates an (empty) anchor point at the center of an IRL image target in-camera. Then moves the virtual world around the IRL camera.
My goal is to drive plane.rotation to always face the camera, while keeping plane.position locked to the anchor point. Additionally, plane.rotation values will be referenced later in development.
const THREE = window.MINDAR.IMAGE.THREE;
document.addEventListener('DOMContentLoaded', () => {
const start = async() => {
// initialize MindAR
const mindarThree = new window.MINDAR.IMAGE.MindARThree({
container: document.body,
imageTargetSrc: '../../assets/targets/testQR.mind',
});
const {renderer, scene, camera} = mindarThree;
// create AR object
const geometry = new THREE.PlaneGeometry(1, 1.25);
const material = new THREE.MeshBasicMaterial({color: 0x00ffff, transparent: true, opacity: 0.5});
const plane = new THREE.Mesh(geometry, material);
// create anchor
const anchor = mindarThree.addAnchor(0);
anchor.group.add(plane);
// start AR
await mindarThree.start();
renderer.setAnimationLoop(() => {
renderer.render(scene, camera);
});
}
start();
});
Everything I've tried so far went into the solutions already massaged into the (functioning draft) code. I have, however, done some research and found a couple avenues that might or might not work. Just tossing them out to see what might stick or inspire another solution. Skill-wise, I'm still in the beginner category, so any help figuring this out is much appreciated.
identify plane object by its group index number;
drive (override lib?) object rotation (x, y, z) to face camera;
possible solutions from dev:
"You can get those values through the anchor object, e.g. anchor.group.position. Meaning that you can use the current three.js API and get those values but without using it for rendering i.e. don't append the renderer.domElement to document."
"You can hack into the source code of mindar (it's open source)."
"Another way might be easier for you to try is to just create another camera yourself. I believe you can have multiple cameras, and just render another layer on top using your new camera."

I think it may be as simple as calling lookAt in the animation loop function:
// start AR
await mindarThree.start();
renderer.setAnimationLoop(() => {
plane.lookAt(new THREE.Vector3());
renderer.render(scene, camera);
});
This assumes the camera is always located at (0,0,0) (i.e., new THREE.Vector3()). This seems to be true from my limited testing. I found it helpful to debug by copy-pasting the MindAR three.js example into this codepen and printing some relevant values to the console.
Also note that, internally, MindAR's three.js module seems to directly modify the world matrix of the anchor.group object without modifying the position/rotation/scale parameters.

Disable collision of body in Cannon.js

I have a bunch of planes that fit together to form terrain. Each individual plane has it's own cannon.js body (I use three.js for rendering visuals) for collision. Due to memory constraints I de-render each object when the player moves to far away from the object. I can de-render objects easily in three.js just by turning them invisible, but there's no clear way to do this in cannon.js. Basically I want to disable a cannon.js object without deleting it outright.
I've already looked through the docs and there's basically nothing on how to do this. I've also seen no questions on any form on this topic.
Example code below to show you how I want to implement this.
//terrain generation
for (z=0; z<6; z++) {
for (x=0; x<6; x++) {
//cannon.js hitbox creation
var groundShape = new CANNON.Box(new CANNON.Vec3(2,0.125,2));
var groundBody = new CANNON.Body({ mass: 0, material: zeromaterial});
groundBody.addShape(groundShape);
groundBody.position.set(x*4,0,z*4);
world.addBody(groundBody);
maparray.push(groundBody);
//three.js plane creation
grassmesh = new THREE.Mesh(grassgeometry, grassmaterial);
grassmesh.castShadow = true;
grassmesh.receiveShadow = true;
grassmesh.position.set(x*4,0,z*4);
scene.add(grassmesh);
maparray.push(grassmesh);
}
}
...
function animate() {
//detect if player is outside of loadDistance of object
for(i=0; i<maparray; i++){
if(Math.abs(maparray[i].position.x - player.position.x) <
loadDistance && Math.abs(maparray[i].position.z -
player.position.z) < loadDistance) {
//code here magically turns off collisions for object.
}
}
}
animate();

To exclude a CANNON.Body from the simulation, run the following:
world.removeBody(groundBody);
To add it back again, run:
world.addBody(groundBody);
It’s perfectly fine to remove and add it back like this. It will help you get better performance when running word.step().

Three.js adding a rigged, animated Asset to an existing Scene

I'm working on a Tamagotchi like browser app in three.js at the moment. But currently I'm stuck with implementing a hand, that pets the avatar when clicked.
The Hand is a rigged Blender model with 2 animations, idle and the poking animation. In the gltf Viewer the model works fine with both animations.
But when added in js, the hand is either completely distorted, or rendered correctly but, positions aren't recognized(for movement with the cursor).
Most of the examples I looked at only added a general scene, but not just one animated model. In both versions of those animations, I get an animation error.
Code for the distorted version:
loader.load('resources/models/gltf/Hand.gltf', function(gltf) {
gltf.scene.traverse(function(node) {
if (node.isMesh) hand = node;
});
//hand.material.morphTargets = true;
scene.add(hand);
mixer = new THREE.AnimationMixer(hand);
clips = hand.animations;
hand = gltf;
scene.add(hand.scene);
});
The second version, where the Hand is rendered correctly, but positions for event handling aren't recognized.
loader.load('resources/models/gltf/Hand.gltf', function(gltf) {
var hand = gltf.scene;
var animations = gltf.animations;
mixer = new THREE.AnimationMixer(hand);
for (var i = 0; i < animations.length; i++) {
mixer.clipAction(animations[i]).play();
}
scene.add(hand);
});
function for idle animation:
function idleAnim() {
var idleClip = THREE.AnimationClip.findByName(clips, "Idle");
var action = mixer.clipAction(idleClip);
action.play();
console.log("idling");
}
Link: https://github.com/JoeJoe49/AnimTest
Thanks in advance and greetings.

In your first example, you're pulling the "hand" object out of your import scene, adding it to your render scene, Then adding the rest of the import scene to your render scene.
My guess is that you need to pull out "hand" from higher in the hierarchy. It probably has a few parent objects that need to come along with to preserve the correct hierarchy for the animation.
It's worth doing a scene.traverse((o)=>{console.log(o)} to get a clear picture of how your scene is being exported. I've found with the blender gltf exporter for instance, there are usually 2 separate parent nodes, one for positioning and one for scaling+rotation, so.. it's worth looking at because it might not be exactly what you expect.
fwiw I grabbed your repo and opened the gltfs in my model previewer, but I didn't seem to see any animations on them. My previewer is set to play all animations it finds, in sequence.. so not sure what's going on there. I'm guessing these are skinnedmeshes and not morphtargets?

p5.speech.js continuous won't become true

I've been playing around with p5.speech.js the past few days. I am able to record myself for short periods of time but it stops short. I later learned that there is a continuous bool that allows you to keep recording so I decided to implement it. I set it to true with the "let continuous = true". When I ran the code the p5.speechRec still said false within the console. When I tested the console.log at the bottom it the output was true as well so I'm a little confused as to if this is a bug, a problem with chrome, or just an error that I am missing. Thanks for your help.
var myRec = new p5.SpeechRec(); // new P5.SpeechRec object
function setup()
{
// graphics stuff:
createCanvas(800, 400);
background(255, 255, 255);
fill(0, 0, 0, 255);
// instructions:
textSize(32);
textAlign(CENTER);
text("say something", width/2, height/2);
let continuous = true;
let interimResults = false;
myRec.start(continuous, interimResults);
console.log(myRec);
function speechRec(){
if (speechRec.resultValue){
createP(speechRec.resultString);
}
}
console.log("cont bool: " + continuous);
}

Questions like these are best answered by looking at the documentation for the library in question. Start by looking at the P5.js libraries page, which leads to the p5.speech documentation page here.
That documentation page says that continuous is a property of the p5.SpeechRec object, and it even links to some example code here.
Basically, you can't just pass in a random value to the start() function and expect it to work. You have to set the continuous variable yourself:
var myRec = new p5.SpeechRec('en-US', parseResult); // new P5.SpeechRec object
myRec.continuous = true; // do continuous recognition
myRec.start(); // start engine
Also, I'm not sure what your speechRec() function inside your setup() function is meant to do since you never call it, but that's not directly related to your question.

Toggle between multiple THREE.EffectComposer scenes using the same renderer in three.js

I am in the process of creating complex scenes with Composer in three.js.
I am wanting to know if it is possible to switch between two scenes that have different composer effects attributed to them. To gain some sort of perspective I have created an example which allows you to toggle between two normally rendered scenes.
Two scene example
From my understanding of how composer works you create an instance of it and then apply a render pass like so:
this.composer = new THREE.EffectComposer(this.renderer.default.init);
this.renderPass = new THREE.RenderPass(this.stage, this.camera);
this.renderPass.renderToScreen = true;
this.composer.addPass(this.renderPass);
and then apply a composer render like so:
this.composer.render();
So my question is if I have a second scene which a composer instance how can I then:
Use the same renderer (if possible)
Toggle between scene 1 and scene 2 like in a similar fashion to my example.

You can just switch from one effectComposer to another one the same way as you switch from one scene to the other. So that would be something like this:
const scenes = [
new THREE.Scene(),
new THREE.Scene()
];
const composers = scenes.map(function(scene) {
const composer = new THREE.EffectComposer(renderer);
// configure render-passes
const renderpass = new THREE.RenderPass(scene, camera);
renderpass.renderToScreen = true;
composer.addPass(renderpass);
scene.composer = composer;
return composer;
});
// then use the composer for the scene
let activeScene = scenes[0];
activeScene.composer.render();
You should even be able to reuse certain render-passes if you want to.

So before I begin I just want to give a shout out to Martin Schuhfuß who provided the insight to this solution.
Disclaimer
I am not an expert or professional Programmer in javascript and I by no means suggest that this is the only way of doing this, my presentation should be taken in an abstract fashion. My thoughts only explain the theoretical premises for which the solution is based on and implementation of this will depend upon your own architecture.
Architecture
I am using an OOP method in this example, you should be able to manipulate the solution to whatever method you are using.
Method
So based on Martin’s current example you can see that we able to add the composer property/object to our scene object this means that scenes could inherit the same composer effects which is brilliant, however In my case I have many scenes with different composer effects so I needed to rethink the problem.
1. Create a Composer object.
So I created an object to put composer objects in and keep to the nice ‘composer’ name convention when reference my effects.
This.composer = {};
2. Create a function to initialise RenderPasses with the concerning scenes.
So it is important to remember that we need to define and initailse the RenderPasses first before will call our composer. RenderPasses allows us to attribute the effects (know has shaders) that we want. In my example I have two scenes therefore I needed to create:
Two RenderPasses.
One RenderPass copied the scene.
The other RenderPass applied a sepia effect to its scene.
Example of Code:
this.init = function() {
stackoverflow.webgl.pass.base = new THREE.RenderPass(stackoverflow.webgl.scene.default,
stackoverflow.webgl.camera);
stackoverflow.webgl.pass.base2 = new THREE.RenderPass(stackoverflow.webgl.scene.test,
stackoverflow.webgl.camera);
stackoverflow.webgl.pass.sepia = new
THREE.ShaderPass(THREE.SepiaShader);
stackoverflow.webgl.pass.sepia.renderToScreen = true;
stackoverflow.webgl.pass.copy = new THREE.ShaderPass(THREE.CopyShader);
stackoverflow.webgl.pass.copy.renderToScreen = true;
}
Note: some file names had to be renamed due to legal reason, but the point being is that this function is called into a larger object.
3. Create a start function and lets assign composers to scenes
The purpose of this function is just to demonstrate how we combine it all together. So the code that we have is something like this.
// so we create an object to use for reference for EffectComposer objects
this.composer = {};
// this property is used to set the scene that we want
// this.scene.default = new THREE.Scene();
this.activeScene = this.scene.default;
// this function is just a wrapper for our code
this.start = function () {
// so call our initialise our RenderPasses
stackoverflow.webgl.pass.init();
// my own method of seting up the WebGLRenderer scene (You do it your Way!)
stackoverflow.webgl.renderer.default.setup;
// Create a new composer for scene 1
this.composer.ui = new THREE.EffectComposer(stackoverflow.webgl.renderer.default.init);
// Create a new composer for scene 1
this.composer.ui2 = new THREE.EffectComposer(stackoverflow.webgl.renderer.default.init);
// Now here is the cool stuff you can assign a composer to each scene
this.scene.default.composer = stackoverflow.webgl.composer.ui;
this.scene.test.composer = stackoverflow.webgl.composer.ui2;
// and i always like to check that things happen and they do ;)
console.log(this.scene.default);
console.log(this.scene.test);
console.log(this.composer);
// so you will need to add the passes some place, I created a function call render, (you do according to your code structure)
stackoverflow.webgl.render();
}
4. Define where you add the passes in your architecture
So now we will need to add the pass declarations (addpass functions from composer) for our effects to take place.
this.render = function () {
stackoverflow.webgl.composer.ui.addPass(stackoverflow.webgl.pass.base);
stackoverflow.webgl.composer.ui.addPass(stackoverflow.webgl.pass.copy);
stackoverflow.webgl.composer.ui2.addPass(stackoverflow.webgl.pass.base2);
stackoverflow.webgl.composer.ui2.addPass(stackoverflow.webgl.pass.sepia);
};
5. Add the EffectComposer render method
So this line of code will need to be placed wherever you do the composer processing (that depends on your setup).
stackoverflow.webgl.activeScene.composer.render();
Please take note that the ‘activecScene’ part makes reference to the actual scene used at the time. It is this that allows us to change the scene.
6. Create a button and toggle between the scenes
So I created two buttons in a dat.gui instance that allows me to toggle between the two scenes.
Again you can create a button, function whatever you like.
// again just change the vale of active scene to the scene you wish to change to
// button 1 value will be something like this:
stackoverflow.webgl.activeScene = stackoverflow.webgl.scene.test;
// button 2 value will takes us back to scene 1
stackoverflow.webgl.activeScene = stackoverflow.webgl.scene.default;
Conclusion
This is my method of achieving this effect, but if there are better or alternative ways then please add to the discussion and share.