Brief description: How can a JSON-character-model be dynamically animated using Three.js and a BVH-motion capture-file? As in: The animation is not baked into the JSON-file but rather loaded and parsed from the BVH-file using BVHLoader.js
I got as far as loading the BVH-file, attaching its data to the skeletonHelper, extracting the bone-positions, animating some boxes based on their position (for testing pursposes) and loading a (hopefully?) rigged JSON-model. See the demo here: https://www.patrik-huebner.com/repo/JSON-BVH/
See a GIF of the animation here: https://www.patrik-huebner.com/repo/JSON-BVH/demo.gif (sadly, I can't yet post images)
I did quite some extensive research on this topic and while there are quite a lot of helpful tips[1] on stackoverflow on exporting a mesh from Blender to JSON using the Three.js-exporter and baking in animations, I was not able to figure out how to
a) set up a JSON-file / rigged model / it's skeleton/bones so that I can dynamically animate it (I used a rigged, low-poly from blendswap.com/blends/view/21948). Are there any recommended settings? In order to be able to load the file at all I have to check the "export scene"-option which I'm not quite certain why that should be necessary.
b) map the BVH-data to the JSON-model
If you have any advice and / or code-samples I can use - or at least some resources or keywords that I can research, I'd be most thankful !
Using the supplied BVHLoader.js I am able to load the animation and attach it to the skeletonHelper
var loader = new THREE.BVHLoader();
loader.load( "models/pirouette.bvh", function( result ) {
skeletonHelper = new THREE.SkeletonHelper( result.skeleton.bones[0] );
skeletonHelper.skeleton = result.skeleton; // allow animation mixer to bind to SkeletonHelper directly
boneContainer = new THREE.Group();
boneContainer.add( result.skeleton.bones[ 0 ] );
scene.add( skeletonHelper );
scene.add( boneContainer );
mixer = new THREE.AnimationMixer( skeletonHelper );
mixer.clipAction( result.clip ).setEffectiveWeight( 1.0 ).play();
} );
Though I am rather certain this is not how its done: I can access the Vector3-data of the skeletonHelper and animate some test-cubes based on those positions (compare code-sample above and my supplied GIF-animation)
for (var i = 0; i < skeletonHelper.bones.length; i++) {
var myPos = skeletonHelper.bones[i].getWorldPosition();
cubeGroup.children[i].position.x = myPos.x;
cubeGroup.children[i].position.y = myPos.y;
cubeGroup.children[i].position.z = myPos.z;
}
Finally, I load the JSON-file and add it to the scene using THREE.ObjectLoader (as the THREE.JSONLoader would not accept my exported model)
// Load the rigged JSON-file based on this character http://www.blendswap.com/blends/view/21948
// converted to JSON with the Three.json-export-plugin
var loader = new THREE.ObjectLoader();
var s = 50;
loader.load(
"models/lowPoly-rigged.json",
function ( obj ) {
var material = new THREE.MeshLambertMaterial({color: 0xff0000});
scene.add( obj );
obj.position.y = 0;
obj.children[0].material = material;
obj.scale.set(s,s,s);
},
function ( xhr ) {
console.log( (xhr.loaded / xhr.total * 100) + '% loaded' );
},
function ( xhr ) {
console.error( 'An error happened' );
}
);
But now: How do I animate the model using the loaded BVH-data. Do I have to match the amount of bones from the JSON-model to the BVH? Can I just supply some data to the model and have it happen "magically"? I have seen some online demos that load BVH-files dynamically and pass the data to a model[2]
[1] Stuff I did research (sadly, I can't post more links) that did not help, is not quite specific to what I am looking to achieve:
stackoverflow.com/questions/20433474/dynamic-bones-animation-in-three-js
stackoverflow.com/questions/29970791/animate-makehuman-character-with-bvh-using-three-js
stackoverflow.com/questions/40719572/three-js-wrong-bones-orientation (this did look the most helpful as it is actually about combining a loaded BVH-file and a model but researching/implementing the code with my own model lead to error-messages about the material which points to some misunderstanding of mine in the export-process from Blender to JSON)
[2] Some resources that were able to handle BVH-files and could attach those animations to a model:
BVH Player by Aki A. on chromeExperiments: chromeexperiments.com/experiment/bvh-player
lo-th.github.io/root/blending2/index_bvh.html
Again, thanks a lot for any input!
Related
I am working on customizing the transform controls available in three.js for my project.
I have already changed the rotation part and now working on translation part.
if you notice in translation Gizmo, there is a XYZ octahedron in the center. I have removed all other planes and arrows and wrote all functionality only on that center mesh, which is working fine.
Now I am only stuck at one small problem, that is the size and position of that controller. I changed that Octahedron to boxGeometry and writing the code to make the size of that controller to be exact size of the selected object. for that I get the idea to make the size of the controller, same as the boxHelper size, which act as outline of object.
when I tried this logic in sample code, where I created a box, and getting the size of box helper and creating another box of same size, it was working fine. but when I am writing same code in threejs transform controls, result is not the same.
below is the geometry code init
XYZ: [[ new THREE.Mesh( new THREE.BoxGeometry( 0.1, 0.1, 0.1 ), pickerMaterial )]],
then I am getting the size of box3 when attaching to any object
this.addBoxHelper = function () {
this.removeBoxHelper();
if(this.object.box3) {
**this.object.box3.getSize(selectionBoxSize);**
console.log(selectionBoxSize)
this.objectBoxHelper = new THREE.Box3Helper(this.object.box3, 0xffff00);
this.objectBoxHelper.canSelect = function () {
return false;
}
this.object.add(this.objectBoxHelper);
}
}
then below is my update function of transform controls
this.update = function () {
if ( scope.object === undefined ) return;
scope.object.updateMatrixWorld();
worldPosition.setFromMatrixPosition( scope.object.matrixWorld );
worldRotation.setFromRotationMatrix( tempMatrix.extractRotation( scope.object.matrixWorld ) );
scope.object.box3.getSize(selectionBoxSize);
scope.object.getWorldPosition(selectionBoxPos);
camera.updateMatrixWorld();
camPosition.setFromMatrixPosition( camera.matrixWorld );
camRotation.setFromRotationMatrix( tempMatrix.extractRotation( camera.matrixWorld ) );
**scaleT = selectionBoxSize;**
//below three lines are for dynamic size change based on camera position..for
//next level functionality
//scaleT.x = worldPosition.distanceTo( camPosition ) / 6 * selectionBoxSize.x;
//scaleT.y = worldPosition.distanceTo( camPosition ) / 6 * selectionBoxSize.y;
//scaleT.z = worldPosition.distanceTo( camPosition ) / 6 * selectionBoxSize.z;
this.position.copy( selectionBoxPos );
this.scale.set( scaleT.x, scaleT.y, scaleT.z);
this.updateMatrixWorld();
below is the output of console log
TransformControls.js:526 Vector3 {x: 10.020332336425781, y: 2.621583938598633, z: 3.503500819206238}
TransformControls.js:601 Vector3 {x: 10.020332336425781, y: 2.621583938598633, z: 3.503500819206238}
as you can see, the scale is the same, but the result is different. see the result in below images.
as you see in images, that red color box at bottom is translation controller but smaller than selection box.
another issue is that pivot of my objects are at bottom, and I want this controller to come at the center of the selection box, that is also not happening with getCenter method of box3.
Please help!! let me know if I am unclear in explaining the issue
If you are getting the bounding box of the object from its geometry, that will be wrong because it doesn't take the objects transform into account. You have to use box3.setFromObject (yourObject) instead. does that help?
I am writing a trace-line function for a visualization project that requires jumping between time step values. My issue is that during rendering, the line created using THREE.js's BufferGeometry and the setDrawRange method, will only be visible if the origin of the line is in the camera's view. Panning away will result in the line disappearing and panning toward the origin of the line (usually 0,0,0) will make it appear again. Is there a reason for this and a way around it? I have tried playing around with render settings.
The code I have included is being used in testing and draws the trace of the object as time progresses.
var traceHandle = {
/* setup() returns trace-line */
setup : function (MAX_POINTS) {
var lineGeo = new THREE.BufferGeometry();
//var MAX_POINTS = 500*10;
var positions = new Float32Array( MAX_POINTS * 3 ); // 3 vertices per point
lineGeo.addAttribute('position', new THREE.BufferAttribute(positions, 3));
var lineMaterial = new THREE.LineBasicMaterial({color:0x00ff00 });
var traceLine = new THREE.Line(lineGeo, lineMaterial);
scene.add(traceLine);
return traceLine;
},
/****
* updateTrace() updates and draws trace line
* Need 'index' saved globally for this
****/
updateTrace : function (traceLine, obj, timeStep, index) {
traceLine.geometry.setDrawRange( 0, timeStep );
traceLine.geometry.dynamic = true;
var positions = traceLine.geometry.attributes.position.array;
positions[index++]=obj.position.x;
positions[index++]=obj.position.y;
positions[index++]=obj.position.z;
// required after the first render
traceLine.geometry.attributes.position.needsUpdate = true;
return index;
}
};
Thanks a lot!
Likely, the bounding sphere is not defined or has radius zero. Since you are adding points dynamically, you can set:
traceLine.frustumCulled = false;
The other option is to make sure the bounding sphere is current, but given your use case, that seems too computationally expensive.
three.js r.73
If I assign framemat to createScene(ID,geometry,1,framemat) it works OK, if I do createScene( ID, geometry, 1, materials[ID] ) it doesn't work.
var jsonLoader = new THREE.JSONLoader(),
paths = [
"obj/jgd/GALD-JGD-frame.json",
"obj/jgd/GALD-JGD-logo.json",
"obj/jgd/GALD-JGD-light.json"
],
materials = [
"framemat",
"logomat",
"lightmat"
],
objNum = paths.length; // Cache obects.length
for ( var id = 0; id < paths.length; id++ ) {
(function(){ // IIFE start
jsonLoader.load( paths[id], function( geometry, materials ) {
createScene( id, geometry, 1, materials[id] )
}
);
})(); // IIFE end
}
CreateScene fun here:
function createScene( id, geometry, scale, material ) {
geometry.computeTangents();
objects[id]= new THREE.Mesh( geometry, material );
objects[id].scale.x = objects[id].scale.y = objects[id].scale.z = scale;
scene.add( objects[id] );
}
Your code is ambiguous : first you define an array called materials then you use the second argument of the JSONLoader and also name it materials to catch the textures in the json file, finally you use materials[id] to pass the material at the mesh creation. Which one is it ? Whereas the first array is still filled with strings, the json can have an array of materials instead of only one, worse of all the rank [id] may not exist in this one.
So you have two solutions :
If you want to create the materials independently, only keep the first array and add the missing function that transforms your strings to materials. Just remove the second argument of the JSONLoader.
If you want to use the material provided in the JSON File, the array materials is useless and can lead to other mistakes. There are two options then :
if you have one material for the whole object, write it this way : createScene(...,..., materials);
if you have several materials for the object, this way : createScene(....,....,new THREE.MeshFaceMaterial(materials);
I have a object that I'm adding into my scene. I also have various other cubes in the scene. I'm using the following code to fire a Ray and see if I can detect a collision:
var ray = new THREE.Raycaster(camera.position, vec);
var intersects = ray.intersectObjects( scene.children );
For some reason, the regular shapes (cubes) are detected, but the objects that loaded from the obj files are not.
var loader = new THREE.OBJMTLLoader();
loader.load( 'models/technicalTable1.obj', 'models/technicalTable1.mtl', function ( obj ) {
obj.scale.set(0.4, 0.4, 0.4);
obj.position.x = - roomWidth/2 + 100;
obj.position.y = 36;
obj.position.z = - roomLength/2 + 25;
scene.add( obj );
}, onProgress, onError );
Thanks for any help!
You need to pass the recursive flag to Raycaster.intersectObjects().
var intersects = ray.intersectObjects( scene.children, true );
three.js r.70
It doesnt work because the OBJMTLLoader creates a Three.Group
The Group does not have a raycaster.intersectObjects
(I just finished tracing it down and it went to a non implemented function
This REALLY limits the the use of OBJ objects.
I have tried JSON but since you can not load materials and have to create UVmap I find that I need a person that can 3D model or not be able to use the intersectObjects.
Not sure if this is a bug
I have some objects added to an Object3D (for grouping elements) and I'm trying to detect clicks on it.
My scene has a size of 600x400, my camera is within a three-object and my event handler code looks like below:
function onDocumentMouseDown( event ) {
event.preventDefault();
var mouse = {};
mouse.x = ( event.clientX / 600 ) * 2 - 1;
mouse.y = - ( event.clientY / 400 ) * 2 + 1;
var vector = new THREE.Vector3( mouse.x, mouse.y, 1 );
projector.unprojectVector( vector, three.camera );
var ray = new THREE.Ray( three.camera.position, vector.subSelf( three.camera.position ).normalize() );
var intersects = ray.intersectObjects( group.children );
alert(intersects.length);
[...]
}
Actually I'm alerting the count of intersected objects. But it stays zero. It couldn't find any intersected objects. I've already played a bit aroud with the x, y and z values of my projection vector - without success.
I've added a stripped down sample for demonstrating this issue on jsfiddle. Maybe someone has a short hint for me what goes wrong with it?
In your fiddle, because you are calling THREE.SceneUtils.createMultiMaterialObject( ), which creates a hierarchical structure, you need to add the recursive flag to ray.intersectObjects().
var intersects = ray.intersectObjects( group.children, true );
EDiT: ray is now an instance of THREE.Raycaster -- not THREE.Ray.
three.js r.58
I had the same problem and WestLangley's answer provides the answer. Great job! For anybody struggling with mouse selection of objects grouped in Object3D wrapper too, I am posting my own solution.
First, I created an array of objects that are selectable - I hope this also saves some performance, as RayCaster doesnt need to search all objects in the scene, but only those you wish to respond to selection. I also attached this array to scene object directly (solely for the fact that it is already accessible from most parts of my app)
scene.selectable = [];
Next step is to push all objects that you wish to make selectable into this array. You will insert only meshes/sprites/etc from your group, not the whole group. Only last line is important here:
var myWrapper = new THREE.Object3D();
var myObject = new THREE.Mesh( something );
myWrapper.add( myObject );
scene.add ( myWrapper );
scene.selectable.push( myObject );
And lastly in your mouse selection routine you will call raycaster like this:
var intersects = ray.intersectObjects( scene.selectable );