I am trying to make a situation wherein a context menu appears when you click on the mesh of a 3D object. Herein what I have currently:
But what is happening is that the context menu is appearing behind the 3D object and so it is not properly visible. I was wondering what could I do to make the context menu appear in front of the 3D object? The raycast part of my code is herein stated:
public raycast(e: MouseEvent): void {
let mesh;
const x = e.clientX - this.rect.left;
const y = e.clientY - this.rect.top;
let scenePointer = new th.Vector2();
scenePointer.x = (x / this.canvas.clientWidth) * 2 - 1;
scenePointer.y = (y / this.canvas.clientHeight) * - 2 + 1;
var intersect;
// Determine the component in-focus using raycasting
this.raycaster.setFromCamera(scenePointer, this.camera);
const intersects = this.raycaster.intersectObjects(this.scene.children);
var menu = document.getElementById("menu")
var rightclick;
if ((<any>e).which) rightclick = ((<any>e).which == 3);
else if ((<any>e).button) rightclick = ((<any>e).button == 2);
if (!rightclick) {
if (intersects.length > 0) {
this.asmStateServ.focusedComponent = this.assemblyFetchServ.guidToName.get(intersects[0].object.name);
mesh = intersects[0].object
if ((<th.MeshPhongMaterial>mesh.material).color.getHex() != 0x00ff00) {
this.asmStateServ.lastColor = (<th.MeshPhongMaterial>mesh.material).color.getHex();
(<th.MeshPhongMaterial>mesh.material).color = new th.Color(0x00ff00);
}
else {
(<th.MeshPhongMaterial>mesh.material).color = new th.Color(this.asmStateServ.lastColor);
}
}
else {
this.asmStateServ.focusedComponent = '';
}
}
else {
if (intersects.length) {
intersect = intersects[0].object;
menu.style.left = x + "px";
menu.style.top = y + "px";
menu.style.display = "";
}
else {
intersect = undefined;
}
}
}
Any kind of help by modifying my part of the code would be much appreciated. :)
I coded a little simulation where you can dig cubes, they fall an stack, you can consume some or make a selection explode.
I started this little project for fun and my aim is to handle as much cubes as possible (100k would be a good start).
The project is simple, 3 possibles actions:
Dig cubes (2k each click)
Consume cubes (50 each click)
Explode cubes (click on two points to form a rectangle)
Currently, on my computer, performance starts to drop when I got about 20k cubes. When you select a large portion of cubes to explode, performance are heavily slowed down too. I'm not sure the way I simplified physics is the best way to go.
Could you give me some hints on how to improve/optimize it ?
Here is the complete code : (The stacking doesn't work in the SO snippet so here is the codepen version)
(() => {
// Variables init
let defaultState = {
cubesPerDig : 2000,
cubesIncome : 0,
total : 0
};
let cubeSize = 2, dropSize = window.innerWidth, downSpeed = 5;
let state,
digButton, // Button to manually dig
gameState, // An object containing the state of the game
cubes, // Array containing all the spawned cubes
heightIndex, // fake physics
cubesPerX, // fake physics helper
playScene; // The gamescene
// App setup
let app = new PIXI.Application();
app.renderer.view.style.position = "absolute";
app.renderer.view.style.display = "block";
app.renderer.autoResize = true;
document.body.appendChild(app.view);
// Resize
function resize() {
app.renderer.resize(window.innerWidth, window.innerHeight);
}
window.onresize = resize;
resize();
// Hello ! we can talk in the chat.txt file
// Issue : When there are more than ~10k cubes, performance start to drop
// To test, click the "mine" button about 5-10 times
// Main state
function play(delta){
// Animate the cubes according to their states
let cube;
for(let c in cubes){
cube = cubes[c];
switch(cube.state) {
case STATE.LANDING:
// fake physics
if(!cube.landed){
if (cube.y < heightIndex[cube.x]) {
cube.y+= downSpeed;
}else if (cube.y >= heightIndex[cube.x]) {
cube.y = heightIndex[cube.x];
cube.landed = 1;
heightIndex[cube.x] -= cubeSize;
}
}
break;
case STATE.CONSUMING:
if(cube.y > -cubeSize){
cube.y -= cube.speed;
}else{
removeCube(c);
}
break;
case STATE.EXPLODING:
if(boundings(c)){
continue;
}
cube.x += cube.eDirX;
cube.y += cube.eDirY;
break;
}
}
updateUI();
}
// Game loop
function gameLoop(delta){
state(delta);
}
// Setup variables and gameState
function setup(){
state = play;
digButton = document.getElementById('dig');
digButton.addEventListener('click', mine);
playScene = new PIXI.Container();
gameState = defaultState;
/* User inputs */
// Mine
document.getElementById('consume').addEventListener('click', () => {consumeCubes(50)});
// Manual explode
let explodeOrigin = null
document.querySelector('canvas').addEventListener('click', e => {
if(!explodeOrigin){
explodeOrigin = {x: e.clientX, y: e.clientY};
}else{
explode(explodeOrigin, {x: e.clientX, y: e.clientY});
explodeOrigin = null;
}
});
window['explode'] = explode;
heightIndex = {};
cubesPerX = [];
// Todo fill with gameState.total cubes
cubes = [];
app.ticker.add(delta => gameLoop(delta));
app.stage.addChild(playScene);
}
/*
* UI
*/
function updateUI(){
document.getElementById('total').innerHTML = cubes.length;
}
/*
* Game logic
*/
// Add cube when user clicks
function mine(){
for(let i = 0; i < gameState.cubesPerDig; i++){
setTimeout(addCube, 5*i);
}
}
// Consume a number of cubes
function consumeCubes(nb){
let candidates = _.sampleSize(cubes.filter(c => !c.eDirX), Math.min(nb, cubes.length));
candidates = candidates.slice(0, nb);
candidates.map(c => {
dropCubes(c.x);
c.state = STATE.CONSUMING;
});
}
const STATE = {
LANDING: 0,
CONSUMING: 1,
EXPLODING: 2
}
// Add a cube
function addCube(){
let c = new cube(cubeSize);
let tres = dropSize / cubeSize / 2;
c.x = window.innerWidth / 2 + (_.random(-tres, tres) * cubeSize);
c.y = 0//-cubeSize;
c.speed = _.random(5,8);
cubes.push(c);
c.landed = !1;
c.state = STATE.LANDING;
if(!cubesPerX[c.x]) cubesPerX[c.x] = [];
if (!heightIndex[c.x]) heightIndex[c.x] = window.innerHeight - cubeSize;
cubesPerX[c.x].push(c);
playScene.addChild(c);
}
// Remove a cube
function removeCube(c){
let cube = cubes[c];
playScene.removeChild(cube);
cubes.splice(c,1);
}
// Delete the cube if offscreen
function boundings(c){
let cube = cubes[c];
if(cube.x < 0 || cube.x + cubeSize > window.innerWidth || cube.y < 0 || cube.y > window.innerHeight)
{
removeCube(c);
return true;
}
}
// explode some cubes
function explode(origin, dest){
if(dest.x < origin.x){
dest = [origin, origin = dest][0]; // swap
}
var candidates = cubes.filter(c => c.state != STATE.EXPLODING && c.x >= origin.x && c.x <= dest.x && c.y >= origin.y && c.y <= dest.y);
if(!candidates.length)
return;
for(let i = origin.x; i <= dest.x; i++){
dropCubes(i);
}
candidates.forEach(c => {
c.explodingSpeed = _.random(5,6);
c.eDirX = _.random(-1,1,1) * c.explodingSpeed * c.speed;
c.eDirY = _.random(-1,1,1) * c.explodingSpeed * c.speed;
c.state = STATE.EXPLODING;
});
}
// Drop cubes
function dropCubes(x){
heightIndex[x] = window.innerHeight - cubeSize;
if(cubesPerX[x] && cubesPerX[x].length)
cubesPerX[x].forEach(c => {
if(c.state == STATE.EXPLODING) return;
c.landed = false; c.state = STATE.LANDING;
});
}
/*
* Graphic display
*/
// Cube definition
function cube(size){
let graphic = new PIXI.Graphics();
graphic.beginFill(Math.random() * 0xFFFFFF);
graphic.drawRect(0, 0, size, size);
graphic.endFill();
return graphic;
}
// Init
setup();
})()
/* styles */
/* called by your view template */
* {
box-sizing: border-box;
}
body, html{
margin: 0;
padding: 0;
color:white;
}
#ui{
position: absolute;
z-index: 2;
top: 0;
width: 0;
left: 0;
bottom: 0;
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/pixi.js/4.8.1/pixi.min.js"></script>
<script>PIXI.utils.skipHello();</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.10/lodash.min.js"></script>
<!-- Click two points to make cubes explode -->
<div id="ui">
<button id="dig">
Dig 2k cubes
</button>
<button id="consume">
Consume 50 cubes
</button>
<p>
Total : <span id="total"></span>
</p>
</div>
Thanks
Using Sprites will always be faster than using Graphics (at least until v5 comes along!)
So I changed your cube creation function to
function cube(size) {
const sprite = new PIXI.Sprite(PIXI.Texture.WHITE);
sprite.tint = Math.random() * 0xFFFFFF;
sprite.width = sprite.height = size;
return sprite;
}
And that boosted the fps for myself
I would like to be able to overlay a html iframe on top of an augmented reality marker, however I cannot get the CSS3DRenderer to show the same result as the WebGLRenderer and I'm not sure where I'm going wrong.
The WebGL renders perfectly, with the mesh following the marker and it's all magic, the CSS3DRenderer however centers the iframe in the middle of the video, inversed and unscaled, and it rotates in the opposite direction.
Thanks to three.js and artoolkit, here is some test code using video input and both renderers.
new Promise(function(resolve,reject) {
var source = document.createElement('video');
source.autoplay = true;
source.playsinline = true;
source.controls = false;
source.loop = true;
source.onplay = function(event) {
resolve(source);
}
source.src = 'data/output_4.ogg';
document.body.appendChild(source);
}).then(function(source) {
var scene = new THREE.Scene();
var camera = new THREE.Camera();
camera.matrixAutoUpdate = false;
scene.add(camera);
var material = new THREE.MeshNormalMaterial({
transparent : true,
opacity : 0.5,
side : THREE.DoubleSide
});
var geometry = new THREE.PlaneGeometry(1,1);
var mesh = new THREE.Mesh(geometry,material);
// mesh.matrixAutoUpdate = false;
scene.add(mesh);
var renderer = new THREE.WebGLRenderer({
antialias : true,
alpha : true
});
renderer.setSize(source.videoWidth,source.videoHeight);
renderer.setClearColor(new THREE.Color('lightgrey'),0);
document.body.appendChild(renderer.domElement);
/*\
cssRenderer
\*/
var cssRenderer = new THREE.CSS3DRenderer();
cssRenderer.setSize(source.videoWidth,source.videoHeight);
var cssScene = new THREE.Scene();
var iframe = document.createElement("iframe");
iframe.src = "/data/index.html";
iframe.style.background = "rgb(0,0,0)";
var iframe3D = new THREE.CSS3DObject(iframe);
// iframe3D.matrixAutoUpdate = false;
cssScene.add(iframe3D);
document.body.appendChild(cssRenderer.domElement);
/*\
arController
\*/
var cameraParameters = new ARCameraParam();
var arController = null;
cameraParameters.onload = function() {
arController = new ARController(source.videoWidth,source.videoHeight,cameraParameters);
arController.addEventListener("getMarker",function(event) {
var modelViewMatrix = new THREE.Matrix4().fromArray(event.data.matrix);
camera.matrix.getInverse(modelViewMatrix);
// mesh.matrix.copy(modelViewMatrix);
// iframe3D.matrix.copy(modelViewMatrix);
});
var cameraViewMatrix = new THREE.Matrix4().fromArray(arController.getCameraMatrix());
camera.projectionMatrix.copy(cameraViewMatrix);
}
cameraParameters.load("data/camera_para.dat");
/*\
animate
\*/
requestAnimationFrame(function animate() {
requestAnimationFrame(animate);
if (!arController) {
return;
}
arController.process(source);
renderer.render(scene,camera);
cssRenderer.render(cssScene,camera);
});
});
I had hoped that rotating the camera instead of the object would provide a solution, alas. It's as if I was missing out some matrix transformation that needs to be applied.
Perhaps there was something with the fov parameters of the camera, or maybe the transform matrix was getting overwritten, but anyway I couldn't locate the problem. So here's an alternative that doesn't use CSS3DRenderer, or THREE.js.
Essentially we can use the coordinates from the marker data itself to create a projection matrix. Many, many thanks to this post which provided most of the code I needed.
function adjugate(m) { // Compute the adjugate of m
return [
m[4]*m[8]-m[7]*m[5],m[7]*m[2]-m[1]*m[8],m[1]*m[5]-m[4]*m[2],
m[6]*m[5]-m[3]*m[8],m[0]*m[8]-m[6]*m[2],m[3]*m[2]-m[0]*m[5],
m[3]*m[7]-m[6]*m[4],m[6]*m[1]-m[0]*m[7],m[0]*m[4]-m[3]*m[1]
];
}
function multiply(a,b) { // multiply two matrices
a = [
a[0],a[3],a[6],
a[1],a[4],a[7],
a[2],a[5],a[8]
];
b = [
b[0],b[3],b[6],
b[1],b[4],b[7],
b[2],b[5],b[8]
];
var m = Array(9);
for (var i = 0; i != 3; ++i) {
for (var j = 0; j != 3; ++j) {
var mij = 0;
for (var k = 0; k != 3; ++k) {
mij += a[3*i + k]*b[3*k + j];
}
m[3*i + j] = mij;
}
}
return [
m[0],m[3],m[6],
m[1],m[4],m[7],
m[2],m[5],m[8]
];
}
function apply(m,v) { // multiply matrix and vector
return [
m[0]*v[0] + m[3]*v[1] + m[6]*v[2],
m[1]*v[0] + m[4]*v[1] + m[7]*v[2],
m[2]*v[0] + m[5]*v[1] + m[8]*v[2]
];
}
//
var iframe = document.createElement("iframe");
iframe.src = "data/index.html";
iframe.style.position = "absolute";
iframe.style.left = "0";
iframe.style.top = "0";
iframe.style.transformOrigin = "0 0";
document.querySelector("main").appendChild(iframe);
var s = [
0,0,1,
iframe.offsetWidth,0,1,
0,iframe.offsetHeight,1
];
var v = apply(adjugate(s),[iframe.offsetWidth,iframe.offsetHeight,1]);
s = multiply(s,[
v[0], 0, 0,
0, v[1], 0,
0, 0, v[2]
]);
arController.addEventListener("getMarker",function(event) {
if (event.data.marker.id === marker) {
var d = [
event.data.marker.vertex[(4 - event.data.marker.dir) % 4][0],event.data.marker.vertex[(4 - event.data.marker.dir) % 4][1],1,
event.data.marker.vertex[(5 - event.data.marker.dir) % 4][0],event.data.marker.vertex[(5 - event.data.marker.dir) % 4][1],1,
event.data.marker.vertex[(7 - event.data.marker.dir) % 4][0],event.data.marker.vertex[(7 - event.data.marker.dir) % 4][1],1
];
var v = apply(adjugate(d),[event.data.marker.vertex[(6 - event.data.marker.dir) % 4][0],event.data.marker.vertex[(6 - event.data.marker.dir) % 4][1],1]);
d = multiply(d,[
v[0],0,0,
0,v[1],0,
0,0,v[2]
]);
var t = multiply(d,adjugate(s));
for (i = 0; i < 9; ++i) {
t[i] = t[i] / t[8];
t[i] = Math.abs(t[i]) < Number.EPSILON ? 0 : t[i];
}
t = [
t[0],t[1],0,t[2],
t[3],t[4],0,t[5],
0,0,1,0,
t[6],t[7],0,t[8]
];
iframe.style.transform = "matrix3d(" + t.join(", ") + ")";
} else {
// mesh.visible = false;
}
});
Might be an answer 4 years overdue, but it might be useful for somebody.
I've created an alternative solution.
You can check it here: https://github.com/jonathanneels/QrA
In essence;
QrA uses tilt-js to simulate 3D-effects (parallax) and AR.js a-box as the startobject.
The iframe gets pushed by the tilt.js functions and the AR cube's .position, .scale and .rotation is used as reference.
Enjoy!
Picking and objects code is one of the most popular:
function Picking(event) {
var raycaster = new THREE.Raycaster();
event.preventDefault();
mouse.x = (event.clientX / renderer.domElement.clientWidth) * 2 - 1;
mouse.y = -(event.clientY / renderer.domElement.clientHeight) * 2 + 1;
raycaster.setFromCamera(mouse, camera);
var intersects = raycaster.intersectObjects(objects);
if (intersects.length > 0) {
if (INTERSECTED != intersects[0].object) {
}
} else {
INTERSECTED = null;
}
}
The description: in scene are two objects - cube and sphere. Sphere is first to camera and cube is second. The Sphere has ID1 and cube ID2. Picking is working.
The problem: after deleting the sphere (scene.remove(sphere)); Picking is giving the ID1, so, it seems like sphere is invisible. What is the problam?
The picture of the example: Picture
This code is not giving the result:
for (i = sphere.children.length - 1; i >= 0 ; i--) {
object = sphere.children[i];
object.geometry.remove;
object.material.remove;
object.geometry.dispose();
object.material.dispose();
scene.remove(object);
sphere.remove(object);
}
or
recur(sphere);
for (var i in objects) {
objects[i].geometry.remove;
objects[i].material.remove;
objects[i].geometry.dispose();
objects[i].material.dispose();
scene.remove(objects[i]);
sphere.remove(objects[i]);
}
function recur(obj) {
if (obj instanceof THREE.Mesh) {
objects.push(obj);
}
for (var i in obj.children) {
recur(obj.children[i]);
}
}
The code that add objects:
var ObjLoader = new THREE.ObjectLoader();
var ii;
var group = new THREE.Object3D();
var oldModel = scene.getObjectByName('group');
if (oldModel !== undefined) { scene.remove(oldModel); }
ObjLoader.load("path/model.json", addModelToScene);
group.name = "group";
scene.add(group);
function addModelToScene(model) {
recur(model);
for (var i in objects) {
objects[i].castShadow = true;
objects[i].receiveShadow = true;
}
model.name = "ModelName";
group.add(model)
}
So, The .json model consists of some objects (1..n);
This .json model is added into group.
With picking the side of the cube (may be material) is selectable, but not removable. but the position can be changed.
Thank you.
Change your picking logic a little, store all your objects into array when creating them:
/*var objects = [];
var group = new THREE.Object3D();
create your mesh add to array*/
objects.push(MyMesh);
/*now we have a reference all the time in objects[0] , objects[1]*/
objects[0].displose(); check syntax been a while*/
customMaterial = new THREE.ShaderMaterial(
{
uniforms: customUniforms,
vertexShader: document.getElementById( 'vertexShader' ).textContent,
fragmentShader: document.getElementById( 'fragmentShader' ).textContent,
//wireframe: true,
side: THREE.FrontSide
} );
function ground (){
var map = new THREE.PlaneGeometry(1024,1024, 128, 128);//Use planebufferGeo
map.dynamic = true;
_terrain = new THREE.Mesh( map, customMaterial );
_terrain.material.needsUpdate = true;
_terrain.geometry.dynamic = true;
_terrain.geometry.vertexColors = true;
_terrain.material.uvsNeedUpdate = true;
_terrain.geometry.normalsNeedUpdate = true;
_terrain.rotation.x = -Math.PI / 2;
objects.push(_terrain);
scene.add(_terrain );
}
So I used this in a situation where it was a picking game type, I used an array to store the mesh and i could manipulate the vertices / faces etc edit the array to make changes.
It was an easier way to compare things and check data.
function animate() {
var raycaster = new THREE.Raycaster();
raycaster.setFromCamera(mouse, camera);
var intersects = raycaster.intersectObjects(objects);
if ( intersects.length > 0 ) {
var faceIndex = intersects[0].faceIndex;
var obj = intersects[0].object;
var geom = obj.geometry;
//etc
I have a multiple models in Collada format (knight.dae & archer.dae).
My problem is that i can't get them all to animate(lets say Idle with is 2-3frames).
When i load the scene i either get only one animated model and one stil model(no animation,no nothing,it's like he's being modeled in 3ds max).
I know my problem is with the skin and morphs but i searched alot and didnt find an answer and due to my lack of experience my attempts have failed so far.
Help pls!
//animation length of the model is 150(and it hosts 4 different animations)
var startFrame = 0, endFrame = 150, totalFrames = endFrame - startFrame, lastFrame;
var urls = [];
var characters = [];
urls.push('3D/archer/archer.dae');
urls.push('3D/archer/archer.dae');
//here's the loader
loader = new THREE.ColladaLoader();
loader.options.convertUpAxis = true;
for (var i=0;i<urls.length;i++) {
loader.load(urls[i],function colladaReady( collada ){
player = collada.scene;
player.scale.x = player.scale.y = player.scale.z =10;
player.position.y=115;
player.position.z=i*200;
player.updateMatrix()
skin = collada.skins [ 0 ];
//skinArray.push(skin);;
var mesh=new THREE.Mesh(new THREE.CubeGeometry(10,20,10,1,1,1));
player.add(mesh);
characters.push(mesh);
scene.add( player );
});
}
//i added the cube because i use raycaster and it doesnt detect collada obj
// Here is where i try my animation.
function animate() {
requestAnimationFrame( animate );
render();
}
function render() {
update();
renderer.render(scene,camera);
}
function update() {
var delta = clock.getDelta();
delta = delta / 2;
if ( t > 1 ) t = 0;
if ( skin )
{
skin.morphTargetInfluences[lastFrame] = 0;
var currentFrame = startFrame + Math.floor(t*totalFrames);
skin.morphTargetInfluences[currentFrame] = 1;
t += delta;
lastFrame = currentFrame;
}
}
Try something like this.... at the beginning:
var skins = [];
At your collada callback, something you already seem to have thought about:
skins.push(collada.skins[0]);
At your renderer, instead of the current if (skin) clause:
t += delta;
lastFrame = currentFrame;
var currentFrame = startFrame + Math.floor(t*totalFrames);
for (var i = 0; i < skins.length; i++) {
var skin = skins[i];
if (skin) {
skin.morphTargetInfluences[lastFrame] = 0;
skin.morphTargetInfluences[currentFrame] = 1;
}
}
Point being, you need to loop all the skins in the update() function. I didn't check the frame handling code very carefully as that was not the question.. If your skins have different amount of frames you need to take those into account in your code (maybe make the lastFrame, currentFrame etc variables to arrays matching the skins array).
if ( skinArray[0] && skinArray[1] )
{
skinArray[0].morphTargetInfluences[lastFrame] = 0;
skinArray[1].morphTargetInfluences[lastFrame] = 0;
var currentFrame = startFrame + Math.floor(t*totalFrames);
skinArray[0].morphTargetInfluences[currentFrame] = 1;
skinArray[1].morphTargetInfluences[currentFrame] = 1;
t += delta;
lastFrame = currentFrame;
}
I came up with this code,it does the job but i just don't like it,mainly because it feels it's hardcoded.So if any of you guys can come up with a more elegant solution i'd be more then happy.