I'm trying to make a WebVR environment using Three.js. I exported some scenes from Cinema4D and loaded them in with the Colladaloader of Three.js. Now I wanted to try this environment in my Google Cardboard but I needed to have the split screen for both my eyes, of course.
I used the npm module three-stereo-effect to achieve the VR effect, but it's overlapping when using it in a cardboard. I looked it up and saw that a lot of WebVR examples had a rounded rectangle for each eye (example of what I mean), not a straight rectangle, I thought I needed to find matrices to fix that (When looking at the examples of this repository). But then I downloaded a VR tunnel racing game and saw that they used straight rectangles and the vision was fine.
Now I'm thinking the eyeSeparation of my stereo effect is incorrect, I saw someone use the property eyeSeparation on the StereoEffect module and tried that out, but I think I shouldn't just be guessing a value...
Am I on the right track here to find a solution? Or am I looking in the total wrong direction why my 3D scene does not give a good vision when using a Cardboard?
This is the code I'm experimenting with.
import {sets} from './data/';
import * as THREE from 'three';
import threeOrbitControls from 'three-orbit-controls';
import ColladaLoader from 'three-collada-loader';
import threeStereoEffect from 'three-stereo-effect';
import {BufferLoader} from './modules/sound';
import {SpawnObject} from './modules/render';
const OrbitControls = threeOrbitControls(THREE);
const StereoEffect = threeStereoEffect(THREE);
let scene, camera, renderer;
let audioCtx, bufferLoader;
const notes = [];
let stereoEffect = null;
const init = () => {
window.AudioContext = window.AudioContext || window.webkitAudioContext;
audioCtx = new AudioContext();
bufferLoader = new BufferLoader(audioCtx);
bufferLoader.load(sets.drums)
.then(data => spawnObject(data));
initEnvironment();
};
const spawnObject = data => {
for (let i = 0;i < 5;i ++) {
const bol = new SpawnObject(`object.dae`, audioCtx, data[0], scene, false);
notes.push(bol);
}
// console.log(notes);
};
const initEnvironment = () => {
scene = new THREE.Scene();
camera = new THREE.PerspectiveCamera(
45, window.innerWidth / window.innerHeight,
1, 10000
);
renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
stereoEffect = new StereoEffect(renderer);
// stereoEffect.eyeSeparation = 1;
stereoEffect.setSize(window.innerWidth, window.innerHeight);
console.log(stereoEffect);
document.querySelector(`main`).appendChild(renderer.domElement);
camera.position.set(0, 0, 2);
camera.lookAt(scene.position);
new OrbitControls(camera);
//LIGHTS
const light = new THREE.PointLight(0xFFFFFF);
light.position.set(0, 0, 9);
light.castShadow = true;
light.shadow.mapSize.width = 1024;
light.shadow.mapSize.height = 1024;
light.shadow.camera.near = 10;
light.shadow.camera.far = 100;
scene.add(light);
// const hemiLight = new THREE.HemisphereLight(0xffffff, 0xffffff, 0.6);
// hemiLight.color.setHSL(0.6, 1, 0.6);
// hemiLight.groundColor.setHSL(0.095, 1, 0.75);
// hemiLight.position.set(0, 500, 0);
// scene.add(hemiLight);
//
// const dirLight = new THREE.DirectionalLight(0xffffff, 1);
// dirLight.color.setHSL(0.1, 1, 0.95);
// dirLight.position.set(- 1, 1.75, 1);
// dirLight.position.multiplyScalar(50);
// scene.add(dirLight);
// dirLight.castShadow = true;
//FLOOR
const matFloor = new THREE.MeshPhongMaterial();
const geoFloor = new THREE.BoxGeometry(2000, 1, 2000);
const mshFloor = new THREE.Mesh(geoFloor, matFloor);
matFloor.color.set(0x212E39);
mshFloor.receiveShadow = true;
mshFloor.position.set(0, - 1, 0);
scene.add(mshFloor);
//ENVIRONMENT
const loader = new ColladaLoader();
loader.load(`../assets/environment.dae`, collada => {
collada.scene.traverse(child => {
child.castShadow = true;
child.receiveShadow = true;
});
scene.add(collada.scene);
render();
});
};
const render = () => {
// stereoEffect.render(scene, camera);
// effect.render(scene, camera);
renderer.shadowMap.enabled = true;
renderer.shadowMap.type = THREE.PCFSoftShadowMap;
renderer.gammaInput = true;
renderer.gammaOutput = true;
renderer.setClearColor(0xdddddd, 1);
stereoEffect.render(scene, camera);
requestAnimationFrame(render);
};
init();
From the PDF "Scalable Multi-view Stereo Camera Array for Real World Real-Time Image Capture and Three-Dimensional Displays":
2.1.1 Binocular Disparity
Binocular disparity is the positional difference between the two retinal projections of a given point in space. This positional difference results from the fact that the two eyes are laterally separated and therefore see the world from the two slightly different vantage points. For the average person the mean lateral separation also known as the interocular is 65mm. Most of the population has an eye separation within ±10mm of the average interocular.
It would seem, with a little testing with friends with a variety of face shapes you will find a happy average for the eyeSeparation value for the device and the people using it. I would then also provide some settings panel which allows a few different settings of the eyeSeparation for users to choose from if they find disparity or overlap in their stereo experience. Normally I think this would be done with a keyboard connected to the same system to dial in the stereo alignment, but you're in cardboard, so the user may need trial and error to get it right.
Related
so I am a newbie towards threejs and basically have just started to learn and develop in threejs. I do understand due to limitations on mobile browsers there are some characteristics that is not present on desktop may be present on mobile. I however have run into a problem that I cant seem to google or find as to why it happened. I have created a very basic scene with the background color of black ‘0x000000’ and in this scene I have a sphere with wireframe as its material and its color is set to white. On desktop this renders perfect and shows up exactly as black background scene with white sphere geometry. However once Ive test deployed on a domain and accessing it through mobile, the scene and sphere color are totally inverted. I am still unsure what is causing it and any answer would be greatly appreciated. Thanks in advance!
This below code is how I create a canvas and a scene and a sphere.
import "../css/style.css";
import * as THREE from "three";
import { OrbitControls } from "three/examples/jsm/controls/OrbitControls.js";
import * as dat from "dat.gui";
// Debug
// const gui = new dat.GUI();
// Canvas
const canvas = document.querySelector("canvas.main-bg");
// Scene
const scene = new THREE.Scene();
scene.background = new THREE.Color(0x000000);
// Objects
const geometry = new THREE.SphereBufferGeometry(0.5, 15, 8);
// Materials
const material = new THREE.MeshBasicMaterial({
color: 0xffffff,
wireframe: true,
});
// Mesh
const sphere = new THREE.Mesh(geometry, material);
// Lights
// Point light
const pointLight = new THREE.PointLight(0x252525, 0.1);
pointLight.position.x = 2;
pointLight.position.y = 3;
pointLight.position.z = 4;
scene.add(pointLight);
// Sizes
const sizes = {
width: window.innerWidth,
height: window.innerHeight,
};
window.addEventListener("resize", () => {
// Update sizes
sizes.width = window.innerWidth;
sizes.height = window.innerHeight;
// Update camera
camera.aspect = sizes.width / sizes.height;
camera.updateProjectionMatrix();
// Update renderer
renderer.setSize(sizes.width, sizes.height);
renderer.setPixelRatio(Math.min(window.devicePixelRatio, 2));
});
// Base camera
const camera = new THREE.PerspectiveCamera(
75,
sizes.width / sizes.height,
0.1,
100
);
camera.position.x = 0;
camera.position.y = 0;
camera.position.z = 2;
scene.add(camera);
// Controls
// const controls = new OrbitControls(camera, canvas)
// controls.enableDamping = true
// Renderer
const renderer = new THREE.WebGLRenderer({
canvas: canvas,
});
renderer.setSize(sizes.width, sizes.height);
renderer.setPixelRatio(Math.min(window.devicePixelRatio, 2));
const clock = new THREE.Clock();
const animate = () =>
const elapsedTime = clock.getElapsedTime();
// Update objects
sphere.rotation.y = -0.2 * elapsedTime;
sphere2.rotation.y = 0.1 * elapsedTime;
sphere3.rotation.x = 0.7 * elapsedTime;
sphere.rotation.x += 0.5 * (targetY - sphere.rotation.x);
sphere.rotation.y += 0.5 * (targetX - sphere.rotation.y);
sphere.position.z += -0.5 * (targetY - sphere.rotation.x);
// Render
renderer.render(scene, camera);
// Call animate again on the next frame
window.requestAnimationFrame(animate);
};
animate();
Nothing else besides this is altering the scene.background color. This is how it looks like on desktop.
desktop image
And this is how it looks like on mobile.mobile
Nothing else is setting the canvas color however the colors are inverted on mobile, Any help is greatly appreciated! Thank you
You have added the CSS rule mix-blend-mode: exclusion to your <canvas>. For some reason, Safari is the only browser that's respecting that rule, and that's why it's showing inverted colors on mobile.
If you want a black background with white wireframes, just get rid of any mix-blend-modes in CSS so you get exactly the color you're defining.
so I'm trying to create an online game using Babylon.js but have run into a problem thats got me a little stumped so hoping someone here would be willing to help me out. Please bear with me on this one, i'm a complete newbie with babylon as i've only every worked with THREE.js. Right now my game consists of a scene compromising of multiple meshes with multiple users represented as avatars (created from basic circle geometry for the moment) loaded into an environment. What I want to do is highlight the outline of these avatars ONLY when they are occluded by any other object, meaning that when they are not occluded they look normal with no highlight but when behind an object their highlighted silhouette can be seen by others (including yourself as you can see your own avatar). This is very akin to effects used in many other video games (see example below).
Example of Effect
Thus far, based on some googling and forum browsing (Babylonjs outline through walls & https://forum.babylonjs.com/t/highlight-through-objects/8002/4) I've figured out how to highlight the outline of objects using Babylon.HighlighLayer and I know that i can render objects above others via RenderingGroups but I can't seem to figure out how to use them in conjunction to create the effect I want. The best i've managed to do is get the highlighted avatar render above everything but I need just the silhouette not the entire mesh. I'm also constrained by the fact that my scene has many meshes in it that are loaded dynamically and i'm also trying to keep things as optimal as possible. Can't afford to use very computationally expensive procedures.
Anybody know of the best way to approach this? Would greatly appreciate any advice or assistance you can provide.Thanks!
So I asked the same question on the babylon forums which helped me to find a solution. All credit goes to the guy's that helped me out over there but just in case someone else comes across this question seeking an answer, here is a link to that forum question https://forum.babylonjs.com/t/showing-highlighted-silhouette-of-mesh-only-when-it-is-occluded/27783/7
Edit:
Ok thought i'd include the two possible solutions here properly as well as their babylon playgrounds. All credit goes to roland & evgeni_popov who came up with these solutions on the forum linked above.
The first solution is easier to implement but slightly less performant than the second solution.
Clone Solution: https://playground.babylonjs.com/#JXYGLT%235
// roland#babylonjs.xyz, 2022
const createScene = function () {
const scene = new BABYLON.Scene(engine);
const camera = new BABYLON.ArcRotateCamera('camera', -Math.PI / 2, Math.PI / 2, 20, new BABYLON.Vector3(0, 0, 0), scene)
camera.attachControl(canvas, true);
const light = new BABYLON.HemisphericLight("light", new BABYLON.Vector3(0, 1, 0), scene);
light.intensity = 0.7;
const wall = BABYLON.MeshBuilder.CreateBox('wall', { width: 5, height: 5, depth: 0.5 }, scene)
wall.position.y = 1
wall.position.z = -2
const sphere = BABYLON.MeshBuilder.CreateSphere('sphere', { diameter: 2, segments: 32 }, scene)
sphere.position.y = 1
const sphereClone = sphere.clone('sphereClone')
sphereClone.setEnabled(false)
const matc = new BABYLON.StandardMaterial("matc", scene);
matc.depthFunction = BABYLON.Constants.ALWAYS;
matc.disableColorWrite = true;
matc.disableDepthWrite = true;
sphereClone.material = matc;
sphere.occlusionQueryAlgorithmType = BABYLON.AbstractMesh.OCCLUSION_ALGORITHM_TYPE_ACCURATE
sphere.occlusionType = BABYLON.AbstractMesh.OCCLUSION_TYPE_STRICT
const hl = new BABYLON.HighlightLayer('hl1', scene, { camera: camera })
hl.addMesh(sphereClone, BABYLON.Color3.Green())
hl.addExcludedMesh(wall);
let t = 0;
scene.onBeforeRenderObservable.add(() => {
sphere.position.x = 10 * Math.cos(t);
sphere.position.z = 100 + 104 * Math.sin(t);
if (sphere.isOccluded) {
sphereClone.setEnabled(true)
sphereClone.position.copyFrom(sphere.position);
} else {
sphereClone.setEnabled(false)
}
t += 0.03;
})
return scene;
};
This second solution is slightly more performant than above as you don't need a clone but involves overriding the AbstactMesh._checkOcclusionQuery function which is the function that updates the isOccluded property for meshes such that the mesh is always rendered even when occluded. There’s no overhead if you are using the occlusion queries only for the purpose of drawing silhouettes however If you are also using them to avoid drawing occluded meshes then there’s an overhead because the meshes will be drawn even if they are occluded. In which case your probably best of going with the first solution
Non-Clone solution: https://playground.babylonjs.com/#JXYGLT#14
// roland#babylonjs.xyz, 2022
const createScene = function () {
const scene = new BABYLON.Scene(engine);
const camera = new BABYLON.ArcRotateCamera('camera', -Math.PI / 2, Math.PI / 2, 20, new BABYLON.Vector3(0, 0, 0), scene)
camera.attachControl(canvas, true);
const light = new BABYLON.HemisphericLight("light", new BABYLON.Vector3(0, 1, 0), scene);
light.intensity = 0.7;
const wall = BABYLON.MeshBuilder.CreateBox('wall', { width: 5, height: 5, depth: 0.5 }, scene)
wall.position.y = 1
wall.position.z = -2
const sphere = BABYLON.MeshBuilder.CreateSphere('sphere', { diameter: 2, segments: 32 }, scene)
sphere.position.y = 1
sphere.occlusionQueryAlgorithmType = BABYLON.AbstractMesh.OCCLUSION_ALGORITHM_TYPE_ACCURATE
sphere.occlusionType = BABYLON.AbstractMesh.OCCLUSION_TYPE_STRICT
const mats = new BABYLON.StandardMaterial("mats", scene);
sphere.material = mats;
const hl = new BABYLON.HighlightLayer('hl1', scene, { camera: camera })
hl.addExcludedMesh(wall);
let t = 0;
const cur = BABYLON.AbstractMesh.prototype._checkOcclusionQuery;
scene.onDisposeObservable.add(() => {
BABYLON.AbstractMesh.prototype._checkOcclusionQuery = cur;
});
BABYLON.AbstractMesh.prototype._checkOcclusionQuery = function() {
cur.apply(this);
return false;
}
scene.onBeforeRenderObservable.add(() => {
sphere.position.x = 10 * Math.cos(t);
sphere.position.z = 100 + 104 * Math.sin(t);
if (sphere.isOccluded) {
hl.addMesh(sphere, BABYLON.Color3.Green())
mats.depthFunction = BABYLON.Constants.ALWAYS;
mats.disableColorWrite = true;
} else {
hl.removeMesh(sphere);
mats.depthFunction = BABYLON.Constants.LESS;
mats.disableColorWrite = false;
}
t += 0.03;
})
return scene;
};
I have scene with elements size over 500 units and i want to create mirror effect for them. to Reach descripted effect i used Reflector library from three.js webgl_mirror example.
I placed mirror on ground and most of meshes disappears or showing only small parts of surface when i set background hdri without its displayes normally. I builded other scene for tests and it looks like this unexpected effect begins when distance between mirror and obiect is over around 75 units (sometimes its less i dont know what its depends).
Image to preview on that effect
Is there any possibility that i could increase range of this clipping box size for that mirror? (i really want to avoid of scaling my actual created scene)
What i already tryed:
-changing my perspective camera far and near distances. - no effect
-manipulate paramets for clipBias and recursion or even increasing texture size. -no effect
-adding multiple lights around elements. -no effect
code that i used for experiment:
sceneSetup = () => {
//initialize
const width = this.mount.clientWidth;
const height = this.mount.clientHeight;
this.scene = new THREE.Scene();
let helperScene = this.scene;
this.camera = new THREE.PerspectiveCamera(60, width / height, 1, 500);
this.camera.position.z = 200;
this.controls = new OrbitControls(this.camera, document.body);
this.renderer = new THREE.WebGLRenderer();
this.renderer.setSize(width, height);
this.mount.appendChild(this.renderer.domElement); //render to container (React staff)
///Load HDR map
new RGBELoader()
.setDataType(THREE.UnsignedByteType)
.load(HdrFile, function(texture) {
var envMap = pmremGenerator.fromEquirectangular(texture).texture;
helperScene.background = envMap; // comment to see issue
helperScene.environment = envMap;
texture.dispose();
pmremGenerator.dispose();
});
var pmremGenerator = new THREE.PMREMGenerator(this.renderer);
pmremGenerator.compileEquirectangularShader();
//create ground mirror
let geometry = new THREE.PlaneBufferGeometry(200, 200);
let groundMirror = new Reflector(geometry, {
clipBias: 0,
textureWidth: 1024,
textureHeight: 1024,
color: 0x889999,
recursion: 1
});
groundMirror .position.z = -20;
groundMirror .rotation.x = Math.PI * -0.5;
//change groundMirror .position.y to -104 and evrything looks fine;
groundMirror .position.y = -105;
this.scene.add(groundMirror );
};
addCustomSceneObjects = () => {
//create cube for reflect
const geometry = new THREE.BoxGeometry(50, 50, 50);
const material = new THREE.MeshPhongMaterial({
color: 0x156289,
emissive: 0x072534,
side: THREE.DoubleSide,
depthTest: true,
depthWrite: true
});
this.cube = new THREE.Mesh(geometry, material);
this.cube.position.y = 0;
this.scene.add(this.cube);
//radding lights
const lights = [];
lights[0] = new THREE.PointLight(0xffffff, 1, 0);
lights[1] = new THREE.PointLight(0xffffff, 1, 0);
lights[2] = new THREE.PointLight(0xffffff, 1, 0);
lights[0].position.set(0, 200, 0);
lights[1].position.set(100, 200, 100);
lights[2].position.set(-100, -200, -100);
this.scene.add(lights[0]);
this.scene.add(lights[1]);
this.scene.add(lights[2]);
};
startAnimationLoop = () => {
//rotate cube
this.cube.rotation.x += 0.01;
this.cube.rotation.y += 0.01;
this.requestID = window.requestAnimationFrame(this.startAnimationLoop);
this.renderer.render(this.scene, this.camera);
};
Basically, anytime I try to load any .obj file, I get the following error.
THREE:ObjectLoader: Can't parse obj/weapon_4.obj. JSON.parse: unexpected character at line 1 column 1 of the JSON data
This happens with any file. No obj files that I've tried work. Here's one of the obj files I've tried to load that didn't work. I'll only post the first five lines since the error states that the "unexpected character" is on line one.
# Exported from Wings 3D 1.5.3
mtllib Tent_Poles_01.mtl
o Mesh1
#63 vertices, 122 faces
v -1.31380400 1.1423300e-15 -1.30752000
... the rest is basically just vertices
I can't figure out what's wrong with the obj files. I'm pretty much certain at this point It's something with my code. Here's my main.js file.
const d = document;
const $ = d.querySelector.bind(d);
let [w, h] = [innerWidth-10, innerHeight-10];
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(75, w/h, 0.1, 1000);
const renderer = new THREE.WebGLRenderer();
const initSize = () => {
[w, h] = [innerWidth-10, innerHeight-10];
renderer.setSize(w, h);
camera.aspect = w/h;
camera.updateProjectionMatrix();
};
addEventListener("resize", initSize);
addEventListener("load", initSize);
d.body.appendChild(renderer.domElement);
const controls = new THREE.OrbitControls(camera, renderer.domElement);
const loader = new THREE.ObjectLoader();
loader.load(
"obj/weapon_4.obj",
(object) => {
scene.add(object); // Error
}
);
const ambientLight = new THREE.AmbientLight(0xFFFFFF, 0.2);
const directionLight = new THREE.DirectionalLight(0xFFFFFF, 10);
const spotLight = new THREE.SpotLight(0xFF45F6, 25);
directionLight.position.set(1, 1, 0);
spotLight.position.set(0, 2, 0);
scene.add(ambientLight);
scene.add(spotLight);
scene.add(directionLight);
camera.position.z = 3;
const update = () => {
// todo...
};
const render = () => {
renderer.render(scene, camera);
};
const GameLoop = () => {
requestAnimationFrame(GameLoop);
update();
render();
};
GameLoop();
If anyone could help me get an obj file loaded that'd be great. I know the lighting is a mess, I was just messing around. I can provide any other files if necessary. Thanks in advance.
The loader is expecting the object to be in JSON Object/Scene file format, your obj file is not in that format.
References:
https://threejs.org/docs/#api/en/loaders/ObjectLoader
https://github.com/mrdoob/three.js/wiki/JSON-Object-Scene-format-4
You'll need to change your file, or use a different loader.
You have to use THREE.OBJLoader like demonstrated in the following example:
https://threejs.org/examples/webgl_loader_obj
You used THREE.ObjectLoader which is intended for loading the three.js JSON format.
I'm creating a 3D game, and I just began. However, I quickly ran into a problem with the localhost GET taking more than 2 minutes, and after like 15-45 seconds of life, then the canvas turns white and in console, I get a warn showing that the WebGL context has been lost. Also, in Task Manager, the game takes up 30% of the CPU and 100% of the GPU.
It is for a new online 3d multiplayer game. I've tried to dispose the memory after a new frame, but that didn't work. I've tried also to pre-load all the textures to use less CPU, but the 30% CPU remains the same. This is my code: (client-side)
var scene = new THREE.Scene();
var camera = new THREE.PerspectiveCamera(45, window.innerWidth / window.innerHeight, 1, 1000)
var renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
scene.autoUpdate = false;
var preLoad = new THREE.TextureLoader();
var sunTexture = preLoad.load("/static/sun-texture.jpg");
var mercuryTexture = preLoad.load("/static/mercury-texture.jpg");
var socketio = io();
var Geometries = [];
socketio.on("TX2", function (data) {
Geometries = [];
var collectedEntities = [];
data.objects.forEach(obj => {
collectedEntities.push(obj);
});
DisplayAllEntities(collectedEntities);
});
function DisplayAllEntities(objects) {
var loader;
objects.forEach(obj => {
if (obj.geometry == "sphere") {
if (obj.type != "ordinary_sphere") {
switch (obj.type) {
case "sun":
var material = new THREE.MeshBasicMaterial({ map: sunTexture });
break;
case "mercury":
var material = new THREE.MeshBasicMaterial({ map: mercuryTexture });
camera.position.z = obj.z + 500;
break;
}
}
loader = new THREE.TextureLoader();
var texture = loader.load(obj.texture)
var geometry = new THREE.SphereGeometry(obj.radius, 50, 50, 0, Math.PI * 2, 0, Math.PI * 2);
var mesh = new THREE.Mesh(geometry, material);
Geometries.push(mesh);
} else if (obj.geometry == "cube") {
loader = new THREE.TextureLoader();
var texture = loader.load(obj.texture)
var geometry = new THREE.CubeGeometry(obj.width, obj.height, obj.depth);
var material = new THREE.MeshBasicMaterial({ map: texture });
var mesh = new THREE.Mesh(geometry, material);
mesh.position = {"x": obj.x, "y": obj.y, "z": obj.z}
Geometries.push(mesh);
}
loader = null;
});
scene.children = [];
scene.dispose();
Geometries.forEach(obj => {
scene.add(obj);
});
render();
}
function render() {
requestAnimationFrame(render);
renderer.render(scene, camera);
}
The server just makes calculations about positions, and sends them to clients to render them.
I expect to get a lower CPU and GPU usage and a much lower load time, but the performance still remains the same.
It is very inefficient to create each object every time you need to render it, and especially to load the corresponding textures. A better solution would be set up the objects beforehand, and then to update these objects continually. This would require
a server emit for initializing (setting up objects, loading textures etc.)
a server emit for game state updates (adding/removing items if needed)
a server emit to update the positions. (this is the one that will execute 60 times per second)
Each object to have a unique id given by the server at creation so that the client knows which object server references.
This involves a bit more effort but would boost performance greatly