How does the .clone() method in Three.js save memory?

How does the .clone() method in Three.js save memory? - javascript

For the game I'm writing, I'm adding 100,000 trees, each of which is a merged geometry. When I add these from a cloned model using tree.clone(), I save tons of memory by doing this, but the game runs at 3 FPS because of the 100k geometries.
In order to get the game up to 60 FPS, I'd need to merge these trees together into a few geometries total. However, when I do this, chrome crashes due to using too much memory.
What's the reason for the extreme memory usage when merging these trees? Is it because I'm undoing the positives of using the .clone() function?

You need to look into instancing, your use case is exactly what the thing is made for.
Alternatively, using BufferGeometry instead of regular geometry, should be less memory intensive.
edit
What's actually eating your memory is the overhead of the merge operation when it operates on THREE.Geometry. The reason for this is that you have to allocate a ton of JS objects, like Vector3,Vector2, Face3, etc. Which then are discarded since the original geometry doesn't exist any more. This stresses everything, even if you didn't experience a crash, you'd experience slow down from garbage collection. The reason buffer geometry works better is because it's using typed arrays. For starters all your floats are floats not doubles, only primitives are being copied around, no object allocation etc etc.
You are eating more memory on the gpu though, since instead of storing one instance of geometry and referring to it in multiple draw calls, you now hold N instances within the same buffer (same data just repeated, and pre-transformed). This is where instancing would help. In summary you have:
Mesh (Node, Object)
Describes a 3d object within a "scene graph". Is it a child of some other node, does it have children, where its positioned (Translation), how its rotated, and is it scaled. This is the THREE.Object3D class. Extending from that is THREE.Mesh that references a geometry and a material, along with some other properties.
Geometry
Holds the geometry data, (which is actually referred to as "mesh" in modeling programs), file formats etc. hence the ambiguity. In your example that would be a single "tree":
describing how far away a leaf is from the root or how segmented is the trunk or branches (vertices)
where the leaves or trunk look up textures (UVS),
how it reacts to light (explicit normals , optional, but there are actually techniques for rendering foliage with overriden/modified/non-regular normals)
What's important to understand is that this stuff exists in "object (model) space". Let's say that a modeler modeled this, this would mean that he designated the object as "vertical" (the trunk goes up say Z axis, the ground is considered XY) thus giving it the initial Rotation, he put the root nicely at 0,0,0 thus giving it the initial Translation, by default we can assume that the Scale part is 1,1,1.
This tree can now be scattered around a 3d scene, be it in a modeling program or three.js. Let's say we import it into something like Blender. It would come in at the center of the world, at 0,0,0, rotated by 0,0,0, and at its own scale of 1,1,1. We realize that the modeler worked in inches, and our world in meters, so we scale it in all three directions by some constant. Now we realize that the tree is under ground, so we move it up by X units, until it sits on the terrain. But now its going through a house, so we move it sideways, or perhaps in all three directions because the house is on a hill and now it sits where we want it. We now observe that the silhouette is not aesthetically pleasing so we rotate it around the "vertical" axis by N degrees.
Let's observe now what happened. We haven't made a single clone of the tree, we scaled it, moved it around, and rotated it.We haven't modified the geometry in any way (adding leaves, branches, deleting something, changing uvs), it's still the same tree. If we say that the geometry is it's own entity, we have a scene graph node that has its TRS set. In context of three.js this is THREE.Mesh (inherits from Object3D), which has .rotation,.scale, position(translation) and finally .geometry (in your case a "tree").
Now lets say that we need a new tree for the forest. This tree is actually going to be the exact copy of the previous tree, but residing at a different place (T), rotated along the Z axis (R), and scaled non-uniformly (S). We only need a new node that has a different TRS, lets call it tree_02, it's using the same geometry, lets call it treeGeometryOption_1. Since tree geometry one has a defined set of UVS, it also has a corresponding texture. A texture goes into a material, and the material describes the properties, how shiny is the leaf, how dull is the trunk, is it using a normal map, does it have a color overlay, etc.
This means that you can have some sort of TreeMasterMaterial that sets these properties, and then have a treeOptionX_material corresponding to a geoemetry. Ie. if a leaf looks up the uvs in some range, the texture there should be green, and more shiny, then the range that trunk looks up.
Now lets reiterate the entire process. We imported the initial tree model, and gave it some scale, rotation and position. This is a node, with a geometry attached to it. We then made multiple copies of that node, which all refers to the same geometry TreeOption1. Since the geometry is the same, all of these clones can have the same material treeOption1_material which has it's own set of textures.
This was all a super lengthy explanation of why the clone code looks like this:
return
new this.constructor( //a new instance of Mesh
this.geometry , //with the sources geometry (reference)
this.material //with the sources material (reference)
).copy(this) //utility to copy other properties per class (Points, Mesh...)
The other answer is misleading and makes it sound like:
return
new this.constructor( //a new instance of Mesh
this.geometry.clone() , //this would be devastating for memory
this.material.clone() //this is actually sort of common to be done, but it would be done after the clone operation
).copy(this)
Let's say we want to tint the trees to have different colors, for example 3.
var materialOptions = [];
colorOptions.forEach( color =>{
var mOption = masterTreeMaterial.clone()
//var mOption = myImportedTreeMesh.material.clone(); //lets say youve loaded the mesh with a material
mOption.color.copy( color ); //generate three copies of the material with different color tints
materialOptions.push( mOption );
});
scatterTrees( myImportedTreeMesh , materialOptions );
//where you would have something like
var newTree = myImportedTreeMesh.clone(); //newTree has the same geometry, same material - the master one
newTree.material = someMaterialOption; //assign it a different material
//set the node TRS
newTree.position.copy( somePosition );
newTree.rotation.copy( someRotation );
newTree.scale.copy( someScale );
Now what happens is that this generates many many draw calls. For each tree to be drawn, a set of low level instructions need to be sent to set up the uniforms (matrices for TRS ), textures (when trees with different materials are drawn) and this produces overhead. If these are combined and the draw call number is reduced, the overhead is reduced, and webgl can handle transforming many vertices, so a low poly object can be drawn at 60fps thousands of times, but not with thousands of draw calls.
This is the cause of your 3 fps result.
Fancy optimizations aside, the brute force way would be to merge multiple trees into a single object. If we combine multiple THREE.Mesh nodes into one, we only have room for one TRS. What do we do with the thousands of individual trees scattered over the terrain, what happened to their TRSs? They get baked into geometry.
First of all, each node now requires a clone of the geometry, since the geometry is going to be modified. The first step is multiplying every vertex by the TRS matrix of that node. This is now a tree that does not sit at 0,0,0 and is no longer in inches, but sits somewhere at XYZ relative to the terrain, and is in meters.
After doing this a thousand times, these thousand individual tree geometries need to be merged into one. Easy peasy, it's just making a new geometry, and filling it's vertices, faces, and uvs with these thousand new geometries. You can imagine the overhead thats involved when these numbers are high, JS has its limitations, GC can be slow, and this is a lot of data, because it's 3d.
This should answer the question from the title. If we go in reverse, we can say that you had a game that was consuming lots of memory ( by having a model - geometry, of a "forest") but was running at 60fps. You used the clone method to save memory, by breaking the forest apart into individual trees, extracting the TRS at every root, and by using a single tree reference for each node.
Now the gpu holds only a low poly model of a tree, instead of holding a giant model of a forest. Memory saved. Draw call abundance.
FPS RIP.
How do both, reduce the number of draw calls, while rendering a model of a tree, not a forest?
By using a feature called instancing. Webgl allows for a special draw call to be issued. It uses an attribute buffer to set up multiple TRS information at the same time. 10000 nodes would have a buffer of 10000 TRS matrices, this is 16 floats, describing a single triangle with no uvs or normals takes 9, so you can see where this is going. You hold one instance of the tree geometry on the gpu, and instead of setting up thousands of draw calls, you set one with the data for all of them. If the objects are static, the overhead is minimal, since you'd set both buffers at once.
Three.js abstracts this quite nicely with THREE.InstancedBufferGeometry (or something like that).
One tree, many nodes:
T memory
N draw calls
One forest, single node:
T * N memory
1 draw call
One tree, instanced many times:
T + N memory (kind of, but it's mostly just T)
1 draw call
/edit
100k is quite a lot, three is probably better at handling this than before, but it used to tank your fps whenever you had more than 65536 vertices. I'm not sure off the top of my head, but it's now addressed by either breaking it up into multiple drawcalls internally, or the fact that webgl can address more than 2^16 vertices.
I save tons of memory by doing this, but the game runs at 3 FPS because of the 100k geometries.
You still have one geometry, you have 100k "nodes" that all point to the same instance of geometry. It's the overhead of the 100k draw calls that is slowing this down.
There are many ways to skin this cat.

The problem of memory has nothing to do with the usage or non-usage of the clone() function, which simply makes a separate copy of your object geometry and material.
The memory problem is due to the fact that the program is reaching the memory limit while merging.
The mergin process is an expensive process during setup (loop through all vertices, faces of both objects and create a third combined object). Plus, the merged geometry you are building is getting too large at a certain point to be processed and causes the browser to crash.
Have you tried to split the merged geometry into smaller chunks of 100 objects instead of using 1 merged object to store the 100K trees?
To optimize the code, I would also look at the tree mesh itself and see if you can reduce the number of vertices and faces.
For the usage of that many trees, you might want to consider using tricks and have only a few real mesh for the trees that are close to the camera and for the ones further away use a "waterdown" version (2-3 planes mesh that would need to be merged as well).

Related

Removing and Adding TextGeometry freezes the scene

I am trying to create a clock with the help of Text Geometry. In order to update the time I need to update the text in Text Geometry which can be done by removing and recreating a new Text Geometry.
Every time I add a new Text Geometry it freezes my browser:
// Remove old mesh
earthClockMesh.geometry.dispose();
earthClockMesh.material.dispose();
group.remove(earthClockMesh);
//add new mesh
earthClockMesh = this.getTextMesh(
new Date(diluatedTime).toLocaleString(),
textMaterial
);
group.add(earthClockMesh);
Anybody know better way way we can update the text in Text Geometry without freezing browser.
Live Example
https://codesandbox.io/s/peaceful-boyd-x859m
You can see the particles gets freeze for a moment when TextGeometry changed

Using THREE.TextBufferGeometry will improve the performance since it produces much less object allocation than TextGeometry. Besides, each instance of THREE.Geometry is internally converted to THREE.BufferGeometry before the first rendering. If you additionally lower the amount of curveSegments, the should be almost no noticeable lag anymore.
three.js R107

The reason for this is that you're deleting the generated geometry, then re-building thousands and thousands of triangles each second. This is very computationally expensive, and you see the animation freeze while the CPU tries to catch up. This is what you're doing:
Text geometry gets disposed
CPU re-builds all characters with an updated second (first bottleneck)
New geometry data gets passed to GPU (second bottleneck)
Scene gets rendered smoothly while no geom is being rebuilt
With real-time graphics (videogames, visualizers, etc), the geometry construction typically happens at the beginning of the app to avoid these mid-game stutters. Try to generate the geometry only once, then swap it out as necessary:
Create a "dictionary" of all 16 necessary characters as individual Mesh objects: 0123456789:/APM,
With this base dictionary, you can .clone() the needed characters, then place them with .position. You can use .clone(false).
At the end of each second, .clone() the geometry from the dictionary into the few characters you need to update.
The beauty of cloning Meshes like this is that the geometry is only generated once. You don't need to spend tons of processing power re-building it.

Three.js - How to group together multiple imported models

I'm attempting to reduce the amount of drawcalls in my THREE.js scene. I have a large 32x32 of tiles that are .gtlf models imported from disk. Some grid tiles get a model (that has just 8 vertices and a texture). However, inspecting this canvas with a third part tool like Spector shows that there's 151 draw calls, and it seems like every model is being drawn separately.
They are created by looping through the grid for every tile, seeing if that tile needs a model, if so, it gets the model data from a Map where it was loaded with the THREE.GLTFLoader. Sometimes, but rarely, the material changes and so may the rotation, but only during setup.
These tiles will never need to be moved or have their transforms changed. They will be static. Is there some sort of optimisation I can perform to group the rendering of these tiles into as few drawcalls as possible, seeing as they are in essence the same object?

You can merge your tiles into one geometry.
Something along the lines of this
const mergedGeom = new THREE.BufferGeometry()
tiles.forEach( tile=>{
const geom = tile.geometry.clone()
geom.applyMatrix(tile.matrixWorld)
mergedGeom.merge(geom)
}
If you do have many different materials and textures you will somehow have to manage that though.

Threejs - Best way to handle partial transforms on geometry vertices

I have mesh with a geometry of about 5k vertices. This is the result of multiple geometries being merged so it's all in a flat array. So far all good, I can modify individual vertices and by setting the flag verticesNeedUpdate=true I can see the changes reflected on my mesh. My problem is that I need to make translations and rotations on some of this vertices and I was wondering if there is better way of applying transforms for each collection apart from modifying each vertex position inside a loop to apply the transforms.
One idea I had was to create a new geometry and assign the vertices subset (by reference) so I could then do the transforms but that seems weird so I stopped and ask this question instead.
I took a look at this example https://threejs.org/examples/?q=constr#webgl_buffergeometry_constructed_from_geometry but I have no idea how would I go about rotating/scaling groups of vertices.
Also from what I understand this will reduce the calls to the GPU by uploading only one set of vertices instead of having hundreds of calls. But then I wonder if modifying this geometry vertices on each frame defeats the all purpose of doing all this?

As i can see, the example creates multiple heartshapes (geometry) and applies a transformation, then the transformed vertices are combined to one geometry (bufferGeometry). So all hearts are in the same geometry and drawn in one call. The downside is, that you can't manipulate the heart individually.
The clue here is that the transformations are done initially and the transformed coords are uploaded to the gpu. You don't want to update the vertices each frame by the cpu.
geometry.lookAt( vector );
geometry.translate( vector.x, vector.y, vector.z );
is responsible for transforming the vertices, before they are added to the bufferGeometry.
If you can add an 'index' to each vertex, you could use a UBO for storing matrices and give vertices different transformations (in vertexshader) within the same drawcall.

Copy mesh thousand times and animate without big performance hit?

I have a demo where i used hundreds of cubes that have exactly same geometry and texture, example:
texture = THREE.ImageUtils.loadTexture ...
material = new THREE.MeshLambertMaterial( map: texture )
geometry = new THREE.BoxGeometry( 1, 1, 1 )
cubes = []
for i in [0..1000]
cubes.push new THREE.Mesh geometry, material
... on every frame
for cube in cubes
// do something with each cube
Once all the cubes are created i start moving them on the screen.
All of them have the same texture, same size, they just change position and rotation. The problem here is that when i start using many hundreds of cubes the computer starts to suffer to render it.
Is there any way i could tell Three.js / WebGL that all those objects are the same object, they are identical copies just in different positions ?
I ready something about BufferGeometry and Geometry2 being able to boost performance for this sort of situation but i'm not exactly sure what would be the best in this case.
Thank you

Is there any way i could tell Three.js / WebGL that all those objects are the
same object, they are identical copies just in different positions ?
Unfortunately there's nothing that can automatically determine and optimize rendercalls in that regard. That would be pretty awesome.
I read something about BufferGeometry and Geometry2 being able to boost performance for this sort of situation but i'm not exactly sure what would be the best in this case.
So, the details here is this: the normal THREE.Geometry-class three.js provides is built for developer-convenience, but is a bit far from how data is handled by WebGL. This is what the DirectGeometry (earlier called Geometry2) and BufferGeometry are for. A BufferGeometry is a representation of how WebGL expects data for drawcalls to be held: it contains a typed-array for every attribute of the geometry. The conversion from Geometry to BufferGeometry happens automatically every time geometry.verticesNeedsUpdate is set to true.
If you don't change any of the attributes, this conversion will happen once per geometry (of which you have 1) so this is completely ok and moving to a buffer-geometry won't help (simply because you are already using it).
The main problem you face with several hundred geometries is the number of drawcalls required to render the scene. Generally speaking, every instance of THREE.Mesh represents a single drawcall. And those drawcalls are expensive: A single drawcall that outputs hundred thousands of triangles is no problem at all, but a thousands of drawcalls with 100 triangles each will very quickly become a serious performance problem.
Now, there are different ways how the number of drawcalls can be reduced using three.js. The first is (as already mentioned in comments) to combine multiple meshes/geometries into a single one (in the end, meshes are just a collection of triangles, so there's no requirement that they form a single "body" or something like that). This isn't too practical in your case as this would involve applying the position and rotation of each of your cubes via JS and update the vertex-arrays accordingly on each frame.
What you are really looking for is a WebGL-feature called geometry instancing.
This is not as easy to use as regular meshes and geometries, but not too complicated either.
With instancing, you can create a huge amount of objects in a single drawcall. All of the rendered objects will share a single geometry (your cube-geometry with its vertices, normals and uv-coordinates). The instancing happens when you add special attributes named InstancedBufferAttribute that can contain independent values for each of the instances. So you could add two per-instance attributes for position and rotation (or a single instance transformation-matrix if you like).
These examples should pretty much be what you are looking for:
http://threejs.org/examples/?q=instancing
The only difficulty with instancing as of now is the material: you will need to provide a custom vertex-shader that knows how to apply your per-instance-attributes to the vertex-positions from the original geometry (this can also be seen in the code of the examples).

You have a webgl tag so Im going to give a non three js answer.
The best way to handle this is to allocate a float texture array made of model transform matrix data (or just vec3 positions if thats all you need). Then you allocate a mesh chunk containing all your cube data. You need to add an additional attribute which I refer to as modelTransform index. For each "cube instance" in the mesh chunk, write the correct modelTransform index value corresponding to the correct offset in the model transform data texture.
On each frame, you calculate the correct model transform data for all the cubes and write to the model transform data texture with correct offsets and such. Upload the texture to GPU on each frame.
In the vertex shader, access the model transform data from the modelTransform index attribute and the float texture. Rest is the same.
This is what I am using in my engine and it works well for smallish objects such as cubes. Note however, updating 150000 cubes on 60 FPS will likely take most of your CPU resources from JS. This is unavoidable regardless of which instancing scheme you take.
If the motion/animation of each cube is fixed, then a even better way to do it is to upload a velocity attribute and initial creation time stamp attribute for each cube instance. On each frame, send the current time as uniform and calculate the position as "pos += attr_velocity * getDeltaTime(attr_initTime, unif_currentTime);". This skips work on CPU all together and allows you to render a much higher number of cubes.

Best practice: Rendering volume (voxel) based data in WebGL

I´m searching for a (or more) best practice(s) for the following problem. I´ll try to describe it as abstract as possible, so the solution can be applied to scenarios i have not yet thought of.
Data available: Voxels (Volumetric Pixels), forming a cube, with coordinates x,y,z and a color attached.
Goal: Use OpenGL to display this data, as you move through it from different sides.
Question: Whats the best practice to render those voxels, depending on the viewpoint? How (which type of Object) can store the data?
Consider the following:
The cube of data can be considered as z layers of x y data. It should
be possible to view, in-between-layers, then the displayed color
should be interpolated from the closest matching voxels.
For my application, i have data sets of (x,y,z)=(512,512,128) and
more, containing medical data (scans of hearts, brains, ...).
What i´ve tried so far:
Evaluated different frameworks (PIXI.js, three.js) and worked through a few WebGL tutorials.
If something is not yet clear enough, please ask.

There are 2 major ways to represent / render 3D datasets. Rasterization and Ray-tracing.
One fair rasterization approach is a surface reconstruction technique by the use of algorithms such as Marching Cubes, Dual Contouring or Dual Marching Cubes.
Three.js have a Marching Cubes implementation in the examples section. You basically create polygons from your voxels for classical rasterization. It may be faster than it seems. Depending the level of detail you want to reach, the process can be fast enough to be done more than 60 times per second, for thousands of vertices.
Although, unless you want to simply represent cubes (I doubt) instead of a surface, you will also need more info associated to each of your voxels rather than only voxel positions and colors.
The other way is raycasting. Unless you find a really efficient raycasting algorithm, you will have serious performance hit with a naive implementation.
You can try to cast rays from your camera position through your data structure, find / stop marching through when you reach a surface and project your intersection point back to screen space with the desired color.
You may draw the resulting pixel in a texture buffer to map it on a full-screen quad with a simple shader.
In both cases, you need more information than just colors and cubes. For example, you need at least density values at each corners of your voxels for Marching cubes or intersection normals along voxels edges (hermite data) for Dual Contouring.
The same for ray-casting, you need at least some density information to figure out where the surface lies or not.
One of the keys is also in how you organize the data in your structure specially for out-of-core accesses.

Develop Reference

JavaScript is the programming language of the Web.