I'm trying to draw colored lines in my browser -- there are no meshes involved. I currently query a MySQL database where geometry and other attributes are stored, and translate these objects into text blocks that each produce a single line object to be read by threejs.
var geometry = new THREE.Geometry();
var material = new THREE.LineBasicMaterial({
color: 0xF2AA20,
linewidth: 5});
var line = new THREE.Line(geometry, material);
geometry.vertices.push(new THREE.Vector3(1040,-406,-760));
geometry.vertices.push(new THREE.Vector3(1040,-406,-709));
scene.add(line);
This ultimately produces a stand-alone html document with data hard-coded: http://thomasshouler.com/datavis/wrapper/axial.html
I need to make this more scalable. We're currently loading ~1,300 of the 170 million records that will ultimately need to be visualized. An initial logical step would seemingly be to store the objects and attributes in separate files (JSON?) that could then be loaded into an html template (certainly cleaner, I expect efficient?).
I assume this would be pretty simple for most, but examples I find are typically far more complex and therefore difficult for me to incorporate into my work. Could somebody please provide an example which loads a single line via JSON format into the browser?
If it's going to be displayed statically (ie no frame by frame updates) then simply drawing on a canvas is probably the best approach as redrawing a few thousand lines might be fine for a GPU, but millions is definitely pushing it (1). linewidth is also not supported everywhere in WebGL but works fine with the canvas methods. Once drawn on a canvas there is no frame by frame redrawing required, and you can bypass three.js entirely if you don't need the 3D. And yes, you should export the data you need from the database as JSON, then process that in a loop rather than building a HTML page with all the instructions.
(1) You wouldn't be able to fill an image with millions of lines and not display a terrible mess in the process, so you'll have to filter the data in some way.
Edit: Also have a look at Circos, which might just be what you need.
Related
I am trying to create a clock with the help of Text Geometry. In order to update the time I need to update the text in Text Geometry which can be done by removing and recreating a new Text Geometry.
Every time I add a new Text Geometry it freezes my browser:
// Remove old mesh
earthClockMesh.geometry.dispose();
earthClockMesh.material.dispose();
group.remove(earthClockMesh);
//add new mesh
earthClockMesh = this.getTextMesh(
new Date(diluatedTime).toLocaleString(),
textMaterial
);
group.add(earthClockMesh);
Anybody know better way way we can update the text in Text Geometry without freezing browser.
Live Example
https://codesandbox.io/s/peaceful-boyd-x859m
You can see the particles gets freeze for a moment when TextGeometry changed
Using THREE.TextBufferGeometry will improve the performance since it produces much less object allocation than TextGeometry. Besides, each instance of THREE.Geometry is internally converted to THREE.BufferGeometry before the first rendering. If you additionally lower the amount of curveSegments, the should be almost no noticeable lag anymore.
three.js R107
The reason for this is that you're deleting the generated geometry, then re-building thousands and thousands of triangles each second. This is very computationally expensive, and you see the animation freeze while the CPU tries to catch up. This is what you're doing:
Text geometry gets disposed
CPU re-builds all characters with an updated second (first bottleneck)
New geometry data gets passed to GPU (second bottleneck)
Scene gets rendered smoothly while no geom is being rebuilt
With real-time graphics (videogames, visualizers, etc), the geometry construction typically happens at the beginning of the app to avoid these mid-game stutters. Try to generate the geometry only once, then swap it out as necessary:
Create a "dictionary" of all 16 necessary characters as individual Mesh objects: 0123456789:/APM,
With this base dictionary, you can .clone() the needed characters, then place them with .position. You can use .clone(false).
At the end of each second, .clone() the geometry from the dictionary into the few characters you need to update.
The beauty of cloning Meshes like this is that the geometry is only generated once. You don't need to spend tons of processing power re-building it.
I'm attempting to reduce the amount of drawcalls in my THREE.js scene. I have a large 32x32 of tiles that are .gtlf models imported from disk. Some grid tiles get a model (that has just 8 vertices and a texture). However, inspecting this canvas with a third part tool like Spector shows that there's 151 draw calls, and it seems like every model is being drawn separately.
They are created by looping through the grid for every tile, seeing if that tile needs a model, if so, it gets the model data from a Map where it was loaded with the THREE.GLTFLoader. Sometimes, but rarely, the material changes and so may the rotation, but only during setup.
These tiles will never need to be moved or have their transforms changed. They will be static. Is there some sort of optimisation I can perform to group the rendering of these tiles into as few drawcalls as possible, seeing as they are in essence the same object?
You can merge your tiles into one geometry.
Something along the lines of this
const mergedGeom = new THREE.BufferGeometry()
tiles.forEach( tile=>{
const geom = tile.geometry.clone()
geom.applyMatrix(tile.matrixWorld)
mergedGeom.merge(geom)
}
If you do have many different materials and textures you will somehow have to manage that though.
Let's say I have a 3DObject in three.js:
const geometry = new THREE.BoxGeometry(1, 1, 1);
const material = new THREE.MeshStandardMaterial({ color: 0xff0000 });
const mesh = new THREE.Mesh(geometry, material);
This mesh is also dynamic, that means, user sets the size of it, which is dynamically set as the scale of the mesh.
widthInput.subscribe(value => this.mesh.scale.x = +value); // just to get the idea
So, I know it's possible to set separate materials to the different sides of geometry. I'm also aware that it should be possible to set it on separate segments of that geometry's sides (if I'd had more).
The problem is that the user can set the width from the range 200 - 260, but I need a different material on the very right of the mesh with a fixed size of 10. I'm not really sure how would I do that without creating another geometry. Is there any way to set the material on the fixed part of the mesh? or is there any way to set the segments the way one of them will always have the fixed size? Thank you in advance.
To visualize the problem (white area needs to have a fixed width of 10 while the red area resizes)
Is there any way to set the material on the fixed part of the mesh?
As you've already mentioned there is a way to set different materials on different parts of the geometry. The problem here is defining what fixed means:
or is there any way to set the segments the way one of them will always have the fixed size?
Yes. You'd have to modify the geometry yourself. Reach into say g.attributes.position.array and modify the vertices that make the segment. It's lower level and different than the scene graph.
There may not be a good reason for wanting to keep everything in the same geometry. It would make sense if you used vertex colors for example to paint the different segments, and perhaps animated the extrusion in GLSL rather than with scale.set(). But since you want to apply different materials and are not writing GLSL, you will end up with multiple draw calls anyway.
You actually can save a bit of memory by storing just the simple cube and referencing it twice, than storing extra vertices and faces. So what you're trying to do is more likely going to consume more memory, and have the same amount of render overhead. In which case, doing everything with the scene graph, with two meshes and one geometry (you dont need to duplicate the box, you only need two nodes) should be as performant, and much easier to work with.
I have a demo where i used hundreds of cubes that have exactly same geometry and texture, example:
texture = THREE.ImageUtils.loadTexture ...
material = new THREE.MeshLambertMaterial( map: texture )
geometry = new THREE.BoxGeometry( 1, 1, 1 )
cubes = []
for i in [0..1000]
cubes.push new THREE.Mesh geometry, material
... on every frame
for cube in cubes
// do something with each cube
Once all the cubes are created i start moving them on the screen.
All of them have the same texture, same size, they just change position and rotation. The problem here is that when i start using many hundreds of cubes the computer starts to suffer to render it.
Is there any way i could tell Three.js / WebGL that all those objects are the same object, they are identical copies just in different positions ?
I ready something about BufferGeometry and Geometry2 being able to boost performance for this sort of situation but i'm not exactly sure what would be the best in this case.
Thank you
Is there any way i could tell Three.js / WebGL that all those objects are the
same object, they are identical copies just in different positions ?
Unfortunately there's nothing that can automatically determine and optimize rendercalls in that regard. That would be pretty awesome.
I read something about BufferGeometry and Geometry2 being able to boost performance for this sort of situation but i'm not exactly sure what would be the best in this case.
So, the details here is this: the normal THREE.Geometry-class three.js provides is built for developer-convenience, but is a bit far from how data is handled by WebGL. This is what the DirectGeometry (earlier called Geometry2) and BufferGeometry are for. A BufferGeometry is a representation of how WebGL expects data for drawcalls to be held: it contains a typed-array for every attribute of the geometry. The conversion from Geometry to BufferGeometry happens automatically every time geometry.verticesNeedsUpdate is set to true.
If you don't change any of the attributes, this conversion will happen once per geometry (of which you have 1) so this is completely ok and moving to a buffer-geometry won't help (simply because you are already using it).
The main problem you face with several hundred geometries is the number of drawcalls required to render the scene. Generally speaking, every instance of THREE.Mesh represents a single drawcall. And those drawcalls are expensive: A single drawcall that outputs hundred thousands of triangles is no problem at all, but a thousands of drawcalls with 100 triangles each will very quickly become a serious performance problem.
Now, there are different ways how the number of drawcalls can be reduced using three.js. The first is (as already mentioned in comments) to combine multiple meshes/geometries into a single one (in the end, meshes are just a collection of triangles, so there's no requirement that they form a single "body" or something like that). This isn't too practical in your case as this would involve applying the position and rotation of each of your cubes via JS and update the vertex-arrays accordingly on each frame.
What you are really looking for is a WebGL-feature called geometry instancing.
This is not as easy to use as regular meshes and geometries, but not too complicated either.
With instancing, you can create a huge amount of objects in a single drawcall. All of the rendered objects will share a single geometry (your cube-geometry with its vertices, normals and uv-coordinates). The instancing happens when you add special attributes named InstancedBufferAttribute that can contain independent values for each of the instances. So you could add two per-instance attributes for position and rotation (or a single instance transformation-matrix if you like).
These examples should pretty much be what you are looking for:
http://threejs.org/examples/?q=instancing
The only difficulty with instancing as of now is the material: you will need to provide a custom vertex-shader that knows how to apply your per-instance-attributes to the vertex-positions from the original geometry (this can also be seen in the code of the examples).
You have a webgl tag so Im going to give a non three js answer.
The best way to handle this is to allocate a float texture array made of model transform matrix data (or just vec3 positions if thats all you need). Then you allocate a mesh chunk containing all your cube data. You need to add an additional attribute which I refer to as modelTransform index. For each "cube instance" in the mesh chunk, write the correct modelTransform index value corresponding to the correct offset in the model transform data texture.
On each frame, you calculate the correct model transform data for all the cubes and write to the model transform data texture with correct offsets and such. Upload the texture to GPU on each frame.
In the vertex shader, access the model transform data from the modelTransform index attribute and the float texture. Rest is the same.
This is what I am using in my engine and it works well for smallish objects such as cubes. Note however, updating 150000 cubes on 60 FPS will likely take most of your CPU resources from JS. This is unavoidable regardless of which instancing scheme you take.
If the motion/animation of each cube is fixed, then a even better way to do it is to upload a velocity attribute and initial creation time stamp attribute for each cube instance. On each frame, send the current time as uniform and calculate the position as "pos += attr_velocity * getDeltaTime(attr_initTime, unif_currentTime);". This skips work on CPU all together and allows you to render a much higher number of cubes.
I am creating a map with leaflet and d3. A lot of circles will be plotted on a map. In terms of browser compatibility, there is an expected limit of how many svg elements the browser can render. In terms of user experience however, I would prefer that the user can see as many elements on the map as possible (otherwise the user might need to zoom in and out constantly and would need to wait for the ajax to return data). There will be some optimisation that I need to consider (user waiting time user vs. server query load vs. what the browser can handle).
See plot, there is a limit right now on the number of points that the server returns and thus only a portion of the map is filled.
The browser cannot handle a fully filled map here and the user would need to wait too long for the server response as well.
I suppose the problem that I am faced with needs to be solved by answering two questions:
Is there a standard in terms of what the average browser can handle in terms of number of simple svg shapes (circles) on a map?
What is the best technique to show as many shapes on the map as possible?
I'm considering the following points but I am unsure if it will help;
use squares instead of circles
use the leaflet API instead of the D3
Speaking in general terms, neither of the points you're considering will help. In both cases, the amount of processing to be done / information to display by the browser will be approximately the same.
Regarding your first question, not that I'm aware of. There are huge variations between browsers and platforms (especially if you consider mobile devices as well) and an average would be almost meaningless. Furthermore, this is changing constantly. I've found that up to about 1000 simple shapes are usually not a problem.
To show as many shapes as possible on the map, I would pre-render them into bitmap tiles and then use either the leaflet API or something like d3.geo.tile (example here) to overlay it on the actual map. This way you can easily scale to millions of points.
Although you can only render ~2-5k SVG elements before you start to see noticeable slowdown (depending on size, use of gradient fills, etc.), you can store and manipulate much larger datasets client-side. You can often handle tens or hundreds of thousands of data points efficiently in SVG: the trick is to be very selective about what you actually render, and to use techniques like debouncing to redraw only when necessary.
(For very large datasets: yes, you'll need to either aggregate/subsample points or pre-render.)
With this in mind, one technique I've used for d3 maps in particular is to use d3.geom.quadtree() to dynamically cull points as the user pans/zooms. More specifically, I avoid drawing points that are either
outside the current map bounding box (since these aren't visible at all), or
too close to other points (since these add visual clutter and are hard to interact with anyways).
In JS-ish pseudocode, this would look roughly like:
function getIndicesToDraw(data, r, bbox) {
var indicesToDraw = [];
var Q = d3.geom.quadtree();
// set bounds in pixel space
for (var i = 0; i < data.length; i++) {
var d = data[i];
var p = getPointForDatum(d);
if (isInsideBoundingBox(bbox, p) && !hasPointWithinRadius(Q, r, p)) {
Q.add(p);
indicesToDraw.push(i);
}
}
return indicesToDraw;
}
function redraw(svg, data, r, bbox) {
var indicesToDraw = getIndicesToDraw(data, r, bbox);
var points = svg.selectAll('.data-point')
.data(indicesToDraw, function(i) { return i; });
// draw new points for points.enter()
points.exit().remove();
// update positions of points (or SVG transforms, etc.)
}
This is a question of cartography as much as it is of technology. Just because you can put thousands of points on a map doesn't mean you should. You should ask yourself if your user needs to see as many points as possible at once, and will understand the data better as a result of it. Will seeing this many points confuse and overwhelm the user, or will it help them accomplish their desired task? Is there any way you could filter the data so that all points do not need to be drawn at once?
The most common solution to this problem is to use clusters, usually with something like Leaflet MarkerCluster. In the past while using D3 and Leaflet I have created a sort of heat map by creating an arbitrary grid, assigning each point to a bin, and applying a color ramp to the grid. However, getting back to your original concern, it is rather computationally intensive.
Depending on which platforms you want to support, be aware that phones and tablets do not always handle SVGs very well, especially while drawing thousands of features.
Another place for potential performance gain is in the delivery of the point geometry. I'm not sure how you are currently loading them, but using a spatially indexed PostGIS table and selecting by bounding box is often quite quick, but because you are drawing points and not polygons, you could even get away with loading them into the browser via CSV, which is substantially smaller than a GeoJSON or Topojson.
I would load all the data at once but only draw circles in the view port that are large enough. On zoom or pan, remove the circles that shouldn't be shown and check if previously hidden circles should be added.
You may use canvas + three.js or webgl where may join another map and 10k animation created 3d models or native code svg some elements, and intercative live time animation in one scene very good. I`m tested for fun this methode.Sry, my bad engl. Another puths - i think may used dlsl shader, opengl+ wasm and so on