Develop Reference

Optimizing smooth tween between svg paths in JavaScript / React Native

I'm currently porting an application to React Native that captures user input as a stroke and animates it to the correct position to match an svg (pictures below). In the web, I use a combination of multiple smoothing libraries & pixijs to achieve perfectly smooth transitions with no artifacts.
With React Native & reanimated I'm limited to functions I can write by hand to handle the interpolation between two paths. Currently what I'm doing is:
Convert the target svg to a fixed number N of points
Smooth the captured input and convert it to a series of N points
Loop over each coordinate and interpolate the value between those two points (linear interpolation)
Run the resulting points array through a Catmull-Rom function
Render the resulting SVG curve
1 & 2 I can cache prior to the animation, but steps 3 4 & 5 need to happen on each render.
Unfortunately, using this method, I'm limited to a value of around 300 N as the maximum amount of points before dropping some frames. I'm also still seeing some artifacts at the end of an animation that I don't know how to fix.
This is sufficient, but given that in the web I can animate tens of thousands of points without dropping frames, I feel like I am missing a key performance optimization here. For example, is there a way to combine steps 3 & 4? Are there more performant algorithms than Catmull-Rom?
Is there a better way to achieve a smooth transition between two vector paths using just pure JavaScript (or dropping into Swift if that is possible)?
Is there something more I can do to remove the artifacts pictured in the last photo? I'm not sure what these are called technically so it's hard for me to research - the catmull-rom spline removed most of them but I still see a few at the tail ends of the animation.
Animation end/start state:
Animation middle state:
Animation start/end state (with artifact):

You might want to have a look at flubber.js
Also why not ditch the catmull-rom for simple linear sections (probably detailed enough with 1000+ points)
If neither helps, or you want to get as fast as possible, you might want to leverage the GPUs power for embarrassingly parallel workflows like the interpolation between to N-sized arrays.
edit:
also consider using the skia renderer which already leverages the gpu and supports stuff perfectly fitting your use-case
import {Canvas, Path, Skia, interpolatePath} from "#shopify/react-native-skia";
//obv. you need to modify this to use your arrays
const path1 = new Path();
path1.moveTo(0, 0);
path1.lineTo(100, 0);
const path2 = new Path();
path2.moveTo(0, 0);
path2.lineTo(0, 100);
//you have to do this dynamically (maybe using skia animations)
let animationProgress = 0.5;
//magic already implemented for you
let path = interpolatePath(animationProgress, [0, 1], [path1, path2]);
const PathDemo = () => {
return (
<Canvas style={{ flex: 1 }}>
<Path
path={path}
color="lightblue"
/>
</Canvas>
);
};

How to optimize performance using instancing in a scene full of skinned-mesh?

I’m working on a web tower defense game based on three.js. Now I’m stuck at optimizing the performance.
My game loads two GLTF models as enemy and tower, both have skinned-mesh. When the player creates a tower or the game spawns an enemy, I use THREE.AnimationUtils.clone to clone the loaded model. Then I add this cloned model to the scene. For animation, I use THREE.AnimationObjectGroup to animate all the enemies.
This results in an average of 370 draw-calls per frame in the performance test with the scene loaded with 45 towers and 70 enemies, which is a nightmare for the game.
I think maybe using instancing can optimize the performance because every tower and enemy share the same model and state in each frame, but only rotation and position are different. But after I studied some examples using instancing, there is no example using instancing with skinned-mesh. (There is a discussion here, but the result here doesn't mention any method with instancing.)
Is there any chance that this can be done with three.js, or some other solution for this situation?
Update
After researched more I found some concepts maybe can help me to implement instancing with skinned-mesh.
Concept
The original post here implement skinned-mesh with instancing in Unity. (It's written in Chinese, I translated the main concept in the following.)
After loaded a skinned-mesh, it has an initial state with all vertices (for clarity, each initial vertex denote as PLT in the following). In any frame of the animation, the final position of PLT (denote as PI) equals to a series of matrix multiplication PI = (M_rootlocal * ... * M_2_3 * M_1_2 * M_bind_1 * PLT) + (M_rootlocal * ... * M_2_3 * M_1_2 * M_bind_2 * PLT) + (...)
M_bind_1 is the bone-binding matrix of bone 1.
M_m_n means the transformation of bone m relative to it's initial state under the coordinate system of bone n.
For simplify, use M_f_i = M_rootlocal * ... * M_2_3 * M_1_2 * M_bind_i to represent the transformation. M_f_i means bone-binding matrix of bone i after multiplication at frame f, so PI = (M_f_1 * PLT) + (M_f_2 * PLT) + (...) Once we know M_f_i, we can calculate the position of every vertex in frame f.
The process above can be done inside GPU by passing M_f_i which wrap as a texture. (Under the premise that the skinned-mesh needs to animate around 10 animations and less amount of bones, the require memory is about 0.75Mb.). Finally, we can pass different frame number f to each instance to render skinned-mesh with animation in one draw-call.
Implement with three.js
I haven't build an example code yet because I don't know the concept can work on WebGL or not (also I'm not familiar with GLSL), but I think the way to implement it with three.js can done as the following.
Follow here to get M_f_i.
Use THREE.InstancedBufferGeometry and THREE.RawShaderMaterial.
In uniforms pass initial geometry, M_f_i and texture.
In vertexShader process PI = (M_f_1 * PLT) + (M_f_2 * PLT) + (...).
In fragmentShader process texture (I have no idea how to do it).
Pass f and other instance attribute using THREE.InstancedBufferAttribute.
Problem
Where is M_f and how to get it by THREE.AnimationClip in step 1?
How to index each PLT (vertex in geometry)?
How to deal with texture?
How to deal with hierarchy Object3D (Object3D.children have THREE.Mesh and THREE.SkinnedMesh at the same time)?
I need someone to tell me this idea works in three.js or not, and how to solve the problem above.

I remember there was a Geometry or Mesh Merge function that was really helping me in the past with such a cases. I recommend you search in that direction.
There are many counterparts to its usage such as loosing the individuality of each 3d object you use but when possible you should use it for static elements like environment objects, in other cases it may be also useful if your individual objects/towers are based on many single 3d objects in the way that they become just one...
From my experience (could vary a lot depending on each computer and size of 3d viewport) at the end you should never have more than 50 (simple) 3d objects in front of your visible camera area and reuse all materials, geometries, mesh... otherwise you'll end up having a very poor performance as soon as something fun is happening in your game.
Hope it helps!

Multiplayer Game - Client Interpolation Calculation?

I am creating a Multiplayer game using socket io in javascript. The game works perfectly at the moment aside from the client interpolation. Right now, when I get a packet from the server, I simply set the clients position to the position sent by the server. Here is what I have tried to do:
getServerInfo(packet) {
var otherPlayer = players[packet.id]; // GET PLAYER
otherPlayer.setTarget(packet.x, packet.y); // SET TARGET TO MOVE TO
...
}
So I set the players Target position. And then in the Players Update method I simply did this:
var update = function(delta) {
if (x != target.x || y != target.y){
var direction = Math.atan2((target.y - y), (target.x - x));
x += (delta* speed) * Math.cos(direction);
y += (delta* speed) * Math.sin(direction);
var dist = Math.sqrt((x - target.x) * (x - target.x) + (y - target.y)
* (y - target.y));
if (dist < treshhold){
x = target.x;
y = target.y;
}
}
}
This basically moves the player in the direction of the target at a fixed speed. The issue is that the player arrives at the target either before or after the next information arrives from the server.
Edit: I have just read Gabriel Bambettas Article on this subject, and he mentions this:
Say you receive position data at t = 1000. You already had received data at t = 900, so you know where the player was at t = 900 and t = 1000. So, from t = 1000 and t = 1100, you show what the other player did from t = 900 to t = 1000. This way you’re always showing the user actual movement data, except you’re showing it 100 ms “late”.
This again assumed that it is exactly 100ms late. If your ping varies a lot, this will not work.
Would you be able to provide some pseudo code so I can get an Idea of how to do this?
I have found this question online here. But none of the answers provide an example of how to do it, only suggestions.

I'm completely fresh to multiplayer game client/server architecture and algorithms, however in reading this question the first thing that came to mind was implementing second-order (or higher) Kalman filters on the relevant variables for each player.
Specifically, the Kalman prediction steps which are much better than simple dead-reckoning. Also the fact that Kalman prediction and update steps work somewhat as weighted or optimal interpolators. And futhermore, the dynamics of players could be encoded directly rather than playing around with abstracted parameterizations used in other methods.
Meanwhile, a quick search led me to this:
An improvement of dead reckoning algorithm using kalman filter for minimizing network traffic of 3d on-line games
The abstract:
Online 3D games require efficient and fast user interaction support
over network, and the networking support is usually implemented using
network game engine. The network game engine should minimize the
network delay and mitigate the network traffic congestion. To minimize
the network traffic between game users, a client-based prediction
(dead reckoning algorithm) is used. Each game entity uses the
algorithm to estimates its own movement (also other entities'
movement), and when the estimation error is over threshold, the entity
sends the UPDATE (including position, velocity, etc) packet to other
entities. As the estimation accuracy is increased, each entity can
minimize the transmission of the UPDATE packet. To improve the
prediction accuracy of dead reckoning algorithm, we propose the Kalman
filter based dead reckoning approach. To show real demonstration, we
use a popular network game (BZFlag), and improve the game optimized
dead reckoning algorithm using Kalman filter. We improve the
prediction accuracy and reduce the network traffic by 12 percents.
Might seem wordy and like a whole new problem to learn what it's all about... and discrete state-space for that matter.
Briefly, I'd say a Kalman filter is a filter that takes into account uncertainty, which is what you've got here. It normally works on measurement uncertainty at a known sample rate, but it could be re-tooled to work with uncertainty in measurement period/phase.
The idea being that in lieu of a proper measurement, you'd simply update with the kalman predictions. The tactic is similar to target tracking applications.
I was recommended them on stackexchange myself - took about a week to figure out how they were relevant but I've since implemented them successfully in vision processing work.
(...it's making me want to experiment with your problem now !)
As I wanted more direct control over the filter, I copied someone else's roll-your-own implementation of a Kalman filter in matlab into openCV (in C++):
void Marker::kalmanPredict(){
//Prediction for state vector
Xx = A * Xx;
Xy = A * Xy;
//and covariance
Px = A * Px * A.t() + Q;
Py = A * Py * A.t() + Q;
}
void Marker::kalmanUpdate(Point2d& measuredPosition){
//Kalman gain K:
Mat tempINVx = Mat(2, 2, CV_64F);
Mat tempINVy = Mat(2, 2, CV_64F);
tempINVx = C*Px*C.t() + R;
tempINVy = C*Py*C.t() + R;
Kx = Px*C.t() * tempINVx.inv(DECOMP_CHOLESKY);
Ky = Py*C.t() * tempINVy.inv(DECOMP_CHOLESKY);
//Estimate of velocity
//units are pixels.s^-1
Point2d measuredVelocity = Point2d(measuredPosition.x - Xx.at<double>(0), measuredPosition.y - Xy.at<double>(0));
Mat zx = (Mat_<double>(2,1) << measuredPosition.x, measuredVelocity.x);
Mat zy = (Mat_<double>(2,1) << measuredPosition.y, measuredVelocity.y);
//kalman correction based on position measurement and velocity estimate:
Xx = Xx + Kx*(zx - C*Xx);
Xy = Xy + Ky*(zy - C*Xy);
//and covariance again
Px = Px - Kx*C*Px;
Py = Py - Ky*C*Py;
}
I don't expect you to be able to use this directly though, but if anyone comes across it and understand what 'A', 'P', 'Q' and 'C' are in state-space (hint hint, state-space understanding is a pre-req here) they'll likely see how connect the dots.
(both matlab and openCV have their own Kalman filter implementations included by the way...)

This question is being left open with a request for more detail, so I’ll try to fill in the gaps of Patrick Klug’s answer. He suggested, reasonably, that you transmit both the current position and the current velocity at each time point.
Since two position and two velocity measurements give a system of four equations, it enables us to solve for a system of four unknowns, namely a cubic spline (which has four coefficients, a, b, c and d). In order for this spline to be smooth, the first and second derivatives (velocity and acceleration) should be equal at the endpoints. There are two standard, equivalent ways of calculating this: Hermite splines (https://en.wikipedia.org/wiki/Cubic_Hermite_spline) and Bézier splines (http://mathfaculty.fullerton.edu/mathews/n2003/BezierCurveMod.html). For a two-dimensional problem such as this, I suggested separating variables and finding splines for both x and y based on the tangent data in the updates, which is called a clamped piecewise cubic Hermite spline. This has several advantages over the splines in the link above, such as cardinal splines, which do not take advantage of that information. The locations and velocities at the control points will match, you can interpolate up to the last update rather than the one before, and you can apply this method just as easily to polar coordinates if the game world is inherently polar like Space wars. (Another approach sometimes used for periodic data is to perform a FFT and do trigonometric interpolation in the frequency domain, but that doesn’t sound applicable here.)
What originally appeared here was a derivation of the Hermite spline using linear algebra in a somewhat unusual way that (unless I made a mistake entering it) would have worked. However, the comments convinced me it would be more helpful to give the standard names for what I was talking about. If you are interested in the mathematical details of how and why this works, this is a better explanation: https://math.stackexchange.com/questions/62360/natural-cubic-splines-vs-piecewise-hermite-splines
A better algorithm than the one I gave is to represent the sample points and first derivatives as a tridiagonal matrix that, multiplied by a column vector of coefficients, produces the boundary conditions, and solve for the coefficients. An alternative is to add control points to a Bézier curve where the tangent lines at the sampled points intersect and on the tangent lines at the endpoints. Both methods produce the same, unique, smooth cubic spline.
One situation you might be able to avoid if you were choosing the points rather than receiving updates is if you get a bad sample of points. You can’t, for example, intersect parallel tangent lines, or tell what happened if it’s back in the same place with a nonzero first derivative. You’d never choose those points for a piecewise spline, but you might get them if an object made a swerve between updates.
If my computer weren’t broken right now, here is where I would put fancy graphics like the ones I posted to TeX.SX. Unfortunately, I have to bow out of those for now.
Is this better than straight linear interpolation? Definitely: linear interpolation will get you straight- line paths, quadratic splines won't be smooth, and higher-order polynomials will likely be overfitted. Cubic splines are the standard way to solve that problem.
Are they better for extrapolation, where you try to predict where a game object will go? Possibly not: this way, you’re assuming that a player who’s accelerating will keep accelerating, rather than that they will immediately stop accelerating, and that could put you much further off. However, the time between updates should be short, so you shouldn’t get too far off.
Finally, you might make things a lot easier on yourself by programming in a bit more conservation of momentum. If there’s a limit to how quickly objects can turn, accelerate or decelerate, their paths will not be able to diverge as much from where you predict based on their last positions and velocities.

Depending on your game you might want to prefer smooth player movement over super-precise location. If so, then I'd suggest to aim for 'eventual consistency'. I think your idea of keeping 'real' and 'simulated' data-points is a good one. Just make sure that from time to time you force the simulated to converge with the real, otherwise the gap will get too big.
Regarding your concern about different movement speed I'd suggest you include the current velocity and direction of the player in addition to the current position in your packet. This will enable you to more smoothly predict where the player would be based on your own framerate/update timing.
Essentially you would calculate the current simulated velocity and direction taking into account the last simulated location and velocity as well as last known location and velocity (put more emphasis on the second) and then simulate new position based on that.
If the gap between simulated and known gets too big, just put more emphasis on the known location and the otherPlayer will catch up quicker.

JS Canvas get pixel value very frequently

I am creating a video game based on Node.js/WebGL/Canvas/PIXI.js.
In this game, blocks have a generic size: they can be circles, polygons, or everything. So, my physical engine needs to know where exactly the things are, what pixels are walls and what pixels are not. Since I think PIXI don't allow this, I create an invisible canvas where I put all the wall's images of the map. Then, I use the function getImageData to create a function "isWall" at (x, y):
function isWall(x, y):
return canvas.getImageData(x, y, 1, 1).data[3] != 0;
However, this is very slow (it takes up to 70% of the CPU time of the game, according to Chrome profiling). Also, since I introduced this function, I sometimes got the error "Oops, WebGL crashed" without any additional advice.
Is there a better method to access the value of the pixel? I thought about storing everything in a static bit array (walls have a fixed size), with 1 corresponding to a wall and 0 to a non-wall. Is it reasonable to have a 10-million-cells array in memory?

Some thoughts:
For first check: Use collision regions for all of your objects. The regions can even be defined for each side depending on shape (ie. complex shapes). Only check for collisions inside intersecting regions.
Use half resolution for hit-test bitmaps (or even 25% if your scenario allow). Our brains are not capable of detecting pixel-accurate collisions when things are moving so this can be taken advantage of.
For complex shapes, pre-store the whole bitmap for it (based on its region(s)) but transform it to a single value typed array like Uint8Array with high and low values (re-use this instead of getting one and one pixels via the context). Subtract object's position and use the result as a delta for your shape region, then hit-testing the "bitmap". If the shape rotates, transform incoming check points accordingly (there is probably a sweet-spot here where updating bitmap becomes faster than transforming a bunch of points etc. You need to test for your scenario).
For close-to-square shaped objects do a compromise and use a simple rectangle check
For circles and ellipses use un-squared values to check distances for radius.
In some cases you can perhaps use collision predictions which you calculate before the games starts and when knowing all objects positions, directions and velocities (calculate the complete motion path, find intersections for those paths, calculate time/distance to those intersections). If your objects change direction etc. due to other events during their path, this will of course not work so well (or try and see if re-calculating is beneficial or not).
I'm sure why you would need 10m stored in memory, it's doable though - but you will need to use something like a quad-tree and split the array up, so it becomes efficient to look up a pixel state. IMO you will only need to store "bits" for the complex shapes, and you can limit it further by defining multiple regions per shape. For simpler shapes just use vectors (rectangles, radius/distance). Do performance tests often to find the right balance.
In any case - these sort of things has to be hand-optimized for the very scenario, so this is just a general take on it. Other factors will affect the approach such as high velocities, rotation, reflection etc. and it will quickly become very broad. Hope this gives some input though.

I use bit arrays to store 0 || 1 info and it works very well.
The information is stored compactly and gets/sets are very fast.
Here is the bit library I use:
https://github.com/drslump/Bits-js/blob/master/lib/Bits.js
I've not tried with 10m bits so you'll have to try it on your own dataset.
The solution you propose is very "flat", meaning each pixel must have a corresponding bit. This results in a large amount of memory being required--even if information is stored as bits.
An alternative testing data ranges instead of testing each pixel:
If the number of wall pixels is small versus the total number of pixels you might try storing each wall as a series of "runs". For example, a wall run might be stored in an object like this (warning: untested code!):
// an object containing all horizontal wall runs
var xRuns={}
// an object containing all vertical wall runs
var yRuns={}
// define a wall that runs on y=50 from x=100 to x=185
// and then runs on x=185 from y=50 to y=225
var y=50;
var x=185;
if(!xRuns[y]){ xRuns[y]=[]; }
xRuns[y].push({start:100,end:185});
if(!yRuns[x]){ yRuns[x]=[]; }
yRuns[x].push({start:50,end:225});
Then you can quickly test an [x,y] against the wall runs like this (warning untested code!):
function isWall(x,y){
if(xRuns[y]){
var a=xRuns[y];
var i=a.length;
do while(i--){
var run=a[i];
if(x>=run.start && x<=run.end){return(true);}
}
}
if(yRuns[x]){
var a=yRuns[x];
var i=a.length;
do while(i--){
var run=a[i];
if(y>=run.start && y<=run.end){return(true);}
}
}
return(false);
}
This should require very few tests because the x & y exactly specify which array of xRuns and yRuns need to be tested.
It may (or may not) be faster than testing the "flat" model because there is overhead getting to the specified element of the flat model. You'd have to perf test using both methods.
The wall-run method would likely require much less memory.
Hope this helps...Keep in mind the wall-run alternative is just off the top of my head and probably requires tweaking ;-)

Click detection in a 2D isometric grid?

I've been doing web development for years now and I'm slowly getting myself involved with game development and for my current project I've got this isometric map, where I need to use an algorithm to detect which field is being clicked on. This is all in the browser with Javascript by the way.
The map
It looks like this and I've added some numbers to show you the structure of the fields (tiles) and their IDs. All the fields have a center point (array of x,y) which the four corners are based on when drawn.
As you can see it's not a diamond shape, but a zig-zag map and there's no angle (top-down view) which is why I can't find an answer myself considering that all articles and calculations are usually based on a diamond shape with an angle.
The numbers
It's a dynamic map and all sizes and numbers can be changed to generate a new map.
I know it isn't a lot of data, but the map is generated based on the map and field sizes.
- Map Size: x:800 y:400
- Field Size: 80x80 (between corners)
- Center position of all the fields (x,y)
The goal
To come up with an algorithm which tells the client (game) which field the mouse is located in at any given event (click, movement etc).
Disclaimer
I do want to mention that I've already come up with a working solution myself, however I'm 100% certain it could be written in a better way (my solution involves a lot of nested if-statements and loops), and that's why I'm asking here.
Here's an example of my solution where I basically find a square with corners in the nearest 4 known positions and then I get my result based on the smallest square between the 2 nearest fields. Does that make any sense?
Ask if I missed something.

Here's what I came up with,
function posInGrid(x, y, length) {
xFromColCenter = x % length - length / 2;
yFromRowCenter = y % length - length / 2;
col = (x - xFromColCenter) / length;
row = (y - yFromRowCenter) / length;
if (yFromRowCenter < xFromColCenter) {
if (yFromRowCenter < (-xFromColCenter))--row;
else++col;
} else if (yFromRowCenter > xFromColCenter) {
if (yFromRowCenter < (-xFromColCenter))--col;
else++row;
}
return "Col:"+col+", Row:"+row+", xFC:"+xFromColCenter+", yFC:"+yFromRowCenter;
}
X and Y are the coords in the image, and length is the spacing of the grid.
Right now it returns a string, just for testing.. result should be row and col, and those are the coordinates I chose: your tile 1 has coords (1,0) tile 2 is(3,0), tile 10 is (0,1), tile 11 is (2,1). You could convert my coordinates to your numbered tiles in a line or two.
And a JSFiddle for testing http://jsfiddle.net/NHV3y/
Cheers.
EDIT: changed the return statement, had some variables I used for debugging left in.

A pixel perfect way of hit detection I've used in the past (in OpenGL, but the concept stands here too) is an off screen rendering of the scene where the different objects are identified with different colors.
This approach requires double the memory and double the rendering but the hit detection of arbitrarily complex scenes is done with a simple color lookup.
Since you want to detect a cell in a grid there are probably more efficient solutions but I wanted to mention this one for it's simplicity and flexibility.

This has been solved before, let me consult my notes...
Here's a couple of good resources:
From Laserbrain Studios, The basics of isometric programming
Useful article in the thread posted here, in Java
Let me know if this helps, and good luck with your game!
This code calculates the position in the grid given the uneven spacing. Should be pretty fast; almost all operations are done mathematically, using just one loop. I'll ponder the other part of the problem later.
def cspot(x,y,length):
l=length
lp=length+1
vlist = [ (l*(k%2))+(lp*((k+1)%2)) for k in range(1,y+1) ]
vlist.append(1)
return x + sum(vlist)