Develop Reference

Converting an equirectangular depth map into 3d point cloud

I have a 2D equirectangular depth map that is a 1024 x 512 array of floats, each ranging between 0 to 1. Here example (truncated to grayscale):
I want to convert it to a set of 3D points but I am having trouble finding the right formula to do so - it's sort of close - pseudocode here (using a vec3() library):
for(var y = 0; y < array_height; ++y) {
var lat = (y / array_height) * 180.0 - 90.0;
var rho = Math.cos(lat * Math.PI / 180.0);
for(var x = 0; x < array_width; ++x) {
var lng = (x / array_width) * 360.0 - 180.0;
var pos = new vec3();
pos.x = (r * Math.cos(lng * Math.PI / 180.0));
pos.y = (Math.sin(lat * Math.PI / 180.0));
pos.z = (r * Math.sin(lng * Math.PI / 180.0));
pos.norm();
var depth = parseFloat(depth[(y * array_width) + x] / 255);
pos.multiply(depth);
// at this point I can plot pos as an X, Y, Z point
}
}
What I end up with isn't quite right and I can't tell why not. I am certain the data is correct. Can anyone suggest what I am doing wrong.
Thank you.
Molly.

Well looks like the texture is half-sphere in spherical coordinates:
x axis is longitude angle a <0,180> [deg]
y axis is latitude angle b <-45,+45> [deg]
intensity is radius r <0,1> [-]
So for each pixel simply:
linearly convert x,y to a,b
in degrees:
a = x*180 / (width -1)
b = -45 + ( y* 90 / (height-1) )
or in radians:
a = x*M_PI / (width -1)
b = -0.25*M_PI + ( 0.5*y*M_PI / (height-1) )
apply spherical to cartesian conversion
x=r*cos(a)*cos(b);
y=r*sin(a)*cos(b);
z=r* sin(b);
Looks like you have wrongly coded this conversion as latitude angle should be in all x,y,z not just y !!! Also you should not normalize the resulting position that would corrupt the shape !!!
store point into point cloud.
When I put all together in VCL/C++ (sorry do not code in javascript):
List<double> pnt; // 3D point list x0,y0,z0,x1,y1,z1,...
void compute()
{
int x,y,xs,ys; // texture positiona and size
double a,b,r,da,db; // spherical positiona and angle steps
double xx,yy,zz; // 3D point
DWORD *p; // texture pixel access
// load and prepare BMP texture
Graphics::TBitmap *bmp=new Graphics::TBitmap;
bmp->LoadFromFile("map.bmp");
bmp->HandleType=bmDIB;
bmp->PixelFormat=pf32bit;
xs=bmp->Width;
ys=bmp->Height;
/*
// 360x180 deg
da=2.0*M_PI/double(xs-1);
db=1.0*M_PI/double(ys-1);
b=-0.5*M_PI;
*/
// 180x90 deg
da=1.0*M_PI/double(xs-1);
db=0.5*M_PI/double(ys-1);
b=-0.25*M_PI;
// proces all its pixels
pnt.num=0;
for ( y=0; y<ys; y++,b+=db)
for (p=(DWORD*)bmp->ScanLine[y],a=0.0,x=0; x<xs; x++,a+=da)
{
// pixel access
r=DWORD(p[x]&255); // obtain intensity from texture <0..255>
r/=255.0; // normalize to <0..1>
// convert to 3D
xx=r*cos(a)*cos(b);
yy=r*sin(a)*cos(b);
zz=r* sin(b);
// store to pointcloud
pnt.add(xx);
pnt.add(yy);
pnt.add(zz);
}
// clean up
delete bmp;
}
Here preview for 180x90 deg:
and preview for 360x180 deg:
Not sure which one is correct (as I do not have any context to your map) but the first option looks more correct to me ...
In case its the second just use different numbers (doubled) for the interpolation in bullet #1
Also if you want to remove the background just ignore r==1 pixels:
simply by testing the intensity to max value (before normalization) in my case by adding this line:
if (r==255) continue;
after this one
r=DWORD(p[x]&255);
In your case (you have <0..1> already) you should test r>=0.9999 or something like that instead.

Calculating vertex position after rotating in 3D space

I am trying to rotate on Y axis in javascript ( In my own 3D engine )
and i tried this function :
function rotateY(amount) {
for (var i = 0; i < points.length; i++) {
points[i].z = Math.sin(amount) * points[i].x + Math.cos(amount) * points[i].z;
points[i].x = Math.cos(amount) * points[i].x - Math.sin(amount) * points[i].z;
}
}
It is rotating, but every time it rotates it changes it's x and z scale so it is getting thinner.. Can you help me how to rotate it properly ? Thanks :)

Assuming i) the rotation is respective to the global origin and not the object itself and ii) that you want to apply a delta (if we can't take these, see below):
For each point:
1. Find the distance of the point relative to the axis.
2. Find the current angle of the point relative to the axis.
3. Use basic 2-D cos/sin polar projection, since the excluded axis is a unit vector.
function rotateY( points, deltaAngle ) {
const _points = points;
if ( ! Array.isArray( points ) ) points = [ points ];
for ( let i = 0; i < points.length; i ++ ) {
const newAngle = Math.atan2( points[ i ].z, points[ i ].x ) + deltaAngle;
const distance = ( points[ i ].x ** 2 + points[ i ].z ** 2 ) ** ( 1 / 2 );
points[ i ].x = distance * Math.cos( newAngle );
points[ i ].z = distance * Math.sin( newAngle );
}
return _points;
}
The algorithm is the same for X and Z rotation, so long as the first axis used in Math.atan2 is the same axis that uses Math.sin.
NOTE: I used the exponentiation operator. I wouldn't use this in production unless you're using Babel/something similar or don't care about IE/old users.
If assumption ii) cannot be taken, we simply want to store the original angles of the points and have newAngle defined as the original angle plus the new angle.
If assumption i) cannot be taken, it gets complicated. If the object's axes are simply offset, you can subtract that offset in newAngle and distance and add it back when setting x and z. If the axes themselves are not respectively parallel to the global axes, you'll want to switch to using a quaternion to avoid gimbal lock. I would suggest copying or at least looking three.js's implementation.

Isometric tilemap using canvas (with click detection)

I am currently developing a game, which requires a map consisting of various tile images. I managed to make them display correctly (see second image) but I am now unsure of how to calculate the clicked tile from the mouse position.
Are there any existing libraries for this purpose?
Please also note, that the tile images aren't drawn perfectly "corner-facing-camera", they are slightly rotated clockwise.

Isometric Transformations
Define a projection
Isometric display is the same as standard display, the only thing that has changed is the direction of the x and y axis. Normally the x axis is defined as (1,0) one unit across and zero down and the y axis is (0,1) zero units across and one down. For isometric (strictly speaking your image is a dimetric projection) you will have something like x axis (0.5,1) and y axis (-1,0.5)
The Matrix
From this you can create a rendering matrix with 6 values Two each for both axes and two for the origin, which I will ignore for now (the origin) and just use the 4 for the axis and assume that the origin is always at 0,0
var dimetricMatrix = [0.5,1.0,-1,0.5]; // x and y axis
Matrix transformation
From that you can get a point on the display that matches a given isometric coordinate. Lets say the blocks are 200 by 200 pixels and that you address each block by the block x and y. Thus the block in the bottom of your image is at x = 2 and y = 1 (the first top block is x = 0, y = 0)
Using the matrix we can get the pixel location of the block
var blockW = 200;
var blockH = 200;
var locX = 2;
var locY = 1;
function getLoc(x,y){
var xx,yy; // intermediate results
var m = dimetricMatrix; // short cut to make code readable
x *= blockW; // scale up
y *= blockH;
// now move along the projection x axis
xx = x * m[0];
yy = x * m[1];
// then add the distance along the y axis
xx += y * m[2];
yy += y * m[3];
return {x : xx, y : yy};
}
Befoer I move on you can see that I have scaled the x and y by the block size. We can simplify the above code and include the scale 200,200 in the matrix
var xAxis = [0.5, 1.0];
var yAxis = [-1, 0.5];
var blockW = 200;
var blockH = 200;
// now create the matrix and scale the x and y axis
var dimetricMatrix = [
xAxis[0] * blockW,
xAxis[1] * blockW,
yAxis[0] * blockH,
yAxis[1] * blockH,
]; // x and y axis
The matrix holds the scale in the x and y axis so that the two numbers for x axis tell us the direction and length of a transformed unit.
Simplify function
And redo the getLoc function for speed and efficiency
function transformPoint(point,matrix,result){
if(result === undefined){
result = {};
}
// now move along the projection x axis
result.x = point.x * matrix[0] + point.y * matrix[2];
result.y = point.x * matrix[1] + point.y * matrix[3];
return result;
}
So pass a point and get a transformed point back. The result argument allows you to pass an existing point and that saves having to allocate a new point if you are doing it often.
var point = {x : 2, y : 1};
var screen = transformPoint(point,dimetricMatrix);
// result is the screen location of the block
// next time
screen = transformPoint(point,dimetricMatrix,screen); // pass the screen obj
// to avoid those too
// GC hits that kill
// game frame rates
Inverting the Matrix
All that is handy but you need the reverse of what we just did. Luckily the way matrices work allows us to reverse the process by inverting the matrix.
function invertMatrix(matrix){
var m = matrix; // shortcut to make code readable
var rm = [0,0,0,0]; // resulting matrix
// get the cross product of the x and y axis. It is the area of the rectangle made by the
// two axis
var cross = m[0] * m[3] - m[1] * m[2]; // I call it the cross but most will call
// it the determinate (I think that cross
// product is more suited to geometry while
// determinate is for maths geeks)
rm[0] = m[3] / cross; // invert both axis and unscale (if cross is 1 then nothing)
rm[1] = -m[1] / cross;
rm[2] = -m[2] / cross;
rm[3] = m[0] / cross;
return rm;
}
Now we can invert our matrix
var dimetricMatrixInv = invertMatrix(dimetricMatrix); // get the invers
And now that we have the inverse matrix we can use the transform function to convert from a screen location to a block location
var screen = {x : 100, y : 200};
var blockLoc = transformPoint(screen, dimetricMatrixInv );
// result is the location of the block
The Matrix for rendering
For a bit of magic the transformation matrix dimetricMatrix can also be used by the 2D canvas, but you need to add the origin.
var m = dimetricMatrix;
ctx.setTransform(m[0], m[1], m[2], m[3], 0, 0); // assume origin at 0,0
Now you can draw a box around the block with
ctx.strokeRect(2,1,1,1); // 3rd by 2nd block 1 by 1 block wide.
The origin
I have left out the origin in all the above, I will leave that up to you to find as there is a trillion pages online about matrices as all 2D and 3D rendering use them and getting a good deep knowledge of them is important if you wish to get into computer visualization.

I need to warp a long image into a 2d circle (webGL). Distortion is expected

I need to take a long (max resolution) image and wrap it into a circle. So imagine bending a steel bar so that it is now circular with each end touching.
I have been banging my head against threejs for the last 8 hours and have so far managed to apply the image as a texture on a circle geometry, but can't figure out how to apply the texture to a long mesh and then warp that mesh appropriately. The warping doesn't need to be (and shouldn't be) animated. What we basically have is a 360 panoramic image that we need to "flatten" into a top-down view.
In lieu of sharing my code (as it's not significantly different), I've so far been playing around with this tutorial:
http://www.johannes-raida.de/tutorials/three.js/tutorial06/tutorial06.htm
And I do (I think) understand the broad strokes at this point.
Other things I've tried is to use just canvas to slice the image up into strips and warp each strip... this was horribly slow and I couldn't get that to work properly either!
Any help/suggestions?

Here's also a shader version: Shadertoy - Circle Distortion
This is the actual code:
#define dPI 6.28318530718 // 2*PI
#define sR 0.3 // small radius
#define bR 1.0 // big radius
void main(void)
{
// calc coordinates on the canvas
vec2 uv = gl_FragCoord.xy / iResolution.xy*2.-vec2(1.);
uv.x *= iResolution.x/iResolution.y;
// calc if it's in the ring area
float k = 0.0;
float d = length(uv);
if(d>sR && d<bR)
k = 1.0;
// calc the texture UV
// y coord is easy, but x is tricky, and certain calcs produce artifacts
vec2 tUV = vec2(0.0,0.0);
// 1st version (with artifact)
//tUV.x = atan(uv.y,uv.x)/dPI;
// 2nd version (more readable version of the 3rd version)
//float disp = 0.0;
//if(uv.x<0.0) disp = 0.5;
//tUV.x = atan(uv.y/uv.x)/dPI+disp;
// 3rd version (no branching, ugly)
tUV.x = atan(uv.y/uv.x)/dPI+0.5*(1.-clamp(uv.x,0.0,1.0)/uv.x);
tUV.y = (d-sR)/(bR-sR);
// output pixel
vec3 col = texture2D(iChannel0, tUV).rgb;
gl_FragColor = vec4(col*k,1.);
}
So you could draw rectangle on the canvas and add this shader code.
I hope this helps.

So here's a function using canvas's context2d that does the job.
The idea is to go around all the circle by a small angular step and to draw a thin slice of 'texture' along the circle radius.
To make it faster, only way i see is to compute by hand the transform to do one single setTransform instead of all this stuff.
The step count is optimal with step = atan(1, radius)
(if you do the scheme it's obvious : to go one y up when you're radius far from the center then tan = 1/radius => step angle = atan(1, radius).)
fiddle is here :
http://jsfiddle.net/gamealchemist/hto1s6fy/
A small example with a cloudy landscape :
// draw the part of img defined by the rect (startX, startY, endX, endY) inside
// the circle of center (cx,cy) between radius (innerRadius -> outerRadius)
// - no check performed -
function drawRectInCircle(img, cx, cy, innerRadius, outerRadius, startX, startY, endX, endY) {
var angle = 0;
var step = 1 * Math.atan2(1, outerRadius);
var limit = 2 * Math.PI;
ctx.save();
ctx.translate(cx, cy);
while (angle < limit) {
ctx.save();
ctx.rotate(angle);
ctx.translate(innerRadius, 0);
ctx.rotate(-Math.PI / 2);
var ratio = angle / limit;
var x = startX + ratio * (endX - startX);
ctx.drawImage(img, x, startY, 1, (endY - startY), 0, 0, 1, (outerRadius - innerRadius));
ctx.restore();
angle += step;
}
ctx.restore();
}

Points on a (un)rotated rectangle

I found this excellent question and answer which starts with x/y (plus the center x/y and degrees/radians) and calculates the rotated-to x'/y'. This calculation works perfectly, but I would like to run it in the opposite direction; starting with x'/y' and degrees/radians, I would like to calculate the originating x/y and the center x/y.
(x', y') = new position
(xc, yc) = center point things rotate around
(x, y) = initial point
theta = counterclockwise rotation in radians (radians = degrees * Pi / 180)
dx = x - xc
dy = y - yc
x' = xc + dx cos(theta) - dy sin(theta)
y' = yc + dx sin(theta) + dy cos(theta)
Or, in JavaScript/jQuery:
XYRotatesTo = function($element, iDegrees, iX, iY, iCenterXPercent, iCenterYPercent) {
var oPos = $element.position(),
iCenterX = ($element.outerWidth() * iCenterXPercent / 100),
iCenterY = ($element.outerHeight() * iCenterYPercent / 100),
iRadians = (iDegrees * Math.PI / 180),
iDX = (oPos.left - iCenterX),
iDY = (oPos.top - iCenterY)
;
return {
x: iCenterX + (iDX * Math.cos(iRadians)) - (iDY * Math.sin(iRadians)),
y: iCenterY + (iDX * Math.sin(iRadians)) + (iDY * Math.cos(iRadians))
};
};
The math/code above solves for the situation in Figure A; it calculates the position of the destination x'/y' (green circle) based on the known values for x/y (red circle), the center x/y (blue star) and the degrees/radians.
But I need math/code to solve for Figure B; where I can find not only the destination x/y (green circle), but also the destination center x/y (green star) from the known values of the starting x/y (grey circle, though probably not needed), the destination x'/y' (red circle) and the degrees/radians.
The code above will solve for the destination x/y (green circle) via iDegrees * -1 (thanks to #andrew cooke's answer which has since been removed by him), but in order to do that I need to feed into it the location of the destination center x/y (green star), and that is the calculations I'm currently missing, as you can see in Diagram C, below:
So... how do I find the coordinates ?/? (green star) given n, A (angle) and x'/y' (red circle)?

You're trying to find an inverse transformation. You start with the composition of two linear transformations, a translation T and a rotation R. You apply R first to a vector x and T second, so the expression is y = TRx. To solve the inverse problem you need the inverse of TR, written (TR)-1, which is equal to R-1T-1. The inverse of the rotation R is just the rotation by the negative of the angle (which you mention). The inverse of the translation is, similarly, the original translation multiplied by -1. So your answer is x = R-1T-1y.
In your present situation, you're given the rotation by means of its angle, but you'll need to compute the translation. You'll need the grey circle, which you didn't think you would need. Apply the rotation R (not its inverse) to the gray circle. Subtract this point from the red circle. This is the original translation T. Reverse the sign to get T-1.