I want to recreate the perspectiveTransform function from OpenCV in JavaScript. The reason I need to do this is because of an issue encountered here. I found the actual C++ implementation for this function here and getPerspectiveTransform, which seems to work completely differently here. They both seem to involve a mess of pointers I don't really understand, with no equivalent in JavaScript so I'm not sure how to write something similar.
I figured it's just some type of matrix math and surely there is a simple way to do it mathematically. If anyone knows anything about matrices / transformations like this I would hugely appreciate some advice / pseudo / code.
Example data I got from running a working version in C++
Input 1 (3x3) matrix
[0.4709243769447963, -0.1089570231317744, 289.4386175367417;
-0.05915378097999304, 0.3816064687798928, 503.5397702666958;
-9.054861751225341e-05, -0.0001637295648326533, 1]
Input 2 (2x4) matrix
[0, 0] [1000, 0] [1000, 740] [0, 740]
Desired output
[289.439, 503.54] [836.068, 488.631] [862.289, 921.962] [237.598,
894.279]
Details of the function
Where I am calling it in JavaScript
var H = new cv.Mat();
H = cv.findHomographyEasy(obj, scene, cv.FM_RANSAC);
var obj_corners = new cv.Point2fVector();
obj_corners[0] = new cv.Point(0,0);
obj_corners[1] = new cv.Point(img1Raw.cols,0);
obj_corners[2] = new cv.Point(img1Raw.cols, img1Raw.rows);
obj_corners[3] = new cv.Point(0, img1Raw.rows);
var scene_corners = new cv.Point2fVector();
scene_corners[0] = new cv.Point(0,0);
scene_corners[1] = new cv.Point(0,0);
scene_corners[2] = new cv.Point(0,0);
scene_corners[3] = new cv.Point(0,0);
// NEED TO REMAKE THIS FUNCTION WHICH DOESN'T WORK IN OPENCV.js
cv.perspectiveTransformEasy(obj_corners, scene_corners, H);
Related
I want to scale a contour in OpenCV.js. I have a valid contour in cnt variable of type cv.Mat (verified it by using drawContours).
I found a function in Python that does everything I need but I have problems converting it to Javascript.
Python version:
def scale_contour(cnt, scale):
M = cv2.moments(cnt)
cx = int(M['m10']/M['m00'])
cy = int(M['m01']/M['m00'])
cnt_norm = cnt - [cx, cy]
cnt_scaled = cnt_norm * scale
cnt_scaled = cnt_scaled + [cx, cy]
cnt_scaled = cnt_scaled.astype(np.int32)
return cnt_scaled
Here's what I started for Javascript:
function scaleContour(cnt, scale) {
console.log("cnt", cnt.data32S, cnt.rows, cnt.cols, cnt.type());
const M = cv.moments(cnt);
const cx = M['m10']/M['m00'];
const cy = M['m01']/M['m00'];
const offset = [Math.ceil(cx), Math.ceil(cy)];
console.log("Offset", offset);
// cannot use convenient Python arithmetics here,
// have to call functions
// although technically we have 1 row 2 cols for a point, but the cnt type is 2-channel CV_32SC2 (12)
// therefore keeping the size 1,1 and leave the second dimension as a channel to be compatible with the contour format
const pointMat = cv.matFromArray(1, 1, cnt.type(), offset);
console.log("pointMat", pointMat.data32S);
const cntNorm = new cv.Mat(cnt.rows, cnt.cols, cnt.type());
cv.subtract(cnt, pointMat, cntNorm); <-- my app crashes here with an exception that has only some random number - OpenCV seems to always do that when I'm doing something wrong or it's out of memory
console.log("ctnorm", cntNorm.data32S);
Unfortunately, I cannot find a good example on Python-like matrix operations in the official OpenCV.js documentation on basic data structures. It just shows how to create matrices but does not explain how to perform simple math operations with a matrix and a point-like value.
Also, I'm not sure when I need new cv.Mat(cnt.rows, cnt.cols, cnt.type()); and when new cv.Mat() is enough. The documentation has both but does not answer what is the rule of thumb to use an empty Mat and when it must be configured with row/col/type.
And the log output for cnt cols and rows is confusing, it prints 75 rows and 1 col, but the data is Int32Array(150). I found that sometimes the second layer of values are designated by type and not cols/rows. That's confusing. How should we know when to use rows=1,cols=2 and when rows=1,cols=2 and a type with 2 channels?
I have a specific problem, I would like to export an indexed geomtry that has a drawrange.
Using the GLTFExporter, after having faced the issue with typescript integration (known issue apparently), I had the bad luck to discover that this was not implemented in the exporter:
// #TODO Indexed buffer geometry with drawRange not supported yet
https://github.com/mrdoob/three.js/blob/master/examples/js/exporters/GLTFExporter.js line 564
Checking the commit history showed me that the last update was 3 months ago and I don't think this is gonna come any time soon.
I tried to remove the index buffer and rewrite my position bufferattribute array base on my draw range but I must do something wrong because it does not work, it simply breaks my geometry.
Would any of you have a work around for me or some explainations on how to proceed with my geometry?
Thank you in advance.
EDIT:
My current work-around is to "de-index" my geometry for the export and keep the drawRange, this case is handled by the exporter. It is not ideal, and it forces me to recreate a full new geometry with new BufferAttributes. But since this operation is only done for the export, I can even have this process happening in an asynchronous way. I wish there was a better way.
Regarding the two variables, this is something I'm going to address in the following PR to make it part of the WebGLUtils and just import it. It doesn't make sense that each one that needs these constants need to redefine them again every time.
As mentionned in my edit, I bypassed my problem by de-indexing of my geometry, it is not the best solution, but since I only need it for this export, here is how I proceeded:
// original attributes
const vertices = geometryTmp.getAttribute("position");
const normals = geometryTmp.getAttribute("normal");
const uv = geometryTmp.getAttribute("uv");
// new buffer arrays
let verticesTmp = new Float32Array(3 * geometryTmp.index.array.length);
let normalTmp = new Float32Array(3 * geometryTmp.index.array.length);
let uvTmp = new Float32Array(2 * geometryTmp.index.array.length);
let j = 0;
for(let i = 0; i < verticesTmp.length; i += 3) {
let index = geometryTmp.index.array[j];
verticesTmp[i] = vertices.getX(index);
verticesTmp[i+1] = vertices.getY(index);
verticesTmp[i+2] = vertices.getZ(index);
normalTmp[i] = normals.getX(index);
normalTmp[i+1] = normals.getY(index);
normalTmp[i+2] = normals.getZ(index);
j++;
}
j = 0;
for(let i = 0; i < uvTmp.length; i += 2) {
let index = geometryTmp.index.array[j];
uvTmp[i] = uv.getX(index);
uvTmp[i+1] = uv.getY(index);
j++;
}
let newGeomtry = new THREE.BufferGeometry();
newGeomtry.addAttribute( 'position', new THREE.BufferAttribute( verticesTmp, 3 ) );
newGeomtry.addAttribute( 'normal', new THREE.BufferAttribute( normalTmp, 3 ) );
newGeomtry.addAttribute( 'uv', new THREE.BufferAttribute( uvTmp, 2 ) );
newGeomtry.drawRange = geometryTmp.drawRange;
mesh.geometry = newGeomtry;
// After I do that to all the meshes I need, them to a new THREE.Scene that will be given to the exporter with truncateDrawRange = true
I hope it helps someone too.
I've been reading this article: http://blogs.adobe.com/cantrell/archives/2011/03/native-cursors-in-air-2-6.html on how to create a native cursor in AIR without having to hack it by moving a sprite in place of a hidden cursor to fake it.
However I'm using HTML/JavaScript instead of ActionScript.
So far I have:
function nativeCursor(){
// load in a bitmap
var loader = new air.Loader();
loader.load(new air.URLRequest('./assets/cursor.png'));
var bitmaps = new air.Vector["<String>"]();
var bmd = new air.BitmapData(32, 32, true, 0x00000000);
var p = new window.runtime.flash.geom.Point(0, 0);
var r = new window.runtime.flash.geom.Rectangle(32 , 0, 32, 32);
var image = new window.runtime.flash.display.Bitmap(loader.content);
bmd.copyPixels([image.bitmapData], r, p);
bitmaps.push(bmd);
var mcd = new window.runtime.flash.ui.MouseCursorData();
mcd.data = bitmaps;
mcd.hotSpot = new Point(0, 0);
mcd.frameRate = 24;
window.runtime.flash.ui.Mouse.registerCursor("defaultCursor", mcd);
window.runtime.flash.ui.Mouse.cursor = "defaultCursor";
}
But I get an error TypeError: Error #1034: Type Coercion failed: cannot convert []#2b9d1f1 to flash.display.BitmapData. for this line: bmd.copyPixels([image.bitmapData], r, p);
If I remove the brackets for that line so it's just: bmd.copyPixels(image.bitmapData, r, p); the error becomes TypeError: Error #2007: Parameter sourceBitmapData must be non-null.
So I'm assuming that the error is because the bitmap data is null... but why? The image is being loaded in fine so is the way I'm trying to get the bitmap data incorrect?
BitmapData, not String
The vector is supposed to be of type BitmapData, not String, that is:
air.Vector["<flash.display.BitmapData>"]
See HTML Developer’s Guide for Adobe AIR - Working with Vectors for more information.
Asynchronous loaders
Also the Loader class probably works asynchronously in JavaScript too, it's not documented properly in the HTML API reference and I've never used JS for AIR development, so I can only assume that, and that one can refer to the AS3 reference for the missing docs, however it makes sense judging from the available examples.
http://help.adobe.com/.../html/flash/display/BitmapData.html#includeExamplesSummary
Loader.bitmapData doesn't exist
There is no bitmapData property on the Loader class, only a content property that holds a DisplayObject, which might actually be a Bitmap object which in turn has a bitmapData property.
An example
Here's some untested example code that should get you started:
var mcd = new window.runtime.flash.ui.MouseCursorData();
mcd.hotSpot = new air.Point(0, 0);
mcd.frameRate = 24;
var loader = new air.Loader();
loader.contentLoaderInfo.addEventListener(air.Event.COMPLETE, function(event)
{
var image = air.Bitmap(loader.content);
var bitmaps = new air.Vector["<flash.display.BitmapData>"]();
bitmaps.push(image.bitmapData);
mcd.data = bitmaps;
air.Mouse.registerCursor('defaultCursor', mcd);
air.Mouse.cursor = 'defaultCursor';
});
var request = new air.URLRequest('./assets/cursor.png');
loader.load(request);
I have been trying to do simple matrix transformations but can't seem to get the hang of it. There's a lot of old code on the internet and I'm not sure whats current. Here's the code.
var matrixIntiial = new THREE.Matrix4();
myObj.matrix.copy( matrixIntiial );
matrixIntiial.makeTranslation(new THREE.Vector3(-100, 50, -100));
myObj.matrixAutoUpdate = false;
myObj.applyMatrix( matrixIntiial );
When I call myObj.applyMatrix( matrixIntiial ); the object disappears. Also I'm not very clear on how to correctly use the myObj.matrixAutoUpdate = false; and the .updateMatrix();. I know manipulating the matrix directly is for advanced users but I would like to know the basics.
You can make a translation like this:
var geometry= new THREE.CircleGeometry(0.05, 10, 0, Math.PI/2);
var translation = new THREE.Matrix4().makeTranslation(10, 0, 0);
geometry.applyMatrix(translation);
I'm trying to make use of the built-in shadow map plugin in three.js. After initial difficulties I have more or less acceptable image with one last glitch. That one being shadow appearing on top some (all?) surfaces, with normal 0,0,1. Below are pictures of the same model.
Three.js
Preview.app (Mac)
And the code used to setup shadows:
var shadowLight = new THREE.DirectionalLight(0xFFFFFF);
shadowLight.position.x = cx + dmax/2;
shadowLight.position.y = cy - dmax/2;
shadowLight.position.z = dmax*1.5;
shadowLight.lookAt(new THREE.Vector3(cx, cy, 0));
shadowLight.target.position.set(cx, cy, 0);
shadowLight.castShadow = true;
shadowLight.onlyShadow = true;
shadowLight.shadowCameraNear = dmax;
shadowLight.shadowCameraFar = dmax*2;
shadowLight.shadowCameraLeft = -dmax/2;
shadowLight.shadowCameraRight = dmax/2;
shadowLight.shadowCameraBottom = -dmax/2;
shadowLight.shadowCameraTop = dmax/2;
shadowLight.shadowBias = 0.005;
shadowLight.shadowDarkness = 0.3;
shadowLight.shadowMapWidth = 2048;
shadowLight.shadowMapHeight = 2048;
// shadowLight.shadowCameraVisible = true;
scene.add(shadowLight);
UPDATE: And a live example over here: http://jsbin.com/okobum/1/edit
Your code looks fine. You just need to play with the shadowLight.shadowBias parameter. This is always a bit tricky. (Note that the bias parameter can be negative.)
EDIT: Tighten up your shadow-camera near and far planes. This will help reduce both shadow acne and peter-panning. For example, your live link, set shadowLight.shadowCameraNear = 3*dmax;. This worked for me.
You can also try adding depth to your table tops, if it's not already there.
You can try setting renderer.shadowMapCullFrontFaces = false. This will cull back faces instead of front ones.