According to OpenCV.js docs to modify a pixel value you can use 3 methods:
Direct data manipulation
at() family of methods
ptr() family of methods
Althougth the header of the section in the docs say "Accessing and Modifying pixel values" it provides examples for retrieving a value but not for modifying them. The problem is that while in C++ this code using the at or ptr methods works:
mat.at<type>(row, col) = value;
The equivalent in javascript is not valid and gives an Invalid left hand side in assignment expression:
mat.floatAt(row, col)) = value;
I could make it work using the direct data manipulation method with:
mat.data[row * this.cols * this.channels() + col * this.channels()] = value;
But this method does not operate in pixel values but in the underlaying array data structure where a pixel may span more than one array index, so is not valid for my use case.
How can a pixel value at [row, col] position in a CvMat be modified using OpenCV.js?
I have been successful in setting single pixels of images (or matrices) with ucharPtr.
Instead of setting it like this:
img.ucharPtr(i, j) = 255
I set it like this:
img.ucharPtr(i, j)[0] = 255
Even if the image is black and white and only has one channel
If you want to set all 4 pixel values, you can do this:
src.ucharPtr(i, j)[0] = 255
src.ucharPtr(i, j)[1] = 255
src.ucharPtr(i, j)[2] = 255
src.ucharPtr(i, j)[3] = 0
Related
I have an array of N HTMLCanvasElements that come from N frames of a video, and I want to compute the "median canvas" in the sense that every component (r, g, b, opacity) of every pixel is the median of the corresponding component in all the canvases.
The video frames are 1280x720, so that the pixels data for every canvas (obtained with canvas.getContext('2d').getImageData(0, 0, canvas.width, canvas.height).data) is a Uint8ClampedArray of length 3.686.400.
The naive way to compute the median is to:
prepare a result Uint8ClampedArray of length 3.686.400
prepare a temporary Uint8ClampedArray of length N
loop from 0 to 3.686.399
a) loop over the N canvases to fill the array
b) compute the median of the array
c) store the median to the result array
But it's very slow, even for 4 canvases.
Is there an efficient way (or existing code) to do that? My question is very similar to Find median of list of images, but I need to to this in JavaScript, not Python.
Note: for b), I use d3.median() which doesn't work on typed arrays, as far as I understand, so that it implies converting to numbers, then converting back to Uint8Clamped.
Note 2: I don't know much of GLSL shaders, but maybe using the GPU would be a way to get faster results. It would require to pass data from the CPU to the GPU though, which takes time if done repeatedly.
Note 3: the naive solution is there: https://observablehq.com/#severo/compute-the-approximate-median-image-of-a-video
You wrote
I use d3.median() which doesn't work on typed arrays…
Although that is not exactly true it leads into the right direction. Internally d3.median() uses the d3.quantile() method which starts off like this:
export default function quantile(values, p, valueof) {
values = Float64Array.from(numbers(values, valueof));
As you can see, this in fact does make use of typed arrays, it is just not your Uint8ClampedArray but a Float64Array instead. Because floating-point arithmetic is much more computation-intensive than its integer counterpart (including the conversion itself) this has a dramatic effect on the performance of your code. Doing this some 3 million times in a tight loop kills the efficiency of your solution.
Since you are retrieving all your pixel values from a Uint8ClampedArray you can be sure that you are always dealing with integers, though. That said, it is fairly easy to build a custom function median(values) derived from d3.median() and d3.quantile():
function median(values) {
// No conversion to floating point values needed.
if (!(n = values.length)) return;
if (n < 2) return d3.min(values);
var n,
i = (n - 1) * 0.5,
i0 = Math.floor(i),
value0 = d3.max(d3.quickselect(values, i0).subarray(0, i0 + 1)),
value1 = d3.min(values.subarray(i0 + 1));
return value0 + (value1 - value0) * (i - i0);
}
On top of getting rid of the problematic conversion on the first line this implementation additionally applies some more micro-optimizations because in your case you are always looking for the 2-quantile (i.e. the median). That might not seem much at first, but doing this multiple million times in a loop it does make a difference.
With minimal changes to your own code you can call it like this:
// medianImageData.data[i] = d3.median(arr); Instead of this use line below.
medianImageData.data[i] = median(arr);
Have a look at my working fork of your Observable notebook.
I have correctly applied the d3 (v 4.0) histogram function to bin an array of data. My code looks like this:
var bins = d3.histogram()
.domain([data_points_min, data_points_max])
.thresholds(8)
(data_points);
Is there a function to retrieve the bin thresholds? I guess I could loop through the array and identify the max in each bin, but that would be tedious. Am guessing there must be a function, including one that produces bin thresholds that are 'pleasing to the human eye' and not some ghastly decimal number.
Pseudo code would be something like this:
var bin_thresholds = bin.thresholds();
There is no method (as far as I know) for returning the lower and upper limits for each bin. There is one method for returning the number of bins...
d3.thresholdSturges(values);
... which is clearly not what you want, specially because you're setting the number of bins already.
However, you don't need to "loop through the array and identify the max in each bin". The histogram generator generates two properties:
x0 - the lower bound of the bin (inclusive).
x1 - the upper bound of the bin (exclusive, except for the last bin).
So, you can loop through the result to get those properties only. For instance:
var data = d3.range(1000).map(() => Math.random() * 20);
var bins = d3.histogram()
.thresholds(8)(data)
bins.forEach(function(d, i) {
console.log("Array number " + i + " --> Lower limit: " + d.x0 + " Upper limit:" + d.x1)
})
<script src="https://d3js.org/d3.v5.min.js"></script>
PS: you don't need to set the domain, since you're just passing the min and max value of the array, which is the default domain:
If domain is specified, sets the domain accessor to the specified function or array and returns this histogram generator. If domain is not specified, returns the current domain accessor, which defaults to extent. (emphasis mine)
I have this image which is completely black except for a white object in the middle (it can be anything but it is always completely white). What I would like to do with nodeJs, is trace the boundary of the object (I would like to find all the white points which are next to black) in the image (performance is key!)
With pngjs I can read an image which gives me an array in which each pixels has 4 values (RGBA). Its a one dimensional array. So, suppose the image is 1000 x 1000 pixels it gives me an array of 1000 x 1000 x 4 = 4000000 entries.
The below expression converts x and y into an array index
var idx = (1000 * y + x) << 2;
data[idx] = 243;
data[idx + 1] = 16;
data[idx + 2] = 16;
Anyway, I could traverse the whole array and register the points where black changes into white, but as I said, performance is very important. I can imagine that some kind of smart iterative search algorithm exists that can follow the boundary somehow :)
Maybe someone knows a library that can help, or an article about how to do this would be great too!!
Check out chain codes like Freeman Code. You need 1 contour point to start with. So just iterate through your lines until you hit your object. Then you walk around your object until you reach your starting point. You will get a code that describes the direction you took for every step. This code can be used to calculate various object features or to just draw the contour of your object.
Btw if your obect is always white and your background is always black you don't have to process 4 channels. Red, green or blue channels contain the same information. Just use either one of them.
Consider the following code (shortened for clarity):
Vertices.chamfer = function(vertices, radius, quality, qualityMin, qualityMax) {
radius = radius || [8];
if (!radius.length)
radius = [radius];
};
I'm reading the first part as (pseudocode):
if (radius array passed to function) then
radius = radius
else
radius = [8] // new array, single element of value 8
end if
but I don't get the second expression (the if(!radius.length) radius = [radius] part).
Could someone explain it to me?
Vertices.chamfer = function(vertices, radius, quality, qualityMin, qualityMax) {
// Set the radius to the value passed in.
// If the value passed in is undefined, set the radius to an
// array containing 8 (the "default")
radius = radius || [8];
// If radius isn't an array (i.e. length is 0 which evaluates to false),
// make it an array.
if (!radius.length)
radius = [radius];
};
The concept used here is known as duck typing - i.e. checking the type of a variable by looking for the presence (or absence) of some characteristic properties/methods. Author of this code assumed that if something has a length property it is an Array, and if not it can be converted to an Array by wrapping it in [].
The second part, converting x to a single-item Array containing x using x = [x] is totally OK.
The problem is there are other types in JavaScript that have a length so this test is not really reliable.
For example: "7".length returns 1, so if you passed the variable as a string instead of a number (easy enough to make this mistake for example by reading values from <input> fields) something would break down the line expecting an Array but getting a String.
Another type that has a length is a Function: (function(a,b){}).length == 2.
So yeah, this is not really good test but the basic idea makes sense. Should have used either Array.isArray or some other property/method that is unique to Arrays.
EDIT: I'd also point out the radius = radius || [8] construct is only OK if 0 is not allowed as the radius argument.
I'm reading the first part as (pseudocode):
if (radius array passed to function) then
radius = radius || [8] checks to see if radius is falsy, which might be not passed in at all (undefined), or passed in as undefined, or passed in as null, 0, "", NaN, or of course false.
But yes, it's probably intended to use [8] if no radius was specified.
but I don't get the second expression (the if(!radius.length) radius = [radius] part).
If radius was given but is either A) An array with no entries (radius.length is 0 and therefore falsy), or B) Not an array (radius.length is undefined and therefore falsy), we create an array and make it the first entry in that array.
That's not at all robust validation, lots of things have length (strings, for instance), but that's probably what it's meant to do.
The robust way to check if something is an array is to use Array.isArray, which was added in ES5 and which can be reliably shimmed/polyfilled for obsolete engines (like the one in IE8).
if imported radius is not an array and it has value, then it returns an array with the radius value inside.
if imported radius is empty then it returns an array with the number 8 inside: [8]
However, inside the pseudocode, it is not gonna get in if statement, if the imported radius is not an array.
Is there a way to detect an empty area, without text or images within a web page, using JavaScript?
More precisely, how to determine whether point [x,y] is within a blank area, like in the following example (marked in red)
EDIT: I want to make my question clearer, I'm building an extension which supposed to mark search results as trustworthy or as spam, I want to put my marking at the end of the text of a result item URL.
I also want to do it in a generic way, so it wouldn't work only in Google web page. an example is shown below:
You can test for genuine white space like this :
function isWhiteSpace(coords) {
var element = document.elementFromPoint(coords.x, coords.y);
var whitespace = $(document).add("body, html");
return (whitespace.get().indexOf(element) > -1) ? true : false;
}
where coords is an object with .x and .y properties.
DEMO
document.elementFromPoint(), documented here, is "an experimental technology", so I wouldn't trust my life to it. Test thoroughly on all target platforms.
Edit
For the full detection of all the white you seek, isWhiteSpace() would be the first of two stages. The second stage would be isVisualWhiteSpace() implemented with #remdevtec's approach for example.
As my isWhiteSpace(coords) is inexpensive, you would perform it first and only if it returned false go for the expensive test. You could use the protective property of ||
var isWhite = isWhiteSpace(coords) || isVisualWhiteSpace(coords);
But your real problem will be writing isVisualWhiteSpace(). I can't help with that.
One approach would be to work with a screenshot of the window.
You can use libraries like html2canvas to load a screenshot to a HTML canvas element.
Next, on window.onclick, use the automatic event parameter to get an RGBA array of the clicked coordinate:
var pixelData = canvas.getContext('2d').getImageData(
event.offsetX,
event.offsetY, 1, 1)
.data;
Now if all (or at least the first three) pixelData's items equal 255, it means that this point is white.
if (pixelData[0] == 255 && pixelData[1] == 255 && pixelData[2] == 255) {
// the clicked point is white,
// and if the background-color is white,
// it can be considered an empty point
}
Of course, the down side is that you have to know the background color of the site you're testing, or the background color of the element you click in.
You can build a matrix with width and length of the page.
Set all matrix cells to zero.
Get all elements of the DOM.
Get x, y, width, and height of each element, this link may help
Retrieve the position (X,Y) of an HTML element
Draw the elements in the matrix
for(k=0;k < dom_elements.length;k++) {
for(i=dom_elements[k].y;i < dom_elements[k].length;i++) {
for(j=dom_elements[k].x;j < dom_elements[k].width;j++) {
matrix[i][j] = 1 ;
}
}
}
And finally check if matrix[i][j] is set to zero or 1