Bilateral filter algorithm - javascript

I'm trying to implement a simple bilateral filter in javascript. This is what I've come up with so far:
// For each pixel
for (var y = kernelSize; y < height-kernelSize; y++) {
for (var x = kernelSize; x < width-kernelSize; x++) {
var pixel = (y*width + x)*4;
var sumWeight = 0;
outputData[pixel] = 0;
outputData[pixel+1] = 0;
outputData[pixel+2] = 0;
outputData[pixel+3] = inputData[pixel+3];
// For each neighbouring pixel
for(var i=-kernelSize; i<=kernelSize; i++) {
for(var j=-kernelSize; j<=kernelSize; j++) {
var kernel = ((y+i)*width+x+j)*4;
var dist = Math.sqrt(i*i+j*j);
var colourDist = Math.sqrt((inputData[kernel]-inputData[pixel])*(inputData[kernel]-inputData[pixel])+
(inputData[kernel+1]-inputData[pixel+1])*(inputData[kernel+1]-inputData[pixel+1])+
(inputData[kernel+2]-inputData[pixel+2])*(inputData[kernel+2]-inputData[pixel+2]));
var curWeight = 1/(Math.exp(dist*dist/72)*Math.exp(colourDist*colourDist*8));
sumWeight += curWeight;
outputData[pixel] += curWeight*inputData[pixel];
outputData[pixel+1] += curWeight*inputData[pixel+1];
outputData[pixel+2] += curWeight*inputData[pixel+2];
}
}
outputData[pixel] /= sumWeight;
outputData[pixel+1] /= sumWeight;
outputData[pixel+2] /= sumWeight;
}
}
inputData is from a html5 canvas object and is in the form of rgba.
My images are either coming up with no changes or with patches of black around edges depending on how i change this formula:
var curWeight = 1/(Math.exp(dist*dist/72)*Math.exp(colourDist*colourDist*8));
Unfortunately I'm still new to html/javascript and image vision algorithms and my search have come up with no answers. My guess is there is something wrong with the way curWeight is calculated. What am I doing wrong here? Should I have converted the input image to CIElab/hsv first?

I'm no Javasript expert: Are the RGB values 0..255? If so, Math.exp(colourDist*colourDist*8) will yield extremely large values - you'll probably want to scale colourDist to the range [0..1].
BTW: Why do you calculate the sqrt of dist and colourDist if you only need the squared distance afterwards?

First of all, your images turn out black/weird in the edges because you don't filter the edges. A short look at your code would show that you begin at (kernelSize,kernelSize) and finish at (width-kernelSize,height-kernelSize) - this means that you only filter a smaller rectangle inside the image where your have a margin of kernelSize on each side which is unfilterred. Without knowing your javscript/html5, I would assume that your outputData array is initialized with zero's (which means black) and then not touching them would leave them black. See my link the comment to your post for code that does handle the edges.
Other than that, follow #nikie's answer - your probably want to make sure the color distance is clamped to the range of [0,1] - youo can do this by adding the line colourDist = colourDist / (MAX_COMP * Math,sqrt(3)) (directly after the first line to calculate it). where MAX_COMP is the maximal value a color component in the image can have (usually 255)

I've found the error in the code. The problem was I was adding each pixel to itself instead of its surrounding neighbours. I'll leave the corrected code here in case anyone needs a bilateral filter algorithm.
outputData[pixel] += curWeight*inputData[kernel];
outputData[pixel+1] += curWeight*inputData[kernel+1];
outputData[pixel+2] += curWeight*inputData[kernel+2];

Related

Algorithm to take explicit heights of images and lay them out in optimally compact "masonry"-style layout?

I am using react-masonry-css to layout some images in a masonry-like layout, but it just is putting the same number of images in each column basically, and it ends up having large discrepancies in the amount of space left at the bottom of each column, as seen here (this is the bottom of all the columns):
(Note, a couple images in this demo are missing, but even if they are all present there are large differences in the bottom space remaining).
It looks like all that library does is put equal number of items in each column, irrespective of the image heights.
I would like for it to distribute the images more optimally, given that I also have included the width and height for each image in the JSON data. How can I use the height to figure out how to optimally place the images in a known number of columns? It seems extremely complicated at first thought, like it would require a sophisticated research-paper-like algorithm. Is that true? If so, what are the general algorithms used to solve this, so I may go about writing a quick implementation in JavaScript? If it's instead straightforward, how is it done?
In my head so far, I would maybe divide the number of images by number of columns to give the first rough estimate of how many per column. Then I would put one image in each column, so there are 1 image in each of the 7 columns. Then I would lay like bricks across the columns next images. I would do the best to keep the column heights the same high, searching through and selecting images that fit an appropriate height.
That is just brainstorming, but already I can see several holes and pitfalls in that non-workable approach. It seems quite hard, wondering if it's a solved problem though, given the images can be widely varied in height.
The end goal is to have all of the columns be roughly the same height, that's it. As close as possible.
If we operationalize “most compact” to mean the shortest bounding box,
then this is an identical-machines
scheduling
problem. Each image corresponds to a job, and each column corresponds to
a machine. The processing time of a job is the height/width ratio of the
corresponding image (plus padding).
Although scheduling is NP-hard, there’s a simple and provably effective
approximation known as Longest Processing-Time
First.
In terms of your problem, for each image in descending order of
height/width, you assign it to the column that is currently the
shortest. (You can shuffle the order of images in each column at the end
to avoid bias towards taller images.) The bounding box will never be
more than 34% longer than it needs to be (well, maybe a little more
because of the interstitials).
// Set up some test data.
const imageCount = 50;
let images = [];
for (let i = 0; i < imageCount; ++i) {
// Allow the images to vary in aspect ratio between 5:1 and 1:5.
images.push({
id: i,
width: Math.random() + 0.25,
height: Math.random() + 0.25,
});
}
// Parameters.
const columnCount = 10;
const thumbnailWidth = 100;
const interstitialHeight = 10;
// Algorithm begins. Initialize empty columns.
let columns = [];
let columnHeights = [];
for (let j = 0; j < columnCount; ++j) {
// This height will be correct once the column is nonempty.
columnHeights.push(-interstitialHeight);
columns.push([]);
}
// Sort the images by aspect ratio descending.
function aspectRatioDescending(a, b) {
return b.height / b.width - a.height / a.width;
}
images.sort(aspectRatioDescending);
// Assign each image to a column.
for (const image of images) {
// Find the shortest column.
let shortest = 0;
for (let j = 1; j < columnCount; ++j) {
if (columnHeights[j] < columnHeights[shortest]) {
shortest = j;
}
}
// Put the image there.
columnHeights[shortest] +=
interstitialHeight + thumbnailWidth * (image.height / image.width);
columns[shortest].push(image);
}
// Shuffle the columns for aesthetic reasons.
for (const column of columns) {
for (let k = 1; k < column.length; ++k) {
const i = Math.floor((k + 1) * Math.random());
let temp = column[i];
column[i] = column[k];
column[k] = temp;
}
}
const maxHeight = Math.max.apply(null, columnHeights);
const minHeight = Math.min.apply(null, columnHeights);
// Analyze the layout.
console.log(
"// The tallest column is %f%% taller than the shortest.",
(100 * (maxHeight - minHeight)) / minHeight
);
// The tallest column is 3.030982959129835% taller than the shortest.

Neural Network, gradient descent only finds the average of the outputs?

This problem is more conceptual than in the code, so the fact that this is written in JS shouldn't matter very much.
So I'm trying to make a Neural Network and I'm testing it by trying to train it to do a simple task - an OR gate (or, really, just any logic gate). I'm using Gradient Descent without any batches for the sake of simplicity (batches seem unnecessary for this task, and the less unnecessary code I have the easier it is to debug).
However, after many iterations the output always converges to the average of the outputs. For example, given this training set:
[0,0] = 0
[0,1] = 1
[1,0] = 1
[1,1] = 0
The outputs, no matter the inputs, always converge around 0.5. If the training set is:
[0,0] = 0,
[0,1] = 1,
[1,0] = 1,
[1,1] = 1
The outputs always converge around 0.75 - the average of all the training outputs. This appears to be true for all combinations of outputs.
It seems like this is happening because whenever it's given something with an output of 0, it changes the weights to get closer to that, and whenever it's given something with an output of 1, it changes the weights to get closer to that, meaning that overtime it will converge around the average.
Here's the backpropagation code (written in Javascript):
this.backpropigate = function(data){
//Sets the inputs
for(var i = 0; i < this.layers[0].length; i ++){
if(i < data[0].length){
this.layers[0][i].output = data[0][i];
}
else{
this.layers[0][i].output = 0;
}
}
this.feedForward(); //Rerun through the NN with the new set outputs
for(var i = this.layers.length-1; i >= 1; i --){
for(var j = 0; j < this.layers[i].length; j ++){
var ref = this.layers[i][j];
//Calculate the gradients for each Neuron
if(i == this.layers.length-1){ //Output layer neurons
var error = ref.output - data[1][j]; //Error
ref.gradient = error * ref.output * (1 - ref.output);
}
else{ //Hidden layer neurons
var gradSum = 0; //Find sum from the next layer
for(var m = 0; m < this.layers[i+1].length; m ++){
var ref2 = this.layers[i+1][m];
gradSum += (ref2.gradient * ref2.weights[j]);
}
ref.gradient = gradSum * ref.output * (1-ref.output);
}
//Update each of the weights based off of the gradient
for(var m = 0; m < ref.weights.length; m ++){
//Find the corresponding neuron in the previous layer
var ref2 = this.layers[i-1][m];
ref.weights[m] -= LEARNING_RATE*ref2.output*ref.gradient;
}
}
}
this.feedForward();
};
Here, the NN is in a structure where each Neuron is an object with inputs, weights, and an output which is calculated based on the inputs/weights, and the Neurons are stored in a 2D 'layers' array where the x dimension is the layer (so, the first layer is the inputs, second hidden, etc.) and the y dimension is a list of the Neuron objects inside of that layer. The 'data' inputted is given in the form [data,correct-output] so like [[0,1],[1]].
Also my LEARNING_RATE is 1 and my hidden layer has 2 Neurons.
I feel like there must be some conceptual issue with my backpropagation method, as I've tested out the other parts of my code (like the feedForward part) and it works fine. I tried to use various sources, though I mostly relied on the wikipedia article on backpropagation and the equations that it gave me.
.
.
I know it may be confusing to read my code, though I tried to make it as simple to understand as possible, but any help would be greatly appreciated.

Rendering too many points on Javascript-player

As part of a project, I have to render a video on a JS-player from a text file which has the points - all the changed coordinates along-with the color in each frame. Below is the code I'm using to draw these point on the screen.
But the issue is that the number of changed pixels in each frame are as high as ~20,000 and I need to display these in less than 30ms (inter-frame time difference). So, when I run this code the browser hangs for almost each frame. Could someone suggest an improvement for this?
Any help is really appreciated.
c.drawImage(img,0,0,800,800);
setInterval(
function(){
while(tArr[index]==time) {
var my_imageData = c.getImageData(0,0,width, height);
color(my_imageData,Math.round(yArr[index]),Math.round(xArr[index]),Math.round(iArr[index]),255);
c.putImageData(my_imageData,0,0);
index=index+1;
}
time = tArr[index];
}
,30);
xArr, yArr, iArr, tArr are arrays of x-coordinate, y-coordinate, intensity value and time of appearance respectively for the corresponding point to be rendered
function color(imageData,x,y,i,a){ //wrapper function to color the point
var index = (x + y * imageData.width) * 4;
imageData.data[index+0] = i;
imageData.data[index+1] = i;
imageData.data[index+2] = i;
imageData.data[index+3] = a;
}

Difficult to solve the phaser sliding puzzle as some parts of the original image is missing

Im trying to create the phaser examples game sliding puzzle
Live example is demonstrated here
But in the output game, some parts of the original image is missing. So it is difficult to solve the puzzle.
I am suspecting the algorithm of cutting the image to pieces is not correct.
The code for pieces is ,
function prepareBoard() {
var piecesIndex = 0,
i, j,
piece;
BOARD_COLS = Math.floor(game.world.width / PIECE_WIDTH);
BOARD_ROWS = Math.floor(game.world.height / PIECE_HEIGHT);
piecesAmount = BOARD_COLS * BOARD_ROWS;
shuffledIndexArray = createShuffledIndexArray();
piecesGroup = game.add.group();
for (i = 0; i < BOARD_ROWS; i++)
{
for (j = 0; j < BOARD_COLS; j++)
{
if (shuffledIndexArray[piecesIndex]) {
piece = piecesGroup.create(j * PIECE_WIDTH, i * PIECE_HEIGHT, "background", shuffledIndexArray[piecesIndex]);
}
else { //initial position of black piece
piece = piecesGroup.create(j * PIECE_WIDTH, i * PIECE_HEIGHT);
piece.black = true;
}
piece.name = 'piece' + i.toString() + 'x' + j.toString();
piece.currentIndex = piecesIndex;
piece.destIndex = shuffledIndexArray[piecesIndex];
piece.inputEnabled = true;
piece.events.onInputDown.add(selectPiece, this);
piece.posX = j;
piece.posY = i;
piecesIndex++;
}
}
}
function createShuffledIndexArray() {
var indexArray = [];
for (var i = 0; i < piecesAmount; i++)
{
indexArray.push(i);
}
return shuffle(indexArray);
}
function shuffle(array) {
var counter = array.length,
temp,
index;
while (counter > 0)
{
index = Math.floor(Math.random() * counter);
counter--;
temp = array[counter];
array[counter] = array[index];
array[index] = temp;
}
return array;
}
Please anyone have any idea ? Please share any algorithm to correctly cut the pieces.
Thanks in advance
iijb
This is the classic 15puzzle because it traditionally has a 4x4 grid with 1 tile missing (4x4-1=15 tiles). However the puzzle can practically be any grid size (4x3, 5x4, 6x6 etc).
You're using a .destIndex property to keep track of their position, but you could just give each tile a numbered index. I think that way it's easier because when all the tiles are ordered the puzzle is solved and it would also help the check-if-solvable-algorithm.
With these kind of sliding tile puzzles, there are two things to consider which are a little tricky, especially the 2nd point:
There is always one tile missing because that is the empty spot that the player can use to slide tiles into. Most commonly, the missing tile is the bottom-right tile of the image.
In your algorithm the blank tile is always the top-left tile of the image.
This is unusual and players might not expect that, however in theory it doesn't really matter and you could make a workable puzzle that way. You then keep track of the empty tile in code by value 1 (or maybe 0 for zero-indexed) because it's the first tile.
Some configurations are unsolvable, i.e. not every random scrambled tiles situation can be solved by sliding the tiles around.
A puzzle is solvable when the number of inversions (switches) needed to solve it is an even number, not odd. So count the number of pairs where a bigger number
is in front of a smaller one (=one inversions). For example in a 3x3 puzzle with the bottom-right tile missing:
5 3 4
2 6 1
8 7
In array it looks like this [5,3,4,2,6,1,8,7,9], so count pairs which are 5-3, 5-4, 5-2, 5-1, 3-2 3-1, 4-2 4-1, 2-1, 6-1, 8-7. This equals 11 pairs, so 11 inversions are needed. This is not an even number, thus this configuration is unsolvable. Btw note that the missing tile has internally the highest possible number, which is 9 in this case.
You can use this method to detect a unsolvable puzzle. All you need to do to make it solvable again is switch any two tiles, so for example the top first two tiles (so 5 and 3 in the example). When the number of switches needed is an even number, it's already solvable and you don't need to do anything.
I've made similar puzzle games, you can see the source code here to see how it works:
Photoscramble v2 (download incl Delphi source)
Photoscramble v1 (download incl BlitzBasic source)

Canvas Maze character's distance from walls

I am working on a 2D maze game with a torch effect in canvas without the use of any raytracing. Everything is working great, however the torch effect's algorithm is causing immense lags in several browsers and computers. (It is weird as well, that the game runs smoother on older computers. The funniest is, IExplorer runs the game without any lags, while mozzila dies on every move..)
My general idea for solving this problem was, to get how far the character is from the walls (4 functions) and make the rest of the maze grey.
Here is an example with the Northern wall detection:
http://webprogramozas.inf.elte.hu/~ce0ta3/beadando/maze_example.png
And an example how it is working at the moment and what I would like to achieve without lag issues.
http://webprogramozas.inf.elte.hu/~ce0ta3/beadando/ce0ta3_html5_maze.html
As I mentioned above, the algorithm that tracks the character's distance from the walls is causing incredible lags.
//Get the character's X,Y position as parameter
function distanceFromNorth (posX,posY)
{
distNorth = 0;
var l = false;
//Start getting charSize x 1 px lines from the character position towards the up, until we reach the max viewDistance or we find a black pixel in the charSize x 1 line.
for (var i = posY; i > posY - viewDistance && !l; i--)
{
var mazeWallData = context.getImageData(posX, i, charSize, 1);
var data = mazeWallData.data;
//Check if there are any black pixels in the line
for (var j = 0; j < 4 * charSize && !l; j += 4)
{
l = (data[j] === 0 && data[j + 1] === 0 && data[j + 2] === 0);
}
distNorth++;
}
return distNorth;
}
I am fairly sure, that the ctx.getImageData() is the most costly method in this linear search and if I only requested this method once for a charSize x viewDistance rectangle, and then check for black pixels in that huge array, then the lag could be reduced greatly. However, I still want to keep searching in lines, because finding only one black pixel will return false distNorth value.
I would be grateful if anyone could convert my code into the form I mentioned in the previous paragraph.
Assuming the image data isnt changing then you can precompute all the pixel values that have black pixel. Then use simple binary search on it to get the any black pixels in the given range.
Algorithm : -
cols[];
rows[];
for(int i=0;i<height;i++) {
for(int j=0;j<width;j++) {
if(pixel(j,i)==black) {
row[i].add(j);
col[j].add(i);
}
}
}
for query on (x,y) :
distance = binarysearch(col[x],y,y-distance) - y

Categories

Resources