Develop Reference

How to improve accuracy of a FeedForward Neural Network?

I want to draw StackOverflow's logo with this Neural Network:
The NN should ideally become [r, g, b] = f([x, y]). In other words, it should return RGB colors for a given pair of coordinates. The FFNN works pretty well for simple shapes like a circle or a box. For example after several thousands epochs a circle looks like this:
Try it yourself: https://codepen.io/adelriosantiago/pen/PoNGeLw
However since StackOverflow's logo is far more complex even after several thousands of iterations the FFNN's results are somewhat poor:
From left to right:
StackOverflow's logo at 256 colors.
With 15 hidden neurons: The left handle never appears.
50 hidden neurons: Pretty poor result in general.
0.03 as learning rate: Shows blue in the results (blue is not in the orignal image)
A time-decreasing learning rate: The left handle appears but other details are now lost.
Try it yourself: https://codepen.io/adelriosantiago/pen/xxVEjeJ
Some parameters of interest are synaptic.Architect.Perceptron definition and learningRate value.
How can I improve the accuracy of this NN?
Could you improve the snippet? If so, please explain what you did. If there is a better NN architecture to tackle this type of job could you please provide an example?
Additional info:
Artificial Neural Network library used: Synaptic.js
To run this example in your localhost: See repository

By adding another layer, you get better results :
let perceptron = new synaptic.Architect.Perceptron(2, 15, 10, 3)
There are small improvements that you can do to improve efficiency (marginally):
Here is my optimized code:
const width = 125
const height = 125
const outputCtx = document.getElementById("output").getContext("2d")
const iterationLabel = document.getElementById("iteration")
const stopAtIteration = 3000
let perceptron = new synaptic.Architect.Perceptron(2, 15, 10, 3)
let iteration = 0
let inputData = (() => {
const tempCtx = document.createElement("canvas").getContext("2d")
tempCtx.drawImage(document.getElementById("input"), 0, 0)
return tempCtx.getImageData(0, 0, width, height)
})()
const getRGB = (img, x, y) => {
var k = (height * y + x) * 4;
return [
img.data[k] / 255, // R
img.data[k + 1] / 255, // G
img.data[k + 2] / 255, // B
//img.data[(height * y + x) * 4 + 3], // Alpha not used
]
}
const paint = () => {
var imageData = outputCtx.getImageData(0, 0, width, height)
for (let x = 0; x < width; x++) {
for (let y = 0; y < height; y++) {
var rgb = perceptron.activate([x / width, y / height])
var k = (height * y + x) * 4;
imageData.data[k] = rgb[0] * 255
imageData.data[k + 1] = rgb[1] * 255
imageData.data[k + 2] = rgb[2] * 255
imageData.data[k + 3] = 255 // Alpha not used
}
}
outputCtx.putImageData(imageData, 0, 0)
setTimeout(train, 0)
}
const train = () => {
iterationLabel.innerHTML = ++iteration
if (iteration > stopAtIteration) return
let learningRate = 0.01 / (1 + 0.0005 * iteration) // Attempt with dynamic learning rate
//let learningRate = 0.01 // Attempt with non-dynamic learning rate
for (let x = 0; x < width; x += 1) {
for (let y = 0; y < height; y += 1) {
perceptron.activate([x / width, y / height])
perceptron.propagate(learningRate, getRGB(inputData, x, y))
}
}
paint()
}
const startTraining = (btn) => {
btn.disabled = true
train()
}
EDIT : I made another CodePen with even better results:
https://codepen.io/xurei/pen/KKzWLxg
It is likely to be over-fitted BTW.
The perceptron definition:
let perceptron = new synaptic.Architect.Perceptron(2, 8, 15, 7, 3)

Taking some insights from the lecture/slides of Bhiksha Raj (from slides 62 onwards), and summarizing as below:
Each node can be assumed like a linear classifier, and combination of several nodes in a single layer of neural networks can approximate any basic shapes. For example, a rectangle can be formed by 4 nodes for each lines, assuming each nodes contributes to one line, and the shape can be approximated by the final output layer.
Falling back to the summary of complex shapes such as circle, it may require infinite nodes in a layer. Or this would likely hold true for a single layer with two disjoint shapes (A non-overlapping triangle and rectangle). However, this can still be learnt using more than 1 hidden layers. Where, the 1st layer learns the basic shapes, followed by 2nd layer approximating their disjoint combinations.
Thus, you can assume that this logo is combination of disjoint rectangles (5 rectangles for orange and 3 rectangles for grey). We can use atleast 32 nodes in 1st hidden layer and few nodes in the 2nd hidden layer. However, we don't have control over what each node learns. Hence, a few more number of neurons than required neurons should be helpful.

Finding the closest indexed color value to the current color in javascript / p5.js

I have an array of "indexed" RGBA color values, and for any given image that I load, I want to be able to run through all the color values of the loaded pixels, and match them to the closest of my indexed color values. So, if the pixel in the image had a color of, say, RGBA(0,0,10,1), and the color RGBA(0,0,0,1) was my closest indexed value, it would adjust the loaded pixel to RGBA(0,0,0,1).
I know PHP has a function imagecolorclosest
int imagecolorclosest( $image, $red, $green, $blue )
Does javascript / p5.js / processing have anything similar? What's the easiest way to compare one color to another. Currently I can read the pixels of the image with this code (using P5.js):
let img;
function preload() {
img = loadImage('assets/00.jpg');
}
function setup() {
image(img, 0, 0, width, height);
let d = pixelDensity();
let fullImage = 4 * (width * d) * (height * d);
loadPixels();
for (let i = 0; i < fullImage; i+=4) {
let curR = pixels[i];
let curG = pixels[i]+1;
let curB = pixels[i+2];
let curA = pixels[i+3];
}
updatePixels();
}

Each color consists 3 color channels. Imagine the color as a point in a 3 dimensional space, where each color channel (red, green, blue) is associated to one dimension. You've to find the closest color (point) by the Euclidean distance. The color with the lowest distance is the "closest" color.
In p5.js you can use p5.Vector for vector arithmetic. The Euclidean distance between to points can be calculated by .dist(). So the distance between points respectively "colors" a and b can be expressed by:
let a = createVector(r1, g1, b1);
let b = createVector(r2, g2, b2);
let distance = a.dist(b);
Use the expression somehow like this:
colorTable = [[r0, g0, b0], [r1, g1, b1] ... ];
int closesetColor(r, g, b) {
let a = createVector(r, g, b);
let minDistance;
let minI;
for (let i=0; i < colorTable; ++i) {
let b = createVector(...colorTable[i]);
let distance = a.dist(b);
if (!minDistance || distance < minDistance) {
minI = i; minDistance = distance;
}
}
return minI;
}
function setup() {
image(img, 0, 0, width, height);
let d = pixelDensity();
let fullImage = 4 * (width * d) * (height * d);
loadPixels();
for (let i = 0; i < fullImage; i+=4) {
let closestI = closesetColor(pixels[i], pixels[i+1], pixels[i+2])
pixels[i] = colorTable[closestI][0];
pixels[i+1] = colorTable[closestI][1];
pixels[i+2] = colorTable[closestI][2];
}
updatePixels();
}

If I understand you correctly you want to keep the colors of an image within a certain limited pallet. If so, you should apply this function to each pixel of your image. It will give you the closest color value to a supplied pixel from a set of limited colors (indexedColors).
// example color pallet (no alpha)
indexedColors = [
[0, 10, 0],
[0, 50, 0]
];
// Takes pixel with no alpha value
function closestIndexedColor(color) {
var closest = {};
var dist;
for (var i = 0; i < indexedColors.length; i++) {
dist = Math.pow(indexedColors[i][0] - color[0], 2);
dist += Math.pow(indexedColors[i][1] - color[1], 2);
dist += Math.pow(indexedColors[i][2] - color[2], 2);
dist = Math.sqrt(dist);
if(!closest.dist || closest.dist > dist){
closest.dist = dist;
closest.color = indexedColors[i];
}
}
// returns closest match as RGB array without alpha
return closest.color;
}
// example usage
closestIndexedColor([0, 20, 0]); // returns [0, 10, 0]
It works the way that the PHP function you mentioned does. If you treat the color values as 3d coordinate points then the closet colors will be the ones with the smallest 3d "distance" between them. This 3d distance is calculated using the distance formula:

Generate a number of random width and height blocks and fit them full width of the HTML5 canvas - EaselJS

I am trying to do the following:
Generate a number (specified by var number) of random width and height blocks
Fill these blocks to 100% width of the canvas
My code so far
function init() {
//find canvas and load images, wait for last image to load
var canvas = document.getElementById("Canvas");
var stage = new createjs.Stage(canvas);
// set width and height
stage.canvas.width = 500;
stage.canvas.height = 500;
var number = 5;
for(var i = 0; i < number; i++){
var shape = new createjs.Shape();
var shapeWidth = Math.floor(Math.random() * 100);
var shapeHeight = 50;
shape.graphics.beginFill('red').drawRect(0, 0, shapeWidth, shapeHeight);
shape.x = i * 51;
shape.y = canvas.height - 50;
stage.addChild(shape);
stage.update();
}
}
JSFiddle: https://jsfiddle.net/n5kgbe3g/1/
Thanks in advance

It can get very complicated to divide a length randomly if you include constraints , such as min and max size of divisions, must maintain true randomness (can't cheat and just add last bit if the divisions don't fit perfectly).
So the easy way is to set a simple constraint. Divide length into n random lengths each length no shorter than x.
// length is the length to divide randomly
// count is the number of divisions
// min is the minimum division size
// array (optional) the array to add the data too. Will be created if not given
// returns array containing count items. Each item is a random value > min
// where the total sum of all the items is === length.
// If the array already has items then new items are added. Only the new
// items will satisfy the length constraint.
// NOTE There is no vetting YOU MUST SUPPLY good arguments or the function
// will return nonsense.
function randomDiv(length,count,min,array = []){
var rand, total = 0; // total to normalise random values
length -= count * min; // get the remaining length after removing length
// used by min length
var startAt = array.length; // If adding to array get the start index
// get a set of random values and sum them
while(array.length - startAt < count){
ran = Math.random();
total += ran;
array.push(ran);
}
// use the total sum of random values to
// normalise the array items to be the correct length
while(count --){
array[startAt + count] = min + array[startAt + count] / total * length;
}
return array;
}
To use
// get 20 box widths to fit canvas width no smaller than 10 pixels
var boxes = randomDiv(canvas.width, 20, 10);
// to reuse the array. Create a referance
var boxes = [];
// each time out call the function pass the referance and empty it or
// result is added
boxes.length = 0;
boxes = randomDiv(canvas.width, 20, 10, boxes);
To create a set of boxes with differing qualities
// 100 small lengths for half the canvas one or more pixels wide
var boxes = randomDiv(canvas.width / 2, 100, 1);
// get 10 more boxes for the remaining half larger than 30 pixels
boxes = randomDiv(canvas.width / 2, 10, 30, boxes);
// shuffle the array
var shuffled = [];
while(boxes.length > 0){
shuffled.push(boxes.splice( Math.floor(Math.random() * boxes.length) , 1)[0]);
}

Paper.js: fastest way to draw many iterated shapes over loop

jsfiddle here: http://jsfiddle.net/yw0w18m3/2/
I'm using paper.js to make a background image that looks somthing like this:
Basically, I'm creating a couple thousand triangles over a loop and rotating them on every other iteration.
function Tri(x, y, rotate) {
var tri = new Path([
new Point((x - 42), (y - 48)),
new Point((x - 42), y),
new Point(x, (y - 24)),
new Point((x - 42), (y - 48))
]);
tri.fillColor = {
hue: Math.random() * 360,
saturation: 0,
brightness: ( (( Math.random() ) * .95) + .3 )
};
if(rotate) { tri.rotate(180); }
}
for (var i = 0; i < 2000; i++) {
rotate = false;
if( i % 2 ) {
rotate = true;
}
new Tri(x, y, rotate);
x = x + 42;
if( x > (winWidth + 42) ) {
x = 0 ;
y = y + 24;
}
}
There seems to be a brief 1-2 second pause/freeze though while the shapes are being drawn. Is there a more efficient way to draw all the shapes first (or push to an array) then add that to the canvas all at once?
I based my code off of the example here: http://paperjs.org/examples/candy-crash/ (click "source" in the upper right corner).
Any help is much appreciated.
Thanks!

I would end up creating two triangles, one rotated, so they don't have to be built from new points each time. Then choose the correct triangle based on the rotation variable and clone it, as opposed to create points and a triangle from scratch each time. Finally, just change the position of the cloned triangle.
Last, I would correct the maxTri so it doesn't do more than it needs to. The paren should follow the 48, not the 24. You're doing an order of magnitude more triangles than needed.
Here's a link to the sketch.paperjs.org solution I created based on your code. I find sketch easier to use than jsfiddle for paper examples.
proto1 = new Path([
new Point(0, -24),
new Point(0, 24),
new Point(42, 0)
]);
proto1.closed = true;
proto2 = proto1.clone();
proto2.rotate(180);
function putTriangle(pos, rotate) {
var tri = (rotate ? proto2 : proto1).clone();
tri.position = pos;
tri.position = tri.position.subtract([21, 0])
tri.fillColor = {
hue: Math.random() * 360,
saturation: 0,
brightness: Math.random() * 0.5 + 0.5
}
}
var tris = [],
x = 42,
y = 24,
rotate,
winWidth = paper.view.size.width,
winHeight = paper.view.size.height,
rows = (winHeight + 48) / 24,
cols = (winWidth + 42) / 42,
numTri = rows * cols,
numTriOrig = (winWidth + 42) / 42 * (winHeight + 48 / 24);
//console.log(numTri, numTriOrig);
x = 0;
y = 0;
for (var row = 0; row < rows; row++) {
rowrotate = row % 2;
for (var col = 0; col <= cols; col++) {
rotate = rowrotate ^ col % 2;
putTriangle([x,y], rotate);
x += 42;
}
x = 0;
y = y + 24;
}

Two thoughts:
I see you use rotate to transform you triangles into place. This is an expensive operation. You could replace the rotate with a less geometric & more arithmetic calculation of the triangles orientation.
Also, I see is that the fill color is being changed with each triangle and state changes (like fill) are modestly expensive. You could group all the similarly colored triangles and draw them in a single batch.

Javascript store array outside dynamic content and reuse if needed

I'm experimenting with some html 5 canvas and at a certain point I tried to update the canvas with new bars (rectangles).
Now the tricky part.
Lets say you have an array with 1800 items(numbers generated by a php request).
You also have an html 5 canvas with the width of 1163px.
Now you need to draw bars(rectangles) in the canvas, but the bars need to remain 2px wide and there must be a 1px margin between all bars.
So the PHP file always returns 1800 numbers (color codes from image) and you need to extract just enough numbers to fit the canvas width(with the margin included(bar+margin)).
The new array of numbers you extract cannot just be the first 100-200 numbers! It must contain the (first or second) and (last or last-1) number.
What I have tried so far
<script>
// 1800 numbers !!! ......... //
arr = [12,1,21,1,2,13,21,32,1,5,4,6,5,4,6,4,8,3,5,4,9,6,8,7,9,1,3,6,4,6,4,5,9,87,4,4,5];
c = getElementById('canvas');
ctx = c.getContex('2d');
var can = 360;
var spliter = 2;
var canWidth = c.width;
var numOfBars = arr.length;
var barwidth = 2;
var margin = 1;
// amount of bars that will fit in the canvas //
var maxbars = (canWidth / (barwidth + margin));
var offset = arr.length / maxbars;
for(var i = 0; i < arr.length; i++) {
if(arr[i] % offset){
ctx.fillStyle = "#000000";
ctx.fillRect(
this.margin + i * this.barwidth / numOfBars,
0,
barwidth + margin,
100
);
}
}
</script>
This does not seem to work how I need it to work.
Thanks in advance.

Maxbars must be an integer right? and same for offset.
After that offset is your step.
var maxbars = Math.floor(canWidth / (barwidth + margin));
var offset = Math.floor(arr.length / maxbars);
for(var i = 0; i < arr.length; i+=offset) {
var colorcode = arr[i];
// fill stuff
}