I'm trying to teach a neural network to decide where to go based on its inputted life level. The neural network will always receive three inputs [x, y, life]. If life => 0.2, it should output the angle from [x, y] to (1, 1). If life < 0.2, it should output the angle from [x, y] to (0, 0).
As the inputs and outputs of neurons should be between 0 and 1, I divide the angle by 2 *Math.PI.
Here is the code:
var network = new synaptic.Architect.Perceptron(3,4,1);
for(var i = 0; i < 50000; i++){
var x = Math.random();
var y = Math.random();
var angle1 = angleToPoint(x, y, 0, 0) / (2 * Math.PI);
var angle2 = angleToPoint(x, y, 1, 1) / (2 * Math.PI);
for(var j = 0; j < 100; j++){
network.activate([x,y,j/100]);
if(j < 20){
network.propagate(0.3, [angle1]);
} else {
network.propagate(0.3, [angle2]);
}
}
}
Try it out here: jsfiddle
So when I enter the following input [0, 1, 0.19], I expect the neural network to output something close to [0.75] (1.5PI / 2PI). But my results are completely inconsistent and show no correlation with any input given at all.
What mistake am I making in teaching my Neural network?
I have managed to teach a neural network to output 1 when input [a, b, c] with c => 0.2 and 0 when input [a, b, c] with c < 0.2. I have also managed to teach it to output an angle to a certain location based on [x, y] input, however I can't seem to combine them.
As requested, I have written some code that uses 2 Neural Networks to get the desired output. The first neural network converts life level to a 0 or a 1, and the second neural network outputs an angle depending on the 0 or 1 it got outputted from the first neural network. This is the code:
// This network outputs 1 when life => 0.2, otherwise 0
var network1 = new synaptic.Architect.Perceptron(3,3,1);
// This network outputs the angle to a certain point based on life
var network2 = new synaptic.Architect.Perceptron(3,3,1);
for (var i = 0; i < 50000; i++){
var x = Math.random();
var y = Math.random();
var angle1 = angleToPoint(x, y, 0, 0) / (2 * Math.PI);
var angle2 = angleToPoint(x, y, 1, 1) / (2 * Math.PI);
for(var j = 0; j < 100; j++){
network1.activate([x,y,j/100]);
if(j < 20){
network1.propagate(0.1, [0]);
} else {
network1.propagate(0.1, [1]);
}
network2.activate([x,y,0]);
network2.propagate(0.1, [angle1]);
network2.activate([x,y,1]);
network2.propagate(0.1, [angle2]);
}
}
Try it out here: jsfiddle
As you can see in this example. It manages to reach the desired output quite closely, by adding more iterations it will come even closer.
Observations
Skewed Distribution sampled as Training set
Your training set is choosing the life parameter inside for(var j = 0; j < 100; j++), which is highly biased towards j>20 and consequently life>0.2. It has 4 times more training data for that subset, which makes your training function prioritize.
Non-shuffled training data
You are training sequentially against the life parameter, which can be harmful. You network will end up giving more attention to the bigger js since it's the most recent reason for network propagations. You should shuffle your training set to avoid this bias.
This will stack with the previous point, because you're again giving more attention to some subset of life values.
You should measure your training performance as well
Your network, despite previous observations, was not really that bad. Your training error was not as huge as your tests. This discrepancy usually means that you're training and testing on different sample distributions.
You could say that you have two classes of data points: the ones with life>0.2 and the others not. But because you introduced a discontinuity in the angleToPoint function, I'd recommend that you separate in three classes: keep a class for life<0.2 (because the function behaves continuously) and split life>0.2 in "above (1,1)" and "below (1,1)."
Network complexity
You could successfully train a network for each task separately. Now you want to stack them. This is quite the purpose of deep learning: each layer builds on the concepts perceived by the previous layer, therefore increasing the complexity of the concepts it can learn.
So instead of using 20 nodes in a single layer, I'd recommend that you use 2 layers of 10 nodes. This matches the classes hierarchy I mentioned in the previous point.
The Code
Running this code I had a training/testing error of 0.0004/0.0002.
https://jsfiddle.net/hekqj5jq/11/
var network = new synaptic.Architect.Perceptron(3,10,10,1);
var trainer = new synaptic.Trainer(network);
var trainingSet = [];
for(var i = 0; i < 50000; i++){
// 1st category: above vector (1,1), measure against (1,1)
var x = getRandom(0.0, 1.0);
var y = getRandom(x, 1.0);
var z = getRandom(0.2, 1);
var angle = angleToPoint(x, y, 1, 1) / (2 * Math.PI);
trainingSet.push({input: [x,y,z], output: [angle]});
// 2nd category: below vector (1,1), measure against (1,1)
var x = getRandom(0.0, 1.0);
var y = getRandom(0.0, x);
var z = getRandom(0.2, 1);
var angle = angleToPoint(x, y, 1, 1) / (2 * Math.PI);
trainingSet.push({input: [x,y,z], output: [angle]});
// 3rd category: above/below vector (1,1), measure against (0,0)
var x = getRandom(0.0, 1.0);
var y = getRandom(0.0, 1.0);
var z = getRandom(0.0, 0.2);
var angle = angleToPoint(x, y, 0, 0) / (2 * Math.PI);
trainingSet.push({input: [x,y,z], output: [angle]});
}
trainer.train(trainingSet, {
rate: 0.1,
error: 0.0001,
iterations: 50,
shuffle: true,
log: 1,
cost: synaptic.Trainer.cost.MSE
});
testSet = [
{input: [0,1,0.25], output: [angleToPoint(0, 1, 1, 1) / (2 * Math.PI)]},
{input: [1,0,0.35], output: [angleToPoint(1, 0, 1, 1) / (2 * Math.PI)]},
{input: [0,1,0.10], output: [angleToPoint(0, 1, 0, 0) / (2 * Math.PI)]},
{input: [1,0,0.15], output: [angleToPoint(1, 0, 0, 0) / (2 * Math.PI)]}
];
$('html').append('<p>Train:</p> ' + JSON.stringify(trainer.test(trainingSet)));
$('html').append('<p>Tests:</p> ' + JSON.stringify(trainer.test(testSet)));
$('html').append('<p>1st:</p> ')
$('html').append('<p>Expect:</p> ' + angleToPoint(0, 1, 1, 1) / (2 * Math.PI));
$('html').append('<p>Received: </p> ' + network.activate([0, 1, 0.25]));
$('html').append('<p>2nd:</p> ')
$('html').append('<p>Expect:</p> ' + angleToPoint(1, 0, 1, 1) / (2 * Math.PI));
$('html').append('<p>Received: </p> ' + network.activate([1, 0, 0.25]));
$('html').append('<p>3rd:</p> ')
$('html').append('<p>Expect:</p> ' + angleToPoint(0, 1, 0, 0) / (2 * Math.PI));
$('html').append('<p>Received: </p> ' + network.activate([0, 1, 0.15]));
$('html').append('<p>4th:</p> ')
$('html').append('<p>Expect:</p> ' + angleToPoint(1, 0, 0, 0) / (2 * Math.PI));
$('html').append('<p>Received: </p> ' + network.activate([1, 0, 0.15]));
function angleToPoint(x1, y1, x2, y2){
var angle = Math.atan2(y2 - y1, x2 - x1);
if(angle < 0){
angle += 2 * Math.PI;
}
return angle;
}
function getRandom (min, max) {
return Math.random() * (max - min) + min;
}
Further Remarks
As I mentioned in the comments and in the chat, there's no such a thing as "angle between (x,y) and (0,0)", because the notion of angle between vectors is usually taken as the difference between their directions and (0,0) has no direction.
Your function angleToPoint(p1, p2) returns instead the direction of (p1-p2). For p2 = (0,0), that means the angle between p1 and the x axis alright. But for p1=(1,1) and p2=(1,0) it will not return 45 degrees. For p1=p2, it's undefined instead of zero.
jsfiddle here: http://jsfiddle.net/yw0w18m3/2/
I'm using paper.js to make a background image that looks somthing like this:
Basically, I'm creating a couple thousand triangles over a loop and rotating them on every other iteration.
function Tri(x, y, rotate) {
var tri = new Path([
new Point((x - 42), (y - 48)),
new Point((x - 42), y),
new Point(x, (y - 24)),
new Point((x - 42), (y - 48))
]);
tri.fillColor = {
hue: Math.random() * 360,
saturation: 0,
brightness: ( (( Math.random() ) * .95) + .3 )
};
if(rotate) { tri.rotate(180); }
}
for (var i = 0; i < 2000; i++) {
rotate = false;
if( i % 2 ) {
rotate = true;
}
new Tri(x, y, rotate);
x = x + 42;
if( x > (winWidth + 42) ) {
x = 0 ;
y = y + 24;
}
}
There seems to be a brief 1-2 second pause/freeze though while the shapes are being drawn. Is there a more efficient way to draw all the shapes first (or push to an array) then add that to the canvas all at once?
I based my code off of the example here: http://paperjs.org/examples/candy-crash/ (click "source" in the upper right corner).
Any help is much appreciated.
Thanks!
I would end up creating two triangles, one rotated, so they don't have to be built from new points each time. Then choose the correct triangle based on the rotation variable and clone it, as opposed to create points and a triangle from scratch each time. Finally, just change the position of the cloned triangle.
Last, I would correct the maxTri so it doesn't do more than it needs to. The paren should follow the 48, not the 24. You're doing an order of magnitude more triangles than needed.
Here's a link to the sketch.paperjs.org solution I created based on your code. I find sketch easier to use than jsfiddle for paper examples.
proto1 = new Path([
new Point(0, -24),
new Point(0, 24),
new Point(42, 0)
]);
proto1.closed = true;
proto2 = proto1.clone();
proto2.rotate(180);
function putTriangle(pos, rotate) {
var tri = (rotate ? proto2 : proto1).clone();
tri.position = pos;
tri.position = tri.position.subtract([21, 0])
tri.fillColor = {
hue: Math.random() * 360,
saturation: 0,
brightness: Math.random() * 0.5 + 0.5
}
}
var tris = [],
x = 42,
y = 24,
rotate,
winWidth = paper.view.size.width,
winHeight = paper.view.size.height,
rows = (winHeight + 48) / 24,
cols = (winWidth + 42) / 42,
numTri = rows * cols,
numTriOrig = (winWidth + 42) / 42 * (winHeight + 48 / 24);
//console.log(numTri, numTriOrig);
x = 0;
y = 0;
for (var row = 0; row < rows; row++) {
rowrotate = row % 2;
for (var col = 0; col <= cols; col++) {
rotate = rowrotate ^ col % 2;
putTriangle([x,y], rotate);
x += 42;
}
x = 0;
y = y + 24;
}
Two thoughts:
I see you use rotate to transform you triangles into place. This is an expensive operation. You could replace the rotate with a less geometric & more arithmetic calculation of the triangles orientation.
Also, I see is that the fill color is being changed with each triangle and state changes (like fill) are modestly expensive. You could group all the similarly colored triangles and draw them in a single batch.
I'm building a patient monitor simulator in JavaScript. This involves drawing parabolas (semi-circles) on an HTML canvas. Normally this wouldn't be an issue with the bezierCurveTo() function however this solution is not applicable here as I need to animate the curve pixel by pixel as demonstrated here http://www.jet5.com/ecg/. From my understanding this will require an array of all points in the curve.
My question is how can I generate this array of points from a provided width and height that I want the curve to be. Is there some sort of special command or algorithm that I can use to obtain these Cartesian coordinates. For a clearer picture of what I need please refer to the following image http://en.ecgpedia.org/wiki/File:Epi_endo_en.png.
A lecturer helped me with the following equation: y = (x - t1) x (t2 - x). Here is my code (I ave created a point object and remember this is for an HTML canvas where 0, 0 is in the top left corner):
var y = 0;
var duration = 200;
var t1 = 0;
var t2 = duration;
var x1 = 0;
var x2 = 0;
for (i = 0; i < duration; i++) {
x1 = i;
x2 = duration - i;
var ctx = canvas.getContext("2d");
y = (x1 - t1) * (t2 - x2)
if (i < duration / 2) {
y = -y;
}
data.push(new point(y));
}
While this partly worked from my understanding this equation wouldn't allow me to specify a height only the parabolas width.
Any help is greatly appreciated
Best thing to do in this kind of mathematical situation is to normalize.
Which means here, try to always go back to the case when x is between 0 and 1.
so if x is in [ t1 ; t2 ]
delta = (x-t1) / (t2 - t1).
now delta moves in [0;1] !!! Magic
In the same way, for your shape functions use normalized function that returns only in [ 0 ; 1].
The example you give in your code becomes :
function shape(x) { return (x<0.5) ? x*(1-x) : -x*(1-x) }
And the code becomes - just to be clear -
for ( x = t1; x<t2; x++ ) {
var delta = (x-t1) / (t2 - t1) ;
y = amplitude * shape(delta);
// do something with (x,y)
}
See a working jsbin here :
http://jsbin.com/wodecowibexa/1/edit?js,output
![var cv = document.getElementById('cv');
var ctx = cv.getContext('2d');
ctx.fillStyle = '#C66';
// shift for all the drawings
var plotShift = {x:0, y:200 } ;
// draw between t1 and t2 with an initial y shift.
// expects an easing function i.e. a function \[0;1\] -> \[0;1\]
// yShift should be the last computed value.
// returns the last computed value to allow chaining.
function drawEasingFunction (yShift, t1, t2, amplitude, func) {
var x=0, y=0;
var duration = t2 - t1;
for (x= t1; x < t2; x++) {
// delta is a figure between 0 and 1
var delta = (x - t1) / duration;
//
y = yShift + amplitude*func(delta);
ctx.fillRect(x+plotShift.x,y+plotShift.y,2,2);
}
return y;
}
var easingFunctions = \[
function (x) { return x} , /* line */
function (x) { return x*x} , /* line */
function (x) { return Math.sqrt(x)} , /* line */
function (x) { return Math.sin(Math.PI*x/2)} , /* sin */
function (x) { return x*(1-x)} , /* that's your function */
\];
var lastY = 0;
var currentX = 0;
var length = 50;
// demo of all functions
for (; currentX < cv.width; currentX+=length) {
// take a random function
var fnIndex = Math.floor(Math.random()*easingFunctions.length) ;
var thisEasingFunction = easingFunctions\[fnIndex\];
// take some random amplitude
var amplitude = (60 + Math.random()*10);
// randomly switch the amplitude sign.
if (Math.random() > 0.5) amplitude = -amplitude;
// draw ! (and store last value)
lastY = drawEasingFunction(lastY, currentX, currentX+length, amplitude, thisEasingFunction);
}][2]
EDIT: So apparently, PI is finite in JavaScript (which makes sense). But that leaves me with a major problem. What's the next best way to calculate the angles I need?
Alright, first, my code:
http://jsfiddle.net/joshlalonde/vtfyj/34/
I'm drawing cubes that open up to a 120 degree angle.
So the coordinates are calculated based on (h)eight and theta (120).
On line 46, I have a for loop that contains a nested for loop used for creating rows/columns.
It's somewhat subtle, but I noticed that the lines aren't matching up exactly. The code for figuring out each cubes position is on line 49. One of the things in the first parameter (my x value) for the origin of the cube is off. Can anyone help figure out what it is?
var cube = new Cube(
origin.x + (j * -w * (Math.PI)) +
(i * w * (Math.PI))
, origin.y + j * (h / 2) +
i * (h / 2) +
(-k*h), h);
Sorry if that's confusing. I,j, and k refer to the variable being incremented by the for loops. So basically, a three dimensional for loop.
I think the problem lies with Math.PI.
The width isn't the problem, or so I believe. I originally used 3.2 (which I somehow guessed and it seemed to line up pretty good. But I have no clue what the magical number is). I'm guessing it has to do with the angle being converted to Radians, but I don't understand why Math.PI/180 isn't the solution. I tried multiple things. 60 (in degrees) * Math.PI/180 doesn't work. What is it for?
EDIT: It might just be a JavaScript related math problem. The math is theoretically correct but can't be calculated correctly. I'll accept the imperfection to spare myself from re-writing code in unorthodox manners. I can tell it would take a lot to circumvent using trig math.
There are 2 problems...
Change line 35 to var w=h*Math.sin(30);. The 30 here matches the this.theta / 4 in the Cube getWidthmethod since this.theta equals 120.
Use the following code to generate the position of your new cube. You don't need Math.Pi. You needed to use both the cube width and height in your calculation.
var cube = new Cube(
origin.x+ -j*w - i*h,
origin.y + -j*w/2 + i*h/2,
h);
Alright I found the solution!
It's really simple - I was using degrees instead of radians.
function Cube(x, y, h) {
this.x = x
this.y = y
this.h = h;
this.theta = 120*Math.PI/180;
this.getWidth = function () {
return (this.h * Math.sin(this.theta / 2));
};
this.width = this.getWidth();
this.getCorner = function () {
return (this.h / 2);
};
this.corner = this.getCorner();
}
So apparently Javascript trig functions use Radians, so that's one problem.
Next fix I made was to the offset of each point in the cube. It doesn't need one! (o.O idk why. But whatever it works. I left the old code just in case I discover why later on).
function draw() {
var canvas = document.getElementById("canvas");
var ctx = canvas.getContext("2d");
ctx.fillStyle = "#000";
ctx.fillRect(0, 0, canvas.width, canvas.height); // Draw a black canvas
var h = 32;
var width = Math.sin(60*Math.PI/180);
var w = h*width;
var row = 9; // column and row will always be same (to make cube)
var column = row;
var area = row * column;
var height = 1;
row--;
column--;
height--;
var origin = {
x: canvas.width / 2,
y: (canvas.height / 2) - (h * column/2) + height*h
};
var offset = Math.sqrt(3)/2;
offset = 1;
for (var i = 0; i <= row; i++) {
for (var j = 0; j <= column; j++) {
for (var k = 0; k <= height; k++) {
var cube = new Cube(
origin.x + (j * -w * offset) +
(i * w * offset)
, origin.y + (j * (h / 2) * offset) +
(i * (h / 2) * offset) +
(-k*h*offset), h);
var cubes = {};
cubes[i+j+k] = cube; // Store to array
if (j == column) {
drawCube(2, cube);
}
if (i == row) {
drawCube(1, cube);
}
if (k == height) {
drawCube(0,cube);
}
}
}
}
}
See the full Jsfiddle here: http://jsfiddle.net/joshlalonde/vtfyj/41/
I'm messing around with some Mandlebrot set stuff because I think the pictures it produces are pretty. I thought I might try to tackle the problem of drawing one in javascript to see what I could do. I looked at a couple algorithms, namely:
http://library.thinkquest.org/26242/full/progs/a2.html
Which I translated into this:
drawGraph: function(canvas,resolution,iterations,colors,coefficent){
var context = canvas.getContext('2d');
for(var m = 0; m < resolution.x; m++){
for(var n = 0; n < resolution.y; n++){
var x = m,
x2 = x*x,
y = n,
y2 = y*y;
var i;
for(i = 1; i < iterations; i++){
if(x2 + y2 > 4) break;
var new_x = x2 - y2 + coefficent.a;
var new_y = 2*x*y + coefficent.b;
x = new_x;
y = new_y;
}
var color = i % colors;
DrawUtils.drawPoint(context,m,n,color);
}
}
}
Which essentially draws a box of one color.
Then I tried this one:
http://en.wikipedia.org/wiki/Mandelbrot_set#Escape_time_algorithm
Which I translated into this:
drawGraph: function(canvas,resolution,iterations,colors,coefficent){
var context = canvas.getContext('2d');
for(var m = 0; m < resolution.x; m++){
for(var n = 0; n < resolution.y; n++){
var x = 0,
y = 0,
x0 = ((m/resolution.x) * 3.5) - 2.5,
y0 = ((n/resolution.y) * 2) - 1;
var i = 0;
while(x*x + y*y < 4 && i < iterations){
var x_temp = x*x - y*y + x0;
y = 2*x*y + y0;
x = x_temp;
i++;
}
var color = 0;
if(x*x + y*y >= 4){
color = i % colors;
}
DrawUtils.drawPoint(context,m,n,color);
}
}
}
Which produces a black box. The wording in the algorithm kind of confused me though since it said x0 and y0 scaled are factors of the pixel, but then after the algorithm, it says the coefficient c = x0 + iy0; so, does that mean I don't pass a predetermined coefficient into the function?
For most of these tests I was using the coefficient 0.25 + 0i, but I tried others that produced the exact same results.
What am I doing wrong here?
First point: you need to be clear about the difference between Julia sets and the Mandelbrot set. Both are insights into the behaviour of f(z) = z^2 + c under iteration, but from different perspectives.
For a Julia set, we fix c and make a plot of how different initial zs behave
For the Mandelbrot set, we make a plot of how the same initial z = 0 behaves for different cs.
With that addressed...
For your first code (which tries to draw the a Julia set for the c in coefficient), your translation from the BASIC in the first page you link to is not quite right. Where that has
‘ run through every point on the screen, setting
‘ m and n to the coordinates
FOR m = x_minimum TO x_maximum STEP x_resolution
FOR n = y_minimum TO y_maximum STEP y_resolution
‘ the initial z value is the current pixel,
‘ so x and y have to be set to m and n
x = m: y = n
you have
for(var m = 0; m < resolution.x; m++){
for(var n = 0; n < resolution.y; n++){
which is close, except for the crucial point that you are not taking any steps to implement STEP x_resolution. Your m is an integer that runs from 0 to resolution.x - 1 in steps of 1; and your x is set to m.
So instead of looking at the complex plane from say -2-2i to 2+2i (a decent viewport for seeing a Julia set), you are instead looking at the complex plane from 0 to resolution.x + resolution.y i, which will have at most a few pixels set in its lower-left corner.
The second code (which attempts to draw a Mandelbrot set) does have code to scale to a correct range, and I can't immediately see what's going wrong - I would debug and see if m/resolution.x is always 0, as #user973572 suggests may be the problem.
In your first example, I think you forgot to update x2 and y2 so they are always the same value. You need to update x2 and y2 before checking if the sum is greater than 4. Something like
for(i = 1; i < iterations; i++){
x2 = x*x,
y2 = y*y
if(x2 + y2 > 4) break;
which is probably wrong because I know nothing about javascript.