I am trying to compute the gradient of a loss, with relation to a network's trainable weights using Tensorflow.js in order to apply these gradients to my network's weight. In python, this is easily done using the tf.gradients() functions, which takes two minimum inputs representing dx and dy.
However, I am not able to reproduce the behavior in Tensorflow.js. I am not sure wether my understanding of the gradient of the loss w.r.t the weights is wrong, or if my code contains mistakes.
I have spent some time analysing the core code of the tfjs-node package to understand how it is done when we call the function tf.model.fit(), but with little success so far.
let model = build_model(); //Two stacked dense layers followed by two parallel dense layers for the output
let loss = compute_loss(...); //This function returns a tf.Tensor of shape [1] containing the mean loss for the batch.
const f = () => loss;
const grad = tf.variableGrads(f);
grad(model.getWeights());
The model.getWeights() function returns an array of tf.variable(), so I assumed the function would compute dL/dW for each layer, which I could apply later to my network's weights, however, that's not quite the case as I get this error :
Error: Cannot compute gradient of y=f(x) with respect to x. Make sure that the f you passed encloses all operations that lead from x to y.
I don't quite understand what does this error means.
How am I supposed to compute the gradient (analog to tf.gradients() in Python) of the loss using Tensorflow.js then ?
Edit :
This is the function computing the loss :
function compute_loss(done, new_state, memory, agent, gamma=0.99) {
let reward_sum = 0.;
if(done) {
reward_sum = 0.;
} else {
reward_sum = agent.call(tf.oneHot(new_state, 12).reshape([1, 9, 12]))
.values.flatten().get(0);
}
let discounted_rewards = [];
let memory_reward_rev = memory.rewards;
for(let reward of memory_reward_rev.reverse()) {
reward_sum = reward + gamma * reward_sum;
discounted_rewards.push(reward_sum);
}
discounted_rewards.reverse();
let onehot_states = [];
for(let state of memory.states) {
onehot_states.push(tf.oneHot(state, 12));
}
let init_onehot = onehot_states[0];
for(let i=1; i<onehot_states.length;i++) {
init_onehot = init_onehot.concat(onehot_states[i]);
}
let log_val = agent.call(
init_onehot.reshape([memory.states.length, 9, 12])
);
let disc_reward_tensor = tf.tensor(discounted_rewards);
let advantage = disc_reward_tensor.reshapeAs(log_val.values).sub(log_val.values);
let value_loss = advantage.square();
log_val.values.print();
let policy = tf.softmax(log_val.logits);
let logits_cpy = log_val.logits.clone();
let entropy = policy.mul(logits_cpy.mul(tf.scalar(-1)));
entropy = entropy.sum();
let memory_actions = [];
for(let i=0; i< memory.actions.length; i++) {
memory_actions.push(new Array(2000).fill(0));
memory_actions[i][memory.actions[i]] = 1;
}
memory_actions = tf.tensor(memory_actions);
let policy_loss = tf.losses.softmaxCrossEntropy(memory_actions.reshape([memory.actions.length, 2000]), log_val.logits);
let value_loss_copy = value_loss.clone();
let entropy_mul = (entropy.mul(tf.scalar(0.01))).mul(tf.scalar(-1));
let total_loss_1 = value_loss_copy.mul(tf.scalar(0.5, dtype='float32'));
let total_loss_2 = total_loss_1.add(policy_loss);
let total_loss = total_loss_2.add(entropy_mul);
total_loss.print();
return total_loss.mean();
}
EDIT 2:
I managed to use the compute_loss as the loss function specified on model.compile(). But then, it is required that it takes only two inputs (predictions, labels), so it's not working out for me, as I want to input multiple parameters.
I am trully lost on the matter.
The error says it all.
Your issue has to do with tf.variableGrads. loss should be a scalar computed using all available tf tensors operators. loss should not return a tensor as indicated in your question.
Here is an example of what loss should be:
const a = tf.variable(tf.tensor1d([3, 4]));
const b = tf.variable(tf.tensor1d([5, 6]));
const x = tf.tensor1d([1, 2]);
const f = () => a.mul(x.square()).add(b.mul(x)).sum(); // f is a function
// df/da = x ^ 2, df/db = x
const {value, grads} = tf.variableGrads(f); // gradient of f as respect of each variable
Object.keys(grads).forEach(varName => grads[varName].print());
/!\ Notice that the gradient is calculated as respect of variables created using tf.variable
Update:
You're not computing the gradients as it should be. Here is the fix.
function compute_loss(done, new_state, memory, agent, gamma=0.99) {
const f = () => { let reward_sum = 0.;
if(done) {
reward_sum = 0.;
} else {
reward_sum = agent.call(tf.oneHot(new_state, 12).reshape([1, 9, 12]))
.values.flatten().get(0);
}
let discounted_rewards = [];
let memory_reward_rev = memory.rewards;
for(let reward of memory_reward_rev.reverse()) {
reward_sum = reward + gamma * reward_sum;
discounted_rewards.push(reward_sum);
}
discounted_rewards.reverse();
let onehot_states = [];
for(let state of memory.states) {
onehot_states.push(tf.oneHot(state, 12));
}
let init_onehot = onehot_states[0];
for(let i=1; i<onehot_states.length;i++) {
init_onehot = init_onehot.concat(onehot_states[i]);
}
let log_val = agent.call(
init_onehot.reshape([memory.states.length, 9, 12])
);
let disc_reward_tensor = tf.tensor(discounted_rewards);
let advantage = disc_reward_tensor.reshapeAs(log_val.values).sub(log_val.values);
let value_loss = advantage.square();
log_val.values.print();
let policy = tf.softmax(log_val.logits);
let logits_cpy = log_val.logits.clone();
let entropy = policy.mul(logits_cpy.mul(tf.scalar(-1)));
entropy = entropy.sum();
let memory_actions = [];
for(let i=0; i< memory.actions.length; i++) {
memory_actions.push(new Array(2000).fill(0));
memory_actions[i][memory.actions[i]] = 1;
}
memory_actions = tf.tensor(memory_actions);
let policy_loss = tf.losses.softmaxCrossEntropy(memory_actions.reshape([memory.actions.length, 2000]), log_val.logits);
let value_loss_copy = value_loss.clone();
let entropy_mul = (entropy.mul(tf.scalar(0.01))).mul(tf.scalar(-1));
let total_loss_1 = value_loss_copy.mul(tf.scalar(0.5, dtype='float32'));
let total_loss_2 = total_loss_1.add(policy_loss);
let total_loss = total_loss_2.add(entropy_mul);
total_loss.print();
return total_loss.mean().asScalar();
}
return tf.variableGrads(f);
}
Notice that you can quickly run into a memory consumption issue. It will advisable to surround the function differentiated with tf.tidy to dispose of the tensors.
Related
I was trying to create sliding window on a real time signal.
For now, I have just simulated real time data getting pushed into a signal array. I am trying to capture a part of the real time signal using sliding window method in a buffer.
The buffer part is in red, and signal is in blue.
Here is the visualization in P5JS Sketch Link
Just to implement the code and visualize the output, I have used p5js. Can anyone please check my code and tell me if I am doing it correctly?
Here is my code --
let sampleLength = 0
let signal = []
let buffer = []
let windowSize = 50
let overlappingSize = 20
// let toggle = false
let t = 0
let pos = 0
function setup() {
createCanvas(500, 400)
sampleLength = width
// create buffer filled with zero
windowSize = floor(sampleLength/2)
for(let i=0;i<windowSize;i++){
buffer.push(100)
}
// setInterval(()=>{
// toggle=!toggle
// },100);
}
function draw() {
background(240);
translate(0,height>>1)
drawWave(buffer,0.5,'red')
drawWave(signal,1,'blue')
generateSignal()
if(pos>(windowSize-overlappingSize)){
buffer = signal.slice(0,windowSize)
pos= 0
}else{
pos++
}
}
function generateSignal(){
if(signal.length<sampleLength){
signal.push(100*sin(2*radians(t)))
t++;
}else{
signal.shift()
}
}
function drawWave(arr,scale,color){
stroke(color);
noFill()
beginShape()
for(let i=0;i<arr.length-1;i++){
vertex(i,arr[i]*scale)
}
endShape()
}
Edit: New example has been added -
As an example without visualization, it seems what below code does is correct.
const arr = [1,2,3,4,5,6,7,8]
const bufferSize = 8
var buffr = []
const overlap = 2
const slideStep = bufferSize-overlap
let i = 0
while(i<32){
buffr = arr.slice(0+i*slideStep,bufferSize+i*slideStep);
console.log(i, buffr);
pushNewValue()
i++
}
function pushNewValue(){
for(let k=0;k<8;k++){
arr.push(arr[arr.length-1]+k)
}
}
Edit 2: Update the code for sliding window and created a class which is applied in a example. Please check the link. Let me know if that is a correct code.
function DSP( windowSize, overlap){
this.arr = Array(windowSize)
.fill()
.map(() => 0);
this.windowSize = windowSize
this.slideStep = windowSize-overlap
}
DSP.prototype.slideWindow = function(currVal, callback){
if (this.arr.length == this.windowSize) {
callback(this.arr.slice())
let del = this.slideStep;
while (del--) {
this.arr.shift();
}
}else{
this.arr.push(currVal)
}
}
I am learning javascript and Ive stumbled upon issue that I do not understand. Could somebody explain to me why in method compareDNA I need to use parentheses while using this.dna and in the previous method it works just fine?
// Returns a random DNA base
const returnRandBase = () => {
const dnaBases = ['A', 'T', 'C', 'G'];
return dnaBases[Math.floor(Math.random() * 4)];
};
// Returns a random single stand of DNA containing 15 bases
const mockUpStrand = () => {
const newStrand = [];
for (let i = 0; i < 15; i++) {
newStrand.push(returnRandBase());
}
return newStrand;
};
function pAequorFactory(specimenNum, dna){
return {
specimenNum,
dna,
mutate(){
let i = Math.floor(Math.random() * 15)
let newGene = returnRandBase()
while (this.dna[i] === newGene){
newGene = returnRandBase()
}
this.dna[i] = newGene
return this.dna
},
compareDNA(object){
let counter = 0
for(let i = 0; i < this.dna().length; i++){
if(this.dna()[i] === object.dna()[i]){
counter++
}
}
let percentage = counter / this.dna().length * 100
return `Specimen number ${this.specimenNum} and specimen number ${object.specimenNum} have ${percentage}% of DNA in common.`
},
}
}
let aligator = pAequorFactory(1, mockUpStrand)
let dog = pAequorFactory(2, mockUpStrand)
console.log(aligator.compareDNA(dog))
console.log(dog.dna().length)
The problem is that the dna that is passed as an argument is a function, so it becomes a method of the returned object, and needs to be called with .dna(). However, this looks like a mistake - actually an array should have been passed:
let aligator = pAequorFactory(1, mockUpStrand())
// ^^
let dog = pAequorFactory(2, mockUpStrand())
// ^^
Then you can access .dna[i] or .dna.length as normal.
If you don't do that, dog.dna() returns a different DNA every time, which doesn't make sense.
using this.dna and in the previous method it works just fine?
Actually, it doesn't. dog.mutate() does return a function with a single integer property. It's supposed to return an array really.
I am having an issue with my merge sort visualizer.
My program has no issues visualizing bubble sort or quick sort, as I can do the swapping operation of css property values in-place, but I am having major issues trying to get merge sort to work properly. The issue arises when I try to update a css property on the dom, it causes the sort to not function.
I have tried passing in copies of the data I wish to sort, and all sorts of weird things I could think of to make it work. I am currently trying to sort by the css property 'maxWidth'. I use that to display how large a div element is in the html file and then visualize the sort from there.
My latest thought has been to set all the div elements to have another css property equal to the maxWidth (I am using fontSize as it does not affect my program) and then sorting based on fontSize, allowing me in theory to change the maxWidth properties of the divs without affecting merge sorts algorithm.
I am including my entire js file as I hope reading my correctly working bubble sort or quick sort functions can help you see what I am trying to achieve. Thank you so much for taking the time to read this and offer any help!
Important Note: I am not trying to visualize the individual steps of merge sort yet because I am unable to update the final result to the html page without affecting the merge sort algorithm. According to console logs, my merge sort algorithm does indeed work, I just can't update the DOM without messing it up. Once I can do that, I will turn it into an asynchronous function using async and await like I previously did with bubble and quick sort.
/********* Generate and Store Divs to be Sorted *************/
const generateSortingDivs = (numOfDivs) => {
const divContainer = document.querySelector('.div-container');
let html = '';
for (let i = 0; i < numOfDivs; i++) {
let r = Math.floor(Math.random() * 100);
html += `<div class='sorting-div' id='id-${i}' style='max-width: ${r}%'> </div>`;
}
divContainer.innerHTML = html;
for(let i = 0; i < numOfDivs; i++) {
let x = document.getElementById('id-' + i);
x.style.fontSize = x.style.maxWidth;
}
}
const storeSortingDivs = () => {
const divContainer = document.querySelector('.div-container');
let divCollection = [];
const numOfDivs = divContainer.childElementCount;
for(let i=0; i<numOfDivs; i++) {
let div = document.getElementById('id-' + i);
divCollection.push(div);
}
return divCollection;
}
/********** SLEEP FUNCTION ************/
//Used to allow asynchronous visualizations of synchronous tasks
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
/******* SWAP FUNCTIONS *********/
//Used for Testing Algorithm before Animating Visualization
const syncSwap = (div1, div2) => {
let tmp = div1.style.maxWidth;
div1.style.maxWidth = div2.style.maxWidth;
div2.style.maxWidth = tmp;
}
async function asyncSwap(div1, div2) {
await sleep(50);
let tmp = div1.style.maxWidth;
div1.style.maxWidth = div2.style.maxWidth;
div2.style.maxWidth = tmp;
}
const swapDivs = (smallerDiv, biggerDiv) => {
return new Promise(resolve => {
setTimeout(() => {
let tmp = smallerDiv.style.maxWidth;
smallerDiv.style.maxWidth = biggerDiv.style.maxWidth;
biggerDiv.style.maxWidth = tmp;
resolve();
}, 50);
});
}
/****************************************/
/*********** SORTING ALGO'S *************/
/****************************************/
/******* BUBBLE SORT ***********/
async function bubbleSort(divCollection) {
displayBubbleSortInfo();
const len = divCollection.length;
for(let i=0; i<len; i++) {
for(let j=0; j<len-i-1; j++) {
divCollection[j].style.backgroundColor = "#FF4949";
divCollection[j+1].style.backgroundColor = "#FF4949";
let numDiv1 = parseInt(divCollection[j].style.maxWidth);
let numDiv2 = parseInt(divCollection[j+1].style.maxWidth);
let div1 = divCollection[j];
let div2 = divCollection[j+1];
if(numDiv1 > numDiv2) {
await swapDivs(div2, div1);
}
divCollection[j].style.backgroundColor = "darkcyan";
divCollection[j+1].style.backgroundColor = "darkcyan";
}
divCollection[len - i - 1].style.backgroundColor = 'black';
}
}
function displayBubbleSortInfo(){
const infoDiv = document.querySelector('.algo-info');
let html = `<h1>Bubble Sort Visualizer</h1>`;
html += `<h2>Time Complexity: O(n^2)</h2>`;
html += `<h3>Space Complexity: O(1)</h3>`;
html += `<p>This sorting algorithm loops through the array and continues to push the
largest found element into the last position, also pushing the last available
position down by one on each iteration. It is guaranteed to run in exactly
O(n^2) time because it is a nested loop that runs completely through.</p>`;
infoDiv.innerHTML = html;
}
/****** QUICK SORT ********/
async function quickSort(divCollection, start, end) {
if(start >= end) return;
let partitionIndex = await partition(divCollection, start, end);
await Promise.all([quickSort(divCollection, start, partitionIndex - 1), quickSort(divCollection, partitionIndex + 1, end)]);
}
/* This function takes last element as pivot, places
the pivot element at its correct position in sorted
array, and places all smaller (smaller than pivot)
to left of pivot and all greater elements to right
of pivot */
async function partition(divCollection, start, end) {
let pivotIndex = start;
let pivotValue = parseInt(divCollection[end].style.maxWidth);
for(let i = start; i < end; i++) {
if(parseInt(divCollection[i].style.maxWidth) < pivotValue) {
await asyncSwap(divCollection[i], divCollection[pivotIndex]);
pivotIndex++;
}
}
await asyncSwap(divCollection[pivotIndex], divCollection[end]);
return pivotIndex;
}
function displayQuickSortInfo(){
const infoDiv = document.querySelector('.algo-info');
let html = `<h1>Quick Sort Visualizer</h1>`;
html += `<h2>Time Complexity: O(n log n)</h2>`;
html += `<h3>Space Complexity: O(log n)</h3>`;
html += `<p>This sorting algorithm uses the idea of a partition to sort
each iteration recursively. You can implement quick sort
in a variety of manners based on the method in which you
pick your "pivot" value to partition the array. In this
visualization, I implemented the method that chooses the
last element of the array as the pivot value. You could
also choose the first value, the middle value, or the median
value based on the first, middle, and last values.</p>`;
infoDiv.innerHTML = html;
}
/* Merge Sort does not sort in place, and thus we have to be
* clever when implementing it and also editing the css style
* of our divs to show the visualization of how the algorithm
* works. My method is to store a copy of the divs, that way
* I can use one to be sorted by merge sort, and the other to
* change the css style property to show the visualization.
* Unlike Quick Sort and Bubble Sort, we are not swapping
* elements when sorting, instead we are merging entire
* arrays together as the name implies. */
function mergeSort(divCollection) {
if(divCollection.length < 2) return divCollection;
let middleIndex = Math.floor(divCollection.length / 2);
let left = divCollection.slice(0, middleIndex);
let right = divCollection.slice(middleIndex);
return merge(mergeSort(left), mergeSort(right));
}
function merge(left, right) {
let mergedCollection = [];
while(left.length && right.length) {
if(parseInt(left[0].style.fontSize) < parseInt(right[0].style.fontSize || right.length === 0)) {
let el = left.shift();
mergedCollection.push(el);
} else {
let el = right.shift();
mergedCollection.push(el);
}
}
let res = mergedCollection.concat(left.slice().concat(right.slice()));
return res;
}
/***** INITIALIZATION FUNCTION *******/
generateSortingDivs(10);
let divs = storeSortingDivs();
let copyDivs = [...divs];
console.log('Original State: ')
console.log(divs);
//bubbleSort(divs);
//displayQuickSortInfo();
//quickSort(divs, 0, divs.length-1);
let x = mergeSort(copyDivs);
console.log('Sorted: ');
console.log(x);
I want to fit a nonlinear multivariable equation using TensorFlow. The equation is given below. The parameters to fit are a0, a1, and a2. The independent variables are S and R, while F is the dependent variable. The corresponding data for S,R,F are provided in the code below, as Sdata, Rdata, and Fdata respectively.
F = a0 + a1*S + a2*R
I
const Sdata = tf.tensor1d([13.8,13.8,20.2,12.1,14.1,29.4,13.7,16.6,18.9,15.5]);
const Fdata = tf.tensor1d([46.7,130.7,78.1,72.2,40.1,78.6,57.4,170.7,80.2,45.2]);
const Rdata = tf.tensor1d([1.5,4.5,2.5,3.0,3.5,3.0,2.5,3.0,3.0,2.5])
const a0 = tf.scalar(Math.random()).variable();
const a1 = tf.scalar(Math.random()).variable();
const a2 = tf.scalar(Math.random()).variable();
const fun = (r,s) => a2.mul(r).add(a1.mul(s)).add(a0)
const cost = (pred, label) => pred.sub(label).square().mean();
const learningRate = 0.01;
const optimizer = tf.train.sgd(learningRate);
// Train the model.
for (let i = 0; i < 800; i++) {
optimizer.minimize(() => cost(fun(Rdata,Sdata), Fdata));
}
As shown in my code, I assumed the function "fun" can take two independent variable. Instead of getting a0 = -6.6986, a1 = 0.8005, and a2 = 25.2523, I am getting NaNs.
Does it mean it is not possible to fit multivariable functions in tensorflow.js? I don't think so. I will appreciate any insight to this.
Because of the learning rate, the model is oscillating to find the best parameters. Actually, the parameters keep on increasing to Infinity.
Tuning the learning rate will allow the model to find the best parameters. In this case 0.001 seems to give good result. If you want to improve the accuracy of the model, you can consider normalizing all your input data to be of the same magnitude order - between 0 and 1
const Sdata = tf.tensor1d([13.8,13.8,20.2,12.1,14.1,29.4,13.7,16.6,18.9,15.5]);
const Fdata = tf.tensor1d([46.7,130.7,78.1,72.2,40.1,78.6,57.4,170.7,80.2,45.2]);
const Rdata = tf.tensor1d([1.5,4.5,2.5,3.0,3.5,3.0,2.5,3.0,3.0,2.5])
const a0 = tf.scalar(Math.random()).variable();
const a1 = tf.scalar(Math.random()).variable();
const a2 = tf.scalar(Math.random()).variable();
const fun = (r,s) => a2.mul(r).add(a1.mul(s)).add(a0)
const cost = (pred, label) => pred.sub(label).square().mean();
const learningRate = 0.001;
const optimizer = tf.train.sgd(learningRate);
// Train the model.
for (let i = 0; i < 800; i++) {
console.log("training")
optimizer.minimize(() => cost(fun(Rdata,Sdata), Fdata));
}
console.log(`a: ${a0.dataSync()}, b: ${a1.dataSync()}, c: ${a2.dataSync()}`);
const preds = fun(Rdata,Sdata).dataSync();
preds.forEach((pred, i) => {
console.log(`x: ${i}, pred: ${pred}`);
});
<html>
<head>
<!-- Load TensorFlow.js -->
<script src="https://cdn.jsdelivr.net/npm/#tensorflow/tfjs#latest"> </script>
</head>
<body>
</body>
</html>
I am currently doing some 3d modeling using babylonjs. I need to create a pressure map from given pressure at specific points. I am doing that using IDW. However this means that even with my map being a size of 70x90 grid requires me to have an array of 25200 (4 rgba values for each pixel) entries. Then this buffer is passed to a RawTexture for assigning it to a material, that is overlaid on the object
I am using a web worker, because I have to update the pressure values every 100ms and I don't want to block the main thread.The issue occurs when I am return that array (created in calculate function) from a service worker.
For some reason the memory usage just keeps going up, without stopping. It eventually goes up to around 1.5 gigabytes and I have to kill it.
The question : Is there any way to prevent this and what could be causing such high memory usage?
Worker:
// #flow
import { find, propEq, both } from 'ramda';
import { colorFromValue } from './color';
import { inverseDistance, distanceValues } from './math';
const findPoint = (x: number, y: number) =>
find(both(propEq('x', x), propEq('y', y)));
const distanceDict = {};
/* eslint-disable */
function calculate(options: Object, pList: Array<*>) {
const points = pList || [];
const { height, width } = options;
const gridWidth = width * 4;
const grid = new Uint8Array(options.width * options.height * 4);
for (let y = 0; y < height; y += 1) {
const rW = y * gridWidth;
for (let i = 0; i < gridWidth; i += 4) {
const index = i + rW;
const x = i / 4;
const dictKey = `${x}--${y}`;
let bottoms = distanceDict[dictKey];
if (bottoms === undefined) {
bottoms = distanceValues(points, x, y);
distanceDict[dictKey] = bottoms;
}
const point = findPoint(x, y)(points);
const value = point !== undefined && point !== null ?
point.value : inverseDistance(points, bottoms);
const color = colorFromValue(value);
grid[index] = color[0];
grid[index + 1] = color[1];
grid[index + 2] = color[2];
grid[index + 3] = 255;
}
}
return grid;
}
self.onmessage = (e) => {
const { points, options } = e.data;
const grid = calculate(options, points);
self.postMessage(grid.buffer, [grid.buffer]);
};
Painting:
modifyNodes = (points: Array<*>) => new Promise((res, rej) => {
this.worker.onmessage = (e) => {
this._texture.update(new Uint8Array(e.data));
res();
}
const data = {
options: this._options,
points,
};
this.worker.postMessage(data);
})
So it seems the issue was in the colorFromValue function that was memoized. Because the values had quite few decimal points it could create up to 9! new entries into cache, so it drove up the memory usage...