Two heatmaps on top of each other with opacity with Plotly.JS - javascript

This bounty has ended. Answers to this question are eligible for a +100 reputation bounty. Bounty grace period ends in 21 hours.
Basj is looking for a canonical answer:
The current answer is useful. One detail: how to have one of the layers (for example, the background layer) as an RGB image instead of just a z-axis heatmap?
I'm trying to port this answer to a 100% Plotly.JS solution.
TL;DR : how to have two heatmaps on top of eacher with an opacity slider, with Plotly.JS (no Python)?
Beginning of solution, but how to add the second trace?
const z = [];
for (let i = 0; i < 500; i++)
z.push(Array.from({ length: 600 }, () => Math.floor(Math.random() * 100)));
const data = [{ z: z, colorscale: "YlGnBu", type: "heatmap" }];
const steps = [];
for (let i = 0; i <= 100; i++)
steps.push({ label: i + "%", execute: true, method: "restyle", args: [{ opacity: i / 100 }] });
const layout = { sliders: [{ name: "slider", steps: steps, active: 100 }] };
Plotly.newPlot("graph", data, layout);
<script src="https://cdn.plot.ly/plotly-2.16.2.min.js"></script>
<div id="graph"></div>
For reference: original Python solution:
from PIL import Image
import plotly.graph_objects as go
import numpy as np
import scipy.misc
imgA = scipy.misc.face()
imgB = Image.fromarray(np.random.random(imgA.shape[:2])*255).convert('RGB')
fig = go.Figure([
go.Image(name='raccoon', z=imgA, opacity=1), # trace 0
go.Image(name='noise', z=imgB, opacity=0.5) # trace 1
])
slider = {
'active': 50,
'currentvalue': {'prefix': 'Noise: '},
'steps': [{
'value': step/100,
'label': f'{step}%',
'visible': True,
'execute': True,
'method': 'restyle',
'args': [{'opacity': step/100}, [1]] # apply to trace [1] only
} for step in range(101)]
}
fig.update_layout(sliders=[slider])
fig.show(renderer='browser')

The second trace goes into the data array as well. The thing to note is that indexing matters : the trace at index 1 is drawn above the trace at index 0, and so on.
For the slider configuration, it should be the same as in python : each step change triggers the same 'restyle' method with the same arguments, ie. Plotly.restyle(graphDiv, ...args), that is, with args such that the method call matches the signature :
Plotly.restyle(graphDiv, update [, traceIndices])
Now, the most important thing is which trace (traceIndices) the slider should target, that is, which index or which name for explicitly named traces (default is all if I'm not wrong), but again here it doesn't change between Python and Javascript.
Here is a full example (play around with it on codepen.io) :
// Random z data
const w = {length: 600};
const h = {length: 400};
const z0 = Array.from(h, () => Array.from(w, () => Math.floor(Math.random() * 100)));
const z1 = Array.from(h, () => Array.from(w, () => Math.floor(Math.random() * 100)));
// Initial opacity for the trace 'above'
const op_init = 0.5;
const data = [
// Nb. Trace 1 drawn on top of trace 0
{type: 'heatmap', z: z0, colorscale: 'Greys'}, // trace 0
{type: 'heatmap', z: z1, colorscale: 'Cividis', opacity: op_init} // trace 1
];
// Steps for the opacity slider
const steps = [];
const n_steps = 100; // number of steps above step 0
for (let i = 0; i <= n_steps; i++) {
steps.push({
label: i + '%',
execute: true,
method: 'restyle',
args: [{
opacity: i/n_steps
}, [1]] // <- Nb. this applies only to trace 1
});
}
const layout = {
width: 600,
sliders: [{
steps: steps,
active: Math.round(op_init * n_steps), // slider default matches op_init
pad: {t: 30},
currentvalue: {prefix: 'opacity: '}
}]
};
Plotly.newPlot('plot', data, layout);
Image vs Heatmap
A Heatmap works only with single channel data (individual value-to-color mappings according to a given colorscale).
When working with rgb (or rgba, rgba256, hsl, hsla), one has to use the image type. The difference is that z must be a 2-dimensional array in which each element is an array of 3 or 4 numbers representing a color (the colormodel should be set accordingly).
For example, setting an rgb image made of noise as the background layer :
const z0 = Array.from(h, () => Array.from(w, () => ['r', 'g', 'b'].map(() => Math.floor(Math.random() * 255)) ));
// ...
const data = [
{type: 'image', z: z0, colormodel: 'rgb'}, // trace 0
{type: 'heatmap', z: z1, colorscale: 'Cividis', opacity: op_init} // trace 1
];
Here a second example where we have an rgb[a] image (DOM object img) and its pixel data represented as a 1-dimensional Uint8Array (uint8Arr), which need to be converted in 2d :
const z0 = [];
const nChannels = uint8Arr.length / img.width / img.height;
const chunkSize = uint8Arr.length / img.height;
const z0_model = nChannels === 4 ? 'rgba' : 'rgb';
for (let i = 0; i < uint8Arr.length; i += chunkSize) {
const chunk = uint8Arr.slice(i, i + chunkSize);
const row = [];
for (let j = 0; j < chunk.length; j += nChannels)
row.push(chunk.slice(j, j + nChannels));
z0.push(row);
}
// ...
const data = [
{type: 'image', z: z0, colormodel: z0_model}, // trace 0
{type: 'heatmap', z: z1, colorscale: 'Cividis', opacity: op_init} // trace 1
];
Nb. When you plot an image, the yaxis is automatically reversed (unless specified otherwise, which would display the image upside down). This affects the orientation of the heatmap y-labels, as they're on the same plot, but only the labels not the data.
Here is the layout settings ensuring that both traces share the same aspect ratio and that the image is oriented correctly :
const layout = {
// ...
xaxis: {anchor: 'y', scaleanchor: 'y', constrain: 'domain'},
yaxis: {anchor: 'x', autorange: 'reversed', constrain: 'domain'},
};

Related

Slow rendering of image + slider with Plotly JS

I'm trying to plot an image/heatmap with a slider that will change the opacity (of the heatmap), and a second slider that will modify a custom parameter on each "onchange" event.
Once this image/heatmap is rendered, no computation should be done, and moving the sliders should be instantaneous. But from what I have tried, moving one slider is very slow (1 second lag between each position), and uses max CPU %.
I'm looking for a JS-only solution (no Python for this part).
How to make a faster slider rendering with Plotly JS?
var z = [], steps = [], i;
for (i = 0; i < 500; i++)
z.push(Array.from({length: 600}, () => Math.floor(Math.random() * 100)));
for (i = 0; i < 100; i++)
steps.push({ label: i, method: 'restyle', args: ['line.color', 'red']});
var data = [{z: z, colorscale: 'YlGnBu', type: 'heatmap'}];
var layout = {title: '', sliders: [{
pad: {t: 5},
len: 1,
x: 0,
currentvalue: {
xanchor: 'right',
prefix: 'i: ',
font: {
color: '#888',
size: 20
}
},
steps: steps
}]};
Plotly.newPlot('myDiv', data, layout);
<script src="https://cdn.plot.ly/plotly-2.16.2.min.js"></script>
<div id="myDiv"></div>
This is because you are using method: 'restyle' with the wrong args, ie. ['line.color', 'red'] the syntax is not correct and there is no line so I guess Plotly (without knowing what to restyle exactly) just redraws the whole plot whenever the slider moves, which is slow.
Basically, you can use the same slider configuration in javascript and in python for the same task (in the end the same Plotly.js slider component will be used).
For example, one can set the opacity of an image according to the slider's position, but for the changes to be applied instantly one needs to set the proper method and args in the slider' steps configuration, excactly as explained in this post :
steps.push({
label: i,
execute: true,
method: 'restyle',
args: [{opacity: i/100}]
});
Here is a full example with two sliders : one that changes the opacity of the heatmap and another one that doesn't touch the plot but only triggers a specific handler :
const z = [];
for (let i=0; i<500; i++) {
z.push(Array.from({length: 600}, () => Math.floor(Math.random() * 100)));
}
const data = [{z: z, colorscale: 'YlGnBu', type: 'heatmap'}];
// Steps for the heatmap opacity slider
const opacity_steps = [];
for (let i = 0; i <= 100; i++) {
opacity_steps.push({
label: i + '%',
execute: true,
method: 'restyle',
args: [{opacity: i/100}]
});
}
// Steps for the custom slider
const custom_steps = [];
for (let i = 50; i <= 200; i++) {
custom_steps.push({
label: i,
execute: false,
method: 'skip',
});
}
const layout = {
title: '',
sliders: [{
name: 'opacity_slider',
steps: opacity_steps,
active: 100,
pad: {t: 30},
currentvalue: {prefix: 'opacity: '}
}, {
name: 'custom_slider',
steps: custom_steps,
pad: {t: 120},
currentvalue: {prefix: 'i: '}
}]
};
Plotly.newPlot('graph', data, layout);
// Retrieve the graph div
const gd = document.getElementById('graph');
// Attach 'plotly_sliderchange' event listener to it (note that we can't specify
// which slider the handler is taking care of using a secondary selector)
gd.on('plotly_sliderchange', function(event) {
// ... so we have to distinguish between the two sliders here.
if (event.slider.name != 'custom_slider')
return;
const slider = event.slider; // the slider emitting the event (object)
const step = event.step; // the currently active step (object)
const prev = event.previousActive; // index of the previously active step
const value = step.value; // captain obvious was here
const index = step._index; // index of the current step
// ...
});
<script src="https://cdn.plot.ly/plotly-2.16.2.min.js"></script>
<div id="graph"></div>

Is there a way to fix a batch size mismatch between output from final model layer and input?

What I've done: I have a complete dataset of 898 labels with a total of ~55,000 images. For purposes of speed, I took 10 of those labels and about ~600 images to test the code below. I've tried changing the batchSize, modifying the data function, but to no avail.
Problem: Error: Batch size mismatch: output dense_Dense1 has 10; expected 500 based on input conv2d_Conv2D1_input.
Goal: Either change the final output of dense_Dense1 to have 500, or change the expected input of conv2d_Conv2D1_input to only 10.
Complete Code:
var tf = require('#tensorflow/tfjs');
var tfnode = require('#tensorflow/tfjs-node');
var fs = require(`fs`)
const numberOfClasses = 10;
const imageWidth = 500;
const imageHeight = 800;
const imageChannels = 3;
const batchSize = 3;
const epochsValue = 5;
const createImage = async (fileName) => {
const imageBuffer = await fs.readFileSync(fileName);
const image = await tfnode.node.decodeImage(imageBuffer);
return image;
}
const labelArray = indice => Array.from({length: numberOfClasses}, (_, k) => k === indice ? 1 : 0)
async function* data() {
for (i = 1; i < numberOfClasses+1; i++) {
for (x = 10; x < 40; x++) {
const feature = await createImage(`./images/${i}/${i}-${x}.png`) ;
const label = tf.tensor1d(labelArray(i))
yield {xs: feature, ys: label};
}
}
}
function onBatchEnd(batch, logs) {
console.log('Accuracy', logs.acc);
}
const main = async () => {
const model = tf.sequential();
model.add(tf.layers.conv2d({
inputShape: [imageWidth, imageHeight, imageChannels],
filters: 8,
kernelSize: 5,
padding: 'same',
activation: 'relu'
}));
model.add(tf.layers.maxPooling2d({
poolSize: 2,
strides: 2
}));
model.add(tf.layers.conv2d({
filters: 16,
kernelSize: 5,
padding: 'same',
activation: 'relu'
}));
model.add(tf.layers.maxPooling2d({
poolSize: 3,
strides: 3
}));
model.add(tf.layers.flatten());
model.add(tf.layers.dense({
units: numberOfClasses,
activation: 'softmax'
}));
model.compile({
optimizer: 'sgd',
loss: 'categoricalCrossentropy',
metrics: ['accuracy']
});
model.summary()
const ds = tf.data.generator(data);
model.fitDataset(ds, {
epochs: 5,
batchSize: 10,
callbacks: {onBatchEnd}
}).then(info => {
console.log('Final accuracy', info.history.acc);
});
}
main()
The error is self explanatory
Error: Batch size mismatch: output dense_Dense1 has 10; expected 500 based on input conv2d_Conv2D1_input.
There is a mismatch of shape between what the model expects and what the dataset has.
dense_Dense1 is the last layer and 10 the number of classes (model.summary() is handy to know the layers' name)
500 is the number of batches - at least that's what the model is getting with fitDataset(). Here is the confusion. 500 is the first axis size of the feature. feature is a 3d tensor with the first axis (image width) being 500. If the 3d tensor is used directly for prediction, the model consider that the image width is the number of batches and the features are 2d tensors. Therefore the feature should rather be a 4d tensor.
To go further, the model is not predicting on a single image at a time though. Since there is later a batch(10). After each iteration of the iterator data*, the features are stacked and after 10 iterations are used for predictions. It is handy because other operators can be used to reshuffle the batch and so on.
Here is the fix:
yield {xs: feature.expandDims(), ys: label.expandDims()};
Or even better you can batch the number of samples before fitting it to the model this way
const ds = tf.data.generator(data).batch(1);
I looked at the tensorflow javascript source code, and found this following snippet from the file https://github.com/tensorflow/tfjs/blob/550e2c7ced057309533f44f76e692fe28aa7a802/tfjs-layers/src/engine/training_dataset.ts on line 202, where you will see the following snippet.
for (let yIndex = 0; yIndex < flattenedYs.length; yIndex++) {
tfc.util.assert(
flattenedYs[yIndex].shape[0] === batchSize,
() => `Batch size mismatch: output ` +
`${model.outputNames[yIndex]} has ${
flattenedYs[yIndex].shape[0]}; ` +
`expected ${batchSize} based on input
${model.inputNames[0]}.`);
}
this is the only place in the entire codebase that generates this error which matches exactly to your error. Though I am not an expert in deep learning, but I can tell the problem is probably at y dimension of your data.
that snippet is coming from a function called standardizeDataIteratorOutput, which one of its forloop inside is trying to validate the y dimension of your dataset flattenedYs[yIndex].shape[0] === batchSize so in a nutshell, I am guessing, your batchSize is set to 10, which does not match the input Y size. you probably need to rearrange the image data to match that.

Failing to execute `connect` on `AudioNode`

I have this script in the header of an HTML file. It works fine with the exception of the line delay.connect(Tone.context.destination); midway down that theoretically adds layering of the sounds in the DuoSynth by starting one of them on a delay.
When it is not commented out, I get this error in the console: Uncaught TypeError: Failed to execute 'connect' on 'AudioNode': No function was found that matched the signature provided.
What am I doing wrong?
<script>
function makeSynth() {
let envelope = {
attack: 0.1,
release: 4,
releaseCurve: 'exponential'
};
let filterEnvelope = {
baseFrequency: 440,
octaves: 3,
attack: 0,
decay: 0,
release: 1000
};
return new Tone.DuoSynth({
harmonicity: .5,
detune : 5000,
volume: -20,
voice0: {
oscillator: { type: 'triangle' },
envelope,
filterEnvelope
},
voice1: {
oscillator: { type: 'sine' },
envelope,
filterEnvelope
},
vibratoRate: 0.5,
vibratoAmount: 0.1 // THIS IS GOOD TO MANIPULATE
});
}
let leftSynth = makeSynth();
let rightSynth = makeSynth();
let leftPanner = new Tone.Panner(-0.5).toMaster();
let rightPanner = new Tone.Panner(0.5).toMaster();
let echo = new Tone.FeedbackDelay('16n', 0.2);
let delay = Tone.context.createDelay(6.0); // Borrow the AudioContext from Tone.js
let delayFade = Tone.context.createGain();
delay.delayTime.value = 6.0;
delayFade.gain.value = 0.75;
leftSynth.connect(leftPanner);
rightSynth.connect(rightPanner);
leftPanner.connect(echo);
rightPanner.connect(echo);
// var c = canvas.getContext('2d');
echo.toMaster();
echo.connect(delay);
delay.connect(Tone.context.destination);
delay.connect(delayFade);
delayFade.connect(delay);
new Tone.Loop(time => {
leftSynth.triggerAttackRelease('C6', '1:2', time);
leftSynth.setNote('G5', '+0:2');
leftSynth.setNote('C4', '+0:4');
leftSynth.triggerAttackRelease('G5', '0:2', '+6:0');
leftSynth.triggerAttackRelease('E4', '0:2', '+11:2');
leftSynth.triggerAttackRelease('E5', '2:0', '+19:0');
leftSynth.setNote('G5', '+19:1:2');
leftSynth.setNote('A5', '+19:3:0');
leftSynth.setNote('G5', '+19:4:2');
}, '34m').start();
new Tone.Loop(time => {
// Trigger D4 after 5 measures and hold for 1 full measure + two 1/4 notes
rightSynth.triggerAttackRelease('G6', '1:2', '+5:0');
// Switch to E4 after one more measure
rightSynth.setNote('B6', '+6:0');
// Trigger B3 after 11 measures + two 1/4 notes + two 1/16 notes. Hold for one measure
rightSynth.triggerAttackRelease('D4', '1m', '+11:2:2');
// Switch to G3 after a 1/2 note more
rightSynth.setNote('G3', '+12:0:2');
// Trigger G4 after 23 measures + two 1/4 notes. Hold for a half note.
rightSynth.triggerAttackRelease('G4', '0:2', '+23:2');
}, '37m').start();
</script>

Spurious point is being added in the end C3 charts

I'm using C3 charts library to draw charts. I send data to the chart using two arrays, which are 'timeArray' and 'dataArray', one for the X-Axis and the other one for Y-Axis respectively. This simple logic was working fine.
Later I had to implement a change such that I had to take average of every three elements of an array and then make a new array and then plot the graph using averaged values.
I started facing a problem that a spurious point was being plotted on the graph. Whenever this error occurs, only one spurious point is added in the end. I've checked the arrays that are used to plot the graph, they do not have that spurious point. When I take the average of every three elements, I face this problem almost every time, however when I take average of 500 or 1000 points I face this error only sometimes.
As you can see in the code I have already tried removing the last point of the final array since the spurious point that was being added was always the last point in the chart. I've also tried changing the graph type, it did not help.
socket.on('get-avg-graph', function(data) {
// dataPoints = Points for Y-Axis
// mili = Points for X-Axis
var dataPoints = data.dataPoints;
var mili = data.mili;
var sumX = 0;
var sumY = 0;
var avgXGraph = 0;
var avgYGraph = 0;
var avgXArray = [];
var avgYArray = [];
for (var i = 0; i < dataPoints.length - 999; i++) {
for (var j = i; j < i + 999; j++) {
sumX = sumX + mili[j];
sumY = sumY + dataPoints[j];
}
if (sumY !== 0) {
avgXGraph = ( sumX / 1000 );
avgXArray.push(avgXGraph);
avgYGraph = ( sumY / 1000 );
avgYArray.push(avgYGraph);
sumX = 0;
sumY = 0;
avgXGraph = 0;
avgYGraph = 0;
}
}
io.emit('get-avg-graph-response', avgXArray, avgYArray);
});
socket.on('get-avg-graph-response', function(avgXArray, avgYArray) {
plot_X_axis = [];
plot_Y_axis = [];
drawChart();
avgXArray.splice( -1, 1);
avgYArray.splice( -1, 1);
plot_X_axis.push.apply(plot_X_axis, avgXArray);
plot_Y_axis.push.apply(plot_Y_axis, avgYArray);
drawChart();
});
function drawChart() {
var graphTitle = $("#test_type_show").val();
dataArray = [];
dataArray[0] = "PRESSURE";
dataArray.push.apply(dataArray, plot_Y_axis);
timeArray = [];
timeArray[0] = "TIME";
timeArray.push.apply(timeArray, plot_X_axis);
if (chart==null) {
chart = c3.generate({
bindto: '#chart1',
title: {
text: graphTitle
},
data: {
x: 'TIME',
columns: [
timeArray,
dataArray
],
type: 'spline'
},
axis: {
x: {show:false},
y: {show: true}
},
grid: {
x: {
show: true
},
y: {
show: true
}
},
point: {
show: false
}
});
} else {
chart.load({
x: 'TIME',
columns: [
timeArray,
dataArray
],
type: 'spline'
});
}
chart.internal.xAxis.g.attr('transform', "translate(0," + chart.internal.y(0) + ")");
chart.internal.yAxis.g.attr('transform', "translate(" + chart.internal.x(0) + ", 0)");
}
I expect the output of the code to be the actual graph without any spurious data added anywhere.

jqxChart with relative values

I was playing around with the waterfall series of the jqxChart.
According to its API, the following piece of code defines the values of the axis, in this case it's the y-axis:
valueAxis:
{
title: {text: 'Population<br>'},
unitInterval: 1000000,
labels:
{
formatFunction: function (value) {
return value / 1000000 + ' M';
}
}
}
Is it possible to define the intervals not with absolute values, but with relative values. So that the interval are e.g. 10% and the overall value is 100%?
Simply doing unitInterval: '10%' doesn't work.
This is how it should look like:
Here is a fiddle.
I think you're looking for these options :
logarithmicScale: true,
logarithmicScaleBase: 1.10,
Example:
valueAxis:
{
title: {text: 'Population<br>'},
logarithmicScale: true,
logarithmicScaleBase: 1.10,
labels:
{
formatFunction: function (value) {
return value / 1000000 + ' M';
}
}
},
Edit:
var accuracy = 2;
var first = data[0].population;
var last = data[data.length - 2].population;
var unit = (100 / last);
// convert raw data to differences
for (var i = 0; i < data.length - 2; i++)
data[i].population = (data[i].population * unit).toFixed(accuracy);

Categories

Resources