Develop Reference

Can the built-in heatmap layer in Google Maps API be optimized for animation?

Context:
The built-in heatmap layer in Google Maps API doesn't have native support for animations, so I've created my own animations using requestAnimationFrame with calls to the Maps API to set the next frame's data. Each frame uses ~2500 data points to render the heatmap and maxes out around 4000 data points.
The problem:
My animation has jank every 5-10 cycles. I don't need the animation to render 60fps, but I do want the framerate to be consistent and each step to be <=100ms. In Devtools, the janky animation frames give me this error at the top of call stack: "Warning: Recurring handler took # ms."
My proposed cause of the problem:
The dev console shows memory leaks,* minor GC, major GC, and DOM GC correlating to Maps API calls. The DOM GC in particular is correlated to the animation jank. (*JS Heap, Nodes, and Listeners increase linearly).
Why I'm asking the question:
I'm fairly new to Google Maps API, requestAnimationFrame, and memory management in JavaScript, so first of all I want to make sure I'm understanding the problem correctly. Secondly, I want to see if there is a way to optimize the heatmap layer for animations, or if my approach is misguided.
Most relevant resources I've consulted:
Google Maps API documentation
Google's guide to web performance
StackOverflow question that asks about very basic, short Maps API heatmap animations
Code
All within a d3.csv('data').
Establish recursive requestAnimationFrame:
const periodDuration = 100; // ms
const timeStart = performance.now();
const frameTick = timestamp => {
if (timestamp - timeStart >= periodDuration) {
timeStart = timestamp;
transitionToNextPeriod(); // shown in next code snippet
}
requestAnimationFrame(frameTick);
};
requestAnimationFrame(frameTick);
transitionToNextPeriod increments time, then filters data, and ends by setting heatmap with updated data:
function transitionToNextPeriod() {
let batchedData = [],
currPeriodData = [],
addMin = 0;
prevMin = min,
prevDate = date,
currMin = 0;
// batch the next 'x' # of mins for one batchedData call after `while` loop
while (++addMin <= 7){
if (prevMin + addMin === 60){
if (hour + 1 === 24){
hour = 0;
date++;
} else {
hour++
}
}
// Now with current minute, filter data and add to batchedData
currMin = ((prevMin + addMin) % 60);
let newTime = `${hour}:${zeroPad(currMin)}`;
// Filter based on current time
let currFilter = dayData.filter( d => d.StartTime.indexOf( ` ${newTime}` ) !== -1);
prevPeriodData = prevPeriodData.filter( d => d.EndTime.indexOf( ` ${newTime}` ) === -1 );
batchedData = [...batchedData, ...currFilter ];
} // End while
batchedData = [...batchedData, ...prevPeriodData];
prevPeriodData = batchedData;
min = currMin;
// Update the maps.LatLng data with new data, then set heatmap data
batchedData.forEach( (d,i) => {
// Check for last datum in current period
if (i + 1 === batchedData.length){
heatmap.setData(currPeriodData); // PROBLEM: about every 7th call to setData results in jank
} else {
currPeriodData.push(new google.maps.LatLng( d.Lat , d.Long ));
}
})
} // End transitionToNextPeriod()

Plotly: How to discard older points?

TL;DR I want to display a long-running strip chart with Plotly.js. I don't know how to discard old points.
Details
The following updater from my CodePen at https://codepen.io/Michael-F-Ellis/pen/QvXPQr does almost what I want. It shows a set of 20 samples in 2 traces that update continuously at 500 msec intervals. At the end of the demo, it plots all the points to show they still exist.
var cnt = 0;
var interval = setInterval(function() {
// Add next point to each trace
Plotly.extendTraces('graph', {
y: [[rand()], [rand()]]
}, [0, 1])
// Display only 20 most recent points
Plotly.relayout('graph', { 'xaxis.range': [cnt-20, cnt]})
cnt = cnt+1;
if(cnt === 100) {
// Before ending the run, show all points
// to demonstrate they still exist in Plotly.
Plotly.relayout('graph', { 'xaxis.range': [0, cnt]});
clearInterval(interval);
}
}, 500);
The problem is that I do want to delete older points. The real application needs to run essentially forever on a system with limited memory. I'm looking for a Plotly call that will drop the oldest N trace points. It needs to be reasonably efficient as performance of the target system is also limited.
Thanks!

https://codepen.io/Michael-F-Ellis/pen/YxeEwm
The above seems workable from a behavioral standpoint. Here's the revised updating routine:
Plotly.plot('graph', data);
var cnt = 0;
var max = 20;
var interval = setInterval(function() {
// Add next point to each trace
Plotly.extendTraces('graph', {
y: [[rand()], [rand()]]
}, [0, 1])
// Keep only 'max' most recent points
if(cnt > max) {
data[0].y.shift();
data[1].y.shift();
}
cnt = cnt+1;
if(cnt === 100) {
// Before ending the run, show all points
// to demonstrate that only 'max' points
// still exist in Plotly.
Plotly.relayout('graph', { 'xaxis.range': [0, cnt]});
clearInterval(interval);
}
}, 500);
The solution is to keep the data object in a var outside of Plotly and use shift() to drop old points from the beginning of the array as new points are added.
I'm open to another solution, especially if there are known memory or performance problems with this approach.

Most performant way to call update loop of a javaScript physics engine

I've written my own HTML5 canvas - javaScript based physics engine to simulate a number of points connected by springs. The current general structure of the program is
function init(){
// A bunch of event listeners
renderer();
physics();
}
var frameTime = 1;
function physics(){
// iterate the physics
parts.update();
setTimeout(physics, frameTime);
}
// render loop
function renderer(){
// draws a rectangle over the last frame
drawBackground();
// renders the objects themselves
parts.draw();
// update the timeout according to an onscreen slider
frameTime = Math.ceil(101 - speed_r.value) / 2;
setTimeout(renderer, 15);
}
The rationale behind the 2 different loops is that the human eye only needs to see 60fps, but doing more updates per second yields better physics.
I've since done more research, and found that the standard way to render animations with javaScript is to call requestAnimationFrame(), which as I understand it has the advantage of not rendering while the tab is deselected, improving battery life. However, due to the dual loop structure, the physics will continue to be calculated and will probably outweigh the renderer overhead.
The question is: What is the most performant and ideally most efficient way to achieve this?

To sync your physics simulation with the wall clock, and render the animation smoothly, you need to use a fixed time step and interpolation. Read this excellent article (see also: archive.org) about both subjects.
Using requestAnimationFrame is a good idea to save battery (it will lower the frame rate if the battery is low on most devices). You can use it for both the physics and rendering loop.
What you have to do is compute the time elapsed since the last frame and then use zero or many fixed steps to keep the physics loop in sync with the current (wall-clock) time. This is how all real-time physics engines work, including Box2D and Bullet Physics.
I made a complete JSFiddle using HTML5 Canvas and JavaScript that implements what you need, based on the article mentioned above. See the code below or open it on JSFiddle.
The integrate function is where you update your physics. In the code it is used to step a spring simulation forward.
var t = 0;
var dt = 0.01;
var currentTime;
var accumulator = 0;
var previousState = { x: 100, v: 0 };
var currentState = { x: 100, v: 0 };
var canvas = document.getElementById("myCanvas");
var ctx = canvas.getContext("2d");
// start animation loop
requestAnimationFrame(animate);
function animate(newTime){
requestAnimationFrame(animate);
if (currentTime) {
var frameTime = newTime - currentTime;
if ( frameTime > 250 )
frameTime = 250;
accumulator += frameTime;
while ( accumulator >= dt )
{
previousState = currentState;
currentState = integrate( currentState, t, dt );
t += dt;
accumulator -= dt;
}
var alpha = accumulator / dt;
var interpolatedPosition = currentState.x * alpha + previousState.x * (1 - alpha);
render( interpolatedPosition );
}
currentTime = newTime;
}
// Move simulation forward
function integrate(state, time, fixedDeltaTime){
var fixedDeltaTimeSeconds = fixedDeltaTime / 1000;
var f = (200 - state.x) * 3;
var v = state.v + f * fixedDeltaTimeSeconds;
var x = state.x + v * fixedDeltaTimeSeconds;
return { x: x, v: v };
}
// Render the scene
function render(position){
// Clear
ctx.fillStyle = 'white';
ctx.fillRect(0,0,canvas.width,canvas.height);
// Draw circle
ctx.fillStyle = 'black';
ctx.beginPath();
ctx.arc(position,100,50,0,2*Math.PI);
ctx.closePath();
ctx.fill();
}

I think I'd look at putting the physics part in a web worker, and having it post updates to the main UI thread, which renders them on a requestAnimationFrame callback. That allows the physics code to run constantly (you don't even need setTimeout looping; although having it yield periodically so it can access messages from the front end — not least "stop"! — would be a good idea), while only updating the display as often as actually needed.
2018 update: As of ES2018, the worker and the main thread could share memory via a SharedArrayBuffer and the features of the Atomics object. Rather than having to break the work up so the worker could process messages, it could just check a location in the shared memory for flags (for instance, the flag saying it needs to stop). The worker could even be suspended right in the middle of the calculation (even in the middle of a standard loop, such as a for or while) and then resumed via Atomics.wait and Atomics.notify.

Memory gradually increased using d3 for real time chart?

I tried to simulate a real time chart with dynamic data using d3.js. I am running this using IE-10 browser.
My Source Code
I come across to a problem where the memory of my IE browser will be gradually increased if left the web application running for a period of time.
I Google searched the possible reason that caused this problem.
Two things come into my mind for discussion:
The timer prevents the garbage collection of IE
The d3 chart does not release memory after data.shift()
My question:
How could I diagnose if my problem actually originated from discussion 1 or 2 or neither?
How could I solve the memory problem?
You might need to download the code and run it with some time and monitor the iexplorer.exe using resource monitor in order to identify the problem.
Thank you.
Source Code:
<html>
<head>
<title>Animated Sparkline using SVG Path and d3.js</title>
<script src="http://mbostock.github.com/d3/d3.v2.js"></script>
<style>
/* tell the SVG path to be a thin blue line without any area fill */
path {
stroke: steelblue;
stroke-width: 1;
fill: none;
}
</style>
</head>
<body>
<span>
<b>Size:</b> 300x30 <b>Interpolation:</b> basis <b>Animation:</b> true <b>Transition:</b> 1000ms <b>Update Frequency:</b> 1000ms
<div id="graph1" class="aGraph" style="width:300px; height:30px;"></div>
</span>
<script>
var myTimer;
function FeedDataToChart(id, width, height, interpolation, animate, updateDelay, transitionDelay, data, startIndex) {
// create an SVG element inside the #graph div that fills 100% of the div
var graph = d3.select(id).append("svg:svg").attr("width", "80%").attr("height", "80%");
// X scale will fit values from 0-10 within pixels 0-100
var x = d3.scale.linear().domain([0, 48]).range([10, width-10]); // starting point is -5 so the first value doesn't show and slides off the edge as part of the transition
// Y scale will fit values from 0-10 within pixels 0-100
var y = d3.scale.linear().domain([0, 20]).range([height-10, 10]);
// create a line object that represents the SVN line we're creating
var line = d3.svg.line()
// assign the X function to plot our line as we wish
.x(function(d,i) {
// verbose logging to show what's actually being done
//console.log('Plotting X value for data point: ' + d + ' using index: ' + i + ' to be at: ' + x(i) + ' using our xScale.');
// return the X coordinate where we want to plot this datapoint
return x(i);
})
.y(function(d) {
// verbose logging to show what's actually being done
//console.log('Plotting Y value for data point: ' + d + ' to be at: ' + y(d) + " using our yScale.");
// return the Y coordinate where we want to plot this datapoint
return y(d);
})
.interpolate(interpolation)
var counter = startIndex;
//var myData = data.slice();
// display the line by appending an svg:path element with the data line we created above
graph.append("svg:path").attr("d", line(data));
// or it can be done like this
function redrawWithAnimation() {
// update with animation
graph.selectAll("path")
.data([data]) // set the new data
.attr("transform", "translate(" + x(1) + ")") // set the transform to the right by x(1) pixels (6 for the scale we've set) to hide the new value
.attr("d", line) // apply the new data values ... but the new value is hidden at this point off the right of the canvas
.transition() // start a transition to bring the new value into view
.ease("linear")
.duration(transitionDelay) // for this demo we want a continual slide so set this to the same as the setInterval amount below
.attr("transform", "translate(" + x(0) + ")"); // animate a slide to the left back to x(0) pixels to reveal the new value
}
function redrawWithoutAnimation() {
// static update without animation
graph.selectAll("path")
.data([data]) // set the new data
.attr("d", line); // apply the new data values
}
function stopTimer()
{
clearInterval(myTimer);
myTimer = null;
graph.selectAll("path").data([data]).remove().append("svg:path").attr("d", line);
buffer = null;
signalGenerator();
}
function startTimer()
{
if (myTimer == null)
{
myTimer = setInterval(function() {
if (counter < data.length - 1)
{
var v = data.shift(); // remove the first element of the array
data.push(v); // add a new element to the array (we're just taking the number we just shifted off the front and appending to the end)
if(animate)
{
redrawWithAnimation();
}
else
{
redrawWithoutAnimation();
}
counter++;
}
else
{
//alert("no more data in buffer");
stopTimer();
counter = startIndex;
}
}, updateDelay);
}
}
startTimer();
}
var buffer;
function signalGenerator()
{
if (buffer == null)
{
buffer = new Array(100);
var i;
for (i = 0; i < buffer.length; i++)
{
buffer[i] = Math.random() * 10;
}
FeedDataToChart("#graph1", 300, 300, "basis", true, 100, 100, buffer, 0);
}
}
function startGenerator()
{
signalGenerator();
}
startGenerator();
</script>
</body>
</html>

I tried as you said for 2 hours and it was initially 56 MB memory usage and in the end around 56.8 MB. It means only 0.8 MB difference with some exceptional cases. But I can help you finding the exact point where memory load is occurring. Just follow the steps one by one.
Open "Developer Tools" of IE by pressing F12
Go to Memory (A camera symbol or CTRL+7)
Click the Start Profiling Session ( Green Play button on top)
Take a Heap Snap Shot to create Base Line.
Now every 10 or 15 minutes take a heap snap shot
Do this for how many hours you require (In your case 2 hours)
Once profiling is done for desired time, stop it and analyze from beginning by comparing Heap Snap Shots.
If memory difference in the beginning and end is so big, check where this memory increase started by analyzing the memory difference in the snap shot.
Here you can check the difference of memory used by the process in terms of bytes or KB.
Check which function or variable or operation is creating the memory issue. Most probably some calculations that are repeatedly carried out so that the variables used in these calculations won't be released from a certain point of time. I saw some "Ba, p", "n, p", "Wa, n, p" etc when analyzed the memory flow. I believe the functions that use these variables are creating the problem for you.
Note
If you use the UI Responsiveness (CTRL+5), you can easily see that the Garbage Collection is carried out by IE automatically.

ThreeJS - how to set current time in Animation

I'm using skinning / skeletal animation in ThreeJS. I have an animation, and I want to be able to move backward and forward through it, and jump to different locations within it, rather than the usual looping behaviour.
The animation is created like this, as in the example:
var animation = new THREE.Animation( mesh, geometry.animation.name );
I have tried updating the animation with negative deltas, as well as setting animation.currentTime directly:
animation.currentTime = animationLocation;
These appear to work only if I move forward in time, but if I go backward the animation breaks and I get an error:
THREE.Animation.update: Warning! Scale out of bounds: ... on bone ...
One thing that does actually work without error is to call stop() and then play() with a new start time:
animation.stop();
animation.play( true, animationLocation );
...however when I look at what these functions are actually doing, they involve many many function calls, looping, resetting transforms etc. This seems like a horrible way to do it, even if it works as a hack.
It may be that this functionality does not exist yet, in which case I'll try to dig in and create a function that does a minimal amount of work, but I'm hoping there is another way that I haven't found.
Can anyone help with this?
[UPDATE]
As an update on my progress, I'll post the best solution I have at this time...
I pulled out the contents of the stop() and play() functions, and stripped out everything I could, making some assumptions about certain values having already been set by 'play()'.
This still seems like it is probably not the best way to do it, but it is doing a bit less work than by just calling stop() then play().
This is what I was able to get it down to:
THREE.Animation.prototype.gotoTime = function( time ) {
//clamp to duration of the animation:
time = THREE.Math.clamp( time, 0, this.length );
this.currentTime = time;
// reset key cache
var h, hl = this.hierarchy.length,
object;
for ( h = 0; h < hl; h ++ ) {
object = this.hierarchy[ h ];
var prevKey = object.animationCache.prevKey;
var nextKey = object.animationCache.nextKey;
prevKey.pos = this.data.hierarchy[ h ].keys[ 0 ];
prevKey.rot = this.data.hierarchy[ h ].keys[ 0 ];
prevKey.scl = this.data.hierarchy[ h ].keys[ 0 ];
nextKey.pos = this.getNextKeyWith( "pos", h, 1 );
nextKey.rot = this.getNextKeyWith( "rot", h, 1 );
nextKey.scl = this.getNextKeyWith( "scl", h, 1 );
}
//isPlaying must be true for update to work due to "early out"
//so remember the current play state:
var wasPlaying = this.isPlaying;
this.isPlaying = true;
//update with a delta time of zero:
this.update( 0 );
//reset the play state:
this.isPlaying = wasPlaying;
}
The main limitation of the function in terms of usefulness is that you can't interpolate from one arbitrary time to another. You can basically just scrub around in the animation.

You can use THREE.Clock and assign startTime, oldTime, elapsedTime.