Web Audio Api: frequency related to elapsed time - javascript

I have a script that gets audio frequencies. I want to relate that frequency with the exact time of the song. I can get the webkitAudioContext currentTime property, but this is not accurate because it starts to count time when saving the sound in the buffer before the song starts.
This is my code:
var context = new webkitAudioContext();
...
function drawSpectrogram(array) {
// copy the current canvas onto the temp canvas
var canvas = document.getElementById("canvas");
tempCtx.drawImage(canvas, 0, 0, 800, 512);
// iterate over the elements from the array
for (var i = 0; i < array.length; i++) {
// draw each pixel with the specific color
var value = array[i];
frequency = frequency + value + ";";
time = time + Math.round((context.currentTime) * 1000000) / 1000000 + ";";
ctx.fillStyle = hot.getColor(value).hex();
// draw the line at the right side of the canvas
ctx.fillRect(800 - 1, 512 - i, 1, 1);
}
}
Thank you!

Save the value of currentTime just before you are ready to begin playing, and use that time value as your "zero-time" marker. Whenever you want to know how long the music has been playing, subtract that time value (which indicates how long setup took) from the currentTime value.
var context = new webkitAudioContext();
// load media, place in buffer, etc...
var startTime = context.currentTime; // everything up to now has been setup
// begin playing...
var currentPlayingTime = context.currentTime - startTime;
Just use context.currentTime - startTime to find how long the song has actually been playing.

I'm not quite sure what you're going for, but you might want to take a look at the High Resolution Time API: http://www.w3.org/TR/hr-time/
You could grab the current time at the same time that you call .noteOn() or .start() on the buffer you're interested in, and then in your DrawSpectrogram function determine the time elapsed to do what you need.

Related

Html5 interpolation-latency problems

I have a node.js server which receives the client position and emits it to everyone once every 100ms. On the client side I have already implemented client prediction, reconciliation and entity interpolation. The problem is that if the latency gets higher, the interpolation is not smooth. There is a constant jitter and the player has different velocities. This is my interpolation code:
function updatePlayers(player) {
var t1 = previousLastUpdate; // timestamp of the previous postion
var t2 = lastUpdate; // timestamp of the actual position
var tickRate = t2 - t1;
if (tickRate < 100)
tickRate = 100;
// sometimes the difference is lower than 100(which is the server update rate) and
//this also creates jitter
var renderTime = Date.now() - tickRate;
if(renderTime >= t1 && renderTime <= t2) {
var portion = renderTime - t1;
var percentage = portion / tickRate;
// interpolate from last known position to actual position
var interpX = lerp(player.lastX, player.getX(), percentage);
var interpY = lerp(player.lastY, player.getY(), percentage);
player.setPrevX(interpX);
player.setPrevY(interpY);
}
}
So, my question is: how to get rid of this problem? I can't use dead reckoning because the players change their direction fast.

Capture photos in interval from canvas

I have a script that allows me to show in canvas the webcam and 'download' a specific frame within some intervals.
I am having trouble when time parameters are big (30 minutes of captures every 2 seconds). It works smoothly for about 15 minutes and then crashes (firefox closes with out of memory error). Also, after restarting firefox sometimes many 0 byte photos are taken during a 3-4 mins and then starts working again. I am running this in an old 2 GB RAM machine placed in the lab, is there a way to reduce memory usage?
Here is the piece of code with parameters and the function realizarCapturas.
I can add the resting code but I think the part to optimize should be this one.
var frecuenciaComienzoCaptura = 1; // how long till next capture
var frecuenciaCaptura = 3; //seconds between photos
var duracion = 5; // amount of photos to capture
function realizarCapturas(){
var i = 1;
var interval = setInterval(function(){
if(i <= duracion){
context.drawImage(video, 0, 0, 640, 480);
var imagen = document.getElementById("imagen");
imagen.href = canvas.toDataURL("image/png");
var now = new Date();
var filename = formatNumber(now.getHours()) + "-" + formatNumber(now.getMinutes()) + "-" + formatNumber(now.getSeconds());
imagen.download = filename + ".png"; // Make sure the browser downloads the image
imagen.click(); // Trigger the click
i = i+1;
}else{
clearInterval(interval);
}
}, frecuenciaCaptura * 1000);
}
setInterval(function(){
realizarCapturas();
}, frecuenciaComienzoCaptura * 1000 * 60 * 60);
realizarCapturas();
}, false);
As a rule NEVER use setInterval as it can be a source of call stack overflows which are very difficult to detect in code.
Your problem is that you are not clearing all the intervals you are generating and thus every 3 seconds you are creating a new interval event. Eventually the time it takes to run the little bit of code will be longer than than can be managed by all the interval events you have created and thus each interval will continue to push their events onto the call stack but will not get a chance to be run until more intervals have been place on the stack eventually causing the crash. Nor does setInterval guarantee the time between events are accurate.
Use setTimeout instead. That way you will only ever generate event as needed and you do not have to keep a handle to turn off events.
Below is your code written so that you will never have a call stack overflow.
var frecuenciaComienzoCaptura = 1 * 1000* 60 * 60; // how long till next capture
var frecuenciaCaptura = 3 * 1000; //seconds between photos
var duracion = 5; // amount of photos to capture
var counter = 0;
// the capture function
var captura = function () {
counter = counter + 1;
if(counter < duracion){ // do we need more images?
// only create timer events as needed.
setTimeout(captura, frecuenciaCaptura); //set time till next image
}
context.drawImage(video, 0, 0, 640, 480);
var imagen = document.getElementById("imagen");
imagen.href = canvas.toDataURL("image/png");
var now = new Date();
var filename = formatNumber(now.getHours()) + "-" + formatNumber(now.getMinutes()) + "-" + formatNumber(now.getSeconds());
imagen.download = filename + ".png"; // Make sure the browser downloads the image
imagen.click(); // Trigger the click
}
function realizarCapturas() {
// request next batch of captures by only creating one timer event as we need
setTimeout(realizarCapturas,frecuenciaComienzoCaptura);
counter = 0; // reset counter
captura(); // capture timages
}
// start captures.
realizarCapturas();

Using while(Date.now() < interval) {} with requestAnimationFrame()

We all know how difficult it is to make a proper update algorithm if certain fps is important or something like this.
Anyway, I just came up with this infinite-ish while cycle hack, which just freezes the program until the next frame, and it seems to work flawlessly.
var then = Date.now()
var fps = 40;
var interval = 1000 / fps;
function mainloop() {
while (Date.now() - then < interval) {} // freezes program until next frame
requestAnimationFrame(mainloop);
then = Date.now();
// update logic goes here
}
mainloop();
I haven't seen this solution anywhere, so I wanted to ask whether it is clean and correct. I know it is bad freezing the program just to wait for something and that piece of code looks terrible, but it seems to work. Is there a cleaner solution that would work similarly to my code?
You can use setTimeout to wait for a certain time but that will not be very precise. However by changing interval all the time you can get the average delay precise enough.
var startTime = Date.now();
var fps = 40;
var frames = 0;
var interval = 1000 / fps;
function mainloop() {
frames++;
var timeElapsed = Date.now() - startTime,
averageFps = 1000 * frames / timeElapsed;
if(averageFps < fps && interval > 0) interval -= 0.1;
if(averageFps > fps) interval += 0.1;
setTimeout(mainloop, interval);
// update logic goes here
}
setTimeout(mainloop, interval);
But there is still the risk that the computer isn't able to meet the requested fps if it's too slow.
Using a while loop to waste time is a bad idea. It just wastes processor time that could be doing something else.
Using SetTimeout as suggested by jishi, is one possible solution. However, that only controls when your code runs. You have no real control over when the browser actually paints. Bascially, the browser will paint the last frame that your code updated.
Thus, another possible solution is to use requestAnimation. In your drawing code, determine the last frame that would have occurred at your preferred rate. Draw that frame. For example...
var start = null;
var fps = 40;
var interval = 1000 / fps;
function mainloop(timeStamp) {
if (!start) {
start = timeStamp;
}
var n = (timeStamp - start) / interval;
// update logic goes here to paint nth frame
requestAnimationFrame(mainloop);
}
mainloop();
In your specific scenario, it would probably be better to delay the mainloop execution using setTimeout, like this:
var nextExecution = Date.now() - then + interval;
if (nextExecution < 0) nextExecution = 0;
setTimeout(mainloop, nextExecution);
This will allow it to do other stuff while waiting for the next frame rendering.

Record audio, sync to loop, offset latency and export portion

I am building a web app which allows users to listen to a loop of instrumental music and then record vocals on top. This is all working using Recorder.js however there are a few problems:
There is latency with recording, so this needs to be set by the user before pressing record.
The exported loop is not always the same length as the sample rate might not match the time needed exactly
However since then I went back to the drawing board and asked: What's best for the user?. This gave me a new set of requirements:
Backing loop plays continuously in the background
Recording starts and stops whenever the user chooses
Recording then plays back in sync with loop (the dead time between loops is automatically filled with blank audio)
User can slide an offset slider to adjust for small timing issues with latency
User can select which portion of the recording to save (same length as original backing loop)
Here's a diagram of how that would look:
Logic I have so far:
// backing loop
a.startTime = 5
a.duration = 10
a.loop = true
// recording
b.startTime = 22.5
b.duration = 15
b.loop = false
// fill blank space + loop
fill = a.duration - (b.duration % a.duration) // 5
c = b.buffers + (fill * blankBuffers)
c.startTime = (context.currentTime - a.startTime) % a.duration
c.duration = 20
c.loop = true
// user corrects timing offset
c.startTime = ((context.currentTime - a.startTime) % a.duration) - offset
// user choose favourite loop
? this is where I start to lose the plot!
Here is an example of chopping the buffers sent from Recorder.js:
// shorten the length of buffers
start = context.sampleRate * 2; // start at 2 seconds
end = context.sampleRate * 3; // end at 3 seconds
buffers.push(buffers.subarray(start, end));
And more example code from the previous versions i've been working on:
https://github.com/mattdiamond/Recorderjs/issues/105
Any help in working out how to slice the buffers for the exported loop or improving this logic would be greatly appreciated!
UPDATE
Using this example I was able to find out how to insert blank space into the recording:
http://mdn.github.io/audio-buffer/
I've now managed to almost replicate the functionality I need, however the white noise seems off. Is there a miscalculation somewhere?
http://kmturley.github.io/Recorderjs/loop.html
I managed to solve this by writing the following logic
diff = track2.startTime - track1.startTime
before = Math.round((diff % track1.duration) * 44100)
after = Math.round((track1.duration - ((diff + track2.duration) % track1.duration)) * 44100)
newAudio = [before data] + [recording data] + [after data]
and in javascript code it looks like this:
var i = 0,
channel = 0,
channelTotal = 2,
num = 0,
vocalsRecording = this.createBuffer(vocalsBuffers, channelTotal),
diff = this.recorder.startTime - backingInstance.startTime + (offset / 1000),
before = Math.round((diff % backingInstance.buffer.duration) * this.context.sampleRate),
after = Math.round((backingInstance.buffer.duration - ((diff + vocalsRecording.duration) % backingInstance.buffer.duration)) * this.context.sampleRate),
audioBuffer = this.context.createBuffer(channelTotal, before + vocalsBuffers[0].length + after, this.context.sampleRate),
buffer = null;
// loop through the audio left, right channels
for (channel = 0; channel < channelTotal; channel += 1) {
buffer = audioBuffer.getChannelData(channel);
// fill the empty space before the recording
for (i = 0; i < before; i += 1) {
buffer[num] = 0;
num += 1;
}
// add the recording data
for (i = 0; i < vocalsBuffers[channel].length; i += 1) {
buffer[num] = vocalsBuffers[channel][i];
num += 1;
}
// fill the empty space at the end of the recording
for (i = 0; i < after; i += 1) {
buffer[num] = 0;
num += 1;
}
}
// now return the new audio which should be the exact same length
return audioBuffer;
You can view a full working example here:
http://kmturley.github.io/Recorderjs/loop.html

Controlling fps with requestAnimationFrame?

It seems like requestAnimationFrame is the de facto way to animate things now. It worked pretty well for me for the most part, but right now I'm trying to do some canvas animations and I was wondering: Is there any way to make sure it runs at a certain fps? I understand that the purpose of rAF is for consistently smooth animations, and I might run the risk of making my animation choppy, but right now it seems to run at drastically different speeds pretty arbitrarily, and I'm wondering if there's a way to combat that somehow.
I'd use setInterval but I want the optimizations that rAF offers (especially automatically stopping when the tab is in focus).
In case someone wants to look at my code, it's pretty much:
animateFlash: function() {
ctx_fg.clearRect(0,0,canvasWidth,canvasHeight);
ctx_fg.fillStyle = 'rgba(177,39,116,1)';
ctx_fg.strokeStyle = 'none';
ctx_fg.beginPath();
for(var i in nodes) {
nodes[i].drawFlash();
}
ctx_fg.fill();
ctx_fg.closePath();
var instance = this;
var rafID = requestAnimationFrame(function(){
instance.animateFlash();
})
var unfinishedNodes = nodes.filter(function(elem){
return elem.timer < timerMax;
});
if(unfinishedNodes.length === 0) {
console.log("done");
cancelAnimationFrame(rafID);
instance.animate();
}
}
Where Node.drawFlash() is just some code that determines radius based off a counter variable and then draws a circle.
How to throttle requestAnimationFrame to a specific frame rate
Demo throttling at 5 FPS: http://jsfiddle.net/m1erickson/CtsY3/
This method works by testing the elapsed time since executing the last frame loop.
Your drawing code executes only when your specified FPS interval has elapsed.
The first part of the code sets some variables used to calculate elapsed time.
var stop = false;
var frameCount = 0;
var $results = $("#results");
var fps, fpsInterval, startTime, now, then, elapsed;
// initialize the timer variables and start the animation
function startAnimating(fps) {
fpsInterval = 1000 / fps;
then = Date.now();
startTime = then;
animate();
}
And this code is the actual requestAnimationFrame loop which draws at your specified FPS.
// the animation loop calculates time elapsed since the last loop
// and only draws if your specified fps interval is achieved
function animate() {
// request another frame
requestAnimationFrame(animate);
// calc elapsed time since last loop
now = Date.now();
elapsed = now - then;
// if enough time has elapsed, draw the next frame
if (elapsed > fpsInterval) {
// Get ready for next frame by setting then=now, but also adjust for your
// specified fpsInterval not being a multiple of RAF's interval (16.7ms)
then = now - (elapsed % fpsInterval);
// Put your drawing code here
}
}
I suggest wrapping your call to requestAnimationFrame in a setTimeout:
const fps = 25;
function animate() {
// perform some animation task here
setTimeout(() => {
requestAnimationFrame(animate);
}, 1000 / fps);
}
animate();
You need to call requestAnimationFrame from within setTimeout, rather than the other way around, because requestAnimationFrame schedules your function to run right before the next repaint, and if you delay your update further using setTimeout you will have missed that time window. However, doing the reverse is sound, since you’re simply waiting a period of time before making the request.
Update 2016/6
The problem throttling the frame rate is that the screen has a constant update rate, typically 60 FPS.
If we want 24 FPS we will never get the true 24 fps on the screen, we can time it as such but not show it as the monitor can only show synced frames at 15 fps, 30 fps or 60 fps (some monitors also 120 fps).
However, for timing purposes we can calculate and update when possible.
You can build all the logic for controlling the frame-rate by encapsulating calculations and callbacks into an object:
function FpsCtrl(fps, callback) {
var delay = 1000 / fps, // calc. time per frame
time = null, // start time
frame = -1, // frame count
tref; // rAF time reference
function loop(timestamp) {
if (time === null) time = timestamp; // init start time
var seg = Math.floor((timestamp - time) / delay); // calc frame no.
if (seg > frame) { // moved to next frame?
frame = seg; // update
callback({ // callback function
time: timestamp,
frame: frame
})
}
tref = requestAnimationFrame(loop)
}
}
Then add some controller and configuration code:
// play status
this.isPlaying = false;
// set frame-rate
this.frameRate = function(newfps) {
if (!arguments.length) return fps;
fps = newfps;
delay = 1000 / fps;
frame = -1;
time = null;
};
// enable starting/pausing of the object
this.start = function() {
if (!this.isPlaying) {
this.isPlaying = true;
tref = requestAnimationFrame(loop);
}
};
this.pause = function() {
if (this.isPlaying) {
cancelAnimationFrame(tref);
this.isPlaying = false;
time = null;
frame = -1;
}
};
Usage
It becomes very simple - now, all that we have to do is to create an instance by setting callback function and desired frame rate just like this:
var fc = new FpsCtrl(24, function(e) {
// render each frame here
});
Then start (which could be the default behavior if desired):
fc.start();
That's it, all the logic is handled internally.
Demo
var ctx = c.getContext("2d"), pTime = 0, mTime = 0, x = 0;
ctx.font = "20px sans-serif";
// update canvas with some information and animation
var fps = new FpsCtrl(12, function(e) {
ctx.clearRect(0, 0, c.width, c.height);
ctx.fillText("FPS: " + fps.frameRate() +
" Frame: " + e.frame +
" Time: " + (e.time - pTime).toFixed(1), 4, 30);
pTime = e.time;
var x = (pTime - mTime) * 0.1;
if (x > c.width) mTime = pTime;
ctx.fillRect(x, 50, 10, 10)
})
// start the loop
fps.start();
// UI
bState.onclick = function() {
fps.isPlaying ? fps.pause() : fps.start();
};
sFPS.onchange = function() {
fps.frameRate(+this.value)
};
function FpsCtrl(fps, callback) {
var delay = 1000 / fps,
time = null,
frame = -1,
tref;
function loop(timestamp) {
if (time === null) time = timestamp;
var seg = Math.floor((timestamp - time) / delay);
if (seg > frame) {
frame = seg;
callback({
time: timestamp,
frame: frame
})
}
tref = requestAnimationFrame(loop)
}
this.isPlaying = false;
this.frameRate = function(newfps) {
if (!arguments.length) return fps;
fps = newfps;
delay = 1000 / fps;
frame = -1;
time = null;
};
this.start = function() {
if (!this.isPlaying) {
this.isPlaying = true;
tref = requestAnimationFrame(loop);
}
};
this.pause = function() {
if (this.isPlaying) {
cancelAnimationFrame(tref);
this.isPlaying = false;
time = null;
frame = -1;
}
};
}
body {font:16px sans-serif}
<label>Framerate: <select id=sFPS>
<option>12</option>
<option>15</option>
<option>24</option>
<option>25</option>
<option>29.97</option>
<option>30</option>
<option>60</option>
</select></label><br>
<canvas id=c height=60></canvas><br>
<button id=bState>Start/Stop</button>
Old answer
The main purpose of requestAnimationFrame is to sync updates to the monitor's refresh rate. This will require you to animate at the FPS of the monitor or a factor of it (ie. 60, 30, 15 FPS for a typical refresh rate # 60 Hz).
If you want a more arbitrary FPS then there is no point using rAF as the frame rate will never match the monitor's update frequency anyways (just a frame here and there) which simply cannot give you a smooth animation (as with all frame re-timings) and you can might as well use setTimeout or setInterval instead.
This is also a well known problem in the professional video industry when you want to playback a video at a different FPS then the device showing it refresh at. Many techniques has been used such as frame blending and complex re-timing re-building intermediate frames based on motion vectors, but with canvas these techniques are not available and the result will always be jerky video.
var FPS = 24; /// "silver screen"
var isPlaying = true;
function loop() {
if (isPlaying) setTimeout(loop, 1000 / FPS);
... code for frame here
}
The reason why we place setTimeout first (and why some place rAF first when a poly-fill is used) is that this will be more accurate as the setTimeout will queue an event immediately when the loop starts so that no matter how much time the remaining code will use (provided it doesn't exceed the timeout interval) the next call will be at the interval it represents (for pure rAF this is not essential as rAF will try to jump onto the next frame in any case).
Also worth to note that placing it first will also risk calls stacking up as with setInterval. setInterval may be slightly more accurate for this use.
And you can use setInterval instead outside the loop to do the same.
var FPS = 29.97; /// NTSC
var rememberMe = setInterval(loop, 1000 / FPS);
function loop() {
... code for frame here
}
And to stop the loop:
clearInterval(rememberMe);
In order to reduce frame rate when the tab gets blurred you can add a factor like this:
var isFocus = 1;
var FPS = 25;
function loop() {
setTimeout(loop, 1000 / (isFocus * FPS)); /// note the change here
... code for frame here
}
window.onblur = function() {
isFocus = 0.5; /// reduce FPS to half
}
window.onfocus = function() {
isFocus = 1; /// full FPS
}
This way you can reduce the FPS to 1/4 etc.
These are all good ideas in theory, until you go deep. The problem is you can't throttle an RAF without de-synchronizing it, defeating it's very purpose for existing. So you let it run at full-speed, and update your data in a separate loop, or even a separate thread!
Yes, I said it. You can do multi-threaded JavaScript in the browser!
There are two methods I know that work extremely well without jank, using far less juice and creating less heat. Accurate human-scale timing and machine efficiency are the net result.
Apologies if this is a little wordy, but here goes...
Method 1: Update data via setInterval, and graphics via RAF.
Use a separate setInterval for updating translation and rotation values, physics, collisions, etc. Keep those values in an object for each animated element. Assign the transform string to a variable in the object each setInterval 'frame'. Keep these objects in an array. Set your interval to your desired fps in ms: ms=(1000/fps). This keeps a steady clock that allows the same fps on any device, regardless of RAF speed. Do not assign the transforms to the elements here!
In a requestAnimationFrame loop, iterate through your array with an old-school for loop-- do not use the newer forms here, they are slow!
for(var i=0; i<sprite.length-1; i++){ rafUpdate(sprite[i]); }
In your rafUpdate function, get the transform string from your js object in the array, and its elements id. You should already have your 'sprite' elements attached to a variable or easily accessible through other means so you don't lose time 'get'-ing them in the RAF. Keeping them in an object named after their html id's works pretty good. Set that part up before it even goes into your SI or RAF.
Use the RAF to update your transforms only, use only 3D transforms (even for 2d), and set css "will-change: transform;" on elements that will change. This keeps your transforms synced to the native refresh rate as much as possible, kicks in the GPU, and tells the browser where to concentrate most.
So you should have something like this pseudocode...
// refs to elements to be transformed, kept in an array
var element = [
mario: document.getElementById('mario'),
luigi: document.getElementById('luigi')
//...etc.
]
var sprite = [ // read/write this with SI. read-only from RAF
mario: { id: mario ....physics data, id, and updated transform string (from SI) here },
luigi: { id: luigi .....same }
//...and so forth
] // also kept in an array (for efficient iteration)
//update one sprite js object
//data manipulation, CPU tasks for each sprite object
//(physics, collisions, and transform-string updates here.)
//pass the object (by reference).
var SIupdate = function(object){
// get pos/rot and update with movement
object.pos.x += object.mov.pos.x; // example, motion along x axis
// and so on for y and z movement
// and xyz rotational motion, scripted scaling etc
// build transform string ie
object.transform =
'translate3d('+
object.pos.x+','+
object.pos.y+','+
object.pos.z+
') '+
// assign rotations, order depends on purpose and set-up.
'rotationZ('+object.rot.z+') '+
'rotationY('+object.rot.y+') '+
'rotationX('+object.rot.x+') '+
'scale3d('.... if desired
; //...etc. include
}
var fps = 30; //desired controlled frame-rate
// CPU TASKS - SI psuedo-frame data manipulation
setInterval(function(){
// update each objects data
for(var i=0; i<sprite.length-1; i++){ SIupdate(sprite[i]); }
},1000/fps); // note ms = 1000/fps
// GPU TASKS - RAF callback, real frame graphics updates only
var rAf = function(){
// update each objects graphics
for(var i=0; i<sprite.length-1; i++){ rAF.update(sprite[i]) }
window.requestAnimationFrame(rAF); // loop
}
// assign new transform to sprite's element, only if it's transform has changed.
rAF.update = function(object){
if(object.old_transform !== object.transform){
element[object.id].style.transform = transform;
object.old_transform = object.transform;
}
}
window.requestAnimationFrame(rAF); // begin RAF
This keeps your updates to the data objects and transform strings synced to desired 'frame' rate in the SI, and the actual transform assignments in the RAF synced to GPU refresh rate. So the actual graphics updates are only in the RAF, but the changes to the data, and building the transform string are in the SI, thus no jankies but 'time' flows at desired frame-rate.
Flow:
[setup js sprite objects and html element object references]
[setup RAF and SI single-object update functions]
[start SI at percieved/ideal frame-rate]
[iterate through js objects, update data transform string for each]
[loop back to SI]
[start RAF loop]
[iterate through js objects, read object's transform string and assign it to it's html element]
[loop back to RAF]
Method 2. Put the SI in a web-worker. This one is FAAAST and smooth!
Same as method 1, but put the SI in web-worker. It'll run on a totally separate thread then, leaving the page to deal only with the RAF and UI. Pass the sprite array back and forth as a 'transferable object'. This is buko fast. It does not take time to clone or serialize, but it's not like passing by reference in that the reference from the other side is destroyed, so you will need to have both sides pass to the other side, and only update them when present, sort of like passing a note back and forth with your girlfriend in high-school.
Only one can read and write at a time. This is fine so long as they check if it's not undefined to avoid an error. The RAF is FAST and will kick it back immediately, then go through a bunch of GPU frames just checking if it's been sent back yet. The SI in the web-worker will have the sprite array most of the time, and will update positional, movement and physics data, as well as creating the new transform string, then pass it back to the RAF in the page.
This is the fastest way I know to animate elements via script. The two functions will be running as two separate programs, on two separate threads, taking advantage of multi-core CPU's in a way that a single js script does not. Multi-threaded javascript animation.
And it will do so smoothly without jank, but at the actual specified frame-rate, with very little divergence.
Result:
Either of these two methods will ensure your script will run at the same speed on any PC, phone, tablet, etc (within the capabilities of the device and the browser, of course).
How to easily throttle to a specific FPS:
// timestamps are ms passed since document creation.
// lastTimestamp can be initialized to 0, if main loop is executed immediately
var lastTimestamp = 0,
maxFPS = 30,
timestep = 1000 / maxFPS; // ms for each frame
function main(timestamp) {
window.requestAnimationFrame(main);
// skip if timestep ms hasn't passed since last frame
if (timestamp - lastTimestamp < timestep) return;
lastTimestamp = timestamp;
// draw frame here
}
window.requestAnimationFrame(main);
Source: A Detailed Explanation of JavaScript Game Loops and Timing by Isaac Sukin
The simplest way
note: It might behave differently on different screens with different frame rate.
const FPS = 30;
let lastTimestamp = 0;
function update(timestamp) {
requestAnimationFrame(update);
if (timestamp - lastTimestamp < 1000 / FPS) return;
/* <<< PUT YOUR CODE HERE >>> */
lastTimestamp = timestamp;
}
update();
var time = 0;
var time_framerate = 1000; //in milliseconds
function animate(timestamp) {
if(timestamp > time + time_framerate) {
time = timestamp;
//your code
}
window.requestAnimationFrame(animate);
}
A simple solution to this problem is to return from the render loop if the frame is not required to render:
const FPS = 60;
let prevTick = 0;
function render()
{
requestAnimationFrame(render);
// clamp to fixed framerate
let now = Math.round(FPS * Date.now() / 1000);
if (now == prevTick) return;
prevTick = now;
// otherwise, do your stuff ...
}
It's important to know that requestAnimationFrame depends on the users monitor refresh rate (vsync). So, relying on requestAnimationFrame for game speed for example will make it unplayable on 200Hz monitors if you're not using a separate timer mechanism in your simulation.
Skipping requestAnimationFrame cause not smooth(desired) animation at custom fps.
// Input/output DOM elements
var $results = $("#results");
var $fps = $("#fps");
var $period = $("#period");
// Array of FPS samples for graphing
// Animation state/parameters
var fpsInterval, lastDrawTime, frameCount_timed, frameCount, lastSampleTime,
currentFps=0, currentFps_timed=0;
var intervalID, requestID;
// Setup canvas being animated
var canvas = document.getElementById("c");
var canvas_timed = document.getElementById("c2");
canvas_timed.width = canvas.width = 300;
canvas_timed.height = canvas.height = 300;
var ctx = canvas.getContext("2d");
var ctx2 = canvas_timed.getContext("2d");
// Setup input event handlers
$fps.on('click change keyup', function() {
if (this.value > 0) {
fpsInterval = 1000 / +this.value;
}
});
$period.on('click change keyup', function() {
if (this.value > 0) {
if (intervalID) {
clearInterval(intervalID);
}
intervalID = setInterval(sampleFps, +this.value);
}
});
function startAnimating(fps, sampleFreq) {
ctx.fillStyle = ctx2.fillStyle = "#000";
ctx.fillRect(0, 0, canvas.width, canvas.height);
ctx2.fillRect(0, 0, canvas.width, canvas.height);
ctx2.font = ctx.font = "32px sans";
fpsInterval = 1000 / fps;
lastDrawTime = performance.now();
lastSampleTime = lastDrawTime;
frameCount = 0;
frameCount_timed = 0;
animate();
intervalID = setInterval(sampleFps, sampleFreq);
animate_timed()
}
function sampleFps() {
// sample FPS
var now = performance.now();
if (frameCount > 0) {
currentFps =
(frameCount / (now - lastSampleTime) * 1000).toFixed(2);
currentFps_timed =
(frameCount_timed / (now - lastSampleTime) * 1000).toFixed(2);
$results.text(currentFps + " | " + currentFps_timed);
frameCount = 0;
frameCount_timed = 0;
}
lastSampleTime = now;
}
function drawNextFrame(now, canvas, ctx, fpsCount) {
// Just draw an oscillating seconds-hand
var length = Math.min(canvas.width, canvas.height) / 2.1;
var step = 15000;
var theta = (now % step) / step * 2 * Math.PI;
var xCenter = canvas.width / 2;
var yCenter = canvas.height / 2;
var x = xCenter + length * Math.cos(theta);
var y = yCenter + length * Math.sin(theta);
ctx.beginPath();
ctx.moveTo(xCenter, yCenter);
ctx.lineTo(x, y);
ctx.fillStyle = ctx.strokeStyle = 'white';
ctx.stroke();
var theta2 = theta + 3.14/6;
ctx.beginPath();
ctx.moveTo(xCenter, yCenter);
ctx.lineTo(x, y);
ctx.arc(xCenter, yCenter, length*2, theta, theta2);
ctx.fillStyle = "rgba(0,0,0,.1)"
ctx.fill();
ctx.fillStyle = "#000";
ctx.fillRect(0,0,100,30);
ctx.fillStyle = "#080";
ctx.fillText(fpsCount,10,30);
}
// redraw second canvas each fpsInterval (1000/fps)
function animate_timed() {
frameCount_timed++;
drawNextFrame( performance.now(), canvas_timed, ctx2, currentFps_timed);
setTimeout(animate_timed, fpsInterval);
}
function animate(now) {
// request another frame
requestAnimationFrame(animate);
// calc elapsed time since last loop
var elapsed = now - lastDrawTime;
// if enough time has elapsed, draw the next frame
if (elapsed > fpsInterval) {
// Get ready for next frame by setting lastDrawTime=now, but...
// Also, adjust for fpsInterval not being multiple of 16.67
lastDrawTime = now - (elapsed % fpsInterval);
frameCount++;
drawNextFrame(now, canvas, ctx, currentFps);
}
}
startAnimating(+$fps.val(), +$period.val());
input{
width:100px;
}
#tvs{
color:red;
padding:0px 25px;
}
H3{
font-weight:400;
}
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<h3>requestAnimationFrame skipping <span id="tvs">vs.</span> setTimeout() redraw</h3>
<div>
<input id="fps" type="number" value="33"/> FPS:
<span id="results"></span>
</div>
<div>
<input id="period" type="number" value="1000"/> Sample period (fps, ms)
</div>
<canvas id="c"></canvas><canvas id="c2"></canvas>
Original code by #tavnab.
For throttling FPS to any value, pls see jdmayfields answer.
However, for a very quick and easy solution to halve your frame rate, you can simply do your computations only every 2nd frame by:
requestAnimationFrame(render);
function render() {
// ... computations ...
requestAnimationFrame(skipFrame);
}
function skipFrame() { requestAnimationFrame(render); }
Similarly you could always call render but use a variable to control whether you do computations this time or not, allowing you to also cut FPS to a third or fourth (in my case, for a schematic webgl-animation 20fps is still enough while considerably lowering computational load on the clients)
I always do it this very simple way without messing with timestamps:
let fps, eachNthFrame, frameCount;
fps = 30;
//This variable specifies how many frames should be skipped.
//If it is 1 then no frames are skipped. If it is 2, one frame
//is skipped so "eachSecondFrame" is renderd.
eachNthFrame = Math.round((1000 / fps) / 16.66);
//This variable is the number of the current frame. It is set to eachNthFrame so that the
//first frame will be renderd.
frameCount = eachNthFrame;
requestAnimationFrame(frame);
//I think the rest is self-explanatory
function frame() {
if (frameCount === eachNthFrame) {
frameCount = 0;
animate();
}
frameCount++;
requestAnimationFrame(frame);
}
Here is an idea to reach desired fps:
detect browser's animationFrameRate (typically 60fps)
build a bitSet, according to animationFrameRate and your disiredFrameRate (say 24fps)
lookup bitSet and conditionally "continue" the animation frame loop
It uses requestAnimationFrame so the actual frame rate won't be greater than animationFrameRate. you may adjust disiredFrameRate according to animationFrameRate.
I wrote a mini lib, and a canvas animation demo.
function filterNums(nums, jitter = 0.2, downJitter = 1 - 1 / (1 + jitter)) {
let len = nums.length;
let mid = Math.floor(len % 2 === 0 ? len / 2 : (len - 1) / 2), low = mid, high = mid;
let lower = true, higher = true;
let sum = nums[mid], count = 1;
for (let i = 1, j, num; i <= mid; i += 1) {
if (higher) {
j = mid + i;
if (j === len)
break;
num = nums[j];
if (num < (sum / count) * (1 + jitter)) {
sum += num;
count += 1;
high = j;
} else {
higher = false;
}
}
if (lower) {
j = mid - i;
num = nums[j];
if (num > (sum / count) * (1 - downJitter)) {
sum += num;
count += 1;
low = j;
} else {
lower = false;
}
}
}
return nums.slice(low, high + 1);
}
function snapToOrRound(n, values, distance = 3) {
for (let i = 0, v; i < values.length; i += 1) {
v = values[i];
if (n >= v - distance && n <= v + distance) {
return v;
}
}
return Math.round(n);
}
function detectAnimationFrameRate(numIntervals = 6) {
if (typeof numIntervals !== 'number' || !isFinite(numIntervals) || numIntervals < 2) {
throw new RangeError('Argument numIntervals should be a number not less than 2');
}
return new Promise((resolve) => {
let num = Math.floor(numIntervals);
let numFrames = num + 1;
let last;
let intervals = [];
let i = 0;
let tick = () => {
let now = performance.now();
i += 1;
if (i < numFrames) {
requestAnimationFrame(tick);
}
if (i === 1) {
last = now;
} else {
intervals.push(now - last);
last = now;
if (i === numFrames) {
let compareFn = (a, b) => a < b ? -1 : a > b ? 1 : 0;
let sortedIntervals = intervals.slice().sort(compareFn);
let selectedIntervals = filterNums(sortedIntervals, 0.2, 0.1);
let selectedDuration = selectedIntervals.reduce((s, n) => s + n, 0);
let seletedFrameRate = 1000 / (selectedDuration / selectedIntervals.length);
let finalFrameRate = snapToOrRound(seletedFrameRate, [60, 120, 90, 30], 5);
resolve(finalFrameRate);
}
}
};
requestAnimationFrame(() => {
requestAnimationFrame(tick);
});
});
}
function buildFrameBitSet(animationFrameRate, desiredFrameRate){
let bitSet = new Uint8Array(animationFrameRate);
let ratio = desiredFrameRate / animationFrameRate;
if(ratio >= 1)
return bitSet.fill(1);
for(let i = 0, prev = -1, curr; i < animationFrameRate; i += 1, prev = curr){
curr = Math.floor(i * ratio);
bitSet[i] = (curr !== prev) ? 1 : 0;
}
return bitSet;
}
let $ = (s, c = document) => c.querySelector(s);
let $$ = (s, c = document) => Array.prototype.slice.call(c.querySelectorAll(s));
async function main(){
let canvas = $('#digitalClock');
let context2d = canvas.getContext('2d');
await new Promise((resolve) => {
if(window.requestIdleCallback){
requestIdleCallback(resolve, {timeout:3000});
}else{
setTimeout(resolve, 0, {didTimeout: false});
}
});
let animationFrameRate = await detectAnimationFrameRate(10); // 1. detect animation frame rate
let desiredFrameRate = 24;
let frameBits = buildFrameBitSet(animationFrameRate, desiredFrameRate); // 2. build a bit set
let handle;
let i = 0;
let count = 0, then, actualFrameRate = $('#actualFrameRate'); // debug-only
let draw = () => {
if(++i >= animationFrameRate){ // shoud use === if frameBits don't change dynamically
i = 0;
/* debug-only */
let now = performance.now();
let deltaT = now - then;
let fps = 1000 / (deltaT / count);
actualFrameRate.textContent = fps;
then = now;
count = 0;
}
if(frameBits[i] === 0){ // 3. lookup the bit set
handle = requestAnimationFrame(draw);
return;
}
count += 1; // debug-only
let d = new Date();
let text = d.getHours().toString().padStart(2, '0') + ':' +
d.getMinutes().toString().padStart(2, '0') + ':' +
d.getSeconds().toString().padStart(2, '0') + '.' +
(d.getMilliseconds() / 10).toFixed(0).padStart(2, '0');
context2d.fillStyle = '#000000';
context2d.fillRect(0, 0, canvas.width, canvas.height);
context2d.font = '36px monospace';
context2d.fillStyle = '#ffffff';
context2d.fillText(text, 0, 36);
handle = requestAnimationFrame(draw);
};
handle = requestAnimationFrame(() => {
then = performance.now();
handle = requestAnimationFrame(draw);
});
/* debug-only */
$('#animationFrameRate').textContent = animationFrameRate;
let frameRateInput = $('#frameRateInput');
let frameRateOutput = $('#frameRateOutput');
frameRateInput.addEventListener('input', (e) => {
frameRateOutput.value = e.target.value;
});
frameRateInput.max = animationFrameRate;
frameRateOutput.value = frameRateOutput.value = desiredFrameRate;
frameRateInput.addEventListener('change', (e) => {
desiredFrameRate = +e.target.value;
frameBits = buildFrameBitSet(animationFrameRate, desiredFrameRate);
});
}
document.addEventListener('DOMContentLoaded', main);
<div>
Animation Frame Rate: <span id="animationFrameRate">--</span>
</div>
<div>
Desired Frame Rate: <input id="frameRateInput" type="range" min="1" max="60" step="1" list="frameRates" />
<output id="frameRateOutput"></output>
<datalist id="frameRates">
<option>15</option>
<option>24</option>
<option>30</option>
<option>48</option>
<option>60</option>
</datalist>
</div>
<div>
Actual Frame Rate: <span id="actualFrameRate">--</span>
</div>
<canvas id="digitalClock" width="240" height="48"></canvas>
Simplified explanation of earlier answer. At least if you want real-time, accurate throttling without the janks, or dropping frames like bombs. GPU and CPU friendly.
setInterval and setTimeout are both CPU-oriented, not GPU.
requestAnimationFrame is purely GPU-oriented.
Run them separately. It's simple and not janky. In your setInterval, update your math and create a little CSS script in a string. With your RAF loop, only use that script to update the new coordinates of your elements. Don't do anything else in the RAF loop.
The RAF is tied inherently to the GPU. Whenever the script does not change (i.e. because the SI is running a gazillion times slower), Chromium-based browsers know they do not need to do anything, because there are no changes. So the on-the-fly script created each "frame", say 60 times per second, is still the same for say 1000 RAF GPU frames, but it knows nothing has changed, and the net result is it wastes no energy on this. If you check in DevTools, you will see your GPU frame-rate registers at the rate delineated by the setInterval.
Truely, it is just that simple. Separate them, and they will cooperate.
No jankies.
I tried multiple solutions provided on this question. Even though the solutions work as expected, they result in not so professional output.
Based on my personal experience, I would highly recommend not to control FPS on the browser side, especially using requestAnimationFrame. Because, when you do that, it'll make the frame rendering experience very choppy, users will clearly see the frames jumping and finally, it won't look real or professional at all.
So, my advice would be to control the FPS from the server side at the time of sending itself and simply render the frames as soon as you receive them on the browser side.
Note: if you still want to control on the client-side, try avoiding
usage of setTimeout or Date object in your logic of controlling fps.
Because, when the FPS is high, these will introduce their own delay in
terms of event loops or object creations.
Here's a good explanation I found: CreativeJS.com, to wrap a setTimeou) call inside the function passed to requestAnimationFrame. My concern with a "plain" requestionAnimationFrame would be, "what if I only want it to animate three times a second?" Even with requestAnimationFrame (as opposed to setTimeout) is that it still wastes (some) amount of "energy" (meaning that the Browser code is doing something, and possibly slowing the system down) 60 or 120 or however many times a second, as opposed to only two or three times a second (as you might want).
Most of the time I run my browsers with JavaScript intentially off for just this reason. But, I'm using Yosemite 10.10.3, and I think there's some kind of timer problem with it - at least on my old system (relatively old - meaning 2011).

Categories

Resources