Develop Reference

How do we schedule a series of oscillator nodes that play for a fixed duration with a smooth transition from one node’s ending to the other?

We are trying to map an array of numbers to sound and are following the approach mentioned in this ‘A Tale of 2 Clocks’ article to schedule oscillator nodes to play in the future. Each oscillator node exponentially ramps to a frequency value corresponding to the data in a fixed duration (this.pointSonificationLength). However, there’s a clicking noise as each node stops, as referenced in this article by Chris Wilson. The example here talks about smoothly stopping a single oscillator. However, we are unable to directly use this approach to smoothen the transition between one oscillator node to the other.
To clarify some of the values, pointTime refers to the node’s number in the order starting from 0, i.e. as the nodes were scheduled, they’d have pointTime = 0, 1, 2, and so forth. this.pointSonificationLength is the constant used to indicate how long the node should play for.
The first general approach was to decrease the gain at the end of the node so the change is almost imperceptible, as was documented in the article above. We tried implementing both methods, including setTargetAtTime and a combination of exponentialRampToValueAtTime and setValueAtTime, but neither worked to remove the click.
We were able to remove the click by changing some of our reasoning, and we scheduled for the gain to start transitioning to 0 at 100ms before the node ends.
However, when we scheduled more than one node, there was now a pause between each node. If we changed the function to start transitioning at 10ms, the gap was removed, but there was still a quiet click.
Our next approach was to have each node fade in as well as fade out. We added in this.delay as a constant to be the amount of time each transition in and out takes.
Below is where we’re currently at in the method to schedule an oscillator node for a given time with a given data point. The actual node scheduling is contained in another method inside the class.
private scheduleOscillatorNode(dataPoint: number, pointTime: number) {
let osc = this.audioCtx.createOscillator()
let amp = this.audioCtx.createGain()
osc.frequency.value = this.previousFrequencyOfset
osc.frequency.exponentialRampToValueAtTime(dataPoint, pointTime + this.pointSonificationLength)
osc.onended = () => this.handleOnEnded()
osc.connect(amp).connect(this.audioCtx.destination)
let nodeStart = pointTime + this.delay * this.numNode;
amp.gain.setValueAtTime(0.00001, nodeStart);
amp.gain.exponentialRampToValueAtTime(1, nodeStart + this.delay);
amp.gain.setValueAtTime(1, nodeStart + this.delay + this.pointSonificationLength);
amp.gain.exponentialRampToValueAtTime(0.00001, nodeStart + this.delay * 2 + this.pointSonificationLength);
osc.start(nodeStart)
osc.stop(nodeStart + this.delay * 2 + this.pointSonificationLength)
this.numNode++;
this.audioQueue.enqueue(osc)
// code to keep track of playback
}
We notice that there is a slight difference between the values we calculate manually and the values we see when we log the time values using the console.log statements, but the difference was too small to potentially be perceivable. As a result, we believe that this may not be causing the clicking noise, since the difference shouldn’t be perceivable if it’s so small. For example, instead of ending at 6.7 seconds, the node would end at 6.699999999999999 seconds, or if the node was meant to end at 5.6 seconds, it would actually end at 5.6000000000000005 seconds.
Is there a way to account for these delays and schedule nodes such that the transition occurs smoothly? Alternatively, is there a different approach that we need to use to make these transitions smooth? Any suggestions and pointers to code samples or other helpful resources would be of great help!

Ramp WebAudio gain, while playback is stopping and starting?

I have an application where audio playback may be starting and stopping, and there are UI controls to ramp the gain to zero or nonzero values. I'm scheduling playback using AudioBufferSourceNode.start, and modulating gain using AudioParam.linearRampToValueAtTime. Playback is sometimes scheduled for a future time. The problem I'm having is that the ramp function only seems to set values when playback is currently happening; so, if we try to set the gain value e.g. between playback being scheduled and playback starting, the new values are lost. I could do a bunch of timing checks, and either ramp or directly set gain depending on whether playback is happening, but this can get messy, and I was wondering whether there was an alternative way to do this which would work independently of playback starting and stopping.
Here is a test case: we create a one-second noise buffer and play it, while also ramping gain to zero. If playback is scheduled for after the ramp has ended (one second), the gain value never gets set and remains at the default, nonzero value.
var ctx = new AudioContext();
var SR = ctx.sampleRate;
var buffer = ctx.createBuffer(1, SR, SR);
var channelData = buffer.getChannelData(0);
for (var i=0; i<SR; i++) {
channelData[i] = Math.random() * 2 - 1;
}
var bufferNode = ctx.createBufferSource();
var gainNode = ctx.createGain();
bufferNode.buffer = buffer;
bufferNode.connect(gainNode);
gainNode.connect(ctx.destination);
gainNode.gain.linearRampToValueAtTime(0, ctx.currentTime + 1);
//XXX if start_delay is greater than 1 (the ramp duration),
// the gain is never changed and remains at 1.
var start_delay = 0;
bufferNode.start(ctx.currentTime + start_delay);

Couple of issues here:
The behavior of linearRampToValueAtTime without a preceeding automation event was originally not well-specified. This has since been fixed in the WebAudio spec. Don't know the status of various browsers on this, but I think Chrome does this correctly.
The automation of the gain node should work as you expect. If it doesn't file a bug against your browser vendor.

My problem seems to have been caused by a bug in Chrome, which was fixed in Chrome 57. Here is the bug report: https://bugs.chromium.org/p/chromium/issues/detail?id=647974
The following comment has a workaround for older versions of Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=647974#c9

requestAnimationFrame [now] vs performance.now() time discrepancy

Assumptions: rAF now time is calculated at the time the set of callbacks are all triggered. Therefore any blocking that happens before the first callback of that frame is called doesn't affect the rAF now and it's accurate--at least for that first callback.
Any performance.now() measurements made before a rAF set is triggered should be earlier than rAF now.
Test: Record before (a baseline time before anything happens). Set the next rAF. Compare rAF now and actual performance.now() to before to see how different they are.
Expected results:
var before = performance.now(), frames = ["with blocking", "with no blocking"], calls = 0;
requestAnimationFrame(function frame(rAFnow) {
var actual = performance.now();
console.log("frame " + (calls + 1) + " " + frames[calls] + ":");
console.log("before frame -> rAF now: " + (rAFnow - before));
console.log("before frame -> rAF actual: " + (actual - before));
if (++calls < frames.length) { before = actual; requestAnimationFrame(frame); }
});
// blocking
for (var i = 0, l = 0; i < 10000000; i++) {
l += i;
}
Observations: When there is blocking before the frame starts, the rAF now time is at times incorrect, even for that first frame. Sometimes the first frame's now is actually an earlier time than the recorded before time.
Whether there is blocking happening before the frame or not, every so often the in-frame time rAFnow will be earlier than the pre-frame time before--even when I setup the rAF after I take my first measurement. This can also happen without any blocking whatsoever, though this is rarer.
(I get a timing error on the first blocking frame most of the time. Getting an issue on the others is rarer, but does happen occasionally if you try running it a few times.)
With more extensive testing, I've found bad times with blocking prior to callback: 1% from 100 frames, no blocking: 0.21645021645021645% from ~400 frames, seemingly caused by opening a window or some other potentially CPU-intensive action by the user.
So it's fairly rare, but the problem is this shouldn't happen at all. If you want to do useful things with them, simulating time, animation, etc., then you need those times to make sense.
I've taken into account what people have said, but maybe I am still not understanding how things work. If this is all per-spec, I'd love some psuedo-code to solidify it in my mind.
And more importantly, if anyone has any suggestions for how I could get around these issues, that would be awesome. The only thing I can think of is taking my own performance.now() measurement every frame and using that--but it seems a bit wasteful, having it effectively run twice every frame, on top of any triggered events and so on.

The timestamp passed in to the requestAnimationFrame() callback is the time of the beginning of the animation frame. Multiple callbacks being invoked during the same frame all receive the same timestamp. Thus, it would be really weird if performance.now() returned a time before the parameter value, but not really weird for it to be after that.
Here's the relevant specification:
When the user agent is to run the animation frame callbacks for a Document document with a timestamp now, it must run the following steps:
If the value returned by the document object’s hidden attribute is true, abort these steps. [PAGE-VISIBILITY]
Let callbacks be a list of the entries in document’s list of animation frame callbacks, in the order in which they were added to the list.
Set document’s list of animation frame callbacks to the empty list.
For each entry in callbacks, in order: invoke the Web IDL callback function, passing now as the only argument, and if an exception is thrown, report the exception.
So you've registered a callback (let's say just one) for the next animation frame. Tick tick tick, BOOM, time for that animation frame to happen:
The JavaScript runtime makes a note of the time and labels that now.
The runtime makes a temporary copy of the list of registered animation frame callbacks, and clears the actual list (so that they're not accidentally invoked if things take so long that the next animation frame comes around).
The list has just one thing in it: your callback. The system invokes that with now as the parameter.
Your callback starts running. Maybe it's never been run before, so the JavaScript optimizer may need to do some work. Or maybe the operating system switches threads to some other system process, like starting up a disk buffer flush or handling some network traffic, or any of dozens of other things.
Oh right, your callback. The browser gets the CPU again and your callback code starts running.
Your code calls performance.now() and compares it to the now value passed in as a parameter.
Because a brief but non-ignorable amount of time may pass between step 1 and step 6, the return value from performance.now() may indicate that a couple of microseconds, or even more than a couple, have elapsed. That is perfectly normal behavior.

I encountered the same issue on chrome, where calls to performance.now () would return a higher value than the now value passed into subsequent callbacks made by window.requestAnimationFrame ()
My workaround was to set the before using the now passed to the callback in the first window.requestAnimationFrame () rather than performance.now (). It seems that using only one of the two functions to measure time guarantees progressing values.
I hope this helps anyone else suffering through this bug.

Safari and requestAnimationFrame gets DOMHighResTimestamp; window.performance not available

I created an animation using requestAnimationFrame. Works fine in Windows Chrome and IE; Safari (tested Safari 6 and 7) breaks. It turns out that rAF get a DOMHighResTimestamp rather than a Date timestamp. That's all fine and good, and what I expected, as it's now part of the spec. However, as far as I've been able to find, there's no way to get the current DOMHighResTimestamp (i.e. window.performance is not available, even with a prefix). So if I create the start time as a Date timestamp, it behaves radically wrong when I try to determine progress within the rAF callback (very small negative numbers).
If you look at this JSBin on Safari, it won't animate at all.
In this JBin, I've made a change to "skip" the first frame (where the time parameter is undefined), so that startTime gets set to the time parameter on the next frame. Seems to work, but skipping a frame seems a bit crappy.
Is there some way to get the current DOMHighResTimestamp in Safari, given the lack of window.performance? Or alternately, force rAF into some sort of legacy mode that forces it to get a Date timestamp instead?
Does anyone know why Safari has this inconsistency, where it provides the parameter in a format that you can't get at any other way?

Performance.now() is only a recommendation as of now. https://developer.mozilla.org/en-US/docs/Web/API/Performance.now() I can only assume it's a matter of time before it's official, seeing as how everyone but Safari has it built in.
Besides that fact use this to your advantage. Since you know requestAnimationFrame returns a DOMHighResTimestamp use that for your timing.
Game.start = function(timestamp){
if(timestamp){
this.time = timestamp;
requestAnimationFrame(Game.loop);
}else{
requestAnimationFrame(Game.start);
}
}
Game.loop = function(timestamp){
Game.tick(timestamp);
... your code
requestAnimationFrame(Game.loop);
}
Game.tick = function(timestamp) {
this.delta = timestamp - this.time;
this.time = timestamp;
};
What I do here, is call Game.start which will begin the loop (I've run into situations where the timestamp was undefined so we try until we get something valid). Once we get that we have our base time and since RAF is going to return a timestamp our tick function never has to call Date.now or performance.now as long as we pass it the timestamp returned by requestAnimationFrame.

Most efficient way to throttle continuous JavaScript execution on a web page

I'd like to continuously execute a piece of JavaScript code on a page, spending all available CPU time I can for it, but allowing browser to be functional and responsive at the same time.
If I just run my code continuously, it freezes the browser's UI and browser starts to complain. Right now I pass a zero timeout to setTimeout, which then does a small chunk of work and loops back to setTimeout. This works, but does not seem to utilize all available CPU. Any better ways of doing this you might think of?
Update: To be more specific, the code in question is rendering frames on canvas continuously. The unit of work here is one frame. We aim for the maximum possible frame rate.

Probably what you want is to centralize everything that happens on the page and use requestAnimationFrame to do all your drawing. So basically you would have a function/class that looks something like this (you'll have to forgive some style/syntax errors I'm used to Mootools classes, just take this as an outline)
var Main = function(){
this.queue = [];
this.actions = {};
requestAnimationFrame(this.loop)
}
Main.prototype.loop = function(){
while (this.queue.length){
var action = this.queue.pop();
this.executeAction(e);
}
//do you rendering here
requestAnimationFrame(this.loop);
}
Main.prototype.addToQueue = function(e){
this.queue.push(e);
}
Main.prototype.addAction = function(target, event, callback){
if (this.actions[target] === void 0) this.actions[target] = {};
if (this.actions[target][event] === void 0) this.actions[target][event] = [];
this.actions[target][event].push(callback);
}
Main.prototype.executeAction = function(e){
if (this.actions[e.target]!==void 0 && this.actions[e.target][e.type]!==void 0){
for (var i=0; i<this.actions[e.target][e.type].length; i++){
this.actions[e.target][e.type](e);
}
}
}
So basically you'd use this class to handle everything that happens on the page. Every event handler would be onclick='Main.addToQueue(event)' or however you want to add your events to your page, you just point them to adding the event to the cue, and just use Main.addAction to direct those events to whatever you want them to do. This way every user action gets executed as soon as your canvas is finished redrawing and before it gets redrawn again. So long as your canvas renders at a decent framerate your app should remain responsive.
EDIT: forgot the "this" in requestAnimationFrame(this.loop)

web workers are something to try
https://developer.mozilla.org/en-US/docs/DOM/Using_web_workers

You can tune your performance by changing the amount of work you do per invocation. In your question you say you do a "small chunk of work". Establish a parameter which controls the amount of work being done and try various values.
You might also try to set the timeout before you do the processing. That way the time spent processing should count towards any minimum the browsers set.
One technique I use is to have a counter in my processing loop counting iterations. Then set up an interval of, say one second, in that function, display the counter and clear it to zero. This provides a rough performance value with which to measure the effects of changes you make.
In general this is likely to be very dependent on specific browsers, even versions of browsers. With tunable parameters and performance measurements you could implement a feedback loop to optimize in real-time.

One can use window.postMessage() to overcome the limitation on the minimum amount of time setTimeout enforces. See this article for details. A demo is available here.