I followed the instructions in this article and created a Javascript metronome. It makes use of the Web Audio API and has audioContext.currentTime at its core for precise timing.
My version, available at this plunker, is a very simplified version of the original, made by Chris Wilson and available here. In order for mine to work, since it uses an actual audio file and doesn't synthesize sounds through an oscillator, you need to download the plunker and this audio file, placing it in the root folder (it's a metronome 'tick' sound, but you could use any sound you want).
It works like a charm - if it weren't for the fact that if the user minimizes the window, the otherwise very accurate metronome starts hiccupping instantly and awfully. I really don't understand what is the problem, here.
Javascript
var context, request, buffer;
var tempo = 120;
var tickTime;
function ticking() {
var source = context.createBufferSource();
source.buffer = buffer;
source.connect(context.destination);
source.start(tickTime);
}
function scheduler() {
while (tickTime < context.currentTime + 0.1) { //while there are notes to schedule, play the last scheduled note and advance the pointer
ticking();
tickTime += 60 / tempo;
}
}
function loadTick() {
request = new XMLHttpRequest(); //Asynchronous http request (you'll need a local server)
request.open('GET', 'tick.wav', true); //You need to download the file # http://s000.tinyupload.com/index.php?file_id=89415137224761217947
request.responseType = 'arraybuffer';
request.onload = function () {
context.decodeAudioData(request.response, function (theBuffer) {
buffer = theBuffer;
});
};
request.send();
}
function start() {
tickTime = context.currentTime;
scheduleTimer = setInterval(function () {
scheduler();
}, 25);
}
window.onload = function () {
window.AudioContext = window.AudioContext || window.webkitAudioContext;
context = new AudioContext();
loadTick();
start();
};
Yeah, this is because the browsers throttle setTimeout and setInterval to once per second when the window loses focus. (This was done to circumvent CPU/power drain due to developers using setTimeout/setInterval for visual animation, and not pausing the animation when the tab lost focus.)
There are two ways around this:
1) increase the "look-ahead" (in your example, 0.1 second) to greater than one second - like, 1.1s. Unfortunately, this would mean you couldn't change things (like stopping the playback, or changing the tempo) without a more-than-one-second lag in the change; so you'd probably want to only increase that value when the blur event was fired on the window, and change it back to 0.1 when the window focus event fired. Still not ideal.
2) Circumvent the throttling. :) It turns out you can do this, because setTimeout/setInterval are NOT throttled in Web Workers! (This approach was originally suggested by someone in the comment thread of my original article at http://www.html5rocks.com/en/tutorials/audio/scheduling/#disqus_thread.) I implemented this for the metronome code in https://github.com/cwilso/metronome: take a look at the js/metronome.js and js/metronomeworker.js. The Worker basically just maintains the timer, and marshals a message back across to the main thread; take a look at https://github.com/cwilso/metronome/blob/master/js/metronome.js#L153, in particular, to see how it's kicked off. You could modify that snippet of code and use metronomeworker.js as-is to fix this.
Related
I am working with HTML5 audio. For my use-case, I need to listen on the audio duration played and once it crosses a certain threshold, pause the audio. So something like:
$(audio).bind('timeupdate', function() {
if (audio.currentTime >= 10){
audio.pause();
}
});
What I am noticing is that by the time my handler executes, audio.currentTime is around 10.12878, 10.34023 etc and hence, some little extra audio is played before it is paused.
Another question seems to have documented the same issue.The question is dated in 2012 so I am wondering if the state of the art has improved.
If not, what other ways exist to do this with more precision? I haven't worked with audio much before and I would really appreciate the help.
According to MDN documentation, timeupdate event
The event frequency is dependant on the system load, but will be
thrown between about 4Hz and 66Hz (assuming the event handlers don't
take longer than 250ms to run). User agents are encouraged to vary the
frequency of the event based on the system load and the average cost
of processing the event each time, so that the UI updates are not any
more frequent than the user agent can comfortably handle while
decoding the video.
To pause audio at 10.0nnnnn, you can utilize requestAnimationFrame; if condition this.currentTime > 9.999 to call .pause() before reaching 10.000000. Note, 9.999 can be adjust to between 9.9 through 9.99750, which could also periodically pause audio at 9.99nnnn; that is, before reaching 10.0
var requestAnimationFrame = window.requestAnimationFrame;
var audio;
function trackCurrentTime() {
// `this` : `audio`
console.log(this.currentTime);
if (this.currentTime > 9.999) {
this.pause();
} else {
requestAnimationFrame(trackCurrentTime.bind(this))
}
}
var request = new XMLHttpRequest();
request.open("GET", "/path/to/audio", true);
request.responseType = "blob";
request.onload = function() {
if (this.status == 200) {
audio = new Audio(URL.createObjectURL(this.response));
audio.onpause = function(e) {
console.log(e.target.currentTime)
}
audio.load();
audio.play();
trackCurrentTime.call(audio);
}
}
request.send();
jsfiddle http://jsfiddle.net/Lg5L4qso/7/
So I researched my way through this problem and came out with the following findings:
1) HTML Audio is no dice for such a scenario. Its a very limited API and doesn't offer fine grain audio control. Web Audio API is the way to go for this.
2) Web Audio API is not the easiest thing to read/understand and is perhaps weakly implemented across various browsers. Its better to find a nice wrapper around it that fallsback to web audio or flash when Web Audio is not available in a browser.
3) Some wrappers exist like howler.js, sound.js and SoundManager2. Off the three SoundManager2 seems the best fit at first thought.
Demo 4G at http://www.schillmania.com/projects/soundmanager2/demo/api/ effectively solves a very similar problem as in the scenario in the question.
Update: SoundManager2 actually doesn't support the web audio api. From the technotes:
SM2 does not presently use the Webkit Audio API. It may be
experimentally or more formally added at some point in the future,
when the API is more universally-supported.
HTML Audio is no dice for such a scenario. Its a very limited API and doesn't offer fine grain audio control. Web Audio API is the way to go for this.
I am having problems when I want to know the current time of a file playing using the Web Audio API. My code plays the file nicely and the current time returned by the getCurrentTime() function is accurate enough when it comes to short files which load fast.
But when I try to load big files, sometimes the current time returned by the getCurrentTime() function is accurate and sometimes not. Sometimes, after waiting for example for 20 seconds to hear the file playing, when it starts playing it says that the current time is about 20 seconds (which is not true because it is just playing the beginning of the file). It happens with any audio format (OGG, MP3, WAV...) but only sometimes.
I am using a slow system (Asus EEE PC 901 with an Intel Atom 1.60 Ghz and 2 GB RAM with Windows XP Home Edition and SP3) and Firefox 41.0.1.
I am not sure, but it seems that the source.start() method starts playing the sound way too late, so the line after calling that method, where I set the value for the startTime variable, is not the real starting time.
Here is the code (simplified):
var context, buffer, startTime, source;
var stopped = true;
function load(file, startAt)
{
//Here creates the AudioContext and loads the file through XHR (AJAX) and gets the buffer. All works fine.
//When it gots the buffer through XHR (AJAX) and all is fine, it calls play(startAt) function immediately.
//Note: normally, startAt is 0.
}
function play(startAt)
{
source = context.createBufferSource(); //Context created before.
source.buffer = buffer; //Buffer got before from XHR (AJAX).
//Creates a gain node to be able to set the volume later:
var gainNode = context.createGain();
source.connect(gainNode);
gainNode.connect(context.destination);
//Plays the sound:
source.loop = false;
source.start(startAt, 0, buffer.duration - 3); //I don't want the last 3 seconds.
//Stores the start time (useful for pause/resume):
startTime = context.currentTime - startAt; //Here I store the startTime but maybe the file has still not begun to play (should it be just startTime = context.currentTime?).
stopped = false;
}
function stop()
{
source.stop(0);
stopped = true;
}
function getCurrentTime()
{
return (stopped) ? 0 : context.currentTime - startTime;
}
How can I detect when exactly the source.start() method starts playing the file? So I can set the startTime variable value just at that moment, and never before.
Thank you very much in advance. I would really appreciate any kind of help.
From MDN (https://developer.mozilla.org/en-US/docs/Web/API/AudioBufferSourceNode/start), about the first parameter of the start() function:
when (Optional)
The time, in seconds, at which the sound should begin to play, in the same time coordinate system used by the AudioContext. If when is less than (AudioContext.currentTime, or if it's 0, the sound begins to play at once. The default value is 0.
There is no evident issue with your code (although there is no example a call to play()): if you call play(0) or play(context.currentTime + someDelayInSeconds), start() should behave as expected. Unfortunately here the issue is that AudioBufferSource is not meant for big files. Again from the MDN doc of AudioBuffer (https://developer.mozilla.org/en-US/docs/Web/API/AudioBuffer):
Objects of these types are designed to hold small audio snippets, typically less than 45 s.
I suspect that for big something doesn't work very well with the "sound begins play at once" assumption (I also experienced it, although 20 seconds seems way too much...). Unfortunately there is no way to get the exact start time of AudioBufferSource in WebAudio yet.
If you don't have any real reason to load this big file with AudioBufferSource, I suggest you use a MediaElementSourceNode (https://developer.mozilla.org/en-US/docs/Web/API/MediaElementAudioSourceNode): as you can see from the example on the linked doc, it allows you to plug a simple HTML5 Audio element into the AudioContext. You then can have all usual control over the element itself, i.e. you also have access to the audioElement.currentTime property, which tells you the current playout time (in this case of the file itself, which is what you need). Additionally, you don't have to handle loading of the file in memory and could start playing as soon as some data is available.
context.currentTime starts counting the second you create the context object. That means if it takes 20 seconds for your audio to load, context.currentTime == 20.
To account for this delay, you can set a simple timer from the time that you create the context to the time that audio loading completes.
var context; //Create your context here
var audioLoadStart = new Date();
//Do audio load
var audioLoadOffset = (new Date() - audioLoadStart) / 1000;
currentTime = context.currentTime - audioLoadOffset - startTime;
I was tracking down some ridiculously high load times that my app's javascript reported, and found that Android (and iOS) pause some JavaScript execution when the window is in the background or the display is off.
On Android, I found that I could use the window.onfocus and onblur events to detect when the app was switching to the background (and js execution would soon be paused, at least for new scripts), but I can't find a way to detect when the screen is turned on or off. Is this possible?
(On Safari, I had similar results except that onfocus and onblur didn't fire reliably.)
There is few options to check it:
Using Visibility API
Using focus and blur events to detect browser tab visibility:
window.addEventListener("focus", handleBrowserState.bind(context, true));
window.addEventListener("blur", handleBrowserState.bind(context, false));
function handleBrowserState(isActive){
// do something
}
Using timers, as mentioned above
I just found a pretty good solution for my use case:
function getTime() {
return (new Date()).getTime();
}
var lastInterval = getTime();
function intervalHeartbeat() {
var now = getTime();
var diff = now - lastInterval;
var offBy = diff - 1000; // 1000 = the 1 second delay I was expecting
lastInterval = now;
if(offBy > 100) { // don't trigger on small stutters less than 100ms
console.log('interval heartbeat - off by ' + offBy + 'ms');
}
}
setInterval(intervalHeartbeat, 1000);
When the screen is turned off (or JS is paused for any reason), the next interval is delayed until JS execution resumes. In my code, I can just adjust the timers by the offBy amount and call it good.
In quick testing, this seemed to work well on both Android 4.2.2's browser and Safari on iOS 6.1.3.
Found a nice function here:
http://rakaz.nl/2009/09/iphone-webapps-101-detecting-essential-information-about-your-iphone.html
(function() {
var timestamp = new Date().getTime();
function checkResume() {
var current = new Date().getTime();
if (current - timestamp > 4000) {
var event = document.createEvent("Events");
event.initEvent("resume", true, true);
document.dispatchEvent(event);
}
timestamp = current;
}
window.setInterval(checkResume, 1000);
})();
To register for event:
addEventListener("resume", function() {
alert('Resuming this webapp');
});
This is consistent with Cordova which also fires the resume event.
what will you do in your script once you now that the screen turns off? Well anyway, you can inject Java objects ( http://developer.android.com/reference/android/webkit/WebView.html#addJavascriptInterface(java.lang.Object,%20java.lang.String) ) to interface with the activity and proxy all information you require in JS world.
I'm having a bit of trouble rewinding audio in Javascript. I basically have a countdown that beeps each second as it gets towards the end of the countdown.
I tried using;
var bip = new Audio("http://www.soundjay.com/button/beep-7.wav");
bip.play();
but it didn't beep every second which I'm guessing has something to do withit having to load a new sound every second. I then tried loading the sound externally and triggering it with the following code.
bip.pause();
bip.currentTime = 0;
console.log(bip.currentTime);
bip.play();
but this only plays the sound once then completely fails to rewind it (this is shown by the console logging a time of 0.19 seconds after the first playthrough).
Is there something I'm missing here?
In google chrome I noticed that it only works if you have the audio file in same domain. I have no problems if the audio file is in same domain. Event setting .currentTime works.
Cross-domain:
var bip = new Audio("http://www.soundjay.com/button/beep-7.wav");
bip.play();
bip.currentTime; //0.10950099676847458
bip.currentTime = 0;
bip.currentTime; //0.10950099676847458
Same-domain:
var bip = new Audio("beep-7.wav");
bip.play();
bip.currentTime; //0.10950099676847458
bip.currentTime = 0;
bip.currentTime; //0
I tried googling for a while and could find nothing in specs about this or even any discussion.
when I want rewind I simply load the audio again:
my_audio.load()
*btw, I also use a eventlistener for 'canplay' to trigger the my_audio.play(). It seems that this is necessary in android, and maybe other devices also*
To further dsdsdsdsd's answer with a bit of paint-by-numbers for the "whole shebang"
NOTE: In my app, loc1 is a dummy that refers to the song's stored location
// ... code before ...
tune = new Audio(loc1); // First, create audio event using js
// set up "song's over' event listener & action
tune.addEventListener('ended', function(){
tune.load(); //reload audio event (and reset currentTime!)
tune.play(); //play audio event for subsequent 'ended' events
},false);
tune.play(); // plays it the first time thru
// ... code after ...
I spent days and days trying to figure out what I was doing wrong, but it all works fine now... at least on the desktop browsers...
As of Chrome version 37.0.2062.120 m, the behaviour described by #Esailija has not changed.
I workaround this issue by encoding the audio data in base64 encoding and feed the data to Audio() using data: URL.
Test code:
snd = new Audio('data:audio/ogg;base64,[...base64 encoded data...]');
snd.onloadeddata = function() {
snd.currentTime = 0;
snd.play();
setTimeout(function() {
snd.currentTime = 0;
snd.play();
}, 200);
};
(I am surprised that there are no bug reports or references on this matter... or maybe my Google-fu is not strong enough.)
I'm trying to make a cross-device/browser image and audio preloading scheme for a GameAPI I'm working on. An audio file will preload, and issue a callback once it completes.
The problem is, audio will not start to load on slow page loads, but will usually work on the second try, probably because it cached it and knows it exists.
I've narrowed it down to the audio.load() function. Getting rid of it solves the problem, but interestingly, my motorola droid needs that function.
What are some experiences you've had with HTML5 audio preloading?
Here's my code. Yes, I know loading images in a separate function could cause a race condition :)
var resourcesLoading = 0;
function loadImage(imgSrc) {
//alert("Starting to load an image");
resourcesLoading++;
var image = new Image();
image.src = imgSrc;
image.onload = function() {
//CODE GOES HERE
//alert("A image has been loaded");
resourcesLoading--;
onResourceLoad();
}
}
function loadSound(soundSrc) {
//alert("Starting to load a sound");
resourcesLoading++;
var loaded = false;
//var soundFile = document.createElement("audio");
var soundFile = document.createElement("audio");
console.log(soundFile);
soundFile.autoplay = false;
soundFile.preload = false;
var src = document.createElement("source");
src.src = soundSrc + ".mp3";
soundFile.appendChild(src);
function onLoad() {
loaded = true;
soundFile.removeEventListener("canplaythrough", onLoad, true);
soundFile.removeEventListener("error", onError, true);
//CODE GOES HERE
//alert("A sound has been loaded");
resourcesLoading--;
onResourceLoad();
}
//Attempt to reload the resource 5 times
var retrys = 4;
function onError(e) {
retrys--;
if(retrys > 0) {
soundFile.load();
} else {
loaded = true;
soundFile.removeEventListener("canplaythrough", onLoad, true);
soundFile.removeEventListener("error", onError, true);
alert("A sound has failed to loaded");
resourcesLoading--;
onResourceLoad();
}
}
soundFile.addEventListener("canplaythrough", onLoad, true);
soundFile.addEventListener("error", onError, true);
}
function onResourceLoad() {
if(resourcesLoading == 0)
onLoaded();
}
It's hard to diagnose the problem because it shows no errors and only fails occasionally.
I got it working. The solution was fairly simple actually:
Basically, it works like this:
channel.load();
channel.volume = 0.00000001;
channel.play();
If it isn't obvious, the load function tells browsers and devices that support it to start loading, and then the sound immediately tries to play with the volume virtually at zero. So, if the load function isn't enough, the fact that the sound 'needs' to be played is enough to trigger a load on all the devices I tested.
The load function may actually be redundant now, but based off the inconsistiency with audio implementation, it probably doesn't hurt to have it.
Edit: After testing this on Opera, Safari, Firefox, and Chrome, it looks like setting the volume to 0 will still preload the resource.
canplaythrough fires when enough data has buffered that it probably could play non-stop to the end if you started playing on that event. The HTML Audio element is designed for streaming, so the file may not have completely finished downloading by the time this event fires.
Contrast this to images which only fire their event once they are completely downloaded.
If you navigate away from the page and the audio has not finished completely downloading, the browser probably doesn't cache it at all. However, if it has finished completely downloading, it probably gets cached, which explains the behavior you've seen.
I'd recommend the HTML5 AppCache to make sure the images and audio are certainly cached.
The AppCache, as suggested above, might be your only solution to keep the audio cached from one browser-session to another (that's not what you asked for, right?). but keep in mind the limited amount of space, some browsers offer. Safari for instance allows the user to change this value in the settings but the default is 5MB - hardly enough to save a bunch of songs, especially if other websites that are frequented by your users use AppCache as well. Also IE <10 does not support AppCache.
Alright so I ran into the same problem recently, and my trick was to use a simple ajax request to load the file entirely once (which end into the cache), and then by loading the sound again directly from the cache and use the event binding canplaythrough.
Using Buzz.js as my HTML5 audio library, my code is basically something like that:
var self = this;
$.get(this.file_name+".mp3", function(data) {
self.sound = new buzz.sound(self.file_name, {formats: [ "mp3" ], preload: true});
self.sound.bind("error", function(e) {
console.log("Music Error: " + this.getErrorMessage());
});
self.sound.decreaseVolume(20);
self.sound.bind("canplaythrough",function(){ self.onSoundLoaded(self); });
});