Develop Reference

Best solution for unlocking web audio

I've been trying to think on some ideas on what I could make with JavaScript using Web Audio API. I know that depending on the user's browser I know that sometimes it won't let you play audio without a user gesture of some sort. I been doing some research on how to do it and they are pretty useful ways but the problem is that some developers found different ways to do it. For example:
Using a audioContext.resume() and audioContext.suspend() methods to unlock web audio by changing it's state:
function unlockAudioContext(context) {
if (context.state !== "suspended") return;
const b = document.body;
const events = ["touchstart", "touchend", "mousedown", "keydown"];
events.forEach(e => b.addEventListener(e, unlock, false));
function unlock() {context.resume().then(clean);}
function clean() {events.forEach(e => b.removeEventListener(e, unlock));}
}
creating an empty buffer and play it to unlock web audio.
var unlocked = false;
var context = new (window.AudioContext || window.webkitAudioContext)();
function init(e) {
if (unlocked) return;
// create empty buffer and play it
var buffer = context.createBuffer(1, 1, 22050);
var source = context.createBufferSource();
source.buffer = buffer;
source.connect(context.destination);
/*
Phonograph.js use this method to start it
source.start(context.currentTime);
paulbakaus.com suggest to use this method to start it
source.noteOn(0);
*/
source.start(context.currentTime) || source.noteOn(0);
setTimeout(function() {
if (!unlocked) {
if (source.playbackState === source.PLAYING_STATE || source.playbackState === source.FINISHED_STATE) {
unlocked = true;
window.removeEventListener("touchend", init, false);
}
}
}, 0);
}
window.addEventListener("touchend", init, false);
I know mostly how both of these methods work but
my question is what is going on here, what is the difference and which method is better etc?
And can someone please explain to me about this source.playbackState from an AudioBufferSourceNode Please? I never heard about that property on there before. It even doesn't have an article or get mentioned in the Mozilla MDN Website.
Also as a bonus question (which you don't have to answer), If both of these methods are useful then could it be possible to put them together as one if you know what I mean?
Sorry if that is a lot to ask. Thanks :)
resources:
https://paulbakaus.com/tutorials/html5/web-audio-on-ios/
https://github.com/Rich-Harris/phonograph/blob/master/src/init.ts
https://www.mattmontag.com/web/unlock-web-audio-in-safari-for-ios-and-macos

Both methods work, but I find the first (resume context in a user gesture) to be cleaner. The AudioBufferSource method is a kind of gross hack for backward compatibility with old sites that started playing buffers in a user gesture. This method doesn't work if you don't start the buffer from a gesture. (I think.)
Which one you want to use is up to you.

Web Audio Api precise looping in different browsers

So what I want is to have constant looping interchanging from different audio sources. For demo purpose I made a little puzzle game - you align numbers in order from 0 to 8 and depending on how you align them different loops are playing. I managed to get the result I want on Chrome Browser, but not on Safari or Firefox. I tried adding a different audio destination or multiple audio contexts but no matter what loop just stops after one iteration in Safari and other browsers except for Chrome.
Here is a link to the demo on code-pen Demo Puzzle with music
please turn down your sound as music might be a little too loud, I didn't master it. And here is basic code I have for Web Audio Api manipulation.
Thanks
*Also it does not work for mobile at all.
const AudioContext = window.AudioContext || window.webkitAudioContext;
var audioContext = new AudioContext();
const audio1 = document.getElementById("aud1");
const audio2 = document.getElementById("aud2");
const audio3 = document.getElementById("aud3");
const audio4 = document.getElementById("aud4");
var chosenTrack = audio2;
let gameStarted = false;
function startGame() {
document.getElementById("sHold").style.display = "none";
document.getElementById("container").style.display = "block";
gameStarted = true;
audioContext.resume();
audioContext = new AudioContext();
audio1.pause();
audio1.play();
audio1.currentTime = 0;
}
setInterval(function() {
if (gameStarted) {
//console.log(audioContext.currentTime );
if (audioContext.currentTime >= 6.4) {
audioContext = new AudioContext();
chosenTrack.pause();
chosenTrack.play();
chosenTrack.currentTime = 0;
}
}
}, 5);

Some thoughts:
You're not really using Web Audio this way, you're still using audio elements as the source which doesn't help if you want to be able to achieve precise timing. You should load them into AudioBuffers and play them using an AudioBufferSourceNode.
If you absolutely want to use audio elements (because the files you use are really massive and you want to stream them) you probably want to use the loop property on it although i doubt if that ends up being precise and gapless.
Never use setInterval to get a callback every frame, use requestAnimationFrame
Don't use setInterval OR requestAnimationFrame to be able to achieve precise audio looping, the javascript thread is not precise enough to do that AND can be held up when other things take a bit more time, too many enemies in screen for example. You should be scheduling ahead of time now and then: https://www.html5rocks.com/en/tutorials/audio/scheduling/
AudioBufferSourceNodes have a loop boolean property which will loop them as precise as possible
Do realise that different audio-decoders (so: different browsers) MIGHT decode audiofiles slightly differently: some may have a few more ms on the start for example. This might become an issue when using multiple looping AudioBufferSourceNodes, which may all be running out of sync after an x amount of time. I always reschedule something on the exact time needed instead of using the loop property.

Create Seamless Loop of Audio - Web

I want to create a seamless loop of an audio file. But in all approaches I used so far, there was a noticeable gap between end & start.
This is what I tried so far:
First approach was to use the audio in the HTML and it loops but there is still a noticeable delay when going from the end of the track to the beginning.
<audio loop autoplay>
<source src="audio.mp3" type="audio/mpeg">
<audio>
Then I tried it from JavaScript with the same result:
let myAudio = new Audio(file);
myAudio.loop = true;
myAudio.play();
After that I tried this (according to this answer)
myAudio.addEventListener(
'timeupdate',
function() {
var buffer = .44;
if (this.currentTime > this.duration - buffer) {
this.currentTime = 0;
this.play();
}
},
false
);
I played around with the buffer but I only got it to reduce the gap but not leave it out entirely.
I turned to the library SeamlessLoop (GitHub) and got it to work to loop seamlessly in Chromium browsers (but not in the latest Safari. Didn't test in other browsers). Code I used for that:
let loop = new SeamlessLoop();
// My File is 58 Seconds long. Btw there aren't any gaps in the file.
loop.addUri(file, 58000, 'sound1');
loop.callback(soundsLoaded);
function soundsLoaded() {
let n = 1;
loop.start('sound' + n);
}
EDIT: I tried another approach: Looping it trough two different audio elements:
var current_player = "a";
var player_a = document.createElement("audio");
var player_b = document.createElement("audio");
player_a.src = "sounds/back_music.ogg";
player_b.src = player_a.src;
function loopIt(){
var player = null;
if(current_player == "a"){
player = player_b;
current_player = "b";
}
else{
player = player_a;
current_player = "a";
}
player.play();
/*
3104.897 is the length of the audio clip in milliseconds.
Received from player.duration.
This is a different file than the first one
*/
setTimeout(loopIt, 3104.897);
}
loopIt();
But as milliseconds in browsers are not consistent or granular enough this doesn't work too well but it does work much better than the normal "loop" property of the audio.
Can anyone guide me into the right direction to loop the audio seamlessly?

You can use the Web Audio API instead. There are a couple of caveats with this, but it will allow you to loop accurately down to the single sample level.
The caveats are that you have to load the entire file into memory. This may not be practical with large files. If the files are only a few seconds it should however not be any problem.
The second is that you have to write control buttons manually (if needed) as the API has a low-level approach. This means play, pause/stop, mute, volume etc. Scanning and possibly pausing can be a challenge of their own.
And lastly, not all browsers support Web Audio API - in this case you will have to fallback to the regular Audio API or even Flash, but if your target is modern browsers this should not be a major problem nowadays.
Example
This will load a 4 bar drum-loop and play without any gap when looped. The main steps are:
It loads the audio from a CORS enabled source (this is important, either use the same domain as your page or set up the external server to allow for cross-origin usage as Dropbox does for us in this example).
AudioContext then decodes the loaded file
The decoded file is used for the source node
The source node is connected to an output
Looping is enabled and the buffer is played from memory.
var actx = new (AudioContext || webkitAudioContext)(),
src = "https://dl.dropboxusercontent.com/s/fdcf2lwsa748qav/drum44.wav",
audioData, srcNode; // global so we can access them from handlers
// Load some audio (CORS need to be allowed or we won't be able to decode the data)
fetch(src, {mode: "cors"}).then(function(resp) {return resp.arrayBuffer()}).then(decode);
// Decode the audio file, then start the show
function decode(buffer) {
actx.decodeAudioData(buffer, playLoop);
}
// Sets up a new source node as needed as stopping will render current invalid
function playLoop(abuffer) {
if (!audioData) audioData = abuffer; // create a reference for control buttons
srcNode = actx.createBufferSource(); // create audio source
srcNode.buffer = abuffer; // use decoded buffer
srcNode.connect(actx.destination); // create output
srcNode.loop = true; // takes care of perfect looping
srcNode.start(); // play...
}
// Simple example control
document.querySelector("button").onclick = function() {
if (srcNode) {
srcNode.stop();
srcNode = null;
this.innerText = "Play";
} else {
playLoop(audioData);
this.innerText = "Stop";
}
};
<button>Stop</button>

There is a very simple solution for that, just use loopify it makes use of the html5 web audio api and works perfectly well with many formats, not only wav as the dev says.
<script src="loopify.js" type="text/javascript"></script>
<script>
loopify("yourfile.mp3|ogg|webm|flac",ready);
function ready(err,loop){
if (err) {
console.warn(err);
}
loop.play();
}
</script>
This will automatically play the file, if you want to have start and stop buttons for example take a look at his demo

Is there a way to capture a browser's audio using Javascript or any other web language?

I want to make a simple website that displays interactive visuals and I would like for some of them to be audio-driven. I want the visitors to be able to drive the visuals with their own choice of music. I've had difficulty finding much documentation on anything other than getting audio input from microphones.
For example, my webpage (in tab 1) is running javascript code X that allows me to process the audio stream playing in another tab of the web browser. Is this possible?

If I understand your question correctly you want to capture audio from sources outside of the current tab.
There is no such api as part of the W3C the spec, probably due to the problems concerning user integrity that would arise if there was. As the browser is essentially executing foreign code browsers tend to be strictly sandboxed and requiring consent to prevent any violations of the users privacy.
If you or the user has access to the media try using the Audio and File APIs. If neither that nor using input audio capturing APIs as you mentioned is sufficient you'll either have to turn to creating a browser extension or to another platform altogether.

You can take microphone stream with WebRtc API
if (!navigator.getUserMedia)
navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia ||
navigator.mozGetUserMedia || navigator.msGetUserMedia;
if (navigator.getUserMedia) {
navigator.getUserMedia({audio:true}, success, function(e) {
});
}
...............
function success(e) {
audioContext = window.AudioContext || window.webkitAudioContext;
context = new audioContext();
// the sample rate is in context.sampleRate
audioInput = context.createMediaStreamSource(e);
var bufferSize = 2048;
recorder = context.createScriptProcessor(bufferSize, 1, 1);
recorder.onaudioprocess = function(e){
console.log ('recording');
var left = e.inputBuffer.getChannelData(0);
// stream
window.Stream.write(convertoFloat32ToInt16(left));
}
audioInput.connect(recorder)
recorder.connect(context.destination);
function convertoFloat32ToInt16(buffer) {
var l = buffer.length;
var buf = new Int16Array(l)
while (l--) {
buf[l] = buffer[l]*0xFFFF; //convert to 16 bit
}
return buf.buffer
}
If you like only visualization of audio stream you can use javascript plugins like enter link description here

WebAudio API "createOscillator" is not a function. Where's it gone?

Trying to make my first foray into the web audio API. At the moment
I'd love a simple audio equivalent of hello world. i.e. a 440HZ sine oscillator.
From the API it seems the function to create one is "createOscillator" which
doesn't seem to exist according to the browsers I'm using, (Canary and Firefox
latest builds). I thought it was obtained through the AudioContext Object like
createBufferSource() but it would appear not. Can anyone shed some light
the matter? Thanks in advance and please find my code below:
<script>
window.onload = init();
function init ()
{
if (typeof AudioContext == "function") {
var audioContext = new AudioContext();
} else if (typeof webkitAudioContext == "function") {
var audioContext = new webkitAudioContext();
}
var source = audioContext.createOscillator();
source.connect(audioContext.destination);
source.start();
}
</script>