Develop Reference

stream.addTrack working in Firefox but not Chrome

I'm having an issue with a project i'm working on for my brother's company and wanted to see if anybody had any ideas on how to fix it.
The company puts on virtual conferences for corporate events, nonprofits, etc. and its common with these things for people to need to pre-record their presentations with a powerpoint slideshow. For non-tech savvy clients (which is most of their clients), the only way to make this happen is to hire a contractor to work with with the client 1 on 1 to get their video recorded. My goal is to make a site where clients can easily record the video and slideshow themself, which would reduce their costs significantly.
Its still very much a work in progress, but what i have so far is live here: https://ezav-redesign-hf4d3.ondigitalocean.app/
The issue is, while the recording function works really well on Firefox, on Chrome it always ends up with no audio. You can see the full source code at the live site, but here is what I think should be the relevant part:
const video = document.createElement("video");
video.muted = true;
video.srcObject = stream;
video.play();
function update() {
ctx.drawImage(video, 300, 0, 720, 720, 1547, 297, 357, 355);
ctx.drawImage(
presentationCanvas,
0,
0,
presentationCanvas.width,
presentationCanvas.height,
20,
125,
presentationCanvas.width * 0.754,
presentationCanvas.height * 0.745
);
requestAnimationFrame(update); // wait for the browser to be ready to present another animation fram.
}
video.addEventListener("loadeddata", function () {
update(); //Start rendering
});
/* RECORDING */
const recStream = canvas.captureStream(30);
var audioCtx = new AudioContext();
// create a stream from our AudioContext
var dest = audioCtx.createMediaStreamDestination();
audioStream = dest.stream;
// connect our video element's output to the stream
var sourceNode = audioCtx.createMediaElementSource(video);
sourceNode.connect(dest);
recStream.addTrack(audioStream.getAudioTracks()[0]);
I've spent most of the weekend googling and tinkering trying to fix this and have tested it on 3 different machines but i can't seem to get anywhere. Im not sure how to pinpoint exactly where its going wrong as the console isnt diplaying any errors. Any ideas would be greatly appreciated. Thanks!

This is a known bug/limitation in Chrome, where they don't pass the audio stream of muted MediaElements to the graph at all anymore.
Luckily in your case it seems you absolutely don't need to go through the MediaElement since you have access to the raw MediaStream. So all you have to do is to get rid of the AudioContext part entirely and just do
stream.getAudioTracks().forEach( (track) => recStream.addTrack( track ) );

Web Audio Api precise looping in different browsers

So what I want is to have constant looping interchanging from different audio sources. For demo purpose I made a little puzzle game - you align numbers in order from 0 to 8 and depending on how you align them different loops are playing. I managed to get the result I want on Chrome Browser, but not on Safari or Firefox. I tried adding a different audio destination or multiple audio contexts but no matter what loop just stops after one iteration in Safari and other browsers except for Chrome.
Here is a link to the demo on code-pen Demo Puzzle with music
please turn down your sound as music might be a little too loud, I didn't master it. And here is basic code I have for Web Audio Api manipulation.
Thanks
*Also it does not work for mobile at all.
const AudioContext = window.AudioContext || window.webkitAudioContext;
var audioContext = new AudioContext();
const audio1 = document.getElementById("aud1");
const audio2 = document.getElementById("aud2");
const audio3 = document.getElementById("aud3");
const audio4 = document.getElementById("aud4");
var chosenTrack = audio2;
let gameStarted = false;
function startGame() {
document.getElementById("sHold").style.display = "none";
document.getElementById("container").style.display = "block";
gameStarted = true;
audioContext.resume();
audioContext = new AudioContext();
audio1.pause();
audio1.play();
audio1.currentTime = 0;
}
setInterval(function() {
if (gameStarted) {
//console.log(audioContext.currentTime );
if (audioContext.currentTime >= 6.4) {
audioContext = new AudioContext();
chosenTrack.pause();
chosenTrack.play();
chosenTrack.currentTime = 0;
}
}
}, 5);

Some thoughts:
You're not really using Web Audio this way, you're still using audio elements as the source which doesn't help if you want to be able to achieve precise timing. You should load them into AudioBuffers and play them using an AudioBufferSourceNode.
If you absolutely want to use audio elements (because the files you use are really massive and you want to stream them) you probably want to use the loop property on it although i doubt if that ends up being precise and gapless.
Never use setInterval to get a callback every frame, use requestAnimationFrame
Don't use setInterval OR requestAnimationFrame to be able to achieve precise audio looping, the javascript thread is not precise enough to do that AND can be held up when other things take a bit more time, too many enemies in screen for example. You should be scheduling ahead of time now and then: https://www.html5rocks.com/en/tutorials/audio/scheduling/
AudioBufferSourceNodes have a loop boolean property which will loop them as precise as possible
Do realise that different audio-decoders (so: different browsers) MIGHT decode audiofiles slightly differently: some may have a few more ms on the start for example. This might become an issue when using multiple looping AudioBufferSourceNodes, which may all be running out of sync after an x amount of time. I always reschedule something on the exact time needed instead of using the loop property.

Web Audio API demo doesn't work on iOS

I'm currently working on adapting this web audio API demo for a project that I am working on, but there is no sound when I test on an iPhone. It works fine on the iPad.
I've searched for solutions and found this thread on StackOverflow with the following snippet of one of the answers:
Safari on iOS 6 effectively starts with the Web Audio API muted. It
will not unmute until you attempt to play a sound in a user input
event (create a buffer source, connect it to destination, and call
noteOn()). After this, it unmutes and audio plays unrestricted and as
it ought to. This is an undocumented aspect of how the Web Audio API
works on iOS 6 (Apple's doc is here, hopefully they update it with a
mention of this soon!)
The user input event should be the onclick event on the play button but changing to use noteOn() instead of start() still doesn't fix it.
Update: I've also tried binding the play button with the touchend event but to no avail.
Here is the function that uses noteOn():
function playNote(buffer, pan, x, y, z, sendGain, mainGain, playbackRate, noteTime) {
// Create the note
var voice = context.createBufferSource();
voice.buffer = buffer;
voice.playbackRate.value = playbackRate;
// Optionally, connect to a panner
var finalNode;
if (pan) {
var panner = context.createPanner();
panner.panningModel = "HRTF";
panner.setPosition(x, y, z);
voice.connect(panner);
finalNode = panner;
} else {
finalNode = voice;
}
// Connect to dry mix
var dryGainNode = context.createGain();
dryGainNode.gain.value = mainGain * effectDryMix;
finalNode.connect(dryGainNode);
dryGainNode.connect(masterGainNode);
// Connect to wet mix
var wetGainNode = context.createGain();
wetGainNode.gain.value = sendGain;
finalNode.connect(wetGainNode);
wetGainNode.connect(convolver);
if (iOS) {
voice.noteOn(noteTime);
}
else {
voice.start(noteTime);
}
}
Any suggestions would be greatly appreciated. Thanks.

I feel really stupid. Apparently, if you have your iPhone on vibrate mode, the sound doesn't play.

The start() method should work fine without the if else statements on iOS as long as you call the function with a user interaction event. Also flip the order you pass y and z to the panner cause z is second for some strange reason.
Here's a working example, change stuff in it to fit what you need, most isn't need and I've got others somewhere that use the dom to add event listeners
<script>
var audioCtx = new (window.AudioContext || window.webkitAudioContext)();
var oscillator = audioCtx.createOscillator();
var gainNode = audioCtx.createGain();
oscillator.connect(gainNode);
gainNode.connect(audioCtx.destination);
oscillator.type = 'sine';
oscillator.frequency.value = 440;
gainNode.gain.value = 1;
</script>
<button onclick="oscillator.start();">play</button>

My own experience has been that sometimes the Web Audio API works on iPhones, sometimes it doesn't. Here is a page that worked 5 minutes ago on my iPhone 6s; 1 minute ago it didn't work; now it does again!
http://www.stephenandrewtaylor.net/dna-sonification/
Here is another one that works intermittently; it worked 2 minutes ago and now it doesn't (the animations work, there is just no audio).
http://www.stephenandrewtaylor.net/lie-sonification/
It might have to do with how many tabs are open in Safari; you could try closing some of your open tabs (right now I have 5 tabs, including the lie-sonificaton page which worked 2 minutes ago but now doesn't). I am also a novice programmer and I'm sure there are much better ways I could be writing the code.

Webaudio sound stops on Chrome for Android after about 2 minutes

I'm running into an issue with WebAudio on Chrome for Android.
I'm experiencing this on a Samsung Galaxy S3 (GT-I9300) with:
Chrome version 44.0.2403.133
Android 4.3.0
Here is the code I'm using to try and isolate the issue:
var audioContext;
if(window.AudioContext) {
audioContext = new AudioContext();
}
var startTime = Date.now();
var lastTrigger;
var gain = audioContext.createGain();
gain.gain.value = 1;
gain.connect(audioContext.destination);
var buttonTrigger = document.getElementById('trigger');
buttonTrigger.addEventListener('click', function(event) {
var oscillator = audioContext.createOscillator();
oscillator.type = "square";
oscillator.frequency.value = 100 + (Math.cos(audioContext.currentTime)*100);
oscillator.connect(gain);
oscillator.start(0);
oscillator.stop(audioContext.currentTime+0.1);
lastTrigger = Date.now();
});
var timer = document.getElementById('timer');
setInterval(function() {
if(lastTrigger) { timer.textContent = Date.now() - lastTrigger; }
}, 1000);
And here it is on jsfiddle
This simply creates an oscillator node and plays on clicking a button. On my phone, if I do not click the button for about a minute and a half or two minutes, I no longer get any sound.
There are no errors thrown.
Does anyone have any experience of this issue and a possible workaround?
This issue originally appeared in a much larger app using Phaser to play sounds from a m4a file, so this is not solely to do with the oscillator.
UPDATE
According to the Chromium bug ticket this issue has now been fixed.

After experiencing the same problem on Android. I found a better solution than playing a "dummy sound" every 30sec.
Just remember the time when you last played over your context:
var lastPlayed = new Date().getTime();
var audioSource = context.createBufferSource();
audioSource.connect( context.destination );
audioSource.buffer = sb;
audioSource.start( 0 );
The next time you play a sample/sound Just check the time passed and reset the AudioContext
if(new Date().getTime()-lastPlayed>30000){ // Time passed since last playing is greater than 30 secs
context.close();
context=new AudioContext();
}
For Android this works like charm.

I think what you're seeing is an auto-shutdown of Web Audio when there's no sound for a while. What happens if you click the button a second time, a second or so after the first? (Web Audio can take some time (order of tens of milliseconds, at least) to restart.)
The suspend()/resume() methods, and looking at the context.state, would be helpful here.

#RaymondToy's comment answers this question. There is a bug with Chrome on Android (at least for Samsung Galaxy S3/4). Webaudio stops playing sounds after a period of inactivity. Where inactivity is essentially silence.
The only work around I can find is to play some kind of sound at intervals. I have experimented with playing a sound every 30 seconds and that stopped the problem.
I also tried playing some kind of silent noise (silent audio buffer or silent part of an m4a audio file or muted sound), neither of which solved the problem.

poor quality audio playback in javascript windows store app

I'm programming a Windows Store app using javascript and having trouble playing audio without poor quality. Usually when a sound plays it sounds as if it is skipping the first few hundred milliseconds of the sound (and sometimes more), resulting in clicks and sometimes almost no sound (because the sound files are short). I have 8 short files (27kb .mp3). The weirdest thing is that the volume of the playback also seems to randomly vary between a quiet and loud level. I'm using a Windows 8.1 tablet with a 1.6GHz processor.
//I declare like this:
var playList = new Array("chord1.mp3", "chord2.mp3", "chord3.mp3", "chord4.mp3", "chord5.mp3", "chord6.mp3", "chord7.mp3", "chord8.mp3" );
var playListIndex = 0;
var players = new Array();
//I initialise like this:
for (var i = 0; i < playList.length; i++) {
players[i] = new Audio();
players[i].src = playList[i];
}
//When it's time to play (listening for pointerdown events) I do this:
players[playListIndex].play();
//I have tried this but it just introduces latency without solving the problem
//new Audio(playList[playListIndex]).play();
//This happens after playing to make the sounds play sequentially.
//The problems occur even when there is no overlap of playback
playListIndex++;
if (playListIndex >= playList.length) {
playListIndex = 0;
}
There is almost nothing else going on in the app. Suggestions much appreciated.