A notable issue that's appearing as I'm building a simple audio streaming element in HTML5 is that the <audio> tag doesn't behave as one would expect in regards to playing and pausing a live audio stream.
I'm using the most basic HTML5 code for streaming the audio, an <audio> tag with controls, the source of which is a live stream.
Current outcome: When the stream is first played, it plays whatever is streaming as expected. When it's paused and played again, however, the audio resumes exactly where it left off when the stream was previously paused. The user is now listening to a delayed version of the stream. This occurrence isn't browser-specific.
Desired outcome: When the stream is paused, I want the stream to stop. When it is played again, I want it resume where the stream is currently at, not where it was when the user paused the stream.
Does anyone know of a way to make this audio stream resume properly after it's been paused?
Some failed attempts I've made to fix this issue:
Altering the currentTime of the audio element does nothing to streaming audio.
I've removed the audio element from the DOM when the user stops stream playback and added it back in when user resumes playback. The stream still continues where the user left off and worse yet downloads another copy of the stream behind the scenes.
I've added a random GET variable to the end of the stream URL every time the stream is played in an attempt to fool the browser into believing that it's playing a new stream. Playback still resumes where the user paused the stream.
Best way to stop a stream, and then start it again seems to be removing the source and then calling load:
var sourceElement = document.querySelector("source");
var originalSourceUrl = sourceElement.getAttribute("src");
var audioElement = document.querySelector("audio");
function pause() {
sourceElement.setAttribute("src", "");
audioElement.pause();
// settimeout, otherwise pause event is not raised normally
setTimeout(function () {
audioElement.load(); // This stops the stream from downloading
});
}
function play() {
if (!sourceElement.getAttribute("src")) {
sourceElement.setAttribute("src", originalSourceUrl);
audioElement.load(); // This restarts the stream download
}
audioElement.play();
}
Resetting the audio source and calling the load() method seems to be the simplest solution when you want to stop downloading from the stream.
Since it's a stream, the browser will stop downloading only when the user gets offline. Resetting is necessary to protect your users from burning through their cellular data or to avoid serving outdated content that the browser downloaded when they paused the audio.
Keep in mind though that when the source attribute is set to an empty string, like so audio.src = "", the audio source will instead be set to the page's hostname. If you use a random word, that word will be appended as a path.
So as seen below, setting audio.src ="", means that audio.src === "https://stacksnippets.net/js". Setting audio.src="meow" will make the source be audio.src === "https://stacksnippets.net/js/meow" instead. Thus the 3d paragraph is not visible.
const audio1 = document.getElementById('audio1');
const audio2 = document.getElementById('audio2');
document.getElementById('p1').innerHTML = `First audio source: ${audio1.src}`;
document.getElementById('p2').innerHTML = `Second audio source: ${audio2.src}`;
if (audio1.src === "") {
document.getElementById('p3').innerHTML = "You can see me because the audio source is set to an empty string";
}
<audio id="audio1" src=""></audio>
<audio id="audio2" src="meow"></audio>
<p id="p1"></p>
<p id="p2"></p>
<p id="p3"></p>
Be aware of that behavior if you do rely on the audio's source at a given moment. Using the about URI scheme seems to trick it into behaving in a more reliable way. So using "about:" or "about:about", "about:blank", etc. will work fine.
const resetAudioSource = "about:"
const audio = document.getElementById('audio');
audio.src = resetAudioSource;
document.getElementById('p1').innerHTML = `Audio source: -- "${audio.src}"`;
// Somewhere else in your code...
if (audio.src === resetAudioSource){
document.getElementById('p2').innerHTML = "You can see me because you reset the audio source."
}
<audio id="audio"></audio>
<p id="p1"></p>
<p id="p2"></p>
Resetting the audio.src and calling the .load() method will make the audio to try to load the new source. The above comes in handy if you want to show a spinner component while the audio is loading, but don't want to also show that component when you reset your audio source.
A working example can be found here: https://jsfiddle.net/v2xuczrq/
If the source is reset using a random word, then you might end up with the loader showing up when you also pause the audio, or until the onError event handler catches it. https://jsfiddle.net/jcwvue0s/
UPDATE: The strings "javascript:;" and "javascript:void(0)" can be used instead of the "about:" URI and this seems to work even better as it will also stop the console warnings caused by "about:".
Note: I'm leaving this answer for the sake of posterity, since it was the best solution I or anyone could come up with at the time for my issue. But I've since marked Ciantic's later idea as the best solution because it actually stops the stream downloading and playback like I originally wanted. Consider that solution instead of this one.
One solution I came up with while troubleshooting this issue was to ignore the play and pause functions on the audio element entirely and just set the volume property of the audio element to 0 when user wishes to stop playback and then set the volume property back to 1 when the user wishes to resume playback.
The JavaScript code for such a function would look much like this if you're using jQuery (also demonstrated in this fiddle):
/*
* Play/Stop Live Audio Streams
* "audioElement" should be a jQuery object
*/
function streamPlayStop(audioElement) {
if (audioElement[0].paused) {
audioElement[0].play();
} else if (!audioElement[0].volume) {
audioElement[0].volume = 1;
} else {
audioElement[0].volume = 0;
}
}
I should caution that even though this achieves the desired functionality for stopping and resuming live audio streams, it isn't ideal because the stream, when stopped, is actually still playing and being downloaded in the background, using up bandwidth in the process.
However, this solution doesn't necessarily take up more bandwidth than just using .play() and .pause() on a streaming audio element. Simply using the audio tag with streaming audio uses up a great deal of bandwidth anyway, because once streaming audio is played, it continues to download the contents of the stream in the background when it is paused.
It should be noted that this method won't work in iOS because of purposefully built-in limitations for iPhones and iPads:
On iOS devices, the audio level is always under the user’s physical control. The volume property is not settable in JavaScript. Reading the volume property always returns 1.
If you choose to use the workaround in this answer, you'll need to create a fallback for iOS devices that uses the play() and pause() functions normally, or your interface will be unable to pause the stream.
Tested #Ciantics code and it worked with some modifications, if you want to use multiple sources.
As the source is getting removed, the HTML audio player becomes inactive, so the source (URL) needs to be added directly after again to become active.
Also added an event listener at the end to connect the function when pausing:
var audioElement = document.querySelector("audio");
var sources = document.querySelector("audio").children;
var sourceList = [];
for(i=0;i<sources.length;i++){
sourceList[i] = sources[i].getAttribute("src");
}
function pause() {
for(i=0;i<sources.length;i++){
sources[i].setAttribute("src", "");
}
audioElement.pause();
// settimeout, otherwise pause event is not raised normally
setTimeout(function () {
audioElement.load(); // This stops the stream from downloading
});
for(i=0;i<sources.length;i++){
if (!sources[i].getAttribute("src")) {
sources[i].setAttribute("src", sourceList[i]);
audioElement.load(); // This restarts the stream download
}
}
}
audioElement.addEventListener("pause", pause);
Related
I'm trying to add filter effects to an audio stream I have playing on my website. I'm able to connect the Tone.js library to the audio stream but I'm not hearing any changes in the audio stream playing on the website. I'm not seeing any errors in the console and I've tried adjusting the filter from 50 to 5000 but nothing seems to have any effect on the audio. Do I need to set up the new Tone.Player() to actually hear the audio? If so, how do you go about setting up the Player if there is no src for the existing audio element.
$('#testBtn').click(async function () {
const audioElement = document.getElementById('theAudioStream');
const mediaElementSource = Tone.context.createMediaElementSource(audioElement);
const filter = new Tone.Filter(50, 'lowpass').toDestination();
Tone.connect(mediaElementSource, filter);
await Tone.start();
console.log("Started?");
});
The audio stream I'm trying to modify is set up from a JsSip call. The code to start the stream is as follows:
var audioStream = document.getElementById('theAudioStream')
//further down in code
currentSession.answer(options);
if (currentSession.connection) {
currentSession.connection.ontrack = function (e) {
audioStream.srcObject = e.streams[0];
audioStream.play();
}
}
I click the test button after the call has started so I know the audio stream is present before initializing the Tone.js Filters
Working solution:
Removing the audioStream.play() from where the JsSIP call is answered solves the issue.
I don't know the exact reason why this solves the issue (it might even be a workaround) but after much trial and error this way allows the audio to be available to ToneJS for effecting.
Any other solutions are welcome.
Safari on iOS puts a scrubber on its lock screen for simple HTMLAudioElements. For example:
const a = new Audio();
a.src = 'https://example.com/audio.m4a'
a.play();
JSFiddle: https://jsfiddle.net/0seckLfd/
The lock screen will allow me to choose a position in the currently playing audio file.
How can I disable the ability for the user to scrub the file on the lock screen? The metadata showing is fine, and being able to pause/play is also acceptable, but I'm also fine with disabling it all if I need to.
DISABLE Player on lock screen completely
if you want to completely remove the lock screen player you could do something like
const a = new Audio();
document.querySelector('button').addEventListener('click', (e) => {
a.src = 'http://sprott.physics.wisc.edu/wop/sounds/Bicycle%20Race-Full.m4a'
a.play();
});
document.addEventListener('visibilitychange', () => {
if (document.hidden) a.src = undefined
})
https://jsfiddle.net/5s8c9eL0/3/
that is stoping the player when changing tab or locking screen
(code to be cleaned improved depending on your needs)
From my understanding you can't block/hide the scrubbing commands unless you can tag the audio as a live stream. That being said, you can use js to refuse scrubbing server-side. Reference the answer here. Although that answer speaks of video, it also works with audio.
The lock screen / control center scrubber can also be avoided by using Web Audio API.
This is an example of preloading a sound and playing it, with commentary and error handling:
try {
// <audio> element is simpler for sound effects,
// but in iOS/iPad it shows up in the Control Center, as if it's music you'd want to play/pause/etc.
// Also, on subsequent plays, it only plays part of the sound.
// And Web Audio API is better for playing sound effects anyway because it can play a sound overlapping with itself, without maintaining a pool of <audio> elements.
window.audioContext = window.audioContext || new AudioContext(); // Interoperate with other things using Web Audio API, assuming they use the same global & pattern.
const audio_buffer_promise =
fetch("audio/sound.wav")
.then(response => response.arrayBuffer())
.then(array_buffer => audioContext.decodeAudioData(array_buffer))
var play_sound = async function () {
audioContext.resume(); // in case it was not allowed to start until a user interaction
// Note that this should be before waiting for the audio buffer,
// so that it works the first time (it would no longer be "within a user gesture")
// This only works if play_sound is called during a user gesture (at least once), otherwise audioContext.resume(); needs to be called externally.
const audio_buffer = await audio_buffer_promise; // Promises can be awaited any number of times. This waits for the fetch the first time, and is instant the next time.
// Note that if the fetch failed, it will not retry. One could instead rely on HTTP caching and just fetch() each time, but that would be a little less efficient as it would need to decode the audio file each time, so the best option might be custom caching with request error handling.
const source = audioContext.createBufferSource();
source.buffer = audio_buffer;
source.connect(audioContext.destination);
source.start();
};
} catch (error) {
console.log("AudioContext not supported", error);
play_sound = function() {
// no-op
// console.log("SFX disabled because AudioContext setup failed.");
};
}
I did a search, in search of a way to help you, but I did not find an effective way to disable the commands, however, I found a way to customize them, it may help you, follow the apple tutorial link
I think what's left to do now is wait, see if ios 13 will bring some option that will do what you want.
Safari on iOS puts a scrubber on its lock screen for simple HTMLAudioElements. For example:
const a = new Audio();
a.src = 'https://example.com/audio.m4a'
a.play();
JSFiddle: https://jsfiddle.net/0seckLfd/
The lock screen will allow me to choose a position in the currently playing audio file.
How can I disable the ability for the user to scrub the file on the lock screen? The metadata showing is fine, and being able to pause/play is also acceptable, but I'm also fine with disabling it all if I need to.
DISABLE Player on lock screen completely
if you want to completely remove the lock screen player you could do something like
const a = new Audio();
document.querySelector('button').addEventListener('click', (e) => {
a.src = 'http://sprott.physics.wisc.edu/wop/sounds/Bicycle%20Race-Full.m4a'
a.play();
});
document.addEventListener('visibilitychange', () => {
if (document.hidden) a.src = undefined
})
https://jsfiddle.net/5s8c9eL0/3/
that is stoping the player when changing tab or locking screen
(code to be cleaned improved depending on your needs)
From my understanding you can't block/hide the scrubbing commands unless you can tag the audio as a live stream. That being said, you can use js to refuse scrubbing server-side. Reference the answer here. Although that answer speaks of video, it also works with audio.
The lock screen / control center scrubber can also be avoided by using Web Audio API.
This is an example of preloading a sound and playing it, with commentary and error handling:
try {
// <audio> element is simpler for sound effects,
// but in iOS/iPad it shows up in the Control Center, as if it's music you'd want to play/pause/etc.
// Also, on subsequent plays, it only plays part of the sound.
// And Web Audio API is better for playing sound effects anyway because it can play a sound overlapping with itself, without maintaining a pool of <audio> elements.
window.audioContext = window.audioContext || new AudioContext(); // Interoperate with other things using Web Audio API, assuming they use the same global & pattern.
const audio_buffer_promise =
fetch("audio/sound.wav")
.then(response => response.arrayBuffer())
.then(array_buffer => audioContext.decodeAudioData(array_buffer))
var play_sound = async function () {
audioContext.resume(); // in case it was not allowed to start until a user interaction
// Note that this should be before waiting for the audio buffer,
// so that it works the first time (it would no longer be "within a user gesture")
// This only works if play_sound is called during a user gesture (at least once), otherwise audioContext.resume(); needs to be called externally.
const audio_buffer = await audio_buffer_promise; // Promises can be awaited any number of times. This waits for the fetch the first time, and is instant the next time.
// Note that if the fetch failed, it will not retry. One could instead rely on HTTP caching and just fetch() each time, but that would be a little less efficient as it would need to decode the audio file each time, so the best option might be custom caching with request error handling.
const source = audioContext.createBufferSource();
source.buffer = audio_buffer;
source.connect(audioContext.destination);
source.start();
};
} catch (error) {
console.log("AudioContext not supported", error);
play_sound = function() {
// no-op
// console.log("SFX disabled because AudioContext setup failed.");
};
}
I did a search, in search of a way to help you, but I did not find an effective way to disable the commands, however, I found a way to customize them, it may help you, follow the apple tutorial link
I think what's left to do now is wait, see if ios 13 will bring some option that will do what you want.
I just built a real-time app using socket.io where a "master" user can trigger sounds on receiving devices (desktop browsers, mobile browsers). That master user sees a list of sound files, and can click "Play" on a sound file.
The audio playback is instant on browsers. On mobiles however, there is a 0.5-2 seconds delay (my Nexus 4 and iPhone 5 about 1 second and iPhone 3GS 1-2 seconds).
I've tried several things to optimize the audio playback to make it faster on mobiles. Right now (at the best "phase" of its optimization I'd say), I combine all the mp3's together in one audio file (it creates .mp3, .ogg, and .mp4 files). I need ideas on how I can further fix / improve this issue. The bottleneck really seems to be in the hmtl 5 audio methods such as .play().
On the receivers I use as such:
<audio id="audioFile" preload="auto">
<source src="/output.m4a" type="audio/mp4"/>
<source src="/output.mp3" type="audio/mpeg"/>
<source src="/output.ogg" type="audio/ogg"/>
<p>Your browser does not support HTML5 audio.</p>
</audio>
In my JS:
var audioFile = document.getElementById('audioFile');
// Little hack for mobile, as only a user generated click will enable us to play the sounds
$('#prepareAudioBtn').on('click', function () {
$(this).hide();
audioFile.play();
audioFile.pause();
audioFile.currentTime = 0;
});
// Master user triggered a sound sprite to play
socket.on('playAudio', function (audioClip) {
if (audioFile.paused)
audioFile.play();
audioFile.currentTime = audioClip.startTime;
// checks every 750ms to pause the clip if the endTime has been reached.
// There is a second of "silence" between each sound sprite so the pause is sure to happen at a correct time.
timeListener(audioClip.endTime);
});
function timeListener(clipEndTime) {
this.clear = function () {
clearInterval(interval);
interval = null;
};
if (interval !== null) {
this.clear();
}
interval = setInterval(function () {
if (audioFile.currentTime >= clipEndTime) {
audioFile.pause();
this.clear();
}
}, 750);
}
Also considered blob for each sound but some sounds can go for minutes so that's why I resorted to combining all sounds together for 1 big audio file (better than several audio tags on the page for each clip)
Instead of pausing / playing, I simply set the volume to 0 when it shouldn't be playing, and back to 1 when it should be playing. The Audio methods currentTime and volume don't slow the audio playback at all even on an iPhone 3GS.
I also added the 'loop' attribute to the audio element so it never has to be .play()'ed again.
It was fruitful to combine all mp3 sounds together because this solutions can work because of that.
Edit: audioElement.muted = true or audioElement.muted = false makes more sense.
Edit2: Can't control volume on user's behalf on iOS so I must pause() and play() the audio element as opposed to just muting and unmuting it.
Your setup is working well on desktop because of the preload attribute.
Unfortunately, here's Apple on the subject of preload:
Safari on iOS never preloads.
And here's MDN:
Note: This value is often ignored on mobile platforms.
The mobile platforms are making a tradeoff to save battery and data usage to only load media when it's actually interacted with by the user or programmatically played (autoplay generally doesn't work for similar reasons).
I think the best you're going to do is combining your tracks together, as you said you've done, so you don't have to pay the initial load-up "cost" as much.
I was having the same delay issue when testing in mobile. I found out what some HTML 5 games are using for audio since games demand very low latencies. Some are using SoundJS. I recommend you try that library out.
You can find a speed comparison between using the HTML Audio tag vs using SoundJS here:
http://www.nickfrazier.com/javascript/audio/ui/2016/08/14/js-sound-libraries.html
(test in mobile to hear the difference)
From my tests SoundJS is much faster.
In fact, it's Good enough to be used in a game, or for sound feedback in a user interface.
Old question but here is my solution using one of the answer above:
const el = document.createElement("audio");
el.muted = true;
el.loop = true;
const source = document.createElement("source");
source.src = lineSe;
source.type = "audio/mpeg";
el.appendChild(source);
// need to call this function after user first interaction, or safari won't do it.
function firstPlay() {
el.play();
}
let timeout = null;
function play() {
// In case user press the button too fast, cancel last timeout
if (lineSeTimeout) {
clearTimeout(timeout);
}
// Back to beginning
el.currentTime = 0;
// unmute
el.muted = false;
// set to mute after the audio finish. In my case 500ms later
// onended event won't work because loop=tue
timeout = setTimeout(() => {
// mute audio again
el.muted = true;
}, 500);
}
So, I am building a web app that has a div with text that changes on various user actions (it's stepping through an array of pieces of text). I'm trying to add audio to it, so I made another array with the sound files in the appropriate positions:
var phrases=['Please hand me the awl.','etc1','etc2','etc3'];
var phrasesAudio=['awl.mp3','etc1.mp3','etc2.mp3','etc3.mp3'];
And on each action completion, a 'counter' variable in incremented, and each array looks for the object at that counter
var audio = document.createElement("audio"),
canPlayMP3 = (typeof audio.canPlayType === "function" &&
audio.canPlayType("audio/mpeg") !== "");
function onAction(){
correct++;
document.getElementById('speech').innerHTML=phrases[correct];
if(canPlayMP3){
snd = new Audio(phrasesAudio[correct]);
}
else{
snd = new Audio(phrasesAudioOgg[correct]);
}
snd.play();
}
(the text replaces a div's HTML and I use .play() for the audio object)...usually works okay (and ALWAYS does in a 'real' browser), but on the iPad (the actual target device) after a few successful iterations, the TEXT continues to progress accurately, but the AUDIO will hiccup and repeat a sound file one or more times. I added some logging and looking there it reports that it's playing screw.mp3 (just an example, no particular file is more or less error prone), but in actuality it plays screwdriver.mp3 (the prior file, if there is an error, the audio always LAGS, never LEADS)...because of this I am thinking that the problem is with my use of .play()...so I tried setting snd=null; between each sound, but nothing changed...Any suggestions for how to proceed? This is my first use of audio elements so any advice is appreciated. Thanks.
edit: I've also tried setting the files with snd.src (based on https://wiki.mozilla.org/Audio_Data_API) and this caused no audio to play
for iPad you need to call snd.load() before snd.play() - otherwise you get "odd" behaviour...
see for some insight:
http://jnjnjn.com/187/playing-audio-on-the-ipad-with-html5-and-javascript/
Autoplay audio files on an iPad with HTML5
EDIT - as per comment from the OP:
Here https://developer.mozilla.org/En/Using_audio_and_video_in_Firefox you can find a tip on halting a currently playing piece of media with
var mediaElement = document.getElementById("myMediaElementID");
mediaElement.pause();
mediaElement.src = "";
and, following that with setting the correct src, load(), play() works great