I am trying to write a small library for convenient manipulations with audio. I know about the autoplay policy for media elements, and I play audio after a user interaction:
const contextClass = window.AudioContext || window.webkitAudioContext;
const context = this.audioContext = new contextClass();
if (context.state === 'suspended') {
const clickCb = () => {
this.playSoundsAfterInteraction();
window.removeEventListener('touchend', clickCb);
this.usingAudios.forEach((audio) => {
if (audio.playAfterInteraction) {
const promise = audio.play();
if (promise !== undefined) {
promise.then(_ => {
}).catch(error => {
// If playing isn't allowed
console.log(error);
});
}
}
});
};
window.addEventListener('touchend', clickCb);
}
On android chrome everything ok and on a desktop browser. But on mobile Safari I am getting such error in promise:
the request is not allowed by the user agent or the platform in the current context safari
I have tried to create audios after an interaction, change their "src" property. In every case, I am getting this error.
I just create audio in js:
const audio = new Audio(base64);
add it to array and try to play. But nothing...
Tried to create and play after a few seconds after interaction - nothing.
Related
I am trying to create a web application with some video chat functionality, but trying to get it to work on mobile (specifically, Chrome for iOS) is giving me fits.
What I would like to do is have users be able to join a game, and join a team within that game. There are two tabs on the page for players - a "Team" tab and a "Game" tab. When the player selects the game tab, they may talk to all participants in the entire game (e.g. to ask the host/moderator a question). When the team tab is selected, the player's stream to the game is muted, and only the player's team can hear them talk. As a result, I believe I need two MediaStream objects for each player - one to stream to the game, and one to stream to the player's team - this way, I can mute one while keeping the other unmuted.
There is an iOS quirk where you can only call the getUserMedia() function once, so I need to clone the stream using MediaStream.clone(). AddVideoStream is a function that just adds the video to the appropriate grid of videos, and it appears to work properly.
The problem is - when I use my iPhone 12 to connect to the game, I can see my video just fine, but when I click over to the "game" tab and look at the second stream, the stream works for a second, and then freezes. The weird thing is, if I open a new tab in Chrome, and then go back to the game tab, both videos seem to run smoothly.
Has anyone ever tried something similar, and figured out why this behavior occurs?
const myPeer = new Peer(undefined);
myPeer.on('open', (userId) => {
myUserId = userId;
console.log(`UserId: ${myUserId}`);
socket.emit('set-peer-id', {
id: userId,
});
});
const myVideo = document.createElement('video');
myVideo.setAttribute('playsinline', true);
myVideo.muted = true;
const myTeamVideo = document.createElement('video');
myTeamVideo.setAttribute('playsinline', true);
myTeamVideo.muted = true;
const myStream =
// (navigator.mediaDevices ? navigator.mediaDevices.getUserMedia : undefined) ||
navigator.mediaDevices ||
navigator.webkitGetUserMedia ||
navigator.mozGetUserMedia;
let myVideoStream;
let myTeamStream;
if (myStream) {
myStream
.getUserMedia({
video: true,
audio: true,
})
.then((stream) => {
myVideoStream = stream;
myTeamStream = stream.clone();
addVideoStream(myTeamVideo, myTeamStream, myUserId, teamVideoGrid);
addVideoStream(myVideo, myVideoStream, myUserId, videoGrid);
myPeer.on('call', (call) => {
call.answer(stream);
const video = document.createElement('video');
video.setAttribute('playsinline', true);
call.on('stream', (userVideoStream) => {
const teammate = teammates.find((t) => {
return t.peerId === call.peer;
});
if (teammate) {
addVideoStream(
video,
userVideoStream,
call.peer,
teamVideoGrid,
teammate.name
);
} else {
addVideoStream(video, userVideoStream, call.peer, videoGrid);
}
});
call.on('close', () => {
console.log(`Call with ${call.peer} closed`);
});
});
socket.on('player-joined', (data) => {
addMessage({
name: 'System',
isHost: false,
message: `${data.name} has joined the game.`,
});
if (data.id !== myUserId) {
if (data.teamId !== teamId) {
connectToNewUser(data.peerId, myVideoStream, videoGrid, data.name);
} else {
connectToNewUser(
data.peerId,
myTeamStream,
teamVideoGrid,
data.name
);
}
}
});
});
}
I'm trying to make a script that gets the audio from an audio element and plays it on headset and laptop speakers at he same time.
Is this possible?
This is the code that I wrote so far:
const outputs = (await navigator.mediaDevices.enumerateDevices()).filter(({ kind, deviceId }) => kind === 'audiooutput' && deviceId !== 'default' && deviceId !== 'communications' );
var players = [];
document.querySelectorAll('audio').forEach((y)=>{
players.push([]); let l = players.length-1;
outputs.forEach((x)=>{
if(players[l].length===0) players[l].push(y);
else {
let p = y.cloneNode();
p.srcObject = players[l][0].srcObject.clone();
players[l].push(p);
}
});
})
players.forEach((a)=>{
a.forEach((o, i)=>{
o.setSinkId(outputs[i].deviceId);
o.play();
})
})
The issue with this code is that it makes the audio only play on the other speaker instead of playing it on both.
Note that the window has access to mic so I can see all the devices from navigator.mediaDevices.enumerateDevices().
The script is intended to work mainly on Edge and Chrome.
I found a solution!
Since I wasn't able to play sound on multiple devices at once in the same window I managed to get this done by creating iframes that contain audio elements.
Then I set to all of those audio elements the same source and I set them to output audio a specific device and I sync the play/stop with event listeners.
This works because iframes are separated by the main window. Actually I think they get their own process (on chromium browsers).
My code looks like this: (note I'm not using static sources, but srcObjects)
navigator.mediaDevices.getUserMedia({audio: true}).then(s=>{
s.getTracks().forEach(x=>x.stop()); //stop mic use because we need only outputs
navigator.mediaDevices.enumerateDevices().then(o=>{
const outputs = o.filter(({ kind, deviceId }) => kind === 'audiooutput' && deviceId !== 'default' && deviceId !== 'communications');
let audioSrc = getAudioSrc(), players = [];
audioSrc.src.pause(); // Pause source to start audio in sync ?
audioSrc.src.addEventListener("pause", () => players.forEach(x=>x.pause()));
audioSrc.src.addEventListener("play", () => players.forEach(x=>x.play()));
outputs.forEach((x)=>{
let ifrm = makeIFrame(), audioEl = makeAudio(ifrm.document, audioSrc.s, x.deviceId);
players.push(audioEl);
});
});
}).catch(e=>console.log(e));
I have an html5 video element I'm trying to increase the volume off.
I'm using the code I found in this answer
However there is no sound coming out of the speakers. If I disable it sound is fine.
videoEl.muted = true //tried with this disabled or enabled
if(!window.audio)
window.audio = amplify(vol)
else
window.audio.amplify(vol)
...
export function amplify(multiplier) {
const media = document.getElementById('videoEl')
//#ts-ignore
var context = new(window.AudioContext || window.webkitAudioContext),
result = {
context: context,
source: context.createMediaElementSource(media),
gain: context.createGain(),
media,
amplify: function(multiplier) {
result.gain.gain.value = multiplier;
},
getAmpLevel: function() {
return result.gain.gain.value;
}
};
result.source.connect(result.gain)
result.gain.connect(context.destination)
result.amplify(multiplier)
return result;
}
That value is set to 3 for testing.
Any idea how why I'm getting no sound?
I also have Howler running for other audio files, could it be blocking the web audio API?
I'm trying to setup audio playback which I cannot get working on Safari 14.0.3, but works fine in Chrome 88.0.4324.146. I have a function that returns a AudioContext or webkitAudioContext. I followed this answer: https://stackoverflow.com/a/29373891/586006
var sounds;
var audioContext
window.onload = function() {
audioContext = returnAudioContext()
sounds = {
drop : new Audio('sounds/drop.mp3'),
slide : new Audio('sounds/slide.mp3'),
win : new Audio('sounds/win.mp3'),
lose : new Audio('sounds/lose.mp3'),
}
playSound(sounds.drop)
}
function returnAudioContext(){
var AudioContext = window.AudioContext // Default
|| window.webkitAudioContext // Safari and old versions of Chrome
|| false;
if (AudioContext) {
return new AudioContext;
}
}
function playSound(sound){
audioContext.resume().then(() => {
console.log("playing sound")
sound.play();
});
}
Live example: http://www.mysterysystem.com/stuff/test.html
I've done my very best to make an example that uses solely the Web Audio API, but alas, Safari is not very compatible with this API. Though it is possible to use it in conjuction with the HTMLAudioElement, but unless you want to manipulate the audio; you won't need it.
The example below will play the drop sounds whenever you click anywhere in the document. It might need 2 clicks as audio in the browser can be very strict on when to play or not.
The playSound function checks if the play() method returns a promise. If it does then that promise should have a .catch() block. Otherwise it will throw the Unhandled Promise Rejection error in Safari.
const sounds = {
drop: new Audio('sounds/drop.mp3'),
slide: new Audio('sounds/slide.mp3'),
win: new Audio('sounds/win.mp3'),
lose: new Audio('sounds/lose.mp3'),
};
function playSound(audio) {
let promise = audio.play();
if (promise !== undefined) {
promise.catch(error => {
console.log(error);
});
}
}
document.addEventListener('click', () => {
playSound(sounds.drop);
});
If you do need to use the Web Audio API to do some stuff, please let me know.
I have a multi peer WebRTC stream using simple-peer and I'm playing the received stream like this:
peer.on("stream", data => {
let audio = document.createElement("audio") as HTMLAudioElement;
audio.src = URL.createObjectURL(data);
audio.play();
});
This works fine on desktop but for chrome on android there is no sound:
Unhandled Promise rejection: play() can only be initiated by a user gesture.
I couldn't find any documentation on how to correctly play the received stream. Do I really have to show a button when the stream is ready?
I have also tried to work around this issue by playing the stream from getUserMedia but this only worked as long as I didn't call audioTag.muted = true which is no solution either because this creates a feedback loop.
let audioTag = document.createElement("audio") as HTMLAudioElement;
audioTag.autoplay = true;
navigator.getUserMedia({video: false, audio: true}, (async stream => {
audioTag.src = window.URL.createObjectURL(stream);
audioTag.muted = true;
// ...
});
Sites like http://talky.io seem to have found a way around this problem though, so what do I have to do?
Check out: https://www.chromium.org/audio-video/autoplay
var promise = document.querySelector('video').play();
if (promise !== undefined) {
promise.then(_ => {
// Autoplay started!
}).catch(error => {
// Autoplay was prevented.
// Show a "Play" button so that user can start playback.
});
}