I want to develop a web App for mobile phones that records audio from the microphone and plays music at the same time.
With getUserMedia() I get the stream and create in my AudioContext a MediaStreamSource. At the same time I create a BufferSource which plays music. In Chrome this setup works. But when I start the same Web app in Chrome on my Nexus 5 and I allow it to use the microphone the music is muted.
Success Callback for getUserMedia:
function gotStream(stream) {
mediaStreamSource = audioContext.createMediaStreamSource(stream);
meter = createAudioMeter(audioContext);
mediaStreamSource.connect(meter);
info = document.getElementById('info');
outputData();
}
Play Music Function:
function playSound(buffer) {
source = audioContext.createBufferSource();
source.buffer = buffer;
gainNode = audioContext.createGain();
source.connect(gainNode);
gainNode.connect(audioContext.destination);
source.start(0);
}
Should that be the expected behaviour or am I doing something wrong?
Related
I am using standard JS speech recognition in my NextJS app, and it works well in Chrome web browser and on Android. But it doesn't work when I try to run it in Chrome on iOS, but in Safari it works well. What can be the issue? I checked Chrome settings and access to mic is allowed there.
This is the speech recognition method that I use:
// new speech recognition object
var SpeechRecognition = SpeechRecognition || webkitSpeechRecognition;
var recognition = new SpeechRecognition();
// This runs when the speech recognition service starts
recognition.onstart = function() {
console.log("We are listening. Try speaking into the microphone.");
};
recognition.onspeechend = function() {
// when user is done speaking
recognition.stop();
}
// This runs when the speech recognition service returns result
recognition.onresult = function(event) {
var transcript = event.results[0][0].transcript;
};
// start recognition
recognition.start();
I have developed an electron app and for the first time ever, my antivirus (ESET) has raised "Webcam access attempt" when the app loads. Has anyone else experienced this?
My app does not use the webcam and I have no code that requires the webcam.
I do have code that accesses audio for audio recording. I have denied access to the webcam in antivirus and the app does function as designed. However, antivirus warning messages appear on every load of the app. As you can imagine, this is not cool.
This has surfaced immediately after updating ESET (v14.2.10.0), so they have some new rule that gets triggered. I have to assume that this is not an ESET over sensitivity to something (I have no idea how AV's function and ‘blaming’ the antivirus doesn’t seem like a sound response to provide users), so I am left questioning my deployment of web-apis in my code.
My audio access uses the native web-apis: AudioContext, Navigator, MediaDevices, MediaRecorder. The key lines of code are below:
// getting list of all AUDIO devices:
// const audioSources = await navigator.mediaDevices.enumerateDevices({ audio: true });
// ^ above does NOT filter by audio only
const audioSources = await navigator.mediaDevices.enumerateDevices();
// creating a recorder object:
const audioContext = new AudioContext();
const dest = audioContext.createMediaStreamDestination();
const audioParams = {
deviceId: "6e5fc2d7ffa5c6c04e06d282a5aa743e983e585a7e12118c80c0cd8646cce4b7", // this ID is from audioSources object
}
const mediaStream = await navigator.mediaDevices.getUserMedia({ audio: audioParams });
const audioIn = audioContext.createMediaStreamSource(mediaStream);
audioIn.connect(dest);
const audioOptions = {
bitsPerSecond: 128000,
mimeType: 'audio/webm; codecs=opus',
};
const recorder = new MediaRecorder(dest.stream, audioOptions);
Because navigator.mediaDevices.enumerateDevices() does not take parameters, such as { audio: true }, enumerateDevices() triggers the camera request.
I use the results of enumerateDevices() to access the device ID, which is then passed into .getUserMedia() to select the specific device. This allows users to select one or more audio inputs for the same recording.
Is there a way of just querying available media for audio devices / excluding video?
Is there another way of identifiying all available audio devices?
How else can I select what device .getUserMedia() returns as a stream?
The only existing information I could find on this was on the shut-down Atom Community forum:
Electron keeps accessing my webcam for NO REASON - two developers discovering the same behaviour in Sept'20 with different Antivirus software. No resolution.
Originally seen using Electron 8.5.0. Issue remains after updating to 13.1.2
Software versions: Electron 13.1.2, ESET 14.2.10.0
I'm developing an application that allows voice recording in the web browser. This is working fine on most browsers but I have some issues with iOS Safari.
Below you can find an extract of the code, it is not complete but it gives an idea of what's going on.
//Triggered when the user clicks on a button that start the recording
function startRecording() {
//Create new audio context
let audioContext = new (window.AudioContext || window.webkitAudioContext);
//Hack polyfill media recorder to re-use the audioContex
window.NewMediaRecorder.changeAudioContext(audioContext);
navigator.mediaDevices.enumerateDevices().then(function (devices) {
console.log('loaded audio devices');
console.log(devices);
devices = devices.filter((d) => d.kind === 'audioinput');
console.log(devices);
console.log('chosen device: ' + devices[0].deviceId);
navigator.mediaDevices.getUserMedia({
audio: {
deviceId : {
exact : devices[0].deviceId
}
}
}).then(function (stream) {
console.log(stream);
let recorder = new NewMediaRecorder(stream);
recorder.addEventListener('dataavailable', function (e) {
document.getElementById('ctrlAudio').src = URL.createObjectURL(e.data);
});
recorder.start();
console.log('stop listening after 15 seconds');
setTimeout(function () {
console.log('15 seconds passed');
console.log("Force stop listening");
recorder.stop();
recorder.stream.getTracks()[0].stop();
}, 15000);
});
});
}
For the record, I'm using audio recorder polyfill (https://ai.github.io/audio-recorder-polyfill/) in order to achieve recording, as MediaRecorder is not yet available on Safari.
The recorder works fine on all navigators (this including OS X Safari), yet on iOS Safari it only records noice. If I set the volume of my speakers at maximal level I can hear myself speak, but it is from "very far away".
All the online dictaphones/recorders that I found have the same issue, they always recording noise. (Tested with an iPhone 5S, 5SE and X, all up to date).
I'm a bit desperate because I already did a lot of research, but I didn't find any solution for this issue.
As required, the AudioContext is created on a user event (in this case a touch on a button).
I even tried to change the gain of but that didn't help.
Trying to access the audio without setting a media device isn't helping.
navigator.mediaDevices.getUserMedia({audio: true})
I want to use the SpeechRecognition api with an audio file (mp3, wave, etc.)
Is that possible?
The short answer is No.
The Web Speech Api Specification does not prohibit this (the browser could allow the end-user to choose a file to use as input), but the audio input stream is never provided to the calling javascript code (in the current draft version), so you don't have any way to read or change the audio that is input to the speech recognition service.
This specification was designed so that the javascript code will only have access to the result text coming from the speech recognition service.
Basicly you may use it only with default audioinput device which is choosen on OS level...
Therefore you just need to play you file into your default audioinput
2 options possible:
1
Install https://www.vb-audio.com/Cable/
Update system settings to use VCable device as default audiooutput and audioinput
Play your file with any audio player you have
Recognize it... e.g. using even standard demo UI https://www.google.com/intl/fr/chrome/demos/speech.html
Tested this today, and it works perfectly :-)
2
THIS IS NOT TESTED BY ME, so I cannot confirm that this is working, but you may feed audio file into chrome using Selenium... just like
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
ChromeOptions options = new ChromeOptions();
options.addArguments("--allow-file-access-from-files",
"--use-fake-ui-for-media-stream",
"--allow-file-access",
"--use-file-for-fake-audio-capture=D:\\PATH\\TO\\WAV\\xxx.wav",
"--use-fake-device-for-media-stream");
capabilities.setCapability(ChromeOptions.CAPABILITY, options);
ChromeDriver driver = new ChromeDriver(capabilities);
But I'm not sure if this stream will replace default audioinput
Andri deleted this post but I will repost it as I believe it to be the most accurate answer, besides the hackish answers above:
According to MDN you CAN'T do that. You can't feed any stream into recognition service
That's a big problem... You even cannot select microphone used by SpeechRecognition
That is done by purpose, Google want's to sell their CLOUD SPEECH API
You need to use services like CLOUD SPEECH API
You could probably just start the SpeechRecognition engine using the mic and playback the audio file via speakers to have feed back into the mic. It worked for me when I tested it.
Yes, it is possible to get the text transcript of the playback of an audio file using webkitSpeechRecognition. The quality of the transcript depends upon the quality of the audio playback.
const recognition = new webkitSpeechRecognition();
const audio = new Audio();
recognition.continuous = true;
recognition.interimResults = true;
recognition.onresult = function(event) {
if (event.results[0].isFinal) {
// do stuff with `event.results[0][0].transcript`
console.log(event.results[0][0].transcript);
recognition.stop();
}
}
recognition.onaudiostart = e => {
console.log("audio capture started");
}
recognition.onaudioend = e => {
console.log("audio capture ended");
}
audio.oncanplay = () => {
recognition.start();
audio.play();
}
audio.src = "/path/to/audio";
jsfiddle https://jsfiddle.net/guest271314/guvn1yq6/
My tablet is running Chrome 52.0.2743.98 but will not output sound when I go to this Web Audio example page.
When I inspect the audio context in the console, I can see that the currentTime is always 0.
Pasting the following code from MDN also produces no sound:
var audioCtx = new (window.AudioContext || window.webkitAudioContext)();
var oscillator = audioCtx.createOscillator();
oscillator.type = 'square';
oscillator.frequency.value = 3000; // value in hertz
oscillator.connect(audioCtx.destination);
oscillator.start();
These two examples work well on my laptop with Chrome 52.0.2743.116.
How can I get Chrome to output sound from the Web Audio API?
For Chrome Android, my recollection is that audio will only start if attached to a user interaction (e.g. a touch or click event). See also https://bugs.chromium.org/p/chromium/issues/detail?id=178297