My goal is to generate an audio at a certain frequency and then check at what frequency it is using the result of FFT.
function speak() {
gb.src = gb.ctx.createOscillator();
gb.src.connect(gb.ctx.destination);
gb.src.start(gb.ctx.currentTime);
gb.src.frequency.value = 1000;
}
function listen() {
navigator.getUserMedia = (navigator.getUserMedia
|| navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia);
navigator.getUserMedia({
audio : true,
video : false
}, function(stream) {
gb.stream = stream;
var input = gb.ctx.createMediaStreamSource(stream);
gb.analyser = gb.ctx.createAnalyser();
gb.analyser.fftSize = gb.FFT_SIZE;
input.connect(gb.analyser);
gb.freqs = new Uint8Array(gb.analyser.frequencyBinCount);
setInterval(detect, gb.BIT_RATE / 2);
}, function(err) {
console.log('The following gUM error occured: ' + err);
});
}
See working example at http://codepen.io/Ovilia/full/hFtrA/ . You may need to put your microphone near the speaker to see the effect.
The problem is, when the frequency is somewhere larger than 15000 (e.g. 16000), there seems not to be any response at high frequency area any more.
Is there any limit of frequency with Web Audio, or is it the limit of my device?
What is the unit of each element when I get from getByteFrequencyData?
Is there any limit of frequency with Web Audio, or is it the limit of my device?
I don't think the WebAudio framework itself limits this. Like the other answers have mentioned here. The limit is probably from microphone's and loudspeaker's physical limits.
I tried to use the my current bookshelf loudspeaker (Kurzweil KS40A) I have along with a decent microphone (Zoom H4). The microphone was about 1 cm from the tweeter.
As you see, with these loudspeakers and microphones aren't able to effeciently generate/capture sounds at those frequencies.
This is more obvious when you look at the Zoom H4's frequency response. Unfortunately I couldn't find a frequency respose for the KS40a.
You can also do something similar using non browser tools to check if you see similar results.
What is the unit of each element when I get from getByteFrequencyData?
The unit of each element from getByteFrequencyData is a normalized magnitude data of from the FFT scaled to fit the dBFS range of the maxDecibles and minDecibles attributes on the AnalyserNode. So a byte value 0 would imply minDecibles (default is -100dBFS) or lower and a byte value of 255 would imply maxDecibles (default is -30dBFS) or higher.
Lookup the concept of Nyquist Frequency - the default sampling rate of web audio is 44.1kHz - this means the theoretical maximum frequency would be 22050 hertz given perfect hardware such as microphone and analog-to-digital converter inside your computer. #Ovilia on that same computer using same microphone record the same input sound and then examine the audio file using a utility like Audacity where you can view the output of its FFT analysis - in Audacity when you open an audio file go to menu Analyze -> Plot Spectrum ... also to see a very nice FFT view click the down arror near left side of waveform view subwindow and pick Spectrogram - another excellent FFT capable audio tool is called Sonic Visualizer - are you now seeing power at frequencies you are not seeing using FFT within web audio ?
I think that the most microphones just works well in the voice range frequency, something around 80 Hz to 1100 Hz
So probably do you have a hardware limit problem, try check with manufacturer or manual the frequency input response from your device !
There is probably an anti-alias low pass filter (between the microphone and the ADC) which has a cut-off below Fs/2 in order to make sure everything is rolled off by that frequency (given a finite filter transition width).
There may also be nulls in the room's acoustics. At frequencies above 2 Khz, it might be only inches from a peak to a null location for the microphone placement.
Related
I've built a system-audio setup with WebAudio in an Angular component. It works well only the bands do not seem to reflect frequency accurately.
See here a test for high, mid, low tone test.
I've gotten pretty far with the native API, accessing a media stream etc. but it's not as helpful as a utility as I thought it would be...
Question:
How would we get the most accurate frequency decibel data?
All sounds seem to be focused in the first 3 bands.
Here is the method which visualizes the media stream (full code on [Github][2])
private repeater() {
this._AFID = requestAnimationFrame(() => this.frameLooper());
// how many values from analyser (the "buffer" size)
this._fbc = this._analyser.frequencyBinCount;
// frequency data is integers on a scale from 0 to 255
this._data = new Uint8Array(this._analyser.frequencyBinCount);
this._analyser.getByteFrequencyData(this._data);
let bandsTemp = [];
// calculate the height of each band element using frequency data
for (var i = 0; i < this._fbc; i++) {
bandsTemp.push({ height: this._data[i] });
}
this.bands = bandsTemp;
}
Boris Smus' Web Audio API book says:
If, however, we want to perform a comprehensive analysis of the whole
audio buffer, we should look to other methods...
Perhaps this method is as good as it gets. What is a better method for more functional frequency analysis?
Thanks for the example in https://stackblitz.com/edit/angular-mediastream-device?file=src%2Fapp%2Fsys-sound%2Fsys-sound.component.ts. You're right that it doesn't work in chrome, but if I use the link to open it in a new window, everything is right.
So, I think you're computing the labels for the graph incorrectly. I assuming they're supposed to represent the frequency of the band. If not, then this answer is wrong.
You have fqRange = sampleRate / bands. Let's assume that sampleRate = 48000 (to keep the numbers simple), and bands = 16. Then fqRange = 3000. First I think you really want either sampleRate/2/bands or sampleRate / fftSize, which is the same thing.
So each of the frequency bins is 1500 Hz wide. Your labels should be 1500*k, for k = 0 to 15. (Although there's more than one way to label these, this is the easiest.) This will cover the range from 0 to 24000 Hz.
And when I play a 12 kHz tone, I see the peak is aroudn 1552 in your code. But with the new labeling, this is the 8'th bin, so 1500*8 = 12000. (Well, there are some differences. My sampleRate is actually 44.1 kHz, so the numbers computed above will be different.)
I am using dash.js library and achieving Adaptive bitrate with DASH protocol for my audio player.
I am facing issue in one of the case, when instead of changing bit rate adaptively, I want to be it specific i.e. 320 kbps. I am using methods which are provided in dash.js library as following.But not able to get a static bitrate segment for my whole audio file.
(function () {
var url = "https://xxxxxxxxxxxx.xxxxxxxx.net/myplaylist.mpd";
var player = dashjs.MediaPlayer().create();
player.initialize(document.querySelector("#audioPlayer"), url, true);
player.setInitialBitrateFor('audio', 320);
player.setQualityFor('audio', 320);
player.setAutoSwitchQualityFor('audio', false);
player.getDebug().setLogToBrowserConsole(false);
})();
so basically there are two options :: auto and 320kbps
auto will allow adaptive bitrate but when 320 kbps selected at any time after that it should only get segments for that bitrate only.
For the later scenario I am facing the issue.
is there any method to do that ? am I missing something here ?
it was not setting the bitrate b'coz it is doing exact matching for bitrate.
As of now, how bitrate is set with dash.js is as following.
when you do player.setInitialBitrateFor('audio', 320); first it will get the bandwidth from mpd files.
then there is a internal mechanism which will divide bandwidth by 1000 and then round of the value. so because off this if your mpd files contains values like bandwidth="320000",
then player.setInitialBitrateFor('audio', 320); will work.
There might be variation in bandwidth like 321684 which will generate bit rate value = 321. in such case you have to do player.setInitialBitrateFor('audio', 321); will work
also setQualityFor method takes index as second parameter.
so one can do
player.setQualityFor('audio', indexValue);
where considering there are three adaption set and
low bitrate (64 kbps) ==> 0 (indexValue)
Medium bitrate (128 kbps) ==> 1 (indexValue)
High bitrate (320 kbps) ==> 2 (indexValue)
I'm trying to set a velocity value, a value that is in a midi signal to gain. The velocity ranges from 0 to 127.
The documentation on the web audio api albeit well done, doesn't really say anything about this.
At the moment I've this to play sounds :
play(key, startTime) {
this.audioContext.decodeAudioData(this.soundContainer[key], (buffer) => {
let source = this.audioContext.createBufferSource();
source.buffer = buffer;
source.connect(this.audioContext.destination);
source.start(startTime);
});
}
I didn't find anything to use the velocity values that range from 0 to 127. However I found gain node that applies a gain.
So my function is now this:
play(key:string, startTime, velocity) {
this.audioContext.decodeAudioData(this.soundContainer[key], (buffer) => {
let source = this.audioContext.createBufferSource();
source.buffer = buffer;
source.connect(this.gainNode);
this.gainNode.connect(this.audioContext.destination);
this.gainNode.gain.value = velocity;
source.start(startTime);
});
}
Eehhh... if I apply the midi velocity value to the gain, I obviously have a sound that is insanely loud. So I'd like to know either of those two questions:
Can I somehow use the velocity value directly ?
How can I convert the velocity value to gain ?
The MIDI specification says:
Interpretation of the Velocity byte is left up to the receiving instrument. Generally, the larger the numeric value of the message, the stronger the velocity-controlled effect. If velocity is applied to volume (output level) for instance, then higher Velocity values will generate louder notes. A value of 64 (40H) would correspond to a mezzo-forte note […] Preferably, application of velocity to volume should be an exponential function.
The General MIDI specifications are not any more concrete.
The DLS Level 1 specification says:
The MIDI Note Velocity value is converted to attenuation in dB by the Concave Transform according to the following formula:
attendB = 20 × log10(1272 / Velocity2)
and fed to control either the volume or envelope generator peak level.
You then have to map this attenuation to the gain factor, i.e., gain = velocity² / 127².
And many hardware synthesizers allow to select different curves to map the velocity to volume.
I don't know if it is correct, because I don't know that much about sound but this seem to work:
this.gainNode.gain.value = velocity / 100 ;
So a velocity of 127 = a gain of 1.27
Ultimately I think what is better is dividing 1 in 127 values and each of those correspond to their respective midi value. However code is easier this way so yeah, it works.
I'm using Recorder.js to record audio from Google Chrome desktop and mobile browsers. In my specific use case I need to record exactly 3 seconds of audio, starting and ending at a specific time.
Now I know that when recording audio, your soundcard cannot work in realtime due to hardware delays, so there is always a memory buffer which allows you to keep up recording without hearing jumps/stutters.
Recorder.js allows you to configure the bufferLen variable exactly for this, while sampleRate is taken automatically from the audio context object. Here is a simplified version of how it works:
var context = new AudioContext();
var recorder;
navigator.getUserMedia({audio: true}, function(stream) {
recorder = new Recorder(context.createMediaStreamSource(stream), {
bufferLen: 4096
});
});
function recordLoop() {
recorder.record();
window.setTimeout(function () {
recorder.stop();
}, 3000);
}
The issue i'm facing is that record() does not offset for the buffer latency and neither does stop(). So instead of getting a three second sound, it's 2.97 seconds and the start is cut off.
This means my recordings don't start in the same place, and also when I loop them, the loops are different lengths depending on your device latency!!
There are two potentially solutions I see here:
Adjust Recorder.js code to offset the buffer automatically against your start/stop times (maybe add new startSync/stopSync functions)
Calculate the latency and create two offset timers to start and stop Recorder.js at the correct points in time.
I'm trying solution 2, because solution 1 requires knowledge of buffer arrays which I don't have :( I believe the calculation for latency is:
var bufferSize = 4096;
var sampleRate = 44100
var latency = (bufferSize / sampleRate) * 2; // 0.18575963718820862 secs
However when I run these calculations in a real test I get:
var duration = 2.972154195011338 secs
var latency = 0.18575963718820862 secs
var total = duration + latency // 3.1579138321995464 secs
Something isn't right, it doesn't make 3 seconds and it's beginning to confuse me now! I've created a working fork of Recorder.js demo with a log:
http://kmturley.github.io/Recorderjs/
Any help would be greatly appreciated. Thanks!
I'm a bit confused by your concern for the latency. Yes, it's true that the minimum possible latency is going to be the related to the length of the buffer but there are many other latencies involved. In any case, the latency has nothing to do with the recording duration, which seems to me to be what your question is about.
If you want to record an exactly 3 second long buffer at 44100 that is 44100*3=132,300 samples. The buffer size is 4096 samples and the system is only going to record an even multiple of that number. Given that the closest you are going to get is to record either 32 or 33 complete buffers. This gives either 131072 (2.97 seconds) or 135168 (3.065 seconds) samples.
You have a couple options here.
Choose a buffer length that evenly divides the sample rate. e.g. 11025. You can then record exactly 12 buffers.
Record slightly longer than the 3.0 seconds you need and then throw the extra 2868 samples away.
I am using the Web Audio API to display a visualization of the audio being played. I have an <audio> element that is controlling the playback, I then hook it up to the Web Audio API with by creating a MediaElementSource node from the <audio> element. That is then connected to a GainNode and an AnalyserNode. The AnalyserNode's smoothingTimeConstant is set to 0.6. The GainNode is then connected to the AudioContext.destination.
I then call my audio processing function: onAudioProcess(). That function will continually call itself using:
audioAnimation = requestAnimationFrame(onAudioProcess);
The function uses the AnalyserNode to getByteFrequencyData from the audio, then loops through the (now populated) Uint8Array and draws each frequency magnitude on the <canvas> element's 2d context. This all works fine.
My issue is that when you pause the <audio> element, my onAudioProcess function continues to loop (by requesting animation frames on itself) which is needlessly eating up CPU cycles. I can cancelAnimationFrame(audioAnimation) but that leaves the last-drawn frequencies on the canvas. I can resolve that by also calling clearRect on the canvas's 2d context, but it looks very odd compared to just letting the audio processing loop continue (which slowly lowers each bar to the bottom of the canvas because of the smoothingTimeConstant).
So what I ended up doing was setting a timeout when the <audio> is paused, prior to canceling the animation frame. Doing this I was able to save CPU cycles when no audio was playing AND I was still able to maintain the smooth lowering of the frequency bars drawn on the <canvas>.
MY QUESTION: How do I accurately calculate the number of milliseconds it takes for a frequency magnitude of 255 to hit 0 (the range is 0-255) based on the AnalyserNode's smoothingTimeConstant value so that I can properly set the timeout to cancel the animation frame?
Based on my reading of the spec, I'd think you'd figure it out like this:
var val = 255
, smooth = 0.6
, sampl = 48000
, i = 0
, ms;
for ( ; val > 0.001; i++ ){
val = ( val + val * smooth ) / 2;
}
ms = ( i / sampl * 1000 );
The problem is that with this kind of averaging, you never really get all the way down to zero - so the loop condition is kind of arbitrary. You can make that number smaller and as you'd expect, the value for ms gets larger.
Anyway, I could be completely off-base here. But a quick look through the actual Chromium source code seems to sort of confirm that this is how it works. Although I'll be the first to admit my C++ is pretty bad.