I have read and implemented in the past something similar to what Chris Wilson described in his article "A Tale of Two Clocks": https://web.dev/audio-scheduling/
I recently found WAAClock which in theory implements the same principle:
https://github.com/sebpiq/WAAClock
I'm working on a MIDI Web App and I want to send MIDI Clock Messages, which requires precise scheduling. I wrote this post in WebMidiJS forum (an amazing lib I'm using in my Project):
https://github.com/djipco/webmidi/discussions/333
Essentially this is my code:
const PULSES_PER_QUARTER_NOTE = 24;
const BPM_ONE_SECOND = 60;
let context = new AudioContext();
let clock = new WAAClock(context);
const calculateClockDelay = bpm => BPM_ONE_SECOND / bpm / PULSES_PER_QUARTER_NOTE;
const startMidiClock = bpm => {
clock.start();
clock.callbackAtTime(function () {
WebMidi.outputs.forEach(outputPort => outputPort.sendClock({}));
}, 0).repeat(calculateClockDelay(bpm));`
}
const stopMidiClock = () => clock.stop();
As described in that posts, I CANNOT get the event to happen with high precision. I see the BPM Meter to slightly DRIFT. I tried sending MIDI Clock from a DAW and the timing is perfect.
I'm using ZERO as tolerance in the clock.callbackAtTime function.
Why Do I see this drift/ slight scheduling error?
Is there any other way to schedule a precise repeating MIDI event with WAAClock?
Is WAAClock capable of precise scheduling as with Chris Wilson's technique?
Thanks a lot!
Danny Bullo
Eh. I'm not deeply familiar with that library - but at a quick glance, I am deeply suspicious that it can do what it claims to. From its code, this snippet makes me VERY suspicious:
this._clockNode.onaudioprocess = function () {
setTimeout(function() { self._tick() }, 0)
}
If I understand it, this is trying to use the scriptprocessornode to get a high-stability, low-latency clock. That's not, unfortunately, something that scriptprocessornode can do. You COULD do something closer to this with audioworklets, except you wouldn't be able to call back out to the main thread to fire the MIDI calls.
I'm not sure you've fully grasped how to apply the TOTC approach to MIDI - but the key is that Web MIDI has a scheduler built in, too: instead of trying to call your Javascript code and outputPort.sendClock() at PRECISELY the right time, you need to schedule ahead a call with (essentially) outputPort.sendClock( time ) - that is, some small amount of time before the clock message needs to be sent, you need to call the MIDIOutput.send() with the timestamp parameter set - a schedule time for precisely when it needs to be sent.
This is gone over in more detail (for audio, though, not MIDI) in https://web.dev/audio-scheduling/#obtaining-rock-solid-timing-by-looking-ahead.
Related
i would like to achieve in the same way discord app is doing, calibrate the audio input to be triggered at a certain volume (Decibel, ...) and not bellow that volume.
I'm creating a video call app and i want before the call allow the user to setup his devices correctly. Actually the sensitivity is too high and there is a lot of undesirable noise (You can even hear a bug fly).
I did a lot of unsuccessful research about how to achieve this with the Web Audio API.
At the beginning i tried to use the GainNode but it's just a way to amplify/attenuate (+/-) the sound like a volume.
After i tried using the BiquadFilterNode but it's just some filters to attenuate/amplify the audio around a certain frequency.
I think there is maybe a way using AudioWorklet but i don't find any clear documentation about how to solve my problem.
I would like in the same way i'm modifying the gain create a function setVolumeThreshold that regarding a certain db (or other kind of value) cut the sound bellow that.
var start = () => navigator.mediaDevices.getUserMedia({audio: true})
.then(stream => audio.srcObject = modifyGain(stream, 2.5))
.catch(e => log(e));
var modifyGain = (stream, gainValue) => {
var ctx = new AudioContext();
var src = ctx.createMediaStreamSource(stream);
var dst = ctx.createMediaStreamDestination();
var gainNode = ctx.createGain();
gainNode.gain.value = gainValue;
[src, gainNode, dst].reduce((a, b) => a && a.connect(b));
return dst.stream;
};
Here a jsFiddle using that code snippet.
Thanks a lot to those will try to help me !
I did a similar project quite a while ago using ScriptProcessorNode - https://github.com/cwilso/volume-meter/. It should be easily portable to AudioWorklet.
Also of interest is the "noise gate" effect in https://github.com/cwilso/Audio-Input-Effects, which I think is closer to what you want?
I'm working on a Javascript Music App that includes a Sequencer. For those who are not familiar, MIDI sequencers work pretty much like this: There is something called PPQ: pulses per quarter note. Each pulse is called "Tick". It depicts how may "subdivisions" there are per quarter note, like resolution. So Sequencers "play" the Events that are in the tracks one Tick at a time: Play Tick1, wait Tick Duration, Play tick2, Tick Duration, and so on.
Now, let's say we have a BPM (Beats per Min) of 120 with PPQ=96 (standard). That means that each Quarter Note Duration is 500ms, and each Tick Duration is 5.20833ms.
What Timer Alternatives we have in Javascript?
1) We have the old setTimeOut. It has several problems: the min. wait time is 4ms. (https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setTimeout#Minimum_delay_and_timeout_nesting)
It is also subject to JITTER/time Variations. It is not precise and it is demanding, as call backs are stacked in the even loop.
2) There is an alternative to setTimeOut/setInterval which involves using requestAnimationFrame(). It is VERY precise and CPU efficient. However, the minimum time it can be set is around 16.7ms (the duration of a Frame in a typical 60FPS monitor)
Is there any other Alternative? To to precisely schedule an event every 2-5ms?
Note: the function done in side the loop, playEventsAtTick() is NOT demanding at all, so it would never take more time to execute than Tick Duration.
Thanks!
Danny Bullo
To maintain any sanity in doing this kind of thing, you're going to want to do the audio processing on a devoted thread. Better yet, use the Web Audio API and let people who have been thinking about these problems for a long time do the hard work of sample-accuracy.
Also check out Web MIDI (chrome only).
Thanks nvioli. I'm aware of Web Audio API. However, I don't think that can help here.
I'm not triggering AUDIO directly: I have MIDI events (or let's say just "EVENTS") stored in the TRACKS. And those events happen at any TICK. So the Sequencer needs to loop every Tick Duration to scan what to play at that particular tick.
Regards,
Danny Bullo
In a separate thread, such as a web worker, you can create an endless loop. In this loop, all you need to do is calculate the time between beats. After the time is valid, you can then send a message to the main process, to do some visuals, play a sound or what ever you would like to do.
Here is a Working example
class MyWorker {
constructor() {
// Keeps the loop running
this.run = true
// Beats per minute
this.bpm = 120
// Time last beat was called
this.lastLoopTime = this.milliseconds
}
get milliseconds() {
return new Date().getTime()
}
start() {
while (this.run) {
// Get the current time
let now = this.milliseconds
// Get the elapsed time between now and the last beat
let updateLength = now - this.lastLoopTime
// If not enough time has passed restart from the beginning of the loop
if (updateLength < (1000 * 60) / this.bpm) continue;
// Enough time has passed update the last time
this.lastLoopTime = now
// Do any processing that you would like here
// Send a message back to the main thread
postMessage({ msg: 'beat', time: now })
}
}
}
new MyWorker().start()
Next we can create the index page, which will run the worker, and flash a square everytime a message comes back from the worker.
<!DOCTYPE html>
<html lang="en">
<head>
<script>
// Start the worker
var myWorker = new Worker('worker.js')
// Listen for messages from the worker
myWorker.onmessage = function (e) {
var msg = e.data
switch (msg.msg) {
// If the message is a `beat` message, flash the square
case 'beat':
let div = document.querySelector('div')
div.classList.add('red')
setTimeout(() => div.classList.remove('red'), 100)
break;
}
}
</script>
<style>
div { width: 100px; height: 100px; border: solid 1px; }
.red { background: red; }
</style>
</head>
<body>
<div></div>
</body>
</html>
Get Off My Lawn: The approach you suggested does not completely work. Let's say I add a method to the web worker to STOP the Sequencer:
stop() {
this.run = false;
}
The problem is that the method myWorker.onmessage = function (e) {...} never get's triggered. I suspect it is because the Web Worker Thread is "TOO BUSY" with the endless loop. any way to solve that?
Also, while playing, it works.....but the CPU goes up considerably..... The only possible Solution would be a Sleep() method, but Real SLEEP that does not exist in Javascript...
Thanks
I want a small short beep noise to get the user's attention occasionally in my Vaadin Framework 8 app.
Example usage: Beep when password entered into a user-authentication page fails to authenticate successfully. User clicks "Sign-in" button, beep happens immediately after the password check flunks.
Another example: When an automatically updated chart trends upward significantly, play 3 beeps in ascending frequency (notes). If chart trends downward, play descending frequency (notes).
I would like to avoid downloading a sound file to the web client, unless that has clear advantages. Seems simpler, lightweight, and hopefully higher performance to use JavaScript or HTML5 to generate a beep on the client itself locally.
I found what looks like a modern JavaScript solution in this Answer by Houshalter. And this sibling Answer by CaptainWiz contains a live demo that seems to work well.
Is there a way to trigger the client-side execution of this JavaScript code from my server-side Vaadin app’s Java code?
Will it be performant? I mean the beep needs to happen very quickly in the context of the user's current action, without an annoying/confusing time delay.
Alternatively, this Answer talks about HTML5 having a new Audio objects feature for playing a sound file. Might that have advantages over invoking a chunk of JavaScript code for sound synthesis?
And another alternative: WebAudio API by W3C as shown in this Answer.
One way to achieve this via an AbstractJavascriptComponent in Vaadin. This
gives a rather direct approach to write Javascript components or make JS libs
accessible without spending too much time on getting a grasp on GWT etc.
The callFunction from an AbstractJavascriptComponent calls directly into
the JS-code in the browser.
Create a Beeper class:
package app.ui
import com.vaadin.annotations.JavaScript
import com.vaadin.ui.AbstractJavaScriptComponent
#JavaScript("js/beeper_connector.js")
class Beeper extends AbstractJavaScriptComponent {
void beep(Integer duration, Integer frequency) {
callFunction('beep', duration, frequency)
}
}
Note the annotation and also create that file in the same package (app.ui),
on that path with that name (js/beeper_connector.js). The file needs at
least to contain a "class" with the name app_ui_Beeper (FQN of the Java class
with the dots replaced by underscores) Add your beep function with params of
types, that can be transported via "JSON":
window.app_ui_Beeper = function() {
var audioCtx = new (window.AudioContext || window.webkitAudioContext || window.audioContext);
this.beep = function(duration, frequency) {
var oscillator = audioCtx.createOscillator();
var gainNode = audioCtx.createGain();
oscillator.connect(gainNode);
gainNode.connect(audioCtx.destination);
if (frequency){oscillator.frequency.value = frequency;}
oscillator.start();
setTimeout(function(){oscillator.stop()}, (duration ? duration : 500));
};
};
This code is lifted from the answer referenced by OP: How do I make Javascript beep?
Now make sure, that you add a Beeper instance somewhere in your main scene
graph in the UI, so it can be accessed from everywhere.
A working example can be found here: https://github.com/christoph-frick/vaadin-webaudio-beep
I'm using MediaCapture in javascript to capture my camera.
I have a Camera class with an initCamera function. The problem is, if I try to re-init my camera in a short time period I will get this error: Hardware MFT failed to start streaming due to lack of hardware resources.
Now I get that this means my camera is still in use. The thing I want to know is:
How do I properly close my camera
How do I check if my camera is in use or unavailable
Here is a piece of code:
function Camera() {
var that = this;
this.mediaCaptureElement = null;
this.initCamera = function() {
if (!that.mediaCaptureElement) {
that.mediaCaptureElement = new Windows.Media.Capture.MediaCapture();
that.mediaCaptureElement.addEventListener("failed", function (e) {
console.warn("The camera has stopped working");
}
that.mediaCaptureElement.initializeAsync().then(function() {
that.mediaCaptureElement.videoDeviceController.primaryUse = Windows.Media.Devices.CaptureUse.photo;
that.getCameraResolution();
that.orientationChanged();
that.startCamera();
});
}
};
The way I re-open my camera currently is by overwriting the camera instance with a new instance of the Camera class.
Thanks in advance.
I had the same problem using MediaCapture in C#.
I had to call Dispose() after StopPreviewAsync in order to correct it :
await cameraControler.MediaCaptureInstance.StopPreviewAsync(); cameraControler.MediaCaptureInstance.Dispose();
Have you seen the Camera Starter Kit UWP sample? It comes in a JS flavor too!
If you want to be able to reliably access the camera shortly after being done using it, you need to make sure you're cleaning up all resources properly. From the code that you've shared, it seems like you're letting the system take care of this, which means your app might be coming back before the system is done closing out all resources.
You should take care of:
Stop any recordings that may be in progress
Stop the preview
Close the MediaCapture
Have a look at the cleanupCameraAsync() method from the sample I linked above for an example on how to implement this.
We have a long piece of video, up to 1 hour long.
We want to show users small 30 second chunks of this video.
It's imperative that the video does not stutter at any point.
The user can't then jump around the rest of the video, they only see the 30 second chunk.
An example would be say, a football match, the whole match is on video but clicking a button in another page would load up the full video and play just a goal.
Is this possible with HTML5 Video?
Would it have anything to do with TimeRanges?
Does the video have to served over a pure streaming protocol?
Can we buffer the full 30 second chunk before playing it?
The goal is to cut down on the workflow required to cut out all the little clips (and the time transcoding these to all the different HTML 5 video formats), we can just throw up a trans-coded piece of footage and send the user to a section of that footage.
Your thoughts and input are most welcome, thanks!
At this point in time HTML5 videos are a real PITA -- we have no real API to control the browser buffering, hence they tend to stutter on slower connections, as the browsers try to buffer intelligently, but usually do quite the opposite.
Additionally, if you only want your users to view a particular 30 second chunk of a video (I assume that would be your way of forcing users to registers to view the full videos), HTML5 is not the right choice -- it would be incredibly simple to abuse your system.
What you really need in this case is a decent Flash Player and a Media Server in the backend -- this is when you have full control.
You could do some of this, but then you'd be subject the the browser's own buffering. (You also can't stop it from buffering beyond X sec)
Best put, you could easily have a custom seek control to restrict the ranges and stop the video when is hits the 30 second chunk.
Also, buffering is not something you can control other the tell the browser not to do it. The rest is automatic and support to force a full buffer has been removed from specs.
Anyways, just letting you know this is terrible practice and it could be done but you'll be potentially running into many issues. You could always use a service like Zencoder to help handle transcoding too. Another alternative would be to have ffmpeg or other software on the server to handle clipping and transcoding.
You can set the time using javascript (the video's currentTime property).
In case you want a custom seekbar you can do something like this:
<input type="range" step="any" id="seekbar">
var seekbar = document.getElementById('seekbar');
function setupSeekbar() {
seekbar.max = video.duration;
}
video.ondurationchange = setupSeekbar;
function seekVideo() {
video.currentTime = seekbar.value;
}
function updateUI() {
seekbar.value = video.currentTime;
}
seekbar.onchange = seekVideo;
video.ontimeupdate = updateUI;
function setupSeekbar() {
seekbar.min = video.startTime;
seekbar.max = video.startTime + video.duration;
}
If the video is streaming you will need to "calculate" the "end" time.
var lastBuffered = video.buffered.end(video.buffered.length-1);
function updateUI() {
var lastBuffered = video.buffered.end(video.buffered.length-1);
seekbar.min = video.startTime;
seekbar.max = lastBuffered;
seekbar.value = video.currentTime;
}