I support several churches that don't have musicians, by providing a little website with a bunch of pure Javascript so they can select music for their services from a collection of about 1100 mp3 and m4a music files. Previously, they created playlists in iTunes or Media Player, but after a track completed, the player would immediately start the next track unless they quickly clicked 'Stop'. So my website allows them to select all their music ahead of time (up to 10 tracks), with a separate "Play" button for each. Hit "Play" and it plays that one track and stops. (Duh.)
I'm encountering delays in loading the files into my "audio" tags - and I need the file to load when they select it so I can display the track duration, which is frequently important to the selection of the music for the service. A delay doesn't occur very often, but often enough to be annoying. Also, the load will occasionally time out completely, even after several attempts. I've experimented played with various techniques, like using setTimeout with different values to allow several seconds before checking if it's loaded, or, loading 5 or 10 times with shorter timeout values until it's loaded. I created a test page that indicates that the timeouts vary greatly - from 2% to 5% of the time, to upwards of 25% occasionally (during tests of 1,000 to 10,000 random loads).
My first technique was relying on events (I tried both 'canplay' and 'canplaythrough' events with minimal difference):
const testAudio = document.getElementById('test-audio');
let timeStart = Date.now();
function loadMusic(p_file) {
testAudio.src = p_file;
testAudio.addEventListener('canplaythrough', musicLoaded);
timeStart = Date.now();
testAudio.load();
}
function musicLoaded() {
console.log('music loaded in ' + (Date.now()-timeStart) + 'ms');
testAudio.removeEventListener('canplaythrough', musicLoaded);
/* should I add/remove the listener each time I change the source file ? */
}
My second approach (from a post here: https://stackoverflow.com/questions/10235919/the-canplay-canplaythrough-events-for-an-html5-video-are-not-called-on-firefox) is to check the 'readyState' of the audio element after a specified timeout, rather than relying on an event. This question specifically addressed Firefox, so I should mention that in my tests Firefox has horrible load times for both the "events" and the "readyState" techniques. Chrome and Edge vary in the range of 2% to 6% load failure due to timeout and Firefox has 27% to 39% load timeouts.
let myTimeout = '';
function loadMusic(p_file) {
myTimeout = setTimeout(fileTimeout, 1000); /* I've tried various values here */
testAudio.src = p_file;
timeStart = Date.now();
testAudio.load();
}
function fileTimeout() {
if (testAudio.readyState > 3) {
console.log('music loaded in ' + (Date.now()-timeStart) + 'ms');
} else {
/* here, I've tried calling loadMusic again 5 to 10 times, which sometimes works */
/* or, just reporting that the load failed... */
console.log('music FAILED to load!');
}
}
I have a shared server hosting plan, and I suspect the delay might be due to traffic on my server. Unfortunately, my hosting service turns a deaf ear to anything that might be application or content related (not surprising). And this isn't worth upgrading to a dedicated server just to eliminate that variable. But I suspect that might be a major factor here.
I need a technique that will always work - even if it takes 30 seconds or more. As long as I can display an intermittent "Still loading..." type message I (and my users) would be satisfied. The "track X won't load" messages happen often enough to be annoying. Early on, I had a few files with bad characters in the file name that needed to be fixed before they would load. So the users think that problem persists. But I know I've fixed all them now.
Any and all suggestions are welcome - but I'd love to keep everything in plain Javascript.
Using an audio constructor:
function loadMusic(p_file) {
myTimeout = setTimeout(fileTimeout, 1000);
let audioConst = new Audio();
audioConst.src = p_file;
timeStart = Date.now();
}
function fileTimeout() {
if (audioConst.readyState > 3) {
console.log('music loaded in ' + (Date.now()-timeStart) + 'ms');
} else {
console.log('music FAILED to load!');
}
myTimeout = '';
}
Related
I'm working on a Javascript Music App that includes a Sequencer. For those who are not familiar, MIDI sequencers work pretty much like this: There is something called PPQ: pulses per quarter note. Each pulse is called "Tick". It depicts how may "subdivisions" there are per quarter note, like resolution. So Sequencers "play" the Events that are in the tracks one Tick at a time: Play Tick1, wait Tick Duration, Play tick2, Tick Duration, and so on.
Now, let's say we have a BPM (Beats per Min) of 120 with PPQ=96 (standard). That means that each Quarter Note Duration is 500ms, and each Tick Duration is 5.20833ms.
What Timer Alternatives we have in Javascript?
1) We have the old setTimeOut. It has several problems: the min. wait time is 4ms. (https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setTimeout#Minimum_delay_and_timeout_nesting)
It is also subject to JITTER/time Variations. It is not precise and it is demanding, as call backs are stacked in the even loop.
2) There is an alternative to setTimeOut/setInterval which involves using requestAnimationFrame(). It is VERY precise and CPU efficient. However, the minimum time it can be set is around 16.7ms (the duration of a Frame in a typical 60FPS monitor)
Is there any other Alternative? To to precisely schedule an event every 2-5ms?
Note: the function done in side the loop, playEventsAtTick() is NOT demanding at all, so it would never take more time to execute than Tick Duration.
Thanks!
Danny Bullo
To maintain any sanity in doing this kind of thing, you're going to want to do the audio processing on a devoted thread. Better yet, use the Web Audio API and let people who have been thinking about these problems for a long time do the hard work of sample-accuracy.
Also check out Web MIDI (chrome only).
Thanks nvioli. I'm aware of Web Audio API. However, I don't think that can help here.
I'm not triggering AUDIO directly: I have MIDI events (or let's say just "EVENTS") stored in the TRACKS. And those events happen at any TICK. So the Sequencer needs to loop every Tick Duration to scan what to play at that particular tick.
Regards,
Danny Bullo
In a separate thread, such as a web worker, you can create an endless loop. In this loop, all you need to do is calculate the time between beats. After the time is valid, you can then send a message to the main process, to do some visuals, play a sound or what ever you would like to do.
Here is a Working example
class MyWorker {
constructor() {
// Keeps the loop running
this.run = true
// Beats per minute
this.bpm = 120
// Time last beat was called
this.lastLoopTime = this.milliseconds
}
get milliseconds() {
return new Date().getTime()
}
start() {
while (this.run) {
// Get the current time
let now = this.milliseconds
// Get the elapsed time between now and the last beat
let updateLength = now - this.lastLoopTime
// If not enough time has passed restart from the beginning of the loop
if (updateLength < (1000 * 60) / this.bpm) continue;
// Enough time has passed update the last time
this.lastLoopTime = now
// Do any processing that you would like here
// Send a message back to the main thread
postMessage({ msg: 'beat', time: now })
}
}
}
new MyWorker().start()
Next we can create the index page, which will run the worker, and flash a square everytime a message comes back from the worker.
<!DOCTYPE html>
<html lang="en">
<head>
<script>
// Start the worker
var myWorker = new Worker('worker.js')
// Listen for messages from the worker
myWorker.onmessage = function (e) {
var msg = e.data
switch (msg.msg) {
// If the message is a `beat` message, flash the square
case 'beat':
let div = document.querySelector('div')
div.classList.add('red')
setTimeout(() => div.classList.remove('red'), 100)
break;
}
}
</script>
<style>
div { width: 100px; height: 100px; border: solid 1px; }
.red { background: red; }
</style>
</head>
<body>
<div></div>
</body>
</html>
Get Off My Lawn: The approach you suggested does not completely work. Let's say I add a method to the web worker to STOP the Sequencer:
stop() {
this.run = false;
}
The problem is that the method myWorker.onmessage = function (e) {...} never get's triggered. I suspect it is because the Web Worker Thread is "TOO BUSY" with the endless loop. any way to solve that?
Also, while playing, it works.....but the CPU goes up considerably..... The only possible Solution would be a Sleep() method, but Real SLEEP that does not exist in Javascript...
Thanks
I have an FFT using canvas that plots a high speed display. I want to optimize the code to have 16 browser windows showing at the same time at 60 fps or close to it. Right now on my machine it runs at 5 fps with 16 windows showing simultaneously.
I was wondering if there was a better way to optimize my code for drawing performance.
With this I am getting 60 fps for up to four simultaneous browser windows but fps drops significantly after that. Right now I am loading all the files into an array buffer and manipulating the points and drawing them at the same time in drawFFT(). Any tips on improving fps performance on multiple browser windows running at the same time?
60 fps animation on multiple windows
99.99% of the time, requestAnimationFrame is the way to do visual animations. It's a great tool, synced with the screen refresh rate, with great timing precision, and battery friendly.
This last advantage is your problem : To save power, browsers do allow the screen synchronization only for the currently focused window at 60fps. All other windows are delayed, on a lower frame-rate.
Since you want to have multiple windows, you'll be able to get only one focused, and thus only one refreshing at 60fps, all others will be slowed down to about 5fps.
How to circumvent this ?
The WebAudioAPI does have its own low-level and high-precision clock system.
By "low-level", I mean that this clock system is not tied to the main js-thread*. On some implementations (chrome) all the WebAudioAPI even runs on a parallel thread. And more importantly to our case, this clock system is not only tied to focused window. This does mean that we can run code, in a background window, at 60fps.
Here is a simple implementation of an timing loop based on the WebAudioAPI clock.
(*Note that while the clock is not tied to the main js thread, the event handler is).
/*
An alternative timing loop, based on AudioContext's clock
#arg callback : a callback function
with the audioContext's currentTime passed as unique argument
#arg frequency : float in ms;
#returns : a stop function
*/
function audioTimerLoop(callback, frequency) {
var freq = frequency / 1000; // AudioContext time parameters are in seconds
var aCtx = new AudioContext();
// Chrome needs our oscillator node to be attached to the destination
// So we create a silent Gain Node
var silence = aCtx.createGain();
silence.gain.value = 0;
silence.connect(aCtx.destination);
onOSCend();
var stopped = false; // A flag to know when we'll stop the loop
function onOSCend() {
var osc = aCtx.createOscillator();
osc.onended = onOSCend; // so we can loop
osc.connect(silence);
osc.start(0); // start it now
osc.stop(aCtx.currentTime + freq); // stop it next frame
callback(aCtx.currentTime); // one frame is done
if (stopped) { // user broke the loop
osc.onended = function() {
aCtx.close(); // clear the audioContext
return;
};
}
};
// return a function to stop our loop
return function() {
stopped = true;
};
}
And call it like that :
var stop_anim = audioTimerLoop(yourCallback, 60); // runs 'yourCallback' every 60ms
And to stop it :
stop_anim();
All right, Now we are able to run smooth animation even on blurred windows.
But I want to run it on 16 windows !
Unfortunately, browsers are tied to hardware limitations when creating AudioContext. (e.g. on my computer, I can not have more than 6 contexts running at the same time.)
Here the solution is to run all the code on a single, master window.
From this master window, you'll first
Open all the other windows, so that you can access their content,
Grab the contexts of your canvas elements,
draw on these contexts
This way, you've got a single update thread, on the main window, and all other windows only have to render.
Live Example
(Be sure to allow popups and to disable your ad-blocker though).
So, I'm developing a plugin for webpack for hot-reloading the chrome extensions.
The biggest problem is, if I call a certain number of times the "chrome.runtime.reload()" this will make Chrome block the extension:
This extension reloaded itself too frequently.
On a old discussion they said that if you call the reload more than 5 times in a 10 seconds time frame, your extension will be blocked.
The thing is, I've throttled the reloading a lot (like max 1 call each 5 seconds) and this still happening. I've searched a lot on the docs. but didn't found anything related to this, so I'm kind of in the dark.
So there's really a threshold for this or you only can call the runtime reload a fixed number of times before it blocks the extension?
UPDATE:
To deal with this problem, I've requested a new feature for the Chromium team, Let disable the "Fast Reload" blocking for unpacked extensions. If anyone have the same problems, please give a star on this feature request :)
When the threshold has been reached (i.e. reloaded 5 times in quick succession), you have to wait at least 10 seconds before the counter resets and the extension can safely be reloaded.
Source (trimmed code to emphasize the relevant logic):
std::pair<base::TimeTicks, int>& reload_info =
last_reload_time_[extension_id];
base::TimeTicks now = base::TimeTicks::Now();
if (reload_info.first.is_null() ||
(now - reload_info.first).InMilliseconds() > kFastReloadTime) {
reload_info.second = 0;
} else {
reload_info.second++;
}
// ....
reload_info.first = now;
ExtensionService* service =
ExtensionSystem::Get(browser_context_)->extension_service();
if (reload_info.second >= kFastReloadCount) {
// ....
base::ThreadTaskRunnerHandle::Get()->PostTask(
FROM_HERE, base::BindOnce(&ExtensionService::TerminateExtension,
service->AsWeakPtr(), extension_id));
extensions::WarningSet warnings;
warnings.insert(
extensions::Warning::CreateReloadTooFrequentWarning(
extension_id));
With kFastReloadTime and kFastReloadCount defined here:
// If an extension reloads itself within this many miliseconds of reloading
// itself, the reload is considered suspiciously fast.
const int kFastReloadTime = 10000;
// After this many suspiciously fast consecutive reloads, an extension will get
// disabled.
const int kFastReloadCount = 5;
I am creating a music sequencing app and I used the methodology explained in this article. However when I play my project in Chrome all is fine, when I use Firefox the timing is erratic and semi analogous to using setTimeout or SetInterval stand alone.
My scheduler code is below. The JS file can be viewed here. The working app can be viewed by going to the same link as the js file and replacing "javascript/index.js" with "index.html" ( Stackoverflow won't let me post that url directly ).
A JSfiddle is available here
If you listen in Chrome and then Firefox you can hear the difference, the latter is not good. I am not sure if this is my fault or just the way Firefox is.
function scheduleFutureNote() {
while (futureNote <= audioContext.currentTime + 0.10) { //_________When you've gotten within a Nth of a second is when you schedule the note
playFutureNote(futureNote);
futureNote += (60 / tempo) / 4;
}
if (timer) {
timer = window.setTimeout(scheduleFutureNote, 0.10); //__________sleep for n milliseconds...then check to see if we're close to next note.
} else {
timer === false
}
};
window.setTimeout() is in MILLISECONDS, not seconds. You need to setTimeout( scheduleFutureNode, 100).
I am trying to capture each frame number of the video however it looks like there is no way of achieving it. So I started my own clock to match the frame numbers of the video but they never match and the difference keeps increasing as the video progress.
Please have a look at my bin. http://jsbin.com/dopuvo/4/edit
I have added the frame number to each frame of the video from Adobe After Effect so I have more accurate information of the difference. The Video is running at 29.97fps and the requestAnimationFrame is also set to increase the number at the same rate, however I am not sure where this difference is coming from.
Sometimes they match and sometimes they don't. I also tried doing it offline but I get the same results. Any help.
I found something on github for this. https://github.com/allensarkisyan/VideoFrame
I have implemented it in this fiddle: https://jsfiddle.net/k0y8tp2v/
var currentFrame = $('#currentFrame');
var video = VideoFrame({
id : 'video',
frameRate: 25,
callback : function(frame) {
currentFrame.html(frame);
}
});
$('#play-pause').click(function(){
if(video.video.paused){
video.video.play();
video.listen('frame');
$(this).html('Pause');
}else{
video.video.pause();
video.stopListen();
$(this).html('Play');
}
});
EDIT: updated fiddle to new video so it works again.
EDIT: As pointed out, the video is 25fps, so I updated it, and while I was there removed reliance on jQuery.
Non jQuery version:
https://jsfiddle.net/k0y8tp2v/1/
var currentFrame = document.getElementById('currentFrame');
var video = VideoFrame({
id : 'video',
frameRate: 25,
callback : function(frame) {
currentFrame.innerHTML = frame ;
}
});
document.getElementById('play-pause').addEventListener('click', function(e){
if(video.video.paused){
video.video.play();
video.listen('frame');
e.target.innerHTML = 'Pause';
}else{
video.video.pause();
video.stopListen();
e.target.innerHTML = 'Play';
}
});
The problem is that setTimeout is not really predictable, so you can't be sure that exactly one new frame has been displayed every time your function runs. You need to check the currentTime of the video every time you update your frame display and multiply that by the frame rate.
Here's a working example: http://jsbin.com/xarekice/1/edit It's off by one frame, but it looks like you may have two frames at the beginning marked "000000".
A few things about the video element that you may want to be aware of:
As you seem to have discovered, there's no reliable way to determine the frame rate, so you have to discover it elsewhere and hard-code it. There are some interesting things going on with video metrics, but they're non-standard, not widely supported and, in my experience, completely ineffective at determining the actual frame rate.
The currentTime is not always exactly representative of what's on the screen. Sometimes it's ahead a little bit, especially when seeking (which is why in my JSBin, I don't update the frame while you're seeking).
I believe currentTime updates on a separate thread from the actual video draw, so it kind of works like it's own clock that just keeps going. It's where the video wants to be, not necessarily where it is. So you can get close, but you need to round the results of the frame calculation, and once in a while, you may be off by one frame.
Starting in M83, Chromium has a requestVideoFrameCallback() API, which might solve your issue.
You can use the mediaTime to get a consistent timestamp, as outlined in this Github issue. From there, you could try something like this:
var frameCounter = (time, metadata) => {
let count = metadata.mediaTime * frameRate;
console.log("Got frame: " + Math.round(count));
// Capture code here.
video.requestVideoFrameCallback(frameCounter);
}
video.requestVideoFrameCallback(frameCounter)
This will only fire on new frames, but you may occasionally miss one (which you can detect from a discontinuity in the metadata.presentedFrames count). You might also be slightly late in capturing the frame (usually 16ms, or one call to window.requestAnimationFrame() later than when the video frame is available).
If you're interested in a high level overview of the API, here's a blogpost, or you can take a look at the API's offical github.