I'm trying to make a cross-device/browser image and audio preloading scheme for a GameAPI I'm working on. An audio file will preload, and issue a callback once it completes.
The problem is, audio will not start to load on slow page loads, but will usually work on the second try, probably because it cached it and knows it exists.
I've narrowed it down to the audio.load() function. Getting rid of it solves the problem, but interestingly, my motorola droid needs that function.
What are some experiences you've had with HTML5 audio preloading?
Here's my code. Yes, I know loading images in a separate function could cause a race condition :)
var resourcesLoading = 0;
function loadImage(imgSrc) {
//alert("Starting to load an image");
resourcesLoading++;
var image = new Image();
image.src = imgSrc;
image.onload = function() {
//CODE GOES HERE
//alert("A image has been loaded");
resourcesLoading--;
onResourceLoad();
}
}
function loadSound(soundSrc) {
//alert("Starting to load a sound");
resourcesLoading++;
var loaded = false;
//var soundFile = document.createElement("audio");
var soundFile = document.createElement("audio");
console.log(soundFile);
soundFile.autoplay = false;
soundFile.preload = false;
var src = document.createElement("source");
src.src = soundSrc + ".mp3";
soundFile.appendChild(src);
function onLoad() {
loaded = true;
soundFile.removeEventListener("canplaythrough", onLoad, true);
soundFile.removeEventListener("error", onError, true);
//CODE GOES HERE
//alert("A sound has been loaded");
resourcesLoading--;
onResourceLoad();
}
//Attempt to reload the resource 5 times
var retrys = 4;
function onError(e) {
retrys--;
if(retrys > 0) {
soundFile.load();
} else {
loaded = true;
soundFile.removeEventListener("canplaythrough", onLoad, true);
soundFile.removeEventListener("error", onError, true);
alert("A sound has failed to loaded");
resourcesLoading--;
onResourceLoad();
}
}
soundFile.addEventListener("canplaythrough", onLoad, true);
soundFile.addEventListener("error", onError, true);
}
function onResourceLoad() {
if(resourcesLoading == 0)
onLoaded();
}
It's hard to diagnose the problem because it shows no errors and only fails occasionally.
I got it working. The solution was fairly simple actually:
Basically, it works like this:
channel.load();
channel.volume = 0.00000001;
channel.play();
If it isn't obvious, the load function tells browsers and devices that support it to start loading, and then the sound immediately tries to play with the volume virtually at zero. So, if the load function isn't enough, the fact that the sound 'needs' to be played is enough to trigger a load on all the devices I tested.
The load function may actually be redundant now, but based off the inconsistiency with audio implementation, it probably doesn't hurt to have it.
Edit: After testing this on Opera, Safari, Firefox, and Chrome, it looks like setting the volume to 0 will still preload the resource.
canplaythrough fires when enough data has buffered that it probably could play non-stop to the end if you started playing on that event. The HTML Audio element is designed for streaming, so the file may not have completely finished downloading by the time this event fires.
Contrast this to images which only fire their event once they are completely downloaded.
If you navigate away from the page and the audio has not finished completely downloading, the browser probably doesn't cache it at all. However, if it has finished completely downloading, it probably gets cached, which explains the behavior you've seen.
I'd recommend the HTML5 AppCache to make sure the images and audio are certainly cached.
The AppCache, as suggested above, might be your only solution to keep the audio cached from one browser-session to another (that's not what you asked for, right?). but keep in mind the limited amount of space, some browsers offer. Safari for instance allows the user to change this value in the settings but the default is 5MB - hardly enough to save a bunch of songs, especially if other websites that are frequented by your users use AppCache as well. Also IE <10 does not support AppCache.
Alright so I ran into the same problem recently, and my trick was to use a simple ajax request to load the file entirely once (which end into the cache), and then by loading the sound again directly from the cache and use the event binding canplaythrough.
Using Buzz.js as my HTML5 audio library, my code is basically something like that:
var self = this;
$.get(this.file_name+".mp3", function(data) {
self.sound = new buzz.sound(self.file_name, {formats: [ "mp3" ], preload: true});
self.sound.bind("error", function(e) {
console.log("Music Error: " + this.getErrorMessage());
});
self.sound.decreaseVolume(20);
self.sound.bind("canplaythrough",function(){ self.onSoundLoaded(self); });
});
Related
I have a html/javascript client that is listening to a mjpeg video stream:
myImg = document.getElementById('my-image');
myImg.src = 'http://myserver.com/camera.mjpeg';
Works fine but if the video stream dies for whatever reason the video feed "freezes" on the last received image and I have no opportunity to display an error to the user. I've see this post that offers a solution (creating a long running ajax request alongside the stream) that only works some of the time. I was hoping there would be a more supported method like through a disconnect event or something.
Even an event for when data is received would be better than nothing. At least that way I could tell if it's been a while since a frame came through. Using addEventListener('load') only works on the very first frame.
Any ideas?
Update:
Based on comments I have tried the following approaches, none of which has worked:
myImg.addEventListener('error', event => { ... });
myImg.addEventListener('stalled', event => { ... });
myImg.addEventListener('suspend', event => { ... });
This is common with a normal implementation of a mjpeg, for example
<video src="http://myserver.com/camera.mjpeg" controls>
Your browser does not support the <code>video</code> element.
</video>
the mjpeg is a series of images and eventually it will not get the next one for whatever reason, breaking the connection. (this is sometimes because the source is cached, causing the browser to use the last image every time). I don't consider this an error, more something to program around with mjpeg streams.
A simple solution you can do, set a refresh rate and set the src continuously refreshing the connection every ~500ms (or less depending on your network connection/resources).
setInterval(function() {
var myImg = document.getElementById('myImg');
myImg.src = 'http://myserver.com/camera.mjpeg?rand=' + Math.random();
}, 5000);
The random number is added to prevent browser side caching in the event the server sends those headers.
Or you can create a ReadableStream, and keep reading a blob of bytes directly into the source of the image. There is a robust example in this repo, from this other question.
In Safari document.readyState will change from interactive to complete.
For example put this before the image loads:
<script>
console.log('Initial ready state', document.readyState);
document.onreadystatechange = function() {
console.log('Ready state changed to:', document.readyState);
}
</script>
And the output will be:
Initial ready state – "loading"
Ready state changed to: – "interactive"
// When the connection disconnects:
Ready state changed to: – "complete"
In google chrome the readyState doesn't stay on interactive, but it looks like chrome is better at reconnecting, so might not be an issue for you.
Edit: One way to make use of this is to drop the image in an iframe, you'll continually get load events in safari (this does not work in chrome).
iframe = document.createElement('iframe')
iframe.onload = console.log
iframe.src = "http://10.0.0.119:8080/stream"
document.body.append(iframe)
Edit2: Another technique -- use image.decode to detect when the connection is down and reload the image:
<img id="stream" src="http://10.0.0.119:8080/stream"></img>
<script>
let image = document.getElementById('stream');
async function check() {
while (true) {
try {
await image.decode();
} catch {
let src = image.src;
image.src = "";
image.src = src;
}
await new Promise((resolve) => setTimeout(resolve, 5000));
}
}
check();
</script>
Would something like this work?
function hasLoaded(myImg) {
return myImg.complete && myImg.naturalHeight !== 0;
}
Following Beau Bouchard answers.
The setInterval timer, works fine but it tends to max out active client listening. ( if ur mjpeg stream are coming directly from an IP Camera). Could possibly create a restreaming server the mjpeg server to allow more clients to be able to be listening to it) Short pooling though does tend to be very resource heavy.
Tried the Restream Api as well. When loading the image back into the img tag, you do get a jittery effect, most likely because the chunks are coming in randomly and not smooth out via time?
In the end, i use the onload img tag event. This triggers whenever an img is loaded. Then a time interval to check if the img tag has stop loading to determine if the mjpeg stream has stop.
Met the same requirement, test on Image onload event and works!!
If FFMPEG feed FFSERVER stop, although the MJPEG in a still image,
After couple times error count, mjpeg FAIL! detected.
<!DOCTYPE HTML>
<HTML>
<HEAD>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<TITLE> mjpeg detect </TITLE>
<script type="text/javascript" language="JavaScript" src="/js/jquery.js"></script>
<script type="text/javascript" language="JavaScript">
//------------------------------------------------------------------------------
// localhost/tool/mjpeg.htm
// document.ready
$(function(){
setTimeout("mjpegRefresh()", 10000);
});
var mTmjpegRefresh, mBmjpegStatus=0, mNmjpegError=0;
var mjpegRefresh = function()
{
clearTimeout(mTmjpegRefresh);
mBmjpegStatus=0;
$('#myMJPEG').attr('src', "http://192.168.1.17:8090/live.mjpeg?rand=" + Math.random());
console.log("mjpeg refresh: ", Math.round( (new Date()).getTime()/1000)) ;
mTmjpegRefresh = setTimeout("mjpegRefresh()", 10000);
mTmjpegStatusCheck = setTimeout("mjpegStatusCheck()", 5000);
}
var mjpegOnload = function()
{
console.log("mjpeg Onload");
mBmjpegStatus=1;
}
var mTmjpegStatusCheck;
var mjpegStatusCheck = function()
{
clearTimeout(mTmjpegStatusCheck);
if(mBmjpegStatus>0)
{
mNmjpegError=0;
}
else
{
mNmjpegError++;
}
if(mNmjpegError>5)
{
console.log("mjpeg FAIL!");
}
mTmjpegStatusCheck = setTimeout("mjpegStatusCheck()", 5000);
}
//------------------------------------------------------------------------------
</script>
</HEAD>
<BODY>
<img src="http://192.168.1.17:8090/live.mjpeg" width="720" height="404" id="myMJPEG" onload="mjpegOnload()">
</BODY>
</HTML>
I am using blob:https as source for my video-tags, like this:
function mk_bloburl(source_id, url) {
var xhr = new XMLHttpRequest();
xhr.open('GET', url, true);
xhr.responseType = 'blob'; //important
xhr.onload = function(e) {
if (this.status == 200) {
var blob = this.response;
var source = document.getElementById(source_id);
var video = source;
if (video.tagName=="SOURCE") {
video = source.parentNode;
}
video.oncanplaythrough = function() {
URL.revokeObjectURL(source.src);
};
source.src = URL.createObjectURL(blob);
video.load();
}
};
xhr.send();
}
mk_bloburl('review-video-source', my_video_url );
Having HTML like this:
<video id="review-video" controls="" width="100%">
<source id="review-video-source" />
</video>
Now if I reload the page and start playing, it works. If I play and let it play through and then replay with no wait-time it works.
If I however reload the page and then wait for a while (like 1-2 minutes) and then press play, it fails.
Message I get in the Chrome browser looks like this:
GET blob:https://example.com/debeecfe-49b0-4c98-87d6-8ead325b2d75 404 (Not Found)
So, it's like the blob is auto-removed from the browser memory after a while. I want to catch the event when it is erased or when I get the 404 by starting the playback, so I can refresh the blob.
I have tried:
var source = document.querySelector("#review-video-source");
source.addEventListener("error", function(event) {
console.debug("An error accoured");
});
But this does not seem to catch the error.
What can I do?
Your main problem is caused by the fact you revoked the blobURI in the canplaythrough event.
The canplaythrough event is just a notice sent by the browser to let us know it thinks it will be able to load and play the whole media without interruption ; it doesn't mean that it has loaded everything yet.
In the case of BlobURI, the connection speed is so fast (it comes from memory) that the browser could think it is able to fetch the StarWars saga in 4k in a blink.
So you get this canplaythrough event really early, but the browser didn't actually uncompressed all the data yet. Still, you revoke the BlobURI, and when the browser tries again to fetch data so it can uncompress and read it, there is nothing anymore at the end of the BlobURI's pointer.
So for your problem, you've got two solutions :
In case you need to play the media only once :
call URL.revokeObjectURL(blobURI) in the ended event. This will fire, the first time the currentTime of the video will reach the end.
If you need to play the video multiple times :
revoke the blobURI in the beforeunload event of the page. This way, your BlobURI pointer is always active for as long as the page is alive, but will not block the whole Blob in memory for longer than the page life (which would happen if you don't revoke the BlobURI at all).
And about how to Detect 404 on video blob:https source, I don't really know a good way, except listening for an unexpected jump to the end, but this should not be needed for blobURIs anyway.
One way to get around it is using the fetch API like so :
function mk_bloburl(source_id, url){
fetch(url)//usual get
.then(response=>{
if(response.status == 404)
//Should set here what is an error (ex: x>299)
Promise.reject("Error 404!");
return response.blob();
})//get blob
.then(blobObj=>{//use blob
var blob = blobObj;
var source = focument.getElementById(source_id);
var video = source;
if(video.tagName === "SOURCE")
video = source.parentNode;
video.oncanplaythrough = ()=>{
URL.revokeObjectUrl(source.src);
};
source.src = URL.revokeObjectURL(blob);
video.load();
})
.catch(error=>{
//There has been an error,
//do something about it
//here
});
}
I followed the instructions in this article and created a Javascript metronome. It makes use of the Web Audio API and has audioContext.currentTime at its core for precise timing.
My version, available at this plunker, is a very simplified version of the original, made by Chris Wilson and available here. In order for mine to work, since it uses an actual audio file and doesn't synthesize sounds through an oscillator, you need to download the plunker and this audio file, placing it in the root folder (it's a metronome 'tick' sound, but you could use any sound you want).
It works like a charm - if it weren't for the fact that if the user minimizes the window, the otherwise very accurate metronome starts hiccupping instantly and awfully. I really don't understand what is the problem, here.
Javascript
var context, request, buffer;
var tempo = 120;
var tickTime;
function ticking() {
var source = context.createBufferSource();
source.buffer = buffer;
source.connect(context.destination);
source.start(tickTime);
}
function scheduler() {
while (tickTime < context.currentTime + 0.1) { //while there are notes to schedule, play the last scheduled note and advance the pointer
ticking();
tickTime += 60 / tempo;
}
}
function loadTick() {
request = new XMLHttpRequest(); //Asynchronous http request (you'll need a local server)
request.open('GET', 'tick.wav', true); //You need to download the file # http://s000.tinyupload.com/index.php?file_id=89415137224761217947
request.responseType = 'arraybuffer';
request.onload = function () {
context.decodeAudioData(request.response, function (theBuffer) {
buffer = theBuffer;
});
};
request.send();
}
function start() {
tickTime = context.currentTime;
scheduleTimer = setInterval(function () {
scheduler();
}, 25);
}
window.onload = function () {
window.AudioContext = window.AudioContext || window.webkitAudioContext;
context = new AudioContext();
loadTick();
start();
};
Yeah, this is because the browsers throttle setTimeout and setInterval to once per second when the window loses focus. (This was done to circumvent CPU/power drain due to developers using setTimeout/setInterval for visual animation, and not pausing the animation when the tab lost focus.)
There are two ways around this:
1) increase the "look-ahead" (in your example, 0.1 second) to greater than one second - like, 1.1s. Unfortunately, this would mean you couldn't change things (like stopping the playback, or changing the tempo) without a more-than-one-second lag in the change; so you'd probably want to only increase that value when the blur event was fired on the window, and change it back to 0.1 when the window focus event fired. Still not ideal.
2) Circumvent the throttling. :) It turns out you can do this, because setTimeout/setInterval are NOT throttled in Web Workers! (This approach was originally suggested by someone in the comment thread of my original article at http://www.html5rocks.com/en/tutorials/audio/scheduling/#disqus_thread.) I implemented this for the metronome code in https://github.com/cwilso/metronome: take a look at the js/metronome.js and js/metronomeworker.js. The Worker basically just maintains the timer, and marshals a message back across to the main thread; take a look at https://github.com/cwilso/metronome/blob/master/js/metronome.js#L153, in particular, to see how it's kicked off. You could modify that snippet of code and use metronomeworker.js as-is to fix this.
I have a web app that loads instagram images and creates a slideshow with background sound. While the images are being loaded, a pre loader is run to show the processing. But it doesn't fully load the mp3 file and when I check in the browser's console it has got 206 partial content. So, on clicking the play button, I have to wait a few second so that music is loaded fully to play the slideshow.
console.log('loading theme: ' + folder);
$.ajaxSetup({
cache: false
});
$.when(
$.getScript(epic.getFrontendURL() + "animations/" + folder + "/js/data.js"),
$.getScript(epic.getFrontendURL() + "animations/" + folder + "/js/main.js"),
$.Deferred(function (deferred) {
$(deferred.resolve);
})
).done(function () {
//some methods
sound_bg.src = epic.getFrontendURL() + "sound/back.mp3?
});
So, is this related somehow, with done which delays the processing.
This behavior is normal. Browsers use HTTP range requests to load audio and video files to minimize bandwidth usage.
If you want the browser to preload the audio, set the preload attribute to auto on the audio element:
<audio src="..." preload="auto"></audio>
I found a way to check if there are errors, if there are sound errors, your website will reload the faulty sound, until it's loaded right. This has worked for me:
var sound1 = new Audio('sfx/1.mp3')
var sound2 = new Audio('sfx/2.mp3')
var sound3 = new Audio('sfx/3.mp3')
var soundChecker = setInterval(function() {
if (sound1.error) {
sound1 = new Audio('sfx/1.mp3')
} else if (sound2.error) {
sound2 = new Audio('sfx/2.mp3')
} else if (sound3.error) {
sound3 = new Audio('sfx/3.mp3')
} else {
clearInterval(soundChecker)
}
}, 1500)
This will check for errors when loading sounds every one and a half seconds (you could modify the time for slower connections), if there is an error with any of the loaded sounds, soundChecker will reload them until all sounds have been loaded successfully (without errors). If all sounds have been loaded successfully the soundChecker interval will clear itself and stop checking.
You will notice that errors such as net::ERR_CACHE_READ_FAILURE 206 (Partial Content) will still be printed in the console but the sound will work, because in a following reload it was successfully loaded.
I'm having a bit of trouble rewinding audio in Javascript. I basically have a countdown that beeps each second as it gets towards the end of the countdown.
I tried using;
var bip = new Audio("http://www.soundjay.com/button/beep-7.wav");
bip.play();
but it didn't beep every second which I'm guessing has something to do withit having to load a new sound every second. I then tried loading the sound externally and triggering it with the following code.
bip.pause();
bip.currentTime = 0;
console.log(bip.currentTime);
bip.play();
but this only plays the sound once then completely fails to rewind it (this is shown by the console logging a time of 0.19 seconds after the first playthrough).
Is there something I'm missing here?
In google chrome I noticed that it only works if you have the audio file in same domain. I have no problems if the audio file is in same domain. Event setting .currentTime works.
Cross-domain:
var bip = new Audio("http://www.soundjay.com/button/beep-7.wav");
bip.play();
bip.currentTime; //0.10950099676847458
bip.currentTime = 0;
bip.currentTime; //0.10950099676847458
Same-domain:
var bip = new Audio("beep-7.wav");
bip.play();
bip.currentTime; //0.10950099676847458
bip.currentTime = 0;
bip.currentTime; //0
I tried googling for a while and could find nothing in specs about this or even any discussion.
when I want rewind I simply load the audio again:
my_audio.load()
*btw, I also use a eventlistener for 'canplay' to trigger the my_audio.play(). It seems that this is necessary in android, and maybe other devices also*
To further dsdsdsdsd's answer with a bit of paint-by-numbers for the "whole shebang"
NOTE: In my app, loc1 is a dummy that refers to the song's stored location
// ... code before ...
tune = new Audio(loc1); // First, create audio event using js
// set up "song's over' event listener & action
tune.addEventListener('ended', function(){
tune.load(); //reload audio event (and reset currentTime!)
tune.play(); //play audio event for subsequent 'ended' events
},false);
tune.play(); // plays it the first time thru
// ... code after ...
I spent days and days trying to figure out what I was doing wrong, but it all works fine now... at least on the desktop browsers...
As of Chrome version 37.0.2062.120 m, the behaviour described by #Esailija has not changed.
I workaround this issue by encoding the audio data in base64 encoding and feed the data to Audio() using data: URL.
Test code:
snd = new Audio('data:audio/ogg;base64,[...base64 encoded data...]');
snd.onloadeddata = function() {
snd.currentTime = 0;
snd.play();
setTimeout(function() {
snd.currentTime = 0;
snd.play();
}, 200);
};
(I am surprised that there are no bug reports or references on this matter... or maybe my Google-fu is not strong enough.)