how to get programDateTime in hls on ios safari? - javascript

I want to get the programDateTime which is a timestamp in the browser to measure the delay time of HLS. I knew how to get it on the desktop version of Chrome and Safari. However, there is no way to get it with iOS Safari. Do you know any good way?
The following is how to get programDateTime using hls.js. However, this method can not be used because iOS plays hls natively.
hls.on(Hls.Events.FRAG_CHANGED, function(id, data){
if (data.frag._url) {
console.log('data',data)
if (data.frag.programDateTime){
var progaramDateTime = data.frag.programDateTime;
} else {
// for macOS Safari
var rawProgaramDateTime = data.frag.tagList[2][1];
rawProgaramDateTime = rawProgaramDateTime.slice(0, -2) + ":" + rawProgaramDateTime.slice(-2);
var progaramDateTime = Date.parse(rawProgaramDateTime);
}
programDateTimeArray.push(progaramDateTime);
console.log('programDateTimeArray',programDateTimeArray)
if (programDateTimeArray.length == 5){
programDateTimeArray.shift();
chunkTime = programDateTimeArray[3] - programDateTimeArray[0];
}
var programDateTime = progaramDateTime - chunkTime - 2000;
mcDelayTime = getCurrentTime() - programDateTime;
console.log(data.frag._url, mcDelayTime+"ミリ秒遅延しています");
}
});

On Safari for native HLS playback you can use this method:
https://developer.apple.com/documentation/webkitjs/htmlmediaelement/1634352-getstartdate
If there is a program date time in the HLS stream the first one will be stored as the date returned by this method.
From there, you should be able to combine it with the video element currentTime attribute to get the current program date time. With those two values you should be able to measure your delay time in a similar fashion.

Related

Blocked attempt to create a WebMediaPlayer as there are too many WebMediaPlayers already in existence

We are working on a Digital Audio Workstation kind of thing in the browser. We need to work with multiple audio files in a tab. We use new Audio(audioUrl) to be able to play the audio in our own audio mixer. It has been working for us up to now.
With the latest version of Chrome (92), we have the problem where the above code snippet causes the following error:
[Intervention] Blocked attempt to create a WebMediaPlayer as there are too many WebMediaPlayers already in existence. See crbug.com/1144736#c27
I cannot access the bug link provided, it says permission denied. And is there a suggested workaround to handle this?
UPDATE:
I moved away from using HTMLAudioElement to AudioBufferSourceNode. Seems like the only straightforward solution as Chrome team is discussing to limit them anyway. Note: We may need more than 1000 audio clips to be played back. This is in reference to the chromium discussion thread where they are going to increase the number of webmediaplayers to 1000 on the next release for August 5 2021.
Chrome 92 has introduced a limit on number of audio and video tags that can be allocated in a particular tab.
75 for desktop browsers and 40 for mobile browsers.
For now the only solution is to limit the number of audio and video tags created in the page. Try reusing the already allocated audio / video elements.
The number can only be increased by passing the following flag when starting up chrome, for example --max-web-media-player-count=5000
(Of course we cannot expect the end user to do this)
Related Source code here:
https://chromium-review.googlesource.com/c/chromium/src/+/2816118
Edit:
Before deallocating the audio/video elements setting the following seems to force clean up of the element.
mediaElement.remove();
mediaElement.srcObject = null;
const MaxWebMediaPlayerCount = 75;
class VideoProducer {
static #documentForVideo
static createVideo() {
if (!this.#documentForVideo || this.#documentForVideo.videoCount === MaxWebMediaPlayerCount) {
const iframeForVideo = document.body.appendChild(document.createElement('iframe'));
iframeForVideo.style.display = 'none';
iframeForVideo.contentDocument.videoCount = 0;
this.#documentForVideo = iframeForVideo.contentDocument;
}
this.#documentForVideo.videoCount++;
const video = this.#documentForVideo.createElement('video');
return video;
}
foo() {
const video = VideoProducer.createVideo();
// ...
}
Yeah me too it broke my game,
This is what I found as a workaround, hope this helps in the mean time:
function playSound( ) {
var jump_sound = new Audio("./jump.mp3");
jump_sound.play();
jump_sound.onended = function(){
this.currentSrc = null;
this.src = "";
this.srcObject = null;
this.remove();
};
}
Note: it still blocks if there's too many concurrent sound but with this code in place the blocking is temporary.
Chrome version 92.0.4515.131 seems to resolve the issue

Trying to run a HTML5 accelerometer listener in the background on mobile

I'm currently in the process of creating a webapp that detects whether the user is sitting or standing. I have had great success with detecting the tilt of the device, changing a boolean variable and sending the correct data to a graph.
The following code runs perfectly in the background, updating the data every second.
var trackInterval = setInterval(function(){
if(isSitting){
addData(myPieChart, "Sitting");
} else{
addData(myPieChart, "Standing");
}
}, 1000)
My issue is that the listening function that changes the variable 'isSitting' does not continue once the browser is closed. This means that the last value of 'isSitting' gets data added to it, even though the device might be tilted otherwise.
This is the code that creates the accelerometer updates:
window.addEventListener("devicemotion", accelerometerUpdate, true);
var isSitting = true;
function accelerometerUpdate(event) {
var aX = event.accelerationIncludingGravity.x * 100 ;
var aY = event.accelerationIncludingGravity.y * 100 ;
var aZ = event.accelerationIncludingGravity.z * 100 ;
if (aY > 600 || aY < -900 ){
isSitting = false;
} else{
isSitting = true;
}
}
I have tried Chrome, Opera and Firefox on my Android device.
Any help or tips to work around this would be greatly appreciated
This sounds like regular behaviour as your javascript code, to be executed, should run in your browser.
To achieve your goal, a possible guess would be to look into a way to use this api in service workers, which is currently not possible with the devicemotion api but should be (at least with chrome) with the new Generic Sensors API described here : https://developers.google.com/web/updates/2017/09/sensors-for-the-web

Create Seamless Loop of Audio - Web

I want to create a seamless loop of an audio file. But in all approaches I used so far, there was a noticeable gap between end & start.
This is what I tried so far:
First approach was to use the audio in the HTML and it loops but there is still a noticeable delay when going from the end of the track to the beginning.
<audio loop autoplay>
<source src="audio.mp3" type="audio/mpeg">
<audio>
Then I tried it from JavaScript with the same result:
let myAudio = new Audio(file);
myAudio.loop = true;
myAudio.play();
After that I tried this (according to this answer)
myAudio.addEventListener(
'timeupdate',
function() {
var buffer = .44;
if (this.currentTime > this.duration - buffer) {
this.currentTime = 0;
this.play();
}
},
false
);
I played around with the buffer but I only got it to reduce the gap but not leave it out entirely.
I turned to the library SeamlessLoop (GitHub) and got it to work to loop seamlessly in Chromium browsers (but not in the latest Safari. Didn't test in other browsers). Code I used for that:
let loop = new SeamlessLoop();
// My File is 58 Seconds long. Btw there aren't any gaps in the file.
loop.addUri(file, 58000, 'sound1');
loop.callback(soundsLoaded);
function soundsLoaded() {
let n = 1;
loop.start('sound' + n);
}
EDIT: I tried another approach: Looping it trough two different audio elements:
var current_player = "a";
var player_a = document.createElement("audio");
var player_b = document.createElement("audio");
player_a.src = "sounds/back_music.ogg";
player_b.src = player_a.src;
function loopIt(){
var player = null;
if(current_player == "a"){
player = player_b;
current_player = "b";
}
else{
player = player_a;
current_player = "a";
}
player.play();
/*
3104.897 is the length of the audio clip in milliseconds.
Received from player.duration.
This is a different file than the first one
*/
setTimeout(loopIt, 3104.897);
}
loopIt();
But as milliseconds in browsers are not consistent or granular enough this doesn't work too well but it does work much better than the normal "loop" property of the audio.
Can anyone guide me into the right direction to loop the audio seamlessly?
You can use the Web Audio API instead. There are a couple of caveats with this, but it will allow you to loop accurately down to the single sample level.
The caveats are that you have to load the entire file into memory. This may not be practical with large files. If the files are only a few seconds it should however not be any problem.
The second is that you have to write control buttons manually (if needed) as the API has a low-level approach. This means play, pause/stop, mute, volume etc. Scanning and possibly pausing can be a challenge of their own.
And lastly, not all browsers support Web Audio API - in this case you will have to fallback to the regular Audio API or even Flash, but if your target is modern browsers this should not be a major problem nowadays.
Example
This will load a 4 bar drum-loop and play without any gap when looped. The main steps are:
It loads the audio from a CORS enabled source (this is important, either use the same domain as your page or set up the external server to allow for cross-origin usage as Dropbox does for us in this example).
AudioContext then decodes the loaded file
The decoded file is used for the source node
The source node is connected to an output
Looping is enabled and the buffer is played from memory.
var actx = new (AudioContext || webkitAudioContext)(),
src = "https://dl.dropboxusercontent.com/s/fdcf2lwsa748qav/drum44.wav",
audioData, srcNode; // global so we can access them from handlers
// Load some audio (CORS need to be allowed or we won't be able to decode the data)
fetch(src, {mode: "cors"}).then(function(resp) {return resp.arrayBuffer()}).then(decode);
// Decode the audio file, then start the show
function decode(buffer) {
actx.decodeAudioData(buffer, playLoop);
}
// Sets up a new source node as needed as stopping will render current invalid
function playLoop(abuffer) {
if (!audioData) audioData = abuffer; // create a reference for control buttons
srcNode = actx.createBufferSource(); // create audio source
srcNode.buffer = abuffer; // use decoded buffer
srcNode.connect(actx.destination); // create output
srcNode.loop = true; // takes care of perfect looping
srcNode.start(); // play...
}
// Simple example control
document.querySelector("button").onclick = function() {
if (srcNode) {
srcNode.stop();
srcNode = null;
this.innerText = "Play";
} else {
playLoop(audioData);
this.innerText = "Stop";
}
};
<button>Stop</button>
There is a very simple solution for that, just use loopify it makes use of the html5 web audio api and works perfectly well with many formats, not only wav as the dev says.
<script src="loopify.js" type="text/javascript"></script>
<script>
loopify("yourfile.mp3|ogg|webm|flac",ready);
function ready(err,loop){
if (err) {
console.warn(err);
}
loop.play();
}
</script>
This will automatically play the file, if you want to have start and stop buttons for example take a look at his demo

Webaudio sound stops on Chrome for Android after about 2 minutes

I'm running into an issue with WebAudio on Chrome for Android.
I'm experiencing this on a Samsung Galaxy S3 (GT-I9300) with:
Chrome version 44.0.2403.133
Android 4.3.0
Here is the code I'm using to try and isolate the issue:
var audioContext;
if(window.AudioContext) {
audioContext = new AudioContext();
}
var startTime = Date.now();
var lastTrigger;
var gain = audioContext.createGain();
gain.gain.value = 1;
gain.connect(audioContext.destination);
var buttonTrigger = document.getElementById('trigger');
buttonTrigger.addEventListener('click', function(event) {
var oscillator = audioContext.createOscillator();
oscillator.type = "square";
oscillator.frequency.value = 100 + (Math.cos(audioContext.currentTime)*100);
oscillator.connect(gain);
oscillator.start(0);
oscillator.stop(audioContext.currentTime+0.1);
lastTrigger = Date.now();
});
var timer = document.getElementById('timer');
setInterval(function() {
if(lastTrigger) { timer.textContent = Date.now() - lastTrigger; }
}, 1000);
And here it is on jsfiddle
This simply creates an oscillator node and plays on clicking a button. On my phone, if I do not click the button for about a minute and a half or two minutes, I no longer get any sound.
There are no errors thrown.
Does anyone have any experience of this issue and a possible workaround?
This issue originally appeared in a much larger app using Phaser to play sounds from a m4a file, so this is not solely to do with the oscillator.
UPDATE
According to the Chromium bug ticket this issue has now been fixed.
After experiencing the same problem on Android. I found a better solution than playing a "dummy sound" every 30sec.
Just remember the time when you last played over your context:
var lastPlayed = new Date().getTime();
var audioSource = context.createBufferSource();
audioSource.connect( context.destination );
audioSource.buffer = sb;
audioSource.start( 0 );
The next time you play a sample/sound Just check the time passed and reset the AudioContext
if(new Date().getTime()-lastPlayed>30000){ // Time passed since last playing is greater than 30 secs
context.close();
context=new AudioContext();
}
For Android this works like charm.
I think what you're seeing is an auto-shutdown of Web Audio when there's no sound for a while. What happens if you click the button a second time, a second or so after the first? (Web Audio can take some time (order of tens of milliseconds, at least) to restart.)
The suspend()/resume() methods, and looking at the context.state, would be helpful here.
#RaymondToy's comment answers this question. There is a bug with Chrome on Android (at least for Samsung Galaxy S3/4). Webaudio stops playing sounds after a period of inactivity. Where inactivity is essentially silence.
The only work around I can find is to play some kind of sound at intervals. I have experimented with playing a sound every 30 seconds and that stopped the problem.
I also tried playing some kind of silent noise (silent audio buffer or silent part of an m4a audio file or muted sound), neither of which solved the problem.

Is it possible, in JavaScript, to detect when the screen is turned off in the Android & iOS browsers

I was tracking down some ridiculously high load times that my app's javascript reported, and found that Android (and iOS) pause some JavaScript execution when the window is in the background or the display is off.
On Android, I found that I could use the window.onfocus and onblur events to detect when the app was switching to the background (and js execution would soon be paused, at least for new scripts), but I can't find a way to detect when the screen is turned on or off. Is this possible?
(On Safari, I had similar results except that onfocus and onblur didn't fire reliably.)
There is few options to check it:
Using Visibility API
Using focus and blur events to detect browser tab visibility:
window.addEventListener("focus", handleBrowserState.bind(context, true));
window.addEventListener("blur", handleBrowserState.bind(context, false));
function handleBrowserState(isActive){
// do something
}
Using timers, as mentioned above
I just found a pretty good solution for my use case:
function getTime() {
return (new Date()).getTime();
}
var lastInterval = getTime();
function intervalHeartbeat() {
var now = getTime();
var diff = now - lastInterval;
var offBy = diff - 1000; // 1000 = the 1 second delay I was expecting
lastInterval = now;
if(offBy > 100) { // don't trigger on small stutters less than 100ms
console.log('interval heartbeat - off by ' + offBy + 'ms');
}
}
setInterval(intervalHeartbeat, 1000);
When the screen is turned off (or JS is paused for any reason), the next interval is delayed until JS execution resumes. In my code, I can just adjust the timers by the offBy amount and call it good.
In quick testing, this seemed to work well on both Android 4.2.2's browser and Safari on iOS 6.1.3.
Found a nice function here:
http://rakaz.nl/2009/09/iphone-webapps-101-detecting-essential-information-about-your-iphone.html
(function() {
var timestamp = new Date().getTime();
function checkResume() {
var current = new Date().getTime();
if (current - timestamp > 4000) {
var event = document.createEvent("Events");
event.initEvent("resume", true, true);
document.dispatchEvent(event);
}
timestamp = current;
}
window.setInterval(checkResume, 1000);
})();
To register for event:
addEventListener("resume", function() {
alert('Resuming this webapp');
});
This is consistent with Cordova which also fires the resume event.
what will you do in your script once you now that the screen turns off? Well anyway, you can inject Java objects ( http://developer.android.com/reference/android/webkit/WebView.html#addJavascriptInterface(java.lang.Object,%20java.lang.String) ) to interface with the activity and proxy all information you require in JS world.

Categories

Resources