Track when the user received first bytes of the video - javascript

There is a web page which has HTML5 video in it. When the user clicked start or when he navigates through the timeline, the video starts (either from start or from the position he selected). But it does not always happens instantly. I wanted to find how much time did it took from the user click event and the time the user received first bytes of the video.
Getting time of userclick is not a problem, but while looking through HTML5 video API here and I was not able to find any event which is close to what I am looking for.
Is it possible to tack such event?

The event(s) you listen for after you receive the click (or "play" or "seeking") event depends on the state of the video before the time of the click.
If you have a fresh, unplayed video element with the preload attribute set to "none", then the first data you're going to receive from the network is the metadata. so you can listen for the "loadedmetadata" event.
If preload is set to "metadata", you might have already loaded metadata, depending on the browser and platform. (e.g., Safari on iPad will not load metadata or anything else until the first user interaction.) In that case, you want to listen for either "loadedmetadata" or "progress". It couldn't hurt to listen for "loadeddata" as well, but I think "progress" fires first.
If preload is set to "auto" or if you've already played some of the video, you might have some actual video data. And while you're likely to have data at the current point on the timeline, you may or may not have it at the seek destination. It depends at least on how far ahead (or behind) you're seeking, how fast data is coming in and how much spare room the browser has in the media cache.
If there is no data at the destination time (you can check this in advance if you want with the .buffered property, see TimeRanges), then the next event you see will be either "loadeddata" or "progress", probably followed by "canplay". If there is enough data buffered at the target time of the seek, then the question doesn't really apply because nothing else will be transferred.
However, in any of the above cases, once there is enough data to display the frame at the new point on that timeline and that data has been decoded, the "seeked" event will fire. So if you were to only pick one (no reason you can't use more), this is the one to pick.

Related

How to update a web page javascript counter live when the browser doesn't have focus?

I am making a browser game in html, css, and javascript, written in perl. Health and stamina are kept in the server and I use javascript to show the user a live updated count of these stats while the current page is loaded. This works fine, however if the user switches tabs or switches away from the browser and leaves it running in the background, the count value you see when you return does not keep up properly. So when you switch back to the browser, your counter might say 50/100 stamina when you actually have 100/100. So when you do something in the game (loads a new page) the server updates the counter to the true amount because the javascript is just keeping time to show the user a "live" rolling view in the browser.
Is there a way to ensure the javascript counter will continue to function even if the page/tab isn't active or on the forefront? Aside from completely re-writing my game to include continuous live server pushes in what is displayed on the browser to the user?
Say you are playing the game. You see your health and stamina regenerating. You switch to another program for a minute, then return to the game in the browser. You notice your health and stamina have not updated while you were away. But when you perform an action in the game, this value is updated to what it should be because it is tracked internally on the server. This is what I would like to fix. Hope that makes sense!
I have not tried anything to fix this issue yet besides searching the web and ending up on this site without a really "good" answer in sight, so I decided to ask the question.
Continuous server pushes wouldn't work either. Anything in the main event loop like a timer, or events happening when it's out of focus, gets slowed down by the browser to conserve resources. Some mobile browsers will stop it together.
The answer to the question is to change how your app keeps track of these stats.
Now some will say to use WebWorkers to run the timer in a separate thread but this won't solve all your issues. You'd still have a different version of the issue, like if someone restored your webpage from sleep or something along those lines. No background task can survive that.
You mention that you track these stats also on the server. That's convenient, so the most obvious thing you should do is detect when the tab comes back into focus using the Window focus event. You would then make all the calls to the server to fetch the most up-to-date stats and reset the timers based on that fresh data. To stop it from showing stale data while the request is in flight, you might choose to show a loading spinner or something during that period.
Another common way of fixing this is you keep around on each timer increment a var which says when the data last came back (a timestamp). When you leave focus, you detect this with the blur event and store that last timestamp somewhere. Then they come back into focus, you handle the focus event and calculate the difference between the current time and the last recorded time before defocus (blur). You may be able to recalculate from this period what the values should be.
But if your server has this info, it'd be far less error-prone and easy to just ask the server when they refocus.

How does HotJar generate their recordings?

Tracking mouse movement/scroll/click events is easy but how do they save the screen and keep it in sync so well?
The pages are rendered very quite well (at least for static HTML pages, haven't tested on Angular or any SPA), the sync is almost perfect.
To generate and upload a 23fps recording of my screen (1920x1080) it would take about 2Mbps of bandwidth. Maybe when recording only when there are some mouse events it would still take some 300-500Kbps on average? That seems way too much...
HTML content and DOM changes get pumped through a websocket and stored by Hotjar (minus sensitive information such as form inputs from the user, unless you've whitelisted them), the CSS isn't stored (it gets loaded by you when you watch the recording).
Because they're only recording user activity and DOM changes, there's a lot less data to record than if they were capturing a full video. The downside is that some Javascript driven widgets won't function correctly in the replay.
Relevant information from Hotjar docs:
When it comes to recordings, changes to the page are captured using the MutationObserver API which is built-in into every modern browser.
This makes it efficient since the change itself is already happening
on the page and the browser MutationObserver API allows us to record
this change which we then parse and also send through the websocket.
At regular short intervals, every 100ms or 10 times per second, the cursor position and scroll position are recorded. Clicks are recorded
when they happen, capturing the position of the cursor relative to the
element being clicked. These are functions which in no way hinder a
user's experience as they only capture the location of the pointer
when a click happens or every 100ms. The events are sent to the Hotjar
servers through frames within the websocket, which is more efficient
than sending XHR requests at regular intervals.
Source: https://help.hotjar.com/hc/en-us/articles/115009335727-Will-Hotjar-Slow-Down-My-Site-

Javascript - hide div 1 sec before live stream on video ends

I have this in page:
<video src="blob://some-live-stream" autoplay></video>
<div id="hideMePlease"> hide 1 sec before video ends</div>
I would like to hide the div 1 sec before the video ends, how can i do?
N.B: i can't know video duration, it's a live stream , and the video autostops so i have no way to stop it myself.
If, as you state, you cannot know the length of the video because it's streaming, it will be impossible (relativistic time travel notwithstanding) for you to schedule an event one second before it finishes.
However, I would at least first try to use the duration property of the video, it may be that metadata is made available as part of the stream early on. If it is, you can use that to schedule the hiding of your div.
As an aside, if you visit the page http://www.w3.org/2010/05/video/mediaevents.html, you'll find that the duration is set correctly as soon as you start playing the video, despite the fact it seems to be streaming from either an MP4, OGG or WEBM file). So it's at least possible, though it may depend on the data stream itself (whether the metadata comes before or after the actual video data).
If the data is not available (I think you get Inf for the duration in that case), then you're just going to have to hide the div at the earliest possible time after that.
That would presumably be when it finishes (fires the onended event).
So, in short, if you can get the duration (or periodically get the time remaining which might be better), use that. Otherwise fall back to hiding it at the end and start hassling w3c to provide the functionality you want in HTML6.

Weird bug on Videojs

I don't know how this happens and I can't see any errors.
I can't seem to navigate through the video the second time I open my page.
See screenshot here:
I have found this error it says,
TypeError: Floating-point value is not finite.
"Video is not ready. (Video.js)"
Help would be really appreciated.
Thanks
When you say "I can't seem to navigate through the video the second time I open my page"? Do you mean you aren't able to play the video at all or that you aren't able to fast-forward and rewind within the playing video?
That you are getting a Type Error: Floating-point value is not finite error means that a certain parameter you're supplying to video.js is of the wrong type. I.e. you probaby supply video.js with a string when it wants an integer (or something similar).
Because it works the first time you load the page, are you trying to resume playback where you left off when you navigated away from the page?
If that's the case you are probably storing the currentTime parameter in a cookie or localStorage value as a string (using jQuery cookies for example these usually get automaticalyl stringified) and forgetting to switch it back to an int when you need video.js to read it back to you. Because what I notice about your screenshot is it seems video.js doesn't know the duration of your video (i.e. the seek time says 0:31 / 0:00)
Here's what you should do:
Clear you cache to get a working first time player load then:
Before starting play back, after playback has finished, and during playback you should log the current playback time signature, i.e.: console.log( videojs("id-of-your-video").currentTime() )
Adding time signature console.logs() to these video.js event callbacks should help you:
durationchange (fired if the total duration of your video changes)
duration (tells you the duration of your video. Try logging this while it works and again after it stops working and see what's changed in the value)
If you're still having trouble try logging values using the video js timeupdate callback. This event is called when the current playback position has changed (can be several times a second). If all else fails this callback might give you some insight into the exact moment you're getting the invalid type value, but it won't help you if you're problems are with trying to resume playback from a .currentTime() value reading from an incorrect type stored in user cookie / sessionStorage / localStorage etc.
Are you Using a Server to execute it or are you running it locally. Because I got some similar issues when I ran it locally. But When I put the files in some server like tomcat or iis or whatever servers it worked. Not sure about the issue just a guess

Record and replay Javascript

I know it is possible to record mouse movements, scrolling and keystrokes. But what about changes to the document? How can I record changes to the document?
Here is my try out. There must be a better more simple way to store all events?
I am thankful for all tips I can get!
<!DOCTYPE html>
<html>
<head>
<title>Record And replay javascript</title>
</head>
<body id="if_no_other_id_exist">
<div style="height:100px;background:#0F0" id="test1">click me</div>
<div style="height:100px;background:#9F9" class="test2">click me</div>
<div style="height:100px;background:#3F9" id="test3">click me</div>
<div style="height:100px;background:#F96" id="test4">click me</div>
<script src="http://code.jquery.com/jquery-latest.min.js"></script>
<script>
$(document).ready(function() {
var the_time_document_is_redy = new Date().getTime();
var the_replay = '';
$('div').live("click", function (){
var the_length_of_visit = new Date().getTime() - the_time_document_is_redy;
// check if the element that is clicked has an id
if (this.id)
{
the_replay =
the_replay
+
"setTimeout(\"$('#"
+
this.id
+
"').trigger('click')\","
+
the_length_of_visit
+
");"
;
alert (
"The following javascript will be included in the file in the replay version:\n\n"
+
the_replay
) // end alert
} // end if
// if it does not have an id, check if the element that is clicked has an class
else if (this.className)
{
// find the closest id to better target the element (needed in my application)
var closest_div_with_id = $(this).closest('[id]').attr('id');
the_replay =
the_replay
+
"setTimeout(\"$('#"
+
closest_div_with_id
+
" ."
+
this.className
+
"').trigger('click')\","
+
the_length_of_visit
+
");"
;
alert (
"The following javascript will be included in the file in the replay version:\n\n"
+
the_replay
) // end alert
} // end if
});
// fall back if there are no other id's
$('body').attr('id','if_no_other_id_exist');
// example of how it will work in the replay version
setTimeout("$('#test1').trigger('click')",10000);
});
</script>
</body>
</html>
I became curious by this question and implemented a proof of concept here
https://codesandbox.io/s/jquery-playground-y46pv?fontsize=14&hidenavigation=1&theme=dark
Using the demo
Press record, click on some circles, type something in the input, press record again to stop the recording and finally click play.
You can tweak the size of the playback by editing the REPLAY_SCALE variable in the source code.
You can control the playback speed by changing the SPEED variable in the source code.
Note, I only tested this on Chrome.
Implementation details:
It monitors mousemove, click and typing events. It should be easily extensible to add others such as scroll, window resizing, hover, focus etc.
Playback creates an <iframe>, injects the original HTML and replays the user events.
The event listeners bypass any event.stopPropagation() by using capture when listening for events on the document.
Displaying playback in a different resolution is done using zoom CSS3.
A transparent canvas could be overlaid to draw the trace lines of the mouse. I use just a simple div so no trace lines.
Considerations:
Imagining we are capturing user events on a real website. Since the page served could change between now and the playback we can't rely on the client's server when replaying the recording in the iframe. Instead we have to snapshot the html, all ajax requests and resource requests made during the recording. In the demo I only snapshot the HTML for simplicity. However in practice, all extra requests would have to be stored on the server in realtime as they are downloaded on the client page. Furthermore, during playback it is important that the requests are played back with the same timeline that they were perceived by the user. To simulate the request timeline, the offset and duration of each request must also be stored. Uploading all page requests as they are downloaded on the client will slow down the client page. One way to optimize this uploading could be to hash the contents of the request before they are uploaded, if the hash is already present on the server, the request data need not be reuploaded. Furthermore, the session of one user can leverage the request data uploaded by another user using this hashing method. Finally, the browser itself need not do the uploading, provided all requests are going through a central server, this snapshotting can happen server side so as not to impact the user's experience.
Careful consideration will be needed when uploading all the user events. Since lots of events will be generated, this means lots of data. Perhaps some compression of the events could be made e.g. losing some of the less important mousemove events. An upload request should not be made per event to minimize number of requests. The events should be buffered until a buffer size or timeout is reached before each batch of events is uploaded. A timeout should be used as the user could close the page at any point thus losing some events.
During playback outgoing POST requests should be mocked to prevent duplicating events elsewhere.
During playback the user agent should be spoofed but this may be unreliable in rendering the original display.
The custom recording code could conflict with client code. e.g. jquery. Namespacing will be required to avoid this.
There might be some edge cases where typing and clicking may not reproduce the same resulting HTML as seen in the original e.g. random numbers, date times. Mutation observers may be required to observe HTML changes, although not supported in all browsers. Screenshots could come in useful here but might prove OTT.
Replaying user actions with just Javascript is a complex problem.
First of all, you can't move mouse cursor, you can't emulate mouseover/hovers also. So there goes away a big part of user interactions with a page.
Second of all, actions, once recorded, for most of the time they have to be replayed in different environment than they were recorded in the first place. I mean you can replay the actions on screen with smaller resolutions, different client browser, different content served based on replaying browser cookies etc.
If you take a time to study available services that enable you to record website visitors actions ( http://clicktale.com, http://userfly.com/ to name a few), you'll see that none of them are capable of fully replaying users actions, especially when it comes to mouseovers, ajax, complex JS widgets.
As to your question for detecting changes made to the DOM - as Chris Biscardi stated in his answer, there are mutation events, that are designed for that. However, keep in mind, that they are not implemented in every browser. Namely, the IE doesn't support them (they will be supported as of IE 9, according to this blog entry on msdn http://blogs.msdn.com/b/ie/archive/2010/03/26/dom-level-3-events-support-in-ie9.aspx).
Relying on those events may be suitable for you, depending on use case.
As to "better more simple way to store all events". There are other ways (syntax wise), of listening for events of your choice, however handling (= storing) them can't be handled in simple way, unless you want to serialize whole event objects which wouldn't be a good idea, if you were to send information about those event to some server to store them. You have to be aware of the fact, that there are massive amount of events popping around while using website, hence massive amount of potential data to be stored/send to the server.
I hope I made myself clear and you find some of those information useful. I myself have been involved in project that aimed to do what you're trying to achive, so I know how complicated can it get once you start digging into the subject.
I believe you are looking for Mutation Events.
http://www.w3.org/TR/2000/REC-DOM-Level-2-Events-20001113/events.html#Events-eventgroupings-mutationevents
Here are some resources for you:
http://tobiasz123.wordpress.com/2009/01/19/utilizing-mutation-events-for-automatic-and-persistent-event-attaching/
http://forum.jquery.com/topic/mutation-events-12-1-2010
https://github.com/jollytoad/jquery.mutation-events
Update:
In Response to comment, a very, very basic implementation:
//callback function
function onNodeInserted(){alert('inserted')}
//add listener to dom(in this case the body tag)
document.body.addEventListener ('DOMNodeInserted', onNodeInserted, false);
//Add element to dom
$('<div>test</div>').appendTo('body')
Like WTK said, you are getting yourself into complex territory.
Record
Save the initial DOM of the page, remove the scripts from it and also you need to change all relative URLs to absolute ones.
Then, record DOM mutations and Keyboard/Mouse event.
Replay
Start with initial saved DOM, apply mutations and events using timestamp order.
In fact, clicks will not do anything because we have removed any scripts. but because we have saved the DOM changes we can replay the effect after the click.
I found these two solutions on github which allows your to capture the events and then replay that on a remote server.
https://github.com/ElliotNB/js-replay
and a more comprehensive solution
https://github.com/rrweb-io/rrweb
https://www.rrweb.io/#demos
Both has demos which you can try.
Lately, we can now use MutationObserver
MutationObserver provides developers with a way to react to changes in
a DOM. It is designed as a replacement for Mutation Events defined in
the DOM3 Events specification.
Slow demo, because the console.log message is huge.
var mutationObserver = new MutationObserver(function(mutations) {
mutations.forEach(function(mutation) {
console.log(mutation)
})
})
mutationObserver.observe(watchme, {
attributes: true,
characterData: true,
childList: true,
subtree: true,
attributeOldValue: true,
characterDataOldValue: true
})
<div id="watchme" contenteditable>
Hello world!
</div>

Categories

Resources