The video in the top is the stream from navigator.mediaDevices.getUserMedia() which is set as the source of a the first video element. The bottom video is the video from MediaRecorder API. The video is played from a Virtual Web Camera (OBS VirtualCam specifically)
The problem I'm having with this
When the video starts to stream for the web camera (once allowed) the media seems to be pushed to the MediaSource but the playback took a while to actually play (about 2-3 seconds). What kind of behaviour is this?
Another problem is the playback in the bottom video is delayed, I have been testing and I am finding that the problem is because of the mediaRecorder.ondataavailable is not triggered fast enough, thus the chucks are delayed. In what way possible that the MediaRecorder API or similar API can be used to record in much lower latency than this.
UPDATE:
What is happening here? 1. Video from camera is play directly in the video element, that element video stream is recorded through MediaRecorder API, recorded video is sent to server (websocket). The video (bottom) is playing the video from websocket. My tests shows that the latency is not from the Websocket but from the either the MediaRecorder or the playback is delayed.
Related
I have a simple swiper.js slideshow with a video in one slide. When the slide is finished, the video resets and resumes when it returns. The video has a size of 54.8MB.
I noticed that the resource consumption was very high and I saw that there were regular requests to the mp4 in the network tab. With each request, the size of the response increases.
I don't understand why this happens, has anyone ever encountered this problem or has an explanation ?
I have a MP4 video of a product which shows a 360 of the product (so it looks like it's spinning) - I have it autoplaying on the page and looks great.
I want the user to be able to click and drag on the video to basically rewind or fastforward the video, so they can kind of scroll back and forth and spin the product around as they wish.
We have used the js-cloudimage-360-view library and used images, but because we wanted this as smooth as possible each colour had 360 images and loading times weren't great, we have MP4s which are small in size and load fast but it's just the srubbing I can't get right.
I'd had a look at this https://codepen.io/webandapp/pen/xEjjOJ too but it's supper jumpy, essentially we want if the user drags to the right it fast forwards as they drag, when they stop it should play from that point (and rewinds if dragging to the left) - this should be as smooth as possible.
The video is just a HTML video element like so:
<video id="three-sixty-video" preload="auto" controls loop playsinline autoplay muted>
<source src="/video/video-1.mp4" type="video/mp4;">
</video>
Scrubbing on a streamed video is limited by the speed at which the video can be downloaded, decoded and displayed.
For a large video on most devices and connections it is not practical/possible to do all this as quickly as a user can scroll along the video timeline.
This is why video containers (mp4 etc) often include a track of thumbnail images which are displayed when the user scrubs along the line.
In other words, it is not actually the video you see in the small image in the timeline, but the appropriate thumbnail for that time in the video.
When the user stop scrubbing, the player then requests that section of the video, decodes and displays it. This is (generally) not immediate.
If your video is small enough and is downloaded completely then you may find you can scrub as quickly as you want, but even then it may be device/client dependent - a quick local test with a number of videos seem to suggest it should scrub ok when available locally:
Using a relatively small 30MB, 50 second local mp4 video, available locally on he computer hard drive (a MAC), scrubbing is relatively smooth using the Quicktime client player.
The same video on the same device being viewed by the Safari browser rather than directly by the QuickTime client appears to be similar, although it is hard to be sure. The playback may well use very similar paths under the skin.
Using a much larger 2G, 4K, 2 minute 30 second video, also locally on the hard drive, scrubbing is again relatively smooth both in Safari and QuickTime.
As a side note, if you do decide the solution is to have the video fully downloaded before you play, that is also an interesting challenge. Most effort is usually focused on the opposite, allowing a video to start playing before it fully downloads. You may find that, providing the video is not too big, having the header info at the end (the MooV atom for mp4) will force the browser to download it fully. This is the opposite of the mp4 faststart you will see discussed some places. I'm not sure how reliable this will be as I think some browser may be clever enough to 'search' in the video by making different requests for bytes at the start and end to find the metadata.
I am working on a WebRTC video streaming between a moving drone and a remote device. Sometimes, the video freezes. I watched some webrtc's and video's statistics and noticed that during freezes, some frames were received, but no frame was displayed on the video.
To explain, I compared:
About of the video element, the difference of webkitdecodedframecount in the last second
About the Inbound Stream (using RTCInboundRtpStreamStats) the difference of framesReceived in the last second
During a ~4s freeze where I was recording my screen, 0 frames were decoded but about 16 frames were received.
So my questions are:
Why is there such a difference between received frames and displayed frames ?
How to display received frames instantly ?
Edit:
At the beginning of the freeze, 4 packets have been lost.
At the end of the freeze, the stream worked well again, with a lower resolution
I'm developing a browser based application/tool in Javascript that runs WebGL to animate images loaded by the user. The animation stuff is working good. Now I want the user to be able to save/export the animation. I'm currently using the HTML MediaRecorder API to record the browser screen on the fly/in real time, generating a WEBM file that saves to their PC.
Overall this works well enough. However if a very large image is being animated, ie 2048x2048 pixels, when the recording process starts the browser FPS drops to about 12. Obviously then the recorded animation looks crap. Now what I can do is detect the FPS, and if it drops below say 24fps I can slow down the animation accordingly. eg if fps drops to 12, I set the animation to half speed while recording. Afterwards I can then run the output video in VLC player, increase playback speed to 200%, and the video looks smooth.
So, my question... Can I somehow set the encoding information of the WEBM file so that it will play smoothly in the player? Tell the player what the FPS is? (ie 12 in the above example). Or tell it what the playback speed is?
How do I know what the default recording FPS is with HTML MediaRecorder? I've tried using the FFMPEG tools on the output files but not really getting much useful information from it (possibly user error here).
(Video encoding noob here, so I'm probably screwing up terminology, but hopefully my problem/intention is explained clearly enough)
EDIT: Is it possible to get the MediaRecorder API to record in step with the animation frame? It seems like it runs independent of the actual browser FPS.
I am developing a time-lapse video player using HTML5 Video element and is controlled by mouse (touch) events. The problem is that the videos which will be played are encoded at a different frame rate, not fixed like common 24 FPS. In native platforms such as iOS, there is a native API which provides video's frame rate. I am wondering if is there any similar API on HTML5 Media API. Without this information, I cannot play a video smoothly on my player.
Mozzila Developer Network does not list any methods for finding the framerate of a media element. However, there is an experimental method called seekToNextFrame which could be used to advance frame by frame. This is only supported by firefox behind a flag.