I'm developing a browser based application/tool in Javascript that runs WebGL to animate images loaded by the user. The animation stuff is working good. Now I want the user to be able to save/export the animation. I'm currently using the HTML MediaRecorder API to record the browser screen on the fly/in real time, generating a WEBM file that saves to their PC.
Overall this works well enough. However if a very large image is being animated, ie 2048x2048 pixels, when the recording process starts the browser FPS drops to about 12. Obviously then the recorded animation looks crap. Now what I can do is detect the FPS, and if it drops below say 24fps I can slow down the animation accordingly. eg if fps drops to 12, I set the animation to half speed while recording. Afterwards I can then run the output video in VLC player, increase playback speed to 200%, and the video looks smooth.
So, my question... Can I somehow set the encoding information of the WEBM file so that it will play smoothly in the player? Tell the player what the FPS is? (ie 12 in the above example). Or tell it what the playback speed is?
How do I know what the default recording FPS is with HTML MediaRecorder? I've tried using the FFMPEG tools on the output files but not really getting much useful information from it (possibly user error here).
(Video encoding noob here, so I'm probably screwing up terminology, but hopefully my problem/intention is explained clearly enough)
EDIT: Is it possible to get the MediaRecorder API to record in step with the animation frame? It seems like it runs independent of the actual browser FPS.
Related
I have a MP4 video of a product which shows a 360 of the product (so it looks like it's spinning) - I have it autoplaying on the page and looks great.
I want the user to be able to click and drag on the video to basically rewind or fastforward the video, so they can kind of scroll back and forth and spin the product around as they wish.
We have used the js-cloudimage-360-view library and used images, but because we wanted this as smooth as possible each colour had 360 images and loading times weren't great, we have MP4s which are small in size and load fast but it's just the srubbing I can't get right.
I'd had a look at this https://codepen.io/webandapp/pen/xEjjOJ too but it's supper jumpy, essentially we want if the user drags to the right it fast forwards as they drag, when they stop it should play from that point (and rewinds if dragging to the left) - this should be as smooth as possible.
The video is just a HTML video element like so:
<video id="three-sixty-video" preload="auto" controls loop playsinline autoplay muted>
<source src="/video/video-1.mp4" type="video/mp4;">
</video>
Scrubbing on a streamed video is limited by the speed at which the video can be downloaded, decoded and displayed.
For a large video on most devices and connections it is not practical/possible to do all this as quickly as a user can scroll along the video timeline.
This is why video containers (mp4 etc) often include a track of thumbnail images which are displayed when the user scrubs along the line.
In other words, it is not actually the video you see in the small image in the timeline, but the appropriate thumbnail for that time in the video.
When the user stop scrubbing, the player then requests that section of the video, decodes and displays it. This is (generally) not immediate.
If your video is small enough and is downloaded completely then you may find you can scrub as quickly as you want, but even then it may be device/client dependent - a quick local test with a number of videos seem to suggest it should scrub ok when available locally:
Using a relatively small 30MB, 50 second local mp4 video, available locally on he computer hard drive (a MAC), scrubbing is relatively smooth using the Quicktime client player.
The same video on the same device being viewed by the Safari browser rather than directly by the QuickTime client appears to be similar, although it is hard to be sure. The playback may well use very similar paths under the skin.
Using a much larger 2G, 4K, 2 minute 30 second video, also locally on the hard drive, scrubbing is again relatively smooth both in Safari and QuickTime.
As a side note, if you do decide the solution is to have the video fully downloaded before you play, that is also an interesting challenge. Most effort is usually focused on the opposite, allowing a video to start playing before it fully downloads. You may find that, providing the video is not too big, having the header info at the end (the MooV atom for mp4) will force the browser to download it fully. This is the opposite of the mp4 faststart you will see discussed some places. I'm not sure how reliable this will be as I think some browser may be clever enough to 'search' in the video by making different requests for bytes at the start and end to find the metadata.
So i'm trying to stream video in segments without using the MediaSource extension. (Because not all browsers support MSE). Now i'm trying to do this by loading two video elements and play the next one at the right moment. But this has a very tiny delay between switching. I tried to keep checking the currenTime of the video and after a tiny fraction play the next video element. But this doesn't really work that well (audio overlaps, or delay)
Mind you that the video are preloaded and loaded from Blob storage. So the loading shouldn't delay the playback.
How can i make this (Or another solution without flash) play smoothly without using MediaSource extensions?
Your best bet is to use two video elements positioned one on top of the other. The first one should be playing the current part (or chunk) of the video. The second video element should be loaded with the blob that contains the next chunk and be paused. It should also be hidden (you can set display:'none' or z-index:-99999). And then when the first video element ends (the end event is dispatched), call the play() method of the second video element, show it, and hide the first one. Rinse and repeat.
This is what the LifemirrorPlayer does.
If the chunks of the stream are perfectly encoded and cut then this technique works. Often it doesn't. Common problems are:
Synchronizing the audio stream is the probably the hardest. After twenty or thirty chunks are played, the audio usually loses sync with the video. This is very annoying for the viewer and hard to detect and fix (i don't know of any solution actually).
The video element doesn't end. This is usually caused by an encoder that has put too much (or too little) content into a chunk. It can also be caused by improperly cut chunks, by a buggy encoder or decoder.
Flickering when the two video elements are swapped. This depends on the browser. However most browsers deal very well with this and the swap is quick and smooth.
I have a <video> element at the top of my page playing an animation in the background (very similar to the one found on the Google Hangouts page). On most computers, the video plays smoothly and it seems fine, but on older hardware there is a lot of lag and the browser slows down quite a lot.
Is there anyway to detect the performance of the video playback? I'd like to either replace the video with a static image or pause the video if performance is an issue.
Use the video metrics if available in your browser. Here's an example using webkit prefixed DroppedFrameCount.
http://git.chromium.org/gitweb/?p=chromium.git;a=blob;f=chrome/test/data/media/html/media_stat_perf.html
See also: http://wiki.whatwg.org/wiki/Video_Metrics for the Moz prefix.
You could use Get the timestamp of loaded page to determine how fast the user loads the page, then if their bandwidth is below a specified threshold, supply an image rather than the variable.
Is there any way to emulate this kind of 360 degree product view scroll/drag functionality with video, so taking a piece of video and being able to scroll it left and right as though it was a set of images? I've found this tutorial which is about the 360langstrasse.sf.tv/page/ site which is pretty interesting, but it's pretty custom to their specific needs and pretty complex. Is there any way to just take some video and have the kind of left/right scroll/drag functionality that you would get with an image-based 360 degree script? I would just like a little video to load in a placeholder, and then you can drag it left and right.
I've thought about converting frames to jpegs and then stitching them into a panorama, but that gets very big very quickly. Would prefer a video solution, where one can just drag the video itself.
You could shoot the video of the object spinning. Then play it forwards for spinning one way and reverse for spinning another.
Using 2 videos could also work - one for spinning one way and the other video the opposite way.
This is possible in Flash but I'm not sure about HTML5 video across browser vendors.
Is it possible to play HTML5 video in reverse?
I have an html5 video app which consists of video + JS swipable playlist + other JS animated overlays.
In iOS, onload performance of the playlist swiping and overlay animation is fine. Once a video has played, even if it is stopped, the playlist swiping and overlay animation is very very slow. Some users think it is not responding at all.
In Chrome/desktop, everything is fine.
If I log status information, the logs look the same regardless of whether a video has played or not. I don't know where to begin to debug. It seems to be a memory issue (?), but not sure how or if it can be addressed.
Anyone have any thoughts?
Javascript in iOS (both safari and UIWebView) execution can be stopped (or killed could be more appropriate term) without throwing any errors if under heavy CPU load to save more power for background native tasks or special tasks like handling video - rendering video even if hardware (GPU) supported still requires more CPU power thus it gets higher priority then javascript and can lead to the javascript acting not as intended or expected - it can work sluggish or not at all in extreme cases.
The only thing I can come with is pausing the video on any user input in iOS - maybe it would help.
regards,
Tom