I have a camera feed coming into a linux machine using a V4l2 interface as the source for a gstreamer pipeline. I'm building an interface to control the camera, and I would like to do so in HTML/javascript, communicating to a local server. The problem is getting a feed from the gst pipeline into the browser. The options for doing so seem to be:
A loopback from gst to a v4l2 device, which is displayed using flash's webcam support
Outputting a MJPEG stream which is displayed in the browser
Outputting a RTSP stream which is displayed by flash
Writing a browser plugin
Overlaying a native X application over the browser
Has anyone had experience solving this problem before? The most important requirement is that the feed be as close to real time as possible. I would like to avoid flash if possible, though it may not be. Any help would be greatly appreciated.
You already thought about multiple solutions. You could also stream in ogg/vorbis/theora or vp8 to an icecast server, see the OLPC GStreamer wiki for examples.
Since you are looking for a python solution as well (according to your tags), have you considered using Flumotion? It's a streaming server written on top of GStreamer with Twisted, and you could integrate it with your own solution. It can stream over HTTP, so you don't need an icecast server.
Depending on the codecs, there are various tweaks to allow low-latency. Typically, with Flumotion, locally, you could get a few seconds latency, and that can be lowered I believe (x264enc can be tweaked to reach less than a second latency, iirc). Typically, you have to reduce the keyframe distance, and also limit the motion-vector estimation to a few nearby frames: that will probably reduce the quality and raise the bitrate though.
What browsers are you targeting? If you ignore Internet Explorer, you should be able to stream OGG/Theora video and/or WebM video direct to the browser using the tag. If you need to support IE as well though you're probably reduced to a flash applet. I just set up a web stream using Flumotion and the free version of Flowplayer http://flowplayer.org/ and it's working very well. Flowplayer has a lot of advanced functionality that I have barely even begun to explore.
Related
I have a web program which makes use of Web Audio API. The issue here is that i want to make it compatible for IE. Is there any alternate for the Web Audio API, so that i can make the same code run on IE specifically?
What are your needs? If you need to do dynamic synthesis, audio routing, etc, you will only be able to achieve that with the Web Audio API, so your IE users are out of luck.
However, if all you need to do is play audio files, then I would recommend that you use howler.js. Howler has great compatibility across different browsers and operating systems, including various versions of IE.
Microsoft have been working on implementing the web audio API, it looks like a preview implementation is available. https://status.modern.ie/webaudioapi If you can wait until the next version of IE which I believe is expected to be released with Windows 10, which should be out this year IIRC, it may not be worth taking the time to implement an alternative.
That said, to answer your question, there's no way that I could find to generate audio with an oscillator in the web without the Web Audio API other than writing flash code, which has its own major disadvantages.
Minor note: you could possibly hack the html5 audio node to play back audio buffers that you generate in javascript code, but keeping everything in sync and preventing jumps in the waveform would be an awful task.
What I am doing is some kind of a video conference tool. I am doing some research on video processing recently and it seems straight forward by using the video element in combination with canvases. But, I am using WebRTC for video streaming to all connected clients, for which I need a MediaStream. So I am looking for a way to retrieve a MediaStream from the canvas element.
On my way, I found this project here called Whammy http://antimatter15.com/wp/2012/08/whammy-a-real-time-javascript-webm-encoder/ which creates a video file from a canvas, but as far as I understand, it is not made for live-streaming it.
One alternative approach would be to do the video processing on the remote client by sending him the stream and the information on how to process it. However this might work pretty well for few clients, but when it comes to multi-user conferences, I feel like it doesn't scale, since real-time video processing is still a calculation intense job. Every client would have to process all video streams from all connected clients.
For me it looks like a one-way-street. Getting video content into a canvas is pretty easy, the other way around is pretty hard. I thought, there might be a library for creating a MediaStream from a canvas element, but I found nothing. Any ideas on how to achieve that?
Best,
Felix
Since this question was posted, there has been not much, but a little progress on this front.
The MediaStream Recording standard allows for recording streams (such as WebRTC) into video file formats, using the MediaRecorder API.
It should work with recent Firefox as well as Chrome (video only) enabling the latter via the experimental flags ( chrome://flags -> Experimental Web Platform features).
Also see this resource for examples.
I'm working on HTML5 app that lets several users to work on one document. I need to add a possibility for users (editing the same document) to talk to each other. And I just don't know how to start with that. Here are my questions
Is there an HTML5 lib allowing to transfer sound from microphone between clients?
What about streaming video from camera?
What is an easiest server-side solution for that?
Any thoughts are strongly appreciated! So don't be shy! :)
UPD: please note that I need an abbility for more then two users to talk.
For this you can use WebRTC.
However, this is a very young and unfinished technology that as already stated is currently available only in Chrome stable and Firefox beta. This means there will probably come changes to the current spec, something to be aware of in case of early implementation. But it allow you to use video and audio communication directly in the browser.
Quick-start here:
http://www.html5rocks.com/en/tutorials/webrtc/basics/
Other options are Flash based plugins such as flash-videoio. This is an open source plugin but will naturally require Adobe Flash installed. This may or may not be a problem depending on the company's security policy.
For technical details on implementation please see examples on the provided links.
For many-to-many you can use either:
"Mesh" - everybody connects to everybody. This however is costly on CPU and mobiles are often left out.
"Star" - everybody goes through the most capable device. However, with many connections this will soon run slow for the device handling all connections.
MCU. Specialized server to handle all connections. If mixes audio and video and handles drop-outs as well without affecting the other callers.
Examples of MCU's:
http://sourceforge.net/projects/mcumediaserver/ (open source)
http://www.medooze.com/products/mcu.aspx (commercial)
you are searching for navigator.getUserMedia()
that allows the various users to share video audio and data.
the support is very low... only chrome and the latest verions of opera and firefox support it.
and totally no support on mobile devices... maybe in the next android chrome... dunno
as there is much to talk about and i have no clue on how u wanna setup everything i suggest u read a little more about that on the urls...
http://caniuse.com/stream
http://www.html5rocks.com/en/tutorials/getusermedia/intro/
http://dev.w3.org/2011/webrtc/editor/getusermedia.html
https://developer.mozilla.org/en-US/docs/WebRTC/navigator.getUserMedia
http://my.opera.com/core/blog/2011/03/23/webcam-orientation-preview
http://simpl.info/getusermedia/
and SERVERSIDE solution nahh... thats not a good solution
clientside is the way to go.
Not sure if you're required to do it yourself from scratch or are able to use third party libraries/tools.
In which case I would recommend using Tokbox which has support for WebRTC and SDK for iOS.
Their API is simple and easy to use.
I understand that the best bet when streaming a video as a source in an HTML5 tag would be .mp4. But let's say that I have a source that only outputs fragmented to an ism/manifest.
Is there any way at all, whether through other libraries or messy hacks, that I can bring this video into something rendered as a tag onscreen? The closest I have found is Walkthrough: Building Your First HTML5 Smooth Streaming Player because it allows this to be done - but I neither have Windows 8, or want to have this running a server capable of .NET. I was hoping there was something, messy or not, that I could achieve this with entirely within javascript and executable locally without a deploy.
Thanks
Firstly, W3C does not provide a standard for adaptive bitrate streaming, yet. So for the time being most browsers only support simple progressive download playback.
Hence, there is no JS implementation of a Smooth Streaming player and Microsoft is not working on one, as far as I know.
The example you provide uses the "Microsoft Smooth Streaming Client SDK Beta 2 for Windows 8" which is a C++ library and is only available for Windows Store Apps development. It has nothing to do with browsers.
So, unfortunately this is not yet possible. Even more, I doubt that this will ever happen, because everybody is waiting for MPEG DASH to be finalized.
UPDATE.
Please, notice that you always can use Siverlight application for playing SmoothStreaming. The referenced HTML5 Player framework is capable of falling back to Silverlight.
no luck for Microsoft Smooth Streaming, but regarding MPEG-DASH which is similar (see http://blog.johndeutscher.com/2013/06/10/mpeg-dash-preview-from-windows-azure-media-services):
"Dash.js is permissively licensed (under the BSD license) and can therefore be studied and reused by anyone seeking to provide their own DASH-AVC/264 compliant player. The goal is to make it easier for third-parties to build adaptive streaming video players."
http://msopentech.com/blog/2013/06/20/ms-open-tech-contributes-to-open-source-adaptive-streaming-video-player/
also see:
https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html
which is required for Dash.js
I am using a embeddable Flash based MP3 player (The standalone Version of Wordpress Audio Player, to be exact) to play audio on the web page of a client. Given the nature of Flash based players - AFAIK they all use Flash's audio/video decoding libraries - the exact player product might not matter, though.
She complains that some specific audio files sometimes tend to play slowly, as if on a tape deck that doesn't run at full speed. Her machine is a 5 year old Sony Vaio with a 1.5 GhZ Pentium M Processor, so it's not top of the line anymore but it should not have trouble decoding anything, let alone a MP3 audio stream.
I can't verify the problem because we can't set up an audio connection (slow Internet on a remote island), nor can I reproduce it on my 3.3 GhZ dual-code Workstation, but this is a person whose error reports have seldom turned out unfounded in the past, so I tend to take them seriously.
I see differences between the encodings of the MP3 files (see below).
My question: Has anybody ever experienced something like this?
If yes, what was the reason?
If it's something in the encoding (which I suspect), could somebody give me pointers towards which setting(s) to change?
This is what Winamp tells me about a file that is sometimes slow:
MPEG-1 Layer 3
128 Kbit, approx. 966 frames
48000Hz Stereo
CRC: Yes, Copyright: No
This is what Winamp tells me about a file that is never slow:
Encoder delay: 576, Zero Padding: 1584
MPEG-1 Layer 3
151 Kbit (VBR), 12405 Frames
44100Hz Joint Stereo
CRC: No, Copyright: No
I think this is a bug with certain versions of the Flash player. I have a Flash-based MP3 audio player that I made with the same problem. The solution is to make sure it is a 44,100 kHz MP3 file... or make your users upgrade their player.
Here is my site with the details:
http://web.archive.org/web/20111227184112/http://www.quickmediasolutions.com/sandbox/audio_player/
This is likely because the sample rate info in the header file does not match the sample rate the file was encoded with. Try fixing the problematic file with one of the MP# fixer utilities: google search.
It is also possible that this is a driver issue on their pc, and reinstalling/updating their audio driver may solve the problem.