I'm developing a web application that captures video from a webcam and saves the stream to Amazon Kinesis.
The first approach I came up with is getUserMedia / mediaRecorder / XMLHttpRequest which posts chunked MKV to my unix server (not AWS), where simple PHP backend proxies that traffic to Kinesis with putMedia.
This should work, but all media streams from user will go through my server which could become a bottleneck. As far as I know, it's not possible to post chunked mkv to Amazon directly from browser due to cross-origin problems. Correct me if I'm wrong or there's a solution for this.
Another thing that I feel I'm missing - is WebRTC. XHR feels a little bit like a legacy in 2019 for streaming media. But if I want this to work, I will need a stack of three servers: webrtc server to establish connection, webrtc->rtsp proxy, and Kinesis gstreamer plugin, which grabs rtsp stream and pushes it to Kinesis. It looks a bit overcomplicated, and media traffic still runs through my server. Or maybe there is a better approach?
I need a suggestion on how to make better architecture for my app. I feel the best solution would be direct webrtc connection with some amazon service, which proxies stream to kinesis. Is it possible?
Thanks!
I was looking into this also for general education/research purpose. The closest example is featured on AWS blog.
And this is github repo. From the README.md
If the source is a sequence of buffered webcam frames, the browser client posts frame data to an API Gateway - Lambda Proxy endpoint, triggering the lambda/WebApi/frame-converter function. This function uses FFmpeg to construct a short MKV fragment out of the image frame sequence. For details on how this API request is executed, see the function-specific documentation.
Related
I'm trying to make a website that will serve as a VoIP recorder app. It will take audio from the microphone, transmit the audio to the server and the server only, and then the server will handle the redistribution of audio to it's connected clients.
Here's what I've tried already:
WebRTC (from what I can tell, it's peer-to-peer only)
MediaRecorder - timeSlice to Socket.IO (only the first packet is playable due to header information)
MediaRecorder - Stopping every few milliseconds, transmitting the audio, and starting again. (is extremely choppy)
The stack I'm set on is NodeJS with Express, but I'm extremely open to any packages that will help.
As far as possibility, I know it is possible because Discord wrote in their own blog that they explicitly do not send packets peer-to-peer because they have large numbers of connected users.
Below is the way I imagine it being setup:
Anyways, hope someone can help - I've been stuck on this for a while. Thanks!
WebRTC is NOT only P2P. You can put a WebRTC Peer on a server (and then have it do fan-out). This is what all major conferencing solutions do. SFU is a very popular deployment style, mesh isn't the only thing you can do.
You can go down the MediaRecorder path, but you are going to hit issues with congestion control/backpressure.
Hi I am wondering how in javascript or reactjs would I read data from a streaming station?
I have googled sadly I have had no luck and I was wondering if anyone knows of a script that can read (icecast ICY metadata?)
Please note that web browsers don't support ICY metadata, so you'd have to implement quite a few things manually and consume the whole stream just for the metadata. I do NOT recommend this.
As you indicate Icecast, the recommended way to get metadata is by querying the JSON endpoint: /status-json.xsl. It's documented.
It sounds like you are custom building for a certain server, so this should be a good approach. Note that you must be running a recent Icecast version (at the very least 2.4.1, but for security reasons better latest).
If you are wondering about accessing random Icecast servers where you have no control over, it becomes complicated: https://stackoverflow.com/a/57353140/2648865
If you want to play a stream and then display it's ICY metadata, look at miknik's answer. (It applies to legacy ICY streams, won't work with WebM or Ogg encapsulated Opus, Vorbis, etc)
I wrote a script that does exactly this.
It implements a service worker and uses the Fetch API and the Readable Streams API to intercept network requests from your page to your streaming server, add the necessary header to the request to initiate in-stream metadata from your streaming server and then extract the metadata from the response while playing the mp3 via the audio element on your page.
Due to restrictions on service workers and the Fetch API my script will only work if your site is served over SSL and your streaming server and website are on the same domain.
You can find the code on Github and a very basic demo of it in action here (open the console window to view the data being passed from the service worker)
I don't know much about stream's but I've found some stuff googling lol
https://www.npmjs.com/package/icy-metadata
https://living-sun.com/es/audio/85978-how-do-i-obtain-shoutcast-ldquonow-playingrdquo-metadata-from-the-stream-audio-stream-metadata-shoutcast-internet-radio.html
also this
Developing the client for the icecast server
its for php but maybe you can translate it to JS.
I am a beginner and I am trying to set up a peer-assisted media streaming system, that will work over the web browser. I wish to a server to 'push' media segments to a few clients and then any of these client browsers to push media segment to other client browsers.
I got to know that HTTP/2.0 can make this possible, but I found examples on a server to a client browser.
I came across WebRTC technology. however, could not find anything like PUSH technique among client browser.
I came across WebSocket technology. I found that it does PUSHing from the only server to the client.
Kindly direct.
You should use WebRTC + Socket or any other signaling media system to do it.
WebRTC will send the media peer to peer and the signaling server should manage which connections to make and when. I've seen similar product on the web yet.
WebSocket is not fast enought to stream media. WebRTC is far better when talking about media.
I'm looking at a way to implement video encoder using web browser. Youtube and Facebook already allow you to go live directly from the web browser. I'm wondering how do they do that?
There are a couple of solutions I've researched:
Using web socket: using web browser to encode the video (using mediarecorder api) and push the encoded video to the server to be broadcast.
Using WebRTC: web browser as a WebRTC peer and another server as the other end to receive the stream and re-broadcast (transcode) using other means (rtmp, hls).
Is there any other tech to implement this that those guys (YouTube, Facebook) are using? Or they also use one of these things?
Thanks
WebRTCHacks has a "how does youtube use webrtc" post here which examines some of the technical details of their implementation.
In addition one of their engineers gave a Talk at WebRTC Boston describing the system which is available on Youtube
Correct, you've hit on two ways to do this. (Note that for the MediaRecorder method, you can use any other method to get the data to the server. Web Sockets is one way... so is a regular HTTP PUT of segments. Or, you could even use a data channel of a WebRTC connection to the server.)
Pretty much everyone uses the WebRTC method, as there are some nice built-in benefits:
Low latency (at the cost of some quality)
Dynamic bitrate
Well-optimized on the client
Able to automatically scale output if there are not enough system resources to continue encoding at a higher frame size
The downsides of the WebRTC method:
Ridiculously complicated stack to maintain server-side.
Lower quality (due to emphasis on low latency, BUT you can tweak this by fiddling with the SDP yourself)
If you go the WebRTC route, consider gstreamer. If you want to go the Web Socket route, I've written a proxy to receive the data and send it off to FFmpeg to be copied over to RTMP. You can find it here: https://github.com/fbsamples/Canvas-Streaming-Example
I am capturing audio data using getUserMedia() and I want to send it to my server so I can save it as a Blob in a MySQL field.
This is all I am trying to do. I have made several attempts to do this using WebRTC, but I don't even know at this point if this is right or even the best way to do this.
Can anybody help me?
Here is the code I am using to capture audio from the microphone:
navigator.getUserMedia({
video:false,
audio:true,
},function(mediaStream){
// output mediaStream to speakers:
var mediaStreamSource=audioContext.createMediaStreamSource(mediaStream);
mediaStreamSource.connect(audioContext.destintion);
// send mediaStream to server:
// WebRTC code? not sure about this...
var RTCconfig={};
var conn=new RTCPeerConnection(RTCconfig);
// ???
},function(error){
console.log('getUserMedia() fail.');
console.log(error);
});
How can I send this mediaStream up to the server?
After Googling around I've been looking into WebRTC, but this seems to be for just peer to peer communication - actually, now I'm looking into this more, I think this is the way to go. It seems to be the way to communicate from the client's browser up to the host webserver, but nothing I try even comes close to working.
I've been going through the W3C documentation (which I am finding way too abstract), and I've been going thru this article on HTML5 Rocks (which is bringing up more questions than answers). Apparently I need a signalling method, can anyone advise which signalling method is best for sending mediaStreams, XHR, XMPP, SIP, Socket.io or something else?
What will I need on the server to support the receiving of WebRTC? My web server is running a basic LAMP stack.
Also, is it best to wait until the mediaStream is finished recording before I send it up to the server, or is it better to send the mediaStream as its being recorded? I want to know if I am going about doing this the right way. I have written file uploaders in javascript and HTML5, but uploading one of these mediaStreams seems hellishly more complicated and I'm not sure if I am approaching it right.
Any help on this would be greatly appreciated.
You cannot upload the live stream itself while it is running. This is because it is a LIVE stream.
So, this leaves you with a handful options.
Record the audio stream using one of the many recorders out there RecordRTC works fairly well. Wait until the stream is completed and then upload the file.
Send smaller chuncks of recorded audio with a timer and merge them again server side. This is an example of this
Send the audio packets as they occur over websockets to your server so that you can manipulate and merge them there. My version of RecordRTC does this.
Make an actual peer connection with your server so it can grab the raw rtp stream and you can record the stream using some lower level code. This can easily be done with the Janus-Gateway.
As for waiting to send the stream vs sending it in chunks, it all depends on how long you are recording. If it is for a longer period of time, I would say sending the recording in chunks or actively sending audio packets over websockets is a better solution as uploading and storing larger audio files from the client side can be arduous for the client.
Firefox actually has a its own solution for recording but it is not supported in chrome so it may not work in your situation.
As an aside, the signalling method mentioned is for session build/destroy and really has nothing to do with the media itself. You would only really worry about this if you were using possibly solution number 4 shown above.
A good API for you would be MediaRecorder API but it is less supported than the Web Audio API, so you can do it using a ScriptNode or use Recorder.js (or base on it to build your own scriptnode).
WebRTC is design as peer-to-peer, but the peer could be a browser and a server. So it's definitely possible to push the stream by WebRTC to a server, then record the stream as a file.
The stream flow is:
Chrome ----WebRTC---> Server ---record---> FLV/MP4
There are lots of servers, like SRS, janus or mediasoup to accept WebRTC stream. Please note that you might need to covert the WebRTC(H.264+Opus) to MP4(H.264+AAC), or just choose SRS which supports this feature.
yes it is possible to send MediaStream to your server, but the only way you can achieve is by going through WebSocket which enable client browser to send data to your server in real time connection. so i recommend you to use websocket