What are the steps and means of capturing microphone audio stream through HTML5 / Javascript (no flash) and then sending it to an already set up icecast server?
The solution must be solely browser/web-based, no additional software.
The server is on Rails 5.0.0.1.
Where should I start?
I'm struggling to find any relevant info online as everything talks about uploading/recording audio files as complete files, not streams.
The solution must be solely browser/web-based, no additional software.
That's not possible at the moment, because there is no way to make a streaming HTTP PUT request. That is, to make an HTTP request either via XHR or Fetch, the request body has to be immutable.
There is a new standard of ReadableStream which will be available any day now, allowing you to make a ReadableStream (your encoded audio) and PUT it to a server. Once that is available, this is possible.
The server is on Rails 5.0.0.1.
Not sure why you mention that since you said it has to all live in the browser. In any case, let me tell you how I implemented this, with a server-side proxy.
Client-side, you need to capture the audio with getUserMedia(). (Be sure to use adapter.js, as the spec has recently changed and you don't want to have to deal with the constraints object changes manually.) Once you have that, you can either send PCM samples (I'd recommend reducing from 32-bit float to 16-bit signed first), or you can use a codec like AAC or MP3 client-side. If you do the codec in the client, you're only going to be able to send one or two streams. If you do the codec server-side, you can derive as many as you want, due to CPU requirements, but you will take more bandwidth depending on the streams you're deriving.
For the codec on the client, you can either use MediaRecorder, or a codec compiled to JavaScript via emscripten (or similar). Aurora.js has some codecs for decoding, but I believe at least one of the codecs in there also had encoding.
To get the data to the server, you need a binary web socket.
On the server side, if you're keeping the codecs there, FFmpeg makes this easy.
Once you have the encoded audio, you need to make your HTTP PUT request to Icecast or similar, as well as update the metadata either out-of-band, or muxed into the container that you're PUT-ing to the server.
Self Promotion: If you don't want to do all of that yourself, I have some code you can license from me called the AudioPump Web Encoder, which does exactly what you ask for. You can modify it for your needs, take components and embed them into your project, etc. E-mail me at brad#audiopump.co if you're interested.
Related
My goal is to create a web-radio station hosted on localhost using JavaScript with node.js server for the backend part and HTML/CSS/JavaScript for the frontend part. My radio is going to be simple, and the goal is to create a server which is going to constantly broadcast a single song (or many songs using the .mp3 format) so they can be consumable from each client connected.
The streaming part is not so hard. The part I am struggling so far, is to achieve the "broadcast" transmission simultaneously to all consumers. The image below explains better my thoughts:
Does anyone have a code example so I can easily understand it and at the same time implement it for my project's use?
Has anyone faced a similar situation like this?
This is exactly what SHOUTcast/Icecast and compatible servers do. You basically just need to copy the audio data to each client.
You can use normal HTTP for the streaming protocol. The clients don't need to know or care that they're streaming live. A simple audio element works:
<audio src="https://stream.example.com/some-stream" preload="none" controls></audio>
On the server-side, you do need to ensure that you're sending a stream aligned in such a way that the client can just pick right up and play it. The good news is that for raw MP3 and ADTS (which normally wraps AAC) streams, all the data they need for playback is in each frame, so you can just "needle drop", stream from an arbitrary position, and the client will figure it out.
Inside your Node.js app, it will look a bit like this:
Input Stream -> Buffer(s) -> Clients HTTP Responses
And in fact, you can even drop the buffer part if you want. As data comes in from the codec, you can just write it to all the clients. The buffer is useful to ensure a fast starting playback. Most clients are going to need some buffered data to sniff stream type and compatibility, as well as to fill their own playback buffer for smooth streaming.
That's really all there is to it!
In Node.js, streams can be used easily to deliver a media content by chunks (as a video for example).
It is also possible to use an adaptative bitrate streaming method like DASH or Http Live Streaming by breaking down segments of the media to send. It looks like it is recommended to implement this second method over the first one in the case of a videos delivery streaming platform.
I would like to know why and what's the difference in terms of benefits and disadvantages to implement an adaptative bitrate streaming method instead of using Node.js native streams for a video streaming app ?
EDIT : Example of using Node.js streams to deliver a media content by chunks :
var videoStream = fs.createReadStream(path, { start, end });
res.writeHead(206, {
"Content-Range": `bytes ${start}-${end}/${size}`,
"Accept-Ranges": "bytes"
});
videoStream.pipe(res);
There is a bit of confusion in the question. The streams in Node.js are just a structure for easily sending streams of data around your application. They're not directly relevant.
I suspect what you're getting at is the difference between regular HTTP progressive streaming (such as piping video data to a response stream to a client) vs. a segmented streaming protocol like DASH or HLS.
Indeed, you can send an adaptive bitrate stream over a regular HTTP progressive stream as well. Nothing prevents you from changing the bitrate on the fly.
The reason folks use DASH and HLS is that they allow re-use of regular static file/blob/object-based CDNs. They don't require specialized server software to stream. Any HTTP server works.
Even if you don't need to use existing CDNs, you might as well stick with DASH/HLS for the benefit of there being client support most everywhere now. You won't have to write your own client to track metrics to decide when to switch bitrates. Using DASH/HLS allows you to be compatible with existing devices, often with writing no additional code at all.
Bringing this over from softwareengineering. Was told this question may be better for stackoverflow.
I am sending a video stream of data to another peer and want to reassemble that data and make that stream the source of a video element. I record the data using the npm package RecordRTC and am getting a Blob of data every 1s.
I send it over an WebRTC Data Channel and initially tried to reassemble the data using the MediaSource API but turns out that MediaSource doesn't support data with a mimetype of video/webm;codecs=vp8,pcm. Are there any thoughts on how to reassemble this stream? Is it possible to modify the MediaSource API?
My only requirement of this stream of data is that the audio be encoded with pcm but if you have any thoughts or questions please let me know!
P.S. I thought opinion based questioned weren't for stackoverflow so that's why I posted there first.
The easiest way to handle this is to proxy the stream through a server where you can return the stream as an HTTP response. Then, you can do something as simple as:
<video src="https://example.com/your-stream"></video>
The downside of course is that now you have to cover the bandwidth cost, since the connection is no longer peer-to-peer.
What would be nice is if you could use a Service Worker and have it return a faked HTTP response from the data you're receiving from the peer. Unfortunately, the browser developers have crippled the Service Worker standards by disabling it if the user reloads the page, or uses privacy modes. (It seems that they assumed Service Workers were only useful for caching.)
Also, a note on WebRTC... what you're doing is fine. You don't want to use the normal WebRTC media streams, as not only are they lossy compressed, but they will drop segments to prioritize staying realtime over quality. This doesn't sound like what you want.
I've been wondering this - is the raw mediastream returned from something like getusermedia what format is that in?
The MediaStream is the raw data, but it isn't accessible directly. If you attach the MediaStream to a Web Audio API graph, whatever format the sound card captured in is converted to 32-bit floating point PCM. At this point, you can use a script processor node to capture the raw PCM data.
Hi I am wondering how in javascript or reactjs would I read data from a streaming station?
I have googled sadly I have had no luck and I was wondering if anyone knows of a script that can read (icecast ICY metadata?)
Please note that web browsers don't support ICY metadata, so you'd have to implement quite a few things manually and consume the whole stream just for the metadata. I do NOT recommend this.
As you indicate Icecast, the recommended way to get metadata is by querying the JSON endpoint: /status-json.xsl. It's documented.
It sounds like you are custom building for a certain server, so this should be a good approach. Note that you must be running a recent Icecast version (at the very least 2.4.1, but for security reasons better latest).
If you are wondering about accessing random Icecast servers where you have no control over, it becomes complicated: https://stackoverflow.com/a/57353140/2648865
If you want to play a stream and then display it's ICY metadata, look at miknik's answer. (It applies to legacy ICY streams, won't work with WebM or Ogg encapsulated Opus, Vorbis, etc)
I wrote a script that does exactly this.
It implements a service worker and uses the Fetch API and the Readable Streams API to intercept network requests from your page to your streaming server, add the necessary header to the request to initiate in-stream metadata from your streaming server and then extract the metadata from the response while playing the mp3 via the audio element on your page.
Due to restrictions on service workers and the Fetch API my script will only work if your site is served over SSL and your streaming server and website are on the same domain.
You can find the code on Github and a very basic demo of it in action here (open the console window to view the data being passed from the service worker)
I don't know much about stream's but I've found some stuff googling lol
https://www.npmjs.com/package/icy-metadata
https://living-sun.com/es/audio/85978-how-do-i-obtain-shoutcast-ldquonow-playingrdquo-metadata-from-the-stream-audio-stream-metadata-shoutcast-internet-radio.html
also this
Developing the client for the icecast server
its for php but maybe you can translate it to JS.
I have a piece of software running on a node in my network that generates RTP streams carried over UDP/IP. Those streams contain streaming data, but not in any standard audio/video format (like H.264, etc.). I would like to have a simple Web app that can hook into these streams, decode the payloads appropriately, and display their contents. I understand that it isn't possible to have direct access to a UDP socket from a browser.
Is there a way to, using JavaScript/HTML5, to read an arbitrary RTP stream (i.e. given a UDP port number to receive the data from)? The software that sends the stream does not implement any of the signaling protocols specified by WebRTC, and I'm unable to change it. I would like to just be able to get at the RTP packet contents; I can handle the decoding and display without much issue.
As far as I know, there is nothing in the set of WebRTC APIs that will allow you to do this. As you have pointed out, there also isn't a direct programmatic way to handle UDP packets in-browser.
You can use Canvas and the Web Audio API to effectively playback arbitrary video, but this takes a ton of CPU. The MediaSource extensions can be used to run data through the browser's codec, but you still have to get the data somehow.
I think the best solution in your case is to make these connections server-side and use something like FFmpeg to output a stream in a codec and container that your browser can handle, and simply play back in a video element. Then, you can connect to whatever you want. I have done similar projects with Node.js which make it very easy to pipe streams through, and on out to the browser.
Another alternative is to use WASM and create your own player for your stream. It's pretty incredible technology of these recent years > 2014. Also as stated by #Brad, WebRTC doesn't support what you need even as of this year 2020.