I am hoping to make an example using the developer build of chrome and being able to use subsonic to stream a binary audio file. So far I have not had any luck though.
Granted my next option will be try to load in the audio files into windowStorage and toss some magic dust on them.
Does anybody know a way to stream a audio file to the audio tag through binary?
Related
I need to record a webpage and save it as a video, in an automated manner, without human interaction.
I am creating a NodeJS app that generates MP4 videos on the request of the user. The user provides an MP3 file, the app generates animated waveforms for the sound file on top of an illustration.
What I came up with so far is a system that opens a generated web page in the backend, plays the audio file, and shows audio visualization for the audio file on an HTML canvas element. On top of another canvas with mainly static components, such as images, that do not animate. The system records this, the output will be a video file. Finally, I will merge the video file with the sound file to create the final file for the user.
I came up with 2 possible solutions but both of them have problems which I am not able to solve at the moment.
Solution #1
Use a headless browser API such as Phantomjs or Puppeteer to snatch a screenshot x time every second and pipe it to FFmpeg.
The problem
The problem with this is that the process is not realtime. It would work fine if it's JUST an animation but mine is dependant on the audio file. The audio file will play-on during the render which results in a glitchy 1FPS-esque video.
Possible solution?
Don't play the audio file live but convert the audio file into raw data. Animate the audio visualization based on the raw data instead.
Not sure how to do this and if it's even possible.
Solution #2
Play, record, and save the animation, all in the frontend.
Could use ccapture.js to record and save a canvas.
Use a headless browser to open the page and save it to disk when it's done playing.
Doesn't sound like it's the best solution.
The problem(s)
I have more than 1 canvas.
It takes a while, especially when the audio file is longer than 10 minutes.
Making users wait for a long time can be a deal-breaker.
Possible solution?
Merge canvases into one.
No idea how to speed up the rendering time and I doubt it's possible this way.
Late answer from someone looking for similar options due to the convenience of some browser SVG APIs:
My first recommendation, as someone who has written a fair amount of my own audio visualization software, is to use a graphics library and language that don't require a browser or GPU, like Gd or Anti-grain Geometry or Cairo with any server-side language. You might also check out Processing.org (which I haven't used), not sure if there's a headless version.
If that's not possible, I've found these so far but haven't tried them:
https://github.com/tungs/timecut
https://github.com/myplanet/headless-render
https://wave.video/blog/how-we-render-animated-content-from-html5-canvas/
So here's the issue that I'm currently having. I need an audio player for iOS that will play mp3. Now at first glance this may seem like a trivial issue, just create an audio tag and give it the URL to the mp3 file. While this technically works, it's basically unusable since iOS needs to download the entire file before starting to play it. This results in a really long wait time when trying to play large mp3 files (could get close to a minute).
So the first thing I tried was to manually mimic the chunking that chrome does for playing mp3 files. I first sent a HEAD request to get the byte length of the audio file. Used that length / duration in seconds to get the average bytes per second and use that data to request chunks based on where the user seeks to. That didn't work since sometimes mp3 files contain metadata that throw off the calculation (like a cover image). Additionally, sometimes mp3 files use VBR (Variable Bit Rate) and then I'm well and truly screwed.
So this lead me to thinking, Safari couldn't possibly require the end user to download an entire mp4 file before playing it. So I took an mp3 file, converted it to mp4 on an online converter and tested my theory out. Voila, it worked. Safari was streaming the mp4 file in chunks and the wait time went to close to 0. Alright, so now all I need to do is convert mp3 files to mp4 files. The problem now is, I have requirement not to use my server for converting these files. I want to offload this expensive operation to the client. After looking around for a bit I found a few wasm libraries to do this, great! Nope. These wasm libraries are huge (24 MB) and would add an unacceptable amount to my already large bundles files. So this brings me to my question. Is there any way to "trick" safari into thinking that my mp3 file is mp4. What I've tried already:
On input change event -> get the file -> clone it into a new Blob with a mimeType of video/mp4 and then upload that file to the server. Chrome plays this file no problem (probably because it detects it's an mp3 file), on Safari however it's unable to play the file.
So my question is, is there any way to package the mp3 file in an mp4 container (Client Side !important) to "force" Safari into chunking the file.
Is there any way to "trick" safari into thinking that my mp3 file is
mp4
mp4 files are not seekable because the client "thinks" they are mp4. They have an index in the file that is required for the seeking to work. That index does not magically exist because the mime type was changed.
is there any way to package the mp3 file in an mp4 container (Client Side...
Yes, totally. But you must write the code and it is very complicated. You can read ISO 14496-12 document to understand how mp4 files are structured. and ISO 11172-3 and ISO 13818-3 to parse the Mp3 into frames that can be written to the MP4
I did this very thing last month, i only needed it for a 128kbps CBR mp3 live stream so i hard coded something to support that specific format, it can be modified to specifically support different bit rates though, but probably not VBR :/. Posting it here in case its of use to anyone dealing with firefox not playing an mp3 using mediasource extensions: https://gist.github.com/fanfare/0fa525af28b275fd6623942d7e9d70dd
I'm trying to capture audio output from the browser and save a recorded file in JavaScript (without using 3rd party apps or browser extensions).
After reviewing the examples at WebRTC samples this task seems to be relatively straightforward when capturing audio from a user's microphone using the MediaStream output of getUserMedia().
Is there a way to capture a MediaStream that is just the browser's audio output? Or is there some better way to access the browser's audio output in a way that can be recorded to a file?
For context, my audio output in the browser may originate from one of several audio libraries (Tone.js, for example) so I'd rather not rely on generating the audio file from the JS library that is generating the audio. I've looked into writing a file from the AudioContext, but I am trying to find some solution that would be audio-source-agnostic.
I"m trying to use the chrome API https://developers.chrome.com/extensions/tabCapture.
How do I get a audio file out of it. For example, if I'm watching youtube and want to export whatever song I'm doing into an audio file MP3, wav, etc. I know there are some chrome extensions out there that does this but I want to know if there is an API.
Thanks!
Use the desktopCapture API with DesktopCaptureSourceType "audio"
Saving the stream may be more challenging. You could create a Web Audio API Audio context and call decodeAudioData() to get raw PCM data (e.g. .wav file). MP3 encoding you might be able to find some kind of Emscripten module for.
Alternatively, it looks like there is a media recorder API (demo) I'm not sure if this is stable in many browsers yet, but I just tried in Chrome and it seemed to work.
So I'm trying to write a server to stream video to a client in html5/javascript, and I'd like to use the already existing framework of the video tag in html5 if that's possible. That being said, I can't find I good source for what the format of the stream is. I think it works using progressive downloading (this is how youtube works as well?), but I can't find what the header of any given progressive download packet should look like.
Can someone point me in the direction of some information about the actual format of the video tag stream? I'm also not 100% devoted to the idea of using the video tag, so if someone has a better alternative, that'd also be great!
You just need an HTTP server such as Apache or Nginx to serve the progressive download of MP4 video files.