Develop Reference

Not able to create seekable video blobs from mediarecorder using EBML.js - MediaRecorder API - Chrome

Using media recorder, I am able to upload and append the video blobs on azure. But combined video is not seekable on download with following code -
var chunks =[];
var mediaRecorder = new MediaRecorder(stream, 'video/x-matroska;codecs=vp8,opus');
mediaRecorder.ondataavailable = function(event) {
if(event.data && event.data.size > 0) {
chunks.push(event.data);
appendBlockToAzure(chunks);
}
};
mediaRecorder.start(10000);
I tried using EBML.js, if I use the following code then I get the seekable video file. This approach needs the file to be processed at the end. Therefore, final file could be of 1GB in size which will take very long time to upload.
var chunks =[];
var mediaRecorder = new MediaRecorder(stream, 'video/x-matroska;codecs=vp8,opus');
mediaRecorder.ondataavailable = function(event) {
if(event.data && event.data.size > 0) {
chunks.push(event.data);
if(mediaRecorder.state == "inactive") { //if media recorder is stopped
var combined = new Blob(chunks, { type: event.data.type });
getSeekableBlob(combined, function (seekableBlob) {
saveCombinedVideoToAzure(seekableBlob);
});
}
}
};
mediaRecorder.start(10000);
That's the reason I want to upload simultaneously to the azure. If I use the following code, then it logs unknown tag warnings and then length error. Also, the video file is not playable.
var seekablechunks =[];
var mediaRecorder = new MediaRecorder(stream, 'video/x-matroska;codecs=vp8,opus');
mediaRecorder.ondataavailable = function(event) {
if(event.data && event.data.size > 0) {
getSeekableBlob(event.data, function (seekableBlob) {
seekablechunks.push(seekableBlob);
saveCombinedVideoToAzure(seekablechunks);
});
}
};
mediaRecorder.start(10000);
Function 'getSeekableBlob':
function getSeekableBlob(inputBlob, callback) {
// EBML.js copyrights goes to: https://github.com/legokichi/ts-ebml
if(typeof EBML === 'undefined') {
throw new Error('Please link: https://www.webrtc- experiment.com/EBML.js');
}
var reader = new EBML.Reader();
var decoder = new EBML.Decoder();
var tools = EBML.tools;
var fileReader = new FileReader();
fileReader.onload = function (e) {
var ebmlElms = decoder.decode(this.result);
ebmlElms.forEach(function (element) {
reader.read(element);
});
reader.stop();
var refinedMetadataBuf = tools.makeMetadataSeekable(reader.metadatas, reader.duration, reader.cues);
var body = this.result.slice(reader.metadataSize);
var newBlob = new Blob([refinedMetadataBuf, body], {
type: 'video/webm'
});
callback(newBlob);
};
fileReader.readAsArrayBuffer(inputBlob);
}
Is there a way to get seekable blobs and upload them to azure?

It's a challenge for an open-ended streaming source for media (for example MediaRecorder) to create a file with SeekHead elements in it. The Seek elements in a SeekHead element contain byte offsets to elements in the file.
MediaRecorder doesn't create segments or SeekHead elements as you have discovered. To do so it would need to be able to see the future to know how big future compressed video and audio elements will be in the file.
A good way for you to handle this problem might be to post-process your uploaded files on a server. You can use ts-ebml to do this in a streaming fashion on a server when a file is completely uploaded.
It's possible, I suppose, to create Javascript software in your browser that can transform the stream of data emitted by MediaRecorder so it's seekable, on the fly. To make your stream seekeable you'd need to insert SeekHead elements every so often. You'd buffer up multiple seconds of the stream, then locate the Cluster elements in each buffer, then write a SeekHead element pointing to some of them. (Chrome's MediaRecorder outputs Clusters beginning with video key frames.) If you succeed in doing this you'll know a lot about Matroska / webm.

Suddenly, our Face on camera web-cam recorder component stopped saving webm blob.
In the console there were warnings about {EBML_ID: "55b0", type: "unknown", ...} during reader.read(element) and then
"Uncaught (in promise) Error: No schema entry found for unknown" in EBMLEncoder.js" at tools.makeMetadataSeekable(...) call.
Ignoring unknown elements from the decoder workarounded the issue:
...
var ebmlElms = decoder.decode(this.result);
ebmlElms.forEach(function (element) {
if (element.type !== 'unknown') {
reader.read(element);
}
});
reader.stop();
...
Related issue on ts-ebml npm package https://github.com/legokichi/ts-ebml/issues/33 with similar workaround

Hls.js record file

Hello and thanks for reading,
I have a Hls stream with an m3u8 playlist.
The Video is playing just fine on an Html page with a Video element and https://github.com/video-dev/hls.js
But if I download the segments to join them they are only white pixels. VLC and FFmpeg can't handle them. VLC shows a white pixel for 10seconds and FFmpeg says that there's no stream in the file.
So now I want to know what this hls.js is doing to make it running. To me a non-js developer it all looks a bit confusing. I was able to understand stuff like which function is called when a new segment is loaded. Unfortunately, I was unable to understand stuff about the data. The one character variables are confusing to me.
For now, I capture the stream of the video element and download it later but I don't like this solution at all.
How to help me
It would be very nice if anyone can tell me how to hook into the
script and tell it to download directly to the disk so I'm independent
of framerate drops.
If anyone can tell how the script is able to convert the data so that
the element can use it and I would be able to implement or do
it with FFmpeg would be really helpful.
I also thought it might be possible to have a listener when the blob
changes to store its contents.
Thanks for everyone helping. I'm trying to find a solution for too many hours now.

I found the solution. After looking at their great event system
https://github.com/video-dev/hls.js/
and this issue which I contributed too and not just copied
https://github.com/video-dev/hls.js/issues/1322
var arrayRecord = [];
function download(data, filename) {
console.log('downloading...');
var blob = new Blob([arrayConcat(data)], {
type: 'application/octet-stream'
});
saveAs(blob, filename);
}
function arrayConcat(inputArray) {
var totalLength = inputArray.reduce(function (prev, cur) {
return prev + cur.length
}, 0);
var result = new Uint8Array(totalLength);
var offset = 0;
inputArray.forEach(function (element) {
result.set(element, offset);
offset += element.length;
});
return result;
}
function saveAs(blob, filename) {
var url = URL.createObjectURL(blob);
var a = document.createElement("a");
document.body.appendChild(a);
a.style = "display: none";
a.href = url;
a.download = filename;
a.click();
window.URL.revokeObjectURL(url);
}
function stopRecord() {
arrayRecord.forEach(function (item) {
download(item.data['video'], "video.mp4");
download(item.data['audio'], "audio.mp4");
item.hls.destroy();
return false;
});
}
function startRecord() {
var video = document.getElementById('video');
var dataStream = {
'video': [],
'audio': []
};
var hls = new Hls();
hls.loadSource("Your playlist");
hls.attachMedia(video);
hls.on(Hls.Events.MANIFEST_PARSED, function () {
video.play();
hls.on(Hls.Events.BUFFER_APPENDING, function (event, data) {
console.log("apending");
dataStream[data.type].push(data.data);
});
});
arrayRecord.push({
hls: hls,
data: dataStream
});
video.onended = function (e) {
stopRecord()
}
}

Javascript | MediaRecorder API - Recorded video file on Desktop does not play in Mobile

I am trying to develop a canva-like Insta story creator using Canvas and MediaRecorder
The app is working perfectly on a desktop browser - I am able to download the file, and play it on desktop. However, when I send that file to my mobile, it doesn't play(even on Insta). I figure this is an issue with codecs - but don't know how to solve the same.
This is the function that handles the mediaRecorderAPI
Is there any mime type that I can use, that is universal and can play for any device?
initRecorder () {
var dl = document.querySelector("#dl")
let videoStream = this.canvas.captureStream(60);
if(this.isAudioPresent) {
videoStream.addTrack(this.audioStream.getAudioTracks()[0])
}
let mediaRecorder = new MediaRecorder(videoStream, {
videoBitsPerSecond : 2500000,
mime: 'video/webm'
});
let chunks = [];
mediaRecorder.onstop = function(e) {
var blob = new Blob(chunks, { 'type' : 'video/webm' });
chunks = [];
var videoURL = URL.createObjectURL(blob);
dl.href = videoURL;
};
mediaRecorder.ondataavailable = function(e) {
e.data.size && chunks.push(e.data);
};
mediaRecorder.start();
setTimeout(function (){ mediaRecorder.stop(); },this.storytime);
}
```

Figured this out: Different browsers use different transcoding. Insta only accepts MP4 transcoding. Hence, you need to use either a transcoder on the frontend(ffmpeg.js or wasm version of ffmpeg) or send your data to backend and handle there(which I ended up doing)

Using MediaSource for video with MediaRecord

I'm trying to broadcast a video from my webcam in javascript. I'm using MediaStream to get the video from my webcam, MediaRecorder to record such video in chunks (which would be transmitted to the server), and MediaSource to assemble these chunks and play them seamlessly in a video container called watchVideo on the source below.
It all works perfectly when i'm capturing only video, i.e. constraints = { video: true } ; but if I add audio, the watchVideo doesn't display anything, and the console shows me the following error:
Uncaught DOMException: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source.
This is the relevant part of the code:
var mime = 'video/webm; codecs=vp8';
if (navigator.mediaDevices) {
constraints = { video: true, audio: true };
navigator.mediaDevices.getUserMedia(constraints)
.then(handleUserMedia)
.catch( err => {
console.log("ERROR: " + err);
})
}
function handleUserMedia(stream) {
source = new MediaSource();
watchVideo.src = window.URL.createObjectURL(source);
source.onsourceopen = () => {
buffer = source.addSourceBuffer(mime);
};
var options = { mimeType: mime };
mediaRecorder = new MediaRecorder(stream, options);
mediaRecorder.ondataavailable = handleDataAvailable;
}
function handleDataAvailable(evt) {
var filereader = new FileReader();
filereader.onload = () => {
buffer.appendBuffer(filereader.result );
};
filereader.readAsArrayBuffer(evt.data);
}

I came across the question and it actually helped me more than many answers related to this topic. I don't know if you are still interested in the answer but I have tried
mime="video/webm; codecs="vp9,opus"
and it worked fine with audio and video, I hope this answer will help you

QuotaExceededError using MediaStream API

My application receives some video chunks from a ServerSentEvent (SSE) and, using MediaStream API, it should append them in a buffer and visualize them into an HTML5 video tag.
The problem is MediaSource API, that stops working when the program tries to append a chunk to the mediaStream buffer.
The error appears when the program tries to append the first chunk.
This is the client-side code:
window.MediaSource = window.MediaSource || window.WebKitMediaSource;
window.URL = window.URL || window.webkitURL;
if (!!!window.MediaSource) {alert('MediaSource API is not available');}
var video = document.getElementById("video");
var mediaSource = new MediaSource();
video.src = window.URL.createObjectURL(mediaSource);
mediaSource.addEventListener('sourceopen', function(e) {
var source = new EventSource('http://localhost:5000/video');
// this is the line that catch the error
var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"');
source.addEventListener('chunkStreamed', function(e){
var chunk = new Blob(JSON.parse(e.data));
console.log("chunk: ", chunk);
var reader = new FileReader();
reader.onload = function(e) {
// error is caused by this line
sourceBuffer.appendBuffer(new Uint8Array(e.target.result));
};
reader.readAsArrayBuffer(chunk);
});
source.addEventListener('endStreaming', function(e){
console.log(e.data);
// mediaSource.endOfStream();
// endOfStream() not here, sourceBuffer.appendBuffer will done after this command and will cause InvalidStateError
});
source.onopen = function(e) {
console.log("EventSource open");
};
source.onerror = function(e) {
console.log("error", e);
source.close();
};
}, false);
and this is the complete error log:
Uncaught QuotaExceededError: An attempt was made to add something to storage that exceeded the quota.
The problem comes out when the app tries to do sourceBuffer.appendBuffer(new Uint8Array(e.target.result));.
I really can't understand way this error appear. Code is really
like the code of this example