I am building an electron in which I need the desktopCapturer api, but I don't fully understand how to use it.
From the api official page (and this example app: https://github.com/hokein/electron-sample-apps/tree/master/desktop-capture) I see that the desktopCapturer only gives me the id's of the sources, not the video streams themselves. For that, I should use navigator.mediaDevices.getUserMedia(). But the constraints object no longer has the mandatory property and because I am using typescript I am getting an error if I try to use it.
I've tried to use the deviceId property instead, but I am getting this error:
Uncaught (in promise) DOMException: Requested device not found (on a device with a webcam I would get the webcam stream instead of that error). Here is my code:
import { desktopCapturer, DesktopCapturerSource } from "electron";
function onLoad(){
desktopCapturer.getSources({
thumbnailSize: {
width: 256,
height: 256,
},
types: ["screen", "window"]
}, (error: Error, srcs: DesktopCapturerSource[]) => {
if (error)
throw error;
let video: HTMLVideoElement | null = document.querySelector("video");
for (let src of srcs)
navigator.mediaDevices.getUserMedia({
video:{
deviceId : src.id
}
}).then((stream:MediaStream)=>{
if(video){
video.srcObject = stream;
video.play();
}
})
})
}
document.addEventListener("DOMContentLoaded", onLoad);
I also tried using navigator.getDisplayMedia(), but I wouldn't get the pop-up prompting to select a source as I would get in Chrome. What should I do to get this working? Thanks in advance!
I found the solution, at least for new since WebRTC is not yet standardized. Copying the navigator object into a variable and casted to any allows the use of the mandatory property on the constraints object since typescript no longer checks for type compatibility
Related
I am using Electron to create a Windows application that creates a fullscreen transparent overlay window. The purpose of this overlay is to:
take a screenshot of the entire screen (not the overlay itself which is transparent, but the screen 'underneath'),
process this image by sending the image as a byte stream to my python server, and
draw some things on the overlay
I am getting stuck on the first step, which is the screenshot capturing process.
I tried option 1, which is to use capturePage():
this.electronService.remote.getCurrentWindow().webContents.capturePage()
.then((img: Electron.NativeImage) => { ... }
but this captures my overlay window only (and not the desktop screen). This will be a blank image which is useless to me.
Option 2 is to use desktopCapturer:
this.electronService.remote.desktopCapturer.getSources({types: ['screen']}).then(sources => {
for (const source of sources) {
if (source.name === 'Screen 1') {
try {
const mediaDevices = navigator.mediaDevices as any;
mediaDevices.getUserMedia({
audio: false,
video: { // this specification is only available for Chrome -> Electron runs on Chromium browser
mandatory: {
chromeMediaSource: 'desktop',
chromeMediaSourceId: source.id,
minWidth: 1280,
maxWidth: 1280,
minHeight: 720,
maxHeight: 720
}
}
}).then((stream: MediaStream) => { // stream.getVideoTracks()[0] contains the video track I need
this.handleStream(stream);
});
} catch (e) {
}
}
}
});
The next step is where it becomes fuzzy for me. What do I do with the acquired MediaStream to get a bytestream from the screenshot out of it? I see plenty of examples how to display this stream on a webpage, but I wish to send it to my backend. This StackOverflow post mentions how to do it, but I am not getting it to work properly. This is how I implemented handleStream():
import * as MediaStreamRecorder from 'msr';
private handleStream(stream: MediaStream): void {
recorder.stop()
const recorder = new MediaStreamRecorder(stream);
recorder.ondataavailable = (blob: Blob) => { // I immediately get a blob, while the linked SO page got an event and had to get a blob through event.data
this.http.post<Result>('http://localhost:5050', blob);
};
// make data available event fire every one second
recorder.start(1000);
}
The blob is not being accepted by the Python server. Upon inspecting the contents of Blob, it's a video as I suspected. I verified this with the following code:
let url = URL.createObjectURL(blob);
window.open(url, '_blank')
which opens the blob in a new window. It displays a video of maybe half a second, but I want to have a static image. So how do I get a specific snapshot out of it? I'm also not sure if simply sending the Javascript blob format in the POST body will do for Python to be correctly interpret it. In Java it works by simply sending a byte[] of the image so I verified that the Python server implementation works as expected.
Any suggestions other than using the desktopCapturer are also fine. This implementation is capturing my mouse as well, which I rather not have. I must admit that I did not expect this feature to be so difficult to implement.
Here's how you take a desktop screenshot:
const { desktopCapturer } = require('electron')
document.getElementById('screenshot-button').addEventListener('click', () => { // The button which takes the screenshot
desktopCapturer.getSources({ types: ['screen'] })
.then( sources => {
document.getElementById('screenshot-image').src = sources[0].thumbnail.toDataURL() // The image to display the screenshot
})
})
Using 'screen' will take a screenshot of the entire desktop.
Using 'windows' will take a screenshot of only the application.
Also refer to these docs: https://www.electronjs.org/docs/api/desktop-capturer
desktopCapturer only takes videos. So you need to get a single frame from it. You can use html5 canvas for that. Here is an example:
https://ourcodeworld.com/articles/read/280/creating-screenshots-of-your-app-or-the-screen-in-electron-framework
Or, use some third party screenshot library available on npm. The one I found needs to have ImageMagick installed on linux, but maybe there are more, or you don't need to support linux. You'll need to do that in the main electron process in which you can do anything that you can do in node.
You can get each frame from taken video like this:
desktopCapturer.getSources({
types: ['window'], thumbnailSize: {
height: 768,
width: 1366
}
}).then(sources => {
for (let s in sources) {
const content = sources[s].thumbnail.toPNG()
console.log(content)
}
})
I want to get a video from the webcam using JS but no footage.
MESSAGE:
DOMException: Could not start video source
App.js
const video = document.getElementById("video");
function startVideo() {
navigator.getUserMedia(
{
video: {}
},
stream => (video.srcObject = stream),
err => console.log(err)
);
}
startVideo();
index.html
...
<body>
<video id="video" width="720" height="540" autoplay muted></video>
</body>
...
thanks for your help
If anyone else is having this problem and nothing else helps. Make sure that your camera is not already claimed/used by a different software/browser.
TLDR: I have tested your code and had to change it a bit:
https://codepen.io/puradawid/pen/PoqxzPQ
It looks like the problem lays here:
navigator.getUserMedia({
video: {}
},
stream => { video.srcObject = stream },
err => console.log(err)
);
Regarding to docs navigator.getUserMedia is deprecated and there is navigator.mediaDevices.getUserMedia that supports it. However, changing that up doesn't solve the correct problem which is your callback functions. This method returns Promise that is controlled by .then(), so changing it allows me to see my face in codepen:
navigator.mediaDevices.getUserMedia({
video: true
}).then(
stream => (video.srcObject = stream),
err => console.log(err)
);
I ran into this problem on certain android devices (Sony XA2) when trying to toggle the camera on a mobile browser because I am calling navigator.mediaDevices.getUserMedia repeatedly on each camera toggle.
The solution that I found was to make sure to stop all the tracks in previous streams that you created.
this.stream.getTracks().forEach(t => {
t.stop();
this.stream.removeTrack(t);
});
Without the previous code you can't seem to toggle camera on certain Android devices: (Demo),
(Code)
The error shown was DOMException: Requested device not found
By stopping previous tracks: you are able to start a new stream:
(Demo)
Code
Note: The following code snippet doesn't seem to execute in stack overflow due to security restrictions, so please use the jsfiddle links.
class Camera {
constructor({ video }) {
this.facingMode = "environment";
this.video = video;
video.onloadedmetadata = () => {
video.play();
};
}
async toggleCamera() {
if (this.facingMode === "environment") {
this.facingMode = "user";
} else {
this.facingMode = "environment";
}
try {
if (this.stream){
/* On some android devices, it is necessary to stop the previous track*/
this.stream.getTracks().forEach(t => t.stop());
}
this.stream = await navigator.mediaDevices.getUserMedia({
video: {
facingMode: this.facingMode,
}
});
} catch (e) {
console.error(e);
}
this.video.srcObject = this.stream;
}
}
const camera = new Camera({
video: document.querySelector("video"),
});
const button = document.querySelector("button");
button.addEventListener('click', () => {
camera.toggleCamera();
});
<button>
Toggle Camera
</button>
<video></video>
In Windows 10 go to settings->privacy->App permission(left side)->Microphone-> enable 'Allow apps to access your microphone'
After that retry with your JS program....It will work!!
If anyone have such an error and you are working on a laptop. You can try bending your laptop monitor in both directions. Sometimes the cable comes loose. This helped in my case.
Also look into Feature-Policy HTTP header, both on the website and on the host web server config, and make sure camera access is allowed.
I have tried all the other solution but nothing is work. Then finally i need to uninstall my camera driver in Device Manager and then scan for hardware changes. Try to run the app again and it's working.
I'm building electron desktop app in windows 10.
electron: 15.3.0
Source video that helped me: https://www.youtube.com/watch?v=XE2ULFlzkxw
I went to the browser settings for camera and noticed that the default camera was showing as "Leap Motion" which is not a standard camera device. I changed to an actual webcam and the problem was solved.
Sometimes this type of error also appears when you try to make a peer-to-peer call system and your tests are done on the same device. The camera is already used by the initiator and the receiver can no longer activate the camera.
I'm taking an image from the webcam and storing it to the server. Everything was working fine until I got the chrome update today. My latest chrome version is:
Version 71.0.3578.80 (64 bit)
This line is throwing an error:
camera.src = window.URL.createObjectURL(stream);
Failed to execute 'createObjectURL' on 'URL': No function was found
that matched the signature provided.
According to this link here. I applied the code
try {
this.srcObject = stream;
} catch (error) {
this.src = window.URL.createObjectURL(stream);
}
Its not displaying the camera feed.
For reference - This jsfiddle code is not working on my chrome anymore.
It's just been removed from the current version of Chrome. I suddenly started getting this error after it updated. I have no idea why it never printed deprecation warnings before today.
Instead of setting the src property to URL.createObjectURL(stream) you're now supposed to set the srcObject property to the stream directly. It seems to be working in Chrome and Firefox.
Source: https://developers.google.com/web/updates/2018/10/chrome-71-deps-rems
Since google update version 71, this issue is generating. I also faced this issue. But now I got the solution to it. Here is the solution:
Replace
videoElement.src = URL.createObjectURL(screenStream);
to
videoElement.srcObject = screenStream;
Have you tried this?
try {
camera.srcObject = stream;
} catch (error) {
camera.src = window.URL.createObjectURL(stream);
}
In chrome, it works fine if you use:
video.srcObject = stream;
Instead of:
this.srcObject = stream;
See printscreen here
I want to capture video with the webcamera.
And there is the right decision:
window.onload = function () {
var video = document.getElementById('video');
var videoStreamUrl = false;
navigator.getUserMedia({video: true}, function (stream) {
videoStreamUrl = window.URL.createObjectURL(stream);
video.src = videoStreamUrl;
}, function () {
console.log('error');
});
};
but produces an error in the browser:
[Deprecation] URL.createObjectURL with media streams is deprecated and will be removed in M68, around July 2018. Please use HTMLMediaElement.srcObject instead. See https://www.chromestatus.com/features/5618491470118912 for more details.
how to use HTMLMediaElement.srcObject for my purposes ? Thanks for your time!
MediaElement.srcObject should allow Blobs, MediaSources and MediaStreams to be played in the MediaElement without the need to bind these sources in the memory for the lifetime of the document like blobURIs do.
(Currently no browser support anything else than MediaStream though...)
Indeed, when you do URL.createObjectURL(MediaStream), you are telling the browser that it should keep alive this Source until your revoke the blobURI, or until the document dies.
In the case of a LocalMediaStream served from a capturing device (camera or microphone), this also means that the browser has to keep the connection to this device open.
Firefox initiated the deprecation of this feature, one year or so ago, since srcObject can provide the same result in better ways, easier to handle for everyone, and hence Chrome seems to finally follow (not sure what's the specs status about this).
So to use it, simply do
MediaElement.srcObject = MediaStream;
Also note that the API you are using is itself deprecated (and not only in FF), and you shouldn't use it anymore. Indeed, the correct API to capture MediaStreams from user Media is the MediaDevices.getUserMedia one.
This API now returns a Promise which gets resolved to the MediaStream.
So a complete correction of your code would be
var video = document.getElementById('video');
navigator.mediaDevices.getUserMedia({
video: true
})
.then(function(stream) {
video.srcObject = stream;
})
.catch(function(error) {
console.log('error', error);
});
<video id="video"></video>
Or as a fiddle since StackSnippets® overprotected iframe may not deal well with gUM.
I'm trying to share my screen with Kurento WebRtc server. But getting this error:
NavigatorUserMediaError {name: "ScreenCaptureError", message: "", constraintName: ""}
There is no errors in Firefox with same code.
Constraints using for webrtc:
var constraints = {
audio: true,
video: {
mandatory : {
chromeMediaSource: 'screen',
maxWidth: 1920,
maxHeight: 1080,
maxFrameRate: 30,
minFrameRate: 15,
minAspectRatio: 1.6
},
optional: []
}
}
var options = {
localVideo : video,
onicecandidate : onIceCandidate,
mediaConstraints : constraints
}
webRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendonly(options,function(error) {
if (error) {
return console.error(error);
}
webRtcPeer.generateOffer(onOfferPresenter);
});
How do I share my screen using chrome and kurento?
Sharing a screen with Kurento through WebRTC, is exactly the same as sharing the webcam: get the stream from the client and negotiate the endpoint. The tricky part when doing screenshare is to get the stream. The kurento-utils-js library will give you a little help on that, as you can create the WebRtcPeer object, in the client, indicating that you want to share your screen or a window. You just need to make sure that you
have an extension installed to do screen-sharing in Chrome. In FF, it's enough to add the domain to the whitelist. Check this extension.
pass a valid sendSource value (screen or window) in the options bag when creating the kurentoUtils.WebRtcPeer object
have a getScreenConstraints method in your window object, as it will be used here. getScreenConstraints should return a valid set of constraints, depending on the browser. YOu can check an implementation of that function here
I think that should be enough. We are doing screen sharing with the library, using our own getScreenConstrains and extension, and it works fine. Once you have that, doing screen sharing with the kurento-utils-js library is quite easy. Just need to pass the sendSource value when creating the peer like so
var constraints = {
audio: false,
video: true
}
var options = {
localVideo: videoInput, //if you want to see what you are sharing
onicecandidate: onIceCandidate,
mediaConstraints: constraints,
sendSource: 'screen'
}
webRtcPeerScreencast = kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options, function(error) {
if (error) return onError(error) //You'll need to use whatever you use for handling errors
this.generateOffer(onOffer)
});
The value of sendSource is a string, and it depends on what you want to share
'screen': will let you share the whole screen. If you have more than one, you can choose which one to share
'window': lets you choose between all open windows
[ 'screen', 'window' ]: WARNING! Only accepted by Chrome, this will let the user choose between full screens or windows.
'webcam': this is the default value of you don't specify anything here. Guess what'll happen ;-)