As I'm playing with WebRTC in Chrome, I'm noticing that the durability of these streams is still somewhat shaky. I need to create a video stream before the element displaying it is shown (technically I only need the audio track initially, but renegotiation without replaceTrack() seems to be an issue all in its own, so I'm enabling both at once for now).
The element is then rendered dynamically by JavaScript and needs to start receiving WebRTC video. The problem is that at the time of WebRTC creation this video element where I want to show it does not yet exist. I don't see a way to tell WebRTC to change the video element being rendered to after the stream starts, is that possible? I was mainly playing with SimpleWebRTC, but am open to using WebRTC directly - from looking at the docs I couldn't find a way to do it with raw WebRTC either. I also tried moving the original video element into the new element, but this causes the video stream to break/stop:
newElement.appendChild(originalWebRTCVideoTag);
Short of killing the entire stream and restarting, what are my options?
UPDATE:
For both approaches, videoTag is a generic DOM video tag, webrtc is an instance of WebRTC object with a working connection established via SimpleWebRTC (simpleWebRtc.webrtc, which SimpleWebRTC wraps around). I'm putting together a JSFiddle right now for those who want to see the actual code but this should be enough information to reproduce this.
// this doesn't seem to be working in stackoverflow, probably because it rejects video camera capture
var simplertc = new SimpleWebRTC({
localVideoEl: 'webrtc-local',
remoteVideosEl: 'webrtc-remote',
media: {"audio": true, "video": {
"optional": [{"minWidth": "640"}, {"minHeight": "480"}], "mandatory": {}
}},
autoRequestMedia: true
});
var webrtc = simplertc.webrtc;
// this portion is overly simplified, in this case there is no point
// in creating this dynamically, in the app I'm working on this element
// is generated much later
$('#dynamic').appendTo('<video id="dynamic-video"></video>');
var videoTag = $('#dynamic-video')[0];
simplertc.on('readyToCall', function() {
simplertc.joinRoom('my-room-875385864'); // random name
// by this time the local video should be ready, we don't need remote ones for our test
// test case 1 (replace with logic from test case 2 if needed)
videoTag.srcObject = webrtc.localStreams[0];
// end test case
});
video {
border: 1px solid red;
width: 200px;
}
/* overlap with original video is intentional to show hardware acceleration effect */
#dynamic {
position: absolute;
border: 1px solid black;
width: 200px;
height: 200px;
left: 100px;
top:50px;
}
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.0.0/jquery.min.js"></script>
<script src="https://simplewebrtc.com/latest-v2.js"></script>
<div id='webrtc'>
<video id='webrtc-local'></video>
<div id='webrtc-remote'></div>
</div>
<div id='dynamic'>
</div>
Approach 1, stumbled upon this by accident while attempting approach 2
Tried the following, it works but much slower than I'd like, about 5 FPS:
// note that I can just as easily use remote streams here
videoTag.srcObject = webrtc.localStreams[0]
Ironically, while messing with this approach more I accidentally overlapped the video regions of the webRTC element and the one generated (videoTag), and even though webRTC is on the background, that corner of videoTag where it overlaps does run in real time, unlike the rest of the element which continues running at 3-5 FPS. This is leading me to believe that the issue here is hardware acceleration. Can I enable it for the videoTag somehow?
Approach 2
var media = new MediaSource();
videoTag.src = URL.createObjectURL(media);
// guessing mimetype from a few WebRTC tutorials I stumbled upon
var srcBuf = media.addSourceBuffer(‘video/webm;codecs=”vp8, vorbis”’);
// need to convert webrtc.localStreams[0] or its video track to a buffer somehow???
srcBuf.appendBuffer(/* buffer */);
FURTHER RESEARCH
This may be a bug in Chrome, a hackerish workaround that seems to work is to make sure the newly generated video elements are completely overlapped by the original video element (even if the original video element is set to render on the background behind all other elements (and behind a non-transparent background). This seems to kick in hardware acceleration.
You can use MediaSource, sourceopen event, .addSourceBuffer(), .appendBuffer(). See HTML5 audio streaming: precisely measure latency?, Unable to stream video over a websocket to Firefox
Related
In my web app I obtain a MediaStream either via getUserMedia or getDisplayMedia. In certain situations, the video track of that stream can change its size. For example, if getDisplayMedia tracks a specific window, that window can change size. Or on a mobile device, when switching from landscape into portrait mode, the camera image will be rotated by 90° (i.e. width and height get swapped).
I know how to get the dimensions of the MediaStream (as described here). I now want to get notified whenever the dimensions change. A callback/event of some sorts.
I already tried MediaStream.onaddtrack (guessing that maybe the track is removed and readded with a different size) and I also tried using a MutationObserver on the <video> element I am showing the stream in. None of these worked. I also checked MDN for other events that might help me, but I didn't find any promising ones.
Is there a way to subscribe to changes of a video track's dimension? A function of mine should be called each time the dimensions change. And if it's not possible and I would need to busy poll: is there a smart way how to make the polling a bit less busy?
You can subscribe to resize event of your video element
const video = document.getElementById("video");
video.addEventListener("resize", (e) => {
const { videoWidth, videoHeight } = video;
// can do smth here
// for example
video.style.width = videoWidth;
video.style.height = videoHeight;
}, false);
I'm building a photo application using electron that loads user photos from the file system. These photos can easily be 7MB or more in size. The application allows the user to switch between photo's using the keyboard arrows, at which point I want the new photo to display extremely fast.
For a 7MB image, just changing the src of an existing image tag in the DOM can take ~200-300ms, webkit must load the file, decode the file, and render the file on the page. The loading and decoding take 100-150ms each. (actually the profiler just says 2 x decoding, but the next step removes one of those decodes, so I presume it's related to the file read).
Preloading an img tag...
var img = new Image();
img.src = "/path/to/photo.jpg"
...means that webkit preloads the file, and this strips the file load time, but there is still a 100-150ms delay in appending like this...
domElement.appendChild(img);
...because the read data must still be decoded for the item to be appended to the DOM.
Is there a way to pre-decode the image, so that appending to the DOM does not have a 100-150ms delay, and only the fast rendering is required?
No you cannot "pre-decode". However, you can pre-append the img in an effectively invisible way by applying the style width: 1px; height: 1px; opacity: 0.01, and webkit won't redo the work if you append again.
You can even remove the image in the mean time, provided it has had time to fully decode, and webkit will hold on to the decoded data (although I'm not sure for how long).
If you want to be absolutely certain it will load fast, you could do one of two things. Reveal the img tag by removing the styles above, or by loading the same img tag in a different part of the DOM, while leaving the 'pre-appended' one in place. This will take between 3ms and 20ms in my experience.
BE CAREFUL regarding cleanup if you are using a lot of user defined photo contents.
In my experience, simply removing numerous img elements from the DOM will cause memory leaks (and electron windows to crash). I would advise that you either set the img.src to null after removing the image from the DOM, or set the entire img to null if you no longer need the Image instance.
You could play with the following code (use images of your own) using the chrome devtools timeline to measure the render speeds of photos in different scenarios.
<style>
/*#preload img {
width: 1px;
height: 1px;
opacity: 0.01;
}*/
</style>
<button>Toggle Image</button>
<div id="container"></div>
<div id="preload"></div>
<script>
"use strict"
const button = document.getElementsByTagName('button')[0]
, container = document.getElementById('container')
, preload = document.getElementById('preload')
, img1 = new Image()
, img2 = new Image()
var current = img2
img1.src = './img1.JPG'
img2.src = './img2.JPG'
preload.appendChild(img2)
setTimeout(function(){
preload.removeChild(preload.childNodes[0])
}, 200)
button.addEventListener('click', function(e){
toggleImg()
})
function toggleImg(){
if (current === img1) {
setImg(img2);
} else {
setImg(img1)
}
}
function setImg(img){
if (container.hasChildNodes()) container.removeChild(container.childNodes[0])
container.appendChild(img)
current = img
}
</script>
I'd suggest experimenting with using the HTML5 Canvas to render your images, that way you can load the next image ahead of time and have more direct control over the caching strategy.
I am having an issue with Video streaming on vlc plugin in Windows safari. I have added windowless="true" attribute in embed tag so that I can display transparent DIV above vlc plug in for Drawing on video. After adding this tag video is stopped and video frame is not received. But When I click on video or draw something on video, video frame refreshed for a while. Even if when I remove windowless="true" attribute video works.
Issue is observed in windows safari only with windowless="true".
I have the exactly same behavior. Windows 10, Safari 5.1.7.
So far the only solution I found is a workaround to force browser updating the player's frame. I do it by quickly adding and removing a "glass" div: a region that occupies whole browser's screen and has an opaque background fill color. Since it's transparent there is no any visible changes on the screen but Safari always redraws overlapped regions even if they are overlapped by a transparent pane. Here is what I do:
function play() {
// ...create player, set properties, etc.
// assuming player taking whole browser's screen.
// It's a glass pane - it should cover whole player's output surface.
var glass = $('<div>');
$('body').append(glass);
// Here we start updating...
setInterval(function() {
$('body').toggleClass('glass');
}, 20); // <== 1000ms / 30fps = 33ms, put 20ms just in case.
}
.glass {
position: absolute;
width: 100%;
height: 100%;
z-index: 10000;
background-color: rgba(40,1,1,0);
}
Of course, you need to adjust z-index so what glass is overlapping player surface but not blocking your custom controls.
Yes, it's an ugly hack but it works as a short term solution.
I'm using Appcelerator and wanted to do some video processing. I came across Seriously.js and saw that you could potentially do some impressive image and video stream manipulation in a "node" pipeline. So before taking on the appcelerator part of this effort, I figured I'd coerce the camera-source example (see: http://brianchirls.github.io/Seriously.js/examples) into doing more than just edge detection. So I quickly added a pixelation effect on top of that. Code looked like this:
<!DOCTYPE html>
<html>
<head>
<title>Seriously.js Camera Demo</title>
</head>
<body>
<canvas id="target" width="640" height="480"></canvas>
<script src="../../seriously.js"></script>
<script src="../../sources/seriously.camera.js"></script>
<script src="../../effects/seriously.edge.js"></script>
<script src="../../effects/seriously.pixelate.js"></script>
<script>
(function() {
//main code goes here
// declare our variables
var seriously, // the main object that holds the entire composition
source, // wrapper object for source video
edge, // edge detection effect
pixelate, // pixelate effect
target; // a wrapper object for our target canvas
if (Seriously.incompatible('camera')) {
document.body.appendChild(document.createTextNode('Sorry, your browser does not support getUserMedia'));
document.querySelector('canvas').style.display = 'none';
return;
}
// construct our seriously object
seriously = new Seriously();
// time to get serious
source = seriously.source('camera');
target = seriously.target('#target');
edge = seriously.effect('edge');
pixelate = seriously.effect('pixelate');
// connect all our nodes in the right order
edge.source = source;
pixelate.source = edge;
target.source = pixelate;
seriously.go();
}());
</script>
</body>
</html>
And cool it worked. But what I really wanted to do is use the blend effect (specifically difference). This takes a top and bottom for two different sources (images or videos, I assume) and performs the specified blend operation between corresponding frames. But what I really want is to have one video stream operated on and have the difference blend effect performed between frames. The closest I could get, which really isn't very close is to use the same video stream as both the top source and bottom source. Of course, there's no difference between them, so I don't really get what I'm after. So I'm guessing I need to access the previous frame, but I don't know how given the operation I see in the API. Here's the code I have:
<!DOCTYPE html>
<html>
<head>
<title>Seriously.js Camera Demo</title>
</head>
<body>
<canvas id="target" width="640" height="480"></canvas>
<script src="../../seriously.js"></script>
<script src="../../sources/seriously.camera.js"></script>
<script src="../../effects/seriously.edge.js"></script>
<script src="../../effects/seriously.blend.js"></script>
<script>
(function() {
//main code goes here
// declare our variables
var seriously, // the main object that holds the entire composition
source, // wrapper object for source video
edge, // edge detection effect
difference, // difference effect
target; // a wrapper object for our target canvas
if (Seriously.incompatible('camera')) {
document.body.appendChild(document.createTextNode('Sorry, your browser does not support getUserMedia'));
document.querySelector('canvas').style.display = 'none';
return;
}
// construct our seriously object
seriously = new Seriously();
// time to get serious
source = seriously.source('camera');
target = seriously.target('#target');
edge = seriously.effect('edge');
difference = seriously.effect('blend', { mode: "difference" } );
// connect all our nodes in the right order
edge.source = source;
difference.top = edge;
difference.bottom = edge; // I really want a frame sooner or later from edge
target.source = difference;
seriously.go();
}());
</script>
</body>
</html>
I look forward to a response.
Seriously.js doesn't do much in the way of manipulating image frames in time, at least not in the core code, since it's designed for processing live video, and storing frames could potentially take up a lot of memory.
However, there is a "freeze" effect plugin that could help. A freeze node has a "frozen" setting that causes it to stop updating, and you can use it to process an older frame. What you'd do is set up two freeze nodes, each taking input from your camera, and alternate which of the two nodes is frozen every time you render a frame. You'd also alternate the inputs of your blend node so the "bottom" input always receives the old frame (the "frozen" node) and the top receives the current frame (the unfrozen node).
It's best to set the bottom and top inputs on the blend node to "select" nodes, which will allow you to swap between the two different freeze nodes without disconnecting and re-connecting nodes on your node graph. This way you avoid any costly operations that sometimes happen when you change the network around. And you can do the swap in a callback to the ".go()" method, which runs before every frame render.
Here's a link to a working example:
https://jsbin.com/hisuha/edit?js
I didn't use an edge filter here because it didn't seem necessary, though you're welcome to give it a shot. I'd try putting it right after the camera node and have both freeze nodes use your edge node as the input. It's also worth noting that this is not quite the same as an optical flow effect, which I'm working on.
I looked around internet and was always looking like from previous 6 months for a script that could load flash content/game while showing an actual loading screen But I always received a few answers:
It is not possible to show an actual loading with Javascript.
You can do it only by adding action script to the flash file maybe they are talking about FLA
Why Don't you show a fake loading screen that appears and show some
seconds and then disappears (the most annoying this of such screen
is that they first make user load 15 seconds then the flash starts
loading, if it starts loading those 15 seconds still it is worth
something it is good BUT making them wait double is really bad)
But at last I found something that I was looking forever. A Jquery based script that shows actual loading (shows ad too) and uses swf Object to talk to flash content too. It is really awesome as it doesn't require you to do changes to the FLA, it is just pure outer environment dealing. So now the question arises what's the issue then. Well the issue is that this script was made for pixels, it works if you are using width and height for flash in pixels, while I can't use pixels as I am using %ages (this way user have ability to go full screen optionally by pressing f11).
So as you can see I want that script to work with %ages that is my problem, but as I mentioned earlier I didn't came here right away I have been asking for help (Actually Begging) in over 14 forums from previous few months and of course some good people still exists some people helped me to reach a certain point (but it didn't solve the problem) So now I will provide some Markup:
Here is link to the script that I am talking about http://www.balloontowerdefense.net/jquery-preloader/jquery-preloader.html (It is the link to the creator of this script)
Here is a link to working example (flash based on Pixels) http://www.balloontowerdefense.net/jquery-preloader/example.html
Some one helped me here but it didn't work 1 month ago. The person told me that I should change the plugin Named as Preroll the changes preferred were these
Modify the plugin to use user-supplied units instead of pixels. To do this, you will need to modify two functions in the plugin, applygameiframe and showgame.
applygameiframe should be changed to:
var applygameiframe = function() {
var gc = '#'+settings.gameframe;
var iframe = $('<iframe />').attr({
"id": settings.gameid,
"src": settings.swf,
"frameborder": 0,
"scrolling": "no",
"marginwidth": 0,
"marginheight": 0
}).css({
"height":'settings.height
"width": settings.width
});
$(gc).append(iframe);
return true;
};
showgame should be changed to:
var showgame = function() {
var ac = '#' + settings.adframe;
var game = '#' + settings.gameframe;
$(ac).hide();
$(game).css({
"width": settings.width,
"height": settings.height
});
};
Once those changes are made, the inline CSS should be set to whatever you supply as parameters (i.e., 100%, 50em, etc.).
I did the changes told to be done as described above to the Preroll plugin and after that this is what I get http://files.cryoffalcon.com/MyFootPrint/fullscreen.html
Now if you let the game load (as loading screen appears) all is well done except that in the end, the game doesn't appear, it loads but when it should skip and make the game appear at that time something goes wrong. (For reference you can see this link http://www.balloontowerdefense.net/jquery-preloader/example.html here when the loading finishes then game appears)
Can Someone Fix this problem?
Note: Sorry for not providing JsFiddle but as I live in Afghanistan with 5KBps speed it is not possible for me.
I didn't provided the HTML, CSS and JS that makes up the whole demo page as I thought it will make the question very long but still if you think I should provide Please let me know in comments.
I tried my best to make the question more relevant with Relevant Markups BUT still If I am missing something I would try my best by editing it and providing it you again.
Being an accountant, I tried my best to use programmers terms, coding is my passion but I am still in learning stage of JS
UPDATE: After solving the problem here you can see now everything is fine. http://files.cryoffalcon.com/MyFootPrint/newfullscreen.html
Credit: Goes to the one who answered this question.
This seems to be just a pure css problem. You're trying to set the width to 100% while the parent of div.gamewrapper has no width or height. That's why the size is 0 and it will not show up.
The trick you need to apply is add the following to your style:
html, body, .gamecontent {
height: 100%;
width: 100%;
}
Update:
Also, remove float: left; from .gamecontent .game, and add a width and height of 1px such that it becomes:
.gamecontent .game {
margin:0px auto 0px auto;
padding:0px;
overflow:hidden;
width : 1px;
height : 1px;
}
Well, after an hour and a half of playing Bloons on your link (my untouched work load can verify that), I feel it's safe to say that the full screen features work exactly as I'd expect them to. I'm using Chrome 18.0.x.
My experience was: Click link, game loads. The loader took about 2 seconds longer to finish then it took for the "Click Here to Show the Game" button appeared. After, an ad appeared for 10seconds and then I clicked "Play" and it went right to the game. Full screen worked correctly to my knowledge, although when I left full screen the game didn't resize back down - the bottom section was cut off.
I know that doesn't answer your question, but perhaps the issue is only in certain browsers?
i found that in FF the problem seems to be the height and width of 100%. I changed the script slightly to:
$(game).css({
"width": window.innerWidth,
"height": window.innerHeight
});
and now the game shows correctly in FF.