In my web app I obtain a MediaStream either via getUserMedia or getDisplayMedia. In certain situations, the video track of that stream can change its size. For example, if getDisplayMedia tracks a specific window, that window can change size. Or on a mobile device, when switching from landscape into portrait mode, the camera image will be rotated by 90° (i.e. width and height get swapped).
I know how to get the dimensions of the MediaStream (as described here). I now want to get notified whenever the dimensions change. A callback/event of some sorts.
I already tried MediaStream.onaddtrack (guessing that maybe the track is removed and readded with a different size) and I also tried using a MutationObserver on the <video> element I am showing the stream in. None of these worked. I also checked MDN for other events that might help me, but I didn't find any promising ones.
Is there a way to subscribe to changes of a video track's dimension? A function of mine should be called each time the dimensions change. And if it's not possible and I would need to busy poll: is there a smart way how to make the polling a bit less busy?
You can subscribe to resize event of your video element
const video = document.getElementById("video");
video.addEventListener("resize", (e) => {
const { videoWidth, videoHeight } = video;
// can do smth here
// for example
video.style.width = videoWidth;
video.style.height = videoHeight;
}, false);
I am trying out Houdini's CSS Paint API on Chrome 70 and I can't this example to work:
https://github.com/w3c/css-houdini-drafts/blob/master/css-paint-api/EXPLAINER.md#drawing-images
I am passing an Image via custom property like this:
--my-image: url("bridge.jpg");
and I use the CSS Properties and Values API Level (enabled via the Experimental Web Platform features flag) to register my image property:
CSS.registerProperty({
name: '--my-image',
syntax: '<image> | none',
initialValue: 'none',
inherits: false,
});
Then I am trying to draw it in my worklet:
class MyWorklet {
static get inputProperties() {
return ['--my-image'];
}
paint(ctx, geom, properties) {
const img = properties.get('--my-image');
console.log(img);
ctx.drawImage(img,10,10,150,180);
}
}
registerPaint('myworklet', MyWorklet);
for displaying the worklet I made a simple 500 * 500 div and added the style:
background-image: paint(myworklet);
The Console tells me that img is a CSSImageValue,
so it seems to work in some way but the image does not get drawn onto the background of the div.
Has anyone managed to get this to work or are some functionalities which are required for this kind of stuff not yet implemented?
Image is not implemented in Chrome till now. Should be available in the latest Safari Technical Preview (v72). But it requires inline code, see https://vitaliy-bobrov.github.io/blog/trying-paint-wroklet-in-safari-tp/
I have this code below:
<script type="text/javascript">
jwplayer("container").setup({
flashplayer: "http://test.captive-portal.com/jwplayer/player.swf",
file: "http://content.captive-portal.com/files/video/cirque-du-soleil/mob.mp4",
image: "http://content.captive-portal.com/files/video/cirque-du-soleil/mob.jpg",
height: 285,
width: 480,
});
</script>
The whole page is here (in case you need it): video page.
What I'm trying to do is to change the 4 lines: file: (...), image (...), height(...) and width (...) depending of the resolution of the window. I have managed to do this with css, so styles are applied correctly, but this is javascript and I don't really know how to modify it. I was trying to place 2 similar scripts on one page in divs and hide the small one in case of big screen or hide a big one in case of small screen but it didn't work. Video didn't play. I think this may be caused by some scripts in the flash player file.
Is there any way to set conditions for those 4 lines depending of the screen resolution?
Thank you very much for your help in advance.
Those parameters are simply an object.
So you should be able to create that object out of the jwplayer call, and simply pass it in. e.g
var params = {};
params.flashplayer = "http://test.captive-portal.com/jwplayer/player.swf";
if(!mobile){ //you need to handle checking for mobile devices
params.file = "http://content.captive-portal.com/files/video/cirque-du-soleil/desktop.mp4";
params.image = "http://content.captive-portal.com/files/video/cirque-du-soleil/desktop.jpg";
params.height = 570;
params.width = 960;
}else{
//set params in same way but with mobile settings
}
jwplayer("container").setup(params);
If you later want to just resize the player. JwPlayer has a 'resize' function that you can call: jwplayer.resize(width,height)
Is there a way to force a high quality THUMBNAIL for YouTube?
My videos are of very high quality and once they start streaming they run fine in 720p, however the thumbnail for the video is of variable quality - sometimes it's high, other times it's really blurry.
Is there a way of forcing a high quality thumbnail? I've found this in the API docs - http://code.google.com/apis/youtube/2.0/reference.html#youtube%5Fdata%5Fapi%5Ftag%5Fmedia:thumbnail but it doesn't detail how to use the media tag.
There is a "maxres" version as well, which is a "full hd" picture in case that the video resolution is high enough.
http://i.ytimg.com/vi/VIDEO_ID/maxresdefault.jpg
However, if the video resolution isn't high enough, this image doesn't seem to be created. So you might want to have a function that shows a lower quality version in case the "maxres" version doesn't exist.
Check out How do I get a YouTube video thumbnail from the YouTube API? for more info.
Thanks, Johan for answer.
http://i.ytimg.com/vi/VIDEO_ID/maxresdefault.jpg
If you also want set playback to HQ use Javascript API
https://developers.google.com/youtube/js_api_reference?hl=fi-FI#Playback_quality
var qualityLevels = self.players[iframeID].getAvailableQualityLevels();
/* Set highest quality */
self.players[iframeID].setPlaybackQuality(qualityLevels[0]);
In order for this to work we first need to call:
self.players[iframeID].playVideo();
Then it will triggrer
self.players[iframeID] = new YT.Player(iframeID, {
events: {
'onReady': function(event) {
var iframe = $(event.target.getIframe());
self.players[iframe.attr('id')].loaded = true;
},
'onStateChange': function(event) {
var iframe = $(event.target.getIframe());
var iframeID = iframe.attr('id');
if(event.data == 1) /* playing */
{
var qualityLevels = self.players[iframeID].getAvailableQualityLevels();
if(self.players[iframeID].getPlaybackQuality() != qualityLevels[0])
{
/* Set highest quality */
self.players[iframeID].setPlaybackQuality(qualityLevels[0]);
}
}
}
}
});
Example:
http://img.youtube.com/vi/xjFouf6j81g/hqdefault.jpg
where xjFouf6j81g, videoid
As far as I know, YouTube only offers 4 thumbnails:
http://i.ytimg.com/vi/VIDEO_ID/default.jpg
http://i.ytimg.com/vi/VIDEO_ID/1.jpg
http://i.ytimg.com/vi/VIDEO_ID/2.jpg
http://i.ytimg.com/vi/VIDEO_ID/3.jpg
The thumbnails are generated once after the upload, so you can't alter the size of them in any way.
Edit:
YouTube JS API lets you force the quality of the video, but I don't know if it will affect the video preview. See the docs for more info.
Issue has been reported here:
https://code.google.com/p/gdata-issues/issues/detail?can=2&start=0&num=100&q=&colspec=API%20ID%20Type%20Status%20Priority%20Stars%20Summary&groupby=&sort=&id=4590
Not a fix or a workaround, but if you want to avoid getting the low-res thumbnail your player should be larger than 430px wide.
So a suggested minimum size for perfect 16:9 aspect would be 432 x 243 pixels and this should get a nice-looking thumbnail.
I'm trying to build an image gallery in Safari that mimics the iPad photo app. It works perfectly, except that once I load more than 6MB or so worth of images either by adding them to the DOM or creating new Image objects, new images either stop loading or the browser crashes. This problem is widespread enough (with everyone else hitting up against the same limit) that I've ruled out my Javascript code as the culprit.
Given that you can stream much more than a few MB in a element or through the in-browser media player, this limit seems unnecessary, and there should be some kind of workaround available. Perhaps by freeing up memory or something else.
I also came across this reference for UIWebView.
"JavaScript allocations are also limited to 10 MB. Safari raises an exception if you exceed this limit on the total memory allocation for JavaScript."
Which matches what I'm seeing fairly well. Is it possible to deallocate objects in Javascript, or does Safari/UIWebView keep a running total and never lets go? Alternately, is there any workaround to load in data another way that doesn't eat up this 10MB?
Update: I think there's an even easier way to do this, depending on your application. Instead of having multiple images, if you simply have one <img> element or Image object (or maybe two, like a 'this' image and a 'next' image if you need animations or transitions) and simply update the .src, .width, .height and so on, you should never get near the 10MB limit. If you wanted to do a carousel application, you'd have to use smaller placeholders first. You might find this technique might be easier to implement.
I think I may actually have found a work-around to this.
Basically, you'll need to do some deeper image management and explicitly shrink any image you don't need. You'd normally do this by using document.removeChild(divMyImageContainer) or $("myimagecontainer").empty() or what have you, but on Mobile Safari this does absolutely nothing; the browser simply never deallocates the memory.
Instead, you need to update the image itself so it takes up very little memory; and you can do that by changing the image's src attribute. The quickest way I know of to do that is to use a data URL. So instead of saying this:
myImage.src="/path/to/image.png"
...say this instead:
myImage.src="data:image/gif;base64,AN_ENCODED_IMAGE_DATA_STRING"
Below is a test to demonstrate it working. In my tests, my large 750KB image would eventually kill the browser and halt all JS exectution. But after resetting src, I"ve been able to load in instances of the image over 170 times. An explanation of how the code works is below as well.
var strImagePath = "http://path/to/your/gigantic/image.jpg";
var arrImages = [];
var imgActiveImage = null
var strNullImage = "data:image/gif;base64,R0lGODlhEAAOALMAAOazToeHh0tLS/7LZv/0jvb29t/f3//Ub//ge8WSLf/rhf/3kdbW1mxsbP//mf///yH5BAAAAAAALAAAAAAQAA4AAARe8L1Ekyky67QZ1hLnjM5UUde0ECwLJoExKcppV0aCcGCmTIHEIUEqjgaORCMxIC6e0CcguWw6aFjsVMkkIr7g77ZKPJjPZqIyd7sJAgVGoEGv2xsBxqNgYPj/gAwXEQA7";
var intTimesViewed = 1;
var divCounter = document.createElement('h1');
document.body.appendChild(divCounter);
var shrinkImages = function() {
var imgStoredImage;
for (var i = arrImages.length - 1; i >= 0; i--) {
imgStoredImage = arrImages[i];
if (imgStoredImage !== imgActiveImage) {
imgStoredImage.src = strNullImage;
}
}
};
var waitAndReload = function() {
this.onload = null;
setTimeout(loadNextImage,2500);
};
var loadNextImage = function() {
var imgImage = new Image();
imgImage.onload = waitAndReload;
document.body.appendChild(imgImage);
imgImage.src = strImagePath + "?" + (Math.random() * 9007199254740992);
imgActiveImage = imgImage;
shrinkImages()
arrImages.push(imgImage);
divCounter.innerHTML = intTimesViewed++;
};
loadNextImage()
This code was written to test my solution, so you'll have to figure out how to apply it to your own code. The code comes in three parts, which I will explain below, but the only really important part is imgStoredImage.src = strNullImage;
loadNextImage() simply loads a new image and calls shrinkImages(). It also assigns an onload event which is used to begin the process of loading another image (bug: I should be clearing this event later, but I'm not).
waitAndReload() is only here to allow the image time to show up on the screen. Mobile Safari is pretty slow and displaying big images, so it needs time after the image has loaded to paint the screen.
shrinkImages() goes through all previously loaded images (except the active one) and changes the .src to the dataurl address.
I'm using a file-folder image for the dataurl here (it was the first dataurl image I could find). I'm using it simply so you can see the script working. You'll probably want to use a transparent gif instead, so use this data url string instead: data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==
The 6.5MB(iPad) / 10MB(iPhone) download limits are calculated based on the number of image elements used to set an image through its src property. Mobile safari doesn't seem to differentiate images loaded from cache or via the network. It also doesn't matter whether the image is injected into the dom or not.
The second part to the solution is that mobile safari seems to be able to load an unlimited number of images via the "background-image" css property.
This proof of concept uses a pool of precacher's which set the background-image properties once successfully downloaded. I know that it's not optimal and doesn't return the used Image downloader to the pool but i'm sure you get the idea :)
The idea is adapted from Rob Laplaca's original canvas workaround http://roblaplaca.com/blog/2010/05/05/ipad-safari-image-limit-workaround/
<!DOCTYPE html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>iPad maximum number of images test</title>
<script type="text/javascript">
var precache = [
new Image(),
new Image(),
new Image(),
new Image()
];
function setImage(precache, item, waiting) {
precache.onload = function () {
item.img.style.backgroundImage = 'url(' + item.url + ')';
if (waiting.length > 0) {
setImage(precache, waiting.shift(), waiting);
}
};
precache.src = item.url;
}
window.onload = function () {
var total = 50,
url = 'http://www.roblaplaca.com/examples/ipadImageLoading/1500.jpg',
queue = [],
versionUrl,
imageSize = 0.5,
mb,
img;
for (var i = 0; i < total; i++) {
mb = document.createElement('div');
mb.innerHTML = ((i + 1) * imageSize) + 'mb';
mb.style.fontSize = '2em';
mb.style.fontWeight = 'bold';
img = new Image();
img.width = 1000;
img.height = 730;
img.style.width = '1000px';
img.style.height = '730px';
img.style.display = 'block';
document.body.appendChild(mb);
document.body.appendChild(img);
queue.push({
img: img,
url: url + '?ver=' + (i + +new Date())
});
}
//
for (var p = 0; p < precache.length; p++) {
if (queue.length > 0) {
setImage(precache[p], queue.shift(), queue);
}
}
};
</script>
</head>
<body>
<p>Loading (roughly half MB) images with the <strong>img tag</strong></p>
</body>
</html>
So far I've had luck using <div> tags instead of <img> tags and setting the image as the div's background-image.
All in all, it's crazy. If the user is making an affirmative request for more image content, then there's no reason why Safari shouldn't allow you to load it.
I've had luck starting with the suggestion of Steve Simitzis, and Andrew.
My project:
PhoneGap-based app with 6 main sections, and about 45 subsections which have a jquery cycle gallery of between 2 and 7 images, each 640 x 440 (215+ images altogether). At first I was using ajax to load page fragments, but I've since switched to a one-page site, with all sections hidden until needed.
Initially, after going through about 20 galleries, I was getting memory warning 1, then 2, then the crash.
After making all the images into divs with the image applied as a background, I could get through more galleries (about 35) in the app before a crash, but after going to previously visited galleries, it would eventually fail.
The solution that seems to be working for me, is to store the background image URL in the div's title attribute, and setting all of the background images to be a blank gif. With 215+ images, I wanted to keep the url someplace in the html for sake of ease and quick reference.
When a subnavigation button is pressed, I rewrite the css background image to the correct source which is contained in the div's title tag, for ONLY the gallery that is showing. This saved me from having to do any fancy javascript to store the correct source image.
var newUrl = $(this).attr('title');
$(this).css('background-image', 'url('+newUrl+')');
When a new subnavigation button is pressed, I rewrite the background image of the last gallery divs to be blank gifs. So, aside from interface gfx, I only have 2-7 images 'active' at all times. With anything else I add that contains images, I just use this "ondemand" technique to swap the title with the background-image.
Now it seems I can use the app indefinitely with no crashes. Don't know if this will help anyone else, and it may not be the most elegant solution, but it provided a fix for me.
On a rails app, I was lazy loading hundreds of mid-size photos (infinite scroll) and inevitably hit the 10Mb limit on the iphone. I tried loading the graphics into a canvas (new Image, src=, then Image.onload) but still hit the same limit. I also tried replacing the img src and removing it (when it went out of viewable area) but still no cigar. In the end, switching out all the img tags w/ div's w/ the photo as background did the trick.
$.ajax({
url:"/listings/"+id+"/big",
async:true,
cache:true,
success:function(data, textStatus, XMLHttpRequest) {
// detect iOS
if (navigator.userAgent.match(/iPhone/i) || navigator.userAgent.match(/iPod/i) || navigator.userAgent.match(/iPad/i)) {
// load html into data
data = $(data);
// replace img w/ div w/ css bg
data.find(".images img").each(function() {
var src = $(this).attr("src").replace(/\s/g,"%20");
var div = $("<div>");
div.css({width:"432px",height:"288px",background:"transparent url("+src+") no-repeat"});
$(this).parent().append(div);
$(this).remove();
});
// remove graphic w/ dynamic dimensions
data.find(".logo").remove();
}
// append element to the page
page.append(data);
}
});
I can now load well over 40Mb of photos on one page w/o hitting the wall. I encountered an odd issue, though, with some of the css background graphics failing to show up. A quick js thread fixed that. Set the div's css bg property every 3 sec's.
setInterval(function() {
$(".big_box .images div.img").each(function() {
$(this).css({background:$(this).css("background")});
});
}, 3000);
You can see this in action at http://fotodeck.com. Check it out on your iphone/ipad.
I was unable to find a solution for this. Here are a couple of methods I tried, and all of them failed:
Simply changed the background of a DIV using div.style.backgroundImage = "url("+base64+")"
Changed the .src of an image using img.src = base64
Removed the old and added the new image using removeChild( document.getElementById("img") ); document.body.appendChild( newImg )
The same as above but with a random height on the new image
Removing and adding the image as a HTML5 canvas object. Also doesn't work, since a new Image(); has to be created, see *
On launch, created a new Image() object, let's call it container. Displayed the image as <canvas>, every time the image changed, I would change container's .src and redraw the canvas using ctx.drawImage( container, 0,0 ).
The sames as the previous, but without actually redrawing the canvas. Simply changing the Image() object's src uses up memory.
A strange thing I noticed: The bug occurs even if the image isn't displayed! For example, when doing this:
var newImg = new Image( 1024, 750 );
newImg.src = newString; // A long base64 string
Every 5 seconds, and nothing else, no loading or displaying the image, of course wrapped up in an object, also crashes the memory after some time!
I encountered an out of memory with Javascript on the iPad when we were trying to refresh an image very often, like every couple of seconds. It was a bug to refresh that often, but Safari crashed out to the home screen. Once I got the refresh timing under control, the web app functioned fine. It seemed as if the Javascript engine couldn't keep up with garbage collection quickly enough to discard all the old images.
There are issues with memory and the way to solve this problem is very simple. 1) Put all your thumbnails in canvas. You will be creating a lot of new Image objects and drawing them into canvas, but if your thumbnail are very small you should be fine. For the container where you will be displaying the real size image, create only one Image object and reuse this object and make sure to also draw it into a canvas. So, every time a user clicks the thumbnail, you will update your main Image object. Do not insert IMG tags in the page. Insert CANVAS tags instead with the correct width and height of the thumbnails and the main display container. iPad will cry foul if you insert too many IMG tags. So, avoid them!!! Insert only canvas. You can then find the canvas object from the page and get the context. So every time the user clicks a thumbnail, you will get the src of the main image (real size image) and draw it to the main canvas, reusing the main Image object and the firing the events. Clearing the events every time at the beginning.
mainDisplayImage.onload = null;
mainDisplayImage.onerror = null;
...
mainDisplayImage.onload = function() { ... Draw it to main canvas }
mainDisplayImage.onerror = function() { ... Draw the error.gif to main canvas }
mainDisplayImage.src = imgsrc_string_url;
I have create 200 thumbnails and each is like 15kb. The real images are like 1 MB each.
I also had similar problems while rendering large lists of images on iPhones.
In my case displaying even 50 images in the list was enough to either crash the browser or occasionally the entire operating system. For some reason any images rendered onto the page weren't garbage collected, even when pooling and recycling just a few onscreen DOM elements or using the images as background-image property. Even displaying the images directly as Data-URIs is enough to count towards the limit.
The solution ended up being rather simple - using position: absolute on the list items allows them to be garbage collected fast enough to not run into a memory limit. This still involved on having only about 20-30 images in the DOM at any moment, creating and removing the item's DOM nodes by scroll positon finally did the trick.
It seems it's particularily dependent on having webkit-transform':'scale3d() applied to any ancestor of the images in the DOM. Relatively flowing a very tall DOM and rendering it on the GPU pisses off a memory leak in webkit renderer, I guess?
I'm running in a similar issue in Chrome too, developing an extension that loads images in the same page (the popup, actually) replacing old images with new ones.
The memory used by the old images (removed from the DOM) is never freed, consuming all the PC memory in a short time.
Have tried various tricks with CSS, without success.
Using hardware with less memory than a PC, like the iPad, this problem arises earlier, naturally.
I filed a bug with jQuery as jQuery trys to handle memory leaks...so I'd consider this a bug. Hopefully the team can come up with some concise and clever way of handling this problem in Mobile Safari soon.
http://dev.jquery.com/ticket/6944#preview