WebRTC Webcam constraints don't adapt to device orientation - javascript

I am implementing a photo-taking application in html and javascript (actually Angular 1.x). The app is mostly used on Android tablets but also phones and Windows computers.
My Problem
when the user flips the tablet (portrait / landscape), the camera "flips" as well. Meaning, when you hold your tablet in landscape mode the camera is in landscape mode as well and when you flip the tablet, the camera is in portrait mode. But the system keeps the width and height parameters of the camera.
This wouldn't be much of a problem if I where to display the video only, but I need to copy the image, crop it, scale it etc. So I need to be sure that the camera's width is actually the width.
My implementation
To give you an idea, I tried to extract the code that is responsible for the camera:
// I have a static array with possible camera resolutions
private resolutions = [
{width: 320, height: 240},
{width: 600, height: 480}
];
// and the camera constraints that I initialize like this
this.constraints = {
audio: false, // no audio needed
video: {
facingMode: "environment", // I want the back camera
width: {
exact: 4096 // initial width
},
height: {
exact: 2160 // initial height
}
}
};
// I use that array to query the system for a camera that fits these resolution constraints (it's recursive, so I call this function with the last index of my resolutions-array)
testCameraResolution(resolutionIndex: number) {
if (this.checking) {
this.constraints.video.width.exact = this.resolutions[resolutionIndex].width; // overwrite the width value of the constraints
this.constraints.video.height.exact = this.resolutions[resolutionIndex].height; // overwrite the height value of the constraints
navigator.mediaDevices.getUserMedia(this.constraints).then((stream: MediaStream) => {
// when the navigator returns a camera with these constraints, I found my resolution and can stop testing.
this.checking = false;
for (let track of stream.getTracks()) {
track.stop(); // I stop the camera stream
}
this.videoResolution.width = this.constraints.video.width.exact;
this.videoResolution.height = this.constraints.video.height.exact;
this.startWebCam(); // and start the webcam
}).catch((error) => {
// no camera was found that matches these constraints, continue testing
if (resolutionIndex > 0) {
this.testCameraResolution(resolutionIndex - 1);
}
});
}
}
How it works
The page loads and my script will try to start the webcam with {width: 600, height: 480}, if the navigator cannot return a camera with these constraints, the script will continue and test {width: 320, height: 240}.
Let's say the navigator can return a camera with 320x240, then the page will finish loading and I can display and manipulate the image. Nice. Always taking it for granted, that the video's width is the number of pixels from "left to right" and it's height spans from "top" to "bottom".
But when I load the page on a tablet and I flip the tablet to "portrait" mode, the navigator still gives me the camera with {width: 320, height: 240}, although it displays it with width=240 and height=320 (in portrait mode). So now all my image manipulation doesn't work anymore because width and height are inversed.
My Solution
So I figured, when the page is in "portrait" mode (the browser window is higher than it is wide), then I just inverse width and height of the camera. And this actually works on a tablet - but of course it doesn't work on a desktop computer.
So here is my actual question
Is there a way to query the "navigator" if the whole device is in "portrait" or "landscape" mode without relying on the browsers width and height?

If you are sure it doesn't necessarily need to work on safari. You can use the Screen API.
https://developer.mozilla.org/en-US/docs/Web/API/Screen/orientation
In that page you have an example.

In short: There is not a solution for all devices.
Like #Nicolas Nobile pointed out, the Screen API is the most useful tool. However, Safari still does not support it so you can use the workaround as described here:
https://stackoverflow.com/a/36650956/10408987 (all Credit goes to him)
Screen API
screen.orientation.onchange = function (){
// logs 'portrait' or 'landscape'
console.log(screen.orientation.type.match(/\w+/)[0]);
};
Detect yourself (on safari):
function getOrientation(){
var orientation = window.innerWidth > window.innerHeight ? "Landscape" : "Portrait";
return orientation;
}
window.onresize = function(){ getOrientation(); }
Just noticed that you asked 2 years ago. I came across the same problem so I thought I answer on how I solved it.

Related

Scaling an iFrame depending on window size and content size

I have a dynamically generated iFrame on my page that loads a website using a variable object.
All that is well understood. My challenge now is that in some cases, say if I am viewing on mobile, the frame width exceeds my mobile device width.
// STATIC VALUE
let screenSize = {
"height": window.innerHeight,
"width" window.innerWidth:
}
// DYNAMICALLY GENERATED VARIABLE
let frameValue = {
"url": "https://example.com",
"height": 913,
"width": 1600
}
//Using this variable, the iframe property is set as follows using javascript
$('#dynamicFrame').attr('src', frameValue.url);
$('#dynamicFrame').width(frameValue.width);
$('#dynamicFrame').height(frameValue.height);
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<!-- HTML DYNAMIC iFRAME -->
<iframe src="" id="dynamicFrame" frameBorder="0" width="100%" height="100%" scrolling="auto"> </iframe>
Need:
I would like an algorithm (or perhaps some code) to perhaps scale or zoom the iframe whilst keeping its aspect ratio.
Meaning I want the content to of frameValue.url (example.com) to load in the iframe as it would while considering frameValue.width & frameValue.height.
Notes:
I don't mind having the iframe look smaller or have dark bands around the edge just like when you watch videos on a mobile device or use zoom or Microsoft teams on a mobile device whilst the person sharing the screen is on a desktop device.
Please feel free to comment if you need further explanation. Thank you.
Is there any reason you aren't using a CSS solution? It'd be a much cleaner solution than setting the height/width attributes.
#dynamicFrame {
/* Swap for your desired aspect ratio */
aspect-ratio 16/9;
width: 100%;
height: auto;
}
Depending on the screen size and orientation, one of the following formulas may be useful.
NB:The orientation can be determined using the dimensions provided. i.e landcape = width > height
let scale = 0;
// If the window is potrait and the frame is landscape orientation
scale = screenSize.height * (1 / frameValue.height);
// If the window is landscape and the frame is potrait, one of the following applies.
scale = dimensions.height/dimensions.width).toFixed(2);
// OR
scale = dimensions.width/dimensions.height).toFixed(2);
// Set the iFrame Dynamic Scale Value
iframe.style.transform = `scale(${scale})`;

How to react to changes in the dimension/size of a video track in a `MediaStream`?

In my web app I obtain a MediaStream either via getUserMedia or getDisplayMedia. In certain situations, the video track of that stream can change its size. For example, if getDisplayMedia tracks a specific window, that window can change size. Or on a mobile device, when switching from landscape into portrait mode, the camera image will be rotated by 90° (i.e. width and height get swapped).
I know how to get the dimensions of the MediaStream (as described here). I now want to get notified whenever the dimensions change. A callback/event of some sorts.
I already tried MediaStream.onaddtrack (guessing that maybe the track is removed and readded with a different size) and I also tried using a MutationObserver on the <video> element I am showing the stream in. None of these worked. I also checked MDN for other events that might help me, but I didn't find any promising ones.
Is there a way to subscribe to changes of a video track's dimension? A function of mine should be called each time the dimensions change. And if it's not possible and I would need to busy poll: is there a smart way how to make the polling a bit less busy?
You can subscribe to resize event of your video element
const video = document.getElementById("video");
video.addEventListener("resize", (e) => {
const { videoWidth, videoHeight } = video;
// can do smth here
// for example
video.style.width = videoWidth;
video.style.height = videoHeight;
}, false);

How to make images stretch to fill the available area in PhotoSwipe

I'm hoping to use PhotoSwipe to display all the images in each article on my responsive site, like the gallery you get when you click or tap on an in-content image in http://dailymail.co.uk. The main use is to give users a big view of pics and allow them to zoom in to them on mobile devices without leaving the article.
PhotoSwipe works very well for this purpose. My JS combs through the article content and builds the PhotoSwipe array from the images it finds. But there's one problem: some of the images in my articles are small. This isn't too bad on a mobile, but on desktop I'm liable to end up with a small image in the middle of a big black gallery area. The Daily Mail get around this problem by always stretching their images up to fit the gallery area - slightly ugly with some pics, but it gives a big view.
Is there any way I can selectively stretch small images in my PhotoSwipe gallery to fill more of the available area? An older version of PhotoSwipe used to have a imageScaleMethod option that would allow you to fit images to the screen size: is there anything similar I can do with the latest version, v4.0.7?
Here's a CodePen which should illustrate what I'm talking about - the third image is the sort of image I'd like to stretch up a bit. My slides in this example are:
var items = [
{
src: 'http://lorempixel.com/1000/750/cats/',
w: 1000,
h: 750,
title: 'Fairly big cat'
},
{
src: 'http://lorempixel.com/1200/900/cats/',
w: 1200,
h: 900,
title: 'Another quite big cat'
},
{
src: 'http://lorempixel.com/200/320/cats/',
w: 200,
h: 320,
title: 'Small cat'
}
];
At first I thought I could solve this by adding a beforeChange listener which zooms each image in to fit the display area as it comes in. Conveniently, each item in PhotoSwipe has a property fitRatio which represents the zoom ratio required to fit the display area.
On line 34 of my CodePen, just before the line that goes gallery.init(); I added:
gallery.listen('beforeChange', function () {
gallery.zoomTo(gallery.currItem.fitRatio, {x: gallery.viewportSize.x / 2, y: gallery.viewportSize.y / 2}, 1);
});
which zoomed the current pic to the right ratio to fit the gallery, centred on the slide's x and y centre, and over a duration of 1 microsecond. (See the PhotoSwipe API documentation for details about the zoomTo method.)
But zooming in deactivates the "click swipe" action on desktop, and I wanted to keep it.
So instead I'm measuring the browser window and setting the image width and height to match. You have to take account of whether the image is landscape vs portrait, and you need to update your dimensions if the user resizes the browser window.
var measureWindow = function () {
windowW = $window.width();
windowH = $window.height();
};
measureWindow();
// If image is landscape or square, set width to window width and height to width * ratio
// If image is portrait, set height to window height and width to height / ratio
var getImageDimensions = function (w, h) {
var ratio = h / w;
if (w >= h) {
return {
width : windowW,
height: windowW * ratio
}
} else {
return {
width : windowH / ratio,
height: windowH
}
}
};
Then as I add each of the images to my array I do
var dimensions = getImageDimensions(width, height),
o = {
el : value,
src: src,
w : dimensions.width,
h : dimensions.height
};
where o is the object representing the image.
I also add a listener to my gallery to call measureWindow again if the user resizes the window. This updates windowW and windowH.
var debouncedMeasure = _.debounce(measureWindow, 250);
gallery.listen('beforeResize', function () {
debouncedMeasure();
});
Strictly speaking I should update the dimensions in my images array when this happans, so new images are correctly sized to the new window dimensions.
Here's the solution I ended up with - you need to view the full preview to see how it works.
EDIT
To get the best results from PhotoSwipe, I've found I have to hide the content in the rest of the page whenever it's active. When I open it I do
html.addClass('photoswipe-active');
gallery.listen('close', function () {
html.removeClass('photoswipe-active');
});
and set visibility:hidden on the other content when that class is present. If you have anything in the page that causes a repaint (like an animated gif) it can cause a stutter in PhotoSwipe's swipe and pinch to zoom gestures. Hiding the content keeps it smooth.
I also had performance problems when I tried to measure the PhotoSwipe element on resize - it seems much better to measure the window instead.
I was using the wordpress plugin and added the following css.
(I then set the thumbnail size in wordpress to 0 0.)
.photoswipe_gallery_holder {
width: 101%;
margin-left: -5px;
}
.photoswipe_gallery {
width: 100%!important;
margin-top: 20px!important;
margin-bottom: 20px!important;
margin-right: 0px!important;
margin-left: 0px!important;
padding-bottom:0px!important;
}
.photoswipe_gallery figure {
width: 100%!important;
-webkit-box-sizing: border-box; /* Safari/Chrome, other WebKit */
-moz-box-sizing: border-box; /* Firefox, other Gecko */
box-sizing: border-box; /* Opera/IE 8+ */
padding: 5px!important;
}
.photoswipe_gallery figure img {
width: 100%!important;
height: auto!important;
padding: 0px!important;
}
.photoswipe_gallery figure a {
width: 100%!important;
height: auto!important;
}
#media screen and (min-width: 480px) {
.photoswipe_gallery figure {
width: 50%!important;
}
}
#media screen and (min-width: 750px) {
.photoswipe_gallery figure {
width: 25%!important;
}
}
The "holder" div is so we can shift things abit to line up. If you use this, you need to set overflow: hidden; on the parent div. My parent div was 968px wide at the widest on a responsive site.
If you look at the source code you can see it is/was a planned feature at some point, as it has parts that say:
// not fully implemented yet
scaleMode: 'fit' // TODO
This is indeed not much of a help, but I'm just saying this as an interesting detail.
My solution to the problem was this: simply set the data-size attribute to a size, that is bigger than most common screen sizes. As I'm generating the markup from PHP, I simply put something like this:
data-size="<?php echo intval($img_x * 1.5); ?>x<?php echo intval($img_y * 1.5); ?>"
The actual number required may vary in your case, but I had all my images already set to a fixed size and the 1.5 multipliers were enough to make them fill most screens.
It is not a "correct" solution, however, it is simple, quick and lightweight for the client (no additional javascript magic is needed). It may be a benefit for simple projects compared to the elaborate JavaScript heavy solution suggested above.
In fact, I'm using MobileDetect so I'm only increasing the image size for desktops, as my original images are suitable for mobiles (and tablets).
data-size="<?php echo intval($img_x * ($mobile_detect->isMobile() ? 1 : 1.5)); ?>x<?php echo intval($img_y * ($mobile_detect->isMobile() ? 1 : 1.5)); ?>"
You need to plan the sizings according to your own needs and you may need to use different multipliers for each image, but my point is that the size manipulation can be done easily in the gallery markup, instead of trying to override the values from JavaScript or to force zooming PhotoSwipe after certain events (which I've tried and worked more-or-less fine for switching the images, but it caused glitches when opening PhotoSwipe for example).

Orientation Portrait and PortraitUpSideDown Only For One Window

I have 10 windows.
The initial window is loginWindow i want to set orientation for Portrait and PortraitUpSideDown.
For remaining windows will have landscape and portrait orientation.
in Tiapp.xml
<key>UISupportedInterfaceOrientations~iphone</key>
<array>
<string>UIInterfaceOrientationPortrait</string>
<string>UIInterfaceOrientationPortraitUpsideDown</string>
<string>UIInterfaceOrientationLandscapeLeft</string>
<string>UIInterfaceOrientationLandscapeRight</string>
</array>
Which set all orientation for my application which enables portrait,portraitupsidedown,landscapeLeft and landscapeRight.
I need those only portrait and portraitUpSideDown for LoginWindow.
Rest of window do have all the orientation which is portrait,portraitupsidedown,landscapeLeft and landscapeRight.
Can any one suggest me how can i able to get this behaviours for my application.
You need to use different windows and define for each window which orientation you want to allow.
I mean, you have to create loginWindow like this:
var loginWindow = Ti.UI.createWindow({
orientationModes : [Ti.UI.PORTRAIT, Ti.UI.UPSIDE_PORTRAIT],
fullscreen : false,
navBarHidden : true
});
winPortrait.orientationModes = [Ti.UI.PORTRAIT, Ti.UI.UPSIDE_PORTRAIT];
Windows where you want to allow all orientations, have to been created like this:
var appWindow = Titanium.UI.createWindow({
width : Ti.UI.FILL,
height : Ti.UI.FILL,
fullscreen : false,
navBarHidden : true,
orientationModes : [Ti.UI.PORTRAIT, Ti.UI.UPSIDE_PORTRAIT, Ti.UI.LANDSCAPE_LEFT, Ti.UI.LANDSCAPE_RIGHT]
});
appWindow.orientationModes = [Ti.UI.PORTRAIT, Ti.UI.UPSIDE_PORTRAIT, Ti.UI.LANDSCAPE_LEFT, Ti.UI.LANDSCAPE_RIGHT];
Hope it helps
Present your loginWindow as a modal view and after that set this methods for desired orientations.
- (BOOL) shouldAutorotate
{
return NO;
}
- (NSUInteger) supportedInterfaceOrientations
{
return UIInterfaceOrientationMaskPortrait;
}
Using different orientation modes for a single app in iOS is not recommended. Please read Orientation design principles
Apple's Developer documentation says: "People expect to use your
app in different orientations, and it’s best when you can fulfill that
expectation." In other words, don't look at handling orientation as a
bother but an opportunity.
Apple further recommends that when choosing to lock or support
orientation, you should consider following these principles:
On iPhone/iPod Touch – Don't mix orientation of windows within a single app; so, either lock orientation for the whole app, or react to orientation changes.
On iPhone – don't support the portrait-upside-down orientation because that could leave the user with their phone upside-down when receiving a phone call.
However you can achieve orientation for particualr window using the orientationMode property of window

How to make window.innerHeight work on mobile devices when zooming?

window.innerHeight
Yes, it will return the value of the browser's height on a mobile device. However, the problem comes (on some browsers) when a user tries to pinch to zoom in or zoom out. The value will not adjust properly and instead still return the full length of the page.
Let's say it was 500px when loaded. The user then zooms in and the height is now 200px. However, the value is still returning 500px.
Does anyone know a method to fix this? Been searching forever.
The way I fixed this was to remove any resize callback in my code. Sounds weird, but it worked for me.
Check out the accepted answer in this link:
Detect page zoom change with jQuery in Safari
If your want innerHeight, may be get original width and then zoomed width, get zoom ratio and then calculate the new Height (after zoom).
This worked for me. The first thing I do is grab window.innerHeight and window.innerWidth from the dom when the page loads so I get the original values and store them in javascript variables. Then in my window.onresize event handler I do this.
var height = null;
var width = null;
if (window.orientation && window.orientation == -90) {
height = myOriginalHeight;
width = myOriginalWidth;
}
else {
height = myOriginalWidth;
width = myOriginalHeight;
}
doCallbacks(width, height);
My app resizes a lot because I attempt to write one ui for all screen types. According to my testing with the app this works on ipad and andriod and all the resizing works when zoomed in or orientation changes which can sometimes cause zoom to occur.
The interesting aspect of this is mobile browsers never actually change screen sizes as they are fixed, they just zoom. But if you resize to original width/height and handle orientation this way it seems to work.

Categories

Resources