Apparent blocking behaviour in JavaScript websocket on mobile Safari - javascript

I've run into a real head-scratcher, and I was hoping someone out there could shed some light on my issue.
The application I'm writing is a JS based client for what is essentially a desktop-sharing a service. The service captures images from the desktop, encodes them, as base64 encoded jpegs and sends them over a websocket to the JS client. The client then displays these images (as data URIs), users can move the mouse over the image as well as click on the image, these mouse events are encoded as commands in XML which are put into a queue and serviced on a timer every 15ms, this way the queue can be scrubbed of redundant or duplicate commands before being sent to the service. These commands are then executed (generating click events on the desktop, moving the mouse, etc.), and new desktop images are generated and the cycle continues.
The whole system works extremely well, except for some very inconsistent behaviour on Safari on the iPad. Essentially, when the user moves their finger around the screen, the client seems to block (or possibly de-prioritize) incoming messages on the websocket, in favour of only sending outgoing messages. The way this is manifest is that as you move your finger around, the display will not appear to update as long as you are touching the screen, then once you raise your finger, a flood of image updates will be received by onMessage(), which then get animated to the screen in rapid succession.
Mobile Safari is the only browser that appears to behave in this way, none of the desktop browsers, or any of the Android tablets I've tested appear to show the same behaviour.
I've put logging into the inbound and outbound methods on the websocket, and it confirms the behaviour I've seen. On Safari, I'll get numerous outbound messages in a row, followed by numerous inbound messages, whereas on Android, I'll see the inbound and outbound messages interleaved as you drag your finger around the screen, as a result the display on Android will continue to update as your are dragging your finger around.
The main reason why I suspect the websocket as the culprit is because the client has a fallback mechanism, so that if a browser does not have websocket support, a pair of XHR objects are created (one for inbound and one for outbound) and used instead of the websocket. If I force mobile Safari to use XHR instead of websockets, the problem goes away. In this case only the communication mechanism changes (all of the code for capturing input events and displaying images stays the same).
I realize that this a pretty specific problem, and without code it will be very hard to diagnose, but I opted not to post code simply due to the sheer volume of code in the client.
If anyone has see behaviour similar to what I've described, or know of any potential reasons for this behaviour, I'd be very thankful for your input.

Depending on the size of the packets, you could face the problem of 'large' messages, being extremely slow on Safari (both on iPad and desktop). Have you tried desktop Safari?
Have a look at this page, to see comparisons of performance between different browsers.
It could be that this is your problem.

Related

How to prevent Firefox mobile app throttling AJAX requests?

I'm making a media player website that lets the user listen to audio in the background. After finishing playback, it makes a handful of AJAX requests to transmit data and switch to the next audio file.
The app works fine on desktop Firefox, but on mobile Firefox, I find that when the app is not active for a certain amount of time, these requests don't occur: my server has no log of receiving the request. I'm guessing this is a battery/bandwidth saving feature, but I haven't found any documentation on this.
However, in this case it makes the site unusable on mobile devices since I can't make those requests. Is there a way to work around this?

How should I create "screen sharing" with SignalR

I know this is not a specific question, but I just want to get design ideas about a screen sharing web site with SignalR.
We want to add a link on our website which is called "Share My Screen" and then our support team be able to see the content of the browser (not whole desktop) and even they be able to click or type on customer browser.
I was thinking to do it as
Taking screen shot from browser by js (by http://html2canvas.hertzen.com for example)
send taken screen shot to server constantly (I don't know how yet)
Server sends the received screen shot to our support team browser
Capturing mouse move and key press on support team browser
Sending this captured data to customer browser
Since each part of this needs a lot of work I just want to gather all possible ideas to find a tested solution
First of all, I do not think your idea of capturing screen is really doable with javascript technology. Security would be a huge issue, you would need to process a high amount of data, and syncing events would be a nightmare no matter how you approach it. Capturing and sharing tab content and events is a much more manageable goal.
If you are aiming to use this for people who can not manage to install a remote control app, then we can count newer technologies such as WebRTC out due to browser compatibility issues.
There is a good blog post discussing this issue here - (Screensharing a browser tab in HTML5?).
I especially like the first method, using Mutation Observer (browser support) and Web Sockets (browser support). It basically syncs two html documents through the use of mutation observer and uses web sockets for communication. You could use SignalR instead of standard web socket API for communication if you prefer.

Detect disconnected websocket in framework after navigation or page unload

I'm using a web framework where everything gets passed through a websocket. New / updated DOM elements are shipped over to the browser, events get shipped back to the server. Works great. Except when the websocket gets closed.
This happens...
when using desktop/mobile Safari's browser navigation buttons are used to leave & come back to the page
on mobile Safari after a timeout when switching to a different app, tab or screen locking
when the WiFi goes down etc.
After that the user simply sees a normal looking page, but everything is obviously dead as no more updates happen and no more events get relayed to the server. I'd like to simply refresh the page in that case. Either everything goes back to normal or the user sees a connection error in case of no network etc.
I tried poking around in the framework's code, adding "window.location.reload(true);" to the "onclose" handler for the websocket kinda does the trick. My state is 100% on the server, so reloading the page will just fix things.
Assuming I can't modify / fix the web framework I'm using, and it doesn't have any onConnectionLost client-side event, what's my best option to detect this scenario and reload the page? Are there any events like 'onPageDisplayedButNotProperlyReloaded', 'onPageVisibleButHasBecomeStaleInTheMeantime', 'onWebSocketsClosed' for me to use? Can I just open some dummy websocket and reload the page when it gets closed or something? Any other good way to detect this?
Thanks!

Detect if monitor is powered on from IE Kiosk mode

I developed a web app to display a slideshow, and want to display it on my secondary monitor (Connected via HDMI) with IE's Kiosk mode on Windows 10. Because of CPU and other resources on the shared server, I want to pause the slideshow when the monitor is powered off. (And therefore nobody is seeing it)
Is there a way to detect connected displays from Internet Explorer? Since this is a one-pc kiosk setup, add-ons, etc. are accepted. Triggering javascript/jquery events would be ideal. Thank you!
No, there is no reliable way to detect if a second monitor is physically switched off but still connected via the cable.
I have to ask though: why do you need to physically switch the second monitor off?
As an alternative could you not:
Have the slideshow stop after a timed duration unless it receives an input?
Have the slideshow only on display at certain times of the day?
Accept events from, say, a node server to control when to and not show the slideshow?
Having said that these threads could provided you, albeit unreliably apparently, what you need:
Is there any way to detect the monitor state in Windows (on or off)?
Monitoring a displays state in python?
You can't do in javascript. Why not try some asp component.
http://msdn.microsoft.com/en-us/library/windows/desktop/dd162617%28v=vs.85%29.aspx
You could potentially write a command line program that sits on a particular port, continuously checks for that locally and then use HTML5 WebSockets in IE to communicate with it?
i.e. C# PowerModeChangedEvent
SystemEvents.PowerModeChanged += new PowerModeChangedEventHandler(
SystemEvents_PowerModeChanged
);
I don't think so....
CPU cycles are paused when the client computer is put into Sleep mode. (win+L)
Start>Control Panel>Power
configures how the monitor(s) behave when the client is powered down or put to sleep mode.
the screen object in js returns the metic values (height/width) of the screen object but not its powered state.
the impact of wasted CPU cycles on a powered down secondary monitor should be un-noticable....
probably you have not selected the option to "Use software rendering instead of GPU rendering" on the Advance tab of internet options....
You will notice that your CPU on your desktop will throttle up and the cooling fan will race if you haven't set the above setting when running graphic intensive web pages or canvas scripts.

Implement locking & unlocking a document while editing

Here's the situation:
I have a web-based ticket application, multiple users.
One problem that might occur (and does happen in the old version I'm replacing) is that user1 opens a ticket, edits it, and saves it. But while he was editing it, user2 also opened and saved the ticked. The changes user2 made will be lost/overwritten by user1.
To prevent this I implemented a locking mechanism, it's fairly simply:
On opening a ticket the PHP script checks for existing locks.
If it doesn't find any, it locks & opens the document.
In JS, setTimeout() and an XmlHttpRequest call to unlocks the ticket after 10 minutes (works w/o problems).
I also set an unload event to unlock the ticket when closing/moving away from the window/tab
The problem sits in step 4: The unload event (& it's friend beforeunload) just doesn't work well enough to implement this reliably (for this feature to have any serious meaning, it needs to be reliable), many browsers don't always fire it when I would like it to be fired (Like pressing back button, hitting F5, closing tab, etc. This varies per browser)
The only alternative I can come up with is using a setTimeout() and XmlHttpRequest() call to a php script to tell it the page is still open. If this "heartbeat" monitor fails we assume the user moved away from the ticket and unlock the document.
This seems horribly inefficient to me and quickly leads to many requests to the server with even a few users.
Anyone got a better idea on how to handle this?
It needs to work in IE8+ and other modern browsers (ideally, Firefox, Webkit, Opera). I don't care about IE6/IE7, our organization doesn't use those).
Using heartbeat pings via XHR is the way to go. Depending on the use case you might also want to send them after the user stopped typing in a field instead of every x seconds - this would ensure the page being kept open but inactive would not keep it locked.
If you send those XHRs after the user stopped typing, use one of the keydown/up/press events and a debounce / throttle script to send the request only when the user stops typing for e.g. 5 seconds and one every x seconds (in case it's likely enough the user will be typing for a long time).
Maybe it's not the best solution, but it's worth looking into it : websockets.
You could establish a connection with the server at page load and when the connection fails (ie the client does not respond to the ping), you can unlock the ticket.
Using something like socket.io ensures you that this procedure will work even on ie8.
The main advantage is that you do not send a request every n seconds, but the server sends you a ping every n seconds and you don't have to care about unload/beforeunload events. If the client doesn't respond to the ping, unlock the ticket.
The main disadvantage is that you need a server to handle all your websocket connections, which can be done in almost any server-side language, but it can be a bit harder than a simple web-service (in case of xhr polling)
Implementing ajax heartbeats or unload handlers to unlock the document automatically is tricky.
You problem is that even if you have support for beforeunload in all browsers that you target, it still might not be called if the browser crashes or the user falls asleep.
Look at how webdav works. You explicitly aquire a lock before you start edit, then you save and release the lock explicitly.
Other users can see who has acquired a lock and admins can release locks that has been left behind by accident.

Categories

Resources