interact.js 1.2.4 drag and gesture action confusion - javascript

I have an issue with interact.js 1.2.4 and iPad Retina iOS 8.1 Mobile Safari, which I need to resolve somehow. The problem occurs in a sorting component nested inside a large sequential learning web-application, where the user solves a succession of tasks. The interact.js plugin is used for adding drag/drop behavior so the user can sort elements in the correct order. The site is live, but the issue I'm about to describe is now persistent and reproducible, so it needs to be solved quick.
Here's the setup. We've initialised a number of draggable elements with
interact('.drag').draggable(...);
and we've registered onstart, onmove and onend listeners, which are handled manually with callback functions.
In most cases, the drag drop behavior acts as planned. But when using the sorting component in certain parts of the application sequence, the actionName of an interaction is inexplicably "gesture" instead of "drag", which makes the validateAction (interact.js) function return null instead of a valid action. This in turn causes the pointerDown (interact.js) handler to never get fired, so the drag motion is never initiated, and the element stays put forever, making it impossible to progress further in the sequence. The behavior only occurs on iPad.
If I run the site in an iPad simulator and inspect the element with Safari's developer tools crosshair functionality, something changes the element, and the next interaction's actionName suddenly becomes "drag", so the component starts to work as intended and the user can proceed in the sequence.
Do you have any idea as to:
What causes the website to relay a "gesture" action instead of a "drag" action?
What excactly happens when I use the inspection crosshair? Why does it change behavior?
How do I solve this issue?
You can see the component code here, but it's part of a major Backbone solution, so this is all I can include.
Thanks in advance.

We still don't know the cause, but seemingly there is a bug in iOS, which causes the application to fire 5 pointer events at once. Thus, the plugin registers the event as a gesture, and the drag drop functionality fails. The plugin creator gave this suggestion, which fixed the issue:
interact(document).on('down', function (event) {
var interaction = event.interaction;
var canGesture = interaction.target && interaction.target.options.gesture.enabled;
var maxAllowedPointers = canGesture? 2 : 1;
while (interaction.pointers.length > maxAllowedPointers) {
interaction.removePointer(interaction.pointers[0]);
}
})

Related

JavaScript: Tracking Selection Change on Firefox for Android

I'm looking for a way to track the text selection on a web page. I need some code to be executed whenever there is a change in selection. I have successfully done this on all major desktop browsers, but the same does not seem to be working on Firefox for Android.
I tried three approaches (none of them worked!):
Tried catching the "mouseup" event and checking if there is a
non-empty text selection. Problem is "mouseup" isn't getting
triggered if a selection was made during the mousedown-move-up
sequence!
Tried doing the same with the "touchend" event - Same result.
Tried catching the "selectionchange" event. I saw that it isn’t
triggered when the selection changes as it needs the config key
"dom.select_events.enabled" to be set. This is false by default and I obviously can't ask my visitors to tweak browser settings :-)
Also, as expected, the first two events don't get triggered if the selection is extended or reduced by dragging the selection start/end markers.
The only solution I can think of now is a periodic poller (using setInterval) that checks if there is a text selection. This is definitely unclean and anti-performance.
Any alternatives and/or advice will be very helpful.
1) Native Text Selection Events
Currently, polling seems to be the only work-around.
However, the selectionchange event is currently experimentally implemented in Firefox for Android Nightly. It can be enabled by setting the dom.select_events.enabled flag to true (defaults to false).
In Nightly builds, this flag already defaults to true, so there is a chance you can use it normally in a couple of months.
2) Find events that work
(UNTESTED!!)
Even if onselectstart cannot be used productively in Firefox for Android yet, the only viable easy solution is polling.
To improve performance and reduce costs, polling can be started on the window blur event. Because whenever the user is making a selection, the focus should be set off the viewport (untested though).
window.addEventListener('blur', function(){
// start polling for text selections
});
When the focus is given back, polling can be stopped.
window.addEventListener('focus', function(){
// stop polling
});
To check if the browser actually supports text selection events, you can use this function:
var browserSupportsTextSelectionEvents = function(){
return !!('onselectstart' in document.body);
}
3) Hacking
A third idea is to disable text selection for mobile Firefox users via CSS (-moz-user-select: none), implement a custom text selection function based on touch start and end positions and pushing the native selection range back to the browser via HTMLElement.setSelectionRange().

WebDriver click() vs JavaScript click()

The Story:
Here on StackOverflow, I've seen users reporting that they cannot click an element via selenium WebDriver "click" command and can work around it with a JavaScript click by executing a script.
Example in Python:
element = driver.find_element_by_id("myid")
driver.execute_script("arguments[0].click();", element)
Example in WebDriverJS/Protractor:
var elm = $("#myid");
browser.executeScript("arguments[0].click();", elm.getWebElement());
The Question:
Why is clicking "via JavaScript" works when a regular WebDriver click does not? When exactly is this happening and what is the downside of this workaround (if any)?
I personally used this workaround without fully understanding why I have to do it and what problems it can lead to.
Contrarily to what the currently accepted answer suggests, there's nothing specific to PhantomJS when it comes to the difference between having WebDriver do a click and doing it in JavaScript.
The Difference
The essential difference between the two methods is common to all browsers and can be explained pretty simply:
WebDriver: When WebDriver does the click, it attempts as best as it can to simulate what happens when a real user uses the browser. Suppose you have an element A which is a button that says "Click me" and an element B which is a div element which is transparent but has its dimensions and zIndex set so that it completely covers A. Then you tell WebDriver to click A. WebDriver will simulate the click so that B receives the click first. Why? Because B covers A, and if a user were to try to click on A, then B would get the event first. Whether or not A would eventually get the click event depends on how B handles the event. At any rate, the behavior with WebDriver in this case is the same as when a real user tries to click on A.
JavaScript: Now, suppose you use JavaScript to do A.click(). This method of clicking does not reproduce what really happens when the user tries to click A. JavaScript sends the click event directly to A, and B will not get any event.
Why a JavaScript Click Works When a WebDriver Click Does Not?
As I mentioned above WebDriver will try to simulate as best it can what happens when a real user is using a browser. The fact of the matter is that the DOM can contain elements that a user cannot interact with, and WebDriver won't allow you to click on these element. Besides the overlapping case I mentioned, this also entails that invisible elements cannot be clicked. A common case I see in Stack Overflow questions is someone who is trying to interact with a GUI element that already exists in the DOM but becomes visible only when some other element has been manipulated. This sometimes happens with dropdown menus: you have to first click on the button the brings up the dropdown before a menu item can be selected. If someone tries to click the menu item before the menu is visible, WebDriver will balk and say that the element cannot be manipulated. If the person then tries to do it with JavaScript, it will work because the event is delivered directly to the element, irrespective of visibility.
When Should You Use JavaScript for Clicking?
If you are using Selenium for testing an application, my answer to this question is "almost never". By and large, your Selenium test should reproduce what a user would do with the browser. Taking the example of the drop down menu: a test should click on the button that brings up the drop down first, and then click on the menu item. If there is a problem with the GUI because the button is invisible, or the button fails to show the menu items, or something similar, then your test will fail and you'll have detected the bug. If you use JavaScript to click around, you won't be able to detect these bugs through automated testing.
I say "almost never" because there may be exceptions where it makes sense to use JavaScript. They should be very rare, though.
If you are using Selenium for scraping sites, then it is not as critical to attempt to reproduce user behavior. So using JavaScript to bypass the GUI is less of an issue.
The click executed by the driver tries to simulate the behavior of a real user as close as possible while the JavaScript HTMLElement.click() performs the default action for the click event, even if the element is not interactable.
The differences are:
The driver ensures that the element is visible by scrolling it into the view and checks that the element is interactable.
The driver will raise an error:
when the element on top at the coordinates of the click is not the targeted element or a descendant
when the element doesn't have a positive size or if it is fully transparent
when the element is a disabled input or button (attribute/property disabled is true)
when the element has the mouse pointer disabled (CSS pointer-events is none)
A JavaScript HTMLElement.click() will always perform the default action or will at best silently fail if the element is a disabled.
The driver is expected to bring the element into focus if it is focusable.
A JavaScript HTMLElement.click() won't.
The driver is expected to emit all the events (mousemove, mousedown, mouseup, click, ...) just like like a real user.
A JavaScript HTMLElement.click() emits only the click event.
The page might rely on these extra events and might behave differently if they are not emitted.
These are the events emitted by the driver for a click with Chrome:
mouseover {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
mousemove {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
mousedown {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
mouseup {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
click {target:#topic, clientX:222, clientY:343, isTrusted:true, ... }
And this is the event emitted with a JavaScript injection:
click {target:#topic, clientX:0, clientY:0, isTrusted:false, ... }
The event emitted by a JavaScript .click() is not trusted and the default action may not be invoked:
https://developer.mozilla.org/en/docs/Web/API/Event/isTrusted
https://googlechrome.github.io/samples/event-istrusted/index.html
Note that some of the drivers are still generating untrusted events. This is the case with PhantomJS as of version 2.1.
The event emitted by a JavaScript .click() doesn't have the coordinates of the click.
The properties clientX, clientY, screenX, screenY, layerX, layerY are set to 0. The page might rely on them and might behave differently.
It may be ok to use a JavaScript .click() to scrap some data, but it is not in a testing context. It defeats the purpose of the test since it doesn't simulate the behavior of a user. So, if the click from the driver fails, then a real user will most likely also fail to perform the same click in the same conditions.
What makes the driver fail to click an element when we expect it to succeed?
The targeted element is not yet visible/interactable due to a delay or a transition effect.
Some examples :
https://developer.mozilla.org/fr/docs/Web (dropdown navigation menu)
http://materializecss.com/side-nav.html (dropdown side bar)
Workarrounds:
Add a waiter to wait for the visibility, a minimum size or a steady position :
// wait visible
browser.wait(ExpectedConditions.visibilityOf(elem), 5000);
// wait visible and not disabled
browser.wait(ExpectedConditions.elementToBeClickable(elem), 5000);
// wait for minimum width
browser.wait(function minimumWidth() {
return elem.getSize().then(size => size.width > 50);
}, 5000);
Retry to click until it succeeds :
browser.wait(function clickSuccessful() {
return elem.click().then(() => true, (ex) => false);
}, 5000);
Add a delay matching the duration of the animation/transition :
browser.sleep(250);
The targeted element ends-up covered by a floating element once scrolled into the view:
The driver automatically scrolls the element into the view to make it visible. If the page contains a floating/sticky element (menu, ads, footer, notification, cookie policy..), the element may end-up covered and will no longer be visible/interactable.
Example: https://twitter.com/?lang=en
Workarounds:
Set the size of the window to a larger one to avoid the scrolling or the floating element.
Mover over the element with a negative Y offset and then click it:
browser.actions()
.mouseMove(elem, {x: 0, y: -250})
.click()
.perform();
Scroll the element to the center of the window before the click:
browser.executeScript(function scrollCenter(elem) {
var win = elem.ownerDocument.defaultView || window,
box = elem.getBoundingClientRect(),
dy = box.top - (win.innerHeight - box.height) / 2;
win.scrollTo(win.pageXOffset, win.pageYOffset + dy);
}, element);
element.click();
Hide the floating element if it can't be avoided:
browser.executeScript(function scrollCenter(elem) {
elem.style.display = 'none';
}, element);
NOTE: let's call 'click' is end-user click. 'js click' is click via JS
Why is clicking "via JavaScript" works when a regular WebDriver click does not?
There are 2 cases for this to happen:
I. If you are using PhamtomJS
Then this is the most common known behavior of PhantomJS . Some elements are sometimes not clickable, for example <div>. This is because PhantomJS was original made for simulating the engine of browsers (like initial HTML + CSS -> computing CSS -> rendering). But it does not mean to be interacted with as an end user's way (viewing, clicking, dragging). Therefore PhamtomJS is only partially supported with end-users interaction.
WHY DOES JS CLICK WORK? As for either click, they are all mean click. It is like a gun with 1 barrel and 2 triggers. One from the viewport, one from JS. Since PhamtomJS great in simulating browser's engine, a JS click should work perfectly.
II. The event handler of "click" got to bind in the bad period of time.
For example, we got a <div>
-> We do some calculation
-> then we bind event of click to the <div>.
-> Plus with some bad coding of angular (e.g. not handling scope's cycle properly)
We may end up with the same result. Click won't work, because WebdriverJS trying to click on the element when it has no click event handler.
WHY DOES JS CLICK WORK? Js click is like injecting js directly into the browser. Possible with 2 ways,
Fist is through devtools console (yes, WebdriverJS does communicate with devtools' console).
Second is inject a <script> tag directly into HTML.
For each browser, the behavior will be different. But regardless, these methods are more complicating than clicking on the button. Click is using what already there (end-users click), js click is going through backdoor.
And for js click will appear to be an asynchronous task. This is related a with a kinda complex topic of 'browser asynchronous task and CPU task scheduling' (read it a while back can't find the article again). For short this will mostly result as js click will need to wait for a cycle of task scheduling of CPU and it will be ran a bit slower after the binding of the click event.
(You could know this case when you found the element sometimes clickable, sometimes not.
)
When exactly is this happening and what is the downside of this
workaround (if any)?
=> As mention above, both mean for one purpose, but about using which entrance:
Click: is using what providing by default of browser.
JS click: is going through backdoor.
=> For performance, it is hard to say because it relies on browsers. But generally:
Click: doesn't mean faster but only signed higher position in schedule list of CPU execution task.
JS click: doesn't mean slower but only it signed into the last position of schedule list of CPU task.
=> Downsides:
Click: doesn't seem to have any downside except you are using PhamtomJS.
JS click: very bad for health. You may accidentally click on something that doesn't there on the view. When you use this, make sure the element is there and available to view and click as the point of view of end-user.
P.S. if you are looking for a solution.
Using PhantomJS? I will suggest using Chrome headless instead. Yes, you can set up Chrome headless on Ubuntu. Thing runs just like Chrome but it only does not have a view and less buggy like PhantomJS.
Not using PhamtomJS but still having problems? I will suggest using ExpectedCondition of Protractor with browser.wait() (check this for more information)
(I want to make it short, but ended up badly. Anything related with theory is complicated to explain...)
Thank you for the good explanation, I was hitting the same issue and your explanation helped to resolve my issue.
button = driver.wait.until(EC.presence_of_element_located(
(By.XPATH, "//div[#id='pagination-element']/nav[1]/span[3]/button[1]/span[1]/i[1]")
))
driver.execute_script("arguments[0].click();", button)
if (theElement.Enabled)
{
if (!theElement.Selected)
{
var driver = (IJavaScriptExecutor)Driver;
driver.ExecuteScript("arguments[0].click();", theElement); //ok
//theElement.Click();//action performed on theElement, then pops exception
}
}
I do not agree we will "almost never" use JS to simulate the click action.
Above theElement.Click(), we'll check the Radio button but then Exception pops as above image.
Actually, this is no page load action after the Click, the click is just to select the Radio button, and I don't know why the WebDriver Click() will cause this exception, can anyone explain why this exception occurred.
I use the Webdriver 3.141.59 and IE 11 and selenium-server-standalone-3.141.59.jar for a remote test.

ArcGIS JavaScript api Map update-end stops firing

We have a complex web mapping app using arcgis JavaScript api. The following code is used to inform the ui that the map layers are loaded:
on(map,"update-end",function(){
//emit event to ui module
});
map.addLayers(layers);
The application frequently adds and removes map layers. However occasionally something is going wrong and although layers get added the update-end event doesn't get fired, despite the layers completing load. After this happens the event wont fire again unless the app reloads.
Does anyone have an idea under what conditions this may occur? How can we prevent the problem?
I usually put my map.on("update-end", endUpdateMap); sentence after any map.addlayer() in the same javascript where you are defining your layers. It works fine for me.
map.on("update-end", endUpdateMap);
function endUpdateMap(error) {
esri.hide(loading);
map.enableMapNavigation();
map.disableDoubleClickZoom();
map.showZoomSlider();
}
I had this issue, I think there is actually a bug in the JSAPI. If you remove a layer whilst it is updating then it actually breaks the "update-start" and "update-end" events, they stop firing completely. I will try and put a test case together at some point but I found with a slow drawing layer, like an image service, the issue was very prevalent.
Anyway, I had to think of a way around it and this is what I came up with:
this.layer.setVisibility(false); // set invisible first to ensure spinner stop
this.map.removeLayer(this.layer);

How to tell what Javascript event just fired

Is there any way in Javascript to tell what the most recently-fired event was? Reason I'm asking is I'm working on a site that isn't behaving properly on one of those new Ultrabooks that's running Windows 8 and is a laptop with a touch screen. If you use the standard mouse functionality (with a touchpad or an actual mouse), things work fine, but if you use the touch screen, things don't.
This only happens with IE; Chrome has its own issues (which I have fixed in the code), and Firefox hasn't given us any problems.
Basically, the functionality we have includes a "hoverIntent" block, and if you use the touch screen on IE, it registers both the "over" and "out" functions, which is a problem.
However, if there was a way for me to tell whether the last thing that happened was that the user TOUCHED THE SCREEN or CLICKED WITH A MOUSE, I'd have a solution in place. But I couldn't tell if there's a way to do that.
The only thing I could find was tacking on ".data('events')" on an element, but what returns is "click" regardless of whether it was an actual mouse click or a tap on the screen.
Is there a way to do this?
The browser does not have a standard way of recording events that happened previously. If you want to know what events happened prior to the current event, then you will have to install an event handler for those events and record them yourself so you can then look back at them at some future time.
For events that propagate, you could install some event handlers on the document object and record both event and target for the last N events.
If you're just trying to figure out what event the current event is, then you can examine the type property of the event object that is passed into the event handler as in e.type.
You can add an event to your function arguments and then use event.type to check which event is triggered.
ex:
var x = function(e) {
alert(e.type);
}
So I found out that IE has a completely different set of touch events from what EVERY OTHER BROWSER IN THE UNIVERSE has. ughhh. Instead of "touchstart," you use "MSPointerDown," etc. My solution was basically to write new event handlers for MSIE's touch device events.

IE 9 & 10 events not firing (consistently)

I am developing a web application. This application due to requirement contains a table with 500+ rows (unpaged). The application is very interactive and so we're using KO behind the scenes.
That said I am noticing that occasionally tabbing on an input text box, does not take you to the nearest text box, instead it takes you to the first input text box on the page. When this happens a couple of things also happen:
The blur event does not trigger.
The focusout event also does not trigger. I have had to work around the issue by detecting mouse click and fire the events off manually using jQuery.
Unobtrusive validation will not work.
View model is not bound ie. changes on the input box is not reflected on the KO view model.
The issue is intermittent, I cannot necessarily reproduce it consistently, but it is happening frequently - and considering money is involved in this application I need this to work 100%.
Also testing the same application on Chrome and Firefox works perfectly. No intermittent issue as detected on IE9 and 10. Is there a known browser bug around this issue? Or am I just missing something?

Categories

Resources