We are using Backbone.js and having issues when running our WebDriver tests. We are getting the following error:
org.openqa.selenium.StaleElementReferenceException: Error Message => 'Element does not exist in cache'
Our understanding is that this is caused when we are finding an element, and executing an action on that element (e.g. click()). The element that we have found has gone 'stale', and we suspect that element has been re-rendered or modified.
We have seen lots of solutions that we are not keen on:
Use Thread.Sleep(...). We don't want explicit sleeps in our code
Using a retry strategy, either as a loop or try-catching the StaleElementReferenceException. We feel this is not the right/clean solution, and is prone to breaking in the future
Some people are using WebDriverWait and waiting until some javascript function execution returns true. We have seen people wait for notifyWhenNoOutstandingRequests(callback) in Angular, but can't find anything obvious for Backbone.
We are hoping there is a clean solution that does not involve explicit sleeping, or some form of looping. Any thoughts?
I looked into WebDriverWaits a bit more and I think i've come up with a combination of expectations that works for us:
wait.until(refreshed(elementToBeClickable(...)));
The refreshed expectation is a wrapper for other expectations that deals with StaleElementReferenceException, and the elementToBeClickable expectation checks the element is clickable. What is interesting is that looking at the source for the built in expectations, some of them deal with StaleElementReferenceExceptions, while others don't (e.g. presenceOfElementLocated) and need to be wrapped in the refreshed expectation, so I think that's what initially threw me off when I first looked at WebDriverWaits.
Related
In my script, I am trying to locate and click one of the many document links, with this syntax:
cy.wait(3000); cy.get('a[href^="/articleDetail/"]').first().click();
I got this error:
CypressError: Timed out retrying: Expected to find element:
'a[href^="/articleDetail/"] but never found it'
The issue is this happens only few times, not all the times. Like 3 out 5 times. How should I solve this issue ?
Testing it via the Selector Playground (as N. suggested) is a good step. What you also can do is investigate screenshots which Cypress can make on failure. That shows the exact state of the application when the failure happened. That usually gives a good hint to the problem.
Besides that you can also try to set the wait to an absurt value like 10000. If Cypress can find the element at that case, the application is slow and therefor Cypress is not waiting long enough.
For different reasons (internet speed, CPU, Memory, errors) your page could take longer to load or not load at all. As a good practice, your page should have a loading system, where it is shown until the page is completely rendered. This way you could have something like cy.get('your-loading-element').should('not.be.visible'), which will hold the next command while the loading is in place.
Waiting is not the right approach as you never know exactly how long it will take and raising the time will only delay your tests.
It is very important to think of your test in the same way a test analyst would execute them, because one of the steps would be to wait the page to be rendered.
Here is some good testing good practices: UI test automation good practices
I am using Protractor and Jasmine.
I'm automating a website that is heavily data-driven with a lot of dynamic elements (elements displayed depend on data available). As a result, I never know exactly how many elements there are, but I need to test each one since data-driven means that just because one works, doesn't mean the rest will.
I'm not sure the best way to go about this - I've done tons of research, but none of the ideas work, or only partially work.
What I've tried:
Throwing an it block into a loop that dynamically grabs the element count
I found that this doesn't work because it appears Jasmine evaluates which / how many tests run at compile. And since I need get to the page before I can grab the count, the count is always 0, so it runs the test 0 times
This only works with static data, but again, my data is dynamic, so this won't work. At least, I couldn't find a way to
Throwing an it block into a loop that loops using a variable and then reassigning the variable in a beforeAll
Same issue as the previous, reassigning doesn't work because Jasmine uses the value that was available on compile
Looping through the elements inside of an 'it' and doing an 'expect' for each element
This works for the most part, but only the first error gets reported. I'd ideally like to see every element that has an issue. Jasmine loops through all elements even when one expect fails, so I'm not sure why it doesn't report them all / or how to report them all
I'd prefer to use solution #3 if I can see all expect failures, but I'm open to any suggestions. If there's a better way, best practice, or a way you handle this instead of what I have listed, I'd like to hear those as well.
Let me know if any more information is needed.
I'm new to Angular2 and have been charged with developing "robust error handling". So far I've followed the simplistic examples (console.logging) for adding custom error handling. But sometimes, if the page completely stops loading due to the error, we will want to redirect the user.
However sometimes, as below, although there are errors, the page otherwise loads completely. Are there are only certain types of errors that stop the page from completely loading? One of the following 6 types perhaps?
Any error can stop your page from rendering, depending on where it occurs in your process. Any error can fail to be caught if it is in a callback or other asynchronous action.
Be careful with terms like "robust error handling" - I've seen huge commercial projects that claim just that, but actually just silently truck over loads of issues, kind of like on error resume next.
I find the golden rules are:
If your app can continue through an exception (such as getting corrupt JSON data from a non-essential service) then that specific case should always be explicitly handled.
Otherwise a unexpected exception should always break something visible.
That second rule is counter intuitive, but it really is best practice. Users will complain about errors that they see, and visible exceptions and crashes will upset them and reduce their confidence in your application.
However, exceptions that they don't see still happened, and because you've silently trucked through them whatever caused them is still there. Silent exceptions cause data to be lost or corrupted. They cause the kind of bugs that you only find out about after 6 months in production. They cause the kind of bugs you can get sued over.
Users will forgive you for obvious errors that you fix quick, they will leave and never come back if you lose data and don't immediately know about it.
Ok, so that all said, the errors you seem to be highlighting are asynchronous, and related to a problem sometimes described as callback hell.
In your screenshot the error is from an HTTP GET request - this will typically be a method where you make an AJAX request, have a callback to fire when it succeeds, but don't have a callback to handle the exception.
Angular2 uses promises, which is the next error line of your screenshot. Promises wrap those callbacks and allow you to chain them - they really help with callback hell, but they're not a magic bullet: you have to make sure that every .then() has an error handler or a following .catch().
However, there is an even better way: with Angular2 you can use TypeScript, and that means you can use async and await. These are syntactic sugar for promises, but they also work with try-catch to make error handling of asynchronous exceptions much easier.
I've blogged about that in a lot more detail than I can fit here.
TL;DR: in Angular2 use async/await (with TS transpilation if you need it) to make sure that your Promise and callback exceptions make it back up, and then handle what you expect/can work around and visibly crash for what you can't.
In an existing Backbone/jQuery/CoffeeScript app I am working on, it appears there is a function (Backbone.Collection.fetch()) called multiple times (sometimes number may vary). I think it might be a timing thing as I am doing alot of nested callbacks (like AJAX etc) and its becoming hard to debug. I should probably try to convert the code to use jQuery deferred but in the mean time, what can I do?
Just tried walking through the code in Chrome, but it appears the code is jumping here and there, maybe its processing different callbacks at the same time?
I am thinking maybe I add a console.log to every function + its arguments, but there must be a better way?
You can add a stack trace to that fetch() function, and see where it's being called from. There are a number of decent stack trace implementations for JS. I've had good success with
Eric Wendelin's version, but there are plenty of others.
With the stack trace, perhaps you can at least see what the most common paths are into that function, and that might help narrow down where to search. It might even make clear the underlying culprit.
I know I can get Selenium 2's webdriver to run JavaScript and get return values but so much asynchronous stuff is happening I would like JavaScript to talk to Selenium instead of the other way around. I have done some searching and haven't found anything like this. Do people just generally use implicitly_wait? That seems likely to fail since it's not possible to time everything? Perfect example would be to let Selenium know when an XHR completed or an asynchronous animation with undetermined execution time.
Is this possible? We're using Selenium 2 with Python on Saucelabs.
You should look into the execute_async_script() method (JavascriptExecutor.executeAsyncScript in Java, IJavaScriptExecutor.ExecuteAsyncScript() in .NET), which allows you to wait for a callback function. The callback function is automatically appended to the arguments array in your JavaScript function. So, assuming you have a JavaScript function already on the page that waits until the condition you want, you could do something like the following (Java code below, C# and Python code should be similar):
String script = "var callback = arguments[arguments.length - 1];"
+ "callback(myJavaScriptFunctionThatWaitsUntilReady());";
driver.manage().timeouts().setScriptTimeout(15, TimeUnit.SECONDS);
((JavascriptExecutor)driver).executeAsyncScript(script);
It might be possible to be even more clever and pass the callback function directly to an event that returns the proper data. You can find more information on the executeAsyncScript() function in the project JavaDocs, and can find sample code for this in the project source tree. There's a great example of waiting for an XHR to complete in the tests in this file.
If this isn't yet available in the version of the Python bindings available for use with SauceLabs, I would expect it to be available before long. Admittedly, in a sense, this is pushing the "poll for desired state" from your test case into JavaScript, but it would make your test more readable.
Theoretically it is possible, but I would advise against it.
The solution would probably have some jQuery running on the site that sets a variable to true when the JavaScript processing has finished.
Set selenium up to loop through a getEval until this variable becomes true and then do something in Selenium.
It would meet your requirements but it's a really bad idea. If for some reason your jQuery doesn't set the trigger variable to true (or whatever state you expect) Selenium will sit there indefinetly. You could put a really long timeout on it, but then what would be the different in just getting Selenium to do a getEval and wait for a specific element to appear?
It sounds like you are trying to overengineer your solution and it will cause you more pain in the future will very few additional benefits.
Not to be overly blunt, but if you want your App to talk to your Test Runner, then you're doing it wrong.
If you need to wait for an XHR to finish, you could try displaying a spinner and then test that the spinner has disappeared to indicate a successful request.
In regards to the animation, when the animation has completed, maybe its callback could add a class indicating that the animation has finished and then you could test for the existence of that class.
Testing animation with selenium is opening a can of worms. The tests can be quite brittle and cause many false positives.
The problem is to do that the calls are asynchronous, and difficult to track the behaviour and change in state of the page.
In my experience the asynchronous call can be so quick that the spinner is never displayed, and the state of the page may skip a state entirely (that Selenium can detect).
Waiting for the state of the page to transition can make the tests less brittle, however the false positives cannot be removed entirely.
I recommend manual testing for animation.