I have a python script that is utilizing selenium with the chrome webdriver. When I load pages sometimes it'll get stuck because some javascript is trying to load. If I run the line:
driver.execute_script("$(window.stop())")
that occasionally (but not reliably) works to simulate the stop browser action. My other thought was to change the page_load_timeout to 5s. That will throw an error when this annoying javascript isn't loading. I want to stop the page from loading with the timeout, but then run the function that I initially wanted to run... How do I accomplish that?
One way is, you have to assert by inspection which element could do the delay and with the element's #id you you can wait for that element to load with a time out.
try:
WebDriverWait(driver, 60).until((EC.visibility_of_element_located(By.Id, 'id')))
except TimeoutException:
pass
Or you can use the staleness_of approach, explained in related SO answers,
#contextlib.contextmanager
def wait_for_page_load(self, timeout=30):
old_page = self.find_element_by_tag_name('html')
yield
WebDriverWait(self, timeout).until(staleness_of(old_page))
Related
I am working on a stabilization of an old application which is developed using jQuery. In order to stabilize it, I added a method which, using javascript, checks for document is ready document.readyState === "complete"and ajax calls have been completed jQuery.active == 0 before interacting with elements.
But I still see following kind of issues -
Even though element is present on a page, Selenium WebDriver unable to find it even with polling mechanism which has some timeout set. However we can find same element, by checking in DOM.
UI is distorted and when WebDriver tries to click on an element, it gives error Element click intercepted. However same element can be clicked manually.
Even application has jQuery, when we execute javascrit jQuery.active == 0 through WebDriver, it results into jQuery is not defined. However if we execute same query through browser console, it returns correct result.
For all above, when I said something worked that means I've checked that in same browser instance which WebDriver had launched.
Could you please help how this can be resolved? I have been struggling for a week to stabilize this.
I'm trying to identify roughly when the DOM is finished updating after a page is loaded via AJAX on any arbitrary website.
My current method first listens for the chrome.webNavigation.onHistoryStateUpdated event in a background script, then executes a content script in which a MutationObserver detects changes to the website's body. From there, unfortunately, it seems like it's a bit more finicky. If I just wait for the first mutation where nodes are added to the DOM, I wind up in many cases (YouTube, to give one example) where the page is still blank. Other more hacky approaches I've considered include things like just using setTimeout or waiting for the page to reach a certain length, but those seem clearly wide open to exception cases.
Is there a more fool-proof way to detect that the DOM has roughly finished updating? It doesn't necessarily have to be perfectly precise, and erring on the side of triggering late in my use case is better than triggering early. Also it isn't important at all that resources like video and images be fully loaded, just that the text contents of the page are basically in place.
Thanks for your help!
First I would like to describe the motivation for my question.
I have a complex web page to test with Selenium + HtmlUnit, which launches diverse javascript scripts. The problem which I describe should be quite common.
On the page there is a button to which jQuery binds a click callback (click event handler) after the page is loaded. There is an explicit Wait (this is a Selenium term) for the button to become clickable in the test code. So as soon as the button becomes clickable, it gets clicked by Selenium. Often, however, this happens before jQuery manages to attach to the button the click event handler. In this case the Selenium test fails.
What I thought to do is to preprocess the web page accessed by HtmlUnit before javascript starts executing on the page, injecting some <script>myownscript()</script>at the beginning of the page (so that it executes before any other script on that page). Then I would be able to know, controlling certain conditions in the Selenium test code, when exactly the attaching of the click event handler has happened (how I exactly do this, depends on the details of the application). If I make Selenium click the button then, the presence of the click event handler will be guaranteed, and the test would proceed further as planned - with no errors due to the missing click event handler.
Let us leave apart the question whether the idea is a good or a bad one (a much simpler one, of course, would be just introducing a large enough delay in the Selenium test code before trying to click the problematic button, but then there might be a problem with the overall duration of tests, because the problem I described is present on many pages of the application being tested).
Are there some hooks in Selenium/HtmlUnit which permit to preprocess the page fetched from the server, injecting a script as I described, before javascript starts executing on the page?
In this case, you can use JavaScriptExecutor. You can add a function to do anything you want in the String script.
WebElement button = driver.findElement(By.id("my-button"));
JavascriptExecutor jsExe = ((JavascriptExecutor) driver);
String script = "console.log(arguments[0].id); return arguments[0].id";
Object oj = jsExe.executeScript(script, button);
String txt = oj.toString();
System.out.println(txt);
Please be careful if you want to use aycn such as setTimeout(), it will return immediately. See an example for async method in my answer at: method execute_script don't wait end of script to return value with selenium in python
I'd like to do things like this;
elem1 = document.getElementById(id1);
elem1.click()
// The page has been changed
elem2 = document.getElementById(id2);
elem2.click()
My problem is, if the second 'getElementById(id2)' is called before the HTML of document.documentElement completes its change, it sometimes fails to get elem2 because there is no HTML element that has id2. (Every javascript sentence in the above code is sent from 3rd party like Selenium Server, and there is a javascript loop that gets the sentence and runs it one by one. You can think the loop as a thing like Selenium Core)
I'm trying to write a tool like Selenium or Watir. So I can't use those tools. I think Selenium or Watir has a way to send the second command after it waits until the page has been completely changed as the result of the first command. I tried events like window.onwebkittransitionend because I'm trying this on Webkit, but it did not work.
I don't think promise or async can solve this problem, because it needs more than enforcing sequences. It needs to verify if the second page is fully loaded, because the second command can be the others like fetching the information of rendered images rather than just click.
Is there any event or a way to check if page change is completed? Oh, and I can't use jQuery unfortunately. I need to do it with plain Javascript.
Thanks for reading my question.
Is JavaScript intended to be running as little as possible on a website/webapp? By that I mean is the usual intention to run through all your js files as soon as the page loads and put them aside, and then when functions come up to execute them right away and be done with it?
I'm working on a project using google maps and I have a custom marker object scripted out, and a debugger has told me that the browser runs through all my js files before anything even appears on the page.
My problem comes in here: I wanted to animate certain markers to bounce up and down continuously with jQuery (similar to OS X icons in the dock) and my several attempts at infinite loop functions all just crash the browser. So I understand that the browser doesn't like that, but is there a way to have a simple script be repeating itself in the background while the user navigates the page? Or is JavaScript just not supposed to be used that way?
(I worked with Flash for a long time so my mindset is still there.)
Yes, Javascript functions should just do their bit and exit as soon as possible. The GUI and the scripts run on the same single thread, so as long as you are inside a Javascript function, nothing shows up in the browser. If you try to use an infinite loop, the browser will appear to freeze.
You use the window.setInterval and window.setTimeout methods to trigger code that runs at a specific time. By running an interval that updates something several times a second, you can create an animation.
You have to set a timer to execute a script after a defined time.
var timer = setTimeout(code, milliseconds);
will execute code in so-and-so milliseconds. Each execution of the script can set a new timer to execute the script again.
You can cancel a timed event using clearTimeout(timer).
Use setTimeout() or setInterval(). The MDC articles on it are pretty good.
You'll need to update inside of functions that run quickly, but get called many times, instead of updating inside of a loop.
Since you said that you are using jQuery, consider using its effects API (e.g., jQuery.animate()), it will make your life much easier!
Personally, I save as much code as possible for execution after the page has loaded, partly by putting all my <script>s at the bottom of <body>. This means a (perceived) reduction in page load time, whilst having all my JS ready to run when need be.
I wouldn't recommend going through everything you need to do at the beginning of the document. Instead, bind things to events such as clicks of buttons, etc.