I am attempting to download a file from a website using Selenium and Python 3. This requires pressing a confirmation button on an overlay window. The overlay window is not within an iFrame - the HTML is simply dynamically added when the overlay appears - but Selenium is not able to find the button by xPath, returning a NoSuchElementException. Am I missing anything that would cause Selenium not to be able to see the element as it appears in the page source? So far as I can tell, Selenium should be able to locate the button with no issue.
#Initialize Driver
driver = webdriver.Safari()
cmd = "osascript -e 'tell application \"Safari\" to set bounds of front window to {0, 22, 1500, 1022}'"
os.system(cmd)
#Call up seach link
driver.get(data_url)
wait_a = WebDriverWait(driver, 15)
element = wait_a.until(EC.presence_of_element_located((By.ID, "md-input-3")))
#Initialize and send login information (defined above)
username = driver.find_element_by_id("md-input-3")
password = driver.find_element_by_id("md-input-6")
username.send_keys(crunchbase_username)
password.send_keys(crunchbase_password)
#Click login button
password.send_keys(Keys.ENTER)
#Wait for results page to finish loading
wait = WebDriverWait(driver, 15)
element = wait.until(EC.title_contains("Signals"))
time.sleep(2)
#Press Download Button
driver.find_element_by_xpath("//button[#aria-label='Export your results']").click()
time.sleep(2)
#Press csv button
driver.find_element_by_xpath("//button[#aria-label='Export to CSV']").click()
time.sleep(2)
#Confirm downlaod
driver.find_element_by_xpath("//*[#id='cdk-overlay-36']/md-dialog-container/confirmation-dialog/dialog-layout/div/md-dialog-actions/div/button[2]").click()
#Close driver
#driver.close()
The page source is overly complicated and highly stylized so I will not include it here, but a screenshot of the relevant section of the code in my browser's web inspector is below. The element which I'm trying to click is highlighted in blue.
Web Inspector Screenshot
I appreciate any help with this.
It is hard to tell without having access to the page under question and being able to see what's going. Few general points:
Try css selectors instead of xpath. They are more robust, easier to work with and fast.
Acvoid using dynamically generated IDs. In your screenshot I can't even see an id that appears in your code.
When you have more than one element of the same kind (like buttons in your case) try getting all webelements under a certain parent and test all of them for having an attribute value that you are looking for.
For example:
elemItems = driver.find_elements_by_css_selector(menuItemSelector)
for element in elements:
if element.text == "export":
elemItems[1].click()
Here, you find all the elements of a certain type (buttons for example) and select one that has "export" text in it.
Before clicking on the element, execute the following lines:
WebElement element = driver.findElement(By.xpath(" path_of_your_module "));
((JavaScriptExecutor) driver). executeScript("argument[0].scrollIntoView(true);", element);
I faced the same issue on a dialogOverlay and I fixed it. I realized that on clicking the overlay button that would bring the overlay, selenium was searching for the element before the overlay loads the dynamic content. So I did this:
def download():
global browser
notFound = True
while(notFound):
try:
elem = browser.find_element(By.ID, 'btnFormat2')
elem.click()
notFound = False
except BaseException:
print("----Error Download Element not found")
download()
The code will continuously look for the element until its loaded on the overlay.
I am trying automation test on following website - http://www.arzoo.com
when we search flight,
I am unable to click select on particular flight.
I used Xpath but it doesn't get the element if it's at bottom or middle of the page so then I need to use:
JavascriptExecutor jsx2 = (JavascriptExecutor)driver;
jsx2.executeScript("window.scrollBy(0,750)", "");
driver.findElements(By.xpath("//a[text()='Select']")).get(15).click();
but I don't want to use scroll to position. different screens will need different sizes.
I planned to use css sector but still no success.
Try the following xpath:
driver.findElement(By.xpath("//a[contains(#class, 'btn-primary')]")).click();
if not, try
driver.findElement(By.xpath("//li[contains(#id, 'result_0')]/div/div/div/div[2]/a")).click();
See the following for reference:
Get Nth child of a node using xpath
Xpath changing after the page gest loaded every time
Wrap the below code inside executeScript()
let allSelectButtons = document.querySelectorAll(".booking-item-flight-details .booking-item-arrival a");
for(i=0;i<allSelectButtons.length;i++) {
allSelectButtons[i].click();
}
TL,DR:
For whatever reason, my selenium python script can't seem to "click" on the buttons I need.
Context:
Hello. My task is one many are probably familiar with: I'd like to automate the process of opening a website, logging in, and clicking on a few drop-down menu links within the website, which will ultimately lead me to a page where I can download a spreadsheet. I'm able to open the web page and log in. To proceed, I have to:
Click on the drop down menu header, and
in the drop down menu, click on the appropriate option.
Here's a snapshot of the pertinent HTML code from the website:
<td class="x-toolbar-cell" id="ext-gen45">
<table id="ext-comp-1045" class="x-btn x-btn-noicon" style="width: auto;" cellspacing="0">
<tbody class="x-btn-small x-btn-icon-small-left">
<tr>
<td class="x-btn-ml"><i> </i></td>
<td class="x-btn-mc">
<em class="x-btn-arrow" unselectable="on">
<button type="button" id="ext-gen46" class=" x-btn-text">Reports</button>
</em>
</td>
<td class="x-btn-mr"><i> </i></td>
</tr>
</tbody>
</table>
</td>
The item I need to "click" has a button tag, specifically:
<button type="button" id="ext-gen46" class=" x-btn-text">Reports</button>
To select it with selenium, I've tried the following:
reports_click_element = browser.find_element_by_id('ext-gen46').click()
and when that failed,
reports_element = browser.find_element_by_xpath("//button[contains(text(), 'Reports')]").click()
That one actually executed without an ExceptionMessage error, but I found out it was selecting other elements in the page that had "Reports" text, as opposed to the particular button I need.
When I've tried to zero in on the button I need clicked, the interpreter returned an error message indicating that the html attributes could not be found.
How can I proceed from here? (Should I be focusing on the unselectable="on" tag in the element right above the button I need clicked?)
Please let me know if I can add anything to the question. Thanks in advance.
Update:
I have switched into an iframe that I believe the menu is a part of- but I still cannot select the button. So far, here is my Python code:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
import time
binary = FirefoxBinary('C:\Program Files (x86)\Mozilla Firefox\Firefox.exe')
browser = webdriver.Firefox(firefox_binary=binary)
browser.get("https://app.website.com")
login_entry(username, password) # this works fine; it's just a user-created function to login. Ignore.
time.sleep(10) # wait for website's markup to load
browser.switch_to.frame(browser.find_element_by_tag_name("iframe"))
time.sleep(10)
# This is the point where I'm trying to click on the "Reports" button
reports_element = browser.find_element_by_xpath("//*[contains(text(), 'Reports')]") #this refers to other elements
reports_element = browser.find_element_by_xpath("//button[contains(text(), 'Reports')][1]") #no luck here either
A couple of cases which came to mind.
More than one element exists but not all are visible
elements = browser.find_elements_by_xpath("//button[contains(text(), 'Reports')]")
for element in elements:
if element.is_displayed():
print "element is visible"
element.click()
else:
print("element is not visible")
print(element)
The element exists, it would be visible but is out of screen.
from selenium.webdriver.common.action_chains import ActionChains
elements = browser.find_elements_by_xpath("//button[contains(text(), 'Reports')]")
for element in elements:
ActionChains(driver).move_to_element(element).perform()
try:
element.click()
except:
print("couldn't click on {}".format(element))
Can you also try to record your clicks and keyboard entries with
Selenium IDE for Firefox? Then save it as a python script and post it as a comment here?
Some ideas:
Print the DOM source of the iframe and check if the html is what you expected:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
import time
binary = FirefoxBinary('C:\Program Files (x86)\Mozilla Firefox\Firefox.exe')
browser = webdriver.Firefox(firefox_binary=binary)
browser.get("https://app.website.com")
login_entry(username, password) # this works fine; it's just a user-created function to login. Ignore.
time.sleep(10) # wait for website's markup to load
browser.switch_to.frame(browser.find_element_by_tag_name("iframe"))
time.sleep(10)
print browser.page_source
It could be, for example, that you switched to the wrong iframe, in case there is more than one in the page.
If the html is correct, then you could try to use the id of the button instead of trying to get it by text:
reports_element = browser.find_element_by_xpath("//button[#id='ext-gen46']")
If that still doesn't work, you can try to click the button with javascript:
browser.execute_script("""
(function() {
document.getElementById("ext-gen46").click();
})()
""")
You can even include jQuery by saving jQuery in a local file, then storing the contents to a variable, and running, like:
with open(JQUERY_PATH) as f:
jquery = f.read()
browser.execute_script(jquery)
Then you could use it to click the button with:
driver.execute_script("""
(function() {
jQuery(document).ready(function($) {
$('#ext-gen46').click();
}, jQuery)
})()
""")
The "page source" is only what comes with the document request. It will not show any DOM elements created via javascript after the page loads. It does sound like your elements are within some sort of iframe. In the browser console, try this and see if it returns any elements:
document.querySelectorAll('iframe')
EDIT for your update:
Once again, the page source is only what is available at document load. Everything that comes afterward that is loaded dynamically can only be seen by using the browser inspector or by getting parts of the document w/ javascript. The link being basically the same html is probably because it is a link that acts with javascript and isn't meant to lead to an actual html document page. You probably need to do in your code is:
browser.switch_to.frame(browser.find_element_by_css_selector('iframe'))
reports_element = browser.find_element_by_link_text('Reports')
reports_element.click()
I would suggest the following things to try:
Switch to the correct frame.
There may be a chance of no frame or one frame or more than one nested frames, where your element can be as a child of another. Therefore, you must switch to the right frame before you find the element. My detailed answer is here.
Use a more restricted XPATH. For example:
report_element =browser.find_element_by_xpath("//td/em/button[text()='Reports']")
You should check whether there is only one iframe on current page. You can use
len(browser.find_elements_by_tag_name('iframe'))
If output is greater than 1 and you use browser.switch_to.frame(browser.find_element_by_tag_name("iframe")), it means that you're trying to switch to first iframe found by webdriver, but you actually might need to switch to another iframe
Good solution is to find required iframe in button's ancestors with F12 or right-click on button + inspect element and use specific attributes of iframe in XPath that will match exact iframe, e.g.
browser.switch_to.frame(browser.find_element_by_xpath('//iframe[#class="iframe_class_name"]'))
Bad solution is to define index of required iframe by exhaustive search like:
browser.switch_to.frame(browser.find_elements_by_tag_name("iframe")[0])
reports_element = browser.find_element_by_xpath("//button[text()='Reports'][contains(#id, 'ext-gen')]")
reports_element.click()
...
browser.switch_to.frame(browser.find_elements_by_tag_name("iframe")[1])
reports_element = browser.find_element_by_xpath("//button[text()='Reports'][contains(#id, 'ext-gen')]")
reports_element.click()
....
I'm doing something similar with Siebel OneView, which puts most of its control buttons and menus inside Java screen elements. These elements prevent me from finding the HTML objects to activate with Selenium, so I have fallen back to using pyautogui:
import pyautogui
import time
#look for and click the Activites menu
try:
x,y = pyautogui.locateCenterOnScreen('acts_button.png')
try:
pyautogui.click(x, y)
except PermissionError:
#allow load - note this is AFTER the click permission error
time.sleep(7)
pass
except TypeError:
<code to deal with NOT finding the button>
This looks for a copy of a previously taken screenshot, stored in the same file location as the python script. In this section the image is called acts_button.png.
[Quick note: There are two try: except: statements in here. The inner one deals with any problems clicking the button as Windows will often throw out permission errors. The outer one is more important and tells your script what to do if it can't find the button. In my code, I try clicking on a preset x,y location with pyautogui.click(958, 169); if that fails, I ask the user for input (Failures detected by the next screen not loading).
The screenshots themselves are created by using commands like this
acts_button = pyautogui.screenshot('acts_button.png', region=(928,162,63,15))
Where the region is a tuple of the following parts of your target button, measured as pixels
left hand edge
top edge
width
height
Now all you need is a way to find the (x,y) values for when you take the screenshot. Fortunately I have code for that too:
#! python3
import pyautogui, sys, time
print('Press Ctrl-C to quit.')
try:
while True:
x, y = pyautogui.position()
positionStr = 'X: ' + str(x).rjust(4) + ' Y: ' + str(y).rjust(4)
print(positionStr, end='')
print('\b' * len(positionStr), end='', flush=True)
time.sleep(1)
except KeyboardInterrupt:
print('\n')
This will pop up a small window with the (x,y) coordinates of the mouse, updating once per second.
The routine to set up a new click event is therefore
Open the site you want to navigate
Use the mouse locator script to find the settings for the screenshot tuple
Use pyautogui.screenshot to save an image of the button
Use the main script to look for and click that image
After a couple of goes you should be able to set up a new click event in less than a minute. You can either use this method where you know Selenium will fail, or as a way of catching and managing exceptions when Selenium might fail.
Why not use python request (with session) and BeautifulSoup modules to do this kind of job (user interact on a website) ?
Problem: Hi Guys, I'm testing a CMS tool using selenium c# but problem is to find a selector for a tiny drop down button because of random ID(all selectors). While it is generating HTML codes but i can not take the help of it as the next time when script runs it changes the IDs (Class name and all other identifiers).
Tried : i tried storing Xpaths of all drop down button on page in an array and next time clicking on the array position of the element but it didnt store any element xpath in array.
please suggest what can i do in this case, possibly its a case of java script enabled page.
HTML Code of element:
<span class="epi-extraIcon epi-pt-contextMenu epi-iconContextMenu" role="presentation" title="Display menu" data-dojo-attach-point="iconNodeMenu" _dijitmenuuniqname_51_43="1"/>
Recently I used selenium in C# and had a few problems like that.
My solution was to use XPath.
I inspected the elements that I needed with firebug (on Mozilla Firefox) to get the Xpath.
After that, I used HtmlAgilityPack nuget to load the page source and select the nodes and then I was able to get the elements.
I also disabled the JQuery animations of the page to avoid some problems.
So, my code for the selection of the nodes was something like that:
var document = new HtmlDocument();
document.LoadHtml(pageSource);
var htmlLoaded = DocumentParsing(document.DocumentNode.SelectNodes(
"/html/body/table[2]/tbody/tr/td/table[2]/tbody/tr/td[1]/font[2]/b[1] |" +
"/html/body/table[2]/tbody/tr/td/table[2]/tbody/tr/td[1]/font[2]/b[2]));
And my code for disable JQuery animations:
try
{
var js = DriverService as IJavaScriptExecutor;
js.ExecuteScript("$.fx.off = !$.fx.off;");
return true;
}
catch (Exception)
{
return false;
}
Hope that helps.
I am working on Selenium WebDriver.
I need to point the mouse to an element and perform click on it and I want to use javascript here instead of Xpaths.
The javascript of that element is not a method so that I can just fire it directly.
I am confused how to create a javascript so that the method when auto-executed should go to that object (I want to point to that object using its javascript only) and perform click.
Element's javascript:
javascript:setParam(paramOrderNbr, '4');
go('survey_editing.jsp','actMoveItemUp);
Please help!
Kumar
try this:
String cssSelector =.... //css selector of the element you want click on
JavascriptExecutor js = (JavascriptExecutor) driver;
StringBuilder stringBuilder = new StringBuilder();
stringBuilder.append("var x = $(\'"+cssSelector+"\');");
stringBuilder.append("x.click();");
js.executeScript(stringBuilder.toString());
hope this works for you
Good job.
But try to modify a lil bit your css selector.
Try simply map[name="edit_1"]> area
But before you try to execute anuthing verify with firebug ( i use firepath, firebug addon in ffox) to verify that your css selector is correct.
Then try execute the code I mentioned above. It always works.
But also is possible to try another approach. If your selenium test is connected with pointing out web element with onmousehover action handling.
Then is possible to user action builder:
WebElement mnuElement;
WebElement submnuElement;
mnEle = driver.findElement(By.Id("mnEle")).click();
sbEle = driver.findElement(By.Id("sbEle")).click();
Actions builder = new Actions(driver);
// Move cursor to the Main Menu Element
builder.moveToElement(mnEle).perform();
// Giving 5 Secs for submenu to be displayed
Thread.sleep(5000L);
// Clicking on the Hidden SubMenu
driver.findElement(By.Id("sbEle")).click();
please inform as soon as you check this one.
I've made a little investigation on your problem. And now I'ma a lil bit frustrated.
Firebug is unable to locate anything which is contained in <script> tags.
See the picture below
So if we are unable of locating element using standard tree DOM model then the last assumption is left (in my opinion). I'll share only the idea I would implement if come across with your problem. Simply try to click on fixed coordinates using js.But this is considered to be bad approach. It is explained here
So returning back to the js locating coordinates to click you can use this
Using described part we locate x, y coordinates of the element we need to locate. And using this
you can actually perform the click.
Something like that:
JavascriptExecutor js = (JavascriptExecutor) driver;
StringBuilder stringBuilder = new StringBuilder();
stringBuilder.append("x.trigger("click", [x, y]);"); //where [x,y] you've already //obtained
js.executeScript(stringBuilder.toString());
By the way, you can get to know about advanced user actions here . I find it quite helpful in some cases.
But it still seems to me that somehow it is possbile to locate your needed element in DOM.
Hope my answer helps somehow)