Develop Reference

selenium click not working on element with onclick="" and bound event listener

I am using Selenium driver to automatically display webpages from StackOverflow. I want all the comments to be displayed in the browser, so I have code to find the elements at the bottom of the comments lists that are like this:
<a class="js-show-link comments-link " title="expand to show all comments on this post" href="#" onclick="">show <b>7</b> more comments</a>
Once I locate this element, I try a .click() method on it, but nothing happens until about the third click.
(One more thing: after I posted the question, I realize that the .click() method works if I employ it after I manually click on the link, and I reload the page. This is strange. Maybe the page needs to be in the cache?)
This used to work fine a few months ago, but now there is an event listener (a little dark icon with the letters "ev" in it) right next to this element, and the onclick attribute is an empty string.
Would the event listening bind affect the behavior of .click()? Is there a solution? Thanks.
I am using selenium in Python 2.7. Here is my code:
link = u'http://stackoverflow.com/questions/549/the-definitive-guide-to-form-based-website-authentication'
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
firefox_capabilities = DesiredCapabilities.FIREFOX
firefox_capabilities['marionette'] = True
driver = webdriver.Firefox(capabilities=firefox_capabilities)
driver.get(link)
myelement = driver.find_element_by_css_selector('[title="expand to show all comments on this post"]')
myelement.click()

Selenium in Python, Unable to Click 'button': Tag not found

TL,DR:
For whatever reason, my selenium python script can't seem to "click" on the buttons I need.
Context:
Hello. My task is one many are probably familiar with: I'd like to automate the process of opening a website, logging in, and clicking on a few drop-down menu links within the website, which will ultimately lead me to a page where I can download a spreadsheet. I'm able to open the web page and log in. To proceed, I have to:
Click on the drop down menu header, and
in the drop down menu, click on the appropriate option.
Here's a snapshot of the pertinent HTML code from the website:
<td class="x-toolbar-cell" id="ext-gen45">
<table id="ext-comp-1045" class="x-btn x-btn-noicon" style="width: auto;" cellspacing="0">
<tbody class="x-btn-small x-btn-icon-small-left">
<tr>
<td class="x-btn-ml"><i> </i></td>
<td class="x-btn-mc">
<em class="x-btn-arrow" unselectable="on">
<button type="button" id="ext-gen46" class=" x-btn-text">Reports</button>
</em>
</td>
<td class="x-btn-mr"><i> </i></td>
</tr>
</tbody>
</table>
</td>
The item I need to "click" has a button tag, specifically:
<button type="button" id="ext-gen46" class=" x-btn-text">Reports</button>
To select it with selenium, I've tried the following:
reports_click_element = browser.find_element_by_id('ext-gen46').click()
and when that failed,
reports_element = browser.find_element_by_xpath("//button[contains(text(), 'Reports')]").click()
That one actually executed without an ExceptionMessage error, but I found out it was selecting other elements in the page that had "Reports" text, as opposed to the particular button I need.
When I've tried to zero in on the button I need clicked, the interpreter returned an error message indicating that the html attributes could not be found.
How can I proceed from here? (Should I be focusing on the unselectable="on" tag in the element right above the button I need clicked?)
Please let me know if I can add anything to the question. Thanks in advance.
Update:
I have switched into an iframe that I believe the menu is a part of- but I still cannot select the button. So far, here is my Python code:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
import time
binary = FirefoxBinary('C:\Program Files (x86)\Mozilla Firefox\Firefox.exe')
browser = webdriver.Firefox(firefox_binary=binary)
browser.get("https://app.website.com")
login_entry(username, password) # this works fine; it's just a user-created function to login. Ignore.
time.sleep(10) # wait for website's markup to load
browser.switch_to.frame(browser.find_element_by_tag_name("iframe"))
time.sleep(10)
# This is the point where I'm trying to click on the "Reports" button
reports_element = browser.find_element_by_xpath("//*[contains(text(), 'Reports')]") #this refers to other elements
reports_element = browser.find_element_by_xpath("//button[contains(text(), 'Reports')][1]") #no luck here either

A couple of cases which came to mind.
More than one element exists but not all are visible
elements = browser.find_elements_by_xpath("//button[contains(text(), 'Reports')]")
for element in elements:
if element.is_displayed():
print "element is visible"
element.click()
else:
print("element is not visible")
print(element)
The element exists, it would be visible but is out of screen.
from selenium.webdriver.common.action_chains import ActionChains
elements = browser.find_elements_by_xpath("//button[contains(text(), 'Reports')]")
for element in elements:
ActionChains(driver).move_to_element(element).perform()
try:
element.click()
except:
print("couldn't click on {}".format(element))
Can you also try to record your clicks and keyboard entries with
Selenium IDE for Firefox? Then save it as a python script and post it as a comment here?

Some ideas:
Print the DOM source of the iframe and check if the html is what you expected:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
import time
binary = FirefoxBinary('C:\Program Files (x86)\Mozilla Firefox\Firefox.exe')
browser = webdriver.Firefox(firefox_binary=binary)
browser.get("https://app.website.com")
login_entry(username, password) # this works fine; it's just a user-created function to login. Ignore.
time.sleep(10) # wait for website's markup to load
browser.switch_to.frame(browser.find_element_by_tag_name("iframe"))
time.sleep(10)
print browser.page_source
It could be, for example, that you switched to the wrong iframe, in case there is more than one in the page.
If the html is correct, then you could try to use the id of the button instead of trying to get it by text:
reports_element = browser.find_element_by_xpath("//button[#id='ext-gen46']")
If that still doesn't work, you can try to click the button with javascript:
browser.execute_script("""
(function() {
document.getElementById("ext-gen46").click();
})()
""")
You can even include jQuery by saving jQuery in a local file, then storing the contents to a variable, and running, like:
with open(JQUERY_PATH) as f:
jquery = f.read()
browser.execute_script(jquery)
Then you could use it to click the button with:
driver.execute_script("""
(function() {
jQuery(document).ready(function($) {
$('#ext-gen46').click();
}, jQuery)
})()
""")

The "page source" is only what comes with the document request. It will not show any DOM elements created via javascript after the page loads. It does sound like your elements are within some sort of iframe. In the browser console, try this and see if it returns any elements:
document.querySelectorAll('iframe')
EDIT for your update:
Once again, the page source is only what is available at document load. Everything that comes afterward that is loaded dynamically can only be seen by using the browser inspector or by getting parts of the document w/ javascript. The link being basically the same html is probably because it is a link that acts with javascript and isn't meant to lead to an actual html document page. You probably need to do in your code is:
browser.switch_to.frame(browser.find_element_by_css_selector('iframe'))
reports_element = browser.find_element_by_link_text('Reports')
reports_element.click()

I would suggest the following things to try:
Switch to the correct frame.
There may be a chance of no frame or one frame or more than one nested frames, where your element can be as a child of another. Therefore, you must switch to the right frame before you find the element. My detailed answer is here.
Use a more restricted XPATH. For example:
report_element =browser.find_element_by_xpath("//td/em/button[text()='Reports']")

You should check whether there is only one iframe on current page. You can use
len(browser.find_elements_by_tag_name('iframe'))
If output is greater than 1 and you use browser.switch_to.frame(browser.find_element_by_tag_name("iframe")), it means that you're trying to switch to first iframe found by webdriver, but you actually might need to switch to another iframe
Good solution is to find required iframe in button's ancestors with F12 or right-click on button + inspect element and use specific attributes of iframe in XPath that will match exact iframe, e.g.
browser.switch_to.frame(browser.find_element_by_xpath('//iframe[#class="iframe_class_name"]'))
Bad solution is to define index of required iframe by exhaustive search like:
browser.switch_to.frame(browser.find_elements_by_tag_name("iframe")[0])
reports_element = browser.find_element_by_xpath("//button[text()='Reports'][contains(#id, 'ext-gen')]")
reports_element.click()
...
browser.switch_to.frame(browser.find_elements_by_tag_name("iframe")[1])
reports_element = browser.find_element_by_xpath("//button[text()='Reports'][contains(#id, 'ext-gen')]")
reports_element.click()
....

I'm doing something similar with Siebel OneView, which puts most of its control buttons and menus inside Java screen elements. These elements prevent me from finding the HTML objects to activate with Selenium, so I have fallen back to using pyautogui:
import pyautogui
import time
#look for and click the Activites menu
try:
x,y = pyautogui.locateCenterOnScreen('acts_button.png')
try:
pyautogui.click(x, y)
except PermissionError:
#allow load - note this is AFTER the click permission error
time.sleep(7)
pass
except TypeError:
<code to deal with NOT finding the button>
This looks for a copy of a previously taken screenshot, stored in the same file location as the python script. In this section the image is called acts_button.png.
[Quick note: There are two try: except: statements in here. The inner one deals with any problems clicking the button as Windows will often throw out permission errors. The outer one is more important and tells your script what to do if it can't find the button. In my code, I try clicking on a preset x,y location with pyautogui.click(958, 169); if that fails, I ask the user for input (Failures detected by the next screen not loading).
The screenshots themselves are created by using commands like this
acts_button = pyautogui.screenshot('acts_button.png', region=(928,162,63,15))
Where the region is a tuple of the following parts of your target button, measured as pixels
left hand edge
top edge
width
height
Now all you need is a way to find the (x,y) values for when you take the screenshot. Fortunately I have code for that too:
#! python3
import pyautogui, sys, time
print('Press Ctrl-C to quit.')
try:
while True:
x, y = pyautogui.position()
positionStr = 'X: ' + str(x).rjust(4) + ' Y: ' + str(y).rjust(4)
print(positionStr, end='')
print('\b' * len(positionStr), end='', flush=True)
time.sleep(1)
except KeyboardInterrupt:
print('\n')
This will pop up a small window with the (x,y) coordinates of the mouse, updating once per second.
The routine to set up a new click event is therefore
Open the site you want to navigate
Use the mouse locator script to find the settings for the screenshot tuple
Use pyautogui.screenshot to save an image of the button
Use the main script to look for and click that image
After a couple of goes you should be able to set up a new click event in less than a minute. You can either use this method where you know Selenium will fail, or as a way of catching and managing exceptions when Selenium might fail.

Why not use python request (with session) and BeautifulSoup modules to do this kind of job (user interact on a website) ?

Selenium c# how to findElement by JavaScript?

I'm working with an old portal and I have to use IE. There are some things that it doesn't find, cause it's part of a <td> menu, I tried to find it by XPath, but doesn't help.
I found the form is being rendered by a JavaScript function. And I'd like to click on them just to execute it, but how can I locate the page elements using selenium WebDriver??
For example: if I had this code
<div class="logout-link ng-scope"
ng-click="login("github")"
ng-show="!me" ng-controller="MenuCtrl">login</div>
How can I execute the ng-click part with the Selenium WebDriver?

Make sure you're trying to find element in the same frame it is located. Answer example: How to switch between frames in Selenium WebDriver using Java
Try to wait for element to appear and be available: http://www.seleniumhq.org/docs/04_webdriver_advanced.jsp
Hopefully you know how to find elements via JS (document.getElementsByClassName('logout-link ng-scope')) and here is answer on hot to use JS in C#: Execute JavaScript using Selenium WebDriver in C# - only difference is that you don't need to return anything - you only need to '.click()'

Why do you want to execute Javascript to locate an element??? Try using WebDriverWait to wait until element visible and clickable as below :-
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
var Login = wait.Until(ExpectedConditions.ElementToBeClickable(By.Xpath("//div[text() = 'login']")));
Login.Click();
Note :- make sure before try this element is not inside any frame
Hope it helps.....:)

Clicking on the web element you create executes the associated function. Look via CSS Selector against the ng-click:
IWebElement elem = driver.FindElement(By.CssSelector("div[ng-click=login("github")]"));
elem.click();
You could also build an action to move to the element and then click on it:
IWebElement elem = driver.FindElement(By.CssSelector("div[ng-click=login("github")]"));
Actions action = new Actions(driver);
action.MoveToElement(elem).Click().Build().Perform();

Selenium and Python can't click href="javascript:void(0);" with .click()

I have been stuck on trying to click the href="javascript:void(0);" on a page.
My problem is that I am not able to select this using selenium, and my end goal is to click on one. The page is fully loaded, and this is what all the links on the page have as the href.
my code is this:
a = soup.find_all('a')
for names in a:
try:
print (names['href'])
if names['href'] == "javascript:void(0);":
print "IM IN HUR"
names.click()
break
except:
continue
But the "name.click()" statement never works.I have not found any way to click on javascript:void(0). Any help would be much appreciated.

You have to use Selenium WebDriver to interact with web browser. Your variable names is an object of Beautiful Soup which is a Python package for parsing HTML/XML, it cannot interact with web browser. Try code below:
aElements = browser.find_elements_by_tag_name("a")
for name in aElements:
if(name.get_attribute("href") is not None and "javascript:void" in name.get_attribute("href")):
print("IM IN HUR")
name.click()
break

What exactly can cause an "HIERARCHY_REQUEST_ERR: DOM Exception 3"-Error?

How exactly does it relate to jQuery? I know the library uses native javascript functions internally, but what exactly is it trying to do whenever such a problem appears?

It means you've tried to insert a DOM node into a place in the DOM tree where it cannot go. The most common place I see this is on Safari which doesn't allow the following:
document.appendChild(document.createElement('div'));
Generally, this is just a mistake where this was actually intended:
document.body.appendChild(document.createElement('div'));
Other causes seen in the wild (summarized from comments):
You are attempting to append a node to itself
You are attempting to append null to a node
You are attempting to append a node to a text node.
Your HTML is invalid (e.g. failing to close your target node)
The browser thinks the HTML you are attempting to append is XML (fix by adding <!doctype html> to your injected HTML, or specifying the content type when fetching via XHR)

If you are getting this error due to a jquery ajax call $.ajax
Then you may need to specify what the dataType is coming back from the server. I have fixed the response a lot using this simple property.
$.ajax({
url: "URL_HERE",
dataType: "html",
success: function(response) {
$('#ELEMENT').html(response);
}
});

Specifically with jQuery you can run into this issue if forget the carets around the html tag when creating elements:
$("#target").append($("div").text("Test"));
Will raise this error because what you meant was
$("#target").append($("<div>").text("Test"));

This error can occur when you try to insert a node into the DOM which is invalid HTML, which can be something as subtle as an incorrect attribute, for example:
// <input> can have a 'type' attribute
var $input = $('<input/>').attr('type', 'text');
$holder.append($input); // OK
// <div> CANNOT have a 'type' attribute
var $div = $('<div></div>').attr('type', 'text');
$holder.append($div); // Error: HIERARCHY_REQUEST_ERR: DOM Exception 3

#Kelly Norton is right in his answer that The browser thinks the HTML you are attempting to append is XML and suggests specifying the content type when fetching via XHR.
It's true however you sometimes use third party libraries that you are not going to modify. It's JQuery UI in my case. Then you should provide the right Content-Type in the response instead of overriding the response type on JavaScript side. Set your Content-Type to text/html and you are fine.
In my case, it was as easy as renaming the file.xhtml to file.html - application server had some extension to MIME types mappings out of the box. When content is dynamic, you can set the content type of response somehow (e.g. res.setContentType("text/html") in Servlet API).

You can see these questions
Getting HIERARCHY_REQUEST_ERR when using Javascript to recursively generate a nested list
or
jQuery UI Dialog with ASP.NET button postback
The conclusion is
when you try to use function append, you should use new variable, like this example
jQuery(function() {
var dlg = jQuery("#dialog").dialog({
draggable: true,
resizable: true,
show: 'Transfer',
hide: 'Transfer',
width: 320,
autoOpen: false,
minHeight: 10,
minwidth: 10
});
dlg.parent().appendTo(jQuery("form:first"));
});
In the example above, uses the var "dlg" to run the function appendTo.
Then error “HIERARCHY_REQUEST_ERR" will not come out again.

I encountered this error when using the Google Chrome extension Sidewiki. Disabling it resolved the issue for me.

I'm going to add one more specific answer here because it was a 2 hour search for the answer...
I was trying to inject a tag into a document. The html was like this:
<map id='imageMap' name='imageMap'>
<area shape='circle' coords='55,28,5' href='#' title='1687.01 - 0 percentile' />
</map>
If you notice, the tag is closed in the preceding example (<area/>). This was not accepted in Chrome browsers. w3schools seems to think it should be closed, and I could not find the official spec on this tag, but it sure doesn't work in Chrome. Firefox will not accept it with <area/> or <area></area> or <area>. Chrome must have <area>. IE accepts anything.
Anyway, this error can be because your HTML is not correct.

I know this thread is old, but I've encountered another cause of the problem which others might find helpful. I was getting the error with Google Analytics trying to append itself to an HTML comment. The offending code:
document.documentElement.firstChild.appendChild(ga);
This was causing the error because my first element was an HTML comment (namely a Dreamweaver template code).
<!-- #BeginTemplate "/Templates/default.dwt.php" -->
I modified the offending code to something admittedly not bulletproof, but better:
document.documentElement.firstChild.nodeType===1 ? document.documentElement.firstChild.appendChild(ga) : document.documentElement.lastChild.appendChild(ga);

If you run into this problem while trying to append a node into another window in Internet Explorer, try using the HTML inside the node instead of the node itself.
myElement.appendChild(myNode.html());
IE doesn't support appending nodes to another window.

This ERROR happened to me in IE9 when I tried to appendChild an dynamically to a which already existed in a window A. Window A would create a child window B. In window B after some user action a function would run and do an appendChild on the form element in window A using window.opener.document.getElementById('formElement').appendChild(input);
This would throw an error. Same with creating the input element using document.createElement('input'); in the child window, passing it as a parameter to the window.opener window A, and there do the append. Only if I created the input element in the same window where I was going to append it, it would succeed without errors.
Thus my conclusion (please verify): no element can be dynamically created (using document.createElement) in one window, and then appended (using .appendChild) to an element in another window (without taking maybe a particular extra step I missed to ensure it is not considered XML or something). This fails in IE9 and throws the error, in FF this works fine though.
PS. I don't use jQuery.

Another reason this can come up is that you are appending before the element is ready e.g.
<body>
<script>
document.body.appendChild(foo);
</script>
</body>
</html>
In this case, you'll need to move the script after the . Not entirely sure if that's kosher, but moving the script after the body doesn't seem to help :/
Instead of moving the script, you can also do the appending in an event handler.

I got that error because I forgot to clone my element.
// creates an error
clone = $("#thing");
clone.appendTo("#somediv");
// does not
clone = $("#thing").clone();
clone.appendTo("#somediv");

Just for reference.
IE will block appending any element created in a different window context from the window context that the element is being appending to.
e.g
var childWindow = window.open('somepage.html');
//will throw the exception in IE
childWindow.document.body.appendChild(document.createElement('div'));
//will not throw exception in IE
childWindow.document.body.appendChild(childWindow.document.createElement('div'));
I haven't figured out how to create a dom element with jQuery using a different window context yet.

I get this error in IE9 if I had disabled script debugging (Internet Explorer) option. If I enable script debugging I don't see the error and the page works fine. This seems odd what is the DOM exception got to do with debugging either enabled or disabled.