Develop Reference

Selenium in Python, Unable to Click 'button': Tag not found

TL,DR:
For whatever reason, my selenium python script can't seem to "click" on the buttons I need.
Context:
Hello. My task is one many are probably familiar with: I'd like to automate the process of opening a website, logging in, and clicking on a few drop-down menu links within the website, which will ultimately lead me to a page where I can download a spreadsheet. I'm able to open the web page and log in. To proceed, I have to:
Click on the drop down menu header, and
in the drop down menu, click on the appropriate option.
Here's a snapshot of the pertinent HTML code from the website:
<td class="x-toolbar-cell" id="ext-gen45">
<table id="ext-comp-1045" class="x-btn x-btn-noicon" style="width: auto;" cellspacing="0">
<tbody class="x-btn-small x-btn-icon-small-left">
<tr>
<td class="x-btn-ml"><i> </i></td>
<td class="x-btn-mc">
<em class="x-btn-arrow" unselectable="on">
<button type="button" id="ext-gen46" class=" x-btn-text">Reports</button>
</em>
</td>
<td class="x-btn-mr"><i> </i></td>
</tr>
</tbody>
</table>
</td>
The item I need to "click" has a button tag, specifically:
<button type="button" id="ext-gen46" class=" x-btn-text">Reports</button>
To select it with selenium, I've tried the following:
reports_click_element = browser.find_element_by_id('ext-gen46').click()
and when that failed,
reports_element = browser.find_element_by_xpath("//button[contains(text(), 'Reports')]").click()
That one actually executed without an ExceptionMessage error, but I found out it was selecting other elements in the page that had "Reports" text, as opposed to the particular button I need.
When I've tried to zero in on the button I need clicked, the interpreter returned an error message indicating that the html attributes could not be found.
How can I proceed from here? (Should I be focusing on the unselectable="on" tag in the element right above the button I need clicked?)
Please let me know if I can add anything to the question. Thanks in advance.
Update:
I have switched into an iframe that I believe the menu is a part of- but I still cannot select the button. So far, here is my Python code:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
import time
binary = FirefoxBinary('C:\Program Files (x86)\Mozilla Firefox\Firefox.exe')
browser = webdriver.Firefox(firefox_binary=binary)
browser.get("https://app.website.com")
login_entry(username, password) # this works fine; it's just a user-created function to login. Ignore.
time.sleep(10) # wait for website's markup to load
browser.switch_to.frame(browser.find_element_by_tag_name("iframe"))
time.sleep(10)
# This is the point where I'm trying to click on the "Reports" button
reports_element = browser.find_element_by_xpath("//*[contains(text(), 'Reports')]") #this refers to other elements
reports_element = browser.find_element_by_xpath("//button[contains(text(), 'Reports')][1]") #no luck here either

A couple of cases which came to mind.
More than one element exists but not all are visible
elements = browser.find_elements_by_xpath("//button[contains(text(), 'Reports')]")
for element in elements:
if element.is_displayed():
print "element is visible"
element.click()
else:
print("element is not visible")
print(element)
The element exists, it would be visible but is out of screen.
from selenium.webdriver.common.action_chains import ActionChains
elements = browser.find_elements_by_xpath("//button[contains(text(), 'Reports')]")
for element in elements:
ActionChains(driver).move_to_element(element).perform()
try:
element.click()
except:
print("couldn't click on {}".format(element))
Can you also try to record your clicks and keyboard entries with
Selenium IDE for Firefox? Then save it as a python script and post it as a comment here?

Some ideas:
Print the DOM source of the iframe and check if the html is what you expected:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
import time
binary = FirefoxBinary('C:\Program Files (x86)\Mozilla Firefox\Firefox.exe')
browser = webdriver.Firefox(firefox_binary=binary)
browser.get("https://app.website.com")
login_entry(username, password) # this works fine; it's just a user-created function to login. Ignore.
time.sleep(10) # wait for website's markup to load
browser.switch_to.frame(browser.find_element_by_tag_name("iframe"))
time.sleep(10)
print browser.page_source
It could be, for example, that you switched to the wrong iframe, in case there is more than one in the page.
If the html is correct, then you could try to use the id of the button instead of trying to get it by text:
reports_element = browser.find_element_by_xpath("//button[#id='ext-gen46']")
If that still doesn't work, you can try to click the button with javascript:
browser.execute_script("""
(function() {
document.getElementById("ext-gen46").click();
})()
""")
You can even include jQuery by saving jQuery in a local file, then storing the contents to a variable, and running, like:
with open(JQUERY_PATH) as f:
jquery = f.read()
browser.execute_script(jquery)
Then you could use it to click the button with:
driver.execute_script("""
(function() {
jQuery(document).ready(function($) {
$('#ext-gen46').click();
}, jQuery)
})()
""")

The "page source" is only what comes with the document request. It will not show any DOM elements created via javascript after the page loads. It does sound like your elements are within some sort of iframe. In the browser console, try this and see if it returns any elements:
document.querySelectorAll('iframe')
EDIT for your update:
Once again, the page source is only what is available at document load. Everything that comes afterward that is loaded dynamically can only be seen by using the browser inspector or by getting parts of the document w/ javascript. The link being basically the same html is probably because it is a link that acts with javascript and isn't meant to lead to an actual html document page. You probably need to do in your code is:
browser.switch_to.frame(browser.find_element_by_css_selector('iframe'))
reports_element = browser.find_element_by_link_text('Reports')
reports_element.click()

I would suggest the following things to try:
Switch to the correct frame.
There may be a chance of no frame or one frame or more than one nested frames, where your element can be as a child of another. Therefore, you must switch to the right frame before you find the element. My detailed answer is here.
Use a more restricted XPATH. For example:
report_element =browser.find_element_by_xpath("//td/em/button[text()='Reports']")

You should check whether there is only one iframe on current page. You can use
len(browser.find_elements_by_tag_name('iframe'))
If output is greater than 1 and you use browser.switch_to.frame(browser.find_element_by_tag_name("iframe")), it means that you're trying to switch to first iframe found by webdriver, but you actually might need to switch to another iframe
Good solution is to find required iframe in button's ancestors with F12 or right-click on button + inspect element and use specific attributes of iframe in XPath that will match exact iframe, e.g.
browser.switch_to.frame(browser.find_element_by_xpath('//iframe[#class="iframe_class_name"]'))
Bad solution is to define index of required iframe by exhaustive search like:
browser.switch_to.frame(browser.find_elements_by_tag_name("iframe")[0])
reports_element = browser.find_element_by_xpath("//button[text()='Reports'][contains(#id, 'ext-gen')]")
reports_element.click()
...
browser.switch_to.frame(browser.find_elements_by_tag_name("iframe")[1])
reports_element = browser.find_element_by_xpath("//button[text()='Reports'][contains(#id, 'ext-gen')]")
reports_element.click()
....

I'm doing something similar with Siebel OneView, which puts most of its control buttons and menus inside Java screen elements. These elements prevent me from finding the HTML objects to activate with Selenium, so I have fallen back to using pyautogui:
import pyautogui
import time
#look for and click the Activites menu
try:
x,y = pyautogui.locateCenterOnScreen('acts_button.png')
try:
pyautogui.click(x, y)
except PermissionError:
#allow load - note this is AFTER the click permission error
time.sleep(7)
pass
except TypeError:
<code to deal with NOT finding the button>
This looks for a copy of a previously taken screenshot, stored in the same file location as the python script. In this section the image is called acts_button.png.
[Quick note: There are two try: except: statements in here. The inner one deals with any problems clicking the button as Windows will often throw out permission errors. The outer one is more important and tells your script what to do if it can't find the button. In my code, I try clicking on a preset x,y location with pyautogui.click(958, 169); if that fails, I ask the user for input (Failures detected by the next screen not loading).
The screenshots themselves are created by using commands like this
acts_button = pyautogui.screenshot('acts_button.png', region=(928,162,63,15))
Where the region is a tuple of the following parts of your target button, measured as pixels
left hand edge
top edge
width
height
Now all you need is a way to find the (x,y) values for when you take the screenshot. Fortunately I have code for that too:
#! python3
import pyautogui, sys, time
print('Press Ctrl-C to quit.')
try:
while True:
x, y = pyautogui.position()
positionStr = 'X: ' + str(x).rjust(4) + ' Y: ' + str(y).rjust(4)
print(positionStr, end='')
print('\b' * len(positionStr), end='', flush=True)
time.sleep(1)
except KeyboardInterrupt:
print('\n')
This will pop up a small window with the (x,y) coordinates of the mouse, updating once per second.
The routine to set up a new click event is therefore
Open the site you want to navigate
Use the mouse locator script to find the settings for the screenshot tuple
Use pyautogui.screenshot to save an image of the button
Use the main script to look for and click that image
After a couple of goes you should be able to set up a new click event in less than a minute. You can either use this method where you know Selenium will fail, or as a way of catching and managing exceptions when Selenium might fail.

Why not use python request (with session) and BeautifulSoup modules to do this kind of job (user interact on a website) ?

JavaScript for Acrobat: duplex printing without using booklets

I want to enable duplex-mode of a printer automatically by just opening a PDF file in a browser. this code works completely fine - also setting it to duplex-mode works. the only big problem is: it is setting the page-handling to booklet-style.
var pp = this.getPrintParams();
pp.printerName = "Company Printer Name";
pp.interactive = pp.constants.interactionLevel.silent;
pp.pageHandling = pp.constants.handling.booklet;
pp.booklet.binding = pp.constants.bookletBindings.Left;
pp.booklet.duplexMode = pp.constants.bookletDuplexModes.BothSides;
this.print(pp);
you find this example a couple of times, also in the official Adobe documentation: https://www.adobe.com/content/dam/Adobe/en/devnet/acrobat/pdfs/js_api_reference.pdf
but it's always just referring to the booklet-page style.
is there a possibility to change to duplex-mode without being in booklet mode? (preferably with pdf js)
update:
I managed to set my duplexmode successfully by changing the PDF's source adding these lines (using FPDF):
/ViewerPreferences<<
/Duplex /DuplexFlipLongEdge
>>
the duplex-mode is set in the print dialog, but it is ignored when you silently print it with adobe js. it works when you manually click the "print" button in the print dialog though.
so my "bonus"-question is: how do i automatically (silently) print it with adobe js without ignoring the document's settings? maybe some delay could help here?

SCRIPT5: Access is denied. [duplicate]

We want to reduce the number of steps it takes for a user to upload a file on our website; so we're using jQuery to open and postback files using the below markup (simplified):
<a onclick="$('#uplRegistrationImage').click();">
Change profile picture
</a>
<!-- Hidden to keep the UI clean -->
<asp:FileUpload ID="uplRegistrationImage"
runat="server"
ClientIDMode="static"
Style="display:none"
onchange="$('#btnSubmitImage').click();" />
<asp:Button runat="server"
ID="btnSubmitImage"
ClientIDMode="static"
Style="display:none"
OnClick="btnSubmitImage_OnClick"
UseSubmitBehavior="False" />
This works absolutely fine in Firefox and Chrome; opening the file dialog when the link is clicked and firing the postback when a file is selected.
However in IE9 after the file upload has loaded and a user has selected a file; insteaed of the OnChange working I get a "SCRIPT5 Access is denied" error. I've tried setting an arbitrary timeout, setting intervals to check if a file is given to no avail.
There are a number of other questions relating to this; however none appear to have a decent answer (One said set the file dialog to be transparent and hover behind a button!)
Has anyone else resolved this? Or is it absolutely necessary that I provide a button for IE users?

For security reasons, what you are trying to do is not possible. It seems to be the IE9 will not let you submit a form in this way unless it was an actual mouse click on the File Upload control that triggers it.
For arguments sake, I was able to use your code to do the submit in the change handler, but it worked only when I clicked the Browse button myself. I even set up polling in the $(document).ready method for a variable set by the change handler that indicates a submission should be triggered - this didn't work either.
The solutions to this problem appear to be:
Styling the control in such a way that it sits behind a button. You mentioned this in your question, but the answer provided by Romas here In JavaScript can I make a "click" event fire programmatically for a file input element? does in fact work (I tried in IE9, Chrome v23 and FF v15).
Using a Flash-based approach (GMail does this). I tried out the Uploadify demo and it seems to work quite nicely.
Styling a File Upload:
http://www.quirksmode.org/dom/inputfile.html
http://www.shauninman.com/archive/2007/09/10/styling_file_inputs_with_css_and_the_dom
References:
jQuery : simulating a click on a <input type="file" /> doesn't work in Firefox?
IE9 file input triggering using Javascript
getting access is denied error on IE8

Hey this solution works.
for download we should be using MSBLOB
$scope.getSingleInvoicePDF = function(invoiceNumberEntity) {
var fileName = invoiceNumberEntity + ".pdf";
var pdfDownload = document.createElement("a");
document.body.appendChild(pdfDownload);
AngularWebService.getFileWithSuffix("ezbillpdfget",invoiceNumberEntity,"pdf" ).then(function(returnedJSON) {
var fileBlob = new Blob([returnedJSON.data], {type: 'application/pdf'});
if (navigator.appVersion.toString().indexOf('.NET') > 0) { // for IE browser
window.navigator.msSaveBlob(fileBlob, fileName);
} else { // for other browsers
var fileURL = window.URL.createObjectURL(fileBlob);
pdfDownload.href = fileURL;
pdfDownload.download = fileName;
pdfDownload.click();
}
});
};

This solution looks like it might work. You'll have to wrap it in a <form> and get it to post in the jquery change handler, and probably handle it in form_load using the __eventtarget or and iframe or whatever it is that web forms uses, but it allows you to select a file, and by submitting the form, it should send it. I can't test it however, since I don't have an environment set up at home.
http://jsfiddle.net/axpLc/1/
<a onclick="$('#inputFile').click();">
Change profile picture
</a>
<div id='divHide'>
<input id='inputFile' type='file' />
</div>
$('#inputFile').change(function() { alert('ran'); });
#divHide { display:none; }

Well, like SLC stated you should utilize the <Form> tag.
First you should indicate the amount of files; which should be determined by your input fields. The second step will be to stack them into an array.
<input type="file" class="upload" name="fileX[]"/>
Then create a loop; by looping it will automatically be determined based on the input field it's currently on.
$("input[#type=file]:nth(" + n +")")
Then you'll notice that each file chosen; will replace the input name to the file-name. That should be a very, very basic way to submit multiple files through jQuery.
If you'd like a single item:
$("input[#type=file]").change(function(){
doIt(this, fileMax);
});
That should create a Div where the maximum file found; and attaches to the onEvent. The correlating code above would need these also:
var fileMax = 3;
<input type="file" class="upload" name="fileX[]" />
This should navigate the DOM parent tree; then create the fields respectively. That is one way; the other way is the one you see above with SLC. There are quite a few ways to do it; it's just how much of jQuery do you want manipulating it?
Hopefully that helps; sorry if I misunderstood your question.

Inline Editing But Instance Doesn't Exist

I have my own custom non-jQuery ajax which I use for programming web applications. I recently ran into problems with IE9 using TinyMCE, so am trying to switch to CKeditor
The editable text is being wrapped in a div, like so:
<div id='content'>
<div id='editable' contenteditable='true'>
page of inline text filled with ajax when links throughout the site are clicked
</div>
</div>
When I try to getData on the editable content using the examples in the documentation, I get an error.
I do this:
CKEDITOR.instances.editable.getData();
And get this:
Uncaught TypeError: Cannot call method 'getData' of undefined
So I figure that it doesn't know where the editor is in the dom... I've tried working through all editors to get the editor name, but that doesn't work-- no name appears to be found.
I've tried this:
for(var i in CKEDITOR.instances) {
alert(CKEDITOR.instances[i].name);
}
The alert is just blank-- so there's no name associated with it apparently.
I should also mention, that despite my best efforts, I cannot seem to get the editable text to have a menu appear above it like it does in the Massive Inline Editing Example
Thanks for any assistance you can bring.
Jason Silver
UPDATE:
I'm showing off my lack of knowledge here, but I had never come across "contenteditable='true'" before, so thought that because I was able to type inline, therefore the editor was instantiated somehow... but now I'm wondering if the editor is even being applied to my div.
UPDATE 2:
When the page is loaded and the script is initially called, the div does not exist. The editable div is sent into the DOM using AJAX. #Zee left a comment below that made me wonder if there is some other command that should be called in order to apply the editor to that div, so I created a button in the page with the following onclick as a way to test this approach: (adapted from the ajax example)
var editor,html='';config = {};editor=CKEDITOR.appendTo('editable',config, html );
That gives the following error in Chrome:
> Uncaught TypeError: Cannot call method 'equals' of undefined
> + CKEDITOR.tools.extend.getEditor ckeditor.js:101
> b ckeditor.js:252
> CKEDITOR.appendTo ckeditor.js:257
> onclick www.pediatricjunction.com:410
Am I headed in the right direction? Is there another way to programmatically tell CKEditor to apply the editor to a div?
UPDATE 3:
Thanks to #Reinmar I had something new to try. The most obvious way for me to test to see if this was the solution was to put a button above the content editable div that called CKEDITOR.inlineAll() and inline('editable') respectively:
<input type='button' onclick=\"CKEDITOR.inlineAll();\" value='InlineAll'/>
<input type='button' onclick=\"CKEDITOR.inline('editable');\" value='Inline'/>
<input type='button' onclick=\"var editor = CKEDITOR.inline( document.getElementById( 'editable' ) );\" value='getElementById'/>
This returned the same type of error in Chrome for all three buttons, namely:
Uncaught TypeError: Cannot call method 'equals' of undefined ckeditor.js:101
+ CKEDITOR.tools.extend.getEditor ckeditor.js:101
CKEDITOR.inline ckeditor.js:249
CKEDITOR.inlineAll ckeditor.js:250
onclick
UPDATE 4:
Upon further fiddling, I've tracked down the problem being related to json2007.js, which is a script I use which works with Real Simple History (RSH.js). These scripts have the purpose of tracking ajax history, so as I move forward and back through the browser, the AJAX page views is not lost.
Here's the fiddle page: http://jsfiddle.net/jasonsilver/3CqPv/2/

When you want to initialize inline editor there are two ways:
If element which is editable (has contenteditable attribute) exists when page is loaded CKEditor will automatically initialize an instance for it. Its name will be taken from that element's id or it will be editor<number>. You can find editors initialized automatically on this sample.
If this element is created dynamically, then you need to initialize editor on your own.
E.g. after appending <div id="editor" contenteditable="true">X</div> to the document you should call:
CKEDITOR.inline( 'editor' )
or
CKEDITOR.inlineAll()
See docs and docs.
You can find editor initialized this way on this sample.
The appendTo method has different use. You can initialize themed (not inline) editor inside specified element. This method also accepts data of editor (as 3rd arg), when all other methods (CKEDITOR.inline, CKEDITOR.replace, CKEDITOR.inlineAll) take data from the element they are replacing/using.
Update
I checked that libraries you use together with CKEditor are poorly written and cause errors you mentioned. Remove json2007.js and rsh.js and CKEditor works fine.

OK, so I have tracked down the problem.
The library I was using for tracking Ajax history and remembering commands for the back button, called Real Simple History, was using a script called json2007 which was intrusive and extended native prototypes to the point where things broke.
RSH.js is kind of old, and I wasn't using it to it's full potential anyway, so my final solution was to rewrite the essential code I needed for that, namely, a listener that watched for anchor (hash) changes in the URL, then parsed those changes and resubmitted the ajax command.
var current_hash = window.location.hash;
function check_hash() {
if ( window.location.hash != current_hash ) {
current_hash = window.location.hash;
refreshAjax();
}
}
hashCheck = setInterval( "check_hash()", 50 );
'refreshAjax()' was an existing function anyway, so this is actually a more elegant solution than I was using with Real Simple History.
After stripping out the json2007.js script, everything else just worked, and CKEditor is beautiful.
Thanks so much for your help, #Reinmar... I appreciate your patience and effort.

Detecting local file drag'n'drop with HTML/JavaScript

There is a HTML textarea. I'm able to catch that event when a local file is dragged and dropped onto the textarea. But how to obtain the name of the dropped file? (To be modified and inserted into the textarea finally.)
The following expressions returns None in that case:
event.dataTransfer.files
event.dataTransfer.getData('text/plain')
I made a short example for Firefox 3 that is my target platform currently.
<script>
function init() {
document.getElementById('x').addEventListener('drop', onDrop, true)
}
function onDrop(event) {
var data = event.dataTransfer.getData('text/plain')
event.preventDefault()
alert('files: ' + event.dataTransfer.files + ' && data: ' + data + '.')
}
</script>
<body onload='init()'>
<textarea cols=70 rows=20 id='x'></textarea>

This is a bit late - but I think what you are looking for is this:
event.dataTransfer.files[0].name
You can also get the following properties:
event.dataTransfer.files[0].size
event.dataTransfer.files[0].type
And you can loop thru these files with the following:
var listOfNames='';
for(var i=0,tot=event.dataTransfer.files.length; i<tot; i++){
listOfNames+=event.dataTransfer.files[i].name + '\r\n';
}
Btw - if you are using jQuery then the dataTransfer object can be found here:
event.originalEvent.dataTransfer.files[0].name

Don't know if it's still relevant, but I faced the same problem. Here's how I solved it:
Create a normal upload form with a single input field (type="file")
Add this HTML attribute to the input field:
dropzone="copy file:image/png file:image/jpg file:image/jpeg"
Set JQuery listener or whatever to catch the "drop"-event on the input field
When you drag & drop a local image on the input field, the "value" attribute is filled automatically and you can process it like any other HTML form.
I also wrapped the form into another HTML element (div), set the size of the div and set overflow:hidden via CSS - this way, you can get rid of the "browse" button. It's not nice, but it works. I also used the AjaxForm plugin to upload the image in the background - works very nice.

as far as I know, you need to obtain an instance of nsIFile in order to get the file path (the File class does not offer this feature).
This MDC page explains how to do this: https://developer.mozilla.org/En/DragDrop/Recommended_Drag_Types#file.
Note that although not listed in the previous link, obtaining an nsIFile instance requires privileges escalation (cf. my answer to Can I drag files from the desktop to a drop area in Firefox 3.5 and initiate an upload? show how to do this).

im doing it by detecting mouseover and mousedown over the "Drop" zone

Alemjerus is correct, you don't have access to what you're looking for.
The behavior you mentioned in reply to his comment is the default behavior of certain browsers. For instance, with the stackoverflow textarea for this entry, if I use Safari and drag a file into it, it places the file's path into the textarea. With firefox 3.5, on the other hand, it attempts to open the file with the browser.
Basically, the "drag and drop" functionality you're attempting to implement is something thats handled by the browser and OS on the client machine -- you can't use Javascript for this purpose.

You can not do it with Javascript because of security reasons. Javascript VM has no direct access to OS file system. You can only drag and drop text.