I have this code:
from selenium.common import TimeoutException, WebDriverException
from selenium import webdriver
import sys
class Main:
def _init_(self, page_url):
self.driver = webdriver.Chrome()
self.element_list = []
self.page_url = page_url
def javascript(self):
self.driver.get(self.page_url)
js_script = """
//Callback function
var done = arguments[arguments.length - 1];
//Take all the events
var array_events = []
var registerOuterHtml = (e) => {
array_events.push
(e.target.outerHTML)
}
var whole = (e) => {
array_events.push
(document.documentElement.outerHTML)
}
var quit = (key) => {
console.log(array_events);
(key.keyCode == 27 )? done(JSON.parse(JSON.stringify(array_events))) :
undefined
}
// Listen to the clicks
getElementHtml = document.addEventListener("click", registerOuterHtml, true)
getDOMHtml = document.addEventListener("click", whole, true)
// Listen to the key "esc" which means user has gathered all needed events
getKey = document.addEventListener("keydown", quit, true)
"""
self.driver.set_script_timeout(10000)
try:
try:
response = self.driver.execute_async_script(js_script)
print(len(response) / 2) #should print the number of times you have clicked for testing purposes
except TimeoutException:
print('Program closed after 10,000 seconds !')
self.driver.quit()
sys.exit()
except WebDriverException:
print(WebDriverException)
print('Driver may be closed incorrectly or unknown error occurred!')
return False
This python code starts a chrome browser with selenium, then the user should click some elements in the page, where the click gets some value using the javascript function being executed by driver.execute_async_script,and then the function is closed when pressing 'esc' button, closing window and returing values as well.
It all works fine, but if the user clicks some elements in the original page and then goes to a different page and clicks some more, the values from the previous page don't get returned (the DOM changes, so the new DOM with the fresh script obviously doesn't know anything what happened before)
I have tried storing the variable array_events in localStorage or sessionStorage, but it returns invalid, so I haven't been able to fix this.
I want the answer to be the JavaScript code correct and ready to be able to handle this task.
Related
I have been trying for so long to find a way to persist variables between page refreshes and different pages in one browser session opened from selenium python.
Unfortunately, neither storing variable in localStorage, sessionStorage or window.name doesn't work after testing so many times and research.
So I have resorted to a python script which continuously repeats driver.execute_script('return variable') and continue to gather data while surfing.
Data that needs to be collected, is a value of element that gets clicked, which is catched by eventListener for click and inserted to local variable I have added to the page.
This all works fine, except for the time where the element that gets clicked, is the actual button that contains a link that redirects page and changes the DOM.
My best guess is that at the same moment, the click, my JavaScript script that stores the variable, my JavaScript script that retrieves the variable, and the page redirect, all almost happen at the same time, suspecting that the change of the DOM happens before the retrieving of the variable, thus canceling any of my efforts to get that data.
This is the code:
from selenium.common import TimeoutException, WebDriverException
from selenium.webdriver.support.ui import WebDriverWait
from selenium import webdriver
class Main:
def __init__(self, page_url):
self.__driver = webdriver.Chrome()
self.__element_list = []
self.__page_url = page_url
def start(self):
program_return = []
self.__driver.get(self.__page_url)
event_js = '''
var array_events = []
var registerOuterHtml = (e) => {
array_events.push(e.target.outerHTML)
window.array_events = array_events
}
var registerUrl = (e) => {
array_events.push(document.documentElement.outerHTML)
}
getElementHtml = document.addEventListener("click", registerOuterHtml, true)
getDOMHtml = document.addEventListener("click", registerUrl, true)
'''
return_js = '''return window.array_events'''
self.__driver.set_script_timeout(10000)
self.__driver.execute_script(event_js)
try:
for _ in range(800):
if array_events := self.__driver.execute_script(return_js):
if array_events[-2:] not in program_return:
program_return.append(array_events[-2:])
else:
try:
WebDriverWait(self.__driver, 0.1).until(
lambda driver: self.__driver.current_url != self.__page_url)
except TimeoutException:
pass
else:
self.__page_url = self.__driver.current_url
self.__driver.execute_script(event_js)
except WebDriverException:
pass
finally:
print(len(program_return)) # should print total number of clicks made.
To test it out, call it like this:
Main('any url you wish').start()
And after clicking, and should at least click a button which changes the page, you can close the window manually and check the results.
Please indent the functions of the class a tab to the right, I can't format it here for the sake of my life!
Any idea or ideally a solution to this problem would be greatly appreciated.
Overall question---Taking for granted that variable persistence between different pages is not possible, How can I get the value of that variable that gets set on the time of click, before the page changes, from the same click action? (Maybe delay whole page...??)
Theoretically you can get some global data before a navigation like:
data = driver.execute_async_script("""
let [resolve] = arguments
window.unload = () => resolve(window.some_global_data)
""")
but it's likely to timeout ... Puppeteer / Playwright are better suited to things like this. There are python ports of them you might try.
I am trying to develop a tool for user to create a simple RPA for themselves by recording the clicking in browser. Using Selenium library, I can execute an event listener javascript into the browser with the return of the clicking node.
from selenium import webdriver
driver = webdriver.Chrome()
test = driver.get('https://www.google.com')
js_script = "var resp = arguments[0]; document.addEventListener('click', function(e) { resp(e.path);}, true)"
response = driver.execute_async_script(js_script)
print(response)
this only able to record the first time click in the browser. How can I continue listen for the Javascript event return in Python whenever the click is triggered in the browser?
The "execute_async_script" awaits a callback from javascript.
Once the awaited (by Python) callback "arguments[0]" is called then, even though the EventListener is still up on the browser, Python got its callback. The function execute_async_script is thus done for.
You can either create a logic in Python that evaluates the value in the response, and according to that recreates another script or do the logic directly in Javascript something like: (User needs to press "q" in order to terminate the event)
js_script = """
//Callback function
var done = arguments[arguments.length - 1];
//Take all the events
var array_events = []
var retour = (e) => {
array_events.push(e.path)
}
var quit = (key) => {
console.log(array_events);
(key.keyCode == 81 )? done(JSON.stringify(array_events)) : undefined
}
// Listen to the clicks
getPath = document.addEventListener("click", retour, true)
// Listen to the key "q" which means user has gathered all needed events
getKey = document.addEventListener("keydown", quit, true)
"""
response = driver.execute_async_script(js_script)
print(response)
Trying to iterate through a list of links that open modal popups, I'm running into an issue with the asynchronous nature of Javascript. I can loop through the links, and I can get Casperjs to click on all of the links. The popup opens up well (and I need to save the content of that popup). However, my code leads to Casperjs skipping every few links -- I suspect that's because of the delay. I need to be sure that every link is clicked and every popup saved. Any hint is highly appreciated!
I'm aware of Casperjs wait and waitForSelector functions, but no matter where I put them -- it still skips some popups. I suppose the reason for this behaviour is the delay, but increasing/decreasing the wait values and places where I tell casperjs to wait don't help.
this.then(function(){
x = 0;
this.each(links,function(self,link){
// I only need links that contain a certain string
if(link.indexOf('jugyoKmkName')>=0) {
var coursetitle = linktexts[x];
this.clickLabel(linktexts[x], 'a');
this.wait(2000, function() {
var coursetitleSplit = coursetitle.split(' ');
var courseid = coursetitleSplit[0];
//this logs the title and id in a file. Works perfectly
var line = courseid+' '+coursetitle+' \\n';
fs.write('/myappdirectory/alldata.txt', line, 'a');
//this logs the popup contents -- but it's completely out of sync
var courseinfo = this.getElementInfo('.rx-dialog-large').html
fs.write('/myappdirectory/'+courseid+'.html', courseinfo, 'w');
});
}
x++;
});
});
I'm logging two things here -- the link text (and some more information) in a running log file. That's working fine -- it catches every link correctly. The link text contains a unique id, which I'm using as a file name to save the popup contents. That's only working on every nth popup -- and the popup contents and the id are out of sync.
To be precise: The first 10 ids in the list are:
20000 -- saved with this id, but contains data of popup 20215
20160 -- saved with this id, but contains data of popup 20307
20211 -- saved with this id, but contains data of popup 20312
20214 ...etc (saved, but with popup from an ID way further down the list)
20215
20225
20235
20236
20307
20308
Obviously, I need the file 2000.html to save the contents of the popup with the ID 20000, 20160 with the contents of 20160 etc.
Presumably this.each(links,...) will run the callback synchronously rather than waiting for each this.wait() call to complete. Instead you'll want to wait until you've written your data to the filesystem before processing the next link. Consider this code instead:
this.then(function() {
function processNthLink(i) {
var self = this;
var link = links[i];
if (link.indexOf('jugyoKmkName')>=0) {
var coursetitle = linktexts[i];
self.clickLabel(linktexts[i], 'a');
self.wait(2000, function() {
var coursetitleSplit = coursetitle.split(' ');
var courseid = coursetitleSplit[0];
var line = courseid+' '+coursetitle+' \\n';
fs.write('/myappdirectory/alldata.txt', line, 'a');
var courseinfo = self.getElementInfo('.rx-dialog-large').html
fs.write('/myappdirectory/'+courseid+'.html', courseinfo, 'w');
if (i < links.length) {
processNthLink(i+1);
}
});
} else if (i < links.length) {
processNthLink(i+1);
}
}
processNthLink(0);
});
In this case the the next link will only be processed after the timeout and write to FS has been completed. In the case that the link doesn't contain the expected string, the next link is processed immediately.
Please find the summary of my query:-
Lets say I start off at my homepage :- https://example.com//homepage
At the homepage I have links to certain documents .Once I click on one of the links it takes me to a page where I am asked to fill information and I can finally submit it.
I fill the information ,click on submit it takes me to final signature page where it asks me to enter my credentials so that it can post a timestamp saying that I have authored this document at this particular time.
Now I close this window and return to the homepage and click on another document link ,WITHOUT REFRESHING THE HOMEPAGE and follow the same steps. For some some odd reason when I complete this document I see that details mentioned in the first report have been leaked\merged\overwritten into this document.BIZZARE!!
So I did a bit of research(soul searching if you may say) and I found the reason could be because the session is still having the old data from the old document ?Basically what I tried was when I manually refreshed the homepage ,this issue didn't occur. So what I am trying to do now is ,every time I SIGN off the document, i.e reach the final signature page and click no 'SUBMIT' button , I want the homepage ,viz https://example.com/homepage to refresh.
Below is the snippet of JSCRIPT that is being called when I click on 'SUBMIT' button on the signature page:-(this is basically the onclick function for that 'SUBMIT' HTML function )
function completeForm(aForm)
{
var foundError = false;
if(validateRequiredInput(aForm.loginID) &
validateRequiredInput(aForm.password) )
{
document.getElementById("submitBtn").disabled = true;
// Modified to get the newWindow request Parameter by Nidhi Prakash Srivastava for clinSIGHT release 2.0.6
var newWindow = "";
if(aForm.getNewWindow != null)
{
newWindow=aForm.getNewWindow.value;
}
// Modified to get the newEPCWindow request Parameter by Nidhi Prakash Srivastava for clinSIGHT release 2.0.10
var newEPCWindow = "";
var newQCWindow = "";
if(aForm.getNewEPCWindow != null && aForm.action.indexOf("?") < 0)
{
newEPCWindow=aForm.getNewEPCWindow.value;
aForm.action = aForm.action+"?op=complete&newWindow="+newWindow+"&newEPCWindow="+newEPCWindow;
}else if(aForm.getNewQCWindow != null && aForm.action.indexOf("?") < 0){
newQCWindow=aForm.getNewQCWindow.value;
aForm.action = aForm.action+"?op=complete&newWindow="+newWindow+"&newQCWindow="+newQCWindow;
}else if(aForm.action.indexOf("?") < 0)
{
aForm.action = aForm.action+"?op=complete&newWindow="+newWindow;
}
onsubmitFormHandler();
aForm.submit();
}
How do I go about achieveing the refresh of the homepage from here? Is this even the right approach ? Any suggestions on the same ?
Usually i don't put this kind of so specific question in SO, but i'm struggling with this issue for days, so i'm seeking for some help here.
I'm building an app to automate a task in web version of Whatsapp (https://web.whatsapp.com/). My goal is to click on a button on the interface to show some options, and then click on the same button again to hide it.
To simulate what i want to do manually :
1 - Open Whatsapp Web.
2 - Click on the 'Attach' button on the upper right corner of the interface, as shown in the image below.
3 - The attach options will show, as the image below :
4 - Click on the 'Attach' button again, and the attach options will hide.
That's it, but i want do this programatically using Javascript (pure JS, no JQuery).
To achieve the task in step 2, i'm using the code below with success :
var nodes = document.getElementsByTagName('span');
if (typeof lastElementId == 'undefined')
var lastElementId = 0;
var result = undefined;
for (var i = 0; i < nodes.length; i++) {
var h = nodes[i].outerHTML;
var flag = false;
flag = (h.toLowerCase().indexOf('data-icon="clip') > -1);
if (flag) {
result = h;
lastElementId = i;
break;
}
}
if (result !== undefined) {
function triggerMouseEvent(node, eventType) {
var clickEvent = document.createEvent('MouseEvents');
clickEvent.initEvent(eventType, true, true);
node.dispatchEvent(clickEvent);
}
triggerMouseEvent(nodes[i], "mouseover");
triggerMouseEvent(nodes[i], "mousedown");
} else {
console.log('Not found');
}
;
The code above will work to do the step 2, but won't work to do step 4. Manually when i click in the Attach button after the options are show, the options will hide. But not using my JS code.
What am i missing here ?
Thanks in advance !
To fix the closing problem:
Right click on the attach element.
Select inspect element in chrome browser
In the right panel select Event Listeners tab and find mousedown section
Click the handler code and detect that we need to pass specific screenX and screenY to satisfy this particular business logic and pass through to n.uie.requestDismiss() part which apparently does what is says.
So now we have enough information to try a possible solution, which apparently works for now. Goes like this:
const toggleAttach = () => {
// select the span with reliable identification like data-*
const clipNode = document.querySelector('[data-icon="clip"]');
// take its element, i.e. the button itself
const clipButtonNode = clipNode.parentNode;
// extract the current offset position relative to the document
// more info here https://developer.mozilla.org/en-US/docs/Web/API/Element/getBoundingClientRect
// we can use this for filling in the non-0 screenX and screenY
const clipButtonNodeClientRect = clipButtonNode.getBoundingClientRect();
clipButtonNode.dispatchEvent(new MouseEvent("mousedown", {
bubbles: true,
cancelable: true,
screenX: clipButtonNodeClientRect.x,
screenY: clipButtonNodeClientRect.y
}));
}
Now to understanding why the first mousedown works for opening:
This is much harder to reverse engineer, but what I managed to find is if you install React DevTools (since whatsapp web is written in React) extension and open its tab in DevTools you will see:
And there you will find:
So we can make a very vague conclusion that opening and closing is handled in separate functions. Rest is up to you to figure out.
Hope this helped.