I am trying to develop a chrome plugin that will record a series of click and play them back
I have seen that you can simulate a click on a DOM element but the problem is in my case, I only have a webGL canvas and the buttons are not directly accessible
I have managed to get the user's click position using this :
document.onclick = function(e)
{
var x = e.pageX;
var y = e.pageY;
console.log(x + " " + y)
//browser.runtime.sendMessage({"message": "open_new_tab", "url": "aze"});
};
But I haven't found anything to use these positions to perform a click action on the page
If I understand your question properly, you want to simulate a click on the webpage?
JavaScript has the click() method exposed for all elements:
element.click()
If you don't know the element, but just the positions:
document.elementFromPoint(x, y).click()
You can dispatch a mouse click at a specific position:
const = (x, y) => {
const ev = new MouseEvent('click', {
'view': window,
'bubbles': true,
'cancelable': true,
'screenX': x,
'screenY': y
})
const el = document.elementFromPoint(x, y);
el.dispatchEvent(ev);
}
All the above should be done via Content Script within the extension.
Related
I am developing an iOS application using Swift that incorporates web content by using WKWebView.
[URL Loading and Goal] Once a URL has been loaded inside this web view, I want to open a keyboard whenever the user wants to type.
[JavaScript Events] As browsing is based on eye coordinates, I want a JavaScript event to be triggered when the user's gaze is focused on an input field. On the basis of eye gaze coordinates, I can determine the underlying element of the webpage document. After this, I want to trigger a mouse event/touch event for clicks within the input element.
However, the system-level keyboard does not appear if I follow these steps.
Code:
touchPoint variable contains the current x and y coordinates of the eye gaze.
#IBOutlet var webView : WKWebView!
let jsStyle = """
function sendTouchEvent(x, y, element, eventType) {
const touchObj = new Touch({
identifier: Date.now(),
target: element,
clientX: x,
clientY: y,
radiusX: 2.5,
radiusY: 2.5,
rotationAngle: 10,
force: 0.5,
});
const touchEvent = new TouchEvent(eventType, {
cancelable: true,
bubbles: true,
touches: [touchObj],
targetTouches: [],
changedTouches: [touchObj],
shiftKey: true,
});
element.dispatchEvent(touchEvent);
}
function click(x, y)
{
var ev = new MouseEvent('click', {
'view': window,
'bubbles': true,
'cancelable': true,
'screenX': x,
'screenY': y
});
var el = document.elementFromPoint(x, y);
if(el.nodeName == "INPUT") {
el.autofocus = true;
el.click();
el.focus();
sendTouchEvent(x, y, el, 'touchstart');
}
try {
el.dispatchEvent(ev);
} catch(error) {
console.log(error);
}
}
click(\(touchPoint.x), \(touchPoint.y));
"""
webView.evaluateJavaScript(jsStyle, completionHandler : { result, error in
print("### Evaluate JavaScript for current click!")
} => Called when eye gaze coordinates are received.
[Temporary Solution] In order to make this work, I created a native UITextField that acts as a proxy for the webpage element and opens the keyboard by becoming the first responder when I determine using JS that the eye gaze is focused on an input element.
My goal, however, is to remove the proxy logic and replace it with JavaScript code that opens the system-level keyboard.
I'm trying to click on a Facebook web game using this js function:
onmousemove = function(e){console.log("mouse location:", e.clientX, e.clientY)}
function click(x, y)
{
var ev = new MouseEvent('click', {
'view': window,
'bubbles': true,
'cancelable': true,
'screenX': x,
'screenY': y
});
var el = document.elementFromPoint(x, y);
el.dispatchEvent(ev);
}
click(796,77)
The click works everywhere in the Facebook page, less inside the game area.
When i hover it with dev tools > inspect element, the game area show: video._1yvx
game area:
https://i.imgur.com/vctnIIA.png
dev tools: https://i.imgur.com/YqmVLYt.png
Looking for help on this
I have a button that is purposely designed to open multiple pages at once, it is the main feature of the button.
I tried using:
(1)
urls.forEach(url => {
window.open(url);
});
(2) promises with a delay on them but that did not work either.
(3) multiple a tags and trying to simulate a human click however, that did not work either.
var element = document.createElement("a");
element.href = tempUrl;
element.innerHTML = "temp";
element.id = "tempAtag";
element.target = "_blank";
document.getElementById("dashboardID").appendChild(element);
element = document.getElementById("tempAtag");
var box = element.getBoundingClientRect(),
coordX = box.left + (box.right - box.left) / 2,
coordY = box.top + (box.bottom - box.top) / 2;
var simulateMouseEvent = function (element, eventName, coordX, coordY) {
element.dispatchEvent(
new MouseEvent(eventName, {
view: window,
bubbles: true,
cancelable: true,
clientX: coordX,
clientY: coordY,
//button: 0,
})
);
};
simulateMouseEvent(element, "mousedown", coordX, coordY);
simulateMouseEvent(element, "mouseup", coordX, coordY);
simulateMouseEvent(element, "click", coordX, coordY);
This does work for the first link but I get a warning in the console saying I am trying to open multiple tabs with only one interaction. So it works for one link but not for the rest.
I got the code from Simulate a REAL HUMAN mouse click in pure javascript?
(Note: I did delete my a tags after each iteration and I did test it out, there are no duplicates)
I ran out of ideas. Any ideas?
(I have looked at other solutions and none of what I came across has worked for me)
The following works with a big IF attached.
That if is you need to click the "allow pop-up" warning that will appear in the address bar and continue to allow pop-ups from that site.
You also need to have pop-ups to load in a new tab.
I still think this is a duplicate question with Open a URL in a new tab (and not a new window)
var id = 0, u = ['12','34','56','78'];
function openNextTab(){
if (u[id]){
var x = window.open("https://some.site/id:"+ u[id],"id"+u[id]);
id++;
setTimeout(openNextTab,2000);
}
}
function openTabs(e){
e.stopPropagation();e.preventDefault();
openNextTab();
}
window.onload = function(){
var b = document.getElementById("openLinks");
b.addEventListener("click",openTabs,false);
}
how do I get the x,y coordinates from a HTML element relative to screen?
I'm using x,y from getBoundingClientRect() as below, but as you can see in the blow image, if I use move the cursor to this x,y position, the curson is in the middle between 0 and + buttons not the 5 button, which is the target button.
What am I missing?
JS code:
var e = document.querySelector('input[id=five]');"
var r = e.getBoundingClientRect();
var x = r.x;
var y = r.y;
MoveMouseTo(x,y); // imaginary call, the real cursor move is done by C# but I can share the code, as needed.
Image:
NOTE: if this aditional info may help, it's a C# application with embedded browser.
const getCoords = (e) => {
var x = e.clientX
var y = e.clientY
var xx = e.screenX
var yy = e.screenY
console.log(x, y, "client")
console.log(xx, yy, "screen")
}
You're going to want to assign this function to a onmousemove event to the outermost div containing the UI for your calculator. This should at least show you the difference between a screen X,Y and a client X,Y
https://www.w3schools.com/code/tryit.asp?filename=FVXZOK1SPTR0
You can try to add an event listener on your element and use the event to retrieve the coordinates of the mouse.
const $element = document.querySelector('input[id=five]');
$element.addEventListener('mousemove', handleMouseMove);
function handleMouseMove (event) {
console.log('Offset from the screen coordinates', event.screenX, event.screenY);
// The client refers to the element you are bound to;
console.log('Offset from the client coordinates', event.clientX, event.clientY);
};
I have an HTML5 canvas based Javascript component that needs to capture and release mouse events. In the control the user clicks an area inside it and drags to affect a change. On PC I would like the user to be able to continue dragging outside of the browser and for the canvas to receive the mouse up event if the button is released outside of the window.
However, according to my reading setCapture and releaseCapture aren't supported on Chrome.
Is there a workaround?
An article written in 2009 details how you can implement cross-browser dragging which will continue to fire mousemove events even if the user's cursor leaves the window.
http://news.qooxdoo.org/mouse-capturing
Here's the essential code from the article:
function draggable(element) {
var dragging = null;
addListener(element, "mousedown", function(e) {
var e = window.event || e;
dragging = {
mouseX: e.clientX,
mouseY: e.clientY,
startX: parseInt(element.style.left),
startY: parseInt(element.style.top)
};
if (element.setCapture) element.setCapture();
});
addListener(element, "losecapture", function() {
dragging = null;
});
addListener(document, "mouseup", function() {
dragging = null;
}, true);
var dragTarget = element.setCapture ? element : document;
addListener(dragTarget, "mousemove", function(e) {
if (!dragging) return;
var e = window.event || e;
var top = dragging.startY + (e.clientY - dragging.mouseY);
var left = dragging.startX + (e.clientX - dragging.mouseX);
element.style.top = (Math.max(0, top)) + "px";
element.style.left = (Math.max(0, left)) + "px";
}, true);
};
draggable(document.getElementById("drag"));
The article contains a pretty good explanation of what's going on, but there are a few gaps where knowledge is assumed. Basically (I think), in Chrome and Safari, if you handle mousemove on the document then, if the user clicks down and holds the mouse, the document will continue receiving mousemove events even if the cursor leaves the window. These events will not propagate to child nodes of the document, so you have to handle it at the document level.
Chrome supports setPointerCapture, which is part of the W3C Pointer events recommendation. Thus an alternative would be to use pointer events and these methods.
You might want to use the jquery Pointer Events Polyfill to support other browsers.