When I open this page with puppeteer, and try to click the element, it throws an Error when it is expected to be possible to click the element.
const puppeteer = require('puppeteer');
const url = "https://www.zapimoveis.com.br/venda/apartamentos/rj+rio-de-janeiro/";
async function run() {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto(url, {waitUntil: 'load'});
const nextPageSelector = "#list > div.pagination > #proximaPagina";
await page.waitForSelector(nextPageSelector, {visible: true})
console.log("Found the button");
await page.click(nextPageSelector)
console.log('clicked');
}
run();
Here's a working version of your code.
const puppeteer = require('puppeteer');
const url = "https://www.zapimoveis.com.br/venda/apartamentos/rj+rio-de-janeiro/";
async function run() {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto(url);
const nextPageSelector = "#list > div.pagination > #proximaPagina";
console.log("Found the button");
await page.evaluate(selector=>{
return document.querySelector(selector).click();
},nextPageSelector)
console.log('clicked');
}
run();
I personally prefer to use page.evaluate, as page.click doesn't work for me neither in some cases and you can execute whatever js on the page directly.
The only thing to know is the syntax :
- first param : function to execute
- second param : the variable(s) to be passed to this function
Found the problem.
When you page.click(selector) or ElementHandle.click(), Puppeteer scrolls to the element, find its center coordinates, and finaly clicks. It uses the function _clickablePoint at node_modules/puppeteer/lib/JSHandle.js to find the coordinates.
The problem with this website (zapimoveis) is that the scroll into the element's viewport is too slow, so Puppeteer can't find its (x,y) center.
One nice way you can click on this element is to use page.evaluate to click it using page javascript. But, there is a gambi's way that I prefer. :) Change the line await page.click(nextPageSelector) by these lines:
try { await page.click(nextPageSelector) } catch (error) {} // sacrifice click :)
await page.waitFor(3000); // time for scrolling
await page.click(nextPageSelector); // this will work
Related
My Code below tries to collect a bunch of hyper links that come under the class name ".jss2". However, I do not think the function within my page.evaluate() is working. When I run the code, the link_list const doesn't get displayed.
I ran the document.querySelectorAll on the Chrome console and that was perfectly fine - really having a hard time with this.
async function testing() {
const browser = await puppeteer.launch({headless:false});
const page = await browser.newPage();
await page.setViewport({width: 1200, height: 800});
await page.goto(url);
const link_list = await this.page.evaluate(() => {
let elements = Array.from(document.querySelectorAll(".jss2"));
let links = elements.map(element => {
return element.href;
});
return (links);
});
console.log(link_list);
}
const link_list = await page.$$eval('.classname', links => links.map(link => link.href));
Found the answer here: PUPPETEER - unable to extract elements on certain websites using page.evaluate(() => document.querySelectorAll())
Apologies for any mistake, am completely new to node.js. I am trying to get a button click event from the existing website. For example if I click login button in a www.xyz.com website I need to get that event and print button clicked in my terminal. I tried puppeteer, cheerio and axios, all these has the option to automate a click but not to read a click event. Can somebody help me with it or advice me an alternate possible approach to get it done.
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch({headless:false});
const page = await browser.newPage();
await page.goto('https://stackoverflow.com/questions/68126167/how-to-read-a-click-event-from-any-current-existing-website/68126333#68126333', {waitUntil: 'networkidle0'});
const element = window.document.getElementById("upvoteBtn")
element.onclick = function() {
console.log("Upvoted!")
}
})();
Getting this error for the above code
With puppeteer, you have 2 different scopes: Node.js scope and browser scope. Variables from one scope are not defined in other one. Here is a way to communicate between these scopes and to achieve your goal:
import puppeteer from 'puppeteer';
const browser = await puppeteer.launch({ headless: false, defaultViewport: null });
try {
const [page] = await browser.pages();
await page.goto('https://example.org/');
await page.exposeFunction('logClickedElement', (name) => {
console.log(`Clicked element: ${name}.`);
});
await page.evaluate(() => {
document.body.addEventListener('click', (event) => {
window.logClickedElement(event.target.tagName);
});
});
} catch (err) { console.error(err); }
If you know the ID of the button you could get the element then assign a function to it's onclick property. Although this may overwrite the existing functionality.
Say the upvote button had an ID of upvoteBtn (which you can give it in the elements panel).
const element = window.document.getElementById("upvoteBtn")
element.onclick = function() {
console.log("Upvoted!")
}
In the case of SO, the button retains its expected functionality and also prints to the console.
I'm using puppeteer to visit a website and click on a link. Unfortunately, I cannot get the click on the link to work. Here is a minimal non-working example:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({headless:false,});
const page = await browser.newPage();
page.setViewport({ width: 1600, height: 900 });
await page.goto('https://dl.acm.org/doi/proceedings/10.1145/3337932');
let arrMainLinks = await page.$$('.issue-item__title');
for(let i = 0; i < arrMainLinks.length; ++i)
{
await page.waitForSelector('.issue-item__title a', {waitUntil: 'networkidle2'});
const x = arrMainLinks[i];
await x.hover();
await x.click();
await page.waitForNavigation({waitUntil: 'networkidle0'});
// do things
await page.goBack();
await page.waitForSelector('body', {waitUntil: 'networkidle2'});
arrMainLinks = await page.$$('.issue-item__title a')
}
})();
Things I've tried:
The Promise.all trick a quick google search suggests
page.eval(x => x.click(), x)
waitFor(20000) before clicking
How can I get this to work and just click on the link?
You need to change the selector:
let arrMainLinks = await page.$$('.issue-item__title a');
The way it is in the question emit a click on the h2 element, changing it to the a element works.
I'm trying to figure out how to get the elements in e.g. a JS gallery that loads its images after they have been clicked on.
I'm using a demo of Viewer.js as an example. The element with the classes .viewer-move.viewer-transition isn't in the initial DOM. After clicking on an image the element is available but if I use $eval the string is empty. If I open the console in the Puppeteer browser and execute document.querySelector('.viewer-move.viewer-transition') I get the element but in the code the element isn't available.
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto('https://fengyuanchen.github.io/viewerjs/');
await page.click('[data-original="images/tibet-1.jpg"]');
let viewer = await page.$eval('.viewer-move.viewer-transition', el => el.innerHTML);
console.log(viewer);
})();
You get the empty string because the element has no content so inner HTML is empty. outerHTML seems working:
const puppeteer = require('puppeteer');
(async function main() {
try {
const browser = await puppeteer.launch({ headless: false });
const [page] = await browser.pages();
await page.goto('https://fengyuanchen.github.io/viewerjs/');
await page.click('[data-original="images/tibet-1.jpg"]');
await page.waitForSelector('.viewer-move.viewer-transition');
const viewer = await page.$eval('.viewer-move.viewer-transition', el => el.outerHTML);
console.log(viewer);
await browser.close();
} catch (err) {
console.error(err);
}
})();
Since you have to wait until it is available, the most convenient method would be to use await page.waitForSelector(".viewer-move.viewer-transition") which would wait util the element is added to DOM, although this has the caveat that this continues execution the moment that the element is added to DOM, even if it is empty/hidden.
I'm trying to fill a form with Puppeteer on a webpage, the input has an id and I'm using it as a selector.
The ID is :
#loginPage:SiteTemplate:formulaire:login-field
When I get the selector from chrome it gives me that :
#loginPage\3a SiteTemplate\3a formulaire\3a login-field
And wether I put the first or the second option in Puppeteer it spits me out this error :
Error: Evaluation failed: DOMException: Failed to execute 'querySelector' on 'Document': '#loginPage:SiteTemplate:formulaire:login-field' is not a valid selector.
Here is the code if needed :
(async () => {
const browser = await puppeteer.launch({headless: false});
const page = await browser.newPage();
await page.goto('XXX');
await page.click(GOTO_LOGIN_BUTTON_SELECTOR)
await page.waitForNavigation({waitUntil: 'load'});
await page.waitFor(EMAIL_SELECTOR); // here
await page.focus(EMAIL_SELECTOR);
await page.keyboard.type(CREDS.email);
await page.focus(PASSWORD_SELECTOR);
await page.keyboard.type(CREDS.password);
await browser.close();
})();
One option, for an ID like that, is to do as follows:
const EMAIL_SELECTOR = '[id="loginPage:SiteTemplate:formulaire:login-field"]';
Or, if that doesn't work, split it up as follows to work around the use of the ::
const EMAIL_SELECTOR = '[id*="loginPage"][id*="SiteTemplate"][id*="formulaire"][id*="login-field"]';
Hopefully one (or both) of those will help!
I'm not sure but I would drop the line await page.waitForNavigation({waitUntil: 'load'});
And replace await page.waitFor(EMAIL_SELECTOR); // here
with await page.waitForSelector(EMAIL_SELECTOR);
And test to see if just using #login-field or other #loginPage > SiteTemplate > formulaire > login-field
I could be wrong as I'm still working this out too.