How to pass the "page" element to a function with puppeteer? - javascript

I’m trying check if elements are available on a page from within a function, if the element is on the page, good, continue with the code, if not, log the error.
Using the try puppeteer page, here is what I tried:
const browser = await puppeteer.launch();
const page = await browser.newPage();
const check = element => {
try {
await page.waitFor(element, {timeout: 1000});
} catch(e) {
console.log("error : ", e)
await browser.close();
}
}
await page.goto('https://www.example.com/');
check("#something");
console.log("done")
await browser.close();
I get Error running your code. SyntaxError: Unexpected identifier. I debugged a bit and it seems that page within the check function is the unexpected identifier. So I tried to pass it in with force like this:
const browser = await puppeteer.launch();
const page = await browser.newPage();
const check = (element, page) => {
try {
await page.waitFor(element, {timeout: 1000});
} catch(e) {
console.log("error : ", e)
await browser.close();
}
}
await page.goto('https://www.example.com/');
check("#something", page);
console.log("done")
await browser.close();
but I get the same Error running your code. SyntaxError: Unexpected identifier error...
What am I doing wrong?

You can use this variant to check if the element is in the page or not.
if (await page.$(selector) !== null) console.log('found');
else console.log('not found');
Now back to your code, it's throwing error because of this function is not async,
const check = async element => { // <-- make it async
try {
await page.waitFor(element, {timeout: 1000});
} catch(e) {
console.log("error : ", e)
await browser.close();
}
}
Anytime you call await, it must be inside an async function. You cannot call await everywhere. So your check function should be called like this,
await check("#something", page);
So altogether we can rewrite the code snippet this way, you can go ahead and try this one.
const browser = await puppeteer.launch();
const page = await browser.newPage();
const check = async(element, page) => (await page.$(element) !== null); // Make it async, return true if the element is visible
await page.goto('https://www.example.com/');
// now lets check for the h1 element on example.com
const foundH1 = await check("h1", page);
console.log(`Element Found? : ${foundH1}`);
// now lets check for the h2 element on example.com
const foundH2 = await check("h2", page);
console.log(`Element Found? : ${foundH2}`);
await browser.close();
Also async functions will return promises, so you have to catch that promise or use another await. Read more about async await here:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
https://ponyfoo.com/articles/understanding-javascript-async-await

Related

How to make puppeteer return something different if a specific element gets loaded

I'm trying to get values from a page using puppeteer, the main goal is to extract a URL if the content is valid, however the page might return an error inside of a p tag.
async function example() {
const browser = await puppeteer.launch({
headless: true,
});
const page = await browser.newPage();
await page.goto('https://example.com');
// The page might load an error inside of a p element
// I don't want to block the rest of the script using await
page.waitForSelector('p', {timeout: 5000})
.then( async () => {
const pString = await page.evaluate(() => {
return document.querySelector('p').innerHTML;
});
console.log(pString);
if(pString.includes('error')) {
console.log('Page contains an error');
//i need to make the main function 'return' if this happens
}
})
// Wait for the link to load if everything is ok
await page.waitForSelector('a', {timeout: 5000});
const link = await page.$('a');
// other stuff
}
I can't figure out a proper way of doing this without having to wait for the timeout

Puppeteer not allowing me to run multiple functions in different blocks of code

I'm in the process of making an Autocheckout bot, I'm attempting to make the section that checks if the item is in stock and I want to make it all different functions in different code blocks. The problem is I cant get it to run.
When I wrap the function in () only the first function runs while the second one does nothing.
Here is the code without the () around the functions, anyone know what I'm doing wrong?
const puppeteer = require ('puppeteer');
const puppeteerExtra = require('puppeteer-extra');
const pluginStealth = require('puppeteer-extra-plugin-stealth');
const rand_url = "https://www.walmart.com/ip/Cyberpunk-2077-Warner-Bros-PlayStation-4/786104378";
async function initBrowser(){
const browser = await puppeteer.launch({args: ["--incognito"],headless:false}); //Launches browser in incognito
const context = await browser.createIncognitoBrowserContext();
const page = await context.newPage(); //Ensures the new page is also incognito
await page.evaluateOnNewDocument(() => {delete navigator.__proto__.webdriver;});
await page.goto(rand_url); //goes to given link
return page;
};
async function checkstock(page){
await page.reload();
let content = await page.evaluate(() => document.body.innerHTML)
$("link[itemprop ='availability']", content).each(function(){
let out_of_stock = $(this).attr('href').toLowerCase().includes("outofstock");
if(out_of_stock){
console.log("Out of Stock");
} else{
await browser.close();
console.log("In Stock")
//await page.waitForSelector("button[class='button spin-button prod-ProductCTA--primary button--primary']", {visible: true,}); //Waits for Add to Cart Button
//await page.$eval("button[class='button spin-button prod-ProductCTA--primary button--primary']", elem => elem.click()); //Clicks Add to cart button
}
});
};
To execute the code do it as follow, but you will get ReferenceError: $ is not defined.
const puppeteer = require ('puppeteer');
const puppeteerExtra = require('puppeteer-extra');
const pluginStealth = require('puppeteer-extra-plugin-stealth');
const rand_url = "https://www.walmart.com/ip/Cyberpunk-2077-Warner-Bros-PlayStation-4/786104378";
async function initBrowser(){
const browser = await puppeteer.launch({args: ["--incognito"],headless:false}); //Launches browser in incognito
const context = await browser.createIncognitoBrowserContext();
const page = await context.newPage(); //Ensures the new page is also incognito
await page.evaluateOnNewDocument(() => {delete navigator.__proto__.webdriver;});
await page.goto(rand_url); //goes to given link
return page;
};
async function checkstock(page){
await page.reload();
let content = await page.evaluate(() => document.body.innerHTML)
console.error(content);
$("link[itemprop ='availability']", content).each(async function(){
let out_of_stock = $(this).attr('href').toLowerCase().includes("outofstock");
if(out_of_stock){
console.log("Out of Stock");
} else{
await browser.close();
}
});
};
(async () => {
const page = await initBrowser()
await checkstock(page)
})()
I debugged your code, and after add to launch.json:
"outputCapture": "std"
I noticed that there is an error in the following line:
await browser.close();
^^^^^
SyntaxError: await is only valid in async function
You need to add async
$("link[itemprop ='availability']", content).each(async function(){

How to reload and wait for an element to appear?

I tried searching for this answer but there doesn't seem to be an answer on the Internet. What I want to do is use node js to reload a page until it finds the element with the query I want. I will be using puppeteer for other parts of the program if that will help.
Ok, I used functions from both answers and came up with this, probably unoptimized code:
const puppeteer = require("puppeteer");
(async () => {
try {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto("http://127.0.0.1:5500/main.html");
await page.waitForSelector("#buy-button");
console.log("worked");
} catch (err) {
console.log(`ERROR: ${err}`);
}
})();
But what I don't know how to do is to reload the page, and keep reloading until the id I want is there. For example, keep reloading youtube until the video you want is there(unpractical example, but I think it gets the point across).
Here's how I solved waiting for an element in puppeteer and reloading the page if it wasn't found;
async waitForSelectorWithReload(selector: string) {
const MAX_TRIES = 5;
let tries = 0;
while (tries <= MAX_TRIES) {
try {
const element = await this.page.waitForSelector(selector, {
timeout: 5000,
});
return element;
} catch (error) {
if (tries === MAX_TRIES) throw error;
tries += 1;
void this.page.reload();
await this.page.waitForNavigation({ waitUntil: 'networkidle0' });
}
}
}
And can be used as;
await waitForSelectorWithReload("input#name")
You can use "waitUntil: "networkidle2" to make sure the page is done loading. Obviously change the url, unless you are actually using evil.com
const puppeteer = require("puppeteer"); // include library
(async () =>{
const browser = await puppeteer.launch(); // run browser
const page = await browser.newPage(); // create new tab
await page.goto(
`http://www.evil.com`,
{
waitUntil: "networkidle2",
}
);
// do your stuff here
await browser.close();
})();
const puppeteer = require('puppeteer');
puppeteer.launch().then(async browser => {
const page = await browser.newPage();
page
.waitForSelector('#myId')
.then(() => console.log('got it'));
browser.close();
});

UnhandledPromiseRejectionWarning: ReferenceError: browser is not defined for azure login with puppeteer

I am trying to automate my application which is running on azure portal using puppeteer. And I am getting following error after entering the password it is not clicking the submit button.
node:55768) UnhandledPromiseRejectionWarning: ReferenceError: browser is not defined
Here is my sample code:
(async () => {
try {
const launchOptions = { headless: false, args: ['--start-maximized'] };
const browser = await puppeteer.launch(launchOptions);
const page = await browser.newPage();
await page.emulate(iPhonex);
await page.goto('https://apps.testpowerapps.com/play/72ff5b93-2327-404d-9423-92eedb44a287?tenantId=n082027');
//Enter User Name
const [userName] = await page.$x('//*[#id="i0116"]');
await userName.type("jyoti.m#azure.com");
const [loginButton] = await page.$x('//*[#id="idSIButton9"]');
await loginButton.press('Enter');
//Enter Password
const [passWord] = await page.$x('//*[#id="i0118"]');
await passWord.type("Pass123");
const [submitButton] = await page.$x('//*[#id="idSIButton9"]');
await submitButton.press('Enter');
//await page.keyboard.press('Enter');
}
catch(error){
console.error(error);
}
finally {
await browser.close();
}
})();
Tried with both way but not working only catch is the xpath for both the pages are same.
const [submitButton] = await page.$x('//*[#id="idSIButton9"]');
await submitButton.press('Enter');
//await page.keyboard.press('Enter');
any clue to resolve this.
You define the browser value in the try but you also use it in the catch. consts are block-scoped, so they are tied to the block, so a different block (the finally) can not see it.
Here is the problem:
try {
const browser = ...;
}
finally {
// different block!
await browser.close();
}
To solve this, move the browser out of the try-catch:
const browser = ...
try {
}
finally {
await browser.close();
}
This way it's available in the finally block.

How to execute a javascript code in puppeteer

I'm new to puppeteer and trying to figure out how to execute a javascript code, provided as a string value, in puppeteer.
For example, the value (which is retrieved from an input) can look like this: document.getElementById('selector').value='some_value';
I've implemented the following code
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.southwest.com/', { waitUntil: 'domcontentloaded' });
const script = await page.evaluate("document.getElementById('LandingAirBookingSearchForm_originationAirportCode').value='Dallas'; document.getElementById('LandingAirBookingSearchForm_originationAirportCode').dispatchEvent(new Event('input',{bubbles:!0}));");
await browser.close();
But it returns the following error:
Evaluation failed: TypeError: Cannot set property 'value' of null
Evaluate ur script on the page in a callback
Wait for the element with the ID of 'LandingAirBookingSearchForm_originationAirportCode' before you execute the script to be sure the side has loaded
const puppeteer = require('puppeteer');
(async function () {
const browser = await puppeteer.launch(/*{headless: false}*/);
const page = await browser.newPage();
await page.goto('https://www.southwest.com/', { waitUntil: 'domcontentloaded' });
await page.waitFor('#LandingAirBookingSearchForm_originationAirportCode');
await page.evaluate(() => {
document.getElementById('LandingAirBookingSearchForm_originationAirportCode').value='Dallas';
document.getElementById('LandingAirBookingSearchForm_originationAirportCode').dispatchEvent(new Event('input',{bubbles:!0}));
});
await browser.close();
})();

Categories

Resources