This Meteor code suppose to navigates to a url and take a screenshot of the last page which has a list. It works fine as is but the strange thing about this code that it calls page.screenshot twice, first without the await then inside an await Promise.all. If I change any thing it will not work, like insert await at the beginning of the first statement, or remove one of page.screenshot statements, or re-arrange them.
Any idea why this is happening and how to fix this issue please?
import { Template } from 'meteor/templating';
import './main.html';
Template.info.events({
'blur #approvalNo'(e, instance) {
let approvalNo = $(e.target).val()
Meteor.call('post99RVD', approvalNo)
},
});
import { Meteor } from 'meteor/meteor';
const puppeteer = require('puppeteer'); //maybe change const to let
const xPath = {
'post99': '//*[#id="LMi3L"]',
'inputCPA': '//*[#id="searchRVDCpaNo"]',
'type': '//*[#id="searchRVDType"]'
}
Meteor.methods({
'post99RVD': async function (approvalNo='34230') {
const browser = await puppeteer.launch({headless: false})
const page = await browser.newPage()
let url = 'https://myrta.com/rvd/'
await page.goto(url)
await page.waitForXPath(xPath.post99)
const post1999 = await page.$x("//a[contains(., 'Post-1999 RVD search')]");
await post1999[0].click()
await page.waitForXPath(xPath.type)
//Select Car in the Type drop down manu
await Promise.all([
page.waitForNavigation(),
page.select("select#searchRVDType", "Car")
]);
//Type the approval number and click the searh button
await page.click('#searchRVDCpaNo')
await page.keyboard.type(approvalNo)
let searchLink = await page.$('input[alt="Search"]')
await searchLink.click()
// the next line took the shot that belongs to the previous page
page.screenshot({path: '/screen_shots/page.png'})
await Promise.all ([
page.waitForNavigation(),
page.screenshot({path: '/screen_shots/page.png'})
])
// browser.close()
}
})
<template name="info">
<input type="text" id="approvalNo" placeholder="Approval No...">
</template>
I suspect that you are misunderstanding how Promise.all works. It does not guarantee any sequencing among the elements in the array. I suspect what you want is really just removing the Promise.all and awaiting the steps in sequence:
await page.waitForNavigation();
// only take a screenshot once navigation has completed
await page.screenshot({path: '/screen_shots/page.png'});
Related
I am attempting to scrape deck lists from aetherhub for personal use. when you get to the page you have to click to make a modal popup and then no matter what I tried I could not make it copy the text in the body of the modal. Second option is to have it copy to the clipboard and then save that to a variable and then work with the string. Bingo! I made it connect and copy and return the deck list. The problem I am having is that I can not get it to work every time. I have tried putting in waits and delays to try and see that would help but i can not seem to get it to work every time. I mostly get this error "Error: Node is either not visible or not an HTMLElement"
const puppeteer = require('puppeteer')
async function getcardlist(url) {
try {
const browser = await puppeteer.launch({headless: false})
const page = await browser.newPage()
const context = await browser.defaultBrowserContext()
await context.overridePermissions(url, ['clipboard-read'])
await page.goto(url, {waitUntil: 'load'})
const exportButton = await page.$('li.nav-item:nth-child(5) > a:nth-child(1)')
await exportButton.click()
await page.waitForSelector('a.mtgaExport')
const mtgaFormatButton = await page.$('a.mtgaExport')
await mtgaFormatButton.click()
await page.waitForSelector('#exportSimpleBtn')
const simplebutton = await page.$('#exportSimpleBtn')
await simplebutton.click()
await page.$('.modal.show', { waitUntil: 'load' })
await page.waitForSelector('.modal-footer > #exportListbtn')
const toClipBoard = await page.$('.modal-footer > #exportListbtn')
await toClipBoard.click()
const copiedText = await page.evaluate(`(async () => await navigator.clipboard.readText())()`)
await browser.close()
return copiedText
} catch (err) {
console.error(err);
}
}
getcardlist('https://aetherhub.com/Deck/rakdos-menacing-menaces')
.then(returnVal => console.log((returnVal)))
When you get the Error
"Error: Node is either not visible or not an HTMLElement"
It's basically saying that's the requested button/element is not found on the page.
so even if you want or do a page.waitForSelector you will get an error (because it does not exist in your DOM). so use headless: false, and inspect element to see if you find your selector
I'm trying to change the URL path because I have path variable I use, I don't want to do it with page.click because I reach a dead end at some point.
My code is:
const generarPDF = async (id, fecha) => {
const usuarios = await Usuarios.find();
usuarios.forEach(async dato => {
const urlBase = 'http://localhost:3000';
const urlDestino = '/monitor/604c058e90de8c58c8c5ddb3';
const navegador = await puppeteer.launch();
const pagina = await navegador.newPage();
await pagina.setViewport({ width: 1920, height: 1080 });
await pagina.goto(urlBase);
await pagina.type('#usuario', dato.usuario);
await pagina.type('#passwd', '1234');
await pagina.click('#ingresar');
await pagina.goto(urlBase+urlDestino)
await pagina.pdf({ path: 'archivos/incidencia1.pdf', format: 'A4' });
})
}
generarPDF();
These three lines are the ones I use to log in
await pagina.type('#usuario', dato.usuario);
await pagina.type('#passwd', '1234');
await pagina.click('#ingresar');
I know I login correctly, the problem is when I do the second page.goto because it logs me out, Is there any way to prevent that from happening. If I put the url manually it works, also if I do page.url() I obtain the correct url, so the problem is that it logs me out.
Thanks for any help :D
When you use .goto(...) puppeeter wait for the page loading.
When you use .click(...) , NOT. ( https://pptr.dev/#?product=Puppeteer&version=v8.0.0&show=api-pageclickselector-options )
probably you change page before the login page endpoint is fully loaded. Try replace:
await pagina.click('#ingresar')
with
const [response] = await Promise.all([
await pagina.click('#ingresar'),
page.waitForNavigation({'waitUntil':'networkidle0')
]);
Ps... usually i prefer, instead of waitForNavigation, waitForSelector. For example, if in the "login confirmation page" there is a div like: <div class="login-conf">
You can write:
await pagina.click('#ingresar');
await waitForSelector('div.login-conf', 'timeout':3000)
// raise an exception if after 3seconds the page is not loaded with this element
I am creating a bot for posting something every day on Instagram, and I want to use creator studio from Facebook. The script below works fine:
const puppeteer = require('puppeteer');
(async () => {
var username = require('./config')
var password = require('./config')
const browser = await puppeteer.launch();
const ig = await browser.newPage();
await ig.setViewport({
width: 1920,
height: 1080
})
await ig.goto('https://business.facebook.com/creatorstudio/');
await ig.click('.rwb8dzxj');
await ig.waitForSelector('#email');
await ig.type('#email', username.username);
await ig.type('#pass', username.password);
await ig.click('#loginbutton');
await ig.waitForSelector('#media_manager_chrome_bar_instagram_icon');
await ig.click('#media_manager_chrome_bar_instagram_icon');
await ig.waitForSelector('[role=presentation]');
await ig.click('[role=presentation]');
await ig.screenshot({path: 'example.png'});
await browser.close();
})().catch(err => {
console.log(err.message);
})
but when I continue and add:
await ig.waitForSelector('[role=menuitem]');
await ig.click('[role=menuitem]');
I get this error:
"Node is either not visible or not an HTMLElement"
The error caused by the click and hover methods called on an ElementHandle can be mitigated by ensuring the element is "in view" separately from the method (despite the docs stating that the method "scrolls it into view if needed", seems like sometimes the element does not become visible).
Assuming the element is indeed attached to DOM - you can check that via isConnected property, try using something like this (the sample is in TypeScript) - scrollIntoView should make sure the element is centered on:
const safeHover = async <E extends ElementHandle<Element>>(
elem: E
) => {
await elem.evaluate((e) =>
e.scrollIntoView({ block: "center", inline: "center" })
);
await elem.click();
};
Note that the sample uses elementHandle's click method, not Page's. Since you call waitForSelector before clicking, use the returned elementHandle:
//...
const pres = await ig.waitForSelector('[role=presentation]');
pres && await safeHover(pres);
//...
Also, these Q&As may prove useful:
Puppeteer in NodeJS reports 'Error: Node is either not visible or not an HTMLElement'
Puppeteer throws error with "Node not visible..."
I'm Aadarshvelu! Recently Started Testing My WebApp Code Using Jest With Puppeteer. So I Have Page Which All Credentials Have Been Filled With Puppeteer.But When SummitButton('signBtn') Clicked POST process Starts
Is There Any Test That Process POST Request?..
Or
How Do I Know Test Has Been Completely Finished?
Or
How to Get The Redirect Page URL While Test Running?
This Is My Code!
const puppeteer = require('puppeteer');
const timeOut = 100 * 1000;
test("Full E2E Test!" , async () => {
const browser = await puppeteer.launch({
headless: false,
slowMo:30,
args: ['--window-size=1920,1080']
});
const page = await browser.newPage();
await page.goto('https://mypass-webapp.herokuapp.com/signUp');
await page.click('input#email');
await page.type('input#email', 'Automation#puppeteer.com');
await page.click('input#username');
await page.type('input#username' , "puppeteer");
await page.click('input#password');
await page.type('input#password' , "puppeteer");
await page.click('#signBtn').then(await console.log(page.url()));
// Here I Need a Test That Checks The Current Page!
await browser.close();
} , timeOut);
Is There Any Test That Process POST Request?..
const [response] = await Promise.all([
page.click('input[type="submit"]'), // After clicking the submit
page.waitForNavigation() // This will set the promise to wait for navigation events
// Then the page will be send POST and navigate to target page
]);
// The promise resolved
How Do I Know Test Has Been Completely Finished?
const [response] = await Promise.all([
page.click('a.my-link'), // Clicking the link will indirectly cause a navigation
page.waitForNavigation('networkidle2') // The promise resolves after navigation has finished after no more than 2 request left
]);
// The promise resolved
How to Get The Redirect Page URL While Test Running?
For example, if the website http://example.com has a single redirect to https://example.com, then the chain will contain one request:
const response = await page.goto('http://example.com');
const chain = response.request().redirectChain();
console.log(chain.length); // Return 1
console.log(chain[0].url()); // Return string 'http://example.com'
If the website https://google.com has no redirects, then the chain will be empty:
const response = await page.goto('https://google.com');
const chain = response.request().redirectChain();
console.log(chain.length); // Return 0
await page.click('#signBtn')
After this simply make another page
const [, page2] = await browser.pages();
And here is your redirect Page Url 👇
const redirectPageUrl = page2.url();
console.log(redirectPageUrl);
I'm trying to login into Instagram with Puppeteer, but somehow I'm unable to do it.
Can you help me?
Here is the link I'm using:
https://www.instagram.com/accounts/login/
I tried different stuff. The last code I tried was this:
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.instagram.com/accounts/login/');
await page.evaluate();
await afterJS.type('#f29d14ae75303cc', 'username');
await afterJS.type('#f13459e80cdd114', 'password');
await page.pdf({path: 'page.pdf', format: 'A4'});
await browser.close();
})();
Thanks in advance!
OK you're on the right track but just need to change a few things.
Firstly, I have no idea where your afterJS variable comes from? Either way you won't need it.
You're asking for data to be typed into the username and password input fields but aren't asking puppeteer to actually click on the log in button to complete the log in process.
page.evaluate() is used to execute JavaScript code inside of the page context (ie. on the web page loaded in the remote browser). So you don't need to use it here.
I would refactor your code to look like the following:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.instagram.com/accounts/login/');
await page.waitForSelector('input[name="username"]');
await page.type('input[name="username"]', 'username');
await page.type('input[name="password"]', 'password');
await page.click('button[type="submit"]');
// Add a wait for some selector on the home page to load to ensure the next step works correctly
await page.pdf({path: 'page.pdf', format: 'A4'});
await browser.close();
})();
Hopefully this sets you down the right path to getting past the login page!
Update 1:
You've enquired about parsing the text of an element on Instagram... unfortunately I don't have an account on there myself so can't really give you an exact solution but hopefully this still proves of some value.
So you're trying to evaluate an elements text, right? You can do this as follows:
const text = await page.$eval(cssSelector, (element) => {
return element.textContent;
});
All you have to do is replace cssSelector with the selector of the element you wish to retrieve the text from.
Update 2:
OK lastly, you've enquired about scrolling down to an element within a parent element. I'm not going to steal the credit from someone else so here's the answer to that:
How to scroll to an element inside a div?
What you'll have to do is basically follow the instructions in there and get that to work with puppeteer similar to as follows:
await page.evaluate(() => {
const lastLink = document.querySelectorAll('h3 > a')[2];
const topPos = lastLink.offsetTop;
const parentDiv = document.querySelector('div[class*="eo2As"]');
parentDiv.scrollTop = topPos;
});
Bear in mind that I haven't tested that code - I've just directly followed the answer in the URL I've provided. It should work!
You can log in to Instagram using the following example code:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
// Wait until page has loaded
await page.goto('https://www.instagram.com/accounts/login/', {
waitUntil: 'networkidle0',
});
// Wait for log in form
await Promise.all([
page.waitForSelector('[name="username"]'),
page.waitForSelector('[name="password"]'),
page.waitForSelector('[name="submit"]'),
]);
// Enter username and password
await page.type('[name="username"]', 'username');
await page.type('[name="password"]', 'password');
// Submit log in credentials and wait for navigation
await Promise.all([
page.click('[type="submit"]'),
page.waitForNavigation({
waitUntil: 'networkidle0',
}),
]);
// Download PDF
await page.pdf({
path: 'page.pdf',
format: 'A4',
});
await browser.close();
})();