Puppeteer: How do I get the value from the loop? - javascript

I need to load the code on the page that will check the value of one element in a loop and after it becomes necessary, then return this value.
How can I do this correctly?

Your question is really vague, so I'm note sure if I understood correctly, but you may be looking for page.evaluate(). With this function you can execute code on browser context and then return some information.
Ex.:
var output = await page.evaluate(() => {
let urlSelector= '#someId > tbody > tr:nth-child(1) > td:nth-child(2) > a';
let desiredUrl = document.querySelector(urlSelector).href;
console.log(desiredUrl);//print the url in browser console
return desiredUrl;
});
console.log(output);//prints the url in node console
In this example I executed the code in browser context and returned a desired url to the "output" variable in my script.

Related

How to make the client-side code wait for the full execution of google.script.run?

I've got a Google Apps Script WebApp that relies on an array of objects that are generated from a Google Spreadsheet. The app uses jquery and miniSearch to provide user functionality.
Currently, I run the server-side function with a success handler at the beginning of the HTML tag and update a "global" variable with the array of objects declared before it.
Index.html:
<script>
let data
google.scripts.run
.withSuccessHandler(payload=>{
data = payload}).getLinks() //[{link:body}, {link1:body1}]
setTimeout(()=>{
const documents = data
miniSearch = new miniSearch(...)
miniSearch.addAll(documents)}, 2500)
...
</script>
Code.gs
function getLinks(){
.
.
.
let values = sheet.getRange(1, 1, lastRow, lastCol)
for (let row = 0; row < lastRow; row++) {
let entry = new Data(row + 1, values[row][0], values[row][1], values[row][2], values[row][3], values[row][4], values[row][5], values[row][6])
allTitles.push(entry)
}
return allTitles
}
I simulate waiting for the google.scripts.run to finish by setting a setTimeout of 2500ms on the miniSearch indexing execution (which relies on the aforementioned array) and most of the time it works.
The problem is, that before the app is ran for the first time in a given period, the contents are not cached and the execution takes longer than the setTimeout, thus, as expected, the searching function bugs out since it's got no data to run on.
My question is: How to make the code wait and confirm that google.scripts.run has returned the needed data?
I have tried doing it with regular promises or async await functions, but to my understanding, google runs its server functions asynchronously and there's no way to tell (in code, that is) if it's finished, I've tried running it as $(function(){google.script.run..}) as to try to load the contents as soon as the DOM is loaded, but to no avail..
The only way to make sure it finishes is to do this. If its unresponsive then the problem lies in getLinks, Data, or whatever miniSearch is.
<script>
const documents = null;
google.scripts.run.withSuccessHandler( function(payload) {
documents = payload;
miniSearch = new miniSearch(...);
miniSearch.addAll(documents);
}.getLinks(); //[{link:body}, {link1:body1}]
...
</script>

callback element doesn't show anything in console [duplicate]

This question already has answers here:
Puppeteer log inside page.evaluate
(13 answers)
Closed 1 year ago.
Here, I am trying to get each players runs and matches using player-URL.
When I passes selector within page.$$eval method and in callback element when I console it,I got nothing in it.
const stats = await page.$$eval(
"div.score-top.sp.text-center > div.container > div.match-summary > div.row > div.col-sm-12 > div.match-in-summary > div.row > div.col-sm-5 > div.matches-runs-wickets > ul.list-inline ",
(elements) => {
console.log("elements", elements);
return elements.map((match) => {
return (obj = {
matches: match.querySelector(" li:nth-Child(1) >span").innerText,
runs: match.querySelector("li:nth-Child(2) > span").innerText,
});
});
}
);
return stats;
}
but when I map over the callback element and return the object then I am getting runs and matches in stats variableThe code detail is mentioned here.
Everything that you console.log in eval function will be displayed in browser's console. See the console of the browser that you have opened and there you will see the logs that you are trying to print.
In your example console.log is happening not on the node.js side, but inside of a headless browser's console. You can see messages if you launch puppeteer with headless: false option.
And if you'd like to receive such console messages in terminal, see this answer.

Code working on DevTools but not working inside page.evaluate()

I'm very new to javascript and Puppeteer as well.
I'm trying to grab some innerHTML from a series of web pages inside a forum. The pages' URLs follow a pattern that has a prefix and '/page-N' at the end, N being the page number.
So I decided to loop through the pages using a for loop and template literals to load a new page URL on each loop, until I reach the final number of pages, contained in the variable C.numberOfPages.
Problem is: the code inside the page.evaluate() function is not working, when I run my code I get the TypeError: Cannot read property of undefined. I've checked and the source of the problem is that document.getElementById('discussion_subentries') is returning undefined.
I've tested the same code that is inside the page.evaluate() function in Chrome Dev Tools and it works fine, returning the innerHTML I wanted. All of those .children[] concatenations were necessary due to the structure of the page I'm scraping, and they work fine at the browser, returning the proper value.
So how do I make it work in my Puppeteer script?
for (let i = 1; i <= C.numberOfPages; i++) {
let URL = `${C.url}page-${i}`;
await page.goto(URL);
await page.waitForSelector('#discussion_subentries');
let pageData = await page.evaluate(() => {
let discussionEntries = document.getElementById('discussion_subentries')
.children[1];
let discussionEntryMessages = [];
for (let j = 0; j < discussionEntries.childElementCount; j++) {
let thisEntryMessage =
discussionEntries.children[j].children[0].children[1].children[1]
.children[1].innerHTML;
discussionEntryMessages.push(thisEntryMessage);
}
return discussionEntryMessages;
});
entryData.discussionEntryMessages.push(pageData);
}
Page evaluate is not the problem, it works 100% as the devtools. The problem is most probably that wait for selector doesnt to its proper job and doesnt wait for the element to be properly loaded before going further. Try to debug with adding some sleep instead of the wait for selector, to confirm that thats the problem.

Script not working on page load, but working from console

I wrote a script that's running from ebay listing iframe. It's working fine, it runs on $(document).ready(), sends an AJAX request to a remote server, gets some info, manipulate the DOM on 'success' callback, everything working perfect...
However, I added a piece of code, which should get the document.referrer, and extract some keywords from it, if they exist. Specifically, if a user searches ebay for a product, and clicks on my product from the results, the function extracts the keywords he entered.
Now, the problem is, that function is not running on page load at all. It seems like it blocks the script when it comes to that part. This is the function:
function getKeywords(){
var index = window.parent.document.referrer.indexOf('_nkw');
if (index >= 0){
var currentIndex = index + 5;
var keywords = '';
while (window.parent.document.referrer[currentIndex] !== '&'){
keywords += window.parent.document.referrer[currentIndex++];
}
keywords = keywords.split('+');
return keywords;
}
}
And I tried calling two logs right after it:
console.log('referrer: ' + window.parent.document.referrer);
console.log(getKeywords());
None of them is working. It's like when it comes to that 'window.parent.document.referrer' part, it stops completely.
But, when I put this all in a console, and run it, it works perfectly. It logs the right referrer, and the right keywords.
Does anybody know what might be the issue here?
The reason it is working on the console is because your window object is the outer window reference and not your iframe. Besides that, on the console:
window.parent === window
// ==> true
So, on in fact you are running window.document.referrer and not your frame's window.parent.document.referrer.
If you want to access your frame's window object you should something like
var myFrame = document.getElementsByClassName('my-frame-class')[0];
myFrame.contentWindow === window
// ==> false
myFrame.contentWindow.parent.window === window
// ==> true
This might help you debug your problem, but I guess the browser is just preventing an inner iframe from accessing the parent's window object.

run javascript via url

My Problem is:
I have an eibport:
www.bab-tec.de/
You can See a live demo in the live section.
Now the problem.
I would like to start or run a javascript on this page with entering the url like:
http:// IP-EIBPORT /#javascript:click(Button)
or in another way.
Can someone helps me?
You may want to setup your script to look for a sepcific hash and run a function based on that.
switch(location.hash){
case "#button":
//click button
break;
case "#other":
//do something else
break;
}
Then use urls like /#button or /#other
The way to do exactly what you want is the following, but it's not recommended!
eval(location.hash.substr(1));
Use a url like /#alert('test'); That will work, but it means anyone can link to your site and execute javascript which is a very unsafe thing to do.
You cannot include JavaScript-invoking code in the URL itself. The best you can do is have a script on the destination page that examines the location, and responds accordingly.
As an example, suppose you wanted to expose sum method via the url. If somebody were to load your page with the following query string: ?method=sum&params=5,5, you could have logic in place to look for these parts, and process them.
var methods = {
sum: function () {
var result = 0; i = -1;
while ( ++i < arguments.length )
result += parseInt( arguments[i], 10 );
return result;
}
};
var method = location.search.match(/method=(.*)&/)[1],
params = location.search.match(/params=(.*)$/)[1].split(",");
var result = methods[ method ].apply( this, params );
alert( result );
See it in action: http://jsfiddle.net/jonathansampson/TF5ta/show/?method=sum&params=5,5

Categories

Resources