I'm trying to understand how the WhatsApp Web front-end page was created, especially the list of contacts on the left side (<div id="pane-side">). The name of the contacts uses the class "emojitext ellipsify" and I am trying to get the name of all contacts by the code:
var contatosTemp = document.getElementsByClassName('emojitext ellipsify');
var contatos = [];
for (var i = 0; i < contatosTemp.length; i = i + 2) { contatos.push(contatosTemp[i].innerText); }
contatos
If the scroll of the list of contacts is at top, the number of contatos are 19. But if I scroll the list, the number of contatos is increased until the number of 29. If I minimize the window, the number of contatos is decreased to 15.
At resume, the number of the list of contacts caught by the JavaScript above changes when the user scrolls the list of contacts.
If you want to understand what I am doing, open your WhatsApp Web, go to browser DevTools > Console and run the JavaScript above.
You will see the size of the array contatos:
Then scroll the pane at left side of screen and execute the JavaScript again. It will show that the array size changed.
I would like to get all the contacts' names but JavaScript only gets part of it. How can I do that?
Use this script that will bring you all the contacts
function contactList(){
var user = Store.Chat.models;
var destiny = [];
user.forEach(function(obj){destiny.push(obj.__x_formattedTitle)});
return destiny }
Related
I have a script that basically take info from a website for multiple users, and put these info in a google spreadsheet, with one sheet per users.
I have a function that remove values of the firstline, resize every columns, and put back the setValues:
function adjustColumnsAndIgnoreFirstLine(sheet) {
Logger.log('--- Adjust columns ---')
const range = sheet.getRange("1:1")
// save the title line
const datas = range.getValues();
// clear it
range.clearContent();
// format without the title lines
var lastColumn = sheet.getLastColumn()
sheet.autoResizeColumns(1, lastColumn);
// set width to a minimum
for (var i = 1; i < 37; i++) { // fixed number of columns
if (sheet.getColumnWidth(i) < 30) {
sheet.setColumnWidth(i, 30);
}
}
// put back titles
range.setValues(datas);
}
my problem is that the script stop executing in the middle of the function. I still have the "execution please wait" popup, but in the logs, the script stopped like there was no error (execution finished) with this as the last log:
And, on the google spreadsheet:
One thing to note is that the problem doesn't comes from the script itself, as I do not encounter this problem on any of my machines, but my client does. My client ran the script on different navigator (chrome and edge), and had the same problem, but on different users (sometimes it blocks at the before-last user, sometimes at the before-before-last user...)
So I'm kinda lost on this problem...
The problem is actually a timeout. Google app script limit the execution time of a script at ~6 minutes.
There is existing issues for this
I'm trying to scrape Instagram using Python and Selenium. Goal is to get the url of all the posts, number of comments, number of likes, etc.
I was able to scrape some data but for some reason the page doesn't show more than 12 latest entries. I'm unable to figure out a way to show all the other entries. I've even tried scrolling down and then reading the page but it's only giving 12. I checked the source and am unable to find how to get the rest of the entries. It looks like the 12 entries are embedded into the script tag and I don't see it anywhere else.
driver = webdriver.Chrome('chromedriver.exe')
driver.get('https://www.instagram.com/fazeapparel/?hl=en')
source = driver.page_source
data=bs(source, 'html.parser')
body = data.find('body')
script = body.find('script', text=lambda t: t.startswith('window._sharedData'))
page_json = script.text.split(' = ', 1)[1].rstrip(';')
data = json.loads(page_json)
Using the data retrieved, I was able to find the information and collect them.
for each in data['entry_data']['ProfilePage'][0]['graphql']['user']['edge_owner_to_timeline_media']['edges']:
link = 'https://www.instagram.com'+'/p/'+each['node']['shortcode']+'/'
posttext = each['node']['edge_media_to_caption']['edges'][0]['node']['text'].replace('\n','')
comments = each['node']['edge_media_to_comment']['count']
likes = each['node']['edge_liked_by']['count']
postimage = each['node']['thumbnail_src']
isvideo = each['node']['is_video']
postdate = time.strftime('%Y %b %d %H:%M:%S', time.localtime(each['node']['taken_at_timestamp']))
links.append([link, posttext, comments, likes, postimage, isvideo, postdate])
I've even created a scroll function to scroll the window and then scraping the data but it's only returning 12.
Is there any way I can get more than 12 entries? This account has 46 entries and I'm unable to find it anywhere in the code. Please Help!
Edit: I think the data is embedded within React so it's not showing all the posts
Have you added using OpenQA.Selenium.Support.UI ? It has a WebDriverWait and you can wait for the element to be visible. Sorry for doing this in C#.
Boxes should returns all the posts.
Again, I know it isn't in Python, but I hope it helps.
IWebDriver driver = new ChromeDriver("C:\\Users\\admin\\downloads", options);
WebDriverWait wait = new WebDriverWait(driver, time);
driver.Navigate().GoToUrl("www.instagram.com\cnn");
IWebElement mainDocument = wait.Until(SeleniumExtras.WaitHelpers.ExpectedConditions.ElementExists(By.TagName("body")));
IWebElement element = mainDocument.FindElements(By.CssSelector("#react-root > section > main > div > div._2z6nI > article > div > div");
IList <IWebElement> boxes = element.FindElements(By.TagName("div"));
foreach (var posts in boxes)
{
//do stuff here
}
EDIT:
It is making a ajax call on the back end to load the next posts when you scroll. One way might be to run a script that scrolls down. You would want to call this script in selenium. I would add logic to with a timer to wait while the script runs and check if it returns "STOP". Any type of thread sleeps blocks the thread. I would use some start of timer to call the method that runs my script.
function scrollDown() {
//once this bottom element disappears we found all the posts
var bottom = document.querySelector('._4emnV')
if (bottom != null) {
window.scroll(0,999999)
}
else
{
return "STOP"
}
}
I am trying to debug a chrome extension. I have opened up the background page in the extension page but I was wondering if there was a way to look at the array type that I have.
For example, I have the code
function main() {
var cells = document.getElementsByClassName('sectionFieldInstructor');
var length = cells.length;
var professors = [];
var profCount = 0;
Where my cells is the teachers in the list. I would like to view which teachers I have in the cells array.
Also, in the inspected page, the teachers name is listed as
A. TEACHER<br/>
Would it be affected by that?
Thanks!
You can do that with Chrome developer tools. The easiset way is to add
var cells = document.getElementsByClassName('sectionFieldInstructor');
console.log(cells); //Add this line
var length = cells.length;
Then it will be displayed in your console. (There is a tab in dev tools called console.)
You can call that function from the console as well by typing:
main();
Hope this helps
I've been able to add chat functionality to my website but I'm wondering if there is a more efficient way to do it. This is the current structure of the chat:
HTML page with javascript functions:
sendChat(elmnt) {
var result = XHR("http_sendChat.aspx","POST",0,elmnt); //xmlhttprequest to send chat message by posting elmnt string to page. 0 used for not polling
}
getChat() {
var result = XHR("http_getChat.aspx","GET",50,""); //polls page every 50ms to get new chat messages
addMessageElmnt(result); //update chat window with new messages
}
On the server side in vb.net http_sendChat.aspx:
Application.Lock
Application("chat") = Application("chat") & Request.Form("message") //Global application object stores chat log
Application.Unlock
On the server side in vb.net http_getChat.aspx:
Dim chatTemp
chatTemp = Mid(Application("chat"),Session("chatIndex")) //fetches whatever chat data hasn't been fetched yet
Session("chatIndex") = Session("chatIndex") + Len(chatTemp) //set index to last read position
Response.Write(chatTemp)
There is some more code that mostly checks to make sure the users account is activated and such but as far as the chat goes, is there a better way to do this? I ask because its fairly slow when there are like 100 people logged in and using the chat.
Client has two websites: desktop and mobile (don't ask why desktop isn't responsive).
Desktop site has many pages that are actually just anchors within a page. E.g. aboutus.html#team that now has to redirect to team.html when someone is viewing on a small mobile device. And all mobile pages have different names than desktop pages (aboutus.html = about-us.html).
Rather than put code on every single page to redirect that individual page (which leaves no way to redirect the anchored URLs) it's been suggested to use an array for the desktopURLs and a related array for the mobileURLs.
So (note I omitted the // after http: so they don't show up as links since I don't have enough points to post links):
var desktopURL = ['http:domain.com/aboutus.html','http:domain.com/aboutus.html#/team','http:domain.com/contactus.html']
var mobileURL = ['http:m.domain.com/about-us.html','http:m.domain.com/team.html','http:m.domain.com/contact-us.html']
Once I have that set up, where desktopURL index 1 = mobileURL index 1.
I need to basically say this in code:
Get the window.location.href and look it up in desktopURL and return the index value.
Once you have the index value of desktopURL for the current window, update window.location.href with the object of the matching index number from the mobileURL array.
So that, when I'm a person on a phone and I go to domain.com/aboutus.html I am automatically redirected to m.domain.com/about-us.html
Note that I cannot change the names of any of the pages on the mobile site (making them match the original site would have made this easy). So I need to figure out a code way to do this and I have researched extensively but am just not sure how to achieve this simple task.
Here is the JS:
var desktopURL = ['aboutus.html','aboutus.html#team','contactus.html']
var mobileURL = ['about-us.html','team.html','contact-us.html']
//Get the name of current page
var currPage=(window.location.href).split("/")[(window.location.href).split("/").length-1];
console.log(currPage);
var index=desktopURL.indexOf(currPage);
if( /Android|webOS|iPhone|iPad|iPod|BlackBerry|IEMobile|Opera Mini/i.test(navigator.userAgent) ) {
window.location.href=mobileURL[index];
}
Check redirecttest[dot]webuda[dot]com
What you are looking to accomplish is better handled with an object lookup. Managing two arrays, as you have suggested, is very fragile solution (no real relationship exists there, which will make it difficult to scale/maintain).
var MOBILE_ROOT = 'm.domain.com';
var LOOKUP = {
'/aboutus.html': '/about-us.html',
'/aboutus.html#/team': '/team.html',
'/contactus.html': '/contact-us.html'
};
function redirect () {
var path = window.location.pathname;
var hash = window.location.hash;
window.location.href = MOBILE_ROOT + LOOKUP[ path + hash ];
}
...
if ( //user agent lookup goes here ) {
redirect();
}