DevTools Google Chrome:
On this site (https://booyah.live/users/41874362/followers), to load the complete list of followers it is necessary to keep scrolling down the page to reload more profiles, but there comes a time when the page weighs so much that the browser crashes and it ends up needing to be closed.
Is there any way to be able to collect the follow buttons without this happening?
The current script I use is:
setInterval(function(){
document.getElementById("layout-content").scrollTo(0, 50000000000000000000000000000000000000);
document.querySelectorAll('.components-button.components-button-size-mini.components-button-type-orange.desktop.components-button-inline').forEach(btn => btn.click());
}, 10)
I use setInterval to create a loop of:
1 - Scrolling the page
2 - Loading more profiles
3 - Clicking the follow buttons
My need:
For the study I'm doing for learning, the idea is that my profile follows all profiles followers of a single most famous profile in order to analyze how many people follow back on this social media.
Additional:
In this answer provided by Leftium, it is possible to follow only one profile:
https://stackoverflow.com/a/67882688/11462274
In this answer given by KCGD, it is possible to collect the entire list of followers but during this collection the profiles are not followed, it is possible to create a list and save the data, but not follow the profiles:
https://stackoverflow.com/a/67865968/11462274
I tried to contact them both, but they haven't returned yet. It was a good way but I couldn't combine the two answers so I can follow all the profiles, I thought about the possibility according to which I would collect the profiles of the KCGD response, I would follow the profiles too, but not only the first one but also the answer of the Leftium.
Would it be possible to take advantage of the loop created by the response from KCGD and from each response, already follow all profiles instead of just the first one as in Leftium's response?
I tried to create but was unsuccessful.
The browser crashes because too much memory is used. As you scroll down the page, the HTML DOM tree is extended and more avatar images are downloaded. These HTML and image resources are not necessary for your goal.
It is possible to avoid crashing by calling the (internal) Booyah API directly. This will be much faster and consume less resources since only the text is transferred. There are two API endpoints of interest:
GET /api/v3/users/[USERID]/followers?cursor=0&count=100
Gets list of followers following a certain user:
[USERID] is the ID of the user being studied (WEEDZAO's id).
cursor is where in the list of followers to start listing. When the page first loads, this is 0. As you scroll down, the following API calls increment this (101, 201, 301...)
count is how many results to return.
Since this is a GET call, you can open this URL in your browser.
POST /api/v3/users/[USERID]/followings
Follows a user (same as clicking their 'Follow' button).
Here [USERID] is ID of the user whose follower list will be updated (your own ID).
A payload must be sent that looks like this: {followee_uid: ID, source: 43}. I'm not sure what source is.
Also a CSRF header must be included.
Because this is a POST type call, it is not possible to open this type of URL directly in your browser.
DELETE /api/v3/users/[USERID]/followings
There is also an API to unfollow a user. (Just for reference).
If you call these API's from outside the browser, you probably need to send session cookies.
This script will list WEEDZAO's first 10 followers, then follow the first one from the list:
You must replace USERID and CSRF_TOKEN with your own values.
You can copy/paste this code into the browser dev console.
Alternatively, you can use this code from a web scraping framework like Puppeteer.
// Find these values in dev console "Network" tab:
var CSRF_TOKEN, USERID, USERID_TARGET, main;
USERID_TARGET = '41874362';
USERID = '12345678';
CSRF_TOKEN = 'MTYy...p8wg';
main = async function() {
var body, followers, json, options, payload, response, url;
// Get list of first 10 followers
console.log(url = `/api/v3/users/${USERID_TARGET}/followers?cursor=0&count=10`);
response = (await fetch(url));
json = (await response.json());
followers = json.follower_list;
console.table(followers);
// Follow first member from list above
console.log(url = `/api/v3/users/${USERID}/followings`);
payload = JSON.stringify({
followee_uid: followers[0].uid,
source: 43
});
response = (await fetch(url, options = {
method: 'POST',
body: payload,
headers: {
'X-CSRF-Token': CSRF_TOKEN
}
}));
body = (await response.text());
return console.log(body);
};
main();
It crashes because the interval is too fast
setInterval(function(){}, 10)
you are trying to call a scroll and click function every 10 milliseconds (that's 100 function call every 1 second). Which also interferes with the server as they fetch new users while scrolling.
Your script could work if you will adjust the interval to atleast 1000 milliseconds (1 second). Of course, it may take a while, but it will work. You should also expect that the page may become laggy specially when the page already loaded tons of users because Virtual Scrolling is not implemented in this page.
Even with slowing down the rate of the scrolling it still really bogs down the browser, the solution to this may be in the API the page contacts. To get the user's followers it contacts the site's V3 API
https://booyah.live/api/v3/users/41874362/followers?cursor=[LAST USER IN API RETURN]&count=100
to get all the users that would show up in the page. I wrote a script that can contact the api over and over again to get all the follower data, just run it in the page's console and use print() when you want to export the data
and copy/paste it into a .json file
//WARNING: THIS SCRIPT USES RECURSION, i have no clue how long the followers list goes so use at your own risk
var followers = []; //data collected from api
function getFollowers(cursor){
httpGet(`https://booyah.live/api/v3/users/41874362/followers?cursor=${cursor}&count=100`, function (data) { //returns data from API for given cursor (user at the end of last follower chunk)
console.log("got cursor: "+cursor);
var _followChunk = JSON.parse(String(data));
console.log(_followChunk)
followers.push(_followChunk.follower_list); //saves followers from chunk
var last_user = _followChunk.follower_list[_followChunk.follower_list.length - 1]; //gets last user of chunk (cursor for the next chunk)
setTimeout(function(){ //1 second timeout so that the API doesnt return "too many requests", not nessicary but you should probably leave this on
getFollowers(last_user.uid); //get next chunk
},1000)
})
}
var print = function(){console.log(JSON.stringify(followers))};
getFollowers(0); //get initial set of followers (cursor 0)
function httpGet(theUrl, callback) {
var xmlHttp = new XMLHttpRequest();
xmlHttp.open("GET", theUrl, false); // false for synchronous request
xmlHttp.setRequestHeader("Cache-Control", "no-store");
xmlHttp.send(null);
callback(xmlHttp.responseText);
};
if you really only need the button elements then the only way is to scroll all the way down for each time it loads new followers, as the page creates the elements as you scroll down
This is a fully working solution that I have tested in my own Chrome browser with a fresh account, successfully following all the follower accounts of the account you are targeting.
UPDATE (2021-06-18)
I've updated my solution to a drastically improved and faster function, rewritten with async/await. This new function reduces the estimated runtime from ~45min to ~10min. 10min is still a long while, but that's to be expected considering the large number of followers the user you are targeting has.
After a few iterations, the latest function not only improves speed, performance, and error reporting, but it also extends what is possible with the function. I provide several example below my solutions of how to use the function completely.
For the sake of de-cluttering my answer, I am removing my older function from this solution altogether, but you can still reference it in my solution's edit history if you like.
TL;DR
Here is the final, fastest, working solution. Make sure to replace PUT_YOUR_CSRF_TOKEN_HERE with your own CSRF token value. Detailed instructions on how to find your CSRF token are below.
You must run this in your console on the Booyah website in order to avoid CORS issues.
const csrf = 'PUT_YOUR_CSRF_TOKEN_HERE';
async function booyahGetAccounts(uid, type = 'followers', follow = 1) {
if (typeof uid !== 'undefined' && !isNaN(uid)) {
const loggedInUserID = window.localStorage?.loggedUID;
if (uid === 0) uid = loggedInUserID;
const unfollow = follow === -1;
if (unfollow) follow = 1;
if (loggedInUserID) {
if (csrf) {
async function getUserData(uid) {
const response = await fetch(`https://booyah.live/api/v3/users/${uid}`),
data = await response.json();
return data.user;
}
const loggedInUserData = await getUserData(loggedInUserID),
targetUserData = await getUserData(uid),
followUser = uid => fetch(`https://booyah.live/api/v3/users/${loggedInUserID}/followings`, { method: (unfollow ? 'DELETE' : 'POST'), headers: { 'X-CSRF-Token': csrf }, body: JSON.stringify({ followee_uid: uid, source: 43 }) }),
logSep = (data = '', usePad = 0) => typeof data === 'string' && usePad ? console.log((data ? data + ' ' : '').padEnd(50, '━')) : console.log('━'.repeat(50),data,'━'.repeat(50));
async function getList(uid, type, follow) {
const isLoggedInUser = uid === loggedInUserID;
if (isLoggedInUser && follow && !unfollow && type === 'followings') {
follow = 0;
console.warn('You alredy follow your followings. `follow` mode switched to `false`. Followings will be retrieved instead of followed.');
}
const userData = await getUserData(uid),
totalCount = userData[type.slice(0,-1)+'_count'] || 0,
totalCountStrLength = totalCount.toString().length;
if (totalCount) {
let userIDsLength = 0;
const userIDs = [],
nickname = userData.nickname,
nicknameStr = `${nickname ? ` of ${nickname}'s ${type}` : ''}`,
alreadyFollowedStr = uid => `User ID ${uid} already followed by ${loggedInUserData.nickname} (Account #${loggedInUserID})`;
async function followerFetch(cursor = 0) {
const fetched = [];
await fetch(`https://booyah.live/api/v3/users/${uid}/${type}?cursor=${cursor}&count=100`).then(res => res.json()).then(data => {
const list = data[type.slice(0,-1)+'_list'];
if (list?.length) fetched.push(...list.map(e => e.uid));
if (fetched.length) {
userIDs.push(...fetched);
userIDsLength += fetched.length;
if (follow) followUser(uid);
console.log(`${userIDsLength.toString().padStart(totalCountStrLength)} (${(userIDsLength / totalCount * 100).toFixed(4)}%)${nicknameStr} ${follow ? 'followed' : 'retrieved'}`);
if (fetched.length === 100) {
followerFetch(data.cursor);
} else {
console.log(`END REACHED. ${userIDsLength} accounts ${follow ? 'followed' : 'retrieved'}.`);
if (!follow) logSep(targetList);
}
}
});
}
await followerFetch();
return userIDs;
} else {
console.log(`This account has no ${type}.`);
}
}
logSep(`${follow ? 'Following' : 'Retrieving'} ${targetUserData.nickname}'s ${type}`, 1);
const targetList = await getList(uid, type, follow);
} else {
console.error('Missing CSRF token. Retrieve your CSRF token from the Network tab in your inspector by clicking into the Network tab item named "bug-report-claims" and then scrolling down in the associated details window to where you see "x-csrf-token". Copy its value and store it into a variable named "csrf" which this function will reference when you execute it.');
}
} else {
console.error('You do not appear to be logged in. Please log in and try again.');
}
} else {
console.error('UID not passed. Pass the UID of the profile you are targeting to this function.');
}
}
booyahGetAccounts(41874362);
Detailed explanation of the process
As the function runs, it logs the progress to the console, both how many users have been followed so far, and how much progress has been made percentage-wise, based on the total number of followers the profile you are targeting has.
Retrieving your CSRF token
The only manual portion of this process is retrieving your CSRF token. This is rather simple though. Once you log into Booyah, navigate to the Network tab of your Chrome console and click on the item named bug-report-claims, then scroll all the way down the details window which appears on the right. There should see x-csrf-token. Store this value as a string variable in your console as csrf, which my function will reference when it runs. This is necessary in order to use the POST method to follow users.
Here is what it will look like:
The solution
The function will loop through all users the account you are targeting follows in batches of 100 (the max amount allowed per GET request) and follow them all. When the end of each batch is met, the next batch is automatically triggered recursively.
🚀 Version 3 (Fastest and most flexible, using async/await and fetch())
My previous two solution versions (🐇 …🐢) can be referenced in this answer's edit history.
Make sure to replace PUT_YOUR_CSRF_TOKEN_HERE with your own CSRF token value. Detailed instructions on how to find your CSRF token are below.
You must run this in your console on the Booyah website in order to avoid CORS issues.
const csrf = 'PUT_YOUR_CSRF_TOKEN_HERE';
async function booyahGetAccounts(uid, type = 'followers', follow = 1) {
if (typeof uid !== 'undefined' && !isNaN(uid)) {
const loggedInUserID = window.localStorage?.loggedUID;
if (uid === 0) uid = loggedInUserID;
const unfollow = follow === -1;
if (unfollow) follow = 1;
if (loggedInUserID) {
if (csrf) {
async function getUserData(uid) {
const response = await fetch(`https://booyah.live/api/v3/users/${uid}`),
data = await response.json();
return data.user;
}
const loggedInUserData = await getUserData(loggedInUserID),
targetUserData = await getUserData(uid),
followUser = uid => fetch(`https://booyah.live/api/v3/users/${loggedInUserID}/followings`, { method: (unfollow ? 'DELETE' : 'POST'), headers: { 'X-CSRF-Token': csrf }, body: JSON.stringify({ followee_uid: uid, source: 43 }) }),
logSep = (data = '', usePad = 0) => typeof data === 'string' && usePad ? console.log((data ? data + ' ' : '').padEnd(50, '━')) : console.log('━'.repeat(50),data,'━'.repeat(50));
async function getList(uid, type, follow) {
const isLoggedInUser = uid === loggedInUserID;
if (isLoggedInUser && follow && !unfollow && type === 'followings') {
follow = 0;
console.warn('You alredy follow your followings. `follow` mode switched to `false`. Followings will be retrieved instead of followed.');
}
const userData = await getUserData(uid),
totalCount = userData[type.slice(0,-1)+'_count'] || 0,
totalCountStrLength = totalCount.toString().length;
if (totalCount) {
let userIDsLength = 0;
const userIDs = [],
nickname = userData.nickname,
nicknameStr = `${nickname ? ` of ${nickname}'s ${type}` : ''}`,
alreadyFollowedStr = uid => `User ID ${uid} already followed by ${loggedInUserData.nickname} (Account #${loggedInUserID})`;
async function followerFetch(cursor = 0) {
const fetched = [];
await fetch(`https://booyah.live/api/v3/users/${uid}/${type}?cursor=${cursor}&count=100`).then(res => res.json()).then(data => {
const list = data[type.slice(0,-1)+'_list'];
if (list?.length) fetched.push(...list.map(e => e.uid));
if (fetched.length) {
userIDs.push(...fetched);
userIDsLength += fetched.length;
if (follow) followUser(uid);
console.log(`${userIDsLength.toString().padStart(totalCountStrLength)} (${(userIDsLength / totalCount * 100).toFixed(4)}%)${nicknameStr} ${follow ? 'followed' : 'retrieved'}`);
if (fetched.length === 100) {
followerFetch(data.cursor);
} else {
console.log(`END REACHED. ${userIDsLength} accounts ${follow ? 'followed' : 'retrieved'}.`);
if (!follow) logSep(targetList);
}
}
});
}
await followerFetch();
return userIDs;
} else {
console.log(`This account has no ${type}.`);
}
}
logSep(`${follow ? 'Following' : 'Retrieving'} ${targetUserData.nickname}'s ${type}`, 1);
const targetList = await getList(uid, type, follow);
} else {
console.error('Missing CSRF token. Retrieve your CSRF token from the Network tab in your inspector by clicking into the Network tab item named "bug-report-claims" and then scrolling down in the associated details window to where you see "x-csrf-token". Copy its value and store it into a variable named "csrf" which this function will reference when you execute it.');
}
} else {
console.error('You do not appear to be logged in. Please log in and try again.');
}
} else {
console.error('UID not passed. Pass the UID of the profile you are targeting to this function.');
}
}
Usage
To run the function (for either of the above solutions), just call the function name with the desired User ID name as an argument, in your example case, 41874362. The function call would look like this:
booyahGetAccounts(41874362);
The function is quite flexible in its abilities though. booyahGetAccounts() accepts three parameters, but only the first is required.
booyahGetAccounts(
uid, // required, no default
type = 'followers', // optional, must be 'followers' or 'followings' -> default: 'followers'
follow = 1 // optional, must be 0, 1, or -1, -> default: 1 (boolean true)
)
The second parameter, type, allows you to choose whether you would like to process the targeted user's followers or followings (the users which that user follows).
The third parameter allows you to choose whether you would like to follow/unfollow the returned users or only retrieve their User IDs. This defaults to 1 (boolean true) which will follow the users returned, but if you only want to test the function and not actually follow the returned users, set this to a falsy value such as 0 or false. Using -1 will unfollow the users returned.
This function intelligently retrieves your own User ID for you from the window.localStorage object, so you don't need to retrieve that yourself. If you would like to process your own followers or followings, simply pass 0 as the main uid parameter value, and the function will default the uid to your own User ID.
Because you can't re-follow users you already follow, if you try to follow your followings, the function will produce the warning You already follow your followings. 'follow' mode switched to 'false'. Followings will be retrieved instead of followed. and instead return them as if you had set the follow parameter to false.
However, it can be very useful to process your own list. For example, if you want to follow all of your own followers back, you could do so like this:
booyahGetAccounts(0); // `type` and `follow` parameters already default to the correct values here
On the other hand, if you were strategically using a follow/unfollow technique in order to increase your number of followers and needed to unfollow all of your followers, you could do so like this:
booyahGetAccounts(0, 'followers', -1);
By setting the follow parameter value to -1, you instruct the function to run its followUser function on all returned User IDs using the DELETE method instead of the POST method, thereby unfollowing those users returned instead of following them.
Desired outcome
Function call
Follow all your own followers
booyahGetAccounts(0, 'followers');
Unfollow all your own followers
booyahGetAccounts(0, 'followers', -1);
Unfollow all your own followings
booyahGetAccounts(0, 'followings', -1);
Follow users that follow User ID #12345
booyahGetAccounts(12345, 'followers');
Follow users followed by User ID #12345
booyahGetAccounts(12345, 'followings');
Retrieve User IDs of accounts following User ID #12345
booyahGetAccounts(12345, 'followers', 0);
Retrieve User IDs of accounts followed by User ID #12345
booyahGetAccounts(12345, 'followings', 0);
Other notes
To improve the performance of this function, as it's very heavy, I've replaced all calls to userIDs.length with a dedicated userIDsLength variable which I add to using += with each iteration rather than calling length each time. Similarly, I store the length of the stringified followerCount in the variable followerCountStrLength rather than calling followerCount.toString().length with each iteration. Because this is a rather heavy function, it is possible for your browser window to crash. However, it should eventually complete.
If the page appears to crash by flickering and auto-closing the console, FIRST try to re-open the console without refreshing the page at all. In my case, the inspector occasionally closed on its own, likely due to the exhaustion from the function, but when I opened the inspector's console again, the function was still running.
I got a click handler that, when clicked, for some reason is updating my value twice (incrementing or decrementing twice). It doesn't happen all the time, but I would say 90% of the time. I want to prevent that and only update the value once.
Let me explain.
First, here's my button template code:
<b-button
variant="link"
class="mb-2"
#click="clueHandler(comment)"
>
Here is the script code for the clueHandler:
async clueHandler(comment) {
const inClues =
this.clues.findIndex((clue) => clue.id === comment.id) > -1
if (this.loggedIn) {
if (inClues) {
await this.$store.dispatch('removeCommentFromCluesFeed', comment)
console.log('dispatched from clueHandler')
} else if (!inClues) {
await this.$store.dispatch('addCommentToCluesFeed', comment)
console.log('dispatched from clueHandler')
}
} else {
this.$router.replace('/login')
}
},
Here's the logic in Vuex for updating the Firestore document property:
async addCommentToCluesFeed({ state }, comment) {
try {
const cluesFeedDoc = this.$fireStore
.collection(`users/${state.userProfile.uid}/clues`)
.doc(comment.id)
await cluesFeedDoc.set(comment)
console.log('clue added from addCommentToCluesFeed action')
await this.$fireStore
.collection('comments')
.doc(comment.id)
.update({
clueVotes: parseInt(comment.clueVotes) + 1 // <<<<------------HERE !!
})
console.log('clue vote increased from addCommentToCluesFeed action')
} catch (error) {
console.error(
'error adding clue from addCommentToCluesFeed action',
error
)
}
},
Here's a screenshot of the area the handler is updating:
What happens is that when you click ONCE on the template button with the clueHandler, the document property clueVotes is incremented twice (or decremented twice depending on whether the clue was already set by the user previously).
Here is an example of what the database shows when a user clicks the button for the first time:
It should only be clueVotes: 1.
Anyone have any advice or thoughts on what is happening or how I can fix? I am assuming this is going to be related to not fully realizing the promise/race completion, etc. Thanks!
I found an excellent explanation of why I was having this issue. Here it is:
https://fireship.io/snippets/firestore-increment-tips/
Basically, I needed to return a special value that can be used with set() or update() that tells the server to increment the field's current value by the given value.
Here's my code update:
.update({
clueVotes: this.$fireStoreObj.FieldValue.increment(1)
})
Context:
I 'm doing a cloud function to send pushes to multiple users. I need to recover the info of each user to know some data like, name, country..etc..
Problem:
Actually I recover the list of user Id's and when I got it, then I create an array of promisesto recover all the info:
var usersPromises = []
for (var i = 0; i < usersInRange.length; i++) {
usersPromises[i] = firestore.collection("users").doc(usersInRange[i])
}
Then I recover and send the push using firestore.getAll():
firestore.getAll(...usersPromises).then(results => {
for(snapshot in results){
if(snapshot.exists){
......
var user = snapshot.data()
......
}else{
......
}
}
})
This solution is actually working "fine" almost all the time. But at this moment the Firestore db has some users that do not exist or something is wrong, because the method getAll()stops before finishing all the promises. I know it because no push is sent, and in the console, just say that the method has finished.
Reading in SO and documentation, I saw, that getAll stops if some promise is "broken". (all or nothing)
And here is where I'm lost. How can I "force" or do in another way, to just "jump" this promises that can't be completed?
P.S:
I tried to do with a "for" but It seems to omit some promises:
for (var i = 0; i < usersPromises.length; i++) {
usersPromises[i]
.get()
.then(snapshot => {
if(snapshot.exists){
......
var user = snapshot.data()
......
}else{
......
}
})
}
I think its not a problem of getAll. I have tested like this:
const firestore = new Firestore();
let doc = []
doc[0] = firestore.doc('test/test');
doc[1] = firestore.doc('test/test1');
doc[2] = firestore.doc('test/doc');
firestore.getAll(...doc)
.then(result=> result.forEach(doc => console.log(doc._fieldsProto)))
.catch(err=>console.log(err));
In my database I have 'test/test' and 'test/doc' document, but I do not have 'test/test1' and results look like this:
So we just get undefined on document that is not exist and that's all. I suggest to add catch and see if there is any exception. When I have been writing the test the function was interrupted by typo mistake in inner function.
I hope this will help!
I am trying to aggregate a list of dates from a data table, written in Angular, in a Protractor test. I'm doing the aggregation from a PageObject class that is called in the Protractor test. I know that my code is successfully grabbing the text I want, but when I try to console.log the returned array, I get an empty array. I'm still new to Javascript/Typescript, Angular, and Protractor and this may be a result of my newness to the asynchronous nature of this development environment.
Code is as follows,
The PageObject SpecMapper class with method:
import { browser, element, by } from 'protractor';
export class SpecMapperPage {
getImportDateSubmittedColumnValues() {
let stringDatesArray: Array<string> = [];
// currently this css selector gets rows in both import and export tables
// TODO: get better identifiers on the import and export tables and columns
element.all(by.css('md-card-content tbody tr.ng-tns-c3-0')).each(function(row, index){
// check outerHTML for presence of "unclickable", the rows in the export table
row.getAttribute('outerHTML').then(function(outerHTML:string) {
// specifically look for rows without unclickable
if(outerHTML.indexOf("unclickable") < 0){
// grab the columns and get the third column, where the date submitted field is
// TODO: get better identifiers on the import and export columns
row.all(by.css("td.ng-tns-c3-0")).get(2).getText().then(function(text:string) {
stringDatesArray.push(text);
});
}
});
});
return stringDatesArray;
}
}
I know it's not the prettiest code, but it's temporary place holder while my devs make me better attributes/classes/ids to grab my variables. Key things to note is that I create a string Array to hold the values I consider relevant to be returned when the method is finished.
I used WebStorm and put a breakpoint at the stringDatesArray.push(text) and return stringDatesArray lines. The first line shows that the text variable has a string variable that I'm looking for and is successfully getting pushed. I see the success in debug mode as I can see the stringDatesArray and see the values in it. The second line though, the array return, shows that the local variable stringDatesArray is empty. This is echoed in the following code when I try to console.log the array:
The Protractor run Spec class with my test in it:
import { SpecMapperPage } from "./app.po";
import {browser, ExpectedConditions} from "protractor";
describe('spec mapper app', () => {
let page: SpecMapperPage;
let PROJECT_ID: string = '57';
let PROJECT_NAME: string = 'DO NOT DELETE - AUTOMATED TESTING PROJECT';
beforeEach(() => {
page = new SpecMapperPage();
});
describe('import/export page', () => {
it('verify sort order is desc', () => {
browser.waitForAngularEnabled(false);
// Step 1: Launch Map Data from Dashboard
page.navigateTo(PROJECT_ID);
browser.driver.sleep(5000).then(() => {
// Verify: Mapping Screen displays
// Verify on the specmapper page by checking the breadcrumbs
expect(page.getProjectNameBreadCrumbText()).toContain(PROJECT_NAME);
expect(page.getProjectMapperBreadCrumbText()).toEqual("MAPPER");
// Verify: Verify Latest Submitted Date is displayed at the top
// Verify: Verify the Submitted Date column is in descending order
console.log(page.getImportDateSubmittedColumnValues());
});
});
});
});
I acknowledge that this code is not actively using the niceties of Protractor, there's a known issue with our app that will not be addressed for a couple of months, so I am accessing the driver directly 99% of the time.
You'll note that I call the method I posted above as the very last line in the browser.driver.sleep().then() clause, page.getImportDateSubmittedColumnValues().
I thought maybe I was running into asynchronous issues with the call being done before the page was loaded, thus I put it in the .then() clause; but learned with debugging that was not the case. This code should work once I have the array returning properly though.
The console.log is printing an empty [] array. That is synonymous with the results I saw when debugging the above method directly in the PageObject SpecMapper class. I wish to do some verification that the strings are returned properly formatted, and then I'm going to do some date order comparisons. I feel like returning an array of data retrieved from a page is not an unusual request, but I can't seem to find a good way to Google what I'm trying to do.
My apologies if I am hitting some very obvious roadblock, I'm still learning the nuances of Typescript/Angular/Protractor. Thank you for your consideration!
My attempted to used collated promises seemed promising, but fell through on execution.
My Updated PageObject SpecMapper Class
import {browser, element, by, protractor} from 'protractor';
export class SpecMapperPage {
getImportDateSubmittedColumnValues() {
let promisesArray = [];
let stringDatesArray: Array<string> = [];
// This CSS selector grabs the import table and any cells with the label .created-date
element.all(by.css('.import-component .created-date')).each(function(cell, index) {
// cell.getText().then(function(text:string) {
// console.log(text);
// });
promisesArray.push(cell.getText());
});
return protractor.promise.all(promisesArray).then(function(results) {
for(let result of results) {
stringDatesArray.push(result);
}
return stringDatesArray;
});
}
}
My Updated Spec test Using The Updated SpecMapper PO Class
import { SpecMapperPage } from "./specMapper.po";
import {browser, ExpectedConditions} from "protractor";
describe('spec mapper app', () => {
let page: SpecMapperPage;
let PROJECT_ID: string = '57';
let PROJECT_NAME: string = 'DO NOT DELETE - AUTOMATED TESTING PROJECT';
beforeEach(() => {
page = new SpecMapperPage();
});
describe('import/export page', () => {
it('TC2963: ImportComponentGrid_ShouldDefaultSortBySubmittedDateInDescendingOrder_WhenPageIsLoaded', () => {
browser.waitForAngularEnabled(false);
// Step 1: Launch Map Data from Dashboard
page.navigateTo(PROJECT_ID);
browser.driver.sleep(5000).then(() => {
// Verify: Mapping Screen displays
// Verify on the specmapper page by checking the breadcrumbs
expect(page.getProjectNameBreadCrumbText()).toContain(PROJECT_NAME);
expect(page.getProjectMapperBreadCrumbText()).toEqual("MAPPER");
// Verify: Verify Latest Submitted Date is displayed at the top
// Verify: Verify the Submitted Date column is in descending order
page.getImportDateSubmittedColumnValues().then(function(results) {
for(let value of results) {
console.log("a value is: " + value);
}
});
});
});
});
});
When I breakpoint in the PO class at the return stringDatesArray; line, I have the following variables in my differing scopes. Note that the promisesArray has 3 objects, but the results array going into the protractor.promise.all( block has 0 objects. I'm not sure what my disconnect is. :/
I think I'm running into a scopes problem that I am having issues understanding. You'll note the commented out promise resolution on the getText(), and this was my POC proving that I am getting the string values I'm expecting, so I'm not sure why it's not working in the Promise Array structure presented as a solution below.
Only other related question that I could find has to do with grabbing a particular row of a table, not specifically aggregating the data to be returned for test verification in Protractor. You can find it here if you're interested.
As you've alluded to your issue is caused by the console.log returning the value of the variable before its actually been populated.
I've taken a snippet from this answer which should allow you to solve it: Is there a way to resolve multiple promises with Protractor?
var x = element(by.id('x')).sendKeys('xxx');
var y = element(by.id('y')).sendKeys('yyy');
var z = element(by.id('z')).sendKeys('zzz');
myFun(x,y,z);
//isEnabled() is contained in the expect() function, so it'll wait for
// myFun() promise to be fulfilled
expect(element(by.id('myButton')).isEnabled()).toBe(true);
// in a common function library
function myFun(Xel,Yel,Zel) {
return protractor.promise.all([Xel,Yel,Zel]).then(function(results){
var xText = results[0];
var yText = results[1];
var zText = results[2];
});
}
So in your code it would be something like
getImportDateSubmittedColumnValues() {
let promisesArray = [];
let stringDatesArray: Array<string> = [];
// currently this css selector gets rows in both import and export tables
// TODO: get better identifiers on the import and export tables and columns
element.all(by.css('md-card-content tbody tr.ng-tns-c3-0')).each(function(row, index){
// check outerHTML for presence of "unclickable", the rows in the export table
row.getAttribute('outerHTML').then(function(outerHTML:string) {
// specifically look for rows without unclickable
if(outerHTML.indexOf("unclickable") < 0){
// grab the columns and get the third column, where the date submitted field is
// TODO: get better identifiers on the import and export columns
promisesArray.push(row.all(by.css("td.ng-tns-c3-0")).get(2).getText());
}
});
});
return protractor.promise.all(promisesArray).then(function(results){
// In here you'll have access to the results
});
}
Theres quite a few different ways you could do it. You could process the data in that method at the end or I think you could return the array within that "then", and access it like so:
page.getImportDateSubmittedColumnValues().then((res) =>{
//And then here you will have access to the array
})
I don't do the Typescript but if you're just looking to get an array of locator texts back from your method, something resembling this should work...
getImportDateSubmittedColumnValues() {
let stringDatesArray: Array<string> = [];
$$('.import-component .created-date').each((cell, index) => {
cell.getText().then(text => {
stringDatesArray.push(text);
});
}).then(() => {
return stringDatesArray;
});
}
The answer ended up related to the answer posted on How do I return the response from an asynchronous call?
The final PageObject class function:
import {browser, element, by, protractor} from 'protractor';
export class SpecMapperPage {
getImportDateSubmittedColumnValues() {
let stringDatesArray: Array<string> = [];
let promisesArray = [];
// return a promise promising that stringDatesArray will have an array of dates
return new Promise((resolve, reject) => {
// This CSS selector grabs the import table and any cells with the label .created-date
element.all(by.css('.import-component .created-date')).map((cell) => {
// Gather all the getText's we want the text from
promisesArray.push(cell.getText());
}).then(() => {
protractor.promise.all(promisesArray).then((results) => {
// Resolve the getText's values and shove into array we want to return
for(let result of results) {
stringDatesArray.push(result);
}
}).then(() => {
// Set the filled array as the resolution to the returned promise
resolve(stringDatesArray);
});
});
});
}
}
The final test class:
import { SpecMapperPage } from "./specMapper.po";
import {browser, ExpectedConditions} from "protractor";
describe('spec mapper app', () => {
let page: SpecMapperPage;
let PROJECT_ID: string = '57';
let PROJECT_NAME: string = 'DO NOT DELETE - AUTOMATED TESTING PROJECT';
beforeEach(() => {
page = new SpecMapperPage();
});
describe('import/export page', () => {
it('TC2963: ImportComponentGrid_ShouldDefaultSortBySubmittedDateInDescendingOrder_WhenPageIsLoaded', () => {
browser.waitForAngularEnabled(false);
// Step 1: Launch Map Data from Dashboard
page.navigateTo(PROJECT_ID);
browser.driver.sleep(5000).then(() => {
// Verify: Mapping Screen displays
// Verify on the specmapper page by checking the breadcrumbs
expect(page.getProjectNameBreadCrumbText()).toContain(PROJECT_NAME);
expect(page.getProjectMapperBreadCrumbText()).toEqual("MAPPER");
// Verify: Verify Latest Submitted Date is displayed at the top
// Verify: Verify the Submitted Date column is in descending order
page.getImportDateSubmittedColumnValues().then((results) => {
console.log(results);
});
});
});
});
});
The biggest thing was waiting for the different calls to get done running and then waiting for the stringDataArray to be filled. That required the promise(resolve,reject) structure I found in the SO post noted above. I ended up using the lambda (()=>{}) function calls instead of declared (function(){}) for a cleaner look, the method works the same either way. None of the other proposed solutions successfully propagated the array of strings back to my test. I'm working in Typescript, with Protractor.
I have an issue where my array is exponentially growing more and more each time I submit a post. I think it is happening in the second observable as that user object is being updated after each post to update the timestamp for when they last updated a post.
I am trying to check on the inside observable if that post is already in the array to prevent the duplicates from being inserted to the array. For some reason this is not working.
loadPosts(url: string) {
switch (url) {
case '/timeline/top':
this.postsService.subscribeAllPosts(this.selectedArea)
.subscribe(posts => {
let container = new Array<PostContainer>();
for (let post of posts) {
this.getEquippedItemsForUsername(post.username).subscribe(x => {
try {
if (container.indexOf(new PostContainer(post, x[0].equippedItems)) === -1) {
container.push(new PostContainer(post, x[0].equippedItems));
}
console.log( container); // grows exponentially after each submitted post
} catch (ex) { }
}
);
}
this.postContainers = container; // postContainers is the array that is being looped over in the html.
});
break;
}
}
I'm not sure whether you are correct about the problem, removing duplicates from your posts would be easy like this:
this.postsService.subscribeAllPosts(this.selectedArea)
.distinct()
.subscribe(posts => {
...
});
Your problem is that by creating a new PostContainer you're creating a new object which is not in container, so it will add every post in posts.
You should instead check that some unique value of post does not exist in any item of container
Something like:
if (container.findIndex((postContainer) => postContainer.id === post.id) === -1) {
continer.push(new PostContainer(post, x[0].equippedItems));
}