Hey i'm trying to make a code that will automate the extraction of all the emails from a website by going through all the links and checking if there's a regex match but i can't figure it out here is what i got.
function getEmails() {
var search_in = document.body.innerHTML;
string_context = search_in.toString();
array_mails = string_context.match(/([a-zA-Z0-9._-]+#[a-zA-Z0-9._-]+\.[a-zA-Z0-9._-]+)/gi);
return array_mails;
}
You have to create a loop that will open every link that is presented on main page create ajax request and for each page opened use your function to get emails from it and push them to some array. Then you will have another array with all results. You will also need to check if your loop isn't infinite. Storing all links that have already been used will be needed.
Related
What I try to do is to loop through an array of Facebook page IDs and to return the code from each event page. Unfortunately, I only get the code of the last page ID in the array but as many times as elements are in the array. E.g. when I have 3 ID's in the array I get 3 times the code of the last page ID.
I already experimented with async await but I had no success.
The expected outcome would be the code of each page.
Thank you for any help and examples.
//Looping through pages
pages.forEach(
function(page) {
//Creating URL
let url = "https://mbasic.facebook.com/"+page+"?v=events";
//Getting URL
driver.get(url).then(
function() {
//Page loaded
driver.getPageSource().then(function(result) {
console.log(result);
});
}
);
}
);
you faced the same issue i did when i created a scraper using python and selenium. Facebook has countermeasure on manual URL change, you cannot change it , i receive the same data again and again even though it was automated. in order to get a good result you need to have access of face books Graph API which provides a complete object of Facebook page with its pagination URL.
or the second way i got it write was i used on click button of selenium browser automation to scroll down the next page.it wont work like you are typing , i prefer the usage of graph API
What is the best practice to create unique shareable urls for some text lists users create?
It's a single page website with a content div where users create text lists. Once they click share, how can I store those values inside a shareable url so that another user going to that address loads the same list?
I'm using html, js, jquery, php.
EDIT: as suggested below i'm already saving the lists on a database (firebase), and each have an unique ID, so I'd need to understand how I can create urls with a list id in it, and how to read the url back.
EDIT 2: so this is the code i'm using right now, combining answers from marzelin and the Alchemist Shahed in my other question about my database structure (Firebase how to find child knowing its id but not its parent's id (js)):
//js inside window load function:
const keyOfDynamicHtmlItemRef = new URL(window.location).searchParams.get("share")
if (keyOfDynamicHtmlItemRef) {
var dynamicHtmlListRef = firebase.database().ref('users');
// var dynamicHtmlItemRef = dynamicHtmlListRef.child(keyOfDynamicHtmlItemRef);
// console.log(keyOfDynamicHtmlItemRef);
// dynamicHtmlItemRef.once("value").then(dynamicHtmlSnap => {
// texta.innerHTML = dynamicHtmlSnap.val();
// });
dynamicHtmlListRef.once('value').then((snapshot)=>{
snapshot.forEach(function(data) {
if (data.key == keyOfDynamicHtmlItemRef) {
myVar = data.c;
myContentDiv.innerHTML = myVar;
}
});
});
}
and i'm simply trying to manually write the url in the searchbar as a first step, as https://example.com/?share=<random list id i copied from db>, but it does nothing.
So the way I would to this is I would have the users share click trigger a save to database saving all the dynamically generated content into a table.
One of the table values would be a randomly generated unique identifier of some sort that I would use as a query in the url like https://www.example.org/?share=skd822475
Then when a user visits the site and that query is in the url id use the unique identifier to look up the database and publish the dynamic content back on the page.
I would also put a half life on the database entry's of say no more than 30 days so that it doesn't clog up the db.
Saving data and creating shareable link:
document.querySelector(".share").addEventListener("click" => {
var dynamicHtmlListRef = firebase.database().ref('dynamic_html');
var dynamicHtmlItemRef = dynamicHtmlListRef.push();
dynamicHtmlItemRef.set(userCreatedDynamicHtml);
var keyOfDynamicHtmlItem = dynamicHtmlItemRef.key;
var linkToDynamicHtmlItem = `${window.location}?share=${keyofDynamicHtmlItem}`;
alert(`link: ${linkToDynamicHtmlItem}`)
})
Showing the dynamic HTML based on query parameters:
const keyOfDynamicHtmlItemRef = new URL(window.location).searchParams.get("share")
if (keyOfDynamicHtmlItemRef) {
var dynamicHtmlListRef = firebase.database().ref('dynamic_html');
var dynamicHtmlItemRef = dynamicHtmlListRef.child(keyOfDynamicHtmlItemRef);
keyOfDynamicHtmlItemRef.once("value").then(dynamicHtmlSnap => {
document.querySelector(".dynamic-html-mountpoint").innerHTML = dynamicHtmlSnap.val();
});
}
Let's start with the first question "How to create urls with a list id in it?"
The thing is that to answer this one we need to answer the second question first witch is
"How to read the url back?"
Consider that you have a php page named "draft". when a user visit https://www.example.com/draft?listId=an_id you will get listId using php like so $_GET("listId") and use that value to retrieve the list data and display the page content.
Now coming back to the first question, if the user share the draft like in social media (ex: facebook) then there is no problem because he will share a link and all his followers and any other user can access it easily. but if the user just save the draft then you will have to change the page url dynamically like this window.history.pushState(null, null, '/draft?listId=your_newly_created_id'); and so the user will copy the url and do whatever he wnt with it (sharing it in stackoverflow maybe example using jsfiddle http://jsfiddle.net/F2es9/ (you can change the url to look like this using 'htaccess' file)) at the end I would like to tell you that we don't "create" urls.
Edit
without using php code (or any other server side code). the difference will be in retrieving the data.
instead of using $_GET("listId") you will use new URL(window.location).searchParams.get("listId") to get the list id in javascript then using this value you can retrieve data from firebase and display your content
So I have one HTML page which consists of a bunch of form elements for the user to fill out. I push all the selections that the user makes into one global variable, allTheData[] inside my only Javascript file.
Then I have a 2nd HTML page which loads in after a user clicks a button. This HTML page is supposed to take some of the data inside the allTheData array and display it. I am calling the function to display allTheData by using:
window.onload = function () {
if (window.location.href.indexOf('Two') > -1) {
carousel();
}
}
function carousel() {
console.log("oh");
alert(allTheData.toString());
}
However, I am finding that nothing gets displayed in my 2nd HTML page and the allTheData array appears to be empty despite it getting it filled out previously in the 1st HTML page. I am pretty confident that I am correctly pushing data into the allTheData array because when I use alert(allTheData.toString()) while i'm still inside my 1st HTML page, all the data gets displayed.
I think there's something happening during my transition from the 1st to 2nd HTML page that causes the allTheData array to empty or something but I am not sure what it is. Please help a newbie out!
Web Storage: This sounds like a job for the window.sessionStorage object, which along with its cousin window.localStorage allows data-as-strings to be saved in the users browser for use across pages on the same domain.
However, keep in mind that they are both Cookie-like features and therefore their effectiveness depends on the user's Cookie preference for each domain.
A simple condition will determine if the web storage option is available, like so...
if (window.sessionStorage) {
// continue with app ...
} else {
// inform user about web storage
// and ask them to accept Cookies
// before reloading the page (or whatever)
}
Saving to and retrieving from web storage requires conversion to-and-from String data types, usually via JSON methods like so...
// save to...
var array = ['item0', 'item1', 2, 3, 'IV'];
sessionStorage.myApp = JSON.stringify(array);
// retrieve from...
var array = JSON.parse(sessionStorage.myApp);
There are more specific methods available than these. Further details and compatibility tables etc in Using the Web Storage API # MDN.
Hope that helps. :)
I'm trying to create a basic time clock web app.
So far, I'm using this script to create this web app which takes the input values and puts them in this spreadsheet for the time stamping part.
I need it to use one of the values from the form and perform a lookup in this sheet (take the longId and find me the name) and return the (name) value to the html page as a verification for the end user that they were identified correctly. Unfortunately, I don't know enough to grasp what I'm doing wrong. Let me know if I need to provide more info.
Edit 1
I'm thinking that I wasn't clear enough. I don't need the user info from entry, I need the user from a lookup. The user will be entering their ID anonymously, I need to match the ID to their info, and bring the info back for them to verify.
Edit 2
Using the link provided by Br. Sayan, I've created this script using this spreadsheet as above to test one piece of this. The web app here spits out: undefined. It should spit out "Student 3" Still not sure what I'm doing wrong.
One way for the next button to grab the student input field:
<input type="submit" onclick="studentName(document.getElementById('student').value)" value="Next..."/>
That sends the value to this func in Javascript.html:
function studentName(value) {
google.script.run
.withSuccessHandler(findSuccess)
.findStudent(value);
}
Which sends it to a findStudent(value) in Code.gs
You do the lookup and the return value goes back to findSuccess( result ) back in Javascript.html. Handle the result from there.
Also consider keeping the stock preventDefault() code that comes with the Web App template in the Help > Welcome Screen.
Please try this one:
(source: technokarak.com)
Also please have a look at:
Retrieve rows from spreadsheet data using GAS
EDIT:
Please make these changes in your function and let us know.
function findValue() {
var data = SpreadsheetApp.openById("15DRZRQ2Hcd7MNnAsu_lnZ6n4kiHeXW_OMPP3squbTLE").getSheetByName("Volatile Data").getDataRange().getValues();
for(i in data) {
if(data[i][3] == 100000003) {
Logger.log("yes");
Logger.log(data[i][0]);
var student = [];
student.push(data[i][0]);
return student;
}
}
}
It is a complicated answer, I have had a lot of success with:
function process(object){
var user = Session.getActiveUser().getEmail();
var key = object.Key;
send(key);
}
function send(k){
var ss =
SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
var lastR = ss.GetLastRow();
ss.GetRange(lastR,1).SetValue(k);
}
On your html button you will need to have inside the tags
onClick="google.script.run
.withSuccessHandler(Success)
.process(this.parentNode);"
In order for this to work, obviously you will need to have your fields named accordingly.
Edit: The only thing I did not include in the code was a Success handler, which will be in your html of the GAS script. This should point you in a direction that can resolve that.
Hope this helps.
Hello I have a small website where data is passed between pages over URL.
My question is can someone break into it and make it pass the same data always?
For example let say, when you click button one, page below is loaded.
example.com?clicked=5
Then at that page I take value 5 and get some more data from user through a form. Then pass all the data to a third page. In this page data is entered to a database. While I observe collected data I saw some unusual combinations of records. How can I verify this?
yes. as javascript is open on the website, everyone can hack it.
you will need to write some code on you backend to validade it.
always think that you user/costumer will try to hack you sytem.
so take precautions like, check if user is the user of the session, if he is logged, if he can do what he is trying to do. check if the record that he is trying get exists.
if u are using a stand alone site, that u made the entire code from the ashes, you will need to implement this things by yourself.
like using the standard php session, making the data validation etc.
or you can find some classes that other people have made, you can find a lot o this on google. as it is a common problem of web programing.
if u are using a backed framework that isnt from another world, probably already has one. sp, go check its documentation.
html:
<a id = 'button-one' name = '5'> Button One </a>
javascript:
window.onload = function() {
document.getElementById('button-one').onclick = function() {
changeURL(this.attributes.name.value);
};
};
function changeURL(data) {
location.hash = data;
}