Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
Is it possible to scrape data from popups, modals or dialog windows?
for example
https://tenders.procurement.gov.ge/public/?lang=en
I need email addresses from the users>suppliers and you can see users list, but I must open popup window in order to check any info about user.
So how could I scrape all emails from these popups ? is it possible?
first screen:
second screen:
third screen:
You can use requests.session for the task:
import re
import requests
from bs4 import BeautifulSoup
base_url = 'https://tenders.procurement.gov.ge/public/?lang=en'
url = 'https://tenders.procurement.gov.ge/public/library/controller.php?action=org_list'
profile_url = 'https://tenders.procurement.gov.ge/public/library/controller.php?action=profile&org_id='
num = re.compile(r'(\d+)')
with requests.session() as s:
# load cookies:
s.get(base_url)
soup = BeautifulSoup(s.get(url).content, 'html.parser')
for tr in soup.select('tr[onclick]'):
n = num.search(tr['onclick']).group(1)
soup2 = BeautifulSoup(s.get(profile_url + n).content, 'html.parser')
email = soup2.select_one('td:contains("E-Mail") + td')
print(email.text)
Prints:
xxx#yandex.ru
xxx#gmail.com
xxx#gmail.com
...and so on.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I have a database called lowdb (it is a nodejs api), it is in the backend.
I want to use it in the html tag, ie the javascript frontend.
How can I import the lowdb into the frontend javascript?
First frontend = browser, backend = nodejs. You surely mean that LowDB is available as a NodeJS module, which means it can be installed with NPM, either for frontend or backend development.
If you're not using NPM for your frontend development, you can still import modules with <script> tags.
Let's see an example using only HTML/javascript:
(from the doc):
<html>
<head>
<script src="https://unpkg.com/lodash#4/lodash.min.js"></script>
<script src="https://unpkg.com/lowdb#0.17/dist/low.min.js"></script>
<script src="https://unpkg.com/lowdb#0.17/dist/LocalStorage.min.js"></script>
</head>
<body>
<button onclick="add()">Add Post</button>
<button onclick="load()">Load Post</button>
<div></div>
</body>
<script>
var adapter = new LocalStorage('db')
var db = low(adapter)
db.defaults({ posts: [] })
.write()
function add() {
// Data is automatically saved to localStorage
db.get('posts')
.push({ title: 'lowdb' })
.write()
}
function load() {
var div = document.querySelector('div')
var text = div.innerText;
text += db.get('posts').map(post=>post.title).join(';');
div.innerText = text
}
</script>
</html>
See this work: https://codepen.io/bcaure/pen/ExgBPBz
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
Is it possible to take a link and access its HTML code through that link? For example I would like to take a link from Amazon and put it within my own HTML code, use JavaScript to getElementsByClassName to get the price from that link and display it back into my HTML code.
It is possible. You could do a GET request to the Amazon page that will give you the html in the response from there you'll have a string now you'll need to format it, last time I used the node module jsdom to do that.
In more detail:
HTTP is a protocol that we use to request data from the server, I've wrote an explanatory node js script:
const https = require('https');
const JSD = require('jsdom');
const { JSDOM } = JSD;
const zlib = require('zlib');
// The http get request
https.get('https://www.amazon.com', (response) => {
html = '';
// we need this because amazon is tricky and encodes the response so it is smaller hence it is faster to send
let gunzip = zlib.createGunzip();
response.pipe(gunzip);
// we need this to get the full html page since it is too big to send in one amazon divides it to chunks
gunzip.on('data', (chunk) => {
html += chunk.toString();
});
// when the transmittion finished we can do wathever we want with it
gunzip.on('end', () => {
let amazon = new JSDOM(html);
console.log(amazon.window.document.querySelector('html').innerHTML);
});
});
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 years ago.
Improve this question
I have a function that writes to the variable files from the input file field. I want to re-load files in the variable after reselecting files in the form. The current script loads the files correctly, but the previous files are in the upload variable. How to clean up the variable upload before each call to the load_files function in such a way that the variable data is available outside of it?
$(document).ready(function() {
$('.choose-input').on('change', load_files);
var upload = [];
function load_files(e) {
var files = e.target.files;
var arr = Array.prototype.slice.call(files);
arr.forEach(function(file) {
upload.push(file);
$('.choose-el').append(file.name);
});
if (upload.length > 0) {
$('.choose-el').append('<span class="start-upload">Start upload</span>');
}
$('.choose-input').val('');
}
});
var upload = [];
$('.choose-input').on('change', function(e) {
upload = [];
load_files(e);
);
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I am still very new at web development, I need some suggestions please. I am busy creating a page with a date selector, then I have a folder with one file saved every day. What I am trying to do is: The user needs to select the date of the file he wants and click download and the file saved on that date needs to be downloaded. Can someone please give me an idea how I can get this to work. I have tried some things with JavaScript and php and could not get a working solution.
This code should do the job.
First we scan path provided and list all files, then check creation date and find our target file.
<?php
$Path = './'; // Set path of files here
$TargetDate = '2016-08-11'; // We find the first file with thi date
$TargetFile = null; // Store result here
// Lets Do It
foreach (glob("$Path/*") as $File) {
$Stat = stat($File);
if (date("Y-m-d", $Stat['ctime']) == $TargetDate) {
$TargetFile = $File;
break;
}
}
// Your File!
if (is_null($TargetFile)) {
echo 'No file found';
} else {
echo $TargetFile;
}
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
Currently I have a object that is used for passing message from client to server.
var JSONMessage = function() {
this.sender = "";
this.method = "";
this.arguments = "";
}
I want this object to be available both to the server (NodeJS) as well as the client (HTML). Currently I am doing the following below the above object.
if ( typeof module === 'undefined' ) {
console.log("must be client side!");
}
else {
module.exports = JSONMessage;
}
And in the nodeJS file I do the following
var JSONMessage = require('./public/js/message');
While in the HTML I can simply include the js file.
My question is, is this the best way sharing code between Node and Javascript?
You should check out the umdjs patterns hosted in this Github Repo:
https://github.com/umdjs/umd
What you are doing will work, but you can eliminate the guess work by using one from that repo.