I'm fetching the stylesheet and replacing all CSS variables with the actual hex value it corresponds to when the user changes the color as desired.
I created an event handler so that when the user clicks the download button, all of the colors he/she selected, would be saved in the stylesheet at that moment, but it doesn't seem to work. I know it's an issue with my understanding of promises as a whole and async await
What I did.
const fetchStyleSheet = async () => {
const res = await fetch("./themes/prism.css");
const orig_css = await res.text();
let updated_css = orig_css;
const regexp = /(?:var\(--)[a-zA-z\-]*(?:\))/g;
let cssVars = orig_css.matchAll(regexp);
cssVars = Array.from(cssVars).flat();
console.log(cssVars)
for await (const variable of cssVars) {
const trimmedVar = variable.slice(6, -1)
const styles = getComputedStyle(document.documentElement)
const value = String(styles.getPropertyValue(`--${trimmedVar}`)).trim()
updated_css = updated_css.replace(variable, value);
}
console.log(updated_css)
return updated_css
}
const main = async () => {
const downloadBtn = document.getElementById('download-btn')
downloadBtn.addEventListener('click', () => {
const updated_css = fetchStyleSheet()
downloadBtn.setAttribute('href', 'data:application/octet-stream;charset=utf-8,' + encodeURIComponent(updated_css))
downloadBtn.setAttribute('download', 'prism-theme.css')
})
}
main()
I can't await the updated_css because it falls into the callback of the click event, which is a new function.
Then I did the following thinking it would work since it was top level.
const downloadBtn = document.getElementById('download-btn')
downloadBtn.addEventListener('click', async () => {
const updated_css = await fetchStyleSheet()
downloadBtn.setAttribute('href', 'data:application/octet-stream;charset=utf-8,' + encodeURIComponent(updated_css))
downloadBtn.setAttribute('download', 'prism-theme.css')
})
That gave me the following error TypeError: NetworkError when attempting to fetch resource.
I understand that calling fetchStyleSheet() only returns a promise object at first and to get the value (which is updated_css), I need to follow it with .then() or await it.
The await is the correct approach to deal with the fetchStyleSheet() call returning a promise, your problem is that the click on the link tries to follow the href attribute immediately - before you set it to that data url. What you would need to do instead is prevent the default action, asynchronously do your stuff, and then re-trigger the click when you're done. Also don't forget to deal with possible exceptions:
const downloadBtn = document.getElementById('download-btn')
downloadBtn.addEventListener('click', async (event) => {
if (!e.isTrusted) return // do nothing on the second run
try {
event.preventDefault()
const updated_css = await fetchStyleSheet()
downloadBtn.setAttribute('href', 'data:application/octet-stream;charset=utf-8,' + encodeURIComponent(updated_css))
downloadBtn.setAttribute('download', 'prism-theme.css')
downloadBtn.click() // simulate a new click
} catch(err) {
console.error(err) // or alert it, or put the message on the page
}
})
Related
I have an async function I have to call it under the map function,
class DataArticle {
id:number,
title:string,
...
user:User // User Entity
}
I want to get articles then assign to each article the author of it:
var dtresult = this.articleRepo.findAll(); // get all articles
const result:DataArticle[] = dtresult.map(async (a:DataArticle) => {
let user = await this.UserRepo.getUser(a.id)
a.user = user ; // assign user to the article after getting the user
return a ;
})
I tried implemeting async function with this way and it doesn't work
You don't need to use another library for this task.
It can be achieved by Promise.All (or Promise.allSettled)
async function getUser(id){return id;}
let data = [...Array(10).keys()]
async function the_caller_function(){
const result = await Promise.all(data.map(async a=> {
let user = await getUser(a)
return {user:a}
}))
console.log(result)
}
the_caller_function();
I'm trying to practice some web scraping with prices from a supermarket. It's with node.js and puppeteer. I can navigate throught the website in beginning with accepting cookies and clicking a "load more button". But then when I try to read div's containing the products with querySelectorAll I get stuck. It returns undefined even though I wait for a specific div to be present. What am I missing?
Problem is at the end of the code block.
const { product } = require("puppeteer");
const scraperObjectAll = {
url: 'https://www.bilkatogo.dk/s/?query=',
async scraper(browser) {
let page = await browser.newPage();
console.log(`Navigating to ${this.url}`);
await page.goto(this.url);
// accept cookies
await page.evaluate(_ => {
CookieInformation.submitAllCategories();
});
var productsRead = 0;
var productsTotal = Number.MAX_VALUE;
while (productsRead < 100) {
// Wait for the required DOM to be rendered
await page.waitForSelector('button.btn.btn-dark.border-radius.my-3');
// Click button to read more products
await page.evaluate(_ => {
document.querySelector("button.btn.btn-dark.border-radius.my-3").click()
});
// Wait for it to load the new products
await page.waitForSelector('div.col-10.col-sm-4.col-lg-2.text-center.mt-4.text-secondary');
// Get number of products read and total
const loadProducts = await page.evaluate(_ => {
let p = document.querySelector("div.col-10.col-sm-4.col-lg-2").innerText.replace("INDLÆS FLERE", "").replace("Du har set ","").replace(" ", "").replace(/(\r\n|\n|\r)/gm,"").split("af ");
return p;
});
console.log("Products (read/total): " + loadProducts);
productsRead = loadProducts[0];
productsTotal = loadProducts[1];
// Now waiting for a div element
await page.waitForSelector('div[data-productid]');
const getProducts = await page.evaluate(_ => {
return document.querySelectorAll('div');
});
// PROBLEM HERE!
// Cannot convert undefined or null to object
console.log("LENGTH: " + Array.from(getProducts).length);
}
The callback passed to page.evaluate runs in the emulated page context, not in the standard scope of the Node script. Expressions can't be passed between the page and the Node script without careful considerations: most importantly, if something isn't serializable (converted into plain JSON), it can't be transferred.
querySelectorAll returns a NodeList, and NodeLists only exist on the front-end, not the backend. Similarly, NodeLists contain HTMLElements, which also only exist on the front-end.
Put all the logic that requires using the data that exists only on the front-end inside the .evaluate callback, for example:
const numberOfDivs = await page.evaluate(_ => {
return document.querySelectorAll('div').length;
});
or
const firstDivText = await page.evaluate(_ => {
return document.querySelector('div').textContent;
});
I have a function that retrieves the data from a document correctly. However, one image has the URL as it's field already. The other image only has a firebase image reference. Before I proceed to another function, I need to wait until the download URL has been fetched. I've attempted it below without much luck, and I'm not entirely sure i've stuck the async in the right place either.
getPhoto(user_id: string) {
this._subscription = this._activatedRoute.params.pipe(
switchMap(params => {
return this.service.getPhoto(params);
})
).subscribe(async(result) => {
const imageOne = result.imageOne;
// Need to await the download URL here
const imageTwo = this.blah(result.imageTwoRef)
this.otherFunction(imageOne, imageTwo)
});
}
blah(reference){
var storage = firebase.storage();
var imageTwo = reference;
var imagePathRef = storage.ref().child(imageTwo);
imagePathRef.getDownloadURL().then((url) => {
return url;
});
}
Using the async keyword only works on function, and by doing so, it will return a promise. So your usage is correct in that instance.
You can use await only in an async function and next to a promise call. It will stop the execution until your promise get resolved.
I think you are almost done. Try it like this and let me know:
getPhoto(user_id: string) {
this._subscription = this._activatedRoute.params.pipe(
switchMap(params => {
return this.service.getPhoto(params);
})
).subscribe(async(result) => {
const imageOne = result.imageOne;
// Need to await the download URL here
const imageTwo = await this.blah(result.imageTwoRef)
this.otherFunction(imageOne, imageTwo);
});
}
async blah(reference){
var storage = firebase.storage();
var imageTwo = reference;
var imagePathRef = storage.ref().child(imageTwo);
const url = await imagePathRef.getDownloadURL();
return url;
}
I made a small webpage that takes information from the "Yahoo Weather" API and displays it in divs on the page.
This is the JS:
const url = 'https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text%3D%22nome%2C%20ak%22)&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys';
let data = 1;
const getWeather = async function() {
const fetchWeather = await fetch(url);
const result = await fetchWeather.json();
return data=result;
}
getWeather();
const showData = async function(info) {
let html = '';
const newInfo = info.query.results.channel.item.forecast.map((item, index) => {
html += `<div id='item${index}'><p>Date: ${item.date}</p>`;
html += `<p>Day: ${item.day}</p>`;
html += `<p>Highest temp: ${item.high}</p>`;
html += `<p>Lowest temp: ${item.low}</p></div>`;
return html;
});
}
const display = async function() {
const info = await showData(data);
weatherInfo.innerHTML = data;
}
display();
My goal is that when the page loads, it displays the information gathered from the promise returned by the API.
I get this error:Uncaught (in promise) TypeError: Cannot read property 'results' of undefined
Basically as far as I understand, by the time the "display()" is invoked, the variable "data" doesn't have anything in it yet.
What I'm trying to achieve is that display() will only work after "data" is defined, but without a for loop or something like that.
Any help will be appreciated!
As far as I understand, by the time the "display()" is invoked, the variable "data" doesn't have anything in it yet.
Yes. Don't use a global data variable at all1. getWeather returns a promise that will fulfill with the data, so you know exactly when it becomes available:
getWeather().then(display);
with
async function getWeather () {
const fetchWeather = await fetch(url);
const result = await fetchWeather.json();
return result; // drop the `data =` assignment
}
async function display(data) {
// ^^^^
const info = await showData(data);
weatherInfo.innerHTML = info;
}
Alternatively, especially when you don't want to use a then chain, just put const data = await getWeather(); in the display function.
1: If you insist to store the data in the global scope because you want to use it in multiple places, put the promise for the data in the variable not the result itself.
Here is my solutiuon:
const url = 'https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text%3D%22nome%2C%20ak%22)&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys';
const getWeather = async function() {
const fetchWeather = await fetch(url);
return await fetchWeather.json();
}
const showData = async function(info) {
let html = '';
info.query.results.channel.item.forecast.map((item, index) => {
html += `<div id='item${index}'><p>Date: ${item.date}</p>`;
html += `<p>Day: ${item.day}</p>`;
html += `<p>Highest temp: ${item.high}</p>`;
html += `<p>Lowest temp: ${item.low}</p></div>`;
});
return html;
}
const display = async function() {
const data = await getWeather();
const info = await showData(data);
weatherInfo.innerHTML = info;
}
display();
https://plnkr.co/edit/1b0fpBji7y6sZPODPDjY?p=preview
I am trying to scrap wikipedia page to fetch list of airlines by first scrapping first page and then going to each individual page of airline to get the website url. I have divided the code in two functions. One to scrap main page and get a new url, and second function to scrap another page from the created url to get the website name from that page. I have used request-promise module for getting the html and then cheerio to parse the data.
export async function getAirlinesWebsites(req,res) {
let response = await request(options_mainpage);
console.log(`Data`);
let $ = cheerio.load(response);
console.log('Response got');
$('tr').each((i,e)=>{
let children = '';
console.log('inside function ', i);
if($(e).children('td').children('a').attr('class') !== 'new') {
children = $(e).children('td').children('a').attr('href');
let wiki_url = 'https://en.wikipedia.org' + children;
console.log(`wiki_url = ${wiki_url}`);
let airline_url = getAirlineUrl(wiki_url);
console.log(`airline_url = ${airline_url}`);
}
})
And then the getAirlineUrl() function will parse another page based on the provided url.
async function getAirlineUrl(url){
const wiki_child_options = {
url : url,
headers : headers
}
let child_response = await request(wiki_child_options);
let $ = cheerio.load(child_response);
let answer = $('.infobox.vcard').children('tbody').children('tr').children('td').children('span.url').text();
return answer;
})
However when I console log the answer variable in the parent function, I get a [object Promise] value instead of a String. How do I resolve this issue?
Async function return promise.In case of that,you need to use then to get resolved response or use await.
This should work if other part of your code is ok.
export async function getAirlinesWebsites(req, res) {
let response = await request(options_mainpage);
console.log(`Data`);
let $ = cheerio.load(response);
console.log("Response got");
$("tr").each(async (i, e) => {
let children = "";
console.log("inside function ", i);
if ($(e).children("td").children("a").attr("class") !== "new") {
children = $(e).children("td").children("a").attr("href");
let wiki_url = "https://en.wikipedia.org" + children;
console.log(`wiki_url = ${wiki_url}`);
let airline_url = await getAirlineUrl(wiki_url);
console.log(`airline_url = ${airline_url}`);
}
});
}
Since your getAirlineUrl function returns a promise, you need to await that promise. You can't have await nested inside of the .each callback because the callback is not an async function, and if it was it wouldn't work still. The best fix is the avoid using .each and just use a loop.
export async function getAirlinesWebsites(req,res) {
let response = await request(options_mainpage);
console.log(`Data`);
let $ = cheerio.load(response);
console.log('Response got');
for (const [i, e] of Array.from($('tr')).entries()) {
let children = '';
console.log('inside function ', i);
if($(e).children('td').children('a').attr('class') !== 'new') {
children = $(e).children('td').children('a').attr('href');
let wiki_url = 'https://en.wikipedia.org' + children;
console.log(`wiki_url = ${wiki_url}`);
let airline_url = await getAirlineUrl(wiki_url);
console.log(`airline_url = ${airline_url}`);
}
}
}