I have been trying to make the text from a raw github file test if the text is also the same as put in the if statement. It doesn't seem to work and the code runs the code in the if statement no matter what.
var current_version = "b-1"
const url = 'https://raw.githubusercontent.com/MythicalTrashcan/MY-Javascript-Bookmarklets/main/Version%20ID'
const version = await fetch(url);
const data = await version.text();
if (current_version != data) {
alert("Outated Bookmarklet!");
}
I was expecting the value to check if b-1 is not equal to the raw github to see if it would run that code inside the if-statement or not.
Based on your code, github returns with new line ending
So remove that \n, the code works
const data = (await version.text()).replace(/\n+/g, '');
Related
Documentation here:
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Examples
Code here:
https://github.com/mdn/webextensions-examples/tree/main/content-script-register
The above example from Firefox's own documentation does not appear to work as expected. Here is the main JS for the extension:
'use strict';
const hostsInput = document.querySelector("#hosts");
const codeInput = document.querySelector("#code");
const defaultHosts = "*://*.org/*";
const defaultCode = "document.body.innerHTML = '<h1>This page has been eaten</h1>'";
hostsInput.value = defaultHosts;
codeInput.value = defaultCode;
function registerScript() {
browser.runtime.sendMessage({
hosts: hostsInput.value.split(","),
code: codeInput.value
});
}
document.querySelector("#register").addEventListener('click', registerScript);
You can see the line const defaultHosts = "*://*.org/*"; which works as expected, however no matter what I do I cannot get it to work for i.e. const defaultHosts = *reddit.com/* or *://*google* etc.
Any ideas why it might be?
A match pattern must specify scheme://host/path, so the first pattern will be *://*.reddit.com/*
A match pattern's last domain cannot be *, so the second pattern cannot be fixed and you'll have to list all top-level domains explicitly (example).
P.S. Although it's also possible to use includeGlobs: ['*.google.*/'] when registering the content script in the background script, but it's a terrible workaround as it'll match the text in the wrong part of a URL like /path/ or search?parameter=value.
I am web scraping using NodeJS/typescript.
I have a problem using puppeteer where I get the fully rendered page (which I verify by running await page.content()). I printed the content and found that it had 26 'a' tags (links). However, when I search with puppeteer, I only get 20.
What is more strange is that sometimes I will get all the 'a' tags on the page and sometimes it gets less 'a' tags than on the page - all without changing the code! It seems to be kind of random.
I've seen some suggestions online saying to use a waitForElement method or something along those lines. Basically, before searching for tags, it ensures an element is on the page. I don't think this would help in my case because clearly puppeteer is getting everything it needs as shown by the await page.content() method.
Does anyone know why this may be happening? Thanks! A simplified snippet of my code is below.
const getLinksFromPage = async (
browser: puppeteer.Browser,
url: string
) => {
const page = await browser.newPage();
const curLink = book.sportsURLs[pageIndex];
await page.goto(url, { waitUntil: 'networkIdle0'});
const html = await page.content(); // this code gets the content and prints it
console.log(html); // so I can verify number of 'a' tags
const rawLinks = await page.$$eval('a', (elements: Element[]) => {
return elements
.map((element: Element) => element.getAttribute('href')!)
});
await page.close();
return rawLinks
};
I have this link https://nfse.blumenau.sc.gov.br/contrib/app/nfse/rel/rp_nfse_v23.aspx?s=61154301&e=00165960000101&f=2BED3D1E8 (if you try to access its gonna ask to solve a captcha but as long as i already have the session, the playwright doesnt need to worry it).
OUT page.goto: net::ERR_ABORTED at https://nfse.blumenau.sc.gov.br/contrib/app/nfse/rel/rp_nfse_v23.aspx?s=61154301&e=00165960000101&f=2BED3D1E8
Anybody knows why playwright cannot access it? I need to download the PDF Buffer of this link.
You can use the fetch API. Something like this:
const fetchResponse = browserContect.request.get('https://nfse.blumenau.sc.gov.br/contrib/app/nfse/rel/rp_nfse_v23.aspx?s=61154301&e=00165960000101&f=2BED3D1E8')
const pdfBuffer = await fetchResponse.body();
Found a Solution:
First i didnt concatenate the cookie on the calling of the function
const cook = 'ASP.NET_SessionId=' + cookie;
await setCookie(cook, urlFinal);
Then i used the got module to put the cookie and get the buffer of the pdf:
response = await got(urlFinal, {cookieJar}).buffer();
Plus: Sometimes it returned a blank pdf (i think because of the timeout of loading it). So i inserted a loop to check the size of the buffer and tried 20 times until it gets more than 'X' of lenght.
for (let j = 0;j<=25;j++){
console.log('Entrei no looping ==> ' + j);
response = await got(urlFinal, {cookieJar}).buffer();
if (response.toString().length>=10000){
j=21;
}
}
console.log('tamanho do buffer ==> ' + response.toString().length);
I tried that already but didnt work either:
const cookie = (await page.context().cookies()).filter(cookie => cookie.name === 'ASP.NET_SessionId')
.map(cookie => cookie.value)[0];
console.log(cookie);
const cookieJar = new CookieJar();
const setCookie = promisify(cookieJar.setCookie.bind(cookieJar));
await setCookie('ASP.NET_SessionId=' + cookie, urlFinal);
const response = await got(urlFinal, {cookieJar}).buffer();
its really a challenge. Because if i wont go with page.goto i loose the session. The code bellow would solve the problem.
await page.goto(urlFinal);
I am struggling for hours trying to get to the iframe but I just can't type in this box for some reason. The HTML does not show input on the page or in the iframe this is the code I tried and was the closest but not really getting to the box to type. this is the part of the HTML I try to get into.
inspect from Chrome
and here is the code I am using
const iframeHandle = await page.$$('iframe');
const contentFrame = await iframeHandle[2].contentFrame();
const tester = await contentFrame.$$('#rte');
and when I run
console.log(tester.length);
I get 1 so i am getting into the iframe but I dont know how to type with in it so far I can see its only an emtpy tag in it
Maybe I am just missing something small any help will be most appreciated
You can utilize the frame call.
So from your code
const iframeHandle = await page.$$('iframe');
await this.browser.frame(iframeHandle);
Or something of the sort, according to your code should get you into that iframe.
Try focus on the input and type
const cardElement = await paymentFrame.$('#cardNumber');
// Input is focused.
await cardElement.focus();
or this should work
const frames = await page.frames();
let iframe = frames.find(f => f.name() === 'any_iframe');
const textInput = await iframe.$('#textInput');
textInput.click(); // this focusses on the element
textInput.type('description text');
I'm trying to write in a text file, but not at the end like appendFile() do or by replacing the entiere content...
I saw it was possible to chose where you want to start with start parameter of fs.createwritestream() -> https://nodejs.org/api/fs.html#fs_fs_createwritestream_path_options
But there is no parameter to say where to stop writting, right ? So it remove all the end of my file after I wrote with this function.
const fs = require('fs');
var logger = fs.createWriteStream('result.csv', {
flags: 'r+',
start: 20 //start to write at the 20th caracter
})
logger.write('5258,525,98951,0,1\n') //example a new line to write
Is there a way to specify where to stop writting in the file to have something like:
....
data from begining
....
5258,525,98951,0,1
...
data till the end
...
I suspect you mean, "Is it possible to insert in the middle of the file." The answer to that is: No, it isn't.
Instead, to insert, you have to:
Determine how big what you're inserting is
Copy the data at your insertion point to that many bytes later in the file
Write your data
Obviously when doing #2 you need to be sure that you're not overwriting data you haven't copied yet (either by reading it all into memory first or by working in blocks, from the end of the file toward the insertion point).
(I've never looked for one, but there may be an npm module out there that does this for you...)
You could read/parse your file at first. Then apply the modifications and save the new file.
Something like:
const fs = require("fs");
const fileData = fs.readFileSync("result.csv", { encoding: "utf8" });
const fileDataArray = fileData.split("\n");
const newData = "5258,525,98951,0,1";
const index = 2; // after each row to insert your data
fileDataArray.splice(index, 0, newData); // insert data into the array
const newFileData = fileDataArray.join("\n"); // create the new file
fs.writeFileSync("result.csv", newFileData, { encoding: "utf8" }); // save it