setInterval doesn't work without an error [closed] - javascript

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 8 months ago.
Improve this question
So i tried to watch a tutorial and make a web scraper, i combined like 5 tutorial worth code into my code so i suppose it's messy.
This is the code:
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
const fs = require('fs');
const { html } = require('cheerio');
const cheerio = require('cheerio');
const { setInterval } = require('timers/promises');
require('dotenv').config();
const accountSid = process.env.TWILIO_ACCOUNT_SID;
const authToken = process.env.TWILIO_AUTH_TOKEN;
const client = require('twilio')(accountSid,authToken);
puppeteer.use(StealthPlugin());
const oldData = "oldData";
async function scrape(){
const browser = await puppeteer.launch({
headless: true,
defaultViewport:{
width: 1920,
height: 1080
}
});
const page = await browser.newPage();
await page.goto('page');
const pageData = await page.evaluate(() => {
return{
width: document.documentElement.clientWidth,
height: document.documentElement.clientHeight,
html: document.documentElement.innerHTML,
};
})
const $ = cheerio.load(pageData.html);
const element = $(".accordion__header-inner:last");
console.log(element.text());
};
const handle = setInterval(scrape(), 4000);
scrape();
I hide some parts that i didn't want to be seen.
So, when i run the code function it logged the element thing twice.
When i delete the interval thing it works only one.
That is the only difference, i want to log the element thing in every 5 seconds.
Any help is appreciated and i hope this isn't a poorly written question.

In your setInterval you are executing the function and passing it's return value as argument, instead of passing function itself as argument.
It should be:
const handle = setInterval(scrape, 4000);
Note, you won't be use this inside the function, because it will be called inside setInterval scope.
If you need to use this, then use bind():
const handle = setInterval(scrape.bind(scrape), 4000);

You seem to be calling the "scrape()" method in two places. Once in the SetInterval() method and then immediately again on the line after. That looks like your problem.

Related

Prevent node.js application from exiting once the process is done

I wrote a really basic nodejs application which look like this:
#!/usr/bin/env node
const inquirer = require("inquirer");
const chalk = require("chalk");
const figlet = require("figlet");
const shell = require("shelljs");
const fs = require('fs');
//Defining some functions
//~~
//The main part of the code
const run = async function() {
const answers = await form();
//other codes
//~~
}
run();
The main purpose of the code is to use the inquirer module to ask some question in the console, and then process the answers in the run() part. It does perfect job on that. It successfully asks the question and do what it should do with the answers.
However, The process would exit once the answer have been processed. What I want is once the answer have been processed, I want it to answer the same question again, and keep repeating that until I terminate the process manually.
I tried this:
for( ; ; ) {
run();
}
However, It would then answer questions again and again without waiting for answers. Here is how the console looked like:
console output
I want it to do these:
Ask question and wait untill I enter the answer.
Process the question (with the codes in run())
Once it's done, go back to question 1.
How can I do this?
Instead of doing
for( ; ; ) {
run();
}
Do this inside run function
while (true) {
const answers = await form();
//other codes
//~~
}

Can't export wsEndpoint created from puppeteer browser

I am trying to open a puppeteer browser upon startup and then exporting the wsEndpoint so that I may use the link to connect to the browser rather than opening a new browser every time I call the function.
Here is the code snippet in the file app.js that is the entry point for node.
const mainFunction = async () => {
const browser = await puppeteer.launch()
const wsEndpoint = browser.wsEndpoint()
return wsEndpoint
}
mainFunction().then(async endpoint => {
console.log(endpoint)
module.exports = endpoint
})
upon startup, the console log above returns a link that I then export
And here is the code snippet in the utility file equities.js
const puppeteer = require("puppeteer")
const endpoint = require("../../app.js")
module.exports = async(symbol)=>{
console.log(endpoint)
const browser = await puppeteer.connect({connectWSEndpoint: endpoint})
}
Every time I call the function, the console log only returns an empty object meaning that the export in app.js failed for some reason. I tried to google a few things and tried different ways of exporting but none seem to work. Can someone help guide me? Thank you so much in advance.
A few things here seem amiss to me -- this code feels like it wasn't tested along the way, leading to multiple points of failure. Try to take smaller steps so you can isolate problems instead of accumulating them.
For starters, the mainFunction code abandons the browser object, creating a leaked subprocess resource can't be closed.
I'd return or store the browser variable along with the endpoint so someone can clean it up through a function. Or just return the browser and let the client code pull the endpoint out of it if they want, as well as manage and close the resource.
Next, the export code:
mainFunction().then(async endpoint => {
console.log(endpoint)
module.exports = endpoint
})
I don't understand the motivation for this extra then wrapper that receives an async resolution function that never uses await. You may think Node awaits all of this code, then sets the module.exports value before the client file's require runs synchronously. That's not the case, which can be determined with a simpler piece of code:
app.js (in the same folder throughout this post for convenience):
const mainFunction = async () => 42;
mainFunction().then(async endpoint => {
console.log("endpoint":, endpoint)
module.exports = endpoint
})
index.js:
const endpoint = require("./app");
console.log("imported:", endpoint);
node index gives me:
imported: {}
endpoint: 42
The promise resolved after the require, which synchronously brought in the default blank object module.exports -- probably not what you expected.
If you have async code, it has to stay async forever, including exports and imports. Try exporting the promise directly, then awaiting it in the client:
app.js:
const mainFunction = async () => 42;
module.exports = mainFunction;
index.js:
const getEndpoint = require("./app");
getEndpoint().then(endpoint => console.log("imported:", endpoint));
Running node index gives me: imported: 42.
The client code in equities.js looks more reasonable because it exports a promise synchronously, but it's going to have to await the endpoint promise it imported anywhere it uses it.
Also, Puppeteer throws on puppeteer.connect({connectWSEndpoint: endpoint}), Error: Exactly one of browserWSEndpoint, browserURL or transport must be passed to puppeteer.connect. I'll leave that up to you to work out based on your goals.
Here's a rewrite sketch that fixes the promise problems, but is only a proof of concept which will need tweaks to do whatever you're trying to do:
app.js:
const puppeteer = require("puppeteer");
const browserPromise = puppeteer.launch();
const endpointPromise = browserPromise
.then(browser => browser.wsEndpoint())
;
module.exports = {browserPromise, endpointPromise};
equities.js:
const puppeteer = require("puppeteer");
const {browserPromise, endpointPromise} = require("./app");
module.exports = async symbol => {
const endpoint = await endpointPromise;
console.log(endpoint);
//const browser = await puppeteer.connect({connectWSEndpoint: endpoint}) // FIXME
const browser = await browserPromise;
await browser.close();
};
index.js:
const equitiesFn = require("./equities");
(async () => {
await equitiesFn();
})();
Run node index and you should see the ws printed.
Note that you can wrap the exported promises in functions or as part of an object which is a layer of abstraction more typical for the interface of a library if you want. But this doesn't change the fundamental asynchrony. The client will call the exported functions and await the endpoint and/or browser promises through that extra layer of indirection,
require("./app").getBrowser().then(browser => /* */);
versus
require("./app").browserPromise.then(browser => /* */);
If you don't want to expose the browser object, that's fine, but I'd suggest exposing a function that closes the underlying browser so you can get a clean exit, e.g.
app.js:
const puppeteer = require("puppeteer");
const browserPromise = puppeteer.launch();
const closeBrowser = () =>
browserPromise.then(browser => browser.close())
;
module.exports = {closeBrowser};
index.js:
require("./app")
.closeBrowser()
.then(() => console.log("closed"))
;

Is there a conventional way to document Jest/Puppeteer test suites? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
Summary of problem: I'm writing several test suites using Jest and Puppeteer to automate end-to-end testing of my Angular JS app. I'm big on documentation, as it is important to help future developers get up to speed more quickly. Unfortunately, I don't know of a conventional/widely accepted method for documenting test suites written with Jest. I've already written an extensive README that explains the tools we're using, how my team configured Jest/Puppeteer, and how to get started writing tests. What I'm specifically wondering about is how to document WITHIN each test suite, or if it is even necessary to spend time doing that (I'm leaning towards yes, it is definitely worth spending time doing that on the latter question).
Here's some sample code that I'd like to document:
// index.spec.js
// Insert comment here describing test file (aka test suite)
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({headless: false});
const page = await browser.newPage();
// Insert comment here describing test suite
describe('load startpage', async () => {
// Insert comment here describing test
test('page loads', async () => {
await page.goto('https://my-site.com', {waitUntil: 'networkidle0'});
});
});
// Insert comment here describing test suite
describe('complete form', async () => {
// Insert comment here describing test
test('populate form', async () => {
let formSelector = 'form[name="form1"]';
await page.waitForSelector(formSelector, {timeout: 3000});
await page.waitForSelector(formSelector+' input', {timeout: 3000});
await page.click(formSelector+' input');
await page.keyboard.type('Hello World');
let submitButtonSelector = 'form[name="form1"] button[type="submit"]';
await page.click(submitButtonSelector);
});
// Insert comment here describing test
test('submit form', async() => {
let submitButtonSelector = 'form[name="form1"] button[type="submit"]';
await page.waitForSelector(submitButtonSelector, {timeout: 3000});
await page.click(submitButtonSelector);
});
});
await browser.close();
})();
What I've already tried:
I've researched a little about the conventional method for documenting Javascript via JSDoc, but I don't really think this applies here because I'm using the Jest and Puppeteer Apis, which I assume are wrappers for native Javascript functions.
Question: Do any of you Jest/Puppeteer hackers know of the proper way to document tests? Thank you in advance!
I'll try and tailor this response to not sound completely opinionated.
Yes, documentation is essential, but too much of it doesn't play well either.
In the case of tests, be it unit/integration/e2e, frameworks already give you all the constructs you'll ever need to specify/document your tests. To my knowledge, there aren't any other conventions to document test suites.
The describe/it/test/etc blocks should be thought of as documentation and they should guide any developer through the intentions of whoever wrote the test.
In rare cases, any other essential commentary can be done inline.
The beauty of tests is that when their specification is well written, it reads like a book when run. And yes, describing your tests in small phrases is hard, just like naming a variable. Takes practice but it's doable.
Any documentation needed besides that you already covered in your README.
Apart from having good test specifications, you'll gain much more value in ensuring the tests are written with consistency instead of trying to explain what's going on in detail in each test, let the code do that.

Getting around an async/await issue

I am creating a simple node script to learn the functionality of Cosmos DB. I want to create a way to not have to provide the following at the top of every async function (yes, I know I could chain the async calls with but that means still means I have to use a new db instance at the top of every function. So, I want to do something like this:
const {database} = await client.databases.createIfNotExists({id: databaseId});
const {container} = await database.containers.createIfNotExists({id: containerId});
With that said, I've bumped my head on this for a few hours and can't find a way to create one database and one container for all my functions to share. The idea (but not implementation because it doesn't work, is to do something like this:
getConnections = async () => {
const {database} = await client.databases.createIfNotExists({id: databaseId});
const {container} = await database.containers.createIfNotExists({id: containerId});
let connections = {};
connections.db = database;
connections.container = container;
return connections;
};
But since the getCoonections method is async (which it must be because the methods that would use it are, too) the function doesn't necessarily finish before the first insert is made in another function, thereby causing an exception.
Has anyone found a way to centralize these objects so I don't have to declare them in each async function of my app?
It sounds like you need to get these connections before the app does anything else. So why not simply make the loading of your app use async/await too?
async function init() {
const connections = await getConnections();
const app = initializeTheRestOfYourApp(connections); // now safe to do inserts
};
init();
This pretty much works now, Not sure why as there is no blocking between this init() and the next async method in the call chain uses the connection, but it's working. – David Starr - Elegant Code just now

Testing Error on node.js function with Mocha [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 7 years ago.
Improve this question
With Node.js installed and testing with mocha,
I have two files, numbers.js and test.js in the same directory
Following the top answer: What is the purpose of Node.js module.exports and how do you use it?
numbers.js
var myFunc = function myFunc() {
return 10;
};
exports.myFunc = myFunc;
test.js
var assert = require('assert');
var numbers = require('./numbers');
describe('numbers', function() {
it('first function returns 10', function() {
var result = numbers.myFunc;
assert.equal(result, 10);
});
});
But when running $ mocha it returns the error:
AssertionError: [Function: myFunc] == 10
What am I doing wrong?
You need to call your myFunc function. Add the parens
describe('numbers', function() {
it('first function returns 10', function() {
var result = numbers.myFunc(); <<<<<<
assert.equal(result, 10);
});
});
First of all, Inspect your numbers module is properly required or not.
var numbers = require('../numbers'); // Check path of file numbers.js
If yes, write :
var result = numbers.myFunc(); // your module exports 'myFunc' property not 'myfunc'
assert.equal(result, 10); // your function return '10' instead of '1'

Categories

Resources