Using chromeless to run tests every 5 minutes to verify I can log into a website and that content is available.
Code looks like this:
const { Chromeless } = require('chromeless')
async function run() {
const chromeless = new Chromeless({
remote: {
endpointUrl: 'https://abc1234567.execute-api.us-west-2.amazonaws.com/dev',
apiKey: 'abc123xyz987'
}
})
const screenshot = await chromeless
.clearCookies()
.setUserAgent('some-user-agent')
.goto('https://www.example.com')
.type('username', 'input[name="username"]')
.type('super_secret', 'input[name="password"]')
.click('#loginButton')
.wait('#TitleTagOnceLoggedIn')
.screenshot()
console.log(screenshot)
const report = await chromeless
.setUserAgent('some-user-agent')
.goto('https://www.example.com/only/accessible/if/logged/in')
.wait('#TitleTagForPageWhenLoggedIn')
.screenshot()
console.log(report)
await chromeless.end()
}
run().catch(console.error.bind(console))
This works fine, however once every 10-20 times I run it, I receive the following error
Chromeless Proxy disconnected due to inactivity (no commands sent for 30 seconds).
In the cloudwatch logs from my lambda function I get a similar error
Timing out. No requests received for 30 seconds
How can I set this to retry if I get the inactivity errors?
Convert
await chromeless.end()
to
chromeless.end().catch(callback).then(callback)
and move it into a non-async function, e.g. function end (callback) {...}.
Then have your code wait for run to finish executing and then execute end with a callback handling the result. See https://github.com/graphcool/chromeless/issues/259
Related
I have used puppeteer for one of my projects to open webpages in headless chrome, do some actions and then close the page. These actions, however, are user dependent. I want to attach a lifetime to the page, where it closes automatically after, say 30 minutes, of opening irrespective of whether any action is performed or not.
I have tried setTimeout() functionality of Node JS but it didn't work (or I just couldn't figure how to make it work).
I have tried the following:
const puppeteer = require('puppeteer-core');
const browser = await puppeteer.connect({browserURL: browser_url});
const page = await browser.newPage();
// timer starts ticking here upon creation of new page (maybe in a subroutine and not block the main thread)
/**
..
Do something
..
*/
// timer ends and closePage() is triggered.
const closePage = (page) => {
if (!page.isClosed()) {
page.close();
}
}
But this gives me the following error:
Error: Protocol error: Connection closed. Most likely the page has been closed.
Your provided code should work as excepted. Are you sure the page is still opened after the timeout and it is indeed the same page?
You can try this wrapper for opening pages and closing them correctly.
// since it is async it won't block the eventloop.
// using `await` will allow other functions to execute.
async function openNewPage(browser, timeoutMs) {
const page = await browser.newPage()
setTimeout(async () => {
// you want to use try/catch for omitting unhandled promise rejections.
try {
if(!page.isClosed()) {
await page.close()
}
} catch(err) {
console.error('unexpected error occured when closing page.', err)
}
}, timeoutMs)
}
// use it like so.
const browser = await puppeteer.connect({browserURL: browser_url});
const min30Ms = 30 * 60 * 1000
const page = await openNewPage(browser, min30Ms);
// ...
The above only closes the Tabs in your browser. For closing the puppeteer instance you would have to call browser.close() which could may be what you want?
page.close returns a promise so you need to define closePage as an async function and use await page.close(). I believe #silvan's answer should address the issue, just make sure to replace if condition
if(page.isClosed())
with
if(!page.isClosed())
I am trying to dump out a few key measurements to console when my test runs, rather than getting them from the reporter output, but I can't see how to grab the time taken for the last step to execute. Here's a simplified version based on the docs for request.timing() but I don't think that what I'm doing is classed as a request:
const { test, expect } = require('#playwright/test');
test('ApplicationLoadTime', async ({ page }) => {
// Wait for applications to load
await page.waitForSelector('img[alt="Application"]');
// Not working! - get time for step execution
const [fir] = await Promise.all([
page.click('text=Further information requested'),
page.waitForSelector('img[alt="Application"]')
]);
console.log(fir.timing());
});
The click on "Further information requested" causes the page to be modified based on an AJAX call in the background and the appearance of the Application img tells me it's finished. Is this possible or do I need to rely on the reports instead?
fir is going to be undefined in your code as page.click() doesn't return anything. You need to wait for the request whose timing you're interested in, use page.waitForEvent('requestfinished') or waitForNavigation:
const { test, expect } = require('#playwright/test');
test('ApplicationLoadTime', async ({ page }) => {
// Wait for applications to load
await page.waitForSelector('img[alt="Application"]');
const [fir] = await Promise.all([
// Wait for the request
page.waitForEvent('requestfinished', r => r.url() == '<url of interest>'),
page.click('text=Further information requested'),
page.waitForSelector('img[alt="Application"]')
]);
console.log(fir.timing());
});
I have a simple update function which sometimes executes really slow compared to other times.
// Five executions with very different execution times
Finished in 1068ms // Almost six times slower than the next execution
Finished in 184ms
Finished in 175ms
Finished in 854ms
Finished in 234ms
The Function is triggered from the frontend and doesn't run on Firebase Cloud Functions.
const startAt = performance.now()
const db = firebase.firestore();
const ref = db.doc(`random/nested/document/${id}`);
ref.update({
na: boolean // a calculated boolean with array.includes(...)
? firebase.firestore.FieldValue.arrayRemove(referenceId)
: firebase.firestore.FieldValue.arrayUnion(referenceId)
})
.then(() => {
let endAt = performance.now();
console.log("Finished in " + (endAt - startAt) + "ms");
});
Is there anything I can improve to fix these performance differences?
Also the longer execution times dont only appear when removing something from an array or adding something to an array. It appears on adding and removing. Sometimes these execution times go up to 3000ms.
Similar to cold-starting a Cloud Function where everything is spun up, initialized and made ready for use, a connection to Cloud Firestore also needs to be resolved through DNS, ID tokens need to be obtained to authenticate the request, a socket to the server opened and any handshakes are exchanged between the server and the SDK.
Any new operations on the database can make use of the previous work taken to initialize the connection and that is why they look like they are faster.
Showing this as loose pseudocode:
let connection = undefined;
function initConnectionToFirestore() {
if (!connection) {
await loadFirebaseConfig();
await Promise.all([
resolveIpAddressOfFirebaseAuth(),
resolveIpAddressOfFirestoreInstance()
]);
await getIdTokenFromFirebaseAuth();
await addAuthenticationToRequest();
connection = await openConnection();
}
return connection;
}
function doUpdate(...args) {
const connection = await initConnectionToFirestore();
// do the work
connection.send(/* ... */);
}
await doUpdate() // has to do work of initConnectionToFirestore
await doUpdate() // reuses previous work
await doUpdate() // reuses previous work
await doUpdate() // reuses previous work
I want this to work on deploy. It does what its meant to do in local development, add a bunch of drugstores to a Firestore database. It also deploys successfully, and it even returns success on google cloud platform when I trigger it.
BUT it doesn't do anything when triggered from google cloud platform. Like i said, only works on development.
The only hint I have is what I get logged in the terminal when I ran it on develop.
firebase > getFarmacias()
'Successfully invoked function.'
firebase > info: User function triggered, starting execution
error: Function returned undefined, expected Promise or value
info: Execution took 20 ms, user function completed successfully
Here is the code of my function.
exports.getFarmacias = functions.pubsub.schedule('0 20 * * *').onRun((context) => {
// Declare http requests
const firstReq = axios.get('https://farmanet.minsal.cl/maps/index.php/ws/getLocalesTurnos')
const secondReq = axios.get('https://farmanet.minsal.cl/maps/index.php/ws/getLocalesUrgencia')
// Trigger both requests with axios.all
axios.all([firstReq, secondReq]).then(axios.spread((turno, urgencia) => {
const farmacias = [...turno.data, ...urgencia.data];
// for each element in 'farmacias' add a new document to 'firestore/farmacias'
farmacias.forEach(farmacia => {
return db.collection('farmacias').add({ farmacia })
})
})).catch(err=> console.log(err));
})
I've read other answers and try to return every line in the code, but I can't get it right.
I wrote an asynchronous function which is supposed to check if a given url returns a '200' from a get and otherwise wait a few seconds to try again a limited number of times. The code works just fine when I run it in my computer using node but when I transfer it to backend it only checks for the site once and then immediately stops when receiving an error. What am I doing wrong?
async function wait(url,delay=10000,attp=3){
let t0 = new Date(Date.now());
let attempts = 0;
console.log('starting...');
async function check(){
if(attempts<attp){
console.log('ATTEMPTS: ',attempts);
return await request.get(url).on('error',
async function(err){
console.log('ERROR: ',err);
attempts+=1;
return await setTimeout(()=>{check()},delay);
}).on('response',
async function(response){
if(response.statusCode===200){
let t1 = new Date(Date.now());
wixData.insert('pagSeguroTimeStats', { 'time': (t1 - t0) / 1000. });
return '200';
}else{
attempts+=1;
console.log('not 200');
return await setTimeout(()=>{check()},delay);
}
});
}else{
return '404';
}
}
return check();
}
Seems that there is a limit to how much time a backend function can run. From the Wix Code forum, it seems that the limit is 14 seconds, although this doesn't look like an official number from Wix.
The 14 second limit only applies to web functions.
Time
Wix allows web modules called from the frontend, HTTP functions, and router hooks to run for up to 14 seconds. This limitation applies to both free and premium sites. Any of these methods that take longer than 14 seconds receives a 504 response code. Note that after 14 seconds the code might still execute, but the connection to the client is closed so the results do not appear in the frontend. This error message appears in your log:
/backend/.js(w)/ timed out because it exceeded the maximum execution time.
I've got a .js function that just stop, and I get no error or anything.