I'm using puppeteer to navigate my website. I want to wait for an api that sometimes gets called and sometimes not. I'm using
await page.waitForResponse((response =>response.url().includes(myurl)), { timeout: 1000 });
to wait for that api. This works fine when the api gets called, but whenever the api doesn't get called, it crashes and the page isn't same anymore. So, I want to deep copy the page so that I can just check for the api via it's copy and even if that page gets damaged. I will have another that I can use.
I think you don't need to copy your page. That's probably not doable very easy and seems like a bit of overkill. Instead, preventing the page from crashing would be a simpler approach.
Try something like this:
async function waitForApi(url, timeoutMs) {
try {
console.log('waiting ', timeoutMs+'ms for special API. url:', url);
const opts = { timeout: timeoutMs || 1000 };
await page.waitForResponse(response => response.url().includes(url), opts);
console.log('Special API was called!.');
return true;
} catch(err) {
console.log('Special Api was appearantly not called. (Or may be failed.. Error:', err);
return false;
}
}
// example call of waitForApi ..
const myUrl = '...'
const apiCalled = await waitForApi(myUrl, 1000)
if(apiCalled) {
// do stuff if you want to..
} else {
// do stuff if you want to..
}
This should now log if the api was called or not and when needed you can handle the cases differently.
Related
I am trying to test a react webapp (created in a separate project), that contains a popup where there's an input containing a google auto-complete for cities:
(I changed text because of language)
I have in "search city" a text input where if data is inserted, google searches for cities and returns results (eg I search Rome, Italy):
When I press "save data" there's a function that checks google results, then closes the popup:
in a file:
export const useGoogleApiDesktop = () => {
let autocompleteService
if (window.google && window.google.maps) {
autocompleteService = new window.google.maps.places.AutocompleteService()
}
}
in another file (the one called):
const googleApi = useGoogleApiDesktop()
const onSubmitClick = useCallback(async () => {
[...]
const res: GoogleApiPlacesResponse = await googleApi.autocompleteService.getPlacePredictions({
input: addressComputed,
types: ['(cities)'],
componentRestrictions: { country: 'it' }
})
}, [])
When I use it in plain browser, everything works fine;
but if I try to launch it with cypress to test it, it returns me this error:
I am trying to avoid this error and simply go on and close the popup, since during my tests I do not need to write anything on that line; I only need to write something on the other textareas and close the popup.
Since I couldn't do it, I've tried to stub that call, but I am totally new in using cy.stub() and does not work:
function selectAddress(bookingConfig) {
// opens the popup
cy.get('.reservationsWhereAdd').click()
// trying to add the google library
const win = cy.state('window')
const document = win.document
const script = document.createElement('script')
script.src = `https://maps.googleapis.com/maps/api/js?key=[myApiKey]&libraries=places&language=it`
script.async = true
// this is commented since I don't think I need it
// window.initMap = function () {
// // JS API is loaded and available
// console.log('lanciato')
// }
// Append the ‘script’ element to ‘head’
document.head.appendChild(script)
// type something in some fields
cy.get('#street').type(bookingConfig.street)
cy.get('#streetNumber').type(bookingConfig.streetNum)
cy.get('#nameOnTheDoorbell').type(bookingConfig.nameOnTheDoorbell)
cy.get('#addressAlias').type(bookingConfig.addressAlias)
// this correctly finds and prints the object
console.log('--->', win.google.maps.places)
cy.stub(googleApi.autocompleteService, 'getPlacePredictions')
// this closes the popup
cy.get('.flex-1 > .btn').click()
}
this cy.stub however does not works, and I don't get why: it says
googleApi is not defined
Any idea on how to solve this? Thanks!
UPDATE:
After the error, working with the cypress window, I manually closed the popup, reopened it, filled the fields, and clicked on save data. It worked, so I added a cy.wait(1000) just after opening the popup and it works for 95% of the times (9 times on 10). Any Idea on how to "wait for loading the google api, then fill the fields"?
As the update block said, I discovered that the problem was that it kept really long time to load the google API, because it's not local and needs time to be retrieved.
So at first I just put a cy.wait(2000) before executing my code; but this couldn't be the answer: what happens if I run the code on a slow network? Or if it takes more time for my application to load?
So, i created a command, that first waits for the google API to load; if it fails to load after 5 attempts, the test fails.
Then, after that, my code is being executed. This way my test won't fail really easily.
Here's the code:
in cypress/support/command.js
Cypress.Commands.add('waitForGoogleApi', () => {
let mapWaitCount = 0
const mapWaitMax = 5
cyMapLoad()
function cyMapLoad() {
mapWaitCount++
cy.window().then(win => {
if (typeof win.google != 'undefined') {
console.log(`Done at attempt #${mapWaitCount}:`, win)
return true
} else if (mapWaitCount <= mapWaitMax) {
console.log('Waiting attempt #' + mapWaitCount) // just log
cy.wait(2000)
cyMapLoad()
} else if (mapWaitCount > mapWaitMax) {
console.log('Failed to load google api')
return false
}
})
}
})
in file you want to use it:
cy.waitForGoogleApi().then(() => {
// here comes the code to execute after loading the google Apis
})
I am trying to dump out a few key measurements to console when my test runs, rather than getting them from the reporter output, but I can't see how to grab the time taken for the last step to execute. Here's a simplified version based on the docs for request.timing() but I don't think that what I'm doing is classed as a request:
const { test, expect } = require('#playwright/test');
test('ApplicationLoadTime', async ({ page }) => {
// Wait for applications to load
await page.waitForSelector('img[alt="Application"]');
// Not working! - get time for step execution
const [fir] = await Promise.all([
page.click('text=Further information requested'),
page.waitForSelector('img[alt="Application"]')
]);
console.log(fir.timing());
});
The click on "Further information requested" causes the page to be modified based on an AJAX call in the background and the appearance of the Application img tells me it's finished. Is this possible or do I need to rely on the reports instead?
fir is going to be undefined in your code as page.click() doesn't return anything. You need to wait for the request whose timing you're interested in, use page.waitForEvent('requestfinished') or waitForNavigation:
const { test, expect } = require('#playwright/test');
test('ApplicationLoadTime', async ({ page }) => {
// Wait for applications to load
await page.waitForSelector('img[alt="Application"]');
const [fir] = await Promise.all([
// Wait for the request
page.waitForEvent('requestfinished', r => r.url() == '<url of interest>'),
page.click('text=Further information requested'),
page.waitForSelector('img[alt="Application"]')
]);
console.log(fir.timing());
});
I have headless off and I want to wait untill user redirect to some page. I could use waitForRequest from puppeteer API but I don't know exact url it just must pass few circumstances.
So I use waitForFunction and check circumstances there but when I redirect to correct URL then I need to refresh page to pass circumstances for some reason.
My code is:
try {
await page.waitForFunction(() => {
if(window &&
window.location &&
window.location.hostname) {
const host = window.location.hostname.split('.');
if(!host.includes('www') &&
!host.includes('new') &&
host.includes('margonem') &&
host.length === 3) {
return true;
}
}
}, {
polling: 200,
timeout: 0
})
} catch (e) {
console.log(e);
}
and when I redirect to URL which pass all of above if's then I need to reload page to actually see that it return true. Why it works like this? I don't want user to be forced to refresh page after he enter correct one.
I have headless off and I want to wait untill user redirect to some page.
just use waitForNavigation().
In combination with a click make sure you use this pattern:
const [response] = await Promise.all([
page.waitForNavigation(waitOptions),
page.click(selector, clickOptions),
]);
waitForNavigation also returns a response object that you can then inspect
I could use waitForRequest from puppeteer API but I don't know exact url it just must pass few circumstances.
in this case puppeteer injects the request as argument and you can just test this in your lambda function. For example:
page.waitForRequest(request => {
return request.url().includes('margonem') && request.method() === 'GET'
})
The simplest way I have found is using waitForFunction, it is simple to modify to fit your specifications as well as compact.
await page.waitForFunction("window.location.pathname == '/Welcome.aspx'")
Sometimes we need to wait until we reach a specific URL.
The best way to handle it is
let holdProgress = true;
while (holdProgress) {
await page.waitFor(300);
if (page.url().includes('/env-selector')) {
holdProgress = false;
}
}
I'm building a very simple scraper to get the 'now playing' info from an online radio station I like to listen too.
It's stored in a simple p element on their site:
data html location
Now using the standard apify/web-scraper I run into a strange issue. The scraping sometimes works, but sometimes doesn't using this code:
async function pageFunction(context) {
const { request, log, jQuery } = context;
const $ = jQuery;
const nowPlaying = $('p.js-playing-now').text();
return {
nowPlaying
};
}
If the scraper works I get this result:
[{"nowPlaying": "Hangover Hotline - hosted by Lamebrane"}]
But if it doesn't I get this:
[{"nowPlaying": ""}]
And there is only a 5 minute difference between the two scrapes. The website doesn't change, the data is always presented in the same way. I tried checking all the boxes to circumvent security and different mixes of options (Use Chrome, Use Stealth, Ignore SSL errors, Ignore CORS and CSP) but that doesn't seem to fix it unfortunately.
Scraping instable
Any suggestions on how I can get this scraping task to constantly return the data I need?
It would be great if you can attach the URL, it will help me to find out the problem.
With the information you provided, I guess that the data you want to are loaded asynchronously. You can use context.waitFor() function.
async function pageFunction(context) {
const { request, log, jQuery } = context;
const $ = jQuery;
await context.waitFor(() => !!$('p.js-playing-now').text());
const nowPlaying = $('p.js-playing-now').text();
return {
nowPlaying
};
}
You can pass the function to wait, and I will wait until the result of the function will be true. You can check the doc.
I am trying to simulate the user operation on CefSharp(OffScreen) using JavaScript.
Once I load the page (https://www.w3.org),
I am trying to search in the search bar,
click the search button
open first result
So I have used,
await browser.EvaluateScriptAsync("document.getElementsByName('q')[0].value = 'CSS';");
await browser.EvaluateScriptAsync("document.getElementById('search-submit').click();");
await browser.EvaluateScriptAsync("document.getElementById('r1-0').click();");
But the issue I am facing is, to take screenshot, I have to use Thread.Sleep(x) in between and at the end for the pages to load before doing the next operation or take the screenshot.
Is there anyway to avoid the SLEEP and detect when the loading is done, to do the next operation?
I tried ExecuteScriptAsync also, same issue with that also.
You can modify and use the LoadPageAsync() function from the OffScreenExample for this purpose.
Just replace alter the parameters by removing the url as you wont be using it, and remove the if statement using the url.
Then on calling the LoadPageAsync after EvaluateScriptAsync with the corresponding browser object.
The Function will look like this after modification.
public static Task LoadPageAsync(IWebBrowser browser)
{
var tcs = new TaskCompletionSource<bool>();
EventHandler<LoadingStateChangedEventArgs> handler = null;
handler = (sender, args) =>
{
if (!args.IsLoading)
{
browser.LoadingStateChanged -= handler;
tcs.TrySetResultAsync(true);
}
};
browser.LoadingStateChanged += handler;
return tcs.Task;
}
Usage will be like,
await browser.EvaluateScriptAsync("document.getElementsByName('q')[0].value = 'CSS';");
await browser.EvaluateScriptAsync("document.getElementById('search-submit').click();");
await LoadPageAsyncCompleted(browser);
await browser.EvaluateScriptAsync("document.getElementById('r1-0').click();");
await LoadPageAsyncCompleted(browser);