Develop Reference

Cypress error when testing nested iframes in headless mode - race condition

I am testing a web app and the test runs reliably in headed mode (cypress open) but has errors in headless mode (cypress run), so it's likely a race condition that I cannot resolve. The error message is:
[36819:0223/163815.745047:ERROR:system_services.cc(34)] SetApplicationIsDaemon: Error Domain=NSOSStatusErrorDomain Code=-50 "paramErr: error in user parameter list" (-50)
This error is mentioned again when Cypress creates a video of the incident:
- Started processing: Compressing to 32 CRF
2022-02-23 17:00:19.700 Cypress Helper[37571:416134] In -[NSApplication(NSQuietSafeQuit) _updateCanQuitQuietlyAndSafely], _LSSetApplicationInformationItem(NSCanQuitQuietlyAndSafely) returned error -50
- Finished processing: /Users/malte.wirz/Documents/iframes-cypress-issue/cypress/videos/iframe-github.js.mp4 (3 seconds)
I created a demo repository here. To reproduce, clone it, run yarn to install, and yarn cypress:run. The test does pass, but with the error mentioned above.
I assume that the error stems from accessing the nested iframes and I tested 5 different approaches so far, but to no avail. I especially made sure that the function getIframeBody waits until each iframe and the requested element is ready. The error also creates a video, but you can only see the successful run, the error message is not visible there.
Any help on how to debug this further is much appreciated.
describe('Testing Iframe within Iframes', () => {
it('Visits the iframe website and accesses the iframe within the iframe', () => {
const getIframeBody = (iframeSelector, elementSelectorInIframe) => {
return cy
.get(iframeSelector)
.its('0.contentDocument.body', {timeout: 30000})
.should((body) => {
expect(Cypress.$(body).has(elementSelectorInIframe).length).gt(0)
})
.then(cy.wrap)
}
// Visiting the page index.html and getting iframe A
cy.visit('index.html').contains('XHR in iframe')
getIframeBody('iframe[data-cy="bankid"]', 'iframe[src="https://tools.bankid.no/bankid-test/auth"]').as('iframeA')
cy.get('#iframeA').within(() => {
getIframeBody('iframe[src="https://tools.bankid.no/bankid-test/auth"]', 'iframe[src^="https://csfe.bankid.no/CentralServerFEJS"]').as('iframeB')
cy.get('#iframeB').within(() => {
getIframeBody('iframe[src^="https://csfe.bankid.no/CentralServerFEJS"]', 'input[type="tel"]').as('iframeC')
// Now we are in the right place and it finds the correct input element.
// However, normal cypress command .type() fails and we have to use library cypress-real-events,
// which provides an event firing system that works literally like in puppeteer
cy.get('#iframeC').find('input[type="tel"]').should('be.visible').realType('12345678912')
// But for the button below, this library now doesn't help anymore:
// "Failed to execute 'getComputedStyle' on 'Window': parameter 1 is not of type 'Element'."
// This was solved by using {scrollBehavior:false}.
cy.get('#iframeC').find('button[type="submit"]').should('be.visible').first().realClick({scrollBehavior:false})
})
})
})
})

I got some feedback that the above "ERROR:system_services.cc(34)" is not critical and does not cause flaky or unsuccessful tests, therefore there are no action points.

How to check if nodejs process failed due to a timeout?

I spawn a synchronous process like this:
child = spawnSync('somecommand', { stdio: 'inherit', cwd: someDir, timeout: 30000});
Is there a reliable way to check if the child process failed due to the timeout option or some other error?

You should check for the log of node which is running down there somewhere. You must be running this with a cli so you can see the log there. Or you can look for where the log is saved. Everything, specially the errors are noted there.

If anyone runs into the same problem, there is a simple way to retrieve the timeout error code at run-time:
if (child.error && child.error.code === "ETIMEDOUT") {
console.log("child timed out");
}

How to ignore the "ResizeObserver loop limit exceeded" in TestCafe

I'm currently using TestCafe for some e2e tests. I've run into the following error
1) - Error in Role initializer -
A JavaScript error occurred on "http://localhost:3000/".
Repeat test actions in the browser and check the console for errors.
If you see this error, it means that the tested website caused it. You can fix it or disable tracking JavaScript errors in TestCafe. To do the latter, enable the "--skip-js-errors" option.
If this error does not occur, please write a new issue at:
"https://github.com/DevExpress/testcafe/issues/new?template=bug-report.md".
JavaScript error details:
ResizeObserver loop limit exceeded
No stack trace available
A bit of research suggests that the ResizeObserver loop limit exceeded issue is a benign error.
However, it causes my test to fail. Is there any way I can ignore this specific error without using the --skip-js-errors flag, as I would prefer to not ignore all JavaScript errors because of this one issue

As far as I understand, this error occurs when ResizeObserver cannot deliver all observations within a single animation frame. A person who is the author of the ResizeObserver specification assures that it can be safely ignored: ResizeObserver loop limit exceeded
Chrome and Firefox don't display it by default. You can only catch it when you set an explicit onerror handler:
window.onerror = e => console.log(e);
You can see that this error is reproduced on the Google Sign In page without TestCafe. I added an onerror handler to the page and got ResizeObserver loop completed with undelivered notifications. in Firefox and ResizeObserver loop limit exceeded in Chrome.
As a workaround, you can specify the --skip-js-errors flag when starting TestCafe. I admit that it's not the best approach since you will suppress all Javascript errors on a tested page.
A more reliable way is to add a global window error handler explicitly in your tests via client scripts:
import { Selector, t } from 'testcafe';
// Constants
const gmailEmailInput = Selector("#identifierId");
const gmailNextButton = Selector(".CwaK9");
const gmailPasswordInput = Selector("input[type='password']");
const explicitErrorHandler = () => {
window.addEventListener('error', e => {
if(e.message === 'ResizeObserver loop completed with undelivered notifications.' ||
e.message === 'ResizeObserver loop limit exceeded') {
e.stopImmediatePropagation();
}
})
}
fixture("Gmail login test")
.clientScripts({ content: `(${explicitErrorHandler.toString()})()` });
test("Not trigger JS error when logging in to Gmail", async testController => {
await testController
.navigateTo("https://mail.google.com")
.typeText(gmailEmailInput, "someuser#gmail.com")
.click(gmailNextButton)
.typeText(gmailPasswordInput, "password")
});
I copypasted the workaround from here.

Why does exception within frame get no notification in qUnit?

I noticed that qUnit doesn't give any notice when an exception happens in a later part of the test. For example, running this in a test():
stop();
function myfun(ed) {
console.log('resumed');
start(); //Resume qunit
ok(1,'entered qunit again');
ok(ed.getContent()== 'expected content') // < causes exception, no getContent() yet.
}
R.tinymce.onAddEditor.add(myfun)
in an inner iframe on the page will cause an exception (TypeError: ed.getContent is not a function),
but nothing in Qunit status area tells this. I see 0 failures.
(R being the inner iframe, using technique here: http://www.mattevanoff.com/2011/01/unit-testing-jquery-w-qunit/) Would I be correct in assuming this isn't the best way to go for testing sequences of UI interaction that cause certain results? Is it always better to use something like selenium, even for some mostly-javascript oriented frontend web-app tests?
As a side note, the Firefox console shows the console.log below the exception here, even though it happened first... why?

If you look into qUnit source code, there are two mechanisms handling exceptions. One is controlled by config.notrycatch setting and will wrap test setup, execution and teardown in try..catch blocks. This approach won't help much with exceptions thrown by asynchronous tests however, qUnit isn't the caller there. This is why there is an additional window.onerror handler controlled by Test.ignoreGlobalErrors setting. Both settings are false by default so that both kinds of exceptions are caught. In fact, the following code (essentially same as yours but without TinyMCE-specific parts) produces the expected results for me:
test("foo", function()
{
stop();
function myfun(ed)
{
start();
ok(1, 'entered qunit again');
throw "bar";
}
setTimeout(myfun, 1000);
});
I first see a passed tests with the message "entered qunit again" and then a failed one with the message: "uncaught exception: bar." As to why this doesn't work for you, I can see the following options:
Your qUnit copy is more than two years old, before qUnit issue 134 was fixed and a global exception handler added.
Your code is changing Test.ignoreGlobalErrors setting (unlikely).
There is an existing window.onerror handler that returns true and thus tells qUnit that the error has been handled. I checked whether TinyMCE adds one by default but it doesn't look like it does.
TinyMCE catches errors in event handlers when calling them. This is the logical thing to do when dealing with multiple callbacks, the usual approach is something like this:
for (var i = 0; i < callbacks.length; i++)
{
try
{
callbacks[i]();
}
catch (e)
{
console.error(e);
}
}
By redirecting all exceptions to console.error this makes sure that exceptions are still reported while all callbacks will be called even if one of them throws an exception. However, since the exception is handled jQuery can no longer catch it. Again, I checked whether TinyMCE implements this pattern - it doesn't look like it.
Update: Turns out there is a fifth option that I didn't think of: the exception is fired inside a frame and qUnit didn't set up its global error handler there (already because tracking frame creation is non-trivial, a new frame can be created any time). This should be easily fixed by adding the following code to the frame:
window.onerror = function()
{
if (parent.onerror)
{
// Forward the call to the parent frame
return parent.onerror.apply(parent, arguments);
}
else
return false;
}
Concerning your side-note: the console object doesn't guarantee you any specific order in which messages appear. In fact, the code console.log("foo");throw "bar"; also shows the exception first, followed by the log message. This indicates that log messages are queued and handled delayed, probably for performance reasons. But you would need to look into the implementation of the console object in Firefox to be certain - this is an implementation detail.

Why there is no error thrown with Strophe.js?

Example code:
var connection = null;
function onConnect(status) {
im_a_big_error.log('wtf');
// Why it doesn't throw me an error here ??
}
$().ready(function() {
connection = new Strophe.Connection('http://localhost:8080/http-bind');
connection.connect('admin#localhost', 'admin', onConnect);
});
It doesn't throw me an error in my Chrome console.
Do you have an idea to resolve this issue?

Yes, Strophe often catch errors by itself and currently doesn't provide any ability to get connection error information. While error catching is ok, the impossibility of catching errors by yourself is not very good. But you can fix it with the following code:
$().ready(function() {
connection = new Strophe.Connection('http://localhost:8080/http-bind');
connection._hitError = function (reqStatus) {
this.errors++;
Strophe.warn("request errored, status: " + reqStatus + ",
number of errors: " + this.errors);
if (this.errors > 4) this._onDisconnectTimeout();
myErrorHandler(reqStatus, this.errors);
};
connection.connect('admin#localhost', 'admin', onConnect);
});
where myErrorHandler is your custom connection error handler.

Yes, strophe swallows errors. Worse; After an error is thrown, the callback won't return true as it should, and strophe will remove the handler. As soon as an error occurs, the callback will never be called again.
I found the code from the current answer a bit hard to use. Internally, we use the following wrapper for every callback;
function callback(cb) {
// Callback wrapper with
// (1) proper error reporting (Strophe swallows errors)
// (2) always returns true to keep the handler installed
return function() {
try {
cb.apply(this, arguments);
} catch (e){
console.log('ERROR: ' + (e.stack ? e.stack : e));
}
// Return true to keep calling the callback.
return true;
};
}
This wrapper would be used as following in the code of the question;
connection.connect('admin#localhost', 'admin', callback(onConnect));

I've been playing with Strophe for a while now and I had to modify its default error handling routine to fit our needs
Strophe.js - log function - by default contains nothing - I added calls to my server side logging service for level === ERROR and level === FATAL
Strophe.js - run function - the default behavior for error is to remove the handler and to rethrow the error - since I already log the error server side I don't rethrow the error and decided to keep the handler (even if it failed). This behavior could make sense (or not) depending on your own implementation - since I use custom messages and have a rather complicated message processing routine I don't want the client to stop just because a message was not properly formatted when sent so I want to keep the handler, error or not. I replace the throw e line inside the run function with result = true;
Strope.js _hitError - as I mentioned, I don't want the client to ever disconnect so I rewrote the default behavior to never disconnect (no matter how high the error counter)
Hope these thoughts are of help to others - leave a comment if you have questions/want details.

I had a similar problem which I fixed using the approach given by tsds above. However with minimal modification. I created two connect methods one as connect and the other as connect_bak I placed the script
this.connection._hitError=function (reqStatus) {
client.connect_bak();
};
in my connectHandler function as well as the connect function. Such that the function is always binded on connect.