I am testing a web app that renders some of its errors as HTML and gives no other indication that a problem occurred. After every click, I would like to see if a .error element exists.
I know I could manually add this after every click, but it's going to obfuscate my test and it will be easy to forget some instances. Is there a way to tell testcafe that a certain condition should make the test fail, even if I'm not explicitly checking for it?
I did something like this:
const scriptContent = `
window.addEventListener('click', function () {
if($('.error').length > 0) {
throw new Error($('.error').text());
}
});
`;
fixture`Main test`
.page`../../dist/index.html`.clientScripts(
{ content: scriptContent }
);
This injects a script onto the page I am testing. After every click, it uses jQuery to see if an error class exists. If it does, it throws an error on the page I'm testing. Testcafe reports the message of that error.
I'm hoping there's a better way.
Related
I am trying to E2E test an auth flow with Cypress that includes a third party method called BankID. BankId is integrated through three nested iframes that I can successfully access. However, when I type into the input field via cy.type('12345678912'), BankId does not register this as trusted events and never unlocks the submit button with the arrow.
According to this issue here, Cypress does not intend to support native browser events and suggests to use the package cypress-real-events. When using this via cy.realType('12345678912'), it actually succeeds in unlocking the submit button. However i can never successfully click the submit button, neither with .click() or even the package method .realClick().
The error is: "Failed to execute 'getComputedStyle' on 'Window': parameter 1 is not of type 'Element'."
I uploaded a sample repository with an minimal testing version here.
Any feedback or hints would be greatly appreciated :)
Here is the relevant code:
/// <reference types="cypress" />
import { skipOn } from '#cypress/skip-test'
describe('Recipe: blogs__iframes', () => {
skipOn('firefox', () => {
it('do it more generically', () => {
const getIframeBody = (identifier) => {
return cy
.get(identifier)
.its('0.contentDocument.body')
.should('not.be.empty')
.then(cy.wrap)
}
// Visiting the page index.html and getting iframe A
cy.visit('index.html').contains('XHR in iframe')
getIframeBody('iframe[data-cy="bankid"]').as('iframeA')
cy.get('#iframeA').within(() => {
getIframeBody('iframe[src="https://tools.bankid.no/bankid-test/auth"]').as('iframeB')
cy.get('#iframeB').within(() => {
getIframeBody('iframe[src^="https://csfe.bankid.no/CentralServerFEJS"]').as('iframeC')
// Now we are in the right place and it finds the correct input element.
// However, normal cypress command .type() fails and we have to use library cypress-real-events,
// which provides an event firing system that works literally like in puppeteer
cy.get('#iframeC').find('input[type="tel"]').should('be.visible').realType('12345678912')
// But for the button below, this library now doesn't help anymore:
// "Failed to execute 'getComputedStyle' on 'Window': parameter 1 is not of type 'Element'."
cy.get('#iframeC').find('button[type="submit"]').should('be.visible').first().realClick()
})
})
})
})
})
I also posted this problem on https://github.com/dmtrKovalenko/cypress-real-events/issues/226 and got an answer there:
Using .realClick({ scrollBehavior: false }); solved the issue.
The problem is if the webapp is not scrolling as expected, therefore leading to Cypress not finding the element. In my case, i made the iframe wider to avoid needing to scroll and the issue was still there, but the workaround solved it anyway.
Following lines of code used to work and stopped working after chrome upgrade to Version 74.0.3729.169 (Official Build) (64-bit). Now I get DOMException even though permission is set correctly. Appreciate if you can explain what is the bug and workaround. Exception details:
message:Document is not focused
name:NotAllowedError
code:0
navigator.permissions.query({ name: 'clipboard-read' }).then(result => {
// If permission to read the clipboard is granted or if the user will
// be prompted to allow it, we proceed.
if (result.state === 'granted' || result.state === 'prompt') {
navigator.clipboard.readText()
.then(text => {
//my code to handle paste
})
.catch(err => {
console.error('Failed to read clipboard contents: ', err);
});
}
}
This seems to happen when executing code from the devtools console or snippets.
Workaround:
You can execute the code below and focus on the window within 3 seconds, by clicking somewhere, or just by pressing <tab>.
e.g. from snippets
Ctrl-Enter
<Tab>
e.g. from console
Enter
<Tab>
setTimeout(async()=>console.log(
await window.navigator.clipboard.readText()), 3000)
The issue I was having was that I had an alert to say that the text had been copied, and that was removing focus from the document. Ironically, this caused the text to not be copied. The workaround was quite simple:
clipboard.writeText(clippy_button.href).then(function(x) {
alert("Link copied to clipboard: " + clippy_button.href);
});
Just show the alert when the Promise is resolved. This might not fix everybody's issue but if you came here based on searching for the error this might be the correct fix for your code.
As Kaiido said, your DOM need to be focused. I had the same problem during my development when i put a breakpoint in the code... The console developper took the focused and the error appear. With the same code and same browser, all work fine if F12 is closed
Problem
It's a security risk, clearly. :)
Solution
I assume you face this when you are trying to call it from the dev tools. Well, to make life easier, I am taking Jannis's answer, to a less adrenaline-oriented way. :)
I am adding a one-time focus listener to window to do the things magically after hitting "tab" from the Devtools.
function readClipboardFromDevTools() {
return new Promise((resolve, reject) => {
const _asyncCopyFn = (async () => {
try {
const value = await navigator.clipboard.readText();
console.log(`${value} is read!`);
resolve(value);
} catch (e) {
reject(e);
}
window.removeEventListener("focus", _asyncCopyFn);
});
window.addEventListener("focus", _asyncCopyFn);
console.log("Hit <Tab> to give focus back to document (or we will face a DOMException);");
});
}
// To call:
readClipboardFromDevTools().then((r) => console.log("Returned value: ", r));
Note: The return value is a Promise as it's an asynchronous call.
if you want to debug a and play around to view result. also can hide this <p></p>.
async function readClipboard () {
if (!navigator.clipboard) {
// Clipboard API not available
return
}
try {
const text = await navigator.clipboard.readText();
document.querySelector('.clipboard-content').innerText = text;
} catch (err) {
console.error('Failed to copy!', err)
}
}
function updateClipboard() {
// Here You Can Debug without DomException
debugger
const clipboard = document.querySelector('.clipboard-content').innerText;
document.querySelector('.clipboard-content').innerText = 'Updated => ' + clipboard;
}
<button onclick="readClipboard()">Paste</button>
<p class="clipboard-content"></p>
<button onclick="updateClipboard()">Edit</button>
As the exception message says, you need to have the Document actively focused in order to use this API.
Suppose there is a p element. you want to copy its innerText.
So, lets not use navigation.clipboard (because 0f the error your are facing)
So below given is a example which copies the innerText of the p element when that button is clicked. your do not to rely upon "clicking" the button manually by using the code below. you can trigger the "click" by executing code like pElement.click() from devtools console.
Your devtools console problem, that #jannis-ioannou mentioned in his post above, will not occur!
function myFunction() {
var copyText = document.getElementById("copy-my-contents");
var range = document.createRange();
var selection = window.getSelection();
range.selectNodeContents(copyText);
selection.removeAllRanges();
selection.addRange(range);
document.execCommand("copy");
}
<p id="copy-my-contents">copy-me</p>
<button onclick="myFunction()">Copy text</button>
I was facing this in a Cypress test, this fixed it for me:
first focus the copy icon/button which is about to be clicked, then
use cy.realClick from cypress-real-events instead of cy.click
I just discovered that I don't need to write any code to debug this. When you pause on a breakpoint in Chrome DevTools, it adds a small yellow box to the page you're debugging which says, "Paused in debugger" and has play and step over buttons. I've never used it before, preferring the more extensive controls in DevTools.
I just discovered that if you use the step over button in this yellow box, the DOM stays focused and no error is thrown.
Cypress - Use the right window/navigator and focus on the document.
I tried to programatically populate the clipboard in my Cypress test so I could paste the contents into a text-area input element.
When I was struggling with this issue I found out that there were two things causing the problem.
The first issue was that I used the wrong window. Inside the test function scope window returns the Window object in test scope, while cy.window() returns the Window object for the Application Under Test (AUT).
Second issue was that document was not in focus, which can be easily resolved by calling cy.window().focus();.
Since both result in the same DOMException:
NotAllowedError: Document is not focused.
It was not always clear that there were 2 issues going on. So when debugging this:
Make sure you use the right window/navigator of the page you are testing.
Make sure that the document is focused on.
See following Cypress tests demonstrating the above:
describe('Clipboard tests', () => {
before(() => {
cy.visit('/');
// Focus on the document
cy.window().focus();
});
it('window object in test scope is not the window of the AUT', () => {
cy.window().then((win) => {
expect(window === win).to.equal(false);
expect(window.navigator === win.navigator).to.equal(false);
})
});
it('See here the different results from the different Window objects', () => {
expect(window.navigator.clipboard.writeText('test').catch(
(exception) => {
expect(exception.name).to.equal('NotAllowedError')
expect(exception.message).to.equal('Document is not focused.')
},
));
cy.window().then((win) => {
return win.navigator.clipboard.writeText('test').then(() => {
return win.navigator.clipboard.readText().then(
result => expect(result).to.equal('test'),
);
});
});
})
});
We saw a handful of these in our production app, and after some digging it turned out that the root cause in our case was the user was copying a lot of data, and switched tabs during the process (thus causing the relevant DOM to lose focus, and triggering the error). We were only able to replicate this by using CPU throttling in chrome, and even then it doesn't happen every time!
There was nothing to be "fixed" in this case - instead we're just catching the error and notifying the user via a toast that the copy failed and that they should try again. Posting in case it's of use to anyone else seeing this!
If you're using Selenium and getting this, you need to bring the test window to the front:
webDriver.switchTo().window(webDriver.getWindowHandle());
I have to do this repeatedly so I have it in a loop with a Thread.sleep() until the the paste works.
Full details: https://stackoverflow.com/a/65248258/145976
Is there a way to get QUnit.js to not run the remaining tests after a single one fails?
Using the following code as an example:
QUnit.test('test1', function(assert) {
assert.equal(1,1);
assert.equal(1,2);
assert.equal(3,3);
});
QUnit.test('test2', function(assert) {
assert.equal(4,4);
assert.equal(5,5);
assert.equal(6,6);
});
Is there some way to get QUnit to stop executing after the assert.equal(1,2)? This means that test2 should never be run.
The best way to stop QUnit after test case fail will be
QUnit.testDone( function( details ) {
if (details.failed>0){
QUnit.config.queue.length = 0;
}
});
Okay, based on my comments above I ran the code below and things to stop as I think you want them to. Again, as I said in the comments, I would really investigate whether this is a good idea. Generally you want your tests to be idempotent such that any one failure does not affect any other test.
Note that we have to set the reorder config option to false here, otherwise QUnit will attempt to run the previously failed test first to "short circuit" things, but you don't want that I'm guessing. I also added a "test0" just to see the fill effect.
QUnit.config.reorder = false;
// This is how we detect the failure and cancel the rest of the tests...
QUnit.testDone(function(details) {
console.log(details);
if (details.name === 'test1' && details.failed) {
throw new Error('Cannot proceed because of failure in test1!');
}
});
QUnit.test('test0', function(assert) {
assert.equal(1,1);
assert.equal(2,2);
assert.equal(3,3);
});
QUnit.test('test1', function(assert) {
assert.equal(1,1);
assert.equal(1,2);
assert.equal(3,3);
});
QUnit.test('test2', function(assert) {
assert.equal(4,4);
assert.equal(5,5);
assert.equal(6,6);
});
You won't get any visual feedback that the tests were canceled because this isn't really interacting with the QUnit UI. However, because we threw an Error object you can open the developer console and see the output there:
I am getting a totally bizarre issue in Chrome v. 33 that looks as if the string comparison operator is broken. It only occurs with the developer tools closed. I have the following function:
function TabSelected(data) {
var tab, was_design;
this.data = data;
tab = this.data.tab;
was_design = tab === 'design';
if (this.data.tab === 'design') {
this.tab = 1;
} else {
this.tab = 2;
console.log('was_design');
console.log(was_design);
console.log('is_design');
console.log(tab === 'design');
}
}
Which I call like so:
new TabSelected({
tab: 'design'
});
I have a setInterval running that runs this code every 50 ms. Most of the time, the if statement picks the first code path, so nothing gets logged to the console. However, after about ~8 seconds, it goes down the else code path. When I open the developer tools afterwards (since the bug doesn't happen when they're closed), I see the following log output:
was_design (index):96624
false (index):96625
is_design (index):96626
true (index):96627
I am... confused by this. I've also tried logging the contents of tab, which is in fact 'design', and logging this, which is a new TabSelected instance.
Am I losing my mind? Is Chrome losing it's mind?
UPDATE: I was able to reproduce it in a simplified setting: http://jsfiddle.net/WBpLG/24/. I'm pretty sure this is a bug with Chrome and I've filed an issue, see answer below.
Make this change and the problem should go away:
if (this.data.tab === 'design') {
to
if (String(this.data.tab) === 'design') {
However, I can confirm that typeof this.data.tab === 'string' both before the if clause and during the else, so I think this is only a partial answer at best.
Alternatively, I can also clear the problem by adjusting NewElementButtonSectionOpened.prototype.previous_requirement on line 59440:
// Create a single instance of the requirement and store it in the closure.
var cachedReq = new TabSelected({ tab: 'design' });
// Now just return that one instance over and over again.
NewElementButtonSectionOpened.prototype.previous_requirement = function() {
// deleted line: return new TabSelected({ tab: 'design' });
return cachedReq;
};
While both of these solutions fix the problem on my machine, it is not clear to me why this works.
I am afraid to mention it, but at one point, I was also able to prevent the error from happening by adding a throw new Error("..."); line in your else block. In other words, changing something in the else block altered the behavior of the if check. My only clue here is that the length of the error message mattered. For a while there, I could clear the error or cause the error consistently by altering the length of an error message that would never be thrown. This is so bizarre that I must surely have been mistaken, and indeed, I can no longer replicate it.
However, this is an extremely large JavaScript file. Maybe there is something to that. Maybe it is just a ghost story. This problem is certainly quite creepy enough without it, but just in case somebody else sees something similar... You aren't alone.
I was able to create a simple reproduction case, so I've filed a bug with Chromium.
The necessary conditions seem to be: a setInterval or repeating setTimeout, an expensive computation in the body of the interval, a call to a new Object passing data that contains a string, and a string comparison.
I noticed that qUnit doesn't give any notice when an exception happens in a later part of the test. For example, running this in a test():
stop();
function myfun(ed) {
console.log('resumed');
start(); //Resume qunit
ok(1,'entered qunit again');
ok(ed.getContent()== 'expected content') // < causes exception, no getContent() yet.
}
R.tinymce.onAddEditor.add(myfun)
in an inner iframe on the page will cause an exception (TypeError: ed.getContent is not a function),
but nothing in Qunit status area tells this. I see 0 failures.
(R being the inner iframe, using technique here: http://www.mattevanoff.com/2011/01/unit-testing-jquery-w-qunit/) Would I be correct in assuming this isn't the best way to go for testing sequences of UI interaction that cause certain results? Is it always better to use something like selenium, even for some mostly-javascript oriented frontend web-app tests?
As a side note, the Firefox console shows the console.log below the exception here, even though it happened first... why?
If you look into qUnit source code, there are two mechanisms handling exceptions. One is controlled by config.notrycatch setting and will wrap test setup, execution and teardown in try..catch blocks. This approach won't help much with exceptions thrown by asynchronous tests however, qUnit isn't the caller there. This is why there is an additional window.onerror handler controlled by Test.ignoreGlobalErrors setting. Both settings are false by default so that both kinds of exceptions are caught. In fact, the following code (essentially same as yours but without TinyMCE-specific parts) produces the expected results for me:
test("foo", function()
{
stop();
function myfun(ed)
{
start();
ok(1, 'entered qunit again');
throw "bar";
}
setTimeout(myfun, 1000);
});
I first see a passed tests with the message "entered qunit again" and then a failed one with the message: "uncaught exception: bar." As to why this doesn't work for you, I can see the following options:
Your qUnit copy is more than two years old, before qUnit issue 134 was fixed and a global exception handler added.
Your code is changing Test.ignoreGlobalErrors setting (unlikely).
There is an existing window.onerror handler that returns true and thus tells qUnit that the error has been handled. I checked whether TinyMCE adds one by default but it doesn't look like it does.
TinyMCE catches errors in event handlers when calling them. This is the logical thing to do when dealing with multiple callbacks, the usual approach is something like this:
for (var i = 0; i < callbacks.length; i++)
{
try
{
callbacks[i]();
}
catch (e)
{
console.error(e);
}
}
By redirecting all exceptions to console.error this makes sure that exceptions are still reported while all callbacks will be called even if one of them throws an exception. However, since the exception is handled jQuery can no longer catch it. Again, I checked whether TinyMCE implements this pattern - it doesn't look like it.
Update: Turns out there is a fifth option that I didn't think of: the exception is fired inside a frame and qUnit didn't set up its global error handler there (already because tracking frame creation is non-trivial, a new frame can be created any time). This should be easily fixed by adding the following code to the frame:
window.onerror = function()
{
if (parent.onerror)
{
// Forward the call to the parent frame
return parent.onerror.apply(parent, arguments);
}
else
return false;
}
Concerning your side-note: the console object doesn't guarantee you any specific order in which messages appear. In fact, the code console.log("foo");throw "bar"; also shows the exception first, followed by the log message. This indicates that log messages are queued and handled delayed, probably for performance reasons. But you would need to look into the implementation of the console object in Firefox to be certain - this is an implementation detail.