I've tried to put the descriptive error message and exit the test run via fail() function but only with a partial success - it appears that I am doing something wrong... Here is my code:
it('set up internal budget', function(done) {
var acceptBudgetButton = element(by.buttonText('Accept Budget'));
page.setInternalBudget(); //setting budget values
browser.wait(function() {
return browser.isElementPresent(acceptBudgetButton);
}, 30000, 'Error - Accept Budget button is not visible.');
acceptBudgetButton.click();
done();
done.fail('Unable to setup internal budget. Terminating test run');
});
When "Accept Budget" button is not available I expected 3 things from this script:
1) See "Accept Budget button is not visible" error, followed by wait time out
2) See "Unable to setup internal budget. Terminating test run" error
3) Expect protractor to immediately exist test run, as we got into failure.
In the reality, only first expectation is met. Script keeps on running and I don't even get "Unable to setup internal budget" error... so it looks like my done.fail statement is being ignored.
What should I change to make all of my dreams/expectations come true?
Yes, nothing would be executed after done. What I would do is to define the browser.wait()'s success and failure handlers. Something along these lines:
browser.wait(function() {
return browser.isElementPresent(acceptBudgetButton);
}, 30000).then(
function () {
acceptBudgetButton.click();
done();
},
function () {
console.log('Error - Accept Budget button is not visible.');
done.fail('Unable to setup internal budget. Terminating test run');
});
Related
I have a very basic karma/jasmine setup with one test suite containing two tests. I expect the first test to fail and the second test to pass.
describe("The system", function() {
it("should fail", function() {
expect(true).toBe(false);
});
it("should succeed", function(done) {
setTimeout(function() {
expect(1).toBe(1);
done();
}, 10);
});
});
However, when I run these tests in the browser and click the Debug button to open the Karma DEBUG RUNNER, I see both tests failing, where the second test fails with the error message of the first test. The regular test run (i.e. not in the Karma DEBUG RUNNER) works as expected.
The error message for the second test is:
Uncaught Expected true to be false.
at UserContext.<anonymous> (http://localhost:9876/base/workingspec.js:4:22) thrown
When I disable or remove the first test, the second test passes.
Why do both tests fail in this case? Why does the second test fail with the error message of the first test?
My test setup contains the following packages/versions:
+-- jasmine-core#2.7.0
+-- karma#1.7.0
+-- karma-chrome-launcher#2.2.0
+-- karma-jasmine#1.1.0
`-- karma-jasmine-html-reporter#0.2.2
The problem is indeed in the Debug.js file of the Karma Debug Runner, as #user907860 already hinted at. It is not particular to Jasmine. I have reported the issue and created a fix which has just been merged into the master branch of Karma, so the next release should fix this issue.
At first I thought it was a bug, but after some research with the brilliant Chrome devtools, it seems that this is an expected behavior, at least by Jasmine. It may be, though, a bug in the Karma framework.
In short, the node_modules/karma/static/debug.js file (which is the js-file for the debug page) has these lines (I have Karma v1.7.0):
for (var i = 0; i < result.log.length; i++) {
// Throwing error without losing stack trace
(function (err) {
setTimeout(function () {
throw err
})
})(result.log[i])
}
If you comment the throw line, and restart the karma server, you'll see only console log messages, which should be expected: first FAIL, then PASS, and then the summary.
Verbosely, the bug in Karma may be in it's behavior to report after each spec.
That is what's happening here step by step (I have version "jasmine-core": "^2.6.4", at least this is in my package.json file):
Jasmine runs the first spec and it fails;
Karma reports on this in the log and adds a function, which throws an error to the stack (I assume the reader knows the asynchronous model in JavaScript, if not, then he must read about this in something like
"Effective JavaScript: 68 Specific Ways to Harness the Power of JavaScript" by David Herman, a true gemstone book, or elsewhere). Also, this is important, though I'm not sure, since I haven't look into the code so deep, it registers sort of "globalError", since on the next step
Jasmine runs the second spec, and calls the getJasmineRequireObj().GlobalErrors function (jasmine.js:2204). The "globalError" is detected and the spec instantly becomes a failure. The asynchronous expect is added to the stack after Karma's function, the one, which throws the error
Then the first function (which throws the error) starts execution, add it throws. This function will always be called before the Jasmine asynchronous spec, since in the debug.js there is no time passed to the setTimeout call:
//notice the absent time argument
setTimeout(function () {
throw err
})
Jasmine runs the assync expect from the second spec and it passes
Karma reports on the second spec as a failure and adds the error throwing (if not commented out, then no error occurs and this spec passes) to the stack
Karma's second error is thrown
Below are screen shots with numbers, put by me to illustrate the steps from the list:
with throw not commented in the debug.js file:
and with the commented throw:
I'm using wedriverio 4.5:
./node_modules/.bin/wdio -v
v4.5.2
I need to wait until some element exists and if it doesn't exist handle this situation.
for example:
let element = browser.element('.unexisting');
browser.waitUntil(
function () {
return element.isExisting();
},
1000,
'Element is not found.'
);
But if element doesn't exist on the page, webdriver marks my test as failed and shows message: 'Timeout of 10000ms exceeded. Try to reduce the run time or increase your timeout for test specs (http://webdriver.io/guide/testrunner/timeouts.html); if returning a Promise, ensure it resolves.'
How can I handle this situation?
I tried try-catch block, but anyway I see same message about timeout and failed test.
I tried element.waitForExist() but behavior is the same
I tried to use error handler (but it doesn't help)
browser.on('error', function(e) {
console.log ('handle browser error');
})
Why don't I see my message 'Element is not found.'?
Thanks!
Make sure your waitForXXX command doesn't take longer than your spec timeout. If it does increase your spec timeout, in your case mochaOpts.timeout. See more here http://webdriver.io/guide/testrunner/timeouts.html#Framework-related-timeouts
I am new to using Jasmine and BDD testing and I am trying to test if a simple page exists.
I have the following code:
describe("when server is running",function () {
it("should serve an index page", function(){
expect(server.app).toBeDefined();
request.get('http://localhost:3000/', function(error, response, body){
expect(response.statusCode).toBeEqual(200);
done();
});
expect(server.app).toBeDefined();
});
});
When the tests are run I get:
Finished in 0.027 seconds
1 test, 2 assertions, 0 failures, 0 skipped
There are obviously 3 assertions and it's the middle one, within the function() call that is not performed (I added the third repetitive one just for checking and commented out each one individually).
I came across the guide at https://semaphoreci.com/community/tutorials/getting-started-with-node-js-and-jasmine and made not of the addition of the done() callback but still no joy.
Is this something wrong with the test, the request or the jasmine-node setup I have.
I'm running this on Ubuntu 14.04.
Thanks.
Is there a way to get QUnit.js to not run the remaining tests after a single one fails?
Using the following code as an example:
QUnit.test('test1', function(assert) {
assert.equal(1,1);
assert.equal(1,2);
assert.equal(3,3);
});
QUnit.test('test2', function(assert) {
assert.equal(4,4);
assert.equal(5,5);
assert.equal(6,6);
});
Is there some way to get QUnit to stop executing after the assert.equal(1,2)? This means that test2 should never be run.
The best way to stop QUnit after test case fail will be
QUnit.testDone( function( details ) {
if (details.failed>0){
QUnit.config.queue.length = 0;
}
});
Okay, based on my comments above I ran the code below and things to stop as I think you want them to. Again, as I said in the comments, I would really investigate whether this is a good idea. Generally you want your tests to be idempotent such that any one failure does not affect any other test.
Note that we have to set the reorder config option to false here, otherwise QUnit will attempt to run the previously failed test first to "short circuit" things, but you don't want that I'm guessing. I also added a "test0" just to see the fill effect.
QUnit.config.reorder = false;
// This is how we detect the failure and cancel the rest of the tests...
QUnit.testDone(function(details) {
console.log(details);
if (details.name === 'test1' && details.failed) {
throw new Error('Cannot proceed because of failure in test1!');
}
});
QUnit.test('test0', function(assert) {
assert.equal(1,1);
assert.equal(2,2);
assert.equal(3,3);
});
QUnit.test('test1', function(assert) {
assert.equal(1,1);
assert.equal(1,2);
assert.equal(3,3);
});
QUnit.test('test2', function(assert) {
assert.equal(4,4);
assert.equal(5,5);
assert.equal(6,6);
});
You won't get any visual feedback that the tests were canceled because this isn't really interacting with the QUnit UI. However, because we threw an Error object you can open the developer console and see the output there:
Example code:
var connection = null;
function onConnect(status) {
im_a_big_error.log('wtf');
// Why it doesn't throw me an error here ??
}
$().ready(function() {
connection = new Strophe.Connection('http://localhost:8080/http-bind');
connection.connect('admin#localhost', 'admin', onConnect);
});
It doesn't throw me an error in my Chrome console.
Do you have an idea to resolve this issue?
Yes, Strophe often catch errors by itself and currently doesn't provide any ability to get connection error information. While error catching is ok, the impossibility of catching errors by yourself is not very good. But you can fix it with the following code:
$().ready(function() {
connection = new Strophe.Connection('http://localhost:8080/http-bind');
connection._hitError = function (reqStatus) {
this.errors++;
Strophe.warn("request errored, status: " + reqStatus + ",
number of errors: " + this.errors);
if (this.errors > 4) this._onDisconnectTimeout();
myErrorHandler(reqStatus, this.errors);
};
connection.connect('admin#localhost', 'admin', onConnect);
});
where myErrorHandler is your custom connection error handler.
Yes, strophe swallows errors. Worse; After an error is thrown, the callback won't return true as it should, and strophe will remove the handler. As soon as an error occurs, the callback will never be called again.
I found the code from the current answer a bit hard to use. Internally, we use the following wrapper for every callback;
function callback(cb) {
// Callback wrapper with
// (1) proper error reporting (Strophe swallows errors)
// (2) always returns true to keep the handler installed
return function() {
try {
cb.apply(this, arguments);
} catch (e){
console.log('ERROR: ' + (e.stack ? e.stack : e));
}
// Return true to keep calling the callback.
return true;
};
}
This wrapper would be used as following in the code of the question;
connection.connect('admin#localhost', 'admin', callback(onConnect));
I've been playing with Strophe for a while now and I had to modify its default error handling routine to fit our needs
Strophe.js - log function - by default contains nothing - I added calls to my server side logging service for level === ERROR and level === FATAL
Strophe.js - run function - the default behavior for error is to remove the handler and to rethrow the error - since I already log the error server side I don't rethrow the error and decided to keep the handler (even if it failed). This behavior could make sense (or not) depending on your own implementation - since I use custom messages and have a rather complicated message processing routine I don't want the client to stop just because a message was not properly formatted when sent so I want to keep the handler, error or not. I replace the throw e line inside the run function with result = true;
Strope.js _hitError - as I mentioned, I don't want the client to ever disconnect so I rewrote the default behavior to never disconnect (no matter how high the error counter)
Hope these thoughts are of help to others - leave a comment if you have questions/want details.
I had a similar problem which I fixed using the approach given by tsds above. However with minimal modification. I created two connect methods one as connect and the other as connect_bak I placed the script
this.connection._hitError=function (reqStatus) {
client.connect_bak();
};
in my connectHandler function as well as the connect function. Such that the function is always binded on connect.