I started to learn ReactJS yesterday (to be used in my next product), I am willing to set up my dev environment but now I'm stuck with Jest...
Having a bluetooth lightbulb on my desk (already op with scripts etc..), I want to get a red light when my tests launched with jest --watch fail (see create-react-app from FB devs here)
The problem is, I don't know how to run a callback after the tests, it seems like no one ran into this issue on the interwebz, no solution found yet for me.
Update:
I am currently using a log file to grep:
lamp.rb
def ci
if File.readlines("path/jest.log").grep(/failed/).any?
File.truncate('path/jest.log', 0)
fail_jest # This method updates my lightbulb :) (red blink)
end
rescue
puts 'No jest log found :('
end
Launching my jest tests like this: unbuffer npm run test |& tee tmp/jest.log
I am still looking for a better solution !
Thanks for your help
Your problem is not specific to React or Jest. When your run jest tests, you are basically running a Node/npm command and when the tests fail the process exists with an unsuccessful exit code. This is the same mechanism used to make automated CI builds fail when tests don't pass. So I'd suggest you start your research from there and depending on your lightbulb's API, it should be straight forward to make it fire events whenever the process fails regardless of the reason.
Related
I have a grouping of Cypress tests that are failing, only in my Jenkins environment. An entire describe block is failing to find an element. Every test in the block starts with the same commands:
describe("This section of tests", () => {
it("Test for something in this section", () => {
cy.login(); // custom login command, works in all other test blocks
setupPage(); // page setup function, no problems in all other test blocks
cy.wait(10000); // desperation wait
cy.get("#toggle-detail-pane-button").click(); // issue here!
// all tests in block run more assertions and commands after this
});
// more similar tests in block
});
// more describe blocks that also use cy.login and setupPage, with no issue
When I run these tests in the cypress test UI, they all pass. When I run them in the terminal on my machine with npx cypress run, they pass. When I ssh into a remote OpenStack terminal which is set up to run my project and run cypress, and run npx cypress run, they pass.
When I run the tests in Jenkins, which uses a handful of identical OpenStack instances to run cypress in parallel, the tests fail, consistently, with the following message:
AssertionError: Timed out retrying after 4000ms: Expected to find element: `#toggle-detail-pane-button`, but never found it.
at Context.eval (http://localhost:3000/__cypress/tests?p=cypress/integration/stuff/mytests.spec.js:519:8)
I have tried reorganizing my tests, rewriting them, etc, but no luck. I cannot figure out what is going wrong here. My Jenkins environment uses a custom sorry-cypress setup, and for some reason, the dashboard is not registering the tests, so I can't go and view any screenshots or videos (that is a whole separate can-of-worms problem).
Why would only these tests fail only in my CI/CD env? How can I even begin to debug this?
Edit: screenshots
I was able to rerun the CI/CD with screenshots and videos, and then I was able to scp the files off of the instance with the failing tests. When running on my machine, cypress has no problem finding the element:
When using the cypress selector playground on my machine, it finds it as well:
But in the screenshot I pulled off of the openstack instance running exactly the same test, it can't find it:
(Sorry for all the green, this is a proprietary app)
What gives?
Please update the default command timeout in cypress.config.js file using below line:
defaultCommandTimeout: 10000
Instead of 10000 you can use any value of time in miliseconds. Using this your, timeout issue will be resolved.
That means there's an issue with your cy.visit() command. The page is not being loaded before the test that's why
I had a similar problem. I solved it by getting the element by a selector or class instead of its id.
I went from:
cy.get('#mat-options-text').click()
to
cy.get('mat-options').click()
Intro
We struggle to setup cypress in the CI runners of gitlab.com. We use the default blueprint from vue-cli to scaffold the project. We tried various of different gitlab.yml configurations. Currently we run out of CI minutes because we tried so many different combinations.
We tried different docker images (from here: https://github.com/cypress-io/cypress-docker-images/) and also followed the best practices from Crypress which we found here: https://gitlab.com/cypress-io/cypress-example-docker-gitlab/
We just had no luck getting it running. After spending hours of hours we are not sure if it's even possible to get Cypress running with the default setup from vue-cli.
We also created an issue on vue repo but it got closed, for reference you can see here: https://github.com/vuejs/vue/issues/10944
We filled out the default vue template for an issue report but since it's not a real "JavaScript" issue it was hard to properly fill it out. But we tried to provide as much information as possible. In the codepen you find our results. The HTML column is the output and the JS column is the YML file. I hope you can use this information somehow
Version
2.6.10
Reproduction link
https://codepen.io/usamahamed/pen/WNbpdPE
Steps to reproduce
this this the gitlab CI pipeline result including in codepen
it give this
CypressError: cy.visit() failed trying to load:
We failed looking for this file at the path:
/builds/room/web/room-ui/
Checking your .yaml file, I think your application is not running.
There is not log of application running
There is no build stage implementation
There is no start application task on the stage test-e2e > before_script
So I would like to suggest:
Check your build stage. Where you make your application start running?
Check your before_script, adding a step "npm ci" like this:
https://gitlab.com/cypress-io/cypress-example-docker-gitlab/blob/master/.gitlab-ci.yml
or
https://github.com/cypress-io/cypress-example-kitchensink/blob/master/.gitlab-ci.yml
You should also wait for your application to be running to start testing. You can use the wait-on module for this: https://github.com/jeffbski/wait-on
From what I know console.log() should work without problem printing to the STDOUT of my console, when running a script.
But in my case I have NPM configured to run Jest when issuing npm test from the shell, and any console.log() inside the test files doesn't print anything on the screen.
I tried also to use process.stdout.write() but still I get no custom output when running npm test.
How am I supposed to debug stuff in my test scripts?
I can't figure out if it is a problem from Node, from NPM, or from Jest.
There's a Jest issue that looks similar to mine but still I cannot solve and get to output a simple string; while there rest of Jest output is echoed as usual.
Anybody experienced a similar problem?
EDIT 1:
I tried running npm test -- --runInBand but the log doesn't appear.
Note
trying to run the command repeatedly like
console.log('foo...');console.log('bar..');console.log('baz..');
I can sometimes see the log, but this is then overwritten/hidden by the rest of the following Jest output.
From the linked issue since the problem is going on since years...
There is a pull request in the alpha release that is going to fix this problem. Looks like the last refining is being worked in the past few hours (see Github issue link).
However I found a solution that works for my simplest case:
just run enabling verbose mode and the log will magically appear!
npm test -- --verbose=true
This will not fix in every case; maybe async/multi-thread/etc will still have problems (please report your experiences, so I can update the answer); fo r example adding a process.exit(1) after the log will hide it again.
Trying to press ctrl + c before the process.exit() runs (with the right timing...) will show that the log is actually there and being overridden.
I will update the answer with news, but probably this will help others in starting with Node/NPM/Jest setups!
Jest's GitHub issue mention some useful details/suggestions:
https://github.com/facebook/jest/issues/2441#issuecomment-713433790
https://github.com/facebook/jest/issues/2441#issuecomment-724131782
https://github.com/facebook/jest/issues/2441#issuecomment-586359238
https://github.com/facebook/jest/issues/2441#issuecomment-611122871
https://github.com/facebook/jest/issues/2441#issuecomment-623856169
aaaand.... https://github.com/facebook/jest/issues/2441#issuecomment-649391333
Jest replaces the global console module; this apparently is the source of the issue. And also the fact that Jest will set it to true automatically if you're running a single test. upside_down_face
This isn't clean, but I've had luck with it, hopefully it'll help you until a fix is published: log some extra newlines after the log that's getting overwritten:
console.log('I want to be seen!');
console.log('\n\n\n\n');
I had this issue and it turns out that in my case, the problem was calling process.exit(1) immediately after writing to the console. When you do this during Jest tests, the output seems to be lost.
The (not-entirely-satisfying) workaround that I settled on is:
console.warn('stuff I want to write to the console');
if (process.env.NODE_ENV === 'test') {
setTimeout(() => process.exit(1), 1000);
} else {
process.exit(1);
}
For this, the NODE_ENV environment variable would need to be set to test in your Jest setup script or elsewhere.
Recent versions of IntelliJ IDEA support the execution of Jest tests.
I couldn't find an option (or even better a shortcut) to update snapshot tests within IntelliJ IDEA.
Is there an option/shortcut to update snapshots within IntelliJ IDEA?
What I have been doing is to right click on the failing Jest test and select the Create option in the pop-up menu to create a new run configuration for just that failing test.
I then add -u to the Jest options and run that specific test (once) to update the snapshot.
It is far from ideal, but you can keep them around for later, if you like, to re-run them with the -u option when needed.
I was wondering the same thing and I asked Jetbrains. They said this feature was requested and you can track the status of it here: https://youtrack.jetbrains.com/issue/WEB-26008
I'm not sure when it will be complete but looks like it is on their radar.
The -u option can be applied as a Jest run config default if you want it to be active for all Jest tests.
I am looking for a way to make karma.js crash when my tests fail so that my build process gets interrupted since that is easier to monitor in a remote building server.
I am using mocha as a reporter and jasmine as the test processor engine.
Is there any option or particular variable in the karma.conf.js file maybe that would allow me to "crash on test fail"
There is no need for a "crash on failure".
Your build process should check the exit code of the Karma process that it launches. If all test pass, the code will be 0. If there is a test failure, the code will be non-zero. (I've just checked and it was 1 when I tried.)
You may be able to get a plugin for karma for your build server that will report the results of your tests.
For example, there's a plugin for karma/teamcity (karma-teamcity-reporter) which allows you to fail the build step, preventing the application from publishing if you have it set up that way.
you can use grunt as automation tool to run your karma test case so that whenever any grunt job(test case) failed it will by default stop execution.