I would like to run only specific Cypress tests related to the application files that I changed, but every time that I make a change all the integration tests are running. I've already checked a tool called cypress grep but it uses tags to map the tests but it is not dynamic or automatic since I need to set the tags and also add the tags manually in the pipeline.
Do you know an alternative tool to solve this issue?
We are using nx with Angular, works quite well https://nx.dev/using-nx/affected. Not sure if it's suitable for your project and if it's worth setting up whole nx.
You'll want to use the find-cypress-specs where you can get a list of changed specs, number, and by tags.
You'll have to form the Cypress run command in your YML file.
I have a weird situation going on with one of the react applications at my company. We have a suite of E2E tests written in testcafe that check our pull requests before we can merge. One portion of the application has about 15 tests associated with it.
When I pull down the changes to my latest branch and run the tests with Testcafe locally via yarn, while my application is pointed at the same exact backend services that we use for all testing, all 15 tests pass.
When the tests are run via github actions, using a docker image of our application as well as testcafe's docker container, 1/15 of the tests fails. The one that fails claims that it can't find certain elements within the dom.
I have spent days debugging this and I'm completely out of ideas. I've tried attempting to select different elements, using selectors from testing library instead of testcafe, implementing various waiting mechanisms to try to give the page additional time to load, none of it seems to matter just for this one single test.
I can't for the life of me figure out why the tests would pass when run locally, and the actual new feature works when tested manually, but only 1/15 tests would consistently fail when run via the docker process.
Does anyone have any ideas why this discrepancy could exist?
I have a website that is running on some server, which is used by a few thousand people. It's built on Java and soy template. I need to test the fronted rendering/js files. Can I do this from my local machine after accessing the web page?
for example: say I need to run a few Javascript test files on facebook.com. I go to facebook.com in my browser, after it is loaded, I need to run a few js files to test it. Is it possible? if yes, should I use Mocha or jest or any other alternate framework?
Only you can do is E2E testing. E2E testing is an UI automation testing. Here are the e2e testing framework choices:
puppeteer
cypress
casperjs
protractor
You can use jestjs with puppeteer.
There are different test types that can be run against code: Unit, Integration, Functional, etc. Some types have different test methodologies: TDD, BDD, etc. In this instance it seems you are wanting Functional Tests usually using the BDD method. In regard to your post, you state Java but tag the post Javascript and mention the Mocha framework. I'm assuming you meant Javascript instead of Java, with that said Mocha does support Functional Tests:
https://mochajs.org/#retry-tests
You will need the Selenium Driver to be installed along with other specific web browser drivers: Chrome, Edge, Gecko. A quick search shows that there might be an easier path to get up and running:
https://www.npmjs.com/package/mocha-webdriver
I have no experience with that library but leave further investigation up to you.
I intend to write tests using Mocha that can be executed using Karma. When I develop code, the application starts on http://localhost:3000 and I want to run tests (through command line - via GRUNT) that validate the status of the DOM.
How do I configure karma to do that?
How do I provide the HTML for that view?
Also, there are multiple pages and each page has a different set of tests. How to architect the solution so that all the tests are executed precisely?
I think all you need is Protractor configured with Karma. Here is example (with Jasmine) - https://github.com/angular/angular-seed/blob/master/test/protractor-conf.js.
It can run browser, simulate any action like real "end" user, and check DOM elements.
If Protractor is replacing Angular Scenario Runner for E2E testing, does that mean I will still be able to use it with Karma as my E2E testing framework ?
Not recommended by the current maintainer of Protractor:
https://github.com/angular/protractor/issues/9#issuecomment-19927049
Protractor and Karma should not be used together; instead they provide separate systems for running tests. Protractor and Karma cover different aspects of testing - Karma is intended mostly for unit tests, while Protractor should be used for end to end testing.
Protractor is built on top of WebDriverJS, which uses a Selenium/WebDriver server to provision browsers and drive test execution. Examples of pure WebDriverJS can be found here: http://code.google.com/p/selenium/wiki/WebDriverJs
And
https://github.com/angular/protractor/issues/9#issuecomment-19931154
Georgios - I think it makes sense to keep Protractor and Karma separate - for end to end tests, you want the native event driving and flexibility of webdriver, while for unit tests you want fast execution and autowatching of files.
UPDATE. Here is a simple package I've created to add minimal Karma setup to any project with one single command npm install min-karma.
I'd like to clarify some possible misconceptions about Karma and Protractor. Karma FAQ actually does refer to Adapter for Angular's Scenario Runner, which, however, seems to be abandoned, with Protractor being recommended instead.
Karma
Karma is a test runner that will run the JavaScript files specified in you configuration file explicitly or using node-globs. (For non-JavaScript external templates, Angular's Unit Testing Guide recommends using Karma html preprocessor to compile them into JavaScript first.)
These can be all your source files, some of them, some of them plus some additional files or files irrelevant to your project, only providing some extra configuration - you name it!
You can have multiple karma config files for different purposes, which you can run in parallel or one-by-one. Each karma process launches its own set of browsers (these are currently available).
This feature of Karma to run only a set of files is what makes it perfect for fast tests running in background upon each source file edit, and get immediate feedback, which is brilliant! The only negative is the "noisy" error reporting that will hopefully improve!
Karma is not only for unit tests
Unit test is for a single unit of your source code. In Angular's case a typical unit is Angular Component (Service, Factory, Provider, Controller, Filter, Directive etc). Remember to keep your Controllers thin, so too many unit tests for latters is a red flag.
In a unit test, every other units of code, on which this unit depends (so-called unit's dependencies) should not be tested at the same time. Instead they should be "mocked", e.g. replaced by something simple like dummy instances. Angular provides great mock environment support. Ideally you want to see all those mocks directly inside your tests, so you never need to wonder where all those dependencies come from.
Karma is just as useful for Integration Tests, where a group of source code units is tested together, with only some of their dependencies being mocked. It is important to remember that any dependency is by default provided from your source code modules (as long as those modules either injected directly in your tests, or they are dependencies of other modules injected (in which case you don't need to inject them, but no harm to do so). The mocked dependencies will override the provided ones.
Running Fast and Frequent is the main feature of Karma. This means you want to avoid any server requests, any database queries, anything that can take longer than fractions of seconds. (Otherwise it will NOT be fast!) Those long processes are ones you want to mock. This also explains why it is a bad practice to put raw low level services like $http directly inside your controllers or any complicated business logic units. By wrapping those low level outside communication services into smaller dedicated services, you make it much easier to "mock them away".
What Karma does not do is running your site as it is, which is what End-to-End (E2E) testing is. In principle, you could use Angular's internal methods to recreate the site or its pieces. Which, for small pieces, can be useful, and a fast way e.g. to test directives.
It is, however, not recommended way to throw complicated code inside your tests. The more you do it, the more chance is that you make errors in that code instead of what you are actually testing.
That is why I personally dislike the often mentioned complicated way of testing methods using low level methods like $http. It works cleaner to isolate any reference to low level methods into dedicated methods of your own, whose single responsibility is to make http requests. These dedicated methods should be able to work with real backend, not a fake one! Which you can easily test - manually or even perfectly fine with Karma running with another special config, as long as you don't mix that config with the one usually used to run Karma regular and fast.
Now, having your dedicated small services tested, you can safely and easily mock them to test your other logic and put these tests into your regular Karma setup.
To summarize. Use Karma to run any set of JavaScript files. It is (should be) fast. You don't see your complete app, so can't test the final result effectively and reliably. Would I run it with Protractor? Why would I? Running Protractor would slow down my tests, defeating the purpose of Karma. It is easy to run Protractor separately.
Protractor
Protractor is:
an end-to-end test framework for AngularJS applications. Protractor runs tests against your application running in a real browser, interacting with it as a user would.
So Protractor does exactly what Karma doesn't - run your real final application. This reveals its both power and limitations:
Running complete application is the only reliable final test that your application works as expected. You can write up complete user story scenarios and put them into your tests!
But it is harder to track errors without isolating individual units of your source code. This is why you still need Karma to test your JavaScript code first.
Now would I want to run Protractor with Karma? I surely can run them in separate terminal windows, in parallel. I could, in principle, have them share test files if I need to, but normally I'd rather not. Why? Because I want to keep my tests small with single dedicated purpose.
The only exception would be a file defining testing macros useful for both runners. This, however, would not be a test file but a macro definition file.
Other than that, I like a clear separation between my tests. Those to be run frequently and fast, and those for the complete app. That makes a clear separation between when using Karma and when Protractor.
Karma is a test runner provided by the Angular team, Karma will execute your tests in multiple browsers which shall ensure that our application is compatible in all browsers.
Unit Test for angular js can be used karma + jasmine
Jasmine is a javascript unit testing framework and will provide us with utilities to test our application. This works best on the Angular framework and thus, our choice of “automated unit testing tool”.
https://github.com/shahing/testingangularjs
And Protractor is an end-to-end test framework for Angular and AngularJS applications.
Protractor runs tests against your application running in a real browser, headless browsers , cross browser testing and can be hosted on saucelabs.
https://github.com/shahing/Protractor-Web-Automation
Yes, you can use karma and protractor together. Karma is used for unit testing the component you created using angular command you can test those components using karma.
Protractor is used for end to end test. It is mainly used for UI testing.