Some background context:
I'm working on a game written in BABYLONJS, which renders 3D graphics inside an HTML5 canvas using webGL. That is to say this project is not a typical Web UI where I need to test DOM elements like button clicks or form submits. BABYLON JS has its own way of simulating pointer events in the context of a 3D scene and you can pass it a pointerInfo object that mocks if a mesh was hit etc., and that's what I would like to use.
My project came bootstrapped with esbuild. I love it because it's fast at transpiling typescript and bundles everything into a single artifact and doesn't produce javascript artifacts next to my typescript files, so my directories are clean.
I started testing using jest. This was fine until I started running into issues where
Window is not defined
would crop up. It was because the code being tested brought other code along that was testing window.navigator... for attributes to see if this was a mobile device or not. I could try to mock the Window object but it is a pain. Also when trying to simulate a click the PointerEvent was not defined. I tried tried adding JSDOM, but that didn't seem to help and I wasn't able to get unblocked. It just seemed like I was trying to use a tool built for node, when I should just test in a real browser.
But Googling "browser based testing" usually finds results I don't want. I'm not looking for full end-to-end user interaction testing. I don't want selenium/chrome driver style of testing because:
it's slow
my project is not a traditional website, I don't have many HTML elements for a user to interact with
I don't want to test the whole stack of logging in, dealing with authentication etc.
I just want to test classes and functions in small unit test level, but I need access to Window and PointerEvent and all the goodies that come with a browser for free.
Next I looked at Jasmine. Jasmine standalone has a browser based SpecRunner.html. It's a single HTML page that you modify. It includes its own jasmine boot scripts that are loaded with script tags, and then your source code files and your test files are also imported as JS files with script tags.
This seemed promising in that the tests run in a browser so presumably they have access to the window object. However, both my specs and my source code are written in typescript, not javascript, so how do I place my code and tests into the SpecRunner.html?
esbuild is only bundling a single output artifact. If instead of using esbuild, If I used the tsc command and pollute my directories full of javascript artifacts, then yes, the jasmine SpecRunner.html would have access to the javascript files, but tsc is slower and js files everywhere is messy.
But before I get too far on this,... I think the downside to this approach is that for every test file I write, I need to manually modify SpecRunner.html to include all the source code to be tested and all the test files which will be annoying to maintain whenever file paths or file names change.
TLDR;
Any advice on what is a good solution to run unit tests with a real browser (not selenium style) when using typescript and esbuild? I don't have a real preference for any particular test framework.
I have a website that is running on some server, which is used by a few thousand people. It's built on Java and soy template. I need to test the fronted rendering/js files. Can I do this from my local machine after accessing the web page?
for example: say I need to run a few Javascript test files on facebook.com. I go to facebook.com in my browser, after it is loaded, I need to run a few js files to test it. Is it possible? if yes, should I use Mocha or jest or any other alternate framework?
Only you can do is E2E testing. E2E testing is an UI automation testing. Here are the e2e testing framework choices:
puppeteer
cypress
casperjs
protractor
You can use jestjs with puppeteer.
There are different test types that can be run against code: Unit, Integration, Functional, etc. Some types have different test methodologies: TDD, BDD, etc. In this instance it seems you are wanting Functional Tests usually using the BDD method. In regard to your post, you state Java but tag the post Javascript and mention the Mocha framework. I'm assuming you meant Javascript instead of Java, with that said Mocha does support Functional Tests:
https://mochajs.org/#retry-tests
You will need the Selenium Driver to be installed along with other specific web browser drivers: Chrome, Edge, Gecko. A quick search shows that there might be an easier path to get up and running:
https://www.npmjs.com/package/mocha-webdriver
I have no experience with that library but leave further investigation up to you.
I have already experimented testing NodeJS libraries using Jasmine or Mocha, but I don't know how to test front-end projects. I have found tutorials online, but everything includes a task manager in the workflow and I would like to know how to do this without one.
I found the following question close to what I am asking:
Using Travis-CI for client-side JavaScript libraries?
In my case, I am using Jasmine and have already set up the Jasmine SpecRunner.html, Jasmine library and spec/mylibSpec.js. The tests pass when I run the SpecRunner.html on my browser.
Now, how do I integrate this with Travis, without Grunt/Gulp/Brunch/etc?
I have heard the words "PhantomJS" and "Selenium" and I think this has to do with what I am trying to accomplish. Is there a "hello, world"-like project with tests and Travis integration one can learn from?
The Travis documentation lists three basic ways to accomplish this:
the PhantomJS headless browser
running Firefox with a virtual display or
using the Saucelabs browser VM service
Testing with PhantomJS is the fastest, since it does not simulate a display (it still allows you to create screenshots, though). PhantomJS comes with a run-jasmine example.
The phantom test script can then be executed directly, simply by running
script: phantomjs run-jasmine.js
in your .travis.yml, without the additional overhead of a build system such as Grunt.
If testing your project requires a real browser GUI, that leaves you with options 2 or 3.
Saucelabs browser VMs have the advantage of real cross-browser testing; if your project is open source, they offer a free plan. They also provide an in-depth tutorial for your specific use case: Travis + Jasmine + Saucelabs, which however does require Grunt in order to run.
I intend to write tests using Mocha that can be executed using Karma. When I develop code, the application starts on http://localhost:3000 and I want to run tests (through command line - via GRUNT) that validate the status of the DOM.
How do I configure karma to do that?
How do I provide the HTML for that view?
Also, there are multiple pages and each page has a different set of tests. How to architect the solution so that all the tests are executed precisely?
I think all you need is Protractor configured with Karma. Here is example (with Jasmine) - https://github.com/angular/angular-seed/blob/master/test/protractor-conf.js.
It can run browser, simulate any action like real "end" user, and check DOM elements.
If Protractor is replacing Angular Scenario Runner for E2E testing, does that mean I will still be able to use it with Karma as my E2E testing framework ?
Not recommended by the current maintainer of Protractor:
https://github.com/angular/protractor/issues/9#issuecomment-19927049
Protractor and Karma should not be used together; instead they provide separate systems for running tests. Protractor and Karma cover different aspects of testing - Karma is intended mostly for unit tests, while Protractor should be used for end to end testing.
Protractor is built on top of WebDriverJS, which uses a Selenium/WebDriver server to provision browsers and drive test execution. Examples of pure WebDriverJS can be found here: http://code.google.com/p/selenium/wiki/WebDriverJs
And
https://github.com/angular/protractor/issues/9#issuecomment-19931154
Georgios - I think it makes sense to keep Protractor and Karma separate - for end to end tests, you want the native event driving and flexibility of webdriver, while for unit tests you want fast execution and autowatching of files.
UPDATE. Here is a simple package I've created to add minimal Karma setup to any project with one single command npm install min-karma.
I'd like to clarify some possible misconceptions about Karma and Protractor. Karma FAQ actually does refer to Adapter for Angular's Scenario Runner, which, however, seems to be abandoned, with Protractor being recommended instead.
Karma
Karma is a test runner that will run the JavaScript files specified in you configuration file explicitly or using node-globs. (For non-JavaScript external templates, Angular's Unit Testing Guide recommends using Karma html preprocessor to compile them into JavaScript first.)
These can be all your source files, some of them, some of them plus some additional files or files irrelevant to your project, only providing some extra configuration - you name it!
You can have multiple karma config files for different purposes, which you can run in parallel or one-by-one. Each karma process launches its own set of browsers (these are currently available).
This feature of Karma to run only a set of files is what makes it perfect for fast tests running in background upon each source file edit, and get immediate feedback, which is brilliant! The only negative is the "noisy" error reporting that will hopefully improve!
Karma is not only for unit tests
Unit test is for a single unit of your source code. In Angular's case a typical unit is Angular Component (Service, Factory, Provider, Controller, Filter, Directive etc). Remember to keep your Controllers thin, so too many unit tests for latters is a red flag.
In a unit test, every other units of code, on which this unit depends (so-called unit's dependencies) should not be tested at the same time. Instead they should be "mocked", e.g. replaced by something simple like dummy instances. Angular provides great mock environment support. Ideally you want to see all those mocks directly inside your tests, so you never need to wonder where all those dependencies come from.
Karma is just as useful for Integration Tests, where a group of source code units is tested together, with only some of their dependencies being mocked. It is important to remember that any dependency is by default provided from your source code modules (as long as those modules either injected directly in your tests, or they are dependencies of other modules injected (in which case you don't need to inject them, but no harm to do so). The mocked dependencies will override the provided ones.
Running Fast and Frequent is the main feature of Karma. This means you want to avoid any server requests, any database queries, anything that can take longer than fractions of seconds. (Otherwise it will NOT be fast!) Those long processes are ones you want to mock. This also explains why it is a bad practice to put raw low level services like $http directly inside your controllers or any complicated business logic units. By wrapping those low level outside communication services into smaller dedicated services, you make it much easier to "mock them away".
What Karma does not do is running your site as it is, which is what End-to-End (E2E) testing is. In principle, you could use Angular's internal methods to recreate the site or its pieces. Which, for small pieces, can be useful, and a fast way e.g. to test directives.
It is, however, not recommended way to throw complicated code inside your tests. The more you do it, the more chance is that you make errors in that code instead of what you are actually testing.
That is why I personally dislike the often mentioned complicated way of testing methods using low level methods like $http. It works cleaner to isolate any reference to low level methods into dedicated methods of your own, whose single responsibility is to make http requests. These dedicated methods should be able to work with real backend, not a fake one! Which you can easily test - manually or even perfectly fine with Karma running with another special config, as long as you don't mix that config with the one usually used to run Karma regular and fast.
Now, having your dedicated small services tested, you can safely and easily mock them to test your other logic and put these tests into your regular Karma setup.
To summarize. Use Karma to run any set of JavaScript files. It is (should be) fast. You don't see your complete app, so can't test the final result effectively and reliably. Would I run it with Protractor? Why would I? Running Protractor would slow down my tests, defeating the purpose of Karma. It is easy to run Protractor separately.
Protractor
Protractor is:
an end-to-end test framework for AngularJS applications. Protractor runs tests against your application running in a real browser, interacting with it as a user would.
So Protractor does exactly what Karma doesn't - run your real final application. This reveals its both power and limitations:
Running complete application is the only reliable final test that your application works as expected. You can write up complete user story scenarios and put them into your tests!
But it is harder to track errors without isolating individual units of your source code. This is why you still need Karma to test your JavaScript code first.
Now would I want to run Protractor with Karma? I surely can run them in separate terminal windows, in parallel. I could, in principle, have them share test files if I need to, but normally I'd rather not. Why? Because I want to keep my tests small with single dedicated purpose.
The only exception would be a file defining testing macros useful for both runners. This, however, would not be a test file but a macro definition file.
Other than that, I like a clear separation between my tests. Those to be run frequently and fast, and those for the complete app. That makes a clear separation between when using Karma and when Protractor.
Karma is a test runner provided by the Angular team, Karma will execute your tests in multiple browsers which shall ensure that our application is compatible in all browsers.
Unit Test for angular js can be used karma + jasmine
Jasmine is a javascript unit testing framework and will provide us with utilities to test our application. This works best on the Angular framework and thus, our choice of “automated unit testing tool”.
https://github.com/shahing/testingangularjs
And Protractor is an end-to-end test framework for Angular and AngularJS applications.
Protractor runs tests against your application running in a real browser, headless browsers , cross browser testing and can be hosted on saucelabs.
https://github.com/shahing/Protractor-Web-Automation
Yes, you can use karma and protractor together. Karma is used for unit testing the component you created using angular command you can test those components using karma.
Protractor is used for end to end test. It is mainly used for UI testing.