I'm developing an Webapp.
It consists of 2 parts. A node rest server and an angularjs client.
The app is structured this way: Rest Server <--> Api Module <--> Angular App
The server is currently well tested.
I have Unit Tests and Integration Tests.
The Integration Tests are accessing a real database and calling the rest api over http.
I think this is as high level as it can get for the server testing.
The integration tests run fast, too.
I'm pretty confident that the way I tested the server is sufficient for my use case and I'm happy with the results.
However I'm struggling how to test the angularjs app.
I have unit tests for the relevant directives and modules. Writing these wasn't an issue.
I would like to write integration tests that cover user scenarios.
Something like a signup scenario: The user visits the website, goes to the signup form, and submits the form with the data.
The angularjs team is moving from ng-scenarios to protractor.
Protractor is using Selenium to run the tests.
Therefore there are two scopes: The app scope and the test scope.
Now I can think of three different abstractions I could use.
And I'm not sure which one suites me best.
Mock the Api Module
Mock the Rest Server
Use the full server
Mock the Api Module
In this case I would need not to setup a server. All Interactions are running in the browser
Advantage:
No server is needed
Disadvantage:
The api is in the browser scope and I have to tamper with this.
I really like this solution, but I find it difficult to mock the Api.
The Api needs to be modified in the browsers scope.
Therefore I need to send the modification from the test to the browser.
This can be done, however I don't see how I could run assertions like mockedApi.method.wasCalledOnce() in the tests scope
Mock the Rest Server
Advantage:
Client would be unchanged
Only one scope to deal with
Disadvantage:
One has to setup the Rest Routes
I could create a complete Mock Rest Server in nodejs.
Protractor Tests are written in nodejs, thus the control of the server can be done in the test.
Before I run the test I can tell the server how to respond.
Something like this: server.onRequest({method: 'GET', url: '/'}).respondWith('hello world')
Then I could do assertions like wasCalledOnce
Use the full Server with Database
Each test is running with a complete server and can add elements to the database.
After each test one can look at the expected elements in the database
Advantage:
Can be pretty sure, that if these tests are running the app is functional in the tested use case
Disadvantage:
I already made a fairly intense integration test with the rest server. This feels like doing the same again.
Setup depends on the full server
Current Conclusion
Mocking the Api would separate the server and the client completely.
Using a Mock Api would be a higher level test, but would require a fake server
Doing a full integration test would give the best reliability, but this is also highly dependant on the server code
What should I pick? What would you do?
I think I answered this same question in the Protractor google group. I am much of the same mind as you about wanting no server but wanting all of my test code in one place (in Protractor) and not split between Protractor and the browser. To enable this, I took matters into my own hand and developed a proxy for the $httpBackend service which runs within Protractor. It allows one to configure the $httpBackend service as if it were running in Protractor. I have been working on it for a while now and its reasonably full featured at this point. It would be great if you could take a look and let me know if I am missing anything important.
https://github.com/kbaltrinic/http-backend-proxy
It is an excellent question, which has nothing to do with a particular tool. I had to face the same problem on a big "greenfield" (ie started from scratch) project.
There is a problem of vocabulary here : the word "mock" is used everywhere, and what you called "integration test" are more "full end-to-end automated functional testing". No offence here, it's just that a clear wording will help to solve the problem.
You actually suggested the correct answer yourself : #2 stub the rest server. #1 is feseable but will be soon too hard to develop and maintain, #3 is an excellent idea but has nothing to do with UI testing and UI validation.
To achieve a high reliability of your front-end, independently of your backend, just stub the rest server, i.e. develop a stupid simple REST server that will idempotent, i. e. will ALWAYS answer the same thing to one http request. Keeping the idempotence principle will make development and test, very, very easier than any other option.
Then for one test, you only check what is displayed on the screen (test the top) and what is send to the server (test the bottom), so that the full UI stack is tested only once.
The full answer to the question should deserve an entire blog article, but I hope you can feel what to do from what I suggest.
Best regards
Here is an approach for writing integration tests for your Angular code. The key concept is to structure your code in a way that lets you invoke the various functions in a way very similar to how it's consumed by the UI. Properly decoupling your code is important to be successful at this though:
More here: http://www.syntaxsuccess.com/viewarticle/angular-integration-tests
This is a great question. This is how I would do it:
As you already have the angular unit tests for the relevant directives and modules this is perfect.
The other thing that is perfect is that your server Integration Tests are accessing a real database and are also making sure the rest api over http works.
So why not just add some high level integration tests that include angular and your server at the same time.
If you can avoid mocking, why not save the work to maintain the extra code, if possible.
Also a good read: http://blog.ericbmerritt.com/2014/03/25/mocking-is-evil.html
Mocking the REST server is the best, cleaner option in my opinion. Try Mountebank (http://www.mbtest.org). An amazing Virtualization Service tool.
Related
I set up a server that provides several endpoints to clients, and I have a client that uses node-rest-client package to post data to the backend.
I would like to write some unit-testing for the client side, making sure the data is properly sent to the endpoint.
I've been thinking of setting a dummy server for the testing, but that doesn't feel right.
How can i acheive such tests?
Please let me know if there's any missing data.
Thanks in advance!
EDIT
I've been looking at supertest package, but it looks like this is used to test the REST API service itself and not to test the REST calls from the client
Setting up a dummmy server is a legitimate approach as part of your "Acceptance Testing" strategy. Unlike Unit Tests, Acceptance Tests test a single component (made up of many Units) in isolation, stubbing out any external factors like databases or back end HTTP services. Having a dummy server can be a good way to go if you want to test how your client side as a whole interacts with the back end HTTP service. Acceptance Tests are generally more wide reaching, fewer in number, and slower than Unit Tests. That's why it's OK to start a dummy HTTP server.
However, if you really want to Unit Test the individual javascript unit that makes calls to the server, then this can actually be a bit tricky. You don't want to start a real HTTP server in Unit Tests because it will make them slow and unable to run in parallel.
The best approach depends on the technology you are using. For example, AngularJS provides a way to Unit Test HTTP calls called $httpBackend. Essentially it is a dummy HTTP server, but it doesn't actually go over HTTP (all requests stay in memory). This is possible because the Angular Team view testability as a priority and designed it this way from the ground up.
I don't think the node-rest-client provides a similar sort of mock HTTP server so it may prove difficult to Unit Test your client. The best approach is probably to wrap all usages of the node-rest-client in a standalone javascript object which itself has no Unit Tests, but is easy to mock out in unit tests for other parts of your client side code. This un-unit tested object can then be tested against a dummy HTTP server as part of a separate test suite, or you could even rely on your Integration Tests to catch any problems with this part.
As always though, it is best to keep your reliance on Integration Tests (which connect to an actual instance of the back end) to a minimum as they are slow and unreliable. These tests should really only be for verifying top level details such as performance, connection details, configuration compatibility etc.
If you take the wrapping approach, it is important to have as little logic in the wrapper as possible, because it will not be Unit Tested!
Note that wrapping all HTTP calls in a javascript service object is probably good practice regardless of your testing strategy, as it will confine all usages of the node-rest-client to one place in your code.
There is an web-application, that needs to be tested. This application uses AJAX and jQuery. Tests have to be written for all possible interactions with the browser and client-side. There are some tools for this, for example, Selenium IDE, but I wonder if it is possible to use any headless browser.
So, requirements for the testing system are:
Query pages from the remote server, simulate browser behavior (basically we give the headless browser the URL, browser fetches the page and launches tests on it);
Inject tested JavaScript or test JavaScript already loaded on the remote page;
Use any of testing frameworks than can be integrated with any of CI software (Jasmine, Mocha etc.).
It is possible to use mocking techniques when dealing with AJAX requests, for example, but I'm trying to test real-life application. Hope that this question will be useful for anybody.
As far as I investigated this topic, there is no means of doing this so far.
In my case I have a server PHP application, that talks with the outside world using REST interface. My JavaScript code talks to server and performs some interface manipulations depending on responses. So, my goal is to test the JavaScript code, but it hardly relies on the server side. So, I have to ways of testing JavaScript:
Using mocks, consider looking up this article. You basically simulate you server-side API. There is a problem with this method - whenever you change your API, you have to perform corresponding changes in your mocks, so the testing set will be actual.
Calling JavaScript testing utilities directly from PHPUnit tests (or whatever server-side testing is used) - there are no solution for this yet, unfortunately. But this method will save a lot of developers time (no need to rewrite mocks for 100-200 example queries), also we can guide the server's behavior on-the-fly.
Please, give a feedback on the second approach. If it is really needed, I guess it make sense to implement it.
I've recently started getting into Angular (with node.js), and many of the tutorials suggest using Protactor, which looks amazing. One thing has me confused though.
I'm used to tests where test data is built before the test, the test is run, and the data is destroyed.
With protractor it seems you start your server, and have your tests run against that server. In the tutorials I've seen, this server is usually the dev environment (populated by seed data I assume). In my experience, the dev database changes as you play around and tweak your app. Furthermore, a protractor test might delete an object, meaning that for the test to be re-run the object would have to be built again.
When using Protractor, what is the standard practice for creating a test environment with before/after hooks for populate. Bonus points if you can point me to some good resources that answer my question.
Depends an how PRO you want to go. Are you interested in testing in dev only? Do you have other environments? How often do you want to test? I test if different environments. One of them has no data because the database is created before running the tests. Other environments have a lot of data.
I gave a talk in the Angular meetup in NY a few months back:
https://github.com/andresdominguez/protractor-meetup
Take a look at slide 35 of the presentation (the link is in the readme file).
I call the rest api directly to generate data for my tests. You can also run a script before you run your tests to make sure that some objects are present.
I am developing an Angular app (Angular 1.1.5). I need some way of mocking REST API responses for e2e tests.
AngularJS (Green & Seshadri) goes into some detail on unit testing HTTP requests with $httpBackend, but has little to say about e2e tests.
The documentation suggests using the ngMockE2E module to mock $httpBackend (which is distinct from the normal unit testing $httpBackend mock.
I spent a few hours investigating the ngMockE2EAPI and, whilst I can see some benefits, there seem to be many downsides. My question is:
What are the advantages of ngMockE2E over a simple server that provides test responses? or, more generally, What is the best way to test an Angular app's interactions with a server?
I can think of a few disadvantages:
I need to change my HTML to bootstrap the app with a different Angular module (a new one that depends on ngMockE2E and my actual application module)
There are no HTTP requests for Firebug/Developer tools to inspect. This is a big one for me. I ended up writing request/response logging methods in my mock backend. I guess most mock back ends contain this kind of code.
There isn't an easy way to extract parameters from URLs. If I wanted to the get the id parameter from /resource/:id, I'd need to give a regex to match the URL (\/resource\/.+$), then split the matching URL string on slash and pick the last element.
The API is clunky. Returning a static object is simple, but dynamically determining which test object to return based on the requested URL is difficult. Some apps may be able to hard-code a complete set of test URLs, but I need to test a large number of resources so this is impractical for me. More complex code means there is more to go wrong.
I can also think of some advantages. When using ngMockE2E:
I don't need to tell my app where the test API is - $httpBackend will inspect all HTTP requests for matches.
I don't need to run a web server at all.
I don't need to restart a server to pick up changes to my test data.
There are fewer overall components in the tests. However, given that this is an integration test, this isn't a huge benefit.
Very interested in thoughts and experiences.
My conclusion, several weeks of development later: I found a simple dummy backend (Node/Express) more flexible and maintanable than the e2e testing mock backend. The most compelling reasons are proper URL pattern matching and real HTTP requests for browser debugging.
When developing an API (in my case an API that wraps around a websockets service), is there a good way of giving it's skeletal implementation to another developer so that he can work independent of me? If my API is still in the design phase for example, I'm thinking that I could give an outline of the methods that will be available and all the possible return types.
I realise that I could just give him a dummy object with all the methods inside but it would be great if that dummy object could be tied in with his unit tests and so on.
I've been looking at [SinonJs][1]: http://sinonjs.org/ and in particular at the Stub functionality it provides. However I am not sure if what I've outlined here is an appropriate use of stubs.
Any advice appreciated!
EDIT:
I guess this question should of been:
'How to write unit tests using Jasmine, Sinon and a mock API'
It was a bit of a silly question because it turns out that Sinon actually has it's own 'fake server' which is very easy to use. In ignorance of this my first idea was to mock or spy on XHR stuff.
The code I ended up with is here: https://gist.github.com/james-gardner/11405316
See 'DummyViewSpec.js' for examples of the fake server stuff. Edits welcome!
If you are asking about an easy way to have a web app (or other REST) API up and running - http://flatironjs.org/ might be a good option for you. It's full JS stack both on the server (node.js) and client. However, you can do an API mockups with any other routing library (search npm - there are A LOT of them).
I hope it helps.
There is a tool to mockup the whole APIs using schema and random data with types
https://github.com/homerquan/kakuen
I created and used it to inject dependance of api