I don't know if I'm asking too much, but I want a library to intercept/mock HTTP requests in a isomorphic/progressive way (i.e. works the same on node.js and browser) for unit/behavior tests. Is there such a thing?
I'm building a client for an API and it must work both on server and browser. Nock is great but only for Node (as it doesn't work with Browserify, I tried).
I could just mock the library I'll use for requests (such as superagent or rest). That, however, would lock me in to some library and would require a major refactor of the tests.
My wish is to avoid duplication of tests and avoid the most checks for environment as possible. And to be agnostic of implementation, hence my need to mock the requests.
I'm almost considering making one myself (or at least a glue between two libraries).
I looked at how superagent were doing their tests. They build a mock Express server to respond in the way they want and check for matches.
But how does that work on the browser?
It turns out they were using Zuul for that.
Zuul is a tool that creates an ad-hoc server to run tests in the browser. It supports qunit, mocha, tape and a few other test frameworks. It also accept a mock server to run during the tests. And it runs tests on Sauce Labs, which have a free plan for open source projects.
This is not the solution I expected but it is a solution.
You can look at my project to see the proper code (permalink to the tree of the last commit by the time of this answer).
Related
I set up a server that provides several endpoints to clients, and I have a client that uses node-rest-client package to post data to the backend.
I would like to write some unit-testing for the client side, making sure the data is properly sent to the endpoint.
I've been thinking of setting a dummy server for the testing, but that doesn't feel right.
How can i acheive such tests?
Please let me know if there's any missing data.
Thanks in advance!
EDIT
I've been looking at supertest package, but it looks like this is used to test the REST API service itself and not to test the REST calls from the client
Setting up a dummmy server is a legitimate approach as part of your "Acceptance Testing" strategy. Unlike Unit Tests, Acceptance Tests test a single component (made up of many Units) in isolation, stubbing out any external factors like databases or back end HTTP services. Having a dummy server can be a good way to go if you want to test how your client side as a whole interacts with the back end HTTP service. Acceptance Tests are generally more wide reaching, fewer in number, and slower than Unit Tests. That's why it's OK to start a dummy HTTP server.
However, if you really want to Unit Test the individual javascript unit that makes calls to the server, then this can actually be a bit tricky. You don't want to start a real HTTP server in Unit Tests because it will make them slow and unable to run in parallel.
The best approach depends on the technology you are using. For example, AngularJS provides a way to Unit Test HTTP calls called $httpBackend. Essentially it is a dummy HTTP server, but it doesn't actually go over HTTP (all requests stay in memory). This is possible because the Angular Team view testability as a priority and designed it this way from the ground up.
I don't think the node-rest-client provides a similar sort of mock HTTP server so it may prove difficult to Unit Test your client. The best approach is probably to wrap all usages of the node-rest-client in a standalone javascript object which itself has no Unit Tests, but is easy to mock out in unit tests for other parts of your client side code. This un-unit tested object can then be tested against a dummy HTTP server as part of a separate test suite, or you could even rely on your Integration Tests to catch any problems with this part.
As always though, it is best to keep your reliance on Integration Tests (which connect to an actual instance of the back end) to a minimum as they are slow and unreliable. These tests should really only be for verifying top level details such as performance, connection details, configuration compatibility etc.
If you take the wrapping approach, it is important to have as little logic in the wrapper as possible, because it will not be Unit Tested!
Note that wrapping all HTTP calls in a javascript service object is probably good practice regardless of your testing strategy, as it will confine all usages of the node-rest-client to one place in your code.
I would like to add integration tests to an (Backbone.js) application I am maintaining, and considering what strategy is preferable and upsides and downsides for each:
Running tests on the client side, with something like jasmine-jquery. Ideally I would like something like ember test helpers which allows writing simple synchronous looking code (while actually running async).
Running tests on the server side using selenium drivers, for example Nightwatch.js.
It is hard to tell which approach has more community and tooling around it, and which projects are more mature. Additionally, I am getting the feeling that running tests on the client side might allow better isolation of tests, while running on the server side, might create tests which run longer and heavier (also to maintain?) but allow to simulate more complex real-user scenarios
Any thoughts would be appreciated
If you care about browser compatibility, then you'll want to run end-to-end tests on different browsers on different platforms, possibly using a cloud-based cross-browser testing service such as SauceLabs, BrowserStack, TestingBot or CrossBrowserTesting.
Since you're using Backbone and are familiar with JavaScript, you might want to pick a JavaScript client for Selenium WebDriver. There are quite a few choices, with Intern leading the pack by far.
There is an web-application, that needs to be tested. This application uses AJAX and jQuery. Tests have to be written for all possible interactions with the browser and client-side. There are some tools for this, for example, Selenium IDE, but I wonder if it is possible to use any headless browser.
So, requirements for the testing system are:
Query pages from the remote server, simulate browser behavior (basically we give the headless browser the URL, browser fetches the page and launches tests on it);
Inject tested JavaScript or test JavaScript already loaded on the remote page;
Use any of testing frameworks than can be integrated with any of CI software (Jasmine, Mocha etc.).
It is possible to use mocking techniques when dealing with AJAX requests, for example, but I'm trying to test real-life application. Hope that this question will be useful for anybody.
As far as I investigated this topic, there is no means of doing this so far.
In my case I have a server PHP application, that talks with the outside world using REST interface. My JavaScript code talks to server and performs some interface manipulations depending on responses. So, my goal is to test the JavaScript code, but it hardly relies on the server side. So, I have to ways of testing JavaScript:
Using mocks, consider looking up this article. You basically simulate you server-side API. There is a problem with this method - whenever you change your API, you have to perform corresponding changes in your mocks, so the testing set will be actual.
Calling JavaScript testing utilities directly from PHPUnit tests (or whatever server-side testing is used) - there are no solution for this yet, unfortunately. But this method will save a lot of developers time (no need to rewrite mocks for 100-200 example queries), also we can guide the server's behavior on-the-fly.
Please, give a feedback on the second approach. If it is really needed, I guess it make sense to implement it.
I am developing an Angular app (Angular 1.1.5). I need some way of mocking REST API responses for e2e tests.
AngularJS (Green & Seshadri) goes into some detail on unit testing HTTP requests with $httpBackend, but has little to say about e2e tests.
The documentation suggests using the ngMockE2E module to mock $httpBackend (which is distinct from the normal unit testing $httpBackend mock.
I spent a few hours investigating the ngMockE2EAPI and, whilst I can see some benefits, there seem to be many downsides. My question is:
What are the advantages of ngMockE2E over a simple server that provides test responses? or, more generally, What is the best way to test an Angular app's interactions with a server?
I can think of a few disadvantages:
I need to change my HTML to bootstrap the app with a different Angular module (a new one that depends on ngMockE2E and my actual application module)
There are no HTTP requests for Firebug/Developer tools to inspect. This is a big one for me. I ended up writing request/response logging methods in my mock backend. I guess most mock back ends contain this kind of code.
There isn't an easy way to extract parameters from URLs. If I wanted to the get the id parameter from /resource/:id, I'd need to give a regex to match the URL (\/resource\/.+$), then split the matching URL string on slash and pick the last element.
The API is clunky. Returning a static object is simple, but dynamically determining which test object to return based on the requested URL is difficult. Some apps may be able to hard-code a complete set of test URLs, but I need to test a large number of resources so this is impractical for me. More complex code means there is more to go wrong.
I can also think of some advantages. When using ngMockE2E:
I don't need to tell my app where the test API is - $httpBackend will inspect all HTTP requests for matches.
I don't need to run a web server at all.
I don't need to restart a server to pick up changes to my test data.
There are fewer overall components in the tests. However, given that this is an integration test, this isn't a huge benefit.
Very interested in thoughts and experiences.
My conclusion, several weeks of development later: I found a simple dummy backend (Node/Express) more flexible and maintanable than the e2e testing mock backend. The most compelling reasons are proper URL pattern matching and real HTTP requests for browser debugging.
I'm developing an Webapp.
It consists of 2 parts. A node rest server and an angularjs client.
The app is structured this way: Rest Server <--> Api Module <--> Angular App
The server is currently well tested.
I have Unit Tests and Integration Tests.
The Integration Tests are accessing a real database and calling the rest api over http.
I think this is as high level as it can get for the server testing.
The integration tests run fast, too.
I'm pretty confident that the way I tested the server is sufficient for my use case and I'm happy with the results.
However I'm struggling how to test the angularjs app.
I have unit tests for the relevant directives and modules. Writing these wasn't an issue.
I would like to write integration tests that cover user scenarios.
Something like a signup scenario: The user visits the website, goes to the signup form, and submits the form with the data.
The angularjs team is moving from ng-scenarios to protractor.
Protractor is using Selenium to run the tests.
Therefore there are two scopes: The app scope and the test scope.
Now I can think of three different abstractions I could use.
And I'm not sure which one suites me best.
Mock the Api Module
Mock the Rest Server
Use the full server
Mock the Api Module
In this case I would need not to setup a server. All Interactions are running in the browser
Advantage:
No server is needed
Disadvantage:
The api is in the browser scope and I have to tamper with this.
I really like this solution, but I find it difficult to mock the Api.
The Api needs to be modified in the browsers scope.
Therefore I need to send the modification from the test to the browser.
This can be done, however I don't see how I could run assertions like mockedApi.method.wasCalledOnce() in the tests scope
Mock the Rest Server
Advantage:
Client would be unchanged
Only one scope to deal with
Disadvantage:
One has to setup the Rest Routes
I could create a complete Mock Rest Server in nodejs.
Protractor Tests are written in nodejs, thus the control of the server can be done in the test.
Before I run the test I can tell the server how to respond.
Something like this: server.onRequest({method: 'GET', url: '/'}).respondWith('hello world')
Then I could do assertions like wasCalledOnce
Use the full Server with Database
Each test is running with a complete server and can add elements to the database.
After each test one can look at the expected elements in the database
Advantage:
Can be pretty sure, that if these tests are running the app is functional in the tested use case
Disadvantage:
I already made a fairly intense integration test with the rest server. This feels like doing the same again.
Setup depends on the full server
Current Conclusion
Mocking the Api would separate the server and the client completely.
Using a Mock Api would be a higher level test, but would require a fake server
Doing a full integration test would give the best reliability, but this is also highly dependant on the server code
What should I pick? What would you do?
I think I answered this same question in the Protractor google group. I am much of the same mind as you about wanting no server but wanting all of my test code in one place (in Protractor) and not split between Protractor and the browser. To enable this, I took matters into my own hand and developed a proxy for the $httpBackend service which runs within Protractor. It allows one to configure the $httpBackend service as if it were running in Protractor. I have been working on it for a while now and its reasonably full featured at this point. It would be great if you could take a look and let me know if I am missing anything important.
https://github.com/kbaltrinic/http-backend-proxy
It is an excellent question, which has nothing to do with a particular tool. I had to face the same problem on a big "greenfield" (ie started from scratch) project.
There is a problem of vocabulary here : the word "mock" is used everywhere, and what you called "integration test" are more "full end-to-end automated functional testing". No offence here, it's just that a clear wording will help to solve the problem.
You actually suggested the correct answer yourself : #2 stub the rest server. #1 is feseable but will be soon too hard to develop and maintain, #3 is an excellent idea but has nothing to do with UI testing and UI validation.
To achieve a high reliability of your front-end, independently of your backend, just stub the rest server, i.e. develop a stupid simple REST server that will idempotent, i. e. will ALWAYS answer the same thing to one http request. Keeping the idempotence principle will make development and test, very, very easier than any other option.
Then for one test, you only check what is displayed on the screen (test the top) and what is send to the server (test the bottom), so that the full UI stack is tested only once.
The full answer to the question should deserve an entire blog article, but I hope you can feel what to do from what I suggest.
Best regards
Here is an approach for writing integration tests for your Angular code. The key concept is to structure your code in a way that lets you invoke the various functions in a way very similar to how it's consumed by the UI. Properly decoupling your code is important to be successful at this though:
More here: http://www.syntaxsuccess.com/viewarticle/angular-integration-tests
This is a great question. This is how I would do it:
As you already have the angular unit tests for the relevant directives and modules this is perfect.
The other thing that is perfect is that your server Integration Tests are accessing a real database and are also making sure the rest api over http works.
So why not just add some high level integration tests that include angular and your server at the same time.
If you can avoid mocking, why not save the work to maintain the extra code, if possible.
Also a good read: http://blog.ericbmerritt.com/2014/03/25/mocking-is-evil.html
Mocking the REST server is the best, cleaner option in my opinion. Try Mountebank (http://www.mbtest.org). An amazing Virtualization Service tool.