It is very difficult to design and create acceptance test cases on-the-fly for agile’s Acceptance Test Driven Development (ATDD), simultaneous to requirements drafting. QualiTest’s ATDD services design acceptance tests to act simultaneously as detailed requirements and test cases, thus producing a definitive set of acceptance criteria in the early development stages.
We have been very impressed with QualiTest and your valuable contribution to our project. As you know the project that we have undertaken is unusual and as such there is very little in the way of benchmarks, documentation or references, made worse by the complexity of replacing an existing ERP system. QualiTest picked up on this and produced documentation explaining the delta between the two systems, very quickly mastered the complexity and was able to form meaningful tests which provided a much greater insight into our new system.
Angus Gow, I Want One Of Those
The ATDD process typically starts with the team (customer, BAs, developers, testers, etc.) writing a number of unambiguous test cases, describing what will be received as inputs and produced as outputs. As the number of test cases grows, more conditions are covered and the design of the product is further refined. Acceptance test cases are typically stored in a table and can be managed in an online web portal or Wiki. This enables all information to be viewed and updated by all the project participants and easily monitored by the target customer. Once the team feels a sufficient set of acceptance tests has been defined, developers can start their work while automated testing scripts are executed in parallel. That is done by running through the test case table and feeding the test cases into the product using hooks (called “fixtures”) which have been embedded in the code by the developers or the testers. This provides the team with a definitive indication of progress and quality in real-time, while simultaneously reducing the effort of testing. It also creates an inherent regression testing library as part of the project without a significant added effort. Once all tests pass, an additional round of regression testing is performed to ensure that changes haven’t broken anything and the functions or features can be released. Iterations tend to be short and focused on a relatively small number of functions or features which allow for continued readjustment and reprioritization.
A noteworthy variant of the ATDD process is the Behavior-Driven Development (BDD) process. While the core of BDD is very similar to ATDD, there is a great difference in the approach to the subject material. In ATDD, the tests are written from a technical engineering point of view. On the other hand, the BDD tests are written in an almost natural language, and describe situations the customer would understand. The tests thus describe functionality in a more abstract fashion than in ATDD. This is beneficial because the tests are much more readable, both for the developers and the customer, and because the tests are a lot more stable, since the abstract functionality of a system changes more rarely than the technical implementation.
One of the tools most widely associated with ATDD is FitNesse. It is a holistic development tool, containing a wiki, a server and an automated testing tool designed specifically for acceptance testing. It currently supports java and .NET technology. Since it is built with acceptance testing in mind, the tool supports extensive descriptions of the tests. BDD, on the other hand, is more widely associated with Cucumber, a ruby-based tool that also supports Java, Flex, .NET, and web-based applications.
Cucumber has a custom-built language that it takes as input (Gherkin), a language which is fine-tuned for user scenarios. Cucumber’s main feature is its ability to effortlessly borrow steps of one test for another, and to easily run selected groups of tests. The way it works is in three layers. The top layer is the test cases written in Gherkin, which are from a customer’s of view and fairly static. The middle layer is the more detailed breakdown of the test steps, where the actual mechanics of the test is written out. The final layer consists of hooks built into the application itself that is written in the native language of the application, and is very technical. The amount of readability and stability versus technicality and detail increases as one goes further up in the layers, providing an easy way to keep the big picture in mind, and to have the customer involved without needing extensive coding skills.