ThoughtWorks' ATLAS project uses some Extreme Programming practices (pair programming, refactoring and others); is coached by two of the best, Ward Cunningham and Martin Fowler; but does not follow the practices rigorously enough to qualify as an "XP project". We have, however, adopted a functional testing approach that may be of interest to the XP community. A business user writes a test scenario and runs it using a Test Harness that provides immediate, detailed feedback on the status of the system. The effects of the tool on our development and testing process have been:
The ATLAS project develops an accounts receivable system for an equipment leasing company. The system bills customers and processes their incoming payments. A large proportion of the application involves batch processing. Ward Cunningham, Martin Fowler and I created the Functional Test Harness to exercise those batch processes. A business user writes a test scenario as a set of tables in an Excel spreadsheet. The Test Harness reads the spreadsheet, runs the application and compares the computed results with the expected results. A simplified example of the Test Harness output is shown below:
Billing Test 1
In the example, Brookfield Zoo rents a bear cage for a year. The test scenario checks that when the billing process is run on February 1, the first two rental periods are billed. The test fails for three reasons:
The Test Harness (written in java) consists of two components:
The Test Framework is a reusable package, while Test Fixtures are application-specific. In other words, one extends the Test Framework by writing a Test Fixture.
We use JUnit for unit testing. Developers unit test new functionality before subjecting it to functional tests. When a functional test fails due to a bug in the code, a developer adds a unit test that elicits the failure and then fixes the code.
Users respond enthusiastically to the Functional Test Harness, and it meets XP's goal of business-driven, automated and repeatable functional testing. As Kent Beck says, until a functional test proves the existence of a program feature, the feature doesn't exist.