Framework History


This framework was born of need, many times. Here I chronicle my implementations to establish a depth of experience and to give credit to those who have helped me through it. Others have had similar experiences; see ReadingList. -- WardCunningham


WyattSoftware (1989) -- My first implementation was done while working on financial software written in Smalltalk. I was inspired by the conciseness and clarity of the test cases GaryGoldberg worked up to test his understanding of financial instruments. These eventually became our test cases. I decided to short-circuit the process by reading his spreadsheets into a test framework I wrote in a day. We already had a well developed table and type conversion framework. The combined framework supported what we would now call a ColumnFixture. AlanDarlington hooked it into our daily build and PaulMcCullough refactored our application from single user to client server with it. Not only did we learn to depend on these cases as much as our own abstractions, we sailed through a big five audit when the auditor admitted that he'd never seen anything like our test browser and passed us before lunch.

The testing framework takes me way back. That was fun getting that to work, fast update of test cases from spreadsheets, automatic batch processing at night. Your "Framework History" was eye-opening -- I didn't know these ideas had a life outside of the particular job. It's the right thing to do, give a name to a concept and propagate it so that future generations can benefit. You could call it the "Testing Meme." -- GaryGoldberg

ThoughtWorks (1999) -- MartinFowler encouraged me to look at ways to get similar functionality with Java. EmmetMurphy and I wrote a framework that read Excel's tab separated text files, ran tests, and wrote results as html. Our first tests followed a format already in use by ThoughtWorks' analysts which lead to a RowFixture like approach. For our first end-to-end test TerriHollar and MattSimons added concrete values to CaraTauber's "Spot Books a Lease" scenario and thta was interpreted using JavaReflection in an ActionFixture like arrangement cooked up by DaveRice and myself.

RoleModelSoftware (2000) -- KenAuer brought me in to launch their automated acceptance testing for a medical instrument. The FoodAndDrugAdministration was clear regarding their audits: if it wasn't written down it didn't happen. A color laser printer was ordered. And just to be sure the auditor understood what he was looking at we added an extra column where the tests would write "PASSED" as they did. We again looked to how the customer liked to write documents, in this case, Microsoft Word. ChrisCollins and I found we could reliably find tabular data in the strange html Word read and wrote. And Ken's modular approach to Java GUI made action fixturing trivial. NathanielTalbott reved this system several times before creating his RubyLanguage based framework.

Vulcan (2002) -- BrianDorsey and I built a version of the framework in Python and fixtured it to talk directly to StoredProcedures in the database. NormParks helped us work up tests that paralleled his GUI calls to the same procs, including retrieving artwork and including it in the html test results. The stock Python html parser choked on Word's output so we were temporarily stumped. The Seattle Python Interest Group (SeaPig) had two suggestions: try parsing with regular expressions and read from a wiki instead. We did both.


But wait, there's more.

EricEvans convinced me that there should be a widely available version of this framework. I wasn't convinced immediately. I knew each implementation I'd done had been wildly different, largely due to differing approaches to fixturing.

I pitched KentBeck on including this experience in his test-first book. He said no. I'd have to work harder myself if I wanted a wide audience to see this work.

I wrote the current parser one evening in my hotel room using index and substring in place of regular expressions. BrianForst reviewed this and the PrimitiveFixture that went with it. I was encouraged.

BretPettichord and BrianMarick hosted a workshop on Agile Acceptance Testing at Agile/XP Universe in Chicago. Conversations with them and JamesBach convinced me that the framework was only a small part of what was needed for testing to be agile, but it was an indispensable part none the less.

I took an iBook with me on vacation in the Oregon High Desert. I wrote all the reflection code early in the morning while my family slept. I used IntellijIdea and refactored mercilessly. I built the wiki on my return and invited the WhosWho of important people to helped me get this to beta. Now here it is. Enjoy.

 

Last edited June 21, 2003
Return to WelcomeVisitors