So we looked into the FIT framework. Specifically, we wanted a suite of tests, written as documents, in a hierarchy that matched our existing data and model documentation hierarchy. (The root of our documentation is http://www.urbansim.org/docs/urbansim2/.) We wanted our first test documents to read as:
Start with a set of good data stored in database X. Alter tables Y and Z to contain bad data. Run the data consistency checker and it should report errors A, B, and C.
We started by writing an HTML document in the desired format (using FrontPage 2000 to make it easy to manipulate the tables). We use nested HTML tables:
Notice that the table reads exactly as the desired prose. After processing, the table is annotated with green (or red) and with explanations of the errors:
Then we set out to subclass the necessary FIT classes to make this work. It was quite simple:
I didn't document the process that first evening: I just sent an email to the developers telling them to use this new framework for writing tests. But it turns out that they didn't need any documentation because the test format was self describing and easy to use. So the developers went to town creating new tests.
Later that week, Ward demo'd the run.cgi feature of writing tests on the wiki, which I liked enough to implement a similar scheme for our pages. Our pages are CVS source controlled rather than being in a wiki, so we implemented a simple httpd server in Perl which, when given a page with the "?run.cgi" annotation, will run the test through the FIT framework and return the result.
The trickest part of setting the whole FIT framework up was extending Parse to handle nested tables. Other than that, it was quite easy to use.
See source for fixtures and the run script.UrbanSim source code (http://www.urbansim.org/download/index.shtml) and look at the urbansim.tests.fit package.
Neat! Nit: "green is good" but "Consistency checking aborted because errors have been found while checking the base data..." is green? --PaulChisholm
|Last edited February 8, 2003
Return to WelcomeVisitors