This is a python translation of fit, by Simon Michael . fit (the Framework for Interactive Testing) is copyright (c) 2002 Cunningham & Cunningham, Inc, and released under the terms of the GNU General Public License version 2 or later. Usage: add this directory to your PYTHONPATH and, eg, python fit/FileRunner.py Documents/arithmetic.html Reports/arithmetic.html Notes: - this is version b021006 (the first) based on java fit version b020927 - has not been made python 1.5 compatible - example Documents have been converted to unix line endings - types are handled differently from the java version, see below - "equal-but-different" results are displayed - Some java class-side methods have become python instance methods - java signatures preserved in comments, remove later ? - name changes to avoid clashes with keywords/builtins/methods: - Fixture.right, wrong -> rights, wrongs - type argument -> atype - Parse.print -> use print str(p) instead - ActionFixture.check -> check_ - emulated multimethods variously by - a single method with optional arguments - multiple methods with differing names - hardcoded reference to a specific implementation, eg Fixture.check() Todo: - why ../fit/fit in tracebacks ? - verbose/debug option - figure out issues with use of rexec - dates support ? - `` enables exact repr-matching ? - don't sys.exit when invoked from python prompt Dynamic types, conversion, rendering, matching etc. Certain issues arise when using fit with a dynamically-typed language like python. Unlike the java version, which knows the expected type of every field, method argument and method return value, we must infer this information from other sources or do without it. Some alternatives and difficulties are noted below. Now, here's the strategy which I think is the most useful and which this version implements: - Infer all types from the test data's format, treating it like a python string literal. - When checking results, compare expected and actual as typed objects using ==. - For passing tests, also compare the repr representations; if they differ display both. We are working with two kinds of table cell - input data and expected results. Setting strategies a. set everything as strings b. choose a type based on the class default for the attribute c. choose a type based on the data format d. b, otherwise c e. other possibilities.. interfaces, markup in the test data Matching strategies 1. value matching, type based only on class defaults If there's an existing attribute and an adapter to it's type, convert expected to that type. Compare with actual's value. con: - fails a lot (unless your methods accept and return strings) since we often can't infer type - expected 1 will match actual 1.0 as per python comparison rules; can't specify a test that requires a floating point result 2. canonical string matching, type based only on class defaults If there's an existing attribute and an adapter to it's type, convert expected to that type. Now convert both expected & actual to their python string literal format (using repr) and compare. con: - fails a lot when a type cannot be inferred (unless your methods accept and return strings) because expected '1' will not match actual 1 - and, in this situation they look the same in the report; expected should have quotes to show that it was treated as a string - more strict - when type can't be inferred, user must write 0.1 not .1 pro: - on the other hand, if you arrange your methods to accept and return strings, this method allows exact control. 3. value matching, type based on class defaults or data format If there's an existing attribute and an adapter corresponding to it's type, convert expected to that type. If not, infer the type from expected's format (eg by doing a python eval). Compare with actual's value. con: - expected 1 will match actual 1.0 as per python comparison rules; this (and other things ?) may cause confusion. Here the passing test should show expected 1, actual 1.0 to reveal what's happening. - can't specify a test that requires a floating point result 4. canonical string matching, type based on class defaults or data format If there's an existing attribute and an adapter to it's type, convert expected to that type. If not, infer the type from expected's format. Then convert both to string literals & compare. 5. value matching, type based only on data format pro: simple, intuitive con: same as 3 6. canonical string matching, type based only on data format pro: simple, exact con: more fragile than value matching 7. original expected & canonical actual string matching For result-matching cells, converting expected to a type and then back to string literal loses information in the case of floating point numbers. Eg an expected '0.666667' will be changed to it's string literal '0.66666700000000001'. When inferring type from data format, this would prevent you testing a method which returns the string '0.666667'. Instead, we could treat expected as the literal result we expect to see, and simply compare it with the string literal form of actual. pro: simple, precise con: more work to specify, less generic There seems to be a choice between exact, language/implementation-specific tests and more lenient tests which could be more robust across different implementations & platforms.