mirror of https://github.com/python/cpython
78 lines
4.0 KiB
Plaintext
78 lines
4.0 KiB
Plaintext
|
Writing Python Test Cases
|
||
|
|
-------------------------
|
||
|
|
Skip Montanaro
|
||
|
|
|
||
|
|
If you add a new module to Python or modify the functionality of an existing
|
||
|
|
module, it is your responsibility to write one or more test cases to test
|
||
|
|
that new functionality. The mechanics of the test system are fairly
|
||
|
|
straightforward. If you are writing test cases for module zyzzyx, you need
|
||
|
|
to create a file in .../Lib/test named test_zyzzyx.py and an expected output
|
||
|
|
file in .../Lib/test/output named test_zyzzyx ("..." represents the
|
||
|
|
top-level directory in the Python source tree, the directory containing the
|
||
|
|
configure script). Generate the initial version of the test output file by
|
||
|
|
executing:
|
||
|
|
|
||
|
|
cd .../Lib/test
|
||
|
|
python regrtest.py -g test_zyzzyx.py
|
||
|
|
|
||
|
|
Any time you modify test_zyzzyx.py you need to generate a new expected
|
||
|
|
output file. Don't forget to desk check the generated output to make sure
|
||
|
|
it's really what you expected to find! To run a single test after modifying
|
||
|
|
a module, simply run regrtest.py without the -g flag:
|
||
|
|
|
||
|
|
cd .../Lib/test
|
||
|
|
python regrtest.py test_zyzzyx.py
|
||
|
|
|
||
|
|
To run the entire test suite, make the "test" target at the top level:
|
||
|
|
|
||
|
|
cd ...
|
||
|
|
make test
|
||
|
|
|
||
|
|
Test cases generate output based upon computed values and branches taken in
|
||
|
|
the code. When executed, regrtest.py compares the actual output generated
|
||
|
|
by executing the test case with the expected output and reports success or
|
||
|
|
failure. It stands to reason that if the actual and expected outputs are to
|
||
|
|
match, they must not contain any machine dependencies. This means
|
||
|
|
your test cases should not print out absolute machine addresses or floating
|
||
|
|
point numbers with large numbers of significant digits.
|
||
|
|
|
||
|
|
Writing good test cases is a skilled task and is too complex to discuss in
|
||
|
|
detail in this short document. Many books have been written on the subject.
|
||
|
|
I'll show my age by suggesting that Glenford Myers' "The Art of Software
|
||
|
|
Testing", published in 1979, is still the best introduction to the subject
|
||
|
|
available. It is short (177 pages), easy to read, and discusses the major
|
||
|
|
elements of software testing, though its publication predates the
|
||
|
|
object-oriented software revolution, so doesn't cover that subject at all.
|
||
|
|
Unfortunately, it is very expensive (about $100 new). If you can borrow it
|
||
|
|
or find it used (around $20), I strongly urge you to pick up a copy.
|
||
|
|
|
||
|
|
As an author of at least part of a module, you will be writing unit tests
|
||
|
|
(isolated tests of functions and objects defined by the module) using white
|
||
|
|
box techniques. (Unlike black box testing, where you only have the external
|
||
|
|
interfaces to guide your test case writing, in white box testing you can see
|
||
|
|
the code being tested and tailor your test cases to exercise it more
|
||
|
|
completely).
|
||
|
|
|
||
|
|
The most important goal when writing test cases is to break things. A test
|
||
|
|
case that doesn't uncover a bug is less valuable than one that does. In
|
||
|
|
designing test cases you should pay attention to the following:
|
||
|
|
|
||
|
|
1. Your test cases should exercise all the functions and objects defined
|
||
|
|
in the module, not just the ones meant to be called by users of your
|
||
|
|
module. This may require you to write test code that uses the module
|
||
|
|
in ways you don't expect (explicitly calling internal functions, for
|
||
|
|
example - see test_atexit.py).
|
||
|
|
|
||
|
|
2. You should consider any boundary values that may tickle exceptional
|
||
|
|
conditions (e.g. if you were testing a division module you might well
|
||
|
|
want to generate tests with numerators and denominators at the limits
|
||
|
|
of floating point and integer numbers on the machine performing the
|
||
|
|
tests as well as a denominator of zero).
|
||
|
|
|
||
|
|
3. You should exercise as many paths through the code as possible. This
|
||
|
|
may not always be possible, but is a goal to strive for. In
|
||
|
|
particular, when considering if statements (or their equivalent), you
|
||
|
|
want to create test cases that exercise both the true and false
|
||
|
|
branches. For while and for statements, you should create test cases
|
||
|
|
that exercise the loop zero, one and multiple times.
|