cpython/Lib/test
Guido van Rossum 54610f31cc output for openpty test 2000-07-01 03:34:14 +00:00
..
output output for openpty test 2000-07-01 03:34:14 +00:00
README Describe a bit about writing test cases for Python... 2000-06-30 06:08:35 +00:00
__init__.py Add __init__.py to test package. 1997-11-06 15:41:23 +00:00
audiotest.au The encoding type was wrong, I think. 1999-02-05 20:57:44 +00:00
autotest.py Restore old behavior of autotest and testall, using regrtest. 1997-03-07 21:05:43 +00:00
greyrgb.uue Use uuencoded test images. 1997-04-16 00:30:45 +00:00
pystone.py Separate out a function pystones(loops=LOOPS) which runs the benchmark 1997-11-06 15:45:05 +00:00
re_tests.py AMK's latest 1998-04-03 21:47:12 +00:00
regex_tests.py 1.5a3 prerelease 1 from AMK 1997-08-13 22:34:14 +00:00
regrtest.py * added a randomize flag and corresponding -r command line argument that 2000-06-30 16:39:27 +00:00
reperf.py Measure performance of sub(), split(), findall(). 1998-07-17 21:10:42 +00:00
sortperf.py Add Tim's worst case scenario. 1998-05-26 15:05:12 +00:00
test_MimeWriter.py Untabified -- to satisfy the tab nanny. 1998-06-09 19:19:40 +00:00
test_al.py typos fixed by Rob Hooft 2000-06-28 14:48:01 +00:00
test_array.py Trent Mick <trentm@activestate.com>: 2000-06-28 17:50:51 +00:00
test_atexit.py Added the atexit module and documentation from Skip Montanaro 2000-06-28 15:07:31 +00:00
test_audioop.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_b1.py typos fixed by Rob Hooft 2000-06-28 14:48:01 +00:00
test_b2.py Revise tests to support str(<long int object>) not appending "L". 1999-12-23 15:36:42 +00:00
test_binascii.py Added test for new crc32() function. 2000-02-16 21:13:06 +00:00
test_binhex.py This test really only tests the binhex module. 1999-10-19 17:48:54 +00:00
test_bsddb.py Remove the temp file when we're done. 1999-03-24 19:03:01 +00:00
test_builtin.py Initial revision 1992-01-27 17:00:37 +00:00
test_cd.py typos fixed by Rob Hooft 2000-06-28 14:48:01 +00:00
test_cl.py typos fixed by Rob Hooft 2000-06-28 14:48:01 +00:00
test_cmath.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_contains.py Deviant1 didn't work as advertised 2000-04-27 21:42:48 +00:00
test_cpickle.py Charles Waldman writes: 2000-04-21 20:49:58 +00:00
test_crypt.py Many scripts, but small changes. Update the way the scripts obtain the 1996-12-20 22:36:52 +00:00
test_dbm.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_dl.py 1. Print the error message (carefully) when a dl.open() fails in verbose mode. 1999-02-23 17:58:48 +00:00
test_errno.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_exceptions.py add minimal test of exception use. verify that each exception can be 2000-06-20 18:52:57 +00:00
test_extcall.py Don't be so strict in checking AttributeError -- the error message 2000-04-10 13:37:14 +00:00
test_fcntl.py Daniel Neri: OpenBSD is just as BSD'ish as the other BSD's ;-) 1999-04-19 17:22:12 +00:00
test_fork1.py Raise ImportError when os.fork does not exist. 2000-05-04 00:36:42 +00:00
test_format.py Marc-Andre Lemburg <mal@lemburg.com>: 2000-06-30 10:26:29 +00:00
test_gc.py remove all prints (since the prints where of memory locations) 2000-06-30 15:13:15 +00:00
test_gdbm.py Added test case for accessing gsbm database by key after it's closed; 2000-02-07 17:15:48 +00:00
test_gl.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_grammar.py add test cases for Greg Ewing's extended call syntax patch 2000-03-28 23:51:17 +00:00
test_grp.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_gzip.py Use binary mode for all gzip files we open. 1999-04-07 19:00:58 +00:00
test_hash.py This patch addresses two main issues: (1) There exist some non-fatal 2000-06-29 19:17:04 +00:00
test_imageop.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_imgfile.py Move unified findfile() into test_support.py 1998-04-23 20:13:30 +00:00
test_linuxaudiodev.py Added test for linnuxaudiodev module; directly adapted from sunaudiodev 2000-06-10 04:22:57 +00:00
test_long.py Christian Tismer: added test to ensure that multiplication commutes. 2000-04-10 17:41:37 +00:00
test_longexp.py verify that Python raises SyntaxError for long and deeply-nested expressions 2000-06-20 19:13:27 +00:00
test_math.py Trent Mick <trentm@activestate.com>: 2000-06-01 17:59:17 +00:00
test_md5.py * wdbframewin.py (re_eval): set __privileged__ in globals so private 1993-12-14 15:54:01 +00:00
test_mmap.py Additional tests for seek() method, written by Trent Mick 2000-06-18 04:47:08 +00:00
test_new.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_nis.py Slight reworking of this test. If nis.maps() gives a nis.error, then 1999-01-28 04:54:33 +00:00
test_ntpath.py Fix the tests now that splitdrive() no longer treats UNC paths special. 1999-04-06 19:38:18 +00:00
test_opcodes.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_openpty.py Thomas Wouters: Test script for openpty() 2000-07-01 01:13:31 +00:00
test_operations.py Initial revision 1992-01-27 17:00:37 +00:00
test_operator.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_pickle.py Basic regr tests for pickle/cPickle 1999-03-25 22:38:49 +00:00
test_pkg.py Test that "import sys.imp" fails as it should. 1998-05-19 15:09:42 +00:00
test_popen2.py Test for popen2 module, by Chris Tismer. 1999-03-11 13:26:23 +00:00
test_pow.py Revise tests to support str(<long int object>) not appending "L". 1999-12-23 15:36:42 +00:00
test_pty.py Thomas Wouters <thomas@xs4all.net>: 2000-06-30 23:22:35 +00:00
test_pwd.py typos fixed by Rob Hooft 2000-06-28 14:48:01 +00:00
test_pyexpat.py Change pyexpat test suite to exercise the .returns_unicode attribute, 2000-06-27 00:37:25 +00:00
test_re.py Switch to sre for regular expression matching (the new mini-re module 2000-06-30 16:25:20 +00:00
test_regex.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_rfc822.py added a test for "To: :" patch 1999-07-12 18:47:00 +00:00
test_rgbimg.py Move unified findfile() into test_support.py 1998-04-23 20:13:30 +00:00
test_rotor.py Oops. Of course, print b should also be print `b`. :-( 1997-08-18 13:42:28 +00:00
test_select.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_sha.py Added Jeremy's test code for the sha module. 1999-03-24 19:04:32 +00:00
test_signal.py os2 patch by Jeff Rush 1997-11-22 21:53:48 +00:00
test_socket.py Skip Montanaro: modify test_socket.py to listen/connect using loopback addr 2000-06-29 17:00:54 +00:00
test_sre.py - pedantic: make sure "python -t" doesn't complain... 2000-06-30 07:50:59 +00:00
test_strftime.py # Bah. The same problem occurred a second time. 1999-04-08 20:22:46 +00:00
test_string.py Marc-Andre Lemburg: 2000-04-11 15:37:24 +00:00
test_strop.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_struct.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_sunaudiodev.py Move unified findfile() into test_support.py 1998-04-23 20:13:30 +00:00
test_support.py Changed runs of 8 spaces to tab -- to satisfy the tab nanny. 1998-06-09 19:20:12 +00:00
test_thread.py Use random instead of whrandom. 1998-05-20 17:05:52 +00:00
test_time.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_timing.py Test of the timing module 1997-01-13 23:03:07 +00:00
test_tokenize.py Move unified findfile() into test_support.py 1998-04-23 20:13:30 +00:00
test_types.py Add tests to exercise sequence operations (multiplication, indexing, 2000-02-23 22:23:17 +00:00
test_ucn.py New test for the ucnhash module. 2000-06-30 09:45:20 +00:00
test_unicode.py Marc-Andre Lemburg <mal@lemburg.com>: 2000-06-30 09:13:35 +00:00
test_unicodedata.py Marc-Andre Lemburg: 2000-03-28 20:29:59 +00:00
test_unpack.py Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
test_userdict.py Test suite for UserDict 1999-03-26 15:32:05 +00:00
test_userlist.py robustify UserList constructor -- will now accept any sequence 2000-03-31 00:17:46 +00:00
test_userstring.py UserString class from Peter Funk <pf@artcom-gmbh.de>. 2000-04-03 03:51:50 +00:00
test_winreg.py The low-level interface is now in _winreg; update the import here. 2000-06-29 19:42:00 +00:00
test_winreg2.py Paul Prescod <paul@prescod.net>: 2000-06-30 19:36:23 +00:00
test_winsound.py Fix spelling error and remove Windows line endings. 2000-04-24 14:01:51 +00:00
test_xmllib.py New version of xmllib from Sjoerd. 1998-01-29 14:55:24 +00:00
test_zipfile.py Test for zipfile.py, by Jim Ahlstrom. 2000-04-10 13:23:04 +00:00
test_zlib.py typos fixed by Rob Hooft 2000-06-28 14:48:01 +00:00
testall.py Restore old behavior of autotest and testall, using regrtest. 1997-03-07 21:05:43 +00:00
testimg.uue Use uuencoded test images. 1997-04-09 21:25:01 +00:00
testimgr.uue Use uuencoded test images. 1997-04-09 21:25:01 +00:00
testrgb.uue Use uuencoded test images. 1997-04-09 21:25:01 +00:00
tokenize_tests.py Tests for tokenize.py (Ka-Ping Yee) 1997-10-27 22:15:06 +00:00

README

			 Writing Python Test Cases
			 -------------------------
			       Skip Montanaro

If you add a new module to Python or modify the functionality of an existing
module, it is your responsibility to write one or more test cases to test
that new functionality.  The mechanics of the test system are fairly
straightforward.  If you are writing test cases for module zyzzyx, you need
to create a file in .../Lib/test named test_zyzzyx.py and an expected output
file in .../Lib/test/output named test_zyzzyx ("..." represents the
top-level directory in the Python source tree, the directory containing the
configure script).  Generate the initial version of the test output file by
executing:

    cd .../Lib/test
    python regrtest.py -g test_zyzzyx.py

Any time you modify test_zyzzyx.py you need to generate a new expected
output file.  Don't forget to desk check the generated output to make sure
it's really what you expected to find!  To run a single test after modifying
a module, simply run regrtest.py without the -g flag:

    cd .../Lib/test
    python regrtest.py test_zyzzyx.py

To run the entire test suite, make the "test" target at the top level:

    cd ...
    make test

Test cases generate output based upon computed values and branches taken in
the code.  When executed, regrtest.py compares the actual output generated
by executing the test case with the expected output and reports success or
failure.  It stands to reason that if the actual and expected outputs are to 
match, they must not contain any machine dependencies.  This means 
your test cases should not print out absolute machine addresses or floating
point numbers with large numbers of significant digits.

Writing good test cases is a skilled task and is too complex to discuss in
detail in this short document.  Many books have been written on the subject.
I'll show my age by suggesting that Glenford Myers' "The Art of Software
Testing", published in 1979, is still the best introduction to the subject
available.  It is short (177 pages), easy to read, and discusses the major
elements of software testing, though its publication predates the
object-oriented software revolution, so doesn't cover that subject at all.
Unfortunately, it is very expensive (about $100 new).  If you can borrow it
or find it used (around $20), I strongly urge you to pick up a copy.

As an author of at least part of a module, you will be writing unit tests
(isolated tests of functions and objects defined by the module) using white
box techniques.  (Unlike black box testing, where you only have the external
interfaces to guide your test case writing, in white box testing you can see
the code being tested and tailor your test cases to exercise it more
completely).

The most important goal when writing test cases is to break things.  A test
case that doesn't uncover a bug is less valuable than one that does.  In
designing test cases you should pay attention to the following:

    1. Your test cases should exercise all the functions and objects defined
       in the module, not just the ones meant to be called by users of your
       module.  This may require you to write test code that uses the module
       in ways you don't expect (explicitly calling internal functions, for
       example - see test_atexit.py).

    2. You should consider any boundary values that may tickle exceptional
       conditions (e.g. if you were testing a division module you might well 
       want to generate tests with numerators and denominators at the limits 
       of floating point and integer numbers on the machine performing the
       tests as well as a denominator of zero).

    3. You should exercise as many paths through the code as possible.  This 
       may not always be possible, but is a goal to strive for.  In
       particular, when considering if statements (or their equivalent), you 
       want to create test cases that exercise both the true and false
       branches.  For while and for statements, you should create test cases 
       that exercise the loop zero, one and multiple times.