imports of test modules now import from the test package. Other
related oddities are also fixed (like DeprecationWarning filters that
weren't specifying the full import part, etc.). Also did a general
code cleanup to remove all "from test.test_support import *"'s. Other
from...import *'s weren't changed.
imports e.g. test_support must do so using an absolute package name
such as "import test.test_support" or "from test import test_support".
This also updates the README in Lib/test, and gets rid of the
duplicate data dirctory in Lib/test/data (replaced by
Lib/email/test/data).
Now Tim and Jack can have at it. :)
-f/--fromfile <filename>
option. This runs all and only the tests named in the file, in the
order given (although -x may weed that list, and -r may shuffle it).
Lines starting with '#' are ignored.
This goes a long way toward helping to automate the binary-search-like
procedure I keep reinventing by hand when a test fails due to interaction
among tests (no failure in isolation, and some unknown number of
predecessor tests need to run first -- now you can stick all the test
names in a file, and comment/uncomment blocks of lines until finding a
minimal set of predecessors).
the local save/modify/restore of sys.stdout, but add machinery so that
regrtest can tell test_support the value of sys.stdout at the time
regrtest.main() started, and test_support can pass that out later to anyone
who needs a "visible" stdout.
somewhere inside a line, use ndiff so that intraline difference marking
can point out what changed within a line. I don't remember diff-style
abbreviations either (haven't used it since '94, except to produce
patches), so say the rest in English too.
the first difference, let the test run till completion, then gather
all the output and compare it to the expected output using difflib.
XXX Still to do: produce diff output that only shows the sections that
differ; currently it produces ndiff-style output because that's the
easiest to produce with difflib, but this becomes a liability when the
output is voluminous and there are only a few differences.
horridly inefficient hack in regrtest's Compare class, but it's about as
clean as can be: regrtest has to set up the Compare instance before
importing a test module, and by the time the module *is* imported it's too
late to change that decision. The good news is that the more tests we
convert to unittest and doctest, the less the inefficiency here matters.
Even now there are few tests with large expected-output files (the new
cost here is a Python-level call per .write() when there's an expected-
output file).
(1) Allow multiple -u options to extend each other (and the initial
value of use_resources passed into regrtest.main()).
(2) When a test is run stand-alone (not via regrtest.py), needed
resources are always granted.
flag, which specifies external or resource intensive tests to
perform. This is used by test_largefile and test_socket_ssl.
-u/--use takes a comma separated list of flags, currently supported:
largefile, network.
usage(): New function. Note that the semantics of main() have changed
slightly; instead of returning an error code, it raises a
SystemExit (via sys.exit()) with the given error code.
main(): use_large_resources => use_resources
Also, added support for long-option alternative to the short
options.
_expectations: Added test_socket_ssl to the list of expectedly skipped
tests.
name of the test, only write the output file if it already exists (and
tell the user to consider removing it). This avoids the generation of
unnecessary turds.
When regrtest.py finds an attribute "test_main" in a test it imports,
regrtest runs the test's test_main after the import. test_threaded_import
needs this else the cross-thread import lock prevents it from making
progress. Other tests can use this hack too, but I doubt it will ever be
popular.
The changes cause compilation failures in any file in the Python
installation lib directory to cause the install to fail. It looks
like compileall.py intended to behave this way, but a change to
py_compile.py and a separate bug defeated it.
Fixes SF bug #412436
This change affects the test suite, which contains several files that
contain intentional errors. The solution is to extend compileall.py
with the ability to skip compilation of selected files.
In the test suite, rename nocaret.py and test_future[3..7].py to start
with badsyntax_nocaret.py and badsyntax_future[3..7].py. Update the
makefile to skip compilation of these files. Update the tests to use
the name names for imports.
NB compileall.py is changed so that compile_dir() returns success only
if all recursive calls to compile_dir() also check success.
1. When running in verbose mode, if any test happens to pass, print
a warning that the apparent success may be bogus (stdout isn't
compared in verbose mode). Been fooled by that too often.
2. When a test fails because the expected stdout doesn't match the
actual stdout, print as much of stdout as did match before the
first failing write. Else we get failures of the form "expected
'a', got 'b'" and a glance at the expected output file shows
500 instances of 'a' -- no idea where it failed, and, as in #1,
trying to run in verbose mode instead doesn't help because
stdout isn't compared then.
First, only report garbage that the GC cannot free. Second, only report
the number of objects found, not their repr(). People can dig deeper on
their own if they find a leak.
comments, docstrings or error messages. I fixed two minor things in
test_winreg.py ("didn't" -> "Didn't" and "Didnt" -> "Didn't").
There is a minor style issue involved: Guido seems to have preferred English
grammar (behaviour, honour) in a couple places. This patch changes that to
American, which is the more prominent style in the source. I prefer English
myself, so if English is preferred, I'd be happy to supply a patch myself ;)
The Compare close contains a close method that checks to see if there
is any unconsumed data in the Compare instance; i.e. if the canonical
output file contains more data than was produced by the current test
run. This method was never called, allowing differences to go
undetected.
Fix is to call close after the test is run (after __import__)
output/test_long and output/test_popen2 needed trivial changes
output/test_select contained lots of text, but test_select.py produced
no output
allows the caller to execute the various tests in pseudo-random order -
default is still to execute tests in the order returned by findtests().
* moved initialization of the various flag variables to the main() function
definition, making it possible to execute regrtest.main() interactively
and still override default behavior.
weird errors. (E.g. see thread "weird bug in test_winreg" in python-dev.)
Since it's actually useful to be able to re-run an individual test
after running test.autotest, we keep the unloading code, but only for
modules whose full name starts with "test.".
by the new '-x' arguments, losing the previous items. Thus,
test_support, test_b1 & test_b2 are executed (and warnings issued).
(Discovered by Vladimir Marangozov.)
default and -v. In default mode, the name of each test is printed.
-v is the same as the old -vv. -q is more quiet than the old default
mode; that's fine I think.
Martin v. Löwis
Guido van Rossum
Tim Peters
Barry Warsaw
Jeremy Hylton
Andrew MacIntyre
Fred Drake
Skip Montanaro
Neal Norwitz
Jack Jansen
Andrew M. Kuchling
Eric S. Raymond
Neil Schemenauer
Trent Mick
Thomas Wouters