Commit Graph

255 Commits

Author SHA1 Message Date
Tim Peters c377b16d12 Since the most likely failure mode for an expected-output test is a change
somewhere inside a line, use ndiff so that intraline difference marking
can point out what changed within a line.  I don't remember diff-style
abbreviations either (haven't used it since '94, except to produce
patches), so say the rest in English too.
2001-09-22 05:31:03 +00:00
Guido van Rossum cf691935bb reportdiff(): print a "plain diff" style diff.
XXX This should really be a unified diff, but I can't be bothered.
2001-09-21 21:06:22 +00:00
Guido van Rossum 0a07639779 Oops. I didn't expect that some tests (test_cookie) have expected
output *and* doctest stuff.  Assuming the doctest stuff comes after the
expected output, this fixes that.
2001-09-21 20:45:44 +00:00
Guido van Rossum 0fcca4e815 Change the way unexpected output is reported: rather than stopping at
the first difference, let the test run till completion, then gather
all the output and compare it to the expected output using difflib.

XXX Still to do: produce diff output that only shows the sections that
differ; currently it produces ndiff-style output because that's the
easiest to produce with difflib, but this becomes a liability when the
output is voluminous and there are only a few differences.
2001-09-21 20:31:52 +00:00
Guido van Rossum eb94905265 Get rid of a superfluous space after "--" in the message printed for a
skipped test -- the print command already supplies a space.
2001-09-18 20:34:19 +00:00
Tim Peters a86f0c17a1 Make test_socketserver require the network resource.
Add it back to the list of tests we expect to skip on Windows.
2001-09-18 02:18:57 +00:00
Tim Peters 7a76d4b4e4 I don't expect test_socketserver to get skipped on Windows anymore. 2001-09-18 00:24:10 +00:00
Tim Peters a0a6222509 Teach regrtest how to pass on doctest failure msgs. This is done via a
horridly inefficient hack in regrtest's Compare class, but it's about as
clean as can be:  regrtest has to set up the Compare instance before
importing a test module, and by the time the module *is* imported it's too
late to change that decision.  The good news is that the more tests we
convert to unittest and doctest, the less the inefficiency here matters.
Even now there are few tests with large expected-output files (the new
cost here is a Python-level call per .write() when there's an expected-
output file).
2001-09-09 06:12:01 +00:00
Guido van Rossum fe3f6969f5 Two small changes to the resource usage option:
(1) Allow multiple -u options to extend each other (and the initial
    value of use_resources passed into regrtest.main()).

(2) When a test is run stand-alone (not via regrtest.py), needed
    resources are always granted.
2001-09-06 16:09:41 +00:00
Martin v. Löwis 0ace326ed2 Patch #453627: Adds a list of tests that are expected to be skipped for UnixWare 7.x systems. 2001-09-05 14:38:48 +00:00
Guido van Rossum aa78236636 Whitespace normalization (tabs -> 4 spaces) in the Mac expectations. 2001-09-02 03:58:41 +00:00
Jack Jansen 49a806edbb Added list of tests expected to be skipped on the mac. 2001-08-28 14:49:00 +00:00
Barry Warsaw 08fca52125 Removed --have-resources flag in favor of the more granular -u/--use
flag, which specifies external or resource intensive tests to
perform.  This is used by test_largefile and test_socket_ssl.

-u/--use takes a comma separated list of flags, currently supported:
largefile, network.

usage(): New function.  Note that the semantics of main() have changed
    slightly; instead of returning an error code, it raises a
    SystemExit (via sys.exit()) with the given error code.

main(): use_large_resources => use_resources
    Also, added support for long-option alternative to the short
    options.

_expectations: Added test_socket_ssl to the list of expectedly skipped
    tests.
2001-08-20 22:33:46 +00:00
Jeremy Hylton 8471a35feb Fix SF bug [ #450245 ] Error in parsing future stmts
Add test case to cover multiple future statements on separate lines of
a module.
2001-08-20 20:33:42 +00:00
Jeremy Hylton 39f77bc90e Modify _Set to support iteration.
Otherwise printlist(surprise) will fail with a TypeError, because map
is called with an argument that doesn't support iteration.
2001-08-12 21:53:08 +00:00
Tim Peters a45da92484 Make the output of tests skipped readable (i.e., deliberately break it
into indented lines each of which probably fits on a typical screen line).
2001-08-12 03:45:50 +00:00
Guido van Rossum f73e30c3e3 Add the list of expected skips for Linux 2.x. Restructured the code a
little bit using a dictionary to avoid more code duplication as
more platforms are supported.
2001-08-12 02:22:19 +00:00
Tim Peters a2be2d624a Move line; reported on python-dev by Mark Favas (thanks!). 2001-08-12 02:01:09 +00:00
Tim Peters b5b7b78414 Teach regrtest which tests we *expect* to skip on Win32. Please teach it
about your platform too.
2001-08-12 01:20:39 +00:00
Fred Drake e51fe8d0a3 runtest(): When generating output, if the result is a single line with the
name of the test, only write the output file if it already exists (and
    tell the user to consider removing it).  This avoids the generation of
    unnecessary turds.
2001-05-29 17:10:51 +00:00
Tim Peters d97422115e Implementing an idea from Guido on the checkins list:
When regrtest.py finds an attribute "test_main" in a test it imports,
regrtest runs the test's test_main after the import.  test_threaded_import
needs this else the cross-thread import lock prevents it from making
progress.  Other tests can use this hack too, but I doubt it will ever be
popular.
2001-05-22 18:28:25 +00:00
Fred Drake ae1bb176be If the file containing expected output does not exist, assume that it
contains a single line of text giving the name of the output file.  This
covers all tests that do not actually produce any output in the test code.
2001-05-21 21:08:12 +00:00
Jeremy Hylton 3090694068 Fix compileall.py so that it fails on SyntaxErrors
The changes cause compilation failures in any file in the Python
installation lib directory to cause the install to fail.  It looks
like compileall.py intended to behave this way, but a change to
py_compile.py and a separate bug defeated it.

Fixes SF bug #412436

This change affects the test suite, which contains several files that
contain intentional errors.  The solution is to extend compileall.py
with the ability to skip compilation of selected files.

In the test suite, rename nocaret.py and test_future[3..7].py to start
with badsyntax_nocaret.py and badsyntax_future[3..7].py.  Update the
makefile to skip compilation of these files.  Update the tests to use
the name names for imports.

NB compileall.py is changed so that compile_dir() returns success only
if all recursive calls to compile_dir() also check success.
2001-04-18 01:19:28 +00:00
Jeremy Hylton 62e2c7e3df Add regression test for future statements. This adds eight files, but
seven are not tests in their own right; these files are mentioned in
regrtest.
2001-02-28 17:48:06 +00:00
Barry Warsaw 22e4182d60 Describe -s a little more generically. 2001-02-23 18:31:40 +00:00
Eric S. Raymond fc170b1fd5 String method conversion. 2001-02-09 11:51:27 +00:00
Tim Peters 1a4d77b252 Christmas present to myself: changed regrtest in two ways:
1. When running in verbose mode, if any test happens to pass, print
   a warning that the apparent success may be bogus (stdout isn't
   compared in verbose mode).  Been fooled by that too often.
2. When a test fails because the expected stdout doesn't match the
   actual stdout, print as much of stdout as did match before the
   first failing write.  Else we get failures of the form "expected
   'a', got 'b'" and a glance at the expected output file shows
   500 instances of 'a' -- no idea where it failed, and, as in #1,
   trying to run in verbose mode instead doesn't help because
   stdout isn't compared then.
2000-12-30 22:21:22 +00:00
Fred Drake 132dce2246 Update the code to better reflect recommended style:
Use != instead of <> since <> is documented as "obsolescent".
Use "is" and "is not" when comparing with None or type objects.
2000-12-12 23:11:42 +00:00
Fred Drake 004d5e6880 Make reindent.py happy (convert everything to 4-space indents!). 2000-10-23 17:22:08 +00:00
Neil Schemenauer 8a00abc0ff Make the regrtest.py -l (findleaks) option considerably less obnoxious.
First, only report garbage that the GC cannot free.  Second, only report
the number of objects found, not their repr().  People can dig deeper on
their own if they find a leak.
2000-10-13 01:32:42 +00:00
Neil Schemenauer d569f23da9 - Replace debugleak flag with findleaks flag. The new SAVEALL GC option is
used to find cyclic garbage produced by tests.
2000-09-22 15:29:28 +00:00
Fred Drake 27c4b39025 When skipping a test, do not include a spurious space between the exception
name and the ":" that separates it from the value.  (Minor cleanup.)
2000-08-23 20:34:40 +00:00
Fred Drake fe5c22a85e When a KeyboardInterrupt is caught, just use the "raise" syntax to
re-raise it instead of re-raising it "manually" the ugly way.
2000-08-18 16:04:05 +00:00
Trent Mick f29f47b38b Add largefile support for Linux64 and WIn64. Add test_largefile and some minor
change to regrtest.py to allow optional running of test_largefile ('cause it's
slow on Win64).

This closes patches:
http://sourceforge.net/patch/index.php?func=detailpatch&patch_id=100510&group_id=5470
and
http://sourceforge.net/patch/index.php?func=detailpatch&patch_id=100511&group_id=5470
2000-08-11 19:02:59 +00:00
Thomas Wouters 3af826ebca Make test_support.TestSkipped errors work the same way as ImportErrors:
mark the test as 'skipped', rather than 'failed'.
2000-08-04 13:17:51 +00:00
Barry Warsaw a873b03ebb Added a -l/--leakdebug option which turns on DEBUG_LEAK if the gc
module is importable.
2000-08-03 15:50:37 +00:00
Thomas Wouters 7e47402264 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
comments, docstrings or error messages. I fixed two minor things in
test_winreg.py ("didn't" -> "Didn't" and "Didnt" -> "Didn't").

There is a minor style issue involved: Guido seems to have preferred English
grammar (behaviour, honour) in a couple places. This patch changes that to
American, which is the more prominent style in the source. I prefer English
myself, so if English is preferred, I'd be happy to supply a patch myself ;)
2000-07-16 12:04:32 +00:00
Jeremy Hylton fff9e20943 fix bug #42 reported by Andrew Dalke
The Compare close contains a close method that checks to see if there
is any unconsumed data in the Compare instance; i.e. if the canonical
output file contains more data than was produced by the current test
run. This method was never called, allowing differences to go
undetected.

Fix is to call close after the test is run (after __import__)

output/test_long and output/test_popen2 needed trivial changes
output/test_select contained lots of text, but test_select.py produced
    no output
2000-07-11 15:15:31 +00:00
Skip Montanaro ab1c7918f6 * added a randomize flag and corresponding -r command line argument that
allows the caller to execute the various tests in pseudo-random order -
  default is still to execute tests in the order returned by findtests().

* moved initialization of the various flag variables to the main() function
  definition, making it possible to execute regrtest.main() interactively
  and still override default behavior.
2000-06-30 16:39:27 +00:00
Guido van Rossum 5193114442 Alas, Vladimir's patch was too aggressive, and started causing really
weird errors.  (E.g. see thread "weird bug in test_winreg" in python-dev.)

Since it's actually useful to be able to re-run an individual test
after running test.autotest, we keep the unloading code, but only for
modules whose full name starts with "test.".
2000-05-05 14:27:39 +00:00
Guido van Rossum 5796d26794 Patch by Vladimir Marangozov to unload additionally imported modules
after each test has been run.  This avoids excessive memory growth
during the tests.
2000-04-21 21:35:06 +00:00
Barry Warsaw e11e3dee3e Added a -s option which is useful for narrowing down memory leaks.
With -s only a single test is run.  The next test run is chosen
sequentially from the list of all tests.
1999-01-28 19:51:51 +00:00
Guido van Rossum 6c74fea07d There was still something wrong. The original NOTTESTS are replaced
by the new '-x' arguments, losing the previous items.  Thus,
test_support, test_b1 & test_b2 are executed (and warnings issued).
(Discovered by Vladimir Marangozov.)
1998-08-25 12:29:08 +00:00
Guido van Rossum 747e1cade6 Should pass explicit arguments to findtests(). Should initialize 'nottests'. 1998-08-24 13:48:36 +00:00
Guido van Rossum 6fd83b7b38 Generalized so it's useful for testing other packages, by Andrew
Kuchling @ CNRI.
1998-08-01 17:04:08 +00:00
Guido van Rossum e87ed5f6d4 Add writelines() method to Compare class. 1998-04-23 13:33:21 +00:00
Guido van Rossum 41360a4696 Mass check-in after untabifying all files that need it. 1998-03-26 19:42:58 +00:00
Guido van Rossum 4e8ef5fcd3 Catch KeyboardInterrupt separately and propagate it, instead of
reporting a "crash".

Use sys.exc_info() instead of sys.exc_type and sys.exc_value.
1997-10-20 23:46:54 +00:00
Guido van Rossum f9fa20b2d2 Pass optional arguments (globals(), locals(), []) to __import__() so
it will run as package test, e.g. this will run the test suite:

    import test.regrtest
    test.regrtest.main()
1997-09-07 16:42:34 +00:00
Guido van Rossum a412220bbf Change default verbosity so that there are only three levels left: -q,
default and -v.  In default mode, the name of each test is printed.
-v is the same as the old -vv.  -q is more quiet than the old default
mode; that's fine I think.
1997-08-18 20:08:24 +00:00
Guido van Rossum e83870131b Don't call sys.exit() all over the place -- simply return the exit
status from main() and call sys.exit(main()) in the startup stub at
the end of the file.
1997-08-14 19:40:34 +00:00
Guido van Rossum 75fce308bc Add flush() method to fake file. 1997-07-17 14:51:37 +00:00
Guido van Rossum 9e48b272b9 Catch all exceptions in test modules. 1997-07-16 01:56:13 +00:00
Guido van Rossum f58ed25967 Add "extra-verbose" mode, triggered by specifying two -v flags. In
this mode, all tests are run in verbose mode with their output to
stdout.  No comparing of output is done.
1997-03-07 21:04:33 +00:00
Guido van Rossum 152494aea2 New regression test harness. See usage message / doc string. 1996-12-20 03:12:20 +00:00