Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs.
Add check for invalid --huntrleaks/-R parameters.
If tests are re-run, use "xxx then yyy" result format (ex: "FAILURE
then SUCCESS") to show that some failing tests have been re-run.
Add also test_regrtest.test_rerun_fail() test.
* "running:" progress: Format number of seconds as hours and minutes
* format_duration(): count also minutes as hours
* Create Lib/test/libregrtest/utils.py
* No longer clear filters, like --match, to re-run failed tests in
verbose mode (-w option).
* Tests result: always indicate if tests have been interrupted.
* Enhance tests summary
* Rename support._match_test() to support.match_test(): make it
public
* Remove support.match_tests global variable. It is replaced with a
new support.set_match_tests() function, so match_test() doesn't
have to check each time if patterns were modified.
* Rewrite match_test(): use different code paths depending on the
kind of patterns for best performances.
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
* Add Lib/test/pythoninfo.py: script collecting various informations
about Python to help debugging test failures.
* regrtest: remove sys.hash_info and sys.flags from header.
* Travis CI, Appveyor: run pythoninfo before tests
* bpo-30764: regrtest: change exit code on failure
* Exit code 2 if failed tests ("bad")
* Exit code 3 if interrupted
* bpo-30764: regrtest: add --fail-env-changed option
If the option is set, mark a test as failed if it alters the
environment, for example if it creates a file without removing it.
* regrtest --list-cases now supports --match and --match-file options.
Example: ./python -m test --list-cases -m FileTests test_os
* --list-cases now also sets support.verbose to False to prevent
messages to stdout when loading test modules.
* Add support._match_test() private function.
Use a build/ directory in the build directory, not in the source
directory, since the source directory may be read-only and must not
be modified.
Fallback on the source directory if the build directory is not
available (missing "abs_builddir" sysconfig variable).
Issue #29362: Catch a crash of a worker process as a normal failure and
continue to run next tests. It allows to get the usual test summary: single
line result (OK/FAIL), total duration, etc.
It's sometimes hard to check quickly if tests succeeded, failed or something
bad happened. I added a final "Result: xxx" line which summarizes all outputs
into a single line, written at the end (it should always be the last line of
the output).
* Rename libregrtest.main_in_temp_cwd() to libregrtest.main()
* Add regrtest.main_in_temp_cwd() alias to libregrtest.main()
* Move old main_in_temp_cwd() code into libregrtest.Regrtest.main()
* Update multiple scripts to call libregrtest.main()
* Fix accumulate_result(): don't use time on interrupted and failed test
* Add unit test for interrupted test
* Add unit test on --slow with interrupted test, with and without
multiprocessing
* Fix "-m test --forever": replace _test_forever() with self._test_forever()
* Add unit test for --forever
* Add unit test for a failing test
* Fix also some pyflakes warnings in libregrtest
* Remove runtest_ns(): pass directly ns to runtest().
* Create also Regrtest.rerun_failed_tests() method.
* Inline again Regrtest.run_test(): it's no more justified to have a method
Slaves (child processes running tests for regrtest -jN) now inherit
--memlimit/-M, --threshold/-t and --nowindows/-n options.
* -M, -t and -n are now supported with -jN
* Factorize code to run tests.
* run_test_in_subprocess() now pass the whole "ns" namespace to the child
process.
Steve Dower
Victor Stinner
Pablo Galindo
Victor Stinner
mlouielu
Brett Cannon