* Clean up code which checked presence of os.{stat,lstat,chmod} (GH-11643)
(cherry picked from commit 8377cd4fcd)
* bpo-36725: regrtest: add TestResult type (GH-12960)
* Add TestResult and MultiprocessResult types to ensure that results
always have the same fields.
* runtest() now handles KeyboardInterrupt
* accumulate_result() and format_test_result() now takes a TestResult
* cleanup_test_droppings() is now called by runtest() and mark the
test as ENV_CHANGED if the test leaks support.TESTFN file.
* runtest() now includes code "around" the test in the test timing
* Add print_warning() in test.libregrtest.utils to standardize how
libregrtest logs warnings to ease parsing the test output.
* support.unload() is now called with abstest rather than test_name
* Rename 'test' variable/parameter to 'test_name'
* dash_R(): remove unused the_module parameter
* Remove unused imports
(cherry picked from commit 4d29983185)
* bpo-36725: Refactor regrtest multiprocessing code (GH-12961)
Rewrite run_tests_multiprocess() function as a new MultiprocessRunner
class with multiple methods to better report errors and stop
immediately when needed.
Changes:
* Worker processes are now killed immediately if tests are
interrupted or if a test does crash (CHILD_ERROR): worker
processes are killed.
* Rewrite how errors in a worker thread are reported to
the main thread. No longer ignore BaseException or parsing errors
silently.
* Remove 'finished' variable: use worker.is_alive() instead
* Always compute omitted tests. Add Regrtest.get_executed() method.
(cherry picked from commit 3cde440f20)
* bpo-36719: regrtest always detect uncollectable objects (GH-12951)
regrtest now always detects uncollectable objects. Previously, the
check was only enabled by --findleaks. The check now also works with
-jN/--multiprocess N.
--findleaks becomes a deprecated alias to --fail-env-changed.
(cherry picked from commit 75120d2205)
* bpo-34060: Report system load when running test suite for Windows (GH-8357)
While Windows exposes the system processor queue length, the raw value
used for load calculations on Unix systems, it does not provide an API
to access the averaged value. Hence to calculate the load we must track
and average it ourselves. We can't use multiprocessing or a thread to
read it in the background while the tests run since using those would
conflict with test_multiprocessing and test_xxsubprocess.
Thus, we use Window's asynchronous IO API to run the tracker in the
background with it sampling at the correct rate. When we wish to access
the load we check to see if there's new data on the stream, if there is,
we update our load values.
(cherry picked from commit e16467af0b)
* bpo-36719: Fix regrtest re-run (GH-12964)
Properly handle a test which fail but then pass.
Add test_rerun_success() unit test.
(cherry picked from commit 837acc1957)
* bpo-36719: regrtest closes explicitly WindowsLoadTracker (GH-12965)
Regrtest.finalize() now closes explicitly the WindowsLoadTracker
instance.
(cherry picked from commit 00db7c73af)
A minor typo in the output of `python -m test -h`.
A space was missing in between two words.
howmuch -> how much
(cherry picked from commit 7108aab49c)
Co-authored-by: sth <sth.dev@tejp.de>
* bpo-34605, libregrtest: Rename --slaveargs to --worker-args (GH-9099)
Rename also run_tests_slave() to run_tests_worker().
(cherry picked from commit 012f5b968a)
* bpo-34279, regrtest: Issue a warning if no tests have been executed (GH-10150)
(cherry picked from commit 9724348b43)
* test_regrtest: remove unused threading import
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.
Same change for MB and GB which become MiB and GiB.
Change the output of Tools/iobench/iobench.py.
Round also the size of the documentation from 5.5 MB to 5 MiB.
When running the test suite using --use=all / -u all, exclude tzdata
since it makes test_datetime too slow (15-20 min on some buildbots)
which then times out on some buildbots.
-u tzdata must now be enabled explicitly, -u tzdata or -u all,tzdata,
to run all test_datetime tests.
Fix also regrtest command line parser to allow passing -u
extralargefile to run test_zipfile64.
Travis CI: remove -tzdata. Replace -u all,-tzdata,-cpu with -u all,-cpu since tzdata is now excluded from -u all.
* bpo-30764: regrtest: change exit code on failure
* Exit code 2 if failed tests ("bad")
* Exit code 3 if interrupted
* bpo-30764: regrtest: add --fail-env-changed option
If the option is set, mark a test as failed if it alters the
environment, for example if it creates a file without removing it.
* Add a new option taking a filename to get a list of test names to
filter tests.
* support.match_tests becomes a list.
* Modify run_unittest() to accept to match the whole test identifier,
not just a part of a test identifier.
For example, the following command only runs test_default_timeout()
of the BarrierTests class of test_threading:
$ ./python -m test -v test_threading -m test.test_threading.BarrierTests.test_default_timeout
Remove also some empty lines from test_regrtest.py to make flake8
tool happy.
* regrtest now uses subprocesses when the -j1 command line option
is used: each test file runs in a fresh child process. Before, the -j1 option
was ignored.
* Tools/buildbot/test.bat script now uses -j1 by default to run
each test file in fresh child process.
Slaves (child processes running tests for regrtest -jN) now inherit
--memlimit/-M, --threshold/-t and --nowindows/-n options.
* -M, -t and -n are now supported with -jN
* Factorize code to run tests.
* run_test_in_subprocess() now pass the whole "ns" namespace to the child
process.
Start to split regrtest.py into smaller parts with the creation of
Lib/test/libregrtest/cmdline.py: code to handle the command line, especially
parsing command line arguments. This part of the code is tested by
test_regrtest.
Victor Stinner
Miss Islington (bot)
Victor Stinner
mlouielu
Xiang Zhang
Alexander Belopolsky
Serhiy Storchaka
Steve Dower
Brett Cannon