cpython

History

Joshua Herman 9e011e7c77 gh-82054: allow test runner to split test_asyncio to execute in parallel by sharding. (#103927 ) This runs test_asyncio sub-tests in parallel using sharding from Cinder. This suite is typically the longest-pole in runs because it is a test package with a lot of further sub-tests otherwise run serially. By breaking out the sub-tests as independent modules we can run a lot more in parallel. After porting we can see the direct impact on a multicore system. Without this change: Running make test is 5 min 26 seconds With this change: Running make test takes 3 min 39 seconds That'll vary based on system and parallelism. On a `-j 4` run similar to what CI and buildbot systems often do, it reduced the overall test suite completion latency by 10%. The drawbacks are that this implementation is hacky and due to the sorting of the tests it obscures when the asyncio tests occur and involves changing CPython test infrastructure but, the wall time saved it is worth it, especially in low-core count CI runs as it pulls a long tail. The win for productivity and reserved CI resource usage is significant. Future tests that deserve to be refactored into split up suites to benefit from are test_concurrent_futures and the way the _test_multiprocessing suite gets run for all start methods. As exposed by passing the -o flag to python -m test to get a list of the 10 longest running tests. --------- Co-authored-by: Carl Meyer <carl@oddbird.net> Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google, LLC]		2023-04-29 18:26:24 -07:00
..
__phello__	…
asyncio	gh-83925: Make asyncio.subprocess communicate similar to non-asyncio (#18650 )	2023-04-27 17:30:26 -07:00
collections	gh-103092: Isolate `_collections` (#103093 )	2023-04-12 18:21:28 +05:30
concurrent	gh-94440: Fix issue of ProcessPoolExecutor shutdown hanging (#94468 )	2023-03-15 21:44:52 -07:00
ctypes	Fix typo in _swappedbytes_ in ctypes comment (#102773 )	2023-03-25 16:38:24 -07:00
curses	gh-60436: fix curses textbox backspace/del (#103783 )	2023-04-26 22:54:07 +02:00
dbm	…
email	GH-103857: Deprecate utcnow and utcfromtimestamp (#103858 )	2023-04-27 11:32:30 -06:00
encodings	gh-98433: Fix quadratic time idna decoding. (#99092 )	2022-11-07 16:54:41 -08:00
ensurepip	gh-103872: Update bundled pip version to 23.1.2 (gh-103873)	2023-04-26 10:22:25 +00:00
html	gh-100210: Correct the comment link for unescaping HTML (#100212 )	2023-02-19 11:18:12 +01:00
http	GH-103857: Deprecate utcnow and utcfromtimestamp (#103858 )	2023-04-27 11:32:30 -06:00
idlelib	Change 'dependant' to 'dependent' (#103745 )	2023-04-24 09:40:30 -04:00
importlib	gh-87729: specialize LOAD_SUPER_ATTR_METHOD (#103809 )	2023-04-25 17:45:51 +00:00
json	gh-96959: Update HTTP links which are redirected to HTTPS (GH-96961)	2022-09-24 14:38:53 +03:00
lib2to3	gh-54781: Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/ (#94049 )	2022-06-21 15:21:22 +02:00
logging	gh-103357: Add logging.Formatter defaults support to logging.config fileConfig and dictConfig (GH-103359)	2023-04-12 08:35:56 +01:00
msilib	gh-91217: deprecate msilib (GH-91515)	2022-04-14 12:50:11 -07:00
multiprocessing	gh-84559: Remove the new multiprocessing warning, too disruptive. (#101551 )	2023-02-03 15:20:46 -08:00
pydoc_data	Python 3.12.0a7	2023-04-04 17:52:42 +02:00
re	gh-91524: Speed up the regular expression substitution (#91525 )	2022-10-23 15:57:30 -07:00
site-packages	…
sqlite3	gh-102628: Fix sqlite3 CLI prompt in IDLE on Windows (#103945 )	2023-04-27 21:22:26 +00:00
test	gh-82054: allow test runner to split test_asyncio to execute in parallel by sharding. (#103927 )	2023-04-29 18:26:24 -07:00
tkinter	gh-103685: Fix tkinter.Menu.index() for Tk 8.7 (#103686 )	2023-04-23 21:31:44 -04:00
tomllib	bpo-40059: Add tomllib (PEP-680) (GH-31498)	2022-03-08 09:26:13 +01:00
turtledemo	gh-91212: Fixed flickering when the tracer is turned off (#95129 )	2022-09-28 23:40:51 -07:00
unittest	gh-62432: unittest runner: Exit code 5 if no tests were run (#102051 )	2023-04-27 01:28:46 +00:00
urllib	gh-81403: Fix for CacheFTPHandler in urllib (#13951 )	2023-04-22 21:41:23 -07:00
venv	gh-95299: Stop installing setuptools as a part of ensurepip and venv (#101039 )	2023-04-17 23:43:34 -05:00
wsgiref	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
xml	gh-83122: Deprecate testing element truth values in `ElementTree` (#31149 )	2023-01-22 17:16:48 -08:00
xmlrpc	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
zipfile	gh-86094: Add support for Unicode Path Extra Field in ZipFile (gh-102566)	2023-04-05 20:54:48 +09:00
zoneinfo	Fix typos in documentation and comments (GH-102374)	2023-03-02 07:26:49 -08:00
__future__.py	gh-93626: Set the release for `__future__.annotations` to `None` (GH-93628)	2022-07-05 10:46:39 +02:00
__hello__.py	bpo-47084: Clear Unicode cached representations on finalization (GH-32032)	2022-03-22 13:53:51 +01:00
_aix_support.py	gh-96305: Fix AIX build by avoiding subprocess during bootstrap (#96429 )	2023-02-02 12:30:49 -08:00
_collections_abc.py	gh-102721: Improve coverage of `_collections_abc._CallableGenericAlias` (#102722 )	2023-03-16 14:47:30 +00:00
_compat_pickle.py	bpo-46565: `del` loop vars that are leaking into module namespaces (GH-30993)	2022-02-03 11:20:08 +02:00
_compression.py	…
_markupbase.py	…
_osx_support.py	…
_py_abc.py	…
_pydecimal.py	gh-86682: Adds sys._getframemodulename as an alternative to using _getframe (GH-99520)	2023-01-13 11:31:06 +00:00
_pyio.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
_pylong.py	gh-102515: Remove unused imports in the `Lib/` directory (#102516 )	2023-03-08 11:45:38 +00:00
_sitebuiltins.py	…
_strptime.py	GH-90750: Use datetime.fromisocalendar in _strptime (#103802 )	2023-04-27 10:27:27 -06:00
_threading_local.py	…
_weakrefset.py	bpo-26579: Add object.__getstate__(). (GH-2821)	2022-04-06 20:00:14 +03:00
abc.py	gh-87864: Use correct function definition syntax in the docs (#103312 )	2023-04-11 16:50:25 +03:00
aifc.py	gh-47061: Deprecate `chunk` (GH-91419)	2022-04-11 15:02:41 -07:00
antigravity.py	…
argparse.py	gh-101979: argparse: fix a bug where parentheses in metavar argument of add_argument() were dropped (#102318 )	2023-03-05 06:54:33 -08:00
ast.py	gh-103285: Rewrite _splitlines_no_ff to improve performance (#103307 )	2023-04-23 23:03:49 -06:00
base64.py	GH-102456: Fix docstring and getopt options for base64 (gh-102457)	2023-04-02 14:05:50 +09:00
bdb.py	gh-101517: fix line number propagation in code generated for except* (#103550 )	2023-04-24 21:58:51 +01:00
bisect.py	GH-102833: Mention the key function in the docstrings (GH-103009)	2023-03-25 02:19:20 -05:00
bz2.py	…
cProfile.py	gh-103935: Use `io.open_code()` when executing code in trace and profile modules (GH-103947)	2023-04-27 20:29:35 +00:00
calendar.py	gh-103636: issue warning for deprecated calendar constants (#103833 )	2023-04-29 01:16:46 -06:00
cgi.py	bpo-47061: deprecate cgi and cgitb (GH-32410)	2022-04-08 17:15:35 -07:00
cgitb.py	bpo-44712: Replace "type(literal)" with corresponding builtin types (GH-27294)	2022-05-08 17:10:11 +03:00
chunk.py	gh-47061: Deprecate `chunk` (GH-91419)	2022-04-11 15:02:41 -07:00
cmd.py	gh-67248: cmd: Sort miscellaneous help topics (#92254 )	2022-05-03 21:36:52 -06:00
code.py	gh-102778: Add sys.last_exc, deprecate sys.last_type, sys.last_value,sys.last_traceback (#102779 )	2023-03-18 11:47:11 +00:00
codecs.py	gh-51511: Note that codecs.open()'s encoding parameter affects automatic conversion to binary mode (#94370 )	2022-10-21 16:01:05 -07:00
codeop.py	gh-96052: codeop: fix handling compiler warnings in incomplete input (GH-96132)	2022-09-16 17:37:30 +03:00
colorsys.py	…
compileall.py	GH-84559: Deprecate fork being the multiprocessing default. (#100618 )	2023-02-02 15:50:35 -08:00
configparser.py	gh-100520: Fix `rst` markup in `configparser` docstrings (#100524 )	2022-12-26 17:08:49 +05:30
contextlib.py	gh-103791: Make contextlib.suppress also act on exceptions within an ExceptionGroup (#103792 )	2023-04-24 22:17:02 +00:00
contextvars.py	…
copy.py	gh-100815: Normalize `types` module usage in `copy` module (#100816 )	2023-01-07 21:29:53 +00:00
copyreg.py	gh-99325: Remove unused `NameError` handling (#99326 )	2022-11-11 09:56:57 +00:00
crypt.py	gh-95231: Disable md5 & crypt modules if FIPS is enabled (GH-94742)	2022-08-15 07:48:07 -07:00
csv.py	gh-67230: add quoting rules to csv module (GH-29469)	2023-04-12 15:32:30 -07:00
dataclasses.py	gh-103449: Fix a bug in dataclass docstring generation (#103454 )	2023-04-17 17:33:22 -06:00
datetime.py	GH-103857: Deprecate utcnow and utcfromtimestamp (#103858 )	2023-04-27 11:32:30 -06:00
decimal.py	…
difflib.py	Correct method name typo (#91970 )	2022-04-27 15:28:56 -06:00
dis.py	GH-99944: Make dis display the value of oparg of KW_NAMES (#103856 )	2023-04-26 19:00:36 +01:00
doctest.py	gh-86682: Adds sys._getframemodulename as an alternative to using _getframe (GH-99520)	2023-01-13 11:31:06 +00:00
enum.py	gh-103596: [Enum] do not shadow mixed-in methods/attributes (GH-103600)	2023-04-18 16:19:23 -07:00
filecmp.py	gh-93991: Use boolean instead of 0/1 for condition check (GH-93992)	2022-06-19 07:12:59 -07:00
fileinput.py	gh-101961 fileinput.hookcompressed should not set the encoding value for the binary mode (gh-102068)	2023-02-21 12:10:29 +09:00
fnmatch.py	gh-89973: Fix re.error in the fnmatch module. (GH-93072)	2022-06-05 11:46:29 +03:00
fractions.py	gh-101825: Clarify that as_integer_ratio() output is always normalized (#101843 )	2023-02-27 19:11:28 +00:00
ftplib.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
functools.py	gh-87634: remove locking from functools.cached_property (GH-101890)	2023-02-22 17:49:22 -08:00
genericpath.py	gh-101196: Make isdir/isfile/exists faster on Windows (GH-101324)	2023-02-08 14:34:24 +00:00
getopt.py	bpo-44712: Replace "type(literal)" with corresponding builtin types (GH-27294)	2022-05-08 17:10:11 +03:00
getpass.py	…
gettext.py	…
glob.py	…
graphlib.py	…
gzip.py	gh-95534: Improve gzip reading speed by 10% (#97664 )	2022-10-16 19:10:58 -07:00
hashlib.py	gh-99108: Refactor _sha256 & _sha512 into _sha2. (#101924 )	2023-02-15 22:08:20 -08:00
heapq.py	Update: usage doc for heappushpop (GH-91451)	2022-04-17 23:12:33 -05:00
hmac.py	…
imaplib.py	gh-94172: Remove keyfile, certfile and check_hostname parameters (#94173 )	2022-11-03 18:32:25 +01:00
imghdr.py	gh-85455: Add missing doc strings and improve docs (#21573 )	2022-10-15 09:31:06 -04:00
inspect.py	gh-103556: [inspect.Signature] disallow pos-or-kw params without default after pos-only with default (#103557 )	2023-04-21 23:45:10 -06:00
io.py	gh-94169: Remove deprecated io.OpenWrapper (#94170 )	2022-06-24 08:46:53 +02:00
ipaddress.py	ipaddress: Remove non-existent ip_str param from docstring (#103461 )	2023-04-14 00:44:10 -07:00
keyword.py	…
linecache.py	gh-92336: linecache.getline should not raise exceptions on decoding errors (GH-94410)	2022-06-30 10:18:18 +01:00
locale.py	gh-74940: Allow fallback to UTF-8 encoding on systems with no locales installed (GH-14925)	2023-04-26 15:33:09 +09:00
lzma.py	…
mailbox.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
mailcap.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
mimetypes.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
modulefinder.py	…
netrc.py	netrc: Remove unused "import shlex" (#93311 )	2022-06-03 20:14:58 -07:00
nntplib.py	gh-91217: deprecate nntplib (GH-91543)	2022-04-15 12:32:56 -07:00
ntpath.py	GH-103220: Fix `ntpath.join()` of partial UNC drive with trailing slash (GH-103221)	2023-04-11 17:26:45 +01:00
nturl2path.py	…
numbers.py	gh-68163: Correct conversion of Rational instances to float (GH-25619)	2022-09-04 13:15:59 +01:00
opcode.py	gh-87729: specialize LOAD_SUPER_ATTR_METHOD (#103809 )	2023-04-25 17:45:51 +00:00
operator.py	…
optparse.py	…
os.py	gh-89727: Improve os.walk complexity (#100671 )	2023-01-02 13:41:19 -08:00
pathlib.py	GH-78079: Fix UNC device path root normalization in pathlib (GH-102003)	2023-04-14 21:55:41 +01:00
pdb.py	gh-103578: Fix pdb reading code with non-utf8 encoding (#103581 )	2023-04-25 23:04:51 -06:00
pickle.py	gh-102799: use exception instance instead of sys.exc_info() (#102885 )	2023-03-31 11:23:02 +01:00
pickletools.py	…
pipes.py	gh-91217: deprecate-pipes (GH-91779)	2022-04-21 19:28:34 -07:00
pkgutil.py	gh-98040: Remove just the `imp` module (#98573 )	2023-04-28 16:17:58 -07:00
platform.py	gh-102491: Remove IronPython version check in sys_version (#102492 )	2023-03-19 15:52:47 -07:00
plistlib.py	gh-101992: update pstlib module documentation (#102133 )	2023-03-04 13:35:25 -08:00
poplib.py	gh-94172: Remove keyfile, certfile and check_hostname parameters (#94173 )	2022-11-03 18:32:25 +01:00
posixpath.py	gh-101196: Make isdir/isfile/exists faster on Windows (GH-101324)	2023-02-08 14:34:24 +00:00
pprint.py	gh-92546: Move pprint benchmark into pyperformance (GH-94613)	2022-07-25 11:30:13 -07:00
profile.py	gh-103935: Use `io.open_code()` when executing code in trace and profile modules (GH-103947)	2023-04-27 20:29:35 +00:00
pstats.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
pty.py	gh-85984: Remove legacy Lib/pty.py code. (#92365 )	2023-02-08 17:00:17 -08:00
py_compile.py	…
pyclbr.py	…
pydoc.py	gh-98040: Remove just the `imp` module (#98573 )	2023-04-28 16:17:58 -07:00
queue.py	gh-90879: Fix missing parameter for put_nowait() (GH-91514)	2022-04-14 17:23:57 +09:00
quopri.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
random.py	Move binomialvariate() to a section for discrete distributions (GH-102955)	2023-03-23 12:10:12 -05:00
reprlib.py	gh-92734: Add indentation feature to reprlib.Repr (GH-92735)	2022-09-08 13:51:44 -05:00
rlcompleter.py	gh-92345: Import rlcompleter before sys.path is extended (#92346 )	2022-05-05 21:24:16 +02:00
runpy.py	gh-98040: Remove just the `imp` module (#98573 )	2023-04-28 16:17:58 -07:00
sched.py	…
secrets.py	gh-99305: Speed up secrets.token_hex() ~2x (#99306 )	2022-11-11 14:45:24 -08:00
selectors.py	bpo-46583: remove unused `sys.version_info` check from `selectors` (GH-31023)	2022-02-02 10:15:02 +02:00
shelve.py	…
shlex.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
shutil.py	gh-82814: fix shutil access error on WSL (#103790 )	2023-04-25 00:45:38 +00:00
signal.py	bpo-27718: Fix help for the signal module (GH-30063)	2021-12-13 11:21:55 +02:00
site.py	gh-102038: Skip a sometimes unnecessary stat in site.py (#102039 )	2023-04-02 15:47:31 -07:00
smtplib.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
sndhdr.py	gh-85525: Indicate supported sound header formats (#21575 )	2022-10-15 09:30:05 -04:00
socket.py	Fix broken link to MSDN (#102355 )	2023-03-13 15:15:52 +02:00
socketserver.py	gh-103673: Add missing ForkingUnixStreamServer and ForkingUnixDatagramServer socketservers (#103674 )	2023-04-24 22:35:52 +00:00
sre_compile.py	bpo-47152: Convert the re module into a package (GH-32177)	2022-04-02 11:35:13 +03:00
sre_constants.py	bpo-47152: Convert the re module into a package (GH-32177)	2022-04-02 11:35:13 +03:00
sre_parse.py	bpo-47152: Convert the re module into a package (GH-32177)	2022-04-02 11:35:13 +03:00
ssl.py	gh-94199: Remove the ssl.wrap_socket() function (#94203 )	2022-07-08 15:20:15 +02:00
stat.py	…
statistics.py	GH-102670: Use sumprod() to simplify, speed up, and improve accuracy of statistics functions (GH-102649)	2023-03-13 20:06:43 -05:00
string.py	…
stringprep.py	…
struct.py	…
subprocess.py	gh-101283: Fix use of unbound variable (GH-101712)	2023-02-08 23:38:56 +00:00
sunau.py	bpo-44712: Replace "type(literal)" with corresponding builtin types (GH-27294)	2022-05-08 17:10:11 +03:00
symtable.py	Change list to view object (#93661 )	2022-06-11 11:54:31 +01:00
sysconfig.py	gh-102515: Remove unused imports in the `Lib/` directory (#102516 )	2023-03-08 11:45:38 +00:00
tabnanny.py	gh-63301: Set exit code when tabnanny CLI exits on error (#7699 )	2023-03-04 13:32:13 -08:00
tarfile.py	gh-102950: Implement PEP 706 – Filter for tarfile.extractall (#102953 )	2023-04-24 10:58:06 +02:00
telnetlib.py	gh-91217: deprecate telnetlib (GH-91958)	2022-04-26 10:45:08 -07:00
tempfile.py	gh-51574: Make tempfile.mkdtemp() always return absolute paths (#94612 )	2023-04-25 16:05:59 +00:00
textwrap.py	bpo-46544: Do not leak `x` and `uspace` in textwrap.TextWrapper (GH-30955)	2022-01-27 13:55:58 +02:00
this.py	…
threading.py	gh-100228: Warn from os.fork() if other threads exist. (#100229 )	2022-12-29 14:41:39 -08:00
timeit.py	gh-90085: Remove vestigial -t and -c timeit options (#94941 )	2022-10-07 15:38:20 -07:00
token.py	gh-102856: Initial implementation of PEP 701 (#102855 )	2023-04-19 11:18:16 -05:00
tokenize.py	bpo-46565: `del` loop vars that are leaking into module namespaces (GH-30993)	2022-02-03 11:20:08 +02:00
trace.py	gh-103935: Use `io.open_code()` when executing code in trace and profile modules (GH-103947)	2023-04-27 20:29:35 +00:00
traceback.py	gh-102778: Add sys.last_exc, deprecate sys.last_type, sys.last_value,sys.last_traceback (#102779 )	2023-03-18 11:47:11 +00:00
tracemalloc.py	…
tty.py	…
turtle.py	gh-101100: Fix Sphinx warnings in `turtle` module (#102340 )	2023-03-13 11:24:52 +02:00
types.py	gh-101688: Implement types.get_original_bases (#101827 )	2023-04-23 20:24:30 +01:00
typing.py	GH-103699: Add `__orig_bases__` to various typing classes (#103698 )	2023-04-23 08:33:39 -06:00
uu.py	gh-91217: deprecate uu (GH-92009)	2022-04-27 20:26:33 -07:00
uuid.py	Update MAC_address wikipedia URL in comments (#29019 )	2023-04-26 14:27:03 -06:00
warnings.py	gh-39615: Add warnings.warn() skip_file_prefixes support (#100840 )	2023-01-27 18:35:14 -08:00
wave.py	gh-77171: Fixes SubFormat check to compare the entire value. Add docs (GH-97509)	2022-09-23 16:08:21 +01:00
weakref.py	Remove unnecessary registration of weakref.WeakSet to _collections_abc.Set (GH-32211)	2022-03-31 09:11:35 -05:00
webbrowser.py	bpo-44844: Enable detection of Microsoft Edge browser in webbrowser module (GH-29908)	2023-04-04 17:00:03 +01:00
xdrlib.py	bpo-45975: Simplify some while-loops with walrus operator (GH-29347)	2022-11-26 14:33:25 -08:00
zipapp.py	bpo-46951: Order contents of zipapps (GH-31713)	2022-05-27 19:04:29 +03:00
zipimport.py	GH-87235: Make sure "python /dev/fd/9 9</path/to/script.py" works on macOS (#99768 )	2022-11-27 11:56:14 +01:00