Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity
cpython/Python
History
mpage 33da0e844c
gh-114271: Fix race in `Thread.join()` (#114839) 
There is a race between when `Thread._tstate_lock` is released[^1] in `Thread._wait_for_tstate_lock()`
and when `Thread._stop()` asserts[^2] that it is unlocked. Consider the following execution
involving threads A, B, and C:

1. A starts.
2. B joins A, blocking on its `_tstate_lock`.
3. C joins A, blocking on its `_tstate_lock`.
4. A finishes and releases its `_tstate_lock`.
5. B acquires A's `_tstate_lock` in `_wait_for_tstate_lock()`, releases it, but is swapped
   out before calling `_stop()`.
6. C is scheduled, acquires A's `_tstate_lock` in `_wait_for_tstate_lock()` but is swapped
   out before releasing it.
7. B is scheduled, calls `_stop()`, which asserts that A's `_tstate_lock` is not held.
   However, C holds it, so the assertion fails.

The race can be reproduced[^3] by inserting sleeps at the appropriate points in
the threading code. To do so, run the `repro_join_race.py` from the linked repo.

There are two main parts to this PR:

1. `_tstate_lock` is replaced with an event that is attached to `PyThreadState`.
   The event is set by the runtime prior to the thread being cleared (in the same
   place that `_tstate_lock` was released). `Thread.join()` blocks waiting for the
   event to be set.
2. `_PyInterpreterState_WaitForThreads()` provides the ability to wait for all
   non-daemon threads to exit. To do so, an `is_daemon` predicate was added to
   `PyThreadState`. This field is set each time a thread is created. `threading._shutdown()`
   now calls into `_PyInterpreterState_WaitForThreads()` instead of waiting on
   `_tstate_lock`s.

[^1]: 441affc9e7/Lib/threading.py (L1201)
[^2]: 441affc9e7/Lib/threading.py (L1115)
[^3]: 8194653279

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
 		2024-03-16 13:56:30 +01:00
..
clinic GH-114695: Add `sys._clear_internal_caches` (GH-115152) 2024-02-12 09:04:36 +00:00
deepfreeze
…
frozen_modules
…
Python-ast.c gh-116437: Use new C API PyDict_Pop() to simplify the code (GH-116438) 2024-03-07 11:21:08 +02:00
Python-tokenize.c gh-112943: Correctly compute end offsets for multiline tokens in the tokenize module (#112949) 2023-12-11 11:44:22 +00:00
README
…
_warnings.c gh-111789: Use PyDict_GetItemRef() in Python/_warnings.c (gh-112080) 2023-11-27 18:58:43 +01:00
adaptive.md
…
asdl.c
…
asm_trampoline.S
…
assemble.c GH-111485: Generate instruction and uop metadata (GH-113287) 2023-12-20 14:27:25 +00:00
ast.c GH-113655: Lower the C recursion limit on various platforms (GH-113944) 2024-01-16 09:32:01 +00:00
ast_opt.c GH-113655: Lower the C recursion limit on various platforms (GH-113944) 2024-01-16 09:32:01 +00:00
ast_unparse.c gh-100227: Move _str_replace_inf to PyInterpreterState (gh-102333) 2023-02-28 14:16:39 -07:00
bltinmodule.c gh-116437: Use new C API PyDict_Pop() to simplify the code (GH-116438) 2024-03-07 11:21:08 +02:00
bootstrap_hash.c gh-110014: Fix bootstrap_hash.c: remove debug code (#110161) 2023-09-30 22:21:20 +00:00
brc.c gh-112175: Add `eval_breaker` to `PyThreadState` (#115194) 2024-02-20 09:57:48 -05:00
bytecodes.c GH-115802: Reduce the size of _INIT_CALL_PY_EXACT_ARGS. (GH-116856) 2024-03-15 17:16:30 +00:00
ceval.c gh-116098: Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)" (GH-116178) 2024-03-01 07:46:33 +01:00
ceval_gil.c gh-116167: Allow disabling the GIL with `PYTHON_GIL=0` or `-X gil=0` (#116338) 2024-03-11 11:02:58 -04:00
ceval_macros.h gh-116098: Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)" (GH-116178) 2024-03-01 07:46:33 +01:00
codecs.c gh-111972: Make Unicode name C APIcapsule initialization thread-safe (#112249) 2023-11-30 11:12:49 +01:00
compile.c gh-111789: Use PyDict_GetItemRef() in Python/compile.c (GH-112083) 2024-02-23 12:35:27 +01:00
condvar.h gh-104530: Enable native Win32 condition variables by default (GH-104531) 2024-02-02 13:50:51 +00:00
context.c gh-111968: Split _Py_dictkeys_freelist out of _Py_dict_freelist (gh-115505) 2024-02-16 01:01:36 +00:00
critical_section.c gh-111569: Implement Python critical section API (gh-111571) 2023-11-08 15:39:29 -07:00
crossinterp.c gh-111696, PEP 737: Add PyType_GetModuleName() function (#116824) 2024-03-14 18:17:43 +00:00
crossinterp_data_lookup.h gh-76785: Improved Subinterpreters Compatibility with 3.12 (gh-115424) 2024-02-13 14:56:49 -07:00
crossinterp_exceptions.h gh-76785: Improved Subinterpreters Compatibility with 3.12 (gh-115424) 2024-02-13 14:56:49 -07:00
dtoa.c gh-111962: Make dtoa thread-safe in `--disable-gil` builds. (#112049) 2023-12-07 13:47:55 +00:00
dup2.c gh-108765: Python.h no longer includes <unistd.h> (#108783) 2023-09-02 16:50:18 +02:00
dynamic_annotations.c
…
dynload_hpux.c gh-88402: Add new sysconfig variables on Windows (GH-110049) 2023-10-04 22:50:29 +00:00
dynload_shlib.c gh-88402: Add new sysconfig variables on Windows (GH-110049) 2023-10-04 22:50:29 +00:00
dynload_stub.c gh-88402: Add new sysconfig variables on Windows (GH-110049) 2023-10-04 22:50:29 +00:00
dynload_win.c gh-88402: Add new sysconfig variables on Windows (GH-110049) 2023-10-04 22:50:29 +00:00
emscripten_signal.c GH-108614: Unbreak emscripten build (GH-109132) 2023-09-08 17:54:45 +01:00
emscripten_trampoline.c gh-106213: Make Emscripten trampolines work with JSPI (GH-106219) 2023-09-15 15:04:21 -07:00
errors.c gh-111375: Use `NULL` rather than `None` in the exception stack to indicate that an exception was handled (#113302) 2023-12-21 01:46:41 +00:00
executor_cases.c.h GH-115802: Reduce the size of _INIT_CALL_PY_EXACT_ARGS. (GH-116856) 2024-03-15 17:16:30 +00:00
fileutils.c gh-112970: Detect and use closefrom() when available (#112969) 2023-12-12 11:25:27 +01:00
flowgraph.c gh-115796: fix exception table construction in _testinternalcapi.assemble_code_object (#115797) 2024-02-22 12:36:44 +00:00
formatter_unicode.c gh-92536: Remove PyUnicode_READY() calls (#105210) 2023-06-02 01:33:17 +02:00
frame.c GH-116098: Remove dead frame object creation code (GH-116687) 2024-03-12 23:35:28 +00:00
frozen.c gh-106560: Fix redundant declarations in Python/frozen.c (#112612) 2023-12-03 12:18:24 +01:00
frozenmain.c gh-110014: Include explicitly <unistd.h> header (#110155) 2023-09-30 20:06:45 +00:00
future.c gh-106320: Remove more private _PyUnicode C API functions (#106382) 2023-07-03 22:35:46 +00:00
gc.c gh-116604: Correctly honor the gc status when calling _Py_RunGC (#116628) 2024-03-12 12:00:49 +00:00
gc_free_threading.c gh-116604: Check for `gcstate->enabled` in _Py_RunGC in free-threaded build (#116663) 2024-03-12 17:12:02 +00:00
gc_gil.c gh-111968: Rename freelist related struct names to Eric's suggestion (gh-115329) 2024-02-14 00:32:51 +00:00
generated_cases.c.h GH-115802: Reduce the size of _INIT_CALL_PY_EXACT_ARGS. (GH-116856) 2024-03-15 17:16:30 +00:00
getargs.c gh-116447: Fix possible UB in `arraymodule` and `getargs` (#116459) 2024-03-08 13:49:52 +03:00
getcompiler.c
…
getcopyright.c Update copyright years to 2024. (GH-113608) 2024-01-16 21:54:05 +01:00
getopt.c gh-110079: Remove extern "C" { ...} in C code (#110080) 2023-09-29 10:56:49 +02:00
getplatform.c
…
getversion.c
…
hamt.c gh-106320: Remove private PyLong C API functions (#108429) 2023-08-24 18:53:50 +02:00
hashtable.c gh-111545: Add Py_HashPointer() function (#112096) 2023-12-06 15:09:22 +01:00
import.c gh-110850: Rename internal PyTime C API functions (#115734) 2024-02-20 22:16:37 +00:00
importdl.c gh-88402: Add new sysconfig variables on Windows (GH-110049) 2023-10-04 22:50:29 +00:00
initconfig.c gh-90300: Fix undocumented envvars in the Python CLI help (GH-116765) 2024-03-14 12:09:19 +02:00
instrumentation.c gh-115832: Fix instrumentation version mismatch during interpreter shutdown (#115856) 2024-03-04 11:29:39 -05:00
intrinsics.c gh-107149: make new opcode util functions private rather than public and unstable (#112042) 2023-11-14 00:31:02 +00:00
jit.c GH-116134: JIT aarch64-pc-windows-msvc (GH-116130) 2024-03-04 10:16:56 -08:00
legacy_tracing.c gh-103615: Use local events for opcode tracing (GH-109472) 2023-11-03 16:39:50 +00:00
lock.c gh-114271: Fix race in `Thread.join()` (#114839) 2024-03-16 13:56:30 +01:00
marshal.c gh-113626: Add allow_code parameter in marshal functions (GH-113648) 2024-01-16 18:05:15 +02:00
modsupport.c gh-84489: Properly handle trailing spaces in Py_BuildValue() format strings (GH-21158) 2023-10-11 14:44:06 +03:00
mysnprintf.c
…
mystrtoul.c gh-108765: Python.h no longer includes <ctype.h> (#108831) 2023-09-03 18:54:27 +02:00
object_stack.c gh-111968: Split _Py_dictkeys_freelist out of _Py_dict_freelist (gh-115505) 2024-02-16 01:01:36 +00:00
opcode_targets.h gh-116381: Remove bad specializations, add fail stats (GH-116464) 2024-03-08 00:21:21 +08:00
optimizer.c gh-116760: Fix pystats for trace attempts (GH-116761) 2024-03-13 22:13:33 +00:00
optimizer_analysis.c GH-116596: Better determination of escaping uops. (GH-116597) 2024-03-11 13:37:48 +00:00
optimizer_bytecodes.c gh-115419: Change default sym to not_null (GH-116562) 2024-03-13 20:57:48 +08:00
optimizer_cases.c.h GH-116422: Factor out eval breaker checks at end of calls into its own micro-op. (GH-116817) 2024-03-14 16:31:47 +00:00
optimizer_symbols.c GH-115819: Eliminate Boolean guards when value is known (GH-116355) 2024-03-05 15:06:00 +00:00
parking_lot.c gh-108724: Fix _PySemaphore_Wait call during thread deletion (#116483) 2024-03-08 15:26:36 -05:00
pathconfig.c gh-110079: Remove extern "C" { ...} in C code (#110080) 2023-09-29 10:56:49 +02:00
perf_trampoline.c gh-113343: Fix error check on mmap(2) (#113342) 2023-12-21 19:28:55 +00:00
preconfig.c gh-106320: Remove private pylifecycle.h functions (#106400) 2023-07-04 09:41:43 +00:00
pyarena.c
…
pyctype.c
…
pyfpe.c
…
pyhash.c gh-111545: Add Py_HashPointer() function (#112096) 2023-12-06 15:09:22 +01:00
pylifecycle.c gh-115168: Add pystats counter for invalidated executors (GH-115169) 2024-02-26 17:51:47 +00:00
pymath.c
…
pystate.c gh-114271: Fix race in `Thread.join()` (#114839) 2024-03-16 13:56:30 +01:00
pystrcmp.c gh-108767: Replace ctype.h functions with pyctype.h functions (#108772) 2023-09-01 18:36:53 +02:00
pystrhex.c gh-108765: pystrhex: Replace stdlib.h abs() with Py_ABS() (#108830) 2023-09-02 23:15:54 +02:00
pystrtod.c gh-108765: Cleanup #include in Python/*.c files (#108977) 2023-09-06 15:56:08 +02:00
pythonrun.c gh-116437: Use new C API PyDict_Pop() to simplify the code (GH-116438) 2024-03-07 11:21:08 +02:00
pytime.c gh-88494: Use QueryPerformanceCounter() for time.monotonic() (#116781) 2024-03-14 16:42:41 +01:00
qsbr.c gh-115103: Fix unregistering of QSBR state (#116480) 2024-03-08 12:39:53 -05:00
specialize.c gh-116381: Remove bad specializations, add fail stats (GH-116464) 2024-03-08 00:21:21 +08:00
stdlib_module_names.h gh-110721: Remove unused code from suggestions.c after moving PyErr_Display to use the traceback module (#113712) 2024-01-08 15:10:45 +00:00
structmember.c gh-115011: Improve support of __index__() in setters of members with unsigned integer type (GH-115029) 2024-02-11 12:45:58 +02:00
suggestions.c gh-113845: Fix a compiler warning in Python/suggestions.c (GH-113949) 2024-01-11 20:31:24 +02:00
symtable.c gh-96497: Mangle name before symtable lookup in 'symtable_extend_namedexpr_scope' (GH-96561) 2024-02-17 12:06:31 +00:00
sysmodule.c gh-116167: Allow disabling the GIL with `PYTHON_GIL=0` or `-X gil=0` (#116338) 2024-03-11 11:02:58 -04:00
thread.c gh-110850: Cleanup PyTime API: PyTime_t are nanoseconds (#115753) 2024-02-21 11:46:00 +01:00
thread_nt.h gh-116012: Preserve GetLastError() across calls to TlsGetValue on Windows (GH-116014) 2024-02-28 13:58:25 +00:00
thread_pthread.h gh-112536: Add TSAN builds on Github Actions (#116872) 2024-03-16 11:10:37 +01:00
thread_pthread_stubs.h GH-110829: Ensure Thread.join() joins the OS thread (#110848) 2023-11-04 13:59:24 +00:00
tier2_engine.md Docs: fix spelling of the word 'transferring' (#116641) 2024-03-13 23:53:32 +01:00
traceback.c GH-113655: Lower the C recursion limit on various platforms (GH-113944) 2024-01-16 09:32:01 +00:00
tracemalloc.c gh-108765: Cleanup #include in Python/*.c files (#108977) 2023-09-06 15:56:08 +02:00
vm-state.md High level docs for the VM state (#111621) 2023-11-01 22:55:10 +00:00

README 

Miscellaneous source files for the main Python shared library