* bpo-36402: Fix threading._shutdown() race condition (GH-13948)
Fix a race condition at Python shutdown when waiting for threads.
Wait until the Python thread state of all non-daemon threads get
deleted (join all non-daemon threads), rather than just wait until
Python threads complete.
* Add threading._shutdown_locks: set of Thread._tstate_lock locks
of non-daemon threads used by _shutdown() to wait until all Python
thread states get deleted. See Thread._set_tstate_lock().
* Add also threading._shutdown_locks_lock to protect access to
threading._shutdown_locks.
* Add test_finalization_shutdown() test.
(cherry picked from commit 468e5fec8a)
* bpo-36402: Fix threading.Thread._stop() (GH-14047)
Remove the _tstate_lock from _shutdown_locks, don't remove None.
(cherry picked from commit 6f75c87375)
Add a new threading.excepthook() function which handles uncaught
Thread.run() exception. It can be overridden to control how uncaught
exceptions are handled.
threading.ExceptHookArgs is not documented on purpose: it should not
be used directly.
* threading.excepthook() and threading.ExceptHookArgs.
* Add _PyErr_Display(): similar to PyErr_Display(), but accept a
'file' parameter.
* Add _thread._excepthook(): C implementation of the exception hook
calling _PyErr_Display().
* Add _thread._ExceptHookArgs: structseq type.
* Add threading._invoke_excepthook_wrapper() which handles the gory
details to ensure that everything remains alive during Python
shutdown.
* Add unit tests.
* bpo-6532: Make the thread id an unsigned integer.
From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".
* Restore a check in thread_get_ident().
threading.Lock.acquire(), threading.RLock.acquire() and socket operations now
use a monotonic clock, instead of the system clock, when a timeout is used.
Due to recent changes, a Thread doesn't know that it's over before
someone calls .join() or .is_alive(). That meant repr(Thread)
continued to include "started" (and not "stopped") before one of
those methods was called, even if hours passed since the thread
ended.
Repaired that.
Antoine Pitrou found a variation that worked for him on the
thread+fork tests, and added an important
self._is_stopped = True
to the after-fork code. I confess I don't know why things passed
before. But then mixing fork with threads is insane ;-)
The fix for issue 18808 left us checking two things to be sure a Thread
was done: an Event (._stopped) and a mutex (._tstate_lock). Clumsy &
brittle. This patch removes the Event, leaving just a happy lock :-)
The bulk of the patch removes two excruciating tests, which were
verifying sanity of the internals of the ._stopped Event after a fork.
Thanks to Antoine Pitrou for verifying that's the only real value
these tests had.
One consequence of moving from an Event to a mutex: waiters (threads
calling Thread.join()) used to block each on their own unique mutex
(internal to the ._stopped event), but now all contend on the same
mutex (._tstate_lock). These approaches have different performance
characteristics on different platforms. I don't think it matters in
this context.
Victor Stinner
Jake Tesler
Carl Bordum Hansen
Dong-hee Na
Skip Montanaro
Antoine Pitrou
Gregory P. Smith
Serhiy Storchaka
Xiang Zhang
Martin Panter
Zachary Ware
Benjamin Peterson
R David Murray
Victor Stinner
Antoine Pitrou
Raymond Hettinger
Tim Peters
Georg Brandl