* bpo-36402: Fix threading._shutdown() race condition (GH-13948)
Fix a race condition at Python shutdown when waiting for threads.
Wait until the Python thread state of all non-daemon threads get
deleted (join all non-daemon threads), rather than just wait until
Python threads complete.
* Add threading._shutdown_locks: set of Thread._tstate_lock locks
of non-daemon threads used by _shutdown() to wait until all Python
thread states get deleted. See Thread._set_tstate_lock().
* Add also threading._shutdown_locks_lock to protect access to
threading._shutdown_locks.
* Add test_finalization_shutdown() test.
(cherry picked from commit 468e5fec8a)
* bpo-36402: Fix threading.Thread._stop() (GH-14047)
Remove the _tstate_lock from _shutdown_locks, don't remove None.
(cherry picked from commit 6f75c87375)
(cherry picked from commit e40a97a721)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-6532: Make the thread id an unsigned integer.
From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".
* Restore a check in thread_get_ident().
threading.Lock.acquire(), threading.RLock.acquire() and socket operations now
use a monotonic clock, instead of the system clock, when a timeout is used.
Due to recent changes, a Thread doesn't know that it's over before
someone calls .join() or .is_alive(). That meant repr(Thread)
continued to include "started" (and not "stopped") before one of
those methods was called, even if hours passed since the thread
ended.
Repaired that.
Antoine Pitrou found a variation that worked for him on the
thread+fork tests, and added an important
self._is_stopped = True
to the after-fork code. I confess I don't know why things passed
before. But then mixing fork with threads is insane ;-)
The fix for issue 18808 left us checking two things to be sure a Thread
was done: an Event (._stopped) and a mutex (._tstate_lock). Clumsy &
brittle. This patch removes the Event, leaving just a happy lock :-)
The bulk of the patch removes two excruciating tests, which were
verifying sanity of the internals of the ._stopped Event after a fork.
Thanks to Antoine Pitrou for verifying that's the only real value
these tests had.
One consequence of moving from an Event to a mutex: waiters (threads
calling Thread.join()) used to block each on their own unique mutex
(internal to the ._stopped event), but now all contend on the same
mutex (._tstate_lock). These approaches have different performance
characteristics on different platforms. I don't think it matters in
this context.
test_is_alive_after_fork is failing on some old Linux kernels, but
passing on all newer ones. Since virtually anything can go wrong
with locks when mixing threads with fork, replace the most likely
cause with a redundant simple data member.
Miss Islington (bot)
Dong-hee Na
Antoine Pitrou
Gregory P. Smith
Serhiy Storchaka
Xiang Zhang
Martin Panter
Zachary Ware
Benjamin Peterson
R David Murray
Victor Stinner
Antoine Pitrou
Raymond Hettinger
Tim Peters
Georg Brandl
Andrew Svetlov