Fix a race condition at Python shutdown when waiting for threads.
Wait until the Python thread state of all non-daemon threads get
deleted (join all non-daemon threads), rather than just wait until
Python threads complete.
* Add threading._shutdown_locks: set of Thread._tstate_lock locks
of non-daemon threads used by _shutdown() to wait until all Python
thread states get deleted. See Thread._set_tstate_lock().
* Add also threading._shutdown_locks_lock to protect access to
threading._shutdown_locks.
* Add test_finalization_shutdown() test.
Add a new threading.excepthook() function which handles uncaught
Thread.run() exception. It can be overridden to control how uncaught
exceptions are handled.
threading.ExceptHookArgs is not documented on purpose: it should not
be used directly.
* threading.excepthook() and threading.ExceptHookArgs.
* Add _PyErr_Display(): similar to PyErr_Display(), but accept a
'file' parameter.
* Add _thread._excepthook(): C implementation of the exception hook
calling _PyErr_Display().
* Add _thread._ExceptHookArgs: structseq type.
* Add threading._invoke_excepthook_wrapper() which handles the gory
details to ensure that everything remains alive during Python
shutdown.
* Add unit tests.
* bpo-6532: Make the thread id an unsigned integer.
From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".
* Restore a check in thread_get_ident().
threading.Lock.acquire(), threading.RLock.acquire() and socket operations now
use a monotonic clock, instead of the system clock, when a timeout is used.
Due to recent changes, a Thread doesn't know that it's over before
someone calls .join() or .is_alive(). That meant repr(Thread)
continued to include "started" (and not "stopped") before one of
those methods was called, even if hours passed since the thread
ended.
Repaired that.
Antoine Pitrou found a variation that worked for him on the
thread+fork tests, and added an important
self._is_stopped = True
to the after-fork code. I confess I don't know why things passed
before. But then mixing fork with threads is insane ;-)
The fix for issue 18808 left us checking two things to be sure a Thread
was done: an Event (._stopped) and a mutex (._tstate_lock). Clumsy &
brittle. This patch removes the Event, leaving just a happy lock :-)
The bulk of the patch removes two excruciating tests, which were
verifying sanity of the internals of the ._stopped Event after a fork.
Thanks to Antoine Pitrou for verifying that's the only real value
these tests had.
One consequence of moving from an Event to a mutex: waiters (threads
calling Thread.join()) used to block each on their own unique mutex
(internal to the ._stopped event), but now all contend on the same
mutex (._tstate_lock). These approaches have different performance
characteristics on different platforms. I don't think it matters in
this context.
test_is_alive_after_fork is failing on some old Linux kernels, but
passing on all newer ones. Since virtually anything can go wrong
with locks when mixing threads with fork, replace the most likely
cause with a redundant simple data member.
* Rename time.steady() to time.monotonic()
* On Windows, time.monotonic() uses GetTickCount/GetTickCount64() instead of
QueryPerformanceCounter()
* time.monotonic() uses CLOCK_HIGHRES if available
* Add time.get_clock_info(), time.perf_counter() and time.process_time()
functions
The public names (Thread, Condition, etc.) used to be factory functions
returning instances of hidden classes (_Thread, _Condition, etc.),
because (if Guido recalls correctly) this code pre-dates the ability to
subclass extension types.
It is now possible to inherit from Thread and other classes, without
having to import the private underscored names like multiprocessing did.
A doc update will follow: a patch is under discussion on the issue.
thread implementation.
Skip test_lock_acquire_interruption() and test_rlock_acquire_interruption() of
test_threadsignals if a thread lock is implemented using a POSIX mutex and a
POSIX condition variable. A POSIX condition variable cannot be interrupted by a
signal (e.g. on Linux, the futex system call is restarted).
and multiprocessing.Process constructors in order to override the
default behaviour of inheriting the daemonic property from the current
thread/process.
deletes its _block attribute, deal with that. This prevents an uncaught
exception in a thread during test_thread.
This refactors a bit to better match what I did in the r87727 backport to 2.7.
needed to be reinitialized after fork(). Adds tests to confirm that they are
and that a potential deadlock and crasher bug are fixed (platform dependant).
Victor Stinner
Jake Tesler
Carl Bordum Hansen
Dong-hee Na
Skip Montanaro
Antoine Pitrou
Gregory P. Smith
Serhiy Storchaka
Xiang Zhang
Martin Panter
Zachary Ware
Benjamin Peterson
R David Murray
Victor Stinner
Antoine Pitrou
Raymond Hettinger
Tim Peters
Georg Brandl
Andrew Svetlov
Charles-François Natali
Ezio Melotti
Brett Cannon
Charles-François Natali
Éric Araujo
Victor Stinner
Gregory P. Smith
Kristján Valur Jónsson
Brett Cannon