Commit Graph

239 Commits

Author SHA1 Message Date
Sam Gross b6228b521b
gh-115035: Mark ThreadHandles as non-joinable earlier after forking (#115042)
This marks dead ThreadHandles as non-joinable earlier in
`PyOS_AfterFork_Child()` before we execute any Python code. The handles
are stored in a global linked list in `_PyRuntimeState` because `fork()`
affects the entire process.
2024-02-06 14:45:04 -05:00
Daniel Hollas 5e390a0fc8
gh-109653: Speedup import of threading module (#114509)
Avoiding an import of functools leads to 50% speedup of import time.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-01-31 09:29:44 +00:00
Nikita Sobolev d96358ff9d
gh-114315: Make `threading.Lock` a real class, not a factory function (#114479)
`threading.Lock` is now the underlying class and is constructable rather than the old
factory function. This allows for type annotations to refer to it which had no non-ugly
way to be expressed prior to this.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2024-01-25 19:46:32 +00:00
Fabio Zadrozny 5a1ecc8cc7
gh-114423: Remove DummyThread from threading._active when thread dies (GH-114424) 2024-01-23 14:12:50 +02:00
Serhiy Storchaka 49785b06de
gh-102512: Turn _DummyThread into _MainThread after os.fork() called from a foreign thread (GH-113261)
Always set a _MainThread as a main thread after os.fork() is called from
a thread started not by the threading module.

A new _MainThread was already set as a new main thread after fork if
threading.current_thread() was not called for a foreign thread before fork.
Now, if it was called before fork, the implicitly created _DummyThread will
be turned into _MainThread after fork.

It fixes, in particularly, an incompatibility of _DummyThread with
the threading shutdown logic which relies on the main thread
having tstate_lock.

Co-authored-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2024-01-22 16:14:42 +02:00
Antoine Pitrou 0e9c364f4a
GH-110829: Ensure Thread.join() joins the OS thread (#110848)
Joining a thread now ensures the underlying OS thread has exited. This is required for safer fork() in multi-threaded processes.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-11-04 13:59:24 +00:00
Eric Snow f5198b09e1
gh-109860: Use a New Thread State When Switching Interpreters, When Necessary (gh-110245)
In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread.  For the main interpreter there wasn't much of a problem, but for subinterpreters we were *mostly* okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()).  There was a good chance that tstate wasn't actually in use by another thread.

However, there are no guarantees of that.  Furthermore, re-using an already used tstate is currently fragile.  To address this, now we create a new thread state in each of those places and use it.

One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).
2023-10-03 09:20:48 -06:00
Eric Snow 1dd9dee45d
gh-105716: Support Background Threads in Subinterpreters Consistently (gh-109921)
The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.

We add PyInterpreterState.threads.main, with some internal C-API functions.
2023-10-02 20:12:12 +00:00
Antoine Pitrou 0eb98837b6
gh-109593: Fix reentrancy issue in multiprocessing resource_tracker (#109629)
---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-09-26 13:57:25 +02:00
Nikita Sobolev 80f30cf51b
gh-102029: Deprecate passing arguments to `_PyRLock` in `threading` (#102071) 2023-08-17 09:19:07 -07:00
Nikita Sobolev e4b88c1e4a
gh-106236: Replace `assert` with `raise RuntimeError` in `threading.py` (#106237)
Replace `assert` with `raise ` in `threading.py` so that -OO does not alter _DummyThread behavior.
2023-07-12 11:07:59 -07:00
Gregory P. Smith 4b56e56c49
gh-104837: Revert "gh-104341: Add a Separate "Running" Lock for Each Thread (gh-104754) (#104838)
gh-104837: Revert "gh-104341: Add a Separate "Running" Lock for Each Thread (gh-104754)"

This reverts commit 097b7830cd.
2023-05-24 01:00:57 -07:00
Eric Snow 097b7830cd
gh-104341: Add a Separate "Running" Lock for Each Thread (gh-104754)
Having a separate lock means Thread.join() doesn't need to wait for the thread to be cleaned up first.  It can wait for the thread's Python target to finish running.  This gives us some flexibility in how we clean up threads.

(This is a minor cleanup as part of a fix for gh-104341.)
2023-05-23 14:29:30 -06:00
Gregory P. Smith 894f2c3c16
gh-100228: Warn from os.fork() if other threads exist. (#100229)
Not comprehensive, best effort warning. There are cases when threads exist on some platforms that this code cannot detect. macOS when API permissions allow and Linux with a readable /proc procfs present are the currently supported cases where a warning should show up reliably.

Starting with a DeprecationWarning for now, it is less disruptive than something like RuntimeWarning and most likely to only be seen in people's CI tests - a good place to start with this messaging.
2022-12-29 14:41:39 -08:00
Eric Snow 4702552885
gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)
Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads.  By default, we were disallowing all three for "isolated" interpreters.  We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API.

Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads.  The default for "isolated" interpreters disables fork, exec, and daemon threads.  Regular threads are allowed by default.  We continue always allowing everything For the main interpreter and the legacy API.

In the code, we add `_PyInterpreterConfig.allow_exec` and  `_PyInterpreterConfig.allow_daemon_threads`.  We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
2022-10-31 12:35:54 -07:00
Daniel Giger 22ed5233b7
gh-96349: fix minor performance regression initializing threading.Event (gh-96350) 2022-08-30 21:10:02 +09:00
Daniel Giger e534440510
fix threading.Event.isSet() docstring (#96297)
fixes gh-96296
2022-08-26 22:06:26 -07:00
Pablo Galindo Salgado e34c82abeb
GH-93503: Add thread-specific APIs to set profiling and tracing functions in the C-API (#93504)
* gh-93503: Add APIs to set profiling and tracing functions in all threads in the C-API

* Use a separate API

* Fix NEWS entry

* Add locks around the loop

* Document ignoring exceptions

* Use the new APIs in the sys module

* Update docs
2022-08-24 23:21:39 +01:00
Victor Stinner 259dd71c32
gh-84623: Remove unused imports in stdlib (#93773) 2022-06-13 16:28:41 +02:00
Serhiy Storchaka 70af994fee
gh-92530: Fix an issue that occurred after interrupting threading.Condition.notify (GH-92534)
If Condition.notify() was interrupted just after it released the waiter lock,
but before removing it from the queue, the following calls of notify() failed
with RuntimeError: cannot release un-acquired lock.
2022-05-16 08:25:29 +03:00
Dong-hee Na c826867b7c
gh-89474: Improve Semaphore/BoundedSemaphore.release() for multiple thread waiting (GH-92447) 2022-05-08 22:33:53 +09:00
Charlie Zhao e466faa9df
bpo-45735: Promise the long-time truth that `args=list` works (GH-30982)
For threads, and for multiprocessing, it's always been the case that ``args=list`` works fine when passed to ``Process()`` or ``Thread()``, and such code is common in the wild. But, according to the docs, only a tuple can be used. This brings the docs into synch with reality.

Doc changes by Charlie Zhao.
Co-authored-by: Tim Peters <tim.peters@gmail.com>
2022-02-25 22:17:13 -06:00
Christian Clauss 745c9d9dfc
Fix typos in the Lib directory (GH-28775)
Fix typos in the Lib directory as identified by codespell.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-10-06 16:13:48 -07:00
Łukasz Langa f1ca5d7f61
[typo] Fix threading.Barrier comment that used confusing punctuation (GH-28623)
Removed extra comma in comment that indicates state of a `Barrier` as it was confusing and breaking the flow while reading.

Co-authored-by: Priyank <5903604+cpriyank@users.noreply.github.com>
2021-09-29 16:11:26 +02:00
Serhiy Storchaka eed32df5b6
bpo-24391: Better reprs for threading objects. (GH-20534)
Add reprs for Semaphore, BoundedSemaphore, Event, and Barrier.
2021-09-29 13:07:58 +03:00
Victor Stinner 95d3137082
bpo-1596321: Fix threading._shutdown() for the main thread (GH-28549)
Fix the threading._shutdown() function when the threading module was
imported first from a thread different than the main thread: no
longer log an error at Python exit.
2021-09-27 23:09:00 +02:00
Victor Stinner a22be4943c
bpo-45274: Fix Thread._wait_for_tstate_lock() race condition (GH-28532)
Fix a race condition in the Thread.join() method of the threading
module. If the function is interrupted by a signal and the signal
handler raises an exception, make sure that the thread remains in a
consistent state to prevent a deadlock.
2021-09-27 14:20:31 +02:00
Victor Stinner 0729694246
bpo-44422: threading.Thread reuses the _delete() method (GH-26741)
The _bootstrap_inner() method of threading.Thread now reuses its
_delete() method rather than accessing _active() directly. It became
possible since _active_limbo_lock became reentrant. Moreover, it no
longer ignores any exception when deleting the thread from the
_active dictionary.
2021-06-16 11:41:17 +02:00
Victor Stinner 243fd01047
bpo-44422: Fix threading.enumerate() reentrant call (GH-26727)
The threading.enumerate() function now uses a reentrant lock to
prevent a hang on reentrant call.
2021-06-15 16:14:24 +02:00
Antoine Pitrou c10c2ec7a0
bpo-37788: Fix reference leak when Thread is never joined (GH-26103)
When a Thread is not joined after it has stopped, its lock may remain in the _shutdown_locks set until interpreter shutdown.  If many threads are created this way, the _shutdown_locks set could therefore grow endlessly.  To avoid such a situation, purge expired locks each time a new one is added or removed.
2021-05-14 12:37:20 -07:00
Irit Katriel 12e7d10dfd
bpo-25821: Fix inaccuracy in threading.enumerate/is_alive documentation (#23192) 2021-05-11 18:55:24 +01:00
Jelle Zijlstra 9825bdfbd5
bpo-43723: Deprecate camelCase aliases from threading (GH-25174)
The snake_case names have existed since Python 2.6, so there is
no reason to keep the old camelCase names around. One similar
method, threading.Thread.isAlive, was already removed in
Python 3.9 (bpo-37804).
2021-04-12 10:42:53 +02:00
BarneyStratford 01c4fddc4b
bpo-41149: Fix a bug in threading that causes fals-y threads callables to fail to start. (GH-21201) 2021-02-02 20:24:24 +00:00
Victor Stinner 5909a494cd
bpo-42350: Fix Thread._reset_internal_locks() (GH-23268)
Fix the threading.Thread class at fork: do nothing if the thread is
already stopped (ex: fork called at Python exit). Previously, an
error was logged in the child process.
2020-11-16 15:20:34 +01:00
Mario Corchero 750c5abf43
bpo-42308: Add threading.__excepthook__ (GH-23218)
Add threading.__excepthook__ to allow retrieving the original value
of threading.excepthook in case it is set to a broken or a different
value.
2020-11-12 18:27:44 +01:00
Mario Corchero 0001a1b69e
bpo-42251: Add gettrace and getprofile to threading (GH-23125)
This allows to retrieve the functions that were set in these two, which might differ from sys.gettrace and sys.getprofile within a thread.
2020-11-04 09:27:43 +00:00
Victor Stinner 98c16c991d
bpo-41833: threading.Thread now uses the target name (GH-22357) 2020-09-23 23:21:19 +02:00
Victor Stinner 14d5331eb5
bpo-40234: Revert "bpo-37266: Daemon threads are now denied in subinterpreters (GH-14049)" (GH-19456)
This reverts commit 066e5b1a91.
2020-04-12 23:45:09 +02:00
Victor Stinner 87255be696
bpo-40089: Add _at_fork_reinit() method to locks (GH-19195)
Add a private _at_fork_reinit() method to _thread.Lock,
_thread.RLock, threading.RLock and threading.Condition classes:
reinitialize the lock after fork in the child process; reset the lock
to the unlocked state.

Rename also the private _reset_internal_locks() method of
threading.Event to _at_fork_reinit().

* Add _PyThread_at_fork_reinit() private function. It is excluded
  from the limited C API.
* threading.Thread._reset_internal_locks() now calls
  _at_fork_reinit() on self._tstate_lock rather than creating a new
  Python lock object.
2020-04-07 23:11:49 +02:00
Kyle Stanley b61b818d91
bpo-39812: Remove daemon threads in concurrent.futures (GH-19149)
Remove daemon threads from :mod:`concurrent.futures` by adding
an internal `threading._register_atexit()`, which calls registered functions
prior to joining all non-daemon threads. This allows for compatibility
with subinterpreters, which don't support daemon threads.
2020-03-27 20:31:22 +01:00
Victor Stinner d8ff44ce4c
bpo-40089: Fix threading._after_fork() (GH-19191)
If fork was not called by a thread spawned by threading.Thread,
threading._after_fork() now creates a _MainThread instance for
_main_thread, instead of a _DummyThread instance.
2020-03-27 17:50:42 +01:00
Serhiy Storchaka 1f21eaa15e
bpo-15999: Clean up of handling boolean arguments. (GH-15610)
* Use the 'p' format unit instead of manually called PyObject_IsTrue().
* Pass boolean value instead 0/1 integers to functions that needs boolean.
* Convert some arguments to boolean only once.
2019-09-01 12:16:51 +03:00
Raymond Hettinger 35f6301d68
bpo-10978: Semaphores can release multiple threads at a time (GH-15588) 2019-08-29 01:45:19 -07:00
Victor Stinner d11c2c6077
Revert "bpo-37788: Fix a reference leak if a thread is not joined (GH-15228)" (GH-15338)
This reverts commit d3dcc92778.
2019-08-20 00:47:07 +01:00
Victor Stinner d3dcc92778
bpo-37788: Fix a reference leak if a thread is not joined (GH-15228)
Add threading.Thread.__del__() method to ensure that the thread state
lock is removed from the _shutdown_locks list when a thread
completes.
2019-08-19 23:37:17 +01:00
Dong-hee Na 44046fe4fc bpo-37804: Remove the deprecated method threading.Thread.isAlive() (GH-15225) 2019-08-12 19:41:08 +02:00
mbarkhau bb110cc2ed Improve threading.daemon docstring (GH-14278)
Rephrase and clarify that "the entire Python program exits when only daemon threads are left". This matches the documentation at https://docs.python.org/3/library/threading.html#thread-objects.
2019-06-22 05:51:06 -07:00
Victor Stinner 066e5b1a91
bpo-37266: Daemon threads are now denied in subinterpreters (GH-14049)
In a subinterpreter, spawning a daemon thread now raises an
exception. Daemon threads were never supported in subinterpreters.
Previously, the subinterpreter finalization crashed with a Pyton
fatal error if a daemon thread was still running.

* Add _thread._is_main_interpreter()
* threading.Thread.start() now raises RuntimeError if the thread is a
  daemon thread and the method is called from a subinterpreter.
* The _thread module now uses Argument Clinic for the new function.
* Use textwrap.dedent() in test_threading.SubinterpThreadingTests
2019-06-14 18:55:22 +02:00
Victor Stinner 6f75c87375
tbpo-36402: Fix threading.Thread._stop() (GH-14047)
Remove the _tstate_lock from _shutdown_locks, don't remove None.
2019-06-13 12:06:24 +02:00
Victor Stinner 468e5fec8a
bpo-36402: Fix threading._shutdown() race condition (GH-13948)
Fix a race condition at Python shutdown when waiting for threads.
Wait until the Python thread state of all non-daemon threads get
deleted (join all non-daemon threads), rather than just wait until
Python threads complete.

* Add threading._shutdown_locks: set of Thread._tstate_lock locks
  of non-daemon threads used by _shutdown() to wait until all Python
  thread states get deleted. See Thread._set_tstate_lock().
* Add also threading._shutdown_locks_lock to protect access to
  threading._shutdown_locks.
* Add test_finalization_shutdown() test.
2019-06-13 01:30:17 +02:00