We should maintain the invariant that a zero `ob_tid` implies the
refcount fields are merged.
* Move the assignment in `_Py_MergeZeroLocalRefcount` to immediately
before the refcount merge.
* Update `_PyTrash_thread_destroy_chain` to set `ob_ref_shared` to
`_Py_REF_MERGED` when setting `ob_tid` to zero.
Also check this invariant with assertions in the GC in debug builds.
That uncovered a bug when running out of memory during GC.
(cherry picked from commit d23be3947c)
Co-authored-by: Sam Gross <colesbury@gmail.com>
This avoids messages like:
ThreadSanitizer: starting new threads after multi-threaded fork is not
supported. Dying (set die_after_fork=0 to override)
(cherry picked from commit 82a4dac9f6)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-121546: Disable contextvar caching on free-threading build (GH-121740)
(cherry picked from commit e904300882)
Co-authored-by: Ken Jin <kenjin@python.org>
See 6b98b274b6 for an explanation of the problem and solution. Here I've applied the solution to channels.
(cherry picked from commit 8b209fd4f8, AKA gh-121805)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Any cross-interpreter mechanism for passing objects between interpreters must be very careful to respect isolation, even when the object is effectively immutable (e.g. int, str). Here this especially relates to when an interpreter sends one of its objects, and then is destroyed while the inter-interpreter machinery (e.g. queue) still holds a reference to the object.
When I added interpreters.Queue, I dealt with that case (using an atexit hook) by silently removing all items from the queue that were added by the finalizing interpreter.
Later, while working on concurrent.futures.InterpreterPoolExecutor (gh-116430), I noticed it was somewhat surprising when items were silently removed from the queue when the originating interpreter was destroyed. (See my comment on that PR.)
It took me a little while to realize what was going on. I expect that users, which much less context than I have, would experience the same pain.
My approach, here, to improving the situation is to give users three options:
1. return a singleton (interpreters.queues.UNBOUND) from Queue.get() in place of each removed item
2. raise an exception (interpreters.queues.ItemInterpreterDestroyed) from Queue.get() in place of each removed item
3. existing behavior: silently remove each item (i.e. Queue.get() skips each one)
The default will now be (1), but users can still explicitly opt in any of them, including to the silent removal behavior.
The behavior for each item may be set with the corresponding Queue.put() call. and a queue-wide default may be set when the queue is created. (This is the same as I did for "synconly".)
(cherry picked from commit 6b98b274b6, AKA gh-116431)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
It is our general practice to make new optional parameters keyword-only,
even if the existing parameters are all positional-or-keyword. Passing
this parameter as positional would look confusing and could be error-prone
if additional parameters are added in the future.
(cherry picked from commit 50eec501fe)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-121660: Fix `ga_getitem` by explicitly checking for `NULL` result (GH-121661)
(cherry picked from commit bb802db8cf)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-120642: Move _PyCode_CODE() to the internal C API (GH-121644)
Move _PyCode_CODE() and _PyCode_NBYTES() macros to the internal C API
since they use _Py_CODEUNIT which is only part of the internal C API.
(cherry picked from commit a2bec77d25)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-73159 Added clarifications in multiprocessing docs on that objects are pickled. (GH-121686)
Added explicit comments about that objects are pickled when transmitted via multiprocessing queues and pipes.
(cherry picked from commit b5805892d5)
Co-authored-by: Ulrik Södergren <ulrik@digitalfotografen.se>
The `allocate_weakref` may return NULL when out of memory. We need to
handle that case and propagate the error.
(cherry picked from commit a640a605a8)
Co-authored-by: Sam Gross <colesbury@gmail.com>
We also need to close the `slave_fd` earlier so that reading from
`master_fd` won't block forever when the subprocess finishes.
(cherry picked from commit abc3aeebdb)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-121497: Make Pyrepl respect correctly the history with input hook set (GH-121498)
(cherry picked from commit 4e36dd7d87)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
gh-121499: Fix multi-line history rendering in the REPL (GH-121531)
(cherry picked from commit 4b9e10d0ea)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
On POSIX systems, excluding macOS framework installs, the lib directory
for the free-threaded build now includes a "t" suffix to avoid conflicts
with a co-located default build installation.
(cherry picked from commit e8c91d90ba)
Co-authored-by: Sam Gross <colesbury@gmail.com>
When builtin static types are initialized for a subinterpreter, various "tp" slots have already been inherited (for the main interpreter). This was interfering with the logic in add_operators() (in Objects/typeobject.c), causing a wrapper to get created when it shouldn't. This change fixes that by preserving the original data from the static type struct and checking that.
(cherry picked from commit 5250a03133, AKA gh-121602)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
This makes select.poll() and kqueue() objects thread-safe in the
free-threaded build. Note that calling close() concurrently with other
functions is still not thread-safe due to races on file descriptors
(gh-121544).
(cherry picked from commit 44937d11a6)
Co-authored-by: Sam Gross <colesbury@gmail.com>
This fixes a mistake in gh-113012 and adds a test that verifies the fix.
(cherry picked from commit 35a67e36aa, AKA gh-121597)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
The functions look thread-safe and I haven't seen any warnings issued
when running the tests locally.
(cherry picked from commit 7641743d48)
Co-authored-by: Sam Gross <colesbury@gmail.com>
The only remaining race in dictobject.c was in _PyDict_CheckConsistency
when the dictionary has shared keys.
(cherry picked from commit 3ec719fabf)
Co-authored-by: Sam Gross <colesbury@gmail.com>
`munmap(NULL)` is not noop, like `free(NULL)` is.
Fixes an observed testsuite hang on 32-bit ARM systems.
(cherry picked from commit a802277914, AKA gh-121491)
Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.
This leads to errors like:
Exception: failed to sync child in 300.3 sec
Use pipes instead to mutually synchronize between parent and child.
(cherry picked from commit af9f6de6ea)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-89364: Export PySignal_SetWakeupFd() function (GH-121537)
Export the PySignal_SetWakeupFd() function. Previously, the function
was documented but it couldn't be used in 3rd party code.
(cherry picked from commit ca0fb3423c)
Co-authored-by: Victor Stinner <vstinner@python.org>
Miss Islington (bot)
Kirill Podoprigora
Sam Gross
Hugo van Kemenade
Ken Jin
Jelle Zijlstra
neonene