* gh-71936: Fix race condition in multiprocessing.Pool
Proxes of shared objects register a Finalizer in BaseProxy._incref(), and it
will call BaseProxy._decref() when it is GCed. This may cause a race condition
with Pool(maxtasksperchild=None) on Windows.
A connection would be closed and raised TypeError when a GC occurs between
_ConnectionBase._check_writable() and _ConnectionBase._send_bytes() in
_ConnectionBase.send() in the second or later task, and a new object
is allocated that shares the id() of a previously deleted one.
Instead of using the id() of the token (or the proxy), use a unique,
non-reusable number.
Co-Authored-By: Akinori Hattori <hattya@gmail.com>
Previously, this would cause an `AttributeError` if the patch stopped more than once after this, and would also disrupt the original patched object.
---------
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
The PyMutex implementation supports unlocking after fork because we
clear the list of waiters in parking_lot.c. This doesn't work as well
for _PyRecursiveMutex because on some systems, such as SerenityOS, the
thread id is not preserved across fork().
Accepting objects with false values (like 0 and []) except empty strings
and byte-like objects and None in urllib.parse functions parse_qsl() and
parse_qs() is now deprecated.
These changes makes it easier to backport the _interpreters, _interpqueues, and _interpchannels modules to Python 3.12.
This involves the following:
* add the _PyXI_GET_STATE() and _PyXI_GET_GLOBAL_STATE() macros
* add _PyXIData_lookup_context_t and _PyXIData_GetLookupContext()
* add _Py_xi_state_init() and _Py_xi_state_fini()
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Removes erroneous explanation of the `global` statement restrictions; a name declared as global can be subsequently bound using any kind of name binding operation.
* Updates `test_global.py` to also test various name-binding scenarios for global
variables to ensure correct behavior
These changes makes it easier to backport the _interpreters, _interpqueues, and _interpchannels modules to Python 3.12.
This involves the following:
* rename several structs and typedefs
* add several typedefs
* stop using the PyThreadState.state field directly in parking_lot.c
Add an entry for the ``--enable-experimental-jit`` option in ``Doc/using/configure.rst``.
This was added as an experimental option in CPython 3.13.
Possible values for it:
* `no` - don't build the JIT.
* `yes` - build the JIT.
* `yes-off` - build the JIT but disable it by default.
* `interpreter` - don't build the JIT but enable tier 2 interpreter instead.
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
The first version had it running two forkserver and one spawn tests underneath each of the _fork, _forkserver, and _spawn test suites that build off the generic one.
This adds to the existing complexity of the multiprocessing test suite by offering BaseTestCase classes another attribute to control which suites they are invoked under. Practicality vs purity here. :/
Net result: we don't over-run the new test and their internal logic is simplified.
Skip `testMakefileCloseSocketDestroy` test if `sys.getrefcount` isn't available. This is necessary for PyPy and other Python implementations that do not have `sys.getrefcount`.
When `zipfile.Path.open` is called, the implementation will check
whether the path already exists in the ZIP file. However, this check is
only required when the ZIP file is in read mode. By swapping arguments
of the `and` operator, the short-circuiting will prevent the check from
being run in write mode.
This change will improve the performance of `open()`, because checking
whether a file exists is slow in write mode, especially when the archive
has many members.
gh-117378: Fix multiprocessing forkserver preload sys.path inheritance.
`sys.path` was not properly being sent from the parent process when launching
the multiprocessing forkserver process to preload imports. This bug has been
there since the forkserver start method was introduced in Python 3.4. It was
always _supposed_ to inherit `sys.path` the same way the spawn method does.
Observable behavior change: A `''` value in `sys.path` will now be replaced in
the forkserver's `sys.path` with an absolute pathname
`os.path.abspath(os.getcwd())` saved at the time that `multiprocessing` was
imported in the parent process as it already was when using the spawn start
method. **This will only be observable during forkserver preload imports**.
The code invoked before calling things in another process already correctly sets `sys.path`.
Which is likely why this went unnoticed for so long as a mere performance issue in
some configurations.
A workaround for the bug on impacted Pythons is to set PYTHONPATH in the
environment before multiprocessing's forkserver process was started. Not perfect
as that is then inherited by other children, etc, but likely good enough for many
people's purposes.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
In `PathBase.resolve()`, raise `UnsupportedOperation` if a non-POSIX path
parser is used (our implementation uses `posixpath._realpath()`, which
produces incorrect results for non-POSIX path flavours.) Also tweak code to
call `self.absolute()` upfront rather than supplying an emulated `getcwd()`
function.
Adjust `PathBase.absolute()` to work somewhat like `resolve()`. If a POSIX
path parser is used, we treat the root directory as the current directory.
This is the simplest useful behaviour for concrete path types without a
current directory cursor.
In the past I've equivocated about whether to require at least one argument
in the `PurePathBase` (and `PathBase`) initializer, and what the default
should be if we make it optional. I now have a local use case that has
persuaded me to make it optional and default to the empty string (a
`zipp.Path`-like class that treats relative and absolute paths similarly.)
Happily this brings the base class more in line with `PurePath` and `Path`.
Ritvik Pasham
Petr Viktorin
Red4Ru
Andrew Svetlov
Yuxuan Zhang
Nice Zombies
Sam Gross
Barney Gale
Serhiy Storchaka
Bénédikt Tran
Stephen Morton
Eric Snow
Hugo van Kemenade
RUANG (James Roy)
Sayandip Dutta
Kumar Aditya
Daehee Kim
sobolevn
Stephen Morton
Sahil Prajapati
Diego Russo
Beomsoo Kim
Ned Batchelder
Tomas R.
Gregory P. Smith
谭九鼎
Rafael Fontenelle
vivodi
Pedro Fonini
Kirill Podoprigora
CF Bolz-Tereick
Jan Hicken
Ken Jin