Increase the backlog for multiprocessing.connection.Listener` objects created
by `multiprocessing.manager` and `multiprocessing.resource_sharer` to
significantly reduce the risk of getting a connection refused error when creating
a `multiprocessing.connection.Connection` to them.
On Windows, `os.path.isabs()` now returns `False` when given a path that
starts with exactly one (back)slash. This is more compatible with other
functions in `os.path`, and with Microsoft's own documentation.
Also adjust `pathlib.PureWindowsPath.is_absolute()` to call
`ntpath.isabs()`, which corrects its handling of partial UNC/device paths
like `//foo`.
Co-authored-by: Jon Foster <jon@jon-foster.co.uk>
Make zipfile.ZipInfo.compress_level public.
A property is used to retain the behavior of the ._compresslevel.
People constructing zipfile.ZipInfo instances to pass into existing APIs to control per-file compression levels already treat this as public, there was never a reason for it not to be.
I used the more modern name compress_level instead of compresslevel as the keyword argument on other ZipFile APIs is called to be consistent with compress_type and a general long term preference of not runningwordstogether without a separator in names.
sqlite3.Connection.iterdump now ensures that foreign key support is
disabled before dumping the database schema, if there is any foreign key
violation.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
test_configdialog.HighPageTest.test_highlight_target_text_mouse fails
if a line of the Highlight tab text sample is not visible. If so, bbox()
in click_char() returns None and the unpacking iteration fails.
This occurred on a Devuan Linux system. Fix by moving the
'see character' call inside click_char, just before the bbox call.
Also, reduce the click_char calls to just one per tag name and
replace the other nested function with a dict comprehension.
On Windows, set _O_NOINHERIT flag on file descriptors
created by os.pipe() and io.WindowsConsoleIO.
Add test_pipe_spawnl() to test_os.
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Previously the C implementation of pickle.Pickler and pickle.Unpickler
classes did not have such methods and they could only be used if
they were overloaded in subclasses or set as instance attributes.
Fixed calling super().persistent_id() and super().persistent_load() in
subclasses of the C implementation of pickle.Pickler and pickle.Unpickler
classes. It no longer causes an infinite recursion.
If other exception was raised during exiting an expired
asyncio.timeout() block, insert TimeoutError in the exception context
just above the CancelledError.
winfo_id() converts the result of "winfo id" command to integer, but
"winfo pathname" command requires an argument to be a hexadecimal number
on Win64.
When an `StopIteration` raises into `asyncio.Future`, this will cause
a thread to hang. This commit address this by not raising an exception
and silently transforming the `StopIteration` with a `RuntimeError`,
which the caller can reconstruct from `fut.exception().__cause__`
It can be used to set the location of a .python_history file
---------
Co-authored-by: Levi Sabah <0xl3vi@gmail.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Allow extraction (no-op) of a "/" folder in a zipfile, they are commonly added by some archive creation tools.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Remove LibreSSL specific workaround ifdefs from `_ssl.c` and delete the non-version-specific `_ssl_data.h` file (relevant for OpenSSL < 1.1.1, which we no longer support per PEP 644).
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Raise auditing events in `pathlib.Path.glob()`, `rglob()` and `walk()`,
but not in `pathlib._abc.PathBase` methods. Also move generation of a
deprecation warning into `pathlib.Path` so it gets the right stack level.
- Only attempt to figure out whether protocol members are "method members" or not if the class is marked as a runtime protocol. This information is irrelevant for non-runtime protocols; we can safely skip the risky introspection for them.
- Only do the risky getattr() calls in one place (the runtime_checkable class decorator), rather than in three places (_ProtocolMeta.__init__, _ProtocolMeta.__instancecheck__ and _ProtocolMeta.__subclasscheck__). This reduces the number of locations in typing.py where the risky introspection could go wrong.
- For runtime protocols, if determining whether a protocol member is callable or not fails, give a better error message. I think it's reasonable for us to reject runtime protocols that have members which raise strange exceptions when you try to access them. PEP-544 clearly states that all protocol member must be callable for issubclass() calls against the protocol to be valid -- and if a member raises when we try to access it, there's no way for us to figure out whether it's a callable member or not!
This ensures the source directory is not modified at build time, and different builds (e.g. different versions or GIL vs no-GIL) do not have conflicts.
* gh-113536: Expose `os.waitid` on macOS
This API has been available on macOS for a long time, but was
explicitly excluded due to unspecified problems with the API
in ancient versions of macOS.
* Document that the API is available on macOS starting in Python 3.13
The length field of StgDictObject for Structure class contains now
the total number of items in ffi_type_pointer.elements (excluding
the trailing null).
The old behavior of using the number of elements in the parent class can
cause the array to be truncated when it is copied, especially when there
are multiple layers of subclassing.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Previously some error messages complained about incompatible
combinations of directives that are not contained in the format string.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
First fix resolve situation when pyexpat module (which contains expat_CAPI
capsule) deallocates before _elementtree, so we need to hold a strong
reference to pyexpat module to.
Second fix resolve situation when module state is deallocated before
deallocation of XMLParser instances, which uses module state to clear
some stuff.
In case the spawned process is setuid, we may not be able to send
signals to it, in which case our .kill() call will raise
PermissionError.
Ignore that in order to avoid .close() raising an exception. Hopefully
the process will exit as a result of receiving EOF on its stdin.
* gh-74573: document that ndbm can silently corrupt databases on macOS
The system ndbm implementation on macOS has an undocumented limitation
on the size of values and can silently corrupt database files when those
are exceeded.
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Previously it worked differently if dst is a symbolic link:
it modified the permission bits of dst itself rather than the file
it points to if follow_symlinks is true or src is not a symbolic link,
and did nothing if follow_symlinks is false and src is a symbolic link.
Also document similar changes in shutil.copystat().
* Sync with importlib_metadata 7.0.0
* Add blurb
* Update docs to reflect changes.
* Link datamodel docs for object.__getitem__
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
* Add what's new for removed __getattr__
* Link datamodel docs for object.__getitem__
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
* Add exclamation point, as that seems to be used for other classes.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
* gh-65701: document that freeze doesn't work with framework builds on macOS
The framework install is inherently incompatible with freeze. Document
that that freeze doesn't work with framework builds and bail out
early when trying to run freeze anyway.
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
The installation of XQuartz on macOS will unconditionally
set the $DISPLAY variable. The X11 server will be launched
when a program tries to access the display. This results
in launching the X11 server when using the webbrowser module,
even though X11 browsers won't be used in practice.
A typo left this check broken so many of us who do out-of-tree builds
were seeing strange failures due to bad `Python/frozen_modules/*.h`
files being picked up from the source tree and used at build time from
different Python versions leading to errors like:
`Fatal Python error: _PyImport_InitCore: failed to initialize importlib`
Or similar once our build got to an "invoke the interpreter"
bootstrapping step due to incorrect bytecode being embedded.
When wrapped, `_SSLProtocolTransport._force_close(exc)` is called just like in the unwrapped scenario `_SelectorTransport._force_close(exc)` or `_ProactorBasePipeTransport._force_close(exc)` would be called, except here the exception needs to be passed through the `SSLProtocol._abort()` method, which didn't accept an exception object.
This commit ensures that this path works, in the same way that the uvloop implementation of SSLProto passes on the exception (on which the current implementation of SSLProto is based).
Change _osx_support.get_platform_osx() to make sure that the
version number in the result includes at least a major and
minor version (e.g. 14.2) even if MACOSX_DEPLOYMENT_TARGET is
set to just a major version (e.g. 14).
This matches the versions expected by pip when selecting
appropriate wheels for installation.
Add support for `os.POSIX_SPAWN_CLOSEFROM` and
`posix_spawn_file_actions_addclosefrom_np` and have the `subprocess` module use
them when available. This means `posix_spawn` can now be used in the default
`close_fds=True` situation on many platforms.
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
* Allow posix_spawn to inherit environment form parent environ variable.
With this change, posix_spawn call can behave similarly to execv with regards to environments when used in subprocess functions.
PR #100161 added fancy float-style formatting for the Fraction type,
but left us in a state where basic formatting for fractions (alignment,
fill, minimum width, thousands separators) still wasn't supported.
This PR adds that support.
---------
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Detect email address parsing errors and return empty tuple to
indicate the parsing error (old API). Add an optional 'strict'
parameter to getaddresses() and parseaddr() functions. Patch by
Thomas Dwyer.
Co-Authored-By: Thomas Dwyer <github@tomd.tel>
On Windows, Process.terminate() no longer sets the returncode
attribute to always call WaitForSingleObject() in Process.wait().
Previously, sometimes the process was still running after
TerminateProcess() even if GetExitCodeProcess() is not STILL_ACTIVE.
It was raised in two cases:
* in the import statement when looking up __import__
* in pickling some builtin type when looking up built-ins iter, getattr, etc.
This affects task creation through either `asyncio.create_task()` or `TaskGroup.create_task()` -- the redundant call to `task.set_name()` is skipped. We still call `set_name()` when a task factory is involved, because the task factory call signature (unfortunately) doesn't take a `name` argument.
glibc-2.34 implements closefrom(3) using the same semantics as on BSD.
Check for closefrom() in configure and use the check result in
fileutils.c, rather than hardcoding a FreeBSD check.
Some implementations of closefrom() return an int. Explicitly discard
the return value by casting it to void, to avoid future compiler
warnings.
Signed-off-by: Sam James <sam@gentoo.org>
* bpo-43120: Add a number of LOG_* constants to syslog
This adds a number of syslog facilities to the syslogmodule.c.
These values are available on macOS.
* Switch contant documentation to the data directive
This fixes a CI warning and matches the pattern
used in the documentation for ``os``.
* Update Doc/library/syslog.rst
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Renamed re.error for clarity, and kept re.error for backward compatibility.
Updated idlelib files at TJR's request.
---------
Co-authored-by: Matthias Bussonnier <mbussonnier@ucmerced.edu>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* gh-112898: warn about unsaved files when quitting IDLE on macOS
Implement the TK function ``::tk::mac::Quit`` on macOS to
ensure that IDLE asks about saving unsaved files when
quitting IDLE.
Co-authored-by: Christopher Chavez chrischavez@gmx.us
On recentish macOS versions the system tar
command includes system metadata (ACLs, extended attributes
and resource forks) in the tar archive, which
shutil.make_archive will not do. This can cause
spurious test failures.
Regression test that vfork is used when expected by subprocess.
This is written integration test style, it uses strace if it is present and appears to work to find out what system call actually gets used in different scenarios.
Test coverage is added for the default behavior and that of each of the specific arguments that must disable the use of vfork. obviously not an entire test matrix, but it covers the most important aspects.
If there are ever issues with this test being flaky or failing on new platforms, rather than try and adapt it for all possible platforms, feel free to narrow the range it gets tested on when appropriate. That is not likely to reduce coverage.
Add private `pathlib._PurePathBase` class: a private superclass of both `PurePath` and `_PathBase`. Unlike `PurePath`, it does not define any of these special methods: `__fspath__`, `__bytes__`, `__reduce__`, `__hash__`, `__eq__`, `__lt__`, `__le__`, `__gt__`, `__ge__`. Its initializer and path joining methods accept only strings, not os.PathLike objects more broadly.
This is important for supporting *virtual paths*: user subclasses of `_PathBase` that provide access to archive files, FTP servers, etc. In these classes, the above methods should be implemented by users only as appropriate, with due consideration for the hash/equality of any backing objects, such as file objects or sockets.
* gh-110820: Make sure processor specific defines are correct for Universal 2 build on macOS
A number of processor specific defines are different for x86-64 and
arm64, and need to be adjusted in pymacconfig.h.
* remove debug stuf
* gh-51944: Add some macOS constants to termios
This changeset adds all public constants in <termio.h>
and <sys/termios.h> on macOS that weren't present
already.
Based on the macOS 14.2 SDK
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
If the input prompt to the builtin input function on terminal has any null
character, then raise ValueError instead of silently truncating it.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Previously arbitrary errors could be cleared during formatting error
messages for ImportError or AttributeError for modules. Now all
unexpected errors are reported.
Use scanning "/dev/fd/" on macOS in support.fd_count(). That's both more efficient than scanning all possible file descriptors, and avoids crashing the interpreter when there are open "guarded" file descriptors.
"Guarded" file descriptors are a macOS feature where file descriptors used by system libraries are marked and cause hard crashes when used by "user" code.
Co-authored-by: Victor Stinner <vstinner@python.org>
Previously, "widget.unbind(sequence, funcid)" destroyed the current binding
for "sequence", leaving "sequence" unbound, and deleted the "funcid"
command.
Now it removes only "funcid" from the binding for "sequence", keeping
other commands, and deletes the "funcid" command.
It leaves "sequence" unbound only if "funcid" was the last bound command.
Co-authored-by: GiovanniL <13402461+GiovaLomba@users.noreply.github.com>
* Implement _Py_HashPointerRaw() as a static inline function.
* Add Py_HashPointer() tests to test_capi.test_hash.
* Keep _Py_HashPointer() function as an alias to Py_HashPointer().
* Ignore os.close() errors when ignore_errors is True.
* Pass os.close() errors to the error handler if specified.
* os.close no longer retried after error.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Previously a symlink attack resistant version of shutil.rmtree() could ignore
or pass to the error handler arbitrary exception when invalid arguments
were provided.
Set MAX_STRUCT_SIZE to 32 in stgdict.c when on Arm platforms.
This because on Arm platforms structs with at most 4 elements of any
floating point type values can be passed through registers. If the type
is double the maximum size of the struct is 32 bytes.
On x86-64 Linux, it's maximum 16 bytes hence we need to differentiate.
Add a track parameter to shared memory to allow resource tracking via the side-launched resource tracker process to be disabled on platforms that use it (POSIX).
This allows people who do not want automated cleanup at process exit because they are using the shared memory with processes not participating in Python's resource tracking to use the shared_memory API.
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Refactor delete-safe symbol handling in subprocess.
Only module globals are force-cleared during interpreter finalization, using a class reference instead of individually listing the constants everywhere is simpler.
Use `_from_parsed_parts()` to create a pre-joined/pre-parsed path, rather
than passing multiple arguments to `with_segments()`
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Restore `subprocess`'s intended use of `vfork()` by default for performance on Linux;
also fixes the behavior of `extra_groups=[]` which was unintentionally broken in 3.12.0:
Fixed a performance regression in 3.12's :mod:`subprocess` on Linux where it
would no longer use the fast-path ``vfork()`` system call when it could have
due to a logic bug, instead falling back to the safe but slower ``fork()``.
Also fixed a security bug introduced in 3.12.0. If a value of ``extra_groups=[]``
was passed to :mod:`subprocess.Popen` or related APIs, the underlying
``setgroups(0, NULL)`` system call to clear the groups list would not be made
in the child process prior to ``exec()``.
The security issue was identified via code inspection in the process of
fixing the first bug. Thanks to @vain for the detailed report and
analysis in the initial bug on Github.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Work around a macOS bug, limit zlib crc32 calls to 1GiB.
Without this, `zlib.crc32` and `binascii.crc32` could produce incorrect
results on multi-gigabyte inputs depending on the macOS version's Apple
supplied zlib implementation.
Change the declaration of the keywords parameter in functions
PyArg_ParseTupleAndKeywords() and PyArg_VaParseTupleAndKeywords() from `char **`
to `char * const *` in C and `const char * const *` in C++.
It makes these functions compatible with argument of type `const char * const *`,
`const char **` or `char * const *` in C++ and `char * const *` in C
without explicit type cast.
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Prevents a segmentation fault in registered hooks for the readline library, but only when the readline module is loaded inside an isolated sub interpreter. The module is single-phase init so loading it fails, but not until the module init function has already run, where the readline hooks get registered.
The readlinestate_global macro was error-prone to PyImport_FindModule returning NULL and crashing in about 18 places. I could reproduce 1 easily, but this PR replaces the macro with a function and adds error conditions to the other functions.
* bpo-32731: Raise OSError from any failure in getpass.getuser()
Previously, if the username was not set in certain environment variables, ImportError escaped on Windows systems, and it was possible for KeyError to escape on other systems if getpwuid() failed.
Add support for TLS-PSK (pre-shared key) to the ssl module.
---------
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Revert commit c8c0afc713 (PR #94532),
which moved `struct.Struct` initialisation from `Struct.__init__` to `Struct.__new__`.
This caused issues with code in the wild that subclasses `struct.Struct`.
Construct only one new list object (using `list.copy()`) when creating a
new path object with a modified tail. This slightly speeds up
`with_name()` and `with_suffix()`
Add upstream Tk patches for three problems affecting tkinter users:
- Update macOS installer to include a fix accepted by upstream Tcl/Tk
for a crash encountered after the first :meth:`tkinter.Tk` instance
is destroyed. (gh-92603)
- Update macOS installer to include an upstream Tcl/Tk fix
for the ``ttk::ThemeChanged`` error encountered in Tkinter. (gh-71383)
- Update macOS installer to include an upstream Tcl/Tk fix for the
``Secure coding is not enabled for restorable state!`` warning
encountered in Tkinter on macOS 14 Sonoma. (gh-110950)
Co-authored-by: Ned Deily <nad@python.org>
This makes the Tier 2 interpreter a little faster.
I calculated by about 3%,
though I hesitate to claim an exact number.
This starts by doubling the trace size limit (to 512),
making it more likely that loops fit in a trace.
The rest of the approach is to only load
`oparg` and `operand` in cases that use them.
The code generator know when these are used.
For `oparg`, it will conditionally emit
```
oparg = CURRENT_OPARG();
```
at the top of the case block.
(The `oparg` variable may be referenced multiple times
by the instructions code block, so it must be in a variable.)
For `operand`, it will use `CURRENT_OPERAND()` directly
instead of referencing the `operand` variable,
which no longer exists.
(There is only one place where this will be used.)
This uses the new mechanism whereby certain uops
are replaced by others during translation,
using the `_PyUop_Replacements` table.
We further special-case the `_FOR_ITER_TIER_TWO` uop
to update the deoptimization target to point
just past the corresponding `END_FOR` opcode.
Two tiny code cleanups are also part of this PR.
Omit the `@interface_scope` from an IPv6 address when used as Host header by `http.client`.
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
Add docstrings to the debugger module. Fix two bugs: initialize Idb.botframe (should be in Bdb); In Idb.in_rpc_code, check whether prev_frame is None before trying to use it. Make other code changes.
Expand test_debugger coverage from 19% to 66%.
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* Run again test_ast_recursion_limit() on WASI platform.
* Add _testinternalcapi.get_c_recursion_remaining().
* Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and
test_trace_unpack_long_sequence() now adjust the maximum recursion
depth depending on the the remaining C recursion.
The "Check if generated files are up to date" job of GitHub Actions
now runs the "autoreconf -ivf -Werror" command instead of the "make
regen-configure" command to avoid depending on the external quay.io
server.
Add Tools/build/regen-configure.sh script to regenerate the configure
with an Ubuntu container image. The
"quay.io/tiran/cpython_autoconf:271" container image
(https://github.com/tiran/cpython_autoconf) is no longer used.
This avoids:
python3.13 Tools/unicode/makeunicodedata.py
python3.13: can't open file '.../build/debug/Tools/unicode/makeunicodedata.py': [Errno 2] No such file or directory
make: *** [Makefile:1498: regen-unicodedata] Error 2
Re-run `make regen-unicodedata` to update the script path in generated files.
Ronald Oussoren
Barney Gale
Brett Cannon
Hugo van Kemenade
Alois Klink
Gregory P. Smith
Ken Jin
Irit Katriel
Steve Dower
Mariusz Felisiak
Terry Jan Reedy
Zackery Spytz
Mark Shannon
Nikita Sobolev
Peter Lazorchak
Victor Stinner
Sam Gross
Serhiy Storchaka
Jamie Phan
AN Long
Stefano Rivera
Kristján Valur Jónsson
Rami
Grigoriev Semyon
Pablo Galindo Salgado
Alex Waygood
Guido van Rossum
wookie184
Filip Łapkiewicz
Itamar Oren
Hugo van Kemenade
Ege Akman
Brandt Bucher
Neil Schemenauer
Jeffrey Kintscher
Samet YASLAN
Ankit Kumar Pandey
Inada Naoki
Vinay Sajip
Gordon P. Hemsley
Yilei Yang
Xu Song
Jeff Allen
Kirill Podoprigora
Allison Karlitskaya
Yan Yanchii
Shantanu
Jason R. Coombs
Carey Metcalfe
Martijn Pieters
Seth Michael Larson
Ethan Furman
Erlend E. Aasland
Marat Idrisov
Seth Michael Larson
Illia Volochii
Stéphane Bidoul
Jakub Kulík
Carson Radtke
Raymond Hettinger
Mark Dickinson
Akshat Khandelwal
Pavol Babinčák
Diego Russo
beavailable
Jamie
J. Nick Koston
Sam James
achhina
colorfulappl
Sidney Markowitz
Søren Løvborg
andrewluotechnologies
Tian Gao
Kushal Das
Miro Hrončok
pan324
Constantin Hong
Russell Keith-Magee
Kamil Turek
Thomas Bininda
chilaxan
William Wen
Rémi Lapeyre
Matt Prodani
Jelle Zijlstra
Anthony Shaw
apaz
kale-smoothie
Jacob Walls
Grant Ramsay
Randolf Scholz
Thomas Wouters
Junya Okabe
Christopher Chavez
Crowthebird
Michael
Tomas R