Add optional 'filter' parameter to iterdump() that allows a "LIKE"
pattern for filtering database objects to dump.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Fix race between `_PyParkingLot_Park` and `_PyParkingLot_UnparkAll` when handling interrupts
There is a potential race when `_PyParkingLot_UnparkAll` is executing in
one thread and another thread is unblocked because of an interrupt in
`_PyParkingLot_Park`. Consider the following scenario:
1. Thread T0 is blocked[^1] in `_PyParkingLot_Park` on address `A`.
2. Thread T1 executes `_PyParkingLot_UnparkAll` on address `A`. It
finds the `wait_entry` for `T0` and unlinks[^2] its list node.
3. Immediately after (2), T0 is woken up due to an interrupt. It
then segfaults trying to unlink[^3] the node that was previously
unlinked in (2).
To fix this we mark each waiter as unparking before releasing the bucket
lock. `_PyParkingLot_Park` will wait to handle the coming wakeup, and not
attempt to unlink the node, when this field is set. `_PyParkingLot_Unpark`
does this already, presumably to handle this case.
* When called with a single argument to get a value, it allow to omit
the minus prefix.
* It can be called with keyword arguments to set attributes.
* w.wm_attributes(return_python_dict=True) returns a dict instead of
a tuple (it will be the default in future).
* Setting wantobjects to 0 no longer affects the result.
When cvs.Error is raised when TypeError is caught,
the TypeError display and 'During handling' note is just noise
with duplicate information. Suppress with 'from None'.
Add test annotations required to run the test suite on iOS (PEP 730).
The majority of the change involve annotating tests that use subprocess,
but are skipped on Emscripten/WASI for other reasons, and including
iOS/tvOS/watchOS under the same umbrella as macOS/darwin checks.
`is_apple` and `is_apple_mobile` test helpers have been added to
identify *any* Apple platform, and "any Apple platform except macOS",
respectively.
Previously, "tag_unbind(tag, sequence, funcid)" methods of Text and
Canvas widgets destroyed the current binding for "sequence", leaving
"sequence" unbound, and deleted the "funcid" command.
Now they remove only "funcid" from the binding for "sequence", keeping
other commands, and delete the "funcid" command.
They leave "sequence" unbound only if "funcid" was the last bound command.
* Fix a RuntimeWarning emitted when assign an integer-like value that
is not an instance of int to an attribute that corresponds to a C
struct member of type T_UINT and T_ULONG.
* Fix a double RuntimeWarning emitted when assign a negative integer value
to an attribute that corresponds to a C struct member of type T_UINT.
Update documentation with `__new__` and `__init__` entries.
Support use of `auto()` in tuple subclasses on member assignment lines. Previously, auto() was only supported on the member definition line either solo or as part of a tuple:
RED = auto()
BLUE = auto(), 'azul'
However, since Python itself supports using tuple subclasses where tuples are expected, e.g.:
from collections import namedtuple
T = namedtuple('T', 'first second third')
def test(one, two, three):
print(one, two, three)
test(*T(4, 5, 6))
# 4 5 6
it made sense to also support tuple subclasses in enum definitions.
If the option with argument has short and long names,
output argument only once, after the long name:
-o, --option ARG description
instead of
-o ARG, --option ARG description
The new `PyList_GetItemRef` is similar to `PyList_GetItem`, but returns
a strong reference instead of a borrowed reference. Additionally, if the
passed "list" object is not a list, the function sets a `TypeError`
instead of calling `PyErr_BadInternalCall()`.
Also fix test_mousewheel: do not skip a check which was broken due to incorrect
delta on Aqua and XQuartz, and probably not because of `.update_idletasks()`.
* gh-112529: Remove PyGC_Head from object pre-header in free-threaded build
This avoids allocating space for PyGC_Head in the free-threaded build.
The GC implementation for free-threaded CPython does not use the
PyGC_Head structure.
* The trashcan mechanism uses the `ob_tid` field instead of `_gc_prev`
in the free-threaded build.
* The GDB libpython.py file now determines the offset of the managed
dict field based on whether the running process is a free-threaded
build. Those are identified by the `ob_ref_local` field in PyObject.
* Fixes `_PySys_GetSizeOf()` which incorrectly incorrectly included the
size of `PyGC_Head` in the size of static `PyTypeObject`.
---------
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
PyObject_GetBuffer() now raises a SystemError if called with
PyBUF_READ or PyBUF_WRITE as flags. These flags should
only be used with the PyMemoryView_* C API.
Return files and directories from `pathlib.Path.glob()` if the pattern ends
with `**`. This is more compatible with `PurePath.full_match()` and with
other glob implementations such as bash and `glob.glob()`. Users can add a
trailing slash to match only directories.
In my previous patch I added a `FutureWarning` with the intention of fixing
this in Python 3.15. Upon further reflection I think this was an
unnecessarily cautious remedy to a clear bug.
Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
My criterion for delayed imports is that they're only worth it if the
majority of users of the module would benefit from it, otherwise you're
just moving latency around unpredictably.
mktime_tz is not used anywhere in the standard library and grep.app
indicates it's not got much use in the ecosystem either.
Distribution.files is not nearly as widely used as other
importlib.metadata APIs, so we defer the csv import.
Before:
```
λ hyperfine -w 8 './python -c "import importlib.metadata"'
Benchmark 1: ./python -c "import importlib.metadata"
Time (mean ± σ): 65.1 ms ± 0.5 ms [User: 55.3 ms, System: 9.8 ms]
Range (min … max): 64.4 ms … 66.4 ms 44 runs
```
After:
```
λ hyperfine -w 8 './python -c "import importlib.metadata"'
Benchmark 1: ./python -c "import importlib.metadata"
Time (mean ± σ): 62.0 ms ± 0.3 ms [User: 52.5 ms, System: 9.6 ms]
Range (min … max): 61.3 ms … 62.8 ms 46 runs
```
for about a 3ms saving with warm disk cache, maybe 7-11ms with cold disk
cache.
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch (https://fredrikbk.com/publications/copy-and-patch.pdf).
See Tools/jit/README.md for more information on how to install the required build-time tooling.
For interpreters that share state with the main interpreter, this points
to the same static memory structure. For interpreters with their own
obmalloc state, it is heap allocated. Add free_obmalloc_arenas() which
will free the obmalloc arenas and radix tree structures for interpreters
with their own obmalloc state.
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
``platform.architecture()`` now returns the format of binaries (e.g. Mach-O) instead of the default empty string.
Co-authored-by: AN Long <aisk@users.noreply.github.com>
Add `ntpath.isreserved()`, which identifies reserved pathnames such as "NUL", "AUX" and "CON".
Deprecate `pathlib.PurePath.is_reserved()`.
---------
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Co-authored-by: Brett Cannon <brett@python.org>
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
`threading.Lock` is now the underlying class and is constructable rather than the old
factory function. This allows for type annotations to refer to it which had no non-ugly
way to be expressed prior to this.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* gh-112529: Implement GC for free-threaded builds
This implements a mark and sweep GC for the free-threaded builds of
CPython. The implementation relies on mimalloc to find GC tracked
objects (i.e., "containers").
The iterator returned by ElementTree.iterparse() may hold on to a file
descriptor. The reference cycle prevented prompt clean-up of the file
descriptor if the returned iterator was not exhausted.
* use the ParkingLot API to manage waiting threads
* use Argument Clinic's critical section directive to protect queue methods
* remove unnecessary overflow check
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Always set a _MainThread as a main thread after os.fork() is called from
a thread started not by the threading module.
A new _MainThread was already set as a new main thread after fork if
threading.current_thread() was not called for a foreign thread before fork.
Now, if it was called before fork, the implicitly created _DummyThread will
be turned into _MainThread after fork.
It fixes, in particularly, an incompatibility of _DummyThread with
the threading shutdown logic which relies on the main thread
having tstate_lock.
Co-authored-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
The terminal CR -> NL mapping setting should be inherited in cbreak mode as OSes do not specify altering it as part of their stty cbreak mode definition.
Use `_make_child_entry()` rather than `_make_child_relpath()` to retrieve
path objects for directories to visit. This saves the allocation of one
path object per directory in user subclasses of `PathBase`, and avoids a
second loop.
This trick does not apply when walking top-down, because users can affect
the walk by modifying *dirnames* in-place.
A side effect of this change is that, in bottom-up mode, subdirectories of
each directory are visited in reverse order, and that this order doesn't
match that of the names in *dirnames*. I suspect this is fine as the
order is arbitrary anyway.
Allow `os.PathLike` objects to be passed as patterns to `pathlib.Path.glob()` and `rglob()`. (It's already possible to use them in `PurePath.match()`)
While we're in the area:
- Allow empty glob patterns in `PathBase` (but not `Path`)
- Speed up globbing in `PathBase` by generating paths with trailing slashes only as a final step, rather than for every intermediate directory.
- Simplify and speed up handling of rare patterns involving both `**` and `..` segments.
Only set filename to cwd if it was caused by failed chdir(cwd).
_fork_exec() now returns "noexec:chdir" for failed chdir(cwd).
Co-authored-by: Robert O'Shea <PurityLake@users.noreply.github.com>
Tkinter is a fact, not necessarily a feature.
Reorganize editor key bindings in a logical order
and remove those that do not work, at least on Windows.
Improve shell bindings list.
If *trackfd* is False, the file descriptor specified by *fileno*
will not be duplicated.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Small performance improvement of getproxies_environment() when there are many environment variables. In a benchmark with 5k environment variables not related to proxies, and 5 specifying proxies, we get a 10% walltime improvement.
`PyComplex_RealAsDouble()`/`PyComplex_ImagAsDouble` now try to convert
an object to a `complex` instance using its `__complex__()` method
before falling back to the ``__float__()`` method.
PyComplex_ImagAsDouble() also will not silently return 0.0 for
non-complex types anymore. Instead we try to call PyFloat_AsDouble()
and return 0.0 only if this call is successful.
Fix a bug in the regex used for parsing a string input to the `fractions.Fraction` constructor. That bug led to an inconsistent exception message being given for some inputs.
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
Increase the backlog for multiprocessing.connection.Listener` objects created
by `multiprocessing.manager` and `multiprocessing.resource_sharer` to
significantly reduce the risk of getting a connection refused error when creating
a `multiprocessing.connection.Connection` to them.
On Windows, `os.path.isabs()` now returns `False` when given a path that
starts with exactly one (back)slash. This is more compatible with other
functions in `os.path`, and with Microsoft's own documentation.
Also adjust `pathlib.PureWindowsPath.is_absolute()` to call
`ntpath.isabs()`, which corrects its handling of partial UNC/device paths
like `//foo`.
Co-authored-by: Jon Foster <jon@jon-foster.co.uk>
Make zipfile.ZipInfo.compress_level public.
A property is used to retain the behavior of the ._compresslevel.
People constructing zipfile.ZipInfo instances to pass into existing APIs to control per-file compression levels already treat this as public, there was never a reason for it not to be.
I used the more modern name compress_level instead of compresslevel as the keyword argument on other ZipFile APIs is called to be consistent with compress_type and a general long term preference of not runningwordstogether without a separator in names.
sqlite3.Connection.iterdump now ensures that foreign key support is
disabled before dumping the database schema, if there is any foreign key
violation.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
test_configdialog.HighPageTest.test_highlight_target_text_mouse fails
if a line of the Highlight tab text sample is not visible. If so, bbox()
in click_char() returns None and the unpacking iteration fails.
This occurred on a Devuan Linux system. Fix by moving the
'see character' call inside click_char, just before the bbox call.
Also, reduce the click_char calls to just one per tag name and
replace the other nested function with a dict comprehension.
On Windows, set _O_NOINHERIT flag on file descriptors
created by os.pipe() and io.WindowsConsoleIO.
Add test_pipe_spawnl() to test_os.
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Previously the C implementation of pickle.Pickler and pickle.Unpickler
classes did not have such methods and they could only be used if
they were overloaded in subclasses or set as instance attributes.
Fixed calling super().persistent_id() and super().persistent_load() in
subclasses of the C implementation of pickle.Pickler and pickle.Unpickler
classes. It no longer causes an infinite recursion.
If other exception was raised during exiting an expired
asyncio.timeout() block, insert TimeoutError in the exception context
just above the CancelledError.
winfo_id() converts the result of "winfo id" command to integer, but
"winfo pathname" command requires an argument to be a hexadecimal number
on Win64.
When an `StopIteration` raises into `asyncio.Future`, this will cause
a thread to hang. This commit address this by not raising an exception
and silently transforming the `StopIteration` with a `RuntimeError`,
which the caller can reconstruct from `fut.exception().__cause__`
It can be used to set the location of a .python_history file
---------
Co-authored-by: Levi Sabah <0xl3vi@gmail.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Allow extraction (no-op) of a "/" folder in a zipfile, they are commonly added by some archive creation tools.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Remove LibreSSL specific workaround ifdefs from `_ssl.c` and delete the non-version-specific `_ssl_data.h` file (relevant for OpenSSL < 1.1.1, which we no longer support per PEP 644).
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Raise auditing events in `pathlib.Path.glob()`, `rglob()` and `walk()`,
but not in `pathlib._abc.PathBase` methods. Also move generation of a
deprecation warning into `pathlib.Path` so it gets the right stack level.
- Only attempt to figure out whether protocol members are "method members" or not if the class is marked as a runtime protocol. This information is irrelevant for non-runtime protocols; we can safely skip the risky introspection for them.
- Only do the risky getattr() calls in one place (the runtime_checkable class decorator), rather than in three places (_ProtocolMeta.__init__, _ProtocolMeta.__instancecheck__ and _ProtocolMeta.__subclasscheck__). This reduces the number of locations in typing.py where the risky introspection could go wrong.
- For runtime protocols, if determining whether a protocol member is callable or not fails, give a better error message. I think it's reasonable for us to reject runtime protocols that have members which raise strange exceptions when you try to access them. PEP-544 clearly states that all protocol member must be callable for issubclass() calls against the protocol to be valid -- and if a member raises when we try to access it, there's no way for us to figure out whether it's a callable member or not!
This ensures the source directory is not modified at build time, and different builds (e.g. different versions or GIL vs no-GIL) do not have conflicts.
* gh-113536: Expose `os.waitid` on macOS
This API has been available on macOS for a long time, but was
explicitly excluded due to unspecified problems with the API
in ancient versions of macOS.
* Document that the API is available on macOS starting in Python 3.13
The length field of StgDictObject for Structure class contains now
the total number of items in ffi_type_pointer.elements (excluding
the trailing null).
The old behavior of using the number of elements in the parent class can
cause the array to be truncated when it is copied, especially when there
are multiple layers of subclassing.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Previously some error messages complained about incompatible
combinations of directives that are not contained in the format string.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
First fix resolve situation when pyexpat module (which contains expat_CAPI
capsule) deallocates before _elementtree, so we need to hold a strong
reference to pyexpat module to.
Second fix resolve situation when module state is deallocated before
deallocation of XMLParser instances, which uses module state to clear
some stuff.
Mariusz Felisiak
Erlend E. Aasland
Ned Deily
Zachary Ware
Serhiy Storchaka
mpage
Mark Shannon
Terry Jan Reedy
Russell Keith-Magee
Nikita Sobolev
Ethan Furman
Stéphane Bidoul
Travis Howse
Kristján Valur Jónsson
Malcolm Smith
Jokimax
Andrew Rogers
Sam Gross
Christopher Chavez
Ayappan Perumal
technillogue
Jamie Phan
Albert Zeyer
Nachtalb
Tian Gao
Daniel Hollas
Diego Russo
Barney Gale
Eugene Toder
Steven Ward
Soumendra Ganguly
Shantanu
Brandt Bucher
Furkan Onder
Hugo van Kemenade
Neil Schemenauer
Aiden Fox Ivey
Rito Takeuchi
Vincent Cunningham
Brett Cannon
Fabio Zadrozny
AN Long
Irit Katriel
Gregory P. Smith
Skip Montanaro
keithasaurus
Phillip Schanely
kcatss
Steve Dower
buermarc
Sheidan
Thomas Wouters
Radislav Chugunov
Mike Zimin
Jérome Perrin
Zackery Spytz
Raphaël Marinier
Kamil Turek
Sergey B Kirpichev
Ronald Oussoren
Crowthebird
Alois Klink
Ken Jin
Peter Lazorchak
Victor Stinner
Stefano Rivera
Rami
Grigoriev Semyon
Pablo Galindo Salgado
Alex Waygood
Guido van Rossum
wookie184
Filip Łapkiewicz
Itamar Oren
Hugo van Kemenade
Jeffrey Kintscher
Samet YASLAN
Ankit Kumar Pandey
Inada Naoki
Vinay Sajip
Gordon P. Hemsley
Yilei Yang
Xu Song
Jeff Allen
Kirill Podoprigora