Commit Graph

8693 Commits

Author SHA1 Message Date
Will Childs-Klein c85d84166a
gh-116333: Relax error string text expectations in SSL-related tests (GH-116334)
* Relax error string text expectations in SSL-related tests

As suggested [here][1], this change relaxes the OpenSSL error string
text expectations in a number of tests. This was specifically done in
support of more easily building CPython [AWS-LC][2], but because AWS-LC
is a fork of [BoringSSL][3], it should increase compatibility with that
library as well.

In addition to the error string relaxations, we also add some guards
around the `tls-unique` channel binding being used with TLSv1.3, as that
feature (described in [RFC 6929][4]) is [not defined][5] for TLSv1.3.

[1]: https://discuss.python.org/t/support-building-ssl-and-hashlib-modules-against-aws-lc/44505/4
[2]: https://github.com/aws/aws-lc
[3]: https://github.com/google/boringssl
[4]: https://datatracker.ietf.org/doc/html/rfc5929#section-3
[5]: https://datatracker.ietf.org/doc/html/rfc8446#appendix-C.5
2024-03-21 19:16:36 +00:00
Victor Stinner abdd1f938f
gh-85283: Build _testconsole extension with limited C API (#117125) 2024-03-21 17:45:43 +01:00
Victor Stinner 8bea6c411d
gh-115754: Add Py_GetConstant() function (#116883)
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.

In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.

Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
2024-03-21 16:07:00 +00:00
Malcolm Smith 1f8b24ef69
gh-71052: Implement `ctypes.util.find_library` on Android (GH-116379) 2024-03-21 14:20:57 +01:00
Tian Gao d16c9d1278
gh-116987: Support class code objects in inspect.findsource() (GH-117025) 2024-03-21 10:30:10 +00:00
Adam Turner 6547330f4e
GH-109653: Defer import of ``importlib.metadata._adapters`` (#109829)
* adapters

* Add comments for deferred imports with links to rationale.

* Add blurb

---------

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2024-03-21 03:49:10 +00:00
Jason R. Coombs 8ad8898420
gh-117089: Move importlib.metadata tests to their own package (#117092)
* Ensure importlib.metadata tests do not leak references in sys.modules.

* Move importlib.metadata tests to their own package for easier syncing with importlib_metadata.

* Update owners and makefile for new directories.

* Add blurb
2024-03-20 17:11:00 -04:00
Petr Viktorin dcaf33a41d
gh-114314: ctypes: remove stgdict and switch to heap types (GH-116458)
Before this change, ctypes classes used a custom dict subclass, `StgDict`,
as their `tp_dict`. This acts like a regular dict but also includes extra information
about the type.

This replaces stgdict by `StgInfo`, a C struct on the type, accessed by
`PyObject_GetTypeData()` (PEP-697).
All usage of `StgDict` (mainly variables named `stgdict`, `dict`, `edict` etc.) is
converted to `StgInfo` (named `stginfo`, `info`, `einfo`, etc.).
Where the dict is actually used for class attributes (as a regular PyDict), it's now
called `attrdict`.

This change -- not overriding `tp_dict` -- is made to make me comfortable with
the next part of this PR: moving the initialization logic from `tp_new` to `tp_init`.

The `StgInfo` is set up in `__init__` of each class, with a guard that prevents
calling `__init__` more than once. Note that abstract classes (like `Array` or
`Structure`) are created using `PyType_FromMetaclass` and do not have
`__init__` called.
Previously, this was done in `__new__`, which also wasn't called for abstract
classes.
Since `__init__` can be called from Python code or skipped, there is a tested
guard to ensure `StgInfo` is initialized exactly once before it's used.

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-20 17:33:08 +01:00
jkriegshauser fc45998007
gh-116773: Ensure overlapped objects on Windows are not deallocated too early by asyncio (GH-116774) 2024-03-20 14:33:28 +00:00
Serhiy Storchaka 519b2ae22b
gh-117021: Fix integer overflow in PyLong_AsPid() on non-Windows 64-bit platforms (GH-117064) 2024-03-20 15:39:53 +02:00
Mark Shannon 15309329b6
GH-108362: Incremental Cycle GC (GH-116206) 2024-03-20 08:54:42 +00:00
Sam Gross 60e105c1c1
gh-113964: Don't prevent new threads until all non-daemon threads exit (#116677)
Starting in Python 3.12, we prevented calling fork() and starting new threads
during interpreter finalization (shutdown). This has led to a number of
regressions and flaky tests. We should not prevent starting new threads
(or `fork()`) until all non-daemon threads exit and finalization starts in
earnest.

This changes the checks to use `_PyInterpreterState_GetFinalizing(interp)`,
which is set immediately before terminating non-daemon threads.
2024-03-19 14:40:20 -04:00
et-repositories 75935746be
gh-116647: Fix recursive child in dataclasses (#116790) 2024-03-19 08:58:40 -06:00
Victor Stinner 27cf3ed00c
gh-90872: Fix subprocess.Popen.wait() for negative timeout (#116989)
On Windows, subprocess.Popen.wait() no longer calls
WaitForSingleObject() with a negative timeout: pass 0 ms if the
timeout is negative.
2024-03-19 14:42:44 +01:00
Russell Keith-Magee 408e127159
gh-114099 - Add iOS framework loading machinery. (GH-116454)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-03-19 08:36:19 -04:00
David Röthlisberger b1bc37597f
gh-116957: configparser: Do post-process values after DuplicateOptionError (GH-116958)
If you catch DuplicateOptionError / DuplicateSectionError when reading a
config file (the intention is to skip invalid config files) and then
attempt to use the ConfigParser instance, any values it *had* read
successfully so far, were stored as a list instead of string! Later
`get` calls would raise "AttributeError: 'list' object has no attribute
'find'" from somewhere deep in the interpolation code.
2024-03-19 11:59:08 +02:00
Bogdan Romanyuk a8e93d3dca
gh-115756: make PyCode_GetFirstFree an unstable API (GH-115781) 2024-03-19 09:20:38 +00:00
Aviel Boag a22d05f04c
gh-105866: fix dataclass with slots=True, weakref_slot=True (#105870)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-03-18 18:53:14 -06:00
Pierre Ossman (ThinLinc team) 4159644177
gh-113538: Add asycio.Server.{close,abort}_clients (redo) (#116784)
These give applications the option of more forcefully terminating client
connections for asyncio servers. Useful when terminating a service and
there is limited time to wait for clients to finish up their work.

This is a do-over with a test fix for gh-114432, which was reverted.
2024-03-18 13:15:53 -07:00
Victor Stinner a9c304cf02
gh-116869: Make C API compatible with ISO C90 (#116950)
Make the C API compatible with -Werror=declaration-after-statement
compiler flag again.
2024-03-18 20:16:58 +01:00
Victor Stinner 590a26010d
gh-116869: Add test_cext test: build a C extension (#116954) 2024-03-18 20:15:20 +01:00
Zachary Ware 849e0716d3
gh-115119: Switch Windows build to mpdecimal external (GH-115182)
This includes adding what should be a relatively temporary
`Modules/_decimal/windows/mpdecimal.h` shim to choose between `mpdecimal32vc.h`
or `mpdecimal64vc.h` based on which of `CONFIG_64` or `CONFIG_32` is defined.
2024-03-18 12:07:25 -05:00
Victor Stinner 1d95451be1
gh-63207: Use GetSystemTimePreciseAsFileTime() in time.time() (#116822) 2024-03-18 17:13:01 +01:00
kernc 52ef4430a9
gh-71765: Fix inspect.getsource() on empty file (GH-20809)
* bpo-27578: Fix inspect.getsource() on empty file

For modules from empty files, `inspect.getsource()` now
returns an empty string, and `inspect.getsourcelines()` returns
a list of one empty string, fixing the expected invariant.

As indicated by `exec('')`, empty strings are valid Python
source code.

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2024-03-18 16:13:02 +01:00
AN Long cd2ed91780
gh-115538: Emit warning when use bool as fd in _io.WindowsConsoleIO (GH-116925) 2024-03-18 11:48:50 +00:00
Victor Stinner 2982bdb936
gh-85283: Build _statistics extension with the limited C API (#116927)
Argument Clinic now inlines _PyArg_CheckPositional() for the limited
C API. The generated code should be as fast or even a little bit
faster.
2024-03-17 18:59:02 +01:00
Victor Stinner 1cf0301086
gh-85283: Build termios extension with the limited C API (#116928) 2024-03-17 15:12:29 +00:00
Victor Stinner 8e3c953b3a
gh-73468: Add math.fma() function (#116667)
Added new math.fma() function, wrapping C99's ``fma()`` operation:
fused multiply-add function.

Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
2024-03-17 13:58:26 +00:00
John Sloboda 649857a157
gh-85287: Change codecs to raise precise UnicodeEncodeError and UnicodeDecodeError (#113674)
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2024-03-17 04:58:42 +00:00
Victor Stinner 5e0a070dfe
gh-116809: Restore removed _PyErr_ChainExceptions1() function (#116900) 2024-03-16 21:37:11 +01:00
mpage 33da0e844c
gh-114271: Fix race in `Thread.join()` (#114839)
There is a race between when `Thread._tstate_lock` is released[^1] in `Thread._wait_for_tstate_lock()`
and when `Thread._stop()` asserts[^2] that it is unlocked. Consider the following execution
involving threads A, B, and C:

1. A starts.
2. B joins A, blocking on its `_tstate_lock`.
3. C joins A, blocking on its `_tstate_lock`.
4. A finishes and releases its `_tstate_lock`.
5. B acquires A's `_tstate_lock` in `_wait_for_tstate_lock()`, releases it, but is swapped
   out before calling `_stop()`.
6. C is scheduled, acquires A's `_tstate_lock` in `_wait_for_tstate_lock()` but is swapped
   out before releasing it.
7. B is scheduled, calls `_stop()`, which asserts that A's `_tstate_lock` is not held.
   However, C holds it, so the assertion fails.

The race can be reproduced[^3] by inserting sleeps at the appropriate points in
the threading code. To do so, run the `repro_join_race.py` from the linked repo.

There are two main parts to this PR:

1. `_tstate_lock` is replaced with an event that is attached to `PyThreadState`.
   The event is set by the runtime prior to the thread being cleared (in the same
   place that `_tstate_lock` was released). `Thread.join()` blocks waiting for the
   event to be set.
2. `_PyInterpreterState_WaitForThreads()` provides the ability to wait for all
   non-daemon threads to exit. To do so, an `is_daemon` predicate was added to
   `PyThreadState`. This field is set each time a thread is created. `threading._shutdown()`
   now calls into `_PyInterpreterState_WaitForThreads()` instead of waiting on
   `_tstate_lock`s.

[^1]: 441affc9e7/Lib/threading.py (L1201)
[^2]: 441affc9e7/Lib/threading.py (L1115)
[^3]: 8194653279

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
2024-03-16 13:56:30 +01:00
Serhiy Storchaka c61cb507c1
gh-116484: Fix collisions between Checkbutton and ttk.Checkbutton default names (GH-116495)
Change automatically generated tkinter.Checkbutton widget names to
avoid collisions with automatically generated tkinter.ttk.Checkbutton
widget names within the same parent widget.
2024-03-16 13:31:19 +02:00
Serhiy Storchaka 1069a462f6
gh-116764: Fix regressions in urllib.parse.parse_qsl() (GH-116801)
* Restore support of None and other false values.
* Raise TypeError for non-zero integers and non-empty sequences.

The regressions were introduced in gh-74668
(bdba8ef42b).
2024-03-16 12:36:05 +02:00
Serhiy Storchaka 269051d20e
gh-90535: Fix support of interval>1 in logging.TimedRotatingFileHandler (GH-116220)
Fix support of interval values > 1 in logging.TimedRotatingFileHandler
for when='MIDNIGHT' and when='Wx'.
2024-03-16 12:29:42 +02:00
Donghee Na ebf29b3a02
gh-112536: Add --tsan test for reasonable TSAN execution times. (gh-116601) 2024-03-16 01:07:16 +09:00
Tian Gao 59e30f41ed
gh-116735: Use `MISSING` for `CALL` event if argument is absent (GH-116737) 2024-03-15 14:46:18 +00:00
Zackery Spytz d180b507c4
gh-63283: IDNA prefix should be case insensitive (GH-17726)
Any capitalization of "xn--" should be acceptable for the ACE prefix
(see https://tools.ietf.org/html/rfc3490#section-5).

Co-authored-by: Pepijn de Vos <pepijndevos@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-03-15 15:38:13 +01:00
Tian Gao a50cf6c3d7
gh-90095: Ignore empty lines and comments in `.pdbrc` (#116834) 2024-03-15 09:36:04 +00:00
Victor Stinner 8fc8fbb43a
gh-85283: Build pwd extension with the limited C API (#116841)
Argument Clinic now uses the PEP 737 "%T" format to format type name
for the limited C API.
2024-03-15 08:49:58 +01:00
vxiiduu be1c808fca
gh-116195: Implements a fast path for nt.getppid (GH-116205)
Use the NtQueryInformationProcess system call to efficiently retrieve the parent process ID in a single step, rather than using the process snapshots API which retrieves large amounts of unnecessary information and is more prone to failure (since it makes heap allocations).

Includes a fallback to the original win32_getppid implementation in case the unstable API appears to return strange results.
2024-03-14 23:09:36 +00:00
Victor Stinner 7bbb9b57e6
gh-111696, PEP 737: Add %T and %N to PyUnicode_FromFormat() (#116839) 2024-03-14 22:23:00 +00:00
Jason R. Coombs 5f52d20a93
gh-116811: Ensure MetadataPathFinder.invalidate_caches is reachable when delegated through PathFinder. (#116812)
* Make MetadataPathFinder a proper classmethod.

* In PathFinder.invalidate_caches, also invoke MetadataPathFinder.invalidate_caches.

* Add blurb
2024-03-14 17:59:00 -04:00
Jason R. Coombs be59aaf3ab
gh-106531: Refresh zipfile._path with zipp 3.18. (#116835)
* gh-106531: Refresh zipfile._path with zipp 3.18.

* Add blurb
2024-03-14 21:53:50 +00:00
Victor Stinner c432df6d56
gh-111696, PEP 737: Add PyType_GetModuleName() function (#116824)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-03-14 18:17:43 +00:00
Victor Stinner 19c3a2ff91
gh-111696, PEP 737: Add PyType_GetFullyQualifiedName() function (#116815)
Rewrite tests on type names in Python, they were written in C.
2024-03-14 16:19:36 +00:00
Victor Stinner 846ad5a26a
gh-88494: Use QueryPerformanceCounter() for time.monotonic() (#116781)
On Windows, time.monotonic() now uses the QueryPerformanceCounter()
clock to have a resolution better than 1 us, instead of the
gGetTickCount64() clock which has a resolution of 15.6 ms.
2024-03-14 16:42:41 +01:00
Victor Stinner 97b80af897
gh-85283: Build fcntl extension with the limited C API (#116791) 2024-03-14 12:01:13 +00:00
Nikita Sobolev 61f576a5ef
gh-113308: Remove some internal parts of `uuid` module (#115934)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-03-14 13:01:41 +03:00
Brett Cannon 3a25d9c5a9
GH-114736: Use WASI SDK 21 (GH-116771) 2024-03-14 01:45:33 +00:00
Brett Cannon 8c094c3095
GH-115983: skip building shared modules for testing under WASI (GH-116528) 2024-03-13 23:25:50 +00:00