Commit Graph

29403 Commits

Author SHA1 Message Date
Alex Waygood 0f8a07d158
gh-118418: Deprecate failing to pass a value to the *type_params* parameter of some private `typing` APIs (#118695) 2024-05-07 11:12:28 +00:00
chrysn 1e428426c8
gh-118650: Exclude `_repr_*` methods from Enum's _sunder_ reservation (GH-118651) 2024-05-07 12:35:51 +02:00
Hugo van Kemenade 6cf70678e2
gh-109975: Update 'What's New in Python 3.13' for beta (#118694) 2024-05-07 12:35:32 +02:00
Youfu Zhang fe47d9bee3
gh-118314: Fix padding edge case in binascii.a2b_base64 strict mode (GH-118320)
Fix an edge case in `binascii.a2b_base64` strict mode, where
excessive padding was not detected when no padding is necessary.

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
2024-05-07 11:18:45 +02:00
Ali Hamdan de1428f8c2
gh-62090: Simplify argparse usage formatting (GH-105039)
Rationale
=========

argparse performs a complex formatting of the usage for argument grouping
and for line wrapping to fit the terminal width. This formatting has been
a constant source of bugs for at least 10 years (see linked issues below)
where defensive assertion errors are triggered or brackets and paranthesis
are not properly handeled.

Problem
=======

The current implementation of argparse usage formatting relies on regular
expressions to group arguments usage only to separate them again later
with another set of regular expressions. This is a complex and error prone
approach that caused all the issues linked below. Special casing certain
argument formats has not solved the problem. The following are some of
the most common issues:
- empty `metavar`
- mutually exclusive groups with `SUPPRESS`ed arguments
- metavars with whitespace
- metavars with brackets or paranthesis

Solution
========

The following two comments summarize the solution:
- https://github.com/python/cpython/issues/82091#issuecomment-1093832187
- https://github.com/python/cpython/issues/77048#issuecomment-1093776995

Mainly, the solution is to rewrite the usage formatting to avoid the
group-then-separate approach. Instead, the usage parts are kept separate
and only joined together at the end. This allows for a much simpler
implementation that is easier to understand and maintain. It avoids the
regular expressions approach and fixes the corresponding issues.

This closes the following GitHub issues:
-  #62090
-  #62549
-  #77048
-  #82091
-  #89743
-  #96310
-  #98666

These PRs become obsolete:
-  #15372
-  #96311
2024-05-07 09:28:51 +02:00
John Sirois 49258efada
gh-118107: Fix zipimporter ZIP64 handling. (GH-118108)
Add missing import to code that handles too large files and offsets.
Use list, not tuple, for a mutable sequence.

Add tests to prevent similar mistakes.

---------

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-05-07 09:23:27 +02:00
Tian Gao 0d9148823d
gh-118414: Fix assertion in YIELD_VALUE when tracing lines or instrs (#118683) 2024-05-06 21:22:59 -07:00
Eric Snow b2cd54a4fb
gh-117953: Always Run Extension Init Func in Main Interpreter First (gh-118157)
This change makes sure all extension/builtin modules have their init function run first by the main interpreter before proceeding with import in the original interpreter (main or otherwise).  This means when the import of a single-phase init module fails in an isolated subinterpreter, it won't tie any global state/callbacks to the subinterpreter.
2024-05-07 04:21:51 +00:00
Brett Simmers 853163d3b5
gh-116322: Enable the GIL while loading C extension modules (#118560)
Add the ability to enable/disable the GIL at runtime, and use that in
the C module loading code.

We can't know before running a module init function if it supports
free-threading, so the GIL is temporarily enabled before doing so. If
the module declares support for running without the GIL, the GIL is
later disabled. Otherwise, the GIL is permanently enabled, and will
never be disabled again for the life of the current interpreter.
2024-05-06 23:07:23 -04:00
James Hilton-Balfe e7dafdc224
gh-110209: Add __class_getitem__ for generator and coroutine (#110212) 2024-05-06 18:41:32 -07:00
Jeong, YunWon 8d8275b0cf
gh-118473: Fix set_asyncgen_hooks not to be partially set when arguments are invalid (#118474) 2024-05-06 17:02:52 -07:00
Jelle Zijlstra e0422198fb
gh-117486: Improve behavior for user-defined AST subclasses (#118212)
Now, such classes will no longer require changes in Python 3.13 in the normal case.
The test suite for robotframework passes with no DeprecationWarnings under this PR.

I also added a new DeprecationWarning for the case where `_field_types` exists
but is incomplete, since that seems likely to indicate a user mistake.
2024-05-06 15:57:27 -07:00
Jelle Zijlstra 8419f01673
gh-118647: Add defaults to typing.Generator and typing.AsyncGenerator (#118648)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-05-06 22:35:06 +00:00
Erlend E. Aasland 325a1da18d
gh-115119: Default to --with-system-libmpdec=yes (#118539)
Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
2024-05-06 21:16:08 +02:00
Tian Gao e5353d49dc
GH-83151: Add closure support to pdb (GH-111094) 2024-05-06 11:34:13 -07:00
Dino Viehland 5a1618a2c8
gh-118362: Fix thread safety around lookups from the type cache in the face of concurrent mutators (#118454)
Add _PyType_LookupRef and use incref before setting attribute on type
Makes setting an attribute on a class and signaling type modified atomic
Avoid adding re-entrancy exposing the type cache in an inconsistent state by decrefing after type is updated
2024-05-06 10:50:35 -07:00
Thomas Grainger e5c699280d
GH-117714: implement athrow().close() and asend().close() using throw (GH-117906)
* GH-117714: replace athrow().close() and asend().close() stubs with implimentations

* test athrow().close() and asend().close() raises RuntimeError

* 📜🤖 Added by blurb_it.

* Update Objects/genobject.c

Co-authored-by: Petr Viktorin <encukou@gmail.com>

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-05-06 17:13:15 +00:00
Petr Viktorin 417dd3aca7
gh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Module_SetGIL (GH-118645) 2024-05-06 18:59:36 +02:00
Serhiy Storchaka 709ca90a00
gh-118271: Support more options for reading/writing images in Tkinter (GH-118273)
* Add PhotoImage.read() to read an image from a file.
* Add PhotoImage.data() to get the image data.
* Add background and grayscale parameters to PhotoImage.write().
2024-05-06 15:06:06 +00:00
Serhiy Storchaka 1b639a04ca
gh-118225: Support more options for copying images in Tkinter (GH-118228)
* Add the PhotoImage method copy_replace() to copy a region
  from one image to other image, possibly with pixel zooming and/or
  subsampling.
* Add from_coords parameter to PhotoImage methods copy(), zoom() and subsample().
* Add zoom and subsample parameters to PhotoImage method copy().
2024-05-06 17:33:15 +03:00
Serhiy Storchaka 0085c3ae8f
gh-116871: Improve name suggestions in tracebacks (GH-116930)
Only include underscored names in name suggestions for AttributeError and
ImportError if the original name was underscored.
2024-05-06 15:53:15 +03:00
Serhiy Storchaka d6fa1d4bee
gh-66543: Add mimetypes.guess_file_type() (GH-117258) 2024-05-06 15:50:52 +03:00
Serhiy Storchaka 153b3f7530
gh-118465: Add __firstlineno__ attribute to class (GH-118475)
It is set by compiler with the line number of the first line of
the class definition.
2024-05-06 12:02:37 +03:00
Pieter Eendebak 05adfbba2a
gh-95382: Improve performance of json encoder with indent (GH-118105) 2024-05-06 11:04:39 +03:00
Terry Jan Reedy 7758be4318
gh-78955: Use user-selected color theme for Help => IDLE Doc (#9502) 2024-05-06 03:55:56 -04:00
Pablo Galindo Salgado 2eb99914c4
Add Lysandros Nikolaou to the News entry of gh-111201 (#118616) 2024-05-05 22:13:50 +00:00
Pablo Galindo Salgado f27f8c790a
gh-111201: A new Python REPL (GH-111567)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-05-05 21:32:23 +02:00
Guido van Rossum 40cc809902
gh-117549: Don't use designated initializers in headers (#118580)
The designated initializer syntax in static inline functions in pycore_backoff.h
causes problems for C++ or MSVC users who aren't yet using C++20.
While internal, pycore_backoff.h is included (indirectly, via pycore_code.h)
by some key 3rd party software that does so for speed.
2024-05-05 19:28:55 +00:00
Guido van Rossum 9c13d9e37a
gh-74929: Rudimentary docs for PEP 667 (#118581)
This is *not* sufficient for the final 3.13 release, but it will do for beta 1:

- What's new entry
- Updated changelog entry (news blurb)
- Mention the proxy for f_globals in the datamodel and Python frame object docs

This doesn't have any C API details (what's new refers to the PEP).
2024-05-05 15:31:26 +00:00
Tian Gao 5a0022a1d7
GH-111744: Make breakpoint() enter the debugger immediately (GH-118579) 2024-05-05 07:05:01 -07:00
Nikita Sobolev 1511bc95c4
gh-101137: Add `text/x-rst` to `mimetypes` (#118593) 2024-05-05 13:39:50 +00:00
Ryan Batchelder b6f0ab5b1c
gh-83505: Add markdown mimetype mapping (#17995) 2024-05-05 13:09:59 +00:00
Hugo van Kemenade 3b32575ed6
gh-118131: Command-line interface for the `random` module (#118132) 2024-05-05 06:30:03 +00:00
Serhiy Storchaka 711c80bfca
gh-118164: Break a loop between _pydecimal and _pylong and optimize int to str conversion (GH-118483)
For converting large ints to strings, CPython invokes a function in _pylong.py,
which uses the decimal module to implement an asymptotically waaaaay
sub-quadratic algorithm. But if the C decimal module isn't available, CPython
uses _pydecimal.py instead. Which in turn frequently does str(int). If the int
is very large, _pylong ends up doing the work, which in turn asks decimal to do
"big" arithmetic, which in turn calls str(big_int), which in turn ... it can
become infinite mutual recursion.

This change introduces a different int->str function that doesn't use decimal.
It's asymptotically worse, "Karatsuba time" instead of quadratic time, so
still a huge improvement. _pylong switches to that when the C decimal isn't
available. It is also used for not too large integers (less than 450_000 bits),
where it is faster (up to 2 times for 30_000 bits) than the asymptotically
better implementation that uses the C decimal.

Co-authored-by: Tim Peters <tim.peters@gmail.com>
2024-05-05 08:20:06 +03:00
Pablo Galindo Salgado 1b22d801b8
gh-118518: Allow perf to work without frame pointers (#112254) 2024-05-05 03:07:29 +02:00
Tim Peters 999f0c5122
gh-118164: str(10**10000) hangs if the C _decimal module is missing (#118503)
* Initial stab.

* Test the tentative fix. Hangs "forever" without this change.

* Move the new test to a better spot.

* New comment to explain why _convert_to_str allows any poewr of 10.

* Fixed a comment, and fleshed out an existing test that appeared unfinished.

* Added temporary asserts. Or maybe permanent ;-)

* Update Lib/_pydecimal.py

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

* Remove the new _convert_to_str().

Serhiy and I independently concluded that exact powers of 10
aren't possible in these contexts, so just checking the
string length is sufficient.

* At least for now, add the asserts to the other block too.

* 📜🤖 Added by blurb_it.

---------

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-05-04 18:22:33 -05:00
Davide Rizzo 08d169f14a
gh-109617: fix ncurses incompatibility on macOS with Xcode 15 (#111258)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2024-05-04 23:41:47 +02:00
Tian Gao f34e965e52
GH-111744: Support opcode events in bdb (GH-111834) 2024-05-04 07:44:49 -07:00
Tian Gao b034f14a4b
gh-74929: Implement PEP 667 (GH-115153) 2024-05-04 12:12:10 +01:00
Tian Gao 00da0afa0d
gh-113081: Print colorized exception just like built-in traceback in pdb (#113082) 2024-05-04 12:26:40 +02:00
Irit Katriel 85af789961
gh-111997: C-API for signalling monitoring events (#116413) 2024-05-04 08:23:50 +00:00
Tian Gao 9c14ed0618
gh-107674: Improve performance of `sys.settrace` (GH-117133)
* Check tracing in RESUME_CHECK

* Only change to RESUME_CHECK if not tracing
2024-05-03 19:49:24 +01:00
Tian Gao 998c3856c1
gh-83856: Honor atexit for all multiprocessing start methods (GH-114279)
Use atexit for all multiprocessing start methods to cleanup.
See the GH-114279 PR discussion and related issue for details as to why.
2024-05-03 11:45:46 -07:00
Erlend E. Aasland dd8f05fee2
gh-115119: Update macOS installer to build with libmpdec 4.0.0 (GH-118382)
Co-authored-by: Ned Deily <nad@python.org>
2024-05-03 16:49:40 +00:00
Erlend E. Aasland 1324502752
gh-115119: Update Windows installer to mpdecimal 4.0.0 (#118506) 2024-05-03 18:00:43 +02:00
Brett Simmers c2627d6eea
gh-116322: Add Py_mod_gil module slot (#116882)
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.

PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.

A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
2024-05-03 11:30:55 -04:00
Sam Gross 2dae505e87
gh-117514: Add `sys._is_gil_enabled()` function (#118514)
The function returns `True` or `False` depending on whether the GIL is
currently enabled. In the default build, it always returns `True`
because the GIL is always enabled.
2024-05-03 11:09:57 -04:00
Carl Meyer c8deb1e4b4
gh-118513: Fix sibling comprehensions with a name bound in one and global in the other (#118526)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-05-03 14:05:19 +00:00
Alexander Kanavin 37ccf16786
gh-101732: Modules/_ssl.c: use Y2038 compatible openssl function when available (GH-118425) 2024-05-03 15:34:05 +02:00
Jelle Zijlstra ca269e58c2
gh-116126: Implement PEP 696 (#116129)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 06:17:32 -07:00
Tian Gao 4e2caf2aa0
gh-118500: Add pdb support for zipapp (#118501) 2024-05-02 21:53:27 +01:00
infohash b28a3339e4
gh-90848: Fixed create_autospec ignoring configure_mock style kwargs (#118163) 2024-05-02 18:36:35 +01:00
Pablo Galindo Salgado 6bcbee09df
gh-93502: Add new C-API functions to trace object creation and destruction (#115945) 2024-05-02 19:30:00 +02:00
Raphael Gaschignard 2770d5caca
gh-105879: Add support for keyword arguments to eval and exec (#105885)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-05-02 08:55:29 -07:00
Sergey B Kirpichev 9789440de3
gh-82062: Fix support of parameter defaults on methods in extension modules (GH-115270)
Now inspect.signature() supports references to the module globals in
parameter defaults on methods in extension modules.  Previously it was
only supported in functions.  The workaround was to specify the fully
qualified name, including the module name.
2024-05-02 17:44:33 +03:00
Steve Dower 81939dad77
gh-118486: Support mkdir(mode=0o700) on Windows (GH-118488) 2024-05-02 15:20:43 +01:00
Crowthebird 7c97dc8c95
gh-118216: Don't consider dotted `__future__` imports (#118267) 2024-05-02 06:32:20 -07:00
Guido van Rossum a524152b8c
gh-118335: Make REGEN_JIT_COMMAND empty if tier2 interpreter enabled (#118493)
Also patch up news blurb for gh-118339
(add warning that PYTHON_UOPS is now PYTHON_JIT).
2024-05-01 16:36:29 -07:00
Nice Zombies a7711a2a4e
gh-117607: Speedup os.path.relpath() (GH-117608) 2024-05-01 22:44:55 +01:00
Victor Stinner b52c753e0f
gh-110850: Add PyTime_TimeRaw() function (#118394)
Add "Raw" variant of PyTime functions:

* PyTime_MonotonicRaw()
* PyTime_PerfCounterRaw()
* PyTime_TimeRaw()

Changes:

* Add documentation and tests. Tests release the GIL while calling
  raw clock functions.
* py_get_system_clock() and py_get_monotonic_clock() now check that
  the GIL is hold by the caller if raise_exc is non-zero.
* Reimplement "Unchecked" functions with raw clock functions.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-05-01 18:05:01 +00:00
Malcolm Smith 75955110a6
gh-116622: Android sysconfig updates (#118352) 2024-05-01 16:47:54 +00:00
Harmen Stoppels 759e8e7ab8
gh-99730: urllib.request: Keep HEAD method on redirect (GH-99731) 2024-05-01 18:01:47 +02:00
Anthony Shaw beb653cc24
gh-117958: Expose JIT code via method in UOpExecutor (#117959) 2024-05-01 07:11:14 -07:00
Pablo Galindo Salgado 4a08a75cf4
gh-99180: Remove traceback anchors in return and assign statements that cover all the displayed range (#112670) 2024-05-01 14:42:10 +01:00
Irit Katriel c1bf4874c1
gh-116767: fix crash on 'async with' with many context managers (GH-118348)
Account for `add_stopiteration_handler` pushing a block for `async with`.
To allow generator functions that previously almost hit the `CO_MAXBLOCKS`
limit by nesting non-async blocks, the limit is increased by 1.
This increase allows one more block in non-generator functions.
2024-05-01 13:01:16 +02:00
Thomas Grainger fc7e1aa3c0
GH-117881: fix athrow().throw()/asend().throw() concurrent access (GH-117882) 2024-05-01 08:44:01 +02:00
Malcolm Smith 2520eed0a5
gh-116622: Add Android testbed (GH-117878)
Add code and config for a minimal Android app, and instructions to build and run it.
Improve Android build instructions in general.
Add a tool subcommand to download the Gradle wrapper (with its binary blob). Android
studio must be downloaded manually (due to the license).
2024-05-01 08:36:45 +02:00
Guido van Rossum 7d83f7bcc4
gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.

We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.

On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.

In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
Victor Stinner 587388ff22
gh-118124: Use static_assert() in Py_BUILD_ASSERT() on C11 (#118398)
Use static_assert() in Py_BUILD_ASSERT() and Py_BUILD_ASSERT_EXPR()
on C11 and newer and C++11 and newer.

Add tests to test_cext and test_cppext.
2024-04-30 22:29:48 +02:00
Shantanu 6999d68d28
gh-118218: Reuse return tuple in itertools.pairwise (GH-118219) 2024-04-30 23:16:52 +03:00
Erlend E. Aasland b568c2c1ff
gh-118406: Add signature for sqlite3.Connection objects (#118428) 2024-04-30 19:58:22 +00:00
Irit Katriel 1f16b4ce56
gh-118272: Clear generator frame's locals when the generator is closed (#118277)
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
2024-04-30 19:32:25 +01:00
Tian Gao d7ac427a79
gh-117618: Make package.module searchable for breakpoints and clean up docs (#117619) 2024-04-30 18:18:01 +00:00
Serhiy Storchaka 17a8af9508
gh-118402: Fix inspect.signature() for functools.cmp_to_key() result (GH-118427) 2024-04-30 17:49:28 +03:00
Malcolm Smith 3b268f4edc
gh-116622: Redirect stdout and stderr to system log when embedded in an Android app (#118063) 2024-04-30 16:00:31 +02:00
Serhiy Storchaka 11f8348d78
gh-118404: Fix inspect.signature() for non-comparable callables (GH-118405) 2024-04-30 15:04:16 +03:00
Savannah Ostrowski 8b56d82c59
GH-118306: Update JIT to use LLVM 18 (GH-118307) 2024-04-29 21:09:16 +00:00
Steve Dower 96d8ca7ad6
gh-118347: Fix Windows installer not updating launcher (GH-118386) 2024-04-29 21:46:25 +01:00
Serhiy Storchaka 444ac0b7a6
gh-118285: Fix signatures of operator.{attrgetter,itemgetter,methodcaller} instances (GH-118316)
* Allow to specify the signature of custom callable instances of extension
  type by the __text_signature__ attribute.
* Specify signatures of operator.attrgetter, operator.itemgetter, and
  operator.methodcaller instances.
2024-04-29 19:30:48 +03:00
Tian Gao 375c94c75d
gh-107674: Lazy load line number to improve performance of tracing (GH-118127) 2024-04-29 09:54:52 +01:00
Erlend E. Aasland c7e7bfc4ca
gh-115119: Detect _decimal dependencies using pkg-config (#115406)
pkg-config is supported for libmpdec 4.0.0 and newer.
2024-04-29 08:58:57 +02:00
Henrik Tunedal 133c1a7cdb
gh-118293: Suppress mouse cursor feedback when launching Windows processes with multiprocessing (GH-118315) 2024-04-28 21:10:44 +00:00
Hugo van Kemenade e0ab642436
gh-101100: Fix Sphinx warnings in `whatsnew/3.9.rst` (#118364) 2024-04-28 20:31:22 +03:00
Jelle Zijlstra 2326d6c868
gh-109118: Make comprehensions work within annotation scopes, but without inlining (#118160)
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-04-28 06:21:28 -07:00
Xie Yanbo 8397d8d300
Correct spelling error in recent NEWS entry (#118308) 2024-04-26 19:06:08 -07:00
Cheryl Sabella 194fd17bc6
bpo-32839: Add the after_info() method for Tkinter widgets (GH-5664) 2024-04-27 00:27:58 +03:00
Tian Gao 0f99861332
gh-117385: Remove unhooked events on sys.settrace (GH-117386) 2024-04-26 17:01:44 +01:00
Kirill Podoprigora 463c20dae9
gh-117928: Bump the minimum Sphinx version to 6.2.1 (#117853)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-04-26 13:10:16 +03:00
Faidon Liambotis fb7f79b4da
gh-117566: fix IPv6Address.is_loopback for IPv4-mapped loopbacks (GH-117567)
While properties like IPv6Address.is_private account for IPv4-mapped
IPv6 addresses, such as for example:

    >>> ipaddress.ip_address("192.168.0.1").is_private
    True
    >>> ipaddress.ip_address("::ffff:192.168.0.1").is_private
    True
...the same doesn't currently apply to the is_loopback property:
    >>> ipaddress.ip_address("127.0.0.1").is_loopback
    True
    >>> ipaddress.ip_address("::ffff:127.0.0.1").is_loopback
    False

At minimum, this inconsistency between different properties is
counter-intuitive. Moreover, ::ffff:127.0.0.0/104 is for all intents and
purposes a loopback address, and should be treated as such.
2024-04-25 15:17:40 +00:00
Nice Zombies 10bb90ed49
gh-102511: Speed up os.path.splitroot() with native helpers (GH-118089) 2024-04-25 10:07:38 +01:00
Erlend E. Aasland e38b43c213
gh-118221: Always use the default row factory in sqlite3.iterdump() (#118223)
sqlite3.iterdump() depends on the row factory returning resulting rows
as tuples; it will fail with custom row factories like for example a
dict factory.

With this commit, we explicitly reset the row factory of the cursor used
by iterdump(), so we always get predictable results. This does not
affect the row factory of the parent connection.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-04-25 10:11:45 +02:00
Steve Dower 4b10e209c7
gh-117786: Fix venv created from Windows Store install by restoring __PYVENV_LAUNCHER__ smuggling (GH-117814) 2024-04-24 23:00:55 +01:00
Serhiy Storchaka 93b7ed7c6b
gh-108191: Add support of positional argument in SimpleNamespace constructor (GH-108195)
SimpleNamespace({'a': 1, 'b': 2}) and SimpleNamespace([('a', 1), ('b', 2)])
are now the same as SimpleNamespace(a=1, b=2).
2024-04-25 00:39:54 +03:00
Eric Snow 03e3e31723
gh-76785: Rename _xxsubinterpreters to _interpreters (gh-117791)
See https://discuss.python.org/t/pep-734-multiple-interpreters-in-the-stdlib/41147/26.
2024-04-24 16:18:24 +00:00
Alex Waygood 8227883d1f
gh-118013: Use weakrefs for the cache key in `inspect._shadowed_dict` (#118202) 2024-04-24 15:55:02 +01:00
Thomas Grainger 7d369d471c
GH-117536: GH-117894: fix athrow().throw(...) unawaited warning (GH-117851) 2024-04-24 14:20:19 +02:00
Hugo van Kemenade 975081b11e
gh-117225: Add color to doctest output (#117583)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-04-24 14:27:40 +03:00
Irit Katriel 0aa0fc3d3c
gh-117901: Add option for compiler's codegen to save nested instruction sequences for introspection (#118007) 2024-04-24 09:46:17 +00:00
Nikita Sobolev 692e902c74
gh-116023: Add `show_empty=False` to `ast.dump` (#116037)
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-04-24 11:02:38 +03:00
Jelle Zijlstra d0b664ee06
gh-118168: Fix Unpack interaction with builtin aliases (#118169)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-04-23 13:40:26 +00:00
Nikita Sobolev de1f686827
gh-118082: Improve `import` without names syntax error message (#118083) 2024-04-23 13:00:52 +01:00
Joe Jevnik eb927e9fc8
gh-68114: Fix handling for removed PyArg_ParseTuple 'w' formatters (GH-8204)
Co-authored-by: Joe Jevnik <joe@quantopian.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-04-23 13:15:15 +02:00
Shantanu 8e86579cae
gh-95754: Better error when script shadows a standard library or third party module (#113769) 2024-04-22 18:24:21 -07:00
Seth Michael Larson c9829eec08
gh-116741: Upgrade libexpat to 2.6.2 (#117296)
Upgrade libexpat to 2.6.2
2024-04-22 18:15:08 -07:00
Guido van Rossum 1b85b3424c
GH-118074: Executors in the COLD_EXITS array are not GC'able (#118117) 2024-04-22 16:20:39 -07:00
Jelle Zijlstra 85f727c5fb
gh-109118: Allow lambdas in annotation scopes in classes (#118019) 2024-04-22 12:50:26 -07:00
tahia 8974a63f5e
bpo-18108: Adding dir_fd and follow_symlinks keyword args to shutil.chown (GH-15811)
* Adding dir_fd and follow_symlinks keyword args to shutil.chown
* Extending test_shutil.TestShutil.test_chown to include new kwargs
* Updating shutil.chown documentation

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Berker Peksag <berker.peksag@gmail.com>
Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
2024-04-22 18:23:36 +00:00
Erlend E. Aasland 550483b7e6
gh-117995: Don't raise DeprecationWarnings for indexed nameless params (#118001)
Filter out '?NNN' placeholders when looking for named params.

Co-authored-by: AN Long <aisk@users.noreply.github.com>
2024-04-22 08:43:20 +02:00
Serhiy Storchaka 51ef89cd9a
gh-115961: Add name and mode attributes for compressed file-like objects (GH-116036)
* Add name and mode attributes for compressed and archived file-like objects
  in modules bz2, lzma, tarfile and zipfile.
* Change the value of the mode attribute of GzipFile from integer (1 or 2)
  to string ('rb' or 'wb').
* Change the value of the mode attribute of ZipExtFile from 'r' to 'rb'.
2024-04-21 11:46:39 +03:00
Andrew Cassidy 5fa5b7facb
gh-91629 Use conf.d configs and fish_add_path to set the PATH when installing for the Fish shell. (GH-91630)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2024-04-21 02:52:58 -04:00
Russell Keith-Magee df987331d8
gh-114099: Formalize Tier 3 status of iOS (GH-118020) 2024-04-20 23:04:08 -04:00
Barney Gale 15fbd53ba9
GH-112855: Speed up `pathlib.PurePath` pickling (#112856)
The second item in the tuple returned from `__reduce__()` is a tuple of arguments to supply to path constructor. Previously we returned the `parts` tuple here, which entailed joining, parsing and normalising the path object, and produced a compact pickle representation.

With this patch, we instead return a tuple of paths that were originally given to the path constructor. This makes pickling much faster (at the expense of compactness).

It's worth noting that, in the olden times, pathlib performed this parsing/normalization up-front in every case, and so using `parts` for pickling was almost free. Nowadays pathlib only parses/normalises paths when it's necessary or advantageous to do so (e.g. computing a path parent, or iterating over a directory, respectively).
2024-04-20 17:46:52 +01:00
Savannah Ostrowski d8f350309d
GH-115874: Fix segfault in FutureIter_dealloc (GH-117741) 2024-04-19 22:30:52 +00:00
Russell Keith-Magee ab99438900
gh-114099: Modify preprocessor symbol usage to support older macOS SDKs (GH-118073)
Co-authored-by: Joshua Root jmr@macports.org
2024-04-19 14:56:33 -04:00
Alex Waygood 1e3e7ce11e
gh-114053: Fix bad interaction of PEP-695, PEP-563 and ``get_type_hints`` (#118009)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-04-19 13:03:44 +00:00
lyc8503 15b3555e4a
gh-116931: Add fileobj parameter check for Tarfile.addfile (GH-117988)
Tarfile.addfile now throws an ValueError when the user passes
in a non-zero size tarinfo but does not provide a fileobj,
instead of writing an incomplete entry.
2024-04-19 11:41:51 +00:00
Tian Gao a09e472992
gh-117535: Change unknown filename of warnings from `sys` to `<sys>` (#118018) 2024-04-18 20:50:09 -07:00
Victor Stinner 340a02b590
gh-117987: Restore several functions removed in Python 3.13 alpha 1 (GH-117993)
Restore these functions removed in Python 3.13 alpha 1:

* Py_SetPythonHome()
* Py_SetProgramName()
* PySys_SetArgvEx()
* PySys_SetArgv()
2024-04-18 15:20:38 +02:00
lyc8503 0a0756c5ed
gh-116932: Remove redundant NEWS entry (#118040) 2024-04-18 13:22:18 +01:00
lyc8503 468b9aeb92
gh-116932: Add note on how to report python documentation theme bugs (GH-117989) 2024-04-18 10:45:25 +02:00
Nice Zombies b848b944bb
gh-117641: Improve the perfornance of posixpath.commonpath() (#117652) 2024-04-18 09:26:34 +02:00
Serhiy Storchaka deaecb88fa
gh-80361: Fix TypeError in email.Message.get_payload() (GH-117994)
It was raised when the charset is rfc2231 encoded, e.g.:

   Content-Type: text/plain; charset*=ansi-x3.4-1968''utf-8
2024-04-17 19:31:26 +03:00
Irit Katriel c179c0e6cb
gh-117680: make _PyInstructionSequence a PyObject and use it in tests (#117629) 2024-04-17 16:42:04 +01:00
CF Bolz-Tereick 5a0209fc23
GH-100242: bring functools.py partial implementation more in line with C code (GH-100244)
in partial.__new__, before checking for the existence of the attribute
'func', first check whether the argument is an instance of partial.
2024-04-17 15:34:46 +02:00
Victor Stinner 75eed5b373
gh-117929: Restore removed PyEval_InitThreads() function (#117931) 2024-04-17 15:01:28 +02:00
Mariusz Felisiak b9b3c455f0
gh-115009: Update macOS installer to use SQLite 3.45.3 (#117443) 2024-04-17 13:59:38 +02:00
Ivan Savin 1aa8bbe62f
bpo-40944: Fix IndexError when parse emails with truncated Message-ID, address, routes, etc (GH-20790)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-04-17 10:14:22 +00:00
Serhiy Storchaka aec1dac4ef
gh-117313: Fix re-folding email messages containing non-standard line separators (GH-117369)
Only treat '\n', '\r' and '\r\n' as line separators in re-folding the email
messages.  Preserve control characters '\v', '\f', '\x1c', '\x1d' and '\x1e'
and Unicode line separators '\x85', '\u2028' and '\u2029' as is.
2024-04-17 13:00:25 +03:00
Serhiy Storchaka 4e502a4997
gh-117394: Speed up os.path.ismount() on Posix (GH-117447)
It is now 2-3 times faster if the user has permissions.
2024-04-17 12:58:19 +03:00
Serhiy Storchaka 51132da0c4
gh-117503: Fix support of non-ASCII user names in posixpath.expanduser() (GH-117504)
They are now supported in bytes paths as well as in string paths.
2024-04-17 12:53:40 +03:00
Joshua Root 8515fd79fe
gh-117845: Detect libedit hook function signature in configure (#117870)
Older libedit versions (like Apple's) use a different type signature
for rl_startup_hook and rl_pre_input_hook. Add a configure check to
determine which signature is accepted by introducing the
Py_RL_STARTUP_HOOK_TAKES_ARGS macro in pyconfig.h.
2024-04-17 11:26:10 +02:00
Serhiy Storchaka f74e51229c
gh-86650: Fix IndexError when parse emails with invalid Message-ID (GH-117934)
In particularly, one-off addresses generated by Microsoft Outlook:
https://learn.microsoft.com/en-us/office/client-developer/outlook/mapi/one-off-addresses

Co-authored-by: fsc-eriker <72394365+fsc-eriker@users.noreply.github.com>
2024-04-17 10:44:41 +03:00
tsufeki 8cc9adbfdd
gh-75171: Fix parsing invalid email address headers starting or ending with a dot (GH-15600)
Co-authored-by: Tim Bell <timothybell@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-04-17 10:39:15 +03:00
Seth Michael Larson d70ee13e57
gh-112844: Fix xz CPE identifier (GH-117656) 2024-04-16 23:11:43 +01:00
Mariusz Felisiak de0dc68b82
gh-115009: Update Windows installer to use SQLite 3.45.3 (GH-117445) 2024-04-16 23:10:05 +01:00
Victor Stinner 919784737c
gh-117645: Increase WASI stack size from 512 KiB to 8 MiB (#117674)
Increase also the initial memory from 10 MiB to 20 MiB.

Reenable test_dynamic on WASI build.
2024-04-16 23:26:54 +02:00
Victor Stinner 3fe03ccea6
gh-117755: Fix mimalloc for huge allocation on s390x (#117809)
Fix mimalloc allocator for huge memory allocation (around
8,589,934,592 GiB) on s390x.

Abort allocation early in mimalloc if the number of slices doesn't
fit into uint32_t, to prevent a integer overflow (cast 64-bit
size_t to uint32_t).
2024-04-16 20:34:12 +00:00
Jeff Glass acf69e09c6
gh-115178: Add Counts of UOp Pairs to pystats (GH-115181) 2024-04-16 14:27:18 +01:00
Mark Shannon c053d52edd
GH-115776: Static object are immortal, so mark them as such. (GH-117673) 2024-04-16 12:51:41 +01:00
Alex Waygood cff0a2db00
gh-117691: Add an appropriate stacklevel for PEP-706 tarfile deprecation warnings (GH-117872) 2024-04-16 13:36:00 +02:00
Douglas Thor 1316692e8c
gh-102402: Fix floating point math issue by using `time.time_ns()` in `logging.LogRecord` (GH-102412) 2024-04-16 10:44:57 +01:00
Inada Naoki 6dc661bc9f
gh-77102: site: try utf-8 and fallback to locale encoding when reading .pth file (#117802) 2024-04-16 12:56:16 +09:00
Hugo van Kemenade 3375282bb8
Docs: add link roles with Sphinx extlinks (#117850)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-04-15 21:22:00 +03:00
Steve Dower 185999bb3a
gh-90329: Add _winapi.GetLongPathName and GetShortPathName and use in venv to reduce warnings (GH-117817) 2024-04-15 15:36:06 +01:00
Mark Shannon 784e076a10
GH-117750: When clearing object's dict, clear inline values but leave dict attached (GH-117808) 2024-04-15 14:45:05 +01:00
Nice Zombies 9ee94d1391
gh-117636: Remove redundant type check in `os.path.join()` (#117638) 2024-04-14 14:04:14 -07:00
Jason R. Coombs 9c93b7402b
gh-117348: restore import time performance of configparser (#117703)
Reduces import time by over 50% (10431µs vs 4350µs on Apple M3 Pro).
2024-04-14 11:10:09 +00:00
Barney Gale a74f117dab
GH-115060: Speed up `pathlib.Path.glob()` by omitting initial `stat()` (#117831)
Since 6258844c, paths that might not exist can be fed into pathlib's
globbing implementation, which will call `os.scandir()` / `os.lstat()` only
when strictly necessary. This allows us to drop an initial `self.is_dir()`
call, which saves a `stat()`.

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-04-14 00:08:03 +01:00
Hugo van Kemenade 56ed979d04
gh-68583: webbrowser: replace `getopt` with `argparse`, add long options (#117047) 2024-04-13 08:56:56 -06:00
Michiel W. Beijen 022ba6d161
gh-102247: http: support rfc9110 status codes (GH-117611)
rfc9110 obsoletes the earlier rfc 7231. This document also includes some
status codes that were previously only used for WebDAV and assigns more
generic names to these status codes.

ref: https://www.rfc-editor.org/rfc/rfc9110.html#name-changes-from-rfc-7231

- http.HTTPStatus.CONTENT_TOO_LARGE (413, previously
  REQUEST_ENTITY_TOO_LARGE)
- http.HTTPStatus.URI_TOO_LONG (414, previously REQUEST_URI_TOO_LONG)
- http.HTTPStatus.RANGE_NOT_SATISFYABLE (416, previously
  REQUEST_RANGE_NOT_SATISFYABLE)
- http.HTTPStatus.UNPROCESSABLE_CONTENT (422, previously
  UNPROCESSABLE_ENTITY)

The new constants are added to http.HTTPStatus and the old constant names are
preserved for backwards compatibility.

References in documentation to the obsoleted rfc 7231 are updated
2024-04-13 07:33:20 -07:00
Barney Gale 30f0643e36
GH-117727: Speed up `pathlib.Path.iterdir()` by using `os.scandir()` (#117728)
Replace use of `os.listdir()` with `os.scandir()`. Forgo setting `_drv`,
`_root` and `_tail_cached`, as these usually aren't needed. Use
`os.DirEntry.path` to set `_str`.
2024-04-12 22:02:39 +00:00
Barney Gale 0eb52f5f26
GH-115060: Speed up `pathlib.Path.glob()` by not scanning literal parts (#117732)
Don't bother calling `os.scandir()` to scan for literal pattern segments,
like `foo` in `foo/*.py`. Instead, append the segment(s) as-is and call
through to the next selector with `exists=False`, which signals that the
path might not exist. Subsequent selectors will call `os.scandir()` or
`os.lstat()` to filter out missing paths as needed.
2024-04-12 22:19:21 +01:00
Erlend E. Aasland deb921f851
gh-117431: Adapt bytes and bytearray .find() and friends to Argument Clinic (#117502)
This change gives a significant speedup, as the METH_FASTCALL calling
convention is now used. The following bytes and bytearray methods are adapted:

- count()
- find()
- index()
- rfind()
- rindex()

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2024-04-12 07:40:55 +00:00
Will Childs-Klein b8eaad3009
gh-117233: Detect support for several hashes at hashlib build time (GH-117234)
Detect libcrypto BLAKE2, Shake, SHA3, and Truncated-SHA512 support at hashlib build time

## BLAKE2

While OpenSSL supports both "b" and "s" variants of the BLAKE2 hash
function, other cryptographic libraries may lack support for one or both
of the variants. This commit modifies `hashlib`'s C code to detect
whether or not the linked libcrypto supports each BLAKE2 variant, and
elides references to each variant's NID accordingly. In cases where the
underlying libcrypto doesn't fully support BLAKE2, CPython's
`./configure` script can be given the following flag to use CPython's
interned BLAKE2 implementation: `--with-builtin-hashlib-hashes=blake2`.

## SHA3, Shake, & truncated SHA512.

Detect BLAKE2, SHA3, Shake, & truncated SHA512 support in the
OpenSSL-ish libcrypto library at build time.  This helps allow hashlib's
`_hashopenssl` to be used with libraries that do not to support every
algorithm that upstream OpenSSL does.  Such as AWS-LC & BoringSSL.

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2024-04-11 16:49:41 +02:00
Bruce Merry 01a51f9494
gh-117722: Fix Stream.readuntil with non-bytes buffer objects (#117723)
gh-16429 introduced support for an iterable of separators in
Stream.readuntil. Since bytes-like types are themselves iterable, this
can introduce ambiguities in deciding whether the argument is an
iterator of separators or a singleton separator. In gh-16429, only 'bytes'
was considered a singleton, but this will break code that passes other
buffer object types.

Fix it by only supporting tuples rather than arbitrary iterables.

Closes gh-117722.
2024-04-11 07:41:55 -07:00
Erlend E. Aasland 044dc496e0
gh-117709: Add vectorcall support for str() with positional-only arguments (#117746)
Fall back to tp_call() for cases when arguments are passed by name.

Co-authored-by: Donghee Na <donghee.na@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-04-11 13:55:37 +00:00
Barney Gale 0cc71bde00
GH-117586: Speed up `pathlib.Path.walk()` by working with strings (#117726)
Move `pathlib.Path.walk()` implementation into `glob._Globber`. The new
`glob._Globber.walk()` classmethod works with strings internally, which is
a little faster than generating `Path` objects and keeping them normalized.
The `pathlib.Path.walk()` method converts the strings back to path objects.

In the private pathlib ABCs, our existing subclass of `_Globber` ensures
that `PathBase` instances are used throughout.

Follow-up to #117589.
2024-04-11 01:26:53 +01:00
Barney Gale 6258844c27
GH-117586: Speed up `pathlib.Path.glob()` by working with strings (#117589)
Move pathlib globbing implementation into a new private class: `glob._Globber`. This class implements fast string-based globbing. It's called by `pathlib.Path.glob()`, which then converts strings back to path objects.

In the private pathlib ABCs, add a `pathlib._abc.Globber` subclass that works with `PathBase` objects rather than strings, and calls user-defined path methods like `PathBase.stat()` rather than `os.stat()`.

This sets the stage for two more improvements:

- GH-115060: Query non-wildcard segments with `lstat()`
- GH-116380: Unify `pathlib` and `glob` implementations of globbing.

No change to the implementations of `glob.glob()` and `glob.iglob()`.
2024-04-10 20:43:07 +01:00
Barney Gale 630df37116
GH-117546: Fix symlink resolution in `os.path.realpath('loop/../link')` (#117568)
Continue resolving symlink targets after encountering a symlink loop, which
matches coreutils `realpath` behaviour.
2024-04-10 18:17:18 +01:00
neonene ef4118222b
gh-117142: Port _ctypes to multi-phase init (GH-117181) 2024-04-10 11:00:01 +00:00
Nikita Sobolev 4bb7d121bc
gh-117692: Fix `AttributeError` in `DocTestFinder` on wrapped `builtin_or_method` (#117699)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-04-10 10:52:47 +01:00
Nice Zombies f90ff03672
gh-117686: Improve the performance of ntpath.expanduser() (#117690)
Refactor out _get_bothseps() call from the loop.
2024-04-10 10:28:48 +02:00
Erlend E. Aasland ca75915779
gh-117648: Amend NEWS entry (#117697)
Make the wording more vague; the performance impact varies a lot
depending on platform and input.
2024-04-09 20:03:47 +00:00
Thomas Wouters d0f93d132f Merge branch 'main' of https://github.com/python/cpython 2024-04-09 20:42:07 +02:00
Ethan Furman e5521bcca9
gh-117663: [Enum] fix _simple_enum's detection of aliases (GH-117664) 2024-04-09 11:31:07 -07:00
Vlad4896 d5f1139c79
gh-117534: Add checking for input parameter in iso_to_ymd (#117543)
Moves the validation for invalid years in the C implementation of the `datetime` module into a common location between `fromisoformat` and `fromisocalendar`, which improves the error message and fixes a failed assertion when parsing invalid ISO 8601 years using one of the "ISO weeks" formats.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-04-09 13:53:00 -04:00
Guido van Rossum fa58e75a86
gh-116720: Fix corner cases of taskgroups (#117407)
This prevents external cancellations of a task group's parent task to
be dropped when an internal cancellation happens at the same time.
Also strengthen the semantics of uncancel() to clear self._must_cancel
when the cancellation count reaches zero.

Co-Authored-By: Tin Tvrtković <tinchester@gmail.com>
Co-Authored-By: Arthur Tacca
2024-04-09 08:17:28 -07:00
Jelle Zijlstra f2132fcd2a
gh-117516: Implement typing.TypeIs (#117517)
See PEP 742.

Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-04-09 10:50:37 +00:00
Thomas Wouters 57aee2a02c Python 3.13.0a6 2024-04-09 11:56:22 +02:00
Nice Zombies 99852d9e65
gh-117648: Improve performance of os.join (#117654)
Replace map() with a method call in the loop body.

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
2024-04-09 10:27:14 +02:00
Chris Markiewicz 19a2202067
gh-117182: Allow lazily loaded modules to modify their own __class__ 2024-04-09 04:08:48 +01:00
Bruce Merry 775912a51d
gh-81322: support multiple separators in StreamReader.readuntil (#16429) 2024-04-08 09:58:02 -07:00
Serhiy Storchaka 24a2bd0481
gh-117642: Fix PEP 737 implementation (GH-117643)
* Fix implementation of %#T and %#N (they were implemented as %T# and
  %N#).
* Restore tests removed in gh-116417.
2024-04-08 16:27:25 +00:00
Nice Zombies 733e56ef96
gh-117584: Raise TypeError for non-paths in posixpath.relpath() (GH-117585) 2024-04-07 12:00:08 +03:00
Laurie O df4d84c3cd
gh-96471: Add asyncio queue shutdown (#104228)
Co-authored-by: Duprat <yduprat@gmail.com>
2024-04-06 07:27:13 -07:00
Steve Dower 687616877b
gh-111140: PyLong_From/AsNativeBytes: Take *flags* rather than just *endianness* (GH-116053) 2024-04-05 16:21:16 +02:00
Barney Gale abfa16b44b
GH-114847: Speed up `posixpath.realpath()` (#114848)
Apply the following optimizations to `posixpath.realpath()`:

- Remove use of recursion
- Construct child paths directly rather than using `join()`
- Use `os.getcwd[b]()` rather than `abspath()`
- Use `startswith(sep)` rather than `isabs()`
- Use slicing rather than `split()`

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-04-05 12:35:01 +00:00
Petr Viktorin 9ceaee74db
gh-116608: importlib.resources: Un-deprecate functional API & add subdirectory support (GH-116609) 2024-04-05 13:55:59 +02:00
Irit Katriel 04697bcfaf
gh-117494: extract the Instruction Sequence data structure into a separate file (#117496) 2024-04-04 15:47:26 +00:00
Guido van Rossum 060a96f1a9
gh-116968: Reimplement Tier 2 counters (#117144)
Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``),
shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The
API used for adaptive specialization counters is changed but the behavior is
(supposed to be) identical.

The behavior of the Tier 2 counters is changed:
- There are no longer dynamic thresholds (we never varied these).
- All counters now use the same exponential backoff.
- The counter for ``JUMP_BACKWARD`` starts counting down from 16.
- The ``temperature`` in side exits starts counting down from 64.
2024-04-04 15:03:27 +00:00
Tony Mountifield 3f5bcc86d0
gh-117467: Add preserving of mailbox owner on flush (GH-117510)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-04-04 13:32:53 +03:00
rsp4jack 85843348c5
gh-117459: Keep the traceback in _convert_future_exc (#117460) 2024-04-03 20:13:32 -07:00
Shantanu b4fe02f595
gh-117205: Increase chunksize when compiling pyc in parallel (#117206) 2024-04-03 15:24:24 -07:00
Steve Dower 985917dc8d
gh-117267: Ensure DirEntry.stat().st_ctime still contains creation time during deprecation period (GH-117354) 2024-04-03 23:14:55 +01:00
Erlend E. Aasland 7ecd55d604
gh-117431: Adapt str.find and friends to Argument Clinic (#117468)
This change gives a significant speedup, as the METH_FASTCALL calling
convention is now used. The following methods are adapted:

- str.count
- str.find
- str.index
- str.rfind
- str.rindex
2024-04-03 17:59:18 +02:00
Barney Gale 345194de8c
GH-114847: Raise FileNotFoundError when getcwd() returns '(unreachable)' (#117481)
On Linux >= 2.6.36 with glibc < 2.27, `getcwd()` can return a relative
pathname starting with '(unreachable)'. We detect this and fail with
ENOENT, matching new glibc behaviour.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-04-03 16:39:40 +01:00
Zackery Spytz fc5f68e58e
gh-59215: unittest: restore _top_level_dir at end of discovery (GH-15242) 2024-04-03 16:17:13 +02:00
Nice Zombies 2ec6bb4111
gh-117381: Improve error messages for ntpath.commonpath() (GH-117382) 2024-04-03 16:10:09 +03:00
Gregory P. Smith 33ee5cb3e9
GH-70647: Deprecate strptime day of month parsing without a year present to avoid leap-year bugs (GH-117107) 2024-04-03 14:19:49 +02:00
Erlend E. Aasland 595bb496b0
gh-117431: Adapt bytes and bytearray .startswith() and .endswith() to Argument Clinic (#117495)
This change gives a significant speedup, as the METH_FASTCALL calling
convention is now used.
2024-04-03 13:11:14 +02:00
Erlend E. Aasland 444156ede4
gh-117431: Adapt str.startswith and str.endswith to Argument Clinic (#117466)
This change gives a significant speedup, as the METH_FASTCALL calling
convention is now used.
2024-04-03 09:11:39 +02:00
Nice Zombies cae4cdd07d
gh-117349: Micro-optimize a few `os.path` functions (#117350)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
2024-04-02 21:32:35 +01:00
Mark Shannon c32dc47aca
GH-115776: Embed the values array into the object, for "normal" Python objects. (GH-116115) 2024-04-02 11:59:21 +01:00
Grigoriev Semyon c97d3af239
gh-109120: Fix syntax error in handlinh of incorrect star expressions (#117444) 2024-04-02 11:42:58 +01:00
Irit Katriel 1d5479b236
gh-117411: move PyFutureFeatures to pycore_symtable.h and make it private (#117412) 2024-04-02 10:34:49 +00:00
Barney Gale fc8007ee36
GH-117337: Deprecate `glob.glob0()` and `glob.glob1()`. (#117371)
These undocumented functions are no longer used by `msilib`, so there's no
reason to keep them around.
2024-04-01 19:37:41 +00:00
Justin Turner Arthur c741ad3537
gh-77714: Provide an async iterator version of as_completed (GH-22491)
* as_completed returns object that is both iterator and async iterator
* Existing tests adjusted to test both the old and new style
* New test to ensure iterator can be resumed
* New test to ensure async iterator yields any passed-in Futures as-is

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2024-04-01 20:07:29 +03:00
Steve (Gadget) Barnes 3de09cadde
gh-91565: Replace bugs.python.org links with Devguide/GitHub ones (GH-91568)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
2024-04-01 13:02:07 +00:00
Jason R. Coombs 019143fecb
gh-117348: Refactored RawConfigParser._read for similicity and comprehensibility (#117372)
* Extract method for _read_inner, reducing complexity and indentation by 1.

* Extract method for _raise_all and yield ParseErrors from _read_inner.

Reduces complexity by 1 and reduces touch points for handling errors in _read_inner.

* Prefer iterators to splat expansion and literal indexing.

* Extract method for _strip_comments. Reduces complexity by 7.

* Model the file lines in a class to encapsulate the comment status and cleaned value.

* Encapsulate the read state as a dataclass

* Extract _handle_continuation_line and _handle_rest methods. Reduces complexity by 8.

* Reindent

* At least for now, collect errors in the ReadState

* Check for missing section header separately.

* Extract methods for _handle_header and _handle_option. Reduces complexity by 6.

* Remove unreachable code. Reduces complexity by 4.

* Remove unreachable branch

* Handle error condition early. Reduces complexity by 1.

* Add blurb

* Move _raise_all to ParsingError, as its behavior is most closely related to the exception class and not the reader.

* Split _strip* into separate methods.

* Refactor _strip_full to compute the strip just once and use 'not any' to determine the factor.

* Replace use of 'sys.maxsize' with direct computation of the stripped value.

* Extract has_comments as a dynamic property.

* Implement clean as a cached property.

* Model comment prefixes in the RawConfigParser within a prefixes namespace.

* Use a regular expression to search for the first match.

Avoids mutating variables and tricky logic and over-computing all of the starts when only the first is relevant.
2024-03-29 16:06:09 -04:00
Pedro Lacerda 54f7e14500
gh-66449: configparser: Add support for unnamed sections (#117273)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2024-03-29 15:05:00 +00:00
傅立业(Chris Fu) 8eec7ed714
gh-117110: Fix subclasses of typing.Any with custom constructors (#117111) 2024-03-29 00:19:20 +00:00
Nice Zombies 14f1ca7d53
gh-117335: Handle non-iterables for `ntpath.commonpath` (GH-117336) 2024-03-28 21:20:08 +00:00
Malcolm Smith 29829b58a8
gh-117294: Report DocTestCase as skipped if all examples in the doctest are skipped (GH-117297) 2024-03-28 21:59:12 +02:00
Joachim Wuttke 9a388b9a64
bpo-43848: explain optional argument mtime in gzip.py. (GH-25410)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-03-28 13:43:07 +00:00
Chris Markiewicz 9a1e55b8c5
gh-117178: Recover lazy loading of self-referential modules (#117179) 2024-03-28 12:59:31 +02:00
Jelle Zijlstra 4c71d51a4b
gh-117266: Fix crashes on user-created AST subclasses (GH-117276)
Fix crashes on user-created AST subclasses
2024-03-28 11:30:31 +01:00
Gregory P. Smith 8cb7d7ff86
gh-117310: Remove extra DECREF on "no ciphers" error path in `_ssl._SSLContext` constructor (#117309)
Remove extra self DECREF on ssl "no ciphers" error path.

This doesn't come up in practice because nobody links against a broken
OpenSSL library that provides nothing.
2024-03-28 11:11:58 +01:00
Russell Keith-Magee 0f27672c50
gh-114099: Add documentation for iOS platform (GH-117057)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Jacob Coffee <jacob@z7x.org>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Ned Deily <nad@python.org>
2024-03-28 04:13:13 -04:00
Russell Keith-Magee f006338017
gh-114099: Additions to standard library to support iOS (GH-117052)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Ned Deily <nad@python.org>
2024-03-28 03:59:33 -04:00
Tim Hatch b44898299a
gh-89739: gh-77140: Support zip64 in zipimport (GH-94146)
* Reads zip64 files as produced by the zipfile module
* Include tests (somewhat slow, however, because of the need to create "large" zips)
* About the same amount of strictness reading invalid zip files as zipfile has
* Still works on files with prepended data (like pex)

There are a lot more test cases at https://github.com/thatch/zipimport64/ that give me confidence that this works for real-world files.

Fixes #89739 and #77140.

---------

Co-authored-by: Itamar Ostricher <itamarost@gmail.com>
Reviewed-by: Gregory P. Smith <greg@krypto.org>
2024-03-28 06:54:51 +00:00
Seth Michael Larson 669ef49c7d
gh-99108: Update and check HACL* version information (GH-117295)
* Update and check HACL* version information
2024-03-27 14:56:14 -07:00
Malcolm Smith 74c8568d07
gh-71042: Add `platform.android_ver` (#116674) 2024-03-27 17:53:27 +01:00
Hugo van Kemenade ce00de4c8c
gh-117225: doctest: only print "and X failed" when non-zero, don't pluralise "1 items" (#117228) 2024-03-27 16:46:35 +02:00
Tian Gao b3e8c78ed7
gh-113548: Allow CLI arguments to `pdb -m` (#113557) 2024-03-27 01:20:12 +00:00
Irit Katriel 79be75735c
gh-115775: Compiler adds __static_attributes__ field to classes (#115913) 2024-03-26 15:18:17 +00:00
Antonio 70969d53a7
gh-97901 add missing text/rtf to mimetypes (GH-97902)
Co-authored-by: Noam Cohen <noam@noam.me>
2024-03-26 15:10:29 +01:00
Paulo Neves 4abca7e1e7
gh-98966: Handle stdout=subprocess.STDOUT (GH-98967)
Explicitly handle the case where stdout=STDOUT
as otherwise the existing error handling gets
confused and reports hard to understand errors.

Signed-off-by: Paulo Neves <ptsneves@gmail.com>
2024-03-26 13:37:50 +01:00
Serhiy Storchaka 9654daf793
gh-66543: Fix mimetype.guess_type() (GH-117217)
Fix parsing of the following corner cases:

* URLs with only a host name
* URLs containing a fragment
* URLs containing a query
* filenames with only a UNC sharepoint on Windows

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2024-03-26 13:26:45 +02:00
Mark Shannon 8bef34f625
GH-117108: Set the "old space bit" to "visited" for all young objects (#117213)
Change old space bit of young objects from 0 to gcstate->visited_space.
This ensures that any object created *and* collected during cycle GC has the bit set correctly.
2024-03-26 11:11:42 +00:00
Pablo Galindo Salgado 61599a48f5
bpo-24612: Improve syntax error for 'not' after an operator (GH-28170)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2024-03-26 10:30:46 +01:00
yevgeny hong ea9a296fce
gh-115627: Fix PySSL_SetError handling SSL_ERROR_SYSCALL (GH-115628)
Python 3.10 changed from using SSL_write() and SSL_read() to SSL_write_ex() and
SSL_read_ex(), but did not update handling of the return value.

Change error handling so that the return value is not examined.
OSError (not EOF) is now returned when retval is 0.

According to *recent* man pages of all functions for which we call
PySSL_SetError, (in OpenSSL 3.0 and 1.1.1), their return value should
be used to determine whether an error happened (i.e. if PySSL_SetError
should be called), but not what kind of error happened (so,
PySSL_SetError shouldn't need retval). To get the error,
we need to use SSL_get_error.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-03-26 08:45:43 +01:00
Victor Stinner d52bdfb19f
gh-83434: Disable XML in regrtest when -R option is used (#117232) 2024-03-26 08:35:59 +01:00
Sebastian Pipping 9f74e86c78
gh-117187: Fix XML tests for vanilla Expat <2.6.0 (GH-117203)
This fixes XML unittest fallout from the https://github.com/python/cpython/issues/115398 security fix.  When configured using `--with-system-expat` on systems with older pre 2.6.0 versions of libexpat, our unittests were failing.

* sax|etree: Simplify Expat version guard where simplifiable

Idea by Matěj Cepl

* sax|etree: Fix reparse deferral tests for vanilla Expat <2.6.0

This *does not fix* the case of distros with an older version of libexpat with the 2.6.0 feature backported as a security fix.  (Ubuntu is a known example of this with its libexpat1 2.5.0-2ubunutu0.1 package)
2024-03-25 18:48:27 -07:00
Jonathan Protzenko 872e212378
gh-99108: Refresh HACL*; update modules accordingly; fix namespacing (GH-117237)
Pulls in a new update from https://github.com/hacl-star/hacl-star and fixes our C "namespacing" done by `Modules/_hacl/refresh.sh`.
2024-03-26 00:35:26 +00:00
Eric V. Smith 8945b7ff55
gh-109870: Dataclasses: batch up exec calls (gh-110851)
Instead of calling `exec()` once for each function added to a dataclass, only call `exec()` once per dataclass. This can lead to speed improvements of up to 20%.
2024-03-25 19:59:14 -04:00
Nice Zombies 0821923aa9
gh-117114: Make os.path.isdevdrive available on all platforms (GH-117115) 2024-03-25 22:55:11 +00:00
Victor Stinner 507896d97d
gh-116936: Add PyType_GetModuleByDef() to the limited C API (#116937) 2024-03-25 16:32:20 +00:00
Serhiy Storchaka 0c1a42cf9c
gh-87193: Support bytes objects with refcount > 1 in _PyBytes_Resize() (GH-117160)
Create a new bytes object and destroy the old one if it has refcount > 1.
2024-03-25 16:32:11 +01:00
Tian Gao 01e7405da4
gh-112948: Make pdb completion similar to repl completion (#112950) 2024-03-25 15:18:09 +00:00
Irit Katriel d610d821fd
gh-112383: teach dis how to interpret ENTER_EXECUTOR (#117171) 2024-03-23 22:32:33 +00:00
Erik Soma f11d0d8be8
gh-91227: Ignore ERROR_PORT_UNREACHABLE in proactor recvfrom() (#32011) 2024-03-23 08:39:35 -07:00
Mark Shannon e28477f214
GH-117108: Change the size of the GC increment to about 1% of the total heap size. (GH-117120) 2024-03-22 18:43:25 +00:00
Serhiy Storchaka e2e0b4b4b9
gh-113024: C API: Add PyObject_GenericHash() function (GH-113025) 2024-03-22 20:19:10 +02:00
Serhiy Storchaka 567ab3bd15
gh-117084: Fix ZIP file extraction for directory entry names with backslashes on Windows (GH-117129) 2024-03-22 20:08:00 +02:00
Jakub Stasiak 40d75c2b7f
GH-113171: Fix "private" (non-global) IP address ranges (GH-113179)
* GH-113171: Fix "private" (really non-global) IP address ranges

The _private_networks variables, used by various is_private
implementations, were missing some ranges and at the same time had
overly strict ranges (where there are more specific ranges considered
globally reachable by the IANA registries).

This patch updates the ranges with what was missing or otherwise
incorrect.

I left 100.64.0.0/10 alone, for now, as it's been made special in [1]
and I'm not sure if we want to undo that as I don't quite understand the
motivation behind it.

The _address_exclude_many() call returns 8 networks for IPv4, 121
networks for IPv6.

[1] https://github.com/python/cpython/issues/61602
2024-03-22 17:49:56 +01:00
Thomas A Caswell c4bf58a14f
gh-116745: Remove all internal usage of @LIBPYTHON@ (#116746)
Replace with MODULE_LDFLAGS.
2024-03-22 00:54:50 +01:00
Malcolm Smith 3ec57307e7
gh-71052: Add Android build script and instructions (#116426) 2024-03-22 00:52:29 +01:00
Will Childs-Klein c85d84166a
gh-116333: Relax error string text expectations in SSL-related tests (GH-116334)
* Relax error string text expectations in SSL-related tests

As suggested [here][1], this change relaxes the OpenSSL error string
text expectations in a number of tests. This was specifically done in
support of more easily building CPython [AWS-LC][2], but because AWS-LC
is a fork of [BoringSSL][3], it should increase compatibility with that
library as well.

In addition to the error string relaxations, we also add some guards
around the `tls-unique` channel binding being used with TLSv1.3, as that
feature (described in [RFC 6929][4]) is [not defined][5] for TLSv1.3.

[1]: https://discuss.python.org/t/support-building-ssl-and-hashlib-modules-against-aws-lc/44505/4
[2]: https://github.com/aws/aws-lc
[3]: https://github.com/google/boringssl
[4]: https://datatracker.ietf.org/doc/html/rfc5929#section-3
[5]: https://datatracker.ietf.org/doc/html/rfc8446#appendix-C.5
2024-03-21 19:16:36 +00:00
Victor Stinner abdd1f938f
gh-85283: Build _testconsole extension with limited C API (#117125) 2024-03-21 17:45:43 +01:00
Victor Stinner 8bea6c411d
gh-115754: Add Py_GetConstant() function (#116883)
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.

In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.

Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
2024-03-21 16:07:00 +00:00
Malcolm Smith 1f8b24ef69
gh-71052: Implement `ctypes.util.find_library` on Android (GH-116379) 2024-03-21 14:20:57 +01:00
Tian Gao d16c9d1278
gh-116987: Support class code objects in inspect.findsource() (GH-117025) 2024-03-21 10:30:10 +00:00
Adam Turner 6547330f4e
GH-109653: Defer import of ``importlib.metadata._adapters`` (#109829)
* adapters

* Add comments for deferred imports with links to rationale.

* Add blurb

---------

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2024-03-21 03:49:10 +00:00
Jason R. Coombs 8ad8898420
gh-117089: Move importlib.metadata tests to their own package (#117092)
* Ensure importlib.metadata tests do not leak references in sys.modules.

* Move importlib.metadata tests to their own package for easier syncing with importlib_metadata.

* Update owners and makefile for new directories.

* Add blurb
2024-03-20 17:11:00 -04:00
Petr Viktorin dcaf33a41d
gh-114314: ctypes: remove stgdict and switch to heap types (GH-116458)
Before this change, ctypes classes used a custom dict subclass, `StgDict`,
as their `tp_dict`. This acts like a regular dict but also includes extra information
about the type.

This replaces stgdict by `StgInfo`, a C struct on the type, accessed by
`PyObject_GetTypeData()` (PEP-697).
All usage of `StgDict` (mainly variables named `stgdict`, `dict`, `edict` etc.) is
converted to `StgInfo` (named `stginfo`, `info`, `einfo`, etc.).
Where the dict is actually used for class attributes (as a regular PyDict), it's now
called `attrdict`.

This change -- not overriding `tp_dict` -- is made to make me comfortable with
the next part of this PR: moving the initialization logic from `tp_new` to `tp_init`.

The `StgInfo` is set up in `__init__` of each class, with a guard that prevents
calling `__init__` more than once. Note that abstract classes (like `Array` or
`Structure`) are created using `PyType_FromMetaclass` and do not have
`__init__` called.
Previously, this was done in `__new__`, which also wasn't called for abstract
classes.
Since `__init__` can be called from Python code or skipped, there is a tested
guard to ensure `StgInfo` is initialized exactly once before it's used.

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-20 17:33:08 +01:00
jkriegshauser fc45998007
gh-116773: Ensure overlapped objects on Windows are not deallocated too early by asyncio (GH-116774) 2024-03-20 14:33:28 +00:00
Serhiy Storchaka 519b2ae22b
gh-117021: Fix integer overflow in PyLong_AsPid() on non-Windows 64-bit platforms (GH-117064) 2024-03-20 15:39:53 +02:00
Mark Shannon 15309329b6
GH-108362: Incremental Cycle GC (GH-116206) 2024-03-20 08:54:42 +00:00
Sam Gross 60e105c1c1
gh-113964: Don't prevent new threads until all non-daemon threads exit (#116677)
Starting in Python 3.12, we prevented calling fork() and starting new threads
during interpreter finalization (shutdown). This has led to a number of
regressions and flaky tests. We should not prevent starting new threads
(or `fork()`) until all non-daemon threads exit and finalization starts in
earnest.

This changes the checks to use `_PyInterpreterState_GetFinalizing(interp)`,
which is set immediately before terminating non-daemon threads.
2024-03-19 14:40:20 -04:00
et-repositories 75935746be
gh-116647: Fix recursive child in dataclasses (#116790) 2024-03-19 08:58:40 -06:00