Commit Graph

53608 Commits

Author SHA1 Message Date
blhsing 83ba8c2bba
gh-70764: inspect.getclosurevars now identifies global variables with LOAD_GLOBAL (#120143) 2024-11-05 15:53:54 -08:00
Barney Gale 5e9168492f
pathlib ABCs: defer path joining (#126409)
Defer joining of path segments in the private `PurePathBase` ABC. The new
behaviour matches how the public `PurePath` class handles path segments.

This removes a hard-to-grok difference between the ABCs and the main
classes. It also slightly reduces the size of `PurePath` objects by
eliminating a `_raw_path` slot.
2024-11-05 21:19:36 +00:00
Stephen Morton 78842e4a98
gh-126417: Register multiprocessing proxy types to an appropriate collections.abc class (#126419) 2024-11-05 15:35:45 +05:30
Peter Bierma 1371295e67
gh-126366: Fix crash if `__iter__` raises an exception during `yield from` (#126369) 2024-11-05 15:26:36 +05:30
Victor Stinner 4a0d574273
gh-120057: Add os.reload_environ() function (#126268)
Replace the os.environ.refresh() method with a new
os.reload_environ() function.

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2024-11-05 08:43:34 +01:00
Serhiy Storchaka d3840503b0
gh-126303: Fix pickling and copying of os.sched_param objects (GH-126336) 2024-11-05 08:23:17 +02:00
Barney Gale 9b7294c3a5
GH-126363: Speed up pattern parsing in `pathlib.Path.glob()` (#126364)
The implementation of `Path.glob()` does rather a hacky thing: it calls
`self.with_segments()` to convert the given pattern to a `Path` object, and
then peeks at the private `_raw_path` attribute to see if pathlib removed a
trailing slash from the pattern.

In this patch, we make `glob()` use a new `_parse_pattern()` classmethod
that splits the pattern into parts while preserving information about any
trailing slash. This skips the cost of creating a `Path` object, and avoids
some path anchor normalization, which makes `Path.glob()` slightly faster.
But mostly it's about making the code less naughty.

Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
2024-11-04 19:29:57 +00:00
mpage 2e95c5ba3b
gh-115999: Implement thread-local bytecode and enable specialization for `BINARY_OP` (#123926)
Each thread specializes a thread-local copy of the bytecode, created on the first RESUME, in free-threaded builds. All copies of the bytecode for a code object are stored in the co_tlbc array on the code object. Threads reserve a globally unique index identifying its copy of the bytecode in all co_tlbc arrays at thread creation and release the index at thread destruction. The first entry in every co_tlbc array always points to the "main" copy of the bytecode that is stored at the end of the code object. This ensures that no bytecode is copied for programs that do not use threads.

Thread-local bytecode can be disabled at runtime by providing either -X tlbc=0 or PYTHON_TLBC=0. Disabling thread-local bytecode also disables specialization.

Concurrent modifications to the bytecode made by the specializing interpreter and instrumentation use atomics, with specialization taking care not to overwrite an instruction that was instrumented concurrently.
2024-11-04 11:13:32 -08:00
Nice Zombies 3032fcd90e
gh-119793: Add optional length-checking to `map()` (GH-120471)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>
2024-11-04 15:00:19 +01:00
sobolevn bfc1d2504c
gh-109413: Add more type hints to `libregrtest` (#126352) 2024-11-04 13:15:57 +03:00
Kumar Aditya fe5a6ab7be
gh-126353: remove implicit creation of loop from `asyncio.get_event_loop` (#126354)
Remove implicit creation of loop from `asyncio.get_event_loop`. This is a step forward of deprecating the policy system of asyncio.
2024-11-04 14:21:20 +05:30
Tomas R. ac556a2ad1
gh-126357: Remove gettext import guards (GH-126358) 2024-11-03 18:54:42 +02:00
Tomas R. dcae5cd6ab
gh-104400: Add more tests to pygettext (GH-108173) 2024-11-03 14:01:09 +00:00
Cody Maloney 556dc9b8a7
gh-113977, gh-120754: Remove unbounded reads from zipfile (GH-122101)
GH-113977, GH-120754: Remove unbounded reads from zipfile

Read without a size may read an unbounded amount of data + allocate
unbounded size buffers. Move to capped size reads to prevent potential
issues.

Co-authored-by: Daniel Hillier <daniel.hillier@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-11-02 22:28:51 -07:00
Cody Maloney 9ad57cf58b
gh-120754: Add a strace helper and test set of syscalls for open().read(), Take 2 (#123413) 2024-11-02 20:37:21 -07:00
Lee Dong Wook bd4be5e67d
gh-126317: Simplify pickle code by using itertools.batched() (GH-126323) 2024-11-02 16:07:32 +02:00
simple-is-great cfb1b2f0cb
gh-125522: Remove bare except in test_zlib.test_flushes (gh-126321) 2024-11-02 08:27:07 +00:00
mpage f0c6fccd08
gh-126255: Ignore warning about JIT being deactivated when perf support is active in `test_embed.InitConfigTests.test_initconfig_api` (#126302)
Temporarily ignore warnings about JIT deactivation when perf support is active. 
This will be reverted as soon as a way is found to determine at run time whether the interpreter was built with JIT. Currently, this is not possible on Windows.


Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2024-11-01 23:10:58 +00:00
sobolevn 28b148fb32
gh-126220: Fix crash on calls to `_lsprof.Profiler` methods with 0 args (backportable) (#126271)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-11-02 00:53:29 +03:00
Cody Maloney 72dd4714f9
gh-120754: _io Ensure stat cache is cleared on fd change (#125166)
Performed an audit of `fileio.c` and `_pyio` and made sure anytime the
fd changes the stat result, if set, is also cleared/changed.

There's one case where it's not cleared, if code would clear it in
__init__, keep the memory allocated and just do another fstat with the
existing memory.
2024-11-01 22:50:49 +01:00
Sergey B Kirpichev 51ef54abc4
gh-125916: Adapt functools.reduce() to Argument Clinic (#125999) 2024-11-01 21:15:39 +01:00
Barney Gale 37651cfbce
GH-125413: pathlib ABCs: use `scandir()` to speed up `walk()` (#126262)
Use the new `PathBase.scandir()` method in `PathBase.walk()`, which greatly
reduces the number of `PathBase.stat()` calls needed when walking.

There are no user-facing changes, because the pathlib ABCs are still
private and `Path.walk()` doesn't use the implementation in its superclass.
2024-11-01 18:52:00 +00:00
Barney Gale 68a51e0178
GH-125413: pathlib ABCs: use `scandir()` to speed up `glob()` (#126261)
Use the new `PathBase.scandir()` method in `PathBase.glob()`, which greatly
reduces the number of `PathBase.stat()` calls needed when globbing.

There are no user-facing changes, because the pathlib ABCs are still
private and `Path.glob()` doesn't use the implementation in its superclass.
2024-11-01 17:48:58 +00:00
Bénédikt Tran 464a7a91d0
gh-97850: remove ``find_loader`` and ``get_loader`` from ``pkgutil`` (#119656)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2024-11-01 18:35:22 +02:00
mpage 821759d631
gh-126211: Exclude preprocessor directives from statements containing escaping calls (#126213)
The cases generator inserts code to save and restore the stack pointer around
statements that contain escaping calls. To find the beginning of such statements,
we would walk backwards from the escaping call until we encountered a token that
was treated as a statement terminator. This set of terminators should include
preprocessor directives.
2024-11-01 08:53:03 -07:00
Bénédikt Tran 32e07fd377
gh-111495: improve test coverage of codecs C API (GH-126030)
For now, skip some crashers (tracked in gh-123378).
2024-11-01 13:28:18 +00:00
Jun Komoda 6c67446a6e
gh-125783: Add more tests to prevent regressions with the combination of ctypes and metaclasses. (GH-126126) 2024-11-01 13:50:02 +01:00
Barney Gale 260843df1b
GH-125413: Add `pathlib.Path.scandir()` method (#126060)
Add `pathlib.Path.scandir()` as a trivial wrapper of `os.scandir()`. This
will be used to implement several `PathBase` methods more efficiently,
including methods that provide `Path.copy()`.
2024-11-01 01:19:01 +00:00
J. Nick Koston dd3c0fa3fd
gh-126156: Improve performance of creating `Morsel` objects (#126157)
Replaces the manually constructed loop with a call to `dict.update`
2024-10-31 12:05:40 -07:00
Peter Bierma 01415213d7
gh-126223: Propagate unicode errors in `_interpreters.create()` (#126224)
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-10-31 17:14:37 +03:00
Sergey B Kirpichev 8c22eba877
gh-90370: Argument Clinic: avoid temporary tuple creation for varargs (#126064)
Avoid temporary tuple creation when all arguments either positional-only
or vararg.

Objects/setobject.c and Modules/gcmodule.c adapted. This fixes slight
performance regression for set methods, introduced by gh-115112.
2024-10-31 11:37:03 +01:00
Nico-Posada d07dcce693
gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (#126103) 2024-10-31 10:47:57 +03:00
Barney Gale 951cb2c369
GH-126205: Fix conversion of UNC paths to file URIs (#126208)
File URIs for Windows UNC paths should begin with two slashes, not four.
2024-10-30 22:56:58 +00:00
Johan Förberg 2b2d607095
gh-121267: Improve performance of tarfile (#121267) (#121269)
Tarfile in the default write mode spends much of its time resolving UIDs
into usernames and GIDs into group names. By caching these mappings, a
significant speedup can be achieved.

In my simple benchmark[1], this extra caching speeds up tarfile by 8x.

[1] https://gist.github.com/jforberg/86af759c796199740c31547ae828aef2

---------

Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-10-30 15:08:30 -07:00
Furkan Onder 4f826214b3
gh-60712: Include the "object" type in the lists of documented types (GH-103036)
* add test for the predefined object's attributes

* Include the "object" type in the lists of documented types

* remove 'or' from augment tuple

* 📜🤖 Added by blurb_it.

* Add cross-reference to news

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>

* Fix format for the function parameter

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>

* Add space

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>

* add reference for the 'object'

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>

* add reference for NotImplemented

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>

* Change ref:`string <textseq>`  as class:`str`

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>

* remove hyphen from `newly-created`

* Update Doc/reference/datamodel.rst

'dictionaries' to 'dict'

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>

* Update predefined attribute types in testPredefinedAttrs

* Change `universal type` as `top type`

* Don't mention about the top type

* Update the description of richcmpfuncs

* Update Doc/library/stdtypes.rst

Co-authored-by: Éric <merwok@netwok.org>

* Revert: Hierarchy Section in Data Model Documentation

* Revert to original explanations of __new__ and __init__ methods in datamodel.rst for improved clarity.

* Update Doc/reference/datamodel.rst

Co-authored-by: Éric <merwok@netwok.org>

* Remove blank line

Co-authored-by: Éric <merwok@netwok.org>

* Use ref:`str <textseq>` instead of :class:`str

Co-authored-by: Éric <merwok@netwok.org>

* Revert changes the description of Other Built-in Types in stdtypes.rst

* Update Doc/reference/datamodel.rst

Co-authored-by: Éric <merwok@netwok.org>

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
2024-10-30 11:20:02 -07:00
Serhiy Storchaka cc9a183993
gh-126068: Fix exceptions in the argparse module (GH-126069)
* Only error messages for ArgumentError and ArgumentTypeError are now
  translated.
* ArgumentError is now only used for command line errors, not for logical
  errors in the program.
* TypeError is now raised instead of ValueError for some logical errors.
2024-10-30 18:14:27 +02:00
Xuanteng Huang 35df4eb959
gh-126072: do not add `None` to `co_consts` if there is no docstring (GH-126101) 2024-10-30 09:01:09 +00:00
Brian Schubert 224c370a36
gh-126139: Improve error message location for future statement with unknown feature (#126140) 2024-10-29 23:57:59 +00:00
Pablo Galindo Salgado 9dfef4e5f4
gh-125588: Teach the python PEG generator the new f-string tokens (#125589)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2024-10-29 23:40:12 +00:00
Kirill Podoprigora 4877e33708
gh-126146: Remove ``__cmp__`` method from tests (#126147)
Remove ``__cmp__`` method from the tests because it was removed in 3.0 version.
2024-10-30 00:47:26 +02:00
Barney Gale 6742f14dfd
GH-125866: Improve tests for `pathname2url()` and `url2pathname()` (#125993)
Merge `URL2PathNameTests` and `PathName2URLTests` test cases (which test
only the Windows-specific implementations from `nturl2path`) into the main
`Pathname_Tests` test case for these functions.

Copy/port some test cases for `pathlib.Path.as_uri()` and `from_uri()`.
2024-10-29 20:44:57 +00:00
Christian Ullrich 802d405ff1
gh-126084: Fix venvwlauncher.exe to run pythonw.exe (GH-126088) 2024-10-29 19:52:52 +00:00
Tomas R. aab58a93ef
gh-118423: Add `INSTRUCTION_SIZE` macro to code generator (GH-125467) 2024-10-29 17:25:05 +00:00
sobolevn b2eaa75b17
gh-126105: Fix crash in `ast` module, when `._fields` is deleted (#126115)
Previously, if the `ast.AST._fields` attribute was deleted, attempts to create a new `as`t node would crash due to the assumption that `_fields` always had a non-NULL value. Now it has been fixed by adding an extra check to ensure that `_fields` does not have a NULL value (this can happen when you manually remove `_fields` attribute).
2024-10-29 17:42:48 +02:00
Mark Shannon faa3272fb8
GH-125837: Split `LOAD_CONST` into three. (GH-125972)
* Add LOAD_CONST_IMMORTAL opcode

* Add LOAD_SMALL_INT opcode

* Remove RETURN_CONST opcode
2024-10-29 11:15:42 +00:00
Taneli Hukkinen 67f5c5bd6f
tomllib: Add a comment about implicit lru_cache bound (GH-126078) 2024-10-29 11:36:14 +01:00
Sergey B Kirpichev 9b14083497
Align functools.reduce() docstring with PEP-257 (#126045)
Yak-shave in preparation for Argument Clinic adaption in gh-125999.
2024-10-29 08:08:08 +00:00
sobolevn aeafaf4cda
gh-126014: Ignore `__pycache__`-only folders in makefile tests (#126066)
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
2024-10-29 10:43:48 +03:00
Tomas R. 6870eb3f73
gh-124295: Skip translation tests when pygettext is missing (GH-126051) 2024-10-27 18:09:08 +00:00
Bénédikt Tran ed5059eeb1
gh-125966: fix use-after-free on `fut->fut_callback0` due to an evil callback's `__eq__` in asyncio (#125967) 2024-10-27 22:40:10 +05:30