Commit Graph

121402 Commits

Author SHA1 Message Date
Serhiy Storchaka 0c1a42cf9c
gh-87193: Support bytes objects with refcount > 1 in _PyBytes_Resize() (GH-117160)
Create a new bytes object and destroy the old one if it has refcount > 1.
2024-03-25 16:32:11 +01:00
Tian Gao 01e7405da4
gh-112948: Make pdb completion similar to repl completion (#112950) 2024-03-25 15:18:09 +00:00
Raymond Hettinger 9db2a8f914
Minor markup and grammar fixes in the statistics docs (gh-117216) 2024-03-25 09:26:42 -05:00
Kirill Podoprigora eebea7e515
gh-117176: Fix compiler warning in Python/optimizer_bytecodes.c (GH-117199) 2024-03-24 20:34:55 +02:00
Totally a booplicate 83485a0953
gh-112571: Move fish venv activation script into the common folder (GH-117169)
pythongh-112571: allow using fish venv activation script on windows

The fish shell can be used on windows under cygwin or msys2.
This change moves the script to the common folder
so the venv module will install it on both posix and nt systems (like the bash script).
2024-03-24 15:48:40 +00:00
Terry Jan Reedy 78a651fd7f
gh-117194: Properly format 'base64' header in What's New (#117198)
It needs 6, not 3, '-'s.
2024-03-24 11:38:34 -04:00
Kerim Kabirov f267d5bf2a
GH-115986 Docs: promote pprint.pp usage as a default (#116614)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-03-24 07:59:14 -06:00
LilKS 39df773217
gh-101760: Improve the imaplib.IMAP4 example (#101764)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2024-03-24 04:01:07 -06:00
Raymond Hettinger a1e948edba
Add cumulative option for the new statistics.kde() function. (#117033) 2024-03-24 04:35:58 -05:00
Irit Katriel d610d821fd
gh-112383: teach dis how to interpret ENTER_EXECUTOR (#117171) 2024-03-23 22:32:33 +00:00
Ken Jin 6c83352bfe
gh-117180: Complete call sequence when trace stack overflow (GH-117184)
---------

Co-authored-by: Peter Lazorchak <lazorchakp@gmail.com>
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2024-03-24 06:19:17 +08:00
Erik Soma f11d0d8be8
gh-91227: Ignore ERROR_PORT_UNREACHABLE in proactor recvfrom() (#32011) 2024-03-23 08:39:35 -07:00
Victor Stinner 9967b568ed
gh-117008: Fix functools test_recursive_pickle() (#117009)
Use support.infinite_recursion() in test_recursive_pickle() of
test_functools to prevent a stack overflow on "ARM64 Windows
Non-Debug" buildbot.

Lower Py_C_RECURSION_LIMIT to 1,000 frames on Windows ARM64.
2024-03-23 13:01:20 +01:00
Barney Gale 72eea512b8
GH-106747: Document another difference between `glob` and `pathlib`. (#116518)
Document that `path.glob()` might return *path*, whereas
`glob.glob(root_dir=path)` will never return an empty string corresponding
to *path*.
2024-03-22 19:14:09 +00:00
Mark Shannon e28477f214
GH-117108: Change the size of the GC increment to about 1% of the total heap size. (GH-117120) 2024-03-22 18:43:25 +00:00
Serhiy Storchaka e2e0b4b4b9
gh-113024: C API: Add PyObject_GenericHash() function (GH-113025) 2024-03-22 20:19:10 +02:00
Serhiy Storchaka 567ab3bd15
gh-117084: Fix ZIP file extraction for directory entry names with backslashes on Windows (GH-117129) 2024-03-22 20:08:00 +02:00
Serhiy Storchaka 5a78f6e798
gh-117134: Microoptimize glob() for include_hidden=True (GH-117135) 2024-03-22 20:03:48 +02:00
Vinay Sajip 00baaa21de
[docs] Fix typo in docstring and add example to logging cookbook. (GH-117157) 2024-03-22 17:25:51 +00:00
Jakub Stasiak 40d75c2b7f
GH-113171: Fix "private" (non-global) IP address ranges (GH-113179)
* GH-113171: Fix "private" (really non-global) IP address ranges

The _private_networks variables, used by various is_private
implementations, were missing some ranges and at the same time had
overly strict ranges (where there are more specific ranges considered
globally reachable by the IANA registries).

This patch updates the ranges with what was missing or otherwise
incorrect.

I left 100.64.0.0/10 alone, for now, as it's been made special in [1]
and I'm not sure if we want to undo that as I don't quite understand the
motivation behind it.

The _address_exclude_many() call returns 8 networks for IPv4, 121
networks for IPv6.

[1] https://github.com/python/cpython/issues/61602
2024-03-22 17:49:56 +01:00
Steve Dower 3be9b9d872
Fix get_packagefamilyname helper function on Windows 32-bit (GH-117153) 2024-03-22 15:00:50 +00:00
NGRsoftlab 63d6f2623e
gh-117068: Remove useless code in bytesio.c:resize_buffer() (GH-117069)
Co-authored-by: i.khabibulin <i.khabibulin@ngrsoftlab.ru>
2024-03-22 11:25:38 +00:00
Petr Viktorin 42ae924d27
gh-117127: glob tests: Reopen dir_fd to pick up directory changes (GH-117128) 2024-03-22 10:42:18 +01:00
Tim Peters 8383915031
GH-116939: Rewrite binarysort() (#116940)
Rewrote binarysort() for clarity.

Also changed the signature to be more coherent (it was mixing sortslice with raw pointers).

No change in method or functionality. However, I left some experiments in, disabled for now
via `#if` tricks. Since this code was first written, some kinds of comparisons have gotten
enormously faster (like for lists of floats), which changes the tradeoffs.

For example, plain insertion sort's simpler innermost loop and highly predictable branches
leave it very competitive (even beating, by a bit) binary insertion when comparisons are
very cheap, despite that it can do many more compares. And it wins big on runs that
are already sorted (moving the next one in takes only 1 compare then).

So I left code for a plain insertion sort, to make future experimenting easier.

Also made the maximum value of minrun a `#define` (``MAX_MINRUN`) to make
experimenting with that easier too.

And another bit of `#if``-disabled code rewrites binary insertion's innermost loop to
remove its unpredictable branch. Surprisingly, this doesn't really seem to help
overall. I'm unclear on why not. It certainly adds more instructions, but they're very
simple, and it's hard to be believe they cost as much as a branch miss.
2024-03-21 22:27:25 -05:00
Guido van Rossum 97ba910e47
gh-108716:: Remove _PyStaticCode_Init/Fini (#117141)
More deepfreeze cleanup.
2024-03-22 01:27:48 +00:00
Eric Snow b3d25df8d3
gh-105716: Fix _PyInterpreterState_IsRunningMain() For Embedders (gh-117140)
When I added _PyInterpreterState_IsRunningMain() and friends last year, I tried to accommodate applications that embed Python but don't call _PyInterpreterState_SetRunningMain() (not that they're expected to).  That mostly worked fine until my recent changes in gh-117049, where the subtleties with the fallback code led to failures; the change ended up breaking test_tools.test_freeze, which exercises a basic embedding situation.

The simplest fix is to drop the fallback code I originally added to _PyInterpreterState_IsRunningMain() (and later to _PyThreadState_IsRunningMain()).  I've kept the fallback in the _xxsubinterpreters module though.  I've also updated Py_FrozenMain() to call _PyInterpreterState_SetRunningMain().
2024-03-21 18:20:20 -06:00
Thomas A Caswell c4bf58a14f
gh-116745: Remove all internal usage of @LIBPYTHON@ (#116746)
Replace with MODULE_LDFLAGS.
2024-03-22 00:54:50 +01:00
Malcolm Smith 3ec57307e7
gh-71052: Add Android build script and instructions (#116426) 2024-03-22 00:52:29 +01:00
Victor Stinner 50f9b0b1e0
gh-117061: Fix test_posix.test_sched_setaffinity() on RHEL9 (#117126)
On RHEL9, sched_setaffinity(0, []) does not fail.
2024-03-21 22:17:09 +00:00
Ned Batchelder 0907871d43
docs: fix over-linking in dataclasses.rst (#117005) 2024-03-21 12:47:09 -07:00
Guido van Rossum 570a82d46a
gh-117045: Add code object to function version cache (#117028)
Changes to the function version cache:

- In addition to the function object, also store the code object,
  and allow the latter to be retrieved even if the function has been evicted.
- Stop assigning new function versions after a critical attribute (e.g. `__code__`)
  has been modified; the version is permanently reset to zero in this case.
- Changes to `__annotations__` are no longer considered critical. (This fixes gh-109998.)

Changes to the Tier 2 optimization machinery:

- If we cannot map a function version to a function, but it is still mapped to a code object,
  we continue projecting the trace.
  The operand of the `_PUSH_FRAME` and `_POP_FRAME` opcodes can be either NULL,
  a function object, or a code object with the lowest bit set.

This allows us to trace through code that calls an ephemeral function,
i.e., a function that may not be alive when we are constructing the executor,
e.g. a generator expression or certain nested functions.
We will lose globals removal inside such functions,
but we can still do other peephole operations
(and even possibly [call inlining](https://github.com/python/cpython/pull/116290),
if we decide to do it), which only need the code object.
As before, if we cannot retrieve the code object from the cache, we stop projecting.
2024-03-21 12:37:41 -07:00
Will Childs-Klein c85d84166a
gh-116333: Relax error string text expectations in SSL-related tests (GH-116334)
* Relax error string text expectations in SSL-related tests

As suggested [here][1], this change relaxes the OpenSSL error string
text expectations in a number of tests. This was specifically done in
support of more easily building CPython [AWS-LC][2], but because AWS-LC
is a fork of [BoringSSL][3], it should increase compatibility with that
library as well.

In addition to the error string relaxations, we also add some guards
around the `tls-unique` channel binding being used with TLSv1.3, as that
feature (described in [RFC 6929][4]) is [not defined][5] for TLSv1.3.

[1]: https://discuss.python.org/t/support-building-ssl-and-hashlib-modules-against-aws-lc/44505/4
[2]: https://github.com/aws/aws-lc
[3]: https://github.com/google/boringssl
[4]: https://datatracker.ietf.org/doc/html/rfc5929#section-3
[5]: https://datatracker.ietf.org/doc/html/rfc8446#appendix-C.5
2024-03-21 19:16:36 +00:00
Sam Gross 1f72fb5447
gh-116522: Refactor `_PyThreadState_DeleteExcept` (#117131)
Split `_PyThreadState_DeleteExcept` into two functions:

- `_PyThreadState_RemoveExcept` removes all thread states other than one
  passed as an argument. It returns the removed thread states as a
  linked list.

- `_PyThreadState_DeleteList` deletes those dead thread states. It may
  call destructors, so we want to "start the world" before calling
  `_PyThreadState_DeleteList` to avoid potential deadlocks.
2024-03-21 11:21:02 -07:00
Michael Droettboom 50369e6c34
gh-116996: Add pystats about _Py_uop_analyse_and_optimize (GH-116997) 2024-03-22 01:27:46 +08:00
Eric Snow 617158e078
gh-76785: Drop PyInterpreterID_Type (gh-117101)
I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation.  Relatively recently I refactored that implementation to no longer rely on InterpreterID objects.  Thus now I'm removing it.
2024-03-21 17:15:02 +00:00
Victor Stinner abdd1f938f
gh-85283: Build _testconsole extension with limited C API (#117125) 2024-03-21 17:45:43 +01:00
Victor Stinner 8bea6c411d
gh-115754: Add Py_GetConstant() function (#116883)
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.

In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.

Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
2024-03-21 16:07:00 +00:00
Eric Snow 5a76d1be8e
gh-105716: Update interp->threads.main After Fork (gh-117049)
I missed this in gh-109921.

We also update Py_Exit() to call _PyInterpreterState_SetNotRunningMain(), if necessary.
2024-03-21 10:06:35 -06:00
Eric Snow bbee57fa8c
gh-76785: Clean Up Interpreter ID Conversions (gh-117048)
Mostly we unify the two different implementations of the conversion code (from PyObject * to int64_t.  We also drop the PyArg_ParseTuple()-style converter function, as well as rename and move PyInterpreterID_LookUp().
2024-03-21 09:56:12 -06:00
Sam Gross e728303532
gh-116522: Stop the world before fork() and during shutdown (#116607)
This changes the free-threaded build to perform a stop-the-world pause
before deleting other thread states when forking and during shutdown.
This fixes some crashes when using multiprocessing and during shutdown
when running with `PYTHON_GIL=0`.

This also changes `PyOS_BeforeFork` to acquire the runtime lock
(i.e., `HEAD_LOCK(&_PyRuntime)`) before forking to ensure that data
protected by the runtime lock (and not just the GIL or stop-the-world)
is in a consistent state before forking.
2024-03-21 10:01:16 -04:00
Malcolm Smith 1f8b24ef69
gh-71052: Implement `ctypes.util.find_library` on Android (GH-116379) 2024-03-21 14:20:57 +01:00
Tian Gao d16c9d1278
gh-116987: Support class code objects in inspect.findsource() (GH-117025) 2024-03-21 10:30:10 +00:00
Adam Turner 6547330f4e
GH-109653: Defer import of ``importlib.metadata._adapters`` (#109829)
* adapters

* Add comments for deferred imports with links to rationale.

* Add blurb

---------

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2024-03-21 03:49:10 +00:00
Jason R. Coombs 667294d5b2
gh-117089: Apply changes from importlib_metadata 7.1.0 (#117094)
* Apply changes from importlib_metadata 7.1.0

* Include the data sources in the makefile (even though they're not needed)
2024-03-20 23:01:24 -04:00
Victor Stinner f4cc77d494
gh-116869: Enable -Werror in test_cext for Free Threading (#117106)
Check for warnings, but don't enable the compiler flag
-Werror=declaration-after-statement.
2024-03-21 00:06:24 +01:00
Victor Stinner 104602a607
gh-105927: Limit PyWeakref_GetRef() to limited C API 3.13 (#117091) 2024-03-20 23:52:23 +01:00
Jason R. Coombs 8ad8898420
gh-117089: Move importlib.metadata tests to their own package (#117092)
* Ensure importlib.metadata tests do not leak references in sys.modules.

* Move importlib.metadata tests to their own package for easier syncing with importlib_metadata.

* Update owners and makefile for new directories.

* Add blurb
2024-03-20 17:11:00 -04:00
Carol Willing 7d446548ef
Fix sort order for "locale encoding" glossary item (#115794)
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
2024-03-20 15:00:59 -06:00
Mark Shannon 63289b9dfb
GH-117066: Tier 2 optimizer: Don't throw away good traces if we can't optimize them perfectly. (GH-117067) 2024-03-20 18:24:02 +00:00
Petr Viktorin dcaf33a41d
gh-114314: ctypes: remove stgdict and switch to heap types (GH-116458)
Before this change, ctypes classes used a custom dict subclass, `StgDict`,
as their `tp_dict`. This acts like a regular dict but also includes extra information
about the type.

This replaces stgdict by `StgInfo`, a C struct on the type, accessed by
`PyObject_GetTypeData()` (PEP-697).
All usage of `StgDict` (mainly variables named `stgdict`, `dict`, `edict` etc.) is
converted to `StgInfo` (named `stginfo`, `info`, `einfo`, etc.).
Where the dict is actually used for class attributes (as a regular PyDict), it's now
called `attrdict`.

This change -- not overriding `tp_dict` -- is made to make me comfortable with
the next part of this PR: moving the initialization logic from `tp_new` to `tp_init`.

The `StgInfo` is set up in `__init__` of each class, with a guard that prevents
calling `__init__` more than once. Note that abstract classes (like `Array` or
`Structure`) are created using `PyType_FromMetaclass` and do not have
`__init__` called.
Previously, this was done in `__new__`, which also wasn't called for abstract
classes.
Since `__init__` can be called from Python code or skipped, there is a tested
guard to ensure `StgInfo` is initialized exactly once before it's used.

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-20 17:33:08 +01:00