Part of #95913
Forward port of #93306, which was a backport of #93185, to address #84694
This adds the What's New entry for the removal of the subinterpreter-related env variable, build-time flag, etc. As @ericsnowcurrently was author of the original changes, I added him as a co-author to the commit.
This addition to the Python 3.11 What's New document were only made to the Python 3.11 branch during the backport process, and not added to the version in `main`. Forward-porting it ensures the docs retain these additions for the future, rather than being lost in a legacy Python versions, allows it to be be edited as part of #95913 , and avoids merge conflicts with routine back-ports of PRs touching it.
I've pulled in the addition exactly as-is with no modifications; any editing will be done in future PRs (and therefore can be reviewed and backported accordingly).
The one other such addition is forward-ported in #98344
This is the next step for deprecating child watchers.
Until we've removed the API completely we have to use it, so this PR is mostly suppressing a lot of warnings when using the API internally.
Once the child watcher API is totally removed, the two child watcher implementations we actually use and need (Pidfd and Thread) will be turned into internal helpers.
* Some formatting changes for general faq
* Use list for Python versioning
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
* New line for list, list for a/b/rc
* Line wrap for 80 chars
* More line wrap
* Remove PythonWin mention.
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
The definition of obj in the `Py_buffer` struct is as a PyObject*
ec091bd47e/Include/pybuffer.h (L22)
PyMemoryView_GET_BASE returns `.obj` - thus its return type
should be a PyObject* (or at least a void*). It definitely
doesn't return `Py_buffer`
* Add walrus operator to the index
* Add named expression to the index
Co-authored-by: Mariatta Wijaya <Mariatta@users.noreply.github.com>
* Fix indentation and add missing newline
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Mariatta Wijaya <Mariatta@users.noreply.github.com>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Remove diff.py and ndiff.py scripts of Tools/scripts/: move them to
Doc/includes/.
* diff.py and ndiff.py files are no longer executable. Remove also
their shebang ("#!/usr/bin/env python3").
* Remove the -profile command from ndiff.py to simply the code.
* Remove ndiff.py copyright and history command. The Python
documentation examples are distributed under the "Zero Clause BSD
License".
In `_warnings.c`, in the C equivalent of `warnings.warn_explicit()`, if the module globals are given (and not None), the warning will attempt to get the source line for the issued warning. To do this, it needs the module's loader.
Previously, it would only look up `__loader__` in the module globals. In https://github.com/python/cpython/issues/86298 we want to defer to the `__spec__.loader` if available.
The first step on this journey is to check that `loader == __spec__.loader` and issue another warning if it is not. This commit does that.
Since this is a PoC, only manual testing for now.
```python
# /tmp/foo.py
import warnings
import bar
warnings.warn_explicit(
'warning!',
RuntimeWarning,
'bar.py', 2,
module='bar knee',
module_globals=bar.__dict__,
)
```
```python
# /tmp/bar.py
import sys
import os
import pathlib
# __loader__ = pathlib.Path()
```
Then running this: `./python.exe -Wdefault /tmp/foo.py`
Produces:
```
bar.py:2: RuntimeWarning: warning!
import os
```
Uncomment the `__loader__ = ` line in `bar.py` and try it again:
```
sys:1: ImportWarning: Module bar; __loader__ != __spec__.loader (<_frozen_importlib_external.SourceFileLoader object at 0x109f7dfa0> != PosixPath('.'))
bar.py:2: RuntimeWarning: warning!
import os
```
Automerge-Triggered-By: GH:warsaw
* Make the same version versionadded oneline
* Format versionadded for enum.rst
* Format versionadded
A single line versionadded was reading better.
Co-authored-by: Senthil Kumaran <senthil@python.org>
The current re.VERBOSE documentation example leaves space for ambiguous
interpretation. One may read that spaces within the `(?:` token are
spaces inside the non-capturing group (such as `(?: )`). This patch
removes the ambiguity by including examples after the statement.
* Add/refine cross references to items in other lang changes section
* Unify context manager exception changes into single non-repetitive item
* More clearly describe the intent and consequences of the -P option
* Apply minor clarifications & copyedits to rest of section
* Tweak the formatting of module references
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Fix backslashes in AF_PIPE (#88355)
The correct syntax for AF_PIPE addresses is `\\.\pipe\blahblah`, not `\.\pipe{blahblah}`, but the syntax markup messed up the backslashes.
* Move Windows py.exe improvements from Typing section to New Features
* Add ref target label and use literal for py.exe
* Be clearer/explict about what legacy version arg components reprisent
* Apply other minor clarity and textual fixes to py.exe launcher text
* Refine phrasing of legacy sentence of py.exe desc
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
It had to live as a global outside of PyConfig for stable ABI reasons in
the pre-3.12 backports.
This removes the `_Py_global_config_int_max_str_digits` and gets rid of
the equivalent field in the internal `struct _is PyInterpreterState` as
code can just use the existing nested config struct within that.
Adds tests to verify unique settings and configs in subinterpreters.
Remove the Tools/demo/ directory which contained old demo scripts. A
copy can be found in the old-demos project:
https://github.com/gvanrossum/old-demos
Remove the following old demo scripts:
* beer.py
* eiffel.py
* hanoi.py
* life.py
* markov.py
* mcast.py
* queens.py
* redemo.py
* rpython.py
* rpythond.py
* sortvisu.py
* spreadsheet.py
* vector.py
Changes:
* Remove a reference to the redemo.py script in the regex howto
documentation.
* Remove a reference to the removed Tools/demo/ directory in the
curses documentation.
* Update PC/layout/ to remove the reference to Tools/demo/ directory.
* gh-97740: Fix bang in Sphinx C domain ref target syntax
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* Add NEWS entry for C domain bang fix
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* Don't parse content as arg in the impl-detail directive
This does not change the (untranslated) output,
but ensures that the doctree node metadata is correct.
which fixes gh-97607 with the text not being translated.
It also simplifies the code and logic
and makes it consistant with the docutils built-in directives.
* Remove unused branch from impl-detail directive handling no-content case
This is not used anywhere in the docs and lacks a clear use case,
and is more likely a mistake which is now flagged at build time.
This simplifies the logic from two code paths to one,
and makes the behavior consistant with similar built-in directives
(e.g. the various admonition types).
* Further simplify impl-detail reST directive code
* gh-95975: Move except/*/finally ref labels to more precise locations
* Add section headers to fix :keyword: role and aid navigation
* Move see also to the introduction rather than a particular subsection
* Fix other minor Sphinx syntax issues with except
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
* Suppress redundant link to same section for except too
* Don't link try/except/else/finally keywords if in the same section
* Format try/except/finally as keywords in modified sections
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Use output from a 3.10+ REPL, showing invalid range, for the
SyntaxError examples in the tutorial introduction page.
Automerge-Triggered-By: GH:iritkatriel
This Monty Python reference is of-its-time. It could seem inappropriate in the context of today's sensibilities around mental health.
Automerge-Triggered-By: GH:iritkatriel
* gh-68163: Correct conversion of Rational instances to float
Also document that numerator/denominator properties are instances of Integral.
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Martin Panter <vadmium@users.noreply.github.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.
This PR comes fresh from a pile of work done in our private PSRT security response team repo.
Signed-off-by: Christian Heimes [Red Hat] <christian@python.org>
Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org>
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).
<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->
I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#). Much of that text wound up in the Issue. Backports PRs already exist. See the issue for links.
* gh-96132: Add some comments and minor fixes missed in the original PR
* Update Doc/using/cmdline.rst
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
⚠️⚠️ Note for reviewers, hackers and fellow systems/low-level/compiler engineers ⚠️⚠️
If you have a lot of experience with this kind of shenanigans and want to improve the **first** version, **please make a PR against my branch** or **reach out by email** or **suggest code changes directly on GitHub**.
If you have any **refinements or optimizations** please, wait until the first version is merged before starting hacking or proposing those so we can keep this PR productive.
The previous wording of this entry suggests that CPython
won't work if optional compiler features are enabled.
That's not the case. The change is that we require C11 rather
than C89.
Note that PEP 7 does say "Python 3.11 and newer versions use C11
without optional features." It is correct there: that's
not a guide for users who compile Python, but for CPython devs
who must avoid the features.
As a consequence of the added test, this commit also includes
fixes for broken examples.
- Add separate namespace for trace tests bco. module level callback
- Move more backup and cursor examples under separate namespaces
datetime.isoformat generates the tzoffset with colons, but there
was no format code to make strftime output the same format.
for simplicity and consistency the %:z formatting behaves mostly
as %z, with the exception of adding colons. this includes the
dynamic behaviour of adding seconds and microseconds only when
needed (when not 0).
this fixes the still open "generate" part of this issue:
https://github.com/python/cpython/issues/69142
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
* gh-93503: Add APIs to set profiling and tracing functions in all threads in the C-API
* Use a separate API
* Fix NEWS entry
* Add locks around the loop
* Document ignoring exceptions
* Use the new APIs in the sys module
* Update docs
* Add support for the BOLT post-link binary optimizer
Using [bolt](https://github.com/llvm/llvm-project/tree/main/bolt)
provides a fairly large speedup without any code or functionality
changes. It provides roughly a 1% speedup on pyperformance, and a
4% improvement on the Pyston web macrobenchmarks.
It is gated behind an `--enable-bolt` configure arg because not all
toolchains and environments are supported. It has been tested on a
Linux x86_64 toolchain, using llvm-bolt built from the LLVM 14.0.6
sources (their binary distribution of this version did not include bolt).
Compared to [a previous attempt](https://github.com/faster-cpython/ideas/issues/224),
this commit uses bolt's preferred "instrumentation" approach, as well as adds some non-PIE
flags which enable much better optimizations from bolt.
The effects of this change are a bit more dependent on CPU microarchitecture
than other changes, since it optimizes i-cache behavior which seems
to be a bit more variable between architectures. The 1%/4% numbers
were collected on an Intel Skylake CPU, and on an AMD Zen 3 CPU I
got a slightly larger speedup (2%/4%), and on a c6i.xlarge EC2 instance
I got a slightly lower speedup (1%/3%).
The low speedup on pyperformance is not entirely unexpected, because
BOLT improves i-cache behavior, and the benchmarks in the pyperformance
suite are small and tend to fit in i-cache.
This change uses the existing pgo profiling task (`python -m test --pgo`),
though I was able to measure about a 1% macrobenchmark improvement by
using the macrobenchmarks as the training task. I personally think that
both the PGO and BOLT tasks should be updated to use macrobenchmarks,
but for the sake of splitting up the work this PR uses the existing pgo task.
* Simplify the build flags
* Add a NEWS entry
* Update Makefile.pre.in
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Update configure.ac
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Add myself to ACKS
* Add docs
* Other review comments
* fix tab/space issue
* Make it more clear that --enable-bolt is experimental
* Add link to bolt's github page
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Treat tp_weakref and tp_dictoffset like other opaque slots for multiple inheritance.
* Document Py_TPFLAGS_MANAGED_DICT and Py_TPFLAGS_MANAGED_WEAKREF in what's new.
- Limited API needs to be enabled per source file
- Some builds don't support Limited API, so Limited API tests must be skipped on those builds
(currently this is `Py_TRACE_REFS`, but that may change.)
- `Py_LIMITED_API` must be defined before `<Python.h>` is included.
This puts the hoop-jumping in `testcapi/parts.h`, so individual
test files can be relatively simple. (Currently that's only
`vectorcall_limited.c`, imagine more.)
C.A.M. Gerlach
Victor Stinner
Raymond Hettinger
Joannah Nanjekye
Stanley
Vinay Sajip
Kumar Aditya
Simon Legner
BiscuitCandy
QuakeIV
Julien Palard
Nikita Sobolev
Jelle Zijlstra
Shantanu
MonadChains
da-woods
Carl Meyer
Géry Ogam
Pablo Galindo Salgado
Hugo van Kemenade
partev
JasonYZ
180909
Eric Wieser
Christopher Chianelli
Barry Warsaw
Xiao Chen
Guido van Rossum
Brett Cannon
Jason R. Coombs
Terry Jan Reedy
Adam Turner
Tshepang Mbambo
Serhiy Storchaka
Jia Junjie
Athos Ribeiro
Shahriar Heidrich
cousteau
Ev2geny
Daniel Giger
Inada Naoki
Ben Faulhaber
matheusja
Rohan Shah
Ivan Kapeykin
Boris Verkhovskiy
Gregory P. Smith
annonm
Will Hawkins
Łukasz Langa
Eddie Hebert
Ofey Chan
Jeff Allen
NoSuck
Andrew Kay
Steve Dower
Harry
Scott Main
Benjamin Peterson
Dong-hee Na
Erlend E. Aasland
adphrost
Laurie O
Alex Waygood
zhanpon
Maksym Medvied
Hendrik Makait
finefoot
Irit Katriel
Pamela Fox
Sergey B Kirpichev
Mehrdad Moradizadeh
Alexandru Mărășteanu
Cheryl Sabella
siph
Gregory P. Smith
Erlend E. Aasland
Piotr Kaznowski
Mark Shannon
Adrian Garcia Badaracco
Tim Burke
Petr Viktorin
Jonathan Oberländer
TW
Ansab Gillani
Ezio Melotti
ov2k
Sam Ezeh
prego
sand8089
Nick Coghlan
Brandt Bucher
MrSuspicious
Kevin Modzelewski