This is the culmination of PEP 684 (and of my 8-year long multi-core Python project)!
Each subinterpreter may now be created with its own GIL (via Py_NewInterpreterFromConfig()). If not so configured then the interpreter will share with the main interpreter--the status quo since subinterpreters were added decades ago. The main interpreter always has its own GIL and subinterpreters from Py_NewInterpreter() will always share with the main interpreter.
Use `io.BufferedWriter` to buffer gzip writes.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Replaces our built-in SHA3 implementation with a verified one from the HACL* project.
This implementation is used when OpenSSL does not provide SHA3 or is not present.
3.11 shiped with a very slow tiny sha3 implementation to get off of the <=3.10 reference implementation that wound up having serious bugs. This brings us back to a reasonably performing built-in implementation consistent with what we've just replaced our other guaranteed available standard hash algorithms with: code from the HACL* project.
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Stop de-duplicating results in `_RecursiveWildcardSelector`. A new
`_DoubleRecursiveWildcardSelector` class is introduced which performs
de-duplication, but this is used _only_ for patterns with multiple
non-adjacent `**` segments, such as `path.glob('**/foo/**')`. By avoiding
the use of a set, `PurePath.__hash__()` is not called, and so paths do not
need to be stringified and case-normalised.
Also merge adjacent '**' segments in patterns.
Improve performance of `pathlib.Path.absolute()` and `cwd()` by joining paths only when necessary. Also improve
performance of `PurePath.is_absolute()` on Posix by skipping path parsing and normalization.
`ast.Num`, `ast.Str`, `ast.Bytes`, `ast.Ellipsis` and `ast.NameConstant` now all emit deprecation warnings on import, access, instantation or `isinstance()` checks.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Remove the unused, private, and undocumented name `_ver` and
the commented-out `print` call.
* Don't add math functions to `__all__`. Beginners should learn
to `import math` to access them.
* Gregor Lindel, who wrote this version of turtle, dropped plans
to implement turtle on another toolkit at least a decade ago.
Drop `_dot` code preparing for this, but add a hint comment.
* `_Screen` is meant to be a singleton class. To enforce that,
it needs either a `__new__` that returns the singleton or
`else...raise` in `__iter__`. Merely removing the `if` clauses
as suggested might break something if a user were to call `_Screen`
directly. Leave the code alone until a problem is evident.
* Turtledemo injects into _Screen both _root and _canvas,
configured as it needs them to be. Making _canvas an `__init__`
option would require skipping some but not all of the lines under
'if _Screen._canvas is None:`. Leave working code alone.
Add `pathlib.PurePath.with_segments()`, which creates a path object from arguments. This method is called whenever a derivative path is created, such as from `pathlib.PurePath.parent`. Subclasses may override this method to share information between path objects.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
This argument allows case-sensitive matching to be enabled on Windows, and
case-insensitive matching to be enabled on Posix.
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
This test can fail unnecessarily. In the test we wait for events on two
file descriptors. This is done in a single call to select.epoll's poll()
function. However, it is valid for the OS to return only one event via
poll() and the next via a subsequent call to poll(). This rarely
happens, but it can cause the test to fail despite properly functioning
polling.
Instead, we poll a second time when necessary.
We now use `_WildcardSelector` to evaluate literal pattern segments, which
allows us to retrieve the real filesystem case.
This change is necessary in order to implement a *case_sensitive* argument
(see GH-81079) and a *follow_symlinks* argument (see GH-77609).
* Remove deprecated classes from pkgutil
* Remove some other PEP 302 obsolescence
* Use find_spec instead of load_module
* Remove more tests of PEP 302 obsolete APIs
* Remove another bunch of tests using obsolete load_modules()
* Remove deleted names from __all__
* Remove obsolete footnote
* imp is removed
* Remove `imp` from generated stdlib names
* What's new and blurb
* Update zipimport documentation for the removed methods
* Fix some Windows tests
* Remove any test (or part of a test) that references `find_module()`.
* Use assertIsNone() / assertIsNotNone() consistently.
* Update Doc/reference/import.rst
* We don't need pkgutil._get_spec() any more either
* test.test_importlib.fixtures.NullFinder
* ...BadLoaderFinder.find_module
* ...test_api.InvalidatingNullFinder.find_module
* ...test.test_zipimport test of z.find_module
* Suppress cross-references to find_loader and find_module
* Suppress cross-references to Finder
* Suppress cross-references to pkgutil.ImpImporter and pkgutil.ImpLoader
---------
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Adam Turner <9087854+aa-turner@users.noreply.github.com>
The bitwise inversion operator on bool returns the bitwise inversion of the
underlying int value; i.e. `~True == -2` such that `bool(~True) == True`.
It's a common pitfall that users mistake `~` as negation operator and actually
want `not`. Supporting `~` is an artifact of bool inheriting from int. Since there
is no real use-case for the current behavior, let's deprecate `~` on bool and
later raise an error. This removes a potential source errors for users.
Full reasoning: https://github.com/python/cpython/issues/82012#issuecomment-1258705971
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Do not expose the local server's on-disk location from `SimpleHTTPRequestHandler` when generating a directory index. (unnecessary information disclosure)
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
We also expose PyInterpreterConfig. This is part of the PEP 684 (per-interpreter GIL) implementation. We will add docs as soon as we can.
FYI, I'm adding the new config field for per-interpreter GIL in gh-99114.
These segments do not require a `stat()` call, as the selector's
`_select_from()` method is called after we've established that the
parent is a directory.
Check that arguments are strings before calling `os.path.join()`.
Also improve performance of `PurePath(PurePath(...))` while we're in the
area: we now use the *unnormalized* string path of such arguments.
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Avoid a potential `ResourceWarning` in `http.client.HTTPConnection`
by closing the proxy / tunnel's CONNECT response explicitly.
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org>
PEP-0682 specified that %-formatting would not support the "z" specifier,
but it was unintentionally allowed for bytes. This PR makes use of the "z"
flag an error for %-formatting in a bytestring.
Issue: #104018
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Add a `teleport` method to `turtle` module turtle instances that acts a lot like `goto`, _but_ ensures the pen is up while warping to the new position to and can control shape filling behavior as part of the jump.
Based on an educator user feature request.
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
The default task name is "Task-<counter>" (if no name is passed in during Task creation).
This is initialized in `Task.__init__` (C impl) using string formatting, which can be quite slow.
Actually using the task name in real world code is not very common, so this is wasted init.
Let's defer this string formatting to the first time the name is read (in `get_name` impl),
so we don't need to pay the string formatting cost if the task name is never read.
We don't change the order in which tasks are assigned numbers (if they are) --
the number is set on task creation, as a PyLong instead of a formatted string.
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
subprocess's communicate(None) closes stdin of the child process, after
sending no (extra) data. Make asyncio variant do the same.
This fixes issues with processes that waits for EOF on stdin before
continuing.
Using `datetime.datetime.utcnow()` and `datetime.datetime.utcfromtimestamp()` will now raise a `DeprecationWarning`.
We also have removed our internal uses of these functions and documented the change.
This change removes the alias of the 'C' locale to 'en_US'. Because of
this alias, it is currently impossible for an application to use
setlocale() to specify a UTF-8 locale on a system that has no locales
installed, but which supports the C.UTF-8 locale/encoding.
gh-82814: Adds `errno.EACCES` to the list of ignored errors on
`_copyxattr`. EPERM and EACCES are different constants but
in general should be treated the same.
News entry authored by: Gregory P. Smith <greg@krypto.org>
This PR makes three minor linting adjustments to the `wasm` module
caught by [ruff](https://github.com/charliermarsh/ruff).
<!-- gh-issue-number: gh-103801 -->
* Issue: gh-103801
<!-- /gh-issue-number -->
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
sockserver gains ForkingUnixStreamServer and ForkingUnixDatagramServer classes for consistency with all of the others. Ironically these existed but were buried in our test suite.
Addresses #103673
<!-- gh-issue-number: gh-103673 -->
* Issue: gh-103673
<!-- /gh-issue-number -->
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
This speeds up `super()` (by around 85%, for a simple one-level
`super().meth()` microbenchmark) by avoiding allocation of a new
single-use `super()` object on each use.
We replace _PyRuntime.tstate_current with a thread-local variable. As part of this change, we add a _Py_thread_local macro in pyport.h (only for the core runtime) to smooth out the compiler differences. The main motivation here is in support of a per-interpreter GIL, but this change also provides some performance improvement opportunities.
Note that we do not provide a fallback to the thread-local, either falling back to the old tstate_current or to thread-specific storage (PyThread_tss_*()). If that proves problematic then we can circle back. I consider it unlikely, but will run the buildbots to double-check.
Also note that this does not change any of the code related to the GILState API, where it uses a thread state stored in thread-specific storage. I suspect we can combine that with _Py_tss_tstate (from here). However, that can be addressed separately and is not urgent (nor critical).
(While this change was mostly done independently, I did take some inspiration from earlier (~2020) work by @markshannon (main...markshannon:threadstate_in_tls) and @vstinner (#23976).)
This is the implementation of PEP683
Motivation:
The PR introduces the ability to immortalize instances in CPython which bypasses reference counting. Tagging objects as immortal allows up to skip certain operations when we know that the object will be around for the entire execution of the runtime.
Note that this by itself will bring a performance regression to the runtime due to the extra reference count checks. However, this brings the ability of having truly immutable objects that are useful in other contexts such as immutable data sharing between sub-interpreters.
* Doc: Fix broken links reported by linkcheck
* Apply suggestions from code review
- Remove extra diff line in faq/library.rst (merwok)
- Use HTTPS to link Unicode 15.0.0 to solve a redirect (hugovk)
- Use wayback machine link for openssl 1.1.0 instead of linking 1.1.1, "as this text mentions a feature from 1.1.0" (hugovk)
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Doc: Make mark-up code as literal
* Doc: Alphabetize items in linkcheck_ignore
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Doc: Improve comment in sphinx conf
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
---------
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* bugfix: let the HTTP- and HTTPSHandlers respect the value of http.client.HTTPConnection.debuglevel
* add tests
* add news
* ReSTify NEWS and reword a bit.
* Address Review Comments.
* Use mock.patch.object instead of settting the module level value.
* Used test values to assert the debuglevel.
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Senthil Kumaran <senthil@python.org>
For example:
class Book(StrEnum):
title = auto()
author = auto()
desc = auto()
Book.author.desc is Book.desc
but
Book.author.title() == 'Author'
is commonly expected. Using upper-case member names avoids this confusion and possible performance impacts.
Co-authored-by: samypr100 <3933065+samypr100@users.noreply.github.com>
Remove the bundled setuptools wheel from ensurepip, and stop installing setuptools in environments created by venv.
Co-Authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
We no longer add a root to device paths such as `//./PhysicalDrive0`,
`//?/BootPartition` and `//./c:` while normalizing. We also avoid adding a
root to incomplete UNC share paths, like `//`, `//a` and `//a/`.
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Fixes unittest.mock.patch not enforcing function signatures for methods
decorated with @classmethod or @staticmethod when patch is called with
autospec=True.
* The majority of the monitoring code is in instrumentation.c
* The new instrumentation bytecodes are in bytecodes.c
* legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
* Made all the command part of the docstring match the official documentation
* Always have a space between the command and the description in docstring
* Added a helper function to format the help message
Before:
```
(Pdb) h a
a(rgs)
Print the argument list of the current function.
(Pdb) h commands
commands [bpnumber]
(com) ...
(com) end
(Pdb)
...
(Pdb) h interact
interact
Start an interactive interpreter whose global namespace
contains all the (global and local) names found in the current scope.
```
After
```
(Pdb) h a
Usage: a(rgs)
Print the argument list of the current function.
(Pdb) h commands
Usage: (Pdb) commands [bpnumber]
(com) ...
(com) end
(Pdb)
...
(Pdb) h interact
Usage: interact
Start an interactive interpreter whose global namespace
contains all the (global and local) names found in the current scope.
```
Automerge-Triggered-By: GH:brandtbucher
Improve performance of path construction by skipping the addition of the path anchor (`drive + root`) to the internal `_parts` list. Rename this attribute to `_tail` for clarity.
* Quote paths in os.spawn tests on Windows so they work with spaces
* Add NEWS entry for os spawn test fix
* Fix code style to avoid double negative in os.spawn tests
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
---------
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Set `SHELL = /bin/sh -e` to ensure that complex recipes fail on the first error rather than incorrectly reporting success.
Co-authored-by: Zachary Ware <zach@python.org>
Several platforms don't define the static_assert macro despite having
compiler support for the _Static_assert keyword. The macro needs to be
defined since it is used unconditionally in the Python code. So it
should always be safe to define it if undefined and not in C++11 (or
later) mode.
Hence, remove the checks for particular platforms or libc versions,
and just define static_assert anytime it needs to be defined but isn't.
That way, all platforms that need the fix will get it, regardless of
whether someone specifically thought of them.
Also document that certain macOS versions are among the platforms that
need this.
The C2x draft (currently expected to become C23) makes static_assert
a keyword to match C++. So only define the macro for up to C17.
Co-authored-by: Victor Stinner <vstinner@python.org>
* bpo-22708: Upgrade HTTP CONNECT to protocol HTTP/1.1 (GH-NNNN)
Use protocol HTTP/1.1 when sending HTTP CONNECT tunnelling requests;
generate Host: headers if one is not already provided (required by
HTTP/1.1), convert IDN domains to punycode in HTTP CONNECT requests.
* Refactor tests to pass under -bb (fix ByteWarnings); missed some lines >80.
* Use consistent 'tunnelling' spelling in Lib/http/client.py
* Lib/test/test_httplib: Remove remnant of obsoleted test.
* Use dict.copy() not copy.copy()
* fix version changed
* Update Lib/http/client.py
Co-authored-by: bgehman <bgehman@users.noreply.github.com>
* Switch to for/else: syntax, as suggested
* Don't use for: else:
* Sure, fine, w/e
* Oops
* 1nm to the left
---------
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: bgehman <bgehman@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Fix an issue where `__new__()` and `__init__()` were not called on subclasses of `pathlib.PurePath` and `Path` in some circumstances.
Paths are now normalized on-demand. This speeds up path construction, `p.joinpath(q)`, and `p / q`.
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
On content update, builds `python` and the docs. Also adds a Dockerfile that should include everything but autoconf 2.69 that's necessary to build CPython and the entire stdlib on Fedora.
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
Co-authored-by: Dusty Phillips <dusty@phillips.codes>
There are some warnings if build python via clang:
Parser/pegen.c:812:31: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
_PyPegen_clear_memo_statistics()
^
void
Parser/pegen.c:820:29: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
_PyPegen_get_memo_statistics()
^
void
Fix it to make clang happy.
Signed-off-by: Chenxi Mao <chenxi.mao@suse.com>
This commit prefixes `__dataclass` to several things in the locals dict:
- Names like `_dflt_` (which cause trouble, see first test)
- Names like `_type_` (not known to be able to cause trouble)
- `_return_type` (not known to able to cause trouble)
- `_HAS_DEFAULT_FACTORY` (which causes trouble, see second test)
In addition, this removes `MISSING` from the locals dict. As far as I can tell, this wasn't needed even in the initial implementation of dataclasses.py (and tests on that version passed with it removed). This makes me wary :-)
This is basically a continuation of #96151, where fixing this was welcomed in https://github.com/python/cpython/pull/98143#issuecomment-1280306360
In PEM, we need to parse until error and then suppress `PEM_R_NO_START_LINE`, because PEM allows arbitrary leading and trailing data. DER, however, does not. Parsing until error and suppressing `ASN1_R_HEADER_TOO_LONG` doesn't quite work because that error also covers some cases that should be rejected.
Instead, check `BIO_eof` early and stop the loop that way.
Automerge-Triggered-By: GH:Yhg1s
Previously, this used to fail:
```py
from typing import *
T = TypeVar("T")
P = ParamSpec("P")
class X(Generic[P]):
f: Callable[P, int]
Y = X[[int, T]]
Z = Y[str]
```
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Also use `raise TimeOut from <CancelledError instance>` so that the CancelledError is set
in the `__cause__` field rather than in the `__context__` field.
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Eric Snow
Arjun
Itamar Ostricher
Jonathan Protzenko
chgnrdv
Barney Gale
Arthur Pastel
Erlend E. Aasland
Oleg Iarygin
Alex Waygood
Terry Jan Reedy
Alexey Namyotkin
Tian Gao
Mark Shannon
Dong-hee Na
Brandt Bucher
Michael Blahay
Shantanu
Jelle Zijlstra
Gleb Smirnoff
Kevin Krakauer
Petr Viktorin
Nikita Sobolev
Adam Turner
Barry Warsaw
Paul Ganssle
Tim Hoffmann
andrei kulakov
Ethan Furman
Juhi Chandalia
Prince Roshan
Jurica Bradarić
Mariusz Felisiak
Rafael Fontenelle
Thomas Grainger
John Belmonte
Franek Magiera
sunmy2019
Carey Metcalfe
Liam Gersten
Pieter Eendebak
Ken Jin
Marek Marczykowski-Górecki
Paul Ganssle
Irit Katriel
Itayazolay
Reese Hyde
Stefano Rivera
Aidan Melen
Paul Moore
Gordon Messmer
Carl Meyer
Samuel Sloniker
Allan Lago
Daniel Versoza
jb2170
Łukasz Langa
Brett Simmers
Christopher Chavez
Senthil Kumaran
James Hilton-Balfe
Adrian Garcia Badaracco
Eddie Elizondo
Eric Wieser
Jason R. Coombs
Wheeler Law
Alexander Belopolsky
Pablo Galindo Salgado
Pradyun Gedam
AN Long
Zachary Ware
Tomas R
Ali-Akber Saifee
Skip Montanaro
Stanislav Syekirin
Bar Harel
David Ellis
C.A.M. Gerlach
Russell Keith-Magee
Michał Górny
Brett Cannon
Jeffrey Newman
Joshua Root
Yeojin Kim
Michael Handler
Charles Machalow
Ned Deily
Thomas Wouters
Steve Dower
Giampaolo Rodola
James De Bias
Furkan Onder
Jeong, YunWon
Bernhard Wagner
Chenxi Mao
JakobDev
David Benjamin
Hugo van Kemenade
MonadChains
Kristján Valur Jónsson