* Add name and mode attributes for compressed and archived file-like objects
in modules bz2, lzma, tarfile and zipfile.
* Change the value of the mode attribute of GzipFile from integer (1 or 2)
to string ('rb' or 'wb').
* Change the value of the mode attribute of ZipExtFile from 'r' to 'rb'.
Tarfile.addfile now throws an ValueError when the user passes
in a non-zero size tarinfo but does not provide a fileobj,
instead of writing an incomplete entry.
The implementation uses 'ptr' for the name of the first parameter of
ctypes.string_at() and ctypes.wstring_at(). Align docs and docstrings
with the naming used in the implementation.
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Since 6258844c, paths that might not exist can be fed into pathlib's
globbing implementation, which will call `os.scandir()` / `os.lstat()` only
when strictly necessary. This allows us to drop an initial `self.is_dir()`
call, which saves a `stat()`.
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
rfc9110 obsoletes the earlier rfc 7231. This document also includes some
status codes that were previously only used for WebDAV and assigns more
generic names to these status codes.
ref: https://www.rfc-editor.org/rfc/rfc9110.html#name-changes-from-rfc-7231
- http.HTTPStatus.CONTENT_TOO_LARGE (413, previously
REQUEST_ENTITY_TOO_LARGE)
- http.HTTPStatus.URI_TOO_LONG (414, previously REQUEST_URI_TOO_LONG)
- http.HTTPStatus.RANGE_NOT_SATISFYABLE (416, previously
REQUEST_RANGE_NOT_SATISFYABLE)
- http.HTTPStatus.UNPROCESSABLE_CONTENT (422, previously
UNPROCESSABLE_ENTITY)
The new constants are added to http.HTTPStatus and the old constant names are
preserved for backwards compatibility.
References in documentation to the obsoleted rfc 7231 are updated
I think the choice of wording in these docs is great and doesn't
need to change. However, it could be useful to explicitly define
this term / the cost of doing so seems relatively low.
gh-16429 introduced support for an iterable of separators in
Stream.readuntil. Since bytes-like types are themselves iterable, this
can introduce ambiguities in deciding whether the argument is an
iterator of separators or a singleton separator. In gh-16429, only 'bytes'
was considered a singleton, but this will break code that passes other
buffer object types.
Fix it by only supporting tuples rather than arbitrary iterables.
Closes gh-117722.
This prevents external cancellations of a task group's parent task to
be dropped when an internal cancellation happens at the same time.
Also strengthen the semantics of uncancel() to clear self._must_cancel
when the cancellation count reaches zero.
Co-Authored-By: Tin Tvrtković <tinchester@gmail.com>
Co-Authored-By: Arthur Tacca
* remove load extension doctest since we cannot skip it conditionally
* remove sys.unraisablehook example; using unraisable hooks is not "an
improved debug experience"
Replace tri-state `follow_symlinks` with boolean `recurse_symlinks` argument. The new argument controls whether symlinks are followed when expanding recursive `**` wildcards. The possible argument values correspond as follows:
follow_symlinks recurse_symlinks
=============== ================
False N/A
None False
True True
We therefore drop support for not following symlinks when expanding non-recursive pattern parts; it wasn't requested in the original issue, and it's a feature not found in any shells.
This makes the API a easier to grok by eliminating `None` as an option.
No news blurb as `follow_symlinks` was new in 3.13.
* as_completed returns object that is both iterator and async iterator
* Existing tests adjusted to test both the old and new style
* New test to ensure iterator can be resumed
* New test to ensure async iterator yields any passed-in Futures as-is
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Jacob Coffee <jacob@z7x.org>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Ned Deily <nad@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Ned Deily <nad@python.org>
* Reads zip64 files as produced by the zipfile module
* Include tests (somewhat slow, however, because of the need to create "large" zips)
* About the same amount of strictness reading invalid zip files as zipfile has
* Still works on files with prepended data (like pex)
There are a lot more test cases at https://github.com/thatch/zipimport64/ that give me confidence that this works for real-world files.
Fixes#89739 and #77140.
---------
Co-authored-by: Itamar Ostricher <itamarost@gmail.com>
Reviewed-by: Gregory P. Smith <greg@krypto.org>
* GH-113171: Fix "private" (really non-global) IP address ranges
The _private_networks variables, used by various is_private
implementations, were missing some ranges and at the same time had
overly strict ranges (where there are more specific ranges considered
globally reachable by the IANA registries).
This patch updates the ranges with what was missing or otherwise
incorrect.
I left 100.64.0.0/10 alone, for now, as it's been made special in [1]
and I'm not sure if we want to undo that as I don't quite understand the
motivation behind it.
The _address_exclude_many() call returns 8 networks for IPv4, 121
networks for IPv6.
[1] https://github.com/python/cpython/issues/61602
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.
In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.
Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
These give applications the option of more forcefully terminating client
connections for asyncio servers. Useful when terminating a service and
there is limited time to wait for clients to finish up their work.
This is a do-over with a test fix for gh-114432, which was reverted.
* GH-65056: Improve the IP address' is_global/is_private documentation
It wasn't clear what the semantics of is_global/is_private are and, when
one gets to the bottom of it, it's not quite so simple (hence the
exceptions listed).
Co-authored-by: Petr Viktorin <encukou@gmail.com>
In Lexical Analysis f-strings section, NULL in the description
of 'literal character' means '\0'. In the format_spec grammar
production, it is wrong with that meaning and redundant if
instead interpreted as <nothing>. Remove it there.
Stop raising `ValueError` from `glob.translate()` when a `**` sub-string
appears in a non-recursive pattern segment. This matches `glob.glob()`
behaviour.
* Update titles and subtitles on landing page template
* address review from gvanrossum
* Edits from hugovk review
* Change word order back. Down the road we should split license and history
On Windows, time.monotonic() now uses the QueryPerformanceCounter()
clock to have a resolution better than 1 us, instead of the
gGetTickCount64() clock which has a resolution of 15.6 ms.
These give applications the option of more forcefully terminating client
connections for asyncio servers. Useful when terminating a service and
there is limited time to wait for clients to finish up their work.
In free-threaded builds, running with `PYTHON_GIL=0` will now disable the
GIL. Follow-up issues track work to re-enable the GIL when loading an
incompatible extension, and to disable the GIL by default.
In order to support re-enabling the GIL at runtime, all GIL-related data
structures are initialized as usual, and disabling the GIL simply sets a flag
that causes `take_gil()` and `drop_gil()` to return early.
* set default return value of functional types as _mock_return_value
* added test of wrapping child attributes
* added backward compatibility with explicit return
* added docs on the order of precedence
* added test to check default return_value
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
This adds `VERIFY_X509_STRICT` to make the default
SSL context perform stricter (per RFC 5280) validation, as well
as `VERIFY_X509_PARTIAL_CHAIN` to enforce more standards-compliant
path-building behavior.
As part of this changeset, I had to tweak `make_ssl_certs.py`
slightly to emit 5280-conforming CA certs. This changeset includes
the regenerated certificates after that change.
Signed-off-by: William Woodruff <william@yossarian.net>
Co-authored-by: Victor Stinner <vstinner@python.org>
Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:
- `xml.etree.ElementTree.XMLParser.flush`
- `xml.etree.ElementTree.XMLPullParser.flush`
- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`
- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`
- `xml.sax.expatreader.ExpatParser.flush`
Based on the "flush" idea from https://github.com/python/cpython/pull/115138#issuecomment-1932444270 .
### Notes
- Please treat as a security fix related to CVE-2023-52425.
Includes code suggested-by: Snild Dolkow <snild@sony.com>
and by core dev Serhiy Storchaka.
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Introduce a new subsubsection, 'Functions', for module level functions,
and place it before the PrettyPrinter class reference.
Also:
- Fix pprint.pprint() references so they properly link to the module
level function.
- Add links to sys.stdout.
Remove a left-over sentence that refers to Py_OptimizeFlag
Remove a left-over sentence that refers to an example that was present in Python 3.10 and was using ``Py_OptimizeFlag``.
Explain the `full_match()` / `glob()` / `rglob()` pattern language in its own section. Move `rglob()` documentation under `glob()` and reduce duplicated text.
* clean up fcntl module doc
* simplify
* a few changes, based on suggestion by CAM-Gerlach
* nitpick ignore for a couple other C functions mentioned in the fcntl module doc
* more changes, especially related to LOCK_* constants
* :data: back to :const:
* Apply suggestions from code review
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
---------
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Content adapted from https://devguide.python.org/development-tools/gdb/#
and https://wiki.python.org/moin/DebuggingWithGdb.
The original content on the Wiki page came from gdb debug help used by
the Launchpad (https://launchpad.net/) team.
Thanks to Anatoly Techtonik and user `rmf` for substantial improvements to the Wiki page.
The history of the Devguide page follows
(with log entries expanded for major content contributions):
Hugo van Kemenade, Sat Dec 30 21:22:04 2023 +0200
Hugo van Kemenade, Fri Dec 8 12:04:32 2023 +0200
Erlend E. Aasland & Hugo van Kemenade, Tue Aug 8 22:05:34 2023 +0200
Satish Mishra, Sat Feb 11 13:54:57 2023 +0530
Hugo van Kemenade, Fri Dec 23 17:33:33 2022 +0200
Skip Montanaro, Hugo, Erlend, & Ezio, Fri Nov 4 05:04:23 2022 -0500
Add a GDB tips section to Advanced Tools (#977)
Adam Turner, Wed Jun 15 21:19:23 2022 +0100
Adam Turner, Tue Jun 14 11:12:26 2022 +0100
Suriyaa, Fri Jun 8 19:39:23 2018 +0200
Jeff Allen, Tue Oct 24 18:12:53 2017 +0100
Jeff Allen, Fri Oct 13 13:43:43 2017 +0100
Mariatta, Wed Jan 4 09:14:55 2017 -0800
Carol Willing, Mon Sep 26 14:50:54 2016 -0700
Zachary Ware, Thu Jul 21 10:42:23 2016 -0500
Georg Brandl, Mon Nov 3 11:28:19 2014 +0100
Add instruction how to activate python-gdb.py
Georg Brandl, Sun Mar 9 10:32:01 2014 +0100
Georg Brandl, Tue Apr 3 09:12:53 2012 +0200
Georg Brandl, Sat Mar 5 17:32:35 2011 +0100
Dave Malcolm, Fri Jan 21 12:34:09 2011 -0500
Add documentation on the gdb extension commands provided in libpython.py
I adapted this from documentation I wrote for the Fedora wiki:
https://fedoraproject.org/wiki/Features/EasierPythonDebugging#New_gdb_commands
reformatting it as rst, and making other minor changes
Brett Cannon, Thu Jan 20 15:16:52 2011 -0800
Dave Malcolm, Thu Jan 20 16:17:23 2011 -0500
Add some notes on the gdb pretty-printer hooks
Antoine Pitrou, Thu Jan 20 21:17:49 2011 +0100
Give an example backtrace
Antoine Pitrou, Thu Jan 20 21:03:06 2011 +0100
Expand explanations about gdb support
Brett Cannon, Thu Jan 20 11:33:36 2011 -0800
Tweak the gdb support title to fit in better with the devguide.
Brett Cannon, Mon Jan 17 21:12:54 2011 +0000
Short README on gdb support.
Co-authored-by: Adam Turner <9087854+aa-turner@users.noreply.github.com>
Co-authored-by: anatoly techtonik <techtonik@gmail.com>
Co-authored-by: Antoine Pitrou <solipsis@pitrou.net>
Co-authored-by: Brett Cannon <brett@python.org>
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
Co-authored-by: Dave Malcolm <dmalcolm@redhat.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Georg Brandl <georg@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Jeff Allen <ja.py@farowl.co.uk>
Co-authored-by: Mariatta <Mariatta@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Satish Mishra <7506satish@gmail.com>
Co-authored-by: Skip Montanaro <skip.montanaro@gmail.com>
Co-authored-by: Suriyaa <isc.suriyaa@gmail.com>
Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
This expands the examples to cover both realistic use cases for the API.
I noticed thing in the test that could be done better so I added those as well: We need to guarantee that all bytes of the result are overwritten and that too many are not written. Tests now pre-fills the result with data in order to ensure that.
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
The charset name "Windows-31J" is registered in the IANA Charset Registry[1]
and is implemented in Python as the cp932 codec.
[1] https://www.iana.org/assignments/charset-reg/windows-31J
Signed-off-by: Masayuki Moriyama <masayuki.moriyama@miraclelinux.com>
Fix the exceptions raised by posixpath.commonpath
Raise ValueError, not IndexError when passed an empty iterable. Raise
TypeError, not ValueError when passed None.
Update documentation for re library to explain that a backreference `\g<0>` is
expanded to the entire string when using Match.expand().
Note that numeric backreferences to group 0 (`\0`) are not supported.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
This adds a safe memory reclamation scheme based on FreeBSD's "GUS" and
quiescent state based reclamation (QSBR). The API provides a mechanism
for callers to detect when it is safe to free memory that may be
concurrently accessed by readers.
* bpo-38364: unwrap partialmethods just like we unwrap partials
The inspect.isgeneratorfunction, inspect.iscoroutinefunction and inspect.isasyncgenfunction already unwrap functools.partial objects, this patch adds support for partialmethod objects as well.
Also: Rename _partialmethod to __partialmethod__.
Since we're checking this attribute on arbitrary function-like objects,
we should use the namespace reserved for core Python.
---------
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Add PythonFinalizationError exception. This exception derived from
RuntimeError is raised when an operation is blocked during the Python
finalization.
The following functions now raise PythonFinalizationError, instead of
RuntimeError:
* _thread.start_new_thread()
* subprocess.Popen
* os.fork()
* os.fork1()
* os.forkpty()
Morever, _winapi.Overlapped finalizer now logs an unraisable
PythonFinalizationError, instead of an unraisable RuntimeError.
Keep the page though, because people might still rely on it (the traffic shows that they do).
Instead of our own manual we now give links to the 3rd-party ones.
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Now the special comparison methods like `__eq__` and `__lt__` return
NotImplemented if one of comparands is date and other is datetime
instead of ignoring the time part and the time zone or forcefully
return "not equal" or raise TypeError.
It makes comparison of date and datetime subclasses more symmetric
and allows to change the default behavior by overriding
the special comparison methods in subclasses.
It is now the same as if date and datetime was independent classes.
By default, it preserves an inconsistent behavior of older Python
versions: packs the count into a 1-tuple if only one or none
options are specified (including 'update'), returns None instead of 0.
Except that setting wantobjects to 0 no longer affects the result.
Add a new parameter return_ints: specifying return_ints=True makes
Text.count() always returning the single count as an integer
instead of a 1-tuple or None.
The `PyDict_SetDefaultRef` function is similar to `PyDict_SetDefault`,
but returns a strong reference through the optional `**result` pointer
instead of a borrowed reference.
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Add optional 'filter' parameter to iterdump() that allows a "LIKE"
pattern for filtering database objects to dump.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Change the somewhat vague "listed below" to "listed in this chapter" in Doc/library/exceptions.rst.
The exceptions are listed in multiple sections after two intermediate sections.
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* When called with a single argument to get a value, it allow to omit
the minus prefix.
* It can be called with keyword arguments to set attributes.
* w.wm_attributes(return_python_dict=True) returns a dict instead of
a tuple (it will be the default in future).
* Setting wantobjects to 0 no longer affects the result.
Update documentation with `__new__` and `__init__` entries.
Support use of `auto()` in tuple subclasses on member assignment lines. Previously, auto() was only supported on the member definition line either solo or as part of a tuple:
RED = auto()
BLUE = auto(), 'azul'
However, since Python itself supports using tuple subclasses where tuples are expected, e.g.:
from collections import namedtuple
T = namedtuple('T', 'first second third')
def test(one, two, three):
print(one, two, three)
test(*T(4, 5, 6))
# 4 5 6
it made sense to also support tuple subclasses in enum definitions.
The new `PyList_GetItemRef` is similar to `PyList_GetItem`, but returns
a strong reference instead of a borrowed reference. Additionally, if the
passed "list" object is not a list, the function sets a `TypeError`
instead of calling `PyErr_BadInternalCall()`.
---------
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Improve misleading TCP server docs and example.
socket.recv(), as documented by the Python reference documentation,
returns at most `bufsize` bytes, and the underlying TCP protocol means
there is no guaranteed correspondence between what is sent by the client
and what is received by the server.
This conflation could mislead readers into thinking that TCP is
datagram-based or has similar semantics, which will likely appear to
work for simple cases, but introduce difficult to reproduce bugs.
The text clearly seems to be referencing `TestFuncAcceptsSequencesMixin`,
for which no target is available. Name the class properly and suppress
the dangling reference.
Return files and directories from `pathlib.Path.glob()` if the pattern ends
with `**`. This is more compatible with `PurePath.full_match()` and with
other glob implementations such as bash and `glob.glob()`. Users can add a
trailing slash to match only directories.
In my previous patch I added a `FutureWarning` with the intention of fixing
this in Python 3.15. Upon further reflection I think this was an
unnecessarily cautious remedy to a clear bug.
edson duarte
Alex Waygood
Nikita Sobolev
Animesh Kumar
Shantanu
Jelle Zijlstra
tahia
Nice Zombies
Kerim Kabirov
Clément Robert
Serhiy Storchaka
Russell Keith-Magee
Quazi Irfan
lyc8503
Irit Katriel
Savannah Ostrowski
Victor Stinner
Mariusz Felisiak
Rafael Fontenelle
Shreyan Avigyan
Miro Hrončok
Steve Dower
Manuel Kaufmann
Hugo van Kemenade
Douglas Thor
Inada Naoki
Erlend E. Aasland
Raymond Hettinger
Galen Rice
Gouvernathor
Barney Gale
Michiel W. Beijen
Tomas R
Bruce Merry
Mikhail B
Laurie O
Guido van Rossum
Anita Hammer
Petr Viktorin
Sam Gross
Vinay Sajip
Zackery Spytz
Gregory P. Smith
Moshe Kaplan
Matthew Davis
Justin Turner Arthur
Adorilson Bezerra
Pedro Lacerda
Christopher Chianelli
Malcolm Smith
Joachim Wuttke
Tim Hatch
Terry Jan Reedy
LilKS
Jakub Stasiak
Ned Batchelder
Carol Willing
Mark Shannon
Sunghyun Kim
Bogdan Romanyuk
Pierre Ossman (ThinLinc team)
jnchen
Tian Gao
Sergey B Kirpichev
Nir Friedman
Tushar Sadhwani
Brett Simmers
Declan
Kerim Kabirov
Nicolas A. Oyarzabal
infohash
Skip Montanaro
William Woodruff
Jason Zhang
Sebastian Pipping
Prince Roshan
cui fliter
Tobias Rautenkranz
pan324
Iman Tabrizian
HarryLHW
AN Long
Carl Bordum Hansen
Ethan Furman
Jan Max Meyer
Tahoma Software
Phil Elson
Antti Haapala
bssyousefi
Matan Perelman
mauricelambert
Oh seungmin
partev
jmcb
Stanley
Pablo Galindo Salgado
NewUserHa
Victor Westerhuis
Eugene Toder
talcs
Naglis Jonaitis
Brian Schubert
Daniel Haag
0xflotus
Masayuki Moriyama
Simon A. Eugster
Daler
Sebastian Rittau
Thomas Grainger
Matthew Hughes
Furkan Onder
Rami
Stevoisiak
Jamie Phan
monkeyman192
Martijn Pieters
David Hewitt
Stanislav Lyu
Christophe Papazian
qqwqqw689
Brandt Bucher
Mike Zimin
Brett Cannon
Edgar Ramírez Mondragón
Finite State Machine
da-woods
Dai Wentao
Kristján Valur Jónsson
John Belmonte
patenaud
Steven Ward
Justin Williams
Michal Kaptur
Oleg Iarygin
Nicholas Hollander
Christophe Nanteuil
technillogue
srinivasan
Pradyot Ranjan
Aidan Holm
Bradley Reynolds