Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Note: This change is not effective on Microsoft Windows.
Cookies can store sensitive information and should therefore be protected
against unauthorized third parties. This is also described in issue #79096.
The filesystem permissions are currently set to 644, everyone can read the
file. This commit changes the permissions to 600, only the creater of the file
can read and modify it. This improves security, because it reduces the attack
surface. Now the attacker needs control of the user that created the cookie or
a ways to circumvent the filesystems permissions.
This change is backwards incompatible. Systems that rely on world-readable
cookies will breake. However, one could argue that those are misconfigured in
the first place.
Character ranges with upper bound less that lower bound (e.g. [c-a])
are now interpreted as empty ranges, for compatibility with other glob
pattern implementations. Previously it was re.error.
Add C++ overloads for _Py_CAST_impl() to handle 0/NULL. This will allow
C++ extensions that pass 0 or NULL to macros using _Py_CAST() to
continue to compile. Without this, you get an error like:
invalid ‘static_cast’ from type ‘int’ to type ‘_object*’
The modern way to use a NULL value in C++ is to use nullptr. However,
we want to not break extensions that do things the old way.
Co-authored-by: serge-sans-paille
When a `_PathParents` object has a drive or a root, the length of the
object is *one less* than than the length of `self._parts`, which resulted
in an off-by-one error when `path.parents[-n]` was fed through to
`self._parts[:-n - 1]`. In particular, `path.parents[-1]` was a malformed
path object with spooky properties.
This is addressed by adding `len(self)` to negative indices.
* bpo-42272: improve message/module warning filter docs
"The Warnings Filter" section of the warnings module documentation
describes the message and module filters as "a string containing a
regular expression". While that is true when they are arguments to the
filterwarnings function, it is not true when they appear in -W or
$PYTHONWARNINGS where they are matched literally (after stripping any
starting/ending whitespace). Update the documentation to note when they
are matched literally. Also clarify that module matches the
"fully-qualified module name", rather than "module name" which is
ambiguous.
skip news (since this is a doc fix)
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
* bpo-42272: remove bad submodule warning filter doc
The `error:::mymodule[.*]` example in the "Describing Warning Filters"
section of the warnings module documentation does not behave as the
comment describes. Since the module portion of the filter string is
interpreted literally, it would match a module with a fully-qualified
name that is literally `mymodule[.*]`.
Unfortunately, there is not a way to match '"module" and any subpackages
of "mymodule"' as documented, since the module part of a filter string
is matched literally. Instead, update the filter and comment to match
only "mymodule".
skip news (since this is a doc fix)
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
* bpo-42272: add warning filter doc changes to NEWS
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
This was added for bpo-40514 (gh-84694) to test out a per-interpreter GIL. However, it has since proven unnecessary to keep the experiment in the repo. (It can be done as a branch in a fork like normal.) So here we are removing:
* the configure option
* the macro
* the code enabled by the macro
Added a new stable API function ``PyType_FromMetaclass``, which mirrors
the behavior of ``PyType_FromModuleAndSpec`` except that it takes an
additional metaclass argument. This is, e.g., useful for language
binding tools that need to store additional information in the type
object.
Python now always use the ``%zu`` and ``%zd`` printf formats to
format a size_t or Py_ssize_t number. Building Python 3.12 requires a
C11 compiler, so these printf formats are now always supported.
* PyObject_Print() and _PyObject_Dump() now use the printf %zd format
to display an object reference count.
* Update PY_FORMAT_SIZE_T comment.
* Remove outdated notes about the %zd format in PyBytes_FromFormat()
and PyUnicode_FromFormat() documentations.
* configure no longer checks for the %zd format and no longer defines
PY_FORMAT_SIZE_T macro in pyconfig.h.
* pymacconfig.h no longer undefines PY_FORMAT_SIZE_T: macOS 10.4 is
no longer supported. Python 3.12 now requires macOS 10.6 (Snow
Leopard) or newer.
Fix __lltrace__ debug feature if the stdout encoding is not UTF-8.
If the stdout encoding is not UTF-8, the first call to
lltrace_resume_frame() indirectly sets lltrace to 0 when calling
unicode_check_encoding_errors() which calls
encodings.search_function().
Generally comparable perf for the "good" case where memchr doesn't
return any collisions (false matches on lower byte) but clearly faster
with collisions.
Also while there, clarify a few things about why we reduce the hash to 32 bits.
Co-authored-by: Eli Libman <eli@hyro.ai>
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
At compile time, '+z' is already properly used with HP aCC, and shared
libraries are correctly linked with '+b'. The '-fPIC' switch can safely be
dropped.
The following sqlite3 features were deprecated in 3.10, scheduled for
removal in 3.12:
- sqlite3.OptimizedUnicode (gh-23163)
- sqlite3.enable_shared_cache (gh-24008)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Signed-off-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
If Condition.notify() was interrupted just after it released the waiter lock,
but before removing it from the queue, the following calls of notify() failed
with RuntimeError: cannot release un-acquired lock.
Avoid mixing declarations and code in the C API to fix the compiler
warning: "ISO C90 forbids mixed declarations and code"
[-Werror=declaration-after-statement].
Remove the PyUnicode_InternImmortal() function and the
SSTATE_INTERNED_IMMORTAL macro.
The PyUnicode_InternImmortal() function is still exported in the
stable ABI. The function is removed from the API.
PyASCIIObject.state.interned size is now a single bit, rather than 2
bits.
Keep SSTATE_NOT_INTERNED and SSTATE_INTERNED_MORTAL macros for
backward compatibility, but no longer use them internally since the
interned member is now a single bit and so can only have two values
(interned or not interned).
Update stats of _PyUnicode_ClearInterned().
The `utc_to_seconds` call can fail, here's a minimal reproducer on
Linux:
TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)"
The old behavior still raised an error in a similar way, but only
because subsequent calculations happened to fail as well. Better to fail
fast.
This also refactors the tests to split out the `fromtimestamp` and
`utcfromtimestamp` tests, and to get us closer to the actual desired
limits of the functions. As part of this, we also changed the way we
detect platforms where the same limits don't necessarily apply (e.g.
Windows).
As part of refactoring the tests to hit this condition explicitly (even
though the user-facing behvior doesn't change in any way we plan to
guarantee), I noticed that there was a difference in the places that
`datetime.utcfromtimestamp` fails in the C and pure Python versions, which
was fixed by skipping the "probe for fold" logic for UTC specifically —
since UTC doesn't have any folds or gaps, we were never going to find a
fold value anyway. This should prevent some failures in the pure python
`utcfromtimestamp` method on timestamps close to 0001-01-01.
There are two separate news entries for this because one is a
potentially user-facing change, the other is an internal code
correctness change that, if anything, changes some error messages. The
two happen to be coupled because of the test refactoring, but they are
probably best thought of as independent changes.
Fixes GH-91581
Remove the token.h header file. There was never any public tokenizer
C API. The token.h header file was only designed to be used by Python
internals.
Move Include/token.h to Include/internal/pycore_token.h. Including
this header file now requires that the Py_BUILD_CORE macro is
defined. It no longer checks for the Py_LIMITED_API macro.
Rename functions:
* PyToken_OneChar() => _PyToken_OneChar()
* PyToken_TwoChars() => _PyToken_TwoChars()
* PyToken_ThreeChars() => _PyToken_ThreeChars()
ElementTree method write() and function tostring() now use the text file's
encoding ("UTF-8" if not available) instead of locale encoding in XML
declaration when encoding="unicode" is specified.
Lines beginning with ``?`` try to help understanding the given diff.
The output can be hard to understand when it contains whitespace characters, such as spaces, tabs or line breaks.
While previously only tabs were mentioned, now all are listed.
Automerge-Triggered-By: GH:rhettinger
We could try to remedy this by taking a slice, but we then run into an issue where the empty string will match altsep on POSIX. That rabbit hole could keep getting deeper.
A proper fix for the original issue involves making pathlib's path normalisation more configurable - in this case we want to retain trailing slashes, but in other we might want to preserve `./` prefixes, or elide `../` segments when we're sure we won't encounter symlinks.
This reverts commit ea2f5bcda1.
* Some handlers were wrongly described as text-encoding only, but actually they can also be used in text-decoding.
* Add more description to each handler.
* Add two REPL examples.
* Add indexes for Error Handler's name.
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Do not spawn ProcessPool workers on demand when they spawn via fork.
This avoids potential deadlocks in the child processes due to forking from
a multithreaded process.
Only sequence of ASCII digits is now accepted as a numerical reference.
The group name in bytes patterns and replacement strings can now only
contain ASCII letters and digits and underscore.
Add methods enterContext() and enterClassContext() in TestCase.
Add method enterAsyncContext() in IsolatedAsyncioTestCase.
Add function enterModuleContext().
Add a closure keyword-only parameter to exec(). It can only be specified when exec-ing a code object that uses free variables. When specified, it must be a tuple, with exactly the number of cell variables referenced by the code object. closure has a default value of None, and it must be None if the code object doesn't refer to any free variables.
- add member() and nonmember() functions
- add deprecation warning for internal classes in enums not
becoming members in 3.13
Co-authored-by: edwardcwang
Prevent `max_tasks_per_child` use with a "fork" mp_context to avoid deadlocks.
Also defaults to "spawn" when no mp_context is supplied for safe convenience.
38f331d introduced a delayed initialization routine to set up
ctypes formattable (`_ctypes_init_fielddesc`), but inadvertently
removed setting the `initialization` flag to 1 to avoid initting
each time.
Add the -P command line option and the PYTHONSAFEPATH environment
variable to not prepend a potentially unsafe path to sys.path.
* Add sys.flags.safe_path flag.
* Add PyConfig.safe_path member.
* Programs/_bootstrap_python.c uses config.safe_path=0.
* Update subprocess._optim_args_from_interpreter_flags() to handle
the -P command line option.
* Modules/getpath.py sets safe_path to 1 if a "._pth" file is
present.
One more thing that can help prevent people from using `preexec_fn`.
Also adds conditional skips to two tests exposing ASAN flakiness on the Ubuntu 20.04 Address Sanitizer Github CI system. When that build is run on more modern systems the "problem" does not show up. It seems ASAN implementation related.
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
``pymain_run_python()`` now imports ``readline`` and ``rlcompleter``
before sys.path is extended to include the current working directory of
an interactive interpreter. Non-interactive interpreters are not
affected.
Also move imports of ``re`` and ``keyword`` module to top level so they
are materialized early, too. The ``keyword`` module is trivial and the
``re`` is already imported via ``inspect`` -> ``linecache``.
#92301: subprocess: Prefer `close_range()` to procfs-based fd closing.
`close_range()` is much faster for large number of file descriptors, e.g.
4 times faster for 1000 descriptors in a Linux 5.16-based environment.
We prefer close_range() only if it's known to be async-signal-safe.
* Map SQLITE_MISUSE to sqlite3.InterfaceError
SQLITE_MISUSE implies misuse of the SQLite C API, which, if it happens,
is _not_ a user error; it is an sqlite3 extension module error.
* Raise better errors when binding parameters fail.
Instead of always raising InterfaceError, guessing what went wrong,
raise accurate exceptions with more accurate error messages.
Fix a crash in subinterpreters related to the garbage collector. When
a subinterpreter is deleted, untrack all objects tracked by its GC.
To prevent a crash in deallocator functions expecting objects to be
tracked by the GC, leak a strong reference to these objects on
purpose, so they are never deleted and their deallocator functions
are not called.
Use FLAG_REF always for interned strings.
Refcounts of interned string is very unstable.
When compiling same source, refcounts of interned string in the output may be 1 or >1.
It makes FLAG_REF usage unstable.
To help reproducible build, use FLAG_REF for interned string even if refcnt(obj)==1.
Help for other actions omit the default value if default is SUPPRESS or
already contains the special format string '%(default)'. Add those
special cases to BooleanOptionalAction's help formatting too.
Fixes https://bugs.python.org/issue44587 so that default=SUPPRESS is not
emitted.
Fixes https://bugs.python.org/issue38956 as this code will detect
whether '%(default)s' has already been specified in the help string.
Signed-off-by: Micky Yun Chan (michiboo): <chanmickyyun@gmail.com>
Co-authored-by: Micky Yun Chan <michan@redhat.com>
This makes macOS gdbm provided by Homebrew not segfault through correct
selection of the linked library (-lgdbm_compat) *AND* the correct ndbm-style
header (gdbm-ndbm.h instead of the invalid ndbm.h).
Since the underlying file-like objects (either `io.BytesIO`,
or a true file object) all implement the `io.IOBase`
interface, the `SpooledTemporaryFile` should as well.
Additionally, since the underlying file object will either be an
instance of an `io.BufferedIOBase` (for binary mode) or an
`io.TextIOBase` (for text mode), methods for these classes were also
implemented.
In every case, the required methods and properties are simply delegated
to the underlying file object.
Co-authored-by: Gary Fernie <Gary.Fernie@skyscanner.net>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
`IPv*Network` and `IPv*Interface` constructors accept a 2-tuple of
(address description, netmask) as the address parameter.
When the tuple-based address is used errors are not propagated
correctly through the `ipaddress.ip_*` helper because of the %-formatting now expecting several arguments:
In [7]: ipaddress.ip_network(("192.168.100.0", "fooo"))
...
TypeError: not all arguments converted during string formatting
Compared to:
In [8]: ipaddress.IPv4Network(("192.168.100.0", "foo"))
...
NetmaskValueError: 'foo' is not a valid netmask
Use an f-string to make sure the error is always properly formatted.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
This mirrors logic in typing.get_args. The trickiness comes from how we
flatten args in collections.abc.Callable, see
https://bugs.python.org/issue42195
If the error handler returns position less or equal than the starting
position of non-encodable characters, most of built-in encoders didn't
properly re-size the output buffer. This led to out-of-bounds writes,
and segfaults.
Raise an ArgumentError when the same subparser name is added twice to an
ArgumentParser. This is consistent with the (default) behavior when the
same option string is added twice to an ArgumentParser.
(Support for `conflict_handler="resolve"` could be considered as a
followup feature, although real use cases seem even rarer than
"resolve"ing option-strings.)
Automerge-Triggered-By: GH:rhettinger
* Check the types of PRECALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS
* fix PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS as well
* fix PRECALL_NO_KW_METHOD_DESCRIPTOR_O
* fix PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST
It was raised if the charset itself contains characters not encodable
in UTF-8 (in particular \udcxx characters representing non-decodable
bytes in the source).
Only sequence of ASCII digits will be accepted as a numerical reference.
The group name in bytes patterns and replacement strings could only
contain ASCII letters and digits and underscore.
Also updated `make -C htmlview` so it used a full path with `file://`, because the original didn't open the page (macOS).
For example:
```sh
cd Doc
# Doesn't open anything:
python3 -c "import webbrowser; webbrowser.open('build/html/index.html')"
# Opens the docs page e.g. file:///Users/hugo/github/cpython/Doc/build/html/index.html :
python3 -c "import os, webbrowser; webbrowser.open('file://' + os.path.realpath('build/html/index.html'))"
```
https://bugs.python.org/issue36329
# Adding 'required' to names in Lib.argparse.Action
gh-91832:
Added 'required' to the list `names` in `Lib.argparse.Action`.
Changed constant strings that test the Action object.
Automerge-Triggered-By: GH:merwok
Reduce the complexity from O((M+N)^2) to O(M*N), where M and N are the length
of __args__ for both operands (1 for operand which is not a UnionType).
As a consequence, the complexity of parameter substitution in UnionType has
been reduced from O(N^3) to O(N^2).
Co-authored-by: Yurii Karabas <1998uriyyo@gmail.com>
The warning emitted by the Python parser for a numeric literal
immediately followed by keyword has been changed from deprecation
warning to syntax warning.
Fix C++ compiler warnings about "old-style cast"
(g++ -Wold-style-cast) in the Python C API. Use C++
reinterpret_cast<> and static_cast<> casts when the Python C API is
used in C++.
Example of fixed warning:
Include/object.h:107:43: error: use of old-style cast to
‘PyObject*’ {aka ‘struct _object*’} [-Werror=old-style-cast]
#define _PyObject_CAST(op) ((PyObject*)(op))
Add _Py_reinterpret_cast() and _Py_static_cast() macros.
Copied from typing-extensions (python/typing#1054, python/typing#1120).
Documentation is intentionally omitted, so we can focus on getting the
runtime part in before the feature freeze.
Just in case there is ever an issue with _posixsubprocess's use of
vfork() due to the complexity of using it properly and potential
directions that Linux platforms where it defaults to on could take, this
adds a failsafe so that users can disable its use entirely by setting
a global flag.
No known reason to disable it exists. But it'd be a shame to encounter
one and not be able to use CPython without patching and rebuilding it.
See the linked issue for some discussion on reasoning.
Also documents the existing way to disable posix_spawn.
Fix signal.NSIG value on FreeBSD to accept signal numbers greater
than 32, like signal.SIGRTMIN and signal.SIGRTMAX.
* Add Py_NSIG constant.
* Add pycore_signal.h internal header file.
* _Py_Sigset_Converter() now includes the range of valid signals in
the error message.
* Add a new :gh:`...` role for GitHub issues.
* Fix a GitHub id to use the :gh: role.
* Add Misc/NEWS entry.
* Refactoring and rephrasing.
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
The test relies on precision being set to 9, but some ways of
invoking this test leave it set to 28 instead. I don't know
exactly how it happens, but setting the precision directly should
make the behavior consistent.
Also inline necessary functionality from `sndhdr` into `email.mime.audio` for `MIMEAudio`.
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
In expression (?(group)...) an appropriate re.error is now
raised if the group number refers to not defined group.
Previously it raised RuntimeError: invalid SRE code.
Macros Py_DECREF, Py_XDECREF, Py_IS_TYPE, _Py_atomic_load_32bit_impl
and _Py_DECREF_SPECIALIZED are redefined as macros
that completely replace the inline functions of the same name.
These three came out in the top four of functions that (in MSVC)
somehow weren't inlined.
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
Deprecate functions:
* PyUnicode_AS_DATA()
* PyUnicode_AS_UNICODE()
* PyUnicode_GET_DATA_SIZE()
* PyUnicode_GET_SIZE()
Previously, these functions were macros and so it wasn't possible to
decorate them with Py_DEPRECATED().
Py_REFCNT(), Py_TYPE(), Py_SIZE() and Py_IS_TYPE() functions argument
type is now "PyObject*", rather than "const PyObject*".
* Replace also "const PyObject*" with "PyObject*" in functions:
* _Py_strhex_impl()
* _Py_strhex_with_sep()
* _Py_strhex_bytes_with_sep()
* Remove _PyObject_CAST_CONST() and _PyVarObject_CAST_CONST() macros.
* Py_IS_TYPE() can now use Py_TYPE() in its implementation.
* Stores all location info in linetable to conform to PEP 626.
* Remove column table from code objects.
* Remove end-line table from code objects.
* Document new location table format
Kalyan
Adrian Garcia Badaracco
Mark Shannon
Andreas Grommek
Hugo van Kemenade
Vinay Sajip
Pascal Wittmann
Davide Rizzo
AN Long
Illia Volochii
Christian Heimes
Serhiy Storchaka
Neil Schemenauer
jackh-ncl
Irit Katriel
Barney Gale
Ken Jin
Petr Viktorin
Dong-hee Na
Eric V. Smith
Éric
Pablo Galindo Salgado
Dennis Sweeney
Kumar Aditya
Kevin Locke
Sergei Izmailov
georgically
oda-gitso
Eric Snow
Yury Selivanov
hfinucane
Wenzel Jakob
Victor Stinner
Éric
Miro Hrončok
Jordan Borean
goldsteinn
Sam Ezeh
Mori Bellamy
Steve Dower
David Machaj
Nikita Sobolev
Michael Osipov
Zackery Spytz
David Foster
Ayappan Perumal
Inada Naoki
David CARLIER
Oleg Iarygin
Erlend Egeberg Aasland
Batuhan Taskaya
Paul Ganssle
ravi140222
Jelle Zijlstra
Raymond Hettinger
狂男风
Bar Harel
Jürgen Gmach
Gregory P. Smith
Cheryl Sabella
Mariusz Felisiak
Ma Lin
Marc Mueller
Matthew Rahtz
Brandt Bucher
larryhastings
Alex Waygood
Ethan Furman
Alexey Izbyshev
Michael Droettboom
cibofo
Soumendra Ganguly
Vlad Hoi
Géry Ogam
Michael J. Sullivan
Joshua Herman
Sam Denton
Kabir Kwatra
kixorz
Zac Hatfield-Dodds
Humbled Drugman
Abhigyan Bose
Toshio Kuratomi
Łukasz Langa
Samodya Abey
Carey Metcalfe
Thomas Cellerier
Itai Steinherz
Shantanu
Ethan Smith
kbeldan
themylogin
Mark Dickinson
Antony Lee
Brett Cannon
Eisuke Kawashima
Yurii Karabas
Alexander Shadchin
Ezio Melotti
Guido van Rossum
Erlend Egeberg Aasland
Iman Kermani
Jakub Kulík
Nick