- check for ``dup()`` libc function
- handle missing ``F_DUPFD`` in ``dup2()`` replacement function
- add workaround for WASI libc bug in MSG_TRUNC
- ESHUTDOWN is missing, use EPIPE instead
- POLLPRI is missing, define as 0 (no-op)
* Add _Py_memory_repeat function to pycore_list
* Add _Py_RefcntAdd function to pycore_object
* Use the new functions in tuplerepeat, list_repeat, and list_inplace_repeat
* gh-95218: Move tests for importlib.resources into test_importlib.resources.
* Also update makefile
* Include test_importlib/resources in code ownership rule.
This PR partially reverts gh-24421 (PR) and fixes the remaining concerns
given in gh-93044 (issue):
- keyword arguments are passed as positional arguments to factory()
- if an argument is not passed to sqlite3.connect(), its default value
is passed to factory()
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-94949: Disallow parsing parenthesised ctx manager with old feature_version
* 📜🤖 Added by blurb_it.
* Allow it with feature_version=(3, 9) as well
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Support for bytes broke sometime between Python 3.2 and 3.6 and has been broken ever since. Trying to bring back supports is surprisingly difficult in the face of -b and checking for keys in sys.path_importer_cache. Since the support was broken for so long, trying to overcome the difficulty of bringing back the support has been deemed not worth it.
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Co-authored-by: Brett Cannon <brett@python.org>
When binding a unix socket to an empty address on Linux, the socket is
automatically bound to an available address in the abstract namespace.
>>> s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
>>> s.bind("")
>>> s.getsockname()
b'\x0075499'
Since python 3.9, the socket is bound to the one address:
>>> s.getsockname()
b'\x00'
And trying to bind multiple sockets will fail with:
Traceback (most recent call last):
File "/home/nsoffer/src/cpython/Lib/test/test_socket.py", line 5553, in testAutobind
s2.bind("")
OSError: [Errno 98] Address already in use
Added 2 tests:
- Auto binding empty address on Linux
- Failing to bind an empty address on other platforms
Fixes f6b3a07b7d (bpo-44493: Add missing terminated NUL in sockaddr_un's length (GH-26866)
* gh-93883: elide traceback indicators when possible
Elide traceback column indicators when the entire line of the
frame is implicated. This reduces traceback length and draws
even more attention to the remaining (very relevant) indicators.
Example:
```
Traceback (most recent call last):
File "query.py", line 99, in <module>
bar()
File "query.py", line 66, in bar
foo()
File "query.py", line 37, in foo
magic_arithmetic('foo')
File "query.py", line 18, in magic_arithmetic
return add_counts(x) / 25
^^^^^^^^^^^^^
File "query.py", line 24, in add_counts
return 25 + query_user(user1) + query_user(user2)
^^^^^^^^^^^^^^^^^
File "query.py", line 32, in query_user
return 1 + query_count(db, response['a']['b']['c']['user'], retry=True)
~~~~~~~~~~~~~~~~~~^^^^^
TypeError: 'NoneType' object is not subscriptable
```
Rather than going out of our way to provide indicator coverage
in every traceback test suite, the indicator test suite should
be responible for sufficient coverage (e.g. by adding a basic
exception group test to ensure that margin strings are covered).
Deprecate typing.Hashable/Sized. Use the collections.abc counterparts directly instead.
To be consistent with PEP 585, deprecated aliases will not raise any DeprecationWarning.
Remove the ssl.wrap_socket() function, deprecated in Python 3.7:
instead, create a ssl.SSLContext object and call its
sl.SSLContext.wrap_socket() method. Any package that still uses
ssl.wrap_socket() is broken and insecure. The function neither sends
a SNI TLS extension nor validates server hostname. Code is subject to
CWE-295 : Improper Certificate Validation.
This removes the performance regression in 3.11, **at the expense of not fixing
the "bug" that allows accessing values from values** (e.g. `Color.RED.BLUE`).
Using the benchmark @markshannon [presented](https://github.com/python/cpython/issues/93910#issuecomment-1165503032), the results are:
| Version | Enum | Fast enum | Normal class |
| --- | --- | --- | --- |
| 3.10 | 2.04 | 0.59 | 0.56 |
| 3.11 | 2.78 | 0.31 | 0.15 |
| This PR | 1.30 | 0.32 | 0.16 |
I share this mostly as information about the source of the regression, as this may be useful. It may be that the lower-risk approach for the beta is just to revert to a previously-known working state.
Inlining of code that corresponds to source code lines, can make it hard to distinguish later between code which is only reachable from except handlers, and that which is reachable in normal control flow. This caused problems with the debugger's jump feature.
This PR turns off the inlining optimisation for code which has line numbers. We still inline things like the implicit "return None".
zipimport: Remove find_loader() and find_module() methods, deprecated
in Python 3.10: use the find_spec() method instead. See PEP 451 for
the rationale.
Add script ``Tools/scripts/check_modules.py`` to check and validate builtin
and shared extension modules. The script also handles ``Modules/Setup`` and
will eventually replace ``setup.py``.
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@protonmail.com>
xml.etree: Remove the ElementTree.Element.copy() method of the pure
Python implementation, deprecated in Python 3.10, use the copy.copy()
function instead. The C implementation of xml.etree has no copy()
method, only a __copy__() method.
Adds `ctypes.c_time_t` to represent the C `time_t` type accurately as its size varies.
Primarily-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google]
Once the task group is shutting down, it should not be possible to create a new task.
Here "shutting down" means `self._aborting` is set, indicating that at least one task
has failed and we have cancelled all others.
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
The inspect version was not working with unittest.mock.AsyncMock.
The fix introduces special-casing of AsyncMock in
`inspect.iscoroutinefunction` equivalent to the one
performed in `asyncio.iscoroutinefunction`.
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Remove the pure Python implementation of hashlib.pbkdf2_hmac(),
deprecated in Python 3.10. Python 3.10 and newer requires OpenSSL
1.1.1 or newer (PEP 644), this OpenSSL version provides a C
implementation of pbkdf2_hmac() which is faster.
``os.geteuid() == 0`` is not a reliable check whether the current user
has the capability to bypass permission checks. Tests now probe for DAC
override.
Remove the locale.format() function, deprecated in Python
3.7: use locale.format_string() instead.
Remove TestFormatPatternArg test case: it is irrelevant for
locale.format_string() which accepts complex formats.
Make _struct.Struct a GC type
This fixes a memory leak in the _struct module, where as soon
as a Struct object is stored in the cache, there's a cycle from
the _struct module to the cache to Struct objects to the Struct
type back to the module. If _struct.Struct is not gc-tracked, that
cycle is never collected.
This PR makes _struct.Struct GC-tracked, and adds a regression test.
`tarfile` already accepts a compressionlevel argument for creating
files. This patch adds the same for stream-based tarfile usage.
The default is 9, the value that was previously hard-coded.
The existing test covering this case passed only incidentally. We
explicitly disallow doing this and add a proper error message.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gzip: Remove the filename attribute of gzip.GzipFile,
deprecated since Python 2.6, use the name attribute instead. In write
mode, the filename attribute added '.gz' file extension if it was not
present.
Remove io.OpenWrapper and _pyio.OpenWrapper, deprecated in Python
3.10: just use :func:`open` instead. The open() (io.open()) function
is a built-in function. Since Python 3.10, _pyio.open() is also a
static method.
* Add an index_pages default list to SimpleHTTPRequestHandler and an
optional constructor parameter that allows the default indexes pages
list to be overridden. This makes it easy to set a new index page name
without having to override send_head.
When used with plain Enum, auto() returns the last numeric value assigned, skipping any incompatible member values (such as strings); starting in 3.13 the default auto() for plain Enums will require all the values to be of compatible types, and will return a new value that is 1 higher than any existing value.
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
* Move Lib/tkinter/test/test_tkinter/ to Lib/test/test_tkinter/.
* Move Lib/tkinter/test/test_ttk/ to Lib/test/test_ttk/.
* Add Lib/test/test_ttk/__init__.py based on test_ttk_guionly.py.
* Add Lib/test/test_tkinter/__init__.py
* Remove old Lib/test/test_tk.py.
* Remove old Lib/test/test_ttk_guionly.py.
* Add __main__ sub-modules.
* Update imports and update references to rename files.
It is no longer changed when create a zip or tar archive.
It is still changed for custom archivers registered with shutil.register_archive_format()
if root_dir is not None.
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Fix an open redirection vulnerability in the `http.server` module when
an URI path starts with `//` that could produce a 301 Location header
with a misleading target. Vulnerability discovered, and logic fix
proposed, by Hamza Avvan (@hamzaavvan).
Test and comments authored by Gregory P. Smith [Google].
Explicitly note that transactions are only closed if there is an open
transation at `__exit__`, and that transactions are not implicitly
opened during `__enter__`.
Co-authored-by: CAM Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
Automerge-Triggered-By: GH:erlend-aasland
Move the follow functions and type from frameobject.h to pyframe.h,
so the standard <Python.h> provide frame getter functions:
* PyFrame_Check()
* PyFrame_GetBack()
* PyFrame_GetBuiltins()
* PyFrame_GetGenerator()
* PyFrame_GetGlobals()
* PyFrame_GetLasti()
* PyFrame_GetLocals()
* PyFrame_Type
Remove #include "frameobject.h" from many C files. It's no longer
needed.
- gh-93957: Provide nicer error reporting from subprocesses in test_venv.EnsurePipTest.test_with_pip.
- Update changelog
This change does three things:
1. Extract a function for trapping output in subprocesses.
2. Emit both stdout and stderr when encountering an error.
3. Apply the change to `ensurepip._uninstall` check.
Deprecate global configuration variable like
Py_IgnoreEnvironmentFlag: the Py_InitializeFromConfig() API should be
instead.
Fix declaration of Py_GETENV(): use PyAPI_FUNC(), not PyAPI_DATA().
test_asyncio, test_logging, test_socket and test_socketserver now
create AF_UNIX domains in the current directory to no longer fail
with OSError("AF_UNIX path too long") if the temporary directory (the
TMPDIR environment variable) is too long.
Modify the following tests to use create_unix_domain_name():
* test_asyncio
* test_logging
* test_socket
* test_socketserver
test_asyncio.utils: remove unused time import.
Revert "bpo-23689: re module, fix memory leak when a match is terminated by a signal or memory allocation failure (GH-32283)"
This reverts commit 6e3eee5c11.
Manual fixups to increase the MAGIC number and to handle conflicts with
a couple of changes that landed after that.
Thanks for reviews by Ma Lin and Serhiy Storchaka.
regrtest now also implements checking for leaked temporary files and
directories when using -jN for N >= 2. Use tempfile.mkdtemp() to
create the temporary directory. Skip this check on WASI.
SharedMemory.unlink() uses the unregister() function from resource_tracker. Previously it was imported in the method, but this can fail if the method is called during interpreter shutdown, for example when unlink is part of a __del__() method.
Moving the import to the top of the file, means that the unregister() method is available during interpreter shutdown.
The register call in SharedMemory.__init__() can also use this imported resource_tracker.
GH-26658 introduced a regression in copy / pickle protocol for combined
`enum.Flag`s. `copy.copy(re.A | re.I)` would fail with
`AttributeError: ASCII|IGNORECASE`.
`enum.Flag` now has a `__reduce_ex__()` method that reduces flags by
combined value, not by combined name.
Align the docs for the following methods with the actual implementation:
- sqlite3.complete_statement()
- sqlite3.Connection.create_function()
- sqlite3.Connection.create_aggregate()
- sqlite3.Connection.set_progress_handler()
The fix involves using pysqlite_check_remaining_sql(), not only to check
for multiple statements, but now also to strip leading comments and
whitespace from SQL statements, so we can improve DML query detection.
pysqlite_check_remaining_sql() is renamed lstrip_sql(), to more
accurately reflect its function, and hardened to handle more SQL comment
corner cases.
When changing PyType_FromMetaclass recently (GH-93012, GH-93466, GH-28748)
I found a bunch of opportunities to improve the code. Here they are.
Fixes: #89546
Automerge-Triggered-By: GH:encukou
Suppress writing an XML declaration in open files in ElementTree.write()
with encoding='unicode' and xml_declaration=None.
If file patch is passed to ElementTree.write() with encoding='unicode',
always open a new file in UTF-8.
Classes ReferenceType, ProxyType and CallableProxyType have now correct
atrtributes __module__, __name__ and __qualname__.
It makes them (types, not instances) pickleable.
Fix the importlib.resources.as_file() context manager to remove the
temporary file if destroyed late during Python finalization: keep a
local reference to the os.remove() function. Patch by Victor Stinner.
Documentation for `pathlib` says:
> Spurious slashes and single dots are collapsed, but double dots ('..') are not, since this would change the meaning of a path in the face of symbolic links:
However, it omits that initial double slashes also aren't collapsed.
Later, in documentation of `PurePath.drive`, `PurePath.root`, and `PurePath.name` it mentions UNC but:
- this abbreviation says nothing to a person who is unaware about existence of UNC (Wikipedia doesn't help either by [giving a disambiguation page](https://en.wikipedia.org/wiki/UNC))
- it shows up only if a person needs to use a specific property or decides to fully learn what the module provides.
For context, see the BPO entry.
This checks the bases of of a type created using the FromSpec
API to inherit the bases metaclasses. The metaclass's alloc
function will be called as is done in `tp_new` for classes
created in Python.
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@protonmail.com>
copy.copy() and copy.deepcopy() now always raise a TypeError if
__reduce__() returns a tuple with length 6 instead of silently ignore
the 6th item or produce incorrect result.
All install targets use the "all" target as synchronization point to
prevent race conditions with PGO builds. PGO builds use recursive make,
which can lead to two parallel `./python setup.py build` processes that
step on each others toes.
"test" targets now correctly compile PGO build in a clean repo.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Note: This change is not effective on Microsoft Windows.
Cookies can store sensitive information and should therefore be protected
against unauthorized third parties. This is also described in issue #79096.
The filesystem permissions are currently set to 644, everyone can read the
file. This commit changes the permissions to 600, only the creater of the file
can read and modify it. This improves security, because it reduces the attack
surface. Now the attacker needs control of the user that created the cookie or
a ways to circumvent the filesystems permissions.
This change is backwards incompatible. Systems that rely on world-readable
cookies will breake. However, one could argue that those are misconfigured in
the first place.
Character ranges with upper bound less that lower bound (e.g. [c-a])
are now interpreted as empty ranges, for compatibility with other glob
pattern implementations. Previously it was re.error.
Add C++ overloads for _Py_CAST_impl() to handle 0/NULL. This will allow
C++ extensions that pass 0 or NULL to macros using _Py_CAST() to
continue to compile. Without this, you get an error like:
invalid ‘static_cast’ from type ‘int’ to type ‘_object*’
The modern way to use a NULL value in C++ is to use nullptr. However,
we want to not break extensions that do things the old way.
Co-authored-by: serge-sans-paille
When a `_PathParents` object has a drive or a root, the length of the
object is *one less* than than the length of `self._parts`, which resulted
in an off-by-one error when `path.parents[-n]` was fed through to
`self._parts[:-n - 1]`. In particular, `path.parents[-1]` was a malformed
path object with spooky properties.
This is addressed by adding `len(self)` to negative indices.
* bpo-42272: improve message/module warning filter docs
"The Warnings Filter" section of the warnings module documentation
describes the message and module filters as "a string containing a
regular expression". While that is true when they are arguments to the
filterwarnings function, it is not true when they appear in -W or
$PYTHONWARNINGS where they are matched literally (after stripping any
starting/ending whitespace). Update the documentation to note when they
are matched literally. Also clarify that module matches the
"fully-qualified module name", rather than "module name" which is
ambiguous.
skip news (since this is a doc fix)
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
* bpo-42272: remove bad submodule warning filter doc
The `error:::mymodule[.*]` example in the "Describing Warning Filters"
section of the warnings module documentation does not behave as the
comment describes. Since the module portion of the filter string is
interpreted literally, it would match a module with a fully-qualified
name that is literally `mymodule[.*]`.
Unfortunately, there is not a way to match '"module" and any subpackages
of "mymodule"' as documented, since the module part of a filter string
is matched literally. Instead, update the filter and comment to match
only "mymodule".
skip news (since this is a doc fix)
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
* bpo-42272: add warning filter doc changes to NEWS
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
Christian Heimes
Stéphane Bidoul
Pablo Galindo Salgado
Kumar Aditya
Jason R. Coombs
Aivars Kalvāns
Steve Dower
Irit Katriel
Shantanu
Pieter Eendebak
Mark Shannon
Oleg Iarygin
Ethan Furman
Serhiy Storchaka
Thomas Grainger
Inada Naoki
Erlend Egeberg Aasland
Stanislav Zmiev
Brandt Bucher
Ken Jin
Erlend Egeberg Aasland
Terry Jan Reedy
Paul Moore
Paul m. p. Peny
Nir Soffer
neonene
Petr Viktorin
Raymond Hettinger
Simon-Martin Schröder
John Belmonte
MonadChains
Victor Stinner
finefoot
Michael Droettboom
Erik De Bonte
Carter Dodd
Erik Montnemery
Łukasz Langa
Ned Batchelder
KotlinIsland
Thomas Perl
Guido van Rossum
Mehdi ABAAKOUK
Robin Plumey
Shixian Li
Thomas B. Brunner
wookie184
Mark Dickinson
Yaron de Leeuw
Gregory Beauregard
Myron Walker
Oscar R
Illia Volochii
chgnrdv
Carl Bordum Hansen
Gregory P. Smith
Ezio Melotti
Jelle Zijlstra
Dennis Sweeney
itssme
Ken Jin
Chris Fernald
samtygier
Ken Jin
jackh-ncl
Barney Gale
Leo Trol
Sebastian Berg
Bluenix
Kalyan
Adrian Garcia Badaracco
Andreas Grommek
Hugo van Kemenade
Vinay Sajip
Pascal Wittmann
Davide Rizzo
AN Long
Neil Schemenauer
Dong-hee Na
Eric V. Smith
Éric
Kevin Locke
Sergei Izmailov
georgically
oda-gitso