* Deprecate imghdr
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Update Doc/whatsnew/3.11.rst
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Inline `imghdr` into `email.mime.image`
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Barry Warsaw <barry@python.org>
I talked to @davidfstr and I offered to implement the runtime part of PEP 655
to make sure we can get it in before the feature freeze. We're going to defer
the documentation to a separate PR, because it can wait until after the feature
freeze.
The runtime implementation conveniently already exists in typing-extensions,
so I largely copied that.
Co-authored-by: David Foster <david@dafoster.net>
* Move the code for generating Modules/_sre/sre_constants.h from
Lib/re/_constants.py into a separate script
Tools/scripts/generate_sre_constants.py.
* Add target `regen-sre` in the makefile.
* Make target `regen-all` depending on `regen-sre`.
Copying and pickling instances of subclasses of builtin types
bytearray, set, frozenset, collections.OrderedDict, collections.deque,
weakref.WeakSet, and datetime.tzinfo now copies and pickles instance attributes
implemented as slots.
unittest.TestCase.assertEquals() alias is depracated. Fix the
warning:
Lib/test/test_capi.py:1100: DeprecationWarning: Please use assertEqual instead.
self.assertEquals(frame.f_locals, _testcapi.frame_getlocals(frame))
The side effect of this bug was that venv environments directly
used the main interpreter instead of the intermediate stub executable,
which can cause problems when a script uses system APIs that
require the use of an application bundle.
In rare cases, capturing group could get wrong result.
Regular expression engines in Perl and Java have similar bugs.
The new behavior now matches the behavior of more modern
RE engines: in the regex module and in PHP, Ruby and Node.js.
The goal here is to reduce potential confusion between
`assert_type(val, type)` and `assert isinstance(val, typ)`.
The former is meant to ask a type checker to confirm a fact, the latter
is meant to tell a type checker a fact. The behaviour of the latter more
closely resembles what I'd expect from the prior phrasing of
"assert [something] to the type checker".
After a long deliberation we ended up feeling that the message argument for Future.cancel(), added in 3.9, was a bad idea, so we're deprecating it in 3.11 and plan to remove it in 3.13.
- Add requires_fork and requires_subprocess to more tests
- Skip extension import tests if dlopen is not available
- Don't assume that _testcapi is a shared extension
- Skip a lot of socket tests that don't work on Emscripten
- Skip mmap tests, mmap emulation is incomplete
- venv does not work yet
- Cannot get libc from executable
The "entire" test suite is now passing on Emscripten with EMSDK from git head (91 suites are skipped).
* Add the metadata_encoding parameter in the zipfile.ZipFile constructor.
* Add the --metadata-encoding option in the zipfile CLI.
Co-authored-by: Stephen J. Turnbull <stephen@xemacs.org>
Replace the child process `typeperf.exe` with a daemon thread that reads the performance counters directly. This prevents the issues that arise from inherited handles in grandchild processes (see issue37531 for discussion).
We only use the load tracker when running tests in multiprocess mode. This prevents inadvertent interactions with tests expecting a single threaded environment. Displaying load is really only helpful for buildbots running in multiprocess mode anyway.
* Moves the bytecode to the end of the corresponding PyCodeObject, and quickens it in-place.
* Removes the almost-always-unused co_varnames, co_freevars, and co_cellvars member caches
* _PyOpcode_Deopt is a new mapping from all opcodes to their un-quickened forms.
* _PyOpcode_InlineCacheEntries is renamed to _PyOpcode_Caches
* _Py_IncrementCountAndMaybeQuicken is renamed to _PyCode_Warmup
* _Py_Quicken is renamed to _PyCode_Quicken
* _co_quickened is renamed to _co_code_adaptive (and is now a read-only memoryview).
* Do not emit unused nonzero opargs anymore in the compiler.
When compiled with `USE_ZLIB_CRC32` defined (`configure` sets this on POSIX systems), `binascii.crc32(...)` failed to compute the correct value when the input data was >= 4GiB. Because the zlib crc32 API is limited to a 32-bit length.
This lines it up with the `zlib.crc32(...)` implementation that doesn't have that flaw.
**Performance:** This also adopts the same GIL releasing for larger inputs logic that `zlib.crc32` has, and causes the Windows build to always use zlib's crc32 instead of our slow C code as zlib is a required build dependency on Windows.
The `_SharedFile` tracks its own virtual position into the file as
`self._pos` and updates it after reading or seeking. `tell()` should
return this position instead of calling into the underlying file object,
since if multiple `_SharedFile` instances are being used concurrently on
the same file, another one may have moved the real file position.
Additionally, calling into the underlying `tell` may expose thread
safety issues in the underlying file object because it was called
without taking the lock.
Define *posix_venv* and *nt_venv* sysconfig installation schemes
to be used for bootstrapping new virtual environments.
Add *venv* sysconfig installation scheme to get the appropriate one of the above.
The schemes are identical to the pre-existing
*posix_prefix* and *nt* install schemes.
The venv module now uses the *venv* scheme to create new virtual environments
instead of hardcoding the paths depending only on the platform. Downstream
Python distributors customizing the *posix_prefix* or *nt* install
scheme in a way that is not compatible with the install scheme used in
virtual environments are encouraged not to customize the *venv* schemes.
When Python itself runs in a virtual environment,
sysconfig.get_default_scheme and
sysconfig.get_preferred_scheme with `key="prefix"` returns
*venv*.
Previously it tested that that the actual output contains every non-whitespace
character from the expected output (ignoring order and repetitions).
Now it will test that the actual output contains the same lines as the expected
output, in the same order, ignoring indentation and empty lines.
* Improve exception compliance with PEP 249
* Raise InterfaceError instead of ProgrammingError for SQLITE_MISUSE.
If SQLITE_MISUSE is raised, it is a sqlite3 module bug. Users of the
sqlite3 module are not responsible for using the SQLite C API correctly.
* Don't overwrite BufferError with ValueError when conversion to BLOB fails.
* Raise ProgrammingError instead of Warning if user tries to execute() more
than one SQL statement.
* Raise ProgrammingError instead of ValueError if an SQL query contains null characters.
* Make sure `_pysqlite_set_result` raises an exception if it returns -1.
wasm32-emscripten does not support exceptfds and requires NULL. Python
now passes NULL instead of a fdset pointer when the input list is empty.
This works fine on all platforms and might even be a tiny bit faster.
Specifically, prepare for starring of tuples via a new genericalias iter type. GenericAlias also partially supports the iterator protocol after this change.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Add new functions to pack and unpack C double (serialize and
deserialize):
* PyFloat_Pack2(), PyFloat_Pack4(), PyFloat_Pack8()
* PyFloat_Unpack2(), PyFloat_Unpack4(), PyFloat_Unpack8()
Document these functions and add unit tests.
Rename private functions and move them from the internal C API
to the public C API:
* _PyFloat_Pack2() => PyFloat_Pack2()
* _PyFloat_Pack4() => PyFloat_Pack4()
* _PyFloat_Pack8() => PyFloat_Pack8()
* _PyFloat_Unpack2() => PyFloat_Unpack2()
* _PyFloat_Unpack4() => PyFloat_Unpack4()
* _PyFloat_Unpack8() => PyFloat_Unpack8()
Replace the "unsigned char*" type with "char*" which is more common
and easy to use.
Add methods __typing_subst__() in TypeVar and ParamSpec.
Simplify code by using more object-oriented approach, especially
the C code for types.GenericAlias and the Python code for
collections.abc.Callable.
GH-30297 removed a duplicate `from test import support` statement from `test_asyncio.test_sslproto`. However, in between that PR being filed and it being merged, GH-31275 removed the _other_ `from test import support` statement. This means that `support` is now undefined in `test_asyncio.test_sslproto`, causing the CI to fail on all platforms for all PRS.
Example:
async with asyncio.timeout(5):
await some_task()
Will interrupt the await and raise TimeoutError if some_task() takes longer than 5 seconds.
Co-authored-by: Guido van Rossum <guido@python.org>
GH-26091 added the _typevar_types and _paramspec_tvars instance
variables to _GenericAlias. However, they were not propagated
consistently. This commit addresses the most prominent deficiency
identified in bpo-46581 (namely their absence from
_GenericAlias.copy_with), but there could be others.
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
- fd inheritance can't be modified because Emscripten doesn't support subprocesses anyway.
- setpriority always fails
- geteuid no longer causes problems with latest emsdk
- umask is a stub
- geteuid / getuid always return 0, but process cannot chown to random uid.
- getgroups always fails.
- geteuid and getegid always return 0 (root), which confuse tarfile and
tests.
- hardlinks (link, linkat) always fails.
- non-encodable file names are not supported by NODERAWFS layer.
- mark more tests with dependency on subprocess and multiprocessing.
Mocking does not work if the module fails to import.
This adds a new standard library module, `tomllib`, for parsing TOML.
The implementation is based on Tomli (https://github.com/hukkin/tomli).
## Steps taken (converting `tomli` to `tomllib`)
- Move everything in `tomli:src/tomli` to `Lib/tomllib`. Exclude `py.typed`.
- Remove `__version__ = ...` line from `Lib/tomllib/__init__.py`
- Move everything in `tomli:tests` to `Lib/test/test_tomllib`. Exclude the following test data dirs recursively:
- `tomli:tests/data/invalid/_external/`
- `tomli:tests/data/valid/_external/`
- Create `Lib/test/test_tomllib/__main__.py`:
```python
import unittest
from . import load_tests
unittest.main()
```
- Add the following to `Lib/test/test_tomllib/__init__.py`:
```python
import os
from test.support import load_package_tests
def load_tests(*args):
return load_package_tests(os.path.dirname(__file__), *args)
```
Also change `import tomli as tomllib` to `import tomllib`.
- In `cpython/Lib/tomllib/_parser.py` replace `__fp` with `fp` and `__s` with
`s`. Add the `/` to `load` and `loads` function signatures.
- Run `make regen-stdlib-module-names`
- Create `Doc/library/tomllib.rst` and reference it in `Doc/library/fileformats.rst`
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
This removes discrepancy between list["int"] and List["int"].
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
When an exception is created in a nested call to PyObject_GetAttr, any
external calls will override the context information of the
AttributeError that we have already placed in the most internal call.
This will cause the suggestions we create to nor work properly as the
attribute name and object that we will be using are the incorrect ones.
To avoid this, we need to check first if these attributes are already
set and bail out if that's the case.
Also made modes containing 'a' or 'x' act the same as a mode containing 'w' when argument is '-'
(so 'a'/'x' return sys.stdout like 'w', and 'ab'/'xb' return sys.stdout.buffer like 'wb').
If Python is built with UBSan, test_ctypes now skips test_shorts().
This change allows to run test_ctypes to check for new UBSan regression,
but the known test_shorts() undefined behavior must be fixed.
If Python is built with UBSan, test_hashlib skips tests on the _sha3
extension which currently has undefined behaviors.
This change allows to run test_hashlib to check for new UBSan regression,
but the known _sha3 undefined behavior must be fixed.
Skip tests on ASAN and/or MSAN builds:
* multiprocessing tests
* test___all__
* test_concurrent_futures
* test_decimal
* test_peg_generator
* test_tools
The ASAN job of GitHub Actions no longer excludes these tests.
Also from the _asyncio C accelerator module,
and adjust one test that the change caused to fail.
For more discussion see the discussion starting here:
https://github.com/python/cpython/pull/31394#issuecomment-1053545331
(Basically, @asvetlov proposed to return False from cancel()
when there is already a pending cancellation, and I went along,
even though it wasn't necessary for the task group implementation,
and @agronholm has come up with a counterexample that fails
because of this change. So now I'm changing it back to the old
semantics (but still bumping the counter) until we can have a
proper discussion about this.)
For threads, and for multiprocessing, it's always been the case that ``args=list`` works fine when passed to ``Process()`` or ``Thread()``, and such code is common in the wild. But, according to the docs, only a tuple can be used. This brings the docs into synch with reality.
Doc changes by Charlie Zhao.
Co-authored-by: Tim Peters <tim.peters@gmail.com>
Remove the undocumented private float.__set_format__() method,
previously known as float.__set_format__() in Python 3.7. Its
docstring said: "You probably don't want to use this function. It
exists mainly to be used in Python's test suite."
Fix a bug in urllib.request.HTTPPasswordMgr.find_user_password() and
urllib.request.HTTPPasswordMgrWithPriorAuth.is_authenticated() which
allowed to bypass authorization. For example, access to URI "example.org/foobar"
was allowed if the user was authorized for URI "example.org/foo".
This changes cancelling() and uncancel() to return the count of pending cancellations.
This can be used to avoid bugs in certain edge cases (e.g. two timeouts going off at the same time).
The locale.getdefaultlocale() function is deprecated and will be
removed in Python 3.13. Use locale.setlocale(),
locale.getpreferredencoding(False) and locale.getlocale() functions
instead.
Fix parsing a numeric literal immediately (without spaces) followed by
"not in" keywords, like in "1not in x". Now the parser only emits
a warning, not a syntax error.
Curly brackets were never allowed in namespace URIs
according to RFC 3986, and so-called namespace-validating
XML parsers have the right to reject them a invalid URIs.
libexpat >=2.4.5 has become strcter in that regard due to
related security issues; with ET.XML instantiating a
namespace-aware parser under the hood, this test has no
future in CPython.
References:
- https://datatracker.ietf.org/doc/html/rfc3968
- https://www.w3.org/TR/xml-names/
Also, test_minidom.py: Support Expat >=2.4.5
It fixes the "Text File Busy" OSError when using 'rmtree' on a
windows-managed filesystem in via the VirtualBox shared folder
(and possible other scenarios like a windows-managed network file
system).
There are several changes:
1. We now don't explicitly check for any base / sub types, because new name check covers it
2. I've also checked that `no_type_check` do not modify foreign functions. It was the same as with `type`s
3. I've also covered `except TypeError` in `no_type_check` with a simple test case, it was not covered at all
4. I also felt like adding `lambda` test is a good idea: because `lambda` is a bit of both in class bodies: a function and an assignment
<!-- issue-number: [bpo-46571](https://bugs.python.org/issue46571) -->
https://bugs.python.org/issue46571
<!-- /issue-number -->
* Remove task group names (for now)
We're not sure that they are needed, and once in the code
we would never be able to get rid of them.
Yury wrote:
> Ideally, there should be a way for someone to build a "trace"
> of taskgroups/task leading to the current running task.
> We could do that using contextvars, but I'm not sure we should
> do that in 3.11.
* Pass optional name on to task in create_task()
* Remove a bunch of unused stuff
* Add exception for uninstantiated interpolation (configparser)
The current feedback when users try to pass an uninstantiated
interpolation into a ConfigParser is an error message that does not help
users solve the problem. This current error of `TypeError: before_set()
missing 1 required positional argument: 'value'` does not display until
the parser is used, which usually results in the assumption that
instantiation of the parser was done correctly. The new exception of
InterpolationTypeError, will be raised on the line where the
ConfigParser is instantiated. This will result in users see the line
that has the error in their backtrace for faster debugging.
There have been a number of bugs created in the issue tracker, which
could have been addressed by:
https://bugs.python.org/issue26831 and https://bugs.python.org/issue26469
* 📜🤖 Added by blurb_it.
* Replace custom Error with TypeError
Per feedback from @iritkatriel, the custom InterpolationTypeError has
been dropped in favour of a TypeError with a custom message, and the
unittests have been expanded.
* More verbose message
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
The `module` parameter carries semantic information about the forward ref.
Forward refs are different if they refer to different module even if they
have the same name. This affects the `__eq__`, `__repr__` and `__hash__` methods.
Co-authored-by: Andreas Hangauer <andreas.hangauer@siemens.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
asyncio/taskgroups.py is an adaptation of taskgroup.py from EdgeDb, with the following key changes:
- Allow creating new tasks as long as the last task hasn't finished
- Raise [Base]ExceptionGroup (directly) rather than TaskGroupError deriving from MultiError
- Instead of monkey-patching the parent task's cancel() method,
add a new public API to Task
The Task class has a new internal flag, `_cancel_requested`, which is set when `.cancel()` is called successfully. The `.cancelling()` method returns the value of this flag. Further `.cancel()` calls while this flag is set return False. To reset this flag, call `.uncancel()`.
Thus, a Task that catches and ignores `CancelledError` should call `.uncancel()` if it wants to be cancellable again; until it does so, it is deemed to be busy with uninterruptible cleanup.
This new Task API helps solve the problem where TaskGroup needs to distinguish between whether the parent task being cancelled "from the outside" vs. "from inside".
Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
Also add .pyi to the python extensions in the "File-open" and "File-save" dialogues.
Add util.py to contain objects that are used in multiple idlelib modules
and have no dependencies on any of them.
Co-authored-by: E-Paine <63801254+E-Paine@users.noreply.github.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
The module parameter carries semantic information about the forward ref.
Show to the user that forward refs with same argument but different
module are different.
Co-authored-by: Andreas Hangauer <andreas.hangauer@siemens.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Ensure strong references are acquired whenever using `set_next()`. Added randomized test cases for `__eq__` methods that sometimes mutate sets when called.
Confirmed with @jaraco that this indeed needs a fix.
A question that came up while I was digging into the code: I think `SelectableGroups` could similarly use `__slots__ = ()`, since its purpose seems only for convenience around `dict`, not to have attributes of its own.
Automerge-Triggered-By: GH:jaraco
Numeric fields of type float, notably mtime, can't be represented
exactly in the ustar header, so the pax header is used. But it is
helpful to set them to the nearest int (i.e. second rather than
nanosecond precision mtimes) in the ustar header as well, for the
benefit of unarchivers that don't understand the pax header.
Add test for tarfile.TarInfo.create_pax_header to confirm correct
behaviour.
In [Lib/test/support/import_helper.py](https://github.com/python/cpython/blob/master/Lib/test/support/import_helper.py), the function `make_legacy_pyc` makes a call to `os.rename` which can fail when the source and target live on different devices. This happens (for example) when `PYTHONPYCACHEPREFIX` is set to a directory anywhere on disk, while a ramdisk is mounted on `/tmp` (the latter of which is the default on various Linux distros). Replacing `os.rename` with `shutil.move` fixes this.
Automerge-Triggered-By: GH:brettcannon
The calendar.LocaleTextCalendar and calendar.LocaleHTMLCalendar
classes module now use locale.getlocale(), instead of using
locale.getdefaultlocale(), if no locale is specified.
ctypes.CFUNCTYPE() and ctypes.WINFUNCTYPE() now fail to create the
type if its "_argtypes_" member contains too many arguments.
Previously, the error was only raised when calling a function.
Change also how CFUNCTYPE() and WINFUNCTYPE() handle KeyError to
prevent creating a chain of exceptions if ctypes.CFuncPtr raises an
error.
Inada Naoki
Alix Lourme
Brett Cannon
Nikita Sobolev
Dustin Rodrigues
Serhiy Storchaka
Jelle Zijlstra
Ilya Leoshkevich
Irit Katriel
Jack DeVries
Erlend Egeberg Aasland
Dennis Sweeney
Adrian Garcia Badaracco
John Belmonte
Petr Viktorin
Mark Shannon
Christian Heimes
Steve Dower
Jeremy Kloth
Géry Ogam
Ken Jin
Victor Stinner
Pablo Galindo
Sam Ezeh
Hugo van Kemenade
Matthieu Dartiailh
Ronald Oussoren
Matthew Rahtz
Shantanu
180909
Jason R. Coombs
Ma Lin
Andrew Svetlov
Oleg Iarygin
Daniël van Noord
Dave Goncalves
Vincent Bernat
Maciej Górski
Jonathan
Kumar Aditya
Jouke Witteveen
Duprat
Tim Peters
Brandt Bucher
Gregory P. Smith
Kevin Mehall
Arie Bovenberg
Miro Hrončok
Bader Zaidan
Carlos Damazio
Maximilian Hils
Michael P. Nitowski
Ned Deily
Pradyun Gedam
Alex Grönholm
andrei kulakov
Gregory Beauregard
Alex Waygood
Matt Bogosian
Stefan Zabka
Taneli Hukkinen
James Hilton-Balfe
Niklas Rosenstein
Jacob Walls
MojoVampire
Ken Jin
Matt Bogosian
slateny
Guido van Rossum
Raymond Hettinger
Jannis Vajen
Tin Tvrtković
Thomas Grainger
Charlie Zhao
Dong-hee Na
Eric V. Smith
Sebastian Pipping
Lital Natan
Josh Smith
Saul Shanabrook
辰冢
Patrick Reader
Brian Faherty
aha79
97littleleaf11
Hakan Çelik
Alex-Blade
Zackery Spytz
Brandt Bucher
Joshua Root
Andrzej Mateja
Barney Gale
Jason Wilkes