dataclasses.replace() now raises TypeError instead of ValueError if
specify keyword argument for a field declared with init=False or miss keyword
argument for required InitVar field.
This method supports file URIs (including variants) as described in RFC 8089, such as URIs generated by `pathlib.Path.as_uri()` and `urllib.request.pathname2url()`.
The method is added to `Path` rather than `PurePath` because it uses `os.fsdecode()`, and so its results vary from system to system. I intend to deprecate `PurePath.as_uri()` and move it to `Path` for the same reason.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* Refactor os_sched_getaffinity_impl(): move variable definitions to
their first assignment.
* Fix test_posix.test_sched_getaffinity(): restore the old CPU mask
when the test completes!
* Doc: Specify that os.cpu_count() counts *logicial* CPUs.
* Doc: Specify that os.sched_getaffinity(0) is related to the calling
thread.
Add private `pathlib._PathBase` class. This will be used by an experimental PyPI package to incubate a `tarfile.TarPath` class.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
concurrent.futures: The *executor manager thread* now catches
exceptions when adding an item to the *call queue*. During Python
finalization, creating a new thread can now raise RuntimeError. Catch
the exception and call terminate_broken() in this case.
Add test_python_finalization_error() to test_concurrent_futures.
concurrent.futures._ExecutorManagerThread changes:
* terminate_broken() no longer calls shutdown_workers() since the
call queue is no longer working anymore (read and write ends of
the queue pipe are closed).
* terminate_broken() now terminates child processes, not only
wait until they complete.
* _ExecutorManagerThread.terminate_broken() now holds shutdown_lock
to prevent race conditons with ProcessPoolExecutor.submit().
multiprocessing.Queue changes:
* Add _terminate_broken() method.
* _start_thread() sets _thread to None on exception to prevent
leaking "dangling threads" even if the thread was not started
yet.
On Windows, multiprocessing Popen.terminate() now catchs
PermissionError and get the process exit code. If the process is
still running, raise again the PermissionError. Otherwise, the
process terminated as expected: store its exit code.
* Refactor zipfile._strip_extra to use higher level abstractions for extras instead of a heavy-state loop.
* Add blurb
* Remove _strip_extra and use _Extra.strip directly.
* Use memoryview to avoid unnecessary copies while splitting Extras.
This fixes issue #105829, https://github.com/python/cpython/issues/105829
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Chris Withers <chris@withers.org>
Co-authored-by: Thomas Moreau <thomas.moreau.2010@gmail.com>
Fix os.stat() and os.DirEntry.stat(): check for exceptions.
Previously, on Python built in debug mode, these functions could
trigger a fatal Python error (and abort the process) when a function
succeeded with an exception set.
_pystat_fromstructstat() now exits immediately if an exception is
raised, rather only checking for exceptions at the end. It fix
following fatal error in fill_time():
Fatal Python error: _Py_CheckSlotResult:
Slot * of type int succeeded with an exception set
Adds APIs to get the TLS certificate chains, verified or full unverified, from SSLSocket and SSLObject.
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
The filename was previously intentionally omitted from exception because
"it might confuse the user". Uncaught exceptions are not generally a
replacement for user-facing error messages, so obscuring this
information only has the effect of making the programmer's life more
difficult.
This feature is off by default via code but on by default via the CLI. The `.gitignore` file contains `*` which causes the entire directory to be ignored.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.
thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.
Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().
Fix a race condition in concurrent.futures. When a process in the
process pool was terminated abruptly (while the future was running or
pending), close the connection write end. If the call queue is
blocked on sending bytes to a worker process, closing the connection
write end interrupts the send, so the queue can be closed.
Changes:
* _ExecutorManagerThread.terminate_broken() now closes
call_queue._writer.
* multiprocessing PipeConnection.close() now interrupts
WaitForMultipleObjects() in _send_bytes() by cancelling the
overlapped operation.
Attempts to pickle or create a shallow or deep copy of codecs streams
now raise a TypeError.
Previously, copying failed with a RecursionError, while pickling
produced wrong results that eventually caused unpickling to fail with
a RecursionError.
add guard so that ``Enum('bar')`` raises a TypeError instead of
creating a new enum class called `bar`. To create the new but
empty class, use:
huh = Enum('bar', names=())
It creates a modified copy of an object by calling the object's
__replace__() method.
It is a generalization of dataclasses.replace(), named tuple's _replace()
method and replace() methods in various classes, and supports all these
stdlib classes.
When overriding the `__new__` method of an enum, the underlying data type should be created directly; i.e. .
member = object.__new__(cls)
member = int.__new__(cls, value)
member = str.__new__(cls, value)
Calling `super().__new__()` finds the lookup version of `Enum.__new__`, and will now raise an exception when detected.
Deprecate passing the callback callable by keyword for the following
sqlite3.Connection APIs:
- set_authorizer(authorizer_callback)
- set_progress_handler(progress_handler, ...)
- set_trace_callback(trace_callback)
The affected parameters will become positional-only in Python 3.15.
Deprecate passing name, number of arguments, and the callable as keyword
arguments, for the following sqlite3.Connection APIs:
- create_function(name, nargs, callable, ...)
- create_aggregate(name, nargs, callable)
The affected parameters will become positional-only in Python 3.15.
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
In the stack call of: _init_read_gz()
```
_read, tarfile.py:548
read, tarfile.py:526
_init_read_gz, tarfile.py:491
```
a try;except exists that uses `self.exception`, so it needs to be set before
calling _init_read_gz().
- Add explanatory comments
- Add return value to connection_close() for propagating errors
- Always check the return value of connection_exec_stmt()
- Assert pre/post state in remove_callbacks()
- Don't log unraisable exceptions in case of interpreter shutdown
- Make sure we're not initialized if reinit fails
- Try to close the database even if ROLLBACK fails
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Deprecate passing optional arguments maxsplit, count and flags in
module-level functions re.split(), re.sub() and re.subn() as positional.
They should only be passed by keyword.
Restore following CPython <= 3.10.5 behavior of shutil.make_archive()
that went away as part of gh-93160:
Do not create an empty archive if root_dir is not a directory, and, in
that case, raise FileNotFoundError or NotADirectoryError regardless
of format choice. Beyond the brought-back behavior, the function may
now also raise these exceptions in dry_run mode.
It did not work in the case of a subpattern containing backtracking.
Temporary implement possessive quantifiers as equivalent greedy qualifiers
in atomic groups.
This patch escapes the class name before embedding it in the regular expression
for `pat` in `doctest.DocTestFinder._find_lineno`. While class names do not
ordinarily contain special characters, it is possible to encounter these when a
class is created dynamically. Escaping the name will correctly return `None` in
this scenario, rather than potentially matching a different class or raising
`re.error` depending on the symbols used.
Represent IPv4-mapped IPv6 address as x❌x❌x❌d.d.d.d,
where the 'x's are the hexadecimal values
of the six high-order 16-bit pieces of the address,
and the 'd's are the decimal values
of the four low-order 8-bit pieces of the address
(standard IPv4 representation).
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
Adding the `rtype_cache` to the `warnings.warn` message improves the
previous, somewhat vague message from
```
/Users/username/cpython/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown
```
to
```
/Users/username/cpython/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown: {'/mp-yor5cvj8', '/mp-10jx8eqr', '/mp-eobsx9tt', '/mp-0lml23vl', '/mp-9dgtsa_m', '/mp-frntyv4s'}
```
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Use lowercase `mail from` and `rcpt to` in `smtplib.SMTP`
SMTP commands are case-insensitive. `smtplib` uses lowercase commands,
however it writes `mail FROM` and `rcpt TO`, lacking consistency.
* gh-105293: Do not call SSL_CTX_set_session_id_context on client side SSL context
Openssl states this is a "server side only" operation.
Calling this on a client side socket can result in unexpected behavior
* Add news entry on SSL "set session id context" changes
gh-86618 assumed a-b-c = a-(b+c) = a-d where d = b+d.
For floats 2.0, 1.0, and 0.9999999999999999, this assumption
is false. The net change of 1.1102230246251565e-16 to 0.0
results in division by 0. Revert the replacement. Add test.
Fixes a hang in multiprocessing process pool executor when a child process crashes and code could otherwise block on writing to the pipe. See GH-94777 for more details.
Some items remained uninitialized if _sre.template() was called with invalid
indices. Then attempt to clear them in the destructor led to dereferencing
of uninitialized pointer.
Prevent `multiprocessing.spawn` from failing to *import* in environments
where `sys.executable` is `None`. This regressed in 3.11 with the addition
of support for path-like objects in multiprocessing.
Adds a test decorator to have tests only run when part of test_multiprocessing_spawn to `_test_multiprocessing.py` so we can start to avoid re-running the same not-global-state specific test in all 3 modes when there is no need.
We match paths using the `_lines` attribute, which is derived from the
path's string representation. The bug arises because an empty path's string
representation is `'.'` (not `''`), which is matched by the `'*'` wildcard.
mock: Add `ThreadingMock` class
Add a new class that allows to wait for a call to happen by using
`Event` objects. This mock class can be used to test and validate
expectations of multithreading code.
It uses two attributes for events to distinguish calls with any argument
and calls with specific arguments.
The calls with specific arguments need a lock to prevent two calls in
parallel from creating the same event twice.
The timeout is configured at class and constructor level to allow users
to set a timeout, we considered passing it as an argument to the
function but it could collide with a function parameter. Alternatively
we also considered passing it as positional only but from an API
caller perspective it was unclear what the first number meant on the
function call, think `mock.wait_until_called(1, "arg1", "arg2")`, where
1 is the timeout.
Lastly we also considered adding the new attributes to magic mock
directly rather than having a custom mock class for multi threading
scenarios, but we preferred to have specialised class that can be
composed if necessary. Additionally, having added it to `MagicMock`
directly would have resulted in `AsyncMock` having this logic, which
would not work as expected, since when if user "waits" on a
coroutine does not have the same meaning as waiting on a standard
call.
Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
If the output arg to Helper() is a stream rather than the default None, which means 'page to stdout', the ImportError from pydoc.resolve is currently not caught in pydoc.doc. The same error is caught when output is None.
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Brings `pathlib.Path.is_dir()` and `in line with `os.DirEntry.is_dir()`, which
will be important for implementing generic path walking and globbing.
Likewise `is_file()`.
This new exception type is raised instead of `NotImplementedError` when
a path operation is not supported. It can be raised from `Path.readlink()`,
`symlink_to()`, `hardlink_to()`, `owner()` and `group()`. In a future
version of pathlib, it will be raised by `AbstractPath` for these methods
and others, such as `AbstractPath.mkdir()` and `unlink()`.
Deprecate two methods of creating typing.TypedDict classes with 0 fields using the functional syntax: `TD = TypedDict("TD")` and `TD = TypedDict("TD", None)`. Both will be disallowed in Python 3.15. To create a TypedDict class with 0 fields, either use `class TD(TypedDict): pass` or `TD = TypedDict("TD", {})`.
Deprecate creating a typing.NamedTuple class using keyword arguments to denote the fields (`NT = NamedTuple("NT", x=int, y=str)`). This will be disallowed in Python 3.15. Use the class-based syntax or the functional syntax instead.
Two methods of creating `NamedTuple` classes with 0 fields using the functional syntax are also deprecated, and will be disallowed in Python 3.15: `NT = NamedTuple("NT")` and `NT = NamedTuple("NT", None)`. To create a `NamedTuple` class with 0 fields, either use `class NT(NamedTuple): pass` or `NT = NamedTuple("NT", [])`.
* GH-104554: Add RTSPS support to `urllib/parse.py`
RTSPS is the permanent scheme defined in
https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml
alongside RTSP and RTSPU schemes.
* 📜🤖 Added by blurb_it.
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Adds start_offset, cache_offset, end_offset, baseopcode,
baseopname, jump_target and oparg to dis.Instruction.
Also slightly improves the disassembly output by allowing
opnames to overflow into the space reserved for opargs.
Fix bugs where exceptions could end up being overwritten
because of deferred error handling.
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
When inverting a Flag member (or boundary STRICT), only consider other canonical flags; when inverting an IntFlag member (or boundary KEEP), also consider aliases.
* support inspect.iscoroutinefunction in create_autospec(async_def)
* test create_autospec with inspect.iscoroutine and inspect.iscoroutinefunction
* test when create_autospec functions check their signature
The original name wasn't as clear as it could have been. This change includes the following:
* rename the function
* change the default value for "disable_check" to False
* add clues to the docstring that folks should probably not use the function
---------
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
This commit introduces a 'walk-and-match' strategy for handling glob patterns that include a non-terminal `**` wildcard, such as `**/*.py`. For this example, the previous implementation recursively walked directories using `os.scandir()` when it expanded the `**` component, and then **scanned those same directories again** when expanded the `*.py` component. This is wasteful.
In the new implementation, any components following a `**` wildcard are used to build a `re.Pattern` object, which is used to filter the results of the recursive walk. A pattern like `**/*.py` uses half the number of `os.scandir()` calls; a pattern like `**/*/*.py` a third, etc.
This new algorithm does not apply if either:
1. The *follow_symlinks* argument is set to `None` (its default), or
2. The pattern contains `..` components.
In these cases we fall back to the old implementation.
This commit also replaces selector classes with selector functions. These generators directly yield results rather calling through to their successors. A new internal `Path._glob()` method takes care to chain these generators together, which simplifies the lazy algorithm and slightly improves performance. It should also be easier to understand and maintain.
`PurePath.match()` now handles the `**` wildcard as in `Path.glob()`, i.e. it matches any number of path segments.
We now compile a `re.Pattern` object for the entire pattern. This is made more difficult by `fnmatch` not treating directory separators as special when evaluating wildcards (`*`, `?`, etc), and so we arrange the path parts onto separate *lines* in a string, and ensure we don't set `re.DOTALL`.
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Add a keyword-only *follow_symlinks* parameter to `pathlib.Path.glob()` and`rglob()`.
When *follow_symlinks* is `None` (the default), these methods follow symlinks except when evaluating "`**`" wildcards. When set to true or false, symlinks are always or never followed, respectively.
For backwards compatibility, accept backslashes as path separators in
`PurePosixPath` if an instance of `PureWindowsPath` is supplied.
This restores behaviour from Python 3.11.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Use `str.lower()` rather than `ntpath.normcase()` to normalize case of
Windows paths. This restores behaviour from Python 3.11.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* Remove the Lib/test/imghdrdata/ directory.
* Copy 5 pictures (gif, png, ppm, pgm, xbm) from removed
Lib/test/imghdrdata/ to a new Lib/test/tkinterdata/ directory.
* Update Sphinx from 4.5 to 6.2 in Doc/requirements.txt.
Reduced _idle_semaphore.release calls in concurrent.futures.thread._worker
_idle_semaphore.release() is now only called if only work_queue is empty.
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
On Linux where the `subprocess` module can use the `vfork` syscall for
faster spawning, prevent the parent process from blocking other threads
by dropping the GIL while it waits for the vfork'ed child process `exec`
outcome. This prevents spawning a binary from a slow filesystem from
blocking the rest of the application.
Fixes#104372.
Sam Gross
Xuehai Pan
Serhiy Storchaka
Alex Waygood
Guido van Rossum
Charles Machalow
Barney Gale
Raymond Hettinger
Victor Stinner
Furkan Onder
James Hilton-Balfe
Jelle Zijlstra
Davide Rizzo
Amin Alaee
Nikita Sobolev
Dale Collison
Antoine Pitrou
Jason R. Coombs
Hugo van Kemenade
Tian Gao
elfstrom
EliseevEgor
AN Long
James Gerity
Mateusz Nowak
Ronan Pigott
Shantanu
Brett Cannon
Gregory P. Smith
buermarc
Irit Katriel
Ethan Furman
Anthony Shaw
Erlend E. Aasland
Corvin
Matthias Bussonnier
Ori Avtalion
denballakh
Peeyush Aggarwal
Chris Markiewicz
Petr Viktorin
balmeida-nokia
Tyler Smart
SKO
6t8k
Finn Womack
Dong-hee Na
Romuald Brunet
Andrew Geng
Mina Galić
Gertjan van Zwieten
Pieter Eendebak
Kumar Aditya
Pablo Galindo Salgado
Michael The
opavliuk
Konstantin
Charlie Zhao
Inada Naoki
James Cave
János Kukovecz
Christopher Chavez
shailshouryya
Paul Moore
Tomas R
htsedebenham
Gilles Bassière
J. Nick Koston
Matthieu Caneill
Grigoriev Semyon
Grant Ramsay
CF Bolz-Tereick
Jim Porter
Prince Roshan
Terry Jan Reedy
Louis Paulot
Radislav Chugunov
Desmond Cheong
Ariel Eizenberg
Carl Meyer
Mario Corchero
Kirill Podoprigora
Łukasz Langa
Gabriel Venberg
James Webber
T. Wouters
zentarim
Erlend E. Aasland
Kumar Aditya
Samet YASLAN
Thomas Grainger
Eric Snow
Zachary Ware
Andrii Kuzmin
Gregory P. Smith