Commit Graph

1021 Commits

Author SHA1 Message Date
Nikita Sobolev 16b46ebd2b
gh-119121: Fix and test `async.staggered.staggered_race` (#119173) 2024-05-20 14:06:50 +03:00
Kirill Podoprigora c3643a1214
gh-118817: Fix `asyncio REPL` on Windows (#118819) 2024-05-09 08:20:46 -07:00
Bruce Merry 01a51f9494
gh-117722: Fix Stream.readuntil with non-bytes buffer objects (#117723)
gh-16429 introduced support for an iterable of separators in
Stream.readuntil. Since bytes-like types are themselves iterable, this
can introduce ambiguities in deciding whether the argument is an
iterator of separators or a singleton separator. In gh-16429, only 'bytes'
was considered a singleton, but this will break code that passes other
buffer object types.

Fix it by only supporting tuples rather than arbitrary iterables.

Closes gh-117722.
2024-04-11 07:41:55 -07:00
Guido van Rossum fa58e75a86
gh-116720: Fix corner cases of taskgroups (#117407)
This prevents external cancellations of a task group's parent task to
be dropped when an internal cancellation happens at the same time.
Also strengthen the semantics of uncancel() to clear self._must_cancel
when the cancellation count reaches zero.

Co-Authored-By: Tin Tvrtković <tinchester@gmail.com>
Co-Authored-By: Arthur Tacca
2024-04-09 08:17:28 -07:00
Bruce Merry 775912a51d
gh-81322: support multiple separators in StreamReader.readuntil (#16429) 2024-04-08 09:58:02 -07:00
Laurie O e16062dd34
gh-96471: Correct documentation for asyncio queue shutdown (#117621) 2024-04-08 14:50:54 +00:00
Laurie O df4d84c3cd
gh-96471: Add asyncio queue shutdown (#104228)
Co-authored-by: Duprat <yduprat@gmail.com>
2024-04-06 07:27:13 -07:00
rsp4jack 85843348c5
gh-117459: Keep the traceback in _convert_future_exc (#117460) 2024-04-03 20:13:32 -07:00
Justin Turner Arthur c741ad3537
gh-77714: Provide an async iterator version of as_completed (GH-22491)
* as_completed returns object that is both iterator and async iterator
* Existing tests adjusted to test both the old and new style
* New test to ensure iterator can be resumed
* New test to ensure async iterator yields any passed-in Futures as-is

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2024-04-01 20:07:29 +03:00
Erik Soma f11d0d8be8
gh-91227: Ignore ERROR_PORT_UNREACHABLE in proactor recvfrom() (#32011) 2024-03-23 08:39:35 -07:00
jkriegshauser fc45998007
gh-116773: Ensure overlapped objects on Windows are not deallocated too early by asyncio (GH-116774) 2024-03-20 14:33:28 +00:00
Pierre Ossman (ThinLinc team) 4159644177
gh-113538: Add asycio.Server.{close,abort}_clients (redo) (#116784)
These give applications the option of more forcefully terminating client
connections for asyncio servers. Useful when terminating a service and
there is limited time to wait for clients to finish up their work.

This is a do-over with a test fix for gh-114432, which was reverted.
2024-03-18 13:15:53 -07:00
Guido van Rossum ba13215eb1
gh-113538: Revert "gh-113538: Add asycio.Server.{close,abort}_clients (#114432)" (#116632)
Revert "gh-113538: Add asycio.Server.{close,abort}_clients (#114432)"

Reason: The new test doesn't always pass:
https://github.com/python/cpython/pull/116423#issuecomment-1989425489

This reverts commit 1d0d49a7e8.
2024-03-12 00:31:49 +00:00
Pierre Ossman (ThinLinc team) 1d0d49a7e8
gh-113538: Add asycio.Server.{close,abort}_clients (#114432)
These give applications the option of more forcefully terminating client
connections for asyncio servers. Useful when terminating a service and
there is limited time to wait for clients to finish up their work.
2024-03-11 12:43:30 -07:00
Jason Zhang ce0ae1d784
gh-115957: Close coroutine if TaskGroup.create_task() raises an error (#116009) 2024-03-06 12:20:26 -08:00
cui fliter e7ba6e9dbe
chore: fix typos (#116345)
Signed-off-by: cui fliter <imcusg@gmail.com>
2024-03-05 09:05:52 -07:00
Rémi Lapeyre b5949eac62
gh-84995: Run sys.__interactivehook__() on asyncio REPL startup (#20517)
This makes the asyncio REPL (`python -m asyncio`) more usable
and similar to the regular REPL.

This exposes register_readline() as a top-level function in site.py,
but it's intentionally undocumented. 

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
Co-authored-by: Itamar Oren <itamarost@gmail.com>
2024-03-01 11:39:16 -08:00
Pierre Ossman (ThinLinc team) 5a1559d949
gh-112997: Don't log arguments in asyncio unless debugging (#115667)
Nothing else in Python generally logs the contents of variables, so this
can be very unexpected for developers and could leak sensitive
information in to terminals and log files.
2024-02-27 17:39:08 -08:00
Pierre Ossman (ThinLinc team) a355f60b03
gh-114914: Avoid keeping dead StreamWriter alive (#115661)
In some cases we might cause a StreamWriter to stay alive even when the
application has dropped all references to it. This prevents us from
doing automatical cleanup, and complaining that the StreamWriter wasn't
properly closed.

Fortunately, the extra reference was never actually used for anything so
we can just drop it.
2024-02-27 17:27:44 -08:00
Jamie Phan 53d5e67804
gh-111358: Fix timeout behaviour in BaseEventLoop.shutdown_default_executor (#115622) 2024-02-19 00:01:00 +00:00
Jamie Phan 73e8637002
gh-113812: Allow DatagramTransport.sendto to send empty data (#115199)
Also include the UDP packet header sizes (8 bytes per packet)
in the buffer size reported to the flow control subsystem.
2024-02-16 18:38:07 -08:00
Travis Howse 94ec2b9c9c
gh-114887 Reject only sockets of type SOCK_STREAM in create_datagram_endpoint() (#114893)
Also improve exception message.

Co-authored-by: Donghee Na <donghee.na92@gmail.com>
2024-02-03 17:14:02 +00:00
Kristján Valur Jónsson 6b53d5fe04
gh-112202: Ensure that condition.notify() succeeds even when racing with Task.cancel() (#112201)
Also did a general cleanup of asyncio locks.py comments and docstrings.
2024-02-03 08:19:37 -08:00
Jamie Phan 80aa7b3688
gh-109534: fix reference leak when SSL handshake fails (#114074) 2024-01-31 16:42:17 -08:00
Matan Perelman 29952c86f3
TaskGroup: Use explicit None check for cancellation error (#114708) 2024-01-29 11:12:33 -08:00
Serhiy Storchaka a53e56e7d8
gh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.create_server() (GH-114420)
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2024-01-22 18:40:35 +02:00
Nikita Sobolev 0554a9594e
gh-114281: Remove incorrect type hints from `asyncio.staggered` (#114282)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2024-01-20 16:49:48 +00:00
Alois Klink dce30c9cbc
gh-95649: Document that asyncio contains uvloop code (#107536)
Some of the asyncio SSL changes in GH-31275 [1] were taken from
v0.16.0 of the uvloop project [2]. In order to comply with the MIT
license, we need to just need to document the copyright information.

[1]: https://github.com/python/cpython/pull/31275
[2]: https://github.com/MagicStack/uvloop/tree/v0.16.0
2024-01-12 22:21:13 +00:00
Serhiy Storchaka aef4a1203c
gh-96037: Always insert TimeoutError when exit an expired asyncio.timeout() block (GH-113819)
If other exception was raised during exiting an expired
asyncio.timeout() block, insert TimeoutError in the exception context
just above the CancelledError.
2024-01-10 12:50:31 +02:00
Jamie Phan 4826d52338
gh-112182: Replace StopIteration with RuntimeError for future (#113220)
When an `StopIteration` raises into `asyncio.Future`, this will cause
a thread to hang. This commit address this by not raising an exception
and silently transforming the `StopIteration` with a `RuntimeError`,
which the caller can reconstruct from `fut.exception().__cause__`
2024-01-09 21:21:00 -08:00
Serhiy Storchaka a5db6a3351
gh-113848: Handle CancelledError subclasses in asyncio TaskGroup() and timeout() (GH-113850) 2024-01-09 21:41:31 +02:00
Kristján Valur Jónsson 52161781a6
GH-111693: Propagate correct asyncio.CancelledError instance out of asyncio.Condition.wait() (#111694)
Also fix a race condition in `asyncio.Semaphore.acquire()` when cancelled.
2024-01-08 11:57:48 -08:00
Guido van Rossum 4681a5271a
gh-113538: Don't error in stream reader protocol callback when task is cancelled (#113690) 2024-01-04 12:20:21 -08:00
Allison Karlitskaya 0187a7e4ec
gh-112800: Ignore PermissionError on SubprocessTransport.close() in asyncio (#112803)
In case the spawned process is setuid, we may not be able to send
signals to it, in which case our .kill() call will raise
PermissionError.

Ignore that in order to avoid .close() raising an exception.  Hopefully
the process will exit as a result of receiving EOF on its stdin.
2023-12-23 16:43:39 -08:00
Martijn Pieters 1ff0238594
GH-113214: Fix SSLProto exception handling in SSL-over-SSL scenarios (#113334)
When wrapped, `_SSLProtocolTransport._force_close(exc)` is called just like in the unwrapped scenario `_SelectorTransport._force_close(exc)` or `_ProactorBasePipeTransport._force_close(exc)` would be called, except here the exception needs to be passed through the `SSLProtocol._abort()` method, which didn't accept an exception object.

This commit ensures that this path works, in the same way that the uvloop implementation of SSLProto passes on the exception (on which the current implementation of SSLProto is based).
2023-12-20 15:09:01 -08:00
beavailable 3aea6c4823
gh-101336: Add keep_alive keyword arg for asyncio create_server() (#112485) 2023-12-13 03:23:29 +00:00
Jamie a3a1cb4845
gh-112622: Pass name to loop create_task method (#112623)
This affects task creation through either `asyncio.create_task()` or `TaskGroup.create_task()` -- the redundant call to `task.set_name()` is skipped. We still call `set_name()` when a task factory is involved, because the task factory call signature (unfortunately) doesn't take a `name` argument.
2023-12-12 17:26:40 -08:00
J. Nick Koston 7e2d93f30b
gh-112989: asyncio: Reduce overhead to connect sockets with SelectorEventLoop (#112991)
_ensure_fd_no_transport had a KeyError in the success path
2023-12-12 16:29:21 -08:00
DPR e0f5127975
gh-109538: Avoid RuntimeError when StreamWriter is deleted with closed loop (#111983)
Issue a ResourceWarning instead.

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-11-15 01:17:51 +00:00
Pierre Ossman (ThinLinc team) 74b868f636
gh-111246: Remove listening Unix socket on close (#111483)
Try to clean up the socket file we create so we don't add unused noise to the file system.
2023-11-08 08:10:10 -08:00
Kumar Aditya 229f44d353
GH-110894: Call loop exception handler for exceptions in client_connected_cb (#111601)
Call loop exception handler for exceptions in `client_connected_cb` of `asyncio.start_server` so that applications can handle it.
2023-11-02 07:38:18 +00:00
Guido van Rossum 2655369559
gh-79033: Try to fix asyncio.Server.wait_closed() again (GH-111336)
* Try to fix asyncio.Server.wait_closed() again

I identified the condition that `wait_closed()` is intended
to wait for: the server is closed *and* there are no more
active connections.

When this condition first becomes true, `_wakeup()` is called
(either from `close()` or from `_detach()`) and it sets `_waiters`
to `None`. So we just check for `self._waiters is None`; if it's
not `None`, we know we have to wait, and do so.

A problem was that the new test introduced in 3.12 explicitly
tested that `wait_closed()` returns immediately when the server
is *not* closed but there are currently no active connections.
This was a mistake (probably a misunderstanding of the intended
semantics). I've fixed the test, and added a separate test that
checks exactly for this scenario.

I also fixed an oddity where in `_wakeup()` the result of the
waiter was set to the waiter itself. This result is not used
anywhere and I changed this to `None`, to avoid a GC cycle.

* Update Lib/asyncio/base_events.py

---------

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
2023-10-28 18:04:29 +00:00
Guido van Rossum c3bb10c930
gh-110205: Fix asyncio ThreadedChildWatcher._join_threads() (#110884)
- `ThreadedChildWatcher.close()` is now *officially* a no-op; `_join_threads()` never did anything.
- Threads created by that class are now named `asyncio-waitpid-NNN`.
- `test.test_asyncio.utils.TestCase.close_loop()` now waits for the child watcher's threads, but not forever; if a thread hangs, it raises `RuntimeError`.
2023-10-27 15:44:12 -07:00
Serhiy Storchaka 6c23635f2b
gh-111085: Fix invalid state handling in TaskGroup and Timeout (#111111)
asyncio.TaskGroup and asyncio.Timeout classes now raise proper RuntimeError
if they are improperly used.

* When they are used without entering the context manager.
* When they are used after finishing.
* When the context manager is entered more than once (simultaneously or
  sequentially).
* If there is no current task when entering the context manager.

They now remain in a consistent state after an exception is thrown,
so subsequent operations can be performed correctly (if they are allowed).

Co-authored-by: James Hilton-Balfe <gobot1234yt@gmail.com>
2023-10-21 22:18:34 +03:00
Bar Harel 7025844f4c
gh-110961: Fixed asyncio.wait docstring to remove deprecated coroutine reference (#111017)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2023-10-18 04:40:26 +00:00
Thomas Grainger 596589104f
remove redundant call to attach_loop in watcher (#110847) 2023-10-14 10:24:02 +02:00
Russell Keith-Magee a7e2a10a85
gh-110771: Decompose run_forever() into parts (#110773)
Effectively introduce an unstable, private (really: protected) API for subclasses that want to override `run_forever()`.
2023-10-13 14:12:32 +00:00
Thomas Grainger 8c6c14b91b
gh-94597: Add asyncio.EventLoop (#110723)
This is needed to pave the way for deprecating and eventually killing the event loop policy system (which is over-engineered and rarely used).
2023-10-12 07:13:57 -07:00
J. Nick Koston 3ac8e6955f
gh-110733: Micro-optimization in BaseEventLoop._run_once (#110735) 2023-10-11 20:59:27 +00:00
Kristján Valur Jónsson 45cf5b0c69
gh-109955 : Update state transition comments for asyncio.Task (#109910)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-09-27 15:24:10 -07:00