Commit Graph

994 Commits

Author SHA1 Message Date
Justin Turner Arthur c741ad3537
gh-77714: Provide an async iterator version of as_completed (GH-22491)
* as_completed returns object that is both iterator and async iterator
* Existing tests adjusted to test both the old and new style
* New test to ensure iterator can be resumed
* New test to ensure async iterator yields any passed-in Futures as-is

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2024-04-01 20:07:29 +03:00
Malcolm Smith 74c8568d07
gh-71042: Add `platform.android_ver` (#116674) 2024-03-27 17:53:27 +01:00
Erik Soma f11d0d8be8
gh-91227: Ignore ERROR_PORT_UNREACHABLE in proactor recvfrom() (#32011) 2024-03-23 08:39:35 -07:00
Will Childs-Klein c85d84166a
gh-116333: Relax error string text expectations in SSL-related tests (GH-116334)
* Relax error string text expectations in SSL-related tests

As suggested [here][1], this change relaxes the OpenSSL error string
text expectations in a number of tests. This was specifically done in
support of more easily building CPython [AWS-LC][2], but because AWS-LC
is a fork of [BoringSSL][3], it should increase compatibility with that
library as well.

In addition to the error string relaxations, we also add some guards
around the `tls-unique` channel binding being used with TLSv1.3, as that
feature (described in [RFC 6929][4]) is [not defined][5] for TLSv1.3.

[1]: https://discuss.python.org/t/support-building-ssl-and-hashlib-modules-against-aws-lc/44505/4
[2]: https://github.com/aws/aws-lc
[3]: https://github.com/google/boringssl
[4]: https://datatracker.ietf.org/doc/html/rfc5929#section-3
[5]: https://datatracker.ietf.org/doc/html/rfc8446#appendix-C.5
2024-03-21 19:16:36 +00:00
jkriegshauser fc45998007
gh-116773: Ensure overlapped objects on Windows are not deallocated too early by asyncio (GH-116774) 2024-03-20 14:33:28 +00:00
Pierre Ossman (ThinLinc team) 4159644177
gh-113538: Add asycio.Server.{close,abort}_clients (redo) (#116784)
These give applications the option of more forcefully terminating client
connections for asyncio servers. Useful when terminating a service and
there is limited time to wait for clients to finish up their work.

This is a do-over with a test fix for gh-114432, which was reverted.
2024-03-18 13:15:53 -07:00
Guido van Rossum ba13215eb1
gh-113538: Revert "gh-113538: Add asycio.Server.{close,abort}_clients (#114432)" (#116632)
Revert "gh-113538: Add asycio.Server.{close,abort}_clients (#114432)"

Reason: The new test doesn't always pass:
https://github.com/python/cpython/pull/116423#issuecomment-1989425489

This reverts commit 1d0d49a7e8.
2024-03-12 00:31:49 +00:00
Pierre Ossman (ThinLinc team) 1d0d49a7e8
gh-113538: Add asycio.Server.{close,abort}_clients (#114432)
These give applications the option of more forcefully terminating client
connections for asyncio servers. Useful when terminating a service and
there is limited time to wait for clients to finish up their work.
2024-03-11 12:43:30 -07:00
Malcolm Smith 872c0714fc
gh-71052: Change Android's `sys.platform` from "linux" to "android"
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-11 19:25:39 +00:00
Jason Zhang ce0ae1d784
gh-115957: Close coroutine if TaskGroup.create_task() raises an error (#116009) 2024-03-06 12:20:26 -08:00
Nikita Sobolev 990a5f17d0
gh-116112: Fix `ResourceWarning` in `test_asyncio.test_stream` (#116371)
Co-authored-by: @CendioOssman
2024-03-06 01:08:18 +03:00
cui fliter e7ba6e9dbe
chore: fix typos (#116345)
Signed-off-by: cui fliter <imcusg@gmail.com>
2024-03-05 09:05:52 -07:00
Malcolm Smith 41d5391c55
gh-71052: Add test exclusions to support running the test suite on Android (#115918) 2024-02-29 22:32:50 +01:00
Pierre Ossman (ThinLinc team) 5a1559d949
gh-112997: Don't log arguments in asyncio unless debugging (#115667)
Nothing else in Python generally logs the contents of variables, so this
can be very unexpected for developers and could leak sensitive
information in to terminals and log files.
2024-02-27 17:39:08 -08:00
Pierre Ossman (ThinLinc team) a355f60b03
gh-114914: Avoid keeping dead StreamWriter alive (#115661)
In some cases we might cause a StreamWriter to stay alive even when the
application has dropped all references to it. This prevents us from
doing automatical cleanup, and complaining that the StreamWriter wasn't
properly closed.

Fortunately, the extra reference was never actually used for anything so
we can just drop it.
2024-02-27 17:27:44 -08:00
Jamie Phan 53d5e67804
gh-111358: Fix timeout behaviour in BaseEventLoop.shutdown_default_executor (#115622) 2024-02-19 00:01:00 +00:00
Jamie Phan 73e8637002
gh-113812: Allow DatagramTransport.sendto to send empty data (#115199)
Also include the UDP packet header sizes (8 bytes per packet)
in the buffer size reported to the flow control subsystem.
2024-02-16 18:38:07 -08:00
Russell Keith-Magee 391659b3da
gh-114099: Add test exclusions to support running the test suite on iOS (#114889)
Add test annotations required to run the test suite on iOS (PEP 730).

The majority of the change involve annotating tests that use subprocess,
but are skipped on Emscripten/WASI for other reasons, and including
iOS/tvOS/watchOS under the same umbrella as macOS/darwin checks.

`is_apple` and `is_apple_mobile` test helpers have been added to
identify *any* Apple platform, and "any Apple platform except macOS",
respectively.
2024-02-05 01:04:57 +01:00
Travis Howse 94ec2b9c9c
gh-114887 Reject only sockets of type SOCK_STREAM in create_datagram_endpoint() (#114893)
Also improve exception message.

Co-authored-by: Donghee Na <donghee.na92@gmail.com>
2024-02-03 17:14:02 +00:00
Kristján Valur Jónsson 6b53d5fe04
gh-112202: Ensure that condition.notify() succeeds even when racing with Task.cancel() (#112201)
Also did a general cleanup of asyncio locks.py comments and docstrings.
2024-02-03 08:19:37 -08:00
Steve Dower c63c6142f9
gh-114272: Fix or skip tests that fail due to spaces in paths (GH-114451) 2024-01-25 00:38:34 +00:00
Serhiy Storchaka f8a79109d0
gh-109862: Fix test_create_subprocess_with_pidfd when it was run separately (GH-113991) 2024-01-15 09:26:49 +02:00
Alois Klink dce30c9cbc
gh-95649: Document that asyncio contains uvloop code (#107536)
Some of the asyncio SSL changes in GH-31275 [1] were taken from
v0.16.0 of the uvloop project [2]. In order to comply with the MIT
license, we need to just need to document the copyright information.

[1]: https://github.com/python/cpython/pull/31275
[2]: https://github.com/MagicStack/uvloop/tree/v0.16.0
2024-01-12 22:21:13 +00:00
Serhiy Storchaka aef4a1203c
gh-96037: Always insert TimeoutError when exit an expired asyncio.timeout() block (GH-113819)
If other exception was raised during exiting an expired
asyncio.timeout() block, insert TimeoutError in the exception context
just above the CancelledError.
2024-01-10 12:50:31 +02:00
Serhiy Storchaka ab0ad62038
gh-113879: Fix ResourceWarning in test_asyncio.test_server (GH-113881) 2024-01-10 12:38:36 +02:00
Jamie Phan 4826d52338
gh-112182: Replace StopIteration with RuntimeError for future (#113220)
When an `StopIteration` raises into `asyncio.Future`, this will cause
a thread to hang. This commit address this by not raising an exception
and silently transforming the `StopIteration` with a `RuntimeError`,
which the caller can reconstruct from `fut.exception().__cause__`
2024-01-09 21:21:00 -08:00
Kristján Valur Jónsson 52161781a6
GH-111693: Propagate correct asyncio.CancelledError instance out of asyncio.Condition.wait() (#111694)
Also fix a race condition in `asyncio.Semaphore.acquire()` when cancelled.
2024-01-08 11:57:48 -08:00
Guido van Rossum 4681a5271a
gh-113538: Don't error in stream reader protocol callback when task is cancelled (#113690) 2024-01-04 12:20:21 -08:00
Yilei Yang 050783cb37
gh-112559: Avoid unnecessary conversion attempts to enum_klass in signal.py (#113040) 2023-12-23 17:07:52 -08:00
Martijn Pieters 1ff0238594
GH-113214: Fix SSLProto exception handling in SSL-over-SSL scenarios (#113334)
When wrapped, `_SSLProtocolTransport._force_close(exc)` is called just like in the unwrapped scenario `_SelectorTransport._force_close(exc)` or `_ProactorBasePipeTransport._force_close(exc)` would be called, except here the exception needs to be passed through the `SSLProtocol._abort()` method, which didn't accept an exception object.

This commit ensures that this path works, in the same way that the uvloop implementation of SSLProto passes on the exception (on which the current implementation of SSLProto is based).
2023-12-20 15:09:01 -08:00
DPR 18c6929469
gh-112186: Improve test case `test_loop_is_closed_resource_warnings` (#112187) 2023-11-19 03:21:34 +00:00
DPR e0f5127975
gh-109538: Avoid RuntimeError when StreamWriter is deleted with closed loop (#111983)
Issue a ResourceWarning instead.

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-11-15 01:17:51 +00:00
Pierre Ossman (ThinLinc team) 74b868f636
gh-111246: Remove listening Unix socket on close (#111483)
Try to clean up the socket file we create so we don't add unused noise to the file system.
2023-11-08 08:10:10 -08:00
Victor Stinner ac01e2243a
gh-111644: Fix asyncio test_unhandled_exceptions() (#111713)
Fix test_unhandled_exceptions() of test_asyncio.test_streams: break
explicitly a reference cycle.

Fix also StreamTests.tearDown(): the loop must not be closed
explicitly, but using set_event_loop() which takes care of shutting
down the executor with executor.shutdown(wait=True).
BaseEventLoop.close() calls executor.shutdown(wait=False).
2023-11-04 00:47:07 +00:00
Kumar Aditya 229f44d353
GH-110894: Call loop exception handler for exceptions in client_connected_cb (#111601)
Call loop exception handler for exceptions in `client_connected_cb` of `asyncio.start_server` so that applications can handle it.
2023-11-02 07:38:18 +00:00
zcxsythenew fa35b9e89b
gh-111347: Remove wrong assertion in test_sendfile (#111377)
Windows is different.
2023-10-29 08:22:15 -07:00
Guido van Rossum 2655369559
gh-79033: Try to fix asyncio.Server.wait_closed() again (GH-111336)
* Try to fix asyncio.Server.wait_closed() again

I identified the condition that `wait_closed()` is intended
to wait for: the server is closed *and* there are no more
active connections.

When this condition first becomes true, `_wakeup()` is called
(either from `close()` or from `_detach()`) and it sets `_waiters`
to `None`. So we just check for `self._waiters is None`; if it's
not `None`, we know we have to wait, and do so.

A problem was that the new test introduced in 3.12 explicitly
tested that `wait_closed()` returns immediately when the server
is *not* closed but there are currently no active connections.
This was a mistake (probably a misunderstanding of the intended
semantics). I've fixed the test, and added a separate test that
checks exactly for this scenario.

I also fixed an oddity where in `_wakeup()` the result of the
waiter was set to the waiter itself. This result is not used
anywhere and I changed this to `None`, to avoid a GC cycle.

* Update Lib/asyncio/base_events.py

---------

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
2023-10-28 18:04:29 +00:00
Guido van Rossum c3bb10c930
gh-110205: Fix asyncio ThreadedChildWatcher._join_threads() (#110884)
- `ThreadedChildWatcher.close()` is now *officially* a no-op; `_join_threads()` never did anything.
- Threads created by that class are now named `asyncio-waitpid-NNN`.
- `test.test_asyncio.utils.TestCase.close_loop()` now waits for the child watcher's threads, but not forever; if a thread hangs, it raises `RuntimeError`.
2023-10-27 15:44:12 -07:00
Serhiy Storchaka 6c23635f2b
gh-111085: Fix invalid state handling in TaskGroup and Timeout (#111111)
asyncio.TaskGroup and asyncio.Timeout classes now raise proper RuntimeError
if they are improperly used.

* When they are used without entering the context manager.
* When they are used after finishing.
* When the context manager is entered more than once (simultaneously or
  sequentially).
* If there is no current task when entering the context manager.

They now remain in a consistent state after an exception is thrown,
so subsequent operations can be performed correctly (if they are allowed).

Co-authored-by: James Hilton-Balfe <gobot1234yt@gmail.com>
2023-10-21 22:18:34 +03:00
Victor Stinner 9a9fba825f
gh-110695: test_asyncio uses 50 ms for clock resolution (#110952)
Before utils.CLOCK_RES constant was added (20 ms), test_asyncio
already used 50 ms.
2023-10-17 11:34:04 +02:00
Russell Keith-Magee a7e2a10a85
gh-110771: Decompose run_forever() into parts (#110773)
Effectively introduce an unstable, private (really: protected) API for subclasses that want to override `run_forever()`.
2023-10-13 14:12:32 +00:00
Thomas Grainger 8c6c14b91b
gh-94597: Add asyncio.EventLoop (#110723)
This is needed to pave the way for deprecating and eventually killing the event loop policy system (which is over-engineered and rarely used).
2023-10-12 07:13:57 -07:00
Victor Stinner e0c4437793
Add support.MS_WINDOWS constant (#110446) 2023-10-06 02:37:28 +02:00
Victor Stinner 1337765225
gh-110335: asyncio test_unix_events cleans multiprocessing (#110336)
test_unix_events tests using the multiprocessing module now call
multiprocessing.util._cleanup_tests().
2023-10-04 09:19:08 +00:00
Victor Stinner c62b49ecc8
gh-110088: Fix asyncio test_prompt_cancellation() (#110157)
Don't measure the CI performance: don't test the maximum elapsed
time. The check failed on a slow CI.
2023-09-30 22:40:10 +02:00
Victor Stinner db0a258e79
gh-110088, gh-109878: Fix test_asyncio timeouts (#110092)
Fix test_asyncio timeouts: don't measure the maximum duration, a test
should not measure a CI performance. Only measure the minimum
duration when a task has a timeout or delay. Add CLOCK_RES to
test_asyncio.utils.
2023-09-29 11:49:30 +00:00
Victor Stinner f29bc9c9a0
gh-109833: Fix asyncio test_wait_for() (#109834)
Expect the test to be "short" but don't measure the exact performance
of the CI. SHORT_TIMEOUT is about 30 seconds whereas the cancelled
coroutine takes around 1 hour.
2023-09-25 13:27:36 +00:00
Alex Waygood 19601efa36
gh-109653: Remove unused imports in the `Lib/` directory (#109803) 2023-09-24 15:07:23 +01:00
Nikita Sobolev 0d20fc7477
gh-109505: Remove unnecessary `hasattr` checks from `test_asyncio` (#109506) 2023-09-22 21:14:15 -07:00
Victor Stinner cbbdf2c144
gh-109709: Fix asyncio test_stdin_broken_pipe() (#109710)
Replace harcoded sleep of 500 ms with synchronization using a pipe.

Fix also Process._feed_stdin(): catch also BrokenPipeError on
stdin.write(input), not only on stdin.drain().
2023-09-22 15:29:42 +02:00