Commit Graph

811 Commits

Author SHA1 Message Date
Richard Kojedzinszky 17ef4319a3
bpo-41891: ensure asyncio.wait_for waits for task completion (#22461) 2020-12-18 09:26:04 -08:00
Chris Jerdonek 66d3b589c4
bpo-38323: Add guard clauses in MultiLoopChildWatcher. (#22756)
This is a trivial refactor in preparation for a fix for bpo-38323.
2020-12-16 19:50:25 +02:00
Victor Stinner 99d28c5670
bpo-40364: asyncio uses os.waitstatus_to_exitcode() (GH-23798)
test_unix_events.py no longer checks if waitstatus_to_exitcode() mock
has been called or not to make the test more functional, rather than
checking the exact implementation.
2020-12-16 12:11:24 +01:00
Serhiy Storchaka 8a62887dfb
bpo-42582: Remove asyncio._all_tasks_compat(). (GH-23664)
It was used to implement now removed asyncio.Task.all_tasks().
2020-12-06 18:54:33 +02:00
Zackery Spytz 8085f742f4
bpo-34215: Clarify IncompleteReadError message when "expected" is None (GH-21925)
Co-Authored-By: Tyler Bell <mrbell321@gmail.com>
2020-11-28 16:27:28 +02:00
Wansoo Kim 5b0194ed31
bpo-41241: Unnecessary Type casting in 'if condition' (GH-21396)
This is my first issue!
So, if there's anything wrong, please tell me!

Also, thank you always for all the contributors!

Automerge-Triggered-By: GH:asvetlov
2020-11-28 03:37:08 -08:00
Yurii Karabas e4fe303b8c
bpo-42392: Remove loop parameter from asyncio.tasks and asyncio.subprocess (GH-23521) 2020-11-28 10:21:17 +02:00
Alex Grönholm e3ef4d7f65
bpo-41332: Added missing connect_accepted_socket() to AbstractEventLoop (GH-21533)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-11-26 12:09:12 +02:00
Yurii Karabas f533cb80cb
bpo-42392: Remove loop parameter from asyncio.streams (GH-23517) 2020-11-26 09:36:37 +02:00
Yurii Karabas b9127dd6ee
bpo-42392: Improve removal of *loop* parameter in asyncio primitives (GH-23499)
* Update code after merge review from 1st1

* Use a sentinel approach for loop parameter
Remove unnecessary _get_running_loop patching

* Use more clear function name (_verify_parameter_is_marker -> _verify_no_loop)

* Add init method to _LoopBoundMixin to check that loop param wasn't used
2020-11-25 06:50:44 -05:00
Yurii Karabas 0ec34cab9d
bpo-42392: Remove loop parameter form asyncio locks and Queue (#23420)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2020-11-24 20:08:54 +02:00
Diogo Dutra 7e5ef0a571
bpo-42140: Improve asyncio.wait function (GH-22938)
# Improve asyncio.wait function

The original code creates the futures set two times.
We can create this set before, avoiding the second creation.

This new behaviour [breaks the aiokafka library](https://github.com/aio-libs/aiokafka/pull/672), because it gives an iterator to that function, so the second iteration become empty.

Automerge-Triggered-By: GH:1st1
2020-11-10 14:12:52 -08:00
Andrew Svetlov 42d873c63a
bpo-42183: Fix a stack overflow error for asyncio Task or Future repr() (GH-23020)
The overflow occurs under some circumstances when a task or future
recursively returns itself.

Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-11-10 15:58:31 +02:00
Jakub Stasiak 3d86d090dc
bpo-42230: Improve asyncio documentation regarding accepting sets vs iterables (GH-23073)
People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] https://github.com/aio-libs/aiokafka/pull/672
[2] 620989bac5/stdlib/3/asyncio/tasks.pyi (L161)
[3] 620989bac5/stdlib/3/asyncio/tasks.pyi (L40)
2020-11-02 12:56:35 +02:00
Victor Stinner 1b0f0e3d7d
bpo-39651: Fix asyncio proactor _write_to_self() (GH-22197)
Fix a race condition in the call_soon_threadsafe() method of
asyncio.ProactorEventLoop: do nothing if the self-pipe socket has
been closed.
2020-09-12 08:50:18 +02:00
Shantanu 0770ad948c
bpo-41696: Fix handling of debug mode in asyncio.run (#22069)
* bpo-41696: Fix handling of debug mode in asyncio.run

This allows PYTHONASYNCIODEBUG or -X dev to enable asyncio debug mode
when using asyncio.run

* 📜🤖 Added by blurb_it.

Co-authored-by: hauntsaninja <>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-09-02 21:54:46 -07:00
Ben Darnell ea5a6363c3
bpo-39010: Fix errors logged on proactor loop restart (#22017)
Stopping and restarting a proactor event loop on windows can lead to
spurious errors logged (ConnectionResetError while reading from the
self pipe). This fixes the issue by ensuring that we don't attempt
to start multiple copies of the self-pipe reading loop.
2020-08-31 12:57:52 -07:00
Elvis Pranskevichus a2118a1462
bpo-37658: Fix asyncio.wait_for() to respect waited task status (#21894)
Currently, if `asyncio.wait_for()` itself is cancelled it will always
raise `CancelledError` regardless if the underlying task is still
running.  This is similar to a race with the timeout, which is handled
already.
2020-08-26 09:42:45 -07:00
Elvis Pranskevichus c517fc7121
bpo-32751: Wait for task cancel in asyncio.wait_for() when timeout <= 0 (#21895)
When I was fixing bpo-32751 back in GH-7216 I missed the case when
*timeout* is zero or negative.  This takes care of that.

Props to @aaliddell for noticing the inconsistency.
2020-08-26 09:42:22 -07:00
Cleber Rosa 1afb42cfa8
bpo-41572: Fix grammar in BaseTransport.close docstring (GH-21914)
Fix grammar in BaseTransport.close docstring.

https://bugs.python.org/issue41572

Signed-off-by: Cleber Rosa <crosa@redhat.com>
2020-08-20 18:10:01 +05:30
James Weaver 29f84294d8
bpo-40782: Change asyncio.AbstractEventLoop.run_in_executor to be a method not a coroutine (GH-21852)
asyncio.AbstractEventLoop.run_in_executor should be a method that returns an asyncio Future, not an async method.
This matches the concrete implementations, and the documentation better.
2020-08-17 07:19:46 -07:00
Victor Stinner 602a971a2a
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
On Windows, fix asyncio recv_into() return value when the socket/pipe
is closed (BrokenPipeError): return 0 rather than an empty byte
string (b'').
2020-08-04 02:40:10 +02:00
Alex Grönholm 0dd98c2d00
bpo-41317: Remove reader on cancellation in asyncio.loop.sock_accept() (#21595) 2020-07-23 12:45:08 -07:00
Vinay Sharma d42528a3a2
bpo-37703: improve asyncio.gather documentation regarding cancellation (GH-15312)
These changes updates the doc to comprehensively mention the behaviour of gather.cancel()

Automerge-Triggered-By: @asvetlov
2020-07-20 01:42:57 -07:00
Tony Solomonik 568fb0ff4a
bpo-41273: asyncio's proactor read transport's better performance by using recv_into instead of recv (#21442)
* bpo-41273: Proactor transport read loop to use recv_into

By using recv_into instead of recv we do not allocate a new buffer each
time _loop_reading calls recv.

This betters performance for any stream using proactor (basically any
asyncio stream on windows).

* bpo-41273: Double proactor read transport buffer size

By doubling the read buffer size we get better performance.
2020-07-14 12:41:24 -07:00
Rémi Lapeyre 004e64e805
bpo-40967: Remove deprecated asyncio.Task.current_task() and asyncio.Task.all_tasks() (GH-20874) 2020-07-01 20:41:21 -07:00
Kyle Stanley 2b201369b4
Fix asyncio.to_thread() documented return type (GH-20547)
When I wrote the documentation for `asyncio.to_thread()`, I mistakenly assumed that `return await loop.run_in_executor(...)` within an async def function would return a Future. In reality, it returns a coroutine.

This likely won't affect typical usage of `asyncio.to_thread()`, but it's important for the documentation to be correct here. In general, we also tend to avoid returning futures from high-level APIs in asyncio.
2020-05-31 00:07:04 -07:00
Fantix King 210a137396
bpo-30064: Fix asyncio loop.sock_* race condition issue (#20369) 2020-05-27 12:47:30 -07:00
Kyle Stanley 0f56263e62
bpo-32309: Add support for contextvars in asyncio.to_thread() (GH-20278)
Allows contextvars from the main thread to be accessed in the separate thread used in `asyncio.to_thread()`. See the [discussion](https://github.com/python/cpython/pull/20143#discussion_r427808225) in GH-20143 for context.

Automerge-Triggered-By: @aeros
2020-05-20 22:20:43 -07:00
Kunal Bhalla f2947e354c
s/wakup/wakeup (GH-20250)
(as title)

Automerge-Triggered-By: @Mariatta
2020-05-20 10:12:37 -07:00
Kyle Stanley cc2bbc2227
bpo-32309: Implement asyncio.to_thread() (GH-20143)
Implements `asyncio.to_thread`, a coroutine for asynchronously running IO-bound functions in a separate thread without blocking the event loop. See the discussion starting from [here](https://github.com/python/cpython/pull/18410#issuecomment-628930973) in GH-18410 for context.

Automerge-Triggered-By: @aeros
2020-05-18 20:03:28 -07:00
Chris Jerdonek da742ba826
bpo-31033: Improve the traceback for cancelled asyncio tasks (GH-19951)
When an asyncio.Task is cancelled, the exception traceback now
starts with where the task was first interrupted.  Previously,
the traceback only had "depth one."
2020-05-17 22:47:31 -07:00
Chris Jerdonek 1ce5841eca
bpo-31033: Add a msg argument to Future.cancel() and Task.cancel() (GH-19979) 2020-05-15 16:55:50 -07:00
romasku 382a5635bd
bpo-40607: Reraise exception during task cancelation in asyncio.wait_for() (GH-20054)
Currently, if asyncio.wait_for() timeout expires, it cancels
inner future and then always raises TimeoutError. In case
those future is task, it can handle cancelation mannually,
and those process can lead to some other exception. Current
implementation silently loses thoses exception.

To resolve this, wait_for will check was the cancelation
successfull or not. In case there was exception, wait_for
will reraise it.

Co-authored-by: Roman Skurikhin <roman.skurikhin@cruxlab.com>
2020-05-15 13:12:05 -07:00
Victor Stinner 90549676e0
bpo-40443: Remove unused imports in the stdlib (GH-19803) 2020-04-30 11:26:33 +02:00
Galden 02152b7332
Fix typo from Lib/asyncio/events.py (GH-19410) 2020-04-14 09:04:32 +09:00
Lidi Zheng 4482337dec
bpo-39764: Make Task.get_stack accept ag_frame (#18669)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-03-02 14:45:54 +02:00
Markus Mohrhard 374d998b50
bpo-39609: set the thread_name_prefix for the default asyncio executor (GH-18458)
Just a small debugging improvement to identify the asyncio executor threads.
2020-02-27 12:01:47 -08:00
Andrew Svetlov 90d9ba6ef1
bpo-34793: Drop old-style context managers in asyncio.locks (GH-17533) 2020-02-01 13:12:52 +02:00
Kyle Stanley 0ca7cc7fc0 bpo-38356: Fix ThreadedChildWatcher thread leak in test_asyncio (GH-16552)
Motivation for this PR (comment from @vstinner in bpo issue):
```
Warning seen o AMD64 Ubuntu Shared 3.x buildbot:
https://buildbot.python.org/all/#/builders/141/builds/2593

test_devnull_output (test.test_a=syncio.test_subprocess.SubprocessThreadedWatcherTests) ...
Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)
```
The following implementation details for the new method are TBD:

1) Public vs private

2) Inclusion in `close()`

3) Name

4) Coroutine vs subroutine method

5) *timeout* parameter

If it's a private method, 3, 4, and 5 are significantly less important.

I started with the most minimal implementation that fixes the dangling threads without modifying the regression tests, which I think is particularly important. I typically try to avoid directly modifying existing tests as much as possible unless it's necessary to do so. However, I am open to changing any part of this.


https://bugs.python.org/issue38356
2020-01-12 03:02:50 -08:00
Andrew Svetlov 10ac0cded2 bpo-39191: Fix RuntimeWarning in asyncio test (GH-17863)
https://bugs.python.org/issue39191
2020-01-07 05:23:01 -08:00
Andrew Svetlov 3a5de51159
Fix #39191: Don't spawn a task before failing (#17796) 2020-01-04 11:10:14 +02:00
Kyle Stanley 89aa7f0ede bpo-34790: Implement deprecation of passing coroutines to asyncio.wait() (GH-16977) 2019-12-30 13:50:19 +02:00
Andrew Svetlov 025eeaa196
Fix import path for asyncio.TimeoutError (#17691) 2019-12-24 12:46:42 +02:00
Kyle Stanley ab513a38c9 bpo-37228: Fix loop.create_datagram_endpoint()'s usage of SO_REUSEADDR (#17311) 2019-12-09 15:21:10 +01:00
Victor Stinner 82b4950b5e
bpo-39006: Fix asyncio when the ssl module is missing (GH-17524)
Fix asyncio when the ssl module is missing: only check for
ssl.SSLSocket instance if the ssl module is available.
2019-12-09 15:02:03 +01:00
idomic 892f9e0777 bpo-37404: Raising value error if an SSLSocket is passed to asyncio functions (GH-16457)
https://bugs.python.org/issue37404
2019-12-07 03:52:35 -08:00
Andrew Svetlov 7ddcd0caa4
bpo-38529: Fix asyncio stream warning (GH-17474) 2019-12-07 13:22:00 +02:00
Batuhan Taşkaya dec367261e bpo-38978: Implement __class_getitem__ for asyncio objects (GH-17491)
https://bugs.python.org/issue38978
2019-12-07 03:05:07 -08:00
Victor Stinner e76ee1a72b
bpo-38982: Fix asyncio PidfdChildWatcher on waitpid() error (GH-17477)
If waitpid() is called elsewhere, waitpid() call fails with
ChildProcessError: use return code 255 in this case, and log a
warning. It ensure that the pidfd file descriptor is closed if this
error occurs.
2019-12-06 16:32:41 +01:00