Commit Graph

152 Commits

Author SHA1 Message Date
Yury Selivanov a9d7e552c7
bpo-32357: Optimize asyncio.iscoroutine() for non-native coroutines (#4915) 2017-12-19 07:18:45 -05:00
Yury Selivanov 1b7c11ff0e
bpo-32348: Optimize asyncio.Future schedule/add/remove callback. (#4907) 2017-12-17 20:19:47 -05:00
Andrew Svetlov 44d1a5912e
bpo-32250: Implement asyncio.current_task() and asyncio.all_tasks() (#4799) 2017-12-16 21:58:38 +02:00
Andrew Svetlov f74ef458ab
bpo-32311: Implement asyncio.create_task() shortcut (#4848)
* Implement functionality
* Add documentation
2017-12-15 07:04:38 +02:00
Yury Selivanov 19a44f63c7
bpo-32327: Convert asyncio functions documented as coroutines to coroutines. (#4872) 2017-12-14 20:53:26 -05:00
Victor Stinner 721e25c653
bpo-32101: Fix tests for PYTHONDEVMODE=1 (#4821)
test_asycio: remove also aio_path which was used when asyncio was
developed outside the stdlib.
2017-12-12 23:15:00 +01:00
Yury Selivanov 3e9751819a
bpo-32273: Move asyncio.test_utils to test.test_asyncio (#4785) 2017-12-11 10:04:40 -05:00
Andrew Svetlov 5f841b5538
bpo-32193: Convert asyncio to async/await usage (#4753)
* Convert asyncio/tasks.py to async/await

* Convert asyncio/queues.py to async/await

* Convert asyncio/test_utils.py to async/await

* Convert asyncio/base_subprocess.py to async/await

* Convert asyncio/subprocess.py to async/await

* Convert asyncio/streams.py to async/await

* Fix comments

* Convert asyncio/locks.py to async/await

* Convert asyncio.sleep to async def

* Add a comment

* Add missing news

* Convert stubs from AbstrctEventLoop to async functions

* Convert subprocess_shell/subprocess_exec

* Convert connect_read_pipe/connect_write_pip to async/await syntax

* Convert create_datagram_endpoint

* Convert create_unix_server/create_unix_connection

* Get rid of old style coroutines in unix_events.py

* Convert selector_events.py to async/await

* Convert wait_closed and create_connection

* Drop redundant line

* Convert base_events.py

* Code cleanup

* Drop redundant comments

* Fix indentation

* Add explicit tests for compatibility between old and new coroutines

* Convert windows event loop to use async/await

* Fix double awaiting of async function

* Convert asyncio/locks.py

* Improve docstring

* Convert tests to async/await

* Convert more tests

* Convert more tests

* Convert more tests

* Convert tests

* Improve test
2017-12-09 00:23:48 +02:00
Victor Stinner 44862df2ee
bpo-32047: -X dev enables asyncio debug mode (#4418)
The new -X dev command line option now also enables asyncio debug
mode.
2017-11-20 07:14:07 -08:00
Antoine Pitrou 921e9432a1
bpo-31970: Reduce performance overhead of asyncio debug mode. (#4314)
* bpo-31970: Reduce performance overhead of asyncio debug mode.
2017-11-07 17:23:29 +01:00
Jelle Zijlstra ac317700ce bpo-30406: Make async and await proper keywords (#1669)
Per PEP 492, 'async' and 'await' should become proper keywords in 3.7.
2017-10-05 23:24:46 -04:00
Victor K 4d07189788 bpo-31556: asyncio.wait_for can cancel futures faster with timeout <= 0 (#3703) 2017-10-05 12:04:39 -04:00
Alexander Mohr de34cbe9cd bpo-31061: fix crash in asyncio speedup module (GH-2966) 2017-08-02 15:31:07 +09:00
Yury Selivanov 7ce1c6fb57 bpo-30508: Don't log exceptions if Task/Future "cancel()" method called (#2050) 2017-06-11 13:49:18 +00:00
INADA Naoki 991adca012 bpo-30048: asyncio: fix Task.cancel() was ignored. (GH-1097)
when there are no more `await` or `yield (from)` before return in coroutine,
cancel was ignored.

example:

    async def coro():
        asyncio.Task.current_task().cancel()
        return 42
    ...
    res = await coro()  # should raise CancelledError
2017-05-11 21:18:38 +09:00
Yury Selivanov 8d26aa930c bpo-29271: Fix Task.current_task and Task.all_tasks to accept None. (#406) 2017-03-02 22:16:33 -05:00
Serhiy Storchaka 70d28a184c Remove unused imports. 2016-12-16 20:00:15 +02:00
Yury Selivanov edfe8869c8 Merge 3.6 (issue #28843) 2016-12-01 11:37:47 -05:00
Yury Selivanov 27182bb232 Merge 3.5 (issue #28703) 2016-11-15 15:21:07 -05:00
Yury Selivanov 0ed20cdfb7 Issue #28703: Fix asyncio.iscoroutinefunction to handle Mock objects. 2016-11-15 15:20:34 -05:00
Yury Selivanov 69312fa4a2 Merge 3.5 (issue #28613) 2016-11-04 14:30:11 -04:00
Yury Selivanov 600a349781 Issue #28613: Fix get_event_loop() to return the current loop
when called from coroutines or callbacks.
2016-11-04 14:29:28 -04:00
Yury Selivanov a0c1ba608e Issue #28544: Implement asyncio.Task in C.
This implementation provides additional 10-20% speed boost for
asyncio programs.

The patch also fixes _asynciomodule.c to use Arguments Clinic, and
makes '_schedule_callbacks' an overridable method (as it was in 3.5).
2016-10-28 12:52:37 -04:00
Yury Selivanov 3d67615a48 Issue #26923: Fix asyncio.Gather to refuse being cancelled once all children are done.
Patch by Johannes Ebke.
2016-10-21 17:22:17 -04:00
Yury Selivanov 4145c83806 Issue #27972: Prohibit Tasks to await on themselves. 2016-10-09 12:19:12 -04:00
Guido van Rossum e3c65a7a22 Misc asyncio improvements from upstream 2016-09-30 08:17:15 -07:00
Yury Selivanov c1cf296de6 asyncio: Remove asyncio.timeout() context manager.
It will probably be added back in Python 3.6, once its compatibility
issues are resolved; see [1] for more details.

[1] https://mail.python.org/pipermail/async-sig/2016-June/000045.html
2016-06-08 13:57:03 -04:00
Victor Stinner c1b578608e Fix test_asyncio.test_timeout_disable()
Issue #26777: Fix random failing of the test on the "AMD64 FreeBSD 9.x 3.5"
buildbot:

    File ".../Lib/test/test_asyncio/test_tasks.py", line 2398, in go
      self.assertTrue(0.09 < dt < 0.11, dt)
    AssertionError: False is not true : 0.11902812402695417

Replace "< 0.11" with "< 0.15".
2016-04-18 10:28:42 +02:00
Victor Stinner 2ba8ece5be asyncio: allow None as wait timeout
Fix GH#325: Allow to pass None as a timeout value to disable timeout logic.

Change written by Andrew Svetlov and merged by Guido van Rossum.
2016-04-01 21:39:09 +02:00
Yury Selivanov dce63234c5 asyncio: Fix @coroutine to recognize CoroWrapper (issue #25647)
Patch by Vladimir Rutsky.
2016-03-02 10:49:16 -05:00
Victor Stinner 0df5313458 test_asyncio: fix test_timeout_time()
Accept time delta up to 0.12 second, instead of 0.11, for the "AMD64 FreeBSD
9.x" buildbot slave.
2016-02-01 12:39:50 +01:00
Andrew Svetlov c07b16b40f Sync with asyncio repo 2016-01-11 08:42:49 +02:00
Yury Selivanov ea4ce1c776 asyncio/tests: Fix a ResourceWarning due to unclosed loop 2015-12-16 20:41:11 -05:00
Yury Selivanov 0ac3a0cd79 asyncio: Make Tasks check if Futures are attached to the same event loop
See https://github.com/python/asyncio/pull/303 for details
2015-12-11 11:33:59 -05:00
Yury Selivanov abe9625eeb asyncio: Fix sporadic failing unittests in debug mode 2015-11-13 12:28:48 -05:00
Yury Selivanov ade0412613 asyncio: Optimize asyncio.sleep(0) 2015-11-05 14:29:04 -05:00
Guido van Rossum 601953b679 Docs and one small improvement for issue #25304, by Vincent Michel. 2015-10-05 16:20:00 -07:00
Guido van Rossum 841d9ee41a Issue #25304: Add asyncio.run_coroutine_threadsafe(). By Vincent Michel. 2015-10-03 08:31:42 -07:00
Yury Selivanov 620279b9ac asyncio: ensure_future() now understands awaitables 2015-10-02 15:00:19 -04:00
Yury Selivanov 233983380d Issue #24867: Fix Task.get_stack() for 'async def' coroutines 2015-08-14 15:30:59 -04:00
Yury Selivanov 29a602a140 Issue #24400: Fix CoroWrapper for 'async def' coroutines 2015-06-24 10:30:14 -04:00
Yury Selivanov 1af2bf75a2 asyncio: Support PEP 492. Issue #24017. 2015-05-11 22:27:25 -04:00
Yury Selivanov 59eb9a4da5 asyncio: async() function is deprecated in favour of ensure_future(). 2015-05-11 14:48:38 -04:00
Yury Selivanov 740169cd24 Sync asyncio changes from the main repo. 2015-05-11 14:23:38 -04:00
Guido van Rossum 0a9933ebf3 Asyncio issue 222 / PR 231 (Victor Stinner) -- fix @coroutine functions without __name__. 2015-05-02 18:38:24 -07:00
Victor Stinner 922bc2ca12 Closes #23219: cancelling asyncio.wait_for() now cancels the task 2015-01-15 16:29:10 +01:00
Victor Stinner 15cc678d89 asyncio: Truncate to 80 columns 2015-01-09 00:09:10 +01:00
Victor Stinner d7ff5a5375 asyncio: sync with Tulip
* Fix pyflakes warnings: remove unused imports and variables
* asyncio.test_support now uses test.support and test.script_helper if available
2014-12-26 21:16:42 +01:00
Victor Stinner dc7765d12c asyncio: sync with Tulip 2014-12-18 12:29:53 +01:00
Yury Selivanov d5a8f5807f asyncio.test_tasks: Fix test_env_var_debug to use correct asyncio module 2014-09-25 19:12:10 -04:00
Victor Stinner 59e0802301 asyncio, Tulip issue 201: Fix a race condition in wait_for()
Don't raise a TimeoutError if we reached the timeout and the future completed
in the same iteration of the event loop. A side effect of the bug is that
Queue.get() looses items.
2014-08-28 11:19:25 +02:00
Victor Stinner 313a980904 asyncio: sync with Tulip
* _WaitHandleFuture.cancel() now notify IocpProactor through the overlapped
  object that the wait was cancelled.
* Optimize IocpProactor.wait_for_handle() gets the result if the wait is
  signaled immediatly.
* Enhance representation of Future and Future subclasses

  - Add "created at filename:lineno" in the representation
  - Add Future._repr_info() method which can be more easily overriden than
    Future.__repr__(). It should now be more easy to enhance Future
    representation without having to modify each subclass. For example,
    _OverlappedFuture and _WaitHandleFuture get the new "created at" information.
  - Use reprlib to format Future result, and function arguments when formatting a
    callback, to limit the length of the representation.

* Fix repr(_WaitHandleFuture)
* _WaitHandleFuture and _OverlappedFuture: hide frames of internal calls in the
  source traceback.
* Cleanup ProactorIocp._poll(): set the timeout to 0 after the first call to
  GetQueuedCompletionStatus()
* test_locks: close the temporary event loop and check the condition lock
* Remove workaround in test_futures, no more needed
2014-07-29 12:58:23 +02:00
Victor Stinner accef5ce36 asyncio: test_as_completed(): disable "slow callback" warning 2014-07-16 18:54:13 +02:00
Victor Stinner e931f7b768 Issue #21163: Fix "destroy pending task" warning in test_wait_errors() 2014-07-16 18:50:39 +02:00
Victor Stinner 770e48d017 asyncio: sync with Tulip
* Tulip issue #182: Improve logs of BaseEventLoop._run_once()

  - Don't log non-blocking poll
  - Only log polling with a timeout if it gets events or if it timed out after
    more than 1 second.

* Fix some pyflakes warnings: remove unused imports
2014-07-11 11:58:33 +02:00
Victor Stinner 737c34fa85 asyncio: sync with Tulip
- CoroWrapper.__del__() now reuses repr(CoroWrapper) to log the "... was never
  yielded from" warning
- Improve CoroWrapper: copy also the qualified name on Python 3.4, not only on
  Python 3.5+
2014-07-11 01:04:16 +02:00
Victor Stinner c39ba7d611 asyncio: sync with Tulip
- repr(Task) and repr(CoroWrapper) now also includes where these objects were
  created. If the coroutine is not a generator (don't use "yield from"), use
  the location of the function, not the location of the coro() wrapper.
- Fix create_task(): truncate the traceback to hide the call to create_task().
2014-07-11 00:21:27 +02:00
Victor Stinner 896a25ab30 asyncio: sync with Tulip
- Tulip issue 185: Add a create_task() method to event loops. The create_task()
  method can be overriden in custom event loop to implement their own task
  class. For example, greenio and Pulsar projects use their own task class. The
  create_task() method is now preferred over creating directly task using the
  Task class.
- tests: fix a warning
- fix typo in the name of a test function
- Update AbstractEventLoop: add new event loop methods; update also the unit test
2014-07-08 11:29:25 +02:00
Victor Stinner a9acbe82e7 Closes #21886, #21447: Fix a race condition in asyncio when setting the result
of a Future with call_soon(). Add an helper, a private method, to set the
result only if the future was not cancelled.
2014-07-05 15:29:41 +02:00
Victor Stinner 2dba23af71 asyncio: sync with Tulip
* _UnixSubprocessTransport: fix file mode of stdin. Open stdin in write mode,
  not in read mode
* Examples: close the event loop at exit
* More reliable CoroWrapper.__del__. If the constructor is interrupted by
  KeyboardInterrupt or the coroutine objet is destroyed lately, some the
  _source_traceback attribute doesn't exist anymore.
* repr(Task): include also the future the task is waiting for
2014-07-03 00:59:00 +02:00
Victor Stinner 1690ed397a asyncio: Fix test_sleep_cancel(): call_later() mock has no self parameter 2014-07-01 12:38:51 +02:00
Victor Stinner b75380f333 asyncio: sync with Tulip
- Sort imports
- Simplify/optimize iscoroutine(). Inline inspect.isgenerator(obj): replace it
  with isinstance(obj, types.GeneratorType)
- CoroWrapper: check at runtime if Python has the yield-from bug #21209.  If
  Python has the bug, check if CoroWrapper.send() was called by yield-from to
  decide if parameters must be unpacked or not.
- Fix "Task was destroyed but it is pending!" warning in
  test_task_source_traceback()
2014-06-30 14:39:11 +02:00
Victor Stinner f951d28ac8 asyncio: sync with Tulip, add a new asyncio.coroutines module 2014-06-29 00:46:45 +02:00
Victor Stinner 09e7590801 asyncio: Fix two "Coroutine xxx was never yielded from" messages in tests 2014-06-28 01:19:11 +02:00
Victor Stinner b9a301a348 asyncio: Fix unit tests on Windows, escape filenames in regex 2014-06-28 00:12:02 +02:00
Victor Stinner 80f53aa9a0 asyncio, Tulip issue 137: In debug mode, save traceback where Future, Task and
Handle objects are created. Pass the traceback to call_exception_handler() in
the 'source_traceback' key.

The traceback is truncated to hide internal calls in asyncio, show only the
traceback from user code.

Add tests for the new source_traceback, and a test for the 'Future/Task
exception was never retrieved' log.
2014-06-27 13:52:20 +02:00
Victor Stinner bbd96c6f47 asyncio, Tulip issue 137: In debug mode, add the traceback where the coroutine
object was created to the "coroutine ... was never yield from" log
2014-06-27 12:28:41 +02:00
Victor Stinner c4cca45baf asyncio: Oops, restore a removed test 2014-06-27 12:23:41 +02:00
Victor Stinner 72502c12a7 Issue #21163: Fix one more "Task was destroyed but it is pending!" log in tests 2014-06-25 23:57:50 +02:00
Victor Stinner 7ba4061098 asyncio: sync with Tulip
- Python issue 21163: Fix more "Task was destroyed but it is pending!" logs in
  tests
- Add test to check that run_until_complete() checks the loop of the future
2014-06-25 23:32:25 +02:00
Victor Stinner 87f16f892c Issue #21163, asyncio: Fix some "Task was destroyed but it is pending!" logs in tests 2014-06-25 23:11:21 +02:00
Victor Stinner 975735f729 asyncio, Tulip issue 177: Rewite repr() of Future, Task, Handle and TimerHandle
- Uniformize repr() output to format "<Class ...>"
- On Python 3.5+, repr(Task) uses the qualified name instead of the short name
  of the coroutine
2014-06-25 21:41:58 +02:00
Victor Stinner df29c4a83d asyncio: repr(Task) now also contains the line number even if the coroutine is
done: use the first line number of the code object instead of the current line
number of the generator frame.

The name of the coroutine is not enough because many coroutines may have the
same name. It's a common case in asyncio tests for example.
2014-06-24 22:57:14 +02:00
Victor Stinner a02f81ff17 asyncio: Log an error if a Task is destroyed while it is still pending 2014-06-24 22:37:53 +02:00
Victor Stinner c082ee692b asyncio: Add an unit test to check that setting the PYTHONASYNCIODEBUG env var
enables debug mode of the event loop.
2014-06-23 00:19:33 +02:00
Victor Stinner 7b7120e159 asyncio: Enable the debug mode of event loops when the PYTHONASYNCIODEBUG
environment variable is set
2014-06-23 00:12:14 +02:00
Victor Stinner 64d750b36b asyncio: Fix pyflakes errors
- Add a missing import
- Remove an unused import
- Remove unused variables
2014-06-18 03:25:23 +02:00
Victor Stinner c73701de72 asyncio: Refactor tests: add a base TestCase class 2014-06-18 01:36:32 +02:00
Victor Stinner 8d3e02ef5a asyncio: Set __qualname__ attribute of CoroWrapper in @coroutine decorator on
Python 3.5

- Drop __slots__ optimization of CoroWrapper to be able to set the __qualname__
  attribute.
- Add tests on __name__, __qualname__ and __module__ of a coroutine function
  and coroutine object.
- Fix test_tasks when run in debug mode (PYTHONASYNCIODEBUG env var set) on
  Python 3.3 or 3.4
2014-06-18 01:14:59 +02:00
Victor Stinner 4802c6ee55 Sync asyncio with Tulip: Fix test_tasks for Python 3.5
On Python 3.5, generator now gets their name from the function, no more from
the code. So we get the expected "notmuch" name instead of the generic "coro"
name.
2014-06-16 17:11:05 +02:00
Victor Stinner 307bccc6ff asyncio: Tulip issue 173: Enhance repr(Handle) and repr(Task)
repr(Handle) is shorter for function: "foo" instead of "<function foo at
0x...>". It now also includes the source of the callback, filename and line
number where it was defined, if available.

repr(Task) now also includes the current position in the code, filename and
line number, if available. If the coroutine (generator) is done, the line
number is omitted and "done" is added.
2014-06-12 18:39:26 +02:00
Guido van Rossum 94ba146d11 asyncio: Add __weakref__ slots to Handle and CoroWrapper. Upstream issue #166. 2014-04-27 10:44:22 -07:00
Guido van Rossum 0cbc76880f asyncio: Add gi_{frame,running,code} properties to CoroWrapper (upstream #163). 2014-04-15 12:06:34 -07:00
Yury Selivanov 09cc169a03 asyncio.tasks: Make sure CoroWrapper.send proxies one argument correctly
Issue #21209.
2014-04-15 12:01:16 -04:00
Yury Selivanov f15f7484bb asyncio.tasks: Fix CoroWrapper to workaround yield-from bug in CPython < 3.4.1
Closes issue #21209.
2014-04-14 22:21:52 -04:00
Victor Stinner f5e37037cc asyncio: Fix pyflakes warnings: remove unused variables and imports 2014-02-26 11:07:42 +01:00
Victor Stinner 7ef60cd8c2 asyncio, Tulip issue #136: Add get/set_debug() methods to BaseEventLoopTests.
Add also a PYTHONASYNCIODEBUG environment variable to debug coroutines since
Python startup, to be able to debug coroutines defined directly in the asyncio
module.
2014-02-19 23:15:02 +01:00
Yury Selivanov b41a42e316 asyncio: pep8-ify the code. 2014-02-18 22:56:15 -05:00
Guido van Rossum b58f053e48 asyncio: Change as_completed() to use a Queue, to avoid O(N**2) behavior. Fixes issue #20566. 2014-02-12 17:58:19 -08:00
Victor Stinner eb74876e99 asyncio, Tulip issue 131: as_completed() and wait() now raises a TypeError if
the list of futures is not a list but a Future, Task or coroutine object
2014-02-11 11:54:08 +01:00
Victor Stinner a125497ea3 asyncio, Tulip issue 126: call_soon(), call_soon_threadsafe(), call_later(),
call_at() and run_in_executor() now raise a TypeError if the callback is a
coroutine function.
2014-02-11 11:34:30 +01:00
Guido van Rossum 313f829ce8 asyncio: Test fix. 2014-02-08 17:35:09 -08:00
Yury Selivanov 622be340fd asyncio.tasks: Fix as_completed, gather & wait to work with duplicate coroutines 2014-02-06 22:06:16 -05:00
Guido van Rossum a849be9c64 asyncio: Fix misc whitespace issues. 2014-01-30 16:05:28 -08:00
Guido van Rossum 48c66c3dd8 asyncio: wait_for() now accepts None as timeout (Victor Stinner). 2014-01-29 14:30:38 -08:00
Victor Stinner 8dffc456d7 Update asyncio from the Tulip project
Major changes:

- StreamReader.readexactly() now raises an IncompleteReadError if the
  end of stream is reached before we received enough bytes, instead of
  returning less bytes than requested.

- Unit tests use the main asyncio module instead of submodules like events

- _UnixWritePipeTransport now also supports character devices, as
  _UnixReadPipeTransport. Patch written by Jonathan Slenders.

- Export more symbols: BaseEventLoop, BaseProactorEventLoop,
  BaseSelectorEventLoop, Queue and Queue sublasses, Empty, Full
2014-01-25 15:32:06 +01:00
Victor Stinner 421e49b5c1 asyncio: wait_for() now cancels the future on timeout. Patch written by Gustavo
Carneiro.
2014-01-23 17:40:59 +01:00
Guido van Rossum 9ba75db3c5 asyncio: Clean up formatting. 2013-12-19 13:49:32 -08:00
Victor Stinner 4c3c699e62 Close #19967: Thanks to the PEP 442, asyncio.Future can use a destructor in
Python 3.4 to log "uncatched" exceptions, instead of the dedicated
_TracebackLogger class.
2013-12-19 22:42:40 +01:00
Guido van Rossum 1a605ed5a3 asyncio: Add Task.current_task() class method. 2013-12-06 12:57:40 -08:00