Commit Graph

552 Commits

Author SHA1 Message Date
Victor Stinner 71080fc351 asyncio: Add asyncio.compat module
Move compatibility helpers for the different Python versions to a new
asyncio.compat module.
2015-07-25 02:23:21 +02:00
Victor Stinner e6ecea53c8 asyncio: sync with github asyncio
* queues: get coroutine from asyncio.coroutines, not from asyncio.tasks
* tets: replace tulip with asyncio in comments
2015-07-09 23:13:50 +02:00
Yury Selivanov 09e6058838 Issue #24450: Proxy gi_yieldfrom & cr_await in asyncio.CoroWrapper 2015-07-03 00:41:16 -04:00
Yury Selivanov 339d5e7d85 Fix asyncio unittests in debug mode 2015-06-24 10:45:44 -04:00
Yury Selivanov 29a602a140 Issue #24400: Fix CoroWrapper for 'async def' coroutines 2015-06-24 10:30:14 -04:00
Yury Selivanov dfbd27f0be asyncio: Merge changes from issue #24400. 2015-06-24 09:41:35 -04:00
Yury Selivanov 70adad2a0e Issue 24017: Fix asyncio.CoroWrapper to support 'async def' coroutines 2015-05-31 21:37:09 -04:00
Yury Selivanov a316085192 Issue 24004: Support Awaitables (pep 492) in @asyncio.coroutine decorator 2015-05-30 21:02:12 -04:00
Yury Selivanov 1ad08a5555 asyncio: Drop some useless code from tasks.py.
See also issue 24017.
2015-05-28 10:52:19 -04:00
Yury Selivanov c58cca5951 asyncio: Use 'collections.abc.Coroutine' in asyncio.iscoroutine (in 3.5) 2015-05-13 15:21:41 -04:00
Yury Selivanov d08c363c26 Sync asyncio code from default branch. 2015-05-13 15:15:56 -04:00
Yury Selivanov e8944cb217 asyncio: Make sure sys.set_coroutine_wrapper is called *only* when loop is running.
Previous approach of installing coroutine wrapper in loop.set_debug() and
uninstalling it in loop.close() was very fragile.  Most of asyncio tests
do not call loop.close() at all.  Since coroutine wrapper is a global
setting, we have to make sure that it's only set when the loop is
running, and is automatically unset when it stops running.

Issue #24017.
2015-05-12 11:43:04 -04:00
Yury Selivanov 1af2bf75a2 asyncio: Support PEP 492. Issue #24017. 2015-05-11 22:27:25 -04:00
Yury Selivanov 59eb9a4da5 asyncio: async() function is deprecated in favour of ensure_future(). 2015-05-11 14:48:38 -04:00
Yury Selivanov 740169cd24 Sync asyncio changes from the main repo. 2015-05-11 14:23:38 -04:00
Guido van Rossum 0a9933ebf3 Asyncio issue 222 / PR 231 (Victor Stinner) -- fix @coroutine functions without __name__. 2015-05-02 18:38:24 -07:00
Guido van Rossum 0bd16bc4cd Fix asyncio issue 235: Queue subclass bug caused by JoinableQueue merge. 2015-04-20 09:24:24 -07:00
Victor Stinner c9d11c341e Issue #23879, asyncio: SelectorEventLoop.sock_connect() must not call connect()
again if the first call to connect() raises an InterruptedError.

When the C function connect() fails with EINTR, the connection runs in
background. We have to wait until the socket becomes writable to be notified
when the connection succeed or fails.
2015-04-07 21:38:04 +02:00
Victor Stinner 79fd962652 asyncio: Fix _SelectorTransport.__repr__() if the event loop is closed 2015-03-27 15:20:08 +01:00
Victor Stinner d6dc7bdaf9 Issue #23456: Add missing @coroutine decorators in asyncio 2015-03-18 11:37:42 +01:00
Victor Stinner 7a82afee70 asyncio: Fix repr(BaseSubprocessTransport) if it didn't start yet
Replace "running" with "not started" and don't show the pid if the subprocess
didn't start yet.
2015-03-10 16:32:29 +01:00
Victor Stinner ce8c7682d6 Issue #23537: Remove 2 unused private methods of asyncio.BaseSubprocessTransport
Methods only raise NotImplementedError and are never used.
2015-02-27 17:49:09 +01:00
Victor Stinner 4cb814c7e1 asyncio, Tulip issue 220: Merge JoinableQueue with Queue.
Merge JoinableQueue with Queue. To more closely match the standard Queue,
asyncio.Queue has "join" and "task_done". JoinableQueue is deleted.

Docstring for Queue.join shouldn't mention threads.

Restore JoinableQueue as a deprecated alias for Queue. To more closely match
the standard Queue, asyncio.Queue has "join" and "task_done".  JoinableQueue
remains as a deprecated alias for Queue to avoid needlessly breaking too much
code that depended on it.

Patch written by A. Jesse Jiryu Davis <jesse@mongodb.com>.
2015-02-17 22:53:28 +01:00
Victor Stinner 4e82fb99a0 asyncio: BaseSubprocessTransport: repr() mentions when the child process is
running
2015-02-17 22:50:33 +01:00
Victor Stinner 8e36812e27 asyncio: BaseSubprocessTransport.close() doesn't try to kill the process if it
already finished
2015-02-10 14:49:32 +01:00
Victor Stinner a87501fd23 asyncio: BaseEventLoop: rename _owner to _thread_id 2015-02-05 11:45:33 +01:00
Victor Stinner 2fc2313038 asyncio: Only call _check_resolved_address() in debug mode
* _check_resolved_address() is implemented with getaddrinfo() which is slow
* If available, use socket.inet_pton() instead of socket.getaddrinfo(), because
  it is much faster

Microbenchmark (timeit) on Fedora 21 (Python 3.4, Linux 3.17, glibc 2.20) to
validate the IPV4 address "127.0.0.1" or the IPv6 address "::1":

* getaddrinfo() 10.4 usec per loop
* inet_pton(): 0.285 usec per loop

On glibc older than 2.14, getaddrinfo() always requests the list of all local
IP addresses to the kernel (using a NETLINK socket). getaddrinfo() has other
known issues, it's better to avoid it when it is possible.
2015-02-04 14:51:23 +01:00
Victor Stinner aa41b9b22b asyncio: BaseSelectorEventLoop uses directly the private _debug attribute
Just try to be consistent: _debug was already used in some places, and always
used in BaseProactorEventLoop.
2015-02-04 14:50:59 +01:00
Victor Stinner 17d87f8ae4 asyncio, Tulip issue 221: Fix doc of QueueEmpty and QueueFull 2015-02-03 15:09:24 +01:00
Victor Stinner 5d44c08f1c Issue #23353, asyncio: Workaround CPython bug #23353
Don't use yield/yield-from in an except block of a generator. Store the
exception and handle it outside the except block.
2015-02-02 18:36:31 +01:00
Victor Stinner f2e43cbbd4 Issue #23347, asyncio: send_signal(), terminate(), kill() don't check if the
transport was closed. The check broken a Tulip example and this limitation is
arbitrary. Check if _proc is None should be enough.

Enhance also close(): do nothing when called the second time.
2015-01-30 01:20:44 +01:00
Victor Stinner 1241ecc21b Issue #23347, asyncio: Make BaseSubprocessTransport.wait() private 2015-01-30 00:16:14 +01:00
Victor Stinner 47cd10d7a9 asyncio: sync with Tulip
Issue #23347: send_signal(), kill() and terminate() methods of
BaseSubprocessTransport now check if the transport was closed and if the
process exited.

Issue #23347: Refactor creation of subprocess transports. Changes on
BaseSubprocessTransport:

* Add a wait() method to wait until the child process exit
* The constructor now accepts an optional waiter parameter. The _post_init()
  coroutine must not be called explicitly anymore. It makes subprocess
  transports closer to other transports, and it gives more freedom if we want
  later to change completly how subprocess transports are created.
* close() now kills the process instead of kindly terminate it: the child
  process may ignore SIGTERM and continue to run. Call explicitly terminate()
  and wait() if you want to kindly terminate the child process.
* close() now logs a warning in debug mode if the process is still running and
  needs to be killed
* _make_subprocess_transport() is now fully asynchronous again: if the creation
  of the transport failed, wait asynchronously for the process eixt. Before the
  wait was synchronous. This change requires close() to *kill*, and not
  terminate, the child process.
* Remove the _kill_wait() method, replaced with a more agressive close()
  method. It fixes _make_subprocess_transport() on error.
  BaseSubprocessTransport.close() calls the close() method of pipe transports,
  whereas _kill_wait() closed directly pipes of the subprocess.Popen object
  without unregistering file descriptors from the selector (which caused severe
  bugs).

These changes simplifies the code of subprocess.py.
2015-01-30 00:05:19 +01:00
Victor Stinner 978a9afc6a Issue #23243, asyncio: Emit a ResourceWarning when an event loop or a transport
is not explicitly closed. Close also explicitly transports in test_sslproto.
2015-01-29 17:50:58 +01:00
Victor Stinner 2934262fd3 asyncio: sync with Tulip
* Cleanup gather(): use cancelled() method instead of using private Future
  attribute
* Fix _UnixReadPipeTransport and _UnixWritePipeTransport. Only start reading
  when connection_made() has been called.
* Issue #23333: Fix BaseSelectorEventLoop._accept_connection(). Close the
  transport on error. In debug mode, log errors using call_exception_handler()
2015-01-29 14:15:19 +01:00
Victor Stinner 54a231d539 asyncio doc: document Protocol state machine 2015-01-29 13:33:15 +01:00
Victor Stinner 47bbea7124 asyncio: sync with Tulip
* _SelectorTransport constructor: extra parameter is now optional
* Fix _SelectorDatagramTransport constructor. Only start reading after
  connection_made() has been called.
* Fix _SelectorSslTransport.close(). Don't call protocol.connection_lost() if
  protocol.connection_made() was not called yet: if the SSL handshake failed or
  is still in progress. The close() method can be called if the creation of the
  connection is cancelled, by a timeout for example.
2015-01-29 02:56:05 +01:00
Victor Stinner 7b5a900e88 asyncio: BaseSubprocessTransport._kill_wait() now also call close()
close() closes pipes, which is not None yet by _kill_wait().
2015-01-29 02:14:30 +01:00
Victor Stinner fa73779b0a asyncio: Fix _SelectorSocketTransport constructor
Only start reading when connection_made() has been called:
protocol.data_received() must not be called before protocol.connection_made().
2015-01-29 00:36:51 +01:00
Victor Stinner f07801bb17 asyncio: SSL transports now clear their reference to the waiter
* Rephrase also the comment explaining why the waiter is not awaken immediatly.
* SSLProtocol.eof_received() doesn't instanciate ConnectionResetError exception
  directly, it will be done by Future.set_exception(). The exception is not
  used if the waiter was cancelled or if there is no waiter.
2015-01-29 00:36:35 +01:00
Victor Stinner b507cbaac5 asyncio: Fix SSLProtocol.eof_received()
Wake-up the waiter if it is not done yet.
2015-01-29 00:35:56 +01:00
Victor Stinner ff018e406c asyncio: sync with Tulip
* Remove unused SSLProtocol._closing attribute
* test_sslproto: skip test if ssl module is missing
* Python issue #23208: Don't use the traceback of the current handle if we
  already know the traceback of the source. The handle may be more revelant,
  but having 3 tracebacks (handle, source, exception) becomes more difficult to
  read. The handle may be preferred later but it requires more work to make
  this choice.
2015-01-28 00:30:40 +01:00
Victor Stinner 41063d2a59 asyncio, Tulip issue 204: Fix IocpProactor.recv()
If ReadFile() fails with ERROR_BROKEN_PIPE, the operation is not pending: don't
register the overlapped.

I don't know if WSARecv() can fail with ERROR_BROKEN_PIPE. Since
Overlapped.WSARecv() already handled ERROR_BROKEN_PIPE, let me guess that it
has the same behaviour than ReadFile().
2015-01-26 22:30:49 +01:00
Victor Stinner 24dfa3c1d6 Issue #23095, asyncio: Fix _WaitHandleFuture.cancel()
If UnregisterWaitEx() fais with ERROR_IO_PENDING, it doesn't mean that the wait
is unregistered yet. We still have to wait until the wait is cancelled.
2015-01-26 22:30:28 +01:00
Victor Stinner e0fd157ba0 Issue #23293, asyncio: Rewrite IocpProactor.connect_pipe() as a coroutine
Use a coroutine with asyncio.sleep() instead of call_later() to ensure that the
schedule call is cancelled.

Add also a unit test cancelling connect_pipe().
2015-01-26 15:04:03 +01:00
Victor Stinner 2a3f38fd29 asyncio: PipeHandle.fileno() now raises an exception if the pipe is closed 2015-01-26 15:03:44 +01:00
Victor Stinner a19b7b3fca asyncio: Fix ProactorEventLoop.start_serving_pipe()
If a client connected before the server was closed: drop the client (close the
pipe) and exit.
2015-01-26 15:03:20 +01:00
Victor Stinner 9b524d59b0 Issue #23208, asyncio: Add BaseEventLoop._current_handle
In debug mode, BaseEventLoop._run_once() now sets the
BaseEventLoop._current_handle attribute to the handle currently executed.
In release mode or when no handle is executed, the attribute is None.

BaseEventLoop.default_exception_handler() displays the traceback of the current
handle if available.
2015-01-26 11:05:12 +01:00
Victor Stinner 3d2256f671 Issue #23293, asyncio: Cleanup IocpProactor.close()
The special case for connect_pipe() is not more needed. connect_pipe() doesn't
use overlapped operations anymore.
2015-01-26 11:02:59 +01:00
Victor Stinner 2596dd0894 asyncio: Close transports on error
Fix create_datagram_endpoint(), connect_read_pipe() and connect_write_pipe():
close the transport if the task is cancelled or on error.
2015-01-26 11:02:18 +01:00
Victor Stinner 2b77c5467f asyncio, Tulip issue 204: Fix IocpProactor.accept_pipe()
Overlapped.ConnectNamedPipe() now returns a boolean: True if the pipe is
connected (if ConnectNamedPipe() failed with ERROR_PIPE_CONNECTED), False if
the connection is in progress.

This change removes multiple hacks in IocpProactor.
2015-01-22 23:50:03 +01:00
Victor Stinner 7ffa2c5fdd Issue #23293, asyncio: Rewrite IocpProactor.connect_pipe()
Add _overlapped.ConnectPipe() which tries to connect to the pipe for
asynchronous I/O (overlapped): call CreateFile() in a loop until it doesn't
fail with ERROR_PIPE_BUSY. Use an increasing delay between 1 ms and 100 ms.

Remove Overlapped.WaitNamedPipeAndConnect() which is no more used.
2015-01-22 22:55:08 +01:00
Victor Stinner 752aba7f99 asyncio: IocpProactor.close() doesn't cancel anymore futures which are already
cancelled
2015-01-22 22:47:13 +01:00
Victor Stinner 1ca9392c70 Issue #23095, asyncio: IocpProactor.close() must not cancel pending
_WaitCancelFuture futures
2015-01-22 00:17:54 +01:00
Victor Stinner 0c2e408833 asyncio: BaseEventLoop._create_connection_transport() catchs any exception, not
only Exception
2015-01-22 00:17:41 +01:00
Victor Stinner d0a28dee78 Issue #23095, asyncio: Rewrite _WaitHandleFuture.cancel()
This change fixes a race conditon related to _WaitHandleFuture.cancel() leading
to Python crash or "GetQueuedCompletionStatus() returned an unexpected event"
logs. Before, the overlapped object was destroyed too early, it was possible
that the wait completed whereas the overlapped object was already destroyed.
Sometimes, a different overlapped was allocated at the same address, leading to
unexpected completition.

_WaitHandleFuture.cancel() now waits until the wait is cancelled to clear its
reference to the overlapped object. To wait until the cancellation is done,
UnregisterWaitEx() is used with an event instead of UnregisterWait().

To wait for this event, a new _WaitCancelFuture class was added. It's a
simplified version of _WaitCancelFuture. For example, its cancel() method calls
UnregisterWait(), not UnregisterWaitEx(). _WaitCancelFuture should not be
cancelled.

The overlapped object is kept alive in _WaitHandleFuture until the wait is
unregistered.

Other changes:

* Add _overlapped.UnregisterWaitEx()
* Remove fast-path in IocpProactor.wait_for_handle() to immediatly set the
  result if the wait already completed. I'm not sure that it's safe to
  call immediatly UnregisterWaitEx() before the completion was signaled.
* Add IocpProactor._unregistered() to forget an overlapped which may never be
  signaled, but may be signaled for the next loop iteration. It avoids to
  block forever IocpProactor.close() if a wait was cancelled, and it may also
  avoid some "... unexpected event ..." warnings.
2015-01-21 23:39:51 +01:00
Victor Stinner 442b0adccd asyncio: pyflakes, remove unused import
tests: Remove unused function; inline another function
2015-01-21 23:39:16 +01:00
Victor Stinner cd0f7f9832 asyncio: Enhance BaseProactorEventLoop._loop_self_reading()
* Handle correctly CancelledError: just exit
* On error, log the exception and exit

Don't try to close the event loop, it is probably running and so it cannot be
closed.
2015-01-21 23:38:37 +01:00
Victor Stinner 406204c8c2 Backout changeset 6ab2575bc12b
StreamWriter: close() now clears the reference to the transport

StreamWriter now raises an exception if it is closed: write(), writelines(),
write_eof(), can_write_eof(), get_extra_info(), drain().
2015-01-15 21:50:19 +01:00
Victor Stinner 922bc2ca12 Closes #23219: cancelling asyncio.wait_for() now cancels the task 2015-01-15 16:29:10 +01:00
Victor Stinner 4bf22e033e asyncio: Close the transport on subprocess creation failure 2015-01-15 14:24:22 +01:00
Victor Stinner fcd58de78f asyncio: Fix _ProactorBasePipeTransport.close()
Set the _read_fut attribute to None after cancelling it.

This change should fix a race condition with
_ProactorWritePipeTransport._pipe_closed().
2015-01-15 13:40:27 +01:00
Victor Stinner 79c93ba47b asyncio: Fix _ProactorBasePipeTransport.__repr__()
Check if the _sock attribute is None to check if the transport is closed.
2015-01-15 13:32:28 +01:00
Victor Stinner 41ed958ee6 Issue #23243: Fix asyncio._UnixWritePipeTransport.close()
Do nothing if the transport is already closed. Before it was not possible to
close the transport twice.
2015-01-15 13:16:50 +01:00
Victor Stinner 7e222f411c SSLProtocol: set the _transport attribute in the constructor 2015-01-15 13:16:27 +01:00
Victor Stinner 791009bb89 Issue #23242: asyncio.SubprocessStreamProtocol now closes the subprocess
transport at subprocess exit.

Clear also its reference to the transport.
2015-01-15 13:16:02 +01:00
Victor Stinner 72bdefb086 Issue #22560: Fix typo: call -> call_soon 2015-01-15 09:44:13 +01:00
Victor Stinner 042dad7232 Issue #22560: Fix SSLProtocol._on_handshake_complete()
Don't call immediatly self._process_write_backlog() but schedule the call using
call_soon(). _on_handshake_complete() can be called indirectly from
_process_write_backlog(), and _process_write_backlog() is not reentrant.
2015-01-15 09:41:48 +01:00
Victor Stinner e7a35717d2 StreamWriter: close() now clears the reference to the transport
StreamWriter now raises an exception if it is closed: write(), writelines(),
write_eof(), can_write_eof(), get_extra_info(), drain().
2015-01-15 09:33:50 +01:00
Victor Stinner 29ad0111bd asyncio: sync with Tulip
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
  ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
  creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
  closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
  not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
  (BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
  readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
  close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
  state in the __repr__() method of _UnixReadPipeTransport and
  _UnixWritePipeTransport classes.
2015-01-15 00:04:21 +01:00
Victor Stinner b92626df5c Issue #23197: On SSL handshake failure on matching hostname, check if the
waiter is cancelled before setting its exception.
2015-01-14 17:13:28 +01:00
Victor Stinner 177e9f0855 Issue #23197, asyncio: On SSL handshake failure, check if the waiter is
cancelled before setting its exception.

* Add unit tests for this case.
* Cleanup also sslproto.py
2015-01-14 16:56:20 +01:00
Victor Stinner f651a60407 Python issue #23173: sync with Tulip
* If an exception is raised during the creation of a subprocess, kill the
  subprocess (close pipes, kill and read the return status). Log an error in
  such case.
* Fix SubprocessStreamProtocol.connection_made() to handle cancelled waiter.
  Add unit test cancelling subprocess methods.
2015-01-14 02:10:33 +01:00
Victor Stinner c2c12e433a Issue #23198: Reactor asyncio.StreamReader
- Add a new _wakeup_waiter() method
- Replace _create_waiter() method with a _wait_for_data() coroutine function
- Use the value None instead of True or False to wake up the waiter
2015-01-14 00:53:37 +01:00
Victor Stinner 231b404cb0 Issue #22560: New SSL implementation based on ssl.MemoryBIO
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.

The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).

Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.

This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!

It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
2015-01-14 00:19:09 +01:00
Victor Stinner b86a96802b Issue #22922: Fix ProactorEventLoop.close()
Close the IocpProactor before closing the event loop. IocpProactor.close() can
call loop.call_soon(), which is forbidden when the event loop is closed.
2015-01-13 16:11:19 +01:00
Victor Stinner 587feb19e8 Issue #23209: Break some reference cycles in asyncio. Patch written by Martin
Richard.
2015-01-09 21:34:27 +01:00
Victor Stinner 70db9e428a asyncio: sync with Tulip
* Tulip issue 184: FlowControlMixin constructor now get the event loop if the
  loop parameter is not set. Add unit tests to ensure that constructor of
  StreamReader and StreamReaderProtocol classes get the event loop.
* Remove outdated TODO/XXX
2015-01-09 21:32:05 +01:00
Victor Stinner 3531d9044d asyncio: sync with Tulip
* Document why set_result() calls are safe
* Cleanup gather(). Use public methods instead of hacks to consume the
  exception of a future.
* sock_connect(): pass directly the fd to _sock_connect_done instead of the
  socket.
2015-01-09 01:42:52 +01:00
Victor Stinner 15cc678d89 asyncio: Truncate to 80 columns 2015-01-09 00:09:10 +01:00
Victor Stinner 8d9c145f61 asyncio: _make_ssl_transport: make the waiter parameter optional 2015-01-08 12:06:36 +01:00
Victor Stinner c447ba04e7 Issue #23140, asyncio: Fix cancellation of Process.wait(). Check the state of
the waiter future before setting its result.
2015-01-06 01:13:49 +01:00
Victor Stinner 8c1a4a2326 Issue #23046: Expose the BaseEventLoop class in the asyncio namespace 2015-01-06 01:03:58 +01:00
Victor Stinner 956de691f8 Issue #22926: In debug mode, call_soon(), call_at() and call_later() methods of
asyncio.BaseEventLoop now use the identifier of the current thread to ensure
that they are called from the thread running the event loop.

Before, the get_event_loop() method was used to check the thread, and no
exception was raised when the thread had no event loop. Now the methods always
raise an exception in debug mode when called from the wrong thread. It should
help to notice misusage of the API.
2014-12-26 21:07:52 +01:00
Victor Stinner 952ec98abb asyncio doc: update also Queue docstrings 2014-12-22 22:09:50 +01:00
Victor Stinner 4d825b45a0 asyncio: IocpProactor.wait_for_handle() test now also checks the result of the
future
2014-12-19 17:10:44 +01:00
Victor Stinner 1b9763d0a9 asyncio: sync with Tulip
* Fix a race condition in BaseSubprocessTransport._try_finish().

  If the process exited before the _post_init() method was called, scheduling
  the call to _call_connection_lost() with call_soon() is wrong:
  connection_made() must be called before connection_lost().

  Reuse the BaseSubprocessTransport._call() method to schedule the call to
  _call_connection_lost() to ensure that connection_made() and
  connection_lost() are called in the correct order.

* Add repr(PipeHandle)

* Fix typo
2014-12-18 23:47:27 +01:00
Victor Stinner 3a1c738e6c Issue #23074: asyncio.get_event_loop() now raises an exception if the thread
has no event loop even if assertions are disabled.
2014-12-18 01:20:10 +01:00
Victor Stinner 1e40f10886 asyncio, tulip issue 209: Fix subprocess for close_fds=False on Python 3.3
Mark the write end of the stdin pipe as non-inheritable.
2014-12-11 23:30:17 +01:00
Victor Stinner df75d5b402 asyncio, tulip issue 202: Add unit test of pause/resume writing for proactor
socket transport
2014-12-11 22:23:19 +01:00
Victor Stinner f3e2e09213 Closes #22429, asyncio: Fix EventLoop.run_until_complete(), don't stop the
event loop if a BaseException is raised, because the event loop is already
stopped.
2014-12-05 01:44:10 +01:00
Victor Stinner 4c85ec99f3 Issue #22922: Fix ProactorEventLoop.close()
Call _stop_accept_futures() before sestting the _closed attribute, otherwise
call_soon() raises an error.
2014-12-05 01:43:42 +01:00
Victor Stinner e80bf0d4a9 Closes #22922: More EventLoop methods fail if the loop is closed. Initial patch
written by Torsten Landschoff.

create_task(), call_at(), call_soon(), call_soon_threadsafe() and
run_in_executor() now raise an error if the event loop is closed.
2014-12-04 23:07:47 +01:00
Victor Stinner fe22e0985a asyncio: Initialize more Future and Task attributes in the class definition to
avoid attribute errors in destructors.
2014-12-04 23:00:13 +01:00
Victor Stinner d87de83582 Closes #22475: asyncio doc, fix Task.get_stack() doc 2014-12-02 17:57:04 +01:00
Serhiy Storchaka 56a6d855e2 Removed duplicated words in in comments and docs. 2014-12-01 18:28:43 +02:00
Victor Stinner 5ef586f25a Closes #22685, asyncio: Set the transport of stdout and stderr StreamReader
objects in the SubprocessStreamProtocol. It allows to pause the transport to
not buffer too much stdout or stderr data.
2014-11-25 17:20:33 +01:00
Benjamin Peterson 7243b574e5 don't require OpenSSL SNI to pass hostname to ssl functions (#22921)
Patch by Donald Stufft.
2014-11-23 17:04:34 -06:00
Victor Stinner 5e63120f8d asyncio: BaseSelectorEventLoop.close() now closes the self-pipe before calling
the parent close() method. If the event loop is already closed, the self-pipe
is not unregistered from the selector.
2014-11-21 00:23:27 +01:00
Victor Stinner 2d99d93d11 asyncio: Coroutine objects are now rejected with a TypeError by the following
functions:

* add_signal_handler()
* call_at()
* call_later()
* call_soon()
* call_soon_threadsafe()
* run_in_executor()

Fix also the error message of add_signal_handler() (fix the name of the
function).
2014-11-20 15:03:52 +01:00
Victor Stinner 662fd5f68e asyncio: Fix formatting of the "Future exception was never retrieved" in
release mode
2014-11-20 14:16:31 +01:00
Guido van Rossum e36fcde383 - Issue #22841: Reject coroutines in asyncio add_signal_handler().
Patch by Ludovic.Gasc.
2014-11-14 11:45:47 -08:00
Victor Stinner 004adb91f6 asyncio: Move loop attribute to _FlowControlMixin
Move the _loop attribute from the constructor of _SelectorTransport,
_ProactorBasePipeTransport and _UnixWritePipeTransport classes to the
constructor of the _FlowControlMixin class.

Add also an assertion to explicit that the parent class must ensure that the
loop is defined (not None)
2014-11-05 15:27:41 +01:00
Antoine Pitrou fd39a89e0e Issue #22641: In asyncio, the default SSL context for client connections is now created using ssl.create_default_context(), for stronger security. 2014-10-15 16:58:21 +02:00
Victor Stinner f67f460242 Issue #18643: asyncio.windows_utils now reuse socket.socketpair() on Windows if
available

Since Python 3.5, socket.socketpair() is now also available on Windows.

Make csock blocking before calling the accept() method, and fix also a typo in
an error message.
2014-10-14 22:56:25 +02:00
Victor Stinner 0e34dc3737 asyncio: enhance protocol representation
Add "closed" or "closing" to repr() of selector and proactor transports
2014-10-12 09:52:11 +02:00
Victor Stinner c8bd53f815 Issue #22601: run_forever() now consumes BaseException of the temporary task
If the coroutine raised a BaseException, consume the exception to not log a
warning. The caller doesn't have access to the local task.
2014-10-11 14:30:18 +02:00
Victor Stinner 68da8fc4c0 Issue #22448: asyncio, cleanup _run_once(), only iterate once to remove delayed
calls that were cancelled.
2014-09-30 18:08:36 +02:00
Yury Selivanov 592ada9b4b asyncio: Improve canceled timer handles cleanup. Closes issue #22448.
Patch by Joshua Moore-Oliva.
2014-09-25 12:07:56 -04:00
Yury Selivanov 8c0e0ab767 asyncio.unix_events: Move import statement to match tulip code 2014-09-24 23:21:39 -04:00
R David Murray 8e069d5ce9 Apply asyncio Task English fixes to docstrings as well.
Also fixed the phrasing in a comment.
2014-09-24 13:13:45 -04:00
Victor Stinner 1b38bc65dd asyncio, Tulip issue 206: In debug mode, keep the callback in the
representation of Handle and TimerHandle after cancel().
2014-09-17 23:24:13 +02:00
Serhiy Storchaka 148679982f Issue #22369: Change "context manager protocol" to "context management protocol". 2014-09-10 23:43:41 +03:00
Victor Stinner d5aeccf976 asyncio, Tulip issue 205: Fix a race condition in BaseSelectorEventLoop.sock_connect()
There is a race condition in create_connection() used with wait_for() to have a
timeout. sock_connect() registers the file descriptor of the socket to be
notified of write event (if connect() raises BlockingIOError). When
create_connection() is cancelled with a TimeoutError, sock_connect() coroutine
gets the exception, but it doesn't unregister the file descriptor for write
event. create_connection() gets the TimeoutError and closes the socket.

If you call again create_connection(), the new socket will likely gets the same
file descriptor, which is still registered in the selector. When sock_connect()
calls add_writer(), it tries to modify the entry instead of creating a new one.

This issue was originally reported in the Trollius project, but the bug comes
from Tulip in fact (Trollius is based on Tulip):
https://bitbucket.org/enovance/trollius/issue/15/after-timeouterror-on-wait_for

This change fixes the race condition. It also makes sock_connect() more
reliable (and portable) is sock.connect() raises an InterruptedError.
2014-08-31 15:07:57 +02:00
Victor Stinner 59e0802301 asyncio, Tulip issue 201: Fix a race condition in wait_for()
Don't raise a TimeoutError if we reached the timeout and the future completed
in the same iteration of the event loop. A side effect of the bug is that
Queue.get() looses items.
2014-08-28 11:19:25 +02:00
Victor Stinner 52bb949fd3 asyncio, tulip issue 203: Add _FlowControlMixin.get_write_buffer_limits() method 2014-08-26 00:22:28 +02:00
Victor Stinner b261475a48 asyncio: sync with Tulip
* PipeServer.close() now cancels the "accept pipe" future which cancels the
  overlapped operation.
* Fix _SelectorTransport.__repr__() if the transport was closed
* Fix debug log in BaseEventLoop.create_connection(): get the socket object
  from the transport because SSL transport closes the old socket and creates a
  new SSL socket object. Remove also the _SelectorSslTransport._rawsock
  attribute: it contained the closed socket (not very useful) and it was not
  used.
* Issue #22063: socket operations (sock_recv, sock_sendall, sock_connect,
  sock_accept) of the proactor event loop don't raise an exception in debug
  mode if the socket are in blocking mode. Overlapped operations also work on
  blocking sockets.
* Fix unit tests in debug mode: mock a non-blocking socket for socket
  operations which now raise an exception if the socket is blocking.
* _fatal_error() method of _UnixReadPipeTransport and _UnixWritePipeTransport
  now log all exceptions in debug mode
* Don't log expected errors in unit tests
* Tulip issue 200: _WaitHandleFuture._unregister_wait() now catchs and logs
  exceptions.
* Tulip issue 200: Log errors in debug mode instead of simply ignoring them.
2014-08-25 23:20:52 +02:00
Victor Stinner 9c9f1f10d3 Close #22063: socket operations (socket,recv, sock_sendall, sock_connect,
sock_accept) now raise an exception in debug mode if sockets are in blocking
mode.
2014-07-29 23:08:17 +02:00
Victor Stinner f2ed889027 asyncio: Use the new os.set_blocking() function of Python 3.5 if available 2014-07-29 23:08:00 +02:00
Victor Stinner 313a980904 asyncio: sync with Tulip
* _WaitHandleFuture.cancel() now notify IocpProactor through the overlapped
  object that the wait was cancelled.
* Optimize IocpProactor.wait_for_handle() gets the result if the wait is
  signaled immediatly.
* Enhance representation of Future and Future subclasses

  - Add "created at filename:lineno" in the representation
  - Add Future._repr_info() method which can be more easily overriden than
    Future.__repr__(). It should now be more easy to enhance Future
    representation without having to modify each subclass. For example,
    _OverlappedFuture and _WaitHandleFuture get the new "created at" information.
  - Use reprlib to format Future result, and function arguments when formatting a
    callback, to limit the length of the representation.

* Fix repr(_WaitHandleFuture)
* _WaitHandleFuture and _OverlappedFuture: hide frames of internal calls in the
  source traceback.
* Cleanup ProactorIocp._poll(): set the timeout to 0 after the first call to
  GetQueuedCompletionStatus()
* test_locks: close the temporary event loop and check the condition lock
* Remove workaround in test_futures, no more needed
2014-07-29 12:58:23 +02:00
Victor Stinner 42d3bdeed6 asyncio, tulip issue 196: ProactorIocp._register() now registers the overlapped
in the _cache dictionary, even if we already got the result. We need to keep a
reference to the overlapped object, otherwise the memory may be reused and
GetQueuedCompletionStatus() may use random bytes and behaves badly.

There is still a hack for ConnectNamedPipe(): the overlapped object is not
register into _cache if the overlapped object completed directly.

Log also an error in debug mode in ProactorIocp._loop() if we get an unexpected
event.

Add a protection in ProactorIocp.close() to avoid blocking, even if it should
not happen. I still don't understand exactly why some the completion of some
overlapped objects are not notified.
2014-07-28 00:18:43 +02:00
Andrew Svetlov f21fcd09c5 Accept optional lock object in Condition ctor (tulip issue #198) 2014-07-26 17:54:34 +03:00
Victor Stinner 51e44ea66a Tulip issue 196: _OverlappedFuture.set_result() now clears its reference to the
overlapped object. IocpProactor._poll() now also ignores false alarms:
GetQueuedCompletionStatus() returns the overlapped but it is still pending.
2014-07-26 00:58:34 +02:00
Victor Stinner 65dd69a3da asyncio: sync with Tulip
* Tulip issue #196: IocpProactor._poll() clears the reference to the
  overlapped operation when the operation is done. It would be better to clear
  the reference in a new _OverlappedFuture.set_result() method, but it cannot
  be done yet because of a weird bug.
* BaseSelectorEventLoop._write_to_self() now logs errors in debug mode.
2014-07-25 22:36:05 +02:00
Victor Stinner 18a28dc5c2 asyncio: sync with Tulip
* Fix _WaitHandleFuture.cancel(): return the result of the parent cancel()
  method.
* _OverlappedFuture.cancel() now clears its reference to the overlapped object.
  Make also the _OverlappedFuture.ov attribute private.
* Check if _WaitHandleFuture completed before unregistering it in the callback.
  Add also _WaitHandleFuture._poll() and repr(_WaitHandleFuture).
* _WaitHandleFuture now unregisters its wait handler if WaitForSingleObject()
  raises an exception.
* _OverlappedFuture.set_exception() now cancels the overlapped operation.
2014-07-25 13:05:20 +02:00
Victor Stinner fea6a100dc asyncio: sync with Tulip
Improve stability of the proactor event loop, especially operations on
overlapped objects:

* Tulip issue 195: Don't call UnregisterWait() twice if a _WaitHandleFuture is
  cancelled twice to fix a crash.
* IocpProactor.close(): cancel futures to cancel overlapped operations, instead
  of cancelling directly overlapped operations. Future objects may not call
  ov.cancel() if the future was cancelled or if the overlapped was already
  cancelled. The cancel() method of the future may also catch exceptions. Log
  also errors on cancellation.
* tests: rename "f" to "fut"
* Add a __repr__() method to IocpProactor
* Add a destructor to IocpProactor which closes it
* _OverlappedFuture.cancel() doesn't cancel the overlapped anymore if it is
  done: if it is already cancelled or completed. Log also an error if the
  cancellation failed.
* Add the address of the overlapped object in repr(_OverlappedFuture)
* _OverlappedFuture truncates the source traceback to hide the call to the
  parent constructor (useless in debug).
2014-07-25 00:54:53 +02:00
Victor Stinner 5006b1fd96 Issue #20055: Fix BaseEventLoop.stop() docstring, incomplete sentence.
Patch written by Saimadhav Heblikar.
2014-07-24 11:34:11 +02:00
Victor Stinner c4c464911a asyncio: sync with Tulip
* Tulip issue 194: Don't use sys.getrefcount() in unit tests
* signal.set_wakeup_fd() can now raise an OSError on Python 3.5
2014-07-23 18:21:45 +02:00
Victor Stinner 31e7bfa6ba asyncio, tulip issue 193: Convert StreamWriter.drain() to a classic coroutine
Replace also _make_drain_waiter() function with a classic _drain_helper()
coroutine.
2014-07-22 12:03:40 +02:00
Victor Stinner be0a2d767c Fix asyncio.__all__: export also unix_events and windows_events symbols
For example, on Windows, it was not possible to get ProactorEventLoop or
DefaultEventLoopPolicy using "from asyncio import *".
2014-07-18 12:44:25 +02:00
Victor Stinner fe5649c7b7 Python issue #21645, Tulip issue 192: Rewrite signal handling
Since Python 3.3, the C signal handler writes the signal number into the wakeup
file descriptor and then schedules the Python call using Py_AddPendingCall().

asyncio uses the wakeup file descriptor to wake up the event loop, and relies
on Py_AddPendingCall() to schedule the final callback with call_soon().

If the C signal handler is called in a thread different than the thread of the
event loop, the loop is awaken but Py_AddPendingCall() was not called yet. In
this case, the event loop has nothing to do and go to sleep again.
Py_AddPendingCall() is called while the event loop is sleeping again and so the
final callback is not scheduled immediatly.

This patch changes how asyncio handles signals. Instead of relying on
Py_AddPendingCall() and the wakeup file descriptor, asyncio now only relies on
the wakeup file descriptor. asyncio reads signal numbers from the wakeup file
descriptor to call its signal handler.
2014-07-17 22:43:40 +02:00
Victor Stinner d55b54d5c0 asyncio, tulip issue 190: Process.communicate() now ignores
ConnectionResetError too
2014-07-17 13:12:03 +02:00
Victor Stinner cc996b5789 asyncio, tulip issue 190: Process.communicate() must ignore BrokenPipeError
If you want to handle the BrokenPipeError, you can easily reimplement
communicate().

Add also a unit test to ensure that stdin.write() + stdin.drain() raises
BrokenPipeError.
2014-07-17 12:25:27 +02:00
Victor Stinner e931f7b768 Issue #21163: Fix "destroy pending task" warning in test_wait_errors() 2014-07-16 18:50:39 +02:00
Victor Stinner f03b3c7564 Issue #21163, asyncio: Ignore "destroy pending task" warnings for private tasks
in gather().
2014-07-16 18:36:24 +02:00
Victor Stinner 1cae9ec966 asyncio tests: make quiet the logs of SSL handshake failures when running tests
in debug mode
2014-07-14 22:26:34 +02:00
Victor Stinner acdb782a83 asyncio: sync with Tulip
* Tulip issue #184: Log subprocess events in debug mode

  - Log stdin, stdout and stderr transports and protocols
  - Log process identifier (pid)
  - Log connection of pipes
  - Log process exit
  - Log Process.communicate() tasks: feed stdin, read stdout and stderr
  - Add __repr__() method to many classes related to subprocesses


* Add BaseSubprocessTransport._pid attribute. Store the pid so it is still
  accessible after the process exited. It's more convinient for debug.

* create_connection(): add the socket in the "connected to" debug log

* Clean up some docstrings and comments. Remove unused unimplemented
  _read_from_self().
2014-07-14 18:33:40 +02:00
Victor Stinner e912e652f8 asyncio: sync with Tulip
* Tulip issue #183: log socket events in debug mode

  - Log most important socket events: socket connected, new client, connection
    reset or closed by peer (EOF), etc.
  - Log time elapsed in DNS resolution (getaddrinfo)
  - Log pause/resume reading
  - Log time of SSL handshake
  - Log SSL handshake errors
  - Add a __repr__() method to many classes

* Fix ProactorEventLoop() in debug mode. ProactorEventLoop._make_self_pipe()
  doesn't call call_soon() directly because it checks for the current loop
  which fails, because the method is called to build the event loop.

* Cleanup _ProactorReadPipeTransport constructor. Not need to set again
  _read_fut attribute to None, it is already done in the base class.
2014-07-12 03:11:53 +02:00
Victor Stinner 8ebeb03740 asyncio: improve the documentation of servers
- Fix the documentation of Server.close(): it closes sockets
- Replace AbstractServer with Server
- Document Server.sockets attribute
2014-07-11 23:47:40 +02:00
Victor Stinner b28dbac86d asyncio, Tulip issue 180: Make Server attributes and methods private
- loop, waiters and active_count attributes are now private
- attach(), detach() and wakeup() methods are now private

The sockets attribute remains public.
2014-07-11 22:52:21 +02:00
Victor Stinner 770e48d017 asyncio: sync with Tulip
* Tulip issue #182: Improve logs of BaseEventLoop._run_once()

  - Don't log non-blocking poll
  - Only log polling with a timeout if it gets events or if it timed out after
    more than 1 second.

* Fix some pyflakes warnings: remove unused imports
2014-07-11 11:58:33 +02:00
Victor Stinner 737c34fa85 asyncio: sync with Tulip
- CoroWrapper.__del__() now reuses repr(CoroWrapper) to log the "... was never
  yielded from" warning
- Improve CoroWrapper: copy also the qualified name on Python 3.4, not only on
  Python 3.5+
2014-07-11 01:04:16 +02:00
Victor Stinner c39ba7d611 asyncio: sync with Tulip
- repr(Task) and repr(CoroWrapper) now also includes where these objects were
  created. If the coroutine is not a generator (don't use "yield from"), use
  the location of the function, not the location of the coro() wrapper.
- Fix create_task(): truncate the traceback to hide the call to create_task().
2014-07-11 00:21:27 +02:00
Victor Stinner f68bd88aa6 asyncio: sync with Tulip
- Issues #21936, #21163: Fix sporadic failures of
  test_future_exception_never_retrieved()
- Handle.cancel() now clears references to callback and args
- In debug mode, repr(Handle) now contains the location where the Handle was
  created.
2014-07-10 22:32:58 +02:00
Victor Stinner bfff45d611 asyncion, Tulip issue 181: BaseEventLoop.create_datagram_endpoint() now waits
until protocol.connection_made() has been called. Document also why transport
constructors use a waiter.
2014-07-08 23:57:31 +02:00
Victor Stinner 896a25ab30 asyncio: sync with Tulip
- Tulip issue 185: Add a create_task() method to event loops. The create_task()
  method can be overriden in custom event loop to implement their own task
  class. For example, greenio and Pulsar projects use their own task class. The
  create_task() method is now preferred over creating directly task using the
  Task class.
- tests: fix a warning
- fix typo in the name of a test function
- Update AbstractEventLoop: add new event loop methods; update also the unit test
2014-07-08 11:29:25 +02:00
Victor Stinner 799a60ccb4 asyncio: sync with Tulip
Backout the "Tulip issue 181: Faster create_connection()" changeset, it was a
mistake.
2014-07-07 18:08:22 +02:00
Victor Stinner 1a870c9132 asyncio: sync with Tulip
- Tulip issue #181: Faster create_connection(). Call directly
  waiter.set_result() in the constructor of _ProactorBasePipeTransport and
  _SelectorSocketTransport, instead of using of delaying the call with
  call_soon().
- Cleanup iscoroutine()
2014-07-07 17:26:54 +02:00
Victor Stinner a9acbe82e7 Closes #21886, #21447: Fix a race condition in asyncio when setting the result
of a Future with call_soon(). Add an helper, a private method, to set the
result only if the future was not cancelled.
2014-07-05 15:29:41 +02:00
Victor Stinner 2dba23af71 asyncio: sync with Tulip
* _UnixSubprocessTransport: fix file mode of stdin. Open stdin in write mode,
  not in read mode
* Examples: close the event loop at exit
* More reliable CoroWrapper.__del__. If the constructor is interrupted by
  KeyboardInterrupt or the coroutine objet is destroyed lately, some the
  _source_traceback attribute doesn't exist anymore.
* repr(Task): include also the future the task is waiting for
2014-07-03 00:59:00 +02:00
Victor Stinner 98b6391fd4 Issue #21163: BaseEventLoop.run_until_complete() and test_utils.run_briefly()
don't log the "destroy pending task" message anymore. The log is redundant for
run_until_complete() and useless in run_briefly().
2014-06-30 14:51:04 +02:00
Victor Stinner b75380f333 asyncio: sync with Tulip
- Sort imports
- Simplify/optimize iscoroutine(). Inline inspect.isgenerator(obj): replace it
  with isinstance(obj, types.GeneratorType)
- CoroWrapper: check at runtime if Python has the yield-from bug #21209.  If
  Python has the bug, check if CoroWrapper.send() was called by yield-from to
  decide if parameters must be unpacked or not.
- Fix "Task was destroyed but it is pending!" warning in
  test_task_source_traceback()
2014-06-30 14:39:11 +02:00
Victor Stinner f951d28ac8 asyncio: sync with Tulip, add a new asyncio.coroutines module 2014-06-29 00:46:45 +02:00
Victor Stinner 80f53aa9a0 asyncio, Tulip issue 137: In debug mode, save traceback where Future, Task and
Handle objects are created. Pass the traceback to call_exception_handler() in
the 'source_traceback' key.

The traceback is truncated to hide internal calls in asyncio, show only the
traceback from user code.

Add tests for the new source_traceback, and a test for the 'Future/Task
exception was never retrieved' log.
2014-06-27 13:52:20 +02:00
Victor Stinner bbd96c6f47 asyncio, Tulip issue 137: In debug mode, add the traceback where the coroutine
object was created to the "coroutine ... was never yield from" log
2014-06-27 12:28:41 +02:00
Victor Stinner 17b53f1301 asyncio: Handle error handler: enhance formatting of the callback 2014-06-26 01:35:45 +02:00
Victor Stinner 975735f729 asyncio, Tulip issue 177: Rewite repr() of Future, Task, Handle and TimerHandle
- Uniformize repr() output to format "<Class ...>"
- On Python 3.5+, repr(Task) uses the qualified name instead of the short name
  of the coroutine
2014-06-25 21:41:58 +02:00
Victor Stinner df29c4a83d asyncio: repr(Task) now also contains the line number even if the coroutine is
done: use the first line number of the code object instead of the current line
number of the generator frame.

The name of the coroutine is not enough because many coroutines may have the
same name. It's a common case in asyncio tests for example.
2014-06-24 22:57:14 +02:00
Victor Stinner a02f81ff17 asyncio: Log an error if a Task is destroyed while it is still pending 2014-06-24 22:37:53 +02:00
Victor Stinner 751c7c0f2d asyncio: Fix BaseEventLoop._assert_is_current_event_loop(): get_event_loop()
raises an exception if there is no current loop
2014-06-23 15:14:13 +02:00
Victor Stinner f328c7dc69 asyncio, Tulip issue 171: BaseEventLoop.close() now raises an exception if the
event loop is running. You must first stop the event loop and then wait until
it stopped, before closing it.
2014-06-23 01:02:37 +02:00
Victor Stinner 1580fe3fce asyncio, Tulip issue 172: only log selector timing in debug mode 2014-06-23 00:31:08 +02:00
Victor Stinner 7b7120e159 asyncio: Enable the debug mode of event loops when the PYTHONASYNCIODEBUG
environment variable is set
2014-06-23 00:12:14 +02:00
Victor Stinner d6de5d8455 asyncio: BaseEventLoop._assert_is_current_event_loop() now only raises an
exception if the current loop is not None.

Guido van Rossum wrote:

"The behavior that you can set the loop to None (and keep track of it
explicitly) is part of the spec, and this should still be supported even in
debug mode. The behavior that we raise an error if you are caught having
multiple active loops per thread is just a debugging heuristic, and it
shouldn't break code that follows the spec."
2014-06-23 00:03:43 +02:00
Victor Stinner 0e6f52a211 asyncio, Tulip issue 105: in debug mode, log callbacks taking more than 100 ms
to be executed.
2014-06-20 17:34:15 +02:00
Victor Stinner d143209d7f Tulip issue 83: document more asyncio functions in docstrings 2014-06-19 17:11:49 +02:00
Victor Stinner 54c4b8e5c1 Closes #21595: asyncio.BaseSelectorEventLoop._read_from_self() now reads all
available bytes from the "self pipe", not only a single byte. This change
reduces the risk of having the pipe full and so getting the innocuous
"BlockingIOError: [Errno 11] Resource temporarily unavailable" message.
2014-06-19 12:59:15 +02:00
Victor Stinner c73701de72 asyncio: Refactor tests: add a base TestCase class 2014-06-18 01:36:32 +02:00
Victor Stinner 8d3e02ef5a asyncio: Set __qualname__ attribute of CoroWrapper in @coroutine decorator on
Python 3.5

- Drop __slots__ optimization of CoroWrapper to be able to set the __qualname__
  attribute.
- Add tests on __name__, __qualname__ and __module__ of a coroutine function
  and coroutine object.
- Fix test_tasks when run in debug mode (PYTHONASYNCIODEBUG env var set) on
  Python 3.3 or 3.4
2014-06-18 01:14:59 +02:00
Victor Stinner 66dc6b0f53 Issue #21723: asyncio.Queue: support any type of number (ex: float) for the
maximum size. Patch written by Vajrasky Kok.
2014-06-17 23:36:21 +02:00
Victor Stinner bc434e2052 asyncio: Task.__repr__() now also handles CoroWrapper 2014-06-17 00:26:36 +02:00
Victor Stinner 307bccc6ff asyncio: Tulip issue 173: Enhance repr(Handle) and repr(Task)
repr(Handle) is shorter for function: "foo" instead of "<function foo at
0x...>". It now also includes the source of the callback, filename and line
number where it was defined, if available.

repr(Task) now also includes the current position in the code, filename and
line number, if available. If the coroutine (generator) is done, the line
number is omitted and "done" is added.
2014-06-12 18:39:26 +02:00
Victor Stinner db74d982d4 Issue #21596: asyncio.wait(): mention that the sequence of futures must not
be empty.
2014-06-10 11:16:05 +02:00
Victor Stinner bb2fc5b2a5 Issue #21326: Add a new is_closed() method to asyncio.BaseEventLoop
Add BaseEventLoop._closed attribute and use it to check if the event loop was
closed or not, instead of checking different attributes in each subclass of
BaseEventLoop.

run_forever() and run_until_complete() methods now raise a RuntimeError('Event loop is
closed') exception if the event loop was closed.

BaseProactorEventLoop.close() now also cancels "accept futures".
2014-06-10 10:23:10 +02:00
Victor Stinner f9e49dd346 Tulip issue 83, Python issue #21252: Fill some XXX docstrings in asyncio 2014-06-05 12:06:44 +02:00
Victor Stinner a9fa2664ab Issue #21119: asyncio: Make sure that socketpair() close sockets on error
Close the listening socket if sock.bind() raises an exception.
2014-06-04 00:12:28 +02:00
Victor Stinner 223a624158 Issue #21119: asyncio now closes sockets on errors
Fix ResourceWarning: create_connection(), create_datagram_endpoint() and
create_unix_server() methods of event loop now close the newly created socket
on error.
2014-06-04 00:11:52 +02:00
Victor Stinner 8d21357fb5 Issue #21601: Document asyncio.Task.cancel(). Initial patch written by Vajrasky
Kok.
2014-06-02 23:06:46 +02:00
Victor Stinner a5b257af22 Issue #21454: Fix asyncio.BaseEventLoop.connect_read_pipe doc
The function sets the the pipe to non-blocking mode.
2014-05-29 00:14:03 +02:00
Andrew Svetlov 3207a03035 Fix for raising exception not derived from BaseException in _SelectorSslTransport.resume_reading 2014-05-27 21:24:43 +03:00
Guido van Rossum bf88ffba5e asyncio: Fix upstream issue 168: StreamReader.read(-1) from pipe may hang if data exceeds buffer limit. 2014-05-12 10:04:37 -07:00
Guido van Rossum 3d1bc608a8 asyncio: Upstream issue #167: remove dead code, by Marc Schlaich. 2014-05-10 15:47:15 -07:00
Guido van Rossum 3d139d8ed6 asyncio: Fix the second half of issue #21447: race in _write_to_self(). 2014-05-06 14:42:40 -07:00
Guido van Rossum 94ba146d11 asyncio: Add __weakref__ slots to Handle and CoroWrapper. Upstream issue #166. 2014-04-27 10:44:22 -07:00
Guido van Rossum 83c1ddda46 asyncio: Be careful accessing instance variables in __del__ (closes #21340). 2014-04-27 10:33:58 -07:00
Guido van Rossum 0cbc76880f asyncio: Add gi_{frame,running,code} properties to CoroWrapper (upstream #163). 2014-04-15 12:06:34 -07:00
Yury Selivanov 09cc169a03 asyncio.tasks: Make sure CoroWrapper.send proxies one argument correctly
Issue #21209.
2014-04-15 12:01:16 -04:00
Yury Selivanov f15f7484bb asyncio.tasks: Fix CoroWrapper to workaround yield-from bug in CPython < 3.4.1
Closes issue #21209.
2014-04-14 22:21:52 -04:00
Victor Stinner 1fd03a4a22 Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError if
path and sock are specified at the same time.
2014-04-07 11:18:54 +02:00
Victor Stinner 4bd652a276 asyncio: Document Task.cancel() properly. 2014-04-07 11:18:06 +02:00
Victor Stinner 93569c2b3d asyncio: Ensure call_soon(), call_later() and call_at() are invoked on current
loop in debug mode. Raise a RuntimeError if the event loop of the current
thread is different.  The check should help to debug thread-safetly issue.
Patch written by David Foster.
2014-03-21 10:00:52 +01:00
Victor Stinner e6a537976e asyncio, Tulip issue 157: Improve test_events.py, avoid run_briefly() which is
not reliable
2014-03-06 01:00:36 +01:00
Victor Stinner eeeebcd816 asyncio: Synchronize with Tulip
* Issue #159: Fix windows_utils.socketpair()

  - Use "127.0.0.1" (IPv4) or "::1" (IPv6) host instead of "localhost", because
    "localhost" may be a different IP address
  - Reject also invalid arguments: only AF_INET/AF_INET6 with SOCK_STREAM (and
    proto=0) are supported

* Reject add/remove reader/writer when event loop is closed.
* Fix ResourceWarning warnings
2014-03-06 00:52:53 +01:00
Victor Stinner d74ac82df9 asyncio, Tulip issue 158: Task._step() now also sets self to None if an
exception is raised.  self is set to None to break a reference cycle.
2014-03-04 23:07:08 +01:00
Victor Stinner c89c8a7be9 asyncio/windows_events.py: use more revelant names to overlapped callbacks
For example: "finish_recv", not just "finish".
2014-02-26 17:35:30 +01:00
Victor Stinner f5e37037cc asyncio: Fix pyflakes warnings: remove unused variables and imports 2014-02-26 11:07:42 +01:00
Victor Stinner 24ba203504 asyncio: Replace "unittest.mock" with "mock" in unit tests
Use "from unittest import mock". It should simplify my work to merge new tests
in Trollius, because Trollius uses "mock" backport for Python 2.
2014-02-26 10:25:02 +01:00
Victor Stinner a90e8edaea asyncio: _check_resolved_address() must also accept IPv6 without flow_info and
scope_id: (host, port).
2014-02-20 21:59:38 +01:00
Victor Stinner 013dece44d asyncio: Fix _check_resolved_address() for IPv6 address 2014-02-20 16:43:09 +01:00
Victor Stinner da492a8c39 asyncio: remove unused imports and unused variables noticed by pyflakes 2014-02-20 10:37:27 +01:00
Victor Stinner 2c7203c6f5 asyncio: Fix _ProactorWritePipeTransport._pipe_closed()
The "exc" variable was not defined, pass a BrokenPipeError exception instead.
2014-02-20 10:33:01 +01:00
Victor Stinner cdb476bd43 asyncio.subprocess: Fix a race condition in communicate()
Use self._loop instead of self._transport._loop, because transport._loop is set
to None at process exit.
2014-02-20 10:12:59 +01:00
Victor Stinner 7ef60cd8c2 asyncio, Tulip issue #136: Add get/set_debug() methods to BaseEventLoopTests.
Add also a PYTHONASYNCIODEBUG environment variable to debug coroutines since
Python startup, to be able to debug coroutines defined directly in the asyncio
module.
2014-02-19 23:15:02 +01:00
Yury Selivanov 2d01c0a080 asyncio: WriteTransport.set_write_buffer_size to call _maybe_pause_protocol 2014-02-19 11:10:52 -05:00
Yury Selivanov b41a42e316 asyncio: pep8-ify the code. 2014-02-18 22:56:15 -05:00
Yury Selivanov b0b0e628ee asyncio: Fix spelling and typos.
Thanks to Vajrasky Kok for discovering some of them.
2014-02-18 22:27:48 -05:00
Victor Stinner 884e40b982 asyncio, Tulip issue 143: UNIX domain methods, fix ResourceWarning and
DeprecationWarning warnings. create_unix_server() closes the socket on any
error, not only on OSError.
2014-02-19 01:45:59 +01:00
Victor Stinner 065ca25aae asyncio, Tulip issue 139: Improve error messages on "fatal errors"
Mention if the error was caused by a read or a write, and be more specific on
the object (ex: "pipe transport" instead of "transport").
2014-02-19 01:40:41 +01:00
Yury Selivanov c098241342 asyncio.transports: Make _ProactorBasePipeTransport use _FlowControlMixin 2014-02-18 18:41:13 -05:00
Yury Selivanov ff827f08ac asyncio: New error handling API. Issue #20681. 2014-02-18 18:02:19 -05:00
Guido van Rossum 59a5533028 asyncio: Make tests pass on Windows. 2014-02-18 10:24:30 -08:00
Yury Selivanov 88a5bf0b2e asyncio: Add support for UNIX Domain Sockets. 2014-02-18 12:15:06 -05:00
Victor Stinner 28773465e6 ayncio, Tulip issue 129: BaseEventLoop.sock_connect() now raises an error if
the address is not resolved (hostname instead of an IP address) for AF_INET and
AF_INET6 address families.
2014-02-13 09:24:37 +01:00
Guido van Rossum b58f053e48 asyncio: Change as_completed() to use a Queue, to avoid O(N**2) behavior. Fixes issue #20566. 2014-02-12 17:58:19 -08:00
Yury Selivanov ee6dc425c8 asyncio.events: Use __slots__ in Handle and TimerHandle 2014-02-12 17:01:52 -05:00
Victor Stinner 2371e25517 Issue #20505: Remove debug code 2014-02-11 17:53:47 +01:00
Victor Stinner eb74876e99 asyncio, Tulip issue 131: as_completed() and wait() now raises a TypeError if
the list of futures is not a list but a Future, Task or coroutine object
2014-02-11 11:54:08 +01:00
Victor Stinner 4e8d2f25e2 asyncio, Tulip issue 130: Add more checks on subprocess_exec/subprocess_shell
parameters
2014-02-11 11:44:56 +01:00
Victor Stinner a125497ea3 asyncio, Tulip issue 126: call_soon(), call_soon_threadsafe(), call_later(),
call_at() and run_in_executor() now raise a TypeError if the callback is a
coroutine function.
2014-02-11 11:34:30 +01:00
Victor Stinner 1db2ba3a92 Issue #20505: use also the monotonic time to decide if asyncio debug traces
should be printed
2014-02-11 10:26:53 +01:00
Victor Stinner 5d1ea04b06 Issue #20505: Oops, only print debug info if selector.select(timeout) took less
than timeout
2014-02-11 10:10:41 +01:00
Victor Stinner 7bff8e1e2b Issue #20505: Improve debug info in asyncio event loop 2014-02-11 10:08:08 +01:00
Victor Stinner 06847d9c8c Issue #20505: Fix TestLoop, set the clock resolution 2014-02-11 09:03:47 +01:00
Victor Stinner 6cf5c96630 Issue #20505: BaseEventLoop uses again the resolution of the clock to decide if
scheduled tasks should be executed or not.
2014-02-10 23:42:32 +01:00
Victor Stinner 1c143b19c8 Issue #20505: Add debug info to analyze sporaric failures of
test_timeout_rounding() on Windows XP buildbots.
2014-02-10 11:47:50 +01:00
Victor Stinner dc62b7e261 asyncio: Tulip issue 112: Inline make_handle() into Handle constructor 2014-02-10 00:45:44 +01:00
Guido van Rossum 09f08fe248 asyncio: Remove more relics of resolution/granularity. 2014-02-08 19:44:02 -08:00
Victor Stinner 0c3949c963 asyncio: Remove Process.subprocess attribute; it's too easy to get inconsistent
Process and Popen objects
2014-02-09 02:51:40 +01:00
Victor Stinner 85310a50a9 Issue #20505: Remove resolution and _granularity from selectors and asyncio
* Remove selectors.BaseSelector.resolution attribute
* Remove asyncio.BaseEventLoop._granularity attribute
2014-02-07 23:34:58 +01:00
Victor Stinner 3e7cc03f63 Issue #20505: add debug info 2014-02-07 17:53:13 +01:00
Yury Selivanov 622be340fd asyncio.tasks: Fix as_completed, gather & wait to work with duplicate coroutines 2014-02-06 22:06:16 -05:00
Yury Selivanov f317cb7a20 asyncio.tasks.gather: Fix docstring 2014-02-06 12:03:53 -05:00
Yury Selivanov f0020f5d77 asyncio.streams.StreamReader: Add 'at_eof()' method 2014-02-06 00:14:30 -05:00
Yury Selivanov e694c9745f asyncio.streams: Use bytebuffer in StreamReader; Add assertion in feed_data 2014-02-05 18:11:13 -05:00
Victor Stinner 83bdfa08f7 asyncio: Fix _ProactorWritePipeTransport._pipe_closed()
Do nothing if the pipe is already closed. _loop_writing() may call
_force_close() when it gets ConnectionResetError.
2014-02-04 08:57:48 +01:00
Victor Stinner b79eb0502c asyncio.subprocess: Replace Process.get_subprocess() method with a
Process.subprocess read-only property
2014-02-03 23:08:14 +01:00
Victor Stinner 915bcb0111 Issue #20400: Merge Tulip into Python: add the new asyncio.subprocess module
* Add a new asyncio.subprocess module
* Add new create_subprocess_exec() and create_subprocess_shell() functions
* The new asyncio.subprocess.SubprocessStreamProtocol creates stream readers
  for stdout and stderr and a stream writer for stdin.
* The new asyncio.subprocess.Process class offers an API close to the
  subprocess.Popen class:

  - pid, returncode, stdin, stdout and stderr attributes
  - communicate(), wait(), send_signal(), terminate() and kill() methods

* Remove STDIN (0), STDOUT (1) and STDERR (2) constants from base_subprocess
  and unix_events, to not be confused with the symbols with the same name of
  subprocess and asyncio.subprocess modules
* _ProactorBasePipeTransport.get_write_buffer_size() now counts also the size
  of the pending write
* _ProactorBaseWritePipeTransport._loop_writing() may now pause the protocol if
  the write buffer size is greater than the high water mark (64 KB by default)
2014-02-01 22:49:59 +01:00
Victor Stinner 1506df2655 Issue #20455: Add a resolution attribute to IocpProactor (1 ms) 2014-01-31 16:26:38 +01:00
Victor Stinner f2e1768bc1 Issue #20455: asyncio: use the same code to round a timeout than the selectors
module

Sort also imports
2014-01-31 16:25:24 +01:00
Victor Stinner b60e9ca69d Issue #20455: asyncio: write a new write pipe transport class for proactor (on
Windows) instead of using the "duplex" pipe transport. The new class uses a
simpler overlapped read to be notified when the pipe is closed. So the protocol
doesn't need to implement eof_received(): connection_lost() is called instead.
_UnixWritePipeTransport has the same approach.
2014-01-31 14:18:18 +01:00
Victor Stinner 61b3c9bacc asyncio: Fix _UnixWritePipeTransport, raise BrokenPipeError when the pipe is
closed, but only if there was pending write
2014-01-31 13:04:28 +01:00
Victor Stinner 49d0f4e428 Issue #20452: Remove debug code, no more needed 2014-01-31 12:59:43 +01:00
Victor Stinner 323748e1d1 asyncio: Fix error message in BaseEventLoop.subprocess_shell(). Patch written
by Vajrasky Kok.
2014-01-31 12:28:30 +01:00
Victor Stinner dcd9740ad2 Issue #20452: select and selectors round (again) timeout away from zero for
poll and epoll

Improve also debug info to analyze the issue
2014-01-31 12:12:53 +01:00
Victor Stinner 31f65044a9 Issue #20452: Oops, fix debug code :-/
Add also event more debug info
2014-01-31 10:55:55 +01:00
Victor Stinner 0278032110 Issue #20452: add more info in case of test_asyncio failure to try to debug the
failure on buildbot "x86 Ubuntu Shared 3.x"
2014-01-31 09:29:35 +01:00
Guido van Rossum a849be9c64 asyncio: Fix misc whitespace issues. 2014-01-30 16:05:28 -08:00
Victor Stinner 1c16537327 asyncio: Fix granularity of test_utils.TestLoop. 2014-01-30 16:05:07 -08:00
Victor Stinner 9572898282 asyncio: Future.set_exception(exc) should instantiate exc if it is a class. 2014-01-30 16:01:54 -08:00
Victor Stinner e623a12297 asyncio: subprocess_shell() and subprocess_exec() now raise ValueError instead of assert.
Moreover, bufsize different than 0 is now considered as an error.
2014-01-29 14:35:15 -08:00