Commit Graph

302 Commits

Author SHA1 Message Date
Victor Stinner 2b77c5467f asyncio, Tulip issue 204: Fix IocpProactor.accept_pipe()
Overlapped.ConnectNamedPipe() now returns a boolean: True if the pipe is
connected (if ConnectNamedPipe() failed with ERROR_PIPE_CONNECTED), False if
the connection is in progress.

This change removes multiple hacks in IocpProactor.
2015-01-22 23:50:03 +01:00
Victor Stinner 7ffa2c5fdd Issue #23293, asyncio: Rewrite IocpProactor.connect_pipe()
Add _overlapped.ConnectPipe() which tries to connect to the pipe for
asynchronous I/O (overlapped): call CreateFile() in a loop until it doesn't
fail with ERROR_PIPE_BUSY. Use an increasing delay between 1 ms and 100 ms.

Remove Overlapped.WaitNamedPipeAndConnect() which is no more used.
2015-01-22 22:55:08 +01:00
Victor Stinner 752aba7f99 asyncio: IocpProactor.close() doesn't cancel anymore futures which are already
cancelled
2015-01-22 22:47:13 +01:00
Victor Stinner 1ca9392c70 Issue #23095, asyncio: IocpProactor.close() must not cancel pending
_WaitCancelFuture futures
2015-01-22 00:17:54 +01:00
Victor Stinner 0c2e408833 asyncio: BaseEventLoop._create_connection_transport() catchs any exception, not
only Exception
2015-01-22 00:17:41 +01:00
Victor Stinner d0a28dee78 Issue #23095, asyncio: Rewrite _WaitHandleFuture.cancel()
This change fixes a race conditon related to _WaitHandleFuture.cancel() leading
to Python crash or "GetQueuedCompletionStatus() returned an unexpected event"
logs. Before, the overlapped object was destroyed too early, it was possible
that the wait completed whereas the overlapped object was already destroyed.
Sometimes, a different overlapped was allocated at the same address, leading to
unexpected completition.

_WaitHandleFuture.cancel() now waits until the wait is cancelled to clear its
reference to the overlapped object. To wait until the cancellation is done,
UnregisterWaitEx() is used with an event instead of UnregisterWait().

To wait for this event, a new _WaitCancelFuture class was added. It's a
simplified version of _WaitCancelFuture. For example, its cancel() method calls
UnregisterWait(), not UnregisterWaitEx(). _WaitCancelFuture should not be
cancelled.

The overlapped object is kept alive in _WaitHandleFuture until the wait is
unregistered.

Other changes:

* Add _overlapped.UnregisterWaitEx()
* Remove fast-path in IocpProactor.wait_for_handle() to immediatly set the
  result if the wait already completed. I'm not sure that it's safe to
  call immediatly UnregisterWaitEx() before the completion was signaled.
* Add IocpProactor._unregistered() to forget an overlapped which may never be
  signaled, but may be signaled for the next loop iteration. It avoids to
  block forever IocpProactor.close() if a wait was cancelled, and it may also
  avoid some "... unexpected event ..." warnings.
2015-01-21 23:39:51 +01:00
Victor Stinner 442b0adccd asyncio: pyflakes, remove unused import
tests: Remove unused function; inline another function
2015-01-21 23:39:16 +01:00
Victor Stinner cd0f7f9832 asyncio: Enhance BaseProactorEventLoop._loop_self_reading()
* Handle correctly CancelledError: just exit
* On error, log the exception and exit

Don't try to close the event loop, it is probably running and so it cannot be
closed.
2015-01-21 23:38:37 +01:00
Victor Stinner 406204c8c2 Backout changeset 6ab2575bc12b
StreamWriter: close() now clears the reference to the transport

StreamWriter now raises an exception if it is closed: write(), writelines(),
write_eof(), can_write_eof(), get_extra_info(), drain().
2015-01-15 21:50:19 +01:00
Victor Stinner 922bc2ca12 Closes #23219: cancelling asyncio.wait_for() now cancels the task 2015-01-15 16:29:10 +01:00
Victor Stinner 4bf22e033e asyncio: Close the transport on subprocess creation failure 2015-01-15 14:24:22 +01:00
Victor Stinner fcd58de78f asyncio: Fix _ProactorBasePipeTransport.close()
Set the _read_fut attribute to None after cancelling it.

This change should fix a race condition with
_ProactorWritePipeTransport._pipe_closed().
2015-01-15 13:40:27 +01:00
Victor Stinner 79c93ba47b asyncio: Fix _ProactorBasePipeTransport.__repr__()
Check if the _sock attribute is None to check if the transport is closed.
2015-01-15 13:32:28 +01:00
Victor Stinner 41ed958ee6 Issue #23243: Fix asyncio._UnixWritePipeTransport.close()
Do nothing if the transport is already closed. Before it was not possible to
close the transport twice.
2015-01-15 13:16:50 +01:00
Victor Stinner 7e222f411c SSLProtocol: set the _transport attribute in the constructor 2015-01-15 13:16:27 +01:00
Victor Stinner 791009bb89 Issue #23242: asyncio.SubprocessStreamProtocol now closes the subprocess
transport at subprocess exit.

Clear also its reference to the transport.
2015-01-15 13:16:02 +01:00
Victor Stinner 72bdefb086 Issue #22560: Fix typo: call -> call_soon 2015-01-15 09:44:13 +01:00
Victor Stinner 042dad7232 Issue #22560: Fix SSLProtocol._on_handshake_complete()
Don't call immediatly self._process_write_backlog() but schedule the call using
call_soon(). _on_handshake_complete() can be called indirectly from
_process_write_backlog(), and _process_write_backlog() is not reentrant.
2015-01-15 09:41:48 +01:00
Victor Stinner e7a35717d2 StreamWriter: close() now clears the reference to the transport
StreamWriter now raises an exception if it is closed: write(), writelines(),
write_eof(), can_write_eof(), get_extra_info(), drain().
2015-01-15 09:33:50 +01:00
Victor Stinner 29ad0111bd asyncio: sync with Tulip
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
  ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
  creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
  closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
  not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
  (BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
  readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
  close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
  state in the __repr__() method of _UnixReadPipeTransport and
  _UnixWritePipeTransport classes.
2015-01-15 00:04:21 +01:00
Victor Stinner b92626df5c Issue #23197: On SSL handshake failure on matching hostname, check if the
waiter is cancelled before setting its exception.
2015-01-14 17:13:28 +01:00
Victor Stinner 177e9f0855 Issue #23197, asyncio: On SSL handshake failure, check if the waiter is
cancelled before setting its exception.

* Add unit tests for this case.
* Cleanup also sslproto.py
2015-01-14 16:56:20 +01:00
Victor Stinner f651a60407 Python issue #23173: sync with Tulip
* If an exception is raised during the creation of a subprocess, kill the
  subprocess (close pipes, kill and read the return status). Log an error in
  such case.
* Fix SubprocessStreamProtocol.connection_made() to handle cancelled waiter.
  Add unit test cancelling subprocess methods.
2015-01-14 02:10:33 +01:00
Victor Stinner c2c12e433a Issue #23198: Reactor asyncio.StreamReader
- Add a new _wakeup_waiter() method
- Replace _create_waiter() method with a _wait_for_data() coroutine function
- Use the value None instead of True or False to wake up the waiter
2015-01-14 00:53:37 +01:00
Victor Stinner 231b404cb0 Issue #22560: New SSL implementation based on ssl.MemoryBIO
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.

The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).

Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.

This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!

It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
2015-01-14 00:19:09 +01:00
Victor Stinner b86a96802b Issue #22922: Fix ProactorEventLoop.close()
Close the IocpProactor before closing the event loop. IocpProactor.close() can
call loop.call_soon(), which is forbidden when the event loop is closed.
2015-01-13 16:11:19 +01:00
Victor Stinner 587feb19e8 Issue #23209: Break some reference cycles in asyncio. Patch written by Martin
Richard.
2015-01-09 21:34:27 +01:00
Victor Stinner 70db9e428a asyncio: sync with Tulip
* Tulip issue 184: FlowControlMixin constructor now get the event loop if the
  loop parameter is not set. Add unit tests to ensure that constructor of
  StreamReader and StreamReaderProtocol classes get the event loop.
* Remove outdated TODO/XXX
2015-01-09 21:32:05 +01:00
Victor Stinner 3531d9044d asyncio: sync with Tulip
* Document why set_result() calls are safe
* Cleanup gather(). Use public methods instead of hacks to consume the
  exception of a future.
* sock_connect(): pass directly the fd to _sock_connect_done instead of the
  socket.
2015-01-09 01:42:52 +01:00
Victor Stinner 15cc678d89 asyncio: Truncate to 80 columns 2015-01-09 00:09:10 +01:00
Victor Stinner 8d9c145f61 asyncio: _make_ssl_transport: make the waiter parameter optional 2015-01-08 12:06:36 +01:00
Victor Stinner c447ba04e7 Issue #23140, asyncio: Fix cancellation of Process.wait(). Check the state of
the waiter future before setting its result.
2015-01-06 01:13:49 +01:00
Victor Stinner 8c1a4a2326 Issue #23046: Expose the BaseEventLoop class in the asyncio namespace 2015-01-06 01:03:58 +01:00
Victor Stinner 956de691f8 Issue #22926: In debug mode, call_soon(), call_at() and call_later() methods of
asyncio.BaseEventLoop now use the identifier of the current thread to ensure
that they are called from the thread running the event loop.

Before, the get_event_loop() method was used to check the thread, and no
exception was raised when the thread had no event loop. Now the methods always
raise an exception in debug mode when called from the wrong thread. It should
help to notice misusage of the API.
2014-12-26 21:07:52 +01:00
Victor Stinner 952ec98abb asyncio doc: update also Queue docstrings 2014-12-22 22:09:50 +01:00
Victor Stinner 4d825b45a0 asyncio: IocpProactor.wait_for_handle() test now also checks the result of the
future
2014-12-19 17:10:44 +01:00
Victor Stinner 1b9763d0a9 asyncio: sync with Tulip
* Fix a race condition in BaseSubprocessTransport._try_finish().

  If the process exited before the _post_init() method was called, scheduling
  the call to _call_connection_lost() with call_soon() is wrong:
  connection_made() must be called before connection_lost().

  Reuse the BaseSubprocessTransport._call() method to schedule the call to
  _call_connection_lost() to ensure that connection_made() and
  connection_lost() are called in the correct order.

* Add repr(PipeHandle)

* Fix typo
2014-12-18 23:47:27 +01:00
Victor Stinner 3a1c738e6c Issue #23074: asyncio.get_event_loop() now raises an exception if the thread
has no event loop even if assertions are disabled.
2014-12-18 01:20:10 +01:00
Victor Stinner 1e40f10886 asyncio, tulip issue 209: Fix subprocess for close_fds=False on Python 3.3
Mark the write end of the stdin pipe as non-inheritable.
2014-12-11 23:30:17 +01:00
Victor Stinner df75d5b402 asyncio, tulip issue 202: Add unit test of pause/resume writing for proactor
socket transport
2014-12-11 22:23:19 +01:00
Victor Stinner f3e2e09213 Closes #22429, asyncio: Fix EventLoop.run_until_complete(), don't stop the
event loop if a BaseException is raised, because the event loop is already
stopped.
2014-12-05 01:44:10 +01:00
Victor Stinner 4c85ec99f3 Issue #22922: Fix ProactorEventLoop.close()
Call _stop_accept_futures() before sestting the _closed attribute, otherwise
call_soon() raises an error.
2014-12-05 01:43:42 +01:00
Victor Stinner e80bf0d4a9 Closes #22922: More EventLoop methods fail if the loop is closed. Initial patch
written by Torsten Landschoff.

create_task(), call_at(), call_soon(), call_soon_threadsafe() and
run_in_executor() now raise an error if the event loop is closed.
2014-12-04 23:07:47 +01:00
Victor Stinner fe22e0985a asyncio: Initialize more Future and Task attributes in the class definition to
avoid attribute errors in destructors.
2014-12-04 23:00:13 +01:00
Victor Stinner d87de83582 Closes #22475: asyncio doc, fix Task.get_stack() doc 2014-12-02 17:57:04 +01:00
Serhiy Storchaka 56a6d855e2 Removed duplicated words in in comments and docs. 2014-12-01 18:28:43 +02:00
Victor Stinner 5ef586f25a Closes #22685, asyncio: Set the transport of stdout and stderr StreamReader
objects in the SubprocessStreamProtocol. It allows to pause the transport to
not buffer too much stdout or stderr data.
2014-11-25 17:20:33 +01:00
Benjamin Peterson 7243b574e5 don't require OpenSSL SNI to pass hostname to ssl functions (#22921)
Patch by Donald Stufft.
2014-11-23 17:04:34 -06:00
Victor Stinner 5e63120f8d asyncio: BaseSelectorEventLoop.close() now closes the self-pipe before calling
the parent close() method. If the event loop is already closed, the self-pipe
is not unregistered from the selector.
2014-11-21 00:23:27 +01:00
Victor Stinner 2d99d93d11 asyncio: Coroutine objects are now rejected with a TypeError by the following
functions:

* add_signal_handler()
* call_at()
* call_later()
* call_soon()
* call_soon_threadsafe()
* run_in_executor()

Fix also the error message of add_signal_handler() (fix the name of the
function).
2014-11-20 15:03:52 +01:00