Commit Graph

94 Commits

Author SHA1 Message Date
Victor Stinner f07801bb17 asyncio: SSL transports now clear their reference to the waiter
* Rephrase also the comment explaining why the waiter is not awaken immediatly.
* SSLProtocol.eof_received() doesn't instanciate ConnectionResetError exception
  directly, it will be done by Future.set_exception(). The exception is not
  used if the waiter was cancelled or if there is no waiter.
2015-01-29 00:36:35 +01:00
Victor Stinner cd0f7f9832 asyncio: Enhance BaseProactorEventLoop._loop_self_reading()
* Handle correctly CancelledError: just exit
* On error, log the exception and exit

Don't try to close the event loop, it is probably running and so it cannot be
closed.
2015-01-21 23:38:37 +01:00
Victor Stinner fcd58de78f asyncio: Fix _ProactorBasePipeTransport.close()
Set the _read_fut attribute to None after cancelling it.

This change should fix a race condition with
_ProactorWritePipeTransport._pipe_closed().
2015-01-15 13:40:27 +01:00
Victor Stinner 79c93ba47b asyncio: Fix _ProactorBasePipeTransport.__repr__()
Check if the _sock attribute is None to check if the transport is closed.
2015-01-15 13:32:28 +01:00
Victor Stinner 29ad0111bd asyncio: sync with Tulip
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
  ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
  creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
  closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
  not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
  (BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
  readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
  close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
  state in the __repr__() method of _UnixReadPipeTransport and
  _UnixWritePipeTransport classes.
2015-01-15 00:04:21 +01:00
Victor Stinner 231b404cb0 Issue #22560: New SSL implementation based on ssl.MemoryBIO
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.

The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).

Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.

This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!

It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
2015-01-14 00:19:09 +01:00
Victor Stinner b86a96802b Issue #22922: Fix ProactorEventLoop.close()
Close the IocpProactor before closing the event loop. IocpProactor.close() can
call loop.call_soon(), which is forbidden when the event loop is closed.
2015-01-13 16:11:19 +01:00
Victor Stinner 70db9e428a asyncio: sync with Tulip
* Tulip issue 184: FlowControlMixin constructor now get the event loop if the
  loop parameter is not set. Add unit tests to ensure that constructor of
  StreamReader and StreamReaderProtocol classes get the event loop.
* Remove outdated TODO/XXX
2015-01-09 21:32:05 +01:00
Victor Stinner 956de691f8 Issue #22926: In debug mode, call_soon(), call_at() and call_later() methods of
asyncio.BaseEventLoop now use the identifier of the current thread to ensure
that they are called from the thread running the event loop.

Before, the get_event_loop() method was used to check the thread, and no
exception was raised when the thread had no event loop. Now the methods always
raise an exception in debug mode when called from the wrong thread. It should
help to notice misusage of the API.
2014-12-26 21:07:52 +01:00
Victor Stinner df75d5b402 asyncio, tulip issue 202: Add unit test of pause/resume writing for proactor
socket transport
2014-12-11 22:23:19 +01:00
Victor Stinner 4c85ec99f3 Issue #22922: Fix ProactorEventLoop.close()
Call _stop_accept_futures() before sestting the _closed attribute, otherwise
call_soon() raises an error.
2014-12-05 01:43:42 +01:00
Victor Stinner 004adb91f6 asyncio: Move loop attribute to _FlowControlMixin
Move the _loop attribute from the constructor of _SelectorTransport,
_ProactorBasePipeTransport and _UnixWritePipeTransport classes to the
constructor of the _FlowControlMixin class.

Add also an assertion to explicit that the parent class must ensure that the
loop is defined (not None)
2014-11-05 15:27:41 +01:00
Victor Stinner 0e34dc3737 asyncio: enhance protocol representation
Add "closed" or "closing" to repr() of selector and proactor transports
2014-10-12 09:52:11 +02:00
Victor Stinner b261475a48 asyncio: sync with Tulip
* PipeServer.close() now cancels the "accept pipe" future which cancels the
  overlapped operation.
* Fix _SelectorTransport.__repr__() if the transport was closed
* Fix debug log in BaseEventLoop.create_connection(): get the socket object
  from the transport because SSL transport closes the old socket and creates a
  new SSL socket object. Remove also the _SelectorSslTransport._rawsock
  attribute: it contained the closed socket (not very useful) and it was not
  used.
* Issue #22063: socket operations (sock_recv, sock_sendall, sock_connect,
  sock_accept) of the proactor event loop don't raise an exception in debug
  mode if the socket are in blocking mode. Overlapped operations also work on
  blocking sockets.
* Fix unit tests in debug mode: mock a non-blocking socket for socket
  operations which now raise an exception if the socket is blocking.
* _fatal_error() method of _UnixReadPipeTransport and _UnixWritePipeTransport
  now log all exceptions in debug mode
* Don't log expected errors in unit tests
* Tulip issue 200: _WaitHandleFuture._unregister_wait() now catchs and logs
  exceptions.
* Tulip issue 200: Log errors in debug mode instead of simply ignoring them.
2014-08-25 23:20:52 +02:00
Victor Stinner 9c9f1f10d3 Close #22063: socket operations (socket,recv, sock_sendall, sock_connect,
sock_accept) now raise an exception in debug mode if sockets are in blocking
mode.
2014-07-29 23:08:17 +02:00
Victor Stinner 18a28dc5c2 asyncio: sync with Tulip
* Fix _WaitHandleFuture.cancel(): return the result of the parent cancel()
  method.
* _OverlappedFuture.cancel() now clears its reference to the overlapped object.
  Make also the _OverlappedFuture.ov attribute private.
* Check if _WaitHandleFuture completed before unregistering it in the callback.
  Add also _WaitHandleFuture._poll() and repr(_WaitHandleFuture).
* _WaitHandleFuture now unregisters its wait handler if WaitForSingleObject()
  raises an exception.
* _OverlappedFuture.set_exception() now cancels the overlapped operation.
2014-07-25 13:05:20 +02:00
Victor Stinner fe5649c7b7 Python issue #21645, Tulip issue 192: Rewrite signal handling
Since Python 3.3, the C signal handler writes the signal number into the wakeup
file descriptor and then schedules the Python call using Py_AddPendingCall().

asyncio uses the wakeup file descriptor to wake up the event loop, and relies
on Py_AddPendingCall() to schedule the final callback with call_soon().

If the C signal handler is called in a thread different than the thread of the
event loop, the loop is awaken but Py_AddPendingCall() was not called yet. In
this case, the event loop has nothing to do and go to sleep again.
Py_AddPendingCall() is called while the event loop is sleeping again and so the
final callback is not scheduled immediatly.

This patch changes how asyncio handles signals. Instead of relying on
Py_AddPendingCall() and the wakeup file descriptor, asyncio now only relies on
the wakeup file descriptor. asyncio reads signal numbers from the wakeup file
descriptor to call its signal handler.
2014-07-17 22:43:40 +02:00
Victor Stinner e912e652f8 asyncio: sync with Tulip
* Tulip issue #183: log socket events in debug mode

  - Log most important socket events: socket connected, new client, connection
    reset or closed by peer (EOF), etc.
  - Log time elapsed in DNS resolution (getaddrinfo)
  - Log pause/resume reading
  - Log time of SSL handshake
  - Log SSL handshake errors
  - Add a __repr__() method to many classes

* Fix ProactorEventLoop() in debug mode. ProactorEventLoop._make_self_pipe()
  doesn't call call_soon() directly because it checks for the current loop
  which fails, because the method is called to build the event loop.

* Cleanup _ProactorReadPipeTransport constructor. Not need to set again
  _read_fut attribute to None, it is already done in the base class.
2014-07-12 03:11:53 +02:00
Victor Stinner b28dbac86d asyncio, Tulip issue 180: Make Server attributes and methods private
- loop, waiters and active_count attributes are now private
- attach(), detach() and wakeup() methods are now private

The sockets attribute remains public.
2014-07-11 22:52:21 +02:00
Victor Stinner bfff45d611 asyncion, Tulip issue 181: BaseEventLoop.create_datagram_endpoint() now waits
until protocol.connection_made() has been called. Document also why transport
constructors use a waiter.
2014-07-08 23:57:31 +02:00
Victor Stinner 799a60ccb4 asyncio: sync with Tulip
Backout the "Tulip issue 181: Faster create_connection()" changeset, it was a
mistake.
2014-07-07 18:08:22 +02:00
Victor Stinner 1a870c9132 asyncio: sync with Tulip
- Tulip issue #181: Faster create_connection(). Call directly
  waiter.set_result() in the constructor of _ProactorBasePipeTransport and
  _SelectorSocketTransport, instead of using of delaying the call with
  call_soon().
- Cleanup iscoroutine()
2014-07-07 17:26:54 +02:00
Victor Stinner a9acbe82e7 Closes #21886, #21447: Fix a race condition in asyncio when setting the result
of a Future with call_soon(). Add an helper, a private method, to set the
result only if the future was not cancelled.
2014-07-05 15:29:41 +02:00
Victor Stinner f328c7dc69 asyncio, Tulip issue 171: BaseEventLoop.close() now raises an exception if the
event loop is running. You must first stop the event loop and then wait until
it stopped, before closing it.
2014-06-23 01:02:37 +02:00
Victor Stinner bb2fc5b2a5 Issue #21326: Add a new is_closed() method to asyncio.BaseEventLoop
Add BaseEventLoop._closed attribute and use it to check if the event loop was
closed or not, instead of checking different attributes in each subclass of
BaseEventLoop.

run_forever() and run_until_complete() methods now raise a RuntimeError('Event loop is
closed') exception if the event loop was closed.

BaseProactorEventLoop.close() now also cancels "accept futures".
2014-06-10 10:23:10 +02:00
Victor Stinner 6f24d83ac6 asyncio: Fix _ProactorWritePipeTransport._pipe_closed()
The "exc" variable was not defined, pass a BrokenPipeError exception instead.
2014-02-20 10:33:01 +01:00
Victor Stinner 0ee29c2c7f asyncio, Tulip issue 139: Improve error messages on "fatal errors"
Mention if the error was caused by a read or a write, and be more specific on
the object (ex: "pipe transport" instead of "transport").
2014-02-19 01:40:41 +01:00
Yury Selivanov 3cb9914488 asyncio.transports: Make _ProactorBasePipeTransport use _FlowControlMixin 2014-02-18 18:41:13 -05:00
Yury Selivanov 569efa2e4b asyncio: New error handling API. Issue #20681. 2014-02-18 18:02:19 -05:00
Victor Stinner 1b0580b320 ayncio, Tulip issue 129: BaseEventLoop.sock_connect() now raises an error if
the address is not resolved (hostname instead of an IP address) for AF_INET and
AF_INET6 address families.
2014-02-13 09:24:37 +01:00
Victor Stinner 85310a50a9 Issue #20505: Remove resolution and _granularity from selectors and asyncio
* Remove selectors.BaseSelector.resolution attribute
* Remove asyncio.BaseEventLoop._granularity attribute
2014-02-07 23:34:58 +01:00
Victor Stinner 83bdfa08f7 asyncio: Fix _ProactorWritePipeTransport._pipe_closed()
Do nothing if the pipe is already closed. _loop_writing() may call
_force_close() when it gets ConnectionResetError.
2014-02-04 08:57:48 +01:00
Victor Stinner 915bcb0111 Issue #20400: Merge Tulip into Python: add the new asyncio.subprocess module
* Add a new asyncio.subprocess module
* Add new create_subprocess_exec() and create_subprocess_shell() functions
* The new asyncio.subprocess.SubprocessStreamProtocol creates stream readers
  for stdout and stderr and a stream writer for stdin.
* The new asyncio.subprocess.Process class offers an API close to the
  subprocess.Popen class:

  - pid, returncode, stdin, stdout and stderr attributes
  - communicate(), wait(), send_signal(), terminate() and kill() methods

* Remove STDIN (0), STDOUT (1) and STDERR (2) constants from base_subprocess
  and unix_events, to not be confused with the symbols with the same name of
  subprocess and asyncio.subprocess modules
* _ProactorBasePipeTransport.get_write_buffer_size() now counts also the size
  of the pending write
* _ProactorBaseWritePipeTransport._loop_writing() may now pause the protocol if
  the write buffer size is greater than the high water mark (64 KB by default)
2014-02-01 22:49:59 +01:00
Victor Stinner 1506df2655 Issue #20455: Add a resolution attribute to IocpProactor (1 ms) 2014-01-31 16:26:38 +01:00
Victor Stinner b60e9ca69d Issue #20455: asyncio: write a new write pipe transport class for proactor (on
Windows) instead of using the "duplex" pipe transport. The new class uses a
simpler overlapped read to be notified when the pipe is closed. So the protocol
doesn't need to implement eof_received(): connection_lost() is called instead.
_UnixWritePipeTransport has the same approach.
2014-01-31 14:18:18 +01:00
Victor Stinner 63b4d4b494 asyncio: _fatal_error() of _UnixWritePipeTransport and _ProactorBasePipeTransport shouldn't log BrokenPipeError nor ConnectionResetError.
(Same behaviour as _SelectorTransport._fatal_error().)
2014-01-29 13:12:03 -08:00
Victor Stinner 8dffc456d7 Update asyncio from the Tulip project
Major changes:

- StreamReader.readexactly() now raises an IncompleteReadError if the
  end of stream is reached before we received enough bytes, instead of
  returning less bytes than requested.

- Unit tests use the main asyncio module instead of submodules like events

- _UnixWritePipeTransport now also supports character devices, as
  _UnixReadPipeTransport. Patch written by Jonathan Slenders.

- Export more symbols: BaseEventLoop, BaseProactorEventLoop,
  BaseSelectorEventLoop, Queue and Queue sublasses, Empty, Full
2014-01-25 15:32:06 +01:00
Victor Stinner 7de2646cdf Cleanup properly proactor event loop
* store the "self reading" future when the "self pipe" is closed (when the
  event loop is closed)
* store "accept" futures to cancel them when we stop serving
* close the "accept socket" if the "accept future" is cancelled

Fix many warnings which can be seen when unit tests are run in verbose mode.
2014-01-11 00:03:21 +01:00
Guido van Rossum ebb8e58f0a asyncio: Write flow control for proactor event loop. 2013-12-04 12:12:07 -08:00
Guido van Rossum 5969128a86 asyncio: Add support for running subprocesses on Windows with the IOCP event loop (Richard Oudkerk). 2013-10-30 14:52:03 -07:00
Antoine Pitrou 4ca7355901 Issue #19299: fix refleak test failures in test_asyncio 2013-10-20 00:54:10 +02:00
Guido van Rossum 57497ad181 Rename Transport.pause/resume to pause_reading/pause_writing. Also relax timeout in test_call_later(). 2013-10-18 07:58:20 -07:00
Guido van Rossum fc29e0f37e Rename the logger to plain "logger". 2013-10-17 15:39:45 -07:00
Guido van Rossum 27b7c7ebf1 Initial checkin of asyncio package (== Tulip, == PEP 3156). 2013-10-17 13:40:50 -07:00