getaddrinfo takes an exclusive lock on some platforms, causing clients to queue
up waiting for the lock if many names are being resolved concurrently. Users
may want to handle name resolution in their own code, for the sake of caching,
using an alternate resolver, or to measure DNS duration separately from
connection duration. Skip getaddrinfo if the "host" passed into
create_connection is already resolved.
See https://github.com/python/asyncio/pull/302 for details.
Patch by A. Jesse Jiryu Davis.
* _check_resolved_address() is implemented with getaddrinfo() which is slow
* If available, use socket.inet_pton() instead of socket.getaddrinfo(), because
it is much faster
Microbenchmark (timeit) on Fedora 21 (Python 3.4, Linux 3.17, glibc 2.20) to
validate the IPV4 address "127.0.0.1" or the IPv6 address "::1":
* getaddrinfo() 10.4 usec per loop
* inet_pton(): 0.285 usec per loop
On glibc older than 2.14, getaddrinfo() always requests the list of all local
IP addresses to the kernel (using a NETLINK socket). getaddrinfo() has other
known issues, it's better to avoid it when it is possible.
* Rephrase also the comment explaining why the waiter is not awaken immediatly.
* SSLProtocol.eof_received() doesn't instanciate ConnectionResetError exception
directly, it will be done by Future.set_exception(). The exception is not
used if the waiter was cancelled or if there is no waiter.
* Handle correctly CancelledError: just exit
* On error, log the exception and exit
Don't try to close the event loop, it is probably running and so it cannot be
closed.
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
(BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
state in the __repr__() method of _UnixReadPipeTransport and
_UnixWritePipeTransport classes.
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.
The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).
Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.
This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!
It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
Close the IocpProactor before closing the event loop. IocpProactor.close() can
call loop.call_soon(), which is forbidden when the event loop is closed.
* Tulip issue 184: FlowControlMixin constructor now get the event loop if the
loop parameter is not set. Add unit tests to ensure that constructor of
StreamReader and StreamReaderProtocol classes get the event loop.
* Remove outdated TODO/XXX
asyncio.BaseEventLoop now use the identifier of the current thread to ensure
that they are called from the thread running the event loop.
Before, the get_event_loop() method was used to check the thread, and no
exception was raised when the thread had no event loop. Now the methods always
raise an exception in debug mode when called from the wrong thread. It should
help to notice misusage of the API.
Move the _loop attribute from the constructor of _SelectorTransport,
_ProactorBasePipeTransport and _UnixWritePipeTransport classes to the
constructor of the _FlowControlMixin class.
Add also an assertion to explicit that the parent class must ensure that the
loop is defined (not None)
* PipeServer.close() now cancels the "accept pipe" future which cancels the
overlapped operation.
* Fix _SelectorTransport.__repr__() if the transport was closed
* Fix debug log in BaseEventLoop.create_connection(): get the socket object
from the transport because SSL transport closes the old socket and creates a
new SSL socket object. Remove also the _SelectorSslTransport._rawsock
attribute: it contained the closed socket (not very useful) and it was not
used.
* Issue #22063: socket operations (sock_recv, sock_sendall, sock_connect,
sock_accept) of the proactor event loop don't raise an exception in debug
mode if the socket are in blocking mode. Overlapped operations also work on
blocking sockets.
* Fix unit tests in debug mode: mock a non-blocking socket for socket
operations which now raise an exception if the socket is blocking.
* _fatal_error() method of _UnixReadPipeTransport and _UnixWritePipeTransport
now log all exceptions in debug mode
* Don't log expected errors in unit tests
* Tulip issue 200: _WaitHandleFuture._unregister_wait() now catchs and logs
exceptions.
* Tulip issue 200: Log errors in debug mode instead of simply ignoring them.
* Fix _WaitHandleFuture.cancel(): return the result of the parent cancel()
method.
* _OverlappedFuture.cancel() now clears its reference to the overlapped object.
Make also the _OverlappedFuture.ov attribute private.
* Check if _WaitHandleFuture completed before unregistering it in the callback.
Add also _WaitHandleFuture._poll() and repr(_WaitHandleFuture).
* _WaitHandleFuture now unregisters its wait handler if WaitForSingleObject()
raises an exception.
* _OverlappedFuture.set_exception() now cancels the overlapped operation.
Since Python 3.3, the C signal handler writes the signal number into the wakeup
file descriptor and then schedules the Python call using Py_AddPendingCall().
asyncio uses the wakeup file descriptor to wake up the event loop, and relies
on Py_AddPendingCall() to schedule the final callback with call_soon().
If the C signal handler is called in a thread different than the thread of the
event loop, the loop is awaken but Py_AddPendingCall() was not called yet. In
this case, the event loop has nothing to do and go to sleep again.
Py_AddPendingCall() is called while the event loop is sleeping again and so the
final callback is not scheduled immediatly.
This patch changes how asyncio handles signals. Instead of relying on
Py_AddPendingCall() and the wakeup file descriptor, asyncio now only relies on
the wakeup file descriptor. asyncio reads signal numbers from the wakeup file
descriptor to call its signal handler.
* Tulip issue #183: log socket events in debug mode
- Log most important socket events: socket connected, new client, connection
reset or closed by peer (EOF), etc.
- Log time elapsed in DNS resolution (getaddrinfo)
- Log pause/resume reading
- Log time of SSL handshake
- Log SSL handshake errors
- Add a __repr__() method to many classes
* Fix ProactorEventLoop() in debug mode. ProactorEventLoop._make_self_pipe()
doesn't call call_soon() directly because it checks for the current loop
which fails, because the method is called to build the event loop.
* Cleanup _ProactorReadPipeTransport constructor. Not need to set again
_read_fut attribute to None, it is already done in the base class.
- loop, waiters and active_count attributes are now private
- attach(), detach() and wakeup() methods are now private
The sockets attribute remains public.
- Tulip issue #181: Faster create_connection(). Call directly
waiter.set_result() in the constructor of _ProactorBasePipeTransport and
_SelectorSocketTransport, instead of using of delaying the call with
call_soon().
- Cleanup iscoroutine()
Add BaseEventLoop._closed attribute and use it to check if the event loop was
closed or not, instead of checking different attributes in each subclass of
BaseEventLoop.
run_forever() and run_until_complete() methods now raise a RuntimeError('Event loop is
closed') exception if the event loop was closed.
BaseProactorEventLoop.close() now also cancels "accept futures".
* Add a new asyncio.subprocess module
* Add new create_subprocess_exec() and create_subprocess_shell() functions
* The new asyncio.subprocess.SubprocessStreamProtocol creates stream readers
for stdout and stderr and a stream writer for stdin.
* The new asyncio.subprocess.Process class offers an API close to the
subprocess.Popen class:
- pid, returncode, stdin, stdout and stderr attributes
- communicate(), wait(), send_signal(), terminate() and kill() methods
* Remove STDIN (0), STDOUT (1) and STDERR (2) constants from base_subprocess
and unix_events, to not be confused with the symbols with the same name of
subprocess and asyncio.subprocess modules
* _ProactorBasePipeTransport.get_write_buffer_size() now counts also the size
of the pending write
* _ProactorBaseWritePipeTransport._loop_writing() may now pause the protocol if
the write buffer size is greater than the high water mark (64 KB by default)
Windows) instead of using the "duplex" pipe transport. The new class uses a
simpler overlapped read to be notified when the pipe is closed. So the protocol
doesn't need to implement eof_received(): connection_lost() is called instead.
_UnixWritePipeTransport has the same approach.
Major changes:
- StreamReader.readexactly() now raises an IncompleteReadError if the
end of stream is reached before we received enough bytes, instead of
returning less bytes than requested.
- Unit tests use the main asyncio module instead of submodules like events
- _UnixWritePipeTransport now also supports character devices, as
_UnixReadPipeTransport. Patch written by Jonathan Slenders.
- Export more symbols: BaseEventLoop, BaseProactorEventLoop,
BaseSelectorEventLoop, Queue and Queue sublasses, Empty, Full
* store the "self reading" future when the "self pipe" is closed (when the
event loop is closed)
* store "accept" futures to cancel them when we stop serving
* close the "accept socket" if the "accept future" is cancelled
Fix many warnings which can be seen when unit tests are run in verbose mode.
Yury Selivanov
Yury Selivanov
Victor Stinner
Guido van Rossum