Issue #26604:
* Add a new optional source parameter to _warnings.warn() and warnings.warn()
* Modify asyncore, asyncio and _pyio modules to set the source parameter when
logging a ResourceWarning warning
getaddrinfo takes an exclusive lock on some platforms, causing clients to queue
up waiting for the lock if many names are being resolved concurrently. Users
may want to handle name resolution in their own code, for the sake of caching,
using an alternate resolver, or to measure DNS duration separately from
connection duration. Skip getaddrinfo if the "host" passed into
create_connection is already resolved.
See https://github.com/python/asyncio/pull/302 for details.
Patch by A. Jesse Jiryu Davis.
This info is required on Python 3.5 and newer to get specific information on
the SSL object, like getting the binary peer certificate (instead of getting
it as text).
again if the first call to connect() raises an InterruptedError.
When the C function connect() fails with EINTR, the connection runs in
background. We have to wait until the socket becomes writable to be notified
when the connection succeed or fails.
* _check_resolved_address() is implemented with getaddrinfo() which is slow
* If available, use socket.inet_pton() instead of socket.getaddrinfo(), because
it is much faster
Microbenchmark (timeit) on Fedora 21 (Python 3.4, Linux 3.17, glibc 2.20) to
validate the IPV4 address "127.0.0.1" or the IPv6 address "::1":
* getaddrinfo() 10.4 usec per loop
* inet_pton(): 0.285 usec per loop
On glibc older than 2.14, getaddrinfo() always requests the list of all local
IP addresses to the kernel (using a NETLINK socket). getaddrinfo() has other
known issues, it's better to avoid it when it is possible.
* Cleanup gather(): use cancelled() method instead of using private Future
attribute
* Fix _UnixReadPipeTransport and _UnixWritePipeTransport. Only start reading
when connection_made() has been called.
* Issue #23333: Fix BaseSelectorEventLoop._accept_connection(). Close the
transport on error. In debug mode, log errors using call_exception_handler()
* _SelectorTransport constructor: extra parameter is now optional
* Fix _SelectorDatagramTransport constructor. Only start reading after
connection_made() has been called.
* Fix _SelectorSslTransport.close(). Don't call protocol.connection_lost() if
protocol.connection_made() was not called yet: if the SSL handshake failed or
is still in progress. The close() method can be called if the creation of the
connection is cancelled, by a timeout for example.
* Rephrase also the comment explaining why the waiter is not awaken immediatly.
* SSLProtocol.eof_received() doesn't instanciate ConnectionResetError exception
directly, it will be done by Future.set_exception(). The exception is not
used if the waiter was cancelled or if there is no waiter.
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
(BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
state in the __repr__() method of _UnixReadPipeTransport and
_UnixWritePipeTransport classes.
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.
The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).
Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.
This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!
It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
* Tulip issue 184: FlowControlMixin constructor now get the event loop if the
loop parameter is not set. Add unit tests to ensure that constructor of
StreamReader and StreamReaderProtocol classes get the event loop.
* Remove outdated TODO/XXX
* Document why set_result() calls are safe
* Cleanup gather(). Use public methods instead of hacks to consume the
exception of a future.
* sock_connect(): pass directly the fd to _sock_connect_done instead of the
socket.
asyncio.BaseEventLoop now use the identifier of the current thread to ensure
that they are called from the thread running the event loop.
Before, the get_event_loop() method was used to check the thread, and no
exception was raised when the thread had no event loop. Now the methods always
raise an exception in debug mode when called from the wrong thread. It should
help to notice misusage of the API.
Move the _loop attribute from the constructor of _SelectorTransport,
_ProactorBasePipeTransport and _UnixWritePipeTransport classes to the
constructor of the _FlowControlMixin class.
Add also an assertion to explicit that the parent class must ensure that the
loop is defined (not None)
There is a race condition in create_connection() used with wait_for() to have a
timeout. sock_connect() registers the file descriptor of the socket to be
notified of write event (if connect() raises BlockingIOError). When
create_connection() is cancelled with a TimeoutError, sock_connect() coroutine
gets the exception, but it doesn't unregister the file descriptor for write
event. create_connection() gets the TimeoutError and closes the socket.
If you call again create_connection(), the new socket will likely gets the same
file descriptor, which is still registered in the selector. When sock_connect()
calls add_writer(), it tries to modify the entry instead of creating a new one.
This issue was originally reported in the Trollius project, but the bug comes
from Tulip in fact (Trollius is based on Tulip):
https://bitbucket.org/enovance/trollius/issue/15/after-timeouterror-on-wait_for
This change fixes the race condition. It also makes sock_connect() more
reliable (and portable) is sock.connect() raises an InterruptedError.
* PipeServer.close() now cancels the "accept pipe" future which cancels the
overlapped operation.
* Fix _SelectorTransport.__repr__() if the transport was closed
* Fix debug log in BaseEventLoop.create_connection(): get the socket object
from the transport because SSL transport closes the old socket and creates a
new SSL socket object. Remove also the _SelectorSslTransport._rawsock
attribute: it contained the closed socket (not very useful) and it was not
used.
* Issue #22063: socket operations (sock_recv, sock_sendall, sock_connect,
sock_accept) of the proactor event loop don't raise an exception in debug
mode if the socket are in blocking mode. Overlapped operations also work on
blocking sockets.
* Fix unit tests in debug mode: mock a non-blocking socket for socket
operations which now raise an exception if the socket is blocking.
* _fatal_error() method of _UnixReadPipeTransport and _UnixWritePipeTransport
now log all exceptions in debug mode
* Don't log expected errors in unit tests
* Tulip issue 200: _WaitHandleFuture._unregister_wait() now catchs and logs
exceptions.
* Tulip issue 200: Log errors in debug mode instead of simply ignoring them.
* Tulip issue #196: IocpProactor._poll() clears the reference to the
overlapped operation when the operation is done. It would be better to clear
the reference in a new _OverlappedFuture.set_result() method, but it cannot
be done yet because of a weird bug.
* BaseSelectorEventLoop._write_to_self() now logs errors in debug mode.
Since Python 3.3, the C signal handler writes the signal number into the wakeup
file descriptor and then schedules the Python call using Py_AddPendingCall().
asyncio uses the wakeup file descriptor to wake up the event loop, and relies
on Py_AddPendingCall() to schedule the final callback with call_soon().
If the C signal handler is called in a thread different than the thread of the
event loop, the loop is awaken but Py_AddPendingCall() was not called yet. In
this case, the event loop has nothing to do and go to sleep again.
Py_AddPendingCall() is called while the event loop is sleeping again and so the
final callback is not scheduled immediatly.
This patch changes how asyncio handles signals. Instead of relying on
Py_AddPendingCall() and the wakeup file descriptor, asyncio now only relies on
the wakeup file descriptor. asyncio reads signal numbers from the wakeup file
descriptor to call its signal handler.
* Tulip issue #183: log socket events in debug mode
- Log most important socket events: socket connected, new client, connection
reset or closed by peer (EOF), etc.
- Log time elapsed in DNS resolution (getaddrinfo)
- Log pause/resume reading
- Log time of SSL handshake
- Log SSL handshake errors
- Add a __repr__() method to many classes
* Fix ProactorEventLoop() in debug mode. ProactorEventLoop._make_self_pipe()
doesn't call call_soon() directly because it checks for the current loop
which fails, because the method is called to build the event loop.
* Cleanup _ProactorReadPipeTransport constructor. Not need to set again
_read_fut attribute to None, it is already done in the base class.
- loop, waiters and active_count attributes are now private
- attach(), detach() and wakeup() methods are now private
The sockets attribute remains public.
- Tulip issue #181: Faster create_connection(). Call directly
waiter.set_result() in the constructor of _ProactorBasePipeTransport and
_SelectorSocketTransport, instead of using of delaying the call with
call_soon().
- Cleanup iscoroutine()
available bytes from the "self pipe", not only a single byte. This change
reduces the risk of having the pipe full and so getting the innocuous
"BlockingIOError: [Errno 11] Resource temporarily unavailable" message.
Add BaseEventLoop._closed attribute and use it to check if the event loop was
closed or not, instead of checking different attributes in each subclass of
BaseEventLoop.
run_forever() and run_until_complete() methods now raise a RuntimeError('Event loop is
closed') exception if the event loop was closed.
BaseProactorEventLoop.close() now also cancels "accept futures".
* Issue #159: Fix windows_utils.socketpair()
- Use "127.0.0.1" (IPv4) or "::1" (IPv6) host instead of "localhost", because
"localhost" may be a different IP address
- Reject also invalid arguments: only AF_INET/AF_INET6 with SOCK_STREAM (and
proto=0) are supported
* Reject add/remove reader/writer when event loop is closed.
* Fix ResourceWarning warnings
Major changes:
- StreamReader.readexactly() now raises an IncompleteReadError if the
end of stream is reached before we received enough bytes, instead of
returning less bytes than requested.
- Unit tests use the main asyncio module instead of submodules like events
- _UnixWritePipeTransport now also supports character devices, as
_UnixReadPipeTransport. Patch written by Jonathan Slenders.
- Export more symbols: BaseEventLoop, BaseProactorEventLoop,
BaseSelectorEventLoop, Queue and Queue sublasses, Empty, Full
between the resolution of the BaseEventLoop.time() method and the resolution of
the selector. The granuarility is used in the scheduler to round time and
deadline.
Yury Selivanov
Serhiy Storchaka
Victor Stinner
Benjamin Peterson
Antoine Pitrou
Andrew Svetlov
Guido van Rossum
Christian Heimes
Guido van Rossum