In this commit:
* Support BufferedProtocol in set_protocol() and start_tls()
* Fix proactor to cancel readers reliably
* Update tests to be compatible with OpenSSL 1.1.1
* Clarify BufferedProtocol docs
* Bump TLS tests timeouts to 60 seconds; eliminate possible race from start_serving
* Rewrite test_start_tls_server_1
bpo-32622, bpo-33353: On macOS, sock.connect() changes the
SO_SNDBUF value. Only set SO_SNDBUF and SO_RCVBUF buffer sizes
once a socket is connected or binded, not before.
bpo-32622, bpo-33353: sendfile() tests of test_asyncio use socket
buffers of 1 kB "to test on relative small data sets". Send only
160 KiB rather 10 MB to make the test much faster.
Shrink also SendfileBase.DATA from 1600 KiB to 160 KiB.
On Linux, 3 test_sock_sendfile_mix_with_regular_send() runs now take
less than 1 second, instead of 18 seconds.
On FreeBSD, the 3 tests didn't hang, but took 3 minutes. Now
the 3 tests pass in less than 1 seconds.
test_asyncio hangs indefinitely on macOS 10.13.2+ on `read_pty_output()`
using the KqueueSelector. Closing `proto.transport` (as is done in
`write_pty_output()`) seems to fix it.
(cherry picked from commit 12f74d8608)
Co-authored-by: Nathan Henrie <n8henrie@users.noreply.github.com>
Also, re-enable test_read_pty_output on macOS.
bpo-31399: Let OpenSSL verify hostname and IP
The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and
X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses.
* Remove match_hostname calls
* Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host()
* Add documentation for OpenSSL 1.0.2 requirement
* Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform.
* Add hostname_checks_common_name
Signed-off-by: Christian Heimes <christian@python.org>
* Make ssh_handshake_timeout None by default.
* Raise ValueError if ssl_handshake_timeout is used without ssl.
* Raise ValueError if ssl_handshake_timeout is not positive.
asyncio.get_event_loop(), and, subsequently asyncio._get_running_loop()
are one of the most frequently executed functions in asyncio. They also
can't be sped up by third-party event loops like uvloop.
When implemented in C they become 4x faster.
* Convert asyncio/tasks.py to async/await
* Convert asyncio/queues.py to async/await
* Convert asyncio/test_utils.py to async/await
* Convert asyncio/base_subprocess.py to async/await
* Convert asyncio/subprocess.py to async/await
* Convert asyncio/streams.py to async/await
* Fix comments
* Convert asyncio/locks.py to async/await
* Convert asyncio.sleep to async def
* Add a comment
* Add missing news
* Convert stubs from AbstrctEventLoop to async functions
* Convert subprocess_shell/subprocess_exec
* Convert connect_read_pipe/connect_write_pip to async/await syntax
* Convert create_datagram_endpoint
* Convert create_unix_server/create_unix_connection
* Get rid of old style coroutines in unix_events.py
* Convert selector_events.py to async/await
* Convert wait_closed and create_connection
* Drop redundant line
* Convert base_events.py
* Code cleanup
* Drop redundant comments
* Fix indentation
* Add explicit tests for compatibility between old and new coroutines
* Convert windows event loop to use async/await
* Fix double awaiting of async function
* Convert asyncio/locks.py
* Improve docstring
* Convert tests to async/await
* Convert more tests
* Convert more tests
* Convert more tests
* Convert tests
* Improve test
* Remove asyncio.selectors and asyncio._overlapped symbols from the
namespace of the asyncio module
* Replace "from asyncio import selectors" with "import selectors"
* Replace "from asyncio import _overlapped" with "import _overlapped"
asyncio.selectors was added to support Python 3.3, which doesn't have
selectors in its standard library, and Python 3.4 in the same code
base. Same rationale for asyncio._overlapped. Python 3.3 reached its
end of life, and asyncio is no more maintained as a third party
module on PyPI.
The test.support.skip_unless_bind_unix_socket() decorator is used to skip
asyncio tests that fail because the platform lacks a functional bind()
function for unix domain sockets (as it is the case for non root users on the
recent Android versions that run now SELinux in enforcing mode).
Call doCleanups() to close the loop after calling
executor.shutdown(wait=True): see TestCase.set_event_loop() of
asyncio.test_utils.
Replace also gc.collect() with support.gc_collect().
* bpo-31034: Reliable signal handler for test_asyncio
Don't rely on the current SIGHUP signal handler, make sure that it's
set to the "default" signal handler: SIG_DFL.
* Add comments
Some built-in coroutine-like objects might not have __name__ or
__qualname__. A good example of such are 'asend', 'aclose' and
'athrow' coroutine methods of asynchronous generators.
getaddrinfo takes an exclusive lock on some platforms, causing clients to queue
up waiting for the lock if many names are being resolved concurrently. Users
may want to handle name resolution in their own code, for the sake of caching,
using an alternate resolver, or to measure DNS duration separately from
connection duration. Skip getaddrinfo if the "host" passed into
create_connection is already resolved.
See https://github.com/python/asyncio/pull/302 for details.
Patch by A. Jesse Jiryu Davis.
This info is required on Python 3.5 and newer to get specific information on
the SSL object, like getting the binary peer certificate (instead of getting
it as text).
* _check_resolved_address() is implemented with getaddrinfo() which is slow
* If available, use socket.inet_pton() instead of socket.getaddrinfo(), because
it is much faster
Microbenchmark (timeit) on Fedora 21 (Python 3.4, Linux 3.17, glibc 2.20) to
validate the IPV4 address "127.0.0.1" or the IPv6 address "::1":
* getaddrinfo() 10.4 usec per loop
* inet_pton(): 0.285 usec per loop
On glibc older than 2.14, getaddrinfo() always requests the list of all local
IP addresses to the kernel (using a NETLINK socket). getaddrinfo() has other
known issues, it's better to avoid it when it is possible.
Issue #23347: send_signal(), kill() and terminate() methods of
BaseSubprocessTransport now check if the transport was closed and if the
process exited.
Issue #23347: Refactor creation of subprocess transports. Changes on
BaseSubprocessTransport:
* Add a wait() method to wait until the child process exit
* The constructor now accepts an optional waiter parameter. The _post_init()
coroutine must not be called explicitly anymore. It makes subprocess
transports closer to other transports, and it gives more freedom if we want
later to change completly how subprocess transports are created.
* close() now kills the process instead of kindly terminate it: the child
process may ignore SIGTERM and continue to run. Call explicitly terminate()
and wait() if you want to kindly terminate the child process.
* close() now logs a warning in debug mode if the process is still running and
needs to be killed
* _make_subprocess_transport() is now fully asynchronous again: if the creation
of the transport failed, wait asynchronously for the process eixt. Before the
wait was synchronous. This change requires close() to *kill*, and not
terminate, the child process.
* Remove the _kill_wait() method, replaced with a more agressive close()
method. It fixes _make_subprocess_transport() on error.
BaseSubprocessTransport.close() calls the close() method of pipe transports,
whereas _kill_wait() closed directly pipes of the subprocess.Popen object
without unregistering file descriptors from the selector (which caused severe
bugs).
These changes simplifies the code of subprocess.py.
* Cleanup gather(): use cancelled() method instead of using private Future
attribute
* Fix _UnixReadPipeTransport and _UnixWritePipeTransport. Only start reading
when connection_made() has been called.
* Issue #23333: Fix BaseSelectorEventLoop._accept_connection(). Close the
transport on error. In debug mode, log errors using call_exception_handler()
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
(BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
state in the __repr__() method of _UnixReadPipeTransport and
_UnixWritePipeTransport classes.
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.
The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).
Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.
This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!
It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
written by Torsten Landschoff.
create_task(), call_at(), call_soon(), call_soon_threadsafe() and
run_in_executor() now raise an error if the event loop is closed.
* PipeServer.close() now cancels the "accept pipe" future which cancels the
overlapped operation.
* Fix _SelectorTransport.__repr__() if the transport was closed
* Fix debug log in BaseEventLoop.create_connection(): get the socket object
from the transport because SSL transport closes the old socket and creates a
new SSL socket object. Remove also the _SelectorSslTransport._rawsock
attribute: it contained the closed socket (not very useful) and it was not
used.
* Issue #22063: socket operations (sock_recv, sock_sendall, sock_connect,
sock_accept) of the proactor event loop don't raise an exception in debug
mode if the socket are in blocking mode. Overlapped operations also work on
blocking sockets.
* Fix unit tests in debug mode: mock a non-blocking socket for socket
operations which now raise an exception if the socket is blocking.
* _fatal_error() method of _UnixReadPipeTransport and _UnixWritePipeTransport
now log all exceptions in debug mode
* Don't log expected errors in unit tests
* Tulip issue 200: _WaitHandleFuture._unregister_wait() now catchs and logs
exceptions.
* Tulip issue 200: Log errors in debug mode instead of simply ignoring them.
* Tulip issue #182: Improve logs of BaseEventLoop._run_once()
- Don't log non-blocking poll
- Only log polling with a timeout if it gets events or if it timed out after
more than 1 second.
* Fix some pyflakes warnings: remove unused imports
- Issues #21936, #21163: Fix sporadic failures of
test_future_exception_never_retrieved()
- Handle.cancel() now clears references to callback and args
- In debug mode, repr(Handle) now contains the location where the Handle was
created.
- Tulip issue 185: Add a create_task() method to event loops. The create_task()
method can be overriden in custom event loop to implement their own task
class. For example, greenio and Pulsar projects use their own task class. The
create_task() method is now preferred over creating directly task using the
Task class.
- tests: fix a warning
- fix typo in the name of a test function
- Update AbstractEventLoop: add new event loop methods; update also the unit test
Handle objects are created. Pass the traceback to call_exception_handler() in
the 'source_traceback' key.
The traceback is truncated to hide internal calls in asyncio, show only the
traceback from user code.
Add tests for the new source_traceback, and a test for the 'Future/Task
exception was never retrieved' log.
repr(Handle) is shorter for function: "foo" instead of "<function foo at
0x...>". It now also includes the source of the callback, filename and line
number where it was defined, if available.
repr(Task) now also includes the current position in the code, filename and
line number, if available. If the coroutine (generator) is done, the line
number is omitted and "done" is added.
* Issue #159: Fix windows_utils.socketpair()
- Use "127.0.0.1" (IPv4) or "::1" (IPv6) host instead of "localhost", because
"localhost" may be a different IP address
- Reject also invalid arguments: only AF_INET/AF_INET6 with SOCK_STREAM (and
proto=0) are supported
* Reject add/remove reader/writer when event loop is closed.
* Fix ResourceWarning warnings
Yury Selivanov
Victor Stinner
Ned Deily
Andrew Svetlov
Christian Heimes
Victor Stinner
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి)
xdegaye
Marat Sharafutdinov
Antoine Pitrou
Xiang Zhang
Yury Selivanov
Guido van Rossum
Yury Selivanov
Gregory P. Smith
Benjamin Peterson
Antoine Pitrou