Issue #26741: asyncio: BaseSubprocessTransport._process_exited() now copies the
return code from the child watched to the returncode attribute of the Popen
object. On Python 3.6, it is required to avoid a ResourceWarning.
Issue #26604:
* Add a new optional source parameter to _warnings.warn() and warnings.warn()
* Modify asyncore, asyncio and _pyio modules to set the source parameter when
logging a ResourceWarning warning
getaddrinfo takes an exclusive lock on some platforms, causing clients to queue
up waiting for the lock if many names are being resolved concurrently. Users
may want to handle name resolution in their own code, for the sake of caching,
using an alternate resolver, or to measure DNS duration separately from
connection duration. Skip getaddrinfo if the "host" passed into
create_connection is already resolved.
See https://github.com/python/asyncio/pull/302 for details.
Patch by A. Jesse Jiryu Davis.
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
This info is required on Python 3.5 and newer to get specific information on
the SSL object, like getting the binary peer certificate (instead of getting
it as text).
Issue #24763: Fix resource warnings when asyncio.BaseSubprocessTransport
constructor fails, if subprocess.Popen raises an exception for example.
Patch written by Martin Richard, test written by me.
* Fix ResourceWarning warnings in test_streams
* Return True from StreamReader.eof_received() to fix
http://bugs.python.org/issue24539 (but still needs a unittest).
Add StreamReader.__repr__() for easy debugging.
* remove unused imports
* Issue #234: Drop JoinableQueue on Python 3.5+
Previous approach of installing coroutine wrapper in loop.set_debug() and
uninstalling it in loop.close() was very fragile. Most of asyncio tests
do not call loop.close() at all. Since coroutine wrapper is a global
setting, we have to make sure that it's only set when the loop is
running, and is automatically unset when it stops running.
Issue #24017.
again if the first call to connect() raises an InterruptedError.
When the C function connect() fails with EINTR, the connection runs in
background. We have to wait until the socket becomes writable to be notified
when the connection succeed or fails.
Merge JoinableQueue with Queue. To more closely match the standard Queue,
asyncio.Queue has "join" and "task_done". JoinableQueue is deleted.
Docstring for Queue.join shouldn't mention threads.
Restore JoinableQueue as a deprecated alias for Queue. To more closely match
the standard Queue, asyncio.Queue has "join" and "task_done". JoinableQueue
remains as a deprecated alias for Queue to avoid needlessly breaking too much
code that depended on it.
Patch written by A. Jesse Jiryu Davis <jesse@mongodb.com>.
* _check_resolved_address() is implemented with getaddrinfo() which is slow
* If available, use socket.inet_pton() instead of socket.getaddrinfo(), because
it is much faster
Microbenchmark (timeit) on Fedora 21 (Python 3.4, Linux 3.17, glibc 2.20) to
validate the IPV4 address "127.0.0.1" or the IPv6 address "::1":
* getaddrinfo() 10.4 usec per loop
* inet_pton(): 0.285 usec per loop
On glibc older than 2.14, getaddrinfo() always requests the list of all local
IP addresses to the kernel (using a NETLINK socket). getaddrinfo() has other
known issues, it's better to avoid it when it is possible.
transport was closed. The check broken a Tulip example and this limitation is
arbitrary. Check if _proc is None should be enough.
Enhance also close(): do nothing when called the second time.
Issue #23347: send_signal(), kill() and terminate() methods of
BaseSubprocessTransport now check if the transport was closed and if the
process exited.
Issue #23347: Refactor creation of subprocess transports. Changes on
BaseSubprocessTransport:
* Add a wait() method to wait until the child process exit
* The constructor now accepts an optional waiter parameter. The _post_init()
coroutine must not be called explicitly anymore. It makes subprocess
transports closer to other transports, and it gives more freedom if we want
later to change completly how subprocess transports are created.
* close() now kills the process instead of kindly terminate it: the child
process may ignore SIGTERM and continue to run. Call explicitly terminate()
and wait() if you want to kindly terminate the child process.
* close() now logs a warning in debug mode if the process is still running and
needs to be killed
* _make_subprocess_transport() is now fully asynchronous again: if the creation
of the transport failed, wait asynchronously for the process eixt. Before the
wait was synchronous. This change requires close() to *kill*, and not
terminate, the child process.
* Remove the _kill_wait() method, replaced with a more agressive close()
method. It fixes _make_subprocess_transport() on error.
BaseSubprocessTransport.close() calls the close() method of pipe transports,
whereas _kill_wait() closed directly pipes of the subprocess.Popen object
without unregistering file descriptors from the selector (which caused severe
bugs).
These changes simplifies the code of subprocess.py.
* Cleanup gather(): use cancelled() method instead of using private Future
attribute
* Fix _UnixReadPipeTransport and _UnixWritePipeTransport. Only start reading
when connection_made() has been called.
* Issue #23333: Fix BaseSelectorEventLoop._accept_connection(). Close the
transport on error. In debug mode, log errors using call_exception_handler()
* _SelectorTransport constructor: extra parameter is now optional
* Fix _SelectorDatagramTransport constructor. Only start reading after
connection_made() has been called.
* Fix _SelectorSslTransport.close(). Don't call protocol.connection_lost() if
protocol.connection_made() was not called yet: if the SSL handshake failed or
is still in progress. The close() method can be called if the creation of the
connection is cancelled, by a timeout for example.
* Rephrase also the comment explaining why the waiter is not awaken immediatly.
* SSLProtocol.eof_received() doesn't instanciate ConnectionResetError exception
directly, it will be done by Future.set_exception(). The exception is not
used if the waiter was cancelled or if there is no waiter.
* Remove unused SSLProtocol._closing attribute
* test_sslproto: skip test if ssl module is missing
* Python issue #23208: Don't use the traceback of the current handle if we
already know the traceback of the source. The handle may be more revelant,
but having 3 tracebacks (handle, source, exception) becomes more difficult to
read. The handle may be preferred later but it requires more work to make
this choice.
If ReadFile() fails with ERROR_BROKEN_PIPE, the operation is not pending: don't
register the overlapped.
I don't know if WSARecv() can fail with ERROR_BROKEN_PIPE. Since
Overlapped.WSARecv() already handled ERROR_BROKEN_PIPE, let me guess that it
has the same behaviour than ReadFile().
If UnregisterWaitEx() fais with ERROR_IO_PENDING, it doesn't mean that the wait
is unregistered yet. We still have to wait until the wait is cancelled.
Use a coroutine with asyncio.sleep() instead of call_later() to ensure that the
schedule call is cancelled.
Add also a unit test cancelling connect_pipe().
In debug mode, BaseEventLoop._run_once() now sets the
BaseEventLoop._current_handle attribute to the handle currently executed.
In release mode or when no handle is executed, the attribute is None.
BaseEventLoop.default_exception_handler() displays the traceback of the current
handle if available.
Overlapped.ConnectNamedPipe() now returns a boolean: True if the pipe is
connected (if ConnectNamedPipe() failed with ERROR_PIPE_CONNECTED), False if
the connection is in progress.
This change removes multiple hacks in IocpProactor.
Add _overlapped.ConnectPipe() which tries to connect to the pipe for
asynchronous I/O (overlapped): call CreateFile() in a loop until it doesn't
fail with ERROR_PIPE_BUSY. Use an increasing delay between 1 ms and 100 ms.
Remove Overlapped.WaitNamedPipeAndConnect() which is no more used.
This change fixes a race conditon related to _WaitHandleFuture.cancel() leading
to Python crash or "GetQueuedCompletionStatus() returned an unexpected event"
logs. Before, the overlapped object was destroyed too early, it was possible
that the wait completed whereas the overlapped object was already destroyed.
Sometimes, a different overlapped was allocated at the same address, leading to
unexpected completition.
_WaitHandleFuture.cancel() now waits until the wait is cancelled to clear its
reference to the overlapped object. To wait until the cancellation is done,
UnregisterWaitEx() is used with an event instead of UnregisterWait().
To wait for this event, a new _WaitCancelFuture class was added. It's a
simplified version of _WaitCancelFuture. For example, its cancel() method calls
UnregisterWait(), not UnregisterWaitEx(). _WaitCancelFuture should not be
cancelled.
The overlapped object is kept alive in _WaitHandleFuture until the wait is
unregistered.
Other changes:
* Add _overlapped.UnregisterWaitEx()
* Remove fast-path in IocpProactor.wait_for_handle() to immediatly set the
result if the wait already completed. I'm not sure that it's safe to
call immediatly UnregisterWaitEx() before the completion was signaled.
* Add IocpProactor._unregistered() to forget an overlapped which may never be
signaled, but may be signaled for the next loop iteration. It avoids to
block forever IocpProactor.close() if a wait was cancelled, and it may also
avoid some "... unexpected event ..." warnings.
* Handle correctly CancelledError: just exit
* On error, log the exception and exit
Don't try to close the event loop, it is probably running and so it cannot be
closed.
StreamWriter: close() now clears the reference to the transport
StreamWriter now raises an exception if it is closed: write(), writelines(),
write_eof(), can_write_eof(), get_extra_info(), drain().
Yury Selivanov
Martin Panter
Yury Selivanov
Victor Stinner
Serhiy Storchaka
Andrew Svetlov
Guido van Rossum