transport was closed. The check broken a Tulip example and this limitation is
arbitrary. Check if _proc is None should be enough.
Enhance also close(): do nothing when called the second time.
Issue #23347: send_signal(), kill() and terminate() methods of
BaseSubprocessTransport now check if the transport was closed and if the
process exited.
Issue #23347: Refactor creation of subprocess transports. Changes on
BaseSubprocessTransport:
* Add a wait() method to wait until the child process exit
* The constructor now accepts an optional waiter parameter. The _post_init()
coroutine must not be called explicitly anymore. It makes subprocess
transports closer to other transports, and it gives more freedom if we want
later to change completly how subprocess transports are created.
* close() now kills the process instead of kindly terminate it: the child
process may ignore SIGTERM and continue to run. Call explicitly terminate()
and wait() if you want to kindly terminate the child process.
* close() now logs a warning in debug mode if the process is still running and
needs to be killed
* _make_subprocess_transport() is now fully asynchronous again: if the creation
of the transport failed, wait asynchronously for the process eixt. Before the
wait was synchronous. This change requires close() to *kill*, and not
terminate, the child process.
* Remove the _kill_wait() method, replaced with a more agressive close()
method. It fixes _make_subprocess_transport() on error.
BaseSubprocessTransport.close() calls the close() method of pipe transports,
whereas _kill_wait() closed directly pipes of the subprocess.Popen object
without unregistering file descriptors from the selector (which caused severe
bugs).
These changes simplifies the code of subprocess.py.
* Cleanup gather(): use cancelled() method instead of using private Future
attribute
* Fix _UnixReadPipeTransport and _UnixWritePipeTransport. Only start reading
when connection_made() has been called.
* Issue #23333: Fix BaseSelectorEventLoop._accept_connection(). Close the
transport on error. In debug mode, log errors using call_exception_handler()
* _SelectorTransport constructor: extra parameter is now optional
* Fix _SelectorDatagramTransport constructor. Only start reading after
connection_made() has been called.
* Fix _SelectorSslTransport.close(). Don't call protocol.connection_lost() if
protocol.connection_made() was not called yet: if the SSL handshake failed or
is still in progress. The close() method can be called if the creation of the
connection is cancelled, by a timeout for example.
* Rephrase also the comment explaining why the waiter is not awaken immediatly.
* SSLProtocol.eof_received() doesn't instanciate ConnectionResetError exception
directly, it will be done by Future.set_exception(). The exception is not
used if the waiter was cancelled or if there is no waiter.
* Remove unused SSLProtocol._closing attribute
* test_sslproto: skip test if ssl module is missing
* Python issue #23208: Don't use the traceback of the current handle if we
already know the traceback of the source. The handle may be more revelant,
but having 3 tracebacks (handle, source, exception) becomes more difficult to
read. The handle may be preferred later but it requires more work to make
this choice.
If ReadFile() fails with ERROR_BROKEN_PIPE, the operation is not pending: don't
register the overlapped.
I don't know if WSARecv() can fail with ERROR_BROKEN_PIPE. Since
Overlapped.WSARecv() already handled ERROR_BROKEN_PIPE, let me guess that it
has the same behaviour than ReadFile().
If UnregisterWaitEx() fais with ERROR_IO_PENDING, it doesn't mean that the wait
is unregistered yet. We still have to wait until the wait is cancelled.
Use a coroutine with asyncio.sleep() instead of call_later() to ensure that the
schedule call is cancelled.
Add also a unit test cancelling connect_pipe().
In debug mode, BaseEventLoop._run_once() now sets the
BaseEventLoop._current_handle attribute to the handle currently executed.
In release mode or when no handle is executed, the attribute is None.
BaseEventLoop.default_exception_handler() displays the traceback of the current
handle if available.
Increase http.client.HTTPConnection test coverage.
Added a new tunnel test to verify setting of _tunnel_host, _tunnel_port,
_tunnel_headers attributes on HTTPConnection object.
script_helper.assert_python_failure(). No such feature has ever existed,
thus it doesn't do what the comment claims. (It does add a 'without'
variable to the environment of the child process but that was not intended)
script_helper.assert_python_failure(). No such feature has ever existed,
thus it doesn't do what the comment claims. (It does add a 'without'
variable to the environment of the child process but that was not intended)
mode. Explicitly remove the PYTHONFAULTHANDLER environment variable before
launching a child interpreter when its presence would impact the test (the
reason -E was being used in the first place).
This enables running the test in an environment where other Python environment variables must be set in order for things to run (such as using PYTHONHOME to
tell an embedded interpreter where it should think it lives).
mode. Explicitly remove the PYTHONFAULTHANDLER environment variable before
launching a child interpreter when its presence would impact the test (the
reason -E was being used in the first place).
This enables running the test in an environment where other Python environment
variables must be set in order for things to run (such as using PYTHONHOME to
tell an embedded interpreter where it should think it lives).
Overlapped.ConnectNamedPipe() now returns a boolean: True if the pipe is
connected (if ConnectNamedPipe() failed with ERROR_PIPE_CONNECTED), False if
the connection is in progress.
This change removes multiple hacks in IocpProactor.
Add _overlapped.ConnectPipe() which tries to connect to the pipe for
asynchronous I/O (overlapped): call CreateFile() in a loop until it doesn't
fail with ERROR_PIPE_BUSY. Use an increasing delay between 1 ms and 100 ms.
Remove Overlapped.WaitNamedPipeAndConnect() which is no more used.
This change fixes a race conditon related to _WaitHandleFuture.cancel() leading
to Python crash or "GetQueuedCompletionStatus() returned an unexpected event"
logs. Before, the overlapped object was destroyed too early, it was possible
that the wait completed whereas the overlapped object was already destroyed.
Sometimes, a different overlapped was allocated at the same address, leading to
unexpected completition.
_WaitHandleFuture.cancel() now waits until the wait is cancelled to clear its
reference to the overlapped object. To wait until the cancellation is done,
UnregisterWaitEx() is used with an event instead of UnregisterWait().
To wait for this event, a new _WaitCancelFuture class was added. It's a
simplified version of _WaitCancelFuture. For example, its cancel() method calls
UnregisterWait(), not UnregisterWaitEx(). _WaitCancelFuture should not be
cancelled.
The overlapped object is kept alive in _WaitHandleFuture until the wait is
unregistered.
Other changes:
* Add _overlapped.UnregisterWaitEx()
* Remove fast-path in IocpProactor.wait_for_handle() to immediatly set the
result if the wait already completed. I'm not sure that it's safe to
call immediatly UnregisterWaitEx() before the completion was signaled.
* Add IocpProactor._unregistered() to forget an overlapped which may never be
signaled, but may be signaled for the next loop iteration. It avoids to
block forever IocpProactor.close() if a wait was cancelled, and it may also
avoid some "... unexpected event ..." warnings.
* Handle correctly CancelledError: just exit
* On error, log the exception and exit
Don't try to close the event loop, it is probably running and so it cannot be
closed.
Python subprocess failure assertion error messages for easier debugging.
Adds a unittest for test.script_helper to confirm that this code works as
it is otherwise uncovered by an already passing test suite that uses it. :)
Python subprocess failure assertion error messages for easier debugging.
Adds a unittest for test.script_helper to confirm that this code works as
it is otherwise uncovered by an already passing test suite that uses it. :)
This commit also removes a couple of non-standard status codes. They were
added as part of edf669b13482, so there is no backwards compatibility issue.
Patch by Demian Brecht.
Override the connect_read_pipe() method of the loop to mock immediatly
pause_reading() and resume_reading() methods.
The test failed randomly on FreeBSD 9 buildbot and on Windows using trollius.
StreamWriter: close() now clears the reference to the transport
StreamWriter now raises an exception if it is closed: write(), writelines(),
write_eof(), can_write_eof(), get_extra_info(), drain().
* Use test_utils.run_briefly() to execute pending calls to really close
transports
* sslproto: mock also _SSLPipe.shutdown(), it's need to close the transport
* pipe test: the test doesn't close explicitly the PipeHandle, so ignore
the warning instead
* test_popen: use the context manager ("with p:") to explicitly close pipes
Don't call immediatly self._process_write_backlog() but schedule the call using
call_soon(). _on_handshake_complete() can be called indirectly from
_process_write_backlog(), and _process_write_backlog() is not reentrant.
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
(BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
state in the __repr__() method of _UnixReadPipeTransport and
_UnixWritePipeTransport classes.
* If an exception is raised during the creation of a subprocess, kill the
subprocess (close pipes, kill and read the return status). Log an error in
such case.
* Fix SubprocessStreamProtocol.connection_made() to handle cancelled waiter.
Add unit test cancelling subprocess methods.
- Add a new _wakeup_waiter() method
- Replace _create_waiter() method with a _wait_for_data() coroutine function
- Use the value None instead of True or False to wake up the waiter
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.
The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).
Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.
This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!
It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
Close the IocpProactor before closing the event loop. IocpProactor.close() can
call loop.call_soon(), which is forbidden when the event loop is closed.
RuntimeError if the selector is closed. And selectors.BaseSelector.close() now
clears its internal reference to the selector mapping to break a reference
cycle. Initial patch written by Martin Richard.
* Tulip issue 184: FlowControlMixin constructor now get the event loop if the
loop parameter is not set. Add unit tests to ensure that constructor of
StreamReader and StreamReaderProtocol classes get the event loop.
* Remove outdated TODO/XXX
importlib.abc.Loader.exec_module() is also defined.
Before this change, create_module() was optional **and** could return
None to trigger default semantics. This change now reduces the
options for choosing default semantics to one and in the most
backporting-friendly way (define create_module() to return None).
* Document why set_result() calls are safe
* Cleanup gather(). Use public methods instead of hacks to consume the
exception of a future.
* sock_connect(): pass directly the fd to _sock_connect_done instead of the
socket.
- clarified the distinction between text encodings and other codecs
- clarified relationship with builtin open and the io module
- consolidated documentation of error handlers into one section
- clarified type constraints of some behaviours
- added tests for some of the new statements in the docs
ssl.PROTOCOL_SSLv23 protocol by default, not ssl.PROTOCOL_SSLv3, for maximum
compatibility and support platforms where ssl.PROTOCOL_SSLv3 support is
disabled.
asyncio.BaseEventLoop now use the identifier of the current thread to ensure
that they are called from the thread running the event loop.
Before, the get_event_loop() method was used to check the thread, and no
exception was raised when the thread had no event loop. Now the methods always
raise an exception in debug mode when called from the wrong thread. It should
help to notice misusage of the API.
Victor Stinner
Stefan Krah
Donald Stufft
Serhiy Storchaka
Benjamin Peterson
Raymond Hettinger
Senthil Kumaran
Ethan Furman
Vinay Sajip
Gregory P. Smith
Zachary Ware
Berker Peksag
Antoine Pitrou
Ned Deily
Mark Dickinson
Brett Cannon
Terry Jan Reedy
Nick Coghlan
R David Murray