Commit Graph

62 Commits

Author SHA1 Message Date
Victor Stinner 5d44c08f1c Issue #23353, asyncio: Workaround CPython bug #23353
Don't use yield/yield-from in an except block of a generator. Store the
exception and handle it outside the except block.
2015-02-02 18:36:31 +01:00
Victor Stinner 1241ecc21b Issue #23347, asyncio: Make BaseSubprocessTransport.wait() private 2015-01-30 00:16:14 +01:00
Victor Stinner 47cd10d7a9 asyncio: sync with Tulip
Issue #23347: send_signal(), kill() and terminate() methods of
BaseSubprocessTransport now check if the transport was closed and if the
process exited.

Issue #23347: Refactor creation of subprocess transports. Changes on
BaseSubprocessTransport:

* Add a wait() method to wait until the child process exit
* The constructor now accepts an optional waiter parameter. The _post_init()
  coroutine must not be called explicitly anymore. It makes subprocess
  transports closer to other transports, and it gives more freedom if we want
  later to change completly how subprocess transports are created.
* close() now kills the process instead of kindly terminate it: the child
  process may ignore SIGTERM and continue to run. Call explicitly terminate()
  and wait() if you want to kindly terminate the child process.
* close() now logs a warning in debug mode if the process is still running and
  needs to be killed
* _make_subprocess_transport() is now fully asynchronous again: if the creation
  of the transport failed, wait asynchronously for the process eixt. Before the
  wait was synchronous. This change requires close() to *kill*, and not
  terminate, the child process.
* Remove the _kill_wait() method, replaced with a more agressive close()
  method. It fixes _make_subprocess_transport() on error.
  BaseSubprocessTransport.close() calls the close() method of pipe transports,
  whereas _kill_wait() closed directly pipes of the subprocess.Popen object
  without unregistering file descriptors from the selector (which caused severe
  bugs).

These changes simplifies the code of subprocess.py.
2015-01-30 00:05:19 +01:00
Victor Stinner 978a9afc6a Issue #23243, asyncio: Emit a ResourceWarning when an event loop or a transport
is not explicitly closed. Close also explicitly transports in test_sslproto.
2015-01-29 17:50:58 +01:00
Victor Stinner 2934262fd3 asyncio: sync with Tulip
* Cleanup gather(): use cancelled() method instead of using private Future
  attribute
* Fix _UnixReadPipeTransport and _UnixWritePipeTransport. Only start reading
  when connection_made() has been called.
* Issue #23333: Fix BaseSelectorEventLoop._accept_connection(). Close the
  transport on error. In debug mode, log errors using call_exception_handler()
2015-01-29 14:15:19 +01:00
Victor Stinner f07801bb17 asyncio: SSL transports now clear their reference to the waiter
* Rephrase also the comment explaining why the waiter is not awaken immediatly.
* SSLProtocol.eof_received() doesn't instanciate ConnectionResetError exception
  directly, it will be done by Future.set_exception(). The exception is not
  used if the waiter was cancelled or if there is no waiter.
2015-01-29 00:36:35 +01:00
Victor Stinner 4bf22e033e asyncio: Close the transport on subprocess creation failure 2015-01-15 14:24:22 +01:00
Victor Stinner 41ed958ee6 Issue #23243: Fix asyncio._UnixWritePipeTransport.close()
Do nothing if the transport is already closed. Before it was not possible to
close the transport twice.
2015-01-15 13:16:50 +01:00
Victor Stinner 29ad0111bd asyncio: sync with Tulip
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
  ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
  creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
  closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
  not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
  (BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
  readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
  close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
  state in the __repr__() method of _UnixReadPipeTransport and
  _UnixWritePipeTransport classes.
2015-01-15 00:04:21 +01:00
Victor Stinner 70db9e428a asyncio: sync with Tulip
* Tulip issue 184: FlowControlMixin constructor now get the event loop if the
  loop parameter is not set. Add unit tests to ensure that constructor of
  StreamReader and StreamReaderProtocol classes get the event loop.
* Remove outdated TODO/XXX
2015-01-09 21:32:05 +01:00
Victor Stinner 15cc678d89 asyncio: Truncate to 80 columns 2015-01-09 00:09:10 +01:00
Victor Stinner 1e40f10886 asyncio, tulip issue 209: Fix subprocess for close_fds=False on Python 3.3
Mark the write end of the stdin pipe as non-inheritable.
2014-12-11 23:30:17 +01:00
Victor Stinner e80bf0d4a9 Closes #22922: More EventLoop methods fail if the loop is closed. Initial patch
written by Torsten Landschoff.

create_task(), call_at(), call_soon(), call_soon_threadsafe() and
run_in_executor() now raise an error if the event loop is closed.
2014-12-04 23:07:47 +01:00
Victor Stinner 2d99d93d11 asyncio: Coroutine objects are now rejected with a TypeError by the following
functions:

* add_signal_handler()
* call_at()
* call_later()
* call_soon()
* call_soon_threadsafe()
* run_in_executor()

Fix also the error message of add_signal_handler() (fix the name of the
function).
2014-11-20 15:03:52 +01:00
Guido van Rossum e36fcde383 - Issue #22841: Reject coroutines in asyncio add_signal_handler().
Patch by Ludovic.Gasc.
2014-11-14 11:45:47 -08:00
Victor Stinner 004adb91f6 asyncio: Move loop attribute to _FlowControlMixin
Move the _loop attribute from the constructor of _SelectorTransport,
_ProactorBasePipeTransport and _UnixWritePipeTransport classes to the
constructor of the _FlowControlMixin class.

Add also an assertion to explicit that the parent class must ensure that the
loop is defined (not None)
2014-11-05 15:27:41 +01:00
Yury Selivanov 8c0e0ab767 asyncio.unix_events: Move import statement to match tulip code 2014-09-24 23:21:39 -04:00
Victor Stinner b261475a48 asyncio: sync with Tulip
* PipeServer.close() now cancels the "accept pipe" future which cancels the
  overlapped operation.
* Fix _SelectorTransport.__repr__() if the transport was closed
* Fix debug log in BaseEventLoop.create_connection(): get the socket object
  from the transport because SSL transport closes the old socket and creates a
  new SSL socket object. Remove also the _SelectorSslTransport._rawsock
  attribute: it contained the closed socket (not very useful) and it was not
  used.
* Issue #22063: socket operations (sock_recv, sock_sendall, sock_connect,
  sock_accept) of the proactor event loop don't raise an exception in debug
  mode if the socket are in blocking mode. Overlapped operations also work on
  blocking sockets.
* Fix unit tests in debug mode: mock a non-blocking socket for socket
  operations which now raise an exception if the socket is blocking.
* _fatal_error() method of _UnixReadPipeTransport and _UnixWritePipeTransport
  now log all exceptions in debug mode
* Don't log expected errors in unit tests
* Tulip issue 200: _WaitHandleFuture._unregister_wait() now catchs and logs
  exceptions.
* Tulip issue 200: Log errors in debug mode instead of simply ignoring them.
2014-08-25 23:20:52 +02:00
Victor Stinner f2ed889027 asyncio: Use the new os.set_blocking() function of Python 3.5 if available 2014-07-29 23:08:00 +02:00
Victor Stinner c4c464911a asyncio: sync with Tulip
* Tulip issue 194: Don't use sys.getrefcount() in unit tests
* signal.set_wakeup_fd() can now raise an OSError on Python 3.5
2014-07-23 18:21:45 +02:00
Victor Stinner fe5649c7b7 Python issue #21645, Tulip issue 192: Rewrite signal handling
Since Python 3.3, the C signal handler writes the signal number into the wakeup
file descriptor and then schedules the Python call using Py_AddPendingCall().

asyncio uses the wakeup file descriptor to wake up the event loop, and relies
on Py_AddPendingCall() to schedule the final callback with call_soon().

If the C signal handler is called in a thread different than the thread of the
event loop, the loop is awaken but Py_AddPendingCall() was not called yet. In
this case, the event loop has nothing to do and go to sleep again.
Py_AddPendingCall() is called while the event loop is sleeping again and so the
final callback is not scheduled immediatly.

This patch changes how asyncio handles signals. Instead of relying on
Py_AddPendingCall() and the wakeup file descriptor, asyncio now only relies on
the wakeup file descriptor. asyncio reads signal numbers from the wakeup file
descriptor to call its signal handler.
2014-07-17 22:43:40 +02:00
Victor Stinner acdb782a83 asyncio: sync with Tulip
* Tulip issue #184: Log subprocess events in debug mode

  - Log stdin, stdout and stderr transports and protocols
  - Log process identifier (pid)
  - Log connection of pipes
  - Log process exit
  - Log Process.communicate() tasks: feed stdin, read stdout and stderr
  - Add __repr__() method to many classes related to subprocesses


* Add BaseSubprocessTransport._pid attribute. Store the pid so it is still
  accessible after the process exited. It's more convinient for debug.

* create_connection(): add the socket in the "connected to" debug log

* Clean up some docstrings and comments. Remove unused unimplemented
  _read_from_self().
2014-07-14 18:33:40 +02:00
Victor Stinner e912e652f8 asyncio: sync with Tulip
* Tulip issue #183: log socket events in debug mode

  - Log most important socket events: socket connected, new client, connection
    reset or closed by peer (EOF), etc.
  - Log time elapsed in DNS resolution (getaddrinfo)
  - Log pause/resume reading
  - Log time of SSL handshake
  - Log SSL handshake errors
  - Add a __repr__() method to many classes

* Fix ProactorEventLoop() in debug mode. ProactorEventLoop._make_self_pipe()
  doesn't call call_soon() directly because it checks for the current loop
  which fails, because the method is called to build the event loop.

* Cleanup _ProactorReadPipeTransport constructor. Not need to set again
  _read_fut attribute to None, it is already done in the base class.
2014-07-12 03:11:53 +02:00
Victor Stinner bfff45d611 asyncion, Tulip issue 181: BaseEventLoop.create_datagram_endpoint() now waits
until protocol.connection_made() has been called. Document also why transport
constructors use a waiter.
2014-07-08 23:57:31 +02:00
Victor Stinner a9acbe82e7 Closes #21886, #21447: Fix a race condition in asyncio when setting the result
of a Future with call_soon(). Add an helper, a private method, to set the
result only if the future was not cancelled.
2014-07-05 15:29:41 +02:00
Victor Stinner 2dba23af71 asyncio: sync with Tulip
* _UnixSubprocessTransport: fix file mode of stdin. Open stdin in write mode,
  not in read mode
* Examples: close the event loop at exit
* More reliable CoroWrapper.__del__. If the constructor is interrupted by
  KeyboardInterrupt or the coroutine objet is destroyed lately, some the
  _source_traceback attribute doesn't exist anymore.
* repr(Task): include also the future the task is waiting for
2014-07-03 00:59:00 +02:00
Victor Stinner f951d28ac8 asyncio: sync with Tulip, add a new asyncio.coroutines module 2014-06-29 00:46:45 +02:00
Victor Stinner f328c7dc69 asyncio, Tulip issue 171: BaseEventLoop.close() now raises an exception if the
event loop is running. You must first stop the event loop and then wait until
it stopped, before closing it.
2014-06-23 01:02:37 +02:00
Victor Stinner f9e49dd346 Tulip issue 83, Python issue #21252: Fill some XXX docstrings in asyncio 2014-06-05 12:06:44 +02:00
Victor Stinner 223a624158 Issue #21119: asyncio now closes sockets on errors
Fix ResourceWarning: create_connection(), create_datagram_endpoint() and
create_unix_server() methods of event loop now close the newly created socket
on error.
2014-06-04 00:11:52 +02:00
Victor Stinner 1fd03a4a22 Issue #21155: asyncio.EventLoop.create_unix_server() now raises a ValueError if
path and sock are specified at the same time.
2014-04-07 11:18:54 +02:00
Victor Stinner b4c9388947 asyncio: remove unused imports and unused variables noticed by pyflakes 2014-02-20 10:37:27 +01:00
Yury Selivanov dec1a45fd1 asyncio: Fix spelling and typos.
Thanks to Vajrasky Kok for discovering some of them.
2014-02-18 22:27:48 -05:00
Victor Stinner 79a295261a asyncio, Tulip issue 143: UNIX domain methods, fix ResourceWarning and
DeprecationWarning warnings. create_unix_server() closes the socket on any
error, not only on OSError.
2014-02-19 01:45:59 +01:00
Victor Stinner 0ee29c2c7f asyncio, Tulip issue 139: Improve error messages on "fatal errors"
Mention if the error was caused by a read or a write, and be more specific on
the object (ex: "pipe transport" instead of "transport").
2014-02-19 01:40:41 +01:00
Yury Selivanov 3cb9914488 asyncio.transports: Make _ProactorBasePipeTransport use _FlowControlMixin 2014-02-18 18:41:13 -05:00
Yury Selivanov 569efa2e4b asyncio: New error handling API. Issue #20681. 2014-02-18 18:02:19 -05:00
Yury Selivanov b057c52b01 asyncio: Add support for UNIX Domain Sockets. 2014-02-18 12:15:06 -05:00
Victor Stinner dc62b7e261 asyncio: Tulip issue 112: Inline make_handle() into Handle constructor 2014-02-10 00:45:44 +01:00
Victor Stinner 915bcb0111 Issue #20400: Merge Tulip into Python: add the new asyncio.subprocess module
* Add a new asyncio.subprocess module
* Add new create_subprocess_exec() and create_subprocess_shell() functions
* The new asyncio.subprocess.SubprocessStreamProtocol creates stream readers
  for stdout and stderr and a stream writer for stdin.
* The new asyncio.subprocess.Process class offers an API close to the
  subprocess.Popen class:

  - pid, returncode, stdin, stdout and stderr attributes
  - communicate(), wait(), send_signal(), terminate() and kill() methods

* Remove STDIN (0), STDOUT (1) and STDERR (2) constants from base_subprocess
  and unix_events, to not be confused with the symbols with the same name of
  subprocess and asyncio.subprocess modules
* _ProactorBasePipeTransport.get_write_buffer_size() now counts also the size
  of the pending write
* _ProactorBaseWritePipeTransport._loop_writing() may now pause the protocol if
  the write buffer size is greater than the high water mark (64 KB by default)
2014-02-01 22:49:59 +01:00
Victor Stinner 61b3c9bacc asyncio: Fix _UnixWritePipeTransport, raise BrokenPipeError when the pipe is
closed, but only if there was pending write
2014-01-31 13:04:28 +01:00
Victor Stinner 73f10fd2f1 asyncio: Fix _make_subprocess_transport(): pass extra value to the constructor. 2014-01-29 14:32:20 -08:00
Guido van Rossum 47fb97e4e6 asyncio: Add write flow control to unix pipes. 2014-01-29 13:20:39 -08:00
Victor Stinner 63b4d4b494 asyncio: _fatal_error() of _UnixWritePipeTransport and _ProactorBasePipeTransport shouldn't log BrokenPipeError nor ConnectionResetError.
(Same behaviour as _SelectorTransport._fatal_error().)
2014-01-29 13:12:03 -08:00
Andrew Svetlov 0742cae335 Merge latest Tulip into asyncio 2014-01-26 18:36:01 +02:00
Guido van Rossum ab27a9fc4b asyncio: Fix race in FastChildWatcher (by its original author, Anthony Baire). 2014-01-25 16:32:17 -08:00
Victor Stinner 8dffc456d7 Update asyncio from the Tulip project
Major changes:

- StreamReader.readexactly() now raises an IncompleteReadError if the
  end of stream is reached before we received enough bytes, instead of
  returning less bytes than requested.

- Unit tests use the main asyncio module instead of submodules like events

- _UnixWritePipeTransport now also supports character devices, as
  _UnixReadPipeTransport. Patch written by Jonathan Slenders.

- Export more symbols: BaseEventLoop, BaseProactorEventLoop,
  BaseSelectorEventLoop, Queue and Queue sublasses, Empty, Full
2014-01-25 15:32:06 +01:00
Guido van Rossum 02757ea7e9 asyncio: Minimal pty support in UNIX read pipe, by Jonathan Slenders. 2014-01-10 13:30:04 -08:00
Guido van Rossum 4835f17c24 asyncio: Fix race in subprocess transport, by Victor Stinner. 2014-01-10 13:28:59 -08:00
Charles-François Natali 74e7cf3f46 Issue #19850: asyncio: Set SA_RESTART when registering a signal handler to
limit EINTR occurrences.
2013-12-05 22:47:19 +01:00