Commit Graph

249 Commits

Author SHA1 Message Date
Antoine Pitrou fd39a89e0e Issue #22641: In asyncio, the default SSL context for client connections is now created using ssl.create_default_context(), for stronger security. 2014-10-15 16:58:21 +02:00
Victor Stinner f67f460242 Issue #18643: asyncio.windows_utils now reuse socket.socketpair() on Windows if
available

Since Python 3.5, socket.socketpair() is now also available on Windows.

Make csock blocking before calling the accept() method, and fix also a typo in
an error message.
2014-10-14 22:56:25 +02:00
Victor Stinner 0e34dc3737 asyncio: enhance protocol representation
Add "closed" or "closing" to repr() of selector and proactor transports
2014-10-12 09:52:11 +02:00
Victor Stinner c8bd53f815 Issue #22601: run_forever() now consumes BaseException of the temporary task
If the coroutine raised a BaseException, consume the exception to not log a
warning. The caller doesn't have access to the local task.
2014-10-11 14:30:18 +02:00
Victor Stinner 68da8fc4c0 Issue #22448: asyncio, cleanup _run_once(), only iterate once to remove delayed
calls that were cancelled.
2014-09-30 18:08:36 +02:00
Yury Selivanov 592ada9b4b asyncio: Improve canceled timer handles cleanup. Closes issue #22448.
Patch by Joshua Moore-Oliva.
2014-09-25 12:07:56 -04:00
Yury Selivanov 8c0e0ab767 asyncio.unix_events: Move import statement to match tulip code 2014-09-24 23:21:39 -04:00
R David Murray 8e069d5ce9 Apply asyncio Task English fixes to docstrings as well.
Also fixed the phrasing in a comment.
2014-09-24 13:13:45 -04:00
Victor Stinner 1b38bc65dd asyncio, Tulip issue 206: In debug mode, keep the callback in the
representation of Handle and TimerHandle after cancel().
2014-09-17 23:24:13 +02:00
Serhiy Storchaka 148679982f Issue #22369: Change "context manager protocol" to "context management protocol". 2014-09-10 23:43:41 +03:00
Victor Stinner d5aeccf976 asyncio, Tulip issue 205: Fix a race condition in BaseSelectorEventLoop.sock_connect()
There is a race condition in create_connection() used with wait_for() to have a
timeout. sock_connect() registers the file descriptor of the socket to be
notified of write event (if connect() raises BlockingIOError). When
create_connection() is cancelled with a TimeoutError, sock_connect() coroutine
gets the exception, but it doesn't unregister the file descriptor for write
event. create_connection() gets the TimeoutError and closes the socket.

If you call again create_connection(), the new socket will likely gets the same
file descriptor, which is still registered in the selector. When sock_connect()
calls add_writer(), it tries to modify the entry instead of creating a new one.

This issue was originally reported in the Trollius project, but the bug comes
from Tulip in fact (Trollius is based on Tulip):
https://bitbucket.org/enovance/trollius/issue/15/after-timeouterror-on-wait_for

This change fixes the race condition. It also makes sock_connect() more
reliable (and portable) is sock.connect() raises an InterruptedError.
2014-08-31 15:07:57 +02:00
Victor Stinner 59e0802301 asyncio, Tulip issue 201: Fix a race condition in wait_for()
Don't raise a TimeoutError if we reached the timeout and the future completed
in the same iteration of the event loop. A side effect of the bug is that
Queue.get() looses items.
2014-08-28 11:19:25 +02:00
Victor Stinner 52bb949fd3 asyncio, tulip issue 203: Add _FlowControlMixin.get_write_buffer_limits() method 2014-08-26 00:22:28 +02:00
Victor Stinner b261475a48 asyncio: sync with Tulip
* PipeServer.close() now cancels the "accept pipe" future which cancels the
  overlapped operation.
* Fix _SelectorTransport.__repr__() if the transport was closed
* Fix debug log in BaseEventLoop.create_connection(): get the socket object
  from the transport because SSL transport closes the old socket and creates a
  new SSL socket object. Remove also the _SelectorSslTransport._rawsock
  attribute: it contained the closed socket (not very useful) and it was not
  used.
* Issue #22063: socket operations (sock_recv, sock_sendall, sock_connect,
  sock_accept) of the proactor event loop don't raise an exception in debug
  mode if the socket are in blocking mode. Overlapped operations also work on
  blocking sockets.
* Fix unit tests in debug mode: mock a non-blocking socket for socket
  operations which now raise an exception if the socket is blocking.
* _fatal_error() method of _UnixReadPipeTransport and _UnixWritePipeTransport
  now log all exceptions in debug mode
* Don't log expected errors in unit tests
* Tulip issue 200: _WaitHandleFuture._unregister_wait() now catchs and logs
  exceptions.
* Tulip issue 200: Log errors in debug mode instead of simply ignoring them.
2014-08-25 23:20:52 +02:00
Victor Stinner 9c9f1f10d3 Close #22063: socket operations (socket,recv, sock_sendall, sock_connect,
sock_accept) now raise an exception in debug mode if sockets are in blocking
mode.
2014-07-29 23:08:17 +02:00
Victor Stinner f2ed889027 asyncio: Use the new os.set_blocking() function of Python 3.5 if available 2014-07-29 23:08:00 +02:00
Victor Stinner 313a980904 asyncio: sync with Tulip
* _WaitHandleFuture.cancel() now notify IocpProactor through the overlapped
  object that the wait was cancelled.
* Optimize IocpProactor.wait_for_handle() gets the result if the wait is
  signaled immediatly.
* Enhance representation of Future and Future subclasses

  - Add "created at filename:lineno" in the representation
  - Add Future._repr_info() method which can be more easily overriden than
    Future.__repr__(). It should now be more easy to enhance Future
    representation without having to modify each subclass. For example,
    _OverlappedFuture and _WaitHandleFuture get the new "created at" information.
  - Use reprlib to format Future result, and function arguments when formatting a
    callback, to limit the length of the representation.

* Fix repr(_WaitHandleFuture)
* _WaitHandleFuture and _OverlappedFuture: hide frames of internal calls in the
  source traceback.
* Cleanup ProactorIocp._poll(): set the timeout to 0 after the first call to
  GetQueuedCompletionStatus()
* test_locks: close the temporary event loop and check the condition lock
* Remove workaround in test_futures, no more needed
2014-07-29 12:58:23 +02:00
Victor Stinner 42d3bdeed6 asyncio, tulip issue 196: ProactorIocp._register() now registers the overlapped
in the _cache dictionary, even if we already got the result. We need to keep a
reference to the overlapped object, otherwise the memory may be reused and
GetQueuedCompletionStatus() may use random bytes and behaves badly.

There is still a hack for ConnectNamedPipe(): the overlapped object is not
register into _cache if the overlapped object completed directly.

Log also an error in debug mode in ProactorIocp._loop() if we get an unexpected
event.

Add a protection in ProactorIocp.close() to avoid blocking, even if it should
not happen. I still don't understand exactly why some the completion of some
overlapped objects are not notified.
2014-07-28 00:18:43 +02:00
Andrew Svetlov f21fcd09c5 Accept optional lock object in Condition ctor (tulip issue #198) 2014-07-26 17:54:34 +03:00
Victor Stinner 51e44ea66a Tulip issue 196: _OverlappedFuture.set_result() now clears its reference to the
overlapped object. IocpProactor._poll() now also ignores false alarms:
GetQueuedCompletionStatus() returns the overlapped but it is still pending.
2014-07-26 00:58:34 +02:00
Victor Stinner 65dd69a3da asyncio: sync with Tulip
* Tulip issue #196: IocpProactor._poll() clears the reference to the
  overlapped operation when the operation is done. It would be better to clear
  the reference in a new _OverlappedFuture.set_result() method, but it cannot
  be done yet because of a weird bug.
* BaseSelectorEventLoop._write_to_self() now logs errors in debug mode.
2014-07-25 22:36:05 +02:00
Victor Stinner 18a28dc5c2 asyncio: sync with Tulip
* Fix _WaitHandleFuture.cancel(): return the result of the parent cancel()
  method.
* _OverlappedFuture.cancel() now clears its reference to the overlapped object.
  Make also the _OverlappedFuture.ov attribute private.
* Check if _WaitHandleFuture completed before unregistering it in the callback.
  Add also _WaitHandleFuture._poll() and repr(_WaitHandleFuture).
* _WaitHandleFuture now unregisters its wait handler if WaitForSingleObject()
  raises an exception.
* _OverlappedFuture.set_exception() now cancels the overlapped operation.
2014-07-25 13:05:20 +02:00
Victor Stinner fea6a100dc asyncio: sync with Tulip
Improve stability of the proactor event loop, especially operations on
overlapped objects:

* Tulip issue 195: Don't call UnregisterWait() twice if a _WaitHandleFuture is
  cancelled twice to fix a crash.
* IocpProactor.close(): cancel futures to cancel overlapped operations, instead
  of cancelling directly overlapped operations. Future objects may not call
  ov.cancel() if the future was cancelled or if the overlapped was already
  cancelled. The cancel() method of the future may also catch exceptions. Log
  also errors on cancellation.
* tests: rename "f" to "fut"
* Add a __repr__() method to IocpProactor
* Add a destructor to IocpProactor which closes it
* _OverlappedFuture.cancel() doesn't cancel the overlapped anymore if it is
  done: if it is already cancelled or completed. Log also an error if the
  cancellation failed.
* Add the address of the overlapped object in repr(_OverlappedFuture)
* _OverlappedFuture truncates the source traceback to hide the call to the
  parent constructor (useless in debug).
2014-07-25 00:54:53 +02:00
Victor Stinner 5006b1fd96 Issue #20055: Fix BaseEventLoop.stop() docstring, incomplete sentence.
Patch written by Saimadhav Heblikar.
2014-07-24 11:34:11 +02:00
Victor Stinner c4c464911a asyncio: sync with Tulip
* Tulip issue 194: Don't use sys.getrefcount() in unit tests
* signal.set_wakeup_fd() can now raise an OSError on Python 3.5
2014-07-23 18:21:45 +02:00
Victor Stinner 31e7bfa6ba asyncio, tulip issue 193: Convert StreamWriter.drain() to a classic coroutine
Replace also _make_drain_waiter() function with a classic _drain_helper()
coroutine.
2014-07-22 12:03:40 +02:00
Victor Stinner be0a2d767c Fix asyncio.__all__: export also unix_events and windows_events symbols
For example, on Windows, it was not possible to get ProactorEventLoop or
DefaultEventLoopPolicy using "from asyncio import *".
2014-07-18 12:44:25 +02:00
Victor Stinner fe5649c7b7 Python issue #21645, Tulip issue 192: Rewrite signal handling
Since Python 3.3, the C signal handler writes the signal number into the wakeup
file descriptor and then schedules the Python call using Py_AddPendingCall().

asyncio uses the wakeup file descriptor to wake up the event loop, and relies
on Py_AddPendingCall() to schedule the final callback with call_soon().

If the C signal handler is called in a thread different than the thread of the
event loop, the loop is awaken but Py_AddPendingCall() was not called yet. In
this case, the event loop has nothing to do and go to sleep again.
Py_AddPendingCall() is called while the event loop is sleeping again and so the
final callback is not scheduled immediatly.

This patch changes how asyncio handles signals. Instead of relying on
Py_AddPendingCall() and the wakeup file descriptor, asyncio now only relies on
the wakeup file descriptor. asyncio reads signal numbers from the wakeup file
descriptor to call its signal handler.
2014-07-17 22:43:40 +02:00
Victor Stinner d55b54d5c0 asyncio, tulip issue 190: Process.communicate() now ignores
ConnectionResetError too
2014-07-17 13:12:03 +02:00
Victor Stinner cc996b5789 asyncio, tulip issue 190: Process.communicate() must ignore BrokenPipeError
If you want to handle the BrokenPipeError, you can easily reimplement
communicate().

Add also a unit test to ensure that stdin.write() + stdin.drain() raises
BrokenPipeError.
2014-07-17 12:25:27 +02:00
Victor Stinner e931f7b768 Issue #21163: Fix "destroy pending task" warning in test_wait_errors() 2014-07-16 18:50:39 +02:00
Victor Stinner f03b3c7564 Issue #21163, asyncio: Ignore "destroy pending task" warnings for private tasks
in gather().
2014-07-16 18:36:24 +02:00
Victor Stinner 1cae9ec966 asyncio tests: make quiet the logs of SSL handshake failures when running tests
in debug mode
2014-07-14 22:26:34 +02:00
Victor Stinner acdb782a83 asyncio: sync with Tulip
* Tulip issue #184: Log subprocess events in debug mode

  - Log stdin, stdout and stderr transports and protocols
  - Log process identifier (pid)
  - Log connection of pipes
  - Log process exit
  - Log Process.communicate() tasks: feed stdin, read stdout and stderr
  - Add __repr__() method to many classes related to subprocesses


* Add BaseSubprocessTransport._pid attribute. Store the pid so it is still
  accessible after the process exited. It's more convinient for debug.

* create_connection(): add the socket in the "connected to" debug log

* Clean up some docstrings and comments. Remove unused unimplemented
  _read_from_self().
2014-07-14 18:33:40 +02:00
Victor Stinner e912e652f8 asyncio: sync with Tulip
* Tulip issue #183: log socket events in debug mode

  - Log most important socket events: socket connected, new client, connection
    reset or closed by peer (EOF), etc.
  - Log time elapsed in DNS resolution (getaddrinfo)
  - Log pause/resume reading
  - Log time of SSL handshake
  - Log SSL handshake errors
  - Add a __repr__() method to many classes

* Fix ProactorEventLoop() in debug mode. ProactorEventLoop._make_self_pipe()
  doesn't call call_soon() directly because it checks for the current loop
  which fails, because the method is called to build the event loop.

* Cleanup _ProactorReadPipeTransport constructor. Not need to set again
  _read_fut attribute to None, it is already done in the base class.
2014-07-12 03:11:53 +02:00
Victor Stinner 8ebeb03740 asyncio: improve the documentation of servers
- Fix the documentation of Server.close(): it closes sockets
- Replace AbstractServer with Server
- Document Server.sockets attribute
2014-07-11 23:47:40 +02:00
Victor Stinner b28dbac86d asyncio, Tulip issue 180: Make Server attributes and methods private
- loop, waiters and active_count attributes are now private
- attach(), detach() and wakeup() methods are now private

The sockets attribute remains public.
2014-07-11 22:52:21 +02:00
Victor Stinner 770e48d017 asyncio: sync with Tulip
* Tulip issue #182: Improve logs of BaseEventLoop._run_once()

  - Don't log non-blocking poll
  - Only log polling with a timeout if it gets events or if it timed out after
    more than 1 second.

* Fix some pyflakes warnings: remove unused imports
2014-07-11 11:58:33 +02:00
Victor Stinner 737c34fa85 asyncio: sync with Tulip
- CoroWrapper.__del__() now reuses repr(CoroWrapper) to log the "... was never
  yielded from" warning
- Improve CoroWrapper: copy also the qualified name on Python 3.4, not only on
  Python 3.5+
2014-07-11 01:04:16 +02:00
Victor Stinner c39ba7d611 asyncio: sync with Tulip
- repr(Task) and repr(CoroWrapper) now also includes where these objects were
  created. If the coroutine is not a generator (don't use "yield from"), use
  the location of the function, not the location of the coro() wrapper.
- Fix create_task(): truncate the traceback to hide the call to create_task().
2014-07-11 00:21:27 +02:00
Victor Stinner f68bd88aa6 asyncio: sync with Tulip
- Issues #21936, #21163: Fix sporadic failures of
  test_future_exception_never_retrieved()
- Handle.cancel() now clears references to callback and args
- In debug mode, repr(Handle) now contains the location where the Handle was
  created.
2014-07-10 22:32:58 +02:00
Victor Stinner bfff45d611 asyncion, Tulip issue 181: BaseEventLoop.create_datagram_endpoint() now waits
until protocol.connection_made() has been called. Document also why transport
constructors use a waiter.
2014-07-08 23:57:31 +02:00
Victor Stinner 896a25ab30 asyncio: sync with Tulip
- Tulip issue 185: Add a create_task() method to event loops. The create_task()
  method can be overriden in custom event loop to implement their own task
  class. For example, greenio and Pulsar projects use their own task class. The
  create_task() method is now preferred over creating directly task using the
  Task class.
- tests: fix a warning
- fix typo in the name of a test function
- Update AbstractEventLoop: add new event loop methods; update also the unit test
2014-07-08 11:29:25 +02:00
Victor Stinner 799a60ccb4 asyncio: sync with Tulip
Backout the "Tulip issue 181: Faster create_connection()" changeset, it was a
mistake.
2014-07-07 18:08:22 +02:00
Victor Stinner 1a870c9132 asyncio: sync with Tulip
- Tulip issue #181: Faster create_connection(). Call directly
  waiter.set_result() in the constructor of _ProactorBasePipeTransport and
  _SelectorSocketTransport, instead of using of delaying the call with
  call_soon().
- Cleanup iscoroutine()
2014-07-07 17:26:54 +02:00
Victor Stinner a9acbe82e7 Closes #21886, #21447: Fix a race condition in asyncio when setting the result
of a Future with call_soon(). Add an helper, a private method, to set the
result only if the future was not cancelled.
2014-07-05 15:29:41 +02:00
Victor Stinner 2dba23af71 asyncio: sync with Tulip
* _UnixSubprocessTransport: fix file mode of stdin. Open stdin in write mode,
  not in read mode
* Examples: close the event loop at exit
* More reliable CoroWrapper.__del__. If the constructor is interrupted by
  KeyboardInterrupt or the coroutine objet is destroyed lately, some the
  _source_traceback attribute doesn't exist anymore.
* repr(Task): include also the future the task is waiting for
2014-07-03 00:59:00 +02:00
Victor Stinner 98b6391fd4 Issue #21163: BaseEventLoop.run_until_complete() and test_utils.run_briefly()
don't log the "destroy pending task" message anymore. The log is redundant for
run_until_complete() and useless in run_briefly().
2014-06-30 14:51:04 +02:00
Victor Stinner b75380f333 asyncio: sync with Tulip
- Sort imports
- Simplify/optimize iscoroutine(). Inline inspect.isgenerator(obj): replace it
  with isinstance(obj, types.GeneratorType)
- CoroWrapper: check at runtime if Python has the yield-from bug #21209.  If
  Python has the bug, check if CoroWrapper.send() was called by yield-from to
  decide if parameters must be unpacked or not.
- Fix "Task was destroyed but it is pending!" warning in
  test_task_source_traceback()
2014-06-30 14:39:11 +02:00
Victor Stinner f951d28ac8 asyncio: sync with Tulip, add a new asyncio.coroutines module 2014-06-29 00:46:45 +02:00