Commit Graph

103 Commits

Author SHA1 Message Date
Andrew Svetlov cc0961c517
[3.6] bpo-32015: Asyncio looping during simultaneously socket read/write an… (GH-4386) (#4393)
* bpo-32015: Asyncio cycling during simultaneously socket read/write and reconnection

* Tests fix

* Tests fix

* News add

* Add new unit tests.
(cherry picked from commit e1d62e0b7c)
2017-11-14 12:14:51 +02:00
Yury Selivanov bb8eb92f46 Merge 3.5 (issue #28369) 2016-10-05 17:49:54 -04:00
Yury Selivanov 5b8d4f97f8 Issue #28369: Raise an error when transport's FD is used with add_reader 2016-10-05 17:48:59 -04:00
Yury Selivanov 50fe7114ae Merge 3.5 (issue #28176) 2016-09-15 17:58:15 -04:00
Yury Selivanov d6c6771fc9 Issue #28176: Fix callbacks race in asyncio.SelectorLoop.sock_connect. 2016-09-15 17:56:36 -04:00
Yury Selivanov d070154fb5 Merge 3.5 (issue #27906) 2016-09-15 14:14:48 -04:00
Yury Selivanov a1b0e7db73 Issue #27906: Fix socket accept exhaustion during high TCP traffic.
Patch by Kevin Conway.
2016-09-15 14:13:15 -04:00
Yury Selivanov 503ba3ed97 Merge 3.5 (issue #27456) 2016-09-11 21:44:17 -04:00
Yury Selivanov 44c19eccf9 Issue #27456: asyncio: Set TCP_NODELAY by default. 2016-09-11 21:39:31 -04:00
Yury Selivanov 966321e236 Merge 3.5 (asyncio) 2016-09-11 21:11:19 -04:00
Yury Selivanov a05a6ef1ca asyncio: Add set_protocol / get_protocol methods to Transports 2016-09-11 21:11:02 -04:00
Yury Selivanov 5eef78b113 Merge 3.5 (asyncio) 2016-06-28 11:00:39 -04:00
Yury Selivanov 63bf487688 asyncio: Use socket specs for getaddrinfo() in sock_connect()
Patch by Martin Richard.  GH PR #365.
2016-06-28 11:00:22 -04:00
Yury Selivanov 6588712bf0 Merge 3.5 (issue #27223) 2016-06-11 11:20:50 -04:00
Yury Selivanov ca2e0a48cf Issue #27223: aio: Fix _read_ready and _write_ready to respect _conn_lost.
Patch by Łukasz Langa.
2016-06-11 11:19:47 -04:00
Yury Selivanov 552bf94648 Merge 3.5 (issue #27136, asyncio) 2016-06-08 12:33:59 -04:00
Yury Selivanov f1c6fa9866 Issue #27136: Fix DNS static resolution; don't use it in getaddrinfo
Patch by A. Jesse Jiryu Davis
2016-06-08 12:33:31 -04:00
Yury Selivanov 8ace2c66d3 Merge 3.5 (Issue #27041) 2016-05-16 15:39:39 -04:00
Yury Selivanov 7661db6228 Issue #27041: asyncio: Add loop.create_future method 2016-05-16 15:38:39 -04:00
Serhiy Storchaka b6a9c9761c Issue #26778: Fixed "a/an/and" typos in code comment, documentation and error
messages.
2016-04-17 09:39:28 +03:00
Serhiy Storchaka 6a7b3a77b4 Issue #26778: Fixed "a/an/and" typos in code comment and documentation. 2016-04-17 08:32:47 +03:00
Victor Stinner 00f4648a39 Merge 3.5 (asyncio) 2016-04-01 21:43:54 +02:00
Victor Stinner c94a93aecb asyncio: Don't log ConnectionAbortedError
Issue #26509: In fatal error handlers, don't log ConnectionAbortedError which
occur on Windows.
2016-04-01 21:43:39 +02:00
Victor Stinner e19558af1b Add a source parameter to warnings.warn()
Issue #26604:

* Add a new optional source parameter to _warnings.warn() and warnings.warn()
* Modify asyncore, asyncio and _pyio modules to set the source parameter when
  logging a ResourceWarning warning
2016-03-23 00:28:08 +01:00
Victor Stinner 3e7230904e Fix error message in asyncio.selector_events.
Patch written by Carlo Beccarini <hackdiablo.cb@gmail.com>.
2016-02-01 12:46:38 +01:00
Yury Selivanov d5c2a62100 asyncio: Skip getaddrinfo if host is already resolved.
getaddrinfo takes an exclusive lock on some platforms, causing clients to queue
up waiting for the lock if many names are being resolved concurrently. Users
may want to handle name resolution in their own code, for the sake of caching,
using an alternate resolver, or to measure DNS duration separately from
connection duration. Skip getaddrinfo if the "host" passed into
create_connection is already resolved.

See https://github.com/python/asyncio/pull/302 for details.

Patch by A. Jesse Jiryu Davis.
2015-12-16 19:31:17 -05:00
Yury Selivanov 5d7e3b6cd2 asyncio: Cleanup Future API
See https://github.com/python/asyncio/pull/292 for details.
2015-11-17 12:19:41 -05:00
Yury Selivanov 5bb1afb332 asyncio: Add Transport.is_closing()
See https://github.com/python/asyncio/pull/291 for details.
2015-11-16 12:43:21 -05:00
Victor Stinner f7dc7fb74d Issue #25114, asyncio: add ssl_object extra info to SSL transports
This info is required on Python 3.5 and newer to get specific information on
the SSL object, like getting the binary peer certificate (instead of getting
it as text).
2015-09-21 18:06:17 +02:00
Yury Selivanov 2a8911c0b7 asyncio: Sync with upstream (compat module) 2015-08-04 15:56:33 -04:00
Victor Stinner c9d11c341e Issue #23879, asyncio: SelectorEventLoop.sock_connect() must not call connect()
again if the first call to connect() raises an InterruptedError.

When the C function connect() fails with EINTR, the connection runs in
background. We have to wait until the socket becomes writable to be notified
when the connection succeed or fails.
2015-04-07 21:38:04 +02:00
Victor Stinner 79fd962652 asyncio: Fix _SelectorTransport.__repr__() if the event loop is closed 2015-03-27 15:20:08 +01:00
Victor Stinner 2fc2313038 asyncio: Only call _check_resolved_address() in debug mode
* _check_resolved_address() is implemented with getaddrinfo() which is slow
* If available, use socket.inet_pton() instead of socket.getaddrinfo(), because
  it is much faster

Microbenchmark (timeit) on Fedora 21 (Python 3.4, Linux 3.17, glibc 2.20) to
validate the IPV4 address "127.0.0.1" or the IPv6 address "::1":

* getaddrinfo() 10.4 usec per loop
* inet_pton(): 0.285 usec per loop

On glibc older than 2.14, getaddrinfo() always requests the list of all local
IP addresses to the kernel (using a NETLINK socket). getaddrinfo() has other
known issues, it's better to avoid it when it is possible.
2015-02-04 14:51:23 +01:00
Victor Stinner aa41b9b22b asyncio: BaseSelectorEventLoop uses directly the private _debug attribute
Just try to be consistent: _debug was already used in some places, and always
used in BaseProactorEventLoop.
2015-02-04 14:50:59 +01:00
Victor Stinner 978a9afc6a Issue #23243, asyncio: Emit a ResourceWarning when an event loop or a transport
is not explicitly closed. Close also explicitly transports in test_sslproto.
2015-01-29 17:50:58 +01:00
Victor Stinner 2934262fd3 asyncio: sync with Tulip
* Cleanup gather(): use cancelled() method instead of using private Future
  attribute
* Fix _UnixReadPipeTransport and _UnixWritePipeTransport. Only start reading
  when connection_made() has been called.
* Issue #23333: Fix BaseSelectorEventLoop._accept_connection(). Close the
  transport on error. In debug mode, log errors using call_exception_handler()
2015-01-29 14:15:19 +01:00
Victor Stinner 47bbea7124 asyncio: sync with Tulip
* _SelectorTransport constructor: extra parameter is now optional
* Fix _SelectorDatagramTransport constructor. Only start reading after
  connection_made() has been called.
* Fix _SelectorSslTransport.close(). Don't call protocol.connection_lost() if
  protocol.connection_made() was not called yet: if the SSL handshake failed or
  is still in progress. The close() method can be called if the creation of the
  connection is cancelled, by a timeout for example.
2015-01-29 02:56:05 +01:00
Victor Stinner fa73779b0a asyncio: Fix _SelectorSocketTransport constructor
Only start reading when connection_made() has been called:
protocol.data_received() must not be called before protocol.connection_made().
2015-01-29 00:36:51 +01:00
Victor Stinner f07801bb17 asyncio: SSL transports now clear their reference to the waiter
* Rephrase also the comment explaining why the waiter is not awaken immediatly.
* SSLProtocol.eof_received() doesn't instanciate ConnectionResetError exception
  directly, it will be done by Future.set_exception(). The exception is not
  used if the waiter was cancelled or if there is no waiter.
2015-01-29 00:36:35 +01:00
Victor Stinner 442b0adccd asyncio: pyflakes, remove unused import
tests: Remove unused function; inline another function
2015-01-21 23:39:16 +01:00
Victor Stinner 29ad0111bd asyncio: sync with Tulip
* PipeHandle now uses None instead of -1 for a closed handle
* Sort imports in windows_utils.
* Fix test_events on Python older than 3.5. Skip SSL tests on the
  ProactorEventLoop if ssl.MemoryIO is missing
* Fix BaseEventLoop._create_connection_transport(). Close the transport if the
  creation of the transport (if the waiter) gets an exception.
* _ProactorBasePipeTransport now sets _sock to None when the transport is
  closed.
* Fix BaseSubprocessTransport.close(). Ignore pipes for which the protocol is
  not set yet (still equal to None).
* TestLoop.close() now calls the close() method of the parent class
  (BaseEventLoop).
* Cleanup BaseSelectorEventLoop: create the protocol on a separated line for
  readability and ease debugging.
* Fix BaseSubprocessTransport._kill_wait(). Set the _returncode attribute, so
  close() doesn't try to terminate the process.
* Tests: explicitly close event loops and transports
* UNIX pipe transports: add closed/closing in repr(). Add "closed" or "closing"
  state in the __repr__() method of _UnixReadPipeTransport and
  _UnixWritePipeTransport classes.
2015-01-15 00:04:21 +01:00
Victor Stinner b92626df5c Issue #23197: On SSL handshake failure on matching hostname, check if the
waiter is cancelled before setting its exception.
2015-01-14 17:13:28 +01:00
Victor Stinner 177e9f0855 Issue #23197, asyncio: On SSL handshake failure, check if the waiter is
cancelled before setting its exception.

* Add unit tests for this case.
* Cleanup also sslproto.py
2015-01-14 16:56:20 +01:00
Victor Stinner 231b404cb0 Issue #22560: New SSL implementation based on ssl.MemoryBIO
The new SSL implementation is based on the new ssl.MemoryBIO which is only
available on Python 3.5. On Python 3.4 and older, the legacy SSL implementation
(using SSL_write, SSL_read, etc.) is used. The proactor event loop only
supports the new implementation.

The new asyncio.sslproto module adds _SSLPipe, SSLProtocol and
_SSLProtocolTransport classes. _SSLPipe allows to "wrap" or "unwrap" a socket
(switch between cleartext and SSL/TLS).

Patch written by Antoine Pitrou. sslproto.py is based on gruvi/ssl.py of the
gruvi project written by Geert Jansen.

This change adds SSL support to ProactorEventLoop on Python 3.5 and newer!

It becomes also possible to implement STARTTTLS: switch a cleartext socket to
SSL.
2015-01-14 00:19:09 +01:00
Victor Stinner 70db9e428a asyncio: sync with Tulip
* Tulip issue 184: FlowControlMixin constructor now get the event loop if the
  loop parameter is not set. Add unit tests to ensure that constructor of
  StreamReader and StreamReaderProtocol classes get the event loop.
* Remove outdated TODO/XXX
2015-01-09 21:32:05 +01:00
Victor Stinner 3531d9044d asyncio: sync with Tulip
* Document why set_result() calls are safe
* Cleanup gather(). Use public methods instead of hacks to consume the
  exception of a future.
* sock_connect(): pass directly the fd to _sock_connect_done instead of the
  socket.
2015-01-09 01:42:52 +01:00
Victor Stinner 15cc678d89 asyncio: Truncate to 80 columns 2015-01-09 00:09:10 +01:00
Victor Stinner 8d9c145f61 asyncio: _make_ssl_transport: make the waiter parameter optional 2015-01-08 12:06:36 +01:00
Victor Stinner 956de691f8 Issue #22926: In debug mode, call_soon(), call_at() and call_later() methods of
asyncio.BaseEventLoop now use the identifier of the current thread to ensure
that they are called from the thread running the event loop.

Before, the get_event_loop() method was used to check the thread, and no
exception was raised when the thread had no event loop. Now the methods always
raise an exception in debug mode when called from the wrong thread. It should
help to notice misusage of the API.
2014-12-26 21:07:52 +01:00
Benjamin Peterson 7243b574e5 don't require OpenSSL SNI to pass hostname to ssl functions (#22921)
Patch by Donald Stufft.
2014-11-23 17:04:34 -06:00