Commit Graph

665 Commits

Author SHA1 Message Date
Miss Islington (bot) 17beebcc8b
bpo-33694: Fix typo in helper function name (GH-7522)
_feed_data_to_bufferred_proto() renamed to
_feed_data_to_buffered_proto() ("bufferred" => "buffered").

Typo spotted by Nathaniel J. Smith.
(cherry picked from commit ff6c077292)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-06-08 01:48:50 -07:00
Miss Islington (bot) 0738443a5b
bpo-33792: Add selector and proactor windows policies (GH-7487)
(cherry picked from commit 8f4042964d)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-06-07 18:31:50 -07:00
Miss Islington (bot) 8fa398d5ce bpo-33694: Fix race condition in asyncio proactor (GH-7498) (GH-7499)
The cancellation of an overlapped WSARecv() has a race condition
which causes data loss because of the current implementation of
proactor in asyncio.

No longer cancel overlapped WSARecv() in _ProactorReadPipeTransport
to work around the race condition.

Remove the optimized recv_into() implementation to get simple
implementation of pause_reading() using the single _pending_data
attribute.

Move _feed_data_to_bufferred_proto() to protocols.py.

Remove set_protocol() method which became useless.
(cherry picked from commit 79790bc35f)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-06-08 00:49:34 +02:00
Miss Islington (bot) 79c7e57c46 bpo-33769: start_tls: Fix error message; cancel callbacks on error (GH-7403) (GH-7428)
In addition to that, mark SSLTransport as "closed" in its "abort()" method to prevent bogus warnings.
(cherry picked from commit 415bc46a78)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-06-05 10:18:20 -04:00
Miss Islington (bot) 87936d03cb bpo-33734: asyncio/ssl: a bunch of bugfixes (GH-7321) (GH-7396)
* Fix AttributeError (not all SSL exceptions have 'errno' attribute)

* Increase default handshake timeout from 10 to 60 seconds
* Make sure start_tls can be cancelled correctly
* Make sure any error in SSLProtocol gets propagated (instead of just being logged)
(cherry picked from commit 9602643120)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-06-04 12:05:46 -04:00
Yury Selivanov af9cda9845
Revert "bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)" (GH-7233)
This reverts commit 2a7eb0b531.
2018-05-29 20:56:33 -04:00
Miss Islington (bot) d8948c5e09
bpo-32751: Wait for task cancellation in asyncio.wait_for() (GH-7216)
Currently, asyncio.wait_for(fut), upon reaching the timeout deadline,
cancels the future and returns immediately.  This is problematic for
when *fut* is a Task, because it will be left running for an arbitrary
amount of time.  This behavior is iself surprising and may lead to
related bugs such as the one described in bpo-33638:

    condition = asyncio.Condition()
    async with condition:
        await asyncio.wait_for(condition.wait(), timeout=0.5)

Currently, instead of raising a TimeoutError, the above code will fail
with `RuntimeError: cannot wait on un-acquired lock`, because
`__aexit__` is reached _before_ `condition.wait()` finishes its
cancellation and re-acquires the condition lock.

To resolve this, make `wait_for` await for the task cancellation.
The tradeoff here is that the `timeout` promise may be broken if the
task decides to handle its cancellation in a slow way.  This represents
a behavior change and should probably not be back-patched to 3.6 and
earlier.
(cherry picked from commit e2b340ab41)

Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
2018-05-29 15:37:06 -07:00
Miss Islington (bot) 036434273e bpo-32684: Fix gather to propagate cancel of itself with return_exceptions (GH-7209) (#7222)
(cherry picked from commit 863b674909)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-05-29 18:29:12 -04:00
Miss Islington (bot) 2a7eb0b531
bpo-22087: Fix Policy.get_event_loop() to detect fork (GH-7208)
Original patch by Dan O'Reilly.
(cherry picked from commit 5d97b7bcc1)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-05-29 14:09:17 -07:00
Miss Islington (bot) 441afbd9c3
bpo-33353: test_asyncio use set_write_buffer_limits() (GH-7200)
Use transport.set_write_buffer_limits() in sendfile tests of
test_asyncio to make sure that the protocol is paused after sending
4 KiB. Previously,
test_sendfile_fallback_close_peer_in_the_middle_of_receiving() failed
on FreeBSD if the DATA was smaller than the default limit of 64 KiB.
(cherry picked from commit 9551f77192)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-05-29 07:21:46 -07:00
Miss Islington (bot) be5d616e55
bpo-33654: Support protocol type switching in SSLTransport.set_protocol() (GH-7194)
(cherry picked from commit 2179022d94)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-05-29 02:19:09 -07:00
Miss Islington (bot) eca085993c
bpo-33674: Pause the transport as early as possible (GH-7192)
(cherry picked from commit f295587c45)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-05-28 22:59:03 -07:00
Miss Islington (bot) 0dd8fd0358 bpo-33674: asyncio: Fix SSLProtocol race (GH-7175) (#7187)
Fix a race condition in SSLProtocol.connection_made() of
asyncio.sslproto: start immediately the handshake instead of using
call_soon(). Previously, data_received() could be called before the
handshake started, causing the handshake to hang or fail.
(cherry picked from commit be00a5583a)

Co-authored-by: Victor Stinner <vstinner@redhat.com>
2018-05-28 21:44:16 -04:00
Miss Islington (bot) 420092e255 bpo-32410: Avoid blocking on file IO in sendfile fallback code (GH-7172) (#7182)
(cherry picked from commit 7165754b6b)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-05-28 21:42:45 -04:00
Miss Islington (bot) ddc613f203
bpo-32610: Fix asyncio.all_tasks() to return only pending tasks. (GH-7174)
(cherry picked from commit 416c1ebd98)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-05-28 17:16:43 -07:00
Miss Islington (bot) 8d8b86116f
bpo-33469: RuntimeError after closing loop that used run_in_executor (GH-7171)
(cherry picked from commit fdccfe09f0)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-05-28 15:50:02 -07:00
Miss Islington (bot) 075c662086 bpo-33672: Fix Task.__repr__ crash with Cython's bogus coroutines (GH-7161) (GH-7173)
(cherry picked from commit 989b9e0e6d)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-05-28 18:31:07 -04:00
Miss Islington (bot) bc3a002e7d
bpo-33654: Support BufferedProtocol in set_protocol() and start_tls() (GH-7130)
In this commit:

* Support BufferedProtocol in set_protocol() and start_tls()
* Fix proactor to cancel readers reliably
* Update tests to be compatible with OpenSSL 1.1.1
* Clarify BufferedProtocol docs
* Bump TLS tests timeouts to 60 seconds; eliminate possible race from start_serving
* Rewrite test_start_tls_server_1
(cherry picked from commit dbf102271f)

Co-authored-by: Yury Selivanov <yury@magic.io>
2018-05-28 11:50:45 -07:00
Miss Islington (bot) f8fdb368e3 bpo-33505: Optimize asyncio.ensure_future by reordering if conditions (GH-6836) (#7162)
(cherry picked from commit e549c4be5f)

Co-authored-by: jimmylai <albert_chs@yahoo.com.tw>
2018-05-28 14:42:50 -04:00
Miss Islington (bot) 1f21ae710d bpo-31647: Fix write_eof() after close() for SelectorSocketTransport (GH-7149) (GH-7153)
Fixed bug where calling write_eof() on a _SelectorSocketTransport after
it's already closed raises AttributeError.
(cherry picked from commit 23f587e395)

Co-authored-by: twisteroid ambassador <twisteroidambassador@users.noreply.github.com>
2018-05-28 12:04:09 -04:00
Miss Islington (bot) b8b800090f bpo-33263: Fix FD leak in _SelectorSocketTransport (GH-6450) (#7022)
* bpo-33263 Fix FD leak in _SelectorSocketTransport. (GH-6450)

Under particular circumstances _SelectorSocketTransport can try to add a reader
even the transport is already being closed. This can lead to FD leak and
invalid stated of the following connections. Fixed the SelectorSocketTransport
to add the reader only if the trasport is still active.
(cherry picked from commit a84d0b361a)

Co-authored-by: Vlad Starostin <drtyrsa@yandex.ru>
2018-05-21 11:35:25 +03:00
Miss Islington (bot) b85115ed4c [3.7] bpo-32262: Fix typo in f-string (GH-7016)
Fix typo from commit 6370f345e1

(cherry picked from commit d361e99868)
Co-authored-by: Christian Heimes <christian@python.org>
2018-05-20 20:22:50 +02:00
Miss Islington (bot) 52d1741c95 bpo-33584: Fix several minor bugs in asyncio. (GH-7003) (GH-7005)
Fix the following bugs in the C implementation:

* get_future_loop() silenced all exceptions raised when look up the get_loop
  attribute, not just an AttributeError.
* enter_task() silenced all exceptions raised when look up the current task,
  not just a KeyError.
* repr() was called for a borrowed link in enter_task() and task_step_impl().
* str() was used instead of repr() in formatting one error message (in
  Python implementation too).
* There where few reference leaks in error cases.
(cherry picked from commit 6655354afc)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2018-05-20 17:34:28 +03:00
Miss Islington (bot) 28ea38b97b
bpo-26819: Prevent proactor double read on resume (GH-6921)
The proactor event loop has a race condition when reading with
pausing/resuming. `resume_reading()` unconditionally schedules the read
function to read from the current future. If `resume_reading()` was
called before the previously scheduled done callback fires, this results
in two attempts to get the data from the most recent read and an
assertion failure. This commit tracks whether or not `resume_reading`
needs to reschedule the callback to restart the loop, preventing a
second attempt to read the data.
(cherry picked from commit 4151061855)

Co-authored-by: CtrlZvi <viz+github@flippedperspective.com>
2018-05-20 03:57:32 -07:00
Serhiy Storchaka fd93666c8a
[3.7] Fix duplicating words words. (GH-6296) (GH-6297)
Most of them have been added in 3.7.
(cherry picked from commit bac2d5ba30)
2018-03-28 23:05:24 +03:00
Miss Islington (bot) 211c0dbc6a Fix senfile typo (GH-6265) (#6274)
* Also in docs
(cherry picked from commit 65a34709f6)

Co-authored-by: Sam Dunster <me@sdunster.com>
2018-03-27 18:34:15 -07:00
Miss Islington (bot) bf0d116517 bpo-33037: Skip sending/receiving after SSL transport closing (GH-6044) (GH-6057)
* Skip write()/data_received() if sslpipe is destroyed
(cherry picked from commit 5e80a71ab6)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2018-03-10 18:27:01 +02:00
Miss Islington (bot) 632c1cb571
bpo-32622: Native sendfile on windows (GH-5565)
* Support sendfile on Windows Proactor event loop naively.
(cherry picked from commit a19fb3c6aa)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2018-02-25 09:10:58 -08:00
Miss Islington (bot) 8caee0fa57
bpo-32841: Fix cancellation in awaiting asyncio.Condition (GH-5665)
(cherry picked from commit 5746510b7a)

Co-authored-by: Bar Harel <bzvi7919@gmail.com>
2018-02-14 01:47:30 -08:00
Miss Islington (bot) 2b5937ec0a bpo-32734: Fix asyncio.Lock multiple acquire safety issue (GH-5466) (#5501)
(cherry picked from commit 2f79c01493)

Co-authored-by: Bar Harel <bzvi7919@gmail.com>
2018-02-02 18:14:38 -05:00
Miss Islington (bot) 71a0b0e6e8 Implement TimerHandle.when() (GH-5473) (GH-5480)
(cherry picked from commit 3d4dbd8f01)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2018-02-01 21:56:03 +02:00
Yury Selivanov 07627e9a6a
bpo-32251: Fix docs (#5408) 2018-01-28 23:51:08 -05:00
Yury Selivanov 631fd38dbf
bpo-32251: Implement asyncio.BufferedProtocol. (#4755) 2018-01-28 16:30:26 -05:00
Yury Selivanov bec2372b7e
bpo-32327: Revert loop.run_in_executor behaviour: return a Future. (#5392)
I've run some tests on 3.7 asyncio and it appears that too many
things assume that run_in_executor returns a Future.
2018-01-28 14:09:40 -05:00
Yury Selivanov 2a2247ce5e
bpo-32622: Normalize ENOTCONN to ConnectionError on macOS (GH-5369)
On mac, sendfile throws ENOTCONN on a repeated sendfile call if
the connection is closed. Normalize it to behave like other systems.
2018-01-27 17:22:01 -05:00
Yury Selivanov b1a6ac4c40
bpo-32622: Enforce sendfile fallback policy for FALLBACK transports (#5364) 2018-01-27 15:52:52 -05:00
Andrew Svetlov 7c684073f9
bpo-32622: Implement loop.sendfile() (#5271) 2018-01-27 21:22:47 +02:00
Christian Heimes 61d478c71c
bpo-31399: Let OpenSSL verify hostname and IP address (#3462)
bpo-31399: Let OpenSSL verify hostname and IP

The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and
X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses.

* Remove match_hostname calls
* Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host()
* Add documentation for OpenSSL 1.0.2 requirement
* Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform.
* Add hostname_checks_common_name

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-27 15:51:38 +01:00
José Melero Fernández c47dacb690 bpo-32574: Fix leaks in asyncio.Queue.put() and .get() (#5208) 2018-01-25 18:45:43 -05:00
Yury Selivanov c9070d03f5
bpo-32662: Implement Server.start_serving() and Server.serve_forever() (#5312)
* bpo-32662: Implement Server.start_serving() and Server.serve_forever()

New methods:

* Server.start_serving(),
* Server.serve_forever(), and
* Server.is_serving().

Add 'start_serving' keyword parameter to loop.create_server() and
loop.create_unix_server().
2018-01-25 18:08:09 -05:00
Andrew Svetlov fe133aad52
bpo-32391: Implement StreamWriter.wait_closed() (#5281) 2018-01-25 00:30:30 +02:00
Nathaniel J. Smith fb5a7ad421 bpo-32636: Fix @asyncio.coroutine debug mode bug exposed by gh-5250 (#5291) 2018-01-24 15:14:33 -05:00
Yury Selivanov 22feeb88b4
bpo-32643: Drop support for a few private Task and Future APIs. (#5293)
Specifically, it's not possible to subclass Task/Future classes
and override the following methods:

* Future._schedule_callbacks
* Task._step
* Task._wakeup
2018-01-24 11:31:01 -05:00
Yury Selivanov f23746a934
bpo-32436: Implement PEP 567 (#5027) 2018-01-22 19:11:18 -05:00
Yury Selivanov a4afcdfa55
bpo-32314: Fix asyncio.run() to cancel runinng tasks on shutdown (#5262) 2018-01-21 14:56:59 -05:00
Nathaniel J. Smith fc2f407829 bpo-32591: Add native coroutine origin tracking (#5250)
* Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth
* Use coroutine origin information in the unawaited coroutine warning
* Stop using set_coroutine_wrapper in asyncio debug mode
* In BaseEventLoop.set_debug, enable debugging in the correct thread
2018-01-21 09:44:07 -05:00
Andrew Svetlov 7464e87a65
bpo-32410: Make SendfileNotAvailableError exception public (#5243) 2018-01-19 20:04:29 +02:00
Andrew Svetlov 6b5a27975a
bpo-32410: Implement loop.sock_sendfile() (#4976) 2018-01-16 19:59:34 +02:00
Andrew Svetlov ffcb4c0165
bpo-32418: Postfix, raise NotImplementdError and close resources in tests (#5052) 2017-12-30 18:52:56 +02:00
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) 1634fc289a bpo-32418: Add get_loop() method on Server, AbstractServer classes (#4997)
* Add abstract get_loop() method to Server, AbstractServer classes.
* Add test cases for get_loop() method in Server, AbstractServer classes
* Add documentation for get_loop() method
2017-12-30 17:09:32 +02:00