Fix memory leaks in asyncio ProactorEventLoop on overlapped operation
failures.
Changes:
* Implement the tp_traverse slot in the _overlapped.Overlapped type
to help to break reference cycles and identify referrers in the
garbage collector.
* Always clear overlapped on failure: not only set type to
TYPE_NOT_STARTED, but release also resources.
(cherry picked from commit 5485085b32)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Fix a memory leak in asyncio in the ProactorEventLoop when ReadFile()
or WSASend() overlapped operation fail immediately: release the
internal buffer.
(cherry picked from commit a234e14839)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* Support sendfile on Windows Proactor event loop naively.
(cherry picked from commit a19fb3c6aa)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
On Python 3.3, use aliases:
* PyMem_RawMalloc = PyMem_Malloc
* PyMem_RawFree = PyMem_Free
These aliases are not need in Python 3.5, but this change makes synchronization
of code base simpler.
Issue #26563: Replace PyMem_Malloc() with PyMem_RawFree() since
PostToQueueCallback() calls PyMem_RawFree() (previously PyMem_Free()) in a new
C thread which doesn't hold the GIL.
If ReadFile() fails with ERROR_BROKEN_PIPE, the operation is not pending: don't
register the overlapped.
I don't know if WSARecv() can fail with ERROR_BROKEN_PIPE. Since
Overlapped.WSARecv() already handled ERROR_BROKEN_PIPE, let me guess that it
has the same behaviour than ReadFile().
Overlapped.ConnectNamedPipe() now returns a boolean: True if the pipe is
connected (if ConnectNamedPipe() failed with ERROR_PIPE_CONNECTED), False if
the connection is in progress.
This change removes multiple hacks in IocpProactor.
Add _overlapped.ConnectPipe() which tries to connect to the pipe for
asynchronous I/O (overlapped): call CreateFile() in a loop until it doesn't
fail with ERROR_PIPE_BUSY. Use an increasing delay between 1 ms and 100 ms.
Remove Overlapped.WaitNamedPipeAndConnect() which is no more used.
This change fixes a race conditon related to _WaitHandleFuture.cancel() leading
to Python crash or "GetQueuedCompletionStatus() returned an unexpected event"
logs. Before, the overlapped object was destroyed too early, it was possible
that the wait completed whereas the overlapped object was already destroyed.
Sometimes, a different overlapped was allocated at the same address, leading to
unexpected completition.
_WaitHandleFuture.cancel() now waits until the wait is cancelled to clear its
reference to the overlapped object. To wait until the cancellation is done,
UnregisterWaitEx() is used with an event instead of UnregisterWait().
To wait for this event, a new _WaitCancelFuture class was added. It's a
simplified version of _WaitCancelFuture. For example, its cancel() method calls
UnregisterWait(), not UnregisterWaitEx(). _WaitCancelFuture should not be
cancelled.
The overlapped object is kept alive in _WaitHandleFuture until the wait is
unregistered.
Other changes:
* Add _overlapped.UnregisterWaitEx()
* Remove fast-path in IocpProactor.wait_for_handle() to immediatly set the
result if the wait already completed. I'm not sure that it's safe to
call immediatly UnregisterWaitEx() before the completion was signaled.
* Add IocpProactor._unregistered() to forget an overlapped which may never be
signaled, but may be signaled for the next loop iteration. It avoids to
block forever IocpProactor.close() if a wait was cancelled, and it may also
avoid some "... unexpected event ..." warnings.
Miss Islington (bot)
Andrew Svetlov
Serhiy Storchaka
Antoine Pitrou
Oren Milman
Steve Dower
Victor Stinner
Guido van Rossum