concurrent.futures: The *executor manager thread* now catches
exceptions when adding an item to the *call queue*. During Python
finalization, creating a new thread can now raise RuntimeError. Catch
the exception and call terminate_broken() in this case.
Add test_python_finalization_error() to test_concurrent_futures.
concurrent.futures._ExecutorManagerThread changes:
* terminate_broken() no longer calls shutdown_workers() since the
call queue is no longer working anymore (read and write ends of
the queue pipe are closed).
* terminate_broken() now terminates child processes, not only
wait until they complete.
* _ExecutorManagerThread.terminate_broken() now holds shutdown_lock
to prevent race conditons with ProcessPoolExecutor.submit().
multiprocessing.Queue changes:
* Add _terminate_broken() method.
* _start_thread() sets _thread to None on exception to prevent
leaking "dangling threads" even if the thread was not started
yet.
Fix deadlocks in :class:`concurrent.futures.ProcessPoolExecutor` when task arguments or results cause pickling or unpickling errors.
This should make sure that calls to the :class:`ProcessPoolExecutor` API always eventually return.
* Make error message more informative
Replace assertions in error-reporting code with more-informative version that doesn't cause confusion over where and what the error is.
* Additional clarification + get travis to check
* Change from SystemError to TypeError
As suggested in PR comment by @pitrou, changing from SystemError; TypeError appears appropriate.
* NEWS file installation; ACKS addition (will do my best to justify it by additional work)
* Making current AssertionErrors in multiprocessing more informative
* Blurb added re multiprocessing managers.py, queues.py cleanup
* Further multiprocessing cleanup - went through pool.py
* Fix two asserts in multiprocessing/util.py
* Most asserts in multiprocessing more informative
* Didn't save right version
* Further work on multiprocessing error messages
* Correct typo
* Correct typo v2
* Blasted colon... serves me right for trying to work on two things at once
* Simplify NEWS entry
* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst
* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst
OK, never mind.
* Corrected (thanks to pitrou) error messages for notify
* Remove extraneous backslash in docstring.
multiprocessing.Queue.join_thread() now waits until the thread
completes, even if the thread was started by the same process which
created the queue.
Fix the following warning which occurs randomly when running
test_handle_called_with_mp_queue of test_logging.QueueListenerTest:
Warning -- threading_cleanup() failed to cleanup -1 threads after 4 sec (count: 0, dangling: 1)
multiprocessing.Queue.get() with a timeout now polls its reader in
non-blocking mode if it succeeded to aquire the lock but the acquire
took longer than the timeout.
Co-Authored-By: Grzegorz Grzywacz <grzgrzgrz3@gmail.com>
* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc
Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.
* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc
Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.
Also, add a multiprocessing.connection.wait(rlist, timeout=None) function
for polling multiple objects at once. Patch by sbt.
Complete changelist from sbt's patch:
* Adds a wait(rlist, timeout=None) function for polling multiple
objects at once. On Unix this is just a wrapper for
select(rlist, [], [], timeout=None).
* Removes use of the SentinelReady exception and the sentinels argument
to certain methods. concurrent.futures.process has been changed to
use wait() instead of SentinelReady.
* Fixes bugs concerning PipeConnection.poll() and messages of zero
length.
* Fixes PipeListener.accept() to call ConnectNamedPipe() with
overlapped=True.
* Fixes Queue.empty() and SimpleQueue.empty() so that they are
threadsafe on Windows.
* Now PipeConnection.poll() and wait() will not modify the pipe except
possibly by consuming a zero length message. (Previously poll()
could consume a partial message.)
* All of multiprocesing's pipe related blocking functions/methods are
now interruptible by SIGINT on Windows.
svn+ssh://pythondev@svn.python.org/python/trunk
........
r65824 | benjamin.peterson | 2008-08-18 13:01:43 -0500 (Mon, 18 Aug 2008) | 1 line
change a few uses of the threading APIs
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r64125 | benjamin.peterson | 2008-06-11 12:27:50 -0500 (Wed, 11 Jun 2008) | 2 lines
give the threading API PEP 8 names
........
Victor Stinner
Nikita Sobolev
Géry Ogam
Dong-hee Na
Batuhan Taşkaya
Zackery Spytz
Victor Stinner
Thomas Moreau
Allen W. Smith, Ph.D
Victor Stinner
grzgrzgrz3
Xiang Zhang
Davin Potts
Serhiy Storchaka
Berker Peksag
Charles-François Natali
Richard Oudkerk
Andrew Svetlov
Antoine Pitrou
Florent Xicluna
R. David Murray
Benjamin Peterson
Jesse Noller