Commit Graph

56 Commits

Author SHA1 Message Date
Victor Stinner 9a8d1d7562
bpo-35424: emit ResourceWarning at multiprocessing.Pool destruction (GH-10974)
multiprocessing.Pool destructor now emits ResourceWarning
if the pool is still running.
2018-12-20 20:33:51 +01:00
Victor Stinner 2dfe3511fe
bpo-35491, multiprocessing: replace "RUN" with RUN (GH-11178) 2018-12-16 23:40:49 +01:00
Victor Stinner 2b417fba25
Add multiprocessing.Pool.__repr__() (GH-11137)
* Add multiprocessing.Pool.__repr__() to ease debug
* RUN, CLOSE and TERMINATE constants values are now strings rather
  than integer to ease debug
2018-12-14 11:13:18 +01:00
Victor Stinner 08c2ba0717
bpo-35477: multiprocessing.Pool.__enter__() fails if called twice (GH-11134)
multiprocessing.Pool.__enter__() now fails if the pool is not
running: "with pool:" fails if used more than once.
2018-12-13 02:15:30 +01:00
Victor Stinner 9dfc754d61
Revert "bpo-34172: multiprocessing.Pool leaks resources after being deleted (GH-8450)" (GH-10971)
This reverts commit 97bfe8d3eb.
2018-12-06 08:51:47 +01:00
Julien Palard 5d236cafd7
bpo-19675: Terminate processes if construction of a pool is failing. (GH-5614) 2018-11-04 23:40:32 +01:00
tzickel 97bfe8d3eb bpo-34172: multiprocessing.Pool leaks resources after being deleted (GH-8450)
Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly.
2018-10-02 23:01:23 +02:00
Antoine Pitrou ab74504346
bpo-32576: use queue.SimpleQueue in critical places (#5216)
Where a queue may be invoked from a weakref callback, we need
to use the reentrant SimpleQueue.
2018-01-18 10:38:03 +01:00
Antoine Pitrou e91c2a5086 [Trivial] Remove now redundant assert (#3245) 2017-08-30 16:57:12 +02:00
Allen W. Smith, Ph.D bd73e72b4a bpo-5001: More-informative multiprocessing error messages (#3079)
* Make error message more informative

Replace assertions in error-reporting code with more-informative version that doesn't cause confusion over where and what the error is.

* Additional clarification + get travis to check

* Change from SystemError to TypeError

As suggested in PR comment by @pitrou, changing from SystemError; TypeError appears appropriate.

* NEWS file installation; ACKS addition (will do my best to justify it by additional work)

* Making current AssertionErrors in multiprocessing more informative

* Blurb added re multiprocessing managers.py, queues.py cleanup

* Further multiprocessing cleanup - went through pool.py

* Fix two asserts in multiprocessing/util.py

* Most asserts in multiprocessing more informative

* Didn't save right version

* Further work on multiprocessing error messages

* Correct typo

* Correct typo v2

* Blasted colon... serves me right for trying to work on two things at once

* Simplify NEWS entry

* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst

* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst

OK, never mind.

* Corrected (thanks to pitrou) error messages for notify

* Remove extraneous backslash in docstring.
2017-08-30 00:52:18 +02:00
Serhiy Storchaka 5affd23e6f bpo-29762: More use "raise from None". (#569)
This hides unwanted implementation details from tracebacks.
2017-04-05 09:37:24 +03:00
Xiang Zhang 794623bdb2 bpo-28699: fix abnormal behaviour of pools in multiprocessing.pool (GH-693)
an exception raised at the very first of an iterable would cause pools behave abnormally
(swallow the exception or hang)
2017-03-29 11:58:54 +08:00
Antoine Pitrou 8988945cdc bpo-29861: release references to multiprocessing Pool tasks (#743)
* bpo-29861: release references to multiprocessing Pool tasks

Release references to tasks, their arguments and their results as soon
as they are finished, instead of keeping them alive until another task
arrives.

* Comments in test
2017-03-24 13:52:11 +01:00
Charles-François Natali 78f55ffc63 Issue #23992: multiprocessing: make MapResult not fail-fast upon exception. 2016-02-10 22:58:18 +00:00
Serhiy Storchaka 63623ac252 Issue #23051: multiprocessing.Pool methods imap() and imap_unordered() now
handle exceptions raised by an iterator.  Patch by Alon Diamant and Davin
Potts.
2015-03-13 08:30:33 +02:00
Serhiy Storchaka 79fbeee237 Issue #23051: multiprocessing.Pool methods imap() and imap_unordered() now
handle exceptions raised by an iterator.  Patch by Alon Diamant and Davin
Potts.
2015-03-13 08:25:26 +02:00
Serhiy Storchaka 465e60e654 Issue #22033: Reprs of most Python implemened classes now contain actual
class name instead of hardcoded one.
2014-07-25 23:36:00 +03:00
Charles-François Natali a924fc7abc Issue #21565: multiprocessing: use contex-manager protocol for synchronization
primitives.
2014-05-25 14:12:12 +01:00
Richard Oudkerk 80a5be1d84 Issue #20980: Stop wrapping exception when using ThreadPool. 2014-03-23 12:30:54 +00:00
Victor Stinner 7fa767e517 Issue #20976: pyflakes: Remove unused imports 2014-03-20 09:16:38 +01:00
Richard Oudkerk 33aaa73cad Merge. 2013-10-28 23:23:04 +00:00
Richard Oudkerk e90cedb711 Issue #19425 -- a pickling error should not cause pool to hang. 2013-10-28 23:11:58 +00:00
Richard Oudkerk b1694cf588 Issue #18999: Make multiprocessing use context objects.
This allows different parts of a program to use different methods for
starting processes without interfering with each other.
2013-10-16 16:41:56 +01:00
Richard Oudkerk 84ed9a68bd Issue #8713: Support alternative start methods in multiprocessing on Unix.
See http://hg.python.org/sandbox/sbt#spawn
2013-08-14 15:35:41 +01:00
Charles-François Natali 37cfb0a920 Issue #17914: Use os.cpu_count() instead of multiprocessing.cpu_count() where
applicable.
2013-06-28 19:25:45 +02:00
Richard Oudkerk f3a0da24f3 Merge 2013-05-06 12:13:50 +01:00
Richard Oudkerk def51ca8ea Issue #17805: Add AsyncResult alias for ApplyResult 2013-05-06 12:10:04 +01:00
Richard Oudkerk 8575783a00 Issue #13813: Embed stringification of remote traceback in local
traceback raised when pool task raises an exception.
2013-05-06 11:38:25 +01:00
Andrew Svetlov f7a17b48d7 Replace IOError with OSError (#16715) 2012-12-25 16:47:37 +02:00
Hynek Schlawack 254af2644a #16307: Fix multiprocessing.Pool.map_async not calling its callbacks
Patch by Janne Karila.
2012-10-27 12:53:02 +02:00
Benjamin Peterson 3095f4724e raise a ValueError instead of an AssertionError when pool is an invalid state 2012-09-25 12:45:42 -04:00
Richard Oudkerk d69cfe88ea Issue #15064: Implement context manager protocol for multiprocessing types 2012-06-18 17:47:52 +01:00
Richard Oudkerk 0f884273b0 Issue #15101: Make pool finalizer avoid joining current thread. 2012-06-18 16:02:49 +01:00
Richard Oudkerk f29ec4b0c8 Issue #15101: Make pool finalizer avoid joining current thread. 2012-06-18 15:54:57 +01:00
Richard Oudkerk 58ba47f97b Merge fixes for #13854 and #12157. 2012-06-07 20:38:11 +01:00
Richard Oudkerk e41682b994 Issue #12157: pool.map() does not handle empty iterable correctly
Initial patch by mouad
2012-06-06 19:04:57 +01:00
Richard Oudkerk 692130a231 Issue #12091: simplify ApplyResult and MapResult with threading.Event
Patch by Charles-François Natali
2012-05-25 13:26:53 +01:00
Richard Oudkerk 3e268aac3b Mark multiprocessing files with "Licensed to PSF under a Contributor Agreement"
instead of BSD licence.
2012-04-30 12:13:55 +01:00
Antoine Pitrou de911b2915 Issue #12708: Add starmap() and starmap_async() methods (similar to itertools.starmap()) to multiprocessing.Pool.
Patch by Hynek Schlawack.
2011-12-21 11:03:24 +01:00
Florent Xicluna 5d1155c08e Closes #13258: Use callable() built-in in the standard library. 2011-10-28 14:45:05 +02:00
Charles-François Natali f8859e1808 Issue #10332: multiprocessing: fix a race condition when a Pool is closed
before all tasks have completed.
2011-10-24 18:45:29 +02:00
Victor Stinner 2fae27b735 Issue #12285: multiprocessing.Pool() raises a ValueError if the number of
processes if negative or null.
2011-06-20 17:53:35 +02:00
Antoine Pitrou bed9a5b6b3 Issue #11814: Fix likely typo in multiprocessing.Pool._terminate(). 2011-04-11 00:20:23 +02:00
Antoine Pitrou 81dee6b4d4 Issue #8428: Fix a race condition in multiprocessing.Pool when terminating
worker processes: new processes would be spawned while the pool is being
shut down.  Patch by Charles-François Natali.
2011-04-11 00:18:59 +02:00
R. David Murray 3fc969a4a2 9162: fix license in multiprocessing files 2010-12-14 01:38:16 +00:00
Ask Solem 2afcbf2249 Issue #9244: multiprocessing.pool: Worker crashes if result can't be encoded 2010-11-09 20:55:52 +00:00
Antoine Pitrou 7c3e577395 Issue #7316: the acquire() method of lock objects in the :mod:`threading`
module now takes an optional timeout argument in seconds.  Timeout support
relies on the system threading library, so as to avoid a semi-busy wait
loop.
2010-04-14 15:44:10 +00:00
Florent Xicluna 998171f0d7 Merged revisions 78777,78787,78790 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78777 | florent.xicluna | 2010-03-08 00:49:03 +0100 (lun, 08 mar 2010) | 4 lines

  Backport the Popen.poll() protection from subprocess to multiprocessing. See #1731717.
  It should fix transient failures on test_multiprocessing.
........
  r78787 | florent.xicluna | 2010-03-08 08:21:16 +0100 (lun, 08 mar 2010) | 2 lines

  Don't fail on a debug() statement, if the worker PID is (still) None.
........
  r78790 | florent.xicluna | 2010-03-08 12:01:39 +0100 (lun, 08 mar 2010) | 2 lines

  On finalize, don't try to join not started process.
........
2010-03-08 13:32:17 +00:00
Jesse Noller 1f0b658638 Merged revisions 77794 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r77794 | jesse.noller | 2010-01-26 22:05:57 -0500 (Tue, 26 Jan 2010) | 1 line

  Issue #6963: Added maxtasksperchild argument to multiprocessing.Pool
........
2010-01-27 03:36:01 +00:00
Alexandre Vassalotti e52e3786c6 Merged revisions 73995,74002,74005,74007-74008,74011,74019-74023 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r73995 | vinay.sajip | 2009-07-13 07:21:05 -0400 (Mon, 13 Jul 2009) | 1 line

  Issue #6314: logging: Extra checks on the "level" argument in more places.
........
  r74002 | marc-andre.lemburg | 2009-07-13 16:23:49 -0400 (Mon, 13 Jul 2009) | 6 lines

  Use a new global DEV_NULL instead of hard-coding /dev/null into the system
  command helper functions.

  See #6479 for some motivation.
........
  r74005 | marc-andre.lemburg | 2009-07-13 17:28:33 -0400 (Mon, 13 Jul 2009) | 6 lines

  Use a different VER command output parser to address the localization
  issues mentioned in #3410.

  Prepare for Windows 7 (still commented out).
........
  r74007 | michael.foord | 2009-07-14 13:58:12 -0400 (Tue, 14 Jul 2009) | 1 line

  Move TestRunner initialisation into unittest.TestProgram.runTests. Fixes issue 6418.
........
  r74008 | benjamin.peterson | 2009-07-14 20:46:42 -0400 (Tue, 14 Jul 2009) | 1 line

  update year
........
  r74011 | ezio.melotti | 2009-07-15 13:07:04 -0400 (Wed, 15 Jul 2009) | 1 line

  methods' names pep8ification
........
  r74019 | amaury.forgeotdarc | 2009-07-15 17:29:27 -0400 (Wed, 15 Jul 2009) | 2 lines

  #6076 Add a title to the IDLE Preferences window.
........
  r74020 | georg.brandl | 2009-07-16 03:18:07 -0400 (Thu, 16 Jul 2009) | 1 line

  #5910: fix kqueue for calls with more than one event.
........
  r74021 | georg.brandl | 2009-07-16 03:33:04 -0400 (Thu, 16 Jul 2009) | 1 line

  #6486: start with built in functions rather than "built in objects".
........
  r74022 | georg.brandl | 2009-07-16 03:38:35 -0400 (Thu, 16 Jul 2009) | 1 line

  #6481: fix typo in os.system() replacement.
........
  r74023 | jesse.noller | 2009-07-16 10:23:04 -0400 (Thu, 16 Jul 2009) | 1 line

  Issue 6433: multiprocessing.pool.map hangs on empty list
........
2009-07-17 09:18:18 +00:00