Commit Graph

64 Commits

Author SHA1 Message Date
Mark Nemec c4b695f85e bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144)
Executors in concurrent.futures accepted tasks after executor was shutdown by interpreter exit. Tasks were left in PENDING state forever. This fix changes submit to instead raise a RuntimeError.
2018-04-10 19:23:14 +02:00
Thomas Moreau 095ee415ce bpo-33056 FIX leaking fd in concurrent.futures.ProcessPoolExecutor (#6084) 2018-03-12 19:18:41 +02:00
INADA Naoki 4666ec597c
bpo-32596: Make lazy-load portable (GH-5316)
Global variables should not used as import target.
Use temporary variable instead.
2018-01-26 10:53:31 +09:00
INADA Naoki 6690bb9f17
bpo-32596: Lazy import concurrent.futures.process and thread (GH-5241) 2018-01-20 09:54:42 +09:00
Antoine Pitrou ab74504346
bpo-32576: use queue.SimpleQueue in critical places (#5216)
Where a queue may be invoked from a weakref callback, we need
to use the reentrant SimpleQueue.
2018-01-18 10:38:03 +01:00
Thomas Moreau 94459fd7dc bpo-31699 Deadlocks in `concurrent.futures.ProcessPoolExecutor` with pickling error (#3895)
Fix deadlocks in :class:`concurrent.futures.ProcessPoolExecutor` when task arguments or results cause pickling or unpickling errors.
This should make sure that calls to the :class:`ProcessPoolExecutor` API always eventually return.
2018-01-05 11:15:54 +01:00
Antoine Pitrou 63ff4131af
bpo-21423: Add an initializer argument to {Process,Thread}PoolExecutor (#4241)
* bpo-21423: Add an initializer argument to {Process,Thread}PoolExecutor

* Fix docstring
2017-11-04 11:05:49 +01:00
Thomas Moreau e8c368df22 bpo-31540: Allow passing multiprocessing context to ProcessPoolExecutor (#3682) 2017-10-03 11:53:17 +02:00
Łukasz Langa 574562c5dd bpo-31641: Allow arbitrary iterables in `concurrent.futures.as_completed()` (#3830)
This was possible before.  GH-1560 introduced a regression after 3.6.2 got
released where only sequences were accepted now.  This commit addresses this
problem.
2017-09-29 14:33:34 -07:00
Antoine Pitrou 2ef37607b7 Fix a c.f.as_completed() refleak previously introduced in bpo-27144 (#3270) 2017-09-03 15:09:23 +02:00
Victor Stinner b713adf27a bpo-31326: ProcessPoolExecutor waits for the call queue thread (#3265)
* bpo-31326: ProcessPoolExecutor waits for the call queue thread

concurrent.futures.ProcessPoolExecutor.shutdown() now explicitly
closes the call queue. Moreover, shutdown(wait=True) now also join
the call queue thread, to prevent leaking a dangling thread.

* Fix for shutdown() being called twice.
2017-09-02 00:25:11 +02:00
Grzegorz Grzywacz 97e1b1c814 bpo-27144: concurrent.futures as_complete and map iterators do not keep reference to returned object (#1560)
* bpo-27144: concurrent.futures as_complie and map iterators do not keep
reference to returned object

* Some nits.  Improve wordings in docstrings and comments, and avoid relying on
sys.getrefcount() in tests.
2017-09-01 18:54:00 +02:00
Victor Stinner bc61315377 bpo-31249: Fix ref cycle in ThreadPoolExecutor (#3178)
* bpo-31249: Fix ref cycle in ThreadPoolExecutor

concurrent.futures: WorkItem.run() used by ThreadPoolExecutor now
breaks a reference cycle between an exception object and the WorkItem
object. ThreadPoolExecutor.shutdown() now also clears its threads
set.

* shutdown() now only clears threads if wait is true.

* Revert changes on shutdown()
2017-08-22 16:50:42 +02:00
Gregory P. Smith a3d91b43c2 bpo-29212: Fix the ugly repr() ThreadPoolExecutor thread name. (#2315)
bpo-29212: Fix the ugly ThreadPoolExecutor thread name.

Fixes the newly introduced ugly default thread name for concurrent.futures
thread.ThreadPoolExecutor threads.  They'll now resemble the old <=3.5
threading default Thread-x names by being named ThreadPoolExecutor-y_n.
2017-06-21 23:41:13 -07:00
Raymond Hettinger 15f44ab043 Issue #27895: Spelling fixes (Contributed by Ville Skyttä). 2016-08-30 10:47:49 -07:00
Gregory P. Smith 50abe877ee Issue #27664: Add to concurrent.futures.thread.ThreadPoolExecutor()
the ability to specify a thread name prefix.
2016-08-07 10:19:20 -07:00
Martin Panter d2ad5718ad Issue #25523: Further a-to-an corrections new in 3.5 2015-11-02 04:20:33 +00:00
Martin Panter 2eb819f7a8 Issue #25523: Merge "a" to "an" fixes from 3.4 into 3.5 2015-11-02 04:04:57 +00:00
Martin Panter 7462b64911 Issue #25523: Correct "a" article to "an" article
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
2015-11-02 03:37:02 +00:00
Antoine Pitrou 1285c9b782 Issue #21817: When an exception is raised in a task submitted to a ProcessPoolExecutor, the remote traceback is now displayed in the parent process.
Patch by Claudiu Popa.
2015-01-17 20:02:14 +01:00
Antoine Pitrou 4aae276eca Issue #11271: concurrent.futures.Executor.map() now takes a *chunksize*
argument to allow batching of tasks in child processes and improve
performance of ProcessPoolExecutor.  Patch by Dan O'Reilly.
2014-10-04 20:20:10 +02:00
Guido van Rossum cfd4661e78 Closes #21527: Add default number of workers to ThreadPoolExecutor. (Claudiu Popa.) 2014-09-02 10:39:18 -07:00
Serhiy Storchaka 465e60e654 Issue #22033: Reprs of most Python implemened classes now contain actual
class name instead of hardcoded one.
2014-07-25 23:36:00 +03:00
Brian Quinlan 20efceb757 Issue #21362: concurrent.futures does not validate that max_workers is proper 2014-05-17 13:51:10 -07:00
Brian Quinlan 2b754f49a5 Issue #20319: concurrent.futures.wait() can block forever even if Futures have completed 2014-02-01 11:49:04 +11:00
Guido van Rossum e6994ff6e3 Fix issue #20367: concurrent.futures.as_completed() for duplicate arguments.
Patch by Glenn Langford.
2014-01-26 09:57:51 -08:00
Richard Oudkerk 1f2eaa97eb Fix import of SimpleQueue. 2013-10-16 17:06:22 +01:00
Ezio Melotti e0c69161bc #18741: merge with 3.3. 2013-08-17 16:13:22 +03:00
Ezio Melotti b5bc353b88 #18741: fix more typos. Patch by Févry Thibault. 2013-08-17 16:11:40 +03:00
Charles-François Natali 37cfb0a920 Issue #17914: Use os.cpu_count() instead of multiprocessing.cpu_count() where
applicable.
2013-06-28 19:25:45 +02:00
Terry Jan Reedy 16b5c13668 Merge 3.3, issue #17047: remove doubled words found in 2.7 to
3.4 Lib/*, as reported by Serhiy Storchaka and Matthew Barnett.
2013-03-11 17:59:07 -04:00
Terry Jan Reedy 98472b8396 Merge 3.2, issue #17047: remove doubled words found in 2.7 to
3.4 Lib/*, as reported by Serhiy Storchaka and Matthew Barnett.
2013-03-11 17:58:27 -04:00
Terry Jan Reedy c30b7b16ea Issue #17047: remove doubled words found in 2.7 to 3.4 Lib/*,
as reported by Serhiy Storchaka and Matthew Barnett.
2013-03-11 17:57:08 -04:00
Eli Bendersky c9d504fc29 Fix docstring typo in concurrent.futures.Future 2013-01-17 06:36:30 -08:00
Eli Bendersky 6a56b8ab9c Fix docstring typo in concurrent.futures.Future 2013-01-17 06:36:30 -08:00
Andrew Svetlov 6b973747f3 Issue #16284: Prevent keeping unnecessary references to worker functions in concurrent.futures ThreadPoolExecutor. 2012-11-03 15:36:01 +02:00
Mark Dickinson 5ee2404d1e Fix concurrent.futures docstring typo: "Request Q" -> "Result Q". 2012-10-20 13:16:49 +01:00
Philip Jenvey 4993cc0a5b utilize yield from 2012-10-01 12:53:43 -07:00
Brian Quinlan a26ad5a0a1 #15015: Fix accessing an non-existing attribute. 2012-06-11 12:59:07 +10:00
Antoine Pitrou 8b34b53c52 Issue #14406: Fix a race condition when using `concurrent.futures.wait(return_when=ALL_COMPLETED)`.
Patch by Matt Joiner.
2012-03-31 20:25:22 +02:00
Antoine Pitrou f70401e842 Issue #14406: Fix a race condition when using `concurrent.futures.wait(return_when=ALL_COMPLETED)`.
Patch by Matt Joiner.
2012-03-31 20:23:30 +02:00
Antoine Pitrou bdb1cf1ca5 Issue #12328: Fix multiprocessing's use of overlapped I/O on Windows.
Also, add a multiprocessing.connection.wait(rlist, timeout=None) function
for polling multiple objects at once.  Patch by sbt.

Complete changelist from sbt's patch:

* Adds a wait(rlist, timeout=None) function for polling multiple
  objects at once.  On Unix this is just a wrapper for
  select(rlist, [], [], timeout=None).

* Removes use of the SentinelReady exception and the sentinels argument
  to certain methods.  concurrent.futures.process has been changed to
  use wait() instead of SentinelReady.

* Fixes bugs concerning PipeConnection.poll() and messages of zero
  length.

* Fixes PipeListener.accept() to call ConnectNamedPipe() with
  overlapped=True.

* Fixes Queue.empty() and SimpleQueue.empty() so that they are
  threadsafe on Windows.

* Now PipeConnection.poll() and wait() will not modify the pipe except
  possibly by consuming a zero length message.  (Previously poll()
  could consume a partial message.)

* All of multiprocesing's pipe related blocking functions/methods are
  now interruptible by SIGINT on Windows.
2012-03-05 19:28:37 +01:00
Ross Lagerwall 66e2fb68ac Issue #12364: Fix a hang in concurrent.futures.ProcessPoolExecutor. 2012-01-08 08:29:40 +02:00
Ross Lagerwall 69f39a53f6 Merge with 3.2 for #12364. 2012-01-08 08:42:03 +02:00
Florent Xicluna 04842a84c3 Remove unused or redundant imports in concurrent.futures and multiprocessing. 2011-11-11 20:05:50 +01:00
Antoine Pitrou 0b447957c7 Close the call queue in concurrent.futures.ProcessPoolExecutor when
shutdown() is called, without waiting for the garbage collector to kick in.
2011-07-16 23:52:02 +02:00
Antoine Pitrou dc19c24832 Silence spurious "broken pipe" tracebacks when shutting down a ProcessPoolExecutor. 2011-07-16 01:51:58 +02:00
Antoine Pitrou d06a065a44 Fix potential resource leaks in concurrent.futures.ProcessPoolExecutor
by joining all queues and processes when shutdown() is called.
2011-07-16 01:13:34 +02:00
Antoine Pitrou 1c405b3e60 Followup to 51c1f2cedb96 (and issue #12456):
qsize() raises NotImplementedError on OS X, don't use it.
2011-07-03 13:17:06 +02:00
Antoine Pitrou 020436b0d4 Issue #12456: fix a possible hang on shutdown of a concurrent.futures.ProcessPoolExecutor. 2011-07-02 21:20:25 +02:00