Commit Graph

26 Commits

Author SHA1 Message Date
Victor Stinner d0adfb25c5 [3.6] bpo-26762, bpo-31019: Backport multiprocessing fixes from master to 3.6 (#2879)
* bpo-26762: Avoid daemon process in _test_multiprocessing (#2842)

test_level() of _test_multiprocessing._TestLogging now uses regular
processes rather than daemon processes to prevent zombi processes
(to not "leak" processes).
(cherry picked from commit 06634950c5)

* test_multiprocessing: Fix dangling process/thread (#2850)

bpo-26762: Fix more dangling processes and threads in
test_multiprocessing:

* Queue: call close() followed by join_thread()
* Process: call join() or self.addCleanup(p.join)
(cherry picked from commit d7e64d9934)

* test_multiprocessing detects dangling per test case (#2841)

bpo-26762: test_multiprocessing now detects dangling processes and
threads per test case classes:

* setUpClass()/tearDownClass() of mixin classes now check if
  multiprocessing.process._dangling or threading._dangling was
  modified to detect "dangling" processses and threads.
* ManagerMixin.tearDownClass() now also emits a warning if it still
  has more than one active child process after 5 seconds.
* tearDownModule() now checks for dangling processes and threads
  before sleep 500 ms. And it now only sleeps if there is a least one
  dangling process or thread.
(cherry picked from commit ffb49408f0)

* bpo-26762: test_multiprocessing close more queues (#2855)

* Close explicitly queues to make sure that we don't leave dangling
  threads
* test_queue_in_process(): remove unused queue
* test_access() joins also the process to fix a random warning
(cherry picked from commit b4c52966c8)

* bpo-31019: Fix multiprocessing.Process.is_alive() (#2875)

multiprocessing.Process.is_alive() now removes the process from the
_children set if the process completed.

The change prevents leaking "dangling" processes.
(cherry picked from commit 2db64823c2)
2017-07-26 04:48:56 +02:00
Antoine Pitrou 38d6a40898 [3.6] Clear potential ref cycle between Process and Process target (GH-2470) (#2471)
* Clear potential ref cycle between Process and Process target

Besides Process.join() not being called, this was an indirect cause of bpo-30775.
The threading module already does this.

* Add issue reference.
(cherry picked from commit 79d37ae979)
2017-06-28 13:08:20 +02:00
Victor Stinner a6d865c128 Issue #25654:
* multiprocessing: open file with closefd=False to avoid ResourceWarning
* _test_multiprocessing: open file with O_EXCL to detect bugs in tests (if a
  previous test forgot to remove TESTFN)
* test_sys_exit(): remove TESTFN after each loop iteration

Initial patch written by Serhiy Storchaka.
2016-03-25 09:29:50 +01:00
Richard Oudkerk 69cce482cb Merge. 2013-11-17 17:30:54 +00:00
Richard Oudkerk 8731d7b3c6 Fix handling of SystemExit and exit code. Patch by Brodie Rao. 2013-11-17 17:24:11 +00:00
Richard Oudkerk e943697750 Issue #19478: Make choice of semaphore prefix more flexible. 2013-11-02 17:05:07 +00:00
Richard Oudkerk b1694cf588 Issue #18999: Make multiprocessing use context objects.
This allows different parts of a program to use different methods for
starting processes without interfering with each other.
2013-10-16 16:41:56 +01:00
Richard Oudkerk 84ed9a68bd Issue #8713: Support alternative start methods in multiprocessing on Unix.
See http://hg.python.org/sandbox/sbt#spawn
2013-08-14 15:35:41 +01:00
Richard Oudkerk 58ba47f97b Merge fixes for #13854 and #12157. 2012-06-07 20:38:11 +01:00
Richard Oudkerk 29471de459 Issue #13854: Properly handle non-integer, non-string arg to SystemExit
Previously multiprocessing only expected int or str.  It also wrongly
used an exit code of 1 when the argument was a string instead of zero.
2012-06-06 19:04:57 +01:00
Richard Oudkerk 3e268aac3b Mark multiprocessing files with "Licensed to PSF under a Contributor Agreement"
instead of BSD licence.
2012-04-30 12:13:55 +01:00
Antoine Pitrou 2d843d2520 Issue #13812: When a multiprocessing Process child raises an exception, flush stderr after printing the exception traceback. 2012-01-27 10:53:35 +01:00
Antoine Pitrou 84a0fbf6b0 Issue #13812: When a multiprocessing Process child raises an exception, flush stderr after printing the exception traceback. 2012-01-27 10:52:37 +01:00
Antoine Pitrou 428bc6c48f Issue #12573: Add resource checks for dangling Thread and Process objects. 2011-07-15 22:15:38 +02:00
Antoine Pitrou c081c0c6a0 Issue #12573: Add resource checks for dangling Thread and Process objects. 2011-07-15 22:12:24 +02:00
Victor Stinner 3bcc0170bd (Merge 3.2) Issue #12310: finalize the old process after _run_after_forkers()
multiprocessing: Process._bootstrap() keeps a reference to the old process to
delay its finalization until after _run_after_forkers() as been executed. This
change should fix a crash on Mac OS X Tiger when a lock is released after a
fork.

Patch written by Charles-François Nataliv and Antoine Pitrou.
2011-06-17 12:36:26 +02:00
Victor Stinner 0f83b1511c Issue #12310: finalize the old process after _run_after_forkers()
multiprocessing: Process._bootstrap() keeps a reference to the old process to
delay its finalization until after _run_after_forkers() as been executed. This
change should fix a crash on Mac OS X Tiger when a lock is released after a
fork.

Patch written by Charles-François Nataliv and Antoine Pitrou.
2011-06-17 12:31:49 +02:00
Antoine Pitrou 176f07dadf Issue #12040: Expose a new attribute `sentinel` on instances of
:class:`multiprocessing.Process`.  Also, fix Process.join() to not use
polling anymore, when given a timeout.
2011-06-06 19:35:31 +02:00
Antoine Pitrou 0bd4deba38 Issue #6064: Add a `daemon` keyword argument to the threading.Thread
and multiprocessing.Process constructors in order to override the
default behaviour of inheriting the daemonic property from the current
thread/process.
2011-02-25 22:07:43 +00:00
R. David Murray 3fc969a4a2 9162: fix license in multiprocessing files 2010-12-14 01:38:16 +00:00
Florent Xicluna b519d23c16 Missing part from r78654 2010-03-04 16:10:55 +00:00
Alexandre Vassalotti c57a84f41a Merged revisions 73694,73708,73738 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r73694 | jesse.noller | 2009-06-29 14:24:26 -0400 (Mon, 29 Jun 2009) | 1 line

  Issue 5740: multiprocessing.connection.* authkey fixes
........
  r73708 | jesse.noller | 2009-06-30 13:11:52 -0400 (Tue, 30 Jun 2009) | 1 line

  Resolves issues 5155, 5313, 5331 - bad file descriptor error with processes in processes
........
  r73738 | r.david.murray | 2009-06-30 22:49:10 -0400 (Tue, 30 Jun 2009) | 2 lines

  Make punctuation prettier and break up run-on sentence.
........
2009-07-17 12:07:01 +00:00
Amaury Forgeot d'Arc 768008c6e2 For some reason sys.stdin may be None on Windows, and makes test_multiprocessing fail.
Since we are closing the fileno anyway, the best is to skip this part.

Now test_multiprocessing should pass on Windows.
2008-08-20 09:04:46 +00:00
Benjamin Peterson 58ea9fedc8 Merged revisions 65864 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r65864 | jesse.noller | 2008-08-19 14:06:19 -0500 (Tue, 19 Aug 2008) | 2 lines

  issue3352: clean up the multiprocessing API to remove many get_/set_ methods and convert them to properties. Update the docs and the examples included.
........
2008-08-19 19:17:39 +00:00
Benjamin Peterson 1fcfe211f4 Merged revisions 64517,64519 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r64517 | benjamin.peterson | 2008-06-24 22:09:05 -0500 (Tue, 24 Jun 2008) | 1 line

  remove bytes alias in multiprocessing
........
  r64519 | benjamin.peterson | 2008-06-25 07:39:05 -0500 (Wed, 25 Jun 2008) | 1 line

  use byte literals in multiprocessing
........
2008-06-25 12:54:22 +00:00
Benjamin Peterson e711cafab1 Merged revisions 64104,64117 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r64104 | benjamin.peterson | 2008-06-10 21:40:25 -0500 (Tue, 10 Jun 2008) | 2 lines

  add the multiprocessing package to fulfill PEP 371
........
  r64117 | benjamin.peterson | 2008-06-11 07:26:31 -0500 (Wed, 11 Jun 2008) | 2 lines

  fix import of multiprocessing by juggling imports
........
2008-06-11 16:44:04 +00:00