Commit Graph

121 Commits

Author SHA1 Message Date
Victor Stinner b4c52966c8 bpo-26762: test_multiprocessing close more queues (#2855)
* Close explicitly queues to make sure that we don't leave dangling
  threads
* test_queue_in_process(): remove unused queue
* test_access() joins also the process to fix a random warning
2017-07-25 02:40:55 +02:00
Victor Stinner ffb49408f0 test_multiprocessing detects dangling per test case (#2841)
bpo-26762: test_multiprocessing now detects dangling processes and
threads per test case classes:

* setUpClass()/tearDownClass() of mixin classes now check if
  multiprocessing.process._dangling or threading._dangling was
  modified to detect "dangling" processses and threads.
* ManagerMixin.tearDownClass() now also emits a warning if it still
  has more than one active child process after 5 seconds.
* tearDownModule() now checks for dangling processes and threads
  before sleep 500 ms. And it now only sleeps if there is a least one
  dangling process or thread.
2017-07-25 01:55:54 +02:00
Victor Stinner d7e64d9934 test_multiprocessing: Fix dangling process/thread (#2850)
bpo-26762: Fix more dangling processes and threads in
test_multiprocessing:

* Queue: call close() followed by join_thread()
* Process: call join() or self.addCleanup(p.join)
2017-07-25 00:33:56 +02:00
Victor Stinner 06634950c5 bpo-26762: Avoid daemon process in _test_multiprocessing (#2842)
test_level() of _test_multiprocessing._TestLogging now uses regular
processes rather than daemon processes to prevent zombi processes
(to not "leak" processes).
2017-07-24 13:02:20 +02:00
Antoine Pitrou 896145d9d2 bpo-26732: fix too many fds in processes started with the "forkserver" method (#2813)
* bpo-26732: fix too many fds in processes started with the "forkserver" method

A child process would inherit as many fds as the number of still-running children.

* Add blurb and test comment
2017-07-22 13:22:54 +02:00
Antoine Pitrou ff92ff5366 Fix test failure without ctypes (#2802) 2017-07-21 13:24:05 +02:00
Gareth Rees 3913bad495 bpo-19896: Add typecodes 'q' and 'Q' to multiprocessing.sharedctypes (#2741)
* bpo-19896: Add typcodes 'q' and 'Q' to multiprocessing.sharedctypes. Patch by Antony Lee.

* Add NEWS entry.

* Slightly tweak NEWS entry

Make it clear this is more of a fix rather than a new feature.
2017-07-21 12:35:33 +02:00
Vitor Pereira ba75af7130 bpo-30794: added kill() method to multiprocessing.Process (#2528)
* bpo-30794: added kill() method to multiprocessing.Process

* Added entries to documentation and NEWS

* Refactored test_terminate and test_kill

* Fix SIGTERM and SIGKILL being used on Windows for the tests

* Added "versionadded" marker to the documentation

* Fix trailing whitespace in doc
2017-07-18 17:34:23 +02:00
Antoine Pitrou 48350412b7 bpo-29293: multiprocessing.Condition.notify() lacks parameter `n` (#2480)
* bpo-29293: multiprocessing.Condition.notify() lacks parameter `n`

* Add NEWS blurb
2017-07-04 08:59:22 +02:00
Antoine Pitrou 79d37ae979 Clear potential ref cycle between Process and Process target (#2470)
* Clear potential ref cycle between Process and Process target

Besides Process.join() not being called, this was an indirect cause of bpo-30775.
The threading module already does this.

* Add issue reference
2017-06-28 12:29:08 +02:00
Antoine Pitrou a79f8faccf bpo-30775: Fix refleaks in test_multiprocessing (#2467)
Forgetting to call Process.join() can keep some resources alive.
2017-06-28 11:21:52 +02:00
Antoine Pitrou 13e96cc596 Fix bpo-30596: Add close() method to multiprocessing.Process (#2010)
* Fix bpo-30596: Add close() method to multiprocessing.Process

* Raise ValueError if close() is called before the Process is finished running

* Add docs

* Add NEWS blurb
2017-06-24 19:22:23 +02:00
Victor Stinner 8f6eeaf21c bpo-30595: Increase test_queue_feeder_donot_stop_onexc() timeout (#2148)
_test_multiprocessing.test_queue_feeder_donot_stop_onexc() now uses a
timeout of 1 second on Queue.get(), instead of 0.1 second, for slow
buildbots.
2017-06-13 23:48:47 +02:00
Antoine Pitrou 1eb6c0074d bpo-24484: Avoid race condition in multiprocessing cleanup (#2159)
* bpo-24484: Avoid race condition in multiprocessing cleanup

The finalizer registry can be mutated while inspected by multiprocessing
at process exit.

* Use test.support.start_threads()

* Add Misc/NEWS
2017-06-13 17:10:39 +02:00
Antoine Pitrou dfd5f34634 Fix bpo-30589: improve Process.exitcode with forkserver (#1989)
* Fix bpo-30589: improve Process.exitcode with forkserver

When the child is killed, Process.exitcode should return -signum, not 255.

* Add Misc/NEWS
2017-06-12 15:28:19 +02:00
grzgrzgrz3 bc50f03db4 bpo-30414: multiprocessing.Queue._feed do not break from main loop on exc (#1683)
* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc

Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.

* bpo-30414: multiprocesing.Queue._feed do not break from main loop on exc

Queue background running thread was not handling exceptions correctly.
Any exception occurred inside thread (putting unpickable object) cause
feeder to finish running. After that every message put into queue is
silently ignored.
2017-05-25 16:22:57 +02:00
Xiang Zhang 6f75bc003a bpo-30301: Fix AttributeError when using SimpleQueue.empty() (#1601)
Under  *spawn* and *forkserver* start methods, SimpleQueue.empty() could
raise AttributeError due to not setting _poll in __setstate__.
2017-05-17 21:04:00 +08:00
Antoine Pitrou 685cdb9acc Relax test timing (bpo-29861) to avoid sporadic failures (#1120) 2017-04-14 13:10:00 +02:00
Xiang Zhang 794623bdb2 bpo-28699: fix abnormal behaviour of pools in multiprocessing.pool (GH-693)
an exception raised at the very first of an iterable would cause pools behave abnormally
(swallow the exception or hang)
2017-03-29 11:58:54 +08:00
Antoine Pitrou 8988945cdc bpo-29861: release references to multiprocessing Pool tasks (#743)
* bpo-29861: release references to multiprocessing Pool tasks

Release references to tasks, their arguments and their results as soon
as they are finished, instead of keeping them alive until another task
arrives.

* Comments in test
2017-03-24 13:52:11 +01:00
Antoine Pitrou ebb39bcc04 Issue #28779: multiprocessing.set_forkserver_preload() would crash the forkserver process if a preloaded module instantiated some multiprocessing objects such as locks. 2016-12-10 17:16:17 +01:00
Antoine Pitrou cd2a201e5b Issue #28779: multiprocessing.set_forkserver_preload() would crash the forkserver process if a preloaded module instantiated some multiprocessing objects such as locks. 2016-12-10 17:13:16 +01:00
Martin Panter 0be894b2f6 Issue #27895: Spelling fixes (Contributed by Ville Skyttä). 2016-09-07 12:03:06 +00:00
R David Murray 44b548dda8 #27364: fix "incorrect" uses of escape character in the stdlib.
And most of the tools.

Patch by Emanual Barry, reviewed by me, Serhiy Storchaka, and
Martin Panter.
2016-09-08 13:59:53 -04:00
Davin Potts 86a7668426 Fixes issue #6766: Updated multiprocessing Proxy Objects to support nesting 2016-09-07 18:48:01 -05:00
Raymond Hettinger 15f44ab043 Issue #27895: Spelling fixes (Contributed by Ville Skyttä). 2016-08-30 10:47:49 -07:00
Victor Stinner a6d865c128 Issue #25654:
* multiprocessing: open file with closefd=False to avoid ResourceWarning
* _test_multiprocessing: open file with O_EXCL to detect bugs in tests (if a
  previous test forgot to remove TESTFN)
* test_sys_exit(): remove TESTFN after each loop iteration

Initial patch written by Serhiy Storchaka.
2016-03-25 09:29:50 +01:00
Antoine Pitrou 58f2bd86fb Issue #26523: The multiprocessing thread pool (multiprocessing.dummy.Pool) was untested. 2016-03-15 10:52:51 +01:00
Antoine Pitrou 62b6a0d70c Issue #26523: The multiprocessing thread pool (multiprocessing.dummy.Pool) was untested. 2016-03-15 10:48:28 +01:00
Charles-François Natali 78f55ffc63 Issue #23992: multiprocessing: make MapResult not fail-fast upon exception. 2016-02-10 22:58:18 +00:00
Berker Peksag 076dbd0560 Fix remaining tests and remove an unused import. 2015-05-06 07:01:52 +03:00
Serhiy Storchaka 84de1bf281 Issue #23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
Patch by Davin Potts.
2015-04-23 11:36:40 +03:00
Serhiy Storchaka 71f73ca7a9 Issue #23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
Patch by Davin Potts.
2015-04-23 11:35:59 +03:00
Serhiy Storchaka 63623ac252 Issue #23051: multiprocessing.Pool methods imap() and imap_unordered() now
handle exceptions raised by an iterator.  Patch by Alon Diamant and Davin
Potts.
2015-03-13 08:30:33 +02:00
Serhiy Storchaka 79fbeee237 Issue #23051: multiprocessing.Pool methods imap() and imap_unordered() now
handle exceptions raised by an iterator.  Patch by Alon Diamant and Davin
Potts.
2015-03-13 08:25:26 +02:00
Serhiy Storchaka eef20de744 Issue #22853: Added regression test for using multiprocessing.Queue at import
time.  Patch by Davin Potts.
2015-03-06 23:33:51 +02:00
Serhiy Storchaka f8904e99c7 Issue #22853: Added regression test for using multiprocessing.Queue at import
time.  Patch by Davin Potts.
2015-03-06 23:32:54 +02:00
Serhiy Storchaka 2eab85b7a2 Issue #21849: Ported from 2.7 tests for non-ASCII data. 2015-02-13 15:19:08 +02:00
Serhiy Storchaka a25c542c47 Issue #21849: Ported from 2.7 tests for non-ASCII data. 2015-02-13 15:13:33 +02:00
Victor Stinner f92766c3b2 (Merge 3.4) test_multiprocessing: tolerate a delta of 30 ms because of bad
clock resolution on Windows
2015-02-05 14:25:53 +01:00
Victor Stinner aad7b2e4f5 test_multiprocessing: tolerate a delta of 30 ms because of bad clock resolution
on Windows
2015-02-05 14:25:05 +01:00
Charles-François Natali 6e20460dc6 Issue #21566: Make use of socket.listen() default backlog. 2014-07-23 19:28:13 +01:00
Richard Oudkerk 80a5be1d84 Issue #20980: Stop wrapping exception when using ThreadPool. 2014-03-23 12:30:54 +00:00
Richard Oudkerk 42a526c60f Issue #20114: Double timeout in test_semaphore_tracker() to reduce
chance of sporadic failures.
2014-02-21 22:29:58 +00:00
Serhiy Storchaka ab0ac27d24 Issue #20315: Removed support for backward compatibility with early 2.x versions.
Removed backward compatibility alias curses.window.nooutrefresh which should
be removed in 2.3.
2014-01-20 21:35:06 +02:00
Serhiy Storchaka 8f8ec92de8 Issue #19936: Added executable bits or shebang lines to Python scripts which
requires them.  Disable executable bits and shebang lines in test and
benchmark files in order to prevent using a random system python, and in
source files of modules which don't provide command line interface.  Fixed
shebang lines in the unittestgui and checkpip scripts.
2014-01-16 17:33:23 +02:00
Zachary Ware 101d9e7250 Issue 19572: More silently skipped tests explicitly skipped. 2013-12-08 00:44:27 -06:00
Richard Oudkerk 35e4ad71ba Merge. 2013-11-17 17:47:00 +00:00
Richard Oudkerk 69cce482cb Merge. 2013-11-17 17:30:54 +00:00
Richard Oudkerk af630c5885 Merge. 2013-11-02 16:47:08 +00:00
Richard Oudkerk 33aaa73cad Merge. 2013-10-28 23:23:04 +00:00
Richard Oudkerk 8acc3c574a Remove test_sigterm(). 2013-10-18 14:42:56 +01:00
Richard Oudkerk cc5c728513 Fix signal handler in test. 2013-10-17 15:22:10 +01:00
Richard Oudkerk 1e2f67c05f Try doing a raw test of os.fork()/os.kill(). 2013-10-17 14:24:06 +01:00
Richard Oudkerk 26f92680da Stop trying to use strace, but add a sleep before terminate(). 2013-10-17 13:56:18 +01:00
Richard Oudkerk e4508eebb4 Try strace instead of gdb to see what wedged child is doing. 2013-10-17 12:10:45 +01:00
Richard Oudkerk d44500affe Try to print a backtrace of wedged child process in test. 2013-10-17 10:38:37 +01:00
Richard Oudkerk b1694cf588 Issue #18999: Make multiprocessing use context objects.
This allows different parts of a program to use different methods for
starting processes without interfering with each other.
2013-10-16 16:41:56 +01:00
Richard Oudkerk bc2bfa6b68 Merge 2013-10-15 16:49:59 +01:00
Richard Oudkerk 7b4a36194a Merge. 2013-10-13 00:52:21 +01:00
Richard Oudkerk 085ab4a553 Merge. 2013-09-29 17:33:04 +01:00
Benjamin Peterson 70157e8f02 merge 3.3 (#19112) 2013-09-29 11:11:10 -04:00
Charles-François Natali 6703bb4984 Issue #18934: Relax test_multiprocessing.test_invalid_handles a bit: we just
want to check that Connection.poll() doesn't crash.
2013-09-06 21:12:22 +02:00
Richard Oudkerk e0d25ce1fe Issue #8713: Print dangling processes/threads, if any. 2013-08-29 14:37:47 +01:00
Richard Oudkerk 968d05594e Issue #8713: Test should not print message about start method. 2013-08-29 12:41:19 +01:00
Richard Oudkerk 6a53af893a Issue #18786: Don't reinstall old SIGUSR1 handler prematurely. 2013-08-28 13:50:19 +01:00
Richard Oudkerk 67e5198dbf Make test for semaphore tracker capture and check the debug output. 2013-08-22 23:37:23 +01:00
Antoine Pitrou f6fbf56071 Issue #18792: Use "127.0.0.1" or "::1" instead of "localhost" as much as possible, since "localhost" goes through a DNS lookup under recent Windows versions. 2013-08-22 00:39:46 +02:00
Richard Oudkerk a01fb39877 Issue #18762: Print debug info on failure to create new forkserver process.
Also modify test code to hopefully avoid deadlock on failure.
2013-08-21 19:45:19 +01:00
Richard Oudkerk 84ed9a68bd Issue #8713: Support alternative start methods in multiprocessing on Unix.
See http://hg.python.org/sandbox/sbt#spawn
2013-08-14 15:35:41 +01:00
Victor Stinner 8a20851010 Issue #25654:
* multiprocessing: open file with closefd=False to avoid ResourceWarning
* _test_multiprocessing: open file with O_EXCL to detect bugs in tests (if a
  previous test forgot to remove TESTFN)
* test_sys_exit(): remove TESTFN after each loop iteration

Initial patch written by Serhiy Storchaka.
2016-03-25 09:29:50 +01:00