Commit Graph

176 Commits

Author SHA1 Message Date
Géry Ogam 5576ddbbbc
bpo-46720: Add support for path-like objects to multiprocessing.set_executable for Windows (GH-31279)
This bring the API to be on a par with Unix-like systems.
2022-04-22 15:47:09 -07:00
Victor Stinner 061a8bf77c
gh-91231: Add shutdown_timeout to multiprocessing BaseManager (#32112)
Add an optional keyword 'shutdown_timeout' parameter to the
multiprocessing.BaseManager constructor. Kill the process if
terminate() takes longer than the timeout.

Multiprocessing tests pass test.support.SHORT_TIMEOUT
to BaseManager.shutdown_timeout.
2022-04-19 16:27:00 +02:00
Victor Stinner 9204bb72a2
bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632)
Skip tests on ASAN and/or MSAN builds:

* multiprocessing tests
* test___all__
* test_concurrent_futures
* test_decimal
* test_peg_generator
* test_tools

The ASAN job of GitHub Actions no longer excludes these tests.
2022-03-01 15:44:08 +01:00
Charlie Zhao e466faa9df
bpo-45735: Promise the long-time truth that `args=list` works (GH-30982)
For threads, and for multiprocessing, it's always been the case that ``args=list`` works fine when passed to ``Process()`` or ``Thread()``, and such code is common in the wild. But, according to the docs, only a tuple can be used. This brings the docs into synch with reality.

Doc changes by Charlie Zhao.
Co-authored-by: Tim Peters <tim.peters@gmail.com>
2022-02-25 22:17:13 -06:00
Kumar Aditya af6b406885
bpo-25066: Added repr for multiprocessing.Event (GH-29749)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-12-09 13:16:45 +00:00
Nikita Sobolev 746d648d47
bpo-45125: Improves pickling docs and tests for `shared_memory` (GH-28294) 2021-10-01 13:45:59 +03:00
Serhiy Storchaka eb4495e8e2
bpo-45310: Fix parrallel shared memory tests (GH-28661)
Add a PID to names of POSIX shared memory objects to allow
running multiprocessing tests (test_multiprocessing_fork,
test_multiprocessing_spawn, etc) in parallel.
2021-10-01 10:56:32 +03:00
Nikita Sobolev f604cf1c37
bpo-45209: fix `UserWarning: resource_tracker` in test_multiprocessing (GH-28377) 2021-09-21 19:49:42 +02:00
Nikita Sobolev 19871fce3b
bpo-45052: Unskips a failing `test_shared_memory_basics` test (GH-28182) 2021-09-06 17:55:34 +01:00
Nikita Sobolev dd7b816ac8
bpo-45042: Now test classes decorated with `requires_hashdigest` are not skipped (GH-28060)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2021-09-04 23:42:36 +03:00
Serhiy Storchaka 2a8127cafe
bpo-25130: Add calls of gc.collect() in tests to support PyPy (GH-28005) 2021-08-29 14:04:40 +03:00
finefoot 85b920498b
bpo-30256: Add manager_owned keyword arg to AutoProxy (GH-16341)
Co-authored-by: Jordan Speicher <jordan@jspeicher.com>
2021-07-01 20:45:02 -07:00
Inada Naoki 35715d1e72
bpo-43651: PEP 597: Fix EncodingWarning in some tests (GH-25145)
* test_asyncio
* test_bz2
* test_math
* test_cmath
* test_cmd_line
* test_cmd_line_script
* test_compile
* test_contextlib
* test_profile
* ctypes/test/test_find
* test_multiprocessing
* test_configparser
* test_csv
* test_dbm_dumb
* test_decimal
* test_difflib
* os.fdopen() calls io.text_encoding() to emit EncodingWarning for right place.
2021-04-04 09:01:23 +09:00
Vinay Sharma 475a5fbb56
bpo-41344: Raise ValueError when creating shared memory of size 0 (GH-21556) 2020-08-30 20:03:11 +01:00
Victor Stinner fbf43f051e
bpo-41521: Rename blacklist parameter to not_exported (GH-21824)
Rename "blacklist" parameter of test.support.check__all__() to
"not_exported".
2020-08-17 07:20:40 +02:00
Hai Shi 598a951844
bpo-40275: Use new test.support helper submodules in tests (GH-21764) 2020-08-07 17:18:38 +02:00
Vinay Sharma bfd0fbdc13
bpo-38169: Increase code coverage for SharedMemory and ShareableList (GH-16139) 2020-07-19 22:35:52 +09:00
Hai Shi 0c4f0f3b29
bpo-40275: Use new test.support helper submodules in tests (GH-21169) 2020-06-30 15:46:31 +02:00
Serhiy Storchaka 9355868458
bpo-41043: Escape literal part of the path for glob(). (GH-20994) 2020-06-20 11:10:31 +03:00
Victor Stinner ddbeb2f3e0
bpo-38377: Add support.skip_if_broken_multiprocessing_synchronize() (GH-20944)
On Linux, skip tests using multiprocessing if the current user cannot
create a file in /dev/shm/ directory. Add the
skip_if_broken_multiprocessing_synchronize() function to the
test.support module.
2020-06-18 14:53:19 +02:00
Christian Heimes b022e5cffb
bpo-17258: Add requires_hashdigest to multiprocessing tests (GH-20412)
Skip some :mod:`multiprocessing` tests when MD5 hash digest is blocked.

Signed-off-by: Christian Heimes <christian@python.org>
2020-06-04 05:48:17 -07:00
Hai Shi e80697d687
bpo-40275: Adding threading_helper submodule in test.support (GH-20263) 2020-05-28 00:10:27 +02:00
idomic db098bc1f0
bpo-39244: multiprocessing return default start method first on macOS (GH-18625) 2020-05-26 17:54:21 +03:00
Hai Shi caa3ef284a
bpo-40135: Fix multiprocessing test_shared_memory_across_processes() (GH-19892)
Don't define shared memory block's name in test_shared_memory_across_processes():
use SharedMemory(create=True) instead.
2020-05-04 17:05:54 +02:00
Victor Stinner 9adccc1384
bpo-30966: Add multiprocessing.SimpleQueue.close() (GH-19735)
Add a new close() method to multiprocessing.SimpleQueue to explicitly
close the queue.

Automerge-Triggered-By: @pitrou
2020-04-27 09:11:10 -07:00
Serhiy Storchaka 16994912c9
bpo-40275: Avoid importing socket in test.support (GH-19603)
* Move socket related functions from test.support to socket_helper.
* Import socket, nntplib and urllib.error lazily in transient_internet().
* Remove importing multiprocess.
2020-04-25 10:06:29 +03:00
Victor Stinner d663d34685
bpo-39983: Add test.support.print_warning() (GH-19683)
Log "Warning -- ..." test warnings into sys.__stderr__ rather than
sys.stderr, to ensure to display them even if sys.stderr is captured.

test.libregrtest.utils.print_warning() now calls
test.support.print_warning().
2020-04-23 19:03:52 +02:00
Antoine Pitrou eba9f6155d
bpo-40330: Fix utf-8 size check in ShareableList (GH-19606)
The item size must be checked after encoding to bytes, not before.

Automerge-Triggered-By: @pitrou
2020-04-20 11:54:55 -07:00
Victor Stinner 278c1e159c
bpo-40094: Add test.support.wait_process() (GH-19254)
Moreover, the following tests now check the child process exit code:

* test_os.PtyTests
* test_mailbox.test_lock_conflict()
* test_tempfile.test_process_awareness()
* test_uuid.testIssue8621()
* multiprocessing resource tracker tests
2020-03-31 20:08:12 +02:00
Batuhan Taşkaya ac10e0c932
bpo-39360: Ensure all workers exit when finalizing a multiprocessing Pool (GH-19009)
When the pull is not used via the context manager or terminate() is called, there is a system in multiprocessing.util that handles finalization of all pools via an atexit handler (the Finalize) class. This class registers the _terminate_pool handler in the registry of finalizers of the module, and that registry is called on interpreter exit via _exit_function. The problem is that the "happy" path with the context manager or manual call to finalize() does some extra steps that _terminate_pool does not. The step that is not executed when the atexit() handler calls _terminate_pool is pinging the _change_notifier queue to unblock the maintenance threads.

This commit moves the notification to the _terminate_pool function so is called from both code paths.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-03-15 19:45:56 +00:00
Pablo Galindo 6012f30bef
bpo-39850: Add support for abstract sockets in multiprocessing (GH-18866) 2020-03-09 13:48:01 +00:00
Christopher Hunt c2ac4cf040
bpo-35727: Use exit code 0 on sys.exit() in multiprocessing.Process. (GH-11538) 2020-02-21 10:33:04 +01:00
Victor Stinner 9707e8e22d
bpo-38546: multiprocessing tests stop the resource tracker (GH-17641)
Multiprocessing and concurrent.futures tests now stop the resource
tracker process when tests complete.

Add ResourceTracker._stop() method to
multiprocessing.resource_tracker.

Add _cleanup_tests() helper function to multiprocessing.util: share
code between multiprocessing and concurrent.futures tests.
2019-12-17 18:37:26 +01:00
Victor Stinner 7772b1af5e
bpo-38614: Use support timeout constants (GH-17572) 2019-12-11 22:17:04 +01:00
Victor Stinner 0d63bacefd
bpo-38614: Use test.support.SHORT_TIMEOUT constant (GH-17566)
Replace hardcoded timeout constants in tests with SHORT_TIMEOUT of
test.support, so it's easier to ajdust this timeout for all tests at
once.

SHORT_TIMEOUT is 30 seconds by default, but it can be longer
depending on --timeout command line option.

The change makes almost all timeouts longer, except
test_reap_children() of test_support which is made 2x shorter:
SHORT_TIMEOUT should be enough. If this test starts to fail,
LONG_TIMEOUT should be used instead.

Uniformize also "from test import support" import in some test files.
2019-12-11 11:30:03 +01:00
Victor Stinner c98b0199a9
bpo-38614: Use test.support.LONG_TIMEOUT constant (GH-17562)
Replace hardcoded timeout constants in tests with LONG_TIMEOUT of
test.support, so it's easier to ajdust this timeout for all tests at
once.

LONG_TIMEOUT is 5 minutes by default, but it can be longer depending
on --timeout command line option.
2019-12-10 21:12:26 +01:00
Jake Tesler c6b20be85c bpo-38707: Fix for multiprocessing.Process MainThread.native_id (GH-17088)
This PR implements a fix for `multiprocessing.Process` objects; the error occurs when Processes are created using either `fork` or `forkserver` as the `start_method`.

In these instances, the `MainThread` of the newly created `Process` object retains all attributes from its parent's `MainThread` object, including the `native_id` attribute. The resulting behavior is such that the new process' `MainThread` captures an incorrect/outdated `native_id` (the parent's instead of its own). 

This change forces the Process object to update its `native_id` attribute during the bootstrap process.

cc @vstinner





https://bugs.python.org/issue38707



Automerge-Triggered-By: @pitrou
2019-11-19 11:50:12 -08:00
Victor Stinner 24c6258269
bpo-38614: Add timeout constants to test.support (GH-16964)
Add timeout constants to test.support:

* LOOPBACK_TIMEOUT
* INTERNET_TIMEOUT
* SHORT_TIMEOUT
* LONG_TIMEOUT
2019-10-30 12:41:43 +01:00
Victor Stinner b0e1ae5f54
bpo-37123: multiprocessing test_mymanager() accepts SIGTERM (GH-16349)
Multiprocessing test test_mymanager() now also expects -SIGTERM, not
only exitcode 0.

bpo-30356: BaseManager._finalize_manager() sends SIGTERM to the
manager process if it takes longer than 1 second to stop, which
happens on slow buildbots.
2019-09-24 14:19:48 +02:00
Victor Stinner 99799c7220
bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)
Multiprocessing tests: increase test_queue_feeder_donot_stop_onexc()
timeout from 1 to 60 seconds.
2019-09-24 12:47:49 +02:00
Vinay Sharma 2fc1160a80 bpo-38018: Fix test for multiprocessing.shared_memory in BSD systems (GH-15821) 2019-09-10 07:48:24 +01:00
Vinay Sharma d14e39c8d9 bpo-38018: Increase code coverage for multiprocessing.shared_memory (GH-15662) 2019-09-09 11:48:55 -05:00
Jakub Kulík 4737265622 bpo-37558: Shared memory tests are failing due to double slashes (GH-14703)
With the addition of shared memory into Python 3.8, we now have three tests failing on Solaris, namely `test_multiprocessing_fork`, `test_multiprocessing_forkserver` and `test_multiprocessing_spawn`. The reason seems to be incorrect name handling which results in two slashes being prepended.


https://bugs.python.org/issue37558
2019-07-11 11:04:09 -07:00
Victor Stinner 8fbeb14312
bpo-37421: multiprocessing tests now stop ForkServer (GH-14601)
multiprocessing tests now stop the ForkServer instance if it's
running: close the "alive" file descriptor to ask the server to stop
and then remove its UNIX address.
2019-07-05 16:15:39 +02:00
Victor Stinner 039fb49c18
bpo-37421: multiprocessing tests call _run_finalizers() (GH-14527)
multiprocessing tests now call explicitly _run_finalizers() to remove
immediately temporary directories created by
multiprocessing.util.get_temp_dir().
2019-07-02 13:32:32 +02:00
Pierre Glaser e1a63c4f21 bpo-37244: Fix test_multiprocessing.test_resource_tracker() (GH-14288)
Increase robustness of test_resource_tracker(): retry for 60 seconds.
2019-06-26 00:30:17 +02:00
Pierre Glaser 594d9b9f58 bpo-36888, test_multiprocessing: Increase test_parent_process timeout (GH-14286) 2019-06-25 22:44:11 +02:00
Thomas Moreau c09a9f56c0 bpo-36888: Add multiprocessing.parent_process() (GH-13247) 2019-05-20 21:37:05 +02:00
Pierre Glaser cbe72d8426 bpo-36867: _test_multiprocessing: avoid weak sync primitive (GH-13292)
Avoid weak sync primitive in multiprocessing resource_tracker test.
2019-05-17 20:20:07 +02:00
Pierre Glaser b1dfcad6f0 bpo-36867: Create the resource_tracker before launching SharedMemoryManagers (GH-13276) 2019-05-13 21:15:32 +02:00