Commit Graph

53 Commits

Author SHA1 Message Date
Gregory P. Smith f6dd14c653
gh-82616: Add process_group support to subprocess.Popen (#23930)
One more thing that can help prevent people from using `preexec_fn`.

Also adds conditional skips to two tests exposing ASAN flakiness on the Ubuntu 20.04 Address Sanitizer Github CI system. When that build is run on more modern systems the "problem" does not show up. It seems ASAN implementation related.

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2022-05-05 16:22:32 -07:00
Alexander Shadchin f882d33778
Fix missing `f` prefix on f-strings (GH-91910) 2022-04-27 09:30:54 +03:00
Gregory P. Smith cd5726fe67
gh-91401: Add a failsafe way to disable vfork. (#91490)
Just in case there is ever an issue with _posixsubprocess's use of
vfork() due to the complexity of using it properly and potential
directions that Linux platforms where it defaults to on could take, this
adds a failsafe so that users can disable its use entirely by setting
a global flag.

No known reason to disable it exists. But it'd be a shame to encounter
one and not be able to use CPython without patching and rebuilding it.

See the linked issue for some discussion on reasoning.

Also documents the existing way to disable posix_spawn.
2022-04-25 16:19:39 -07:00
Géry Ogam 5576ddbbbc
bpo-46720: Add support for path-like objects to multiprocessing.set_executable for Windows (GH-31279)
This bring the API to be on a par with Unix-like systems.
2022-04-22 15:47:09 -07:00
Inada Naoki 80017752ba
bpo-43651: Fix test_compileall with PEP 597 (GH-25128) 2021-04-02 09:01:57 +09:00
Dong-hee Na e1945307d3
bpo-40221: Update multiprocessing to use _at_fork_reinit (GH-19477) 2020-04-14 22:15:52 +09:00
Pablo Galindo 6012f30bef
bpo-39850: Add support for abstract sockets in multiprocessing (GH-18866) 2020-03-09 13:48:01 +00:00
Victor Stinner 9707e8e22d
bpo-38546: multiprocessing tests stop the resource tracker (GH-17641)
Multiprocessing and concurrent.futures tests now stop the resource
tracker process when tests complete.

Add ResourceTracker._stop() method to
multiprocessing.resource_tracker.

Add _cleanup_tests() helper function to multiprocessing.util: share
code between multiprocessing and concurrent.futures tests.
2019-12-17 18:37:26 +01:00
Gregory P. Smith f3751efb5c
bpo-38417: Add umask support to subprocess (GH-16726)
On POSIX systems, allow the umask to be set in the child process before we exec.
2019-10-12 13:24:56 -07:00
Patrick McLean 2b2ead7438 bpo-36046: Add user and group parameters to subprocess (GH-11950)
* subprocess: Add user, group and extra_groups paremeters to subprocess.Popen

This adds a `user` parameter to the Popen constructor that will call
setreuid() in the child before calling exec(). This allows processes
running as root to safely drop privileges before running the subprocess
without having to use a preexec_fn.

This also adds a `group` parameter that will call setregid() in
the child process before calling exec().

Finally an `extra_groups` parameter was added that will call
setgroups() to set the supplimental groups.
2019-09-12 18:15:44 +01:00
Min ho Kim 39d87b5471 Fix typos mostly in comments, docs and test names (GH-15209) 2019-08-30 16:21:19 -04:00
Victor Stinner 9d40554e0d
bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572)
Fix multiprocessing.util.get_temp_dir() finalizer: clear also the
'tempdir' configuration of the current process, so next call to
get_temp_dir() will create a new temporary directory, rather than
reusing the removed temporary directory.
2019-07-04 12:28:55 +02:00
Thomas Moreau c09a9f56c0 bpo-36888: Add multiprocessing.parent_process() (GH-13247) 2019-05-20 21:37:05 +02:00
Antoine Pitrou e756f66c83
bpo-31804: Fix multiprocessing.Process with broken standard streams (#6079)
In some conditions the standard streams will be None or closed in the child process (for example if using "pythonw" instead of "python" on Windows).  Avoid failing with a non-0 exit code in those conditions.

Report and initial patch by poxthegreat.
2018-03-11 19:21:38 +01:00
Allen W. Smith, Ph.D bd73e72b4a bpo-5001: More-informative multiprocessing error messages (#3079)
* Make error message more informative

Replace assertions in error-reporting code with more-informative version that doesn't cause confusion over where and what the error is.

* Additional clarification + get travis to check

* Change from SystemError to TypeError

As suggested in PR comment by @pitrou, changing from SystemError; TypeError appears appropriate.

* NEWS file installation; ACKS addition (will do my best to justify it by additional work)

* Making current AssertionErrors in multiprocessing more informative

* Blurb added re multiprocessing managers.py, queues.py cleanup

* Further multiprocessing cleanup - went through pool.py

* Fix two asserts in multiprocessing/util.py

* Most asserts in multiprocessing more informative

* Didn't save right version

* Further work on multiprocessing error messages

* Correct typo

* Correct typo v2

* Blasted colon... serves me right for trying to work on two things at once

* Simplify NEWS entry

* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst

* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst

OK, never mind.

* Corrected (thanks to pitrou) error messages for notify

* Remove extraneous backslash in docstring.
2017-08-30 00:52:18 +02:00
Antoine Pitrou 1eb6c0074d bpo-24484: Avoid race condition in multiprocessing cleanup (#2159)
* bpo-24484: Avoid race condition in multiprocessing cleanup

The finalizer registry can be mutated while inspected by multiprocessing
at process exit.

* Use test.support.start_threads()

* Add Misc/NEWS
2017-06-13 17:10:39 +02:00
Serhiy Storchaka 66bffd1663 bpo-30065: Fixed arguments validation in _posixsubprocess.fork_exec(). (#1110) 2017-04-19 21:12:46 +03:00
Victor Stinner a6d865c128 Issue #25654:
* multiprocessing: open file with closefd=False to avoid ResourceWarning
* _test_multiprocessing: open file with O_EXCL to detect bugs in tests (if a
  previous test forgot to remove TESTFN)
* test_sys_exit(): remove TESTFN after each loop iteration

Initial patch written by Serhiy Storchaka.
2016-03-25 09:29:50 +01:00
Serhiy Storchaka 465e60e654 Issue #22033: Reprs of most Python implemened classes now contain actual
class name instead of hardcoded one.
2014-07-25 23:36:00 +03:00
Charles-François Natali a924fc7abc Issue #21565: multiprocessing: use contex-manager protocol for synchronization
primitives.
2014-05-25 14:12:12 +01:00
Victor Stinner 7fa767e517 Issue #20976: pyflakes: Remove unused imports 2014-03-20 09:16:38 +01:00
Victor Stinner 67973c0279 Issue #18865: remove unused import from multiprocessing.util.spawnv_passfds() 2013-08-28 12:21:47 +02:00
Richard Oudkerk 0d097b6299 Issue #18865: PEP 446 makes multiprocessing.util.pipe() unnecessary. 2013-08-28 11:25:34 +01:00
Victor Stinner daf455554b Issue #18571: Implementation of the PEP 446: file descriptors and file handles
are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
2013-08-28 00:53:59 +02:00
Richard Oudkerk 84ed9a68bd Issue #8713: Support alternative start methods in multiprocessing on Unix.
See http://hg.python.org/sandbox/sbt#spawn
2013-08-14 15:35:41 +01:00
Richard Oudkerk 409c31390f Issue #17555: Fix ForkAwareThreadLock so that size of after fork
registry does not grow exponentially with generation of process.
2013-04-17 20:58:00 +01:00
Andrew Svetlov 5b89840d9c Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:26:36 +02:00
Andrew Svetlov 737fb89dd1 Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:14:22 +02:00
Richard Oudkerk e8cd6bb127 Issue #15881: Clarify comment in exit function 2012-09-13 17:27:15 +01:00
Alexander Belopolsky 5bedef3e64 Issue #15881: Fixed 3.2 backport. 2012-09-09 13:31:08 -04:00
Alexander Belopolsky 7f704c11db Fixed whitespace 2012-09-09 13:25:06 -04:00
Alexander Belopolsky 36351564a2 Fixed whitespace 2012-09-09 13:22:45 -04:00
Alexander Belopolsky f36c49d124 Issue #15881: Fixed atexit hook in multiprocessing. 2012-09-09 13:20:58 -04:00
Alexander Belopolsky 59fb38b582 Issue #15881: Fixed atexit hook in multiprocessing. 2012-09-09 13:16:15 -04:00
Richard Oudkerk 73d9a292ae Issue #13841: Make child processes exit using sys.exit() on Windows 2012-06-14 15:30:10 +01:00
Richard Oudkerk ad06444d6c Make Finalize reserve a reference to os.getpid in case called at shutdown 2012-06-04 18:58:59 +01:00
Richard Oudkerk 739ae5692e Issue #14548: Make multiprocessing finalizers check pid before running
This protects from possibilty of gc running just after fork.
2012-05-25 13:54:53 +01:00
Antoine Pitrou fa67aa0d72 Fix whitespace 2012-05-18 18:33:32 +02:00
Antoine Pitrou ebdcd859e5 Move private function _args_from_interpreter_flags() to subprocess.py, so
that it can be imported when threads are disabled.
(followup to issue #12098)
2012-05-18 18:33:07 +02:00
Richard Oudkerk 77c84f2def #12098: Make multiprocessing's child processes inherit sys.flags on Windows
Initial patch by Sergey Mezentsev.
2012-05-18 14:28:02 +01:00
Richard Oudkerk 59d5404bc7 Issue #14753: Make multiprocessing treat negative timeouts as it did in 3.2
In Python 3.2 and earlier, Process.join() and Connection.poll()
treated negative timeouts as zero timeouts.  Earlier versions from
the 3.3 line of development treat them as infinite timeouts.

The patch reverts to the old behaviour.
2012-05-10 16:11:12 +01:00
Richard Oudkerk 3e268aac3b Mark multiprocessing files with "Licensed to PSF under a Contributor Agreement"
instead of BSD licence.
2012-04-30 12:13:55 +01:00
Florent Xicluna 04842a84c3 Remove unused or redundant imports in concurrent.futures and multiprocessing. 2011-11-11 20:05:50 +01:00
Antoine Pitrou 24d659daaf Use InterruptedError instead of checking for EINTR 2011-10-23 23:49:42 +02:00
Antoine Pitrou 71a28a9174 Rebind locally the globals which can be looked up at shutdown
(to avoid the warnings seen on a buildbot)
2011-07-09 01:03:00 +02:00
Antoine Pitrou 34b312e33d Whitespace normalization 2011-06-06 19:36:01 +02:00
Antoine Pitrou 176f07dadf Issue #12040: Expose a new attribute `sentinel` on instances of
:class:`multiprocessing.Process`.  Also, fix Process.join() to not use
polling anymore, when given a timeout.
2011-06-06 19:35:31 +02:00
R. David Murray 3fc969a4a2 9162: fix license in multiprocessing files 2010-12-14 01:38:16 +00:00
Jesse Noller 41faa543b6 merge r68915 to py3k 2009-01-25 03:45:53 +00:00
Jesse Noller 9a0fc97bf4 merge cl r68737 to py3k 2009-01-18 21:12:58 +00:00