Commit Graph

22 Commits

Author SHA1 Message Date
Pierre Glaser f22cc69b01 bpo-36867: Make semaphore_tracker track other system resources (GH-13222)
The multiprocessing.resource_tracker replaces the multiprocessing.semaphore_tracker module. Other than semaphores, resource_tracker also tracks shared_memory segments. Patch by Pierre Glaser.
2019-05-10 22:59:08 +02:00
Antoine Pitrou fc6b348b12
bpo-31308: If multiprocessing's forkserver dies, launch it again when necessary (#3246)
* bpo-31308: If multiprocessing's forkserver dies, launch it again when necessary.

* Fix test on Windows

* Add NEWS entry

* Adopt a different approach: ignore SIGINT and SIGTERM, as in semaphore tracker.

* Fix comment

* Make sure the test doesn't muck with process state

* Also test previously-started processes

* Update 2017-08-30-17-59-36.bpo-31308.KbexyC.rst

* Avoid masking SIGTERM in forkserver.  It's not necessary and causes a race condition in test_many_processes.
2017-11-03 13:34:22 +01:00
Allen W. Smith, Ph.D bd73e72b4a bpo-5001: More-informative multiprocessing error messages (#3079)
* Make error message more informative

Replace assertions in error-reporting code with more-informative version that doesn't cause confusion over where and what the error is.

* Additional clarification + get travis to check

* Change from SystemError to TypeError

As suggested in PR comment by @pitrou, changing from SystemError; TypeError appears appropriate.

* NEWS file installation; ACKS addition (will do my best to justify it by additional work)

* Making current AssertionErrors in multiprocessing more informative

* Blurb added re multiprocessing managers.py, queues.py cleanup

* Further multiprocessing cleanup - went through pool.py

* Fix two asserts in multiprocessing/util.py

* Most asserts in multiprocessing more informative

* Didn't save right version

* Further work on multiprocessing error messages

* Correct typo

* Correct typo v2

* Blasted colon... serves me right for trying to work on two things at once

* Simplify NEWS entry

* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst

* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst

OK, never mind.

* Corrected (thanks to pitrou) error messages for notify

* Remove extraneous backslash in docstring.
2017-08-30 00:52:18 +02:00
Antoine Pitrou 896145d9d2 bpo-26732: fix too many fds in processes started with the "forkserver" method (#2813)
* bpo-26732: fix too many fds in processes started with the "forkserver" method

A child process would inherit as many fds as the number of still-running children.

* Add blurb and test comment
2017-07-22 13:22:54 +02:00
Antoine Pitrou 13e96cc596 Fix bpo-30596: Add close() method to multiprocessing.Process (#2010)
* Fix bpo-30596: Add close() method to multiprocessing.Process

* Raise ValueError if close() is called before the Process is finished running

* Add docs

* Add NEWS blurb
2017-06-24 19:22:23 +02:00
Antoine Pitrou 2b5cc5ebaf bpo-30643: Fix race condition in signal wakeup in forkserver (followup to PR #1989) (#2139)
* Fix race condition in signal wakeup in forkserver (followup to PR #1989)

There's an admittedly well-known race condition where ECHILD can arrive
just before the C function epoll_wait() and the latter wouldn't therefore
return EINTR.  The solution is to use set_wakeup_fd(), which was designed
to avoid such race conditions.

* Reset wakeup fd in child
2017-06-13 09:46:06 +02:00
Antoine Pitrou dfd5f34634 Fix bpo-30589: improve Process.exitcode with forkserver (#1989)
* Fix bpo-30589: improve Process.exitcode with forkserver

When the child is killed, Process.exitcode should return -signum, not 255.

* Add Misc/NEWS
2017-06-12 15:28:19 +02:00
Antoine Pitrou 346cbd351e bpo-16500: Allow registering at-fork handlers (#1715)
* bpo-16500: Allow registering at-fork handlers

* Address Serhiy's comments

* Add doc for new C API

* Add doc for new Python-facing function

* Add NEWS entry + doc nit
2017-05-27 17:50:54 +02:00
Jon Dufresne 3972628de3 bpo-30296 Remove unnecessary tuples, lists, sets, and dicts (#1489)
* Replaced list(<generator expression>) with list comprehension
* Replaced dict(<generator expression>) with dict comprehension
* Replaced set(<list literal>) with set literal
* Replaced builtin func(<list comprehension>) with func(<generator
  expression>) when supported (e.g. any(), all(), tuple(), min(), &
  max())
2017-05-18 07:35:54 -07:00
Antoine Pitrou 6dd4d734ed bpo-30185: avoid KeyboardInterrupt tracebacks in forkserver (#1319)
* bpo-30185: avoid KeyboardInterrupt tracebacks in forkserver

* Tweak comment
2017-05-04 16:44:53 +02:00
Davin Potts 5458647bb8 Issue #28053: Applying refactorings, docs and other cleanup to follow. 2016-09-09 18:03:10 -05:00
Victor Stinner a6d865c128 Issue #25654:
* multiprocessing: open file with closefd=False to avoid ResourceWarning
* _test_multiprocessing: open file with O_EXCL to detect bugs in tests (if a
  previous test forgot to remove TESTFN)
* test_sys_exit(): remove TESTFN after each loop iteration

Initial patch written by Serhiy Storchaka.
2016-03-25 09:29:50 +01:00
Charles-François Natali 6e6c59b508 Issue #23285: PEP 475 -- Retry system calls failing with EINTR. 2015-02-07 13:27:50 +00:00
Charles-François Natali 6e20460dc6 Issue #21566: Make use of socket.listen() default backlog. 2014-07-23 19:28:13 +01:00
Richard Oudkerk b1694cf588 Issue #18999: Make multiprocessing use context objects.
This allows different parts of a program to use different methods for
starting processes without interfering with each other.
2013-10-16 16:41:56 +01:00
Charles-François Natali e241ac9283 Issue #18934: multiprocessing: use selectors module. 2013-09-05 20:46:49 +02:00
Richard Oudkerk 0d097b6299 Issue #18865: PEP 446 makes multiprocessing.util.pipe() unnecessary. 2013-08-28 11:25:34 +01:00
Victor Stinner daf455554b Issue #18571: Implementation of the PEP 446: file descriptors and file handles
are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
2013-08-28 00:53:59 +02:00
Richard Oudkerk 7d2d43c0b1 Stop making fork server have copy of semaphore_tracker_fd. 2013-08-22 11:38:57 +01:00
Richard Oudkerk 0718f70131 Issue #18762: Fix EBADF error when using forkserver. 2013-08-22 11:38:55 +01:00
Richard Oudkerk a01fb39877 Issue #18762: Print debug info on failure to create new forkserver process.
Also modify test code to hopefully avoid deadlock on failure.
2013-08-21 19:45:19 +01:00
Richard Oudkerk 84ed9a68bd Issue #8713: Support alternative start methods in multiprocessing on Unix.
See http://hg.python.org/sandbox/sbt#spawn
2013-08-14 15:35:41 +01:00