Commit Graph

69 Commits

Author SHA1 Message Date
Victor Stinner 061a8bf77c
gh-91231: Add shutdown_timeout to multiprocessing BaseManager (#32112)
Add an optional keyword 'shutdown_timeout' parameter to the
multiprocessing.BaseManager constructor. Kill the process if
terminate() takes longer than the timeout.

Multiprocessing tests pass test.support.SHORT_TIMEOUT
to BaseManager.shutdown_timeout.
2022-04-19 16:27:00 +02:00
Nikita Sobolev 0cbdd21311
bpo-46565: `del` loop vars that are leaking into module namespaces (GH-30993) 2022-02-03 11:20:08 +02:00
Zackery Spytz d097876111
bpo-38840: Incorrect __all__ in multiprocessing.managers (GH-18034)
This was causing test___all__ to fail on platforms lacking a shared
memory implementation.

Co-Authored-By: Xavier de Gaye <xdegaye@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-08-09 18:44:55 +02:00
Elisha Hollander 62d55a4d11
Remove unnecessary pass statements (GH-27103) 2021-07-13 15:02:30 +02:00
finefoot 85b920498b
bpo-30256: Add manager_owned keyword arg to AutoProxy (GH-16341)
Co-authored-by: Jordan Speicher <jordan@jspeicher.com>
2021-07-01 20:45:02 -07:00
Victor Stinner 7c29ae1f05
bpo-43867: multiprocessing Server catchs SystemExit (GH-25441)
The multiprocessing Server class now explicitly catchs SystemExit and
closes the client connection in this case. It happens when the
Server.serve_client() method reachs the end of file (EOF).
2021-04-16 19:42:34 +02:00
Galden c606624af8
Fix two typos in multiprocessing (GH-19571) 2020-04-18 08:58:29 +02:00
Batuhan Taşkaya 0361556537
bpo-39481: PEP 585 for a variety of modules (GH-19423)
- concurrent.futures
- ctypes
- http.cookies
- multiprocessing
- queue
- tempfile
- unittest.case
- urllib.parse
2020-04-10 07:46:36 -07:00
Pablo Galindo 6012f30bef
bpo-39850: Add support for abstract sockets in multiprocessing (GH-18866) 2020-03-09 13:48:01 +00:00
Pablo Galindo 293dd23477
Remove binding of captured exceptions when not used to reduce the chances of creating cycles (GH-17246)
Capturing exceptions into names can lead to reference cycles though the __traceback__ attribute of the exceptions in some obscure cases that have been reported previously and fixed individually. As these variables are not used anyway, we can remove the binding to reduce the chances of creating reference cycles.

See for example GH-13135
2019-11-19 21:34:03 +00:00
Serhiy Storchaka 142566c028
[3.9] bpo-37116: Use PEP 570 syntax for positional-only parameters. (GH-12620)
Turn deprecation warnings added in 3.8 into TypeError.
2019-06-05 18:22:31 +03:00
Serhiy Storchaka 2085bd0877
bpo-37116: Use PEP 570 syntax for positional-only parameters. (GH-13700) 2019-06-01 11:00:15 +03:00
Pierre Glaser b1dfcad6f0 bpo-36867: Create the resource_tracker before launching SharedMemoryManagers (GH-13276) 2019-05-13 21:15:32 +02:00
Pierre Glaser d0d64ad1f5 bpo-36368: Ignore SIGINT in SharedMemoryManager servers. (GH-12483)
Fix a bug crashing SharedMemoryManager instances in interactive sessions after
a Ctrl-C (KeyboardInterrupt) was sent.
2019-05-10 20:42:35 +02:00
Serhiy Storchaka d53cf99dca
bpo-36542: Allow to overwrite the signature for Python functions. (GH-12705) 2019-05-06 22:40:27 +03:00
Serhiy Storchaka 42a139ed88
bpo-36492: Deprecate passing some arguments as keyword arguments. (GH-12637)
Deprecated passing the following arguments as keyword arguments:

- "func" in functools.partialmethod(), weakref.finalize(),
  profile.Profile.runcall(), cProfile.Profile.runcall(),
  bdb.Bdb.runcall(), trace.Trace.runfunc() and
  curses.wrapper().
- "function" in unittest.addModuleCleanup() and
  unittest.TestCase.addCleanup().
- "fn" in the submit() method of concurrent.futures.ThreadPoolExecutor
  and concurrent.futures.ProcessPoolExecutor.
- "callback" in contextlib.ExitStack.callback(),
  contextlib.AsyncExitStack.callback() and
  contextlib.AsyncExitStack.push_async_callback().
- "c" and "typeid" in the create() method of multiprocessing.managers.Server
  and multiprocessing.managers.SharedMemoryServer.
- "obj" in weakref.finalize().

Also allowed to pass arbitrary keyword arguments (even "self" and "func")
if the above arguments are passed as positional argument.
2019-04-01 09:16:35 +03:00
Davin Potts e895de3e7f
bpo-35813: Tests and docs for shared_memory (#11816)
* Added tests for shared_memory submodule.

* Added tests for ShareableList.

* Fix bug in allocationn size during creation of empty ShareableList illuminated by existing test run on Linux.

* Initial set of docs for shared_memory module.

* Added docs for ShareableList, added doctree entry for shared_memory submodule, name refactoring for greater clarity.

* Added examples to SharedMemoryManager docs, for ease of documentation switched away from exclusively registered functions to some explicit methods on SharedMemoryManager.

* Wording tweaks to docs.

* Fix test failures on Windows.

* Added tests around SharedMemoryManager.

* Documentation tweaks.

* Fix inappropriate test on Windows.

* Further documentation tweaks.

* Fix bare exception.

* Removed __copyright__.

* Fixed typo in doc, removed comment.

* Updated SharedMemoryManager preliminary tests to reflect change of not supporting all registered functions on SyncManager.

* Added Sphinx doctest run controls.

* CloseHandle should be in a finally block in case MapViewOfFile fails.

* Missed opportunity to use with statement.

* Switch to self.addCleanup to spare long try/finally blocks and save one indentation, change to use decorator to skip test instead.

* Simplify the posixshmem extension module.

Provide shm_open() and shm_unlink() functions.  Move other
functionality into the shared_memory.py module.

* Added to doc around size parameter of SharedMemory.

* Changed PosixSharedMemory.size to use os.fstat.

* Change SharedMemory.buf to a read-only property as well as NamedSharedMemory.size.

* Marked as provisional per PEP411 in docstring.

* Changed SharedMemoryTracker to be private.

* Removed registered Proxy Objects from SharedMemoryManager.

* Removed shareable_wrap().

* Removed shareable_wrap() and dangling references to it.

* For consistency added __reduce__ to key classes.

* Fix for potential race condition on Windows for O_CREX.

* Remove unused imports.

* Update access to kernel32 on Windows per feedback from eryksun.

* Moved kernel32 calls to _winapi.

* Removed ShareableList.copy as redundant.

* Changes to _winapi use from eryksun feedback.

* Adopt simpler SharedMemory API, collapsing PosixSharedMemory and WindowsNamedSharedMemory into one.

* Fix missing docstring on class, add test for ignoring size when attaching.

* Moved SharedMemoryManager to managers module, tweak to fragile test.

* Tweak to exception in OpenFileMapping suggested by eryksun.

* Mark a few dangling bits as private as suggested by Giampaolo.
2019-02-23 22:08:16 -06:00
Rémi Lapeyre a31f4cc881 bpo-35918: Remove broken has_key method and add test (#11819) 2019-02-11 16:37:24 -08:00
Serhiy Storchaka e0e5065dae
bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113) 2018-09-17 14:24:01 +03:00
Victor Stinner c2368cbc83
bpo-34054: multiprocessing uses time.monotonic() (GH-8118)
The multiprocessing module now uses the monotonic clock
time.monotonic() instead of the system clock time.time() to implement
timeouts.
2018-07-06 13:51:52 +02:00
Allen W. Smith, Ph.D bd73e72b4a bpo-5001: More-informative multiprocessing error messages (#3079)
* Make error message more informative

Replace assertions in error-reporting code with more-informative version that doesn't cause confusion over where and what the error is.

* Additional clarification + get travis to check

* Change from SystemError to TypeError

As suggested in PR comment by @pitrou, changing from SystemError; TypeError appears appropriate.

* NEWS file installation; ACKS addition (will do my best to justify it by additional work)

* Making current AssertionErrors in multiprocessing more informative

* Blurb added re multiprocessing managers.py, queues.py cleanup

* Further multiprocessing cleanup - went through pool.py

* Fix two asserts in multiprocessing/util.py

* Most asserts in multiprocessing more informative

* Didn't save right version

* Further work on multiprocessing error messages

* Correct typo

* Correct typo v2

* Blasted colon... serves me right for trying to work on two things at once

* Simplify NEWS entry

* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst

* Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst

OK, never mind.

* Corrected (thanks to pitrou) error messages for notify

* Remove extraneous backslash in docstring.
2017-08-30 00:52:18 +02:00
Allen W. Smith, Ph.D 48d9823a0e bpo-5001, bpo-31169: Fix two uninformative asserts in multiprocessing/managers.py (#3078)
* Make error message more informative

Replace assertions in error-reporting code with more-informative version that doesn't cause confusion over where and what the error is.

* Additional clarification + get travis to check

* Change from SystemError to TypeError

As suggested in PR comment by @pitrou, changing from SystemError; TypeError appears appropriate.

* NEWS file installation; ACKS addition (will do my best to justify it by additional work)
2017-08-12 17:37:09 +02:00
Antoine Pitrou 48350412b7 bpo-29293: multiprocessing.Condition.notify() lacks parameter `n` (#2480)
* bpo-29293: multiprocessing.Condition.notify() lacks parameter `n`

* Add NEWS blurb
2017-07-04 08:59:22 +02:00
Serhiy Storchaka bdf6b910f9 bpo-29776: Use decorator syntax for properties. (#585) 2017-03-19 08:40:32 +02:00
Davin Potts 5458647bb8 Issue #28053: Applying refactorings, docs and other cleanup to follow. 2016-09-09 18:03:10 -05:00
Davin Potts 70897ec54c Issue #21201: Improves readability of multiprocessing error message from server to client for certain exceptions 2016-09-08 14:47:23 -05:00
Davin Potts 37156a70b9 Issue #21201: Improves readability of multiprocessing error message from server to client for certain exceptions 2016-09-08 14:40:36 -05:00
Davin Potts 86a7668426 Fixes issue #6766: Updated multiprocessing Proxy Objects to support nesting 2016-09-07 18:48:01 -05:00
Serhiy Storchaka 6a7b3a77b4 Issue #26778: Fixed "a/an/and" typos in code comment and documentation. 2016-04-17 08:32:47 +03:00
Serhiy Storchaka 465e60e654 Issue #22033: Reprs of most Python implemened classes now contain actual
class name instead of hardcoded one.
2014-07-25 23:36:00 +03:00
Charles-François Natali a924fc7abc Issue #21565: multiprocessing: use contex-manager protocol for synchronization
primitives.
2014-05-25 14:12:12 +01:00
Richard Oudkerk 80a5be1d84 Issue #20980: Stop wrapping exception when using ThreadPool. 2014-03-23 12:30:54 +00:00
Richard Oudkerk b1694cf588 Issue #18999: Make multiprocessing use context objects.
This allows different parts of a program to use different methods for
starting processes without interfering with each other.
2013-10-16 16:41:56 +01:00
Richard Oudkerk 84ed9a68bd Issue #8713: Support alternative start methods in multiprocessing on Unix.
See http://hg.python.org/sandbox/sbt#spawn
2013-08-14 15:35:41 +01:00
Richard Oudkerk 1b6348e084 Issue #17261: Ensure multiprocessing's proxies use proper address. 2013-07-02 13:38:58 +01:00
Richard Oudkerk e3e8bcf3e7 Issue #17261: Ensure multiprocessing's proxies use proper address. 2013-07-02 13:37:43 +01:00
Andrew Svetlov f7a17b48d7 Replace IOError with OSError (#16715) 2012-12-25 16:47:37 +02:00
Richard Oudkerk ac38571f00 Issue #15064: Make BaseManager.__enter__() start server if necessary. 2012-06-18 21:29:30 +01:00
Richard Oudkerk 3049f1243e Increase timeout used when waiting for manager to shutdown cleanly
before resorting to terminate()
2012-06-15 20:08:29 +01:00
Richard Oudkerk 3730a17a58 Issue #14059: Implement multiprocessing.Barrier 2012-06-15 18:26:07 +01:00
Richard Oudkerk 73d9a292ae Issue #13841: Make child processes exit using sys.exit() on Windows 2012-06-14 15:30:10 +01:00
Richard Oudkerk f0604fddc3 Issue #3518: Remove references to non-existent BaseManager.from_address()
method
2012-06-11 17:56:08 +01:00
Richard Oudkerk 1074a9294b Remove __getslice__, __setslice__, __delslice__ methods from proxies
Proxy classes in multiprocessing do not need these methods in Python 3.x.
2012-05-29 12:01:45 +01:00
Richard Oudkerk 41eb85b194 Make AcquirerProxy.acquire() support timeout argument 2012-05-06 16:45:02 +01:00
Richard Oudkerk a6becaa9cb Fix dangling warning for test_multiprocessing 2012-05-03 18:29:02 +01:00
Richard Oudkerk 3e268aac3b Mark multiprocessing files with "Licensed to PSF under a Contributor Agreement"
instead of BSD licence.
2012-04-30 12:13:55 +01:00
Charles-François Natali c8ce715a82 Issue #14087: multiprocessing: add Condition.wait_for(). Patch by sbt. 2012-04-17 18:45:57 +02:00
Charles-François Natali 78c1b42fde Issue #13565: Increase multiprocessing's server socket backlog, to avoid
dropped connections in case of simultaneous connection requests.
2011-12-23 19:07:58 +01:00
Charles-François Natali fe8039b3e5 Issue #13565: Increase multiprocessing's server socket backlog, to avoid
dropped connections in case of simultaneous connection requests.
2011-12-23 19:06:48 +01:00
Antoine Pitrou de911b2915 Issue #12708: Add starmap() and starmap_async() methods (similar to itertools.starmap()) to multiprocessing.Pool.
Patch by Hynek Schlawack.
2011-12-21 11:03:24 +01:00