Commit Graph

91 Commits

Author SHA1 Message Date
Christian Heimes dde91b1953
bpo-1635741: Fix NULL ptr deref in multiprocessing (GH-22880)
Commit 1d541c25c8 introduced a NULL
pointer dereference in error path.

Signed-off-by: Christian Heimes <christian@python.org>
2020-10-22 03:20:36 -07:00
Mohamed Koubaa 1d541c25c8
bpo-1635741: Port multiprocessing ext to multiphase init (GH-21378)
Port the _multiprocessing extension module to multiphase initialization (PEP 489).
2020-08-11 12:32:35 +02:00
Zackery Spytz 545b54d2ab
bpo-20175: Convert Modules/_multiprocessing to the Argument Clinic (GH-14245) 2020-07-12 19:11:11 +03:00
Serhiy Storchaka 578c3955e0
bpo-37999: No longer use __int__ in implicit integer conversions. (GH-15636)
Only __index__ should be used to make integer conversions lossless.
2020-05-26 18:43:38 +03:00
Victor Stinner 4a21e57fe5
bpo-40268: Remove unused structmember.h includes (GH-19530)
If only offsetof() is needed: include stddef.h instead.

When structmember.h is used, add a comment explaining that
PyMemberDef is used.
2020-04-15 02:35:41 +02:00
Victor Stinner 7dc140126e
bpo-39511: Fix multiprocessing semlock_acquire() (GH-18298)
The Python C API must not be used when the GIL is released: only
access Py_None when the GIL is hold.
2020-02-01 01:25:59 +01:00
Sergey Fedoseev 32f825393e Remove some unused defines in multiprocessing.h. (GH-15661)
It looks like they are unused since
87cf220972.
2019-09-07 10:44:36 +03:00
Rémi Lapeyre 4901fe274b bpo-37034: Display argument name on errors with keyword arguments with Argument Clinic. (GH-13593) 2019-08-29 17:49:08 +03:00
Jeroen Demeyer 530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00
Serhiy Storchaka 3191391515
bpo-36127: Argument Clinic: inline parsing code for keyword parameters. (GH-12058) 2019-03-14 10:32:22 +02:00
Serhiy Storchaka d53fe5f407
bpo-36254: Fix invalid uses of %d in format strings in C. (GH-12264) 2019-03-13 22:59:55 +02:00
Davin Potts e895de3e7f
bpo-35813: Tests and docs for shared_memory (#11816)
* Added tests for shared_memory submodule.

* Added tests for ShareableList.

* Fix bug in allocationn size during creation of empty ShareableList illuminated by existing test run on Linux.

* Initial set of docs for shared_memory module.

* Added docs for ShareableList, added doctree entry for shared_memory submodule, name refactoring for greater clarity.

* Added examples to SharedMemoryManager docs, for ease of documentation switched away from exclusively registered functions to some explicit methods on SharedMemoryManager.

* Wording tweaks to docs.

* Fix test failures on Windows.

* Added tests around SharedMemoryManager.

* Documentation tweaks.

* Fix inappropriate test on Windows.

* Further documentation tweaks.

* Fix bare exception.

* Removed __copyright__.

* Fixed typo in doc, removed comment.

* Updated SharedMemoryManager preliminary tests to reflect change of not supporting all registered functions on SyncManager.

* Added Sphinx doctest run controls.

* CloseHandle should be in a finally block in case MapViewOfFile fails.

* Missed opportunity to use with statement.

* Switch to self.addCleanup to spare long try/finally blocks and save one indentation, change to use decorator to skip test instead.

* Simplify the posixshmem extension module.

Provide shm_open() and shm_unlink() functions.  Move other
functionality into the shared_memory.py module.

* Added to doc around size parameter of SharedMemory.

* Changed PosixSharedMemory.size to use os.fstat.

* Change SharedMemory.buf to a read-only property as well as NamedSharedMemory.size.

* Marked as provisional per PEP411 in docstring.

* Changed SharedMemoryTracker to be private.

* Removed registered Proxy Objects from SharedMemoryManager.

* Removed shareable_wrap().

* Removed shareable_wrap() and dangling references to it.

* For consistency added __reduce__ to key classes.

* Fix for potential race condition on Windows for O_CREX.

* Remove unused imports.

* Update access to kernel32 on Windows per feedback from eryksun.

* Moved kernel32 calls to _winapi.

* Removed ShareableList.copy as redundant.

* Changes to _winapi use from eryksun feedback.

* Adopt simpler SharedMemory API, collapsing PosixSharedMemory and WindowsNamedSharedMemory into one.

* Fix missing docstring on class, add test for ignoring size when attaching.

* Moved SharedMemoryManager to managers module, tweak to fragile test.

* Tweak to exception in OpenFileMapping suggested by eryksun.

* Mark a few dangling bits as private as suggested by Giampaolo.
2019-02-23 22:08:16 -06:00
Davin Potts e5ef45b8f5
bpo-35813: Added shared_memory submodule of multiprocessing. (#11664)
Added shared_memory submodule to multiprocessing in time for first alpha with cross-platform tests soon to follow.
2019-02-01 22:52:23 -06:00
Zackery Spytz 4c49da0cb7 bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015)
Set MemoryError when appropriate, add missing failure checks,
and fix some potential leaks.
2018-12-07 12:11:30 +02:00
Serhiy Storchaka 62be74290a
bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749)
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
2018-11-27 13:27:31 +02:00
Siddhesh Poyarekar 55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
Victor Stinner 828ca59208 bpo-31653: Remove deadcode in semlock_acquire() (#4091)
Fix the following Coverity warning:

>>>     CID 1420038:  Control flow issues  (DEADCODE)
>>>     Execution cannot reach this statement: "res = sem_trywait(self->han...".
321                     res = sem_trywait(self->handle);

The deadcode was introduced by the commit
c872d39d32.
2017-10-23 13:57:51 -07:00
Antoine Pitrou c872d39d32 bpo-31653: Don't release the GIL if we can acquire a multiprocessing semaphore immediately (#4078) 2017-10-22 13:10:46 +02:00
Serhiy Storchaka 55fe1ae970 bpo-30022: Get rid of using EnvironmentError and IOError (except test… (#1051) 2017-04-16 10:46:38 +03:00
Serhiy Storchaka aefa7ebf0f bpo-6532: Make the thread id an unsigned integer. (#781)
* bpo-6532: Make the thread id an unsigned integer.

From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".

* Restore a check in thread_get_ident().
2017-03-23 14:48:39 +01:00
Victor Stinner 408bfa6d96 Issue #28152: Fix -Wunreachable-code warning on clang
Replace 0 with (0) to ignore a compiler warning about dead code on
"((int)(SEM_VALUE_MAX) < 0)": SEM_VALUE_MAX is not negative on Linux.
2016-12-05 18:00:42 +01:00
Benjamin Peterson ed4aa83ff7 require a long long data type (closes #27961) 2016-09-05 17:44:18 -07:00
Berker Peksag bd8b443742 Issue #27591: Merge from 3.5 2016-07-23 07:13:41 +03:00
Berker Peksag 1fd497ed91 Issue #27591: Set sigint_event to NULL if _PyOS_IsMainThread() returns false
Patch by Chris Angelico.
2016-07-23 07:13:14 +03:00
Berker Peksag 1a269d09f6 Issue #26924: Fix Windows buildbots
sem_unlink is defined as

    #define SEM_UNLINK(name) 0

under Windows.
2016-05-07 21:13:50 +03:00
Berker Peksag 0ce9cd985b Issue #26924: Do not define _multiprocessing.sem_unlink under Android
Android declares sem_unlink but doesn't implement it.
2016-05-07 20:39:20 +03:00
Richard Oudkerk e0e65817e5 Issue #21704: Fix build error for _multiprocessing when semaphores
are not available.  Patch by Arfrever Frehtes Taifersar Arahesis.
2014-07-28 23:01:02 +01:00
Richard Oudkerk b988ee0632 Fix conversion from Py_ssize_t to int. 2013-09-07 17:40:45 +01:00
Richard Oudkerk 84ed9a68bd Issue #8713: Support alternative start methods in multiprocessing on Unix.
See http://hg.python.org/sandbox/sbt#spawn
2013-08-14 15:35:41 +01:00
Victor Stinner 640c35ce13 Reuse Py_MIN and Py_MAX macros: remove duplicate MIN/MAX macros
multiprocessing.h: remove unused MIN and MAX macros
2013-06-04 23:14:37 +02:00
doko@ubuntu.com 94fd53ba4e - Fix typos in the multiprocessing module. 2013-05-15 18:08:03 +02:00
doko@ubuntu.com 9df891ca41 - Fix typos in the multiprocessing module. 2013-05-15 18:06:56 +02:00
Martin v. Löwis c8c6563097 Replace WaitForMultipleObjects with WaitForMultipleObjectEx,
for better WinRT compatibility.
2013-01-25 14:29:13 +01:00
Martin v. Löwis b26a9b10ea Replace WaitForSingleObject with WaitForSingleObjectEx,
for better WinRT compatibility.
2013-01-25 14:25:48 +01:00
Richard Oudkerk 950fab07aa Issue #9586: Merge. 2013-01-01 17:40:58 +00:00
Richard Oudkerk 9b4599322d Issue #9586: Merge 2013-01-01 17:36:53 +00:00
Richard Oudkerk 9866231eab Issue #9586: Redefine SEM_FAILED on MacOSX to keep compiler happy. 2013-01-01 17:29:44 +00:00
Andrew Svetlov 2606a6f197 Issue #16719: Get rid of WindowsError. Use OSError instead
Patch by Serhiy Storchaka.
2012-12-19 14:33:35 +02:00
Richard Oudkerk 8fb9f4cf7b Get rid of circular import and eliminate unprefixed exported symbols
from _multiprocessing.
2012-10-07 18:08:47 +01:00
Richard Oudkerk 197651b4e3 Issue #10133: Make multiprocessing deallocate buffer if socket read fails.
Patch by Hallvard B Furuseth.
2012-06-11 15:12:12 +01:00
Richard Oudkerk 26cdf1fe5b Make multiprocessing's shared memory use memoryview instead of raw pointer 2012-05-26 22:09:59 +01:00
Richard Oudkerk 7ef909cdd7 Fix for issue 14725 for 3.2 branch 2012-05-05 20:41:23 +01:00
Richard Oudkerk 3e268aac3b Mark multiprocessing files with "Licensed to PSF under a Contributor Agreement"
instead of BSD licence.
2012-04-30 12:13:55 +01:00
Antoine Pitrou 23bba4ca39 Issue #11750: The Windows API functions scattered in the _subprocess and
_multiprocessing.win32 modules now live in a single module "_winapi".
Patch by sbt.
2012-04-18 20:51:15 +02:00
Antoine Pitrou bdb1cf1ca5 Issue #12328: Fix multiprocessing's use of overlapped I/O on Windows.
Also, add a multiprocessing.connection.wait(rlist, timeout=None) function
for polling multiple objects at once.  Patch by sbt.

Complete changelist from sbt's patch:

* Adds a wait(rlist, timeout=None) function for polling multiple
  objects at once.  On Unix this is just a wrapper for
  select(rlist, [], [], timeout=None).

* Removes use of the SentinelReady exception and the sentinels argument
  to certain methods.  concurrent.futures.process has been changed to
  use wait() instead of SentinelReady.

* Fixes bugs concerning PipeConnection.poll() and messages of zero
  length.

* Fixes PipeListener.accept() to call ConnectNamedPipe() with
  overlapped=True.

* Fixes Queue.empty() and SimpleQueue.empty() so that they are
  threadsafe on Windows.

* Now PipeConnection.poll() and wait() will not modify the pipe except
  possibly by consuming a zero length message.  (Previously poll()
  could consume a partial message.)

* All of multiprocesing's pipe related blocking functions/methods are
  now interruptible by SIGINT on Windows.
2012-03-05 19:28:37 +01:00
Stefan Krah a03422f5d3 Issue #14125: Fix multiprocessing refleak on Windows. Patch by sbt. 2012-02-27 13:51:02 +01:00
Charles-François Natali ed4a8fc095 Issue #8184: multiprocessing: On Windows, don't set SO_REUSEADDR on Connection
sockets, and set FILE_FLAG_FIRST_PIPE_INSTANCE on named pipes, to make sure two
listeners can't bind to the same socket/pipe (or any existing socket/pipe).
2012-02-08 21:15:58 +01:00
Amaury Forgeot d'Arc 7e447c8224 Fix compilation warning on Windows 2011-12-17 10:23:14 +01:00
Antoine Pitrou ab0e9f7089 Issue #10350: Read and save errno before calling a function which might overwrite it.
Original patch by Hallvard B Furuseth.
2011-12-16 12:29:37 +01:00
Antoine Pitrou c345ce1a69 Issue #10350: Read and save errno before calling a function which might overwrite it.
Original patch by Hallvard B Furuseth.
2011-12-16 12:28:32 +01:00