Issue #22952: improve multiprocessing doc introduction and defer notes until appropriate.

Patch by Davin Potts.
This commit is contained in:
Antoine Pitrou 2015-01-11 15:09:27 +01:00
parent a024f80a71
commit a8efb6b080
1 changed files with 52 additions and 31 deletions

View File

@ -18,41 +18,27 @@ to this, the :mod:`multiprocessing` module allows the programmer to fully
leverage multiple processors on a given machine. It runs on both Unix and
Windows.
.. warning::
The :mod:`multiprocessing` module also introduces APIs which do not have
analogs in the :mod:`threading` module. A prime example of this is the
:class:`Pool` object which offers a convenient means of parallelizing the
execution of a function across multiple input values, distributing the
input data across processes (data parallelism). The following example
demonstrates the common practice of defining such functions in a module so
that child processes can successfully import that module. This basic example
of data parallelism using :class:`Pool`, ::
Some of this package's functionality requires a functioning shared semaphore
implementation on the host operating system. Without one, the
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
import it will result in an :exc:`ImportError`. See
:issue:`3770` for additional information.
from multiprocessing import Pool
.. note::
def f(x):
return x*x
Functionality within this package requires that the ``__main__`` module be
importable by the children. This is covered in :ref:`multiprocessing-programming`
however it is worth pointing out here. This means that some examples, such
as the :class:`multiprocessing.Pool` examples will not work in the
interactive interpreter. For example::
if __name__ == '__main__':
p = Pool(5)
print(p.map(f, [1, 2, 3]))
>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
... return x*x
...
>>> p.map(f, [1,2,3])
Process PoolWorker-1:
Process PoolWorker-2:
Process PoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
will print to standard output ::
(If you try this it will actually output three full tracebacks
interleaved in a semi-random fashion, and then you may have to
stop the master process somehow.)
[1, 4, 9]
The :class:`Process` class
@ -99,7 +85,6 @@ For an explanation of why (on Windows) the ``if __name__ == '__main__'`` part is
necessary, see :ref:`multiprocessing-programming`.
Exchanging objects between processes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -175,6 +160,14 @@ that only one process prints to standard output at a time::
Without using the lock output from the different processes is liable to get all
mixed up.
.. warning::
Some of this package's functionality requires a functioning shared semaphore
implementation on the host operating system. Without one, the
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
import it will result in an :exc:`ImportError`. See
:issue:`3770` for additional information.
Sharing state between processes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -290,6 +283,34 @@ For example::
Note that the methods of a pool should only ever be used by the
process which created it.
.. note::
Functionality within this package requires that the ``__main__`` module be
importable by the children. This is covered in :ref:`multiprocessing-programming`
however it is worth pointing out here. This means that some examples, such
as the :class:`Pool` examples will not work in the interactive interpreter.
For example::
>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
... return x*x
...
>>> p.map(f, [1,2,3])
Process PoolWorker-1:
Process PoolWorker-2:
Process PoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
(If you try this it will actually output three full tracebacks
interleaved in a semi-random fashion, and then you may have to
stop the master process somehow.)
Reference
---------