Issue #22952: improve multiprocessing doc introduction and defer notes until appropriate.

Patch by Davin Potts.
This commit is contained in:
Antoine Pitrou 2015-01-11 15:06:39 +01:00
commit 2cae11e87e
1 changed files with 52 additions and 30 deletions

View File

@ -16,41 +16,27 @@ to this, the :mod:`multiprocessing` module allows the programmer to fully
leverage multiple processors on a given machine. It runs on both Unix and
Windows.
.. note::
The :mod:`multiprocessing` module also introduces APIs which do not have
analogs in the :mod:`threading` module. A prime example of this is the
:class:`~multiprocessing.pool.Pool` object which offers a convenient means of
parallelizing the execution of a function across multiple input values,
distributing the input data across processes (data parallelism). The following
example demonstrates the common practice of defining such functions in a module
so that child processes can successfully import that module. This basic example
of data parallelism using :class:`~multiprocessing.pool.Pool`, ::
Some of this package's functionality requires a functioning shared semaphore
implementation on the host operating system. Without one, the
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
import it will result in an :exc:`ImportError`. See
:issue:`3770` for additional information.
from multiprocessing import Pool
.. note::
def f(x):
return x*x
Functionality within this package requires that the ``__main__`` module be
importable by the children. This is covered in :ref:`multiprocessing-programming`
however it is worth pointing out here. This means that some examples, such
as the :class:`multiprocessing.pool.Pool` examples will not work in the
interactive interpreter. For example::
if __name__ == '__main__':
with Pool(5) as p:
print(p.map(f, [1, 2, 3]))
>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
... return x*x
...
>>> p.map(f, [1,2,3])
Process PoolWorker-1:
Process PoolWorker-2:
Process PoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
will print to standard output ::
(If you try this it will actually output three full tracebacks
interleaved in a semi-random fashion, and then you may have to
stop the master process somehow.)
[1, 4, 9]
The :class:`Process` class
@ -276,6 +262,14 @@ that only one process prints to standard output at a time::
Without using the lock output from the different processes is liable to get all
mixed up.
.. note::
Some of this package's functionality requires a functioning shared semaphore
implementation on the host operating system. Without one, the
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
import it will result in an :exc:`ImportError`. See
:issue:`3770` for additional information.
Sharing state between processes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -406,6 +400,34 @@ For example::
Note that the methods of a pool should only ever be used by the
process which created it.
.. note::
Functionality within this package requires that the ``__main__`` module be
importable by the children. This is covered in :ref:`multiprocessing-programming`
however it is worth pointing out here. This means that some examples, such
as the :class:`multiprocessing.pool.Pool` examples will not work in the
interactive interpreter. For example::
>>> from multiprocessing import Pool
>>> p = Pool(5)
>>> def f(x):
... return x*x
...
>>> p.map(f, [1,2,3])
Process PoolWorker-1:
Process PoolWorker-2:
Process PoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
AttributeError: 'module' object has no attribute 'f'
(If you try this it will actually output three full tracebacks
interleaved in a semi-random fashion, and then you may have to
stop the master process somehow.)
Reference
---------