Issue #22952: improve multiprocessing doc introduction and defer notes until appropriate.
Patch by Davin Potts.
This commit is contained in:
parent
a024f80a71
commit
a8efb6b080
|
@ -18,41 +18,27 @@ to this, the :mod:`multiprocessing` module allows the programmer to fully
|
|||
leverage multiple processors on a given machine. It runs on both Unix and
|
||||
Windows.
|
||||
|
||||
.. warning::
|
||||
The :mod:`multiprocessing` module also introduces APIs which do not have
|
||||
analogs in the :mod:`threading` module. A prime example of this is the
|
||||
:class:`Pool` object which offers a convenient means of parallelizing the
|
||||
execution of a function across multiple input values, distributing the
|
||||
input data across processes (data parallelism). The following example
|
||||
demonstrates the common practice of defining such functions in a module so
|
||||
that child processes can successfully import that module. This basic example
|
||||
of data parallelism using :class:`Pool`, ::
|
||||
|
||||
Some of this package's functionality requires a functioning shared semaphore
|
||||
implementation on the host operating system. Without one, the
|
||||
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
|
||||
import it will result in an :exc:`ImportError`. See
|
||||
:issue:`3770` for additional information.
|
||||
from multiprocessing import Pool
|
||||
|
||||
.. note::
|
||||
def f(x):
|
||||
return x*x
|
||||
|
||||
Functionality within this package requires that the ``__main__`` module be
|
||||
importable by the children. This is covered in :ref:`multiprocessing-programming`
|
||||
however it is worth pointing out here. This means that some examples, such
|
||||
as the :class:`multiprocessing.Pool` examples will not work in the
|
||||
interactive interpreter. For example::
|
||||
if __name__ == '__main__':
|
||||
p = Pool(5)
|
||||
print(p.map(f, [1, 2, 3]))
|
||||
|
||||
>>> from multiprocessing import Pool
|
||||
>>> p = Pool(5)
|
||||
>>> def f(x):
|
||||
... return x*x
|
||||
...
|
||||
>>> p.map(f, [1,2,3])
|
||||
Process PoolWorker-1:
|
||||
Process PoolWorker-2:
|
||||
Process PoolWorker-3:
|
||||
Traceback (most recent call last):
|
||||
Traceback (most recent call last):
|
||||
Traceback (most recent call last):
|
||||
AttributeError: 'module' object has no attribute 'f'
|
||||
AttributeError: 'module' object has no attribute 'f'
|
||||
AttributeError: 'module' object has no attribute 'f'
|
||||
will print to standard output ::
|
||||
|
||||
(If you try this it will actually output three full tracebacks
|
||||
interleaved in a semi-random fashion, and then you may have to
|
||||
stop the master process somehow.)
|
||||
[1, 4, 9]
|
||||
|
||||
|
||||
The :class:`Process` class
|
||||
|
@ -99,7 +85,6 @@ For an explanation of why (on Windows) the ``if __name__ == '__main__'`` part is
|
|||
necessary, see :ref:`multiprocessing-programming`.
|
||||
|
||||
|
||||
|
||||
Exchanging objects between processes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
@ -175,6 +160,14 @@ that only one process prints to standard output at a time::
|
|||
Without using the lock output from the different processes is liable to get all
|
||||
mixed up.
|
||||
|
||||
.. warning::
|
||||
|
||||
Some of this package's functionality requires a functioning shared semaphore
|
||||
implementation on the host operating system. Without one, the
|
||||
:mod:`multiprocessing.synchronize` module will be disabled, and attempts to
|
||||
import it will result in an :exc:`ImportError`. See
|
||||
:issue:`3770` for additional information.
|
||||
|
||||
|
||||
Sharing state between processes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
@ -290,6 +283,34 @@ For example::
|
|||
Note that the methods of a pool should only ever be used by the
|
||||
process which created it.
|
||||
|
||||
.. note::
|
||||
|
||||
Functionality within this package requires that the ``__main__`` module be
|
||||
importable by the children. This is covered in :ref:`multiprocessing-programming`
|
||||
however it is worth pointing out here. This means that some examples, such
|
||||
as the :class:`Pool` examples will not work in the interactive interpreter.
|
||||
For example::
|
||||
|
||||
>>> from multiprocessing import Pool
|
||||
>>> p = Pool(5)
|
||||
>>> def f(x):
|
||||
... return x*x
|
||||
...
|
||||
>>> p.map(f, [1,2,3])
|
||||
Process PoolWorker-1:
|
||||
Process PoolWorker-2:
|
||||
Process PoolWorker-3:
|
||||
Traceback (most recent call last):
|
||||
Traceback (most recent call last):
|
||||
Traceback (most recent call last):
|
||||
AttributeError: 'module' object has no attribute 'f'
|
||||
AttributeError: 'module' object has no attribute 'f'
|
||||
AttributeError: 'module' object has no attribute 'f'
|
||||
|
||||
(If you try this it will actually output three full tracebacks
|
||||
interleaved in a semi-random fashion, and then you may have to
|
||||
stop the master process somehow.)
|
||||
|
||||
|
||||
Reference
|
||||
---------
|
||||
|
|
Loading…
Reference in New Issue