mirror of https://github.com/python/cpython
gh-114756: Update FAQ section on removing the GIL (#114957)
Update FAQ section on removing the GIL to reflect recent progress on PEP 703 and PEP 684. Co-authored-by: AN Long <aisk@users.noreply.github.com>
This commit is contained in:
parent
d7334e2c20
commit
0e2ab73dc3
|
@ -405,22 +405,37 @@ lists. When in doubt, use a mutex!
|
|||
Can't we get rid of the Global Interpreter Lock?
|
||||
------------------------------------------------
|
||||
|
||||
.. XXX link to dbeazley's talk about GIL?
|
||||
|
||||
The :term:`global interpreter lock` (GIL) is often seen as a hindrance to Python's
|
||||
deployment on high-end multiprocessor server machines, because a multi-threaded
|
||||
Python program effectively only uses one CPU, due to the insistence that
|
||||
(almost) all Python code can only run while the GIL is held.
|
||||
|
||||
Back in the days of Python 1.5, Greg Stein actually implemented a comprehensive
|
||||
patch set (the "free threading" patches) that removed the GIL and replaced it
|
||||
with fine-grained locking. Adam Olsen recently did a similar experiment
|
||||
in his `python-safethread <https://code.google.com/archive/p/python-safethread>`_
|
||||
project. Unfortunately, both experiments exhibited a sharp drop in single-thread
|
||||
performance (at least 30% slower), due to the amount of fine-grained locking
|
||||
necessary to compensate for the removal of the GIL.
|
||||
With the approval of :pep:`703` work is now underway to remove the GIL from the
|
||||
CPython implementation of Python. Initially it will be implemented as an
|
||||
optional compiler flag when building the interpreter, and so separate
|
||||
builds will be available with and without the GIL. Long-term, the hope is
|
||||
to settle on a single build, once the performance implications of removing the
|
||||
GIL are fully understood. Python 3.13 is likely to be the first release
|
||||
containing this work, although it may not be completely functional in this
|
||||
release.
|
||||
|
||||
This doesn't mean that you can't make good use of Python on multi-CPU machines!
|
||||
The current work to remove the GIL is based on a
|
||||
`fork of Python 3.9 with the GIL removed <https://github.com/colesbury/nogil>`_
|
||||
by Sam Gross.
|
||||
Prior to that,
|
||||
in the days of Python 1.5, Greg Stein actually implemented a comprehensive
|
||||
patch set (the "free threading" patches) that removed the GIL and replaced it
|
||||
with fine-grained locking. Adam Olsen did a similar experiment
|
||||
in his `python-safethread <https://code.google.com/archive/p/python-safethread>`_
|
||||
project. Unfortunately, both of these earlier experiments exhibited a sharp
|
||||
drop in single-thread
|
||||
performance (at least 30% slower), due to the amount of fine-grained locking
|
||||
necessary to compensate for the removal of the GIL. The Python 3.9 fork
|
||||
is the first attempt at removing the GIL with an acceptable performance
|
||||
impact.
|
||||
|
||||
The presence of the GIL in current Python releases
|
||||
doesn't mean that you can't make good use of Python on multi-CPU machines!
|
||||
You just have to be creative with dividing the work up between multiple
|
||||
*processes* rather than multiple *threads*. The
|
||||
:class:`~concurrent.futures.ProcessPoolExecutor` class in the new
|
||||
|
@ -434,22 +449,13 @@ thread of execution is in the C code and allow other threads to get some work
|
|||
done. Some standard library modules such as :mod:`zlib` and :mod:`hashlib`
|
||||
already do this.
|
||||
|
||||
It has been suggested that the GIL should be a per-interpreter-state lock rather
|
||||
than truly global; interpreters then wouldn't be able to share objects.
|
||||
Unfortunately, this isn't likely to happen either. It would be a tremendous
|
||||
amount of work, because many object implementations currently have global state.
|
||||
For example, small integers and short strings are cached; these caches would
|
||||
have to be moved to the interpreter state. Other object types have their own
|
||||
free list; these free lists would have to be moved to the interpreter state.
|
||||
And so on.
|
||||
|
||||
And I doubt that it can even be done in finite time, because the same problem
|
||||
exists for 3rd party extensions. It is likely that 3rd party extensions are
|
||||
being written at a faster rate than you can convert them to store all their
|
||||
global state in the interpreter state.
|
||||
|
||||
And finally, once you have multiple interpreters not sharing any state, what
|
||||
have you gained over running each interpreter in a separate process?
|
||||
An alternative approach to reducing the impact of the GIL is
|
||||
to make the GIL a per-interpreter-state lock rather than truly global.
|
||||
This was :ref:`first implemented in Python 3.12 <whatsnew312-pep684>` and is
|
||||
available in the C API. A Python interface to it is expected in Python 3.13.
|
||||
The main limitation to it at the moment is likely to be 3rd party extension
|
||||
modules, since these must be written with multiple interpreters in mind in
|
||||
order to be usable, so many older extension modules will not be usable.
|
||||
|
||||
|
||||
Input and Output
|
||||
|
|
Loading…
Reference in New Issue