mirror of https://github.com/python/cpython
[Bug #580462] Mention changes to GC API
Mention portable strptime() Move C-level sections farther down in the file
This commit is contained in:
parent
a982eb1eb5
commit
ef5d06bd3f
|
@ -17,8 +17,6 @@
|
||||||
%
|
%
|
||||||
% Bug #580462: changes to GC API
|
% Bug #580462: changes to GC API
|
||||||
%
|
%
|
||||||
% Pure Python strptime implementation by Brett Cannon. See SF patch 474274.
|
|
||||||
%
|
|
||||||
|
|
||||||
%\section{Introduction \label{intro}}
|
%\section{Introduction \label{intro}}
|
||||||
|
|
||||||
|
@ -602,88 +600,6 @@ In 2.3, you get this:
|
||||||
\end{itemize}
|
\end{itemize}
|
||||||
|
|
||||||
|
|
||||||
%======================================================================
|
|
||||||
\section{Specialized Object Allocator (pymalloc)\label{section-pymalloc}}
|
|
||||||
|
|
||||||
An experimental feature added to Python 2.1 was a specialized object
|
|
||||||
allocator called pymalloc, written by Vladimir Marangozov. Pymalloc
|
|
||||||
was intended to be faster than the system \cfunction{malloc()} and have
|
|
||||||
less memory overhead for typical allocation patterns of Python
|
|
||||||
programs. The allocator uses C's \cfunction{malloc()} function to get
|
|
||||||
large pools of memory, and then fulfills smaller memory requests from
|
|
||||||
these pools.
|
|
||||||
|
|
||||||
In 2.1 and 2.2, pymalloc was an experimental feature and wasn't
|
|
||||||
enabled by default; you had to explicitly turn it on by providing the
|
|
||||||
\longprogramopt{with-pymalloc} option to the \program{configure}
|
|
||||||
script. In 2.3, pymalloc has had further enhancements and is now
|
|
||||||
enabled by default; you'll have to supply
|
|
||||||
\longprogramopt{without-pymalloc} to disable it.
|
|
||||||
|
|
||||||
This change is transparent to code written in Python; however,
|
|
||||||
pymalloc may expose bugs in C extensions. Authors of C extension
|
|
||||||
modules should test their code with the object allocator enabled,
|
|
||||||
because some incorrect code may cause core dumps at runtime. There
|
|
||||||
are a bunch of memory allocation functions in Python's C API that have
|
|
||||||
previously been just aliases for the C library's \cfunction{malloc()}
|
|
||||||
and \cfunction{free()}, meaning that if you accidentally called
|
|
||||||
mismatched functions, the error wouldn't be noticeable. When the
|
|
||||||
object allocator is enabled, these functions aren't aliases of
|
|
||||||
\cfunction{malloc()} and \cfunction{free()} any more, and calling the
|
|
||||||
wrong function to free memory may get you a core dump. For example,
|
|
||||||
if memory was allocated using \cfunction{PyObject_Malloc()}, it has to
|
|
||||||
be freed using \cfunction{PyObject_Free()}, not \cfunction{free()}. A
|
|
||||||
few modules included with Python fell afoul of this and had to be
|
|
||||||
fixed; doubtless there are more third-party modules that will have the
|
|
||||||
same problem.
|
|
||||||
|
|
||||||
As part of this change, the confusing multiple interfaces for
|
|
||||||
allocating memory have been consolidated down into two API families.
|
|
||||||
Memory allocated with one family must not be manipulated with
|
|
||||||
functions from the other family.
|
|
||||||
|
|
||||||
There is another family of functions specifically for allocating
|
|
||||||
Python \emph{objects} (as opposed to memory).
|
|
||||||
|
|
||||||
\begin{itemize}
|
|
||||||
\item To allocate and free an undistinguished chunk of memory use
|
|
||||||
the ``raw memory'' family: \cfunction{PyMem_Malloc()},
|
|
||||||
\cfunction{PyMem_Realloc()}, and \cfunction{PyMem_Free()}.
|
|
||||||
|
|
||||||
\item The ``object memory'' family is the interface to the pymalloc
|
|
||||||
facility described above and is biased towards a large number of
|
|
||||||
``small'' allocations: \cfunction{PyObject_Malloc},
|
|
||||||
\cfunction{PyObject_Realloc}, and \cfunction{PyObject_Free}.
|
|
||||||
|
|
||||||
\item To allocate and free Python objects, use the ``object'' family
|
|
||||||
\cfunction{PyObject_New()}, \cfunction{PyObject_NewVar()}, and
|
|
||||||
\cfunction{PyObject_Del()}.
|
|
||||||
\end{itemize}
|
|
||||||
|
|
||||||
Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides
|
|
||||||
debugging features to catch memory overwrites and doubled frees in
|
|
||||||
both extension modules and in the interpreter itself. To enable this
|
|
||||||
support, turn on the Python interpreter's debugging code by running
|
|
||||||
\program{configure} with \longprogramopt{with-pydebug}.
|
|
||||||
|
|
||||||
To aid extension writers, a header file \file{Misc/pymemcompat.h} is
|
|
||||||
distributed with the source to Python 2.3 that allows Python
|
|
||||||
extensions to use the 2.3 interfaces to memory allocation and compile
|
|
||||||
against any version of Python since 1.5.2. You would copy the file
|
|
||||||
from Python's source distribution and bundle it with the source of
|
|
||||||
your extension.
|
|
||||||
|
|
||||||
\begin{seealso}
|
|
||||||
|
|
||||||
\seeurl{http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Objects/obmalloc.c}
|
|
||||||
{For the full details of the pymalloc implementation, see
|
|
||||||
the comments at the top of the file \file{Objects/obmalloc.c} in the
|
|
||||||
Python source code. The above link points to the file within the
|
|
||||||
SourceForge CVS browser.}
|
|
||||||
|
|
||||||
\end{seealso}
|
|
||||||
|
|
||||||
|
|
||||||
%======================================================================
|
%======================================================================
|
||||||
\section{New and Improved Modules}
|
\section{New and Improved Modules}
|
||||||
|
|
||||||
|
@ -831,6 +747,13 @@ documentation for details.
|
||||||
% XXX add a link to the module docs?
|
% XXX add a link to the module docs?
|
||||||
(Contributed by Greg Ward.)
|
(Contributed by Greg Ward.)
|
||||||
|
|
||||||
|
\item The \module{time} module's \function{strptime()} function has
|
||||||
|
long been an annoyance because it uses the platform C library's
|
||||||
|
\function{strptime()} implementation, and different platforms
|
||||||
|
sometimes have odd bugs. Brett Cannon contributed a portable
|
||||||
|
implementation that's written in pure Python, which should behave
|
||||||
|
identically on all platforms.
|
||||||
|
|
||||||
\item The DOM implementation
|
\item The DOM implementation
|
||||||
in \module{xml.dom.minidom} can now generate XML output in a
|
in \module{xml.dom.minidom} can now generate XML output in a
|
||||||
particular encoding, by specifying an optional encoding argument to
|
particular encoding, by specifying an optional encoding argument to
|
||||||
|
@ -839,6 +762,88 @@ the \method{toxml()} and \method{toprettyxml()} methods of DOM nodes.
|
||||||
\end{itemize}
|
\end{itemize}
|
||||||
|
|
||||||
|
|
||||||
|
%======================================================================
|
||||||
|
\section{Specialized Object Allocator (pymalloc)\label{section-pymalloc}}
|
||||||
|
|
||||||
|
An experimental feature added to Python 2.1 was a specialized object
|
||||||
|
allocator called pymalloc, written by Vladimir Marangozov. Pymalloc
|
||||||
|
was intended to be faster than the system \cfunction{malloc()} and have
|
||||||
|
less memory overhead for typical allocation patterns of Python
|
||||||
|
programs. The allocator uses C's \cfunction{malloc()} function to get
|
||||||
|
large pools of memory, and then fulfills smaller memory requests from
|
||||||
|
these pools.
|
||||||
|
|
||||||
|
In 2.1 and 2.2, pymalloc was an experimental feature and wasn't
|
||||||
|
enabled by default; you had to explicitly turn it on by providing the
|
||||||
|
\longprogramopt{with-pymalloc} option to the \program{configure}
|
||||||
|
script. In 2.3, pymalloc has had further enhancements and is now
|
||||||
|
enabled by default; you'll have to supply
|
||||||
|
\longprogramopt{without-pymalloc} to disable it.
|
||||||
|
|
||||||
|
This change is transparent to code written in Python; however,
|
||||||
|
pymalloc may expose bugs in C extensions. Authors of C extension
|
||||||
|
modules should test their code with the object allocator enabled,
|
||||||
|
because some incorrect code may cause core dumps at runtime. There
|
||||||
|
are a bunch of memory allocation functions in Python's C API that have
|
||||||
|
previously been just aliases for the C library's \cfunction{malloc()}
|
||||||
|
and \cfunction{free()}, meaning that if you accidentally called
|
||||||
|
mismatched functions, the error wouldn't be noticeable. When the
|
||||||
|
object allocator is enabled, these functions aren't aliases of
|
||||||
|
\cfunction{malloc()} and \cfunction{free()} any more, and calling the
|
||||||
|
wrong function to free memory may get you a core dump. For example,
|
||||||
|
if memory was allocated using \cfunction{PyObject_Malloc()}, it has to
|
||||||
|
be freed using \cfunction{PyObject_Free()}, not \cfunction{free()}. A
|
||||||
|
few modules included with Python fell afoul of this and had to be
|
||||||
|
fixed; doubtless there are more third-party modules that will have the
|
||||||
|
same problem.
|
||||||
|
|
||||||
|
As part of this change, the confusing multiple interfaces for
|
||||||
|
allocating memory have been consolidated down into two API families.
|
||||||
|
Memory allocated with one family must not be manipulated with
|
||||||
|
functions from the other family.
|
||||||
|
|
||||||
|
There is another family of functions specifically for allocating
|
||||||
|
Python \emph{objects} (as opposed to memory).
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
\item To allocate and free an undistinguished chunk of memory use
|
||||||
|
the ``raw memory'' family: \cfunction{PyMem_Malloc()},
|
||||||
|
\cfunction{PyMem_Realloc()}, and \cfunction{PyMem_Free()}.
|
||||||
|
|
||||||
|
\item The ``object memory'' family is the interface to the pymalloc
|
||||||
|
facility described above and is biased towards a large number of
|
||||||
|
``small'' allocations: \cfunction{PyObject_Malloc},
|
||||||
|
\cfunction{PyObject_Realloc}, and \cfunction{PyObject_Free}.
|
||||||
|
|
||||||
|
\item To allocate and free Python objects, use the ``object'' family
|
||||||
|
\cfunction{PyObject_New()}, \cfunction{PyObject_NewVar()}, and
|
||||||
|
\cfunction{PyObject_Del()}.
|
||||||
|
\end{itemize}
|
||||||
|
|
||||||
|
Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides
|
||||||
|
debugging features to catch memory overwrites and doubled frees in
|
||||||
|
both extension modules and in the interpreter itself. To enable this
|
||||||
|
support, turn on the Python interpreter's debugging code by running
|
||||||
|
\program{configure} with \longprogramopt{with-pydebug}.
|
||||||
|
|
||||||
|
To aid extension writers, a header file \file{Misc/pymemcompat.h} is
|
||||||
|
distributed with the source to Python 2.3 that allows Python
|
||||||
|
extensions to use the 2.3 interfaces to memory allocation and compile
|
||||||
|
against any version of Python since 1.5.2. You would copy the file
|
||||||
|
from Python's source distribution and bundle it with the source of
|
||||||
|
your extension.
|
||||||
|
|
||||||
|
\begin{seealso}
|
||||||
|
|
||||||
|
\seeurl{http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Objects/obmalloc.c}
|
||||||
|
{For the full details of the pymalloc implementation, see
|
||||||
|
the comments at the top of the file \file{Objects/obmalloc.c} in the
|
||||||
|
Python source code. The above link points to the file within the
|
||||||
|
SourceForge CVS browser.}
|
||||||
|
|
||||||
|
\end{seealso}
|
||||||
|
|
||||||
|
|
||||||
% ======================================================================
|
% ======================================================================
|
||||||
\section{Build and C API Changes}
|
\section{Build and C API Changes}
|
||||||
|
|
||||||
|
@ -846,6 +851,33 @@ Changes to Python's build process and to the C API include:
|
||||||
|
|
||||||
\begin{itemize}
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item The C-level interface to the garbage collector has been changed,
|
||||||
|
to make it easier to write extension types that support garbage
|
||||||
|
collection, and to make it easier to debug misuses of the functions.
|
||||||
|
Various functions have slightly different semantics, so a bunch of
|
||||||
|
functions had to be renamed. Extensions that use the old API will
|
||||||
|
still compile but will \emph{not} participate in garbage collection,
|
||||||
|
so updating them for 2.3 should be considered fairly high priority.
|
||||||
|
|
||||||
|
To upgrade an extension module to the new API, perform the following
|
||||||
|
steps:
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Rename \cfunction{Py_TPFLAGS_GC} to \cfunction{PyTPFLAGS_HAVE_GC}.
|
||||||
|
|
||||||
|
\item Use \cfunction{PyObject_GC_New} or \cfunction{PyObject_GC_NewVar} to
|
||||||
|
allocate objects, and \cfunction{PyObject_GC_Del} to deallocate them.
|
||||||
|
|
||||||
|
\item Rename \cfunction{PyObject_GC_Init} to \cfunction{PyObject_GC_Track} and
|
||||||
|
\cfunction{PyObject_GC_Fini} to \cfunction{PyObject_GC_UnTrack}.
|
||||||
|
|
||||||
|
\item Remove \cfunction{PyGC_HEAD_SIZE} from object size calculations.
|
||||||
|
|
||||||
|
\item Remove calls to \cfunction{PyObject_AS_GC} and \cfunction{PyObject_FROM_GC}.
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
|
||||||
\item Python can now optionally be built as a shared library
|
\item Python can now optionally be built as a shared library
|
||||||
(\file{libpython2.3.so}) by supplying \longprogramopt{enable-shared}
|
(\file{libpython2.3.so}) by supplying \longprogramopt{enable-shared}
|
||||||
when running Python's \file{configure} script. (Contributed by Ondrej
|
when running Python's \file{configure} script. (Contributed by Ondrej
|
||||||
|
|
Loading…
Reference in New Issue