From abfd6388cdba376a72686df52a9a98b2d558271b Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Thu, 29 Apr 2021 13:06:59 +0200 Subject: [PATCH] bpo-43774: Enhance debug build documentation (GH-25712) * Add also references to --with-trace-refs option. * Move _ob_next and _ob_prev at the end, since they don't exist by default and are related to debug. --- Doc/c-api/intro.rst | 32 +++++------------------------- Doc/c-api/typeobj.rst | 43 +++++++++++++++++++++-------------------- Doc/using/configure.rst | 43 +++++++++++++++++++++++------------------ Doc/whatsnew/3.8.rst | 4 ++-- 4 files changed, 53 insertions(+), 69 deletions(-) diff --git a/Doc/c-api/intro.rst b/Doc/c-api/intro.rst index 1223059ce9a..2d85d30702d 100644 --- a/Doc/c-api/intro.rst +++ b/Doc/c-api/intro.rst @@ -749,34 +749,12 @@ It is also implied by the presence of the not-Python-specific :c:macro:`_DEBUG` macro. When :c:macro:`Py_DEBUG` is enabled in the Unix build, compiler optimization is disabled. -In addition to the reference count debugging described below, the following -extra checks are performed: +In addition to the reference count debugging described below, extra checks are +performed, see :ref:`Python Debug Build `. -* Extra checks are added to the object allocator. - -* Extra checks are added to the parser and compiler. - -* Downcasts from wide types to narrow types are checked for loss of information. - -* A number of assertions are added to the dictionary and set implementations. - In addition, the set object acquires a :meth:`test_c_api` method. - -* Sanity checks of the input arguments are added to frame creation. - -* The storage for ints is initialized with a known invalid pattern to catch - reference to uninitialized digits. - -* Low-level tracing and extra exception checking are added to the runtime - virtual machine. - -* Extra checks are added to the memory arena implementation. - -* Extra debugging is added to the thread module. - -There may be additional checks not mentioned here. - -Defining :c:macro:`Py_TRACE_REFS` enables reference tracing. When defined, a -circular doubly linked list of active objects is maintained by adding two extra +Defining :c:macro:`Py_TRACE_REFS` enables reference tracing +(see the :option:`configure --with-trace-refs option <--with-trace-refs>`). +When defined, a circular doubly linked list of active objects is maintained by adding two extra fields to every :c:type:`PyObject`. Total allocations are tracked as well. Upon exit, all existing references are printed. (In interactive mode this happens after every statement run by the interpreter.) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 2af3b9a2e9d..85f0262d101 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -482,27 +482,6 @@ metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that it type objects) *must* have the :attr:`ob_size` field. -.. c:member:: PyObject* PyObject._ob_next - PyObject* PyObject._ob_prev - - These fields are only present when the macro ``Py_TRACE_REFS`` is defined. - - Their initialization to ``NULL`` is taken care of by the - ``PyObject_HEAD_INIT`` macro. For :ref:`statically allocated objects - `, these fields always remain ``NULL``. For :ref:`dynamically - allocated objects `, these two fields are used to link the - object into a doubly-linked list of *all* live objects on the heap. - - This could be used for various debugging purposes; currently the only uses - are the :func:`sys.getobjects` function and to print the objects that are - still alive at the end of a run when the environment variable - :envvar:`PYTHONDUMPREFS` is set. - - **Inheritance:** - - These fields are not inherited by subtypes. - - .. c:member:: Py_ssize_t PyObject.ob_refcnt This is the type object's reference count, initialized to ``1`` by the @@ -540,6 +519,28 @@ type objects) *must* have the :attr:`ob_size` field. This field is inherited by subtypes. +.. c:member:: PyObject* PyObject._ob_next + PyObject* PyObject._ob_prev + + These fields are only present when the macro ``Py_TRACE_REFS`` is defined + (see the :option:`configure --with-trace-refs option <--with-trace-refs>`). + + Their initialization to ``NULL`` is taken care of by the + ``PyObject_HEAD_INIT`` macro. For :ref:`statically allocated objects + `, these fields always remain ``NULL``. For :ref:`dynamically + allocated objects `, these two fields are used to link the + object into a doubly-linked list of *all* live objects on the heap. + + This could be used for various debugging purposes; currently the only uses + are the :func:`sys.getobjects` function and to print the objects that are + still alive at the end of a run when the environment variable + :envvar:`PYTHONDUMPREFS` is set. + + **Inheritance:** + + These fields are not inherited by subtypes. + + PyVarObject Slots ----------------- diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index e7fad7f2e62..4f3953ea5d7 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -208,43 +208,48 @@ recommended for best performance. .. _debug-build: -Debug build ------------ +Python Debug Build +------------------ A debug build is Python built with the :option:`--with-pydebug` configure option. Effects of a debug build: -* Define ``Py_DEBUG`` and ``Py_REF_DEBUG`` macros. +* Display all warnings by default: the list of default warning filters is empty + in the :mod:`warnings` module. * Add ``d`` to :data:`sys.abiflags`. * Add :func:`sys.gettotalrefcount` function. * Add :option:`-X showrefcount <-X>` command line option. * Add :envvar:`PYTHONTHREADDEBUG` environment variable. * Add support for the ``__ltrace__`` variable: enable low-level tracing in the bytecode evaluation loop if the variable is defined. -* The list of default warning filters is empty in the :mod:`warnings` module. -* Install debug hooks on memory allocators to detect buffer overflow and other - memory errors: see :c:func:`PyMem_SetupDebugHooks`. -* Build Python with assertions (don't set ``NDEBUG`` macro): - ``assert(...);`` and ``_PyObject_ASSERT(...);``. - See also the :option:`--with-assertions` configure option. -* Unicode and int objects are created with their memory filled with a pattern - to help detecting uninitialized bytes. -* Many functions ensure that are not called with an exception raised, since - they can clear or replace the current exception. -* The garbage collector (:func:`gc.collect` function) runs some basic checks on - objects consistency. -* More generally, add runtime checks, code surroundeded by ``#ifdef Py_DEBUG`` - and ``#endif``. +* Install :ref:`debug hooks on memory allocators ` + to detect buffer overflow and other memory errors. +* Define ``Py_DEBUG`` and ``Py_REF_DEBUG`` macros. +* Add runtime checks: code surroundeded by ``#ifdef Py_DEBUG`` and ``#endif``. + Enable ``assert(...)`` and ``_PyObject_ASSERT(...)`` assertions: don't set + the ``NDEBUG`` macro (see also the :option:`--with-assertions` configure + option). Main runtime checks: + + * Add sanity checks on the function arguments. + * Unicode and int objects are created with their memory filled with a pattern + to detect usage of uninitialized objects. + * Ensure that functions which can clear or replace the current exception are + not called with an exception raised. + * The garbage collector (:func:`gc.collect` function) runs some basic checks + on objects consistency. + * The :c:macro:`Py_SAFE_DOWNCAST()` macro checks for integer underflow and + overflow when downcasting from wide types to narrow types. See also the :ref:`Python Development Mode ` and the :option:`--with-trace-refs` configure option. .. versionchanged:: 3.8 Release builds and debug builds are now ABI compatible: defining the - ``Py_DEBUG`` macro no longer implies the ``Py_TRACE_REFS`` macro, which - introduces the only ABI incompatibility. + ``Py_DEBUG`` macro no longer implies the ``Py_TRACE_REFS`` macro (see the + :option:`--with-trace-refs` option), which introduces the only ABI + incompatibility. Debug options diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index c958bf4db09..f1a903624f4 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -215,8 +215,8 @@ Release builds and :ref:`debug builds ` are now ABI compatible: def ``Py_DEBUG`` macro no longer implies the ``Py_TRACE_REFS`` macro, which introduces the only ABI incompatibility. The ``Py_TRACE_REFS`` macro, which adds the :func:`sys.getobjects` function and the :envvar:`PYTHONDUMPREFS` -environment variable, can be set using the new ``./configure --with-trace-refs`` -build option. +environment variable, can be set using the new :option:`./configure +--with-trace-refs <--with-trace-refs>` build option. (Contributed by Victor Stinner in :issue:`36465`.) On Unix, C extensions are no longer linked to libpython except on Android