cpython/Include/pymem.h

114 lines
4.0 KiB
C
Raw Normal View History

/* The PyMem_ family: low-level memory allocation interfaces.
See objimpl.h for the PyObject_ memory family.
*/
#ifndef Py_PYMEM_H
#define Py_PYMEM_H
#include "pyport.h"
#ifdef __cplusplus
extern "C" {
#endif
/* BEWARE:
Each interface exports both functions and macros. Extension modules should
use the functions, to ensure binary compatibility across Python versions.
Because the Python implementation is free to change internal details, and
the macros may (or may not) expose details for speed, if you do use the
macros you must recompile your extensions with each Python release.
Never mix calls to PyMem_ with calls to the platform malloc/realloc/
calloc/free. For example, on Windows different DLLs may end up using
different heaps, and if you use PyMem_Malloc you'll get the memory from the
heap used by the Python DLL; it could be a disaster if you free()'ed that
directly in your own extension. Using PyMem_Free instead ensures Python
can return the memory to the proper heap. As another example, in
PYMALLOC_DEBUG mode, Python wraps all calls to all PyMem_ and PyObject_
memory functions in special debugging wrappers that add additional
debugging info to dynamic memory blocks. The system routines have no idea
what to do with that stuff, and the Python wrappers have no idea what to do
with raw blocks obtained directly by the system routines then.
*/
/*
* Raw memory interface
* ====================
*/
/* Functions
Functions supplying platform-independent semantics for malloc/realloc/
free. These functions make sure that allocating 0 bytes returns a distinct
non-NULL pointer (whenever possible -- if we're flat out of memory, NULL
may be returned), even if the platform malloc and realloc don't.
Returned pointers must be checked for NULL explicitly. No action is
performed on failure (no exception is set, no warning is printed, etc).
*/
extern DL_IMPORT(void *) PyMem_Malloc(size_t);
extern DL_IMPORT(void *) PyMem_Realloc(void *, size_t);
extern DL_IMPORT(void) PyMem_Free(void *);
/* Starting from Python 1.6, the wrappers Py_{Malloc,Realloc,Free} are
no longer supported. They used to call PyErr_NoMemory() on failure. */
/* Macros. */
#ifdef PYMALLOC_DEBUG
/* Redirect all memory operations to Python's debugging allocator. */
#define PyMem_MALLOC PyObject_MALLOC
#define PyMem_REALLOC PyObject_REALLOC
#else /* ! PYMALLOC_DEBUG */
#ifdef MALLOC_ZERO_RETURNS_NULL
#define PyMem_MALLOC(n) malloc((n) ? (n) : 1)
#else
#define PyMem_MALLOC malloc
2002-03-18 17:06:21 -04:00
#endif
/* Caution: whether MALLOC_ZERO_RETURNS_NULL is #defined has nothing to
do with whether platform realloc(non-NULL, 0) normally frees the memory
or returns NULL. Rather than introduce yet another config variation,
just make a realloc to 0 bytes act as if to 1 instead. */
#define PyMem_REALLOC(p, n) realloc((p), (n) ? (n) : 1)
#endif /* PYMALLOC_DEBUG */
/* In order to avoid breaking old code mixing PyObject_{New, NEW} with
PyMem_{Del, DEL} and PyMem_{Free, FREE}, the PyMem "release memory"
functions have to be redirected to the object deallocator. */
#define PyMem_FREE PyObject_FREE
/*
* Type-oriented memory interface
* ==============================
*
* These are carried along for historical reasons. There's rarely a good
* reason to use them anymore (you can just as easily do the multiply and
* cast yourself).
*/
#define PyMem_New(type, n) \
( (type *) PyMem_Malloc((n) * sizeof(type)) )
#define PyMem_NEW(type, n) \
( (type *) PyMem_MALLOC((n) * sizeof(type)) )
#define PyMem_Resize(p, type, n) \
( (p) = (type *) PyMem_Realloc((p), (n) * sizeof(type)) )
#define PyMem_RESIZE(p, type, n) \
( (p) = (type *) PyMem_REALLOC((p), (n) * sizeof(type)) )
/* In order to avoid breaking old code mixing PyObject_{New, NEW} with
PyMem_{Del, DEL} and PyMem_{Free, FREE}, the PyMem "release memory"
functions have to be redirected to the object deallocator. */
#define PyMem_Del PyObject_Free
#define PyMem_DEL PyObject_FREE
Give Python a debug-mode pymalloc, much as sketched on Python-Dev. When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug allocator. This can be done independent of build type (release or debug). A debug build automatically defines PYMALLOC_DEBUG when pymalloc is enabled. It's a detected error to define PYMALLOC_DEBUG when pymalloc isn't enabled. Two debugging entry points defined only under PYMALLOC_DEBUG: + _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb) to sanity-check a memory block obtained from pymalloc. It sprays info to stderr (see next) and dies via Py_FatalError if the block is detectably damaged. + _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info about a debug memory block to stderr. A tiny start at implementing "API family" checks isn't good for anything yet. _PyMalloc_DebugRealloc() has been optimized to do little when the new size is <= old size. However, if the new size is larger, it really can't call the underlying realloc() routine without either violating its contract, or knowing something non-trivial about how the underlying realloc() works. A memcpy is always done in this case. This was a disaster for (and only) one of the std tests: test_bufio creates single text file lines up to a million characters long. On Windows, fileobject.c's get_line() uses the horridly funky getline_via_fgets(), which keeps growing and growing a string object hoping to find a newline. It grew the string object 1000 bytes each time, so for a million-character string it took approximately forever (I gave up after a few minutes). So, also: fileobject.c, getline_via_fgets(): When a single line is outrageously long, grow the string object at a mildly exponential rate, instead of just 1000 bytes at a time. That's enough so that a debug-build test_bufio finishes in about 5 seconds on my Win98SE box. I'm curious to try this on Win2K, because it has very different memory behavior than Win9X, and test_bufio always took a factor of 10 longer to complete on Win2K. It *could* be that the endless reallocs were simply killing it on Win2K even in the release build.
2002-03-23 06:03:50 -04:00
#ifdef __cplusplus
}
#endif
#endif /* !Py_PYMEM_H */