Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity
cpython/Include/pymem.h

114 lines
4.0 KiB
C
Raw Normal View History

Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
/* The PyMem_ family: low-level memory allocation interfaces.
See objimpl.h for the PyObject_ memory family.
*/
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 19:19:30 -03:00
#ifndef Py_PYMEM_H
#define Py_PYMEM_H
#include "pyport.h"
#ifdef __cplusplus
extern "C" {
#endif
/* BEWARE:
Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
Each interface exports both functions and macros. Extension modules should
use the functions, to ensure binary compatibility across Python versions.
Because the Python implementation is free to change internal details, and
the macros may (or may not) expose details for speed, if you do use the
macros you must recompile your extensions with each Python release.
Never mix calls to PyMem_ with calls to the platform malloc/realloc/
calloc/free. For example, on Windows different DLLs may end up using
different heaps, and if you use PyMem_Malloc you'll get the memory from the
heap used by the Python DLL; it could be a disaster if you free()'ed that
directly in your own extension. Using PyMem_Free instead ensures Python
can return the memory to the proper heap. As another example, in
PYMALLOC_DEBUG mode, Python wraps all calls to all PyMem_ and PyObject_
memory functions in special debugging wrappers that add additional
debugging info to dynamic memory blocks. The system routines have no idea
what to do with that stuff, and the Python wrappers have no idea what to do
with raw blocks obtained directly by the system routines then.
*/
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 19:19:30 -03:00
/*
* Raw memory interface
* ====================
*/
Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
/* Functions
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 19:19:30 -03:00
Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
Functions supplying platform-independent semantics for malloc/realloc/
free. These functions make sure that allocating 0 bytes returns a distinct
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 04:22:56 -03:00
non-NULL pointer (whenever possible -- if we're flat out of memory, NULL
may be returned), even if the platform malloc and realloc don't.
Returned pointers must be checked for NULL explicitly. No action is
Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
performed on failure (no exception is set, no warning is printed, etc).
*/
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 04:22:56 -03:00
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 19:19:30 -03:00
extern DL_IMPORT(void *) PyMem_Malloc(size_t);
extern DL_IMPORT(void *) PyMem_Realloc(void *, size_t);
extern DL_IMPORT(void) PyMem_Free(void *);
/* Starting from Python 1.6, the wrappers Py_{Malloc,Realloc,Free} are
no longer supported. They used to call PyErr_NoMemory() on failure. */
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 04:22:56 -03:00
/* Macros. */
Moving pymalloc along. + Redirect PyMem_{Del, DEL} to the object allocator's free() when pymalloc is enabled. Needed so old extensions can continue to mix PyObject_New with PyMem_DEL. + This implies that pgen needs to be able to see the PyObject_XYZ declarations too. pgenheaders.h now includes Python.h. An implication is that I expect obmalloc.o needs to get linked into pgen on non-Windows boxes. + When PYMALLOC_DEBUG is defined, *all* Py memory API functions now funnel through the debug allocator wrapper around pymalloc. This is the default in a debug build. + That caused compile.c to fail: it indirectly mixed PyMem_Malloc with raw platform free() in one place. This is verbotten.
2002-04-21 23:33:27 -03:00
#ifdef PYMALLOC_DEBUG
/* Redirect all memory operations to Python's debugging allocator. */
#define PyMem_MALLOC PyObject_MALLOC
#define PyMem_REALLOC PyObject_REALLOC
#else /* ! PYMALLOC_DEBUG */
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 04:22:56 -03:00
#ifdef MALLOC_ZERO_RETURNS_NULL
#define PyMem_MALLOC(n) malloc((n) ? (n) : 1)
#else
#define PyMem_MALLOC malloc
Drop the PyCore_* memory API.
2002-03-18 17:06:21 -04:00
#endif
Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 04:22:56 -03:00
/* Caution: whether MALLOC_ZERO_RETURNS_NULL is #defined has nothing to
do with whether platform realloc(non-NULL, 0) normally frees the memory
or returns NULL. Rather than introduce yet another config variation,
just make a realloc to 0 bytes act as if to 1 instead. */
#define PyMem_REALLOC(p, n) realloc((p), (n) ? (n) : 1)
Moving pymalloc along. + Redirect PyMem_{Del, DEL} to the object allocator's free() when pymalloc is enabled. Needed so old extensions can continue to mix PyObject_New with PyMem_DEL. + This implies that pgen needs to be able to see the PyObject_XYZ declarations too. pgenheaders.h now includes Python.h. An implication is that I expect obmalloc.o needs to get linked into pgen on non-Windows boxes. + When PYMALLOC_DEBUG is defined, *all* Py memory API functions now funnel through the debug allocator wrapper around pymalloc. This is the default in a debug build. + That caused compile.c to fail: it indirectly mixed PyMem_Malloc with raw platform free() in one place. This is verbotten.
2002-04-21 23:33:27 -03:00
#endif /* PYMALLOC_DEBUG */
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 04:22:56 -03:00
Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
/* In order to avoid breaking old code mixing PyObject_{New, NEW} with
PyMem_{Del, DEL} and PyMem_{Free, FREE}, the PyMem "release memory"
functions have to be redirected to the object deallocator. */
#define PyMem_FREE PyObject_FREE
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 19:19:30 -03:00
/*
* Type-oriented memory interface
* ==============================
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 04:22:56 -03:00
*
* These are carried along for historical reasons. There's rarely a good
Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
* reason to use them anymore (you can just as easily do the multiply and
* cast yourself).
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 19:19:30 -03:00
*/
#define PyMem_New(type, n) \
( (type *) PyMem_Malloc((n) * sizeof(type)) )
#define PyMem_NEW(type, n) \
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 04:22:56 -03:00
( (type *) PyMem_MALLOC((n) * sizeof(type)) )
Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
#define PyMem_Resize(p, type, n) \
( (p) = (type *) PyMem_Realloc((p), (n) * sizeof(type)) )
First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with PyMem_{Del, DEL} doesn't work yet (compilation problems). pyport.h: _PyMem_EXTRA is gone. pmem.h: Repaired comments. PyMem_{Malloc, MALLOC} and PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when asking for 0 bytes, and when passing a NULL pointer to the latter. object.c: PyMem_{Malloc, Realloc} just call their macro versions now, since the latter take care of the x-platform 0 and NULL stuff by themselves now. pypcre.c, grow_stack(): So sue me. On two lines, this called PyMem_RESIZE to grow a "const" area. It's not legit to realloc a const area, so the compiler warned given the new expansion of PyMem_RESIZE. It would have gotten the same warning before if it had used PyMem_Resize() instead; the older macro version, but not the function version, silently cast away the constness. IMO that was a wrong thing to do, and the docs say the macro versions of PyMem_xyz are deprecated anyway. If somebody else is resizing const areas with the macro spelling, they'll get a warning when they recompile now too.
2002-04-12 04:22:56 -03:00
#define PyMem_RESIZE(p, type, n) \
( (p) = (type *) PyMem_REALLOC((p), (n) * sizeof(type)) )
Moving pymalloc along. As threatened, PyMem_{Free, FREE} also invoke the object deallocator now when pymalloc is enabled (well, it does when pymalloc isn't enabled too, but in that case "the object deallocator" is plain free()). This is maximally backward-compatible, but it leaves a bitter aftertaste. Also massive reworking of comments.
2002-04-28 01:11:46 -03:00
/* In order to avoid breaking old code mixing PyObject_{New, NEW} with
PyMem_{Del, DEL} and PyMem_{Free, FREE}, the PyMem "release memory"
functions have to be redirected to the object deallocator. */
#define PyMem_Del PyObject_Free
#define PyMem_DEL PyObject_FREE
Give Python a debug-mode pymalloc, much as sketched on Python-Dev. When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug allocator. This can be done independent of build type (release or debug). A debug build automatically defines PYMALLOC_DEBUG when pymalloc is enabled. It's a detected error to define PYMALLOC_DEBUG when pymalloc isn't enabled. Two debugging entry points defined only under PYMALLOC_DEBUG: + _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb) to sanity-check a memory block obtained from pymalloc. It sprays info to stderr (see next) and dies via Py_FatalError if the block is detectably damaged. + _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info about a debug memory block to stderr. A tiny start at implementing "API family" checks isn't good for anything yet. _PyMalloc_DebugRealloc() has been optimized to do little when the new size is <= old size. However, if the new size is larger, it really can't call the underlying realloc() routine without either violating its contract, or knowing something non-trivial about how the underlying realloc() works. A memcpy is always done in this case. This was a disaster for (and only) one of the std tests: test_bufio creates single text file lines up to a million characters long. On Windows, fileobject.c's get_line() uses the horridly funky getline_via_fgets(), which keeps growing and growing a string object hoping to find a newline. It grew the string object 1000 bytes each time, so for a million-character string it took approximately forever (I gave up after a few minutes). So, also: fileobject.c, getline_via_fgets(): When a single line is outrageously long, grow the string object at a mildly exponential rate, instead of just 1000 bytes at a time. That's enough so that a debug-build test_bufio finishes in about 5 seconds on my Win98SE box. I'm curious to try this on Win2K, because it has very different memory behavior than Win9X, and test_bufio always took a factor of 10 longer to complete on Win2K. It *could* be that the endless reallocs were simply killing it on Win2K even in the release build.
2002-03-23 06:03:50 -04:00
patch from Vladimir (move Py_Mem* interface to Include/pymem.h)
2000-07-31 19:19:30 -03:00
#ifdef __cplusplus
}
#endif
#endif /* !Py_PYMEM_H */