Commit Graph

37 Commits

Author SHA1 Message Date
Benjamin Peterson ca47063998 replace Py_(u)intptr_t with the c99 standard types 2016-09-06 13:47:26 -07:00
Victor Stinner ca79ccd9e6 Issue #26588:
* Optimize tracemalloc_add_trace(): modify hashtable entry data (trace) if the
  memory block is already tracked, rather than trying to remove the old trace
  and then add a new trace.
* Add _Py_HASHTABLE_ENTRY_WRITE_DATA() macro
2016-03-23 09:38:54 +01:00
Victor Stinner f9a71153e9 Issue #26588: remove debug traces from _tracemalloc. 2016-03-22 23:54:42 +01:00
Victor Stinner ff9c5346ea Issue #26588: fix compilation on Windows 2016-03-22 16:29:02 +01:00
Victor Stinner 10b73e1748 Add C functions _PyTraceMalloc_Track()
Issue #26530:

* Add C functions _PyTraceMalloc_Track() and _PyTraceMalloc_Untrack() to track
  memory blocks using the tracemalloc module.
* Add _PyTraceMalloc_GetTraceback() to get the traceback of an object.
2016-03-22 13:39:05 +01:00
Victor Stinner 34be807ca4 Add PYTHONMALLOC env var
Issue #26516:

* Add PYTHONMALLOC environment variable to set the Python memory
  allocators and/or install debug hooks.
* PyMem_SetupDebugHooks() can now also be used on Python compiled in release
  mode.
* The PYTHONMALLOCSTATS environment variable can now also be used on Python
  compiled in release mode. It now has no effect if set to an empty string.
* In debug mode, debug hooks are now also installed on Python memory allocators
  when Python is configured without pymalloc.
2016-03-14 12:04:26 +01:00
Victor Stinner d8f0d922d5 Issue #21233: Rename the C structure "PyMemAllocator" to "PyMemAllocatorEx" to
make sure that the code using it will be adapted for the new "calloc" field
(instead of crashing).
2014-06-02 21:57:10 +02:00
Victor Stinner db067af12a Issue #21233: Add new C functions: PyMem_RawCalloc(), PyMem_Calloc(),
PyObject_Calloc(), _PyObject_GC_Calloc(). bytes(int) and bytearray(int) are now
using ``calloc()`` instead of ``malloc()`` for large objects which is faster
and use less memory (until the bytearray buffer is filled with data).
2014-05-02 22:31:14 +02:00
Martin v. Löwis 1c0689c613 Issue #19526: Exclude all new API from the stable ABI. 2014-01-03 21:36:49 +01:00
Victor Stinner 49fc8ece81 Issue #18203: Add _PyMem_RawStrdup() and _PyMem_Strdup()
Replace strdup() with _PyMem_RawStrdup() or _PyMem_Strdup(), depending if the
GIL is held or not.
2013-07-07 23:30:24 +02:00
Victor Stinner 0507bf56f0 Issue #3329: Implement the PEP 445
Add new enum:

* PyMemAllocatorDomain

Add new structures:

* PyMemAllocator
* PyObjectArenaAllocator

Add new functions:

* PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree()
* PyMem_GetAllocator(), PyMem_SetAllocator()
* PyObject_GetArenaAllocator(), PyObject_SetArenaAllocator()
* PyMem_SetupDebugHooks()

Changes:

* PyMem_Malloc()/PyObject_Realloc() now always call malloc()/realloc(), instead
  of calling PyObject_Malloc()/PyObject_Realloc() in debug mode.
* PyObject_Malloc()/PyObject_Realloc() now falls back to
  PyMem_Malloc()/PyMem_Realloc() for allocations larger than 512 bytes.
* Redesign debug checks on memory block allocators as hooks, instead of using C
  macros
2013-07-07 02:05:46 +02:00
Victor Stinner 36f01ad9ac Revert changeset 6661a8154eb3: Issue #3329: Add new APIs to customize memory allocators
The new API require more discussion.
2013-06-15 03:37:01 +02:00
Victor Stinner 4d7056258b Issue #3329: Add new APIs to customize memory allocators
* Add a new PyMemAllocators structure
* New functions:

  - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree(): GIL-free memory
    allocator functions
  - PyMem_GetRawAllocators(), PyMem_SetRawAllocators()
  - PyMem_GetAllocators(), PyMem_SetAllocators()
  - PyMem_SetupDebugHooks()
  - _PyObject_GetArenaAllocators(), _PyObject_SetArenaAllocators()

* Add unit test for PyMem_Malloc(0) and PyObject_Malloc(0)
* Add unit test for new get/set allocators functions
* PyObject_Malloc() now falls back on PyMem_Malloc() instead of malloc() if
  size is bigger than SMALL_REQUEST_THRESHOLD, and PyObject_Realloc() falls
  back on PyMem_Realloc() instead of realloc()
* PyMem_Malloc() and PyMem_Realloc() now always call malloc() and realloc(),
  instead of calling PyObject_Malloc() and PyObject_Realloc() in debug mode
2013-06-15 00:37:46 +02:00
Mark Dickinson bbe63069ef Merged revisions 78189 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78189 | mark.dickinson | 2010-02-14 13:40:30 +0000 (Sun, 14 Feb 2010) | 1 line

  Silence more 'comparison between signed and unsigned' warnings.
........
2010-02-14 14:08:54 +00:00
Mark Dickinson 629dfe28f5 Merged revisions 76740 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r76740 | mark.dickinson | 2009-12-10 10:36:32 +0000 (Thu, 10 Dec 2009) | 8 lines

  Replace the size check for PyMem_MALLOC and PyMem_REALLOC with an almost
  equivalent[*] check that doesn't produce compiler warnings about a 'x < 0'
  check on an unsigned type.

  [*] it's equivalent for inputs of type size_t or Py_ssize_t, or any smaller
  unsigned or signed integer type.
........
2009-12-10 10:39:08 +00:00
Kristján Valur Jónsson ae4cfb1bb3 http://bugs.python.org/issue6836
Merging revisions 75103,75104 from trunk to py3k
2009-09-28 13:45:02 +00:00
Georg Brandl d492ad80c8 Merged revisions 65182 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r65182 | gregory.p.smith | 2008-07-22 06:46:32 +0200 (Tue, 22 Jul 2008) | 7 lines

  Issue #2620: Overflow checking when allocating or reallocating memory
  was not always being done properly in some python types and extension
  modules.  PyMem_MALLOC, PyMem_REALLOC, PyMem_NEW and PyMem_RESIZE have
  all been updated to perform better checks and places in the code that
  would previously leak memory on the error path when such an allocation
  failed have been fixed.
........
2008-07-23 16:13:07 +00:00
Amaury Forgeot d'Arc 9c74b14fe9 Merged revisions 64114 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r64114 | gregory.p.smith | 2008-06-11 09:41:16 +0200 (mer., 11 juin 2008) | 6 lines

  Merge in release25-maint r60793:

   Added checks for integer overflows, contributed by Google. Some are
   only available if asserts are left in the code, in cases where they
   can't be triggered from Python code.
........
2008-06-18 00:47:36 +00:00
Guido van Rossum 360e4b8fb1 Merged revisions 55325-55327 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

................
  r55326 | guido.van.rossum | 2007-05-14 15:07:35 -0700 (Mon, 14 May 2007) | 2 lines

  Don't use err.message, use err.args[0].
................
  r55327 | guido.van.rossum | 2007-05-14 15:11:37 -0700 (Mon, 14 May 2007) | 259 lines

  Merged revisions 54988-55226,55228-55323 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r54995 | neal.norwitz | 2007-04-26 23:45:32 -0700 (Thu, 26 Apr 2007) | 3 lines

    This gets the test working on Solaris.  It seems a little hokey to me,
    but the test passed on Linux and Solaris, hopefully other platforms too.
  ........
    r55002 | georg.brandl | 2007-04-27 12:20:00 -0700 (Fri, 27 Apr 2007) | 2 lines

    Version fix (bug #1708710)
  ........
    r55021 | neal.norwitz | 2007-04-29 16:53:24 -0700 (Sun, 29 Apr 2007) | 1 line

    There really are some tests that are problematic.
  ........
    r55024 | kristjan.jonsson | 2007-04-30 08:17:46 -0700 (Mon, 30 Apr 2007) | 1 line

    Complete revamp of PCBuild8 directory.  Use subdirectories for each project under the main pcbuild solution.  Now make extensive use of property sheets to simplify project configuration. x64 build fully supported, and the process for building PGO version (Profiler Guided Optimization) simplified. All projects are now present, except _ssl, which needs to be reimplemented.  Also, some of the projects that require external libraries need extra work to fully compile on x64.
  ........
    r55025 | thomas.heller | 2007-04-30 08:44:17 -0700 (Mon, 30 Apr 2007) | 4 lines

    Make sure to call PyErr_NoMemory() in several places where
    PyMem_Malloc() could potentially fail.

    Will backport to the release25-maint branch.
  ........
    r55027 | thomas.heller | 2007-04-30 09:04:57 -0700 (Mon, 30 Apr 2007) | 8 lines

    When accessing the .value attribute of a c_wchar_p instance, and the
    instance does not point to a valid wchar_t zero-terminated string,
    raise a ValueError.  c_char_p does this already.

    The ValueError message now contains the correct pointer address.

    Will backport to release25-maint.
  ........
    r55036 | georg.brandl | 2007-04-30 23:04:11 -0700 (Mon, 30 Apr 2007) | 2 lines

    Bug #1710295: exceptions are now new-style classes.
  ........
    r55038 | georg.brandl | 2007-04-30 23:08:15 -0700 (Mon, 30 Apr 2007) | 2 lines

    Patch #1710352: add missing public functions to locale.__all__.
  ........
    r55041 | vinay.sajip | 2007-05-01 03:20:03 -0700 (Tue, 01 May 2007) | 1 line

    Added new optional credentials argument to SMTPHandler.__init__, and smtp.login() is now called in SMTPHandler.emit() if credentials are specified.
  ........
    r55042 | vinay.sajip | 2007-05-01 03:21:45 -0700 (Tue, 01 May 2007) | 1 line

    Added documentation for new optional credentials argument to SMTPHandler.__init__().
  ........
    r55070 | neal.norwitz | 2007-05-01 21:47:55 -0700 (Tue, 01 May 2007) | 3 lines

    Stop using PyMem_FREE while the GIL is not held.  For details see:
    http://mail.python.org/pipermail/python-dev/2007-May/072896.html
  ........
    r55080 | armin.rigo | 2007-05-02 12:23:31 -0700 (Wed, 02 May 2007) | 6 lines

    Fix for #1303614 and #1174712:
    - __dict__ descriptor abuse for subclasses of built-in types
    - subclassing from both ModuleType and another built-in types

    Thanks zseil for the patch.
  ........
    r55083 | georg.brandl | 2007-05-02 13:02:29 -0700 (Wed, 02 May 2007) | 3 lines

    Actually raise an exception before calling ast_error_finish.
    Triggers an assertion otherwise.
  ........
    r55087 | neal.norwitz | 2007-05-02 23:47:18 -0700 (Wed, 02 May 2007) | 1 line

    Handle a couple of uncaught errors.  This should be backported
  ........
    r55090 | neal.norwitz | 2007-05-03 00:20:57 -0700 (Thu, 03 May 2007) | 4 lines

    Remove dead code.  This code couldn't be reached because earlier in
    the function there is another check for z != Py_None.
  ........
    r55092 | thomas.heller | 2007-05-03 05:02:08 -0700 (Thu, 03 May 2007) | 1 line

    Fix building _ctypes.pyd for x64 / Windows.
  ........
    r55093 | thomas.heller | 2007-05-03 05:05:20 -0700 (Thu, 03 May 2007) | 1 line

    Don't truncate pointers to integers (on win64 platform).
  ........
    r55094 | walter.doerwald | 2007-05-03 08:13:55 -0700 (Thu, 03 May 2007) | 3 lines

    Clarify the behaviour of PyUnicode_DecodeUTF16(): A BOM is only skipped
    in native order mode, and only if it's the first two bytes.
  ........
    r55101 | kristjan.jonsson | 2007-05-03 13:04:53 -0700 (Thu, 03 May 2007) | 2 lines

    Fix pcbuild8 after recent overhaul:  Added the version resource to python26.dll. Adjust stacksize to 2Mb and made large address aware for 32 bits, and set stacksize to 3Mb for 64 bits.
    Todo:  Set .dll optimized load addresses, and side-by-side packaging of the python26.dll.
  ........
    r55102 | kristjan.jonsson | 2007-05-03 13:09:56 -0700 (Thu, 03 May 2007) | 1 line

    Fix those parts in the testsuite that assumed that sys.maxint would cause overflow on x64.  Now the testsuite is well behaved on that platform.
  ........
    r55103 | kristjan.jonsson | 2007-05-03 13:27:03 -0700 (Thu, 03 May 2007) | 11 lines

    Fix problems in x64 build that were discovered by the testsuite:
    - Reenable modules on x64 that had been disabled aeons ago for Itanium.
    - Cleared up confusion about compilers for 64 bit windows.  There is only Itanium and x64.  Added macros MS_WINI64 and MS_WINX64 for those rare cases where it matters, such as the disabling of modules above.
    - Set target platform (_WIN32_WINNT and WINVER) to 0x0501 (XP) for x64, and 0x0400 (NT 4.0) otherwise, which are the targeted minimum platforms.
    - Fixed thread_nt.h.  The emulated InterlockedCompareExchange function didn?\194?\180t work on x64, probaby due to the lack of a "volatile" specifier.  Anyway, win95 is no longer a target platform.
    - Itertools module used wrong constant to check for overflow in count()
    - PyInt_AsSsize_t couldn't deal with attribute error when accessing the __long__ member.
    - PyLong_FromSsize_t() incorrectly specified that the operand were unsigned.

    With these changes, the x64 passes the testsuite, for those modules present.
  ........
    r55107 | kristjan.jonsson | 2007-05-03 17:25:08 -0700 (Thu, 03 May 2007) | 1 line

    Revert compiler comment to AMD64 for x64/AMD64 builds.
  ........
    r55115 | thomas.heller | 2007-05-04 00:14:39 -0700 (Fri, 04 May 2007) | 4 lines

    Fix some ctypes test crashes, when running with a debug Python
    version on win64 by using proper argtypes and restype function
    attributes.
  ........
    r55117 | thomas.heller | 2007-05-04 01:20:41 -0700 (Fri, 04 May 2007) | 4 lines

    On 64-bit Windows, ffi_arg must be 8 bytes long.  This fixes the
    remaining crashes in the ctypes tests, when functions return float or
    double types.
  ........
    r55120 | kristjan.jonsson | 2007-05-04 08:48:15 -0700 (Fri, 04 May 2007) | 1 line

    Update the pcbuild8 solution.  Straightened out the _ctypes project by using a .vsproj file and a masm64.rules file to avoid redundancy
  ........
    r55121 | kristjan.jonsson | 2007-05-04 10:28:06 -0700 (Fri, 04 May 2007) | 1 line

    Minor fix of PCBuild8/_ctypes vcproj, moving include dir into the .vsprops file.
  ........
    r55129 | thomas.heller | 2007-05-04 12:54:22 -0700 (Fri, 04 May 2007) | 3 lines

    Do not truncate 64-bit pointers to 32-bit integers.

    Fixes SF #1703286, will backport to release25-maint.
  ........
    r55131 | thomas.heller | 2007-05-04 12:56:32 -0700 (Fri, 04 May 2007) | 1 line

    Oops, these tests do not run on Windows CE.
  ........
    r55140 | brett.cannon | 2007-05-04 18:34:02 -0700 (Fri, 04 May 2007) | 2 lines

    Deprecate BaseException.message as per PEP 352.
  ........
    r55154 | georg.brandl | 2007-05-05 11:55:37 -0700 (Sat, 05 May 2007) | 2 lines

    Bug #1713535: typo in logging example.
  ........
    r55158 | vinay.sajip | 2007-05-06 10:53:37 -0700 (Sun, 06 May 2007) | 1 line

    Updates of recent changes to logging.
  ........
    r55165 | neal.norwitz | 2007-05-07 00:02:26 -0700 (Mon, 07 May 2007) | 1 line

    Verify changes to the trunk go to the normal checkins list
  ........
    r55169 | kristjan.jonsson | 2007-05-07 09:46:54 -0700 (Mon, 07 May 2007) | 1 line

    As per Armin Rigo's suggestion, remove special handing from intobject.c to deal with the peculiarities of classobject's implementation of the number protocol.  The nb_long method of classobject now falls back to nb_int if there is no __long__ attribute present.
  ........
    r55197 | collin.winter | 2007-05-08 21:14:36 -0700 (Tue, 08 May 2007) | 9 lines

    Fix a bug in test.test_support.open_urlresource().

    If the call to requires() doesn't precede the filesystem check, we get the following situation:
    1. ./python Lib/test/regrtest.py test_foo # test needs urlfetch, not enabled, so skipped
    2. ./python Lib/test/regrtest.py -u urlfetch test_foo # test runs
    3. ./python Lib/test/regrtest.py test_foo # test runs (!)

    By moving the call to requires() *before* the filesystem check, the fact that fetched files are cached on the local disk becomes an implementation detail, rather than a semantics-changing point of note.
  ........
    r55198 | neal.norwitz | 2007-05-08 23:43:15 -0700 (Tue, 08 May 2007) | 1 line

    Add markup for True/False.  Will backport
  ........
    r55205 | walter.doerwald | 2007-05-09 11:10:47 -0700 (Wed, 09 May 2007) | 4 lines

    Backport checkin:
    Fix a segfault when b"" was passed to b2a_qp() -- it was using strchr()
    instead of memchr().
  ........
    r55241 | neal.norwitz | 2007-05-10 22:55:15 -0700 (Thu, 10 May 2007) | 6 lines

    Don't ever report a failure when the sum of the reference count differences
    are zero.  This should help reduce the false positives.

    The message about references leaking is maintained to provide as much
    info as possible rather than simply suppressing the message at the source.
  ........
    r55242 | neal.norwitz | 2007-05-10 23:23:01 -0700 (Thu, 10 May 2007) | 1 line

    Fix typo in docstring (the module is popen2, not 3).
  ........
    r55244 | neal.norwitz | 2007-05-10 23:56:52 -0700 (Thu, 10 May 2007) | 1 line

    Remove trailing whitespace in docstring
  ........
    r55245 | neal.norwitz | 2007-05-10 23:57:33 -0700 (Thu, 10 May 2007) | 1 line

    Deprecate os.popen* and popen2 module in favor of the subprocess module.
  ........
    r55247 | neal.norwitz | 2007-05-11 00:13:30 -0700 (Fri, 11 May 2007) | 1 line

    Deprecate os.popen* and popen2 module in favor of the subprocess module. (forgot the doc)
  ........
    r55253 | georg.brandl | 2007-05-11 02:41:37 -0700 (Fri, 11 May 2007) | 3 lines

    Remove an XXX that is unnecessary.
  ........
    r55258 | georg.brandl | 2007-05-11 04:04:26 -0700 (Fri, 11 May 2007) | 2 lines

    Patch #1714700: clarify os.linesep vs. tfiles opened in text mode.
     (backport)
  ........
    r55259 | georg.brandl | 2007-05-11 04:43:56 -0700 (Fri, 11 May 2007) | 2 lines

    Update DDJ link.
  ........
    r55273 | raymond.hettinger | 2007-05-11 10:59:59 -0700 (Fri, 11 May 2007) | 1 line

    Better tests for posixpath.commonprefix
  ........
    r55287 | georg.brandl | 2007-05-12 14:06:41 -0700 (Sat, 12 May 2007) | 2 lines

    Bug #1046945: document SWIG options of distutils.
  ........
    r55290 | georg.brandl | 2007-05-13 01:04:07 -0700 (Sun, 13 May 2007) | 2 lines

    Add bz2 to content encodings.
  ........
    r55297 | neal.norwitz | 2007-05-13 13:45:05 -0700 (Sun, 13 May 2007) | 3 lines

    Remove Amoeba doc which was removed in version 1.0! according to Misc/HISTORY.
    Hopefully Guido won't shed a tear. :-)
  ........
    r55298 | neal.norwitz | 2007-05-13 13:54:19 -0700 (Sun, 13 May 2007) | 1 line

    Remove references to stdwin which was removed long ago.
  ........
    r55299 | neal.norwitz | 2007-05-13 14:13:42 -0700 (Sun, 13 May 2007) | 3 lines

    Remove support for freebsd[23] which haven't been released since 2000
    or earlier.  http://www.freebsd.org/releases/index.html
  ........
    r55320 | raymond.hettinger | 2007-05-14 13:52:31 -0700 (Mon, 14 May 2007) | 1 line

    Small speedup.
  ........
................
2007-05-14 22:51:27 +00:00
Thomas Wouters 49fd7fa443 Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
number of tests, all because of the codecs/_multibytecodecs issue described
here (it's not a Py3K issue, just something Py3K discovers):
http://mail.python.org/pipermail/python-dev/2006-April/064051.html

Hye-Shik Chang promised to look for a fix, so no need to fix it here. The
tests that are expected to break are:

test_codecencodings_cn
test_codecencodings_hk
test_codecencodings_jp
test_codecencodings_kr
test_codecencodings_tw
test_codecs
test_multibytecodec

This merge fixes an actual test failure (test_weakref) in this branch,
though, so I believe merging is the right thing to do anyway.
2006-04-21 10:40:58 +00:00
Martin v. Löwis 39f59b089d Remove MALLOC_ZERO_RETURNS_NULL. 2002-11-23 09:13:40 +00:00
Mark Hammond 91a681debf Excise DL_EXPORT from Include.
Thanks to Skip Montanaro and Kalle Svensson for the patches.
2002-08-12 07:21:58 +00:00
Tim Peters 8b078f95e0 Moving pymalloc along.
As threatened, PyMem_{Free, FREE} also invoke the object deallocator now
when pymalloc is enabled (well, it does when pymalloc isn't enabled too,
but in that case "the object deallocator" is plain free()).

This is maximally backward-compatible, but it leaves a bitter aftertaste.

Also massive reworking of comments.
2002-04-28 04:11:46 +00:00
Tim Peters 51e7f5caba Moving pymalloc along.
+ Redirect PyMem_{Del, DEL} to the object allocator's free() when
  pymalloc is enabled.  Needed so old extensions can continue to
  mix PyObject_New with PyMem_DEL.

+ This implies that pgen needs to be able to see the PyObject_XYZ
  declarations too.  pgenheaders.h now includes Python.h.  An
  implication is that I expect obmalloc.o needs to get linked into
  pgen on non-Windows boxes.

+ When PYMALLOC_DEBUG is defined, *all* Py memory API functions
  now funnel through the debug allocator wrapper around pymalloc.
  This is the default in a debug build.

+ That caused compile.c to fail:  it indirectly mixed PyMem_Malloc
  with raw platform free() in one place.  This is verbotten.
2002-04-22 02:33:27 +00:00
Tim Peters af3e8de580 First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).

pyport.h:  _PyMem_EXTRA is gone.

pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.

object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.

pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
Neil Schemenauer 20d0a1a071 Remove PyMalloc_* symbols. PyObject_Malloc now uses pymalloc if
it's enabled.
2002-04-12 02:39:18 +00:00
Tim Peters 7ccfadf3a8 New PYMALLOC_DEBUG function void _PyMalloc_DebugDumpStats(void).
This displays stats about the # of arenas, pools, blocks and bytes, to
stderr, both used and reserved but unused.

CAUTION:  Because PYMALLOC_DEBUG is on, the debug malloc routine adds
16 bytes to each request.  This makes each block appear two size classes
higher than it would be if PYMALLOC_DEBUG weren't on.

So far, playing with this confirms the obvious:  there's a lot of activity
in the "small dict" size class, but nothing in the core makes any use of
the 8-byte or 16-byte classes.
2002-04-01 06:04:21 +00:00
Tim Peters d1139e043c PYMALLOC_DEBUG routines: The "check API family" gimmick was going nowhere
fast, and just cluttered the code.  Get rid of it for now.  If a compelling
case can be made for it, easy to restore it later.
2002-03-28 07:32:11 +00:00
Tim Peters ddea208be9 Give Python a debug-mode pymalloc, much as sketched on Python-Dev.
When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug
allocator.  This can be done independent of build type (release or debug).
A debug build automatically defines PYMALLOC_DEBUG when pymalloc is
enabled.  It's a detected error to define PYMALLOC_DEBUG when pymalloc
isn't enabled.

Two debugging entry points defined only under PYMALLOC_DEBUG:

+ _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb)
  to sanity-check a memory block obtained from pymalloc.  It sprays
  info to stderr (see next) and dies via Py_FatalError if the block is
  detectably damaged.

+ _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info
  about a debug memory block to stderr.

A tiny start at implementing "API family" checks isn't good for
anything yet.

_PyMalloc_DebugRealloc() has been optimized to do little when the new
size is <= old size.  However, if the new size is larger, it really
can't call the underlying realloc() routine without either violating its
contract, or knowing something non-trivial about how the underlying
realloc() works.  A memcpy is always done in this case.

This was a disaster for (and only) one of the std tests:  test_bufio
creates single text file lines up to a million characters long.  On
Windows, fileobject.c's get_line() uses the horridly funky
getline_via_fgets(), which keeps growing and growing a string object
hoping to find a newline.  It grew the string object 1000 bytes each
time, so for a million-character string it took approximately forever
(I gave up after a few minutes).

So, also:

fileobject.c, getline_via_fgets():  When a single line is outrageously
long, grow the string object at a mildly exponential rate, instead of
just 1000 bytes at a time.

That's enough so that a debug-build test_bufio finishes in about 5 seconds
on my Win98SE box.  I'm curious to try this on Win2K, because it has very
different memory behavior than Win9X, and test_bufio always took a factor
of 10 longer to complete on Win2K.  It *could* be that the endless
reallocs were simply killing it on Win2K even in the release build.
2002-03-23 10:03:50 +00:00
Tim Peters fbb556df15 Arrange to export the _PyMalloc_{Malloc, Realloc, Free} entry points. On
Windows some modules are considered (by me, and I don't care what anyone
else thinks about this <wink>) to be part of "the core" despite that they
happen to be compiled into separate DLLs (the "to DLL or not to DLL?"
question on Windows is nearly arbitrary).  Making the pymalloc entry
points available to them allows the Windows build to complete without
incident when WITH_PYMALLOC is #define'd.

Note that this isn't unprecedented.  Other "private API" functions we
export include _PySequence_IterSearch, _PyEval_SliceIndex, _PyCodec_Lookup,
_Py_ZeroStruct, _Py_TrueStruct, _PyLong_New and _PyModule_Clear.
2002-03-20 04:02:31 +00:00
Neil Schemenauer 25f3dc21b5 Drop the PyCore_* memory API. 2002-03-18 21:06:21 +00:00
Neil Schemenauer 11f5be8d88 Simpilify PyCore_* macros by assuming the function prototypes for
malloc() and free() don't change.
2002-03-18 18:13:41 +00:00
Tim Peters a5d78cc208 Whether platform malloc(0) returns NULL has nothing to do with whether
platform realloc(p, 0) returns NULL, so MALLOC_ZERO_RETURNS_NULL can
be correctly undefined yet realloc(p, 0) can return NULL anyway.

Prevent realloc(p, 0) doing free(p) and returning NULL via a different
hack.  Would probably be better to get rid of MALLOC_ZERO_RETURNS_NULL
entirely.

Bugfix candidate.
2002-03-02 08:43:19 +00:00
Tim Peters 76f373d081 Repair more now-obsolete references to config.h. 2001-07-26 21:34:59 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Vladimir Marangozov dcb45c34f5 Fix a typo in the PyMem_Resize macro, found by Andrew Kuchling 2000-08-13 11:59:08 +00:00
Peter Schneider-Kamp 25f68944c2 patch from Vladimir (move Py_Mem* interface to Include/pymem.h) 2000-07-31 22:19:30 +00:00