Commit Graph

57 Commits

Author SHA1 Message Date
Victor Stinner 7076bef8ba
bpo-45434: Remove useless space in includes (GH-28963)
Micro-optimize spaces!
2021-10-15 01:50:04 +02:00
Victor Stinner 645ed62fb4
bpo-43774: Remove unused PYMALLOC_DEBUG macro (GH-25711)
Enhance also the documentation of debug hooks on memory allocators.
2021-04-29 10:47:47 +02:00
Petr Viktorin 9d6a2d0ee7
bpo-43795: PEP-652: Clean up the stable ABI/limited API (GH-25482)
- `_Py_EncodeLocaleRaw`, which is private by name, undocumented,
  and wasn't exported in `python3.dll`, is moved to a private header.
- `_Py_HashSecret_Initialized`, again private by name, undocumented,
  and not exported in `python3.dll`, is excluded with `Py_LIMITED_API`.
- `PyMarshal_*` and `PyMember_*One` functions, declared in private headers and
  not exported in `python3.dll`, are removed from `Doc/data/stable_abi.dat`.
- `PyMem_Calloc` which *was* exported in `python3dll.c`, is moved to public
  headers where it joins its other `PyMem_*` friends.

Only the last change is documented in the blurb; others are not user-visible.
(Nothing uses `Doc/data/stable_abi.dat` yet.)


https://bugs.python.org/issue43795
2021-04-23 14:17:58 +02:00
Victor Stinner 32bd68c839
bpo-42519: Replace PyObject_MALLOC() with PyObject_Malloc() (GH-23587)
No longer use deprecated aliases to functions:

* Replace PyObject_MALLOC() with PyObject_Malloc()
* Replace PyObject_REALLOC() with PyObject_Realloc()
* Replace PyObject_FREE() with PyObject_Free()
* Replace PyObject_Del() with PyObject_Free()
* Replace PyObject_DEL() with PyObject_Free()
2020-12-01 10:37:39 +01:00
Victor Stinner 00d7abd7ef
bpo-42519: Replace PyMem_MALLOC() with PyMem_Malloc() (GH-23586)
No longer use deprecated aliases to functions:

* Replace PyMem_MALLOC() with PyMem_Malloc()
* Replace PyMem_REALLOC() with PyMem_Realloc()
* Replace PyMem_FREE() with PyMem_Free()
* Replace PyMem_Del() with PyMem_Free()
* Replace PyMem_DEL() with PyMem_Free()

Modify also the PyMem_DEL() macro to use directly PyMem_Free().
2020-12-01 09:56:42 +01:00
Victor Stinner 40e547dfbb
bpo-39542: Make _Py_NewReference() opaque in C API (GH-18346)
_Py_NewReference() becomes a regular opaque function, rather than a
static inline function in the C API (object.h), to better hide
implementation details.

Move _Py_tracemalloc_config from public pymem.h to internal
pycore_pymem.h header.

Make _Py_AddToAllObjects() private.
2020-02-05 01:11:10 +01:00
Victor Stinner 9820c07e41
bpo-35134: Add Include/cpython/pymem.h (GH-12840)
Move unstable CPython API from Include/pymem.h into a new
Include/cpython/pymem.h header file.
2019-04-15 17:00:19 +02:00
Victor Stinner 2b00db6855
bpo-36389: _PyObject_IsFreed() now also detects uninitialized memory (GH-12770)
Replace _PyMem_IsFreed() function with _PyMem_IsPtrFreed() inline
function. The function is now way more efficient, it became a simple
comparison on integers, rather than a short loop. It detects also
uninitialized bytes and "forbidden bytes" filled by debug hooks
on memory allocators.

Add unit tests on _PyObject_IsFreed().
2019-04-11 11:33:27 +02:00
Victor Stinner 2be00d987d
bpo-35081: Move Py_BUILD_CORE code to internal/mem.h (GH-10249)
* Add #include "internal/mem.h" to C files using
  _PyMem_SetDefaultAllocator().
* Include/internal/mem.h now requires Py_BUILD_CORE to be defined.
2018-10-31 20:19:24 +01:00
Victor Stinner 626bff8568
bpo-9263: Dump Python object on GC assertion failure (GH-10062)
Changes:

* Add _PyObject_AssertFailed() function.
* Add _PyObject_ASSERT() and _PyObject_ASSERT_WITH_MSG() macros.
* gc_decref(): replace assert() with _PyObject_ASSERT_WITH_MSG() to
  dump the faulty object if the assertion fails.

_PyObject_AssertFailed() calls:

* _PyMem_DumpTraceback(): try to log the traceback where the object
  memory has been allocated if tracemalloc is enabled.
* _PyObject_Dump(): log repr(obj).
* Py_FatalError(): log the current Python traceback.

_PyObject_AssertFailed() uses _PyObject_IsFreed() heuristic to check
if the object memory has been freed by a debug hook on Python memory
allocators.

Initial patch written by David Malcolm.

Co-Authored-By: David Malcolm <dmalcolm@redhat.com>
2018-10-25 17:31:10 +02:00
Victor Stinner 6279c1c500
bpo-35053: Add Include/tracemalloc.h (GH-10091)
* Modify object.h to ensure that pymem.h is included,
  to get _Py_tracemalloc_config variable.
* Move _PyTraceMalloc_XXX() functions to tracemalloc.h,
  they need PyObject type. Break circular dependency between pymem.h
  and object.h.
2018-10-25 15:54:13 +02:00
Victor Stinner 9e00e80e21
bpo-35053: Enhance tracemalloc to trace free lists (GH-10063)
tracemalloc now tries to update the traceback when an object is
reused from a "free list" (optimization for faster object creation,
used by the builtin list type for example).

Changes:

* Add _PyTraceMalloc_NewReference() function which tries to update
  the Python traceback of a Python object.
* _Py_NewReference() now calls _PyTraceMalloc_NewReference().
* Add an unit test.
2018-10-25 13:31:16 +02:00
Victor Stinner 82af0b63b0
bpo-9263: _PyObject_Dump() detects freed memory (GH-10061)
_PyObject_Dump() now uses an heuristic to check if the object memory
has been freed: log "<freed object>" in that case.

The heuristic rely on the debug hooks on Python memory allocators
which fills the memory with DEADBYTE (0xDB) when memory is
deallocated. Use PYTHONMALLOC=debug to always enable these debug
hooks.
2018-10-23 17:39:40 +02:00
Victor Stinner 6bf992a1ac
bpo-32030: Add pymain_get_global_config() (#4735)
* Py_Main() now starts by reading Py_xxx configuration variables to
  only work on its own private structure, and then later writes back
  the configuration into these variables.
* Replace Py_GETENV() with pymain_get_env_var() which ignores empty
  variables.
* Add _PyCoreConfig.dump_refs
* Add _PyCoreConfig.malloc_stats
* _PyObject_DebugMallocStats() is now responsible to check if debug
  hooks are installed. The function returns 1 if stats were written,
  or 0 if the hooks are disabled. Mark _PyMem_PymallocEnabled() as
  static.
2017-12-06 17:26:10 +01:00
Victor Stinner 5d39e04290
bpo-32030: Rework memory allocators (#4625)
* Fix _PyMem_SetupAllocators("debug"): always restore allocators to
  the defaults, rather than only caling _PyMem_SetupDebugHooks().
* Add _PyMem_SetDefaultAllocator() helper to set the "default"
  allocator.
* Add _PyMem_GetAllocatorsName(): get the name of the allocators
* main() now uses debug hooks on memory allocators if Py_DEBUG is
  defined, rather than calling directly malloc()
* Document default memory allocators in C API documentation
* _Py_InitializeCore() now fails with a fatal user error if
  PYTHONMALLOC value is an unknown memory allocator, instead of
  failing with a fatal internal error.
* Add new tests on the PYTHONMALLOC environment variable
* Add support.with_pymalloc()
* Add the _testcapi.WITH_PYMALLOC constant and expose it as
   support.with_pymalloc().
* sysconfig.get_config_var('WITH_PYMALLOC') doesn't work on Windows, so
   replace it with support.with_pymalloc().
* pythoninfo: add _testcapi collector for pymem
2017-11-29 17:20:38 +01:00
Victor Stinner 46972b7bc3
bpo-32030: Add _PyMainInterpreterConfig_ReadEnv() (#4542)
Py_GetPath() and Py_Main() now call
_PyMainInterpreterConfig_ReadEnv() to share the same code to get
environment variables.

Changes:

* Add _PyMainInterpreterConfig_ReadEnv()
* Add _PyMainInterpreterConfig_Clear()
* Add _PyMem_RawWcsdup()
* _PyMainInterpreterConfig: rename pythonhome to home
* Rename _Py_ReadMainInterpreterConfig() to
  _PyMainInterpreterConfig_Read()
* Use _Py_INIT_USER_ERR(), instead of _Py_INIT_ERR(), for decoding
  errors: the user is able to fix the issue, it's not a bug in
  Python. Same change was made in _Py_INIT_NO_MEMORY().
* Remove _Py_GetPythonHomeWithConfig()
2017-11-24 22:55:40 +01:00
Victor Stinner f7e5b56c37
bpo-32030: Split Py_Main() into subfunctions (#4399)
* Don't use "Python runtime" anymore to parse command line options or
  to get environment variables: pymain_init() is now a strict
  separation.
* Use an error message rather than "crashing" directly with
  Py_FatalError(). Limit the number of calls to Py_FatalError(). It
  prepares the code to handle errors more nicely later.
* Warnings options (-W, PYTHONWARNINGS) and "XOptions" (-X) are now
  only added to the sys module once Python core is properly
  initialized.
* _PyMain is now the well identified owner of some important strings
  like: warnings options, XOptions, and the "program name". The
  program name string is now properly freed at exit.
  pymain_free() is now responsible to free the "command" string.
* Rename most methods in Modules/main.c to use a "pymain_" prefix to
  avoid conflits and ease debug.
* Replace _Py_CommandLineDetails_INIT with memset(0)
* Reorder a lot of code to fix the initialization ordering. For
  example, initializing standard streams now comes before parsing
  PYTHONWARNINGS.
* Py_Main() now handles errors when adding warnings options and
  XOptions.
* Add _PyMem_GetDefaultRawAllocator() private function.
* Cleanup _PyMem_Initialize(): remove useless global constants: move
  them into _PyMem_Initialize().
* Call _PyRuntime_Initialize() as soon as possible:
  _PyRuntime_Initialize() now returns an error message on failure.
* Add _PyInitError structure and following macros:

  * _Py_INIT_OK()
  * _Py_INIT_ERR(msg)
  * _Py_INIT_USER_ERR(msg): "user" error, don't abort() in that case
  * _Py_INIT_FAILED(err)
2017-11-15 15:48:08 -08:00
luzpaz a5293b4ff2 Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
Victor Stinner 5ea4c06773 bpo-30054: Expose tracemalloc C API (#1236)
* Make PyTraceMalloc_Track() and PyTraceMalloc_Untrack() functions
  public (remove the "_" prefix)
* Remove the _PyTraceMalloc_domain_t type: use directly unsigned
  int.
* Document methods

Note: methods are already tested in test_tracemalloc.
2017-06-20 17:46:36 +02:00
Serhiy Storchaka 34d0ac8027 Issue #29058: All stable API extensions added after Python 3.2 are now
available only when Py_LIMITED_API is set to the PY_VERSION_HEX value of
the minimum Python version supporting this API.
2016-12-27 14:57:39 +02:00
Benjamin Peterson ca47063998 replace Py_(u)intptr_t with the c99 standard types 2016-09-06 13:47:26 -07:00
Victor Stinner ca79ccd9e6 Issue #26588:
* Optimize tracemalloc_add_trace(): modify hashtable entry data (trace) if the
  memory block is already tracked, rather than trying to remove the old trace
  and then add a new trace.
* Add _Py_HASHTABLE_ENTRY_WRITE_DATA() macro
2016-03-23 09:38:54 +01:00
Victor Stinner f9a71153e9 Issue #26588: remove debug traces from _tracemalloc. 2016-03-22 23:54:42 +01:00
Victor Stinner ff9c5346ea Issue #26588: fix compilation on Windows 2016-03-22 16:29:02 +01:00
Victor Stinner 10b73e1748 Add C functions _PyTraceMalloc_Track()
Issue #26530:

* Add C functions _PyTraceMalloc_Track() and _PyTraceMalloc_Untrack() to track
  memory blocks using the tracemalloc module.
* Add _PyTraceMalloc_GetTraceback() to get the traceback of an object.
2016-03-22 13:39:05 +01:00
Victor Stinner 34be807ca4 Add PYTHONMALLOC env var
Issue #26516:

* Add PYTHONMALLOC environment variable to set the Python memory
  allocators and/or install debug hooks.
* PyMem_SetupDebugHooks() can now also be used on Python compiled in release
  mode.
* The PYTHONMALLOCSTATS environment variable can now also be used on Python
  compiled in release mode. It now has no effect if set to an empty string.
* In debug mode, debug hooks are now also installed on Python memory allocators
  when Python is configured without pymalloc.
2016-03-14 12:04:26 +01:00
Victor Stinner d8f0d922d5 Issue #21233: Rename the C structure "PyMemAllocator" to "PyMemAllocatorEx" to
make sure that the code using it will be adapted for the new "calloc" field
(instead of crashing).
2014-06-02 21:57:10 +02:00
Victor Stinner db067af12a Issue #21233: Add new C functions: PyMem_RawCalloc(), PyMem_Calloc(),
PyObject_Calloc(), _PyObject_GC_Calloc(). bytes(int) and bytearray(int) are now
using ``calloc()`` instead of ``malloc()`` for large objects which is faster
and use less memory (until the bytearray buffer is filled with data).
2014-05-02 22:31:14 +02:00
Martin v. Löwis 1c0689c613 Issue #19526: Exclude all new API from the stable ABI. 2014-01-03 21:36:49 +01:00
Victor Stinner 49fc8ece81 Issue #18203: Add _PyMem_RawStrdup() and _PyMem_Strdup()
Replace strdup() with _PyMem_RawStrdup() or _PyMem_Strdup(), depending if the
GIL is held or not.
2013-07-07 23:30:24 +02:00
Victor Stinner 0507bf56f0 Issue #3329: Implement the PEP 445
Add new enum:

* PyMemAllocatorDomain

Add new structures:

* PyMemAllocator
* PyObjectArenaAllocator

Add new functions:

* PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree()
* PyMem_GetAllocator(), PyMem_SetAllocator()
* PyObject_GetArenaAllocator(), PyObject_SetArenaAllocator()
* PyMem_SetupDebugHooks()

Changes:

* PyMem_Malloc()/PyObject_Realloc() now always call malloc()/realloc(), instead
  of calling PyObject_Malloc()/PyObject_Realloc() in debug mode.
* PyObject_Malloc()/PyObject_Realloc() now falls back to
  PyMem_Malloc()/PyMem_Realloc() for allocations larger than 512 bytes.
* Redesign debug checks on memory block allocators as hooks, instead of using C
  macros
2013-07-07 02:05:46 +02:00
Victor Stinner 36f01ad9ac Revert changeset 6661a8154eb3: Issue #3329: Add new APIs to customize memory allocators
The new API require more discussion.
2013-06-15 03:37:01 +02:00
Victor Stinner 4d7056258b Issue #3329: Add new APIs to customize memory allocators
* Add a new PyMemAllocators structure
* New functions:

  - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree(): GIL-free memory
    allocator functions
  - PyMem_GetRawAllocators(), PyMem_SetRawAllocators()
  - PyMem_GetAllocators(), PyMem_SetAllocators()
  - PyMem_SetupDebugHooks()
  - _PyObject_GetArenaAllocators(), _PyObject_SetArenaAllocators()

* Add unit test for PyMem_Malloc(0) and PyObject_Malloc(0)
* Add unit test for new get/set allocators functions
* PyObject_Malloc() now falls back on PyMem_Malloc() instead of malloc() if
  size is bigger than SMALL_REQUEST_THRESHOLD, and PyObject_Realloc() falls
  back on PyMem_Realloc() instead of realloc()
* PyMem_Malloc() and PyMem_Realloc() now always call malloc() and realloc(),
  instead of calling PyObject_Malloc() and PyObject_Realloc() in debug mode
2013-06-15 00:37:46 +02:00
Mark Dickinson bbe63069ef Merged revisions 78189 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78189 | mark.dickinson | 2010-02-14 13:40:30 +0000 (Sun, 14 Feb 2010) | 1 line

  Silence more 'comparison between signed and unsigned' warnings.
........
2010-02-14 14:08:54 +00:00
Mark Dickinson 629dfe28f5 Merged revisions 76740 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r76740 | mark.dickinson | 2009-12-10 10:36:32 +0000 (Thu, 10 Dec 2009) | 8 lines

  Replace the size check for PyMem_MALLOC and PyMem_REALLOC with an almost
  equivalent[*] check that doesn't produce compiler warnings about a 'x < 0'
  check on an unsigned type.

  [*] it's equivalent for inputs of type size_t or Py_ssize_t, or any smaller
  unsigned or signed integer type.
........
2009-12-10 10:39:08 +00:00
Kristján Valur Jónsson ae4cfb1bb3 http://bugs.python.org/issue6836
Merging revisions 75103,75104 from trunk to py3k
2009-09-28 13:45:02 +00:00
Georg Brandl d492ad80c8 Merged revisions 65182 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r65182 | gregory.p.smith | 2008-07-22 06:46:32 +0200 (Tue, 22 Jul 2008) | 7 lines

  Issue #2620: Overflow checking when allocating or reallocating memory
  was not always being done properly in some python types and extension
  modules.  PyMem_MALLOC, PyMem_REALLOC, PyMem_NEW and PyMem_RESIZE have
  all been updated to perform better checks and places in the code that
  would previously leak memory on the error path when such an allocation
  failed have been fixed.
........
2008-07-23 16:13:07 +00:00
Amaury Forgeot d'Arc 9c74b14fe9 Merged revisions 64114 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r64114 | gregory.p.smith | 2008-06-11 09:41:16 +0200 (mer., 11 juin 2008) | 6 lines

  Merge in release25-maint r60793:

   Added checks for integer overflows, contributed by Google. Some are
   only available if asserts are left in the code, in cases where they
   can't be triggered from Python code.
........
2008-06-18 00:47:36 +00:00
Guido van Rossum 360e4b8fb1 Merged revisions 55325-55327 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

................
  r55326 | guido.van.rossum | 2007-05-14 15:07:35 -0700 (Mon, 14 May 2007) | 2 lines

  Don't use err.message, use err.args[0].
................
  r55327 | guido.van.rossum | 2007-05-14 15:11:37 -0700 (Mon, 14 May 2007) | 259 lines

  Merged revisions 54988-55226,55228-55323 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r54995 | neal.norwitz | 2007-04-26 23:45:32 -0700 (Thu, 26 Apr 2007) | 3 lines

    This gets the test working on Solaris.  It seems a little hokey to me,
    but the test passed on Linux and Solaris, hopefully other platforms too.
  ........
    r55002 | georg.brandl | 2007-04-27 12:20:00 -0700 (Fri, 27 Apr 2007) | 2 lines

    Version fix (bug #1708710)
  ........
    r55021 | neal.norwitz | 2007-04-29 16:53:24 -0700 (Sun, 29 Apr 2007) | 1 line

    There really are some tests that are problematic.
  ........
    r55024 | kristjan.jonsson | 2007-04-30 08:17:46 -0700 (Mon, 30 Apr 2007) | 1 line

    Complete revamp of PCBuild8 directory.  Use subdirectories for each project under the main pcbuild solution.  Now make extensive use of property sheets to simplify project configuration. x64 build fully supported, and the process for building PGO version (Profiler Guided Optimization) simplified. All projects are now present, except _ssl, which needs to be reimplemented.  Also, some of the projects that require external libraries need extra work to fully compile on x64.
  ........
    r55025 | thomas.heller | 2007-04-30 08:44:17 -0700 (Mon, 30 Apr 2007) | 4 lines

    Make sure to call PyErr_NoMemory() in several places where
    PyMem_Malloc() could potentially fail.

    Will backport to the release25-maint branch.
  ........
    r55027 | thomas.heller | 2007-04-30 09:04:57 -0700 (Mon, 30 Apr 2007) | 8 lines

    When accessing the .value attribute of a c_wchar_p instance, and the
    instance does not point to a valid wchar_t zero-terminated string,
    raise a ValueError.  c_char_p does this already.

    The ValueError message now contains the correct pointer address.

    Will backport to release25-maint.
  ........
    r55036 | georg.brandl | 2007-04-30 23:04:11 -0700 (Mon, 30 Apr 2007) | 2 lines

    Bug #1710295: exceptions are now new-style classes.
  ........
    r55038 | georg.brandl | 2007-04-30 23:08:15 -0700 (Mon, 30 Apr 2007) | 2 lines

    Patch #1710352: add missing public functions to locale.__all__.
  ........
    r55041 | vinay.sajip | 2007-05-01 03:20:03 -0700 (Tue, 01 May 2007) | 1 line

    Added new optional credentials argument to SMTPHandler.__init__, and smtp.login() is now called in SMTPHandler.emit() if credentials are specified.
  ........
    r55042 | vinay.sajip | 2007-05-01 03:21:45 -0700 (Tue, 01 May 2007) | 1 line

    Added documentation for new optional credentials argument to SMTPHandler.__init__().
  ........
    r55070 | neal.norwitz | 2007-05-01 21:47:55 -0700 (Tue, 01 May 2007) | 3 lines

    Stop using PyMem_FREE while the GIL is not held.  For details see:
    http://mail.python.org/pipermail/python-dev/2007-May/072896.html
  ........
    r55080 | armin.rigo | 2007-05-02 12:23:31 -0700 (Wed, 02 May 2007) | 6 lines

    Fix for #1303614 and #1174712:
    - __dict__ descriptor abuse for subclasses of built-in types
    - subclassing from both ModuleType and another built-in types

    Thanks zseil for the patch.
  ........
    r55083 | georg.brandl | 2007-05-02 13:02:29 -0700 (Wed, 02 May 2007) | 3 lines

    Actually raise an exception before calling ast_error_finish.
    Triggers an assertion otherwise.
  ........
    r55087 | neal.norwitz | 2007-05-02 23:47:18 -0700 (Wed, 02 May 2007) | 1 line

    Handle a couple of uncaught errors.  This should be backported
  ........
    r55090 | neal.norwitz | 2007-05-03 00:20:57 -0700 (Thu, 03 May 2007) | 4 lines

    Remove dead code.  This code couldn't be reached because earlier in
    the function there is another check for z != Py_None.
  ........
    r55092 | thomas.heller | 2007-05-03 05:02:08 -0700 (Thu, 03 May 2007) | 1 line

    Fix building _ctypes.pyd for x64 / Windows.
  ........
    r55093 | thomas.heller | 2007-05-03 05:05:20 -0700 (Thu, 03 May 2007) | 1 line

    Don't truncate pointers to integers (on win64 platform).
  ........
    r55094 | walter.doerwald | 2007-05-03 08:13:55 -0700 (Thu, 03 May 2007) | 3 lines

    Clarify the behaviour of PyUnicode_DecodeUTF16(): A BOM is only skipped
    in native order mode, and only if it's the first two bytes.
  ........
    r55101 | kristjan.jonsson | 2007-05-03 13:04:53 -0700 (Thu, 03 May 2007) | 2 lines

    Fix pcbuild8 after recent overhaul:  Added the version resource to python26.dll. Adjust stacksize to 2Mb and made large address aware for 32 bits, and set stacksize to 3Mb for 64 bits.
    Todo:  Set .dll optimized load addresses, and side-by-side packaging of the python26.dll.
  ........
    r55102 | kristjan.jonsson | 2007-05-03 13:09:56 -0700 (Thu, 03 May 2007) | 1 line

    Fix those parts in the testsuite that assumed that sys.maxint would cause overflow on x64.  Now the testsuite is well behaved on that platform.
  ........
    r55103 | kristjan.jonsson | 2007-05-03 13:27:03 -0700 (Thu, 03 May 2007) | 11 lines

    Fix problems in x64 build that were discovered by the testsuite:
    - Reenable modules on x64 that had been disabled aeons ago for Itanium.
    - Cleared up confusion about compilers for 64 bit windows.  There is only Itanium and x64.  Added macros MS_WINI64 and MS_WINX64 for those rare cases where it matters, such as the disabling of modules above.
    - Set target platform (_WIN32_WINNT and WINVER) to 0x0501 (XP) for x64, and 0x0400 (NT 4.0) otherwise, which are the targeted minimum platforms.
    - Fixed thread_nt.h.  The emulated InterlockedCompareExchange function didn?\194?\180t work on x64, probaby due to the lack of a "volatile" specifier.  Anyway, win95 is no longer a target platform.
    - Itertools module used wrong constant to check for overflow in count()
    - PyInt_AsSsize_t couldn't deal with attribute error when accessing the __long__ member.
    - PyLong_FromSsize_t() incorrectly specified that the operand were unsigned.

    With these changes, the x64 passes the testsuite, for those modules present.
  ........
    r55107 | kristjan.jonsson | 2007-05-03 17:25:08 -0700 (Thu, 03 May 2007) | 1 line

    Revert compiler comment to AMD64 for x64/AMD64 builds.
  ........
    r55115 | thomas.heller | 2007-05-04 00:14:39 -0700 (Fri, 04 May 2007) | 4 lines

    Fix some ctypes test crashes, when running with a debug Python
    version on win64 by using proper argtypes and restype function
    attributes.
  ........
    r55117 | thomas.heller | 2007-05-04 01:20:41 -0700 (Fri, 04 May 2007) | 4 lines

    On 64-bit Windows, ffi_arg must be 8 bytes long.  This fixes the
    remaining crashes in the ctypes tests, when functions return float or
    double types.
  ........
    r55120 | kristjan.jonsson | 2007-05-04 08:48:15 -0700 (Fri, 04 May 2007) | 1 line

    Update the pcbuild8 solution.  Straightened out the _ctypes project by using a .vsproj file and a masm64.rules file to avoid redundancy
  ........
    r55121 | kristjan.jonsson | 2007-05-04 10:28:06 -0700 (Fri, 04 May 2007) | 1 line

    Minor fix of PCBuild8/_ctypes vcproj, moving include dir into the .vsprops file.
  ........
    r55129 | thomas.heller | 2007-05-04 12:54:22 -0700 (Fri, 04 May 2007) | 3 lines

    Do not truncate 64-bit pointers to 32-bit integers.

    Fixes SF #1703286, will backport to release25-maint.
  ........
    r55131 | thomas.heller | 2007-05-04 12:56:32 -0700 (Fri, 04 May 2007) | 1 line

    Oops, these tests do not run on Windows CE.
  ........
    r55140 | brett.cannon | 2007-05-04 18:34:02 -0700 (Fri, 04 May 2007) | 2 lines

    Deprecate BaseException.message as per PEP 352.
  ........
    r55154 | georg.brandl | 2007-05-05 11:55:37 -0700 (Sat, 05 May 2007) | 2 lines

    Bug #1713535: typo in logging example.
  ........
    r55158 | vinay.sajip | 2007-05-06 10:53:37 -0700 (Sun, 06 May 2007) | 1 line

    Updates of recent changes to logging.
  ........
    r55165 | neal.norwitz | 2007-05-07 00:02:26 -0700 (Mon, 07 May 2007) | 1 line

    Verify changes to the trunk go to the normal checkins list
  ........
    r55169 | kristjan.jonsson | 2007-05-07 09:46:54 -0700 (Mon, 07 May 2007) | 1 line

    As per Armin Rigo's suggestion, remove special handing from intobject.c to deal with the peculiarities of classobject's implementation of the number protocol.  The nb_long method of classobject now falls back to nb_int if there is no __long__ attribute present.
  ........
    r55197 | collin.winter | 2007-05-08 21:14:36 -0700 (Tue, 08 May 2007) | 9 lines

    Fix a bug in test.test_support.open_urlresource().

    If the call to requires() doesn't precede the filesystem check, we get the following situation:
    1. ./python Lib/test/regrtest.py test_foo # test needs urlfetch, not enabled, so skipped
    2. ./python Lib/test/regrtest.py -u urlfetch test_foo # test runs
    3. ./python Lib/test/regrtest.py test_foo # test runs (!)

    By moving the call to requires() *before* the filesystem check, the fact that fetched files are cached on the local disk becomes an implementation detail, rather than a semantics-changing point of note.
  ........
    r55198 | neal.norwitz | 2007-05-08 23:43:15 -0700 (Tue, 08 May 2007) | 1 line

    Add markup for True/False.  Will backport
  ........
    r55205 | walter.doerwald | 2007-05-09 11:10:47 -0700 (Wed, 09 May 2007) | 4 lines

    Backport checkin:
    Fix a segfault when b"" was passed to b2a_qp() -- it was using strchr()
    instead of memchr().
  ........
    r55241 | neal.norwitz | 2007-05-10 22:55:15 -0700 (Thu, 10 May 2007) | 6 lines

    Don't ever report a failure when the sum of the reference count differences
    are zero.  This should help reduce the false positives.

    The message about references leaking is maintained to provide as much
    info as possible rather than simply suppressing the message at the source.
  ........
    r55242 | neal.norwitz | 2007-05-10 23:23:01 -0700 (Thu, 10 May 2007) | 1 line

    Fix typo in docstring (the module is popen2, not 3).
  ........
    r55244 | neal.norwitz | 2007-05-10 23:56:52 -0700 (Thu, 10 May 2007) | 1 line

    Remove trailing whitespace in docstring
  ........
    r55245 | neal.norwitz | 2007-05-10 23:57:33 -0700 (Thu, 10 May 2007) | 1 line

    Deprecate os.popen* and popen2 module in favor of the subprocess module.
  ........
    r55247 | neal.norwitz | 2007-05-11 00:13:30 -0700 (Fri, 11 May 2007) | 1 line

    Deprecate os.popen* and popen2 module in favor of the subprocess module. (forgot the doc)
  ........
    r55253 | georg.brandl | 2007-05-11 02:41:37 -0700 (Fri, 11 May 2007) | 3 lines

    Remove an XXX that is unnecessary.
  ........
    r55258 | georg.brandl | 2007-05-11 04:04:26 -0700 (Fri, 11 May 2007) | 2 lines

    Patch #1714700: clarify os.linesep vs. tfiles opened in text mode.
     (backport)
  ........
    r55259 | georg.brandl | 2007-05-11 04:43:56 -0700 (Fri, 11 May 2007) | 2 lines

    Update DDJ link.
  ........
    r55273 | raymond.hettinger | 2007-05-11 10:59:59 -0700 (Fri, 11 May 2007) | 1 line

    Better tests for posixpath.commonprefix
  ........
    r55287 | georg.brandl | 2007-05-12 14:06:41 -0700 (Sat, 12 May 2007) | 2 lines

    Bug #1046945: document SWIG options of distutils.
  ........
    r55290 | georg.brandl | 2007-05-13 01:04:07 -0700 (Sun, 13 May 2007) | 2 lines

    Add bz2 to content encodings.
  ........
    r55297 | neal.norwitz | 2007-05-13 13:45:05 -0700 (Sun, 13 May 2007) | 3 lines

    Remove Amoeba doc which was removed in version 1.0! according to Misc/HISTORY.
    Hopefully Guido won't shed a tear. :-)
  ........
    r55298 | neal.norwitz | 2007-05-13 13:54:19 -0700 (Sun, 13 May 2007) | 1 line

    Remove references to stdwin which was removed long ago.
  ........
    r55299 | neal.norwitz | 2007-05-13 14:13:42 -0700 (Sun, 13 May 2007) | 3 lines

    Remove support for freebsd[23] which haven't been released since 2000
    or earlier.  http://www.freebsd.org/releases/index.html
  ........
    r55320 | raymond.hettinger | 2007-05-14 13:52:31 -0700 (Mon, 14 May 2007) | 1 line

    Small speedup.
  ........
................
2007-05-14 22:51:27 +00:00
Thomas Wouters 49fd7fa443 Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
number of tests, all because of the codecs/_multibytecodecs issue described
here (it's not a Py3K issue, just something Py3K discovers):
http://mail.python.org/pipermail/python-dev/2006-April/064051.html

Hye-Shik Chang promised to look for a fix, so no need to fix it here. The
tests that are expected to break are:

test_codecencodings_cn
test_codecencodings_hk
test_codecencodings_jp
test_codecencodings_kr
test_codecencodings_tw
test_codecs
test_multibytecodec

This merge fixes an actual test failure (test_weakref) in this branch,
though, so I believe merging is the right thing to do anyway.
2006-04-21 10:40:58 +00:00
Martin v. Löwis 39f59b089d Remove MALLOC_ZERO_RETURNS_NULL. 2002-11-23 09:13:40 +00:00
Mark Hammond 91a681debf Excise DL_EXPORT from Include.
Thanks to Skip Montanaro and Kalle Svensson for the patches.
2002-08-12 07:21:58 +00:00
Tim Peters 8b078f95e0 Moving pymalloc along.
As threatened, PyMem_{Free, FREE} also invoke the object deallocator now
when pymalloc is enabled (well, it does when pymalloc isn't enabled too,
but in that case "the object deallocator" is plain free()).

This is maximally backward-compatible, but it leaves a bitter aftertaste.

Also massive reworking of comments.
2002-04-28 04:11:46 +00:00
Tim Peters 51e7f5caba Moving pymalloc along.
+ Redirect PyMem_{Del, DEL} to the object allocator's free() when
  pymalloc is enabled.  Needed so old extensions can continue to
  mix PyObject_New with PyMem_DEL.

+ This implies that pgen needs to be able to see the PyObject_XYZ
  declarations too.  pgenheaders.h now includes Python.h.  An
  implication is that I expect obmalloc.o needs to get linked into
  pgen on non-Windows boxes.

+ When PYMALLOC_DEBUG is defined, *all* Py memory API functions
  now funnel through the debug allocator wrapper around pymalloc.
  This is the default in a debug build.

+ That caused compile.c to fail:  it indirectly mixed PyMem_Malloc
  with raw platform free() in one place.  This is verbotten.
2002-04-22 02:33:27 +00:00
Tim Peters af3e8de580 First stab at rationalizing the PyMem_ API. Mixing PyObject_xyz with
PyMem_{Del, DEL} doesn't work yet (compilation problems).

pyport.h:  _PyMem_EXTRA is gone.

pmem.h:  Repaired comments.  PyMem_{Malloc, MALLOC} and
PyMem_{Realloc, REALLOC} now make the same x-platform guarantees when
asking for 0 bytes, and when passing a NULL pointer to the latter.

object.c:  PyMem_{Malloc, Realloc} just call their macro versions
now, since the latter take care of the x-platform 0 and NULL stuff
by themselves now.

pypcre.c, grow_stack():  So sue me.  On two lines, this called
PyMem_RESIZE to grow a "const" area.  It's not legit to realloc a
const area, so the compiler warned given the new expansion of
PyMem_RESIZE.  It would have gotten the same warning before if it
had used PyMem_Resize() instead; the older macro version, but not the
function version, silently cast away the constness.  IMO that was a wrong
thing to do, and the docs say the macro versions of PyMem_xyz are
deprecated anyway.  If somebody else is resizing const areas with the
macro spelling, they'll get a warning when they recompile now too.
2002-04-12 07:22:56 +00:00
Neil Schemenauer 20d0a1a071 Remove PyMalloc_* symbols. PyObject_Malloc now uses pymalloc if
it's enabled.
2002-04-12 02:39:18 +00:00
Tim Peters 7ccfadf3a8 New PYMALLOC_DEBUG function void _PyMalloc_DebugDumpStats(void).
This displays stats about the # of arenas, pools, blocks and bytes, to
stderr, both used and reserved but unused.

CAUTION:  Because PYMALLOC_DEBUG is on, the debug malloc routine adds
16 bytes to each request.  This makes each block appear two size classes
higher than it would be if PYMALLOC_DEBUG weren't on.

So far, playing with this confirms the obvious:  there's a lot of activity
in the "small dict" size class, but nothing in the core makes any use of
the 8-byte or 16-byte classes.
2002-04-01 06:04:21 +00:00
Tim Peters d1139e043c PYMALLOC_DEBUG routines: The "check API family" gimmick was going nowhere
fast, and just cluttered the code.  Get rid of it for now.  If a compelling
case can be made for it, easy to restore it later.
2002-03-28 07:32:11 +00:00
Tim Peters ddea208be9 Give Python a debug-mode pymalloc, much as sketched on Python-Dev.
When WITH_PYMALLOC is defined, define PYMALLOC_DEBUG to enable the debug
allocator.  This can be done independent of build type (release or debug).
A debug build automatically defines PYMALLOC_DEBUG when pymalloc is
enabled.  It's a detected error to define PYMALLOC_DEBUG when pymalloc
isn't enabled.

Two debugging entry points defined only under PYMALLOC_DEBUG:

+ _PyMalloc_DebugCheckAddress(const void *p) can be used (e.g., from gdb)
  to sanity-check a memory block obtained from pymalloc.  It sprays
  info to stderr (see next) and dies via Py_FatalError if the block is
  detectably damaged.

+ _PyMalloc_DebugDumpAddress(const void *p) can be used to spray info
  about a debug memory block to stderr.

A tiny start at implementing "API family" checks isn't good for
anything yet.

_PyMalloc_DebugRealloc() has been optimized to do little when the new
size is <= old size.  However, if the new size is larger, it really
can't call the underlying realloc() routine without either violating its
contract, or knowing something non-trivial about how the underlying
realloc() works.  A memcpy is always done in this case.

This was a disaster for (and only) one of the std tests:  test_bufio
creates single text file lines up to a million characters long.  On
Windows, fileobject.c's get_line() uses the horridly funky
getline_via_fgets(), which keeps growing and growing a string object
hoping to find a newline.  It grew the string object 1000 bytes each
time, so for a million-character string it took approximately forever
(I gave up after a few minutes).

So, also:

fileobject.c, getline_via_fgets():  When a single line is outrageously
long, grow the string object at a mildly exponential rate, instead of
just 1000 bytes at a time.

That's enough so that a debug-build test_bufio finishes in about 5 seconds
on my Win98SE box.  I'm curious to try this on Win2K, because it has very
different memory behavior than Win9X, and test_bufio always took a factor
of 10 longer to complete on Win2K.  It *could* be that the endless
reallocs were simply killing it on Win2K even in the release build.
2002-03-23 10:03:50 +00:00
Tim Peters fbb556df15 Arrange to export the _PyMalloc_{Malloc, Realloc, Free} entry points. On
Windows some modules are considered (by me, and I don't care what anyone
else thinks about this <wink>) to be part of "the core" despite that they
happen to be compiled into separate DLLs (the "to DLL or not to DLL?"
question on Windows is nearly arbitrary).  Making the pymalloc entry
points available to them allows the Windows build to complete without
incident when WITH_PYMALLOC is #define'd.

Note that this isn't unprecedented.  Other "private API" functions we
export include _PySequence_IterSearch, _PyEval_SliceIndex, _PyCodec_Lookup,
_Py_ZeroStruct, _Py_TrueStruct, _PyLong_New and _PyModule_Clear.
2002-03-20 04:02:31 +00:00