Commit Graph

124 Commits

Author SHA1 Message Date
Victor Stinner 45b9e6a61f
gh-108765: Move standard includes to Python.h (#108769)
* Move <ctype.h>, <limits.h> and <stdarg.h> standard includes to
  Python.h.
* Move "pystats.h" include from object.h to Python.h.
* Remove redundant "pymem.h" include in objimpl.h and "pyport.h"
  include in pymem.h; Python.h already includes them earlier.
* Remove redundant <wchar.h> include in unicodeobject.h; Python.h
  already includes it.
* Move _SGI_MP_SOURCE define from Python.h to pyport.h.
* pycore_condvar.h includes explicitly <unistd.h> for the
  _POSIX_THREADS macro.
2023-09-01 21:03:20 +02:00
Jacob Bower cbd3fbfb6e
gh-102013: Add PyUnstable_GC_VisitObjects (#102014) 2023-03-14 01:35:54 +00:00
Victor Stinner 44c8e68b8c
gh-87347: Fix PyObject_NEW() regression (#94234)
Don't add parenthesis around the type parameter.

Add unit tests on PyObject_NEW() and similar functions.
2022-06-26 12:40:17 +02:00
Victor Stinner 7ad6f74fcf
gh-87347: Add parenthesis around macro arguments (#93915)
Add unit test on Py_MEMBER_SIZE() and some other macros.
2022-06-20 16:04:52 +02:00
Victor Stinner fbd5539a54
gh-92135: Rename _Py_reinterpret_cast() to _Py_CAST() (#92230)
Rename also _Py_static_cast() to _Py_STATIC_CAST().
2022-05-03 16:37:06 +02:00
Victor Stinner 031397063e
gh-92135: Fix _Py_reinterpret_cast() for const (#92138)
Fix C++ compiler warnings on cast macros, like _PyObject_CAST(), when
casting a constant expression to a non constant type: use
const_cast<> in C++.

* In C++, Py_SAFE_DOWNCAST() now uses static_cast<> rather than
  reinterpret_cast<>.
* Add tests to the _testcppext C++ extension.
* test_cppext no longer captures stdout in verbose mode.
2022-05-02 17:07:00 +02:00
Victor Stinner 29e2245ad5
gh-91320: Add _Py_reinterpret_cast() macro (#91959)
Fix C++ compiler warnings about "old-style cast"
(g++ -Wold-style-cast) in the Python C API.  Use C++
reinterpret_cast<> and static_cast<> casts when the Python C API is
used in C++.

Example of fixed warning:

    Include/object.h:107:43: error: use of old-style cast to
    ‘PyObject*’ {aka ‘struct _object*’} [-Werror=old-style-cast]
    #define _PyObject_CAST(op) ((PyObject*)(op))

Add _Py_reinterpret_cast() and _Py_static_cast() macros.
2022-04-27 10:40:57 +02:00
Victor Stinner 7076bef8ba
bpo-45434: Remove useless space in includes (GH-28963)
Micro-optimize spaces!
2021-10-15 01:50:04 +02:00
Victor Stinner 645ed62fb4
bpo-43774: Remove unused PYMALLOC_DEBUG macro (GH-25711)
Enhance also the documentation of debug hooks on memory allocators.
2021-04-29 10:47:47 +02:00
scoder 3cc481b9de
bpo-28254: Add a C-API for controlling the GC state (GH-25687)
Add new C-API functions to control the state of the garbage collector:
PyGC_Enable(), PyGC_Disable(), PyGC_IsEnabled(),
corresponding to the functions in the gc module.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-28 18:12:16 +02:00
Victor Stinner 32bd68c839
bpo-42519: Replace PyObject_MALLOC() with PyObject_Malloc() (GH-23587)
No longer use deprecated aliases to functions:

* Replace PyObject_MALLOC() with PyObject_Malloc()
* Replace PyObject_REALLOC() with PyObject_Realloc()
* Replace PyObject_FREE() with PyObject_Free()
* Replace PyObject_Del() with PyObject_Free()
* Replace PyObject_DEL() with PyObject_Free()
2020-12-01 10:37:39 +01:00
Victor Stinner 00d7abd7ef
bpo-42519: Replace PyMem_MALLOC() with PyMem_Malloc() (GH-23586)
No longer use deprecated aliases to functions:

* Replace PyMem_MALLOC() with PyMem_Malloc()
* Replace PyMem_REALLOC() with PyMem_Realloc()
* Replace PyMem_FREE() with PyMem_Free()
* Replace PyMem_Del() with PyMem_Free()
* Replace PyMem_DEL() with PyMem_Free()

Modify also the PyMem_DEL() macro to use directly PyMem_Free().
2020-12-01 09:56:42 +01:00
Victor Stinner 04fc4f2a46
bpo-40989: PyObject_INIT() becomes an alias to PyObject_Init() (GH-20901)
The PyObject_INIT() and PyObject_INIT_VAR() macros become aliases to,
respectively, PyObject_Init() and PyObject_InitVar() functions.

Rename _PyObject_INIT() and _PyObject_INIT_VAR() static inline
functions to, respectively, _PyObject_Init() and _PyObject_InitVar(),
and move them to pycore_object.h. Remove their return value:
their return type becomes void.

The _datetime module is now built with the Py_BUILD_CORE_MODULE macro
defined.

Remove an outdated comment on _Py_tracemalloc_config.
2020-06-16 01:28:07 +02:00
Pablo Galindo f13072b8a8
bpo-40241: Add PyObject_GC_IsTracked and PyObject_GC_IsFinalized to the public C-API (GH-19461)
Add the functions PyObject_GC_IsTracked and PyObject_GC_IsFinalized to the public API to allow to query if Python objects are being currently tracked or have been already finalized by the garbage collector respectively.
2020-04-11 01:21:54 +01:00
Victor Stinner 9205520d8c
bpo-40170: PyObject_NEW() becomes an alias to PyObject_New() (GH-19379)
The PyObject_NEW() macro becomes an alias to the PyObject_New()
macro, and the PyObject_NEW_VAR() macro becomes an alias to the
PyObject_NewVar() macro, to hide implementation details. They no
longer access directly the PyTypeObject.tp_basicsize member.

Exclude _PyObject_SIZE() and _PyObject_VAR_SIZE() macros from
the limited C API.

Replace PyObject_NEW() with PyObject_New() and replace
PyObject_NEW_VAR() with PyObject_NewVar().
2020-04-08 00:38:15 +02:00
Victor Stinner f58bd7c169
bpo-39542: Make PyObject_INIT() opaque in limited C API (GH-18363)
In the limited C API, PyObject_INIT() and PyObject_INIT_VAR() are now
defined as aliases to PyObject_Init() and PyObject_InitVar() to make
their implementation opaque. It avoids to leak implementation details
in the limited C API.

Exclude the following functions from the limited C API, move them
from object.h to cpython/object.h:

* _Py_NewReference()
* _Py_ForgetReference()
* _PyTraceMalloc_NewReference()
* _Py_GetRefTotal()
2020-02-05 13:12:19 +01:00
Eddie Elizondo 364f0b0f19 bpo-35810: Incref heap-allocated types in PyObject_Init (GH-11661)
* Incref heap-allocated types in PyObject_Init
* Add documentation and porting notes to What's New
2019-03-27 12:52:18 +01:00
Victor Stinner e421106b9e
bpo-35134: Create Include/cpython/ subdirectory (GH-10624)
Include/*.h should be the "portable Python API", whereas
Include/cpython/*.h should be the "CPython API": CPython
implementation details.

Changes:

* Create Include/cpython/ subdirectory
* "make install" now creates $prefix/include/cpython and copy
  Include/cpython/* to $prefix/include/cpython
* Create Include/cpython/objimpl.h: move objimpl.h code
  surrounded by "#ifndef Py_LIMITED_API" to cpython/objimpl.h.
* objimpl.h now includes cpython/objimpl.h
* Windows installer (MSI) now also install Include/ subdirectories:
  Include/cpython/ and Include/internal/.
2018-11-23 17:00:00 +01:00
Victor Stinner b509d52083
bpo-35059: PyObject_INIT() casts to PyObject* (GH-10674)
PyObject_INIT() and PyObject_INIT_VAR() now cast their first argument
to PyObject*, as done in Python 3.7.

Revert partially commit b4435e20a9.
2018-11-23 14:27:38 +01:00
Victor Stinner 2ff8fb7639
bpo-35059: Add _PyObject_CAST() macro (GH-10645)
Add _PyObject_CAST() and _PyVarObject_CAST() macros to cast argument
to PyObject* and PyVarObject* properly.
2018-11-22 02:57:29 +01:00
Victor Stinner bcda8f1d42
bpo-35081: Add Include/internal/pycore_object.h (GH-10640)
Move _PyObject_GC_TRACK() and _PyObject_GC_UNTRACK() from
Include/objimpl.h to Include/internal/pycore_object.h.
2018-11-21 22:27:47 +01:00
Victor Stinner 3e21ad1a25
bpo-35081: Move _PyGC_FINALIZED() back to C API (GH-10626)
Partially revert commit 1a6be91e6f,
move back PyGC API from the internal API to the C API:

* _PyGCHead_NEXT(g), _PyGCHead_SET_NEXT(g, p)
* _PyGCHead_PREV(g), _PyGCHead_SET_PREV(g, p)
* _PyGCHead_FINALIZED(g), _PyGCHead_SET_FINALIZED(g)
* _PyGC_FINALIZED(o), _PyGC_SET_FINALIZED(o)
* _PyGC_PREV_MASK_FINALIZED
* _PyGC_PREV_MASK_COLLECTING
* _PyGC_PREV_SHIFT
* _PyGC_PREV_MASK

_PyObject_GC_TRACK(o) and _PyObject_GC_UNTRACK(o) remain in the
internal API.
2018-11-21 00:43:09 +01:00
Victor Stinner 1a6be91e6f
bpo-35081: Make some _PyGC macros internal (GH-10507)
* Move "GC" macros together:

  * PyObject_IS_GC()
  * _Py_AS_GC()
  * _PyObject_GC_IS_TRACKED()
  * _PyObject_GC_MAY_BE_TRACKED()

* Mark other GC macros as internal (#ifdef Py_BUILD_CORE):

  * _PyGCHead_NEXT(g), _PyGCHead_SET_NEXT(g, p)
  * _PyGCHead_PREV(g), _PyGCHead_SET_PREV(g, p)
  * _PyGCHead_FINALIZED(g), _PyGCHead_SET_FINALIZED(g)
  * _PyGC_FINALIZED(o), _PyGC_SET_FINALIZED(o)
  * _PyObject_GC_TRACK(o), _PyObject_GC_UNTRACK(o)
  * _PyGC_PREV_MASK_FINALIZED
  * _PyGC_PREV_MASK_COLLECTING
  * _PyGC_PREV_SHIFT
  * _PyGC_PREV_MASK

* Replace _PyGC_generation0 with _PyRuntime.gc.generation0
* _queuemodule.c: replace _PyObject_GC_UNTRACK()
  with with PyObject_GC_UnTrack()
* Document that  _PyObject_GC_TRACK() _PyObject_GC_UNTRACK() macros
  have been removed from the public C API.
2018-11-13 12:52:18 +01:00
Victor Stinner 542497aa9f
bpo-35059: Remove Py_STATIC_INLINE() macro (GH-10216)
"static inline" should be used directly. Forcing the compiler to
inline is not recommended.
2018-10-29 14:49:24 +01:00
Victor Stinner b4435e20a9
bpo-35059: Convert PyObject_INIT() to function (GH-10077)
* Convert PyObject_INIT() and PyObject_INIT_VAR() macros to static
  inline functions.
* Fix usage of these functions: cast to PyObject* or PyVarObject*.
2018-10-26 14:35:00 +02:00
INADA Naoki 5ac9e6eee5
bpo-33597: Reduce PyGC_Head size (GH-7043) 2018-07-10 17:19:53 +09:00
Christian Tismer ea62ce7f4f bpo-33738: Fix macros which contradict PEP 384 (GH-7477)
During development of the limited API support for PySide,
we saw an error in a macro that accessed a type field.

This patch fixes the 7 errors in the Python headers.
Macros which were not written as capitals were implemented
as function.

To do the necessary analysis again, a script was included that
parses all headers and looks for "->tp_" in serctions which can
be reached with active limited API.

It is easily possible to call this script as a test.

Error listing:

../../Include/objimpl.h:243
#define PyObject_IS_GC(o) (PyType_IS_GC(Py_TYPE(o)) && \
    (Py_TYPE(o)->tp_is_gc == NULL || Py_TYPE(o)->tp_is_gc(o)))
Action: commented only

../../Include/objimpl.h:362
#define PyType_SUPPORTS_WEAKREFS(t) ((t)->tp_weaklistoffset > 0)
Action: commented only

../../Include/objimpl.h:364
#define PyObject_GET_WEAKREFS_LISTPTR(o) \
    ((PyObject **) (((char *) (o)) + Py_TYPE(o)->tp_weaklistoffset))
Action: commented only

../../Include/pyerrors.h:143
#define PyExceptionClass_Name(x) \
     ((char *)(((PyTypeObject*)(x))->tp_name))
Action: implemented function

../../Include/abstract.h:593
#define PyIter_Check(obj) \
    ((obj)->ob_type->tp_iternext != NULL && \
     (obj)->ob_type->tp_iternext != &_PyObject_NextNotImplemented)
Action: implemented function

../../Include/abstract.h:713
#define PyIndex_Check(obj)                              \
    ((obj)->ob_type->tp_as_number != NULL &&            \
     (obj)->ob_type->tp_as_number->nb_index != NULL)
Action: implemented function

../../Include/abstract.h:924
#define PySequence_ITEM(o, i)\
    ( Py_TYPE(o)->tp_as_sequence->sq_item(o, i) )
Action: commented only
2018-06-09 14:32:25 -04:00
Alexey 517da1e58f Fix typo in Include/objimpl.h, the word "has" was missing (GH-5568)
It now reads: ...be aware that Python has no control over...
2018-02-06 16:07:30 -07:00
Victor Stinner 6bf992a1ac
bpo-32030: Add pymain_get_global_config() (#4735)
* Py_Main() now starts by reading Py_xxx configuration variables to
  only work on its own private structure, and then later writes back
  the configuration into these variables.
* Replace Py_GETENV() with pymain_get_env_var() which ignores empty
  variables.
* Add _PyCoreConfig.dump_refs
* Add _PyCoreConfig.malloc_stats
* _PyObject_DebugMallocStats() is now responsible to check if debug
  hooks are installed. The function returns 1 if stats were written,
  or 0 if the hooks are disabled. Mark _PyMem_PymallocEnabled() as
  static.
2017-12-06 17:26:10 +01:00
Serhiy Storchaka 34d0ac8027 Issue #29058: All stable API extensions added after Python 3.2 are now
available only when Py_LIMITED_API is set to the PY_VERSION_HEX value of
the minimum Python version supporting this API.
2016-12-27 14:57:39 +02:00
Serhiy Storchaka 9fab79bcb5 Issue #26900: Excluded underscored names and other private API from limited API. 2016-09-11 11:03:14 +03:00
Łukasz Langa fef7e94fa1 Don't run garbage collection on interpreter exit if it was explicitly disabled
by the user.
2016-09-09 21:47:46 -07:00
Victor Stinner db067af12a Issue #21233: Add new C functions: PyMem_RawCalloc(), PyMem_Calloc(),
PyObject_Calloc(), _PyObject_GC_Calloc(). bytes(int) and bytearray(int) are now
using ``calloc()`` instead of ``malloc()`` for large objects which is faster
and use less memory (until the bytearray buffer is filled with data).
2014-05-02 22:31:14 +02:00
Benjamin Peterson bdc4b02985 cast negative numbers to size_t before shifting them (#20929) 2014-03-14 20:15:29 -05:00
Serhiy Storchaka 46e1ce214b Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 20:17:03 +03:00
Serhiy Storchaka 9594942716 Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 19:40:23 +03:00
Antoine Pitrou 796564c27b Issue #18112: PEP 442 implementation (safe object finalization). 2013-07-30 19:59:21 +02:00
Victor Stinner 0507bf56f0 Issue #3329: Implement the PEP 445
Add new enum:

* PyMemAllocatorDomain

Add new structures:

* PyMemAllocator
* PyObjectArenaAllocator

Add new functions:

* PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree()
* PyMem_GetAllocator(), PyMem_SetAllocator()
* PyObject_GetArenaAllocator(), PyObject_SetArenaAllocator()
* PyMem_SetupDebugHooks()

Changes:

* PyMem_Malloc()/PyObject_Realloc() now always call malloc()/realloc(), instead
  of calling PyObject_Malloc()/PyObject_Realloc() in debug mode.
* PyObject_Malloc()/PyObject_Realloc() now falls back to
  PyMem_Malloc()/PyMem_Realloc() for allocations larger than 512 bytes.
* Redesign debug checks on memory block allocators as hooks, instead of using C
  macros
2013-07-07 02:05:46 +02:00
Victor Stinner 36f01ad9ac Revert changeset 6661a8154eb3: Issue #3329: Add new APIs to customize memory allocators
The new API require more discussion.
2013-06-15 03:37:01 +02:00
Victor Stinner 4d7056258b Issue #3329: Add new APIs to customize memory allocators
* Add a new PyMemAllocators structure
* New functions:

  - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree(): GIL-free memory
    allocator functions
  - PyMem_GetRawAllocators(), PyMem_SetRawAllocators()
  - PyMem_GetAllocators(), PyMem_SetAllocators()
  - PyMem_SetupDebugHooks()
  - _PyObject_GetArenaAllocators(), _PyObject_SetArenaAllocators()

* Add unit test for PyMem_Malloc(0) and PyObject_Malloc(0)
* Add unit test for new get/set allocators functions
* PyObject_Malloc() now falls back on PyMem_Malloc() instead of malloc() if
  size is bigger than SMALL_REQUEST_THRESHOLD, and PyObject_Realloc() falls
  back on PyMem_Realloc() instead of realloc()
* PyMem_Malloc() and PyMem_Realloc() now always call malloc() and realloc(),
  instead of calling PyObject_Malloc() and PyObject_Realloc() in debug mode
2013-06-15 00:37:46 +02:00
Antoine Pitrou fef34e3186 Issue #17937: Try harder to collect cyclic garbage at shutdown. 2013-05-19 01:11:58 +02:00
Gregory P. Smith e348c8d154 Using 'long double' to force this structure to be worst case aligned is no
longer required as of Python 2.5+ when the gc_refs changed from an int (4
bytes) to a Py_ssize_t (8 bytes) as the minimum size is 16 bytes.

The use of a 'long double' triggered a warning by Clang trunk's
Undefined-Behavior Sanitizer as on many platforms a long double requires
16-byte alignment but the Python memory allocator only guarantees 8 byte
alignment.

So our code would allocate and use these structures with technically improper
alignment.  Though it didn't matter since the 'dummy' field is never used.
This silences that warning.

Spelunking into code history, the double was added in 2001 to force better
alignment on some platforms and changed to a long double in 2002 to appease
Tru64.  That issue should no loner be present since the upgrade from int to
Py_ssize_t where the minimum structure size increased to 16 (unless anyone
knows of a platform where ssize_t is 4 bytes?) or 24 bytes depending on if the
build uses 4 or 8 byte pointers.

We can probably get rid of the double and this union hack all together today.
That is a slightly more invasive change that can be left for later.

A more correct non-hacky alternative if any alignment issues are still found
would be to use a compiler specific alignment declaration on the structure and
determine which value to use at configure time.
2012-12-10 18:05:05 -08:00
Antoine Pitrou f9d0b1256f Issue #13390: New function :func:`sys.getallocatedblocks()` returns the number of memory blocks currently allocated.
Also, the ``-R`` option to regrtest uses this function to guard against memory allocation leaks.
2012-12-09 14:28:26 +01:00
Antoine Pitrou ca8aa4acf6 Issue #15144: Fix possible integer overflow when handling pointers as integer values, by using Py_uintptr_t instead of size_t.
Patch by Serhiy Storchaka.
2012-09-20 20:56:47 +02:00
David Malcolm 49526f48fc Issue #14785: Add sys._debugmallocstats() to help debug low-level memory allocation issues 2012-06-22 14:55:41 -04:00
Antoine Pitrou 23683ef26d Issue #10333: Remove ancient GC API, which has been deprecated since
Python 2.2.
2011-01-04 00:00:31 +00:00
Martin v. Löwis 4d0d471a80 Merge branches/pep-0384. 2010-12-03 20:14:31 +00:00
Antoine Pitrou f95a1b3c53 Recorded merge of revisions 81029 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

  Untabify C files. Will watch buildbots.
........
2010-05-09 15:52:27 +00:00
Kristján Valur Jónsson ae4cfb1bb3 http://bugs.python.org/issue6836
Merging revisions 75103,75104 from trunk to py3k
2009-09-28 13:45:02 +00:00
Antoine Pitrou 3a652b1d0a Merged revisions 70546 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r70546 | antoine.pitrou | 2009-03-23 19:41:45 +0100 (lun., 23 mars 2009) | 9 lines

  Issue #4688: Add a heuristic so that tuples and dicts containing only
  untrackable objects are not tracked by the garbage collector. This can
  reduce the size of collections and therefore the garbage collection overhead
  on long-running programs, depending on their particular use of datatypes.

  (trivia: this makes the "binary_trees" benchmark from the Computer Language
  Shootout 40% faster)
........
2009-03-23 18:52:06 +00:00