Commit Graph

99 Commits

Author SHA1 Message Date
INADA Naoki 5ac9e6eee5
bpo-33597: Reduce PyGC_Head size (GH-7043) 2018-07-10 17:19:53 +09:00
Christian Tismer ea62ce7f4f bpo-33738: Fix macros which contradict PEP 384 (GH-7477)
During development of the limited API support for PySide,
we saw an error in a macro that accessed a type field.

This patch fixes the 7 errors in the Python headers.
Macros which were not written as capitals were implemented
as function.

To do the necessary analysis again, a script was included that
parses all headers and looks for "->tp_" in serctions which can
be reached with active limited API.

It is easily possible to call this script as a test.

Error listing:

../../Include/objimpl.h:243
#define PyObject_IS_GC(o) (PyType_IS_GC(Py_TYPE(o)) && \
    (Py_TYPE(o)->tp_is_gc == NULL || Py_TYPE(o)->tp_is_gc(o)))
Action: commented only

../../Include/objimpl.h:362
#define PyType_SUPPORTS_WEAKREFS(t) ((t)->tp_weaklistoffset > 0)
Action: commented only

../../Include/objimpl.h:364
#define PyObject_GET_WEAKREFS_LISTPTR(o) \
    ((PyObject **) (((char *) (o)) + Py_TYPE(o)->tp_weaklistoffset))
Action: commented only

../../Include/pyerrors.h:143
#define PyExceptionClass_Name(x) \
     ((char *)(((PyTypeObject*)(x))->tp_name))
Action: implemented function

../../Include/abstract.h:593
#define PyIter_Check(obj) \
    ((obj)->ob_type->tp_iternext != NULL && \
     (obj)->ob_type->tp_iternext != &_PyObject_NextNotImplemented)
Action: implemented function

../../Include/abstract.h:713
#define PyIndex_Check(obj)                              \
    ((obj)->ob_type->tp_as_number != NULL &&            \
     (obj)->ob_type->tp_as_number->nb_index != NULL)
Action: implemented function

../../Include/abstract.h:924
#define PySequence_ITEM(o, i)\
    ( Py_TYPE(o)->tp_as_sequence->sq_item(o, i) )
Action: commented only
2018-06-09 14:32:25 -04:00
Alexey 517da1e58f Fix typo in Include/objimpl.h, the word "has" was missing (GH-5568)
It now reads: ...be aware that Python has no control over...
2018-02-06 16:07:30 -07:00
Victor Stinner 6bf992a1ac
bpo-32030: Add pymain_get_global_config() (#4735)
* Py_Main() now starts by reading Py_xxx configuration variables to
  only work on its own private structure, and then later writes back
  the configuration into these variables.
* Replace Py_GETENV() with pymain_get_env_var() which ignores empty
  variables.
* Add _PyCoreConfig.dump_refs
* Add _PyCoreConfig.malloc_stats
* _PyObject_DebugMallocStats() is now responsible to check if debug
  hooks are installed. The function returns 1 if stats were written,
  or 0 if the hooks are disabled. Mark _PyMem_PymallocEnabled() as
  static.
2017-12-06 17:26:10 +01:00
Serhiy Storchaka 34d0ac8027 Issue #29058: All stable API extensions added after Python 3.2 are now
available only when Py_LIMITED_API is set to the PY_VERSION_HEX value of
the minimum Python version supporting this API.
2016-12-27 14:57:39 +02:00
Serhiy Storchaka 9fab79bcb5 Issue #26900: Excluded underscored names and other private API from limited API. 2016-09-11 11:03:14 +03:00
Łukasz Langa fef7e94fa1 Don't run garbage collection on interpreter exit if it was explicitly disabled
by the user.
2016-09-09 21:47:46 -07:00
Victor Stinner db067af12a Issue #21233: Add new C functions: PyMem_RawCalloc(), PyMem_Calloc(),
PyObject_Calloc(), _PyObject_GC_Calloc(). bytes(int) and bytearray(int) are now
using ``calloc()`` instead of ``malloc()`` for large objects which is faster
and use less memory (until the bytearray buffer is filled with data).
2014-05-02 22:31:14 +02:00
Benjamin Peterson bdc4b02985 cast negative numbers to size_t before shifting them (#20929) 2014-03-14 20:15:29 -05:00
Serhiy Storchaka 46e1ce214b Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 20:17:03 +03:00
Serhiy Storchaka 9594942716 Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 19:40:23 +03:00
Antoine Pitrou 796564c27b Issue #18112: PEP 442 implementation (safe object finalization). 2013-07-30 19:59:21 +02:00
Victor Stinner 0507bf56f0 Issue #3329: Implement the PEP 445
Add new enum:

* PyMemAllocatorDomain

Add new structures:

* PyMemAllocator
* PyObjectArenaAllocator

Add new functions:

* PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree()
* PyMem_GetAllocator(), PyMem_SetAllocator()
* PyObject_GetArenaAllocator(), PyObject_SetArenaAllocator()
* PyMem_SetupDebugHooks()

Changes:

* PyMem_Malloc()/PyObject_Realloc() now always call malloc()/realloc(), instead
  of calling PyObject_Malloc()/PyObject_Realloc() in debug mode.
* PyObject_Malloc()/PyObject_Realloc() now falls back to
  PyMem_Malloc()/PyMem_Realloc() for allocations larger than 512 bytes.
* Redesign debug checks on memory block allocators as hooks, instead of using C
  macros
2013-07-07 02:05:46 +02:00
Victor Stinner 36f01ad9ac Revert changeset 6661a8154eb3: Issue #3329: Add new APIs to customize memory allocators
The new API require more discussion.
2013-06-15 03:37:01 +02:00
Victor Stinner 4d7056258b Issue #3329: Add new APIs to customize memory allocators
* Add a new PyMemAllocators structure
* New functions:

  - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree(): GIL-free memory
    allocator functions
  - PyMem_GetRawAllocators(), PyMem_SetRawAllocators()
  - PyMem_GetAllocators(), PyMem_SetAllocators()
  - PyMem_SetupDebugHooks()
  - _PyObject_GetArenaAllocators(), _PyObject_SetArenaAllocators()

* Add unit test for PyMem_Malloc(0) and PyObject_Malloc(0)
* Add unit test for new get/set allocators functions
* PyObject_Malloc() now falls back on PyMem_Malloc() instead of malloc() if
  size is bigger than SMALL_REQUEST_THRESHOLD, and PyObject_Realloc() falls
  back on PyMem_Realloc() instead of realloc()
* PyMem_Malloc() and PyMem_Realloc() now always call malloc() and realloc(),
  instead of calling PyObject_Malloc() and PyObject_Realloc() in debug mode
2013-06-15 00:37:46 +02:00
Antoine Pitrou fef34e3186 Issue #17937: Try harder to collect cyclic garbage at shutdown. 2013-05-19 01:11:58 +02:00
Gregory P. Smith e348c8d154 Using 'long double' to force this structure to be worst case aligned is no
longer required as of Python 2.5+ when the gc_refs changed from an int (4
bytes) to a Py_ssize_t (8 bytes) as the minimum size is 16 bytes.

The use of a 'long double' triggered a warning by Clang trunk's
Undefined-Behavior Sanitizer as on many platforms a long double requires
16-byte alignment but the Python memory allocator only guarantees 8 byte
alignment.

So our code would allocate and use these structures with technically improper
alignment.  Though it didn't matter since the 'dummy' field is never used.
This silences that warning.

Spelunking into code history, the double was added in 2001 to force better
alignment on some platforms and changed to a long double in 2002 to appease
Tru64.  That issue should no loner be present since the upgrade from int to
Py_ssize_t where the minimum structure size increased to 16 (unless anyone
knows of a platform where ssize_t is 4 bytes?) or 24 bytes depending on if the
build uses 4 or 8 byte pointers.

We can probably get rid of the double and this union hack all together today.
That is a slightly more invasive change that can be left for later.

A more correct non-hacky alternative if any alignment issues are still found
would be to use a compiler specific alignment declaration on the structure and
determine which value to use at configure time.
2012-12-10 18:05:05 -08:00
Antoine Pitrou f9d0b1256f Issue #13390: New function :func:`sys.getallocatedblocks()` returns the number of memory blocks currently allocated.
Also, the ``-R`` option to regrtest uses this function to guard against memory allocation leaks.
2012-12-09 14:28:26 +01:00
Antoine Pitrou ca8aa4acf6 Issue #15144: Fix possible integer overflow when handling pointers as integer values, by using Py_uintptr_t instead of size_t.
Patch by Serhiy Storchaka.
2012-09-20 20:56:47 +02:00
David Malcolm 49526f48fc Issue #14785: Add sys._debugmallocstats() to help debug low-level memory allocation issues 2012-06-22 14:55:41 -04:00
Antoine Pitrou 23683ef26d Issue #10333: Remove ancient GC API, which has been deprecated since
Python 2.2.
2011-01-04 00:00:31 +00:00
Martin v. Löwis 4d0d471a80 Merge branches/pep-0384. 2010-12-03 20:14:31 +00:00
Antoine Pitrou f95a1b3c53 Recorded merge of revisions 81029 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines

  Untabify C files. Will watch buildbots.
........
2010-05-09 15:52:27 +00:00
Kristján Valur Jónsson ae4cfb1bb3 http://bugs.python.org/issue6836
Merging revisions 75103,75104 from trunk to py3k
2009-09-28 13:45:02 +00:00
Antoine Pitrou 3a652b1d0a Merged revisions 70546 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r70546 | antoine.pitrou | 2009-03-23 19:41:45 +0100 (lun., 23 mars 2009) | 9 lines

  Issue #4688: Add a heuristic so that tuples and dicts containing only
  untrackable objects are not tracked by the garbage collector. This can
  reduce the size of collections and therefore the garbage collection overhead
  on long-running programs, depending on their particular use of datatypes.

  (trivia: this makes the "binary_trees" benchmark from the Computer Language
  Shootout 40% faster)
........
2009-03-23 18:52:06 +00:00
Christian Heimes 90aa7646af #1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT. 2007-12-19 02:45:37 +00:00
Neal Norwitz 30d1c51ac9 Remove _PyObject_Del 2007-08-19 22:48:23 +00:00
Martin v. Löwis 9f2e346911 Merged revisions 56467-56482 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

................
  r56477 | martin.v.loewis | 2007-07-21 09:04:38 +0200 (Sa, 21 Jul 2007) | 11 lines

  Merged revisions 56466-56476 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r56476 | martin.v.loewis | 2007-07-21 08:55:02 +0200 (Sa, 21 Jul 2007) | 4 lines

    PEP 3123: Provide forward compatibility with Python 3.0, while keeping
    backwards compatibility. Add Py_Refcnt, Py_Type, Py_Size, and
    PyVarObject_HEAD_INIT.
  ........
................
  r56478 | martin.v.loewis | 2007-07-21 09:47:23 +0200 (Sa, 21 Jul 2007) | 2 lines

  PEP 3123: Use proper C inheritance for PyObject.
................
  r56479 | martin.v.loewis | 2007-07-21 10:06:55 +0200 (Sa, 21 Jul 2007) | 3 lines

  Add longintrepr.h to Python.h, so that the compiler can
  see that PyFalse is really some kind of PyObject*.
................
  r56480 | martin.v.loewis | 2007-07-21 10:47:18 +0200 (Sa, 21 Jul 2007) | 2 lines

  Qualify SHIFT, MASK, BASE.
................
  r56482 | martin.v.loewis | 2007-07-21 19:10:57 +0200 (Sa, 21 Jul 2007) | 2 lines

  Correctly refer to _ob_next.
................
2007-07-21 17:22:18 +00:00
Guido van Rossum 3cf5b1eef9 Get rid of most of the flags (in tp_flags) that keep track of various
variations of the type struct and its attachments.  In Py3k, all type
structs have to have all fields -- no binary backwards compatibility.
Had to change the complex object to a new-style number!
2006-07-27 21:53:35 +00:00
Thomas Wouters 49fd7fa443 Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
number of tests, all because of the codecs/_multibytecodecs issue described
here (it's not a Py3K issue, just something Py3K discovers):
http://mail.python.org/pipermail/python-dev/2006-April/064051.html

Hye-Shik Chang promised to look for a fix, so no need to fix it here. The
tests that are expected to break are:

test_codecencodings_cn
test_codecencodings_hk
test_codecencodings_jp
test_codecencodings_kr
test_codecencodings_tw
test_codecs
test_multibytecodec

This merge fixes an actual test failure (test_weakref) in this branch,
though, so I believe merging is the right thing to do anyway.
2006-04-21 10:40:58 +00:00
Neal Norwitz 7b216c52e4 Make PyGC_Collect() use Py_ssize_t. 2006-03-04 20:01:53 +00:00
Martin v. Löwis 6db0e00d57 Change GC refcount to Py_ssize_t. 2006-03-01 16:56:25 +00:00
Martin v. Löwis 41290685f9 Change _PyObject_GC_Resize to expect Py_ssize_t. 2006-02-16 14:56:14 +00:00
Martin v. Löwis 18e165558b Merge ssize_t branch. 2006-02-15 17:27:45 +00:00
Barry Warsaw b941825fa2 Fix PyGC_Collect() to be exported from the built DLL on Windows. (Fix given
by Matt Messier).
2006-01-26 18:59:06 +00:00
Georg Brandl 08c02dbb85 [ 1243081 ] repair typos 2005-07-22 18:39:19 +00:00
Tim Peters eda29306b3 Formalize that the Py_VISIT macro requires that the tp_traverse
implementation it's used in must give its arguments specific names.
2004-07-15 04:05:59 +00:00
Jim Fulton aa6389e13b Documented the new Py_VISIT macro to simplify implementation of
tp_traverse handlers. (Tim made me do it. ;)
2004-07-14 19:08:17 +00:00
Guido van Rossum e13ddc9ec8 - New C API PyGC_Collect(), same as calling gc.collect().
- Call this in Py_Finalize().
- Expand the Misc/NEWS text on PY_LONG_LONG.
2003-04-17 17:29:22 +00:00
Mark Hammond 91a681debf Excise DL_EXPORT from Include.
Thanks to Skip Montanaro and Kalle Svensson for the patches.
2002-08-12 07:21:58 +00:00
Tim Peters 943382c8e5 Removed WITH_CYCLE_GC #ifdef-ery. Holes:
+ I'm not sure what to do about configure.in.  Left it alone.

+ Ditto pyexpat.c.  Fred or Martin will know what to do.
2002-07-07 03:59:34 +00:00
Tim Peters 6fc13d9595 Finished transitioning to using gc_refs to track gc objects' states.
This was mostly a matter of adding comments and light code rearrangement.
Upon untracking, gc_next is still set to NULL.  It's a cheap way to
provoke memory faults if calling code is insane.  It's also used in some
way by the trashcan mechanism.
2002-07-02 18:12:35 +00:00
Tim Peters ea405639bf Reserved another gc_refs value for untracked objects. Every live gc
object should now have a well-defined gc_refs value, with clear transitions
among gc_refs states.  As a result, none of the visit_XYZ traversal
callbacks need to check IS_TRACKED() anymore, and those tests were removed.
(They were already looking for objects with specific gc_refs states, and
the gc_refs state of an untracked object can no longer match any other
gc_refs state by accident.)
Added more asserts.
I expect that the gc_next == NULL indicator for an untracked object is
now redundant and can also be removed, but I ran out of time for this.
2002-07-02 00:52:30 +00:00
Neil Schemenauer b1094f0b1b _PyGC_generation0 is now a pointer 2002-05-04 05:36:06 +00:00
Tim Peters 8b078f95e0 Moving pymalloc along.
As threatened, PyMem_{Free, FREE} also invoke the object deallocator now
when pymalloc is enabled (well, it does when pymalloc isn't enabled too,
but in that case "the object deallocator" is plain free()).

This is maximally backward-compatible, but it leaves a bitter aftertaste.

Also massive reworking of comments.
2002-04-28 04:11:46 +00:00
Tim Peters 0e871188e8 _PyObject_DebugDumpStats: renamed to _PyObject_DebugMallocStats.
Added code to call this when PYMALLOC_DEBUG is enabled, and envar
PYTHONMALLOCSTATS is set, whenever a new arena is obtained and once
late in the Python shutdown process.
2002-04-13 08:29:14 +00:00
Tim Peters e9e7452505 First cut at repairing out-of-date comments; make alignment of macro defs
all the same within the #ifdef WITH_PYMALLOC block.
2002-04-12 05:21:34 +00:00
Neil Schemenauer 3e7b893899 Remove PyMalloc_* symbols. PyObject_Malloc now uses pymalloc if
it's enabled.

Allow PyObject_Del, PyObject_Free, and PyObject_GC_Del to be used as
function designators.  Provide source compatibility macros.

Make PyObject_GC_Track and PyObject_GC_UnTrack functions instead of
trivial macros wrapping functions.
2002-04-12 02:38:45 +00:00
Neil Schemenauer ef99723b66 Add _Py_AS_GC macro. It will be used by the trashcan code on object.c. 2002-03-28 21:06:16 +00:00
Neil Schemenauer ffd5399728 Make PyObject_{NEW,New,Del,DEL} always use the standard malloc (PyMem_*)
and not pymalloc.  Add the functions PyMalloc_New, PyMalloc_NewVar, and
PyMalloc_Del that will use pymalloc if it's enabled.   If pymalloc is
not enabled then they use the standard malloc (PyMem_*).
2002-03-22 15:25:18 +00:00