Commit Graph

368 Commits

Author SHA1 Message Date
Victor Stinner 0135598d72
bpo-40241: Add pycore_gc.h header file (GH-19494)
Move the PyGC_Head structure and the following private macros to the
internal C API:

* _PyGCHead_FINALIZED()
* _PyGCHead_NEXT()
* _PyGCHead_PREV()
* _PyGCHead_SET_FINALIZED()
* _PyGCHead_SET_NEXT()
* _PyGCHead_SET_PREV()
* _PyGC_FINALIZED()
* _PyGC_PREV_MASK
* _PyGC_PREV_MASK_COLLECTING
* _PyGC_PREV_MASK_FINALIZED
* _PyGC_PREV_SHIFT
* _PyGC_SET_FINALIZED()
* _PyObject_GC_IS_TRACKED()
* _PyObject_GC_MAY_BE_TRACKED()
* _Py_AS_GC(o)

Keep the private _PyGC_FINALIZED() macro in the public C API for
backward compatibility with Python 3.8: make it an alias to the new
PyObject_GC_IsFinalized() function.

Move the SIZEOF_PYGC_HEAD constant from _testcapi module to
_testinternalcapi module.
2020-04-13 11:38:42 +02:00
Pablo Galindo f13072b8a8
bpo-40241: Add PyObject_GC_IsTracked and PyObject_GC_IsFinalized to the public C-API (GH-19461)
Add the functions PyObject_GC_IsTracked and PyObject_GC_IsFinalized to the public API to allow to query if Python objects are being currently tracked or have been already finalized by the garbage collector respectively.
2020-04-11 01:21:54 +01:00
Inada Naoki 3a8c56295d
Revert "bpo-39087: Add _PyUnicode_GetUTF8Buffer()" (GH-18985)
* Revert "bpo-39087: Add _PyUnicode_GetUTF8Buffer() (GH-17659)"

This reverts commit c7ad974d34.

* Update unicodeobject.h
2020-03-14 15:59:27 +09:00
Inada Naoki c7ad974d34
bpo-39087: Add _PyUnicode_GetUTF8Buffer() (GH-17659)
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-03-14 12:43:18 +09:00
Victor Stinner 3f2f4fefca
bpo-39947: Move get_recursion_depth() to _testinternalcapi (GH-18974)
Move get_recursion_depth() function from _testcapi to
_testinternalcapi to avoid accessing PyThreadState attributes
directly in _testcapi.
2020-03-13 13:07:31 +01:00
Victor Stinner 3225b9f973
bpo-39877: Remove useless PyEval_InitThreads() calls (GH-18883)
Py_Initialize() calls PyEval_InitThreads() since Python 3.7. It's no
longer needed to call it explicitly.
2020-03-09 20:56:57 +01:00
Serhiy Storchaka e5ccc94bbb
bpo-38643: Raise SystemError instead of crashing when PyNumber_ToBase is called with invalid base. (GH-18863) 2020-03-09 20:03:38 +02:00
Serhiy Storchaka 28d0bcac8b
bpo-38913: Fix segfault in Py_BuildValue("(s#O)", ...) if entered with exception raised. (GH-18656) 2020-03-02 08:42:39 +02:00
Petr Viktorin ffd9753a94
bpo-39245: Switch to public API for Vectorcall (GH-18460)
The bulk of this patch was generated automatically with:

    for name in \
        PyObject_Vectorcall \
        Py_TPFLAGS_HAVE_VECTORCALL \
        PyObject_VectorcallMethod \
        PyVectorcall_Function \
        PyObject_CallOneArg \
        PyObject_CallMethodNoArgs \
        PyObject_CallMethodOneArg \
    ;
    do
        echo $name
        git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
    done

    old=_PyObject_FastCallDict
    new=PyObject_VectorcallDict
    git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"

and then cleaned up:

- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
2020-02-11 17:46:57 +01:00
Victor Stinner d2ec81a8c9
bpo-39573: Add Py_SET_TYPE() function (GH-18394)
Add Py_SET_TYPE() function to set the type of an object.
2020-02-07 09:17:07 +01:00
Victor Stinner daa9756cb6
bpo-39573: Use Py_TYPE() macro in Modules directory (GH-18393)
Replace direct access to PyObject.ob_type with Py_TYPE().
2020-02-07 03:37:06 +01:00
Victor Stinner c86a11221d
bpo-39573: Add Py_SET_REFCNT() function (GH-18389)
Add a Py_SET_REFCNT() function to set the reference counter of an
object.
2020-02-07 01:24:29 +01:00
Victor Stinner a93c51e3a8
bpo-39573: Use Py_REFCNT() macro (GH-18388)
Replace direct acccess to PyObject.ob_refcnt with usage of the
Py_REFCNT() macro.
2020-02-07 00:38:59 +01:00
Victor Stinner 49932fec62
bpo-39542: Simplify _Py_NewReference() (GH-18332)
* Remove _Py_INC_REFTOTAL and _Py_DEC_REFTOTAL macros: modify
  directly _Py_RefTotal.
* _Py_ForgetReference() is no longer defined if the Py_TRACE_REFS
  macro is not defined.
* Remove _Py_NewReference() implementation from object.c:
  unify the two implementations in object.h inline function.
* Fix Py_TRACE_REFS build: _Py_INC_TPALLOCS() macro has been removed.
2020-02-03 17:55:04 +01:00
Victor Stinner c6e5c1123b
bpo-39489: Remove COUNT_ALLOCS special build (GH-18259)
Remove:

* COUNT_ALLOCS macro
* sys.getcounts() function
* SHOW_ALLOC_COUNT code in listobject.c
* SHOW_TRACK_COUNT code in tupleobject.c
* PyConfig.show_alloc_count field
* -X showalloccount command line option
* @test.support.requires_type_collecting decorator
2020-02-03 15:17:15 +01:00
Sergey Fedoseev 1f9f69dd4c bpo-27961: Replace PY_LLONG_MAX, PY_LLONG_MIN and PY_ULLONG_MAX with standard macros (GH-15385)
Use standard constants LLONG_MIN, LLONG_MAX and ULLONG_MAX.
2019-12-05 15:55:28 +01:00
Victor Stinner 310e2d2517
bpo-36854: Fix refleak in subinterpreter (GH-17331)
finalize_interp_clear() now explicitly clears the codec registry and
then trigger a GC collection to clear all references.
2019-11-22 10:58:00 +01:00
Sergey Fedoseev a9ed91e6c2 bpo-27961: Replace PY_LONG_LONG with long long. (GH-15386) 2019-10-21 09:49:48 +03:00
Neil Schemenauer 392a13bb93
bpo-38006: Add unit test for weakref clear bug (GH-16788) 2019-10-15 20:56:48 -07:00
Victor Stinner 6876257eaa
bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.

Modify the following functions to also work in release mode:

* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()

Other changes:

* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
  (equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
  with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
  attempt to render repr() at the end.
2019-10-07 18:42:01 +02:00
Victor Stinner aca8c406ad
bpo-38321: Fix _testcapimodule.c warning (GH-16494)
Fix the following warning:

    modules\_testcapimodule.c(6409):
    warning C4146: unary minus operator applied to unsigned type,
    result still unsigned
2019-09-30 21:14:26 +02:00
Eddie Elizondo 3368f3c6ae bpo-38140: Make dict and weakref offsets opaque for C heap types (#16076)
* Make dict and weakref offsets opaque for C heap types

* Add news
2019-09-19 17:29:05 +01:00
Eddie Elizondo a67ac2f2d9 bpo-38150 Fix refleak in the finalizer of a _testcapimodule type (GH-16115)
The PyLong created in the finalizer was not being cleaned up


https://bugs.python.org/issue38150



Automerge-Triggered-By: @matrixise
2019-09-13 09:48:03 -07:00
Petr Viktorin 5e9caeec76
bpo-37879: Fix warnings in _testcapimodule (GH-16004) 2019-09-12 10:12:53 +01:00
Eddie Elizondo ff023ed36e bpo-37879: Suppress subtype_dealloc decref when base type is a C heap type (GH-15323)
The instance destructor for a type is responsible for preparing
an instance for deallocation by decrementing the reference counts
of its referents.

If an instance belongs to a heap type, the type object of an instance
has its reference count decremented while for static types, which
are permanently allocated, the type object is unaffected by the
instance destructor.

Previously, the default instance destructor searched the class
hierarchy for an inherited instance destructor and, if present,
would invoke it.

Then, if the instance type is a heap type, it would decrement the
reference count of that heap type.  However, this could result in the
premature destruction of a type because the inherited instance
destructor should have already decremented the reference count
of the type object.

This change avoids the premature destruction of the type object
by suppressing the decrement of its reference count when an
inherited, non-default instance destructor has been invoked.

Finally, an assertion on the Py_SIZE of a type was deleted.  Heap
types have a non zero size, making this into an incorrect assertion.

https://github.com/python/cpython/pull/15323
2019-09-11 10:17:13 +01:00
Petr Viktorin f958377b67 bpo-37499: Test various C calling conventions (GH-15776)
Add functions with various calling conventions to `_testcapi`, expose them as module-level functions, bound methods, class methods, and static methods, and test calling them and introspecting them through GDB.


https://bugs.python.org/issue37499


Co-authored-by: Jeroen Demeyer <J.Demeyer@UGent.be>
Automerge-Triggered-By: @pganssle
2019-09-10 04:21:09 -07:00
Sergey Fedoseev 92709a263e bpo-37840: Fix handling of negative indices in bytearray_getitem() (GH-15250) 2019-09-09 09:28:34 -07:00
Min ho Kim 39d87b5471 Fix typos mostly in comments, docs and test names (GH-15209) 2019-08-30 16:21:19 -04:00
Joannah Nanjekye 2c5fb17118 bpo-36833: Add tests for Datetime C API Macros (GH-14842)
Added tests for PyDateTime_xxx_GET_xxx() macros of the C API of
the datetime module.
2019-08-29 14:54:46 +02:00
Greg Price 9ece4a5057 Unmark files as executable that can't actually be executed. (GH-15353)
There are plenty of legitimate scripts in the tree that begin with a
`#!`, but also a few that seem to be marked executable by mistake.

Found them with this command -- it gets executable files known to Git,
filters to the ones that don't start with a `#!`, and then unmarks
them as executable:

    $ git ls-files --stage \
      | perl -lane 'print $F[3] if (!/^100644/)' \
      | while read f; do
          head -c2 "$f" | grep -qxF '#!' \
          || chmod a-x "$f"; \
        done

Looking at the list by hand confirms that we didn't sweep up any
files that should have the executable bit after all.  In particular

 * The `.psd` files are images from Photoshop.

 * The `.bat` files sure look like things that can be run.
   But we have lots of other `.bat` files, and they don't have
   this bit set, so it must not be needed for them.



Automerge-Triggered-By: @benjaminp
2019-08-20 21:53:59 -07:00
Joannah Nanjekye 9e66aba999 bpo-15913: Implement PyBuffer_SizeFromFormat() (GH-13873)
Implement PyBuffer_SizeFromFormat() function (previously
documented but not implemented): call struct.calcsize().
2019-08-20 15:46:36 +01:00
Hai Shi 5623ac87bb bpo-37476: Adding tests for asutf8 and asutf8andsize (GH-14531) 2019-07-20 15:56:23 +08:00
Jeroen Demeyer 7f41c8e0dd bpo-37493: use _PyObject_CallNoArg in more places (GH-14575) 2019-07-04 19:35:31 +09:00
Victor Stinner 3bf0f3ad20
bpo-37169: Rewrite _PyObject_IsFreed() unit tests (GH-13888)
Replace two Python function calls with a single one to ensure that no
memory allocation is done between the invalid object is created and
when _PyObject_IsFreed() is called.
2019-06-07 16:22:21 +02:00
Zackery Spytz dc2476500d bpo-37170: Fix the cast on error in PyLong_AsUnsignedLongLongMask() (GH-13860) 2019-06-06 22:39:23 +02:00
Victor Stinner e225bebc14
Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714)" (GH-13780)
This reverts commit 6a150bcaeb.
2019-06-03 18:14:24 +02:00
Petr Viktorin e584cbff1e
bpo-36027 bpo-36974: Fix "incompatible pointer type" compiler warnings (GH-13758) 2019-06-03 01:08:14 +02:00
Petr Viktorin fb9423fd0a
bpo-36974: Make tp_call=PyVectorcall_Call work for inherited types (GH-13699)
When inheriting a heap subclass from a vectorcall class that sets
`.tp_call=PyVectorcall_Call` (as recommended in PEP 590), the subclass does
not inherit `_Py_TPFLAGS_HAVE_VECTORCALL`, and thus `PyVectorcall_Call` does
not work for it.

This attempts to solve the issue by:
* always inheriting `tp_vectorcall_offset` unless `tp_call` is overridden
  in the subclass
* inheriting _Py_TPFLAGS_HAVE_VECTORCALL for static types, unless `tp_call`
  is overridden
* making `PyVectorcall_Call` ignore `_Py_TPFLAGS_HAVE_VECTORCALL`

This means it'll be ever more important to only call `PyVectorcall_Call`
on classes that support vectorcall. In `PyVectorcall_Call`'s intended role
as `tp_call` filler, that's not a problem.
2019-06-02 23:52:20 +02:00
Eric Snow 6a150bcaeb
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714) 2019-06-01 15:39:46 -06:00
Zackery Spytz c7f803b08e bpo-36379: __ipow__ must be a ternaryfunc, not a binaryfunc (GH-13546)
If a type's __ipow__ method was implemented in C, attempting to use
the *modulo* parameter would cause crashes.


https://bugs.python.org/issue36379
2019-05-31 02:46:36 -07:00
Jeroen Demeyer 530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00
Jeroen Demeyer 735e8afa9e bpo-36974: inherit the vectorcall protocol (GH-13498) 2019-05-30 12:43:19 +02:00
Jeroen Demeyer aacc77fbd7 bpo-36974: implement PEP 590 (GH-13185)
Co-authored-by: Jeroen Demeyer <J.Demeyer@UGent.be>
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-29 20:31:52 +02:00
Jeroen Demeyer eb65e2443a bpo-36922: implement PEP-590 Py_TPFLAGS_METHOD_DESCRIPTOR (GH-13338)
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-28 14:42:53 +02:00
Victor Stinner 71c52e3048
bpo-36829: Add _PyErr_WriteUnraisableMsg() (GH-13488)
* sys.unraisablehook: add 'err_msg' field to UnraisableHookArgs.
* Use _PyErr_WriteUnraisableMsg() in _ctypes _DictRemover_call()
  and gc delete_garbage().
2019-05-27 08:57:14 +02:00
Victor Stinner ef9d9b6312
bpo-36829: Add sys.unraisablehook() (GH-13187)
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).

Changes:

* Add an internal UnraisableHookArgs type used to pass arguments to
  sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
  remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
  while using it
2019-05-22 11:28:22 +02:00
Edison A 98ff4d5fb6 bpo-36782: Created C API wrappers and added missing tests for functions in the PyDateTimeAPI. (#13088)
* created a c API wrapper for pyDate_FromDate and added the test

* 📜🤖 Added by blurb_it.

* fixed auto-alignment by vscode

* made changes as per PEP7

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Refactored code as per requested changes

* Remove Whitespace to Fix failed travis build

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Add a new line at end of ACKS

* Added C API function for PyDateTime_FromDateAndTime

* Added a test for the C API wrapper of PyDateTime_FromDateAndTime

* Added C API function for PyDateTime_FromDateAndTime

* Added a test for the C API wrapper of PyDateTime_FromDateAndTimeAndFold

* Remove Whitespace using patchcheck

* Added a C API function for PyTime_FromTime

* Added a test for the C API wrapper of PyTime_FromTime

* Added a C API function for PyTime_FromTimeAndFold

* Added a test for the C API wrapper of PyTime_FromTimeAndFold

* Added a C API function for PyDelta_FromDSU

* Added a test for the C API wrapper of PyDelta_FromDSU

* Refactor code, re-edit lines longer than 80 chars

* Fix Whitespace issues in DatetimeTester

* List all tests that were added in this PR

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Reformat code as per PEP7 guidelines

* Remove unused varibles from another function

* Added specific tests for the Fold Attribute

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Reformat code according to requested changes

* Reformat code to PEP7 Guidelines

* Reformat code to PEP7 Guidelines

* Re-add name to blurb

* Added a backtick to blurb file

* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst

* Remove the need to initialize mandatory parameters

* Make the macro parameter mandatory

* Re-arrange the order of unit-test args

* Removed the need to initialize macro

change all the int macro = 0 to int macro; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>

* Removed the need to initialize macro

change all the `int macro = 0` to `int macro`; now that macro is required

Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
2019-05-17 22:28:42 +02:00
Victor Stinner b16b4e4592
bpo-36763: Add PyMemAllocatorName (GH-13387)
* Add PyMemAllocatorName enum
* _PyPreConfig.allocator type becomes PyMemAllocatorName, instead of
  char*
* Remove _PyPreConfig_Clear()
* Add _PyMem_GetAllocatorName()
* Rename _PyMem_GetAllocatorsName() to
  _PyMem_GetCurrentAllocatorName()
* Remove _PyPreConfig_SetAllocator(): just call
  _PyMem_SetupAllocators() directly, we don't have do reallocate the
  configuration with the new allocator anymore!
* _PyPreConfig_Write() parameter becomes const, as it should be in
  the first place!
2019-05-17 15:20:52 +02:00
Jeroen Demeyer 351c67416b bpo-35983: skip trashcan for subclasses (GH-11841)
Add new trashcan macros to deal with a double deallocation that could occur when the `tp_dealloc` of a subclass calls the `tp_dealloc` of a base class and that base class uses the trashcan mechanism.

Patch by Jeroen Demeyer.
2019-05-10 19:21:10 +02:00
Paul Ganssle 4d8c8c0ad6 bpo-36025: Fix PyDate_FromTimestamp API (GH-11922)
In the process of converting the date.fromtimestamp function to use
argument clinic in GH-8535, the C API for PyDate_FromTimestamp was
inadvertently changed to expect a timestamp object rather than an
argument tuple.

This PR fixes this backwards-incompatible change by adding a new wrapper
function for the C API function that unwraps the argument tuple and
passes it to the underlying function.

This PR also adds tests for both PyDate_FromTimestamp and
PyDateTime_FromTimestamp to prevent any further regressions.
2019-04-27 22:39:40 +03:00