* bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.
Modify the following functions to also work in release mode:
* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()
Other changes:
* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
(equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
attempt to render repr() at the end.
(cherry picked from commit 6876257eaa)
* bpo-36389: Fix _PyBytesWriter in release mode (GH-16624)
Fix _PyBytesWriter API when Python is built in release mode with
assertions.
(cherry picked from commit 60ec6efd96)
* bpo-38070: Enhance visit_decref() debug trace (GH-16631)
subtract_refs() now pass the parent object to visit_decref() which
pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails,
the parent is used in debug trace, rather than the freed object. The
parent object is more likely to contain useful information. Freed
objects cannot be inspected are are displayed as "<object at xxx is
freed>" with no other detail.
(cherry picked from commit 4d5f94b8cd)
* Fix also a typo in PYMEM_DEADBYTE macro comment
* bpo-36389: Add newline to _PyObject_AssertFailed() (GH-16629)
Add a newline between the verbose object dump and the Py_FatalError()
logs for readability.
(cherry picked from commit 7775349895)
The PyLong created in the finalizer was not being cleaned up
https://bugs.python.org/issue38150
Automerge-Triggered-By: @matrixise
(cherry picked from commit a67ac2f2d9)
Co-authored-by: Eddie Elizondo <eelizondo@fb.com>
The instance destructor for a type is responsible for preparing
an instance for deallocation by decrementing the reference counts
of its referents.
If an instance belongs to a heap type, the type object of an instance
has its reference count decremented while for static types, which
are permanently allocated, the type object is unaffected by the
instance destructor.
Previously, the default instance destructor searched the class
hierarchy for an inherited instance destructor and, if present,
would invoke it.
Then, if the instance type is a heap type, it would decrement the
reference count of that heap type. However, this could result in the
premature destruction of a type because the inherited instance
destructor should have already decremented the reference count
of the type object.
This change avoids the premature destruction of the type object
by suppressing the decrement of its reference count when an
inherited, non-default instance destructor has been invoked.
Finally, an assertion on the Py_SIZE of a type was deleted. Heap
types have a non zero size, making this into an incorrect assertion.
https://github.com/python/cpython/pull/15323.
(cherry picked from commit ff023ed36e)
Fixup: https://github.com/python/cpython/pull/16004.
(cherry picked from commit 5e9caeec76)
Co-authored-by: Eddie Elizondo <eduardo.elizondorueda@gmail.com>
Replace two Python function calls with a single one to ensure that no
memory allocation is done between the invalid object is created and
when _PyObject_IsFreed() is called.
(cherry picked from commit 3bf0f3ad20)
When inheriting a heap subclass from a vectorcall class that sets
`.tp_call=PyVectorcall_Call` (as recommended in PEP 590), the subclass does
not inherit `_Py_TPFLAGS_HAVE_VECTORCALL`, and thus `PyVectorcall_Call` does
not work for it.
This attempts to solve the issue by:
* always inheriting `tp_vectorcall_offset` unless `tp_call` is overridden
in the subclass
* inheriting _Py_TPFLAGS_HAVE_VECTORCALL for static types, unless `tp_call`
is overridden
* making `PyVectorcall_Call` ignore `_Py_TPFLAGS_HAVE_VECTORCALL`
This means it'll be ever more important to only call `PyVectorcall_Call`
on classes that support vectorcall. In `PyVectorcall_Call`'s intended role
as `tp_call` filler, that's not a problem.
* sys.unraisablehook: add 'err_msg' field to UnraisableHookArgs.
* Use _PyErr_WriteUnraisableMsg() in _ctypes _DictRemover_call()
and gc delete_garbage().
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).
Changes:
* Add an internal UnraisableHookArgs type used to pass arguments to
sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
while using it
* created a c API wrapper for pyDate_FromDate and added the test
* 📜🤖 Added by blurb_it.
* fixed auto-alignment by vscode
* made changes as per PEP7
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Refactored code as per requested changes
* Remove Whitespace to Fix failed travis build
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Add a new line at end of ACKS
* Added C API function for PyDateTime_FromDateAndTime
* Added a test for the C API wrapper of PyDateTime_FromDateAndTime
* Added C API function for PyDateTime_FromDateAndTime
* Added a test for the C API wrapper of PyDateTime_FromDateAndTimeAndFold
* Remove Whitespace using patchcheck
* Added a C API function for PyTime_FromTime
* Added a test for the C API wrapper of PyTime_FromTime
* Added a C API function for PyTime_FromTimeAndFold
* Added a test for the C API wrapper of PyTime_FromTimeAndFold
* Added a C API function for PyDelta_FromDSU
* Added a test for the C API wrapper of PyDelta_FromDSU
* Refactor code, re-edit lines longer than 80 chars
* Fix Whitespace issues in DatetimeTester
* List all tests that were added in this PR
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Reformat code as per PEP7 guidelines
* Remove unused varibles from another function
* Added specific tests for the Fold Attribute
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Reformat code according to requested changes
* Reformat code to PEP7 Guidelines
* Reformat code to PEP7 Guidelines
* Re-add name to blurb
* Added a backtick to blurb file
* Update 2019-05-04-21-25-19.bpo-36782.h3oPIb.rst
* Remove the need to initialize mandatory parameters
* Make the macro parameter mandatory
* Re-arrange the order of unit-test args
* Removed the need to initialize macro
change all the int macro = 0 to int macro; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Removed the need to initialize macro
change all the `int macro = 0` to `int macro`; now that macro is required
Co-Authored-By: Paul Ganssle <pganssle@users.noreply.github.com>
* Add PyMemAllocatorName enum
* _PyPreConfig.allocator type becomes PyMemAllocatorName, instead of
char*
* Remove _PyPreConfig_Clear()
* Add _PyMem_GetAllocatorName()
* Rename _PyMem_GetAllocatorsName() to
_PyMem_GetCurrentAllocatorName()
* Remove _PyPreConfig_SetAllocator(): just call
_PyMem_SetupAllocators() directly, we don't have do reallocate the
configuration with the new allocator anymore!
* _PyPreConfig_Write() parameter becomes const, as it should be in
the first place!
Add new trashcan macros to deal with a double deallocation that could occur when the `tp_dealloc` of a subclass calls the `tp_dealloc` of a base class and that base class uses the trashcan mechanism.
Patch by Jeroen Demeyer.
In the process of converting the date.fromtimestamp function to use
argument clinic in GH-8535, the C API for PyDate_FromTimestamp was
inadvertently changed to expect a timestamp object rather than an
argument tuple.
This PR fixes this backwards-incompatible change by adding a new wrapper
function for the C API function that unwraps the argument tuple and
passes it to the underlying function.
This PR also adds tests for both PyDate_FromTimestamp and
PyDateTime_FromTimestamp to prevent any further regressions.
Add a new _testinternalcapi module to test the internal C API.
Move _Py_GetConfigsAsDict() function to the internal C API:
_testembed now uses _testinternalcapi to access the function.
Change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined.
The Py_BUILD_CORE_MODULE define must be now be used to build a C
extension as a dynamic library accessing Python internals: export the
PyInit_xxx() function in DLL exports on Windows.
Changes:
* Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE now imply
Py_BUILD_CORE directy in pyport.h.
* ceval.c compilation now fails with an error if Py_BUILD_CORE is not
defined, just to ensure that Python is build with the correct
defines.
* setup.py now compiles _pickle.c with Py_BUILD_CORE_MODULE define.
* setup.py compiles _json.c with Py_BUILD_CORE_MODULE define, rather
than Py_BUILD_CORE_BUILTIN define
* PCbuild/pythoncore.vcxproj: Add Py_BUILD_CORE_BUILTIN define.
This is effectively an un-revert of #11617 and #12024 (reverted in #12159). Portions of those were merged in other PRs (with lower risk) and this represents the remainder. Note that I found 3 different bugs in the original PRs and have fixed them here.
Replace _PyMem_IsFreed() function with _PyMem_IsPtrFreed() inline
function. The function is now way more efficient, it became a simple
comparison on integers, rather than a short loop. It detects also
uninitialized bytes and "forbidden bytes" filled by debug hooks
on memory allocators.
Add unit tests on _PyObject_IsFreed().
* Add _Py_GetConfigsAsDict() function to get all configurations as a
dict.
* dump_config() of _testembed.c now dumps preconfig as a separated
key: call _Py_GetConfigsAsDict().
* Make _PyMainInterpreterConfig_AsDict() private.
* Revert "bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003)"
This reverts commit bcfa450f21.
* Revert "bpo-33608: Simplify ceval's DISPATCH by hoisting eval_breaker ahead of time. (gh-12062)"
This reverts commit bda918bf65.
* Revert "bpo-33608: Use _Py_AddPendingCall() in _PyCrossInterpreterData_Release(). (gh-12024)"
This reverts commit b05b711a2c.
* Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)"
This reverts commit ef4ac967e2.
As in title, expose C `raise` function as `raise_function` in `signal` module. Also drop existing `raise_signal` in `_testcapi` module and replace all usages with new function.
https://bugs.python.org/issue35568
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
* Fix _PyCoreConfig_SetGlobalConfig(): set also Py_FrozenFlag
* Fix _PyCoreConfig_AsDict(): export also xoptions
* Add _Py_GetGlobalVariablesAsDict() and _testcapi.get_global_config()
* test.pythoninfo: dump also global configuration variables
* _testembed now serializes global, core and main configurations
using JSON to reuse _Py_GetGlobalVariablesAsDict(),
_PyCoreConfig_AsDict() and _PyMainInterpreterConfig_AsDict(),
rather than duplicating code.
* test_embed.InitConfigTests now test much more configuration
variables
* Fix _PyMainInterpreterConfig_Copy():
copy 'install_signal_handlers' attribute
* Add _PyMainInterpreterConfig_AsDict()
* Add unit tests on the main interpreter configuration
to test_embed.InitConfigTests
* test.pythoninfo: log also main_config
* _PyTuple_ITEMS() gives access to the tuple->ob_item field and cast the
first argument to PyTupleObject*. This internal macro is only usable if
Py_BUILD_CORE is defined.
* Replace &PyTuple_GET_ITEM(ob, 0) with _PyTuple_ITEMS(ob).
* Replace PyTuple_GET_ITEM(op, 1) with &_PyTuple_ITEMS(ob)[1].
_testcapimodule.c must not include pycore_pathconfig.h, since it's an
internal header files.
Changes:
* Add _PyCoreConfig_AsDict() function to coreconfig.c.
* Remove pycore_pathconfig.h include from _testcapimodule.h.
* pycore_pathconfig.h now requires Py_BUILD_CORE to be defined.
* _testcapimodule.c compilation now fails if it's built with
Py_BUILD_CORE defined.
* And pycore_lifecycle.h and pycore_pathconfig.h headers to
Include/internal/
* Move Py_BUILD_CORE specific code from coreconfig.h and
pylifecycle.h to pycore_pathconfig.h and pycore_lifecycle.h
* Move _Py_wstrlist_XXX() definitions and _PyPathConfig code
from pycore_state.h to pycore_pathconfig.h
* Move "Init" and "Fini" function definitions from pylifecycle.c to
pycore_lifecycle.h.
If Py_BUILD_CORE is defined, the PyThreadState_GET() macro access
_PyRuntime which comes from the internal pycore_state.h header.
Public headers must not require internal headers.
Move PyThreadState_GET() and _PyInterpreterState_GET_UNSAFE() from
Include/pystate.h to Include/internal/pycore_state.h, and rename
PyThreadState_GET() to _PyThreadState_GET() there.
The PyThreadState_GET() macro of pystate.h is now redefined when
pycore_state.h is included, to use the fast _PyThreadState_GET().
Changes:
* Add _PyThreadState_GET() macro
* Replace "PyThreadState_GET()->interp" with
_PyInterpreterState_GET_UNSAFE()
* Replace PyThreadState_GET() with _PyThreadState_GET() in internal C
files (compiled with Py_BUILD_CORE defined), but keep
PyThreadState_GET() in the public header files.
* _testcapimodule.c: replace PyThreadState_GET() with
PyThreadState_Get(); the module is not compiled with Py_BUILD_CORE
defined.
* pycore_state.h now requires Py_BUILD_CORE to be defined.
* Add Py_STATIC_INLINE() macro to declare a "static inline" function.
If the compiler supports it, try to always inline the function even if no
optimization level was specified.
* Modify pydtrace.h to use Py_STATIC_INLINE() when WITH_DTRACE is
not defined.
* Add an unit test on Py_DECREF() to make sure that
_Py_NegativeRefcount() reports the correct filename.
* Revert "bpo-34589: Add -X coerce_c_locale command line option (GH-9378)"
This reverts commit dbdee0073c.
* Revert "bpo-34589: C locale coercion off by default (GH-9073)"
This reverts commit 7a0791b699.
* Revert "bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)"
This reverts commit 188ebfa475.
_PyCoreConfig:
* Rename coerce_c_locale to _coerce_c_locale
* Rename coerce_c_locale_warn to _coerce_c_locale_warn
These fields are now private (name prefixed by "_").
* Add _testcapi.get_coreconfig() to get the _PyCoreConfig of the
interpreter
* test.pythoninfo now gets the core configuration using
_testcapi.get_coreconfig()
Serhiy Storchaka
Victor Stinner
Stéphane Wirtel
Petr Viktorin
Miss Islington (bot)
Jeroen Demeyer
Victor Stinner
Zackery Spytz
Eric Snow
Edison A
Paul Ganssle
Vladimir Matveev
Eddie Elizondo
jdemeyer