Several built-in and standard library types now ensure that their internal result tuples are always tracked by the garbage collector:
- collections.OrderedDict.items
- dict.items
- enumerate
- functools.reduce
- itertools.combinations
- itertools.combinations_with_replacement
- itertools.permutations
- itertools.product
- itertools.zip_longest
- zip
Previously, they could have become untracked by a prior garbage collection.
(cherry picked from commit 226a012d1c)
* There were leaks if Py_tp_bases is used more than once or if some call is
failed before setting tp_bases.
* There was a crash if the bases argument or the Py_tp_bases slot is not a tuple.
* The documentation was not accurate.
(cherry picked from commit 1db76394ea)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
func_dealloc() does not handle partially-created objects. Best not to give it any.
(cherry picked from commit 350526105f)
Co-authored-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
Enable recursion checks which were disabled when get __bases__ of
non-type objects in issubclass() and isinstance() and when intern
strings. It fixes a stack overflow when getting __bases__ leads
to infinite recursion.
Originally recursion checks was disabled for PyDict_GetItem() which
silences all errors including the one raised in case of detected
recursion and can return incorrect result. But now the code uses
PyDict_GetItemWithError() and PyDict_SetDefault() instead.
(cherry picked from commit 9ece9cd65c)
When allocating MemoryError classes, there is some logic to use
pre-allocated instances in a freelist only if the type that is being
allocated is not a subclass of MemoryError. Unfortunately in the
destructor this logic is not present so the freelist is altered even
with subclasses of MemoryError..
(cherry picked from commit 9b648a95cc)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>.
(cherry picked from commit 87e91ae2e5)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Walk down the MRO backwards to find the type that originally defined the final `tp_setattro`, then make sure we are not jumping over intermediate C-level bases with the Python-level call.
Automerge-Triggered-By: @gvanrossum
(cherry picked from commit c53b310e59)
Co-authored-by: scoder <stefan_ml@behnel.de>
The issue is triggered by the bytearray() + bytearray() operation.
Detected by GCC 10 static analysis tool.
(cherry picked from commit 61fc23ca10)
Co-authored-by: stratakis <cstratak@redhat.com>
Unexpected errors in calling the __iter__ method are no longer
masked by TypeError in the "in" operator and functions
operator.contains(), operator.indexOf() and operator.countOf().
(cherry picked from commit cafe1b6e9d)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
PyDescr_NewMethod() and PyCFunction_NewEx() now include the method
name in the SystemError "bad call flags" error message to ease debug.
(cherry picked from commit c7d2d69d95)
Objects do not own weak references to them directly through the __weakref__ list so these
do not need to be traversed by the GC.
(cherry picked from commit 0c2b509)
Hold reference of __bases__ tuple until tuple item is done with, because by
dropping the reference the item may be destroyed.
(cherry picked from commit 1c56f8ffad)
Co-authored-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
The fix for [bpo-39386](https://bugs.python.org/issue39386) attempted to make it so you couldn't reuse a
agen.aclose() coroutine object. It accidentally also prevented you
from calling aclose() at all on an async generator that was already
closed or exhausted. This commit fixes it so we're only blocking the
actually illegal cases, while allowing the legal cases.
The new tests failed before this patch. Also confirmed that this fixes
the test failures we were seeing in Trio with Python dev builds:
https://github.com/python-trio/trio/pull/1396https://bugs.python.org/issue39606
(cherry picked from commit 925dc7fb1d)
Co-authored-by: Nathaniel J. Smith <njs@pobox.com>
https://bugs.python.org/issue39606
Automerge-Triggered-By: @njsmith
gcc -Wcast-qual turns up a number of instances of casting away constness of pointers. Some of these can be safely modified, by either:
Adding the const to the type cast, as in:
- return _PyUnicode_FromUCS1((unsigned char*)s, size);
+ return _PyUnicode_FromUCS1((const unsigned char*)s, size);
or, Removing the cast entirely, because it's not necessary (but probably was at one time), as in:
- PyDTrace_FUNCTION_ENTRY((char *)filename, (char *)funcname, lineno);
+ PyDTrace_FUNCTION_ENTRY(filename, funcname, lineno);
These changes will not change code, but they will make it much easier to check for errors in consts
(cherry picked from commit e6be9b59a9)
Co-authored-by: Andy Lester <andy@petdance.com>
Improvements in listsort.txt and a comment in sortperf.py.
Automerge-Triggered-By: @csabella
(cherry picked from commit 24e5ad4689)
Co-authored-by: Stefan Pochmann <stefan.pochmann@gmail.com>
Some objects like Py_None are not initialized with conventional means
that prepare the circular linked list pointers, leaving them unlinked
from the rest of the objects. For those objects, NULL pointers does
not mean that they are freed, so we need to skip the check in those
cases.
(cherry picked from commit 36e33c360e)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.8] bpo-38588: Fix possible crashes in dict and list when calling PyObject_RichCompareBool (GH-17734)
Take strong references before calling PyObject_RichCompareBool to protect against the case
where the object dies during the call.
(cherry picked from commit 2d5bf568ea)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Update Objects/listobject.c
@methane's suggestion
Co-Authored-By: Inada Naoki <songofacandy@gmail.com>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
Hold strong references to list elements while calling PyObject_RichCompareBool().
(cherry picked from commit d9e561d23d)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Ignore `GeneratorExit` exceptions when throwing an exception into the `aclose` coroutine of an asynchronous generator.
https://bugs.python.org/issue35409
(cherry picked from commit 8e0de2a480)
Co-authored-by: Vincent Michel <vxgmichel@gmail.com>
The reverse iterator for empty dictionaries was not handling correctly shared-key dictionaries.
(cherry picked from commit 24dc2f8c56)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.
Modify the following functions to also work in release mode:
* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()
Other changes:
* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
(equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
attempt to render repr() at the end.
(cherry picked from commit 6876257eaa)
* bpo-36389: Fix _PyBytesWriter in release mode (GH-16624)
Fix _PyBytesWriter API when Python is built in release mode with
assertions.
(cherry picked from commit 60ec6efd96)
* bpo-38070: Enhance visit_decref() debug trace (GH-16631)
subtract_refs() now pass the parent object to visit_decref() which
pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails,
the parent is used in debug trace, rather than the freed object. The
parent object is more likely to contain useful information. Freed
objects cannot be inspected are are displayed as "<object at xxx is
freed>" with no other detail.
(cherry picked from commit 4d5f94b8cd)
* Fix also a typo in PYMEM_DEADBYTE macro comment
* bpo-36389: Add newline to _PyObject_AssertFailed() (GH-16629)
Add a newline between the verbose object dump and the Py_FatalError()
logs for readability.
(cherry picked from commit 7775349895)
The implementation of weakref.proxy's methods call back into the Python
API using a borrowed references of the weakly referenced object
(acquired via PyWeakref_GET_OBJECT). This API call may delete the last
reference to the object (either directly or via GC), leaving a dangling
pointer, which can be subsequently dereferenced.
To fix this, claim a temporary ownership of the referenced object when
calling the appropriate method. Some functions because at the moment they
do not need to access the borrowed referent, but to protect against
future changes to these functions, ownership need to be fixed in
all potentially affected methods..
(cherry picked from commit 10cd00a9e3)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Document that lnotab can contain invalid bytecode offsets (because of
terrible reasons that are difficult to fix). Make dis.findlinestarts()
ignore invalid offsets in lnotab. All other uses of lnotab in CPython
(various reimplementations of addr2line or line2addr in Python, C and gdb)
already ignore this, because they take an address to look for, instead.
Add tests for the result of dis.findlinestarts() on wacky constructs in
test_peepholer.py, because it's the easiest place to add them.
(cherry picked from commit c8165036f3)
Make negative interpreter id to raise ValueError instead of RuntimeError.
(cherry picked from commit 543a3951a1)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Python now dumps path configuration if it fails to import the Python
codecs of the filesystem and stdio encodings.
(cherry picked from commit fcdb027234)
* bpo-38070: _Py_DumpTraceback() writes <no Python frame> (GH-16244)
When a Python thread has no frame, _Py_DumpTraceback() and
_Py_DumpTracebackThreads() now write "<no Python frame>", rather than
writing nothing.
(cherry picked from commit 8fa3e1740b)
* bpo-38070: Enhance _PyObject_Dump() (GH-16243)
_PyObject_Dump() now dumps the object address for freed objects and
objects with ob_type=NULL.
(cherry picked from commit b39afb7876)
* bpo-38070: Add _PyRuntimeState.preinitializing (GH-16245)
Add _PyRuntimeState.preinitializing field: set to 1 while
Py_PreInitialize() is running.
_PyRuntimeState: rename also pre_initialized field to preinitialized.
(cherry picked from commit d3b904144e)
* bpo-38070: Py_FatalError() logs runtime state (GH-16246)
(cherry picked from commit 1ce16fb097)
Even when the helper is not started yet.
This behavior follows conventional generator one.
There is no reason for `async_generator_athrow` to handle `gen.throw()` differently.
https://bugs.python.org/issue38013
(cherry picked from commit c275312a62)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
* Fix a crash in comparing with float (and maybe other crashes).
* They are now never equal to strings and non-integer numbers.
* Comparison with a large number no longer raises OverflowError.
* Arbitrary exceptions no longer silenced in constructors and comparisons.
* TypeError raised in the constructor contains now the name of the type.
* Accept only ChannelID and int-like objects in channel functions.
* Accept only InterpreterId, int-like objects and str in the InterpreterId constructor.
* Accept int-like objects, not just int in interpreter related functions.
(cherry picked from commit bf169915ec)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
(cherry picked from commit 279f44678c)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Brandt Bucher
Miss Islington (bot)
Serhiy Storchaka
Pablo Galindo
scoder
Victor Stinner
Dong-hee Na
Benjamin Peterson
Neil Schemenauer
Gregory P. Smith
Victor Stinner