* bpo-36389: _PyObject_IsFreed() now also detects uninitialized memory (GH-12770)
Replace _PyMem_IsFreed() function with _PyMem_IsPtrFreed() inline
function. The function is now way more efficient, it became a simple
comparison on integers, rather than a short loop. It detects also
uninitialized bytes and "forbidden bytes" filled by debug hooks
on memory allocators.
Add unit tests on _PyObject_IsFreed().
(cherry picked from commit 2b00db6855)
* bpo-36389: Change PyMem_SetupDebugHooks() constants (GH-12782)
Modify CLEANBYTE, DEADDYTE and FORBIDDENBYTE constants: use 0xCD,
0xDD and 0xFD, rather than 0xCB, 0xBB and 0xFB, to use the same byte
patterns than Windows CRT debug malloc() and free().
(cherry picked from commit 4c409beb4c)
The bug occurred when the encoded surrogate character is passed
to the incremental decoder in two chunks.
(cherry picked from commit 7a465cb5ee)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
`object.__new__` and `object.__init__` do take one argument each,
they just don't take extra user supplied arguments.
Patch by Sanyam Khurana.
(cherry picked from commit 5105483acb)
Co-authored-by: Sanyam Khurana <8039608+CuriousLearner@users.noreply.github.com>
Not using `__class_getitem__()` fallback if there is a non-subcriptable metaclass was caused by a certain asymmetry between how `PySequenceMethods` and `PyMappingMethods` are used in `PyObject_GetItem`. This PR removes this asymmetry. No tests failed, so I assume it was not intentional.
(cherry picked from commit ac28147e78)
Co-authored-by: Ivan Levkivskyi <levkivskyi@gmail.com>
Fix a crash in slice_richcompare(): use strong references rather than
stolen references for the two temporary internal tuples.
The crash (or assertion error) occurred if a garbage collection
occurred during slice_richcompare(), especially while calling
PyObject_RichCompare(t1, t2, op).
(cherry picked from commit dcb68f47f7)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Format characters "%s" and "%V" in PyUnicode_FromFormat() and "%s" in PyBytes_FromFormat()
no longer read memory past the limit if precision is specified.
(cherry picked from commit d586ccb04f)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Add also tests for PyUnicode_FromFormat() and PyBytes_FromFormat()
with empty result.
(cherry picked from commit 44cc4822bb)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Fix an assertion error in format() in debug build for floating point
formatting with "n" format, zero padding and small width. Release build is
not impacted. Patch by Karthikeyan Singaravelan.
(cherry picked from commit 3f7983a25a)
Co-authored-by: Xtreak <tir.karthi@gmail.com>
There is already a `Py_ssize_t i` defined at function scope that is used
for similar loops. By removing the local `int i` declaration that `i` is
used, which has the appropriate type.
(cherry picked from commit f8b534477a)
Co-authored-by: sth <sth.dev@tejp.de>
* bpo-35454: Fix miscellaneous minor issues in error handling.
* Fix a null pointer dereference.
(cherry picked from commit 8905fcc85a)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Fix memory leak in PyUnicode_EncodeLocale() and
PyUnicode_EncodeFSDefault() on error handling.
Fix unicode_encode_locale() error handling.
(cherry picked from commit bde9d6bbb4)
Fix also return type for few other functions (clear, releasebuffer).
(cherry picked from commit d4f9cf5545)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Copy code from master: add assertions on start and value, replace 'i'
iterator with 'end' pointer for the loop stop condition.
_PyUnicode_FastFill(): fix type of 'data', it must not be constant,
since data is modified by FILL().
Fix str.format(), float.__format__() and complex.__format__() methods
for non-ASCII decimal point when using the "n" formatter.
Rewrite _PyUnicode_InsertThousandsGrouping(): it now requires
a _PyUnicodeWriter object for the buffer and a Python str object
for digits.
(cherry picked from commit 59423e3ddd)
* bpo-9263: _PyObject_Dump() detects freed memory (GH-10061)
_PyObject_Dump() now uses an heuristic to check if the object memory
has been freed: log "<freed object>" in that case.
The heuristic rely on the debug hooks on Python memory allocators
which fills the memory with DEADBYTE (0xDB) when memory is
deallocated. Use PYTHONMALLOC=debug to always enable these debug
hooks.
(cherry picked from commit 82af0b63b0)
* bpo-9263: Fix _PyObject_Dump() for freed object (#10661)
If _PyObject_Dump() detects that the object is freed, don't try to
dump it (exit immediately).
Enhance also _PyObject_IsFreed(): it now detects if the pointer
itself looks like freed memory.
(cherry picked from commit 2cf5d32fd9)
coro->cr_origin wasn't initialized if compute_cr_origin() failed in
PyCoro_New(), which would cause a crash during the coroutine's
deallocation.
https://bugs.python.org/issue35269
(cherry picked from commit 062a57bf4b)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Discovered using clang's MemorySanitizer when it ran python3's
test_fstring test_misformed_unicode_character_name.
An msan build will fail by simply executing: ./python -c 'u"\N"'
(cherry picked from commit 746b2d35ea)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
https://bugs.python.org/issue35214
Rename our new MEMORY_SANITIZER define to _Py_MEMORY_SANITIZER.
Project based C Preprocessor namespacing at its finest. :P
(cherry picked from commit 3015fb8ce4)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
clang's MemorySanitizer understand getc() but does not understand
getc_unlocked(). Workaround: Don't use it on msan builds.
(cherry picked from commit e6c77d8301)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
This function may access memory which is mapped but is considered
free by libc allocator. It behaves so by design, therefore we
need to suppress sanitizer reports.
GCC doesn't support MSan, so disable only TSan for it.
(cherry picked from commit fd3a91cbf9)
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Constructing bytes from mutating list could cause a crash.
(cherry picked from commit 914f9a078f)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Address a C undefined behavior signed integer overflow issue in set object table resizing. Our -fwrapv compiler flag and practical reasons why sets are unlikely to get this large should mean this was never an issue but it was incorrect code that generates code analysis warnings.
Co-authored-by: Sergey Fedoseev <fedoseev.sergey@gmail.com>
The _PyLong_FromByteArray() call in int_from_bytes_impl() was
unchecked.
(cherry picked from commit 7bb9cd0a67)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
formatfloat() was not checking if PyBytes_FromStringAndSize()
failed, which could lead to a null pointer dereference in
_PyBytes_FormatEx().
(cherry picked from commit 96c5932794)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)`
should use it instead of dict ordering.
https://bugs.python.org/issue34320
(cherry picked from commit 2aaf98c16a)
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
Many type object initializations labeled a field "tp_size" in the
comment, but the name of that field is tp_basicsize..
(cherry picked from commit 0e0bc4e221)
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
When calling tp_descr_get(self, obj, type), make sure that
we own a strong reference to "self".
(cherry picked from commit 8f735485ac)
Co-authored-by: jdemeyer <jdemeyer@cage.ugent.be>
Also, propagate the error from PyNumber_AsSsize_t() because we don't care
only about OverflowError which is not reported if the second argument is NULL.
Reported by Svace static analyzer.
(cherry picked from commit 7ecae3ca0b)
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Victor Stinner
Miss Islington (bot)
Serhiy Storchaka
Zackery Spytz
Benjamin Peterson