Fix memory leak in PyUnicode_EncodeLocale() and
PyUnicode_EncodeFSDefault() on error handling.
Fix unicode_encode_locale() error handling.
(cherry picked from commit bde9d6bbb4)
Fix also return type for few other functions (clear, releasebuffer).
(cherry picked from commit d4f9cf5545)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Copy code from master: add assertions on start and value, replace 'i'
iterator with 'end' pointer for the loop stop condition.
_PyUnicode_FastFill(): fix type of 'data', it must not be constant,
since data is modified by FILL().
Fix str.format(), float.__format__() and complex.__format__() methods
for non-ASCII decimal point when using the "n" formatter.
Rewrite _PyUnicode_InsertThousandsGrouping(): it now requires
a _PyUnicodeWriter object for the buffer and a Python str object
for digits.
(cherry picked from commit 59423e3ddd)
* bpo-9263: _PyObject_Dump() detects freed memory (GH-10061)
_PyObject_Dump() now uses an heuristic to check if the object memory
has been freed: log "<freed object>" in that case.
The heuristic rely on the debug hooks on Python memory allocators
which fills the memory with DEADBYTE (0xDB) when memory is
deallocated. Use PYTHONMALLOC=debug to always enable these debug
hooks.
(cherry picked from commit 82af0b63b0)
* bpo-9263: Fix _PyObject_Dump() for freed object (#10661)
If _PyObject_Dump() detects that the object is freed, don't try to
dump it (exit immediately).
Enhance also _PyObject_IsFreed(): it now detects if the pointer
itself looks like freed memory.
(cherry picked from commit 2cf5d32fd9)
coro->cr_origin wasn't initialized if compute_cr_origin() failed in
PyCoro_New(), which would cause a crash during the coroutine's
deallocation.
https://bugs.python.org/issue35269
(cherry picked from commit 062a57bf4b)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Discovered using clang's MemorySanitizer when it ran python3's
test_fstring test_misformed_unicode_character_name.
An msan build will fail by simply executing: ./python -c 'u"\N"'
(cherry picked from commit 746b2d35ea)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
https://bugs.python.org/issue35214
Rename our new MEMORY_SANITIZER define to _Py_MEMORY_SANITIZER.
Project based C Preprocessor namespacing at its finest. :P
(cherry picked from commit 3015fb8ce4)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
clang's MemorySanitizer understand getc() but does not understand
getc_unlocked(). Workaround: Don't use it on msan builds.
(cherry picked from commit e6c77d8301)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
This function may access memory which is mapped but is considered
free by libc allocator. It behaves so by design, therefore we
need to suppress sanitizer reports.
GCC doesn't support MSan, so disable only TSan for it.
(cherry picked from commit fd3a91cbf9)
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Constructing bytes from mutating list could cause a crash.
(cherry picked from commit 914f9a078f)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Address a C undefined behavior signed integer overflow issue in set object table resizing. Our -fwrapv compiler flag and practical reasons why sets are unlikely to get this large should mean this was never an issue but it was incorrect code that generates code analysis warnings.
Co-authored-by: Sergey Fedoseev <fedoseev.sergey@gmail.com>
The _PyLong_FromByteArray() call in int_from_bytes_impl() was
unchecked.
(cherry picked from commit 7bb9cd0a67)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
formatfloat() was not checking if PyBytes_FromStringAndSize()
failed, which could lead to a null pointer dereference in
_PyBytes_FormatEx().
(cherry picked from commit 96c5932794)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)`
should use it instead of dict ordering.
https://bugs.python.org/issue34320
(cherry picked from commit 2aaf98c16a)
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
Many type object initializations labeled a field "tp_size" in the
comment, but the name of that field is tp_basicsize..
(cherry picked from commit 0e0bc4e221)
Co-authored-by: Peter Eisentraut <peter@eisentraut.org>
When calling tp_descr_get(self, obj, type), make sure that
we own a strong reference to "self".
(cherry picked from commit 8f735485ac)
Co-authored-by: jdemeyer <jdemeyer@cage.ugent.be>
Also, propagate the error from PyNumber_AsSsize_t() because we don't care
only about OverflowError which is not reported if the second argument is NULL.
Reported by Svace static analyzer.
(cherry picked from commit 7ecae3ca0b)
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Check that the size of the varnames tuple is enough at least for all arguments.
(cherry picked from commit bd47384e07)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
`_PyUnicode_TransformDecimalAndSpaceToASCII()` missed trailing NUL char.
It caused buffer overflow in `_Py_string_to_number_with_underscores()`.
This bug is introduced in 9b6c60cb.
(cherry picked from commit 16dfca4d82)
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
Fix clang ubsan (undefined behavior sanitizer) warnings in dictobject.c by
adjusting how the internal struct _dictkeysobject shared keys structure is
declared.
This remains ABI compatible. We get rid of the union at the end of the
struct being used for conveinence to avoid typecasting in favor of char[]
variable length array at the end of a struct. This is known to clang to be
used for variable sized objects and will not cause an undefined behavior
problem. Similarly, char arrays do not have strict aliasing undefined
behavior when cast.
PEP-007 does not currently list variable length arrays (VLAs) as allowed
in our subset of C99. If this turns out to be a problem, the fix to this is
to change the char `dk_indices[]` into `dk_indices[1]` and restore the
three size computation subtractions this change removes:
`- Py_MEMBER_SIZE(PyDictKeysObject, dk_indices)`
If this works as is I'll make a separate PR to update PEP-007.
(cherry picked from commit 397f1b28c4)
This code doesn't have effect on the final result, but causes
GCC 8 warnings and can have an undefined behavior.
(cherry picked from commit 827d49f3cf)
Multi-phase initialized modules allow m_traverse to be called while the
module is still being initialized, so module authors may need to account
for that.
(cherry picked from commit c2b0b12d1a)
Co-authored-by: Marcel Plch <gmarcel.plch@gmail.com>
Zackery Spytz
Miss Islington (bot)
Victor Stinner
Serhiy Storchaka
Benjamin Peterson
xdegaye