Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.
Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that. This lets our build succeed under the memory sanitizer. not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.
(cherry picked from commit 1584a00815)
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
This typo doesn't affect the result because wrong bits are discarded
on implicit conversion to unsigned char, but it trips UBSan
with -fsanitize=implicit-integer-truncation.
https://bugs.python.org/issue35194
(cherry picked from commit 7a69cf47a9)
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Two kind of mistakes:
1. Missed space. After concatenating there is no space between words.
2. Missed comma. Causes unintentional concatenating in a list of strings.
(cherry picked from commit 34fd4c2019)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Some methods in the os module can accept path-like objects. This is documented in the general documentation but not in the function docstrings. To keep both in sync, the docstrings need to be updated to reflect that path-like objects are also accepted..
(cherry picked from commit b942707fc2)
Co-authored-by: BNMetrics <luna@bnmetrics.com>
_io.IncrementalNewlineDecoder's initializer possibly assigns out-of-range
value to the bitwise struct field.
(cherry picked from commit b08746bfdf)
Co-authored-by: Xiang Zhang <angwerzx@126.com>
* Fix potential division by zero in BZ2_Malloc()
* Avoid division by zero in PyLzma_Malloc()
* Avoid division by zero and integer overflow in PyZlib_Malloc()
Reported by Svace static analyzer.
(cherry picked from commit 3d4fabb2a4)
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Declare functions with EXTINLINE:
* mpd_del()
* mpd_uint_zero()
* mpd_qresize()
* mpd_qresize_zero()
* mpd_minalloc()
These functions are implemented with "inline" or "ALWAYS_INLINE", but
declared without inline which cause linker error on Visual Studio in
Debug mode when using /Ob1.
(cherry picked from commit 3b1cba3701)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
References could leak, NULL could be dereferenced, and the Expat parser could
be double freed when some errors raised.
(cherry picked from commit 9f3ed3e213)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
* Use _PyUnicode_Copy in sanitize_isoformat_str
* Use repr in fromisoformat error message
This reverses commit 67b74a98b2 per Serhiy Storchaka's suggestion:
I suggested to use %R in the error message because including the raw
string can be confusing in the case of empty string, or string
containing trailing whitespaces, invisible or unprintable characters.
We agree that it is better to change both the C and pure Python versions
to use repr.
* Retain non-sanitized dtstr for error printing
This does not create an extra string, it just holds on to a reference to
the original input string for purposes of creating the error message.
* PEP 7 fixes to from_isoformat
* Separate handling of Unicode and other errors
In the initial implementation, errors other than encoding errors would
both raise an error indicating an invalid format, which would not be
true for errors like MemoryError.
* Drop needs_decref from _sanitize_isoformat_str
Instead _sanitize_isoformat_str returns a new reference, even to the
original string.
(cherry picked from commit 3df85404d4)
Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
Guard the `CLOCK_GETTIME` et al macros in `timemodule` based on the availability of the parent functions
(cherry picked from commit 94451182cc)
Co-authored-by: Max Bélanger <aeromax@gmail.com>
path_error() uses GetLastError() on Windows, but some os functions
are implemented via CRT APIs which report errors via errno.
This may result in raising OSError with invalid error code (such
as zero).
Introduce posix_path_error() function and use it where appropriate.
(cherry picked from commit 834603112e)
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
On failure, _PyBytes_Resize() will deallocate the bytes object and set
"result" to NULL.
https://bugs.python.org/issue34824
(cherry picked from commit 365ad2ead5)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
C implementation of xml.etree.ElementTree.Element.__setstate__()
leaked references to children when called for already initialized
element.
(cherry picked from commit 6f906b3d72)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
On macOS, fix reading from and writing into a file with a size larger than 2 GiB.
(cherry picked from commit 74a8b6ea7e)
Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
Restores the use of pyexpatns.h to isolate our embedded copy of the expat C
library so that its symbols do not conflict at link or dynamic loading time
with an embedding application or other extension modules with their own
version of libexpat.
5dc3f23b5fGH-diff-3afaf7274c90ce1b7405f75ad825f545 inadvertently removed it when upgrading expat.
(cherry picked from commit 9d4712bc8f)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
https://bugs.python.org/issue35011
Methods find(), findtext() and findall() of xml.etree.ElementTree.Element
were not able to find chldren which are instances of Element subclasses.
(cherry picked from commit b11c5667f9)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
The C implementation of asyncio.Task currently fails to perform the
cancellation cleanup correctly in the following scenario.
async def task1():
async def task2():
await task3 # task3 is never cancelled
asyncio.current_task().cancel()
await asyncio.create_task(task2())
The actuall error is a hardcoded call to `future_cancel()` instead of
calling the `cancel()` method of a future-like object.
Thanks to Vladimir Matveev for noticing the code discrepancy and to
Yury Selivanov for coming up with a pathological scenario..
(cherry picked from commit 548ce9dedd)
Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
https://bugs.python.org/issue34872
_pickle.Unpickler.__init__() should return -1 if Pdata_New() fails, not 1.
(cherry picked from commit 4b430e5f69)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Improvements:
1. Include the number of valid data characters in the error message.
2. Mention "number of data characters" rather than "length".
https://bugs.python.org/issue34736
(cherry picked from commit 1fba2ffc37)
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
Fix a crash with musl libc (on Alpine Linux) when the script filename
specified on the command line doesn't exist. pymain_open_filename()
now gets the current core configuration from the interpreter state.
Modify the code to make it closer to the master branch:
* Rename _Py_CommandLineDetails to _PyCmdline
* Remove _PyMain.config: replaced with a local variable
'local_config' in pymain_init()
* Reorganize pymain_main(): move code using the "local config"
into pymain_init()
* As soon as possible, switch from the local config to the core
configuration attached to the interpreter.
Add SSLContext.post_handshake_auth and
SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake
authentication.
Signed-off-by: Christian Heimes <christian@python.org>q
https://bugs.python.org/issue34670.
(cherry picked from commit 9fb051f032)
Co-authored-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue34670
OpenSSL follows the convention that whenever you call a function, it
returns an error indicator value; and if this value is negative, then
you need to go look at the actual error code to see what happened.
Commit c6fd1c1c3a introduced a small mistake in
_ssl__SSLSocket_shutdown_impl: instead of checking whether the error
indicator was negative, it started checking whether the actual error
code was negative, and it turns out that the error codes are never
negative. So the effect was that 'unwrap()' lost the ability to raise
SSL errors.
https://bugs.python.org/issue34759
(cherry picked from commit c0da582b22)
Co-authored-by: Nathaniel J. Smith <njs@pobox.com>
The C accelerated _elementtree module now initializes hash randomization
salt from _Py_HashSecret instead of libexpat's default CPRNG.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue34623
(cherry picked from commit cb5778f00c)
Co-authored-by: Christian Heimes <christian@python.org>
* bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)
_PyCoreConfig:
* Rename coerce_c_locale to _coerce_c_locale
* Rename coerce_c_locale_warn to _coerce_c_locale_warn
These fields are now private (name prefixed by "_").
(cherry picked from commit 188ebfa475)
* bpo-34589: C locale coercion off by default (GH-9073)
Py_Initialize() and Py_Main() cannot enable the C locale coercion
(PEP 538) anymore: it is always disabled. It can now only be enabled
by the Python program ("python3).
test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8
nor PYTHONCOERCECLOCALE, these variables are already set in the
parent.
(cherry picked from commit 7a0791b699)
* bpo-34589: Add -X coerce_c_locale command line option (GH-9378)
Add a new -X coerce_c_locale command line option to control C locale
coercion (PEP 538).
(cherry picked from commit dbdee0073c)
Include ``openssl/dh.h`` header file to fix implicit function declaration of ``DH_free()``.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
(cherry picked from commit b3a271fc0c)
Co-authored-by: Alexandru Ardelean <ardeleanalex@gmail.com>
[bpo-34658](https://www.bugs.python.org/issue34658): Fix a rare interpreter unhandled exception state SystemError only
seen when using subprocess with a preexec_fn while an after_parent handler has
been registered with os.register_at_fork and the fork system call fails.
https://bugs.python.org/issue34658
(cherry picked from commit a20b6adb5a)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Gregory P. Smith
Miss Islington (bot)
BNMetrics
Serhiy Storchaka
Stéphane Wirtel
Elvis Pranskevichus
Yury Selivanov
Victor Stinner
Christian Heimes
Benjamin Peterson