Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.
Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that. This lets our build succeed under the memory sanitizer. not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.
(cherry picked from commit 1584a00815)
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
* Fix multiple typos in code comments
* Add spacing in comments (test_logging.py, test_math.py)
* Fix spaces at the beginning of comments in test_logging.py
Parse more env vars in Py_Main():
* Add more options to _PyCoreConfig:
* faulthandler
* tracemalloc
* importtime
* Move code to parse environment variables from _Py_InitializeCore()
to Py_Main(). This change fixes a regression from Python 3.6:
PYTHONUNBUFFERED is now read before calling pymain_init_stdio().
* _PyFaulthandler_Init() and _PyTraceMalloc_Init() now take an
argument to decide if the module has to be enabled at startup.
* tracemalloc_start() is now responsible to check the maximum number
of frames.
Other changes:
* Cleanup Py_Main():
* Rename some pymain_xxx() subfunctions
* Add pymain_run_python() subfunction
* Cleanup Py_NewInterpreter()
* _PyInterpreterState_Enable() now reports failure
* init_hash_secret() now considers pyurandom() failure as an "user
error": don't fail with abort().
* pymain_optlist_append() and pymain_strdup() now sets err on memory
allocation failure.
* Don't use "Python runtime" anymore to parse command line options or
to get environment variables: pymain_init() is now a strict
separation.
* Use an error message rather than "crashing" directly with
Py_FatalError(). Limit the number of calls to Py_FatalError(). It
prepares the code to handle errors more nicely later.
* Warnings options (-W, PYTHONWARNINGS) and "XOptions" (-X) are now
only added to the sys module once Python core is properly
initialized.
* _PyMain is now the well identified owner of some important strings
like: warnings options, XOptions, and the "program name". The
program name string is now properly freed at exit.
pymain_free() is now responsible to free the "command" string.
* Rename most methods in Modules/main.c to use a "pymain_" prefix to
avoid conflits and ease debug.
* Replace _Py_CommandLineDetails_INIT with memset(0)
* Reorder a lot of code to fix the initialization ordering. For
example, initializing standard streams now comes before parsing
PYTHONWARNINGS.
* Py_Main() now handles errors when adding warnings options and
XOptions.
* Add _PyMem_GetDefaultRawAllocator() private function.
* Cleanup _PyMem_Initialize(): remove useless global constants: move
them into _PyMem_Initialize().
* Call _PyRuntime_Initialize() as soon as possible:
_PyRuntime_Initialize() now returns an error message on failure.
* Add _PyInitError structure and following macros:
* _Py_INIT_OK()
* _Py_INIT_ERR(msg)
* _Py_INIT_USER_ERR(msg): "user" error, don't abort() in that case
* _Py_INIT_FAILED(err)
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.
Same change for MB and GB which become MiB and GiB.
Change the output of Tools/iobench/iobench.py.
Round also the size of the documentation from 5.5 MB to 5 MiB.
Use the _PyTime_t type rather than double for the faulthandler
timeout in dump_traceback_later().
This change should fix the following Coverity warning:
CID 1420311: Incorrect expression (UNINTENDED_INTEGER_DIVISION)
Dividing integer expressions "9223372036854775807LL" and "1000LL",
and then converting the integer quotient to type "double". Any
remainder, or fractional part of the quotient, is ignored.
if ((timeout * 1e6) >= (double) PY_TIMEOUT_MAX) {
The warning comes from (double)PY_TIMEOUT_MAX with:
#define PY_TIMEOUT_MAX (PY_LLONG_MAX / 1000)
See PEP 539 for details.
Highlights of changes:
- Add Thread Specific Storage (TSS) API
- Document the Thread Local Storage (TLS) API as deprecated
- Update code that used TLS API to use TSS API
* bpo-30557: faulthandler now correctly filters and displays exception codes on Windows
* Adds test for non-fatal exceptions.
* Adds bpo number to comment.
* bpo-30125: Cleanup faulthandler.c
* Use size_t type for iterators
* Add { ... }
* bpo-30125: Fix faulthandler.disable() on Windows
On Windows, faulthandler.disable() now removes the exception handler
installed by faulthandler.enable().
* bpo-6532: Make the thread id an unsigned integer.
From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".
* Restore a check in thread_get_ident().
Issue #23848, #26622:
* faulthandler now only logs fatal Windows exceptions.
* write error code as decimal, not as hexadecimal
* replace "Windows exception" with "Windows fatal exception"
Issue #23848: On Windows, faulthandler.enable() now also installs an exception
handler to dump the traceback of all Python threads on any Windows exception,
not only on UNIX signals (SIGSEGV, SIGFPE, SIGABRT).
Issue #26563:
* Add _PyGILState_GetInterpreterStateUnsafe() function: the single
PyInterpreterState used by this process' GILState implementation.
* Enhance _Py_DumpTracebackThreads() to retrieve the interpreter state from
autoInterpreterState in last resort. The function now accepts NULL for interp
and current_tstate parameters.
* test_faulthandler: fix a ResourceWarning when test is interrupted by CTRL+c
Issue #26154: Add a new private _PyThreadState_UncheckedGet() function which
gets the current thread state, but don't call Py_FatalError() if it is NULL.
Python 3.5.1 removed the _PyThreadState_Current symbol from the Python C API to
no more expose complex and private atomic types. Atomic types depends on the
compiler or can even depend on compiler options. The new function
_PyThreadState_UncheckedGet() allows to get the variable value without having
to care of the exact implementation of atomic types.
Changes:
* Replace direct usage of the _PyThreadState_Current variable with a call to
_PyThreadState_UncheckedGet().
* In pystate.c, replace direct usage of the _PyThreadState_Current variable
with the PyThreadState_GET() macro for readability.
* Document also PyThreadState_Get() in pystate.h
to its signal handlers.
Use also _Py_write_noraise() instead of write() to retry write() if it is
interrupted by a signal (fail with EINTR).
faulthandler.dump_traceback() also calls PyErr_CheckSignals() to call the
Python signal handler if a signal was received.
Issue #23654: Turn off ICC's tail call optimization for the stack_overflow
generator. ICC turns the recursive tail call into a loop.
Patch written by Matt Frank.
_read_null(), because _read_null() cannot be used on AIX. On AIX, reading from
NULL is allowed: the first page of memory is a mapped read-only on AIX.
_read_null() and _sigabrt() don't accept parameters.
- Use _testcapi.raise_signal() in test_signal
- close also os.pipe() file descriptors in some test_signal tests where they
were not closed properly
- Remove faulthandler._sigill() and faulthandler._sigbus(): reuse
_testcapi.raise_signal() in test_faulthandler
Gregory P. Smith
Mike
Victor Stinner
Masayuki Yamamoto
Antoine Pitrou
Steve Dower
Christophe Zeitouny
Serhiy Storchaka
Benjamin Peterson
doko@ubuntu.com
R David Murray