Commit Graph

116 Commits

Author SHA1 Message Date
Victor Stinner 62183b8d6d
bpo-40268: Remove explicit pythread.h includes (#19529)
Remove explicit pythread.h includes: it is always included
by Python.h.
2020-04-15 02:04:42 +02:00
Andy Lester 7668a8bc93
Use calloc-based functions, not malloc. (GH-19152) 2020-03-24 23:26:44 -05:00
Victor Stinner 8fb02b6e19
bpo-39947: Add PyThreadState_GetInterpreter() (GH-18981)
Add PyThreadState_GetInterpreter(tstate): get the interpreter of a
Python thread state.
2020-03-13 23:38:08 +01:00
Victor Stinner 2a4903fcce
bpo-38631: Add _Py_NO_RETURN to functions calling Py_FatalError() (GH-18278)
Add _Py_NO_RETURN to functions calling Py_FatalError():

* _PyObject_AssertFailed()
* dummy_dealloc()
* faulthandler_fatal_error_thread()
* none_dealloc()
* notimplemented_dealloc()
2020-01-30 13:09:11 +01:00
Victor Stinner 8b787964e0
bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 (GH-17467)
Use the "volatile" keyword to prevent tail call optimization
on any compiler, rather than relying on compiler specific pragma.
2019-12-04 21:10:06 +01:00
Brandt Bucher ac2235432c bpo-38823: Fix refleaks in faulthandler init error path on Windows (GH-17250) 2019-11-20 00:13:05 +01:00
Victor Stinner 0a963fbc9c
bpo-38203: faulthandler.dump_traceback_later() is always available (GH-16249)
dump_traceback_later() and cancel_dump_traceback_later() functions of
the faulthandler module are always available since Python 3.7.
2019-09-18 14:15:10 +02:00
Thomas A Caswell e278335a6e bpo-37933: Fix faulthandler.cancel_dump_traceback_later() (GH-15440)
Fix faulthandler.cancel_dump_traceback_later() call
if cancel_dump_traceback_later() was not called previously.
2019-08-29 18:30:04 +02:00
Victor Stinner d8c5adf6f8
bpo-37851: faulthandler allocates its stack on demand (GH-15358)
The faulthandler module no longer allocates its alternative stack at
Python startup. Now the stack is only allocated at the first
faulthandler usage.

faulthandler no longer ignores memory allocation failure when
allocating the stack. sigaltstack() failure now raises an OSError
exception, rather than being ignored.

The alternative stack is no longer used if sigaction() is
not available. In practice, sigaltstack() should only be available
when sigaction() is avaialble, so this change should have no effect
in practice.

faulthandler.dump_traceback_later() internal locks are now only
allocated at the first dump_traceback_later() call, rather than
always being allocated at Python startup.
2019-08-21 13:40:42 +01:00
Victor Stinner ac827edc49
bpo-21131: Fix faulthandler.register(chain=True) stack (GH-15276)
faulthandler now allocates a dedicated stack of SIGSTKSZ*2 bytes,
instead of just SIGSTKSZ bytes. Calling the previous signal handler
in faulthandler signal handler uses more than SIGSTKSZ bytes of stack
memory on some platforms.
2019-08-14 23:35:27 +02:00
Jeroen Demeyer 762f93ff2e bpo-37337: Add _PyObject_CallMethodNoArgs() (GH-14267) 2019-07-08 17:19:25 +09:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Victor Stinner ed48866c55
bpo-35134: Split traceback.h header (GH-13430)
Add new Include/cpython/traceback.h and Include/internal/traceback.h
header files.
2019-05-20 00:14:57 +02:00
Victor Stinner 871ff77c1c
bpo-36763: Add _PyInitError functions (GH-13395)
* Add _PyInitError functions:

  * _PyInitError_Ok()
  * _PyInitError_Error()
  * _PyInitError_NoMemory()
  * _PyInitError_Exit()
  * _PyInitError_IsError()
  * _PyInitError_IsExit()
  * _PyInitError_Failed()

* frozenmain.c and _testembed.c now use functions rather than macros.
* Move _Py_INIT_xxx() macros to the internal API.
* Move _PyWstrList_INIT macro to the internal API.
2019-05-17 23:54:00 +02:00
Xi Ruoyao 6236c9823e bpo-36856: Handle possible overflow in faulthandler_stack_overflow (GH-13205) 2019-05-11 19:13:23 +02:00
Victor Stinner b84cb70880
bpo-36734: Fix compilation of faulthandler.c on HP-UX (GH-12970)
Initialize "stack_t current_stack" to zero using memset().
2019-04-30 12:19:34 +02:00
Serhiy Storchaka 62be74290a
bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749)
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
2018-11-27 13:27:31 +02:00
Gregory P. Smith 1584a00815
bpo-35214: Initial clang MemorySanitizer support (GH-10479)
Adds configure flags for msan and ubsan builds to make it easier to enable.
These also encode the detail that address sanitizer and memory sanitizer
should disable pymalloc.

Define MEMORY_SANITIZER when appropriate at build time and adds workarounds
to existing code to mark things as initialized where the sanitizer is otherwise unable to
determine that.  This lets our build succeed under the memory sanitizer.  not all tests
pass without sanitizer failures yet but we're in pretty good shape after this.
2018-11-12 12:07:14 -08:00
Siddhesh Poyarekar 55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
Mike 53f7a7c281 bpo-32297: Few misspellings found in Python source code comments. (#4803)
* Fix multiple typos in code comments

* Add spacing in comments (test_logging.py, test_math.py)

* Fix spaces at the beginning of comments in test_logging.py
2017-12-14 13:04:53 +02:00
Victor Stinner 48d4dd974f
bpo-32252: Fix faulthandler_suppress_crash_report() (#4794)
Fix faulthandler_suppress_crash_report() used to prevent core dump files
when testing crashes. getrlimit() returns zero on success.
2017-12-11 13:57:12 +01:00
Victor Stinner a7368ac636
bpo-32030: Enhance Py_Main() (#4412)
Parse more env vars in Py_Main():

* Add more options to _PyCoreConfig:

  * faulthandler
  * tracemalloc
  * importtime

* Move code to parse environment variables from _Py_InitializeCore()
  to Py_Main(). This change fixes a regression from Python 3.6:
  PYTHONUNBUFFERED is now read before calling pymain_init_stdio().
* _PyFaulthandler_Init() and _PyTraceMalloc_Init() now take an
  argument to decide if the module has to be enabled at startup.
* tracemalloc_start() is now responsible to check the maximum number
  of frames.

Other changes:

* Cleanup Py_Main():

  * Rename some pymain_xxx() subfunctions
  * Add pymain_run_python() subfunction

* Cleanup Py_NewInterpreter()
* _PyInterpreterState_Enable() now reports failure
* init_hash_secret() now considers pyurandom() failure as an "user
  error": don't fail with abort().
* pymain_optlist_append() and pymain_strdup() now sets err on memory
  allocation failure.
2017-11-15 18:11:45 -08:00
Victor Stinner f7e5b56c37
bpo-32030: Split Py_Main() into subfunctions (#4399)
* Don't use "Python runtime" anymore to parse command line options or
  to get environment variables: pymain_init() is now a strict
  separation.
* Use an error message rather than "crashing" directly with
  Py_FatalError(). Limit the number of calls to Py_FatalError(). It
  prepares the code to handle errors more nicely later.
* Warnings options (-W, PYTHONWARNINGS) and "XOptions" (-X) are now
  only added to the sys module once Python core is properly
  initialized.
* _PyMain is now the well identified owner of some important strings
  like: warnings options, XOptions, and the "program name". The
  program name string is now properly freed at exit.
  pymain_free() is now responsible to free the "command" string.
* Rename most methods in Modules/main.c to use a "pymain_" prefix to
  avoid conflits and ease debug.
* Replace _Py_CommandLineDetails_INIT with memset(0)
* Reorder a lot of code to fix the initialization ordering. For
  example, initializing standard streams now comes before parsing
  PYTHONWARNINGS.
* Py_Main() now handles errors when adding warnings options and
  XOptions.
* Add _PyMem_GetDefaultRawAllocator() private function.
* Cleanup _PyMem_Initialize(): remove useless global constants: move
  them into _PyMem_Initialize().
* Call _PyRuntime_Initialize() as soon as possible:
  _PyRuntime_Initialize() now returns an error message on failure.
* Add _PyInitError structure and following macros:

  * _Py_INIT_OK()
  * _Py_INIT_ERR(msg)
  * _Py_INIT_USER_ERR(msg): "user" error, don't abort() in that case
  * _Py_INIT_FAILED(err)
2017-11-15 15:48:08 -08:00
Victor Stinner 8c663fd60e
Replace KB unit with KiB (#4293)
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.

Same change for MB and GB which become MiB and GiB.

Change the output of Tools/iobench/iobench.py.

Round also the size of the documentation from 5.5 MB to 5 MiB.
2017-11-08 14:44:44 -08:00
Victor Stinner 93fd478231 faulthandler: use _PyTime_t rather than double for timeout (#4139)
Use the _PyTime_t type rather than double for the faulthandler
timeout in dump_traceback_later().

This change should fix the following Coverity warning:

CID 1420311:  Incorrect expression  (UNINTENDED_INTEGER_DIVISION)
Dividing integer expressions "9223372036854775807LL" and "1000LL",
and then converting the integer quotient to type "double". Any
remainder, or fractional part of the quotient, is ignored.

    if ((timeout * 1e6) >= (double) PY_TIMEOUT_MAX) {

The warning comes from (double)PY_TIMEOUT_MAX with:

    #define PY_TIMEOUT_MAX (PY_LLONG_MAX / 1000)
2017-10-27 07:27:12 -07:00
Victor Stinner 6e3d6b5dc2 bpo-31701: faulthandler: ignore MSC and COM Windows exception (#3929)
bpo-31701: On Windows, faulthandler.enable() now ignores MSC and COM
exceptions.
2017-10-09 09:52:32 -07:00
Masayuki Yamamoto 731e189014 bpo-25658: Implement PEP 539 for Thread Specific Storage (TSS) API (GH-1362)
See PEP 539 for details.

Highlights of changes:

- Add Thread Specific Storage (TSS) API
- Document the Thread Local Storage (TLS) API as deprecated
- Update code that used TLS API to use TSS API
2017-10-06 20:41:34 +10:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Steve Dower e6a23c8f9a bpo-30557: faulthandler now correctly filters and displays exception codes on Windows (#1924)
* bpo-30557: faulthandler now correctly filters and displays exception codes on Windows

* Adds test for non-fatal exceptions.

* Adds bpo number to comment.
2017-06-05 15:54:15 -07:00
Victor Stinner 46c2b81026 bpo-30125: Fix faulthandler.disable() on Windows (#1240)
* bpo-30125: Cleanup faulthandler.c

* Use size_t type for iterators
* Add { ... }

* bpo-30125: Fix faulthandler.disable() on Windows

On Windows, faulthandler.disable() now removes the exception handler
installed by faulthandler.enable().
2017-04-21 18:06:13 +02:00
Christophe Zeitouny 20fbf8accd faulthandler: Restore the old sigaltstack during teardown (#777) 2017-03-23 18:14:29 +01:00
Serhiy Storchaka aefa7ebf0f bpo-6532: Make the thread id an unsigned integer. (#781)
* bpo-6532: Make the thread id an unsigned integer.

From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".

* Restore a check in thread_get_ident().
2017-03-23 14:48:39 +01:00
Serhiy Storchaka 228b12edcc Issue #28999: Use Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE wherever
possible.  Patch is writen with Coccinelle.
2017-01-23 09:47:21 +02:00
Victor Stinner 9a2329f9e1 Issue #28152: Fix -Wunreachable-code warnings on Clang
Don't declare dead code when the code is declared with Clang.
2016-12-05 17:56:36 +01:00
Steve Dower 940f33a50f Issue #23524: Finish removing _PyVerify_fd from sources 2016-09-08 11:21:54 -07:00
Benjamin Peterson ca47063998 replace Py_(u)intptr_t with the c99 standard types 2016-09-06 13:47:26 -07:00
Victor Stinner 3466bde1cc Avoid calling functions with an empty string as format string
Directly pass NULL rather than an empty string.
2016-09-05 18:16:01 -07:00
doko@ubuntu.com bc7315068f - make some internal symbols static 2016-05-18 01:06:01 +02:00
Serhiy Storchaka b6a9c9761c Issue #26778: Fixed "a/an/and" typos in code comment, documentation and error
messages.
2016-04-17 09:39:28 +03:00
Serhiy Storchaka 6a7b3a77b4 Issue #26778: Fixed "a/an/and" typos in code comment and documentation. 2016-04-17 08:32:47 +03:00
Serhiy Storchaka ec39756960 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:50:03 +03:00
Serhiy Storchaka 48842714b9 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:45:48 +03:00
Victor Stinner 82d44f0598 Issue #23848: Fix usage of _Py_DumpDecimal() 2016-03-23 18:37:54 +01:00
Victor Stinner 928ad285b5 Issue #23848: Try to fix test_faulthandler on ARM
Restore the previous code for stack_overflow().
2016-03-23 15:19:12 +01:00
Victor Stinner 412a5e7e23 faulthandler: only log fatal exceptions
Issue #23848, #26622:

* faulthandler now only logs fatal Windows exceptions.
* write error code as decimal, not as hexadecimal
* replace "Windows exception" with "Windows fatal exception"
2016-03-23 14:44:14 +01:00
Victor Stinner 404cdc5a92 faulthandler: add Windows exception handler
Issue #23848: On Windows, faulthandler.enable() now also installs an exception
handler to dump the traceback of all Python threads on any Windows exception,
not only on UNIX signals (SIGSEGV, SIGFPE, SIGABRT).
2016-03-23 10:39:17 +01:00
Victor Stinner 861d9abfcf faulthandler now works in non-Python threads
Issue #26563:

* Add _PyGILState_GetInterpreterStateUnsafe() function: the single
  PyInterpreterState used by this process' GILState implementation.
* Enhance _Py_DumpTracebackThreads() to retrieve the interpreter state from
  autoInterpreterState in last resort. The function now accepts NULL for interp
  and current_tstate parameters.
* test_faulthandler: fix a ResourceWarning when test is interrupted by CTRL+c
2016-03-16 22:45:24 +01:00
Victor Stinner 57003f81ea faulthandler: Test Py_FatalError() with GIL released
Issue #26558.
2016-03-15 17:23:35 +01:00
Victor Stinner bfd316e750 Add _PyThreadState_UncheckedGet()
Issue #26154: Add a new private _PyThreadState_UncheckedGet() function which
gets the current thread state, but don't call Py_FatalError() if it is NULL.

Python 3.5.1 removed the _PyThreadState_Current symbol from the Python C API to
no more expose complex and private atomic types. Atomic types depends on the
compiler or can even depend on compiler options. The new function
_PyThreadState_UncheckedGet() allows to get the variable value without having
to care of the exact implementation of atomic types.

Changes:

* Replace direct usage of the _PyThreadState_Current variable with a call to
  _PyThreadState_UncheckedGet().
* In pystate.c, replace direct usage of the _PyThreadState_Current variable
  with the PyThreadState_GET() macro for readability.
* Document also PyThreadState_Get() in pystate.h
2016-01-20 11:12:38 +01:00
Serhiy Storchaka 5a57ade58e Issue #20440: Massive replacing unsafe attribute setting code with special
macro Py_SETREF.
2015-12-24 10:35:59 +02:00