Commit Graph

278 Commits

Author SHA1 Message Date
Dennis Sweeney 3289209716
Fix refleaks in PyErr_SetHandledException (GH-91627) 2022-04-17 02:52:53 -04:00
Irit Katriel 5d421d7342
gh-90501: Add PyErr_GetHandledException and PyErr_SetHandledException (GH-30531) 2022-04-15 19:57:47 +01:00
Oleg Iarygin 4060111f9d
bpo-46816: Remove declarations for non-__STDC__ compilers (GH-31466)
After Python switched to C11, there is a guarantee that every compiler
conforms to ISO C so checks of __STDC__ have no sense.
2022-02-26 00:16:59 +01:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Victor Stinner 1781d55eb3
bpo-46417: _curses uses PyStructSequence_NewType() (GH-30736)
The _curses module now creates its ncurses_version type as a heap
type using PyStructSequence_NewType(), rather than using a static
type.

* Move _PyStructSequence_FiniType() definition to pycore_structseq.h.
* test.pythoninfo: log curses.ncurses_version.
2022-01-21 03:30:20 +01:00
Victor Stinner e9e3eab0b8
bpo-46417: Finalize structseq types at exit (GH-30645)
Add _PyStructSequence_FiniType() and _PyStaticType_Dealloc()
functions to finalize a structseq static type in Py_Finalize().
Currrently, these functions do nothing if Python is built in release
mode.

Clear static types:

* AsyncGenHooksType: sys.set_asyncgen_hooks()
* FlagsType: sys.flags
* FloatInfoType: sys.float_info
* Hash_InfoType: sys.hash_info
* Int_InfoType: sys.int_info
* ThreadInfoType: sys.thread_info
* UnraisableHookArgsType: sys.unraisablehook
* VersionInfoType: sys.version
* WindowsVersionType: sys.getwindowsversion()
2022-01-21 01:42:25 +01:00
Irit Katriel 396b58345f
bpo-45711: Remove type and traceback from exc_info (GH-30122)
* Do not PUSH/POP traceback or type to the stack as part of exc_info

* Remove exc_traceback and exc_type from _PyErr_StackItem

* Add to what's new, because this change breaks things like Cython
2021-12-17 14:46:22 +00:00
Eric Snow c8749b5783
bpo-46008: Make runtime-global object/type lifecycle functions and state consistent. (gh-29998)
This change is strictly renames and moving code around.  It helps in the following ways:

* ensures type-related init functions focus strictly on one of the three aspects (state, objects, types)
* passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter
* consistent naming conventions help make what's going on more clear
* keeping API related to a type in the corresponding header file makes it more obvious where to look for it

https://bugs.python.org/issue46008
2021-12-09 12:59:26 -07:00
Irit Katriel 2109f7880b
bpo-45711: Remove unnecessary normalization of exc_info (GH-29922) 2021-12-08 20:08:06 +00:00
Irit Katriel 8a45ca542a
bpo-45711: Change exc_info related APIs to derive type and traceback from the exception instance (GH-29780) 2021-11-30 22:37:04 +00:00
Irit Katriel c456dfafe9
bpo-45711: use exc_value instead of exc_type to determine if exc_info is valid. Add more assertions. (GH-29627) 2021-11-25 09:41:28 +00:00
Pablo Galindo Salgado fdcc46d955
bpo-45848: Allow the parser to get error lines from encoded files (GH-29646) 2021-11-20 15:36:07 +01:00
Victor Stinner aac29af678
bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
Include <stdlib.h> explicitly in C files.

Python.h includes <wchar.h>.
2021-10-13 19:25:53 +02:00
Victor Stinner d943d19172
bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
  PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
  defined to get access to internal _PyObject_CallNoArgs().
2021-10-12 08:38:19 +02:00
Victor Stinner ce3489cfdb
bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
2021-10-12 00:42:23 +02:00
Irit Katriel 9e31b3952f
bpo-41031: Match C and Python code formatting of unprintable exceptions and exceptions in the __main__ module. (GH-28139) 2021-09-05 18:54:13 +03:00
Irit Katriel b4b6342848
bpo-45083: Include the exception class qualname when formatting an exception (GH-28119)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-09-03 09:30:17 +02:00
Irit Katriel d5c217475c
bpo-25782: avoid hang in PyErr_SetObject when current exception has a cycle in its context chain (GH-27626)
Co-authored-by: Dennis Sweeney 36520290+sweeneyde@users.noreply.github.com
2021-08-10 11:37:25 +02:00
Noah 83ca46b778
closes bpo-39091: Fix segfault when Exception constructor returns non-exception for gen.throw. (#17658)
Co-authored-by: Benjamin Peterson <benjamin@python.org>
2021-08-02 19:17:18 -07:00
Mark Shannon ae0a2b7562
bpo-44590: Lazily allocate frame objects (GH-27077)
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.

* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.

* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.

* Move ownership of InterpreterFrame in generator from frame object to generator object.

* Do not create frame objects for Python calls.

* Do not create frame objects for generators.
2021-07-26 11:22:16 +01:00
Inada Naoki ae3c66acb8
bpo-44094: Remove deprecated PyErr_ APIs. (GH-26011)
These APIs are deprecated since Python 3.3. They are not documented too.
2021-05-13 13:45:26 -07:00
Pablo Galindo a77aac4fca
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525)
To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way:

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^
SyntaxError: Generator expression must be parenthesized

becomes

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
2021-04-23 14:27:05 +01:00
Victor Stinner 442ad74fc2
bpo-43687: Py_Initialize() creates singletons earlier (GH-25147)
Reorganize pycore_interp_init() to initialize singletons before the
the first PyType_Ready() call. Fix an issue when Python is configured
using --without-doc-strings.
2021-04-02 15:28:13 +02:00
Victor Stinner 815506d852
bpo-32381: Rewrite PyErr_ProgramText() (GH-23700)
PyErr_ProgramText() now calls PyErr_ProgramTextObject().
2020-12-08 23:51:26 +01:00
Hai Shi 0f91f586ae
bpo-42599: Remove useless PyModule_GetWarningsModule() (GH-23691)
Removed PyModule_GetWarningsModule() which is useless due to 
the _warnings module was converted to a builtin module in 2.6.
2020-12-08 15:42:42 +01:00
Mark Shannon 4e7a69bdb6
bpo-42500: Fix recursion in or after except (GH-23568)
* Use counter, rather boolean state when handling soft overflows.
2020-12-02 13:30:55 +00:00
Serhiy Storchaka b510e101f8
bpo-42152: Use PyDict_Contains and PyDict_SetDefault if appropriate. (GH-22986)
If PyDict_GetItemWithError is only used to check whether the key is in dict,
it is better to use PyDict_Contains instead.

And if it is used in combination with PyDict_SetItem, PyDict_SetDefault can
replace the combination.
2020-10-26 12:47:57 +02:00
Serhiy Storchaka 98c4433a81
bpo-41991: Remove _PyObject_HasAttrId (GH-22629)
It can silence arbitrary exceptions.
2020-10-10 22:23:42 +03:00
Lysandros Nikolaou 113e2b0a07
bpo-40985: Show correct SyntaxError text when last line has a LINECONT (GH-20888)
When a file ends with a line that contains a line continuation character
the text of the emitted SyntaxError is empty, contrary to the old
parser, where the error text contained the text of the last line.
2020-06-16 01:27:33 +01:00
Hai Shi 1c209e3b53
Remove redundant var in PyErr_NewException() (GH-20850) 2020-06-13 15:58:12 +09:00
Victor Stinner 3026cad59b
bpo-40826: Add _Py_EnsureTstateNotNULL() macro (GH-20571)
Add _Py_EnsureTstateNotNULL(tstate) macro: call Py_FatalError() if
tstate is NULL, the error message contains the current function name.
2020-06-01 16:02:40 +02:00
Chris Jerdonek 7c30d12bd5
bpo-40696: Fix a hang that can arise after gen.throw() (GH-20287)
This updates _PyErr_ChainStackItem() to use _PyErr_SetObject()
instead of _PyErr_ChainExceptions(). This prevents a hang in
certain circumstances because _PyErr_SetObject() performs checks
to prevent cycles in the exception context chain while
_PyErr_ChainExceptions() doesn't.
2020-05-22 13:33:27 -07:00
Chris Jerdonek da742ba826
bpo-31033: Improve the traceback for cancelled asyncio tasks (GH-19951)
When an asyncio.Task is cancelled, the exception traceback now
starts with where the task was first interrupted.  Previously,
the traceback only had "depth one."
2020-05-17 22:47:31 -07:00
Victor Stinner b0be6b3b94
bpo-29587: _PyErr_ChainExceptions() checks exception (GH-19902)
_PyErr_ChainExceptions() now ensures that the first parameter is an
exception type, as done by _PyErr_SetObject().

* The following function now check PyExceptionInstance_Check() in an
  assertion using a new _PyBaseExceptionObject_cast() helper
  function:

  * PyException_GetTraceback(), PyException_SetTraceback()
  * PyException_GetCause(), PyException_SetCause()
  * PyException_GetContext(), PyException_SetContext()

* PyExceptionClass_Name() now checks PyExceptionClass_Check() with an
  assertion.

* Remove XXX comment and add gi_exc_state variable to _gen_throw().

* Remove comment from test_generators
2020-05-05 17:07:41 +02:00
Victor Stinner 4386b9045e
bpo-40429: PyThreadState_GetFrame() returns a strong ref (GH-19781)
The PyThreadState_GetFrame() function now returns a strong reference
to the frame.
2020-04-29 03:01:43 +02:00
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Victor Stinner 1c1e68cf3e
bpo-38644: Use _PySys_Audit(): pass tstate explicitly (GH-19183)
Add the dependency to tstate more explicit.
2020-03-27 15:11:45 +01:00
Andy Lester 557287075c
bpo-39573: Use Py_IS_TYPE() macro to check for types (GH-18809)
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-03-06 23:53:17 +01:00
Andy Lester dffe4c0709
bpo-39573: Finish converting to new Py_IS_TYPE() macro (GH-18601) 2020-03-04 14:15:20 +01:00
Andy Lester 7386a70746
closes bpo-39630: Update pointers to string literals to be const char *. (GH-18510) 2020-02-13 20:42:56 -08:00
Petr Viktorin ffd9753a94
bpo-39245: Switch to public API for Vectorcall (GH-18460)
The bulk of this patch was generated automatically with:

    for name in \
        PyObject_Vectorcall \
        Py_TPFLAGS_HAVE_VECTORCALL \
        PyObject_VectorcallMethod \
        PyVectorcall_Function \
        PyObject_CallOneArg \
        PyObject_CallMethodNoArgs \
        PyObject_CallMethodOneArg \
    ;
    do
        echo $name
        git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
    done

    old=_PyObject_FastCallDict
    new=PyObject_VectorcallDict
    git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"

and then cleaned up:

- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
2020-02-11 17:46:57 +01:00
Hai Shi 46874c26ee
bpo-39487: Merge duplicated _Py_IDENTIFIER identifiers in C code (GH-18254)
Moving repetitive `_Py_IDENTIFIER` instances to a global location helps identify them more easily in regards to sub-interpreter support.
2020-01-30 15:20:25 -08:00
Victor Stinner 3f12ac18a4
bpo-39164: Fix compiler warning in PyErr_GetExcInfo() (GH-18010)
The function has no return value.
2020-01-15 11:23:25 +01:00
Julien Danjou 3430c55417 bpo-39164: Add private _PyErr_GetExcInfo() function (GH-17752)
This adds a new function named _PyErr_GetExcInfo() that is a variation of the
original PyErr_GetExcInfo() taking a PyThreadState as its first argument.
That function allows to retrieve the exceptions information of any Python
thread -- not only the current one.
2020-01-13 17:30:14 +01:00
Steve Dower bea33f5e1d
bpo-38920: Add audit hooks for when sys.excepthook and sys.unraisable hooks are invoked (GH-17392)
Also fixes some potential segfaults in unraisable hook handling.
2019-11-28 08:46:11 -08:00
Victor Stinner d12d0e7c0f
bpo-38733: PyErr_Occurred() caller must hold the GIL (GH-17080)
bpo-3605, bpo-38733: Optimize _PyErr_Occurred(): remove "tstate ==
NULL" test.

Py_FatalError() no longer calls PyErr_Occurred() if called without
holding the GIL. So PyErr_Occurred() no longer has to support
tstate==NULL case.

_Py_CheckFunctionResult(): use directly _PyErr_Occurred() to avoid
explicit "!= NULL" test.
2019-11-07 12:42:07 +01:00
Victor Stinner 1726909094
bpo-38644: Pass tstate to _Py_CheckFunctionResult() (GH-17050)
* Add tstate parameter to _Py_CheckFunctionResult()
* Add _PyErr_FormatFromCauseTstate()
* Replace PyErr_XXX(...) with _PyErr_XXX(state, ...)
2019-11-05 01:22:12 +01:00
Jeroen Demeyer 762f93ff2e bpo-37337: Add _PyObject_CallMethodNoArgs() (GH-14267) 2019-07-08 17:19:25 +09:00
Jeroen Demeyer 196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Victor Stinner 838f26402d
bpo-36710: Pass explicitly tstate in sysmodule.c (GH-14060)
* Replace global var Py_VerboseFlag with interp->config.verbose.
* Add _PyErr_NoMemory(tstate) function.
* Add tstate parameter to _PyEval_SetCoroutineOriginTrackingDepth()
  and move the function to the internal API.
* Replace _PySys_InitMain(runtime, interp)
  with _PySys_InitMain(runtime, tstate).
2019-06-13 22:41:23 +02:00