Commit Graph

205 Commits

Author SHA1 Message Date
Serhiy Storchaka 3a8c1ca7e7
gh-117764: Fix and add signatures for many builtins (GH-117769) 2024-04-12 13:56:41 +03:00
Irit Katriel 1d5479b236
gh-117411: move PyFutureFeatures to pycore_symtable.h and make it private (#117412) 2024-04-02 10:34:49 +00:00
Mark Shannon 17b73ab99e
GH-113655: Lower the C recursion limit on various platforms (GH-113944) 2024-01-16 09:32:01 +00:00
Serhiy Storchaka 06c47a305d
Remove dead code left after gh-110721 (#111905) 2023-11-10 22:30:01 +00:00
Serhiy Storchaka a12f624a9d
Remove unnecessary includes (GH-111633) 2023-11-02 10:42:58 +02:00
Pablo Galindo Salgado abb15420c1
gh-109181: Speed up Traceback object creation by lazily compute the line number (#111548)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2023-10-31 15:02:31 +00:00
Pablo Galindo Salgado e7331365b4
gh-110721: Use the traceback module for PyErr_Display() and fallback to the C implementation (#110702) 2023-10-12 14:52:14 +00:00
Lysandros Nikolaou 17d65547df
gh-104169: Fix test_peg_generator after tokenizer refactoring (#110727)
* Fix test_peg_generator after tokenizer refactoring
* Remove references to tokenizer.c in comments etc.
2023-10-12 09:34:35 +02:00
Victor Stinner 7513994c92
gh-110014: Include explicitly <unistd.h> header (#110155)
* Remove unused <locale.h> includes.
* Remove unused <fcntl.h> include in traceback.h.
* Remove redundant <assert.h> and <stddef.h> includes. They  are already
  included by "Python.h".
* Remove <object.h> include in faulthandler.c. Python.h already includes it.
* Add missing <stdbool.h> in pycore_pythread.h if HAVE_PTHREAD_STUBS
  is defined.
* Fix also warnings in pthread_stubs.h: don't redefine macros if they
  are already defined, like the __NEED_pthread_t macro.
2023-09-30 20:06:45 +00:00
Victor Stinner 2e37a38bcb
gh-110052: Fix faulthandler for freed tstate (#110069)
faulthandler now detected freed interp and freed tstate, and no
longer dereference them.
2023-09-29 02:04:06 +00:00
Pablo Galindo Salgado 6275c67ea6
gh-106922: Fix error location for constructs with spaces and parentheses (#108959) 2023-09-08 17:18:35 +01:00
Victor Stinner b298b395e8
gh-108765: Cleanup #include in Python/*.c files (#108977)
Mention one symbol imported by each #include.
2023-09-06 15:56:08 +02:00
Victor Stinner a071ecb4d1
gh-106320: Remove private _PySys functions (#108452)
Move private functions to the internal C API (pycore_sysmodule.h):

* _PySys_GetAttr()
* _PySys_GetSizeOf()

No longer export most of these functions.

Fix also a typo in Include/cpython/optimizer.h: add a missing space.
2023-08-24 20:02:09 +00:00
Victor Stinner 6541fe4ad7
Ignore _Py_write_noraise() result: cast to (void) (#108291)
Code using _Py_write_noraise() usually cannot report. Ignore errors
is the least surprising behavior for users.
2023-08-22 14:28:20 +00:00
Mark Shannon 006e44f950
GH-108035: Remove the `_PyCFrame` struct as it is no longer needed for performance. (GH-108036) 2023-08-17 11:16:03 +01:00
Victor Stinner 1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Mark Shannon 7199584ac8
GH-100987: Allow objects other than code objects as the "executable" of an internal frame. (GH-105727)
* Add table describing possible executable classes for out-of-process debuggers.

* Remove shim code object creation code as it is no longer needed.

* Make lltrace a bit more robust w.r.t. non-standard frames.
2023-06-14 13:46:37 +01:00
Irit Katriel f990bb8b2d
gh-105148: make _PyASTOptimizeState internal to ast_opt.c (#105149) 2023-05-31 20:21:46 +01:00
Mark Shannon a0df9ee8fc
GH-96803: Add three C-API functions to make _PyInterpreterFrame less opaque for users of PEP 523. (GH-96849) 2023-05-05 17:53:07 +01:00
Irit Katriel 6372e290c0
gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (#102760) 2023-03-16 19:03:52 +00:00
Mark Shannon feec49c407
GH-101578: Normalize the current exception (GH-101607)
* Make sure that the current exception is always normalized.

* Remove redundant type and traceback fields for the current exception.

* Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException

* Add new API functions: PyException_GetArgs, PyException_SetArgs
2023-02-08 09:31:12 +00:00
Victor Stinner 135ec7cefb
gh-99537: Use Py_SETREF() function in C code (#99657)
Fix potential race condition in code patterns:

* Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
* Replace "Py_XDECREF(var); var = new;" with "Py_XSETREF(var, new);"
* Replace "Py_CLEAR(var); var = new;" with "Py_XSETREF(var, new);"

Other changes:

* Replace "old = var; var = new; Py_DECREF(var)"
  with "Py_SETREF(var, new);"
* Replace "old = var; var = new; Py_XDECREF(var)"
  with "Py_XSETREF(var, new);"
* And remove the "old" variable.
2022-11-22 13:39:11 +01:00
Batuhan Taskaya 57be545959
gh-99103: Normalize specialized traceback anchors against the current line (GH-99145)
Automerge-Triggered-By: GH:isidentical
2022-11-12 15:37:25 -08:00
Mark Shannon 1e197e63e2
GH-96421: Insert shim frame on entry to interpreter (GH-96319)
* Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault()

* Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
2022-11-10 12:34:57 +00:00
Victor Stinner 231d83b724
gh-99300: Use Py_NewRef() in Python/ directory (#99317)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Python/ directory.

Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
2022-11-10 11:23:36 +01:00
John Belmonte da717519ec
gh-93883: elide traceback indicators when possible (#93994)
* gh-93883: elide traceback indicators when possible

Elide traceback column indicators when the entire line of the
frame is implicated.  This reduces traceback length and draws
even more attention to the remaining (very relevant) indicators.

Example:
```
Traceback (most recent call last):
  File "query.py", line 99, in <module>
    bar()
  File "query.py", line 66, in bar
    foo()
  File "query.py", line 37, in foo
    magic_arithmetic('foo')
  File "query.py", line 18, in magic_arithmetic
    return add_counts(x) / 25
           ^^^^^^^^^^^^^
  File "query.py", line 24, in add_counts
    return 25 + query_user(user1) + query_user(user2)
                ^^^^^^^^^^^^^^^^^
  File "query.py", line 32, in query_user
    return 1 + query_count(db, response['a']['b']['c']['user'], retry=True)
                               ~~~~~~~~~~~~~~~~~~^^^^^
TypeError: 'NoneType' object is not subscriptable
```

Rather than going out of our way to provide indicator coverage
in every traceback test suite, the indicator test suite should
be responible for sufficient coverage (e.g. by adding a basic
exception group test to ensure that margin strings are covered).
2022-07-11 07:40:53 +01:00
Victor Stinner 27b9894033
gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938)
Move the follow functions and type from frameobject.h to pyframe.h,
so the standard <Python.h> provide frame getter functions:

* PyFrame_Check()
* PyFrame_GetBack()
* PyFrame_GetBuiltins()
* PyFrame_GetGenerator()
* PyFrame_GetGlobals()
* PyFrame_GetLasti()
* PyFrame_GetLocals()
* PyFrame_Type

Remove #include "frameobject.h" from many C files. It's no longer
needed.
2022-06-19 12:02:33 +02:00
Inada Naoki f9c9354a7a
gh-92536: PEP 623: Remove wstr and legacy APIs from Unicode (GH-92537) 2022-05-12 14:48:38 +09:00
Brandt Bucher ef6a482b02
bpo-47177: Replace `f_lasti` with `prev_instr` (GH-32208) 2022-04-07 12:31:01 -07:00
Victor Stinner 85addfb9c6
bpo-35134: Remove the Include/code.h header file (GH-32385)
Remove the Include/code.h header file. C extensions should only
include the main <Python.h> header file.

Python.h includes directly Include/cpython/code.h instead.
2022-04-07 02:29:52 +02:00
Victor Stinner f0bc694856
bpo-47164: Add _PyCFunction_CAST() macro (GH-32192)
Use the macro in C files of the Python/ directory.
2022-03-31 10:02:34 +02:00
Victor Stinner c14d7e4b81
bpo-47164: Add _PyASCIIObject_CAST() macro (GH-32191)
Add macros to cast objects to PyASCIIObject*, PyCompactUnicodeObject*
and PyUnicodeObject*: _PyASCIIObject_CAST(),
_PyCompactUnicodeObject_CAST() and _PyUnicodeObject_CAST(). Using
these new macros make the code more readable and check their argument
with: assert(PyUnicode_Check(op)).

Remove redundant assert(PyUnicode_Check(op)) in macros using directly
or indirectly these new CAST macros.

Replacing existing casts with these macros.
2022-03-31 09:59:27 +02:00
Victor Stinner 87af12bff3
bpo-46836: Rename InterpreterFrame to _PyInterpreterFrame (GH-31583)
Rename also struct _interpreter_frame to struct _PyInterpreterFrame.

Reduce risk of name conflicts if a project includes pycore_frame.h.
2022-02-25 16:22:00 +01:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Kumar Aditya 41026c3155
bpo-45855: Replaced deprecated `PyImport_ImportModuleNoBlock` with PyImport_ImportModule (GH-30046) 2021-12-12 10:45:20 +02:00
Irit Katriel d596acbd3b
bpo-45635: standardize error handling in traceback.c (GH-29905)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-12-07 16:17:22 +00:00
Victor Stinner b919d8105c
bpo-45831: _Py_DumpASCII() uses a single write() call if possible (GH-29596)
If the string is ASCII only and doesn't need to escape characters,
write the whole string with a single write() syscall.
2021-11-17 21:12:20 +01:00
Irit Katriel 3509b26c91
bpo-45292: [PEP 654] Update traceback display code to work with exception groups (GH-29207) 2021-11-05 09:39:18 +00:00
Mark Shannon f291404a80
bpo-45637: Store the frame pointer in the cframe (GH-29267)
* Rename 'frame' to 'current_frame'
2021-10-28 13:59:11 +01:00
Victor Stinner 713bb19356
bpo-45434: Mark the PyTokenizer C API as private (GH-28924)
Rename PyTokenize functions to mark them as private:

* PyTokenizer_FindEncodingFilename() => _PyTokenizer_FindEncodingFilename()
* PyTokenizer_FromString() => _PyTokenizer_FromString()
* PyTokenizer_FromFile() => _PyTokenizer_FromFile()
* PyTokenizer_FromUTF8() => _PyTokenizer_FromUTF8()
* PyTokenizer_Free() => _PyTokenizer_Free()
* PyTokenizer_Get() => _PyTokenizer_Get()

Remove the unused PyTokenizer_FindEncoding() function.

import.c: remove unused #include "errcode.h".
2021-10-13 17:22:14 +02:00
Victor Stinner 97308dfcdc
bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h (GH-28922) 2021-10-13 15:03:35 +02:00
Victor Stinner 7cdc2a0f4b
pycore_pystate.h no longer redefines PyThreadState_GET() (GH-28921)
Redefining the PyThreadState_GET() macro in pycore_pystate.h is
useless since it doesn't affect files not including it. Either use
_PyThreadState_GET() directly, or don't use pycore_pystate.h internal
C API. For example, the _testcapi extension don't use the internal C
API, but use the public PyThreadState_Get() function instead.

Replace PyThreadState_Get() with _PyThreadState_GET(). The
_PyThreadState_GET() macro is more efficient than PyThreadState_Get()
and PyThreadState_GET() function calls which call fail with a fatal
Python error.

posixmodule.c and _ctypes extension now include <windows.h> before
pycore header files (like pycore_call.h).

_PyTraceback_Add() now uses _PyErr_Fetch()/_PyErr_Restore() instead
of PyErr_Fetch()/PyErr_Restore().

The _decimal and _xxsubinterpreters extensions are now built with the
Py_BUILD_CORE_MODULE macro defined to get access to the internal C
API.
2021-10-13 14:09:13 +02:00
Serhiy Storchaka 60b9e040c9
bpo-45355: Use sizeof(_Py_CODEUNIT) instead of literal 2 for the size of the code unit (GH-28711) 2021-10-03 21:22:42 +03:00
Irit Katriel f71300cb04
bpo-1514420: Do not attempt to open files with names in <>s when formatting an exception (GH-28143)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-09-20 17:10:30 +02:00
Victor Stinner 888d4cc06b
bpo-45061: Enhance faulthandler traceback wit no Python frame (GH-28090)
Fix indentation of <no Python frame> message in a faulthandler
traceback or a Fatal Python error traceback. Example:

Current thread 0x00007f03896fb740 (most recent call first):
  Garbage-collecting
  <no Python frame>
2021-08-31 17:34:29 +02:00
Mark Shannon ae0a2b7562
bpo-44590: Lazily allocate frame objects (GH-27077)
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.

* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.

* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.

* Move ownership of InterpreterFrame in generator from frame object to generator object.

* Do not create frame objects for Python calls.

* Do not create frame objects for generators.
2021-07-26 11:22:16 +01:00
Batuhan Taskaya 3e235e0447
bpo-43950: support some multi-line expressions for PEP 657 (GH-27339)
This is basically something that I noticed up while fixing test runs for another issue. It is really common to have multiline calls, and when they fail the display is kind of weird since we omit the annotations. E.g;

```
 $ ./python t.py
Traceback (most recent call last):
  File "/home/isidentical/cpython/cpython/t.py", line 11, in <module>
    frame_1()
    ^^^^^^^^^
  File "/home/isidentical/cpython/cpython/t.py", line 5, in frame_1
    frame_2(              
  File "/home/isidentical/cpython/cpython/t.py", line 2, in frame_2
    return a / 0 / b / c
           ~~^~~
ZeroDivisionError: division by zero
```

This patch basically adds support for annotating the rest of the line, if the instruction covers multiple lines (start_line != end_line).

Automerge-Triggered-By: GH:isidentical
2021-07-25 15:01:44 -07:00
Batuhan Taskaya ef8b8535cb
bpo-43950: check against the raw string, not the pyobject (GH-27337) 2021-07-24 20:49:17 +03:00
Batuhan Taskaya c8362314cc
bpo-43950: ensure source_line is present when specialising the traceback (GH-27313) 2021-07-24 13:50:19 +01:00
Batuhan Taskaya fbc349ff79
bpo-43950: Distinguish errors happening on character offset decoding (GH-27217) 2021-07-20 16:42:12 +01:00