Commit Graph

193 Commits

Author SHA1 Message Date
Kirill Podoprigora 1a84bdc237
gh-121163: Add "all" as an valid alias for "always" in warnings.simplefilter() (#121164)
Add support for ``all`` as an valid alias for ``always`` in ``warnings.simplefilter()``
and ``warnings.filterswarnings()``.
2024-06-30 19:48:00 +02:00
Kirill Podoprigora 100c7ab00a
gh-119049: Fix incorrect display of warning which is constructed by C API (GH-119063)
The source line was not displayed if the warnings module had not yet
been imported.
2024-05-16 20:27:59 +00:00
Brett Simmers c2627d6eea
gh-116322: Add Py_mod_gil module slot (#116882)
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.

PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.

A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
2024-05-03 11:30:55 -04:00
Tian Gao a09e472992
gh-117535: Change unknown filename of warnings from `sys` to `<sys>` (#118018) 2024-04-18 20:50:09 -07:00
Erlend E. Aasland 05e0b67a43
gh-116664: In _warnings.c, make filters_version access thread-safe (#117374)
- assert that the lock is held in already_warned()
- protect 'filters_version' increment in warnings_filters_mutated_impl()
2024-03-29 21:23:28 +01:00
Erlend E. Aasland ddf95b5f16
gh-116664: Fix unused var warnings in _warnings.c in non-free-threaded builds (#117373)
The warnings were introduced by commit c1712ef06.
2024-03-29 17:26:06 +00:00
Erlend E. Aasland c1712ef066
gh-116664: Make module state Py_SETREF's in _warnings thread-safe (#116959)
Mark the swap operations as critical sections.

Add an internal Py_BEGIN_CRITICAL_SECTION_MUT API that takes a PyMutex
pointer instead of a PyObject pointer.
2024-03-28 15:05:08 +00:00
Erlend E. Aasland 762f489b31
gh-116664: Ensure thread-safe dict access in _warnings (#116768)
Replace _PyDict_GetItemWithError() with PyDict_GetItemRef().
2024-03-18 09:37:48 +00:00
Serhiy Storchaka 936c503a44
gh-111789: Use PyDict_GetItemRef() in Python/_warnings.c (gh-112080) 2023-11-27 18:58:43 +01:00
Victor Stinner 546cab8444
gh-106320: Remove private _PyTraceback functions (#108453)
Move private functions to the internal C API (pycore_traceback.h):

* _Py_DisplaySourceLine()
* _PyTraceback_Add()
2023-08-24 23:35:47 +00:00
Victor Stinner a071ecb4d1
gh-106320: Remove private _PySys functions (#108452)
Move private functions to the internal C API (pycore_sysmodule.h):

* _PySys_GetAttr()
* _PySys_GetSizeOf()

No longer export most of these functions.

Fix also a typo in Include/cpython/optimizer.h: add a missing space.
2023-08-24 20:02:09 +00:00
Victor Stinner fd66baf34a
gh-106320: Remove private _PyDict C API (#107145)
Move private _PyDict functions to the internal C API (pycore_dict.h):

* _PyDict_Contains_KnownHash()
* _PyDict_DebugMallocStats()
* _PyDict_DelItemIf()
* _PyDict_GetItemWithError()
* _PyDict_HasOnlyStringKeys()
* _PyDict_MaybeUntrack()
* _PyDict_MergeEx()

No longer export these functions.
2023-07-24 14:02:03 +00:00
Serhiy Storchaka be1b968dc1
gh-106521: Remove _PyObject_LookupAttr() function (GH-106642) 2023-07-12 08:57:10 +03:00
Victor Stinner c9ce983ae1
gh-106320: Remove private pylifecycle.h functions (#106400)
Remove private pylifecycle.h functions: move them to the internal C
API ( pycore_atexit.h, pycore_pylifecycle.h and pycore_signal.h). No
longer export most of these functions.

Move _testcapi.test_atexit() to _testinternalcapi.
2023-07-04 09:41:43 +00:00
Mark Shannon 7199584ac8
GH-100987: Allow objects other than code objects as the "executable" of an internal frame. (GH-105727)
* Add table describing possible executable classes for out-of-process debuggers.

* Remove shim code object creation code as it is no longer needed.

* Make lltrace a bit more robust w.r.t. non-standard frames.
2023-06-14 13:46:37 +01:00
Erlend E. Aasland 567d6ae8e7
gh-105375: Improve PyErr_WarnExplicit() error handling (#105610)
Bail on first error to prevent exceptions from possibly being
overwritten.
2023-06-11 21:23:28 +02:00
Victor Stinner ef300937c2
gh-92536: Remove PyUnicode_READY() calls (#105210)
Since Python 3.12, PyUnicode_READY() does nothing and always
returns 0.
2023-06-02 01:33:17 +02:00
Kumar Aditya 7fc542c88d
GH-89091: raise `RuntimeWarning` for unawaited async generator methods (#104611) 2023-05-26 16:53:29 +05:30
Eric Snow 26baa747c2
gh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalization (gh-104437)
With the move to a per-interpreter GIL, this check slipped through the cracks.
2023-05-15 13:59:26 -06:00
Eric Snow a9c6e0618f
gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)
Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules.  We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
2023-05-05 21:11:27 +00:00
Eric Snow b2fc549278
gh-101758: Clean Up Uses of Import State (gh-101919)
This change is almost entirely moving code around and hiding import state behind internal API.  We introduce no changes to behavior, nor to non-internal API.  (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.)  The motivation is to simplify a number of upcoming changes.

Specific changes:

* move existing import-related code to import.c, wherever possible
* add internal API for interacting with import state (both global and per-interpreter)
* use only API outside of import.c (to limit churn there when changing the location, etc.)
* consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly)
* add macros for import state in import.c (to simplify changing the location)
* group code in import.c into sections
*remove _PyState_AddModule()

https://github.com/python/cpython/issues/101758
2023-02-15 15:32:31 -07:00
Irit Katriel c4170c36b0
gh-39615: fix warning on return type mismatch (#101407) 2023-01-29 16:41:27 -08:00
Gregory P. Smith 052f53d65d
gh-39615: Add warnings.warn() skip_file_prefixes support (#100840)
`warnings.warn()` gains the ability to skip stack frames based on code
filename prefix rather than only a numeric `stacklevel=` via a new
`skip_file_prefixes=` keyword argument.
2023-01-27 18:35:14 -08:00
Victor Stinner 135ec7cefb
gh-99537: Use Py_SETREF() function in C code (#99657)
Fix potential race condition in code patterns:

* Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
* Replace "Py_XDECREF(var); var = new;" with "Py_XSETREF(var, new);"
* Replace "Py_CLEAR(var); var = new;" with "Py_XSETREF(var, new);"

Other changes:

* Replace "old = var; var = new; Py_DECREF(var)"
  with "Py_SETREF(var, new);"
* Replace "old = var; var = new; Py_XDECREF(var)"
  with "Py_XSETREF(var, new);"
* And remove the "old" variable.
2022-11-22 13:39:11 +01:00
Victor Stinner d8f239d86e
gh-99300: Use Py_NewRef() in Python/ directory (#99302)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Python/ directory.
2022-11-10 09:03:39 +01:00
Dong-hee Na a04656ec32
gh-97841: Add methoddef for _filters_mutated (gh-98115) 2022-10-09 20:16:33 +09:00
Barry Warsaw 13d4489142
gh-86298: Ensure that __loader__ and __spec__.loader agree in warnings.warn_explicit() (GH-97803)
In `_warnings.c`, in the C equivalent of `warnings.warn_explicit()`, if the module globals are given (and not None), the warning will attempt to get the source line for the issued warning.  To do this, it needs the module's loader.

Previously, it would only look up `__loader__` in the module globals.  In https://github.com/python/cpython/issues/86298 we want to defer to the `__spec__.loader` if available.

The first step on this journey is to check that `loader == __spec__.loader` and issue another warning if it is not.  This commit does that.

Since this is a PoC, only manual testing for now.

```python
# /tmp/foo.py
import warnings

import bar

warnings.warn_explicit(
    'warning!',
    RuntimeWarning,
    'bar.py', 2,
    module='bar knee',
    module_globals=bar.__dict__,
    )
```

```python
# /tmp/bar.py
import sys
import os
import pathlib

# __loader__ = pathlib.Path()
```

Then running this: `./python.exe -Wdefault /tmp/foo.py`

Produces:

```
bar.py:2: RuntimeWarning: warning!
  import os
```

Uncomment the `__loader__ = ` line in `bar.py` and try it again:

```
sys:1: ImportWarning: Module bar; __loader__ != __spec__.loader (<_frozen_importlib_external.SourceFileLoader object at 0x109f7dfa0> != PosixPath('.'))
bar.py:2: RuntimeWarning: warning!
  import os
```

Automerge-Triggered-By: GH:warsaw
2022-10-06 19:32:53 -07:00
Oleg Iarygin 41e0585ffa
gh-91102: Port 8-argument _warnings.warn_explicit to Argument Clinic (#92891)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2022-07-20 22:24:51 +02:00
Victor Stinner 27b9894033
gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938)
Move the follow functions and type from frameobject.h to pyframe.h,
so the standard <Python.h> provide frame getter functions:

* PyFrame_Check()
* PyFrame_GetBack()
* PyFrame_GetBuiltins()
* PyFrame_GetGenerator()
* PyFrame_GetGlobals()
* PyFrame_GetLasti()
* PyFrame_GetLocals()
* PyFrame_Type

Remove #include "frameobject.h" from many C files. It's no longer
needed.
2022-06-19 12:02:33 +02:00
Kumar Aditya cb04a09d2d
GH-93207: Remove HAVE_STDARG_PROTOTYPES configure check for stdarg.h (#93215) 2022-05-27 13:30:45 +02:00
Victor Stinner f0bc694856
bpo-47164: Add _PyCFunction_CAST() macro (GH-32192)
Use the macro in C files of the Python/ directory.
2022-03-31 10:02:34 +02:00
Eric Snow 1f455361ec
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366)
https://bugs.python.org/issue46765
2022-02-22 17:23:51 -07:00
Eric Snow 12360aa159
bpo-46541: Discover the global strings. (gh-31346)
Instead of manually enumerating the global strings in generate_global_objects.py, we extrapolate the list from usage of _Py_ID() and _Py_STR() in the source files.

This is partly inspired by gh-31261.

https://bugs.python.org/issue46541
2022-02-14 17:36:51 -07:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Mark Shannon ae0a2b7562
bpo-44590: Lazily allocate frame objects (GH-27077)
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.

* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.

* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.

* Move ownership of InterpreterFrame in generator from frame object to generator object.

* Do not create frame objects for Python calls.

* Do not create frame objects for generators.
2021-07-26 11:22:16 +01:00
Ammar Askar 5644c7b3ff
bpo-43950: Print columns in tracebacks (PEP 657) (GH-26958)
The traceback.c and traceback.py mechanisms now utilize the newly added code.co_positions and PyCode_Addr2Location
to print carets on the specific expressions involved in a traceback.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2021-07-05 00:14:33 +01:00
Mark Shannon b11a951f16
bpo-44032: Move data stack to thread from FrameObject. (GH-26076)
* Remove 'zombie' frames. We won't need them once we are allocating fixed-size frames.

* Add co_nlocalplus field to code object to avoid recomputing size of locals + frees + cells.

* Move locals, cells and freevars out of frame object into separate memory buffer.

* Use per-threadstate allocated memory chunks for local variables.

* Move globals and builtins from frame object to per-thread stack.

* Move (slow) locals frame object to per-thread stack.

* Move internal frame functions to internal header.
2021-05-21 10:57:35 +01:00
Victor Stinner bcb094b41f
bpo-43268: Pass interp rather than tstate to internal functions (GH-24580)
Pass the current interpreter (interp) rather than the current Python
thread state (tstate) to internal functions which only use the
interpreter.

Modified functions:

* _PyXXX_Fini() and _PyXXX_ClearFreeList() functions
* _PyEval_SignalAsyncExc(), make_pending_calls()
* _PySys_GetObject(), sys_set_object(), sys_set_object_id(), sys_set_object_str()
* should_audit(), set_flags_from_config(), make_flags()
* _PyAtExit_Call()
* init_stdio_encoding()
* etc.
2021-02-19 15:10:45 +01:00
Victor Stinner 6f4635fe20
bpo-1635741: Port _warnings to the multi-phase init (GH-23379)
Port the _warnings extension module to the multi-phase initialization
API (PEP 489).
2020-11-19 00:19:06 +01:00
Victor Stinner d1e38d4023
bpo-40998: Fix a refleak in create_filter() (GH-23365) 2020-11-18 06:57:10 -08:00
Victor Stinner ef75a625cd
bpo-42260: Initialize time and warnings earlier at startup (GH-23249)
* Call _PyTime_Init() and _PyWarnings_InitState() earlier during the
  Python initialization.
* Inline _PyImportHooks_Init() into _PySys_InitCore().
* The _warnings initialization function no longer call
  _PyWarnings_InitState() to prevent resetting filters_version to 0.
* _PyWarnings_InitState() now returns an int and no longer clear the
  state in case of error (it's done anyway at Python exit).
* Rework init_importlib(), fix refleaks on errors.
2020-11-12 15:14:13 +01:00
Victor Stinner 58ca33b467
bpo-1635741: Fix ref leak in _PyWarnings_Init() error path (GH-23151)
Replace PyModule_AddObject() with PyModule_AddObjectRef() in the
_warnings module to fix a reference leak on error.

Use also PyModule_AddObjectRef() in importdl.c.
2020-11-04 17:33:06 +01:00
Victor Stinner 45df61fd2d
bpo-26789: Fix logging.FileHandler._open() at exit (GH-23053)
The logging.FileHandler class now keeps a reference to the builtin
open() function to be able to open or reopen the file during Python
finalization.

Fix errors like:

    Exception ignored in: (...)
    Traceback (most recent call last):
      (...)
      File ".../logging/__init__.py", line 1463, in error
      File ".../logging/__init__.py", line 1577, in _log
      File ".../logging/__init__.py", line 1587, in handle
      File ".../logging/__init__.py", line 1649, in callHandlers
      File ".../logging/__init__.py", line 948, in handle
      File ".../logging/__init__.py", line 1182, in emit
      File ".../logging/__init__.py", line 1171, in _open
    NameError: name 'open' is not defined
2020-11-02 23:17:46 +01:00
Victor Stinner c9bc290dd6
bpo-42161: Use _PyLong_GetZero() and _PyLong_GetOne() (GH-22995)
Use _PyLong_GetZero() and _PyLong_GetOne()
in Objects/ and Python/ directories.
2020-10-27 02:24:34 +01:00
Victor Stinner 1bcc32f062
bpo-39465: Use _PyInterpreterState_GET() (GH-20788)
Replace _PyThreadState_GET() with _PyInterpreterState_GET() in:

* get_small_int()
* gcmodule.c: add also get_gc_state() function
* _PyTrash_deposit_object()
* _PyTrash_destroy_chain()
* warnings_get_state()
* Py_GetRecursionLimit()

Cleanup listnode.c: add 'parser' variable.
2020-06-10 20:08:26 +02:00
Victor Stinner 7036477323
bpo-40421: Add PyFrame_GetBack() function (GH-19765)
New PyFrame_GetBack() function: get the frame next outer frame.

Replace frame->f_back with PyFrame_GetBack(frame) in most code but
frameobject.c, ceval.c and genobject.c.
2020-04-29 03:28:46 +02:00
Victor Stinner 8852ad4208
bpo-40429: PyFrame_GetCode() now returns a strong reference (GH-19773) 2020-04-29 01:28:13 +02:00
Victor Stinner 6d86a2331e
bpo-40429: PyFrame_GetCode() result cannot be NULL (GH-19772)
Add frame_nslots() to factorize duplicate code.
2020-04-29 00:56:58 +02:00
Victor Stinner a42ca74fa3
bpo-40421: Add PyFrame_GetCode() function (GH-19757)
PyFrame_GetCode(frame): return a borrowed reference to the frame
code.

Replace frame->f_code with PyFrame_GetCode(frame) in most code,
except in frameobject.c, genobject.c and ceval.c.

Also add PyFrame_GetLineNumber() to the limited C API.
2020-04-28 19:01:31 +02:00
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00