Commit Graph

419 Commits

Author SHA1 Message Date
Miss Islington (bot) 148beb6de9
[3.13] gh-121957: Emit audit events for `python -i` and `python -m asyncio` (GH-121958) (GH-122115)
Relatedly, emit the `cpython.run_startup` event from the Python version of
`PYTHONSTARTUP` handling.
(cherry picked from commit dc93d1125f)

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2024-07-22 13:36:57 +02:00
Miss Islington (bot) 449529a8c2
[3.13] gh-121925: Fix uninitialized variables in `main.c` (GH-121926) (#121931)
gh-121925: Fix uninitialized variables in `main.c` (GH-121926)
(cherry picked from commit f4bc84d261)

Co-authored-by: sobolevn <mail@sobolevn.me>
2024-07-17 17:00:42 +00:00
Miss Islington (bot) 3d9692dbf8
[3.13] gh-120678: pyrepl: Include globals from modules passed with `-i` (GH-120904) (#121916)
(cherry picked from commit ac07451116)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2024-07-17 16:52:46 +02:00
Miss Islington (bot) 2dd07fd899
[3.13] gh-120346: Respect PYTHON_BASIC_REPL when running in interactive inspect mode (GH-120349) (#120351)
(cherry picked from commit ec3af291fe)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2024-06-11 16:40:15 +00:00
Pablo Galindo Salgado f27f8c790a
gh-111201: A new Python REPL (GH-111567)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-05-05 21:32:23 +02:00
Pablo Galindo Salgado 90a1b2859f
gh-67224: Show source lines in tracebacks when using the -c option when running Python (#111200) 2023-10-26 15:17:28 +09:00
Eric Snow 1dd9dee45d
gh-105716: Support Background Threads in Subinterpreters Consistently (gh-109921)
The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.

We add PyInterpreterState.threads.main, with some internal C-API functions.
2023-10-02 20:12:12 +00:00
Eric Snow a040a32ea2
gh-109853: Fix sys.path[0] For Subinterpreters (gh-109994)
This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all. This PR does not address the broader concerns from gh-109853.
2023-10-02 19:59:05 +00:00
Victor Stinner 8b626a47ba
gh-110079: Remove extern "C" { ...} in C code (#110080) 2023-09-29 10:56:49 +02:00
Victor Stinner 15c5a50797
gh-106320: Remove private pythonrun API (#108599)
Remove these private functions from the public C API:

* _PyRun_AnyFileObject()
* _PyRun_InteractiveLoopObject()
* _PyRun_SimpleFileObject()
* _Py_SourceAsString()

Move them to the internal C API: add a new pycore_pythonrun.h header
file. No longer export these functions.
2023-08-29 04:18:52 +02:00
Victor Stinner 424049cc11
gh-105145: Remove old functions to config Python init (#105154)
Remove the following old functions to configure the Python
initialization, deprecated in Python 3.11:

* PySys_AddWarnOptionUnicode()
* PySys_AddWarnOption()
* PySys_AddXOption()
* PySys_HasWarnOptions()
* PySys_SetArgvEx()
* PySys_SetArgv()
* PySys_SetPath()
* Py_SetPath()
* Py_SetProgramName()
* Py_SetPythonHome()
* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()

Most of these functions are kept in the stable ABI, except:

* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()

Update Doc/extending/embedding.rst and Doc/extending/extending.rst to
use the new PyConfig API.

_testembed.c:

* check_stdio_details() now sets stdio_encoding and stdio_errors
  of PyConfig.
* Add definitions of functions removed from the API but kept in the
  stable ABI.
* test_init_from_config() and test_init_read_set() now use
  PyConfig_SetString() instead of PyConfig_SetBytesString().

Remove _Py_ClearStandardStreamEncoding() internal function.
2023-06-01 09:14:02 +02:00
Eric Snow 5f55067e23
gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)
https://github.com/python/cpython/issues/81057
2022-11-16 09:37:14 -07:00
Christian Heimes d291a82df3
gh-90473: Check for HAVE_KILL in main.c:exit_sigint (GH-95165) 2022-07-23 12:29:00 +02:00
Victor Stinner 0ff626f210
gh-77782: Deprecate global configuration variable (#93943)
Deprecate global configuration variable like
Py_IgnoreEnvironmentFlag: the Py_InitializeFromConfig() API should be
instead.

Fix declaration of Py_GETENV(): use PyAPI_FUNC(), not PyAPI_DATA().
2022-06-17 16:12:00 +02:00
Victor Stinner ada8b6d1b1
gh-57684: Add -P cmdline option and PYTHONSAFEPATH env var (#31542)
Add the -P command line option and the PYTHONSAFEPATH environment
variable to not prepend a potentially unsafe path to sys.path.

* Add sys.flags.safe_path flag.
* Add PyConfig.safe_path member.
* Programs/_bootstrap_python.c uses config.safe_path=0.
* Update subprocess._optim_args_from_interpreter_flags() to handle
  the -P command line option.
* Modules/getpath.py sets safe_path to 1 if a "._pth" file is
  present.
2022-05-06 01:34:11 +02:00
Christian Heimes 8122e8d501
gh-92345: Import rlcompleter before sys.path is extended (#92346)
``pymain_run_python()`` now imports ``readline`` and ``rlcompleter``
before sys.path is extended to include the current working directory of
an interactive interpreter. Non-interactive interpreters are not
affected.

Also move imports of ``re`` and ``keyword`` module to top level so they
are materialized early, too. The ``keyword`` module is trivial and the
``re`` is already imported via ``inspect`` -> ``linecache``.
2022-05-05 21:24:16 +02:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Steve Dower 99fcf15052
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
2021-12-03 00:08:42 +00:00
Victor Stinner d943d19172
bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
  PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
  defined to get access to internal _PyObject_CallNoArgs().
2021-10-12 08:38:19 +02:00
Victor Stinner ce3489cfdb
bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
2021-10-12 00:42:23 +02:00
borispopoff c7c3b7db29
Typo in comment (GH-24199)
Automerge-Triggered-By: GH:Mariatta
2021-01-25 14:51:48 -08:00
Victor Stinner a12491681f
bpo-32381: pymain_run_command() uses PyCF_IGNORE_COOKIE (GH-23724)
The coding cookie (ex: "# coding: latin1") is now ignored in the
command passed to the -c command line option.

Since pymain_run_command() uses UTF-8, pass PyCF_IGNORE_COOKIE
compiler flag to the parser.

pymain_run_python() no longer propages compiler flags between
function calls.
2020-12-23 19:16:56 +01:00
Victor Stinner ceb420251c
bpo-32381: pymain_run_file() uses PySys_FormatStderr() (GH-23778)
If config->run_filename doesn't exist, log the error into sys.stderr
using "%R" format, to escape properly unencodable characters (usually
with backslashreplace).
2020-12-15 16:25:27 +01:00
Victor Stinner a82f63f5af
bpo-32381: Add _PyRun_AnyFileObject() (GH-23723)
pymain_run_file() no longer encodes the filename: pass the filename
as an object to the new _PyRun_AnyFileObject() function.

Add new private functions:

* _PyRun_AnyFileObject()
* _PyRun_InteractiveLoopObject()
* _Py_FdIsInteractive()
2020-12-09 22:37:27 +01:00
Victor Stinner 550e4673be
bpo-32381: Add _PyRun_SimpleFileObject() (GH-23709)
pymain_run_startup() now pass the filename as a Python object to
_PyRun_SimpleFileObject().
2020-12-09 00:32:54 +01:00
Thomas Grainger a68a2ad19c
bpo-41602: raise SIGINT exit code on KeyboardInterrupt from pymain_run_module (#21956)
Closes bpo issue 41602
2020-09-22 08:53:03 -07:00
Serhiy Storchaka a7dc714701
bpo-41094: Additional fix for PYTHONSTARTUP. (GH-21119) 2020-06-24 19:46:30 +03:00
Serhiy Storchaka 6c6810d989
bpo-41094: Fix decoding errors with audit when open files. (GH-21095) 2020-06-24 08:46:05 +03:00
Victor Stinner 4f98f465f1
bpo-40268: Remove unused imports in pylifecycle.c (GH-19533)
Remove unused imports in files:

* initconfig.c
* main.c
* preconfig.h
* pylifecycle.c
* python.c
* pythonrun.c
2020-04-15 04:01:58 +02:00
Victor Stinner d9ea5cae1d
bpo-40268: Remove unused pycore_pymem.h includes (GH-19531) 2020-04-15 02:57:50 +02:00
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Victor Stinner 81a7be3fa2
bpo-40268: Rename _PyInterpreterState_GET_UNSAFE() (GH-19509)
Rename _PyInterpreterState_GET_UNSAFE() to _PyInterpreterState_GET()
for consistency with _PyThreadState_GET() and to have a shorter name
(help to fit into 80 columns).

Add also "assert(tstate != NULL);" to the function.
2020-04-14 15:14:01 +02:00
Victor Stinner 4a3fe08353
bpo-40268: Include explicitly pycore_interp.h (GH-19505)
pycore_pystate.h no longer includes pycore_interp.h:
it's now included explicitly in files accessing PyInterpreterState.
2020-04-14 14:26:24 +02:00
Victor Stinner da7933ecc3
bpo-40268: Add _PyInterpreterState_GetConfig() (GH-19492)
Don't access PyInterpreterState.config member directly anymore, but
use new functions:

* _PyInterpreterState_GetConfig()
* _PyInterpreterState_SetConfig()
* _Py_GetConfig()
2020-04-13 03:04:28 +02:00
Victor Stinner 61691d8336
bpo-38353: Cleanup includes in the internal C API (GH-16548)
Use forward declaration of types to avoid includes in the internal C
API. Add also comment to justify other includes.
2019-10-02 23:51:20 +02:00
Victor Stinner 5e0ea7540f
bpo-37474: Don't call fedisableexcept() on FreeBSD (GH-16515)
On FreeBSD, Python no longer calls fedisableexcept() at startup to
control the floating point control mode. The call became useless
since FreeBSD 6: it became the default mode.
2019-10-01 13:12:29 +02:00
Victor Stinner 8462a4936b
bpo-38304: PyConfig_InitPythonConfig() cannot fail anymore (GH-16509)
PyConfig_InitPythonConfig() and PyConfig_InitIsolatedConfig() no
longer return PyStatus: they cannot fail anymore.
2019-10-01 12:06:16 +02:00
Victor Stinner 3c30a76f3d
bpo-38304: Remove PyConfig.struct_size (GH-16500) (GH-16508)
For now, we'll rely on the fact that the config structures aren't covered by the stable ABI.

We may revisit this in the future if we further explore the idea of offering a stable embedding API.

(cherry picked from commit bdace21b76)
2019-10-01 10:56:37 +02:00
Victor Stinner 441b10cf28
bpo-38304: Add PyConfig.struct_size (GH-16451)
Add a new struct_size field to PyPreConfig and PyConfig structures to
allow to modify these structures in the future without breaking the
backward compatibility.

* Replace private _config_version field with public struct_size field
  in PyPreConfig and PyConfig.
* Public PyPreConfig_InitIsolatedConfig() and
  PyPreConfig_InitPythonConfig()
  return type becomes PyStatus, instead of void.
* Internal _PyConfig_InitCompatConfig(),
  _PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(),
  _PyPreConfig_InitFromPreConfig() return type becomes PyStatus,
  instead of void.
* Remove _Py_CONFIG_VERSION
* Update the Initialization Configuration documentation.
2019-09-28 04:28:35 +02:00
Serhiy Storchaka 1f21eaa15e
bpo-15999: Clean up of handling boolean arguments. (GH-15610)
* Use the 'p' format unit instead of manually called PyObject_IsTrue().
* Pass boolean value instead 0/1 integers to functions that needs boolean.
* Convert some arguments to boolean only once.
2019-09-01 12:16:51 +03:00
Steve Dower e226e83d36
bpo-37363: Add audit events on startup for the run commands (GH-14524) 2019-07-01 16:03:53 -07:00
Victor Stinner 67310023f2
bpo-36763: Use PyConfig_Clear() (GH-14445)
Stop using "static PyConfig", PyConfig must now always use
dynamically allocated strings: use PyConfig_SetString(),
PyConfig_SetArgv() and PyConfig_Clear().
2019-07-01 19:52:45 +02:00
Victor Stinner 37d66d7d4b
bpo-37253: Add _PyCompilerFlags_INIT macro (GH-14018)
Add a new _PyCompilerFlags_INIT macro to initialize PyCompilerFlags
variables, rather than initializing cf_flags and cf_feature_version
explicitly in each variable.
2019-06-13 02:16:41 +02:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Erik Janssens 925af1d99b bpo-36965: Fix includes in main.c on Windows with non-MSC compilers (GH-13421)
Include windows.h rather than crtdbg.h to get STATUS_CONTROL_C_EXIT constant.
Moreover, include windows.h on Windows, not only when MSC is used.
2019-05-21 12:11:11 +02:00
Victor Stinner 12083284c5
bpo-36763: _Py_RunMain() doesn't call Py_Exit() anymore (GH-13390)
Py_Main() and _Py_RunMain() now return the exitcode rather than
calling Py_Exit(exitcode) when calling PyErr_Print() if the current
exception type is SystemExit.

* Add _Py_HandleSystemExit().
* Add pymain_exit_err_print().
* Add pymain_exit_print().
2019-05-17 23:05:29 +02:00
Victor Stinner cab5d0741e
bpo-36763: Add _PyCoreConfig_InitPythonConfig() (GH-13388)
Add new functions to get the Python interpreter behavior:

* _PyPreConfig_InitPythonConfig()
* _PyCoreConfig_InitPythonConfig()

Add new functions to get an isolated configuration:

* _PyPreConfig_InitIsolatedConfig()
* _PyCoreConfig_InitIsolatedConfig()

Replace _PyPreConfig_INIT and _PyCoreConfig_INIT with new functions
_PyPreConfig_Init() and _PyCoreConfig_Init().

_PyCoreConfig: set configure_c_stdio and parse_argv to 0 by default
to behave as Python 3.6 in the default configuration.

_PyCoreConfig_Read() no longer sets coerce_c_locale_warn to 1 if it's
equal to 0. coerce_c_locale_warn must now be set to -1 (ex: using
_PyCoreConfig_InitPythonConfig()) to enable C locale coercion
warning.

Add unit tests for _PyCoreConfig_InitPythonConfig()
and _PyCoreConfig_InitIsolatedConfig().

Changes:

* Rename _PyCoreConfig_GetCoreConfig() to _PyPreConfig_GetCoreConfig()
* Fix core_read_precmdline(): handle parse_argv=0
* Fix _Py_PreInitializeFromCoreConfig(): pass coreconfig.argv
  to _Py_PreInitializeFromPyArgv(), except if parse_argv=0
2019-05-17 19:01:14 +02:00
Victor Stinner fed02e15b3
bpo-36763: Remove _PyCoreConfig.program (GH-13373)
Use _PyCoreConfig.program_name instead.
2019-05-17 11:12:09 +02:00
Victor Stinner ae239f6b06
bpo-36763: Add _PyCoreConfig.parse_argv (GH-13361)
* _PyCoreConfig_Read() doesn't parse nor update argv
  if parse_argv is 0.
* Move path configuration fields in _PyCoreConfig.
* Add an unit test for parse_argv=0.
* Remove unused "done": label in _Py_RunMain().
2019-05-16 17:02:56 +02:00
Victor Stinner dbacfc2273
bpo-36763: _PyInitError always use int for exitcode (GH-13360)
We cannot use "unsigned int" for exitcode on Windows, since
Py_Main() and _Py_RunMain() always return an "int".

Changes:

* _PyPathConfig_ComputeSysPath0() now returns -1 if an exception is
  raised.
* pymain_run_python() no longer uses _PyInitError but display the
  exception and set exitcode to 1 in case of error.
* Fix _Py_RunMain(): return an exitcode rather than calling
  exit() on pymain_run_python() failure.
* _Py_ExitInitError() no longer uses ExitProcess() on Windows, use
  exit() on all platforms.
* _Py_ExitInitError() now fails with a fatal error if 'err' is not an
  error not an exit.
2019-05-16 16:39:26 +02:00