mirror of https://github.com/python/cpython
bpo-34206: Improve docs and test coverage for pre-init functions (#8023)
- move the Py_Main documentation from the very high level API section to the initialization and finalization section - make it clear that it encapsulates a full Py_Initialize/Finalize cycle of its own - point out that exactly which settings will be read and applied correctly when Py_Main is called after a separate runtime initialization call is version dependent - be explicit that Py_IsInitialized can be called prior to initialization - actually test that Py_IsInitialized can be called prior to initialization - flush stdout in the embedding tests that run code so it appears in the expected order when running with "-vv" - make "-vv" on the subinterpreter embedding tests less spammy --------- Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
This commit is contained in:
Alyssa Coghlan
GitHub
parent
93b9e6bd7d
commit
7c4b6a68f2
|
|
@ -7,7 +7,8 @@
|
|||
Initialization, Finalization, and Threads
|
||||
*****************************************
|
||||
|
||||
See also the :ref:`Python Initialization Configuration <init-config>`.
|
||||
See :ref:`Python Initialization Configuration <init-config>` for details
|
||||
on how to configure the interpreter prior to initialization.
|
||||
|
||||
.. _pre-init-safe:
|
||||
|
||||
|
|
@ -21,6 +22,15 @@ a few functions and the :ref:`global configuration variables
|
|||
|
||||
The following functions can be safely called before Python is initialized:
|
||||
|
||||
* Functions that initialize the interpreter:
|
||||
|
||||
* :c:func:`Py_Initialize`
|
||||
* :c:func:`Py_InitializeEx`
|
||||
* :c:func:`Py_InitializeFromConfig`
|
||||
* :c:func:`Py_BytesMain`
|
||||
* :c:func:`Py_Main`
|
||||
* the runtime pre-initialization functions covered in :ref:`init-config`
|
||||
|
||||
* Configuration functions:
|
||||
|
||||
* :c:func:`PyImport_AppendInittab`
|
||||
|
|
@ -32,6 +42,7 @@ The following functions can be safely called before Python is initialized:
|
|||
* :c:func:`Py_SetProgramName`
|
||||
* :c:func:`Py_SetPythonHome`
|
||||
* :c:func:`PySys_ResetWarnOptions`
|
||||
* the configuration functions covered in :ref:`init-config`
|
||||
|
||||
* Informative functions:
|
||||
|
||||
|
|
@ -43,10 +54,12 @@ The following functions can be safely called before Python is initialized:
|
|||
* :c:func:`Py_GetCopyright`
|
||||
* :c:func:`Py_GetPlatform`
|
||||
* :c:func:`Py_GetVersion`
|
||||
* :c:func:`Py_IsInitialized`
|
||||
|
||||
* Utilities:
|
||||
|
||||
* :c:func:`Py_DecodeLocale`
|
||||
* the status reporting and utility functions covered in :ref:`init-config`
|
||||
|
||||
* Memory allocators:
|
||||
|
||||
|
|
@ -62,11 +75,13 @@ The following functions can be safely called before Python is initialized:
|
|||
|
||||
.. note::
|
||||
|
||||
The following functions **should not be called** before
|
||||
:c:func:`Py_Initialize`: :c:func:`Py_EncodeLocale`, :c:func:`Py_GetPath`,
|
||||
Despite their apparent similarity to some of the functions listed above,
|
||||
the following functions **should not be called** before the interpreter has
|
||||
been initialized: :c:func:`Py_EncodeLocale`, :c:func:`Py_GetPath`,
|
||||
:c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix`,
|
||||
:c:func:`Py_GetProgramFullPath`, :c:func:`Py_GetPythonHome`,
|
||||
:c:func:`Py_GetProgramName` and :c:func:`PyEval_InitThreads`.
|
||||
:c:func:`Py_GetProgramName`, :c:func:`PyEval_InitThreads`, and
|
||||
:c:func:`Py_RunMain`.
|
||||
|
||||
|
||||
.. _global-conf-vars:
|
||||
|
|
@ -346,34 +361,42 @@ Initializing and finalizing the interpreter
|
|||
this should be called before using any other Python/C API functions; see
|
||||
:ref:`Before Python Initialization <pre-init-safe>` for the few exceptions.
|
||||
|
||||
This initializes
|
||||
the table of loaded modules (``sys.modules``), and creates the fundamental
|
||||
modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. It also initializes
|
||||
the module search path (``sys.path``). It does not set ``sys.argv``; use
|
||||
the new :c:type:`PyConfig` API of the :ref:`Python Initialization
|
||||
Configuration <init-config>` for that. This is a no-op when called for a
|
||||
second time
|
||||
(without calling :c:func:`Py_FinalizeEx` first). There is no return value; it is a
|
||||
fatal error if the initialization fails.
|
||||
This initializes the table of loaded modules (``sys.modules``), and creates
|
||||
the fundamental modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`.
|
||||
It also initializes the module search path (``sys.path``). It does not set
|
||||
``sys.argv``; use the :ref:`Python Initialization Configuration <init-config>`
|
||||
API for that. This is a no-op when called for a second time (without calling
|
||||
:c:func:`Py_FinalizeEx` first). There is no return value; it is a fatal
|
||||
error if the initialization fails.
|
||||
|
||||
Use the :c:func:`Py_InitializeFromConfig` function to customize the
|
||||
Use :c:func:`Py_InitializeFromConfig` to customize the
|
||||
:ref:`Python Initialization Configuration <init-config>`.
|
||||
|
||||
.. note::
|
||||
On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``, which will
|
||||
also affect non-Python uses of the console using the C Runtime.
|
||||
On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``,
|
||||
which will also affect non-Python uses of the console using the C Runtime.
|
||||
|
||||
|
||||
.. c:function:: void Py_InitializeEx(int initsigs)
|
||||
|
||||
This function works like :c:func:`Py_Initialize` if *initsigs* is ``1``. If
|
||||
*initsigs* is ``0``, it skips initialization registration of signal handlers, which
|
||||
might be useful when Python is embedded.
|
||||
*initsigs* is ``0``, it skips initialization registration of signal handlers,
|
||||
which may be useful when CPython is embedded as part of a larger application.
|
||||
|
||||
Use the :c:func:`Py_InitializeFromConfig` function to customize the
|
||||
Use :c:func:`Py_InitializeFromConfig` to customize the
|
||||
:ref:`Python Initialization Configuration <init-config>`.
|
||||
|
||||
|
||||
.. c:function:: PyStatus Py_InitializeFromConfig(const PyConfig *config)
|
||||
|
||||
Initialize Python from *config* configuration, as described in
|
||||
:ref:`init-from-config`.
|
||||
|
||||
See the :ref:`init-config` section for details on pre-initializing the
|
||||
interpreter, populating the runtime configuration structure, and querying
|
||||
the returned status structure.
|
||||
|
||||
|
||||
.. c:function:: int Py_IsInitialized()
|
||||
|
||||
Return true (nonzero) when the Python interpreter has been initialized, false
|
||||
|
|
@ -440,12 +463,111 @@ Initializing and finalizing the interpreter
|
|||
|
||||
.. versionadded:: 3.6
|
||||
|
||||
|
||||
.. c:function:: void Py_Finalize()
|
||||
|
||||
This is a backwards-compatible version of :c:func:`Py_FinalizeEx` that
|
||||
disregards the return value.
|
||||
|
||||
|
||||
.. c:function:: int Py_BytesMain(int argc, char **argv)
|
||||
|
||||
Similar to :c:func:`Py_Main` but *argv* is an array of bytes strings,
|
||||
allowing the calling application to delegate the text decoding step to
|
||||
the CPython runtime.
|
||||
|
||||
.. versionadded:: 3.8
|
||||
|
||||
|
||||
.. c:function:: int Py_Main(int argc, wchar_t **argv)
|
||||
|
||||
The main program for the standard interpreter, encapsulating a full
|
||||
initialization/finalization cycle, as well as additional
|
||||
behaviour to implement reading configurations settings from the environment
|
||||
and command line, and then executing ``__main__`` in accordance with
|
||||
:ref:`using-on-cmdline`.
|
||||
|
||||
This is made available for programs which wish to support the full CPython
|
||||
command line interface, rather than just embedding a Python runtime in a
|
||||
larger application.
|
||||
|
||||
The *argc* and *argv* parameters are similar to those which are passed to a
|
||||
C program's :c:func:`main` function, except that the *argv* entries are first
|
||||
converted to ``wchar_t`` using :c:func:`Py_DecodeLocale`. It is also
|
||||
important to note that the argument list entries may be modified to point to
|
||||
strings other than those passed in (however, the contents of the strings
|
||||
pointed to by the argument list are not modified).
|
||||
|
||||
The return value will be ``0`` if the interpreter exits normally (i.e.,
|
||||
without an exception), ``1`` if the interpreter exits due to an exception,
|
||||
or ``2`` if the argument list does not represent a valid Python command
|
||||
line.
|
||||
|
||||
Note that if an otherwise unhandled :exc:`SystemExit` is raised, this
|
||||
function will not return ``1``, but exit the process, as long as
|
||||
``Py_InspectFlag`` is not set. If ``Py_InspectFlag`` is set, execution will
|
||||
drop into the interactive Python prompt, at which point a second otherwise
|
||||
unhandled :exc:`SystemExit` will still exit the process, while any other
|
||||
means of exiting will set the return value as described above.
|
||||
|
||||
In terms of the CPython runtime configuration APIs documented in the
|
||||
:ref:`runtime configuration <init-config>` section (and without accounting
|
||||
for error handling), ``Py_Main`` is approximately equivalent to::
|
||||
|
||||
PyConfig config;
|
||||
PyConfig_InitPythonConfig(&config);
|
||||
PyConfig_SetArgv(&config, argc, argv);
|
||||
Py_InitializeFromConfig(&config);
|
||||
PyConfig_Clear(&config);
|
||||
|
||||
Py_RunMain();
|
||||
|
||||
In normal usage, an embedding application will call this function
|
||||
*instead* of calling :c:func:`Py_Initialize`, :c:func:`Py_InitializeEx` or
|
||||
:c:func:`Py_InitializeFromConfig` directly, and all settings will be applied
|
||||
as described elsewhere in this documentation. If this function is instead
|
||||
called *after* a preceding runtime initialization API call, then exactly
|
||||
which environmental and command line configuration settings will be updated
|
||||
is version dependent (as it depends on which settings correctly support
|
||||
being modified after they have already been set once when the runtime was
|
||||
first initialized).
|
||||
|
||||
|
||||
.. c:function:: int Py_RunMain(void)
|
||||
|
||||
Executes the main module in a fully configured CPython runtime.
|
||||
|
||||
Executes the command (:c:member:`PyConfig.run_command`), the script
|
||||
(:c:member:`PyConfig.run_filename`) or the module
|
||||
(:c:member:`PyConfig.run_module`) specified on the command line or in the
|
||||
configuration. If none of these values are set, runs the interactive Python
|
||||
prompt (REPL) using the ``__main__`` module's global namespace.
|
||||
|
||||
If :c:member:`PyConfig.inspect` is not set (the default), the return value
|
||||
will be ``0`` if the interpreter exits normally (that is, without raising
|
||||
an exception), or ``1`` if the interpreter exits due to an exception. If an
|
||||
otherwise unhandled :exc:`SystemExit` is raised, the function will immediately
|
||||
exit the process instead of returning ``1``.
|
||||
|
||||
If :c:member:`PyConfig.inspect` is set (such as when the :option:`-i` option
|
||||
is used), rather than returning when the interpreter exits, execution will
|
||||
instead resume in an interactive Python prompt (REPL) using the ``__main__``
|
||||
module's global namespace. If the interpreter exited with an exception, it
|
||||
is immediately raised in the REPL session. The function return value is
|
||||
then determined by the way the *REPL session* terminates: returning ``0``
|
||||
if the session terminates without raising an unhandled exception, exiting
|
||||
immediately for an unhandled :exc:`SystemExit`, and returning ``1`` for
|
||||
any other unhandled exception.
|
||||
|
||||
This function always finalizes the Python interpreter regardless of whether
|
||||
it returns a value or immediately exits the process due to an unhandled
|
||||
:exc:`SystemExit` exception.
|
||||
|
||||
See :ref:`Python Configuration <init-python-config>` for an example of a
|
||||
customized Python that always runs in isolated mode using
|
||||
:c:func:`Py_RunMain`.
|
||||
|
||||
|
||||
Process-wide parameters
|
||||
=======================
|
||||
|
||||
|
|
|
|||
|
|
@ -1356,14 +1356,13 @@ the :option:`-X` command line option.
|
|||
The ``show_alloc_count`` field has been removed.
|
||||
|
||||
|
||||
.. _init-from-config:
|
||||
|
||||
Initialization with PyConfig
|
||||
----------------------------
|
||||
|
||||
Function to initialize Python:
|
||||
|
||||
.. c:function:: PyStatus Py_InitializeFromConfig(const PyConfig *config)
|
||||
|
||||
Initialize Python from *config* configuration.
|
||||
Initializing the interpreter from a populated configuration struct is handled
|
||||
by calling :c:func:`Py_InitializeFromConfig`.
|
||||
|
||||
The caller is responsible to handle exceptions (error or exit) using
|
||||
:c:func:`PyStatus_Exception` and :c:func:`Py_ExitStatusException`.
|
||||
|
|
@ -1835,26 +1834,6 @@ return ``-1`` on error:
|
|||
}
|
||||
|
||||
|
||||
Py_RunMain()
|
||||
============
|
||||
|
||||
.. c:function:: int Py_RunMain(void)
|
||||
|
||||
Execute the command (:c:member:`PyConfig.run_command`), the script
|
||||
(:c:member:`PyConfig.run_filename`) or the module
|
||||
(:c:member:`PyConfig.run_module`) specified on the command line or in the
|
||||
configuration.
|
||||
|
||||
By default and when if :option:`-i` option is used, run the REPL.
|
||||
|
||||
Finally, finalizes Python and returns an exit status that can be passed to
|
||||
the ``exit()`` function.
|
||||
|
||||
See :ref:`Python Configuration <init-python-config>` for an example of
|
||||
customized Python always running in isolated mode using
|
||||
:c:func:`Py_RunMain`.
|
||||
|
||||
|
||||
Runtime Python configuration API
|
||||
================================
|
||||
|
||||
|
|
|
|||
|
|
@ -25,30 +25,6 @@ are only passed to these functions if it is certain that they were created by
|
|||
the same library that the Python runtime is using.
|
||||
|
||||
|
||||
.. c:function:: int Py_Main(int argc, wchar_t **argv)
|
||||
|
||||
The main program for the standard interpreter. This is made available for
|
||||
programs which embed Python. The *argc* and *argv* parameters should be
|
||||
prepared exactly as those which are passed to a C program's :c:func:`main`
|
||||
function (converted to wchar_t according to the user's locale). It is
|
||||
important to note that the argument list may be modified (but the contents of
|
||||
the strings pointed to by the argument list are not). The return value will
|
||||
be ``0`` if the interpreter exits normally (i.e., without an exception),
|
||||
``1`` if the interpreter exits due to an exception, or ``2`` if the parameter
|
||||
list does not represent a valid Python command line.
|
||||
|
||||
Note that if an otherwise unhandled :exc:`SystemExit` is raised, this
|
||||
function will not return ``1``, but exit the process, as long as
|
||||
:c:member:`PyConfig.inspect` is zero.
|
||||
|
||||
|
||||
.. c:function:: int Py_BytesMain(int argc, char **argv)
|
||||
|
||||
Similar to :c:func:`Py_Main` but *argv* is an array of bytes strings.
|
||||
|
||||
.. versionadded:: 3.8
|
||||
|
||||
|
||||
.. c:function:: int PyRun_AnyFile(FILE *fp, const char *filename)
|
||||
|
||||
This is a simplified interface to :c:func:`PyRun_AnyFileExFlags` below, leaving
|
||||
|
|
|
|||
|
|
@ -168,7 +168,8 @@ class EmbeddingTestsMixin:
|
|||
# Parse the line from the loop. The first line is the main
|
||||
# interpreter and the 3 afterward are subinterpreters.
|
||||
interp = Interp(*match.groups())
|
||||
if support.verbose > 1:
|
||||
if support.verbose > 2:
|
||||
# 5 lines per pass is super-spammy, so limit that to -vvv
|
||||
print(interp)
|
||||
self.assertTrue(interp.interp)
|
||||
self.assertTrue(interp.tstate)
|
||||
|
|
@ -279,6 +280,10 @@ class EmbeddingTests(EmbeddingTestsMixin, unittest.TestCase):
|
|||
"""
|
||||
env = dict(os.environ, PYTHONPATH=os.pathsep.join(sys.path))
|
||||
out, err = self.run_embedded_interpreter("test_pre_initialization_api", env=env)
|
||||
if support.verbose > 1:
|
||||
print()
|
||||
print(out)
|
||||
print(err)
|
||||
if MS_WINDOWS:
|
||||
expected_path = self.test_exe
|
||||
else:
|
||||
|
|
@ -296,6 +301,10 @@ class EmbeddingTests(EmbeddingTestsMixin, unittest.TestCase):
|
|||
env['PYTHONPATH'] = os.pathsep.join(sys.path)
|
||||
out, err = self.run_embedded_interpreter(
|
||||
"test_pre_initialization_sys_options", env=env)
|
||||
if support.verbose > 1:
|
||||
print()
|
||||
print(out)
|
||||
print(err)
|
||||
expected_output = (
|
||||
"sys.warnoptions: ['once', 'module', 'default']\n"
|
||||
"sys._xoptions: {'not_an_option': '1', 'also_not_an_option': '2'}\n"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,2 @@
|
|||
Added ``Py_IsInitialized`` to the list of APIs that are safe to call before
|
||||
the interpreter is initialized, and updated the embedding tests to cover it.
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
The :c:func:`Py_Main` documentation moved from the "Very High Level API" section to the
|
||||
"Initialization and Finalization" section.
|
||||
|
||||
Also make it explicit that we expect ``Py_Main`` to typically be called instead
|
||||
of ``Py_Initialize`` rather than after it (since ``Py_Main`` makes its own
|
||||
call to ``Py_Initialize``). Document that calling both is
|
||||
supported but is version dependent on which settings
|
||||
will be applied correctly.
|
||||
|
|
@ -311,14 +311,36 @@ static int test_pre_initialization_api(void)
|
|||
_Py_EMBED_PREINIT_CHECK("Checking Py_SetProgramName\n");
|
||||
Py_SetProgramName(program);
|
||||
|
||||
_Py_EMBED_PREINIT_CHECK("Checking !Py_IsInitialized pre-initialization\n");
|
||||
if (Py_IsInitialized()) {
|
||||
fprintf(stderr, "Fatal error: initialized before initialization!\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
_Py_EMBED_PREINIT_CHECK("Initializing interpreter\n");
|
||||
Py_Initialize();
|
||||
|
||||
_Py_EMBED_PREINIT_CHECK("Checking Py_IsInitialized post-initialization\n");
|
||||
if (!Py_IsInitialized()) {
|
||||
fprintf(stderr, "Fatal error: not initialized after initialization!\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
_Py_EMBED_PREINIT_CHECK("Check sys module contents\n");
|
||||
PyRun_SimpleString("import sys; "
|
||||
"print('sys.executable:', sys.executable)");
|
||||
PyRun_SimpleString(
|
||||
"import sys; "
|
||||
"print('sys.executable:', sys.executable); "
|
||||
"sys.stdout.flush(); "
|
||||
);
|
||||
_Py_EMBED_PREINIT_CHECK("Finalizing interpreter\n");
|
||||
Py_Finalize();
|
||||
|
||||
_Py_EMBED_PREINIT_CHECK("Checking !Py_IsInitialized post-finalization\n");
|
||||
if (Py_IsInitialized()) {
|
||||
fprintf(stderr, "Fatal error: still initialized after finalization!\n");
|
||||
return 1;
|
||||
}
|
||||
|
||||
_Py_EMBED_PREINIT_CHECK("Freeing memory allocated by Py_DecodeLocale\n");
|
||||
PyMem_RawFree(program);
|
||||
return 0;
|
||||
|
|
@ -364,12 +386,15 @@ static int test_pre_initialization_sys_options(void)
|
|||
_Py_EMBED_PREINIT_CHECK("Initializing interpreter\n");
|
||||
_testembed_Py_InitializeFromConfig();
|
||||
_Py_EMBED_PREINIT_CHECK("Check sys module contents\n");
|
||||
PyRun_SimpleString("import sys; "
|
||||
PyRun_SimpleString(
|
||||
"import sys; "
|
||||
"print('sys.warnoptions:', sys.warnoptions); "
|
||||
"print('sys._xoptions:', sys._xoptions); "
|
||||
"warnings = sys.modules['warnings']; "
|
||||
"latest_filters = [f[0] for f in warnings.filters[:3]]; "
|
||||
"print('warnings.filters[:3]:', latest_filters)");
|
||||
"print('warnings.filters[:3]:', latest_filters); "
|
||||
"sys.stdout.flush(); "
|
||||
);
|
||||
_Py_EMBED_PREINIT_CHECK("Finalizing interpreter\n");
|
||||
Py_Finalize();
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue