Fix the Python path configuration used to initialized sys.path at
Python startup. Paths are no longer encoded to UTF-8/strict to avoid
encoding errors if it contains surrogate characters (bytes paths are
decoded with the surrogateescape error handler).
getpath_basename() and getpath_dirname() functions no longer encode
the path to UTF-8/strict, but work directly on Unicode strings. These
functions now use PyUnicode_FindChar() and PyUnicode_Substring() on
the Unicode path, rather than strrchr() on the encoded bytes string.
The side effect of this bug was that venv environments directly
used the main interpreter instead of the intermediate stub executable,
which can cause problems when a script uses system APIs that
require the use of an application bundle.
* Check NS API return values for NULL to prevent segfault in
``_bootstrap_python``.
* Set modPathInitialized to 1 so the ``decode_to_dict`` path is used.
Signed-off-by: Christian Heimes <christian@python.org>
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.
This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
The recently added PyConfig.stdlib_dir was being set with ".." entries. When __file__ was added for from modules this caused a problem on out-of-tree builds. This PR fixes that by normalizing "stdlib_dir" when it is calculated in getpath.c.
https://bugs.python.org/issue45506
During runtime startup we figure out the stdlib dir but currently throw that information away. This change preserves it and exposes it via PyConfig.stdlib_dir, _Py_GetStdlibDir(), and sys._stdlib_dir.
https://bugs.python.org/issue45211
This accomplishes 2 things:
* consolidates some common code between getpath.c and getpathp.c
* makes the helpers available to code in other files
FWIW, the signature of the join_relfile() function (in fileutils.c) intentionally mirrors that of Windows' PathCchCombineEx().
Note that this change is mostly moving code around. No behavior is meant to change.
https://bugs.python.org/issue45211
Co-authored-by: Lawrence D’Anna <lawrence_danna@apple.com>
* Add support for macOS 11 and Apple Silicon (aka arm64)
As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy
* Support building on recent versions of macOS while deploying to older versions
This allows building installers on macOS 11 while still supporting macOS 10.9.
Followup of bpo-40854, there is one remaining usage of PLATLIBDIR
which should be replaced by config->platlibdir.
test_sys checks that sys.platlibdir attribute exists and is a string.
Update Makefile: getpath.c and sysmodule.c no longer need PLATLIBDIR
macro, PyConfig.platlibdir member is used instead.
Co-authored-by: Sandro Mani <manisandro@gmail.com>
Add --with-platlibdir option to the configure script: name of the
platform-specific library directory, stored in the new sys.platlitdir
attribute. It is used to build the path of platform-specific dynamic
libraries and the path of the standard library.
It is equal to "lib" on most platforms. On Fedora and SuSE, it is
equal to "lib64" on 64-bit systems.
Co-Authored-By: Jan Matějek <jmatejek@suse.com>
Co-Authored-By: Matěj Cepl <mcepl@cepl.eu>
Co-Authored-By: Charalampos Stratakis <cstratak@redhat.com>
* _Py_FindEnvConfigValue() now returns a string allocated
by PyMem_RawMalloc().
* calculate_init() now decodes VPATH macro.
* Add calculate_open_pyenv() function.
* Add substring() and joinpath2() functions.
* Fix add_exe_suffix()
And a few cleanup changes.
Following symbolic links is now limited to 40 attempts, just to
prevent loops.
Add subfunctions:
* Add resolve_symlinks()
* Add calculate_argv0_path_framework()
* Add calculate_which()
* Add calculate_program_macos()
Fix also _Py_wreadlink(): readlink() result type is Py_ssize_t, not
int.
* search_for_prefix() directly calls reduce() if found is greater
than 0.
* Add calculate_pybuilddir() subfunction.
* search_for_prefix(): add path string buffer for readability.
* Fix some error handling code paths: release resources on error.
* calculate_read_pyenv(): rename tmpbuffer to filename.
* test.pythoninfo now also logs windows.dll_path
Refactor path configuration code:
* read_pth_file() now returns PyStatus to report errors, rather than
calling Py_FatalError().
* Move argv0_path and zip_path buffers out of PyCalculatePath
structures.
* On Windows, _PyPathConfig.home is now preferred over PyConfig.home.
* _PyConfig_InitPathConfig() now starts by copying the global path
configuration, and then override values set in PyConfig.
* _PyPathConfig_Calculate() implementations no longer override
_PyPathConfig fields which are already computed. For example,
if _PyPathConfig.prefix is not NULL, leave it unchanged.
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() doesn't
call _PyPathConfig_Calculate() anymore.
* _PyPathConfig_Calculate() no longer uses PyConfig,
except to initialize PyCalculatePath structure.
* pathconfig_calculate(): remove useless temporary
"_PyPathConfig new_config" variable.
* calculate_module_search_path(): remove hack to workaround memory
allocation failure, call Py_FatalError() instead.
* Fix get_program_full_path(): handle memory allocation failure.
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() now
uses its value.
* Py_Initialize() now longer copies path configuration from PyConfig
to the global path configuration (_Py_path_config).
Python now gets the absolute path of the script filename specified on
the command line (ex: "python3 script.py"): the __file__ attribute of
the __main__ module, sys.argv[0] and sys.path[0] become an absolute
path, rather than a relative path.
* Add _Py_isabs() and _Py_abspath() functions.
* _PyConfig_Read() now tries to get the absolute path of
run_filename, but keeps the relative path if _Py_abspath() fails.
* Reimplement os._getfullpathname() using _Py_abspath().
* Use _Py_isabs() in getpath.c.
* Add _PyInitError functions:
* _PyInitError_Ok()
* _PyInitError_Error()
* _PyInitError_NoMemory()
* _PyInitError_Exit()
* _PyInitError_IsError()
* _PyInitError_IsExit()
* _PyInitError_Failed()
* frozenmain.c and _testembed.c now use functions rather than macros.
* Move _Py_INIT_xxx() macros to the internal API.
* Move _PyWstrList_INIT macro to the internal API.
* Add a private _Py_InitializeMain() function.
* Add again _PyCoreConfig._init_main.
* _Py_InitializeFromConfig() now uses _init_main to decide
if _Py_InitializeMainInterpreter() should be called.
* _PyCoreConfig: rename _frozen to pathconfig_warnings, its value is
now the opposite of Py_FrozenFlag.
* Add an unit test for _init_main=0 and _Py_InitializeMain().
* Remove _PyInitError.user_err field and _Py_INIT_USER_ERR() macro:
use _Py_INIT_ERR() instead. _Py_ExitInitError() now longer calls
abort() on error: exit with exit code 1 instead.
* Add _PyInitError._type private field.
* exitcode field type is now unsigned int on Windows.
* Rename prefix field to _func.
* Rename msg field to err_msg.
Victor Stinner
Serhiy Storchaka
Ronald Oussoren
neonene
Steve Dower
Christian Heimes
Eric Snow
Christian Clauss
stratakis
Sandro Mani
Pablo Galindo
Victor Stinner
Orivej Desh
Min ho Kim