Commit Graph

184 Commits

Author SHA1 Message Date
Ronald Oussoren 6aaf4cd866
bpo-46890: Fix setting of sys._base_executable with framework builds on macOS (GH-31958)
The side effect of this bug was that venv environments directly
used the main interpreter instead of the intermediate stub executable,
which can cause problems when a script uses system APIs that
require the use of an application bundle.
2022-04-05 02:05:36 -04:00
neonene d4e64cd4b0
bpo-46362: Ensure ntpath.abspath() uses the Windows API correctly (GH-30571)
This makes ntpath.abspath()/getpath_abspath() follow normpath(), since some WinAPIs such as PathCchSkipRoot() require backslashed paths.
2022-01-13 23:35:42 +00:00
Steve Dower bfc59ed0a0
bpo-46049: Fixes ._pth support on non-Windows (GH-30051) 2021-12-11 15:06:17 +00:00
Steve Dower 971ece8e17
bpo-46048: Fix parsing of single character lines in getpath readlines() (GH-30048) 2021-12-11 13:43:40 +00:00
Steve Dower 7778116c2f
bpo-46015: Fixes calculation of sys.path in a venv on Windows (GH-29992)
Also ensures that pybuilddir.txt is written early enough in the build to be picked up by later steps.
2021-12-08 19:25:58 +00:00
Christian Heimes 06c4ae8b13
bpo-45582: Fix framework path and bootstrap build (GH-29954)
* Check NS API return values for NULL to prevent segfault in
  ``_bootstrap_python``.
* Set modPathInitialized to 1 so the ``decode_to_dict`` path is used.

Signed-off-by: Christian Heimes <christian@python.org>
2021-12-07 19:09:53 +01:00
Christian Heimes f16f93e527
bpo-45582: framework build: modPath must not be const (GH-29944)
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
2021-12-06 19:13:12 +01:00
neonene af1db4eb55
bpo-45582: Fix getpath_isxfile() and test_embed on Windows (GH-29930) 2021-12-06 17:25:19 +00:00
Christian Heimes 612e59b53f
bpo-45950: Fix macOS framework builds of _bootstrap_python (GH-29936) 2021-12-06 17:13:53 +01:00
Christian Heimes 628abe4463
bpo-45582: Fix signature of _Py_Get_Getpath_CodeObject (GH-29921) 2021-12-05 18:41:46 +01:00
neonene 7d7c91a8e8
bpo-45582: Add a NOT operator to the condition in getpath_isxfile (GH-29906) 2021-12-03 22:04:11 +00:00
Christian Heimes ccb73a0d50
bpo-45582: Fix out-of-tree build issues with new getpath (GH-29902) 2021-12-03 14:47:06 +01:00
Steve Dower 99fcf15052
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
2021-12-03 00:08:42 +00:00
Eric Snow 17c61045c5
bpo-45506: Normalize _PyPathConfig.stdlib_dir when calculated. (#29040)
The recently added PyConfig.stdlib_dir was being set with ".." entries. When __file__ was added for from modules this caused a problem on out-of-tree builds. This PR fixes that by normalizing "stdlib_dir" when it is calculated in getpath.c.

https://bugs.python.org/issue45506
2021-10-22 17:20:03 -06:00
Victor Stinner aac29af678
bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
Include <stdlib.h> explicitly in C files.

Python.h includes <wchar.h>.
2021-10-13 19:25:53 +02:00
Christian Clauss dd02a696e5
Fix typos in the Modules directory (GH-28761) 2021-10-07 01:34:42 -07:00
Eric Snow 0c50b8c0b8
bpo-45211: Remember the stdlib dir during startup. (gh-28586)
During runtime startup we figure out the stdlib dir but currently throw that information away. This change preserves it and exposes it via PyConfig.stdlib_dir, _Py_GetStdlibDir(), and sys._stdlib_dir.

https://bugs.python.org/issue45211
2021-09-28 12:18:28 -06:00
Eric Snow ae7839bbe8
bpo-45211: Move helpers from getpath.c to internal API. (gh-28550)
This accomplishes 2 things:

* consolidates some common code between getpath.c and getpathp.c
* makes the helpers available to code in other files

FWIW, the signature of the join_relfile() function (in fileutils.c) intentionally mirrors that of Windows' PathCchCombineEx().

Note that this change is mostly moving code around. No behavior is meant to change.

https://bugs.python.org/issue45211
2021-09-27 10:00:32 -06:00
stratakis bdb56902a3
bpo-44285: getpath.c: Assert that env_file is NULL during an error check (GH-26486)
This was flagged by a static analyzer, but the logic of why this is NULL on error is hard to follow for humans as well.
2021-06-02 16:54:33 +02:00
Ronald Oussoren 41761933c1
bpo-41100: Support macOS 11 and Apple Silicon (GH-22855)
Co-authored-by:  Lawrence D’Anna <lawrence_danna@apple.com>

* Add support for macOS 11 and Apple Silicon (aka arm64)
   
  As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy

* Support building on recent versions of macOS while deploying to older versions

  This allows building installers on macOS 11 while still supporting macOS 10.9.
2020-11-08 10:05:27 +01:00
Victor Stinner d72b9644a3
bpo-40947: getpath.c uses PyConfig.platlibdir (GH-20807)
Followup of bpo-40854, there is one remaining usage of PLATLIBDIR
which should be replaced by config->platlibdir.

test_sys checks that sys.platlibdir attribute exists and is a string.

Update Makefile: getpath.c and sysmodule.c no longer need PLATLIBDIR
macro, PyConfig.platlibdir member is used instead.

Co-authored-by: Sandro Mani <manisandro@gmail.com>
2020-06-11 17:28:52 +02:00
Sandro Mani 8f023a2f66
bpo-40854: Allow overriding sys.platlibdir via PYTHONPLATLIBDIR env-var (GH-20605) 2020-06-08 17:28:11 +02:00
Pablo Galindo 2a561b5f68
bpo-37616: Handle version information more gracefully in getpath.c (GH-20214) 2020-05-19 17:28:05 +01:00
Pablo Galindo d4fe098d1e
Python 3.10.0a0 (GH-20198) 2020-05-19 03:33:01 +01:00
Victor Stinner 361dcdcefc
bpo-40268: Remove unused osdefs.h includes (GH-19532)
When the include is needed, add required symbol in a comment.
2020-04-15 03:24:57 +02:00
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Victor Stinner 8510f43078
bpo-1294959: Add sys.platlibdir attribute (GH-18381)
Add --with-platlibdir option to the configure script: name of the
platform-specific library directory, stored in the new sys.platlitdir
attribute. It is used to build the path of platform-specific dynamic
libraries and the path of the standard library.

It is equal to "lib" on most platforms. On Fedora and SuSE, it is
equal to "lib64" on 64-bit systems.

Co-Authored-By: Jan Matějek <jmatejek@suse.com>
Co-Authored-By: Matěj Cepl <mcepl@cepl.eu>
Co-Authored-By: Charalampos Stratakis <cstratak@redhat.com>
2020-03-10 09:53:09 +01:00
Victor Stinner b6e0fc7426
bpo-38353: Fix typos in calculate_argv0_path_framework() (GH-16695)
bpo-38353, bpo-38429: Fix typos introduced by commit
c02b41b1fb in
calculate_argv0_path_framework() of getpath.c.
2019-10-10 15:42:30 +02:00
Victor Stinner b96145a6b5
bpo-38353: Simplify calculate_pybuilddir() (GH-16614)
Calling _Py_wfopen() is enough to check if filename is an existing
file or not. There is no need to check first isfile().
2019-10-07 12:56:59 +02:00
Victor Stinner c02b41b1fb
bpo-38353: getpath.c: allocates strings on the heap (GH-16585)
* _Py_FindEnvConfigValue() now returns a string allocated
  by PyMem_RawMalloc().
* calculate_init() now decodes VPATH macro.
* Add calculate_open_pyenv() function.
* Add substring() and joinpath2() functions.

* Fix add_exe_suffix()

And a few cleanup changes.
2019-10-04 19:53:43 +02:00
Victor Stinner abd7cd856b
bpo-38353: getpath.c uses dynamically allocated strings (GH-16582)
* PyCalculatePath: add "_macro" to the 3 fields storing macro values
* Add dynamically allocated strings to PyCalculatePath
2019-10-04 15:25:50 +02:00
Victor Stinner f0c85793d2
bpo-38353: Rework ismodule() in getpath.c (GH-16574)
* ismodule() no longer modify the input string.
* copy_absolute(), joinpath(): rename parameters and local variables.
2019-10-04 03:53:26 +02:00
Victor Stinner 03a8a56fac
bpo-38353: Add subfunctions to getpath.c (GH-16572)
Following symbolic links is now limited to 40 attempts, just to
prevent loops.

Add subfunctions:

* Add resolve_symlinks()
* Add calculate_argv0_path_framework()
* Add calculate_which()
* Add calculate_program_macos()

Fix also _Py_wreadlink(): readlink() result type is Py_ssize_t, not
int.
2019-10-04 02:22:39 +02:00
Victor Stinner 86ec5c65fe
bpo-38353: Fix calculate_argv0_path() for symlinks (GH-16549)
calculate_argv0_path() must read argv0_path link, not read
program_full_path link.
2019-10-03 23:17:00 +02:00
Victor Stinner 221fd84703
bpo-38234: Cleanup getpath.c (GH-16367)
* search_for_prefix() directly calls reduce() if found is greater
  than 0.
* Add calculate_pybuilddir() subfunction.
* search_for_prefix(): add path string buffer for readability.
* Fix some error handling code paths: release resources on error.
* calculate_read_pyenv(): rename tmpbuffer to filename.
* test.pythoninfo now also logs windows.dll_path
2019-09-25 02:54:25 +02:00
Victor Stinner bb6bf7d342
bpo-38234: Add tests for Python init path config (GH-16358) 2019-09-24 18:21:02 +02:00
Victor Stinner 85ce0a7178
bpo-38234: read_pth_file() now returns PyStatus (GH-16338)
Refactor path configuration code:

* read_pth_file() now returns PyStatus to report errors, rather than
  calling Py_FatalError().
* Move argv0_path and zip_path buffers out of PyCalculatePath
  structures.
* On Windows, _PyPathConfig.home is now preferred over PyConfig.home.
2019-09-24 00:55:48 +02:00
Victor Stinner 9c42f8cda5
bpo-38234: Fix _PyConfig_InitPathConfig() (GH-16335)
* _PyConfig_InitPathConfig() now starts by copying the global path
  configuration, and then override values set in PyConfig.
* _PyPathConfig_Calculate() implementations no longer override
  _PyPathConfig fields which are already computed. For example,
  if _PyPathConfig.prefix is not NULL, leave it unchanged.
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() doesn't
  call _PyPathConfig_Calculate() anymore.
* _PyPathConfig_Calculate() no longer uses PyConfig,
  except to initialize PyCalculatePath structure.
* pathconfig_calculate(): remove useless temporary
  "_PyPathConfig new_config" variable.
* calculate_module_search_path(): remove hack to workaround memory
  allocation failure, call Py_FatalError() instead.
* Fix get_program_full_path(): handle memory allocation failure.
2019-09-23 18:47:29 +02:00
Victor Stinner e267793aa4
bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298)
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() now
  uses its value.
* Py_Initialize() now longer copies path configuration from PyConfig
  to the global path configuration (_Py_path_config).
2019-09-21 01:50:16 +02:00
Orivej Desh 09090d04ef bpo-37649: Fix exec_prefix check (GH-14897) 2019-09-09 09:05:21 -07:00
Min ho Kim c4cacc8c5e Fix typos in comments, docs and test names (#15018)
* Fix typos in comments, docs and test names

* Update test_pyparse.py

account for change in string length

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: Dealloccte -> Deallocate

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Update posixmodule checksum.

* Reverse idlelib changes.
2019-07-30 18:16:13 -04:00
Victor Stinner 3029035ef3
bpo-20443: Fix calculate_program_full_path() warning (GH-14446)
Don't call _Py_isabs() with a bytes string (char*), the function
expects as wide string.
2019-06-28 16:49:38 +02:00
Victor Stinner 3939c321c9
bpo-20443: _PyConfig_Read() gets the absolute path of run_filename (GH-14053)
Python now gets the absolute path of the script filename specified on
the command line (ex: "python3 script.py"): the __file__ attribute of
the __main__ module, sys.argv[0] and sys.path[0] become an absolute
path, rather than a relative path.

* Add _Py_isabs() and _Py_abspath() functions.
* _PyConfig_Read() now tries to get the absolute path of
  run_filename, but keeps the relative path if _Py_abspath() fails.
* Reimplement os._getfullpathname() using _Py_abspath().
* Use _Py_isabs() in getpath.c.
2019-06-25 15:02:43 +02:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Victor Stinner 871ff77c1c
bpo-36763: Add _PyInitError functions (GH-13395)
* Add _PyInitError functions:

  * _PyInitError_Ok()
  * _PyInitError_Error()
  * _PyInitError_NoMemory()
  * _PyInitError_Exit()
  * _PyInitError_IsError()
  * _PyInitError_IsExit()
  * _PyInitError_Failed()

* frozenmain.c and _testembed.c now use functions rather than macros.
* Move _Py_INIT_xxx() macros to the internal API.
* Move _PyWstrList_INIT macro to the internal API.
2019-05-17 23:54:00 +02:00
Victor Stinner 9ef5dcaa0b
bpo-36763: Add _Py_InitializeMain() (GH-13362)
* Add a private _Py_InitializeMain() function.
* Add again _PyCoreConfig._init_main.
* _Py_InitializeFromConfig() now uses _init_main to decide
  if _Py_InitializeMainInterpreter() should be called.
* _PyCoreConfig: rename _frozen to pathconfig_warnings, its value is
  now the opposite of Py_FrozenFlag.
* Add an unit test for _init_main=0 and _Py_InitializeMain().
2019-05-16 17:38:16 +02:00
Victor Stinner db71975431
bpo-36763: Rework _PyInitError API (GH-13031)
* Remove _PyInitError.user_err field and _Py_INIT_USER_ERR() macro:
  use _Py_INIT_ERR() instead. _Py_ExitInitError() now longer calls
  abort() on error: exit with exit code 1 instead.
* Add _PyInitError._type private field.
* exitcode field type is now unsigned int on Windows.
* Rename prefix field to _func.
* Rename msg field to err_msg.
2019-05-01 05:35:33 +02:00
Victor Stinner faddaedd05
bpo-36352: Avoid hardcoded MAXPATHLEN size in getpath.c (GH-12423)
* Use Py_ARRAY_LENGTH() rather than hardcoded MAXPATHLEN in getpath.c.
* Pass string length to functions modifying strings.
2019-03-19 02:58:14 +01:00
Victor Stinner 5f9cf23502
bpo-36301: Error if decoding pybuilddir.txt fails (GH-12422)
Python initialization now fails if decoding pybuilddir.txt
configuration file fails at startup.

_PyPathConfig_Calculate() now reports memory allocation failure and
decoding error on decoding pybuilddir.txt content from
UTF-8/surrogateescape.
2019-03-19 01:46:25 +01:00
Victor Stinner 7b14f0c02c
bpo-36352: Add error handling to getpath.c (GH-12421)
Replace Py_FatalError() with _PyInitError to let the caller handle
the fatal error.
2019-03-18 23:54:59 +01:00