Commit Graph

125 Commits

Author SHA1 Message Date
Victor Stinner 97308dfcdc
bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h (GH-28922) 2021-10-13 15:03:35 +02:00
Christian Clauss 5f401f1040
Fix typos in the Objects directory (GH-28766) 2021-10-06 16:57:10 -07:00
Steve Dower 6c942a86a4
bpo-45375: Fix off by one error in buffer allocation (GH-28764) 2021-10-06 20:09:33 +01:00
Steve Dower 5146877623
bpo-45375: Fix assertion failure due to searching for stdlib in unnormalised paths (GH-28735) 2021-10-05 13:37:43 +01:00
Eric Snow 0c50b8c0b8
bpo-45211: Remember the stdlib dir during startup. (gh-28586)
During runtime startup we figure out the stdlib dir but currently throw that information away. This change preserves it and exposes it via PyConfig.stdlib_dir, _Py_GetStdlibDir(), and sys._stdlib_dir.

https://bugs.python.org/issue45211
2021-09-28 12:18:28 -06:00
Eric Snow ae7839bbe8
bpo-45211: Move helpers from getpath.c to internal API. (gh-28550)
This accomplishes 2 things:

* consolidates some common code between getpath.c and getpathp.c
* makes the helpers available to code in other files

FWIW, the signature of the join_relfile() function (in fileutils.c) intentionally mirrors that of Windows' PathCchCombineEx().

Note that this change is mostly moving code around. No behavior is meant to change.

https://bugs.python.org/issue45211
2021-09-27 10:00:32 -06:00
Mohamad Mansour 8f943ca257
[codemod] Fix non-matching bracket pairs (GH-28473)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-09-22 01:09:00 +02:00
Steve Dower 936a660945
bpo-41304: Ensure python3x._pth is loaded on Windows (GH-21495) 2020-07-15 22:56:49 +01:00
Steve Dower dcbaa1b49c
bpo-29778: Ensure python3.dll is loaded from correct locations when Python is embedded (GH-21297)
Also enables using debug build of `python3_d.dll`
Reference: CVE-2020-15523
2020-07-06 17:32:00 +01:00
Minmin Gong 98e42d1f88
bpo-35890: Use RegQueryInfoKeyW and CryptAcquireContextW explicitly (GH-19974) 2020-05-18 17:50:03 +01:00
Victor Stinner 361dcdcefc
bpo-40268: Remove unused osdefs.h includes (GH-19532)
When the include is needed, add required symbol in a comment.
2020-04-15 03:24:57 +02:00
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Zackery Spytz 676b105111
bpo-8901: Windows registry path is now ignored with the -E option (GH-18169) 2020-03-30 17:04:45 +01:00
Andy Lester 7668a8bc93
Use calloc-based functions, not malloc. (GH-19152) 2020-03-24 23:26:44 -05:00
Steve Dower 6a65eba44b
bpo-39401: Avoid unsafe DLL load on Windows 7 and earlier (GH-18231)
As Windows 7 is not supported by Python 3.9, we just replace the dynamic load with a static import. Backports will have a different fix to ensure they continue to behave the same.
2020-01-29 13:46:33 +11:00
Victor Stinner c02b41b1fb
bpo-38353: getpath.c: allocates strings on the heap (GH-16585)
* _Py_FindEnvConfigValue() now returns a string allocated
  by PyMem_RawMalloc().
* calculate_init() now decodes VPATH macro.
* Add calculate_open_pyenv() function.
* Add substring() and joinpath2() functions.

* Fix add_exe_suffix()

And a few cleanup changes.
2019-10-04 19:53:43 +02:00
Victor Stinner 61691d8336
bpo-38353: Cleanup includes in the internal C API (GH-16548)
Use forward declaration of types to avoid includes in the internal C
API. Add also comment to justify other includes.
2019-10-02 23:51:20 +02:00
Victor Stinner dec39716ca
bpo-38322: Fix gotlandmark() of PC/getpathp.c (GH-16489)
Write the filename into a temporary buffer instead of reusing prefix.
The problem is that join() modifies prefix inplace. If prefix is not
normalized, join() can make prefix shorter and so gotlandmark()
does modify prefix instead of returning it unmodified.
2019-09-30 14:49:34 +02:00
Victor Stinner 8bf39b606e
bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402)
* Add test_embed.test_init_setpath_config(): test Py_SetPath()
  with PyConfig.
* test_init_setpath() and test_init_setpythonhome() no longer call
  Py_SetProgramName(), but use the default program name.
* _PyPathConfig: isolated, site_import  and base_executable
  fields are now only available on Windows.
* If executable is set explicitly in the configuration, ignore
  calculated base_executable: _PyConfig_InitPathConfig() copies
  executable to base_executable.
* Complete path config documentation.
2019-09-26 02:22:35 +02:00
Victor Stinner 221fd84703
bpo-38234: Cleanup getpath.c (GH-16367)
* search_for_prefix() directly calls reduce() if found is greater
  than 0.
* Add calculate_pybuilddir() subfunction.
* search_for_prefix(): add path string buffer for readability.
* Fix some error handling code paths: release resources on error.
* calculate_read_pyenv(): rename tmpbuffer to filename.
* test.pythoninfo now also logs windows.dll_path
2019-09-25 02:54:25 +02:00
Victor Stinner 85ce0a7178
bpo-38234: read_pth_file() now returns PyStatus (GH-16338)
Refactor path configuration code:

* read_pth_file() now returns PyStatus to report errors, rather than
  calling Py_FatalError().
* Move argv0_path and zip_path buffers out of PyCalculatePath
  structures.
* On Windows, _PyPathConfig.home is now preferred over PyConfig.home.
2019-09-24 00:55:48 +02:00
Victor Stinner 9c42f8cda5
bpo-38234: Fix _PyConfig_InitPathConfig() (GH-16335)
* _PyConfig_InitPathConfig() now starts by copying the global path
  configuration, and then override values set in PyConfig.
* _PyPathConfig_Calculate() implementations no longer override
  _PyPathConfig fields which are already computed. For example,
  if _PyPathConfig.prefix is not NULL, leave it unchanged.
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() doesn't
  call _PyPathConfig_Calculate() anymore.
* _PyPathConfig_Calculate() no longer uses PyConfig,
  except to initialize PyCalculatePath structure.
* pathconfig_calculate(): remove useless temporary
  "_PyPathConfig new_config" variable.
* calculate_module_search_path(): remove hack to workaround memory
  allocation failure, call Py_FatalError() instead.
* Fix get_program_full_path(): handle memory allocation failure.
2019-09-23 18:47:29 +02:00
Victor Stinner fcdb027234
bpo-38236: Dump path config at first import error (GH-16300)
Python now dumps path configuration if it fails to import the Python
codecs of the filesystem and stdio encodings.
2019-09-23 14:45:47 +02:00
Victor Stinner e267793aa4
bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298)
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() now
  uses its value.
* Py_Initialize() now longer copies path configuration from PyConfig
  to the global path configuration (_Py_path_config).
2019-09-21 01:50:16 +02:00
Victor Stinner c422167749
bpo-38234: Remove _PyPathConfig.dll_path (GH-16307)
The DLL path is not computed from any user configuration and cannot
be configured by PyConfig. Instead, add a new _Py_dll_path global variable.

Remove _PyConfig_SetPathConfig(): replaced with _PyPathConfig_Init().

Py_Initialize() now longer sets the "global path configuration",
but only initialize _Py_dll_path.
2019-09-21 01:02:56 +02:00
Steve Dower 9048c49322
bpo-37369: Fix initialization of sys members when launched via an app container (GH-14428)
sys._base_executable is now always defined on all platforms, and can be overridden through configuration.
Also adds test.support.PythonSymlink to encapsulate platform-specific logic for symlinking sys.executable
2019-06-29 10:34:11 -07:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Victor Stinner 410759fba8
bpo-36763: Remove _PyCoreConfig.dll_path (GH-13402) 2019-05-18 04:17:01 +02:00
Victor Stinner 871ff77c1c
bpo-36763: Add _PyInitError functions (GH-13395)
* Add _PyInitError functions:

  * _PyInitError_Ok()
  * _PyInitError_Error()
  * _PyInitError_NoMemory()
  * _PyInitError_Exit()
  * _PyInitError_IsError()
  * _PyInitError_IsExit()
  * _PyInitError_Failed()

* frozenmain.c and _testembed.c now use functions rather than macros.
* Move _Py_INIT_xxx() macros to the internal API.
* Move _PyWstrList_INIT macro to the internal API.
2019-05-17 23:54:00 +02:00
Steve Dower 1c3de541e6
bpo-34977: Use venv redirector instead of original python.exe on Windows (GH-11029) 2018-12-10 08:11:21 -08:00
Victor Stinner cb0b78a070
Revert "bpo-34977: Add Windows App Store package (GH-10245)" (GH-11019)
This reverts commit 468a15aaf9.
2018-12-07 12:57:43 +01:00
Zackery Spytz 4c49da0cb7 bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015)
Set MemoryError when appropriate, add missing failure checks,
and fix some potential leaks.
2018-12-07 12:11:30 +02:00
Steve Dower 468a15aaf9
bpo-34977: Add Windows App Store package (GH-10245) 2018-12-06 21:09:20 -08:00
Steve Dower 177a41a07b
bpo-34725: Adds _Py_SetProgramFullPath so embedders may override sys.executable (GH-9860) 2018-11-17 20:41:48 -08:00
Victor Stinner 621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
erikjanss 6cf8255912 bpo-34217: Use lowercase header for Windows (GH-8453) 2018-07-25 09:41:46 +09:00
Victor Stinner d19d8d5279
bpo-34170: Add _PyCoreConfig.isolated (GH-8417)
* _PyCoreConfig: add isolated and site_import attributes
* Replace Py_IgnoreEnvironment with config->ignore_environment when
  reading the current configuration
* _PyCoreConfig_Read() now sets ignore_environment, utf8_mode,
  isolated and site_import from Py_IgnoreEnvironment, Py_UTF8Mode,
  Py_IsolatedFlag and Py_NoSiteFlag
* _Py_InitializeCore() now sets Py_xxx flags from the configuration
* pymain_read_conf() now uses _PyCoreConfig_Copy() to save/restore
  the configuration.
* Rename _disable_importlib of _PyCoreConfig to _install_importlib
* _PyCoreConfig_SetGlobalConfig() now also set
  Py_HashRandomizationFlag
* Replace !Py_NoSiteFlag with core_config->site_import
2018-07-24 13:55:48 +02:00
Victor Stinner f2626ce6d4
bpo-34170: _PyCoreConfig_Read() leaves Py_IsolatedFlag unchanged (GH-8361)
* _PyCoreConfig_Read() no longer directly modifies Py_IsolatedFlag
  and Py_NoSiteFlag global configuration flags. The function now
  requires two pointers to integer, so these flags can be set later,
  to avoid side effets in _PyCoreConfig_Read().
* pathconfig_global_init() now leaves Py_IsolatedFlag and
  Py_NoSiteFlag unchanged.
* Fix pathconfig_global_init(): avoid computing the path
  configuration twice, use _PyCoreConfig_SetPathConfig().
2018-07-21 03:54:20 +02:00
Victor Stinner b1147e43da
bpo-34170: Rework _PyCoreConfig_Read() to avoid side effect (GH-8353)
Rework _PyCoreConfig_Read() function which *reads* core configuration
to not *modify* the path configuration.

A new _PyCoreConfig_SetPathConfig() function now recreates the path
configuration from the core configuration. This function is now
called very late in _Py_InitializeCore(), just before calling
initimport().

Changes:

* Add _PyCoreConfig.dll_path
* Py_SetPath() now fails with a fatal python error on memory
   allocation failure.
* Rename _PyPathConfig_Calculate() to _PyPathConfig_Calculate_impl()
* Replace _PyPathConfig_Init() with _PyPathConfig_Calculate(): the
  function now requires a _PyPathConfig
* Add _PyPathConfig_SetGlobal() to set the _Py_path_config global
  variable.
* Add _PyCoreConfig_InitPathConfig(): compute the path configuration
* Add _PyCoreConfig_SetPathConfig(): set path configuration from core
  configuration
* Rename wstrlist_append() to _Py_wstrlist_append()
* _Py_wstrlist_append() now handles integer overflow.
2018-07-21 02:06:16 +02:00
Steve Dower 48e8c82fc6
bpo-32457: Improves handling of denormalized executable path when launching Python (GH-5756) 2018-02-22 10:39:26 -08:00
Victor Stinner 8ded5b8037
bpo-32030: Add _PyCoreConfig.module_search_paths (#4954)
_PyCoreConfig_Read() is now responsible to compute sys.path.
So sys.path is now computed before calling _Py_InitializeCore().

Changes:

* Add module_search_path, module_search_paths, executable, prefix,
  base_prefix, exec_prefix and base_exec_prefix to _PyCoreConfig.
* _PyMainInterpreterConfig_Read() now only converts wchar_t** lists
  into a Python list, it doesn't compute sys.path anymore.
2018-01-24 17:03:28 +01:00
Victor Stinner 9bee329130
bpo-32030: Add _Py_FindEnvConfigValue() (#4963)
Add a new _Py_FindEnvConfigValue() function: code shared between
Windows and Unix implementations of _PyPathConfig_Calculate() to read
the pyenv.cfg file.

_Py_FindEnvConfigValue() now uses _Py_DecodeUTF8_surrogateescape()
instead of using a Python Unicode string, the Python API must not be
used early during Python initialization. Same change in Unix
search_for_exec_prefix(): use _Py_DecodeUTF8_surrogateescape().

Cleanup also encode_current_locale(): PyMem_RawFree/PyMem_Free can be
called with NULL.

Fix also "NUL byte" => "NULL byte" typo.
2017-12-21 16:49:13 +01:00
Victor Stinner b5fd9ad05e
bpo-32030: Rewrite _PyMainInterpreterConfig (#4854)
_PyMainInterpreterConfig now contains Python objects, whereas
_PyCoreConfig contains wchar_t* strings.

Core config:

* Rename _PyMainInterpreterConfig_ReadEnv() to _PyCoreConfig_ReadEnv()
* Move 3 strings from _PyMainInterpreterConfig to _PyCoreConfig:
  module_search_path_env, home, program_name.
* Add _PyCoreConfig_Clear()
* _PyPathConfig_Calculate() now takes core config rather than main
  config
* _PyMainInterpreterConfig_Read() now requires also a core config

Main config:

* Add _PyMainInterpreterConfig.module_search_path: sys.path list
* Add _PyMainInterpreterConfig.argv: sys.argv list
* _PyMainInterpreterConfig_Read() now computes module_search_path
2017-12-14 02:20:52 +01:00
Serhiy Storchaka 4ae06c5337
bpo-32241: Add the const qualifire to declarations of umodifiable strings. (#4748) 2017-12-12 13:55:04 +02:00
Victor Stinner 31a8393cf6
Revert "bpo-32197: Try to fix a compiler error on OS X introduced in bpo-32030. (#4681)" (#4694)
* Revert "bpo-32197: Try to fix a compiler error on OS X introduced in bpo-32030. (#4681)"

This reverts commit 13badcbc60.

Re-apply commits:

* "bpo-32030: _PyPathConfig_Init() sets home and program_name (#4673)"
  commit af5a895073.
* "bpo-32030: Fix config_get_program_name() on macOS (#4669)"
  commit e23c06e2b0.
* "bpo-32030: Add Python/pathconfig.c (#4668)"
  commit 0ea395ae96.
* "bpo-32030: Don't call _PyPathConfig_Fini() in Py_FinalizeEx() (#4667)"
  commit ebac19dad6.
* "bpo-32030: Fix Py_GetPath(): init program_name (#4665)"
  commit 9ac3d88827.

* Fix compilation error on macOS
2017-12-04 13:39:15 +01:00
Serhiy Storchaka 13badcbc60
bpo-32197: Try to fix a compiler error on OS X introduced in bpo-32030. (#4681)
* Revert "bpo-32030: _PyPathConfig_Init() sets home and program_name (#4673)"

This reverts commit af5a895073.

* Revert "bpo-32030: Fix config_get_program_name() on macOS (#4669)"

This reverts commit e23c06e2b0.

* Revert "bpo-32030: Add Python/pathconfig.c (#4668)"

This reverts commit 0ea395ae96.

* Revert "bpo-32030: Don't call _PyPathConfig_Fini() in Py_FinalizeEx() (#4667)"

This reverts commit ebac19dad6.

* Revert "bpo-32030: Fix Py_GetPath(): init program_name (#4665)"

This reverts commit 9ac3d88827.
2017-12-02 21:36:00 +02:00
Victor Stinner af5a895073
bpo-32030: _PyPathConfig_Init() sets home and program_name (#4673)
_PyPathConfig_Init() now also initialize home and program_name:

* Rename existing _PyPathConfig_Init() to _PyPathConfig_Calculate().
  Add a new _PyPathConfig_Init() function in pathconfig.c which
  handles the _Py_path_config variable and call
  _PyPathConfig_Calculate().
* Add home and program_name fields to _PyPathConfig.home
* _PyPathConfig_Init() now initialize home and program_name
  from main_config
* Py_SetProgramName(), Py_SetPythonHome() and Py_GetPythonHome() now
  calls Py_FatalError() on failure, instead of silently ignoring
  failures.
* config_init_home() now gets directly _Py_path_config.home to only
  get the value set by Py_SetPythonHome(), or NULL if
  Py_SetPythonHome() was not called.
* config_get_program_name() now gets directly
  _Py_path_config.program_name to only get the value set by
  Py_SetProgramName(), or NULL if Py_SetProgramName() was not called.
* pymain_init_python() doesn't call Py_SetProgramName() anymore,
  _PyPathConfig_Init() now always sets the program name
* Call _PyMainInterpreterConfig_Read() in
  pymain_parse_cmdline_envvars_impl() to control the memory allocator
* C API documentation: it's no more safe to call Py_GetProgramName()
  before Py_Initialize().
2017-12-02 10:11:32 +01:00
Victor Stinner 0ea395ae96
bpo-32030: Add Python/pathconfig.c (#4668)
* Factorize code from PC/getpathp.c and Modules/getpath.c to remove
  duplicated code
* rename pathconfig_clear() to _PyPathConfig_Clear()
* Inline _PyPathConfig_Fini() in pymain_impl() and then remove it,
  since it's a oneliner
2017-12-01 20:50:58 +01:00
Victor Stinner ebac19dad6
bpo-32030: Don't call _PyPathConfig_Fini() in Py_FinalizeEx() (#4667)
Changes:

* _PyPathConfig_Fini() cannot be called in Py_FinalizeEx().
  Py_Initialize() and Py_Finalize() can be called multiple times, but
  it must not "forget" parameters set by Py_SetProgramName(),
  Py_SetPath() or Py_SetPythonHome(), whereas _PyPathConfig_Fini()
  clear all these parameters.
* config_get_program_name() and calculate_program_full_path() now
  also decode paths using Py_DecodeLocale() to use the
  surrogateescape error handler, rather than decoding using
  mbstowcs() which is strict.
* Change _Py_CheckPython3() prototype: () => (void)
* Truncate a few lines which were too long
2017-12-01 20:09:52 +01:00