Commit Graph

66 Commits

Author SHA1 Message Date
Inada Naoki 6773203487
bpo-47000: Add `locale.getencoding()` (GH-32068) 2022-04-09 09:54:54 +09:00
Steve Dower 99fcf15052
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
2021-12-03 00:08:42 +00:00
Victor Stinner a6c3b0faa1
bpo-45954: Rename PyConfig.no_debug_ranges to code_debug_ranges (GH-29886)
Rename PyConfig.no_debug_ranges to PyConfig.code_debug_ranges and
invert the value.

Document -X no_debug_ranges and PYTHONNODEBUGRANGES env var in
PyConfig.code_debug_ranges documentation.
2021-12-02 11:43:37 +01:00
Eric Snow b9cdd0fb9c
bpo-45020: Default to using frozen modules unless running from source tree. (gh-28940)
The default was "off".  Switching it to "on" means users get the benefit of frozen stdlib modules without having to do anything.  There's a special-case for running-in-source-tree, so contributors don't get surprised when their stdlib changes don't get used.

https://bugs.python.org/issue45020
2021-10-16 13:16:08 -06:00
Victor Stinner aac29af678
bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
Include <stdlib.h> explicitly in C files.

Python.h includes <wchar.h>.
2021-10-13 19:25:53 +02:00
Pablo Galindo Salgado db2b6a20cd
bpo-45445: Fail if an invalid X-option is provided in the command line (GH-28823) 2021-10-13 18:08:19 +01:00
Victor Stinner 489176e428
bpo-45434: Convert Py_GETENV() macro to a function (GH-28912)
Avoid calling directly getenv() in the header file.
2021-10-13 03:39:50 +02:00
Christian Clauss db693df3e1
Fix typos in the Python directory (GH-28767) 2021-10-06 15:55:27 -07:00
Eric Snow 0c50b8c0b8
bpo-45211: Remember the stdlib dir during startup. (gh-28586)
During runtime startup we figure out the stdlib dir but currently throw that information away. This change preserves it and exposes it via PyConfig.stdlib_dir, _Py_GetStdlibDir(), and sys._stdlib_dir.

https://bugs.python.org/issue45211
2021-09-28 12:18:28 -06:00
Eric Snow ae7839bbe8
bpo-45211: Move helpers from getpath.c to internal API. (gh-28550)
This accomplishes 2 things:

* consolidates some common code between getpath.c and getpathp.c
* makes the helpers available to code in other files

FWIW, the signature of the join_relfile() function (in fileutils.c) intentionally mirrors that of Windows' PathCchCombineEx().

Note that this change is mostly moving code around. No behavior is meant to change.

https://bugs.python.org/issue45211
2021-09-27 10:00:32 -06:00
Eric Snow a65c86889e
bpo-45020: Add -X frozen_modules=[on|off] to explicitly control use of frozen modules. (gh-28320)
Currently we freeze several modules into the runtime. For each of these modules it is essential to bootstrapping the runtime that they be frozen. Any other stdlib module that we later freeze into the runtime is not essential. We can just as well import from the .py file.  This PR lets users explicitly choose which should be used, with the new "-X frozen_modules=[on|off]" CLI flag. The default is "off" for now.

https://bugs.python.org/issue45020
2021-09-14 17:31:45 -06:00
Dong-hee Na c2c857b40f
bpo-44895: Introduce PYTHONDUMPREFSFILE variable for refcount dumping (GH-27767) 2021-08-18 00:52:50 +09:00
Ammar Askar 4823d9a512
bpo-43950: Add option to opt-out of PEP-657 (GH-27023)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
2021-07-07 20:07:12 +01:00
Inada Naoki 4827483f47
bpo-43510: Implement PEP 597 opt-in EncodingWarning. (GH-19481)
See [PEP 597](https://www.python.org/dev/peps/pep-0597/).

* Add `-X warn_default_encoding` and `PYTHONWARNDEFAULTENCODING`.
* Add EncodingWarning
* Add io.text_encoding()
* open(), TextIOWrapper() emits EncodingWarning when encoding is omitted and warn_default_encoding is enabled.
* _pyio.TextIOWrapper() uses UTF-8 as fallback default encoding used when failed to import locale module. (used during building Python)
* bz2, configparser, gzip, lzma, pathlib, tempfile modules use io.text_encoding().
* What's new entry
2021-03-29 12:28:14 +09:00
Victor Stinner bcb094b41f
bpo-43268: Pass interp rather than tstate to internal functions (GH-24580)
Pass the current interpreter (interp) rather than the current Python
thread state (tstate) to internal functions which only use the
interpreter.

Modified functions:

* _PyXXX_Fini() and _PyXXX_ClearFreeList() functions
* _PyEval_SignalAsyncExc(), make_pending_calls()
* _PySys_GetObject(), sys_set_object(), sys_set_object_id(), sys_set_object_str()
* should_audit(), set_flags_from_config(), make_flags()
* _PyAtExit_Call()
* init_stdio_encoding()
* etc.
2021-02-19 15:10:45 +01:00
Pablo Galindo 99b594404d
bpo-42521: Add note about 'Python -d' only working on debug builds (GH-23607) 2020-12-02 17:56:17 +00:00
Serhiy Storchaka 14d81dcaf8
bpo-42260: Improve error handling in _PyConfig_FromDict (GH-23488) 2020-11-24 14:07:32 +02:00
Victor Stinner ace3f9a0ce
bpo-42260: Fix _PyConfig_Read() if compute_path_config=0 (GH-23220)
Fix _PyConfig_Read() if compute_path_config=0: use values set by
Py_SetPath(), Py_SetPythonHome() and Py_SetProgramName(). Add
compute_path_config parameter to _PyConfig_InitPathConfig().

The following functions now return NULL if called before
Py_Initialize():

* Py_GetExecPrefix()
* Py_GetPath()
* Py_GetPrefix()
* Py_GetProgramFullPath()
* Py_GetProgramName()
* Py_GetPythonHome()

These functions no longer automatically computes the Python Path
Configuration. Moreover, Py_SetPath() no longer computes
program_full_path.
2020-11-10 21:10:22 +01:00
Victor Stinner 9e1b828265
bpo-42260: Compute the path config in the main init (GH-23211)
The path configuration is now computed in the "main" initialization.
The core initialization no longer computes it.

* Add _PyConfig_Read() function to read the configuration without
  computing the path configuration.
* pyinit_core() no longer computes the path configuration: it is now
  computed by init_interp_main().
* The path configuration output members of PyConfig are now optional:

  * executable
  * base_executable
  * prefix
  * base_prefix
  * exec_prefix
  * base_exec_prefix

* _PySys_UpdateConfig() now skips NULL strings in PyConfig.
* _testembed: Rename test_set_config() to test_init_set_config() for
  consistency with other tests.
2020-11-10 13:21:52 +01:00
Victor Stinner dc42af8fd1
bpo-42260: PyConfig_Read() only parses argv once (GH-23168)
The PyConfig_Read() function now only parses PyConfig.argv arguments
once: PyConfig.parse_argv is set to 2 after arguments are parsed.
Since Python arguments are strippped from PyConfig.argv, parsing
arguments twice would parse the application options as Python
options.

* Rework the PyConfig documentation.
* Fix _testinternalcapi.set_config() error handling.
* SetConfigTests no longer needs parse_argv=0 when restoring the old
  configuration.
2020-11-05 18:58:07 +01:00
Victor Stinner f3cb814315
bpo-42260: Add _PyConfig_FromDict() (GH-23167)
* Rename config_as_dict() to _PyConfig_AsDict().
* Add 'module_search_paths_set' to _PyConfig_AsDict().
* Add _PyConfig_FromDict().
* Add get_config() and set_config() to _testinternalcapi.
* Add config_check_consistency().
2020-11-05 18:12:33 +01:00
Victor Stinner 048a35659a
bpo-42260: Add _PyInterpreterState_SetConfig() (GH-23158)
* Inline _PyInterpreterState_SetConfig(): replace it with
  _PyConfig_Copy().
* Add _PyErr_SetFromPyStatus()
* Add _PyInterpreterState_GetConfigCopy()
* Add a new _PyInterpreterState_SetConfig() function.
* Add an unit which gets, modifies, and sets the config.
2020-11-05 00:45:56 +01:00
Victor Stinner 3529718925
bpo-42236: os.device_encoding() respects UTF-8 Mode (GH-23119)
On Unix, the os.device_encoding() function now returns 'UTF-8' rather
than the device encoding if the Python UTF-8 Mode is enabled.
2020-11-04 11:20:10 +01:00
Victor Stinner 4b9aad4999
bpo-42236: Enhance init and encoding documentation (GH-23109)
Enhance the documentation of the Python startup, filesystem encoding
and error handling, locale encoding. Add a new "Python UTF-8 Mode"
section.

* Add "locale encoding" and "filesystem encoding and error handler"
  to the glossary
* Remove documentation from Include/cpython/initconfig.h: move it to
  Doc/c-api/init_config.rst.
* Doc/c-api/init_config.rst:

  * Document command line options and environment variables
  * Document default values.

* Add a new "Python UTF-8 Mode" section in Doc/library/os.rst.
* Add warnings to Py_DecodeLocale() and Py_EncodeLocale() docs.
* Document how Python selects the filesystem encoding and error
  handler at a single place: PyConfig.filesystem_encoding and
  PyConfig.filesystem_errors.
* PyConfig: move orig_argv member at the right place.
2020-11-02 16:49:54 +01:00
Victor Stinner e662c398d8
bpo-42236: Use UTF-8 encoding if nl_langinfo(CODESET) fails (GH-23086)
If the nl_langinfo(CODESET) function returns an empty string, Python
now uses UTF-8 as the filesystem encoding.

In May 2010 (commit b744ba1d14), I
modified Python to log a warning and use UTF-8 as the filesystem
encoding (instead of None) if nl_langinfo(CODESET) returns an empty
string.

In August 2020 (commit 94908bbc15), I
modified Python startup to fail with a fatal error and a specific
error message if nl_langinfo(CODESET) returns an empty string. The
intent was to prevent guessing the encoding and also investigate user
configuration where this case happens.

In 10 years (2010 to 2020), I saw zero user report about the error
message related to nl_langinfo(CODESET) returning an empty string.

Today, UTF-8 became the defacto standard and it's safe to make the
assumption that the user expects UTF-8. For example,
nl_langinfo(CODESET) can return an empty string on macOS if the
LC_CTYPE locale is not supported, and UTF-8 is the default encoding
on macOS.

While this change is likely to not affect anyone in practice, it
should make UTF-8 lover happy ;-)

Rewrite also the documentation explaining how Python selects the
filesystem encoding and error handler.
2020-11-01 23:07:23 +01:00
Victor Stinner 82458b6cdb
bpo-42236: Enhance _locale._get_locale_encoding() (GH-23083)
* Rename _Py_GetLocaleEncoding() to _Py_GetLocaleEncodingObject()
* Add _Py_GetLocaleEncoding() which returns a wchar_t* string to
  share code between _Py_GetLocaleEncodingObject()
  and config_get_locale_encoding().
* _Py_GetLocaleEncodingObject() now decodes nl_langinfo(CODESET)
  from the current locale encoding with surrogateescape,
  rather than using UTF-8.
2020-11-01 20:59:35 +01:00
Victor Stinner 710e826307
bpo-42208: Add _Py_GetLocaleEncoding() (GH-23050)
_io.TextIOWrapper no longer calls getpreferredencoding(False) of
_bootlocale to get the locale encoding, but calls
_Py_GetLocaleEncoding() instead.

Add config_get_fs_encoding() sub-function. Reorganize also
config_get_locale_encoding() code.
2020-10-31 01:02:09 +01:00
Samuel Marks c322948892
bpo-41819: Fix compiler warning in init_dump_ascii_wstr() (GH-22332)
Fix the compiler warning:

format specifies type `wint_t` (aka `int`) but the argument has type `unsigned int`
2020-09-21 10:35:17 +02:00
Victor Stinner 640e8e1d5f
Fix compiler warnings in init_dump_ascii_wstr() (GH-22150)
Fix GCC 9.3 (using -O3) warnings on x86:

initconfig.c: In function ‘init_dump_ascii_wstr’:
initconfig.c:2679:34: warning: format ‘%lc’ expects argument of type
‘wint_t’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
 2679 |             PySys_WriteStderr("%lc", ch);
initconfig.c:2682:38: warning: format ‘%x’ expects argument of type
‘unsigned int’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
 2682 |             PySys_WriteStderr("\\x%02x", ch);
initconfig.c:2686:38: warning: format ‘%x’ expects argument of type
‘unsigned int’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
 2686 |             PySys_WriteStderr("\\U%08x", ch);
initconfig.c:2690:38: warning: format ‘%x’ expects argument of type
‘unsigned int’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
 2690 |             PySys_WriteStderr("\\u%04x", ch);
2020-09-09 12:07:17 +02:00
Serhiy Storchaka 58de1dd6a8
bpo-41525: Make the Python program help ASCII-only (GH-21836) 2020-09-09 01:28:02 +01:00
Victor Stinner 8f42748ded
bpo-29778: test_embed tests the path configuration (GH-21306) 2020-07-08 00:20:37 +02:00
Victor Stinner dd8a93e23b
bpo-23427: Add sys.orig_argv attribute (GH-20729)
Add sys.orig_argv attribute: the list of the original command line
arguments passed to the Python executable.

Rename also PyConfig._orig_argv to PyConfig.orig_argv and
document it.
2020-06-30 00:49:03 +02:00
Victor Stinner e2d47a0568
bpo-40910: PyConfig_Clear() clears _orig_argv (GH-20886)
bpo-40910, bpo-40953: PyConfig_Clear() clears _orig_argv.
2020-06-15 16:27:47 +02:00
Pablo Galindo 1ed83adb0e
bpo-40939: Remove the old parser (GH-20768)
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
2020-06-11 17:30:46 +01:00
Victor Stinner e81f6e687d
bpo-40910: Export Py_GetArgcArgv() function (GH-20721)
Export explicitly the Py_GetArgcArgv() function to the C API and
document the function. Previously, it was exported implicitly which
no longer works since Python is built with -fvisibility=hidden.

* Add PyConfig._orig_argv member.
* Py_InitializeFromConfig() no longer calls _PyConfig_Write() twice.
* PyConfig_Read() no longer initializes Py_GetArgcArgv(): it is now
  _PyConfig_Write() responsibility.
* _PyConfig_Write() result type becomes PyStatus instead of void.
* Write an unit test on Py_GetArgcArgv().
2020-06-08 18:12:59 +02:00
Sandro Mani 8f023a2f66
bpo-40854: Allow overriding sys.platlibdir via PYTHONPLATLIBDIR env-var (GH-20605) 2020-06-08 17:28:11 +02:00
Victor Stinner 252346acd9
bpo-40453: Add PyConfig._isolated_subinterpreter (GH-19820)
An isolated subinterpreter cannot spawn threads, spawn a child
process or call os.fork().

* Add private _Py_NewInterpreter(isolated_subinterpreter) function.
* Add isolated=True keyword-only parameter to
  _xxsubinterpreters.create().
* Allow again os.fork() in "non-isolated" subinterpreters.
2020-05-01 11:33:44 +02:00
Victor Stinner 1def7754b7
bpo-40334: Rename PyConfig.use_peg to _use_peg_parser (GH-19670)
* Rename PyConfig.use_peg to _use_peg_parser
* Document PyConfig._use_peg_parser and mark it a deprecated
* Mark -X oldparser option and PYTHONOLDPARSER env var as deprecated
  in the documentation.
* Add use_old_parser() and skip_if_new_parser() to test.support
* Remove sys.flags.use_peg: use_old_parser() uses
  _testinternalcapi.get_configs() instead.
* Enhance test_embed tests
* subprocess._args_from_interpreter_flags() copies -X oldparser
2020-04-23 03:03:24 +02:00
Pablo Galindo c5fc156852
bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-04-22 23:29:27 +01:00
Victor Stinner 4f98f465f1
bpo-40268: Remove unused imports in pylifecycle.c (GH-19533)
Remove unused imports in files:

* initconfig.c
* main.c
* preconfig.h
* pylifecycle.c
* python.c
* pythonrun.c
2020-04-15 04:01:58 +02:00
Victor Stinner d9ea5cae1d
bpo-40268: Remove unused pycore_pymem.h includes (GH-19531) 2020-04-15 02:57:50 +02:00
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Victor Stinner 4a3fe08353
bpo-40268: Include explicitly pycore_interp.h (GH-19505)
pycore_pystate.h no longer includes pycore_interp.h:
it's now included explicitly in files accessing PyInterpreterState.
2020-04-14 14:26:24 +02:00
Victor Stinner da7933ecc3
bpo-40268: Add _PyInterpreterState_GetConfig() (GH-19492)
Don't access PyInterpreterState.config member directly anymore, but
use new functions:

* _PyInterpreterState_GetConfig()
* _PyInterpreterState_SetConfig()
* _Py_GetConfig()
2020-04-13 03:04:28 +02:00
Ronald Oussoren 044cf94f61
bpo-22490: Remove __PYVENV_LAUNCHER__ from environment during launch (GH-9516)
* bpo-22490: Remove "__PYVENV_LAUNCHER__" from the shell environment on macOS

This changeset removes the environment varialbe "__PYVENV_LAUNCHER__"
during interpreter launch as it is only needed to communicate between
the stub executable in framework installs and the actual interpreter.

Leaving the environment variable present may lead to misbehaviour when
launching other scripts.

* Actually commit the changes for issue 22490...

* Correct typo

Co-Authored-By: Nicola Soranzo <nicola.soranzo@gmail.com>

* Run make patchcheck

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
2020-03-22 14:31:46 -04:00
Victor Stinner ff4584caca
bpo-39947: Use _PyInterpreterState_GET_UNSAFE() (GH-18978)
Replace _PyInterpreterState_Get() function call with
_PyInterpreterState_GET_UNSAFE() macro which is more efficient but
don't check if tstate or interp is NULL.

_Py_GetConfigsAsDict() now uses _PyThreadState_GET().
2020-03-13 18:03:56 +01:00
Andy Lester aa450a0364
closes bpo-39886: Remove unused arg from config_get_stdio_errors. (GH-18823) 2020-03-07 09:36:04 -08:00
Victor Stinner c6e5c1123b
bpo-39489: Remove COUNT_ALLOCS special build (GH-18259)
Remove:

* COUNT_ALLOCS macro
* sys.getcounts() function
* SHOW_ALLOC_COUNT code in listobject.c
* SHOW_TRACK_COUNT code in tupleobject.c
* PyConfig.show_alloc_count field
* -X showalloccount command line option
* @test.support.requires_type_collecting decorator
2020-02-03 15:17:15 +01:00
Pablo Galindo 41f0ef6abb bpo-39427: Document -X opt options in the CLI --help and the man page (GH-18131)
https://bugs.python.org/issue39427



Automerge-Triggered-By: @pablogsal
2020-01-22 17:03:04 -08:00
Inada Naoki 95826c773a
Add PYTHONUTF8 to commandline usage. (GH-17587)
Co-Authored-By: Victor Stinner <vstinner@python.org>
2019-12-14 14:27:32 +09:00