For now, we'll rely on the fact that the config structures aren't covered by the stable ABI.
We may revisit this in the future if we further explore the idea of offering a stable embedding API.
Fix a bug due to the interaction of weakrefs and the cyclic garbage
collector. We must clear any weakrefs in garbage in order to prevent
their callbacks from executing and causing a crash.
(cherry picked from commit bcda460baf)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
Write the filename into a temporary buffer instead of reusing prefix.
The problem is that join() modifies prefix inplace. If prefix is not
normalized, join() can make prefix shorter and so gotlandmark()
does modify prefix instead of returning it unmodified.
test_hmac and test_hashlib test built-in hashing implementations and
OpenSSL-based hashing implementations. Add more checks to skip OpenSSL
implementations when a strict crypto policy is active.
Use EVP_DigestInit_ex() instead of EVP_DigestInit() to initialize the
EVP context. The EVP_DigestInit() function clears alls flags and breaks
usedforsecurity flag again.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38270.
(cherry picked from commit 9055815809)
Co-authored-by: Christian Heimes <christian@python.org>
Fix warnings options priority: PyConfig.warnoptions has the highest
priority, as stated in the PEP 587.
* Document options order in PyConfig.warnoptions documentation.
* Make PyWideStringList_INIT macro private: replace "Py" prefix
with "_Py".
* test_embed: add test_init_warnoptions().
(cherry picked from commit fb4ae152a9)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Document that lnotab can contain invalid bytecode offsets (because of
terrible reasons that are difficult to fix). Make dis.findlinestarts()
ignore invalid offsets in lnotab. All other uses of lnotab in CPython
(various reimplementations of addr2line or line2addr in Python, C and gdb)
already ignore this, because they take an address to look for, instead.
Add tests for the result of dis.findlinestarts() on wacky constructs in
test_peepholer.py, because it's the easiest place to add them.
(cherry picked from commit c8165036f3)
* bpo-38216: Allow bypassing input validation
* bpo-36274: Also allow the URL encoding to be overridden.
* bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL.
* Call with skip_host to avoid tripping on the host checking in the URL.
* Remove obsolete comment.
* Make _prepare_path_encoding its own attr.
This makes overriding just that simpler.
Also, don't use the := operator to make backporting easier.
* Add a news entry.
* _prepare_path_encoding -> _encode_prepared_path()
* Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.
(cherry picked from commit 7774d7831e)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Add a new struct_size field to PyPreConfig and PyConfig structures to
allow to modify these structures in the future without breaking the
backward compatibility.
* Replace private _config_version field with public struct_size field
in PyPreConfig and PyConfig.
* Public PyPreConfig_InitIsolatedConfig() and
PyPreConfig_InitPythonConfig()
return type becomes PyStatus, instead of void.
* Internal _PyConfig_InitCompatConfig(),
_PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(),
_PyPreConfig_InitFromPreConfig() return type becomes PyStatus,
instead of void.
* Remove _Py_CONFIG_VERSION
* Update the Initialization Configuration documentation.
(cherry picked from commit 441b10cf28)
Escape the server title of xmlrpc.server.DocXMLRPCServer
when rendering the document page as HTML.
(cherry picked from commit e8650a4f8c)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
test_ssl now handles disabled TLS/SSL versions better. OpenSSL's crypto
policy and run-time settings are recognized and tests for disabled versions
are skipped.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38275
(cherry picked from commit df6ac7e2b8)
* bpo-38234: Py_SetPath() uses the program full path (GH-16357)
Py_SetPath() now sets sys.executable to the program full path
(Py_GetProgramFullPath()), rather than to the program name
(Py_GetProgramName()).
Fix also memory leaks in pathconfig_set_from_config().
(cherry picked from commit 1ce152a42e)
* bpo-38234: Add tests for Python init path config (GH-16358)
(cherry picked from commit bb6bf7d342)
* bpo-38234: test_embed: test pyvenv.cfg and pybuilddir.txt (GH-16366)
Add test_init_pybuilddir() and test_init_pyvenv_cfg() to test_embed
to test pyvenv.cfg and pybuilddir.txt configuration files.
Fix sysconfig._generate_posix_vars(): pybuilddir.txt uses UTF-8
encoding, not ASCII.
(cherry picked from commit 52ad33abbf)
* bpo-38234: Cleanup getpath.c (GH-16367)
* search_for_prefix() directly calls reduce() if found is greater
than 0.
* Add calculate_pybuilddir() subfunction.
* search_for_prefix(): add path string buffer for readability.
* Fix some error handling code paths: release resources on error.
* calculate_read_pyenv(): rename tmpbuffer to filename.
* test.pythoninfo now also logs windows.dll_path
(cherry picked from commit 221fd84703)
* bpo-38234: Fix test_embed pathconfig tests (GH-16390)
bpo-38234: On macOS and FreeBSD, the temporary directory can be
symbolic link. For example, /tmp can be a symbolic link to /var/tmp.
Call realpath() to resolve all symbolic links.
(cherry picked from commit 00508a7407)
* bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402)
* Add test_embed.test_init_setpath_config(): test Py_SetPath()
with PyConfig.
* test_init_setpath() and test_init_setpythonhome() no longer call
Py_SetProgramName(), but use the default program name.
* _PyPathConfig: isolated, site_import and base_executable
fields are now only available on Windows.
* If executable is set explicitly in the configuration, ignore
calculated base_executable: _PyConfig_InitPathConfig() copies
executable to base_executable.
* Complete path config documentation.
(cherry picked from commit 8bf39b606e)
* bpo-38234: Complete init config documentation (GH-16404)
(cherry picked from commit 88feaecd46)
* bpo-38234: Fix test_embed.test_init_setpath_config() on FreeBSD (GH-16406)
Explicitly preinitializes with a Python preconfiguration to avoid
Py_SetPath() implicit preinitialization with a compat
preconfiguration.
Fix also test_init_setpath() and test_init_setpythonhome() on macOS:
use self.test_exe as the executable (and base_executable), rather
than shutil.which('python3').
(cherry picked from commit 49d99f01e6)
* bpo-38234: Py_Initialize() sets global path configuration (GH-16421)
* Py_InitializeFromConfig() now writes PyConfig path configuration to
the global path configuration (_Py_path_config).
* Add test_embed.test_get_pathconfig().
* Fix typo in _PyWideStringList_Join().
(cherry picked from commit 12f2f177fc)
The private keys for test_ssl were encrypted with 3DES in traditional
PKCSGH-5 format. 3DES and the digest algorithm of PKCSGH-5 are blocked by
some strict crypto policies. Use PKCSGH-8 format with AES256 encryption
instead.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38271
Automerge-Triggered-By: @tiran
(cherry picked from commit bfd0c963d8)
Co-authored-by: Christian Heimes <christian@python.org>
Make negative interpreter id to raise ValueError instead of RuntimeError.
(cherry picked from commit 543a3951a1)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Make it easier to run and test Python on systems with restrict crypto policies:
* add requires_hashdigest to test.support to check if a hash digest algorithm is available and working
* avoid MD5 in test_hmac
* replace MD5 with SHA256 in test_tarfile
* mark network tests that require MD5 for MD5-based digest auth or CRAM-MD5
https://bugs.python.org/issue38270
(cherry picked from commit c64a1a61e6)
Co-authored-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38270
Automerge-Triggered-By: @tiran
If Python is installed, skip test_tools.test_pathfix test because
Tools/scripts/pathfix.py script is not installed.
(cherry picked from commit 3f43ceff18)
* bpo-37064: Add option -k to Tools/scripts/pathfix.py (GH-15548)
Add flag -k to pathscript.py script: preserve shebang flags.
(cherry picked from commit 50254ac4c1)
* bpo-37064: Add option -a to pathfix.py tool (GH-15717)
Add option -a to Tools/Scripts/pathfix.py script: add flags.
(cherry picked from commit 1dc1acbd73)
The fix in PR 13261 handled the underlying issue about the spec for specific methods not being applied correctly, but it didn't fix the issue that was causing the misleading error message.
The code currently grabs a list of responses from _call_matcher (which may include exceptions). But it doesn't reach inside the list when checking if the result is an exception. This results in a misleading error message when one of the provided calls does not match the spec.
https://bugs.python.org/issue36871
Automerge-Triggered-By: @gpshead
(cherry picked from commit b5a7a4f0c2)
Co-authored-by: Samuel Freilich <sfreilich@google.com>
https://bugs.python.org/issue36871
Automerge-Triggered-By: @gpshead
Multiprocessing test test_mymanager() now also expects -SIGTERM, not
only exitcode 0.
bpo-30356: BaseManager._finalize_manager() sends SIGTERM to the
manager process if it takes longer than 1 second to stop, which
happens on slow buildbots.
(cherry picked from commit b0e1ae5f54)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Multiprocessing tests: increase test_queue_feeder_donot_stop_onexc()
timeout from 1 to 60 seconds.
(cherry picked from commit 99799c7220)
Co-authored-by: Victor Stinner <vstinner@python.org>
This fixes the exception '`ValueError: invalid literal for int() with base 10`
if `str(gdbval)` returns a hexadecimal value (e.g. '0xa0'). This is the case if
the output-radix is set to 16 in gdb. See
https://sourceware.org/gdb/onlinedocs/gdb/Numbers.html for more information.
(cherry picked from commit 6f53d34fb0)
Co-authored-by: Marc Hartmayer <marc1006@users.noreply.github.com>
Refactor path configuration code:
* read_pth_file() now returns PyStatus to report errors, rather than
calling Py_FatalError().
* Move argv0_path and zip_path buffers out of PyCalculatePath
structures.
* On Windows, _PyPathConfig.home is now preferred over PyConfig.home.
(cherry picked from commit 85ce0a7178)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* _PyConfig_InitPathConfig() now starts by copying the global path
configuration, and then override values set in PyConfig.
* _PyPathConfig_Calculate() implementations no longer override
_PyPathConfig fields which are already computed. For example,
if _PyPathConfig.prefix is not NULL, leave it unchanged.
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() doesn't
call _PyPathConfig_Calculate() anymore.
* _PyPathConfig_Calculate() no longer uses PyConfig,
except to initialize PyCalculatePath structure.
* pathconfig_calculate(): remove useless temporary
"_PyPathConfig new_config" variable.
* calculate_module_search_path(): remove hack to workaround memory
allocation failure, call Py_FatalError() instead.
* Fix get_program_full_path(): handle memory allocation failure.
(cherry picked from commit 9c42f8cda5)
Python now dumps path configuration if it fails to import the Python
codecs of the filesystem and stdio encodings.
(cherry picked from commit fcdb027234)
Victor Stinner
Miss Islington (bot)
Victor Stinner
Christian Heimes
Yury Selivanov
Lisa Roach
Gregory P. Smith
Benjamin Peterson
Vinay Sajip
Michael Felt