The test.support.wait_process() function now uses a timeout of
LONG_TIMEOUT seconds by default, instead of SHORT_TIMEOUT. It
doesn't matter if a Python buildbot is slower, it only matters that
the process completes. The timeout should just be shorter than
"forever".
(see https://github.com/python/cpython/issues/98608)
This change does the following:
1. change the argument to a new `_PyInterpreterConfig` struct
2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig` instead of `isolated_subinterpreter`)
3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings
* allow_fork
* allow_subprocess
* allow_threads
4. add `PyInterpreterState.feature_flags` to store those settings
5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *`
6. drop `PyConfig._isolated_interpreter`
The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three. None of that changes here; the defaults are preserved.
Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved. This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`. I'd rather just avoid that mess from the start for `_PyInterpreterConfig`. We can preserve it later if we find an actual need.
This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead).
(Note that this PR adds "private" symbols. We'll probably make them public, and add docs, in a separate change.)
test_tools.test_sundry() now uses an unittest mock to prevent the
logging module to register a real "atfork" function which kept the
logging module dictionary alive. So the logging module can be
properly unloaded. Previously, the logging module was loaded before
test_sundry(), but it's no longer the case since recent test_tools
sub-tests removals.
Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.
This PR comes fresh from a pile of work done in our private PSRT security response team repo.
Signed-off-by: Christian Heimes [Red Hat] <christian@python.org>
Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org>
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).
<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->
I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#). Much of that text wound up in the Issue. Backports PRs already exist. See the issue for links.
- Limited API needs to be enabled per source file
- Some builds don't support Limited API, so Limited API tests must be skipped on those builds
(currently this is `Py_TRACE_REFS`, but that may change.)
- `Py_LIMITED_API` must be defined before `<Python.h>` is included.
This puts the hoop-jumping in `testcapi/parts.h`, so individual
test files can be relatively simple. (Currently that's only
`vectorcall_limited.c`, imagine more.)
``os.geteuid() == 0`` is not a reliable check whether the current user
has the capability to bypass permission checks. Tests now probe for DAC
override.
test_asyncio, test_logging, test_socket and test_socketserver now
create AF_UNIX domains in the current directory to no longer fail
with OSError("AF_UNIX path too long") if the temporary directory (the
TMPDIR environment variable) is too long.
Modify the following tests to use create_unix_domain_name():
* test_asyncio
* test_logging
* test_socket
* test_socketserver
test_asyncio.utils: remove unused time import.
* Replace time.sleep(0.010) with sleeping_retry() to
use an exponential sleep.
* support.wait_process(): reuse sleeping_retry().
* _test_eintr: remove unused variables.
- WASI's ``gethostname()`` is a stub that always fails with OSError
``ENOTSUP``
- skip mailcap ``test`` if subprocess is not available
- WASI process_time clock does not work.
- Add requires_fork and requires_subprocess to more tests
- Skip extension import tests if dlopen is not available
- Don't assume that _testcapi is a shared extension
- Skip a lot of socket tests that don't work on Emscripten
- Skip mmap tests, mmap emulation is incomplete
- venv does not work yet
- Cannot get libc from executable
The "entire" test suite is now passing on Emscripten with EMSDK from git head (91 suites are skipped).
- fd inheritance can't be modified because Emscripten doesn't support subprocesses anyway.
- setpriority always fails
- geteuid no longer causes problems with latest emsdk
- umask is a stub
- geteuid / getuid always return 0, but process cannot chown to random uid.
- getgroups always fails.
- geteuid and getegid always return 0 (root), which confuse tarfile and
tests.
- hardlinks (link, linkat) always fails.
- non-encodable file names are not supported by NODERAWFS layer.
- mark more tests with dependency on subprocess and multiprocessing.
Mocking does not work if the module fails to import.
In [Lib/test/support/import_helper.py](https://github.com/python/cpython/blob/master/Lib/test/support/import_helper.py), the function `make_legacy_pyc` makes a call to `os.rename` which can fail when the source and target live on different devices. This happens (for example) when `PYTHONPYCACHEPREFIX` is set to a directory anywhere on disk, while a ramdisk is mounted on `/tmp` (the latter of which is the default on various Linux distros). Replacing `os.rename` with `shutil.move` fixes this.
Automerge-Triggered-By: GH:brettcannon
skip_if_buggy_ucrt_strfptime() of test.support now uses
locale.getpreferredencoding(False) instead of
locale.getdefaultlocale() to get the Windows code page.
It was added as part of #29222 to avoid running freeze tool tests on the
buildbots but the logic was wrong so it did not skip tests on typical posix
setup buildbots where the worker is launched from cron via an @reboot task and
thus have no USER environment variable. This uses the canonical
`getpass.getuser()` API rather than rolling its own attempt.
Irit Katriel
Nikita Sobolev
Victor Stinner
Henry Schreiner
Eric Snow
Mark Shannon
Dennis Sweeney
Gregory P. Smith
Petr Viktorin
Christian Heimes
Stanislav Zmiev
Serhiy Storchaka
Jason R. Coombs
Jelle Zijlstra
Inada Naoki
Brett Cannon
Oleg Iarygin
Jason Wilkes
Gregory P. Smith
Pablo Galindo Salgado