Commit Graph

119142 Commits

Author SHA1 Message Date
Nikita Sobolev 31097df611
gh-101100: Fix sphinx warnings in `library/site.rst` (#110144)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-10-01 20:18:19 +03:00
dependabot[bot] a431a0f988
build(deps): bump hypothesis from 6.84.0 to 6.87.1 in /Tools (#110174)
Bumps [hypothesis](https://github.com/HypothesisWorks/hypothesis) from 6.84.0 to 6.87.1.
- [Release notes](https://github.com/HypothesisWorks/hypothesis/releases)
- [Commits](https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-6.84.0...hypothesis-python-6.87.1)

---
updated-dependencies:
- dependency-name: hypothesis
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-01 17:25:29 +01:00
Alex Waygood d642c5bbf5
gh-110180: Remove unused `_PickleUsingNameMixin` class from `typing` (#110181) 2023-10-01 16:28:02 +01:00
Barney Gale 15de493395
GH-107465: Add `pathlib.Path.from_uri()` classmethod. (#107640)
This method supports file URIs (including variants) as described in RFC 8089, such as URIs generated by `pathlib.Path.as_uri()` and `urllib.request.pathname2url()`.

The method is added to `Path` rather than `PurePath` because it uses `os.fsdecode()`, and so its results vary from system to system. I intend to deprecate `PurePath.as_uri()` and move it to `Path` for the same reason.

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-10-01 16:14:02 +01:00
Nikita Sobolev 06faa9a39b
gh-110160: Fix flaky `test_find_periodic_pattern` in `string_tests` (#110170) 2023-10-01 16:20:01 +02:00
dependabot[bot] 038c3564fb
PEG generator: bump types-setuptools from 68.1.0.1 to 68.2.0.0 (#110175)
build(deps-dev): bump types-setuptools in /Tools

Bumps [types-setuptools](https://github.com/python/typeshed) from 68.1.0.1 to 68.2.0.0.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-setuptools
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-01 12:38:46 +01:00
Raymond Hettinger 62405c7867
gh-110150: Fix base case handling in quantiles() (gh-110151) 2023-09-30 23:35:54 -05:00
Victor Stinner a46e960768
gh-109649: Use os.process_cpu_count() (#110165)
Replace os.cpu_count() with os.process_cpu_count() in modules:

* compileall
* concurrent.futures
* multiprocessing

Replace os.cpu_count() with os.process_cpu_count() in programs:

* _decimal deccheck.py test
* freeze.py
* multissltests.py
* python -m test (regrtest)
* wasm_build.py

Other changes:

* test.pythoninfo logs os.process_cpu_count().
* regrtest gets os.process_cpu_count() / os.cpu_count() in headers.
2023-10-01 03:14:57 +02:00
Victor Stinner 53eb9a676f
gh-110152: regrtest handles cross compilation and HOSTRUNNER (#110156)
* _add_python_opts() now handles cross compilation and HOSTRUNNER.
* display_header() now tells if Python is cross-compiled, display
  HOSTRUNNER, and get the host platform.
* Remove Tools/scripts/run_tests.py script.
* Remove "make hostrunnertest": use "make buildbottest"
  or "make test" instead.
2023-09-30 22:37:23 +00:00
Victor Stinner d3728ddc57
gh-110014: Fix bootstrap_hash.c: remove debug code (#110161)
Oops, I commited debug code by mistake, sorry about that.
2023-09-30 22:21:20 +00:00
Victor Stinner c81521020d
gh-109649: Add os.process_cpu_count() function (#109907)
* Refactor os_sched_getaffinity_impl(): move variable definitions to
  their first assignment.
* Fix test_posix.test_sched_getaffinity(): restore the old CPU mask
  when the test completes!
* Doc: Specify that os.cpu_count() counts *logicial* CPUs.
* Doc: Specify that os.sched_getaffinity(0) is related to the calling
  thread.
2023-10-01 00:12:51 +02:00
Victor Stinner 2c234196ea
gh-109276: regrtest: add WORKER_FAILED state (#110148)
Rename WORKER_ERROR to WORKER_BUG. Add WORKER_FAILED state: it does
not stop the manager, whereas WORKER_BUG does.

Change also TestResults.display_result() order: display failed tests
at the end, the important important information.

WorkerThread now tries to get the signal name for negative exit code.
2023-09-30 22:48:26 +02:00
Victor Stinner c62b49ecc8
gh-110088: Fix asyncio test_prompt_cancellation() (#110157)
Don't measure the CI performance: don't test the maximum elapsed
time. The check failed on a slow CI.
2023-09-30 22:40:10 +02:00
Victor Stinner 7513994c92
gh-110014: Include explicitly <unistd.h> header (#110155)
* Remove unused <locale.h> includes.
* Remove unused <fcntl.h> include in traceback.h.
* Remove redundant <assert.h> and <stddef.h> includes. They  are already
  included by "Python.h".
* Remove <object.h> include in faulthandler.c. Python.h already includes it.
* Add missing <stdbool.h> in pycore_pythread.h if HAVE_PTHREAD_STUBS
  is defined.
* Fix also warnings in pthread_stubs.h: don't redefine macros if they
  are already defined, like the __NEED_pthread_t macro.
2023-09-30 20:06:45 +00:00
Victor Stinner 0def8c712b
gh-109748: Fix again venv test_zippath_from_non_installed_posix() (#110149)
Call also copy_python_src_ignore() on listdir() names.

shutil.copytree(): replace set() with an empty tuple. An empty tuple
becomes a constant in the compiler and checking if an item is in an
empty tuple is cheap.
2023-09-30 20:23:26 +02:00
Victor Stinner 74e425ec18
gh-110014: Fix _POSIX_THREADS and _POSIX_SEMAPHORES usage (#110139)
* pycore_pythread.h is now the central place to make sure that
  _POSIX_THREADS and _POSIX_SEMAPHORES macros are defined if
  available.
* Make sure that pycore_pythread.h is included when _POSIX_THREADS
  and _POSIX_SEMAPHORES macros are tested.
* PY_TIMEOUT_MAX is now defined as a constant, since its value
  depends on _POSIX_THREADS, instead of being defined as a macro.
* Prevent integer overflow in the preprocessor when computing
  PY_TIMEOUT_MAX_VALUE on Windows:
  replace "0xFFFFFFFELL * 1000 < LLONG_MAX"
  with "0xFFFFFFFELL < LLONG_MAX / 1000".
* Document the change and give hints how to fix affected code.
* Add an exception for PY_TIMEOUT_MAX  name to smelly.py
* Add PY_TIMEOUT_MAX to the stable ABI
2023-09-30 19:25:54 +02:00
Victor Stinner f3bb00ea12
gh-107954: Refactor initconfig.c: add CONFIG_SPEC (#110146)
Add a specification of the PyConfig structure to factorize the code.
2023-09-30 17:24:06 +00:00
Barney Gale 89966a694b
GH-89812: Add `pathlib._PathBase` (#106337)
Add private `pathlib._PathBase` class. This will be used by an experimental PyPI package to incubate a `tarfile.TarPath` class.

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-09-30 15:45:01 +01:00
Adam Turner 0449fe999d
GH-101100: Fix reference warnings for ``gettext`` (#110115) 2023-09-30 05:10:07 -06:00
Adam Turner cbdacc738a
GH-101100: Fix reference warnings for ``namedtuple`` (#110113) 2023-09-29 23:32:35 -07:00
Raymond Hettinger 613c0d4e86
Add example for linear_regression() with proportional=True. (gh-110133) 2023-09-29 23:18:12 -05:00
Victor Stinner 14098b78f7
gh-107888: Fix test_mmap PROT_EXEC comment (#110125) 2023-09-29 23:56:19 +02:00
Sam Gross 2973970af8
gh-110119: Temporarily skip test_cppext on --disable-gil builds. (#110123)
The current version of pip does not support "t" in the ABI flags. Skip
the test in `--disable-gil` builds until we can update pip.
2023-09-29 13:50:51 -07:00
Victor Stinner 5ae6c6d053
gh-109566: regrtest --fast-ci no longer enables --nowindows (#110121)
The --nowindows option is deprecated and does nothing but logs a
warning.
2023-09-29 19:49:39 +00:00
Victor Stinner 9c91141fff
gh-109566: Remove make testall (#110122)
Remove "make testall" target: use "make buildbottest" instead.
2023-09-29 21:36:03 +02:00
Victor Stinner 6351842121
gh-109047: concurrent.futures catches PythonFinalizationError (#109810)
concurrent.futures: The *executor manager thread* now catches
exceptions when adding an item to the *call queue*. During Python
finalization, creating a new thread can now raise RuntimeError. Catch
the exception and call terminate_broken() in this case.

Add test_python_finalization_error() to test_concurrent_futures.

concurrent.futures._ExecutorManagerThread changes:

* terminate_broken() no longer calls shutdown_workers() since the
  call queue is no longer working anymore (read and write ends of
  the queue pipe are closed).
* terminate_broken() now terminates child processes, not only
  wait until they complete.
* _ExecutorManagerThread.terminate_broken() now holds shutdown_lock
  to prevent race conditons with ProcessPoolExecutor.submit().

multiprocessing.Queue changes:

* Add _terminate_broken() method.
* _start_thread() sets _thread to None on exception to prevent
  leaking "dangling threads" even if the thread was not started
  yet.
2023-09-29 19:31:19 +00:00
Victor Stinner f3df8fa669
gh-109566: PCbuild/rt.bat now uses --fast-ci (#110120)
Replace "--fail-env-changed --fail-rerun" with "--fast-ci".

Tools/buildbot/test.bat pass --slow-ci which has the priority over
--fast-ci.
2023-09-29 19:16:29 +00:00
Furkan Onder 3439cb0049
gh-66143: Allow copying and pickling of CodecInfo object (GH-109235)
Co-authored-by: Robert Lehmann <mail@robertlehmann.de>
2023-09-29 20:07:09 +03:00
James Hilton-Balfe 7d57288f6d
gh-109495: Remove unused slots from the Python implementation of datetime (GH-109494) 2023-09-29 19:57:32 +03:00
Steve Dower 20bc5f7c28
gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore (GH-110108) 2023-09-29 16:24:38 +01:00
Eric Snow 6364873d2a
gh-110024: Fix Pointer Type Warnings (gh-110053)
The warnings were introduced by gh-109794 (for gh-109793).
2023-09-29 09:20:23 -06:00
Victor Stinner 9c73a9acec
gh-109592: test_eintr tolerates 20 ms when comparing timings (#110102) 2023-09-29 15:20:59 +02:00
Victor Stinner 86e76ab8af
gh-110031: Skip test_threading fork tests if ASAN (#110100)
Skip test_threading tests using thread+fork if Python is built with
Address Sanitizer (ASAN).
2023-09-29 12:41:33 +00:00
Victor Stinner 743e3572ee
gh-109974: Fix more threading lock_tests race conditions (#110089)
* Add context manager on Bunch class.
* Bunch now catchs exceptions on executed functions and re-raise them
  at __exit__() as an ExceptionGroup.
* Rewrite BarrierProxy.test_default_timeout(). Use a single thread.
  Only check that barrier.wait() blocks for at least default timeout
  seconds.
* test_with(): inline _with() function.
2023-09-29 12:21:18 +00:00
Donghee Na 501939c9c1
gh-105323: Update readline module to detect apple editline variant (gh-108665) 2023-09-29 21:18:18 +09:00
Victor Stinner db0a258e79
gh-110088, gh-109878: Fix test_asyncio timeouts (#110092)
Fix test_asyncio timeouts: don't measure the maximum duration, a test
should not measure a CI performance. Only measure the minimum
duration when a task has a timeout or delay. Add CLOCK_RES to
test_asyncio.utils.
2023-09-29 11:49:30 +00:00
Jacob Coffee e27adc68cc
gh-109634: Fix `:samp:` syntax (GH-110073) 2023-09-29 14:21:34 +03:00
Victor Stinner e260087a8e
gh-108716: make regen-global-objects no longer builds deepfreeze.c (#110078)
Remove more references to now unused Python/deepfreeze/deepfreeze.c.
2023-09-29 10:17:49 +00:00
Victor Stinner 8b626a47ba
gh-110079: Remove extern "C" { ...} in C code (#110080) 2023-09-29 10:56:49 +02:00
Yuki K bfd94ab9e9
gh-101100: Fix references to ``URLError`` and ``HTTPError`` in ``howto/urllib2.rst`` (#107966)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-09-29 08:35:29 +00:00
Maciej Olko 8898a8683b
gh-101100: Fix Sphinx warnings in `tutorial/controlflow.rst` (#109424)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-09-29 11:27:43 +03:00
Nikita Sobolev f1b1680a72
gh-109961: Use proper `module` for `copy` method docs (#110027)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-09-29 11:26:55 +03:00
Nikita Sobolev d102d39bbe
gh-101100: Fix sphinx warnings in `library/difflib.rst` (#110074) 2023-09-29 11:03:59 +03:00
Pieter Eendebak 05079d93e4
gh-109868: Skip deepcopy memo check for empty memo (GH-109869) 2023-09-29 10:28:01 +03:00
Jelle Zijlstra 7dc2c5093e
gh-110045: Update symtable module for PEP 695 (#110066) 2023-09-29 02:08:04 +00:00
Victor Stinner 2e37a38bcb
gh-110052: Fix faulthandler for freed tstate (#110069)
faulthandler now detected freed interp and freed tstate, and no
longer dereference them.
2023-09-29 02:04:06 +00:00
Victor Stinner 235aacdeed
gh-109566: regrtest _add_python_opts() handles KeyboardInterrupt (#110062)
In the subprocess code path, wait until the child process completes
with a timeout of EXIT_TIMEOUT seconds.

Fix create_worker_process() regression: use start_new_session=True if
USE_PROCESS_GROUP is true.

WorkerThread.wait_stopped() uses a timeout of 60 seconds, instead of
30 seconds.
2023-09-29 00:51:22 +00:00
Victor Stinner bd4518c60c
gh-110036: multiprocessing Popen.terminate() catches PermissionError (#110037)
On Windows, multiprocessing Popen.terminate() now catchs
PermissionError and get the process exit code. If the process is
still running, raise again the PermissionError. Otherwise, the
process terminated as expected: store its exit code.
2023-09-29 02:41:12 +02:00
Victor Stinner 4e356ad183
gh-109974: Fix threading lock_tests race conditions (#110057)
Fix race conditions in test_threading lock tests. Wait until a
condition is met rather than using time.sleep() with a hardcoded
number of seconds.

* Replace sleeping loops with support.sleeping_retry() which raises
  an exception on timeout.
* Add wait_threads_blocked(nthread) which computes a sleep depending
  on the number of threads. Remove _wait() function.
* test_set_and_clear(): use a way longer Event.wait() timeout.
* BarrierTests.test_repr(): wait until the 2 threads are waiting for
  the barrier. Use a way longer timeout for Barrier.wait() timeout.
* test_thread_leak() no longer needs to count
  len(threading.enumerate()): Bunch uses
  threading_helper.wait_threads_exit() internally which does it in
  wait_for_finished().
* Add BaseLockTests.wait_phase() which implements a timeout.
  test_reacquire() and test_recursion_count() use wait_phase().
2023-09-29 02:34:27 +02:00
Victor Stinner 5fdcea7440
gh-109960: Remove test_pty timeout of 10 seconds (#110058)
In 2003, test_pty got a hardcoded timeout of 10 seconds to prevent
hanging on AIX & HPUX "if run after test_openpty":
commit 7d8145268e. Since 2003, test_pty
was no longer reported to hang on AIX. But today, the test can fail
simply because a CI is busy running other tests in parallel.
The timeout of 10 seconds is no longer needed, just remove it.
Moreover, regrtest now has multiple built-in generic timeout
mecanisms.
2023-09-29 00:01:24 +00:00