Commit Graph

119025 Commits

Author SHA1 Message Date
Guido van Rossum d67edcf0b3
gh-109979: Auto-generate the target for DEOPT_IF() (#110193)
In Python/bytecodes.c, you now write
```
    DEOPT_IF(condition);
```
The code generator expands this to
```
    DEOPT_IF(condition, opcode);
```
where `opcode` is the name of the unspecialized instruction.
This works inside macro expansions too.

**CAVEAT:** The entire `DEOPT_IF(condition)` statement must be on a single line.
If it isn't, the substitution will fail; an error will be printed by the code generator
and the C compiler will report some errors.
2023-10-03 10:13:50 -07:00
Victor Stinner d73501602f
gh-108867: Add PyThreadState_GetUnchecked() function (#108870)
Add PyThreadState_GetUnchecked() function: similar to
PyThreadState_Get(), but don't issue a fatal error if it is NULL. The
caller is responsible to check if the result is NULL. Previously,
this function was private and known as _PyThreadState_UncheckedGet().
2023-10-03 16:53:51 +00:00
Victor Stinner 6ab6040054
gh-110276: No longer ignore PROFILE_TASK failure silently (#110295) 2023-10-03 18:43:23 +02:00
Victor Stinner eeb4e974d0
gh-107073: Mention pythoncapi-compat for PyObject_VisitManagedDict() (#110291) 2023-10-03 15:50:10 +00:00
Eric Snow f5198b09e1
gh-109860: Use a New Thread State When Switching Interpreters, When Necessary (gh-110245)
In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread.  For the main interpreter there wasn't much of a problem, but for subinterpreters we were *mostly* okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()).  There was a good chance that tstate wasn't actually in use by another thread.

However, there are no guarantees of that.  Furthermore, re-using an already used tstate is currently fragile.  To address this, now we create a new thread state in each of those places and use it.

One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).
2023-10-03 09:20:48 -06:00
Lincoln 4227bfa8b2
Enhanced sqlite3 connection context management documentation with contextlib.closing gh-109234 (#109322)
* Enhanced sqlite3 connection context management documentation with contextlib.closing

* 📜🤖 Added by blurb_it.

* Fixed gitignore spelling error from nitignore to gitignore

* Renamed .gitignore to .nitignore

* Added generated doctests

* Deleted sqlite3 generated files

* Removed white-space changes

* Removed News entry from the doc

* Expanded a note that context manager can be used for connection management using contextlib.closing

* Removed repeated contextlib.closing code snippet

* Expanded the note around usage of context manageer for sqlite3 connection management

* Deleted extra white-spaces

* Deleted extra white-space

* re-arranged context manager wording

* Re-arranged word layout on how to use context manager

* Fix whitespace errors

* Remove unneeded change in .gitignore

* Added suggested changes

* Added suggested change redirecting to the contextlib.closing implementation

* Added closing keyword

* Removed line 2473

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-10-03 17:32:43 +03:00
Xuehai Pan 2d4865d775
gh-110267: Add tests for pickling and copying PyStructSequence objects (GH-110272) 2023-10-03 17:09:49 +03:00
Alex Waygood f1663a492e
Bump various dependencies in `Doc/requirements-oldest-sphinx.txt` (#110278)
This resolves a Dependabot security alert on the repository for urllib3==2.0.4.
2023-10-03 14:10:03 +01:00
Alex Waygood dddc757303
gh-110276: Run `test_str`, not `test_unicode`, as part of the PGO build (#110277)
`test_unicode` does not exist
2023-10-03 13:54:21 +01:00
Nikita Sobolev bb2e96f6f4
gh-109956: Also test typing.NamedTuple with copy.replace() (GH-109957) 2023-10-03 14:13:13 +03:00
Lele Gaifax 8c071373f1
Fix typo in py312 whatsnew: add missing closing paren (#110255) 2023-10-03 07:10:16 +01:00
Hugo van Kemenade 1001233fe1
Remove unused Misc/requirements-test.txt (#110240) 2023-10-03 08:01:50 +03:00
Guido van Rossum 6678ef41d2
Add --inline-caches flag to dis command line (#110249) 2023-10-03 00:49:34 +00:00
Alex Waygood 21a6263020
gh-109653: Reduce the import time of `random` by 60% (#110221) 2023-10-02 22:56:31 +00:00
Nikita Sobolev 4596c76d1a
gh-110241: Add missing error check to `record_eval` in `_testinternalcapi` (#110242) 2023-10-02 21:19:32 +00:00
Eric Snow a8f5dab58d
gh-76785: Module-level Fixes for test.support.interpreters (gh-110236)
* add RecvChannel.close() and SendChannel.close()
* make RecvChannel and SendChannel shareable
* expose ChannelEmptyError and ChannelNotEmptyError
2023-10-02 20:47:41 +00:00
Alex Waygood 014aacda62
Enable ruff on `Lib/test/test_typing.py` (#110179) 2023-10-02 14:13:48 -06:00
Eric Snow 1dd9dee45d
gh-105716: Support Background Threads in Subinterpreters Consistently (gh-109921)
The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.

We add PyInterpreterState.threads.main, with some internal C-API functions.
2023-10-02 20:12:12 +00:00
Eric Snow a040a32ea2
gh-109853: Fix sys.path[0] For Subinterpreters (gh-109994)
This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all. This PR does not address the broader concerns from gh-109853.
2023-10-02 19:59:05 +00:00
Victor Stinner fc2cb86d21
gh-107073: Make PyObject_VisitManagedDict() public (#108763)
Make PyObject_VisitManagedDict() and PyObject_ClearManagedDict()
functions public in Python 3.13 C API.

* Rename _PyObject_VisitManagedDict() to PyObject_VisitManagedDict().
* Rename _PyObject_ClearManagedDict() to PyObject_ClearManagedDict().
* Document these functions.
2023-10-02 19:24:08 +02:00
Victor Stinner 6387b5313c
gh-108494: Document how to add a project in PCbuild/readme.txt (#110077)
Add _testclinic_limited to Tools/msi/test/test_files.wxs.
2023-10-02 18:53:38 +02:00
Victor Stinner 4d0d1c3866
gh-110014: Remove PY_TIMEOUT_MAX from limited C API (#110217)
If the timeout is greater than PY_TIMEOUT_MAX,
PyThread_acquire_lock_timed() uses a timeout of PY_TIMEOUT_MAX
microseconds, which is around 280.6 years. This case is unlikely and
limiting a timeout to 280.6 years sounds like a reasonable trade-off.

The constant PY_TIMEOUT_MAX is not used in PyPI top 5,000 projects.
2023-10-02 18:07:56 +02:00
Jelle Zijlstra 732ad44cec
gh-110178: Use fewer weakrefs in test_typing.py (#110194)
Confirmed that without the C changes from #108517, this test still segfaults with only 10 weakrefs.
2023-10-02 08:03:53 -07:00
numbermaniac 8d92b6eff3
3.12 What's New: Remove duplicate "up to" (#110219) 2023-10-02 13:13:44 +00:00
Steve Dower 1b3bc610fd
gh-83180: Made launcher treat shebang 'python' tags as low priority so that active virtual environments are preferred (GH-108101) 2023-10-02 12:22:55 +00:00
Adam Turner 6139bf5e0c
GH-109190: Announce final release in What's New in Python 3.12 (#110117)
Prepare What's New in Python 3.12 for final release
2023-10-02 13:15:58 +02:00
AN Long 44b1e4ea48
gh-108963: using random to generate unique string in sys.intern test (#109491) 2023-10-02 13:07:56 +02:00
Adam Turner f16e81f368
GH-92584: Move installation schemes overview to sysconfig docs (#108018)
* Add new installation path functions subsection

* Add content from install/index to sysconfig

* Fix table

* Update note about installers

* Clean up the list of schemes, remove references to Distutils
2023-10-02 12:40:03 +02:00
Hugo van Kemenade 9cb8927bfc
Docs: bump Pygments to fix contrast ratios to meet WCAG AA guidelines (#110208) 2023-10-02 13:31:23 +03:00
Charles Machalow 29b875bb93
gh-109590: Update shutil.which on Windows to prefer a PATHEXT extension on executable files (GH-109995)
The default arguments for shutil.which() request an executable file, but extensionless files are not executable on Windows and should be ignored.
2023-10-02 09:27:30 +01:00
Ned Batchelder 29c3a445d9
fix misaligned versionchanged blocks in sqlite3 docs (GH-110191) 2023-10-01 20:34:09 -07:00
Victor Stinner 65c285062c
gh-110164: regrtest disables random if SOURCE_DATE_EPOCH (#110168)
If the SOURCE_DATE_EPOCH environment variable is defined, regrtest
now disables randomization of tests.
2023-10-01 22:41:03 +02:00
Quentin Agren adf0f15a06
gh-110138: Improve grammar in idiomatic usage of ``__main__.py`` (#110142) 2023-10-01 13:32:43 -05:00
Nikita Sobolev 31097df611
gh-101100: Fix sphinx warnings in `library/site.rst` (#110144)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-10-01 20:18:19 +03:00
dependabot[bot] a431a0f988
build(deps): bump hypothesis from 6.84.0 to 6.87.1 in /Tools (#110174)
Bumps [hypothesis](https://github.com/HypothesisWorks/hypothesis) from 6.84.0 to 6.87.1.
- [Release notes](https://github.com/HypothesisWorks/hypothesis/releases)
- [Commits](https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-6.84.0...hypothesis-python-6.87.1)

---
updated-dependencies:
- dependency-name: hypothesis
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-01 17:25:29 +01:00
Alex Waygood d642c5bbf5
gh-110180: Remove unused `_PickleUsingNameMixin` class from `typing` (#110181) 2023-10-01 16:28:02 +01:00
Barney Gale 15de493395
GH-107465: Add `pathlib.Path.from_uri()` classmethod. (#107640)
This method supports file URIs (including variants) as described in RFC 8089, such as URIs generated by `pathlib.Path.as_uri()` and `urllib.request.pathname2url()`.

The method is added to `Path` rather than `PurePath` because it uses `os.fsdecode()`, and so its results vary from system to system. I intend to deprecate `PurePath.as_uri()` and move it to `Path` for the same reason.

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-10-01 16:14:02 +01:00
Nikita Sobolev 06faa9a39b
gh-110160: Fix flaky `test_find_periodic_pattern` in `string_tests` (#110170) 2023-10-01 16:20:01 +02:00
dependabot[bot] 038c3564fb
PEG generator: bump types-setuptools from 68.1.0.1 to 68.2.0.0 (#110175)
build(deps-dev): bump types-setuptools in /Tools

Bumps [types-setuptools](https://github.com/python/typeshed) from 68.1.0.1 to 68.2.0.0.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-setuptools
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-01 12:38:46 +01:00
Raymond Hettinger 62405c7867
gh-110150: Fix base case handling in quantiles() (gh-110151) 2023-09-30 23:35:54 -05:00
Victor Stinner a46e960768
gh-109649: Use os.process_cpu_count() (#110165)
Replace os.cpu_count() with os.process_cpu_count() in modules:

* compileall
* concurrent.futures
* multiprocessing

Replace os.cpu_count() with os.process_cpu_count() in programs:

* _decimal deccheck.py test
* freeze.py
* multissltests.py
* python -m test (regrtest)
* wasm_build.py

Other changes:

* test.pythoninfo logs os.process_cpu_count().
* regrtest gets os.process_cpu_count() / os.cpu_count() in headers.
2023-10-01 03:14:57 +02:00
Victor Stinner 53eb9a676f
gh-110152: regrtest handles cross compilation and HOSTRUNNER (#110156)
* _add_python_opts() now handles cross compilation and HOSTRUNNER.
* display_header() now tells if Python is cross-compiled, display
  HOSTRUNNER, and get the host platform.
* Remove Tools/scripts/run_tests.py script.
* Remove "make hostrunnertest": use "make buildbottest"
  or "make test" instead.
2023-09-30 22:37:23 +00:00
Victor Stinner d3728ddc57
gh-110014: Fix bootstrap_hash.c: remove debug code (#110161)
Oops, I commited debug code by mistake, sorry about that.
2023-09-30 22:21:20 +00:00
Victor Stinner c81521020d
gh-109649: Add os.process_cpu_count() function (#109907)
* Refactor os_sched_getaffinity_impl(): move variable definitions to
  their first assignment.
* Fix test_posix.test_sched_getaffinity(): restore the old CPU mask
  when the test completes!
* Doc: Specify that os.cpu_count() counts *logicial* CPUs.
* Doc: Specify that os.sched_getaffinity(0) is related to the calling
  thread.
2023-10-01 00:12:51 +02:00
Victor Stinner 2c234196ea
gh-109276: regrtest: add WORKER_FAILED state (#110148)
Rename WORKER_ERROR to WORKER_BUG. Add WORKER_FAILED state: it does
not stop the manager, whereas WORKER_BUG does.

Change also TestResults.display_result() order: display failed tests
at the end, the important important information.

WorkerThread now tries to get the signal name for negative exit code.
2023-09-30 22:48:26 +02:00
Victor Stinner c62b49ecc8
gh-110088: Fix asyncio test_prompt_cancellation() (#110157)
Don't measure the CI performance: don't test the maximum elapsed
time. The check failed on a slow CI.
2023-09-30 22:40:10 +02:00
Victor Stinner 7513994c92
gh-110014: Include explicitly <unistd.h> header (#110155)
* Remove unused <locale.h> includes.
* Remove unused <fcntl.h> include in traceback.h.
* Remove redundant <assert.h> and <stddef.h> includes. They  are already
  included by "Python.h".
* Remove <object.h> include in faulthandler.c. Python.h already includes it.
* Add missing <stdbool.h> in pycore_pythread.h if HAVE_PTHREAD_STUBS
  is defined.
* Fix also warnings in pthread_stubs.h: don't redefine macros if they
  are already defined, like the __NEED_pthread_t macro.
2023-09-30 20:06:45 +00:00
Victor Stinner 0def8c712b
gh-109748: Fix again venv test_zippath_from_non_installed_posix() (#110149)
Call also copy_python_src_ignore() on listdir() names.

shutil.copytree(): replace set() with an empty tuple. An empty tuple
becomes a constant in the compiler and checking if an item is in an
empty tuple is cheap.
2023-09-30 20:23:26 +02:00
Victor Stinner 74e425ec18
gh-110014: Fix _POSIX_THREADS and _POSIX_SEMAPHORES usage (#110139)
* pycore_pythread.h is now the central place to make sure that
  _POSIX_THREADS and _POSIX_SEMAPHORES macros are defined if
  available.
* Make sure that pycore_pythread.h is included when _POSIX_THREADS
  and _POSIX_SEMAPHORES macros are tested.
* PY_TIMEOUT_MAX is now defined as a constant, since its value
  depends on _POSIX_THREADS, instead of being defined as a macro.
* Prevent integer overflow in the preprocessor when computing
  PY_TIMEOUT_MAX_VALUE on Windows:
  replace "0xFFFFFFFELL * 1000 < LLONG_MAX"
  with "0xFFFFFFFELL < LLONG_MAX / 1000".
* Document the change and give hints how to fix affected code.
* Add an exception for PY_TIMEOUT_MAX  name to smelly.py
* Add PY_TIMEOUT_MAX to the stable ABI
2023-09-30 19:25:54 +02:00
Victor Stinner f3bb00ea12
gh-107954: Refactor initconfig.c: add CONFIG_SPEC (#110146)
Add a specification of the PyConfig structure to factorize the code.
2023-09-30 17:24:06 +00:00