Commit Graph

118512 Commits

Author SHA1 Message Date
Victor Stinner 2bd960b579
gh-108337: Add pyatomic.h header (#108701)
This adds a new header that provides atomic operations on common data
types. The intention is that this will be exposed through Python.h,
although that is not the case yet. The only immediate use is in
the test file.

Co-authored-by: Sam Gross <colesbury@gmail.com>
2023-08-31 21:41:18 +00:00
Victor Stinner c6d56135e1
gh-108638: Fix tests when _stat extension is missing (#108689)
Fix test_inspect and test_pydoc when the _stat extension is missing.
Skip tests relying on _stat when _stat is missing.
2023-08-31 23:14:23 +02:00
Ethan Furman d48760b2f1
gh-108682: [Enum] raise TypeError if super().__new__ called in custom __new__ (GH-108704)
When overriding the `__new__` method of an enum, the underlying data type should be created directly; i.e. .

    member = object.__new__(cls)
    member = int.__new__(cls, value)
    member = str.__new__(cls, value)

Calling `super().__new__()` finds the lookup version of `Enum.__new__`, and will now raise an exception when detected.
2023-08-31 12:45:12 -07:00
Victor Stinner 13a00078b8
gh-108634: Py_TRACE_REFS uses a hash table (#108663)
Python built with "configure --with-trace-refs" (tracing references)
is now ABI compatible with Python release build and debug build.
Moreover, it now also supports the Limited API.

Change Py_TRACE_REFS build:

* Remove _PyObject_EXTRA_INIT macro.
* The PyObject structure no longer has two extra members (_ob_prev
  and _ob_next).
* Use a hash table (_Py_hashtable_t) to trace references (all
  objects): PyInterpreterState.object_state.refchain.
* Py_TRACE_REFS build is now ABI compatible with release build and
  debug build.
* Limited C API extensions can now be built with Py_TRACE_REFS:
  xxlimited, xxlimited_35, _testclinic_limited.
* No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2()
  functions to PyModule_Create2TraceRefs() and
  PyModule_FromDefAndSpec2TraceRefs().
* _Py_PrintReferenceAddresses() is now called before
  finalize_interp_delete() which deletes the refchain hash table.
* test_tracemalloc find_trace() now also filters by size to ignore
  the memory allocated by _PyRefchain_Trace().

Test changes for Py_TRACE_REFS:

* Add test.support.Py_TRACE_REFS constant.
* Add test_sys.test_getobjects() to test sys.getobjects() function.
* test_exceptions skips test_recursion_normalizing_with_no_memory()
  and test_memory_error_in_PyErr_PrintEx() if Python is built with
  Py_TRACE_REFS.
* test_repl skips test_no_memory().
* test_capi skisp test_set_nomemory().
2023-08-31 18:33:34 +02:00
Alex Waygood 013a99a47b
gh-97850: Note in py312 whatsnew that `importlib.util.set_loader` and `importlib.util.module_for_loader` have been removed (#108719)
Note in py312 whatsnew that `importlib.util.set_loader` and `importlib.util.module_for_loader` have been removed
2023-08-31 15:35:23 +01:00
Victor Stinner dd32611f4f
gh-106320: winconsoleio.c includes pycore_pyerrors.h (#108720)
Fix compiler warning:

    warning C4013: '_PyErr_ChainExceptions1' undefined
2023-08-31 14:13:53 +00:00
Victor Stinner bd58389cdd
Run make regen-global-objects (#108714) 2023-08-31 15:37:14 +02:00
Victor Stinner 79823c103b
gh-106320: Remove private _PyErr_ChainExceptions() (#108713)
Remove _PyErr_ChainExceptions(), _PyErr_ChainExceptions1() and
_PyErr_SetFromPyStatus() functions from the public C API.

* Move the private _PyErr_ChainExceptions() and
  _PyErr_ChainExceptions1() function to the internal C API
  (pycore_pyerrors.h).
* Move the private _PyErr_SetFromPyStatus() to the internal C API
  (pycore_initconfig.h).
* No longer export the _PyErr_ChainExceptions() function.
* Move run_in_subinterp_with_config() from _testcapi to
  _testinternalcapi.
2023-08-31 13:53:19 +02:00
Mark Shannon 059bd4d299
GH-108614: Remove non-debug uses of `#if TIER_ONE` and `#if TIER_TWO` from `_POP_FRAME` op. (GH-108685) 2023-08-31 11:34:52 +01:00
Victor Stinner 194c6fb85e
gh-106320: Don't export _Py_ForgetReference() function (#108712)
There is no need to export the _Py_ForgetReference() function of the
Py_TRACE_REFS build. It's not used by shared extensions. Enhance also
its comment.
2023-08-31 09:15:31 +00:00
Victor Stinner 9c03215a3e
gh-107149: Make PyUnstable_ExecutableKinds public (#108440)
Move PyUnstable_ExecutableKinds and associated macros from the
internal C API to the public C API.

Rename constants: replace "PY_" prefix with "PyUnstable_" prefix.
2023-08-31 09:56:06 +02:00
Alex Waygood b89b838ebc
gh-108455: peg_generator: use `types-setuptools==68.1.0.1` in CI (#108697) 2023-08-31 08:41:25 +01:00
Nikita Sobolev 991e4e76b5
gh-101100: Fix sphinx warnings in `threading.rst` (#108684)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-08-31 10:28:27 +03:00
Carl Meyer 157b89e55e
gh-108696: revert bypassing import cache in test_import helper (#108698) 2023-08-30 18:15:31 -06:00
Carl Meyer d52c4482a8
gh-108654: restore comprehension locals before handling exception (#108659)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2023-08-30 17:50:50 -06:00
Victor Stinner f59c66e8c8
gh-108297: Remove test_crashers (#108690)
The test was skipped in 2011 by
commit 89ba56d5fb.

Scripts in Lib/test/crashers/ do not crash on a reliable way. They
rely on undefined behaviors, like state of the stack memory, and so
may or may not crash. It is not worth it to make sure that they crash
in a continious integration, they should be run manually time to time
instead.
2023-08-30 21:33:04 +02:00
albanD add8d45cbe
gh-108520: Fix bad fork detection in nested multiprocessing use case (#108568)
gh-107275 introduced a regression where a SemLock would fail being passed along nested child processes, as the `is_fork_ctx` attribute would be left missing after the first deserialization.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-08-30 17:07:41 +00:00
Erlend E. Aasland 2a3926fa51
gh-108590: Revert gh-108657 (commit 400a1cebc) (#108686)
Reverted per Serhiy's request.
2023-08-30 14:53:10 +00:00
Victor Stinner 2928e5dc65
gh-108494: Argument Clinic: Document how to generate code that uses the limited C API (#108584)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-08-30 16:02:48 +02:00
Victor Stinner e012cf771b
Document Python build requirements (#108646)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-30 15:28:58 +02:00
Hugo van Kemenade c7cef54631
gh-101100: Fix Sphinx warnings in the Logging Cookbook (#108678) 2023-08-30 14:37:45 +03:00
kato8966 38ab0dba80
Fix typo in multiprocessing docs (#108666) 2023-08-30 12:05:29 +01:00
Sebastian Rittau 6c484c39be
gh-108669: unittest: Fix documentation for TestResult.collectedDurations (#108670) 2023-08-30 09:36:59 +00:00
Corvin 400a1cebc7
gh-108590: Fix sqlite3.iterdump for invalid Unicode in TEXT columns (#108657)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-30 09:06:21 +00:00
Serhiy Storchaka 210a5d7b8b
Revert "gh-103224: Use the realpath of the Python executable in `test_venv` (GH-103243)" (GH-108667)
This reverts commit 85b0b0cd94.

It broke builtbots.
2023-08-30 11:41:04 +03:00
Victor Stinner 24b9bdd6ea
gh-106320: Remove private _Py_ForgetReference() (#108664)
Move the private _Py_ForgetReference() function to the internal C API
(pycore_object.h).
2023-08-30 03:34:43 +00:00
sterliakov 14ec0bb7c3
Mention Ellipsis pickling in the docs (#103660) 2023-08-30 01:11:31 +00:00
Serhiy Storchaka d08d49dd09
Revert "Use non alternate name for Kyiv (GH-108533)" (GH-108649)
This reverts commit 7659128b9d.

It broke tests on the Debian and macOS buildbots.
2023-08-29 23:08:45 +03:00
Erlend E. Aasland 0b0c1d046c
gh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (#108632)
Deprecate passing the callback callable by keyword for the following
sqlite3.Connection APIs:

- set_authorizer(authorizer_callback)
- set_progress_handler(progress_handler, ...)
- set_trace_callback(trace_callback)

The affected parameters will become positional-only in Python 3.15.
2023-08-29 22:02:12 +02:00
Alex Waygood 77e8f233ac
gh-108455: peg_generator: install two stubs packages before running mypy (#108637) 2023-08-29 20:14:08 +01:00
Erlend E. Aasland 8178a88bd8
gh-107801: Improve the accuracy of io.IOBase.seek docs (#108268)
- Add param docstrings
- Link to os.SEEK_* constants
- Mention the return value in the initial paragraph

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-08-29 19:13:38 +00:00
Guido van Rossum 59e46932c8
gh-108488: Initialize JUMP_BACKWARD cache to 0, not 17 (#108591)
This mis-initialization caused the executor optimization to kick in sooner than intended. It also set the lower 4 bits of the counter to `1` -- those bits are supposed to be reserved (the actual counter is in the upper 12 bits).
2023-08-29 18:14:56 +00:00
Guido van Rossum 4f22152713
gh-107557: Remove unnecessary SAVE_IP instructions (#108583)
Also remove NOP instructions.

The "stubs" are not optimized in this fashion (their SAVE_IP should always be preserved since it's where to jump next, and they don't contain NOPs by their nature).
2023-08-29 16:51:51 +00:00
Victor Stinner 83e191ba76
test_sys: remove debug print() (#108642) 2023-08-29 16:04:07 +00:00
Victor Stinner b62a76043e
gh-108638: Fix stat.filemode() when _stat is missing (#108639)
Change the pure Python implementation of stat.filemode() for unknown
file type: use "?", as done by the _stat.filemode().

test_stat skips TestFilemodeCStat if the _stat extension is missing.
2023-08-29 15:46:46 +00:00
Jochen Sprickerhof 7659128b9d
Use non alternate name for Kyiv (GH-108533)
tzdata provides Kiev as an alternative to Kyiv:

https://sources.debian.org/src/tzdata/2023c-10/backward/?hl=314#L314

But Debian moved it to the tzdata-legacy package breaking the test:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1050530

This patch switches to the name provided by tzdata.
2023-08-29 18:00:43 +03:00
Serhiy Storchaka 9205dfeca5
gh-108635: Make parameters of some implementations of special methods positional-only (GH-108636) 2023-08-29 17:55:56 +03:00
Nikita Sobolev f8be2e262c
gh-108455: peg generator: Use `strict_optional=True` for `grammar_parser` (#108629) 2023-08-29 13:02:52 +01:00
Nikita Sobolev 30305d6d01
gh-108623: Fix compile warning in `Modules/_multiprocessing/semaphore.c` (#108624) 2023-08-29 13:43:11 +02:00
Victor Stinner bf08131e0a
gh-108494: Don't build _testclinic_limited with TraceRefs (#108608)
If Python is built with ./configure --with-trace-refs, don't build
the _testclinic_limited extension. The limited C API (Py_LIMITED_API)
is not compatible with Py_TRACE_REFS.
2023-08-29 13:39:42 +02:00
Victor Stinner e675e515ae
gh-108494: Argument Clinic: fix option group for Limited C API (#108574)
Use PyTuple_Size() instead of PyTuple_GET_SIZE().
2023-08-29 13:33:28 +02:00
Alex Waygood 0d140b8c5e
gh-108455: peg_generator: make the mypy config slightly stricter (#108627)
* Enable `--no-implicit-reexport`

* Enable the `truthy-bool` error code

* Enable the `ignore-without-code` error code

* Explicitly note that `--warn-unreachable` cannot yet be enabled
2023-08-29 11:23:22 +00:00
Alex Waygood c879b9fddf
gh-108455: Simplify the peg_generator mypy config (#108620)
make it easier to see exactly which options from mypy's `--strict` mode can't currently be enabled
2023-08-29 10:24:06 +00:00
Serhiy Storchaka ecb2bf02a4
gh-108617: Extend interactive session tests for sqlite3 (GH-108556) 2023-08-29 13:20:32 +03:00
Erlend E. Aasland c8847841cc
gh-108550: Fix sqlite3 CLI regression from gh-108551 (#108618) 2023-08-29 09:39:42 +00:00
Edward Schauman-Haigh 6eaddc10e9
gh-108558: Improve sqlite3 row factory tests (#108578)
Add test_sqlite_row_keys() to explicitly test sqlite3.Row.keys().

Cleanups:
- Reduce test noise by converting docstrings to regular comments
- Reduce boilerplate code by adding a setUp() method to RowFactoryTests

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-08-29 08:51:36 +00:00
Brett Cannon 5f85b443f7
GH-106176, GH-104702: Fix reference leak when importing across multiple threads (#108497) 2023-08-29 09:17:25 +02:00
Victor Stinner c780698e9b
gh-106320: Fix test_peg_generator: _Py_UniversalNewlineFgetsWithSize() (#108609)
Fix test_peg_generator by exporting the
_Py_UniversalNewlineFgetsWithSize() function.
2023-08-29 03:40:13 +00:00
Victor Stinner b6de2850f2
gh-106320: Remove private _PyObject_GetState() (#108606)
Move the private _PyObject_GetState() function to the internal C API
(pycore_object.h).
2023-08-29 03:38:51 +00:00
Victor Stinner fadc2dc7df
gh-106320: Remove private _PyOS_IsMainThread() function (#108605)
Move the following private API to the internal C API
(pycore_signal.h): _PyOS_IsMainThread() and _PyOS_SigintEvent().
2023-08-29 03:20:31 +00:00