Commit Graph

121943 Commits

Author SHA1 Message Date
Sam Gross 608192ee2f
gh-118527: Use deferred reference counting for C functions on modules (#118529)
This addresses a scaling bottleneck in the free-threaded build when
calling functions like `math.floor()` concurrently from multiple
threads.
2024-05-03 11:33:05 -04:00
Brett Simmers c2627d6eea
gh-116322: Add Py_mod_gil module slot (#116882)
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.

PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.

A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
2024-05-03 11:30:55 -04:00
mpage 3e818afb9b
gh-118495: Skip test using threads after forking when running with TSAN (#118530)
This is unsupported. Note that `skip_unless_reliable_fork()` checks for
the conditions used by the decorators that were removed, along with checking
for TSAN.
2024-05-03 11:14:26 -04:00
Sam Gross 2dae505e87
gh-117514: Add `sys._is_gil_enabled()` function (#118514)
The function returns `True` or `False` depending on whether the GIL is
currently enabled. In the default build, it always returns `True`
because the GIL is always enabled.
2024-05-03 11:09:57 -04:00
Sam Gross 24e643d4ef
gh-118527: Use `_Py_ID(__main__)` for main module name (#118528)
Most module names are interned and immortalized, but the main
module was not. This partially addresses a scaling bottleneck in the
free-threaded when creating closure concurrently in the main module.
2024-05-03 11:05:30 -04:00
Carl Meyer c8deb1e4b4
gh-118513: Fix sibling comprehensions with a name bound in one and global in the other (#118526)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-05-03 14:05:19 +00:00
Alexander Kanavin 37ccf16786
gh-101732: Modules/_ssl.c: use Y2038 compatible openssl function when available (GH-118425) 2024-05-03 15:34:05 +02:00
Jelle Zijlstra ca269e58c2
gh-116126: Implement PEP 696 (#116129)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 06:17:32 -07:00
Nice Zombies 852263e108
gh-117492: Clarify documentation of `typing.Never` (#117678)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-05-03 06:02:11 -07:00
Eric Snow f201628073
gh-117953: Other Cleanups in the Extensions Machinery (gh-118206)
This change will make some later changes simpler.
2024-05-03 00:51:43 +00:00
Brett Simmers f8290df63f
gh-116738: Make `_codecs` module thread-safe (#117530)
The module itself is a thin wrapper around calls to functions in
`Python/codecs.c`, so that's where the meaningful changes happened:

- Move codecs-related state that lives on `PyInterpreterState` to a
  struct declared in `pycore_codecs.h`.

- In free-threaded builds, add a mutex to `codecs_state` to synchronize
  operations on `search_path`. Because `search_path_mutex` is used as a
  normal mutex and not a critical section, we must be extremely careful
  with operations called while holding it.

- The codec registry is explicitly initialized as part of
  `_PyUnicode_InitEncodings` to simplify thread-safety.
2024-05-02 18:25:36 -04:00
Tian Gao 4e2caf2aa0
gh-118500: Add pdb support for zipapp (#118501) 2024-05-02 21:53:27 +01:00
Dino Viehland e54b0c8a4a
gh-118519: Fix empty weakref list check (#118520)
Fix empty list check
2024-05-02 13:03:29 -07:00
Dino Viehland 1e67b9207c
gh-117657: Fix TSAN list set failure (#118260)
* Fix TSAN list set failure

* Relaxed atomic is sufficient, add targetted test

* More list

* Remove atomic assign in list

* Fixup white space
2024-05-02 13:03:05 -07:00
Steve Dower 8ed5466795
gh-118486: Switch mkdir(mode=0o700) on Windows to use OWNER RIGHTS instead of CURRENT_USER (GH-118515) 2024-05-02 19:43:54 +01:00
Sam Gross 16acecd779
Remove stray `__cplusplus` guard in sysmodule.c (#118511) 2024-05-02 13:51:28 -04:00
Sam Gross 83c51da6ce
gh-118413: Fix test_release_task_refs on free-threaded build (#118494)
The `time.sleep()` call should happen before the GC to give the worker
threads time to clean-up their remaining references to objs.
Additionally, use `support.gc_collect()` instead of `gc.collect()`
just in case the extra GC calls matter.
2024-05-02 13:41:15 -04:00
infohash b28a3339e4
gh-90848: Fixed create_autospec ignoring configure_mock style kwargs (#118163) 2024-05-02 18:36:35 +01:00
Pablo Galindo Salgado 6bcbee09df
gh-93502: Add new C-API functions to trace object creation and destruction (#115945) 2024-05-02 19:30:00 +02:00
Raphael Gaschignard 2770d5caca
gh-105879: Add support for keyword arguments to eval and exec (#105885)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-05-02 08:55:29 -07:00
Mark Shannon 72867c962c
GH-118095: Unify the behavior of tier 2 FOR_ITER branch micro-ops (GH-118420)
* Target _FOR_ITER_TIER_TWO at POP_TOP following the matching END_FOR

* Modify _GUARD_NOT_EXHAUSTED_RANGE, _GUARD_NOT_EXHAUSTED_LIST and _GUARD_NOT_EXHAUSTED_TUPLE so that they also target the POP_TOP following the matching END_FOR
2024-05-02 16:17:59 +01:00
Sergey B Kirpichev 9789440de3
gh-82062: Fix support of parameter defaults on methods in extension modules (GH-115270)
Now inspect.signature() supports references to the module globals in
parameter defaults on methods in extension modules.  Previously it was
only supported in functions.  The workaround was to specify the fully
qualified name, including the module name.
2024-05-02 17:44:33 +03:00
Steve Dower 81939dad77
gh-118486: Support mkdir(mode=0o700) on Windows (GH-118488) 2024-05-02 15:20:43 +01:00
Serhiy Storchaka b3372481b6
gh-117903: Clarify that the staticmethod descriptor is callable (GH-117925) 2024-05-02 16:56:33 +03:00
NGRsoftlab 7d2ffada0a
gh-116180: Check the globals argument in PyRun_* C API (GH-116637)
It used to crash when passing NULL or non-dict as globals.
Now it sets a SystemError.
2024-05-02 16:43:03 +03:00
Crowthebird 7c97dc8c95
gh-118216: Don't consider dotted `__future__` imports (#118267) 2024-05-02 06:32:20 -07:00
Mark Shannon 67bba9dd0f
GH-117442: Check eval-breaker at start (rather than end) of tier 2 loops (GH-118482) 2024-05-02 13:10:31 +01:00
Andrej f8e088df2a
gdb/libpython.py: Update PyLongObjectPtr docstring (GH-118438) 2024-05-02 08:57:45 +00:00
Andrew Zipperer a6b610a94b
docs: typo: tiny grammar change: "pointed by" -> "pointed to by" (#118411)
* docs: tiny grammar change: "pointed by" -> "pointed to by"

This commit uses "file pointed to by" to replace "file pointed by" in
 - doc for shutil.copytree
 - docstring for shutil.copytree
 - docstring _abc.PathBase.open
 - docstring for pathlib.Path.open
 - doc for os.copy_file_range
 - doc for os.splice

The docs use "file pointed to by" more frequently than
"file pointed by". So, this commit replaces the uses of
"file pointed by" in order to make the uses consistent
through the docs.

```bash
$ grep -ri 'pointed to by' cpython/
```
yields more results than
```bash
$ grep -ri 'pointed by' cpython/
```

Separately:

There are two occurrences of "tree pointed by":
 - cpython/Doc/library/xml.etree.elementtree.rst for
     `xml.etree.ElementInclude.include`
 - cpython/Lib/xml/etree/ElementInclude.py for `include`

For those uses of "tree pointed by", I expect "tree pointed to by"
instead. However, I found enough uses online of (a) "tree pointed by"
rather than (b) "tree pointed to by" to convince me that (a) is in
common use.

So, this commit does not replace those occurrences of "tree pointed by"
to "tree pointed to by". But I will replace them if a reviewer
believes it is correct to replace them.

* docs: typo: "exists and executable" -> "exists and is executable"

---------

Co-authored-by: Andrew-Zipperer <atzipperer@gmail.com>
2024-05-02 05:37:12 +00:00
Guido van Rossum a37b093228
gh-118335: Rename --experimental-interpreter on Windows to --experimental-jit-interpreter (#118497)
Also fix docs for this in whatsnew.
2024-05-02 00:48:34 +00:00
Eric Snow 526ca4c09e
gh-117953: Work Relative to Specific Extension Kinds in the Import Machinery (gh-118205)
This change will make some later changes simpler.
2024-05-01 17:40:28 -06:00
Guido van Rossum a524152b8c
gh-118335: Make REGEN_JIT_COMMAND empty if tier2 interpreter enabled (#118493)
Also patch up news blurb for gh-118339
(add warning that PYTHON_UOPS is now PYTHON_JIT).
2024-05-01 16:36:29 -07:00
Filipe Laíns 97feb4a78b
GH-118174: specify the type for the path argument of shutil.which 2024-05-02 00:13:32 +01:00
Sam Gross c408c36e9b
gh-118413: Temporarily skip `test_release_task_refs` in free-threaded builds (#118491) 2024-05-01 17:58:22 -04:00
Nice Zombies a7711a2a4e
gh-117607: Speedup os.path.relpath() (GH-117608) 2024-05-01 22:44:55 +01:00
Brandt Bucher 424438b11e
GH-113464: Display a warning when building the JIT (GH-118481) 2024-05-01 21:35:49 +00:00
Mark Shannon 39981fd07a
GH-118095: Make sure that progress is made if there are pending calls being handled. (GH-118484) 2024-05-01 22:18:31 +01:00
mpage 8a50544a99
gh-118433: Temporarily skip `test_interrupt_main_subthread` in free-threaded builds (#118485)
Free-threaded builds can intermittently tickle a longstanding bug (24 years!)
in the implementation of `threading.Condition`, leading to flakiness in the
test suite. Fixing the underlying issue will require more discussion, and will
likely apply to most of the concurrency primitives in the `threading` module
that are written in Python. See gh-118433 for more details.
2024-05-01 20:59:12 +00:00
Irit Katriel 6763bfcc0f
gh-118272: set stacktop to 0 before freeing contents, to avoid access to invalid objects during GC (#118478) 2024-05-01 21:51:40 +01:00
dependabot[bot] 1161ab9085
build(deps-dev): bump types-psutil from 5.9.5.20240316 to 5.9.5.20240423 in /Tools (#118464)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-01 19:15:30 +00:00
Hugo van Kemenade 3b3f8dea57
gh-117225: Move colorize functionality to own internal module (#118283) 2024-05-01 12:27:06 -06:00
Hugo van Kemenade 164e2c31c0
gh-117225: Document colour use in `doctest` (#118268) 2024-05-01 12:25:11 -06:00
Victor Stinner b52c753e0f
gh-110850: Add PyTime_TimeRaw() function (#118394)
Add "Raw" variant of PyTime functions:

* PyTime_MonotonicRaw()
* PyTime_PerfCounterRaw()
* PyTime_TimeRaw()

Changes:

* Add documentation and tests. Tests release the GIL while calling
  raw clock functions.
* py_get_system_clock() and py_get_monotonic_clock() now check that
  the GIL is hold by the caller if raise_exc is non-zero.
* Reimplement "Unchecked" functions with raw clock functions.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-05-01 18:05:01 +00:00
Jason R. Coombs a8bcf3ec32
Expand the 'Extending' docs with an example. (#113187)
* Expand the 'Extending' docs to provide a minimal example. Closes python/importlib_metadata#427.

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-05-01 17:04:02 +00:00
Malcolm Smith 75955110a6
gh-116622: Android sysconfig updates (#118352) 2024-05-01 16:47:54 +00:00
dependabot[bot] 6d12f4469c
Bump types-setuptools from 69.2.0.20240317 to 69.5.0.20240423 in /Tools (#118463)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-01 17:07:29 +01:00
dependabot[bot] 2d161cb899
Bump mypy from 1.9.0 to 1.10.0 in /Tools (#118461)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-01 17:06:54 +01:00
Harmen Stoppels 759e8e7ab8
gh-99730: urllib.request: Keep HEAD method on redirect (GH-99731) 2024-05-01 18:01:47 +02:00
Brandt Bucher 49baa656cb
GH-115802: Use the GHC calling convention in JIT code (GH-118287) 2024-05-01 08:05:53 -07:00
Anthony Shaw beb653cc24
gh-117958: Expose JIT code via method in UOpExecutor (#117959) 2024-05-01 07:11:14 -07:00