Commit Graph

122839 Commits

Author SHA1 Message Date
AraHaan 0e77540d86
Fixed regenerating files in a checkout path with spaces (GH-121384) 2024-07-05 17:33:52 +01:00
Sergey B Kirpichev d4faa7bd32
gh-121149: improve accuracy of builtin sum() for complex inputs (gh-121176) 2024-07-05 10:01:05 -05:00
Serhiy Storchaka cecd6012b0
gh-59110: Fix a debug output for implicit directories (GH-121375) 2024-07-05 11:44:07 +03:00
Josh Brobst db39bc42f9
gh-121390: tracemalloc: Fix tracebacks memory leak (#121391)
The tracemalloc_tracebacks hash table has traceback keys and NULL
values, but its destructors do not reflect this -- key_destroy_func is
NULL while value_destroy_func is raw_free. Swap these to free the
traceback keys instead.
2024-07-05 08:39:48 +02:00
Ali Tavallaie cb688bab08
gh-90437: Fix __main__.py documentation wording (GH-116309)
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Frank Dana <ferdnyc@gmail.com>
2024-07-04 15:49:14 -07:00
Victor Stinner 5f660e8e2c
gh-121084: Fix test_typing random leaks (#121360)
Clear typing ABC caches when running tests for refleaks (-R option):
call _abc_caches_clear() on typing abstract classes and their
subclasses.
2024-07-04 19:38:30 +02:00
Gabriele N. Tornetta f5c8d67de6
gh-106597: Remove unnecessary CFrame offsets (#121369) 2024-07-04 17:28:23 +00:00
Serhiy Storchaka 17d5b9df10
gh-59110: zipimport: support namespace packages when no directory entry exists (GH-121233) 2024-07-04 15:04:24 +00:00
Nice Zombies db1729143d
gh-118507: Amend news entry to mention ntpath.isfile bugfix (GH-120817) 2024-07-04 15:56:06 +01:00
Irit Katriel 67a05de17c
gh-121272: move async for/with validation from compiler to symtable (#121361) 2024-07-04 14:47:21 +01:00
Jongbum Won 715ec630dd
gh-121355: Fix incorrect word in simple_stmts.rst (#121356) 2024-07-04 06:34:34 -07:00
Cody Maloney 06a1c3fb24
gh-120754: Update estimated_size in C truncate (#121357)
Sometimes a large file is truncated (test_largefile). While
estimated_size is used as a estimate (the read will stil get the number
of bytes in the file), that it is much larger than the actual size of
data can result in a significant over allocation and sometimes lead to
a MemoryError / running out of memory.

This brings the C implementation to match the Python _pyio
implementation.
2024-07-04 12:59:18 +00:00
Irit Katriel 19d1e43e43
gh-121352: use _Py_SourceLocation in symtable (#121353) 2024-07-04 11:28:44 +01:00
Cody Maloney 2f5f19e783
gh-120754: Reduce system calls in full-file FileIO.readall() case (#120755)
This reduces the system call count of a simple program[0] that reads all
the `.rst` files in Doc by over 10% (5706 -> 4734 system calls on my
linux system, 5813 -> 4875 on my macOS)

This reduces the number of `fstat()` calls always and seek calls most
the time. Stat was always called twice, once at open (to error early on
directories), and a second time to get the size of the file to be able
to read the whole file in one read. Now the size is cached with the
first call.

The code keeps an optimization that if the user had previously read a
lot of data, the current position is subtracted from the number of bytes
to read. That is somewhat expensive so only do it on larger files,
otherwise just try and read the extra bytes and resize the PyBytes as
needeed.

I built a little test program to validate the behavior + assumptions
around relative costs and then ran it under `strace` to get a log of the
system calls. Full samples below[1].

After the changes, this is everything in one `filename.read_text()`:

```python3
openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3`
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0`
ioctl(3, TCGETS, 0x7ffdfac04b40)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343
read(3, "", 1)                          = 0
close(3)                                = 0
```

This does make some tradeoffs
1. If the file size changes between open() and readall(), this will
still get all the data but might have more read calls.
2. I experimented with avoiding the stat + cached result for small files
in general, but on my dev workstation at least that tended to reduce
performance compared to using the fstat().

[0]

```python3
from pathlib import Path

nlines = []
for filename in Path("cpython/Doc").glob("**/*.rst"):
    nlines.append(len(filename.read_text()))
```

[1]
Before small file:

```
openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0
ioctl(3, TCGETS, 0x7ffe52525930)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0
read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343
read(3, "", 1)                          = 0
close(3)                                = 0
```

After small file:

```
openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0
ioctl(3, TCGETS, 0x7ffdfac04b40)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343
read(3, "", 1)                          = 0
close(3)                                = 0
```

Before large file:

```
openat(AT_FDCWD, "cpython/Doc/c-api/typeobj.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0
ioctl(3, TCGETS, 0x7ffe52525930)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0
read(3, ".. highlight:: c\n\n.. _type-struc"..., 133105) = 133104
read(3, "", 1)                          = 0
close(3)                                = 0
```

After large file:

```
openat(AT_FDCWD, "cpython/Doc/c-api/typeobj.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0
ioctl(3, TCGETS, 0x7ffdfac04b40)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, ".. highlight:: c\n\n.. _type-struc"..., 133105) = 133104
read(3, "", 1)                          = 0
close(3)                                = 0
```

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-07-04 09:17:00 +02:00
Bénédikt Tran 9728ead361
gh-121141: add support for `copy.replace` to AST nodes (#121162) 2024-07-03 20:10:54 -07:00
Chris Markiewicz 94f50f8ee6
gh-117983: Defer import of threading for lazy module loading (#120233)
As noted in gh-117983, the import importlib.util can be triggered at
interpreter startup under some circumstances, so adding threading makes
it a potentially obligatory load.
Lazy loading is not used in the stdlib, so this removes an unnecessary
load for the majority of users and slightly increases the cost of the
first lazily loaded module.

An obligatory threading load breaks gevent, which monkeypatches the
stdlib. Although unsupported, there doesn't seem to be an offsetting
benefit to breaking their use case.

For reference, here are benchmarks for the current main branch:

```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
  Time (mean ± σ):       9.7 ms ±   0.7 ms    [User: 7.7 ms, System: 1.8 ms]
  Range (min … max):     8.4 ms …  13.1 ms    313 runs
```

And with this patch:

```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
  Time (mean ± σ):       8.4 ms ±   0.7 ms    [User: 6.8 ms, System: 1.4 ms]
  Range (min … max):     7.2 ms …  11.7 ms    352 runs
```

Compare to:

```
❯ hyperfine -w 8 './python -c pass'
Benchmark 1: ./python -c pass
  Time (mean ± σ):       7.6 ms ±   0.6 ms    [User: 5.9 ms, System: 1.6 ms]
  Range (min … max):     6.7 ms …  11.3 ms    390 runs
```

This roughly halves the import time of importlib.util.
2024-07-03 20:50:46 +00:00
Tian Gao e245ed7d1e
gh-118714: Make the pdb post-mortem restart/quit behavior more reasonable (#118725) 2024-07-03 11:30:20 -07:00
Victor Stinner f8373db153
gh-112136: Restore removed _PyArg_Parser (#121262)
Restore the private _PyArg_Parser structure and the private
_PyArg_ParseTupleAndKeywordsFast() function, previously removed
in Python 3.13 alpha 1.

Recreate Include/cpython/modsupport.h header file.
2024-07-03 18:36:57 +02:00
Max Muoto 7c66906802
gh-121300: Add `replace` to `copy.__all__` (#121302) 2024-07-03 20:33:56 +05:30
Stefano Rivera ca2e876500
gh-121201: Disable perf_trampoline on riscv64 for now (#121328)
Disable perf_trampoline on riscv64 for now

Until support is added in perf_jit_trampoline.c

gh-120089 was incomplete.
2024-07-03 15:44:34 +01:00
Diego Russo 84512c0e7f
GH-119726: Emit AArch64 trampolines out-of-line (GH-121280) 2024-07-03 06:22:21 -07:00
Vinay Sajip 26d24eeb90
gh-121035: Update PNG image for logging flow diagram. (GH-121323) 2024-07-03 12:33:28 +01:00
Sergey B Kirpichev afee76b6eb
gh-121245: a regression test for site.register_readline() (#121259) 2024-07-03 11:45:43 +01:00
Ken Jin 722229e5dc
gh-121263: Macro-ify most stackref functions for MSVC (GH-121270)
Macro-ify most stackref functions for MSVC
2024-07-03 17:49:31 +08:00
Irit Katriel 93156880ef
gh-121272: set ste_coroutine during symtable construction (#121297)
compiler no longer modifies the symtable after this.
2024-07-03 10:18:34 +01:00
Sergey B Kirpichev 51c4a324c0
gh-61103: Support float and long double complex types in ctypes module (#121248)
This amends 6988ff02a5: memory allocation for
stginfo->ffi_type_pointer.elements in PyCSimpleType_init() should be
more generic (perhaps someday fmt->pffi_type->elements will be not a
two-elements array).

It should finally resolve #61103.

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-07-03 11:08:11 +02:00
Gabriele N. Tornetta c9bdfbe868
gh-106597: Add more offsets to _Py_DebugOffsets (#121311)
Add more offsets to _Py_DebugOffsets

We add a few more offsets that are required by some out-of-process
tools, such as [Austin](https://github.com/p403n1x87/austin).
2024-07-03 08:53:44 +00:00
Amin Alaee 9d3c9b822c
Docs: Add `os.splice` flags argument (#109847)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Blaise Pabon <blaise@gmail.com>
2024-07-03 13:40:57 +05:30
da-woods 4232976b02
docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (#112237) 2024-07-03 13:35:02 +05:30
dependabot[bot] f49c83aa66
build(deps): bump hypothesis from 6.100.2 to 6.104.2 in /Tools (#121218)
Bumps [hypothesis](https://github.com/HypothesisWorks/hypothesis) from 6.100.2 to 6.104.2.
- [Release notes](https://github.com/HypothesisWorks/hypothesis/releases)
- [Commits](https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-6.100.2...hypothesis-python-6.104.2)

---
updated-dependencies:
- dependency-name: hypothesis
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-03 13:22:59 +05:30
byundojin f65d17bf47
updated tp_flags initialization to use inplace or (#120625) 2024-07-03 13:21:25 +05:30
AN Long ff5751a208
gh-111872: Document the max_children attribute for `socketserver.ForkingMixIn` (#118134) 2024-07-03 13:16:57 +05:30
AN Long 705a123898
gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (#121260) 2024-07-03 13:05:05 +05:30
Serhiy Storchaka ff5806c78e
gh-121027: Make the functools.partial object a method descriptor (GH-121089)
Co-authored-by: d.grigonis <dgrigonis@users.noreply.github.com>
2024-07-03 09:02:15 +03:00
Barney Gale f09d184821
GH-73991: Support copying directory symlinks on older Windows (#120807)
Check for `ERROR_INVALID_PARAMETER` when calling `_winapi.CopyFile2()` and
raise `UnsupportedOperation`. In `Path.copy()`, handle this exception and
fall back to the `PathBase.copy()` implementation.
2024-07-03 04:30:29 +01:00
Vinay Sajip 089835469d
gh-121035: Further improve logging flow diagram with respect to dark/light modes. (GH-121265) 2024-07-02 18:57:34 +01:00
Pablo Galindo Salgado b180788d4a
gh-115773: Add sizes to debug offset structure (#120112) 2024-07-02 17:54:33 +00:00
Sam Gross 8e8d202f55
gh-117139: Add _PyTuple_FromStackRefSteal and use it (#121244)
Avoids the extra conversion from stack refs to PyObjects.
2024-07-02 12:30:14 -04:00
Irit Katriel 1ac273224a
gh-121272: move __future__ import validation from compiler to symtable (#121273) 2024-07-02 16:22:08 +00:00
Bénédikt Tran 6343486eb6
gh-121165: protect macro expansion of `ADJUST_INDICES` with do-while(0) (#121166) 2024-07-02 16:27:51 +05:30
Bénédikt Tran 15232a0819
gh-121210: handle nodes with missing attributes/fields in `ast.compare` (#121211) 2024-07-02 16:23:17 +05:30
Sergey B Kirpichev 7a807c3efa
gh-121245: Amend d611c4c8e9 (correct import) (#121255)
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
2024-07-02 09:40:01 +00:00
Victor Stinner 7435f053b4
Move get_signal_name() to test.support (#121251)
* Move get_signal_name() from test.libregrtest to test.support.
* Use get_signal_name() in support.script_helper.
* support.script_helper now decodes stdout and stderr from UTF-8,
  instead of ASCII, if a command failed.
2024-07-02 10:34:13 +02:00
Vinay Sajip bfe0e4d769
gh-121035: Improve logging flow diagram for dark/light modes. (GH-121254) 2024-07-02 09:13:37 +01:00
Rafael Fontenelle 4f1e1dff89
Fix phrasing in paragraphs with leading "similar" (#121135) 2024-07-01 17:36:27 -07:00
Diego Russo 966260841b
GH-119726: Use LDR for AArch64 trampolines (GH-121001) 2024-07-01 15:52:33 -07:00
sobolevn 1dc9a4f6b2
gh-121196: Document `dict.fromkeys` params as pos-only (#121197) 2024-07-01 23:27:04 +03:00
Brandt Bucher 33903c53db
GH-116017: Get rid of _COLD_EXITs (GH-120960) 2024-07-01 13:17:40 -07:00
AN Long 294e724964
gh-117657: Fix data races reported by TSAN in some set methods (#120914)
Refactor the fast Unicode hash check into `_PyObject_HashFast` and use relaxed
atomic loads in the free-threaded build.

After this change, the TSAN doesn't report data races for this method.
2024-07-01 15:11:39 -04:00
Eric Snow 9bcb7d8c6f
gh-121110: Temporarily Skip test_basic_multiple_interpreters_reset_each (gh-121236)
This will allow Py_TRACE_REFS builds to pass the test suite, until the underlying issue can be resolved.
2024-07-01 17:58:25 +00:00