Prevents a segmentation fault in registered hooks for the readline library, but only when the readline module is loaded inside an isolated sub interpreter. The module is single-phase init so loading it fails, but not until the module init function has already run, where the readline hooks get registered.
The readlinestate_global macro was error-prone to PyImport_FindModule returning NULL and crashing in about 18 places. I could reproduce 1 easily, but this PR replaces the macro with a function and adds error conditions to the other functions.
* bpo-32731: Raise OSError from any failure in getpass.getuser()
Previously, if the username was not set in certain environment variables, ImportError escaped on Windows systems, and it was possible for KeyError to escape on other systems if getpwuid() failed.
Add support for TLS-PSK (pre-shared key) to the ssl module.
---------
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Revert commit c8c0afc713 (PR #94532),
which moved `struct.Struct` initialisation from `Struct.__init__` to `Struct.__new__`.
This caused issues with code in the wild that subclasses `struct.Struct`.
Construct only one new list object (using `list.copy()`) when creating a
new path object with a modified tail. This slightly speeds up
`with_name()` and `with_suffix()`
Add upstream Tk patches for three problems affecting tkinter users:
- Update macOS installer to include a fix accepted by upstream Tcl/Tk
for a crash encountered after the first :meth:`tkinter.Tk` instance
is destroyed. (gh-92603)
- Update macOS installer to include an upstream Tcl/Tk fix
for the ``ttk::ThemeChanged`` error encountered in Tkinter. (gh-71383)
- Update macOS installer to include an upstream Tcl/Tk fix for the
``Secure coding is not enabled for restorable state!`` warning
encountered in Tkinter on macOS 14 Sonoma. (gh-110950)
Co-authored-by: Ned Deily <nad@python.org>
This makes the Tier 2 interpreter a little faster.
I calculated by about 3%,
though I hesitate to claim an exact number.
This starts by doubling the trace size limit (to 512),
making it more likely that loops fit in a trace.
The rest of the approach is to only load
`oparg` and `operand` in cases that use them.
The code generator know when these are used.
For `oparg`, it will conditionally emit
```
oparg = CURRENT_OPARG();
```
at the top of the case block.
(The `oparg` variable may be referenced multiple times
by the instructions code block, so it must be in a variable.)
For `operand`, it will use `CURRENT_OPERAND()` directly
instead of referencing the `operand` variable,
which no longer exists.
(There is only one place where this will be used.)
This uses the new mechanism whereby certain uops
are replaced by others during translation,
using the `_PyUop_Replacements` table.
We further special-case the `_FOR_ITER_TIER_TWO` uop
to update the deoptimization target to point
just past the corresponding `END_FOR` opcode.
Two tiny code cleanups are also part of this PR.
Omit the `@interface_scope` from an IPv6 address when used as Host header by `http.client`.
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
Add docstrings to the debugger module. Fix two bugs: initialize Idb.botframe (should be in Bdb); In Idb.in_rpc_code, check whether prev_frame is None before trying to use it. Make other code changes.
Expand test_debugger coverage from 19% to 66%.
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* Run again test_ast_recursion_limit() on WASI platform.
* Add _testinternalcapi.get_c_recursion_remaining().
* Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and
test_trace_unpack_long_sequence() now adjust the maximum recursion
depth depending on the the remaining C recursion.
The "Check if generated files are up to date" job of GitHub Actions
now runs the "autoreconf -ivf -Werror" command instead of the "make
regen-configure" command to avoid depending on the external quay.io
server.
Add Tools/build/regen-configure.sh script to regenerate the configure
with an Ubuntu container image. The
"quay.io/tiran/cpython_autoconf:271" container image
(https://github.com/tiran/cpython_autoconf) is no longer used.
This avoids:
python3.13 Tools/unicode/makeunicodedata.py
python3.13: can't open file '.../build/debug/Tools/unicode/makeunicodedata.py': [Errno 2] No such file or directory
make: *** [Makefile:1498: regen-unicodedata] Error 2
Re-run `make regen-unicodedata` to update the script path in generated files.
In non-debug more the check for the "errors" argument is skipped,
and then PyUnicode_AsUTF8() can fail, but its result was not checked.
Co-authored-by: Victor Stinner <vstinner@python.org>
* Fix crash when encoding is not string or None.
* Fix crash when both line_buffering and write_through raise exception
when converted ti int.
* Add a number of tests for constructor and reconfigure() method
with invalid arguments.
_PyDict_Pop_KnownHash(): remove the default value and the return type
becomes an int.
Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
The `@critical_section` directive instructs Argument Clinic to generate calls
to `Py_BEGIN_CRITICAL_SECTION()` and `Py_END_CRITICAL_SECTION()` around the
bound function. In `--disable-gil` builds, these calls will lock and unlock
the `self` object. They are no-ops in the default build.
This is used in one place (`_io._Buffered.close`) as a demonstration.
Subsequent PRs will use it more widely in the `_io.Buffered` bindings.
Add `glob.translate()` function that converts a pathname with shell wildcards to a regular expression. The regular expression is used by pathlib to implement `match()` and `glob()`.
This function differs from `fnmatch.translate()` in that wildcards do not match path separators by default, and that a `*` pattern segment matches precisely one path segment. When *recursive* is set to true, `**` pattern segments match any number of path segments, and `**` cannot appear outside its own segment.
In pathlib, this change speeds up directory walking (because `_make_child_relpath()` does less work), makes path objects smaller (they don't need a `_lines` slot), and removes the need for some gnarly code.
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* Split list_extend() into two sub-functions: list_extend_fast() and
list_extend_iter().
* list_inplace_concat() no longer has to call Py_DECREF() on the
list_extend() result, since list_extend() now returns an int.
New methods to access mailbox.Maildir message info and flags:
get_info, set_info, get_flags, set_flags, add_flag, remove_flag.
These methods speed up accessing a message's info and/or flags and are
useful when it is not necessary to access the message's contents,
as when iterating over a Maildir to find messages with specific flags.
---------
* Add more str type checking
* modernize to f-strings instead of %
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Make `multiprocessing.managers.{DictProxy,ListProxy}` generic for type annotation use. `ListProxy[str]` for example.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Now all results from worker processes are aggregated and
displayed together as a summary at the end of a regrtest run.
The traditional trace is left in place for use with sequential
in-process test runs but now raises a warning that those
numbers are not precise.
`-T -j` requires `--with-pydebug` as it relies on `-Xpresite=`.
Drop posix.fallocate() under WASI.
The underlying POSIX function, posix_fallocate(), was found to vary too
much between implementations to remain in WASI. As such, while it was
available in WASI preview1, it's been dropped in preview2.
Critical sections are helpers to replace the global interpreter lock
with finer grained locking. They provide similar guarantees to the GIL
and avoid the deadlock risk that plain locking involves. Critical
sections are implicitly ended whenever the GIL would be released. They
are resumed when the GIL would be acquired. Nested critical sections
behave as if the sections were interleaved.
* Remove mention of "vsapi" element type from the documentation.
* Add tests for element_create() and other ttk.Style methods.
* Add examples for element_create() in the documentation.
* Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)"
This reverts commit d9b606b3d0.
* Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)"
This reverts commit cde1071b2a.
* Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)"
This reverts commit d731579bfb.
* Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)"
This reverts commit d8f32be5b6.
* Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)"
This reverts commit 37e4e20eaa.
Joining a thread now ensures the underlying OS thread has exited. This is required for safer fork() in multi-threaded processes.
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Use Argument Clinic for time.clock_gettime() and
time.clock_gettime_ns() functions.
Benchmark on time.clock_gettime_ns():
import time
import pyperf
runner = pyperf.Runner()
runner.timeit(
'clock_gettime_ns(CLOCK_MONOTONIC_COARSE)',
setup='import time; clock_gettime_ns=time.clock_gettime_ns; CLOCK_MONOTONIC_COARSE=6',
stmt='clock_gettime_ns(CLOCK_MONOTONIC_COARSE)')
Result on Linux with CPU isolation:
Mean +- std dev: [ref] 134 ns +- 1 ns -> [change] 55.7 ns +- 1.4 ns: 2.41x faster
- There is no longer a separate Python/executor.c file.
- Conventions in Python/bytecodes.c are slightly different -- don't use `goto error`,
you must use `GOTO_ERROR(error)` (same for others like `unused_local_error`).
- The `TIER_ONE` and `TIER_TWO` symbols are only valid in the generated (.c.h) files.
- In Lib/test/support/__init__.py, `Py_C_RECURSION_LIMIT` is imported from `_testcapi`.
- On Windows, in debug mode, stack allocation grows from 8MiB to 12MiB.
- **Beware!** This changes the env vars to enable uops and their debugging
to `PYTHON_UOPS` and `PYTHON_LLTRACE`.
* Add mimalloc v2.12
Modified src/alloc.c to remove include of alloc-override.c and not
compile new handler.
Did not include the following files:
- include/mimalloc-new-delete.h
- include/mimalloc-override.h
- src/alloc-override-osx.c
- src/alloc-override.c
- src/static.c
- src/region.c
mimalloc is thread safe and shares a single heap across all runtimes,
therefore finalization and getting global allocated blocks across all
runtimes is different.
* mimalloc: minimal changes for use in Python:
- remove debug spam for freeing large allocations
- use same bytes (0xDD) for freed allocations in CPython and mimalloc
This is important for the test_capi debug memory tests
* Don't export mimalloc symbol in libpython.
* Enable mimalloc as Python allocator option.
* Add mimalloc MIT license.
* Log mimalloc in Lib/test/pythoninfo.py.
* Document new mimalloc support.
* Use macro defs for exports as done in:
https://github.com/python/cpython/pull/31164/
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
* Try to fix asyncio.Server.wait_closed() again
I identified the condition that `wait_closed()` is intended
to wait for: the server is closed *and* there are no more
active connections.
When this condition first becomes true, `_wakeup()` is called
(either from `close()` or from `_detach()`) and it sets `_waiters`
to `None`. So we just check for `self._waiters is None`; if it's
not `None`, we know we have to wait, and do so.
A problem was that the new test introduced in 3.12 explicitly
tested that `wait_closed()` returns immediately when the server
is *not* closed but there are currently no active connections.
This was a mistake (probably a misunderstanding of the intended
semantics). I've fixed the test, and added a separate test that
checks exactly for this scenario.
I also fixed an oddity where in `_wakeup()` the result of the
waiter was set to the waiter itself. This result is not used
anywhere and I changed this to `None`, to avoid a GC cycle.
* Update Lib/asyncio/base_events.py
---------
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
It now always returns an integer if one or less counting options are specified.
Previously it could return a single count as a 1-tuple, an integer (only if
option "update" was specified) or None if no items found.
The result is now the same if wantobjects is set to 0.
Fixes#109894
* set `interp.static_objects.last_resort_memory_error.args` to empty tuple to avoid crash on `PyErr_Display()` call
* allow `_PyExc_InitGlobalObjects()` to be called on subinterpreter init
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Before, the '-s STRIPDIR' option on
compileall lead to some surprising results as it only strips away
path components that match, but leaves alone the non-matching ones
interspersed in between.
For example, with: python -m compileall -s/path/to/another/src
/path/to/build/src/file.py
The resulting written path will be: build/file.py
This fix only strips directories that are a fully matching prefix of the
source path. If a stripdir is provided that is not a valid prefix, a
warning will be displayed (which can be silenced with '-qq').
asyncio.TaskGroup and asyncio.Timeout classes now raise proper RuntimeError
if they are improperly used.
* When they are used without entering the context manager.
* When they are used after finishing.
* When the context manager is entered more than once (simultaneously or
sequentially).
* If there is no current task when entering the context manager.
They now remain in a consistent state after an exception is thrown,
so subsequent operations can be performed correctly (if they are allowed).
Co-authored-by: James Hilton-Balfe <gobot1234yt@gmail.com>
Test case matching patterns specified by options --match, --ignore,
--matchfile and --ignorefile are now tested in the order of
specification, and the last match determines whether the test case be run
or ignored.
Add PyUnicode_AsUTF8() function to the limited C API.
multiprocessing posixshmem now uses PyUnicode_AsUTF8() instead of
PyUnicode_AsUTF8AndSize(): the extension is built with the limited C
API. The function now raises an exception if the filename contains an
embedded null character instead of truncating silently the filename.
* PyUnicode_AsUTF8() now raises an exception if the string contains
embedded null characters.
* Update related C API tests (test_capi.test_unicode).
* type_new_set_doc() uses PyUnicode_AsUTF8AndSize() to silently
truncate doc containing null bytes.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Build the _multiprocessing.posixshmem extension with the Limited C
API.
* Add <errno.h> include.
* Replace PyUnicode_AsUTF8() with PyUnicode_AsUTF8AndSize().
* Replace PyStructSequence_SET_ITEM() with
PyStructSequence_SetItem().
* Replace PyTuple_GET_SIZE() with PyTuple_Size().
* Replace PyTuple_GET_ITEM() with PyTuple_GetItem().
Fix detection of gdb built without Python scripting support.
* check_usable_gdb() doesn't check gdb exit code when calling
run_gdb().
* Use shutil.which() to get the path to the gdb program.
The PySys_Audit() function was added in Python 3.8 by the PEP 578
"Python Runtime Audit Hooks".
Add also PySys_AuditTuple() to the limited C API, function added
to Python 3.13.
Move non-limited "PerfMap" C API from Include/sysmodule.h to
Include/cpython/sysmodule.h.
Move the following private functions and structures to
pycore_modsupport.h internal C API:
* _PyArg_BadArgument()
* _PyArg_CheckPositional()
* _PyArg_NoKeywords()
* _PyArg_NoPositional()
* _PyArg_ParseStack()
* _PyArg_ParseStackAndKeywords()
* _PyArg_Parser structure
* _PyArg_UnpackKeywords()
* _PyArg_UnpackKeywordsWithVararg()
* _PyArg_UnpackStack()
* _Py_ANY_VARARGS()
Changes:
* Python/getargs.h now includes pycore_modsupport.h to export
functions.
* clinic.py now adds pycore_modsupport.h when one of these functions
is used.
* Add pycore_modsupport.h includes when a C extension uses one of
these functions.
* Define Py_BUILD_CORE_MODULE in C extensions which now include
directly or indirectly (via code generated by Argument Clinic)
pycore_modsupport.h:
* _csv
* _curses_panel
* _dbm
* _gdbm
* _multiprocessing.posixshmem
* _sqlite.row
* _statistics
* grp
* resource
* syslog
* _testcapi: bad_get() no longer uses METH_FASTCALL calling
convention but METH_VARARGS. Replace _PyArg_UnpackStack() with
PyArg_ParseTuple().
* _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined
by _testcapi sub-modules which need the internal C API
(pycore_modsupport.h): exceptions.c, float.c, vectorcall.c,
watchers.c.
* Remove Include/cpython/modsupport.h header file.
Include/modsupport.h no longer includes the removed header file.
* Fix mypy clinic.py
Add PyMem_RawMalloc(), PyMem_RawCalloc(), PyMem_RawRealloc() and
PyMem_RawFree() to the limited C API.
These functions were added by Python 3.4 and are needed to port
stdlib extensions to the limited C API, like grp and pwd.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
zipinfo now supports the full range of values in the TZ string
determined by RFC 8536 and detects all invalid formats.
Both Python and C implementations now raise exceptions of the same
type on invalid data.
Now re.error is raised instead of OverflowError or RuntimeError for
too large width of look-behind pattern.
The limit is increased to 2**32-1 (was 2**31-1).
* tty.setraw() and tty.setcbreak() previously returned partially modified
list of the original tty attributes. Now they return the correct list of
the original tty attributes
* tty.cfmakeraw() and tty.cfmakecbreak() now make a copy of the list of
special characters before modifying it.
It already mostly worked, except in the case when invalid keyword
argument with non-ASCII name was passed to function with non-ASCII
parameter names. Then it crashed in the debug mode.
Pablo Galindo Salgado
Irit Katriel
Rémi Lapeyre
Matt Prodani
Brett Cannon
Jelle Zijlstra
Tian Gao
Anthony Shaw
apaz
Serhiy Storchaka
kale-smoothie
Jacob Walls
Zackery Spytz
Alex Waygood
Grant Ramsay
Mark Dickinson
Barney Gale
Randolf Scholz
Thomas Wouters
Junya Okabe
Christopher Chavez
AN Long
Guido van Rossum
Hugo van Kemenade
Crowthebird
Michael
Nikita Sobolev
Sam Gross
Victor Stinner
Tomas R
Miro Hrončok
DPR
adder32
Davide Rizzo
Markus Mohrhard
Stephen Gildea
Łukasz Langa
Zac Hatfield-Dodds
Nicolas Tessore
Samuel Thibault
Mark Shannon
Donghee Na
Pierre Ossman (ThinLinc team)
Stefan
Masayuki Moriyama
Charles Machalow
CF Bolz-Tereick
Antoine Pitrou
Philippe Cerfon
Kumar Aditya
Yilei Yang
Thomas Grainger
Dino Viehland
c-bata
Sergey B Kirpichev
Raymond Hettinger
gsallam
zcxsythenew
Jokimax
Brandt Bucher
Savannah Ostrowski
Radislav Chugunov
hetmankp
Furkan Onder
Ayappan Perumal
YAMAMOTO Takashi
Tamás Hegedűs
Terry Jan Reedy
Adam Turner
Joshua Root
Shantanu
Russell Keith-Magee