The object's inline values may be marked invalid if the materialized
dict was already initialized and then deleted.
(cherry picked from commit 162b41f577)
Co-authored-by: Sam Gross <colesbury@gmail.com>
When using the ** operator or pow() with Fraction as the base
and an exponent that is not rational, a float, or a complex, the
fraction is no longer converted to a float.
(cherry picked from commit b9965ef282)
Co-authored-by: Joshua Herman <30265+zitterbewegung@users.noreply.github.com>
console.compile with the "single" param throws an exception when
there are multiple statements, never allowing to adding newlines
to a pasted code block (gh-121610)
This adds a few extra checks to allow extending when in an indented
block, and tests for a few examples.
(cherry picked from commit 7d111dac16)
Co-authored-by: saucoide <32314353+saucoide@users.noreply.github.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
The fallback repl does not support "exit" without parentheses, so the
test would hang until the timeout expired.
(cherry picked from commit 4134261ab8)
Co-authored-by: Sam Gross <colesbury@gmail.com>
This avoids messages like:
ThreadSanitizer: starting new threads after multi-threaded fork is not
supported. Dying (set die_after_fork=0 to override)
(cherry picked from commit 82a4dac9f6)
Co-authored-by: Sam Gross <colesbury@gmail.com>
See 6b98b274b6 for an explanation of the problem and solution. Here I've applied the solution to channels.
(cherry picked from commit 8b209fd4f8, AKA gh-121805)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Any cross-interpreter mechanism for passing objects between interpreters must be very careful to respect isolation, even when the object is effectively immutable (e.g. int, str). Here this especially relates to when an interpreter sends one of its objects, and then is destroyed while the inter-interpreter machinery (e.g. queue) still holds a reference to the object.
When I added interpreters.Queue, I dealt with that case (using an atexit hook) by silently removing all items from the queue that were added by the finalizing interpreter.
Later, while working on concurrent.futures.InterpreterPoolExecutor (gh-116430), I noticed it was somewhat surprising when items were silently removed from the queue when the originating interpreter was destroyed. (See my comment on that PR.)
It took me a little while to realize what was going on. I expect that users, which much less context than I have, would experience the same pain.
My approach, here, to improving the situation is to give users three options:
1. return a singleton (interpreters.queues.UNBOUND) from Queue.get() in place of each removed item
2. raise an exception (interpreters.queues.ItemInterpreterDestroyed) from Queue.get() in place of each removed item
3. existing behavior: silently remove each item (i.e. Queue.get() skips each one)
The default will now be (1), but users can still explicitly opt in any of them, including to the silent removal behavior.
The behavior for each item may be set with the corresponding Queue.put() call. and a queue-wide default may be set when the queue is created. (This is the same as I did for "synconly".)
(cherry picked from commit 6b98b274b6, AKA gh-116431)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
It is our general practice to make new optional parameters keyword-only,
even if the existing parameters are all positional-or-keyword. Passing
this parameter as positional would look confusing and could be error-prone
if additional parameters are added in the future.
(cherry picked from commit 50eec501fe)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
We also need to close the `slave_fd` earlier so that reading from
`master_fd` won't block forever when the subprocess finishes.
(cherry picked from commit abc3aeebdb)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-121497: Make Pyrepl respect correctly the history with input hook set (GH-121498)
(cherry picked from commit 4e36dd7d87)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
gh-121499: Fix multi-line history rendering in the REPL (GH-121531)
(cherry picked from commit 4b9e10d0ea)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
On POSIX systems, excluding macOS framework installs, the lib directory
for the free-threaded build now includes a "t" suffix to avoid conflicts
with a co-located default build installation.
(cherry picked from commit e8c91d90ba)
Co-authored-by: Sam Gross <colesbury@gmail.com>
When builtin static types are initialized for a subinterpreter, various "tp" slots have already been inherited (for the main interpreter). This was interfering with the logic in add_operators() (in Objects/typeobject.c), causing a wrapper to get created when it shouldn't. This change fixes that by preserving the original data from the static type struct and checking that.
(cherry picked from commit 5250a03133, AKA gh-121602)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
This fixes a mistake in gh-113012 and adds a test that verifies the fix.
(cherry picked from commit 35a67e36aa, AKA gh-121597)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
On heavily loaded machines, the subprocess may finish its sleep before
the parent process manages to synchronize with it via a failed lock.
This leads to errors like:
Exception: failed to sync child in 300.3 sec
Use pipes instead to mutually synchronize between parent and child.
(cherry picked from commit af9f6de6ea)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-121571: Do not use `EnvironmentError` in tests, use `OSError` instead (GH-121572)
(cherry picked from commit e2822360da)
Co-authored-by: sobolevn <mail@sobolevn.me>
gh-117657: Skip test when running under TSan (GH-121549)
The ProcessPoolForkserver combined with resource_tracker starts a thread
after forking, which is not supported by TSan.
Also skip test_multiprocessing_fork for the same reason
(cherry picked from commit 04397434aa)
Co-authored-by: Sam Gross <colesbury@gmail.com>
The change in gh-118157 (b2cd54a) should have also updated clear_singlephase_extension() but didn't. We fix that here. Note that clear_singlephase_extension() (AKA _PyImport_ClearExtension()) is only used in tests.
(cherry picked from commit 15d48aea02, AKA gh-121503)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-121351: Skip test_not_wiping_history_file() if no readline (GH-121422)
(cherry picked from commit 68e279b37a)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
gh-121359: Run test_pyrepl in isolated mode (GH-121414)
run_repl() now pass the -I option (isolated mode) to Python if the
'env' parameter is not set.
(cherry picked from commit 6239d41527)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121084: Fix test_typing random leaks (GH-121360)
Clear typing ABC caches when running tests for refleaks (-R option):
call _abc_caches_clear() on typing abstract classes and their
subclasses.
(cherry picked from commit 5f660e8e2c)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-117983: Defer import of threading for lazy module loading (GH-120233)
As noted in gh-117983, the import importlib.util can be triggered at
interpreter startup under some circumstances, so adding threading makes
it a potentially obligatory load.
Lazy loading is not used in the stdlib, so this removes an unnecessary
load for the majority of users and slightly increases the cost of the
first lazily loaded module.
An obligatory threading load breaks gevent, which monkeypatches the
stdlib. Although unsupported, there doesn't seem to be an offsetting
benefit to breaking their use case.
For reference, here are benchmarks for the current main branch:
```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
Time (mean ± σ): 9.7 ms ± 0.7 ms [User: 7.7 ms, System: 1.8 ms]
Range (min … max): 8.4 ms … 13.1 ms 313 runs
```
And with this patch:
```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
Time (mean ± σ): 8.4 ms ± 0.7 ms [User: 6.8 ms, System: 1.4 ms]
Range (min … max): 7.2 ms … 11.7 ms 352 runs
```
Compare to:
```
❯ hyperfine -w 8 './python -c pass'
Benchmark 1: ./python -c pass
Time (mean ± σ): 7.6 ms ± 0.6 ms [User: 5.9 ms, System: 1.6 ms]
Range (min … max): 6.7 ms … 11.3 ms 390 runs
```
This roughly halves the import time of importlib.util.
(cherry picked from commit 94f50f8ee6)
Co-authored-by: Chris Markiewicz <effigies@gmail.com>
gh-118714: Make the pdb post-mortem restart/quit behavior more reasonable (GH-118725)
(cherry picked from commit e245ed7d1e)
Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
* [3.13] gh-120782: Update internal type cache when reloading datetime
When reloading _datetime module, the single-phase version did not invoke the PyInit__datetime function, whereas the current multi-phase version updates the static types through the module init. The outdated static type cache in the interpreter state needs to be invalidated at the end of reloading the multi-phase module.
This will allow Py_TRACE_REFS builds to pass the test suite, until the underlying issue can be resolved.
(cherry picked from commit 9bcb7d8c6f, AKA gh-121236)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-121200: Fix test_expanduser_pwd2() of test_posixpath (GH-121228)
Call getpwnam() to get pw_dir, since it can be different than
getpwall() pw_dir.
(cherry picked from commit 02cb5fdee3)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121200: Log pwd entry in test_expanduser_pwd2() (GH-121207)
Use subTest() to log the pwd entry in test_expanduser_pwd2() of
test_posixpath to help debugging.
(cherry picked from commit 05a6f8da60)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121084: Call _abc_registry_clear() when checking refleaks (GH-121191)
dash_R_cleanup() now calls _abc_registry_clear() before calling again
register().
(cherry picked from commit c766ad206e)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121188: Sanitize invalid XML characters in regrtest (GH-121195)
When creating the JUnit XML file, regrtest now escapes characters
which are invalid in XML, such as the chr(27) control character used
in ANSI escape sequences.
(cherry picked from commit af8c3d7a26)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-87744: fix waitpid race while calling send_signal in asyncio (GH-121126)
asyncio earlier relied on subprocess module to send signals to the process, this has some drawbacks one being that subprocess module unnecessarily calls waitpid on child processes and hence it races with asyncio implementation which internally uses child watchers. To mitigate this, now asyncio sends signals directly to the process without going through the subprocess on non windows systems. On Windows it fallbacks to subprocess module handling but on windows there are no child watchers so this issue doesn't exists altogether.
(cherry picked from commit bd473aa598)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
* parse_intermixed_args() now raises ArgumentError instead of calling
error() if exit_on_error is false.
* Internal code now always raises ArgumentError instead of calling
error(). It is then caught at the higher level and error() is called if
exit_on_error is true.
(cherry picked from commit 81a654a342)
The check for whether the log file is a real file is expensive on NFS
filesystems. This commit reorders the rollover condition checking to
not do the file type check if the expected file size is less than the
rotation threshold.
(cherry picked from commit e9b4ec614b)
Co-authored-by: Craig Robson <craig@zhatt.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
gh-119521: Remove _IncompleteInputError from the docs (GH-120993)
(cherry picked from commit 1167a9a30b)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-121008: Fix idlelib.run tests (GH-121046)
When testing IDLE, don't create a Tk to avoid side effects such as
installing a PyOS_InputHook hook.
(cherry picked from commit 44eafd6688)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-120380: fix Python implementation of `pickle.Pickler` for `bytes` and `bytearray` objects in protocol version 5. (GH-120422)
(cherry picked from commit 7595e6743a)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Add an InternalDocs file describing how interning should work and how to use it.
* Add internal functions to *explicitly* request what kind of interning is done:
- `_PyUnicode_InternMortal`
- `_PyUnicode_InternImmortal`
- `_PyUnicode_InternStatic`
* Switch uses of `PyUnicode_InternInPlace` to those.
* Disallow using `_Py_SetImmortal` on strings directly.
You should use `_PyUnicode_InternImmortal` instead:
- Strings should be interned before immortalization, otherwise you're possibly
interning a immortalizing copy.
- `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
`SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
backports, as they are now part of public API and version-specific ABI.
* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.
* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
- `_Py_ID`
- `_Py_STR` (including the empty string)
- one-character latin-1 singletons
Now, when you intern a singleton, that exact singleton will be interned.
* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).
* Intern `_Py_STR` singletons at startup.
* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.
* Beef up the tests. Cover internal details (marked with `@cpython_only`).
* Add lots of assertions
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
- gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680)
(cherry picked from commit ce1064e4c9)
- gh-119521: Use `PyAPI_DATA`, not `extern`, for `_PyExc_IncompleteInputError` (GH-120955)
(cherry picked from commit ac61d58db0)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-120373: Mark test_audit.test_http as requiring the network resource (GH-120374)
(cherry picked from commit b0e1c51882)
Co-authored-by: Itamar Oren <itamarost@gmail.com>
Now the null character is always represented as \xc0\x80 for
Tcl_NewStringObj().
(cherry picked from commit c38e2f64d0)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
The integer part of the timestamp can be rounded up, while the millisecond
calculation truncates, causing the log timestamp to be wrong by up to 999 ms
(affected roughly 1 in 8 million timestamps).
(cherry picked from commit 1500a23f33)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Accessing the Tkinter object's string representation no longer converts
the underlying Tcl object to a string on Windows.
(cherry picked from commit f4ddaa3967)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Also refactor the "state" option tests for other ttk widgets.
(cherry picked from commit 974a978631)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
The tests are now passed with the current version of Tcl/Tk under
development (8.7b1+ and 9.0b3+).
The following changes were also made to make the tests more flexible:
* Helper methods like checkParam() now interpret the expected error message
as a regular expression instead of a literal.
* Add support of new arguments in checkEnumParam():
- allow_empty=True skips testing with empty string;
- fullname= specifies the name for error message if it differs from the
option name;
- sort=True sorts values for error message.
* Add support of the allow_empty argument in checkReliefParam():
allow_empty=True adds an empty string to the list of accepted values.
* Attributes _clip_highlightthickness, _clip_pad and _clip_borderwidth
specify how negative values of options -highlightthickness, -padx, -pady
and -borderwidth are handled.
* Use global variables for some common error messages.
(cherry picked from commit 6ad26de6e8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
gh-120732: Fix `name` passing to `Mock`, when using kwargs to `create_autospec` (GH-120737)
(cherry picked from commit 1e4815692f)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
gh-120635: Avoid leaking processes in test_pyrepl (GH-120676)
If the child process takes longer than SHORT_TIMEOUT seconds to
complete, kill the process but then wait until it completes with no
timeout to not leak child processes.
(cherry picked from commit 0f3e36454d)
Co-authored-by: Victor Stinner <vstinner@python.org>
The `inspect.ismethoddescriptor()` function did not check for the lack of
`__delete__()` and, consequently, erroneously returned True when applied
to *data* descriptors with only `__get__()` and `__delete__()` defined.
(cherry picked from commit dacc5ac71a)
Co-authored-by: Jan Kaliszewski <zuo@kaliszewski.net>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
gh-120633: Move scrollbar and remove tear-off menus in turtledemo (GH-120634)
(cherry picked from commit 89f7208f67)
Co-authored-by: Wulian233 <71213467+Wulian233@users.noreply.github.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
This exposes `PyUnstable_Object_ClearWeakRefsNoCallbacks` as an unstable
C-API function to provide a thread-safe mechanism for clearing weakrefs
without executing callbacks.
Some C-API extensions need to clear weakrefs without calling callbacks,
such as after running finalizers like we do in subtype_dealloc.
Previously they could use `_PyWeakref_ClearRef` on each weakref, but
that's not thread-safe in the free-threaded build.
(cherry picked from commit e8752d7b80)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-120659: Skip `test_freethreading` with GIL (GH-120660)
(cherry picked from commit 360f14a493)
Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>
gh-119897: Add test for lambda generator invocation (GH-120658)
(cherry picked from commit 73dc1c678e)
gh-120467: Add test for lambda generator invocation
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Mocking only works if sys.modules['pydoc'] and pydoc are the same,
but some pydoc functions reload the module and change sys.modules.
Ensure that sys.modules['pydoc'] is always restored after the corresponding
tests.
(cherry picked from commit 2cf47389e2)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-120182 added new global state (interp_count), but didn't add thread-safety for it. This change eliminates the possible race.
(cherry picked from commit 2c66318cdc, AKA gh-120529)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Fix a race in `PyMember_GetOne` and `PyMember_SetOne` for `Py_T_OBJECT_EX`.
These functions implement `__slots__` accesses for Python objects.
(cherry picked from commit 362cd2680b)
Co-authored-by: Daniele Parmeggiani <8658291+dpdani@users.noreply.github.com>
gh-115649: Copy the filename into main interpreter before intern in import.c (GH-120315)
(cherry picked from commit 28140d1f2d)
Co-authored-by: AN Long <aisk@users.noreply.github.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
gh-120567: Clarify weekday return in calendar.monthrange docstring (GH-120570)
(cherry picked from commit bd4516d9ef)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
gh-112346: Always set OS byte to 255, simpler gzip.compress function. (GH-120486)
This matches the output behavior in 3.10 and earlier; the optimization in 3.11 allowed the zlib library's "os" value to be filled in instead in the circumstance when mtime was 0. this keeps things consistent.
(cherry picked from commit 08d09cf5ba)
Co-authored-by: Ruben Vorderman <r.h.p.vorderman@lumc.nl>
When help() is called with non-string argument, use __qualname__ or
__name__ if available, otherwise use "{typename} object".
(cherry picked from commit 31d1d72d7e)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-120544: Add `else: fail()` to tests where exception is expected (GH-120545)
(cherry picked from commit 42ebdd83bb)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
gh-119819: Conditional skip of logging tests that require multiprocessing subprocess support (GH-120476)
Skip tests that require multiprocessing subprocess support.
(cherry picked from commit 92f6d400f7)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
In gh-120009 I used an atexit hook to finalize the _datetime module's static types at interpreter shutdown. However, atexit hooks are executed very early in finalization, which is a problem in the few cases where a subclass of one of those static types is still alive until the final GC collection. The static builtin types don't have this probably because they are finalized toward the end, after the final GC collection. To avoid the problem for _datetime, I have applied a similar approach here.
Also, credit goes to @mgorny and @neonene for the new tests.
FYI, I would have liked to take a slightly cleaner approach with managed static types, but wanted to get a smaller fix in first for the sake of backporting. I'll circle back to the cleaner approach with a future change on the main branch.
(cherry picked from commit b2e71ff4f8, AKA gh-120182)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-120361: Add `nonmember` test with enum flags inside to `test_enum` (GH-120364)
* gh-120361: Add `nonmember` test with enum flags inside to `test_enum`
(cherry picked from commit 7fadfd82eb)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
gh-114053: Fix bad interaction of PEP 695, PEP 563 and `inspect.get_annotations` (GH-120270)
(cherry picked from commit 42351c3b9a)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Check if the DateTime C-API type matches the datetime.date type on main and shared/isolated subinterpreters.
(cherry picked from commit 50a389565a, AKA gh-119604)
Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
gh-120418: Don't assume wheeldata is deleted if `WHEEL_PKG_DIR` is set (GH-120419)
Remove wheeldata from both sides of the `assertEqual`, so that we're
*actually* ignoring it from the test set.
This test is only making assertions about the source tree, no code is
being executed that would do anything different based on the value of
`WHEEL_PKG_DIR`.
(cherry picked from commit 030b452e34)
Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
The _strptime module object was cached in a static local variable (in the datetime.strptime() implementation). That's a problem when it crosses isolation boundaries, such as reinitializing the runtme or between interpreters. This change fixes the problem by dropping the static variable, instead always relying on the normal sys.modules cache (via PyImport_Import()).
(cherry picked from commit 127c1d2771, AKA gh-120224)
Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe (GH-120165)
(cherry picked from commit e16aed63f6)
Co-authored-by: Ken Jin <kenjin@python.org>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
gh-93691: fix too broad source locations of for statement iterators (GH-120330)
(cherry picked from commit 97b69db167)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
gh-120128: fix description of argument to ipaddress.collapse_addresses() (GH-120131)
The argument to collapse_addresses() is now described as an *iterable*
(rather than *iterator*).
(cherry picked from commit f878d46e56)
Co-authored-by: Jan Kaliszewski <zuo@kaliszewski.net>
gh-119517: Fixes for pasting in pyrepl (GH-120253)
* Remove pyrepl's optimization for self-insert
This will be replaced by a less specialized optimization.
* Use line-buffering when pyrepl echoes pastes
Previously echoing was totally suppressed until the entire command had
been pasted and the terminal ended paste mode, but this gives the user
no feedback to indicate that an operation is in progress. Drawing
something to the screen once per line strikes a balance between
perceived responsiveness and performance.
* Remove dead code from pyrepl
`msg_at_bottom` is always true.
* Speed up pyrepl's screen rendering computation
The Reader in pyrepl doesn't hold a complete representation of the
screen area being drawn as persistent state. Instead, it recomputes it,
on each keypress. This is fast enough for a few hundred bytes, but
incredibly slow as the input buffer grows into the kilobytes (likely
because of pasting).
Rather than making some expensive and expansive changes to the repl's
internal representation of the screen, add some caching: remember some
data from one refresh to the next about what was drawn to the screen
and, if we don't find anything that has invalidated the results that
were computed last time around, reuse them. To keep this caching as
simple as possible, all we'll do is look for lines in the buffer that
were above the cursor the last time we were asked to update the screen,
and that are still above the cursor now. We assume that nothing can
affect a line that comes before both the old and new cursor location
without us being informed. Based on this assumption, we can reuse old
lines, which drastically speeds up the overwhelmingly common case where
the user is typing near the end of the buffer.
* Speed up pyrepl prompt drawing
Cache the `can_colorize()` call rather than repeatedly recomputing it.
This call looks up an environment variable, and is called once per
character typed at the REPL. The environment variable lookup shows up as
a hot spot when profiling, and we don't expect this to change while the
REPL is running.
* Speed up pasting multiple lines into the REPL
Previously, we were checking whether the command should be accepted each
time a line break was encountered, but that's not the expected behavior.
In bracketed paste mode, we expect everything pasted to be part of
a single block of code, and encountering a newline shouldn't behave like
a user pressing <Enter> to execute a command. The user should always
have a chance to review the pasted command before running it.
* Use a read buffer for input in pyrepl
Previously we were reading one byte at a time, which causes much slower
IO than necessary. Instead, read in chunks, processing previously read
data before asking for more.
* Optimize finding width of a single character
`wlen` finds the width of a multi-character string by adding up the
width of each character, and then subtracting the width of any escape
sequences. It's often called for single character strings, however,
which can't possibly contain escape sequences. Optimize for that case.
* Optimize disp_str for ASCII characters
Since every ASCII character is known to display as single width, we can
avoid not only the Unicode data lookup in `disp_str` but also the one
hidden in `str_width` for them.
* Speed up cursor movements in long pyrepl commands
When the current pyrepl command buffer contains many lines, scrolling up
becomes slow. We have optimizations in place to reuse lines above the
cursor position from one refresh to the next, but don't currently try to
reuse lines below the cursor position in the same way, so we wind up
with quadratic behavior where all lines of the buffer below the cursor
are recomputed each time the cursor moves up another line.
Optimize this by only computing one screen's worth of lines beyond the
cursor position. Any lines beyond that can't possibly be shown by the
console, and bounding this makes scrolling up have linear time
complexity instead.
---------
(cherry picked from commit 32a0faba43)
Signed-off-by: Matt Wozniski <mwozniski@bloomberg.net>
Co-authored-by: Matt Wozniski <mwozniski@bloomberg.net>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
This makes the pure Python implementation consistent with the C implementation.
(cherry picked from commit 34f5ae69fe)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
gh-65454: avoid triggering call to a PropertyMock in NonCallableMock.__setattr__ (GH-120019)
(cherry picked from commit 9e9ee50421)
Co-authored-by: blhsing <blhsing@gmail.com>
gh-119600: mock: do not access attributes of original when new_callable is set (GH-119601)
In order to patch flask.g e.g. as in GH-84982, that
proxies getattr must not be invoked. For that,
mock must not try to read from the original
object. In some cases that is unavoidable, e.g.
when doing autospec. However, patch("flask.g",
new_callable=MagicMock) should be entirely safe.
(cherry picked from commit 422c4fc855)
Co-authored-by: Robert Collins <robert.collins@cognite.com>
If the Helper() class was initialized with an output, the topics, keywords
and symbols help still use the pager instead of the output.
Change the behavior so the output is used if available while keeping the
previous behavior if no output was configured.
(cherry picked from commit 2080425154)
Co-authored-by: Enrico Tröger <enrico.troeger@uvena.de>
gh-120121: Add InvalidStateError to concurrent.futures.__all__ (GH-120123)
(cherry picked from commit 5d59b870ef)
Co-authored-by: AN Long <aisk@users.noreply.github.com>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
gh-120225: fix crash in compiler on empty block at end of exception handler (GH-120235)
(cherry picked from commit 4fc82b6d3b)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
gh-94808: Add test coverage for "starred kind" in _PyPegen_set_expr_context (GH-119222)
Add test coverage for "starred kind" in _PyPegen_set_expr_context
(cherry picked from commit 8231a24454)
Co-authored-by: Mark Jason Dominus (陶敏修) <mjd@pobox.com>
* Use new methods for tracing Tcl variable.
* Fix Combobox.current() for empty combobox.
(cherry picked from commit d68a22e7a6)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-119577: Adjust DeprecationWarning when testing element truth values in ElementTree (GH-119762)
Adjust DeprecationWarning when testing element truth values in ElementTree, we're planning to go with the more natural True return rather than a disruptive harder to code around exception raise, and are deferring the behavior change for a few more releases.
(cherry picked from commit 6b606522ca)
Co-authored-by: Jacob Walls <jacobtylerwalls@gmail.com>
The modified context caused tests failures in several other tests.
(cherry picked from commit 2d7ff6e0e7)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
This matches the default GIL switch interval. It greatly speeds up the
free-threaded build: previously, it spent nearly all its time in
`gc.collect()`.
(cherry picked from commit 4bba1c9e6c)
Co-authored-by: Sam Gross <colesbury@gmail.com>
The `test_imaplib` was taking 40+ minutes in the refleak build bots because
the tests waiting on a client `self._setup()` was creating a client that
prevented progress until its connection timed out, which scaled with the
global timeout.
We should set `connect=False` for the tests that don't want `_setup()` to
create a client.
(cherry picked from commit 710cbea660)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit d9095194dd)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
The process is expected to time out. In the refleak builds,
`support.SHORT_TIMEOUT` is often five minutes and we run the tests six
times, so test_signal was taking >30 minutes.
(cherry picked from commit d419d468ff)
Co-authored-by: Sam Gross <colesbury@gmail.com>
* gh-120041: Do not use append_to_screen when completions are visible (GH-120042)
(cherry picked from commit 8fc7653766)
* gh-120041: Refactor check for visible completion menu in completing_reader (GH-120055)
(cherry picked from commit bf8e5e53d0)
---------
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Fix os.major(), os.minor() and os.makedev().
Support device numbers larger than 2**63-1.
Support non-existent device number (NODEV).
(cherry picked from commit 7111d9605f)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-106531: Apply changes from importlib_resources 6.3.2 (GH-117054)
Apply changes from importlib_resources 6.3.2.
(cherry picked from commit 8d63c8d47b)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
gh-119070: Update test_shebang_executable_extension to always use non-installed version (GH-119846)
(cherry picked from commit 5c48eb0cc6)
Co-authored-by: Steve Dower <steve.dower@python.org>
The free-threaded build currently immortalizes objects that use deferred
reference counting (see gh-117783). This typically happens once the
first non-main thread is created, but the behavior can be suppressed for
tests, in subinterpreters, or during a compile() call.
This fixes a race condition involving the tracking of whether the
behavior is suppressed.
(cherry picked from commit 47fb4327b5)
gh-119856: Support exiting help() with just "exit" (GH-119858)
(cherry picked from commit 4223f1d828)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-111201: Skip pyrepl Windows tests earlier (#119848)
Don't attempt to load pyrepl Windows console if platforms others than
Windows. For example, the import can fail if ctypes is missing.
(cherry picked from commit 91601a5596)
Implement `shutil._rmtree_safe_fd()` using a list as a stack to avoid emitting recursion errors on deeply nested trees.
`shutil._rmtree_unsafe()` was fixed in a150679f90.
(cherry picked from commit 53b1981fb0)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
dSupport non-dict globals in LOAD_FROM_DICT_OR_GLOBALS
The implementation basically copies LOAD_GLOBAL. Possibly it could be deduplicated,
but that seems like it may get hairy since the two operations have different operands.
This is important to fix in 3.14 for PEP 649, but it's a bug in earlier versions too,
and we should backport to 3.13 and 3.12 if possible.
(cherry picked from commit 80a4e38994)
* Add docs for new APIs
* Add soft-deprecation notices
* Add What's New porting entries
* Update comments referencing `PyFrame_LocalsToFast()` to mention the proxy instead
* Other related cleanups found when looking for refs to the deprecated APIs
(cherry picked from commit 3859e09e3d)
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Make sure that `gilstate_counter` is not zero in when calling
`PyThreadState_Clear()`. A destructor called from `PyThreadState_Clear()` may
call back into `PyGILState_Ensure()` and `PyGILState_Release()`. If
`gilstate_counter` is zero, it will try to create a new thread state before
the current active thread state is destroyed, leading to an assertion failure
or crash.
(cherry picked from commit bcc1be39cb)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-119548: Add a 'clear' command to the REPL (GH-119549)
(cherry picked from commit e3bac04c37)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 9732ed5ca9)
Co-authored-by: James De Bias <81095953+DBJim@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
PhotoImage.get() retruns a string instead of a 3-tuple of integers
in this case.
(cherry picked from commit e875c2d752)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
GH-89727: Fix FD leak on `os.fwalk()` generator finalization. (GH-119766)
Follow-up to 3c890b50. Ensure we `os.close()` open file descriptors when
the `os.fwalk()` generator is finalized.
(cherry picked from commit a5fef800d3)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
GH-89727: Fix `os.fwalk()` recursion error on deep trees (GH-119638)
Implement `os.fwalk()` using a list as a stack to avoid emitting recursion
errors on deeply nested trees.
(cherry picked from commit 3c890b503c)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
gh-119260: Clarify is_dataclass Behavior for Subclasses in Documentation and Tests (GH-119480)
(cherry picked from commit bf4ff3ad2e)
Co-authored-by: Aditya Borikar <adityaborikar2@gmail.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
This is an unrevert of d58ebf0 (gh-119636), which was reverted by 9216a53 (gh-119639) due to problems which have been resolved.
This is minimal support for multiphase init. Subinterpreters are not supported yet. That will be addressed in a later change.
(cherry picked from commit 3e8b609)
Co-authored-by: Erlend E. Aasland erlend@python.org
GH-119169: Implement `pathlib.Path.walk()` using `os.walk()` (GH-119573)
For silly reasons, pathlib's generic implementation of `walk()` currently
resides in `glob._Globber`. This commit moves it into
`pathlib._abc.PathBase.walk()` where it really belongs, and makes
`pathlib.Path.walk()` call `os.walk()`.
(cherry picked from commit 7ff61f51b6)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
GH-89727: Partially fix `shutil.rmtree()` recursion error on deep trees (GH-119634)
Make `shutil._rmtree_unsafe()` call `os.walk()`, which is implemented
without recursion.
`shutil._rmtree_safe_fd()` is not affected and can still raise a recursion
error.
(cherry picked from commit a150679f90)
Co-authored-by: Barney Gale <barney.gale@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-119443: Turn off from __future__ import annotations in REPL (GH-119493)
(cherry picked from commit a8e35e8eba)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-119581: Add a test of InitVar with name shadowing (GH-119582)
(cherry picked from commit 6ec371223d)
Co-authored-by: Steven Troxler <steven.troxler@gmail.com>
Miss Islington (bot)
Pablo Galindo Salgado
Kirill Podoprigora
Ken Jin
Jelle Zijlstra
neonene
Serhiy Storchaka
Victor Stinner
Miro Hrončok
Thomas Wouters
devdanzin
Petr Viktorin
Tian Gao
Mark Shannon
Eric Snow
Xie Yanbo
Łukasz Langa
Sam Gross