This fixes a crash when running the PyO3 test suite on the free-threaded
build. The `qsbr` field is initialized after the `PyThreadState` is
added to the interpreter's linked list -- it might still be NULL.
Instead, we "steal" the queue of to-be-freed memory blocks. This is
always initialized (possibly empty) and protected by the stop the world
pause.
We previously used `AC_RUN_IF_ELSE` with a short test program to detect
if `-latomic` is needed, but that requires choosing a specific default
value when cross-compiling because the test program is not run.
Some cross compilation targets like `wasm32-emscripten` do not support
`-latomic`, while other cross compilation targets, like
`arm-linux-gnueabi` require it.
It is an alternate constructor which only accepts a single numeric argument.
Unlike to Decimal.from_float() it accepts also Decimal.
Unlike to the standard constructor, it does not accept strings and tuples.
It is an alternative constructor which only accepts a single numeric argument.
Unlike to Fraction.from_float() and Fraction.from_decimal() it accepts any
real numbers supported by the standard constructor (int, float, Decimal,
Rational numbers, objects with as_integer_ratio()).
Unlike to the standard constructor, it does not accept strings.
Lock `ZoneInfoType` to protect accesses to `ZONEINFO_STRONG_CACHE`.
Refactor the `tp_new` handler to use Argument Clinic so that we can just
use `@critical_section` annotations on the relevant functions.
Also use `PyDict_SetDefaultRef` instead of `PyDict_SetDefault` when
inserting into the `TIMEDELTA_CACHE`.
`PurePath.__init__()` incorrectly uses the `_raw_paths` of a given
`PurePath` object with a different flavour, even though the procedure to
join path segments can differ between flavours.
This change makes the `_raw_paths`-enabled deferred joining apply _only_
when the path flavours match.
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
In some locales (like French or Hebrew) the full or abbreviated names of
the default month and weekday used in __calc_date_time can be part of
other name or constant part of the %c format. The month name can also
match %m with constant suffix (like in Japanese). So the code failed to
correctly distinguish formats %a, %A, %b, %B and %m.
Cycle all month and all days of the week to find the variable part
and distinguish %a from %A and %b from %B or %m.
Fixed locales for the following languges:
Arabic, Bislama, Breton, Bodo, Kashubian, Chuvash, Estonian, French, Irish,
Ge'ez, Gurajati, Manx Gaelic, Hebrew, Hindi, Chhattisgarhi, Haitian Kreyol,
Japanese, Kannada, Korean, Marathi, Malay, Norwegian, Nynorsk, Punjabi,
Rajasthani, Tok Pisin, Yoruba, Yue Chinese, Yau/Nungon and Chinese.
Co-authored-by: Eli Bendersky <eliben@gmail.com>
This follows GNU gzip, which defaults to using 0 as the mtime
for compressing stdin, where no file mtime is involved.
This makes the output of gzip.compress() deterministic by default,
greatly helping reproducible builds.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Raise ValueError in add_argument() if either explicit nargs=0 or action
that does not consume arguments (like 'store_const' or 'store_true') is
specified for positional argument.
Previously, all nested mutually exclusive groups lost their connection
to the group containing them and were displayed as belonging directly
to the parser.
Co-authored-by: Danica J. Sutherland <djsutherland@users.noreply.github.com>
memcopy'ing arbitrary values to _Bool variable triggers undefined
behaviour. Avoid this.
We assume that `false` is represented by all zero bytes.
Credits to Alex Gaynor.
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
* Add definitions for "context", "current context", and "context
management protocol".
* Update related definitions to be consistent with the new
definitions.
* Restructure the documentation for the `contextvars.Context` class
to prepare for adding context manager support, and for consistency
with the definitions.
* Use `testcode` and `testoutput` to test the `Context.run` example.
* Expand the documentation for the `Py_CONTEXT_EVENT_ENTER` and
`Py_CONTEXT_EVENT_EXIT` events to clarify and to prepare for
planned changes.
As argparse now detects by default when the code was run as a module.
This leads to using the actual executable name instead of simply "python"
to display in the usage message ("usage: python -m ...").
Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.
This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
failures identified by gh-124785.
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Barry Warsaw <barry@python.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
This allows testing Y2038 with system time set to after that,
so that actual Y2038 issues can be exposed, and not masked
by expired certificate errors.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
If the PYTHON_BASIC_REPL environment variable is set, the site module
no longer imports the _pyrepl module.
Moreover, the site module now respects -E and -I command line
options: ignore PYTHON_BASIC_REPL in this case.
- move the Py_Main documentation from the very high level API section
to the initialization and finalization section
- make it clear that it encapsulates a full Py_Initialize/Finalize
cycle of its own
- point out that exactly which settings will be read and applied
correctly when Py_Main is called after a separate runtime
initialization call is version dependent
- be explicit that Py_IsInitialized can be called prior to
initialization
- actually test that Py_IsInitialized can be called prior to
initialization
- flush stdout in the embedding tests that run code so it appears
in the expected order when running with "-vv"
- make "-vv" on the subinterpreter embedding tests less spammy
---------
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
The function now sets temporarily the LC_CTYPE locale to the locale
of the category that determines the requested value if the locales are
different and the resulting string is non-ASCII.
This temporary change affects other threads.
The term "free variable" has unfortunately become genuinely
ambiguous over the years (presumably due to the names of
some relevant code object instance attributes).
While we can't eliminate that ambiguity at this late date, we can
at least alert people to the potential ambiguity by describing
both the formal meaning of the term and the common
alternative use as a direct synonym for "closure variable".
---------
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
Increase currently inadequate vertical spacing for the IDLE browsers (path,
module, and stack) on high-resolution monitors.
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
This allows direct intersphinx references to APIs via references
like `` :func:`importlib.metadata.version` ``.
---------
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* gh-117151: increase default buffer size of shutil.copyfileobj() to 256k.
it was set to 16k in the 1990s.
it was raised to 64k in 2019. the discussion at the time mentioned another 5% improvement by raising to 128k and settled for a very conservative setting.
it's 2024 now, I think it should be revisited to match modern hardware. I am measuring 0-15% performance improvement when raising to 256k on various types of disk. there is no downside as far as I can tell.
this function is only intended for sequential copy of full files (or file like objects). it's the typical use case that benefits from larger operations.
for reference, I came across this function while trying to profile pip that is using it to copy files when installing python packages.
* add news
---------
Co-authored-by: rmorotti <romain.morotti@man.com>
Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
This is actually an upstream problem in curses, and has been reported
to them already:
https://lists.gnu.org/archive/html/bug-ncurses/2024-09/msg00101.html
This is a nice workaround in the meantime to prevent the segfault.
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Previously, formatting help output or error message for positional argument
with a tuple metavar raised exception.
Co-authored-by: Cyker Way <cykerway@gmail.com>
It can now have one of three forms:
* basename(argv0) -- for simple scripts
* python arv0 -- for directories, ZIP files, etc
* python -m module -- for imported modules
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
This is to allow the `dataclasses.make_dataclass` infrastructure to be used with another decorator that's compliant with `typing.dataclass_transform`. The new `decorator` argument to `dataclasses.make_dataclass` is `dataclasses.dataclass`, which used to be hard coded.
* Revert "GH-124639: add back loop param to staggered_race (#124700)"
This reverts commit e0a41a5dd1.
* Revert "gh-124309: Modernize the `staggered_race` implementation to support eager task factories (#124390)"
This reverts commit de929f353c.
Resolve a memory leak introduced in CPython 3.10's :mod:`ssl` when the :attr:`ssl.SSLSocket.session` property was accessed. Speeds up read and write access to said property by no longer unnecessarily cloning session objects via serialization.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Positional arguments with nargs equal to '?' or '*' no longer check
default against choices.
Optional arguments with nargs equal to '?' no longer check const
against choices.
* Setting the __module__ attribute for a class now removes the
__firstlineno__ item from the type's dict.
* The _collections_abc and _pydecimal modules now completely replace the
collections.abc and decimal modules after importing them. This
allows to get the source of classes and functions defined in these
modules.
* inspect.findsource() now checks whether the first line number for a
class is out of bound.
Fix a bug that can cause a crash when sub-interpreters use "basic"
single-phase extension modules. Shared objects could refer to PyGC_Head
nodes that had been freed as part of interpreter shutdown.
Change the default multiprocessing start method away from fork to forkserver or spawn on the remaining platforms where it was fork. See the issue for context. This makes the default far more thread safe (other than for people spawning threads at import time... - don't do that!).
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Add NEWS.d entry
* Allow ISO-8601 24:00 alternative to midnight on datetime.time.fromisoformat()
* Allow ISO-8601 24:00 alternative to midnight on datetime.datetime.fromisoformat()
* Add NEWS.d entry
* Improve error message when hour is 24 and minute/second/microsecond is not 0
* Add tests for 24:00 fromisoformat
* Remove duplicate call to days_in_month() by storing in variable
* Add Python implementation
* Fix Lint
* Fix differing error msg in datetime.fromisoformat implementations when 24hrs has non-zero time component(s)
* Fix using time components inside tzinfo in Python implementation
* Don't parse tzinfo in C implementation when invalid iso midnight
* Remove duplicated variable in datetime test assertion line
* Add self to acknowledgements
* Remove duplicate NEWS entry
* Linting
* Add missing test case for when wrapping the year makes it invalid (too large)
Make `versionchanged:: next`` expand to current (unreleased) version.
When a new CPython release is cut, the release manager will replace
all such occurences of "next" with the just-released version.
(See the issue for release-tools and devguide PRs.)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Lib/test/certdata: do not hardcode reference cert data into tests
The script was simply printing the reference data and asking
users to update it by hand into the test suites. This can
be easily improved by writing the data into files and
having the test cases load the files.
* make_ssl_certs: make it possible to pass in expiration dates from command line
Note that in this commit, the defaults are same as they were,
so if nothing is specified the script works as before.
---------
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
* Detect source file encoding.
* Use the "replace" error handler even for UTF-8 (default) encoding.
* Remove the BOM.
* Fix detection of too long lines if they contain NUL.
* Return the head rather than the tail for truncated long lines.
This allows to use positional argument with nargs='*' and without default
in mutually exclusive group and improves error message about required
arguments.
Arguments with the value identical to the default value (e.g. booleans,
small integers, empty or 1-character strings) are no longer considered
"not present".
- If setting `_fields_` fails, e.g. with AttributeError, don't set the attribute in `__dict__`
- Document the “finalization” behaviour
- Beef up tests: add `getattr`, test Union as well as Structure
- Put common functionality in a common function
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
The code changes for warning related to `__package__` landed in Python 3.12. `__cached__` doesn't have any changes as it isn't used but only set by the import system.
In case of usage a long command along with max_help_position more than
the length of the command, the command's help was incorrectly started
on the new line.
Co-authored-by: Pavel Ditenbir <pavel.ditenbir@gmail.com>
https://github.com/python/cpython/issues/88496 replaced text.update with text.update_idletasks in colorizer.py and outwin.py to fix test failures on macOS. While theoretically correct, the result was Shell freezing when receiving continuous short strings to print. Test: `while 1: 1`.
The guess is that there is no idle time in which to do the screen update. Reverting the change in one of the files,
outwin, fixes the issue. Colorizer runs ever 1/20 second and seems to work fine.
When running test-outwin on macOS, alias 'update'
to 'update_idletasks on the text used for testing.
Add a helper function that checks whether the test suite is running
inside a systemd-nspawn container, and skip the few tests failing
with `--suppress-sync=true` in that case. The tests are failing because
`--suppress-sync=true` stubs out `fsync()`, `fdatasync()` and `msync()`
calls, and therefore they always return success without checking for
invalid arguments.
Call `os.open(__file__, os.O_RDONLY | os.O_SYNC)` and check the errno to
detect whether `--suppress-sync=true` is actually used, and skip
the tests only in that scenario.
Debian (and derivatives) provide a /usr/bin/pager binary, managed by the
alternatives system, that always points to an available pager utility.
Allow _pyrepl to use it, to follow system policy.
This is a very trivial change, from a patch that Debian has been
carrying since 2.7 era. Seems appropriate to upstream.
https://bugs.debian.org/799555
Buildbot failure on Windows 10 with MSC v.1916 64 bit (AMD64):
FAIL: testFmod (test.test_math.MathTests.testFmod)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\buildarea\3.x.bolen-windows10\build\Lib\test\test_math.py", line 605, in testFmod
self.ftest('fmod(-10, 1)', math.fmod(-10, 1), -0.0)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\buildarea\3.x.bolen-windows10\build\Lib\test\test_math.py", line 258, in ftest
self.fail("{}: {}".format(name, failure))
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: fmod(-10, 1): expected -0.0, got 0.0 (zero has wrong sign)
Here Windows loose sign of the result; if y is nonzero, the result
should have the same sign as x.
This amends commit 28aea5d07d.
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
pyrepl: Support Del, PgUp, and PgDn on TERM=vt100
From Fedora's /etc/inputrc:
"\e[5~": history-search-backward
"\e[6~": history-search-forward
"\e[3~": delete-char
Fixes https://github.com/python/cpython/issues/124027
Use a `_PyStackRef` and defer the reference to `f_executable` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
* gh-116608: Apply style and compatibility changes from importlib_metadata.
* gh-121735: Ensure module-adjacent resources are loadable from a zipfile.
* gh-121735: Allow all modules to be processed by the ZipReader.
* Add blurb
* Remove update-zips script, unneeded.
* Remove unnecessary references to removed static fixtures.
* Remove zipdata fixtures, unused.
POSIX allows errno to be negative.
Even though all currently supported platforms have non-negative errno,
relying on a quirk like that would make Python less portable.
* Raise PicklingError instead of UnicodeEncodeError, ValueError
and AttributeError in both implementations.
* Chain the original exception to the pickle-specific one as __context__.
* Include the error message of ImportError and some AttributeError in
the PicklingError error message.
* Unify error messages between Python and C implementations.
* Refer to documented __reduce__ and __newobj__ callables instead of
internal methods (e.g. save_reduce()) or pickle opcodes (e.g. NEWOBJ).
* Include more details in error messages (what expected, what got).
* Avoid including a potentially long repr of an arbitrary object in
error messages.
This switches the main pyrepl event loop to always be non-blocking so that it
can listen to incoming interruptions from other threads.
This also resolves invalid display of exceptions from other threads
(gh-123178).
This also fixes freezes with pasting and an active input hook.
Improve import time of `socket` by writing `socket.errorTab`
as a constant and lazy import modules.
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Increases the multiprocessing connection buffer size from 8k to 64k for efficiency, without overallocating.
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Add PyConfig_Get(), PyConfig_GetInt(), PyConfig_Set() and
PyConfig_Names() functions to get and set the current runtime Python
configuration.
Add visibility and "sys spec" to config and preconfig specifications.
_PyConfig_AsDict() now converts PyConfig.xoptions as a dictionary.
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Remove backtracking when parsing tarfile headers
* Rewrite PAX header parsing to be stricter
* Optimize parsing of GNU extended sparse headers v0.0
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* urljoin() with relative reference "?" sets empty query and removes fragment.
* Preserve empty components (authority, params, query, fragment) in urljoin().
* Preserve empty components (authority, params, query) in urldefrag().
Also refactor the code and get rid of double _coerce_args() and
_coerce_result() calls in urljoin(), urldefrag(), urlparse() and
urlunparse().
When checking if the registering browser is the "OS preferred browser", do not use a substring search - that makes no sense: one can have a preferred browser that looks like a super-string of a known browser, e.g. "firefox-nightly" vs "firefox".
https://github.com/python/cpython/issues/108172 explains in more detail, and lays out a potential better future enhancement for this case of just using xdg-open. We'll go with this for now.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
See https://github.com/python/cpython/issues/121313 for analysis, but this greatly reduces memory overallocation and overhead when multiprocessing is sending non-small data over its pipes between processes.
The `zip_next` function uses a common optimization technique for methods
that generate tuples. The iterator maintains an internal reference to
the returned tuple. When the method is called again, it checks if the
internal tuple's reference count is 1. If so, the tuple can be reused.
However, this approach is not safe under the free-threading build:
after checking the reference count, another thread may perform the same
check and also reuse the tuple. This can result in a double decref on
the items of the replaced tuple and a double incref (memory leak) on
the items of the tuple being set.
This adds a function, `_PyObject_IsUniquelyReferenced` that
encapsulates the stricter logic necessary for the free-threaded build:
the internal tuple must be owned by the current thread, have a local
refcount of one, and a shared refcount of zero.
* Make `weakref.WeakSet` safe against concurrent mutations while it is being iterated.
`_IterationGuard` is no longer used for `WeakSet`, it now relies on copying the underlying set which is an atomic operation while iterating so that it can be modified by other threads.
Per feedback from Paul Moore on GH-123158, it's better to defer making
`Path.delete()` public than ship it with under-designed error handling
capabilities.
We leave a remnant `_delete()` method, which is used by `move()`. Any
functionality not needed by `move()` is deleted.
These two methods accept an *existing* directory path, onto which we join
the source path's base name to form the final target path.
A possible alternative implementation is to check for directories in
`copy()` and `move()` and adjust the target path, which is done in several
`shutil` functions. This behaviour is helpful in a shell context, but
less so in a stored program that explicitly specifies destinations. For
example, a user that calls `Path('foo.py').copy('bar.py')` might not
imagine that `bar.py/foo.py` would be created, but under the alternative
implementation this will happen if `bar.py` is an existing directory.
When display lines above the cursor come from the cache, the first line
to not come from the cache may be a wrapped line, starting half way
through a logical line in the buffer. Detect and handle this case to
avoid accidentally drawing a stray prompt in the middle of a logical
line.
Sam Gross
Tian Gao
Petr Viktorin
Thomas Grainger
Steven Jin
Kirill Podoprigora
Wulian
Serhiy Storchaka
rindeal
Barney Gale
Kumar Aditya
Bernhard M. Wiedemann
Jelle Zijlstra
sobolevn
Y5
Richard Hansen
Mark Shannon
Sergey B Kirpichev
Pablo Galindo Salgado
Savannah Ostrowski
Eric Snow
Tomas R.
Alex Waygood
Alexander Kanavin
Victor Stinner
Bénédikt Tran
Alyssa Coghlan
Cody Maloney
Zhikang Yan
morotti
Irit Katriel
Brandt Bucher
Diego Russo
Jeremy Maitin-Shepard
Peter Bierma
Taneli Hukkinen
Bartosz Sławecki
Victorien
T. Wouters
Jeffrey R. Van Voorst
Steve Dower
Dino Viehland
Ned Deily
Kira
Mariatta
Tony Roberts
Malcolm Smith
Neil Schemenauer
Gregory P. Smith
Ron Frederick
dgpb
Nice Zombies
TizzySaurus
Zachary Ware
Raymond Hettinger
Emily Morehouse
Eric V. Smith
Jérémie Detrey
Payton
Xie Yanbo
Jason Fried
Donghee Na
Wulian
Brett Cannon
Terry Jan Reedy
Michał Górny
AN Long
Brian Schubert
luk1337
Jakub Kulík
Stefano Rivera
Jacek
Nate Ohlson
neonene
Rafael Fontenelle
Russell Keith-Magee
Hugo van Kemenade
Wenzel Jakob
Benjamin Peterson
Yngve Mardal Moe
Jason R. Coombs
Miro Hrončok
Anders Kaseorg
adang1345
Tushar Sadhwani
Jérôme Duval
algonell
Seth Michael Larson
Łukasz Langa
aorcajo
Furkan Onder
nkinnan
Bar Harel
devdanzin
Yorik Hansen
Inada Naoki
Yoda
Wei-Hsiang (Matt) Wang
Oded Arbel
Alexander P.
Matt Wozniski
Bogdan Romanyuk
Alexandr Mitin
Pieter Eendebak