Make SSL objects thread safe in Free Theaded build by
using critical sections.
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* Convert paths in venv activate script when using Git Bash under Windows
With https://github.com/python/cpython/pull/112508 the check to converts paths when running on Windows was changed from using the non-posix environment variable `$OSTYPE` to using `uname` instead.
However this missed the fact that when running under Git Bash on Windows, uname reports `MINGW*` (`$OSTYPE` is still `msys`).
This results in `$PATH` being set to something like `D:\a\github-actions-shells\github-actions-shells\venv/Scripts:…`, instead of `/d/a/github-actions-shells/github-actions-shells/venv/Scripts`.
Notably, the Git Bash is the bash shell that’s used for GitHub Actions Windows runners, and ships with VSCode.
Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.
* Characters not encodable in the current locale are now acceptable in
the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
longer recombinated.
* Embedded null character no longer terminates the format string.
This fixes also gh-78662 and gh-124531.
Rewriting the day and date every tick somehow prevented them from being removed either by clicking STOP or loading another example. The solution is to rewrite them only when they change.
This is an implementation of InterpreterPoolExecutor that builds on ThreadPoolExecutor.
(Note that this is not tied to PEP 734, which is strictly about adding a new stdlib module.)
Possible future improvements:
* support passing a script for the initializer or to submit()
* support passing (most) arbitrary functions without pickling
* support passing closures
* optionally exec functions against __main__ instead of the their original module
There was a deadlock when `ProcessPoolExecutor` shuts down at the same
time that a queueing thread handles an error processing a task.
Don't use `_shutdown_lock` to protect the `_ThreadWakeup` pipes -- use
an internal lock instead. This fixes the ordering deadlock where the
`ExecutorManagerThread` holds the `_shutdown_lock` and joins the
queueing thread, while the queueing thread is attempting to acquire the
`_shutdown_lock` while closing the `_ThreadWakeup`.
On Arm v5 it is not possible to get the thread ID via c13 register
hence the illegal instruction. The c13 register started to provide
thread ID since Arm v6K architecture variant. Other variants of
Arm v6 (T2, Z and base) don’t provide the thread ID via c13.
For the sake of simplicity we group v5 and v6 together and
consider that instructions for Arm v7 only.
Workaround for old libffi versions is added.
Module ctypes now supports C11 double complex only with libffi >= 3.3.0.
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
This fixes a crash when running the PyO3 test suite on the free-threaded
build. The `qsbr` field is initialized after the `PyThreadState` is
added to the interpreter's linked list -- it might still be NULL.
Instead, we "steal" the queue of to-be-freed memory blocks. This is
always initialized (possibly empty) and protected by the stop the world
pause.
We previously used `AC_RUN_IF_ELSE` with a short test program to detect
if `-latomic` is needed, but that requires choosing a specific default
value when cross-compiling because the test program is not run.
Some cross compilation targets like `wasm32-emscripten` do not support
`-latomic`, while other cross compilation targets, like
`arm-linux-gnueabi` require it.
It is an alternate constructor which only accepts a single numeric argument.
Unlike to Decimal.from_float() it accepts also Decimal.
Unlike to the standard constructor, it does not accept strings and tuples.
It is an alternative constructor which only accepts a single numeric argument.
Unlike to Fraction.from_float() and Fraction.from_decimal() it accepts any
real numbers supported by the standard constructor (int, float, Decimal,
Rational numbers, objects with as_integer_ratio()).
Unlike to the standard constructor, it does not accept strings.
Lock `ZoneInfoType` to protect accesses to `ZONEINFO_STRONG_CACHE`.
Refactor the `tp_new` handler to use Argument Clinic so that we can just
use `@critical_section` annotations on the relevant functions.
Also use `PyDict_SetDefaultRef` instead of `PyDict_SetDefault` when
inserting into the `TIMEDELTA_CACHE`.
`PurePath.__init__()` incorrectly uses the `_raw_paths` of a given
`PurePath` object with a different flavour, even though the procedure to
join path segments can differ between flavours.
This change makes the `_raw_paths`-enabled deferred joining apply _only_
when the path flavours match.
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
In some locales (like French or Hebrew) the full or abbreviated names of
the default month and weekday used in __calc_date_time can be part of
other name or constant part of the %c format. The month name can also
match %m with constant suffix (like in Japanese). So the code failed to
correctly distinguish formats %a, %A, %b, %B and %m.
Cycle all month and all days of the week to find the variable part
and distinguish %a from %A and %b from %B or %m.
Fixed locales for the following languges:
Arabic, Bislama, Breton, Bodo, Kashubian, Chuvash, Estonian, French, Irish,
Ge'ez, Gurajati, Manx Gaelic, Hebrew, Hindi, Chhattisgarhi, Haitian Kreyol,
Japanese, Kannada, Korean, Marathi, Malay, Norwegian, Nynorsk, Punjabi,
Rajasthani, Tok Pisin, Yoruba, Yue Chinese, Yau/Nungon and Chinese.
Co-authored-by: Eli Bendersky <eliben@gmail.com>
This follows GNU gzip, which defaults to using 0 as the mtime
for compressing stdin, where no file mtime is involved.
This makes the output of gzip.compress() deterministic by default,
greatly helping reproducible builds.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Raise ValueError in add_argument() if either explicit nargs=0 or action
that does not consume arguments (like 'store_const' or 'store_true') is
specified for positional argument.
Previously, all nested mutually exclusive groups lost their connection
to the group containing them and were displayed as belonging directly
to the parser.
Co-authored-by: Danica J. Sutherland <djsutherland@users.noreply.github.com>
memcopy'ing arbitrary values to _Bool variable triggers undefined
behaviour. Avoid this.
We assume that `false` is represented by all zero bytes.
Credits to Alex Gaynor.
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
* Add definitions for "context", "current context", and "context
management protocol".
* Update related definitions to be consistent with the new
definitions.
* Restructure the documentation for the `contextvars.Context` class
to prepare for adding context manager support, and for consistency
with the definitions.
* Use `testcode` and `testoutput` to test the `Context.run` example.
* Expand the documentation for the `Py_CONTEXT_EVENT_ENTER` and
`Py_CONTEXT_EVENT_EXIT` events to clarify and to prepare for
planned changes.
As argparse now detects by default when the code was run as a module.
This leads to using the actual executable name instead of simply "python"
to display in the usage message ("usage: python -m ...").
Fix a crash caused by immortal interned strings being shared between
sub-interpreters that use basic single-phase init. In that case, the string
can be used by an interpreter that outlives the interpreter that created and
interned it. For interpreters that share obmalloc state, also share the
interned dict with the main interpreter.
This is an un-revert of gh-124646 that then addresses the Py_TRACE_REFS
failures identified by gh-124785.
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Barry Warsaw <barry@python.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
This allows testing Y2038 with system time set to after that,
so that actual Y2038 issues can be exposed, and not masked
by expired certificate errors.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
If the PYTHON_BASIC_REPL environment variable is set, the site module
no longer imports the _pyrepl module.
Moreover, the site module now respects -E and -I command line
options: ignore PYTHON_BASIC_REPL in this case.
- move the Py_Main documentation from the very high level API section
to the initialization and finalization section
- make it clear that it encapsulates a full Py_Initialize/Finalize
cycle of its own
- point out that exactly which settings will be read and applied
correctly when Py_Main is called after a separate runtime
initialization call is version dependent
- be explicit that Py_IsInitialized can be called prior to
initialization
- actually test that Py_IsInitialized can be called prior to
initialization
- flush stdout in the embedding tests that run code so it appears
in the expected order when running with "-vv"
- make "-vv" on the subinterpreter embedding tests less spammy
---------
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
The function now sets temporarily the LC_CTYPE locale to the locale
of the category that determines the requested value if the locales are
different and the resulting string is non-ASCII.
This temporary change affects other threads.
The term "free variable" has unfortunately become genuinely
ambiguous over the years (presumably due to the names of
some relevant code object instance attributes).
While we can't eliminate that ambiguity at this late date, we can
at least alert people to the potential ambiguity by describing
both the formal meaning of the term and the common
alternative use as a direct synonym for "closure variable".
---------
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
Increase currently inadequate vertical spacing for the IDLE browsers (path,
module, and stack) on high-resolution monitors.
---------
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
This allows direct intersphinx references to APIs via references
like `` :func:`importlib.metadata.version` ``.
---------
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
* gh-117151: increase default buffer size of shutil.copyfileobj() to 256k.
it was set to 16k in the 1990s.
it was raised to 64k in 2019. the discussion at the time mentioned another 5% improvement by raising to 128k and settled for a very conservative setting.
it's 2024 now, I think it should be revisited to match modern hardware. I am measuring 0-15% performance improvement when raising to 256k on various types of disk. there is no downside as far as I can tell.
this function is only intended for sequential copy of full files (or file like objects). it's the typical use case that benefits from larger operations.
for reference, I came across this function while trying to profile pip that is using it to copy files when installing python packages.
* add news
---------
Co-authored-by: rmorotti <romain.morotti@man.com>
Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
This is actually an upstream problem in curses, and has been reported
to them already:
https://lists.gnu.org/archive/html/bug-ncurses/2024-09/msg00101.html
This is a nice workaround in the meantime to prevent the segfault.
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Previously, formatting help output or error message for positional argument
with a tuple metavar raised exception.
Co-authored-by: Cyker Way <cykerway@gmail.com>
It can now have one of three forms:
* basename(argv0) -- for simple scripts
* python arv0 -- for directories, ZIP files, etc
* python -m module -- for imported modules
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
This is to allow the `dataclasses.make_dataclass` infrastructure to be used with another decorator that's compliant with `typing.dataclass_transform`. The new `decorator` argument to `dataclasses.make_dataclass` is `dataclasses.dataclass`, which used to be hard coded.
* Revert "GH-124639: add back loop param to staggered_race (#124700)"
This reverts commit e0a41a5dd1.
* Revert "gh-124309: Modernize the `staggered_race` implementation to support eager task factories (#124390)"
This reverts commit de929f353c.
Resolve a memory leak introduced in CPython 3.10's :mod:`ssl` when the :attr:`ssl.SSLSocket.session` property was accessed. Speeds up read and write access to said property by no longer unnecessarily cloning session objects via serialization.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Positional arguments with nargs equal to '?' or '*' no longer check
default against choices.
Optional arguments with nargs equal to '?' no longer check const
against choices.
* Setting the __module__ attribute for a class now removes the
__firstlineno__ item from the type's dict.
* The _collections_abc and _pydecimal modules now completely replace the
collections.abc and decimal modules after importing them. This
allows to get the source of classes and functions defined in these
modules.
* inspect.findsource() now checks whether the first line number for a
class is out of bound.
Fix a bug that can cause a crash when sub-interpreters use "basic"
single-phase extension modules. Shared objects could refer to PyGC_Head
nodes that had been freed as part of interpreter shutdown.
Change the default multiprocessing start method away from fork to forkserver or spawn on the remaining platforms where it was fork. See the issue for context. This makes the default far more thread safe (other than for people spawning threads at import time... - don't do that!).
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Add NEWS.d entry
* Allow ISO-8601 24:00 alternative to midnight on datetime.time.fromisoformat()
* Allow ISO-8601 24:00 alternative to midnight on datetime.datetime.fromisoformat()
* Add NEWS.d entry
* Improve error message when hour is 24 and minute/second/microsecond is not 0
* Add tests for 24:00 fromisoformat
* Remove duplicate call to days_in_month() by storing in variable
* Add Python implementation
* Fix Lint
* Fix differing error msg in datetime.fromisoformat implementations when 24hrs has non-zero time component(s)
* Fix using time components inside tzinfo in Python implementation
* Don't parse tzinfo in C implementation when invalid iso midnight
* Remove duplicated variable in datetime test assertion line
* Add self to acknowledgements
* Remove duplicate NEWS entry
* Linting
* Add missing test case for when wrapping the year makes it invalid (too large)
Make `versionchanged:: next`` expand to current (unreleased) version.
When a new CPython release is cut, the release manager will replace
all such occurences of "next" with the just-released version.
(See the issue for release-tools and devguide PRs.)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* Lib/test/certdata: do not hardcode reference cert data into tests
The script was simply printing the reference data and asking
users to update it by hand into the test suites. This can
be easily improved by writing the data into files and
having the test cases load the files.
* make_ssl_certs: make it possible to pass in expiration dates from command line
Note that in this commit, the defaults are same as they were,
so if nothing is specified the script works as before.
---------
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
* Detect source file encoding.
* Use the "replace" error handler even for UTF-8 (default) encoding.
* Remove the BOM.
* Fix detection of too long lines if they contain NUL.
* Return the head rather than the tail for truncated long lines.
This allows to use positional argument with nargs='*' and without default
in mutually exclusive group and improves error message about required
arguments.
Arguments with the value identical to the default value (e.g. booleans,
small integers, empty or 1-character strings) are no longer considered
"not present".
- If setting `_fields_` fails, e.g. with AttributeError, don't set the attribute in `__dict__`
- Document the “finalization” behaviour
- Beef up tests: add `getattr`, test Union as well as Structure
- Put common functionality in a common function
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
The code changes for warning related to `__package__` landed in Python 3.12. `__cached__` doesn't have any changes as it isn't used but only set by the import system.
In case of usage a long command along with max_help_position more than
the length of the command, the command's help was incorrectly started
on the new line.
Co-authored-by: Pavel Ditenbir <pavel.ditenbir@gmail.com>
https://github.com/python/cpython/issues/88496 replaced text.update with text.update_idletasks in colorizer.py and outwin.py to fix test failures on macOS. While theoretically correct, the result was Shell freezing when receiving continuous short strings to print. Test: `while 1: 1`.
The guess is that there is no idle time in which to do the screen update. Reverting the change in one of the files,
outwin, fixes the issue. Colorizer runs ever 1/20 second and seems to work fine.
When running test-outwin on macOS, alias 'update'
to 'update_idletasks on the text used for testing.
Add a helper function that checks whether the test suite is running
inside a systemd-nspawn container, and skip the few tests failing
with `--suppress-sync=true` in that case. The tests are failing because
`--suppress-sync=true` stubs out `fsync()`, `fdatasync()` and `msync()`
calls, and therefore they always return success without checking for
invalid arguments.
Call `os.open(__file__, os.O_RDONLY | os.O_SYNC)` and check the errno to
detect whether `--suppress-sync=true` is actually used, and skip
the tests only in that scenario.
Debian (and derivatives) provide a /usr/bin/pager binary, managed by the
alternatives system, that always points to an available pager utility.
Allow _pyrepl to use it, to follow system policy.
This is a very trivial change, from a patch that Debian has been
carrying since 2.7 era. Seems appropriate to upstream.
https://bugs.debian.org/799555
Buildbot failure on Windows 10 with MSC v.1916 64 bit (AMD64):
FAIL: testFmod (test.test_math.MathTests.testFmod)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\buildarea\3.x.bolen-windows10\build\Lib\test\test_math.py", line 605, in testFmod
self.ftest('fmod(-10, 1)', math.fmod(-10, 1), -0.0)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\buildarea\3.x.bolen-windows10\build\Lib\test\test_math.py", line 258, in ftest
self.fail("{}: {}".format(name, failure))
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: fmod(-10, 1): expected -0.0, got 0.0 (zero has wrong sign)
Here Windows loose sign of the result; if y is nonzero, the result
should have the same sign as x.
This amends commit 28aea5d07d.
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Alex Waygood
Tian Gao
Peter Bierma
Julien
Serhiy Storchaka
Adam Turner
Pablo Galindo Salgado
Nice Zombies
Sam Gross
chrysn
Jelle Zijlstra
George Pittock
Wulian
Jonathan Protzenko
Savannah Ostrowski
Eric Snow
Steve Dower
Vincent Fazio
Diego Russo
Hugo van Kemenade
Mikhail Efimov
Petr Viktorin
Thomas Grainger
Steven Jin
Kirill Podoprigora
rindeal
Barney Gale
Kumar Aditya
Bernhard M. Wiedemann
sobolevn
Y5
Richard Hansen
Mark Shannon
Sergey B Kirpichev
Tomas R.
Alexander Kanavin
Victor Stinner
Bénédikt Tran
Alyssa Coghlan
Cody Maloney
Zhikang Yan
morotti
Irit Katriel
Brandt Bucher
Jeremy Maitin-Shepard
Taneli Hukkinen
Bartosz Sławecki
Victorien
T. Wouters
Jeffrey R. Van Voorst
Dino Viehland
Ned Deily
Kira
Mariatta
Tony Roberts
Malcolm Smith
Neil Schemenauer
Gregory P. Smith
Ron Frederick
dgpb
TizzySaurus
Zachary Ware
Raymond Hettinger
Emily Morehouse
Eric V. Smith
Jérémie Detrey
Payton
Xie Yanbo
Jason Fried
Donghee Na
Wulian
Brett Cannon
Terry Jan Reedy
Michał Górny
AN Long
Brian Schubert
luk1337
Jakub Kulík
Stefano Rivera
Jacek
Nate Ohlson
neonene
Rafael Fontenelle
Russell Keith-Magee
Wenzel Jakob
Benjamin Peterson