This will keep us from adding new unsupported (i.e. non-const) C global variables, which would break interpreter isolation.
FYI, historically it is very uncommon for new global variables to get added. Furthermore, it is rare for new code to break the c-analyzer. So the check should almost always pass unnoticed.
Note that I've removed test_check_c_globals. A test wasn't a great fit conceptually and was super slow on debug builds. A CI check is a better fit.
This also resolves gh-100237.
https://github.com/python/cpython/issues/81057
Skip `TestGetAsyncGenState` and restoring of the default event loop policy in `test_inspect` if platform lacks working socket support.
Fixes#11590
Automerge-Triggered-By: GH:kumaraditya303
It is possible but unlikely for the `python_tzpath_context` function to fail between the start of the `try` block and the point where `os.environ.get` succeeds, in which case `old_env` will be undefined. In this case, we want to take no action.
Practically speaking this will really only happen in an error condition anyway, so it doesn't really matter, but we should probably do it right anyway.
Add `MS_WINDOWS_DESKTOP`, `MS_WINDOWS_APPS`, `MS_WINDOWS_SYSTEM` and `MS_WINDOWS_GAMES` preprocessor definitions to allow switching off functionality missing from particular API partitions ("partitions" are used in Windows to identify overlapping subsets of APIs).
CPython only officially supports `MS_WINDOWS_DESKTOP` and `MS_WINDOWS_SYSTEM` (APPS is included by normal desktop builds, but APPS without DESKTOP is not covered). Other configurations are a convenience for people building their own runtimes.
`MS_WINDOWS_GAMES` is for the Xbox subset of the Windows API, which is also available on client OS, but is restricted compared to `MS_WINDOWS_DESKTOP`. These restrictions may change over time, as they relate to the build headers rather than the OS support, and so we assume that Xbox builds will use the latest available version of the GDK.
Since Mercurial removal from bitbucket.org, some links are broken.
They are replaced by github.com or webarchive.org links if available. Otherwise, they are removed.
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
The previous `_parse_args()` method pulled the `_parts` out of any supplied `PurePath` objects; these were subsequently joined in `_from_parts()` using `os.path.join()`. This is actually a slower form of joining than calling `fspath()` on the path object, because it doesn't take advantage of the fact that the contents of `_parts` is normalized!
This reduces the time taken to run `PurePath("foo", "bar")` by ~20%, and the time taken to run `PurePath(p, "cheese")`, where `p = PurePath("/foo", "bar", "baz")`, by ~40%.
Automerge-Triggered-By: GH:AlexWaygood
As part of investigation issue https://github.com/python/cpython/issues/102433, I discovered what I believe to be an error where two classes `CI` and `DI` are not being used. The assertions beneath them act on `C` and `D`, duplicating existing assertions in this test.
Automerge-Triggered-By: GH:AlexWaygood
Found some duplicate `to`s in the documentation and some code comments and fixed them.
[Misc/NEWS.d/3.12.0a1.rst](ed55c69ebd/Misc/NEWS.d/3.12.0a1.rst) also contains two duplicate `to`s, but I wasn't sure if it's ok to touch that file. Looks auto generated. I'm happy to amend the PR if requested. :)
Automerge-Triggered-By: GH:AlexWaygood
Refactor DynOptionMenu's initializer to not copy kwargs dict and use subscripting;
improve its htest.
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Make docstrings for `as_integer_ratio` consistent across types, and document that
the returned pair is always normalized (coprime integers, with positive denominator).
---------
Co-authored-by: Owain Davies <116417456+OTheDev@users.noreply.github.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
This PR adds a private `Fraction._from_coprime_ints` classmethod for internal creations of `Fraction` objects, replacing the use of `_normalize=False` in the existing constructor. This speeds up creation of `Fraction` objects arising from calculations. The `_normalize` argument to the `Fraction` constructor has been removed.
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
This is related to fixing the refleaks introduced by commit 096d009. I haven't been able to find the leak yet, but these changes are a consequence of that effort. This includes some cleanup, some tweaks to the existing tests, and a bunch of new test cases. The only change here that might have impact outside the tests in question is in imp.py, where I update imp.load_dynamic() to use spec_from_file_location() instead of creating a ModuleSpec directly.
Also note that I've updated the tests to only skip if we're checking for refleaks (regrtest's --huntrleaks), whereas in gh-101969 I had skipped the tests entirely. The tests will be useful for some upcoming work and I'd rather the refleaks not hold that up. (It isn't clear how quickly we'll be able to fix the leaking code, though it will certainly be done in the short term.)
https://github.com/python/cpython/issues/102251
The argument is used as a switch and corresponds to a boolean logic. Therefore it is more intuitive to use the corresponding constant `False` as default value instead of the integer `0`.
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
* Don't deadlock on shutdown if test_current_{exception,frames} fails
These tests spawn a thread that waits on a threading.Event. If the test fails any of its assertions, the Event won't be signaled and the thread will wait indefinitely, causing a deadlock when threading._shutdown() tries to join all outstanding threads.
Co-authored-by: Brett Simmers <bsimmers@meta.com>
* Add a news entry
* Fix whitespace
---------
Co-authored-by: Brett Simmers <bsimmers@meta.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
lzma.LZMADecompressor and bz2.BZ2Decompressor objects caused
segfaults when their `__init__()` methods were not called.
lzma.LZMADecompressor, lzma.LZMACompressor, bz2.BZ2Compressor,
and bz2.BZ2Decompressor objects would leak locks and internal buffers
when their `__init__()` methods were called multiple times.
https://bugs.python.org/issue23224
- partial tests for cosh/sinh overflows (L535 and L771). I doubt
both ||-ed conditions could be tested.
- removed inaccessible case in sqrt (L832): ax=ay=0 is handled
above (L823) because fabs() is exact. Also added test (checked
with mpmath and gmpy2) for second condition on that line.
- some trivial tests for isclose (cover all conditions on L1217-1218)
- add comment for uncovered L1018
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
gh-101891 is causing failures under `$> ./python -m test test_imp -R 3:3`. Furthermore, with that fixed, "test_singlephase_variants" is leaking references. This change addresses the first part, but skips the leaking tests until we can follow up with a fix.
https://github.com/python/cpython/issues/101758
We're adding the function back, only for the stable ABI symbol and not as any form of API. I had removed it yesterday.
This undocumented "private" function was added with the implementation for PEP 3121 (3.0, 2007) for internal use and later moved out of the limited API (3.6, 2016) and then into the internal API (3.9, 2019). I removed it completely yesterday, including from the stable ABI manifest (where it was added because the symbol happened to be exported). It's unlikely that anyone is using _PyState_AddModule(), especially any stable ABI extensions built against 3.2-3.5, but we're playing it safe.
https://github.com/python/cpython/issues/101758
* fileutils: handle non-blocking pipe IO on Windows
Handle erroring operations on non-blocking pipes by reading the _doserrno code.
Limit writes on non-blocking pipes that are too large.
* Support blocking functions on Windows
Use the GetNamedPipeHandleState and SetNamedPipeHandleState Win32 API functions to add support for os.get_blocking and os.set_blocking.
This merges their code. They're backed by the same single HACL* static library, having them be a single module simplifies maintenance.
This should unbreak the wasm enscripten builds that currently fail due to linking in --whole-archive mode and the HACL* library appearing twice.
Long unnoticed error fixed: _sha512.SHA384Type was doubly assigned and was actually SHA512Type. Nobody depends on those internal names.
Also rename LIBHACL_ make vars to LIBHACL_SHA2_ in preperation for other future HACL things.
Enforcing (optionally) the restriction set by PEP 489 makes sense. Furthermore, this sets the stage for a potential restriction related to a per-interpreter GIL.
This change includes the following:
* add tests for extension module subinterpreter compatibility
* add _PyInterpreterConfig.check_multi_interp_extensions
* add Py_RTFLAGS_MULTI_INTERP_EXTENSIONS
* add _PyImport_CheckSubinterpIncompatibleExtensionAllowed()
* fail iff the module does not implement multi-phase init and the current interpreter is configured to check
https://github.com/python/cpython/issues/98627
This change is almost entirely moving code around and hiding import state behind internal API. We introduce no changes to behavior, nor to non-internal API. (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.) The motivation is to simplify a number of upcoming changes.
Specific changes:
* move existing import-related code to import.c, wherever possible
* add internal API for interacting with import state (both global and per-interpreter)
* use only API outside of import.c (to limit churn there when changing the location, etc.)
* consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly)
* add macros for import state in import.c (to simplify changing the location)
* group code in import.c into sections
*remove _PyState_AddModule()
https://github.com/python/cpython/issues/101758
The new test exercises the most important variants for single-phase init extension modules. We also add some explanation about those variants to import.c.
https://github.com/python/cpython/issues/101758
`socket.getaddrinfo()` no longer raises `OverflowError` based on the **port** argument. Error reporting (or not) for its value is left up to the underlying C library `getaddrinfo()` implementation.
Utilize new functions termios.tcgetwinsize() and termios.tcsetwinsize in test_pty.py.
Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Refactored the implementation of pty.fork to use os.login_tty.
A DeprecationWarning is now raised by pty.master_open() and pty.slave_open(). They were
undocumented and deprecated long long ago in the docstring in favor of pty.openpty.
Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* Make sure that the current exception is always normalized.
* Remove redundant type and traceback fields for the current exception.
* Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException
* Add new API functions: PyException_GetArgs, PyException_SetArgs
replacing hashlib primitives (for the non-OpenSSL case) with verified implementations from HACL*. This is the first PR in the series, and focuses specifically on SHA2-256 and SHA2-224.
This PR imports Hacl_Streaming_SHA2 into the Python tree. This is the HACL* implementation of SHA2, which combines a core implementation of SHA2 along with a layer of buffer management that allows updating the digest with any number of bytes. This supersedes the previous implementation in the tree.
@franziskuskiefer was kind enough to benchmark the changes: in addition to being verified (thus providing significant safety and security improvements), this implementation also provides a sizeable performance boost!
```
---------------------------------------------------------------
Benchmark Time CPU Iterations
---------------------------------------------------------------
Sha2_256_Streaming 3163 ns 3160 ns 219353 // this PR
LibTomCrypt_Sha2_256 5057 ns 5056 ns 136234 // library used by Python currently
```
The changes in this PR are as follows:
- import the subset of HACL* that covers SHA2-256/224 into `Modules/_hacl`
- rewire sha256module.c to use the HACL* implementation
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
This PR fixes the buildbot failures introduced by the merge of #5561, by restricting the relevant tests to something that should work on both 32-bit and 64-bit platforms. It also silences some compiler warnings introduced in that PR.
That causes the test to fail when run using a high UID as that ancient format
cannot represent it. The current default (PAX) and the old default (GNU) both
support high UIDs.
The summary of this diff is that it:
* adds a `_ctypes_alloc_format_padding` function to append strings like `37x` to a format string to indicate 37 padding bytes
* removes the branches that amount to "give up on producing a valid format string if the struct is packed"
* combines the resulting adjacent `if (isStruct) {`s now that neither is `if (isStruct && !isPacked) {`
* invokes `_ctypes_alloc_format_padding` to add padding between structure fields, and after the last structure field. The computation used for the total size is unchanged from ctypes already used.
This patch does not affect any existing aligment computation; all it does is use subtraction to deduce the amount of paddnig introduced by the existing code.
---
Without this fix, it would never include padding bytes - an assumption that was only
valid in the case when `_pack_` was set - and this case was explicitly not implemented.
This should allow conversion from ctypes structs to numpy structs
Fixes https://github.com/numpy/numpy/issues/10528
Update bundled pip version to 23.0
This is the current latest version of `pip`.
---------
Co-authored-by: Pradyun Gedam <pradyunsg@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Fix the behaviour of the `__sizeof__` method (and hence the results returned by `sys.getsizeof`) for subclasses of `int`. Previously, `int` subclasses gave identical results to the `int` base class, ignoring the presence of the instance dictionary.
<!-- gh-issue-number: gh-101266 -->
* Issue: gh-101266
<!-- /gh-issue-number -->
This starts the process. Users who don't specify their own start method
and use the default on platforms where it is 'fork' will see a
DeprecationWarning upon multiprocessing.Pool() construction or upon
multiprocessing.Process.start() or concurrent.futures.ProcessPool use.
See the related issue and documentation within this change for details.
`warnings.warn()` gains the ability to skip stack frames based on code
filename prefix rather than only a numeric `stacklevel=` via a new
`skip_file_prefixes=` keyword argument.
Fixes a reference counting issue with `ctypes.Structure` when a `from_param()` method call is used and the structure size is larger than a C pointer `sizeof(void*)`.
This problem existed for a very long time, but became more apparent in 3.8+ by change likely due to garbage collection cleanup timing changes.
When testing element truth values, emit a DeprecationWarning in all implementations.
This had emitted a FutureWarning in the rarely used python-only implementation since ~2.7 and has always been documented as a behavior not to rely on.
Matching an element in a tree search but having it test False can be unexpected. Raising the warning enables making the choice to finally raise an exception for this ambiguous behavior in the future.
This PR adds support for float-style formatting for `Fraction` objects: it supports the `"e"`, `"E"`, `"f"`, `"F"`, `"g"`, `"G"` and `"%"` presentation types, and all the various bells and whistles of the formatting mini-language for those presentation types. The behaviour almost exactly matches that of `float`, but the implementation works with the exact `Fraction` value and does not do an intermediate conversion to `float`, and so avoids loss of precision or issues with numbers that are outside the dynamic range of the `float` type.
Note that the `"n"` presentation type is _not_ supported. That support could be added later if people have a need for it.
There's one corner-case where the behaviour differs from that of float: for the `float` type, if explicit alignment is specified with a fill character of `'0'` and alignment type `'='`, then thousands separators (if specified) are inserted into the padding string:
```python
>>> format(3.14, '0=11,.2f')
'0,000,003.14'
```
The exact same effect can be achieved by using the `'0'` flag:
```python
>>> format(3.14, '011,.2f')
'0,000,003.14'
```
For `Fraction`, only the `'0'` flag has the above behaviour with respect to thousands separators: there's no special-casing of the particular `'0='` fill-character/alignment combination. Instead, we treat the fill character `'0'` just like any other:
```python
>>> format(Fraction('3.14'), '0=11,.2f')
'00000003.14'
>>> format(Fraction('3.14'), '011,.2f')
'0,000,003.14'
```
The `Fraction` formatter is also stricter about combining these two things: it's not permitted to use both the `'0'` flag _and_ explicit alignment, on the basis that we should refuse the temptation to guess in the face of ambiguity. `float` is less picky:
```python
>>> format(3.14, '0<011,.2f')
'3.140000000'
>>> format(Fraction('3.14'), '0<011,.2f')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/mdickinson/Repositories/python/cpython/Lib/fractions.py", line 414, in __format__
raise ValueError(
ValueError: Invalid format specifier '0<011,.2f' for object of type 'Fraction'; can't use explicit alignment when zero-padding
```
Use C long arithmetic instead of PyLong arithmetic to compute the range length, where possible.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
The zipfile.Path open() and read_text() encoding parameter can be supplied as a positional argument without causing a TypeError again. 3.10.0b1 included a regression that made it keyword only.
Documentation update included as users writing code to be compatible with a wide range of versions will need to consider this for some time.
Fix a bug where `Path` takes and ignores `**kwargs` by adding to `PurePath` class `__init__` method which can take only positional arguments.
Automerge-Triggered-By: GH:brettcannon
Partially revert changes made in GH-93453.
asyncio.DefaultEventLoopPolicy.get_event_loop() now emits a
DeprecationWarning and creates and sets a new event loop instead of
raising a RuntimeError if there is no current event loop set.
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
This brings the Python implementation of `ntpath.normpath()` in line with the C implementation added in 99fcf15
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Mock objects which are not unsafe will now raise an AttributeError when accessing an
attribute that matches the name of an assertion but without the prefix `assert_`, e.g. accessing `called_once` instead of `assert_called_once`.
This is in addition to this already happening for accessing attributes with prefixes assert, assret, asert, aseert, and assrt.
Make some trivial performance optimizations in Fraction
Uses private class attributes `_numerator` and `_denominator` in place of the `numerator` and `denominator` property accesses.
Co-authored-by: hauntsaninja <hauntsaninja@gmail.com>
Increase performance of the `absolute()` method by calling `os.getcwd()` directly, rather than using the `Path.cwd()` class method. This avoids constructing an extra `Path` object (and the parsing/normalization that comes with it).
Decrease performance of the `cwd()` class method by calling the `Path.absolute()` method, rather than using `os.getcwd()` directly. This involves constructing an extra `Path` object. We do this to maintain a longstanding pattern where `os` functions are called from only one place, which allows them to be more readily replaced by users. As `cwd()` is generally called at most once within user programs, it's a good bargain.
```shell
# before
$ ./python -m timeit -s 'from pathlib import Path; p = Path("foo", "bar")' 'p.absolute()'
50000 loops, best of 5: 9.04 usec per loop
# after
$ ./python -m timeit -s 'from pathlib import Path; p = Path("foo", "bar")' 'p.absolute()'
50000 loops, best of 5: 5.02 usec per loop
```
Automerge-Triggered-By: GH:AlexWaygood
Fixes behaviour where int (and subtypes like bool) __sizeof__ under-reports true size as it did not take into account the size 1 `ob_digit` array for the zero int.
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
Not comprehensive, best effort warning. There are cases when threads exist on some platforms that this code cannot detect. macOS when API permissions allow and Linux with a readable /proc procfs present are the currently supported cases where a warning should show up reliably.
Starting with a DeprecationWarning for now, it is less disruptive than something like RuntimeWarning and most likely to only be seen in people's CI tests - a good place to start with this messaging.
Kumar Aditya
Nikita Sobolev
Irit Katriel
yonatanp
Jamoo721
Martin Breuss
Andre Hora
JosephSBoyle
Steve Dower
Dong-hee Na
Eric Snow
Raymond Hettinger
T
谭九鼎
Hugo van Kemenade
gaogaotiantian
Thomas Krennwallner
Alex Waygood
Brandt Bucher
Ilya V. Schurov
Xuehai Pan
Barney Gale
Paul Ganssle
David Hewitt
Max Bachmann
sblondon
Thomas Wouters
Carl Meyer
Partha P. Mukherjee
Yeojin Kim
Marta Gómez Macías
Dustin Rodrigues
Jaysinh Shukla
Gouvernathor
Alexey Izbyshev
Raj
Michael K
Hyunkyun Moon
Stefan Pochmann
Petr Viktorin
Steven Troxler
Sergey B Kirpichev
Dennis Sweeney
Rotzbua
Eclips4
Jason R. Coombs
Pradyun Gedam
SKO
Ionite
Jacob Bower
Erlend E. Aasland
Zackery Spytz
Mark Shannon
Hyunkyun Moon
Vo Hoang Long
Gihwan Kim
Tim Hatch
Jean-Christophe Amiel
Furkan Onder
Jan Gosmann
Rayyan Ansari
penguin_wwy
Gregory P. Smith
Radek Smejkal
Soumendra Ganguly
mjoerg
Ikko Eltociear Ashimine
Oleg Iarygin
Michael Droettboom
Zachary Ware
Matthieu Dartiailh
Jonathan Protzenko
Mark Dickinson
Ethan Furman
Matty G
mrh1997
Eric Wieser
Alexander Belopolsky
Ruben Vorderman
Ayappan Perumal
beavailable
Yukihiro Nakadaira
Shantanu
achhina
Jacob Walls
Andrew Hong
Pieter Eendebak
Kamil Turek
J. Nick Koston
Jean-Paul Calderone
Thomas Grainger
Bob Kline
Yurii Karabas
Guido van Rossum
Serhiy Storchaka
Carlton Gibson
dgelessus
Chris Withers
Nnarol
Christian Klein
Stanislav Zmiev
Rupa Lahiri
Terry Jan Reedy
tqxia
Samet YASLAN