Commit Graph

27419 Commits

Author SHA1 Message Date
Steve Dower cb35882773
gh-102519: Add os.listdrives, os.listvolumes and os.listmounts on Windows (GH-102544) 2023-03-10 12:21:37 +00:00
David Hewitt 71cf7c3ddd
gh-102378: don't bother stripping `/` from __text_signature__ (#102379) 2023-03-09 15:06:20 -08:00
Max Bachmann c6858d1e7f
gh-102255: Improve build support for Windows API partitions (GH-102256)
Add `MS_WINDOWS_DESKTOP`, `MS_WINDOWS_APPS`, `MS_WINDOWS_SYSTEM` and `MS_WINDOWS_GAMES` preprocessor definitions to allow switching off functionality missing from particular API partitions ("partitions" are used in Windows to identify overlapping subsets of APIs).
CPython only officially supports `MS_WINDOWS_DESKTOP` and `MS_WINDOWS_SYSTEM` (APPS is included by normal desktop builds, but APPS without DESKTOP is not covered). Other configurations are a convenience for people building their own runtimes.
`MS_WINDOWS_GAMES` is for the Xbox subset of the Windows API, which is also available on client OS, but is restricted compared to `MS_WINDOWS_DESKTOP`. These restrictions may change over time, as they relate to the build headers rather than the OS support, and so we assume that Xbox builds will use the latest available version of the GDK.
2023-03-09 21:09:12 +00:00
Kumar Aditya 1a84cc007e
GH-102397: Fix segfault from race condition in signal handling (#102399)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-03-08 13:29:39 +05:30
Thomas Wouters f9774e57d8 Python 3.12.0a6 2023-03-07 22:48:18 +01:00
Irit Katriel a33ca2ad1f
gh-102493: fix normalization in PyErr_SetObject (#102502)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-03-07 13:27:46 -08:00
Pablo Galindo Salgado f533f216e6
gh-102416: Do not memoize incorrectly loop rules in the parser (#102467) 2023-03-06 14:41:53 +01:00
Barney Gale 6716254e71
GH-101362: Optimise PurePath(PurePath(...)) (GH-101667)
The previous `_parse_args()` method pulled the `_parts` out of any supplied `PurePath` objects; these were subsequently joined in `_from_parts()` using `os.path.join()`. This is actually a slower form of joining than calling `fspath()` on the path object, because it doesn't take advantage of the fact that the contents of `_parts` is normalized!

This reduces the time taken to run `PurePath("foo", "bar")` by ~20%, and the time taken to run `PurePath(p, "cheese")`, where `p = PurePath("/foo", "bar", "baz")`, by ~40%.

Automerge-Triggered-By: GH:AlexWaygood
2023-03-05 15:50:21 -08:00
Barney Gale 3e60e0213e
GH-101362: Check pathlib.Path flavour compatibility at import time (GH-101664)
This saves a comparison in `pathlib.Path.__new__()` and reduces the time taken to run `Path()` by ~5%.

Automerge-Triggered-By: GH:AlexWaygood
2023-03-05 14:46:45 -08:00
Barney Gale 3572c861d8
GH-101362: Call join() only when >1 argument supplied to pathlib.PurePath() (#101665)
GH-101362: Call join() only when >1 argument supplied to pathlib.PurePath

This reduces the time taken to run `PurePath("foo")` by ~15%
2023-03-05 22:00:56 +00:00
Yeojin Kim 9a478be1a4
gh-101979: argparse: fix a bug where parentheses in metavar argument of add_argument() were dropped (#102318) 2023-03-05 06:54:33 -08:00
Marta Gómez Macías 66aa78cbe6
gh-102356: Add thrashcan macros to filter object dealloc (#102426)
Add thrashcan macros to the deallocator of the filter objects to protect against deeply nested destruction of chains of nested filters.
2023-03-05 12:00:41 +01:00
Matthias Görgens eff9f43924
gh-96821: Add config option `--with-strict-overflow` (#96823)
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Shantanu <hauntsaninja@gmail.com>
2023-03-04 13:39:52 -08:00
Jaysinh Shukla 81763341ed
gh-63301: Set exit code when tabnanny CLI exits on error (#7699)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-03-04 13:32:13 -08:00
Byeongmin Choi 77a3196b7c
gh-101863: Fix wrong comments in EUC-KR codec (gh-102417) 2023-03-05 01:01:54 +09:00
Gouvernathor 90801e48fd
gh-102302 Micro-optimize `inspect.Parameter.__hash__` (#102303) 2023-03-04 15:08:57 +00:00
Alexey Izbyshev c2bd55d26f
gh-102179: Fix `os.dup2` error reporting for negative fds (#102180) 2023-03-04 19:54:08 +05:30
Raj 705487c655
gh-101892: Fix `SystemError` when a callable iterator call exhausts the iterator (#101896)
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-03-04 19:51:29 +05:30
Max Bachmann c1748ed59d
gh-102344: Reimplement winreg QueryValue / SetValue using QueryValueEx / SetValueEx (GH-102345)
The newer APIs are more widely available than the old ones, and are called in a way to preserve functionality.
2023-03-01 14:50:38 +00:00
Max Bachmann 938e36f824
gh-102336: Remove code specifically for handling Windows 7 (GH-102337) 2023-03-01 00:31:21 +00:00
Petr Viktorin 6b2d7c0ddb
gh-101101: Unstable C API tier (PEP 689) (GH-101102) 2023-02-28 09:31:01 +01:00
Steven Troxler 0f89acf6cc
gh-101561: Add typing.override decorator (#101564)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-02-27 13:16:11 -08:00
Sergey B Kirpichev 4f3786b761
gh-101773: Optimize creation of Fractions in private methods (#101780)
This PR adds a private `Fraction._from_coprime_ints` classmethod for internal creations of `Fraction` objects, replacing the use of `_normalize=False` in the existing constructor. This speeds up creation of `Fraction` objects arising from calculations. The `_normalize` argument to the `Fraction` constructor has been removed.

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-02-27 18:53:22 +00:00
Dennis Sweeney e3c3f9fec0
gh-102250: Fix double-decref in COMPARE_AND_BRANCH error case (GH-102287) 2023-02-27 10:46:40 +00:00
Rotzbua f3cb15c88a
gh-91038: Change default argument value to `False` instead of `0` (#31621)
The argument is used as a switch and corresponds to a boolean logic. Therefore it is more intuitive to use the corresponding constant `False` as default value instead of the integer `0`.

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-02-26 18:10:34 -08:00
Pradyun Gedam 89d9ff0f48
gh-101997: Update bundled pip version to 23.0.1 (#101998) 2023-02-25 12:00:12 +00:00
Kumar Aditya 5f11478ce7
GH-102126: fix deadlock at shutdown when clearing thread states (#102222) 2023-02-25 12:21:36 +05:30
SKO 56e93c8020
gh-95675: fix uid and gid at test_add_dir_getmember (gh-102207)
Co-authored-by: Seonkyo Ok <seonkyo.ok@linecorp.com>
2023-02-25 11:26:40 +09:00
Ionite 54dfa14c5a
gh-101765: Fix SystemError / segmentation fault in iter `__reduce__` when internal access of `builtins.__dict__` exhausts the iterator (#101769) 2023-02-24 15:02:04 -08:00
Max Bachmann 1fa38906f0
gh-102141: replace use of getpid on Windows with GetCurrentProcessId (GH-102142) 2023-02-24 12:38:21 +00:00
Yeojin Kim 347f7406df
gh-81652: Add MAP_ALIGNED_SUPER FreeBSD and MAP_CONCEAL OpenBSD constants (gh-102191) 2023-02-24 19:26:51 +09:00
Jacob Bower 0c857865e4
Fix deadlock on shutdown if test_current_{exception,frames} fails (#102019)
* Don't deadlock on shutdown if test_current_{exception,frames} fails

These tests spawn a thread that waits on a threading.Event. If the test fails any of its assertions, the Event won't be signaled and the thread will wait indefinitely, causing a deadlock when threading._shutdown() tries to join all outstanding threads.

Co-authored-by: Brett Simmers <bsimmers@meta.com>

* Add a news entry

* Fix whitespace

---------

Co-authored-by: Brett Simmers <bsimmers@meta.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-02-23 14:57:06 -08:00
Zackery Spytz 665730d217
bpo-23224: Fix segfaults and multiple leaks in the lzma and bz2 modules (GH-7822)
lzma.LZMADecompressor and bz2.BZ2Decompressor objects caused
segfaults when their `__init__()` methods were not called.

lzma.LZMADecompressor, lzma.LZMACompressor, bz2.BZ2Compressor,
and bz2.BZ2Decompressor objects would leak locks and internal buffers
when their `__init__()` methods were called multiple times.


https://bugs.python.org/issue23224
2023-02-23 06:00:58 -08:00
Erlend E. Aasland 5b9573eed4
gh-101578: Fixup NEWS and add What's New entry for new exception APIs (#102157) 2023-02-23 13:19:21 +01:00
Mark Shannon 22b8d77b98
GH-100719: Remove redundant `gi_code` field from generator object. (GH-100749) 2023-02-23 10:19:01 +00:00
Irit Katriel 572223f9ce
Revert "bpo-46978: Correct docstrings for in-place builtin operators #31802) (#102146)
Revert "bpo-46978: Correct docstrings for in-place builtin operators (#31802)"

This reverts commit 128379b8cd.
2023-02-23 10:17:44 +00:00
Carl Meyer 056dfc71dc
gh-87634: remove locking from functools.cached_property (GH-101890)
Remove the undocumented locking capabilities of functools.cached_property.
2023-02-22 17:49:22 -08:00
Jonathan Protzenko fcadc7e405
gh-99108: Import MD5 and SHA1 from HACL* (#102089)
Replaces our fallback non-OpenSSL MD5 and SHA1 implementations with those from HACL* as we've already done with SHA2.
2023-02-22 13:18:43 -08:00
Mark Shannon 7c106a443f
GH-100982: Restrict `FOR_ITER_RANGE` to a single instruction to allow instrumentation. (GH-101985) 2023-02-22 11:11:57 +00:00
Eli Schwartz 3ba7743b06
gh-99942: python.pc on android/cygwin should link to libpython per configure.ac (GH-100356)
In commit 254b309c80 a previous change to avoid linking to libpython was partially reverted for Android (and later Cygwin as well), to add back the link flags. This was applied to distutils and to python-config.sh, but not to python.pc.

Add it back to python.pc as well.

Automerge-Triggered-By: GH:gpshead
2023-02-21 17:21:24 -08:00
Hyunkyun Moon d5c7954d0c
gh-95672 fix typo SkitTest to SkipTest (gh-102119)
Co-authored-by: HyunKyun Moon <hyunkyun.moon@linecorp.com>
2023-02-22 02:39:00 +09:00
Vo Hoang Long 0d4c7fcd4f
gh-101936: Update the default value of fp from io.StringIO to io.BytesIO (gh-102100)
Co-authored-by: Long Vo <long.vo@linecorp.com>
2023-02-22 00:14:41 +09:00
Gihwan Kim 6f25657b83
gh-101961 fileinput.hookcompressed should not set the encoding value for the binary mode (gh-102068) 2023-02-21 12:10:29 +09:00
Irit Katriel 022b44f254
gh-102056: Fix a few bugs in error handling of exception printing code (#102078) 2023-02-20 22:16:09 +00:00
Jason R. Coombs 36854bbb24
gh-101566: Sync with zipp 3.14. (GH-102018) 2023-02-20 13:01:58 -08:00
Tim Hatch 59e86caca8
gh-88233: zipfile: handle extras after a zip64 extra (GH-96161)
Previously, any data _after_ the zip64 extra would be removed.

With many new tests.

Fixes #88233

Automerge-Triggered-By: GH:jaraco
2023-02-20 09:07:03 -08:00
Steve Dower a99eb5cd99
gh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (GH-101912)
* gh-101907: Removes use of non-standard C++ extension from Include/cpython/code.h

* Make cases_generator correct on Windows
2023-02-20 14:56:48 +00:00
Mark Dickinson b1b375e267
gh-97786: Fix compiler warnings in pytime.c (#101826)
Fixes compiler warnings in pytime.c.
2023-02-19 17:16:11 -08:00
Mark Dickinson b513c46d99
gh-85417: Clarify behaviour on branch cuts in cmath module (#102046)
This PR updates the cmath module documentation to reflect the reality that Python is almost always (and as far as I can tell, that "almost" can be omitted) running on a machine whose C double supports signed zeros.

* Removes misleading references to functions being continuous from above / below / the left / the right at branch cuts
* Expands the note on branch cuts at the top of the module documentation to explain the double-sided sign-of-zero-based behaviour
2023-02-19 19:15:44 +00:00
Furkan Onder 61f1e67c6f
GH-84783: Make the slice object hashable (GH-101264) 2023-02-18 18:22:02 -06:00
Jason R. Coombs 5170caf305
gh-97930: Apply changes from importlib_resources 5.12. (GH-102010) 2023-02-18 16:29:22 -05:00
Nicko van Someren 128379b8cd
bpo-46978: Correct docstrings for in-place builtin operators (#31802) 2023-02-18 18:44:41 +00:00
Eclips4 89413bbccb
gh-101967: add a missing error check (#101968) 2023-02-18 00:52:23 +00:00
Barney Gale 072011b3c3
gh-100809: Fix handling of drive-relative paths in pathlib.Path.absolute() (GH-100812)
Resolving the drive independently uses the OS API, which ensures it starts from the current directory on that drive.
2023-02-17 14:08:14 +00:00
Barney Gale d401b20630
gh-101360: Fix anchor matching in pathlib.PureWindowsPath.match() (GH-101363)
Use `fnmatch` to match path and pattern anchors, just as we do for other
path parts. This allows patterns such as `'*:/Users/*'` to be matched.
2023-02-17 14:05:38 +00:00
Yeojin Kim 3c0a31cbfd
Docs: fix typos in PyFunction_WatchCallback docs and in 3.12 NEWS (GH-101980)
- possitibility => possibility
- disaallowed => disallowed
2023-02-17 00:47:02 -08:00
Eric Snow 4d8959b73a
gh-101758: Add _PyState_AddModule() Back for the Stable ABI (gh-101956)
We're adding the function back, only for the stable ABI symbol and not as any form of API. I had removed it yesterday.

This undocumented "private" function was added with the implementation for PEP 3121 (3.0, 2007) for internal use and later moved out of the limited API (3.6, 2016) and then into the internal API (3.9, 2019). I removed it completely yesterday, including from the stable ABI manifest (where it was added because the symbol happened to be exported). It's unlikely that anyone is using _PyState_AddModule(), especially any stable ABI extensions built against 3.2-3.5, but we're playing it safe.

https://github.com/python/cpython/issues/101758
2023-02-16 14:05:31 -07:00
Kumar Aditya a5024a261a
GH-96764: rewrite `asyncio.wait_for` to use `asyncio.timeout` (#98518)
Changes `asyncio.wait_for` to use `asyncio.timeout` as its underlying implementation.
2023-02-17 00:18:21 +05:30
Eli Schwartz 226484e475
gh-99942: correct the pkg-config/python-config flags for cygwin/android 2023-02-16 17:57:59 +00:00
Eclips4 68bd8c5e2e
gh-101952: Fix possible segfault in `BUILD_SET` opcode (#101958) 2023-02-16 09:46:43 -08:00
Rayyan Ansari 739c026f44
gh-101881: Support (non-)blocking read/write functions on Windows pipes (GH-101882)
* fileutils: handle non-blocking pipe IO on Windows

Handle erroring operations on non-blocking pipes by reading the _doserrno code.
Limit writes on non-blocking pipes that are too large.

* Support blocking functions on Windows

Use the GetNamedPipeHandleState and SetNamedPipeHandleState Win32 API functions to add support for os.get_blocking and os.set_blocking.
2023-02-16 14:52:24 +00:00
Gregory P. Smith 0b13575e74
gh-99108: Refactor _sha256 & _sha512 into _sha2. (#101924)
This merges their code. They're backed by the same single HACL* static library, having them be a single module simplifies maintenance.

This should unbreak the wasm enscripten builds that currently fail due to linking in --whole-archive mode and the HACL* library appearing twice.

Long unnoticed error fixed: _sha512.SHA384Type was doubly assigned and was actually SHA512Type. Nobody depends on those internal names.

Also rename LIBHACL_ make vars to LIBHACL_SHA2_ in preperation for other future HACL things.
2023-02-15 22:08:20 -08:00
Eric Snow 89ac665891
gh-98627: Add an Optional Check for Extension Module Subinterpreter Compatibility (gh-99040)
Enforcing (optionally) the restriction set by PEP 489 makes sense. Furthermore, this sets the stage for a potential restriction related to a per-interpreter GIL.

This change includes the following:

* add tests for extension module subinterpreter compatibility
* add _PyInterpreterConfig.check_multi_interp_extensions
* add Py_RTFLAGS_MULTI_INTERP_EXTENSIONS
* add _PyImport_CheckSubinterpIncompatibleExtensionAllowed()
* fail iff the module does not implement multi-phase init and the current interpreter is configured to check

https://github.com/python/cpython/issues/98627
2023-02-15 18:16:00 -07:00
Eric Snow b2fc549278
gh-101758: Clean Up Uses of Import State (gh-101919)
This change is almost entirely moving code around and hiding import state behind internal API.  We introduce no changes to behavior, nor to non-internal API.  (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.)  The motivation is to simplify a number of upcoming changes.

Specific changes:

* move existing import-related code to import.c, wherever possible
* add internal API for interacting with import state (both global and per-interpreter)
* use only API outside of import.c (to limit churn there when changing the location, etc.)
* consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly)
* add macros for import state in import.c (to simplify changing the location)
* group code in import.c into sections
*remove _PyState_AddModule()

https://github.com/python/cpython/issues/101758
2023-02-15 15:32:31 -07:00
Erlend E. Aasland c1ce0d178f
gh-99138: Isolate _zoneinfo (#99218)
* Convert zone info type to heap type and add it to module state
* Add global variables to module state
2023-02-15 22:58:48 +01:00
Erlend E. Aasland 8a2b7ee64d
gh-101693: In sqlite3, deprecate using named placeholders with parameters supplied as a sequence (#101698) 2023-02-15 06:27:16 +01:00
Irit Katriel 81e3aa835c
gh-101799: implement PREP_RERAISE_STAR as an intrinsic function (#101800) 2023-02-14 11:54:13 +00:00
Jonathan Protzenko e5da9ab2c8
gh-99108: Import SHA2-384/512 from HACL* (#101707)
Replace the builtin hashlib implementations of SHA2-384 and SHA2-512
originally from LibTomCrypt with formally verified, side-channel resistant
code from the [HACL*](https://github.com/hacl-star/hacl-star/) project.
The builtins remain a fallback only used when OpenSSL does not provide them.
2023-02-14 01:25:16 -08:00
Sam James 8be8101bca
gh-101857: Allow xattr detection on musl libc (#101858)
Previously, we checked exclusively for `__GLIBC__` (AND'd with some other
conditions). Checking for `__linux__` instead should be fine.

This fixes using e.g. `os.listxattr()` on systems using musl libc.

Bug: https://bugs.gentoo.org/894130

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-02-13 23:21:58 -08:00
Radek Smejkal 928752ce4c
gh-74895: getaddrinfo no longer raises OverflowError (#2435)
`socket.getaddrinfo()` no longer raises `OverflowError` based on the **port** argument. Error reporting (or not) for its value is left up to the underlying C library `getaddrinfo()` implementation.
2023-02-13 17:37:34 -08:00
Steve Dower 0c6fe81dce
gh-101849: Add upgrade codes for old versions of launcher that ended up with later version numbers (GH-101877) 2023-02-13 20:33:48 +00:00
Mark Shannon 160f2fe2b9
GH-87849: Simplify stack effect of SEND and specialize it for generators and coroutines. (GH-101788) 2023-02-13 11:24:55 +00:00
Gregory P. Smith dfc2e065a2
gh-89792: Limit test_tools freeze test build parallelism based on the number of cores (#101841)
unhardcode freeze test build parallelism. base it on the number of cpus, don't use more than max(2, os.cpu_count()/3).
2023-02-11 22:07:52 -08:00
Soumendra Ganguly da2fb92643
gh-85984: Utilize new "winsize" functions from termios in pty tests. (#101831)
Utilize new functions termios.tcgetwinsize() and termios.tcsetwinsize in test_pty.py.

Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-02-11 21:24:43 -08:00
Gregory P. Smith 1d194235e4
gh-89792: Prevent test_tools from copying 1000M of "source" in freeze test (#101837)
Prevent test_tools from copying 1000M of "source"
    
It doesn't need a git repo, just the checkout.  We skip .git metadata, Doc/build, Doc/venv, and `__pycache__` subdirs, that developers often have in their clients to reduce the size of the source tree copy ten-fold.

This should significantly reduce IO and presumably time on buildbots during this long test.
2023-02-11 20:54:28 -08:00
Steve Dower e1aadedf09
gh-101763: Update bundled copy of libffi to 3.4.4 on Windows (GH-101784) 2023-02-10 16:57:30 +00:00
Irit Katriel 366b949058
gh-101517: make bdb avoid looking up in linecache with lineno=None (#101787) 2023-02-10 16:49:29 +00:00
Erlend E. Aasland 5d15224011
gh-101759: Update Windows installer to SQLite 3.40.1 (#101762) 2023-02-10 17:38:26 +01:00
Erlend E. Aasland 826bf0e695
gh-101277: Finalise isolating itertools (GH-101305)
Add repeat, islice, chain, tee, teedataobject, and batched types to module state.

Automerge-Triggered-By: GH:erlend-aasland
2023-02-10 03:58:14 -08:00
Erlend E. Aasland d40a23c0a1
gh-101759: Update macOS installer to SQLite 3.40.1 (#101761) 2023-02-10 08:25:02 +01:00
Dong-hee Na 5b946d3719
gh-101430: Update tracemalloc to handle presize properly. (gh-101745) 2023-02-10 08:30:03 +09:00
Gregory P. Smith b41c47cd06
gh-101726: Update the OpenSSL version to 1.1.1t (GH-101727)
Fixes CVE-2023-0286 (High) and a couple of Medium security issues.
https://www.openssl.org/news/secadv/20230207.txt
2023-02-09 17:40:51 +00:00
Mark Shannon ecfd2d37c5
GH-99293: Document that `Py_TPFLAGS_VALID_VERSION_TAG` shouldn't be used. (#GH-101736)
Document that Py_TPFLAGS_VALID_VERSION_TAG shouldn't be used.
2023-02-09 14:05:53 +01:00
Soumendra Ganguly 244d4cd9d2
gh-85984: Remove legacy Lib/pty.py code. (#92365)
Refactored the implementation of pty.fork to use os.login_tty.

A DeprecationWarning is now raised by pty.master_open() and pty.slave_open(). They were
undocumented and deprecated long long ago in the docstring in favor of pty.openpty.

Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-02-08 17:00:17 -08:00
Oleg Iarygin 23751ed826
gh-101283: Improved fallback logic for subprocess with shell=True on Windows (GH-101286) 2023-02-08 22:12:19 +00:00
Kumar Aditya d9de079248
GH-101696: invalidate type version tag in `_PyStaticType_Dealloc` (#101697) 2023-02-08 23:32:15 +05:30
Michał Górny 2a8bf25804
gh-100221: Fix creating dirs in `make sharedinstall` (GH-100329)
Fix creating install directories in `make sharedinstall` if they exist already outside `DESTDIR`.  The previous make rules assumed that the directories would be created via a dependency on a rule for `$(DESTSHARED)` that did not fire if the directory did exist outside `$(DESTDIR)`.

While technically `$(DESTDIR)` could be prepended to the rule name, moving the rules for creating directories straight into the `sharedinstall` rule seems to fit the common practices better. Since the rule explicitly checks whether the individual directories exist anyway, there seems to be no reason to rely on make determining that implicitly as well.
2023-02-08 08:50:43 -08:00
Michael Droettboom 86ebd5c3fa
gh-101196: Make isdir/isfile/exists faster on Windows (GH-101324)
Co-authored-by: Eryk Sun <eryksun@gmail.com>
2023-02-08 14:34:24 +00:00
David Hewitt 3a88de7a0a
gh-101614: Don't treat python3_d.dll as a Python DLL when checking extension modules for incompatibility (GH-101615) 2023-02-08 14:23:57 +00:00
Mark Shannon feec49c407
GH-101578: Normalize the current exception (GH-101607)
* Make sure that the current exception is always normalized.

* Remove redundant type and traceback fields for the current exception.

* Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException

* Add new API functions: PyException_GetArgs, PyException_SetArgs
2023-02-08 09:31:12 +00:00
Nikita Sobolev 790ff6bc6a
gh-101446: Change `repr` of `collections.OrderedDict` (#101661) 2023-02-07 20:01:10 -06:00
penguin_wwy 753fc8a5d6
gh-101632: Add the new RETURN_CONST opcode (#101633) 2023-02-07 22:32:21 +00:00
Thomas Wouters 0d3d5007b1 Merge branch 'main' of https://github.com/python/cpython into main 2023-02-07 23:13:53 +01:00
Oleg Iarygin f87f6e2396
gh-97725: Fix documentation for the default file of `asyncio.Task.print_stack` (#101652) 2023-02-07 23:34:31 +05:30
Thomas Wouters 3c67ec394f Python 3.12.0a5 2023-02-07 13:21:15 +01:00
Łukasz Langa 7990324048
[gh-101072] Fix Blurb for GH-101127 2023-02-07 10:50:39 +01:00
Matthieu Dartiailh ae62bddaf8
gh-101072: support default and kw default in PyEval_EvalCodeEx for 3.11+ (#101127)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-02-07 10:34:21 +01:00
Jonathan Protzenko 1fcc0efdaa
gh-99108: Replace SHA2-224 & 256 with verified code from HACL* (#99109)
replacing hashlib primitives (for the non-OpenSSL case) with verified implementations from HACL*. This is the first PR in the series, and focuses specifically on SHA2-256 and SHA2-224.

This PR imports Hacl_Streaming_SHA2 into the Python tree. This is the HACL* implementation of SHA2, which combines a core implementation of SHA2 along with a layer of buffer management that allows updating the digest with any number of bytes. This supersedes the previous implementation in the tree.

@franziskuskiefer was kind enough to benchmark the changes: in addition to being verified (thus providing significant safety and security improvements), this implementation also provides a sizeable performance boost!

```
---------------------------------------------------------------
Benchmark                     Time             CPU   Iterations
---------------------------------------------------------------
Sha2_256_Streaming            3163 ns      3160 ns       219353     // this PR
LibTomCrypt_Sha2_256          5057 ns      5056 ns       136234     // library used by Python currently
``` 

The changes in this PR are as follows:
- import the subset of HACL* that covers SHA2-256/224 into `Modules/_hacl`
- rewire sha256module.c to use the HACL* implementation

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-02-06 18:11:01 -08:00
Eric Snow 132b3f8302
gh-59956: Partial Fix for GILState API Compatibility with Subinterpreters (gh-101431)
The GILState API (PEP 311) implementation from 2003 made the assumption that only one thread state would ever be used for any given OS thread, explicitly disregarding the case of subinterpreters.  However, PyThreadState_Swap() still facilitated switching between subinterpreters, meaning the "current" thread state (holding the GIL), and the GILState thread state could end up out of sync, causing problems (including crashes).

This change addresses the issue by keeping the two in sync in PyThreadState_Swap().  I verified the fix against gh-99040.

Note that the other GILState-subinterpreter incompatibility (with autoInterpreterState) is not resolved here.

https://github.com/python/cpython/issues/59956
2023-02-06 14:39:25 -07:00
Eclips4 b96b344f25
gh-101562: typing: add tests for inheritance with NotRequired & Required in parent fields (#101563) 2023-02-06 11:28:24 -08:00
Steve Dower 7a253103d4
gh-101543: Ensure Windows registry path is only used when stdlib can't be found (GH-101544) 2023-02-06 15:55:32 +00:00
Dong-hee Na 9ef7e75434
gh-101372: Fix unicodedata.is_normalized to properly handle the UCD 3… (gh-101388) 2023-02-06 13:58:00 +09:00
Ethan Furman ef7c2bfcf1
gh-101541: [Enum] create flag psuedo-member without calling original __new__ (GH-101590) 2023-02-05 19:29:06 -08:00
Gregory P. Smith ffcb8220d7
gh-101334: Don't force USTAR format in test_tarfile. (GH-101572)
That causes the test to fail when run using a high UID as that ancient format
cannot represent it. The current default (PAX) and the old default (GNU) both
support high UIDs.
2023-02-05 09:44:57 -08:00
mrh1997 f7e9fbacb2
bpo-33591: Add support for path like objects to `ctypes.CDLL` (#7032)
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-02-05 23:06:57 +05:30
Eric Wieser 90d85a9b41
gh-76961: Fix the PEP3118 format string for ctypes.Structure (#5561)
The summary of this diff is that it:

* adds a `_ctypes_alloc_format_padding` function to append strings like `37x` to a format string to indicate 37 padding bytes
* removes the branches that amount to "give up on producing a valid format string if the struct is packed"
* combines the resulting adjacent `if (isStruct) {`s now that neither is `if (isStruct && !isPacked) {`
* invokes `_ctypes_alloc_format_padding` to add padding between structure fields, and after the last structure field. The computation used for the total size is unchanged from ctypes already used.

This patch does not affect any existing aligment computation; all it does is use subtraction to deduce the amount of paddnig introduced by the existing code.

---

Without this fix, it would never include padding bytes - an assumption that was only
valid in the case when `_pack_` was set - and this case was explicitly not implemented.

This should allow conversion from ctypes structs to numpy structs

Fixes https://github.com/numpy/numpy/issues/10528
2023-02-05 17:10:53 +00:00
Mark Dickinson 0672a6c23b
Revert "gh-89381: Fix invalid signatures of math/cmath.log (#101404)" (#101580)
This reverts commit 0ef92d9793.
2023-02-05 16:36:33 +00:00
Pradyun Gedam 19ac43629e
gh-101570: Update bundled pip version to 23.0 (#101571)
Update bundled pip version to 23.0

This is the current latest version of `pip`.

---------

Co-authored-by: Pradyun Gedam <pradyunsg@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-02-05 11:30:44 +00:00
Mark Dickinson 39017e04b5
gh-101266: Fix __sizeof__ for subclasses of int (#101394)
Fix the behaviour of the `__sizeof__` method (and hence the results returned by `sys.getsizeof`) for subclasses of `int`. Previously, `int` subclasses gave identical results to the `int` base class, ignoring the presence of the instance dictionary.

<!-- gh-issue-number: gh-101266 -->
* Issue: gh-101266
<!-- /gh-issue-number -->
2023-02-05 10:02:53 +00:00
Ruben Vorderman a89e6713c4
gh-101322: Ensure test_zlib.ZlibDecompressorTest runs, fix errors in ZlibDecompressor (#101323)
* Ensure test_zlib.ZlibDecompressorTest actually runs, fix errors in ZlibDecompressor.
2023-02-04 12:07:30 -08:00
Dong-hee Na 144aaa74bb
gh-101282: Update BOLT --split-functions flag not to use deprecated u… (gh-101557)
gh-101282: Update BOLT --split-functions flag not to use deprecated usage
2023-02-04 16:55:31 +09:00
Gregory P. Smith d4c410f0f9
gh-84559: Remove the new multiprocessing warning, too disruptive. (#101551)
This reverts the core of #100618 while leaving relevant documentation
improvements and minor refactorings in place.
2023-02-03 15:20:46 -08:00
Steve Dower f6c53b80a1
gh-101522: Allow overriding Windows dependencies versions and paths using MSBuild properties (GH-101523) 2023-02-03 18:08:34 +00:00
Gregory P. Smith 0ca67e6313
GH-84559: Deprecate fork being the multiprocessing default. (#100618)
This starts the process. Users who don't specify their own start method
and use the default on platforms where it is 'fork' will see a
DeprecationWarning upon multiprocessing.Pool() construction or upon
multiprocessing.Process.start() or concurrent.futures.ProcessPool use.

See the related issue and documentation within this change for details.
2023-02-02 15:50:35 -08:00
Ayappan Perumal 618b7a8260
gh-98705: Fix AIX build by undefining `__bool__` in C (#98768) 2023-02-02 12:32:33 -08:00
Ayappan Perumal ba4731d149
gh-96305: Fix AIX build by avoiding subprocess during bootstrap (#96429)
* Fix AIX build by avoiding `subprocess` during bootstrap.
2023-02-02 12:30:49 -08:00
Steve Dower eda60916bc
gh-101467: Correct py.exe handling of prefix matches and cases when only one runtime is installed (GH-101468) 2023-02-01 21:06:56 +00:00
beavailable cc407b9de6
gh-101317: Add `ssl_shutdown_timeout` parameter for `asyncio.StreamWriter.start_tls` (#101335)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-02-01 16:33:59 +05:30
Dong-hee Na e867c1b753
gh-101400: Fix incorrect lineno in exception message on continue/break which are not in a loop (#101413) 2023-01-30 23:33:54 +00:00
Mark Shannon c1b1f51cd1
GH-101291: Refactor the `PyLongObject` struct into object header and PyLongValue struct. (GH-101292) 2023-01-30 10:03:04 +00:00
Sergey B Kirpichev 0ef92d9793
gh-89381: Fix invalid signatures of math/cmath.log (#101404) 2023-01-29 11:50:10 -08:00
Gregory P. Smith 052f53d65d
gh-39615: Add warnings.warn() skip_file_prefixes support (#100840)
`warnings.warn()` gains the ability to skip stack frames based on code
filename prefix rather than only a numeric `stacklevel=` via a new
`skip_file_prefixes=` keyword argument.
2023-01-27 18:35:14 -08:00
Steve Dower 737d367b1f
gh-77532: Minor tweaks to allow compiling with PlatformToolset=ClangCL on Windows (GH-101352)
To use this, ensure that clang support was selected in Visual Studio Installer, then set the PlatformToolset environment variable to "ClangCL" and build as normal from the command line.
It remains unsupported, but at least is possible now for experimentation.
2023-01-27 14:45:08 +00:00
Barney Gale e5b08ddddf
gh-101000: Add os.path.splitroot() (#101002)
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-01-27 00:28:27 +00:00
Steve Dower 8d18d1ffd5
gh-99834: Update bundled copy of Tcl/Tk to 8.6.13.0 on Windows (GH-101307) 2023-01-26 20:47:24 +00:00
Oleg Iarygin 409f5337a3
gh-60580: Fix a wrong type of `ctypes.wintypes.BYTE` (#97579)
Created from a patch file attached to an issue, by Anatoly Techtonik.
2023-01-26 18:16:27 +04:00
Yukihiro Nakadaira dfad678d70
gh-99952: [ctypes] fix refcount issues in from_param() result. (#100169)
Fixes a reference counting issue with `ctypes.Structure` when a `from_param()` method call is used and the structure size is larger than a C pointer `sizeof(void*)`.

This problem existed for a very long time, but became more apparent in 3.8+ by change likely due to garbage collection cleanup timing changes.
2023-01-26 00:28:34 -08:00
Oleg Iarygin 73245d084e
gh-94518: Rename `group*` to `extra_group*` to avoid confusion (#101054)
* Rename `group*` to `extra_group*` to avoid confusion
* Rename `num_groups` into `extra_group_size`
* Rename `groups_list` to `extra_groups_packed`
2023-01-25 22:50:33 -08:00
Shantanu a178ba82bf
gh-101326: Fix regression when passing None to FutureIter.throw (#101327) 2023-01-25 12:01:01 -08:00
Mark Shannon f02fa64bf2
GH-100762: Don't call `gen.throw()` in `gen.close()`, unless necessary. (GH-101013)
* Store exception stack depth in YIELD_VALUE's oparg and use it avoid expensive gen.throw() in gen.close() where possible.
2023-01-24 17:25:37 +00:00
Martin Boisvert daec3a463c
gh-101135: Add backwards compatibility to Windows launcher for older 32-bit versions (GH-101138)
Python 2.x and up to 3.4 did not contain the "-32" in their registry name, so the 32 and 64-bit installs were treated equal. Since 3.5/PEP 514 this is no longer true, but we still want to detect the EOL versions correctly in case people are still using them.

Additionally, the code to replace a node with one with a lower sort key was buggy (wrong node chosen, replace never happened since parent was always NULL, replaced node never freed, etc)
2023-01-24 16:35:16 +00:00
Erlend E. Aasland fee7a995a1
gh-92123: Adapt _elementtree to multi-phase init (PEP 489) (#101285) 2023-01-24 17:00:24 +01:00
Gregory Szorc 7589d713a1
gh-101060: conditionally add `-fno-reorder-blocks-and-partition` (gh-101061) 2023-01-24 18:34:44 +09:00
ram vikram singh 7f95ec3e74
gh-101152: Implement PEP 699 (GH-101193)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Ken Jin <kenjin@python.org>
2023-01-24 17:29:22 +08:00
Gregory P. Smith b724ac2fe7
gh-100795: Don't call freeaddrinfo on failure. (#101252)
When getaddrinfo returns an error, the output pointer is in an unknown state
Don't call freeaddrinfo on it.  See the issue for discussion and details with
links to reasoning.  _Most_ libc getaddrinfo implementations never modify the
output pointer unless they are returning success.

Co-authored-by: Sergey G. Brester <github@sebres.de>
Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2023-01-23 15:03:26 -08:00
Nikita Sobolev 807d6b576f
gh-101015: Fix `typing.get_type_hints` with unpacked `*tuple` (PEP 646) (#101031) 2023-01-23 07:52:58 +00:00
Jacob Walls d717be04dc
gh-83122: Deprecate testing element truth values in `ElementTree` (#31149)
When testing element truth values, emit a DeprecationWarning in all implementations.

This had emitted a FutureWarning in the rarely used python-only implementation since ~2.7 and has always been documented as a behavior not to rely on.

Matching an element in a tree search but having it test False can be unexpected. Raising the warning enables making the choice to finally raise an exception for this ambiguous behavior in the future.
2023-01-22 17:16:48 -08:00
Mark Dickinson 3e09f3152e
gh-67790: Support float-style formatting for Fraction instances (#100161)
This PR adds support for float-style formatting for `Fraction` objects: it supports the `"e"`, `"E"`, `"f"`, `"F"`, `"g"`, `"G"` and `"%"` presentation types, and all the various bells and whistles of the formatting mini-language for those presentation types. The behaviour almost exactly matches that of `float`, but the implementation works with the exact `Fraction` value and does not do an intermediate conversion to `float`, and so avoids loss of precision or issues with numbers that are outside the dynamic range of the `float` type.

Note that the `"n"` presentation type is _not_ supported. That support could be added later if people have a need for it.

There's one corner-case where the behaviour differs from that of float: for the `float` type, if explicit alignment is specified with a fill character of `'0'` and alignment type `'='`, then thousands separators (if specified) are inserted into the padding string:

```python
>>> format(3.14, '0=11,.2f')
'0,000,003.14'
```

The exact same effect can be achieved by using the `'0'` flag:

```python
>>> format(3.14, '011,.2f')
'0,000,003.14'
```

For `Fraction`, only the `'0'` flag has the above behaviour with respect to thousands separators: there's no special-casing of the particular `'0='` fill-character/alignment combination. Instead, we treat the fill character `'0'` just like any other:

```python
>>> format(Fraction('3.14'), '0=11,.2f')
'00000003.14'
>>> format(Fraction('3.14'), '011,.2f')
'0,000,003.14'
```

The `Fraction` formatter is also stricter about combining these two things: it's not permitted to use both the `'0'` flag _and_ explicit alignment, on the basis that we should refuse the temptation to guess in the face of ambiguity. `float` is less picky:

```python
>>> format(3.14, '0<011,.2f')
'3.140000000'
>>> format(Fraction('3.14'), '0<011,.2f')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mdickinson/Repositories/python/cpython/Lib/fractions.py", line 414, in __format__
    raise ValueError(
ValueError: Invalid format specifier '0<011,.2f' for object of type 'Fraction'; can't use explicit alignment when zero-padding
```
2023-01-22 18:44:49 +00:00
Kumar Aditya b53bad6dd0
Revert "gh-100795: avoid unexpected `freeaddrinfo` after failed `getaddrinfo` (#101220)" (#101238)
Revert "gh-100795: avoid unexpected `freeaddrinfo` after failed `getaddrinfo` (#101220)"

This reverts commit 5f08fe4a2c.
2023-01-22 14:17:01 +05:30
Sergey G. Brester 5f08fe4a2c
gh-100795: avoid unexpected `freeaddrinfo` after failed `getaddrinfo` (#101220)
Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2023-01-22 13:40:00 +05:30
achhina 95f5b05a8c
GH-88597: Added command line interface to UUID module. (#99463)
The `uuid` module now supports command line usage.

```python
❯ ./python.exe -m uuid             
5f2d57b1-90e8-417c-ba5d-69b9b6f74289

❯ ./python.exe -m uuid -h          
usage: uuid.py [-h] [-u {uuid1,uuid3,uuid4,uuid5}] [-ns NAMESPACE] [-n NAME]
...
```
2023-01-21 22:59:31 -08:00
Pieter Eendebak f63f525e16
gh-100726: Optimize construction of range object for medium sized integers (#100810)
Use C long arithmetic instead of PyLong arithmetic to compute the range length, where possible.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-01-21 19:33:08 +00:00
Kamil Turek b4e11a7985
gh-99266: ctypes: Preserve more detailed exception in `ArgumentError`
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-01-21 19:14:43 +05:30
Mark Dickinson 401fdf9c85
gh-101037: Fix potential memory underallocation for zeros of int subtypes (#101038)
This PR fixes object allocation in long_subtype_new to ensure that there's at least one digit in all cases, and makes sure that the value of that digit is copied over from the source long.

Needs backport to 3.11, but not any further: the change to require at least one digit was only introduced for Python 3.11.

Fixes #101037.
2023-01-21 10:23:59 +00:00
J. Nick Koston 9e947675ae
gh-101143: Remove references to `TimerHandle` from `asyncio.base_events.BaseEventLoop._add_callback` (#101197)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-01-21 14:46:07 +05:30
Jean-Paul Calderone 3325f054e3
gh-91351: Fix some bugs in importlib handling of re-entrant imports (GH-94504)
Co-authored-by: Brett Cannon <brett@python.org>
2023-01-20 16:00:39 -08:00
Thomas Grainger 6b3993c556
gh-100750: pass encoding kwarg in lib/platform.py (#100751)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2023-01-20 23:25:38 +01:00
Erlend E. Aasland 3847a6c64b
gh-92123: Convert `_elementtree` types to heap types (#99221) 2023-01-20 12:40:06 +01:00
richardhob 3fa8fe7177
gh-88324: Clarify documentation for redirected stdout/stderr when using subprocess in Linux (#94035)
* Update description of stdout, stderr, and stdin.

Changes:
- Move the ``None`` option (which is default) to the front of the list
  of input options
- Move the ``None`` option description up to make the default behavior
  more clear (No redirection)
- Remove mention of Child File Descriptors from ``None`` option description
2023-01-19 23:56:13 -08:00
Gregory P. Smith 5927013e47
gh-101144: Allow open and read_text encoding to be positional. (#101145)
The zipfile.Path open() and read_text() encoding parameter can be supplied as a positional argument without causing a TypeError again. 3.10.0b1 included a regression that made it keyword only.

Documentation update included as users writing code to be compatible with a wide range of versions will need to consider this for some time.
2023-01-19 23:04:30 -08:00
Irit Katriel e9ccfe4a63
gh-100712: make it possible to disable specialization (for debugging) (#100713) 2023-01-19 18:14:55 +00:00
Kushal Das 75c8133efe
gh-100340: Allows -Wno-int-conversion for wasm (#100341)
Fixes #100340 allows -Wno-int-conversion for wasm
2023-01-18 19:43:49 +01:00
Paul Moore f34176b77f
gh-82052: Don't send partial UTF-8 sequences to the Windows API (GH-101103)
Don't send partial UTF-8 sequences to the Windows API
2023-01-17 19:23:06 +00:00
Pablo Galindo Salgado 1de4395f62
gh-101046: Fix a potential memory leak in the parser when raising MemoryError (#101051) 2023-01-16 18:45:37 +00:00
Steve Dower df10571a13
gh-100320: Fix path calculations on Windows when python.exe is moved outside of the normal location (GH-100947) 2023-01-16 16:05:39 +00:00
Mark Shannon 7b14c2ef19
GH-100982: Add `COMPARE_AND_BRANCH` instruction (GH-100983) 2023-01-16 12:35:21 +00:00
Oleg Iarygin 124af17b6e
gh-94518: [_posixsubprocess] Replace variable validity flags with reserved values (#94687)
Have _posixsubprocess.c stop using boolean flags to say if gid and uid values were supplied and action is required.  Such an implicit "either initialized or look somewhere else" confused both the reader (another mental connection to constantly track between functions) and a compiler (warnings on potentially uninitialized variables being passed). Instead, we can utilize a special group/user id as a flag value -1 defined by POSIX but used nowhere else. Namely:

gid: call_setgid = False → gid = -1

uid: call_setuid = False → uid = -1

groups: call_setgroups = False → groups = NULL (obtained with (groups_list != Py_None) ? groups : NULL)

This PR is required for #94519.
2023-01-14 12:11:04 -08:00
Yurii Karabas 080cb27829
gh-74033: Fix bug when Path takes and ignores **kwargs (GH-19632)
Fix a bug where `Path` takes and ignores `**kwargs` by adding to `PurePath`  class `__init__` method which can take only positional arguments.

Automerge-Triggered-By: GH:brettcannon
2023-01-13 16:05:43 -08:00
Guido van Rossum 1bc7a73683
GH-100573: Fix server hang caused by os.stat() on named pipe (Windows) (#100959) 2023-01-13 13:24:57 -08:00
Serhiy Storchaka e5bd5ad70d
gh-100160: Restore and deprecate implicit creation of an event loop (GH-100410)
Partially revert changes made in GH-93453.

asyncio.DefaultEventLoopPolicy.get_event_loop() now emits a
DeprecationWarning and creates and sets a new event loop instead of
raising a RuntimeError if there is no current event loop set.

Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2023-01-13 14:40:29 +02:00
Steve Dower 468c3bf798
gh-100247: Fix py.exe launcher not using entire shebang command for finding custom commands (GH-100944) 2023-01-13 11:49:01 +00:00
Steve Dower b5d4347950
gh-86682: Adds sys._getframemodulename as an alternative to using _getframe (GH-99520)
Also updates calls in collections, doctest, enum, and typing modules to use _getframemodulename first when available.
2023-01-13 11:31:06 +00:00
Raymond Hettinger 94fc7706b7
GH-100942: Fix incorrect cast in property_copy(). (#100965) 2023-01-12 16:13:56 -06:00
Barney Gale 005e69403d
gh-96290: Support partial/invalid UNC drives in ntpath.normpath() and splitdrive() (GH-100351)
This brings the Python implementation of `ntpath.normpath()` in line with the C implementation added in 99fcf15

Co-authored-by: Eryk Sun <eryksun@gmail.com>
2023-01-12 19:24:57 +00:00
Michał Górny 02a72f080d
gh-98636: Fix detecting gdbm_compat for _dbm module (#98643)
Fix the gdbm_compat library detection logic to actually check for
-lgdbm_compat independently of the ndbm detection.
This fixes the build failure with `--with-dbmliborder=gdbm`,
and implicit fallback to ndbm with the default value.
2023-01-11 22:46:28 +01:00
Mark Shannon 6e4e14d98f
GH-100923: Embed jump mask in `COMPARE_OP` oparg (GH-100924) 2023-01-11 20:40:43 +00:00
Kumar Aditya 762745a124
GH-100892: Fix race in clearing `threading.local` (#100922) 2023-01-11 16:03:31 +05:30
Thomas Wouters 3d5d3f7af6 Python 3.12.0a4 2023-01-10 13:09:15 +01:00
Brandt Bucher f07daaf4f7
GH-100117: Make `co_lines` more efficient (GH-100447) 2023-01-10 10:56:53 +00:00
Ned Deily 57a5259a43
gh-100180: Update macOS installer to OpenSSL 1.1.1s (GH-100908) 2023-01-10 00:10:31 -05:00
Steve Dower d7ab7149f8
gh-100180: Update Windows installer to OpenSSL 1.1.1s (GH-100901) 2023-01-09 23:50:59 +00:00
Brandt Bucher 61762b9387
GH-100126: Skip incomplete frames in more places (GH-100613) 2023-01-09 12:20:04 -08:00
C.A.M. Gerlach f08209874e
gh-99191: Use correct check for MSVC C++ version support in _wmimodule.cpp (GH-100381) 2023-01-09 17:48:24 +00:00
dgelessus 837ba05267
GH-81061: Fix refcount issue when returning `None` from a `ctypes.py_object` callback (#13364) 2023-01-09 15:43:04 +05:30
dsentinel 7a50d6b5b0
GH-100813: Add `socket.IP_PKTINFO` constant (#10294) 2023-01-09 14:30:40 +05:30
Nikita Sobolev bc0a686f82
gh-87447: Fix walrus comprehension rebind checking (#100581)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2023-01-08 15:51:29 -07:00
Raymond Hettinger 9a68ff12c3
GH-100805: Support numpy.array() in random.choice(). (GH-100830) 2023-01-08 13:40:35 -06:00
Mark Dickinson 87d3bd0e02
gh-100833: Remove 'volatile' qualifiers in fsum algorithm (#100845)
This PR removes the `volatile` qualifier on various intermediate quantities
in the `math.fsum` implementation, and updates the notes preceding the
algorithm accordingly (as well as fixing some of the exsting notes). See
the linked issue #100833 for discussion.
2023-01-08 19:40:15 +00:00
Nikita Sobolev b034fd3e59
gh-100689: Revert "bpo-41798: pyexpat: Allocate the expat_CAPI on the heap memory (GH-24061)" (#100745)
* gh-100689: Revert "bpo-41798: pyexpat: Allocate the expat_CAPI on the heap memory (GH-24061)"

This reverts commit 7c83eaa536.
2023-01-08 18:24:40 +05:30
Sergey B Kirpichev 909982e82a
gh-91851: Micro optimizations for arithmetic between Fractions (#25518)
Adapted from
https://github.com/python/cpython/pull/24779/commits/046c84e8f9

This makes arithmetic between Fractions with small components
just as fast as before python/cpython#24779, at some expense of
mixed arithmetic (e.g. Fraction + int).
2023-01-08 00:34:20 -08:00
Shantanu a2141882f2
gh-100776: Fix misleading default value in help(input) (#100788) 2023-01-08 13:27:41 +05:30
Nikita Sobolev 6746135b07
gh-100792: Make `email.message.Message.__contains__` twice as fast (#100793) 2023-01-07 13:26:05 -08:00
Raymond Hettinger 47b9f83a83
GH-100485: Add math.sumprod() (GH-100677) 2023-01-07 12:46:35 -06:00
Nikita Sobolev 9e7d7266ec
gh-96127: Fix `inspect.signature` call on mocks (#96335) 2023-01-07 10:49:15 +00:00
Christian Klein 1d4d677d1c
gh-100690: Raise an AttributeError when the assert_ prefix is forgotten when using Mock (#100691)
Mock objects which are not unsafe will now raise an AttributeError when accessing an
attribute that matches the name of an assertion but without the prefix `assert_`, e.g. accessing `called_once` instead of `assert_called_once`.

This is in addition to this already happening for accessing attributes with prefixes assert, assret, asert, aseert, and assrt.
2023-01-06 18:38:50 +00:00
Sergey B Kirpichev 0e640260da
gh-91851: Trivial optimizations in Fraction (#100791)
Make some trivial performance optimizations in Fraction

Uses private class attributes `_numerator` and `_denominator` in place of the `numerator` and `denominator` property accesses.

Co-authored-by: hauntsaninja <hauntsaninja@gmail.com>
2023-01-06 15:37:34 +00:00
Mark Shannon 78068126a1
GH-99005: More intrinsics (GH-100774)
* Remove UNARY_POSITIVE, LIST_TO_TUPLE and ASYNC_GEN_WRAP, replacing them with intrinsics.
2023-01-06 14:47:57 +00:00
Carl Meyer 0a7936a38f
gh-90104: avoid RecursionError on recursive dataclass field repr (gh-100756)
Avoid RecursionError on recursive dataclass field repr
2023-01-05 19:19:40 -05:00
Dustin Spicuzza cc8748712e
gh-86082: bpo-41916: allow cross-compiled python to have -pthread set for CXX (#22525)
When cross-compiling, the compile/run test for -pthread always fails so -pthread
will never be automatically set without an override from the cache. ac_cv_pthread
can already be overridden, so do the same thing for ac_cv_cxx_thread.
2023-01-05 14:57:31 -08:00
Barney Gale 7fba99eadb
gh-100562: improve performance of `pathlib.Path.absolute()` (GH-100563)
Increase performance of the `absolute()` method by calling `os.getcwd()` directly, rather than using the `Path.cwd()` class method. This avoids constructing an extra `Path` object (and the parsing/normalization that comes with it).

Decrease performance of the `cwd()` class method by calling the `Path.absolute()` method, rather than using `os.getcwd()` directly. This involves constructing an extra `Path` object. We do this to maintain a longstanding pattern where `os` functions are called from only one place, which allows them to be more readily replaced by users. As `cwd()` is generally called at most once within user programs, it's a good bargain.

```shell
# before
$ ./python -m timeit -s 'from pathlib import Path; p = Path("foo", "bar")' 'p.absolute()'
50000 loops, best of 5: 9.04 usec per loop
# after
$ ./python -m timeit -s 'from pathlib import Path; p = Path("foo", "bar")' 'p.absolute()'
50000 loops, best of 5: 5.02 usec per loop
```

Automerge-Triggered-By: GH:AlexWaygood
2023-01-05 14:11:50 -08:00
Mark Shannon 28187141cc
GH-99005: Add `CALL_INTRINSIC_1` instruction (GH-100771)
* Remove PRINT_EXPR instruction

* Remove STOPITERATION_ERROR instruction

* Remove IMPORT_STAR instruction
2023-01-05 16:05:51 +00:00
Mark Shannon f20c553a45
GH-100288: Remove LOAD_ATTR_METHOD_WITH_DICT instruction. (GH-100753) 2023-01-05 12:20:09 +00:00
Christian Klein 7f1eefc6f4
gh-100739: Respect mock spec when checking for unsafe prefixes (#100740)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2023-01-04 22:31:29 +00:00
Mark Shannon 15aecf8dd7
GH-100719: Remove the `co_nplaincellvars` field from code objects. (GH-100721) 2023-01-04 15:41:39 +00:00
Irit Katriel c31e356a10
gh-100720: refactor calculation of number of frame slots for a code object into the new function _PyFrame_NumSlotsForCodeObject (#100722) 2023-01-04 13:37:06 +00:00
Kumar Aditya ba8dcdbcab
GH-86508: skip binding to local addresses of different family in `asyncio.open_connection` (#100615) 2023-01-04 13:30:26 +05:30
Ethan Furman a286caa937
gh-91219: http - use subclassing to override index_pages attribute (GH-100731)
Remove previously added parameter to `__init__`, and recommend subclassing to modify the `index_pages` attribute instead.
2023-01-03 15:20:08 -08:00
L. A. F. Pereira e6d4440782
gh-100146: Steal references from stack when building a list (#100147)
When executing the BUILD_LIST opcode, steal the references from the stack,
in a manner similar to the BUILD_TUPLE opcode.  Implement this by offloading
the logic to a new private API, _PyList_FromArraySteal(), that works similarly
to _PyTuple_FromArraySteal().

This way, instead of performing multiple stack pointer adjustments while the
list is being initialized, the stack is adjusted only once and a fast memory
copy operation is performed in one fell swoop.
2023-01-03 10:49:49 -08:00
Thomas Grainger b3722ca058
gh-95882: fix regression in the traceback of exceptions propagated from inside a contextlib context manager (#95883) 2023-01-03 15:47:13 +00:00
Eli Schwartz 8586949833
gh-89419: gdb: fix bug causing `AttributeError` in py-locals when no frame is available (#100611) 2023-01-03 20:26:57 +05:30
Stanislav Zmiev 73097d91a6
gh-89727: Improve os.walk complexity (#100671) 2023-01-02 13:41:19 -08:00
Ionite d7e7f79ca7
gh-100637: Fix int and bool __sizeof__ calculation to include the 1 element ob_digit array for 0 and False (#100663)
Fixes behaviour where int (and subtypes like bool) __sizeof__ under-reports true size as it did not take into account the size 1 `ob_digit` array for the zero int.

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-01-02 21:11:49 +00:00
Jason R. Coombs 447d061bc7
gh-97930: Apply changes from importlib_resources 5.10. (GH-100598) 2023-01-01 11:07:32 -05:00
Gabriele N. Tornetta d52d4942cf
gh-100649: Update native_thread_id after fork (gh-100650)
Update native_thread_id after fork
2023-01-01 23:20:38 +09:00
Shantanu e83f88a455
gh-100488: Add is_integer method to fractions.Fraction (#100489) 2023-01-01 01:44:48 -07:00
Éric 46521826cb
gh-95778: add doc missing in some places (GH-100627) 2022-12-30 13:21:15 -08:00
mathieui f4fcfdf8c5
gh-100616: Document 'attr' parameter for window.vline() in curses module (#24961)
Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
2022-12-30 08:35:04 -08:00
Nikita Sobolev 79c10b7da8
gh-99433: Fix `doctest` failure on `types.MethodWrapperType` (#99434) 2022-12-30 17:25:01 +05:30
Gregory P. Smith 894f2c3c16
gh-100228: Warn from os.fork() if other threads exist. (#100229)
Not comprehensive, best effort warning. There are cases when threads exist on some platforms that this code cannot detect. macOS when API permissions allow and Linux with a readable /proc procfs present are the currently supported cases where a warning should show up reliably.

Starting with a DeprecationWarning for now, it is less disruptive than something like RuntimeWarning and most likely to only be seen in people's CI tests - a good place to start with this messaging.
2022-12-29 14:41:39 -08:00
Zachary Ware 2df82db485
gh-100540: Remove obsolete Modules/_ctypes/darwin/ dlfcn shim (GH-100541)
As far as I can tell, this hasn't been actually used since Mac OS X 10.2.
2022-12-29 16:13:28 -06:00
Samet YASLAN f10f503b24
gh-100585: Fixed a bug where importlib.resources.as_file was leaving file pointers open (GH-100586)
* gh-100585: Fixed open fp bug in the imporlib module

* Added news for gh-100585
2022-12-28 16:31:43 -05:00
Hugo van Kemenade 9eca7235af
gh-94172: Update docs for params removed in 3.12 (#100431)
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
2022-12-28 14:43:19 -05:00
Zachary Ware 25590eb5de
gh-100540: Remove obsolete '--with-system-ffi' configure option (GH-100544)
It has had no effect on non-macOS platforms for a long time, and has had
the non-obvious effect of invoking `pkg_config` and not setting
`-DUSING_APPLE_OS_LIBFFI` on macOS since GH-22855.
2022-12-28 13:36:03 -06:00
Kumar Aditya 7cf164ad5e
GH-100342: check for allocation failure in AC `*args` parsing (#100343) 2022-12-28 09:46:28 +05:30
Pieter Eendebak b3da698952
gh-94603: micro optimize list.pop (gh-94604) 2022-12-27 19:55:54 +09:00
Nikita Sobolev ce39aaffee
gh-99509: Add `__class_getitem__` to `multiprocessing.queues.Queue` (#99511) 2022-12-26 20:50:55 -08:00
Jakub Kuczys 3ccc98fc24
Fix name of removed `inspect.Signature.from_builtin` method in 3.11.0a2 changelog (#100525) 2022-12-26 17:04:18 +05:30
Glyph b9aa14a484
gh-100519: simplification to `eff_request_host` in cookiejar.py (#99588)
`IPV4_RE` includes a `.`, and the `.find(".") == -1` included here is already testing to make sure there's no dot, so this part of the expression is tautological. Instead use more modern `in` syntax to make it clear what the check is doing here. The simplified implementation more clearly matches the wording in RFC 2965.

Co-authored-by: hauntsaninja <hauntsaninja@gmail.com>
2022-12-24 18:14:51 -06:00
Shantanu 046cbc2080
gh-100472: Fix docs claim that compileall parameters could be bytes (#100473) 2022-12-24 14:22:49 -06:00
Shantanu e4b43ebb3a
gh-100287: Fix unittest.mock.seal with AsyncMock (#100496) 2022-12-24 19:39:39 +00:00
James Frost 46e6a28308
gh-100474: Fix handling of dirs named index.html in http.server (GH-100475)
If you had a directory called index.html or index.htm within a directory, it would cause http.server to return a 404 Not Found error instead of the directory listing. This came about due to not checking that the index was a regular file.

I have also added a test case for this situation.

Automerge-Triggered-By: GH:merwok
2022-12-24 10:28:59 -08:00
Nikita Sobolev bdfb694386
gh-100357: Convert several functions in `bltinsmodule` to AC (#100358) 2022-12-24 06:45:47 -08:00
Kumar Aditya c122390a55
GH-91166: Implement zero copy writes for `SelectorSocketTransport` in asyncio (#31871)
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2022-12-24 11:21:11 +05:30
Shantanu 3e46f9fe05
gh-100268: Add is_integer method to int (#100439)
This improves the lives of type annotation users of `float` - which type checkers implicitly treat as `int|float` because that is what most code actually wants. Before this change a `.is_integer()` method could not be assumed to exist on things annotated as `: float` due to the method not existing on both types.
2022-12-23 18:30:27 -08:00
Illia Volochii a23cb72ac8
gh-100454: Start running SSL tests with OpenSSL 3.1.0-beta1 (#100456) 2022-12-23 18:04:20 -08:00
Sebastian Berg 474220e3a5
gh-99947: Ensure unreported errors are chained for SystemError during import (GH-99946) 2022-12-23 15:43:19 -08:00
Barney Gale a68e585c8b
gh-68320, gh-88302 - Allow for private `pathlib.Path` subclassing (GH-31691)
Users may wish to define subclasses of `pathlib.Path` to add or modify
existing methods. Before this change, attempting to instantiate a subclass
raised an exception like:

    AttributeError: type object 'PPath' has no attribute '_flavour'

Previously the `_flavour` attribute was assigned as follows:

    PurePath._flavour        = xxx not set!! xxx
    PurePosixPath._flavour   = _PosixFlavour()
    PureWindowsPath._flavour = _WindowsFlavour()

This change replaces it with a `_pathmod` attribute, set as follows:

    PurePath._pathmod        = os.path
    PurePosixPath._pathmod   = posixpath
    PureWindowsPath._pathmod = ntpath

Functionality from `_PosixFlavour` and `_WindowsFlavour` is moved into
`PurePath` as underscored-prefixed classmethods. Flavours are removed.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Brett Cannon <brett@python.org>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Eryk Sun <eryksun@gmail.com>
2022-12-23 14:52:23 -08:00
Raymond Hettinger 5d84966cce
GH-100425: Improve accuracy of builtin sum() for float inputs (GH-100426) 2022-12-23 14:35:58 -08:00
Filipe Laíns 1ecfd1ebf1
bpo-40447: accept all path-like objects in compileall.compile_file (#19883)
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Filipe Laíns <lains@riseup.net>
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2022-12-23 14:19:16 -06:00
Nikita Sobolev 745545b5bb
gh-99482: remove `jython` compatibility parts from stdlib and tests (#99484) 2022-12-23 14:17:24 -06:00
Carl Meyer c5726b727e
gh-83076: 3.8x speed improvement in (Async)Mock instantiation (#100252) 2022-12-23 19:41:37 +00:00
Dennis Sweeney a98d9ea56e
gh-94155: Reduce hash collisions for code objects (#100183)
* Uses a better hashing algorithm to get better dispersion and remove commutativity.

* Incorporates `co_firstlineno`, `Py_SIZE(co)`, and bytecode instructions.

* This is now the entire set of criteria used in `code_richcompare`, except for `_PyCode_ConstantKey` (which would incorporate the types of `co_consts` rather than just their values).
2022-12-23 13:15:47 -05:00
Ken Jin 36d358348d
Revert "gh-100288: Specialise LOAD_ATTR_METHOD for managed dictionaries (GH-100289)" (#100468)
This reverts commit c3c7848a48.
2022-12-24 01:48:43 +08:00
Ken Jin c3c7848a48
gh-100288: Specialise LOAD_ATTR_METHOD for managed dictionaries (GH-100289) 2022-12-24 00:26:42 +08:00
Pieter Eendebak 7fc7909677
gh-92216: improve performance of `hasattr` for type objects (GH-99979) 2022-12-23 23:23:36 +08:00
Bill Fisher 88d565f32a
gh-99110: Initialize `frame->previous` in init_frame to fix segmentation fault when accessing `frame.f_back` (#100182) 2022-12-23 20:15:53 +05:30
Eric Wieser 84bc6a4f25
gh-76963: PEP3118 itemsize of an empty ctypes array should not be 0 (GH-5576)
The itemsize returned in a memoryview of a ctypes array is now computed from the item type, instead of dividing the total size by the length and assuming that the length is not zero.
2022-12-23 08:23:19 +00:00
Alex Waygood 9cdb642997
gh-85432: Harmonise parameter names between C and pure-Python implementations of `datetime.time.strftime`, `datetime.datetime.fromtimestamp` (#99993) 2022-12-22 15:57:18 -06:00
Itamar Ostricher 4cc63e0d4e
gh-100344: Add C implementation for `asyncio.current_task` (#100345)
Co-authored-by: pranavtbhat
2022-12-22 19:08:12 +05:30
Brandt Bucher 3c033a2e6f
GH-99554: Pack location tables more effectively (GH-99556) 2022-12-21 16:41:18 -08:00
amaajemyfren bbf4a66a18
GH-69564: Clarify use of octal format of mode argument in help(os.chmod) (#20621)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-12-21 23:18:25 +05:30
Hai Shi d713c54ac8
gh-78878: Fix crash when creating an instance of `_ctypes.CField` (#14837) 2022-12-21 23:01:17 +05:30
Dominic Socular 12be23cf3c
gh-100374: Fixed a bug in socket.getfqdn() (gh-100375) 2022-12-21 22:25:04 +09:00
Kumar Aditya a7715ccfba
GH-100133: fix `asyncio` subprocess losing `stderr` and `stdout` output (#100154) 2022-12-21 15:20:26 +05:30
Kumar Aditya 4994f2488f
GH-100363: Speed up `asyncio.get_running_loop` (#100364) 2022-12-21 15:19:44 +05:30
Shantanu 79311cbfe7
gh-85267: Improvements to inspect.signature __text_signature__ handling (#98796)
This makes a couple related changes to inspect.signature's behaviour
when parsing a signature from `__text_signature__`.

First, `inspect.signature` is documented as only raising ValueError or
TypeError. However, in some cases, we could raise RuntimeError.  This PR
changes that, thereby fixing #83685.

(Note that the new ValueErrors in RewriteSymbolics are caught and then
reraised with a message)

Second, `inspect.signature` could randomly drop parameters that it
didn't understand (corresponding to `return None` in the `p` function).
This is the core issue in #85267. I think this is very surprising
behaviour and it seems better to fail outright.

Third, adding this new failure broke a couple tests. To fix them (and to
e.g. allow `inspect.signature(select.epoll.register)` as in #85267), I
add constant folding of a couple binary operations to RewriteSymbolics.

(There's some discussion of making signature expression evaluation
arbitrary powerful in #68155. I think that's out of scope. The
additional constant folding here is pretty straightforward, useful, and
not much of a slippery slope)

Fourth, while #85267 is incorrect about the cause of the issue, it turns
out if you had consecutive newlines in __text_signature__, you'd get
`tokenize.TokenError`.

Finally, the `if name is invalid:` code path was dead, since
`parse_name` never returned `invalid`.
2022-12-20 19:29:02 -08:00
Dennis Sweeney c18d831188
gh-100188: Reduce misses in BINARY_SUBSCR_(LIST/TUPLE)_INT (#100189)
Don't specialize if the index is negative.
2022-12-20 15:46:16 -05:00
Nikita Sobolev 44892d45b0
gh-99576: Fix cookiejar file that was not truncated for some classes (GH-99616)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-12-20 17:51:26 +01:00
František Nesveda d98ca8172c
gh-99925: Fix inconsistency in `json.dumps()` error messages (GH-99926) 2022-12-20 12:54:56 +02:00
Richard Kojedzinszky a6331b605e
gh-100348: Fix ref cycle in `asyncio._SelectorSocketTransport` with `_read_ready_cb` (#100349) 2022-12-20 16:10:56 +05:30