Commit Graph

27364 Commits

Author SHA1 Message Date
Yeojin Kim 3c0a31cbfd
Docs: fix typos in PyFunction_WatchCallback docs and in 3.12 NEWS (GH-101980)
- possitibility => possibility
- disaallowed => disallowed
2023-02-17 00:47:02 -08:00
Eric Snow 4d8959b73a
gh-101758: Add _PyState_AddModule() Back for the Stable ABI (gh-101956)
We're adding the function back, only for the stable ABI symbol and not as any form of API. I had removed it yesterday.

This undocumented "private" function was added with the implementation for PEP 3121 (3.0, 2007) for internal use and later moved out of the limited API (3.6, 2016) and then into the internal API (3.9, 2019). I removed it completely yesterday, including from the stable ABI manifest (where it was added because the symbol happened to be exported). It's unlikely that anyone is using _PyState_AddModule(), especially any stable ABI extensions built against 3.2-3.5, but we're playing it safe.

https://github.com/python/cpython/issues/101758
2023-02-16 14:05:31 -07:00
Kumar Aditya a5024a261a
GH-96764: rewrite `asyncio.wait_for` to use `asyncio.timeout` (#98518)
Changes `asyncio.wait_for` to use `asyncio.timeout` as its underlying implementation.
2023-02-17 00:18:21 +05:30
Eli Schwartz 226484e475
gh-99942: correct the pkg-config/python-config flags for cygwin/android 2023-02-16 17:57:59 +00:00
Eclips4 68bd8c5e2e
gh-101952: Fix possible segfault in `BUILD_SET` opcode (#101958) 2023-02-16 09:46:43 -08:00
Rayyan Ansari 739c026f44
gh-101881: Support (non-)blocking read/write functions on Windows pipes (GH-101882)
* fileutils: handle non-blocking pipe IO on Windows

Handle erroring operations on non-blocking pipes by reading the _doserrno code.
Limit writes on non-blocking pipes that are too large.

* Support blocking functions on Windows

Use the GetNamedPipeHandleState and SetNamedPipeHandleState Win32 API functions to add support for os.get_blocking and os.set_blocking.
2023-02-16 14:52:24 +00:00
Gregory P. Smith 0b13575e74
gh-99108: Refactor _sha256 & _sha512 into _sha2. (#101924)
This merges their code. They're backed by the same single HACL* static library, having them be a single module simplifies maintenance.

This should unbreak the wasm enscripten builds that currently fail due to linking in --whole-archive mode and the HACL* library appearing twice.

Long unnoticed error fixed: _sha512.SHA384Type was doubly assigned and was actually SHA512Type. Nobody depends on those internal names.

Also rename LIBHACL_ make vars to LIBHACL_SHA2_ in preperation for other future HACL things.
2023-02-15 22:08:20 -08:00
Eric Snow 89ac665891
gh-98627: Add an Optional Check for Extension Module Subinterpreter Compatibility (gh-99040)
Enforcing (optionally) the restriction set by PEP 489 makes sense. Furthermore, this sets the stage for a potential restriction related to a per-interpreter GIL.

This change includes the following:

* add tests for extension module subinterpreter compatibility
* add _PyInterpreterConfig.check_multi_interp_extensions
* add Py_RTFLAGS_MULTI_INTERP_EXTENSIONS
* add _PyImport_CheckSubinterpIncompatibleExtensionAllowed()
* fail iff the module does not implement multi-phase init and the current interpreter is configured to check

https://github.com/python/cpython/issues/98627
2023-02-15 18:16:00 -07:00
Eric Snow b2fc549278
gh-101758: Clean Up Uses of Import State (gh-101919)
This change is almost entirely moving code around and hiding import state behind internal API.  We introduce no changes to behavior, nor to non-internal API.  (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.)  The motivation is to simplify a number of upcoming changes.

Specific changes:

* move existing import-related code to import.c, wherever possible
* add internal API for interacting with import state (both global and per-interpreter)
* use only API outside of import.c (to limit churn there when changing the location, etc.)
* consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly)
* add macros for import state in import.c (to simplify changing the location)
* group code in import.c into sections
*remove _PyState_AddModule()

https://github.com/python/cpython/issues/101758
2023-02-15 15:32:31 -07:00
Erlend E. Aasland c1ce0d178f
gh-99138: Isolate _zoneinfo (#99218)
* Convert zone info type to heap type and add it to module state
* Add global variables to module state
2023-02-15 22:58:48 +01:00
Erlend E. Aasland 8a2b7ee64d
gh-101693: In sqlite3, deprecate using named placeholders with parameters supplied as a sequence (#101698) 2023-02-15 06:27:16 +01:00
Irit Katriel 81e3aa835c
gh-101799: implement PREP_RERAISE_STAR as an intrinsic function (#101800) 2023-02-14 11:54:13 +00:00
Jonathan Protzenko e5da9ab2c8
gh-99108: Import SHA2-384/512 from HACL* (#101707)
Replace the builtin hashlib implementations of SHA2-384 and SHA2-512
originally from LibTomCrypt with formally verified, side-channel resistant
code from the [HACL*](https://github.com/hacl-star/hacl-star/) project.
The builtins remain a fallback only used when OpenSSL does not provide them.
2023-02-14 01:25:16 -08:00
Sam James 8be8101bca
gh-101857: Allow xattr detection on musl libc (#101858)
Previously, we checked exclusively for `__GLIBC__` (AND'd with some other
conditions). Checking for `__linux__` instead should be fine.

This fixes using e.g. `os.listxattr()` on systems using musl libc.

Bug: https://bugs.gentoo.org/894130

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-02-13 23:21:58 -08:00
Radek Smejkal 928752ce4c
gh-74895: getaddrinfo no longer raises OverflowError (#2435)
`socket.getaddrinfo()` no longer raises `OverflowError` based on the **port** argument. Error reporting (or not) for its value is left up to the underlying C library `getaddrinfo()` implementation.
2023-02-13 17:37:34 -08:00
Steve Dower 0c6fe81dce
gh-101849: Add upgrade codes for old versions of launcher that ended up with later version numbers (GH-101877) 2023-02-13 20:33:48 +00:00
Mark Shannon 160f2fe2b9
GH-87849: Simplify stack effect of SEND and specialize it for generators and coroutines. (GH-101788) 2023-02-13 11:24:55 +00:00
Gregory P. Smith dfc2e065a2
gh-89792: Limit test_tools freeze test build parallelism based on the number of cores (#101841)
unhardcode freeze test build parallelism. base it on the number of cpus, don't use more than max(2, os.cpu_count()/3).
2023-02-11 22:07:52 -08:00
Soumendra Ganguly da2fb92643
gh-85984: Utilize new "winsize" functions from termios in pty tests. (#101831)
Utilize new functions termios.tcgetwinsize() and termios.tcsetwinsize in test_pty.py.

Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-02-11 21:24:43 -08:00
Gregory P. Smith 1d194235e4
gh-89792: Prevent test_tools from copying 1000M of "source" in freeze test (#101837)
Prevent test_tools from copying 1000M of "source"
    
It doesn't need a git repo, just the checkout.  We skip .git metadata, Doc/build, Doc/venv, and `__pycache__` subdirs, that developers often have in their clients to reduce the size of the source tree copy ten-fold.

This should significantly reduce IO and presumably time on buildbots during this long test.
2023-02-11 20:54:28 -08:00
Steve Dower e1aadedf09
gh-101763: Update bundled copy of libffi to 3.4.4 on Windows (GH-101784) 2023-02-10 16:57:30 +00:00
Irit Katriel 366b949058
gh-101517: make bdb avoid looking up in linecache with lineno=None (#101787) 2023-02-10 16:49:29 +00:00
Erlend E. Aasland 5d15224011
gh-101759: Update Windows installer to SQLite 3.40.1 (#101762) 2023-02-10 17:38:26 +01:00
Erlend E. Aasland 826bf0e695
gh-101277: Finalise isolating itertools (GH-101305)
Add repeat, islice, chain, tee, teedataobject, and batched types to module state.

Automerge-Triggered-By: GH:erlend-aasland
2023-02-10 03:58:14 -08:00
Erlend E. Aasland d40a23c0a1
gh-101759: Update macOS installer to SQLite 3.40.1 (#101761) 2023-02-10 08:25:02 +01:00
Dong-hee Na 5b946d3719
gh-101430: Update tracemalloc to handle presize properly. (gh-101745) 2023-02-10 08:30:03 +09:00
Gregory P. Smith b41c47cd06
gh-101726: Update the OpenSSL version to 1.1.1t (GH-101727)
Fixes CVE-2023-0286 (High) and a couple of Medium security issues.
https://www.openssl.org/news/secadv/20230207.txt
2023-02-09 17:40:51 +00:00
Mark Shannon ecfd2d37c5
GH-99293: Document that `Py_TPFLAGS_VALID_VERSION_TAG` shouldn't be used. (#GH-101736)
Document that Py_TPFLAGS_VALID_VERSION_TAG shouldn't be used.
2023-02-09 14:05:53 +01:00
Soumendra Ganguly 244d4cd9d2
gh-85984: Remove legacy Lib/pty.py code. (#92365)
Refactored the implementation of pty.fork to use os.login_tty.

A DeprecationWarning is now raised by pty.master_open() and pty.slave_open(). They were
undocumented and deprecated long long ago in the docstring in favor of pty.openpty.

Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-02-08 17:00:17 -08:00
Oleg Iarygin 23751ed826
gh-101283: Improved fallback logic for subprocess with shell=True on Windows (GH-101286) 2023-02-08 22:12:19 +00:00
Kumar Aditya d9de079248
GH-101696: invalidate type version tag in `_PyStaticType_Dealloc` (#101697) 2023-02-08 23:32:15 +05:30
Michał Górny 2a8bf25804
gh-100221: Fix creating dirs in `make sharedinstall` (GH-100329)
Fix creating install directories in `make sharedinstall` if they exist already outside `DESTDIR`.  The previous make rules assumed that the directories would be created via a dependency on a rule for `$(DESTSHARED)` that did not fire if the directory did exist outside `$(DESTDIR)`.

While technically `$(DESTDIR)` could be prepended to the rule name, moving the rules for creating directories straight into the `sharedinstall` rule seems to fit the common practices better. Since the rule explicitly checks whether the individual directories exist anyway, there seems to be no reason to rely on make determining that implicitly as well.
2023-02-08 08:50:43 -08:00
Michael Droettboom 86ebd5c3fa
gh-101196: Make isdir/isfile/exists faster on Windows (GH-101324)
Co-authored-by: Eryk Sun <eryksun@gmail.com>
2023-02-08 14:34:24 +00:00
David Hewitt 3a88de7a0a
gh-101614: Don't treat python3_d.dll as a Python DLL when checking extension modules for incompatibility (GH-101615) 2023-02-08 14:23:57 +00:00
Mark Shannon feec49c407
GH-101578: Normalize the current exception (GH-101607)
* Make sure that the current exception is always normalized.

* Remove redundant type and traceback fields for the current exception.

* Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException

* Add new API functions: PyException_GetArgs, PyException_SetArgs
2023-02-08 09:31:12 +00:00
Nikita Sobolev 790ff6bc6a
gh-101446: Change `repr` of `collections.OrderedDict` (#101661) 2023-02-07 20:01:10 -06:00
penguin_wwy 753fc8a5d6
gh-101632: Add the new RETURN_CONST opcode (#101633) 2023-02-07 22:32:21 +00:00
Thomas Wouters 0d3d5007b1 Merge branch 'main' of https://github.com/python/cpython into main 2023-02-07 23:13:53 +01:00
Oleg Iarygin f87f6e2396
gh-97725: Fix documentation for the default file of `asyncio.Task.print_stack` (#101652) 2023-02-07 23:34:31 +05:30
Thomas Wouters 3c67ec394f Python 3.12.0a5 2023-02-07 13:21:15 +01:00
Łukasz Langa 7990324048
[gh-101072] Fix Blurb for GH-101127 2023-02-07 10:50:39 +01:00
Matthieu Dartiailh ae62bddaf8
gh-101072: support default and kw default in PyEval_EvalCodeEx for 3.11+ (#101127)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-02-07 10:34:21 +01:00
Jonathan Protzenko 1fcc0efdaa
gh-99108: Replace SHA2-224 & 256 with verified code from HACL* (#99109)
replacing hashlib primitives (for the non-OpenSSL case) with verified implementations from HACL*. This is the first PR in the series, and focuses specifically on SHA2-256 and SHA2-224.

This PR imports Hacl_Streaming_SHA2 into the Python tree. This is the HACL* implementation of SHA2, which combines a core implementation of SHA2 along with a layer of buffer management that allows updating the digest with any number of bytes. This supersedes the previous implementation in the tree.

@franziskuskiefer was kind enough to benchmark the changes: in addition to being verified (thus providing significant safety and security improvements), this implementation also provides a sizeable performance boost!

```
---------------------------------------------------------------
Benchmark                     Time             CPU   Iterations
---------------------------------------------------------------
Sha2_256_Streaming            3163 ns      3160 ns       219353     // this PR
LibTomCrypt_Sha2_256          5057 ns      5056 ns       136234     // library used by Python currently
``` 

The changes in this PR are as follows:
- import the subset of HACL* that covers SHA2-256/224 into `Modules/_hacl`
- rewire sha256module.c to use the HACL* implementation

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-02-06 18:11:01 -08:00
Eric Snow 132b3f8302
gh-59956: Partial Fix for GILState API Compatibility with Subinterpreters (gh-101431)
The GILState API (PEP 311) implementation from 2003 made the assumption that only one thread state would ever be used for any given OS thread, explicitly disregarding the case of subinterpreters.  However, PyThreadState_Swap() still facilitated switching between subinterpreters, meaning the "current" thread state (holding the GIL), and the GILState thread state could end up out of sync, causing problems (including crashes).

This change addresses the issue by keeping the two in sync in PyThreadState_Swap().  I verified the fix against gh-99040.

Note that the other GILState-subinterpreter incompatibility (with autoInterpreterState) is not resolved here.

https://github.com/python/cpython/issues/59956
2023-02-06 14:39:25 -07:00
Eclips4 b96b344f25
gh-101562: typing: add tests for inheritance with NotRequired & Required in parent fields (#101563) 2023-02-06 11:28:24 -08:00
Steve Dower 7a253103d4
gh-101543: Ensure Windows registry path is only used when stdlib can't be found (GH-101544) 2023-02-06 15:55:32 +00:00
Dong-hee Na 9ef7e75434
gh-101372: Fix unicodedata.is_normalized to properly handle the UCD 3… (gh-101388) 2023-02-06 13:58:00 +09:00
Ethan Furman ef7c2bfcf1
gh-101541: [Enum] create flag psuedo-member without calling original __new__ (GH-101590) 2023-02-05 19:29:06 -08:00
Gregory P. Smith ffcb8220d7
gh-101334: Don't force USTAR format in test_tarfile. (GH-101572)
That causes the test to fail when run using a high UID as that ancient format
cannot represent it. The current default (PAX) and the old default (GNU) both
support high UIDs.
2023-02-05 09:44:57 -08:00
mrh1997 f7e9fbacb2
bpo-33591: Add support for path like objects to `ctypes.CDLL` (#7032)
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-02-05 23:06:57 +05:30
Eric Wieser 90d85a9b41
gh-76961: Fix the PEP3118 format string for ctypes.Structure (#5561)
The summary of this diff is that it:

* adds a `_ctypes_alloc_format_padding` function to append strings like `37x` to a format string to indicate 37 padding bytes
* removes the branches that amount to "give up on producing a valid format string if the struct is packed"
* combines the resulting adjacent `if (isStruct) {`s now that neither is `if (isStruct && !isPacked) {`
* invokes `_ctypes_alloc_format_padding` to add padding between structure fields, and after the last structure field. The computation used for the total size is unchanged from ctypes already used.

This patch does not affect any existing aligment computation; all it does is use subtraction to deduce the amount of paddnig introduced by the existing code.

---

Without this fix, it would never include padding bytes - an assumption that was only
valid in the case when `_pack_` was set - and this case was explicitly not implemented.

This should allow conversion from ctypes structs to numpy structs

Fixes https://github.com/numpy/numpy/issues/10528
2023-02-05 17:10:53 +00:00
Mark Dickinson 0672a6c23b
Revert "gh-89381: Fix invalid signatures of math/cmath.log (#101404)" (#101580)
This reverts commit 0ef92d9793.
2023-02-05 16:36:33 +00:00
Pradyun Gedam 19ac43629e
gh-101570: Update bundled pip version to 23.0 (#101571)
Update bundled pip version to 23.0

This is the current latest version of `pip`.

---------

Co-authored-by: Pradyun Gedam <pradyunsg@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-02-05 11:30:44 +00:00
Mark Dickinson 39017e04b5
gh-101266: Fix __sizeof__ for subclasses of int (#101394)
Fix the behaviour of the `__sizeof__` method (and hence the results returned by `sys.getsizeof`) for subclasses of `int`. Previously, `int` subclasses gave identical results to the `int` base class, ignoring the presence of the instance dictionary.

<!-- gh-issue-number: gh-101266 -->
* Issue: gh-101266
<!-- /gh-issue-number -->
2023-02-05 10:02:53 +00:00
Ruben Vorderman a89e6713c4
gh-101322: Ensure test_zlib.ZlibDecompressorTest runs, fix errors in ZlibDecompressor (#101323)
* Ensure test_zlib.ZlibDecompressorTest actually runs, fix errors in ZlibDecompressor.
2023-02-04 12:07:30 -08:00
Dong-hee Na 144aaa74bb
gh-101282: Update BOLT --split-functions flag not to use deprecated u… (gh-101557)
gh-101282: Update BOLT --split-functions flag not to use deprecated usage
2023-02-04 16:55:31 +09:00
Gregory P. Smith d4c410f0f9
gh-84559: Remove the new multiprocessing warning, too disruptive. (#101551)
This reverts the core of #100618 while leaving relevant documentation
improvements and minor refactorings in place.
2023-02-03 15:20:46 -08:00
Steve Dower f6c53b80a1
gh-101522: Allow overriding Windows dependencies versions and paths using MSBuild properties (GH-101523) 2023-02-03 18:08:34 +00:00
Gregory P. Smith 0ca67e6313
GH-84559: Deprecate fork being the multiprocessing default. (#100618)
This starts the process. Users who don't specify their own start method
and use the default on platforms where it is 'fork' will see a
DeprecationWarning upon multiprocessing.Pool() construction or upon
multiprocessing.Process.start() or concurrent.futures.ProcessPool use.

See the related issue and documentation within this change for details.
2023-02-02 15:50:35 -08:00
Ayappan Perumal 618b7a8260
gh-98705: Fix AIX build by undefining `__bool__` in C (#98768) 2023-02-02 12:32:33 -08:00
Ayappan Perumal ba4731d149
gh-96305: Fix AIX build by avoiding subprocess during bootstrap (#96429)
* Fix AIX build by avoiding `subprocess` during bootstrap.
2023-02-02 12:30:49 -08:00
Steve Dower eda60916bc
gh-101467: Correct py.exe handling of prefix matches and cases when only one runtime is installed (GH-101468) 2023-02-01 21:06:56 +00:00
beavailable cc407b9de6
gh-101317: Add `ssl_shutdown_timeout` parameter for `asyncio.StreamWriter.start_tls` (#101335)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-02-01 16:33:59 +05:30
Dong-hee Na e867c1b753
gh-101400: Fix incorrect lineno in exception message on continue/break which are not in a loop (#101413) 2023-01-30 23:33:54 +00:00
Mark Shannon c1b1f51cd1
GH-101291: Refactor the `PyLongObject` struct into object header and PyLongValue struct. (GH-101292) 2023-01-30 10:03:04 +00:00
Sergey B Kirpichev 0ef92d9793
gh-89381: Fix invalid signatures of math/cmath.log (#101404) 2023-01-29 11:50:10 -08:00
Gregory P. Smith 052f53d65d
gh-39615: Add warnings.warn() skip_file_prefixes support (#100840)
`warnings.warn()` gains the ability to skip stack frames based on code
filename prefix rather than only a numeric `stacklevel=` via a new
`skip_file_prefixes=` keyword argument.
2023-01-27 18:35:14 -08:00
Steve Dower 737d367b1f
gh-77532: Minor tweaks to allow compiling with PlatformToolset=ClangCL on Windows (GH-101352)
To use this, ensure that clang support was selected in Visual Studio Installer, then set the PlatformToolset environment variable to "ClangCL" and build as normal from the command line.
It remains unsupported, but at least is possible now for experimentation.
2023-01-27 14:45:08 +00:00
Barney Gale e5b08ddddf
gh-101000: Add os.path.splitroot() (#101002)
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-01-27 00:28:27 +00:00
Steve Dower 8d18d1ffd5
gh-99834: Update bundled copy of Tcl/Tk to 8.6.13.0 on Windows (GH-101307) 2023-01-26 20:47:24 +00:00
Oleg Iarygin 409f5337a3
gh-60580: Fix a wrong type of `ctypes.wintypes.BYTE` (#97579)
Created from a patch file attached to an issue, by Anatoly Techtonik.
2023-01-26 18:16:27 +04:00
Yukihiro Nakadaira dfad678d70
gh-99952: [ctypes] fix refcount issues in from_param() result. (#100169)
Fixes a reference counting issue with `ctypes.Structure` when a `from_param()` method call is used and the structure size is larger than a C pointer `sizeof(void*)`.

This problem existed for a very long time, but became more apparent in 3.8+ by change likely due to garbage collection cleanup timing changes.
2023-01-26 00:28:34 -08:00
Oleg Iarygin 73245d084e
gh-94518: Rename `group*` to `extra_group*` to avoid confusion (#101054)
* Rename `group*` to `extra_group*` to avoid confusion
* Rename `num_groups` into `extra_group_size`
* Rename `groups_list` to `extra_groups_packed`
2023-01-25 22:50:33 -08:00
Shantanu a178ba82bf
gh-101326: Fix regression when passing None to FutureIter.throw (#101327) 2023-01-25 12:01:01 -08:00
Mark Shannon f02fa64bf2
GH-100762: Don't call `gen.throw()` in `gen.close()`, unless necessary. (GH-101013)
* Store exception stack depth in YIELD_VALUE's oparg and use it avoid expensive gen.throw() in gen.close() where possible.
2023-01-24 17:25:37 +00:00
Martin Boisvert daec3a463c
gh-101135: Add backwards compatibility to Windows launcher for older 32-bit versions (GH-101138)
Python 2.x and up to 3.4 did not contain the "-32" in their registry name, so the 32 and 64-bit installs were treated equal. Since 3.5/PEP 514 this is no longer true, but we still want to detect the EOL versions correctly in case people are still using them.

Additionally, the code to replace a node with one with a lower sort key was buggy (wrong node chosen, replace never happened since parent was always NULL, replaced node never freed, etc)
2023-01-24 16:35:16 +00:00
Erlend E. Aasland fee7a995a1
gh-92123: Adapt _elementtree to multi-phase init (PEP 489) (#101285) 2023-01-24 17:00:24 +01:00
Gregory Szorc 7589d713a1
gh-101060: conditionally add `-fno-reorder-blocks-and-partition` (gh-101061) 2023-01-24 18:34:44 +09:00
ram vikram singh 7f95ec3e74
gh-101152: Implement PEP 699 (GH-101193)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Ken Jin <kenjin@python.org>
2023-01-24 17:29:22 +08:00
Gregory P. Smith b724ac2fe7
gh-100795: Don't call freeaddrinfo on failure. (#101252)
When getaddrinfo returns an error, the output pointer is in an unknown state
Don't call freeaddrinfo on it.  See the issue for discussion and details with
links to reasoning.  _Most_ libc getaddrinfo implementations never modify the
output pointer unless they are returning success.

Co-authored-by: Sergey G. Brester <github@sebres.de>
Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2023-01-23 15:03:26 -08:00
Nikita Sobolev 807d6b576f
gh-101015: Fix `typing.get_type_hints` with unpacked `*tuple` (PEP 646) (#101031) 2023-01-23 07:52:58 +00:00
Jacob Walls d717be04dc
gh-83122: Deprecate testing element truth values in `ElementTree` (#31149)
When testing element truth values, emit a DeprecationWarning in all implementations.

This had emitted a FutureWarning in the rarely used python-only implementation since ~2.7 and has always been documented as a behavior not to rely on.

Matching an element in a tree search but having it test False can be unexpected. Raising the warning enables making the choice to finally raise an exception for this ambiguous behavior in the future.
2023-01-22 17:16:48 -08:00
Mark Dickinson 3e09f3152e
gh-67790: Support float-style formatting for Fraction instances (#100161)
This PR adds support for float-style formatting for `Fraction` objects: it supports the `"e"`, `"E"`, `"f"`, `"F"`, `"g"`, `"G"` and `"%"` presentation types, and all the various bells and whistles of the formatting mini-language for those presentation types. The behaviour almost exactly matches that of `float`, but the implementation works with the exact `Fraction` value and does not do an intermediate conversion to `float`, and so avoids loss of precision or issues with numbers that are outside the dynamic range of the `float` type.

Note that the `"n"` presentation type is _not_ supported. That support could be added later if people have a need for it.

There's one corner-case where the behaviour differs from that of float: for the `float` type, if explicit alignment is specified with a fill character of `'0'` and alignment type `'='`, then thousands separators (if specified) are inserted into the padding string:

```python
>>> format(3.14, '0=11,.2f')
'0,000,003.14'
```

The exact same effect can be achieved by using the `'0'` flag:

```python
>>> format(3.14, '011,.2f')
'0,000,003.14'
```

For `Fraction`, only the `'0'` flag has the above behaviour with respect to thousands separators: there's no special-casing of the particular `'0='` fill-character/alignment combination. Instead, we treat the fill character `'0'` just like any other:

```python
>>> format(Fraction('3.14'), '0=11,.2f')
'00000003.14'
>>> format(Fraction('3.14'), '011,.2f')
'0,000,003.14'
```

The `Fraction` formatter is also stricter about combining these two things: it's not permitted to use both the `'0'` flag _and_ explicit alignment, on the basis that we should refuse the temptation to guess in the face of ambiguity. `float` is less picky:

```python
>>> format(3.14, '0<011,.2f')
'3.140000000'
>>> format(Fraction('3.14'), '0<011,.2f')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mdickinson/Repositories/python/cpython/Lib/fractions.py", line 414, in __format__
    raise ValueError(
ValueError: Invalid format specifier '0<011,.2f' for object of type 'Fraction'; can't use explicit alignment when zero-padding
```
2023-01-22 18:44:49 +00:00
Kumar Aditya b53bad6dd0
Revert "gh-100795: avoid unexpected `freeaddrinfo` after failed `getaddrinfo` (#101220)" (#101238)
Revert "gh-100795: avoid unexpected `freeaddrinfo` after failed `getaddrinfo` (#101220)"

This reverts commit 5f08fe4a2c.
2023-01-22 14:17:01 +05:30
Sergey G. Brester 5f08fe4a2c
gh-100795: avoid unexpected `freeaddrinfo` after failed `getaddrinfo` (#101220)
Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2023-01-22 13:40:00 +05:30
achhina 95f5b05a8c
GH-88597: Added command line interface to UUID module. (#99463)
The `uuid` module now supports command line usage.

```python
❯ ./python.exe -m uuid             
5f2d57b1-90e8-417c-ba5d-69b9b6f74289

❯ ./python.exe -m uuid -h          
usage: uuid.py [-h] [-u {uuid1,uuid3,uuid4,uuid5}] [-ns NAMESPACE] [-n NAME]
...
```
2023-01-21 22:59:31 -08:00
Pieter Eendebak f63f525e16
gh-100726: Optimize construction of range object for medium sized integers (#100810)
Use C long arithmetic instead of PyLong arithmetic to compute the range length, where possible.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-01-21 19:33:08 +00:00
Kamil Turek b4e11a7985
gh-99266: ctypes: Preserve more detailed exception in `ArgumentError`
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-01-21 19:14:43 +05:30
Mark Dickinson 401fdf9c85
gh-101037: Fix potential memory underallocation for zeros of int subtypes (#101038)
This PR fixes object allocation in long_subtype_new to ensure that there's at least one digit in all cases, and makes sure that the value of that digit is copied over from the source long.

Needs backport to 3.11, but not any further: the change to require at least one digit was only introduced for Python 3.11.

Fixes #101037.
2023-01-21 10:23:59 +00:00
J. Nick Koston 9e947675ae
gh-101143: Remove references to `TimerHandle` from `asyncio.base_events.BaseEventLoop._add_callback` (#101197)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-01-21 14:46:07 +05:30
Jean-Paul Calderone 3325f054e3
gh-91351: Fix some bugs in importlib handling of re-entrant imports (GH-94504)
Co-authored-by: Brett Cannon <brett@python.org>
2023-01-20 16:00:39 -08:00
Thomas Grainger 6b3993c556
gh-100750: pass encoding kwarg in lib/platform.py (#100751)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2023-01-20 23:25:38 +01:00
Erlend E. Aasland 3847a6c64b
gh-92123: Convert `_elementtree` types to heap types (#99221) 2023-01-20 12:40:06 +01:00
richardhob 3fa8fe7177
gh-88324: Clarify documentation for redirected stdout/stderr when using subprocess in Linux (#94035)
* Update description of stdout, stderr, and stdin.

Changes:
- Move the ``None`` option (which is default) to the front of the list
  of input options
- Move the ``None`` option description up to make the default behavior
  more clear (No redirection)
- Remove mention of Child File Descriptors from ``None`` option description
2023-01-19 23:56:13 -08:00
Gregory P. Smith 5927013e47
gh-101144: Allow open and read_text encoding to be positional. (#101145)
The zipfile.Path open() and read_text() encoding parameter can be supplied as a positional argument without causing a TypeError again. 3.10.0b1 included a regression that made it keyword only.

Documentation update included as users writing code to be compatible with a wide range of versions will need to consider this for some time.
2023-01-19 23:04:30 -08:00
Irit Katriel e9ccfe4a63
gh-100712: make it possible to disable specialization (for debugging) (#100713) 2023-01-19 18:14:55 +00:00
Kushal Das 75c8133efe
gh-100340: Allows -Wno-int-conversion for wasm (#100341)
Fixes #100340 allows -Wno-int-conversion for wasm
2023-01-18 19:43:49 +01:00
Paul Moore f34176b77f
gh-82052: Don't send partial UTF-8 sequences to the Windows API (GH-101103)
Don't send partial UTF-8 sequences to the Windows API
2023-01-17 19:23:06 +00:00
Pablo Galindo Salgado 1de4395f62
gh-101046: Fix a potential memory leak in the parser when raising MemoryError (#101051) 2023-01-16 18:45:37 +00:00
Steve Dower df10571a13
gh-100320: Fix path calculations on Windows when python.exe is moved outside of the normal location (GH-100947) 2023-01-16 16:05:39 +00:00
Mark Shannon 7b14c2ef19
GH-100982: Add `COMPARE_AND_BRANCH` instruction (GH-100983) 2023-01-16 12:35:21 +00:00
Oleg Iarygin 124af17b6e
gh-94518: [_posixsubprocess] Replace variable validity flags with reserved values (#94687)
Have _posixsubprocess.c stop using boolean flags to say if gid and uid values were supplied and action is required.  Such an implicit "either initialized or look somewhere else" confused both the reader (another mental connection to constantly track between functions) and a compiler (warnings on potentially uninitialized variables being passed). Instead, we can utilize a special group/user id as a flag value -1 defined by POSIX but used nowhere else. Namely:

gid: call_setgid = False → gid = -1

uid: call_setuid = False → uid = -1

groups: call_setgroups = False → groups = NULL (obtained with (groups_list != Py_None) ? groups : NULL)

This PR is required for #94519.
2023-01-14 12:11:04 -08:00
Yurii Karabas 080cb27829
gh-74033: Fix bug when Path takes and ignores **kwargs (GH-19632)
Fix a bug where `Path` takes and ignores `**kwargs` by adding to `PurePath`  class `__init__` method which can take only positional arguments.

Automerge-Triggered-By: GH:brettcannon
2023-01-13 16:05:43 -08:00
Guido van Rossum 1bc7a73683
GH-100573: Fix server hang caused by os.stat() on named pipe (Windows) (#100959) 2023-01-13 13:24:57 -08:00
Serhiy Storchaka e5bd5ad70d
gh-100160: Restore and deprecate implicit creation of an event loop (GH-100410)
Partially revert changes made in GH-93453.

asyncio.DefaultEventLoopPolicy.get_event_loop() now emits a
DeprecationWarning and creates and sets a new event loop instead of
raising a RuntimeError if there is no current event loop set.

Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2023-01-13 14:40:29 +02:00
Steve Dower 468c3bf798
gh-100247: Fix py.exe launcher not using entire shebang command for finding custom commands (GH-100944) 2023-01-13 11:49:01 +00:00
Steve Dower b5d4347950
gh-86682: Adds sys._getframemodulename as an alternative to using _getframe (GH-99520)
Also updates calls in collections, doctest, enum, and typing modules to use _getframemodulename first when available.
2023-01-13 11:31:06 +00:00
Raymond Hettinger 94fc7706b7
GH-100942: Fix incorrect cast in property_copy(). (#100965) 2023-01-12 16:13:56 -06:00
Barney Gale 005e69403d
gh-96290: Support partial/invalid UNC drives in ntpath.normpath() and splitdrive() (GH-100351)
This brings the Python implementation of `ntpath.normpath()` in line with the C implementation added in 99fcf15

Co-authored-by: Eryk Sun <eryksun@gmail.com>
2023-01-12 19:24:57 +00:00
Michał Górny 02a72f080d
gh-98636: Fix detecting gdbm_compat for _dbm module (#98643)
Fix the gdbm_compat library detection logic to actually check for
-lgdbm_compat independently of the ndbm detection.
This fixes the build failure with `--with-dbmliborder=gdbm`,
and implicit fallback to ndbm with the default value.
2023-01-11 22:46:28 +01:00
Mark Shannon 6e4e14d98f
GH-100923: Embed jump mask in `COMPARE_OP` oparg (GH-100924) 2023-01-11 20:40:43 +00:00
Kumar Aditya 762745a124
GH-100892: Fix race in clearing `threading.local` (#100922) 2023-01-11 16:03:31 +05:30
Thomas Wouters 3d5d3f7af6 Python 3.12.0a4 2023-01-10 13:09:15 +01:00
Brandt Bucher f07daaf4f7
GH-100117: Make `co_lines` more efficient (GH-100447) 2023-01-10 10:56:53 +00:00
Ned Deily 57a5259a43
gh-100180: Update macOS installer to OpenSSL 1.1.1s (GH-100908) 2023-01-10 00:10:31 -05:00
Steve Dower d7ab7149f8
gh-100180: Update Windows installer to OpenSSL 1.1.1s (GH-100901) 2023-01-09 23:50:59 +00:00
Brandt Bucher 61762b9387
GH-100126: Skip incomplete frames in more places (GH-100613) 2023-01-09 12:20:04 -08:00
C.A.M. Gerlach f08209874e
gh-99191: Use correct check for MSVC C++ version support in _wmimodule.cpp (GH-100381) 2023-01-09 17:48:24 +00:00
dgelessus 837ba05267
GH-81061: Fix refcount issue when returning `None` from a `ctypes.py_object` callback (#13364) 2023-01-09 15:43:04 +05:30
dsentinel 7a50d6b5b0
GH-100813: Add `socket.IP_PKTINFO` constant (#10294) 2023-01-09 14:30:40 +05:30
Nikita Sobolev bc0a686f82
gh-87447: Fix walrus comprehension rebind checking (#100581)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2023-01-08 15:51:29 -07:00
Raymond Hettinger 9a68ff12c3
GH-100805: Support numpy.array() in random.choice(). (GH-100830) 2023-01-08 13:40:35 -06:00
Mark Dickinson 87d3bd0e02
gh-100833: Remove 'volatile' qualifiers in fsum algorithm (#100845)
This PR removes the `volatile` qualifier on various intermediate quantities
in the `math.fsum` implementation, and updates the notes preceding the
algorithm accordingly (as well as fixing some of the exsting notes). See
the linked issue #100833 for discussion.
2023-01-08 19:40:15 +00:00
Nikita Sobolev b034fd3e59
gh-100689: Revert "bpo-41798: pyexpat: Allocate the expat_CAPI on the heap memory (GH-24061)" (#100745)
* gh-100689: Revert "bpo-41798: pyexpat: Allocate the expat_CAPI on the heap memory (GH-24061)"

This reverts commit 7c83eaa536.
2023-01-08 18:24:40 +05:30
Sergey B Kirpichev 909982e82a
gh-91851: Micro optimizations for arithmetic between Fractions (#25518)
Adapted from
https://github.com/python/cpython/pull/24779/commits/046c84e8f9

This makes arithmetic between Fractions with small components
just as fast as before python/cpython#24779, at some expense of
mixed arithmetic (e.g. Fraction + int).
2023-01-08 00:34:20 -08:00
Shantanu a2141882f2
gh-100776: Fix misleading default value in help(input) (#100788) 2023-01-08 13:27:41 +05:30
Nikita Sobolev 6746135b07
gh-100792: Make `email.message.Message.__contains__` twice as fast (#100793) 2023-01-07 13:26:05 -08:00
Raymond Hettinger 47b9f83a83
GH-100485: Add math.sumprod() (GH-100677) 2023-01-07 12:46:35 -06:00
Nikita Sobolev 9e7d7266ec
gh-96127: Fix `inspect.signature` call on mocks (#96335) 2023-01-07 10:49:15 +00:00
Christian Klein 1d4d677d1c
gh-100690: Raise an AttributeError when the assert_ prefix is forgotten when using Mock (#100691)
Mock objects which are not unsafe will now raise an AttributeError when accessing an
attribute that matches the name of an assertion but without the prefix `assert_`, e.g. accessing `called_once` instead of `assert_called_once`.

This is in addition to this already happening for accessing attributes with prefixes assert, assret, asert, aseert, and assrt.
2023-01-06 18:38:50 +00:00
Sergey B Kirpichev 0e640260da
gh-91851: Trivial optimizations in Fraction (#100791)
Make some trivial performance optimizations in Fraction

Uses private class attributes `_numerator` and `_denominator` in place of the `numerator` and `denominator` property accesses.

Co-authored-by: hauntsaninja <hauntsaninja@gmail.com>
2023-01-06 15:37:34 +00:00
Mark Shannon 78068126a1
GH-99005: More intrinsics (GH-100774)
* Remove UNARY_POSITIVE, LIST_TO_TUPLE and ASYNC_GEN_WRAP, replacing them with intrinsics.
2023-01-06 14:47:57 +00:00
Carl Meyer 0a7936a38f
gh-90104: avoid RecursionError on recursive dataclass field repr (gh-100756)
Avoid RecursionError on recursive dataclass field repr
2023-01-05 19:19:40 -05:00
Dustin Spicuzza cc8748712e
gh-86082: bpo-41916: allow cross-compiled python to have -pthread set for CXX (#22525)
When cross-compiling, the compile/run test for -pthread always fails so -pthread
will never be automatically set without an override from the cache. ac_cv_pthread
can already be overridden, so do the same thing for ac_cv_cxx_thread.
2023-01-05 14:57:31 -08:00
Barney Gale 7fba99eadb
gh-100562: improve performance of `pathlib.Path.absolute()` (GH-100563)
Increase performance of the `absolute()` method by calling `os.getcwd()` directly, rather than using the `Path.cwd()` class method. This avoids constructing an extra `Path` object (and the parsing/normalization that comes with it).

Decrease performance of the `cwd()` class method by calling the `Path.absolute()` method, rather than using `os.getcwd()` directly. This involves constructing an extra `Path` object. We do this to maintain a longstanding pattern where `os` functions are called from only one place, which allows them to be more readily replaced by users. As `cwd()` is generally called at most once within user programs, it's a good bargain.

```shell
# before
$ ./python -m timeit -s 'from pathlib import Path; p = Path("foo", "bar")' 'p.absolute()'
50000 loops, best of 5: 9.04 usec per loop
# after
$ ./python -m timeit -s 'from pathlib import Path; p = Path("foo", "bar")' 'p.absolute()'
50000 loops, best of 5: 5.02 usec per loop
```

Automerge-Triggered-By: GH:AlexWaygood
2023-01-05 14:11:50 -08:00
Mark Shannon 28187141cc
GH-99005: Add `CALL_INTRINSIC_1` instruction (GH-100771)
* Remove PRINT_EXPR instruction

* Remove STOPITERATION_ERROR instruction

* Remove IMPORT_STAR instruction
2023-01-05 16:05:51 +00:00
Mark Shannon f20c553a45
GH-100288: Remove LOAD_ATTR_METHOD_WITH_DICT instruction. (GH-100753) 2023-01-05 12:20:09 +00:00
Christian Klein 7f1eefc6f4
gh-100739: Respect mock spec when checking for unsafe prefixes (#100740)
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2023-01-04 22:31:29 +00:00
Mark Shannon 15aecf8dd7
GH-100719: Remove the `co_nplaincellvars` field from code objects. (GH-100721) 2023-01-04 15:41:39 +00:00
Irit Katriel c31e356a10
gh-100720: refactor calculation of number of frame slots for a code object into the new function _PyFrame_NumSlotsForCodeObject (#100722) 2023-01-04 13:37:06 +00:00
Kumar Aditya ba8dcdbcab
GH-86508: skip binding to local addresses of different family in `asyncio.open_connection` (#100615) 2023-01-04 13:30:26 +05:30
Ethan Furman a286caa937
gh-91219: http - use subclassing to override index_pages attribute (GH-100731)
Remove previously added parameter to `__init__`, and recommend subclassing to modify the `index_pages` attribute instead.
2023-01-03 15:20:08 -08:00
L. A. F. Pereira e6d4440782
gh-100146: Steal references from stack when building a list (#100147)
When executing the BUILD_LIST opcode, steal the references from the stack,
in a manner similar to the BUILD_TUPLE opcode.  Implement this by offloading
the logic to a new private API, _PyList_FromArraySteal(), that works similarly
to _PyTuple_FromArraySteal().

This way, instead of performing multiple stack pointer adjustments while the
list is being initialized, the stack is adjusted only once and a fast memory
copy operation is performed in one fell swoop.
2023-01-03 10:49:49 -08:00
Thomas Grainger b3722ca058
gh-95882: fix regression in the traceback of exceptions propagated from inside a contextlib context manager (#95883) 2023-01-03 15:47:13 +00:00
Eli Schwartz 8586949833
gh-89419: gdb: fix bug causing `AttributeError` in py-locals when no frame is available (#100611) 2023-01-03 20:26:57 +05:30
Stanislav Zmiev 73097d91a6
gh-89727: Improve os.walk complexity (#100671) 2023-01-02 13:41:19 -08:00
Ionite d7e7f79ca7
gh-100637: Fix int and bool __sizeof__ calculation to include the 1 element ob_digit array for 0 and False (#100663)
Fixes behaviour where int (and subtypes like bool) __sizeof__ under-reports true size as it did not take into account the size 1 `ob_digit` array for the zero int.

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-01-02 21:11:49 +00:00
Jason R. Coombs 447d061bc7
gh-97930: Apply changes from importlib_resources 5.10. (GH-100598) 2023-01-01 11:07:32 -05:00
Gabriele N. Tornetta d52d4942cf
gh-100649: Update native_thread_id after fork (gh-100650)
Update native_thread_id after fork
2023-01-01 23:20:38 +09:00
Shantanu e83f88a455
gh-100488: Add is_integer method to fractions.Fraction (#100489) 2023-01-01 01:44:48 -07:00
Éric 46521826cb
gh-95778: add doc missing in some places (GH-100627) 2022-12-30 13:21:15 -08:00
mathieui f4fcfdf8c5
gh-100616: Document 'attr' parameter for window.vline() in curses module (#24961)
Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
2022-12-30 08:35:04 -08:00
Nikita Sobolev 79c10b7da8
gh-99433: Fix `doctest` failure on `types.MethodWrapperType` (#99434) 2022-12-30 17:25:01 +05:30
Gregory P. Smith 894f2c3c16
gh-100228: Warn from os.fork() if other threads exist. (#100229)
Not comprehensive, best effort warning. There are cases when threads exist on some platforms that this code cannot detect. macOS when API permissions allow and Linux with a readable /proc procfs present are the currently supported cases where a warning should show up reliably.

Starting with a DeprecationWarning for now, it is less disruptive than something like RuntimeWarning and most likely to only be seen in people's CI tests - a good place to start with this messaging.
2022-12-29 14:41:39 -08:00
Zachary Ware 2df82db485
gh-100540: Remove obsolete Modules/_ctypes/darwin/ dlfcn shim (GH-100541)
As far as I can tell, this hasn't been actually used since Mac OS X 10.2.
2022-12-29 16:13:28 -06:00
Samet YASLAN f10f503b24
gh-100585: Fixed a bug where importlib.resources.as_file was leaving file pointers open (GH-100586)
* gh-100585: Fixed open fp bug in the imporlib module

* Added news for gh-100585
2022-12-28 16:31:43 -05:00
Hugo van Kemenade 9eca7235af
gh-94172: Update docs for params removed in 3.12 (#100431)
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
2022-12-28 14:43:19 -05:00
Zachary Ware 25590eb5de
gh-100540: Remove obsolete '--with-system-ffi' configure option (GH-100544)
It has had no effect on non-macOS platforms for a long time, and has had
the non-obvious effect of invoking `pkg_config` and not setting
`-DUSING_APPLE_OS_LIBFFI` on macOS since GH-22855.
2022-12-28 13:36:03 -06:00
Kumar Aditya 7cf164ad5e
GH-100342: check for allocation failure in AC `*args` parsing (#100343) 2022-12-28 09:46:28 +05:30
Pieter Eendebak b3da698952
gh-94603: micro optimize list.pop (gh-94604) 2022-12-27 19:55:54 +09:00
Nikita Sobolev ce39aaffee
gh-99509: Add `__class_getitem__` to `multiprocessing.queues.Queue` (#99511) 2022-12-26 20:50:55 -08:00
Jakub Kuczys 3ccc98fc24
Fix name of removed `inspect.Signature.from_builtin` method in 3.11.0a2 changelog (#100525) 2022-12-26 17:04:18 +05:30
Glyph b9aa14a484
gh-100519: simplification to `eff_request_host` in cookiejar.py (#99588)
`IPV4_RE` includes a `.`, and the `.find(".") == -1` included here is already testing to make sure there's no dot, so this part of the expression is tautological. Instead use more modern `in` syntax to make it clear what the check is doing here. The simplified implementation more clearly matches the wording in RFC 2965.

Co-authored-by: hauntsaninja <hauntsaninja@gmail.com>
2022-12-24 18:14:51 -06:00
Shantanu 046cbc2080
gh-100472: Fix docs claim that compileall parameters could be bytes (#100473) 2022-12-24 14:22:49 -06:00
Shantanu e4b43ebb3a
gh-100287: Fix unittest.mock.seal with AsyncMock (#100496) 2022-12-24 19:39:39 +00:00
James Frost 46e6a28308
gh-100474: Fix handling of dirs named index.html in http.server (GH-100475)
If you had a directory called index.html or index.htm within a directory, it would cause http.server to return a 404 Not Found error instead of the directory listing. This came about due to not checking that the index was a regular file.

I have also added a test case for this situation.

Automerge-Triggered-By: GH:merwok
2022-12-24 10:28:59 -08:00
Nikita Sobolev bdfb694386
gh-100357: Convert several functions in `bltinsmodule` to AC (#100358) 2022-12-24 06:45:47 -08:00
Kumar Aditya c122390a55
GH-91166: Implement zero copy writes for `SelectorSocketTransport` in asyncio (#31871)
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2022-12-24 11:21:11 +05:30
Shantanu 3e46f9fe05
gh-100268: Add is_integer method to int (#100439)
This improves the lives of type annotation users of `float` - which type checkers implicitly treat as `int|float` because that is what most code actually wants. Before this change a `.is_integer()` method could not be assumed to exist on things annotated as `: float` due to the method not existing on both types.
2022-12-23 18:30:27 -08:00
Illia Volochii a23cb72ac8
gh-100454: Start running SSL tests with OpenSSL 3.1.0-beta1 (#100456) 2022-12-23 18:04:20 -08:00
Sebastian Berg 474220e3a5
gh-99947: Ensure unreported errors are chained for SystemError during import (GH-99946) 2022-12-23 15:43:19 -08:00
Barney Gale a68e585c8b
gh-68320, gh-88302 - Allow for private `pathlib.Path` subclassing (GH-31691)
Users may wish to define subclasses of `pathlib.Path` to add or modify
existing methods. Before this change, attempting to instantiate a subclass
raised an exception like:

    AttributeError: type object 'PPath' has no attribute '_flavour'

Previously the `_flavour` attribute was assigned as follows:

    PurePath._flavour        = xxx not set!! xxx
    PurePosixPath._flavour   = _PosixFlavour()
    PureWindowsPath._flavour = _WindowsFlavour()

This change replaces it with a `_pathmod` attribute, set as follows:

    PurePath._pathmod        = os.path
    PurePosixPath._pathmod   = posixpath
    PureWindowsPath._pathmod = ntpath

Functionality from `_PosixFlavour` and `_WindowsFlavour` is moved into
`PurePath` as underscored-prefixed classmethods. Flavours are removed.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Brett Cannon <brett@python.org>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Eryk Sun <eryksun@gmail.com>
2022-12-23 14:52:23 -08:00
Raymond Hettinger 5d84966cce
GH-100425: Improve accuracy of builtin sum() for float inputs (GH-100426) 2022-12-23 14:35:58 -08:00
Filipe Laíns 1ecfd1ebf1
bpo-40447: accept all path-like objects in compileall.compile_file (#19883)
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Filipe Laíns <lains@riseup.net>
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2022-12-23 14:19:16 -06:00
Nikita Sobolev 745545b5bb
gh-99482: remove `jython` compatibility parts from stdlib and tests (#99484) 2022-12-23 14:17:24 -06:00
Carl Meyer c5726b727e
gh-83076: 3.8x speed improvement in (Async)Mock instantiation (#100252) 2022-12-23 19:41:37 +00:00
Dennis Sweeney a98d9ea56e
gh-94155: Reduce hash collisions for code objects (#100183)
* Uses a better hashing algorithm to get better dispersion and remove commutativity.

* Incorporates `co_firstlineno`, `Py_SIZE(co)`, and bytecode instructions.

* This is now the entire set of criteria used in `code_richcompare`, except for `_PyCode_ConstantKey` (which would incorporate the types of `co_consts` rather than just their values).
2022-12-23 13:15:47 -05:00
Ken Jin 36d358348d
Revert "gh-100288: Specialise LOAD_ATTR_METHOD for managed dictionaries (GH-100289)" (#100468)
This reverts commit c3c7848a48.
2022-12-24 01:48:43 +08:00
Ken Jin c3c7848a48
gh-100288: Specialise LOAD_ATTR_METHOD for managed dictionaries (GH-100289) 2022-12-24 00:26:42 +08:00
Pieter Eendebak 7fc7909677
gh-92216: improve performance of `hasattr` for type objects (GH-99979) 2022-12-23 23:23:36 +08:00
Bill Fisher 88d565f32a
gh-99110: Initialize `frame->previous` in init_frame to fix segmentation fault when accessing `frame.f_back` (#100182) 2022-12-23 20:15:53 +05:30
Eric Wieser 84bc6a4f25
gh-76963: PEP3118 itemsize of an empty ctypes array should not be 0 (GH-5576)
The itemsize returned in a memoryview of a ctypes array is now computed from the item type, instead of dividing the total size by the length and assuming that the length is not zero.
2022-12-23 08:23:19 +00:00
Alex Waygood 9cdb642997
gh-85432: Harmonise parameter names between C and pure-Python implementations of `datetime.time.strftime`, `datetime.datetime.fromtimestamp` (#99993) 2022-12-22 15:57:18 -06:00
Itamar Ostricher 4cc63e0d4e
gh-100344: Add C implementation for `asyncio.current_task` (#100345)
Co-authored-by: pranavtbhat
2022-12-22 19:08:12 +05:30
Brandt Bucher 3c033a2e6f
GH-99554: Pack location tables more effectively (GH-99556) 2022-12-21 16:41:18 -08:00
amaajemyfren bbf4a66a18
GH-69564: Clarify use of octal format of mode argument in help(os.chmod) (#20621)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-12-21 23:18:25 +05:30
Hai Shi d713c54ac8
gh-78878: Fix crash when creating an instance of `_ctypes.CField` (#14837) 2022-12-21 23:01:17 +05:30
Dominic Socular 12be23cf3c
gh-100374: Fixed a bug in socket.getfqdn() (gh-100375) 2022-12-21 22:25:04 +09:00
Kumar Aditya a7715ccfba
GH-100133: fix `asyncio` subprocess losing `stderr` and `stdout` output (#100154) 2022-12-21 15:20:26 +05:30
Kumar Aditya 4994f2488f
GH-100363: Speed up `asyncio.get_running_loop` (#100364) 2022-12-21 15:19:44 +05:30
Shantanu 79311cbfe7
gh-85267: Improvements to inspect.signature __text_signature__ handling (#98796)
This makes a couple related changes to inspect.signature's behaviour
when parsing a signature from `__text_signature__`.

First, `inspect.signature` is documented as only raising ValueError or
TypeError. However, in some cases, we could raise RuntimeError.  This PR
changes that, thereby fixing #83685.

(Note that the new ValueErrors in RewriteSymbolics are caught and then
reraised with a message)

Second, `inspect.signature` could randomly drop parameters that it
didn't understand (corresponding to `return None` in the `p` function).
This is the core issue in #85267. I think this is very surprising
behaviour and it seems better to fail outright.

Third, adding this new failure broke a couple tests. To fix them (and to
e.g. allow `inspect.signature(select.epoll.register)` as in #85267), I
add constant folding of a couple binary operations to RewriteSymbolics.

(There's some discussion of making signature expression evaluation
arbitrary powerful in #68155. I think that's out of scope. The
additional constant folding here is pretty straightforward, useful, and
not much of a slippery slope)

Fourth, while #85267 is incorrect about the cause of the issue, it turns
out if you had consecutive newlines in __text_signature__, you'd get
`tokenize.TokenError`.

Finally, the `if name is invalid:` code path was dead, since
`parse_name` never returned `invalid`.
2022-12-20 19:29:02 -08:00
Dennis Sweeney c18d831188
gh-100188: Reduce misses in BINARY_SUBSCR_(LIST/TUPLE)_INT (#100189)
Don't specialize if the index is negative.
2022-12-20 15:46:16 -05:00
Nikita Sobolev 44892d45b0
gh-99576: Fix cookiejar file that was not truncated for some classes (GH-99616)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-12-20 17:51:26 +01:00
František Nesveda d98ca8172c
gh-99925: Fix inconsistency in `json.dumps()` error messages (GH-99926) 2022-12-20 12:54:56 +02:00
Richard Kojedzinszky a6331b605e
gh-100348: Fix ref cycle in `asyncio._SelectorSocketTransport` with `_read_ready_cb` (#100349) 2022-12-20 16:10:56 +05:30
Thomas Grainger 79ccc03b62
gh-89051: Add ssl.OP_LEGACY_SERVER_CONNECT (#93927)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Fixes https://github.com/python/cpython/issues/89051
2022-12-20 09:10:30 +02:00
Jon Burdo 797edb28c3
gh-89727: Fix os.walk RecursionError on deep trees (#99803)
Use a stack to implement os.walk iteratively instead of recursively to
avoid hitting recursion limits on deeply nested trees.
2022-12-19 10:59:01 -08:00
Carlton Gibson 532aa4e4e0
gh-94912: Added marker for non-standard coroutine function detection (#99247)
This introduces a new decorator `@inspect.markcoroutinefunction`,
which, applied to a sync function, makes it appear async to
`inspect.iscoroutinefunction()`.
2022-12-18 11:13:24 -08:00
Serhiy Storchaka 0fe61d0838
gh-100272: Fix JSON serialization of OrderedDict (GH-100273)
It now preserves the order of keys.
2022-12-17 14:32:48 +02:00
colorfulappl efbb1eb9f5
gh-99240: Reset pointer to NULL when the pointed memory is freed in argument parsing (#99890)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2022-12-17 12:07:44 +05:30
Barney Gale 5a991da329
gh-78707: deprecate passing >1 argument to `PurePath.[is_]relative_to()` (GH-94469)
This brings `relative_to()` and `is_relative_to()` more in line with other pathlib methods like `rename()` and `symlink_to()`.

Resolves #78707.
2022-12-16 16:14:27 -08:00
yonillasky 432117cd1f
gh-99540: Constant hash for _PyNone_Type to aid reproducibility (GH-99541)
Needed for ASLR builds of Python.
2022-12-16 12:36:13 -08:00
Ethan Furman a5a7cea202
gh-100039: enhance __signature__ to work with str and callables (GH-100168)
Callables should be either class- or static-methods.
Enum now uses the classmethod version to greatly improve the help
given for enums and flags.
2022-12-16 12:30:47 -08:00
Brandt Bucher 9076455d1b
GH-90043: Handle NaNs in COMPARE_OP_FLOAT_JUMP (GH-100278) 2022-12-16 10:18:31 -08:00
Ateeq Sharfuddin 101cfe679f
gh-78997: AttributeError if loading fails in LibraryLoader.__getattr__
Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
Co-authored-by: Filipe Laíns <filipe.lains@gmail.com>
Co-authored-by: Eric Wieser <wieser.eric@gmail.com>
2022-12-15 22:59:37 +00:00
Raymond Hettinger b430399d41
GH-100234: Set a default value for random.expovariate() (GH-100235) 2022-12-15 12:40:45 -06:00
Mark Shannon 6997e77bdf
GH-100222: Redefine _Py_CODEUNIT as a union to clarify structure of code unit. (GH-100223) 2022-12-14 11:12:53 +00:00
Michael Droettboom 1583c6e326
GH-100143: Improve collecting pystats for parts of runs (GH-100144)
* pystats off by default

* Add -Xpystats flag

* Always dump pystats, even if turned off
2022-12-12 14:50:43 +00:00
Irit Katriel e4ea33b178
gh-99955: standardize return values of functions in compiler's code-gen (#100010) 2022-12-12 14:22:15 +00:00
GalaxySnail 158b8a0721
gh-79218: Define `MS_WIN64` macro for Mingw-w64 64bit on Windows (GH-100137) 2022-12-12 13:39:23 +00:00
Gregory P. Smith 2e279e85fe
gh-88500: Reduce memory use of `urllib.unquote` (#96763)
`urllib.unquote_to_bytes` and `urllib.unquote` could both potentially generate `O(len(string))` intermediate `bytes` or `str` objects while computing the unquoted final result depending on the input provided. As Python objects are relatively large, this could consume a lot of ram.

This switches the implementation to using an expanding `bytearray` and a generator internally instead of precomputed `split()` style operations.

Microbenchmarks with some antagonistic inputs like `mess = "\u0141%%%20a%fe"*1000` show this is 10-20% slower for unquote and unquote_to_bytes and no different for typical inputs that are short or lack much unicode or % escaping. But the functions are already quite fast anyways so not a big deal.  The slowdown scales consistently linear with input size as expected.

Memory usage observed manually using `/usr/bin/time -v` on `python -m timeit` runs of larger inputs. Unittesting memory consumption is difficult and does not seem worthwhile.

Observed memory usage is ~1/2 for `unquote()` and <1/3 for `unquote_to_bytes()` using `python -m timeit -s 'from urllib.parse import unquote, unquote_to_bytes; v="\u0141%01\u0161%20"*500_000' 'unquote_to_bytes(v)'` as a test.
2022-12-10 16:17:39 -08:00
DarioDaF 1bb68ba6d9
gh-99941: Ensure that asyncio.Protocol.data_received receives immutable bytes (#100053) 2022-12-10 15:07:02 -08:00
Kai Zhang 228c92eb5c
gh-99582: freeze `zipimport` into `_bootstrap_python` (#99583)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-12-10 15:05:56 +05:30
andrei kulakov d0679c1239
bpo-44512: Fix handling of extrasactions arg to csv.DictWriter with mixed or upper case (#26924) 2022-12-09 16:14:33 +00:00
Shreyan Avigyan a29a7b9b78
bpo-43984: Allow winreg.SetValueEx to set -1 without treating it as an error (GH-25775) 2022-12-09 12:47:18 +00:00
Mark Shannon fb713b2183
GH-98522: Add version number to code objects. (GH-98525)
* Add version number to code object for better versioning of functions.

* Improves specialization for closures and list comprehensions.
2022-12-09 12:18:45 +00:00
Christian Rendina 3c5355496b
gh-88267: Avoid DLL exporting functions from static builds on Windows(GH-99888) 2022-12-09 11:16:15 +00:00
Ken Jin 748c6c0921
GH-100110: Specialize FOR_ITER for tuples (GH-100109)
* Specialize FOR_ITER for tuples
2022-12-09 10:27:01 +00:00
Ethan Furman ded02ca54d
gh-100098: [Enum] insist on actual tuples, no subclasses, for auto (GH-100099)
When checking for auto() instances, only top-level usage is supported,
which means either alone or as part of a regular tuple. Other
containers, such as lists, dicts, or namedtuples, will not have auto()
transformed into a value.
2022-12-07 22:58:08 -08:00
Matthieu Baerts cce8362960
gh-98030: socket: add missing TCP socket options (#98031)
A few TCP socket options have been added to the Linux kernel these last
few years.

This commit adds all the ones available in Linux 6.0:

  https://elixir.bootlin.com/linux/v6.0/source/include/uapi/linux/tcp.h#L91

While at it, the TCP_FASTOPEN option has been moved lower in the list
just to keep the same order as in tcp.h to ease future synchronisations.

Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
2022-12-07 20:59:37 -08:00
Takeshi KOMIYA 90d5c9b195
gh-92120: The docstring of enum.Enum is invalid in reST (GH-92122)
Closes #92120
2022-12-07 18:24:52 -08:00
Dong-hee Na dc8a86893d
gh-98778: Update HTTPError to initialize properly even if fp is None (gh-99966) 2022-12-08 11:20:34 +09:00
Victor Stinner 3c89202247
gh-100086: Add build info to test.libregrtest (#100093)
The Python test runner (libregrtest) now logs Python build information like
"debug" vs "release" build, or LTO and PGO optimizations.
2022-12-08 01:38:47 +01:00
Carl Meyer 68e41295b8
gh-83035: handle decorator with nested parens in inspect.getsource (#99654) 2022-12-07 08:55:12 -08:00
Victor Stinner b11a384dc7
gh-98724: Fix Py_CLEAR() macro side effects (#99100) (#100070)
The Py_CLEAR(), Py_SETREF() and Py_XSETREF() macros now only evaluate
their arguments once. If an argument has side effects, these side
effects are no longer duplicated.

Use temporary variables to avoid duplicating side effects of macro
arguments. If available, use _Py_TYPEOF() to avoid type punning.
Otherwise, use memcpy() for the assignment to prevent a
miscompilation with strict aliasing caused by type punning.

Add _Py_TYPEOF() macro: __typeof__() on GCC and clang.

Add test_py_clear() and test_py_setref() unit tests to _testcapi.
2022-12-07 15:22:38 +01:00
Matěj Cepl 7031275776
gh-93018: Fix for the compatibility problems with expat (gh-93900) 2022-12-07 15:55:49 +09:00
Pablo Galindo Salgado 97e7004cfe
gh-100050: Fix an assertion error when raising unclosed parenthesis errors in the tokenizer (GH-100065)
Automerge-Triggered-By: GH:pablogsal
2022-12-06 15:09:56 -08:00
Ethan Furman 679efbb080
gh-94943: [Enum] improve repr() when inheriting from a dataclass (GH-99740)
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
2022-12-06 13:43:41 -08:00
Thomas Wouters b6bd7ffcbc Python 3.12.0a3 2022-12-06 19:33:02 +01:00
Serhiy Storchaka fd38a2f0ec
gh-93453: No longer create an event loop in get_event_loop() (#98440)
asyncio.get_event_loop() now always return either running event loop or
the result of get_event_loop_policy().get_event_loop() call. The latter
should now raise an RuntimeError if no current event loop was set
instead of creating and setting a new event loop.

It affects also a number of asyncio functions and constructors which
call get_event_loop() implicitly: ensure_future(), shield(), gather(),
etc.

DeprecationWarning is no longer emitted if there is no running event loop but
the current event loop was set.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-12-06 18:42:12 +01:00
Brandt Bucher b72014c783
GH-99729: Unlink frames before clearing them (GH-100030) 2022-12-06 14:01:38 +00:00
Erik De Bonte 5c19050546
gh-99957: Add `frozen_default` parameter on `dataclass_transform` (#99958) 2022-12-05 19:35:43 -08:00
Gregory P. Smith 7e29398407
gh-100001: Also escape \s in http.server log messages. (#100038)
Also \ escape \s in the http.server BaseHTTPRequestHandler.log_message so
that it is technically possible to parse the line and reconstruct what the
original data was.  Without this a \xHH is ambiguious as to if it is a hex
replacement we put in or the characters r"\x" came through in the original
request line.
2022-12-05 14:27:55 -08:00
Hugo van Kemenade f49c735e52
Use sphinxext-opengraph to generate OpenGraph metadata (#99931)
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
2022-12-05 23:26:28 +02:00
Gregory P. Smith d8ab0a4dfa
gh-100001: Omit control characters in http.server stderr logs. (#100002)
Replace control characters in http.server.BaseHTTPRequestHandler.log_message with an escaped \xHH sequence to avoid causing problems for the terminal the output is printed to.
2022-12-05 12:55:45 -08:00
Eric Snow 530cc9dbb6
gh-99741: Implement Multi-Phase Init for the _xxsubinterpreters Module (gh-99742)
_xxsubinterpreters is an internal module used for testing.

https://github.com/python/cpython/issues/99741
2022-12-05 13:40:20 -07:00
Michael Förderer 124ecd6576
bpo-44817: Ignore additional errors in ntpath.realpath (GH-27574) 2022-12-05 17:34:00 +00:00
Petr Viktorin f24738742c
gh-98680: Add PyBUF_* constants to the Limited API listing (GH-100018)
``PyBUF_*`` constants are marked as part of Limited API of Python 3.11+.
These were available in 3.11.0 with `Py_LIMITED_API` defined for 3.11,
and are necessary to use the buffer API. Omitting them in `stable_abi.toml`
was a mistake.
2022-12-05 18:02:36 +01:00
Victor Stinner 2488c1e1b6
gh-99892: test_unicodedata: skip test on download failure (#100011)
Skip test_normalization() of test_unicodedata if it fails to download
NormalizationTest.txt file from pythontest.net.
2022-12-05 16:37:40 +01:00
Felix Ye 854a878e4f
gh-98248: Normalizing the error messages in function struct.pack (GH-98252)
Provide consistent and informative error messages in function struct.pack when its integral arguments are not in range.
2022-12-04 20:24:18 +00:00
AlexTate 72ec518203
gh-98458: unittest: bugfix for infinite loop while handling chained exceptions that contain cycles (#98459)
* Bugfix addressing infinite loop while handling self-referencing chained exception in TestResult._clean_tracebacks()
* Bugfix extended to properly handle exception cycles in _clean_tracebacks. The "seen" set follows the approach used in the TracebackException class (thank you @iritkatriel for pointing it out)
* adds a test for a single chained exception that holds a self-loop in its __cause__ and __context__ attributes
2022-12-04 11:37:55 -08:00
Serhiy Storchaka a87c46eab3
bpo-15999: Accept arbitrary values for boolean parameters. (#15609)
builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
2022-12-03 11:52:21 -08:00
Alexander Kanavin c68573b339
gh-99934: test_marshal.py: add more elements in test_deterministic_sets (GH-99935)
Existing elements do produce different output on x86_64, but they
do not on x86. Let's make the data longer to ensure it differs.
2022-12-03 10:53:20 -06:00
Itamar Ostricher 3c137dc613
GH-91054: Add code object watchers API (GH-99859)
* Add API to allow extensions to set callback function on creation and destruction of PyCodeObject

Co-authored-by: Ye11ow-Flash <janshah@cs.stonybrook.edu>
2022-12-02 17:28:27 +00:00
Serhiy Storchaka f08e52ccb0
gh-99612: Fix PyUnicode_DecodeUTF8Stateful() for ASCII-only data (GH-99613)
Previously *consumed was not set in this case.
2022-12-01 14:54:51 +02:00
Serhiy Storchaka 787764219f
gh-89189: More compact range iterator (GH-27986) 2022-11-30 23:04:30 +02:00
Géry Ogam 9628136fac
bpo-47220: Document the optional callback parameter of weakref.WeakMethod (GH-25491) 2022-11-30 13:44:10 -06:00
Pablo Galindo Salgado 417206a05c
gh-99891: Fix infinite recursion in the tokenizer when showing warnings (GH-99893)
Automerge-Triggered-By: GH:pablogsal
2022-11-30 03:36:06 -08:00
Wenzel Jakob c314198fad
gh-98253: Break potential reference cycles in external code worsened by typing.py lru_cache (#98591) 2022-11-30 08:33:32 +00:00
Dong-hee Na 8bb2303fd7
gh-99127: Allow some features of syslog to the main interpreter only (gh-99128) 2022-11-30 07:58:20 +09:00