Commit Graph

14021 Commits

Author SHA1 Message Date
Victor Stinner 113053a070
gh-110850: Fix _PyTime_FromSecondsDouble() API (#116606)
Return 0 on success. Set an exception and return -1 on error.

Fix os.timerfd_settime(): properly report exceptions on
_PyTime_FromSecondsDouble() failure.

No longer export _PyTime_FromSecondsDouble().
2024-03-11 16:35:29 +00:00
Victor Stinner 546eb7a3be
gh-116417: Build _testinternalcapi with limited C API version 3.5 (#116598) 2024-03-11 15:20:04 +01:00
Nikita Sobolev ffd79bea0f
gh-116545: Fix error handling in `mkpwent` in `pwdmodule` (#116548) 2024-03-11 13:58:24 +03:00
Victor Stinner 1cc02ca063
gh-116417: Move 4 limited C API test files to _testlimitedcapi (#116571)
Move the following files from Modules/_testcapi/ to
Modules/_testlimitedcapi/:

* bytearray.c
* bytes.c
* pyos.c
* sys.c

Changes:

* Replace PyBytes_AS_STRING() with PyBytes_AsString().
* Replace PyBytes_GET_SIZE() with PyBytes_Size().
* Update related test_capi tests.
* Copy Modules/_testcapi/util.h to Modules/_testlimitedcapi/util.h.
2024-03-11 10:28:16 +00:00
Victor Stinner c5fa796619
gh-116417: Fix make check-c-globals for _testlimitedcapi (#116570)
* Remove unused '_testcapimodule' global in Modules/_testcapi/unicode.c.
* Update c-analyzer to not use the internal C API in
  _testlimitedcapi.c.
2024-03-10 20:19:47 +00:00
Victor Stinner 729bfb3105
gh-116417: Avoid PyFloat_AS_DOUBLE() in AC limited C API (#116568)
Argument Clinic no longer calls PyFloat_AS_DOUBLE() when the usage of
the limited C API is requested.
2024-03-10 20:42:40 +01:00
Nikita Sobolev b4b4e764a7
gh-116520: Fix error handling in `os_get_terminal_size_impl` in `posixmodule` (#116521) 2024-03-09 14:18:38 +03:00
Nikita Sobolev fdb2d90a27
gh-116447: Fix possible UB in `arraymodule` and `getargs` (#116459) 2024-03-08 13:49:52 +03:00
Erlend E. Aasland d864b0094f
gh-116303: Explicitly check for the _testsinglephase module in configure.ac (#116479) 2024-03-07 23:42:43 +00:00
Victor Stinner d9bcdda39c
gh-116417: Add _testlimitedcapi C extension (#116419)
Add a new C extension "_testlimitedcapi" which is only built with the
limited C API.

Move heaptype_relative.c and vectorcall_limited.c from
Modules/_testcapi/ to Modules/_testlimitedcapi/.

* configure: add _testlimitedcapi test extension.
* Update generate_stdlib_module_names.py.
* Update make check-c-globals.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-07 18:31:12 +00:00
Serhiy Storchaka 72d3cc94cd
gh-116437: Use new C API PyDict_Pop() to simplify the code (GH-116438) 2024-03-07 11:21:08 +02:00
Nikita Sobolev 882fcede83
gh-116448: Handle errors correctly in `os_waitid_impl` in `posixmodule` (#116449) 2024-03-07 08:28:48 +00:00
Nikita Sobolev 40b79efae7
gh-116386: Fix format string "%ld" warning in `_xxinterpqueuesmodule` (#116387) 2024-03-07 09:44:26 +03:00
Russell Keith-Magee b33980a2e3
gh-114099 - Add iOS testbed, plus Makefile target to invoke it. (gh-115930) 2024-03-06 23:24:52 -05:00
mpage c62144a02c
gh-114271: Make `_thread.lock` thread-safe in free-threaded builds (#116433)
Previously, the `locked` field was set after releasing the lock. This reverses
the order so that the `locked` field is set while the lock is still held.

There is still one thread-safety issue where `locked` is checked prior to
releasing the lock, however, in practice that will only be an issue when
unlocking the lock is contended, which should be rare.
2024-03-06 15:46:36 -05:00
Nikita Sobolev 22ccf13b33
gh-116404: Handle errors correctly in `wait_helper` in `posixmodule` (#116405) 2024-03-06 08:46:47 +00:00
Eric Snow dab85e0189
gh-76785: Use PRId64 to Fix a Compiler Warning on Windows (gh-116369)
I accidentally introduced the warning in gh-116328.
2024-03-05 18:51:04 +00:00
Eric Snow 4402b3cbcf
gh-76785: Minor Improvements to "interpreters" Module (gh-116328)
This includes adding pickle support to various classes, and small changes to improve the maintainability of the low-level _xxinterpqueues module.
2024-03-05 08:54:46 -07:00
Eric Snow eb22e2b251
gh-115490: Make the interpreter.channels and interpreter.queues Modules Handle Reloading Properly (gh-115493)
The problem manifested when the .py module got reloaded and the corresponding extension module didn't. The .py module registers types with the extension and the extension was not allowing that to happen more than once. The solution: let it happen more than once.
2024-03-04 20:59:30 +00:00
Eric Snow 01440d3a39
gh-76785: Simplify Channels XID Types (gh-116318)
I had added an extra cleanup abstraction a while back that has turned out to be unnecessary.
2024-03-04 19:32:39 +00:00
Yuriy Chernyshov 9b9e819b51
gh-116116: Backport blake2 change to fix building with clang-cl on windows-i686 (GH-116117) 2024-03-04 16:59:57 +00:00
Raymond Hettinger 15dc2979bc
Consistently spell out *predicate* instead of *pred*. (gh-116308) 2024-03-04 15:51:29 +00:00
Sergey B Kirpichev 002a5948fc
gh-108562: Fix compiler warnings for libmpdec (#114751)
If awailable, enable -fstrict-overflow for libmpdec. Also
shut off false positive warnings (-Warray-bounds).

The later was backported from mpdecimal-4.0.0.
2024-03-03 08:25:39 +01:00
Eric Snow cad3745b87
gh-116102: Silence a Compiler Warning in _xxinterpqueues (gh-116230) 2024-03-02 00:28:05 +00:00
mpage 9e88173d36
gh-114271: Make `_thread.ThreadHandle` thread-safe in free-threaded builds (GH-115190)
Make `_thread.ThreadHandle` thread-safe in free-threaded builds

We protect the mutable state of `ThreadHandle` using a `_PyOnceFlag`.
Concurrent operations (i.e. `join` or `detach`) on `ThreadHandle` block
until it is their turn to execute or an earlier operation succeeds.
Once an operation has been applied successfully all future operations
complete immediately.

The `join()` method is now idempotent. It may be called multiple times
but the underlying OS thread will only be joined once. After `join()`
succeeds, any future calls to `join()` will succeed immediately.

The internal thread handle `detach()` method has been removed.
2024-03-01 13:43:12 -08:00
Malcolm Smith e6e35327d8
gh-115773: Add missing preprocessor guard in _testexternalinspection (#116212)
Add missing preprocessor guard in _testexternalinspection
2024-03-01 17:50:48 +00:00
Brett Simmers 2e94a6687c
gh-116099: Fix refcounting bug in `_queueobj_shared()` (gh-116164)
This code decrefs `qidobj` twice in some paths. Since `qidobj` isn't used after
the first `Py_DECREF()`, remove the others, and replace the `Py_DECREF()` with
`Py_CLEAR()` to make it clear that the variable is dead.

With this fix, `python -mtest test_interpreters -R 3:3 -mtest_queues` no longer
fails with `_Py_NegativeRefcount: Assertion failed: object has negative ref
count`.
2024-03-01 01:02:12 +00:00
AN Long ca56c3a172
gh-103092: Add a mutex to make the PRNG state of rotatingtree concurrent-safe (#115301) 2024-03-01 00:04:16 +01:00
Sebastian Pipping 6a95676bb5
gh-115398: Expose Expat >=2.6.0 reparse deferral API (CVE-2023-52425) (GH-115623)
Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:

- `xml.etree.ElementTree.XMLParser.flush`
- `xml.etree.ElementTree.XMLPullParser.flush`
- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`
- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`
- `xml.sax.expatreader.ExpatParser.flush`

Based on the "flush" idea from https://github.com/python/cpython/pull/115138#issuecomment-1932444270 .

### Notes

- Please treat as a security fix related to CVE-2023-52425.

Includes code suggested-by: Snild Dolkow <snild@sony.com>
and by core dev Serhiy Storchaka.
2024-02-29 14:52:50 -08:00
Malcolm Smith fa1d675309
gh-71052: Fix several Android build issues (#115955)
This change is part of the work on PEP-738: Adding Android as a 
supported platform.

* Remove the "1.0" suffix from libpython's filename on Android, which 
  would prevent Gradle from packaging it into an app. 
* Simplify the build command in the Makefile so that libpython always 
  gets given an SONAME with the `-Wl-h` argument, even if the SONAME is
  identical to the actual filename.
* Disable a number of functions on Android which can be compiled and 
  linked against, but always fail at runtime. As a result, the native
  _multiprocessing module is no longer built for Android.
* gh-115390 (bee7bb331) added some pre-determined results to the 
  configure script for things that can't be autodetected when
  cross-compiling; this change adds Android to these where appropriate.
* Add a couple more pre-determined results for Android, and making them 
  cover iOS as well. This means the --enable-ipv6 configure option will 
  no longer be required on either platform.
2024-02-29 22:58:20 +01:00
Eric Snow e80abd57a8
gh-76785: Update test.support.interpreters to Align With PEP 734 (gh-115566)
This brings the code under test.support.interpreters, and the corresponding extension modules, in line with recent updates to PEP 734.

(Note: PEP 734 has not been accepted at this time.  However, we are using an internal copy of the implementation in the test suite to exercise the existing subinterpreters feature.)
2024-02-28 16:08:08 -07:00
Sam Gross df5212df6c
gh-112529: Simplify PyObject_GC_IsTracked and PyObject_GC_IsFinalized (#114732) 2024-02-28 15:37:59 -05:00
Pablo Galindo Salgado 1752b51012
gh-115773: Add tests to exercise the _Py_DebugOffsets structure (#115774) 2024-02-28 10:17:34 +00:00
Mark Shannon 6ecfcfe894
GH-115816: Assorted naming and formatting changes to improve maintainability. (GH-115987)
* Rename _Py_UOpsAbstractInterpContext to _Py_UOpsContext and _Py_UOpsSymType to _Py_UopsSymbol.

* #define shortened form of _Py_uop_... names for improved readability.
2024-02-27 13:25:02 +00:00
Mark Shannon 10fbcd6c5d
GH-115816: Make tier2 optimizer symbols testable, and add a few tests. (GH-115953) 2024-02-27 10:51:26 +00:00
Michael Droettboom b05afdd5ec
gh-115168: Add pystats counter for invalidated executors (GH-115169) 2024-02-26 17:51:47 +00:00
Yuriy Chernyshov 96c10c6485
gh-115882: Reference Unknwn.h for ctypes on Windows (GH-115350)
This allows the module to be compiled with WIN32_LEAN_AND_MEAN enabled
2024-02-26 17:21:55 +00:00
Furkan Onder 8f5be78bce
gh-72249: Include the module name in the repr of partial object (GH-101910)
Co-authored-by: Anilyka Barry <vgr255@live.ca>
2024-02-25 22:55:19 +02:00
Serhiy Storchaka 79811ededd
gh-115886: Handle embedded null characters in shared memory name (GH-115887)
shm_open() and shm_unlink() now check for embedded null characters in
the name and raise an error instead of silently truncating it.
2024-02-25 11:31:03 +02:00
Serhiy Storchaka c688c0f130
gh-67044: Always quote or escape \r and \n in csv.writer() (GH-115741) 2024-02-23 22:25:09 +02:00
Petr Viktorin 8aa372edcd
gh-115714: Don't use CLOCK_PROCESS_CPUTIME_ID and times() on WASI (GH-115757)
* gh-115714: Don't use CLOCK_PROCESS_CPUTIME_ID and times() on WASI

* Add blurb
2024-02-22 12:39:45 +01:00
Malcolm Smith 7f5e3f04f8
gh-111225: Link extension modules against libpython on Android (#115780)
Part of the work on PEP 738: Adding Android as a supported platform.

* Rename the LIBPYTHON variable to MODULE_LDFLAGS, to more accurately 
  reflect its purpose.
* Edit makesetup to use MODULE_LDFLAGS when linking extension modules.
* Edit the Makefile so that extension modules depend on libpython on 
  Android and Cygwin.
* Restore `-fPIC` on Android. It was removed several years ago with a 
  note that the toolchain used it automatically, but this is no longer
  the case. Omitting it causes all linker commands to fail with an error
  like `relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used against
  symbol '_Py_FalseStruct'; recompile with -fPIC`.
2024-02-21 23:18:57 +00:00
Victor Stinner e4c34f04a1
gh-110850: Cleanup PyTime API: PyTime_t are nanoseconds (#115753)
PyTime_t no longer uses an arbitrary unit, it's always a number of
nanoseconds (64-bit signed integer).

* Rename _PyTime_FromNanosecondsObject() to _PyTime_FromLong().
* Rename _PyTime_AsNanosecondsObject() to _PyTime_AsLong().
* Remove pytime_from_nanoseconds().
* Remove pytime_as_nanoseconds().
* Remove _PyTime_FromNanoseconds().
2024-02-21 11:46:00 +01:00
Victor Stinner 77430b6a32
gh-110850: Replace private _PyTime_MAX with public PyTime_MAX (#115751)
Remove references to the old names _PyTime_MIN
and _PyTime_MAX, now that PyTime_MIN and
PyTime_MAX are public.

Replace also _PyTime_MIN with PyTime_MIN.
2024-02-21 08:11:40 +00:00
Victor Stinner 145bc2d638
gh-110850: Use public PyTime functions (#115746)
Replace private _PyTime functions with public PyTime functions.

random_seed_time_pid() now reports errors to its caller.
2024-02-20 23:31:30 +00:00
Victor Stinner 52d1477566
gh-110850: Rename internal PyTime C API functions (#115734)
Rename functions:

* _PyTime_GetSystemClock() => _PyTime_TimeUnchecked()
* _PyTime_GetPerfCounter() => _PyTime_PerfCounterUnchecked()
* _PyTime_GetMonotonicClock() => _PyTime_MonotonicUnchecked()
* _PyTime_GetSystemClockWithInfo() => _PyTime_TimeWithInfo()
* _PyTime_GetMonotonicClockWithInfo() => _PyTime_MonotonicWithInfo()
* _PyTime_GetMonotonicClockWithInfo() => _PyTime_MonotonicWithInfo()

Changes:

* Remove "typedef PyTime_t PyTime_t;" which was
  "typedef PyTime_t _PyTime_t;" before a previous rename.
* Update comments of "Unchecked" functions.
* Remove invalid PyTime_Time() comment.
2024-02-20 22:16:37 +00:00
Victor Westerhuis e1fdc3c323
gh-104061: Add socket.SO_BINDTOIFINDEX constant (GH-104062)
Add socket.SO_BINDTOIFINDEX constant

This socket option avoids a race condition between SO_BINDTODEVICE and network interface renaming.
2024-02-20 23:08:15 +02:00
Guido van Rossum 142502ea8d
Tier 2 cleanups and tweaks (#115534)
* Rename `_testinternalcapi.get_{uop,counter}_optimizer` to `new_*_optimizer`
* Use `_PyUOpName()` instead of` _PyOpcode_uop_name[]`
* Add `target` to executor iterator items -- `list(ex)` now returns `(opcode, oparg, target, operand)` quadruples
* Add executor methods `get_opcode()` and `get_oparg()` to get `vmdata.opcode`, `vmdata.oparg`
* Define a helper for printing uops, and unify various places where they are printed
* Add a hack to summarize_stats.py to fix legacy uop names (e.g. `POP_TOP` -> `_POP_TOP`)
* Define helpers in `test_opt.py` for accessing the set or list of opnames of an executor
2024-02-20 20:24:35 +00:00
Victor Stinner d207c7cd5a
gh-110850: Cleanup pycore_time.h includes (#115724)
<pycore_time.h> include is no longer needed to get the PyTime_t type
in internal header files. This type is now provided by <Python.h>
include. Add <pycore_time.h> includes to C files instead.
2024-02-20 16:50:43 +00:00
Serhiy Storchaka 937d282150
gh-115712: Support CSV dialects with delimiter=' ' and skipinitialspace=True (GH-115721)
Restore support of such combination, disabled in gh-113796.

csv.writer() now quotes empty fields if delimiter is a space and
skipinitialspace is true and raises exception if quoting is not possible.
2024-02-20 18:09:50 +02:00