Commit Graph

13952 Commits

Author SHA1 Message Date
Steve Dower ea25f32d5f
gh-89240: Enable multiprocessing on Windows to use large process pools (GH-107873)
We add _winapi.BatchedWaitForMultipleObjects to wait for larger numbers of handles.
This is an internal module, hence undocumented, and should be used with caution.
Check the docstring for info before using BatchedWaitForMultipleObjects.
2024-02-13 00:28:35 +00:00
Steve Dower 7861dfd26a
gh-111140: Adds PyLong_AsNativeBytes and PyLong_FromNative[Unsigned]Bytes functions (GH-114886) 2024-02-12 20:13:13 +00:00
mpage de7d67b19b
gh-114271: Make `PyInterpreterState.threads.count` thread-safe in free-threaded builds (gh-115093)
Use atomics to mutate PyInterpreterState.threads.count.
2024-02-12 10:44:00 -07:00
Petr Viktorin 879f4546bf
gh-110850: Add PyTime_t C API (GH-115215)
* gh-110850: Add PyTime_t C API

Add PyTime_t API:

* PyTime_t type.
* PyTime_MIN and PyTime_MAX constants.
* PyTime_AsSecondsDouble(), PyTime_Monotonic(),
  PyTime_PerfCounter() and PyTime_GetSystemClock() functions.

Co-authored-by: Victor Stinner <vstinner@python.org>
2024-02-12 18:13:10 +01:00
Nikita Sobolev 91bf01d4b1
gh-87804: Fix the refleak in error handling of `_pystatvfs_fromstructstatfs` (#115335)
It was the macro expansion! Sorry!
2024-02-12 19:27:27 +03:00
Kirill Podoprigora 93ac78ac3e
gh-115058: Add ``reset_rare_event_counters`` function in `_testinternalcapi` (GH-115128) 2024-02-12 16:05:30 +00:00
Eugene Toder 46190d9ea8
gh-89039: Call subclass constructors in datetime.*.replace (GH-114780)
When replace() method is called on a subclass of datetime, date or time,
properly call derived constructor. Previously, only the base class's
constructor was called.

Also, make sure to pass non-zero fold values when creating subclasses in
various methods. Previously, fold was silently ignored.
2024-02-12 14:44:56 +02:00
John Belmonte 72340d15cd
gh-114563: C decimal falls back to pydecimal for unsupported format strings (GH-114879)
Immediate merits:
* eliminate complex workarounds for 'z' format support
  (NOTE: mpdecimal recently added 'z' support, so this becomes
  efficient in the long term.)
* fix 'z' format memory leak
* fix 'z' format applied to 'F'
* fix missing '#' format support

Suggested and prototyped by Stefan Krah.

Fixes gh-114563, gh-91060

Co-authored-by: Stefan Krah <skrah@bytereef.org>
2024-02-12 13:17:51 +02:00
Nikita Sobolev 54bde5dcc3
gh-87804: Fix error handling and style in `_pystatvfs_fromstructstatfs` (#115236) 2024-02-12 10:27:12 +03:00
Serhiy Storchaka b104360788
gh-49766: Make date-datetime comparison more symmetric and flexible (GH-114760)
Now the special comparison methods like `__eq__` and `__lt__` return
NotImplemented if one of comparands is date and other is datetime
instead of ignoring the time part and the time zone or forcefully
return "not equal" or raise TypeError.

It makes comparison of date and datetime subclasses more symmetric
and allows to change the default behavior by overriding
the special comparison methods in subclasses.

It is now the same as if date and datetime was independent classes.
2024-02-11 13:06:43 +02:00
Soumendra Ganguly bf75f1b147
gh-85984: Add _POSIX_VDISABLE from unistd.h to termios module. (#114985)
Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2024-02-11 10:29:44 +00:00
Nikita Sobolev 3a5b38e3b4
gh-114670: Fix `_testbuffer` module initialization (#114672) 2024-02-11 00:48:28 +03:00
Mike Zimin 9d1a353230
gh-114894: add array.array.clear() method (#114919)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: AN Long <aisk@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-02-10 07:59:46 -08:00
Ronald Oussoren 6e222a55b1
GH-87804: Fix counter overflow in statvfs on macOS (#99570)
On macOS the statvfs interface returns block counts as
32-bit integers, and that results in bad reporting for
larger disks.

Therefore reimplement statvfs in terms of statfs, which
does use 64-bit integers for block counts.

Tested using a sparse filesystem image of 100TB.
2024-02-10 11:16:45 +01:00
Sam Gross a3af3cb4f4
gh-110481: Implement inter-thread queue for biased reference counting (#114824)
Biased reference counting maintains two refcount fields in each object:
`ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two
fields. In some cases, when refcounting operations are split across threads,
the ob_ref_shared field can be negative (although the total refcount must be
at least zero). In this case, the thread that decremented the refcount
requests that the owning thread give up ownership and merge the refcount
fields.
2024-02-09 17:08:32 -05:00
Serhiy Storchaka 846fd721d5
gh-115059: Flush the underlying write buffer in io.BufferedRandom.read1() (GH-115163) 2024-02-09 12:36:12 +02:00
Artem Chernyshev 9e90313320
gh-115136: Fix possible NULL deref in getpath_joinpath() (GH-115137)
Signed-off-by: Artem Chernyshev <artem.chernyshev@red-soft.ru>
2024-02-08 08:40:38 +00:00
Alex Gaynor 38b970dfcc
When the Py_CompileStringExFlags fuzzer encounters a SystemError, abort (#115147)
This allows us to catch bugs beyond memory corruption and assertions.
2024-02-07 17:21:33 -05:00
Sam Gross ef3ceab09d
gh-112066: Use `PyDict_SetDefaultRef` in place of `PyDict_SetDefault`. (#112211)
This changes a number of internal usages of `PyDict_SetDefault` to use `PyDict_SetDefaultRef`.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-02-07 13:43:18 -05:00
Mark Shannon 8a3c499ffe
GH-108362: Revert "GH-108362: Incremental GC implementation (GH-108038)" (#115132)
Revert "GH-108362: Incremental GC implementation (GH-108038)"

This reverts commit 36518e69d7.
2024-02-07 12:38:34 +00:00
Dino Viehland 92abb01240
gh-112075: Add critical sections for most dict APIs (#114508)
Starts adding thread safety to dict objects.


Use @critical_section for APIs which are exposed via argument clinic and don't directly correlate with a public C API which needs to acquire the lock
Use a _lock_held suffix for keeping changes to complicated functions simple and just wrapping them with a critical section
Acquire and release the lock in an existing function where it won't be overly disruptive to the existing logic
2024-02-06 14:03:43 -08:00
Sam Gross b6228b521b
gh-115035: Mark ThreadHandles as non-joinable earlier after forking (#115042)
This marks dead ThreadHandles as non-joinable earlier in
`PyOS_AfterFork_Child()` before we execute any Python code. The handles
are stored in a global linked list in `_PyRuntimeState` because `fork()`
affects the entire process.
2024-02-06 14:45:04 -05:00
Sam Gross 7fdd4235d7
gh-112529: Stop the world around gc.get_referents (#114823)
We do not want to add locking in `tp_traverse` slot implementations.
Instead, stop the world when calling `gc.get_referents`. Note that the the
stop the world call is a no-op in the default build.

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2024-02-06 11:45:42 -05:00
Sam Gross de61d4bd4d
gh-112066: Add `PyDict_SetDefaultRef` function. (#112123)
The `PyDict_SetDefaultRef` function is similar to `PyDict_SetDefault`,
but returns a strong reference through the optional `**result` pointer
instead of a borrowed reference.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-02-06 11:36:23 -05:00
Mariusz Felisiak 1a10437a14
gh-91602: Add iterdump() support for filtering database objects (#114501)
Add optional 'filter' parameter to iterdump() that allows a "LIKE"
pattern for filtering database objects to dump.

Co-authored-by: Erlend E. Aasland <erlend@python.org>
2024-02-06 12:34:56 +01:00
Serhiy Storchaka 652fbf88c4
gh-82626: Emit a warning when bool is used as a file descriptor (GH-111275) 2024-02-05 22:51:11 +02:00
Erlend E. Aasland 09096a1647
gh-115015: Argument Clinic: fix generated code for METH_METHOD methods without params (#115016) 2024-02-05 21:49:17 +01:00
Mark Shannon 36518e69d7
GH-108362: Incremental GC implementation (GH-108038) 2024-02-05 18:28:51 +00:00
Nikita Sobolev 87cd20a567
gh-115026: Argument Clinic: handle PyBuffer_FillInfo errors in generated code (#115027) 2024-02-05 11:45:09 +01:00
Nikita Sobolev 929d44e15a
gh-114685: PyBuffer_FillInfo() now raises on PyBUF_{READ,WRITE} (GH-114802) 2024-02-04 19:16:43 +00:00
Jason Zhang efc489021c
gh-111417: Remove unused code block in math.trunc() and round() (GH-111454)
_PyObject_LookupSpecial() now ensures that the type is ready.
2024-02-03 17:11:10 +02:00
Sam Gross d0f1307580
gh-114329: Add `PyList_GetItemRef` function (GH-114504)
The new `PyList_GetItemRef` is similar to `PyList_GetItem`, but returns
a strong reference instead of a borrowed reference. Additionally, if the
passed "list" object is not a list, the function sets a `TypeError`
instead of calling `PyErr_BadInternalCall()`.
2024-02-02 14:03:15 +01:00
Mark Shannon 0e71a295e9
GH-113710: Add a "globals to constants" pass (GH-114592)
Converts specializations of `LOAD_GLOBAL` into constants during tier 2 optimization.
2024-02-02 12:14:34 +00:00
Sam Gross 587d480203
gh-112529: Remove PyGC_Head from object pre-header in free-threaded build (#114564)
* gh-112529: Remove PyGC_Head from object pre-header in free-threaded build

This avoids allocating space for PyGC_Head in the free-threaded build.
The GC implementation for free-threaded CPython does not use the
PyGC_Head structure.

 * The trashcan mechanism uses the `ob_tid` field instead of `_gc_prev`
   in the free-threaded build.
 * The GDB libpython.py file now determines the offset of the managed
   dict field based on whether the running process is a free-threaded
   build. Those are identified by the `ob_ref_local` field in PyObject.
 * Fixes `_PySys_GetSizeOf()` which incorrectly incorrectly included the
   size of `PyGC_Head` in the size of static `PyTypeObject`.
2024-02-01 12:29:19 -08:00
Serhiy Storchaka b7688ef71e
gh-114685: Check flags in PyObject_GetBuffer() (GH-114707)
PyObject_GetBuffer() now raises a SystemError if called with
PyBUF_READ or PyBUF_WRITE as flags. These flags should
only be used with the PyMemoryView_* C API.
2024-01-31 13:11:35 +02:00
Diego Russo a06b606462
gh-110190: Fix ctypes structs with array on Windows ARM64 (GH-114753) 2024-01-30 23:53:04 +00:00
Eugene Toder 1f515e8a10
gh-112919: Speed-up datetime, date and time.replace() (GH-112921)
Use argument clinic and call new_* functions directly. This speeds up
these functions 6x to 7.5x when calling with keyword arguments.
2024-01-30 15:19:46 +00:00
Serhiy Storchaka ea30a28c3e
gh-113732: Fix support of QUOTE_NOTNULL and QUOTE_STRINGS in csv.reader (GH-113738) 2024-01-30 14:21:12 +02:00
Serhiy Storchaka aa3402ad45
gh-114678: Fix incorrect deprecation warning for 'N' specifier in Decimal format (GH-114683)
Co-authored-by: Stefan Krah <skrah@bytereef.org>
2024-01-29 19:58:31 +02:00
Soumendra Ganguly e351ca3c20
gh-85984: Add POSIX pseudo-terminal functions. (GH-102413)
Signed-off-by: Soumendra Ganguly <soumendraganguly@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-01-29 16:10:28 +00:00
Petr Viktorin 15fe8cea17
gh-91325: Skip Stable ABI checks with Py_TRACE_REFS special build (GH-92046)
Skip Stable ABI checks with Py_TRACE_REFS special build

This build is not compatible with Py_LIMITED_API nor with
the stable ABI.
2024-01-29 16:45:31 +01:00
mpage c87233fd3f
gh-112050: Adapt collections.deque to Argument Clinic (#113963) 2024-01-29 15:08:23 +00:00
Steve Dower 102569d150
Use Unicode unconditionally for _winapi.CreateFile (GH-114611)
Currently it switches based on build settings, but argument clinic does not handle it correctly.
2024-01-26 17:27:29 +00:00
Erlend E. Aasland dcd28b5c35
gh-114569: Use PyMem_* APIs for most non-PyObject uses (#114574)
Fix usage in Modules, Objects, and Parser subdirectories.
2024-01-26 10:11:35 +00:00
Michael Droettboom d0f7f5c41d
gh-114312: Fix rare event counter tests on aarch64 (GH-114554) 2024-01-26 10:10:03 +00:00
Nikita Sobolev d96358ff9d
gh-114315: Make `threading.Lock` a real class, not a factory function (#114479)
`threading.Lock` is now the underlying class and is constructable rather than the old
factory function. This allows for type annotations to refer to it which had no non-ugly
way to be expressed prior to this.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2024-01-25 19:46:32 +00:00
Michael Droettboom ea3cd0498c
gh-114312: Collect stats for unlikely events (GH-114493) 2024-01-25 11:10:51 +00:00
Serhiy Storchaka d22c066b80
gh-114492: Initialize struct termios before calling tcgetattr() (GH-114495)
On Alpine Linux it could leave some field non-initialized.
2024-01-23 23:27:04 +02:00
mpage 925907ea36
gh-113884: Make queue.SimpleQueue thread-safe when the GIL is disabled (#114161)
* use the ParkingLot API to manage waiting threads
* use Argument Clinic's critical section directive to protect queue methods
* remove unnecessary overflow check

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-01-23 20:25:41 +01:00
Serhiy Storchaka 1b719b39b9
gh-114321: Expose more constants in the fcntl module (GH-114322) 2024-01-22 18:09:22 +02:00