Commit Graph

14519 Commits

Author SHA1 Message Date
Bénédikt Tran 403f3ddedc
gh-123961: Remove global variable `ModDict` in `_cursesmodule.c` (#123962) 2024-09-13 10:33:13 +00:00
Sam Gross b2afe2aae4
gh-123923: Defer refcounting for `f_executable` in `_PyInterpreterFrame` (#123924)
Use a `_PyStackRef` and defer the reference to `f_executable` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
2024-09-12 12:37:06 -04:00
Bénédikt Tran 2938c3dec9
gh-123290: fix reference leaks in the error-branches of `_cursesmodule.c` (#123953) 2024-09-11 15:49:33 +00:00
Serhiy Storchaka c0c2aa7644
gh-122213: Add notes for pickle serialization errors (GH-122214)
This allows to identify the source of the error.
2024-09-09 21:28:55 +03:00
Jérôme Duval 65fcaa38ad
gh-84808: socket.connect_ex: Handle negative errno (GH-122304)
POSIX allows errno to be negative.
Even though all currently supported platforms have non-negative errno,
relying on a quirk like that would make Python less portable.
2024-09-09 14:59:13 +00:00
algonell 9017b95ff2
Fix typos (#123775) 2024-09-09 14:58:26 +02:00
Serhiy Storchaka b2a8c38bb2
gh-122311: Improve and unify pickle errors (GH-122771)
* Raise PicklingError instead of UnicodeEncodeError, ValueError
  and AttributeError in both implementations.
* Chain the original exception to the pickle-specific one as __context__.
* Include the error message of ImportError and some AttributeError in
  the PicklingError error message.
* Unify error messages between Python and C implementations.
* Refer to documented __reduce__ and __newobj__ callables instead of
  internal methods (e.g. save_reduce()) or pickle opcodes (e.g. NEWOBJ).
* Include more details in error messages (what expected, what got).
* Avoid including a potentially long repr of an arbitrary object in
  error messages.
2024-09-09 15:04:51 +03:00
Bénédikt Tran 05a401a5c3
chore: decimal module macro cleanup (#123791)
* protect macros expansion via `do { ... } while (0)` constructions in `_decimal.c`

* Use public macro `Py_UNUSED`

This replaces the usages of the `UNUSED` macro which
was not consistent with the `Py_UNUSED` macro itself.

In addition, this amends the parameter names so that
they match their semantic meanings.

* Remove redundant `PyCFunction` casts
2024-09-09 11:24:24 +02:00
Adam Turner 93b61bc124
gh-123843: Remove broken links to the Zope DateTimeWiki (#123846)
Co-authored-by: Conrad Bhuiyan-Volkoff <hi@cbv.im>
2024-09-08 22:39:23 -04:00
neonene 853588e24c
gh-123657: Fix crash and refleak in `decimal.getcontext()` (GH-123703) 2024-09-06 13:15:23 +02:00
Furkan Onder f8f7500168
gh-123718: Fix implicit declaration of 'explicit_memset' for NetBSD 10.0 (#123719)
Fix implicit declaration of 'explicit_memset' for NetBSD 10.0 in Lib_Memzero0.c.
2024-09-06 00:09:04 +00:00
nkinnan b5aa271f86
gh-123476: Add support for TCP_QUICKACK socket setting to Windows (#123478)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
2024-09-05 22:59:48 +02:00
Petr Viktorin 16be8db6be
gh-123465: Allow Py_RELATIVE_OFFSET for __*offset__ members (GH-123474) 2024-09-05 14:14:05 +02:00
Petr Viktorin ce9f84a47b
gh-97588: Move ctypes struct/union layout logic to Python (GH-123352)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-09-05 11:20:07 +02:00
Seth Michael Larson 40bdb0deee
gh-123678: Upgrade libexpat 2.6.3 (#123689)
Upgrade libexpat 2.6.3
2024-09-04 12:57:16 -07:00
Peter Bierma 2daed5f7a7
gh-123504: Fix regression in `_tkinter` initializer (#123662)
* Add module traverse function to _tkinter.
* Set m_size to -1 (instead of 0).
2024-09-04 13:43:50 +02:00
Peter Bierma a8bc03696c
gh-123504: Fix reference leak in finalization of `_tkinter` (#123505) 2024-09-03 20:35:57 +00:00
Victor Stinner 33b790978d
gh-107954, PEP 741: Add PyConfig_Get()/Set() functions (#123472)
Add PyConfig_Get(), PyConfig_GetInt(), PyConfig_Set() and
PyConfig_Names() functions to get and set the current runtime Python
configuration.

Add visibility and "sys spec" to config and preconfig specifications.

_PyConfig_AsDict() now converts PyConfig.xoptions as a dictionary.

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-09-02 23:25:08 +02:00
Petr Viktorin 57c471a688
gh-123091: Use more _Py_IsImmortalLoose() (GH-123602)
Switch more _Py_IsImmortal(...) assertions to _Py_IsImmortalLoose(...)

The remaining calls to _Py_IsImmortal are in free-threaded-only code,
initialization of core objects, tests, and guards that fall back to
code that works with mortal objects.
2024-09-02 18:17:48 +02:00
Victor Stinner d8e69b2c1b
gh-122854: Add Py_HashBuffer() function (#122855) 2024-08-30 15:42:27 +00:00
Victor Stinner 3d60dfbe17
gh-121645: Add PyBytes_Join() function (#121646)
* Replace _PyBytes_Join() with PyBytes_Join().
* Keep _PyBytes_Join() as an alias to PyBytes_Join().
2024-08-30 12:57:33 +00:00
Kirill Podoprigora 8aaf7525ab
gh-123189: [`Modules/blake2module.c`]: Move function definitions and their usage under the macros (#123190) 2024-08-30 13:28:32 +03:00
Serhiy Storchaka 32c7dbb2bc
gh-121485: Always use 64-bit integers for integers bits count (GH-121486)
Use 64-bit integers instead of platform specific size_t or Py_ssize_t
to represent the number of bits in Python integer.
2024-08-30 08:13:24 +03:00
sobolevn 58ce131037
gh-123446: Fix empty function names in `TypeError`s in `_csv` module (#123462) 2024-08-29 11:12:37 +03:00
sobolevn 303f92a9ce
gh-123446: Fix empty function names in `TypeError`s in `_csv` module (#123461) 2024-08-29 06:53:57 +00:00
Serhiy Storchaka 0c3ea30238
gh-123431: Harmonize extension code checks in pickle (GH-123434)
This checks are redundant in normal circumstances and can only work if
the extension registry was intentionally broken.

* The Python implementation now raises exception for the extension code
  with false boolean value.
* Simplify the C code. RuntimeError is now raised in explicit checks.
* Add many tests.
2024-08-29 08:26:16 +03:00
Peter Bierma c9930f5022
gh-123448: Move `_PyNoDefault_Type` to the static types array (#123449) 2024-08-28 18:27:40 -07:00
Wulian 9e108b8719
Fix typos in docs, error messages and comments (#123336)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-08-28 14:41:04 +03:00
Victor Stinner 4c6dca8292
gh-120389: Add PyLong_FromInt64() and PyLong_AsInt64() (#120390)
Add new functions to convert C <stdint.h> numbers from/to Python int:

* PyLong_FromInt32()
* PyLong_FromUInt32()
* PyLong_FromInt64()
* PyLong_FromUInt64()
* PyLong_AsInt32()
* PyLong_AsUInt32()
* PyLong_AsInt64()
* PyLong_AsUInt64()
2024-08-28 10:16:13 +00:00
Sergey B Kirpichev 2f20f5a9bc
gh-111495: Add tests for PyNumber C API (#111996) 2024-08-26 15:59:22 +02:00
Sergey B Kirpichev dbc1752d41
gh-111495: Add tests for PyTuple C API (#118757)
Co-authored-by: kalyanr <kalyan.ben10@live.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-08-26 11:57:52 +02:00
neonene 5ff638f1b5
gh-123243: Fix reference leak in `_decimal` (#123244) 2024-08-24 09:37:01 +05:30
blhsing 126910edba
gh-122272: Guarantee specifiers %F and %C for datetime.strftime to be 0-padded (GH-122436) 2024-08-23 18:45:03 +03:00
Bar Harel 90b6d0e0f8
gh-123213: Fixed xml.etree.ElementTree.Element.extend and assignment to no longer hide exceptions (GH-123214) 2024-08-23 12:12:58 +03:00
Jonathan Protzenko a64aa47302
GH-99108: Amend Modules/Setup (#123146) 2024-08-23 08:40:31 +02:00
Wulian 94036e43a8
Fix typos in comments (#123201) 2024-08-21 12:49:23 +00:00
Sergey B Kirpichev b9e10d1a0f
gh-122081: fixed crash in decimal.IEEEContext() (#122082)
* gh-122081: fixed crash in decimal.IEEEContext()

Now

$ ./configure CFLAGS=-DEXTRA_FUNCTIONALITY -q && make -s && \
     ./python -m test test_decimal

- PASS

* Apply suggestions from code review

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Update Misc/NEWS.d/next/Library/2024-07-21-10-45-24.gh-issue-122081.dNrYMq.rst

* Apply suggestions from code review

---------

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2024-08-19 07:51:38 +00:00
Sergey B Kirpichev 19be0ee931
gh-122681: remove m_atan2()/c_atan2() helpers (#122715) 2024-08-17 13:48:16 +05:30
Jonathan Protzenko 325e9b8ef4
gh-99108: Add HACL* Blake2 implementation to hashlib (GH-119316)
This replaces the existing hashlib Blake2 module with a single implementation that uses HACL\*'s Blake2b/Blake2s implementations. We added support for all the modes exposed by the Python API, including tree hashing, leaf nodes, and so on. We ported and merged all of these changes upstream in HACL\*, added test vectors based on Python's existing implementation, and exposed everything needed for hashlib.

This was joint work done with @R1kM.

See the PR for much discussion and benchmarking details.   TL;DR: On many systems, 8-50% faster (!) than `libb2`, on some systems it appeared 10-20% slower than `libb2`.
2024-08-13 21:42:19 +00:00
Eric Snow 503af8fe9a
gh-117482: Make the Slot Wrapper Inheritance Tests Much More Thorough (gh-122867)
There were a still a number of gaps in the tests, including not looking
at all the builtin types and not checking wrappers in subinterpreters
that weren't in the main interpreter. This fixes all that.

I considered incorporating the names of the PyTypeObject fields
(a la gh-122866), but figured doing so doesn't add much value.
2024-08-12 19:19:33 +00:00
Serhiy Storchaka f9ba1f35c7
gh-122688: Add more tests for var-positional parameters in Argument Clinic (GH-122900) 2024-08-12 16:52:48 +03:00
Kumar Aditya 3aaed083a3
GH-120974: make `_asyncio.all_tasks` thread safe (#122801)
Make `_asyncio.all_tasks` thread safe, also changes state lock to use critical section.
2024-08-11 20:49:31 +05:30
Nico Mexis 5580f31c56
gh-115808: Add ``is_none`` and ``is_not_none`` to ``operator`` (#115814)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-08-10 20:16:34 +01:00
Victor Stinner b4a316087c
gh-111495: Fix refleaks in test_capi.test_eval tests (#122851) 2024-08-09 09:47:00 +00:00
Serhiy Storchaka 8393608dd9
gh-122688: Fix support of var-positional parameter in Argument Clinic (GH-122689)
* Parameters after the var-positional parameter are now keyword-only
  instead of positional-or-keyword.
* Correctly calculate min_kw_only.
* Raise errors for invalid combinations of the var-positional parameter
  with "*", "/" and deprecation markers.
2024-08-09 09:22:41 +00:00
Sergey B Kirpichev b6e745a27e
gh-121268: Remove workarounds for non-IEEE 754 systems in cmath (#122716)
As now building Python now requires support of IEEE 754 floating point
numbers.
2024-08-09 11:17:40 +02:00
Malcolm Smith 2f5c3b09e4
gh-116622: Rename build variable MODULE_LDFLAGS back to LIBPYTHON (#122764)
(LIBPYTHON was renamed MODULE_LDFLAGS in commit 7f5e3f04f.)
2024-08-09 00:00:55 +02:00
Victor Stinner bf8b374639
gh-111495: Add more tests on PyEval C APIs (#122789)
* Add Lib/test/test_capi/test_eval.py
* Add Modules/_testlimitedcapi/eval.c
2024-08-08 14:16:20 +02:00
Erlend E. Aasland e006c7371d
gh-105201: Add PyIter_NextItem() (#122331)
Return -1 and set an exception on error; return 0 if the iterator is
exhausted, and return 1 if the next item was fetched successfully.

Prefer this API to PyIter_Next(), which requires the caller to use
PyErr_Occurred() to differentiate between iterator exhaustion and errors.

Co-authered-by: Irit Katriel <iritkatriel@yahoo.com>
2024-08-08 00:47:15 +02:00
Serhiy Storchaka 540fcc62f5
gh-118814: Fix the TypeVar constructor when name is passed by keyword (GH-122664)
Fix _PyArg_UnpackKeywordsWithVararg for the case when argument for
positional-or-keyword parameter is passed by keyword.
There was only one such case in the stdlib -- the TypeVar constructor.
2024-08-07 23:30:10 +03:00
Lucas Esposito 0bd93755f3
gh-100256: Skip inaccessible registry keys in the WinAPI mimetype implementation (GH-122047) 2024-08-07 16:07:26 +01:00
Victor Stinner 4767a6e31c
gh-122728: Fix SystemError in PyEval_GetLocals() (#122735)
Fix PyEval_GetLocals() to avoid SystemError ("bad argument to
internal function"). Don't redefine the 'ret' variable in the if
block.

Add an unit test on PyEval_GetLocals().
2024-08-06 23:01:44 +02:00
Sergey B Kirpichev 0b433aa9df
gh-122681: merge m_atan2() and c_atan2() helper functions (#122682) 2024-08-06 14:43:13 +02:00
Kirill Podoprigora 94a4bd79a7
gh-122704: Fix reference leak in Modules/_pickle.c (GH-122705) 2024-08-06 08:57:36 +03:00
Serhiy Storchaka 1bb955a2fe
gh-122459: Optimize pickling by name objects without __module__ (GH-122460) 2024-08-05 16:21:32 +03:00
Sergey B Kirpichev e6fad7a0e3
gh-122637: fix tanh(±0+infj) and tanh(±0+nanj) to return ±0+nanj (#122638)
As per C11 DR#471, ctanh (0 + i NaN) and ctanh (0 + i Inf) should return
0 + i NaN (with "invalid" exception in the second case).  This has
corresponding implications for ctan(z), as its errors and special cases
are handled as if the operation is implemented by -i*ctanh(i*z).
This patch fixes cmath's code to do same.

Glibs patch: https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=d15e83c5f5231d971472b5ffc9219d54056ca0f1
2024-08-04 10:05:30 +01:00
Sergey B Kirpichev 3462a80d2c
gh-121889: cmath.acosh(0+nanj) returns nan+pi/2j (#121892)
As per C11 DR#471 (adjusted resolution accepted for C17), cacosh (0 +
iNaN) should return NaN ± i pi/2, not NaN + iNaN.  This patch
fixes cmath's code to do same.
2024-08-04 09:53:17 +01:00
Sam Gross b5e6fb39a2
gh-120974: Make asyncio `swap_current_task` safe in free-threaded build (#122317)
* gh-120974: Make asyncio `swap_current_task` safe in free-threaded build
2024-08-02 19:02:08 +05:30
Victor Stinner addbb73927
Update PyObject_Del() documentation (#122597)
Replace PyMem_Del() with PyMem_Free().
2024-08-02 12:13:33 +02:00
Victor Stinner fda6bd842a
Replace PyObject_Del with PyObject_Free (#122453)
PyObject_Del() is just a alias to PyObject_Free() kept for backward
compatibility. Use directly PyObject_Free() instead.
2024-08-01 14:12:33 +02:00
Malcolm Smith 06656e259b
gh-116622: Don't expose `FICLONE` ioctl on Android (#122522)
Don't expose `FICLONE` ioctl on Android

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
2024-08-01 00:23:10 +00:00
Cody Maloney a9344cdffa
gh-121381 Remove subprocess._USE_VFORK escape hatch (#121383)
This flag was added as an escape hatch in gh-91401 and backported to
Python 3.10. The flag broke at some point between its addition and now.
As there is currently no publicly known environments that require this,
remove it rather than work on fixing it.

This leaves the flag in the subprocess module to not break code which
may have used / checked the flag itself.

discussion: https://discuss.python.org/t/subprocess-use-vfork-escape-hatch-broken-fix-or-remove/56915/2
2024-07-30 18:39:54 -07:00
Serhiy Storchaka 68840e91ac
gh-122311: Fix a refleak in pickle (GH-122411) 2024-07-29 21:52:48 +03:00
Serhiy Storchaka 3b034d26eb
gh-122311: Fix some error messages in pickle (GH-122386) 2024-07-29 11:49:13 +03:00
Peter Bierma c08696286f
gh-122332: Fix missing `NULL` check in `asyncio.Task.get_coro` (#122338) 2024-07-27 11:57:48 +05:30
Marc Mueller aef95eb107
gh-121489: Export private _PyBytes_Join() again (#122267) 2024-07-25 17:31:30 +02:00
Serhiy Storchaka dc07f65a53
gh-82951: Fix serializing by name in pickle protocols < 4 (GH-122149)
Serializing objects with complex __qualname__ (such as unbound methods and
nested classes) by name no longer involves serializing parent objects by value
in pickle protocols < 4.
2024-07-25 08:45:19 +00:00
Mikołaj Kuranowski a3327dbfd4
gh-113785: csv: fields starting with escapechar are not quoted (GH-122110) 2024-07-25 10:04:47 +03:00
Mark Shannon e55b05f29e
GH-121832: Assert that the version number of static builtin types is not changed by PyType_Modified. (GH-122182)
Update datetime module and test_type_cache.py to not call PyType_Modified.
2024-07-24 10:22:51 +01:00
Sam Gross a15feded71
gh-120974: Make _asyncio._leave_task atomic in the free-threaded build (#122139)
* gh-120974: Make _asyncio._leave_task atomic in the free-threaded build

Update `_PyDict_DelItemIf` to allow for an argument to be passed to the
predicate.
2024-07-23 17:06:03 +00:00
Serhiy Storchaka e6b25e9a09
gh-122163: Add notes for JSON serialization errors (GH-122165)
This allows to identify the source of the error.
2024-07-23 20:02:54 +03:00
Sam Gross c908d1f87d
gh-120974: Use common freelist code in asyncio (#122132)
This refactors asyncio to use the common freelist helper functions and
macros. As a side effect, the freelist for _asyncio.Future is now
re-enabled in the free-threaded build.
2024-07-23 21:54:24 +05:30
Sam Gross 47847aa8ef
gh-120974: Make _asyncio._enter_task atomic in the free-threaded build (#122138)
Use `PyDict_SetDefaultRef` to set the current task in a single operation
under the dictionary's lock.
2024-07-23 14:17:14 +05:30
Łukasz Langa dc93d1125f
gh-121957: Emit audit events for `python -i` and `python -m asyncio` (GH-121958)
Relatedly, emit the `cpython.run_startup` event from the Python version of
`PYTHONSTARTUP` handling.
2024-07-22 13:04:08 +02:00
Sam Gross 97248204a1
gh-121621: Disable asyncio freelist in free-threaded build (#122046)
The futureobj freelist isn't thread-safe. We intend to re-enable the
freelist in a thread-safe way for 3.14 (but not 3.13).
2024-07-19 15:26:29 -04:00
mpage e059aa6b01
gh-120973: Fix thread-safety issues with `threading.local` (#121655)
This is a small refactoring to the current design that allows us to
avoid manually iterating over threads.

This should also fix gh-118490.
2024-07-19 13:22:02 -04:00
Rodrigo Oliveira d66b06107b
gh-118830: Bump pickle.DEFAULT_PROTOCOL to 5 (GH-119340) 2024-07-19 16:47:10 +02:00
Serhiy Storchaka 1a0c7b9ba4
gh-121905: Consistently use "floating-point" instead of "floating point" (GH-121907) 2024-07-19 08:06:02 +00:00
Tian Gao eaf094c09b
gh-120289: Add external timer in traverse of _lsprof.Profiler (#121998) 2024-07-18 20:46:24 -07:00
Tian Gao 1ab1778283
gh-120289: Disallow disable() and clear() in external timer to prevent use-after-free (#120297) 2024-07-18 12:47:22 -07:00
Sam Gross 81fd625b5c
gh-121621: Move asyncio_running_loop to private struct (#121939)
This avoids changing the ABI and keeps the field in the private struct.
2024-07-17 15:21:24 -07:00
sobolevn f4bc84d261
gh-121925: Fix uninitialized variables in `main.c` (#121926) 2024-07-17 16:13:37 +00:00
Alex Waygood ac07451116
gh-120678: pyrepl: Include globals from modules passed with `-i` (GH-120904)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2024-07-17 16:18:42 +02:00
Ken Jin 69c68de43a
gh-121621: Move asyncio running loop to thread state (GH-121695) 2024-07-17 01:09:58 +08:00
Petr Viktorin b4aedb23ae
gh-113993: Don't immortalize in PyUnicode_InternInPlace; keep immortalizing in other API (#121364)
* Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs

* Document immortality in some functions that take `const char *`

This is PyUnicode_InternFromString;
PyDict_SetItemString, PyObject_SetAttrString;
PyObject_DelAttrString; PyUnicode_InternFromString;
and the PyModule_Add convenience functions.

Always point out a non-immortalizing alternative.

* Don't immortalize user-provided attr names in _ctypes
2024-07-16 15:36:21 +02:00
sobolevn 8b6d475581
gh-121791: Check for `NULL` in `MethodDescriptor2_new` in `_testcapi` (#121792) 2024-07-16 11:29:33 +03:00
Eric Snow 8b209fd4f8
gh-76785: Expand How Interpreter Channels Handle Interpreter Finalization (gh-121805)
See 6b98b274b6 for an explanation of the problem and solution.  Here I've applied the solution to channels.
2024-07-15 19:43:59 +00:00
Eric Snow 6b98b274b6
gh-76785: Expand How Interpreter Queues Handle Interpreter Finalization (gh-116431)
Any cross-interpreter mechanism for passing objects between interpreters must be very careful to respect isolation, even when the object is effectively immutable (e.g. int, str).  Here this especially relates to when an interpreter sends one of its objects, and then is destroyed while the inter-interpreter machinery (e.g. queue) still holds a reference to the object.

When I added interpreters.Queue, I dealt with that case (using an atexit hook) by silently removing all items from the queue that were added by the finalizing interpreter.

Later, while working on concurrent.futures.InterpreterPoolExecutor (gh-116430), I noticed it was somewhat surprising when items were silently removed from the queue when the originating interpreter was destroyed.  (See my comment on that PR.) 
 It took me a little while to realize what was going on.  I expect that users, which much less context than I have, would experience the same pain.

My approach, here, to improving the situation is to give users three options:

1. return a singleton (interpreters.queues.UNBOUND) from Queue.get() in place of each removed item
2. raise an exception (interpreters.queues.ItemInterpreterDestroyed) from Queue.get() in place of each removed item
3. existing behavior: silently remove each item (i.e. Queue.get() skips each one)

The default will now be (1), but users can still explicitly opt in any of them, including to the silent removal behavior.

The behavior for each item may be set with the corresponding Queue.put() call. and a queue-wide default may be set when the queue is created.  (This is the same as I did for "synconly".)
2024-07-15 12:49:23 -06:00
Kumar Aditya 48042c52a6
fix outdated comments in asyncio (#121783) 2024-07-15 05:59:19 +00:00
Ken Jin 5d6861ad06
gh-121621: Use PyMutex for writes to asyncio state (#121622)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
2024-07-14 12:22:56 +00:00
Savannah Ostrowski 65fededf9c
Update retroactive comments from GH-117741 (segfault in `FutureIter_dealloc`) (GH-121638)
Address comments
2024-07-12 10:34:30 +02:00
Sam Gross e8c91d90ba
gh-121103: Put free-threaded libraries in `lib/python3.14t` (#121293)
On POSIX systems, excluding macOS framework installs, the lib directory
for the free-threaded build now includes a "t" suffix to avoid conflicts
with a co-located default build installation.
2024-07-11 16:21:37 -04:00
Sam Gross 44937d11a6
gh-121592: Make select.poll() and related objects thread-safe (#121594)
This makes select.poll() and kqueue() objects thread-safe in the
free-threaded build. Note that calling close() concurrently with other
functions is still not thread-safe due to races on file descriptors
(gh-121544).
2024-07-11 10:21:09 -04:00
Irit Katriel 6557af6698
gh-121554: remove unnecessary internal functions in compile.c (#121555)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2024-07-10 23:48:37 +01:00
Eric Snow 35a67e36aa
gh-121596: Fix Sharing Interpreter Channels (gh-121597)
This fixes a mistake in gh-113012 and adds a test that verifies the fix.
2024-07-10 21:31:09 +00:00
Sam Gross 1d3cf79a50
gh-121368: Fix seq lock memory ordering in _PyType_Lookup (#121388)
The `_PySeqLock_EndRead` function needs an acquire fence to ensure that
the load of the sequence happens after any loads within the read side
critical section. The missing fence can trigger bugs on macOS arm64.

Additionally, we need a release fence in `_PySeqLock_LockWrite` to
ensure that the sequence update is visible before any modifications to
the cache entry.
2024-07-08 14:52:07 -04:00
Max Muoto 5289550b33
gh-121374: Correct docstrings in `_interpchannels` (gh-121418) 2024-07-08 10:32:17 -06:00
Serhiy Storchaka 8ecb8962e3
gh-121288: Make error message for index() methods consistent (GH-121395)
Make error message for index() methods consistent

Remove the repr of the searched value (which can be arbitrary large)
from ValueError messages for list.index(), range.index(), deque.index(),
deque.remove() and ShareableList.index().  Make the error messages
consistent with error messages for other index() and remove()
methods.
2024-07-05 10:50:45 -07:00
Cody Maloney 06a1c3fb24
gh-120754: Update estimated_size in C truncate (#121357)
Sometimes a large file is truncated (test_largefile). While
estimated_size is used as a estimate (the read will stil get the number
of bytes in the file), that it is much larger than the actual size of
data can result in a significant over allocation and sometimes lead to
a MemoryError / running out of memory.

This brings the C implementation to match the Python _pyio
implementation.
2024-07-04 12:59:18 +00:00
Cody Maloney 2f5f19e783
gh-120754: Reduce system calls in full-file FileIO.readall() case (#120755)
This reduces the system call count of a simple program[0] that reads all
the `.rst` files in Doc by over 10% (5706 -> 4734 system calls on my
linux system, 5813 -> 4875 on my macOS)

This reduces the number of `fstat()` calls always and seek calls most
the time. Stat was always called twice, once at open (to error early on
directories), and a second time to get the size of the file to be able
to read the whole file in one read. Now the size is cached with the
first call.

The code keeps an optimization that if the user had previously read a
lot of data, the current position is subtracted from the number of bytes
to read. That is somewhat expensive so only do it on larger files,
otherwise just try and read the extra bytes and resize the PyBytes as
needeed.

I built a little test program to validate the behavior + assumptions
around relative costs and then ran it under `strace` to get a log of the
system calls. Full samples below[1].

After the changes, this is everything in one `filename.read_text()`:

```python3
openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3`
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0`
ioctl(3, TCGETS, 0x7ffdfac04b40)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343
read(3, "", 1)                          = 0
close(3)                                = 0
```

This does make some tradeoffs
1. If the file size changes between open() and readall(), this will
still get all the data but might have more read calls.
2. I experimented with avoiding the stat + cached result for small files
in general, but on my dev workstation at least that tended to reduce
performance compared to using the fstat().

[0]

```python3
from pathlib import Path

nlines = []
for filename in Path("cpython/Doc").glob("**/*.rst"):
    nlines.append(len(filename.read_text()))
```

[1]
Before small file:

```
openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0
ioctl(3, TCGETS, 0x7ffe52525930)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0
read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343
read(3, "", 1)                          = 0
close(3)                                = 0
```

After small file:

```
openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0
ioctl(3, TCGETS, 0x7ffdfac04b40)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343
read(3, "", 1)                          = 0
close(3)                                = 0
```

Before large file:

```
openat(AT_FDCWD, "cpython/Doc/c-api/typeobj.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0
ioctl(3, TCGETS, 0x7ffe52525930)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0
read(3, ".. highlight:: c\n\n.. _type-struc"..., 133105) = 133104
read(3, "", 1)                          = 0
close(3)                                = 0
```

After large file:

```
openat(AT_FDCWD, "cpython/Doc/c-api/typeobj.rst", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0
ioctl(3, TCGETS, 0x7ffdfac04b40)        = -1 ENOTTY (Inappropriate ioctl for device)
lseek(3, 0, SEEK_CUR)                   = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, ".. highlight:: c\n\n.. _type-struc"..., 133105) = 133104
read(3, "", 1)                          = 0
close(3)                                = 0
```

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-07-04 09:17:00 +02:00
Sergey B Kirpichev 51c4a324c0
gh-61103: Support float and long double complex types in ctypes module (#121248)
This amends 6988ff02a5: memory allocation for
stginfo->ffi_type_pointer.elements in PyCSimpleType_init() should be
more generic (perhaps someday fmt->pffi_type->elements will be not a
two-elements array).

It should finally resolve #61103.

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-07-03 11:08:11 +02:00
AN Long 705a123898
gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in rotatingtree.c (#121260) 2024-07-03 13:05:05 +05:30
Serhiy Storchaka ff5806c78e
gh-121027: Make the functools.partial object a method descriptor (GH-121089)
Co-authored-by: d.grigonis <dgrigonis@users.noreply.github.com>
2024-07-03 09:02:15 +03:00
AN Long 294e724964
gh-117657: Fix data races reported by TSAN in some set methods (#120914)
Refactor the fast Unicode hash check into `_PyObject_HashFast` and use relaxed
atomic loads in the free-threaded build.

After this change, the TSAN doesn't report data races for this method.
2024-07-01 15:11:39 -04:00
AN Long 8a5176772c
gh-117657: Use critical section to make _socket.socket.close thread safe (GH-120490) 2024-07-01 16:38:30 +02:00
Will Childs-Klein 56a3ce2715
gh-117784: Only reference PHA functions ifndef SSL_VERIFY_POST_HANDSHAKE (GH-117785)
With this change, builds with OpenSSL forks that don't have this functionalty
(like AWS-LC or BoringSSL) will require less patching.
2024-07-01 15:28:35 +02:00
Sergey B Kirpichev 6988ff02a5
gh-61103: Support double complex (_Complex) type in ctypes (#120894)
Example:

```pycon
>>> import ctypes
>>> ctypes.__STDC_IEC_559_COMPLEX__
1
>>> libm = ctypes.CDLL('libm.so.6')
>>> libm.clog.argtypes = [ctypes.c_double_complex]
>>> libm.clog.restype = ctypes.c_double_complex
>>> libm.clog(1+1j)
(0.34657359027997264+0.7853981633974483j)
```

Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-07-01 10:54:33 +02:00
Victor Stinner a0b8b342c5
gh-121199: Use _Py__has_attribute() in timemodule.c (#121203)
Use the _Py__has_attribute() macro in timemodule.c and
bootstrap_hash.c to fix a build error on old GCC versions (GCC 4.8.5
on s390x).
2024-07-01 08:49:33 +00:00
Erlend E. Aasland f80376b129
gh-113565: Improve and harden detection of curses dependencies (#119816)
1. Use pkg-config to check for ncursesw/panelw. If that fails, use 
   pkg-config to check for ncurses/panel.
2. Regardless of pkg-config output, search for curses/panel headers, so
   we're sure we have all defines in pyconfig.h.
3. Regardless of pkg-config output, check if libncurses or libncursesw
   contains the 'initscr' symbol; if it does _and_ pkg-config failed
   earlier, add the resulting -llib linker option to CURSES_LIBS.
   Ditto for 'update_panels' and PANEL_LIBS.
4. Wrap the rest of the checks with WITH_SAVE_ENV and make sure we're 
   using updated LIBS and CPPFLAGS for those.

Add the PY_CHECK_CURSES convenience macro.
2024-07-01 08:10:03 +00:00
blhsing 6d34938dc8
gh-120713: Normalize year with century for datetime.strftime (GH-120820) 2024-06-29 09:32:42 +03:00
Justin Applegate 92893fd8dc
gh-121137: Add missing Py_DECREF calls for ADDITEMS opcode of _pickle.c (#121136)
PyObject_GetAttr returns a new reference, but this reference is never decremented using Py_DECREF, so Py_DECREF calls to this referece are added
2024-06-28 14:43:45 -07:00
Serhiy Storchaka db96edd6d1
gh-121027: Add a future warning in functools.partial.__get__ (#121086) 2024-06-27 11:47:20 +00:00
Victor Stinner 12af8ec864
gh-121040: Use __attribute__((fallthrough)) (#121044)
Fix warnings when using -Wimplicit-fallthrough compiler flag.

Annotate explicitly "fall through" switch cases with a new
_Py_FALLTHROUGH macro which uses __attribute__((fallthrough)) if
available. Replace "fall through" comments with _Py_FALLTHROUGH.

Add _Py__has_attribute() macro. No longer define __has_attribute()
macro if it's not defined. Move also _Py__has_builtin() at the top
of pyport.h.

Co-Authored-By: Nikita Sobolev <mail@sobolevn.me>
2024-06-27 09:58:44 +00:00
Victor Stinner 9e4a81f00f
gh-120642: Move private PyCode APIs to the internal C API (#120643)
* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()
2024-06-26 13:54:03 +02:00
Kumar Aditya 82235449b8
gh-107803: fix thread safety issue in double linked list implementation (#121007) 2024-06-26 05:11:32 +00:00
Eric Snow a905721b9c
gh-120838: Add _PyThreadState_WHENCE_FINI (gh-121010)
We also add _PyThreadState_NewBound() and drop _PyThreadState_SetWhence().

This change only affects internal API.
2024-06-25 14:35:12 -06:00
Raymond Hettinger 9b32b89074
Add fast path in count_elements (gh-120983) 2024-06-25 03:10:00 -05:00
Steve Dower e731554337
Fixes loop variables to be the same types as their limit (GH-120958) 2024-06-24 17:11:47 +01:00
Victor Stinner 2e157851e3
gh-119182: Add PyUnicodeWriter_WriteUCS4() function (#120849) 2024-06-24 17:40:39 +02:00
Serhiy Storchaka c38e2f64d0
gh-119614: Fix truncation of strings with embedded null characters in Tkinter (GH-120909)
Now the null character is always represented as \xc0\x80 for
Tcl_NewStringObj().
2024-06-24 12:17:25 +03:00
Serhiy Storchaka 02df679574
Use _PyLong_IsNegative instead of _PyLong_Sign if appropriate. (GH-120493)
It is faster and more obvious.
2024-06-24 09:49:01 +03:00
Serhiy Storchaka f4ddaa3967
gh-101830: Fix Tcl_Obj to string conversion (GH-120884)
Accessing the Tkinter object's string representation no longer converts
the underlying Tcl object to a string on Windows.
2024-06-23 16:34:14 +03:00
Kumar Aditya 4717aaa1a7
GH-107803: double linked list implementation for asyncio tasks (GH-107804)
* linked list

* add tail optmiization to linked list

* wip

* wip

* wip

* more fixes

* finally it works

* add tests

* remove weakreflist

* add some comments

* reduce code duplication in _asynciomodule.c

* address some review comments

* add invariants about the state of the linked list

* add better explanation

* clinic regen

* reorder branches for better branch prediction

* Update Modules/_asynciomodule.c

* Apply suggestions from code review

Co-authored-by: Itamar Oren <itamarost@gmail.com>

* fix capturing of eager tasks

* add comment to task finalization

* fix tests and couple c implmentation to c task

improved linked-list logic and more comments

* fix test

---------

Co-authored-by: Itamar Oren <itamarost@gmail.com>
2024-06-22 10:58:35 -07:00
Sam Gross 8f17d69b7b
gh-119344: Make critical section API public (#119353)
This makes the following macros public as part of the non-limited C-API for
locking a single object or two objects at once.

* `Py_BEGIN_CRITICAL_SECTION(op)` / `Py_END_CRITICAL_SECTION()`
* `Py_BEGIN_CRITICAL_SECTION2(a, b)` / `Py_END_CRITICAL_SECTION2()`

The supporting functions and structs used by the macros are also exposed for
cases where C macros are not available.
2024-06-21 15:50:18 -04:00
Victor Stinner 913a956d85
gh-119182: Rewrite PyUnicodeWriter tests in Python (#120845) 2024-06-21 20:15:06 +02:00
Victor Stinner 4123226bbd
gh-119182: Add PyUnicodeWriter_DecodeUTF8Stateful() (#120639)
Add PyUnicodeWriter_WriteWideChar() and
PyUnicodeWriter_DecodeUTF8Stateful() functions.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-06-21 19:33:15 +02:00
neonene a81d434c06
gh-120782: Update internal type cache when reloading datetime (#120829) 2024-06-21 22:39:33 +05:30
Petr Viktorin 6f1d448bc1
gh-113993: Allow interned strings to be mortal, and fix related issues (GH-120520)
* Add an InternalDocs file describing how interning should work and how to use it.

* Add internal functions to *explicitly* request what kind of interning is done:
  - `_PyUnicode_InternMortal`
  - `_PyUnicode_InternImmortal`
  - `_PyUnicode_InternStatic`

* Switch uses of `PyUnicode_InternInPlace` to those.

* Disallow using `_Py_SetImmortal` on strings directly.
  You should use `_PyUnicode_InternImmortal` instead:
  - Strings should be interned before immortalization, otherwise you're possibly
    interning a immortalizing copy.
  - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
    `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
    backports, as they are now part of public API and version-specific ABI.

* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.

* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
  - `_Py_ID`
  - `_Py_STR` (including the empty string)
  - one-character latin-1 singletons

  Now, when you intern a singleton, that exact singleton will be interned.

* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).

* Intern `_Py_STR` singletons at startup.

* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.

* Beef up the tests. Cover internal details (marked with `@cpython_only`).

* Add lots of assertions

Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
2024-06-21 17:19:31 +02:00
Sam Gross 3af7263037
gh-117511: Make PyMutex public in the non-limited API (#117731) 2024-06-20 11:29:08 -04:00
Jelle Zijlstra e8e151d471
gh-120780: Show attribute name for LOAD_SPECIAL in dis output (#120781) 2024-06-20 07:07:24 -07:00
Mark Shannon 00257c746c
GH-119462: Enforce invariants of type versioning (GH-120731)
* Remove uses of Py_TPFLAGS_VALID_VERSION_TAG
2024-06-19 17:38:45 +01:00
Barney Gale 20d5b84f57
GH-73991: Add follow_symlinks argument to `pathlib.Path.copy()` (#120519)
Add support for not following symlinks in `pathlib.Path.copy()`.

On Windows we add the `COPY_FILE_COPY_SYMLINK` flag is following symlinks is disabled. If the source is symlink to a directory, this call will fail with `ERROR_ACCESS_DENIED`. In this case we add `COPY_FILE_DIRECTORY` to the flags and retry. This can fail on old Windowses, which we note in the docs.

No news as `copy()` was only just added.
2024-06-19 00:59:54 +00:00
Sam Gross e8752d7b80
gh-118789: Add `PyUnstable_Object_ClearWeakRefsNoCallbacks` (#118807)
This exposes `PyUnstable_Object_ClearWeakRefsNoCallbacks` as an unstable
C-API function to provide a thread-safe mechanism for clearing weakrefs
without executing callbacks.

Some C-API extensions need to clear weakrefs without calling callbacks,
such as after running finalizers like we do in subtype_dealloc.
Previously they could use `_PyWeakref_ClearRef` on each weakref, but
that's not thread-safe in the free-threaded build.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-06-18 09:57:23 -04:00
Bénédikt Tran 7c5da94b5d
gh-120674: Protect multi-line macros in _testbuffer.c and _testcapimodule.c (#120675)
Add do { ... } while (0) pattern.
2024-06-18 12:04:52 +00:00
Victor Stinner 5c4235cd8c
gh-119182: Add PyUnicodeWriter C API (#119184) 2024-06-17 17:10:52 +02:00
Bénédikt Tran 4bf17c381f
gh-119933: Improve ``SyntaxError`` message for invalid type parameters expressions (#119976)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-17 06:51:03 -07:00
Victor Stinner c2d5df5787
gh-83754: Use the Py_TYPE() macro (#120599)
Don't access directly PyObject.ob_type, but use the Py_TYPE() macro
instead.
2024-06-17 10:34:29 +02:00
Nikita Sobolev 3df2022931
gh-120586: Fix several "unused function" warnings in `posixmodule.c` (#120588) 2024-06-17 09:44:13 +03:00
Nikita Sobolev b337aefd3e
gh-120584: Fix "unused thread_critical_sections" warning in `test_critical_sections` (#120585) 2024-06-16 20:13:56 +03:00
Eric Snow b2e71ff4f8
gh-120161: Fix a Crash in the _datetime Module (gh-120182)
In gh-120009 I used an atexit hook to finalize the _datetime module's static types at interpreter shutdown.  However, atexit hooks are executed very early in finalization, which is a problem in the few cases where a subclass of one of those static types is still alive until the final GC collection.  The static builtin types don't have this probably because they are finalized toward the end, after the final GC collection.  To avoid the problem for _datetime, I have applied a similar approach here.

Also, credit goes to @mgorny and @neonene for the new tests.

FYI, I would have liked to take a slightly cleaner approach with managed static types, but wanted to get a smaller fix in first for the sake of backporting.  I'll circle back to the cleaner approach with a future change on the main branch.
2024-06-14 13:29:09 -06:00
neonene 50a389565a
gh-117398: Add datetime C-API type check test for subinterpreters (gh-119604)
Check if the DateTime C-API type matches the datetime.date type on main and shared/isolated subinterpreters.
2024-06-13 12:05:03 -06:00
neonene 127c1d2771
gh-71587: Drop local reference cache to `_strptime` module in `_datetime` (gh-120224)
The _strptime module object was cached in a static local variable (in the datetime.strptime() implementation).  That's a problem when it crosses isolation boundaries, such as reinitializing the runtme or between interpreters.  This change fixes the problem by dropping the static variable, instead always relying on the normal sys.modules cache (via PyImport_Import()).
2024-06-12 10:46:39 -06:00
Victor Stinner 42b25dd61f
gh-120155: Add assertion to sre.c match_getindex() (#120402)
Add an assertion to help static analyzers to detect that i*2 cannot
overflow.
2024-06-12 15:27:07 +02:00
Bénédikt Tran 32d3e05fe6
gh-120029: remove unused macros in ``symtable.c`` (#120222)
Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-06-12 09:23:45 -04:00
Bénédikt Tran 755dab719d
gh-120029: make `symtable.Symbol.__repr__` correctly reflect the compiler's flags, add methods (#120099)
Expose :class:`symtable.Symbol` methods :meth:`~symtable.Symbol.is_free_class`,
:meth:`~symtable.Symbol.is_comp_iter` and :meth:`~symtable.Symbol.is_comp_cell`.

---------

Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-06-12 05:14:50 -06:00
Nikita Sobolev f5a9c34f38
gh-120056: Add `IP_RECVERR`, `IP_RECVORIGDSTADDR`, `IP_RECVTTL` to `socket` module (#120058)
* gh-120056: Add `IP_RECVERR` and `IP_RECVTTL` to `socket` module

* Fix news

* Address review

* Update NEWS
2024-06-11 21:00:56 -04:00
Pablo Galindo Salgado ec3af291fe
gh-120346: Respect PYTHON_BASIC_REPL when running in interactive inspect mode (#120349) 2024-06-11 16:15:01 +00:00
Victor Stinner 7aff2de62b
gh-120057: Add os.environ.refresh() method (#120059) 2024-06-10 16:34:17 +00:00
Victor Stinner b90bd3e5bb
gh-120155: Fix Coverity issue in zoneinfo load_data() (#120232)
Declare the 'rv' varaible at the top of the load_data() function to
make sure that it's initialized before the first 'goto error' which
uses 'rv' (return rv).

Fix the Coverity issue:

Error: UNINIT (CWE-457):
Python-3.12.2/Modules/_zoneinfo.c:1233:5: skipped_decl: Jumping over declaration of ""rv"".
Python-3.12.2/Modules/_zoneinfo.c:1284:5: uninit_use: Using uninitialized value ""rv"".
  1282|       }
  1283|
  1284|->     return rv;
  1285|   }
  1286|
2024-06-10 11:54:35 +02:00
Clinton e5a7bc6f2e
gh-120296: Fix format string of fcntl.ioctl() audit (#120301) 2024-06-10 08:17:50 +00:00
neonene 38a25e9560
gh-120244: Fix re.sub() reference leak (GH-120245) 2024-06-08 10:22:07 +00:00
Saul Shanabrook 55402d3232
gh-119258: Eliminate Type Guards in Tier 2 Optimizer with Watcher (GH-119365)
Co-authored-by: parmeggiani <parmeggiani@spaziodati.eu>
Co-authored-by: dpdani <git@danieleparmeggiani.me>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
Co-authored-by: Ken Jin <kenjin@python.org>
2024-06-08 17:41:45 +08:00
Jacob Walls 6b606522ca
gh-119577: Adjust DeprecationWarning when testing element truth values in ElementTree (GH-119762)
Adjust DeprecationWarning when testing element truth values in ElementTree, we're planning to go with the more natural True return rather than a disruptive harder to code around exception raise, and are deferring the behavior change for a few more releases.
2024-06-06 20:18:30 -07:00
Sam Gross e21057b999
gh-117657: Fix TSAN race involving import lock (#118523)
This adds a `_PyRecursiveMutex` type based on `PyMutex` and uses that
for the import lock. This fixes some data races in the free-threaded
build and generally simplifies the import lock code.
2024-06-06 13:40:58 -04:00
Victor Stinner 78634cfa3d
gh-120155: Initialize variables in _tkinter.c (#120156)
Initialize variables in _tkinter.c to make static analyzers happy.
2024-06-06 17:31:33 +02:00
benchatt 14e3c7071b
gh-115225: Raise error on unsupported ISO 8601 time strings (#119339)
Some time strings that contain fractional hours or minutes are permitted
by ISO 8601, but such strings are very unlikely to be intentional. The
current parser does not parse such strings correctly or raise an error.
This change raises a ValueError when hours or minutes contain a decimal mark.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-06-05 13:35:40 -04:00
Pablo Galindo Salgado d9095194dd
gh-119842: Honor PyOS_InputHook in the new REPL (GH-119843)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2024-06-04 20:32:43 +02:00
Christopher Chavez e079935282
gh-112672: Fix builtin Tkinter with Tcl 9.0 (GH-112681)
* Add declaration of Tcl_AppInit(), missing in Tcl 9.0.
* Use Tcl_Size instead of int where needed.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-06-04 19:47:15 +03:00
Serhiy Storchaka 7111d9605f
gh-89928: Fix integer conversion of device numbers (GH-31794)
Fix os.major(), os.minor() and os.makedev().
Support device numbers larger than 2**63-1.
Support non-existent device number (NODEV).
2024-06-04 19:36:37 +03:00
Bénédikt Tran ff1857d6ed
gh-120029: export `DEF_TYPE_PARAM` compiler flag (#120028) 2024-06-04 07:24:22 -07:00
Eric Snow 105f22ea46
gh-117398: Use Per-Interpreter State for the _datetime Static Types (gh-119929)
We make use of the same mechanism that we use for the static builtin types.  This required a few tweaks.

The relevant code could use some cleanup but I opted to avoid the significant churn in this change.  I'll tackle that separately.

This change is the final piece needed to make _datetime support multiple interpreters.  I've updated the module slot accordingly.
2024-06-03 17:09:18 -06:00
Eric Snow dba7a167db
gh-117142: Support Importing ctypes in Isolated Interpreters (gh-119991)
This makes the support official.

Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-06-03 16:42:48 -06:00
Eric Snow d82a7ba041
gh-117398: Add datetime Module State (gh-119810)
I was able to make use of the existing datetime_state struct, but there was one tricky thing I had to sort out.  We mostly aren't converting to heap types, so we can't use things like PyType_GetModuleByDef() to look up the module state.  The solution I came up with is somewhat novel, but I consider it straightforward.  Also, it shouldn't have much impact on performance.

In summary, this main changes here are:

* I've added some macros to help hide how various objects relate to module state
* as a solution to the module state lookup problem, I've stored the last loaded module on the current interpreter's internal dict (actually a weakref)
* if the static type method is used after the module has been deleted, it is reloaded
* to avoid extra work when loading the module, we directly copy the objects (new refs only) from the old module state into the new state if the old module hasn't been deleted yet
* during module init we set various objects on the static types' __dict__s; to simplify things, we only do that the first time; once those static types have a separate __dict__ per interpreter, we'll do it every time
* we now clear the module state when the module is destroyed (before, we were leaking everything in _datetime_global_state)
2024-06-03 15:56:00 -06:00
Sam Gross 47fb4327b5
gh-117657: Fix race involving immortalizing objects (#119927)
The free-threaded build currently immortalizes objects that use deferred
reference counting (see gh-117783). This typically happens once the
first non-main thread is created, but the behavior can be suppressed for
tests, in subinterpreters, or during a compile() call.

This fixes a race condition involving the tracking of whether the
behavior is suppressed.
2024-06-03 20:58:41 +00:00
Sergey B Kirpichev 61d3ab32da
gh-116560: Add PyLong_GetSign() public function (#116561)
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-06-03 14:06:31 +02:00
Radislav Chugunov 52586f930f
gh-119506: fix `_io.TextIOWrapper.write()` write during flush (#119507)
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2024-06-03 16:47:36 +09:00
Victor Stinner 64ff1e217d
gh-119770: Make termios ioctl() constants positive (#119840) 2024-05-31 17:18:40 +02:00
Sam Gross bcc1be39cb
gh-119585: Fix crash involving `PyGILState_Release()` and `PyThreadState_Clear()` (#119753)
Make sure that `gilstate_counter` is not zero in when calling
`PyThreadState_Clear()`. A destructor called from `PyThreadState_Clear()` may
call back into `PyGILState_Ensure()` and `PyGILState_Release()`. If
`gilstate_counter` is zero, it will try to create a new thread state before
the current active thread state is destroyed, leading to an assertion failure
or crash.
2024-05-31 10:50:52 -04:00
Christopher Chavez 94e9585e99
gh-103194: Fix Tkinter’s Tcl value type handling for Tcl 8.7/9.0 (GH-103846)
Some of standard Tcl types were renamed, removed, or no longer
registered in Tcl 8.7/9.0. This change fixes automatic conversion of Tcl
values to Python values to avoid returning a Tcl_Obj where the primary
Python types (int, bool, str, bytes) were returned in older Tcl.
2024-05-31 11:23:53 +03:00
Irit Katriel 13a5fdc72f
gh-119744: move a few functions from compile.c to flowgraph.c (#119745) 2024-05-30 21:55:06 +01:00
Steve Dower 78d697b7d5
gh-119690: Adds Unicode support for named pipes in _winapi (GH-119717) 2024-05-29 16:51:09 +01:00
Kirill Podoprigora 34f9b3e724
gh-119655: Fix reference leak in the ``_datetimemodule.c`` (gh-119713) 2024-05-29 09:43:03 -06:00
Matthias Görgens 18c1a8d3a8
gh-97588: Align ctypes struct layout to GCC/MSVC (GH-97702)
Structure layout, and especially bitfields, sometimes resulted in clearly
wrong behaviour like overlapping fields. This fixes

Co-authored-by: Gregory P. Smith <gps@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-05-29 12:02:53 +02:00
Victor Stinner 7ca74a760a
gh-119661: Add _Py_SINGLETON() include in Argumenet Clinic (#119712)
When the _Py_SINGLETON() is used, Argument Clinic now adds an
explicit "pycore_runtime.h" include to get the macro. Previously, the
macro may or may not be included indirectly by another include.
2024-05-29 11:37:04 +02:00
Sergey B Kirpichev cd11ff12ac
gh-119613: Use C99+ functions instead of Py_IS_NAN/INFINITY/FINITE (#119619) 2024-05-29 09:51:19 +02:00
Eric Snow 548a11d5cf
gh-117398: Convert datetime.IsoCalendarDate To A Heap Type (gh-119637)
This is the only static type in the module that we will not keep static.
2024-05-28 16:42:23 -06:00
Petr Viktorin a89fc26f4a
gh-117398: gh-119655: datetime: Init static state once & don't free it (GH-119662)
- While datetime uses global state, only initialize it once.
- While `capi` is static, don't free it (thanks @neonene in https://github.com/python/cpython/pull/119641/files#r1616710048)
2024-05-28 17:27:52 +00:00
Serhiy Storchaka b313cc68d5
gh-117557: Improve error messages when a string, bytes or bytearray of length 1 are expected (GH-117631) 2024-05-28 12:01:37 +03:00
Erlend E. Aasland 3e8b60905e
gh-117398: Add multiphase support to _datetime (gh-119373)
This is minimal support.  Subinterpreters are not supported yet.  That will be addressed in a later change.

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-05-27 16:02:46 -06:00
Eric Snow ae7b17673f
gh-119584: Fix test_import Failed Assertion (gh-119623)
The fix in gh-119561 introduced an assertion that doesn't hold true if any of the three new test extension modules are loaded more than once.  This is fine normally but breaks if the new test_check_state_first() is run more than once, which happens for refleak checking and with the regrtest --forever flag.  We fix that here by clearing each of the three modules after loading them.  We also tweak a check in _modules_by_index_check().
2024-05-27 19:35:30 +00:00
Eric Snow 0c5ebe13e9
gh-119560: Drop an Invalid Assert in PyState_FindModule() (gh-119561)
The assertion was added in gh-118532 but was based on the invalid assumption that PyState_FindModule() would only be called with an already-initialized module def.  I've added a test to make sure we don't make that assumption again.
2024-05-25 19:30:48 +00:00
Nice Zombies 96b392df30
gh-118263: Add additional arguments to path_t (Argument Clinic type) in posixmodule (GH-118355) 2024-05-24 19:04:17 +01:00
Victor Stinner 92fab3356f
gh-69214: Fix fcntl.ioctl() request type (#119498)
Use an 'unsigned long' instead of an 'unsigned int' for the request
parameter of fcntl.ioctl() to support requests larger than UINT_MAX.
2024-05-24 14:31:40 +02:00
Victor Stinner e94dbe4ed8
gh-119461: Fix ThreadedVSOCKSocketStreamTest (#119465)
Add socket.VMADDR_CID_LOCAL constant.

Fix ThreadedVSOCKSocketStreamTest: if get_cid() returns the host
address or the "any" address, use the local communication address
(loopback): VMADDR_CID_LOCAL.

On Linux 6.9, apparently, the /dev/vsock device is now available but
get_cid() returns VMADDR_CID_ANY (-1).
2024-05-23 23:26:09 +02:00
Eric Snow b30d30c747
gh-117398: Statically Allocate the Datetime C-API (GH-119472) 2024-05-23 21:15:52 +02:00
Petr Viktorin a192547dfe
gh-117142: Slightly hacky fix for memory leak of StgInfo (GH-119424)
Add a funciton that inlines PyObject_GetTypeData and skips
type-checking, so it doesn't need access to the CType_Type object.
This will break if the memory layout changes, but should
be an acceptable solution to enable ctypes in subinterpreters in
Python 3.13.

Mark _ctypes as safe for multiple interpreters

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
2024-05-23 16:01:37 +00:00
neonene e12a6780bb
gh-117142: ctypes: Clean up c-analyzer .tsv files (GH-117544)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-05-22 20:30:41 +00:00
Geoffrey Thomas ef172521a9
Remove almost all unpaired backticks in docstrings (#119231)
As reported in #117847 and #115366, an unpaired backtick in a docstring
tends to confuse e.g. Sphinx running on subclasses of standard library
objects, and the typographic style of using a backtick as an opening
quote is no longer in favor. Convert almost all uses of the form

    The variable `foo' should do xyz

to

    The variable 'foo' should do xyz

and also fix up miscellaneous other unpaired backticks (extraneous /
missing characters).

No functional change is intended here other than in human-readable
docstrings.
2024-05-22 12:35:18 -04:00
Eric Snow 81865002ae
gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across Interpreters (gh-119331)
_PyArg_Parser holds static global data generated for modules by Argument Clinic.  The _PyArg_Parser.kwtuple field is a tuple object, even though it's stored within a static global.  In some cases the tuple is statically allocated and thus it's okay that it gets shared by multiple interpreters.  However, in other cases the tuple is set lazily, allocated from the heap using the active interprepreter at the point the tuple is needed.

This is a problem once that interpreter is destroyed since _PyArg_Parser.kwtuple becomes at dangling pointer, leading to crashes.  It isn't a problem if the tuple is allocated under the main interpreter, since its lifetime is bound to the lifetime of the runtime.  The solution here is to temporarily switch to the main interpreter.  The alternative would be to always statically allocate the tuple.

This change also fixes a bug where only the most recent parser was added to the global linked list.
2024-05-22 09:57:52 -06:00
Irit Katriel 6e9863d7a3
gh-118692: Avoid creating unnecessary StopIteration instances for monitoring (#119216) 2024-05-21 20:42:51 +00:00
Nice Zombies b64182550f
gh-118507 : Refactor `nt._path_is*` to improve applicability for other cases (GH-118755) 2024-05-21 21:36:36 +01:00
Arnon Yaari 87939bd579
gh-117657: Fix itertools.count thread safety (#119268)
Fix itertools.count in free-threading mode
2024-05-21 10:16:34 -07:00
Erlend E. Aasland af359cee75
gh-118928: sqlite3: correctly bail if sequences of params are used with named placeholders (#119197) 2024-05-20 09:44:42 -04:00
Kirill Podoprigora 100c7ab00a
gh-119049: Fix incorrect display of warning which is constructed by C API (GH-119063)
The source line was not displayed if the warnings module had not yet
been imported.
2024-05-16 20:27:59 +00:00
Steve Dower 94591dca51
gh-118486: Simplify test_win32_mkdir_700 to check the exact ACL (GH-119056) 2024-05-15 11:59:41 +01:00
Erlend E. Aasland d8e0e00919
gh-118928: sqlite3: disallow sequences of params with named placeholders (#118929)
Follow-up of gh-101693. The previous DeprecationWarning is replaced with 
raising sqlite3.ProgrammingError.

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-05-14 16:10:55 +00:00
Nikita Sobolev fc75792594
gh-118998: Handle errors correctly in `tmtotuple` in `timemodule` (#118999) 2024-05-14 00:20:59 +03:00
Victor Stinner 7d7eec595a
gh-117873: Revert _posixshmem.shm_open() change (#118901) 2024-05-13 16:03:52 +02:00
Hugo van Kemenade a019347947
gh-118924: Remove `sqlite3.version` and `sqlite3.version_info` (#118925) 2024-05-10 20:42:34 +00:00
Jelle Zijlstra 13d7cf997b
gh-118895: Call PyType_Ready() on typing.NoDefault (#118897) 2024-05-10 08:42:00 -07:00