Commit Graph

13834 Commits

Author SHA1 Message Date
Serhiy Storchaka 548ce0923b
gh-110815: Improve tests for PyArg_ParseTupleAndKeywords() (GH-110817) 2023-10-13 16:05:01 +03:00
Erlend E. Aasland ec5622d197
gh-71587: Establish global state in `_datetime` (#110475)
* Use explicit initialiser for m_base
* Add module state stub; establish global state on stack
* Put conversion factors in state struct
* Move PyDateTime_TimeZone_UTC to state
* Move PyDateTime_Epoch to state struct
* Fix ref leaks in and clean up initialisation
2023-10-12 10:28:10 +02:00
Lysandros Nikolaou 17d65547df
gh-104169: Fix test_peg_generator after tokenizer refactoring (#110727)
* Fix test_peg_generator after tokenizer refactoring
* Remove references to tokenizer.c in comments etc.
2023-10-12 09:34:35 +02:00
Victor Stinner 88ecb190f3
gh-85283: _stat extension now uses the limited C API (#110711)
gh-85283: _stat extension uses the limited C API

The _stat C extension is now built with the limited C API.
2023-10-12 00:06:20 +02:00
Serhiy Storchaka eb50cd37ea
gh-110289: C API: Add PyUnicode_EqualToUTF8() and PyUnicode_EqualToUTF8AndSize() functions (GH-110297) 2023-10-11 16:41:58 +03:00
Serhiy Storchaka 5c6e85480a
gh-84489: C API: Add tests for Py_BuildValue() (GH-110596) 2023-10-11 10:08:51 +03:00
Nikita Sobolev 9cfb4e0d1e
gh-110525: Add tests for internal `set` CAPI (GH-110630) 2023-10-10 19:00:05 +03:00
Nikita Sobolev 344d3a222a
gh-110590: Fix a bug where _sre.compile would overwrite exceptions (#110591)
TypeError would be overwritten by OverflowError
if 'code' param contained non-ints.
2023-10-10 10:15:12 +00:00
Donghee Na 0362cbf908
gh-109595: Add -Xcpu_count=<n> cmdline for container users (#109667)
---------

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2023-10-10 19:00:09 +09:00
Eric Snow 757cc35b6b
gh-84570: Implement Waiting in SendChannel.send() (gh-110565)
We had been faking it (poorly).

We will add timeouts separately.
2023-10-10 09:35:14 +00:00
Donghee Na 67e8d416cc
gh-109693: Use pyatomic.h for signal module (gh-110480) 2023-10-10 08:26:29 +09:00
Jakub Kulík 3b1580af07
gh-78469: Declare missing sethostname for Solaris 10 (#109447)
Add OS version specific macro for Solaris: Py_SUNOS_VERSION.
2023-10-09 21:18:05 +00:00
Illia Volochii ea7b53ff67
gh-107652: Set up CIFuzz to run fuzz targets continuously (#107653)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-10-09 09:30:10 -06:00
Eric Snow 7bd560ce8d
gh-76785: Add SendChannel.send_buffer() (#110246)
(This is still a test module.)
2023-10-09 07:39:51 -06:00
Bo Anderson f4cb0d27cc
gh-109191: Fix build with newer editline (gh-110239) 2023-10-09 13:21:20 +00:00
Nikita Sobolev ea39c877c0
gh-110525: Cover PySet_Add corner case with frozenset objects (GH-110544) 2023-10-09 14:04:49 +03:00
Nikita Sobolev 89df5b73d0
gh-110549: Remove unused includes of <stddef.h> in _testcapi (GH-110552) 2023-10-09 10:37:57 +00:00
Nikita Sobolev c49edd7d9c
gh-110525: Add CAPI tests for set and frozenset objects (GH-110526) 2023-10-09 11:57:48 +03:00
Serhiy Storchaka d96a8cdfe2
gh-89902: Deprecate non-standard format specifier "N" for Decimal (GH-110508)
It was not documented and only supported in the C implementation.
2023-10-08 10:01:39 +03:00
Masaru Tsuchiyama de2a4036cb
gh-108277: Add os.timerfd_create() function (#108382)
Add wrapper for timerfd_create, timerfd_settime, and timerfd_gettime to os module.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2023-10-07 19:33:22 +02:00
Eric Snow 92ca90b762
gh-76785: Support Running Some Functions in Subinterpreters (gh-110251)
This specifically refers to `test.support.interpreters.Interpreter.run()`.
2023-10-06 17:52:22 -06:00
Charlie Zhao de1052245f
gh-106078: Suppress the warning caused by multi-phase initialization of `decimal` (#107524) 2023-10-07 00:15:19 +01:00
Shantanu f013b47504
gh-110489: Optimise math.ceil for known exact float (#108801)
This matches a similar optimisation done for math.floor in
https://github.com/python/cpython/pull/21072
2023-10-06 22:57:18 +00:00
박문식 d33aa18f15
gh-82367: Use `FindFirstFile` Win32 API in `ntpath.realpath()` (GH-110298)
* Use `FindFirstFile` Win32 API to fix a bug where `ntpath.realpath()`
breaks out of traversing a series of paths where a (handled)
`ERROR_ACCESS_DENIED` or `ERROR_SHARING_VIOLATION` occurs.
* Update docs to reflect that `ntpath.realpath()` eliminates MS-DOS
style names.
2023-10-05 09:49:07 -05:00
Nikita Sobolev 2bbbab212f
gh-110365: Fix error overwrite in `termios.tcsetattr` (#110366)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-10-05 11:02:52 +00:00
Sam Gross cf6f23b0e3
gh-88402: Add new sysconfig variables on Windows (GH-110049)
Co-authored-by: Filipe Laíns <filipe.lains@gmail.com>
2023-10-04 22:50:29 +00:00
Eric Snow ee317f7ddd
gh-76785: Print the Traceback from Interpreter.run() (gh-110322)
This is a temporary solution.  The full fix may involve serializing the traceback in some form.

(FYI, I merged this yesterday and the reverted it due to buildbot failures.  See gh-110248.)
2023-10-04 11:57:03 -06:00
Mark Shannon bf4bc36069
GH-109369: Merge all eval-breaker flags and monitoring version into one word. (GH-109846) 2023-10-04 16:09:48 +01:00
Nikita Sobolev 43baddc2b9
gh-110260: Check for PyList_SetItem() errors in termios module (GH-110261) 2023-10-04 15:42:17 +03:00
Eric Snow 269005e784
gh-110300: Fix Refleaks in test_interpreters and test__xxinterpchannels (gh-110318) 2023-10-04 00:36:50 +00:00
Eric Snow 4467d2c3ac
Revert "gh-76785: Print the Traceback from Interpreter.run() (gh-110248)" (gh-110314)
This reverts commit 6bc889aedc.

That commit is causing some buildbot failures.
2023-10-03 21:18:25 +00:00
Eric Snow 6bc889aedc
gh-76785: Print the Traceback from Interpreter.run() (gh-110248)
This is a temporary fix. The full fix may involve serializing the traceback in some form.
2023-10-03 14:41:02 -06:00
Victor Stinner d73501602f
gh-108867: Add PyThreadState_GetUnchecked() function (#108870)
Add PyThreadState_GetUnchecked() function: similar to
PyThreadState_Get(), but don't issue a fatal error if it is NULL. The
caller is responsible to check if the result is NULL. Previously,
this function was private and known as _PyThreadState_UncheckedGet().
2023-10-03 16:53:51 +00:00
Eric Snow f5198b09e1
gh-109860: Use a New Thread State When Switching Interpreters, When Necessary (gh-110245)
In a few places we switch to another interpreter without knowing if it has a thread state associated with the current thread.  For the main interpreter there wasn't much of a problem, but for subinterpreters we were *mostly* okay re-using the tstate created with the interpreter (located via PyInterpreterState_ThreadHead()).  There was a good chance that tstate wasn't actually in use by another thread.

However, there are no guarantees of that.  Furthermore, re-using an already used tstate is currently fragile.  To address this, now we create a new thread state in each of those places and use it.

One consequence of this change is that PyInterpreterState_ThreadHead() may not return NULL (though that won't happen for the main interpreter).
2023-10-03 09:20:48 -06:00
Nikita Sobolev 4596c76d1a
gh-110241: Add missing error check to `record_eval` in `_testinternalcapi` (#110242) 2023-10-02 21:19:32 +00:00
Eric Snow a8f5dab58d
gh-76785: Module-level Fixes for test.support.interpreters (gh-110236)
* add RecvChannel.close() and SendChannel.close()
* make RecvChannel and SendChannel shareable
* expose ChannelEmptyError and ChannelNotEmptyError
2023-10-02 20:47:41 +00:00
Eric Snow 1dd9dee45d
gh-105716: Support Background Threads in Subinterpreters Consistently (gh-109921)
The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.

We add PyInterpreterState.threads.main, with some internal C-API functions.
2023-10-02 20:12:12 +00:00
Eric Snow a040a32ea2
gh-109853: Fix sys.path[0] For Subinterpreters (gh-109994)
This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all. This PR does not address the broader concerns from gh-109853.
2023-10-02 19:59:05 +00:00
Victor Stinner fc2cb86d21
gh-107073: Make PyObject_VisitManagedDict() public (#108763)
Make PyObject_VisitManagedDict() and PyObject_ClearManagedDict()
functions public in Python 3.13 C API.

* Rename _PyObject_VisitManagedDict() to PyObject_VisitManagedDict().
* Rename _PyObject_ClearManagedDict() to PyObject_ClearManagedDict().
* Document these functions.
2023-10-02 19:24:08 +02:00
Victor Stinner a46e960768
gh-109649: Use os.process_cpu_count() (#110165)
Replace os.cpu_count() with os.process_cpu_count() in modules:

* compileall
* concurrent.futures
* multiprocessing

Replace os.cpu_count() with os.process_cpu_count() in programs:

* _decimal deccheck.py test
* freeze.py
* multissltests.py
* python -m test (regrtest)
* wasm_build.py

Other changes:

* test.pythoninfo logs os.process_cpu_count().
* regrtest gets os.process_cpu_count() / os.cpu_count() in headers.
2023-10-01 03:14:57 +02:00
Victor Stinner c81521020d
gh-109649: Add os.process_cpu_count() function (#109907)
* Refactor os_sched_getaffinity_impl(): move variable definitions to
  their first assignment.
* Fix test_posix.test_sched_getaffinity(): restore the old CPU mask
  when the test completes!
* Doc: Specify that os.cpu_count() counts *logicial* CPUs.
* Doc: Specify that os.sched_getaffinity(0) is related to the calling
  thread.
2023-10-01 00:12:51 +02:00
Victor Stinner 7513994c92
gh-110014: Include explicitly <unistd.h> header (#110155)
* Remove unused <locale.h> includes.
* Remove unused <fcntl.h> include in traceback.h.
* Remove redundant <assert.h> and <stddef.h> includes. They  are already
  included by "Python.h".
* Remove <object.h> include in faulthandler.c. Python.h already includes it.
* Add missing <stdbool.h> in pycore_pythread.h if HAVE_PTHREAD_STUBS
  is defined.
* Fix also warnings in pthread_stubs.h: don't redefine macros if they
  are already defined, like the __NEED_pthread_t macro.
2023-09-30 20:06:45 +00:00
Donghee Na 501939c9c1
gh-105323: Update readline module to detect apple editline variant (gh-108665) 2023-09-29 21:18:18 +09:00
Victor Stinner 8b626a47ba
gh-110079: Remove extern "C" { ...} in C code (#110080) 2023-09-29 10:56:49 +02:00
Jelle Zijlstra 7dc2c5093e
gh-110045: Update symtable module for PEP 695 (#110066) 2023-09-29 02:08:04 +00:00
Victor Stinner 2e37a38bcb
gh-110052: Fix faulthandler for freed tstate (#110069)
faulthandler now detected freed interp and freed tstate, and no
longer dereference them.
2023-09-29 02:04:06 +00:00
Amin Alaee 7df8b16d28
gh-109782: Ensure `os.path.isdir` has the same signature on all platforms (GH-109790) 2023-09-28 16:17:30 +01:00
Ammar Askar a829356f86
gh-109098: Fuzz re module instead of internal sre (#109911)
* gh-109098: Fuzz re module instead of internal sre
* Fix c-analyzer globals test failure
* Put globals exception in ignored.tsv
2023-09-26 15:35:49 -07:00
Antoine Pitrou 0eb98837b6
gh-109593: Fix reentrancy issue in multiprocessing resource_tracker (#109629)
---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-09-26 13:57:25 +02:00
Serhiy Storchaka 8ac2085b80
gh-109631: Allow interruption of short repeated regex matches (GH-109867)
Counting for signal checking now continues in new match from the point where
it ended in the previous match instead of starting from 0.
2023-09-26 10:56:33 +03:00
Radislav Chugunov 1b8f2366b3
gh-109795: `_thread.start_new_thread`: allocate thread bootstate using raw memory allocator (#109808) 2023-09-25 17:38:06 +02:00
Victor Stinner eeb9070437
gh-109723: Fix build of _testclinic_limited on WASM (#109842)
Make sure that the Py_BUILD_CORE macro is not defined.
2023-09-25 14:43:54 +00:00
Victor Stinner 8c521f090e
gh-104469: Convert _testcapi/vectorcall_limited.c to use AC (#109691)
Co-authored-by: nahyeon <55136494+nahyeon-an@users.noreply.github.com>
2023-09-25 13:24:19 +02:00
Serhiy Storchaka b8d1744e7b
gh-109611: Add convenient C API function _PyFile_Flush() (GH-109612) 2023-09-23 09:35:30 +03:00
Victor Stinner 09a25616a9
gh-109723: Disable Py_BUILD_CORE in _testcapi (#109727)
Make sure that the internal C API is not tested by mistake by
_testcapi.

Undefine Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE macros in
Modules/_testcapi/parts.h: move code from _testcapimodule.c.

heaptype_relative.c and vectorcall_limited.c are using the limited C
API which is incompatible with the internal C API.

Move test_long_numbits() from _testcapi to _testinternalcapi since it
uses the internal C API "pycore_long.h".

Fix Modules/_testcapi/pyatomic.c: don't include Python.h directly,
just include _testcapi/parts.h.

Ajust "make check-c-globals" for these changes.
2023-09-22 14:54:37 +00:00
Sam Gross 2aceb21ae6
gh-109693: Remove pycore_atomic_funcs.h (#109694)
_PyUnicode_FromId() now uses pyatomic.h functions instead.
2023-09-21 22:57:20 +02:00
Victor Stinner d4cea794a7
gh-109613: _pystat_fromstructstat() checks for exceptions (#109618)
Fix os.stat() and os.DirEntry.stat(): check for exceptions.
Previously, on Python built in debug mode, these functions could
trigger a fatal Python error (and abort the process) when a function
succeeded with an exception set.

_pystat_fromstructstat() now exits immediately if an exception is
raised, rather only checking for exceptions at the end. It fix
following fatal error in fill_time():

    Fatal Python error: _Py_CheckSlotResult:
    Slot * of type int succeeded with an exception set
2023-09-21 09:55:06 +02:00
James Gerity def828995a
fixes gh-109559: Update `unicodedata` for Unicode 15.1.0 (GH-109560)
---------

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2023-09-19 22:07:47 -07:00
Ronan Pigott ddf2e953c2
gh-109033: Return filename with os.utime errors (#109034)
The filename was previously intentionally omitted from exception because
"it might confuse the user". Uncaught exceptions are not generally a
replacement for user-facing error messages, so obscuring this
information only has the effect of making the programmer's life more
difficult.
2023-09-20 01:18:23 +02:00
Eric Snow fd7e08a6f3
gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data (gh-109556)
This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters.
2023-09-19 15:01:34 -06:00
Sam Gross 0c89056fe5
gh-108724: Add PyMutex and _PyParkingLot APIs (gh-109344)
PyMutex is a one byte lock with fast, inlineable lock and unlock functions for the common uncontended case.  The design is based on WebKit's WTF::Lock.

PyMutex is built using the _PyParkingLot APIs, which provides a cross-platform futex-like API (based on WebKit's WTF::ParkingLot).  This internal API will be used for building other synchronization primitives used to implement PEP 703, such as one-time initialization and events.

This also includes tests and a mini benchmark in Tools/lockbench/lockbench.py to compare with the existing PyThread_type_lock.

Uncontended acquisition + release:
* Linux (x86-64): PyMutex: 11 ns, PyThread_type_lock: 44 ns
* macOS (arm64): PyMutex: 13 ns, PyThread_type_lock: 18 ns
* Windows (x86-64): PyMutex: 13 ns, PyThread_type_lock: 38 ns

PR Overview:

The primary purpose of this PR is to implement PyMutex, but there are a number of support pieces (described below).

* PyMutex:  A 1-byte lock that doesn't require memory allocation to initialize and is generally faster than the existing PyThread_type_lock.  The API is internal only for now.
* _PyParking_Lot:  A futex-like API based on the API of the same name in WebKit.  Used to implement PyMutex.
* _PyRawMutex:  A word sized lock used to implement _PyParking_Lot.
* PyEvent:  A one time event.  This was used a bunch in the "nogil" fork and is useful for testing the PyMutex implementation, so I've included it as part of the PR.
* pycore_llist.h:  Defines common operations on doubly-linked list.  Not strictly necessary (could do the list operations manually), but they come up frequently in the "nogil" fork. ( Similar to https://man.freebsd.org/cgi/man.cgi?queue)

---------

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2023-09-19 09:54:29 -06:00
Serhiy Storchaka ed582a2ed9
gh-109469: Silence compiler warnings on string comparisons in _testcapi (GH-109533) 2023-09-19 08:12:29 +03:00
Xuehai Pan 74f315edd0
gh-102757: fix function signature mismatch for `functools.reduce` between code and documentation (#102759) 2023-09-18 10:42:58 -06:00
Victor Stinner 0bb0d88e2d
gh-109496: Detect Py_DECREF() after dealloc in debug mode (#109539)
On a Python built in debug mode, Py_DECREF() now calls
_Py_NegativeRefcount() if the object is a dangling pointer to
deallocated memory: memory filled with 0xDD "dead byte" by the debug
hook on memory allocators. The fix is to check the reference count
*before* checking for _Py_IsImmortal().

Add test_decref_freed_object() to test_capi.test_misc.
2023-09-18 14:59:09 +00:00
Serhiy Storchaka add16f1a5e
gh-108511: Add C API functions which do not silently ignore errors (GH-109025)
Add the following functions:

* PyObject_HasAttrWithError()
* PyObject_HasAttrStringWithError()
* PyMapping_HasKeyWithError()
* PyMapping_HasKeyStringWithError()
2023-09-17 14:23:31 +03:00
Nikita Sobolev 47af188593
Add missing `PyDoc_STR` calls (#109393)
In files:

* Modules/_ctypes/cfield.c
* Modules/_struct.c
* Objects/dictobject.c
* Objects/typevarobject.c
* Objects/unionobject.c
2023-09-15 15:10:48 +02:00
Serhiy Storchaka 1f885df2a5
gh-107782: Use _testcapi to test non-representable signatures (GH-109325)
Builtin functions and methods that have non-representable signatures today
will have representable signatures yesterday, and they will become unusable
for testing this feature.

So we need to add special functions and methods to the _testcapi module
that always have non-representable signatures.
2023-09-14 09:12:17 +03:00
Charlie Zhao b39f65a495
gh-108346: Fix failed benchmark in decimal (#108353)
Fix benchmark in decimal to work again after the int str conversion limits.
2023-09-12 21:17:55 -07:00
Victor Stinner 517cd82ea7
gh-108987: Fix _thread.start_new_thread() race condition (#109135)
Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.

thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.

Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().
2023-09-11 17:27:03 +02:00
Serhiy Storchaka b4131a13cb
gh-109050: Remove remaining tests for legacy Unicode C API (GH-109068) 2023-09-09 05:44:46 +00:00
Victor Stinner f63d37877a
gh-104690: thread_run() checks for tstate dangling pointer (#109056)
thread_run() of _threadmodule.c now calls
_PyThreadState_CheckConsistency() to check if tstate is a dangling
pointer when Python is built in debug mode.

Rename ceval_gil.c is_tstate_valid() to
_PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
2023-09-08 11:50:46 +02:00
Victor Stinner b0edf3b98e
GH-91079: Rename C_RECURSION_LIMIT to Py_C_RECURSION_LIMIT (#108507)
Symbols of the C API should be prefixed by "Py_" to avoid conflict
with existing names in 3rd party C extensions on "#include <Python.h>".

test.pythoninfo now logs Py_C_RECURSION_LIMIT constant and other
_testcapi and _testinternalcapi constants.
2023-09-08 09:48:28 +00:00
Anders Kaseorg f42edf1e7b
gh-109045: Remove remaining LIMITED_API_AVAILABLE checks in tests (#109046)
Commit 13a00078b8 (#108663) made all
Python builds compatible with the Limited API, and removed the
LIMITED_API_AVAILABLE flag.  However, some tests were still checking
for that flag, so they were now being incorrectly skipped.  Remove
these checks to let these tests run again.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-09-07 04:42:58 +02:00
Victor Stinner 19eddb515a
gh-107211: No longer export internal _PyLong_FromUid() (#109037)
No longer export _PyLong_FromUid() and _Py_Sigset_Converter()
internal C API function.
2023-09-07 02:09:06 +02:00
Serhiy Storchaka 6f3c138dfa
gh-108751: Add copy.replace() function (GH-108752)
It creates a modified copy of an object by calling the object's
__replace__() method.

It is a generalization of dataclasses.replace(), named tuple's _replace()
method and replace() methods in various classes, and supports all these
stdlib classes.
2023-09-06 23:55:42 +03:00
Serhiy Storchaka bf414b7fcb
C API tests: use special markers to test that output parameters were set (GH-109014) 2023-09-06 22:02:01 +03:00
Victor Stinner a0773b89df
gh-108753: Enhance pystats (#108754)
Statistics gathering is now off by default. Use the "-X pystats"
command line option or set the new PYTHONSTATS environment variable
to 1 to turn statistics gathering on at Python startup.

Statistics are no longer dumped at exit if statistics gathering was
off or statistics have been cleared.

Changes:

* Add PYTHONSTATS environment variable.
* sys._stats_dump() now returns False if statistics are not dumped
  because they are all equal to zero.
* Add PyConfig._pystats member.
* Add tests on sys functions and on setting PyConfig._pystats to 1.
* Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h
  header files.
* Rename '_py_stats' variable to '_Py_stats'.
* Exclude Include/cpython/pystats.h from the Py_LIMITED_API.
* Move pystats.h include from object.h to Python.h.
* Add _Py_StatsOn() and _Py_StatsOff() functions. Remove
  '_py_stats_struct' variable from the API: make it static in
  specialize.c.
* Document API in Include/pystats.h and Include/cpython/pystats.h.
* Complete pystats documentation in Doc/using/configure.rst.
* Don't write "all zeros" stats: if _stats_off() and _stats_clear()
  or _stats_dump() were called.
* _PyEval_Fini() now always call _Py_PrintSpecializationStats() which
  does nothing if stats are all zeros.

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2023-09-06 15:54:59 +00:00
Nikita Sobolev 65a2bce704
gh-108717: Speedup `os.DirEntry.is_junction` function (#108718) 2023-09-06 08:52:53 +00:00
Petr Viktorin 230649f538
gh-108294: Add error handling for time.sleep audit event (GH-108363)
I've also cleaned the tests up a bit to make this easier to test.
2023-09-05 10:25:08 +02:00
Victor Stinner 676593859e
gh-106320: Remove private _PyErr_WriteUnraisableMsg() (#108863)
Move the private _PyErr_WriteUnraisableMsg() functions to the
internal C API (pycore_pyerrors.h).

Move write_unraisable_exc() from _testcapi to _testinternalcapi.
2023-09-05 01:54:55 +02:00
Victor Stinner 5a79d2ae57
Revert "gh-46376: Return existing pointer when possible in ctypes (#1… (#108688)
This reverts commit 08447b5deb.

Revert also _ctypes.c changes of the PyDict_ContainsString() change,
commit 6726626646.
2023-09-04 11:21:47 +02:00
Victor Stinner 03c4080c71
gh-108765: Python.h no longer includes <ctype.h> (#108831)
Remove <ctype.h> in C files which don't use it; only sre.c and
_decimal.c still use it.

Remove _PY_PORT_CTYPE_UTF8_ISSUE code from pyport.h:

* Code added by commit b5047fd019
  in 2004 for MacOSX and FreeBSD.
* Test removed by commit 52ddaefb6b
  in 2007, since Python str type now uses locale independent
  functions like Py_ISALPHA() and Py_TOLOWER() and the Unicode
  database.

Modules/_sre/sre.c replaces _PY_PORT_CTYPE_UTF8_ISSUE with new
functions: sre_isalnum(), sre_tolower(), sre_toupper().

Remove unused includes:

* _localemodule.c: remove <stdio.h>.
* getargs.c: remove <float.h>.
* dynload_win.c: remove <direct.h>, it no longer calls _getcwd()
  since commit fb1f68ed7c (in 2001).
2023-09-03 18:54:27 +02:00
Serhiy Storchaka 1796c191b4
gh-108494: Argument Clinic: inline parsing code for positional-only parameters in the limited C API (GH-108622) 2023-09-03 17:28:14 +03:00
Sergey B Kirpichev 9c995abd78
gh-102837: improve test coverage for math module (#102523)
- input checks for math_1(L989), math_1a(L1023), math_2(L1064,L1071), hypot(L2682), log(L2307), ldexp(L2168), ceil(L1165), floor(L1236,L1239) and dist(L2587,L2588,L2628).
- drop inaccessible "if" branch (L3518) in perm_comb_small()
- improve fsum coverage for exceptional cases (L1433,L1438,L1451,L1497), ditto fmod(L2378)
- rewrite modf to fix inaccessible case(L2229), ditto for pow(L2988)
    
(all line numbers are wrt the main branch at 5e6661bce9)
2023-09-03 01:48:47 -07:00
Jakub Kulík ecc61a6d76
gh-108765: include <unistd.h> in termios.c on Solaris (#108825) 2023-09-02 18:57:47 +00:00
Victor Stinner 0e6d582b3b
gh-63760: Don't declare gethostname() on Solaris (#108817)
Since 2005, Solaris defines gethostname(). socketmodule.c no longer
has to define gethostname() for Solaris.

Oracle Solaris and OpenSolaris have patches to remove the
gethostname() definition in Python:

* https://github.com/oracle/solaris-userland/blob/master/components/python/python37/patches/15-gethostname.patch
* https://github.com/OpenIndiana/oi-userland/blob/oi/hipster/components/python/python37/patches/15-gethostname.patch
* https://github.com/omniosorg/omnios-build/blob/master/build/python27/patches/24-gethostname.patch

Co-authored-by: Jakub Kulík <Kulikjak@gmail.com>
2023-09-02 17:54:59 +00:00
Victor Stinner bac1e6d695
gh-108765: multiprocessing.h includes <unistd.h> (#108823) 2023-09-02 17:54:39 +00:00
Victor Stinner e7de0c5901
gh-108765: Python.h no longer includes <sys/time.h> (#108775)
Python.h no longer includes <time.h>, <sys/select.h> and <sys/time.h>
standard header files.

* Add <time.h> include to xxsubtype.c.
* Add <sys/time.h> include to posixmodule.c and semaphore.c.
* readline.c includes <sys/select.h> instead of <sys/time.h>.
* resource.c no longer includes <time.h> and <sys/time.h>.
2023-09-02 17:51:19 +02:00
Victor Stinner 594b00057e
gh-108765: Python.h no longer includes <unistd.h> (#108783) 2023-09-02 16:50:18 +02:00
Victor Stinner 1f3e797dc0
gh-108765: Remove old prototypes from pyport.h (#108782)
Move prototypes of gethostname(), _getpty() and struct termios from
pyport.h to the C code using them: posixmodule.c, socketmodule.c and
termios.c.

Replace "#ifdef SOLARIS" with "#ifdef __sun".
2023-09-02 15:46:43 +02:00
Sam Gross aa52888e6a
gh-108777: Split _PyTime tests from _testinternalcapi.c (gh-108787) 2023-09-02 00:46:36 +00:00
Serhiy Storchaka 0e01fac315
Add Modules/_testcapi/util.h header (GH-108774)
It contains common macros used in C API tests.
2023-09-01 21:42:42 +03:00
Victor Stinner 578ebc5d5f
gh-108767: Replace ctype.h functions with pyctype.h functions (#108772)
Replace <ctype.h> locale dependent functions with Python "pyctype.h"
locale independent functions:

* Replace isalpha() with Py_ISALPHA().
* Replace isdigit() with Py_ISDIGIT().
* Replace isxdigit() with Py_ISXDIGIT().
* Replace tolower() with Py_TOLOWER().

Leave Modules/_sre/sre.c unchanged, it uses locale dependent
functions on purpose.

Include explicitly <ctype.h> in _decimal.c to get isascii().
2023-09-01 18:36:53 +02:00
Victor Stinner 3edcf743e8
gh-106320: Remove private _PyLong_Sign() (#108743)
Move the private _PyLong_Sign() and _PyLong_NumBits() functions
to the internal C API (pycore_long.h).

Modules/_testcapi/long.c now uses the internal C API.
2023-09-01 09:13:07 +02:00
Victor Stinner ad73674283
gh-107603: Argument Clinic: Only include pycore_gc.h if needed (#108726)
Argument Clinic now only includes pycore_gc.h if PyGC_Head is needed,
and only includes pycore_runtime.h if _Py_ID() is needed.

* Add 'condition' optional argument to Clinic.add_include().
* deprecate_keyword_use() includes pycore_runtime.h when using
  the _PyID() function.
* Fix rendering of includes: comments start at the column 35.
* Mark PC/clinic/_wmimodule.cpp.h and
  "Objects/stringlib/clinic/*.h.h" header files as generated in
  .gitattributes.

Effects:

* 42 header files generated by AC no longer include the internal C
  API, instead of 4 header files before. For example,
  Modules/clinic/_abc.c.h no longer includes the internal C API.
* Fix _testclinic_depr.c.h: it now always includes pycore_runtime.h
  to get _Py_ID().
2023-08-31 23:42:34 +02:00
Victor Stinner 2bd960b579
gh-108337: Add pyatomic.h header (#108701)
This adds a new header that provides atomic operations on common data
types. The intention is that this will be exposed through Python.h,
although that is not the case yet. The only immediate use is in
the test file.

Co-authored-by: Sam Gross <colesbury@gmail.com>
2023-08-31 21:41:18 +00:00
Victor Stinner 13a00078b8
gh-108634: Py_TRACE_REFS uses a hash table (#108663)
Python built with "configure --with-trace-refs" (tracing references)
is now ABI compatible with Python release build and debug build.
Moreover, it now also supports the Limited API.

Change Py_TRACE_REFS build:

* Remove _PyObject_EXTRA_INIT macro.
* The PyObject structure no longer has two extra members (_ob_prev
  and _ob_next).
* Use a hash table (_Py_hashtable_t) to trace references (all
  objects): PyInterpreterState.object_state.refchain.
* Py_TRACE_REFS build is now ABI compatible with release build and
  debug build.
* Limited C API extensions can now be built with Py_TRACE_REFS:
  xxlimited, xxlimited_35, _testclinic_limited.
* No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2()
  functions to PyModule_Create2TraceRefs() and
  PyModule_FromDefAndSpec2TraceRefs().
* _Py_PrintReferenceAddresses() is now called before
  finalize_interp_delete() which deletes the refchain hash table.
* test_tracemalloc find_trace() now also filters by size to ignore
  the memory allocated by _PyRefchain_Trace().

Test changes for Py_TRACE_REFS:

* Add test.support.Py_TRACE_REFS constant.
* Add test_sys.test_getobjects() to test sys.getobjects() function.
* test_exceptions skips test_recursion_normalizing_with_no_memory()
  and test_memory_error_in_PyErr_PrintEx() if Python is built with
  Py_TRACE_REFS.
* test_repl skips test_no_memory().
* test_capi skisp test_set_nomemory().
2023-08-31 18:33:34 +02:00
Victor Stinner dd32611f4f
gh-106320: winconsoleio.c includes pycore_pyerrors.h (#108720)
Fix compiler warning:

    warning C4013: '_PyErr_ChainExceptions1' undefined
2023-08-31 14:13:53 +00:00
Victor Stinner 79823c103b
gh-106320: Remove private _PyErr_ChainExceptions() (#108713)
Remove _PyErr_ChainExceptions(), _PyErr_ChainExceptions1() and
_PyErr_SetFromPyStatus() functions from the public C API.

* Move the private _PyErr_ChainExceptions() and
  _PyErr_ChainExceptions1() function to the internal C API
  (pycore_pyerrors.h).
* Move the private _PyErr_SetFromPyStatus() to the internal C API
  (pycore_initconfig.h).
* No longer export the _PyErr_ChainExceptions() function.
* Move run_in_subinterp_with_config() from _testcapi to
  _testinternalcapi.
2023-08-31 13:53:19 +02:00
Erlend E. Aasland 0b0c1d046c
gh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (#108632)
Deprecate passing the callback callable by keyword for the following
sqlite3.Connection APIs:

- set_authorizer(authorizer_callback)
- set_progress_handler(progress_handler, ...)
- set_trace_callback(trace_callback)

The affected parameters will become positional-only in Python 3.15.
2023-08-29 22:02:12 +02:00
Erlend E. Aasland 8178a88bd8
gh-107801: Improve the accuracy of io.IOBase.seek docs (#108268)
- Add param docstrings
- Link to os.SEEK_* constants
- Mention the return value in the initial paragraph

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-08-29 19:13:38 +00:00
Nikita Sobolev 30305d6d01
gh-108623: Fix compile warning in `Modules/_multiprocessing/semaphore.c` (#108624) 2023-08-29 13:43:11 +02:00
Victor Stinner bf08131e0a
gh-108494: Don't build _testclinic_limited with TraceRefs (#108608)
If Python is built with ./configure --with-trace-refs, don't build
the _testclinic_limited extension. The limited C API (Py_LIMITED_API)
is not compatible with Py_TRACE_REFS.
2023-08-29 13:39:42 +02:00
Victor Stinner 15c5a50797
gh-106320: Remove private pythonrun API (#108599)
Remove these private functions from the public C API:

* _PyRun_AnyFileObject()
* _PyRun_InteractiveLoopObject()
* _PyRun_SimpleFileObject()
* _Py_SourceAsString()

Move them to the internal C API: add a new pycore_pythonrun.h header
file. No longer export these functions.
2023-08-29 04:18:52 +02:00
Victor Stinner 39506ee565
gh-108240: Add pycore_capsule.h internal header file (#108596)
Move _PyCapsule_SetTraverse() definition to a new internal header
file: pycore_capsule.h.
2023-08-29 01:20:02 +00:00
Victor Stinner 0b6a4cb0df
gh-107149: Rename _PyUnstable_GetUnaryIntrinsicName() function (#108441)
* Rename _PyUnstable_GetUnaryIntrinsicName() to
  PyUnstable_GetUnaryIntrinsicName()
* Rename _PyUnstable_GetBinaryIntrinsicName()
  to PyUnstable_GetBinaryIntrinsicName().
2023-08-29 01:42:24 +02:00
Erlend E. Aasland 4116592b6f
gh-108278: Deprecate passing the three first params as keyword args for sqlite3 UDF creation APIs (#108281)
Deprecate passing name, number of arguments, and the callable as keyword
arguments, for the following sqlite3.Connection APIs:

- create_function(name, nargs, callable, ...)
- create_aggregate(name, nargs, callable)

The affected parameters will become positional-only in Python 3.15.
2023-08-28 13:32:07 +00:00
Serhiy Storchaka bc5356bb5d
gh-108494: Argument Clinic: fix support of Limited C API (GH-108536) 2023-08-28 16:04:27 +03:00
Erlend E. Aasland 38afa4af9b
gh-107801: Document io.TextIOWrapper.tell (#108265) 2023-08-27 20:35:27 +00:00
Clément Robert 09343dba44
Clarify distinction between datetime module and class in deprecation messages (GH-108073) 2023-08-27 16:09:40 +03:00
Serhiy Storchaka 2b15536fa9
gh-107913: Fix possible losses of OSError error codes (GH-107930)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
2023-08-27 00:35:06 +03:00
denballakh e407cea193
gh-107406: Add better `struct.Struct` repr (#107407) 2023-08-26 15:24:16 +05:30
Victor Stinner 8ba4714611
gh-106320: Remove private AC converter functions (#108505)
Move these private functions to the internal C API
(pycore_abstract.h):

* _Py_convert_optional_to_ssize_t()
* _PyNumber_Index()

Argument Clinic now emits #include "pycore_abstract.h" when these
functions are used.

The parser of the c-analyzer tool now uses a list of files which use
the limited C API, rather than a list of files using the internal C
API.
2023-08-26 04:05:17 +02:00
Victor Stinner 6353c21b78
gh-106320: Remove private _PyLong_FileDescriptor_Converter() (#108503)
Move the private _PyLong converter functions to the internal C API

* _PyLong_FileDescriptor_Converter(): moved to pycore_fileutils.h
* _PyLong_Size_t_Converter(): moved to pycore_long.h

Argument Clinic now emits includes for pycore_fileutils.h and
pycore_long.h when these functions are used.
2023-08-26 03:18:09 +02:00
Victor Stinner 713afb8804
gh-106320: Remove private _PyLong converter functions (#108499)
Move these private functions to the internal C API (pycore_long.h):

* _PyLong_UnsignedInt_Converter()
* _PyLong_UnsignedLongLong_Converter()
* _PyLong_UnsignedLong_Converter()
* _PyLong_UnsignedShort_Converter()

Argument Clinic now emits #include "pycore_long.h" when these
functions are used.
2023-08-26 02:24:27 +02:00
Victor Stinner 86bc9e35c4
gh-108494: AC supports pos-only args in limited C API (#108498)
AC now checks for "#define Py_LIMITED_API" pattern to use the limited
C API.
2023-08-26 00:39:24 +02:00
Victor Stinner 1dd9510977
gh-108494: Argument Clinic partial supports of Limited C API (#108495)
Argument Clinic now has a partial support of the
Limited API:

* Add --limited option to clinic.c.
* Add '_testclinic_limited' extension which is built with
  the limited C API version 3.13.
* For now, hardcode in clinic.py that "_testclinic_limited.c" targets
  the limited C API.
2023-08-25 23:22:08 +02:00
Victor Stinner 546cab8444
gh-106320: Remove private _PyTraceback functions (#108453)
Move private functions to the internal C API (pycore_traceback.h):

* _Py_DisplaySourceLine()
* _PyTraceback_Add()
2023-08-24 23:35:47 +00:00
Victor Stinner b32d4cad15
gh-108444: Replace _PyLong_AsInt() with PyLong_AsInt() (#108459)
Change generated by the command:

sed -i -e 's!_PyLong_AsInt!PyLong_AsInt!g' \
    $(find -name "*.c" -o -name "*.h")
2023-08-25 01:01:30 +02:00
Victor Stinner 4e5a7284ee
gh-108444: Argument Clinic uses PyLong_AsInt() (#108458)
Argument Clinic now uses the new public PyLong_AsInt(), rather than
the old name _PyLong_AsInt().
2023-08-25 00:51:22 +02:00
Victor Stinner be436e08b8
gh-108444: Add PyLong_AsInt() public function (#108445)
* Rename _PyLong_AsInt() to PyLong_AsInt().
* Add documentation.
* Add test.
* For now, keep _PyLong_AsInt() as an alias to PyLong_AsInt().
2023-08-24 23:55:30 +02:00
Victor Stinner a071ecb4d1
gh-106320: Remove private _PySys functions (#108452)
Move private functions to the internal C API (pycore_sysmodule.h):

* _PySys_GetAttr()
* _PySys_GetSizeOf()

No longer export most of these functions.

Fix also a typo in Include/cpython/optimizer.h: add a missing space.
2023-08-24 20:02:09 +00:00
Victor Stinner 26893016a7
gh-106320: Remove private _PyDict functions (#108449)
Move private functions to the internal C API (pycore_dict.h):

* _PyDictView_Intersect()
* _PyDictView_New()
* _PyDict_ContainsId()
* _PyDict_DelItemId()
* _PyDict_DelItem_KnownHash()
* _PyDict_GetItemIdWithError()
* _PyDict_GetItem_KnownHash()
* _PyDict_HasSplitTable()
* _PyDict_NewPresized()
* _PyDict_Next()
* _PyDict_Pop()
* _PyDict_SetItemId()
* _PyDict_SetItem_KnownHash()
* _PyDict_SizeOf()

No longer export most of these functions.

Move also the _PyDictViewObject structure to the internal C API.

Move dict_getitem_knownhash() function from _testcapi to the
_testinternalcapi extension. Update test_capi.test_dict for this
change.
2023-08-24 20:01:50 +00:00
Victor Stinner c494fb333b
gh-106320: Remove private _PyEval function (#108433)
Move private _PyEval functions to the internal C API
(pycore_ceval.h):

* _PyEval_GetBuiltin()
* _PyEval_GetBuiltinId()
* _PyEval_GetSwitchInterval()
* _PyEval_MakePendingCalls()
* _PyEval_SetProfile()
* _PyEval_SetSwitchInterval()
* _PyEval_SetTrace()

No longer export most of these functions.
2023-08-24 20:25:22 +02:00
Victor Stinner fa626b8ca0
gh-107178: Remove _testcapi.test_dict_capi() (#108436)
The new _testcapi.test_dict tests are more complete,
_testcapi.test_dict_capi() became redundant.
2023-08-24 17:43:29 +00:00
Victor Stinner 480a337366
gh-106320: Remove private _PyContext_NewHamtForTests() (#108434)
Move the function to the internal C API.
2023-08-24 19:37:41 +02:00
Victor Stinner c55e73112c
gh-106320: Remove private PyLong C API functions (#108429)
Remove private PyLong C API functions:

* _PyLong_AsByteArray()
* _PyLong_DivmodNear()
* _PyLong_Format()
* _PyLong_Frexp()
* _PyLong_FromByteArray()
* _PyLong_FromBytes()
* _PyLong_GCD()
* _PyLong_Lshift()
* _PyLong_Rshift()

Move these functions to the internal C API. No longer export
_PyLong_FromBytes() function.
2023-08-24 18:53:50 +02:00
Victor Stinner f1ae706ca5
gh-107211: No longer export internal functions (7) (#108425)
No longer export _PyUnicode_FromId() internal C API function.

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.

Update Tools/build/generate_token.py to update Include/internal/pycore_token.h
comments.
2023-08-24 17:40:56 +02:00
Victor Stinner 6726626646
gh-108314: Add PyDict_ContainsString() function (#108323) 2023-08-24 15:59:12 +02:00
Victor Stinner 513c89d012
gh-108240: Add _PyCapsule_SetTraverse() internal function (#108339)
The _socket extension uses _PyCapsule_SetTraverse() to visit and clear
the socket type in the garbage collector. So the _socket.socket type
can be cleared in some corner cases when it wasn't possible before.
2023-08-24 00:19:11 +02:00
Petr Viktorin 31b61d19ab
gh-108294: Add time.sleep audit event (GH-108298) 2023-08-23 11:00:22 +02:00
Victor Stinner c1c9cd61da
gh-89745: Remove commented code in getpath.c (#108307) 2023-08-22 20:25:06 +02:00
Victor Stinner 615f6e946d
gh-106320: Remove _PyDict_GetItemStringWithError() function (#108313)
Remove private _PyDict_GetItemStringWithError() function of the
public C API: the new PyDict_GetItemStringRef() can be used instead.

* Move private _PyDict_GetItemStringWithError() to the internal C API.
* _testcapi get_code_extra_index() uses PyDict_GetItemStringRef().
  Avoid using private functions in _testcapi which tests the public C
  API.
2023-08-22 18:17:25 +00:00
Victor Stinner 6541fe4ad7
Ignore _Py_write_noraise() result: cast to (void) (#108291)
Code using _Py_write_noraise() usually cannot report. Ignore errors
is the least surprising behavior for users.
2023-08-22 14:28:20 +00:00
Erlend E. Aasland 1a1bfc2891
gh-105539: Emit ResourceWarning if sqlite3 database is not closed explicitly (#108015) 2023-08-22 13:10:29 +02:00
Erlend E. Aasland 7f87ebbc3f
gh-107801: Improve the accuracy of io.TextIOWrapper.seek docs (#107933)
Clearly document the supported seek() operations:

- Rewind to the start of the stream
- Restore a previous stream position (given by tell())
- Fast-forward to the end of the stream
2023-08-22 09:37:53 +02:00
Victor Stinner 0dd3fc2a64
gh-108216: Cleanup #include in internal header files (#108228)
* Add missing includes.
* Remove unused includes.
* Update old include/symbol names to newer names.
* Mention at least one included symbol.
* Sort includes.
* Update Tools/cases_generator/generate_cases.py used to generated
  pycore_opcode_metadata.h.
* Update Parser/asdl_c.py used to generate pycore_ast.h.
* Cleanup also includes in _testcapimodule.c and _testinternalcapi.c.
2023-08-21 18:05:59 +00:00
Victor Stinner 21c0844742
gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221)
* pycore_intrinsics.h does nothing if included twice
  (add #ifndef and #define).
* Update Tools/cases_generator/generate_cases.py to generate the
  Py_BUILD_CORE test.
* _bz2, _lzma, _opcode and zlib extensions now define the
  Py_BUILD_CORE_MODULE macro to use internal headers
  (pycore_code.h, pycore_intrinsics.h and pycore_blocks_output_buffer.h).
2023-08-21 19:15:52 +02:00
Serhiy Storchaka 60942cccb1
gh-95065, gh-107704: Argument Clinic: support multiple '/ [from ...]' and '* [from ...]' markers (GH-108132) 2023-08-21 13:59:58 +00:00
Serhiy Storchaka 633ea217a8
gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918)
Such C API functions as PyErr_SetString(), PyErr_Format(),
PyErr_SetFromErrnoWithFilename() and many others no longer crash or
ignore errors if it failed to format the error message or decode the
filename. Instead, they keep a corresponding error.
2023-08-19 14:51:03 +03:00
Serhiy Storchaka 2f311437cd
gh-107704: Argument Clinic: add support for deprecating keyword use of parameters (GH-107984)
It is now possible to deprecate passing keyword arguments for
keyword-or-positional parameters with Argument Clinic, using the new
'/ [from X.Y]' syntax.
(To be read as "positional-only from Python version X.Y")

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-19 10:13:35 +03:00
Erlend E. Aasland dd4442c8f5
gh-107801: Improve the accuracy of os.lseek docs (#107935)
- name the last parameter *whence*, like it is for seek() methods on
  file objects
- add param docstrings
- structure the valid *whence* params

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-08-18 19:53:51 +02:00
Erlend E. Aasland fd19509220
gh-108083: Don't ignore exceptions in sqlite3.Connection.__init__() and .close() (#108084)
- Add explanatory comments
- Add return value to connection_close() for propagating errors
- Always check the return value of connection_exec_stmt()
- Assert pre/post state in remove_callbacks()
- Don't log unraisable exceptions in case of interpreter shutdown
- Make sure we're not initialized if reinit fails
- Try to close the database even if ROLLBACK fails

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-18 11:39:12 +00:00
Victor Stinner 3ff5ef2ad3
gh-108014: Add Py_IsFinalizing() function (#108032)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-18 12:34:41 +02:00
Irit Katriel 0b243c2f66
gh-105481: opcode.h is no longer generated during the build (#108080) 2023-08-17 17:07:58 +01:00
Mark Shannon 006e44f950
GH-108035: Remove the `_PyCFrame` struct as it is no longer needed for performance. (GH-108036) 2023-08-17 11:16:03 +01:00
SKO abd9cc52d9
gh-100061: Proper fix of the bug in the matching of possessive quantifiers (GH-102612)
Restore the global Input Stream pointer after trying to match a sub-pattern.

Co-authored-by: Ma Lin <animalize@users.noreply.github.com>
2023-08-16 10:43:45 +03:00
Finn Womack 0932272431
gh-106242: Fix path truncation in os.path.normpath (GH-106816) 2023-08-15 16:33:00 +01:00
Erlend E. Aasland 13c36dc9ae
gh-93057: Deprecate positional use of optional sqlite3.connect() params (#107948) 2023-08-15 08:09:56 +00:00
Dong-hee Na 580f357c66
Revert "gh-104469 : Convert _testcapi/vectorcall_limited.c to use AC … (gh-107951)
Revert "gh-104469 : Convert _testcapi/vectorcall_limited.c to use AC (gh-107857)"

This reverts commit 2e27da1895.
2023-08-14 23:25:57 +00:00
Erlend E. Aasland 6fbaba552a
gh-107938: Synchonise the signature of of sqlite3.connect and sqlite3.Connection.__init__ (#107939) 2023-08-14 16:37:44 +02:00
Erlend E. Aasland 9b75ada6e4
gh-107880: Teach Argument Clinic to clone __init__ and __new__ methods (#107885) 2023-08-13 12:13:11 +02:00
Nikita Sobolev 7ddc1eaff1
Improve `_typing.__doc__` (#107908) 2023-08-13 10:24:59 +01:00
nahyeon 2e27da1895
gh-104469 : Convert _testcapi/vectorcall_limited.c to use AC (gh-107857) 2023-08-12 09:57:35 +09:00
Serhiy Storchaka 04cc01453d
gh-106844: Fix issues in _winapi.LCMapStringEx (GH-107832)
* Strings with length from 2**31-1 to 2**32-2 always caused MemoryError,
   it doesn't matter how much memory is available.
* Strings with length exactly 2**32-1 caused OSError.
* Strings longer than 2**32-1 characters were truncated due to integer overflow bug.
* Strings containing the null character were truncated at the first null character.

Now strings longer than 2**31-1 characters caused OverflowError and the null character is allowed.
2023-08-11 21:13:46 +03:00
Serhiy Storchaka 3901c991e1
gh-84805: Autogenerate signature for METH_NOARGS and METH_O extension functions (GH-107794) 2023-08-11 18:08:38 +03:00
Mark Shannon 1d976b2da2
GH-106485: Handle dict subclasses correctly when dematerializing `__dict__` (GH-107837) 2023-08-10 13:34:00 +01:00
Erlend E. Aasland 39ef93edb9
gh-95065: Argument Clinic: Add functional tests of deprecated positionals (#107768)
Move the "deprecated positinal" tests from clinic.test.c to
_testclinic.c. Mock PY_VERSION_HEX in order to prevent generated
compiler warnings/errors to trigger. Put clinic code for deprecated
positionals in Modules/clinic/_testclinic_depr_star.c.h for easy
inspection of the generated code.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-10 07:19:05 +00:00
Kostya Farber 0f2fb6efb4
gh-107689: Add docstring to `ctypes.Array` (#107697) 2023-08-10 06:39:14 +00:00
Mina Galić f50c17243a
GH-107812: extend `socket`'s netlink support to FreeBSD (gh-107813) 2023-08-10 00:47:46 +00:00
Carl Meyer 2ec16fed14
gh-91054: make code watcher tests resilient to other watchers (#107821) 2023-08-09 16:42:32 -06:00
Tomas R de72677f8a
gh-107659: Add docstrings for ctypes.pointer and ctypes.POINTER (#107660) 2023-08-08 08:20:10 +00:00
Serhiy Storchaka bea5f93196
gh-107735: Add C API tests for PySys_GetObject() and PySys_SetObject() (GH-107736) 2023-08-07 22:29:01 +03:00
Serhiy Storchaka 16c9415fba
gh-107178: Add the C API tests for the Abstract Objects Layer (GH-107179)
Cover all the Mapping Protocol, almost all the Sequence Protocol
(except PySequence_Fast) and a part of the Object Protocol.

Move existing tests to Lib/test/test_capi/test_abstract.py and
Modules/_testcapi/abstract.c.

Add also tests for PyDict C API.
2023-08-07 18:51:43 +03:00
Brandt Bucher 05a824f294
GH-84436: Skip refcounting for known immortals (GH-107605) 2023-08-04 16:24:50 -07:00
Mark Shannon 2ba7c7f7b1
Add some GC stats to Py_STATS (GH-107581) 2023-08-04 10:34:23 +01:00
Erlend E. Aasland a443c310ac
gh-107609: Fix duplicate module check in Argument Clinic (#107610)
Also remove duplicate module def from _testcapi.
2023-08-04 07:28:25 +02:00
Pablo Galindo Salgado 77e09192b5
gh-107077: Raise SSLCertVerificationError even if the error is set via SSL_ERROR_SYSCALL (#107586)
Co-authored-by: T. Wouters <thomas@python.org>
2023-08-03 12:37:14 +00:00
Tomas R 62a3a15119
gh-107455: ctypes: Improve error messages when converting to an incompatible type (#107456) 2023-08-03 10:59:03 +05:30
Irit Katriel 6ef8f8ca88
gh-105481: the ENABLE_SPECIALIZATION flag does not need to be generated by the build script, or exposed in opcode.py (#107534) 2023-08-01 17:05:00 +00:00
Pieter Eendebak f7c9144c2c
gh-89013: Improve the performance of methodcaller (lazy version) (gh-107201) 2023-08-01 15:45:51 +09:00
Konstantin 08447b5deb
gh-46376: Return existing pointer when possible in ctypes (#107131) 2023-07-31 09:10:53 +02:00
Charlie Zhao 3979150a0d
gh-106263: Fix segfault in `signaldict_repr` in `_decimal` module (#106270)
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-07-30 13:58:54 +05:30
justdan6 11c055f5ff
gh-106881: Check for linux/limits.h before including it (#107397)
* Check for linux/limits.h before including it

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-07-28 15:08:43 -07:00
Charlie Zhao a43cc3fa1f
gh-106078: Isolate `decimal` module (#107287) 2023-07-28 11:28:14 +05:30
Eric Snow 8bdae1424b
gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359)
The _xxsubinterpreters module should not rely on internal API.  Some of the functions it uses were recently moved there however.  Here we move them back (and expose them properly).
2023-07-27 15:30:16 -06:00
Christopher Chavez f01e4cedba
gh-104432: Use `memcpy()` to avoid misaligned loads (#104433)
Fix potential unaligned memory access on C APIs involving returned sequences
of `char *` pointers within the :mod:`grp` and :mod:`socket` modules. These
were revealed using a ``-fsaniziter=alignment`` build on ARM macOS.
2023-07-27 19:20:25 +00:00
Christopher Chavez 6e850c30bb
gh-103731: Remove unneeded checks for TCL_WIDE_INT_TYPE (GH-103732) 2023-07-27 11:25:19 +03:00
shailshouryya 4b2e54bd3c
gh-107279 Add `<stddef.h>` to `Modules/zlibmodule.c` to fix failing builds (#107280) 2023-07-27 12:26:39 +05:30
Christopher Chavez d96ca41688
gh-103735: Tkinter: remove handling for uninteresting "procbody" Tcl value type (GH-103736) 2023-07-26 12:11:50 +03:00
Christopher Chavez b5ae7c4984
gh-106350: Tkinter: do not ignore return value of `mp_init()` (GH-106351) 2023-07-25 21:52:07 +03:00
Victor Stinner 1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Serhiy Storchaka 329e4a1a3f
gh-86493: Modernize modules initialization code (GH-106858)
Use PyModule_Add() or PyModule_AddObjectRef() instead of soft deprecated
PyModule_AddObject().
2023-07-25 14:34:49 +03:00
Victor Stinner 307186704d
gh-106320: Remove private _PyMem API (#107187)
Move private _PyMem functions to the internal C API (pycore_pymem.h):

* _PyMem_GetCurrentAllocatorName()
* _PyMem_RawStrdup()
* _PyMem_RawWcsdup()
* _PyMem_Strdup()

No longer export these functions.

Move pymem_getallocatorsname() function from _testcapi to _testinternalcapi,
since the API moved to the internal C API.
2023-07-24 18:48:06 +00:00
Victor Stinner fd66baf34a
gh-106320: Remove private _PyDict C API (#107145)
Move private _PyDict functions to the internal C API (pycore_dict.h):

* _PyDict_Contains_KnownHash()
* _PyDict_DebugMallocStats()
* _PyDict_DelItemIf()
* _PyDict_GetItemWithError()
* _PyDict_HasOnlyStringKeys()
* _PyDict_MaybeUntrack()
* _PyDict_MergeEx()

No longer export these functions.
2023-07-24 14:02:03 +00:00
Jérôme Carretero 8de8a817ee
Docs: fix typo in os.pwrite docstring (#107087) 2023-07-23 14:53:25 -07:00
Victor Stinner 7d41ead919
gh-106320: Remove _PyBytes_Join() C API (#107144)
Move private _PyBytes functions to the internal C API
(pycore_bytesobject.h):

* _PyBytes_DecodeEscape()
* _PyBytes_FormatEx()
* _PyBytes_FromHex()
* _PyBytes_Join()

No longer export these functions.
2023-07-23 20:10:12 +00:00
Victor Stinner 0d6dfd68d2
gh-106320: Remove private _PyObject C API (#107147)
Move private debug _PyObject functions to the internal C API
(pycore_object.h):

* _PyDebugAllocatorStats()
* _PyObject_CheckConsistency()
* _PyObject_DebugTypeStats()
* _PyObject_IsFreed()

No longer export most of these functions, except of
_PyObject_IsFreed().

Move test functions using _PyObject_IsFreed() from _testcapi to
_testinternalcapi. check_pyobject_is_freed() test no longer catch
_testcapi.error: the tested function cannot raise _testcapi.error.
2023-07-23 20:09:08 +00:00
Victor Stinner adb27ea2d5
gh-106320: Remove _PyIsSelectable_fd() C API (#107142)
Move _PyIsSelectable_fd() macro to the internal C API
(pycore_fileutils.h).
2023-07-23 19:07:12 +00:00
Dong-hee Na 0ae4870d09
gh-107122: Add clear method to dbm.ndbm module (gh-107126) 2023-07-23 14:26:23 +00:00
Dong-hee Na b273837fea
gh-107122: Add clear method to dbm.gdbm.module (gh-107127) 2023-07-23 13:51:12 +00:00
Victor Stinner 22422e9d1a
gh-106320: Remove private _PyInterpreterID C API (#107053)
Move the private _PyInterpreterID C API to the internal C API: add a
new pycore_interp_id.h header file.

Remove Include/interpreteridobject.h and
Include/cpython/interpreteridobject.h header files.
2023-07-22 19:31:55 +00:00
Victor Stinner 5e4af2a3e9
gh-106320: Move private _PySet API to the internal API (#107041)
* Add pycore_setobject.h header file.
* Move the following API to the internal C API:

  * _PySet_Dummy
  * _PySet_NextEntry()
  * _PySet_Update()
2023-07-22 17:04:34 +02:00
Victor Stinner d228825e08
gh-106320: Remove _PyOS_ReadlineTState API (#107034)
Remove _PyOS_ReadlineTState variable from the public C API.
The symbol is still exported for the readline shared extension.
2023-07-22 14:45:56 +00:00
Victor Stinner eda9ce1487
gh-106320: Move _PyNone_Type to the internal C API (#107030)
Move private types _PyNone_Type and _PyNotImplemented_Type to
internal C API.
2023-07-22 14:12:17 +00:00
Victor Stinner 89f9875448
gh-106320: Move private _PyHash API to the internal C API (#107026)
* No longer export most private _PyHash symbols, only export the ones
  which are needed by shared extensions.
* Modules/_xxtestfuzz/fuzzer.c now uses the internal C API.
2023-07-22 13:49:37 +00:00
Victor Stinner 41ca164551
gh-106004: Add PyDict_GetItemRef() function (#106005)
* Add PyDict_GetItemRef() and PyDict_GetItemStringRef() functions.
  Add these functions to the stable ABI version 3.13.
* Add unit tests on the PyDict C API in test_capi.
2023-07-21 23:10:51 +02:00
Serhiy Storchaka 8d397ee825
gh-47146: Fix reference counting in _testcapi.structmember initializer (GH-106862) 2023-07-21 12:30:14 +03:00
Irit Katriel 9c81fc2dbe
gh-105481: do not auto-generate pycore_intrinsics.h (#106913) 2023-07-20 17:46:04 +01:00
Charlie Zhao a1620dd7b7
gh-106078: Prepare to isolate decimal module (#106880)
* move signal_map to global_state

* move cond_map to global_state
2023-07-20 09:24:35 +00:00