Commit Graph

13788 Commits

Author SHA1 Message Date
Amin Alaee 7df8b16d28
gh-109782: Ensure `os.path.isdir` has the same signature on all platforms (GH-109790) 2023-09-28 16:17:30 +01:00
Ammar Askar a829356f86
gh-109098: Fuzz re module instead of internal sre (#109911)
* gh-109098: Fuzz re module instead of internal sre
* Fix c-analyzer globals test failure
* Put globals exception in ignored.tsv
2023-09-26 15:35:49 -07:00
Antoine Pitrou 0eb98837b6
gh-109593: Fix reentrancy issue in multiprocessing resource_tracker (#109629)
---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-09-26 13:57:25 +02:00
Serhiy Storchaka 8ac2085b80
gh-109631: Allow interruption of short repeated regex matches (GH-109867)
Counting for signal checking now continues in new match from the point where
it ended in the previous match instead of starting from 0.
2023-09-26 10:56:33 +03:00
Radislav Chugunov 1b8f2366b3
gh-109795: `_thread.start_new_thread`: allocate thread bootstate using raw memory allocator (#109808) 2023-09-25 17:38:06 +02:00
Victor Stinner eeb9070437
gh-109723: Fix build of _testclinic_limited on WASM (#109842)
Make sure that the Py_BUILD_CORE macro is not defined.
2023-09-25 14:43:54 +00:00
Victor Stinner 8c521f090e
gh-104469: Convert _testcapi/vectorcall_limited.c to use AC (#109691)
Co-authored-by: nahyeon <55136494+nahyeon-an@users.noreply.github.com>
2023-09-25 13:24:19 +02:00
Serhiy Storchaka b8d1744e7b
gh-109611: Add convenient C API function _PyFile_Flush() (GH-109612) 2023-09-23 09:35:30 +03:00
Victor Stinner 09a25616a9
gh-109723: Disable Py_BUILD_CORE in _testcapi (#109727)
Make sure that the internal C API is not tested by mistake by
_testcapi.

Undefine Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE macros in
Modules/_testcapi/parts.h: move code from _testcapimodule.c.

heaptype_relative.c and vectorcall_limited.c are using the limited C
API which is incompatible with the internal C API.

Move test_long_numbits() from _testcapi to _testinternalcapi since it
uses the internal C API "pycore_long.h".

Fix Modules/_testcapi/pyatomic.c: don't include Python.h directly,
just include _testcapi/parts.h.

Ajust "make check-c-globals" for these changes.
2023-09-22 14:54:37 +00:00
Sam Gross 2aceb21ae6
gh-109693: Remove pycore_atomic_funcs.h (#109694)
_PyUnicode_FromId() now uses pyatomic.h functions instead.
2023-09-21 22:57:20 +02:00
Victor Stinner d4cea794a7
gh-109613: _pystat_fromstructstat() checks for exceptions (#109618)
Fix os.stat() and os.DirEntry.stat(): check for exceptions.
Previously, on Python built in debug mode, these functions could
trigger a fatal Python error (and abort the process) when a function
succeeded with an exception set.

_pystat_fromstructstat() now exits immediately if an exception is
raised, rather only checking for exceptions at the end. It fix
following fatal error in fill_time():

    Fatal Python error: _Py_CheckSlotResult:
    Slot * of type int succeeded with an exception set
2023-09-21 09:55:06 +02:00
James Gerity def828995a
fixes gh-109559: Update `unicodedata` for Unicode 15.1.0 (GH-109560)
---------

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2023-09-19 22:07:47 -07:00
Ronan Pigott ddf2e953c2
gh-109033: Return filename with os.utime errors (#109034)
The filename was previously intentionally omitted from exception because
"it might confuse the user". Uncaught exceptions are not generally a
replacement for user-facing error messages, so obscuring this
information only has the effect of making the programmer's life more
difficult.
2023-09-20 01:18:23 +02:00
Eric Snow fd7e08a6f3
gh-76785: Use Pending Calls When Releasing Cross-Interpreter Data (gh-109556)
This fixes some crashes in the _xxinterpchannels module, due to a race between interpreters.
2023-09-19 15:01:34 -06:00
Sam Gross 0c89056fe5
gh-108724: Add PyMutex and _PyParkingLot APIs (gh-109344)
PyMutex is a one byte lock with fast, inlineable lock and unlock functions for the common uncontended case.  The design is based on WebKit's WTF::Lock.

PyMutex is built using the _PyParkingLot APIs, which provides a cross-platform futex-like API (based on WebKit's WTF::ParkingLot).  This internal API will be used for building other synchronization primitives used to implement PEP 703, such as one-time initialization and events.

This also includes tests and a mini benchmark in Tools/lockbench/lockbench.py to compare with the existing PyThread_type_lock.

Uncontended acquisition + release:
* Linux (x86-64): PyMutex: 11 ns, PyThread_type_lock: 44 ns
* macOS (arm64): PyMutex: 13 ns, PyThread_type_lock: 18 ns
* Windows (x86-64): PyMutex: 13 ns, PyThread_type_lock: 38 ns

PR Overview:

The primary purpose of this PR is to implement PyMutex, but there are a number of support pieces (described below).

* PyMutex:  A 1-byte lock that doesn't require memory allocation to initialize and is generally faster than the existing PyThread_type_lock.  The API is internal only for now.
* _PyParking_Lot:  A futex-like API based on the API of the same name in WebKit.  Used to implement PyMutex.
* _PyRawMutex:  A word sized lock used to implement _PyParking_Lot.
* PyEvent:  A one time event.  This was used a bunch in the "nogil" fork and is useful for testing the PyMutex implementation, so I've included it as part of the PR.
* pycore_llist.h:  Defines common operations on doubly-linked list.  Not strictly necessary (could do the list operations manually), but they come up frequently in the "nogil" fork. ( Similar to https://man.freebsd.org/cgi/man.cgi?queue)

---------

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2023-09-19 09:54:29 -06:00
Serhiy Storchaka ed582a2ed9
gh-109469: Silence compiler warnings on string comparisons in _testcapi (GH-109533) 2023-09-19 08:12:29 +03:00
Xuehai Pan 74f315edd0
gh-102757: fix function signature mismatch for `functools.reduce` between code and documentation (#102759) 2023-09-18 10:42:58 -06:00
Victor Stinner 0bb0d88e2d
gh-109496: Detect Py_DECREF() after dealloc in debug mode (#109539)
On a Python built in debug mode, Py_DECREF() now calls
_Py_NegativeRefcount() if the object is a dangling pointer to
deallocated memory: memory filled with 0xDD "dead byte" by the debug
hook on memory allocators. The fix is to check the reference count
*before* checking for _Py_IsImmortal().

Add test_decref_freed_object() to test_capi.test_misc.
2023-09-18 14:59:09 +00:00
Serhiy Storchaka add16f1a5e
gh-108511: Add C API functions which do not silently ignore errors (GH-109025)
Add the following functions:

* PyObject_HasAttrWithError()
* PyObject_HasAttrStringWithError()
* PyMapping_HasKeyWithError()
* PyMapping_HasKeyStringWithError()
2023-09-17 14:23:31 +03:00
Nikita Sobolev 47af188593
Add missing `PyDoc_STR` calls (#109393)
In files:

* Modules/_ctypes/cfield.c
* Modules/_struct.c
* Objects/dictobject.c
* Objects/typevarobject.c
* Objects/unionobject.c
2023-09-15 15:10:48 +02:00
Serhiy Storchaka 1f885df2a5
gh-107782: Use _testcapi to test non-representable signatures (GH-109325)
Builtin functions and methods that have non-representable signatures today
will have representable signatures yesterday, and they will become unusable
for testing this feature.

So we need to add special functions and methods to the _testcapi module
that always have non-representable signatures.
2023-09-14 09:12:17 +03:00
Charlie Zhao b39f65a495
gh-108346: Fix failed benchmark in decimal (#108353)
Fix benchmark in decimal to work again after the int str conversion limits.
2023-09-12 21:17:55 -07:00
Victor Stinner 517cd82ea7
gh-108987: Fix _thread.start_new_thread() race condition (#109135)
Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.

thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.

Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().
2023-09-11 17:27:03 +02:00
Serhiy Storchaka b4131a13cb
gh-109050: Remove remaining tests for legacy Unicode C API (GH-109068) 2023-09-09 05:44:46 +00:00
Victor Stinner f63d37877a
gh-104690: thread_run() checks for tstate dangling pointer (#109056)
thread_run() of _threadmodule.c now calls
_PyThreadState_CheckConsistency() to check if tstate is a dangling
pointer when Python is built in debug mode.

Rename ceval_gil.c is_tstate_valid() to
_PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
2023-09-08 11:50:46 +02:00
Victor Stinner b0edf3b98e
GH-91079: Rename C_RECURSION_LIMIT to Py_C_RECURSION_LIMIT (#108507)
Symbols of the C API should be prefixed by "Py_" to avoid conflict
with existing names in 3rd party C extensions on "#include <Python.h>".

test.pythoninfo now logs Py_C_RECURSION_LIMIT constant and other
_testcapi and _testinternalcapi constants.
2023-09-08 09:48:28 +00:00
Anders Kaseorg f42edf1e7b
gh-109045: Remove remaining LIMITED_API_AVAILABLE checks in tests (#109046)
Commit 13a00078b8 (#108663) made all
Python builds compatible with the Limited API, and removed the
LIMITED_API_AVAILABLE flag.  However, some tests were still checking
for that flag, so they were now being incorrectly skipped.  Remove
these checks to let these tests run again.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2023-09-07 04:42:58 +02:00
Victor Stinner 19eddb515a
gh-107211: No longer export internal _PyLong_FromUid() (#109037)
No longer export _PyLong_FromUid() and _Py_Sigset_Converter()
internal C API function.
2023-09-07 02:09:06 +02:00
Serhiy Storchaka 6f3c138dfa
gh-108751: Add copy.replace() function (GH-108752)
It creates a modified copy of an object by calling the object's
__replace__() method.

It is a generalization of dataclasses.replace(), named tuple's _replace()
method and replace() methods in various classes, and supports all these
stdlib classes.
2023-09-06 23:55:42 +03:00
Serhiy Storchaka bf414b7fcb
C API tests: use special markers to test that output parameters were set (GH-109014) 2023-09-06 22:02:01 +03:00
Victor Stinner a0773b89df
gh-108753: Enhance pystats (#108754)
Statistics gathering is now off by default. Use the "-X pystats"
command line option or set the new PYTHONSTATS environment variable
to 1 to turn statistics gathering on at Python startup.

Statistics are no longer dumped at exit if statistics gathering was
off or statistics have been cleared.

Changes:

* Add PYTHONSTATS environment variable.
* sys._stats_dump() now returns False if statistics are not dumped
  because they are all equal to zero.
* Add PyConfig._pystats member.
* Add tests on sys functions and on setting PyConfig._pystats to 1.
* Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h
  header files.
* Rename '_py_stats' variable to '_Py_stats'.
* Exclude Include/cpython/pystats.h from the Py_LIMITED_API.
* Move pystats.h include from object.h to Python.h.
* Add _Py_StatsOn() and _Py_StatsOff() functions. Remove
  '_py_stats_struct' variable from the API: make it static in
  specialize.c.
* Document API in Include/pystats.h and Include/cpython/pystats.h.
* Complete pystats documentation in Doc/using/configure.rst.
* Don't write "all zeros" stats: if _stats_off() and _stats_clear()
  or _stats_dump() were called.
* _PyEval_Fini() now always call _Py_PrintSpecializationStats() which
  does nothing if stats are all zeros.

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2023-09-06 15:54:59 +00:00
Nikita Sobolev 65a2bce704
gh-108717: Speedup `os.DirEntry.is_junction` function (#108718) 2023-09-06 08:52:53 +00:00
Petr Viktorin 230649f538
gh-108294: Add error handling for time.sleep audit event (GH-108363)
I've also cleaned the tests up a bit to make this easier to test.
2023-09-05 10:25:08 +02:00
Victor Stinner 676593859e
gh-106320: Remove private _PyErr_WriteUnraisableMsg() (#108863)
Move the private _PyErr_WriteUnraisableMsg() functions to the
internal C API (pycore_pyerrors.h).

Move write_unraisable_exc() from _testcapi to _testinternalcapi.
2023-09-05 01:54:55 +02:00
Victor Stinner 5a79d2ae57
Revert "gh-46376: Return existing pointer when possible in ctypes (#1… (#108688)
This reverts commit 08447b5deb.

Revert also _ctypes.c changes of the PyDict_ContainsString() change,
commit 6726626646.
2023-09-04 11:21:47 +02:00
Victor Stinner 03c4080c71
gh-108765: Python.h no longer includes <ctype.h> (#108831)
Remove <ctype.h> in C files which don't use it; only sre.c and
_decimal.c still use it.

Remove _PY_PORT_CTYPE_UTF8_ISSUE code from pyport.h:

* Code added by commit b5047fd019
  in 2004 for MacOSX and FreeBSD.
* Test removed by commit 52ddaefb6b
  in 2007, since Python str type now uses locale independent
  functions like Py_ISALPHA() and Py_TOLOWER() and the Unicode
  database.

Modules/_sre/sre.c replaces _PY_PORT_CTYPE_UTF8_ISSUE with new
functions: sre_isalnum(), sre_tolower(), sre_toupper().

Remove unused includes:

* _localemodule.c: remove <stdio.h>.
* getargs.c: remove <float.h>.
* dynload_win.c: remove <direct.h>, it no longer calls _getcwd()
  since commit fb1f68ed7c (in 2001).
2023-09-03 18:54:27 +02:00
Serhiy Storchaka 1796c191b4
gh-108494: Argument Clinic: inline parsing code for positional-only parameters in the limited C API (GH-108622) 2023-09-03 17:28:14 +03:00
Sergey B Kirpichev 9c995abd78
gh-102837: improve test coverage for math module (#102523)
- input checks for math_1(L989), math_1a(L1023), math_2(L1064,L1071), hypot(L2682), log(L2307), ldexp(L2168), ceil(L1165), floor(L1236,L1239) and dist(L2587,L2588,L2628).
- drop inaccessible "if" branch (L3518) in perm_comb_small()
- improve fsum coverage for exceptional cases (L1433,L1438,L1451,L1497), ditto fmod(L2378)
- rewrite modf to fix inaccessible case(L2229), ditto for pow(L2988)
    
(all line numbers are wrt the main branch at 5e6661bce9)
2023-09-03 01:48:47 -07:00
Jakub Kulík ecc61a6d76
gh-108765: include <unistd.h> in termios.c on Solaris (#108825) 2023-09-02 18:57:47 +00:00
Victor Stinner 0e6d582b3b
gh-63760: Don't declare gethostname() on Solaris (#108817)
Since 2005, Solaris defines gethostname(). socketmodule.c no longer
has to define gethostname() for Solaris.

Oracle Solaris and OpenSolaris have patches to remove the
gethostname() definition in Python:

* https://github.com/oracle/solaris-userland/blob/master/components/python/python37/patches/15-gethostname.patch
* https://github.com/OpenIndiana/oi-userland/blob/oi/hipster/components/python/python37/patches/15-gethostname.patch
* https://github.com/omniosorg/omnios-build/blob/master/build/python27/patches/24-gethostname.patch

Co-authored-by: Jakub Kulík <Kulikjak@gmail.com>
2023-09-02 17:54:59 +00:00
Victor Stinner bac1e6d695
gh-108765: multiprocessing.h includes <unistd.h> (#108823) 2023-09-02 17:54:39 +00:00
Victor Stinner e7de0c5901
gh-108765: Python.h no longer includes <sys/time.h> (#108775)
Python.h no longer includes <time.h>, <sys/select.h> and <sys/time.h>
standard header files.

* Add <time.h> include to xxsubtype.c.
* Add <sys/time.h> include to posixmodule.c and semaphore.c.
* readline.c includes <sys/select.h> instead of <sys/time.h>.
* resource.c no longer includes <time.h> and <sys/time.h>.
2023-09-02 17:51:19 +02:00
Victor Stinner 594b00057e
gh-108765: Python.h no longer includes <unistd.h> (#108783) 2023-09-02 16:50:18 +02:00
Victor Stinner 1f3e797dc0
gh-108765: Remove old prototypes from pyport.h (#108782)
Move prototypes of gethostname(), _getpty() and struct termios from
pyport.h to the C code using them: posixmodule.c, socketmodule.c and
termios.c.

Replace "#ifdef SOLARIS" with "#ifdef __sun".
2023-09-02 15:46:43 +02:00
Sam Gross aa52888e6a
gh-108777: Split _PyTime tests from _testinternalcapi.c (gh-108787) 2023-09-02 00:46:36 +00:00
Serhiy Storchaka 0e01fac315
Add Modules/_testcapi/util.h header (GH-108774)
It contains common macros used in C API tests.
2023-09-01 21:42:42 +03:00
Victor Stinner 578ebc5d5f
gh-108767: Replace ctype.h functions with pyctype.h functions (#108772)
Replace <ctype.h> locale dependent functions with Python "pyctype.h"
locale independent functions:

* Replace isalpha() with Py_ISALPHA().
* Replace isdigit() with Py_ISDIGIT().
* Replace isxdigit() with Py_ISXDIGIT().
* Replace tolower() with Py_TOLOWER().

Leave Modules/_sre/sre.c unchanged, it uses locale dependent
functions on purpose.

Include explicitly <ctype.h> in _decimal.c to get isascii().
2023-09-01 18:36:53 +02:00
Victor Stinner 3edcf743e8
gh-106320: Remove private _PyLong_Sign() (#108743)
Move the private _PyLong_Sign() and _PyLong_NumBits() functions
to the internal C API (pycore_long.h).

Modules/_testcapi/long.c now uses the internal C API.
2023-09-01 09:13:07 +02:00
Victor Stinner ad73674283
gh-107603: Argument Clinic: Only include pycore_gc.h if needed (#108726)
Argument Clinic now only includes pycore_gc.h if PyGC_Head is needed,
and only includes pycore_runtime.h if _Py_ID() is needed.

* Add 'condition' optional argument to Clinic.add_include().
* deprecate_keyword_use() includes pycore_runtime.h when using
  the _PyID() function.
* Fix rendering of includes: comments start at the column 35.
* Mark PC/clinic/_wmimodule.cpp.h and
  "Objects/stringlib/clinic/*.h.h" header files as generated in
  .gitattributes.

Effects:

* 42 header files generated by AC no longer include the internal C
  API, instead of 4 header files before. For example,
  Modules/clinic/_abc.c.h no longer includes the internal C API.
* Fix _testclinic_depr.c.h: it now always includes pycore_runtime.h
  to get _Py_ID().
2023-08-31 23:42:34 +02:00
Victor Stinner 2bd960b579
gh-108337: Add pyatomic.h header (#108701)
This adds a new header that provides atomic operations on common data
types. The intention is that this will be exposed through Python.h,
although that is not the case yet. The only immediate use is in
the test file.

Co-authored-by: Sam Gross <colesbury@gmail.com>
2023-08-31 21:41:18 +00:00
Victor Stinner 13a00078b8
gh-108634: Py_TRACE_REFS uses a hash table (#108663)
Python built with "configure --with-trace-refs" (tracing references)
is now ABI compatible with Python release build and debug build.
Moreover, it now also supports the Limited API.

Change Py_TRACE_REFS build:

* Remove _PyObject_EXTRA_INIT macro.
* The PyObject structure no longer has two extra members (_ob_prev
  and _ob_next).
* Use a hash table (_Py_hashtable_t) to trace references (all
  objects): PyInterpreterState.object_state.refchain.
* Py_TRACE_REFS build is now ABI compatible with release build and
  debug build.
* Limited C API extensions can now be built with Py_TRACE_REFS:
  xxlimited, xxlimited_35, _testclinic_limited.
* No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2()
  functions to PyModule_Create2TraceRefs() and
  PyModule_FromDefAndSpec2TraceRefs().
* _Py_PrintReferenceAddresses() is now called before
  finalize_interp_delete() which deletes the refchain hash table.
* test_tracemalloc find_trace() now also filters by size to ignore
  the memory allocated by _PyRefchain_Trace().

Test changes for Py_TRACE_REFS:

* Add test.support.Py_TRACE_REFS constant.
* Add test_sys.test_getobjects() to test sys.getobjects() function.
* test_exceptions skips test_recursion_normalizing_with_no_memory()
  and test_memory_error_in_PyErr_PrintEx() if Python is built with
  Py_TRACE_REFS.
* test_repl skips test_no_memory().
* test_capi skisp test_set_nomemory().
2023-08-31 18:33:34 +02:00
Victor Stinner dd32611f4f
gh-106320: winconsoleio.c includes pycore_pyerrors.h (#108720)
Fix compiler warning:

    warning C4013: '_PyErr_ChainExceptions1' undefined
2023-08-31 14:13:53 +00:00
Victor Stinner 79823c103b
gh-106320: Remove private _PyErr_ChainExceptions() (#108713)
Remove _PyErr_ChainExceptions(), _PyErr_ChainExceptions1() and
_PyErr_SetFromPyStatus() functions from the public C API.

* Move the private _PyErr_ChainExceptions() and
  _PyErr_ChainExceptions1() function to the internal C API
  (pycore_pyerrors.h).
* Move the private _PyErr_SetFromPyStatus() to the internal C API
  (pycore_initconfig.h).
* No longer export the _PyErr_ChainExceptions() function.
* Move run_in_subinterp_with_config() from _testcapi to
  _testinternalcapi.
2023-08-31 13:53:19 +02:00
Erlend E. Aasland 0b0c1d046c
gh-108278: Deprecate passing the first param of sqlite3.Connection callback APIs by keyword (#108632)
Deprecate passing the callback callable by keyword for the following
sqlite3.Connection APIs:

- set_authorizer(authorizer_callback)
- set_progress_handler(progress_handler, ...)
- set_trace_callback(trace_callback)

The affected parameters will become positional-only in Python 3.15.
2023-08-29 22:02:12 +02:00
Erlend E. Aasland 8178a88bd8
gh-107801: Improve the accuracy of io.IOBase.seek docs (#108268)
- Add param docstrings
- Link to os.SEEK_* constants
- Mention the return value in the initial paragraph

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-08-29 19:13:38 +00:00
Nikita Sobolev 30305d6d01
gh-108623: Fix compile warning in `Modules/_multiprocessing/semaphore.c` (#108624) 2023-08-29 13:43:11 +02:00
Victor Stinner bf08131e0a
gh-108494: Don't build _testclinic_limited with TraceRefs (#108608)
If Python is built with ./configure --with-trace-refs, don't build
the _testclinic_limited extension. The limited C API (Py_LIMITED_API)
is not compatible with Py_TRACE_REFS.
2023-08-29 13:39:42 +02:00
Victor Stinner 15c5a50797
gh-106320: Remove private pythonrun API (#108599)
Remove these private functions from the public C API:

* _PyRun_AnyFileObject()
* _PyRun_InteractiveLoopObject()
* _PyRun_SimpleFileObject()
* _Py_SourceAsString()

Move them to the internal C API: add a new pycore_pythonrun.h header
file. No longer export these functions.
2023-08-29 04:18:52 +02:00
Victor Stinner 39506ee565
gh-108240: Add pycore_capsule.h internal header file (#108596)
Move _PyCapsule_SetTraverse() definition to a new internal header
file: pycore_capsule.h.
2023-08-29 01:20:02 +00:00
Victor Stinner 0b6a4cb0df
gh-107149: Rename _PyUnstable_GetUnaryIntrinsicName() function (#108441)
* Rename _PyUnstable_GetUnaryIntrinsicName() to
  PyUnstable_GetUnaryIntrinsicName()
* Rename _PyUnstable_GetBinaryIntrinsicName()
  to PyUnstable_GetBinaryIntrinsicName().
2023-08-29 01:42:24 +02:00
Erlend E. Aasland 4116592b6f
gh-108278: Deprecate passing the three first params as keyword args for sqlite3 UDF creation APIs (#108281)
Deprecate passing name, number of arguments, and the callable as keyword
arguments, for the following sqlite3.Connection APIs:

- create_function(name, nargs, callable, ...)
- create_aggregate(name, nargs, callable)

The affected parameters will become positional-only in Python 3.15.
2023-08-28 13:32:07 +00:00
Serhiy Storchaka bc5356bb5d
gh-108494: Argument Clinic: fix support of Limited C API (GH-108536) 2023-08-28 16:04:27 +03:00
Erlend E. Aasland 38afa4af9b
gh-107801: Document io.TextIOWrapper.tell (#108265) 2023-08-27 20:35:27 +00:00
Clément Robert 09343dba44
Clarify distinction between datetime module and class in deprecation messages (GH-108073) 2023-08-27 16:09:40 +03:00
Serhiy Storchaka 2b15536fa9
gh-107913: Fix possible losses of OSError error codes (GH-107930)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
2023-08-27 00:35:06 +03:00
denballakh e407cea193
gh-107406: Add better `struct.Struct` repr (#107407) 2023-08-26 15:24:16 +05:30
Victor Stinner 8ba4714611
gh-106320: Remove private AC converter functions (#108505)
Move these private functions to the internal C API
(pycore_abstract.h):

* _Py_convert_optional_to_ssize_t()
* _PyNumber_Index()

Argument Clinic now emits #include "pycore_abstract.h" when these
functions are used.

The parser of the c-analyzer tool now uses a list of files which use
the limited C API, rather than a list of files using the internal C
API.
2023-08-26 04:05:17 +02:00
Victor Stinner 6353c21b78
gh-106320: Remove private _PyLong_FileDescriptor_Converter() (#108503)
Move the private _PyLong converter functions to the internal C API

* _PyLong_FileDescriptor_Converter(): moved to pycore_fileutils.h
* _PyLong_Size_t_Converter(): moved to pycore_long.h

Argument Clinic now emits includes for pycore_fileutils.h and
pycore_long.h when these functions are used.
2023-08-26 03:18:09 +02:00
Victor Stinner 713afb8804
gh-106320: Remove private _PyLong converter functions (#108499)
Move these private functions to the internal C API (pycore_long.h):

* _PyLong_UnsignedInt_Converter()
* _PyLong_UnsignedLongLong_Converter()
* _PyLong_UnsignedLong_Converter()
* _PyLong_UnsignedShort_Converter()

Argument Clinic now emits #include "pycore_long.h" when these
functions are used.
2023-08-26 02:24:27 +02:00
Victor Stinner 86bc9e35c4
gh-108494: AC supports pos-only args in limited C API (#108498)
AC now checks for "#define Py_LIMITED_API" pattern to use the limited
C API.
2023-08-26 00:39:24 +02:00
Victor Stinner 1dd9510977
gh-108494: Argument Clinic partial supports of Limited C API (#108495)
Argument Clinic now has a partial support of the
Limited API:

* Add --limited option to clinic.c.
* Add '_testclinic_limited' extension which is built with
  the limited C API version 3.13.
* For now, hardcode in clinic.py that "_testclinic_limited.c" targets
  the limited C API.
2023-08-25 23:22:08 +02:00
Victor Stinner 546cab8444
gh-106320: Remove private _PyTraceback functions (#108453)
Move private functions to the internal C API (pycore_traceback.h):

* _Py_DisplaySourceLine()
* _PyTraceback_Add()
2023-08-24 23:35:47 +00:00
Victor Stinner b32d4cad15
gh-108444: Replace _PyLong_AsInt() with PyLong_AsInt() (#108459)
Change generated by the command:

sed -i -e 's!_PyLong_AsInt!PyLong_AsInt!g' \
    $(find -name "*.c" -o -name "*.h")
2023-08-25 01:01:30 +02:00
Victor Stinner 4e5a7284ee
gh-108444: Argument Clinic uses PyLong_AsInt() (#108458)
Argument Clinic now uses the new public PyLong_AsInt(), rather than
the old name _PyLong_AsInt().
2023-08-25 00:51:22 +02:00
Victor Stinner be436e08b8
gh-108444: Add PyLong_AsInt() public function (#108445)
* Rename _PyLong_AsInt() to PyLong_AsInt().
* Add documentation.
* Add test.
* For now, keep _PyLong_AsInt() as an alias to PyLong_AsInt().
2023-08-24 23:55:30 +02:00
Victor Stinner a071ecb4d1
gh-106320: Remove private _PySys functions (#108452)
Move private functions to the internal C API (pycore_sysmodule.h):

* _PySys_GetAttr()
* _PySys_GetSizeOf()

No longer export most of these functions.

Fix also a typo in Include/cpython/optimizer.h: add a missing space.
2023-08-24 20:02:09 +00:00
Victor Stinner 26893016a7
gh-106320: Remove private _PyDict functions (#108449)
Move private functions to the internal C API (pycore_dict.h):

* _PyDictView_Intersect()
* _PyDictView_New()
* _PyDict_ContainsId()
* _PyDict_DelItemId()
* _PyDict_DelItem_KnownHash()
* _PyDict_GetItemIdWithError()
* _PyDict_GetItem_KnownHash()
* _PyDict_HasSplitTable()
* _PyDict_NewPresized()
* _PyDict_Next()
* _PyDict_Pop()
* _PyDict_SetItemId()
* _PyDict_SetItem_KnownHash()
* _PyDict_SizeOf()

No longer export most of these functions.

Move also the _PyDictViewObject structure to the internal C API.

Move dict_getitem_knownhash() function from _testcapi to the
_testinternalcapi extension. Update test_capi.test_dict for this
change.
2023-08-24 20:01:50 +00:00
Victor Stinner c494fb333b
gh-106320: Remove private _PyEval function (#108433)
Move private _PyEval functions to the internal C API
(pycore_ceval.h):

* _PyEval_GetBuiltin()
* _PyEval_GetBuiltinId()
* _PyEval_GetSwitchInterval()
* _PyEval_MakePendingCalls()
* _PyEval_SetProfile()
* _PyEval_SetSwitchInterval()
* _PyEval_SetTrace()

No longer export most of these functions.
2023-08-24 20:25:22 +02:00
Victor Stinner fa626b8ca0
gh-107178: Remove _testcapi.test_dict_capi() (#108436)
The new _testcapi.test_dict tests are more complete,
_testcapi.test_dict_capi() became redundant.
2023-08-24 17:43:29 +00:00
Victor Stinner 480a337366
gh-106320: Remove private _PyContext_NewHamtForTests() (#108434)
Move the function to the internal C API.
2023-08-24 19:37:41 +02:00
Victor Stinner c55e73112c
gh-106320: Remove private PyLong C API functions (#108429)
Remove private PyLong C API functions:

* _PyLong_AsByteArray()
* _PyLong_DivmodNear()
* _PyLong_Format()
* _PyLong_Frexp()
* _PyLong_FromByteArray()
* _PyLong_FromBytes()
* _PyLong_GCD()
* _PyLong_Lshift()
* _PyLong_Rshift()

Move these functions to the internal C API. No longer export
_PyLong_FromBytes() function.
2023-08-24 18:53:50 +02:00
Victor Stinner f1ae706ca5
gh-107211: No longer export internal functions (7) (#108425)
No longer export _PyUnicode_FromId() internal C API function.

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.

Update Tools/build/generate_token.py to update Include/internal/pycore_token.h
comments.
2023-08-24 17:40:56 +02:00
Victor Stinner 6726626646
gh-108314: Add PyDict_ContainsString() function (#108323) 2023-08-24 15:59:12 +02:00
Victor Stinner 513c89d012
gh-108240: Add _PyCapsule_SetTraverse() internal function (#108339)
The _socket extension uses _PyCapsule_SetTraverse() to visit and clear
the socket type in the garbage collector. So the _socket.socket type
can be cleared in some corner cases when it wasn't possible before.
2023-08-24 00:19:11 +02:00
Petr Viktorin 31b61d19ab
gh-108294: Add time.sleep audit event (GH-108298) 2023-08-23 11:00:22 +02:00
Victor Stinner c1c9cd61da
gh-89745: Remove commented code in getpath.c (#108307) 2023-08-22 20:25:06 +02:00
Victor Stinner 615f6e946d
gh-106320: Remove _PyDict_GetItemStringWithError() function (#108313)
Remove private _PyDict_GetItemStringWithError() function of the
public C API: the new PyDict_GetItemStringRef() can be used instead.

* Move private _PyDict_GetItemStringWithError() to the internal C API.
* _testcapi get_code_extra_index() uses PyDict_GetItemStringRef().
  Avoid using private functions in _testcapi which tests the public C
  API.
2023-08-22 18:17:25 +00:00
Victor Stinner 6541fe4ad7
Ignore _Py_write_noraise() result: cast to (void) (#108291)
Code using _Py_write_noraise() usually cannot report. Ignore errors
is the least surprising behavior for users.
2023-08-22 14:28:20 +00:00
Erlend E. Aasland 1a1bfc2891
gh-105539: Emit ResourceWarning if sqlite3 database is not closed explicitly (#108015) 2023-08-22 13:10:29 +02:00
Erlend E. Aasland 7f87ebbc3f
gh-107801: Improve the accuracy of io.TextIOWrapper.seek docs (#107933)
Clearly document the supported seek() operations:

- Rewind to the start of the stream
- Restore a previous stream position (given by tell())
- Fast-forward to the end of the stream
2023-08-22 09:37:53 +02:00
Victor Stinner 0dd3fc2a64
gh-108216: Cleanup #include in internal header files (#108228)
* Add missing includes.
* Remove unused includes.
* Update old include/symbol names to newer names.
* Mention at least one included symbol.
* Sort includes.
* Update Tools/cases_generator/generate_cases.py used to generated
  pycore_opcode_metadata.h.
* Update Parser/asdl_c.py used to generate pycore_ast.h.
* Cleanup also includes in _testcapimodule.c and _testinternalcapi.c.
2023-08-21 18:05:59 +00:00
Victor Stinner 21c0844742
gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221)
* pycore_intrinsics.h does nothing if included twice
  (add #ifndef and #define).
* Update Tools/cases_generator/generate_cases.py to generate the
  Py_BUILD_CORE test.
* _bz2, _lzma, _opcode and zlib extensions now define the
  Py_BUILD_CORE_MODULE macro to use internal headers
  (pycore_code.h, pycore_intrinsics.h and pycore_blocks_output_buffer.h).
2023-08-21 19:15:52 +02:00
Serhiy Storchaka 60942cccb1
gh-95065, gh-107704: Argument Clinic: support multiple '/ [from ...]' and '* [from ...]' markers (GH-108132) 2023-08-21 13:59:58 +00:00
Serhiy Storchaka 633ea217a8
gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918)
Such C API functions as PyErr_SetString(), PyErr_Format(),
PyErr_SetFromErrnoWithFilename() and many others no longer crash or
ignore errors if it failed to format the error message or decode the
filename. Instead, they keep a corresponding error.
2023-08-19 14:51:03 +03:00
Serhiy Storchaka 2f311437cd
gh-107704: Argument Clinic: add support for deprecating keyword use of parameters (GH-107984)
It is now possible to deprecate passing keyword arguments for
keyword-or-positional parameters with Argument Clinic, using the new
'/ [from X.Y]' syntax.
(To be read as "positional-only from Python version X.Y")

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-19 10:13:35 +03:00
Erlend E. Aasland dd4442c8f5
gh-107801: Improve the accuracy of os.lseek docs (#107935)
- name the last parameter *whence*, like it is for seek() methods on
  file objects
- add param docstrings
- structure the valid *whence* params

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-08-18 19:53:51 +02:00
Erlend E. Aasland fd19509220
gh-108083: Don't ignore exceptions in sqlite3.Connection.__init__() and .close() (#108084)
- Add explanatory comments
- Add return value to connection_close() for propagating errors
- Always check the return value of connection_exec_stmt()
- Assert pre/post state in remove_callbacks()
- Don't log unraisable exceptions in case of interpreter shutdown
- Make sure we're not initialized if reinit fails
- Try to close the database even if ROLLBACK fails

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-18 11:39:12 +00:00
Victor Stinner 3ff5ef2ad3
gh-108014: Add Py_IsFinalizing() function (#108032)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-18 12:34:41 +02:00
Irit Katriel 0b243c2f66
gh-105481: opcode.h is no longer generated during the build (#108080) 2023-08-17 17:07:58 +01:00
Mark Shannon 006e44f950
GH-108035: Remove the `_PyCFrame` struct as it is no longer needed for performance. (GH-108036) 2023-08-17 11:16:03 +01:00
SKO abd9cc52d9
gh-100061: Proper fix of the bug in the matching of possessive quantifiers (GH-102612)
Restore the global Input Stream pointer after trying to match a sub-pattern.

Co-authored-by: Ma Lin <animalize@users.noreply.github.com>
2023-08-16 10:43:45 +03:00
Finn Womack 0932272431
gh-106242: Fix path truncation in os.path.normpath (GH-106816) 2023-08-15 16:33:00 +01:00
Erlend E. Aasland 13c36dc9ae
gh-93057: Deprecate positional use of optional sqlite3.connect() params (#107948) 2023-08-15 08:09:56 +00:00
Dong-hee Na 580f357c66
Revert "gh-104469 : Convert _testcapi/vectorcall_limited.c to use AC … (gh-107951)
Revert "gh-104469 : Convert _testcapi/vectorcall_limited.c to use AC (gh-107857)"

This reverts commit 2e27da1895.
2023-08-14 23:25:57 +00:00
Erlend E. Aasland 6fbaba552a
gh-107938: Synchonise the signature of of sqlite3.connect and sqlite3.Connection.__init__ (#107939) 2023-08-14 16:37:44 +02:00
Erlend E. Aasland 9b75ada6e4
gh-107880: Teach Argument Clinic to clone __init__ and __new__ methods (#107885) 2023-08-13 12:13:11 +02:00
Nikita Sobolev 7ddc1eaff1
Improve `_typing.__doc__` (#107908) 2023-08-13 10:24:59 +01:00
nahyeon 2e27da1895
gh-104469 : Convert _testcapi/vectorcall_limited.c to use AC (gh-107857) 2023-08-12 09:57:35 +09:00
Serhiy Storchaka 04cc01453d
gh-106844: Fix issues in _winapi.LCMapStringEx (GH-107832)
* Strings with length from 2**31-1 to 2**32-2 always caused MemoryError,
   it doesn't matter how much memory is available.
* Strings with length exactly 2**32-1 caused OSError.
* Strings longer than 2**32-1 characters were truncated due to integer overflow bug.
* Strings containing the null character were truncated at the first null character.

Now strings longer than 2**31-1 characters caused OverflowError and the null character is allowed.
2023-08-11 21:13:46 +03:00
Serhiy Storchaka 3901c991e1
gh-84805: Autogenerate signature for METH_NOARGS and METH_O extension functions (GH-107794) 2023-08-11 18:08:38 +03:00
Mark Shannon 1d976b2da2
GH-106485: Handle dict subclasses correctly when dematerializing `__dict__` (GH-107837) 2023-08-10 13:34:00 +01:00
Erlend E. Aasland 39ef93edb9
gh-95065: Argument Clinic: Add functional tests of deprecated positionals (#107768)
Move the "deprecated positinal" tests from clinic.test.c to
_testclinic.c. Mock PY_VERSION_HEX in order to prevent generated
compiler warnings/errors to trigger. Put clinic code for deprecated
positionals in Modules/clinic/_testclinic_depr_star.c.h for easy
inspection of the generated code.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-10 07:19:05 +00:00
Kostya Farber 0f2fb6efb4
gh-107689: Add docstring to `ctypes.Array` (#107697) 2023-08-10 06:39:14 +00:00
Mina Galić f50c17243a
GH-107812: extend `socket`'s netlink support to FreeBSD (gh-107813) 2023-08-10 00:47:46 +00:00
Carl Meyer 2ec16fed14
gh-91054: make code watcher tests resilient to other watchers (#107821) 2023-08-09 16:42:32 -06:00
Tomas R de72677f8a
gh-107659: Add docstrings for ctypes.pointer and ctypes.POINTER (#107660) 2023-08-08 08:20:10 +00:00
Serhiy Storchaka bea5f93196
gh-107735: Add C API tests for PySys_GetObject() and PySys_SetObject() (GH-107736) 2023-08-07 22:29:01 +03:00
Serhiy Storchaka 16c9415fba
gh-107178: Add the C API tests for the Abstract Objects Layer (GH-107179)
Cover all the Mapping Protocol, almost all the Sequence Protocol
(except PySequence_Fast) and a part of the Object Protocol.

Move existing tests to Lib/test/test_capi/test_abstract.py and
Modules/_testcapi/abstract.c.

Add also tests for PyDict C API.
2023-08-07 18:51:43 +03:00
Brandt Bucher 05a824f294
GH-84436: Skip refcounting for known immortals (GH-107605) 2023-08-04 16:24:50 -07:00
Mark Shannon 2ba7c7f7b1
Add some GC stats to Py_STATS (GH-107581) 2023-08-04 10:34:23 +01:00
Erlend E. Aasland a443c310ac
gh-107609: Fix duplicate module check in Argument Clinic (#107610)
Also remove duplicate module def from _testcapi.
2023-08-04 07:28:25 +02:00
Pablo Galindo Salgado 77e09192b5
gh-107077: Raise SSLCertVerificationError even if the error is set via SSL_ERROR_SYSCALL (#107586)
Co-authored-by: T. Wouters <thomas@python.org>
2023-08-03 12:37:14 +00:00
Tomas R 62a3a15119
gh-107455: ctypes: Improve error messages when converting to an incompatible type (#107456) 2023-08-03 10:59:03 +05:30
Irit Katriel 6ef8f8ca88
gh-105481: the ENABLE_SPECIALIZATION flag does not need to be generated by the build script, or exposed in opcode.py (#107534) 2023-08-01 17:05:00 +00:00
Pieter Eendebak f7c9144c2c
gh-89013: Improve the performance of methodcaller (lazy version) (gh-107201) 2023-08-01 15:45:51 +09:00
Konstantin 08447b5deb
gh-46376: Return existing pointer when possible in ctypes (#107131) 2023-07-31 09:10:53 +02:00
Charlie Zhao 3979150a0d
gh-106263: Fix segfault in `signaldict_repr` in `_decimal` module (#106270)
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-07-30 13:58:54 +05:30
justdan6 11c055f5ff
gh-106881: Check for linux/limits.h before including it (#107397)
* Check for linux/limits.h before including it

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-07-28 15:08:43 -07:00
Charlie Zhao a43cc3fa1f
gh-106078: Isolate `decimal` module (#107287) 2023-07-28 11:28:14 +05:30
Eric Snow 8bdae1424b
gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359)
The _xxsubinterpreters module should not rely on internal API.  Some of the functions it uses were recently moved there however.  Here we move them back (and expose them properly).
2023-07-27 15:30:16 -06:00
Christopher Chavez f01e4cedba
gh-104432: Use `memcpy()` to avoid misaligned loads (#104433)
Fix potential unaligned memory access on C APIs involving returned sequences
of `char *` pointers within the :mod:`grp` and :mod:`socket` modules. These
were revealed using a ``-fsaniziter=alignment`` build on ARM macOS.
2023-07-27 19:20:25 +00:00
Christopher Chavez 6e850c30bb
gh-103731: Remove unneeded checks for TCL_WIDE_INT_TYPE (GH-103732) 2023-07-27 11:25:19 +03:00
shailshouryya 4b2e54bd3c
gh-107279 Add `<stddef.h>` to `Modules/zlibmodule.c` to fix failing builds (#107280) 2023-07-27 12:26:39 +05:30
Christopher Chavez d96ca41688
gh-103735: Tkinter: remove handling for uninteresting "procbody" Tcl value type (GH-103736) 2023-07-26 12:11:50 +03:00
Christopher Chavez b5ae7c4984
gh-106350: Tkinter: do not ignore return value of `mp_init()` (GH-106351) 2023-07-25 21:52:07 +03:00
Victor Stinner 1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Serhiy Storchaka 329e4a1a3f
gh-86493: Modernize modules initialization code (GH-106858)
Use PyModule_Add() or PyModule_AddObjectRef() instead of soft deprecated
PyModule_AddObject().
2023-07-25 14:34:49 +03:00
Victor Stinner 307186704d
gh-106320: Remove private _PyMem API (#107187)
Move private _PyMem functions to the internal C API (pycore_pymem.h):

* _PyMem_GetCurrentAllocatorName()
* _PyMem_RawStrdup()
* _PyMem_RawWcsdup()
* _PyMem_Strdup()

No longer export these functions.

Move pymem_getallocatorsname() function from _testcapi to _testinternalcapi,
since the API moved to the internal C API.
2023-07-24 18:48:06 +00:00
Victor Stinner fd66baf34a
gh-106320: Remove private _PyDict C API (#107145)
Move private _PyDict functions to the internal C API (pycore_dict.h):

* _PyDict_Contains_KnownHash()
* _PyDict_DebugMallocStats()
* _PyDict_DelItemIf()
* _PyDict_GetItemWithError()
* _PyDict_HasOnlyStringKeys()
* _PyDict_MaybeUntrack()
* _PyDict_MergeEx()

No longer export these functions.
2023-07-24 14:02:03 +00:00
Jérôme Carretero 8de8a817ee
Docs: fix typo in os.pwrite docstring (#107087) 2023-07-23 14:53:25 -07:00
Victor Stinner 7d41ead919
gh-106320: Remove _PyBytes_Join() C API (#107144)
Move private _PyBytes functions to the internal C API
(pycore_bytesobject.h):

* _PyBytes_DecodeEscape()
* _PyBytes_FormatEx()
* _PyBytes_FromHex()
* _PyBytes_Join()

No longer export these functions.
2023-07-23 20:10:12 +00:00
Victor Stinner 0d6dfd68d2
gh-106320: Remove private _PyObject C API (#107147)
Move private debug _PyObject functions to the internal C API
(pycore_object.h):

* _PyDebugAllocatorStats()
* _PyObject_CheckConsistency()
* _PyObject_DebugTypeStats()
* _PyObject_IsFreed()

No longer export most of these functions, except of
_PyObject_IsFreed().

Move test functions using _PyObject_IsFreed() from _testcapi to
_testinternalcapi. check_pyobject_is_freed() test no longer catch
_testcapi.error: the tested function cannot raise _testcapi.error.
2023-07-23 20:09:08 +00:00
Victor Stinner adb27ea2d5
gh-106320: Remove _PyIsSelectable_fd() C API (#107142)
Move _PyIsSelectable_fd() macro to the internal C API
(pycore_fileutils.h).
2023-07-23 19:07:12 +00:00
Dong-hee Na 0ae4870d09
gh-107122: Add clear method to dbm.ndbm module (gh-107126) 2023-07-23 14:26:23 +00:00
Dong-hee Na b273837fea
gh-107122: Add clear method to dbm.gdbm.module (gh-107127) 2023-07-23 13:51:12 +00:00
Victor Stinner 22422e9d1a
gh-106320: Remove private _PyInterpreterID C API (#107053)
Move the private _PyInterpreterID C API to the internal C API: add a
new pycore_interp_id.h header file.

Remove Include/interpreteridobject.h and
Include/cpython/interpreteridobject.h header files.
2023-07-22 19:31:55 +00:00
Victor Stinner 5e4af2a3e9
gh-106320: Move private _PySet API to the internal API (#107041)
* Add pycore_setobject.h header file.
* Move the following API to the internal C API:

  * _PySet_Dummy
  * _PySet_NextEntry()
  * _PySet_Update()
2023-07-22 17:04:34 +02:00
Victor Stinner d228825e08
gh-106320: Remove _PyOS_ReadlineTState API (#107034)
Remove _PyOS_ReadlineTState variable from the public C API.
The symbol is still exported for the readline shared extension.
2023-07-22 14:45:56 +00:00
Victor Stinner eda9ce1487
gh-106320: Move _PyNone_Type to the internal C API (#107030)
Move private types _PyNone_Type and _PyNotImplemented_Type to
internal C API.
2023-07-22 14:12:17 +00:00
Victor Stinner 89f9875448
gh-106320: Move private _PyHash API to the internal C API (#107026)
* No longer export most private _PyHash symbols, only export the ones
  which are needed by shared extensions.
* Modules/_xxtestfuzz/fuzzer.c now uses the internal C API.
2023-07-22 13:49:37 +00:00
Victor Stinner 41ca164551
gh-106004: Add PyDict_GetItemRef() function (#106005)
* Add PyDict_GetItemRef() and PyDict_GetItemStringRef() functions.
  Add these functions to the stable ABI version 3.13.
* Add unit tests on the PyDict C API in test_capi.
2023-07-21 23:10:51 +02:00
Serhiy Storchaka 8d397ee825
gh-47146: Fix reference counting in _testcapi.structmember initializer (GH-106862) 2023-07-21 12:30:14 +03:00
Irit Katriel 9c81fc2dbe
gh-105481: do not auto-generate pycore_intrinsics.h (#106913) 2023-07-20 17:46:04 +01:00
Charlie Zhao a1620dd7b7
gh-106078: Prepare to isolate decimal module (#106880)
* move signal_map to global_state

* move cond_map to global_state
2023-07-20 09:24:35 +00:00
Serhiy Storchaka a293fa5915
gh-86493: Use PyModule_Add() instead of PyModule_AddObjectRef() (GH-106860) 2023-07-18 23:59:53 +03:00
Irit Katriel 40f3f11a77
gh-105481: Generate the opcode lists in dis from data extracted from bytecodes.c (#106758) 2023-07-18 19:42:44 +01:00
Serhiy Storchaka 3e65baee72
gh-86493: Fix possible leaks in some modules initialization (GH-106768)
Fix _ssl, _stat, _testinternalcapi, _threadmodule, cmath, math, posix, time.
2023-07-18 10:50:47 +03:00
Serhiy Storchaka 745492355b
gh-86493: Fix possible leaks in modules initialization: _curses_panel, _decimal, posix, xxsubtype (GH-106767) 2023-07-18 09:00:22 +03:00
Nikita Sobolev ebf2c56b33
gh-106831: Fix NULL check of d2i_SSL_SESSION() result in _ssl.c (#106832) 2023-07-17 19:55:40 +00:00
Victor Stinner ad95c7253a
gh-106687: _ssl: use uint64_t for SSL options (#106700)
SSL_CTX_get_options() uses uint64_t for options:
https://www.openssl.org/docs/man3.1/man3/SSL_CTX_get_options.html

Fix this compiler warning on Windows with MSC:

    conversion from 'uint64_t' to 'long', possible loss of data
2023-07-17 17:55:30 +02:00
Inada Naoki 2566b74b26
gh-81283: compiler: remove indent from docstring (#106411)
Co-authored-by: Éric <merwok@netwok.org>
2023-07-15 19:33:32 +09:00
Irit Katriel 6a70edf24c
gh-105481: expose opcode metadata via the _opcode module (#106688) 2023-07-14 18:41:52 +01:00
Grant Ramsay 21d98be422
gh-105293: Do not call SSL_CTX_set_session_id_context on client side SSL context (#105295)
* gh-105293: Do not call SSL_CTX_set_session_id_context on client side SSL context

Openssl states this is a "server side only" operation.
Calling this on a client side socket can result in unexpected behavior

* Add news entry on SSL "set session id context" changes
2023-07-14 00:10:54 -07:00
Dennis Sweeney ab86426a34
gh-105235: Prevent reading outside buffer during mmap.find() (#105252)
* Add a special case for s[-m:] == p in _PyBytes_Find

* Add tests for _PyBytes_Find

* Make sure that start <= end in mmap.find
2023-07-12 22:50:45 -04:00
Serhiy Storchaka be1b968dc1
gh-106521: Remove _PyObject_LookupAttr() function (GH-106642) 2023-07-12 08:57:10 +03:00
Serhiy Storchaka 4bf43710d1
gh-106307: C API: Add PyMapping_GetOptionalItem() function (GH-106308)
Also add PyMapping_GetOptionalItemString() function.
2023-07-11 23:04:12 +03:00
Charlie Zhao e5c32a811c
gh-106078: Move external C-API functions to decimal module global state (#106616) 2023-07-11 16:36:42 +05:30
Eric Snow a840806d33
gh-105227: Add PyType_GetDict() (GH-105747)
This compensates for static builtin types having `tp_dict` set to `NULL`.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2023-07-10 18:41:02 +02:00
Charlie Zhao 93846657a3
gh-106078: Move static variables initialized once to decimal module global state (#106475) 2023-07-10 17:27:25 +05:30
Kumar Aditya 8fb6edf479
GH-104787: use managed weakrefs in `_asyncio` (#106516) 2023-07-10 17:18:03 +05:30
Serhiy Storchaka 51ea664d18
gh-99593: Add tests for Unicode C API (part 3) (GH-104728)
Add tests for codecs.
2023-07-10 14:04:34 +03:00
Victor Stinner ee46cb6aa9
gh-105927: PyWeakref_GetRef() returns 1 on success (#106561)
PyWeakref_GetRef() now returns 1 on success, and return 0 if the
reference is dead.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-07-09 15:50:26 +00:00
littlebutt's workshop d137c2cae2
gh-104469: Convert_testcapi/vectorcall.c to use AC (gh-106557) 2023-07-09 21:08:18 +09:00
Serhiy Storchaka b305c69d10
gh-106508: Improve debugging of the _sre module (GH-106509)
Now the VERBOSE macro can control tracing on per-pattern basis:

* 0 -- disabled
* 1 -- only if the DEBUG flag set
* 2 -- always
2023-07-08 18:00:39 +03:00
Radislav Chugunov 69a39bd9ad
gh-105873: Make `_xxsubinterpreters` use exception type name in shared exception (#105874) 2023-07-08 08:44:50 +00:00
Radislav Chugunov 2ef1dc37f0
gh-106524: Fix a crash in _sre.template() (GH-106525)
Some items remained uninitialized if _sre.template() was called with invalid
indices. Then attempt to clear them in the destructor led to dereferencing
of uninitialized pointer.
2023-07-08 10:47:01 +03:00
Charlie Zhao 1c9e493462
gh-106078: Move static objects related to `CONTEXTVAR` to the decimal module global state (#106395)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2023-07-08 13:14:24 +05:30
Mark Shannon 318ea2c72e
GH-106360: Support very basic superblock introspection (#106422)
* Add len() and indexing support to uop superblocks.
2023-07-04 17:23:00 +01:00
Nikita Sobolev 80f1c6c49b
gh-106406: Fix _Py_IsInterpreterFinalizing() in _winapi.c (#106408) 2023-07-04 16:19:20 +02:00
Victor Stinner c9ce983ae1
gh-106320: Remove private pylifecycle.h functions (#106400)
Remove private pylifecycle.h functions: move them to the internal C
API ( pycore_atexit.h, pycore_pylifecycle.h and pycore_signal.h). No
longer export most of these functions.

Move _testcapi.test_atexit() to _testinternalcapi.
2023-07-04 09:41:43 +00:00
Victor Stinner 8a73b57b9b
gh-106320: Remove _PyUnicode_TransformDecimalAndSpaceToASCII() (#106398)
Remove private _PyUnicode_TransformDecimalAndSpaceToASCII() and other
private _PyUnicode C API functions: move them to the internal C API
(pycore_unicodeobject.h). No longer most of these functions.

Replace _testcapi.unicode_transformdecimalandspacetoascii() with
_testinternal._PyUnicode_TransformDecimalAndSpaceToASCII().
2023-07-04 08:59:09 +00:00
Victor Stinner ec931fc394
gh-106320: Remove _PyBytesWriter C API (#106399)
Remove the _PyBytesWriter C API: move it to the internal C API
(pycore_bytesobject.h).
2023-07-04 08:27:23 +00:00
Victor Stinner 2e92edbf6d
gh-106320: Remove private _PyImport C API functions (#106383)
* Remove private _PyImport C API functions: move them to the internal
  C API (pycore_import.h).
* No longer export most of these private functions.
* _testcapi avoids private _PyImport_GetModuleAttrString().
2023-07-03 23:02:07 +00:00
Charlie Zhao bf06c6825c
gh-106078: Move `context template` to decimal module global state (#106346) 2023-07-04 01:40:01 +05:30
Carl Meyer 58906213cc
gh-91053: make func watcher tests resilient to other func watchers (#106286) 2023-07-03 08:25:22 -06:00
Victor Stinner c5afc97fc2
gh-106320: Remove private _PyErr C API functions (#106356)
Remove private _PyErr C API functions: move them to the internal
C API (pycore_pyerrors.h).
2023-07-03 10:48:50 +00:00
Victor Stinner 35963da40f
gh-106320: Create pycore_modsupport.h header file (#106355)
Remove the following functions from the C API, move them to the internal C
API: add a new pycore_modsupport.h internal header file:

* PyModule_CreateInitialized()
* _PyArg_NoKwnames()
* _Py_VaBuildStack()

No longer export these functions.
2023-07-03 09:39:11 +00:00
Victor Stinner 5ccbbe5bb9
gh-106320: Move _PyUnicodeWriter to the internal C API (#106342)
Move also _PyUnicode_FormatAdvancedWriter().

CJK codecs and multibytecodec.c now define the Py_BUILD_CORE_MODULE
macro.
2023-07-03 10:23:43 +02:00
Victor Stinner 4e3aa7cd31
gh-106320: _testcapi avoids private _PyUnicode_EqualToASCIIString() (#106341)
Replace private _PyUnicode_EqualToASCIIString() with public
PyUnicode_CompareWithASCIIString().
2023-07-02 21:56:58 +00:00
Victor Stinner c38c66687f
gh-106320: Add pycore_complexobject.h header file (#106339)
Add internal pycore_complexobject.h header file.

Move _Py_c_xxx() functions and _PyComplex_FormatAdvancedWriter()
function to this new header file.
2023-07-02 21:19:59 +00:00
Charlie Zhao dbefa88b27
gh-106078: Move DecimalException to _decimal state (#106301) 2023-07-02 16:50:40 +00:00
Victor Stinner bc7eb17084
gh-106320: Use _PyInterpreterState_GET() (#106336)
Replace PyInterpreterState_Get() with inlined
_PyInterpreterState_GET().
2023-07-02 16:37:37 +00:00
Inada Naoki d5bd32fb48
gh-104922: remove PY_SSIZE_T_CLEAN (#106315) 2023-07-02 15:07:46 +09:00
Victor Stinner 8571b271e7
gh-106320: Remove private _PyInterpreterState functions (#106325)
Remove private _PyThreadState and _PyInterpreterState C API
functions: move them to the internal C API (pycore_pystate.h and
pycore_interp.h). Don't export most of these functions anymore, but
still export functions used by tests.

Remove _PyThreadState_Prealloc() and _PyThreadState_Init() from the C
API, but keep it in the stable API.
2023-07-02 01:39:38 +00:00
Victor Stinner feb51f3a64
gh-106320: Remove private _PyTraceMalloc C API functions (#106324)
* Remove private _PyTraceMalloc C API functions: move them to the
  internal C API.
* Don't export most of these functions anymore, but still export
  _PyTraceMalloc_GetTraceback() used by tests.
* Rename Include/tracemalloc.h to Include/cpython/tracemalloc.h
2023-07-02 00:49:18 +00:00
Victor Stinner 18b1fdebe0
gh-106320: Remove _PyInterpreterState_Get() alias (#106321)
Replace calls to the (removed) slow _PyInterpreterState_Get() with
fast inlined _PyInterpreterState_GET() function.
2023-07-01 23:44:07 +00:00
Victor Stinner 46d77610fc
gh-106316: Remove pytime.h header file (#106317)
Remove the "cpython/pytime.h" header file: it only contained private
functions. Move functions to the internal pycore_time.h header file.

Move tests from _testcapi to _testinternalcapi. Rename also test
methods to have the same name than tested C functions.

No longer export these functions:

* _PyTime_Add()
* _PyTime_As100Nanoseconds()
* _PyTime_FromMicrosecondsClamp()
* _PyTime_FromTimespec()
* _PyTime_FromTimeval()
* _PyTime_GetPerfCounterWithInfo()
* _PyTime_MulDiv()
2023-07-01 22:27:18 +00:00
Victor Stinner 2efdd2a14e
gh-106023: Remove _PyObject_FastCall() function (#106265) 2023-06-30 11:46:43 +02:00
Victor Stinner 0b51463862
Remove private _PyCodec_Lookup() function (#106269)
Remove the following private functions of the C API:

* _PyCodecInfo_GetIncrementalDecoder()
* _PyCodecInfo_GetIncrementalEncoder()
* _PyCodec_DecodeText()
* _PyCodec_EncodeText()
* _PyCodec_Forget()
* _PyCodec_Lookup()
* _PyCodec_LookupTextEncoding()

Move these functions to a new pycore_codecs.h internal header file.

These functions are no longer exported.
2023-06-30 09:34:01 +00:00
Victor Stinner 8c5f74fc89
gh-106023: Update code using _PyObject_FastCall() (#106257)
Replace _PyObject_FastCall() calls with PyObject_Vectorcall().
2023-06-30 01:05:01 +00:00