Commit Graph

12897 Commits

Author SHA1 Message Date
Gregory P. Smith 4a08c4c469
bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186)
bpo-47151: Fallback to fork when vfork fails in subprocess. An OS kernel can specifically decide to disallow vfork() in a process. No need for that to prevent us from launching subprocesses.
2022-03-31 13:42:28 -07:00
Mark Shannon 74b95d86e0
bpo-40421: Add missing getters for frame object attributes to C-API. (GH-32114) 2022-03-31 17:13:25 +01:00
Erlend Egeberg Aasland b36d222110
bpo-45847: Port _tkinter to PY_STDLIB_MOD (GH-31698)
- Remove ``--with-tclk-*`` options from `configure`
- Use pkg-config to detect `_tkinter` dependencies (Tcl/Tk, X11)
- Manual override via environment variables `TCLTK_CFLAGS` and `TCLTK_LIBS`
2022-03-31 03:19:08 -07:00
Victor Stinner c14d7e4b81
bpo-47164: Add _PyASCIIObject_CAST() macro (GH-32191)
Add macros to cast objects to PyASCIIObject*, PyCompactUnicodeObject*
and PyUnicodeObject*: _PyASCIIObject_CAST(),
_PyCompactUnicodeObject_CAST() and _PyUnicodeObject_CAST(). Using
these new macros make the code more readable and check their argument
with: assert(PyUnicode_Check(op)).

Remove redundant assert(PyUnicode_Check(op)) in macros using directly
or indirectly these new CAST macros.

Replacing existing casts with these macros.
2022-03-31 09:59:27 +02:00
Andrew Svetlov a5ba445322
asyncio.Task: rename internal nested variable to don't hide another declaration from outer scope (GH-32181) 2022-03-30 00:33:51 +03:00
Ma Lin 356997cccc
bpo-35859: Fix a few long-standing bugs in re engine (GH-12427)
In rare cases, capturing group could get wrong result.

Regular expression engines in Perl and Java have similar bugs.
The new behavior now matches the behavior of more modern
RE engines: in the regex module and in PHP, Ruby and Node.js.
2022-03-29 17:31:01 +03:00
Pieter Eendebak 850687df47
bpo-47070: Add _PyBytes_Repeat() (GH-31999)
Use it where appropriate: the repeat functions of `array.array`, `bytes`, `bytearray`, and `str`.
2022-03-28 04:43:45 -04:00
ty f6b3a07b7d
bpo-44493: Add missing terminated NUL in sockaddr_un's length (GH-26866)
Add missing terminated NUL in sockaddr_un's length

- Linux: https://man7.org/linux/man-pages/man7/unix.7.html
- *BSD: SUN_LEN
2022-03-27 13:22:22 -07:00
Christian Heimes 5fd8c574e0
bpo-47098: Replace Keccak Code Package with tiny_sha3 (GH-32060) 2022-03-26 21:36:08 +01:00
Christian Heimes b16b6bb8da
bpo-47095: Use libb2 to provide blake2 implementation (GH-32059) 2022-03-26 20:52:24 +01:00
Christian Heimes 48e2010d92
bpo-47101: list only activated algorithms in hashlib.algorithms_available (GH-32076) 2022-03-23 13:30:05 -07:00
Andrew Svetlov 0360e9f346
bpo-46829: Deprecate passing a message into Future.cancel() and Task.cancel() (GH-31840)
After a long deliberation we ended up feeling that the message argument for Future.cancel(), added in 3.9, was a bad idea, so we're deprecating it in 3.11 and plan to remove it in 3.13.
2022-03-23 08:43:05 -07:00
Serhiy Storchaka 492d4109f4
bpo-42885: Optimize search for regular expressions starting with "\A" or "^" (GH-32021)
Affected functions are re.search(), re.split(), re.findall(), re.finditer()
and re.sub().
2022-03-22 17:27:55 +02:00
Mark Shannon 49daf6dba8
bpo-47045: Remove `f_state` field (GH-31963)
* Remove the f_state field from _PyInterpreterFrame

*  Make ownership of the frame explicit, replacing the is_generator field with an owner field.
2022-03-22 12:57:19 +00:00
Serhiy Storchaka 345b390ed6
bpo-433030: Add support of atomic grouping in regular expressions (GH-31982)
* Atomic grouping: (?>...).
* Possessive quantifiers: x++, x*+, x?+, x{m,n}+.
  Equivalent to (?>x+), (?>x*), (?>x?), (?>x{m,n}).

Co-authored-by: Jeffrey C. Jacobs <timehorse@users.sourceforge.net>
2022-03-21 18:28:22 +02:00
Serhiy Storchaka 08eb754d84
bpo-23691: Protect the re.finditer() iterator from re-entering (GH-32012) 2022-03-21 13:00:43 +02:00
Gregory P. Smith 9d1c4d69db
bpo-38256: Fix binascii.crc32() when inputs are 4+GiB (GH-32000)
When compiled with `USE_ZLIB_CRC32` defined (`configure` sets this on POSIX systems), `binascii.crc32(...)` failed to compute the correct value when the input data was >= 4GiB. Because the zlib crc32 API is limited to a 32-bit length.

This lines it up with the `zlib.crc32(...)` implementation that doesn't have that flaw.

**Performance:** This also adopts the same GIL releasing for larger inputs logic that `zlib.crc32` has, and causes the Windows build to always use zlib's crc32 instead of our slow C code as zlib is a required build dependency on Windows.
2022-03-20 12:28:15 -07:00
Ma Lin b3f2d4c8ba
bpo-47040: improve document of checksum functions (gh-31955)
Clarifies a versionchanged note on crc32 & adler32 docs that the workaround is only needed for Python 2 and earlier.
Also cleans up an unnecessary intermediate variable in the implementation.

Authored-By: Ma Lin / animalize
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2022-03-19 14:42:04 -07:00
Andrew Svetlov 0a8b8e0d26
bpo-47057: Use FASTCALL convention for FutureIter.throw() (GH-31973)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-03-19 14:01:46 +02:00
Pablo Galindo Salgado 8e3fde728f
bpo-46968: Check for 'sys/auxv.h' in the configure script (GH-31961) 2022-03-18 05:03:22 -07:00
David CARLIER 33698e8ff4
bpo-46030: socket module add couple of FreeBSD constants. (GH-30018)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2022-03-17 22:40:00 +02:00
Serhiy Storchaka c2e3c06139
bpo-46996: Remove support of Tcl/Tk < 8.5.12 (GH-31839) 2022-03-17 13:05:52 +02:00
Erlend Egeberg Aasland 4674fd4e93
bpo-44859: Raise more accurate exceptions in `sqlite3` (GH-27695)
* Improve exception compliance with PEP 249
* Raise InterfaceError instead of ProgrammingError for SQLITE_MISUSE.
  If SQLITE_MISUSE is raised, it is a sqlite3 module bug. Users of the
  sqlite3 module are not responsible for using the SQLite C API correctly.
* Don't overwrite BufferError with ValueError when conversion to BLOB fails.
* Raise ProgrammingError instead of Warning if user tries to execute() more
  than one SQL statement.
* Raise ProgrammingError instead of ValueError if an SQL query contains null characters.
* Make sure `_pysqlite_set_result` raises an exception if it returns -1.
2022-03-16 22:58:25 -07:00
Andrew Svetlov 30b5d41fab
bpo-47039: Normalize repr() of asyncio future and task objects (GH-31950) 2022-03-17 03:03:09 +02:00
Christian Heimes a4674f0194
bpo-40280: Detect presence of time.tzset and thread_time clock (GH-31898) 2022-03-15 21:55:35 +01:00
Oleg Iarygin 13b0412223
bpo-46920: Remove code that has explainers why it was disabled (GH-31813) 2022-03-14 17:04:22 +01:00
Oleg Iarygin e885ac3d5f
bpo-46920: Remove code that has no explainer why it was disabled (GH-31814) 2022-03-14 17:02:32 +01:00
Christian Heimes f00ced8396
bpo-40280: select: Use NULL for empty fdset (GH-31865)
wasm32-emscripten does not support exceptfds and requires NULL. Python
now passes NULL instead of a fdset pointer when the input list is empty.
This works fine on all platforms and might even be a tiny bit faster.
2022-03-14 14:40:28 +01:00
Andrew Svetlov 9523c0d84f
bpo-46994: Accept explicit contextvars.Context in asyncio create_task() API (GH-31837) 2022-03-14 13:54:13 +02:00
Andrew Svetlov 690490e4de
bpo-47003: Cleanup _overlapped module (GH-31848) 2022-03-13 23:28:45 +02:00
Alex Grönholm 9f04ee569c
bpo-46805: Add low level UDP socket functions to asyncio (GH-31455) 2022-03-13 18:42:29 +02:00
Victor Stinner 882d8096c2
bpo-46906: Add PyFloat_Pack8() to the C API (GH-31657)
Add new functions to pack and unpack C double (serialize and
deserialize):

* PyFloat_Pack2(), PyFloat_Pack4(), PyFloat_Pack8()
* PyFloat_Unpack2(), PyFloat_Unpack4(), PyFloat_Unpack8()

Document these functions and add unit tests.

Rename private functions and move them from the internal C API
to the public C API:

* _PyFloat_Pack2() => PyFloat_Pack2()
* _PyFloat_Pack4() => PyFloat_Pack4()
* _PyFloat_Pack8() => PyFloat_Pack8()
* _PyFloat_Unpack2() => PyFloat_Unpack2()
* _PyFloat_Unpack4() => PyFloat_Unpack4()
* _PyFloat_Unpack8() => PyFloat_Unpack8()

Replace the "unsigned char*" type with "char*" which is more common
and easy to use.
2022-03-12 00:10:02 +01:00
Christian Heimes ecfff63e06
bpo-40280: Disable AF_UNIX, AF_PACKET, SO_REUSE* on Emscripten (#31829)
Emscripten's socket emulation is limited. AF_UNIX, AF_PACKET, setsockopt(), and most SO_* constants are not supported.
2022-03-11 23:25:14 +01:00
Oleksandr Pavlyk 3b128c0548
bpo-46968: Fix faulthandler for Sapphire Rapids Xeon (GH-31789)
In Linux kernel 5.14 one can dynamically request size of altstacksize
based on hardware capabilities with getauxval(AT_MINSIGSTKSZ).

This changes allows for Python extension's request to Linux kernel
to use AMX_TILE instruction set on Sapphire Rapids Xeon processor
to succeed, unblocking use of the ISA in frameworks.

Introduced HAVE_LINUX_AUXVEC_H in configure.ac and pyconfig.h.in
Used cpython_autoconf:269 docker container to generate configure.
2022-03-11 23:19:35 +01:00
Victor Stinner dc374ac7b0
bpo-46968: Add os.sysconf_names['SC_MINSIGSTKSZ'] (GH-31824) 2022-03-11 23:01:40 +01:00
Oleg Iarygin f84c867dd7
Remove an old, elementtree-specific leak detector (GH-31811) 2022-03-11 09:47:42 -05:00
Erlend Egeberg Aasland 2d5835a019
sqlite3: normalise pre-acronym determiners (GH-31772)
For consistency, replace "a SQL" with "an SQL".
2022-03-10 17:52:47 -08:00
Erlend Egeberg Aasland e801e88744
bpo-45138: Revert GH-28240: Expand traced SQL statements (GH-31788)
This reverts commit d1777515f9.

Automerge-Triggered-By: GH:JelleZijlstra
2022-03-09 09:39:49 -08:00
Erlend Egeberg Aasland d1777515f9
bpo-45138: Expand traced SQL statements in `sqlite3` trace callback (GH-28240) 2022-03-08 18:46:40 -08:00
Erlend Egeberg Aasland b33a1ae703
Docstring: replace pysqlite with sqlite3 (GH-31758)
Replace two instances of "pysqlite" with "sqlite3" in sqlite3
docstrings. Also reword "is a no-op" to "does nothing" for clarity.
2022-03-08 18:45:48 -08:00
Christian Heimes c8a47e76a3
bpo-23325: Fix SIG_IGN and SIG_DFL int comparison in signal module (GH-31759) 2022-03-08 10:22:32 -08:00
Erlend Egeberg Aasland 4d95fa1ac5
bpo-46878: Purge 'non-standard' from sqlite3 docstrings (GH-31612) 2022-03-07 20:18:41 -08:00
Steve Dower 176835c3d5
bpo-46932: Update bundled libexpat to 2.4.7 (GH-31736) 2022-03-07 21:46:18 +00:00
Christian Heimes ca9689f8da
bpo-46933: Make pwd module optional (GH-31700)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2022-03-07 13:36:47 +01:00
Kumar Aditya 5c06dba21b
bpo-46937: convert remaining functions to AC in _weakref (GH-31705) 2022-03-07 18:57:45 +09:00
Christian Heimes 55a5e17d19
bpo-45582: Don't fail if ENV_PATH is None in getpath.py (GH-31699) 2022-03-06 20:49:27 +01:00
Serhiy Storchaka 6927632492
Remove trailing spaces (GH-31695) 2022-03-05 17:47:00 +02:00
slateny cedd2473a9
bpo-25415: Remove confusing sentence from IOBase docstrings (PR-31631) 2022-03-04 12:35:52 -05:00
Dong-hee Na d168c728f7
bpo-46541: Remove usage of _Py_IDENTIFIER from lzma module (GH-31683) 2022-03-05 01:38:56 +09:00
Victor Stinner 65b92ccdec
bpo-46913: Fix test_faulthandler.test_read_null() on UBSan (GH31672)
Disable undefined behavior sanitizer (UBSan) on
faulthandler._read_null().
2022-03-04 00:25:03 +01:00
Victor Stinner 4173d677a1
bpo-46913: Fix test_faulthandler.test_sigfpe() on UBSAN (GH-31662)
Disable undefined behavior sanitizer (UBSAN) on
faulthandler_sigfpe().
2022-03-03 21:45:01 +01:00
Erlend Egeberg Aasland 88567a9970
bpo-46874: Speed up sqlite3 user-defined aggregate 'step' method (GH-31604) 2022-03-03 22:54:36 +09:00
Victor Stinner b6b711a1aa
bpo-46848: Move _PyBytes_Find() to internal C API (GH-31642)
Move _PyBytes_Find() and _PyBytes_ReverseFind() functions to the
internal C API.

bytesobject.c now includes pycore_bytesobject.h.
2022-03-02 14:15:26 +01:00
Dennis Sweeney 6ddb09f35b
bpo-46848: Use stringlib/fastsearch in mmap (GH-31625)
Speed up mmap.find(). Add _PyBytes_Find() and _PyBytes_ReverseFind().
2022-03-01 23:46:30 -05:00
Kumar Aditya e91b0a7139
bpo-46541: remove usage of _Py_IDENTIFIER from _ssl module (GH-31599) 2022-03-02 00:38:21 +09:00
Dong-hee Na 0cc6364185
bpo-46541: Remove usage of _Py_IDENTIFIER from multibytecodec (GH-31475) 2022-03-01 23:35:43 +09:00
Guido van Rossum 7d611b4cab
bpo-46771: Remove two controversial lines from Task.cancel() (GH-31623)
Also from the _asyncio C accelerator module,
and adjust one test that the change caused to fail.

For more discussion see the discussion starting here:
https://github.com/python/cpython/pull/31394#issuecomment-1053545331

(Basically, @asvetlov proposed to return False from cancel()
when there is already a pending cancellation, and I went along,
even though it wasn't necessary for the task group implementation,
and @agronholm has come up with a counterexample that fails
because of this change.  So now I'm changing it back to the old
semantics (but still bumping the counter) until we can have a
proper discussion about this.)
2022-02-28 15:15:56 -08:00
Erlend Egeberg Aasland c32aef4853
bpo-46541: Remove unneeded visits from sqlite3 (GH-31609) 2022-02-28 18:07:40 +09:00
Dong-hee Na 088dd76dba
bpo-46541: Remove unnecessary Py_VISIT (GH-31608) 2022-02-28 08:06:58 +01:00
Victor Stinner e02c47528b
bpo-46606: os.getgroups() doesn't overallocate (GH-31569) 2022-02-27 00:14:28 +01:00
Victor Stinner 87af12bff3
bpo-46836: Rename InterpreterFrame to _PyInterpreterFrame (GH-31583)
Rename also struct _interpreter_frame to struct _PyInterpreterFrame.

Reduce risk of name conflicts if a project includes pycore_frame.h.
2022-02-25 16:22:00 +01:00
Petr Viktorin 2c228a7b8f
bpo-46748: Don't import <stdbool.h> in public headers (GH-31553)
<stdbool.h> is the standard/modern way to define embedd/extends Python free to define bool, true and false, but there are existing applications that use slightly different redefinitions, which fail if the header is included.

It's OK to use stdbool outside the public headers, though.

https://bugs.python.org/issue46748
2022-02-25 09:25:54 +01:00
Inada Naoki ad6c7003e3
bpo-46606: Remove redundant +1. (GH-31561) 2022-02-25 14:13:14 +09:00
Victor Stinner 1b2611eb02
bpo-46656: Remove Py_NO_NAN macro (GH-31160)
Building Python now requires support for floating point Not-a-Number
(NaN): remove the Py_NO_NAN macro.
2022-02-25 01:32:57 +01:00
Christian Heimes 38f331d465
bpo-45898: Remove duplicate symbols from _ctypes/cfield.c (GH-29791) 2022-02-24 20:51:57 +01:00
Tin Tvrtković 7fce1063b6
bpo-46771: Implement task cancel requests counter (GH-31513)
This changes cancelling() and uncancel() to return the count of pending cancellations.

This can be used to avoid bugs in certain edge cases (e.g. two timeouts going off at the same time).
2022-02-23 18:17:00 -08:00
Victor Stinner 9bbdde2180
bpo-45412: Add _PY_SHORT_FLOAT_REPR macro (GH-31171)
Remove the HAVE_PY_SET_53BIT_PRECISION macro (moved to the internal
C API).

* Move HAVE_PY_SET_53BIT_PRECISION macro to pycore_pymath.h.
* Replace PY_NO_SHORT_FLOAT_REPR macro with _PY_SHORT_FLOAT_REPR
  macro which is always defined. gcc -Wundef emits a warning when
  using _PY_SHORT_FLOAT_REPR but the macro is not defined, if
  pycore_pymath.h include was forgotten.
2022-02-23 18:16:23 +01:00
Dong-hee Na 1935e1cc28
bpo-46794: Bump up the libexpat version into 2.4.6 (GH-31487) 2022-02-23 10:40:30 +09:00
Eric Snow 1f455361ec
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366)
https://bugs.python.org/issue46765
2022-02-22 17:23:51 -07:00
Inada Naoki 74127b89a8
bpo-46606: Reduce stack usage of getgroups and setgroups (GH-31073)
NGROUPS_MAX was 32 before Linux 2.6.4 but 65536 since Linux 2.6.4.
2022-02-22 11:59:27 +09:00
Andrew Svetlov 4140bcb1cd
bpo-45390: Propagate CancelledError's message from cancelled task to its awaiter (GH-31383)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2022-02-21 22:59:04 +02:00
Dong-hee Na 2b86616456
bpo-46541: Remove usage of _Py_IDENTIFIER from pyexpat (GH-31468) 2022-02-21 23:46:52 +09:00
Christian Heimes be095f6c32
bpo-46232: Fix parsing of certs with bit string in DN (GH-30351) 2022-02-20 21:42:31 +01:00
Yilei "Dolee" Yang 6312c1052c
bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397)
The libexpat 2.4.1 upgrade from  introduced the following new exported symbols:

* `testingAccountingGetCountBytesDirect`
* `testingAccountingGetCountBytesIndirect`
* `unsignedCharToPrintable`
* `XML_SetBillionLaughsAttackProtectionActivationThreshold`
* `XML_SetBillionLaughsAttackProtectionMaximumAmplification`

We need to adjust [Modules/expat/pyexpatns.h](https://github.com/python/cpython/blob/master/Modules/expat/pyexpatns.h)

(The newer libexpat upgrade  has no new symbols).

Automerge-Triggered-By: GH:gpshead
2022-02-18 14:33:06 -08:00
Dong-hee Na 8cb5f707a8
bpo-46541: Remove usage of _Py_IDENTIFIER from array module (GH-31376) 2022-02-17 13:02:17 +09:00
Dong-hee Na e8a19b092f
bpo-46541: Remove usage of _Py_IDENTIFIER from mmap module (GH-31375) 2022-02-17 01:04:38 +09:00
Erlend Egeberg Aasland b2077117d1
bpo-46541: Replace _Py_IDENTIFIER with _Py_ID in sqlite3 (GH-31351) 2022-02-17 00:24:44 +09:00
Dong-hee Na d64f3caebe
bpo-46541: Remove usage of _Py_IDENTIFIER from csv module (GH-31372) 2022-02-17 00:24:03 +09:00
Dong-hee Na e59309b9d0
bpo-46541: Remove usage of _Py_IDENTIFIER from dbms modules (GH-31358) 2022-02-16 18:57:28 +09:00
Guido van Rossum 602630ac18
bpo-46752: Add TaskGroup; add Task..cancelled(),.uncancel() (GH-31270)
asyncio/taskgroups.py is an adaptation of taskgroup.py from EdgeDb, with the following key changes:

- Allow creating new tasks as long as the last task hasn't finished
- Raise [Base]ExceptionGroup (directly) rather than TaskGroupError deriving from MultiError
- Instead of monkey-patching the parent task's cancel() method,
  add a new public API to Task

The Task class has a new internal flag, `_cancel_requested`, which is set when `.cancel()` is called successfully. The `.cancelling()` method returns the value of this flag. Further `.cancel()` calls while this flag is set return False. To reset this flag, call `.uncancel()`.

Thus, a Task that catches and ignores `CancelledError` should call `.uncancel()` if it wants to be cancellable again; until it does so, it is deemed to be busy with uninterruptible cleanup.

This new Task API helps solve the problem where TaskGroup needs to distinguish between whether the parent task being cancelled "from the outside" vs. "from inside".

Co-authored-by: Yury Selivanov <yury@edgedb.com>
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2022-02-15 15:42:04 -08:00
DongGeon Lee 278fdd3e3a
bpo-46728: fix docstring of combinations_with_replacement for consistency (GH-31293) 2022-02-15 09:02:21 +09:00
Cyril Jouve 8aaaf7e182
bpo-46400: Update libexpat from 2.4.1 to 2.4.4 (GH-31022) 2022-02-13 00:29:41 +09:00
Jacob Walls 168fd6453b
bpo-45948: Remove constructor discrepancy in C version of ElementTree.XMLParser (GH-31152)
Both implementations accept target=None now.
2022-02-12 10:27:02 +02:00
Petr Viktorin 204946986f
bpo-46613: Add PyType_GetModuleByDef to the public API (GH-31081)
* Make PyType_GetModuleByDef public (remove underscore)

Co-authored-by: Victor Stinner <vstinner@python.org>
2022-02-11 17:22:11 +01:00
Dong-hee Na 0ac5372bf6
bpo-46323: Fix double-free issue for borrowed refs (GH-31272) 2022-02-11 17:48:47 +09:00
Dennis Sweeney 0a145069e8
bpo-44953: Add vectorcall for itemgetter and attrgetter instances (GH-27828) 2022-02-10 16:57:47 -05:00
Dong-hee Na db052851a7
bpo-46323: Allow alloca(0) for python callback function of ctypes (GH-31249) 2022-02-10 19:10:07 +09:00
Dong-hee Na d18120cd67
bpo-46323: Reduce stack usage of ctypes python callback function. (GH-31224) 2022-02-10 03:10:11 +09:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Dong-hee Na b5527688aa
bpo-46323: Use PyObject_Vectorcall while calling ctypes callback function (GH-31138) 2022-02-08 22:09:17 +09:00
Dong-hee Na e959dd9f5c
bpo-46323 Fix ref leak if ctypes.CFuncPtr raises an error. (GH-31209) 2022-02-08 14:22:13 +09:00
Victor Stinner f20ca766fe
bpo-46670: Fix #ifdef in sha3module.c (GH-31180)
* Test if HAVE_ALIGNED_REQUIRED is defined, not its value.
* Define explicitly NOT_PYTHON macro to 0.

Fix "gcc -Wundef" warnings.
2022-02-07 16:22:24 +01:00
Victor Stinner 4f1d3f33dd
bpo-46670: Remove unused macros in the Modules directory (GH-31194)
* bpo-46670: Remove unused macros in the Modules directory

* Add again LINKAT_DIR_FD_CONVERTER: generated by Argument Clinic
2022-02-07 16:21:31 +01:00
Victor Stinner d3e53bc532
bpo-39277: Fix PY_TIMEOUT_MAX cast in _threadmodule.c (GH-31195)
Cast PY_TIMEOUT_MAX to double, not to _PyTime_t.

Fix the clang warning:

Modules/_threadmodule.c:1648:26: warning: implicit conversion from
'_PyTime_t' (aka 'long') to 'double' changes value from
9223372036854775 to 9223372036854776
[-Wimplicit-const-int-float-conversion]
    double timeout_max = (_PyTime_t)PY_TIMEOUT_MAX * 1e-6;
                         ^~~~~~~~~~~~~~~~~~~~~~~~~ ~
2022-02-07 16:21:09 +01:00
Victor Stinner 4cce1352bb
bpo-46323: _ctypes.CFuncPtr fails if _argtypes_ is too long (GH-31188)
ctypes.CFUNCTYPE() and ctypes.WINFUNCTYPE() now fail to create the
type if its "_argtypes_" member contains too many arguments.
Previously, the error was only raised when calling a function.

Change also how CFUNCTYPE() and WINFUNCTYPE() handle KeyError to
prevent creating a chain of exceptions if ctypes.CFuncPtr raises an
error.
2022-02-07 14:53:15 +01:00
Zackery Spytz 59e004af63
bpo-40479: Fix undefined behavior in Modules/_hashopenssl.c (GH-31153)
va_end() must be called before returning.
2022-02-06 23:40:54 -08:00
Victor Stinner b556f53785
bpo-46670: Test if a macro is defined, not its value (GH-31178)
* audioop.c: #ifdef WORDS_BIGENDIAN
* ctypes.h: #ifdef USING_MALLOC_CLOSURE_DOT_C
* _ctypes/malloc_closure.c: #ifdef HAVE_FFI_CLOSURE_ALLOC
  and #ifdef USING_APPLE_OS_LIBFFI
* pytime.c: #ifdef __APPLE__
* unicodeobject.c: #ifdef HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION
2022-02-07 01:46:51 +01:00
Christian Heimes 96b344c2f1
bpo-40280: Address more test failures on Emscripten (GH-31050)
Co-authored-by: Brett Cannon <brett@python.org>
2022-02-05 20:52:01 +01:00
Benjamin Peterson 1aa6be06c4
closes bpo-46626: Expose IP_BIND_ADDRESS_NO_PORT socket option. (GH-31106) 2022-02-03 10:46:50 -08:00
Petr Viktorin 0ef0853012
bpo-46433: _PyType_GetModuleByDef: handle static types in MRO (GH-30696)
Automerge-Triggered-By: GH:encukou
2022-02-02 07:57:51 -08:00
Christian Heimes f66c857572
bpo-45459: Add Py_buffer to limited API (GH-29991)
- [x] ``Py_buffer`` struct
- [x] ``PyBuffer_*()`` API functions
- [x] ``PyBUF_*`` constants
- [x] ``Py_bf_getbuffer`` and ``Py_bf_releasebuffer`` type slots
- [x] ``PyMemoryView_FromBuffer()`` API
- [x] tests for limited API
- [x] ``make regen-limited-abi``
- [x] documentation update
- [ ] export ``PyPickleBuffer*()`` API ???
2022-02-02 07:03:10 -08:00
Kumar Aditya a05866ce3e
Remove Python 3.3 compatibility code from overlapped.c (GH-31049) 2022-02-02 10:16:36 +02:00
Victor Stinner 6c6a153dee
bpo-46417: signal: move siginfo_type to the module state (GH-30964) 2022-01-27 21:21:50 +01:00
Victor Stinner af32b3ef1f
bpo-40170: PyType_SUPPORTS_WEAKREFS() becomes a regular function (GH-30938)
Convert the PyType_SUPPORTS_WEAKREFS() macro to a regular function.
It no longer access the PyTypeObject.tp_weaklistoffset member
directly.

Add _PyType_SUPPORTS_WEAKREFS() static inline functions, used
internally by Python for best performance.
2022-01-27 03:00:55 +01:00
Erlend Egeberg Aasland 3eb3b4f270
bpo-43853: Expand test suite for SQLite UDF's (GH-27642) 2022-01-26 08:26:16 -08:00
Christian Heimes 6e5a193816
bpo-46513: Remove AC_C_CHAR_UNSIGNED / __CHAR_UNSIGNED__ (GH-30851) 2022-01-26 01:03:49 -08:00
Mark Dickinson 83a0ef2162
bpo-29882: Fix portability bug introduced in GH-30774 (#30794) 2022-01-23 09:59:34 +00:00
Victor Stinner 9c8e490b8f
bpo-46417: Clear _io module static objects at exit (GH-30807)
Add _PyIO_Fini() function, called by finalize_interp_clear(). It
clears static objects used by the _io extension module.
2022-01-22 23:22:20 +01:00
Kumar Aditya ea5b96842e
bpo-46469: Make asyncio generic classes return GenericAlias (GH-30777)
* bpo-46469: Make asyncio generic classes return GenericAlias

* 📜🤖 Added by blurb_it.

* Update Misc/NEWS.d/next/Library/2022-01-22-05-05-08.bpo-46469.plUab5.rst

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-01-22 13:28:53 +02:00
Erlend Egeberg Aasland 38afeb1a33
bpo-46249: Move set lastrowid out of the sqlite3 query loop (GH-30489) 2022-01-22 18:40:22 +09:00
Victor Stinner 8ee07dda13
bpo-46417: Add _PyType_GetSubclasses() function (GH-30761)
Add a new _PyType_GetSubclasses() function to get type's subclasses.

_PyType_GetSubclasses(type) returns a list which holds strong
refererences to subclasses. It is safer than iterating on
type->tp_subclasses which yields weak references and can be modified
in the loop.

_PyType_GetSubclasses(type) now holds a reference to the tp_subclasses
dict while creating the list of subclasses.

set_collection_flag_recursive() of _abc.c now uses
_PyType_GetSubclasses().
2022-01-21 23:29:10 +01:00
Thomas Klausner 40fcd16889
bpo-30512: Add CAN Socket support for NetBSD (GH-30066) 2022-01-21 09:44:05 +02:00
Victor Stinner d013b24135
bpo-46417: signal uses PyStructSequence_NewType() (GH-30735)
The signal module now creates its struct_siginfo type as a heap type
using PyStructSequence_NewType(), rather than using a static type.

Add 'siginfo_type' member to the global signal_state_t structure.
2022-01-21 04:02:38 +01:00
Victor Stinner 1781d55eb3
bpo-46417: _curses uses PyStructSequence_NewType() (GH-30736)
The _curses module now creates its ncurses_version type as a heap
type using PyStructSequence_NewType(), rather than using a static
type.

* Move _PyStructSequence_FiniType() definition to pycore_structseq.h.
* test.pythoninfo: log curses.ncurses_version.
2022-01-21 03:30:20 +01:00
Victor Stinner 17f268a4ae
bpo-46417: time module uses PyStructSequence_NewType() (GH-30734)
The time module now creates its struct_time type as a heap
type using PyStructSequence_NewType(), rather than using a static
type.

* Add a module state to the time module: add traverse, clear and free
  functions.
* Use PyModule_AddType().
* Remove the 'initialized' variable.
2022-01-21 02:52:43 +01:00
Victor Stinner f389b37fb1
bpo-46417: _thread uses PyStructSequence_NewType() (GH-30733)
The _thread module now creates its _ExceptHookArgs type as a heap
type using PyStructSequence_NewType(), rather than using a static
type.
2022-01-21 02:51:04 +01:00
Zane Bitter 27df7566bc
bpo-41857: mention timeout argument units in select.poll() and select.depoll() doc-strings (GH-22406) 2022-01-21 01:08:44 +02:00
Steve Dower 7407fe4c25
bpo-46028: Calculate base_executable by resolving symlinks in a venv (GH-30144) 2022-01-18 15:46:26 +00:00
Brandt Bucher 5cd9a162cd
bpo-46361: Fix "small" `int` caching (GH-30583) 2022-01-16 16:06:37 +00:00
Mark Dickinson d02c5e9b55
bpo-46258: Streamline isqrt fast path (#30333) 2022-01-15 09:58:04 +00:00
Christian Heimes cfbde65df3
bpo-46383: Fix signature of zoneinfo module_free function (GH-30607) 2022-01-15 09:52:19 +01:00
Victor Stinner 7c770d3350
bpo-46280: Fix tracemalloc_copy_domain() (GH-30591)
Test if tracemalloc_copy_traces() failed to allocated memory in
tracemalloc_copy_domain().
2022-01-14 05:11:38 +01:00
Eric Snow 322f962f3e
bpo-45953: Statically initialize all the non-object PyInterpreterState fields we can. (gh-30589)
https://bugs.python.org/issue45953
2022-01-13 17:17:28 -07:00
neonene d4e64cd4b0
bpo-46362: Ensure ntpath.abspath() uses the Windows API correctly (GH-30571)
This makes ntpath.abspath()/getpath_abspath() follow normpath(), since some WinAPIs such as PathCchSkipRoot() require backslashed paths.
2022-01-13 23:35:42 +00:00
Christian Heimes 276c234ce0
bpo-40479: Fix typo, flag must be set for OpenSSL < 3.0.0 (GH-30584) 2022-01-13 21:47:42 +01:00
Victor Stinner 1a4d1c1c9b
bpo-46070: _PyGC_Fini() untracks objects (GH-30577)
Py_EndInterpreter() now explicitly untracks all objects currently
tracked by the GC. Previously, if an object was used later by another
interpreter, calling PyObject_GC_UnTrack() on the object crashed if
the previous or the next object of the PyGC_Head structure became a
dangling pointer.
2022-01-13 19:28:32 +01:00
Christian Heimes 443b308fee
bpo-40479: Fix hashlib's usedforsecurity for OpenSSL 3.0.0 (GH-30455) 2022-01-13 09:46:38 +01:00
Christian Heimes a6ca8eee22
bpo-46315: Add ifdef HAVE_ feature checks for WASI compatibility (GH-30507) 2022-01-13 09:46:04 +01:00
Eric Snow ed57b36c32
bpo-45953: Statically allocate the main interpreter (and initial thread state). (gh-29883)
Previously, the main interpreter was allocated on the heap during runtime initialization.  Here we instead embed it into _PyRuntimeState, which means it is statically allocated as part of the _PyRuntime global.  The same goes for the initial thread state (of each interpreter, including the main one).  Consequently there are fewer allocations during runtime/interpreter init, fewer possible failures, and better memory locality.

FYI, this also helps efforts to consolidate globals, which in turns helps work on subinterpreter isolation.

https://bugs.python.org/issue45953
2022-01-12 16:28:46 -07:00
Christian Heimes e34c9367f8
bpo-40280: Allow to compile _testcapi as builtin module (GH-30559) 2022-01-12 20:27:37 +01:00
Christian Heimes 43839ba438
bpo-40280: Add --with-emscripten-target to build for browser or node (GH-30552)
Co-authored-by: Ethan Smith <ethan@ethanhs.me>
2022-01-12 16:08:19 +01:00
Victor Stinner 08bc1bad11
bpo-46303: Fix fileutils.h compiler warnings (GH-30550)
Add missing pycore_fileutils.h include in _tkinter.c and
_testconsole.c.
2022-01-12 00:35:26 +01:00
Victor Stinner ea1a54506b
bpo-46303: Move fileutils.h private functions to internal C API (GH-30484)
Move almost all private functions of Include/cpython/fileutils.h to
the internal C API Include/internal/pycore_fileutils.h.

Only keep _Py_fopen_obj() in Include/cpython/fileutils.h, since it's
used by _testcapi which must not use the internal C API.

Move EncodeLocaleEx() and DecodeLocaleEx() functions from _testcapi
to _testinternalcapi, since the C API moved to the internal C API.
2022-01-11 11:56:16 +01:00
Serhiy Storchaka 2d787971c6
bpo-37295: Use constant-time comb() and perm() for larger n depending on k (GH-30305) 2022-01-09 15:32:25 +02:00
Daniel c9dc1f491e
bpo-46297: Fix interpreter crash on startup with multiple PythonPaths set in registry (GH-30466) 2022-01-07 22:26:00 +00:00
Erlend Egeberg Aasland b127e70a8a
bpo-46070: Fix asyncio initialisation guard (GH-30423)
If init flag is set, exit successfully immediately.
If not, only set the flag after successful initialization.
2022-01-07 15:08:19 +01:00
Erlend Egeberg Aasland f1a58441ee
bpo-44092: Remove unused member `reset` from `sqlite3.Cursor` (GH-30377)
Automerge-Triggered-By: GH:pablogsal
2022-01-03 15:47:16 -08:00
Erlend Egeberg Aasland 9d6a239a34
bpo-44092: Don't reset statements/cursors before rollback (GH-26026)
In SQLite versions pre 3.7.11, pending statements would block a rollback.  This is no longer the case, so remove the workaround.
2022-01-03 19:02:39 +00:00
David CARLIER c960b191b8
bpo-46222: posixmodule sendfile FreeBSD's constants updates. (GH-30327)
* posixodule sendfile FreeBSD's constants updates.

* 📜🤖 Added by blurb_it.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-01-03 15:01:04 +02:00
Irit Katriel a82baed0e9
bpo-45615: Add missing test for printing traceback for non-exception. Fix traceback.py (GH-30091) 2022-01-02 09:34:03 +00:00
Mark Dickinson 0b58bac3e7
bpo-37295: More direct computation of power-of-two factor in math.comb (GH-30313)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2021-12-31 19:52:27 +00:00
Sebastian Pipping e18d81569f
bpo-45321: Add missing error codes to module `xml.parsers.expat.errors` (GH-30188)
The idea is to ensure that module `xml.parsers.expat.errors`
contains all known error codes and messages,
even when CPython is compiled or run with an outdated version of libexpat.

https://bugs.python.org/issue45321
2021-12-31 10:57:00 +01:00
David CARLIER 66c47b63a0
bpo-46176: mmap module adding MAP_STACK constant. (GH-30252) 2021-12-29 22:52:29 +09:00
Mark Dickinson 02b5417f11
bpo-37295: Speed up math.comb(n, k) for 0 <= k <= n <= 67 (GH-30275) 2021-12-28 12:26:40 +00:00
neonene 6214caafbe
bpo-40915: Avoid compiler warnings by fixing mmapmodule conversion from LARGE_INTEGER to Py_ssize_t (GH-30175) 2021-12-18 13:03:43 +00:00
Irit Katriel 396b58345f
bpo-45711: Remove type and traceback from exc_info (GH-30122)
* Do not PUSH/POP traceback or type to the stack as part of exc_info

* Remove exc_traceback and exc_type from _PyErr_StackItem

* Add to what's new, because this change breaks things like Cython
2021-12-17 14:46:22 +00:00
Mark Shannon 342b93f9f2
bpo-46072: Add --with-pystats configure option to simplify gathering of VM stats (GH-30116)
* Simplify specialization stats collection macros.

* Add --enable-pystats option to configure.

* Update specialization summary script to handle larger number of kinds
2021-12-15 15:32:32 +00:00
Christian Heimes 74821b3053
bpo-46023: Skip build if module is marked as DISABLED (GH-30100) 2021-12-14 15:42:46 +01:00
Gareth Rees a62be77266
bpo-45643: Add signal.SIGSTKFLT on platforms where this is defined (GH-29266) 2021-12-13 18:22:43 +01:00
Kumar Aditya 41026c3155
bpo-45855: Replaced deprecated `PyImport_ImportModuleNoBlock` with PyImport_ImportModule (GH-30046) 2021-12-12 10:45:20 +02:00
Steve Dower bfc59ed0a0
bpo-46049: Fixes ._pth support on non-Windows (GH-30051) 2021-12-11 15:06:17 +00:00
Steve Dower 971ece8e17
bpo-46048: Fix parsing of single character lines in getpath readlines() (GH-30048) 2021-12-11 13:43:40 +00:00
neonene 3f398a77d3
bpo-45582: Fix test_embed failure during a PGO build on Windows (GH-30014)
This defines VPATH differently in PGO instrumentation builds, to account for a different default output directory. It also adds sys._vpath on Windows to make the value available to sysconfig so that it can be used in tests.
2021-12-10 17:13:55 +00:00
Christian Heimes 036bbb1d1b
bpo-46023: Fix makesetup handling of disabled rule (GH-30001) 2021-12-10 14:11:55 +01:00
Thomas Klausner 2fb797e93c
bpo-46000: Improve NetBSD curses compatibility (GH-29947) 2021-12-10 11:41:47 +02:00
Gabriele N. Tornetta 50669083fe
bpo-43931: Export Python version as API data (GH-25577)
When Python is embedded in other applications, it is not easy to determine which version of Python is being used. This change exposes the Python version as part of the API data. Tools like Austin (https://github.com/P403n1x87/austin) can benefit from this data when targeting applications like uWSGI, as the Python version can then be inferred systematically by looking at the exported symbols rather than relying on unreliable pattern matching or other hacks (like remote code execution etc...).

Automerge-Triggered-By: GH:pablogsal
2021-12-09 17:52:05 -08:00
Steve Dower 3363e1cb05
bpo-46018: Ensure that math.expm1 does not raise on underflow (GH-29997) 2021-12-09 18:31:54 +00:00
Pablo Galindo Salgado f0d290d25c
bpo-46025: Fix a crash in the atexit module for auto-unregistering functions (GH-30002) 2021-12-09 13:53:44 +00:00
David CARLIER 267539bff7
bpo-46016: fcntl module add FreeBSD's F_DUP2FD_CLOEXEC flag support (GH-29993) 2021-12-08 23:28:51 +01:00
Steve Dower 7778116c2f
bpo-46015: Fixes calculation of sys.path in a venv on Windows (GH-29992)
Also ensures that pybuilddir.txt is written early enough in the build to be picked up by later steps.
2021-12-08 19:25:58 +00:00
Steve Dower b0b3086279
bpo-45582: Write empty pybuilddir.txt on Windows to allow relocatable build directories (GH-29979) 2021-12-08 02:18:21 +00:00
Eric Snow 758b74e71e
bpo-46008: Add _PyInterpreterState_Main(). (gh-29978)
PyInterpreterState_Main() is a plain function exposed in the public C-API. For internal usage we can take the more efficient approach in this PR.

https://bugs.python.org/issue46008
2021-12-07 18:56:06 -07:00
Eric Snow 1f384e3184
bpo-46008: Stop calling _PyThreadState_Init() in new_threadstate(). (gh-29973)
This simplifies new_threadstate().  We also rename _PyThreadState_Init() to _PyThreadState_SetCurrent() to reflect what it actually does.

https://bugs.python.org/issue46008
2021-12-07 17:26:29 -07:00
Eric Snow 313f92a57b
bpo-46008: Move thread-related interpreter state into a sub-struct. (gh-29971)
This parallels _PyRuntimeState.interpreters.  Doing this helps make it more clear what part of PyInterpreterState relates to its threads.

https://bugs.python.org/issue46008
2021-12-07 14:03:47 -07:00
Christian Heimes 91b59a3fcd
bpo-45847: Revert Port _ctypes partly to PY_STDLIB_MOD (GH-29747) (GH-29969) 2021-12-07 20:56:41 +01:00
Christian Heimes 06c4ae8b13
bpo-45582: Fix framework path and bootstrap build (GH-29954)
* Check NS API return values for NULL to prevent segfault in
  ``_bootstrap_python``.
* Set modPathInitialized to 1 so the ``decode_to_dict`` path is used.

Signed-off-by: Christian Heimes <christian@python.org>
2021-12-07 19:09:53 +01:00
Mark Shannon 8319114fee
bpo-45947: Place dict and values pointer at fixed (negative) offset just before GC header. (GH-29879)
* Place __dict__ immediately before GC header for plain Python objects.

* Fix up lazy dict creation logic to use managed dict pointers.

* Manage values pointer, placing them directly before managed dict pointers.

* Convert hint-based load/store attr specialization target managed dict classes.

* Specialize LOAD_METHOD for managed dict objects.

* Remove unsafe _PyObject_GC_Calloc function.

* Remove unsafe _PyObject_GC_Malloc() function.

* Add comment explaning use of Py_TPFLAGS_MANAGED_DICT.
2021-12-07 16:02:53 +00:00
Steve Dower b7ef27bc08
bpo-45582: Ensure PYTHONHOME still overrides detected build prefixes (GH-29948) 2021-12-07 00:07:35 +00:00
Christian Heimes f16f93e527
bpo-45582: framework build: modPath must not be const (GH-29944)
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
2021-12-06 19:13:12 +01:00
neonene af1db4eb55
bpo-45582: Fix getpath_isxfile() and test_embed on Windows (GH-29930) 2021-12-06 17:25:19 +00:00
Christian Heimes 612e59b53f
bpo-45950: Fix macOS framework builds of _bootstrap_python (GH-29936) 2021-12-06 17:13:53 +01:00
Serhiy Storchaka 60c320c38e
bpo-37295: Optimize math.comb() and math.perm() (GH-29090)
For very large numbers use divide-and-conquer algorithm for getting
benefit of Karatsuba multiplication of large numbers.

Do calculations completely in C unsigned long long instead of Python
integers if possible.
2021-12-05 22:26:10 +02:00
Christian Heimes 628abe4463
bpo-45582: Fix signature of _Py_Get_Getpath_CodeObject (GH-29921) 2021-12-05 18:41:46 +01:00
Serhiy Storchaka d15cdb2f32
bpo-27946: Fix possible crash in ElementTree.Element (GH-29915)
Getting an attribute via attrib.get() simultaneously with replacing
the attrib dict can lead to access to deallocated dict.
2021-12-05 14:22:54 +02:00
Christian Heimes 64be8d369b
bpo-45847: Update whatsnew and add place holder entries for missing extensions (GH-29914) 2021-12-04 15:14:48 +01:00
Christian Heimes 4045392e0e
bpo-45847: Port _ctypes partly to PY_STDLIB_MOD (GH-29747)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-12-04 11:21:43 +01:00
neonene 7d7c91a8e8
bpo-45582: Add a NOT operator to the condition in getpath_isxfile (GH-29906) 2021-12-03 22:04:11 +00:00
Irit Katriel 2ff758bd1a
bpo-45711: [asyncio] Normalize exceptions immediately after Fetch, before they are stored as StackItem, which should be normalized (GH-29890) 2021-12-03 19:05:14 +00:00
Christian Heimes ccb73a0d50
bpo-45582: Fix out-of-tree build issues with new getpath (GH-29902) 2021-12-03 14:47:06 +01:00
Steve Dower 99fcf15052
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
2021-12-03 00:08:42 +00:00
Gideon 6266e4af87
bpo-45917: Add math.exp2() method - return 2 raised to the power of x (GH-29829)
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2021-11-29 18:55:43 +00:00
Erlend Egeberg Aasland c4a69a4ad0
bpo-45828: Use unraisable exceptions within sqlite3 callbacks (FH-29591) 2021-11-29 15:22:32 +00:00
Mark Shannon 60929576e4
bpo-45786: Allocate space for frame in frame object. (GH-29729) 2021-11-29 12:34:59 +00:00
Christian Heimes 4dc198dedd
bpo-45847: Fix _multiprocessing on platforms without sem_open (GH-29833) 2021-11-29 10:36:10 +01:00
Erlend Egeberg Aasland aaf42222cf
bpo-45847: Port _multiprocessing to PY_STDLIB_MOD (GH-29768) 2021-11-28 13:00:51 +01:00
Ma Lin 7edb6270a7
bpo-41735: Fix thread lock in zlib.Decompress.flush() may go wrong (GH-29587)
* Fix thread lock in zlib.Decompress.flush() may go wrong

Getting `.unconsumed_tail` before acquiring the thread lock may mix up decompress state.
2021-11-26 16:18:17 -08:00
Christian Heimes 4ebde73b8e
bpo-40280: Move hard-coded feature checks to configure (GH-29789)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-26 19:36:48 +01:00
Christian Heimes 8caceb7a47
bpo-40280: Add configure check for socket shutdown (GH-29795) 2021-11-26 15:16:54 +01:00
Christian Heimes ee1e2c604c
bpo-40280: Use Setup.stdlib static for wasm builds (GH-29784)
``Modules/Setup.stdlib`` contains ``Setup`` lines for all stdlib extension modules for which ``configure`` has detected their dependencies. The file is not used yet and still under development. To use the file, do ``ln -sfr Modules/Setup.stdlib Modules/Setup.local``.
2021-11-26 08:26:49 +01:00
Christian Heimes dc19e86983
bpo-41498: Fix build on platforms without sigset_t (GH-29770) 2021-11-25 12:28:41 +01:00
Pablo Galindo Salgado abfc794bbf
bpo-45822: Minor cleanups to the test_Py_CompileString test (GH-29750) 2021-11-24 18:30:03 +00:00
Christian Heimes 8af6481f6b
bpo-45847: Port _uuid to PY_STDLIB_MOD (GH-29741) 2021-11-24 10:20:37 +01:00
Erlend Egeberg Aasland 324527012f
bpo-45847: Port _posixshmem to PY_STDLIB_MOD (GH-29738) 2021-11-24 08:19:17 +01:00
Christian Heimes 9cf5646bb4
bpo-45847: Port _gdbm to PY_STDLIB_MOD (GH-29720) 2021-11-23 22:58:38 +01:00
Christian Heimes b9e9292d75
bpo-45847: Port _ssl and _hashlib to PY_STDLIB_MOD (GH-29727) 2021-11-23 22:58:13 +01:00
Christian Heimes 095bc8f0d6
bpo-45847: Port _crypt to PY_STDLIB_MOD (GH-29725) 2021-11-23 22:26:50 +01:00
Erlend Egeberg Aasland d82f2caf94
bpo-45847: Port _socket to PY_STDLIB_MOD (GH-29713) 2021-11-23 08:52:05 +01:00
Erlend Egeberg Aasland 2dc7d3dda6
bpo-45847: Port xxlimited and xxlimited_35 to PY_STDLIB_MOD (GH-29707) 2021-11-22 21:27:05 +01:00
Christian Heimes 0e1c2f3ef8
bpo-45847: port _struct to PY_STDLIB_MOD (GH-29706) 2021-11-22 15:58:43 +01:00
Erlend Egeberg Aasland eee683cbde
bpo-45847: Port audioop, _csv, and _posixsubprocess to PY_STDLIB_MOD_SIMPLE (GH-29705)
Automerge-Triggered-By: GH:tiran
2021-11-22 06:37:25 -08:00
Christian Heimes c6dec7e27a
bpo-45847: Port nis module to PY_STDLIB_MOD (GH-29699) 2021-11-22 15:18:41 +01:00
Christian Heimes d9cedabeba
bpo-45847: Port compression libs to PY_STDLIB_MOD (GH-29702) 2021-11-22 14:52:29 +01:00
Erlend Egeberg Aasland b451673f93
bpo-45847: Port mmap, select, and _xxsubinterpreters to Py_STDLIB_MOD (GH-29703) 2021-11-22 14:49:58 +01:00
Erlend Egeberg Aasland 5b946cadaa
bpo-45847: Port fcntl to Py_STDLIB_MOD (GH-29696)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-22 14:02:27 +01:00
Erlend Egeberg Aasland 39f7d2ff01
bpo-45847: Port _lfprof, _opcode, _asyncio, _queue, _statistics, and _typing to PY_STDLIB_MOD_SIMPLE (GH-29690)
Automerge-Triggered-By: GH:tiran
2021-11-22 01:57:50 -08:00
Erlend Egeberg Aasland 718cee08cc
bpo-45847: Port _bisect, _heapq, _json, _pickle, _random, and _zoneinfo to PY_STDLIB_MOD_SIMPLE (GH-29689)
Automerge-Triggered-By: GH:tiran
2021-11-22 00:45:41 -08:00
Erlend Egeberg Aasland 133c65a870
bpo-45847: Port array, _contextvars, math, and cmath to PY_STDLIB_MOD_SIMPLE (GH-29688) 2021-11-22 09:05:36 +01:00
Christian Heimes 2afa1a1266
bpo-45847: Port codecs and unicodedata to PY_STDLIB_MOD (GH-29685) 2021-11-21 14:08:47 +01:00
Christian Heimes f201d261cf
bpo-45847: Port grp, spwd, termios, resource, syslog to PY_STDLIB_MOD (GH-29668) 2021-11-21 10:45:31 +01:00
Christian Heimes f36c69a26e
bpo-45847: Port test modules to PY_STDLIB_MOD (GH-29660) 2021-11-20 16:43:10 +01:00
Christian Heimes 5596909eac
bpo-45847: Port _scproxy to PY_STDLIB_MOD (GH-29644) 2021-11-20 10:18:48 +01:00
Christian Heimes c8c21bdd19
bpo-45847: Port builtin hashlib extensions to PY_STDLIB_MOD (GH-29642) 2021-11-19 20:20:32 +01:00
Christian Heimes 7e44dc0ba7
bpo-45573: Move mandatory core modules to Modules/Setup.bootstrap (GH-29616) 2021-11-19 16:40:57 +01:00
Erlend Egeberg Aasland 29e5874d5a
bpo-45774: Autoconfiscate SQLite detection (GH-29507)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-19 15:10:41 +01:00
Christian Heimes e4bb22fabb
bpo-45573: Add Modules/Setup.stdlib with conditional modules (GH-29615) 2021-11-18 14:40:01 +01:00
Erlend Egeberg Aasland 0920b61a0c
bpo-45512: Use Argument Clinic to set sqlite3 isolation level (GH-29593) 2021-11-18 18:18:09 +09:00
Christian Heimes 25ecc040d0
bpo-45573: Introduce extension module flags in Makefile (GH-29594)
``configure`` now uses a standardized format to forward state, compiler
flags, and linker flags to ``Makefile``, ``setup.py``, and
``Modules/Setup``. ``makesetup`` use the new variables by default if a
module line does not contain any compiler or linker flags. ``setup.py``
has a new function ``addext()``.

For a module ``egg``, configure adds:

* ``MODULE_EGG`` with value yes, missing, disabled, or n/a
* ``MODULE_EGG_CFLAGS``
* ``MODULE_EGG_LDFLAGS``

``Makefile.pre.in`` may also provide ``MODULE_EGG_DEPS`` that lists
dependencies such as header files and static libs.

Signed-off-by: Christian Heimes <christian@python.org>
2021-11-18 09:18:44 +01:00
Dong-hee Na e002bbc6cc
bpo-45512: Simplify manage isolation level (GH-29562) 2021-11-17 21:47:02 +09:00
Pablo Galindo Salgado da20d7401d
bpo-45822: Respect PEP 263's coding cookies in the parser even if flags are not provided (GH-29582) 2021-11-16 12:30:47 -08:00
Erlend Egeberg Aasland 9d6215a54c
bpo-45126: Harden `sqlite3` connection initialisation (GH-28227) 2021-11-16 15:53:35 +01:00
Dong-hee Na 55868f1a33
bpo-45429: Support CREATE_WAITABLE_TIMER_HIGH_RESOLUTION if possible (GH-29203) 2021-11-16 22:41:20 +09:00
Mark Shannon b931077375
bpo-45753: Make recursion checks more efficient. (GH-29524)
* Uses recursion remaining, instead of recursion depth to speed up check against recursion limit.
2021-11-16 11:01:57 +00:00
Erlend Egeberg Aasland 822c3dcce3
bpo-45512: Raise exception if sqlite3.Connection.__init__ is called with bad isolation level (#29561)
* bpo-45512: Raise sqlite3.Connection.__init__ is called with bad isolation level

* Also explicitly test allowed isolation levels

* Use subTest for better error messages if something goes wrong

* Update Lib/test/test_sqlite3/test_dbapi.py

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2021-11-15 21:55:38 +09:00
Erlend Egeberg Aasland b567b9d74b
bpo-45512: Simplify isolation_level handling in `sqlite3` (GH-29053) 2021-11-15 16:50:59 +09:00
Christian Heimes 464e6616be
bpo-45800: Move pyexpat build setup into configure (GH-29547)
Settings for :mod:`pyexpat` C extension are now detected by ``configure``.
The bundled ``expat`` library is built in ``Makefile``.

Signed-off-by: Christian Heimes <christian@python.org>
2021-11-14 10:02:24 +01:00
Christian Heimes 0486570f7b
bpo-45798: Move _decimal build setup into configure (GH-29541)
Settings for :mod:`decimal` internal C extension are now detected by
:program:`configure`. The bundled `libmpdec` library is built in
``Makefile``.

Signed-off-by: Christian Heimes <christian@python.org>
2021-11-13 13:56:16 +01:00
Christian Heimes 0a9f69539b
bpo-45747: Detect gdbm/dbm dependencies in configure (GH-29467)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-10 20:26:55 +01:00
Erlend Egeberg Aasland c1323d4b8c
bpo-45754: Use correct SQLite limit when checking statement length (GH-29489) 2021-11-10 18:46:11 +00:00
Irit Katriel 05fbd60147
bpo-45711: Use _PyErr_ClearExcState instead of setting only exc_value to NULL (GH-29404) 2021-11-10 16:57:14 +00:00
Christian Heimes cbab997efb
bpo-45723: Prepare support for autoconf 2.71 (GH-29441) 2021-11-08 19:31:14 +01:00
Christian Heimes 24af9a40a8
bpo-45743: Move __APPLE_USE_RFC_3542 into socketmodule.c (GH-29456) 2021-11-08 12:48:34 +01:00
180909 122ca4d73f
remove the repeat 'the' (#29455) 2021-11-08 10:19:54 +02:00
Christian Heimes e9594f6747
bpo-45731: Handle --enable-loadable-sqlite-extensions in configure (GH-29434) 2021-11-06 10:30:37 +01:00
Erlend Egeberg Aasland c273986711
bpo-45613: Set `sqlite3.threadsafety` dynamically (GH-29227)
Use the compile-time selected default SQLite threaded mode to set the
DB-API 2.0 attribute 'threadsafety'

Mappings:
  - SQLITE_THREADSAFE=0 => threadsafety=0
  - SQLITE_THREADSAFE=1 => threadsafety=3
  - SQLITE_THREADSAFE=2 => threadsafety=1
2021-11-03 21:01:37 +00:00
Erlend Egeberg Aasland 456e27ac0a
bpo-24139: Add support for SQLite extended result codes (GH-28076) 2021-11-02 23:49:38 +00:00
Erlend Egeberg Aasland 401272e6e6
bpo-42064: Adapt `sqlite3` to multi-phase init (PEP 489) (GH-29234) 2021-11-02 16:35:51 +01:00
Erlend Egeberg Aasland b6b38a8226
bpo-45243: Add support for setting/getting `sqlite3` connection limits (GH-28463) 2021-11-01 22:50:53 +00:00
Erlend Egeberg Aasland 39b4d5938c
bpo-45634: Don't combine error checks when adding `sqlite3` int constants (GH-29251) 2021-10-31 19:01:19 +09:00
Erlend Egeberg Aasland e2e62b3808
bpo-45581: Raise `MemoryError` in `sqlite3.connect` if SQLite signals memory error (GH-29171) 2021-10-29 22:21:58 +02:00
Christian Heimes f0150ac94a
bpo-45548: Some test modules must be built as shared libs (GH-29268)
Some test cases don't work when test modules are static extensions.

Add dependency on Modules/config.c to trigger a rebuild whenever a
module build type is changed.

``makesetup`` puts shared extensions into ``Modules/`` directory. Create
symlinks from pybuilddir so the extensions can be imported.

Note: It is not possible to use the content of pybuilddir.txt as a build
target. Makefile evaluates target variables in the first pass. The
pybuilddir.txt file does not exist at that point.
2021-10-29 17:49:57 +02:00
Tim Golden 7bddd96982
bpo-45621: Small changes to mmap (GH-29247)
* Small tidy-ups / comments
* Use randomized names when testing tagged mmaps to avoid any risk of parallel tests treading on each others' toes
2021-10-29 09:20:21 +01:00
Mark Shannon f291404a80
bpo-45637: Store the frame pointer in the cframe (GH-29267)
* Rename 'frame' to 'current_frame'
2021-10-28 13:59:11 +01:00
Christian Heimes 4c95fb4640
bpo-45548: Fix out-of-tree and Debian builds (GH-29263)
Add Modules subdirs to SRCDIRS to generate directories for out-of-tree
object files.

Debian wants ncurses lib. Works on Fedora, too.

Debian also needs pkg-config to detect correct flags.

Remove more outdated comments. Makefile now tracks header dependencies
-lintl is injected by configure when needed. Build _dbm with
gdbm-compat.

Group some modules by purpose. socket, select, and mmap work on Windows,
too.
2021-10-28 11:57:38 +02:00
Erlend Egeberg Aasland 8f24b7dbcb
bpo-42064: Convert `sqlite3` global state to module state (GH-29073) 2021-10-27 13:12:21 +02:00
Benjamin Peterson 19a6c41e56
Remove unused variables. (GH-29231) 2021-10-26 16:22:34 -07:00
Tim Golden aea5ecc458
bpo-40915: Fix mmap resize bugs on Windows (GH-29213)
(original patch by eryksun)

Correctly hand various failure modes when resizing an mmap on Windows:

* Resizing a pagefile-backed mmap now creates a new mmap and copies data
* Attempting to resize when another mapping is held on the same file raises an OSError
* Attempting to resize a nametagged mmap raises an OSError if another mapping is held with the same nametag
2021-10-26 22:56:43 +01:00
Christian Heimes b5ee79494b
bpo-45548: makesetup improvements (GH-29225)
* record which modules are build as shared extensions
* put object files in same directory as source files
* remove dependency on deleted _math.c

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-26 10:58:43 +02:00
Christian Heimes 2b8677a3cd
bpo-45548: FreeBSD doesn't like auto vars in makesetup (GH-29216) 2021-10-25 20:52:55 +02:00
Christian Heimes ece916e92c
bpo-45548: Add missing extensions to Modules/Setup (GH-29199)
Adds _ctypes, _decimal, _multiprocessing, _posixshmem, _scproxy, _sqlite3,
and _uuid.

Use Makefile variables to build OpenSSL and TCL/TK modules.

Drop ``-lm`` from math lines. Extensions are always linked against libm.

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-25 20:01:52 +02:00
Christian Heimes 81669d1b7b
bpo-45595: Make extensions depend on header files (GH-29198)
``setup.py`` and ``makesetup`` now track build dependencies on all Python
header files and module specific header files.

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-25 20:01:03 +02:00
Christian Heimes fa26245a1c
bpo-45548: Remove _math.c workarounds for pre-C99 libm (GH-29179)
The :mod:`math` and :mod:`cmath` implementation now require a C99 compatible
``libm`` and no longer ship with workarounds for missing acosh, asinh,
expm1, and log1p functions.

The changeset also removes ``_math.c`` and moves the last remaining
workaround into ``_math.h``. This simplifies static builds with
``Modules/Setup`` and resolves symbol conflicts.

Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
Co-authored-by: Brett Cannon <brett@python.org>
Signed-off-by: Christian Heimes <christian@python.org>
2021-10-25 01:25:27 -07:00
Eric Snow 17c61045c5
bpo-45506: Normalize _PyPathConfig.stdlib_dir when calculated. (#29040)
The recently added PyConfig.stdlib_dir was being set with ".." entries. When __file__ was added for from modules this caused a problem on out-of-tree builds. This PR fixes that by normalizing "stdlib_dir" when it is calculated in getpath.c.

https://bugs.python.org/issue45506
2021-10-22 17:20:03 -06:00
Christian Heimes f6e8b80d20
bpo-45571: use PY_CFLAGS_NODIST for shared Modules/Setup (GH-29161) 2021-10-22 20:02:54 +02:00
Christian Heimes ec93721e00
bpo-45570: Simplify setup macros for pyexpat (GH-29159)
* ``HAVE_EXPAT_CONFIG_H`` is not used by our code and not used by
  system-wide expat header files
* ``USE_PYEXPAT_CAPI`` is no longer used by our code
* ``XML_POOR_ENTROPY`` should be defined in expat_config.h

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-22 09:28:23 -07:00
Christian Heimes 03e9f5dc75
bpo-43974: Move Py_BUILD_CORE_MODULE into module code (GH-29157)
setup.py no longer defines Py_BUILD_CORE_MODULE. Instead every
module defines the macro before #include "Python.h" unless
Py_BUILD_CORE_BUILTIN is already defined.

Py_BUILD_CORE_BUILTIN is defined for every module that is built by
Modules/Setup.

The PR also simplifies Modules/Setup. Makefile and makesetup
already define Py_BUILD_CORE_BUILTIN and include Modules/internal
for us.

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-22 15:36:28 +02:00
Brett Cannon 01cf4fb9c1
bpo-45548: Make `Modules/Setup` easier to read (GH-29143) 2021-10-21 17:07:26 -07:00
Petr Viktorin 8a310dd5f4
bpo-45315: PyType_FromSpec: Copy spec->name and have the type own the memory for its name (GH-29103) 2021-10-21 11:46:20 +02:00
Brett Cannon dd86f63b55
bpo-45548: add some missing entries to `Modules/Setup` (GH-29115)
Also remove a duplicate entry for `_weakref`.
2021-10-20 17:34:49 -07:00
Zackery Spytz d8e1819251
bpo-41374: Remove obsolete exclusion of netinet/tcp.h on Cygwin (GH-21649) 2021-10-20 18:14:59 +01:00
Jeong YunWon d2cd5eef0c
bpo-45532: Replace 'default' with 'main' as default in sys.version (GH-29100) 2021-10-21 02:09:03 +09:00
Erlend Egeberg Aasland 09c04e7f0d
bpo-42064: Add module backref to `sqlite3` callback context (GH-28242) 2021-10-19 15:44:45 +02:00
Victor Stinner 52af0756b2
bpo-45434: Only exclude <stdlib.h> in Python 3.11 limited C API (GH-29027)
The Python 3.11 limited C API no longer includes stdlib.h, stdio.h,
string.h and errno.h.

* Exclude Py_MEMCPY() from Python 3.11 limited C API.
* xxlimited C extension is now built with Python 3.11 limited C API.
2021-10-19 12:10:22 +02:00
Victor Stinner 5f09bb021a
bpo-35134: Add Include/cpython/longobject.h (GH-29044)
Move Include/longobject.h non-limited API to a new
Include/cpython/longobject.h header file.

Move the following definitions to the internal C API:

* _PyLong_DigitValue
* _PyLong_FormatAdvancedWriter()
* _PyLong_FormatWriter()
2021-10-19 02:04:52 +02:00
Victor Stinner 547d26aa08
bpo-43760: Add PyThreadState_EnterTracing() (GH-28542)
Add PyThreadState_EnterTracing() and PyThreadState_LeaveTracing()
functions to the limited C API to suspend and resume tracing and
profiling.

Add an unit test on the PyThreadState C API to _testcapi.

Add also internal _PyThreadState_DisableTracing() and
_PyThreadState_ResetTracing().
2021-10-15 16:06:30 +02:00
Victor Stinner 354c35220d
bpo-45482: Rename namespaceobject.h to pycore_namespace.h (GH-28975)
Rename Include/namespaceobject.h to
Include/internal/pycore_namespace.h.

The _testmultiphase extension is now built with the
Py_BUILD_CORE_MODULE macro defined to access _PyNamespace_Type.

object.c: remove unused "pycore_context.h" include.
2021-10-15 15:21:21 +02:00
Victor Stinner 063abd931f
bpo-35081: Move interpreteridobject.h to Include/internal/ (GH-28969)
Move the interpreteridobject.h header file from Include/ to
Include/internal/. It only provides private functions.
2021-10-15 11:56:34 +02:00
Victor Stinner 8e5de40f90
bpo-35134: Move classobject.h to Include/cpython/ (GH-28968)
Move classobject.h, context.h, genobject.h and longintrepr.h header
files from Include/ to Include/cpython/.

Remove redundant "#ifndef Py_LIMITED_API" in context.h.

Remove explicit #include "longintrepr.h" in C files. It's not needed,
Python.h already includes it.
2021-10-15 09:46:29 +02:00
Victor Stinner 0a883a76cd
bpo-35134: Add Include/cpython/floatobject.h (GH-28957)
Split Include/floatobject.h into sub-files: add
Include/cpython/floatobject.h and
Include/internal/pycore_floatobject.h.
2021-10-14 23:41:06 +02:00
Victor Stinner 3cc56c828d
bpo-45439: Move _PyObject_VectorcallTstate() to pycore_call.h (GH-28893)
* Move _PyObject_VectorcallTstate() and _PyObject_FastCallTstate() to
  pycore_call.h (internal C API).
* Convert PyObject_CallOneArg(), PyObject_Vectorcall(),
  _PyObject_FastCall() and PyVectorcall_Function() static inline
  functions to regular functions.
* Add _PyVectorcall_FunctionInline() static inline function.
* PyObject_Vectorcall(), _PyObject_FastCall(), and
  PyObject_CallOneArg() now call _PyThreadState_GET() rather
  than PyThreadState_Get().
2021-10-14 21:53:04 +02:00
Serhiy Storchaka 39aa98346d
bpo-45467: Fix IncrementalDecoder and StreamReader in the "raw-unicode-escape" codec (GH-28944)
They support now splitting escape sequences between input chunks.

Add the third parameter "final" in codecs.raw_unicode_escape_decode().
It is True by default to match the former behavior.
2021-10-14 20:04:19 +03:00
Dong-hee Na d413c50363
no-issue: Make silence about warning '_POSIX_C_SOURCE redefined' (GH-28948) 2021-10-15 00:59:56 +09:00
Serhiy Storchaka c96d1546b1
bpo-45461: Fix IncrementalDecoder and StreamReader in the "unicode-escape" codec (GH-28939)
They support now splitting escape sequences between input chunks.

Add the third parameter "final" in codecs.unicode_escape_decode().
It is True by default to match the former behavior.
2021-10-14 13:17:00 +03:00
Victor Stinner aac29af678
bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
Include <stdlib.h> explicitly in C files.

Python.h includes <wchar.h>.
2021-10-13 19:25:53 +02:00
Victor Stinner bbe7497c5a
bpo-45434: Remove pystrhex.h header file (GH-28923)
Move Include/pystrhex.h to Include/internal/pycore_strhex.h.
The header file only contains private functions.

The following C extensions are now built with Py_BUILD_CORE_MODULE
macro defined to get access to the internal C API:

* _blake2
* _hashopenssl
* _md5
* _sha1
* _sha3
* _ssl
* binascii
2021-10-13 15:22:35 +02:00
Victor Stinner 97308dfcdc
bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h (GH-28922) 2021-10-13 15:03:35 +02:00
Victor Stinner 7cdc2a0f4b
pycore_pystate.h no longer redefines PyThreadState_GET() (GH-28921)
Redefining the PyThreadState_GET() macro in pycore_pystate.h is
useless since it doesn't affect files not including it. Either use
_PyThreadState_GET() directly, or don't use pycore_pystate.h internal
C API. For example, the _testcapi extension don't use the internal C
API, but use the public PyThreadState_Get() function instead.

Replace PyThreadState_Get() with _PyThreadState_GET(). The
_PyThreadState_GET() macro is more efficient than PyThreadState_Get()
and PyThreadState_GET() function calls which call fail with a fatal
Python error.

posixmodule.c and _ctypes extension now include <windows.h> before
pycore header files (like pycore_call.h).

_PyTraceback_Add() now uses _PyErr_Fetch()/_PyErr_Restore() instead
of PyErr_Fetch()/PyErr_Restore().

The _decimal and _xxsubinterpreters extensions are now built with the
Py_BUILD_CORE_MODULE macro defined to get access to the internal C
API.
2021-10-13 14:09:13 +02:00
Victor Stinner c63623a0a6
bpo-45434: bytearrayobject.h no longer includes <stdarg.h> (GH-28913)
bytearrayobject.h and _lzmamodule.c don't use va_list and so don't
need to include <stdarg.h>.
2021-10-13 04:37:55 +02:00
Erlend Egeberg Aasland cfb1df3b71
bpo-44991: Normalise function and collation callback naming (GH-28209) 2021-10-12 13:38:49 +02:00
Victor Stinner d943d19172
bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
  PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
  defined to get access to internal _PyObject_CallNoArgs().
2021-10-12 08:38:19 +02:00
Victor Stinner ce3489cfdb
bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
2021-10-12 00:42:23 +02:00
Dong-hee Na ab62051152
bpo-20028: Empty escapechar/quotechar is not allowed for csv.Dialect (GH-28833) 2021-10-11 20:08:15 +09:00
Serhiy Storchaka b454e8e4df
bpo-27580: Add support of null characters in the csv module. (GH-28808) 2021-10-09 19:17:43 +03:00
Dong-hee Na ec04db74e2
bpo-20028: Keep original exception when PyUnicode_GetLength return -1 (GH-28832) 2021-10-10 00:16:12 +09:00
Dong-hee Na 34bbc87b2d
bpo-20028: Improve error message of csv.Dialect when initializing (GH-28705) 2021-10-09 23:50:12 +09:00
Jeong YunWon 9f7a94fd66
bpo-45407: Remove outdated XXX comment from Struct___init___impl (GH-28805) 2021-10-08 15:45:11 +09:00
Matthias Reichl 392a898353
bpo-45262, asyncio: Fix cache of the running loop holder (GH-28796)
Prevent use-after-free of running loop holder via cache.
2021-10-08 00:46:49 +02:00
Erlend Egeberg Aasland 3f2c433da5
bpo-45041: Restore `sqlite3` executescript behaviour for `SELECT` queries (GH-28509)
* bpo-45041: Restore sqlite3 executescript behaviour for select queries

* Add regression test
2021-10-07 10:16:45 +01:00
Christian Clauss dd02a696e5
Fix typos in the Modules directory (GH-28761) 2021-10-07 01:34:42 -07:00
Ammar Askar db72e58ea5
bpo-29505: Add fuzzer for ast.literal_eval (GH-28777)
This supercedes https://github.com/python/cpython/pull/3437 and fuzzes the method we recommend for unsafe inputs, `ast.literal_eval`. This should exercise the tokenizer and parser.
2021-10-06 16:22:09 -07:00
Mark Shannon a7252f88d3
bpo-40116: Add insertion order bit-vector to dict values to allow dicts to share keys more freely. (GH-28520) 2021-10-06 13:19:53 +01:00
Hai Shi b9bb74871b
bpo-44050: Extension modules can share state when they don't support sub-interpreters. (GH-27794)
Automerge-Triggered-By: GH:encukou
2021-10-05 06:19:32 -07:00
Serhiy Storchaka 60b9e040c9
bpo-45355: Use sizeof(_Py_CODEUNIT) instead of literal 2 for the size of the code unit (GH-28711) 2021-10-03 21:22:42 +03:00
Serhiy Storchaka a5a56154f1
Remove trailing spaces. (GH-28706) 2021-10-03 16:58:14 +03:00
TAGAMI Yukihiro 0742abdc48
bpo-45329: Fix freed memory access in pyexpat.c (GH-28649) 2021-10-02 12:57:13 +03:00
AngstyDuck a450398933
bpo-44687: Ensure BufferedReader objects with unread buffers can peek even when the underlying file is closed (GH-28457) 2021-10-01 21:11:08 +01:00
Dong-hee Na 9eed75fde2
bpo-45332: Fix broken Decimal test and benchmark (GH-28680) 2021-10-01 22:49:46 +09:00
Julien Palard 9ce0f48e91
hashlib: Fix old message about unicode objects. (GH-28653) 2021-10-01 14:38:49 +02:00
Victor Stinner 833fdf126c
bpo-41710: Add private _PyDeadline_Get() function (GH-28674)
Add a private C API for deadlines: add _PyDeadline_Init() and
_PyDeadline_Get() functions.

* Add _PyTime_Add() and _PyTime_Mul() functions which compute t1+t2
  and t1*t2 and clamp the result on overflow.
* _PyTime_MulDiv() now uses _PyTime_Add() and _PyTime_Mul().
2021-10-01 13:29:25 +02:00
Victor Stinner 54957f16a6
bpo-41710: gc_collect_main() uses _PyTime_GetPerfCounter() (GH-28676)
If the DEBUG_STATS debug flag is set, gc_collect_main() now uses
_PyTime_GetPerfCounter() instead of _PyTime_GetMonotonicClock() to
measure the elapsed time.

On Windows, _PyTime_GetMonotonicClock() only has a resolution of 15.6
ms, whereas _PyTime_GetPerfCounter() is closer to a resolution of 100
ns.
2021-10-01 13:29:00 +02:00
Victor Stinner 37b8294d62
bpo-41710: PyThread_acquire_lock_timed() clamps the timout (GH-28643)
PyThread_acquire_lock_timed() now clamps the timeout into the
[_PyTime_MIN; _PyTime_MAX] range (_PyTime_t type) if it is too large,
rather than calling Py_FatalError() which aborts the process.

PyThread_acquire_lock_timed() no longer uses
MICROSECONDS_TO_TIMESPEC() to compute sem_timedwait() argument, but
_PyTime_GetSystemClock() and _PyTime_AsTimespec_truncate().

Fix _thread.TIMEOUT_MAX value on Windows: the maximum timeout is
0x7FFFFFFF milliseconds (around 24.9 days), not 0xFFFFFFFF
milliseconds (around 49.7 days).

Set PY_TIMEOUT_MAX to 0x7FFFFFFF milliseconds, rather than 0xFFFFFFFF
milliseconds.

Fix PY_TIMEOUT_MAX overflow test: replace (us >= PY_TIMEOUT_MAX) with
(us > PY_TIMEOUT_MAX).
2021-09-30 10:16:51 +02:00