Commit Graph

12185 Commits

Author SHA1 Message Date
Dong-hee Na a0ccc404ca
bpo-44113: Update __xxtestfuzz not to use Py_SetProgramName (GH-26083) 2021-05-13 08:22:18 +09:00
Erlend Egeberg Aasland 504ffdae4e
bpo-40645: Fix ref leaks in _hashopenssl (GH-26079) 2021-05-12 19:20:41 +01:00
Erlend Egeberg Aasland e5ba1fe995
bpo-44116: Add GC support to _csv heap types (GH-26074) 2021-05-12 19:18:58 +01:00
Petr Viktorin 3611db53f5
bpo-40645: Fix reference leak in the _hashopenssl extension (GH-26072)
The `PyModule_AddObjectRef` function doesn't steal a reference, so an extra `Py_DECREF` is needed.

Automerge-Triggered-By: GH:tiran
2021-05-12 09:04:34 -07:00
Dong-hee Na 9b06e4b535
Use get_binascii_state instead of PyModule_GetState (GH-26069) 2021-05-13 00:09:30 +09:00
Ken Jin 2b458c1dba
bpo-44089: Allow subclassing of ``csv.Error`` (GH-26008)
* fix subclass error

* Update 2021-05-09-22-52-34.bpo-44089.IoANsN.rst
2021-05-12 06:47:11 -07:00
Inada Naoki 4d4be47705
Do not use Py_ssize_clean_t (GH-25940) 2021-05-08 10:17:37 +09:00
Mark Shannon adcd220556
bpo-40222: "Zero cost" exception handling (GH-25729)
"Zero cost" exception handling.

* Uses a lookup table to determine how to handle exceptions.
* Removes SETUP_FINALLY and POP_TOP block instructions, eliminating (most of) the runtime overhead of try statements.
* Reduces the size of the frame object by about 60%.
2021-05-07 15:19:19 +01:00
Erlend Egeberg Aasland 0988ed09b4
bpo-44047: Remove unused argument to _pysqlite_seterror (GH-25915) 2021-05-07 16:15:01 +03:00
Inada Naoki 9ad8f109ac
bpo-44029: Remove Py_UNICODE APIs (GH-25881)
Remove deprecated `Py_UNICODE` APIs: `PyUnicode_Encode`,
`PyUnicode_EncodeUTF7`, `PyUnicode_EncodeUTF8`,
`PyUnicode_EncodeUTF16`, `PyUnicode_EncodeUTF32`,
`PyUnicode_EncodeLatin1`, `PyUnicode_EncodeMBCS`,
`PyUnicode_EncodeDecimal`, `PyUnicode_EncodeRawUnicodeEscape`,
`PyUnicode_EncodeCharmap`, `PyUnicode_EncodeUnicodeEscape`,
`PyUnicode_TransformDecimalToASCII`, `PyUnicode_TranslateCharmap`,
`PyUnicodeEncodeError_Create`, `PyUnicodeTranslateError_Create`.

See :pep:`393` and :pep:`624` for reference.
2021-05-07 15:58:29 +09:00
Inada Naoki 4ebf4a6bfa
bpo-40943: Fix skipitem() didn't raise SystemError (GH-25937)
`convertitem()` raises `SystemError` when '#' is used without `PY_SSIZE_T_CLEAN`.
This commit makes `skipitem()` raise it too.
2021-05-07 11:56:48 +09:00
Gregory P. Smith da5c808fb5
bpo-36515: Disable unaligned memory access in _sha3 on ARM (GH-25927)
Contributed-By: Matthias Klose

Automerge-Triggered-By: GH:tiran
2021-05-05 13:55:33 -07:00
Brad Larsen 23362f8c30
Eliminate duplicated assignment in _randommodule.c (GH-25904) 2021-05-04 20:41:11 -07:00
Raymond Hettinger 6fdc4d37f3
bpo-40521: Convert deque freelist from global vars to instance vars (GH-25906) 2021-05-04 17:08:31 -07:00
Ken Jin 2f5baa1750
Add C-API tests (#25886) 2021-05-04 13:07:31 +02:00
Dennis Sweeney 80a2a4ed7d
bpo-38530: Refactor and improve AttributeError suggestions (GH-25776)
- Make case-swaps half the cost of any other edit
- Refactor Levenshtein code to not use memory allocator, and to bail early on no match.
- Add comments to Levenshtein distance code
- Add test cases for Levenshtein distance behind a debug macro
- Set threshold to `(name_size + item_size + 3) * MOVE_COST / 6`.
  - Reasoning: similar to `difflib.SequenceMatcher.ratio()` >= 2/3:
```
"Multiset Jaccard similarity" >= 2/3
matching letters / total letters >= 2/3
(name_size - distance + item_size - distance) / (name_size + item_size) >= 2/3
1 - (2*distance) / (name_size + item_size) >= 2/3
1/3 >= (2*distance) / (name_size + item_size)
(name_size + item_size) / 6 >= distance
With rounding:
(name_size + item_size + 3) // 6 >= distance
```

Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-05-03 16:47:27 +01:00
Pablo Galindo c2931d31f8
bpo-43916: Move the _PyStructSequence_InitType function to the internal API (GH-25854) 2021-05-03 15:50:24 +01:00
Batuhan Taskaya ad106c68eb
bpo-42725: Render annotations effectless on symbol table with PEP 563 (GH-25583) 2021-05-03 10:43:00 +03:00
Mark Shannon 33ec88ac81
bpo-43977: Make sure that tp_flags for pattern matching are inherited correctly. (GH-25813) 2021-05-03 00:38:22 +01:00
Erlend Egeberg Aasland cbb7b9ed4a
bpo-43434: Clean up sqlite3.connect() after GH-25818 (GH-25823) 2021-05-02 23:10:51 +01:00
Erlend Egeberg Aasland c96cc089f6
bpo-43434: Move sqlite3.connect audit events to sqlite3.Connection.__init__ (GH-25818) 2021-05-02 22:25:17 +01:00
Zackery Spytz 73766b0341
bpo-32745: Fix a regression in the handling of ctypes' c_wchar_p type (#8721)
Embedded nulls would cause a ValueError to be raised. Thanks go to
Eryk Sun for their analysis.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-05-02 12:40:01 +02:00
Christian Heimes 91554e4c5c
bpo-43908: Mark ssl, hash, and hmac types as immutable (GH-25792)
Signed-off-by: Christian Heimes <christian@python.org>
2021-05-02 09:47:45 +02:00
Christian Heimes e983252b51
bpo-43998: Default to TLS 1.2 and increase cipher suite security (GH-25778)
The ssl module now has more secure default settings. Ciphers without forward
secrecy or SHA-1 MAC are disabled by default. Security level 2 prohibits
weak RSA, DH, and ECC keys with less than 112 bits of security.
:class:`~ssl.SSLContext` defaults to minimum protocol version TLS 1.2.
Settings are based on Hynek Schlawack's research.

```
$ openssl version
OpenSSL 1.1.1k  FIPS 25 Mar 2021
$ openssl ciphers -v '@SECLEVEL=2:ECDH+AESGCM:ECDH+CHACHA20:ECDH+AES:DHE+AES:!aNULL:!eNULL:!aDSS:!SHA1:!AESCCM'
TLS_AES_256_GCM_SHA384  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 Kx=any      Au=any  Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(128) Mac=AEAD
TLS_AES_128_CCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESCCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
```

Signed-off-by: Christian Heimes <christian@python.org>
2021-05-01 20:53:10 +02:00
Ma Lin 251ffa9d2b
bpo-41486: Fix initial buffer size can't > UINT32_MAX in zlib module (GH-25738)
* Fix initial buffer size can't > UINT32_MAX in zlib module

After commit f9bedb630e, in 64-bit build,
if the initial buffer size > UINT32_MAX, ValueError will be raised.

These two functions are affected:
1. zlib.decompress(data, /, wbits=MAX_WBITS, bufsize=DEF_BUF_SIZE)
2. zlib.Decompress.flush([length])

This commit re-allows the size > UINT32_MAX.

* adds curly braces per PEP 7.

* Renames `Buffer_*` to `OutputBuffer_*` for clarity
2021-04-30 16:32:49 -07:00
Victor Stinner 665c7746fc
bpo-43916: _md5.md5 uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25753)
The following types use Py_TPFLAGS_DISALLOW_INSTANTIATION flag:

* _md5.md5
* _sha1.sha1
* _sha256.sha224
* _sha256.sha256
* _sha512.sha384
* _sha512.sha512
2021-04-30 18:40:30 +02:00
Victor Stinner 7dcf0f6db3
bpo-43916: select.devpoll uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25751) 2021-04-30 18:19:57 +02:00
Erlend Egeberg Aasland 9746cda705
bpo-43916: Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to selected types (GH-25748)
Apply Py_TPFLAGS_DISALLOW_INSTANTIATION to the following types:

* _dbm.dbm
* _gdbm.gdbm
* _multibytecodec.MultibyteCodec
* _sre..SRE_Scanner
* _thread._localdummy
* _thread.lock
* _winapi.Overlapped
* array.arrayiterator
* functools.KeyWrapper
* functools._lru_list_elem
* pyexpat.xmlparser
* re.Match
* re.Pattern
* unicodedata.UCD
* zlib.Compress
* zlib.Decompress
2021-04-30 16:04:57 +02:00
Erlend Egeberg Aasland 387397f8a4
bpo-43916: select.poll uses Py_TPFLAGS_DISALLOW_INSTANTIATION (GH-25750) 2021-04-30 15:49:17 +02:00
Victor Stinner 0cad068ec1
bpo-43916: Remove _disabled_new() function (GH-25745)
posix and _hashlib use the new Py_TPFLAGS_DISALLOW_INSTANTIATION
flag on their heap types, rather than using a custom tp_new function
(_disabled_new).
2021-04-30 14:06:49 +02:00
Victor Stinner 3bb09947ec
bpo-43916: Add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag (GH-25721)
Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow
creating type instances: set tp_new to NULL and don't create the
"__new__" key in the type dictionary.

The flag is set automatically on static types if tp_base is NULL or
&PyBaseObject_Type and tp_new is NULL.

Use the flag on the following types:

* _curses.ncurses_version type
* _curses_panel.panel
* _tkinter.Tcl_Obj
* _tkinter.tkapp
* _tkinter.tktimertoken
* _xxsubinterpretersmodule.ChannelID
* sys.flags type
* sys.getwindowsversion() type
* sys.version_info type

Update MyStr example in the C API documentation to use
Py_TPFLAGS_DISALLOW_INSTANTIATION.

Add _PyStructSequence_InitType() function to create a structseq type
with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set.

type_new() calls _PyType_CheckConsistency() at exit.
2021-04-30 12:46:15 +02:00
Mark Shannon 069e81ab3d
bpo-43977: Use tp_flags for collection matching (GH-25723)
* Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes.

* Set relevant flags on collections.abc.Sequence and Mapping.

* Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes.

* Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING.

* Add NEWS

* Remove interpreter-state map_abc and seq_abc fields.
2021-04-30 09:50:28 +01:00
Erlend Egeberg Aasland c6ad03fddf
bpo-43908: Make array.array type immutable (GH-25696)
Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-29 08:47:48 +02:00
Erlend Egeberg Aasland 5daf70b22e
bpo-43908: Make re types immutable (GH-25697)
Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-29 08:47:11 +02:00
Victor Stinner 103d5e420d
bpo-28254: _posixsubprocess uses PyGC_Enable/PyGC_Disable (GH-25693) 2021-04-28 19:09:29 +02:00
scoder 3cc481b9de
bpo-28254: Add a C-API for controlling the GC state (GH-25687)
Add new C-API functions to control the state of the garbage collector:
PyGC_Enable(), PyGC_Disable(), PyGC_IsEnabled(),
corresponding to the functions in the gc module.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2021-04-28 18:12:16 +02:00
Ma Lin f9bedb630e
bpo-41486: Faster bz2/lzma/zlib via new output buffering (GH-21740)
Faster bz2/lzma/zlib via new output buffering.
Also adds .readall() function to _compression.DecompressReader class
to take best advantage of this in the consume-all-output at once scenario.

Often a 5-20% speedup in common scenarios due to less data copying.

Contributed by Ma Lin.
2021-04-27 23:58:54 -07:00
Victor Stinner a5e64444e6
bpo-43963: Add _signal module state (GH-25676)
* Add signal_state_t structure and signal_global_state variable.
* Add a module state to the _signal module.
* Move and rename variables:

  * DefaultHandler becomes state->default_handler
  * IgnoreHandler becomes state->ignore_handler
  * sigint_event becomes state->sigint_event
  * ItimerError becomes modstate->itimer_error

* Rename SetHandler() to set_handler() to be consistent with
  get_handler().
2021-04-28 03:02:55 +02:00
Victor Stinner a09766deab
bpo-43963: Fix import _signal in subinterpreters (GH-25674)
Importing the _signal module in a subinterpreter has no longer side
effects.

signal_module_exec() no longer modifies Handlers and no longer attempts
to set SIGINT signal handler in subinterpreters.
2021-04-28 01:50:04 +02:00
Ma Lin 93f411838a
Fix thread locks in zlib module may go wrong in rare case. (#22126)
Setting `next_in` before acquiring the thread lock may mix up compress/decompress state in other threads.
2021-04-27 10:37:11 +02:00
Erlend Egeberg Aasland 7244c0060d
bpo-43762: Add audit events for loading of sqlite3 extensions (GH-25246) 2021-04-27 00:16:46 +01:00
Christian Heimes 666991fc59
bpo-18233: Add internal methods to access peer chain (GH-25467)
The internal `_ssl._SSLSocket` object now provides methods to retrieve
the peer cert chain and verified cert chain as a list of Certificate
objects. Certificate objects have methods to convert the cert to a dict,
PEM, or DER (ASN.1).

These are private APIs for now. There is a slim chance to stabilize the
approach and provide a public API for 3.10. Otherwise I'll provide a
stable API in 3.11.

Signed-off-by: Christian Heimes <christian@python.org>
2021-04-26 15:01:40 +02:00
Serhiy Storchaka 172c0f2752
bpo-39529: Deprecate creating new event loop in asyncio.get_event_loop() (GH-23554)
asyncio.get_event_loop() emits now a deprecation warning when it creates a new event loop.
In future releases it will became an alias of asyncio.get_running_loop().
2021-04-25 13:40:44 +03:00
Segev Finer 5e437fb872
bpo-30555: Fix WindowsConsoleIO fails in the presence of fd redirection (GH-1927)
This works by not caching the handle and instead getting the handle from
the file descriptor each time, so that if the actual handle changes by
fd redirection closing/opening the console handle beneath our feet, we
will keep working correctly.
2021-04-23 23:00:27 +01:00
Steve Dower 019e9e8168
bpo-43538: Add extra arguments to os.startfile (GH-25538) 2021-04-23 18:03:17 +01:00
Zackery Spytz 7d37b86ad4
bpo-35114: Make ssl.RAND_status() return a bool (GH-20063) 2021-04-23 18:07:37 +02:00
Christian Heimes b9ad88be03
bpo-43920: Make load_verify_locations(cadata) error message consistent (GH-25554)
Signed-off-by: Christian Heimes <christian@python.org>
2021-04-23 13:51:40 +02:00
Erlend Egeberg Aasland e9194ea6ea
bpo-43852: Improve tuple creation in sqlite3 (GH-25421) 2021-04-23 14:21:08 +03:00
Steve Dower e07d809889
bpo-38222: Check specifically for a drive, not just a colon (GH-25540) 2021-04-23 00:30:37 +01:00
Steve Dower fe63a401a9
bpo-38822: Fixed os.stat failing on inaccessible directories. (GH-25527)
It would just fail if the path was inaccessible and had a trailing slash. It should fall back to the parent directory's metadata.
2021-04-22 20:45:02 +01:00