Commit Graph

12149 Commits

Author SHA1 Message Date
Ma Lin f9bedb630e
bpo-41486: Faster bz2/lzma/zlib via new output buffering (GH-21740)
Faster bz2/lzma/zlib via new output buffering.
Also adds .readall() function to _compression.DecompressReader class
to take best advantage of this in the consume-all-output at once scenario.

Often a 5-20% speedup in common scenarios due to less data copying.

Contributed by Ma Lin.
2021-04-27 23:58:54 -07:00
Victor Stinner a5e64444e6
bpo-43963: Add _signal module state (GH-25676)
* Add signal_state_t structure and signal_global_state variable.
* Add a module state to the _signal module.
* Move and rename variables:

  * DefaultHandler becomes state->default_handler
  * IgnoreHandler becomes state->ignore_handler
  * sigint_event becomes state->sigint_event
  * ItimerError becomes modstate->itimer_error

* Rename SetHandler() to set_handler() to be consistent with
  get_handler().
2021-04-28 03:02:55 +02:00
Victor Stinner a09766deab
bpo-43963: Fix import _signal in subinterpreters (GH-25674)
Importing the _signal module in a subinterpreter has no longer side
effects.

signal_module_exec() no longer modifies Handlers and no longer attempts
to set SIGINT signal handler in subinterpreters.
2021-04-28 01:50:04 +02:00
Ma Lin 93f411838a
Fix thread locks in zlib module may go wrong in rare case. (#22126)
Setting `next_in` before acquiring the thread lock may mix up compress/decompress state in other threads.
2021-04-27 10:37:11 +02:00
Erlend Egeberg Aasland 7244c0060d
bpo-43762: Add audit events for loading of sqlite3 extensions (GH-25246) 2021-04-27 00:16:46 +01:00
Christian Heimes 666991fc59
bpo-18233: Add internal methods to access peer chain (GH-25467)
The internal `_ssl._SSLSocket` object now provides methods to retrieve
the peer cert chain and verified cert chain as a list of Certificate
objects. Certificate objects have methods to convert the cert to a dict,
PEM, or DER (ASN.1).

These are private APIs for now. There is a slim chance to stabilize the
approach and provide a public API for 3.10. Otherwise I'll provide a
stable API in 3.11.

Signed-off-by: Christian Heimes <christian@python.org>
2021-04-26 15:01:40 +02:00
Serhiy Storchaka 172c0f2752
bpo-39529: Deprecate creating new event loop in asyncio.get_event_loop() (GH-23554)
asyncio.get_event_loop() emits now a deprecation warning when it creates a new event loop.
In future releases it will became an alias of asyncio.get_running_loop().
2021-04-25 13:40:44 +03:00
Segev Finer 5e437fb872
bpo-30555: Fix WindowsConsoleIO fails in the presence of fd redirection (GH-1927)
This works by not caching the handle and instead getting the handle from
the file descriptor each time, so that if the actual handle changes by
fd redirection closing/opening the console handle beneath our feet, we
will keep working correctly.
2021-04-23 23:00:27 +01:00
Steve Dower 019e9e8168
bpo-43538: Add extra arguments to os.startfile (GH-25538) 2021-04-23 18:03:17 +01:00
Zackery Spytz 7d37b86ad4
bpo-35114: Make ssl.RAND_status() return a bool (GH-20063) 2021-04-23 18:07:37 +02:00
Christian Heimes b9ad88be03
bpo-43920: Make load_verify_locations(cadata) error message consistent (GH-25554)
Signed-off-by: Christian Heimes <christian@python.org>
2021-04-23 13:51:40 +02:00
Erlend Egeberg Aasland e9194ea6ea
bpo-43852: Improve tuple creation in sqlite3 (GH-25421) 2021-04-23 14:21:08 +03:00
Steve Dower e07d809889
bpo-38222: Check specifically for a drive, not just a colon (GH-25540) 2021-04-23 00:30:37 +01:00
Steve Dower fe63a401a9
bpo-38822: Fixed os.stat failing on inaccessible directories. (GH-25527)
It would just fail if the path was inaccessible and had a trailing slash. It should fall back to the parent directory's metadata.
2021-04-22 20:45:02 +01:00
Raymond Hettinger a07da09ad5
bpo-43475: Fix worst case collision behavior for NaN instances (GH-25493) 2021-04-22 08:34:57 -07:00
Steve Dower dc516ef839
bpo-26227: Fixes decoding of host names on Windows from ANSI instead of UTF-8 (GH-25510) 2021-04-22 00:18:20 +01:00
Victor Stinner cdad2724e6
bpo-40137: Add pycore_moduleobject.h internal header (GH-25507)
Add pycore_moduleobject.h internal header file with static inline
functions to access module members:

* _PyModule_GetDict()
* _PyModule_GetDef()
* _PyModule_GetState()

These functions don't check at runtime if their argument has a valid
type and can be inlined even if Python is not built with LTO.

_PyType_GetModuleByDef() uses _PyModule_GetDef().

Replace PyModule_GetState() with _PyModule_GetState() in the
extension modules, considered as performance sensitive:

* _abc
* _functools
* _operator
* _pickle
* _queue
* _random
* _sre
* _struct
* _thread
* _winapi
* array
* posix

The following extensions are now built with the Py_BUILD_CORE_MODULE
macro defined, to be able to use the internal pycore_moduleobject.h
header: _abc, array, _operator, _queue, _sre, _struct.
2021-04-22 00:52:52 +02:00
Steve Dower 7b86e47617
bpo-43472: Ensure PyInterpreterState_New audit events are raised when called through _xxsubinterpreters module (GH-25506) 2021-04-21 23:34:29 +01:00
Raymond Hettinger 139c232f38
bpo-40137: Move state lookups out of the critical path (GH-25492) 2021-04-21 15:22:22 -07:00
Christian Heimes 3309113d61
bpo-43799: Also define SSLv3_method() (GH-25481)
Signed-off-by: Christian Heimes <christian@python.org>
2021-04-20 18:10:10 +02:00
l0x 64d975202f
bpo-40849: Expose X509_V_FLAG_PARTIAL_CHAIN ssl flag (GH-20463)
This short PR exposes an openssl flag that  wasn't exposed. I've also updated to doc to reflect the change. It's heavily inspired by 990fcaac3c.
2021-04-19 04:51:18 -07:00
Christian Heimes 2875c603b2
bpo-43880: Show DeprecationWarnings for deprecated ssl module features (GH-25455)
* ssl.OP_NO_SSLv2
* ssl.OP_NO_SSLv3
* ssl.OP_NO_TLSv1
* ssl.OP_NO_TLSv1_1
* ssl.OP_NO_TLSv1_2
* ssl.OP_NO_TLSv1_3
* ssl.PROTOCOL_SSLv2
* ssl.PROTOCOL_SSLv3
* ssl.PROTOCOL_SSLv23 (alias for PROTOCOL_TLS)
* ssl.PROTOCOL_TLS
* ssl.PROTOCOL_TLSv1
* ssl.PROTOCOL_TLSv1_1
* ssl.PROTOCOL_TLSv1_2
* ssl.TLSVersion.SSLv3
* ssl.TLSVersion.TLSv1
* ssl.TLSVersion.TLSv1_1
* ssl.wrap_socket()
* ssl.RAND_pseudo_bytes()
* ssl.RAND_egd() (already removed since it's not supported by OpenSSL 1.1.1)
* ssl.SSLContext() without a protocol argument
* ssl.match_hostname()
* hashlib.pbkdf2_hmac() (pure Python implementation, fast OpenSSL
  function will stay)

Signed-off-by: Christian Heimes <christian@python.org>
2021-04-19 07:27:10 +02:00
Christian Heimes 89d1550d14
bpo-42854: Use SSL_read/write_ex() (GH-25468)
The ssl module now uses ``SSL_read_ex`` and ``SSL_write_ex``
internally. The functions support reading and writing of data larger
than 2 GB. Writing zero-length data no longer fails with a protocol
violation error.

Signed-off-by: Christian Heimes <christian@python.org>
2021-04-19 06:55:30 +02:00
Christian Heimes aa6da32edc
bpo-43362: Fix invalid free and return check in _sha3 module (GH-25463)
Commit 93d50a6a8d / GH-21855 changed the
order of variable definitions, which introduced a potential invalid free
bug. Py_buffer object is now initialized earlier and the result of
Keccak initialize is verified.

Co-authored-by: Alex Henrie <alexhenrie24@gmail.com>
Signed-off-by: Christian Heimes <christian@python.org>

Co-authored-by: Alex Henrie <alexhenrie24@gmail.com>
2021-04-18 08:39:39 +02:00
Christian Heimes 7f1305ef9e
bpo-42333: Port _ssl extension to multiphase initialization (PEP 489) (GH-23253)
- Introduce sslmodule_slots
- Introduce sslmodulestate
- Use sslmodulestate
- Get rid of PyState_FindModule
- Move new structs and helpers to header file
- Use macros to access state
- Keep a strong ref to socket type
2021-04-17 20:06:38 +02:00
Christian Heimes b8d0fa035d
bpo-43669: Remove OpenSSL 0.9 to 1.1.0 specific documentation (GH-25453) 2021-04-17 15:49:50 +02:00
Christian Heimes 39258d3595
bpo-43669: PEP 644: Require OpenSSL 1.1.1 or newer (GH-23014)
- Remove HAVE_X509_VERIFY_PARAM_SET1_HOST check
- Update hashopenssl to require OpenSSL 1.1.1
- multissltests only OpenSSL > 1.1.0
- ALPN is always supported
- SNI is always supported
- Remove deprecated NPN code. Python wrappers are no-op.
- ECDH is always supported
- Remove OPENSSL_VERSION_1_1 macro
- Remove locking callbacks
- Drop PY_OPENSSL_1_1_API macro
- Drop HAVE_SSL_CTX_CLEAR_OPTIONS macro
- SSL_CTRL_GET_MAX_PROTO_VERSION is always defined now
- security level is always available now
- get_num_tickets is available with TLS 1.3
- X509_V_ERR MISMATCH is always available now
- Always set SSL_MODE_RELEASE_BUFFERS
- X509_V_FLAG_TRUSTED_FIRST is always available
- get_ciphers is always supported
- SSL_CTX_set_keylog_callback is always available
- Update Modules/Setup with static link example
- Mention PEP in whatsnew
- Drop 1.0.2 and 1.1.0 from GHA tests
2021-04-17 11:36:35 +02:00
Christian Heimes b467d9a240
bpo-43522: Fix SSLContext.hostname_checks_common_name (GH-24899)
Fix problem with ssl.SSLContext.hostname_checks_common_name. OpenSSL does not
copy hostflags from *struct SSL_CTX* to *struct SSL*.

Signed-off-by: Christian Heimes <christian@python.org>
2021-04-17 10:07:19 +02:00
Erlend Egeberg Aasland 5cb601f956
bpo-43296: Handle sqlite3_value_blob() errors (GH-24674) 2021-04-15 00:09:11 +03:00
Erlend Egeberg Aasland def919342f
bpo-43505: Explicitly initialize and shutdown sqlite3 (GH-25404) 2021-04-14 17:50:16 +03:00
Erlend Egeberg Aasland c1ae741997
bpo-43265: Improve sqlite3.Connection.backup error handling (GH-24586) 2021-04-14 14:45:49 +03:00
Erlend Egeberg Aasland 6f1e8ccffa
bpo-43752: Fix sqlite3 regression for zero-sized blobs with converters (GH-25228) 2021-04-14 14:18:49 +03:00
Christian Heimes a4833883c9
bpo-43799: OpenSSL 3.0.0: declare OPENSSL_API_COMPAT 1.1.1 (GH-25329)
Signed-off-by: Christian Heimes <christian@python.org>
2021-04-13 08:17:26 +02:00
Victor Stinner 09bbebea16
bpo-43753: Add Py_Is() and Py_IsNone() functions (GH-25227)
Add the Py_Is(x, y) function to test if the 'x' object is the 'y'
object, the same as "x is y" in Python. Add also the Py_IsNone(),
Py_IsTrue(), Py_IsFalse() functions to test if an object is,
respectively, the None singleton, the True singleton or the False
singleton.
2021-04-11 00:17:39 +02:00
Christian Heimes 6f37ebc61e
bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default (GH-25309)
Signed-off-by: Christian Heimes <christian@python.org>
2021-04-09 17:59:21 +02:00
Christian Heimes 150af75432
bpo-43788: Generate version specific _ssl_data.h (GH-25300)
Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: GH:tiran
2021-04-09 08:02:00 -07:00
Christian Heimes d3b73f32ef
bpo-43789: OpenSSL 3.0.0 Don't call passwd callback again in error case (GH-25303) 2021-04-09 15:23:38 +02:00
Steve Dower 04732ca993
bpo-43105: Importlib now resolves relative paths when creating module spec objects from file locations (GH-25121) 2021-04-07 01:02:07 +01:00
Serhiy Storchaka b1dc1aacf8
bpo-43084: Return bool instead of int from curses.window.enclose() (GH-24398) 2021-04-05 16:50:24 +03:00
Zackery Spytz afd1265058
bpo-31956: Add start and stop parameters to array.index() (GH-25059)
Co-Authored-By: Anders Lorentsen <Phaqui@gmail.com>
2021-04-03 00:28:35 +09:00
Victor Stinner 240bcf82a1
bpo-41111: xxlimited.c defines Py_LIMITED_API (GH-25151)
xxlimited.c and xxlimited_35.c now define the Py_LIMITED_API macro,
rather than having to do it in the build recipe.

Co-authored-by: Hai Shi <shihai1992@gmail.com>
2021-04-02 16:48:11 +02:00
Inada Naoki bec8c787ec
bpo-43510: Fix emitting EncodingWarning from _io module. (GH-25146)
I forget to check PyErr_WarnEx() return value. But it will fail when -Werror is used.
2021-04-02 17:38:59 +09:00
Inada Naoki cfa176685a
Revert "bpo-43510: PEP 597: Accept `encoding="locale"` in binary mode (GH-25103)" (#25108)
This reverts commit ff3c9739bd.
2021-03-31 18:49:41 +09:00
Inada Naoki ff3c9739bd
bpo-43510: PEP 597: Accept `encoding="locale"` in binary mode (GH-25103)
It make `encoding="locale"` usable everywhere `encoding=None` is
allowed.
2021-03-31 14:26:08 +09:00
Antoine Pitrou 73b20ae2fb
bpo-41369: Finish updating the vendored libmpdec to version 2.5.1 (GH-24962)
Complete the update to libmpdec-2.5.1.

Co-authored-by: Stefan Krah <skrah@bytereef.org>
2021-03-30 18:11:06 +02:00
Inada Naoki 068ebf9729
bpo-33164: blake2: Fix Coverity scan (GH-25060) 2021-03-30 12:25:28 +09:00
Pablo Galindo 70cdf1812c
bpo-40645: Fix reference leak in the _hashopenssl extension (GH-25063) 2021-03-29 06:17:40 -07:00
Inada Naoki 4827483f47
bpo-43510: Implement PEP 597 opt-in EncodingWarning. (GH-19481)
See [PEP 597](https://www.python.org/dev/peps/pep-0597/).

* Add `-X warn_default_encoding` and `PYTHONWARNDEFAULTENCODING`.
* Add EncodingWarning
* Add io.text_encoding()
* open(), TextIOWrapper() emits EncodingWarning when encoding is omitted and warn_default_encoding is enabled.
* _pyio.TextIOWrapper() uses UTF-8 as fallback default encoding used when failed to import locale module. (used during building Python)
* bz2, configparser, gzip, lzma, pathlib, tempfile modules use io.text_encoding().
* What's new entry
2021-03-29 12:28:14 +09:00
Christian Heimes 933dfd7504
bpo-40645: use C implementation of HMAC (GH-24920)
- [x] fix tests
- [ ] add test scenarios for old/new code.

Signed-off-by: Christian Heimes <christian@python.org>
2021-03-27 06:55:03 -07:00
Raymond Hettinger 3bb19873ab
Revert "bpo-40521: Remove freelist from collections.deque() (GH-21073)" (GH-24944)
This reverts commit 32f2eda859.
It can be re-applied if the subinterpreter PEP is approved.
Otherwise, the commit degraded performance with no offsetting
benefit.
2021-03-25 13:32:23 -07:00