Commit Graph

138 Commits

Author SHA1 Message Date
Tomas R a6465605c1
gh-111916: Make hashlib related modules thread-safe without the GIL (#111981)
Always use an individual lock on hash objects when in free-threaded builds.

Fixes #111916
2023-11-15 23:53:38 +00:00
Serhiy Storchaka 329e4a1a3f
gh-86493: Modernize modules initialization code (GH-106858)
Use PyModule_Add() or PyModule_AddObjectRef() instead of soft deprecated
PyModule_AddObject().
2023-07-25 14:34:49 +03:00
Victor Stinner 89f9875448
gh-106320: Move private _PyHash API to the internal C API (#107026)
* No longer export most private _PyHash symbols, only export the ones
  which are needed by shared extensions.
* Modules/_xxtestfuzz/fuzzer.c now uses the internal C API.
2023-07-22 13:49:37 +00:00
Serhiy Storchaka a293fa5915
gh-86493: Use PyModule_Add() instead of PyModule_AddObjectRef() (GH-106860) 2023-07-18 23:59:53 +03:00
Inada Naoki d5bd32fb48
gh-104922: remove PY_SSIZE_T_CLEAN (#106315) 2023-07-02 15:07:46 +09:00
Serhiy Storchaka 1d33d53780
gh-106033: Get rid of new occurrences of PyDict_GetItem and PyObject_HasAttr (GH-106034)
These functions are broken by design because they discard any exceptions raised
inside, including MemoryError and KeyboardInterrupt.  They should not be
used in new code.
2023-06-23 20:10:32 +03:00
Victor Stinner ef300937c2
gh-92536: Remove PyUnicode_READY() calls (#105210)
Since Python 3.12, PyUnicode_READY() does nothing and always
returns 0.
2023-06-02 01:33:17 +02:00
Gregory P. Smith 2e5d8a90aa
gh-99108: Release the GIL around hashlib built-in computation (#104675)
This matches the GIL releasing behavior of our existing `_hashopenssl`
module, extending it to the HACL* built-ins.

Includes adding comments to better describe the ENTER/LEAVE macros
purpose and explain the lock strategy in both existing and new code.
2023-05-23 00:06:41 +00:00
Eric Snow a9c6e0618f
gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)
Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules.  We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
2023-05-05 21:11:27 +00:00
Dong-hee Na efb0a2cf3a
gh-103256: Fix hmac algorithm to support fallback implementation (gh-103286)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-04-07 10:51:29 +09:00
chgnrdv 85ba8a3e03
gh-102650: Remove duplicate include directives from multiple source files (#102651)
Remove duplicate include directives from multiple source files
2023-03-13 13:25:17 -06:00
Thomas A Caswell 0709586744
DOC: correct bytesarray -> bytearray in comments (GH-92410) 2022-06-21 22:04:03 +02:00
Kumar Aditya cb04a09d2d
GH-93207: Remove HAVE_STDARG_PROTOTYPES configure check for stdarg.h (#93215) 2022-05-27 13:30:45 +02:00
Christian Heimes 48e2010d92
bpo-47101: list only activated algorithms in hashlib.algorithms_available (GH-32076) 2022-03-23 13:30:05 -07:00
Zackery Spytz 59e004af63
bpo-40479: Fix undefined behavior in Modules/_hashopenssl.c (GH-31153)
va_end() must be called before returning.
2022-02-06 23:40:54 -08:00
Christian Heimes 276c234ce0
bpo-40479: Fix typo, flag must be set for OpenSSL < 3.0.0 (GH-30584) 2022-01-13 21:47:42 +01:00
Christian Heimes 443b308fee
bpo-40479: Fix hashlib's usedforsecurity for OpenSSL 3.0.0 (GH-30455) 2022-01-13 09:46:38 +01:00
Christian Heimes 03e9f5dc75
bpo-43974: Move Py_BUILD_CORE_MODULE into module code (GH-29157)
setup.py no longer defines Py_BUILD_CORE_MODULE. Instead every
module defines the macro before #include "Python.h" unless
Py_BUILD_CORE_BUILTIN is already defined.

Py_BUILD_CORE_BUILTIN is defined for every module that is built by
Modules/Setup.

The PR also simplifies Modules/Setup. Makefile and makesetup
already define Py_BUILD_CORE_BUILTIN and include Modules/internal
for us.

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-22 15:36:28 +02:00
Victor Stinner bbe7497c5a
bpo-45434: Remove pystrhex.h header file (GH-28923)
Move Include/pystrhex.h to Include/internal/pycore_strhex.h.
The header file only contains private functions.

The following C extensions are now built with Py_BUILD_CORE_MODULE
macro defined to get access to the internal C API:

* _blake2
* _hashopenssl
* _md5
* _sha1
* _sha3
* _ssl
* binascii
2021-10-13 15:22:35 +02:00
Gregory P. Smith c10392e7dd
bpo-44145: Release the GIL around HMAC_Update. (GH-26157)
It was always meant to be released for parallelization.
This now matches the other similar code in the module.

Thanks michaelforney for noticing!
2021-05-17 00:35:16 -07:00
Erlend Egeberg Aasland 504ffdae4e
bpo-40645: Fix ref leaks in _hashopenssl (GH-26079) 2021-05-12 19:20:41 +01:00
Petr Viktorin 3611db53f5
bpo-40645: Fix reference leak in the _hashopenssl extension (GH-26072)
The `PyModule_AddObjectRef` function doesn't steal a reference, so an extra `Py_DECREF` is needed.

Automerge-Triggered-By: GH:tiran
2021-05-12 09:04:34 -07:00
Christian Heimes 91554e4c5c
bpo-43908: Mark ssl, hash, and hmac types as immutable (GH-25792)
Signed-off-by: Christian Heimes <christian@python.org>
2021-05-02 09:47:45 +02:00
Victor Stinner 0cad068ec1
bpo-43916: Remove _disabled_new() function (GH-25745)
posix and _hashlib use the new Py_TPFLAGS_DISALLOW_INSTANTIATION
flag on their heap types, rather than using a custom tp_new function
(_disabled_new).
2021-04-30 14:06:49 +02:00
Christian Heimes 39258d3595
bpo-43669: PEP 644: Require OpenSSL 1.1.1 or newer (GH-23014)
- Remove HAVE_X509_VERIFY_PARAM_SET1_HOST check
- Update hashopenssl to require OpenSSL 1.1.1
- multissltests only OpenSSL > 1.1.0
- ALPN is always supported
- SNI is always supported
- Remove deprecated NPN code. Python wrappers are no-op.
- ECDH is always supported
- Remove OPENSSL_VERSION_1_1 macro
- Remove locking callbacks
- Drop PY_OPENSSL_1_1_API macro
- Drop HAVE_SSL_CTX_CLEAR_OPTIONS macro
- SSL_CTRL_GET_MAX_PROTO_VERSION is always defined now
- security level is always available now
- get_num_tickets is available with TLS 1.3
- X509_V_ERR MISMATCH is always available now
- Always set SSL_MODE_RELEASE_BUFFERS
- X509_V_FLAG_TRUSTED_FIRST is always available
- get_ciphers is always supported
- SSL_CTX_set_keylog_callback is always available
- Update Modules/Setup with static link example
- Mention PEP in whatsnew
- Drop 1.0.2 and 1.1.0 from GHA tests
2021-04-17 11:36:35 +02:00
Christian Heimes a4833883c9
bpo-43799: OpenSSL 3.0.0: declare OPENSSL_API_COMPAT 1.1.1 (GH-25329)
Signed-off-by: Christian Heimes <christian@python.org>
2021-04-13 08:17:26 +02:00
Pablo Galindo 70cdf1812c
bpo-40645: Fix reference leak in the _hashopenssl extension (GH-25063) 2021-03-29 06:17:40 -07:00
Christian Heimes 933dfd7504
bpo-40645: use C implementation of HMAC (GH-24920)
- [x] fix tests
- [ ] add test scenarios for old/new code.

Signed-off-by: Christian Heimes <christian@python.org>
2021-03-27 06:55:03 -07:00
Victor Stinner 32bd68c839
bpo-42519: Replace PyObject_MALLOC() with PyObject_Malloc() (GH-23587)
No longer use deprecated aliases to functions:

* Replace PyObject_MALLOC() with PyObject_Malloc()
* Replace PyObject_REALLOC() with PyObject_Realloc()
* Replace PyObject_FREE() with PyObject_Free()
* Replace PyObject_Del() with PyObject_Free()
* Replace PyObject_DEL() with PyObject_Free()
2020-12-01 10:37:39 +01:00
Serhiy Storchaka 686c203cd4
bpo-42423: Accept single base class in PyType_FromModuleAndSpec() (GH-23441) 2020-11-22 13:25:02 +02:00
Christian Heimes 46f59ebd01
bpo-1635741: Port _hashlib to multiphase initialization (GH-23358)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-18 16:12:13 +01:00
Christian Heimes db5aed931f
bpo-40791: Use CRYPTO_memcmp() for compare_digest (#20456)
hashlib.compare_digest uses OpenSSL's CRYPTO_memcmp() function
when OpenSSL is available.

Note: The _operator module is a builtin module. I don't want to add
libcrypto dependency to libpython. Therefore I duplicated the wrapper
function and added a copy to _hashopenssl.c.
2020-05-27 21:50:06 +02:00
Christian Heimes 20c22db602
bpo-40671: Prepare _hashlib for PEP 489 (GH-20180) 2020-05-25 10:44:51 +02:00
Christian Heimes aca4670ad6
bpo-40645: restrict HMAC key len to INT_MAX (GH-20238)
Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-19 15:35:51 -07:00
Christian Heimes 62ecd8a8f9
bpo-37630: Fix spelling shake128 -> shake_128 (GH-20154) 2020-05-17 18:32:38 +02:00
Christian Heimes 54f2898fe7
bpo-40645: Implement HMAC in C (GH-20129)
The internal module ``_hashlib`` wraps and exposes OpenSSL's HMAC API. The
new code will be used in Python 3.10 after the internal implementation
details of the pure Python HMAC module are no longer part of the public API.

The code is based on a patch by Petr Viktorin for RHEL and Python 3.6.

Co-Authored-By: Petr Viktorin <encukou@gmail.com>
2020-05-17 13:49:10 +02:00
Christian Heimes d5b3f6b7f9
bpo-37630: Use SHA3 and SHAKE XOF from OpenSSL (GH-16049)
OpenSSL 1.1.1 comes with SHA3 and SHAKE builtin.

Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-16 13:27:06 -07:00
Christian Heimes c087a268a4
bpo-40515: Require OPENSSL_THREADS (GH-19953)
The ``ssl`` and ``hashlib`` modules now actively check that OpenSSL is
build with thread support. Python 3.7.0 made thread support mandatory and no
longer works safely with a no-thread builds.

Signed-off-by: Christian Heimes <christian@python.org>
2020-05-15 11:55:25 -07:00
Christian Heimes 16d4e6f6f5
bpo-40479: Fix hashlib issue with OpenSSL 3.0.0 (GH-20107)
OpenSSL 3.0.0-alpha2 was released today. The FIPS_mode() function has
been deprecated and removed. It no longer makes sense with the new
provider and context system in OpenSSL 3.0.0.

EVP_default_properties_is_fips_enabled() is good enough for our needs in
unit tests. It's an internal API, too.

Signed-off-by: Christian Heimes <christian@python.org>
2020-05-15 09:28:05 -07:00
Victor Stinner e3dfb9b967
bpo-9216: Expose OpenSSL FIPS_mode() as _hashlib.get_fips_mode() (GH-19703)
test.pythoninfo logs OpenSSL FIPS_mode() and Linux
/proc/sys/crypto/fips_enabled in a new "fips" section.

Co-Authored-By: Petr Viktorin <encukou@gmail.com>
2020-04-29 09:04:22 -07:00
Victor Stinner 4a21e57fe5
bpo-40268: Remove unused structmember.h includes (GH-19530)
If only offsetof() is needed: include stddef.h instead.

When structmember.h is used, add a comment explaining that
PyMemberDef is used.
2020-04-15 02:35:41 +02:00
Hai Shi f707d94af6
bpo-39968: Convert extension modules' macros of get_module_state() to inline functions (GH-19017) 2020-03-16 14:15:01 +01:00
Alexandru Ardelean 6552563b3d bpo-38684: haslib: fix build when Blake2 not enabled in OpenSSL (#17043) 2019-11-04 15:55:56 +01:00
Christian Heimes 9055815809 bpo-38270: More fixes for strict crypto policy (GH-16418)
test_hmac and test_hashlib test built-in hashing implementations and
OpenSSL-based hashing implementations. Add more checks to skip OpenSSL
implementations when a strict crypto policy is active.

Use EVP_DigestInit_ex() instead of EVP_DigestInit() to initialize the
EVP context. The EVP_DigestInit() function clears alls flags and breaks
usedforsecurity flag again.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue38270
2019-09-27 06:03:53 -07:00
Christian Heimes df69e75edc
bpo-38142: Updated _hashopenssl.c to be PEP 384 compliant (#16071)
* Updated _hashopenssl.c to be PEP 384 compliant
* Remove refleak test from test_hashlib. The updated type no longer accepts random arguments to __init__.
2019-09-25 23:03:30 +02:00
Christian Heimes 724f1a5723 bpo-33936: Don't call obsolete init methods with OpenSSL 1.1.0+ (GH-16140)
``OPENSSL_VERSION_1_1`` was never defined in ``_hashopenssl.c``. 


https://bugs.python.org/issue33936
2019-09-16 12:10:05 -07:00
Christian Heimes eb2b0c694a
bpo-38153: detect shake independently from sha3 (GH-16143)
XOF digests (SHAKE) are not available in OpenSSL 1.1.0 but SHA3 fixed-length digests are.

Signed-off-by: Christian Heimes <christian@python.org>
2019-09-14 17:29:54 +02:00
Serhiy Storchaka 279f44678c
bpo-37206: Unrepresentable default values no longer represented as None. (GH-13933)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
2019-09-14 12:24:05 +03:00
Christian Heimes 995b5d38e7 bpo-38153: Normalize hashlib algorithm names (GH-16083)
Signed-off-by: Christian Heimes <christian@python.org>
2019-09-13 14:31:19 +01:00
Christian Heimes 7cad53e6b0 bpo-9216: Add usedforsecurity to hashlib constructors (GH-16044)
The usedforsecurity keyword only argument added to the hash constructors is useful for FIPS builds and similar restrictive environment with non-technical requirements that legacy algorithms be forbidden by their implementations without being explicitly annotated as not being used for any security related purposes.  Linux distros with FIPS support benefit from this being standard rather than making up their own way(s) to do it.

Contributed and Signed-off-by: Christian Heimes christian@python.org
2019-09-12 19:30:00 -05:00