* bpo-40826: Add _Py_EnsureTstateNotNULL() macro (GH-20571)
Add _Py_EnsureTstateNotNULL(tstate) macro: call Py_FatalError() if
tstate is NULL, the error message contains the current function name.
(cherry picked from commit 3026cad59b)
* bpo-40826: PyOS_InterruptOccurred() requires GIL (GH-20578)
PyOS_InterruptOccurred() now fails with a fatal error if it is called
with the GIL released.
(cherry picked from commit cbe1296922)
* bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20579)
Fix GIL usage in PyOS_Readline(): lock the GIL to set an exception.
Pass tstate to my_fgets() and _PyOS_WindowsConsoleReadline(). Cleanup
these functions.
(cherry picked from commit c353764fd5)
* bpo-40826: Add _PyOS_InterruptOccurred(tstate) function (GH-20599)
my_fgets() now calls _PyOS_InterruptOccurred(tstate) to check for
pending signals, rather calling PyOS_InterruptOccurred().
my_fgets() is called with the GIL released, whereas
PyOS_InterruptOccurred() must be called with the GIL held.
test_repl: use text=True and avoid SuppressCrashReport in
test_multiline_string_parsing().
Fix my_fgets() on Windows: fgets(fp) does crash if fileno(fp) is closed.
(cherry picked from commit fa7ab6aa0f)
* bpo-40630: Add tracemalloc.reset_peak (GH-20102, cherrypick 8b62644)
The reset_peak function sets the peak memory size to the current size,
representing a resetting of that metric. This allows for recording the
peak of specific sections of code, ignoring other code that may have
had a higher peak (since the most recent `tracemalloc.start()` or
tracemalloc.clear_traces()` call).
* Adjust docs to point to 3.9
Fix :mod:`ssl`` code to be compatible with OpenSSL 1.1.x builds that use
``no-deprecated`` and ``--api=1.1.0``.
Note: Tests assume full OpenSSL API and fail with limited API.
Signed-off-by: Christian Heimes <christian@python.org>
Co-authored-by: Mark Wright <gienah@gentoo.org>
(cherry picked from commit a871f692b4)
Co-authored-by: Christian Heimes <christian@python.org>
Recent changes to _datetimemodule broke compilation on mingw; see the comments in this change for details.
FWIW, @corona10: this issue is why `PyType_FromModuleAndSpec` & friends take the `bases` argument at run time.
(cherry picked from commit 459acc5516)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Heap types now always visit the type in tp_traverse. See added docs for details.
This reverts commit 0169d3003b.
Automerge-Triggered-By: @encukou
(cherry picked from commit 1cf15af9a6)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
hashlib.compare_digest uses OpenSSL's CRYPTO_memcmp() function
when OpenSSL is available.
Note: The _operator module is a builtin module. I don't want to add
libcrypto dependency to libpython. Therefore I duplicated the wrapper
function and added a copy to _hashopenssl.c..
(cherry picked from commit db5aed931f)
Co-authored-by: Christian Heimes <christian@python.org>
If ctypes fails to convert the result of a callback or if a ctypes
callback function raises an exception, sys.unraisablehook is now
called with an exception set. Previously, the error was logged into
stderr by PyErr_Print().
(cherry picked from commit 10228bad04)
Co-authored-by: Victor Stinner <vstinner@python.org>
```
D:\a\cpython\cpython\Modules\_zoneinfo.c(903,52): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data [D:\a\cpython\cpython\PCbuild\_zoneinfo.vcxproj]
D:\a\cpython\cpython\Modules\_zoneinfo.c(904,44): warning C4267: '=': conversion from 'size_t' to 'unsigned int', possible loss of data [D:\a\cpython\cpython\PCbuild\_zoneinfo.vcxproj]
D:\a\cpython\cpython\Modules\_zoneinfo.c(1772,31): warning C4244: '=': conversion from 'ssize_t' to 'uint8_t', possible loss of data [D:\a\cpython\cpython\PCbuild\_zoneinfo.vcxproj]
```
(cherry picked from commit e4799b9594)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
ctypes now raises an ArgumentError when a callback
is invoked with more than 1024 arguments.
The ctypes module allocates arguments on the stack in
ctypes_callproc() using alloca(), which is problematic
when large numbers of arguments are passed. Instead
of a stack overflow, this commit raises an ArgumentError
if more than 1024 parameters are passed.
(cherry picked from commit 29a1384c04)
Co-authored-by: Sean Gillespie <sean@swgillespie.me>
struct.error is now raised if there is a null character in a struct
format string.
(cherry picked from commit 3f59b55316)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
The scripts in `Tools/peg_generator/scripts` mostly assume that
`ast.parse` and `compile` use the old parser, since this was the
state of things, while we were developing them. They need to be
updated to always use the correct parser. `_peg_parser` is being
extended to support both parsing and compiling with both parsers.
(cherry picked from commit 9645930b5b)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
This updates _PyErr_ChainStackItem() to use _PyErr_SetObject()
instead of _PyErr_ChainExceptions(). This prevents a hang in
certain circumstances because _PyErr_SetObject() performs checks
to prevent cycles in the exception context chain while
_PyErr_ChainExceptions() doesn't.
(cherry picked from commit 7c30d12bd5)
Co-authored-by: Chris Jerdonek <chris.jerdonek@gmail.com>
Signed-off-by: Christian Heimes <christian@python.org>
Automerge-Triggered-By: @tiran
(cherry picked from commit aca4670ad6)
Co-authored-by: Christian Heimes <christian@python.org>
Remove --with-experimental-isolated-subinterpreters configure option
in Python 3.9: the experiment continues in the master branch, but
it's no longer needed in 3.9.
This allows building with older versions of the Windows SDK where the value is not defined.
(cherry picked from commit 711f9e180a)
Co-authored-by: Minmin Gong <gongminmin@msn.com>
Ifdef is not necessary, as AF_INET6 is supported from Windows Vista, and other code in overlapped.c uses AF_INET6 and is not ifdef'd.
Change the raised exception so users are not fooled to think it comes from Windows API.
Automerge-Triggered-By: @njsmith
When an asyncio.Task is cancelled, the exception traceback now
starts with where the task was first interrupted. Previously,
the traceback only had "depth one."
The internal module ``_hashlib`` wraps and exposes OpenSSL's HMAC API. The
new code will be used in Python 3.10 after the internal implementation
details of the pure Python HMAC module are no longer part of the public API.
The code is based on a patch by Petr Viktorin for RHEL and Python 3.6.
Co-Authored-By: Petr Viktorin <encukou@gmail.com>
{date, datetime}.isocalendar() now return a private custom named tuple object
IsoCalendarDate rather than a simple tuple.
In order to leave IsocalendarDate as a private class and to improve what
backwards compatibility is offered for pickling the result of a
datetime.isocalendar() call, add a __reduce__ method to the named tuples that
reduces them to plain tuples. (This is the part of this PR most likely to cause
problems — if it causes major issues, switching to a strucseq or equivalent
would be prudent).
The pure python implementation of IsoCalendarDate uses positional-only
arguments, since it is private and only constructed by position anyway; the
equivalent change in the argument clinic on the C side would require us to move
the forward declaration of the type above the clinic import for whatever
reason, so it seems preferable to hold off on that for now.
bpo-24416: https://bugs.python.org/issue24416
Original PR by Dong-hee Na with only minor alterations by Paul Ganssle.
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
On AIX, time.thread_time() is now implemented with thread_cputime()
which has nanosecond resolution, rather than
clock_gettime(CLOCK_THREAD_CPUTIME_ID) which has a resolution of 10 ms.
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).
This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:
1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)
Differences from the reference implementation:
- The module is arranged slightly differently: the accelerated module is
`_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
some changes in the test support function. (Suggested by Victor
Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
because we may do some refactoring in the future that would likely
require this separation anyway; we may:
- include the property tests
- automatically run all the tests against both pure Python and C,
rather than manually constructing C and Python test classes (similar
to the way this works with test_datetime.py, which generates C
and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
CI, though only on the coverage jobs. Introducing a PyPI dependency as
part of the CI build was controversial, and this is seen as less of a
major change, since the coverage jobs already depend on pip and PyPI.
Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:
- Fixed reference and memory leaks
With much debugging help from Pablo Galindo
- Added smoke tests ensuring that the C and Python modules are built
The import machinery can be somewhat fragile, and the "seamlessly falls
back to pure Python" nature of this module makes it so that a problem
building the C extension or a failure to import the pure Python version
might easily go unnoticed.
- Adjustments to zoneinfo.__dir__
Suggested by Petr Viktorin.
- Slight refactorings as suggested by Steve Dower.
- Removed unnecessary if check on std_abbr
Discovered this because of a missing line in branch coverage.
OpenSSL can be build without support for TLS 1.0 and 1.1. The ssl module
now correctly adheres to OPENSSL_NO_TLS1 and OPENSSL_NO_TLS1_1 flags.
Also update multissltest to test with latest OpenSSL and LibreSSL
releases.
Signed-off-by: Christian Heimes <christian@python.org>
Automerge-Triggered-By: @tiran
The ``ssl`` and ``hashlib`` modules now actively check that OpenSSL is
build with thread support. Python 3.7.0 made thread support mandatory and no
longer works safely with a no-thread builds.
Signed-off-by: Christian Heimes <christian@python.org>
OpenSSL 3.0.0-alpha2 was released today. The FIPS_mode() function has
been deprecated and removed. It no longer makes sense with the new
provider and context system in OpenSSL 3.0.0.
EVP_default_properties_is_fips_enabled() is good enough for our needs in
unit tests. It's an internal API, too.
Signed-off-by: Christian Heimes <christian@python.org>
Change spelling of a #define in _tkinter.c from HAVE_LIBTOMMAMTH to HAVE_LIBTOMMATH, since this is used to keep track of tclTomMath.h, not tclTomMamth.h. No other file seems to refer to this variable.
Pass PEP 573 defining_class to os.DirEntry methods. The module state
is now retrieve from defining_class rather than Py_TYPE(self), to
support subclasses (even if DirEntry doesn't support subclasses yet).
* Pass the module rather than defining_class to DirEntry_fetch_stat().
* Only get the module state once in _posix_clear(),
_posix_traverse() and _posixmodule_exec().
Move PyInterpreterState.fs_codec into a new
PyInterpreterState.unicode structure.
Give a name to the fs_codec structure and use this structure in
unicodeobject.c.
_Py_hashtable_t values become regular "void *" pointers.
* Add _Py_hashtable_entry_t.data member
* Remove _Py_hashtable_t.data_size member
* Remove _Py_hashtable_t.get_func member. It is no longer needed
to specialize _Py_hashtable_get() for a specific value size, since
all entries now have the same size (void*).
* Remove the following macros:
* _Py_HASHTABLE_GET()
* _Py_HASHTABLE_SET()
* _Py_HASHTABLE_SET_NODATA()
* _Py_HASHTABLE_POP()
* Rename _Py_hashtable_pop() to _Py_hashtable_steal()
* _Py_hashtable_foreach() callback now gets key and value rather than
entry.
* Remove _Py_hashtable_value_destroy_func type. value_destroy_func
callback now only has a single parameter: data (void*).
Rewrite _tracemalloc to store "trace_t*" rather than directly
"trace_t" in traces hash tables. Traces are now allocated on the heap
memory, outside the hash table.
Add tracemalloc_copy_traces() and tracemalloc_copy_domains() helper
functions.
Remove _Py_hashtable_copy() function since there is no API to copy a
key or a value.
Remove also _Py_hashtable_delete() function which was commented.
Rewrite _Py_hashtable_t type to always store the key as
a "const void *" pointer. Add an explicit "key" member to
_Py_hashtable_entry_t.
Remove _Py_hashtable_t.key_size member.
hash and compare functions drop their hash table parameter, and their
'key' parameter type becomes "const void *".
Rewrite how the _tracemalloc module stores traces of other domains.
Rather than storing the domain inside the key, it now uses a new hash
table with the domain as the key, and the data is a per-domain traces
hash table.
* Add tracemalloc_domain hash table.
* Remove _Py_tracemalloc_config.use_domain.
* Remove pointer_t and related functions.
Miss Islington (bot)
Victor Stinner
Huon Wilson
Christian Heimes
Minmin Gong
Kjell Braden
Chris Jerdonek
Shantanu
Batuhan Taskaya
Paul Ganssle
Batuhan Taskaya
Andrew York
Dong-hee Na
Steve Dower