They can be exposed when some C API calls fail due to lack of
memory.
* Failed Py_BuildValue() could cause an assertion error in the
following TextIOWrapper.tell().
* input_chunk could be decrefed twice in TextIOWrapper.seek()
after failed Py_BuildValue().
* initvalue could leak in StringIO.__getstate__() after failed
PyDict_Copy().
(cherry picked from commit fdb5a50ef3)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* `flags` is indeed deprecated, but there is a validation on its value for
backwards compatibility reasons. This adds mention of this in the docs.
* The docs say that `sizehint` is deprecated and ignored, but it is still
used when `epoll_create1()` is unavailable. This adds mention of this in
the docs.
* `sizehint=-1` is acceptable again, and is replaced with `FD_SETSIZE-1`.
This is needed to have a default value available at the Python level,
since `FD_SETSIZE` is not exposed to Python. (see: bpo-31938)
* Reject `sizehint=0` since it is invalid to pass on to `epoll_create()`.
The relevant tests have also been updated.
(cherry picked from commit 0cdf5f4289)
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
bz2, lzma: When Decompressor.__init__() is called twice, free the old
lock to not leak memory.
(cherry picked from commit 9b7cf75721)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Fix "LC_ALL=C python3.7 -V": reset properly the command line parser
when the encoding changes after reading the Python configuration.
Fix pymain_read_conf(): use memset(0) to reset properly cmdline.
(cherry picked from commit 6c5a4b3156)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Hangul composition check boundaries are wrong for the second character
([0x1161, 0x1176) instead of [0x1161, 0x1176]) and third character ((0x11A7, 0x11C3)
instead of [0x11A7, 0x11C3]).
(cherry picked from commit d134809cd3)
Co-authored-by: Wonsup Yoon <pusnow@me.com>
The documentation for CERT_NONE, CERT_OPTIONAL, and CERT_REQUIRED were
misleading and partly wrong. It fails to explain that OpenSSL behaves
differently in client and server mode. Also OpenSSL does validate the
cert chain everytime. With SSL_VERIFY_NONE a validation error is not
fatal in client mode and does not request a client cert in server mode.
Also discourage people from using CERT_OPTIONAL in client mode.
(cherry picked from commit ef24b6c54d)
Co-authored-by: Christian Heimes <christian@python.org>
A datetime object d is aware if d.tzinfo is not None and
d.tzinfo.utcoffset(d) does not return None. If d.tzinfo is None,
or if d.tzinfo is not None but d.tzinfo.utcoffset(d) returns None,
d is naive.
This commit ensures that instances with non-None d.tzinfo, but
d.tzinfo.utcoffset(d) returning None are treated as naive.
In addition, C acceleration code will raise TypeError if
d.tzinfo.utcoffset(d) returns an object with the type other than
timedelta.
* Updated the documentation.
Assume that the term "naive" is defined elsewhere and remove the
not entirely correct clarification. Thanks, Tim.
(cherry picked from commit 877b23202b)
Co-authored-by: Alexander Belopolsky <abalkin@users.noreply.github.com>
Move the floor() call into fbound() to call floor() on a double
rather than an int. The change should enhance the rounding.
Document also (int)double rounding mode.
(cherry picked from commit 45e4efba7f)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Fix a crash in Python initialization when parsing the command line
options.
Fix memcpy() size parameter: previously, we read one wchar_t after
the end of _PyOS_optarg. Moreover, don't copy the trailingg NUL
character: we write it manually anyway.
Thanks Christoph Gohlke for the bug report and the fix!
(cherry picked from commit 58d1683255)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Use uuid_enc_be() if available to encode UUID to bytes as big endian.
(cherry picked from commit 17d8830312)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Fixed a leak when the GC fails to add an object with __del__ into
the gc.garbage list.
* PyGC_Collect() can now be called when an exception is set and
preserves it.
* Fixed an undefined behavior with comparing a dead pointer with NULL.
(cherry picked from commit 301e3cc8a5)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Fix the following bugs in the C implementation:
* get_future_loop() silenced all exceptions raised when look up the get_loop
attribute, not just an AttributeError.
* enter_task() silenced all exceptions raised when look up the current task,
not just a KeyError.
* repr() was called for a borrowed link in enter_task() and task_step_impl().
* str() was used instead of repr() in formatting one error message (in
Python implementation too).
* There where few reference leaks in error cases.
(cherry picked from commit 6655354afc)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
The editline emulation needs to be initialized *after* the name is
defined. This fixes the long open issue.
(cherry picked from commit c2f082e9d1)
Co-authored-by: Zvezdan Petkovic <zpetkovic@acm.org>
The ssl module now contains OP_NO_RENEGOTIATION constant, available with
OpenSSL 1.1.0h or 1.1.1.
Note, OpenSSL 1.1.0h hasn't been released yet.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 67c4801663)
Co-authored-by: Christian Heimes <christian@python.org>
Starting with glibc 2.27.9000-xxx, sigaddset() can return EINVAL for some
reserved signal numbers between 1 and NSIG. The `range(1, NSIG)` idiom
is commonly used to select all signals for blocking with `pthread_sigmask`.
So we ignore the sigaddset() return value until we expose sigfillset()
to provide a better idiom.
(cherry picked from commit 25038ecfb6)
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Setup modules are no longer built with -DPy_BUILD_CORE by default,
as using that flag may now require including additional internal-only header files.
Instead, only the modules that specifically need it use that setting.
(cherry picked from commit 063db62aab)
Co-authored-by: xdegaye <xdegaye@gmail.com>
bpo-32844: subprocess: Fix a potential misredirection of a low fd to stderr.
When redirecting, subprocess attempts to achieve the following state:
each fd to be redirected to is less than or equal to the fd
it is redirected from, which is necessary because redirection
occurs in the ascending order of destination descriptors.
It fails to do so in a couple of corner cases,
for example, if 1 is redirected to 2 and 0 is closed in the parent.
(cherry picked from commit 0e7144b064)
Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
Harden ssl module against LibreSSL CVE-2018-8970.
X509_VERIFY_PARAM_set1_host() is called with an explicit namelen. A new test
ensures that NULL bytes are not allowed.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit d02ac25ab0)
Co-authored-by: Christian Heimes <christian@python.org>
LibreSSL 2.7 introduced OpenSSL 1.1.0 API. The ssl module now detects
LibreSSL 2.7 and only provides API shims for OpenSSL < 1.1.0 and
LibreSSL < 2.7.
Documentation updates and fixes for failing tests will be provided in
another patch set.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 4ca0739c9d)
Co-authored-by: Christian Heimes <christian@python.org>
bpo-33018 (GH-5944) fixed bpo-32999 too. So fc7df0e6 is not required
anymore. Revert it except test case.
(cherry picked from commit f757b72b25)
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
This improves error message for situations when a non-class is
checked w.r.t. an abstract base class.
(cherry picked from commit 40472dd42d)
Co-authored-by: jab <jab@users.noreply.github.com>
fstat may block for long time if the file descriptor is on a
non-responsive NFS server, hanging all threads. Most fstat() calls are
handled by _Py_fstat(), releasing the GIL internally, but but
_Py_fstat_noraise() does not release the GIL, and most calls release the
GIL explicitly around it.
This patch fixes last 2 calls to _Py_fstat_no_raise(), avoiding hangs
when calling:
- mmap.mmap()
- os.urandom()
- random.seed()
(cherry picked from commit 4484f9dca9)
Co-authored-by: Nir Soffer <nirsof@gmail.com>
Multi-phase initialized modules allow m_traverse to be called while the
module is still being initialized, so module authors may need to account
for that.
(cherry picked from commit c2b0b12d1a)
Co-authored-by: Marcel Plch <gmarcel.plch@gmail.com>
OpenSSL 1.1 has introduced a new API to set the minimum and maximum
supported protocol version. The API is easier to use than the old
OP_NO_TLS1 option flags, too.
Since OpenSSL has no call to set minimum version to highest supported,
the implementation emulate maximum_version = MINIMUM_SUPPORTED and
minimum_version = MAXIMUM_SUPPORTED by figuring out the minumum and
maximum supported version at compile time.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 698dde16f6)
Co-authored-by: Christian Heimes <christian@python.org>
* bpo-32947: OpenSSL 1.1.1-pre1 / TLS 1.3 fixes
Misc fixes and workarounds for compatibility with OpenSSL 1.1.1-pre1 and
TLS 1.3 support. With OpenSSL 1.1.1, Python negotiates TLS 1.3 by
default. Some test cases only apply to TLS 1.2. Other tests currently
fail because the threaded or async test servers stop after failure.
I'm going to address these issues when OpenSSL 1.1.1 reaches beta.
OpenSSL 1.1.1 has added a new option OP_ENABLE_MIDDLEBOX_COMPAT for TLS
1.3. The feature is enabled by default for maximum compatibility with
broken middle boxes. Users should be able to disable the hack and CPython's test suite needs
it to verify default options.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 05d9fe32a1)
Co-authored-by: Christian Heimes <christian@python.org>
* Support sendfile on Windows Proactor event loop naively.
(cherry picked from commit a19fb3c6aa)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
The 'optimization' is for space in the executable file, not for run time.
(cherry picked from commit 186b606d8a)
Co-authored-by: Cheryl Sabella <cheryl.sabella@gmail.com>
The ssl module now detects missing NPN support in LibreSSL.
Co-Authored-By: Bernard Spil <brnrd@FreeBSD.org>
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 6cdb7954b0)
Co-authored-by: Christian Heimes <christian@python.org>
* The SSLSocket is no longer implemented on top of SSLObject to
avoid an extra level of indirection.
* Owner and session are now handled in the internal constructor.
* _ssl._SSLSocket now uses the same method names as SSLSocket and
SSLObject.
* Channel binding type check is now handled in C code. Channel binding
is always available.
The patch also changes the signature of SSLObject.__init__(). In my
opinion it's fine. A SSLObject is not a user-constructable object.
SSLContext.wrap_bio() is the only valid factory.
(cherry picked from commit 141c5e8c24)
Co-authored-by: Christian Heimes <christian@python.org>
Previously, the ssl module stored international domain names (IDNs)
as U-labels. This is problematic for a number of reasons -- for
example, it made it impossible for users to use a different version
of IDNA than the one built into Python.
After this change, we always convert to A-labels as soon as possible,
and use them for all internal processing. In particular, server_hostname
attribute is now an A-label, and on the server side there's a new
sni_callback that receives the SNI servername as an A-label rather than
a U-label.
(cherry picked from commit 11a1493bc4)
Co-authored-by: Christian Heimes <christian@python.org>
The CPython runtime assumes that there is a one-to-one relationship (for a given interpreter) between PyThreadState and OS threads. Sending and receiving on a channel in the same interpreter was causing crashes because of this (specifically due to a check in PyThreadState_Swap()). The solution is to not switch threads if the interpreter is the same.
(cherry picked from commit f53d9f2778)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
os.dup2() tests for dup3() system call availability at runtime,
but doesn't remember the result across calls, repeating
the test on each call with inheritable=False.
Since the caller of os.dup2() is expected to hold the GIL,
fix this by making the variable holding the test result static.
(cherry picked from commit b3caf388a0)
Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
A closing parentheses was missing.
Signed-off-by: Ngie Cooper <yaneurabeya@gmail.com>
(cherry picked from commit 7745ec4e35)
Co-authored-by: ngie-eign <1574099+ngie-eign@users.noreply.github.com>
The PrintNameOffset field of the reparse data buffer
was treated as a number of characters instead of bytes.
(cherry picked from commit 3c34aad4e7)
Co-authored-by: SSE4 <tomskside@gmail.com>
Fix a rare but potential pre-exec child process deadlock in subprocess on POSIX systems when marking file descriptors inheritable on exec in the child process. This bug appears to have been introduced in 3.4 with the inheritable file descriptors support.
This also changes Python/fileutils.c `set_inheritable` to use the "slow" two `fcntl` syscall path instead of the "fast" single `ioctl` syscall path when asked to be async signal safe (by way of being asked not to raise exceptions). `ioctl` is not a POSIX async-signal-safe approved function.
ref: http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html
(cherry picked from commit c1e46e94de)
Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
Fix typos found by codespell in docs, docstrings, and comments.
(cherry picked from commit c3d9508ff2)
Co-authored-by: Leo Arias <leo.arias@canonical.com>
This reverts commit 72a0d218dc.
The reverted commit had a few issues so it was unanimously decided
to undo it. See the bpo issue for details.
(cherry picked from commit 383b32fe10)
Co-authored-by: Yury Selivanov <yury@magic.io>
Fix socket(fileno=fd) by auto-detecting the socket's family, type,
and proto from the file descriptor. The auto-detection can be overruled
by passing in family, type, and proto explicitly.
Without the fix, all socket except for TCP/IP over IPv4 are basically broken:
>>> s = socket.create_connection(('www.python.org', 443))
>>> s
<socket.socket fd=3, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=6, laddr=('2003:58:bc4a:3b00:56ee:75ff:fe47:ca7b', 59730, 0, 0), raddr=('2a04:4e42:1b::223', 443, 0, 0)>
>>> socket.socket(fileno=s.fileno())
<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('2003:58:bc4a:3b00::%2550471192', 59730, 0, 2550471192), raddr=('2a04:4e42:1b:0:700c:e70b:ff7f:0%2550471192', 443, 0, 2550471192)>
Signed-off-by: Christian Heimes <christian@python.org>
Miss Islington (bot)
Eric Snow
Pablo Galindo
Serhiy Storchaka
Ivan Levkivskyi
Victor Stinner
Stéphane Wirtel
Christian Heimes