Backport of TLS 1.3 related fixes from 3.7.
Misc fixes and workarounds for compatibility with OpenSSL 1.1.1 from git
master and TLS 1.3 support. With OpenSSL 1.1.1, Python negotiates TLS 1.3 by
default. Some test cases only apply to TLS 1.2.
OpenSSL 1.1.1 has added a new option OP_ENABLE_MIDDLEBOX_COMPAT for TLS
1.3. The feature is enabled by default for maximum compatibility with
broken middle boxes. Users should be able to disable the hack and CPython's test suite needs
it to verify default options
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 2a4ee8aa01)
The options OP_NO_COMPRESSION, OP_CIPHER_SERVER_PREFERENCE,
OP_SINGLE_DH_USE, OP_SINGLE_ECDH_USE, OP_NO_SSLv2 (except
for PROTOCOL_SSLv2), and OP_NO_SSLv3 (except for PROTOCOL_SSLv3)
are set by default. The initial cipher suite list contains only
HIGH ciphers, no NULL ciphers and MD5 ciphers (except for PROTOCOL_SSLv2).
(cherry picked from commit 358cfd426c)
Change TLS 1.3 cipher suite settings for compatibility with OpenSSL
1.1.1-pre6 and newer. OpenSSL 1.1.1 will have TLS 1.3 cipers enabled by
default.
Also update multissltests to test with latest OpenSSL.
Signed-off-by: Christian Heimes <christian@python.org>.
(cherry picked from commit 3e630c541b)
Co-authored-by: Christian Heimes <christian@python.org>
Fix a NULL pointer deref in ssl module. The cert parser did not handle CRL
distribution points with empty DP or URI correctly. A malicious or buggy
certificate can result into segfault.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue35746
(cherry picked from commit a37f52436f)
Co-authored-by: Christian Heimes <christian@python.org>
This reverts commit aa58050843.
Due to regressions found with using Tk 8.6.9.1, build the python.org
macOS installers with Tcl/Tk 8.6.8 as used in previous releases.
check_environ() of distutils.utils now catchs KeyError on calling
pwd.getpwuid(): don't create the HOME environment variable in this
case.
(cherry picked from commit 17d0c0595e)
bpo-34279, bpo-35412: support.run_unittest() no longer raises
TestDidNotRun if a test result contains skipped tests. The
exception is now only raised if no test have been run and no test
have been skipped.
(cherry picked from commit 3a8f4fef4a)
* bpo-16039: CVE-2013-1752: Change use of readline() in
imaplib.IMAP4_SSL to limit line length. Remove IMAP4_SSL.readline()
and IMAP4_SSL.read() to inherit safe IMAP4 implementation.
* bpo-20118: reenable test_linetoolong() of test_imaplib
on ThreadedNetworkedTests and ThreadedNetworkedTestsSSL.
The test now sets the _MAXLINE limit to 10 characters.
Fix xml.dom.minidom cloneNode() on a document with an entity: pass
the correct arguments to the user data handler of an entity (fix an
old copy/paste mistake).
Bug spotted and fix proposed by Charalampos Stratakis, initial
reproducer written by Petr Viktorin.
Co-Authored-By: Charalampos Stratakis <cstratak@redhat.com>
Co-Authored-By: Petr Viktorin <encukou@gmail.com>
(cherry picked from commit 8e04186889)
* posixpath.expanduser() now returns the input path unchanged if
the HOME environment variable is not set and pwd.getpwuid() raises
KeyError (the current user identifier doesn't exist in the password
database).
* Add test_no_home_directory() to test_site.
(cherry picked from commit f2f4555d82)
When Python is compiled in debug mode, PyMem_Malloc() uses debug
hooks, but it also uses pymalloc allocator instead of malloc().
Problem: pymalloc is not thread-safe, whereas PyMem_Malloc() is
thread-safe in release mode (it's a thin wrapper to malloc() in this
case).
Modify the debug hook to use malloc() for PyMem_Malloc().
Fix an undefined behaviour in the pthread implementation of
PyThread_start_new_thread(): add a function wrapper to always return
NULL.
Add pythread_callback struct and pythread_wrapper() to thread_pthread.h.
(cherry picked from commit 9eea6eaf23)
Discovered using clang's MemorySanitizer.
A msan build will fail by simply executing: ./python -c 'u"\N"'
(cherry picked from commit 746b2d3)
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
_io.IncrementalNewlineDecoder's initializer possibly assigns out-of-range
value to the bitwise struct field.
(cherry picked from commit b08746bfdf)
Co-authored-by: Xiang Zhang <angwerzx@126.com>
Adding `max_num_fields` to `cgi.FieldStorage` to make DOS attacks harder by
limiting the number of `MiniFieldStorage` objects created by `FieldStorage`.
(cherry picked from commit 209144831b)
Prior to this revision, after the shutdown of a `BaseServer`,
the server accepted a last single request
if it was sent between the server socket polling
and the polling timeout.
This can be problematic for instance for a server restart
for which you do not want to interrupt the service,
by not closing the listening socket during the restart.
One request failed because of this behavior.
Note that only one request failed,
following requests were not accepted, as expected.
(cherry picked from commit 10cb3760e8)
The xml.sax and xml.dom.domreg modules now obey
sys.flags.ignore_environment.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue34791
(cherry picked from commit 223e501fb9)
Co-authored-by: Christian Heimes <christian@python.org>
Verify the value for the parameter '-s' of the cProfile CLI. Patch by Robert
Kuska.
Co-authored-by: Robert Kuska <rkuska@gmail.com>
(cherry picked from commit fcd5e84a51)
https://bugs.python.org/issue23420
python-gdb.py now handles errors on computing the line number
of a Python frame.
Changes:
* PyFrameObjectPtr.current_line_num() now catchs any Exception on
calling addr2line(), instead of failing with a surprising "<class
'TypeError'> 'FakeRepr' object is not subscriptable" error.
* All callers of current_line_num() now handle current_line_num()
returning None.
* PyFrameObjectPtr.current_line() now also catchs IndexError on
getting a line from the Python source file.
(cherry picked from commit 2e438cc255)
The bytearray constructor converted unexpected exceptions
(e.g. MemoryError and KeyboardInterrupt) to TypeError.
(cherry picked from commit e890421e33)
When Python is built with the intel control-flow protection flags,
-mcet -fcf-protection, gdb is not able to read the stack without
actually jumping inside the function. This means an extra
'next' command is required to make the $pc (program counter)
enter the function and make the stack of the function exposed to gdb.
test_gdb: get_gdb_repr() now uses the "backtrace 1" command after
breakpoint, as in the master branch.
Co-Authored-By: Marcel Plch <gmarcel.plch@gmail.com>
(cherry picked from commit 9b7c74ca32)
(cherry picked from commit 79d21331e6)
Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly.
This makes streamed zips compatible with MacOS Archive Utility and
other applications.
(cherry picked from commit 4ba3b50bfe)
Co-authored-by: Silas Sewell <silas@sewell.org>
Update all test certs and keys to use future proof crypto settings:
* 3072 bit RSA keys
* SHA-256 signature
Signed-off-by: Christian Heimes <christian@python.org>.
(cherry picked from commit e6dac00779)
https://bugs.python.org/issue34542
The C accelerated _elementtree module now initializes hash randomization
salt from _Py_HashSecret instead of libexpat's default CPRNG.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue34623.
(cherry picked from commit cb5778f00c)
Co-authored-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue34623
Include ``openssl/dh.h`` header file to fix implicit function declaration of ``DH_free()``.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>.
(cherry picked from commit b3a271fc0c)
Co-authored-by: Alexandru Ardelean <ardeleanalex@gmail.com>
https://bugs.python.org/issue34710
Python 2 never checked for I/O error when reading .py files and
thus could mistake an I/O error for EOF and create incorrect .pyc
files.
This adds an check for this and aborts on an error.
The change to Tools/scripts/diff.py effectively backports part of
a2637729f2.
The test code changed in Doc/library/difflib.rst is not present in current 3.x.
Read from data socket to avoid "[SSL] shutdown while in init" exception
during shutdown of the dummy server.
Signed-off-by: Christian Heimes <christian@python.org>
<!-- issue-number: [bpo-34391](https://www.bugs.python.org/issue34391) -->
https://bugs.python.org/issue34391
<!-- /issue-number -->.
(cherry picked from commit 1590c39336)
Co-authored-by: Christian Heimes <christian@python.org>
Downstream vendors have started to deprecate weak keys. Update all RSA keys
and DH params to use at least 2048 bits.
Finite field DH param file use RFC 7919 values, generated with
certtool --get-dh-params --sec-param=high
Signed-off-by: Christian Heimes <christian@python.org>.
(cherry picked from commit 88bfd0bce0)
Co-authored-by: Christian Heimes <christian@python.org>
Some MacOS-tk combinations need .update_idletasks().
The call is both unneeded and innocuous on Linux and Windows.
Patch by Kevin Walzer.
(cherry picked from commit 9beaef6225)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
imaplib now allows MOVE command in IMAP4.uid() (RFC 6851:
IMAP MOVE Extension) and potentially as a name of supported
method of IMAP4 object.
(cherry picked from commit caa331d492)
Co-authored-by: Matěj Cepl <mcepl@cepl.eu>
There was a missing check for integer overflow, several function calls
were not checked for failure, and allocated memory was not freed if an
error occurred.
The Opera Browser was using a outdated command line invocation that resulted in an incorrect URL being opened in the browser when requested using the webbrowser module.
* Correct the arguments passed to the Opera Browser when opening a new URL..
(cherry picked from commit 3cf1f154ed)
Co-authored-by: Bumsik Kim <k.bumsik@gmail.com>
Most of the change involves fixing up the test suite, which previously made
the assumption that there wouldn't be a new line if the input didn't end in
one.
Contributed by Ammar Askar.
(cherry picked from commit c4ef4896ea)
Fix passing lists and tuples of strings containing special characters
'"', '\\', '{', '}' and '\n' as options to tkinter.ttk widgets.
(cherry picked from commit 5bb5bbfca8)
* bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686)
* Enhance also format_duration(): work on integers and rounds towards
+infinity (math.ceil).
* Write unit tests on format_duration()
(cherry picked from commit 4ffe9c2b25)
* bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736)
regrtest: Add warning when using less than 3 warmup runs like -R 1:3.
(cherry picked from commit cac4fef886)
* bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735)
Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs.
Add check for invalid --huntrleaks/-R parameters.
(cherry picked from commit 58ed7307ea)
Fix test_gdbm.test_reorganize() on macOS with gdbm 1.15: add a
larger value to make sure that the file size changes.
(cherry picked from commit 13c79c677f)
* bpo-32962: python-gdb catchs ValueError on read_var() (GH-7692)
python-gdb now catchs ValueError on read_var(): when Python has no
debug symbols for example.
(cherry picked from commit 019d33b7a4)
* bpo-32962: python-gdb catchs UnicodeDecodeError (GH-7693)
python-gdb now catchs UnicodeDecodeError exceptions when calling
string().
(cherry picked from commit d22fc0bc7d)
bpo-29367: python-gdb.py now supports also method-wrapper
(wrapperobject) objects.
(cherry picked from commit 611083331d)
Hangul composition check boundaries are wrong for the second character
([0x1161, 0x1176) instead of [0x1161, 0x1176]) and third character ((0x11A7, 0x11C3)
instead of [0x11A7, 0x11C3])..
(cherry picked from commit d134809cd3)
Co-authored-by: Wonsup Yoon <pusnow@me.com>
- bugfix and test for fragile metavar handling in argparse (see
bpo-24089, bpo-14046, bpo-25058, bpo-11874)
- also fixes some incorrect tests that did not make 1-element tuples correctly
(cherry picked from commit 66f02aa32f)
Co-authored-by: wim glenn <wim.glenn@gmail.com>
Rename Lib/test/bisect.py to Lib/test/bisect_cmd.py. The old name was
in conflict with Lib/bisect.py, causing test failures, depending how
tests were run.
For example, "python2.7 Lib/test/test_httpservers.py" imported
indirectly Lib/test/bisect.py instead of Lib/bisect.py
on "import bisect".
* Fixed a leak when the GC fails to add an object with __del__ into
the gc.garbage list.
* PyGC_Collect() can now be called when an exception is set and
preserves it.
(cherry picked from commit 301e3cc8a5)
uuid._ipconfig_getnode did not validate the maximum length of the value,
so long as the value had the same type of formatting as a MAC address.
This let it select DUIDs as MAC addresses. It now requires an exact
length match..
(cherry picked from commit c66c342cb4)
Co-authored-by: CtrlZvi <viz+github@flippedperspective.com>
The robotparser's __str__ representation now includes wildcard
entries.
(cherry picked from commit c3fa1f2b93)
Co-authored-by: Michael Lazar <lazar.michael22@gmail.com>.
By spec, the "long double" in _PyGC_Head requires the union to always be 16-byte
aligned. However, obmalloc only yields 8-byte alignment. Compilers including GCC
8 are starting to use alignment information to do store-merging. So, the "long
double" needs to be changed to a simple "double" as was long ago done in Python
3 by e348c8d154. For 2.7, we need to add some
dummy padding to make sure _PyGC_Head stays the same size.
These include:
- bpo-32726: Provide an additional, more modern macOS installer variant that
supports macOS 10.9+ systems in 64-bit mode only. Upgrade the supplied
third-party libraries to OpenSSL 1.0.2n and SQLite 3.22.0. The 10.9+
installer now supplies its own private copy of Tcl/Tk 8.6.8.
- bpo-24414: Default macOS deployment target is now set by ``configure`` to
the build system's OS version (as is done by Python 3), not ``10.4``;
override with, for example, ``./configure MACOSX_DEPLOYMENT_TARGET=10.4``.
- bpo-19019: All 2.7 macOS installer variants now supply their own version
of ``OpenSSL 1.0.2``; the Apple-supplied SSL libraries and root
certificates are not longer used. The ``Installer Certificate`` command
in ``/Applications/Python 2.7`` may be used to download and install a
default set of root certificates from the third-party ``certifi`` package.
- bpo-11485: python.org macOS Pythons no longer supply a default SDK value
(e.g. ``-isysroot /``) or specific compiler version default (e.g.
``gcc-4.2``) when building extension modules. Use ``CC``, ``SDKROOT``,
and ``DEVELOPER_DIR`` environment variables to override compilers or to
use an SDK. See Apple's ``xcrun`` man page for more info.
- prepare for pending Apple removal of 32-bit support in future macOS release
Allow ttk.Treeview.insert to insert iid that has a false boolean value.
Note iid=0 and iid=False would be same.
(cherry picked from commit 3ab44c0783)
Co-authored-by: Garvit Khatri <garvitdelhi@gmail.com>
This change generally splits the xmlparser creation code into an unsafe part with "rollback" error handling and a safe "object initialisation done" part with normal decref cleanup.
LibreSSL 2.7 introduced OpenSSL 1.1.0 API. The ssl module now detects
LibreSSL 2.7 and only provides API shims for OpenSSL < 1.1.0 and
LibreSSL < 2.7.
Documentation updates and fixes for failing tests will be provided in
another patch set.
Signed-off-by: Christian Heimes <christian@python.org>.
(cherry picked from commit 4ca0739c9d)
Co-authored-by: Christian Heimes <christian@python.org>
The regex to decode a number in fpformat is susceptible to catastrophic backtracking. This is a potential DOS vector if a server is using fpformat on untrusted number strings.
Replace it with an equivalent non-vulnerable regex. The match behavior of the new regex is slightly different. It captures the whole integer part of the number in one group, Leading zeros are stripped off later.
* Prevent low-grade poplib REDOS (CVE-2018-1060)
The regex to test a mail server's timestamp is susceptible to
catastrophic backtracking on long evil responses from the server.
Happily, the maximum length of malicious inputs is 2K thanks
to a limit introduced in the fix for CVE-2013-1752.
A 2KB evil response from the mail server would result in small slowdowns
(milliseconds vs. microseconds) accumulated over many apop calls.
This is a potential DOS vector via accumulated slowdowns.
Replace it with a similar non-vulnerable regex.
The new regex is RFC compliant.
The old regex was non-compliant in edge cases.
* Prevent difflib REDOS (CVE-2018-1061)
The default regex for IS_LINE_JUNK is susceptible to
catastrophic backtracking.
This is a potential DOS vector.
Replace it with an equivalent non-vulnerable regex.
Also introduce unit and REDOS tests for difflib.
Co-authored-by: Tim Peters <tim.peters@gmail.com>
Co-authored-by: Christian Heimes <christian@python.org>.
(cherry picked from commit 0e6c8ee235)
The ctypes module used to depend on indirect linking for dlopen. The shared
extension is now explicitly linked against libdl on platforms with dl.
Signed-off-by: Christian Heimes <christian@python.org>.
(cherry picked from commit 5bb9692575)
Co-authored-by: Christian Heimes <christian@python.org>
The ssl module now detects missing NPN support in LibreSSL.
Co-Authored-By: Bernard Spil <brnrd@FreeBSD.org>
Signed-off-by: Christian Heimes <christian@python.org>.
(cherry picked from commit 6cdb7954b0)
Co-authored-by: Christian Heimes <christian@python.org>
The SSL module no longer sends IP addresses in SNI TLS extension on
platforms with OpenSSL 1.0.2+ or inet_pton.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit e9370a4738)
Co-authored-by: Christian Heimes <christian@python.org>
Debian Unstable has disabled TLS 1.0 and 1.1 for SSLv23_METHOD(). Change
TLS/SSL protocol of some tests to PROTOCOL_TLS or PROTOCOL_TLSv1_2 to
make them pass on Debian.
Signed-off-by: Christian Heimes <christian@python.org>
Make test.support.temp_cwd() fork-safe. The context manager test.support.temp_cwd() no longer removes the temporary directory when executing in a process other than the parent it entered from.
If a forked child exits the context manager it won't do the cleanup..
(cherry picked from commit 33dddac00b)
Co-authored-by: Anselm Kruis <a.kruis@science-computing.de>
Initialize self._ssnd_chunk so that aifc.Error is raised as intended,
not AttributeError.
(cherry picked from commit 80d20b918b)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
stratakis
Victor Stinner
Miss Islington (bot)
Serhiy Storchaka
Sanyam Khurana
Ned Deily
Myles Borins
Zackery Spytz
Steve Dower
Julien Palard
Gregory P. Smith
Benjamin Peterson
matthewbelisle-wf
Denis Ledoux
Tal Einat
Cheryl Sabella
Stéphane Wirtel
tzickel
Andrés Delfino
Christian Heimes
Vladimir Matveev
xdegaye
Chih-Hsuan Yen
Mickaël Schoentgen
Terry Jan Reedy
Xtreak
Pablo Galindo
Ammar Askar
native-api
Xiang Zhang
Antoine Pitrou
Bo Bayles
Florian Weimer
Steve Dower
scoder
Jamie Davis
Alexey Izbyshev
bennorth
Anselm Kruis