In this commit:
* Support BufferedProtocol in set_protocol() and start_tls()
* Fix proactor to cancel readers reliably
* Update tests to be compatible with OpenSSL 1.1.1
* Clarify BufferedProtocol docs
* Bump TLS tests timeouts to 60 seconds; eliminate possible race from start_serving
* Rewrite test_start_tls_server_1
(cherry picked from commit dbf102271f)
Co-authored-by: Yury Selivanov <yury@magic.io>
Fixed bug where calling write_eof() on a _SelectorSocketTransport after
it's already closed raises AttributeError.
(cherry picked from commit 23f587e395)
Co-authored-by: twisteroid ambassador <twisteroidambassador@users.noreply.github.com>
The failure may be due to the use oF ZFS, a case we already ignore
for Solaris-based systems where ZFS is frequently used.
(cherry picked from commit 09c4a7dee2)
Co-authored-by: Ned Deily <nad@python.org>
Pickles of type variables and subscripted generics are now future-proof
and compatible with older Python versions.
(cherry picked from commit 09f3221fbb)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
bpo-32374, bpo-33629: Use support.SuppressCrashReport() in
test_bad_traverse() of MultiPhaseExtensionModuleTests to prevent
leaking a core dump file.
(cherry picked from commit 483000e164)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
bpo-26510 in 3.7.0a2 changed the behavior of argparse to make
subparsers required by default, returning to the behavior of 2.7
and 3.2. The behavior was changed in 3.3 to be no longer required.
While it might make more sense to have the default to required,
compatibility with 3.3 through 3.6 is probably less disruptive
than trying to reintroduce compatibility with 2.7 at this point.
This change restores the 3.6 behavior.
(cherry picked from commit 8ebf5ceb0f)
Co-authored-by: Ned Deily <nad@python.org>
Add a new block_on_close class attribute to ForkingMixIn and
ThreadingMixIn classes of socketserver to opt-in for pre-3.7 behaviour.
(cherry picked from commit 453bd0bc65)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-33353: test_asyncio uses smaller sendfile data (#7083)
bpo-32622, bpo-33353: sendfile() tests of test_asyncio use socket
buffers of 1 kB "to test on relative small data sets". Send only
160 KiB rather 10 MB to make the test much faster.
Shrink also SendfileBase.DATA from 1600 KiB to 160 KiB.
On Linux, 3 test_sock_sendfile_mix_with_regular_send() runs now take
less than 1 second, instead of 18 seconds.
On FreeBSD, the 3 tests didn't hang, but took 3 minutes. Now
the 3 tests pass in less than 1 seconds.
(cherry picked from commit 2932755cc1)
* bpo-33353: test_asyncio set SO_SNDBUF after connect (GH-7086)
bpo-32622, bpo-33353: On macOS, sock.connect() changes the
SO_SNDBUF value. Only set SO_SNDBUF and SO_RCVBUF buffer sizes
once a socket is connected or binded, not before.
(cherry picked from commit b97de3dd86)
TLS 1.3 behaves slightly different than TLS 1.2. Session tickets and TLS
client cert auth are now handled after the initialy handshake. Tests now
either send/recv data to trigger session and client certs. Or tests
ignore ConnectionResetError / BrokenPipeError on the server side to
handle clients that force-close the socket fd.
To test TLS 1.3, OpenSSL 1.1.1-pre7-dev (git master + OpenSSL PR
https://github.com/openssl/openssl/pull/6340) is required.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 529525fb5a)
bpo-33604: Bump removal notice from 3.6 to 3.8 and change PendingDeprecationWarning to DeprecationWarning as we had intended to do earlier...
(cherry picked from commit 8bb0b5b03c)
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
Change TLS 1.3 cipher suite settings for compatibility with OpenSSL
1.1.1-pre6 and newer. OpenSSL 1.1.1 will have TLS 1.3 cipers enabled by
default.
Also update multissltests and Travis config to test with latest OpenSSL.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit e8eb6cb792)
Co-authored-by: Christian Heimes <christian@python.org>
* bpo-33263 Fix FD leak in _SelectorSocketTransport. (GH-6450)
Under particular circumstances _SelectorSocketTransport can try to add a reader
even the transport is already being closed. This can lead to FD leak and
invalid stated of the following connections. Fixed the SelectorSocketTransport
to add the reader only if the trasport is still active.
(cherry picked from commit a84d0b361a)
Co-authored-by: Vlad Starostin <drtyrsa@yandex.ru>
uuid._ipconfig_getnode did not validate the maximum length of the value,
so long as the value had the same type of formatting as a MAC address.
This let it select DUIDs as MAC addresses. It now requires an exact
length match.
(cherry picked from commit c66c342cb4)
Co-authored-by: CtrlZvi <viz+github@flippedperspective.com>
Fix the following bugs in the C implementation:
* get_future_loop() silenced all exceptions raised when look up the get_loop
attribute, not just an AttributeError.
* enter_task() silenced all exceptions raised when look up the current task,
not just a KeyError.
* repr() was called for a borrowed link in enter_task() and task_step_impl().
* str() was used instead of repr() in formatting one error message (in
Python implementation too).
* There where few reference leaks in error cases.
(cherry picked from commit 6655354afc)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
The proactor event loop has a race condition when reading with
pausing/resuming. `resume_reading()` unconditionally schedules the read
function to read from the current future. If `resume_reading()` was
called before the previously scheduled done callback fires, this results
in two attempts to get the data from the most recent read and an
assertion failure. This commit tracks whether or not `resume_reading`
needs to reschedule the callback to restart the loop, preventing a
second attempt to read the data.
(cherry picked from commit 4151061855)
Co-authored-by: CtrlZvi <viz+github@flippedperspective.com>
`obj.async` is now a syntax error, so the warning/shim is
quasi-unnecessary.
(cherry picked from commit f90f5d5c1d)
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
This condition as already tested before this code is called.
(cherry picked from commit 9285835a05)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
To be more consistent with other code (and so people stop hassling me!), reflow the dataclasses comments to not use a single space indentation when continuing a paragraph of text.
(cherry picked from commit f8e7549490)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Don't crash on warnings.warn_explicit() if module_globals is not a dict.
(cherry picked from commit b056562860)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Added to the eye-verified htest, not to the unittests.
Also remove some stray leftover comments.
(cherry picked from commit 389a48ede9)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Also bump PendingDeprecationWarning to DeprecationWarning.
(cherry picked from commit 9eb40bc38d)
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
The urllib.robotparser's __str__ representation now includes wildcard
entries and the "Crawl-delay" and "Request-rate" fields.
(cherry picked from commit bd08a0af2d)
Co-authored-by: Michael Lazar <lazar.michael22@gmail.com>
This happens in the NixOS build sandbox, for example, where the only
other user is nobody with home directory /.
(cherry picked from commit 5c0d462689)
Co-authored-by: Anders Kaseorg <andersk@mit.edu>
Pass os.environ's copy to new process created at test_posix:
test_specify_environment. Otherwise important variables such as
LD_LIBRARY_PATH are not set and the child process might not work at all
in an environment where such variables are required for Python to function.
(cherry picked from commit 7ec8f28656)
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Previously emitted cover files only when --missing option was used.
(cherry picked from commit 47ab15470d)
Co-authored-by: Michael Selik <mike@selik.org>
In text and entry boxes, this affects selection by double-click,
movement left/right by control-left/right, and deletion left/right
by control-BACKSPACE/DEL.
(cherry picked from commit 5ff3a161c8)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Fix test_embed.test_pre_initialization_sys_options() when building with --enable-shared
(cherry picked from commit 4114846265)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
External importers were being added in both phases of the import
system initialisation.
They're only supposed to be added in the second phase, after the
import machinery has been appropriately configured.
(cherry picked from commit 0977091dca)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
as docstrings and translatable strings, and rejects
bytes literals and f-string expressions.
(cherry picked from commit 69524821a8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
`test_urllib2net.OtherNetworkTests.test_sites_no_connection_close`
used `http://www.imdb.com/` but it is moved to https so the test is
not valid anymore. Skip test for the moment to allow CI to proceed.
(cherry picked from commit 36d56ea826)
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
Adds some working and markup fixes that I missed
in the initial commit for this issue.
(Follow-up to GH-6419)
(cherry picked from commit 1a5c4bdb6e)
Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
The pydoc CLI assumed -m pydoc would add the empty string
to sys.path, and hence got confused when it switched to
adding the full initial working directory instead.
This refactors the pydoc CLI path manipulation to be
more testable, and ensures it won't accidentally
remove the standard library directory containing
pydoc itself from sys.path.
(cherry picked from commit 82a9481059)
Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
Executors in concurrent.futures accepted tasks after executor was shutdown by interpreter exit. Tasks were left in PENDING state forever. This fix changes submit to instead raise a RuntimeError.
(cherry picked from commit c4b695f85e)
Co-authored-by: Mark Nemec <mrknmc@me.com>
The protocol parameter can be any protocol supported by the
pickle module.
(cherry picked from commit 2ef65f346a)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
When bpo-12015 got resolved by increasing the length of the
random string generated by _RandomNameSequence from six to eight
characters, the docstring of the class was not adjusted accordingly.
(cherry picked from commit 9c463ec88b)
Co-authored-by: Wolfgang Maier <wolfgang.maier@biologie.uni-freiburg.de>
An entry of None in sys.path_importer_cache represents a negative/missing finder for a path, so clearing it out makes sense.
(cherry picked from commit 9e2be60634)
* Add a test for frozen with unhashable field value.
* Improve a comment.
(cherry picked from commit 74940913d2)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
This removes a hack and replaces it with a proper
mapping {'list': 'List', 'dict': 'Dict', ...}.
(cherry picked from commit 2a363d2930)
Co-authored-by: Ivan Levkivskyi <levkivskyi@gmail.com>
Different paths are executed for normal exit and for leaving
the 'async with' block with 'break', 'continue' or 'return'.
(cherry picked from commit 2eeac269dd)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
A 'u' string prefix cannot be paired with either 'r' or 'f'. Consistently
color as much of the prefix, starting at the right, as is valid.
(cherry picked from commit da58533ac6)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
This also fixes python/typingGH-512
This also fixes python/typingGH-511
As was discussed in both issues, some typing forms deserve to be treated
as immutable by copy and pickle modules, so that:
* copy(X) is X
* deepcopy(X) is X
* loads(dumps(X)) is X GH- pickled by reference
This PR adds such behaviour to:
* Type variables
* Special forms like Union, Any, ClassVar
* Unsubscripted generic aliases to containers like List, Mapping, Iterable
This not only resolves inconsistencies mentioned in the issues, but also
improves backwards compatibility with previous versions of Python
(including 3.6).
Note that this requires some dances with __module__ for type variables
(similar to NamedTuple) because the class TypeVar itself is define in typing,
while type variables should get module where they were defined.
https://bugs.python.org/issue32873
(cherry picked from commit 834940375a)
Co-authored-by: Ivan Levkivskyi <levkivskyi@gmail.com>
bpo-32844: subprocess: Fix a potential misredirection of a low fd to stderr.
When redirecting, subprocess attempts to achieve the following state:
each fd to be redirected to is less than or equal to the fd
it is redirected from, which is necessary because redirection
occurs in the ascending order of destination descriptors.
It fails to do so in a couple of corner cases,
for example, if 1 is redirected to 2 and 0 is closed in the parent.
(cherry picked from commit 0e7144b064)
Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
This is part of PEP 487 and the descriptor protocol.
(cherry picked from commit de7a2f04d6)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Allow ttk.Treeview.insert to insert iid that has a false boolean value.
Note iid=0 and iid=False would be same.
(cherry picked from commit 3ab44c0783)
Co-authored-by: Garvit Khatri <garvitdelhi@gmail.com>
Also, re-enable a test for ClassVars with default_factory.
(cherry picked from commit 2b75fc2bc9)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Historically, -m added the empty string as sys.path
zero, meaning it resolved imports against the current
working directory, the same way -c and the interactive
prompt do.
This changes the sys.path initialisation to add the
*starting* working directory as sys.path[0] instead,
such that changes to the working directory while the
program is running will have no effect on imports
when using the -m switch.
(cherry picked from commit d5d9e02dd3)
Harden ssl module against LibreSSL CVE-2018-8970.
X509_VERIFY_PARAM_set1_host() is called with an explicit namelen. A new test
ensures that NULL bytes are not allowed.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit d02ac25ab0)
Co-authored-by: Christian Heimes <christian@python.org>
- new test case for pre-initialization of sys.warnoptions and sys._xoptions
- restored ability to call these APIs prior to Py_Initialize
- updated the docs for the affected APIs to make it clear they can be
called before Py_Initialize
- also enhanced the existing embedding test cases
to check for expected settings in the sys module
(cherry picked from commit bc77eff8b9)
Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
* Change _hash_action to be a function table lookup, instead of a list
of strings which is then tested with if statements.
(cherry picked from commit 01d618c560)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
- When adding a single element to a list, use .append() instead of
+= and creating a new list.
- For consistency, import the copy module, instead of just deepcopy. This
leaves only a module at the class level, instead of a function.
- Improve some comments.
- Improve some whitespace.
- Use tuples instead of lists.
- Simplify a test.
(cherry picked from commit f96ddade00)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Commit 141c5e8c re-added match_hostname() call. The resurrection of the
function call was never intended and was solely a merge mistake.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit e42ae91509)
Co-authored-by: Christian Heimes <christian@python.org>
If a dataclass has a member variable that's of type Field, but it doesn't have a type annotation, raise TypeError.
(cherry picked from commit 56970b8ce9)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
This improves error message for situations when a non-class is
checked w.r.t. an abstract base class.
(cherry picked from commit 40472dd42d)
Co-authored-by: jab <jab@users.noreply.github.com>
- Add missing 'Field' to __all__.
- Improve tests to catch this.
(cherry picked from commit 8e4560a9da)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Fix the way that new annotations in a class are detected.
(cherry picked from commit 8f6eccdc64)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
The result of host() was not empty when the network is constructed by a tuple containing an
integer mask and only 1 bit left for addresses.
(cherry picked from commit 10b134a07c)
Co-authored-by: Xiang Zhang <angwerzx@126.com>
If the class has a member that's a MemberDescriptorType, it's not a default value, it's from that member being in __slots__.
(cherry picked from commit 7389fd935c)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
If a non-dataclass derives from a frozen dataclass, allow attributes to be set.
Require either all of the dataclasses in a class hierarchy to be frozen, or all non-frozen.
Store `@dataclass` parameters on the class object under `__dataclass_params__`. This is needed to detect frozen base classes.
(cherry picked from commit f199bc655e)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
This reverts commit ac317700ce.
(Reverts only the lib2to3 part.)
(cherry picked from commit f64aae46da)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Multi-phase initialized modules allow m_traverse to be called while the
module is still being initialized, so module authors may need to account
for that.
(cherry picked from commit c2b0b12d1a)
Co-authored-by: Marcel Plch <gmarcel.plch@gmail.com>
New tests also added.
I also made the comments in line with the builtin Grammar/Grammar. PEP 306 was
withdrawn, Kees Blom's railroad program has been lost to the sands of time for
at least 16 years now (I found a python-dev post from people looking for it).
(cherry picked from commit b51f5de711)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
In some conditions the standard streams will be None or closed in the child process (for example if using "pythonw" instead of "python" on Windows). Avoid failing with a non-0 exit code in those conditions.
Report and initial patch by poxthegreat.
(cherry picked from commit e756f66c83)
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
* Skip write()/data_received() if sslpipe is destroyed
(cherry picked from commit 5e80a71ab6)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
test_asyncio hangs indefinitely on macOS 10.13.2+ on `read_pty_output()`
using the KqueueSelector. Closing `proto.transport` (as is done in
`write_pty_output()`) seems to fix it.
(cherry picked from commit 12f74d8608)
Co-authored-by: Nathan Henrie <n8henrie@users.noreply.github.com>
* bpo-32991: Add test capturing expectation.
DocTestFinder.find should return an empty list for doctests in a namespace package.
* bpo-32991: Restore expectation that inspect.getfile on a namespace package raises TypeError.
(cherry picked from commit b9650a04a8)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Like Python, IDLE optionally runs one startup file in the Shell window
before presenting the first interactive input prompt. For IDLE,
option -s runs a file named in environmental variable IDLESTARTUP or
PYTHONSTARTUP; -r file runs file. Python sets __file__ to the startup
file name before running the file and unsets it before the first
prompt. IDLE now does the same when run normally, without the -n
option.
(cherry picked from commit 22c82be5df)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* Prevent low-grade poplib REDOS (CVE-2018-1060)
The regex to test a mail server's timestamp is susceptible to
catastrophic backtracking on long evil responses from the server.
Happily, the maximum length of malicious inputs is 2K thanks
to a limit introduced in the fix for CVE-2013-1752.
A 2KB evil response from the mail server would result in small slowdowns
(milliseconds vs. microseconds) accumulated over many apop calls.
This is a potential DOS vector via accumulated slowdowns.
Replace it with a similar non-vulnerable regex.
The new regex is RFC compliant.
The old regex was non-compliant in edge cases.
* Prevent difflib REDOS (CVE-2018-1061)
The default regex for IS_LINE_JUNK is susceptible to
catastrophic backtracking.
This is a potential DOS vector.
Replace it with an equivalent non-vulnerable regex.
Also introduce unit and REDOS tests for difflib.
Co-authored-by: Tim Peters <tim.peters@gmail.com>
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Jamie Davis <davisjam@vt.edu>
(cherry picked from commit 0e6c8ee235)
OpenSSL 1.1 has introduced a new API to set the minimum and maximum
supported protocol version. The API is easier to use than the old
OP_NO_TLS1 option flags, too.
Since OpenSSL has no call to set minimum version to highest supported,
the implementation emulate maximum_version = MINIMUM_SUPPORTED and
minimum_version = MAXIMUM_SUPPORTED by figuring out the minumum and
maximum supported version at compile time.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 698dde16f6)
Co-authored-by: Christian Heimes <christian@python.org>
Direct instantiation of SSLSocket and SSLObject objects is now prohibited.
The constructors were never documented, tested, or designed as public
constructors. The SSLSocket constructor had limitations. For example it was
not possible to enabled hostname verification except was
ssl_version=PROTOCOL_TLS_CLIENT with cert_reqs=CERT_REQUIRED.
SSLContext.wrap_socket() and SSLContext.wrap_bio are the recommended API
to construct SSLSocket and SSLObject instances. ssl.wrap_socket() is
also deprecated.
The only test case for direct instantiation was added a couple of days
ago for IDNA testing.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 9d50ab563d)
Co-authored-by: Christian Heimes <christian@python.org>
* bpo-32947: OpenSSL 1.1.1-pre1 / TLS 1.3 fixes
Misc fixes and workarounds for compatibility with OpenSSL 1.1.1-pre1 and
TLS 1.3 support. With OpenSSL 1.1.1, Python negotiates TLS 1.3 by
default. Some test cases only apply to TLS 1.2. Other tests currently
fail because the threaded or async test servers stop after failure.
I'm going to address these issues when OpenSSL 1.1.1 reaches beta.
OpenSSL 1.1.1 has added a new option OP_ENABLE_MIDDLEBOX_COMPAT for TLS
1.3. The feature is enabled by default for maximum compatibility with
broken middle boxes. Users should be able to disable the hack and CPython's test suite needs
it to verify default options.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 05d9fe32a1)
Co-authored-by: Christian Heimes <christian@python.org>
This restriction will be relaxed at a future date.
(cherry picked from commit 2fa6b9eae0)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
unsafe_hash=False is now the default. It is the same behavior as the old hash=None parameter. unsafe_hash=True will try to add __hash__. If it already exists, TypeError is raised.
(cherry picked from commit dbf9cff48a)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
* Support sendfile on Windows Proactor event loop naively.
(cherry picked from commit a19fb3c6aa)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
This code never did anything correct or useful. The class attribute will never be affected, and the condition will never be true.
(cherry picked from commit 5fb632e831)
Co-authored-by: Aaron Gallagher <habnabit@users.noreply.github.com>
Add tests to verify connection with secp384r1 ECDH curves.
(cherry picked from commit b7b9225831)
Co-authored-by: Christian Heimes <christian@python.org>
* The SSLSocket is no longer implemented on top of SSLObject to
avoid an extra level of indirection.
* Owner and session are now handled in the internal constructor.
* _ssl._SSLSocket now uses the same method names as SSLSocket and
SSLObject.
* Channel binding type check is now handled in C code. Channel binding
is always available.
The patch also changes the signature of SSLObject.__init__(). In my
opinion it's fine. A SSLObject is not a user-constructable object.
SSLContext.wrap_bio() is the only valid factory.
(cherry picked from commit 141c5e8c24)
Co-authored-by: Christian Heimes <christian@python.org>
ssl.match_hostname() has been simplified and no longer depends on re and
ipaddress module for wildcard and IP addresses. Error reporting for invalid
wildcards has been improved.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit aef1283ba4)
Co-authored-by: Christian Heimes <christian@python.org>
Previously, the ssl module stored international domain names (IDNs)
as U-labels. This is problematic for a number of reasons -- for
example, it made it impossible for users to use a different version
of IDNA than the one built into Python.
After this change, we always convert to A-labels as soon as possible,
and use them for all internal processing. In particular, server_hostname
attribute is now an A-label, and on the server side there's a new
sni_callback that receives the SNI servername as an A-label rather than
a U-label.
(cherry picked from commit 11a1493bc4)
Co-authored-by: Christian Heimes <christian@python.org>
Make test.support.temp_cwd() fork-safe. The context manager test.support.temp_cwd() no longer removes the temporary directory when executing in a process other than the parent it entered from.
If a forked child exits the context manager it won't do the cleanup.
(cherry picked from commit 33dddac00b)
Co-authored-by: Anselm Kruis <a.kruis@science-computing.de>
dump is similar to print but less flexible. lastopenbracketpos is now always initialized in _study2, as was stmt_bracketing, so the class settings are not needed. get_last_open_bracket_pos is never called.
(cherry picked from commit 451d1edaf4)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
There are no code changes other than comments and docstrings.
(cherry picked from commit c84cf6c03f)
Co-authored-by: Cheryl Sabella <cheryl.sabella@gmail.com>
The CPython runtime assumes that there is a one-to-one relationship (for a given interpreter) between PyThreadState and OS threads. Sending and receiving on a channel in the same interpreter was causing crashes because of this (specifically due to a check in PyThreadState_Swap()). The solution is to not switch threads if the interpreter is the same.
(cherry picked from commit f53d9f2778)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Initialize self._ssnd_chunk so that aifc.Error is raised as intended,
not AttributeError.
(cherry picked from commit 80d20b918b)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Using the system and place-dependent default encoding for open()
is a bad idea for IDLE's system and location-independent files.
(cherry picked from commit 688722cedd)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
The PrintNameOffset field of the reparse data buffer
was treated as a number of characters instead of bytes.
(cherry picked from commit 3c34aad4e7)
Co-authored-by: SSE4 <tomskside@gmail.com>
GUI test test_file_buttons() only looks at initial ascii-only lines,
but failed on systems where open() defaults to 'ascii' because
readline() internally reads and decodes far enough ahead to encounter
a non-ascii character in CREDITS.txt.
(cherry picked from commit f34e03ec0e)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
fnmatch.translate() no longer produces patterns which contain set
operations.
Sets starting with '[' or containing '--', '&&', '~~' or '||' will
be interpreted differently in regular expressions in future versions.
Currently they emit warnings. fnmatch.translate() now avoids producing
patterns containing such sets by accident.
(cherry picked from commit 23cdbfa744)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
it was using our mocked listdir to check when the files were gone.
(cherry picked from commit 4ad703b7ca)
Co-authored-by: Bernhard M. Wiedemann <githubbmw@lsmod.de>
Add entry for uniform lower context; add 'since' to be explicit.
(cherry picked from commit 05e806767b)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Previously the module name was used, which broke relative imports when pdb was run against a plain module or submodule.
(cherry picked from commit 38bfa8418f)
Co-authored-by: Mario Corchero <mariocj89@gmail.com>
Fix typos found by codespell in docs, docstrings, and comments.
(cherry picked from commit c3d9508ff2)
Co-authored-by: Leo Arias <leo.arias@canonical.com>
This reverts commit 72a0d218dc.
The reverted commit had a few issues so it was unanimously decided
to undo it. See the bpo issue for details.
(cherry picked from commit 383b32fe10)
Co-authored-by: Yury Selivanov <yury@magic.io>
No longer add /Library/Python/3.x/site-packages, the Apple-supplied
system Python site-packages directory, to sys.path for macOS framework
builds in case Apple ships a version of Python 3. A similar change
was made earlier to Python 2.7 where it was found that the coupling
between the system Python and a user-installed framework Python often
caused confusion or pip install failures.
Fix two (in my opinion) spurious failure conditions in the lib2to3.tests.test_parser.TestParserIdempotency test_parser test.
Use the same encoding found in the initial file to write a temp file for a diff. This retains the BOM if the encoding was initially utf-8-sig.
If the file cannot be parsed using the normal grammar, try again with no print statement which should succeed for valid files using future print_function
For case (1), the driver was correctly handling a BOM in a utf-8 file, but then the test was not writing a comparison file using 'utf-8-sig' to diff against, so the BOM got removed. I don't think that is the fault of the parser, and lib2to3 will retain the BOM.
For case (2), lib2to3 pre-detects the use of from __future__ import print_function or allows the user to force this interpretation with a -p flag, and then selects a different grammar with the print statement removed. That makes the test cases unfair to this test as the driver itself doesn't know which grammar to use. As a minimal fix, the test will try using a grammar with the print statement, and if that fails fall back on a grammar without it. A more thorough handling of the idempotency test would to be to parse all files using both grammars and ignore if one of the two failed but otherwise check both. I didn't think this was necessary but can change.
This allows the compression level to be specified when writing zipfiles
(for the entire file *and* overridden on a per-file basis).
Contributed by Bo Bayles
Do not allow receiving a SIGINT to cause the subprocess module to trigger an
immediate SIGKILL of the child process. SIGINT is normally sent to all child
processes by the OS at the same time already as was the established normal
behavior in 2.7 and 3.2. This behavior change was introduced during the fix to https://bugs.python.org/issue12494 and is generally surprising to command line
tool users who expect other tools launched in child processes to get their own
SIGINT and do their own cleanup.
In Python 3.3-3.6 subprocess.call and subprocess.run would immediately
SIGKILL the child process upon receiving a SIGINT (which raises a
KeyboardInterrupt). We now give the child a small amount of time to
exit gracefully before resorting to a SIGKILL.
This is also the case for subprocess.Popen.__exit__ which would
previously block indefinitely waiting for the child to die. This was
hidden from many users by virtue of subprocess.call and subprocess.run
sending the signal immediately.
Behavior change: subprocess.Popen.__exit__ will not block indefinitely
when the exiting exception is a KeyboardInterrupt. This is done for
user friendliness as people expect their ^C to actually happen. This
could cause occasional orphaned Popen objects when not using `call` or
`run` with a child process that hasn't exited.
Refactoring involved: The Popen.wait method deals with the
KeyboardInterrupt second chance, existing platform specific internals
have been renamed to _wait().
Also fixes comment typos.
Fix socket(fileno=fd) by auto-detecting the socket's family, type,
and proto from the file descriptor. The auto-detection can be overruled
by passing in family, type, and proto explicitly.
Without the fix, all socket except for TCP/IP over IPv4 are basically broken:
>>> s = socket.create_connection(('www.python.org', 443))
>>> s
<socket.socket fd=3, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=6, laddr=('2003:58:bc4a:3b00:56ee:75ff:fe47:ca7b', 59730, 0, 0), raddr=('2a04:4e42:1b::223', 443, 0, 0)>
>>> socket.socket(fileno=s.fileno())
<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('2003:58:bc4a:3b00::%2550471192', 59730, 0, 2550471192), raddr=('2a04:4e42:1b:0:700c:e70b:ff7f:0%2550471192', 443, 0, 2550471192)>
Signed-off-by: Christian Heimes <christian@python.org>
The commit removes one unnecessary "if" clause in genobject.c. That "if" clause was masking un-awaited coroutines warnings just to make writing unittests more convenient.
Add test cases for IDNA 2003 and 2008 host names. IDNA 2003
internationalized host names are working since bpo-31399 has landed. IDNA
2008 deviations are still broken and will be fixed in another patch.
Signed-off-by: Christian Heimes <christian@python.org>
Until now Python used a hard coded white list of default TLS cipher
suites. The old approach has multiple downsides. OpenSSL's default
selection was completely overruled. Python did neither benefit from new
cipher suites (ChaCha20, TLS 1.3 suites) nor blacklisted cipher suites.
For example we used to re-enable 3DES.
Python now defaults to OpenSSL DEFAULT cipher suite selection and black
lists all unwanted ciphers. Downstream vendors can override the default
cipher list with --with-ssl-default-suites.
Signed-off-by: Christian Heimes <christian@python.org>
This test is unstable and currently prevents to make any new change
since the test always fails on Travis CI.
Skip the test to get more time to fix it.
To mitigate the situation when the buildbot is under load
and is unable to send/receive data fast enough:
* reduce the size of the payload
* set a generous timeout for socket ops
Better account for single-line compound statements and
semi-colon separated statements when suggesting
Py3 replacements for Py2 print statements.
Initial patch by Nitish Chandra.
The arguments to a generator function which is declared as a
contextmanager are stored inside the context manager, and
thus are kept alive, even when it is used as a regular context
manager, and not as a function decorator (where it needs
the original arguments to recreate the generator on each
call).
This is a possible unnecessary memory leak, so this changes
contextmanager.__enter__ to release the saved arguments,
as that method being called means that particular CM instance
isn't going to need to recreate the underlying generator.
Patch by Martin Teichmann.
Class authors no longer need to specify repr=False if they want to provide a custom __repr__ for dataclasses. The same thing applies for the other dunder methods that the dataclass decorator adds. If dataclass finds that a dunder methods is defined in the class, it will not overwrite it.
bpo-31399: Let OpenSSL verify hostname and IP
The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and
X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses.
* Remove match_hostname calls
* Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host()
* Add documentation for OpenSSL 1.0.2 requirement
* Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform.
* Add hostname_checks_common_name
Signed-off-by: Christian Heimes <christian@python.org>
Miss Islington (bot)
Victor Stinner
Serhiy Storchaka
Eric Snow
Pablo Galindo
Ned Deily
Zsolt Dollenstein
Ethan Furman
Brett Cannon
Nick Coghlan
Ivan Levkivskyi
xdegaye
Christian Heimes
Barry Warsaw
Xiang Zhang
Bernhard M. Wiedemann
Stéphane Wirtel
Benjamin Peterson
John Jolly
Anders Lorentsen
Eric Appelt
Bo Bayles
Gregory P. Smith
Mark Shannon
Yury Selivanov
jayyyin
Raymond Hettinger
Andrew Svetlov
Victor Stinner
Nathaniel J. Smith
embg
Eric V. Smith
larryhastings
Nitish Chandra
Mario Corchero
Martin Teichmann
INADA Naoki
Коренберг Марк