Update all test certs and keys to use future proof crypto settings:
* 3072 bit RSA keys
* SHA-256 signature
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit e6dac00779)
* Fix Tools/clinic/clinic_test.py: add missing
FakeClinic.destination_buffers attribute and pass a file argument
to Clinic().
* Rename Tools/clinic/clinic_test.py to Lib/test/test_clinic.py:
add temporary Tools/clinic/ to sys.path to import the clinic
module.
Co-Authored-By: Pablo Galindo <pablogsal@gmail.com>
(cherry picked from commit 65fc98e7b1)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Sometimes some versions of the shared libraries that are part of the
traceback are compiled in optimised mode and the Program Counter (PC)
is not present, not allowing gdb to walk the frames back. When this
happens, the Python bindings of gdb raise an exception, making the
test impossible to succeed.
(cherry picked from commit f2ef51f8be)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-34485: stdout uses surrogateescape on POSIX locale (GH-8986)
Standard streams like sys.stdout now use the "surrogateescape" error
handler, instead of "strict", on the POSIX locale (when the C locale is not
coerced and the UTF-8 Mode is disabled).
Add tests on sys.stdout.errors with LC_ALL=POSIX.
Fix the error handler of standard streams like sys.stdout:
PYTHONIOENCODING=":" is now ignored instead of setting the error handler to
"strict".
(cherry picked from commit 315877dc36)
* The UTF-8 Mode is now also enabled by the "POSIX" locale, not only
by the "C" locale.
* On FreeBSD, Py_DecodeLocale() and Py_EncodeLocale() now also forces
the ASCII encoding if the LC_CTYPE locale is "POSIX", not only if
the LC_CTYPE locale is "C".
* test_utf8_mode.test_cmd_line() checks also that the command line
arguments are decoded from UTF-8 when the the UTF-8 Mode is enabled
with POSIX locale or C locale.
(cherry picked from commit 5cb258950c)
Remove "trace.cover" left from previous test runs before testing
that it is no longer created.
(cherry picked from commit b44a1d4f71)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
An old apparent AIX behavior workaround in test_subprocess's
test_undecodable_env is no longer needed.
(cherry picked from commit 89d79b1449)
Co-authored-by: Michael Felt <aixtools@users.noreply.github.com>
Avoids an integer underflow in the time module's year handling code.
(cherry picked from commit 76be0fffff)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
The current C implementations **crash** if the input includes a surrogate
Unicode code point, which is not possible to encode in UTF-8.
Important notes:
1. It is possible to pass a non-UTF-8 string as a separator to the
`.isoformat()` methods.
2. The pure-Python `datetime.fromisoformat()` implementation accepts
strings with a surrogate as the separator.
In `datetime.fromisoformat()`, in the special case of non-UTF-8 separators,
this implementation will take a performance hit by making a copy of the
input string and replacing the separator with 'T'.
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Co-authored-by: Paul Ganssle <paul@ganssle.io>
(cherry picked from commit 096329f0b2)
Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
The missing NULL check was reported by Svace static analyzer.
(cherry picked from commit cdbf50cba1)
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Read from data socket to avoid "[SSL] shutdown while in init" exception
during shutdown of the dummy server.
Signed-off-by: Christian Heimes <christian@python.org>
<!-- issue-number: [bpo-34391](https://www.bugs.python.org/issue34391) -->
https://bugs.python.org/issue34391
<!-- /issue-number -->
(cherry picked from commit 1590c39336)
Co-authored-by: Christian Heimes <christian@python.org>
Downstream vendors have started to deprecate weak keys. Update all RSA keys
and DH params to use at least 2048 bits.
Finite field DH param file use RFC 7919 values, generated with
certtool --get-dh-params --sec-param=high
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 88bfd0bce0)
Co-authored-by: Christian Heimes <christian@python.org>
The main cause of this bug was fixed as part of bpo-31908.
(cherry picked from commit c8b0dbc492)
Co-authored-by: Berker Peksag <berker.peksag@gmail.com>
* -X dev: it is now possible to override the memory allocator using
PYTHONMALLOC even if the developer mode is enabled.
* Add _Py_InitializeFromConfig()
* Add _Py_Initialize_ReadEnvVars() to set global configuration
variables from environment variables
* Fix the code to initialize Python: Py_Initialize() now also reads
environment variables
* _Py_InitializeCore() can now be called twice: the second call
only replaces the configuration.
* Write unit tests on Py_Initialize() and the different ways to
configure Python
* The isolated mode now always sets Py_IgnoreEnvironmentFlag and
Py_NoUserSiteDirectory to 1.
* pymain_read_conf() now saves/restores the configuration
if the encoding changed
Multiprocessing test_timeout() now accepts a delta of 100 ms instead
of just 50 ms, since the test failed with 135.8 ms instead of the
expected 200 ms.
(cherry picked from commit 5640d030e1)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* Fix integer overflow in os.readv(), os.writev(), os.preadv()
and os.pwritev() and in os.sendfile() with headers or trailers
arguments (on BSD-based OSes and MacOS).
* Fix sending the part of the file in os.sendfile() on MacOS.
Using the trailers argument could cause sending more bytes from
the input file than was specified.
Thanks Ned Deily for testing on 32-bit MacOS.
(cherry picked from commit 9d5727326a)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* help(hashlib) didn't work because of incorrect module name in blake2b and
blake2s classes.
* Constructors blake2*(), sha3_*(), shake_*() and keccak_*() incorrectly
accepted keyword argument "string" for binary data, but documented as
accepting the "data" keyword argument. Now this parameter is positional-only.
* Keyword-only parameters in blake2b() and blake2s() were not documented as
keyword-only.
* Default value for some parameters of blake2b() and blake2s() was None,
which is not acceptable value.
* The length argument for shake_*.digest() was wrapped out to 32 bits.
* The argument for shake_128.digest() and shake_128.hexdigest() was not
positional-only as intended.
* TypeError messages for incorrect arguments in all constructors sha3_*(),
shake_*() and keccak_*() incorrectly referred to sha3_224.
Also made the following enhancements:
* More accurately specified input and result types for strings, bytes and
bytes-like objects.
* Unified positional parameter names for update() and constructors.
* Improved formatting.
(cherry picked from commit f1d36d8efa)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Enable and fix SMTPUTF8SimTests in test_smtplib.
The tests for SMTPUTF8SimTests in test_smtplib.py were not actually
being run because test_smtplib was still using the 'test_main' pattern,
and the class was never added to test_main.
Additionally, one of the tests needed to be moved to the non-UTF8 server
class because it relies on the server not being UTF-8 compatible (and it
had a bug in in).
(cherry picked from commit 48ed88a93b)
Co-authored-by: chason <chason@gmail.com>
On Windows, passing a negative value to local results in an OSError because localtime_s on Windows does not support negative timestamps. Unfortunately this means that fold detection for timestamps between 0 and max_fold_seconds will result in this OSError since we subtract max_fold_seconds from the timestamp to detect a fold. However, since we know there haven't been any folds in the interval [0, max_fold_seconds) in any timezone, we can hackily just forego fold detection for this time range on Windows.
(cherry picked from commit 96d1e69a12)
Co-authored-by: Ammar Askar <ammar_askar@hotmail.com>
Now base64.Error is always raised instead of UnboundLocalError or
OverflowError.
(cherry picked from commit ac0b3c2f4d)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
When Python is installed on Windows, python -m test test_tools failed
because it tried to run Tools\scripts\2to3.py which requires an
argument. Skip this script. On other platforms or on Windows but when
run from source code (not installed), the script is called "2to3"
instead of "2to.py" and so was already skipped.
Modify also the unit test to unload all modules which have been
loaded by the test.
(cherry picked from commit 752d4b7531)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
The test failed on my laptop because the busy loop took 15.9 ms
whereas the test expects at least 20 ms. Modify test_process_time()
as test_thread_time() has been modified recently: only require 15 ms
instead of 20 ms.
(cherry picked from commit e78dace8dc)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Py_Main() can again be called after Py_Initialize(), as in Python
3.6. The new configuration is ignored, except of
_PyMainInterpreterConfig.argv which is used to update sys.argv.
(cherry picked from commit fb47bca9ee)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Increase the timeout: give timeout x 4 instead of timeout x 2 to
threads to wait until the Event is set, but reduce the sleep from 500
ms to 250 ms. So the test should be more reliable and faster!
(cherry picked from commit 81950495ba)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-34130: Fix test_signal.test_socket() (GH-8326)
test_signal.test_socket(): On Windows, sometimes even if the C signal handler
succeed to write the signal number into the write end of the socketpair, the
test fails with a BlockingIOError on the non-blocking read.recv(1) because the
read end of the socketpair didn't receive the byte yet.
Fix the race condition on Windows by setting the read end as blocking.
(cherry picked from commit 99bb6df66a)
* bpo-34130: Fix test_signal.test_warn_on_full_buffer() (GH-8327)
On Windows, sometimes test_signal.test_warn_on_full_buffer() fails to
fill the socketpair buffer. In that case, the C signal handler
succeed to write into the socket, it doesn't log the expected send
error, and so the test fail.
On Windows, the test now uses a timeout of 50 ms to fill the
socketpair buffer to fix this race condition.
Other changes:
* Begin with large chunk size to fill the buffer to speed up the
test.
* Add error messages to assertion errors to more easily identify
which assertion failed.
* Don't set the read end of the socketpair as non-blocking.
(cherry picked from commit 686b4b5ff2)
`_PyUnicode_TransformDecimalAndSpaceToASCII()` missed trailing NUL char.
It caused buffer overflow in `_Py_string_to_number_with_underscores()`.
This bug is introduced in 9b6c60cb.
(cherry picked from commit 16dfca4d82)
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
The test failed on AMD64 Debian root 3.x buildbot because the busy
loop of 100 ms only increased time.thread_time() by 19.9 ms which is
smaller than 20 ms. Modify the test to tolerate a delta of at least
15 ms instead of 20 ms.
(cherry picked from commit d6345def68)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Increase the timeout from 1 min to 5 min.
Replace also time.time() with time.monotonic() for timeouts.
(cherry picked from commit 3ad8decd76)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Change test_logging.SMTPHandlerTest timeout from 8 seconds to 1
minute. The test failed randomly on the slow x86 Gentoo Refleaks 3.7
buildbot.
(cherry picked from commit 31b50b8cbf)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
webbrowser._synthesize() called webbrowser.register() with
outdated signature.
Co-Authored-By: John Still <john@jmsdvl.com>
(cherry picked from commit 25b804a9c2)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Always return bytes from _HackedGetData.get_data().
Ensure the imp.load_source shim always returns bytes by reopening the file in
binary mode if needed. Hash-based pycs have to receive the source code in bytes.
It's tempting to change imp.get_suffixes() to always return 'rb' as a mode, but
that breaks some stdlib tests and likely 3rdparty code, too.
(cherry picked from commit b0274f2cdd)
Co-authored-by: Benjamin Peterson <benjamin@python.org>
Most of the change involves fixing up the test suite, which previously made
the assumption that there wouldn't be a new line if the input didn't end in
one.
Contributed by Ammar Askar.
(cherry picked from commit c4ef4896ea)
subprocess.Popen now copies the startupinfo argument to leave it
unchanged: it will modify the copy, so that the same STARTUPINFO
object can be used multiple times.
Add subprocess.STARTUPINFO._copy() private method.
Python 3.7 backport from master makes the copy() private: renamed to
_copy().
(cherry picked from commit 483422f57e)
Result of function interpreter_requires_environment() depends on os.environ.
This was not covered by the tests, leading to fail when PYTHONHOME was set.
(cherry picked from commit a390cb6b6f)
Co-authored-by: Lorenz Mende <Lorenz.mende@gmail.com>
Fix test_forkserver_sigkill() of test_multiprocessing_forkserver:
give more time to the first child process to complete, double the
sleep in the parent process.
Reduce also the child process sleep from 1000 ms to 500 ms, to not change
the total duration of the test.
(cherry picked from commit 07888e1cce)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
PyErr_Print() will not return when the exception is a SystemExit, so
decref the __main__ module object in that case.
(cherry picked from commit d8cba5d16f)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
The Opera Browser was using a outdated command line invocation that resulted in an incorrect URL being opened in the browser when requested using the webbrowser module.
* Correct the arguments passed to the Opera Browser when opening a new URL.
(cherry picked from commit 3cf1f154ed)
Co-authored-by: Bumsik Kim <k.bumsik@gmail.com>
When hunting memory leaks using -R 3:3, test_imap_unordered() of
test_multiprocessing leaks randomly a few memory blocks. It is a
false alarm: when testing using -R 3:20 for example, no leak is
detected.
Modify test_imap_unordered() to be closer to test_imap():
* Only test 10 numbers instead of 1000: it's a pool of 4 processes, so
10 is enough to test at least one number per process
* Use chunksize=100 instead of chunksize=53 to mimick test_imap()
(cherry picked from commit 23401fb960)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* `flags` is indeed deprecated, but there is a validation on its value for
backwards compatibility reasons. This adds mention of this in the docs.
* The docs say that `sizehint` is deprecated and ignored, but it is still
used when `epoll_create1()` is unavailable. This adds mention of this in
the docs.
* `sizehint=-1` is acceptable again, and is replaced with `FD_SETSIZE-1`.
This is needed to have a default value available at the Python level,
since `FD_SETSIZE` is not exposed to Python. (see: bpo-31938)
* Reject `sizehint=0` since it is invalid to pass on to `epoll_create()`.
The relevant tests have also been updated.
(cherry picked from commit 0cdf5f4289)
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
Fix passing lists and tuples of strings containing special characters
'"', '\\', '{', '}' and '\n' as options to tkinter.ttk widgets.
(cherry picked from commit 5bb5bbfca8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
bpo-30339, bpo-33913:
* Increase timeout from 10 seconds to 1 minute in
test_source_main_skipped_in_children source of
test_multiprocessing_main_handling.
* Replace time.time() with time.monotonic().
* On timeout, include the duration in the error message.
(cherry picked from commit 64737e9ae2)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
test_mymanager_context() now also accepts -SIGTERM as an expected
exitcode for the manager process. The process is killed with SIGTERM
if it takes longer than 1 second to stop.
(cherry picked from commit fbd7172325)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Tolerate a different of 50 ms, instead of just 30 ms, in
test_timeout() of multiprocessing tests. This change should fix such
test failure on Windows:
FAIL: test_timeout (test.test_multiprocessing_spawn.WithProcessesTestQueue)
Traceback (most recent call last):
File "lib\test\_test_multiprocessing.py", line 753, in test_timeout
self.assertGreaterEqual(delta, 0.170)
AssertionError: 0.16138982772827148 not greater than or equal to 0.17
(cherry picked from commit f15f66d275)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686)
* Enhance also format_duration(): work on integers and rounds towards
+infinity (math.ceil).
* Write unit tests on format_duration()
(cherry picked from commit 4ffe9c2b25)
* bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736)
regrtest: Add warning when using less than 3 warmup runs like -R 1:3.
(cherry picked from commit cac4fef886)
* bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735)
Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs.
Add check for invalid --huntrleaks/-R parameters.
(cherry picked from commit 58ed7307ea)
Calling Py_Initialize() twice does nothing, instead of failing with a
fatal error: restore the Python 3.6 behaviour.
(cherry picked from commit 209abf7469)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
with debuglevel=1 only the header keys got printed. With
this change the header values get printed as well and the single
header entries get '\n' as a separator.
(cherry picked from commit 936f03e7fa)
Co-authored-by: Marco Strigl <mstrigl@suse.com>
* Test exists(), lexists(), isdir(), isfile(), islink(), ismount()
with bytes paths.
* Remove unneeded silencing DeprecationWarning for ismount() with
bytes path.
* Test common functions with unencodable and undecodable paths.
* Minor clean up and refactoring.
(cherry picked from commit 17a0088e26)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-32962: python-gdb catchs ValueError on read_var() (GH-7692)
python-gdb now catchs ValueError on read_var(): when Python has no
debug symbols for example.
(cherry picked from commit 019d33b7a4)
* bpo-32962: python-gdb catchs UnicodeDecodeError (GH-7693)
python-gdb now catchs UnicodeDecodeError exceptions when calling
string().
(cherry picked from commit d22fc0bc7d)
* bpo-32962: Fix test_gdb failure in debug build with -mcet -fcf-protection -O0 (GH-6754)
When Python is built with the intel control-flow protection flags,
-mcet -fcf-protection, gdb is not able to read the stack without
actually jumping inside the function. This means an extra
'next' command is required to make the $pc (program counter)
enter the function and make the stack of the function exposed to gdb.
(cherry picked from commit 9b7c74ca32)
Hangul composition check boundaries are wrong for the second character
([0x1161, 0x1176) instead of [0x1161, 0x1176]) and third character ((0x11A7, 0x11C3)
instead of [0x11A7, 0x11C3]).
(cherry picked from commit d134809cd3)
Co-authored-by: Wonsup Yoon <pusnow@me.com>
When attempting to base64-decode a payload of invalid length (1 mod 4),
properly recognize and handle it. The given data will be returned as-is,
i.e. not decoded, along with a new defect, InvalidBase64LengthDefect.
(cherry picked from commit c3f55be7dd)
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
The documentation for CERT_NONE, CERT_OPTIONAL, and CERT_REQUIRED were
misleading and partly wrong. It fails to explain that OpenSSL behaves
differently in client and server mode. Also OpenSSL does validate the
cert chain everytime. With SSL_VERIFY_NONE a validation error is not
fatal in client mode and does not request a client cert in server mode.
Also discourage people from using CERT_OPTIONAL in client mode.
(cherry picked from commit ef24b6c54d)
Co-authored-by: Christian Heimes <christian@python.org>
A datetime object d is aware if d.tzinfo is not None and
d.tzinfo.utcoffset(d) does not return None. If d.tzinfo is None,
or if d.tzinfo is not None but d.tzinfo.utcoffset(d) returns None,
d is naive.
This commit ensures that instances with non-None d.tzinfo, but
d.tzinfo.utcoffset(d) returning None are treated as naive.
In addition, C acceleration code will raise TypeError if
d.tzinfo.utcoffset(d) returns an object with the type other than
timedelta.
* Updated the documentation.
Assume that the term "naive" is defined elsewhere and remove the
not entirely correct clarification. Thanks, Tim.
(cherry picked from commit 877b23202b)
Co-authored-by: Alexander Belopolsky <abalkin@users.noreply.github.com>
If tests are re-run, use "xxx then yyy" result format (ex: "FAILURE
then SUCCESS") to show that some failing tests have been re-run.
Add also test_regrtest.test_rerun_fail() test.
(cherry picked from commit c45fc7673e)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* Added a test case for strftime("%z").
The added test checks a case with UTC offest expressed in an integer
number of seconds.
* Added a test comparing naive and aware datetimes.
Check that a greater than comparison of a naive datetime instance with
an aware one raises a TypeError.
* Test datetime in fold or in gap comparison both ways.
(cherry picked from commit 4c3e39f61c)
Co-authored-by: Alexander Belopolsky <abalkin@users.noreply.github.com>
This solves a regression in logging config due to changes in BPO-23835.
(cherry picked from commit 214f18e49f)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
- bugfix and test for fragile metavar handling in argparse (see
bpo-24089, bpo-14046, bpo-25058, bpo-11874)
- also fixes some incorrect tests that did not make 1-element tuples correctly
(cherry picked from commit 66f02aa32f)
Co-authored-by: wim glenn <wim.glenn@gmail.com>
_feed_data_to_bufferred_proto() renamed to
_feed_data_to_buffered_proto() ("bufferred" => "buffered").
Typo spotted by Nathaniel J. Smith.
(cherry picked from commit ff6c077292)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
The cancellation of an overlapped WSARecv() has a race condition
which causes data loss because of the current implementation of
proactor in asyncio.
No longer cancel overlapped WSARecv() in _ProactorReadPipeTransport
to work around the race condition.
Remove the optimized recv_into() implementation to get simple
implementation of pause_reading() using the single _pending_data
attribute.
Move _feed_data_to_bufferred_proto() to protocols.py.
Remove set_protocol() method which became useless.
(cherry picked from commit 79790bc35f)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
It wasn't testing functionality. Now it is (on Linux anyways).
(cherry picked from commit 5f3d04fa4e)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* bpo-33789: test_asyncio: Fix ResourceWarning (GH-7460)
* Close sockets and streams to fix ResourceWarning warnings
* Catch also OSError to hide a traceback on an expected handshake
error
(cherry picked from commit 0eba7c3913)
* bpo-33789, test_asyncio: Hide PendingDeprecationWarning (GH-7461)
Hide PendingDeprecationWarning in test__register_task_3().
(cherry picked from commit 7ed61e9431)
* bpo-32676, test_asyncio: Fix warning in test_error_in_call_soon() (GH-7462)
Fix "<CoroWrapper ...> was never yielded from" warning in
PyTask_PyFuture_Tests.test_error_in_call_soon() of
test_asyncio.test_tasks.
Close manually the coroutine on error.
(cherry picked from commit 9f04f0df6f)
Substract one because listdir() opens internally a file
descriptor to list the content of the /proc/self/fd/ directory.
Add test_support.test_fd_count().
Move also MAXFD code before msvcrt.CrtSetReportMode(), to make sure
that the report mode is always restored on failure.
(cherry picked from commit 492d6424a7)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
testModeStrings and testTruncateOnWindows were depended on
a file leaked in other tests.
Also improve cleaning up after tests.
(cherry picked from commit c2745d2d05)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
With addCleanup() f.close() was executed after tearDown().
(cherry picked from commit 6592d7fe11)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Fix test_warnings.test_module_globals() when python3 is run with
-Werror.
(cherry picked from commit e292b75e3e)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
* Fix AttributeError (not all SSL exceptions have 'errno' attribute)
* Increase default handshake timeout from 10 to 60 seconds
* Make sure start_tls can be cancelled correctly
* Make sure any error in SSLProtocol gets propagated (instead of just being logged)
(cherry picked from commit 9602643120)
Co-authored-by: Yury Selivanov <yury@magic.io>
Separate tests leaked files or were depended on files leaked in other tests.
(cherry picked from commit 027f95c736)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Use also support.SOCK_MAX_SIZE, not only support.PIPE_MAX_SIZE, to
get the size for a blocking send into a multiprocessing pipe.
(cherry picked from commit 252f6abe0a)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Fix test_ignore() of multiprocessing tests like
test_multiprocessing_forkserver: use support.PIPE_MAX_SIZE to make
sure that send_bytes() blocks.
(cherry picked from commit 5d6c7ed5e3)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Currently, asyncio.wait_for(fut), upon reaching the timeout deadline,
cancels the future and returns immediately. This is problematic for
when *fut* is a Task, because it will be left running for an arbitrary
amount of time. This behavior is iself surprising and may lead to
related bugs such as the one described in bpo-33638:
condition = asyncio.Condition()
async with condition:
await asyncio.wait_for(condition.wait(), timeout=0.5)
Currently, instead of raising a TimeoutError, the above code will fail
with `RuntimeError: cannot wait on un-acquired lock`, because
`__aexit__` is reached _before_ `condition.wait()` finishes its
cancellation and re-acquires the condition lock.
To resolve this, make `wait_for` await for the task cancellation.
The tradeoff here is that the `timeout` promise may be broken if the
task decides to handle its cancellation in a slow way. This represents
a behavior change and should probably not be back-patched to 3.6 and
earlier.
(cherry picked from commit e2b340ab41)
Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
Use transport.set_write_buffer_limits() in sendfile tests of
test_asyncio to make sure that the protocol is paused after sending
4 KiB. Previously,
test_sendfile_fallback_close_peer_in_the_middle_of_receiving() failed
on FreeBSD if the DATA was smaller than the default limit of 64 KiB.
(cherry picked from commit 9551f77192)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Remove the docstring attribute of AST types and restore docstring
expression as a first stmt in their body.
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
In this commit:
* Support BufferedProtocol in set_protocol() and start_tls()
* Fix proactor to cancel readers reliably
* Update tests to be compatible with OpenSSL 1.1.1
* Clarify BufferedProtocol docs
* Bump TLS tests timeouts to 60 seconds; eliminate possible race from start_serving
* Rewrite test_start_tls_server_1
(cherry picked from commit dbf102271f)
Co-authored-by: Yury Selivanov <yury@magic.io>
Fixed bug where calling write_eof() on a _SelectorSocketTransport after
it's already closed raises AttributeError.
(cherry picked from commit 23f587e395)
Co-authored-by: twisteroid ambassador <twisteroidambassador@users.noreply.github.com>
The failure may be due to the use oF ZFS, a case we already ignore
for Solaris-based systems where ZFS is frequently used.
(cherry picked from commit 09c4a7dee2)
Co-authored-by: Ned Deily <nad@python.org>
bpo-32374, bpo-33629: Use support.SuppressCrashReport() in
test_bad_traverse() of MultiPhaseExtensionModuleTests to prevent
leaking a core dump file.
(cherry picked from commit 483000e164)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
bpo-26510 in 3.7.0a2 changed the behavior of argparse to make
subparsers required by default, returning to the behavior of 2.7
and 3.2. The behavior was changed in 3.3 to be no longer required.
While it might make more sense to have the default to required,
compatibility with 3.3 through 3.6 is probably less disruptive
than trying to reintroduce compatibility with 2.7 at this point.
This change restores the 3.6 behavior.
(cherry picked from commit 8ebf5ceb0f)
Co-authored-by: Ned Deily <nad@python.org>
* bpo-33353: test_asyncio uses smaller sendfile data (#7083)
bpo-32622, bpo-33353: sendfile() tests of test_asyncio use socket
buffers of 1 kB "to test on relative small data sets". Send only
160 KiB rather 10 MB to make the test much faster.
Shrink also SendfileBase.DATA from 1600 KiB to 160 KiB.
On Linux, 3 test_sock_sendfile_mix_with_regular_send() runs now take
less than 1 second, instead of 18 seconds.
On FreeBSD, the 3 tests didn't hang, but took 3 minutes. Now
the 3 tests pass in less than 1 seconds.
(cherry picked from commit 2932755cc1)
* bpo-33353: test_asyncio set SO_SNDBUF after connect (GH-7086)
bpo-32622, bpo-33353: On macOS, sock.connect() changes the
SO_SNDBUF value. Only set SO_SNDBUF and SO_RCVBUF buffer sizes
once a socket is connected or binded, not before.
(cherry picked from commit b97de3dd86)
TLS 1.3 behaves slightly different than TLS 1.2. Session tickets and TLS
client cert auth are now handled after the initialy handshake. Tests now
either send/recv data to trigger session and client certs. Or tests
ignore ConnectionResetError / BrokenPipeError on the server side to
handle clients that force-close the socket fd.
To test TLS 1.3, OpenSSL 1.1.1-pre7-dev (git master + OpenSSL PR
https://github.com/openssl/openssl/pull/6340) is required.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 529525fb5a)
bpo-33604: Bump removal notice from 3.6 to 3.8 and change PendingDeprecationWarning to DeprecationWarning as we had intended to do earlier...
(cherry picked from commit 8bb0b5b03c)
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
Change TLS 1.3 cipher suite settings for compatibility with OpenSSL
1.1.1-pre6 and newer. OpenSSL 1.1.1 will have TLS 1.3 cipers enabled by
default.
Also update multissltests and Travis config to test with latest OpenSSL.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit e8eb6cb792)
Co-authored-by: Christian Heimes <christian@python.org>
* bpo-33263 Fix FD leak in _SelectorSocketTransport. (GH-6450)
Under particular circumstances _SelectorSocketTransport can try to add a reader
even the transport is already being closed. This can lead to FD leak and
invalid stated of the following connections. Fixed the SelectorSocketTransport
to add the reader only if the trasport is still active.
(cherry picked from commit a84d0b361a)
Co-authored-by: Vlad Starostin <drtyrsa@yandex.ru>
The proactor event loop has a race condition when reading with
pausing/resuming. `resume_reading()` unconditionally schedules the read
function to read from the current future. If `resume_reading()` was
called before the previously scheduled done callback fires, this results
in two attempts to get the data from the most recent read and an
assertion failure. This commit tracks whether or not `resume_reading`
needs to reschedule the callback to restart the loop, preventing a
second attempt to read the data.
(cherry picked from commit 4151061855)
Co-authored-by: CtrlZvi <viz+github@flippedperspective.com>
`obj.async` is now a syntax error, so the warning/shim is
quasi-unnecessary.
(cherry picked from commit f90f5d5c1d)
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
Don't crash on warnings.warn_explicit() if module_globals is not a dict.
(cherry picked from commit b056562860)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Also bump PendingDeprecationWarning to DeprecationWarning.
(cherry picked from commit 9eb40bc38d)
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
The urllib.robotparser's __str__ representation now includes wildcard
entries and the "Crawl-delay" and "Request-rate" fields.
(cherry picked from commit bd08a0af2d)
Co-authored-by: Michael Lazar <lazar.michael22@gmail.com>
This happens in the NixOS build sandbox, for example, where the only
other user is nobody with home directory /.
(cherry picked from commit 5c0d462689)
Co-authored-by: Anders Kaseorg <andersk@mit.edu>
Pass os.environ's copy to new process created at test_posix:
test_specify_environment. Otherwise important variables such as
LD_LIBRARY_PATH are not set and the child process might not work at all
in an environment where such variables are required for Python to function.
(cherry picked from commit 7ec8f28656)
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Previously emitted cover files only when --missing option was used.
(cherry picked from commit 47ab15470d)
Co-authored-by: Michael Selik <mike@selik.org>
Fix test_embed.test_pre_initialization_sys_options() when building with --enable-shared
(cherry picked from commit 4114846265)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
External importers were being added in both phases of the import
system initialisation.
They're only supposed to be added in the second phase, after the
import machinery has been appropriately configured.
(cherry picked from commit 0977091dca)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
as docstrings and translatable strings, and rejects
bytes literals and f-string expressions.
(cherry picked from commit 69524821a8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
`test_urllib2net.OtherNetworkTests.test_sites_no_connection_close`
used `http://www.imdb.com/` but it is moved to https so the test is
not valid anymore. Skip test for the moment to allow CI to proceed.
(cherry picked from commit 36d56ea826)
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
Adds some working and markup fixes that I missed
in the initial commit for this issue.
(Follow-up to GH-6419)
(cherry picked from commit 1a5c4bdb6e)
Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
The pydoc CLI assumed -m pydoc would add the empty string
to sys.path, and hence got confused when it switched to
adding the full initial working directory instead.
This refactors the pydoc CLI path manipulation to be
more testable, and ensures it won't accidentally
remove the standard library directory containing
pydoc itself from sys.path.
(cherry picked from commit 82a9481059)
Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
Executors in concurrent.futures accepted tasks after executor was shutdown by interpreter exit. Tasks were left in PENDING state forever. This fix changes submit to instead raise a RuntimeError.
(cherry picked from commit c4b695f85e)
Co-authored-by: Mark Nemec <mrknmc@me.com>
An entry of None in sys.path_importer_cache represents a negative/missing finder for a path, so clearing it out makes sense.
(cherry picked from commit 9e2be60634)
* Add a test for frozen with unhashable field value.
* Improve a comment.
(cherry picked from commit 74940913d2)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
This removes a hack and replaces it with a proper
mapping {'list': 'List', 'dict': 'Dict', ...}.
(cherry picked from commit 2a363d2930)
Co-authored-by: Ivan Levkivskyi <levkivskyi@gmail.com>
Different paths are executed for normal exit and for leaving
the 'async with' block with 'break', 'continue' or 'return'.
(cherry picked from commit 2eeac269dd)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
This also fixes python/typingGH-512
This also fixes python/typingGH-511
As was discussed in both issues, some typing forms deserve to be treated
as immutable by copy and pickle modules, so that:
* copy(X) is X
* deepcopy(X) is X
* loads(dumps(X)) is X GH- pickled by reference
This PR adds such behaviour to:
* Type variables
* Special forms like Union, Any, ClassVar
* Unsubscripted generic aliases to containers like List, Mapping, Iterable
This not only resolves inconsistencies mentioned in the issues, but also
improves backwards compatibility with previous versions of Python
(including 3.6).
Note that this requires some dances with __module__ for type variables
(similar to NamedTuple) because the class TypeVar itself is define in typing,
while type variables should get module where they were defined.
https://bugs.python.org/issue32873
(cherry picked from commit 834940375a)
Co-authored-by: Ivan Levkivskyi <levkivskyi@gmail.com>
bpo-32844: subprocess: Fix a potential misredirection of a low fd to stderr.
When redirecting, subprocess attempts to achieve the following state:
each fd to be redirected to is less than or equal to the fd
it is redirected from, which is necessary because redirection
occurs in the ascending order of destination descriptors.
It fails to do so in a couple of corner cases,
for example, if 1 is redirected to 2 and 0 is closed in the parent.
(cherry picked from commit 0e7144b064)
Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
This is part of PEP 487 and the descriptor protocol.
(cherry picked from commit de7a2f04d6)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Also, re-enable a test for ClassVars with default_factory.
(cherry picked from commit 2b75fc2bc9)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Historically, -m added the empty string as sys.path
zero, meaning it resolved imports against the current
working directory, the same way -c and the interactive
prompt do.
This changes the sys.path initialisation to add the
*starting* working directory as sys.path[0] instead,
such that changes to the working directory while the
program is running will have no effect on imports
when using the -m switch.
(cherry picked from commit d5d9e02dd3)
Harden ssl module against LibreSSL CVE-2018-8970.
X509_VERIFY_PARAM_set1_host() is called with an explicit namelen. A new test
ensures that NULL bytes are not allowed.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit d02ac25ab0)
Co-authored-by: Christian Heimes <christian@python.org>
- new test case for pre-initialization of sys.warnoptions and sys._xoptions
- restored ability to call these APIs prior to Py_Initialize
- updated the docs for the affected APIs to make it clear they can be
called before Py_Initialize
- also enhanced the existing embedding test cases
to check for expected settings in the sys module
(cherry picked from commit bc77eff8b9)
Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
If a dataclass has a member variable that's of type Field, but it doesn't have a type annotation, raise TypeError.
(cherry picked from commit 56970b8ce9)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
- Add missing 'Field' to __all__.
- Improve tests to catch this.
(cherry picked from commit 8e4560a9da)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Fix the way that new annotations in a class are detected.
(cherry picked from commit 8f6eccdc64)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
The result of host() was not empty when the network is constructed by a tuple containing an
integer mask and only 1 bit left for addresses.
(cherry picked from commit 10b134a07c)
Co-authored-by: Xiang Zhang <angwerzx@126.com>
If the class has a member that's a MemberDescriptorType, it's not a default value, it's from that member being in __slots__.
(cherry picked from commit 7389fd935c)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
If a non-dataclass derives from a frozen dataclass, allow attributes to be set.
Require either all of the dataclasses in a class hierarchy to be frozen, or all non-frozen.
Store `@dataclass` parameters on the class object under `__dataclass_params__`. This is needed to detect frozen base classes.
(cherry picked from commit f199bc655e)
Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
Multi-phase initialized modules allow m_traverse to be called while the
module is still being initialized, so module authors may need to account
for that.
(cherry picked from commit c2b0b12d1a)
Co-authored-by: Marcel Plch <gmarcel.plch@gmail.com>
In some conditions the standard streams will be None or closed in the child process (for example if using "pythonw" instead of "python" on Windows). Avoid failing with a non-0 exit code in those conditions.
Report and initial patch by poxthegreat.
(cherry picked from commit e756f66c83)
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
* Skip write()/data_received() if sslpipe is destroyed
(cherry picked from commit 5e80a71ab6)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
test_asyncio hangs indefinitely on macOS 10.13.2+ on `read_pty_output()`
using the KqueueSelector. Closing `proto.transport` (as is done in
`write_pty_output()`) seems to fix it.
(cherry picked from commit 12f74d8608)
Co-authored-by: Nathan Henrie <n8henrie@users.noreply.github.com>
Zackery Spytz
Miss Islington (bot)
Serhiy Storchaka
Victor Stinner
Ammar Askar
Xtreak
Ned Deily
Xiang Zhang
Yury Selivanov
Eric Snow
Pablo Galindo
Zsolt Dollenstein
Ethan Furman
Brett Cannon
Nick Coghlan
xdegaye