For 3.7.1rc1 and 3.6.7rc1 we used a pre-release development
snapshot of Tk 8.6 to pick up some post-8.6.8 fixes for macOS.
But the snapshot introduced at least one regression (bpo-34927).
For rc2, revert to using the standard release 8.6.8 for now.
This reverts commit d9cfe5ed2c.
When Python is built with the intel control-flow protection flags,
-mcet -fcf-protection, gdb is not able to read the stack without
actually jumping inside the function. This means an extra
'next' command is required to make the $pc (program counter)
enter the function and make the stack of the function exposed to gdb.
Co-Authored-By: Marcel Plch <gmarcel.plch@gmail.com>
(cherry picked from commit 9b7c74ca32)
(cherry picked from commit 79d21331e6)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Let .chm document display non-ASCII characters properly
Escape the `body` part of .chm source file to 7-bit ASCII, to fix visual effect on some MBCS Windows systems.
(cherry picked from commit 6261ae9b01)
Co-authored-by: animalize <animalize@users.noreply.github.com>
bpo-6721: When os.fork() was called while another thread holds a logging lock, the child process may deadlock when it tries to log. This fixes that by acquiring all logging locks before fork and releasing them afterwards.
A regression test that fails before this change is included.
Within the new unittest itself: There is a small _potential_ due to mixing of fork and a thread in the child process if the parent's thread happened to hold a non-reentrant library call lock (malloc?) when the os.fork() happens. buildbots and time will tell if this actually manifests itself in this test or not. :/ A functionality test that avoids that would be a challenge.
An alternate test that isn't trying to produce the deadlock itself but just checking that the release and acquire calls are made would be the next best alternative if so.
(cherry picked from commit 19003841e9)
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google]
The C implementation of asyncio.Task currently fails to perform the
cancellation cleanup correctly in the following scenario.
async def task1():
async def task2():
await task3 # task3 is never cancelled
asyncio.current_task().cancel()
await asyncio.create_task(task2())
The actuall error is a hardcoded call to `future_cancel()` instead of
calling the `cancel()` method of a future-like object.
Thanks to Vladimir Matveev for noticing the code discrepancy and to
Yury Selivanov for coming up with a pathological scenario..
(cherry picked from commit 548ce9dedd)
Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
https://bugs.python.org/issue34872
formatfloat() was not checking if PyBytes_FromStringAndSize()
failed, which could lead to a null pointer dereference in
_PyBytes_FormatEx().
(cherry picked from commit 96c5932794)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly.
(cherry picked from commit 97bfe8d3eb)
Co-authored-by: tzickel <tzickel@users.noreply.github.com>
* Compiling a string annotation containing a lambda with keyword-only
argument without default value caused a crash.
* Remove the final "*" (it is incorrect syntax) in the representation of
lambda without *args and keyword-only arguments when compile from AST.
* Improve the representation of lambda without arguments.
(cherry picked from commit 2a2940e5c3)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Use a monotonic clock to compute timeouts in :meth:`Executor.map` and :func:`as_completed`, in order to prevent timeouts from deviating when the system clock is adjusted.
This may not be sufficient on all systems. On POSIX for example, the actual waiting (e.g. in ``sem_timedwait``) is specified to rely on the CLOCK_REALTIME clock.
(cherry picked from commit a94ee12c26)
Co-authored-by: orlnub123 <orlnub123@gmail.com>
Have macOS 10.9+ installer builds for 3.7.1rc and 3.6.7rc use a development
snapshot of Tk 8.6 (post-8.6.8) to mitigate certain scroller issues seen
with IDLE and tkinter apps.
When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)`
should use it instead of dict ordering.
https://bugs.python.org/issue34320
(cherry picked from commit 2aaf98c16a)
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
The GlobalLock() call in UpdateDropDescription() was not checked for
failure.
https://bugs.python.org/issue34770
(cherry picked from commit f6c8007a29)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
The SAX parser no longer processes general external entities by default
to increase security. Before, the parser created network connections
to fetch remote files or loaded local files from the file system for DTD
and entities.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue17239.
(cherry picked from commit 17b1d5d4e3)
Co-authored-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue17239
Fix a crash with musl libc (on Alpine Linux) when the script filename
specified on the command line doesn't exist. pymain_open_filename()
now gets the current core configuration from the interpreter state.
Modify the code to make it closer to the master branch:
* Rename _Py_CommandLineDetails to _PyCmdline
* Remove _PyMain.config: replaced with a local variable
'local_config' in pymain_init()
* Reorganize pymain_main(): move code using the "local config"
into pymain_init()
* As soon as possible, switch from the local config to the core
configuration attached to the interpreter.
Add SSLContext.post_handshake_auth and
SSLSocket.verify_client_post_handshake for TLS 1.3 post-handshake
authentication.
Signed-off-by: Christian Heimes <christian@python.org>q
https://bugs.python.org/issue34670.
(cherry picked from commit 9fb051f032)
Co-authored-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue34670
This makes streamed zips compatible with MacOS Archive Utility and
other applications.
(cherry picked from commit 4ba3b50bfe)
Co-authored-by: Silas Sewell <silas@sewell.org>
We cannot simply call locale.getpreferredencoding() here,
as GDB might have been linked against a different version
of Python with a different encoding and coercion policy
with respect to PEP 538 and PEP 540.
Thanks to Victor Stinner for a hint on how to fix this.
(cherry picked from commit 7279b5125e)
Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
The C accelerated _elementtree module now initializes hash randomization
salt from _Py_HashSecret instead of libexpat's default CPRNG.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue34623
(cherry picked from commit cb5778f00c)
Co-authored-by: Christian Heimes <christian@python.org>
* bpo-34589: Make _PyCoreConfig.coerce_c_locale private (GH-9371)
_PyCoreConfig:
* Rename coerce_c_locale to _coerce_c_locale
* Rename coerce_c_locale_warn to _coerce_c_locale_warn
These fields are now private (name prefixed by "_").
(cherry picked from commit 188ebfa475)
* bpo-34589: C locale coercion off by default (GH-9073)
Py_Initialize() and Py_Main() cannot enable the C locale coercion
(PEP 538) anymore: it is always disabled. It can now only be enabled
by the Python program ("python3).
test_embed: get_filesystem_encoding() doesn't have to set PYTHONUTF8
nor PYTHONCOERCECLOCALE, these variables are already set in the
parent.
(cherry picked from commit 7a0791b699)
* bpo-34589: Add -X coerce_c_locale command line option (GH-9378)
Add a new -X coerce_c_locale command line option to control C locale
coercion (PEP 538).
(cherry picked from commit dbdee0073c)
The test tries to fill the receiver's socket buffer and expects an
error. But the RDS protocol doesn't require that. Moreover, the Linux
implementation of RDS expects that the producer of the messages
reduces its rate, it's not the role of the receiver to trigger an
error.
The test fails on Fedora 28 by design, so remove it.
(cherry picked from commit 7484bdfd1e)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Include ``openssl/dh.h`` header file to fix implicit function declaration of ``DH_free()``.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
(cherry picked from commit b3a271fc0c)
Co-authored-by: Alexandru Ardelean <ardeleanalex@gmail.com>
* [3.7] Fix test_asyncio for AIX - do not call transport.get_extra_info('sockname') (GH-8907).
(cherry picked from commit 413118ebf3)
Co-authored-by: Michael Felt <aixtools@users.noreply.github.com>
* Update test_events.py
[bpo-34658](https://www.bugs.python.org/issue34658): Fix a rare interpreter unhandled exception state SystemError only
seen when using subprocess with a preexec_fn while an after_parent handler has
been registered with os.register_at_fork and the fork system call fails.
https://bugs.python.org/issue34658
(cherry picked from commit a20b6adb5a)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
This causes the tearDown code to only unimport the test modules specifically created as part of each test via the self.mkhier method rather than abusing test.support.modules_setup() and the scary test.support.modules_cleanup() code.
https://bugs.python.org/issue34200
(cherry picked from commit 4ae8ece5cd)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
When subprocess.Popen() stdin= stdout= or stderr= handles are specified
and appear in pass_fds=, don't close the original fds after dup'ing them.
This implementation and unittest primarily came from @izbyshev (see the PR)
See also b89b52f284
This also removes the old manual p2cread, c2pwrite, and errwrite closing logic
as inheritable flags and _close_open_fds takes care of that properly today without special treatment.
This code is within child_exec() where it is the only thread so there is no
race condition between the dup and _Py_set_inheritable_async_safe call.
(cherry picked from commit ce34410b8b)
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google]
Remove ellipsis that look like continuation prompts,
has a side benefit of putting rest of error message in proper text color.
(cherry picked from commit f019579828)
Co-authored-by: Lew Kurtz <37632626+lew18@users.noreply.github.com>
The recursive frame pruning code always undercounted the number of elided frames
by one. That is, in the "[Previous line repeated N more times]" message, N would
always be one too few. Near the recursive pruning cutoff, one frame could be
silently dropped. That situation is demonstrated in the OP of the bug report.
The fix is to start the identical frame counter at 1.
(cherry picked from commit d545869d08)
Co-authored-by: Benjamin Peterson <benjamin@python.org>
Pass the user/group name as Unicode to the formatting function,
instead of always decoding a bytes string from UTF-8..
(cherry picked from commit 28658485a5)
Co-authored-by: William Grzybowski <wg@FreeBSD.org>
This caused installation errors in some cases on Windows.
Patch by Julien Malard.
(cherry picked from commit 0afada163c)
Co-authored-by: Julien Malard <julien.malard@mail.mcgill.ca>
Some methods of the SMTP class use mutable default arguments. Specially
`send_message` is affected as it mutates one of the args by appending items
to it, which has side effects on further calls.
(cherry picked from commit d5fbe9b1a3)
Co-authored-by: Pablo Aguiar <scorphus@gmail.com>
When calling tp_descr_get(self, obj, type), make sure that
we own a strong reference to "self".
(cherry picked from commit 8f735485ac)
Co-authored-by: jdemeyer <jdemeyer@cage.ugent.be>
Update all test certs and keys to use future proof crypto settings:
* 3072 bit RSA keys
* SHA-256 signature
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit e6dac00779)
Fix for invalid assert on big output of multiprocessing.Process.
(cherry picked from commit 266f4904a2)
Co-authored-by: Alexander Buchkovsky <olex.buchkovsky@gmail.com>
distutils.spawn.find_executable() now falls back on os.defpath if the
PATH environment variable is not set.
(cherry picked from commit 39487196c8)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Added previously missing "--list" argument.
Made "--list" and "--list-paths" behavior consistent with the corresponding "-0" and "-0p" arguments.
(cherry picked from commit aada63b20e)
Co-authored-by: Brendan Gerrity <brerrity@gmail.com>
* bpo-34485: stdout uses surrogateescape on POSIX locale (GH-8986)
Standard streams like sys.stdout now use the "surrogateescape" error
handler, instead of "strict", on the POSIX locale (when the C locale is not
coerced and the UTF-8 Mode is disabled).
Add tests on sys.stdout.errors with LC_ALL=POSIX.
Fix the error handler of standard streams like sys.stdout:
PYTHONIOENCODING=":" is now ignored instead of setting the error handler to
"strict".
(cherry picked from commit 315877dc36)
* The UTF-8 Mode is now also enabled by the "POSIX" locale, not only
by the "C" locale.
* On FreeBSD, Py_DecodeLocale() and Py_EncodeLocale() now also forces
the ASCII encoding if the LC_CTYPE locale is "POSIX", not only if
the LC_CTYPE locale is "C".
* test_utf8_mode.test_cmd_line() checks also that the command line
arguments are decoded from UTF-8 when the the UTF-8 Mode is enabled
with POSIX locale or C locale.
(cherry picked from commit 5cb258950c)
An old apparent AIX behavior workaround in test_subprocess's
test_undecodable_env is no longer needed.
(cherry picked from commit 89d79b1449)
Co-authored-by: Michael Felt <aixtools@users.noreply.github.com>
Avoids an integer underflow in the time module's year handling code.
(cherry picked from commit 76be0fffff)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
The current C implementations **crash** if the input includes a surrogate
Unicode code point, which is not possible to encode in UTF-8.
Important notes:
1. It is possible to pass a non-UTF-8 string as a separator to the
`.isoformat()` methods.
2. The pure-Python `datetime.fromisoformat()` implementation accepts
strings with a surrogate as the separator.
In `datetime.fromisoformat()`, in the special case of non-UTF-8 separators,
this implementation will take a performance hit by making a copy of the
input string and replacing the separator with 'T'.
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Co-authored-by: Paul Ganssle <paul@ganssle.io>
(cherry picked from commit 096329f0b2)
Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
The missing NULL check was reported by Svace static analyzer.
(cherry picked from commit cdbf50cba1)
Co-authored-by: Alexey Izbyshev <izbyshev@ispras.ru>
Read from data socket to avoid "[SSL] shutdown while in init" exception
during shutdown of the dummy server.
Signed-off-by: Christian Heimes <christian@python.org>
<!-- issue-number: [bpo-34391](https://www.bugs.python.org/issue34391) -->
https://bugs.python.org/issue34391
<!-- /issue-number -->
(cherry picked from commit 1590c39336)
Co-authored-by: Christian Heimes <christian@python.org>
Downstream vendors have started to deprecate weak keys. Update all RSA keys
and DH params to use at least 2048 bits.
Finite field DH param file use RFC 7919 values, generated with
certtool --get-dh-params --sec-param=high
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 88bfd0bce0)
Co-authored-by: Christian Heimes <christian@python.org>
Valgrind isn't seeing PyObject_Free/PyObject_Realloc, but using _PyObject_Free/_PyObject_Realloc works.
(cherry picked from commit db6075ab3a)
Co-authored-by: Paul Price <price@astro.princeton.edu>
* -X dev: it is now possible to override the memory allocator using
PYTHONMALLOC even if the developer mode is enabled.
* Add _Py_InitializeFromConfig()
* Add _Py_Initialize_ReadEnvVars() to set global configuration
variables from environment variables
* Fix the code to initialize Python: Py_Initialize() now also reads
environment variables
* _Py_InitializeCore() can now be called twice: the second call
only replaces the configuration.
* Write unit tests on Py_Initialize() and the different ways to
configure Python
* The isolated mode now always sets Py_IgnoreEnvironmentFlag and
Py_NoUserSiteDirectory to 1.
* pymain_read_conf() now saves/restores the configuration
if the encoding changed
* make CallTip and ToolTip sub-classes of a common abstract base class
* remove ListboxToolTip (unused and ugly)
* greatly increase test coverage
* tested on Windows, Linux and macOS
(cherry picked from commit 87e59ac11e)
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
Some MacOS-tk combinations need .update_idletasks().
The call is both unneeded and innocuous on Linux and Windows.
Patch by Kevin Waltzer.
(cherry picked from commit 9beaef6225)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Ned Deily
Miss Islington (bot)
Serhiy Storchaka
Stéphane Wirtel
Elvis Pranskevichus
Christian Heimes
Victor Stinner
Ethan Furman
Andrew Svetlov
Benjamin Peterson
orlnub123
Tal Einat
William Grzybowski
Zackery Spytz