Replace tkinter tkSimpleDialog.askinteger with a standard IDLE query dialog.
The new box checks for positivity before returning.
(cherry picked from commit 363fab83b8)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Move required blanking of error text to non-overridden entry_ok().
(Omit news item.)
(cherry picked from commit e53a3932cb)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Since clicking to get an IDLE context menu moves the cursor,
any text selection should be and now is cleared.
(cherry picked from commit 4ca060d8ad)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
It appears standard that moving the text insert cursor away from a selection clears the
selection. Clearing prevents accidental deletion of a possibly off-screen bit of text.
The update is for Ln and Col on the status bar.
(cherry picked from commit 2522db11df)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
`list(sys.modules.items())` was apparently not immune to "dictionary
changed size during iteration" errors.
Tested internally using an integration test that has run into this a couple of times in the past two years. With this patch applied, the test is no longer flaky.
(cherry picked from commit 85cf1d514b)
Co-authored-by: Gregory P. Smith <gps@google.com>
Objects do not own weak references to them directly through the __weakref__ list so these
do not need to be traversed by the GC.
(cherry picked from commit 0c2b509)
Open issue in the BPO indicated a desire to make the implementation of
codecs.open() at parity with io.open(), which implements a try/except to
assure file stream gets closed before an exception is raised.
(cherry picked from commit 2565edec2c)
Co-authored-by: Chris A <christopher.aporta@gmail.com>
Fix compileall.compile_dir() ddir= behavior on sub-packages.
Fixes compileall.compile_dir's ddir parameter and compileall command
line flag `-d` to no longer write the wrong pathname to the generated
pyc file for submodules beneath the root of the directory tree being
compiled. This fixes a regression introduced with Python 3.5.
Tests backported from GH 02673352b5, the
implementation is different due to intervening code changes. But still
quiet simple.
Why was the bug ever introduced? The refactoring to add parallel
execution kept the ddir -> dfile computations but discarded the results
instead of sending them to compile_file(). This fixes that. Lack of tests
meant this went unnoticed.
* bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest authentication
- The 'qop' value in the 'WWW-Authenticate' header is optional. The
presence of 'qop' in the header should be checked before its value
is parsed with 'split'.
Signed-off-by: Stephen Balousek <stephen@balousek.net>
* bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest authentication
- Add NEWS item
Signed-off-by: Stephen Balousek <stephen@balousek.net>
* Update Misc/NEWS.d/next/Library/2020-02-06-05-33-52.bpo-39548.DF4FFe.rst
Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
(cherry picked from commit 5e260e0fde)
Co-authored-by: Stephen Balousek <sbalousek@users.noreply.github.com>
Previously, the button-up part of selecting with a mouse was treated as a click
that meant 'jump' to this line, which modified the context and undid the selection
(cherry picked from commit c705fd1e89)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Trying to decode an invalid string with the punycode codec
shoud raise UnicodeError.
(cherry picked from commit ba22e8f174)
Co-authored-by: Berker Peksag <berker.peksag@gmail.com>
Appears to be obsolete since 75bb54c3d8.
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit 4015d1cda3)
Co-authored-by: Daniel Hahler <git@thequod.de>
Fix a regression where the C pickle module wouldn't allow unpickling from a
file-like object that doesn't expose a readinto() method.
(cherry picked from commit 9f37872e30)
Co-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Hold reference of __bases__ tuple until tuple item is done with, because by
dropping the reference the item may be destroyed.
(cherry picked from commit 1c56f8ffad)
Co-authored-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
Make it export the correct init symbol also on Windows.
https://bugs.python.org/issue39432
(cherry picked from commit 9538bc9185)
Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
When `allow_abbrev` was first added, disabling the abbreviation of
long options broke the grouping of short flags ([bpo-26967](https://bugs.python.org/issue26967)). As a fix,
b1e4d1b603 (contained in v3.8) ignores `allow_abbrev=False` for a
given argument string if the string does _not_ start with "--"
(i.e. it doesn't look like a long option).
This fix, however, doesn't take into account that long options can
start with alternative characters specified via `prefix_chars`,
introducing a regression: `allow_abbrev=False` has no effect on long
options that start with an alternative prefix character.
The most minimal fix would be to replace the "starts with --" check
with a "starts with two prefix_chars characters". But
`_get_option_tuples` already distinguishes between long and short
options, so let's instead piggyback off of that check by moving the
`allow_abbrev` condition into `_get_option_tuples`.
https://bugs.python.org/issue39546
(cherry picked from commit 8edfc47bae)
Co-authored-by: Kyle Meyer <kyle@kyleam.com>
https://bugs.python.org/issue39546
Automerge-Triggered-By: @encukou
Test when find_good_parse_start should return 0.
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit ffda25f6b8)
Co-authored-by: Cheryl Sabella <cheryl.sabella@gmail.com>
This reverts commit 0f3187c1ce.
The change broke the backwards compatibility of parsing behavior in a
patch release of Python (3.8.1). A decision was taken to revert this
patch in 3.8.2.
In https://bugs.python.org/issue27657 it was decided that the previous
behavior like
>>> urlparse('localhost:8080')
ParseResult(scheme='', netloc='', path='localhost:8080', params='', query='', fragment='')
>>> urlparse('undefined:8080')
ParseResult(scheme='', netloc='', path='undefined:8080', params='', query='', fragment='')
needs to be preserved in patch releases as number of users rely upon it.
Explicitly mention the releases involved with the revert in NEWS.
Adopt the wording suggested by @ned-deily.
The fix for [bpo-39386](https://bugs.python.org/issue39386) attempted to make it so you couldn't reuse a
agen.aclose() coroutine object. It accidentally also prevented you
from calling aclose() at all on an async generator that was already
closed or exhausted. This commit fixes it so we're only blocking the
actually illegal cases, while allowing the legal cases.
The new tests failed before this patch. Also confirmed that this fixes
the test failures we were seeing in Trio with Python dev builds:
https://github.com/python-trio/trio/pull/1396https://bugs.python.org/issue39606
(cherry picked from commit 925dc7fb1d)
Co-authored-by: Nathaniel J. Smith <njs@pobox.com>
https://bugs.python.org/issue39606
Automerge-Triggered-By: @njsmith
bpo-21016, bpo-1294959: The pydoc and trace modules now use the
sysconfig module to get the path to the Python standard library, to
support uncommon installation path like /usr/lib64/python3.9/ on
Fedora.
Co-Authored-By: Jan Matějek <jmatejek@suse.com>
(cherry picked from commit 4fac7ed43e)
Co-authored-by: Victor Stinner <vstinner@python.org>
* Always set the text attribute.
* Correct the offset attribute for non-ascii sources.
(cherry picked from commit 0cc6b5e559)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Improve zipfile.Path performance on zipfiles with a large number of entries.
* 📜🤖 Added by blurb_it.
* Add bpo to blurb
* Sync with importlib_metadata 1.5 (6fe70ca)
* Update blurb.
* Remove compatibility code
* Add stubs module, omitted from earlier commit
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit e5bd73632e)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
* Add tests for case insensitive check of types and extensions as fallback.
* Add tests for data url with no comma.
* Add tests for read_mime_types.
* Add tests for the mimetypes cli and refactor __main__ code to private function.
* Restore mimetypes.knownfiles value at the end of the test.
(cherry picked from commit d8efc14951)
Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
In the font configuration window, remove duplicated font names.
(cherry picked from commit ed335cf53b)
Co-authored-by: Victor Stinner <vstinner@python.org>
Some numerator types used (specifically NumPy) decides to not
return a Python boolean for the "a != b" operation. Using the equivalent
call to bool() guarantees a bool return also for such types.
(cherry picked from commit 427c84f13f)
Co-authored-by: Sebastian Berg <sebastian@sipsolutions.net>
When called on a closed object, readinto() segfaults on account
of a write to a freed buffer:
==220553== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==220553== Access not within mapped region at address 0x2A
==220553== at 0x48408A0: memmove (vg_replace_strmem.c:1272)
==220553== by 0x58DB0C: _buffered_readinto_generic (bufferedio.c:972)
==220553== by 0x58DCBA: _io__Buffered_readinto_impl (bufferedio.c:1053)
==220553== by 0x58DCBA: _io__Buffered_readinto (bufferedio.c.h:253)
Reproducer:
reader = open ("/dev/zero", "rb")
_void = reader.read (42)
reader.close ()
reader.readinto (bytearray (42)) GH-GH-GH- BANG!
The problem exists since 2012 when commit dc469454ec added code
to free the read buffer on close().
Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>
(cherry picked from commit cb1c0746f2)
Co-authored-by: Philipp Gesang <phg@phi-gamma.net>
Adds an additional assertion check based on a race condition for `test__xxsubinterpreters.DestroyTests.test_still_running` discovered in the bpo issue.
https://bugs.python.org/issue37224
(cherry picked from commit f03a8f8d50)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
Improvements in listsort.txt and a comment in sortperf.py.
Automerge-Triggered-By: @csabella
(cherry picked from commit 24e5ad4689)
Co-authored-by: Stefan Pochmann <stefan.pochmann@gmail.com>
Previously, a calltip might be left after SyntaxError, KeyboardInterrupt, or Shell Restart.
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Tal Einat <taleinat+github@gmail.com>
(cherry picked from commit bfdeaa37b3)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Skip test_zipfile.test_add_file_after_2107() if time.localtime()
fails with OverflowError. It is the case on AIX 6.1 for example.
(cherry picked from commit c232c9110c)
Co-authored-by: Victor Stinner <vstinner@python.org>
Replace check for whether something is a method in the mock module. The
previous version fails on PyPy, because there no method wrappers exist
(everything looks like a regular Python-defined function). Thus the
isinstance(getattr(result, '__get__', None), MethodWrapperTypes) check
returns True for any descriptor, not just methods.
This condition could also return erroneously True in CPython for
C-defined descriptors.
Instead to decide whether something is a method, just check directly
whether it's a function defined on the class. This passes all tests on
CPython and fixes the bug on PyPy.
(cherry picked from commit a327677905)
Co-authored-by: Carl Friedrich Bolz-Tereick <cfbolz@gmx.de>
Co-authored-by: Carl Friedrich Bolz-Tereick <cfbolz@gmx.de>
In bpo-36264 os.path.expanduser was changed to ignore HOME on Windows.
Path.expanduser/home still honored HOME despite being documented as behaving the same
as os.path.expanduser. This makes them also ignore HOME so that both implementations
behave the same way again.
(cherry picked from commit c45a2aa9e2)
Co-authored-by: Christoph Reiter <reiter.christoph@gmail.com>
Expose dialog buttons to test code and complete their test coverage.
Complete test coverage for highlights and keys tabs.
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit dd023ad161)
Co-authored-by: Cheryl Sabella <cheryl.sabella@gmail.com>
If an autospecced object is attached using attach_mock the
child would be a function with mock object as attribute from
which signature has to be derived.
(cherry picked from commit 66b00a9d3a)
Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
Use `from ... import ...` to ensure module is fully loaded before accessing its attributes.
(cherry picked from commit 9017e0bd5e)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Add test for nested async decorator patch.
* Add test for side_effect and wraps with a function.
* Add test for side_effect with an exception in the iterable.
(cherry picked from commit 54f743eb31)
Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
Some portions of the pickle documentation hadn't been updated for the pickle protocol changes in Python 3.8 (new protocol 5, default protocol 4). This PR fixes those docs.
https://bugs.python.org/issue39426
(cherry picked from commit e9652e8d58)
Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
When communicate() is called in a loop, it crashes when the child process
has already closed any piped standard stream, but still continues to be running
Co-authored-by: Andriy Maletsky <andriy.maletsky@gmail.com>.
(cherry picked from commit d3ae95e1e9)
Co-authored-by: Alex Rebert <alex@forallsecure.com>
https://bugs.python.org/issue35182
* bpo-39421: Fix posible crash in heapq with custom comparison operators
* fixup! bpo-39421: Fix posible crash in heapq with custom comparison operators
* fixup! fixup! bpo-39421: Fix posible crash in heapq with custom comparison operators
(cherry picked from commit 79f89e6e5a)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
IDLE does not pass a non-default _synchre in any of its calls to
pyparse.find_good_parse_start.
(cherry picked from commit f9e07e116c)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
As described in RFC 1952, section 2.3.1, the XFL (eXtra FLags) byte of a
gzip member header should indicate whether the DEFLATE algorithm was
tuned for speed or compression ratio. Prior to this patch, archives
emitted by the `gzip` module always indicated maximum compression.
(cherry picked from commit eab3b3f1c6)
Co-authored-by: William Chargin <wchargin@gmail.com>
Fix comments and add tests for editor newline_and_indent_event method.
Remove unused None default for function parameter of pyparse find_good_parse_start method
and code triggered by that default.
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit ec64640a2c)
Co-authored-by: Cheryl Sabella <cheryl.sabella@gmail.com>
Copying property objects results in a TypeError. Steps to reproduce:
```
>>> import copy
>>> obj = property()
>>> copy.copy(obj)
````
This affects both shallow and deep copying.
My idea for a fix is to add property objects to the list of "atomic" objects in the copy module.
These already include types like functions and type objects.
I also added property objects to the unit tests test_copy_atomic and test_deepcopy_atomic. This is my first PR, and it's highly likely I've made some mistake, so please be kind :)
https://bugs.python.org/issue38293
(cherry picked from commit 9f3fc6c5b4)
Co-authored-by: Guðni Natan Gunnarsson <1493259+GudniNatan@users.noreply.github.com>
Although the underlying libffi issue remains open, adding these
checks have caused problems in third-party projects which are in
widespread use. See the issue for examples.
The corresponding tests have also been skipped.
(cherry picked from commit c12440c371)
Motivation for this PR (comment from @vstinner in bpo issue):
```
Warning seen o AMD64 Ubuntu Shared 3.x buildbot:
https://buildbot.python.org/all/GH-/builders/141/builds/2593
test_devnull_output (test.test_a=syncio.test_subprocess.SubprocessThreadedWatcherTests) ...
Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)
```
The following implementation details for the new method are TBD:
1) Public vs private
2) Inclusion in `close()`
3) Name
4) Coroutine vs subroutine method
5) *timeout* parameter
If it's a private method, 3, 4, and 5 are significantly less important.
I started with the most minimal implementation that fixes the dangling threads without modifying the regression tests, which I think is particularly important. I typically try to avoid directly modifying existing tests as much as possible unless it's necessary to do so. However, I am open to changing any part of this.
https://bugs.python.org/issue38356
(cherry picked from commit 0ca7cc7fc0)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
* bpo-39297: Update for importlib_metadata 1.4. Includes performance updates.
* 📜🤖 Added by blurb_it.
* Update blurb
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit 136735c1a2)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Host as None in address raises TypeError since it should be string, bytes or bytearray.
(cherry picked from commit 43682f1e39)
Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
* Add test cases for dataclasses.
* Add test for repr output of field.
* Add test for ValueError to be raised when both default and default_factory are passed.
(cherry picked from commit eef1b027ab)
Co-authored-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
Automerge-Triggered-By: @ericvsmith
Correctly parenthesize filter-based statements that contain lambda
expressions in lib2to3.
(cherry picked from commit b821173b54)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
Fixes error attempting to bind to IPv4 address.
(cherry picked from commit 7cdc31a14c)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
tkinter.ttk.Scale().configure([name]) now returns a configuration tuple for name
or a list thereof for all options. Based on patch Giovanni Lombardo.
(cherry picked from commit 5ea7bb25e3)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Ignore leading dots and no longer ignore a trailing newline.
(cherry picked from commit 6a265f0d0c)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.8] bpo-38588: Fix possible crashes in dict and list when calling PyObject_RichCompareBool (GH-17734)
Take strong references before calling PyObject_RichCompareBool to protect against the case
where the object dies during the call.
(cherry picked from commit 2d5bf568ea)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Update Objects/listobject.c
@methane's suggestion
Co-Authored-By: Inada Naoki <songofacandy@gmail.com>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
Hold strong references to list elements while calling PyObject_RichCompareBool().
(cherry picked from commit d9e561d23d)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Fix test_ressources_gced_in_workers() of test_concurrent_futures:
explicitly stop the manager to prevent leaking a child process
running in the background after the test completes.
(cherry picked from commit 673c39331f)
Multiprocessing and concurrent.futures tests now stop the resource
tracker process when tests complete.
Add ResourceTracker._stop() method to
multiprocessing.resource_tracker.
Add _cleanup_tests() helper function to multiprocessing.util: share
code between multiprocessing and concurrent.futures tests.
(cherry picked from commit 9707e8e22d)
Commit 6b5b013bcc ("bpo-26978: Implement pathlib.Path.link_to (Using
os.link) (GH-12990)") introduced a new link_to method in pathlib. However,
this makes pathlib crash when the 'os' module is missing a 'link' method.
Fix this by checking for the presence of the 'link' method on pathlib
module import, and if it's not present, turn it into a runtime error like
those emitted when there is no lchmod() or symlink().
Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
(cherry picked from commit 092435e932)
Co-authored-by: Toke Høiland-Jørgensen <toke@redhat.com>
When parsing an "elif" node, lineno and col_offset of the node now point to the "elif" keyword and not to its condition, making it consistent with the "if" node.
https://bugs.python.org/issue39031
Automerge-Triggered-By: @pablogsal
(cherry picked from commit 025a602af7)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* bpo-39022, bpo-38594: Sync with importlib_metadata 1.3 including improved docs for custom finders and better serialization support in EntryPoints.
* 📜🤖 Added by blurb_it.
* Correct module reference
(cherry picked from commit b7a0109cd2)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
This has happened on some versions of Ubuntu.
(cherry picked from commit bbc4162baf)
Co-authored-by: JohnnyNajera <58344607+JohnnyNajera@users.noreply.github.com>
Also allows winreg.CloseKey() to accept same types as other functions.
(cherry picked from commit ee17e37356)
Co-authored-by: Steve Dower <steve.dower@python.org>
Fix asyncio when the ssl module is missing: only check for
ssl.SSLSocket instance if the ssl module is available.
(cherry picked from commit 82b4950b5e)
Co-authored-by: Victor Stinner <vstinner@python.org>
Fix test_pty: if the process is the session leader, closing the
master file descriptor raises a SIGHUP signal: simply ignore SIGHUP
when running the tests.
(cherry picked from commit a1838ec259)
Co-authored-by: Victor Stinner <vstinner@python.org>
This means a clearer message is now shown when patch.object is called with two string arguments, rather than a class and a string argument.
(cherry picked from commit cd90a52983)
Co-authored-by: Elena Oat <oat.elena@gmail.com>
Fix a potential IndexError when passing an empty value to the message-id
parser. Instead, HeaderParseError should be raised.
(cherry picked from commit 3ae4ea1931)
Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
This adds a new InvalidMessageID token to the email header parser which can be
used to represent invalid message-id headers in the parse tree.
(cherry picked from commit 68157da8b4)
Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
now contextvars.ContextVar "__class_getitem__" method returns ContextVar class, not None.
https://bugs.python.org/issue38979
Automerge-Triggered-By: @asvetlov
(cherry picked from commit 28c91631c2)
Co-authored-by: AMIR <31338382+amiremohamadi@users.noreply.github.com>
test_openssl_version now accepts version 3.0.0.
getpeercert() no longer returns IPv6 addresses with a trailing new line.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38820
(cherry picked from commit 2b7de6696b)
Co-authored-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38820
Automerge-Triggered-By: @tiran
Break cycle generated when saving an exception in socket.py, codeop.py and dyld.py as they keep alive not only the exception but user objects through the ``__traceback__`` attribute.
https://bugs.python.org/issue36820
Automerge-Triggered-By: @pablogsal
(cherry picked from commit b64334cb93)
Co-authored-by: Mario Corchero <mcorcherojim@bloomberg.net>
parse_message_id() was improperly using a token defined inside an exception
handler, which was raising `UnboundLocalError` on parsing an invalid value.
https://bugs.python.org/issue38698
(cherry picked from commit bb815499af)
Co-authored-by: Claudiu Popa <pcmanticore@gmail.com>
Make ssl tests less strict and also accept TLSv3 as the default maximum
version. This change unbreaks test_min_max_version on Fedora 32.
https://bugs.python.org/issue38815
(cherry picked from commit 34864d1cff)
Co-authored-by: torsava <torsava@redhat.com>
SpooledTemporaryFile.rollback() might cause data corruption
when it is in text mode.
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit ea9835c5d1)
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
Extra newlines are removed at the end of non-shell files. If the file only has newlines after stripping other trailing whitespace, all are removed, as is done by patchcheck.py.
(cherry picked from commit 6bf644ec82)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* fix HTTP Digest handling in request.py
There is a bug triggered when server replies to a request with `WWW-Authenticate: Digest` where `qop="auth,auth-int"` rather than mere `qop="auth"`. Having both `auth` and `auth-int` is legitimate according to the `qop-options` rule in §3.2.1 of [[https://www.ietf.org/rfc/rfc2617.txt|RFC 2617]]:
> qop-options = "qop" "=" <"> 1GH-qop-value <">
> qop-value = "auth" | "auth-int" | token
> **qop-options**: [...] If present, it is a quoted string **of one or more** tokens indicating the "quality of protection" values supported by the server. The value `"auth"` indicates authentication; the value `"auth-int"` indicates authentication with integrity protection
This is description confirmed by the definition of the [//n//]`GH-`[//m//]//rule// extended-BNF pattern defined in §2.1 of [[https://www.ietf.org/rfc/rfc2616.txt|RFC 2616]] as 'a comma-separated list of //rule// with at least //n// and at most //m// items'.
When this reply is parsed by `get_authorization`, request.py only tests for identity with `'auth'`, failing to recognize it as one of the supported modes the server announced, and claims that `"qop 'auth,auth-int' is not supported"`.
* 📜🤖 Added by blurb_it.
* bpo-38686 review fix: remember why.
* fix trailing space in Lib/urllib/request.py
Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
(cherry picked from commit 14a89c4798)
Co-authored-by: PypeBros <PypeBros@users.noreply.github.com>
The regex http.cookiejar.LOOSE_HTTP_DATE_RE was vulnerable to regular
expression denial of service (REDoS).
LOOSE_HTTP_DATE_RE.match is called when using http.cookiejar.CookieJar
to parse Set-Cookie headers returned by a server.
Processing a response from a malicious HTTP server can lead to extreme
CPU usage and execution will be blocked for a long time.
The regex contained multiple overlapping \s* capture groups.
Ignoring the ?-optional capture groups the regex could be simplified to
\d+-\w+-\d+(\s*\s*\s*)$
Therefore, a long sequence of spaces can trigger bad performance.
Matching a malicious string such as
LOOSE_HTTP_DATE_RE.match("1-c-1" + (" " * 2000) + "!")
caused catastrophic backtracking.
The fix removes ambiguity about which \s* should match a particular
space.
You can create a malicious server which responds with Set-Cookie headers
to attack all python programs which access it e.g.
from http.server import BaseHTTPRequestHandler, HTTPServer
def make_set_cookie_value(n_spaces):
spaces = " " * n_spaces
expiry = f"1-c-1{spaces}!"
return f"b;Expires={expiry}"
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.log_request(204)
self.send_response_only(204) GH- Don't bother sending Server and Date
n_spaces = (
int(self.path[1:]) GH- Can GET e.g. /100 to test shorter sequences
if len(self.path) > 1 else
65506 GH- Max header line length 65536
)
value = make_set_cookie_value(n_spaces)
for i in range(99): GH- Not necessary, but we can have up to 100 header lines
self.send_header("Set-Cookie", value)
self.end_headers()
if __name__ == "__main__":
HTTPServer(("", 44020), Handler).serve_forever()
This server returns 99 Set-Cookie headers. Each has 65506 spaces.
Extracting the cookies will pretty much never complete.
Vulnerable client using the example at the bottom of
https://docs.python.org/3/library/http.cookiejar.html :
import http.cookiejar, urllib.request
cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
r = opener.open("http://localhost:44020/")
The popular requests library was also vulnerable without any additional
options (as it uses http.cookiejar by default):
import requests
requests.get("http://localhost:44020/")
* Regression test for http.cookiejar REDoS
If we regress, this test will take a very long time.
* Improve performance of http.cookiejar.ISO_DATE_RE
A string like
"444444" + (" " * 2000) + "A"
could cause poor performance due to the 2 overlapping \s* groups,
although this is not as serious as the REDoS in LOOSE_HTTP_DATE_RE was.
(cherry picked from commit 1b779bfb85)
Co-authored-by: bcaller <bcaller@users.noreply.github.com>
test_capi: trashcan tests now require the test "cpu" resource.
(cherry picked from commit 0127bb1c5c)
Co-authored-by: Victor Stinner <vstinner@python.org>
on platforms lacking a functional bind() for named unix domain sockets
https://bugs.python.org/issue38841
Automerge-Triggered-By: @asvetlov
(cherry picked from commit 559bad1a70)
Co-authored-by: xdegaye <xdegaye@gmail.com>
These Format menu functions (default shortcuts Alt-T and Alt-U)
were mistakenly disabled in 3.7.5 and 3.8.0.
(cherry picked from commit b8462477bf)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
This PR implements a fix for `multiprocessing.Process` objects; the error occurs when Processes are created using either `fork` or `forkserver` as the `start_method`.
In these instances, the `MainThread` of the newly created `Process` object retains all attributes from its parent's `MainThread` object, including the `native_id` attribute. The resulting behavior is such that the new process' `MainThread` captures an incorrect/outdated `native_id` (the parent's instead of its own).
This change forces the Process object to update its `native_id` attribute during the bootstrap process.
cc @vstinner
https://bugs.python.org/issue38707
Automerge-Triggered-By: @pitrou
(cherry picked from commit c6b20be85c)
Co-authored-by: Jake Tesler <jake.tesler@gmail.com>
Ignore `GeneratorExit` exceptions when throwing an exception into the `aclose` coroutine of an asynchronous generator.
https://bugs.python.org/issue35409
(cherry picked from commit 8e0de2a480)
Co-authored-by: Vincent Michel <vxgmichel@gmail.com>
changed 'This is bad class design, but save some typing'
into 'This is bad class design, but saves some typing'.
(cherry picked from commit d0acdfcf34)
Co-authored-by: Jason (Perry) Taylor <jtaylor@seek.com.au>
Ensure isabs() is always True for \\?\ prefixed paths
Avoid unnecessary usage of readlink() to avoid resolving broken links incorrectly
Ensure shutil tests run in test directory
if parent `__init__` is not called from a constructor of object derived from `asyncio.Future`
https://bugs.python.org/issue38785
(cherry picked from commit dad6be5ffe)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
This immediately toggles shell, editor, and output windows, but does not affect other input widgets.
(cherry picked from commit 9c2844927d)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
This change, which follows the behavior of C stdio's fdopen and Python 2's file object, allows pipes to be opened in append mode..
(cherry picked from commit 74fa9f723f)
* "Return true/false" is replaced with "Return ``True``/``False``"
if the function actually returns a bool.
* Fixed formatting of some True and False literals (now in monospace).
* Replaced "True/False" with "true/false" if it can be not only bool.
* Replaced some 1/0 with True/False if it corresponds the code.
* "Returns <bool>" is replaced with "Return <bool>".
(cherry picked from commit 138ccbb022)
Whenever I use `path.suffix` I have to check again whether it includes the dot or not. I decided to add it to the docstring so I won't have to keep checking.
https://bugs.python.org/issue38422
Automerge-Triggered-By: @pitrou
(cherry picked from commit 8d4fef4ee2)
Co-authored-by: Ram Rachum <ram@rachum.com>
* Add missing test class (mistake in GH-4455)
* Increase coverage with 4 more test cases
* Rename neg_uid to huge_uid in test_modified_uid_huge
* Replace test_main() with unittest.main()
* Update plistlib docs.
(cherry picked from commit d0d9f7cfa3)
Co-authored-by: Jon Janzen <jjjonjanzen@gmail.com>
These are valid even in python 2.7
https://bugs.python.org/issue33348
Automerge-Triggered-By: @gpshead
(cherry picked from commit 96b06aefe2)
Co-authored-by: Zsolt Dollenstein <zsol.zsol@gmail.com>
bpo-34679, bpo-38563: asyncio.ProactorEventLoop.close() now only calls
signal.set_wakeup_fd() in the main thread.
(cherry picked from commit 1b53a24fb4)
Co-authored-by: Victor Stinner <vstinner@python.org>
The reverse iterator for empty dictionaries was not handling correctly shared-key dictionaries.
(cherry picked from commit 24dc2f8c56)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* bpo-27657: Fix urlparse() with numeric paths
Revert parsing decision from bpo-754016 in favor of the documented
consensus in bpo-16932 of how to treat strings without a // to
designate the netloc.
* bpo-22891: Remove urlsplit() optimization for 'http' prefixed inputs.
(cherry picked from commit 5a88d50ff0)
Co-authored-by: Tim Graham <timograham@gmail.com>
Also updates the documentation to clarify the situation surrounding
the digestmod parameter that is required despite its position in the
argument list as of 3.8.0 as well as removing old python2 era
references to "binary strings".
We indavertently had this raise ValueError in 3.8.0 for the missing
arg. This is not considered an API change as no reasonable code would
be catching this missing argument error in order to handle it.
(cherry picked from commit f33c57d5c7)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
main() is now responsible to send the ANSWER, rather than
ServerProto. main() now waits until it got the HELLO before sending
the ANSWER over the new transport.
Previously, there was a race condition between main() replacing the
protocol and the protocol sending the ANSWER once it gets the HELLO.
TLSv1.3 was disabled for the test: reenable it.
(cherry picked from commit fab4ef2df0)
Co-authored-by: Victor Stinner <vstinner@python.org>
* bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.
Modify the following functions to also work in release mode:
* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()
Other changes:
* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
(equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
attempt to render repr() at the end.
(cherry picked from commit 6876257eaa)
* bpo-36389: Fix _PyBytesWriter in release mode (GH-16624)
Fix _PyBytesWriter API when Python is built in release mode with
assertions.
(cherry picked from commit 60ec6efd96)
* bpo-38070: Enhance visit_decref() debug trace (GH-16631)
subtract_refs() now pass the parent object to visit_decref() which
pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails,
the parent is used in debug trace, rather than the freed object. The
parent object is more likely to contain useful information. Freed
objects cannot be inspected are are displayed as "<object at xxx is
freed>" with no other detail.
(cherry picked from commit 4d5f94b8cd)
* Fix also a typo in PYMEM_DEADBYTE macro comment
* bpo-36389: Add newline to _PyObject_AssertFailed() (GH-16629)
Add a newline between the verbose object dump and the Py_FatalError()
logs for readability.
(cherry picked from commit 7775349895)
* [3.8] bpo-38456: Use /bin/true in test_subprocess (GH-16736)
Instead of sys.executable, "-c", "pass" or "import sys; sys.exit(0)"
use /bin/true when it is available. On a reasonable machine this
shaves up to two seconds wall time off the otherwise ~40sec execution
on a --with-pydebug build. It should be more notable on many
buildbots or overloaded slower I/O systems (CI, etc)..
(cherry picked from commit 67b93f80c7)
* Handle when there is no 'true' command
backport of 46113e0cf3 by Pablo Galindo.
The symbol table handing of PEP572's assignment expressions is not resolving correctly the scope of some variables in presence of global/nonlocal keywords in conjunction with comprehensions.
(cherry picked from commit fd5c414880)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
KeyError should cause a failure in parsing the encoded word and should be caught and raised as a _InvalidEWError instead.
(cherry picked from commit 65dcc8a8dc)
Co-authored-by: Andrei Troie <andreitroie90@gmail.com>
This reverts commit 87bd2071c7.
The reason for revert is a regression caused by the change in 3.8.0rc1, see bpo-38449 for more details.
https://bugs.python.org/issue38449
(cherry picked from commit 19a3d87300)
Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
Add SMTPNotSupportedError in the exports of smtplib
Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
(cherry picked from commit 3faf826e58)
Co-authored-by: nde <denayer.norman@gmail.com>
pathfix.py: Assume all files that end on '.py' are Python scripts when working recursively.
(cherry picked from commit 2b7dc40b2a)
Co-authored-by: Ruediger Pluem <r.pluem@gmx.de>
Add missing stat.S_IFDOOR, stat.S_IFPORT, stat.S_IFWHT,
stat.S_ISDOOR, stat.S_ISPORT, and stat.S_ISWHT values to
the Python implementation of the stat module.
(cherry picked from commit 7bb14316b8)
Co-authored-by: Ronan Lamy <ronan.lamy@gmail.com>
* [bpo-38379](https://bugs.python.org/issue38379): when a finalizer resurrects an object,
nothing is actually collected in this run of gc.
Change the stats to relect that truth.
(cherry picked from commit ecbf35f933)
Co-authored-by: Tim Peters <tim.peters@gmail.com>
https://bugs.python.org/issue38379
Automerge-Triggered-By: @pablogsal
bpo-37531, bpo-38207: On timeout, regrtest no longer attempts to call
`popen.communicate() again: it can hang until all child processes
using stdout and stderr pipes completes. Kill the worker process and
ignores its output.
Reenable test_regrtest.test_multiprocessing_timeout().
bpo-37531: Change also the faulthandler timeout of the main process
from 1 minute to 5 minutes, for Python slowest buildbots.
(cherry picked from commit 0ec618af98)
Co-authored-by: Victor Stinner <vstinner@python.org>
The implementation of weakref.proxy's methods call back into the Python
API using a borrowed references of the weakly referenced object
(acquired via PyWeakref_GET_OBJECT). This API call may delete the last
reference to the object (either directly or via GC), leaving a dangling
pointer, which can be subsequently dereferenced.
To fix this, claim a temporary ownership of the referenced object when
calling the appropriate method. Some functions because at the moment they
do not need to access the borrowed referent, but to protect against
future changes to these functions, ownership need to be fixed in
all potentially affected methods..
(cherry picked from commit 10cd00a9e3)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
It now escapes them with a backslash, as the regular Python interpreter.
Added the "errors" field to the standard streams.
(cherry picked from commit b690a2759e)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
On Windows use UTF-16 (or UTF-32 for 32-bit Tcl_UniChar) with the
"surrogatepass" error handler for converting to/from Tcl Unicode objects.
On Linux use UTF-8 with the "surrogateescape" error handler for converting
to/from Tcl String objects.
Converting strings from Tcl to Python and back now never fails
(except MemoryError).
(cherry picked from commit 06cb94bc84)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-36670, regrtest: Fix WindowsLoadTracker() for partial line (GH-16550)
WindowsLoadTracker.read_output() now uses a short buffer for
incomplete line.
(cherry picked from commit 3e04cd268e)
* bpo-36670: Enhance regrtest WindowsLoadTracker (GH-16553)
The last line is now passed to the parser even if it does not end
with a newline, but only if it's a valid value.
(cherry picked from commit c65119d5bf)
* bpo-36670: Enhance regrtest (GH-16556)
* Add log() method: add timestamp and load average prefixes
to main messages.
* WindowsLoadTracker:
* LOAD_FACTOR_1 is now computed using SAMPLING_INTERVAL
* Initialize the load to the arithmetic mean of the first 5 values
of the Processor Queue Length value (so over 5 seconds), rather
than 0.0.
* Handle BrokenPipeError and when typeperf exit.
* format_duration(1.5) now returns '1.5 sec', rather than
'1 sec 500 ms'
(cherry picked from commit 098e25672f)
test.pythoninfo now logs environment variables used by OpenSSL and
Python ssl modules, and logs attributes of 3 SSL contexts
(SSLContext, default HTTPS context, stdlib context).
(cherry picked from commit b3e7045f83)
Co-authored-by: Victor Stinner <vstinner@python.org>
* Fix TestWorkerProcess.__repr__(): start_time is only valid
if _popen is not None.
* Fix _kill(): don't set _killed to True if _popen is None.
* _run_process(): only set _killed to False after calling
run_test_in_subprocess().
(cherry picked from commit 2ea71a07d0)
Co-authored-by: Victor Stinner <vstinner@python.org>
* Windows: Fix counter name in WindowsLoadTracker. Counter names are
localized: use the registry to get the counter name. Original
change written by Lorenz Mende.
* Regrtest.main() now ensures that the Windows load tracker is also
killed if an exception is raised
* TestWorkerProcess now ensures that worker processes are no longer
running before exiting: kill also worker processes when an
exception is raised.
* Enhance regrtest messages and warnings: include test name,
duration, add a worker identifier, etc.
* Rename MultiprocessRunner to TestWorkerProcess
* Use print_warning() to display warnings.
Co-Authored-By: Lorenz Mende <Lorenz.mende@gmail.com>
(cherry picked from commit 982bfa4da0)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
test_hmac and test_hashlib test built-in hashing implementations and
OpenSSL-based hashing implementations. Add more checks to skip OpenSSL
implementations when a strict crypto policy is active.
Use EVP_DigestInit_ex() instead of EVP_DigestInit() to initialize the
EVP context. The EVP_DigestInit() function clears alls flags and breaks
usedforsecurity flag again.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38270.
(cherry picked from commit 9055815809)
Co-authored-by: Christian Heimes <christian@python.org>
Fix warnings options priority: PyConfig.warnoptions has the highest
priority, as stated in the PEP 587.
* Document options order in PyConfig.warnoptions documentation.
* Make PyWideStringList_INIT macro private: replace "Py" prefix
with "_Py".
* test_embed: add test_init_warnoptions().
(cherry picked from commit fb4ae152a9)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
Document that lnotab can contain invalid bytecode offsets (because of
terrible reasons that are difficult to fix). Make dis.findlinestarts()
ignore invalid offsets in lnotab. All other uses of lnotab in CPython
(various reimplementations of addr2line or line2addr in Python, C and gdb)
already ignore this, because they take an address to look for, instead.
Add tests for the result of dis.findlinestarts() on wacky constructs in
test_peepholer.py, because it's the easiest place to add them.
(cherry picked from commit c8165036f3)
* bpo-38216: Allow bypassing input validation
* bpo-36274: Also allow the URL encoding to be overridden.
* bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL.
* Call with skip_host to avoid tripping on the host checking in the URL.
* Remove obsolete comment.
* Make _prepare_path_encoding its own attr.
This makes overriding just that simpler.
Also, don't use the := operator to make backporting easier.
* Add a news entry.
* _prepare_path_encoding -> _encode_prepared_path()
* Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.
(cherry picked from commit 7774d7831e)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Escape the server title of xmlrpc.server.DocXMLRPCServer
when rendering the document page as HTML.
(cherry picked from commit e8650a4f8c)
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
test_ssl now handles disabled TLS/SSL versions better. OpenSSL's crypto
policy and run-time settings are recognized and tests for disabled versions
are skipped.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue38275
(cherry picked from commit df6ac7e2b8)
Dong-hee Na
Miss Islington (bot)
Pablo Galindo
Serhiy Storchaka
Gregory P. Smith
Stefan Krah
Łukasz Langa
Senthil Kumaran
Steve Dower
Lysandros Nikolaou
Antoine Pitrou
mbarkhau
Matthew Kokotovich
Alex Rebert
Victor Stinner
Karthikeyan Singaravelan
Andrew Svetlov
Terry Jan Reedy
Bar Harel
Ned Deily
Bruno P. Kinoshita
Lisa Roach
Benjamin Peterson
Jon Janzen
Vinay Sajip
Senthil Kumaran
Abhilash Raj
Christian Heimes
Yury Selivanov