Commit Graph

105448 Commits

Author SHA1 Message Date
Miss Islington (bot) c93d68bbb9
bpo-39008: Require Py_ssize_t for PySys_Audit formats rather than raise a deprecation warning (GH-17540)
(cherry picked from commit b8cbe74c34)

Co-authored-by: Steve Dower <steve.dower@python.org>
2019-12-09 11:22:30 -08:00
Miss Islington (bot) 3c5feaffde bpo-38992: avoid fsum test failure from constant-folding (GH-17513) (GH-17530)
* Issue 38992: avoid fsum test failure

* Add NEWS entry
(cherry picked from commit bba873e633)

Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
2019-12-09 11:32:34 -06:00
Miss Islington (bot) c5a2a974d3
Fix APPX registry key generation (GH-17489)
(cherry picked from commit e89e159b18)

Co-authored-by: Steve Dower <steve.dower@python.org>
2019-12-09 09:02:22 -08:00
Miss Islington (bot) 0d57db27f2 bpo-34776: Fix dataclasses to support __future__ "annotations" mode (GH-9518) (#17531)
(cherry picked from commit d219cc4180)

Co-authored-by: Yury Selivanov <yury@magic.io>
2019-12-09 17:07:51 +01:00
Miss Islington (bot) 79c29742a8 bpo-37228: Fix loop.create_datagram_endpoint()'s usage of SO_REUSEADDR (GH-17311) (#17529)
(cherry picked from commit ab513a38c9)

Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2019-12-09 15:39:54 +01:00
Miss Islington (bot) b22183f273
bpo-39006: Fix asyncio when the ssl module is missing (GH-17524)
Fix asyncio when the ssl module is missing: only check for
ssl.SSLSocket instance if the ssl module is available.
(cherry picked from commit 82b4950b5e)

Co-authored-by: Victor Stinner <vstinner@python.org>
2019-12-09 06:19:48 -08:00
Victor Stinner 0381ea79ac
bpo-38916: Document array.array deprecation (GH-17523)
array.array: Document that tostring() and fromstring() deprecated
aliases will be removed in Python 3.9.
2019-12-09 14:46:33 +01:00
Miss Islington (bot) d08fd298dc
bpo-38547: Fix test_pty if the process is the session leader (GH-17519)
Fix test_pty: if the process is the session leader, closing the
master file descriptor raises a SIGHUP signal: simply ignore SIGHUP
when running the tests.
(cherry picked from commit a1838ec259)

Co-authored-by: Victor Stinner <vstinner@python.org>
2019-12-09 03:15:23 -08:00
Miss Islington (bot) 4594565b56 bpo-38669: patch.object now raises a helpful error (GH17510)
This means a clearer message is now shown when patch.object is called with two string arguments, rather than a class and a string argument.
(cherry picked from commit cd90a52983)

Co-authored-by: Elena Oat <oat.elena@gmail.com>
2019-12-09 06:59:04 +00:00
Miss Islington (bot) 184a3812b8
bpo-38673: dont switch to ps2 if the line starts with comment or whitespace (GH-17421)
https://bugs.python.org/issue38673
(cherry picked from commit 109fc2792a)

Co-authored-by: Batuhan Taşkaya <47358913+isidentical@users.noreply.github.com>
2019-12-08 20:56:19 -08:00
Miss Islington (bot) 2abd3a8f58
bpo-38708: email: Fix a potential IndexError when parsing Message-ID (GH-17504)
Fix a potential IndexError when passing an empty value to the message-id
parser. Instead, HeaderParseError should be raised.
(cherry picked from commit 3ae4ea1931)

Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
2019-12-08 18:12:50 -08:00
Miss Islington (bot) f66f4a09d0
bpo-38698: Add a new InvalidMessageID token to email header parser. (GH-17503)
This adds a new InvalidMessageID token to the email header parser which can be
used to represent invalid message-id headers in the parse tree.
(cherry picked from commit 68157da8b4)

Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
2019-12-08 18:11:31 -08:00
Miss Islington (bot) 960fca1a58
bpo-38979: fix ContextVar "__class_getitem__" method (GH-17497)
now contextvars.ContextVar "__class_getitem__" method returns ContextVar class, not None.

https://bugs.python.org/issue38979

Automerge-Triggered-By: @asvetlov
(cherry picked from commit 28c91631c2)

Co-authored-by: AMIR <31338382+amiremohamadi@users.noreply.github.com>
2019-12-08 04:49:07 -08:00
Miss Islington (bot) 9d3cacd590
[3.8] bpo-38820: OpenSSL 3.0.0 compatibility. (GH-17190) (GH-17499)
test_openssl_version now accepts version 3.0.0.

getpeercert() no longer returns IPv6 addresses with a trailing new line.

Signed-off-by: Christian Heimes <christian@python.org>


https://bugs.python.org/issue38820
(cherry picked from commit 2b7de6696b)


Co-authored-by: Christian Heimes <christian@python.org>


https://bugs.python.org/issue38820



Automerge-Triggered-By: @tiran
2019-12-07 09:20:27 -08:00
Andrew Svetlov 930cef2770
[3.8] bpo-37404: Raising value error if an SSLSocket is passed to asyncio functions (GH-16457) (#17496)
https://bugs.python.org/issue37404
(cherry picked from commit 892f9e0777)

Co-authored-by: idomic <michael.ido@gmail.com>
2019-12-07 14:44:20 +02:00
Miss Islington (bot) ce0a2a8620
Make repr of C accelerated TaskWakeupMethWrapper the same as of pure Python version (GH-17484)
(cherry picked from commit 969ae7aca8)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2019-12-07 03:41:41 -08:00
Miss Islington (bot) 7fde4f446a
bpo-38529: Fix asyncio stream warning (GH-17474)
(cherry picked from commit 7ddcd0caa4)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2019-12-07 03:39:57 -08:00
Miss Islington (bot) 836cf31a3c
bpo-37931: Fix crash on OSX re-initializing os.environ (GH-15428)
On most platforms, the `environ` symbol is accessible everywhere.

In a dylib on OSX, it's not easily accessible, you need to find it with
_NSGetEnviron.

The code was caching the *value* of environ. But a setenv() can change the value,
leaving garbage at the old value. Fix: don't cache the value of environ, just
read it every time.
(cherry picked from commit 723f71abf7)

Co-authored-by: Benoit Hudson <benoit@imgspc.com>
2019-12-06 11:32:33 -08:00
Steve Dower c9f480d2cc
bpo-33125: Add support for building and releasing Windows ARM64 packages (GH-17480)
Note that the support is not actually enabled yet, and so we won't be publishing these packages. However, for those who want to build it themselves (even by reusing the Azure Pipelines definition), it's now relatively easy to enable.
2019-12-06 09:40:39 -08:00
Miss Islington (bot) 681285d052
bpo-36820: Break unnecessary cycle in socket.py, codeop.py and dyld.py (GH-13135)
Break cycle generated when saving an exception in socket.py, codeop.py and dyld.py as they keep alive not only the exception but user objects through the ``__traceback__`` attribute.

https://bugs.python.org/issue36820

Automerge-Triggered-By: @pablogsal
(cherry picked from commit b64334cb93)

Co-authored-by: Mario Corchero <mcorcherojim@bloomberg.net>
2019-12-06 06:59:49 -08:00
Miss Islington (bot) e21aa61e96
bpo-38698: Prevent UnboundLocalError to pop up in parse_message_id (GH-17277)
parse_message_id() was improperly using a token defined inside an exception
handler, which was raising `UnboundLocalError` on parsing an invalid value.

https://bugs.python.org/issue38698
(cherry picked from commit bb815499af)

Co-authored-by: Claudiu Popa <pcmanticore@gmail.com>
2019-12-05 09:42:01 -08:00
Miss Islington (bot) cfdaf92221
[3.8] bpo-38270: Fix indentation of test_hmac assertions (GH-17446) (GH-17450)
Since c64a1a61e6 two assertions were indented and thus ignored when running test_hmac.

This PR fixes it. As the change is quite trivial I didn't add a NEWS entry.


https://bugs.python.org/issue38270
(cherry picked from commit 894331838b)


Co-authored-by: stratakis <cstratak@redhat.com>


https://bugs.python.org/issue38270



Automerge-Triggered-By: @tiran
2019-12-05 08:51:30 -08:00
Miss Islington (bot) f4a21d3b23
bpo-38965: Fix faulthandler._stack_overflow() on GCC 10 (GH-17467)
Use the "volatile" keyword to prevent tail call optimization
on any compiler, rather than relying on compiler specific pragma.
(cherry picked from commit 8b787964e0)

Co-authored-by: Victor Stinner <vstinner@python.org>
2019-12-04 12:30:31 -08:00
Miss Islington (bot) 68669ef788
bpo-38634: Allow non-apple build to cope with libedit (GH-16986)
The readline module now detects if Python is linked to libedit at runtime
on all platforms.  Previously, the check was only done on macOS.

If Python is used as a library by a binary linking to libedit, the linker
resolves the rl_initialize symbol required by the readline module against
libedit instead of libreadline, which leads to a segfault.

Take advantage of the existing supporting code to have readline module being
compatible with both situations.
(cherry picked from commit 7105319ada)

Co-authored-by: serge-sans-paille <serge.guelton@telecom-bretagne.eu>
2019-12-04 08:21:16 -08:00
Miss Islington (bot) a75cad440a
bpo-33684: json.tool: Use utf-8 for infile and outfile. (GH-17460)
(cherry picked from commit 808769f3a4)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2019-12-04 01:57:55 -08:00
Miss Islington (bot) baf07395ea
bpo-27873: Update docstring for multiprocessing.Pool.map (GH-17436)
Update docstring for `multiprocessing.Pool.map` to mention `pool.starmap()`.

Prev PR: https://github.com/python/cpython/pull/17367  @aeros

https://bugs.python.org/issue27873
(cherry picked from commit eb48a451e3)

Co-authored-by: An Long <aisk@users.noreply.github.com>
2019-12-03 15:37:40 -08:00
Miss Islington (bot) 8859fc6294
bpo-38945: UU Encoding: Don't let newline in filename corrupt the output format (GH-17418)
(cherry picked from commit a62ad4730c)

Co-authored-by: Matthew Rollings <1211162+stealthcopter@users.noreply.github.com>
2019-12-02 14:44:44 -08:00
Miss Islington (bot) 9e728806d0
bpo-38815: Accept TLSv3 default in min max test (GH-NNNN) (GH-17437)
Make ssl tests less strict and also accept TLSv3 as the default maximum
version. This change unbreaks test_min_max_version on Fedora 32.

https://bugs.python.org/issue38815
(cherry picked from commit 34864d1cff)

Co-authored-by: torsava <torsava@redhat.com>
2019-12-02 08:34:44 -08:00
Miss Islington (bot) 4f1eaf0280
bpo-38449: Add URL delimiters test cases (GH-16729)
* bpo-38449: Add tricky test cases

* bpo-38449: Reflect codereview
(cherry picked from commit 2fe4c48917)

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2019-12-01 15:24:17 -08:00
Miss Islington (bot) d300c0e845
document threading.Lock.locked() (GH-17427)
(cherry picked from commit fdafa1d0ed)

Co-authored-by: idomic <michael.ido@gmail.com>
2019-12-01 12:14:26 -08:00
Miss Islington (bot) 5f234538ab
Fix typos (GH-17423)
(cherry picked from commit 575d0b46d1)

Co-authored-by: Ofek Lev <ofekmeister@gmail.com>
2019-11-30 21:52:39 -08:00
Jules Lasne (jlasne) 305189ecdc [3.8] Added missing coma after end of list in subprocess.rst (GH-17389)
(cherry picked from commit f25875af42)
2019-11-28 22:47:45 -06:00
Steve Dower b74a6f14b9
bpo-38920: Add audit hooks for when sys.excepthook and sys.unraisablehook are invoked (GH-17392)
Also fixes some potential segfaults in unraisable hook handling.
2019-11-28 08:46:23 -08:00
Tzu-ping Chung 18d8edbbb6 bpo-38928: Remove upgrade_dependencies() from venv doc (GH-17410) 2019-11-28 15:44:08 +00:00
Miss Islington (bot) c0db88f6ab
bpo-38524: clarify example a bit and improve formatting (GH-17406)
(cherry picked from commit 02519f75d1)

Co-authored-by: Tal Einat <taleinat+github@gmail.com>
2019-11-27 21:29:02 -08:00
Miss Islington (bot) d21b8e82dd
bpo-26730: Fix SpooledTemporaryFile data corruption (GH-17400)
SpooledTemporaryFile.rollback() might cause data corruption
when it is in text mode.

Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit ea9835c5d1)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2019-11-27 21:23:14 -08:00
Miss Islington (bot) 0f9c9d5328
bpo-38524: document implicit and explicit calling of descriptors' __set_name__ (GH-17364)
(cherry picked from commit 1bddf890e5)

Co-authored-by: Florian Dahlitz <f2dahlitz@freenet.de>
2019-11-27 00:53:52 -08:00
Bruno P. Kinoshita 65c92c5870 [3.8] bpo-38688, shutil.copytree: consume iterator and create list of entries to prevent infinite recursion (GH-17397)
(cherry picked from commit 9bbcbc9f6d)

Co-authored-by: Bruno P. Kinoshita <kinow@users.noreply.github.com>
2019-11-27 12:49:37 +08:00
Terry Jan Reedy a9c86f5e1a
[3.8] bpo-38862: IDLE Strip Trailing Whitespace fixes end newlines (GH-17366)
Extra newlines are removed at the end of non-shell files. If the file only has newlines after stripping other trailing whitespace, all are removed, as is done by patchcheck.py.
(cherry picked from commit 6bf644ec82)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2019-11-26 20:13:23 -05:00
Miss Islington (bot) 191f94cca6
bpo-38922: Raise code.__new__ audit event when code object replace() is called (GH-17394)
(cherry picked from commit c7c01ab1e5)

Co-authored-by: Steve Dower <steve.dower@python.org>
2019-11-26 16:46:32 -08:00
Miss Islington (bot) 86d9933cc6
bpo-38892: Improve docs for audit event (GH-17361)
(cherry picked from commit e563a155be)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2019-11-26 09:14:48 -08:00
Miss Islington (bot) dadff6f661
Remove use of deprecated `array.fromstring` method (GH-17332)
(cherry picked from commit 386d00cc34)

Co-authored-by: David Coles <coles.david@gmail.com>
2019-11-26 00:38:41 -08:00
Miss Islington (bot) 089387ed1f
bpo-21063: Improve module synopsis for distutils (GH-17363)
(cherry picked from commit f8a6316778)

Co-authored-by: Sanchit Khurana <54467174+GeniusLearner@users.noreply.github.com>
2019-11-25 14:26:43 -08:00
Miss Islington (bot) b9e5547f58
bpo-38686: fix HTTP Digest handling in request.py (GH-17045)
* fix HTTP Digest handling in request.py

There is a bug triggered when server replies to a request with `WWW-Authenticate: Digest` where `qop="auth,auth-int"` rather than mere `qop="auth"`. Having both `auth` and `auth-int` is legitimate according to the `qop-options` rule in §3.2.1 of [[https://www.ietf.org/rfc/rfc2617.txt|RFC 2617]]:
>      qop-options       = "qop" "=" <"> 1GH-qop-value <">
>      qop-value         = "auth" | "auth-int" | token
> **qop-options**: [...] If present, it is a quoted string **of one or more** tokens indicating the "quality of protection" values supported by the server.  The value `"auth"` indicates authentication; the value `"auth-int"` indicates authentication with integrity protection

This is description confirmed by the definition of the [//n//]`GH-`[//m//]//rule// extended-BNF pattern defined in §2.1 of [[https://www.ietf.org/rfc/rfc2616.txt|RFC 2616]] as 'a comma-separated list of //rule// with at least //n// and at most //m// items'.

When this reply is parsed by `get_authorization`, request.py only tests for identity with `'auth'`, failing to recognize it as one of the supported modes the server announced, and claims that `"qop 'auth,auth-int' is not supported"`.

* 📜🤖 Added by blurb_it.

* bpo-38686 review fix: remember why.

* fix trailing space in Lib/urllib/request.py

Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
(cherry picked from commit 14a89c4798)

Co-authored-by: PypeBros <PypeBros@users.noreply.github.com>
2019-11-22 15:36:38 -08:00
Miss Islington (bot) ca5fafc2bb
closes bpo-29275: Remove Y2K reference from time module docs (GH-17321)
The Y2K reference is not needed as it only points out that Python's use
of C standard functions doesn't generally suffer from Y2K issues; the
point regarding conventions for conversion of 2-digit years in
:func:`strptime` is still valid.
(cherry picked from commit 42bc60ead3)

Co-authored-by: Callum Ward <wards.callum@gmail.com>
2019-11-22 09:03:50 -08:00
Miss Islington (bot) a1e1be4c49
bpo-38804: Fix REDoS in http.cookiejar (GH-17157)
The regex http.cookiejar.LOOSE_HTTP_DATE_RE was vulnerable to regular
expression denial of service (REDoS).

LOOSE_HTTP_DATE_RE.match is called when using http.cookiejar.CookieJar
to parse Set-Cookie headers returned by a server.
Processing a response from a malicious HTTP server can lead to extreme
CPU usage and execution will be blocked for a long time.

The regex contained multiple overlapping \s* capture groups.
Ignoring the ?-optional capture groups the regex could be simplified to

    \d+-\w+-\d+(\s*\s*\s*)$

Therefore, a long sequence of spaces can trigger bad performance.

Matching a malicious string such as

    LOOSE_HTTP_DATE_RE.match("1-c-1" + (" " * 2000) + "!")

caused catastrophic backtracking.

The fix removes ambiguity about which \s* should match a particular
space.

You can create a malicious server which responds with Set-Cookie headers
to attack all python programs which access it e.g.

    from http.server import BaseHTTPRequestHandler, HTTPServer

    def make_set_cookie_value(n_spaces):
        spaces = " " * n_spaces
        expiry = f"1-c-1{spaces}!"
        return f"b;Expires={expiry}"

    class Handler(BaseHTTPRequestHandler):
        def do_GET(self):
            self.log_request(204)
            self.send_response_only(204)  GH- Don't bother sending Server and Date
            n_spaces = (
                int(self.path[1:])  GH- Can GET e.g. /100 to test shorter sequences
                if len(self.path) > 1 else
                65506  GH- Max header line length 65536
            )
            value = make_set_cookie_value(n_spaces)
            for i in range(99):  GH- Not necessary, but we can have up to 100 header lines
                self.send_header("Set-Cookie", value)
            self.end_headers()

    if __name__ == "__main__":
        HTTPServer(("", 44020), Handler).serve_forever()

This server returns 99 Set-Cookie headers. Each has 65506 spaces.
Extracting the cookies will pretty much never complete.

Vulnerable client using the example at the bottom of
https://docs.python.org/3/library/http.cookiejar.html :

    import http.cookiejar, urllib.request
    cj = http.cookiejar.CookieJar()
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    r = opener.open("http://localhost:44020/")

The popular requests library was also vulnerable without any additional
options (as it uses http.cookiejar by default):

    import requests
    requests.get("http://localhost:44020/")

* Regression test for http.cookiejar REDoS

If we regress, this test will take a very long time.

* Improve performance of http.cookiejar.ISO_DATE_RE

A string like

"444444" + (" " * 2000) + "A"

could cause poor performance due to the 2 overlapping \s* groups,
although this is not as serious as the REDoS in LOOSE_HTTP_DATE_RE was.
(cherry picked from commit 1b779bfb85)

Co-authored-by: bcaller <bcaller@users.noreply.github.com>
2019-11-22 06:42:06 -08:00
Miss Islington (bot) c3cd0de9ec bpo-22367: Update test_fcntl.py for spawn process mode (GH-17154) (GH-17252)
(cherry picked from commit 9960230f76)

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2019-11-22 15:15:36 +01:00
Victor Stinner 84c36c152a
bpo-36854: Fix reference counter in PyInit__testcapi() (GH-17338)
Increment properly Py_True/Py_False reference counter for
_testcapi.WITH_PYMALLOC variable.
2019-11-22 13:39:36 +01:00
Miss Islington (bot) 107ed88cde
bpo-38526: Fix zipfile.Path method name to be the correct one (GH-17317)
(cherry picked from commit 65444cf7fe)

Co-authored-by: Claudiu Popa <pcmanticore@gmail.com>
2019-11-21 13:41:20 -08:00
Lisa Roach b2744c1be7 [3.8] bpo-38857: AsyncMock fix for awaitable values and StopIteration fix [3.8] (GH-17269) (#17304)
(cherry picked from commit 046442d02b)

Co-authored-by: Jason Fried <fried@fb.com>
2019-11-21 20:14:32 +02:00