On most platforms, the `environ` symbol is accessible everywhere.
In a dylib on OSX, it's not easily accessible, you need to find it with
_NSGetEnviron.
The code was caching the *value* of environ. But a setenv() can change the value,
leaving garbage at the old value. Fix: don't cache the value of environ, just
read it every time.
(cherry picked from commit 723f71abf7)
Co-authored-by: Benoit Hudson <benoit@imgspc.com>
Note that the support is not actually enabled yet, and so we won't be publishing these packages. However, for those who want to build it themselves (even by reusing the Azure Pipelines definition), it's now relatively easy to enable.
Break cycle generated when saving an exception in socket.py, codeop.py and dyld.py as they keep alive not only the exception but user objects through the ``__traceback__`` attribute.
https://bugs.python.org/issue36820
Automerge-Triggered-By: @pablogsal
(cherry picked from commit b64334cb93)
Co-authored-by: Mario Corchero <mcorcherojim@bloomberg.net>
parse_message_id() was improperly using a token defined inside an exception
handler, which was raising `UnboundLocalError` on parsing an invalid value.
https://bugs.python.org/issue38698
(cherry picked from commit bb815499af)
Co-authored-by: Claudiu Popa <pcmanticore@gmail.com>
Use the "volatile" keyword to prevent tail call optimization
on any compiler, rather than relying on compiler specific pragma.
(cherry picked from commit 8b787964e0)
Co-authored-by: Victor Stinner <vstinner@python.org>
The readline module now detects if Python is linked to libedit at runtime
on all platforms. Previously, the check was only done on macOS.
If Python is used as a library by a binary linking to libedit, the linker
resolves the rl_initialize symbol required by the readline module against
libedit instead of libreadline, which leads to a segfault.
Take advantage of the existing supporting code to have readline module being
compatible with both situations.
(cherry picked from commit 7105319ada)
Co-authored-by: serge-sans-paille <serge.guelton@telecom-bretagne.eu>
Make ssl tests less strict and also accept TLSv3 as the default maximum
version. This change unbreaks test_min_max_version on Fedora 32.
https://bugs.python.org/issue38815
(cherry picked from commit 34864d1cff)
Co-authored-by: torsava <torsava@redhat.com>
SpooledTemporaryFile.rollback() might cause data corruption
when it is in text mode.
Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit ea9835c5d1)
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
Extra newlines are removed at the end of non-shell files. If the file only has newlines after stripping other trailing whitespace, all are removed, as is done by patchcheck.py.
(cherry picked from commit 6bf644ec82)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* fix HTTP Digest handling in request.py
There is a bug triggered when server replies to a request with `WWW-Authenticate: Digest` where `qop="auth,auth-int"` rather than mere `qop="auth"`. Having both `auth` and `auth-int` is legitimate according to the `qop-options` rule in §3.2.1 of [[https://www.ietf.org/rfc/rfc2617.txt|RFC 2617]]:
> qop-options = "qop" "=" <"> 1GH-qop-value <">
> qop-value = "auth" | "auth-int" | token
> **qop-options**: [...] If present, it is a quoted string **of one or more** tokens indicating the "quality of protection" values supported by the server. The value `"auth"` indicates authentication; the value `"auth-int"` indicates authentication with integrity protection
This is description confirmed by the definition of the [//n//]`GH-`[//m//]//rule// extended-BNF pattern defined in §2.1 of [[https://www.ietf.org/rfc/rfc2616.txt|RFC 2616]] as 'a comma-separated list of //rule// with at least //n// and at most //m// items'.
When this reply is parsed by `get_authorization`, request.py only tests for identity with `'auth'`, failing to recognize it as one of the supported modes the server announced, and claims that `"qop 'auth,auth-int' is not supported"`.
* 📜🤖 Added by blurb_it.
* bpo-38686 review fix: remember why.
* fix trailing space in Lib/urllib/request.py
Co-Authored-By: Brandt Bucher <brandtbucher@gmail.com>
(cherry picked from commit 14a89c4798)
Co-authored-by: PypeBros <PypeBros@users.noreply.github.com>
The Y2K reference is not needed as it only points out that Python's use
of C standard functions doesn't generally suffer from Y2K issues; the
point regarding conventions for conversion of 2-digit years in
:func:`strptime` is still valid.
(cherry picked from commit 42bc60ead3)
Co-authored-by: Callum Ward <wards.callum@gmail.com>
The regex http.cookiejar.LOOSE_HTTP_DATE_RE was vulnerable to regular
expression denial of service (REDoS).
LOOSE_HTTP_DATE_RE.match is called when using http.cookiejar.CookieJar
to parse Set-Cookie headers returned by a server.
Processing a response from a malicious HTTP server can lead to extreme
CPU usage and execution will be blocked for a long time.
The regex contained multiple overlapping \s* capture groups.
Ignoring the ?-optional capture groups the regex could be simplified to
\d+-\w+-\d+(\s*\s*\s*)$
Therefore, a long sequence of spaces can trigger bad performance.
Matching a malicious string such as
LOOSE_HTTP_DATE_RE.match("1-c-1" + (" " * 2000) + "!")
caused catastrophic backtracking.
The fix removes ambiguity about which \s* should match a particular
space.
You can create a malicious server which responds with Set-Cookie headers
to attack all python programs which access it e.g.
from http.server import BaseHTTPRequestHandler, HTTPServer
def make_set_cookie_value(n_spaces):
spaces = " " * n_spaces
expiry = f"1-c-1{spaces}!"
return f"b;Expires={expiry}"
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
self.log_request(204)
self.send_response_only(204) GH- Don't bother sending Server and Date
n_spaces = (
int(self.path[1:]) GH- Can GET e.g. /100 to test shorter sequences
if len(self.path) > 1 else
65506 GH- Max header line length 65536
)
value = make_set_cookie_value(n_spaces)
for i in range(99): GH- Not necessary, but we can have up to 100 header lines
self.send_header("Set-Cookie", value)
self.end_headers()
if __name__ == "__main__":
HTTPServer(("", 44020), Handler).serve_forever()
This server returns 99 Set-Cookie headers. Each has 65506 spaces.
Extracting the cookies will pretty much never complete.
Vulnerable client using the example at the bottom of
https://docs.python.org/3/library/http.cookiejar.html :
import http.cookiejar, urllib.request
cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
r = opener.open("http://localhost:44020/")
The popular requests library was also vulnerable without any additional
options (as it uses http.cookiejar by default):
import requests
requests.get("http://localhost:44020/")
* Regression test for http.cookiejar REDoS
If we regress, this test will take a very long time.
* Improve performance of http.cookiejar.ISO_DATE_RE
A string like
"444444" + (" " * 2000) + "A"
could cause poor performance due to the 2 overlapping \s* groups,
although this is not as serious as the REDoS in LOOSE_HTTP_DATE_RE was.
(cherry picked from commit 1b779bfb85)
Co-authored-by: bcaller <bcaller@users.noreply.github.com>
test_capi: trashcan tests now require the test "cpu" resource.
(cherry picked from commit 0127bb1c5c)
Co-authored-by: Victor Stinner <vstinner@python.org>
This PR will make the following changes to the [_Built-in Functions_](https://docs.python.org/3/library/functions.html) chapter of the library documentation:
- improve hyperlinks in Sphinx roles (trailing 's' belong to hyperlinks).
Automerge-Triggered-By: @csabella
(cherry picked from commit d67279147a)
Co-authored-by: Géry Ogam <gery.ogam@gmail.com>
Fixed an incorrect sentence in Doc/c-api/mapping.rst I fell on
while translating the file.
skip issue
Automerge-Triggered-By: @csabella
(cherry picked from commit 06ca2a2be9)
Co-authored-by: Aveheuzed <a.masson555@ntymail.com>
on platforms lacking a functional bind() for named unix domain sockets
https://bugs.python.org/issue38841
Automerge-Triggered-By: @asvetlov
(cherry picked from commit 559bad1a70)
Co-authored-by: xdegaye <xdegaye@gmail.com>
These Format menu functions (default shortcuts Alt-T and Alt-U)
were mistakenly disabled in 3.7.5 and 3.8.0.
(cherry picked from commit b8462477bf)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
This PR implements a fix for `multiprocessing.Process` objects; the error occurs when Processes are created using either `fork` or `forkserver` as the `start_method`.
In these instances, the `MainThread` of the newly created `Process` object retains all attributes from its parent's `MainThread` object, including the `native_id` attribute. The resulting behavior is such that the new process' `MainThread` captures an incorrect/outdated `native_id` (the parent's instead of its own).
This change forces the Process object to update its `native_id` attribute during the bootstrap process.
cc @vstinner
https://bugs.python.org/issue38707
Automerge-Triggered-By: @pitrou
(cherry picked from commit c6b20be85c)
Co-authored-by: Jake Tesler <jake.tesler@gmail.com>
Ignore `GeneratorExit` exceptions when throwing an exception into the `aclose` coroutine of an asynchronous generator.
https://bugs.python.org/issue35409
(cherry picked from commit 8e0de2a480)
Co-authored-by: Vincent Michel <vxgmichel@gmail.com>
Andrew Svetlov
Miss Islington (bot)
Steve Dower
Jules Lasne (jlasne)
Tzu-ping Chung
Bruno P. Kinoshita
Terry Jan Reedy
Victor Stinner
Lisa Roach