Commit Graph

458 Commits

Author SHA1 Message Date
Serhiy Storchaka 44e458357f
gh-123067: Fix quadratic complexity in parsing "-quoted cookie values with backslashes (GH-123075)
This fixes CVE-2024-7592.
2024-08-17 16:30:52 +03:00
Serhiy Storchaka 1a0c7b9ba4
gh-121905: Consistently use "floating-point" instead of "floating point" (GH-121907) 2024-07-19 08:06:02 +00:00
Idan Kapustian 192d17c3fd
gh-120485: Add an override of `allow_reuse_port` on classes subclassing `socketserver.TCPServer` (GH-120488)
Co-authored-by: Vinay Sajip <vinay_sajip@yahoo.co.uk>
2024-06-16 13:15:03 +01:00
Geoffrey Thomas ef172521a9
Remove almost all unpaired backticks in docstrings (#119231)
As reported in #117847 and #115366, an unpaired backtick in a docstring
tends to confuse e.g. Sphinx running on subclasses of standard library
objects, and the typographic style of using a backtick as an opening
quote is no longer in favor. Convert almost all uses of the form

    The variable `foo' should do xyz

to

    The variable 'foo' should do xyz

and also fix up miscellaneous other unpaired backticks (extraneous /
missing characters).

No functional change is intended here other than in human-readable
docstrings.
2024-05-22 12:35:18 -04:00
Serhiy Storchaka d6fa1d4bee
gh-66543: Add mimetypes.guess_file_type() (GH-117258) 2024-05-06 15:50:52 +03:00
Michiel W. Beijen 022ba6d161
gh-102247: http: support rfc9110 status codes (GH-117611)
rfc9110 obsoletes the earlier rfc 7231. This document also includes some
status codes that were previously only used for WebDAV and assigns more
generic names to these status codes.

ref: https://www.rfc-editor.org/rfc/rfc9110.html#name-changes-from-rfc-7231

- http.HTTPStatus.CONTENT_TOO_LARGE (413, previously
  REQUEST_ENTITY_TOO_LARGE)
- http.HTTPStatus.URI_TOO_LONG (414, previously REQUEST_URI_TOO_LONG)
- http.HTTPStatus.RANGE_NOT_SATISFYABLE (416, previously
  REQUEST_RANGE_NOT_SATISFYABLE)
- http.HTTPStatus.UNPROCESSABLE_CONTENT (422, previously
  UNPROCESSABLE_ENTITY)

The new constants are added to http.HTTPStatus and the old constant names are
preserved for backwards compatibility.

References in documentation to the obsoleted rfc 7231 are updated
2024-04-13 07:33:20 -07:00
Derek Higgins 465db27cb9
gh-100985: Consistently wrap IPv6 IP address during CONNECT (GH-100986)
Update _get_hostport to always remove square brackets
from IPv6 addresses. Then add them if needed
in "CONNECT .." and "Host: ".
2024-02-17 10:10:12 +00:00
Illia Volochii 41336a72b9
gh-113199: Make read1() and readline() of HTTPResponse close IO after reading all data (GH-113200) 2023-12-18 22:17:16 +02:00
Michael ce1096f974
gh-73561: Omit interface scope from IPv6 when used as Host header (#93324)
Omit the `@interface_scope` from an IPv6 address when used as Host header by `http.client`.

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
2023-11-19 22:37:13 +00:00
Gregory P. Smith 59073c9ab8
gh-109096: Deprecate `http.server.CGIHTTPRequestHandler` (#109387)
Deprecate `http.server.CGIHTTPRequestHandler`.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-09-15 14:26:45 -07:00
Nikita Sobolev 490295d651
gh-105626: Change the default return value of `HTTPConnection.get_proxy_response_headers` (#105628) 2023-07-13 23:55:49 -07:00
Oleg Iarygin 6c81d7572e
gh-104924: Fix `read()able` in `http.client` log messages (gh-104926) 2023-05-26 20:39:51 +09:00
Alexey Namyotkin 85ec192ac4
gh-69152: add method get_proxy_response_headers to HTTPConnection class (#104248)
Add http.client.HTTPConnection method get_proxy_response_headers() - this is a followup to https://github.com/python/cpython/pull/26152 which added it as a non-public attribute. This way we don't pre-compute a headers dictionary that most users will never access. The new method is properly public and documented and triggers full proxy header parsing into a dict only when actually called.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-16 06:20:30 +00:00
Ben Kallus cf720acfcb
gh-103204: `http.server` - Enforce that HTTP version numbers must consist only of digits (#103205)
Reject HTTP requests with invalid http/x.y version numbers: x or y being non-digits or too-long.

---------

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-12 13:25:58 -07:00
Alexey Namyotkin 1afe0e0320
gh-69152: Add _proxy_response_headers attribute to HTTPConnection (#26152)
Add _proxy_response_headers attribute to HTTPConnection (#26152)

---------

Co-authored-by: Senthil Kumaran <senthil@python.org>
2023-05-05 18:52:24 +00:00
Ethan Furman c7c3a60c88
gh-104049: do not expose on-disk location from SimpleHTTPRequestHandler (#104067)
Do not expose the local server's on-disk location from `SimpleHTTPRequestHandler` when generating a directory index. (unnecessary information disclosure)

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2023-05-03 03:42:00 +00:00
Thomas Grainger 9de0cf20fa
GH-103472: close response in HTTPConnection._tunnel (#103473)
Avoid a potential `ResourceWarning` in `http.client.HTTPConnection`
by closing the proxy / tunnel's CONNECT response explicitly.

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-02 03:59:42 +00:00
Paul Ganssle 0b7fd8ffc5
GH-103857: Deprecate utcnow and utcfromtimestamp (#103858)
Using `datetime.datetime.utcnow()` and `datetime.datetime.utcfromtimestamp()` will now raise a `DeprecationWarning`.

We also have removed our internal uses of these functions and documented the change.
2023-04-27 11:32:30 -06:00
Michael Handler 1a8f862e32
gh-66897: Upgrade HTTP CONNECT to protocol HTTP/1.1 (#8305)
* bpo-22708: Upgrade HTTP CONNECT to protocol HTTP/1.1 (GH-NNNN)

Use protocol HTTP/1.1 when sending HTTP CONNECT tunnelling requests;
generate Host: headers if one is not already provided (required by
HTTP/1.1), convert IDN domains to punycode in HTTP CONNECT requests.

* Refactor tests to pass under -bb (fix ByteWarnings); missed some lines >80.

* Use consistent 'tunnelling' spelling in Lib/http/client.py

* Lib/test/test_httplib: Remove remnant of obsoleted test.

* Use dict.copy() not copy.copy()

* fix version changed

* Update Lib/http/client.py

Co-authored-by: bgehman <bgehman@users.noreply.github.com>

* Switch to for/else: syntax, as suggested

* Don't use for: else:

* Sure, fine, w/e

* Oops

* 1nm to the left

---------

Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: bgehman <bgehman@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-04-04 21:55:24 -07:00
Bernhard Wagner d052a383f1
gh-103112: Add http.client.HTTPResponse.read docstring and fix pydoc output (#103113)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Éric <merwok@netwok.org>
2023-03-29 15:21:56 -04:00
Ethan Furman a286caa937
gh-91219: http - use subclassing to override index_pages attribute (GH-100731)
Remove previously added parameter to `__init__`, and recommend subclassing to modify the `index_pages` attribute instead.
2023-01-03 15:20:08 -08:00
Glyph b9aa14a484
gh-100519: simplification to `eff_request_host` in cookiejar.py (#99588)
`IPV4_RE` includes a `.`, and the `.find(".") == -1` included here is already testing to make sure there's no dot, so this part of the expression is tautological. Instead use more modern `in` syntax to make it clear what the check is doing here. The simplified implementation more clearly matches the wording in RFC 2965.

Co-authored-by: hauntsaninja <hauntsaninja@gmail.com>
2022-12-24 18:14:51 -06:00
James Frost 46e6a28308
gh-100474: Fix handling of dirs named index.html in http.server (GH-100475)
If you had a directory called index.html or index.htm within a directory, it would cause http.server to return a 404 Not Found error instead of the directory listing. This came about due to not checking that the index was a regular file.

I have also added a test case for this situation.

Automerge-Triggered-By: GH:merwok
2022-12-24 10:28:59 -08:00
Nikita Sobolev 44892d45b0
gh-99576: Fix cookiejar file that was not truncated for some classes (GH-99616)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-12-20 17:51:26 +01:00
Gregory P. Smith 7e29398407
gh-100001: Also escape \s in http.server log messages. (#100038)
Also \ escape \s in the http.server BaseHTTPRequestHandler.log_message so
that it is technically possible to parse the line and reconstruct what the
original data was.  Without this a \xHH is ambiguious as to if it is a hex
replacement we put in or the characters r"\x" came through in the original
request line.
2022-12-05 14:27:55 -08:00
Gregory P. Smith d8ab0a4dfa
gh-100001: Omit control characters in http.server stderr logs. (#100002)
Replace control characters in http.server.BaseHTTPRequestHandler.log_message with an escaped \xHH sequence to avoid causing problems for the terminal the output is printed to.
2022-12-05 12:55:45 -08:00
Nick Drozd 024ac542d7
bpo-45975: Simplify some while-loops with walrus operator (GH-29347) 2022-11-26 14:33:25 -08:00
Victor Stinner ef0e72b31d
gh-94172: Remove keyfile, certfile and check_hostname parameters (#94173)
Remove the keyfile, certfile and check_hostname parameters,
deprecated since Python 3.6, in modules: ftplib, http.client,
imaplib, poplib and smtplib. Use the context parameter (ssl_context
in imaplib) instead.

Parameters following the removed parameters become keyword-only
parameters.

ftplib: Remove the FTP_TLS.ssl_version class attribute: use the
context parameter instead.
2022-11-03 18:32:25 +01:00
Boris Verkhovskiy 0ea8b925d0
Document that MozillaCookieJar works for curl's cookie files (#91852)
MozillaCookieJar works for curl's cookies
2022-10-03 15:07:54 -07:00
Alexandru Mărășteanu 0ed778835d
gh-95149: Enhance `http.HTTPStatus` with properties that indicate the HTTP status category (GH-95453) 2022-08-30 11:11:44 -07:00
Victor Stinner e87ada48a9
Run Tools/scripts/reindent.py (#94225)
Reindent files which were not properly formatted (PEP 8: 4 spaces).

Remove also some trailing spaces.
2022-06-26 10:34:06 +02:00
Victor Stinner f0b234e6ed
gh-94172: urllib.request avoids deprecated check_hostname (#94193)
The urllib.request no longer uses the deprecated check_hostname
parameter of the http.client module.

Add private http.client._create_https_context() helper to http.client,
used by urllib.request.

Remove the now redundant check on check_hostname and verify_mode in
http.client: the SSLContext.check_hostname setter already implements
the check.
2022-06-24 17:45:28 +02:00
Myron Walker 9a95fa9267
gh-91219: Add an index_pages default list and parameter to SimpleHTTPRequestHandler (GH-31985)
* Add an index_pages default list to SimpleHTTPRequestHandler and an
optional constructor parameter that allows the default indexes pages
list to be overridden.  This makes it easy to set a new index page name
without having to override send_head.
2022-06-23 13:30:44 -07:00
Gregory P. Smith 4abab6b603
gh-87389: Fix an open redirection vulnerability in http.server. (#93879)
Fix an open redirection vulnerability in the `http.server` module when
an URI path starts with `//` that could produce a 301 Location header
with a misleading target.  Vulnerability discovered, and logic fix
proposed, by Hamza Avvan (@hamzaavvan).

Test and comments authored by Gregory P. Smith [Google].
2022-06-21 13:16:57 -07:00
Pascal Wittmann c6f6ede728
gh-79096: Protect cookie file created by {LWP,Mozilla}CookieJar.save() (GH-93463)
Note: This change is not effective on Microsoft Windows.

Cookies can store sensitive information and should therefore be protected
against unauthorized third parties. This is also described in issue #79096.

The filesystem permissions are currently set to 644, everyone can read the
file. This commit changes the permissions to 600, only the creater of the file
can read and modify it. This improves security, because it reduces the attack
surface. Now the attacker needs control of the user that created the cookie or
a ways to circumvent the filesystems permissions.

This change is backwards incompatible. Systems that rely on world-readable
cookies will breake. However, one could argue that those are misconfigured in
the first place.
2022-06-07 10:11:03 +02:00
cibofo 9a0a7b4868
gh-91996: Add an HTTPMethod StrEnum to http (GH-91997)
* Add HTTPMethod enum to http

Create a StrEnum for the 9 common HTTP methods.

Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
2022-05-05 15:39:02 -07:00
Géry Ogam 2d30adee72
bpo-46285: Add command-line option -p/--protocol to module http.server (#30999)
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-05-02 16:28:45 -06:00
Iman Kermani 615b24c80b
bpo-42066: CookieJar cookies should not be sorted (GH-22745) 2022-04-20 20:45:24 -05:00
Nick b6d5e3c3c9
bpo-46075: Store localhost cookies in CookieJar (#30108)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-04-19 13:08:06 -07:00
Dong-hee Na 0bb40a42d7
closes bpo-46736: SimpleHTTPRequestHandler now uses HTML5. (GH-31533)
Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2022-02-23 09:59:32 -08:00
Géry Ogam 2d080347d7
bpo-46436: Fix command-line option -d/--directory in module http.server (GH-30701)
Fix command-line option -d/--directory in http.server main
function that was ignored when combined with --cgi.

Automerge-Triggered-By: GH:merwok
2022-02-03 07:51:05 -08:00
Nikita Sobolev 0cbdd21311
bpo-46565: `del` loop vars that are leaking into module namespaces (GH-30993) 2022-02-03 11:20:08 +02:00
180909 c5c365220e
bpo-28953: Use `raise from` when raising new IncompleteRead (GH-29861)
Automerge-Triggered-By: GH:asvetlov
2021-12-06 16:10:49 -08:00
rtobar 0571b934f5
bpo-45328: Avoid failure in OSs without TCP_NODELAY support (GH-28646)
Operating systems without support for TCP_NODELAY will raise an OSError
when trying to set the socket option, but the show can still go on.
2021-10-06 19:49:44 +02:00
Binbin 17b16e13bb
Fix typos in multiple files (GH-26689)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-06-12 22:47:44 -04:00
Stephen Rosen fb42725561
bpo-43972: Set content-length to 0 for http.server.SimpleHTTPRequestHandler 301s (GH-25705)
* Set content-length for simple http server 301s

When http.server.SimpleHTTPRequestHandler sends a 301 (Moved
Permanently) due to a missing file, it does not set a Content-Length
of 0. Unfortunately, certain clients can be left waiting for the
connection to be closed in this circumstance, even though no body
will be sent. At time of writing, both curl and Firefox demonstrate
this behavior.

* Test Content-Length on simple http server redirect

When serving a redirect, the SimpleHTTPRequestHandler will now send
`Content-Length: 0`. Several tests for http.server already cover
various behaviors and checks including redirection. This change only
adds one check for the expected Content-Length on the simplest case
for a redirect.

* Add news entry for SimpleHTTPRequestHandler fix

* Clarify the specific kind of 301

Co-authored-by: Senthil Kumaran <skumaran@gatech.edu>
2021-05-06 12:25:52 -07:00
Gen Xu 47895e31b6
bpo-44022: Fix http client infinite line reading (DoS) after a HTTP 100 Continue (GH-25916)
Fixes http.client potential denial of service where it could get stuck reading lines from a malicious server after a 100 Continue response.

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2021-05-05 15:42:41 -07:00
Saiyang Gou 927b841c21
bpo-37363: Add audit events to the `http.client` module (GH-21321)
Add audit events to the `http.client` module

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2021-04-23 12:19:08 +02:00
Ethan Furman a02cb474f9
bpo-38659: [Enum] add _simple_enum decorator (GH-25497)
add:

* `_simple_enum` decorator to transform a normal class into an enum
* `_test_simple_enum` function to compare
* `_old_convert_` to enable checking `_convert_` generated enums

`_simple_enum` takes a normal class and converts it into an enum:

    @simple_enum(Enum)
    class Color:
        RED = 1
        GREEN = 2
        BLUE = 3

`_old_convert_` works much like` _convert_` does, using the original logic:

    # in a test file
    import socket, enum
    CheckedAddressFamily = enum._old_convert_(
            enum.IntEnum, 'AddressFamily', 'socket',
            lambda C: C.isupper() and C.startswith('AF_'),
            source=_socket,
            )

`_test_simple_enum` takes a traditional enum and a simple enum and
compares the two:

    # in the REPL or the same module as Color
    class CheckedColor(Enum):
        RED = 1
        GREEN = 2
        BLUE = 3

    _test_simple_enum(CheckedColor, Color)

    _test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-21 10:20:44 -07:00
Ethan Furman 503cdc7c12
Revert "bpo-38659: [Enum] add _simple_enum decorator (GH-25285)" (GH-25476)
This reverts commit dbac8f40e8.
2021-04-19 19:12:24 -07:00