Commit Graph

423 Commits

Author SHA1 Message Date
cibofo 9a0a7b4868
gh-91996: Add an HTTPMethod StrEnum to http (GH-91997)
* Add HTTPMethod enum to http

Create a StrEnum for the 9 common HTTP methods.

Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
2022-05-05 15:39:02 -07:00
Géry Ogam 2d30adee72
bpo-46285: Add command-line option -p/--protocol to module http.server (#30999)
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-05-02 16:28:45 -06:00
Iman Kermani 615b24c80b
bpo-42066: CookieJar cookies should not be sorted (GH-22745) 2022-04-20 20:45:24 -05:00
Nick b6d5e3c3c9
bpo-46075: Store localhost cookies in CookieJar (#30108)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-04-19 13:08:06 -07:00
Dong-hee Na 0bb40a42d7
closes bpo-46736: SimpleHTTPRequestHandler now uses HTML5. (GH-31533)
Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2022-02-23 09:59:32 -08:00
Géry Ogam 2d080347d7
bpo-46436: Fix command-line option -d/--directory in module http.server (GH-30701)
Fix command-line option -d/--directory in http.server main
function that was ignored when combined with --cgi.

Automerge-Triggered-By: GH:merwok
2022-02-03 07:51:05 -08:00
Nikita Sobolev 0cbdd21311
bpo-46565: `del` loop vars that are leaking into module namespaces (GH-30993) 2022-02-03 11:20:08 +02:00
180909 c5c365220e
bpo-28953: Use `raise from` when raising new IncompleteRead (GH-29861)
Automerge-Triggered-By: GH:asvetlov
2021-12-06 16:10:49 -08:00
rtobar 0571b934f5
bpo-45328: Avoid failure in OSs without TCP_NODELAY support (GH-28646)
Operating systems without support for TCP_NODELAY will raise an OSError
when trying to set the socket option, but the show can still go on.
2021-10-06 19:49:44 +02:00
Binbin 17b16e13bb
Fix typos in multiple files (GH-26689)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-06-12 22:47:44 -04:00
Stephen Rosen fb42725561
bpo-43972: Set content-length to 0 for http.server.SimpleHTTPRequestHandler 301s (GH-25705)
* Set content-length for simple http server 301s

When http.server.SimpleHTTPRequestHandler sends a 301 (Moved
Permanently) due to a missing file, it does not set a Content-Length
of 0. Unfortunately, certain clients can be left waiting for the
connection to be closed in this circumstance, even though no body
will be sent. At time of writing, both curl and Firefox demonstrate
this behavior.

* Test Content-Length on simple http server redirect

When serving a redirect, the SimpleHTTPRequestHandler will now send
`Content-Length: 0`. Several tests for http.server already cover
various behaviors and checks including redirection. This change only
adds one check for the expected Content-Length on the simplest case
for a redirect.

* Add news entry for SimpleHTTPRequestHandler fix

* Clarify the specific kind of 301

Co-authored-by: Senthil Kumaran <skumaran@gatech.edu>
2021-05-06 12:25:52 -07:00
Gen Xu 47895e31b6
bpo-44022: Fix http client infinite line reading (DoS) after a HTTP 100 Continue (GH-25916)
Fixes http.client potential denial of service where it could get stuck reading lines from a malicious server after a 100 Continue response.

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2021-05-05 15:42:41 -07:00
Saiyang Gou 927b841c21
bpo-37363: Add audit events to the `http.client` module (GH-21321)
Add audit events to the `http.client` module

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2021-04-23 12:19:08 +02:00
Ethan Furman a02cb474f9
bpo-38659: [Enum] add _simple_enum decorator (GH-25497)
add:

* `_simple_enum` decorator to transform a normal class into an enum
* `_test_simple_enum` function to compare
* `_old_convert_` to enable checking `_convert_` generated enums

`_simple_enum` takes a normal class and converts it into an enum:

    @simple_enum(Enum)
    class Color:
        RED = 1
        GREEN = 2
        BLUE = 3

`_old_convert_` works much like` _convert_` does, using the original logic:

    # in a test file
    import socket, enum
    CheckedAddressFamily = enum._old_convert_(
            enum.IntEnum, 'AddressFamily', 'socket',
            lambda C: C.isupper() and C.startswith('AF_'),
            source=_socket,
            )

`_test_simple_enum` takes a traditional enum and a simple enum and
compares the two:

    # in the REPL or the same module as Color
    class CheckedColor(Enum):
        RED = 1
        GREEN = 2
        BLUE = 3

    _test_simple_enum(CheckedColor, Color)

    _test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-21 10:20:44 -07:00
Ethan Furman 503cdc7c12
Revert "bpo-38659: [Enum] add _simple_enum decorator (GH-25285)" (GH-25476)
This reverts commit dbac8f40e8.
2021-04-19 19:12:24 -07:00
Ethan Furman dbac8f40e8
bpo-38659: [Enum] add _simple_enum decorator (GH-25285)
add:

_simple_enum decorator to transform a normal class into an enum
_test_simple_enum function to compare
_old_convert_ to enable checking _convert_ generated enums
_simple_enum takes a normal class and converts it into an enum:

@simple_enum(Enum)
class Color:
    RED = 1
    GREEN = 2
    BLUE = 3

_old_convert_ works much like _convert_ does, using the original logic:

# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
        enum.IntEnum, 'AddressFamily', 'socket',
        lambda C: C.isupper() and C.startswith('AF_'),
        source=_socket,
        )

test_simple_enum takes a traditional enum and a simple enum and
compares the two:

# in the REPL or the same module as Color
class CheckedColor(Enum):
    RED = 1
    GREEN = 2
    BLUE = 3

_test_simple_enum(CheckedColor, Color)

_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-19 18:04:53 -07:00
Géry Ogam b6884ad268
Update client.py (GH-24827) 2021-03-13 04:22:03 -08:00
Gregory P. Smith c25910a135
bpo-43332: Buffer proxy connection setup packets before sending. (GH-24780)
We now buffer the CONNECT request + tunnel HTTP headers into a single
send call.  This prevents the OS from generating multiple network
packets for connection setup when not necessary, improving efficiency.
2021-03-07 23:35:13 -08:00
Fernando Toledo 711381dfb0
Fix typo in docstring (GH-23515) 2020-12-21 16:06:31 +02:00
Senthil Kumaran da3d2abe6b
GH-5054: CGIHTTPRequestHandler.run_cgi() HTTP_ACCEPT improperly parsed (#23638) 2020-12-05 05:26:24 -08:00
Senthil Kumaran 3ec9d01901
Remove the conditional for setting query. (#23604) 2020-12-02 19:48:14 -08:00
Christian Heimes 03c8ddd9e9
bpo-42413: socket.timeout is now an alias of TimeoutError (GH-23413)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-20 00:26:07 -08:00
Christian Heimes f97406be4c
bpo-40968: Send http/1.1 ALPN extension (#20959)
Signed-off-by: Christian Heimes <christian@python.org>
2020-11-13 16:37:52 +01:00
Jacob Neil Taylor 16ee68da6e
bpo-38976: Add support for HTTP Only flag in MozillaCookieJar (#17471)
Add support for HTTP Only flag in MozillaCookieJar

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2020-10-23 15:48:55 -07:00
AMIR 8ca8a2e8fb
bpo-39603: Prevent header injection in http methods (GH-18485)
reject control chars in http method in http.client.putrequest to prevent http header injection
2020-07-18 13:16:10 -07:00
Bruce Merry 152f0b8bee
bpo-41002: Optimize HTTPResponse.read with a given amount (GH-20943)
I've done the implementation for both non-chunked and chunked reads. I haven't benchmarked chunked reads because I don't currently have a convenient way to generate a high-bandwidth chunked stream, but I don't see any reason that it shouldn't enjoy the same benefits that the non-chunked case does. I've used the benchmark attached to the bpo bug to verify that performance now matches the unsized read case.

Automerge-Triggered-By: @methane
2020-06-24 23:30:21 -07:00
Batuhan Taşkaya 0361556537
bpo-39481: PEP 585 for a variety of modules (GH-19423)
- concurrent.futures
- ctypes
- http.cookies
- multiprocessing
- queue
- tempfile
- unittest.case
- urllib.parse
2020-04-10 07:46:36 -07:00
Victor Stinner 9a679a0e47
bpo-40094: CGIHTTPRequestHandler logs exit code (GH-19285)
CGIHTTPRequestHandler of http.server now logs the CGI script exit
code, rather than the CGI script exit status of os.waitpid().

For example, if the script is killed by signal 11, it now logs:
"CGI script exit code -11."
2020-04-02 03:42:05 +02:00
Ross 61ac612e78
bpo-39507: Add HTTP status 418 "I'm a Teapot" (GH-18291) 2020-03-15 14:24:23 +02:00
Ashwin Ramaswami 9165addc22
bpo-38576: Disallow control characters in hostnames in http.client (GH-18995)
Add host validation for control characters for more CVE-2019-18348 protection.
2020-03-14 11:56:06 -07:00
Dong-hee Na da52be4769
bpo-39509: Update HTTP status code to follow IANA (GH-18294)
Add status codes 103 EARLY_HINTS and 425 TOO_EARLY.
2020-03-14 16:12:01 +02:00
An Long 5907e61a8d bpo-35292: Avoid calling mimetypes.init when http.server is imported (GH-17822) 2020-01-08 10:28:14 -08:00
Jason R. Coombs 7cdc31a14c
bpo-38907: Suppress any exception when attempting to set V6ONLY. (GH-17864)
Fixes error attempting to bind to IPv4 address.
2020-01-06 07:59:36 -05:00
Jason R. Coombs ee94bdb059
bpo-38907: In http.server script, restore binding to IPv4 on Windows. (GH-17851) 2020-01-05 22:32:19 -05:00
bcaller 1b779bfb85 bpo-38804: Fix REDoS in http.cookiejar (GH-17157)
The regex http.cookiejar.LOOSE_HTTP_DATE_RE was vulnerable to regular
expression denial of service (REDoS).

LOOSE_HTTP_DATE_RE.match is called when using http.cookiejar.CookieJar
to parse Set-Cookie headers returned by a server.
Processing a response from a malicious HTTP server can lead to extreme
CPU usage and execution will be blocked for a long time.

The regex contained multiple overlapping \s* capture groups.
Ignoring the ?-optional capture groups the regex could be simplified to

    \d+-\w+-\d+(\s*\s*\s*)$

Therefore, a long sequence of spaces can trigger bad performance.

Matching a malicious string such as

    LOOSE_HTTP_DATE_RE.match("1-c-1" + (" " * 2000) + "!")

caused catastrophic backtracking.

The fix removes ambiguity about which \s* should match a particular
space.

You can create a malicious server which responds with Set-Cookie headers
to attack all python programs which access it e.g.

    from http.server import BaseHTTPRequestHandler, HTTPServer

    def make_set_cookie_value(n_spaces):
        spaces = " " * n_spaces
        expiry = f"1-c-1{spaces}!"
        return f"b;Expires={expiry}"

    class Handler(BaseHTTPRequestHandler):
        def do_GET(self):
            self.log_request(204)
            self.send_response_only(204)  # Don't bother sending Server and Date
            n_spaces = (
                int(self.path[1:])  # Can GET e.g. /100 to test shorter sequences
                if len(self.path) > 1 else
                65506  # Max header line length 65536
            )
            value = make_set_cookie_value(n_spaces)
            for i in range(99):  # Not necessary, but we can have up to 100 header lines
                self.send_header("Set-Cookie", value)
            self.end_headers()

    if __name__ == "__main__":
        HTTPServer(("", 44020), Handler).serve_forever()

This server returns 99 Set-Cookie headers. Each has 65506 spaces.
Extracting the cookies will pretty much never complete.

Vulnerable client using the example at the bottom of
https://docs.python.org/3/library/http.cookiejar.html :

    import http.cookiejar, urllib.request
    cj = http.cookiejar.CookieJar()
    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
    r = opener.open("http://localhost:44020/")

The popular requests library was also vulnerable without any additional
options (as it uses http.cookiejar by default):

    import requests
    requests.get("http://localhost:44020/")

* Regression test for http.cookiejar REDoS

If we regress, this test will take a very long time.

* Improve performance of http.cookiejar.ISO_DATE_RE

A string like

"444444" + (" " * 2000) + "A"

could cause poor performance due to the 2 overlapping \s* groups,
although this is not as serious as the REDoS in LOOSE_HTTP_DATE_RE was.
2019-11-22 15:22:11 +01:00
Siwon Kang 91daa9d722 bpo-38863: Improve is_cgi() in http.server (GH-17312)
is_cgi() function of http.server library does not currently handle a
cgi script if one of the cgi_directories is located at the
sub-directory of given path. Since is_cgi() in CGIHTTPRequestHandler
class separates given path into (dir, rest) based on the first seen
'/', multi-level directories like /sub/dir/cgi-bin/hello.py is divided
into head=/sub, rest=dir/cgi-bin/hello.py then check whether '/sub'
exists in cgi_directories = [..., '/sub/dir/cgi-bin'].
This patch makes the is_cgi() keep expanding dir part to the next '/'
then checking if that expanded path exists in the cgi_directories.

Signed-off-by: Siwon Kang <kkangshawn@gmail.com>





https://bugs.python.org/issue38863
2019-11-22 01:13:05 -08:00
Jason R. Coombs 7774d7831e
bpo-38216, bpo-36274: Allow subclasses to separately override validation and encoding behavior (GH-16448)
* bpo-38216: Allow bypassing input validation

* bpo-36274: Also allow the URL encoding to be overridden.

* bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL.

* Call with skip_host to avoid tripping on the host checking in the URL.

* Remove obsolete comment.

* Make _prepare_path_encoding its own attr.

This makes overriding just that simpler.

Also, don't use the := operator to make backporting easier.

* Add a news entry.

* _prepare_path_encoding -> _encode_prepared_path()

* Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.
2019-09-28 08:32:01 -04:00
Xtreak bb41147eab bpo-12144: Handle cookies with expires attribute in CookieJar.make_cookies (GH-13921)
Handle time comparison for cookies with `expires` attribute when `CookieJar.make_cookies` is called.

Co-authored-by: Demian Brecht <demianbrecht@gmail.com>


https://bugs.python.org/issue12144



Automerge-Triggered-By: @asvetlov
2019-09-13 04:29:00 -07:00
Géry Ogam 781266ebb6 bpo-35640: Allow passing PathLike arguments to SimpleHTTPRequestHandler (GH-11398) 2019-09-11 14:03:46 +01:00
Raymond Hettinger 8f080b0995
bpo-26589: Add http status code 451 (GH-15413) 2019-08-23 10:19:15 -07:00
Florian Wendelborn 5c72badd06 Correct description of HTTP status code 308. (GH-15078)
Permanent redirect was explained as a temporary redirect.
2019-08-03 11:16:34 -07:00
Christian Heimes d1bd6e79da bpo-37440: Enable TLS 1.3 post-handshake auth in http.client (GH-14448)
Post-handshake authentication is required for conditional client cert authentication with TLS 1.3.


https://bugs.python.org/issue37440
2019-06-30 23:32:24 -07:00
Serhiy Storchaka 96aeaec647
bpo-36793: Remove unneeded __str__ definitions. (GH-13081)
Classes that define __str__ the same as __repr__ can
just inherit it from object.
2019-05-06 22:29:40 +03:00
Gregory P. Smith b7378d7728
bpo-30458: Use InvalidURL instead of ValueError. (GH-13044)
Use http.client.InvalidURL instead of ValueError as the new error case's exception.
2019-05-01 16:39:21 -04:00
Gregory P. Smith c4e671eec2
bpo-30458: Disallow control chars in http URLs. (GH-12755)
Disallow control chars in http URLs in urllib.urlopen.  This addresses a potential security problem for applications that do not sanity check their URLs where http request headers could be injected.
2019-04-30 19:12:21 -07:00
Inada Naoki d6bf6f2d0c
bpo-36050: optimize HTTPResponse.read() (GH-12698)
* No need to chunking for now.
* No need to partial read caused by EINTR for now.
2019-04-06 18:06:19 +09:00
Matt Houglum 461c416dd7 bpo-36522: Print all values for headers with multiple values. (GH-12681) 2019-04-04 07:36:47 +03:00
Xtreak 0e1f1f0105 bpo-35647: Fix path check in cookiejar (#11436)
* Refactor cookie path check as per RFC 6265

* Add tests for prefix match of path

* Add news entry

* Fix set_ok_path and refactor tests

* Use slice for last letter
2019-03-10 10:12:28 -07:00
Xtreak ca7fe50635 bpo-35121: prefix dot in domain for proper subdomain validation (GH-10258)
Don't send cookies of domain A without Domain attribute to domain B when domain A is a suffix match of domain B while using a cookiejar with `http.cookiejar.DefaultCookiePolicy` policy.  Patch by Karthikeyan Singaravelan.
2019-03-09 21:09:48 -05:00
Stéphane Wirtel 4b219ce81e bpo-36043: FileCookieJar supports os.PathLike (GH-11945)
https://bugs.python.org/issue36043
2019-03-01 12:40:54 -08:00