Commit Graph

187 Commits

Author SHA1 Message Date
Illia Volochii 41336a72b9
gh-113199: Make read1() and readline() of HTTPResponse close IO after reading all data (GH-113200) 2023-12-18 22:17:16 +02:00
Michael ce1096f974
gh-73561: Omit interface scope from IPv6 when used as Host header (#93324)
Omit the `@interface_scope` from an IPv6 address when used as Host header by `http.client`.

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
2023-11-19 22:37:13 +00:00
Nikita Sobolev e57ecf6bbc
gh-108303: Move all certificates to `Lib/test/certdata/` (#109489) 2023-09-16 18:47:18 +02:00
Serhiy Storchaka 1e0d62793a
gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') (GH-108480) 2023-09-05 17:56:30 +03:00
Nikita Sobolev 490295d651
gh-105626: Change the default return value of `HTTPConnection.get_proxy_response_headers` (#105628) 2023-07-13 23:55:49 -07:00
Alexey Namyotkin 85ec192ac4
gh-69152: add method get_proxy_response_headers to HTTPConnection class (#104248)
Add http.client.HTTPConnection method get_proxy_response_headers() - this is a followup to https://github.com/python/cpython/pull/26152 which added it as a non-public attribute. This way we don't pre-compute a headers dictionary that most users will never access. The new method is properly public and documented and triggers full proxy header parsing into a dict only when actually called.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-16 06:20:30 +00:00
Alexey Namyotkin 1afe0e0320
gh-69152: Add _proxy_response_headers attribute to HTTPConnection (#26152)
Add _proxy_response_headers attribute to HTTPConnection (#26152)

---------

Co-authored-by: Senthil Kumaran <senthil@python.org>
2023-05-05 18:52:24 +00:00
Thomas Grainger 9de0cf20fa
GH-103472: close response in HTTPConnection._tunnel (#103473)
Avoid a potential `ResourceWarning` in `http.client.HTTPConnection`
by closing the proxy / tunnel's CONNECT response explicitly.

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-05-02 03:59:42 +00:00
Michael Handler 1a8f862e32
gh-66897: Upgrade HTTP CONNECT to protocol HTTP/1.1 (#8305)
* bpo-22708: Upgrade HTTP CONNECT to protocol HTTP/1.1 (GH-NNNN)

Use protocol HTTP/1.1 when sending HTTP CONNECT tunnelling requests;
generate Host: headers if one is not already provided (required by
HTTP/1.1), convert IDN domains to punycode in HTTP CONNECT requests.

* Refactor tests to pass under -bb (fix ByteWarnings); missed some lines >80.

* Use consistent 'tunnelling' spelling in Lib/http/client.py

* Lib/test/test_httplib: Remove remnant of obsoleted test.

* Use dict.copy() not copy.copy()

* fix version changed

* Update Lib/http/client.py

Co-authored-by: bgehman <bgehman@users.noreply.github.com>

* Switch to for/else: syntax, as suggested

* Don't use for: else:

* Sure, fine, w/e

* Oops

* 1nm to the left

---------

Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: bgehman <bgehman@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-04-04 21:55:24 -07:00
Alex Waygood 401d7a7f00
gh-102515: Remove unused imports in the `Lib/` directory (#102516) 2023-03-08 11:45:38 +00:00
Victor Stinner ef0e72b31d
gh-94172: Remove keyfile, certfile and check_hostname parameters (#94173)
Remove the keyfile, certfile and check_hostname parameters,
deprecated since Python 3.6, in modules: ftplib, http.client,
imaplib, poplib and smtplib. Use the context parameter (ssl_context
in imaplib) instead.

Parameters following the removed parameters become keyword-only
parameters.

ftplib: Remove the FTP_TLS.ssl_version class attribute: use the
context parameter instead.
2022-11-03 18:32:25 +01:00
Alexandru Mărășteanu 0ed778835d
gh-95149: Enhance `http.HTTPStatus` with properties that indicate the HTTP status category (GH-95453) 2022-08-30 11:11:44 -07:00
Christian Heimes deeaac49e2
bpo-40280: Skip socket, fork, subprocess tests on Emscripten (GH-31986)
- Add requires_fork and requires_subprocess to more tests
- Skip extension import tests if dlopen is not available
- Don't assume that _testcapi is a shared extension
- Skip a lot of socket tests that don't work on Emscripten
- Skip mmap tests, mmap emulation is incomplete
- venv does not work yet
- Cannot get libc from executable

The "entire" test suite is now passing on Emscripten with EMSDK from git head (91 suites are skipped).
2022-03-22 03:04:36 -07:00
Gregory P. Smith e60ab843cb
bpo-44022: Improve the regression test. (GH-26503)
It wasn't actually detecting the regression due to the
assertion being too lenient.
2021-06-02 20:43:38 -07:00
Gen Xu 47895e31b6
bpo-44022: Fix http client infinite line reading (DoS) after a HTTP 100 Continue (GH-25916)
Fixes http.client potential denial of service where it could get stuck reading lines from a malicious server after a 100 Continue response.

Co-authored-by: Gregory P. Smith <greg@krypto.org>
2021-05-05 15:42:41 -07:00
Inada Naoki fa51c0c448
bpo-43651: Fix EncodingWarning in tests. (GH-25655)
* test_httplib
* test_httpservers
* test_logging
2021-04-29 11:34:56 +09:00
Ethan Furman a02cb474f9
bpo-38659: [Enum] add _simple_enum decorator (GH-25497)
add:

* `_simple_enum` decorator to transform a normal class into an enum
* `_test_simple_enum` function to compare
* `_old_convert_` to enable checking `_convert_` generated enums

`_simple_enum` takes a normal class and converts it into an enum:

    @simple_enum(Enum)
    class Color:
        RED = 1
        GREEN = 2
        BLUE = 3

`_old_convert_` works much like` _convert_` does, using the original logic:

    # in a test file
    import socket, enum
    CheckedAddressFamily = enum._old_convert_(
            enum.IntEnum, 'AddressFamily', 'socket',
            lambda C: C.isupper() and C.startswith('AF_'),
            source=_socket,
            )

`_test_simple_enum` takes a traditional enum and a simple enum and
compares the two:

    # in the REPL or the same module as Color
    class CheckedColor(Enum):
        RED = 1
        GREEN = 2
        BLUE = 3

    _test_simple_enum(CheckedColor, Color)

    _test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-21 10:20:44 -07:00
Ethan Furman 503cdc7c12
Revert "bpo-38659: [Enum] add _simple_enum decorator (GH-25285)" (GH-25476)
This reverts commit dbac8f40e8.
2021-04-19 19:12:24 -07:00
Ethan Furman dbac8f40e8
bpo-38659: [Enum] add _simple_enum decorator (GH-25285)
add:

_simple_enum decorator to transform a normal class into an enum
_test_simple_enum function to compare
_old_convert_ to enable checking _convert_ generated enums
_simple_enum takes a normal class and converts it into an enum:

@simple_enum(Enum)
class Color:
    RED = 1
    GREEN = 2
    BLUE = 3

_old_convert_ works much like _convert_ does, using the original logic:

# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
        enum.IntEnum, 'AddressFamily', 'socket',
        lambda C: C.isupper() and C.startswith('AF_'),
        source=_socket,
        )

test_simple_enum takes a traditional enum and a simple enum and
compares the two:

# in the REPL or the same module as Color
class CheckedColor(Enum):
    RED = 1
    GREEN = 2
    BLUE = 3

_test_simple_enum(CheckedColor, Color)

_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-19 18:04:53 -07:00
Gregory P. Smith c25910a135
bpo-43332: Buffer proxy connection setup packets before sending. (GH-24780)
We now buffer the CONNECT request + tunnel HTTP headers into a single
send call.  This prevents the OS from generating multiple network
packets for connection setup when not necessary, improving efficiency.
2021-03-07 23:35:13 -08:00
Angelin BOOZ 68526fe258
bpo-40084: Enum - dir() includes member attributes (GH-19219) 2020-09-21 06:11:06 -07:00
Victor Stinner fabd7bb8e0
bpo-41521: Replace whitelist/blacklist with allowlist/denylist (GH-21822)
Automerge-Triggered-By: @tiran
2020-08-11 06:26:59 -07:00
AMIR 8ca8a2e8fb
bpo-39603: Prevent header injection in http methods (GH-18485)
reject control chars in http method in http.client.putrequest to prevent http header injection
2020-07-18 13:16:10 -07:00
Hai Shi 883bc63833
bpo-40275: Use new test.support helper submodules in tests (GH-21314) 2020-07-06 11:12:49 +02:00
Bruce Merry 152f0b8bee
bpo-41002: Optimize HTTPResponse.read with a given amount (GH-20943)
I've done the implementation for both non-chunked and chunked reads. I haven't benchmarked chunked reads because I don't currently have a convenient way to generate a high-bandwidth chunked stream, but I don't see any reason that it shouldn't enjoy the same benefits that the non-chunked case does. I've used the benchmark attached to the bpo bug to verify that performance now matches the unsized read case.

Automerge-Triggered-By: @methane
2020-06-24 23:30:21 -07:00
Serhiy Storchaka bfb1cf4465
bpo-40275: Move transient_internet from test.support to socket_helper (GH-19711) 2020-04-29 10:36:20 +03:00
Serhiy Storchaka 16994912c9
bpo-40275: Avoid importing socket in test.support (GH-19603)
* Move socket related functions from test.support to socket_helper.
* Import socket, nntplib and urllib.error lazily in transient_internet().
* Remove importing multiprocess.
2020-04-25 10:06:29 +03:00
Ross 61ac612e78
bpo-39507: Add HTTP status 418 "I'm a Teapot" (GH-18291) 2020-03-15 14:24:23 +02:00
Ashwin Ramaswami 9165addc22
bpo-38576: Disallow control characters in hostnames in http.client (GH-18995)
Add host validation for control characters for more CVE-2019-18348 protection.
2020-03-14 11:56:06 -07:00
Dong-hee Na da52be4769
bpo-39509: Update HTTP status code to follow IANA (GH-18294)
Add status codes 103 EARLY_HINTS and 425 TOO_EARLY.
2020-03-14 16:12:01 +02:00
Jason R. Coombs 7774d7831e
bpo-38216, bpo-36274: Allow subclasses to separately override validation and encoding behavior (GH-16448)
* bpo-38216: Allow bypassing input validation

* bpo-36274: Also allow the URL encoding to be overridden.

* bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL.

* Call with skip_host to avoid tripping on the host checking in the URL.

* Remove obsolete comment.

* Make _prepare_path_encoding its own attr.

This makes overriding just that simpler.

Also, don't use the := operator to make backporting easier.

* Add a news entry.

* _prepare_path_encoding -> _encode_prepared_path()

* Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.
2019-09-28 08:32:01 -04:00
Raymond Hettinger 8f080b0995
bpo-26589: Add http status code 451 (GH-15413) 2019-08-23 10:19:15 -07:00
Pablo Galindo aa542c2cf2
bpo-37795: Capture DeprecationWarnings in the test suite (GH-15184) 2019-08-08 23:25:46 +01:00
Christian Heimes d1bd6e79da bpo-37440: Enable TLS 1.3 post-handshake auth in http.client (GH-14448)
Post-handshake authentication is required for conditional client cert authentication with TLS 1.3.


https://bugs.python.org/issue37440
2019-06-30 23:32:24 -07:00
Gregory P. Smith 2cc0223f43
bpo-35925: Skip SSL tests that fail due to weak external certs. (GH-13124)
Modern Linux distros such as Debian Buster have default OpenSSL system
configurations that reject connections to servers with weak certificates
by default.  This causes our test suite run with external networking
resources enabled to skip these tests when they encounter such a failure.

Fixing the network servers is a separate issue.
2019-05-06 17:54:06 -04:00
Giampaolo Rodola eb7e29f2a9
bpo-35934: Add socket.create_server() utility function (GH-11784) 2019-04-09 00:34:02 +02:00
Matt Houglum 461c416dd7 bpo-36522: Print all values for headers with multiple values. (GH-12681) 2019-04-04 07:36:47 +03:00
Marco Strigl 936f03e7fa bpo-33365: print the header values beside the keys (GH-6611)
with debuglevel=1 only the header keys got printed. With
this change the header values get printed as well and the single
header entries get '\n' as a separator.
2018-06-19 16:20:58 +03:00
Serhiy Storchaka f8a4c03ede
bpo-30399: Get rid of trailing comma in the repr of BaseException. (#1650) 2017-11-15 17:53:28 +02:00
Nir Soffer ad455cd924 bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279)
blocksize was hardcoded to 8192, preventing efficient upload when using
file-like body. Add blocksize argument to __init__, so users can
configure the blocksize to fit their needs.

I tested this uploading data from /dev/zero to a web server dropping the
received data, to test the overhead of the HTTPConnection.send() with a
file-like object.

Here is an example 10g upload with the default buffer size (8192):

$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 17.53 seconds (584.00m/s)

real	0m17.574s
user	0m8.887s
sys	0m5.971s

Same with 512k blocksize:

$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 6.60 seconds (1551.15m/s)

real	0m6.641s
user	0m3.426s
sys	0m2.162s

In real world usage the difference will be smaller, depending on the
local and remote storage and the network.

See https://github.com/nirs/http-bench for more info.
2017-11-06 13:16:37 -08:00
Vitor Pereira 52ad72dd0a bpo-30553: Add status code 421 to http.HTTPStatus (GH-2589) 2017-10-26 21:49:19 +03:00
Antoine Pitrou 88c60c9668 Trivial cleanups following bpo-31370 (#3649)
* Trivial cleanups following bpo-31370

* Also cleanup the "importlib._bootstrap_external" module
2017-09-18 23:50:44 +02:00
Christian Heimes a170fa162d bpo-31346: Use PROTOCOL_TLS_CLIENT/SERVER (#3058)
Replaces PROTOCOL_TLSv* and PROTOCOL_SSLv23 with PROTOCOL_TLS_CLIENT and
PROTOCOL_TLS_SERVER.

Signed-off-by: Christian Heimes <christian@python.org>
2017-09-15 20:27:30 +02:00
Martin Panter fd1df3fef2 Issue #28394: Merge typo fixes from 3.5 into 3.6 2016-10-10 00:28:38 +00:00
Martin Panter d2a584b47b Issue #28394: Spelling and typo fixes in code comments and changelog
Includes patch by Ville Skyttä.
2016-10-10 00:24:34 +00:00
Christian Heimes 8d14abc800 Issue 28022: Catch deprecation warning in test_httplib, reported by Martin Panter 2016-09-11 19:54:43 +02:00
R David Murray dc1650ca06 #22233: Only split headers on \r and/or \n, per email RFCs.
Original patch by Martin Panter, new policy fixes by me.
2016-09-07 17:44:34 -04:00
R David Murray 1badd28163 Merge: #22233: Only split headers on \r and/or \n, per email RFCs. 2016-09-07 17:46:55 -04:00
Martin Panter ef91bb2660 Issue #12319: Always send file request bodies using chunked encoding
The previous attempt to determine the file’s Content-Length gave a false
positive for pipes on Windows.

Also, drop the special case for sending zero-length iterable bodies.
2016-08-27 01:39:26 +00:00
Martin Panter 3c0d0baf2b Issue #12319: Support for chunked encoding of HTTP request bodies
When the body object is a file, its size is no longer determined with
fstat(), since that can report the wrong result (e.g. reading from a pipe).
Instead, determine the size using seek(), or fall back to chunked encoding
for unseekable files.

Also, change the logic for detecting text files to check for TextIOBase
inheritance, rather than inspecting the “mode” attribute, which may not
exist (e.g. BytesIO and StringIO).  The Content-Length for text files is no
longer determined ahead of time, because the original logic could have been
wrong depending on the codec and newline translation settings.

Patch by Demian Brecht and Rolf Krahl, with a few tweaks by me.
2016-08-24 06:33:33 +00:00