Commit Graph

162 Commits

Author SHA1 Message Date
Ashwin Ramaswami 9165addc22
bpo-38576: Disallow control characters in hostnames in http.client (GH-18995)
Add host validation for control characters for more CVE-2019-18348 protection.
2020-03-14 11:56:06 -07:00
Jason R. Coombs 7774d7831e
bpo-38216, bpo-36274: Allow subclasses to separately override validation and encoding behavior (GH-16448)
* bpo-38216: Allow bypassing input validation

* bpo-36274: Also allow the URL encoding to be overridden.

* bpo-38216, bpo-36274: Add tests demonstrating a hook for overriding validation, test demonstrating override encoding, and a test to capture expectation of the interface for the URL.

* Call with skip_host to avoid tripping on the host checking in the URL.

* Remove obsolete comment.

* Make _prepare_path_encoding its own attr.

This makes overriding just that simpler.

Also, don't use the := operator to make backporting easier.

* Add a news entry.

* _prepare_path_encoding -> _encode_prepared_path()

* Once again separate the path validation and request encoding, drastically simplifying the behavior. Drop the guarantee that all processing happens in _prepare_path.
2019-09-28 08:32:01 -04:00
Christian Heimes d1bd6e79da bpo-37440: Enable TLS 1.3 post-handshake auth in http.client (GH-14448)
Post-handshake authentication is required for conditional client cert authentication with TLS 1.3.


https://bugs.python.org/issue37440
2019-06-30 23:32:24 -07:00
Serhiy Storchaka 96aeaec647
bpo-36793: Remove unneeded __str__ definitions. (GH-13081)
Classes that define __str__ the same as __repr__ can
just inherit it from object.
2019-05-06 22:29:40 +03:00
Gregory P. Smith b7378d7728
bpo-30458: Use InvalidURL instead of ValueError. (GH-13044)
Use http.client.InvalidURL instead of ValueError as the new error case's exception.
2019-05-01 16:39:21 -04:00
Gregory P. Smith c4e671eec2
bpo-30458: Disallow control chars in http URLs. (GH-12755)
Disallow control chars in http URLs in urllib.urlopen.  This addresses a potential security problem for applications that do not sanity check their URLs where http request headers could be injected.
2019-04-30 19:12:21 -07:00
Inada Naoki d6bf6f2d0c
bpo-36050: optimize HTTPResponse.read() (GH-12698)
* No need to chunking for now.
* No need to partial read caused by EINTR for now.
2019-04-06 18:06:19 +09:00
Matt Houglum 461c416dd7 bpo-36522: Print all values for headers with multiple values. (GH-12681) 2019-04-04 07:36:47 +03:00
Marco Strigl 936f03e7fa bpo-33365: print the header values beside the keys (GH-6611)
with debuglevel=1 only the header keys got printed. With
this change the header values get printed as well and the single
header entries get '\n' as a separator.
2018-06-19 16:20:58 +03:00
Christian Heimes 61d478c71c
bpo-31399: Let OpenSSL verify hostname and IP address (#3462)
bpo-31399: Let OpenSSL verify hostname and IP

The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and
X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses.

* Remove match_hostname calls
* Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host()
* Add documentation for OpenSSL 1.0.2 requirement
* Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform.
* Add hostname_checks_common_name

Signed-off-by: Christian Heimes <christian@python.org>
2018-01-27 15:51:38 +01:00
Mike 53f7a7c281 bpo-32297: Few misspellings found in Python source code comments. (#4803)
* Fix multiple typos in code comments

* Add spacing in comments (test_logging.py, test_math.py)

* Fix spaces at the beginning of comments in test_logging.py
2017-12-14 13:04:53 +02:00
Nir Soffer ad455cd924 bpo-31945: Configurable blocksize in HTTP(S)Connection (#4279)
blocksize was hardcoded to 8192, preventing efficient upload when using
file-like body. Add blocksize argument to __init__, so users can
configure the blocksize to fit their needs.

I tested this uploading data from /dev/zero to a web server dropping the
received data, to test the overhead of the HTTPConnection.send() with a
file-like object.

Here is an example 10g upload with the default buffer size (8192):

$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 17.53 seconds (584.00m/s)

real	0m17.574s
user	0m8.887s
sys	0m5.971s

Same with 512k blocksize:

$ time ~/src/cpython/release/python upload-httplib.py 10 https://localhost:8000/
Uploaded 10.00g in 6.60 seconds (1551.15m/s)

real	0m6.641s
user	0m3.426s
sys	0m2.162s

In real world usage the difference will be smaller, depending on the
local and remote storage and the network.

See https://github.com/nirs/http-bench for more info.
2017-11-06 13:16:37 -08:00
remitamine a632d00a1c Remove duplicate line in Lib/http/client.py (#1665) 2017-05-19 15:28:35 +03:00
Serhiy Storchaka 2e576f5aec bpo-30144: Import collections ABC from collections.abc rather than collections. (#1263) 2017-04-24 09:05:00 +03:00
Serhiy Storchaka 70d28a184c Remove unused imports. 2016-12-16 20:00:15 +02:00
Martin Panter 741d4940fe Issue #23214: Remove BufferedReader.read1(-1) workaround 2016-10-21 00:52:04 +00:00
Christian Heimes d04863771b Issue #28022: Deprecate ssl-related arguments in favor of SSLContext.
The deprecation include manual creation of SSLSocket and certfile/keyfile
(or similar) in ftplib, httplib, imaplib, smtplib, poplib and urllib.

ssl.wrap_socket() is not marked as deprecated yet.
2016-09-10 23:23:33 +02:00
Eric V. Smith 451d0e38fc Issue 27948: Allow backslashes in the literal string portion of f-strings, but not in the expressions. Also, require expressions to begin and end with literal curly braces. 2016-09-09 21:56:20 -04:00
Senthil Kumaran 10427f4485 [merge from 3.5] - Issue28010 - Make http.client.HTTPConnection.putrequest
documentation consistent with the code.
2016-09-08 14:29:23 -07:00
Senthil Kumaran 5dc504c3c9 Issue28010 - Make http.client.HTTPConnection.putrequest documentation consistent with the code. 2016-09-08 14:28:01 -07:00
Martin Panter 0be894b2f6 Issue #27895: Spelling fixes (Contributed by Ville Skyttä). 2016-09-07 12:03:06 +00:00
R David Murray 44b548dda8 #27364: fix "incorrect" uses of escape character in the stdlib.
And most of the tools.

Patch by Emanual Barry, reviewed by me, Serhiy Storchaka, and
Martin Panter.
2016-09-08 13:59:53 -04:00
Eric V. Smith 6e025608a2 Issue 27921: Remove backslash from another f-string. 2016-09-03 10:43:20 -04:00
Raymond Hettinger 15f44ab043 Issue #27895: Spelling fixes (Contributed by Ville Skyttä). 2016-08-30 10:47:49 -07:00
Martin Panter ef91bb2660 Issue #12319: Always send file request bodies using chunked encoding
The previous attempt to determine the file’s Content-Length gave a false
positive for pipes on Windows.

Also, drop the special case for sending zero-length iterable bodies.
2016-08-27 01:39:26 +00:00
Martin Panter 3c0d0baf2b Issue #12319: Support for chunked encoding of HTTP request bodies
When the body object is a file, its size is no longer determined with
fstat(), since that can report the wrong result (e.g. reading from a pipe).
Instead, determine the size using seek(), or fall back to chunked encoding
for unseekable files.

Also, change the logic for detecting text files to check for TextIOBase
inheritance, rather than inspecting the “mode” attribute, which may not
exist (e.g. BytesIO and StringIO).  The Content-Length for text files is no
longer determined ahead of time, because the original logic could have been
wrong depending on the codec and newline translation settings.

Patch by Demian Brecht and Rolf Krahl, with a few tweaks by me.
2016-08-24 06:33:33 +00:00
Martin Panter c86c91aab0 Merge typo fixes from 3.5 2016-04-05 06:20:32 +00:00
Martin Panter cc71a795df Fix typos in documentation and comments 2016-04-05 06:19:42 +00:00
Martin Panter ec195fba5b Issue #26499: Merge HTTPResponse fix from 3.5 2016-03-17 07:05:34 +00:00
Martin Panter ce911c3fed Issue #26499: Fixes to HTTPResponse.readline() and read1(), by Silent Ghost 2016-03-17 06:42:48 +00:00
Martin Panter 192697e33b Issue #26045: Merge http.client error addition from 3.5 2016-02-09 11:57:11 +00:00
Martin Panter 44391481d7 Issue #26045: Add UTF-8 suggestion to error in http.client
Based on patch by Guido van Rossum.
2016-02-09 10:20:52 +00:00
Serhiy Storchaka 3fd4a735d8 Issue #25899: Converted non-ASCII characters in docstrings and manpage
to ASCII replacements.  Removed UTF-8 BOM from Misc/NEWS.
Original patch by Chris Angelico.
2015-12-18 13:10:37 +02:00
Raymond Hettinger 15b87bfedc Add in missing docstrings. 2015-08-18 22:03:08 -07:00
Serhiy Storchaka 2116b12da5 Issue #23865: close() methods in multiple modules now are idempotent and more
robust at shutdown. If needs to release multiple resources, they are released
even if errors are occured.
2015-04-10 13:29:28 +03:00
Serhiy Storchaka 7e7a3dba5f Issue #23865: close() methods in multiple modules now are idempotent and more
robust at shutdown. If needs to release multiple resources, they are released
even if errors are occured.
2015-04-10 13:24:41 +03:00
R David Murray cae7bdb424 #3566: Clean up handling of remote server disconnects.
This changeset does two things: introduces a new RemoteDisconnected exception
(that subclasses ConnectionResetError and BadStatusLine) so that a remote
server disconnection can be detected by client code (and provides a better
error message for debugging purposes), and ensures that the client socket is
closed if a ConnectionError happens, so that the automatic re-connection code
can work if the application handles the error and continues on.

Tests are added that confirm that a connection is re-used or not re-used
as appropriate to the various combinations of protocol version and headers.

Patch by Martin Panter, reviewed by Demian Brecht.  (Tweaked only slightly by
me.)
2015-04-05 19:26:29 -04:00
R David Murray 0a0d20edfb Merge: #23539: Set Content-Length to 0 for PUT, POST, and PATCH if body is None. 2015-03-22 15:19:01 -04:00
R David Murray beed8402ca #23539: Set Content-Length to 0 for PUT, POST, and PATCH if body is None.
Some http servers will reject PUT, POST, and PATCH requests if they
do not have a Content-Length header.

Patch by James Rutherford, with additional cleaning up of the
'request' documentation by me.
2015-03-22 15:18:23 -04:00
Serhiy Storchaka b669bfc2be Issue #22928: Disabled HTTP header injections in http.client.
Original patch by Demian Brecht.
2015-03-12 11:15:15 +02:00
Serhiy Storchaka a112a8ae47 Issue #22928: Disabled HTTP header injections in http.client.
Original patch by Demian Brecht.
2015-03-12 11:13:36 +02:00
Berker Peksag 8e28679417 Issue #23439: Add missing entries to http.client.__all__.
Also, document the LineTooLong exception since it can be raised by
the members of public API (e.g. http.client.HTTPResponse).

Patch by Martin Panter.
2015-02-20 09:45:05 +02:00
Berker Peksag babc688180 Issue #23439: Add missing entries to http.client.__all__.
Also, document the LineTooLong exception since it can be raised by
the members of public API (e.g. http.client.HTTPResponse).

Patch by Martin Panter.
2015-02-20 09:39:38 +02:00
Berker Peksag ab53ab0a84 Issue #13128: Print response headers for CONNECT requests when debuglevel > 0.
Patch by Demian Brecht.
2015-02-03 12:22:11 +02:00
Benjamin Peterson 9d8a3ad02a http.client: disable Nagle's algorithm (closes #23302)
Patch by Demian Brecht.
2015-01-23 11:02:57 -05:00
Serhiy Storchaka e4db76967d Issue #21793: Added http.HTTPStatus enums (i.e. HTTPStatus.OK,
HTTPStatus.NOT_FOUND).  Patch by Demian Brecht.
2014-12-23 16:28:28 +02:00
Serhiy Storchaka 4ac7ed97a8 Issue #22095: Fixed HTTPConnection.set_tunnel with default port. The port
value in the host header was set to "None".  Patch by Demian Brecht.
2014-12-12 09:29:15 +02:00
Benjamin Peterson a090f01bb6 HTTPSConnection: prefer the context's check_hostname attribute over the constructor parameter (#22959) 2014-12-07 13:18:25 -05:00
Serhiy Storchaka b491e0521f Issue #21032. Fixed socket leak if HTTPConnection.getresponse() fails.
Original patch by Martin Panter.
2014-12-01 13:07:45 +02:00
Benjamin Peterson 7243b574e5 don't require OpenSSL SNI to pass hostname to ssl functions (#22921)
Patch by Donald Stufft.
2014-11-23 17:04:34 -06:00