Commit Graph

86 Commits

Author SHA1 Message Date
Jeremy Hylton 2c178253bd SF bug 874842 and patch 997626: httplib bugs
Hack httplib to work with broken Akamai proxies.
Make sure that httplib doesn't add extract Accept-Encoding or
Content-Length headers if the client has already set them.
2004-08-07 16:28:14 +00:00
Hye-Shik Chang 39aef79821 Fix a bug that robotparser starves memory when the server responses
in HTTP/0.9 due to dissonance of httplib.LineAndFileWrapper and
urllib.addbase.
2004-06-05 13:30:56 +00:00
Raymond Hettinger bac788a3cd Replace str.find()!=1 with the more readable "in" operator. 2004-05-04 09:21:43 +00:00
Tim Peters 58eb11cf62 Whitespace normalization. 2004-01-18 20:29:55 +00:00
Martin v. Löwis af7dc8d8b8 Patch #831747: Add skip_accept_encoding parameter to putrequest. 2003-11-19 19:51:55 +00:00
Alex Martelli 0c5b4ad8f2 fixed obvious bug in _send_header as per SF bug #831271
(same as commit of Sun Nov 2 to the release23-maint branch)
2003-11-09 16:41:38 +00:00
Martin v. Löwis 11892ecd6d Patch #817854: Add missing operations for SSLFile. Fixes #792101.
Backported to 2.3.
2003-10-27 14:07:53 +00:00
Jeremy Hylton b67695238d Fix sf bug 666219: assertion error in httplib.
The obvious way for this assertion to fail is if the LineAndFileWrapper constructor is called when an empty line.  Raise a BadStatusError before the call.
2003-06-29 17:55:05 +00:00
Greg Stein 616a58d79a Deal with a couple XXX comments which asked questions.
In response to "shouldn't the client close the file?", the answer is
"no". The original design behind HTTPConnection is that the client did
not have to worry about it. The response would close itself when you
read the last of the data from it. This closing also dealt with
allowing the connection to perform another request/response (if it was
a persistent connection).

However... the auto-close behavior broke compatibility with the
classic httplib.HTTP class' behavior when a zero-length response body
was present. In that situation, the HTTPResponse object was
auto-closing it since there was no data present, and for an HTTP/1.0
connection-close socket (or an HTTP/0.9 request) connection, that also
ended up closing the socket. When an httplib.HTTP user went to read
the socket... boom. A patch to correct the auto-close (for compat with
old httplib users) was added in rev 1.22.

But for non-zero-length *chunked* bodies, we should keep the
auto-close behavior. The library user is not reading the socket (they
can't cuz of the chunked response we just got done handling), so they
should be immune to the response closing the socket. In fact, I would
like to see (one day) the auto-close restored, and the HTTP subclass
would simply have a flag to disable that behavior (for back-compat
purposes).
2003-06-24 06:35:19 +00:00
Martin v. Löwis 1867f24416 Always unwrap _socketobj in socket.ssl. Revert httplib.py 1.25.
Fixes #754447.
2003-06-14 13:30:53 +00:00
Tim Peters c2659cff5d Whitespace normalization. 2003-05-12 20:19:37 +00:00
Jeremy Hylton c1b2cb9d8f SF bug 622042: Don't expect response body from HEAD request.
Bug fix candidate.
2003-05-05 16:13:58 +00:00
Martin v. Löwis 2548c730c1 Implement IDNA (Internationalized Domain Names in Applications). 2003-04-18 10:39:54 +00:00
Raymond Hettinger 4922768d02 Reverted the previous change to read() and readline().
Kevin Jacobs found that the code simplification did not
exactly match the semantics of the original.  Regression
test cases were requested.
2003-03-06 16:31:48 +00:00
Raymond Hettinger b2e0b92ef1 Module review:
* Replaced "while 1" with "while True"
* Rewrote read() and readline() for clarity and speed.
* Replaced variable 'list' with 'hlist'
* Used augmented assignment in two places.
2003-02-26 22:45:18 +00:00
Skip Montanaro 74b9a7a71f typo 2003-02-25 17:48:15 +00:00
Tim Peters 77c06fbf94 Whitespace normalization. 2002-11-24 02:35:35 +00:00
Jeremy Hylton 77628658ca Remove some test code.
The buggy yahoo server was fixed, and the opalgroup test files are
gone.
2002-11-13 17:30:32 +00:00
Jeremy Hylton 22b3a49d3c Fix SF bug #637789: Handle Proxy-Connection header.
Also, remove unused local variable noted by pychecker.
2002-11-13 17:27:43 +00:00
Tim Peters 230a60c6ec Whitespace normalization. 2002-11-09 05:08:07 +00:00
Jeremy Hylton d4c472c3e2 Move code for reading chunked responses in helper function,
along with some small changes (e.g. use of +=).
2002-09-03 20:49:06 +00:00
Jeremy Hylton d229b3ae04 SF bug [ 600488 ] Robustness tweak to httplib.py
If the transfer-encoding is unknown, ignore it.  Suggested by Tom
Emerson.
2002-09-03 19:24:24 +00:00
Tim Peters 469cdad822 Whitespace normalization. 2002-08-08 20:19:19 +00:00
Skip Montanaro 186bec2f8d typo 2002-07-25 16:10:38 +00:00
Jeremy Hylton e3252ec6cf Fix typos and such caught by the pycheckerbot. 2002-07-16 21:41:43 +00:00
Tim Peters c411dbaeee Whitespace normalization. 2002-07-16 21:35:23 +00:00
Jeremy Hylton 8531b1b28d Send HTTP requests with a single send() call instead of many.
The implementation now stores all the lines of the request in a buffer
and makes a single send() call when the request is finished,
specifically when endheaders() is called.

This appears to improve performance.  The old code called send() for
each line.  The sends are all short, so they caused bad interactions
with the Nagle algorithm and delayed acknowledgements.  In simple
tests, the second packet was delayed by 100s of ms.  The second send was
delayed by the Nagle algorithm, waiting for the ack.  The delayed ack
strategy delays the ack in hopes of piggybacking it on a data packet,
but the server won't send any data until it receives the complete
request.

This change minimizes the problem that Nagle + delayed ack will cause
a problem, although a request large enough to be broken into two
packets will still suffer some delay.  Luckily the MSS is large enough
to accomodate most single packets.

XXX Bug fix candidate?
2002-07-16 21:21:11 +00:00
Jeremy Hylton 39c03808c7 Change _begin() back to begin().
Client code could create responses explicitly.
2002-07-12 14:04:09 +00:00
Jeremy Hylton 29d27ac4fe Fix for SF bug 579107.
The recent SSL changes resulted in important, but subtle changes to
close() semantics.  Since builtin socket makefile() is not called for
SSL connections, we don't get separately closeable fds for connection
and response.  Comments in the code explain how to restore makefile
semantics.

Bug fix candidate.
2002-07-09 21:22:36 +00:00
Jeremy Hylton 6d0a4c79cf Fix for SF bug #432621: httplib: multiple Set-Cookie headers
If multiple header fields with the same name occur, they are combined
according to the rules in RFC 2616 sec 4.2:

Appending each subsequent field-value to the first, each separated by
a comma. The order in which header fields with the same field-name are
received is significant to the interpretation of the combined field
value.
2002-07-07 16:51:37 +00:00
Jeremy Hylton 12f4f35f6e Fix SF bug #575360
Subclasses of Exception that define an __init__ must call
Exception.__init__ or define self.args.  Otherwise, str() will fail.

Bug fix candidate.
2002-07-06 18:55:01 +00:00
Jeremy Hylton d46aa37d35 Handle HTTP/0.9 responses.
Section 19.6 of RFC 2616 (HTTP/1.1):

   It is beyond the scope of a protocol specification to mandate
   compliance with previous versions. HTTP/1.1 was deliberately
   designed, however, to make supporting previous versions easy....

   And we would expect HTTP/1.1 clients to:

      - recognize the format of the Status-Line for HTTP/1.0 and 1.1
        responses;

      - understand any valid response in the format of HTTP/0.9, 1.0, or
        1.1.

The changes to the code do handle response in the format of HTTP/0.9.
Some users may consider this a bug because all responses with a
sufficiently corrupted status line will look like an HTTP/0.9
response.  These users can pass strict=1 to the HTTP constructors to
get a BadStatusLine exception instead.

While this is a new feature of sorts, it enhances the robustness of
the code (be tolerant in what you accept).  Thus, I consider it a bug
fix candidate.

XXX strict needs to be documented.
2002-07-06 18:48:07 +00:00
Jeremy Hylton fbd79944a8 Convert raise to call exception class. Add whitespace. 2002-07-02 20:19:08 +00:00
Jeremy Hylton 7c75c99a10 Simplify HTTPSConnection constructor.
See discussion in SF bug 458463.
2002-06-28 23:38:14 +00:00
Jeremy Hylton be4fcf1875 Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
The HTTPResponse class now handles 100 continue responses, instead of
choking on them.  It detects them internally in the _begin() method
and ignores them.  Based on a patch by Bob Kline.

This closes SF bugs 498149 and 551273.

The FakeSocket class (for SSL) is now usable with HTTP/1.1
connections.  The old version of the code could not work with
persistent connections, because the makefile() implementation read
until EOF before returning.  If the connection is persistent, the
server sends a response and leaves the connection open.  A client that
reads until EOF will block until the server gives up on the connection
-- more than a minute in my test case.

The problem was fixed by implementing a reasonable makefile().  It
reads data only when it is needed by the layers above it.  It's
implementation uses an internal buffer with a default size of 8192.

Also, rename begin() method of HTTPResponse to _begin() because it
should only be called by the HTTPConnection.
2002-06-28 22:38:01 +00:00
Raymond Hettinger 54f0222547 SF 563203. Replaced 'has_key()' with 'in'. 2002-06-01 14:18:47 +00:00
Martin v. Löwis 6af3e2dc31 Forward port of patch # 500311: Work around for buggy https servers.
Fixes #494762.
2002-04-20 07:47:40 +00:00
Skip Montanaro f3f87f743e oops - export InvalidURL in __all__ 2002-03-24 16:56:45 +00:00
Skip Montanaro 9d38997e8c add InvalidURL exception - raised if port is given but empty or non-numeric 2002-03-24 16:53:50 +00:00
Andrew M. Kuchling a3c0b9334e [Bug #531616] Make HTTPS work again by adding a sendall method to the
FakeSocket class.  Without it, the sendall() call got the method on
the underlying socket object, and that messed up SSL.

Does httplib use other methods of sockets that FakeSocket doesn't support?
Someone should take a look...  (I'll try to give it a once-over.)

2.2.1 bugfix candidate.
2002-03-18 22:51:48 +00:00
Jeremy Hylton 3921ff675e Fix SF bug 525520.
Don't automatically add a Host: header if the headers passed to
request() already has a Host key.
2002-03-09 06:07:23 +00:00
Jeremy Hylton 8acf1e0e01 SF bug report #405939: wrong Host header with proxy
In August, Greg said this looked good, so I'm going ahead with it.

The fix is different from the one in the bug report.  Instead of using
a regular expression to extract the host from the url, I use
urlparse.urlsplit.

Martin commented that the patch doesn't address URLs that have basic
authentication username and password in the header.  I don't see any
code anywhere in httplib that supports this feature, so I'm not going
to address it for this fix.

Bug fix candidate.
2002-03-08 19:35:51 +00:00
Martin v. Löwis e12454f44a The Grande 'sendall()' patch, copied from release21-maint. Fixes #516715.
Replaces calls to socket.send() (which isn't guaranteed to send all data)
with the new socket.sendall() method.
2002-02-16 23:06:19 +00:00
Neal Norwitz 22c5d77d47 SF #515011, cleanup: remove "or 0" condition 2002-02-11 17:59:51 +00:00
Tim Peters f3623f310e Somebody checked in a version of httplib that doesn't even compile --
SyntaxError.  Fix it.
2001-10-11 18:15:51 +00:00
Jeremy Hylton 6459c8d067 Fix for SF buf #458835
Try to be systematic about dealing with socket and ssl exceptions in
FakeSocket.makefile().  The previous version of the code caught all
ssl errors and treated them as EOF, even though most of the errors
don't mean EOF.

An SSL error can mean on of three things:

    1. The SSL/TLS connection was closed.
    2. The operation should be retried.
    3. An error occurred.

Also, if a socket error occurred and the error was EINTR, retry the
call.  Otherwise, it was a legitimate error and the caller should
receive the exception.
2001-10-11 17:47:22 +00:00
Martin v. Löwis 322c0d187d Only close sockets if they have been created. Reported by Blake Winton. 2001-10-07 08:53:32 +00:00
Greg Stein 81937a4a12 Resolve patch #449367.
For the HTTPS class (when available), ensure that the x509 certificate data
gets passed through to the HTTPSConnection class. Create a new
HTTPS.__init__ to do this, and refactor the HTTP.__init__ into a new _setup
method for both init's to call.

Note: this is solved differently from the patch, which advocated a new
**x509 parameter on the base HTTPConnection class. But that would open
HTTPConnection to arbitrary (ignored) parameters, so was not as desirable.
2001-08-18 09:20:23 +00:00
Martin v. Löwis 2ad2569c72 Initialize msg to avoid unbound locals. 2001-07-31 08:40:21 +00:00
Martin v. Löwis 4eb5940a4d Untabify IPv6 changes. 2001-07-26 13:37:33 +00:00