Commit Graph

136 Commits

Author SHA1 Message Date
Jeremy Hylton 2c178253bd SF bug 874842 and patch 997626: httplib bugs
Hack httplib to work with broken Akamai proxies.
Make sure that httplib doesn't add extract Accept-Encoding or
Content-Length headers if the client has already set them.
2004-08-07 16:28:14 +00:00
Hye-Shik Chang 39aef79821 Fix a bug that robotparser starves memory when the server responses
in HTTP/0.9 due to dissonance of httplib.LineAndFileWrapper and
urllib.addbase.
2004-06-05 13:30:56 +00:00
Raymond Hettinger bac788a3cd Replace str.find()!=1 with the more readable "in" operator. 2004-05-04 09:21:43 +00:00
Tim Peters 58eb11cf62 Whitespace normalization. 2004-01-18 20:29:55 +00:00
Martin v. Löwis af7dc8d8b8 Patch #831747: Add skip_accept_encoding parameter to putrequest. 2003-11-19 19:51:55 +00:00
Alex Martelli 0c5b4ad8f2 fixed obvious bug in _send_header as per SF bug #831271
(same as commit of Sun Nov 2 to the release23-maint branch)
2003-11-09 16:41:38 +00:00
Martin v. Löwis 11892ecd6d Patch #817854: Add missing operations for SSLFile. Fixes #792101.
Backported to 2.3.
2003-10-27 14:07:53 +00:00
Jeremy Hylton b67695238d Fix sf bug 666219: assertion error in httplib.
The obvious way for this assertion to fail is if the LineAndFileWrapper constructor is called when an empty line.  Raise a BadStatusError before the call.
2003-06-29 17:55:05 +00:00
Greg Stein 616a58d79a Deal with a couple XXX comments which asked questions.
In response to "shouldn't the client close the file?", the answer is
"no". The original design behind HTTPConnection is that the client did
not have to worry about it. The response would close itself when you
read the last of the data from it. This closing also dealt with
allowing the connection to perform another request/response (if it was
a persistent connection).

However... the auto-close behavior broke compatibility with the
classic httplib.HTTP class' behavior when a zero-length response body
was present. In that situation, the HTTPResponse object was
auto-closing it since there was no data present, and for an HTTP/1.0
connection-close socket (or an HTTP/0.9 request) connection, that also
ended up closing the socket. When an httplib.HTTP user went to read
the socket... boom. A patch to correct the auto-close (for compat with
old httplib users) was added in rev 1.22.

But for non-zero-length *chunked* bodies, we should keep the
auto-close behavior. The library user is not reading the socket (they
can't cuz of the chunked response we just got done handling), so they
should be immune to the response closing the socket. In fact, I would
like to see (one day) the auto-close restored, and the HTTP subclass
would simply have a flag to disable that behavior (for back-compat
purposes).
2003-06-24 06:35:19 +00:00
Martin v. Löwis 1867f24416 Always unwrap _socketobj in socket.ssl. Revert httplib.py 1.25.
Fixes #754447.
2003-06-14 13:30:53 +00:00
Tim Peters c2659cff5d Whitespace normalization. 2003-05-12 20:19:37 +00:00
Jeremy Hylton c1b2cb9d8f SF bug 622042: Don't expect response body from HEAD request.
Bug fix candidate.
2003-05-05 16:13:58 +00:00
Martin v. Löwis 2548c730c1 Implement IDNA (Internationalized Domain Names in Applications). 2003-04-18 10:39:54 +00:00
Raymond Hettinger 4922768d02 Reverted the previous change to read() and readline().
Kevin Jacobs found that the code simplification did not
exactly match the semantics of the original.  Regression
test cases were requested.
2003-03-06 16:31:48 +00:00
Raymond Hettinger b2e0b92ef1 Module review:
* Replaced "while 1" with "while True"
* Rewrote read() and readline() for clarity and speed.
* Replaced variable 'list' with 'hlist'
* Used augmented assignment in two places.
2003-02-26 22:45:18 +00:00
Skip Montanaro 74b9a7a71f typo 2003-02-25 17:48:15 +00:00
Tim Peters 77c06fbf94 Whitespace normalization. 2002-11-24 02:35:35 +00:00
Jeremy Hylton 77628658ca Remove some test code.
The buggy yahoo server was fixed, and the opalgroup test files are
gone.
2002-11-13 17:30:32 +00:00
Jeremy Hylton 22b3a49d3c Fix SF bug #637789: Handle Proxy-Connection header.
Also, remove unused local variable noted by pychecker.
2002-11-13 17:27:43 +00:00
Tim Peters 230a60c6ec Whitespace normalization. 2002-11-09 05:08:07 +00:00
Jeremy Hylton d4c472c3e2 Move code for reading chunked responses in helper function,
along with some small changes (e.g. use of +=).
2002-09-03 20:49:06 +00:00
Jeremy Hylton d229b3ae04 SF bug [ 600488 ] Robustness tweak to httplib.py
If the transfer-encoding is unknown, ignore it.  Suggested by Tom
Emerson.
2002-09-03 19:24:24 +00:00
Tim Peters 469cdad822 Whitespace normalization. 2002-08-08 20:19:19 +00:00
Skip Montanaro 186bec2f8d typo 2002-07-25 16:10:38 +00:00
Jeremy Hylton e3252ec6cf Fix typos and such caught by the pycheckerbot. 2002-07-16 21:41:43 +00:00
Tim Peters c411dbaeee Whitespace normalization. 2002-07-16 21:35:23 +00:00
Jeremy Hylton 8531b1b28d Send HTTP requests with a single send() call instead of many.
The implementation now stores all the lines of the request in a buffer
and makes a single send() call when the request is finished,
specifically when endheaders() is called.

This appears to improve performance.  The old code called send() for
each line.  The sends are all short, so they caused bad interactions
with the Nagle algorithm and delayed acknowledgements.  In simple
tests, the second packet was delayed by 100s of ms.  The second send was
delayed by the Nagle algorithm, waiting for the ack.  The delayed ack
strategy delays the ack in hopes of piggybacking it on a data packet,
but the server won't send any data until it receives the complete
request.

This change minimizes the problem that Nagle + delayed ack will cause
a problem, although a request large enough to be broken into two
packets will still suffer some delay.  Luckily the MSS is large enough
to accomodate most single packets.

XXX Bug fix candidate?
2002-07-16 21:21:11 +00:00
Jeremy Hylton 39c03808c7 Change _begin() back to begin().
Client code could create responses explicitly.
2002-07-12 14:04:09 +00:00
Jeremy Hylton 29d27ac4fe Fix for SF bug 579107.
The recent SSL changes resulted in important, but subtle changes to
close() semantics.  Since builtin socket makefile() is not called for
SSL connections, we don't get separately closeable fds for connection
and response.  Comments in the code explain how to restore makefile
semantics.

Bug fix candidate.
2002-07-09 21:22:36 +00:00
Jeremy Hylton 6d0a4c79cf Fix for SF bug #432621: httplib: multiple Set-Cookie headers
If multiple header fields with the same name occur, they are combined
according to the rules in RFC 2616 sec 4.2:

Appending each subsequent field-value to the first, each separated by
a comma. The order in which header fields with the same field-name are
received is significant to the interpretation of the combined field
value.
2002-07-07 16:51:37 +00:00
Jeremy Hylton 12f4f35f6e Fix SF bug #575360
Subclasses of Exception that define an __init__ must call
Exception.__init__ or define self.args.  Otherwise, str() will fail.

Bug fix candidate.
2002-07-06 18:55:01 +00:00
Jeremy Hylton d46aa37d35 Handle HTTP/0.9 responses.
Section 19.6 of RFC 2616 (HTTP/1.1):

   It is beyond the scope of a protocol specification to mandate
   compliance with previous versions. HTTP/1.1 was deliberately
   designed, however, to make supporting previous versions easy....

   And we would expect HTTP/1.1 clients to:

      - recognize the format of the Status-Line for HTTP/1.0 and 1.1
        responses;

      - understand any valid response in the format of HTTP/0.9, 1.0, or
        1.1.

The changes to the code do handle response in the format of HTTP/0.9.
Some users may consider this a bug because all responses with a
sufficiently corrupted status line will look like an HTTP/0.9
response.  These users can pass strict=1 to the HTTP constructors to
get a BadStatusLine exception instead.

While this is a new feature of sorts, it enhances the robustness of
the code (be tolerant in what you accept).  Thus, I consider it a bug
fix candidate.

XXX strict needs to be documented.
2002-07-06 18:48:07 +00:00
Jeremy Hylton fbd79944a8 Convert raise to call exception class. Add whitespace. 2002-07-02 20:19:08 +00:00
Jeremy Hylton 7c75c99a10 Simplify HTTPSConnection constructor.
See discussion in SF bug 458463.
2002-06-28 23:38:14 +00:00
Jeremy Hylton be4fcf1875 Fixes for two separate HTTP/1.1 bugs: 100 responses and HTTPS connections.
The HTTPResponse class now handles 100 continue responses, instead of
choking on them.  It detects them internally in the _begin() method
and ignores them.  Based on a patch by Bob Kline.

This closes SF bugs 498149 and 551273.

The FakeSocket class (for SSL) is now usable with HTTP/1.1
connections.  The old version of the code could not work with
persistent connections, because the makefile() implementation read
until EOF before returning.  If the connection is persistent, the
server sends a response and leaves the connection open.  A client that
reads until EOF will block until the server gives up on the connection
-- more than a minute in my test case.

The problem was fixed by implementing a reasonable makefile().  It
reads data only when it is needed by the layers above it.  It's
implementation uses an internal buffer with a default size of 8192.

Also, rename begin() method of HTTPResponse to _begin() because it
should only be called by the HTTPConnection.
2002-06-28 22:38:01 +00:00
Raymond Hettinger 54f0222547 SF 563203. Replaced 'has_key()' with 'in'. 2002-06-01 14:18:47 +00:00
Martin v. Löwis 6af3e2dc31 Forward port of patch # 500311: Work around for buggy https servers.
Fixes #494762.
2002-04-20 07:47:40 +00:00
Skip Montanaro f3f87f743e oops - export InvalidURL in __all__ 2002-03-24 16:56:45 +00:00
Skip Montanaro 9d38997e8c add InvalidURL exception - raised if port is given but empty or non-numeric 2002-03-24 16:53:50 +00:00
Andrew M. Kuchling a3c0b9334e [Bug #531616] Make HTTPS work again by adding a sendall method to the
FakeSocket class.  Without it, the sendall() call got the method on
the underlying socket object, and that messed up SSL.

Does httplib use other methods of sockets that FakeSocket doesn't support?
Someone should take a look...  (I'll try to give it a once-over.)

2.2.1 bugfix candidate.
2002-03-18 22:51:48 +00:00
Jeremy Hylton 3921ff675e Fix SF bug 525520.
Don't automatically add a Host: header if the headers passed to
request() already has a Host key.
2002-03-09 06:07:23 +00:00
Jeremy Hylton 8acf1e0e01 SF bug report #405939: wrong Host header with proxy
In August, Greg said this looked good, so I'm going ahead with it.

The fix is different from the one in the bug report.  Instead of using
a regular expression to extract the host from the url, I use
urlparse.urlsplit.

Martin commented that the patch doesn't address URLs that have basic
authentication username and password in the header.  I don't see any
code anywhere in httplib that supports this feature, so I'm not going
to address it for this fix.

Bug fix candidate.
2002-03-08 19:35:51 +00:00
Martin v. Löwis e12454f44a The Grande 'sendall()' patch, copied from release21-maint. Fixes #516715.
Replaces calls to socket.send() (which isn't guaranteed to send all data)
with the new socket.sendall() method.
2002-02-16 23:06:19 +00:00
Neal Norwitz 22c5d77d47 SF #515011, cleanup: remove "or 0" condition 2002-02-11 17:59:51 +00:00
Tim Peters f3623f310e Somebody checked in a version of httplib that doesn't even compile --
SyntaxError.  Fix it.
2001-10-11 18:15:51 +00:00
Jeremy Hylton 6459c8d067 Fix for SF buf #458835
Try to be systematic about dealing with socket and ssl exceptions in
FakeSocket.makefile().  The previous version of the code caught all
ssl errors and treated them as EOF, even though most of the errors
don't mean EOF.

An SSL error can mean on of three things:

    1. The SSL/TLS connection was closed.
    2. The operation should be retried.
    3. An error occurred.

Also, if a socket error occurred and the error was EINTR, retry the
call.  Otherwise, it was a legitimate error and the caller should
receive the exception.
2001-10-11 17:47:22 +00:00
Martin v. Löwis 322c0d187d Only close sockets if they have been created. Reported by Blake Winton. 2001-10-07 08:53:32 +00:00
Greg Stein 81937a4a12 Resolve patch #449367.
For the HTTPS class (when available), ensure that the x509 certificate data
gets passed through to the HTTPSConnection class. Create a new
HTTPS.__init__ to do this, and refactor the HTTP.__init__ into a new _setup
method for both init's to call.

Note: this is solved differently from the patch, which advocated a new
**x509 parameter on the base HTTPConnection class. But that would open
HTTPConnection to arbitrary (ignored) parameters, so was not as desirable.
2001-08-18 09:20:23 +00:00
Martin v. Löwis 2ad2569c72 Initialize msg to avoid unbound locals. 2001-07-31 08:40:21 +00:00
Martin v. Löwis 4eb5940a4d Untabify IPv6 changes. 2001-07-26 13:37:33 +00:00
Martin v. Löwis a43c2f845e Patch #401196: Use getaddrinfo and AF_INET6 in TCP servers and clients. 2001-07-24 20:34:08 +00:00
Skip Montanaro 951a8841d1 more public symbols for __all__ 2001-06-01 16:25:38 +00:00
Jeremy Hylton 23d4047790 SF patch #405845 by Martin von Löwis
Fixes SF bug #405427.
If an http response has a bogus return code, e.g. 400.100, raise
BadStatusLine.
2001-04-13 14:57:08 +00:00
Jeremy Hylton 42dd01add5 An ssl-wrapped socket now returns '' on EOF, just like a regular
socket -- as suggested by Clarence Gardner.

Fix httplib to comply with the new ssl-socket interface.
2001-02-01 23:35:20 +00:00
Skip Montanaro 2dd4276559 added a few more __all__ lists
fixed typo in ihooks docstring
2001-01-23 15:35:05 +00:00
Guido van Rossum fd97a919ff Revert a change I accidentally checked in together with Ping's
"smallest patch ever".
2001-01-15 14:34:20 +00:00
Tim Peters 07e99cb774 Whitespace normalization. 2001-01-14 23:47:14 +00:00
Guido van Rossum f6922aa435 SF Patch #103232 by dougfort: Preserve Nonstandard Port Number in Host
Header

Dougfort's comments: httplib does not include ':port ' in the HTTP 1.1
'Host:' header.  This causes problems if the server is not listening
on Port 80.  The test case I use is the login to /manage under Zope,
with Zope listening on port 8080. Zope returns a <frameset> with the
<frame> source URLs lacking the :8080.
2001-01-14 21:03:01 +00:00
Tim Peters 5ceadc8cba No text file relying on significant trailing whitespace is robust under
modification.  Removed the need for that.
2001-01-13 19:16:21 +00:00
Guido van Rossum b2825205a2 SF Patch #103225 by Ping: httplib: smallest Python patch ever
The ASCII-art diagram at the top of httplib contains a backslash at
  the end of a line, which causes Python to remove the newline. This
  one-character patch adds a space after the backslash so it will
  appear at the end of the line in the docstring as intended.
2001-01-13 16:55:33 +00:00
Guido van Rossum 34735a6682 Get rid of string functions.
There should really be a little tool to help with this -- it's rather
tedious and there are lots of special cases!
2000-12-15 15:09:42 +00:00
Guido van Rossum 0aee7220db Hoepeful fix for SF bug #123924: Windows - using OpenSSL, problem with
socket in httplib.py.

The bug reports that on Windows, you must pass sock._sock to the
socket.ssl() call.  But on Unix, you must pass sock itself.  (sock is
a wrapper on Windows but not on Unix; the ssl() call wants the real
socket object, not the wrapper.)

So we see if sock has an _sock attribute and if so, extract it.

Unfortunately, the submitter of the bug didn't confirm that this patch
works, so I'll just have to believe it (can't test it myself since I
don't have OpenSSL on Windows set up, and that's a nontrivial thing I
believe).
2000-12-11 20:32:20 +00:00
Jeremy Hylton 110941a4ba If the status line is invalid, assume it is a pre-1.0 response. The
msg/headers are empty and the entire response is treated as the body.
2000-10-12 19:58:36 +00:00
Martin v. Löwis d7bf974af4 Indent _connection_class so that it becomes HTTPS._connection_class. 2000-09-21 22:09:47 +00:00
Jeremy Hylton 30f867423a Do not close socket when a Content-Length is 0. This make the
interface consistent: The client is responsible for closing the
socket, regardless of the amount of data received.

Restore suport for set_debuglevel call.
2000-09-18 22:50:38 +00:00
Jeremy Hylton 30a818196e cope with weird Content-Length values returned from servers by
ignoring them; e.g. Zope sometimes returns 13497L
2000-09-14 20:34:27 +00:00
Jeremy Hylton 4d746fca3d add docstring explaining makefile limitation
fix support for passing http version to connect in HTTP
2000-08-23 20:34:17 +00:00
Jeremy Hylton 29b8d5acd0 add support for HTTPS
Modify HTTP to use delegation instead of inheritance.  The
_connection_class attribute of the class defines what class to
delegate to.  The HTTPS class is a subclass of HTTP that redefines
_connection_class.
2000-08-01 17:33:32 +00:00
Greg Stein dd6eefb348 no changes other than indentation level (now 4) and comment reflow.
use "cvs diff -b" to verify.
2000-07-18 09:09:48 +00:00
Greg Stein 5e0fa4065f initial commit of a new HTTP library, supporting HTTP/1.1 and persistent
connections.
2000-06-26 08:28:01 +00:00
Guido van Rossum 974f70d97b Add call to putheader('Host', 'www.python.org') to the example. 2000-05-19 23:06:45 +00:00
Guido van Rossum 93a7c0fe6b Fredrik Lundh:
This fixes a bunch of socket.connect(host, post) calls.  Note that I
haven't tested all modules -- I don't have enough servers here...
2000-03-28 21:45:46 +00:00
Fred Drake 13a2c279c5 Untabify to pass the -tt test. 2000-02-10 17:17:14 +00:00
Guido van Rossum 09c8b6c3e4 OpenSSL support. This is based on patches for a version of SSLeay by
Brian E Gallew, which were improved and adapted to OpenSSL 0.9.4 by
Laszlo Kovacs of HP.  Both have kindly given permission to include
the patches in the Python distribution.  Final formatting by GvR.
1999-12-07 21:37:17 +00:00
Guido van Rossum 29c4688659 Patch by Tim O'Malley for servers that send a response looking just like
HTTP/1.x 200
instead of
    HTTP/1.x 200 OK
1998-01-19 22:25:24 +00:00
Guido van Rossum 41999c164e Doc strings by Mitch Chapman (with a little reformatting).
Also reformatted the whole module with 4 spaces and no tabs.
1997-12-09 00:12:23 +00:00
Guido van Rossum 2567dd6d44 Two changes:
(1) No longer close self.sock; close it on close().  (Guido)

(2) Don't use regular expressions for what can be done simply with
string.split() -- regex is thread unsafe.  (Jeremy)

(3) Delete unused imports.  (Jeremy)
1997-08-11 21:52:17 +00:00
Guido van Rossum 8b82baa398 In the HTTP reply header, be more accepting in the HTTP version. 1996-12-10 15:58:22 +00:00
Jack Jansen 2bb57b81f2 Changed makefile() args from r/w to rb/wb, for non-unix compatability. 1996-02-14 16:06:24 +00:00
Guido van Rossum 928fcede65 actualized example/reference, fix bug w/ nonnumeric port 1995-09-30 16:50:46 +00:00
Guido van Rossum a0dfc7ad65 fixed the test program 1995-09-07 19:28:19 +00:00
Guido van Rossum 65ab98c427 use mimetools; add close() 1995-08-07 20:13:02 +00:00
Guido van Rossum 4cdcef7e29 discard endrequest(); minor stuff; rfc822 no seek flag 1995-06-22 18:48:48 +00:00
Guido van Rossum 2922c6dabb Changes to use default argument values where possible 1994-05-06 14:28:19 +00:00
Guido van Rossum 76ca3c17f0 Fix bug with somtimes uninitialized port 1994-02-22 16:06:02 +00:00
Guido van Rossum 23acc9590c Towards a standard access mechanism for URLs. 1994-02-21 16:36:04 +00:00