to GET, so it loses its payload. So, it also must lose the
headers related to the payload (if it has no content any more,
it shouldn't indicate content length and type).
alone class. This addresses the primary concern in
http://bugs.python.org/issue1706815
python-dev discussion here:
http://mail.python.org/pipermail/python-dev/2007-July/073749.html
I chose IOError rather than EnvironmentError as the base class since
socket objects are often used as transparent duck typed file objects
in code already prepared to deal with IOError exceptions.
also a minor fix:
urllib2 - fix a couple places where IOError was raised rather than URLError.
for better or worse, URLError already inherits from IOError so
this won't break any existing code.
test_urllib2net - replace bad ftp urls.
with tests in test_urllib2net.py (must have network resource
enabled to execute them). Also modified test_urllib2.py because
testing mock classes must take it into acount. Docs are also
updated.
Python 2.5.
Also remove gopher support from urllib/urllib2. As both imported gopherlib the
usage of the support would have raised a DeprecationWarning.
base64.encodestring() for encoding authentication data.
encodestring() can include newlines for very long input, which
produced broken HTTP headers.
2.4 backport candidate, probably.
test suite.
For urllib2, move the import of gopherlib into the
only function that uses it: users (including the
test suite) certainly shouldn't see a deprecation
warning just because they import urllib2! If they
actually use gopher_open(), fine, _then_ they should
see a deprecation warning.
Pass the full URL to find_user_password(), in particular so that hosts
with port numbers can be looked up.
Also specify the digest algorithm, even if it's MD5. Titus Brown
verified that this fixes a problem with LiveJournal.
The change to use the newer httplib interface admitted the possibility
that we'd get an HTTP/1.1 chunked response, but the code didn't handle
it correctly. The raw socket object can't be pass to addinfourl(),
because it would read the undecoded response. Instead, addinfourl()
must call HTTPResponse.read(), which will handle the decoding.
One extra wrinkle is that the HTTPReponse object can't be passed to
addinfourl() either, because it doesn't implement readline() or
readlines(). As a quick hack, use socket._fileobject(), which
implements those methods on top of a read buffer. (suggested by mwh)
Finally, add some tests based on test_urllibnet.
Thanks to Andrew Sawyers for originally reporting the chunked problem.
Invoke the standard error handlers for non-200 responses.
Always supply a "Connection: close" header to prevent the server from
leaving the connection open. Downstream users of the socket may
attempt recv()/read() with no arguments, which would block if the
connection were kept open.
The chief benefit of this change is that requests will now use
HTTP/1.1 instead of HTTP/1.0. Bump the module version number as part
of the change.
There are two possible incompatibilities that we'll need to watch out
for when we get to an alpha release. We may get a different class of
exceptions out of httplib, and the do_open() method changed its
signature. The latter is only important if anyone actually subclasses
AbstractHTTPHandler.
John J. Lee writes: "the patch makes it possible to implement
functionality like HTTP cookie handling, Refresh handling,
etc. etc. using handler objects. At the moment urllib2's handler
objects aren't quite up to the job, which results in a lot of
cut-n-paste and subclassing. I believe the changes are
backwards-compatible, with the exception of people who've
reimplemented build_opener()'s functionality -- those people would
need to call opener.add_handler(HTTPErrorProcessor).
The main change is allowing handlers to implement
methods like:
http_request(request)
http_response(request, response)
In addition to the usual
http_open(request)
http_error{_*}(...)
"
Note that the change isn't well documented at least in part because
handlers aren't well documented at all. Need to fix this.
Add a bunch of new tests. It appears that none of these tests
actually use the network, so they don't need to be guarded by a
resource flag.
The patch was tweaked slightly. It's get a different mechanism for
generating the cnonce which uses /dev/urandom when possible to
generate less-easily-guessed random input.
Also rearrange the imports so that they are alphabetical and
duplicates are eliminated.
Add a few XXX comments about things left undone and things that could
be improved.
have to insert it in front of other classes, nor do dirty tricks like
inserting a "dummy" HTTPHandler after a ProxyHandler when building an
opener with proxy support.
The latest changes to the redirect handler couldn't possibly have been
tested, because they did not compute a newurl and failed with a
NameError. The __name__ == "__main__": block has a test for
redirects.
Also, fix SF bug 723831. A urlopen() that failed because the host was
not found raised a socket.gaierror unlike earlier versions of
urllib2. The problem is that httplib actually establishes the
connection at a different point starting with Python 2.2. Move the
try/except to endheaders(), which is where the connection gets
established.
Georg Brandl
Facundo Batista
Neal Norwitz
Gregory P. Smith
Brett Cannon
Tim Peters
Andrew M. Kuchling
Raymond Hettinger
Johannes Gijsbers
Jeremy Hylton
Anthony Baxter
Martin v. Löwis
Kurt B. Kaiser
Mark Hammond
Walter Dörwald
Gustavo Niemeyer