a search path setup, some of these hosts resolve to the wrong address.
By appending a period to the hostname, the hostname should only resolve
to what we want it to resolve to. Hopefully this doesn't break different bots.
Also add more info to failure message to aid debugging test failure.
a search path setup, some of these hosts resolve to the wrong address.
By appending a period to the hostname, the hostname should only resolve
to what we want it to resolve to. Hopefully this doesn't break different bots.
The change to use the newer httplib interface admitted the possibility
that we'd get an HTTP/1.1 chunked response, but the code didn't handle
it correctly. The raw socket object can't be pass to addinfourl(),
because it would read the undecoded response. Instead, addinfourl()
must call HTTPResponse.read(), which will handle the decoding.
One extra wrinkle is that the HTTPReponse object can't be passed to
addinfourl() either, because it doesn't implement readline() or
readlines(). As a quick hack, use socket._fileobject(), which
implements those methods on top of a read buffer. (suggested by mwh)
Finally, add some tests based on test_urllibnet.
Thanks to Andrew Sawyers for originally reporting the chunked problem.
Neal Norwitz
Georg Brandl
Jeremy Hylton