Commit Graph

265 Commits

Author SHA1 Message Date
Senthil Kumaran 4af40d2173 Fix for Issue1648102, based on the MSDN spec: If this parameter specifies the
"<local>" macro as the only entry, this function bypasses any host name that
does not contain a period.
2009-05-01 05:59:52 +00:00
Senthil Kumaran 7c2867fcb1 Fix for the Issue918368 - urllib doesn't correct server returned urls 2009-04-21 03:24:19 +00:00
Senthil Kumaran 5e95e763e1 Fix for bugs: Issue4675 and Issue4962. 2009-03-30 21:51:50 +00:00
Benjamin Peterson 373498fc67 Merged revisions 70521 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r70521 | benjamin.peterson | 2009-03-22 12:45:11 -0500 (Sun, 22 Mar 2009) | 1 line

  close the file even if an exception occurs #5536
........
2009-03-22 17:49:21 +00:00
Benjamin Peterson b364bfe2f4 close the file even if an exception occurs #5536 2009-03-22 17:45:11 +00:00
Kristján Valur Jónsson 84040dbe81 Issue 4336: Let users of HTTPConnection.endheaders() submit a message body to the function if required. 2009-01-09 20:27:16 +00:00
Benjamin Peterson 2c7470d951 #3879 fix a regression in urllib.getproxies_environment
reviewers: Benjamin, Georg
2008-09-21 21:27:51 +00:00
Brett Cannon 8bb8fa5dd6 Handle urllib's renaming for Python 3.0:
* Deprecate urllib.urlopen() in favor of urllib2.urlopen() for 3.0.
* Update docs to mention split/rename of the module and deprecation of
  urlopen().

Changes to lib2to3 are in a separate commit. Work is for issue #2885.
2008-07-02 01:57:08 +00:00
Facundo Batista 4f1b1ed975 Fixed the semantic of timeout for socket.create_connection and
all the upper level libraries that use it, including urllib2.
Added and fixed some tests, and changed docs correspondingly.
Thanks to John J Lee for the patch and the pusing, :)
2008-05-29 16:39:26 +00:00
Ronald Oussoren 099646f29a MacOSX: ctypes annotation in implementation of getproxies_macosx_sysconf
getproxies_macosx_sysconf uses ctypes to call SystemConfiguration APIs. This
checkin adds ctypes annotation to specify the right argument types for the
API's that are used.

This is needed to be able to use urllib on a 64-bit system, without
annotations you'd get a hard crash.
2008-05-18 20:09:54 +00:00
Ronald Oussoren 9dd6b1db5b MacOSX: remove dependency on Carbon package for urllib
This patch removes the dependency on the Carbon package from urllib.
The mac-specific code for getting proxy configuration is now writting in
Python using ctypes and uses the SystemConfiguration framework instead of
InternetConfig. Also provides a mac-specific implementation of proxy_bypass.
2008-05-12 11:31:05 +00:00
Georg Brandl d5e6cf2b15 #1664522: in urllib, don't read non-existing directories in ftp mode,
returning a 0-byte file -- raise an IOError instead.
Original patch from Phil Knirsch.
2008-01-20 12:18:17 +00:00
Georg Brandl 2235011d49 #856047: respect the ``no_proxy`` env var when checking for proxies
in urllib and using the other ``_proxy`` env vars.
Original patch by Donovan Baarda.
2008-01-20 12:05:43 +00:00
Georg Brandl 9b0d46db11 #1178141: add addinfourl.code to get http status code from urllib. 2008-01-20 11:43:03 +00:00
Kurt B. Kaiser 0f7c25d20f Issue1177
r58207 and r58247 patch logic is reversed.  I noticed this when I
tried to use urllib to retrieve a file which required auth.

Fix that and add a test for 401 error to verify.
2008-01-02 04:11:28 +00:00
Georg Brandl 9b915673b7 #1177: accept 2xx responses for https too, not only http. 2007-09-24 18:08:24 +00:00
Sean Reifscheider a1afbf617d issue1177: Ported Facundo's from urllib2 to urllib, accepting 2xx responses. 2007-09-19 07:52:56 +00:00
Bill Janssen 426ea0a864 This contains a number of things:
1) Improve the documentation of the SSL module, with a fuller
   explanation of certificate usage, another reference, proper
   formatting of this and that.

2) Fix Windows bug in ssl.py, and general bug in sslsocket.close().
   Remove some unused code from ssl.py.  Allow accept() to be called on
   sslsocket sockets.

3) Use try-except-else in import of ssl in socket.py.  Deprecate use of
   socket.ssl().

4) Remove use of socket.ssl() in every library module, except for
   test_socket_ssl.py and test_ssl.py.
2007-08-29 22:35:05 +00:00
Facundo Batista 711a54ebde Added an optional timeout parameter to urllib.ftpwrapper, with tests
(for this and a basic one, because there weren't any). Changed also
NEWS, but didn't find documentation for this function, assumed it
wasn't public...
2007-05-24 17:50:54 +00:00
Brett Cannon d75f043c33 Remove the gopherlib module. It has been raising a DeprecationWarning since
Python 2.5.

Also remove gopher support from urllib/urllib2.  As both imported gopherlib the
usage of the support would have raised a DeprecationWarning.
2007-05-16 22:42:29 +00:00
Neal Norwitz ce55e21c70 Try to get test_urllib to pass on Windows by closing the file.
I'm guessing that's the problem.  h.getfile() must be called *after*
h.getreply() and the fp can be None.

I'm not entirely convinced this is the best fix (or even correct).
The buildbots will tell us if things improve or not.  I don't
know if this needs to be backported (assuming it actually works).
2007-03-20 08:14:57 +00:00
Georg Brandl 027ac24650 Bug #767111: fix long-standing bug in urllib which caused an
AttributeError instead of an IOError when the server's response didn't
contain a valid HTTP status line.
 (backport from rev. 54376)
2007-03-14 08:27:57 +00:00
Georg Brandl f66b6039c1 Bug #767111: fix long-standing bug in urllib which caused an
AttributeError instead of an IOError when the server's response didn't
contain a valid HTTP status line.
2007-03-14 08:27:52 +00:00
Collin Winter 98d1a4d727 Patch #1678662: ftp.python.org does not exist. So the testcode in urllib.py must use a more
stable FTP.
Backported from r54278.
2007-03-12 01:57:49 +00:00
Collin Winter 071d1ae136 Patch #1678662: ftp.python.org does not exist. So the testcode in urllib.py must use a more
stable FTP.
Will backport.
2007-03-12 01:55:54 +00:00
Georg Brandl 5a096e1b10 Use new email module names (#1637162, #1637159, #1637157). 2007-01-22 19:40:21 +00:00
Andrew M. Kuchling 0faf90acc4 [Patch #1617413 from Dug Song] Fix HTTP Basic authentication via HTTPS 2006-12-19 15:12:23 +00:00
Andrew M. Kuchling 522785732f [Patch #1617413 from Dug Song] Fix HTTP Basic authentication via HTTPS 2006-12-19 15:11:41 +00:00
Andrew M. Kuchling ff9e7abac8 [Patch #1574068 by Scott Dial] urllib and urllib2 were using
base64.encodestring() for encoding authentication data.
encodestring() can include newlines for very long input, which
produced broken HTTP headers.

2.4 backport candidate, probably.
2006-10-27 17:13:33 +00:00
Andrew M. Kuchling 872dba4253 [Patch #1574068 by Scott Dial] urllib and urllib2 were using
base64.encodestring() for encoding authentication data.
encodestring() can include newlines for very long input, which
produced broken HTTP headers.
2006-10-27 17:11:23 +00:00
Georg Brandl 0619a329e8 Bug #1459963: properly capitalize HTTP header names. 2006-07-26 07:40:17 +00:00
Georg Brandl 44a118af50 Patch #1470976: don't NLST files when retrieving over FTP. 2006-04-30 09:23:59 +00:00
Neal Norwitz c5d0dbd328 Fix a couple of strings that were no-ops. urllib.open_file was a docstring
in 2.4, so put it back.  The string in telnetlib looks like a comment.
2006-04-09 04:00:49 +00:00
Georg Brandl 80bb2bb7eb Revert r43399. 2006-03-28 19:19:56 +00:00
Georg Brandl f1349cd05d Bug #1459963: urllib2 now normalizes HTTP header names correctly
with title().
2006-03-28 12:40:24 +00:00
Georg Brandl 1c168d8eeb Bug #1457264: parse http://host?query correctly in urllib 2006-03-26 20:59:38 +00:00
Georg Brandl 1f63670a2a Patch #1426648: urllib proxy_bypass broken 2006-02-18 23:10:23 +00:00
Tim Peters 92037a15a9 Whitespace normalization. 2006-01-24 22:44:08 +00:00
Martin v. Löwis 3e86595280 Patch #1349118: urllib2 now supports user:pass@ style proxy
specifications, raises IOErrors when proxies for unsupported protocols
are defined, and uses the https proxy on https redirections.
2006-01-24 15:51:21 +00:00
Georg Brandl c0b24734e0 Bug #649974: make docstrings for url2pathname consistent 2005-12-26 22:53:56 +00:00
Georg Brandl 1f663574ee bug #1365984: urllib and data: URLs. Problem was that cStringIO objects cannot be assigned attributes on the fly. 2005-11-26 16:50:44 +00:00
Raymond Hettinger 4b0f20def3 Teach unquote() to handle unicode inputs 2005-10-15 16:41:53 +00:00
Raymond Hettinger cf6b6326e5 Corrected version of 1.170 2005-09-10 18:17:54 +00:00
Raymond Hettinger 2bdec7bfb0 Revert 1.170. Add tests. 2005-09-10 14:30:09 +00:00
Raymond Hettinger 803ce801ab Simplify and speed-up unquote(). 2005-09-10 06:49:04 +00:00
Raymond Hettinger 957b126649 Simplify and speed-up quote_plus(). 2005-09-10 02:27:41 +00:00
Raymond Hettinger 199d2f7997 SF #1285086: urllib.quote is too slow
Simplify and speed-up quote() function.
2005-09-09 22:27:13 +00:00
Georg Brandl 5a650a253c patch [ 810023 ] Fix for off-by-one bug in urllib.URLopener.retrieve 2005-08-26 08:51:34 +00:00
Georg Brandl b925602f16 Patch [ 1062060 ] fix for 1016880 urllib.urlretrieve silently truncates dwnld 2005-08-24 18:46:39 +00:00
Raymond Hettinger a617271dbd Use cStringIO where available. 2004-12-31 19:15:26 +00:00
Anthony Baxter 3dd9e46161 Added a usegmt flag to email.Utils.formatdate - this allows it to be
used to replace rfc822.formatdate for protocols like HTTP (where 'GMT' must
be the timezone string).
2004-10-11 13:53:08 +00:00
Tim Peters 182b5aca27 Whitespace normalization, via reindent.py. 2004-07-18 06:16:08 +00:00
Jack Jansen 11d9b06283 Fix for #779167: use InternetConfig proxy settings on MacOSX (in addition
to unix-style).
2004-07-16 11:45:00 +00:00
Brett Cannon aaeffaf01e Replace sequential split/join calls on strings with a single replace call.
Thanks Andrew Gaul.
2004-03-23 23:50:17 +00:00
Brett Cannon 69200fa85b Replace code in urllib for basejoin (undocumented) with urlparse.urljoin .
Test suites for urllib and urlparse run with each other's function to verify
correctness of replacement and both test suites pass.

Bumped urllib's __version__ attribute up a minor number.
2004-03-23 21:26:39 +00:00
Walter Dörwald 70a6b49821 Replace backticks with repr() or "%r"
From SF patch #852334.
2004-02-12 17:35:32 +00:00
Andrew M. Kuchling ff638ea6db [Bug #776542] open_https() generates a bad Authorization header because it calls .putheader() wrongly. Reported by Steffen Ries. 2003-08-29 18:12:23 +00:00
Guido van Rossum fa19f7c20d More fixes according to SF 549151:
- When redirecting, always use GET.  This is common practice and
  more-or-less sanctioned by the HTTP standard.

- Add a handler for 307 redirection, which becomes an error for POST,
  but a regular redirect for GET and HEAD.
2003-05-16 01:46:51 +00:00
Raymond Hettinger 024aaa1bfe SF Patch 549151: urllib2 POSTs on redirect
(contributed by John J Lee)
2003-04-24 15:32:12 +00:00
Brett Cannon 7d618c731c Fix docstring for URLOpener.retrieve() in regards to opening a local file 2003-04-24 02:43:20 +00:00
Raymond Hettinger 42182ebaf6 SF 698520: Iterator for urllib.URLOpener
Contributed by Brett Cannon.
2003-03-09 05:33:33 +00:00
Guido van Rossum 68468eba63 Get rid of many apply() calls. 2003-02-27 20:14:51 +00:00
Jeremy Hylton 3bd6fde4e3 Use fdopen() to create file from fd. 2002-10-11 14:36:24 +00:00
Jack Jansen 4ef1103b71 When testing for localhost/ first map to lower case. Spotted by Skip. 2002-09-12 20:14:04 +00:00
Jack Jansen 3ae2dc5e5e Treat file://localhost/ as local too (same as file:/ and file:///).
Fixes #607789, bugfix candidate.
2002-09-12 19:47:52 +00:00
Raymond Hettinger f2e45dd9dd Modify splituser() method to allow an @ in the userinfo field.
Jeremy reported that this is not allowed by RFC 2396; however,
other tools support unescaped @'s so we should also.

Apply SF patch 596581 closing bug 581529.
2002-08-18 20:08:56 +00:00
Guido van Rossum 3b0a3293c3 Massive changes from SF 589982 (tempfile.py rewrite, by Zack
Weinberg).  This changes all uses of deprecated tempfile functions to
the recommended ones.
2002-08-09 16:38:32 +00:00
Neal Norwitz 60e04cd317 Fix SF #565414, FancyURLopener() needs to support **kwargs
since the URLopener base class does and **kwargs are used in urlopen.
2002-06-11 13:38:51 +00:00
Walter Dörwald 65230a2de7 Remove uses of the string and types modules:
x in string.whitespace => x.isspace()
type(x) in types.StringTypes => isinstance(x, basestring)
isinstance(x, types.StringTypes) => isinstance(x, basestring)
type(x) is types.StringType => isinstance(x, str)
type(x) == types.StringType => isinstance(x, str)
string.split(x, ...) => x.split(...)
string.join(x, y) => y.join(x)
string.zfill(x, ...) => x.zfill(...)
string.count(x, ...) => x.count(...)
hasattr(types, "UnicodeType") => try: unicode except NameError:
type(x) != types.TupleTuple => not isinstance(x, tuple)
isinstance(x, types.TupleType) => isinstance(x, tuple)
type(x) is types.IntType => isinstance(x, int)

Do not mention the string module in the rlcompleter docstring.

This partially applies SF patch http://www.python.org/sf/562373
(with basestring instead of string). (It excludes the changes to
unittest.py and does not change the os.stat stuff.)
2002-06-03 15:58:32 +00:00
Raymond Hettinger 10ff706e27 Replaced boolean tests with is None. 2002-06-02 03:04:52 +00:00
Raymond Hettinger 54f0222547 SF 563203. Replaced 'has_key()' with 'in'. 2002-06-01 14:18:47 +00:00
Guido van Rossum 4b46c0a15f Don't require Unicode support. 2002-05-24 17:58:05 +00:00
Guido van Rossum a2da305211 Fix from SF bug #541980 (Jacques A. Vidrine).
When os.stat() for a file raises OSError, turn it into IOError per
documentation.

Bugfix candidate.
2002-04-15 00:25:01 +00:00
Fred Drake df6eca7eb7 Support manual proxy configuration for simple urlopen() operations.
This change is similar to the supplied patch, but does not save the opener
when a proxy configuration is specified.
This closes SF patch #523415.
2002-04-04 20:41:34 +00:00
Guido van Rossum 64e5aa9391 Fix for a bug in the fix for SF bug 503031. This time the OP verified
that it works.

Bugfix candidate (this and the previous checkin, obviously).
2002-04-02 14:38:16 +00:00
Guido van Rossum b955d6c41e Hopeful fix for SF bug 503031: urllib.py: open_http() host problem.
I really can't test this, but from reading the discussion in that bug
report, it's likely that this works.  It may also close a whole bunch
of other bug reports related to urllib and proxies on Windows, but who
knows.
2002-03-31 23:38:48 +00:00
Neal Norwitz aad1849e25 time and socket were already imported in the module, no need to re-import 2002-03-26 16:25:01 +00:00
Walter Dörwald 92b48b739f use stat attributes instead of tuple entries
and remove the unneccessary "import stat" statement.
2002-03-22 17:30:38 +00:00
Andrew M. Kuchling 56a42356b7 To make 'urllib.py -t' run again, change FTP URL to a file that actually
exists.
2002-03-18 22:18:46 +00:00
Neal Norwitz bc9bc187aa SF #515024 remove unused variable 2002-02-11 18:06:21 +00:00
Guido van Rossum b931bf3c55 SF patch #490515 (Joe A) urllib.open_https() protocol issue
open_http():
    In urllib.py library module, URLopener.open_https()
    returns a class instance of addinfourl() with its
    self.url property missing the protocol.

    Instead of "https://www.someurl.com", it becomes
    "://www.someurl.com".
2001-12-08 17:09:07 +00:00
Fred Drake c680ae8002 Added missing parameter in call to http_error_default();
reported by Neal Norwitz.
2001-10-13 18:37:07 +00:00
Jeremy Hylton 6d8c1aabff Add content-type header to ftp URLs (SF patch #454553)
Modify rfc822.formatdate() to always generate English names,
regardless of locale.  This is required by RFC 1123.

In open_local_file() of urllib and urllib2, use new formatdate() from
rfc822.
2001-08-27 20:16:53 +00:00
Guido van Rossum 88e0b5bee0 SF patch #454553 by Walter Dörwald: auto-guess content-type header for
ftp urls.
2001-08-23 13:38:15 +00:00
Martin v. Löwis 58682b7fe5 Only catch the errors that can actually occur, as reported in bug #411881. 2001-08-11 15:02:57 +00:00
Tim Peters ab9ba27dc0 Whitespace normalization. 2001-08-09 21:40:30 +00:00
Tim Peters 55c12d4d5b SF patch #403640: incomplete proxy handling in URLLIB
Look specific to Windows.  Don't know whether it works.
2001-08-09 18:04:14 +00:00
Guido van Rossum f0713d3f4d SF Patch #420725 by Walter Doerwald:
For local files urllib.py doesn't return the MIME
  headers that the documentation says it does:

  http://www.python.org/doc/current/lib/module-
  urllib.html#l2h-2187 states that "When the method is
  local-file, returned headers will include a Date
  representing the file's last-modified time, a Content-
  Length giving file size, and a Content-Type containing
  a guess at the file's type"

  But in Python 2.1 the only header that gets returned
  is the Content-Type:

  >>> import urllib
  >>> f = urllib.urlopen("gurk.txt")
  >>> f.info().headers
  ['Content-Type: text/plain\n']
2001-08-09 17:43:35 +00:00
Fred Drake ec3dfdee6a Only write out one blank line before the request data.
This closes SF patch #419459.
2001-07-04 05:18:29 +00:00
Guido van Rossum b8bf3bece2 Fix SF bug [ #416231 ] urllib.basejoin fails to apply some ../.
Reported by Juan M. Bello Rivas.
2001-04-15 20:47:33 +00:00
Moshe Zadka 5d87d47295 fixing 408085 - redirect from https becomes http
Even though relative redirects are illegal, they are common
urllib treated every relative redirect as though it was to http,
even if the original was https://
As long as we're compensating for server bugs, might as well do
it properly.
2001-04-09 14:54:21 +00:00
Skip Montanaro 44d5e0c418 updated __all__ to include several other names 2001-03-13 19:47:16 +00:00
Jack Jansen 282fed1363 Grr, splittag was also missing from __all__. 2001-03-05 13:45:38 +00:00
Jack Jansen 49985638fa Added url2pathname and pathname2url to __all__. 2001-03-05 13:41:14 +00:00
Guido van Rossum d74fb6b12a RISCOS changes by dschwertberger. 2001-03-02 06:43:49 +00:00
Skip Montanaro 40fc16059f final round of __all__ lists (I hope) - skipped urllib2 because Moshe may be
giving it a slight facelift
2001-03-01 04:27:19 +00:00
Tim Peters 85ba673b0a Whitespace normalization. 2001-02-28 08:26:44 +00:00
Moshe Zadka e99bd17ed6 Fixing bug #227562 by calling URLopener.http_error_default when
an invalid 401 request is being handled.
2001-02-27 06:27:04 +00:00
Skip Montanaro c3e11d6569 provide simple recovery/escape from apparent redirect recursion. If the
number of entries into http_error_302 exceeds the value set for the maxtries
attribute (which defaults to 10), the recursion is exited by calling
the http_error_500 method (or if that is not defined, http_error_default).
2001-02-15 16:56:36 +00:00
Tim Peters 658cba6706 Whitespace normalization. 2001-02-09 20:06:00 +00:00