Commit Graph

283 Commits

Author SHA1 Message Date
SH b15bde8058 bpo-35907, CVE-2019-9948: urllib rejects local_file:// scheme (GH-11842)
CVE-2019-9948: Avoid file reading as disallowing the unnecessary URL scheme in urllib.urlopen().
2019-05-21 23:12:23 +02:00
Victor Stinner d4324baca4 bpo-30500: urllib: Simplify splithost by calling into urlparse. (#1849) (#2294)
The current regex based splitting produces a wrong result. For example::

  http://abc#@def

Web browsers parse that URL as ``http://abc/#@def``, that is, the host
is ``abc``, the path is ``/``, and the fragment is ``#@def``.
(cherry picked from commit 90e01e50ef)
2017-06-20 16:20:36 +02:00
Xiang Zhang 98b1c82675 Issue #29142: Fix suffixes in no_proxy handling in urllib.
In urllib, suffixes in no_proxy environment variable with
leading dots could match related hostnames again (e.g. .b.c matches a.b.c).
Patch by Milan Oberkirch.
2017-01-09 11:43:24 +08:00
Raymond Hettinger 019a2e225b Issue #22450: Use "Accept: */*" in the default headers for urllib 2016-09-09 16:23:06 -07:00
Senthil Kumaran 75d7b615ba Prevent HTTPoxy attack (CVE-2016-1000110)
Ignore the HTTP_PROXY variable when REQUEST_METHOD environment is set, which
indicates that the script is in CGI mode.

Issue reported and patch contributed by Rémi Rampin.
2016-07-30 05:49:53 -07:00
Senthil Kumaran b5a67dec98 Closes issue26960.
Backport issue16270: Fix for urllib hanging with ftp urls.
2016-05-10 01:12:55 -07:00
Martin Panter 064ee4d28c Issue #26864: Fix case insensitivity and suffix comparison with no_proxy
Patch by Xiang Zhang.
2016-04-30 01:03:40 +00:00
Senthil Kumaran b31c87bfcf backport fix for Issue #26804.
urllib.request will prefer lower_case proxy environment variables over
UPPER_CASE or Mixed_Case ones.
2016-04-25 09:17:54 -07:00
Martin Panter ade4097671 Issue #12923: Reset FancyURLopener's redirect counter even on exception
Based on patches by Brian Brazil and Daniel Rocco.
2016-02-04 06:01:35 +00:00
Serhiy Storchaka 1aa2c0f073 Issue #23865: close() methods in multiple modules now are idempotent and more
robust at shutdown. If needs to release multiple resources, they are released
even if errors are occured.
2015-04-10 13:24:10 +03:00
Victor Stinner 5ab1c01bfd Issue #23881: urllib.ftpwrapper constructor now closes the socket if the FTP
connection failed.
2015-04-07 12:47:57 +02:00
Benjamin Peterson b206473ef8 give urllib.urlopen a context parameter (closes #22927) 2014-11-23 20:55:24 -06:00
Serhiy Storchaka 326b5ab05a Issue #20270: urllib and urlparse now support empty ports. 2014-01-18 18:30:09 +02:00
Senthil Kumaran 964c25f1d9 Fix #17967 - Fix related to regression on Windows.
os.path.join(*self.dirs) produces an invalid path on windows.
ftp paths are always forward-slash seperated like this. /pub/dir.
2013-06-02 11:59:09 -07:00
Senthil Kumaran 7351b66eb9 Fix thishost helper funtion in urllib. Returns the ipaddress of localhost when
hostname is resolvable by socket.gethostname for local machine. This all fixes
certain freebsd builtbot failures.
2013-06-01 11:11:30 -07:00
Senthil Kumaran 243cb807e9 Fix #17967: For ftp urls CWD to target instead of hopping to each directory
towards target. This fixes a bug where target is accessible, but parent
directories are restricted.
2013-06-01 08:24:31 -07:00
Serhiy Storchaka 923baea9f9 Issue #1285086: Get rid of the refcounting hack and speed up urllib.unquote(). 2013-03-14 21:31:09 +02:00
Senthil Kumaran 4c59211bd5 Fix the urllib closing issue which hangs on particular ftp urls/ftp servers. closes issue11199 2012-03-15 13:24:40 -07:00
Senthil Kumaran 58c6062068 Fix Issue6631 - Disallow relative files paths in urllib*.open() 2012-01-21 11:43:02 +08:00
Senthil Kumaran bcd833f30f - Issue #13642: Unquote before b64encoding user:password during Basic
Authentication. Patch contributed by Joonas Kuorilehto and Michele Orrù.
2012-01-11 00:09:24 +08:00
Charles-François Natali f3d35f0efe Issue #8035: urllib: Fix a bug where the client could remain stuck after a
redirection or an error.
2011-12-18 15:52:48 +01:00
Senthil Kumaran 87e585581d Port to 2.7 - issue 10817 - Fix urlretrieve function to raise
ContentTooShortError even when reporthook is None. Patch by Jyrki Pulliainen.
2011-11-01 02:44:45 +08:00
Senthil Kumaran ef651ac162 whitespace fix. 2011-08-07 10:03:58 +08:00
Senthil Kumaran 351950fbfc Remove the old dead test program within the module in 2.7. 2011-08-07 10:02:49 +08:00
Senthil Kumaran b5bd4c88b5 Fix closes issue12698 - make the no_proxy environment variable handling a bit lenient (accomodate spaces in between the items) 2011-08-06 12:24:33 +08:00
Nadeem Vawda a620facc1f Fix typo in dbf1e1a27427 that was causing some buildbots to fail. 2011-07-23 17:04:42 +02:00
Nadeem Vawda b42c53e442 Issue #10883: Fix socket leaks in urllib.request.
* ftpwrapper now uses reference counting to ensure that the underlying socket
  is closed when the ftpwrapper object is no longer in use
* ftplib.FTP.ntransfercmd() now closes the socket if an error occurs

Initial patch by Victor Stinner.
2011-07-23 15:51:16 +02:00
Guido van Rossum 07ef62c47c Merge issue 11662 from 2.6. 2011-03-29 12:53:55 -07:00
Guido van Rossum 079381d236 Merge issue 11662 from 2.5. 2011-03-29 12:51:16 -07:00
guido@google.com f1509306d2 Add tests for the urllib[2] vulnerability. Change to raise exceptions. 2011-03-28 13:47:01 -07:00
guido@google.com 2bc23b8448 Add FTP to the allowed url schemes. Add Misc/NEWS. 2011-03-24 10:44:17 -07:00
guido@google.com 60a4a90c8d Issue 22663: fix redirect vulnerability in urllib/urllib2. 2011-03-24 08:07:45 -07:00
Ronald Oussoren 1aa999c49e Issue #11500: Fixed a bug in the os x proxy bypass code for fully qualified
IP addresses in the proxy exception list.
2011-03-14 18:53:59 -04:00
Senthil Kumaran 0b7cac11bf Merged revisions 86676 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r86676 | senthil.kumaran | 2010-11-22 12:48:26 +0800 (Mon, 22 Nov 2010) | 4 lines

  Fix Issue4493 - urllib2 adds '/' to the path component of url, when it does not
  starts with one. This behavior is exhibited by browser and other clients.
........
2010-11-22 05:04:33 +00:00
Senthil Kumaran 9fce551e0e Merged revisions 86520 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r86520 | senthil.kumaran | 2010-11-18 23:36:41 +0800 (Thu, 18 Nov 2010) | 3 lines

  Fix Issue2244 - urllib unquotes user and password info multiple times - Patch by Theodore Turocy
........
2010-11-20 11:24:08 +00:00
Florent Xicluna 6980342c34 Rolled back revisions 81259,81265 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

(due to 2.6.6 release candidate freeze)
2010-08-12 22:39:08 +00:00
Florent Xicluna 1c24592b92 Merged revisions 81053,81259,81265 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81053 | florent.xicluna | 2010-05-10 21:59:22 +0200 (lun., 10 mai 2010) | 2 lines

  Add a link on maketrans().
........
  r81259 | florent.xicluna | 2010-05-17 12:39:07 +0200 (lun., 17 mai 2010) | 2 lines

  Slight style cleanup.
........
  r81265 | florent.xicluna | 2010-05-17 15:35:09 +0200 (lun., 17 mai 2010) | 2 lines

  Issue #1285086: Speed up urllib.quote and urllib.unquote for simple cases.
........
2010-08-09 22:05:50 +00:00
Senthil Kumaran 880685f698 Reverting the checkin made in revision 82940, as it was adding new parameters to quote function in a bugfix release.
Discussed in issue1712522
2010-07-22 01:47:30 +00:00
Senthil Kumaran c7743aaac3 Fix Issue9301 - urllib.quote(None) to raise TypeError 2010-07-19 17:35:50 +00:00
Senthil Kumaran 5dba6dfe6a Fixing Issue1712522 - urllib.quote to support Unicode. The default
encoding='utf-8' and errors='strict'.
2010-07-18 02:27:10 +00:00
Senthil Kumaran 73ae8dde6f Merged revisions 82897 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/release27-maint

................
  r82897 | senthil.kumaran | 2010-07-15 01:52:17 +0530 (Thu, 15 Jul 2010) | 9 lines

  Merged revisions 82895 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/branches/py3k

  ........
    r82895 | senthil.kumaran | 2010-07-15 01:40:52 +0530 (Thu, 15 Jul 2010) | 3 lines

    Fix a mistake, https proxy shoud be https://
  ........
................
2010-07-14 20:25:15 +00:00
Senthil Kumaran 0fdd385e2c Merged revisions 82895 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r82895 | senthil.kumaran | 2010-07-15 01:40:52 +0530 (Thu, 15 Jul 2010) | 3 lines

  Fix a mistake, https proxy shoud be https://
........
2010-07-14 20:22:17 +00:00
Senthil Kumaran 7a4e837943 Merged revisions 82892 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/release27-maint

................
  r82892 | senthil.kumaran | 2010-07-15 00:55:26 +0530 (Thu, 15 Jul 2010) | 9 lines

  Merged revisions 82890 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/branches/py3k

  ........
    r82890 | senthil.kumaran | 2010-07-15 00:45:23 +0530 (Thu, 15 Jul 2010) | 3 lines

    Fix: Issue6853 - Get HTTPS system proxy in Windows.
  ........
................
2010-07-14 19:28:35 +00:00
Senthil Kumaran 836a2bba2f Merged revisions 82890 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r82890 | senthil.kumaran | 2010-07-15 00:45:23 +0530 (Thu, 15 Jul 2010) | 3 lines

  Fix: Issue6853 - Get HTTPS system proxy in Windows.
........
2010-07-14 19:25:26 +00:00
Ronald Oussoren 68f74ba556 Merged revisions 82284 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r82284 | ronald.oussoren | 2010-06-27 15:59:39 +0200 (Sun, 27 Jun 2010) | 4 lines

  Fix for Issue8883: without this patch test_urllib will fail
  when there is a bare IP address in the "Bypass proxy settings
  for these Hosts & Domains" list on MacOSX.
........
2010-06-27 14:00:56 +00:00
Ronald Oussoren b96fbb8d00 Fix for Issue8883: without this patch test_urllib will fail
when there is a bare IP address in the "Bypass proxy settings
for these Hosts & Domains" list on MacOSX.
2010-06-27 13:59:39 +00:00
R. David Murray bfbdefe539 Issue 8143: sync unquote in urlparse with urllib; add comment about doing so.
unquote is duplicated in the two files to avoid a circular reference.
(This is fixed in Python3.)  Updates keep getting made to the public unquote
without fixing the urlparse one, however, so this fix syncs the two
and adds a comment to both to make sure changes are applied to both.
2010-05-25 15:20:46 +00:00
Florent Xicluna af87f9f09f Issue #1285086: Speed up urllib.quote and urllib.unquote for simple cases. 2010-05-17 13:35:09 +00:00
Florent Xicluna e127e24359 Slight style cleanup. 2010-05-17 10:39:07 +00:00
Ronald Oussoren 9545a23c7f In a number of places code still revers
to "sys.platform == 'mac'" and that is
dead code because it refers to a platform
that is no longer supported (and hasn't been
supported for several releases).

Fixes issue #7908 for the trunk.
2010-05-05 19:09:31 +00:00