Guido van Rossum
c94f16f156
Oops! Of course, Tim is right -- when the item is not a hex number,
...
the '%' should be put back in.
1998-06-29 00:42:54 +00:00
Guido van Rossum
52e86ad05b
Speed-up unquote(), inspired by post from Daniel Walton.
1998-06-28 23:49:35 +00:00
Guido van Rossum
2349015a87
Rewrite the (test) main program so that when used as a script, it can
...
retrieve one or more URLs to stdout. Use -t to run the self-test.
1998-06-25 02:39:00 +00:00
Guido van Rossum
ae9ee7329d
Use the getpass module instead of having platform-specific echo on/off
...
code here.
1998-06-12 14:21:13 +00:00
Guido van Rossum
e0c0da98d8
Patches to make the proxy code work again. (Why does that always break
...
as soon as I change things even just a little bit? :-) Even works
when accessing a password-protected page through the proxy. Prompted
by complaints from, and correct operation verified by, Nigel O'Brian.
1998-05-05 13:58:13 +00:00
Guido van Rossum
0eae8fba81
Feeble attempt at making urlopen more robust -- don't call splituser()
...
when splithost() returned no useable host, to avoid calling
splituser() on None.
1998-04-27 15:19:17 +00:00
Guido van Rossum
c74521acc4
Oops -- remove some debug print statements!
1998-04-11 01:18:35 +00:00
Guido van Rossum
0454b51282
Oops, pulled over by the tab police :-)
1998-04-03 15:57:58 +00:00
Guido van Rossum
b5916ab065
Change by Sjoerd (with minor reformatting):
...
guess the mime type of a local file.
Change suggested by Sjoerd (with different implementation):
when retrieve() creates a temporary file, preserve the suffix.
Corrollary of the first change:
also return the mime type of a local file in retrieve().
1998-04-03 15:56:16 +00:00
Guido van Rossum
a08fabad72
A few lines were indented using spaces instead of tabs -- fix them.
1998-03-30 17:17:24 +00:00
Guido van Rossum
7e7ca0ba17
A few lines were indented using spaces instead of tabs -- fix them.
1998-03-26 21:01:39 +00:00
Guido van Rossum
6d4d1c2a25
Added support for "data" URL, by Sjoerd Mullender.
1998-03-12 14:32:55 +00:00
Guido van Rossum
8a666e7c56
Fix a horrible race condition -- various routines were storing the
...
most recently opened URL in self.openedurl of the URLopener instance.
This doesn't really work if multiple threads share the same opener
instance!
Fix: openedurl was actually simply the type prefix (e.g. "http:")
followed by the rest of the URL; since the rest of the URL is
available and the type is effectively determined by where you are in
the code, I can reconstruct the full URL easily, e.g. "http:" + url.
1998-02-13 01:39:16 +00:00
Guido van Rossum
03710d2a40
Two suggested features by Sjoerd:
...
- use the tempcache in the open() method, too.
- use the "unwrap"ped url as key for the tempcache.
1998-02-05 16:22:27 +00:00
Guido van Rossum
c5d8fed261
(1) Use matchobj.groups(), not matchbj.group() to get all groups.
...
(2) Provisional hack to avoid dying when trying to turn echo on or off
on Macs, where os.system() doesn't exist.
1998-02-05 16:21:28 +00:00
Guido van Rossum
e7579624ef
Fix bad new bug in ftp code -- the test for existing file using NLST
...
would set the transfer to text mode instead of the specified mode.
1998-01-19 22:26:54 +00:00
Guido van Rossum
d499004860
Solve two annoying problems with ftp URLs for Jack: when repeated
...
retrieving files from the same host and directory, you had to close
the previous instance before opening a new one; and retrieving a
non-existent file would return an empty file. (The latter fix relies
on maybe an undocumented property of NLST -- NLST of a file returns
just that file, while NLST of a non-existent file returns nothing. A
side effect, unfortunately, seems to be that now ftp-retrieving an
*empty* directory may fail. Ah well.)
1997-12-28 04:21:20 +00:00
Guido van Rossum
c0f29c2d31
When a port is specified in an ftp:// URL, must convert it to a number!
1997-12-02 20:26:21 +00:00
Guido van Rossum
83600050d0
Assert that the proxies object passed in to the URLopener constructor
...
is indeed a dictionary (or a mapping).
1997-11-18 15:50:39 +00:00
Guido van Rossum
036309b13e
This should hopefully finally clean up the remaining __del__ related
...
problems with this module, even if an instance of a derived class is
kept alive longer than the urllib module itself...
1997-10-27 18:56:19 +00:00
Fred Drake
654451dc54
splitpasswd(): The parameter is named "user", not "host".
1997-10-14 13:30:57 +00:00
Guido van Rossum
332e14437c
Use sys.exc_info() where needed.
...
Use "re" module, making it threadsafe.
1997-09-29 23:23:46 +00:00
Guido van Rossum
10499324cf
Initialize self.__tempfiles to [] in the constructor (else it remains
...
a shared class variable -- but each instance will attempt to clean it
up entirely ob cleanup).
1997-09-08 02:16:33 +00:00
Guido van Rossum
2b3fd76cc7
One patch from Sjoerd and one from Jack.
...
Sjoerd: add separate administration of temporary files created y
URLopener.retrieve() so cleanup can properly remove them. The old
code removed everything in tempcache which was a bad idea if the user
had passed a non-temp file into it. (I added a line to delete the
tempcache in cleanup() -- it still seems to make sense.)
Jack: in basejoin(), interpret relative paths starting in "../". This
is necessary if the server uses symbolic links.
1997-09-03 22:36:15 +00:00
Guido van Rossum
ab0abdcef8
Explicitly close the socket and temp file in URLopener.retrieve(), so
...
that multiple retrievals using the same connection will work.
This leaves open the more general problem that after
f = urlopen("ftp://...")
f must be closed before another retrieval from the same host should be
attempted.
1997-08-26 19:06:40 +00:00
Guido van Rossum
b6784dc53f
Interpret three slashes in file: URL as local file (for Netscape on
...
Windows/Mac).
1997-08-20 23:34:01 +00:00
Guido van Rossum
f668d17e01
Clear the ftp cache when it contains more than 10 entries.
1997-06-06 21:11:11 +00:00
Guido van Rossum
2966b32205
Catch *all* errors that ftplib can raise (ftplib.all_errors) rather
...
than just the four.
Also folded some long lines.
1997-06-06 17:44:07 +00:00
Guido van Rossum
c24751b57a
Add Host: header to URL request.
1997-06-03 14:34:19 +00:00
Guido van Rossum
1aec3f087e
lowercase proxies env variables, for Windows.
1997-05-28 15:37:19 +00:00
Guido van Rossum
ab0d1afdf3
spliturl() should not throw away everything past first newline
1997-04-16 15:17:06 +00:00
Guido van Rossum
54a1d0bc69
(Jack+Guido:) interpret ../ in basejoin() according to RFC1808.
1997-04-11 19:09:51 +00:00
Guido van Rossum
c511aee0e3
Open files in binary mode.
1997-04-11 19:01:48 +00:00
Guido van Rossum
fd79566d0f
Fix two small bugs with proxies.
1997-04-02 05:46:35 +00:00
Guido van Rossum
838cb28290
Put a new, more useful, set of references in the leading comment.
1997-02-10 17:51:56 +00:00
Guido van Rossum
d23d9409f3
Move the 'import os' in URLopener.cleanup() to inside the block
...
guarded by 'if self.tempcache', to reduce the likelihood of this
causing an exception when invoked during __del__...
1997-01-30 15:54:58 +00:00
Guido van Rossum
0564e12367
Added quote_plus() and unquote_plus(), to do space/plus substitutions
...
for form field values.
1996-12-13 14:47:36 +00:00
Guido van Rossum
bd01374da8
Add optional data argument to urlopen() and open_http(), to trigger POST.
1996-12-10 16:00:28 +00:00
Guido van Rossum
29e77816f5
Added safeguard against failure in __del__.
1996-11-27 19:39:58 +00:00
Guido van Rossum
3c8484e866
When re-raising an exception raised by a module used internally as
...
IOError, keep the traceback.
1996-11-20 22:02:24 +00:00
Guido van Rossum
c5d7e80739
Fix the way the Authorization header is sent (how could this have worked?).
1996-11-11 19:01:17 +00:00
Guido van Rossum
5b1b33c7f4
Fix another case where...
1996-10-22 13:28:37 +00:00
Guido van Rossum
b030bc026e
Fix some cases where self.openedurl wasn't set.
1996-10-10 16:01:16 +00:00
Guido van Rossum
e6ad8913e2
One fix by sjoerd and one suggested by him. Bumped __version__ to 1.5.
...
Fix 1: add a method geturl() to the file-like object returned by urlopen().
Fix 2: treat http error 301 the same as error 302.
1996-09-10 17:02:56 +00:00
Guido van Rossum
78c9637075
Bump exposed __version__ to 1.4.
...
Correctly handle a URL containing user:password@host when using a proxy.
1996-08-26 18:09:59 +00:00
Guido van Rossum
f8abb38737
Slightly faster (un)quoting.
1996-08-26 15:56:12 +00:00
Guido van Rossum
2281d35578
add nturl2path
1996-06-26 19:47:37 +00:00
Guido van Rossum
84a00a80a2
Change defn of splitnport() to only accept valid digit strings.
1996-06-17 17:11:40 +00:00
Guido van Rossum
53725a2858
Added splitnport(), which is like splitport() but returns a numeric port,
...
is forgiving about semi-numeric port numbers, and allows you to specify
a default port (default is -1, None returned for nonnumeric port).
1996-06-13 19:12:35 +00:00
Guido van Rossum
a7e4b28422
Support optional filename argument for retrieve() and urlretrieve(),
...
to specify where it should go (if specified, even local files will be
copied into the given file).
1996-06-11 00:16:27 +00:00
Guido van Rossum
71ac945321
move mac url2path conversion to separate module
1996-03-21 16:31:41 +00:00
Guido van Rossum
442e7202f8
Added proxy handling; upped version.
...
(Proxy handling uses <proto>_proxy environment variables by default.)
1996-03-20 15:33:11 +00:00
Jack Jansen
0d12ead05c
Try to normalize urls referring to local files (code copied from posixpath)
1996-02-14 16:05:20 +00:00
Guido van Rossum
8c8a02a258
speed up unquote() by using atoi() instead of eval()
1996-01-26 17:41:44 +00:00
Jack Jansen
e8ea21b0fe
Added pathname2url and url2pathname methods (only correct for unix and
...
mac, so far)
1995-12-21 15:43:53 +00:00
Jack Jansen
dc3e3f69db
Fixed local file access for macintosh
1995-12-15 13:22:13 +00:00
Sjoerd Mullender
e0371b8415
Fixed basejoin. There were two main problems:
...
- basejoin('some/url', '#name') would strip the last component of
some/url and resturn some/#name.
- basejoin('file.html', 'relative/path') would return something like
file:/relative/path, making a relative path into an absolute one.
These bugs are fixed by some drastic changes. No scheme is added when
none is present (i.e. it works as replacement for posix.joinpath).
If a scheme is present in the second argument, it is returned
unprocessed. No hostname are added in this case. If no scheme is
present, the scheme of the first argument, if present, is used.
The algorithm is commented profusely.
Also fixed a typo in a comment.
1995-11-10 10:36:07 +00:00
Guido van Rossum
ca44540bc8
support overriding how to open unknown url types
1995-08-29 19:19:12 +00:00
Sjoerd Mullender
9c4585acfa
Removed addbase.__del__ because it can't work.
...
If code keeps a reference to self.fp or any of its methods, you don't
want to close self.fp just because no explicit reference is kept to self.
1995-08-15 11:33:39 +00:00
Guido van Rossum
30642ab29f
changed version :-)
1995-08-10 19:44:54 +00:00
Guido van Rossum
bbb0a05972
use mimetools; add error handling and authentication
1995-08-04 04:29:05 +00:00
Guido van Rossum
6cb15a0572
add User-agent hdr; read and close the file upon http error
1995-06-22 19:00:13 +00:00
Guido van Rossum
a1124700f8
Add hacks for switching protocol and path but leaving host unchanged
1994-12-30 17:18:59 +00:00
Guido van Rossum
fa59e83813
Fix bug if tmpcache is None
1994-09-21 11:36:19 +00:00
Guido van Rossum
3bb5448767
New way of generating .pyc files, thanks to Sjoerd.
...
urllib.py: '+' is not always safe (even though the RFC says so :-( )
whrandom.py: throw away top bits of time to avoid overflow on Mac
(where times can be negative)
1994-08-29 10:52:58 +00:00
Guido van Rossum
7aeb4b9ce8
* Lib/linecache.py: don't crash on empty filename
...
* Lib/macpath.py: don't return trailing colon for dirname()
(XXX won't do for volume names -- but otherwise glob(':*:*.py')
loops forever)
* Lib/traceback.py: print SyntaxError correctly
* Lib/stat.py: moved to posixstat.py; added macstat.py which has
the constants for the Mac; and created new stat.py which includes
the right one
* Lib/urllib.py: fix caching bug (by disabling the cache)
1994-08-23 13:32:20 +00:00
Guido van Rossum
3f9a6ec9e6
* Lib/rfc822.py: fix two bugs: error in readheaders interpreting
...
regex.match() result, and wrong logic in getfirstmatchingheader()
when the same header occurs twice consecutively
1994-08-12 13:16:50 +00:00
Guido van Rossum
7c395db298
* Lib/urllib.py: implemented new quoting rules; added splituser,
...
splitpasswd, splitattr, splitvalue; new ftp syntax (user:passwd,
cwd to each subdir, type={a,i,d})
1994-07-04 22:14:49 +00:00
Guido van Rossum
590b289672
Added tests for missing host to open_http and open_gopher
1994-04-18 09:39:56 +00:00
Guido van Rossum
7c6ebb572e
Renamed urlopen.py to urllib.py.
1994-03-22 12:05:32 +00:00