tarfile unnecessarily checked the existence of numerical user and group ids on
extraction. If one of them did not exist the respective id of the current user
(i.e. root) was used for the file and ownership information was lost. (Patch
by Sebastien Luttringer)
'latin-1' and 'utf-8'.
These are optimized in the Python Unicode implementation
to result in more direct processing, bypassing the codec
registry.
Also see issue11303.
svn+ssh://pythondev@svn.python.org/python/branches/py3k
........
r88528 | lars.gustaebel | 2011-02-23 12:42:22 +0100 (Wed, 23 Feb 2011) | 16 lines
Issue #11224: Improved sparse file read support (r85916) introduced a
regression in _FileInFile which is used in file-like objects returned
by TarFile.extractfile(). The inefficient design of the
_FileInFile.read() method causes various dramatic side-effects and
errors:
- The data segment of a file member is read completely into memory
every(!) time a small block is accessed. This is not only slow
but may cause unexpected MemoryErrors with very large files.
- Reading members from compressed tar archives is even slower
because of the excessive backwards seeking which is done when the
same data segment is read over and over again.
- As a backwards seek on a TarFile opened in stream mode is not
possible, using extractfile() fails with a StreamError.
........
regression in _FileInFile which is used in file-like objects returned
by TarFile.extractfile(). The inefficient design of the
_FileInFile.read() method causes various dramatic side-effects and
errors:
- The data segment of a file member is read completely into memory
every(!) time a small block is accessed. This is not only slow
but may cause unexpected MemoryErrors with very large files.
- Reading members from compressed tar archives is even slower
because of the excessive backwards seeking which is done when the
same data segment is read over and over again.
- As a backwards seek on a TarFile opened in stream mode is not
possible, using extractfile() fails with a StreamError.
keyword-only argument. The preceding positional argument was deprecated,
so it made no sense to add filter as a positional argument.
(Patch reviewed by Brian Curtin and Anthony Long.)
extensions. Thus, in addition to GNUTYPE_SPARSE headers, sparse
information in pax headers created by GNU tar can now be decoded.
All three formats 0.0, 0.1 and 1.0 are supported.
On filesystems that support this, holes in files are now restored
whenever a sparse member is extracted.
uname and gname field.
If tarfile creates a new archive and adds a file with a
uid/gid that doesn't have a corresponding name on the
system (e.g. because the user/group account was deleted) it
uses the empty string in the uname/gname field now instead
of "root". Using "root" as the default was a bad idea
because on extraction the uname/gname fields are supposed
to override the uid/gid fields. So, all archive members
with nameless uids/gids belonged to the root user after
extraction.
using WindowsError in a try/except. Only add WindowsError to the list of
exceptions to catch when we are actually running on Windows.
Additionally, add a call that was left out in test_posixpath.
Thanks Amaury, Antoine, and Jason.
Added Windows support for os.symlink when run on Windows 6.0 or greater,
aka Vista. Previous Windows versions will raise NotImplementedError
when trying to symlink.
Includes numerous test updates and additions to test_os, including
a symlink_support module because of the fact that privilege escalation
is required in order to run the tests to ensure that the user is able
to create symlinks. By default, accounts do not have the required
privilege, so the escalation code will have to be exposed later (or
documented on how to do so). I'll be following up with that work next.
Note that the tests use ctypes, which was agreed on during the PyCon
language summit.
svn+ssh://pythondev@svn.python.org/python/trunk
........
r81667 | lars.gustaebel | 2010-06-03 14:34:14 +0200 (Thu, 03 Jun 2010) | 8 lines
Issue #8741: Fixed the TarFile.makelink() method that is responsible
for extracting symbolic and hard link entries as regular files as a
work-around on platforms that do not support filesystem links.
This stopped working reliably after a change in r74571. I also added
a few tests for this functionality.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r81663 | lars.gustaebel | 2010-06-03 11:56:22 +0200 (Thu, 03 Jun 2010) | 4 lines
Issue #8833: tarfile created hard link entries with a size
field != 0 by mistake. The associated testcase did not
expose this bug because it was broken too.
........
tarfile is now able to read and write pax headers with a
"hdrcharset=BINARY" record. This record was introduced in
POSIX.1-2008 as a method to store unencoded binary strings that
cannot be translated to UTF-8. In practice, this is just a workaround
that allows a tar implementation to store filenames that do not
comply with the current filesystem encoding and thus cannot be
decoded correctly.
Additionally, tarfile works around a bug in current versions of GNU
tar: undecodable filenames are stored as-is in a pax header without a
"hdrcharset" record being added. Technically, these headers are
invalid, but tarfile manages to read them correctly anyway.
svn+ssh://pythondev@svn.python.org/python/trunk
........
r78623 | lars.gustaebel | 2010-03-03 12:55:48 +0100 (Wed, 03 Mar 2010) | 3 lines
Issue #7232: Add support for the context manager protocol
to the TarFile class.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r76780 | lars.gustaebel | 2009-12-13 12:32:27 +0100 (Sun, 13 Dec 2009) | 21 lines
Issue #7357: No longer suppress fatal extraction errors by
default.
TarFile's errorlevel argument controls how errors are
handled that occur during extraction. There are three
possible levels 0, 1 and 2. If errorlevel is set to 1 or 2
fatal errors (e.g. a full filesystem) are raised as
exceptions. If it is set to 0, which is the default value,
extraction errors are suppressed, and error messages are
written to the debug log instead. But, if the debug log is
not activated, which is the default as well, all these
errors go unnoticed.
The original intention was to imitate GNU tar which tries
to extract as many members as possible instead of stopping
on the first error. It turns out that this is no good
default behaviour for a tar library. This patch simply
changes the default value for the errorlevel argument from
0 to 1, so that fatal extraction errors are raised as
EnvironmentError exceptions.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r76443 | lars.gustaebel | 2009-11-22 19:30:53 +0100 (Sun, 22 Nov 2009) | 24 lines
Issue #6123: Fix opening empty archives and files.
(Note that an empty archive is not the same as an empty file. An
empty archive contains no members and is correctly terminated with an
EOF block full of zeros. An empty file contains no data at all.)
The problem was that although tarfile was able to create empty
archives, it failed to open them raising a ReadError. On the other
hand, tarfile opened empty files without error in most read modes and
presented them as empty archives. (However, some modes still raised
errors: "r|gz" raised ReadError, but "r:gz" worked, "r:bz2" even
raised EOFError.)
In order to get a more fine-grained control over the various internal
error conditions I now split up the HeaderError exception into a
number of meaningful sub-exceptions. This makes it easier in the
TarFile.next() method to react to the different conditions in the
correct way.
The visible change in its behaviour now is that tarfile will open
empty archives correctly and raise ReadError consistently for empty
files.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r76381 | lars.gustaebel | 2009-11-18 21:24:54 +0100 (Wed, 18 Nov 2009) | 3 lines
Issue #7341: Close the internal file object in the TarFile
constructor in case of an error.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r74750 | lars.gustaebel | 2009-09-12 12:28:15 +0200 (Sat, 12 Sep 2009) | 9 lines
Issue #6856: Add a filter keyword argument to TarFile.add().
The filter argument must be a function that takes a TarInfo
object argument, changes it and returns it again. If the
function returns None the TarInfo object will be excluded
from the archive.
The exclude argument is deprecated from now on, because it
does something similar but is not as flexible.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r74571 | lars.gustaebel | 2009-08-28 21:23:44 +0200 (Fri, 28 Aug 2009) | 7 lines
Issue #6054: Do not normalize stored pathnames.
No longer use tarfile.normpath() on pathnames. Store pathnames
unchanged, i.e. do not remove "./", "../" and "//" occurrences.
However, still convert absolute to relative paths.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r70523 | lars.gustaebel | 2009-03-22 21:09:33 +0100 (Sun, 22 Mar 2009) | 5 lines
Issue #5068: Fixed the tarfile._BZ2Proxy.read() method that would loop
forever on incomplete input. That caused tarfile.open() to hang when used
with mode 'r' or 'r:bz2' and a fileobj argument that contained no data or
partial bzip2 compressed data.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r60568 | christian.heimes | 2008-02-04 19:48:38 +0100 (Mon, 04 Feb 2008) | 1 line
Increase debugging to investige failing tests on some build bots
........
r60570 | christian.heimes | 2008-02-04 20:30:05 +0100 (Mon, 04 Feb 2008) | 1 line
Small adjustments for test compact freelist test. It's no passing on Windows as well.
........
r60573 | amaury.forgeotdarc | 2008-02-04 21:53:14 +0100 (Mon, 04 Feb 2008) | 2 lines
Correct quotes in NEWS file
........
r60575 | amaury.forgeotdarc | 2008-02-04 22:45:05 +0100 (Mon, 04 Feb 2008) | 13 lines
#1750076: Debugger did not step on every iteration of a while statement.
The mapping between bytecode offsets and source lines (lnotab) did not contain
an entry for the beginning of the loop.
Now it does, and the lnotab can be a bit larger:
in particular, several statements on the same line generate several entries.
However, this does not bother the settrace function, which will trigger only
one 'line' event.
The lnotab seems to be exactly the same as with python2.4.
........
r60584 | amaury.forgeotdarc | 2008-02-05 01:26:21 +0100 (Tue, 05 Feb 2008) | 3 lines
Change r60575 broke test_compile:
there is no need to emit co_lnotab item when both offsets are zeros.
........
r60587 | skip.montanaro | 2008-02-05 03:32:16 +0100 (Tue, 05 Feb 2008) | 1 line
sync with most recent version from python-mode sf project
........
r60588 | lars.gustaebel | 2008-02-05 12:51:40 +0100 (Tue, 05 Feb 2008) | 5 lines
Issue #2004: Use mode 0700 for temporary directories and default
permissions for missing directories.
(will backport to 2.5)
........
r60590 | georg.brandl | 2008-02-05 13:01:24 +0100 (Tue, 05 Feb 2008) | 2 lines
Convert external links to internal links. Fixes#2010.
........
r60592 | marc-andre.lemburg | 2008-02-05 15:50:40 +0100 (Tue, 05 Feb 2008) | 3 lines
Keep distutils Python 2.1 compatible (or even Python 2.4 in this case).
........
r60593 | andrew.kuchling | 2008-02-05 17:06:57 +0100 (Tue, 05 Feb 2008) | 5 lines
Update PEP URL.
(This code is duplicated between pydoc and DocXMLRPCServer; maybe it
should be refactored as a GHOP project.)
2.5.2 backport candidate.
........
r60596 | guido.van.rossum | 2008-02-05 18:32:15 +0100 (Tue, 05 Feb 2008) | 2 lines
In the experimental 'Scanner' feature, the group count was set wrong.
........
r60602 | facundo.batista | 2008-02-05 20:03:32 +0100 (Tue, 05 Feb 2008) | 3 lines
Issue 1951. Converts wave test cases to unittest.
........
r60603 | georg.brandl | 2008-02-05 20:07:10 +0100 (Tue, 05 Feb 2008) | 2 lines
Actually run the test.
........
r60604 | skip.montanaro | 2008-02-05 20:24:30 +0100 (Tue, 05 Feb 2008) | 2 lines
correct object name
........
r60605 | georg.brandl | 2008-02-05 20:58:17 +0100 (Tue, 05 Feb 2008) | 7 lines
* Use the same code to profile for test_profile and test_cprofile.
* Convert both to unittest.
* Use the same unit testing code.
* Include the expected output in both test files.
* Make it possible to regenerate the expected output by running
the file as a script with an '-r' argument.
........
r60613 | raymond.hettinger | 2008-02-06 02:49:00 +0100 (Wed, 06 Feb 2008) | 1 line
Sync-up with Py3k work.
........
r60614 | christian.heimes | 2008-02-06 13:44:34 +0100 (Wed, 06 Feb 2008) | 1 line
Limit free list of method and builtin function objects to 256 entries each.
........
r60616 | christian.heimes | 2008-02-06 14:33:44 +0100 (Wed, 06 Feb 2008) | 7 lines
Unified naming convention for free lists and their limits. All free lists
in Object/ are named ``free_list``, the counter ``numfree`` and the upper
limit is a macro ``PyName_MAXFREELIST`` inside an #ifndef block.
The chances should make it easier to adjust Python for platforms with
less memory, e.g. mobile phones.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r59704 | christian.heimes | 2008-01-04 04:15:05 +0100 (Fri, 04 Jan 2008) | 1 line
Moved include "Python.h" in front of other imports to silence a warning.
........
r59706 | raymond.hettinger | 2008-01-04 04:22:53 +0100 (Fri, 04 Jan 2008) | 10 lines
Minor fix-ups to named tuples:
* Make the _replace() method respect subclassing.
* Using property() to make _fields read-only wasn't a good idea.
It caused len(Point._fields) to fail.
* Add note to _cast() about length checking and alternative with the star-operator.
........
r59707 | jeffrey.yasskin | 2008-01-04 09:01:23 +0100 (Fri, 04 Jan 2008) | 3 lines
Make math.{floor,ceil}({int,long}) return float again for backwards
compatibility after r59671 made them return integral types.
........
r59709 | christian.heimes | 2008-01-04 14:21:07 +0100 (Fri, 04 Jan 2008) | 1 line
Bug #1713: posixpath.ismount() claims symlink to a mountpoint is a mountpoint.
........
r59712 | lars.gustaebel | 2008-01-04 15:00:33 +0100 (Fri, 04 Jan 2008) | 5 lines
Issue #1735: TarFile.extractall() now correctly sets
directory permissions and times.
(will backport to 2.5)
........
r59714 | andrew.kuchling | 2008-01-04 15:47:17 +0100 (Fri, 04 Jan 2008) | 1 line
Update links to bug/patch tracker
........
r59716 | christian.heimes | 2008-01-04 16:23:30 +0100 (Fri, 04 Jan 2008) | 1 line
Added interface to Windows' WSAIoctl and a simple example for a network sniffer.
........
r59717 | christian.heimes | 2008-01-04 16:29:00 +0100 (Fri, 04 Jan 2008) | 1 line
And here is the rest of Hirokazu Yamamoto's patch for VS6.0 support. Thanks Hiro!
........
r59719 | christian.heimes | 2008-01-04 16:34:06 +0100 (Fri, 04 Jan 2008) | 1 line
Reverted last transaction. It's the wrong branch.
........
r59721 | christian.heimes | 2008-01-04 16:48:06 +0100 (Fri, 04 Jan 2008) | 1 line
socket.ioctl is only available on Windows
........
r59722 | andrew.kuchling | 2008-01-04 19:24:41 +0100 (Fri, 04 Jan 2008) | 1 line
Fix markup
........
r59723 | andrew.kuchling | 2008-01-04 19:25:05 +0100 (Fri, 04 Jan 2008) | 1 line
Fix markup
........
r59725 | guido.van.rossum | 2008-01-05 01:59:59 +0100 (Sat, 05 Jan 2008) | 3 lines
Patch #1725 by Mark Dickinson, fixes incorrect conversion of -1e1000
and adds errors for -0x.
........
r59726 | guido.van.rossum | 2008-01-05 02:21:57 +0100 (Sat, 05 Jan 2008) | 2 lines
Patch #1698 by Senthil: allow '@' in username when parsed by urlparse.py.
........
r59727 | raymond.hettinger | 2008-01-05 02:35:43 +0100 (Sat, 05 Jan 2008) | 1 line
Improve namedtuple's _cast() method with a docstring, new name, and error-checking.
........
r59728 | raymond.hettinger | 2008-01-05 03:17:24 +0100 (Sat, 05 Jan 2008) | 1 line
Add error-checking to namedtuple's _replace() method.
........
r59730 | fred.drake | 2008-01-05 05:38:38 +0100 (Sat, 05 Jan 2008) | 2 lines
clean up a comment
........
r59731 | jeffrey.yasskin | 2008-01-05 09:47:13 +0100 (Sat, 05 Jan 2008) | 11 lines
Continue rolling back pep-3141 changes that changed behavior from 2.5. This
round included:
* Revert round to its 2.6 behavior (half away from 0).
* Because round, floor, and ceil always return float again, it's no
longer necessary to have them delegate to __xxx___, so I've ripped
that out of their implementations and the Real ABC. This also helps
in implementing types that work in both 2.6 and 3.0: you return int
from the __xxx__ methods, and let it get enabled by the version
upgrade.
* Make pow(-1, .5) raise a ValueError again.
........
r59736 | andrew.kuchling | 2008-01-05 16:13:49 +0100 (Sat, 05 Jan 2008) | 1 line
Fix comment typo
........
r59738 | thomas.heller | 2008-01-05 18:15:44 +0100 (Sat, 05 Jan 2008) | 1 line
Add myself.
........
r59739 | georg.brandl | 2008-01-05 18:49:17 +0100 (Sat, 05 Jan 2008) | 2 lines
Fix C++-style comment.
........
r59742 | georg.brandl | 2008-01-05 20:28:16 +0100 (Sat, 05 Jan 2008) | 2 lines
Remove with_statement future imports from 2.6 docs.
........
r59743 | georg.brandl | 2008-01-05 20:29:45 +0100 (Sat, 05 Jan 2008) | 2 lines
Simplify index entries; fix#1712.
........
r59744 | georg.brandl | 2008-01-05 20:44:22 +0100 (Sat, 05 Jan 2008) | 2 lines
Doc patch #1730 from Robin Stocker; minor corrections mostly to os.rst.
........
r59749 | georg.brandl | 2008-01-05 21:29:13 +0100 (Sat, 05 Jan 2008) | 2 lines
Revert socket.rst to unix-eol.
........
r59750 | georg.brandl | 2008-01-05 21:33:46 +0100 (Sat, 05 Jan 2008) | 2 lines
Set native svn:eol-style property for text files.
........
r59752 | georg.brandl | 2008-01-05 21:46:29 +0100 (Sat, 05 Jan 2008) | 2 lines
#1719: capitalization error in "UuidCreate".
........
r59753 | georg.brandl | 2008-01-05 22:02:25 +0100 (Sat, 05 Jan 2008) | 2 lines
Repair markup.
........
r59754 | georg.brandl | 2008-01-05 22:10:50 +0100 (Sat, 05 Jan 2008) | 2 lines
Use markup.
........
r59757 | christian.heimes | 2008-01-05 22:35:52 +0100 (Sat, 05 Jan 2008) | 1 line
Final adjustments for #1601
........
r59758 | guido.van.rossum | 2008-01-05 23:19:06 +0100 (Sat, 05 Jan 2008) | 3 lines
Patch #1637: fix urlparse for URLs like 'http://x.com?arg=/foo'.
Fix by John Nagle.
........
r59759 | guido.van.rossum | 2008-01-05 23:20:01 +0100 (Sat, 05 Jan 2008) | 2 lines
Add John Nagle (of issue #1637).
........
r59765 | raymond.hettinger | 2008-01-06 10:02:24 +0100 (Sun, 06 Jan 2008) | 1 line
Small code simplification. Forgot that classmethods can be called from intances.
........
r59766 | martin.v.loewis | 2008-01-06 11:09:48 +0100 (Sun, 06 Jan 2008) | 2 lines
Use vcbuild for VS 2009.
........
r59767 | martin.v.loewis | 2008-01-06 12:03:43 +0100 (Sun, 06 Jan 2008) | 2 lines
Package using VS 2008.
........
r59768 | martin.v.loewis | 2008-01-06 12:13:16 +0100 (Sun, 06 Jan 2008) | 2 lines
Don't try to package msvcr90 for the moment.
........
r59769 | georg.brandl | 2008-01-06 15:17:36 +0100 (Sun, 06 Jan 2008) | 4 lines
#1696393: don't check for '.' and '..' in ntpath.walk since
they aren't returned from os.listdir anymore.
Reported by Michael Haggerty.
........
r59770 | georg.brandl | 2008-01-06 15:27:15 +0100 (Sun, 06 Jan 2008) | 3 lines
#1742: don't raise exception on os.path.relpath("a", "a"), but return os.curdir.
Reported by Jesse Towner.
........
r59771 | georg.brandl | 2008-01-06 15:33:52 +0100 (Sun, 06 Jan 2008) | 2 lines
#1591: Clarify docstring of Popen3.
........
r59772 | georg.brandl | 2008-01-06 16:30:34 +0100 (Sun, 06 Jan 2008) | 2 lines
#1680: fix context manager example function name.
........
r59773 | georg.brandl | 2008-01-06 16:34:57 +0100 (Sun, 06 Jan 2008) | 2 lines
#1755097: document default values for [].sort() and sorted().
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r59260 | lars.gustaebel | 2007-12-01 22:02:12 +0100 (Sat, 01 Dec 2007) | 5 lines
Issue #1531: Read fileobj from the current offset, do not seek to
the start.
(will backport to 2.5)
........
r59262 | georg.brandl | 2007-12-01 23:24:47 +0100 (Sat, 01 Dec 2007) | 4 lines
Document PyEval_* functions from ceval.c.
Credits to Michael Sloan from GHOP.
........
r59263 | georg.brandl | 2007-12-01 23:27:56 +0100 (Sat, 01 Dec 2007) | 2 lines
Add a few refcount data entries.
........
r59264 | georg.brandl | 2007-12-01 23:38:48 +0100 (Sat, 01 Dec 2007) | 4 lines
Add test suite for cmd module.
Written by Michael Schneider for GHOP.
........
r59265 | georg.brandl | 2007-12-01 23:42:46 +0100 (Sat, 01 Dec 2007) | 3 lines
Add examples to the ElementTree documentation.
Written by h4wk.cz for GHOP.
........
r59266 | georg.brandl | 2007-12-02 00:12:45 +0100 (Sun, 02 Dec 2007) | 3 lines
Add "Using Python on Windows" document, by Robert Lehmann.
Written for GHOP.
........
r59271 | georg.brandl | 2007-12-02 15:34:34 +0100 (Sun, 02 Dec 2007) | 3 lines
Add example to mmap docs.
Written for GHOP by Rafal Rawicki.
........
r59272 | georg.brandl | 2007-12-02 15:37:29 +0100 (Sun, 02 Dec 2007) | 2 lines
Convert bdb.rst line endings to Unix style.
........
r59274 | georg.brandl | 2007-12-02 15:58:50 +0100 (Sun, 02 Dec 2007) | 4 lines
Add more entries to the glossary.
Written by Jeff Wheeler for GHOP.
........
No detailed change log; just check out the change log for the py3k-pep3137
branch. The most obvious changes:
- str8 renamed to bytes (PyString at the C level);
- bytes renamed to buffer (PyBytes at the C level);
- PyString and PyUnicode are no longer compatible.
I.e. we now have an immutable bytes type and a mutable bytes type.
The behavior of PyString was modified quite a bit, to make it more
bytes-like. Some changes are still on the to-do list.
Lars Gustäbel
Georg Brandl
Senthil Kumaran
Benjamin Peterson
Marc-André Lemburg
Raymond Hettinger
Martin v. Löwis
Antoine Pitrou
Brian Curtin
Victor Stinner
Ronald Oussoren
Mark Dickinson
Christian Heimes
Guido van Rossum