Commit Graph

106 Commits

Author SHA1 Message Date
Lars Gustäbel 01277d166a Merge with 3.2: Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile.
The nti() function that converts a number field from a tar header to a number
failed to decode GNU tar specific base-256 fields. I also added support for
decoding and encoding negative base-256 number fields.
2011-10-14 12:53:10 +02:00
Lars Gustäbel ac3d137a30 Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile.
The nti() function that converts a number field from a tar header to a number
failed to decode GNU tar specific base-256 fields. I also added support for
decoding and encoding negative base-256 number fields.
2011-10-14 12:46:40 +02:00
Victor Stinner 9c3de4a883 Issue #12326: don't test the major version of sys.platform
Use startswith, instead of ==, when testing sys.platform to support
new platforms like Linux 3 or OpenBSD 5.
2011-08-17 20:49:41 +02:00
Georg Brandl 74b6abf61f Merge with 3.2. 2011-08-13 11:48:40 +02:00
Georg Brandl 3abb372c81 Fix #11513: wrong exception handling for the case that GzipFile itself raises an IOError. 2011-08-13 11:48:12 +02:00
Victor Stinner bf816223df Issue #12451: Add support.create_empty_file()
We don't need to create a temporary buffered binary or text file object just to
create an empty file.

Replace also os.fdopen(handle).close() by os.close(handle).
2011-06-30 23:25:47 +02:00
Victor Stinner 4e86d5b88d Replace open(filename, 'rU') by open(filename, 'r')
The U flag is no more used (but still accepted for backward compatibility).
2011-05-04 13:55:36 +02:00
Senthil Kumaran 4a0afa224c Wrap the correct test with the skip decorator for the issue10761.
merge from 3.2.
2011-04-30 06:12:25 +08:00
Senthil Kumaran be5dbebeaa merge from 3.1 2011-04-30 06:09:51 +08:00
Senthil Kumaran f108f8258e Wrap the testskip decorator for the proper test to resolve bb failure. 2011-04-30 06:06:28 +08:00
Senthil Kumaran 2ea72709a3 merge from 3.2 2011-04-28 17:08:41 +08:00
Senthil Kumaran c20566cdf8 merge from 3.1 2011-04-28 17:08:12 +08:00
Senthil Kumaran e3bdcf4ffd skip the extractall test on platforms where os.symlink is not available. 2011-04-28 17:05:55 +08:00
Senthil Kumaran 7274828b48 merge from 3.2 2011-04-28 15:53:59 +08:00
Senthil Kumaran 8a410d319a merge from 3.1 2011-04-28 15:53:09 +08:00
Senthil Kumaran 123932f237 Add tests for tarfile extractall feature when with symlinks 2011-04-28 15:38:12 +08:00
Marc-André Lemburg 8f36af7a4c Normalize the encoding names for Latin-1 and UTF-8 to
'latin-1' and 'utf-8'.

These are optimized in the Python Unicode implementation
to result in more direct processing, bypassing the codec
registry.

Also see issue11303.
2011-02-25 15:42:01 +00:00
Lars Gustäbel 9f6cbe09cc Merged revisions 88528 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r88528 | lars.gustaebel | 2011-02-23 12:42:22 +0100 (Wed, 23 Feb 2011) | 16 lines

  Issue #11224: Improved sparse file read support (r85916) introduced a
  regression in _FileInFile which is used in file-like objects returned
  by TarFile.extractfile(). The inefficient design of the
  _FileInFile.read() method causes various dramatic side-effects and
  errors:

    - The data segment of a file member is read completely into memory
      every(!) time a small block is accessed. This is not only slow
      but may cause unexpected MemoryErrors with very large files.
    - Reading members from compressed tar archives is even slower
      because of the excessive backwards seeking which is done when the
      same data segment is read over and over again.
    - As a backwards seek on a TarFile opened in stream mode is not
      possible, using extractfile() fails with a StreamError.
........
2011-02-23 11:52:31 +00:00
Lars Gustäbel dd071045e7 Issue #11224: Improved sparse file read support (r85916) introduced a
regression in _FileInFile which is used in file-like objects returned
by TarFile.extractfile(). The inefficient design of the
_FileInFile.read() method causes various dramatic side-effects and
errors:

  - The data segment of a file member is read completely into memory
    every(!) time a small block is accessed. This is not only slow
    but may cause unexpected MemoryErrors with very large files.
  - Reading members from compressed tar archives is even slower
    because of the excessive backwards seeking which is done when the
    same data segment is read over and over again.
  - As a backwards seek on a TarFile opened in stream mode is not
    possible, using extractfile() fails with a StreamError.
2011-02-23 11:42:22 +00:00
Raymond Hettinger a63a312a3f Issue #11014: Make 'filter' argument in tarfile.Tarfile.add() into a
keyword-only argument.  The preceding positional argument was deprecated,
so it made no sense to add filter as a positional argument.

(Patch reviewed by Brian Curtin and Anthony Long.)
2011-01-26 20:34:14 +00:00
Brian Curtin 3b4499c5c7 Fix #9333. The symlink function is always available now, raising OSError
when the user doesn't hold the symbolic link privilege rather than hiding it.
2010-12-28 14:31:47 +00:00
Brian Curtin 52173d4959 Fix #9333. Expose os.symlink on Windows only when usable.
In order to create symlinks on Windows, SeCreateSymbolicLinkPrivilege
is an account privilege that is required to be held by the user. Not only
must the privilege be enabled for the account, the activated privileges for
the currently running application must be adjusted to enable the requested
privilege.

Rather than exposing an additional function to be called prior to the user's
first os.symlink call, we handle the AdjustTokenPrivileges Windows API call
internally and only expose os.symlink when the privilege escalation was
successful.

Due to the change of only exposing os.symlink when it's available, we can
go back to the original test skipping methods of checking via `hasattr`.
2010-12-02 18:29:18 +00:00
Ezio Melotti b3aedd4862 #9424: Replace deprecated assert* methods in the Python test suite. 2010-11-20 19:04:17 +00:00
Martin v. Löwis be647e288d Remove extracted trees at the end of the test. 2010-11-01 22:08:46 +00:00
Martin v. Löwis 16f344df36 Issue #10184: Touch directories only once when extracting a tarfile. 2010-11-01 21:39:13 +00:00
Antoine Pitrou e1eca4e3f5 Issue #10233: Close file objects in a timely manner in the tarfile module
and its test suite.
2010-10-29 23:49:49 +00:00
Victor Stinner 14d8fe7de8 test_tarfile: woops, remove +0.001 which was used to debug 2010-10-29 11:02:06 +00:00
Victor Stinner 26bfb5ab82 test_tarfile: dump mtime as hexadecimal on test_extractall failure 2010-10-29 10:59:08 +00:00
Lars Gustäbel 9cbdd75ec5 Add read support for all missing variants of the GNU sparse
extensions. Thus, in addition to GNUTYPE_SPARSE headers, sparse
information in pax headers created by GNU tar can now be decoded.
All three formats 0.0, 0.1 and 1.0 are supported.
On filesystems that support this, holes in files are now restored
whenever a sparse member is extracted.
2010-10-29 09:08:19 +00:00
Martin v. Löwis 57f75799bf Revert r85801. The test ought to pass without rounding. 2010-10-23 19:02:30 +00:00
Victor Stinner 2b9af63b4f Try to fix test_tarfile on AMD64 debian parallel 3.x
tarinfo.mtime is an int, whereas getmtime() is a float and it might be
different by something like 3 µs.
2010-10-23 01:15:30 +00:00
Antoine Pitrou 605c293031 Further tarfile / test_tarfile cleanup 2010-09-23 20:15:14 +00:00
Antoine Pitrou ab58b5f90e Try harder to fix test_tarfile under Windows buildbots 2010-09-23 19:39:35 +00:00
Antoine Pitrou 95f5560b46 Try to fix test_tarfile issues on Windows buildbots by closing file
objects explicitly instead of letting them linger on.
2010-09-23 18:36:46 +00:00
Brian Curtin 74e4561a3c Re-flow several long lines from #1578269. 2010-07-09 15:58:59 +00:00
Brian Curtin d40e6f70a5 Implement #1578269. Patch by Jason R. Coombs.
Added Windows support for os.symlink when run on Windows 6.0 or greater,
aka Vista. Previous Windows versions will raise NotImplementedError
when trying to symlink.

Includes numerous test updates and additions to test_os, including
a symlink_support module because of the fact that privilege escalation
is required in order to run the tests to ensure that the user is able
to create symlinks. By default, accounts do not have the required
privilege, so the escalation code will have to be exposed later (or
documented on how to do so). I'll be following up with that work next.

Note that the tests use ctypes, which was agreed on during the PyCon
language summit.
2010-07-08 21:39:08 +00:00
Lars Gustäbel 1b51272b1b Merged revisions 81667 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81667 | lars.gustaebel | 2010-06-03 14:34:14 +0200 (Thu, 03 Jun 2010) | 8 lines

  Issue #8741: Fixed the TarFile.makelink() method that is responsible
  for extracting symbolic and hard link entries as regular files as a
  work-around on platforms that do not support filesystem links.

  This stopped working reliably after a change in r74571. I also added
  a few tests for this functionality.
........
2010-06-03 12:45:16 +00:00
Lars Gustäbel 547f808e26 Merged revisions 81665 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r81665 | lars.gustaebel | 2010-06-03 12:11:52 +0200 (Thu, 03 Jun 2010) | 11 lines

  Merged revisions 81663 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r81663 | lars.gustaebel | 2010-06-03 11:56:22 +0200 (Thu, 03 Jun 2010) | 4 lines

    Issue #8833: tarfile created hard link entries with a size
    field != 0 by mistake. The associated testcase did not
    expose this bug because it was broken too.
  ........
................
2010-06-03 10:15:18 +00:00
Lars Gustäbel 2470ff19e6 Merged revisions 81663 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81663 | lars.gustaebel | 2010-06-03 11:56:22 +0200 (Thu, 03 Jun 2010) | 4 lines

  Issue #8833: tarfile created hard link entries with a size
  field != 0 by mistake. The associated testcase did not
  expose this bug because it was broken too.
........
2010-06-03 10:11:52 +00:00
Lars Gustäbel 1465cc2887 Issue #8633: Support for POSIX.1-2008 binary pax headers.
tarfile is now able to read and write pax headers with a
"hdrcharset=BINARY" record. This record was introduced in
POSIX.1-2008 as a method to store unencoded binary strings that
cannot be translated to UTF-8. In practice, this is just a workaround
that allows a tar implementation to store filenames that do not
comply with the current filesystem encoding and thus cannot be
decoded correctly.
Additionally, tarfile works around a bug in current versions of GNU
tar: undecodable filenames are stored as-is in a pax header without a
"hdrcharset" record being added. Technically, these headers are
invalid, but tarfile manages to read them correctly anyway.
2010-05-17 18:02:50 +00:00
Victor Stinner de629d46f2 Issue #8390: tarfile uses surrogateespace as the default error handler
(instead of replace in read mode or strict in write mode)
2010-05-05 21:43:57 +00:00
Lars Gustäbel f7317f9efc Merged revisions 80618 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r80618 | lars.gustaebel | 2010-04-29 17:37:02 +0200 (Thu, 29 Apr 2010) | 10 lines

  Merged revisions 80616 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r80616 | lars.gustaebel | 2010-04-29 17:23:38 +0200 (Thu, 29 Apr 2010) | 4 lines

    Issue #8464: tarfile.open(name, mode="w|") no longer creates
    files with execute permissions set.
  ........
................
2010-04-29 15:42:25 +00:00
Lars Gustäbel d6eb70b7b4 Merged revisions 80616 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r80616 | lars.gustaebel | 2010-04-29 17:23:38 +0200 (Thu, 29 Apr 2010) | 4 lines

  Issue #8464: tarfile.open(name, mode="w|") no longer creates
  files with execute permissions set.
........
2010-04-29 15:37:02 +00:00
Benjamin Peterson 886af966d8 Merged revisions 78966,78970,79018,79026-79027,79055,79156,79159,79163-79164,79173,79176,79194,79208,79212 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78966 | florent.xicluna | 2010-03-14 10:20:59 -0500 (Sun, 14 Mar 2010) | 2 lines

  Do not hardcode Expat version.  It's possible to build Python with --with-system-expat option.
........
  r78970 | benjamin.peterson | 2010-03-14 21:58:24 -0500 (Sun, 14 Mar 2010) | 1 line

  this little exception dance is pointless
........
  r79018 | collin.winter | 2010-03-16 22:04:01 -0500 (Tue, 16 Mar 2010) | 1 line

  Delete unused import.
........
  r79026 | vinay.sajip | 2010-03-17 10:05:57 -0500 (Wed, 17 Mar 2010) | 1 line

  Issue #8162: logging: Clarified docstring and documentation for disable function.
........
  r79027 | collin.winter | 2010-03-17 12:36:16 -0500 (Wed, 17 Mar 2010) | 1 line

  Avoid hardcoding refcounts in tests.
........
  r79055 | benjamin.peterson | 2010-03-18 16:30:48 -0500 (Thu, 18 Mar 2010) | 1 line

  remove installation of deleted test/output dir
........
  r79156 | florent.xicluna | 2010-03-20 17:21:02 -0500 (Sat, 20 Mar 2010) | 2 lines

  Cleanup test_struct using check_warnings.
........
  r79159 | florent.xicluna | 2010-03-20 17:26:42 -0500 (Sat, 20 Mar 2010) | 2 lines

  Cleanup test_tarfile, and use check_warnings.
........
  r79163 | michael.foord | 2010-03-20 19:53:39 -0500 (Sat, 20 Mar 2010) | 1 line

  A faulty load_tests in a test module no longer halts test discovery. A placeholder test, that reports the failure, is created instead.
........
  r79164 | michael.foord | 2010-03-20 19:55:58 -0500 (Sat, 20 Mar 2010) | 1 line

  Change order of arguments in a unittest function.
........
  r79173 | georg.brandl | 2010-03-21 04:09:38 -0500 (Sun, 21 Mar 2010) | 1 line

  Document that GzipFile supports iteration.
........
  r79176 | georg.brandl | 2010-03-21 04:17:41 -0500 (Sun, 21 Mar 2010) | 1 line

  Introduce copy by slicing, used in later chapters.
........
  r79194 | florent.xicluna | 2010-03-21 06:58:11 -0500 (Sun, 21 Mar 2010) | 2 lines

  Use assertRaises and add a specific warning filter.
........
  r79208 | andrew.kuchling | 2010-03-21 13:47:12 -0500 (Sun, 21 Mar 2010) | 1 line

  Add items
........
  r79212 | georg.brandl | 2010-03-21 14:01:38 -0500 (Sun, 21 Mar 2010) | 1 line

  Fix plural.
........
2010-03-21 23:13:07 +00:00
Georg Brandl 1b37e8728c Merged revisions 78093 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78093 | georg.brandl | 2010-02-07 18:03:15 +0100 (So, 07 Feb 2010) | 1 line

  Remove unused imports in test modules.
........
2010-03-14 10:45:50 +00:00
Lars Gustäbel 0138581c43 Merged revisions 78623 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78623 | lars.gustaebel | 2010-03-03 12:55:48 +0100 (Wed, 03 Mar 2010) | 3 lines

  Issue #7232: Add support for the context manager protocol
  to the TarFile class.
........
2010-03-03 12:08:54 +00:00
Benjamin Peterson 577473fe68 use assert[Not]In where appropriate
A patch from Dave Malcolm.
2010-01-19 00:09:57 +00:00
Lars Gustäbel 45711a7237 Merged revisions 76453 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r76453 | lars.gustaebel | 2009-11-23 16:48:33 +0100 (Mon, 23 Nov 2009) | 10 lines

  Merged revisions 76452 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r76452 | lars.gustaebel | 2009-11-23 16:46:19 +0100 (Mon, 23 Nov 2009) | 3 lines

    Add a testcase that checks if the TarFile constructor successfully
    closes the internal file object in case of an error (issue #7341).
  ........
................
2009-11-23 16:04:57 +00:00
Lars Gustäbel b7f09238d8 Merged revisions 76452 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r76452 | lars.gustaebel | 2009-11-23 16:46:19 +0100 (Mon, 23 Nov 2009) | 3 lines

  Add a testcase that checks if the TarFile constructor successfully
  closes the internal file object in case of an error (issue #7341).
........
2009-11-23 15:48:33 +00:00
Lars Gustäbel 9520a430ef Merged revisions 76443 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r76443 | lars.gustaebel | 2009-11-22 19:30:53 +0100 (Sun, 22 Nov 2009) | 24 lines

  Issue #6123: Fix opening empty archives and files.

  (Note that an empty archive is not the same as an empty file. An
  empty archive contains no members and is correctly terminated with an
  EOF block full of zeros. An empty file contains no data at all.)

  The problem was that although tarfile was able to create empty
  archives, it failed to open them raising a ReadError. On the other
  hand, tarfile opened empty files without error in most read modes and
  presented them as empty archives. (However, some modes still raised
  errors: "r|gz" raised ReadError, but "r:gz" worked, "r:bz2" even
  raised EOFError.)

  In order to get a more fine-grained control over the various internal
  error conditions I now split up the HeaderError exception into a
  number of meaningful sub-exceptions. This makes it easier in the
  TarFile.next() method to react to the different conditions in the
  correct way.

  The visible change in its behaviour now is that tarfile will open
  empty archives correctly and raise ReadError consistently for empty
  files.
........
2009-11-22 18:48:49 +00:00