Commit Graph

117 Commits

Author SHA1 Message Date
Giampaolo Rodola c7f02a9659
bpo-33671 / shutil.copyfile: use memoryview() with dynamic size on Windows (#7681)
bpo-33671
* use memoryview() with size == file size on Windows, see https://github.com/python/cpython/pull/7160#discussion_r195405230
* release intermediate (sliced) memoryview immediately
* replace "OSX" occurrences with "macOS"
* add some unittests for copyfileobj()
2018-06-19 08:27:29 -07:00
Giampaolo Rodola 4a172ccc73
bpo-33671: efficient zero-copy for shutil.copy* functions (Linux, OSX and Win) (#7160)
* have shutil.copyfileobj use sendfile() if possible

* refactoring: use ctx manager

* add test with non-regular file obj

* emulate case where file size can't be determined

* reference _copyfileobj_sendfile directly

* add test for offset() at certain position

* add test for empty file

* add test for non regular file dst

* small refactoring

* leave copyfileobj() alone in order to not introduce any incompatibility

* minor refactoring

* remove old test

* update docstring

* update docstring; rename exception class

* detect platforms which only support file to socket zero copy

* don't run test on platforms where file-to-file zero copy is not supported

* use tempfiles

* reset verbosity

* add test for smaller chunks

* add big file size test

* add comment

* update doc

* update whatsnew doc

* update doc

* catch Exception

* remove unused import

* add test case for error on second sendfile() call

* turn docstring into comment

* add one more test

* update comment

* add Misc/NEWS entry

* get rid of COPY_BUFSIZE; it belongs to another PR

* update doc

* expose posix._fcopyfile() for OSX

* merge from linux branch

* merge from linux branch

* expose fcopyfile

* arg clinic for the win implementation

* convert path type to path_t

* expose CopyFileW

* fix windows tests

* release GIL

* minor refactoring

* update doc

* update comment

* update docstrings

* rename functions

* rename test classes

* update doc

* update doc

* update docstrings and comments

* avoid do import nt|posix modules if unnecessary

* set nt|posix modules to None if not available

* micro speedup

* update description

* add doc note

* use better wording in doc

* rename function using 'fastcopy' prefix instead of 'zerocopy'

* use :ref: in rst doc

* change wording in doc

* add test to make sure sendfile() doesn't get called aymore in case it doesn't support file to file copies

* move CopyFileW in _winapi and actually expose CopyFileExW instead

* fix line endings

* add tests for mode bits

* add docstring

* remove test file mode class; let's keep it for later when Istart addressing OSX fcopyfile() specific copies

* update doc to reflect new changes

* update doc

* adjust tests on win

* fix argument clinic error

* update doc

* OSX: expose copyfile(3) instead of fcopyfile(3); also expose flags arg to python

* osx / copyfile: use path_t instead of char

* do not set dst name in the OSError exception in order to remain consistent with platforms which cannot do that (e.g. linux)

* add same file test

* add test for same file

* have osx copyfile() pre-emptively check if src and dst are the same, otherwise it will return immedialtey and src file content gets deleted

* turn PermissionError into appropriate SameFileError

* expose ERROR_SHARING_VIOLATION in order to raise more appropriate SameFileError

* honour follow_symlinks arg when using CopyFileEx

* update Misc/NEWS

* expose CreateDirectoryEx mock

* change C type

* CreateDirectoryExW actual implementation

* provide specific makedirs() implementation for win

* fix typo

* skeleton for SetNamedSecurityInfo

* get security info for src path

* finally set security attrs

* add unit tests

* mimick os.makedirs() behavior and raise if dst dir exists

* set 2 paths for OSError object

* set 2 paths for OSError object

* expand windows test

* in case of exception on os.sendfile() set filename and filename2 exception attributes

* set 2 filenames (src, dst) for OSError in case copyfile() fails on OSX

* update doc

* do not use CreateDirectoryEx() in copytree() if source dir is a symlink (breaks test_copytree_symlink_dir); instead just create a plain dir and remain consistent with POSIX implementation

* use bytearray() and readinto()

* use memoryview() with bytearray()

* refactoring + introduce a new _fastcopy_binfileobj() fun

* remove CopyFileEx and other C wrappers

* remove code related to CopyFileEx

* Recognize binary files in copyfileobj()
...and use fastest _fastcopy_binfileobj() when possible

* set 1MB copy bufsize on win; also add a global _COPY_BUFSIZE variable

* use ctx manager for memoryview()

* update doc

* remove outdated doc

* remove last CopyFileEx remnants

* OSX - use fcopyfile(3) instead of copyfile(3)

...as an extra safety measure: in case src/dst are "exotic" files (non
regular or living on a network fs etc.) we better fail on open() instead
of copyfile(3) as we're not quite sure what's gonna happen in that
case.

* update doc
2018-06-12 23:04:50 +02:00
Cheryl Sabella ee3b83547c bpo-26330: Update shutil.disk_usage() documentation (GH-5184)
Clarify that on Windows, path must be a directory.
On Unix, path can be a file or a directory.
2018-01-14 21:08:37 -08:00
Jelle Zijlstra a12df7b7d4 bpo-30218: support path-like objects in shutil.unpack_archive() (GH-1367)
Thanks to Jelle Zijlstra for the patch.
2017-05-05 14:27:12 -07:00
Mariatta 70ee0cd5c2 bpo-29784: Fix the reference to shutil.copy in the docs (GH-602) 2017-03-10 18:17:21 -08:00
Berker Peksag fe607beef7 Merge from 3.5 2016-12-27 15:09:35 +03:00
Berker Peksag 8e2bdc8fe8 Move versionadded inside of the get_terminal_size() block 2016-12-27 15:09:11 +03:00
Martin Panter fff07e34fa Merge spelling and grammar from 3.5 2016-12-18 05:37:21 +00:00
Martin Panter 2f9171d900 Fix spelling and grammar in code comments and documentation 2016-12-18 01:23:09 +00:00
Serhiy Storchaka 9bb6fe5274 Issue #14061: Misc fixes and cleanups in archiving code in shutil.
Imporoved the documentation and tests for make_archive() and unpack_archive().
Improved error handling when corresponding compress module is not available.
Brake circular dependency between shutil and tarfile modules.
2016-12-16 19:00:55 +02:00
Serhiy Storchaka 20cdffd830 Issue #14061: Misc fixes and cleanups in archiving code in shutil.
Imporoved the documentation and tests for make_archive() and unpack_archive().
Improved error handling when corresponding compress module is not available.
Brake circular dependency between shutil and tarfile modules.
2016-12-16 18:58:33 +02:00
Martin Panter 99b6f28351 Merge doc fixups from 3.5 2016-11-21 04:22:22 +00:00
Martin Panter 357ed2e577 Change double hyphens (en dashes) to em (longer) dashes 2016-11-21 00:15:20 +00:00
Vinay Sajip dd917f84e3 Closes #27904: Improved logging statements to defer formatting until needed. 2016-08-31 08:22:29 +01:00
Martin Panter 1050d2d0c7 Issue #26462: Doc: reduce literal_block warnings, fix syntax highlighting.
Patch by Julien Palard.
2016-07-26 11:18:21 +02:00
Terry Jan Reedy fa089b9b0b Issue #22558: Add remaining doc links to source code for Python-coded modules.
Reformat header above separator line (added if missing) to a common format.
Patch by Yoni Lavi.
2016-06-11 15:02:54 -04:00
Serhiy Storchaka 6a7b3a77b4 Issue #26778: Fixed "a/an/and" typos in code comment and documentation. 2016-04-17 08:32:47 +03:00
Martin Panter 2eb819f7a8 Issue #25523: Merge "a" to "an" fixes from 3.4 into 3.5 2015-11-02 04:04:57 +00:00
Martin Panter 7462b64911 Issue #25523: Correct "a" article to "an" article
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
2015-11-02 03:37:02 +00:00
Martin Panter 84835ab1cb Issue #25161: Merge full stops from 3.4 into 3.5 2015-10-10 10:44:25 +00:00
Martin Panter d21e0b52f1 Issue #25161: Add full stops in documentation; patch by Takase Arihiro 2015-10-10 10:36:22 +00:00
Senthil Kumaran b4760efbad Back porting changeset db302b88fdb6 to 3.4 branch, which fixed multiple documentation typos.
Related Issues:

#issue21528
#issue24453
2015-06-14 17:35:37 -07:00
Benjamin Peterson 67057ab57c merge 3.4 (#22933) 2015-03-22 10:15:12 -04:00
Benjamin Peterson 218144a94d clarify behavior of shutil.move when destination exists (closes #22933)
Patch by Mike Short.
2015-03-22 10:11:54 -04:00
Georg Brandl 455de40a6e merge with 3.4 2014-10-31 10:54:10 +01:00
Georg Brandl 36ac510956 Closes #22771: "verbose" is unused; deprecate it. 2014-10-31 10:54:06 +01:00
Georg Brandl e21a531ef1 merge with 3.4 2014-10-31 10:39:29 +01:00
Georg Brandl 9b1b0e54e7 #22613: elaborate on arguments of shutil archival functions (thanks Jacques Ducasse) 2014-10-31 10:02:40 +01:00
Berker Peksag ae04ba1952 Fix usage of the versionchanged directive.
Patch by Akira Li.
2014-08-11 18:10:25 +03:00
Serhiy Storchaka f3440c6881 Issue #5411: Fixed version number. 2014-08-06 18:55:54 +03:00
Serhiy Storchaka 1121377cdb Issue #5411: Added support for the "xztar" format in the shutil module. 2014-08-06 18:50:19 +03:00
R David Murray 6ffface429 #19840: Add copy_function to shutil.move.
Patch by Claudiu Popa.
2014-06-11 14:40:13 -04:00
Donald Stufft 8b852f111e Fix Issue #21528 - Fix documentation typos 2014-05-20 12:58:38 -04:00
Tim Golden ba74885e0c Issue19643 Fix whitespace 2014-05-07 18:08:08 +01:00
Tim Golden 783377998a Issue19643 Add an example of shutil.rmtree which shows how to cope with readonly files on Windows 2014-05-07 18:05:45 +01:00
Victor Stinner 1d006a2c0e Backout a1a05e2724dd: shutil.which(bytes) is a new feature and my patch does not work on Windows 2013-12-16 23:39:40 +01:00
Victor Stinner fffb96ba66 Issue #18283: shutil.which() now supports bytes argument, not only text argument. 2013-12-16 22:48:48 +01:00
Serhiy Storchaka 2ba095c6c5 Fix shutil.which() test for issue #16993. 2013-01-22 10:32:16 +02:00
Serhiy Storchaka 80c88f4304 Fix shutil.which() test for issue #16993. 2013-01-22 10:31:36 +02:00
Serhiy Storchaka 5b987c2e68 Issue #16993: shutil.which() now preserves the case of the path and extension
on Windows.
2013-01-21 15:01:34 +02:00
Serhiy Storchaka 014791f848 Issue #16993: shutil.which() now preserves the case of the path and extension
on Windows.
2013-01-21 15:00:27 +02:00
Andrew Svetlov 2606a6f197 Issue #16719: Get rid of WindowsError. Use OSError instead
Patch by Serhiy Storchaka.
2012-12-19 14:33:35 +02:00
Andrew Svetlov 88a3e1f46b Merge: use OSError instead of os.error in the docs. 2012-12-15 22:59:55 +02:00
Andrew Svetlov 618c2e13ca Use OSError instead of os.error in the docs. 2012-12-15 22:59:24 +02:00
Hynek Schlawack 27ddb576f1 #1492704: Ensure and document backward compatibility of the change 2012-10-28 13:59:27 +01:00
Hynek Schlawack 4865376c44 Closes #1492704: Make shutil.copyfile() raise a distinct SameFileError
Patch by Atsuo Ishimoto.
2012-10-07 12:49:58 +02:00
Larry Hastings 60eba57f3e Cleanup/rewrite shutil docs regarding follow_symlinks and copying attributes. 2012-09-21 10:12:14 -07:00
Hynek Schlawack 26fe37dd3f #1492704: Backout and wait for 3.4 2012-07-19 21:41:02 +02:00
Hynek Schlawack 77d3283370 #1492704: Make shutil.copyfile() raise a distinct SameFileError
Patch by Atsuo Ishimoto.
2012-07-19 20:23:49 +02:00
Larry Hastings 7aa2c8baae Issue #15202: Additional documentation fixes inadvertently omitted
from previous checkin.
2012-07-15 16:58:29 -07:00