Commit Graph

547 Commits

Author SHA1 Message Date
Antoine Pitrou 56452eea39 Issue #22982: Improve BOM handling when seeking to multiple positions of a writable text file. 2015-04-13 20:02:33 +02:00
Antoine Pitrou 85e3ee749c Issue #22982: Improve BOM handling when seeking to multiple positions of a writable text file. 2015-04-13 20:01:21 +02:00
Antoine Pitrou cb46f0ecb0 Issue #23309: Avoid a deadlock at shutdown if a daemon thread is aborted
while it is holding a lock to a buffered I/O object, and the main thread
tries to use the same I/O object (typically stdout or stderr).  A fatal
error is emitted instead.
2015-04-13 19:48:19 +02:00
Antoine Pitrou 25f85d4bd5 Issue #23309: Avoid a deadlock at shutdown if a daemon thread is aborted
while it is holding a lock to a buffered I/O object, and the main thread
tries to use the same I/O object (typically stdout or stderr).  A fatal
error is emitted instead.
2015-04-13 19:41:47 +02:00
Steve Dower a1c7e727c8 Issue #23668: Suppresses invalid parameter handler around chsize calls. 2015-04-12 00:26:43 -04:00
Steve Dower 8fc8980c96 Issue #23524: Replace _PyVerify_fd function with calls to _set_thread_local_invalid_parameter_handler. 2015-04-12 00:26:27 -04:00
Serhiy Storchaka 3d2279f9a8 Issue #21859: Corrected FileIO docstrings. 2015-04-10 16:08:43 +03:00
Berker Peksag ea6d5592f2 Issue #23796: peak and read1 methods of BufferedReader now raise ValueError
if they called on a closed object.

Patch by John Hergenroeder.
2015-05-12 17:13:56 +03:00
Steve Dower fe0a41aae4 Issue #23668: Adds support for os.truncate and os.ftruncate on Windows 2015-03-20 19:50:46 -07:00
Serhiy Storchaka cd092efb16 Issue #21859: Corrected FileIO docstrings. 2015-04-10 16:09:13 +03:00
Serhiy Storchaka 5056769b36 Replaced "string" with "bytes object" in docstrings of binary I/O objects. 2015-04-10 02:19:57 +03:00
Serhiy Storchaka b817b77a8c Replaced "string" with "bytes object" in docstrings of binary I/O objects. 2015-04-10 02:18:44 +03:00
Victor Stinner e134a7fe36 Issue #23752: _Py_fstat() is now responsible to raise the Python exception
Add _Py_fstat_noraise() function when a Python exception is not welcome.
2015-03-30 10:09:31 +02:00
Serhiy Storchaka 2e1c4e5db2 Issue #23785: Fixed memory leak in TextIOWrapper.tell() in rare circumstances. 2015-03-30 10:00:40 +03:00
Serhiy Storchaka 04d09ebd39 Issue #23785: Fixed memory leak in TextIOWrapper.tell() in rare circumstances. 2015-03-30 09:58:41 +03:00
Victor Stinner 520bddf79a Issue #23752: When built from an existing file descriptor, io.FileIO() now only
calls fstat() once. Before fstat() was called twice, which was not necessary.
2015-03-30 03:21:06 +02:00
Serhiy Storchaka 7665be6087 Issue #21802: The reader in BufferedRWPair now is closed even when closing
writer failed in BufferedRWPair.close().
2015-03-24 23:21:57 +02:00
Serhiy Storchaka 8be6be427d Issue #21802: The reader in BufferedRWPair now is closed even when closing
writer failed in BufferedRWPair.close().
2015-03-24 23:23:28 +02:00
Victor Stinner f329878e74 Issue #23753: Python doesn't support anymore platforms without stat() or
fstat(), these functions are always required.

Remove HAVE_STAT and HAVE_FSTAT defines, and stop supporting DONT_HAVE_STAT and
DONT_HAVE_FSTAT.
2015-03-24 10:27:50 +01:00
Victor Stinner 66aab0c4b5 Issue #23708: Add _Py_read() and _Py_write() functions to factorize code handle
EINTR error and special cases for Windows.

These functions now truncate the length to PY_SSIZE_T_MAX to have a portable
and reliable behaviour. For example, read() result is undefined if counter is
greater than PY_SSIZE_T_MAX on Linux.
2015-03-19 22:53:20 +01:00
Serhiy Storchaka 009b811d67 Removed unintentional trailing spaces in non-external and non-generated C files. 2015-03-18 21:53:15 +02:00
Steve Dower 8acde7dcce Issue #23524: Change back to using Windows errors for _Py_fstat instead of the errno shim. 2015-03-07 18:14:07 -08:00
Victor Stinner 4a7cc88472 Issue #23571: PyObject_Call(), PyCFunction_Call() and call_function() now
raise a SystemError if a function returns a result and raises an exception.
The SystemError is chained to the previous exception.

Refactor also PyObject_Call() and PyCFunction_Call() to make them more readable.

Remove some checks which became useless (duplicate checks).

Change reviewed by Serhiy Storchaka.
2015-03-06 23:35:27 +01:00
Steve Dower d81431f587 Issue #23524: Replace _PyVerify_fd function with calling _set_thread_local_invalid_parameter_handler on every thread. 2015-03-06 14:47:02 -08:00
Victor Stinner 9672da7bb4 Issue #23285: Fix handling of EINTR in fileio.c
Fix handling of EINTR: don't return None if PyErr_CheckSignals() raised an
exception.

Initialize also the length outside the loop to only initialize it once.
2015-03-04 18:40:10 +01:00
Steve Dower f2f373f593 Issue #23152: Implement _Py_fstat() to support files larger than 2 GB on Windows.
fstat() may fail with EOVERFLOW on files larger than 2 GB because the file size type is an signed 32-bit integer.
2015-02-21 08:44:05 -08:00
Serhiy Storchaka 254dd59068 Issue #5700: io.FileIO() called flush() after closing the file.
flush() was not called in close() if closefd=False.
2015-02-21 00:35:53 +02:00
Serhiy Storchaka a3712a9a6c Issue #5700: io.FileIO() called flush() after closing the file.
flush() was not called in close() if closefd=False.
2015-02-21 00:35:09 +02:00
Serhiy Storchaka 483405bcca Issue #22883: Got rid of outdated references to PyInt and PyString in comments. 2015-02-17 10:14:30 +02:00
Charles-François Natali 6e6c59b508 Issue #23285: PEP 475 -- Retry system calls failing with EINTR. 2015-02-07 13:27:50 +00:00
Stefan Krah 650c1e818d Issue #14203: Remove obsolete support for view==NULL in bytesiobuf_getbuffer()
and array_buffer_getbuf().
2015-02-03 21:43:23 +01:00
Serhiy Storchaka 38c30e6c8e Issue #15381: Fixed a bug in BytesIO.write().
It was expected that string_size == PyBytes_GET_SIZE(buf) if the buffer is
shared, but truncate() and __setstate__() can set string_size without
unsharing the buffer.
2015-02-03 18:51:58 +02:00
Serhiy Storchaka b9765eec5c Issue #15381: Try to fix refcount bug. Empty and 1-byte buffers are always shared. 2015-02-03 14:57:49 +02:00
Serhiy Storchaka 87d0b45485 Issue #15381: Optimized io.BytesIO to make less allocations and copyings. 2015-02-03 11:30:10 +02:00
Serhiy Storchaka 32ca3dcb97 Issue #23099: Closing io.BytesIO with exported buffer is rejected now to
prevent corrupting exported buffer.
2015-02-03 09:30:51 +02:00
Serhiy Storchaka c057c3859c Issue #23099: Closing io.BytesIO with exported buffer is rejected now to
prevent corrupting exported buffer.
2015-02-03 02:00:18 +02:00
Serhiy Storchaka 3dd3e26680 Issue #22896: Avoid to use PyObject_AsCharBuffer(), PyObject_AsReadBuffer()
and PyObject_AsWriteBuffer().
2015-02-03 01:25:42 +02:00
Serhiy Storchaka 4fdb68491e Issue #22896: Avoid to use PyObject_AsCharBuffer(), PyObject_AsReadBuffer()
and PyObject_AsWriteBuffer().
2015-02-03 01:21:08 +02:00
Benjamin Peterson 98beb7599e merge 3.4 (#23093) 2014-12-21 21:00:51 -06:00
Benjamin Peterson 10e76b67c9 allow more operations to work on detached streams (closes #23093)
Patch by Martin Panter.
2014-12-21 20:51:50 -06:00
Serhiy Storchaka 4954f9fcab Issue #17401: Output the closefd attribute as boolean. 2014-12-02 23:39:56 +02:00
Benjamin Peterson 994c7f76a3 merge 3.4 (#22849) 2014-11-12 10:23:35 -05:00
Benjamin Peterson 6c14f23100 fix possible double free in TextIOWrapper.__init__ (closes #22849) 2014-11-12 10:19:46 -05:00
Robert Collins 933430ab69 Issue #17401: document closefd in io.FileIO docs and add to repr
closefd was documented in the open docs but not the matching FileIO
class documented. Further, closefd, part of the core state for the
object was not shown.

In review it was noted that the open docs are a little confusing about
the interaction between closefd and paths, so tweaked them at the same
time.
2014-10-18 13:32:43 +13:00
Serhiy Storchaka 78184af9b5 Issue #21715: Extracted shared complicated code in the _io module to new
_PyErr_ChainExceptions() function.
2014-10-08 22:32:50 +03:00
Serhiy Storchaka e2bd2a7186 Issue #21715: Extracted shared complicated code in the _io module to new
_PyErr_ChainExceptions() function.
2014-10-08 22:31:52 +03:00
Benjamin Peterson 58ee2d31ab merge 3.4 (#22517) 2014-09-29 22:49:05 -04:00
Benjamin Peterson c44eb73473 merge 3.3 (#22517) 2014-09-29 22:48:51 -04:00
Benjamin Peterson bbd0a323ae clear BufferedRWPair weakrefs on deallocation (closes #22517) 2014-09-29 22:46:57 -04:00
Berker Peksag 43705d76aa Issue #21860: Correct docstrings of FileIO.seek() and FileIO.truncate() methods.
Patch by Terry Chia.
2014-09-24 12:44:06 +03:00
Berker Peksag b87630c273 Issue #21860: Correct docstrings of FileIO.seek() and FileIO.truncate() methods.
Patch by Terry Chia.
2014-09-24 12:43:29 +03:00
Serhiy Storchaka d8a1447c99 Issue #22215: Now ValueError is raised instead of TypeError when str or bytes
argument contains not permitted null character or byte.
2014-09-06 20:07:17 +03:00
Victor Stinner 706768c687 Issue #22156: Fix some "comparison between signed and unsigned integers"
compiler warnings in the Modules/ subdirectory.
2014-08-16 01:03:39 +02:00
Serhiy Storchaka d7728cafc5 Issue #15381: Optimized line reading in io.BytesIO. 2014-08-14 22:26:38 +03:00
Antoine Pitrou cc66a73d27 Issue #22003: When initialized from a bytes object, io.BytesIO() now
defers making a copy until it is mutated, improving performance and
memory use on some use cases.

Patch by David Wilson.
2014-07-29 19:41:11 -04:00
Benjamin Peterson e865128605 properly decref the return value of close() 2014-07-04 17:00:25 -07:00
Benjamin Peterson 4f654fbe30 properly decref the return value of close() 2014-07-04 17:00:25 -07:00
Victor Stinner 6680e9f5fd (Merge 3.4) Issue #21090: io.FileIO.readall() does not ignore I/O errors
anymore. Before, it ignored I/O errors if at least the first C call read()
succeed.
2014-07-02 23:00:38 +02:00
Victor Stinner e10920f0d1 Issue #21090: io.FileIO.readall() does not ignore I/O errors anymore. Before,
it ignored I/O errors if at least the first C call read() succeed.
2014-07-02 22:59:31 +02:00
Antoine Pitrou de68722ca0 Issue #21679: Prevent extraneous fstat() calls during open(). Patch by Bohuslav Kabrda. 2014-06-29 20:07:28 -04:00
Benjamin Peterson a96fea03e8 add BufferedIOBase.readinto1 (closes #20578)
Patch by Nikolaus Rath.
2014-06-22 14:17:44 -07:00
Victor Stinner 7d7e7756d9 Issue #10310: Use "unsigned int field:1" instead of "signed int field:1" in a
private structure of the _io module to fix a compiler warning (overflow when
assigning the value 1). Fix also a cast in
incrementalnewlinedecoder_setstate().  Patch written by Hallvard B Furuseth.
2014-06-17 23:31:25 +02:00
Serhiy Storchaka ec87a13e5d PyErr_NormalizeException doesn't like being called with an exception set
(issues #21677, #21310).
2014-06-11 07:19:39 +03:00
Serhiy Storchaka 76d3f14e01 PyErr_NormalizeException doesn't like being called with an exception set
(issues #21677, #21310).
2014-06-11 07:18:53 +03:00
Serhiy Storchaka 3a56117a60 Issue #21310: Fixed possible resource leak in failed open(). 2014-06-09 13:35:43 +03:00
Serhiy Storchaka f10063e3c3 Issue #21310: Fixed possible resource leak in failed open(). 2014-06-09 13:32:34 +03:00
Serhiy Storchaka 85e4235c0e Issue #21677: Fixed chaining nonnormalized exceptions in io close() methods. 2014-06-09 09:15:42 +03:00
Serhiy Storchaka 8a8f7f9830 Issue #21677: Fixed chaining nonnormalized exceptions in io close() methods. 2014-06-09 09:13:04 +03:00
Benjamin Peterson 4fb01ffe66 backout 0fb7789b5eeb for test breakage (#20578) 2014-06-07 23:18:12 -07:00
Benjamin Peterson 10e847bbc7 add BufferedIOBase.readinto1 (closes #20578)
Patch by Nikolaus Rath.
2014-06-07 20:06:48 -07:00
Antoine Pitrou 38ca5a7b6d Issue #21396: Fix TextIOWrapper(..., write_through=True) to not force a flush() on the underlying binary stream.
Patch by akira.
2014-05-09 00:31:32 +02:00
Antoine Pitrou c644e7c39f Issue #21396: Fix TextIOWrapper(..., write_through=True) to not force a flush() on the underlying binary stream.
Patch by akira.
2014-05-09 00:24:50 +02:00
Antoine Pitrou b8503896ad Issue #21057: TextIOWrapper now allows the underlying binary stream's read() or read1() method to return an arbitrary bytes-like object (such as a memoryview).
Patch by Nikolaus Rath.
2014-04-29 10:14:02 +02:00
Andrew Kuchling 764662020b #15840: make docs consistent by saying operations on closed files raise ValueError.
Patch by Caelyn McAulay.

Neither Caelyn nor I could find any cases in 2.7 or 3.4/5 where an
operation on a closed stream raised IOError; generally the C
implementations have a macro to check for the stream being closed, and
these macros all raised ValueError.  If we find any, a new bug should
be opened.
2014-04-15 21:11:36 -04:00
Georg Brandl 2fc8f773e1 Issue #20404: reject non-text encodings early in TextIOWrapper. 2014-03-02 09:18:31 +01:00
Serhiy Storchaka 5bdfc51950 Issue #17671: Fixed a crash when use non-initialized io.BufferedRWPair.
Based on patch by Stephen Tu.
2014-02-12 10:55:07 +02:00
Serhiy Storchaka 61e2493b83 Issue #17671: Fixed a crash when use non-initialized io.BufferedRWPair.
Based on patch by Stephen Tu.
2014-02-12 10:52:35 +02:00
Nick Coghlan a9b15241c6 Close #20404: blacklist non-text encodings in io.TextIOWrapper
- io.TextIOWrapper (and hence the open() builtin) now use the
  internal codec marking system added for issue #19619
- also tweaked the C code to only look up the encoding once,
  rather than multiple times
- the existing output type checks remain in place to deal with
  unmarked third party codecs.
2014-02-04 22:11:18 +10:00
Antoine Pitrou 712cb734bd Issue #20037: Avoid crashes when doing text I/O late at interpreter shutdown. 2013-12-21 15:51:54 +01:00
Serhiy Storchaka 6787a3806e Issue #15204: Deprecated the 'U' mode in file-like objects. 2013-11-23 22:12:06 +02:00
Victor Stinner 3f36a5736b Issue #19515: Remove identifiers duplicated in the same file.
Patch written by Andrei Dorian Duma.
2013-11-12 21:39:02 +01:00
Victor Stinner aa5bbfaa77 Issue #19437: Fix _io._IOBase.close(), handle _PyObject_SetAttrId() failure 2013-11-08 00:29:41 +01:00
Victor Stinner d9d0419a9f Issue #19512: fileio_init() reuses PyId_name identifier instead of "name"
literal string
2013-11-06 23:50:10 +01:00
Victor Stinner cc024d1820 Issue #18408: Fix iobase_readline(), handle PyByteArray_Resize() failure 2013-10-29 02:23:46 +01:00
Antoine Pitrou 4d397008cd Issue #19356: Avoid using a C variabled named "_self", it's a reserved word in some C compilers. 2013-10-23 19:21:55 +02:00
Antoine Pitrou 09fcb72048 Issue #19356: Avoid using a C variabled named "_self", it's a reserved word in some C compilers. 2013-10-23 19:20:21 +02:00
Antoine Pitrou fd4722cacf Issue #9548: Add a minimal "_bootlocale" module that is imported by the _io module instead of the full locale module. 2013-10-12 00:13:50 +02:00
Antoine Pitrou e619427f7e Issue #18876: The FileIO.mode attribute now better reflects the actual mode under which the file was opened.
Patch by Erik Bray.
2013-09-04 20:52:14 +02:00
Antoine Pitrou e93b63b74b Issue #18876: The FileIO.mode attribute now better reflects the actual mode under which the file was opened.
Patch by Erik Bray.
2013-09-04 20:46:33 +02:00
Victor Stinner daf455554b Issue #18571: Implementation of the PEP 446: file descriptors and file handles
are now created non-inheritable; add functions os.get/set_inheritable(),
os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
2013-08-28 00:53:59 +02:00
Serhiy Storchaka edd0de58a8 Issue #13461: Fix a crash in the TextIOWrapper.tell method on 64-bit platforms.
Patch by Yogesh Chaudhari.
2013-08-20 20:07:50 +03:00
Serhiy Storchaka ec67d187ee Issue #13461: Fix a crash in the TextIOWrapper.tell method on 64-bit platforms.
Patch by Yogesh Chaudhari.
2013-08-20 20:04:47 +03:00
Antoine Pitrou 932ff83682 Issue #18608: Avoid keeping a strong reference to the locale module inside the _io module. 2013-08-01 21:04:50 +02:00
Christian Heimes 72f455e96c Fix use of uninitialized scalar variable, see 3f994367a979
CID 1058763
2013-07-31 01:33:50 +02:00
Antoine Pitrou 796564c27b Issue #18112: PEP 442 implementation (safe object finalization). 2013-07-30 19:59:21 +02:00
Victor Stinner ace47d7efd Issue #18408: PyEval_EvalFrameEx() and PyEval_CallObjectWithKeywords() now fail
with an assertion error if they are called with an exception set
(PyErr_Occurred()).

If these functions are called with an exception set, the exception may be
cleared and so the caller looses its exception.

Add also assertions to PyEval_CallObjectWithKeywords() and call_function() to
check if the function succeed with no exception set, or the function failed
with an exception set.
2013-07-18 01:41:08 +02:00
Victor Stinner 85c761d3d3 Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure
bytes is NULL on _PyBytes_Resize() failure
2013-07-16 21:36:02 +02:00
Richard Oudkerk 9ba6962a04 Issue #18344: Fix potential ref-leaks in _bufferedreader_read_all(). 2013-07-15 16:10:28 +01:00
Richard Oudkerk 9ad51ec81b Issue #18344: Fix potential ref-leaks in _bufferedreader_read_all(). 2013-07-15 16:05:22 +01:00
Victor Stinner 14b9b11098 If MS_WIN64 is defined, MS_WINDOWS is also defined: #ifdef can be simplified. 2013-06-25 00:37:25 +02:00
Victor Stinner 9a282975ef Issue #9566: _io: Use Py_SAFE_DOWNCAST for fix a compiler warning on Windows x64 2013-06-24 23:01:33 +02:00
Christian Heimes 9975877f46 Check for correct macro, code uses S_ISDIR(). 2013-06-23 23:52:40 +02:00
Christian Heimes 91e8b8180d Check for correct macro, code uses S_ISDIR(). 2013-06-23 23:51:44 +02:00
Andrew Kuchling f567727abc Merge with 3.3 2013-06-16 13:02:55 -04:00
Andrew Kuchling c7b6c50f29 Describe 'surrogateescape' in the documentation.
Also, improve some docstring descriptions of the 'errors' parameter.

Closes #14015.
2013-06-16 12:58:48 -04:00
Serhiy Storchaka 37a79a12d1 Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw
stream's read() returns more bytes than requested.
2013-05-28 16:24:45 +03:00
Serhiy Storchaka 281945f427 Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw
stream's read() returns more bytes than requested.
2013-05-28 16:27:08 +03:00
Victor Stinner 3e269397e3 FileIO.readall(): remove trailing space from an exception message 2013-05-18 00:38:43 +02:00
Richard Oudkerk af7260e81a Issue #15758: Fix FileIO.readall() so it no longer has O(n**2) complexity. 2013-05-17 23:34:42 +01:00
Terry Jan Reedy 16c41d8520 Merge 3.3, issue #17047: remove doubled words found in 2.7 to 3.4
Modules/*, as reported by Serhiy Storchaka and Matthew Barnett.
2013-03-11 17:46:07 -04:00
Terry Jan Reedy 09b7503bc9 Merge 3.2, Issue #17047: remove doubled words found in 2.7 to 3.4
Modules/*, as reported by Serhiy Storchaka and Matthew Barnett.
2013-03-11 17:45:12 -04:00
Terry Jan Reedy 0158af38b7 Issue #17047: remove doubled words found in 2.7 to 3.4 Modules/*,
as reported by Serhiy Storchaka and Matthew Barnett.
2013-03-11 17:42:46 -04:00
R David Murray d7c59e101f Merge #17275: Fix class name in init errors in C bufferedio classes.
This fixes an apparent copy-and-paste error.

Patch by Manuel Jacob.
2013-02-23 22:21:48 -05:00
R David Murray 9f10f56d13 Merge #17275: Fix class name in init errors in C bufferedio classes.
This fixes an apparent copy-and-paste error.

Patch by Manuel Jacob.
2013-02-23 22:07:55 -05:00
R David Murray 67bfe80758 #17275: Fix class name in init errors in C bufferedio classes.
This fixes an apparent copy-and-paste error.

Patch by Manuel Jacob.
2013-02-23 21:51:05 -05:00
Serhiy Storchaka cce1b8eda8 Issue #17106: Fix a segmentation fault in io.TextIOWrapper when an underlying
stream or a decoder produces data of an unexpected type (i.e. when
io.TextIOWrapper initialized with text stream or use bytes-to-bytes codec).
2013-02-03 17:09:17 +02:00
Serhiy Storchaka d03ce4ae3d Issue #17106: Fix a segmentation fault in io.TextIOWrapper when an underlying
stream or a decoder produces data of an unexpected type (i.e. when
io.TextIOWrapper initialized with text stream or use bytes-to-bytes codec).
2013-02-03 17:07:32 +02:00
Serhiy Storchaka 94dc6736bd Issue #17106: Fix a segmentation fault in io.TextIOWrapper when an underlying
stream or a decoder produces data of an unexpected type (i.e. when
io.TextIOWrapper initialized with text stream or use bytes-to-bytes codec).
2013-02-03 17:03:31 +02:00
Gregory P. Smith 84d0bf94b0 Additional fix for issue #12268: The io module file object write methods no
longer abort early when a write system call is interrupted (EINTR).
2013-02-01 13:10:33 -08:00
Gregory P. Smith 99ec7f6f3e Additional fix for issue #12268: The io module file object write methods no
longer abort early when a write system call is interrupted (EINTR).
2013-02-01 13:08:23 -08:00
Gregory P. Smith b9817b01ed Additional fix for Issue #12268: The io module file object writelines() methods no longer abort early when one of its write system calls is interrupted (EINTR). 2013-02-01 13:03:39 -08:00
Serhiy Storchaka 441d30fac7 Issue #15989: Fix several occurrences of integer overflow
when result of PyLong_AsLong() narrowed to int without checks.

This is a backport of changesets 13e2e44db99d and 525407d89277.
2013-01-19 12:26:26 +02:00
Serhiy Storchaka 9101e23ff6 Issue #15989: Fix several occurrences of integer overflow
when result of PyLong_AsLong() narrowed to int without checks.

This is a backport of changesets 13e2e44db99d and 525407d89277.
2013-01-19 12:41:45 +02:00
Serhiy Storchaka 7898043868 Issue #15989: Fix several occurrences of integer overflow
when result of PyLong_AsLong() narrowed to int without checks.
2013-01-15 01:12:17 +02:00
Victor Stinner fd53a5a011 (Merge 3.3) Issue #16367: Fix FileIO.readall() on Windows for files larger than 2 GB. 2013-01-03 03:38:38 +01:00
Victor Stinner 6f84659e5e (Merge 3.2) Issue #16367: Fix FileIO.readall() on Windows for files larger than 2 GB. 2013-01-03 03:37:47 +01:00
Victor Stinner c44057dfbd Issue #16367: Fix FileIO.readall() on Windows for files larger than 2 GB 2013-01-03 03:33:21 +01:00
Benjamin Peterson 4c05969fc4 merge 3.3 (#16597) 2012-12-20 11:55:16 -06:00
Benjamin Peterson 68623614f0 call close on the underlying stream even if flush raises (closes #16597)
Patch by Serhiy Storchaka.
2012-12-20 11:53:11 -06:00
Andrew Svetlov a191959849 Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:27:16 +02:00
Andrew Svetlov 5b89840d9c Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:26:36 +02:00
Andrew Svetlov 737fb89dd1 Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:14:22 +02:00
Victor Stinner 292c835548 Issue #15478: Raising an OSError doesn't decode or encode the filename anymore
Pass the original filename argument to OSError constructor, instead of trying
to encode it to or decode it from the filesystem encoding. This change avoids
an additionnal UnicodeDecodeError on Windows if the filename cannot be decoded
from the filesystem encoding (ANSI code page).
2012-10-30 02:17:38 +01:00
Christian Heimes 743e0cd6b5 Issue #16166: Add PY_LITTLE_ENDIAN and PY_BIG_ENDIAN macros and unified
endianess detection and handling.
2012-10-17 23:52:17 +02:00
Jesus Cea dc469454ec Closes #15488: Closed files keep their buffer alive 2012-10-04 12:37:56 +02:00
Ezio Melotti 6d10b4d591 #15796: merge with 3.2. 2012-09-18 07:21:18 +03:00
Ezio Melotti 16d2b47837 #15796: Fix \n in readline docstring. Patch by Serhiy Storchaka. 2012-09-18 07:20:18 +03:00
Christian Heimes f47d79fec1 Fixed reference leak in error branch of _bufferedreader_read_all(). The variable data can contain a bytes object but it wasn't cleaned up when PyList_New() failed. CID 715364 2012-09-10 17:46:09 +02:00
Christian Heimes 89ff3c7f20 Fixed out-of-bounce write to rawmode buffer. The fixed size buffer wasn't enlarged for the new 'x' flag. The buffer may contain the 5 flags xrwa+ and the \0 byte 2012-09-10 03:50:48 +02:00
Stefan Krah ea94ba45fa Merge 3.2. 2012-09-08 11:19:27 +02:00
Stefan Krah 96efdd422c Issue #15868: Fix refleak in bytesio.c (Coverity #715365). 2012-09-08 11:12:33 +02:00
Antoine Pitrou 11946fbe80 Issue #15841: The readable(), writable() and seekable() methods of BytesIO
and StringIO objects now raise ValueError when the object has been closed.
Patch by Alessandro Moura.
2012-09-05 20:13:48 +02:00
Antoine Pitrou 1d857453b7 Issue #15841: The readable(), writable() and seekable() methods of BytesIO
and StringIO objects now raise ValueError when the object has been closed.
Patch by Alessandro Moura.
2012-09-05 20:11:49 +02:00
Antoine Pitrou 721738fbee Issue #15604: Update uses of PyObject_IsTrue() to check for and handle errors correctly.
Patch by Serhiy Storchaka.
2012-08-15 23:20:39 +02:00
Antoine Pitrou 6f430e4963 Issue #15604: Update uses of PyObject_IsTrue() to check for and handle errors correctly.
Patch by Serhiy Storchaka.
2012-08-15 23:18:25 +02:00
Victor Stinner d1f9352bd4 (Merge 3.2) open() / TextIOWrapper doc: make it explicit than newline='\n'
doesn't translate newlines on output.
2012-08-04 01:22:07 +02:00
Victor Stinner 401e17d0f0 open() / TextIOWrapper doc: make it explicit than newline='\n' doesn't
translate newlines on output.
2012-08-04 01:18:56 +02:00
Antoine Pitrou 3c25dfbf13 Make TextIOWrapper's documentation clearer by copying the newline argument's description from open(). 2012-08-04 00:56:19 +02:00
Antoine Pitrou 0c1c0d42dc Make TextIOWrapper's documentation clearer by copying the newline argument's description from open(). 2012-08-04 00:55:38 +02:00
Antoine Pitrou 57911f6c1a Issue #15489: Add a __sizeof__ implementation for BytesIO objects.
Patch by Serhiy Storchaka.
2012-07-30 00:01:44 +02:00
Antoine Pitrou 8f328d0c1d Issue #15489: Add a __sizeof__ implementation for BytesIO objects.
Patch by Serhiy Storchaka.
2012-07-30 00:01:06 +02:00
Antoine Pitrou a264384fe6 Issue #15487: Add a __sizeof__ implementation for buffered I/O objects.
Patch by Serhiy Storchaka.
2012-07-29 19:04:57 +02:00
Antoine Pitrou 10f0c50a0b Issue #15487: Add a __sizeof__ implementation for buffered I/O objects.
Patch by Serhiy Storchaka.
2012-07-29 19:02:46 +02:00
Florent Xicluna 109d57358e Issue #13248: io: Remove obsolete argument "max_buffer_size" of BufferedWriter and BufferedRWPair. 2012-07-07 17:03:22 +02:00
Antoine Pitrou 7d7f40c613 Issue #15247: FileIO now raises an error when given a file descriptor pointing to a directory. 2012-07-06 18:52:58 +02:00
Antoine Pitrou 9235b254dc Issue #15247: FileIO now raises an error when given a file descriptor pointing to a directory. 2012-07-06 18:48:24 +02:00
Gregory P. Smith 990a5feba7 Fixes issue #12268: File readline, readlines and read() or readall() methods
no longer lose data when an underlying read system call is interrupted.
IOError is no longer raised due to a read system call returning EINTR
from within these methods.
2012-06-24 00:23:47 -07:00
Gregory P. Smith 5135992164 Fixes issue #12268: File readline, readlines and read() or readall() methods
no longer lose data when an underlying read system call is interrupted.
IOError is no longer raised due to a read system call returning EINTR
from within these methods.
2012-06-23 23:55:39 -07:00
Jesus Cea 9436361e4c Closes #10142: Support for SEEK_HOLE/SEEK_DATA 2012-06-22 18:32:07 +02:00
Hynek Schlawack 7f59fd7c7c Simplify code in fileio_init
If an identical code line is in both at the end of if and else, it can as well
stand after the block. :) The code is from 464cf523485e, I didn't see it before
checking the commits in the web interface of course.
2012-06-22 09:32:22 +02:00
Hynek Schlawack 69168354c2 #10053: Don't close FDs when FileIO.__init__ fails
Loosely based on the work by Hirokazu Yamamoto.
2012-06-21 20:58:31 +02:00
Hynek Schlawack 9ed8b4e4ca #10053: Don't close FDs when FileIO.__init__ fails
Loosely based on the work by Hirokazu Yamamoto.
2012-06-21 20:20:25 +02:00
Victor Stinner f86a5e8a93 Close #11022: TextIOWrapper doesn't call locale.setlocale() anymore
open() and io.TextIOWrapper are now calling locale.getpreferredencoding(False)
instead of locale.getpreferredencoding() in text mode if the encoding is not
specified. Don't change temporary the locale encoding using locale.setlocale(),
use the current locale encoding instead of the user preferred encoding.

Explain also in open() documentation that locale.getpreferredencoding(False) is
called if the encoding is not specified.
2012-06-05 13:43:22 +02:00
Hynek Schlawack 9866d96e48 #4841: Fix FileIO constructor to honor closefd when called repeatedly
Patch by Victor Stinner.
2012-05-25 10:27:43 +02:00
Hynek Schlawack 2cc7156515 #4841: Fix FileIO constructor to honor closefd when called repeatedly
Patch by Victor Stinner.
2012-05-25 10:05:53 +02:00
Jesus Cea 990eff0776 Backing out 86dc014cdd74. Not ready yet 2012-04-26 17:05:31 +02:00
Jesus Cea 2b47f0a23f Close #10142: Support for SEEK_HOLE/SEEK_DATA 2012-04-26 16:39:35 +02:00
Antoine Pitrou a3f4457b17 Speed up reading of small files. This avoids multiple C read() calls on pyc files. 2012-04-17 13:50:58 +02:00
Ross Lagerwall 0f9eec19ee Don't Py_DECREF NULL variable in io.IncrementalNewlineDecoder.
Found with Clang's Static Analyzer.
2012-04-07 07:09:57 +02:00
Antoine Pitrou 6211b88161 Issue #14437: Fix building the _io module under Cygwin. 2012-03-31 23:50:31 +02:00
Antoine Pitrou d0acb411ef Issue #14387: Do not include accu.h from Python.h. 2012-03-22 14:42:18 +01:00
Brett Cannon efb00c0cc1 Issue #14153 Create _Py_device_encoding() to prevent _io from having to import
the os module.
2012-02-29 18:31:31 -05:00
Éric Araujo 408026c7e8 Merge 3.2 2012-02-26 04:07:37 +01:00
Éric Araujo fab976624d Fix typo in “seperat{or,ion}” 2012-02-26 02:14:08 +01:00
Benjamin Peterson 23d7f12ffb use new generic __dict__ descriptor implementations 2012-02-19 20:02:57 -05:00
Antoine Pitrou 7ab4af0427 Issue #13848: open() and the FileIO constructor now check for NUL characters in the file name.
Patch by Hynek Schlawack.
2012-01-29 18:43:36 +01:00
Antoine Pitrou 1334884ff2 Issue #13848: open() and the FileIO constructor now check for NUL characters in the file name.
Patch by Hynek Schlawack.
2012-01-29 18:36:34 +01:00
Antoine Pitrou bb5b92d324 Merge refleak fixes from 3.2 2012-01-18 16:19:19 +01:00
Antoine Pitrou fc1b6f0078 Fix the _io module leaking references when a sub-interpreter is created. 2012-01-18 16:13:56 +01:00
Charles-François Natali d612de10e5 Issue #12760: Refer to the new 'x' open mode as "exclusive creation" mode. 2012-01-14 11:51:00 +01:00
Charles-François Natali dc3044c704 Issue #12760: Add a create mode to open(). Patch by David Townshend. 2012-01-09 22:40:02 +01:00
Antoine Pitrou ab0e9f7089 Issue #10350: Read and save errno before calling a function which might overwrite it.
Original patch by Hallvard B Furuseth.
2011-12-16 12:29:37 +01:00
Antoine Pitrou c345ce1a69 Issue #10350: Read and save errno before calling a function which might overwrite it.
Original patch by Hallvard B Furuseth.
2011-12-16 12:28:32 +01:00
Victor Stinner f8facacf30 Fix compiler warnings 2011-11-22 02:30:47 +01:00
Antoine Pitrou 7fe601c5bf Issue #13322: Fix BufferedWriter.write() to ensure that BlockingIOError is
raised when the wrapped raw file is non-blocking and the write would block.
Previous code assumed that the raw write() would raise BlockingIOError, but
RawIOBase.write() is defined to returned None when the call would block.
Patch by sbt.
2011-11-21 20:22:01 +01:00
Antoine Pitrou 58fcf9f801 Issue #13322: Fix BufferedWriter.write() to ensure that BlockingIOError is
raised when the wrapped raw file is non-blocking and the write would block.
Previous code assumed that the raw write() would raise BlockingIOError, but
RawIOBase.write() is defined to returned None when the call would block.
Patch by sbt.
2011-11-21 20:16:44 +01:00
Victor Stinner 9e30aa52fd Fix misuse of PyUnicode_GET_SIZE() => PyUnicode_GET_LENGTH()
And PyUnicode_GetSize() => PyUnicode_GetLength()
2011-11-21 02:49:52 +01:00
Antoine Pitrou e532456965 Issue #13393: In TextIOWrapper.read(n), try to read `n` characters as
once rather than limit ourselves to the default chunk size.
2011-11-19 00:39:01 +01:00
Antoine Pitrou 56a220ae4e Issue #13393: BufferedReader.read1() now asks the full requested size to
the raw stream instead of limiting itself to the buffer size.
2011-11-16 00:56:10 +01:00
Antoine Pitrou 5c398e8395 Restore performance of special casings for utf-16 and utf-32 in TextIOWrapper 2011-11-13 04:11:37 +01:00
Antoine Pitrou c28e2e53ba In text I/O, optimize scanning for new lines with 1-byte unicode chars 2011-11-13 03:53:42 +01:00
Antoine Pitrou c1b0bfdb04 Fix memory leak in io.StringIO 2011-11-12 22:34:28 +01:00
Antoine Pitrou de20b0b50e Issue #13149: Speed up append-only StringIO objects.
This is very similar to the "lazy strings" idea.
2011-11-10 21:47:38 +01:00
Martin v. Löwis 767046aab1 Replace {Get,Set,Has}AttrString with *AttrId. 2011-10-14 15:35:36 +02:00
Ross Lagerwall 59142db6d3 Issue #12797: Added custom opener parameter to builtin open() and FileIO.open(). 2011-10-31 20:34:46 +02:00
Martin v. Löwis bd928fef42 Rename _Py_identifier to _Py_IDENTIFIER. 2011-10-14 10:20:37 +02:00
Nadeem Vawda 72d6a13413 Merge #13159: Replace FileIO's quadratic-time buffer growth algorithm with a linear-time one.
Also fix the bz2 module, which suffered from the same problem.
2011-10-13 13:38:14 +02:00
Nadeem Vawda d41a98bdd9 Issue #13159: Replace FileIO's quadratic-time buffer growth algorithm with a linear-time one.
Also fix the bz2 module, whose classes used the same algorithm.
2011-10-13 13:34:16 +02:00
Antoine Pitrou 6b4883dec0 PEP 3151 / issue #12555: reworking the OS and IO exception hierarchy. 2011-10-12 02:54:14 +02:00
Victor Stinner c5af7730e3 Fix FileIO.readall() (new_buffersize()) for large files
Truncate the buffer size to PY_SSIZE_T_MAX.
2011-10-11 23:00:31 +02:00
Victor Stinner a2a6477ba0 Fix io.FileIO.readall() on Windows 64 bits
Use Py_off_t type (64 bits) instead of off_t (32 bits).
2011-10-11 22:45:02 +02:00
Victor Stinner c4f281eba3 Fix misuse of PyUnicode_GET_SIZE, use PyUnicode_GET_LENGTH instead 2011-10-11 22:11:42 +02:00
Martin v. Löwis afe55bba33 Add API for static strings, primarily good for identifiers.
Thanks to Konrad Schöbel and Jasper Schulz for helping with the mass-editing.
2011-10-09 10:38:36 +02:00
Martin v. Löwis c47adb04b3 Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE. 2011-10-07 20:55:35 +02:00
Charles-François Natali 42c28cdd1d Issue #13070: Fix a crash when a TextIOWrapper caught in a reference cycle
would be finalized after the reference to its underlying BufferedRWPair's
writer got cleared by the GC.
2011-10-05 19:53:43 +02:00
Antoine Pitrou 1e44fecc52 Issue #13087: BufferedReader.seek() now always raises UnsupportedOperation
if the underlying raw stream is unseekable, even if the seek could be
satisfied using the internal buffer.  Patch by John O'Connor.
2011-10-04 12:26:20 +02:00
Charles-François Natali b619bb27ed Issue #13070: Fix a crash when a TextIOWrapper caught in a reference cycle
would be finalized after the reference to its underlying BufferedRWPair's
writer got cleared by the GC.
2011-10-05 19:55:56 +02:00
Victor Stinner e1335c711c Fix usage og PyUnicode_READY() 2011-10-04 20:53:03 +02:00
Antoine Pitrou bf009f0bce Issue #13087: BufferedReader.seek() now always raises UnsupportedOperation
if the underlying raw stream is unseekable, even if the seek could be
satisfied using the internal buffer.  Patch by John OConnor.
2011-10-04 12:28:52 +02:00
Victor Stinner fe9a861e74 fileio_init() checks for failure on conversion to Py_UNICODE* 2011-09-29 23:19:04 +02:00
Victor Stinner f7b8cb605d _io.textio: fix character type, use Py_UCS4 instead of Py_UNICODE 2011-09-29 03:28:17 +02:00
Martin v. Löwis d63a3b8beb Implement PEP 393. 2011-09-28 07:41:54 +02:00
Benjamin Peterson 6d8f744730 merge 3.2 (#1616) 2011-09-06 07:56:47 -04:00
Benjamin Peterson f22913b8c3 cast to getter 2011-09-06 07:55:34 -04:00
Benjamin Peterson 9fd5374d4e merge 3.2 (#12878) 2011-09-03 09:32:24 -04:00
Benjamin Peterson f6f3a35447 add a __dict__ descr for IOBase (closes #12878) 2011-09-03 09:26:20 -04:00
Antoine Pitrou e8bb1a0229 Issue #12213: Fix a buffering bug with interleaved reads and writes that
could appear on BufferedRandom streams.
2011-08-20 14:52:04 +02:00
Antoine Pitrou e05565ec5a Issue #12213: Fix a buffering bug with interleaved reads and writes that
could appear on BufferedRandom streams.
2011-08-20 14:39:23 +02:00
Antoine Pitrou d42c1d09e9 Issue #12591: Allow io.TextIOWrapper to work with raw IO objects (without
a read1() method), and add a *write_through* parameter to
mandate unbuffered writes.
2011-07-23 21:50:21 +02:00
Antoine Pitrou e96ec68101 Issue #12591: Allow io.TextIOWrapper to work with raw IO objects (without
a read1() method), and add an undocumented *write_through* parameter to
mandate unbuffered writes.
2011-07-23 21:46:35 +02:00
Antoine Pitrou 90ce72dd06 Raise ValueError when attempting to set the _CHUNK_SIZE attribute of a TextIOWrapper to a huge value, not TypeError. 2011-07-13 21:08:56 +02:00
Antoine Pitrou cb4ae815b5 Raise ValueError when attempting to set the _CHUNK_SIZE attribute of a TextIOWrapper to a huge value, not TypeError. 2011-07-13 21:07:49 +02:00
Victor Stinner d9fc85db7f (merge 3.2) Issue #9611, #9015: FileIO.read() clamps the length to INT_MAX on Windows. 2011-07-05 11:34:18 +02:00
Victor Stinner c655a726db Issue #9611, #9015: FileIO.read() clamps the length to INT_MAX on Windows. 2011-07-05 11:31:49 +02:00
Victor Stinner bc93a116eb Close #12229: Remove an unused argument of _bufferedreader_peek_unlocked(),
io.BufferedReader._peek_unlocked(). Patch written by John O'Connor.
2011-06-01 00:01:24 +02:00
Victor Stinner b57f108b03 Issue #12175: BufferedReader.read(-1) now calls raw.readall() if available. 2011-05-26 00:19:38 +02:00
Victor Stinner e9d44ccb22 Issue #12175: FileIO.readall() now only reads the file position and size once. 2011-05-26 00:16:44 +02:00
Victor Stinner d2780aedce (Merge 3.2) Issue #12175: RawIOBase.readall() now returns None if read()
returns None.
2011-05-25 22:51:16 +02:00
Victor Stinner 988512cfd7 (Merge 3.1) Issue #12175: RawIOBase.readall() now returns None if read()
returns None.
2011-05-25 22:49:15 +02:00
Victor Stinner a80987f20d Issue #12175: RawIOBase.readall() now returns None if read() returns None. 2011-05-25 22:47:16 +02:00
Victor Stinner af62c7d3de (Merge 3.2) Issue #12175: FileIO.readall() now raises a ValueError instead of
an IOError if the file is closed.
2011-05-25 22:13:47 +02:00
Victor Stinner 4767114e77 (Merge 3.1) Issue #12175: FileIO.readall() now raises a ValueError instead of
an IOError if the file is closed.
2011-05-25 22:11:55 +02:00
Victor Stinner b79f28ccbd Issue #12175: FileIO.readall() now raises a ValueError instead of an IOError if
the file is closed.
2011-05-25 22:09:03 +02:00
Victor Stinner fd8211372d Fast path for IncrementalNewlineDecoder.decode() in io.TextIOWrapper.read(-1)
Copy/paste code from textiowrapper_read_chunk().
2011-05-25 22:01:33 +02:00
Antoine Pitrou b89b31a158 Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence
on a file opened in read+write mode (namely: reading, seeking a bit forward,
writing, then seeking before the previous write but still within buffered
data, and writing again).
2011-05-13 00:25:53 +02:00
Antoine Pitrou 00dd182b8e Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence
on a file opened in read+write mode (namely: reading, seeking a bit forward,
writing, then seeking before the previous write but still within buffered
data, and writing again).
2011-05-13 00:16:28 +02:00
Antoine Pitrou 7c40489180 Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence
on a file opened in read+write mode (namely: reading, seeking a bit forward,
writing, then seeking before the previous write but still within buffered
data, and writing again).
2011-05-13 00:13:33 +02:00
Antoine Pitrou 4e19e11958 Fix compile error under Windows 2011-05-12 02:07:00 +02:00
Antoine Pitrou 3486a98dcd Issue #9971: Write an optimized implementation of BufferedReader.readinto().
Patch by John O'Connor.
2011-05-12 01:57:53 +02:00
Victor Stinner b938bcd211 (merge) Issue #11395: io.FileIO().write() clamps the data length to 32,767
bytes on Windows if the file is a TTY to workaround a Windows bug. The Windows
console returns an error (12: not enough space error) on writing into stdout if
stdout mode is binary and the length is greater than 66,000 bytes (or less,
depending on heap usage).
2011-03-20 23:37:55 +01:00
Victor Stinner e0daff1c61 Issue #11395: io.FileIO().write() clamps the data length to 32,767 bytes on
Windows if the file is a TTY to workaround a Windows bug. The Windows console
returns an error (12: not enough space error) on writing into stdout if
stdout mode is binary and the length is greater than 66,000 bytes (or less,
depending on heap usage).
2011-03-20 23:36:35 +01:00
Ezio Melotti 3b3499ba69 #11565: Merge with 3.1. 2011-03-16 11:35:38 +02:00
Ezio Melotti 13925008dc #11565: Fix several typos. Patch by Piotr Kasprzyk. 2011-03-16 11:05:33 +02:00
Ezio Melotti b88ed1549e #11565: Merge with 3.2. 2011-03-16 11:38:59 +02:00
Antoine Pitrou d843c2d86f Merged revisions 88610 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r88610 | antoine.pitrou | 2011-02-25 22:24:11 +0100 (ven., 25 févr. 2011) | 4 lines

  Issue #10956: Buffered I/O classes retry reading or writing after a signal
  has arrived and the handler returned successfully.
........
2011-02-25 21:34:39 +00:00
Antoine Pitrou 707ce82cab Issue #10956: Buffered I/O classes retry reading or writing after a signal
has arrived and the handler returned successfully.
2011-02-25 21:24:11 +00:00
Antoine Pitrou 211b81dd09 Issue #11114: Fix catastrophic performance of tell() on text files (up
to 1000x faster in some cases).  It is still one to two order of magnitudes
slower than binary tell().
2011-02-25 20:27:33 +00:00
Brett Cannon b94767ff44 Issue #8914: fix various warnings from the Clang static analyzer v254. 2011-02-22 20:15:44 +00:00
Victor Stinner 7234479580 Issue #9611: remove useless and dangerous explicit conversion to size_t 2011-01-11 00:04:12 +00:00
Antoine Pitrou a4815caa7c Issue #10872: The repr() of TextIOWrapper objects now includes the mode
if available.

(at Georg's request)
2011-01-09 20:38:15 +00:00