Commit Graph

174 Commits

Author SHA1 Message Date
Neil Schemenauer db564238db Revert "bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit. (#1908)" (#3337)
This reverts commit e38d12ed34.
2017-09-04 22:13:17 -07:00
Neil Schemenauer e38d12ed34 bpo-17852: Maintain a list of BufferedWriter objects. Flush them on exit. (#1908)
* Maintain a list of BufferedWriter objects.  Flush them on exit.

In Python 3, the buffer and the underlying file object are separate
and so the order in which objects are finalized matters.  This is
unlike Python 2 where the file and buffer were a single object and
finalization was done for both at the same time.  In Python 3, if
the file is finalized and closed before the buffer then the data in
the buffer is lost.

This change adds a doubly linked list of open file buffers.  An atexit
hook ensures they are flushed before proceeding with interpreter
shutdown.  This is addition does not remove the need to properly close
files as there are other reasons why buffered data could get lost during
finalization.

Initial patch by Armin Rigo.

* Use weakref.WeakSet instead of WeakKeyDictionary.

* Simplify buffered double-linked list types.

* In _flush_all_writers(), suppress errors from flush().

* Remove NEWS entry, use blurb.
2017-09-04 20:18:38 -07:00
Serhiy Storchaka 55fe1ae970 bpo-30022: Get rid of using EnvironmentError and IOError (except test… (#1051) 2017-04-16 10:46:38 +03:00
Serhiy Storchaka 762bf40438 bpo-29852: Argument Clinic Py_ssize_t converter now supports None (#716)
if pass `accept={int, NoneType}`.
2017-03-30 09:15:31 +03:00
Serhiy Storchaka aefa7ebf0f bpo-6532: Make the thread id an unsigned integer. (#781)
* bpo-6532: Make the thread id an unsigned integer.

From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".

* Restore a check in thread_get_ident().
2017-03-23 14:48:39 +01:00
Serhiy Storchaka fff9a31a91 bpo-29865: Use PyXXX_GET_SIZE macros rather than Py_SIZE for concrete types. (#748) 2017-03-21 08:53:25 +02:00
Serhiy Storchaka a5af6e1af7 bpo-25455: Fixed crashes in repr of recursive buffered file-like objects. (#514) 2017-03-19 19:25:29 +02:00
Serhiy Storchaka 228b12edcc Issue #28999: Use Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE wherever
possible.  Patch is writen with Coccinelle.
2017-01-23 09:47:21 +02:00
Victor Stinner 61bdb0d319 Use _PyObject_CallMethodIdObjArgs() in _io
Issue #28915: Replace _PyObject_CallMethodId() with
_PyObject_CallMethodIdObjArgs() when the format string was only made of "O"
formats, PyObject* arguments.

_PyObject_CallMethodIdObjArgs() avoids the creation of a temporary tuple and
doesn't have to parse a format string.
2016-12-09 15:39:28 +01:00
Serhiy Storchaka 85b0f5beb1 Added the const qualifier to char* variables that refer to readonly internal
UTF-8 represenatation of Unicode objects.
2016-11-20 10:16:47 +02:00
Martin Panter ccb2c0e310 Issue #23214: Implement optional BufferedReader, BytesIO read1() argument 2016-10-20 23:48:14 +00:00
Martin Panter c249221dfd Issue #20699: Merge io bytes-like fixes from 3.5 2016-05-28 01:07:08 +00:00
Martin Panter 6bb91f3b6e Issue #20699: Document that “io” methods accept bytes-like objects
This matches the usage of ZipFile and BufferedWriter. This still requires
return values to be bytes() objects.

Also document and test that the write() methods should only access their
argument before they return.
2016-05-28 00:41:57 +00:00
Serhiy Storchaka ec39756960 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:50:03 +03:00
Serhiy Storchaka 48842714b9 Issue #22570: Renamed Py_SETREF to Py_XSETREF. 2016-04-06 09:45:48 +03:00
Martin Panter 047f3b7376 Issue #22854: Merge UnsupportedOperation fixes from 3.5 2016-03-31 08:25:59 +00:00
Martin Panter 754aab28ed Issue #22854: Clarify documentation about UnsupportedOperation and add tests
Also change BufferedReader.writable() and BufferedWriter.readable() to always
return False.
2016-03-31 07:21:56 +00:00
Serhiy Storchaka bdb908ea54 Issue #20440: Applied yet one patch for using Py_SETREF.
The patch is automatically generated, it replaces the code that uses Py_CLEAR.
2015-12-27 12:38:28 +02:00
Serhiy Storchaka 4a1e70fc31 Issue #20440: Applied yet one patch for using Py_SETREF.
The patch is automatically generated, it replaces the code that uses Py_CLEAR.
2015-12-27 12:36:18 +02:00
Serhiy Storchaka ef1585eb9a Issue #25923: Added more const qualifiers to signatures of static and private functions. 2015-12-25 20:01:53 +02:00
Serhiy Storchaka 5c4064e8bd Issue #25421: __sizeof__ methods of builtin types now use dynamic basic size.
This allows sys.getsize() to work correctly with their subclasses with
__slots__ defined.
2015-12-19 20:05:25 +02:00
Steve Dower 6baa0f9805 Fixes cast warning in bufferedio.c 2015-05-23 08:59:25 -07:00
Berker Peksag d10d6ae2fa Issue #23796: peak and read1 methods of BufferedReader now raise ValueError
if they called on a closed object.

Patch by John Hergenroeder.
2015-05-12 17:01:05 +03:00
Larry Hastings dbfdc380df Issue #24001: Argument Clinic converters now use accept={type}
instead of types={'type'} to specify the types the converter accepts.
2015-05-04 06:59:46 -07:00
Serhiy Storchaka f24131ff31 Issue #20175: Converted the _io module to Argument Clinic. 2015-04-16 11:19:43 +03:00
Antoine Pitrou cb46f0ecb0 Issue #23309: Avoid a deadlock at shutdown if a daemon thread is aborted
while it is holding a lock to a buffered I/O object, and the main thread
tries to use the same I/O object (typically stdout or stderr).  A fatal
error is emitted instead.
2015-04-13 19:48:19 +02:00
Antoine Pitrou 25f85d4bd5 Issue #23309: Avoid a deadlock at shutdown if a daemon thread is aborted
while it is holding a lock to a buffered I/O object, and the main thread
tries to use the same I/O object (typically stdout or stderr).  A fatal
error is emitted instead.
2015-04-13 19:41:47 +02:00
Serhiy Storchaka 8be6be427d Issue #21802: The reader in BufferedRWPair now is closed even when closing
writer failed in BufferedRWPair.close().
2015-03-24 23:23:28 +02:00
Serhiy Storchaka 7665be6087 Issue #21802: The reader in BufferedRWPair now is closed even when closing
writer failed in BufferedRWPair.close().
2015-03-24 23:21:57 +02:00
Serhiy Storchaka 009b811d67 Removed unintentional trailing spaces in non-external and non-generated C files. 2015-03-18 21:53:15 +02:00
Victor Stinner 4a7cc88472 Issue #23571: PyObject_Call(), PyCFunction_Call() and call_function() now
raise a SystemError if a function returns a result and raises an exception.
The SystemError is chained to the previous exception.

Refactor also PyObject_Call() and PyCFunction_Call() to make them more readable.

Remove some checks which became useless (duplicate checks).

Change reviewed by Serhiy Storchaka.
2015-03-06 23:35:27 +01:00
Benjamin Peterson 98beb7599e merge 3.4 (#23093) 2014-12-21 21:00:51 -06:00
Benjamin Peterson 10e76b67c9 allow more operations to work on detached streams (closes #23093)
Patch by Martin Panter.
2014-12-21 20:51:50 -06:00
Serhiy Storchaka 78184af9b5 Issue #21715: Extracted shared complicated code in the _io module to new
_PyErr_ChainExceptions() function.
2014-10-08 22:32:50 +03:00
Serhiy Storchaka e2bd2a7186 Issue #21715: Extracted shared complicated code in the _io module to new
_PyErr_ChainExceptions() function.
2014-10-08 22:31:52 +03:00
Benjamin Peterson 58ee2d31ab merge 3.4 (#22517) 2014-09-29 22:49:05 -04:00
Benjamin Peterson c44eb73473 merge 3.3 (#22517) 2014-09-29 22:48:51 -04:00
Benjamin Peterson bbd0a323ae clear BufferedRWPair weakrefs on deallocation (closes #22517) 2014-09-29 22:46:57 -04:00
Benjamin Peterson a96fea03e8 add BufferedIOBase.readinto1 (closes #20578)
Patch by Nikolaus Rath.
2014-06-22 14:17:44 -07:00
Serhiy Storchaka 76d3f14e01 PyErr_NormalizeException doesn't like being called with an exception set
(issues #21677, #21310).
2014-06-11 07:18:53 +03:00
Serhiy Storchaka 8a8f7f9830 Issue #21677: Fixed chaining nonnormalized exceptions in io close() methods. 2014-06-09 09:13:04 +03:00
Serhiy Storchaka 5bdfc51950 Issue #17671: Fixed a crash when use non-initialized io.BufferedRWPair.
Based on patch by Stephen Tu.
2014-02-12 10:55:07 +02:00
Serhiy Storchaka 61e2493b83 Issue #17671: Fixed a crash when use non-initialized io.BufferedRWPair.
Based on patch by Stephen Tu.
2014-02-12 10:52:35 +02:00
Antoine Pitrou 712cb734bd Issue #20037: Avoid crashes when doing text I/O late at interpreter shutdown. 2013-12-21 15:51:54 +01:00
Victor Stinner 3f36a5736b Issue #19515: Remove identifiers duplicated in the same file.
Patch written by Andrei Dorian Duma.
2013-11-12 21:39:02 +01:00
Antoine Pitrou 796564c27b Issue #18112: PEP 442 implementation (safe object finalization). 2013-07-30 19:59:21 +02:00
Victor Stinner ace47d7efd Issue #18408: PyEval_EvalFrameEx() and PyEval_CallObjectWithKeywords() now fail
with an assertion error if they are called with an exception set
(PyErr_Occurred()).

If these functions are called with an exception set, the exception may be
cleared and so the caller looses its exception.

Add also assertions to PyEval_CallObjectWithKeywords() and call_function() to
check if the function succeed with no exception set, or the function failed
with an exception set.
2013-07-18 01:41:08 +02:00
Richard Oudkerk 9ba6962a04 Issue #18344: Fix potential ref-leaks in _bufferedreader_read_all(). 2013-07-15 16:10:28 +01:00
Richard Oudkerk 9ad51ec81b Issue #18344: Fix potential ref-leaks in _bufferedreader_read_all(). 2013-07-15 16:05:22 +01:00
Serhiy Storchaka 37a79a12d1 Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw
stream's read() returns more bytes than requested.
2013-05-28 16:24:45 +03:00
Serhiy Storchaka 281945f427 Issue #18025: Fixed a segfault in io.BufferedIOBase.readinto() when raw
stream's read() returns more bytes than requested.
2013-05-28 16:27:08 +03:00
R David Murray d7c59e101f Merge #17275: Fix class name in init errors in C bufferedio classes.
This fixes an apparent copy-and-paste error.

Patch by Manuel Jacob.
2013-02-23 22:21:48 -05:00
R David Murray 9f10f56d13 Merge #17275: Fix class name in init errors in C bufferedio classes.
This fixes an apparent copy-and-paste error.

Patch by Manuel Jacob.
2013-02-23 22:07:55 -05:00
R David Murray 67bfe80758 #17275: Fix class name in init errors in C bufferedio classes.
This fixes an apparent copy-and-paste error.

Patch by Manuel Jacob.
2013-02-23 21:51:05 -05:00
Benjamin Peterson 4c05969fc4 merge 3.3 (#16597) 2012-12-20 11:55:16 -06:00
Benjamin Peterson 68623614f0 call close on the underlying stream even if flush raises (closes #16597)
Patch by Serhiy Storchaka.
2012-12-20 11:53:11 -06:00
Jesus Cea dc469454ec Closes #15488: Closed files keep their buffer alive 2012-10-04 12:37:56 +02:00
Christian Heimes 8f734ebe94 Fixed reference leak in error branch of _bufferedreader_read_all(). The variable data can contain a bytes object but it wasn't cleaned up when PyList_New() failed. CID 715364 2012-09-10 17:46:09 +02:00
Antoine Pitrou a264384fe6 Issue #15487: Add a __sizeof__ implementation for buffered I/O objects.
Patch by Serhiy Storchaka.
2012-07-29 19:04:57 +02:00
Antoine Pitrou 10f0c50a0b Issue #15487: Add a __sizeof__ implementation for buffered I/O objects.
Patch by Serhiy Storchaka.
2012-07-29 19:02:46 +02:00
Florent Xicluna 109d57358e Issue #13248: io: Remove obsolete argument "max_buffer_size" of BufferedWriter and BufferedRWPair. 2012-07-07 17:03:22 +02:00
Gregory P. Smith 990a5feba7 Fixes issue #12268: File readline, readlines and read() or readall() methods
no longer lose data when an underlying read system call is interrupted.
IOError is no longer raised due to a read system call returning EINTR
from within these methods.
2012-06-24 00:23:47 -07:00
Gregory P. Smith 5135992164 Fixes issue #12268: File readline, readlines and read() or readall() methods
no longer lose data when an underlying read system call is interrupted.
IOError is no longer raised due to a read system call returning EINTR
from within these methods.
2012-06-23 23:55:39 -07:00
Jesus Cea 9436361e4c Closes #10142: Support for SEEK_HOLE/SEEK_DATA 2012-06-22 18:32:07 +02:00
Jesus Cea 990eff0776 Backing out 86dc014cdd74. Not ready yet 2012-04-26 17:05:31 +02:00
Jesus Cea 2b47f0a23f Close #10142: Support for SEEK_HOLE/SEEK_DATA 2012-04-26 16:39:35 +02:00
Antoine Pitrou 7fe601c5bf Issue #13322: Fix BufferedWriter.write() to ensure that BlockingIOError is
raised when the wrapped raw file is non-blocking and the write would block.
Previous code assumed that the raw write() would raise BlockingIOError, but
RawIOBase.write() is defined to returned None when the call would block.
Patch by sbt.
2011-11-21 20:22:01 +01:00
Antoine Pitrou 58fcf9f801 Issue #13322: Fix BufferedWriter.write() to ensure that BlockingIOError is
raised when the wrapped raw file is non-blocking and the write would block.
Previous code assumed that the raw write() would raise BlockingIOError, but
RawIOBase.write() is defined to returned None when the call would block.
Patch by sbt.
2011-11-21 20:16:44 +01:00
Antoine Pitrou 56a220ae4e Issue #13393: BufferedReader.read1() now asks the full requested size to
the raw stream instead of limiting itself to the buffer size.
2011-11-16 00:56:10 +01:00
Martin v. Löwis 767046aab1 Replace {Get,Set,Has}AttrString with *AttrId. 2011-10-14 15:35:36 +02:00
Martin v. Löwis bd928fef42 Rename _Py_identifier to _Py_IDENTIFIER. 2011-10-14 10:20:37 +02:00
Antoine Pitrou 6b4883dec0 PEP 3151 / issue #12555: reworking the OS and IO exception hierarchy. 2011-10-12 02:54:14 +02:00
Martin v. Löwis afe55bba33 Add API for static strings, primarily good for identifiers.
Thanks to Konrad Schöbel and Jasper Schulz for helping with the mass-editing.
2011-10-09 10:38:36 +02:00
Charles-François Natali 42c28cdd1d Issue #13070: Fix a crash when a TextIOWrapper caught in a reference cycle
would be finalized after the reference to its underlying BufferedRWPair's
writer got cleared by the GC.
2011-10-05 19:53:43 +02:00
Antoine Pitrou 1e44fecc52 Issue #13087: BufferedReader.seek() now always raises UnsupportedOperation
if the underlying raw stream is unseekable, even if the seek could be
satisfied using the internal buffer.  Patch by John O'Connor.
2011-10-04 12:26:20 +02:00
Charles-François Natali b619bb27ed Issue #13070: Fix a crash when a TextIOWrapper caught in a reference cycle
would be finalized after the reference to its underlying BufferedRWPair's
writer got cleared by the GC.
2011-10-05 19:55:56 +02:00
Antoine Pitrou bf009f0bce Issue #13087: BufferedReader.seek() now always raises UnsupportedOperation
if the underlying raw stream is unseekable, even if the seek could be
satisfied using the internal buffer.  Patch by John OConnor.
2011-10-04 12:28:52 +02:00
Antoine Pitrou e8bb1a0229 Issue #12213: Fix a buffering bug with interleaved reads and writes that
could appear on BufferedRandom streams.
2011-08-20 14:52:04 +02:00
Antoine Pitrou e05565ec5a Issue #12213: Fix a buffering bug with interleaved reads and writes that
could appear on BufferedRandom streams.
2011-08-20 14:39:23 +02:00
Victor Stinner bc93a116eb Close #12229: Remove an unused argument of _bufferedreader_peek_unlocked(),
io.BufferedReader._peek_unlocked(). Patch written by John O'Connor.
2011-06-01 00:01:24 +02:00
Victor Stinner b57f108b03 Issue #12175: BufferedReader.read(-1) now calls raw.readall() if available. 2011-05-26 00:19:38 +02:00
Antoine Pitrou b89b31a158 Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence
on a file opened in read+write mode (namely: reading, seeking a bit forward,
writing, then seeking before the previous write but still within buffered
data, and writing again).
2011-05-13 00:25:53 +02:00
Antoine Pitrou 00dd182b8e Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence
on a file opened in read+write mode (namely: reading, seeking a bit forward,
writing, then seeking before the previous write but still within buffered
data, and writing again).
2011-05-13 00:16:28 +02:00
Antoine Pitrou 7c40489180 Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence
on a file opened in read+write mode (namely: reading, seeking a bit forward,
writing, then seeking before the previous write but still within buffered
data, and writing again).
2011-05-13 00:13:33 +02:00
Antoine Pitrou 4e19e11958 Fix compile error under Windows 2011-05-12 02:07:00 +02:00
Antoine Pitrou 3486a98dcd Issue #9971: Write an optimized implementation of BufferedReader.readinto().
Patch by John O'Connor.
2011-05-12 01:57:53 +02:00
Antoine Pitrou d843c2d86f Merged revisions 88610 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r88610 | antoine.pitrou | 2011-02-25 22:24:11 +0100 (ven., 25 févr. 2011) | 4 lines

  Issue #10956: Buffered I/O classes retry reading or writing after a signal
  has arrived and the handler returned successfully.
........
2011-02-25 21:34:39 +00:00
Antoine Pitrou 707ce82cab Issue #10956: Buffered I/O classes retry reading or writing after a signal
has arrived and the handler returned successfully.
2011-02-25 21:24:11 +00:00
Antoine Pitrou 6cfc5124f2 Merged revisions 87427 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r87427 | antoine.pitrou | 2010-12-21 22:20:59 +0100 (mar., 21 déc. 2010) | 3 lines

  Issue #10750: The `raw` attribute of buffered IO objects is now read-only.
........
2010-12-21 21:26:09 +00:00
Antoine Pitrou 7f8f41808b Issue #10750: The `raw` attribute of buffered IO objects is now read-only. 2010-12-21 21:20:59 +00:00
Antoine Pitrou 976157f9f3 Merged revisions 86981,86984 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r86981 | antoine.pitrou | 2010-12-03 19:41:39 +0100 (ven., 03 déc. 2010) | 5 lines

  Issue #10478: Reentrant calls inside buffered IO objects (for example by
  way of a signal handler) now raise a RuntimeError instead of freezing the
  current process.
........
  r86984 | antoine.pitrou | 2010-12-03 20:14:17 +0100 (ven., 03 déc. 2010) | 3 lines

  Add an "advanced topics" section to the io doc.
........
2010-12-03 19:21:49 +00:00
Antoine Pitrou f3b68b3f98 Issue #10478: Reentrant calls inside buffered IO objects (for example by
way of a signal handler) now raise a RuntimeError instead of freezing the
current process.
2010-12-03 18:41:39 +00:00
Matthias Klose bee3316083 Modules/_io/bufferedio.c (buffered_dealloc_warn): Make it static. 2010-11-16 20:07:51 +00:00
Antoine Pitrou 243757eb79 Issue #10180: Pickling file objects is now explicitly forbidden, since
unpickling them produced nonsensical results.
2010-11-05 21:15:39 +00:00
Antoine Pitrou e033e06db0 Issue #10093: ResourceWarnings are now issued when files and sockets are
deallocated without explicit closing.  These warnings are silenced by
default, except in pydebug mode.
2010-10-29 10:38:18 +00:00
Antoine Pitrou 16b11de04e Merged revisions 84239 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r84239 | antoine.pitrou | 2010-08-21 21:09:32 +0200 (sam., 21 août 2010) | 4 lines

  Issue #9617: Signals received during a low-level write operation aren't
  ignored by the buffered IO layer anymore.
........
2010-08-21 19:17:57 +00:00
Antoine Pitrou b46b9d59ef Issue #9617: Signals received during a low-level write operation aren't
ignored by the buffered IO layer anymore.
2010-08-21 19:09:32 +00:00
Antoine Pitrou 00091cada6 Merged revisions 83944 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r83944 | antoine.pitrou | 2010-08-11 15:31:33 +0200 (mer., 11 août 2010) | 6 lines

  Issue #9550: a BufferedReader could issue an additional read when the
  original read request had been satisfied, which can block indefinitely
  when the underlying raw IO channel is e.g. a socket.  Report and original
  patch by Jason V. Miller.
........
2010-08-11 13:38:10 +00:00
Antoine Pitrou 32cfedeb1c Issue #9550: a BufferedReader could issue an additional read when the
original read request had been satisfied, which can block indefinitely
when the underlying raw IO channel is e.g. a socket.  Report and original
patch by Jason V. Miller.
2010-08-11 13:31:33 +00:00
Antoine Pitrou f6df1ee171 Merged revisions 83411 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r83411 | antoine.pitrou | 2010-08-01 18:53:42 +0200 (dim., 01 août 2010) | 4 lines

  Issue #9448: Fix a leak of OS resources (mutexes or semaphores) when
  re-initializing a buffered IO object by calling its `__init__` method.
........
2010-08-01 16:57:42 +00:00