Some incremental decoders return multiple characters, even when fed with
only one more byte. In this case the tell() state must subtract the
number of extra characters.
"Universal newline" is now an incremental decoder wrapping the initial one,
with its own additional buffer (if '\r' is seen at the end of the input).
A decoder allows the tell() funtion to record the state of the translation.
This also simplifies the readline() process.
Now test_netrc passes on Windows, as well as many new tests in test_io.py
No detailed change log; just check out the change log for the py3k-pep3137
branch. The most obvious changes:
- str8 renamed to bytes (PyString at the C level);
- bytes renamed to buffer (PyBytes at the C level);
- PyString and PyUnicode are no longer compatible.
I.e. we now have an immutable bytes type and a mutable bytes type.
The behavior of PyString was modified quite a bit, to make it more
bytes-like. Some changes are still on the to-do list.
Add a closefd flag to open() which can be set to False to prevent closing
the file descriptor when close() is called or when the object is destroyed.
Useful to ensure that sys.std{in,out,err} keep their file descriptors open
when Python is uninitialized. (This was always a feature in 2.x, it just
wasn't implemented in 3.0 yet.)
This patch corrects a problem in test_file.py on Windows:
f.truncate() seeks to the truncation point, but does not empty the
buffers. In the test, f.tell() returns -1...
From now on, trying to write str to a binary stream
is an error (I'm still working on the reverse).
There are still (at least) two failing tests:
- test_asynchat
- test_urllib2_localnet
but I'm sure these will be fixed by someone.
- Replace all asserts by ValuleErrors or TypeErrors as appropriate.
- Add _checkReadable, _checkWritable methods; these check self.closed too.
- Add a test that everything exported by io.py exists, and is either
an exception or an IOBase instance (except for the open function).
- Default buffering to 1 if isatty() (I had to tweak this to enforce
the *default* bit -- GvR).
Completely get rid of StringIO.py and cStringIO.c.
I had to fix a few tests and modules beyond what Christian did, and
invent a few conventions. E.g. in elementtree, I chose to
write/return Unicode strings whe no encoding is given, but bytes when
an explicit encoding is given. Also mimetools was made to always
assume binary files.
If a makefile()-generated object is open and its parent socket is
closed, the parent socket should remain open until the child is
closed, too. The code to support this is moderately complex and
requires one extra slots in the socket object.
This change fixes httplib so that several urllib2net test cases pass
again.
Add SocketCloser class to socket.py, which encapsulates the
refcounting logic for sockets after makefile() has been called.
Move SocketIO class from io module to socket module. It's only use is
to implement the raw I/O methods on top of a socket to support
makefile().
Add unittests to test_socket to cover various patterns of close and
makefile.
This makes test_resource.py pass, and I think it's the right thing
to do: if you're closing a file after encountering an I/O error
there's nothing you can do about it. If you want the error, you
can call flush() yourself.
to even the most basic file object (I also added readall() which may
be a better API). Also, not all the tests requiring specific failure
modes could be saved. And there were the usual str/bytes issues.
I made sure test_io.py still passes (io.py is now most thoroughly
tested by combining test_file.py and test_io.py).
io.open() now takes all positional parameters (so we can conveniently
call it from C code).
test_tarfile.py no longer uses u"..." literals, but is otherwise still
badly broken.
This is a checkpoint; some more stuff now breaks.
recv()) now return bytes, not str or str8. The socket.py code is
redone; it now subclasses _socket.socket and instead of having its own
_fileobject for makefile(), it uses io.SocketIO. Some stuff in io.py
was moved around to make this work. (I really need to rethink my
policy regarding readline() and read(-1) on raw files; and readline()
on buffered files ought to use peeking(). Later.)
Changes to io.py, necessary to make this work:
- Redid io.StringIO as a TextIOWrapper on top of a BytesIO instance.
- Got rid of _MemoryIOMixin, folding it into BytesIO instead.
- The read() functions that take -1 to mean "eveything" now also take None.
- Added readline() support to BufferedIOBase. :-(
encoding) which makes the buffer mutable. Strings are encoded on the way in
and decoded on the way out.
Use io.StringIO in test_codecs.py.
Fix the base64_codec test in test_codecs.py.
Don't read more than one line when reading text from a tty device.
Add peek() and read1() methods.
Return str instead of unicode when return ASCII characters in text mode.
Fix the truncate() semantics -- it should not affect the current position.
Switch wave.py/chunk.py to struct.unpack_from() to support bytes.
Don't use writelines() on binary files (test_fileinput.py).
write() returns the number of bytes/characters written/buffered.
FileIO.close() calls self.flush().
Implement readinto() for buffered readers.
Tests th check all these.
Test proper behavior of __enter__/__exit__.
Add closed attribute.
Support int argument to open() -- wrapping a file descriptor.
For b/w compat, support readline(n).
Support readlines() and readlines(n).
Flush on __del__.
Added some XXX comments.
I cleaned up whitespace but otherwise didn't change it.
This will need work to reflect the tentative decision to drop nonblocking I/O
support from the buffering layers.
Also implemented Neal's suggestion (add fileno() to SocketIO)
and some unrelated changes, e.g. remove Google copyright
and make BytesIO a subclass of BufferedIOBase.
Guido van Rossum
Georg Brandl
Amaury Forgeot d'Arc
Christian Heimes
Brett Cannon
Neal Norwitz
Martin v. Löwis
Jeremy Hylton
Walter Dörwald