bpo-39674: Revert "bpo-37330: open() no longer accept 'U' in file mode (GH-16959)" (GH-18767)

This reverts commit e471e72977.

The mode will be removed from Python 3.10.
This commit is contained in:
Victor Stinner 2020-03-04 18:50:22 +01:00 committed by GitHub
parent 00c77ae55a
commit 942f7a2dea
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
13 changed files with 99 additions and 54 deletions

View File

@ -197,9 +197,6 @@ wider range of codecs when working with binary files:
*buffering* has the same meaning as for the built-in :func:`open` function. *buffering* has the same meaning as for the built-in :func:`open` function.
It defaults to -1 which means that the default buffer size will be used. It defaults to -1 which means that the default buffer size will be used.
.. versionchanged:: 3.9
The ``'U'`` mode has been removed.
.. function:: EncodedFile(file, data_encoding, file_encoding=None, errors='strict') .. function:: EncodedFile(file, data_encoding, file_encoding=None, errors='strict')

View File

@ -148,8 +148,8 @@ available for subclassing as well:
The sequence must be accessed in strictly sequential order; random access The sequence must be accessed in strictly sequential order; random access
and :meth:`~io.TextIOBase.readline` cannot be mixed. and :meth:`~io.TextIOBase.readline` cannot be mixed.
With *mode* you can specify which file mode will be passed to :func:`open`. With *mode* you can specify which file mode will be passed to :func:`open`. It
It must be ``'r'`` or ``'rb'``. must be one of ``'r'``, ``'rU'``, ``'U'`` and ``'rb'``.
The *openhook*, when given, must be a function that takes two arguments, The *openhook*, when given, must be a function that takes two arguments,
*filename* and *mode*, and returns an accordingly opened file-like object. You *filename* and *mode*, and returns an accordingly opened file-like object. You
@ -166,14 +166,15 @@ available for subclassing as well:
.. versionchanged:: 3.2 .. versionchanged:: 3.2
Can be used as a context manager. Can be used as a context manager.
.. deprecated:: 3.4
The ``'rU'`` and ``'U'`` modes.
.. deprecated:: 3.8 .. deprecated:: 3.8
Support for :meth:`__getitem__` method is deprecated. Support for :meth:`__getitem__` method is deprecated.
.. versionchanged:: 3.8 .. versionchanged:: 3.8
The keyword parameter *mode* and *openhook* are now keyword-only. The keyword parameter *mode* and *openhook* are now keyword-only.
.. versionchanged:: 3.9
The ``'rU'`` and ``'U'`` modes have been removed.
**Optional in-place filtering:** if the keyword argument ``inplace=True`` is **Optional in-place filtering:** if the keyword argument ``inplace=True`` is

View File

@ -1090,6 +1090,12 @@ are always available. They are listed here in alphabetical order.
first decoded using a platform-dependent encoding or using the specified first decoded using a platform-dependent encoding or using the specified
*encoding* if given. *encoding* if given.
There is an additional mode character permitted, ``'U'``, which no longer
has any effect, and is considered deprecated. It previously enabled
:term:`universal newlines` in text mode, which became the default behaviour
in Python 3.0. Refer to the documentation of the
:ref:`newline <open-newline-parameter>` parameter for further details.
.. note:: .. note::
Python doesn't depend on the underlying operating system's notion of text Python doesn't depend on the underlying operating system's notion of text
@ -1246,6 +1252,10 @@ are always available. They are listed here in alphabetical order.
* The file is now non-inheritable. * The file is now non-inheritable.
.. deprecated-removed:: 3.4 3.10
The ``'U'`` mode.
.. versionchanged:: .. versionchanged::
3.5 3.5
@ -1261,10 +1271,6 @@ are always available. They are listed here in alphabetical order.
* On Windows, opening a console buffer may return a subclass of * On Windows, opening a console buffer may return a subclass of
:class:`io.RawIOBase` other than :class:`io.FileIO`. :class:`io.RawIOBase` other than :class:`io.FileIO`.
.. versionchanged:: 3.9
The ``'U'`` mode has been removed.
.. function:: ord(c) .. function:: ord(c)
Given a string representing one Unicode character, return an integer Given a string representing one Unicode character, return an integer

View File

@ -625,14 +625,6 @@ that may require changes to your code.
Changes in the Python API Changes in the Python API
------------------------- -------------------------
* :func:`open`, :func:`io.open`, :func:`codecs.open` and
:class:`fileinput.FileInput` no longer accept ``'U'`` ("universal newline")
in the file mode. This flag was deprecated since Python 3.3. In Python 3, the
"universal newline" is used by default when a file is open in text mode. The
:ref:`newline parameter <open-newline-parameter>` of :func:`open` controls
how universal newlines works.
(Contributed by Victor Stinner in :issue:`37330`.)
* :func:`__import__` and :func:`importlib.util.resolve_name` now raise * :func:`__import__` and :func:`importlib.util.resolve_name` now raise
:exc:`ImportError` where it previously raised :exc:`ValueError`. Callers :exc:`ImportError` where it previously raised :exc:`ValueError`. Callers
catching the specific exception type and supporting both Python 3.9 and catching the specific exception type and supporting both Python 3.9 and

View File

@ -71,6 +71,7 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
'b' binary mode 'b' binary mode
't' text mode (default) 't' text mode (default)
'+' open a disk file for updating (reading and writing) '+' open a disk file for updating (reading and writing)
'U' universal newline mode (deprecated)
========= =============================================================== ========= ===============================================================
The default mode is 'rt' (open for reading text). For binary random The default mode is 'rt' (open for reading text). For binary random
@ -86,6 +87,10 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
returned as strings, the bytes having been first decoded using a returned as strings, the bytes having been first decoded using a
platform-dependent encoding or using the specified encoding if given. platform-dependent encoding or using the specified encoding if given.
'U' mode is deprecated and will raise an exception in future versions
of Python. It has no effect in Python 3. Use newline to control
universal newlines mode.
buffering is an optional integer used to set the buffering policy. buffering is an optional integer used to set the buffering policy.
Pass 0 to switch buffering off (only allowed in binary mode), 1 to select Pass 0 to switch buffering off (only allowed in binary mode), 1 to select
line buffering (only usable in text mode), and an integer > 1 to indicate line buffering (only usable in text mode), and an integer > 1 to indicate
@ -171,7 +176,7 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
if errors is not None and not isinstance(errors, str): if errors is not None and not isinstance(errors, str):
raise TypeError("invalid errors: %r" % errors) raise TypeError("invalid errors: %r" % errors)
modes = set(mode) modes = set(mode)
if modes - set("axrwb+t") or len(mode) > len(modes): if modes - set("axrwb+tU") or len(mode) > len(modes):
raise ValueError("invalid mode: %r" % mode) raise ValueError("invalid mode: %r" % mode)
creating = "x" in modes creating = "x" in modes
reading = "r" in modes reading = "r" in modes
@ -180,6 +185,13 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None,
updating = "+" in modes updating = "+" in modes
text = "t" in modes text = "t" in modes
binary = "b" in modes binary = "b" in modes
if "U" in modes:
if creating or writing or appending or updating:
raise ValueError("mode U cannot be combined with 'x', 'w', 'a', or '+'")
import warnings
warnings.warn("'U' mode is deprecated",
DeprecationWarning, 2)
reading = True
if text and binary: if text and binary:
raise ValueError("can't have text and binary mode at once") raise ValueError("can't have text and binary mode at once")
if creating + reading + writing + appending > 1: if creating + reading + writing + appending > 1:

View File

@ -209,10 +209,15 @@ class FileInput:
self._isstdin = False self._isstdin = False
self._backupfilename = None self._backupfilename = None
# restrict mode argument to reading modes # restrict mode argument to reading modes
if mode not in ('r', 'rb'): if mode not in ('r', 'rU', 'U', 'rb'):
raise ValueError("FileInput opening mode must be 'r' or 'rb'") raise ValueError("FileInput opening mode must be one of "
"'r', 'rU', 'U' and 'rb'")
if 'U' in mode:
import warnings
warnings.warn("'U' mode is deprecated",
DeprecationWarning, 2)
self._mode = mode self._mode = mode
self._write_mode = mode.replace('r', 'w') self._write_mode = mode.replace('r', 'w') if 'U' not in mode else 'w'
if openhook: if openhook:
if inplace: if inplace:
raise ValueError("FileInput cannot use an opening hook in inplace mode") raise ValueError("FileInput cannot use an opening hook in inplace mode")

View File

@ -225,7 +225,7 @@ def load_module(name, file, filename, details):
""" """
suffix, mode, type_ = details suffix, mode, type_ = details
if mode and (not mode.startswith('r') or '+' in mode): if mode and (not mode.startswith(('r', 'U')) or '+' in mode):
raise ValueError('invalid file open mode {!r}'.format(mode)) raise ValueError('invalid file open mode {!r}'.format(mode))
elif file is None and type_ in {PY_SOURCE, PY_COMPILED}: elif file is None and type_ in {PY_SOURCE, PY_COMPILED}:
msg = 'file object required for import (type code {})'.format(type_) msg = 'file object required for import (type code {})'.format(type_)

View File

@ -712,23 +712,11 @@ class UTF16Test(ReadTest, unittest.TestCase):
self.addCleanup(support.unlink, support.TESTFN) self.addCleanup(support.unlink, support.TESTFN)
with open(support.TESTFN, 'wb') as fp: with open(support.TESTFN, 'wb') as fp:
fp.write(s) fp.write(s)
with codecs.open(support.TESTFN, 'r', with support.check_warnings(('', DeprecationWarning)):
encoding=self.encoding) as reader: reader = codecs.open(support.TESTFN, 'U', encoding=self.encoding)
with reader:
self.assertEqual(reader.read(), s1) self.assertEqual(reader.read(), s1)
def test_invalid_modes(self):
for mode in ('U', 'rU', 'r+U'):
with self.assertRaises(ValueError) as cm:
codecs.open(support.TESTFN, mode, encoding=self.encoding)
self.assertIn('invalid mode', str(cm.exception))
for mode in ('rt', 'wt', 'at', 'r+t'):
with self.assertRaises(ValueError) as cm:
codecs.open(support.TESTFN, mode, encoding=self.encoding)
self.assertIn("can't have text and binary mode at once",
str(cm.exception))
class UTF16LETest(ReadTest, unittest.TestCase): class UTF16LETest(ReadTest, unittest.TestCase):
encoding = "utf-16-le" encoding = "utf-16-le"
ill_formed_sequence = b"\x80\xdc" ill_formed_sequence = b"\x80\xdc"

View File

@ -226,11 +226,19 @@ class FileInputTests(BaseTests, unittest.TestCase):
self.assertEqual(fi.fileno(), -1) self.assertEqual(fi.fileno(), -1)
def test_opening_mode(self): def test_opening_mode(self):
# invalid modes try:
for mode in ('w', 'rU', 'U'): # invalid mode, should raise ValueError
with self.subTest(mode=mode): fi = FileInput(mode="w")
with self.assertRaises(ValueError): self.fail("FileInput should reject invalid mode argument")
FileInput(mode=mode) except ValueError:
pass
# try opening in universal newline mode
t1 = self.writeTmp(b"A\nB\r\nC\rD", mode="wb")
with check_warnings(('', DeprecationWarning)):
fi = FileInput(files=t1, mode="U")
with check_warnings(('', DeprecationWarning)):
lines = list(fi)
self.assertEqual(lines, ["A\n", "B\n", "C\n", "D"])
def test_stdin_binary_mode(self): def test_stdin_binary_mode(self):
with mock.patch('sys.stdin') as m_stdin: with mock.patch('sys.stdin') as m_stdin:
@ -977,6 +985,10 @@ class Test_hook_encoded(unittest.TestCase):
self.assertEqual(lines, expected_lines) self.assertEqual(lines, expected_lines)
check('r', ['A\n', 'B\n', 'C\n', 'D\u20ac']) check('r', ['A\n', 'B\n', 'C\n', 'D\u20ac'])
with self.assertWarns(DeprecationWarning):
check('rU', ['A\n', 'B\n', 'C\n', 'D\u20ac'])
with self.assertWarns(DeprecationWarning):
check('U', ['A\n', 'B\n', 'C\n', 'D\u20ac'])
with self.assertRaises(ValueError): with self.assertRaises(ValueError):
check('rb', ['A\n', 'B\r\n', 'C\r', 'D\u20ac']) check('rb', ['A\n', 'B\r\n', 'C\r', 'D\u20ac'])

View File

@ -3900,6 +3900,16 @@ class MiscIOTest(unittest.TestCase):
self.assertEqual(f.mode, "wb") self.assertEqual(f.mode, "wb")
f.close() f.close()
with support.check_warnings(('', DeprecationWarning)):
f = self.open(support.TESTFN, "U")
self.assertEqual(f.name, support.TESTFN)
self.assertEqual(f.buffer.name, support.TESTFN)
self.assertEqual(f.buffer.raw.name, support.TESTFN)
self.assertEqual(f.mode, "U")
self.assertEqual(f.buffer.mode, "rb")
self.assertEqual(f.buffer.raw.mode, "rb")
f.close()
f = self.open(support.TESTFN, "w+") f = self.open(support.TESTFN, "w+")
self.assertEqual(f.mode, "w+") self.assertEqual(f.mode, "w+")
self.assertEqual(f.buffer.mode, "rb+") # Does it really matter? self.assertEqual(f.buffer.mode, "rb+") # Does it really matter?
@ -3913,13 +3923,6 @@ class MiscIOTest(unittest.TestCase):
f.close() f.close()
g.close() g.close()
def test_removed_u_mode(self):
# "U" mode has been removed in Python 3.9
for mode in ("U", "rU", "r+U"):
with self.assertRaises(ValueError) as cm:
self.open(support.TESTFN, mode)
self.assertIn('invalid mode', str(cm.exception))
def test_open_pipe_with_append(self): def test_open_pipe_with_append(self):
# bpo-27805: Ignore ESPIPE from lseek() in open(). # bpo-27805: Ignore ESPIPE from lseek() in open().
r, w = os.pipe() r, w = os.pipe()

View File

@ -0,0 +1,3 @@
Revert "bpo-37330: open() no longer accept 'U' in file mode". The "U" mode of
open() is kept in Python 3.9 to ease transition from Python 2.7, but will be
removed in Python 3.10.

View File

@ -138,6 +138,7 @@ Character Meaning
'b' binary mode 'b' binary mode
't' text mode (default) 't' text mode (default)
'+' open a disk file for updating (reading and writing) '+' open a disk file for updating (reading and writing)
'U' universal newline mode (deprecated)
========= =============================================================== ========= ===============================================================
The default mode is 'rt' (open for reading text). For binary random The default mode is 'rt' (open for reading text). For binary random
@ -153,6 +154,10 @@ bytes objects without any decoding. In text mode (the default, or when
returned as strings, the bytes having been first decoded using a returned as strings, the bytes having been first decoded using a
platform-dependent encoding or using the specified encoding if given. platform-dependent encoding or using the specified encoding if given.
'U' mode is deprecated and will raise an exception in future versions
of Python. It has no effect in Python 3. Use newline to control
universal newlines mode.
buffering is an optional integer used to set the buffering policy. buffering is an optional integer used to set the buffering policy.
Pass 0 to switch buffering off (only allowed in binary mode), 1 to select Pass 0 to switch buffering off (only allowed in binary mode), 1 to select
line buffering (only usable in text mode), and an integer > 1 to indicate line buffering (only usable in text mode), and an integer > 1 to indicate
@ -228,12 +233,12 @@ static PyObject *
_io_open_impl(PyObject *module, PyObject *file, const char *mode, _io_open_impl(PyObject *module, PyObject *file, const char *mode,
int buffering, const char *encoding, const char *errors, int buffering, const char *encoding, const char *errors,
const char *newline, int closefd, PyObject *opener) const char *newline, int closefd, PyObject *opener)
/*[clinic end generated code: output=aefafc4ce2b46dc0 input=1543f4511d2356a5]*/ /*[clinic end generated code: output=aefafc4ce2b46dc0 input=7295902222e6b311]*/
{ {
unsigned i; unsigned i;
int creating = 0, reading = 0, writing = 0, appending = 0, updating = 0; int creating = 0, reading = 0, writing = 0, appending = 0, updating = 0;
int text = 0, binary = 0; int text = 0, binary = 0, universal = 0;
char rawmode[6], *m; char rawmode[6], *m;
int line_buffering, is_number; int line_buffering, is_number;
@ -291,6 +296,10 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode,
case 'b': case 'b':
binary = 1; binary = 1;
break; break;
case 'U':
universal = 1;
reading = 1;
break;
default: default:
goto invalid_mode; goto invalid_mode;
} }
@ -313,6 +322,18 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode,
*m = '\0'; *m = '\0';
/* Parameters validation */ /* Parameters validation */
if (universal) {
if (creating || writing || appending || updating) {
PyErr_SetString(PyExc_ValueError,
"mode U cannot be combined with 'x', 'w', 'a', or '+'");
goto error;
}
if (PyErr_WarnEx(PyExc_DeprecationWarning,
"'U' mode is deprecated", 1) < 0)
goto error;
reading = 1;
}
if (text && binary) { if (text && binary) {
PyErr_SetString(PyExc_ValueError, PyErr_SetString(PyExc_ValueError,
"can't have text and binary mode at once"); "can't have text and binary mode at once");

View File

@ -36,6 +36,7 @@ PyDoc_STRVAR(_io_open__doc__,
"\'b\' binary mode\n" "\'b\' binary mode\n"
"\'t\' text mode (default)\n" "\'t\' text mode (default)\n"
"\'+\' open a disk file for updating (reading and writing)\n" "\'+\' open a disk file for updating (reading and writing)\n"
"\'U\' universal newline mode (deprecated)\n"
"========= ===============================================================\n" "========= ===============================================================\n"
"\n" "\n"
"The default mode is \'rt\' (open for reading text). For binary random\n" "The default mode is \'rt\' (open for reading text). For binary random\n"
@ -51,6 +52,10 @@ PyDoc_STRVAR(_io_open__doc__,
"returned as strings, the bytes having been first decoded using a\n" "returned as strings, the bytes having been first decoded using a\n"
"platform-dependent encoding or using the specified encoding if given.\n" "platform-dependent encoding or using the specified encoding if given.\n"
"\n" "\n"
"\'U\' mode is deprecated and will raise an exception in future versions\n"
"of Python. It has no effect in Python 3. Use newline to control\n"
"universal newlines mode.\n"
"\n"
"buffering is an optional integer used to set the buffering policy.\n" "buffering is an optional integer used to set the buffering policy.\n"
"Pass 0 to switch buffering off (only allowed in binary mode), 1 to select\n" "Pass 0 to switch buffering off (only allowed in binary mode), 1 to select\n"
"line buffering (only usable in text mode), and an integer > 1 to indicate\n" "line buffering (only usable in text mode), and an integer > 1 to indicate\n"
@ -318,4 +323,4 @@ _io_open_code(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObjec
exit: exit:
return return_value; return return_value;
} }
/*[clinic end generated code: output=680e4b488c7da8a1 input=a9049054013a1b77]*/ /*[clinic end generated code: output=3df6bc6d91697545 input=a9049054013a1b77]*/