diff --git a/Doc/glossary.rst b/Doc/glossary.rst index de6cd89bb45..d0b1922d109 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -630,6 +630,13 @@ Glossary object has a type. An object's type is accessible as its :attr:`__class__` attribute or can be retrieved with ``type(obj)``. + universal newlines + A manner of interpreting text streams in which all of the following are + recognized as ending a line: the Unix end-of-line convention ``'\n'``, + the Windows convention ``'\r\n'``, and the old Macintosh convention + ``'\r'``. See :pep:`278` and :pep:`3116`, as well as + :func:`str.splitlines` for an additional use. + view The objects returned from :meth:`dict.viewkeys`, :meth:`dict.viewvalues`, and :meth:`dict.viewitems` are called dictionary views. They are lazy diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst index 79058fbd498..6793c44de99 100644 --- a/Doc/library/bz2.rst +++ b/Doc/library/bz2.rst @@ -42,6 +42,9 @@ Here is a summary of the features offered by the bz2 module: Handling of compressed files is offered by the :class:`BZ2File` class. +.. index:: + single: universal newlines; bz2.BZ2File class + .. class:: BZ2File(filename[, mode[, buffering[, compresslevel]]]) Open a bz2 file. Mode can be either ``'r'`` or ``'w'``, for reading (default) @@ -50,7 +53,7 @@ Handling of compressed files is offered by the :class:`BZ2File` class. unbuffered, and larger numbers specify the buffer size; the default is ``0``. If *compresslevel* is given, it must be a number between ``1`` and ``9``; the default is ``9``. Add a ``'U'`` to mode to open the file for input - with universal newline support. Any line ending in the input file will be + in :term:`universal newlines` mode. Any line ending in the input file will be seen as a ``'\n'`` in Python. Also, a file so opened gains the attribute :attr:`newlines`; the value for this attribute is one of ``None`` (no newline read yet), ``'\r'``, ``'\n'``, ``'\r\n'`` or a tuple containing all the diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index 30ac614a333..4383cd4e5a6 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -859,13 +859,17 @@ available. They are listed here in alphabetical order. binary mode, on systems that differentiate between binary and text files; on systems that don't have this distinction, adding the ``'b'`` has no effect. + .. index:: + single: universal newlines; open() built-in function + In addition to the standard :c:func:`fopen` values *mode* may be ``'U'`` or - ``'rU'``. Python is usually built with universal newline support; supplying + ``'rU'``. Python is usually built with :term:`universal newlines` support; + supplying ``'U'`` opens the file as a text file, but lines may be terminated by any of the following: the Unix end-of-line convention ``'\n'``, the Macintosh convention ``'\r'``, or the Windows convention ``'\r\n'``. All of these external representations are seen as ``'\n'`` by the Python program. If Python is built - without universal newline support a *mode* with ``'U'`` is the same as normal + without universal newlines support a *mode* with ``'U'`` is the same as normal text mode. Note that file objects so opened also have an attribute called :attr:`newlines` which has a value of ``None`` (if no newlines have yet been seen), ``'\n'``, ``'\r'``, ``'\r\n'``, or a tuple containing all the newline diff --git a/Doc/library/io.rst b/Doc/library/io.rst index e1fa15552f8..bb7c34fb486 100644 --- a/Doc/library/io.rst +++ b/Doc/library/io.rst @@ -92,7 +92,7 @@ Module Interface ``'b'`` binary mode ``'t'`` text mode (default) ``'+'`` open a disk file for updating (reading and writing) - ``'U'`` universal newline mode (for backwards compatibility; should + ``'U'`` universal newlines mode (for backwards compatibility; should not be used in new code) ========= =============================================================== @@ -141,14 +141,18 @@ Module Interface used. Any other error handling name that has been registered with :func:`codecs.register_error` is also valid. - *newline* controls how universal newlines works (it only applies to text + .. index:: + single: universal newlines; open() (in module io) + + *newline* controls how :term:`universal newlines` works (it only applies + to text mode). It can be ``None``, ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It works as follows: * On input, if *newline* is ``None``, universal newlines mode is enabled. Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, and these are translated into ``'\n'`` before being returned to the caller. If it is - ``''``, universal newline mode is enabled, but line endings are returned to + ``''``, universal newlines mode is enabled, but line endings are returned to the caller untranslated. If it has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated. @@ -754,13 +758,17 @@ Text I/O sequences) can be used. Any other error handling name that has been registered with :func:`codecs.register_error` is also valid. + .. index:: + single: universal newlines; io.TextIOWrapper class + *newline* controls how line endings are handled. It can be ``None``, ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It works as follows: - * On input, if *newline* is ``None``, universal newlines mode is enabled. + * On input, if *newline* is ``None``, :term:`universal newlines` mode is + enabled. Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, and these are translated into ``'\n'`` before being returned to the caller. If it is - ``''``, universal newline mode is enabled, but line endings are returned to + ``''``, universal newlines mode is enabled, but line endings are returned to the caller untranslated. If it has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated. @@ -817,10 +825,13 @@ Text I/O output.close() +.. index:: + single: universal newlines; io.IncrementalNewlineDecoder class + .. class:: IncrementalNewlineDecoder - A helper codec that decodes newlines for universal newlines mode. It - inherits :class:`codecs.IncrementalDecoder`. + A helper codec that decodes newlines for :term:`universal newlines` mode. + It inherits :class:`codecs.IncrementalDecoder`. Advanced topics diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index abe305250b4..4dfed58f83b 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1181,10 +1181,13 @@ string functions based on regular expressions. ``' 1 2 3 '.split(None, 1)`` returns ``['1', '2 3 ']``. +.. index:: + single: universal newlines; str.splitlines method + .. method:: str.splitlines([keepends]) Return a list of the lines in the string, breaking at line boundaries. - This method uses the universal newlines approach to splitting lines. + This method uses the :term:`universal newlines` approach to splitting lines. Line breaks are not included in the resulting list unless *keepends* is given and true. @@ -2558,16 +2561,19 @@ the particular object. form ``<...>``. This is a read-only attribute and may not be present on all file-like objects. + .. index:: + single: universal newlines; file.newlines attribute + .. attribute:: file.newlines - If Python was built with universal newlines enabled (the default) this + If Python was built with :term:`universal newlines` enabled (the default) this read-only attribute exists, and for files opened in universal newline read mode it keeps track of the types of newlines encountered while reading the file. The values it can take are ``'\r'``, ``'\n'``, ``'\r\n'``, ``None`` (unknown, no newlines read yet) or a tuple containing all the newline types seen, to indicate that multiple newline conventions were encountered. For - files not opened in universal newline read mode the value of this attribute + files not opened in universal newlines read mode the value of this attribute will be ``None``. diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 1f7315922cd..9f823235b48 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -220,9 +220,12 @@ default values. The arguments that are most commonly needed are: the stderr data from the child process should be captured into the same file handle as for stdout. + .. index:: + single: universal newlines; subprocess module + When *stdout* or *stderr* are pipes and *universal_newlines* is - :const:`True` then all line endings will be converted to ``'\n'`` as - described for the universal newlines `'U'`` mode argument to :func:`open`. + ``True`` then all line endings will be converted to ``'\n'`` as described + for the :term:`universal newlines` `'U'`` mode argument to :func:`open`. If *shell* is :const:`True`, the specified command will be executed through the shell. This can be useful if you are using Python primarily for the @@ -382,8 +385,9 @@ functions. .. _side-by-side assembly: http://en.wikipedia.org/wiki/Side-by-Side_Assembly - If *universal_newlines* is :const:`True`, the file objects stdout and stderr are - opened as text files, but lines may be terminated by any of ``'\n'``, the Unix + If *universal_newlines* is ``True``, the file objects *stdout* and *stderr* are + opened as text files in :term:`universal newlines` mode. Lines may be + terminated by any of ``'\n'``, the Unix end-of-line convention, ``'\r'``, the old Macintosh convention or ``'\r\n'``, the Windows convention. All of these external representations are seen as ``'\n'`` by the Python program. diff --git a/Doc/library/urllib.rst b/Doc/library/urllib.rst index 2d52f95275d..1cb3faf4125 100644 --- a/Doc/library/urllib.rst +++ b/Doc/library/urllib.rst @@ -34,7 +34,7 @@ High-level interface Open a network object denoted by a URL for reading. If the URL does not have a scheme identifier, or if it has :file:`file:` as its scheme identifier, this - opens a local file (without universal newlines); otherwise it opens a socket to + opens a local file (without :term:`universal newlines`); otherwise it opens a socket to a server somewhere on the network. If the connection cannot be made the :exc:`IOError` exception is raised. If all went well, a file-like object is returned. This supports the following methods: :meth:`read`, :meth:`readline`, diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst index 68e8dfac4e1..77a1eecf127 100644 --- a/Doc/library/zipfile.rst +++ b/Doc/library/zipfile.rst @@ -163,13 +163,17 @@ ZipFile Objects Return a list of archive members by name. + .. index:: + single: universal newlines; zipfile.ZipFile.open method + .. method:: ZipFile.open(name[, mode[, pwd]]) Extract a member from the archive as a file-like object (ZipExtFile). *name* is the name of the file in the archive, or a :class:`ZipInfo` object. The *mode* - parameter, if included, must be one of the following: ``'r'`` (the default), - ``'U'``, or ``'rU'``. Choosing ``'U'`` or ``'rU'`` will enable universal newline + parameter, if included, must be one of the following: ``'r'`` (the default), + ``'U'``, or ``'rU'``. Choosing ``'U'`` or ``'rU'`` will enable + :term:`universal newline ` support in the read-only object. *pwd* is the password used for encrypted files. Calling :meth:`open` on a closed ZipFile will raise a :exc:`RuntimeError`. diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index ce22b67188b..a9766a4b95d 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -1024,5 +1024,5 @@ which may be useful to pass around for use by :keyword:`exec`. .. rubric:: Footnotes .. [#] Note that the parser only accepts the Unix-style end of line convention. - If you are reading the code from a file, make sure to use universal - newline mode to convert Windows or Mac-style newlines. + If you are reading the code from a file, make sure to use + :term:`universal newlines` mode to convert Windows or Mac-style newlines. diff --git a/Doc/whatsnew/2.3.rst b/Doc/whatsnew/2.3.rst index b034eb2a1f2..fbde6e54565 100644 --- a/Doc/whatsnew/2.3.rst +++ b/Doc/whatsnew/2.3.rst @@ -366,6 +366,9 @@ Under MacOS, :func:`os.listdir` may now return Unicode filenames. .. ====================================================================== +.. index:: + single: universal newlines; What's new + PEP 278: Universal Newline Support ================================== @@ -378,7 +381,8 @@ two-character sequence of a carriage return plus a newline. Python's file objects can now support end of line conventions other than the one followed by the platform on which Python is running. Opening a file with the -mode ``'U'`` or ``'rU'`` will open a file for reading in universal newline mode. +mode ``'U'`` or ``'rU'`` will open a file for reading in +:term:`universal newlines` mode. All three line ending conventions will be translated to a ``'\n'`` in the strings returned by the various file methods such as :meth:`read` and :meth:`readline`. diff --git a/Doc/whatsnew/2.4.rst b/Doc/whatsnew/2.4.rst index 97a02938bd3..e9677ac14bb 100644 --- a/Doc/whatsnew/2.4.rst +++ b/Doc/whatsnew/2.4.rst @@ -411,6 +411,9 @@ error streams will be. You can provide a file object or a file descriptor, or you can use the constant ``subprocess.PIPE`` to create a pipe between the subprocess and the parent. +.. index:: + single: universal newlines; What's new + The constructor has a number of handy options: * *close_fds* requests that all file descriptors be closed before running the @@ -424,7 +427,7 @@ The constructor has a number of handy options: * *preexec_fn* is a function that gets called before the child is started. * *universal_newlines* opens the child's input and output using Python's - universal newline feature. + :term:`universal newlines` feature. Once you've created the :class:`Popen` instance, you can call its :meth:`wait` method to pause until the subprocess has exited, :meth:`poll` to check if it's diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index ff599c8d8a2..503302449f0 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -1338,9 +1338,12 @@ complete list of changes, or look through the SVN logs for all the details. .. XXX need to provide some more detail here + .. index:: + single: universal newlines; What's new + * The :mod:`fileinput` module was made more flexible. Unicode filenames are now supported, and a *mode* parameter that defaults to ``"r"`` was added to the - :func:`input` function to allow opening files in binary or universal-newline + :func:`input` function to allow opening files in binary or :term:`universal newlines` mode. Another new parameter, *openhook*, lets you use a function other than :func:`open` to open the input files. Once you're iterating over the set of files, the :class:`FileInput` object's new :meth:`fileno` returns the file diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index 796963a17b9..c9336cb2f4e 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -1067,9 +1067,12 @@ the :mod:`io` module: The :class:`BytesIO` class supports reading, writing, and seeking over an in-memory buffer. + .. index:: + single: universal newlines; What's new + * :class:`TextIOBase`: Provides functions for reading and writing strings (remember, strings will be Unicode in Python 3.0), - and supporting universal newlines. :class:`TextIOBase` defines + and supporting :term:`universal newlines`. :class:`TextIOBase` defines the :meth:`readline` method and supports iteration upon objects.