diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst index 70750352815..d0374521cd1 100644 --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -29,7 +29,8 @@ The primary entry point is a :term:`generator`: which must be a callable object which provides the same interface as the :meth:`readline` method of built-in file objects (see section :ref:`bltin-file-objects`). Each call to the function should return one line - of input as a string. + of input as a string. Alternately, *readline* may be a callable object that + signals completion by raising :exc:`StopIteration`. The generator produces 5-tuples with these members: the token type; the token string; a 2-tuple ``(srow, scol)`` of ints specifying the row and column