Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity

show a common usage of detect_encoding

This commit is contained in:
Benjamin Peterson 2010-03-18 22:43:41 +00:00
parent 0088893447
commit b3a482962d
1 changed files with 11 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
Doc/library/tokenize.rst
View File

@ -98,7 +98,17 @@ function it uses to do this is available:
but disagree, a SyntaxError will be raised. Note that if the BOM is found,
``'utf-8-sig'`` will be returned as an encoding.
If no encoding is specified, then the default of ``'utf-8'`` will be returned.
If no encoding is specified, then the default of ``'utf-8'`` will be
returned.
:func:`detect_encoding` is useful for robustly reading Python source files.
A common pattern for this follows::
def read_python_source(file_name):
with open(file_name, "rb") as fp:
encoding = tokenize.detect_encoding(fp.readline)[0]
with open(file_name, "r", encoding=encoding) as fp:
return fp.read()
Example of a script re-writer that transforms float literals into Decimal