Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity

bpo-43014: Improve performance of tokenize.tokenize by 20-30%

This commit is contained in:
Anthony Sottile 2021-01-24 01:23:17 -08:00 committed by GitHub
parent bf9239bb61
commit 15bd9efd01
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Lib/tokenize.py
View File

@ -27,6 +27,7 @@ __credits__ = ('GvR, ESR, Tim Peters, Thomas Wouters, Fred Drake, '
from builtins import open as _builtin_open
from codecs import lookup, BOM_UTF8
import collections
import functools
from io import TextIOWrapper
import itertools as _itertools
import re
@ -95,6 +96,7 @@ def _all_string_prefixes():
result.add(''.join(u))
return result
@functools.lru_cache
def _compile(expr):
return re.compile(expr, re.UNICODE)

1
Misc/NEWS.d/next/Library/2021-01-24-00-37-40.bpo-43014.BVPhEr.rst Normal file
View File

@ -0,0 +1 @@
Improve performance of :mod:`tokenize` by 20-30%. Patch by Anthony Sottile.