cpython/Lib/lib2to3/fixes/fix_unicode.py

26 lines
691 B
Python
Raw Normal View History

"""Fixer that changes unicode to str, unichr to chr, and u"..." into "...".
"""
import re
from ..pgen2 import token
Merged revisions 64286 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r64286 | benjamin.peterson | 2008-06-14 21:31:05 -0500 (Sat, 14 Jun 2008) | 49 lines Merged revisions 63661,63666,63695,63711,63729,63769,63790,63880,63886 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r63661 | georg.brandl | 2008-05-26 05:26:20 -0500 (Mon, 26 May 2008) | 2 lines Add import fixes for dbm package. ........ r63666 | georg.brandl | 2008-05-26 05:49:09 -0500 (Mon, 26 May 2008) | 2 lines Add xmlrpc package fixes. ........ r63695 | georg.brandl | 2008-05-26 10:14:33 -0500 (Mon, 26 May 2008) | 2 lines Add fixer entries for http package. ........ r63711 | benjamin.peterson | 2008-05-26 13:43:51 -0500 (Mon, 26 May 2008) | 2 lines add import mapping for test.test_support -> test.support ........ r63729 | benjamin.peterson | 2008-05-26 16:31:03 -0500 (Mon, 26 May 2008) | 2 lines mapping for commands module -> subprocess ........ r63769 | brett.cannon | 2008-05-29 00:13:13 -0500 (Thu, 29 May 2008) | 1 line Fixer for UserString.UserString over to the collections module. ........ r63790 | brett.cannon | 2008-05-29 14:13:51 -0500 (Thu, 29 May 2008) | 4 lines Add a fixer for UserList. Closes issue #2878. Thanks to Quentin Gallet-Gilles for the patch. ........ r63880 | collin.winter | 2008-06-01 18:09:38 -0500 (Sun, 01 Jun 2008) | 6 lines Move lib2to3/fixes/{basefix,util}.py down to lib2to3/. This is step 1 of turning lib2to3/ into a general-purpose refactoring library, reusable by other projects. ........ r63886 | collin.winter | 2008-06-01 22:15:01 -0500 (Sun, 01 Jun 2008) | 5 lines Allow refactoring tools to specify a directory for fixer modules. This is step 2 of turning lib2to3/ into a general-purpose refactoring library, reusable by other projects. Step 1: r63880. ........ ................
2008-06-14 23:57:40 -03:00
from .. import fixer_base
Merged revisions 73370 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r73370 | benjamin.peterson | 2009-06-11 17:06:46 -0500 (Thu, 11 Jun 2009) | 105 lines Merged revisions 72523,72950-72951,72994,73003,73033,73036-73040,73091-73093,73096,73179-73181,73192,73231,73244,73255-73256,73365 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r72523 | benjamin.peterson | 2009-05-09 14:42:26 -0500 (Sat, 09 May 2009) | 1 line remove parenthesis ........ r72950 | benjamin.peterson | 2009-05-26 18:19:45 -0500 (Tue, 26 May 2009) | 1 line remove unused imports ........ r72951 | benjamin.peterson | 2009-05-26 18:27:00 -0500 (Tue, 26 May 2009) | 1 line this is no longer executable ........ r72994 | benjamin.peterson | 2009-05-28 15:32:54 -0500 (Thu, 28 May 2009) | 1 line fix test_all_fixers on Windows #6134 ........ r73003 | benjamin.peterson | 2009-05-28 21:57:28 -0500 (Thu, 28 May 2009) | 4 lines make 2to3 test utilities easier to use with other applications (3to2) Patch by Joe Amenta ........ r73033 | benjamin.peterson | 2009-05-29 16:58:32 -0500 (Fri, 29 May 2009) | 1 line update grammar for multi with statement ........ r73036 | benjamin.peterson | 2009-05-29 17:33:20 -0500 (Fri, 29 May 2009) | 1 line simplify fix_unicode ........ r73037 | benjamin.peterson | 2009-05-29 17:53:03 -0500 (Fri, 29 May 2009) | 1 line add custom error for pattern syntax errors ........ r73038 | benjamin.peterson | 2009-05-29 17:55:00 -0500 (Fri, 29 May 2009) | 1 line complain if details are attached to a token ........ r73039 | benjamin.peterson | 2009-05-29 18:00:28 -0500 (Fri, 29 May 2009) | 1 line add a test for whitespace ........ r73040 | benjamin.peterson | 2009-05-29 18:01:17 -0500 (Fri, 29 May 2009) | 1 line a fix for emacs highlighting ........ r73091 | benjamin.peterson | 2009-05-31 20:55:25 -0500 (Sun, 31 May 2009) | 1 line deprecate set_prefix() and get_prefix() in favor of a prefix property ........ r73092 | benjamin.peterson | 2009-05-31 21:00:51 -0500 (Sun, 31 May 2009) | 1 line change hideous java naming scheme ........ r73093 | benjamin.peterson | 2009-05-31 21:01:39 -0500 (Sun, 31 May 2009) | 1 line remove dated comment ........ r73096 | benjamin.peterson | 2009-05-31 21:40:53 -0500 (Sun, 31 May 2009) | 1 line group tests ........ r73179 | benjamin.peterson | 2009-06-03 13:09:53 -0500 (Wed, 03 Jun 2009) | 1 line handle the case where there's multiple trailers #6185 ........ r73180 | benjamin.peterson | 2009-06-03 13:18:05 -0500 (Wed, 03 Jun 2009) | 1 line scrap __main__ section ........ r73181 | benjamin.peterson | 2009-06-03 13:24:48 -0500 (Wed, 03 Jun 2009) | 1 line remove shebang lines and __main__ sections ........ r73192 | benjamin.peterson | 2009-06-03 19:16:30 -0500 (Wed, 03 Jun 2009) | 4 lines actually test something here Thanks to Joe Amenta for noticing.y ........ r73231 | benjamin.peterson | 2009-06-04 13:38:50 -0500 (Thu, 04 Jun 2009) | 1 line remove unused variable ........ r73244 | benjamin.peterson | 2009-06-05 08:39:25 -0500 (Fri, 05 Jun 2009) | 1 line allow fixers to give different options in setUp ........ r73255 | benjamin.peterson | 2009-06-06 11:23:46 -0500 (Sat, 06 Jun 2009) | 1 line fix the except fixer on one line suites #6222 ........ r73256 | benjamin.peterson | 2009-06-06 11:27:40 -0500 (Sat, 06 Jun 2009) | 1 line test one-line else and finally clauses ........ r73365 | benjamin.peterson | 2009-06-11 17:01:32 -0500 (Thu, 11 Jun 2009) | 1 line normalize whitespace ........ ................
2009-06-11 20:47:38 -03:00
_mapping = {"unichr" : "chr", "unicode" : "str"}
_literal_re = re.compile(r"[uU][rR]?[\'\"]")
Merged revisions 64286 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r64286 | benjamin.peterson | 2008-06-14 21:31:05 -0500 (Sat, 14 Jun 2008) | 49 lines Merged revisions 63661,63666,63695,63711,63729,63769,63790,63880,63886 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r63661 | georg.brandl | 2008-05-26 05:26:20 -0500 (Mon, 26 May 2008) | 2 lines Add import fixes for dbm package. ........ r63666 | georg.brandl | 2008-05-26 05:49:09 -0500 (Mon, 26 May 2008) | 2 lines Add xmlrpc package fixes. ........ r63695 | georg.brandl | 2008-05-26 10:14:33 -0500 (Mon, 26 May 2008) | 2 lines Add fixer entries for http package. ........ r63711 | benjamin.peterson | 2008-05-26 13:43:51 -0500 (Mon, 26 May 2008) | 2 lines add import mapping for test.test_support -> test.support ........ r63729 | benjamin.peterson | 2008-05-26 16:31:03 -0500 (Mon, 26 May 2008) | 2 lines mapping for commands module -> subprocess ........ r63769 | brett.cannon | 2008-05-29 00:13:13 -0500 (Thu, 29 May 2008) | 1 line Fixer for UserString.UserString over to the collections module. ........ r63790 | brett.cannon | 2008-05-29 14:13:51 -0500 (Thu, 29 May 2008) | 4 lines Add a fixer for UserList. Closes issue #2878. Thanks to Quentin Gallet-Gilles for the patch. ........ r63880 | collin.winter | 2008-06-01 18:09:38 -0500 (Sun, 01 Jun 2008) | 6 lines Move lib2to3/fixes/{basefix,util}.py down to lib2to3/. This is step 1 of turning lib2to3/ into a general-purpose refactoring library, reusable by other projects. ........ r63886 | collin.winter | 2008-06-01 22:15:01 -0500 (Sun, 01 Jun 2008) | 5 lines Allow refactoring tools to specify a directory for fixer modules. This is step 2 of turning lib2to3/ into a general-purpose refactoring library, reusable by other projects. Step 1: r63880. ........ ................
2008-06-14 23:57:40 -03:00
class FixUnicode(fixer_base.BaseFix):
Merged revisions 73370 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r73370 | benjamin.peterson | 2009-06-11 17:06:46 -0500 (Thu, 11 Jun 2009) | 105 lines Merged revisions 72523,72950-72951,72994,73003,73033,73036-73040,73091-73093,73096,73179-73181,73192,73231,73244,73255-73256,73365 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r72523 | benjamin.peterson | 2009-05-09 14:42:26 -0500 (Sat, 09 May 2009) | 1 line remove parenthesis ........ r72950 | benjamin.peterson | 2009-05-26 18:19:45 -0500 (Tue, 26 May 2009) | 1 line remove unused imports ........ r72951 | benjamin.peterson | 2009-05-26 18:27:00 -0500 (Tue, 26 May 2009) | 1 line this is no longer executable ........ r72994 | benjamin.peterson | 2009-05-28 15:32:54 -0500 (Thu, 28 May 2009) | 1 line fix test_all_fixers on Windows #6134 ........ r73003 | benjamin.peterson | 2009-05-28 21:57:28 -0500 (Thu, 28 May 2009) | 4 lines make 2to3 test utilities easier to use with other applications (3to2) Patch by Joe Amenta ........ r73033 | benjamin.peterson | 2009-05-29 16:58:32 -0500 (Fri, 29 May 2009) | 1 line update grammar for multi with statement ........ r73036 | benjamin.peterson | 2009-05-29 17:33:20 -0500 (Fri, 29 May 2009) | 1 line simplify fix_unicode ........ r73037 | benjamin.peterson | 2009-05-29 17:53:03 -0500 (Fri, 29 May 2009) | 1 line add custom error for pattern syntax errors ........ r73038 | benjamin.peterson | 2009-05-29 17:55:00 -0500 (Fri, 29 May 2009) | 1 line complain if details are attached to a token ........ r73039 | benjamin.peterson | 2009-05-29 18:00:28 -0500 (Fri, 29 May 2009) | 1 line add a test for whitespace ........ r73040 | benjamin.peterson | 2009-05-29 18:01:17 -0500 (Fri, 29 May 2009) | 1 line a fix for emacs highlighting ........ r73091 | benjamin.peterson | 2009-05-31 20:55:25 -0500 (Sun, 31 May 2009) | 1 line deprecate set_prefix() and get_prefix() in favor of a prefix property ........ r73092 | benjamin.peterson | 2009-05-31 21:00:51 -0500 (Sun, 31 May 2009) | 1 line change hideous java naming scheme ........ r73093 | benjamin.peterson | 2009-05-31 21:01:39 -0500 (Sun, 31 May 2009) | 1 line remove dated comment ........ r73096 | benjamin.peterson | 2009-05-31 21:40:53 -0500 (Sun, 31 May 2009) | 1 line group tests ........ r73179 | benjamin.peterson | 2009-06-03 13:09:53 -0500 (Wed, 03 Jun 2009) | 1 line handle the case where there's multiple trailers #6185 ........ r73180 | benjamin.peterson | 2009-06-03 13:18:05 -0500 (Wed, 03 Jun 2009) | 1 line scrap __main__ section ........ r73181 | benjamin.peterson | 2009-06-03 13:24:48 -0500 (Wed, 03 Jun 2009) | 1 line remove shebang lines and __main__ sections ........ r73192 | benjamin.peterson | 2009-06-03 19:16:30 -0500 (Wed, 03 Jun 2009) | 4 lines actually test something here Thanks to Joe Amenta for noticing.y ........ r73231 | benjamin.peterson | 2009-06-04 13:38:50 -0500 (Thu, 04 Jun 2009) | 1 line remove unused variable ........ r73244 | benjamin.peterson | 2009-06-05 08:39:25 -0500 (Fri, 05 Jun 2009) | 1 line allow fixers to give different options in setUp ........ r73255 | benjamin.peterson | 2009-06-06 11:23:46 -0500 (Sat, 06 Jun 2009) | 1 line fix the except fixer on one line suites #6222 ........ r73256 | benjamin.peterson | 2009-06-06 11:27:40 -0500 (Sat, 06 Jun 2009) | 1 line test one-line else and finally clauses ........ r73365 | benjamin.peterson | 2009-06-11 17:01:32 -0500 (Thu, 11 Jun 2009) | 1 line normalize whitespace ........ ................
2009-06-11 20:47:38 -03:00
PATTERN = "STRING | 'unicode' | 'unichr'"
def transform(self, node, results):
if node.type == token.NAME:
Merged revisions 73370 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r73370 | benjamin.peterson | 2009-06-11 17:06:46 -0500 (Thu, 11 Jun 2009) | 105 lines Merged revisions 72523,72950-72951,72994,73003,73033,73036-73040,73091-73093,73096,73179-73181,73192,73231,73244,73255-73256,73365 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r72523 | benjamin.peterson | 2009-05-09 14:42:26 -0500 (Sat, 09 May 2009) | 1 line remove parenthesis ........ r72950 | benjamin.peterson | 2009-05-26 18:19:45 -0500 (Tue, 26 May 2009) | 1 line remove unused imports ........ r72951 | benjamin.peterson | 2009-05-26 18:27:00 -0500 (Tue, 26 May 2009) | 1 line this is no longer executable ........ r72994 | benjamin.peterson | 2009-05-28 15:32:54 -0500 (Thu, 28 May 2009) | 1 line fix test_all_fixers on Windows #6134 ........ r73003 | benjamin.peterson | 2009-05-28 21:57:28 -0500 (Thu, 28 May 2009) | 4 lines make 2to3 test utilities easier to use with other applications (3to2) Patch by Joe Amenta ........ r73033 | benjamin.peterson | 2009-05-29 16:58:32 -0500 (Fri, 29 May 2009) | 1 line update grammar for multi with statement ........ r73036 | benjamin.peterson | 2009-05-29 17:33:20 -0500 (Fri, 29 May 2009) | 1 line simplify fix_unicode ........ r73037 | benjamin.peterson | 2009-05-29 17:53:03 -0500 (Fri, 29 May 2009) | 1 line add custom error for pattern syntax errors ........ r73038 | benjamin.peterson | 2009-05-29 17:55:00 -0500 (Fri, 29 May 2009) | 1 line complain if details are attached to a token ........ r73039 | benjamin.peterson | 2009-05-29 18:00:28 -0500 (Fri, 29 May 2009) | 1 line add a test for whitespace ........ r73040 | benjamin.peterson | 2009-05-29 18:01:17 -0500 (Fri, 29 May 2009) | 1 line a fix for emacs highlighting ........ r73091 | benjamin.peterson | 2009-05-31 20:55:25 -0500 (Sun, 31 May 2009) | 1 line deprecate set_prefix() and get_prefix() in favor of a prefix property ........ r73092 | benjamin.peterson | 2009-05-31 21:00:51 -0500 (Sun, 31 May 2009) | 1 line change hideous java naming scheme ........ r73093 | benjamin.peterson | 2009-05-31 21:01:39 -0500 (Sun, 31 May 2009) | 1 line remove dated comment ........ r73096 | benjamin.peterson | 2009-05-31 21:40:53 -0500 (Sun, 31 May 2009) | 1 line group tests ........ r73179 | benjamin.peterson | 2009-06-03 13:09:53 -0500 (Wed, 03 Jun 2009) | 1 line handle the case where there's multiple trailers #6185 ........ r73180 | benjamin.peterson | 2009-06-03 13:18:05 -0500 (Wed, 03 Jun 2009) | 1 line scrap __main__ section ........ r73181 | benjamin.peterson | 2009-06-03 13:24:48 -0500 (Wed, 03 Jun 2009) | 1 line remove shebang lines and __main__ sections ........ r73192 | benjamin.peterson | 2009-06-03 19:16:30 -0500 (Wed, 03 Jun 2009) | 4 lines actually test something here Thanks to Joe Amenta for noticing.y ........ r73231 | benjamin.peterson | 2009-06-04 13:38:50 -0500 (Thu, 04 Jun 2009) | 1 line remove unused variable ........ r73244 | benjamin.peterson | 2009-06-05 08:39:25 -0500 (Fri, 05 Jun 2009) | 1 line allow fixers to give different options in setUp ........ r73255 | benjamin.peterson | 2009-06-06 11:23:46 -0500 (Sat, 06 Jun 2009) | 1 line fix the except fixer on one line suites #6222 ........ r73256 | benjamin.peterson | 2009-06-06 11:27:40 -0500 (Sat, 06 Jun 2009) | 1 line test one-line else and finally clauses ........ r73365 | benjamin.peterson | 2009-06-11 17:01:32 -0500 (Thu, 11 Jun 2009) | 1 line normalize whitespace ........ ................
2009-06-11 20:47:38 -03:00
new = node.clone()
new.value = _mapping[node.value]
return new
elif node.type == token.STRING:
Merged revisions 73370 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r73370 | benjamin.peterson | 2009-06-11 17:06:46 -0500 (Thu, 11 Jun 2009) | 105 lines Merged revisions 72523,72950-72951,72994,73003,73033,73036-73040,73091-73093,73096,73179-73181,73192,73231,73244,73255-73256,73365 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r72523 | benjamin.peterson | 2009-05-09 14:42:26 -0500 (Sat, 09 May 2009) | 1 line remove parenthesis ........ r72950 | benjamin.peterson | 2009-05-26 18:19:45 -0500 (Tue, 26 May 2009) | 1 line remove unused imports ........ r72951 | benjamin.peterson | 2009-05-26 18:27:00 -0500 (Tue, 26 May 2009) | 1 line this is no longer executable ........ r72994 | benjamin.peterson | 2009-05-28 15:32:54 -0500 (Thu, 28 May 2009) | 1 line fix test_all_fixers on Windows #6134 ........ r73003 | benjamin.peterson | 2009-05-28 21:57:28 -0500 (Thu, 28 May 2009) | 4 lines make 2to3 test utilities easier to use with other applications (3to2) Patch by Joe Amenta ........ r73033 | benjamin.peterson | 2009-05-29 16:58:32 -0500 (Fri, 29 May 2009) | 1 line update grammar for multi with statement ........ r73036 | benjamin.peterson | 2009-05-29 17:33:20 -0500 (Fri, 29 May 2009) | 1 line simplify fix_unicode ........ r73037 | benjamin.peterson | 2009-05-29 17:53:03 -0500 (Fri, 29 May 2009) | 1 line add custom error for pattern syntax errors ........ r73038 | benjamin.peterson | 2009-05-29 17:55:00 -0500 (Fri, 29 May 2009) | 1 line complain if details are attached to a token ........ r73039 | benjamin.peterson | 2009-05-29 18:00:28 -0500 (Fri, 29 May 2009) | 1 line add a test for whitespace ........ r73040 | benjamin.peterson | 2009-05-29 18:01:17 -0500 (Fri, 29 May 2009) | 1 line a fix for emacs highlighting ........ r73091 | benjamin.peterson | 2009-05-31 20:55:25 -0500 (Sun, 31 May 2009) | 1 line deprecate set_prefix() and get_prefix() in favor of a prefix property ........ r73092 | benjamin.peterson | 2009-05-31 21:00:51 -0500 (Sun, 31 May 2009) | 1 line change hideous java naming scheme ........ r73093 | benjamin.peterson | 2009-05-31 21:01:39 -0500 (Sun, 31 May 2009) | 1 line remove dated comment ........ r73096 | benjamin.peterson | 2009-05-31 21:40:53 -0500 (Sun, 31 May 2009) | 1 line group tests ........ r73179 | benjamin.peterson | 2009-06-03 13:09:53 -0500 (Wed, 03 Jun 2009) | 1 line handle the case where there's multiple trailers #6185 ........ r73180 | benjamin.peterson | 2009-06-03 13:18:05 -0500 (Wed, 03 Jun 2009) | 1 line scrap __main__ section ........ r73181 | benjamin.peterson | 2009-06-03 13:24:48 -0500 (Wed, 03 Jun 2009) | 1 line remove shebang lines and __main__ sections ........ r73192 | benjamin.peterson | 2009-06-03 19:16:30 -0500 (Wed, 03 Jun 2009) | 4 lines actually test something here Thanks to Joe Amenta for noticing.y ........ r73231 | benjamin.peterson | 2009-06-04 13:38:50 -0500 (Thu, 04 Jun 2009) | 1 line remove unused variable ........ r73244 | benjamin.peterson | 2009-06-05 08:39:25 -0500 (Fri, 05 Jun 2009) | 1 line allow fixers to give different options in setUp ........ r73255 | benjamin.peterson | 2009-06-06 11:23:46 -0500 (Sat, 06 Jun 2009) | 1 line fix the except fixer on one line suites #6222 ........ r73256 | benjamin.peterson | 2009-06-06 11:27:40 -0500 (Sat, 06 Jun 2009) | 1 line test one-line else and finally clauses ........ r73365 | benjamin.peterson | 2009-06-11 17:01:32 -0500 (Thu, 11 Jun 2009) | 1 line normalize whitespace ........ ................
2009-06-11 20:47:38 -03:00
if _literal_re.match(node.value):
new = node.clone()
new.value = new.value[1:]
return new