This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
This commit simplifies async/await tokenization in tokenizer.c,
tokenize.py & lib2to3/tokenize.py. Previous solution was to keep
a stack of async-def & def blocks, whereas the new approach is just
to remember position of the outermost async-def block.
This change won't bring any parsing performance improvements, but
it makes the code much easier to read and validate.
This commit fixes how one-line async-defs and defs are tracked
by tokenizer. It allows to correctly parse invalid code such
as:
>>> async def f():
... def g(): pass
... async = 10
and valid code such as:
>>> async def f():
... async def g(): pass
... await z
As a consequence, is is now possible to have one-line
'async def foo(): await ..' functions:
>>> async def foo(): return await bar()
requires them. Disable executable bits and shebang lines in test and
benchmark files in order to prevent using a random system python, and in
source files of modules which don't provide command line interface. Fixed
shebang lines in the unittestgui and checkpip scripts.
requires them. Disable executable bits and shebang lines in test and
benchmark files in order to prevent using a random system python, and in
source files of modules which don't provide command line interface. Fixed
shebang line to use python3 executable in the unittestgui script.
* The first line of Python script could be executed twice when the source
encoding (not equal to 'utf-8') was specified on the second line.
* Now the source encoding declaration on the second line isn't effective if
the first line contains anything except a comment.
* As a consequence, 'python -x' works now again with files with the source
encoding declarations specified on the second file, and can be used again
to make Python batch files on Windows.
* The tokenize module now ignore the source encoding declaration on the second
line if the first line contains anything except a comment.
* IDLE now ignores the source encoding declaration on the second line if the
first line contains anything except a comment.
* 2to3 and the findnocoding.py script now ignore the source encoding
declaration on the second line if the first line contains anything except
a comment.
* The first line of Python script could be executed twice when the source
encoding (not equal to 'utf-8') was specified on the second line.
* Now the source encoding declaration on the second line isn't effective if
the first line contains anything except a comment.
* As a consequence, 'python -x' works now again with files with the source
encoding declarations specified on the second file, and can be used again
to make Python batch files on Windows.
* The tokenize module now ignore the source encoding declaration on the second
line if the first line contains anything except a comment.
* IDLE now ignores the source encoding declaration on the second line if the
first line contains anything except a comment.
* 2to3 and the findnocoding.py script now ignore the source encoding
declaration on the second line if the first line contains anything except
a comment.
directory tree instead of overwriting the input files. Adds three command
line options: -o/--output-dir, -W/--write-unchanged-files and --add-suffix.
Feature backports into stable release branches for 2to3 are allowed by
a special exemption:
http://mail.python.org/pipermail/python-dev/2011-December/115089.html
with line feeds. This was ported from the sandbox to the 3.3 branch, but
didn't make it into 3.2.
- Re-enable lib2to3's test_parser.py tests, though with an expected failure
(see issue 13125).
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r88661 | benjamin.peterson | 2011-02-26 16:06:24 -0600 (Sat, 26 Feb 2011) | 6 lines
fix refactoring on formfeed characters #11250
This is because text.splitlines() is not the same as
list(StringIO.StringIO(text)).
........
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r88535 | brett.cannon | 2011-02-23 13:46:46 -0600 (Wed, 23 Feb 2011) | 1 line
Add lib2to3.__main__ for easy testing from the console.
........
svn+ssh://pythondev@svn.python.org/python/branches/py3k
................
r85510 | benjamin.peterson | 2010-10-14 18:00:04 -0500 (Thu, 14 Oct 2010) | 61 lines
Merged revisions 83852-83853,83857,84042,84216,84274-84276,84375,85388,85478,85506-85508 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r83852 | benjamin.peterson | 2010-08-08 15:45:44 -0500 (Sun, 08 Aug 2010) | 1 line
wrap with parens
........
r83853 | benjamin.peterson | 2010-08-08 15:46:31 -0500 (Sun, 08 Aug 2010) | 1 line
use parens
........
r83857 | benjamin.peterson | 2010-08-08 15:59:49 -0500 (Sun, 08 Aug 2010) | 1 line
things which use touch_import should be pre order
........
r84042 | george.boutsioukis | 2010-08-14 16:10:19 -0500 (Sat, 14 Aug 2010) | 2 lines
This revision incorporates into the 2to3 tool the new, faster, tree matching algorithm developed during a GSOC project. The algorithm resides in the two added modules, btm_matcher and btm_utils. New code has been added to drive the new matching process in refactor.py and a few minor changes were made in other modules. A BM_compatible flag(False by default) has been added in fixer_base and it is set to True in most of the current fixers.
........
r84216 | benjamin.peterson | 2010-08-19 16:44:05 -0500 (Thu, 19 Aug 2010) | 1 line
allow star_expr in testlist_gexp
........
r84274 | benjamin.peterson | 2010-08-22 18:40:46 -0500 (Sun, 22 Aug 2010) | 1 line
wrap long line
........
r84275 | benjamin.peterson | 2010-08-22 18:42:22 -0500 (Sun, 22 Aug 2010) | 1 line
cleanup
........
r84276 | benjamin.peterson | 2010-08-22 18:51:01 -0500 (Sun, 22 Aug 2010) | 1 line
when there's a None value and a traceback, don't call type with it #9661
........
r84375 | george.boutsioukis | 2010-08-31 08:38:53 -0500 (Tue, 31 Aug 2010) | 3 lines
Idiomatic code changes & stylistic issues fixed in the BottomMatcher module. Thanks to Benjamin Peterson for taking the time to review the code.
........
r85388 | benjamin.peterson | 2010-10-12 17:27:44 -0500 (Tue, 12 Oct 2010) | 1 line
fix urllib fixer with multiple as imports on a line #10069
........
r85478 | benjamin.peterson | 2010-10-14 08:09:56 -0500 (Thu, 14 Oct 2010) | 1 line
stop abusing docstrings
........
r85506 | benjamin.peterson | 2010-10-14 17:45:19 -0500 (Thu, 14 Oct 2010) | 1 line
kill sibling import
........
r85507 | benjamin.peterson | 2010-10-14 17:54:15 -0500 (Thu, 14 Oct 2010) | 1 line
remove trailing whitespace
........
r85508 | benjamin.peterson | 2010-10-14 17:55:28 -0500 (Thu, 14 Oct 2010) | 1 line
typo
........
................
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r83852 | benjamin.peterson | 2010-08-08 15:45:44 -0500 (Sun, 08 Aug 2010) | 1 line
wrap with parens
........
r83853 | benjamin.peterson | 2010-08-08 15:46:31 -0500 (Sun, 08 Aug 2010) | 1 line
use parens
........
r83857 | benjamin.peterson | 2010-08-08 15:59:49 -0500 (Sun, 08 Aug 2010) | 1 line
things which use touch_import should be pre order
........
r84042 | george.boutsioukis | 2010-08-14 16:10:19 -0500 (Sat, 14 Aug 2010) | 2 lines
This revision incorporates into the 2to3 tool the new, faster, tree matching algorithm developed during a GSOC project. The algorithm resides in the two added modules, btm_matcher and btm_utils. New code has been added to drive the new matching process in refactor.py and a few minor changes were made in other modules. A BM_compatible flag(False by default) has been added in fixer_base and it is set to True in most of the current fixers.
........
r84216 | benjamin.peterson | 2010-08-19 16:44:05 -0500 (Thu, 19 Aug 2010) | 1 line
allow star_expr in testlist_gexp
........
r84274 | benjamin.peterson | 2010-08-22 18:40:46 -0500 (Sun, 22 Aug 2010) | 1 line
wrap long line
........
r84275 | benjamin.peterson | 2010-08-22 18:42:22 -0500 (Sun, 22 Aug 2010) | 1 line
cleanup
........
r84276 | benjamin.peterson | 2010-08-22 18:51:01 -0500 (Sun, 22 Aug 2010) | 1 line
when there's a None value and a traceback, don't call type with it #9661
........
r84375 | george.boutsioukis | 2010-08-31 08:38:53 -0500 (Tue, 31 Aug 2010) | 3 lines
Idiomatic code changes & stylistic issues fixed in the BottomMatcher module. Thanks to Benjamin Peterson for taking the time to review the code.
........
r85388 | benjamin.peterson | 2010-10-12 17:27:44 -0500 (Tue, 12 Oct 2010) | 1 line
fix urllib fixer with multiple as imports on a line #10069
........
r85478 | benjamin.peterson | 2010-10-14 08:09:56 -0500 (Thu, 14 Oct 2010) | 1 line
stop abusing docstrings
........
r85506 | benjamin.peterson | 2010-10-14 17:45:19 -0500 (Thu, 14 Oct 2010) | 1 line
kill sibling import
........
r85507 | benjamin.peterson | 2010-10-14 17:54:15 -0500 (Thu, 14 Oct 2010) | 1 line
remove trailing whitespace
........
r85508 | benjamin.peterson | 2010-10-14 17:55:28 -0500 (Thu, 14 Oct 2010) | 1 line
typo
........
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r82779 | benjamin.peterson | 2010-07-10 14:45:08 -0500 (Sat, 10 Jul 2010) | 1 line
typo in attribute name #9217
........
r82855 | benjamin.peterson | 2010-07-13 16:27:38 -0500 (Tue, 13 Jul 2010) | 1 line
remove more extraneous commas #9245
........
r83740 | alexandre.vassalotti | 2010-08-05 01:58:36 -0500 (Thu, 05 Aug 2010) | 4 lines
Issue 5077: Update fixer for the other functions gone from the operator module.
Patch by Meador Inge.
........
r83789 | benjamin.peterson | 2010-08-07 17:45:14 -0500 (Sat, 07 Aug 2010) | 1 line
cleanup and use unicode consistently
........
r83790 | benjamin.peterson | 2010-08-07 17:52:06 -0500 (Sat, 07 Aug 2010) | 1 line
unicode literal
........
r83791 | benjamin.peterson | 2010-08-07 17:52:55 -0500 (Sat, 07 Aug 2010) | 1 line
.get() is pointless here
........
r83797 | benjamin.peterson | 2010-08-07 18:54:51 -0500 (Sat, 07 Aug 2010) | 1 line
add a function to find how a node is indented
........
r83798 | benjamin.peterson | 2010-08-07 18:55:28 -0500 (Sat, 07 Aug 2010) | 1 line
when splitting import statements, use correct indentation #9386
........
r83799 | benjamin.peterson | 2010-08-07 18:57:43 -0500 (Sat, 07 Aug 2010) | 1 line
double quotes
........
r83800 | benjamin.peterson | 2010-08-07 18:58:52 -0500 (Sat, 07 Aug 2010) | 1 line
add another test
........
r83801 | benjamin.peterson | 2010-08-07 19:02:10 -0500 (Sat, 07 Aug 2010) | 1 line
cleanup; style-nits
........
r83803 | benjamin.peterson | 2010-08-07 19:05:08 -0500 (Sat, 07 Aug 2010) | 1 line
slightly more explicit
........
r83811 | benjamin.peterson | 2010-08-07 22:56:44 -0500 (Sat, 07 Aug 2010) | 4 lines
Fix node.pre_order() to call the right method on its children.
This was a rather tragic copy-paste error.
........
r83827 | benjamin.peterson | 2010-08-08 08:12:48 -0500 (Sun, 08 Aug 2010) | 1 line
cause test to actually run and fix it
........
r83844 | benjamin.peterson | 2010-08-08 13:46:37 -0500 (Sun, 08 Aug 2010) | 1 line
fix whitespace
........
svn+ssh://pythondev@svn.python.org/python/trunk
................
r78408 | benjamin.peterson | 2010-02-23 20:24:35 -0600 (Tue, 23 Feb 2010) | 9 lines
Merged revisions 78407 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r78407 | benjamin.peterson | 2010-02-23 20:21:34 -0600 (Tue, 23 Feb 2010) | 1 line
rewrite test to not rely on __doc__ being present
........
................
svn+ssh://pythondev@svn.python.org/python/trunk
........
r78403 | r.david.murray | 2010-02-23 21:08:28 -0500 (Tue, 23 Feb 2010) | 3 lines
The primary copy of lib2to3 is not trunk, so the lib2to3 change
should not have been included in the -OO patch, back it out.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r78351 | r.david.murray | 2010-02-22 19:24:49 -0500 (Mon, 22 Feb 2010) | 5 lines
Issue 6292: for the moment at least, the test suite passes if run
with -OO. Tests requiring docstrings are skipped. Patch by
Brian Curtin, thanks to Matias Torchinsky for helping review and
improve the patch.
........
svn+ssh://pythondev@svn.python.org/python/branches/py3k
................
r77900 | martin.v.loewis | 2010-02-01 02:18:51 +0100 (Mo, 01 Feb 2010) | 24 lines
Merged revisions 77898 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
................
r77898 | martin.v.loewis | 2010-02-01 02:15:39 +0100 (Mo, 01 Feb 2010) | 17 lines
Merged revisions 77855-77856,77870 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r77855 | benjamin.peterson | 2010-01-30 17:32:05 +0100 (Sa, 30 Jan 2010) | 1 line
don't return node if it is not changed
........
r77856 | benjamin.peterson | 2010-01-30 17:35:29 +0100 (Sa, 30 Jan 2010) | 1 line
return None to indicate no change
........
r77870 | benjamin.peterson | 2010-01-31 02:21:26 +0100 (So, 31 Jan 2010) | 1 line
never return the original node given to transform()
........
................
................
svn+ssh://pythondev@svn.python.org/python/trunk
................
r77898 | martin.v.loewis | 2010-02-01 02:15:39 +0100 (Mo, 01 Feb 2010) | 17 lines
Merged revisions 77855-77856,77870 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r77855 | benjamin.peterson | 2010-01-30 17:32:05 +0100 (Sa, 30 Jan 2010) | 1 line
don't return node if it is not changed
........
r77856 | benjamin.peterson | 2010-01-30 17:35:29 +0100 (Sa, 30 Jan 2010) | 1 line
return None to indicate no change
........
r77870 | benjamin.peterson | 2010-01-31 02:21:26 +0100 (So, 31 Jan 2010) | 1 line
never return the original node given to transform()
........
................
svn+ssh://pythondev@svn.python.org/python/trunk
................
r77160 | benjamin.peterson | 2009-12-30 13:44:23 -0600 (Wed, 30 Dec 2009) | 9 lines
Merged revisions 77158 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r77158 | benjamin.peterson | 2009-12-30 13:41:03 -0600 (Wed, 30 Dec 2009) | 1 line
clean up logging's global state after the test finishes
........
................
svn+ssh://pythondev@svn.python.org/python/trunk
................
r76062 | benjamin.peterson | 2009-11-02 12:12:12 -0600 (Mon, 02 Nov 2009) | 70 lines
Merged revisions 74359,75081,75088,75213,75278,75303,75427-75428,75734-75736,75865,76059-76061 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r74359 | benjamin.peterson | 2009-08-12 17:23:13 -0500 (Wed, 12 Aug 2009) | 1 line
don't pass the deprecated print_function option
........
r75081 | benjamin.peterson | 2009-09-26 22:02:57 -0500 (Sat, 26 Sep 2009) | 1 line
let 2to3 work with extended iterable unpacking
........
r75088 | benjamin.peterson | 2009-09-27 11:25:21 -0500 (Sun, 27 Sep 2009) | 1 line
look on the type only for __call__
........
r75213 | benjamin.peterson | 2009-10-03 10:09:46 -0500 (Sat, 03 Oct 2009) | 5 lines
revert 75212; it's not correct
People can use isinstance(x, collections.Callable) if they expect objects with
__call__ in their instance dictionaries.
........
r75278 | benjamin.peterson | 2009-10-07 16:25:56 -0500 (Wed, 07 Oct 2009) | 4 lines
fix whitespace problems with fix_idioms #3563
Patch by Joe Amenta.
........
r75303 | benjamin.peterson | 2009-10-09 16:59:11 -0500 (Fri, 09 Oct 2009) | 1 line
port latin-1 and utf-8 cookie improvements
........
r75427 | benjamin.peterson | 2009-10-14 20:35:57 -0500 (Wed, 14 Oct 2009) | 1 line
force floor division
........
r75428 | benjamin.peterson | 2009-10-14 20:39:21 -0500 (Wed, 14 Oct 2009) | 1 line
silence -3 warnings about __hash__
........
r75734 | benjamin.peterson | 2009-10-26 16:25:53 -0500 (Mon, 26 Oct 2009) | 2 lines
warn on map(None, ...) with more than 2 arguments #7203
........
r75735 | benjamin.peterson | 2009-10-26 16:28:25 -0500 (Mon, 26 Oct 2009) | 1 line
remove unused result
........
r75736 | benjamin.peterson | 2009-10-26 16:29:02 -0500 (Mon, 26 Oct 2009) | 1 line
using get() here is a bit pointless
........
r75865 | benjamin.peterson | 2009-10-27 15:49:00 -0500 (Tue, 27 Oct 2009) | 1 line
explain reason for warning
........
r76059 | benjamin.peterson | 2009-11-02 11:43:47 -0600 (Mon, 02 Nov 2009) | 1 line
tuples are no longer used for children
........
r76060 | benjamin.peterson | 2009-11-02 11:55:40 -0600 (Mon, 02 Nov 2009) | 1 line
revert r76059; apparently some fixers rely on Leaf no () for children
........
r76061 | benjamin.peterson | 2009-11-02 12:06:17 -0600 (Mon, 02 Nov 2009) | 1 line
make fix_tuple_params keep the tree valid #7253
........
................
svn+ssh://pythondev@svn.python.org/python/trunk
................
r72494 | benjamin.peterson | 2009-05-08 20:01:14 -0500 (Fri, 08 May 2009) | 21 lines
Merged revisions 72491-72493 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r72491 | benjamin.peterson | 2009-05-08 19:33:27 -0500 (Fri, 08 May 2009) | 7 lines
make 2to3 use unicode internally on 2.x
This started out as a fix for #2660, but became this large refactoring
when I realized the dire state this was in. 2to3 now uses
tokenize.detect_encoding to decode the files correctly into unicode.
........
r72492 | benjamin.peterson | 2009-05-08 19:35:38 -0500 (Fri, 08 May 2009) | 1 line
remove compat code
........
r72493 | benjamin.peterson | 2009-05-08 19:54:15 -0500 (Fri, 08 May 2009) | 1 line
add a test for \r\n newlines
........
................
svn+ssh://pythondev@svn.python.org/python/trunk
................
r72368 | benjamin.peterson | 2009-05-05 18:13:58 -0500 (Tue, 05 May 2009) | 53 lines
Merged revisions 68503,68507,68694,69054,69673,69679-69681,70991,70999,71003,71695 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r68503 | benjamin.peterson | 2009-01-10 14:14:49 -0600 (Sat, 10 Jan 2009) | 1 line
use variable
........
r68507 | benjamin.peterson | 2009-01-10 15:13:16 -0600 (Sat, 10 Jan 2009) | 1 line
rewrap
........
r68694 | benjamin.peterson | 2009-01-17 17:55:59 -0600 (Sat, 17 Jan 2009) | 1 line
test for specific node type
........
r69054 | guilherme.polo | 2009-01-28 10:01:54 -0600 (Wed, 28 Jan 2009) | 2 lines
Added mapping for the ttk module.
........
r69673 | benjamin.peterson | 2009-02-16 09:38:22 -0600 (Mon, 16 Feb 2009) | 1 line
fix handling of as imports #5279
........
r69679 | benjamin.peterson | 2009-02-16 11:36:06 -0600 (Mon, 16 Feb 2009) | 1 line
make Base.get_next_sibling() and Base.get_prev_sibling() properties
........
r69680 | benjamin.peterson | 2009-02-16 11:41:48 -0600 (Mon, 16 Feb 2009) | 1 line
normalize docstrings in pytree according to PEP 11
........
r69681 | benjamin.peterson | 2009-02-16 11:43:09 -0600 (Mon, 16 Feb 2009) | 1 line
use a set
........
r70991 | benjamin.peterson | 2009-04-01 15:54:50 -0500 (Wed, 01 Apr 2009) | 1 line
map urllib.urlopen to urllib.request.open #5637
........
r70999 | benjamin.peterson | 2009-04-01 17:36:47 -0500 (Wed, 01 Apr 2009) | 1 line
add very alpha support to 2to3 for running concurrently with multiprocessing
........
r71003 | benjamin.peterson | 2009-04-01 18:10:43 -0500 (Wed, 01 Apr 2009) | 1 line
fix when multiprocessing is not available or used
........
r71695 | benjamin.peterson | 2009-04-17 22:21:29 -0500 (Fri, 17 Apr 2009) | 1 line
refactor multiprocessing support, so it's less hacky to employ and only loads mp when needed
........
................
svn+ssh://pythondev@svn.python.org/python/trunk
................
r68197 | benjamin.peterson | 2009-01-03 10:34:02 -0600 (Sat, 03 Jan 2009) | 55 lines
Merged revisions 67900-67901,67919,67928,67984,67991-67993,68106-68108,68110 via svnmerge from
svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3
........
r67900 | benjamin.peterson | 2008-12-22 14:02:45 -0600 (Mon, 22 Dec 2008) | 4 lines
fix_execfile: wrap the open(fn).read() call in compile(), so the filename is preserved
also add unittests for the fixer
........
r67901 | benjamin.peterson | 2008-12-22 14:09:55 -0600 (Mon, 22 Dec 2008) | 1 line
remove unused import
........
r67919 | benjamin.peterson | 2008-12-23 13:12:22 -0600 (Tue, 23 Dec 2008) | 1 line
copy permission bits from the backup to the original
........
r67928 | benjamin.peterson | 2008-12-26 20:49:30 -0600 (Fri, 26 Dec 2008) | 1 line
don't be so idiot about multiple local imports in fix_import; still won't handle absolute and local imports on the same line
........
r67984 | benjamin.peterson | 2008-12-28 09:55:16 -0600 (Sun, 28 Dec 2008) | 1 line
don't need loop
........
r67991 | benjamin.peterson | 2008-12-28 14:30:26 -0600 (Sun, 28 Dec 2008) | 1 line
actually call finish_tree()
........
r67992 | benjamin.peterson | 2008-12-28 14:34:47 -0600 (Sun, 28 Dec 2008) | 1 line
remove useless test
........
r67993 | benjamin.peterson | 2008-12-28 15:04:32 -0600 (Sun, 28 Dec 2008) | 1 line
update pyk3's test grammar
........
r68106 | benjamin.peterson | 2008-12-31 11:53:58 -0600 (Wed, 31 Dec 2008) | 1 line
#2734 don't convert every instance of long (eg if it's an attribute)
........
r68107 | benjamin.peterson | 2008-12-31 11:55:10 -0600 (Wed, 31 Dec 2008) | 1 line
add another test
........
r68108 | benjamin.peterson | 2008-12-31 12:00:12 -0600 (Wed, 31 Dec 2008) | 1 line
don't change long even if it's the only argument name
........
r68110 | benjamin.peterson | 2008-12-31 14:13:26 -0600 (Wed, 31 Dec 2008) | 1 line
remove unused import
........
................