cpython/Lib/lib2to3/Grammar.txt

155 lines
6.4 KiB
Plaintext
Raw Normal View History

# Grammar for 2to3. This grammar supports Python 2.x and 3.x.
# NOTE WELL: You should also follow all the steps listed at
# https://devguide.python.org/grammar/
# Start symbols for the grammar:
# file_input is a module or sequence of commands read from an input file;
# single_input is a single interactive statement;
# eval_input is the input for the eval() and input() functions.
# NB: compound_stmt in single_input is followed by extra NEWLINE!
file_input: (NEWLINE | stmt)* ENDMARKER
single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE
eval_input: testlist NEWLINE* ENDMARKER
decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
decorators: decorator+
decorated: decorators (classdef | funcdef | async_funcdef)
async_funcdef: ASYNC funcdef
funcdef: 'def' NAME parameters ['->' test] ':' suite
parameters: '(' [typedargslist] ')'
typedargslist: ((tfpdef ['=' test] ',')*
('*' [tname] (',' tname ['=' test])* [',' ['**' tname [',']]] | '**' tname [','])
| tfpdef ['=' test] (',' tfpdef ['=' test])* [','])
tname: NAME [':' test]
tfpdef: tname | '(' tfplist ')'
tfplist: tfpdef (',' tfpdef)* [',']
varargslist: ((vfpdef ['=' test] ',')*
('*' [vname] (',' vname ['=' test])* [',' ['**' vname [',']]] | '**' vname [','])
| vfpdef ['=' test] (',' vfpdef ['=' test])* [','])
vname: NAME
vfpdef: vname | '(' vfplist ')'
vfplist: vfpdef (',' vfpdef)* [',']
stmt: simple_stmt | compound_stmt
simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE
small_stmt: (expr_stmt | print_stmt | del_stmt | pass_stmt | flow_stmt |
import_stmt | global_stmt | exec_stmt | assert_stmt)
expr_stmt: testlist_star_expr (annassign | augassign (yield_expr|testlist) |
Merged revisions 76062 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r76062 | benjamin.peterson | 2009-11-02 12:12:12 -0600 (Mon, 02 Nov 2009) | 70 lines Merged revisions 74359,75081,75088,75213,75278,75303,75427-75428,75734-75736,75865,76059-76061 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r74359 | benjamin.peterson | 2009-08-12 17:23:13 -0500 (Wed, 12 Aug 2009) | 1 line don't pass the deprecated print_function option ........ r75081 | benjamin.peterson | 2009-09-26 22:02:57 -0500 (Sat, 26 Sep 2009) | 1 line let 2to3 work with extended iterable unpacking ........ r75088 | benjamin.peterson | 2009-09-27 11:25:21 -0500 (Sun, 27 Sep 2009) | 1 line look on the type only for __call__ ........ r75213 | benjamin.peterson | 2009-10-03 10:09:46 -0500 (Sat, 03 Oct 2009) | 5 lines revert 75212; it's not correct People can use isinstance(x, collections.Callable) if they expect objects with __call__ in their instance dictionaries. ........ r75278 | benjamin.peterson | 2009-10-07 16:25:56 -0500 (Wed, 07 Oct 2009) | 4 lines fix whitespace problems with fix_idioms #3563 Patch by Joe Amenta. ........ r75303 | benjamin.peterson | 2009-10-09 16:59:11 -0500 (Fri, 09 Oct 2009) | 1 line port latin-1 and utf-8 cookie improvements ........ r75427 | benjamin.peterson | 2009-10-14 20:35:57 -0500 (Wed, 14 Oct 2009) | 1 line force floor division ........ r75428 | benjamin.peterson | 2009-10-14 20:39:21 -0500 (Wed, 14 Oct 2009) | 1 line silence -3 warnings about __hash__ ........ r75734 | benjamin.peterson | 2009-10-26 16:25:53 -0500 (Mon, 26 Oct 2009) | 2 lines warn on map(None, ...) with more than 2 arguments #7203 ........ r75735 | benjamin.peterson | 2009-10-26 16:28:25 -0500 (Mon, 26 Oct 2009) | 1 line remove unused result ........ r75736 | benjamin.peterson | 2009-10-26 16:29:02 -0500 (Mon, 26 Oct 2009) | 1 line using get() here is a bit pointless ........ r75865 | benjamin.peterson | 2009-10-27 15:49:00 -0500 (Tue, 27 Oct 2009) | 1 line explain reason for warning ........ r76059 | benjamin.peterson | 2009-11-02 11:43:47 -0600 (Mon, 02 Nov 2009) | 1 line tuples are no longer used for children ........ r76060 | benjamin.peterson | 2009-11-02 11:55:40 -0600 (Mon, 02 Nov 2009) | 1 line revert r76059; apparently some fixers rely on Leaf no () for children ........ r76061 | benjamin.peterson | 2009-11-02 12:06:17 -0600 (Mon, 02 Nov 2009) | 1 line make fix_tuple_params keep the tree valid #7253 ........ ................
2009-11-02 14:16:28 -04:00
('=' (yield_expr|testlist_star_expr))*)
annassign: ':' test ['=' test]
Merged revisions 76062 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r76062 | benjamin.peterson | 2009-11-02 12:12:12 -0600 (Mon, 02 Nov 2009) | 70 lines Merged revisions 74359,75081,75088,75213,75278,75303,75427-75428,75734-75736,75865,76059-76061 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r74359 | benjamin.peterson | 2009-08-12 17:23:13 -0500 (Wed, 12 Aug 2009) | 1 line don't pass the deprecated print_function option ........ r75081 | benjamin.peterson | 2009-09-26 22:02:57 -0500 (Sat, 26 Sep 2009) | 1 line let 2to3 work with extended iterable unpacking ........ r75088 | benjamin.peterson | 2009-09-27 11:25:21 -0500 (Sun, 27 Sep 2009) | 1 line look on the type only for __call__ ........ r75213 | benjamin.peterson | 2009-10-03 10:09:46 -0500 (Sat, 03 Oct 2009) | 5 lines revert 75212; it's not correct People can use isinstance(x, collections.Callable) if they expect objects with __call__ in their instance dictionaries. ........ r75278 | benjamin.peterson | 2009-10-07 16:25:56 -0500 (Wed, 07 Oct 2009) | 4 lines fix whitespace problems with fix_idioms #3563 Patch by Joe Amenta. ........ r75303 | benjamin.peterson | 2009-10-09 16:59:11 -0500 (Fri, 09 Oct 2009) | 1 line port latin-1 and utf-8 cookie improvements ........ r75427 | benjamin.peterson | 2009-10-14 20:35:57 -0500 (Wed, 14 Oct 2009) | 1 line force floor division ........ r75428 | benjamin.peterson | 2009-10-14 20:39:21 -0500 (Wed, 14 Oct 2009) | 1 line silence -3 warnings about __hash__ ........ r75734 | benjamin.peterson | 2009-10-26 16:25:53 -0500 (Mon, 26 Oct 2009) | 2 lines warn on map(None, ...) with more than 2 arguments #7203 ........ r75735 | benjamin.peterson | 2009-10-26 16:28:25 -0500 (Mon, 26 Oct 2009) | 1 line remove unused result ........ r75736 | benjamin.peterson | 2009-10-26 16:29:02 -0500 (Mon, 26 Oct 2009) | 1 line using get() here is a bit pointless ........ r75865 | benjamin.peterson | 2009-10-27 15:49:00 -0500 (Tue, 27 Oct 2009) | 1 line explain reason for warning ........ r76059 | benjamin.peterson | 2009-11-02 11:43:47 -0600 (Mon, 02 Nov 2009) | 1 line tuples are no longer used for children ........ r76060 | benjamin.peterson | 2009-11-02 11:55:40 -0600 (Mon, 02 Nov 2009) | 1 line revert r76059; apparently some fixers rely on Leaf no () for children ........ r76061 | benjamin.peterson | 2009-11-02 12:06:17 -0600 (Mon, 02 Nov 2009) | 1 line make fix_tuple_params keep the tree valid #7253 ........ ................
2009-11-02 14:16:28 -04:00
testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [',']
augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' |
'<<=' | '>>=' | '**=' | '//=')
# For normal and annotated assignments, additional restrictions enforced by the interpreter
print_stmt: 'print' ( [ test (',' test)* [','] ] |
'>>' test [ (',' test)+ [','] ] )
del_stmt: 'del' exprlist
pass_stmt: 'pass'
flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt
break_stmt: 'break'
continue_stmt: 'continue'
return_stmt: 'return' [testlist]
yield_stmt: yield_expr
raise_stmt: 'raise' [test ['from' test | ',' test [',' test]]]
import_stmt: import_name | import_from
import_name: 'import' dotted_as_names
import_from: ('from' ('.'* dotted_name | '.'+)
'import' ('*' | '(' import_as_names ')' | import_as_names))
import_as_name: NAME ['as' NAME]
dotted_as_name: dotted_name ['as' NAME]
import_as_names: import_as_name (',' import_as_name)* [',']
dotted_as_names: dotted_as_name (',' dotted_as_name)*
dotted_name: NAME ('.' NAME)*
global_stmt: ('global' | 'nonlocal') NAME (',' NAME)*
exec_stmt: 'exec' expr ['in' test [',' test]]
assert_stmt: 'assert' test [',' test]
compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef | decorated | async_stmt
async_stmt: ASYNC (funcdef | with_stmt | for_stmt)
if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
while_stmt: 'while' test ':' suite ['else' ':' suite]
for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
try_stmt: ('try' ':' suite
((except_clause ':' suite)+
['else' ':' suite]
['finally' ':' suite] |
'finally' ':' suite))
Merged revisions 73370 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r73370 | benjamin.peterson | 2009-06-11 17:06:46 -0500 (Thu, 11 Jun 2009) | 105 lines Merged revisions 72523,72950-72951,72994,73003,73033,73036-73040,73091-73093,73096,73179-73181,73192,73231,73244,73255-73256,73365 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r72523 | benjamin.peterson | 2009-05-09 14:42:26 -0500 (Sat, 09 May 2009) | 1 line remove parenthesis ........ r72950 | benjamin.peterson | 2009-05-26 18:19:45 -0500 (Tue, 26 May 2009) | 1 line remove unused imports ........ r72951 | benjamin.peterson | 2009-05-26 18:27:00 -0500 (Tue, 26 May 2009) | 1 line this is no longer executable ........ r72994 | benjamin.peterson | 2009-05-28 15:32:54 -0500 (Thu, 28 May 2009) | 1 line fix test_all_fixers on Windows #6134 ........ r73003 | benjamin.peterson | 2009-05-28 21:57:28 -0500 (Thu, 28 May 2009) | 4 lines make 2to3 test utilities easier to use with other applications (3to2) Patch by Joe Amenta ........ r73033 | benjamin.peterson | 2009-05-29 16:58:32 -0500 (Fri, 29 May 2009) | 1 line update grammar for multi with statement ........ r73036 | benjamin.peterson | 2009-05-29 17:33:20 -0500 (Fri, 29 May 2009) | 1 line simplify fix_unicode ........ r73037 | benjamin.peterson | 2009-05-29 17:53:03 -0500 (Fri, 29 May 2009) | 1 line add custom error for pattern syntax errors ........ r73038 | benjamin.peterson | 2009-05-29 17:55:00 -0500 (Fri, 29 May 2009) | 1 line complain if details are attached to a token ........ r73039 | benjamin.peterson | 2009-05-29 18:00:28 -0500 (Fri, 29 May 2009) | 1 line add a test for whitespace ........ r73040 | benjamin.peterson | 2009-05-29 18:01:17 -0500 (Fri, 29 May 2009) | 1 line a fix for emacs highlighting ........ r73091 | benjamin.peterson | 2009-05-31 20:55:25 -0500 (Sun, 31 May 2009) | 1 line deprecate set_prefix() and get_prefix() in favor of a prefix property ........ r73092 | benjamin.peterson | 2009-05-31 21:00:51 -0500 (Sun, 31 May 2009) | 1 line change hideous java naming scheme ........ r73093 | benjamin.peterson | 2009-05-31 21:01:39 -0500 (Sun, 31 May 2009) | 1 line remove dated comment ........ r73096 | benjamin.peterson | 2009-05-31 21:40:53 -0500 (Sun, 31 May 2009) | 1 line group tests ........ r73179 | benjamin.peterson | 2009-06-03 13:09:53 -0500 (Wed, 03 Jun 2009) | 1 line handle the case where there's multiple trailers #6185 ........ r73180 | benjamin.peterson | 2009-06-03 13:18:05 -0500 (Wed, 03 Jun 2009) | 1 line scrap __main__ section ........ r73181 | benjamin.peterson | 2009-06-03 13:24:48 -0500 (Wed, 03 Jun 2009) | 1 line remove shebang lines and __main__ sections ........ r73192 | benjamin.peterson | 2009-06-03 19:16:30 -0500 (Wed, 03 Jun 2009) | 4 lines actually test something here Thanks to Joe Amenta for noticing.y ........ r73231 | benjamin.peterson | 2009-06-04 13:38:50 -0500 (Thu, 04 Jun 2009) | 1 line remove unused variable ........ r73244 | benjamin.peterson | 2009-06-05 08:39:25 -0500 (Fri, 05 Jun 2009) | 1 line allow fixers to give different options in setUp ........ r73255 | benjamin.peterson | 2009-06-06 11:23:46 -0500 (Sat, 06 Jun 2009) | 1 line fix the except fixer on one line suites #6222 ........ r73256 | benjamin.peterson | 2009-06-06 11:27:40 -0500 (Sat, 06 Jun 2009) | 1 line test one-line else and finally clauses ........ r73365 | benjamin.peterson | 2009-06-11 17:01:32 -0500 (Thu, 11 Jun 2009) | 1 line normalize whitespace ........ ................
2009-06-11 20:47:38 -03:00
with_stmt: 'with' with_item (',' with_item)* ':' suite
with_item: test ['as' expr]
with_var: 'as' expr
# NB compile.c makes sure that the default except clause is last
except_clause: 'except' [test [(',' | 'as') test]]
suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT
# Backward compatibility cruft to support:
# [ x for x in lambda: True, lambda: False if x() ]
# even while also allowing:
# lambda x: 5 if x else 2
# (But not a mix of the two)
testlist_safe: old_test [(',' old_test)+ [',']]
old_test: or_test | old_lambdef
old_lambdef: 'lambda' [varargslist] ':' old_test
test: or_test ['if' or_test 'else' test] | lambdef
or_test: and_test ('or' and_test)*
and_test: not_test ('and' not_test)*
not_test: 'not' not_test | comparison
comparison: expr (comp_op expr)*
comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'
Merged revisions 76062 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r76062 | benjamin.peterson | 2009-11-02 12:12:12 -0600 (Mon, 02 Nov 2009) | 70 lines Merged revisions 74359,75081,75088,75213,75278,75303,75427-75428,75734-75736,75865,76059-76061 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r74359 | benjamin.peterson | 2009-08-12 17:23:13 -0500 (Wed, 12 Aug 2009) | 1 line don't pass the deprecated print_function option ........ r75081 | benjamin.peterson | 2009-09-26 22:02:57 -0500 (Sat, 26 Sep 2009) | 1 line let 2to3 work with extended iterable unpacking ........ r75088 | benjamin.peterson | 2009-09-27 11:25:21 -0500 (Sun, 27 Sep 2009) | 1 line look on the type only for __call__ ........ r75213 | benjamin.peterson | 2009-10-03 10:09:46 -0500 (Sat, 03 Oct 2009) | 5 lines revert 75212; it's not correct People can use isinstance(x, collections.Callable) if they expect objects with __call__ in their instance dictionaries. ........ r75278 | benjamin.peterson | 2009-10-07 16:25:56 -0500 (Wed, 07 Oct 2009) | 4 lines fix whitespace problems with fix_idioms #3563 Patch by Joe Amenta. ........ r75303 | benjamin.peterson | 2009-10-09 16:59:11 -0500 (Fri, 09 Oct 2009) | 1 line port latin-1 and utf-8 cookie improvements ........ r75427 | benjamin.peterson | 2009-10-14 20:35:57 -0500 (Wed, 14 Oct 2009) | 1 line force floor division ........ r75428 | benjamin.peterson | 2009-10-14 20:39:21 -0500 (Wed, 14 Oct 2009) | 1 line silence -3 warnings about __hash__ ........ r75734 | benjamin.peterson | 2009-10-26 16:25:53 -0500 (Mon, 26 Oct 2009) | 2 lines warn on map(None, ...) with more than 2 arguments #7203 ........ r75735 | benjamin.peterson | 2009-10-26 16:28:25 -0500 (Mon, 26 Oct 2009) | 1 line remove unused result ........ r75736 | benjamin.peterson | 2009-10-26 16:29:02 -0500 (Mon, 26 Oct 2009) | 1 line using get() here is a bit pointless ........ r75865 | benjamin.peterson | 2009-10-27 15:49:00 -0500 (Tue, 27 Oct 2009) | 1 line explain reason for warning ........ r76059 | benjamin.peterson | 2009-11-02 11:43:47 -0600 (Mon, 02 Nov 2009) | 1 line tuples are no longer used for children ........ r76060 | benjamin.peterson | 2009-11-02 11:55:40 -0600 (Mon, 02 Nov 2009) | 1 line revert r76059; apparently some fixers rely on Leaf no () for children ........ r76061 | benjamin.peterson | 2009-11-02 12:06:17 -0600 (Mon, 02 Nov 2009) | 1 line make fix_tuple_params keep the tree valid #7253 ........ ................
2009-11-02 14:16:28 -04:00
star_expr: '*' expr
expr: xor_expr ('|' xor_expr)*
xor_expr: and_expr ('^' and_expr)*
and_expr: shift_expr ('&' shift_expr)*
shift_expr: arith_expr (('<<'|'>>') arith_expr)*
arith_expr: term (('+'|'-') term)*
term: factor (('*'|'@'|'/'|'%'|'//') factor)*
factor: ('+'|'-'|'~') factor | power
power: [AWAIT] atom trailer* ['**' factor]
atom: ('(' [yield_expr|testlist_gexp] ')' |
'[' [listmaker] ']' |
'{' [dictsetmaker] '}' |
'`' testlist1 '`' |
NAME | NUMBER | STRING+ | '.' '.' '.')
listmaker: (test|star_expr) ( comp_for | (',' (test|star_expr))* [','] )
testlist_gexp: (test|star_expr) ( comp_for | (',' (test|star_expr))* [','] )
lambdef: 'lambda' [varargslist] ':' test
trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
subscriptlist: subscript (',' subscript)* [',']
subscript: test | [test] ':' [test] [sliceop]
sliceop: ':' [test]
Merged revisions 76062 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r76062 | benjamin.peterson | 2009-11-02 12:12:12 -0600 (Mon, 02 Nov 2009) | 70 lines Merged revisions 74359,75081,75088,75213,75278,75303,75427-75428,75734-75736,75865,76059-76061 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r74359 | benjamin.peterson | 2009-08-12 17:23:13 -0500 (Wed, 12 Aug 2009) | 1 line don't pass the deprecated print_function option ........ r75081 | benjamin.peterson | 2009-09-26 22:02:57 -0500 (Sat, 26 Sep 2009) | 1 line let 2to3 work with extended iterable unpacking ........ r75088 | benjamin.peterson | 2009-09-27 11:25:21 -0500 (Sun, 27 Sep 2009) | 1 line look on the type only for __call__ ........ r75213 | benjamin.peterson | 2009-10-03 10:09:46 -0500 (Sat, 03 Oct 2009) | 5 lines revert 75212; it's not correct People can use isinstance(x, collections.Callable) if they expect objects with __call__ in their instance dictionaries. ........ r75278 | benjamin.peterson | 2009-10-07 16:25:56 -0500 (Wed, 07 Oct 2009) | 4 lines fix whitespace problems with fix_idioms #3563 Patch by Joe Amenta. ........ r75303 | benjamin.peterson | 2009-10-09 16:59:11 -0500 (Fri, 09 Oct 2009) | 1 line port latin-1 and utf-8 cookie improvements ........ r75427 | benjamin.peterson | 2009-10-14 20:35:57 -0500 (Wed, 14 Oct 2009) | 1 line force floor division ........ r75428 | benjamin.peterson | 2009-10-14 20:39:21 -0500 (Wed, 14 Oct 2009) | 1 line silence -3 warnings about __hash__ ........ r75734 | benjamin.peterson | 2009-10-26 16:25:53 -0500 (Mon, 26 Oct 2009) | 2 lines warn on map(None, ...) with more than 2 arguments #7203 ........ r75735 | benjamin.peterson | 2009-10-26 16:28:25 -0500 (Mon, 26 Oct 2009) | 1 line remove unused result ........ r75736 | benjamin.peterson | 2009-10-26 16:29:02 -0500 (Mon, 26 Oct 2009) | 1 line using get() here is a bit pointless ........ r75865 | benjamin.peterson | 2009-10-27 15:49:00 -0500 (Tue, 27 Oct 2009) | 1 line explain reason for warning ........ r76059 | benjamin.peterson | 2009-11-02 11:43:47 -0600 (Mon, 02 Nov 2009) | 1 line tuples are no longer used for children ........ r76060 | benjamin.peterson | 2009-11-02 11:55:40 -0600 (Mon, 02 Nov 2009) | 1 line revert r76059; apparently some fixers rely on Leaf no () for children ........ r76061 | benjamin.peterson | 2009-11-02 12:06:17 -0600 (Mon, 02 Nov 2009) | 1 line make fix_tuple_params keep the tree valid #7253 ........ ................
2009-11-02 14:16:28 -04:00
exprlist: (expr|star_expr) (',' (expr|star_expr))* [',']
testlist: test (',' test)* [',']
dictsetmaker: ( ((test ':' test | '**' expr)
(comp_for | (',' (test ':' test | '**' expr))* [','])) |
((test | star_expr)
(comp_for | (',' (test | star_expr))* [','])) )
classdef: 'class' NAME ['(' [arglist] ')'] ':' suite
arglist: argument (',' argument)* [',']
# "test '=' test" is really "keyword '=' test", but we have no such token.
# These need to be in a single rule to avoid grammar that is ambiguous
# to our LL(1) parser. Even though 'test' includes '*expr' in star_expr,
# we explicitly match '*' here, too, to give it proper precedence.
# Illegal combinations and orderings are blocked in ast.c:
# multiple (test comp_for) arguments are blocked; keyword unpackings
# that precede iterable unpackings are blocked; etc.
argument: ( test [comp_for] |
test '=' test |
'**' test |
'*' test )
comp_iter: comp_for | comp_if
comp_for: [ASYNC] 'for' exprlist 'in' testlist_safe [comp_iter]
comp_if: 'if' old_test [comp_iter]
testlist1: test (',' test)*
# not used in grammar, but may appear in "node" passed from Parser to Compiler
encoding_decl: NAME
2014-04-10 01:23:18 -03:00
yield_expr: 'yield' [yield_arg]
yield_arg: 'from' test | testlist