cpython/Parser
Serhiy Storchaka 8ac658114d
bpo-30455: Generate all token related code and docs from Grammar/Tokens. (GH-10370)
"Include/token.h", "Lib/token.py" (containing now some data moved from
"Lib/tokenize.py") and new files "Parser/token.c" (containing the code
moved from "Parser/tokenizer.c") and "Doc/library/token-list.inc" (included
in "Doc/library/token.rst") are now generated from "Grammar/Tokens" by
"Tools/scripts/generate_token.py". The script overwrites files only if
needed and can be used on the read-only sources tree.

"Lib/symbol.py" is now generated by "Tools/scripts/generate_symbol_py.py"
instead of been executable itself.

Added new make targets "regen-token" and "regen-symbol" which are now
dependencies of "regen-all".

The documentation contains now strings for operators and punctuation tokens.
2018-12-22 11:18:40 +02:00
..
Python.asdl bpo-32892: Use ast.Constant instead of specific constant AST types. (GH-9445) 2018-09-27 17:42:37 +03:00
acceler.c
asdl.py Add ast.Constant 2016-01-26 00:40:57 +01:00
asdl_c.py bpo-35177, Python-ast.h: Fix "Yield" compiler warning (GH-10664) 2018-11-22 18:38:38 +01:00
bitset.c
firstsets.c
grammar.c properly free memory in pgen 2016-09-18 18:00:25 -07:00
grammar1.c bpo-31338 (#3374) 2017-09-14 18:13:16 -07:00
listnode.c Use NULL rather than 0. (#778) 2017-03-23 17:53:47 +02:00
metagrammar.c
myreadline.c bpo-35436: Add missing PyErr_NoMemory() calls and other minor bug fixes. (GH-11015) 2018-12-07 12:11:30 +02:00
node.c replace PY_SIZE_MAX with SIZE_MAX 2016-09-07 09:26:18 -07:00
parser.c Issue #26130: Remove redundant variable 's' from Parser/parser.c 2016-03-28 00:45:28 +03:00
parser.h bpo-32150: Expand tabs to spaces in C files. (#4583) 2017-11-28 17:56:10 +02:00
parsetok.c bpo-34683: Make SyntaxError column offsets consistently 1-indexed (gh-9338) 2018-09-24 14:12:49 -07:00
parsetok_pgen.c
pgen.c properly free memory in pgen 2016-09-18 18:00:25 -07:00
pgenmain.c bpo-35081: Rename internal headers (GH-10275) 2018-11-12 16:53:38 +01:00
printgrammar.c closes bpo-34646: Remove PyAPI_* macros from declarations. (GH-9218) 2018-09-12 12:06:42 -07:00
token.c bpo-30455: Generate all token related code and docs from Grammar/Tokens. (GH-10370) 2018-12-22 11:18:40 +02:00
tokenizer.c bpo-30455: Generate all token related code and docs from Grammar/Tokens. (GH-10370) 2018-12-22 11:18:40 +02:00
tokenizer.h bpo-33306: Improve SyntaxError messages for unbalanced parentheses. (GH-6516) 2018-12-17 17:34:14 +02:00
tokenizer_pgen.c