9932a22897
bpo-33416: Add end positions to Python AST (GH-11605)
...
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST. Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
```python
class C:
pass
pass
```
the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.
An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.
2019-01-22 11:18:22 +00:00
62be74290a
bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749)
...
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
2018-11-27 13:27:31 +02:00
3bb183d7fb
bpo-35177, Python-ast.h: Fix "Yield" compiler warning (GH-10664)
...
Partially revert commit 5f2df88b63e50d23914e97ec778861a52abdeaad:
add "#undef Yield" to .c files after including Python-ast.h.
Fix the warning:
winbase.h(102): warning C4005: 'Yield': macro redefinition
2018-11-22 18:38:38 +01:00
5f2df88b63
bpo-35177: Add dependencies between header files (GH-10361)
...
* ast.h now includes Python-ast.h and node.h
* parsetok.h now includes node.h and grammar.h
* symtable.h now includes Python-ast.h
* Modify asdl_c.py to enhance Python-ast.h:
* Add #ifndef/#define Py_PYTHON_AST_H to be able to include the header
twice
* Add "extern { ... }" for C++
* Undefine "Yield" macro conflicting with winbase.h
* Remove "#undef Yield" from C files, it's now done in Python-ast.h
* Remove now useless includes in C files
2018-11-12 00:56:19 +01:00
e5362eaa75
bpo-33308: Fix a crash in the parser module when convert an ST object. ( #6519 )
...
Converting with line_info=False and col_info=True crashed before.
2018-04-19 01:55:37 +03:00
e8b1965639
bpo-23699: Use a macro to reduce boilerplate code in rich comparison functions (GH-793)
2017-11-02 20:32:54 +10:00
a79f4c2195
bpo-30070: Fixed leaks and crashes in errors handling in the parser module. ( #1131 )
2017-04-19 21:09:21 +03:00
228b12edcc
Issue #28999 : Use Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE wherever
...
possible. Patch is writen with Coccinelle.
2017-01-23 09:47:21 +02:00
06515833fe
Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
...
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
53595c4879
replace custom validation logic in the parse module with a simple DFA validator ( closes #26526 )
...
Patch from A. Skrobov.
2016-06-02 11:30:18 -07:00
ef1585eb9a
Issue #25923 : Added more const qualifiers to signatures of static and private functions.
2015-12-25 20:01:53 +02:00
2d06e84455
Issue #25923 : Added the const qualifier to static constant arrays.
2015-12-25 19:53:18 +02:00
5c4064e8bd
Issue #25421 : __sizeof__ methods of builtin types now use dynamic basic size.
...
This allows sys.getsize() to work correctly with their subclasses with
__slots__ defined.
2015-12-19 20:05:25 +02:00
14acf5f41d
Issue #24791 : Fix grammar regression for call syntax: 'g(*a or b)'.
2015-08-05 17:54:10 -04:00
de12b79cd6
allow test node after ** in calls ( closes #24176 )
2015-05-16 09:44:45 -04:00
7544508f02
PEP 0492 -- Coroutines with async and await syntax. Issue #24017 .
2015-05-11 22:57:16 -04:00
025e9ebd0a
PEP 448: additional unpacking generalizations ( closes #2292 )
...
Patch by Neil Girdhar.
2015-05-05 20:16:41 -04:00
009b811d67
Removed unintentional trailing spaces in non-external and non-generated C files.
2015-03-18 21:53:15 +02:00
f4c68db162
(Merge 3.3) parser: fix usage of Py_BuildValue() to build a parser error
...
Fix typo: "os" format => "Os"
2014-01-02 11:50:10 +01:00
5f8d485982
parser: fix usage of Py_BuildValue() to build a parser error
...
Fix typo: "os" format => "Os"
2014-01-02 11:49:27 +01:00
14e461d5b9
Close #11619 : The parser and the import machinery do not encode Unicode
...
filenames anymore on Windows.
2013-08-26 22:28:21 +02:00
6684bdf73d
Issue #18408 : Fix typo in build_node_tree() of the parser module
...
Type "o" format of Py_BuildValue() is invalid: it must be "O".
2013-07-17 00:13:52 +02:00
df4572cc71
Issue #18408 : parser module: fix error handling in node2tuple()
...
Handle PyLong_FromLong() and PyUnicode_FromString() failures
2013-07-12 01:35:10 +02:00
3bd6abd129
Issue #18408 : Fix parser.sequence2st() and parser.tuple2st(): raise MemoryError
...
on memory allocation failure
Instead of ignoring the memory allocation failure and create invalid objects.
2013-07-12 01:33:59 +02:00
9101e23ff6
Issue #15989 : Fix several occurrences of integer overflow
...
when result of PyLong_AsLong() narrowed to int without checks.
This is a backport of changesets 13e2e44db99d and 525407d89277.
2013-01-19 12:41:45 +02:00
7898043868
Issue #15989 : Fix several occurrences of integer overflow
...
when result of PyLong_AsLong() narrowed to int without checks.
2013-01-15 01:12:17 +02:00
5b89840d9c
Issue #16714 : use 'raise' exceptions, don't 'throw'.
...
Patch by Serhiy Storchaka.
2012-12-18 21:26:36 +02:00
737fb89dd1
Issue #16714 : use 'raise' exceptions, don't 'throw'.
...
Patch by Serhiy Storchaka.
2012-12-18 21:14:22 +02:00
721738fbee
Issue #15604 : Update uses of PyObject_IsTrue() to check for and handle errors correctly.
...
Patch by Serhiy Storchaka.
2012-08-15 23:20:39 +02:00
6f430e4963
Issue #15604 : Update uses of PyObject_IsTrue() to check for and handle errors correctly.
...
Patch by Serhiy Storchaka.
2012-08-15 23:18:25 +02:00
88ca04e6a8
MERGE: Closes #15512 : Correct __sizeof__ support for parser
2012-08-03 14:29:26 +02:00
e9c5318967
Closes #15512 : Correct __sizeof__ support for parser
2012-08-03 14:28:37 +02:00
943cab2fec
Issue #14741 : Merge fix from 3.2.
2012-05-07 17:25:14 +01:00
da029fb293
Issue #14741 : Fix missing support for ellipsis in parser module.
2012-05-07 17:24:04 +01:00
72f6095d4f
Issue #14697 : Merge fix from 3.2.
2012-05-07 16:36:33 +01:00
11c1dee183
Issue #14697 : Fix missing parser module support for set displays and set comprehensions.
2012-05-07 16:34:34 +01:00
cf360b9209
Issue #14701 : Add missing support for 'raise ... from' in parser module.
2012-05-07 12:01:27 +01:00
9fad160411
Issue #14701 : Merge fix from 3.2.
2012-05-07 12:03:11 +01:00
57404891a0
Issue #14696 : Merge from 3.2
2012-04-29 22:20:01 +01:00
407b3bd89b
Issue #14696 : Fix parser module to understand 'nonlocal' declarations.
2012-04-29 22:18:31 +01:00
b63fd2a408
Issue #9154 : Merge fix from 3.2.
2012-04-29 18:35:56 +01:00
ea7e9f9a83
Issue #9154 : Fix parser module to understand function annotations.
2012-04-29 18:34:40 +01:00
1f7ce62bd6
Implement PEP 380 - 'yield from' ( closes #11682 )
2012-01-13 21:43:40 +10:00
bd928fef42
Rename _Py_identifier to _Py_IDENTIFIER.
2011-10-14 10:20:37 +02:00
1ee1b6fe0d
Use identifier API for PyObject_GetAttrString.
2011-10-10 18:11:30 +02:00
f0cdbad1b3
always clear parser error
2011-06-05 22:14:05 -05:00
f719957d7a
only clear the parser error if it's set ( closes #12264 )
2011-06-04 22:06:42 -05:00
7f2fee3640
Issue #10785 : Store the filename as Unicode in the Python parser.
2011-04-05 00:39:01 +02:00
4969f709cc
#11515 : Merge with 3.1.
2011-03-15 05:59:46 +02:00
42da663e6f
#11515 : fix several typos. Patch by Piotr Kasprzyk.
2011-03-15 05:18:48 +02:00
Ivan Levkivskyi
Serhiy Storchaka
Victor Stinner
stratakis
Benjamin Peterson
Yury Selivanov
Victor Stinner
Andrew Svetlov
Antoine Pitrou
Jesus Cea
Mark Dickinson
Nick Coghlan
Martin v. Löwis
Victor Stinner
Ezio Melotti