When parsing an "elif" node, lineno and col_offset of the node now point to the "elif" keyword and not to its condition, making it consistent with the "if" node.
https://bugs.python.org/issue39031
Automerge-Triggered-By: @pablogsal
Nested BinOp instances (e.g. a+b+c) had a wrong col_offset for the
second BinOp (e.g. 2 instead of 0 in the example). Fix it by using the
correct st node to copy the line and col_offset from in ast.c.
This commit contains the implementation of PEP570: Python positional-only parameters.
* Update Grammar/Grammar with new typedarglist and varargslist
* Regenerate grammar files
* Update and regenerate AST related files
* Update code object
* Update marshal.c
* Update compiler and symtable
* Regenerate importlib files
* Update callable objects
* Implement positional-only args logic in ceval.c
* Regenerate frozen data
* Update standard library to account for positional-only args
* Add test file for positional-only args
* Update other test files to account for positional-only args
* Add News entry
* Update inspect module and related tests
The value is a string for string and byte literals, None otherwise.
It is 'u' for u"..." literals, 'b' for b"..." literals, '' for "..." literals.
The 'r' (raw) prefix is ignored.
Does not apply to f-strings.
This appears sufficient to make mypy capable of using the stdlib ast module instead of typed_ast (assuming a mypy patch I'm working on).
WIP: I need to make the tests pass. @ilevkivskyi @serhiy-storchaka
https://bugs.python.org/issue36280
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST. Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
```python
class C:
pass
pass
```
the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.
An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.
The lineno and col_offset attributes of AST nodes for list comprehensions,
generator expressions and tuples are now point to the opening parenthesis or
square brace. For tuples without parenthesis they point to the position
of the first item.
Remove the docstring attribute of AST types and restore docstring
expression as a first stmt in their body.
Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
bpo-29463 added optional "docstring" field to 4 AST types.
While it is optional, it breaks backward compatibility because AST constructor
requires number of positional argument is same to number of fields.
AST types accepts empty arguments, and incomplete keyword arguments.
But it's not big problem because field can be filled after creation, and checked when compiling.
So stop requiring complete set of fields for positional arguments too.
* bpo-29463: Add docstring field to some AST nodes.
ClassDef, ModuleDef, FunctionDef, and AsyncFunctionDef has docstring
field for now. It was first statement of there body.
* fix document. thanks travis!
* doc fixes
The compile ignores constant statements and emit a SyntaxWarning warning.
Don't emit the warning for string statement because triple quoted string is a
common syntax for multiline comments.
Don't emit the warning on ellipis neither: 'def f(): ...' is a legit syntax for
abstract functions.
Changes:
* test_ast: ignore SyntaxWarning when compiling test statements. Modify
test_load_const() to use assignment expressions rather than constant
expression.
* test_code: add more kinds of constant statements, ignore SyntaxWarning when
testing that the compiler removes constant statements.
* test_grammar: ignore SyntaxWarning on the statement "1"
Issue #26146: Add a new kind of AST node: ast.Constant. It can be used by
external AST optimizers, but the compiler does not emit directly such node.
An optimizer can replace the following AST nodes with ast.Constant:
* ast.NameConstant: None, False, True
* ast.Num: int, float, complex
* ast.Str: str
* ast.Bytes: bytes
* ast.Tuple if items are constants too: tuple
* frozenset
Update code to accept ast.Constant instead of ast.Num and/or ast.Str:
* compiler
* docstrings
* ast.literal_eval()
* Tools/parser/unparse.py
svn+ssh://pythondev@svn.python.org/python/trunk
........
r75149 | gregory.p.smith | 2009-09-29 16:56:31 -0500 (Tue, 29 Sep 2009) | 3 lines
Mention issue6972 in extractall docs about overwriting things outside of
the supplied path.
........
r75260 | andrew.kuchling | 2009-10-05 16:24:20 -0500 (Mon, 05 Oct 2009) | 1 line
Wording fix
........
r75261 | andrew.kuchling | 2009-10-05 16:24:35 -0500 (Mon, 05 Oct 2009) | 1 line
Fix narkup
........
r75262 | andrew.kuchling | 2009-10-05 16:25:03 -0500 (Mon, 05 Oct 2009) | 1 line
Document 'skip' parameter to constructor
........
r75263 | andrew.kuchling | 2009-10-05 16:25:35 -0500 (Mon, 05 Oct 2009) | 1 line
Note side benefit of socket.create_connection()
........
r75265 | andrew.kuchling | 2009-10-05 17:31:11 -0500 (Mon, 05 Oct 2009) | 1 line
Reword sentence
........
r75266 | andrew.kuchling | 2009-10-05 17:32:48 -0500 (Mon, 05 Oct 2009) | 1 line
Use standard comma punctuation; reword some sentences in the docs
........
r75267 | andrew.kuchling | 2009-10-05 17:42:56 -0500 (Mon, 05 Oct 2009) | 1 line
Backport r73983: Document the thousands separator.
........
r75292 | benjamin.peterson | 2009-10-08 22:11:36 -0500 (Thu, 08 Oct 2009) | 1 line
death to old CVS keyword
........
r75300 | benjamin.peterson | 2009-10-09 16:48:14 -0500 (Fri, 09 Oct 2009) | 1 line
fix some coding style
........
r75376 | benjamin.peterson | 2009-10-11 20:26:07 -0500 (Sun, 11 Oct 2009) | 1 line
platform we don't care about
........
r75405 | neil.schemenauer | 2009-10-14 12:17:14 -0500 (Wed, 14 Oct 2009) | 4 lines
Issue #1754094: Improve the stack depth calculation in the compiler.
There should be no other effect than a small decrease in memory use.
Patch by Christopher Tur Lesniewski-Laas.
........
r75429 | benjamin.peterson | 2009-10-14 20:47:28 -0500 (Wed, 14 Oct 2009) | 1 line
pep8ify if blocks
........
r75430 | benjamin.peterson | 2009-10-14 20:49:37 -0500 (Wed, 14 Oct 2009) | 1 line
use floor division and add a test that exercises the tabsize codepath
........
r75431 | benjamin.peterson | 2009-10-14 20:56:25 -0500 (Wed, 14 Oct 2009) | 1 line
change test to what I intended
........
r75432 | benjamin.peterson | 2009-10-14 22:05:39 -0500 (Wed, 14 Oct 2009) | 1 line
some cleanups
........
r75433 | benjamin.peterson | 2009-10-14 22:06:55 -0500 (Wed, 14 Oct 2009) | 1 line
make inspect.isabstract() always return a boolean; add a test for it, too #7069
........
r75437 | benjamin.peterson | 2009-10-15 10:44:46 -0500 (Thu, 15 Oct 2009) | 1 line
only clear a module's __dict__ if the module is the only one with a reference to it #7140
........
r75445 | vinay.sajip | 2009-10-16 09:06:44 -0500 (Fri, 16 Oct 2009) | 1 line
Issue #7120: logging: Removed import of multiprocessing which is causing crash in GAE.
........
r75501 | antoine.pitrou | 2009-10-18 13:37:11 -0500 (Sun, 18 Oct 2009) | 3 lines
Add a comment about unreachable code, and fix a typo
........
r75551 | benjamin.peterson | 2009-10-19 22:14:10 -0500 (Mon, 19 Oct 2009) | 1 line
use property api
........
r75572 | benjamin.peterson | 2009-10-20 16:55:17 -0500 (Tue, 20 Oct 2009) | 1 line
clarify buffer arg #7178
........
r75589 | benjamin.peterson | 2009-10-21 21:26:47 -0500 (Wed, 21 Oct 2009) | 1 line
whitespace
........
r75590 | benjamin.peterson | 2009-10-21 21:36:47 -0500 (Wed, 21 Oct 2009) | 1 line
rewrite to be nice to other implementations
........
r75591 | benjamin.peterson | 2009-10-21 21:50:38 -0500 (Wed, 21 Oct 2009) | 4 lines
rewrite for style, clarify, and comments
Also, use the hasattr() like scheme of allowing BaseException exceptions through.
........
r75657 | antoine.pitrou | 2009-10-24 07:41:27 -0500 (Sat, 24 Oct 2009) | 3 lines
Fix compilation error in debug mode.
........
r75742 | benjamin.peterson | 2009-10-26 17:51:16 -0500 (Mon, 26 Oct 2009) | 1 line
use 'is' instead of id()
........
r75868 | benjamin.peterson | 2009-10-27 15:59:18 -0500 (Tue, 27 Oct 2009) | 1 line
test expect base classes
........
r75952 | georg.brandl | 2009-10-29 15:38:32 -0500 (Thu, 29 Oct 2009) | 1 line
Use the correct function name in docstring.
........
r75953 | georg.brandl | 2009-10-29 15:39:50 -0500 (Thu, 29 Oct 2009) | 1 line
Remove mention of the old -X command line switch.
........
r75954 | georg.brandl | 2009-10-29 15:53:00 -0500 (Thu, 29 Oct 2009) | 1 line
Use constants instead of magic integers for test result. Do not re-run with --verbose3 for environment changing tests.
........
r75955 | georg.brandl | 2009-10-29 15:54:03 -0500 (Thu, 29 Oct 2009) | 1 line
Use a single style for all the docstrings in the math module.
........
r75956 | georg.brandl | 2009-10-29 16:16:34 -0500 (Thu, 29 Oct 2009) | 1 line
I do not think the "railroad" program mentioned is still available.
........
r75957 | georg.brandl | 2009-10-29 16:44:56 -0500 (Thu, 29 Oct 2009) | 1 line
Fix constant name.
........
r76057 | benjamin.peterson | 2009-11-02 09:06:45 -0600 (Mon, 02 Nov 2009) | 1 line
prevent a rather unlikely segfault
........
r76105 | georg.brandl | 2009-11-04 01:38:12 -0600 (Wed, 04 Nov 2009) | 1 line
#7259: show correct equivalent for operator.i* operations in docstring; fix minor issues in operator docs.
........
r76139 | benjamin.peterson | 2009-11-06 19:04:38 -0600 (Fri, 06 Nov 2009) | 1 line
spelling
........
r76143 | georg.brandl | 2009-11-07 02:26:07 -0600 (Sat, 07 Nov 2009) | 1 line
#7271: fix typo.
........
r76162 | benjamin.peterson | 2009-11-08 22:10:53 -0600 (Sun, 08 Nov 2009) | 1 line
discuss how to use -p
........
r76223 | georg.brandl | 2009-11-12 02:29:46 -0600 (Thu, 12 Nov 2009) | 1 line
Give the profile module a module directive.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r73376 | benjamin.peterson | 2009-06-11 17:29:23 -0500 (Thu, 11 Jun 2009) | 1 line
remove check for case handled in sub-function
........
r73393 | alexandre.vassalotti | 2009-06-12 13:56:57 -0500 (Fri, 12 Jun 2009) | 2 lines
Clear reference to the static PyExc_RecursionErrorInst in _PyExc_Fini.
........
r73398 | alexandre.vassalotti | 2009-06-12 15:57:12 -0500 (Fri, 12 Jun 2009) | 3 lines
Add const qualifier to PyErr_SetFromErrnoWithFilename and to
PyErr_SetFromErrnoWithUnicodeFilename.
........
r73400 | alexandre.vassalotti | 2009-06-12 16:43:47 -0500 (Fri, 12 Jun 2009) | 2 lines
Delete outdated make file for building the parser with MSVC 6.
........
r73404 | benjamin.peterson | 2009-06-12 20:40:00 -0500 (Fri, 12 Jun 2009) | 1 line
keep the slice.step field as NULL if no step expression is given
........
r73405 | benjamin.peterson | 2009-06-12 22:46:30 -0500 (Fri, 12 Jun 2009) | 1 line
prevent import statements from assigning to None
........
r73409 | benjamin.peterson | 2009-06-13 08:06:21 -0500 (Sat, 13 Jun 2009) | 1 line
allow importing from a module named None if it has an 'as' clause
........
r73419 | benjamin.peterson | 2009-06-13 11:19:19 -0500 (Sat, 13 Jun 2009) | 1 line
set Print.values to NULL if there are no values
........
r73420 | benjamin.peterson | 2009-06-13 12:08:53 -0500 (Sat, 13 Jun 2009) | 1 line
give a better error message when deleting ()
........
r73421 | benjamin.peterson | 2009-06-13 15:23:33 -0500 (Sat, 13 Jun 2009) | 1 line
when no module is given in a 'from' relative import, make ImportFrom.module NULL
........
r73432 | amaury.forgeotdarc | 2009-06-14 16:20:40 -0500 (Sun, 14 Jun 2009) | 3 lines
#6227: Because of a wrong indentation, the test was not testing what it should.
Ensure that the snippet in doctest_aliases actually contains aliases.
........
r73457 | benjamin.peterson | 2009-06-16 18:13:09 -0500 (Tue, 16 Jun 2009) | 1 line
add underscores
........
r73460 | benjamin.peterson | 2009-06-16 22:23:04 -0500 (Tue, 16 Jun 2009) | 1 line
remove unused 'encoding' member from the compiler struct
........
r73485 | benjamin.peterson | 2009-06-19 17:07:47 -0500 (Fri, 19 Jun 2009) | 1 line
remove duplicate test
........
r73486 | benjamin.peterson | 2009-06-19 17:09:17 -0500 (Fri, 19 Jun 2009) | 1 line
add missing assertion #6313
........
r73488 | benjamin.peterson | 2009-06-19 17:16:28 -0500 (Fri, 19 Jun 2009) | 1 line
show that this one isn't used
........
r73489 | benjamin.peterson | 2009-06-19 17:21:12 -0500 (Fri, 19 Jun 2009) | 1 line
use closures
........
r73501 | benjamin.peterson | 2009-06-21 18:01:07 -0500 (Sun, 21 Jun 2009) | 1 line
don't need to add the name 'lambda' as assigned
........
r73502 | benjamin.peterson | 2009-06-21 18:03:36 -0500 (Sun, 21 Jun 2009) | 1 line
remove tmpname support since it's no longer used
........
r73513 | benjamin.peterson | 2009-06-22 20:18:57 -0500 (Mon, 22 Jun 2009) | 1 line
fix grammar
........
r73514 | benjamin.peterson | 2009-06-22 22:01:56 -0500 (Mon, 22 Jun 2009) | 1 line
remove some unused symtable constants
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r68450 | jeffrey.yasskin | 2009-01-09 10:47:07 -0600 (Fri, 09 Jan 2009) | 3 lines
Fix issue 4884, preventing a crash in the socket code when python is compiled
with llvm-gcc and run with a glibc <2.10.
........
r68480 | vinay.sajip | 2009-01-10 07:38:04 -0600 (Sat, 10 Jan 2009) | 1 line
Minor documentation changes cross-referencing NullHandler to the documentation on configuring logging in a library.
........
r68481 | vinay.sajip | 2009-01-10 07:42:04 -0600 (Sat, 10 Jan 2009) | 1 line
Corrected an incorrect self-reference.
........
r68493 | benjamin.peterson | 2009-01-10 11:18:55 -0600 (Sat, 10 Jan 2009) | 1 line
rewrite verbose conditionals
........
r68495 | benjamin.peterson | 2009-01-10 11:36:44 -0600 (Sat, 10 Jan 2009) | 1 line
tp_iter only exists with Py_TPFLAGS_HAVE_ITER #4901
........
r68501 | vinay.sajip | 2009-01-10 13:22:57 -0600 (Sat, 10 Jan 2009) | 1 line
Corrected minor typo and added .currentmodule directives to fix missing cross-references.
........
r68512 | benjamin.peterson | 2009-01-10 16:42:10 -0600 (Sat, 10 Jan 2009) | 1 line
make tests fail if they can't be imported
........
r68514 | benjamin.peterson | 2009-01-10 17:41:59 -0600 (Sat, 10 Jan 2009) | 1 line
move seealso to a more appropiate place
........
r68515 | benjamin.peterson | 2009-01-10 17:49:08 -0600 (Sat, 10 Jan 2009) | 1 line
macos 9 isn't supported
........
r68534 | gregory.p.smith | 2009-01-11 11:53:33 -0600 (Sun, 11 Jan 2009) | 2 lines
correct email address
........
r68535 | gregory.p.smith | 2009-01-11 11:57:54 -0600 (Sun, 11 Jan 2009) | 9 lines
Update the documentation for binascii and zlib crc32/adler32 functions
to better describe the signed vs unsigned return value behavior on
different platforms and versions of python. Mention the workaround to
make them all return the same thing by using & 0xffffffff.
Fixes issue4903.
Also needs to be merged into release26-maint, release30-maint, & py3k.
........
r68536 | benjamin.peterson | 2009-01-11 13:48:15 -0600 (Sun, 11 Jan 2009) | 1 line
add email addresses
........
r68552 | vinay.sajip | 2009-01-12 14:36:18 -0600 (Mon, 12 Jan 2009) | 1 line
Minor changes/corrections in markup.
........
r68563 | benjamin.peterson | 2009-01-12 19:49:10 -0600 (Mon, 12 Jan 2009) | 1 line
small logic correction
........
r68570 | raymond.hettinger | 2009-01-13 03:08:32 -0600 (Tue, 13 Jan 2009) | 5 lines
Issue 4922: Incorrect comments for MutableSet.add() and MutableSet.discard().
Needs to be backported to 2.6 and forward ported to 3.0 and 3.1.
........
r68571 | armin.ronacher | 2009-01-13 05:52:23 -0600 (Tue, 13 Jan 2009) | 3 lines
ast.literal_eval can properly evaluate complex numbers now. This fixes issue4907.
........
r68572 | andrew.kuchling | 2009-01-13 07:40:54 -0600 (Tue, 13 Jan 2009) | 1 line
Note that first coord. is left alone
........
r68575 | thomas.heller | 2009-01-13 11:32:28 -0600 (Tue, 13 Jan 2009) | 1 line
Fix refcount leak in error cases. Bug found by coverity.
........
r68582 | georg.brandl | 2009-01-13 16:14:01 -0600 (Tue, 13 Jan 2009) | 2 lines
Use assertRaises.
........
r68596 | amaury.forgeotdarc | 2009-01-13 17:39:22 -0600 (Tue, 13 Jan 2009) | 3 lines
#1162154: inspect.getmembers() now skips attributes that raise AttributeError,
e.g. a __slots__ attribute which has not been set.
........
r68623 | vinay.sajip | 2009-01-15 16:48:13 -0600 (Thu, 15 Jan 2009) | 1 line
Made minor changes/corrections in markup. Added a couple of section headings.
........
r68624 | vinay.sajip | 2009-01-15 17:04:47 -0600 (Thu, 15 Jan 2009) | 1 line
Minor changes/corrections in markup.
........
r68628 | benjamin.peterson | 2009-01-15 20:55:24 -0600 (Thu, 15 Jan 2009) | 1 line
compare with == not is #4946
........
Serhiy Storchaka
Batuhan Taşkaya
Lysandros Nikolaou
Raymond Hettinger
Min ho Kim
Pablo Galindo
Carl Friedrich Bolz-Tereick
Guido van Rossum
Ivan Levkivskyi
Anthony Sottile
guoci
Oren Milman
Matthias Bussonnier
INADA Naoki
Yury Selivanov
Berker Peksag
Victor Stinner
Benjamin Peterson
Yury Selivanov
Ezio Melotti
Brett Cannon
Mark Dickinson
Nick Coghlan
Amaury Forgeot d'Arc
Florent Xicluna
Georg Brandl
Raymond Hettinger