traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Add the "ast" module, containing helpers to ease use of the "_ast" classes.

This commit is contained in:
Georg Brandl 2008-06-10 07:45:28 +00:00
parent df7036d2fa
commit 8509db5a21
8 changed files with 662 additions and 95 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Doc/ACKS.txt
View File

@ -157,6 +157,7 @@ docs@python.org), and we'll be glad to correct the problem.
* Bernhard Reiter
* Armin Rigo
* Wes Rishel
* Armin Ronacher
* Jim Roskind
* Guido van Rossum
* Donald Wallace Rouse II

85
Doc/library/_ast.rst
View File

@ -1,85 +0,0 @@
.. _ast:
Abstract Syntax Trees
=====================
.. module:: _ast
:synopsis: Abstract Syntax Tree classes.
.. sectionauthor:: Martin v. Löwis <martin@v.loewis.de>
.. versionadded:: 2.5
The ``_ast`` module helps Python applications to process trees of the Python
abstract syntax grammar. The abstract syntax itself might change with each
Python release; this module helps to find out programmatically what the current
grammar looks like.
An abstract syntax tree can be generated by passing :data:`_ast.PyCF_ONLY_AST`
as a flag to the :func:`compile` builtin function. The result will be a tree of
objects whose classes all inherit from :class:`_ast.AST`.
A modified abstract syntax tree can be compiled into a Python code object using
the built-in :func:`compile` function.
The actual classes are derived from the ``Parser/Python.asdl`` file, which is
reproduced below. There is one class defined for each left-hand side symbol in
the abstract grammar (for example, ``_ast.stmt`` or ``_ast.expr``). In addition,
there is one class defined for each constructor on the right-hand side; these
classes inherit from the classes for the left-hand side trees. For example,
``_ast.BinOp`` inherits from ``_ast.expr``. For production rules with
alternatives (aka "sums"), the left-hand side class is abstract: only instances
of specific constructor nodes are ever created.
Each concrete class has an attribute ``_fields`` which gives the names of all
child nodes.
Each instance of a concrete class has one attribute for each child node, of the
type as defined in the grammar. For example, ``_ast.BinOp`` instances have an
attribute ``left`` of type ``_ast.expr``. Instances of ``_ast.expr`` and
``_ast.stmt`` subclasses also have lineno and col_offset attributes. The lineno
is the line number of source text (1 indexed so the first line is line 1) and
the col_offset is the utf8 byte offset of the first token that generated the
node. The utf8 offset is recorded because the parser uses utf8 internally.
If these attributes are marked as optional in the grammar (using a question
mark), the value might be ``None``. If the attributes can have zero-or-more
values (marked with an asterisk), the values are represented as Python lists.
All possible attributes must be present and have valid values when compiling an
AST with :func:`compile`.
The constructor of a class ``_ast.T`` parses their arguments as follows:
* If there are positional arguments, there must be as many as there are items in
``T._fields``; they will be assigned as attributes of these names.
* If there are keyword arguments, they will set the attributes of the same names
to the given values.
For example, to create and populate a ``UnaryOp`` node, you could use ::
node = _ast.UnaryOp()
node.op = _ast.USub()
node.operand = _ast.Num()
node.operand.n = 5
node.operand.lineno = 0
node.operand.col_offset = 0
node.lineno = 0
node.col_offset = 0
or the more compact ::
node = _ast.UnaryOp(_ast.USub(), _ast.Num(5, lineno=0, col_offset=0),
lineno=0, col_offset=0)
Abstract Grammar
----------------
The module defines a string constant ``__version__`` which is the decimal
Subversion revision number of the file shown below.
The abstract grammar is currently defined as follows:
.. literalinclude:: ../../Parser/Python.asdl

257
Doc/library/ast.rst Normal file
View File

@ -0,0 +1,257 @@
.. _ast:
Abstract Syntax Trees
=====================
.. module:: ast
:synopsis: Abstract Syntax Tree classes and manipulation.
.. sectionauthor:: Martin v. Löwis <martin@v.loewis.de>
.. sectionauthor:: Georg Brandl <georg@python.org>
.. versionadded:: 2.5
The low-level ``_ast`` module containing only the node classes.
.. versionadded:: 2.6
The high-level ``ast`` module containing all helpers.
The :mod:`ast` module helps Python applications to process trees of the Python
abstract syntax grammar. The abstract syntax itself might change with each
Python release; this module helps to find out programmatically what the current
grammar looks like.
An abstract syntax tree can be generated by passing :data:`_ast.PyCF_ONLY_AST`
as a flag to the :func:`compile` builtin function, or using the :func:`parse`
helper provided in this module. The result will be a tree of objects whose
classes all inherit from :class:`ast.AST`.
A modified abstract syntax tree can be compiled into a Python code object using
the built-in :func:`compile` function.
Node classes
------------
.. class:: AST
This is the base of all AST node classes. The actual node classes are
derived from the :file:`Parser/Python.asdl` file, which is reproduced
:ref:`below <abstract-grammar>`. They are defined in the :mod:`_ast` C
module and re-exported in :mod:`ast`.
There is one class defined for each left-hand side symbol in the abstract
grammar (for example, :class:`ast.stmt` or :class:`ast.expr`). In addition,
there is one class defined for each constructor on the right-hand side; these
classes inherit from the classes for the left-hand side trees. For example,
:class:`ast.BinOp` inherits from :class:`ast.expr`. For production rules
with alternatives (aka "sums"), the left-hand side class is abstract: only
instances of specific constructor nodes are ever created.
.. attribute:: _fields
Each concrete class has an attribute :attr:`_fields` which gives the names
of all child nodes.
Each instance of a concrete class has one attribute for each child node,
of the type as defined in the grammar. For example, :class:`ast.BinOp`
instances have an attribute :attr:`left` of type :class:`ast.expr`.
If these attributes are marked as optional in the grammar (using a
question mark), the value might be ``None``. If the attributes can have
zero-or-more values (marked with an asterisk), the values are represented
as Python lists. All possible attributes must be present and have valid
values when compiling an AST with :func:`compile`.
.. attribute:: lineno
col_offset
Instances of :class:`ast.expr` and :class:`ast.stmt` subclasses have
:attr:`lineno` and :attr:`col_offset` attributes. The :attr:`lineno` is
the line number of source text (1-indexed so the first line is line 1) and
the :attr:`col_offset` is the UTF-8 byte offset of the first token that
generated the node. The UTF-8 offset is recorded because the parser uses
UTF-8 internally.
The constructor of a class :class:`ast.T` parses its arguments as follows:
* If there are positional arguments, there must be as many as there are items
in :attr:`T._fields`; they will be assigned as attributes of these names.
* If there are keyword arguments, they will set the attributes of the same
names to the given values.
For example, to create and populate an :class:`ast.UnaryOp` node, you could
use ::
node = ast.UnaryOp()
node.op = ast.USub()
node.operand = ast.Num()
node.operand.n = 5
node.operand.lineno = 0
node.operand.col_offset = 0
node.lineno = 0
node.col_offset = 0
or the more compact ::
node = ast.UnaryOp(ast.USub(), ast.Num(5, lineno=0, col_offset=0),
lineno=0, col_offset=0)
.. _abstract-grammar:
Abstract Grammar
----------------
The module defines a string constant ``__version__`` which is the decimal
Subversion revision number of the file shown below.
The abstract grammar is currently defined as follows:
.. literalinclude:: ../../Parser/Python.asdl
:mod:`ast` Helpers
------------------
.. versionadded:: 2.6
Apart from the node classes, :mod:`ast` module defines these utility functions
and classes for traversing abstract syntax trees:
.. function:: parse(expr, filename='<unknown>', mode='exec')
Parse an expression into an AST node. Equivalent to ``compile(expr,
filename, mode, PyCF_ONLY_AST)``.
.. function:: literal_eval(node_or_string)
Safely evaluate an expression node or a string containing a Python
expression. The string or node provided may only consist of the following
Python literal structures: strings, numbers, tuples, lists, dicts, booleans,
and ``None``.
This can be used for safely evaluating strings containing Python expressions
from untrusted sources without the need to parse the values oneself.
.. function:: get_docstring(node, clean=True):
Return the docstring of the given *node* (which must be a
:class:`FunctionDef`, :class:`ClassDef` or :class:`Module` node), or ``None``
if it has no docstring. If *clean* is true, clean up the docstring's
indentation with :func:`inspect.cleandoc`.
.. function:: fix_missing_locations(node)
When you compile a node tree with :func:`compile`, the compiler expects
:attr:`lineno` and :attr:`col_offset` attributes for every node that supports
them. This is rather tedious to fill in for generated nodes, so this helper
adds these attributes recursively where not already set, by setting them to
the values of the parent node. It works recursively starting at *node*.
.. function:: increment_lineno(node, n=1)
Increment the line number of each node in the tree starting at *node* by *n*.
This is useful to "move code" to a different location in a file.
.. function:: copy_location(new_node, old_node)
Copy source location (:attr:`lineno` and :attr:`col_offset`) from *old_node*
to *new_node* if possible, and return *new_node*.
.. function:: iter_fields(node)
Yield a tuple of ``(fieldname, value)`` for each field in ``node._fields``
that is present on *node*.
.. function:: iter_child_nodes(node)
Yield all direct child nodes of *node*, that is, all fields that are nodes
and all items of fields that are lists of nodes.
.. function:: walk(node)
Recursively yield all child nodes of *node*, in no specified order. This is
useful if you only want to modify nodes in place and don't care about the
context.
.. class:: NodeVisitor()
A node visitor base class that walks the abstract syntax tree and calls a
visitor function for every node found. This function may return a value
which is forwarded by the `visit` method.
This class is meant to be subclassed, with the subclass adding visitor
methods.
.. method:: visit(node)
Visit a node. The default implementation calls the method called
:samp:`self.visit_{classname}` where *classname* is the name of the node
class, or :meth:`generic_visit` if that method doesn't exist.
.. method:: generic_visit(node)
This visitor calls :meth:`visit` on all children of the node.
Note that child nodes of nodes that have a custom visitor method won't be
visited unless the visitor calls :meth:`generic_visit` or visits them
itself.
Don't use the :class:`NodeVisitor` if you want to apply changes to nodes
during traversal. For this a special visitor exists
(:class:`NodeTransformer`) that allows modifications.
.. class:: NodeTransformer()
A :class:`NodeVisitor` subclass that walks the abstract syntax tree and
allows modification of nodes.
The `NodeTransformer` will walk the AST and use the return value of the
visitor methods to replace or remove the old node. If the return value of
the visitor method is ``None``, the node will be removed from its location,
otherwise it is replaced with the return value. The return value may be the
original node in which case no replacement takes place.
Here is an example transformer that rewrites all occurrences of name lookups
(``foo``) to ``data['foo']``::
class RewriteName(NodeTransformer):
def visit_Name(self, node):
return copy_location(Subscript(
value=Name(id='data', ctx=Load()),
slice=Index(value=Str(s=node.id)),
ctx=node.ctx
), node)
Keep in mind that if the node you're operating on has child nodes you must
either transform the child nodes yourself or call the :meth:`generic_visit`
method for the node first.
For nodes that were part of a collection of statements (that applies to all
statement nodes), the visitor may also return a list of nodes rather than
just a single node.
Usually you use the transformer like this::
node = YourTransformer().visit(node)
.. function:: dump(node, annotate_fields=True, include_attributes=False)
Return a formatted dump of the tree in *node*. This is mainly useful for
debugging purposes. The returned string will show the names and the values
for fields. This makes the code impossible to evaluate, so if evaluation is
wanted *annotate_fields* must be set to False. Attributes such as line
numbers and column offsets are dumped by default. If this is wanted,
*include_attributes* can be set to ``True``.

2
Doc/library/language.rst
View File

@ -15,7 +15,7 @@ These modules include:
.. toctree::
parser.rst
_ast.rst
ast.rst
symbol.rst
token.rst
keyword.rst

300
Lib/ast.py Normal file
View File

@ -0,0 +1,300 @@
# -*- coding: utf-8 -*-
"""
ast
~~~
The `ast` module helps Python applications to process trees of the Python
abstract syntax grammar. The abstract syntax itself might change with
each Python release; this module helps to find out programmatically what
the current grammar looks like and allows modifications of it.
An abstract syntax tree can be generated by passing `ast.PyCF_ONLY_AST` as
a flag to the `compile()` builtin function or by using the `parse()`
function from this module. The result will be a tree of objects whose
classes all inherit from `ast.AST`.
A modified abstract syntax tree can be compiled into a Python code object
using the built-in `compile()` function.
Additionally various helper functions are provided that make working with
the trees simpler. The main intention of the helper functions and this
module in general is to provide an easy to use interface for libraries
that work tightly with the python syntax (template engines for example).
:copyright: Copyright 2008 by Armin Ronacher.
:license: Python License.
"""
from _ast import *
def parse(expr, filename='<unknown>', mode='exec'):
"""
Parse an expression into an AST node.
Equivalent to compile(expr, filename, mode, PyCF_ONLY_AST).
"""
return compile(expr, filename, mode, PyCF_ONLY_AST)
def literal_eval(node_or_string):
"""
Safely evaluate an expression node or a string containing a Python
expression. The string or node provided may only consist of the following
Python literal structures: strings, numbers, tuples, lists, dicts, booleans,
and None.
"""
_safe_names = {'None': None, 'True': True, 'False': False}
if isinstance(node_or_string, basestring):
node_or_string = parse(node_or_string, mode='eval')
if isinstance(node_or_string, Expression):
node_or_string = node_or_string.body
def _convert(node):
if isinstance(node, Str):
return node.s
elif isinstance(node, Num):
return node.n
elif isinstance(node, Tuple):
return tuple(map(_convert, node.elts))
elif isinstance(node, List):
return list(map(_convert, node.elts))
elif isinstance(node, Dict):
return dict((_convert(k), _convert(v)) for k, v
in zip(node.keys, node.values))
elif isinstance(node, Name):
if node.id in _safe_names:
return _safe_names[node.id]
raise ValueError('malformed string')
return _convert(node_or_string)
def dump(node, annotate_fields=True, include_attributes=False):
"""
Return a formatted dump of the tree in *node*. This is mainly useful for
debugging purposes. The returned string will show the names and the values
for fields. This makes the code impossible to evaluate, so if evaluation is
wanted *annotate_fields* must be set to False. Attributes such as line
numbers and column offsets are dumped by default. If this is wanted,
*include_attributes* can be set to True.
"""
def _format(node):
if isinstance(node, AST):
fields = [(a, _format(b)) for a, b in iter_fields(node)]
rv = '%s(%s' % (node.__class__.__name__, ', '.join(
('%s=%s' % field for field in fields)
if annotate_fields else
(b for a, b in fields)
))
if include_attributes and node._attributes:
rv += fields and ', ' or ' '
rv += ', '.join('%s=%s' % (a, _format(getattr(node, a)))
for a in node._attributes)
return rv + ')'
elif isinstance(node, list):
return '[%s]' % ', '.join(_format(x) for x in node)
return repr(node)
if not isinstance(node, AST):
raise TypeError('expected AST, got %r' % node.__class__.__name__)
return _format(node)
def copy_location(new_node, old_node):
"""
Copy source location (`lineno` and `col_offset` attributes) from
*old_node* to *new_node* if possible, and return *new_node*.
"""
for attr in 'lineno', 'col_offset':
if attr in old_node._attributes and attr in new_node._attributes \
and hasattr(old_node, attr):
setattr(new_node, attr, getattr(old_node, attr))
return new_node
def fix_missing_locations(node):
"""
When you compile a node tree with compile(), the compiler expects lineno and
col_offset attributes for every node that supports them. This is rather
tedious to fill in for generated nodes, so this helper adds these attributes
recursively where not already set, by setting them to the values of the
parent node. It works recursively starting at *node*.
"""
def _fix(node, lineno, col_offset):
if 'lineno' in node._attributes:
if not hasattr(node, 'lineno'):
node.lineno = lineno
else:
lineno = node.lineno
if 'col_offset' in node._attributes:
if not hasattr(node, 'col_offset'):
node.col_offset = col_offset
else:
col_offset = node.col_offset
for child in iter_child_nodes(node):
_fix(child, lineno, col_offset)
_fix(node, 1, 0)
return node
def increment_lineno(node, n=1):
"""
Increment the line number of each node in the tree starting at *node* by *n*.
This is useful to "move code" to a different location in a file.
"""
if 'lineno' in node._attributes:
node.lineno = getattr(node, 'lineno', 0) + n
for child in walk(node):
if 'lineno' in child._attributes:
child.lineno = getattr(child, 'lineno', 0) + n
return node
def iter_fields(node):
"""
Yield a tuple of ``(fieldname, value)`` for each field in ``node._fields``
that is present on *node*.
"""
for field in node._fields:
try:
yield field, getattr(node, field)
except AttributeError:
pass
def iter_child_nodes(node):
"""
Yield all direct child nodes of *node*, that is, all fields that are nodes
and all items of fields that are lists of nodes.
"""
for name, field in iter_fields(node):
if isinstance(field, AST):
yield field
elif isinstance(field, list):
for item in field:
if isinstance(item, AST):
yield item
def get_docstring(node, clean=True):
"""
Return the docstring for the given node or None if no docstring can
be found. If the node provided does not have docstrings a TypeError
will be raised.
"""
if not isinstance(node, (FunctionDef, ClassDef, Module)):
raise TypeError("%r can't have docstrings" % node.__class__.__name__)
if node.body and isinstance(node.body[0], Expr) and \
isinstance(node.body[0].value, Str):
if clean:
import inspect
return inspect.cleandoc(node.body[0].value.s)
return node.body[0].value.s
def walk(node):
"""
Recursively yield all child nodes of *node*, in no specified order. This is
useful if you only want to modify nodes in place and don't care about the
context.
"""
from collections import deque
todo = deque([node])
while todo:
node = todo.popleft()
todo.extend(iter_child_nodes(node))
yield node
class NodeVisitor(object):
"""
A node visitor base class that walks the abstract syntax tree and calls a
visitor function for every node found. This function may return a value
which is forwarded by the `visit` method.
This class is meant to be subclassed, with the subclass adding visitor
methods.
Per default the visitor functions for the nodes are ``'visit_'`` +
class name of the node. So a `TryFinally` node visit function would
be `visit_TryFinally`. This behavior can be changed by overriding
the `visit` method. If no visitor function exists for a node
(return value `None`) the `generic_visit` visitor is used instead.
Don't use the `NodeVisitor` if you want to apply changes to nodes during
traversing. For this a special visitor exists (`NodeTransformer`) that
allows modifications.
"""
def visit(self, node):
"""Visit a node."""
method = 'visit_' + node.__class__.__name__
visitor = getattr(self, method, self.generic_visit)
return visitor(node)
def generic_visit(self, node):
"""Called if no explicit visitor function exists for a node."""
for field, value in iter_fields(node):
if isinstance(value, list):
for item in value:
if isinstance(item, AST):
self.visit(item)
elif isinstance(value, AST):
self.visit(value)
class NodeTransformer(NodeVisitor):
"""
A :class:`NodeVisitor` subclass that walks the abstract syntax tree and
allows modification of nodes.
The `NodeTransformer` will walk the AST and use the return value of the
visitor methods to replace or remove the old node. If the return value of
the visitor method is ``None``, the node will be removed from its location,
otherwise it is replaced with the return value. The return value may be the
original node in which case no replacement takes place.
Here is an example transformer that rewrites all occurrences of name lookups
(``foo``) to ``data['foo']``::
class RewriteName(NodeTransformer):
def visit_Name(self, node):
return copy_location(Subscript(
value=Name(id='data', ctx=Load()),
slice=Index(value=Str(s=node.id)),
ctx=node.ctx
), node)
Keep in mind that if the node you're operating on has child nodes you must
either transform the child nodes yourself or call the :meth:`generic_visit`
method for the node first.
For nodes that were part of a collection of statements (that applies to all
statement nodes), the visitor may also return a list of nodes rather than
just a single node.
Usually you use the transformer like this::
node = YourTransformer().visit(node)
"""
def generic_visit(self, node):
for field, old_value in iter_fields(node):
old_value = getattr(node, field, None)
if isinstance(old_value, list):
new_values = []
for value in old_value:
if isinstance(value, AST):
value = self.visit(value)
if value is None:
continue
elif not isinstance(value, AST):
new_values.extend(value)
continue
new_values.append(value)
old_value[:] = new_values
elif isinstance(old_value, AST):
new_node = self.visit(old_value)
if new_node is None:
delattr(node, field)
else:
setattr(node, field, new_node)
return node

109
Lib/test/test_ast.py
View File

@ -1,6 +1,6 @@
import sys, itertools, unittest
from test import test_support
import _ast
import ast
def to_tuple(t):
if t is None or isinstance(t, (basestring, int, long, complex)):
@ -123,9 +123,9 @@ eval_tests = [
class AST_Tests(unittest.TestCase):
def _assert_order(self, ast_node, parent_pos):
if not isinstance(ast_node, _ast.AST) or ast_node._fields is None:
if not isinstance(ast_node, ast.AST) or ast_node._fields is None:
return
if isinstance(ast_node, (_ast.expr, _ast.stmt, _ast.excepthandler)):
if isinstance(ast_node, (ast.expr, ast.stmt, ast.excepthandler)):
node_pos = (ast_node.lineno, ast_node.col_offset)
self.assert_(node_pos >= parent_pos)
parent_pos = (ast_node.lineno, ast_node.col_offset)
@ -142,29 +142,29 @@ class AST_Tests(unittest.TestCase):
(single_tests, single_results, "single"),
(eval_tests, eval_results, "eval")):
for i, o in itertools.izip(input, output):
ast_tree = compile(i, "?", kind, _ast.PyCF_ONLY_AST)
ast_tree = compile(i, "?", kind, ast.PyCF_ONLY_AST)
self.assertEquals(to_tuple(ast_tree), o)
self._assert_order(ast_tree, (0, 0))
def test_nodeclasses(self):
x = _ast.BinOp(1, 2, 3, lineno=0)
x = ast.BinOp(1, 2, 3, lineno=0)
self.assertEquals(x.left, 1)
self.assertEquals(x.op, 2)
self.assertEquals(x.right, 3)
self.assertEquals(x.lineno, 0)
# node raises exception when not given enough arguments
self.assertRaises(TypeError, _ast.BinOp, 1, 2)
self.assertRaises(TypeError, ast.BinOp, 1, 2)
# can set attributes through kwargs too
x = _ast.BinOp(left=1, op=2, right=3, lineno=0)
x = ast.BinOp(left=1, op=2, right=3, lineno=0)
self.assertEquals(x.left, 1)
self.assertEquals(x.op, 2)
self.assertEquals(x.right, 3)
self.assertEquals(x.lineno, 0)
# this used to fail because Sub._fields was None
x = _ast.Sub()
x = ast.Sub()
def test_pickling(self):
import pickle
@ -181,8 +181,99 @@ class AST_Tests(unittest.TestCase):
ast2 = mod.loads(mod.dumps(ast, protocol))
self.assertEquals(to_tuple(ast2), to_tuple(ast))
class ASTHelpers_Test(unittest.TestCase):
def test_parse(self):
a = ast.parse('foo(1 + 1)')
b = compile('foo(1 + 1)', '<unknown>', 'exec', ast.PyCF_ONLY_AST)
self.assertEqual(ast.dump(a), ast.dump(b))
def test_dump(self):
node = ast.parse('spam(eggs, "and cheese")')
self.assertEqual(ast.dump(node),
"Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load()), "
"args=[Name(id='eggs', ctx=Load()), Str(s='and cheese')], "
"keywords=[], starargs=None, kwargs=None))])"
)
self.assertEqual(ast.dump(node, annotate_fields=False),
"Module([Expr(Call(Name('spam', Load()), [Name('eggs', Load()), "
"Str('and cheese')], [], None, None))])"
)
self.assertEqual(ast.dump(node, include_attributes=True),
"Module(body=[Expr(value=Call(func=Name(id='spam', ctx=Load(), "
"lineno=1, col_offset=0), args=[Name(id='eggs', ctx=Load(), "
"lineno=1, col_offset=5), Str(s='and cheese', lineno=1, "
"col_offset=11)], keywords=[], starargs=None, kwargs=None, "
"lineno=1, col_offset=0), lineno=1, col_offset=0)])"
)
def test_copy_location(self):
src = ast.parse('1 + 1', mode='eval')
src.body.right = ast.copy_location(ast.Num(2), src.body.right)
self.assertEqual(ast.dump(src, include_attributes=True),
'Expression(body=BinOp(left=Num(n=1, lineno=1, col_offset=0), '
'op=Add(), right=Num(n=2, lineno=1, col_offset=4), lineno=1, '
'col_offset=0))'
)
def test_fix_missing_locations(self):
src = ast.parse('write("spam")')
src.body.append(ast.Expr(ast.Call(ast.Name('spam', ast.Load()),
[ast.Str('eggs')], [], None, None)))
self.assertEqual(src, ast.fix_missing_locations(src))
self.assertEqual(ast.dump(src, include_attributes=True),
"Module(body=[Expr(value=Call(func=Name(id='write', ctx=Load(), "
"lineno=1, col_offset=0), args=[Str(s='spam', lineno=1, "
"col_offset=6)], keywords=[], starargs=None, kwargs=None, "
"lineno=1, col_offset=0), lineno=1, col_offset=0), "
"Expr(value=Call(func=Name(id='spam', ctx=Load(), lineno=1, "
"col_offset=0), args=[Str(s='eggs', lineno=1, col_offset=0)], "
"keywords=[], starargs=None, kwargs=None, lineno=1, "
"col_offset=0), lineno=1, col_offset=0)])"
)
def test_increment_lineno(self):
src = ast.parse('1 + 1', mode='eval')
self.assertEqual(ast.increment_lineno(src, n=3), src)
self.assertEqual(ast.dump(src, include_attributes=True),
'Expression(body=BinOp(left=Num(n=1, lineno=4, col_offset=0), '
'op=Add(), right=Num(n=1, lineno=4, col_offset=4), lineno=4, '
'col_offset=0))'
)
def test_iter_fields(self):
node = ast.parse('foo()', mode='eval')
d = dict(ast.iter_fields(node.body))
self.assertEqual(d.pop('func').id, 'foo')
self.assertEqual(d, {'keywords': [], 'kwargs': None,
'args': [], 'starargs': None})
def test_iter_child_nodes(self):
node = ast.parse("spam(23, 42, eggs='leek')", mode='eval')
self.assertEqual(len(list(ast.iter_child_nodes(node.body))), 4)
iterator = ast.iter_child_nodes(node.body)
self.assertEqual(next(iterator).id, 'spam')
self.assertEqual(next(iterator).n, 23)
self.assertEqual(next(iterator).n, 42)
self.assertEqual(ast.dump(next(iterator)),
"keyword(arg='eggs', value=Str(s='leek'))"
)
def test_get_docstring(self):
node = ast.parse('def foo():\n """line one\n line two"""')
self.assertEqual(ast.get_docstring(node.body[0]),
'line one\nline two')
def test_literal_eval(self):
self.assertEqual(ast.literal_eval('[1, 2, 3]'), [1, 2, 3])
self.assertEqual(ast.literal_eval('{"foo": 42}'), {"foo": 42})
self.assertEqual(ast.literal_eval('(True, False, None)'), (True, False, None))
self.assertRaises(ValueError, ast.literal_eval, 'foo()')
def test_main():
test_support.run_unittest(AST_Tests)
test_support.run_unittest(AST_Tests, ASTHelpers_Test)
def main():
if __name__ != '__main__':

1
Misc/ACKS
View File

@ -574,6 +574,7 @@ Andy Robinson
Kevin Rodgers
Giampaolo Rodola
Mike Romberg
Armin Ronacher
Case Roole
Timothy Roscoe
Jim Roskind

2
Misc/NEWS
View File

@ -77,6 +77,8 @@ Extension Modules
Library
-------
- Added the ast module.
- Factored out the indentation cleaning from inspect.getdoc() into
inspect.cleandoc() to ease standalone use.