traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Use the "howto" document class rather than the "manual" class. 

Remove the module index; there aren't enough modules documented yet
for this to make sense.

Add a couple more index entries, fixed a few typos, and adjusted a few
more things for consistency.
This commit is contained in:
Fred Drake 2001-08-15 17:01:34 +00:00
parent 27413a5eda
commit 834a85a235
1 changed files with 82 additions and 86 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

168
Tools/compiler/doc/compiler.tex
View File

@ -5,7 +5,7 @@
%
% http://www.python.org/doc/current/doc/doc.html
\documentclass{manual}
\documentclass{howto}
\title{Python compiler package}
@ -15,7 +15,7 @@
% the rest is at your discretion.
\authoraddress{
PythonLabs \\
Zope Corp. \\
Zope Corporation \\
Email: \email{jeremy@zope.com}
}
@ -36,13 +36,6 @@
\maketitle
% This makes the contents more accessible from the front page of the HTML.
\ifhtml
\chapter*{Front Matter\label{front}}
\fi
%\input{copyright}
\begin{abstract}
\noindent
@ -55,15 +48,19 @@ generate Python bytecode from the tree.
\tableofcontents
\chapter{Introduction\label{Introduction}}
\section{Introduction\label{Introduction}}
XXX Need basic intro
XXX what are the major advantages... the abstract syntax is much
closer to the python source...
\section{The basic interface}
\declaremodule{}{compiler}
The top-level of the package defines four functions.
\begin{funcdesc}{parse}{buf}
@ -79,20 +76,22 @@ specified by \var{path}. It is equivalent to
\code{parse(open(\var{path}).read())}.
\end{funcdesc}
\begin{funcdesc}{walk}{ast, visitor, \optional{verbose=None}}
\begin{funcdesc}{walk}{ast, visitor\optional{, verbose}}
Do a pre-order walk over the abstract syntax tree \var{ast}. Call the
appropriate method on the \var{visitor} instance for each node
encountered.
encountered.
\end{funcdesc}
\begin{funcdesc}{compile}{filename}
Compile the file \var{filename} and generated \var{filename}.pyc.
\begin{funcdesc}{compile}{path}
Compile the file \var{path} and generate the corresponding \file{.pyc}
file.
\end{funcdesc}
The \module{compiler} package contains the following modules:
\module{ast}, \module{consts}, \module{future}, \module{misc},
\module{pyassem}, \module{pycodegen}, \module{symbols},
\module{transformer}, and \module{visitor}.
\refmodule[compiler.ast]{ast}, \module{consts}, \module{future},
\module{misc}, \module{pyassem}, \module{pycodegen}, \module{symbols},
\module{transformer}, and \refmodule[compiler.visitor]{visitor}.
\section{Limitations}
@ -106,38 +105,43 @@ incomplete. For example, the compiler package does not raise an error
if a name appears more than once in an argument list:
\code{def f(x, x): ...}
\chapter{Python Abstract Syntax}
\section{Introduction}
\section{Python Abstract Syntax}
The \module{compiler.ast} module defines an abstract syntax for
Python. In the abstract syntax tree, each node represents a syntactic
construct. The root of the tree is \class{Module} object.
The abstract syntax offers a higher level interface to parsed Python
source code. The \module{parser} module and the compiler written in C
for the Python interpreter use a concrete syntax tree. The concrete
syntax is tied closely to the grammar description used for the Python
parser. Instead of a single node for a construct, there are often
several levels of nested nodes that are introduced by Python's
precedence rules.
source code. The \ulink{\module{parser}}
{http://www.python.org/doc/current/lib/module-parser.html}
module and the compiler written in C for the Python interpreter use a
concrete syntax tree. The concrete syntax is tied closely to the
grammar description used for the Python parser. Instead of a single
node for a construct, there are often several levels of nested nodes
that are introduced by Python's precedence rules.
The abstract syntax tree is created by the
\module{compiler.transformer} module. The transformer relies on the
builtin Python parser to generate a concrete syntax tree. It
generates an abstract syntax tree from the concrete tree.
The \module{transformer} module was created by Greg Stein and Bill
Tutt for the Python-to-C compiler. The current version contains a
The \module{transformer} module was created by Greg
Stein\index{Stein, Greg} and Bill Tutt\index{Tutt, Bill} for an
experimental Python-to-C compiler. The current version contains a
number of modifications and improvements, but the basic form of the
abstract syntax and of the transformer are due to Stein and Tutt.
\section{AST Nodes}
The \module{ast} module is generated from a text file that describes
each node type and its elements. Each node type is represented as a
class that inherits from the abstract base class \class{ast.Node} and
defines a set of named attributes for child nodes.
\declaremodule{}{compiler.ast}
The \module{compiler.ast} module is generated from a text file that
describes each node type and its elements. Each node type is
represented as a class that inherits from the abstract base class
\class{compiler.ast.Node} and defines a set of named attributes for
child nodes.
\begin{classdesc}{Node}{}
@ -153,26 +157,27 @@ defines a set of named attributes for child nodes.
Each \class{Node} instance has a \member{lineno} attribute which may
be \code{None}. XXX Not sure what the rules are for which nodes
will have a useful lineno.
\begin{methoddesc}{getChildren}{}
Returns a flattened list of the child nodes and objects in the
order they occur. Specifically, the order of the nodes is the
order in which they appear in the Python grammar. Not all of the
children are \class{Node} instances. The names of functions and
classes, for example, are plain strings.
\end{methoddesc}
\begin{methoddesc}{getChildNodes}{}
Returns a flattened list of the child nodes in the order they
occur. This method is like \method{getChildNodes()}, except that it
only returns those children that are \class{Node} instances.
\end{methoddesc}
\end{classdesc}
All \class{Node} objects offer the following methods:
\begin{methoddesc}{getChildren}{}
Returns a flattened list of the child nodes and objects in the
order they occur. Specifically, the order of the nodes is the
order in which they appear in the Python grammar. Not all of the
children are \class{Node} instances. The names of functions and
classes, for example, are plain strings.
\end{methoddesc}
\begin{methoddesc}{getChildNodes}{}
Returns a flattened list of the child nodes in the order they
occur. This method is like \method{getChildren()}, except that it
only returns those children that are \class{Node} instances.
\end{methoddesc}
Two examples illustrate the general structure of \class{Node}
classes. The while statement is defined by the following grammar
production:
classes. The \keyword{while} statement is defined by the following
grammar production:
\begin{verbatim}
while_stmt: "while" expression ":" suite
@ -182,11 +187,11 @@ while_stmt: "while" expression ":" suite
The \class{While} node has three attributes: \member{test},
\member{body}, and \member{else_}. (If the natural name for an
attribute is also a Python reserved word, it can't be used as an
attribute name. An underscore is appended to the word to make it
legal, hence \code{else_} instead of \code{else}.)
attribute name. An underscore is appended to the word to make it a
legal identifier, hence \member{else_} instead of \keyword{else}.)
The if statement is more complicated because it can include several
tests.
The \keyword{if} statement is more complicated because it can include
several tests.
\begin{verbatim}
if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
@ -194,16 +199,16 @@ if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
The \class{If} node only defines two attributes: \member{tests} and
\member{else_}. The \member{tests} attribute is a sequence of test
expression, consequent body pairs. There is one pair of each if/elif
clause. The first element of the pair is the test expression. The
second elements is a \class{Stmt} node that contains the code to
execute if the test is true.
expression, consequent body pairs. There is one pair for each
\keyword{if}/\keyword{elif} clause. The first element of the pair is
the test expression. The second elements is a \class{Stmt} node that
contains the code to execute if the test is true.
The \method{getChildren()} method of \class{If} returns a flat list of
child nodes. If there are three if/elif clauses and no else clause,
then \method{getChildren()} will return a list of six elements: the
first test expression, the first \class{Stmt}, the second text
expression, etc.
child nodes. If there are three \keyword{if}/\keyword{elif} clauses
and no \keyword{else} clause, then \method{getChildren()} will return
a list of six elements: the first test expression, the first
\class{Stmt}, the second text expression, etc.
The following table lists each of the \class{Node} subclasses defined
in \module{compiler.ast} and each of the public attributes available
@ -215,6 +220,7 @@ returned by \method{getChildren()} and \method{getChildNodes()}.
\input{asttable}
\section{Assignment nodes}
There is a collection of nodes used to represent assignments. Each
@ -229,9 +235,12 @@ Each \class{Node} in the list will be one of the following classes:
XXX Explain what the AssXXX nodes are for. Mention \code{a.b.c = 2}
as an example. Explain what the flags are for.
\chapter{Using Visitors to Walk ASTs}
The visitor pattern is ... The \module{compiler} package uses a
\section{Using Visitors to Walk ASTs}
\declaremodule{}{compiler.visitor}
The visitor pattern is ... The \refmodule{compiler} package uses a
variant on the visitor pattern that takes advantage of Python's
introspection features to elminiate the need for much of the visitor's
infrastructure.
@ -243,6 +252,9 @@ rest.
XXX The magic \method{visit()} method for visitors.
\begin{funcdesc}{walk}{tree, visitor\optional{, verbose}}
\end{funcdesc}
\begin{classdesc}{ASTVisitor}{}
The \class{ASTVisitor} is responsible for walking over the tree in the
@ -259,48 +271,32 @@ the visitor argument by adding a visit method to the visitor; this
method can be used to visit a particular child node. If no visitor is
found for a particular node type, the \method{default()} method is
called.
\end{classdesc}
\class{ASTVisitor} objects have the following methods:
XXX describe extra arguments
\begin{methoddesc}{default}{node\optional{, *args}}
\begin{methoddesc}{default}{node\optional{, \moreargs}}
\end{methoddesc}
\begin{methoddesc}{dispatch}{node\optional{, *args}}
\begin{methoddesc}{dispatch}{node\optional{, \moreargs}}
\end{methoddesc}
\begin{methoddesc}{preorder}{tree, visitor}
\end{methoddesc}
\end{classdesc}
\begin{funcdesc}{walk}{tree, visitor\optional{, verbose=None}}
\end{funcdesc}
\section{Bytecode Generation}
\chapter{Bytecode Generation}
The code generator is a visit that emits bytecodes. Each visit method
The code generator is a visitor that emits bytecodes. Each visit method
can call the \method{emit()} method to emit a new bytecode. The basic
code generator is specialized for modules, classes, and functions. An
assembler converts that emitted instructions to the low-level bytecode
format. It handles things like generator of constant lists of code
objects and calculation of jump offsets.
%
% The ugly "%begin{latexonly}" pseudo-environments are really just to
% keep LaTeX2HTML quiet during the \renewcommand{} macros; they're
% not really valuable.
%
% If you don't want the Module Index, you can remove all of this up
% until the second \input line.
%
%begin{latexonly}
\renewcommand{\indexname}{Module Index}
%end{latexonly}
\input{mod\jobname.ind} % Module Index
%begin{latexonly}
\renewcommand{\indexname}{Index}
%end{latexonly}
\input{\jobname.ind} % Index
\input{compiler.ind} % Index
\end{document}