Use the "howto" document class rather than the "manual" class.
Remove the module index; there aren't enough modules documented yet for this to make sense. Add a couple more index entries, fixed a few typos, and adjusted a few more things for consistency.
This commit is contained in:
parent
27413a5eda
commit
834a85a235
|
@ -5,7 +5,7 @@
|
||||||
%
|
%
|
||||||
% http://www.python.org/doc/current/doc/doc.html
|
% http://www.python.org/doc/current/doc/doc.html
|
||||||
|
|
||||||
\documentclass{manual}
|
\documentclass{howto}
|
||||||
|
|
||||||
\title{Python compiler package}
|
\title{Python compiler package}
|
||||||
|
|
||||||
|
@ -15,7 +15,7 @@
|
||||||
% the rest is at your discretion.
|
% the rest is at your discretion.
|
||||||
\authoraddress{
|
\authoraddress{
|
||||||
PythonLabs \\
|
PythonLabs \\
|
||||||
Zope Corp. \\
|
Zope Corporation \\
|
||||||
Email: \email{jeremy@zope.com}
|
Email: \email{jeremy@zope.com}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -36,13 +36,6 @@
|
||||||
|
|
||||||
\maketitle
|
\maketitle
|
||||||
|
|
||||||
% This makes the contents more accessible from the front page of the HTML.
|
|
||||||
\ifhtml
|
|
||||||
\chapter*{Front Matter\label{front}}
|
|
||||||
\fi
|
|
||||||
|
|
||||||
%\input{copyright}
|
|
||||||
|
|
||||||
\begin{abstract}
|
\begin{abstract}
|
||||||
|
|
||||||
\noindent
|
\noindent
|
||||||
|
@ -55,15 +48,19 @@ generate Python bytecode from the tree.
|
||||||
|
|
||||||
\tableofcontents
|
\tableofcontents
|
||||||
|
|
||||||
\chapter{Introduction\label{Introduction}}
|
|
||||||
|
\section{Introduction\label{Introduction}}
|
||||||
|
|
||||||
XXX Need basic intro
|
XXX Need basic intro
|
||||||
|
|
||||||
XXX what are the major advantages... the abstract syntax is much
|
XXX what are the major advantages... the abstract syntax is much
|
||||||
closer to the python source...
|
closer to the python source...
|
||||||
|
|
||||||
|
|
||||||
\section{The basic interface}
|
\section{The basic interface}
|
||||||
|
|
||||||
|
\declaremodule{}{compiler}
|
||||||
|
|
||||||
The top-level of the package defines four functions.
|
The top-level of the package defines four functions.
|
||||||
|
|
||||||
\begin{funcdesc}{parse}{buf}
|
\begin{funcdesc}{parse}{buf}
|
||||||
|
@ -79,20 +76,22 @@ specified by \var{path}. It is equivalent to
|
||||||
\code{parse(open(\var{path}).read())}.
|
\code{parse(open(\var{path}).read())}.
|
||||||
\end{funcdesc}
|
\end{funcdesc}
|
||||||
|
|
||||||
\begin{funcdesc}{walk}{ast, visitor, \optional{verbose=None}}
|
\begin{funcdesc}{walk}{ast, visitor\optional{, verbose}}
|
||||||
Do a pre-order walk over the abstract syntax tree \var{ast}. Call the
|
Do a pre-order walk over the abstract syntax tree \var{ast}. Call the
|
||||||
appropriate method on the \var{visitor} instance for each node
|
appropriate method on the \var{visitor} instance for each node
|
||||||
encountered.
|
encountered.
|
||||||
\end{funcdesc}
|
\end{funcdesc}
|
||||||
|
|
||||||
\begin{funcdesc}{compile}{filename}
|
\begin{funcdesc}{compile}{path}
|
||||||
Compile the file \var{filename} and generated \var{filename}.pyc.
|
Compile the file \var{path} and generate the corresponding \file{.pyc}
|
||||||
|
file.
|
||||||
\end{funcdesc}
|
\end{funcdesc}
|
||||||
|
|
||||||
The \module{compiler} package contains the following modules:
|
The \module{compiler} package contains the following modules:
|
||||||
\module{ast}, \module{consts}, \module{future}, \module{misc},
|
\refmodule[compiler.ast]{ast}, \module{consts}, \module{future},
|
||||||
\module{pyassem}, \module{pycodegen}, \module{symbols},
|
\module{misc}, \module{pyassem}, \module{pycodegen}, \module{symbols},
|
||||||
\module{transformer}, and \module{visitor}.
|
\module{transformer}, and \refmodule[compiler.visitor]{visitor}.
|
||||||
|
|
||||||
|
|
||||||
\section{Limitations}
|
\section{Limitations}
|
||||||
|
|
||||||
|
@ -106,38 +105,43 @@ incomplete. For example, the compiler package does not raise an error
|
||||||
if a name appears more than once in an argument list:
|
if a name appears more than once in an argument list:
|
||||||
\code{def f(x, x): ...}
|
\code{def f(x, x): ...}
|
||||||
|
|
||||||
\chapter{Python Abstract Syntax}
|
|
||||||
|
|
||||||
\section{Introduction}
|
\section{Python Abstract Syntax}
|
||||||
|
|
||||||
The \module{compiler.ast} module defines an abstract syntax for
|
The \module{compiler.ast} module defines an abstract syntax for
|
||||||
Python. In the abstract syntax tree, each node represents a syntactic
|
Python. In the abstract syntax tree, each node represents a syntactic
|
||||||
construct. The root of the tree is \class{Module} object.
|
construct. The root of the tree is \class{Module} object.
|
||||||
|
|
||||||
The abstract syntax offers a higher level interface to parsed Python
|
The abstract syntax offers a higher level interface to parsed Python
|
||||||
source code. The \module{parser} module and the compiler written in C
|
source code. The \ulink{\module{parser}}
|
||||||
for the Python interpreter use a concrete syntax tree. The concrete
|
{http://www.python.org/doc/current/lib/module-parser.html}
|
||||||
syntax is tied closely to the grammar description used for the Python
|
module and the compiler written in C for the Python interpreter use a
|
||||||
parser. Instead of a single node for a construct, there are often
|
concrete syntax tree. The concrete syntax is tied closely to the
|
||||||
several levels of nested nodes that are introduced by Python's
|
grammar description used for the Python parser. Instead of a single
|
||||||
precedence rules.
|
node for a construct, there are often several levels of nested nodes
|
||||||
|
that are introduced by Python's precedence rules.
|
||||||
|
|
||||||
The abstract syntax tree is created by the
|
The abstract syntax tree is created by the
|
||||||
\module{compiler.transformer} module. The transformer relies on the
|
\module{compiler.transformer} module. The transformer relies on the
|
||||||
builtin Python parser to generate a concrete syntax tree. It
|
builtin Python parser to generate a concrete syntax tree. It
|
||||||
generates an abstract syntax tree from the concrete tree.
|
generates an abstract syntax tree from the concrete tree.
|
||||||
|
|
||||||
The \module{transformer} module was created by Greg Stein and Bill
|
The \module{transformer} module was created by Greg
|
||||||
Tutt for the Python-to-C compiler. The current version contains a
|
Stein\index{Stein, Greg} and Bill Tutt\index{Tutt, Bill} for an
|
||||||
|
experimental Python-to-C compiler. The current version contains a
|
||||||
number of modifications and improvements, but the basic form of the
|
number of modifications and improvements, but the basic form of the
|
||||||
abstract syntax and of the transformer are due to Stein and Tutt.
|
abstract syntax and of the transformer are due to Stein and Tutt.
|
||||||
|
|
||||||
|
|
||||||
\section{AST Nodes}
|
\section{AST Nodes}
|
||||||
|
|
||||||
The \module{ast} module is generated from a text file that describes
|
\declaremodule{}{compiler.ast}
|
||||||
each node type and its elements. Each node type is represented as a
|
|
||||||
class that inherits from the abstract base class \class{ast.Node} and
|
The \module{compiler.ast} module is generated from a text file that
|
||||||
defines a set of named attributes for child nodes.
|
describes each node type and its elements. Each node type is
|
||||||
|
represented as a class that inherits from the abstract base class
|
||||||
|
\class{compiler.ast.Node} and defines a set of named attributes for
|
||||||
|
child nodes.
|
||||||
|
|
||||||
\begin{classdesc}{Node}{}
|
\begin{classdesc}{Node}{}
|
||||||
|
|
||||||
|
@ -153,26 +157,27 @@ defines a set of named attributes for child nodes.
|
||||||
Each \class{Node} instance has a \member{lineno} attribute which may
|
Each \class{Node} instance has a \member{lineno} attribute which may
|
||||||
be \code{None}. XXX Not sure what the rules are for which nodes
|
be \code{None}. XXX Not sure what the rules are for which nodes
|
||||||
will have a useful lineno.
|
will have a useful lineno.
|
||||||
|
|
||||||
\begin{methoddesc}{getChildren}{}
|
|
||||||
Returns a flattened list of the child nodes and objects in the
|
|
||||||
order they occur. Specifically, the order of the nodes is the
|
|
||||||
order in which they appear in the Python grammar. Not all of the
|
|
||||||
children are \class{Node} instances. The names of functions and
|
|
||||||
classes, for example, are plain strings.
|
|
||||||
\end{methoddesc}
|
|
||||||
|
|
||||||
\begin{methoddesc}{getChildNodes}{}
|
|
||||||
Returns a flattened list of the child nodes in the order they
|
|
||||||
occur. This method is like \method{getChildNodes()}, except that it
|
|
||||||
only returns those children that are \class{Node} instances.
|
|
||||||
\end{methoddesc}
|
|
||||||
|
|
||||||
\end{classdesc}
|
\end{classdesc}
|
||||||
|
|
||||||
|
All \class{Node} objects offer the following methods:
|
||||||
|
|
||||||
|
\begin{methoddesc}{getChildren}{}
|
||||||
|
Returns a flattened list of the child nodes and objects in the
|
||||||
|
order they occur. Specifically, the order of the nodes is the
|
||||||
|
order in which they appear in the Python grammar. Not all of the
|
||||||
|
children are \class{Node} instances. The names of functions and
|
||||||
|
classes, for example, are plain strings.
|
||||||
|
\end{methoddesc}
|
||||||
|
|
||||||
|
\begin{methoddesc}{getChildNodes}{}
|
||||||
|
Returns a flattened list of the child nodes in the order they
|
||||||
|
occur. This method is like \method{getChildren()}, except that it
|
||||||
|
only returns those children that are \class{Node} instances.
|
||||||
|
\end{methoddesc}
|
||||||
|
|
||||||
Two examples illustrate the general structure of \class{Node}
|
Two examples illustrate the general structure of \class{Node}
|
||||||
classes. The while statement is defined by the following grammar
|
classes. The \keyword{while} statement is defined by the following
|
||||||
production:
|
grammar production:
|
||||||
|
|
||||||
\begin{verbatim}
|
\begin{verbatim}
|
||||||
while_stmt: "while" expression ":" suite
|
while_stmt: "while" expression ":" suite
|
||||||
|
@ -182,11 +187,11 @@ while_stmt: "while" expression ":" suite
|
||||||
The \class{While} node has three attributes: \member{test},
|
The \class{While} node has three attributes: \member{test},
|
||||||
\member{body}, and \member{else_}. (If the natural name for an
|
\member{body}, and \member{else_}. (If the natural name for an
|
||||||
attribute is also a Python reserved word, it can't be used as an
|
attribute is also a Python reserved word, it can't be used as an
|
||||||
attribute name. An underscore is appended to the word to make it
|
attribute name. An underscore is appended to the word to make it a
|
||||||
legal, hence \code{else_} instead of \code{else}.)
|
legal identifier, hence \member{else_} instead of \keyword{else}.)
|
||||||
|
|
||||||
The if statement is more complicated because it can include several
|
The \keyword{if} statement is more complicated because it can include
|
||||||
tests.
|
several tests.
|
||||||
|
|
||||||
\begin{verbatim}
|
\begin{verbatim}
|
||||||
if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
|
if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
|
||||||
|
@ -194,16 +199,16 @@ if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
|
||||||
|
|
||||||
The \class{If} node only defines two attributes: \member{tests} and
|
The \class{If} node only defines two attributes: \member{tests} and
|
||||||
\member{else_}. The \member{tests} attribute is a sequence of test
|
\member{else_}. The \member{tests} attribute is a sequence of test
|
||||||
expression, consequent body pairs. There is one pair of each if/elif
|
expression, consequent body pairs. There is one pair for each
|
||||||
clause. The first element of the pair is the test expression. The
|
\keyword{if}/\keyword{elif} clause. The first element of the pair is
|
||||||
second elements is a \class{Stmt} node that contains the code to
|
the test expression. The second elements is a \class{Stmt} node that
|
||||||
execute if the test is true.
|
contains the code to execute if the test is true.
|
||||||
|
|
||||||
The \method{getChildren()} method of \class{If} returns a flat list of
|
The \method{getChildren()} method of \class{If} returns a flat list of
|
||||||
child nodes. If there are three if/elif clauses and no else clause,
|
child nodes. If there are three \keyword{if}/\keyword{elif} clauses
|
||||||
then \method{getChildren()} will return a list of six elements: the
|
and no \keyword{else} clause, then \method{getChildren()} will return
|
||||||
first test expression, the first \class{Stmt}, the second text
|
a list of six elements: the first test expression, the first
|
||||||
expression, etc.
|
\class{Stmt}, the second text expression, etc.
|
||||||
|
|
||||||
The following table lists each of the \class{Node} subclasses defined
|
The following table lists each of the \class{Node} subclasses defined
|
||||||
in \module{compiler.ast} and each of the public attributes available
|
in \module{compiler.ast} and each of the public attributes available
|
||||||
|
@ -215,6 +220,7 @@ returned by \method{getChildren()} and \method{getChildNodes()}.
|
||||||
|
|
||||||
\input{asttable}
|
\input{asttable}
|
||||||
|
|
||||||
|
|
||||||
\section{Assignment nodes}
|
\section{Assignment nodes}
|
||||||
|
|
||||||
There is a collection of nodes used to represent assignments. Each
|
There is a collection of nodes used to represent assignments. Each
|
||||||
|
@ -229,9 +235,12 @@ Each \class{Node} in the list will be one of the following classes:
|
||||||
XXX Explain what the AssXXX nodes are for. Mention \code{a.b.c = 2}
|
XXX Explain what the AssXXX nodes are for. Mention \code{a.b.c = 2}
|
||||||
as an example. Explain what the flags are for.
|
as an example. Explain what the flags are for.
|
||||||
|
|
||||||
\chapter{Using Visitors to Walk ASTs}
|
|
||||||
|
|
||||||
The visitor pattern is ... The \module{compiler} package uses a
|
\section{Using Visitors to Walk ASTs}
|
||||||
|
|
||||||
|
\declaremodule{}{compiler.visitor}
|
||||||
|
|
||||||
|
The visitor pattern is ... The \refmodule{compiler} package uses a
|
||||||
variant on the visitor pattern that takes advantage of Python's
|
variant on the visitor pattern that takes advantage of Python's
|
||||||
introspection features to elminiate the need for much of the visitor's
|
introspection features to elminiate the need for much of the visitor's
|
||||||
infrastructure.
|
infrastructure.
|
||||||
|
@ -243,6 +252,9 @@ rest.
|
||||||
|
|
||||||
XXX The magic \method{visit()} method for visitors.
|
XXX The magic \method{visit()} method for visitors.
|
||||||
|
|
||||||
|
\begin{funcdesc}{walk}{tree, visitor\optional{, verbose}}
|
||||||
|
\end{funcdesc}
|
||||||
|
|
||||||
\begin{classdesc}{ASTVisitor}{}
|
\begin{classdesc}{ASTVisitor}{}
|
||||||
|
|
||||||
The \class{ASTVisitor} is responsible for walking over the tree in the
|
The \class{ASTVisitor} is responsible for walking over the tree in the
|
||||||
|
@ -259,48 +271,32 @@ the visitor argument by adding a visit method to the visitor; this
|
||||||
method can be used to visit a particular child node. If no visitor is
|
method can be used to visit a particular child node. If no visitor is
|
||||||
found for a particular node type, the \method{default()} method is
|
found for a particular node type, the \method{default()} method is
|
||||||
called.
|
called.
|
||||||
|
\end{classdesc}
|
||||||
|
|
||||||
|
\class{ASTVisitor} objects have the following methods:
|
||||||
|
|
||||||
XXX describe extra arguments
|
XXX describe extra arguments
|
||||||
|
|
||||||
\begin{methoddesc}{default}{node\optional{, *args}}
|
\begin{methoddesc}{default}{node\optional{, \moreargs}}
|
||||||
\end{methoddesc}
|
\end{methoddesc}
|
||||||
|
|
||||||
\begin{methoddesc}{dispatch}{node\optional{, *args}}
|
\begin{methoddesc}{dispatch}{node\optional{, \moreargs}}
|
||||||
\end{methoddesc}
|
\end{methoddesc}
|
||||||
|
|
||||||
\begin{methoddesc}{preorder}{tree, visitor}
|
\begin{methoddesc}{preorder}{tree, visitor}
|
||||||
\end{methoddesc}
|
\end{methoddesc}
|
||||||
|
|
||||||
\end{classdesc}
|
|
||||||
|
|
||||||
\begin{funcdesc}{walk}{tree, visitor\optional{, verbose=None}}
|
\section{Bytecode Generation}
|
||||||
\end{funcdesc}
|
|
||||||
|
|
||||||
\chapter{Bytecode Generation}
|
The code generator is a visitor that emits bytecodes. Each visit method
|
||||||
|
|
||||||
The code generator is a visit that emits bytecodes. Each visit method
|
|
||||||
can call the \method{emit()} method to emit a new bytecode. The basic
|
can call the \method{emit()} method to emit a new bytecode. The basic
|
||||||
code generator is specialized for modules, classes, and functions. An
|
code generator is specialized for modules, classes, and functions. An
|
||||||
assembler converts that emitted instructions to the low-level bytecode
|
assembler converts that emitted instructions to the low-level bytecode
|
||||||
format. It handles things like generator of constant lists of code
|
format. It handles things like generator of constant lists of code
|
||||||
objects and calculation of jump offsets.
|
objects and calculation of jump offsets.
|
||||||
|
|
||||||
%
|
|
||||||
% The ugly "%begin{latexonly}" pseudo-environments are really just to
|
|
||||||
% keep LaTeX2HTML quiet during the \renewcommand{} macros; they're
|
|
||||||
% not really valuable.
|
|
||||||
%
|
|
||||||
% If you don't want the Module Index, you can remove all of this up
|
|
||||||
% until the second \input line.
|
|
||||||
%
|
|
||||||
%begin{latexonly}
|
|
||||||
\renewcommand{\indexname}{Module Index}
|
|
||||||
%end{latexonly}
|
|
||||||
\input{mod\jobname.ind} % Module Index
|
|
||||||
|
|
||||||
%begin{latexonly}
|
\input{compiler.ind} % Index
|
||||||
\renewcommand{\indexname}{Index}
|
|
||||||
%end{latexonly}
|
|
||||||
\input{\jobname.ind} % Index
|
|
||||||
|
|
||||||
\end{document}
|
\end{document}
|
||||||
|
|
Loading…
Reference in New Issue