traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

* ext.tex: did most of the tedious changes from plain ASCII text to LaTeX. 

* text2latex.py: automatically put function names in \code{}.
* lib.tex, ref.tex, ext.tex, qua.tex, tut.tex: use new P.O.Box number in
  address.
This commit is contained in:
Guido van Rossum 1993-11-05 17:11:16 +00:00
parent 7a2dba2a00
commit db65a6ce55
11 changed files with 592 additions and 518 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

534
Doc/ext.tex
View File

@ -1,11 +1,11 @@
\documentstyle[twoside,11pt,myformat]{report}
\documentstyle[twoside,11pt,myformat,times]{report}
\title{\bf Extending and Embedding the Python Interpreter}
\author{
Guido van Rossum \\
Dept. CST, CWI, Kruislaan 413 \\
1098 SJ Amsterdam, The Netherlands \\
Dept. CST, CWI, P.O. Box 94079 \\
1090 GB Amsterdam, The Netherlands \\
E-mail: {\tt guido@cwi.nl}
}
@ -39,12 +39,13 @@ interpreter as a library package from applications using Python as an
\pagenumbering{arabic}
\chapter{Extending Python with C or C++ code}
It is quite easy to add non-standard built-in modules to Python, if
you know how to program in C. A built-in module known to the Python
programmer as foo is generally implemented in a file called
foomodule.c. The standard built-in modules also adhere to this
programmer as \code{foo} is generally implemented in a file called
\file{foomodule.c}. The standard built-in modules also adhere to this
convention, and in fact some of them form excellent examples of how to
create an extension.
@ -53,11 +54,12 @@ Python: implement new data types and provide access to system calls or
C library functions. Since the latter is usually the most important
reason for adding an extension, I'll concentrate on adding "wrappers"
around C library functions; the concrete example uses the wrapper for
system() in module posix, found in (of course) the file posixmodule.c.
\code{system()} in module posix, found in (of course) the file
posixmodule.c.
It is important not to be impressed by the size and complexity of
the average extension module; much of this is straightforward
"boilerplate" code (starting right with the copyright notice!).
``boilerplate'' code (starting right with the copyright notice!).
Let's skip the boilerplate and jump right to an interesting function:
@ -100,20 +102,20 @@ otherwise the Python user could cause a core dump by passing the wrong
arguments (or no arguments at all). Because argument checking and
converting arguments to C is such a common task, there's a general
function in the Python interpreter which combines these tasks:
getargs(). It uses a template string to determine both the types of
the Python argument and the types of the C variables into which it
should store the converted values.
\code{getargs()}. It uses a template string to determine both the
types of the Python argument and the types of the C variables into
which it should store the converted values.
When getargs returns nonzero, the argument list has the right type and
its components have been stored in the variables whose addresses are
passed. When it returns zero, an error has occurred. In the latter
case it has already raised an appropriate exception by calling
err_setstr(), so the calling function can just return NULL.
\code{err_setstr()}, so the calling function can just return NULL.
The form of the format string is described at the end of this file.
(There are convenience macros getstrarg(), getintarg(), etc., for many
common forms of argument lists. These are relics from the past; it's
better to call getargs() directly.)
(There are convenience macros \code{getstrarg()}, \code{getintarg()},
etc., for many common forms of argument lists. These are relics from
the past; it's better to call \code{getargs()} directly.)
\section{Intermezzo: errors and exceptions}
@ -122,56 +124,58 @@ An important convention throughout the Python interpreter is the
following: when a function fails, it should set an exception condition
and return an error value (often a NULL pointer). Exceptions are set
in a global variable in the file errors.c; if this variable is NULL no
exception has occurred. A second variable is the "associated value"
exception has occurred. A second variable is the ``associated value''
of the exception.
The file errors.h declares a host of err_* functions to set various
types of exceptions. The most common one is err_setstr() -- its
arguments are an exception object (e.g. RuntimeError -- actually it
types of exceptions. The most common one is \code{err_setstr()} --- its
arguments are an exception object (e.g. RuntimeError --- actually it
can be any string object) and a C string indicating the cause of the
error (this is converted to a string object and stored as the
"associated value" of the exception). Another useful function is
err_errno(), which only takes an exception argument and constructs the
associated value by inspection of the (UNIX) global variable errno.
``associated value'' of the exception). Another useful function is
\code{err_errno()}, which only takes an exception argument and
constructs the associated value by inspection of the (UNIX) global
variable errno.
You can test non-destructively whether an exception has been set with
err_occurred(). However, most code never calls err_occurred() to see
whether an error occurred or not, but relies on error return values
from the functions it calls instead:
\code{err_occurred()}. However, most code never calls
\code{err_occurred()} to see whether an error occurred or not, but
relies on error return values from the functions it calls instead:
When a function that calls another function detects that the called
function fails, it should return an error value but not set an
condition -- one is already set. The caller is then supposed to also
condition --- one is already set. The caller is then supposed to also
return an error indication to *its* caller, again *without* calling
err_setstr(), and so on -- the most detailed cause of the error was
already reported by the function that detected it in the first place.
Once the error has reached Python's interpreter main loop, this aborts
the currently executing Python code and tries to find an exception
handler specified by the Python programmer.
\code{err_setstr()}, and so on --- the most detailed cause of the error
was already reported by the function that detected it in the first
place. Once the error has reached Python's interpreter main loop,
this aborts the currently executing Python code and tries to find an
exception handler specified by the Python programmer.
To ignore an exception set by a function call that failed, the
exception condition must be cleared explicitly by calling err_clear().
The only time C code should call err_clear() is if it doesn't want to
pass the error on to the interpreter but wants to handle it completely
by itself (e.g. by trying something else or pretending nothing
happened).
exception condition must be cleared explicitly by calling
\code{err_clear()}. The only time C code should call
\code{err_clear()} is if it doesn't want to pass the error on to the
interpreter but wants to handle it completely by itself (e.g. by
trying something else or pretending nothing happened).
Finally, the function err_get() gives you both error variables
Finally, the function \code{err_get()} gives you both error variables
*and clears them*. Note that even if an error occurred the second one
may be NULL. I doubt you will need to use this function.
Note that a failing malloc() call must also be turned into an
exception -- the direct caller of malloc() (or realloc()) must call
err_nomem() and return a failure indicator itself. All the
object-creating functions (newintobject() etc.) already do this, so
only if you call malloc() directly this note is of importance.
Note that a failing \code{malloc()} call must also be turned into an
exception --- the direct caller of \code{malloc()} (or
\code{realloc()}) must call \code{err_nomem()} and return a failure
indicator itself. All the object-creating functions
(\code{newintobject()} etc.) already do this, so only if you call
\code{malloc()} directly this note is of importance.
Also note that, with the important exception of getargs(), functions
Also note that, with the important exception of \code{getargs()}, functions
that return an integer status usually use 0 for success and -1 for
failure.
Finally, be careful about cleaning up garbage (making appropriate
[X]DECREF() calls) when you return an error!
[\code{X}]\code{DECREF()} calls) when you return an error!
\section{Back to the example}
@ -186,7 +190,7 @@ bit:
It returns NULL (the error indicator for functions of this kind) if an
error is detected in the argument list, relying on the exception set
by getargs(). The string value of the argument is now copied to the
by \code{getargs()}. The string value of the argument is now copied to the
local variable 'command'.
If a Python function is called with multiple arguments, the argument
@ -195,18 +199,18 @@ instance, to explicitly create the tuple containing the arguments
first and make the call later.
The next statement in posix_system is a call tothe C library function
system(), passing it the string we just got from getargs():
\code{system()}, passing it the string we just got from \code{getargs()}:
\begin{verbatim}
sts = system(command);
\end{verbatim}
Python strings may contain internal null bytes; but if these occur in
this example the rest of the string will be ignored by system().
this example the rest of the string will be ignored by \code{system()}.
Finally, posix.system() must return a value: the integer status
returned by the C library system() function. This is done by the
function newintobject(), which takes a (long) integer as parameter.
Finally, posix.\code{system()} must return a value: the integer status
returned by the C library \code{system()} function. This is done by the
function \code{newintobject()}, which takes a (long) integer as parameter.
\begin{verbatim}
return newintobject((long)sts);
@ -223,13 +227,13 @@ this idiom:
'None' is a unique Python object representing 'no value'. It differs
from NULL, which means 'error' in most contexts (except when passed as
a function argument -- there it means 'no arguments').
a function argument --- there it means 'no arguments').
\section{The module's function table}
I promised to show how I made the function posix_system() available to
Python programs. This is shown later in posixmodule.c:
I promised to show how I made the function \code{posix_system()}
available to Python programs. This is shown later in posixmodule.c:
\begin{verbatim}
static struct methodlist posix_methods[] = {
@ -246,27 +250,27 @@ Python programs. This is shown later in posixmodule.c:
}
\end{verbatim}
(The actual initposix() is somewhat more complicated, but most
(The actual \code{initposix()} is somewhat more complicated, but most
extension modules are indeed as simple as that.) When the Python
program first imports module 'posix', initposix() is called, which
calls initmodule() with specific parameters. This creates a module
object (which is inserted in the table sys.modules under the key
'posix'), and adds built-in-function objects to the newly created
module based upon the table (of type struct methodlist) that was
passed as its second parameter. The function initmodule() returns a
pointer to the module object that it creates, but this is unused here.
It aborts with a fatal error if the module could not be initialized
satisfactorily.
program first imports module 'posix', \code{initposix()} is called,
which calls \code{initmodule()} with specific parameters. This
creates a module object (which is inserted in the table sys.modules
under the key 'posix'), and adds built-in-function objects to the
newly created module based upon the table (of type struct methodlist)
that was passed as its second parameter. The function
\code{initmodule()} returns a pointer to the module object that it
creates, but this is unused here. It aborts with a fatal error if the
module could not be initialized satisfactorily.
\section{Calling the module initialization function}
There is one more thing to do: telling the Python module to call the
initfoo() function when it encounters an 'import foo' statement.
\code{initfoo()} function when it encounters an 'import foo' statement.
This is done in the file config.c. This file contains a table mapping
module names to parameterless void function pointers. You need to add
a declaration of initfoo() somewhere early in the file, and a line
saying
a declaration of \code{initfoo()} somewhere early in the file, and a
line saying
\begin{verbatim}
{"foo", initfoo},
@ -274,12 +278,12 @@ saying
to the initializer for inittab[]. It is conventional to include both
the declaration and the initializer line in preprocessor commands
\verb\#ifdef USE_FOO\ / \verb\#endif\, to make it easy to turn the foo
extension on or off. Note that the Macintosh version uses a different
configuration file, distributed as configmac.c. This strategy may be
extended to other operating system versions, although usually the
standard config.c file gives a pretty useful starting point for a new
config*.c file.
\code{\#ifdef USE_FOO} / \code{\#endif}, to make it easy to turn the
foo extension on or off. Note that the Macintosh version uses a
different configuration file, distributed as configmac.c. This
strategy may be extended to other operating system versions, although
usually the standard config.c file gives a pretty useful starting
point for a new config*.c file.
And, of course, I forgot the Makefile. This is actually not too hard,
just follow the examples for, say, AMOEBA. Just find all occurrences
@ -287,8 +291,8 @@ of the string AMOEBA in the Makefile and do the same for FOO that's
done for AMOEBA...
(Note: if you are using dynamic loading for your extension, you don't
need to edit config.c and the Makefile. See "./DYNLOAD" for more info
about this.)
need to edit config.c and the Makefile. See \file{./DYNLOAD} for more
info about this.)
\section{Calling Python functions from C}
@ -296,7 +300,7 @@ about this.)
The above concentrates on making C functions accessible to the Python
programmer. The reverse is also often useful: calling Python
functions from C. This is especially the case for libraries that
support so-called "callback" functions. If a C interface makes heavy
support so-called ``callback'' functions. If a C interface makes heavy
use of callbacks, the equivalent Python often needs to provide a
callback mechanism to the Python programmer; the implementation may
require calling the Python callback functions from a C callback.
@ -305,14 +309,14 @@ Other uses are also possible.
Fortunately, the Python interpreter is easily called recursively, and
there is a standard interface to call a Python function. I won't
dwell on how to call the Python parser with a particular string as
input -- if you're interested, have a look at the implementation of
the "-c" command line option in pythonmain.c.
input --- if you're interested, have a look at the implementation of
the \samp{-c} command line option in pythonmain.c.
Calling a Python function is easy. First, the Python program must
somehow pass you the Python function object. You should provide a
function (or some other interface) to do this. When this function is
called, save a pointer to the Python function object (be careful to
INCREF it!) in a global variable -- or whereever you see fit.
INCREF it!) in a global variable --- or whereever you see fit.
For example, the following function might be part of a module
definition:
@ -333,9 +337,9 @@ definition:
\end{verbatim}
Later, when it is time to call the function, you call the C function
call_object(). This function has two arguments, both pointers to
arbitrary Python objects: the Python function, and the argument. The
argument can be NULL to call the function without arguments. For
\code{call_object()}. This function has two arguments, both pointers
to arbitrary Python objects: the Python function, and the argument.
The argument can be NULL to call the function without arguments. For
example:
\begin{verbatim}
@ -345,21 +349,22 @@ example:
result = call_object(my_callback, (object *)NULL);
\end{verbatim}
call_object() returns a Python object pointer: this is
the return value of the Python function. call_object() is
"reference-count-neutral" with respect to its arguments, but the
return value is "new": either it is a brand new object, or it is an
\code{call_object()} returns a Python object pointer: this is
the return value of the Python function. \code{call_object()} is
``reference-count-neutral'' with respect to its arguments, but the
return value is ``new'': either it is a brand new object, or it is an
existing object whose reference count has been incremented. So, you
should somehow apply DECREF to the result, even (especially!) if you
are not interested in its value.
Before you do this, however, it is important to check that the return
value isn't NULL. If it is, the Python function terminated by raising
an exception. If the C code that called call_object() is called from
Python, it should now return an error indication to its Python caller,
so the interpreter can print a stack trace, or the calling Python code
can handle the exception. If this is not possible or desirable, the
exception should be cleared by calling err_clear(). For example:
an exception. If the C code that called \code{call_object()} is
called from Python, it should now return an error indication to its
Python caller, so the interpreter can print a stack trace, or the
calling Python code can handle the exception. If this is not possible
or desirable, the exception should be cleared by calling
\code{err_clear()}. For example:
\begin{verbatim}
if (result == NULL)
@ -369,13 +374,14 @@ exception should be cleared by calling err_clear(). For example:
\end{verbatim}
Depending on the desired interface to the Python callback function,
you may also have to provide an argument to call_object(). In some
cases the argument is also provided by the Python program, through the
same interface that specified the callback function. It can then be
saved and used in the same manner as the function object. In other
cases, you may have to construct a new object to pass as argument. In
this case you must dispose of it as well. For example, if you want to
pass an integral event code, you might use the following code:
you may also have to provide an argument to \code{call_object()}. In
some cases the argument is also provided by the Python program,
through the same interface that specified the callback function. It
can then be saved and used in the same manner as the function object.
In other cases, you may have to construct a new object to pass as
argument. In this case you must dispose of it as well. For example,
if you want to pass an integral event code, you might use the
following code:
\begin{verbatim}
object *argument;
@ -391,8 +397,8 @@ pass an integral event code, you might use the following code:
Note the placement of DECREF(argument) immediately after the call,
before the error check! Also note that strictly spoken this code is
not complete: newintobject() may run out of memory, and this should be
checked.
not complete: \code{newintobject()} may run out of memory, and this
should be checked.
In even more complicated cases you may want to pass the callback
function multiple arguments. To this end you have to construct (and
@ -401,13 +407,15 @@ errror checks and reference count manipulation) are left as an
exercise for the reader; most of this is also needed when returning
multiple values from a function.
XXX TO DO: explain objects and reference counting.
XXX TO DO: explain objects.
XXX TO DO: defining new object types.
\section{Format strings for getargs()}
\section{Format strings for {\tt getargs()}}
The getargs() function is declared in "modsupport.h" as follows:
The \code{getargs()} function is declared in \file{modsupport.h} as
follows:
\begin{verbatim}
int getargs(object *arg, char *format, ...);
@ -416,89 +424,85 @@ The getargs() function is declared in "modsupport.h" as follows:
The remaining arguments must be addresses of variables whose type is
determined by the format string. For the conversion to succeed, the
`arg' object must match the format and the format must be exhausted.
Note that while getargs() checks that the Python object really is of
the specified type, it cannot check that the addresses provided in the
call match: if you make mistakes there, your code will probably dump
core.
Note that while \code{getargs()} checks that the Python object really
is of the specified type, it cannot check that the addresses provided
in the call match: if you make mistakes there, your code will probably
dump core.
A format string consists of a single `format unit'. A format unit
describes one Python object; it is usually a single character or a
parenthesized string. The type of a format units is determined from
its first character, the `format letter':
's' (string)
The Python object must be a string object. The C argument
must be a char** (i.e., the address of a character pointer),
and a pointer to the C string contained in the Python object
is stored into it. If the next character in the format string
is \verb\'#'\, another C argument of type int* must be present, and
the length of the Python string (not counting the trailing
zero byte) is stored into it.
\begin{description}
'z' (string or zero, i.e., NULL)
Like 's', but the object may also be None. In this case the
string pointer is set to NULL and if a \verb\'#'\ is present the size
it set to 0.
\item[\samp{s} (string)]
The Python object must be a string object. The C argument must be a
char** (i.e. the address of a character pointer), and a pointer to
the C string contained in the Python object is stored into it. If the
next character in the format string is \samp{\#}, another C argument
of type int* must be present, and the length of the Python string (not
counting the trailing zero byte) is stored into it.
'b' (byte, i.e., char interpreted as tiny int)
The object must be a Python integer. The C argument must be a
char*.
\item[\samp{z} (string or zero, i.e. \code{NULL})]
Like \samp{s}, but the object may also be None. In this case the
string pointer is set to NULL and if a \samp{\#} is present the size
it set to 0.
'h' (half, i.e., short)
The object must be a Python integer. The C argument must be a
short*.
\item[\samp{b} (byte, i.e. char interpreted as tiny int)]
The object must be a Python integer. The C argument must be a char*.
'i' (int)
The object must be a Python integer. The C argument must be
an int*.
\item[\samp{h} (half, i.e. short)]
The object must be a Python integer. The C argument must be a short*.
'l' (long)
The object must be a (plain!) Python integer. The C argument
must be a long*.
\item[\samp{i} (int)]
The object must be a Python integer. The C argument must be an int*.
'c' (char)
The Python object must be a string of length 1. The C
argument must be a char*. (Don't pass an int*!)
\item[\samp{l} (long)]
The object must be a (plain!) Python integer. The C argument must be
a long*.
'f' (float)
The object must be a Python int or float. The C argument must
be a float*.
\item[\samp{c} (char)]
The Python object must be a string of length 1. The C argument must
be a char*. (Don't pass an int*!)
'd' (double)
The object must be a Python int or float. The C argument must
be a double*.
\item[\samp{f} (float)]
The object must be a Python int or float. The C argument must be a
float*.
'S' (string object)
The object must be a Python string. The C argument must be an
object** (i.e., the address of an object pointer). The C
program thus gets back the actual string object that was
passed, not just a pointer to its array of characters and its
size as for format character 's'.
\item[\samp{d} (double)]
The object must be a Python int or float. The C argument must be a
double*.
'O' (object)
The object can be any Python object, including None, but not
NULL. The C argument must be an object**. This can be used
if an argument list must contain objects of a type for which
no format letter exist: the caller must then check that it has
the right type.
\item[\samp{S} (string object)]
The object must be a Python string. The C argument must be an
object** (i.e. the address of an object pointer). The C program thus
gets back the actual string object that was passed, not just a pointer
to its array of characters and its size as for format character
\samp{s}.
'(' (tuple)
The object must be a Python tuple. Following the '('
character in the format string must come a number of format
units describing the elements of the tuple, followed by a ')'
character. Tuple format units may be nested. (There are no
exceptions for empty and singleton tuples; "()" specifies an
empty tuple and "(i)" a singleton of one integer. Normally
you don't want to use the latter, since it is hard for the
user to specify.
\item[\samp{O} (object)]
The object can be any Python object, including None, but not NULL.
The C argument must be an object**. This can be used if an argument
list must contain objects of a type for which no format letter exist:
the caller must then check that it has the right type.
\item[\samp{(} (tuple)]
The object must be a Python tuple. Following the \samp{(} character
in the format string must come a number of format units describing the
elements of the tuple, followed by a \samp{)} character. Tuple
format units may be nested. (There are no exceptions for empty and
singleton tuples; \samp{()} specifies an empty tuple and \samp{(i)} a
singleton of one integer. Normally you don't want to use the latter,
since it is hard for the user to specify.
\end{description}
More format characters will probably be added as the need arises. It
should be allowed to use Python long integers whereever integers are
expected, and perform a range check. (A range check is in fact always
necessary for the 'b', 'h' and 'i' format letters, but this is
currently not implemented.)
necessary for the \samp{b}, \samp{h} and \samp{i} format
letters, but this is currently not implemented.)
Some example calls:
@ -533,14 +537,14 @@ Some example calls:
\end{verbatim}
Note that a format string must consist of a single unit; strings like
\verb\'is'\ and \verb\'(ii)s#'\ are not valid format strings. (But
\verb\'s#'\ is.)
\samp{is} and \samp{(ii)s\#} are not valid format strings. (But
\samp{s\#} is.)
The getargs() function does not support variable-length argument
lists. In simple cases you can fake these by trying several calls to
getargs() until one succeeds, but you must take care to call
err_clear() before each retry. For example:
The \code{getargs()} function does not support variable-length
argument lists. In simple cases you can fake these by trying several
calls to
\code{getargs()} until one succeeds, but you must take care to call
\code{err_clear()} before each retry. For example:
\begin{verbatim}
static object *my_method(self, args) object *self, *args; {
@ -561,46 +565,47 @@ err_clear() before each retry. For example:
\end{verbatim}
(It is possible to think of an extension to the definition of format
strings to accomodate this directly, e.g., placing a '|' in a tuple
might specify that the remaining arguments are optional. getargs()
should then return 1 + the number of variables stored into.)
strings to accomodate this directly, e.g., placing a \samp{|} in a
tuple might specify that the remaining arguments are optional.
\code{getargs()} should then return one more than the number of
variables stored into.)
Advanced users note: If you set the `varargs' flag in the method list
for a function, the argument will always be a tuple (the `raw argument
list'). In this case you must enclose single and empty argument lists
in parentheses, e.g., "(s)" and "()".
in parentheses, e.g., \samp{(s)} and \samp{()}.
\section{The mkvalue() function}
\section{The {\tt mkvalue()} function}
This function is the counterpart to getargs(). It is declared in
"modsupport.h" as follows:
This function is the counterpart to \code{getargs()}. It is declared
in \file{modsupport.h} as follows:
\begin{verbatim}
object *mkvalue(char *format, ...);
\end{verbatim}
It supports exactly the same format letters as getargs(), but the
arguments (which are input to the function, not output) must not be
pointers, just values. If a byte, short or float is passed to a
It supports exactly the same format letters as \code{getargs()}, but
the arguments (which are input to the function, not output) must not
be pointers, just values. If a byte, short or float is passed to a
varargs function, it is widened by the compiler to int or double, so
'b' and 'h' are treated as 'i' and 'f' is treated as 'd'. 'S' is
treated as 'O', 's' is treated as 'z'. \verb\'z#'\ and \verb\'s#'\
are supported: a second argument specifies the length of the data
(negative means use strlen()). 'S' and 'O' add a reference to their
argument (so you should DECREF it if you've just created it and aren't
going to use it again).
\samp{b} and \samp{h} are treated as \samp{i} and \samp{f} is
treated as \samp{d}. \samp{S} is treated as \samp{O}, \samp{s} is
treated as \samp{z}. \samp{z\#} and \samp{s\#} are supported: a
second argument specifies the length of the data (negative means use
\code{strlen()}). \samp{S} and \samp{O} add a reference to their
argument (so you should \code{DECREF()} it if you've just created it
and aren't going to use it again).
If the argument for 'O' or 'S' is a NULL pointer, it is assumed that
this was caused because the call producing the argument found an error
and set an exception. Therefore, mkvalue() will return NULL but won't
set an exception if one is already set. If no exception is set,
SystemError is set.
If the argument for \samp{O} or \samp{S} is a NULL pointer, it is
assumed that this was caused because the call producing the argument
found an error and set an exception. Therefore, \code{mkvalue()} will
return \code{NULL} but won't set an exception if one is already set.
If no exception is set, \code{SystemError} is set.
If there is an error in the format string, the SystemError exception
is set, since it is the calling C code's fault, not that of the Python
user who sees the exception.
If there is an error in the format string, the \code{SystemError}
exception is set, since it is the calling C code's fault, not that of
the Python user who sees the exception.
Example:
@ -610,99 +615,124 @@ Example:
returns a tuple containing two zeros. (Outer parentheses in the
format string are actually superfluous, but you can use them for
compatibility with getargs(), which requires them if more than one
argument is expected.)
compatibility with \code{getargs()}, which requires them if more than
one argument is expected.)
\section{Reference counts}
Here's a useful explanation of INCREF and DECREF by Sjoerd Mullender.
Here's a useful explanation of \code{INCREF()} and \code{DECREF()}
(after an original by Sjoerd Mullender).
Use XINCREF or XDECREF instead of INCREF/DECREF when the argument may
be NULL.
Use \code{XINCREF()} or \code{XDECREF()} instead of \code{INCREF()} /
\code{DECREF()} when the argument may be \code{NULL}.
The basic idea is, if you create an extra reference to an object, you
must INCREF it, if you throw away a reference to an object, you must
DECREF it. Functions such as newstringobject, newsizedstringobject,
newintobject, etc. create a reference to an object. If you want to
throw away the object thus created, you must use DECREF.
must \code{INCREF()} it, if you throw away a reference to an object,
you must \code{DECREF()} it. Functions such as
\code{newstringobject()}, \code{newsizedstringobject()},
\code{newintobject()}, etc. create a reference to an object. If you
want to throw away the object thus created, you must use
\code{DECREF()}.
If you put an object into a tuple, list, or dictionary, the idea is
that you usually don't want to keep a reference of your own around, so
Python does not INCREF the elements. It does DECREF the old value.
If you put an object into a tuple or list using \code{settupleitem()}
or \code{setlistitem()}, the idea is that you usually don't want to
keep a reference of your own around, so Python does not
\code{INCREF()} the elements. It does \code{DECREF()} the old value.
This means that if you put something into such an object using the
functions Python provides for this, you must INCREF the object if you
want to keep a separate reference to the object around. Also, if you
replace an element, you should INCREF the old element first if you
want to keep it. If you didn't INCREF it before you replaced it, you
are not allowed to look at it anymore, since it may have been freed.
functions Python provides for this, you must \code{INCREF()} the
object if you also want to keep a separate reference to the object around.
Also, if you replace an element, you should \code{INCREF()} the old
element first if you want to keep it. If you didn't \code{INCREF()}
it before you replaced it, you are not allowed to look at it anymore,
since it may have been freed.
Returning an object to Python (i.e., when your module function
returns) creates a reference to an object, but it does not change the
reference count. When your module does not keep another reference to
the object, you should not INCREF or DECREF it. When you do keep a
reference around, you should INCREF the object. Also, when you return
a global object such as None, you should INCREF it.
Returning an object to Python (i.e. when your C function returns)
creates a reference to an object, but it does not change the reference
count. When your code does not keep another reference to the object,
you should not \code{INCREF()} or \code{DECREF()} it (assuming it is a
newly created object). When you do keep a reference around, you
should \code{INCREF()} the object. Also, when you return a global
object such as \code{None}, you should \code{INCREF()} it.
If you want to return a tuple, you should consider using mkvalue.
Mkvalue creates a new tuple with a reference count of 1 which you can
return. If any of the elements you put into the tuple are objects,
they are INCREFfed by mkvalue. If you don't want to keep references
to those elements around, you should DECREF them after having called
mkvalue.
If you want to return a tuple, you should consider using
\code{mkvalue()}. This function creates a new tuple with a reference
count of 1 which you can return. If any of the elements you put into
the tuple are objects (format codes \samp{O} or \samp{S}), they
are \code{INCREF()}'ed by \code{mkvalue()}. If you don't want to keep
references to those elements around, you should \code{DECREF()} them
after having called \code{mkvalue()}.
Usually you don't have to worry about arguments. They are INCREFfed
before your function is called and DECREFfed after your function
returns. When you keep a reference to an argument, you should INCREF
it and DECREF when you throw it away. Also, when you return an
argument, you should INCREF it, because returning the argument creates
an extra reference to it.
Usually you don't have to worry about arguments. They are
\code{INCREF()}'ed before your function is called and
\code{DECREF()}'ed after your function returns. When you keep a
reference to an argument, you should \code{INCREF()} it and
\code{DECREF()} when you throw it away. Also, when you return an
argument, you should \code{INCREF()} it, because returning the
argument creates an extra reference to it.
If you use getargs() to parse the arguments, you can get a reference
to an object (by using "O" in the format string). This object was not
INCREFfed, so you should not DECREF it. If you want to keep the
object, you must INCREF it yourself.
If you use \code{getargs()} to parse the arguments, you can get a
reference to an object (by using \samp{O} in the format string). This
object was not \code{INCREF()}'ed, so you should not \code{DECREF()}
it. If you want to keep the object, you must \code{INCREF()} it
yourself.
If you create your own type of objects, you should use \code{NEWOBJ()}
to create the object. This sets the reference count to 1. If you
want to throw away the object, you should use \code{DECREF()}. When
the reference count reaches zero, your type's \code{dealloc()}
function is called. In it, you should \code{DECREF()} all object to
which you keep references in your object, but you should not use
\code{DECREF()} on your object. You should use \code{DEL()} instead.
\section{Using C++}
It is possible to write extension modules in C++. Some restrictions
apply: since the main program (the Python interpreter) is compiled and
linked by the C compiler, global or static objects with constructors
cannot be used. All functions that will be called directly or
indirectly (i.e. via function pointers) by the Python interpreter will
have to be declared using \code{extern "C"}; this applies to all
`methods' as well as to the module's initialization function.
It is unnecessary to enclose the Python header files in
\code{extern "C" \{...\}} --- they do this already.
If you create your own type of objects, you should use NEWOBJ to
create the object. This sets the reference count to 1. If you want
to throw away the object, you should use DECREF. When the reference
count reaches 0, the dealloc function is called. In it, you should
DECREF all object to which you keep references in your object, but you
should not use DECREF on your object. You should use DEL instead.
\chapter{Embedding Python in another application}
Embedding Python is similar to extending it, but not quite. The
difference is that when you extend Python, the main program of the
application is still the Python interpreter, while of you embed
Python, the main program may have nothing to do with Python --
Python, the main program may have nothing to do with Python ---
instead, some parts of the application occasionally call the Python
interpreter to run some Python code.
So if you are embedding Python, you are providing your own main
program. One of the things this main program has to do is initialize
the Python interpreter. At the very least, you have to call the
function initall(). There are optional calls to pass command line
arguments to Python. Then later you can call the interpreter from any
part of the application.
function \code{initall()}. There are optional calls to pass command
line arguments to Python. Then later you can call the interpreter
from any part of the application.
There are several different ways to call the interpreter: you can pass
a string containing Python statements to run_command(), or you can
pass a stdio file pointer and a file name (for identification in error
messages only) to run_script(). You can also call the lower-level
operations described (partly) in the file \verb\<pythonroot>/misc/EXTENDING\
to construct and use Python objects.
a string containing Python statements to \code{run_command()}, or you
can pass a stdio file pointer and a file name (for identification in
error messages only) to \code{run_script()}. You can also call the
lower-level operations described in the previous chapters to construct
and use Python objects.
A simple demo of embedding Python can be found in the directory
\verb\<pythonroot>/embed/\.
\file{<pythonroot>/embed}.
\section{Using C++}
It is also possible to embed Python in a C++ program; how this is done
exactly will depend on the details of the C++ system used; in general
you will need to write the main program in C++, enclosing the include
files in \verb\"extern "C" { ... }"\, and compile and link this with
the C++ compiler. (There is no need to recompile Python itself with
C++.)
you will need to write the main program in C++, and use the C++
compiler to compile and link your program. There is no need to
recompile Python itself with C++.
\input{ext.ind}

534
Doc/ext/ext.tex
View File

@ -1,11 +1,11 @@
\documentstyle[twoside,11pt,myformat]{report}
\documentstyle[twoside,11pt,myformat,times]{report}
\title{\bf Extending and Embedding the Python Interpreter}
\author{
Guido van Rossum \\
Dept. CST, CWI, Kruislaan 413 \\
1098 SJ Amsterdam, The Netherlands \\
Dept. CST, CWI, P.O. Box 94079 \\
1090 GB Amsterdam, The Netherlands \\
E-mail: {\tt guido@cwi.nl}
}
@ -39,12 +39,13 @@ interpreter as a library package from applications using Python as an
\pagenumbering{arabic}
\chapter{Extending Python with C or C++ code}
It is quite easy to add non-standard built-in modules to Python, if
you know how to program in C. A built-in module known to the Python
programmer as foo is generally implemented in a file called
foomodule.c. The standard built-in modules also adhere to this
programmer as \code{foo} is generally implemented in a file called
\file{foomodule.c}. The standard built-in modules also adhere to this
convention, and in fact some of them form excellent examples of how to
create an extension.
@ -53,11 +54,12 @@ Python: implement new data types and provide access to system calls or
C library functions. Since the latter is usually the most important
reason for adding an extension, I'll concentrate on adding "wrappers"
around C library functions; the concrete example uses the wrapper for
system() in module posix, found in (of course) the file posixmodule.c.
\code{system()} in module posix, found in (of course) the file
posixmodule.c.
It is important not to be impressed by the size and complexity of
the average extension module; much of this is straightforward
"boilerplate" code (starting right with the copyright notice!).
``boilerplate'' code (starting right with the copyright notice!).
Let's skip the boilerplate and jump right to an interesting function:
@ -100,20 +102,20 @@ otherwise the Python user could cause a core dump by passing the wrong
arguments (or no arguments at all). Because argument checking and
converting arguments to C is such a common task, there's a general
function in the Python interpreter which combines these tasks:
getargs(). It uses a template string to determine both the types of
the Python argument and the types of the C variables into which it
should store the converted values.
\code{getargs()}. It uses a template string to determine both the
types of the Python argument and the types of the C variables into
which it should store the converted values.
When getargs returns nonzero, the argument list has the right type and
its components have been stored in the variables whose addresses are
passed. When it returns zero, an error has occurred. In the latter
case it has already raised an appropriate exception by calling
err_setstr(), so the calling function can just return NULL.
\code{err_setstr()}, so the calling function can just return NULL.
The form of the format string is described at the end of this file.
(There are convenience macros getstrarg(), getintarg(), etc., for many
common forms of argument lists. These are relics from the past; it's
better to call getargs() directly.)
(There are convenience macros \code{getstrarg()}, \code{getintarg()},
etc., for many common forms of argument lists. These are relics from
the past; it's better to call \code{getargs()} directly.)
\section{Intermezzo: errors and exceptions}
@ -122,56 +124,58 @@ An important convention throughout the Python interpreter is the
following: when a function fails, it should set an exception condition
and return an error value (often a NULL pointer). Exceptions are set
in a global variable in the file errors.c; if this variable is NULL no
exception has occurred. A second variable is the "associated value"
exception has occurred. A second variable is the ``associated value''
of the exception.
The file errors.h declares a host of err_* functions to set various
types of exceptions. The most common one is err_setstr() -- its
arguments are an exception object (e.g. RuntimeError -- actually it
types of exceptions. The most common one is \code{err_setstr()} --- its
arguments are an exception object (e.g. RuntimeError --- actually it
can be any string object) and a C string indicating the cause of the
error (this is converted to a string object and stored as the
"associated value" of the exception). Another useful function is
err_errno(), which only takes an exception argument and constructs the
associated value by inspection of the (UNIX) global variable errno.
``associated value'' of the exception). Another useful function is
\code{err_errno()}, which only takes an exception argument and
constructs the associated value by inspection of the (UNIX) global
variable errno.
You can test non-destructively whether an exception has been set with
err_occurred(). However, most code never calls err_occurred() to see
whether an error occurred or not, but relies on error return values
from the functions it calls instead:
\code{err_occurred()}. However, most code never calls
\code{err_occurred()} to see whether an error occurred or not, but
relies on error return values from the functions it calls instead:
When a function that calls another function detects that the called
function fails, it should return an error value but not set an
condition -- one is already set. The caller is then supposed to also
condition --- one is already set. The caller is then supposed to also
return an error indication to *its* caller, again *without* calling
err_setstr(), and so on -- the most detailed cause of the error was
already reported by the function that detected it in the first place.
Once the error has reached Python's interpreter main loop, this aborts
the currently executing Python code and tries to find an exception
handler specified by the Python programmer.
\code{err_setstr()}, and so on --- the most detailed cause of the error
was already reported by the function that detected it in the first
place. Once the error has reached Python's interpreter main loop,
this aborts the currently executing Python code and tries to find an
exception handler specified by the Python programmer.
To ignore an exception set by a function call that failed, the
exception condition must be cleared explicitly by calling err_clear().
The only time C code should call err_clear() is if it doesn't want to
pass the error on to the interpreter but wants to handle it completely
by itself (e.g. by trying something else or pretending nothing
happened).
exception condition must be cleared explicitly by calling
\code{err_clear()}. The only time C code should call
\code{err_clear()} is if it doesn't want to pass the error on to the
interpreter but wants to handle it completely by itself (e.g. by
trying something else or pretending nothing happened).
Finally, the function err_get() gives you both error variables
Finally, the function \code{err_get()} gives you both error variables
*and clears them*. Note that even if an error occurred the second one
may be NULL. I doubt you will need to use this function.
Note that a failing malloc() call must also be turned into an
exception -- the direct caller of malloc() (or realloc()) must call
err_nomem() and return a failure indicator itself. All the
object-creating functions (newintobject() etc.) already do this, so
only if you call malloc() directly this note is of importance.
Note that a failing \code{malloc()} call must also be turned into an
exception --- the direct caller of \code{malloc()} (or
\code{realloc()}) must call \code{err_nomem()} and return a failure
indicator itself. All the object-creating functions
(\code{newintobject()} etc.) already do this, so only if you call
\code{malloc()} directly this note is of importance.
Also note that, with the important exception of getargs(), functions
Also note that, with the important exception of \code{getargs()}, functions
that return an integer status usually use 0 for success and -1 for
failure.
Finally, be careful about cleaning up garbage (making appropriate
[X]DECREF() calls) when you return an error!
[\code{X}]\code{DECREF()} calls) when you return an error!
\section{Back to the example}
@ -186,7 +190,7 @@ bit:
It returns NULL (the error indicator for functions of this kind) if an
error is detected in the argument list, relying on the exception set
by getargs(). The string value of the argument is now copied to the
by \code{getargs()}. The string value of the argument is now copied to the
local variable 'command'.
If a Python function is called with multiple arguments, the argument
@ -195,18 +199,18 @@ instance, to explicitly create the tuple containing the arguments
first and make the call later.
The next statement in posix_system is a call tothe C library function
system(), passing it the string we just got from getargs():
\code{system()}, passing it the string we just got from \code{getargs()}:
\begin{verbatim}
sts = system(command);
\end{verbatim}
Python strings may contain internal null bytes; but if these occur in
this example the rest of the string will be ignored by system().
this example the rest of the string will be ignored by \code{system()}.
Finally, posix.system() must return a value: the integer status
returned by the C library system() function. This is done by the
function newintobject(), which takes a (long) integer as parameter.
Finally, posix.\code{system()} must return a value: the integer status
returned by the C library \code{system()} function. This is done by the
function \code{newintobject()}, which takes a (long) integer as parameter.
\begin{verbatim}
return newintobject((long)sts);
@ -223,13 +227,13 @@ this idiom:
'None' is a unique Python object representing 'no value'. It differs
from NULL, which means 'error' in most contexts (except when passed as
a function argument -- there it means 'no arguments').
a function argument --- there it means 'no arguments').
\section{The module's function table}
I promised to show how I made the function posix_system() available to
Python programs. This is shown later in posixmodule.c:
I promised to show how I made the function \code{posix_system()}
available to Python programs. This is shown later in posixmodule.c:
\begin{verbatim}
static struct methodlist posix_methods[] = {
@ -246,27 +250,27 @@ Python programs. This is shown later in posixmodule.c:
}
\end{verbatim}
(The actual initposix() is somewhat more complicated, but most
(The actual \code{initposix()} is somewhat more complicated, but most
extension modules are indeed as simple as that.) When the Python
program first imports module 'posix', initposix() is called, which
calls initmodule() with specific parameters. This creates a module
object (which is inserted in the table sys.modules under the key
'posix'), and adds built-in-function objects to the newly created
module based upon the table (of type struct methodlist) that was
passed as its second parameter. The function initmodule() returns a
pointer to the module object that it creates, but this is unused here.
It aborts with a fatal error if the module could not be initialized
satisfactorily.
program first imports module 'posix', \code{initposix()} is called,
which calls \code{initmodule()} with specific parameters. This
creates a module object (which is inserted in the table sys.modules
under the key 'posix'), and adds built-in-function objects to the
newly created module based upon the table (of type struct methodlist)
that was passed as its second parameter. The function
\code{initmodule()} returns a pointer to the module object that it
creates, but this is unused here. It aborts with a fatal error if the
module could not be initialized satisfactorily.
\section{Calling the module initialization function}
There is one more thing to do: telling the Python module to call the
initfoo() function when it encounters an 'import foo' statement.
\code{initfoo()} function when it encounters an 'import foo' statement.
This is done in the file config.c. This file contains a table mapping
module names to parameterless void function pointers. You need to add
a declaration of initfoo() somewhere early in the file, and a line
saying
a declaration of \code{initfoo()} somewhere early in the file, and a
line saying
\begin{verbatim}
{"foo", initfoo},
@ -274,12 +278,12 @@ saying
to the initializer for inittab[]. It is conventional to include both
the declaration and the initializer line in preprocessor commands
\verb\#ifdef USE_FOO\ / \verb\#endif\, to make it easy to turn the foo
extension on or off. Note that the Macintosh version uses a different
configuration file, distributed as configmac.c. This strategy may be
extended to other operating system versions, although usually the
standard config.c file gives a pretty useful starting point for a new
config*.c file.
\code{\#ifdef USE_FOO} / \code{\#endif}, to make it easy to turn the
foo extension on or off. Note that the Macintosh version uses a
different configuration file, distributed as configmac.c. This
strategy may be extended to other operating system versions, although
usually the standard config.c file gives a pretty useful starting
point for a new config*.c file.
And, of course, I forgot the Makefile. This is actually not too hard,
just follow the examples for, say, AMOEBA. Just find all occurrences
@ -287,8 +291,8 @@ of the string AMOEBA in the Makefile and do the same for FOO that's
done for AMOEBA...
(Note: if you are using dynamic loading for your extension, you don't
need to edit config.c and the Makefile. See "./DYNLOAD" for more info
about this.)
need to edit config.c and the Makefile. See \file{./DYNLOAD} for more
info about this.)
\section{Calling Python functions from C}
@ -296,7 +300,7 @@ about this.)
The above concentrates on making C functions accessible to the Python
programmer. The reverse is also often useful: calling Python
functions from C. This is especially the case for libraries that
support so-called "callback" functions. If a C interface makes heavy
support so-called ``callback'' functions. If a C interface makes heavy
use of callbacks, the equivalent Python often needs to provide a
callback mechanism to the Python programmer; the implementation may
require calling the Python callback functions from a C callback.
@ -305,14 +309,14 @@ Other uses are also possible.
Fortunately, the Python interpreter is easily called recursively, and
there is a standard interface to call a Python function. I won't
dwell on how to call the Python parser with a particular string as
input -- if you're interested, have a look at the implementation of
the "-c" command line option in pythonmain.c.
input --- if you're interested, have a look at the implementation of
the \samp{-c} command line option in pythonmain.c.
Calling a Python function is easy. First, the Python program must
somehow pass you the Python function object. You should provide a
function (or some other interface) to do this. When this function is
called, save a pointer to the Python function object (be careful to
INCREF it!) in a global variable -- or whereever you see fit.
INCREF it!) in a global variable --- or whereever you see fit.
For example, the following function might be part of a module
definition:
@ -333,9 +337,9 @@ definition:
\end{verbatim}
Later, when it is time to call the function, you call the C function
call_object(). This function has two arguments, both pointers to
arbitrary Python objects: the Python function, and the argument. The
argument can be NULL to call the function without arguments. For
\code{call_object()}. This function has two arguments, both pointers
to arbitrary Python objects: the Python function, and the argument.
The argument can be NULL to call the function without arguments. For
example:
\begin{verbatim}
@ -345,21 +349,22 @@ example:
result = call_object(my_callback, (object *)NULL);
\end{verbatim}
call_object() returns a Python object pointer: this is
the return value of the Python function. call_object() is
"reference-count-neutral" with respect to its arguments, but the
return value is "new": either it is a brand new object, or it is an
\code{call_object()} returns a Python object pointer: this is
the return value of the Python function. \code{call_object()} is
``reference-count-neutral'' with respect to its arguments, but the
return value is ``new'': either it is a brand new object, or it is an
existing object whose reference count has been incremented. So, you
should somehow apply DECREF to the result, even (especially!) if you
are not interested in its value.
Before you do this, however, it is important to check that the return
value isn't NULL. If it is, the Python function terminated by raising
an exception. If the C code that called call_object() is called from
Python, it should now return an error indication to its Python caller,
so the interpreter can print a stack trace, or the calling Python code
can handle the exception. If this is not possible or desirable, the
exception should be cleared by calling err_clear(). For example:
an exception. If the C code that called \code{call_object()} is
called from Python, it should now return an error indication to its
Python caller, so the interpreter can print a stack trace, or the
calling Python code can handle the exception. If this is not possible
or desirable, the exception should be cleared by calling
\code{err_clear()}. For example:
\begin{verbatim}
if (result == NULL)
@ -369,13 +374,14 @@ exception should be cleared by calling err_clear(). For example:
\end{verbatim}
Depending on the desired interface to the Python callback function,
you may also have to provide an argument to call_object(). In some
cases the argument is also provided by the Python program, through the
same interface that specified the callback function. It can then be
saved and used in the same manner as the function object. In other
cases, you may have to construct a new object to pass as argument. In
this case you must dispose of it as well. For example, if you want to
pass an integral event code, you might use the following code:
you may also have to provide an argument to \code{call_object()}. In
some cases the argument is also provided by the Python program,
through the same interface that specified the callback function. It
can then be saved and used in the same manner as the function object.
In other cases, you may have to construct a new object to pass as
argument. In this case you must dispose of it as well. For example,
if you want to pass an integral event code, you might use the
following code:
\begin{verbatim}
object *argument;
@ -391,8 +397,8 @@ pass an integral event code, you might use the following code:
Note the placement of DECREF(argument) immediately after the call,
before the error check! Also note that strictly spoken this code is
not complete: newintobject() may run out of memory, and this should be
checked.
not complete: \code{newintobject()} may run out of memory, and this
should be checked.
In even more complicated cases you may want to pass the callback
function multiple arguments. To this end you have to construct (and
@ -401,13 +407,15 @@ errror checks and reference count manipulation) are left as an
exercise for the reader; most of this is also needed when returning
multiple values from a function.
XXX TO DO: explain objects and reference counting.
XXX TO DO: explain objects.
XXX TO DO: defining new object types.
\section{Format strings for getargs()}
\section{Format strings for {\tt getargs()}}
The getargs() function is declared in "modsupport.h" as follows:
The \code{getargs()} function is declared in \file{modsupport.h} as
follows:
\begin{verbatim}
int getargs(object *arg, char *format, ...);
@ -416,89 +424,85 @@ The getargs() function is declared in "modsupport.h" as follows:
The remaining arguments must be addresses of variables whose type is
determined by the format string. For the conversion to succeed, the
`arg' object must match the format and the format must be exhausted.
Note that while getargs() checks that the Python object really is of
the specified type, it cannot check that the addresses provided in the
call match: if you make mistakes there, your code will probably dump
core.
Note that while \code{getargs()} checks that the Python object really
is of the specified type, it cannot check that the addresses provided
in the call match: if you make mistakes there, your code will probably
dump core.
A format string consists of a single `format unit'. A format unit
describes one Python object; it is usually a single character or a
parenthesized string. The type of a format units is determined from
its first character, the `format letter':
's' (string)
The Python object must be a string object. The C argument
must be a char** (i.e., the address of a character pointer),
and a pointer to the C string contained in the Python object
is stored into it. If the next character in the format string
is \verb\'#'\, another C argument of type int* must be present, and
the length of the Python string (not counting the trailing
zero byte) is stored into it.
\begin{description}
'z' (string or zero, i.e., NULL)
Like 's', but the object may also be None. In this case the
string pointer is set to NULL and if a \verb\'#'\ is present the size
it set to 0.
\item[\samp{s} (string)]
The Python object must be a string object. The C argument must be a
char** (i.e. the address of a character pointer), and a pointer to
the C string contained in the Python object is stored into it. If the
next character in the format string is \samp{\#}, another C argument
of type int* must be present, and the length of the Python string (not
counting the trailing zero byte) is stored into it.
'b' (byte, i.e., char interpreted as tiny int)
The object must be a Python integer. The C argument must be a
char*.
\item[\samp{z} (string or zero, i.e. \code{NULL})]
Like \samp{s}, but the object may also be None. In this case the
string pointer is set to NULL and if a \samp{\#} is present the size
it set to 0.
'h' (half, i.e., short)
The object must be a Python integer. The C argument must be a
short*.
\item[\samp{b} (byte, i.e. char interpreted as tiny int)]
The object must be a Python integer. The C argument must be a char*.
'i' (int)
The object must be a Python integer. The C argument must be
an int*.
\item[\samp{h} (half, i.e. short)]
The object must be a Python integer. The C argument must be a short*.
'l' (long)
The object must be a (plain!) Python integer. The C argument
must be a long*.
\item[\samp{i} (int)]
The object must be a Python integer. The C argument must be an int*.
'c' (char)
The Python object must be a string of length 1. The C
argument must be a char*. (Don't pass an int*!)
\item[\samp{l} (long)]
The object must be a (plain!) Python integer. The C argument must be
a long*.
'f' (float)
The object must be a Python int or float. The C argument must
be a float*.
\item[\samp{c} (char)]
The Python object must be a string of length 1. The C argument must
be a char*. (Don't pass an int*!)
'd' (double)
The object must be a Python int or float. The C argument must
be a double*.
\item[\samp{f} (float)]
The object must be a Python int or float. The C argument must be a
float*.
'S' (string object)
The object must be a Python string. The C argument must be an
object** (i.e., the address of an object pointer). The C
program thus gets back the actual string object that was
passed, not just a pointer to its array of characters and its
size as for format character 's'.
\item[\samp{d} (double)]
The object must be a Python int or float. The C argument must be a
double*.
'O' (object)
The object can be any Python object, including None, but not
NULL. The C argument must be an object**. This can be used
if an argument list must contain objects of a type for which
no format letter exist: the caller must then check that it has
the right type.
\item[\samp{S} (string object)]
The object must be a Python string. The C argument must be an
object** (i.e. the address of an object pointer). The C program thus
gets back the actual string object that was passed, not just a pointer
to its array of characters and its size as for format character
\samp{s}.
'(' (tuple)
The object must be a Python tuple. Following the '('
character in the format string must come a number of format
units describing the elements of the tuple, followed by a ')'
character. Tuple format units may be nested. (There are no
exceptions for empty and singleton tuples; "()" specifies an
empty tuple and "(i)" a singleton of one integer. Normally
you don't want to use the latter, since it is hard for the
user to specify.
\item[\samp{O} (object)]
The object can be any Python object, including None, but not NULL.
The C argument must be an object**. This can be used if an argument
list must contain objects of a type for which no format letter exist:
the caller must then check that it has the right type.
\item[\samp{(} (tuple)]
The object must be a Python tuple. Following the \samp{(} character
in the format string must come a number of format units describing the
elements of the tuple, followed by a \samp{)} character. Tuple
format units may be nested. (There are no exceptions for empty and
singleton tuples; \samp{()} specifies an empty tuple and \samp{(i)} a
singleton of one integer. Normally you don't want to use the latter,
since it is hard for the user to specify.
\end{description}
More format characters will probably be added as the need arises. It
should be allowed to use Python long integers whereever integers are
expected, and perform a range check. (A range check is in fact always
necessary for the 'b', 'h' and 'i' format letters, but this is
currently not implemented.)
necessary for the \samp{b}, \samp{h} and \samp{i} format
letters, but this is currently not implemented.)
Some example calls:
@ -533,14 +537,14 @@ Some example calls:
\end{verbatim}
Note that a format string must consist of a single unit; strings like
\verb\'is'\ and \verb\'(ii)s#'\ are not valid format strings. (But
\verb\'s#'\ is.)
\samp{is} and \samp{(ii)s\#} are not valid format strings. (But
\samp{s\#} is.)
The getargs() function does not support variable-length argument
lists. In simple cases you can fake these by trying several calls to
getargs() until one succeeds, but you must take care to call
err_clear() before each retry. For example:
The \code{getargs()} function does not support variable-length
argument lists. In simple cases you can fake these by trying several
calls to
\code{getargs()} until one succeeds, but you must take care to call
\code{err_clear()} before each retry. For example:
\begin{verbatim}
static object *my_method(self, args) object *self, *args; {
@ -561,46 +565,47 @@ err_clear() before each retry. For example:
\end{verbatim}
(It is possible to think of an extension to the definition of format
strings to accomodate this directly, e.g., placing a '|' in a tuple
might specify that the remaining arguments are optional. getargs()
should then return 1 + the number of variables stored into.)
strings to accomodate this directly, e.g., placing a \samp{|} in a
tuple might specify that the remaining arguments are optional.
\code{getargs()} should then return one more than the number of
variables stored into.)
Advanced users note: If you set the `varargs' flag in the method list
for a function, the argument will always be a tuple (the `raw argument
list'). In this case you must enclose single and empty argument lists
in parentheses, e.g., "(s)" and "()".
in parentheses, e.g., \samp{(s)} and \samp{()}.
\section{The mkvalue() function}
\section{The {\tt mkvalue()} function}
This function is the counterpart to getargs(). It is declared in
"modsupport.h" as follows:
This function is the counterpart to \code{getargs()}. It is declared
in \file{modsupport.h} as follows:
\begin{verbatim}
object *mkvalue(char *format, ...);
\end{verbatim}
It supports exactly the same format letters as getargs(), but the
arguments (which are input to the function, not output) must not be
pointers, just values. If a byte, short or float is passed to a
It supports exactly the same format letters as \code{getargs()}, but
the arguments (which are input to the function, not output) must not
be pointers, just values. If a byte, short or float is passed to a
varargs function, it is widened by the compiler to int or double, so
'b' and 'h' are treated as 'i' and 'f' is treated as 'd'. 'S' is
treated as 'O', 's' is treated as 'z'. \verb\'z#'\ and \verb\'s#'\
are supported: a second argument specifies the length of the data
(negative means use strlen()). 'S' and 'O' add a reference to their
argument (so you should DECREF it if you've just created it and aren't
going to use it again).
\samp{b} and \samp{h} are treated as \samp{i} and \samp{f} is
treated as \samp{d}. \samp{S} is treated as \samp{O}, \samp{s} is
treated as \samp{z}. \samp{z\#} and \samp{s\#} are supported: a
second argument specifies the length of the data (negative means use
\code{strlen()}). \samp{S} and \samp{O} add a reference to their
argument (so you should \code{DECREF()} it if you've just created it
and aren't going to use it again).
If the argument for 'O' or 'S' is a NULL pointer, it is assumed that
this was caused because the call producing the argument found an error
and set an exception. Therefore, mkvalue() will return NULL but won't
set an exception if one is already set. If no exception is set,
SystemError is set.
If the argument for \samp{O} or \samp{S} is a NULL pointer, it is
assumed that this was caused because the call producing the argument
found an error and set an exception. Therefore, \code{mkvalue()} will
return \code{NULL} but won't set an exception if one is already set.
If no exception is set, \code{SystemError} is set.
If there is an error in the format string, the SystemError exception
is set, since it is the calling C code's fault, not that of the Python
user who sees the exception.
If there is an error in the format string, the \code{SystemError}
exception is set, since it is the calling C code's fault, not that of
the Python user who sees the exception.
Example:
@ -610,99 +615,124 @@ Example:
returns a tuple containing two zeros. (Outer parentheses in the
format string are actually superfluous, but you can use them for
compatibility with getargs(), which requires them if more than one
argument is expected.)
compatibility with \code{getargs()}, which requires them if more than
one argument is expected.)
\section{Reference counts}
Here's a useful explanation of INCREF and DECREF by Sjoerd Mullender.
Here's a useful explanation of \code{INCREF()} and \code{DECREF()}
(after an original by Sjoerd Mullender).
Use XINCREF or XDECREF instead of INCREF/DECREF when the argument may
be NULL.
Use \code{XINCREF()} or \code{XDECREF()} instead of \code{INCREF()} /
\code{DECREF()} when the argument may be \code{NULL}.
The basic idea is, if you create an extra reference to an object, you
must INCREF it, if you throw away a reference to an object, you must
DECREF it. Functions such as newstringobject, newsizedstringobject,
newintobject, etc. create a reference to an object. If you want to
throw away the object thus created, you must use DECREF.
must \code{INCREF()} it, if you throw away a reference to an object,
you must \code{DECREF()} it. Functions such as
\code{newstringobject()}, \code{newsizedstringobject()},
\code{newintobject()}, etc. create a reference to an object. If you
want to throw away the object thus created, you must use
\code{DECREF()}.
If you put an object into a tuple, list, or dictionary, the idea is
that you usually don't want to keep a reference of your own around, so
Python does not INCREF the elements. It does DECREF the old value.
If you put an object into a tuple or list using \code{settupleitem()}
or \code{setlistitem()}, the idea is that you usually don't want to
keep a reference of your own around, so Python does not
\code{INCREF()} the elements. It does \code{DECREF()} the old value.
This means that if you put something into such an object using the
functions Python provides for this, you must INCREF the object if you
want to keep a separate reference to the object around. Also, if you
replace an element, you should INCREF the old element first if you
want to keep it. If you didn't INCREF it before you replaced it, you
are not allowed to look at it anymore, since it may have been freed.
functions Python provides for this, you must \code{INCREF()} the
object if you also want to keep a separate reference to the object around.
Also, if you replace an element, you should \code{INCREF()} the old
element first if you want to keep it. If you didn't \code{INCREF()}
it before you replaced it, you are not allowed to look at it anymore,
since it may have been freed.
Returning an object to Python (i.e., when your module function
returns) creates a reference to an object, but it does not change the
reference count. When your module does not keep another reference to
the object, you should not INCREF or DECREF it. When you do keep a
reference around, you should INCREF the object. Also, when you return
a global object such as None, you should INCREF it.
Returning an object to Python (i.e. when your C function returns)
creates a reference to an object, but it does not change the reference
count. When your code does not keep another reference to the object,
you should not \code{INCREF()} or \code{DECREF()} it (assuming it is a
newly created object). When you do keep a reference around, you
should \code{INCREF()} the object. Also, when you return a global
object such as \code{None}, you should \code{INCREF()} it.
If you want to return a tuple, you should consider using mkvalue.
Mkvalue creates a new tuple with a reference count of 1 which you can
return. If any of the elements you put into the tuple are objects,
they are INCREFfed by mkvalue. If you don't want to keep references
to those elements around, you should DECREF them after having called
mkvalue.
If you want to return a tuple, you should consider using
\code{mkvalue()}. This function creates a new tuple with a reference
count of 1 which you can return. If any of the elements you put into
the tuple are objects (format codes \samp{O} or \samp{S}), they
are \code{INCREF()}'ed by \code{mkvalue()}. If you don't want to keep
references to those elements around, you should \code{DECREF()} them
after having called \code{mkvalue()}.
Usually you don't have to worry about arguments. They are INCREFfed
before your function is called and DECREFfed after your function
returns. When you keep a reference to an argument, you should INCREF
it and DECREF when you throw it away. Also, when you return an
argument, you should INCREF it, because returning the argument creates
an extra reference to it.
Usually you don't have to worry about arguments. They are
\code{INCREF()}'ed before your function is called and
\code{DECREF()}'ed after your function returns. When you keep a
reference to an argument, you should \code{INCREF()} it and
\code{DECREF()} when you throw it away. Also, when you return an
argument, you should \code{INCREF()} it, because returning the
argument creates an extra reference to it.
If you use getargs() to parse the arguments, you can get a reference
to an object (by using "O" in the format string). This object was not
INCREFfed, so you should not DECREF it. If you want to keep the
object, you must INCREF it yourself.
If you use \code{getargs()} to parse the arguments, you can get a
reference to an object (by using \samp{O} in the format string). This
object was not \code{INCREF()}'ed, so you should not \code{DECREF()}
it. If you want to keep the object, you must \code{INCREF()} it
yourself.
If you create your own type of objects, you should use \code{NEWOBJ()}
to create the object. This sets the reference count to 1. If you
want to throw away the object, you should use \code{DECREF()}. When
the reference count reaches zero, your type's \code{dealloc()}
function is called. In it, you should \code{DECREF()} all object to
which you keep references in your object, but you should not use
\code{DECREF()} on your object. You should use \code{DEL()} instead.
\section{Using C++}
It is possible to write extension modules in C++. Some restrictions
apply: since the main program (the Python interpreter) is compiled and
linked by the C compiler, global or static objects with constructors
cannot be used. All functions that will be called directly or
indirectly (i.e. via function pointers) by the Python interpreter will
have to be declared using \code{extern "C"}; this applies to all
`methods' as well as to the module's initialization function.
It is unnecessary to enclose the Python header files in
\code{extern "C" \{...\}} --- they do this already.
If you create your own type of objects, you should use NEWOBJ to
create the object. This sets the reference count to 1. If you want
to throw away the object, you should use DECREF. When the reference
count reaches 0, the dealloc function is called. In it, you should
DECREF all object to which you keep references in your object, but you
should not use DECREF on your object. You should use DEL instead.
\chapter{Embedding Python in another application}
Embedding Python is similar to extending it, but not quite. The
difference is that when you extend Python, the main program of the
application is still the Python interpreter, while of you embed
Python, the main program may have nothing to do with Python --
Python, the main program may have nothing to do with Python ---
instead, some parts of the application occasionally call the Python
interpreter to run some Python code.
So if you are embedding Python, you are providing your own main
program. One of the things this main program has to do is initialize
the Python interpreter. At the very least, you have to call the
function initall(). There are optional calls to pass command line
arguments to Python. Then later you can call the interpreter from any
part of the application.
function \code{initall()}. There are optional calls to pass command
line arguments to Python. Then later you can call the interpreter
from any part of the application.
There are several different ways to call the interpreter: you can pass
a string containing Python statements to run_command(), or you can
pass a stdio file pointer and a file name (for identification in error
messages only) to run_script(). You can also call the lower-level
operations described (partly) in the file \verb\<pythonroot>/misc/EXTENDING\
to construct and use Python objects.
a string containing Python statements to \code{run_command()}, or you
can pass a stdio file pointer and a file name (for identification in
error messages only) to \code{run_script()}. You can also call the
lower-level operations described in the previous chapters to construct
and use Python objects.
A simple demo of embedding Python can be found in the directory
\verb\<pythonroot>/embed/\.
\file{<pythonroot>/embed}.
\section{Using C++}
It is also possible to embed Python in a C++ program; how this is done
exactly will depend on the details of the C++ system used; in general
you will need to write the main program in C++, enclosing the include
files in \verb\"extern "C" { ... }"\, and compile and link this with
the C++ compiler. (There is no need to recompile Python itself with
C++.)
you will need to write the main program in C++, and use the C++
compiler to compile and link your program. There is no need to
recompile Python itself with C++.
\input{ext.ind}

4
Doc/lib.tex
View File

@ -7,8 +7,8 @@
\author{
Guido van Rossum \\
Dept. CST, CWI, Kruislaan 413 \\
1098 SJ Amsterdam, The Netherlands \\
Dept. CST, CWI, P.O. Box 94079 \\
1090 GB Amsterdam, The Netherlands \\
E-mail: {\tt guido@cwi.nl}
}

4
Doc/lib/lib.tex
View File

@ -7,8 +7,8 @@
\author{
Guido van Rossum \\
Dept. CST, CWI, Kruislaan 413 \\
1098 SJ Amsterdam, The Netherlands \\
Dept. CST, CWI, P.O. Box 94079 \\
1090 GB Amsterdam, The Netherlands \\
E-mail: {\tt guido@cwi.nl}
}

4
Doc/qua.tex
View File

@ -6,8 +6,8 @@ Interactively Testing Remote Servers Using the Python Programming Language
\author{
Guido van Rossum \\
CWI, dept. CST; Kruislaan 413 \\
1098 SJ Amsterdam, The Netherlands \\
Dept. CST, CWI, P.O. Box 94079 \\
1090 GB Amsterdam, The Netherlands \\
E-mail: {\tt guido@cwi.nl}
\and
Jelke de Boer \\

4
Doc/ref.tex
View File

@ -4,8 +4,8 @@
\author{
Guido van Rossum \\
Dept. CST, CWI, Kruislaan 413 \\
1098 SJ Amsterdam, The Netherlands \\
Dept. CST, CWI, P.O. Box 94079 \\
1090 GB Amsterdam, The Netherlands \\
E-mail: {\tt guido@cwi.nl}
}

4
Doc/ref/ref.tex
View File

@ -4,8 +4,8 @@
\author{
Guido van Rossum \\
Dept. CST, CWI, Kruislaan 413 \\
1098 SJ Amsterdam, The Netherlands \\
Dept. CST, CWI, P.O. Box 94079 \\
1090 GB Amsterdam, The Netherlands \\
E-mail: {\tt guido@cwi.nl}
}

7
Doc/text2latex.py
View File

@ -1,6 +1,7 @@
import os
import sys
import regex
import regsub
import string
import getopt
@ -31,6 +32,9 @@ def process(fi, fo):
if fmt:
nextline = '\n'
line = fmt % string.strip(line)
if '(' in line:
line = regsub.gsub('[a-zA-Z0-9_]+()',
'{\\\\tt \\0}', line)
elif inverbatim:
if blank.match(line) >= 0 and \
indented.match(nextline) < 0:
@ -43,6 +47,9 @@ def process(fi, fo):
fo.write('\\begin{verbatim}\n')
if inverbatim:
line = string.expandtabs(line, 4)
elif not fmt and '(' in line:
line = regsub.gsub('[a-zA-Z0-9_]+()',
'\\\\code{\\0}', line)
fo.write(line)
#main()

7
Doc/tools/text2latex.py
View File

@ -1,6 +1,7 @@
import os
import sys
import regex
import regsub
import string
import getopt
@ -31,6 +32,9 @@ def process(fi, fo):
if fmt:
nextline = '\n'
line = fmt % string.strip(line)
if '(' in line:
line = regsub.gsub('[a-zA-Z0-9_]+()',
'{\\\\tt \\0}', line)
elif inverbatim:
if blank.match(line) >= 0 and \
indented.match(nextline) < 0:
@ -43,6 +47,9 @@ def process(fi, fo):
fo.write('\\begin{verbatim}\n')
if inverbatim:
line = string.expandtabs(line, 4)
elif not fmt and '(' in line:
line = regsub.gsub('[a-zA-Z0-9_]+()',
'\\\\code{\\0}', line)
fo.write(line)
#main()

4
Doc/tut.tex
View File

@ -6,8 +6,8 @@
\author{
Guido van Rossum \\
Dept. CST, CWI, Kruislaan 413 \\
1098 SJ Amsterdam, The Netherlands \\
Dept. CST, CWI, P.O. Box 94079 \\
1090 GB Amsterdam, The Netherlands \\
E-mail: {\tt guido@cwi.nl}
}

4
Doc/tut/tut.tex
View File

@ -6,8 +6,8 @@
\author{
Guido van Rossum \\
Dept. CST, CWI, Kruislaan 413 \\
1098 SJ Amsterdam, The Netherlands \\
Dept. CST, CWI, P.O. Box 94079 \\
1090 GB Amsterdam, The Netherlands \\
E-mail: {\tt guido@cwi.nl}
}