500 lines
19 KiB
TeX
500 lines
19 KiB
TeX
\section{\module{cgi} ---
|
|
Common Gateway Interface support.}
|
|
\declaremodule{standard}{cgi}
|
|
|
|
\modulesynopsis{Common Gateway Interface support, used to interpret
|
|
forms in server-side scripts.}
|
|
|
|
\indexii{WWW}{server}
|
|
\indexii{CGI}{protocol}
|
|
\indexii{HTTP}{protocol}
|
|
\indexii{MIME}{headers}
|
|
\index{URL}
|
|
|
|
|
|
Support module for CGI (Common Gateway Interface) scripts.%
|
|
\index{Common Gateway Interface}
|
|
|
|
This module defines a number of utilities for use by CGI scripts
|
|
written in Python.
|
|
|
|
\subsection{Introduction}
|
|
\nodename{cgi-intro}
|
|
|
|
A CGI script is invoked by an HTTP server, usually to process user
|
|
input submitted through an HTML \code{<FORM>} or \code{<ISINDEX>} element.
|
|
|
|
Most often, CGI scripts live in the server's special \file{cgi-bin}
|
|
directory. The HTTP server places all sorts of information about the
|
|
request (such as the client's hostname, the requested URL, the query
|
|
string, and lots of other goodies) in the script's shell environment,
|
|
executes the script, and sends the script's output back to the client.
|
|
|
|
The script's input is connected to the client too, and sometimes the
|
|
form data is read this way; at other times the form data is passed via
|
|
the ``query string'' part of the URL. This module is intended
|
|
to take care of the different cases and provide a simpler interface to
|
|
the Python script. It also provides a number of utilities that help
|
|
in debugging scripts, and the latest addition is support for file
|
|
uploads from a form (if your browser supports it --- Grail 0.3 and
|
|
Netscape 2.0 do).
|
|
|
|
The output of a CGI script should consist of two sections, separated
|
|
by a blank line. The first section contains a number of headers,
|
|
telling the client what kind of data is following. Python code to
|
|
generate a minimal header section looks like this:
|
|
|
|
\begin{verbatim}
|
|
print "Content-type: text/html" # HTML is following
|
|
print # blank line, end of headers
|
|
\end{verbatim}
|
|
|
|
The second section is usually HTML, which allows the client software
|
|
to display nicely formatted text with header, in-line images, etc.
|
|
Here's Python code that prints a simple piece of HTML:
|
|
|
|
\begin{verbatim}
|
|
print "<TITLE>CGI script output</TITLE>"
|
|
print "<H1>This is my first CGI script</H1>"
|
|
print "Hello, world!"
|
|
\end{verbatim}
|
|
|
|
(It may not be fully legal HTML according to the letter of the
|
|
standard, but any browser will understand it.)
|
|
|
|
\subsection{Using the cgi module}
|
|
\nodename{Using the cgi module}
|
|
|
|
Begin by writing \samp{import cgi}. Do not use \samp{from cgi import
|
|
*} --- the module defines all sorts of names for its own use or for
|
|
backward compatibility that you don't want in your namespace.
|
|
|
|
It's best to use the \class{FieldStorage} class. The other classes
|
|
defined in this module are provided mostly for backward compatibility.
|
|
Instantiate it exactly once, without arguments. This reads the form
|
|
contents from standard input or the environment (depending on the
|
|
value of various environment variables set according to the CGI
|
|
standard). Since it may consume standard input, it should be
|
|
instantiated only once.
|
|
|
|
The \class{FieldStorage} instance can be accessed as if it were a Python
|
|
dictionary. For instance, the following code (which assumes that the
|
|
\code{content-type} header and blank line have already been printed)
|
|
checks that the fields \code{name} and \code{addr} are both set to a
|
|
non-empty string:
|
|
|
|
\begin{verbatim}
|
|
form = cgi.FieldStorage()
|
|
form_ok = 0
|
|
if form.has_key("name") and form.has_key("addr"):
|
|
if form["name"].value != "" and form["addr"].value != "":
|
|
form_ok = 1
|
|
if not form_ok:
|
|
print "<H1>Error</H1>"
|
|
print "Please fill in the name and addr fields."
|
|
return
|
|
...further form processing here...
|
|
\end{verbatim}
|
|
|
|
Here the fields, accessed through \samp{form[\var{key}]}, are
|
|
themselves instances of \class{FieldStorage} (or
|
|
\class{MiniFieldStorage}, depending on the form encoding).
|
|
|
|
If the submitted form data contains more than one field with the same
|
|
name, the object retrieved by \samp{form[\var{key}]} is not a
|
|
\class{FieldStorage} or \class{MiniFieldStorage}
|
|
instance but a list of such instances. If you expect this possibility
|
|
(i.e., when your HTML form comtains multiple fields with the same
|
|
name), use the \function{type()} function to determine whether you
|
|
have a single instance or a list of instances. For example, here's
|
|
code that concatenates any number of username fields, separated by
|
|
commas:
|
|
|
|
\begin{verbatim}
|
|
username = form["username"]
|
|
if type(username) is type([]):
|
|
# Multiple username fields specified
|
|
usernames = ""
|
|
for item in username:
|
|
if usernames:
|
|
# Next item -- insert comma
|
|
usernames = usernames + "," + item.value
|
|
else:
|
|
# First item -- don't insert comma
|
|
usernames = item.value
|
|
else:
|
|
# Single username field specified
|
|
usernames = username.value
|
|
\end{verbatim}
|
|
|
|
If a field represents an uploaded file, the value attribute reads the
|
|
entire file in memory as a string. This may not be what you want.
|
|
You can test for an uploaded file by testing either the filename
|
|
attribute or the file attribute. You can then read the data at
|
|
leasure from the file attribute:
|
|
|
|
\begin{verbatim}
|
|
fileitem = form["userfile"]
|
|
if fileitem.file:
|
|
# It's an uploaded file; count lines
|
|
linecount = 0
|
|
while 1:
|
|
line = fileitem.file.readline()
|
|
if not line: break
|
|
linecount = linecount + 1
|
|
\end{verbatim}
|
|
|
|
The file upload draft standard entertains the possibility of uploading
|
|
multiple files from one field (using a recursive
|
|
\mimetype{multipart/*} encoding). When this occurs, the item will be
|
|
a dictionary-like \class{FieldStorage} item. This can be determined
|
|
by testing its \member{type} attribute, which should be
|
|
\mimetype{multipart/form-data} (or perhaps another MIME type matching
|
|
\mimetype{multipart/*}). In this case, it can be iterated over
|
|
recursively just like the top-level form object.
|
|
|
|
When a form is submitted in the ``old'' format (as the query string or
|
|
as a single data part of type
|
|
\mimetype{application/x-www-form-urlencoded}), the items will actually
|
|
be instances of the class \class{MiniFieldStorage}. In this case, the
|
|
list, file and filename attributes are always \code{None}.
|
|
|
|
|
|
\subsection{Old classes}
|
|
|
|
These classes, present in earlier versions of the \module{cgi} module,
|
|
are still supported for backward compatibility. New applications
|
|
should use the \class{FieldStorage} class.
|
|
|
|
\class{SvFormContentDict} stores single value form content as
|
|
dictionary; it assumes each field name occurs in the form only once.
|
|
|
|
\class{FormContentDict} stores multiple value form content as a
|
|
dictionary (the form items are lists of values). Useful if your form
|
|
contains multiple fields with the same name.
|
|
|
|
Other classes (\class{FormContent}, \class{InterpFormContentDict}) are
|
|
present for backwards compatibility with really old applications only.
|
|
If you still use these and would be inconvenienced when they
|
|
disappeared from a next version of this module, drop me a note.
|
|
|
|
|
|
\subsection{Functions}
|
|
\nodename{Functions in cgi module}
|
|
|
|
These are useful if you want more control, or if you want to employ
|
|
some of the algorithms implemented in this module in other
|
|
circumstances.
|
|
|
|
\begin{funcdesc}{parse}{fp}
|
|
Parse a query in the environment or from a file (default
|
|
\code{sys.stdin}).
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{parse_qs}{qs\optional{, keep_blank_values, strict_parsing}}
|
|
Parse a query string given as a string argument (data of type
|
|
\mimetype{application/x-www-form-urlencoded}). Data are
|
|
returned as a dictionary. The dictionary keys are the unique query
|
|
variable names and the values are lists of vales for each name.
|
|
|
|
The optional argument \var{keep_blank_values} is
|
|
a flag indicating whether blank values in
|
|
URL encoded queries should be treated as blank strings.
|
|
A true value indicates that blanks should be retained as
|
|
blank strings. The default false value indicates that
|
|
blank values are to be ignored and treated as if they were
|
|
not included.
|
|
|
|
The optional argument \var{strict_parsing} is a flag indicating what
|
|
to do with parsing errors. If false (the default), errors
|
|
are silently ignored. If true, errors raise a ValueError
|
|
exception.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{parse_qsl}{qs\optional{, keep_blank_values, strict_parsing}}
|
|
Parse a query string given as a string argument (data of type
|
|
\mimetype{application/x-www-form-urlencoded}). Data are
|
|
returned as a list of name, value pairs.
|
|
|
|
The optional argument \var{keep_blank_values} is
|
|
a flag indicating whether blank values in
|
|
URL encoded queries should be treated as blank strings.
|
|
A true value indicates that blanks should be retained as
|
|
blank strings. The default false value indicates that
|
|
blank values are to be ignored and treated as if they were
|
|
not included.
|
|
|
|
The optional argument \var{strict_parsing} is a flag indicating what
|
|
to do with parsing errors. If false (the default), errors
|
|
are silently ignored. If true, errors raise a ValueError
|
|
exception.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{parse_multipart}{fp, pdict}
|
|
Parse input of type \mimetype{multipart/form-data} (for
|
|
file uploads). Arguments are \var{fp} for the input file and
|
|
\var{pdict} for the dictionary containing other parameters of
|
|
\code{content-type} header
|
|
|
|
Returns a dictionary just like \function{parse_qs()} keys are the
|
|
field names, each value is a list of values for that field. This is
|
|
easy to use but not much good if you are expecting megabytes to be
|
|
uploaded --- in that case, use the \class{FieldStorage} class instead
|
|
which is much more flexible. Note that \code{content-type} is the
|
|
raw, unparsed contents of the \code{content-type} header.
|
|
|
|
Note that this does not parse nested multipart parts --- use
|
|
\class{FieldStorage} for that.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{parse_header}{string}
|
|
Parse a header like \code{content-type} into a main
|
|
content-type and a dictionary of parameters.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{test}{}
|
|
Robust test CGI script, usable as main program.
|
|
Writes minimal HTTP headers and formats all information provided to
|
|
the script in HTML form.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{print_environ}{}
|
|
Format the shell environment in HTML.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{print_form}{form}
|
|
Format a form in HTML.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{print_directory}{}
|
|
Format the current directory in HTML.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{print_environ_usage}{}
|
|
Print a list of useful (used by CGI) environment variables in
|
|
HTML.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{escape}{s\optional{, quote}}
|
|
Convert the characters
|
|
\character{\&}, \character{<} and \character{>} in string \var{s} to
|
|
HTML-safe sequences. Use this if you need to display text that might
|
|
contain such characters in HTML. If the optional flag \var{quote} is
|
|
true, the double quote character (\character{"}) is also translated;
|
|
this helps for inclusion in an HTML attribute value, e.g. in \code{<A
|
|
HREF="...">}.
|
|
\end{funcdesc}
|
|
|
|
|
|
\subsection{Caring about security}
|
|
|
|
There's one important rule: if you invoke an external program (e.g.
|
|
via the \function{os.system()} or \function{os.popen()} functions),
|
|
make very sure you don't pass arbitrary strings received from the
|
|
client to the shell. This is a well-known security hole whereby
|
|
clever hackers anywhere on the web can exploit a gullible CGI script
|
|
to invoke arbitrary shell commands. Even parts of the URL or field
|
|
names cannot be trusted, since the request doesn't have to come from
|
|
your form!
|
|
|
|
To be on the safe side, if you must pass a string gotten from a form
|
|
to a shell command, you should make sure the string contains only
|
|
alphanumeric characters, dashes, underscores, and periods.
|
|
|
|
|
|
\subsection{Installing your CGI script on a Unix system}
|
|
|
|
Read the documentation for your HTTP server and check with your local
|
|
system administrator to find the directory where CGI scripts should be
|
|
installed; usually this is in a directory \file{cgi-bin} in the server tree.
|
|
|
|
Make sure that your script is readable and executable by ``others''; the
|
|
\UNIX{} file mode should be \code{0755} octal (use \samp{chmod 0755
|
|
\var{filename}}). Make sure that the first line of the script contains
|
|
\code{\#!} starting in column 1 followed by the pathname of the Python
|
|
interpreter, for instance:
|
|
|
|
\begin{verbatim}
|
|
#!/usr/local/bin/python
|
|
\end{verbatim}
|
|
|
|
Make sure the Python interpreter exists and is executable by ``others''.
|
|
|
|
Make sure that any files your script needs to read or write are
|
|
readable or writable, respectively, by ``others'' --- their mode
|
|
should be \code{0644} for readable and \code{0666} for writable. This
|
|
is because, for security reasons, the HTTP server executes your script
|
|
as user ``nobody'', without any special privileges. It can only read
|
|
(write, execute) files that everybody can read (write, execute). The
|
|
current directory at execution time is also different (it is usually
|
|
the server's cgi-bin directory) and the set of environment variables
|
|
is also different from what you get at login. In particular, don't
|
|
count on the shell's search path for executables (\envvar{PATH}) or
|
|
the Python module search path (\envvar{PYTHONPATH}) to be set to
|
|
anything interesting.
|
|
|
|
If you need to load modules from a directory which is not on Python's
|
|
default module search path, you can change the path in your script,
|
|
before importing other modules, e.g.:
|
|
|
|
\begin{verbatim}
|
|
import sys
|
|
sys.path.insert(0, "/usr/home/joe/lib/python")
|
|
sys.path.insert(0, "/usr/local/lib/python")
|
|
\end{verbatim}
|
|
|
|
(This way, the directory inserted last will be searched first!)
|
|
|
|
Instructions for non-\UNIX{} systems will vary; check your HTTP server's
|
|
documentation (it will usually have a section on CGI scripts).
|
|
|
|
|
|
\subsection{Testing your CGI script}
|
|
|
|
Unfortunately, a CGI script will generally not run when you try it
|
|
from the command line, and a script that works perfectly from the
|
|
command line may fail mysteriously when run from the server. There's
|
|
one reason why you should still test your script from the command
|
|
line: if it contains a syntax error, the Python interpreter won't
|
|
execute it at all, and the HTTP server will most likely send a cryptic
|
|
error to the client.
|
|
|
|
Assuming your script has no syntax errors, yet it does not work, you
|
|
have no choice but to read the next section.
|
|
|
|
|
|
\subsection{Debugging CGI scripts}
|
|
|
|
First of all, check for trivial installation errors --- reading the
|
|
section above on installing your CGI script carefully can save you a
|
|
lot of time. If you wonder whether you have understood the
|
|
installation procedure correctly, try installing a copy of this module
|
|
file (\file{cgi.py}) as a CGI script. When invoked as a script, the file
|
|
will dump its environment and the contents of the form in HTML form.
|
|
Give it the right mode etc, and send it a request. If it's installed
|
|
in the standard \file{cgi-bin} directory, it should be possible to send it a
|
|
request by entering a URL into your browser of the form:
|
|
|
|
\begin{verbatim}
|
|
http://yourhostname/cgi-bin/cgi.py?name=Joe+Blow&addr=At+Home
|
|
\end{verbatim}
|
|
|
|
If this gives an error of type 404, the server cannot find the script
|
|
-- perhaps you need to install it in a different directory. If it
|
|
gives another error (e.g. 500), there's an installation problem that
|
|
you should fix before trying to go any further. If you get a nicely
|
|
formatted listing of the environment and form content (in this
|
|
example, the fields should be listed as ``addr'' with value ``At Home''
|
|
and ``name'' with value ``Joe Blow''), the \file{cgi.py} script has been
|
|
installed correctly. If you follow the same procedure for your own
|
|
script, you should now be able to debug it.
|
|
|
|
The next step could be to call the \module{cgi} module's
|
|
\function{test()} function from your script: replace its main code
|
|
with the single statement
|
|
|
|
\begin{verbatim}
|
|
cgi.test()
|
|
\end{verbatim}
|
|
|
|
This should produce the same results as those gotten from installing
|
|
the \file{cgi.py} file itself.
|
|
|
|
When an ordinary Python script raises an unhandled exception
|
|
(e.g. because of a typo in a module name, a file that can't be opened,
|
|
etc.), the Python interpreter prints a nice traceback and exits.
|
|
While the Python interpreter will still do this when your CGI script
|
|
raises an exception, most likely the traceback will end up in one of
|
|
the HTTP server's log file, or be discarded altogether.
|
|
|
|
Fortunately, once you have managed to get your script to execute
|
|
\emph{some} code, it is easy to catch exceptions and cause a traceback
|
|
to be printed. The \function{test()} function below in this module is
|
|
an example. Here are the rules:
|
|
|
|
\begin{enumerate}
|
|
\item Import the traceback module before entering the \keyword{try}
|
|
... \keyword{except} statement
|
|
|
|
\item Assign \code{sys.stderr} to be \code{sys.stdout}
|
|
|
|
\item Make sure you finish printing the headers and the blank line
|
|
early
|
|
|
|
\item Wrap all remaining code in a \keyword{try} ... \keyword{except}
|
|
statement
|
|
|
|
\item In the except clause, call \function{traceback.print_exc()}
|
|
\end{enumerate}
|
|
|
|
For example:
|
|
|
|
\begin{verbatim}
|
|
import sys
|
|
import traceback
|
|
print "Content-type: text/html"
|
|
print
|
|
sys.stderr = sys.stdout
|
|
try:
|
|
...your code here...
|
|
except:
|
|
print "\n\n<PRE>"
|
|
traceback.print_exc()
|
|
\end{verbatim}
|
|
|
|
Notes: The assignment to \code{sys.stderr} is needed because the
|
|
traceback prints to \code{sys.stderr}.
|
|
The \code{print "{\e}n{\e}n<PRE>"} statement is necessary to
|
|
disable the word wrapping in HTML.
|
|
|
|
If you suspect that there may be a problem in importing the traceback
|
|
module, you can use an even more robust approach (which only uses
|
|
built-in modules):
|
|
|
|
\begin{verbatim}
|
|
import sys
|
|
sys.stderr = sys.stdout
|
|
print "Content-type: text/plain"
|
|
print
|
|
...your code here...
|
|
\end{verbatim}
|
|
|
|
This relies on the Python interpreter to print the traceback. The
|
|
content type of the output is set to plain text, which disables all
|
|
HTML processing. If your script works, the raw HTML will be displayed
|
|
by your client. If it raises an exception, most likely after the
|
|
first two lines have been printed, a traceback will be displayed.
|
|
Because no HTML interpretation is going on, the traceback will
|
|
readable.
|
|
|
|
|
|
\subsection{Common problems and solutions}
|
|
|
|
\begin{itemize}
|
|
\item Most HTTP servers buffer the output from CGI scripts until the
|
|
script is completed. This means that it is not possible to display a
|
|
progress report on the client's display while the script is running.
|
|
|
|
\item Check the installation instructions above.
|
|
|
|
\item Check the HTTP server's log files. (\samp{tail -f logfile} in a
|
|
separate window may be useful!)
|
|
|
|
\item Always check a script for syntax errors first, by doing something
|
|
like \samp{python script.py}.
|
|
|
|
\item When using any of the debugging techniques, don't forget to add
|
|
\samp{import sys} to the top of the script.
|
|
|
|
\item When invoking external programs, make sure they can be found.
|
|
Usually, this means using absolute path names --- \envvar{PATH} is
|
|
usually not set to a very useful value in a CGI script.
|
|
|
|
\item When reading or writing external files, make sure they can be read
|
|
or written by every user on the system.
|
|
|
|
\item Don't try to give a CGI script a set-uid mode. This doesn't work on
|
|
most systems, and is a security liability as well.
|
|
\end{itemize}
|
|
|