1104 lines
44 KiB
TeX
1104 lines
44 KiB
TeX
\documentclass{howto}
|
|
\usepackage{distutils}
|
|
|
|
% TODO:
|
|
% Fill in XXX comments
|
|
|
|
\title{Installing Python Modules}
|
|
|
|
% The audience for this document includes people who don't know anything
|
|
% about Python and aren't about to learn the language just in order to
|
|
% install and maintain it for their users, i.e. system administrators.
|
|
% Thus, I have to be sure to explain the basics at some point:
|
|
% sys.path and PYTHONPATH at least. Should probably give pointers to
|
|
% other docs on "import site", PYTHONSTARTUP, PYTHONHOME, etc.
|
|
%
|
|
% Finally, it might be useful to include all the material from my "Care
|
|
% and Feeding of a Python Installation" talk in here somewhere. Yow!
|
|
|
|
\input{boilerplate}
|
|
|
|
\author{Greg Ward}
|
|
\authoraddress{
|
|
\strong{Python Software Foundation}\\
|
|
Email: \email{distutils-sig@python.org}
|
|
}
|
|
|
|
\makeindex
|
|
|
|
\begin{document}
|
|
|
|
\maketitle
|
|
|
|
\begin{abstract}
|
|
\noindent
|
|
This document describes the Python Distribution Utilities
|
|
(``Distutils'') from the end-user's point-of-view, describing how to
|
|
extend the capabilities of a standard Python installation by building
|
|
and installing third-party Python modules and extensions.
|
|
\end{abstract}
|
|
|
|
%\begin{abstract}
|
|
%\noindent
|
|
%Abstract this!
|
|
%\end{abstract}
|
|
|
|
|
|
% The ugly "%begin{latexonly}" pseudo-environment suppresses the table
|
|
% of contents for HTML generation.
|
|
%
|
|
%begin{latexonly}
|
|
\tableofcontents
|
|
%end{latexonly}
|
|
|
|
|
|
\section{Introduction}
|
|
\label{intro}
|
|
|
|
Although Python's extensive standard library covers many programming
|
|
needs, there often comes a time when you need to add some new
|
|
functionality to your Python installation in the form of third-party
|
|
modules. This might be necessary to support your own programming, or to
|
|
support an application that you want to use and that happens to be
|
|
written in Python.
|
|
|
|
In the past, there has been little support for adding third-party
|
|
modules to an existing Python installation. With the introduction of
|
|
the Python Distribution Utilities (Distutils for short) in Python 2.0,
|
|
this changed.
|
|
|
|
This document is aimed primarily at the people who need to install
|
|
third-party Python modules: end-users and system administrators who just
|
|
need to get some Python application running, and existing Python
|
|
programmers who want to add some new goodies to their toolbox. You
|
|
don't need to know Python to read this document; there will be some
|
|
brief forays into using Python's interactive mode to explore your
|
|
installation, but that's it. If you're looking for information on how
|
|
to distribute your own Python modules so that others may use them, see
|
|
the \citetitle[../dist/dist.html]{Distributing Python Modules} manual.
|
|
|
|
|
|
\subsection{Best case: trivial installation}
|
|
\label{trivial-install}
|
|
|
|
In the best case, someone will have prepared a special version of the
|
|
module distribution you want to install that is targeted specifically at
|
|
your platform and is installed just like any other software on your
|
|
platform. For example, the module developer might make an executable
|
|
installer available for Windows users, an RPM package for users of
|
|
RPM-based Linux systems (Red Hat, SuSE, Mandrake, and many others), a
|
|
Debian package for users of Debian-based Linux systems, and so forth.
|
|
|
|
In that case, you would download the installer appropriate to your
|
|
platform and do the obvious thing with it: run it if it's an executable
|
|
installer, \code{rpm --install} it if it's an RPM, etc. You don't need
|
|
to run Python or a setup script, you don't need to compile
|
|
anything---you might not even need to read any instructions (although
|
|
it's always a good idea to do so anyways).
|
|
|
|
Of course, things will not always be that easy. You might be interested
|
|
in a module distribution that doesn't have an easy-to-use installer for
|
|
your platform. In that case, you'll have to start with the source
|
|
distribution released by the module's author/maintainer. Installing
|
|
from a source distribution is not too hard, as long as the modules are
|
|
packaged in the standard way. The bulk of this document is about
|
|
building and installing modules from standard source distributions.
|
|
|
|
|
|
\subsection{The new standard: Distutils}
|
|
\label{new-standard}
|
|
|
|
If you download a module source distribution, you can tell pretty
|
|
quickly if it was packaged and distributed in the standard way, i.e.
|
|
using the Distutils. First, the distribution's name and version number
|
|
will be featured prominently in the name of the downloaded archive, e.g.
|
|
\file{foo-1.0.tar.gz} or \file{widget-0.9.7.zip}. Next, the archive
|
|
will unpack into a similarly-named directory: \file{foo-1.0} or
|
|
\file{widget-0.9.7}. Additionally, the distribution will contain a
|
|
setup script \file{setup.py}, and a file named \file{README.txt} or possibly
|
|
just \file{README}, which should explain that building and installing the
|
|
module distribution is a simple matter of running
|
|
|
|
\begin{verbatim}
|
|
python setup.py install
|
|
\end{verbatim}
|
|
|
|
If all these things are true, then you already know how to build and
|
|
install the modules you've just downloaded: Run the command above.
|
|
Unless you need to install things in a non-standard way or customize the
|
|
build process, you don't really need this manual. Or rather, the above
|
|
command is everything you need to get out of this manual.
|
|
|
|
|
|
\section{Standard Build and Install}
|
|
\label{standard-install}
|
|
|
|
As described in section~\ref{new-standard}, building and installing
|
|
a module distribution using the Distutils is usually one simple command:
|
|
|
|
\begin{verbatim}
|
|
python setup.py install
|
|
\end{verbatim}
|
|
|
|
On \UNIX, you'd run this command from a shell prompt; on Windows, you
|
|
have to open a command prompt window (``DOS box'') and do it there; on
|
|
Mac OS X, you open a \command{Terminal} window to get a shell prompt.
|
|
|
|
|
|
\subsection{Platform variations}
|
|
\label{platform-variations}
|
|
|
|
You should always run the setup command from the distribution root
|
|
directory, i.e. the top-level subdirectory that the module source
|
|
distribution unpacks into. For example, if you've just downloaded a
|
|
module source distribution \file{foo-1.0.tar.gz} onto a
|
|
\UNIX{} system, the normal thing to do is:
|
|
|
|
\begin{verbatim}
|
|
gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0
|
|
cd foo-1.0
|
|
python setup.py install
|
|
\end{verbatim}
|
|
|
|
On Windows, you'd probably download \file{foo-1.0.zip}. If you
|
|
downloaded the archive file to \file{C:\textbackslash{}Temp}, then it
|
|
would unpack into \file{C:\textbackslash{}Temp\textbackslash{}foo-1.0};
|
|
you can use either a archive manipulator with a graphical user interface
|
|
(such as WinZip) or a command-line tool (such as \program{unzip} or
|
|
\program{pkunzip}) to unpack the archive. Then, open a command prompt
|
|
window (``DOS box''), and run:
|
|
|
|
\begin{verbatim}
|
|
cd c:\Temp\foo-1.0
|
|
python setup.py install
|
|
\end{verbatim}
|
|
|
|
\subsection{Splitting the job up}
|
|
\label{splitting-up}
|
|
|
|
Running \code{setup.py install} builds and installs all modules in one
|
|
run. If you prefer to work incrementally---especially useful if you
|
|
want to customize the build process, or if things are going wrong---you
|
|
can use the setup script to do one thing at a time. This is
|
|
particularly helpful when the build and install will be done by
|
|
different users---for example, you might want to build a module distribution
|
|
and hand it off to a system administrator for installation (or do it
|
|
yourself, with super-user privileges).
|
|
|
|
For example, you can build everything in one step, and then install
|
|
everything in a second step, by invoking the setup script twice:
|
|
|
|
\begin{verbatim}
|
|
python setup.py build
|
|
python setup.py install
|
|
\end{verbatim}
|
|
|
|
If you do this, you will notice that running the \command{install}
|
|
command first runs the \command{build} command, which---in this
|
|
case---quickly notices that it has nothing to do, since everything in
|
|
the \file{build} directory is up-to-date.
|
|
|
|
You may not need this ability to break things down often if all you do
|
|
is install modules downloaded off the 'net, but it's very handy for more
|
|
advanced tasks. If you get into distributing your own Python modules
|
|
and extensions, you'll run lots of individual Distutils commands on
|
|
their own.
|
|
|
|
|
|
\subsection{How building works}
|
|
\label{how-build-works}
|
|
|
|
As implied above, the \command{build} command is responsible for putting
|
|
the files to install into a \emph{build directory}. By default, this is
|
|
\file{build} under the distribution root; if you're excessively
|
|
concerned with speed, or want to keep the source tree pristine, you can
|
|
change the build directory with the \longprogramopt{build-base} option.
|
|
For example:
|
|
|
|
\begin{verbatim}
|
|
python setup.py build --build-base=/tmp/pybuild/foo-1.0
|
|
\end{verbatim}
|
|
|
|
(Or you could do this permanently with a directive in your system or
|
|
personal Distutils configuration file; see
|
|
section~\ref{config-files}.) Normally, this isn't necessary.
|
|
|
|
The default layout for the build tree is as follows:
|
|
|
|
\begin{verbatim}
|
|
--- build/ --- lib/
|
|
or
|
|
--- build/ --- lib.<plat>/
|
|
temp.<plat>/
|
|
\end{verbatim}
|
|
|
|
where \code{<plat>} expands to a brief description of the current
|
|
OS/hardware platform and Python version. The first form, with just a
|
|
\file{lib} directory, is used for ``pure module distributions''---that
|
|
is, module distributions that include only pure Python modules. If a
|
|
module distribution contains any extensions (modules written in C/\Cpp),
|
|
then the second form, with two \code{<plat>} directories, is used. In
|
|
that case, the \file{temp.\filevar{plat}} directory holds temporary
|
|
files generated by the compile/link process that don't actually get
|
|
installed. In either case, the \file{lib} (or
|
|
\file{lib.\filevar{plat}}) directory contains all Python modules (pure
|
|
Python and extensions) that will be installed.
|
|
|
|
In the future, more directories will be added to handle Python scripts,
|
|
documentation, binary executables, and whatever else is needed to handle
|
|
the job of installing Python modules and applications.
|
|
|
|
|
|
\subsection{How installation works}
|
|
\label{how-install-works}
|
|
|
|
After the \command{build} command runs (whether you run it explicitly,
|
|
or the \command{install} command does it for you), the work of the
|
|
\command{install} command is relatively simple: all it has to do is copy
|
|
everything under \file{build/lib} (or \file{build/lib.\filevar{plat}})
|
|
to your chosen installation directory.
|
|
|
|
If you don't choose an installation directory---i.e., if you just run
|
|
\code{setup.py install}---then the \command{install} command installs to
|
|
the standard location for third-party Python modules. This location
|
|
varies by platform and by how you built/installed Python itself. On
|
|
\UNIX{} (and Mac OS X, which is also Unix-based),
|
|
it also depends on whether the module distribution
|
|
being installed is pure Python or contains extensions (``non-pure''):
|
|
\begin{tableiv}{l|l|l|c}{textrm}%
|
|
{Platform}{Standard installation location}{Default value}{Notes}
|
|
\lineiv{\UNIX{} (pure)}
|
|
{\filenq{\filevar{prefix}/lib/python\shortversion/site-packages}}
|
|
{\filenq{/usr/local/lib/python\shortversion/site-packages}}
|
|
{(1)}
|
|
\lineiv{\UNIX{} (non-pure)}
|
|
{\filenq{\filevar{exec-prefix}/lib/python\shortversion/site-packages}}
|
|
{\filenq{/usr/local/lib/python\shortversion/site-packages}}
|
|
{(1)}
|
|
\lineiv{Windows}
|
|
{\filenq{\filevar{prefix}}}
|
|
{\filenq{C:\textbackslash{}Python}}
|
|
{(2)}
|
|
\end{tableiv}
|
|
|
|
\noindent Notes:
|
|
\begin{description}
|
|
\item[(1)] Most Linux distributions include Python as a standard part of
|
|
the system, so \filevar{prefix} and \filevar{exec-prefix} are usually
|
|
both \file{/usr} on Linux. If you build Python yourself on Linux (or
|
|
any \UNIX-like system), the default \filevar{prefix} and
|
|
\filevar{exec-prefix} are \file{/usr/local}.
|
|
\item[(2)] The default installation directory on Windows was
|
|
\file{C:\textbackslash{}Program Files\textbackslash{}Python} under
|
|
Python 1.6a1, 1.5.2, and earlier.
|
|
\end{description}
|
|
|
|
\filevar{prefix} and \filevar{exec-prefix} stand for the directories
|
|
that Python is installed to, and where it finds its libraries at
|
|
run-time. They are always the same under Windows, and very
|
|
often the same under \UNIX and Mac OS X. You can find out what your Python
|
|
installation uses for \filevar{prefix} and \filevar{exec-prefix} by
|
|
running Python in interactive mode and typing a few simple commands.
|
|
Under \UNIX, just type \code{python} at the shell prompt. Under
|
|
Windows, choose \menuselection{Start \sub Programs \sub Python
|
|
\shortversion \sub Python (command line)}.
|
|
Once the interpreter is started, you type Python code at the
|
|
prompt. For example, on my Linux system, I type the three Python
|
|
statements shown below, and get the output as shown, to find out my
|
|
\filevar{prefix} and \filevar{exec-prefix}:
|
|
|
|
\begin{verbatim}
|
|
Python 2.4 (#26, Aug 7 2004, 17:19:02)
|
|
Type "help", "copyright", "credits" or "license" for more information.
|
|
>>> import sys
|
|
>>> sys.prefix
|
|
'/usr'
|
|
>>> sys.exec_prefix
|
|
'/usr'
|
|
\end{verbatim}
|
|
|
|
If you don't want to install modules to the standard location, or if you
|
|
don't have permission to write there, then you need to read about
|
|
alternate installations in section~\ref{alt-install}. If you want to
|
|
customize your installation directories more heavily, see
|
|
section~\ref{custom-install} on custom installations.
|
|
|
|
|
|
% This rather nasty macro is used to generate the tables that describe
|
|
% each installation scheme. It's nasty because it takes two arguments
|
|
% for each "slot" in an installation scheme, there will soon be more
|
|
% than five of these slots, and TeX has a limit of 10 arguments to a
|
|
% macro. Uh-oh.
|
|
|
|
\newcommand{\installscheme}[8]
|
|
{\begin{tableiii}{l|l|l}{textrm}
|
|
{Type of file}
|
|
{Installation Directory}
|
|
{Override option}
|
|
\lineiii{pure module distribution}
|
|
{\filevar{#1}\filenq{#2}}
|
|
{\longprogramopt{install-purelib}}
|
|
\lineiii{non-pure module distribution}
|
|
{\filevar{#3}\filenq{#4}}
|
|
{\longprogramopt{install-platlib}}
|
|
\lineiii{scripts}
|
|
{\filevar{#5}\filenq{#6}}
|
|
{\longprogramopt{install-scripts}}
|
|
\lineiii{data}
|
|
{\filevar{#7}\filenq{#8}}
|
|
{\longprogramopt{install-data}}
|
|
\end{tableiii}}
|
|
|
|
|
|
\section{Alternate Installation}
|
|
\label{alt-install}
|
|
|
|
Often, it is necessary or desirable to install modules to a location
|
|
other than the standard location for third-party Python modules. For
|
|
example, on a \UNIX{} system you might not have permission to write to the
|
|
standard third-party module directory. Or you might wish to try out a
|
|
module before making it a standard part of your local Python
|
|
installation. This is especially true when upgrading a distribution
|
|
already present: you want to make sure your existing base of scripts
|
|
still works with the new version before actually upgrading.
|
|
|
|
The Distutils \command{install} command is designed to make installing
|
|
module distributions to an alternate location simple and painless. The
|
|
basic idea is that you supply a base directory for the installation, and
|
|
the \command{install} command picks a set of directories (called an
|
|
\emph{installation scheme}) under this base directory in which to
|
|
install files. The details differ across platforms, so read whichever
|
|
of the following sections applies to you.
|
|
|
|
|
|
\subsection{Alternate installation: the home scheme}
|
|
\label{alt-install-prefix}
|
|
|
|
The idea behind the ``home scheme'' is that you build and maintain a
|
|
personal stash of Python modules. This scheme's name is derived from
|
|
the idea of a ``home'' directory on \UNIX, since it's not unusual for
|
|
a \UNIX{} user to make their home directory have a layout similar to
|
|
\file{/usr/} or \file{/usr/local/}. This scheme can be used by
|
|
anyone, regardless of the operating system their installing for.
|
|
|
|
Installing a new module distribution is as simple as
|
|
|
|
\begin{verbatim}
|
|
python setup.py install --home=<dir>
|
|
\end{verbatim}
|
|
|
|
where you can supply any directory you like for the
|
|
\longprogramopt{home} option. On \UNIX, lazy typists can just type a
|
|
tilde (\code{\textasciitilde}); the \command{install} command will
|
|
expand this to your home directory:
|
|
|
|
\begin{verbatim}
|
|
python setup.py install --home=~
|
|
\end{verbatim}
|
|
|
|
The \longprogramopt{home} option defines the installation base
|
|
directory. Files are installed to the following directories under the
|
|
installation base as follows:
|
|
\installscheme{home}{/lib/python}
|
|
{home}{/lib/python}
|
|
{home}{/bin}
|
|
{home}{/share}
|
|
|
|
|
|
\versionchanged[The \longprogramopt{home} option used to be supported
|
|
only on \UNIX]{2.4}
|
|
|
|
|
|
\subsection{Alternate installation: \UNIX{} (the prefix scheme)}
|
|
\label{alt-install-home}
|
|
|
|
The ``prefix scheme'' is useful when you wish to use one Python
|
|
installation to perform the build/install (i.e., to run the setup
|
|
script), but install modules into the third-party module directory of a
|
|
different Python installation (or something that looks like a different
|
|
Python installation). If this sounds a trifle unusual, it is---that's
|
|
why the ``home scheme'' comes first. However, there are at least two
|
|
known cases where the prefix scheme will be useful.
|
|
|
|
First, consider that many Linux distributions put Python in \file{/usr},
|
|
rather than the more traditional \file{/usr/local}. This is entirely
|
|
appropriate, since in those cases Python is part of ``the system''
|
|
rather than a local add-on. However, if you are installing Python
|
|
modules from source, you probably want them to go in
|
|
\file{/usr/local/lib/python2.\filevar{X}} rather than
|
|
\file{/usr/lib/python2.\filevar{X}}. This can be done with
|
|
|
|
\begin{verbatim}
|
|
/usr/bin/python setup.py install --prefix=/usr/local
|
|
\end{verbatim}
|
|
|
|
Another possibility is a network filesystem where the name used to write
|
|
to a remote directory is different from the name used to read it: for
|
|
example, the Python interpreter accessed as \file{/usr/local/bin/python}
|
|
might search for modules in \file{/usr/local/lib/python2.\filevar{X}},
|
|
but those modules would have to be installed to, say,
|
|
\file{/mnt/\filevar{@server}/export/lib/python2.\filevar{X}}. This
|
|
could be done with
|
|
|
|
\begin{verbatim}
|
|
/usr/local/bin/python setup.py install --prefix=/mnt/@server/export
|
|
\end{verbatim}
|
|
|
|
In either case, the \longprogramopt{prefix} option defines the
|
|
installation base, and the \longprogramopt{exec-prefix} option defines
|
|
the platform-specific installation base, which is used for
|
|
platform-specific files. (Currently, this just means non-pure module
|
|
distributions, but could be expanded to C libraries, binary executables,
|
|
etc.) If \longprogramopt{exec-prefix} is not supplied, it defaults to
|
|
\longprogramopt{prefix}. Files are installed as follows:
|
|
|
|
\installscheme{prefix}{/lib/python2.\filevar{X}/site-packages}
|
|
{exec-prefix}{/lib/python2.\filevar{X}/site-packages}
|
|
{prefix}{/bin}
|
|
{prefix}{/share}
|
|
|
|
There is no requirement that \longprogramopt{prefix} or
|
|
\longprogramopt{exec-prefix} actually point to an alternate Python
|
|
installation; if the directories listed above do not already exist, they
|
|
are created at installation time.
|
|
|
|
Incidentally, the real reason the prefix scheme is important is simply
|
|
that a standard \UNIX{} installation uses the prefix scheme, but with
|
|
\longprogramopt{prefix} and \longprogramopt{exec-prefix} supplied by
|
|
Python itself as \code{sys.prefix} and \code{sys.exec\_prefix}. Thus,
|
|
you might think you'll never use the prefix scheme, but every time you
|
|
run \code{python setup.py install} without any other options, you're
|
|
using it.
|
|
|
|
Note that installing extensions to an alternate Python installation has
|
|
no effect on how those extensions are built: in particular, the Python
|
|
header files (\file{Python.h} and friends) installed with the Python
|
|
interpreter used to run the setup script will be used in compiling
|
|
extensions. It is your responsibility to ensure that the interpreter
|
|
used to run extensions installed in this way is compatible with the
|
|
interpreter used to build them. The best way to do this is to ensure
|
|
that the two interpreters are the same version of Python (possibly
|
|
different builds, or possibly copies of the same build). (Of course, if
|
|
your \longprogramopt{prefix} and \longprogramopt{exec-prefix} don't even
|
|
point to an alternate Python installation, this is immaterial.)
|
|
|
|
|
|
\subsection{Alternate installation: Windows (the prefix scheme)}
|
|
\label{alt-install-windows}
|
|
|
|
Windows has no concept of a user's home directory, and since the
|
|
standard Python installation under Windows is simpler than under
|
|
\UNIX, the \longprogramopt{prefix} option has traditionally been used
|
|
to install additional packages in separate locations on Windows.
|
|
|
|
\begin{verbatim}
|
|
python setup.py install --prefix="\Temp\Python"
|
|
\end{verbatim}
|
|
|
|
to install modules to the
|
|
\file{\textbackslash{}Temp\textbackslash{}Python} directory on the
|
|
current drive.
|
|
|
|
The installation base is defined by the \longprogramopt{prefix} option;
|
|
the \longprogramopt{exec-prefix} option is not supported under Windows.
|
|
Files are installed as follows:
|
|
\installscheme{prefix}{}
|
|
{prefix}{}
|
|
{prefix}{\textbackslash{}Scripts}
|
|
{prefix}{\textbackslash{}Data}
|
|
|
|
|
|
|
|
\section{Custom Installation}
|
|
\label{custom-install}
|
|
|
|
Sometimes, the alternate installation schemes described in
|
|
section~\ref{alt-install} just don't do what you want. You might
|
|
want to tweak just one or two directories while keeping everything under
|
|
the same base directory, or you might want to completely redefine the
|
|
installation scheme. In either case, you're creating a \emph{custom
|
|
installation scheme}.
|
|
|
|
You probably noticed the column of ``override options'' in the tables
|
|
describing the alternate installation schemes above. Those options are
|
|
how you define a custom installation scheme. These override options can
|
|
be relative, absolute, or explicitly defined in terms of one of the
|
|
installation base directories. (There are two installation base
|
|
directories, and they are normally the same---they only differ when you
|
|
use the \UNIX{} ``prefix scheme'' and supply different
|
|
\longprogramopt{prefix} and \longprogramopt{exec-prefix} options.)
|
|
|
|
For example, say you're installing a module distribution to your home
|
|
directory under \UNIX---but you want scripts to go in
|
|
\file{\textasciitilde/scripts} rather than \file{\textasciitilde/bin}.
|
|
As you might expect, you can override this directory with the
|
|
\longprogramopt{install-scripts} option; in this case, it makes most
|
|
sense to supply a relative path, which will be interpreted relative to
|
|
the installation base directory (your home directory, in this case):
|
|
|
|
\begin{verbatim}
|
|
python setup.py install --home=~ --install-scripts=scripts
|
|
\end{verbatim}
|
|
|
|
Another \UNIX{} example: suppose your Python installation was built and
|
|
installed with a prefix of \file{/usr/local/python}, so under a standard
|
|
installation scripts will wind up in \file{/usr/local/python/bin}. If
|
|
you want them in \file{/usr/local/bin} instead, you would supply this
|
|
absolute directory for the \longprogramopt{install-scripts} option:
|
|
|
|
\begin{verbatim}
|
|
python setup.py install --install-scripts=/usr/local/bin
|
|
\end{verbatim}
|
|
|
|
(This performs an installation using the ``prefix scheme,'' where the
|
|
prefix is whatever your Python interpreter was installed with---
|
|
\file{/usr/local/python} in this case.)
|
|
|
|
If you maintain Python on Windows, you might want third-party modules to
|
|
live in a subdirectory of \filevar{prefix}, rather than right in
|
|
\filevar{prefix} itself. This is almost as easy as customizing the
|
|
script installation directory---you just have to remember that there are
|
|
two types of modules to worry about, pure modules and non-pure modules
|
|
(i.e., modules from a non-pure distribution). For example:
|
|
|
|
\begin{verbatim}
|
|
python setup.py install --install-purelib=Site --install-platlib=Site
|
|
\end{verbatim}
|
|
|
|
The specified installation directories are relative to
|
|
\filevar{prefix}. Of course, you also have to ensure that these
|
|
directories are in Python's module search path, such as by putting a
|
|
\file{.pth} file in \filevar{prefix}. See section~\ref{search-path}
|
|
to find out how to modify Python's search path.
|
|
|
|
If you want to define an entire installation scheme, you just have to
|
|
supply all of the installation directory options. The recommended way
|
|
to do this is to supply relative paths; for example, if you want to
|
|
maintain all Python module-related files under \file{python} in your
|
|
home directory, and you want a separate directory for each platform that
|
|
you use your home directory from, you might define the following
|
|
installation scheme:
|
|
|
|
\begin{verbatim}
|
|
python setup.py install --home=~ \
|
|
--install-purelib=python/lib \
|
|
--install-platlib=python/lib.$PLAT \
|
|
--install-scripts=python/scripts
|
|
--install-data=python/data
|
|
\end{verbatim}
|
|
% $ % -- bow to font-lock
|
|
|
|
or, equivalently,
|
|
|
|
\begin{verbatim}
|
|
python setup.py install --home=~/python \
|
|
--install-purelib=lib \
|
|
--install-platlib='lib.$PLAT' \
|
|
--install-scripts=scripts
|
|
--install-data=data
|
|
\end{verbatim}
|
|
% $ % -- bow to font-lock
|
|
|
|
\code{\$PLAT} is not (necessarily) an environment variable---it will be
|
|
expanded by the Distutils as it parses your command line options, just
|
|
as it does when parsing your configuration file(s).
|
|
|
|
Obviously, specifying the entire installation scheme every time you
|
|
install a new module distribution would be very tedious. Thus, you can
|
|
put these options into your Distutils config file (see
|
|
section~\ref{config-files}):
|
|
|
|
\begin{verbatim}
|
|
[install]
|
|
install-base=$HOME
|
|
install-purelib=python/lib
|
|
install-platlib=python/lib.$PLAT
|
|
install-scripts=python/scripts
|
|
install-data=python/data
|
|
\end{verbatim}
|
|
|
|
or, equivalently,
|
|
|
|
\begin{verbatim}
|
|
[install]
|
|
install-base=$HOME/python
|
|
install-purelib=lib
|
|
install-platlib=lib.$PLAT
|
|
install-scripts=scripts
|
|
install-data=data
|
|
\end{verbatim}
|
|
|
|
Note that these two are \emph{not} equivalent if you supply a different
|
|
installation base directory when you run the setup script. For example,
|
|
|
|
\begin{verbatim}
|
|
python setup.py --install-base=/tmp
|
|
\end{verbatim}
|
|
|
|
would install pure modules to \filevar{/tmp/python/lib} in the first
|
|
case, and to \filevar{/tmp/lib} in the second case. (For the second
|
|
case, you probably want to supply an installation base of
|
|
\file{/tmp/python}.)
|
|
|
|
You probably noticed the use of \code{\$HOME} and \code{\$PLAT} in the
|
|
sample configuration file input. These are Distutils configuration
|
|
variables, which bear a strong resemblance to environment variables.
|
|
In fact, you can use environment variables in config files on
|
|
platforms that have such a notion but the Distutils additionally
|
|
define a few extra variables that may not be in your environment, such
|
|
as \code{\$PLAT}. (And of course, on systems that don't have
|
|
environment variables, such as Mac OS 9, the configuration
|
|
variables supplied by the Distutils are the only ones you can use.)
|
|
See section~\ref{config-files} for details.
|
|
|
|
% XXX need some Windows examples---when would custom
|
|
% installation schemes be needed on those platforms?
|
|
|
|
|
|
% XXX I'm not sure where this section should go.
|
|
\subsection{Modifying Python's Search Path}
|
|
\label{search-path}
|
|
|
|
When the Python interpreter executes an \keyword{import} statement, it
|
|
searches for both Python code and extension modules along a search
|
|
path. A default value for the path is configured into the Python
|
|
binary when the interpreter is built. You can determine the path by
|
|
importing the \module{sys} module and printing the value of
|
|
\code{sys.path}.
|
|
|
|
\begin{verbatim}
|
|
$ python
|
|
Python 2.2 (#11, Oct 3 2002, 13:31:27)
|
|
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2
|
|
Type ``help'', ``copyright'', ``credits'' or ``license'' for more information.
|
|
>>> import sys
|
|
>>> sys.path
|
|
['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2',
|
|
'/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload',
|
|
'/usr/local/lib/python2.3/site-packages']
|
|
>>>
|
|
\end{verbatim} % $ <-- bow to font-lock
|
|
|
|
The null string in \code{sys.path} represents the current working
|
|
directory.
|
|
|
|
The expected convention for locally installed packages is to put them
|
|
in the \file{.../site-packages/} directory, but you may want to
|
|
install Python modules into some arbitrary directory. For example,
|
|
your site may have a convention of keeping all software related to the
|
|
web server under \file{/www}. Add-on Python modules might then belong
|
|
in \file{/www/python}, and in order to import them, this directory
|
|
must be added to \code{sys.path}. There are several different ways to
|
|
add the directory.
|
|
|
|
The most convenient way is to add a path configuration file to a
|
|
directory that's already on Python's path, usually to the
|
|
\file{.../site-packages/} directory. Path configuration files have an
|
|
extension of \file{.pth}, and each line must contain a single path
|
|
that will be appended to \code{sys.path}. (Because the new paths are
|
|
appended to \code{sys.path}, modules in the added directories will not
|
|
override standard modules. This means you can't use this mechanism
|
|
for installing fixed versions of standard modules.)
|
|
|
|
Paths can be absolute or relative, in which case they're relative to
|
|
the directory containing the \file{.pth} file. Any directories added
|
|
to the search path will be scanned in turn for \file{.pth} files. See
|
|
\citetitle[http://www.python.org/dev/doc/devel/lib/module-site.html]
|
|
{site module documentation} for more information.
|
|
|
|
A slightly less convenient way is to edit the \file{site.py} file in
|
|
Python's standard library, and modify \code{sys.path}. \file{site.py}
|
|
is automatically imported when the Python interpreter is executed,
|
|
unless the \programopt{-S} switch is supplied to suppress this
|
|
behaviour. So you could simply edit \file{site.py} and add two lines to it:
|
|
|
|
\begin{verbatim}
|
|
import sys
|
|
sys.path.append('/www/python/')
|
|
\end{verbatim}
|
|
|
|
However, if you reinstall the same major version of Python (perhaps
|
|
when upgrading from 2.2 to 2.2.2, for example) \file{site.py} will be
|
|
overwritten by the stock version. You'd have to remember that it was
|
|
modified and save a copy before doing the installation.
|
|
|
|
There are two environment variables that can modify \code{sys.path}.
|
|
\envvar{PYTHONHOME} sets an alternate value for the prefix of the
|
|
Python installation. For example, if \envvar{PYTHONHOME} is set to
|
|
\samp{/www/python}, the search path will be set to \code{['',
|
|
'/www/python/lib/python2.2/', '/www/python/lib/python2.3/plat-linux2',
|
|
...]}.
|
|
|
|
The \envvar{PYTHONPATH} variable can be set to a list of paths that
|
|
will be added to the beginning of \code{sys.path}. For example, if
|
|
\envvar{PYTHONPATH} is set to \samp{/www/python:/opt/py}, the search
|
|
path will begin with \code{['/www/python', '/opt/py']}. (Note that
|
|
directories must exist in order to be added to \code{sys.path}; the
|
|
\module{site} module removes paths that don't exist.)
|
|
|
|
Finally, \code{sys.path} is just a regular Python list, so any Python
|
|
application can modify it by adding or removing entries.
|
|
|
|
|
|
\section{Distutils Configuration Files}
|
|
\label{config-files}
|
|
|
|
As mentioned above, you can use Distutils configuration files to record
|
|
personal or site preferences for any Distutils options. That is, any
|
|
option to any command can be stored in one of two or three (depending on
|
|
your platform) configuration files, which will be consulted before the
|
|
command-line is parsed. This means that configuration files will
|
|
override default values, and the command-line will in turn override
|
|
configuration files. Furthermore, if multiple configuration files
|
|
apply, values from ``earlier'' files are overridden by ``later'' files.
|
|
|
|
|
|
\subsection{Location and names of config files}
|
|
\label{config-filenames}
|
|
|
|
The names and locations of the configuration files vary slightly across
|
|
platforms. On \UNIX and Mac OS X, the three configuration files (in the order they
|
|
are processed) are:
|
|
\begin{tableiii}{l|l|c}{textrm}
|
|
{Type of file}{Location and filename}{Notes}
|
|
\lineiii{system}{\filenq{\filevar{prefix}/lib/python\filevar{ver}/distutils/distutils.cfg}}{(1)}
|
|
\lineiii{personal}{\filenq{\$HOME/.pydistutils.cfg}}{(2)}
|
|
\lineiii{local}{\filenq{setup.cfg}}{(3)}
|
|
\end{tableiii}
|
|
|
|
And on Windows, the configuration files are:
|
|
\begin{tableiii}{l|l|c}{textrm}
|
|
{Type of file}{Location and filename}{Notes}
|
|
\lineiii{system}{\filenq{\filevar{prefix}\textbackslash{}Lib\textbackslash{}distutils\textbackslash{}distutils.cfg}}{(4)}
|
|
\lineiii{personal}{\filenq{\%HOME\%\textbackslash{}pydistutils.cfg}}{(5)}
|
|
\lineiii{local}{\filenq{setup.cfg}}{(3)}
|
|
\end{tableiii}
|
|
|
|
\noindent Notes:
|
|
\begin{description}
|
|
\item[(1)] Strictly speaking, the system-wide configuration file lives
|
|
in the directory where the Distutils are installed; under Python 1.6
|
|
and later on \UNIX, this is as shown. For Python 1.5.2, the Distutils
|
|
will normally be installed to
|
|
\file{\filevar{prefix}/lib/python1.5/site-packages/distutils},
|
|
so the system configuration file should be put there under Python
|
|
1.5.2.
|
|
\item[(2)] On \UNIX, if the \envvar{HOME} environment variable is not
|
|
defined, the user's home directory will be determined with the
|
|
\function{getpwuid()} function from the standard
|
|
\ulink{\module{pwd}}{../lib/module-pwd.html} module.
|
|
\item[(3)] I.e., in the current directory (usually the location of the
|
|
setup script).
|
|
\item[(4)] (See also note (1).) Under Python 1.6 and later, Python's
|
|
default ``installation prefix'' is \file{C:\textbackslash{}Python}, so
|
|
the system configuration file is normally
|
|
\file{C:\textbackslash{}Python\textbackslash{}Lib\textbackslash{}distutils\textbackslash{}distutils.cfg}.
|
|
Under Python 1.5.2, the default prefix was
|
|
\file{C:\textbackslash{}Program~Files\textbackslash{}Python}, and the
|
|
Distutils were not part of the standard library---so the system
|
|
configuration file would be
|
|
\file{C:\textbackslash{}Program~Files\textbackslash{}Python\textbackslash{}distutils\textbackslash{}distutils.cfg}
|
|
in a standard Python 1.5.2 installation under Windows.
|
|
\item[(5)] On Windows, if the \envvar{HOME} environment variable is not
|
|
defined, no personal configuration file will be found or used. (In
|
|
other words, the Distutils make no attempt to guess your home
|
|
directory on Windows.)
|
|
\end{description}
|
|
|
|
|
|
\subsection{Syntax of config files}
|
|
\label{config-syntax}
|
|
|
|
The Distutils configuration files all have the same syntax. The config
|
|
files are grouped into sections. There is one section for each Distutils
|
|
command, plus a \code{global} section for global options that affect
|
|
every command. Each section consists of one option per line, specified
|
|
as \code{option=value}.
|
|
|
|
For example, the following is a complete config file that just forces
|
|
all commands to run quietly by default:
|
|
|
|
\begin{verbatim}
|
|
[global]
|
|
verbose=0
|
|
\end{verbatim}
|
|
|
|
If this is installed as the system config file, it will affect all
|
|
processing of any Python module distribution by any user on the current
|
|
system. If it is installed as your personal config file (on systems
|
|
that support them), it will affect only module distributions processed
|
|
by you. And if it is used as the \file{setup.cfg} for a particular
|
|
module distribution, it affects only that distribution.
|
|
|
|
You could override the default ``build base'' directory and make the
|
|
\command{build*} commands always forcibly rebuild all files with the
|
|
following:
|
|
|
|
\begin{verbatim}
|
|
[build]
|
|
build-base=blib
|
|
force=1
|
|
\end{verbatim}
|
|
|
|
which corresponds to the command-line arguments
|
|
|
|
\begin{verbatim}
|
|
python setup.py build --build-base=blib --force
|
|
\end{verbatim}
|
|
|
|
except that including the \command{build} command on the command-line
|
|
means that command will be run. Including a particular command in
|
|
config files has no such implication; it only means that if the command
|
|
is run, the options in the config file will apply. (Or if other
|
|
commands that derive values from it are run, they will use the values in
|
|
the config file.)
|
|
|
|
You can find out the complete list of options for any command using the
|
|
\longprogramopt{help} option, e.g.:
|
|
|
|
\begin{verbatim}
|
|
python setup.py build --help
|
|
\end{verbatim}
|
|
|
|
and you can find out the complete list of global options by using
|
|
\longprogramopt{help} without a command:
|
|
|
|
\begin{verbatim}
|
|
python setup.py --help
|
|
\end{verbatim}
|
|
|
|
See also the ``Reference'' section of the ``Distributing Python
|
|
Modules'' manual.
|
|
|
|
\section{Building Extensions: Tips and Tricks}
|
|
\label{building-ext}
|
|
|
|
Whenever possible, the Distutils try to use the configuration
|
|
information made available by the Python interpreter used to run the
|
|
\file{setup.py} script. For example, the same compiler and linker
|
|
flags used to compile Python will also be used for compiling
|
|
extensions. Usually this will work well, but in complicated
|
|
situations this might be inappropriate. This section discusses how to
|
|
override the usual Distutils behaviour.
|
|
|
|
\subsection{Tweaking compiler/linker flags}
|
|
\label{tweak-flags}
|
|
|
|
Compiling a Python extension written in C or \Cpp{} will sometimes
|
|
require specifying custom flags for the compiler and linker in order
|
|
to use a particular library or produce a special kind of object code.
|
|
This is especially true if the extension hasn't been tested on your
|
|
platform, or if you're trying to cross-compile Python.
|
|
|
|
In the most general case, the extension author might have foreseen
|
|
that compiling the extensions would be complicated, and provided a
|
|
\file{Setup} file for you to edit. This will likely only be done if
|
|
the module distribution contains many separate extension modules, or
|
|
if they often require elaborate sets of compiler flags in order to work.
|
|
|
|
A \file{Setup} file, if present, is parsed in order to get a list of
|
|
extensions to build. Each line in a \file{Setup} describes a single
|
|
module. Lines have the following structure:
|
|
|
|
\begin{alltt}
|
|
\var{module} ... [\var{sourcefile} ...] [\var{cpparg} ...] [\var{library} ...]
|
|
\end{alltt}
|
|
|
|
Let's examine each of the fields in turn.
|
|
|
|
\begin{itemize}
|
|
|
|
\item \var{module} is the name of the extension module to be built,
|
|
and should be a valid Python identifier. You can't just change
|
|
this in order to rename a module (edits to the source code would
|
|
also be needed), so this should be left alone.
|
|
|
|
\item \var{sourcefile} is anything that's likely to be a source code
|
|
file, at least judging by the filename. Filenames ending in
|
|
\file{.c} are assumed to be written in C, filenames ending in
|
|
\file{.C}, \file{.cc}, and \file{.c++} are assumed to be
|
|
\Cpp, and filenames ending in \file{.m} or \file{.mm} are
|
|
assumed to be in Objective C.
|
|
|
|
\item \var{cpparg} is an argument for the C preprocessor,
|
|
and is anything starting with \programopt{-I}, \programopt{-D},
|
|
\programopt{-U} or \programopt{-C}.
|
|
|
|
\item \var{library} is anything ending in \file{.a} or beginning with
|
|
\programopt{-l} or \programopt{-L}.
|
|
\end{itemize}
|
|
|
|
If a particular platform requires a special library on your platform,
|
|
you can add it by editing the \file{Setup} file and running
|
|
\code{python setup.py build}. For example, if the module defined by the line
|
|
|
|
\begin{verbatim}
|
|
foo foomodule.c
|
|
\end{verbatim}
|
|
|
|
must be linked with the math library \file{libm.a} on your platform,
|
|
simply add \programopt{-lm} to the line:
|
|
|
|
\begin{verbatim}
|
|
foo foomodule.c -lm
|
|
\end{verbatim}
|
|
|
|
Arbitrary switches intended for the compiler or the linker can be
|
|
supplied with the \programopt{-Xcompiler} \var{arg} and
|
|
\programopt{-Xlinker} \var{arg} options:
|
|
|
|
\begin{verbatim}
|
|
foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm
|
|
\end{verbatim}
|
|
|
|
The next option after \programopt{-Xcompiler} and
|
|
\programopt{-Xlinker} will be appended to the proper command line, so
|
|
in the above example the compiler will be passed the \programopt{-o32}
|
|
option, and the linker will be passed \programopt{-shared}. If a
|
|
compiler option requires an argument, you'll have to supply multiple
|
|
\programopt{-Xcompiler} options; for example, to pass \code{-x c++} the
|
|
\file{Setup} file would have to contain
|
|
\code{-Xcompiler -x -Xcompiler c++}.
|
|
|
|
Compiler flags can also be supplied through setting the
|
|
\envvar{CFLAGS} environment variable. If set, the contents of
|
|
\envvar{CFLAGS} will be added to the compiler flags specified in the
|
|
\file{Setup} file.
|
|
|
|
|
|
\subsection{Using non-Microsoft compilers on Windows \label{non-ms-compilers}}
|
|
\sectionauthor{Rene Liebscher}{R.Liebscher@gmx.de}
|
|
|
|
\subsubsection{Borland \Cpp}
|
|
|
|
This subsection describes the necessary steps to use Distutils with the
|
|
Borland \Cpp{} compiler version 5.5.
|
|
%Should we mention that users have to create cfg-files for the compiler?
|
|
%see also http://community.borland.com/article/0,1410,21205,00.html
|
|
|
|
First you have to know that Borland's object file format (OMF) is
|
|
different from the format used by the Python version you can download
|
|
from the Python or ActiveState Web site. (Python is built with
|
|
Microsoft Visual \Cpp, which uses COFF as the object file format.)
|
|
For this reason you have to convert Python's library
|
|
\file{python24.lib} into the Borland format. You can do this as
|
|
follows:
|
|
|
|
\begin{verbatim}
|
|
coff2omf python24.lib python24_bcpp.lib
|
|
\end{verbatim}
|
|
|
|
The \file{coff2omf} program comes with the Borland compiler. The file
|
|
\file{python24.lib} is in the \file{Libs} directory of your Python
|
|
installation. If your extension uses other libraries (zlib,...) you
|
|
have to convert them too.
|
|
|
|
The converted files have to reside in the same directories as the
|
|
normal libraries.
|
|
|
|
How does Distutils manage to use these libraries with their changed
|
|
names? If the extension needs a library (eg. \file{foo}) Distutils
|
|
checks first if it finds a library with suffix \file{_bcpp}
|
|
(eg. \file{foo_bcpp.lib}) and then uses this library. In the case it
|
|
doesn't find such a special library it uses the default name
|
|
(\file{foo.lib}.)\footnote{This also means you could replace all
|
|
existing COFF-libraries with OMF-libraries of the same name.}
|
|
|
|
To let Distutils compile your extension with Borland \Cpp{} you now have
|
|
to type:
|
|
|
|
\begin{verbatim}
|
|
python setup.py build --compiler=bcpp
|
|
\end{verbatim}
|
|
|
|
If you want to use the Borland \Cpp{} compiler as the default, you
|
|
could specify this in your personal or system-wide configuration file
|
|
for Distutils (see section~\ref{config-files}.)
|
|
|
|
\begin{seealso}
|
|
\seetitle[http://www.borland.com/bcppbuilder/freecompiler/]
|
|
{\Cpp{}Builder Compiler}
|
|
{Information about the free \Cpp{} compiler from Borland,
|
|
including links to the download pages.}
|
|
|
|
\seetitle[http://www.cyberus.ca/\~{}g_will/pyExtenDL.shtml]
|
|
{Creating Python Extensions Using Borland's Free Compiler}
|
|
{Document describing how to use Borland's free command-line \Cpp
|
|
compiler to build Python.}
|
|
\end{seealso}
|
|
|
|
|
|
\subsubsection{GNU C / Cygwin / MinGW}
|
|
|
|
This section describes the necessary steps to use Distutils with the
|
|
GNU C/\Cpp{} compilers in their Cygwin and MinGW
|
|
distributions.\footnote{Check
|
|
\url{http://sources.redhat.com/cygwin/} and
|
|
\url{http://www.mingw.org/} for more information}
|
|
For a Python interpreter that was built with Cygwin, everything should
|
|
work without any of these following steps.
|
|
|
|
These compilers require some special libraries.
|
|
This task is more complex than for Borland's \Cpp, because there is no
|
|
program to convert the library.
|
|
% I don't understand what the next line means. --amk
|
|
% (inclusive the references on data structures.)
|
|
|
|
First you have to create a list of symbols which the Python DLL exports.
|
|
(You can find a good program for this task at
|
|
\url{http://starship.python.net/crew/kernr/mingw32/Notes.html}, see at
|
|
PExports 0.42h there.)
|
|
|
|
\begin{verbatim}
|
|
pexports python24.dll >python24.def
|
|
\end{verbatim}
|
|
|
|
Then you can create from these information an import library for gcc.
|
|
|
|
\begin{verbatim}
|
|
dlltool --dllname python24.dll --def python24.def --output-lib libpython24.a
|
|
\end{verbatim}
|
|
|
|
The resulting library has to be placed in the same directory as
|
|
\file{python24.lib}. (Should be the \file{libs} directory under your
|
|
Python installation directory.)
|
|
|
|
If your extension uses other libraries (zlib,...) you might
|
|
have to convert them too.
|
|
The converted files have to reside in the same directories as the normal
|
|
libraries do.
|
|
|
|
To let Distutils compile your extension with Cygwin you now have to type
|
|
|
|
\begin{verbatim}
|
|
python setup.py build --compiler=cygwin
|
|
\end{verbatim}
|
|
|
|
and for Cygwin in no-cygwin mode\footnote{Then you have no
|
|
\POSIX{} emulation available, but you also don't need
|
|
\file{cygwin1.dll}.} or for MinGW type:
|
|
|
|
\begin{verbatim}
|
|
python setup.py build --compiler=mingw32
|
|
\end{verbatim}
|
|
|
|
If you want to use any of these options/compilers as default, you should
|
|
consider to write it in your personal or system-wide configuration file
|
|
for Distutils (see section~\ref{config-files}.)
|
|
|
|
\begin{seealso}
|
|
\seetitle[http://www.zope.org/Members/als/tips/win32_mingw_modules]
|
|
{Building Python modules on MS Windows platform with MinGW}
|
|
{Information about building the required libraries for the MinGW
|
|
environment.}
|
|
|
|
\seeurl{http://pyopengl.sourceforge.net/ftp/win32-stuff/}
|
|
{Converted import libraries in Cygwin/MinGW and Borland format,
|
|
and a script to create the registry entries needed for Distutils
|
|
to locate the built Python.}
|
|
\end{seealso}
|
|
|
|
|
|
|
|
\end{document}
|