2000-02-25 20:52:48 -04:00
|
|
|
\documentclass{howto}
|
|
|
|
\usepackage{ltxmarkup}
|
2000-04-09 01:32:40 -03:00
|
|
|
\usepackage{times}
|
2000-04-09 01:06:44 -03:00
|
|
|
\usepackage{distutils}
|
2000-02-25 20:52:48 -04:00
|
|
|
|
2000-04-09 01:06:44 -03:00
|
|
|
\title{Distributing Python Modules}
|
2000-02-25 20:52:48 -04:00
|
|
|
|
|
|
|
\author{Greg Ward}
|
|
|
|
\authoraddress{E-mail: \email{gward@python.net}}
|
|
|
|
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-02-25 20:52:48 -04:00
|
|
|
\begin{document}
|
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\maketitle
|
|
|
|
\tableofcontents
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
\section{Introduction}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{intro}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
In the past, Python module developers have not had much infrastructure
|
|
|
|
support for distributing modules, nor have Python users had much support
|
|
|
|
for installing and maintaining third-party modules. With the
|
|
|
|
introduction of the Python Distribution Utilities (Distutils for short)
|
|
|
|
in Python 1.6, this situation should start to improve.
|
|
|
|
|
|
|
|
This document only covers using the Distutils to distribute your Python
|
|
|
|
modules. Using the Distutils does not tie you to Python 1.6, though:
|
|
|
|
the Distutils work just fine with Python 1.5, and it is reasonable (and
|
|
|
|
expected to become commonplace) to expect users of Python 1.5 to
|
|
|
|
download and install the Distutils separately before they can install
|
|
|
|
your modules. Python 1.6 users, of course, won't have to add anything
|
|
|
|
to their Python installation in order to use the Distutils to install
|
|
|
|
third-party modules.
|
|
|
|
|
|
|
|
This document concentrates on the role of developer/distributor: if
|
|
|
|
you're looking for information on installing Python modules, you should
|
|
|
|
refer to the ``Installing Python Modules'' manual.
|
|
|
|
|
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\section{Concepts \& Terminology}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{concepts}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
Using the Distutils is quite simple, both for module developers and for
|
|
|
|
users/administrators installing third-party modules. As a developer,
|
|
|
|
your responsibilites (apart from writing solid, well-documented and
|
|
|
|
well-tested code, of course!) are:
|
|
|
|
\begin{itemize}
|
|
|
|
\item write a setup script (\file{setup.py} by convention)
|
|
|
|
\item (optional) write a setup configuration file
|
|
|
|
\item create a source distribution
|
|
|
|
\item (optional) create one or more built (binary) distributions
|
|
|
|
\end{itemize}
|
|
|
|
Each of these tasks is covered in this document.
|
|
|
|
|
|
|
|
Not all module developers have access to a multitude of platforms, so
|
|
|
|
it's not always feasible to expect them to create a multitude of built
|
|
|
|
distributions. It is hoped that a class of intermediaries, called
|
|
|
|
\emph{packagers}, will arise to take address this need. Packagers will
|
|
|
|
take source distributions released by module developers, build them on
|
|
|
|
one or more platforms, and release the resulting built distributions.
|
|
|
|
Thus, users on the most popular platforms will be able to install most
|
|
|
|
popular Python module distributions in the most natural way for their
|
|
|
|
platform, without having to run a single setup script or compile a line
|
|
|
|
of code.
|
|
|
|
|
|
|
|
|
|
|
|
\subsection{A simple example}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{simple-example}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
The setup script is usually quite simple, although since it's written in
|
|
|
|
Python, there are no arbitrary limits to what you can do. If all you
|
|
|
|
want to do is distribute a module called \module{foo}, contained in a
|
|
|
|
file \file{foo.py}, then you can get away with as little as this:
|
|
|
|
\begin{verbatim}
|
|
|
|
from distutils.core import setup
|
|
|
|
setup (name = "foo",
|
|
|
|
version = "1.0",
|
|
|
|
py_modules = ["foo"])
|
|
|
|
\end{verbatim}
|
|
|
|
Some observations:
|
|
|
|
\begin{itemize}
|
|
|
|
\item all information that you supply to the Distutils is supplied as
|
2000-04-09 01:32:40 -03:00
|
|
|
keyword arguments to the \function{setup()} function
|
2000-04-09 01:06:44 -03:00
|
|
|
\item those keyword arguments fall into two categories: package
|
|
|
|
meta-data (name, version number) and information about what's in the
|
|
|
|
package (list of pure modules, in this case)
|
|
|
|
\item modules are specified by module name, not filename (the same will
|
|
|
|
hold true for packages and extensions)
|
|
|
|
\item it's recommended that you supply a little more meta-data, in
|
|
|
|
particular your name, email address and a URL for the project
|
|
|
|
\end{itemize}
|
|
|
|
|
|
|
|
To create a source distribution for this module, you would run
|
|
|
|
\begin{verbatim}
|
|
|
|
python setup.py sdist
|
|
|
|
\end{verbatim}
|
|
|
|
which will create an archive file (e.g., tarball on Unix, zip file on
|
|
|
|
Windows) containing your setup script, \file{setup.py}, and your module,
|
|
|
|
\file{foo.py}. The archive file will be named \file{Foo-1.0.tar.gz} (or
|
|
|
|
\file{.zip}), and will unpack into a directory \file{Foo-1.0}.
|
|
|
|
|
|
|
|
If an end-user wishes to install your \module{foo} module, all she has
|
2000-05-25 22:04:47 -03:00
|
|
|
to do is download \file{Foo-1.0.tar.gz} (or \file{.zip}), unpack it,
|
2000-04-09 01:06:44 -03:00
|
|
|
and---from the \file{Foo-1.0} directory---run
|
|
|
|
\begin{verbatim}
|
|
|
|
python setup.py install
|
|
|
|
\end{verbatim}
|
|
|
|
which will ultimately copy \file{foo.py} to the appropriate directory
|
|
|
|
for third-party modules in their Python installation.
|
|
|
|
|
|
|
|
This simple example demonstrates some fundamental concepts of the
|
|
|
|
Distutils: first, both developers and installers have the same basic
|
|
|
|
user interface, i.e. the setup script. The difference is which
|
|
|
|
Distutils \emph{commands} they use: the \command{sdist} command is
|
|
|
|
almost exclusively for module developers, while \command{install} is
|
|
|
|
more often for installers (although most developers will want to install
|
|
|
|
their own code occasionally).
|
|
|
|
|
|
|
|
\XXX{only partially implemented}%
|
|
|
|
If you want to make things really easy for your users, you can create
|
|
|
|
one or more built distributions for them. For instance, if you are
|
|
|
|
running on a Windows machine, and want to make things easy for other
|
|
|
|
Windows users, you can create an executable installer (the most
|
|
|
|
appropriate type of built distribution for this platform) with the
|
2000-05-25 22:04:47 -03:00
|
|
|
\command{bdist\_wininst} command. For example:
|
2000-04-09 01:06:44 -03:00
|
|
|
\begin{verbatim}
|
2000-05-25 22:04:47 -03:00
|
|
|
python setup.py bdist_wininst
|
2000-04-09 01:06:44 -03:00
|
|
|
\end{verbatim}
|
2000-04-09 01:32:40 -03:00
|
|
|
will create an executable installer, \file{Foo-1\_0.exe}, in the current
|
2000-04-09 01:06:44 -03:00
|
|
|
directory.
|
|
|
|
|
2000-05-25 22:04:47 -03:00
|
|
|
(Another way to create executable installers for Windows is with the
|
|
|
|
\command{bdist\_wise} command, which uses Wise---the commercial
|
|
|
|
installer-generator used to create Python's own installer---to create
|
|
|
|
the installer. Wise-based installers are more appropriate for large,
|
|
|
|
industrial-strength applications that need the full capabilities of a
|
|
|
|
``real'' installer. \command{bdist\_wininst} creates a self-extracting
|
|
|
|
zip file with a minimal user interface, which is enough for small- to
|
|
|
|
medium-sized module collections. You'll need to have version XXX of
|
|
|
|
Wise installed on your system for the \command{bdist\_wise} to work;
|
|
|
|
it's available from \url{http://foo/bar/baz}.)
|
|
|
|
|
|
|
|
Other \command{bdist} commands exist for other platforms: for example,
|
|
|
|
\command{bdist\_rpm} for RPM-based Linux systems, (\command{bdist\_deb})
|
|
|
|
for Debian-based Linux systems, and so forth. See
|
|
|
|
section~\ref{bdist-cmds} for details on all the \command{bdist}
|
|
|
|
commands.
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{General Python terminology}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{python-terms}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
If you're reading this document, you probably have a good idea of what
|
|
|
|
modules, extensions, and so forth are. Nevertheless, just to be sure
|
|
|
|
that everyone is operating from a common starting point, we offer the
|
|
|
|
following glossary of common Python terms:
|
|
|
|
\begin{description}
|
|
|
|
\item[module] the basic unit of code reusability in Python: a block of
|
|
|
|
code imported by some other code. There are three types of modules
|
|
|
|
that concern us here: pure Python modules, extension modules, and
|
|
|
|
packages.
|
|
|
|
\item[pure Python module] a module written in Python and contained in a
|
|
|
|
single \file{.py} file (and possibly associated \file{.pyc} and/or
|
|
|
|
\file{.pyo} files). Sometimes referred to as a ``pure module.''
|
|
|
|
\item[extension module] a module written in the low-level language of
|
|
|
|
the Python implemention: C/C++ for CPython, Java for JPython.
|
|
|
|
Typically contained in a single dynamically loadable pre-compiled
|
|
|
|
file, e.g. a shared object (\file{.so}) file for CPython extensions on
|
|
|
|
Unix, a DLL (given the \file{.pyd} extension) for CPython extensions
|
|
|
|
on Windows, or a Java class file for JPython extensions.
|
|
|
|
\item[package] a module that contains other modules; typically contained
|
|
|
|
in a directory in the filesystem and distinguished from other
|
|
|
|
directories by the presence of a file \file{\_\_init\_\_.py}.
|
2000-05-25 22:04:47 -03:00
|
|
|
\item[root package] the ``package'' that modules not in a package live
|
|
|
|
in. The vast majority of the standard library is in the root package,
|
|
|
|
as are many small, standalone third-party modules that don't belong to
|
|
|
|
a larger module collection. (The root package isn't really a package,
|
|
|
|
since it doesn't have an \file{\_\_init\_\_.py} file. But we have to
|
|
|
|
call it something.)
|
2000-04-09 01:06:44 -03:00
|
|
|
\end{description}
|
|
|
|
|
|
|
|
|
|
|
|
\subsection{Distutils-specific terminology}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{distutils-term}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
The following terms apply more specifically to the domain of
|
|
|
|
distributing Python modules using the Distutils:
|
|
|
|
\begin{description}
|
|
|
|
\item[module distribution] a collection of Python modules distributed
|
|
|
|
together as a single downloadable resource and meant to be installed
|
|
|
|
\emph{en masse}. Examples of some well-known module distributions are
|
|
|
|
Numeric Python, PyXML, PIL (the Python Imaging Library), or
|
|
|
|
mxDateTime. (This would be called a \emph{package}, except that term
|
2000-05-25 22:04:47 -03:00
|
|
|
is already taken in the Python context: a single module distribution
|
|
|
|
may contain zero, one, or many Python packages.)
|
2000-04-09 01:06:44 -03:00
|
|
|
\item[pure module distribution] a module distribution that contains only
|
|
|
|
pure Python modules and packages. Sometimes referred to as a ``pure
|
|
|
|
distribution.''
|
|
|
|
\item[non-pure module distribution] a module distribution that contains
|
|
|
|
at least one extension module. Sometimes referred to as a ``non-pure
|
|
|
|
distribution.''
|
|
|
|
\item[distribution root] the top-level directory of your source tree (or
|
|
|
|
source distribution); the directory where \file{setup.py} exists and
|
|
|
|
is run from
|
|
|
|
\end{description}
|
|
|
|
|
|
|
|
|
|
|
|
\section{Writing the Setup Script}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{setup-script}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
The setup script is the centre of all activity in building,
|
|
|
|
distributing, and installing modules using the Distutils. The main
|
|
|
|
purpose of the setup script is to describe your module distribution to
|
2000-04-19 19:48:09 -03:00
|
|
|
the Distutils, so that the various commands that operate on your modules
|
2000-05-25 22:04:47 -03:00
|
|
|
do the right thing. As we saw in section~\ref{simple-example} above,
|
|
|
|
the setup script consists mainly of a call to \function{setup()}, and
|
|
|
|
all information supplied to the Distutils is supplied as keyword
|
2000-04-09 01:32:40 -03:00
|
|
|
arguments to \function{setup()}.
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
Here's a slightly more involved example, which we'll follow for the next
|
|
|
|
couple of sections: the Distutils' own setup script. (Keep in mind that
|
|
|
|
although the Distutils are included with Python 1.6, they also have an
|
|
|
|
independent existence so that Python 1.5 users can use them to install
|
2000-05-25 22:04:47 -03:00
|
|
|
other module distributions. The Distutils' own setup script is used to
|
|
|
|
install the package into Python 1.5.)
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
\begin{verbatim}
|
|
|
|
#!/usr/bin/env python
|
|
|
|
|
|
|
|
from distutils.core import setup
|
|
|
|
|
|
|
|
setup (name = "Distutils",
|
|
|
|
version = "1.0",
|
|
|
|
description = "Python Module Distribution Utilities",
|
|
|
|
author = "Greg Ward",
|
|
|
|
author_email = "gward@python.net",
|
|
|
|
url = "http://www.python.org/sigs/distutils-sig/",
|
|
|
|
|
|
|
|
packages = ['distutils', 'distutils.command'],
|
|
|
|
)
|
|
|
|
\end{verbatim}
|
|
|
|
There are only two differences between this and the trivial one-file
|
2000-04-28 14:12:24 -03:00
|
|
|
distribution presented in section~\ref{simple-example}: more
|
2000-04-09 01:06:44 -03:00
|
|
|
meta-data, and the specification of pure Python modules by package,
|
|
|
|
rather than by module. This is important since the Distutils consist of
|
|
|
|
a couple of dozen modules split into (so far) two packages; an explicit
|
|
|
|
list of every module would be tedious to generate and difficult to
|
|
|
|
maintain.
|
|
|
|
|
2000-04-13 22:53:36 -03:00
|
|
|
Note that any pathnames (files or directories) supplied in the setup
|
|
|
|
script should be written using the Unix convention, i.e.
|
|
|
|
slash-separated. The Distutils will take care of converting this
|
2000-05-25 22:04:47 -03:00
|
|
|
platform-neutral representation into whatever is appropriate on your
|
2000-04-13 22:53:36 -03:00
|
|
|
current platform before actually using the pathname. This makes your
|
|
|
|
setup script portable across operating systems, which of course is one
|
|
|
|
of the major goals of the Distutils. In this spirit, all pathnames in
|
2000-05-25 22:04:47 -03:00
|
|
|
this document are slash-separated (Mac OS programmers should keep in
|
|
|
|
mind that the \emph{absence} of a leading slash indicates a relative
|
|
|
|
path, the opposite of the Mac OS convention with colons).
|
2000-04-13 22:53:36 -03:00
|
|
|
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
\subsection{Package directories}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{package-dirs}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
The \option{packages} option tells the Distutils to process (build,
|
|
|
|
distribute, install, etc.) all pure Python modules found in each package
|
|
|
|
mentioned in the \option{packages} list. In order to do this, of
|
|
|
|
course, there has to be a correspondence between package names and
|
|
|
|
directories in the filesystem. The default correspondence is the most
|
2000-04-19 19:36:24 -03:00
|
|
|
obvious one, i.e. package \module{distutils} is found in the directory
|
2000-04-09 01:06:44 -03:00
|
|
|
\file{distutils} relative to the distribution root. Thus, when you say
|
|
|
|
\code{packages = ['foo']} in your setup script, you are promising that
|
|
|
|
the Distutils will find a file \file{foo/\_\_init\_\_.py} (which might
|
|
|
|
be spelled differently on your system, but you get the idea) relative to
|
|
|
|
the directory where your setup script lives. (If you break this
|
|
|
|
promise, the Distutils will issue a warning but process the broken
|
|
|
|
package anyways.)
|
|
|
|
|
|
|
|
If you use a different convention to lay out your source directory,
|
|
|
|
that's no problem: you just have to supply the \option{package\_dir}
|
|
|
|
option to tell the Distutils about your convention. For example, say
|
|
|
|
you keep all Python source under \file{lib}, so that modules not in any
|
2000-04-19 19:36:24 -03:00
|
|
|
package are right in \file{lib}, modules in the \module{foo} package
|
2000-04-09 01:06:44 -03:00
|
|
|
are in \file{lib/foo}, and so forth. Then you would put
|
|
|
|
\begin{verbatim}
|
|
|
|
package_dir = {'': 'lib'}
|
|
|
|
\end{verbatim}
|
|
|
|
in your setup script. (The keys to this dictionary are package names,
|
|
|
|
and an empty package name stands for the ``root package,'' i.e. no
|
|
|
|
package at all. The values are directory names relative to your
|
|
|
|
distribution root.) In this case, when you say
|
|
|
|
\code{packages = ['foo']}, you are promising that the file
|
|
|
|
\file{lib/foo/\_\_init\_\_.py} exists.
|
|
|
|
|
2000-04-19 19:36:24 -03:00
|
|
|
Another possible convention is to put the \module{foo} package right in
|
|
|
|
\file{lib}, the \module{foo.bar} package in \file{lib/bar}, etc. This
|
2000-04-09 01:06:44 -03:00
|
|
|
would be written in the setup script as
|
|
|
|
\begin{verbatim}
|
|
|
|
package_dir = {'foo': 'lib'}
|
|
|
|
\end{verbatim}
|
2000-05-25 22:04:47 -03:00
|
|
|
A \code{\var{package}: \var{dir}} entry in the \option{package\_dir}
|
|
|
|
dictionary implicitly applies to all packages below \var{package}, so
|
|
|
|
the \module{foo.bar} case is automatically handled here. In this
|
|
|
|
example, having \code{packages = ['foo', 'foo.bar']} tells the Distutils
|
|
|
|
to look for \file{lib/\_\_init\_\_.py} and
|
|
|
|
\file{lib/bar/\_\_init\_\_.py}. (Keep in mind that although
|
|
|
|
\option{package\_dir} applies recursively, you must explicitly list all
|
|
|
|
packages in \option{packages}: the Distutils will \emph{not} recursively
|
|
|
|
scan your source tree looking for any directory with an
|
|
|
|
\file{\_\_init\_\_.py} file.)
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{Listing individual modules}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{listing-modules}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
For a small module distribution, you might prefer to list all modules
|
|
|
|
rather than listing packages---especially the case of a single module
|
|
|
|
that goes in the ``root package'' (i.e., no package at all). This
|
2000-04-28 14:12:24 -03:00
|
|
|
simplest case was shown in section~\ref{simple-example}; here is a
|
2000-04-09 01:06:44 -03:00
|
|
|
slightly more involved example:
|
|
|
|
\begin{verbatim}
|
|
|
|
py_modules = ['mod1', 'pkg.mod2']
|
|
|
|
\end{verbatim}
|
|
|
|
This describes two modules, one of them in the ``root'' package, the
|
2000-04-19 19:48:09 -03:00
|
|
|
other in the \module{pkg} package. Again, the default package/directory
|
|
|
|
layout implies that these two modules can be found in \file{mod1.py} and
|
|
|
|
\file{pkg/mod2.py}, and that \file{pkg/\_\_init\_\_.py} exists as well.
|
|
|
|
And again, you can override the package/directory layout using the
|
2000-05-25 22:04:47 -03:00
|
|
|
\option{package\_dir} option.
|
|
|
|
|
|
|
|
|
|
|
|
\subsection{Describing extension modules}
|
|
|
|
\label{sec:describing-extensions}
|
|
|
|
|
|
|
|
\XXX{be sure to describe the whole \code{build\_info} dict, including
|
|
|
|
\code{extra\_compile\_args} and \code{extra\_link\_args}}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\section{Writing the Setup Configuration File}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{setup-config}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
\XXX{not implemented yet!}
|
|
|
|
|
|
|
|
Often, it's not possible to write down everything needed to build a
|
|
|
|
distribution \emph{a priori}. You need to get some information from the
|
|
|
|
user, or from the user's system, in order to proceed. For example, you
|
|
|
|
might include an optional extension module that provides an interface to
|
|
|
|
a particular C library. If that library is installed on the user's
|
|
|
|
system, then you can build your optional extension---but you need to
|
|
|
|
know where to find the header and library file. If it's not installed,
|
|
|
|
you need to know this so you can omit your optional extension.
|
|
|
|
|
|
|
|
The preferred way to do this, of course, would be for you to tell the
|
|
|
|
Distutils which optional features (C libraries, system calls, external
|
|
|
|
utilities, etc.) you're looking for, and it would inspect the user's
|
|
|
|
system and try to find them. This functionality may appear in a future
|
|
|
|
version of the Distutils, but it isn't there now. So, for the time
|
|
|
|
being, we rely on the user building and installing your software to
|
|
|
|
provide the necessary information. The vehicle for doing so is the
|
|
|
|
setup configuration file, \file{setup.cfg}.
|
|
|
|
|
|
|
|
\XXX{need more here!}
|
|
|
|
|
|
|
|
|
|
|
|
\section{Creating a Source Distribution}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{source-dist}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-28 14:12:24 -03:00
|
|
|
As shown in section~\ref{simple-example}, you use the
|
2000-04-09 01:06:44 -03:00
|
|
|
\command{sdist} command to create a source distribution. In the
|
|
|
|
simplest case,
|
|
|
|
\begin{verbatim}
|
|
|
|
python setup.py sdist
|
|
|
|
\end{verbatim}
|
|
|
|
(assuming you haven't specified any \command{sdist} options in the setup
|
|
|
|
script or config file), \command{sdist} creates the the archive of the
|
|
|
|
default format for the current platform. The default formats are:
|
|
|
|
\begin{tableii}{ll}{textrm}%
|
|
|
|
{Platform}{Default archive format for source distributions}
|
|
|
|
\lineii{Unix}{gzipped tar file (\file{.tar.gz})}
|
|
|
|
\lineii{Windows}{zip file}
|
|
|
|
\end{tableii}
|
2000-04-19 19:48:09 -03:00
|
|
|
You can specify as many formats as you like using the
|
|
|
|
\longprogramopt{formats} option, for example:
|
2000-04-09 01:06:44 -03:00
|
|
|
\begin{verbatim}
|
|
|
|
python setup.py sdist --formats=gztar,zip
|
|
|
|
\end{verbatim}
|
|
|
|
to create a gzipped tarball and a zip file. The available formats are:
|
2000-04-13 22:53:36 -03:00
|
|
|
\begin{tableiii}{l|l|c}{code}%
|
|
|
|
{Format}{Description}{Notes}
|
|
|
|
\lineiii{zip}{zip file (\file{.zip})}{(1)}
|
|
|
|
\lineiii{gztar}{gzipped tar file (\file{.tar.gz})}{(2)}
|
|
|
|
\lineiii{ztar}{compressed tar file (\file{.tar.Z})}{}
|
|
|
|
\lineiii{tar}{tar file (\file{.tar})}{}
|
|
|
|
\end{tableiii}
|
|
|
|
|
|
|
|
\noindent Notes:
|
|
|
|
\begin{description}
|
|
|
|
\item[(1)] default on Windows
|
|
|
|
\item[(2)] default on Unix
|
|
|
|
\end{description}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{The manifest and manifest template}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{manifest}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
Without any additional information, the \command{sdist} command puts a
|
|
|
|
minimal set of files into the source distribution:
|
|
|
|
\begin{itemize}
|
2000-04-09 01:32:40 -03:00
|
|
|
\item all Python source files implied by the \option{py\_modules} and
|
2000-04-09 01:06:44 -03:00
|
|
|
\option{packages} options
|
2000-04-09 01:32:40 -03:00
|
|
|
\item all C source files mentioned in the \option{ext\_modules} or
|
2000-04-09 01:06:44 -03:00
|
|
|
\option{libraries} options (\XXX{getting C library sources currently
|
2000-04-09 01:32:40 -03:00
|
|
|
broken -- no get\_source\_files() method in build\_clib.py!})
|
2000-04-09 01:06:44 -03:00
|
|
|
\item anything that looks like a test script: \file{test/test*.py}
|
|
|
|
(currently, the Distutils don't do anything with test scripts except
|
|
|
|
include them in source distributions, but in the future there will be
|
|
|
|
a standard for testing Python module distributions)
|
|
|
|
\item \file{README.txt} (or \file{README}) and \file{setup.py}
|
|
|
|
\end{itemize}
|
|
|
|
Sometimes this is enough, but usually you will want to specify
|
|
|
|
additional files to distribute. The typical way to do this is to write
|
|
|
|
a \emph{manifest template}, called \file{MANIFEST.in} by default. The
|
|
|
|
\command{sdist} command processes this template and generates a manifest
|
|
|
|
file, \file{MANIFEST}. (If you prefer, you can skip the manifest
|
|
|
|
template and generate the manifest yourself: it just lists one file per
|
|
|
|
line.)
|
|
|
|
|
|
|
|
The manifest template has one command per line, where each command
|
|
|
|
specifies a set of files to include or exclude from the source
|
|
|
|
distribution. For an example, again we turn to the Distutils' own
|
|
|
|
manifest template:
|
|
|
|
\begin{verbatim}
|
|
|
|
include *.txt
|
2000-04-21 01:35:25 -03:00
|
|
|
recursive-include examples *.txt *.py
|
2000-04-09 01:06:44 -03:00
|
|
|
prune examples/sample?/build
|
|
|
|
\end{verbatim}
|
|
|
|
The meanings should be fairly clear: include all files in the
|
|
|
|
distribution root matching \code{*.txt}, all files anywhere under the
|
|
|
|
\file{examples} directory matching \code{*.txt} or \code{*.py}, and
|
|
|
|
exclude all directories matching \code{examples/sample?/build}. There
|
|
|
|
are several other commands available in the manifest template
|
2000-04-28 14:12:24 -03:00
|
|
|
mini-language; see section~\ref{sdist-cmd}.
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
The order of commands in the manifest template very much matters:
|
|
|
|
initially, we have the list of default files as described above, and
|
|
|
|
each command in the template adds to or removes from that list of files.
|
|
|
|
When we have fully processed the manifest template, we have our complete
|
|
|
|
list of files. This list is written to the manifest for future
|
|
|
|
reference, and then used to build the source distribution archive(s).
|
|
|
|
|
2000-04-13 22:53:36 -03:00
|
|
|
Following the Distutils' own manifest template, let's trace how the
|
|
|
|
\command{sdist} command will build the list of files to include in the
|
|
|
|
Distutils source distribution:
|
2000-04-09 01:06:44 -03:00
|
|
|
\begin{enumerate}
|
|
|
|
\item include all Python source files in the \file{distutils} and
|
|
|
|
\file{distutils/command} subdirectories (because packages
|
|
|
|
corresponding to those two directories were mentioned in the
|
|
|
|
\option{packages} option in the setup script)
|
|
|
|
\item include \file{test/test*.py} (always included)
|
|
|
|
\item include \file{README.txt} and \file{setup.py} (always included)
|
|
|
|
\item include \file{*.txt} in the distribution root (this will find
|
|
|
|
\file{README.txt} a second time, but such redundancies are weeded out
|
|
|
|
later)
|
|
|
|
\item in the sub-tree under \file{examples}, include anything matching
|
|
|
|
\file{*.txt}
|
|
|
|
\item in the sub-tree under \file{examples}, include anything matching
|
|
|
|
\file{*.py}
|
|
|
|
\item remove all files in the sub-trees starting at directories matching
|
|
|
|
\file{examples/sample?/build}---this may exclude files included by the
|
|
|
|
previous two steps, so it's important that the \code{prune} command in
|
|
|
|
the manifest template comes after the two \code{recursive-include}
|
|
|
|
commands
|
2000-04-09 01:32:40 -03:00
|
|
|
\end{enumerate}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-13 22:53:36 -03:00
|
|
|
Just like in the setup script, file and directory names in the manifest
|
|
|
|
template should always be slash-separated; the Distutils will take care
|
|
|
|
of converting them to the standard representation on your platform.
|
|
|
|
That way, the manifest template is portable across operating systems.
|
|
|
|
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
\subsection{Manifest-related options}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{manifest-options}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
The normal course of operations for the \command{sdist} command is as
|
|
|
|
follows:
|
|
|
|
\begin{itemize}
|
2000-04-13 22:53:36 -03:00
|
|
|
\item if the manifest file, \file{MANIFEST} doesn't exist, read
|
|
|
|
\file{MANIFEST.in} and create the manifest
|
|
|
|
\item if \file{MANIFEST.in} is more recent than \file{MANIFEST},
|
|
|
|
recreate \file{MANIFEST} by reading \file{MANIFEST.in}
|
2000-04-09 01:06:44 -03:00
|
|
|
\item use the list of files now in \file{MANIFEST} (either just
|
|
|
|
generated or read in) to create the source distribution archive(s)
|
|
|
|
\end{itemize}
|
|
|
|
There are a couple of options that modify this behaviour.
|
|
|
|
|
|
|
|
First, you might want to force the manifest to be regenerated---for
|
|
|
|
example, if you have added or removed files or directories that match an
|
|
|
|
existing pattern in the manifest template, you should regenerate the
|
|
|
|
manifest:
|
|
|
|
\begin{verbatim}
|
|
|
|
python setup.py sdist --force-manifest
|
|
|
|
\end{verbatim}
|
|
|
|
\XXX{this is stupid, but is there a better way to do it without
|
|
|
|
reprocessing MANIFEST.in every single bloody time?}
|
|
|
|
|
|
|
|
Or, you might just want to (re)generate the manifest, but not create a
|
|
|
|
source distribution:
|
|
|
|
\begin{verbatim}
|
|
|
|
python setup.py sdist --manifest-only
|
|
|
|
\end{verbatim}
|
2000-04-19 19:34:11 -03:00
|
|
|
(\longprogramopt{manifest-only} implies \longprogramopt{force-manifest}.)
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
If you don't want to use the default file set, you can supply the
|
2000-04-19 19:48:09 -03:00
|
|
|
\longprogramopt{no-defaults} option. If you use
|
|
|
|
\longprogramopt{no-defaults} and don't supply a manifest template (or
|
|
|
|
it's empty, or nothing matches the patterns in it), then your source
|
|
|
|
distribution will be empty.
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\section{Creating Built Distributions}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{built-dist}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-13 22:53:36 -03:00
|
|
|
A ``built distribution'' is what you're probably used to thinking of
|
|
|
|
either as a ``binary package'' or an ``installer'' (depending on your
|
|
|
|
background). It's not necessarily binary, though, because it might
|
|
|
|
contain only Python source code and/or byte-code; and we don't call it a
|
|
|
|
package, because that word is already spoken for in Python. (And
|
|
|
|
``installer'' is a term specific to the Windows world. \XXX{do Mac
|
|
|
|
people use it?})
|
|
|
|
|
|
|
|
A built distribution is how you make life as easy as possible for
|
|
|
|
installers of your module distribution: for users of RPM-based Linux
|
|
|
|
systems, it's a binary RPM; for Windows users, it's an executable
|
|
|
|
installer; for Debian-based Linux users, it's a Debian package; and so
|
|
|
|
forth. Obviously, no one person will be able to create built
|
|
|
|
distributions for every platform under the sun, so the Distutils is
|
|
|
|
designed to enable module developers to concentrate on their
|
|
|
|
specialty---writing code and creating source distributions---while an
|
|
|
|
intermediary species of \emph{packager} springs up to turn source
|
|
|
|
distributions into build distributions for as many platforms as there
|
|
|
|
are packagers.
|
|
|
|
|
|
|
|
Of course, the module developer could be his own packager; or the
|
|
|
|
packager could be a volunteer ``out there'' somewhere who has access to
|
|
|
|
a platform which the original developer does not; or it could be
|
|
|
|
software periodically grabbing new source distributions and turning them
|
|
|
|
into built distributions for as many platforms as the software has
|
|
|
|
access to. Regardless of the nature of the beast, a packager uses the
|
|
|
|
setup script and the \command{bdist} command family to generate built
|
|
|
|
distributions.
|
|
|
|
|
|
|
|
As a simple example, if I run the following command in the Distutils
|
|
|
|
source tree:
|
|
|
|
\begin{verbatim}
|
|
|
|
python setup.py bdist
|
|
|
|
\end{verbatim}
|
|
|
|
then the Distutils builds my module distribution (the Distutils itself
|
|
|
|
in this case), does a ``fake'' installation (also in the \file{build}
|
|
|
|
directory), and creates the default type of built distribution for my
|
|
|
|
platform. In Distutils 0.8, only two types of built distribution are
|
|
|
|
supported: \code{gztar} (default on non-Linux Unix) and \code{zip}
|
|
|
|
(default on Windows). Thus, the above command on a Unix system creates
|
|
|
|
\file{Distutils-0.8.built-posix.tar.gz}; unpacking this tarball from
|
|
|
|
Python's \filevar{prefix} directory installs the Distutils just as
|
|
|
|
though you had downloaded the source distribution and run \code{python
|
|
|
|
setup.py install}. Obviously, for pure Python distributions, this
|
|
|
|
isn't a huge win---but for non-pure distributions, which include
|
|
|
|
extensions that would need to be compiled, it can mean the difference
|
|
|
|
between someone being able to use your extensions or not.
|
|
|
|
|
|
|
|
\XXX{filenames are inaccurate here!}
|
|
|
|
|
2000-04-19 19:48:09 -03:00
|
|
|
The \command{bdist} command has a \longprogramopt{format} option,
|
|
|
|
similar to the \command{sdist} command, that you can use to select which
|
|
|
|
formats to generate: for example,
|
2000-04-13 22:53:36 -03:00
|
|
|
\begin{verbatim}
|
|
|
|
python setup.py bdist --format=zip
|
|
|
|
\end{verbatim}
|
|
|
|
would, when run on a Unix system, create
|
|
|
|
\file{Distutils-0.8.built-posix.tar.gz}---again, this archive would be
|
|
|
|
unpacked from Python's \filevar{prefix} directory to install the
|
|
|
|
Distutils.
|
|
|
|
|
|
|
|
The available formats for built distributions are:
|
|
|
|
\begin{tableiii}{l|l|c}{code}%
|
|
|
|
{Format}{Description}{Notes}
|
|
|
|
\lineiii{zip}{zip file (\file{.zip})}{(1)}
|
|
|
|
\lineiii{gztar}{gzipped tar file (\file{.tar.gz})}{(2)}
|
|
|
|
\lineiii{ztar}{compressed tar file (\file{.tar.Z})}{}
|
|
|
|
\lineiii{tar}{tar file (\file{.tar})}{}
|
|
|
|
\lineiii{rpm}{RPM}{(3)}
|
|
|
|
\lineiii{srpm}{source RPM}{}
|
|
|
|
\lineiii{wise}{Wise installer for Windows}{}
|
|
|
|
\end{tableiii}
|
|
|
|
|
|
|
|
\noindent Notes:
|
|
|
|
\begin{description}
|
|
|
|
\item[(1)] default on Windows
|
|
|
|
\item[(2)] default on Unix
|
|
|
|
\item[(3)] not implemented yet; will be default on RPM-based Linux
|
|
|
|
systems
|
|
|
|
\item[(5)] not implemented yet; will be default on Windows
|
|
|
|
\end{description}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-13 22:53:36 -03:00
|
|
|
You don't have to use the \command{bdist} command with the
|
2000-04-19 19:48:09 -03:00
|
|
|
\longprogramopt{formats} option; you can also use the command that
|
|
|
|
directly implements the format you're interested in. Many of these
|
2000-04-13 22:53:36 -03:00
|
|
|
\command{bdist} ``sub-commands'' actually generate several similar
|
|
|
|
formats; for instance, the \command{bdist\_dumb} command generates all
|
|
|
|
the ``dumb'' archive formats (\code{tar}, \code{ztar}, \code{gztar}, and
|
|
|
|
\code{zip}), and \command{bdist\_rpm} generates both binary and source
|
|
|
|
RPMs. The \command{bdist} sub-commands, and the formats generated by
|
|
|
|
each, are:
|
|
|
|
\begin{tableii}{l|l}{command}%
|
|
|
|
{Command}{Formats}
|
|
|
|
\lineii{bdist\_dumb}{tar, ztar, gztar, zip}
|
|
|
|
\lineii{bdist\_rpm}{rpm, srpm}
|
|
|
|
\lineii{bdist\_wise}{wise}
|
|
|
|
\end{tableii}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
\section{Examples}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{examples}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{Pure Python distribution (by module)}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{pure-mod}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{Pure Python distribution (by package)}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{pure-pkg}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{Single extension module}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{single-ext}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{Multiple extension modules}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{multiple-ext}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{Putting it all together}
|
|
|
|
|
|
|
|
|
2000-04-24 23:57:36 -03:00
|
|
|
|
|
|
|
\section{Extending the Distutils}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{extending}
|
2000-04-24 23:57:36 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{Extending existing commands}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{extend-existing}
|
2000-04-24 23:57:36 -03:00
|
|
|
|
|
|
|
|
|
|
|
\subsection{Writing new commands}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{new-commands}
|
2000-04-24 23:57:36 -03:00
|
|
|
|
|
|
|
|
|
|
|
|
2000-04-09 01:06:44 -03:00
|
|
|
\section{Reference}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{ref}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsection{Building modules: the \protect\command{build} command family}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{build-cmds}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsubsection{\protect\command{build}}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{build-cmd}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsubsection{\protect\command{build\_py}}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{build-py-cmd}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsubsection{\protect\command{build\_ext}}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{build-ext-cmd}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsubsection{\protect\command{build\_clib}}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{build-clib-cmd}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsection{Installing modules: the \protect\command{install} command family}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{install-cmd}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-05-11 21:58:18 -03:00
|
|
|
The install command ensures that the build commands have been run and then
|
|
|
|
runs the subcommands \command{install\_lib},
|
|
|
|
\command{install\_data} and
|
|
|
|
\command{install\_scripts}.
|
|
|
|
|
|
|
|
\subsubsection{\protect\command{install\_lib}}
|
|
|
|
\label{sec:install-lib-cmd}
|
|
|
|
|
|
|
|
\subsubsection{\protect\command{install\_data}}
|
|
|
|
\label{sec:install-data-cmd}
|
|
|
|
This command installs all data files provided with the distribution.
|
|
|
|
|
|
|
|
\subsubsection{\protect\command{install\_scripts}}
|
|
|
|
\label{sec:install-scripts-cmd}
|
|
|
|
This command installs all (Python) scripts in the distribution.
|
|
|
|
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsection{Cleaning up: the \protect\command{clean} command}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{clean-cmd}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsection{Creating a source distribution: the \protect\command{sdist} command}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{sdist-cmd}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
\XXX{fragment moved down from above: needs context!}
|
|
|
|
The manifest template commands are:
|
|
|
|
\begin{tableii}{ll}{command}{Command}{Description}
|
2000-04-21 01:35:25 -03:00
|
|
|
\lineii{include \var{pat1} \var{pat2} ... }
|
|
|
|
{include all files matching any of the listed patterns}
|
|
|
|
\lineii{exclude \var{pat1} \var{pat2} ... }
|
|
|
|
{exclude all files matching any of the listed patterns}
|
|
|
|
\lineii{recursive-include \var{dir} \var{pat1} \var{pat2} ... }
|
|
|
|
{include all files under \var{dir} matching any of the listed patterns}
|
|
|
|
\lineii{recursive-exclude \var{dir} \var{pat1} \var{pat2} ...}
|
|
|
|
{exclude all files under \var{dir} matching any of the listed patterns}
|
|
|
|
\lineii{global-include \var{pat1} \var{pat2} ...}
|
|
|
|
{include all files anywhere in the source tree matching
|
|
|
|
any of the listed patterns}
|
|
|
|
\lineii{global-exclude \var{pat1} \var{pat2} ...}
|
|
|
|
{exclude all files anywhere in the source tree matching
|
|
|
|
any of the listed patterns}
|
2000-04-09 01:06:44 -03:00
|
|
|
\lineii{prune \var{dir}}{exclude all files under \var{dir}}
|
|
|
|
\lineii{graft \var{dir}}{include all files under \var{dir}}
|
|
|
|
\end{tableii}
|
|
|
|
The patterns here are Unix-style ``glob'' patterns: \code{*} matches any
|
|
|
|
sequence of regular filename characters, \code{?} matches any single
|
|
|
|
regular filename character, and \code{[\var{range}]} matches any of the
|
|
|
|
characters in \var{range} (e.g., \code{a-z}, \code{a-zA-Z},
|
2000-04-09 01:32:40 -03:00
|
|
|
\code{a-f0-9\_.}). The definition of ``regular filename character'' is
|
2000-04-09 01:06:44 -03:00
|
|
|
platform-specific: on Unix it is anything except slash; on Windows
|
|
|
|
anything except backslash or colon; on Mac OS anything except colon.
|
|
|
|
\XXX{Windows and Mac OS support not there yet}
|
|
|
|
|
|
|
|
|
2000-04-19 19:48:09 -03:00
|
|
|
\subsection{Creating a ``built'' distribution: the
|
|
|
|
\protect\command{bdist} command family}
|
2000-04-28 14:12:24 -03:00
|
|
|
\label{bdist-cmds}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsubsection{\protect\command{blib}}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsubsection{\protect\command{blib\_dumb}}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsubsection{\protect\command{blib\_rpm}}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
2000-04-09 01:32:40 -03:00
|
|
|
\subsubsection{\protect\command{blib\_wise}}
|
2000-04-09 01:06:44 -03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2000-02-25 20:52:48 -04:00
|
|
|
\end{document}
|