\documentclass{howto} \usepackage{distutils} % $Id$ % TODO % Document extension.read_setup_file % Document build_clib command % \title{Distributing Python Modules} \author{Greg Ward} \authoraddress{Email: \email{distutils-sig@python.org}} \makeindex \begin{document} \maketitle \begin{abstract} \noindent This document describes the Python Distribution Utilities (``Distutils'') from the module developer's point of view, describing how to use the Distutils to make Python modules and extensions easily available to a wider audience with very little overhead for build/release/install mechanics. \end{abstract} % The ugly "%begin{latexonly}" pseudo-environment supresses the table % of contents for HTML generation. % %begin{latexonly} \tableofcontents %end{latexonly} \section{Introduction} \label{intro} This document covers using the Distutils to distribute your Python modules, concentrating on the role of developer/distributor: if you're looking for information on installing Python modules, you should refer to the \citetitle[../inst/inst.html]{Installing Python Modules} manual. \section{Concepts \& Terminology} \label{concepts} Using the Distutils is quite simple, both for module developers and for users/administrators installing third-party modules. As a developer, your responsibilities (apart from writing solid, well-documented and well-tested code, of course!) are: \begin{itemize} \item write a setup script (\file{setup.py} by convention) \item (optional) write a setup configuration file \item create a source distribution \item (optional) create one or more built (binary) distributions \end{itemize} Each of these tasks is covered in this document. Not all module developers have access to a multitude of platforms, so it's not always feasible to expect them to create a multitude of built distributions. It is hoped that a class of intermediaries, called \emph{packagers}, will arise to address this need. Packagers will take source distributions released by module developers, build them on one or more platforms, and release the resulting built distributions. Thus, users on the most popular platforms will be able to install most popular Python module distributions in the most natural way for their platform, without having to run a single setup script or compile a line of code. \subsection{A Simple Example} \label{simple-example} The setup script is usually quite simple, although since it's written in Python, there are no arbitrary limits to what you can do with it, though you should be careful about putting arbitrarily expensive operations in your setup script. Unlike, say, Autoconf-style configure scripts, the setup script may be run multiple times in the course of building and installing your module distribution. If you need to insert potentially expensive processing steps into the Distutils chain, see section~\ref{extending} on extending the Distutils. If all you want to do is distribute a module called \module{foo}, contained in a file \file{foo.py}, then your setup script can be as simple as this: \begin{verbatim} from distutils.core import setup setup(name="foo", version="1.0", py_modules=["foo"]) \end{verbatim} Some observations: \begin{itemize} \item most information that you supply to the Distutils is supplied as keyword arguments to the \function{setup()} function \item those keyword arguments fall into two categories: package metadata (name, version number) and information about what's in the package (a list of pure Python modules, in this case) \item modules are specified by module name, not filename (the same will hold true for packages and extensions) \item it's recommended that you supply a little more metadata, in particular your name, email address and a URL for the project (see section~\ref{setup-script} for an example) \end{itemize} To create a source distribution for this module, you would create a setup script, \file{setup.py}, containing the above code, and run: \begin{verbatim} python setup.py sdist \end{verbatim} which will create an archive file (e.g., tarball on \UNIX, ZIP file on Windows) containing your setup script \file{setup.py}, and your module \file{foo.py}. The archive file will be named \file{foo-1.0.tar.gz} (or \file{.zip}), and will unpack into a directory \file{foo-1.0}. If an end-user wishes to install your \module{foo} module, all she has to do is download \file{foo-1.0.tar.gz} (or \file{.zip}), unpack it, and---from the \file{foo-1.0} directory---run \begin{verbatim} python setup.py install \end{verbatim} which will ultimately copy \file{foo.py} to the appropriate directory for third-party modules in their Python installation. This simple example demonstrates some fundamental concepts of the Distutils. First, both developers and installers have the same basic user interface, i.e. the setup script. The difference is which Distutils \emph{commands} they use: the \command{sdist} command is almost exclusively for module developers, while \command{install} is more often for installers (although most developers will want to install their own code occasionally). If you want to make things really easy for your users, you can create one or more built distributions for them. For instance, if you are running on a Windows machine, and want to make things easy for other Windows users, you can create an executable installer (the most appropriate type of built distribution for this platform) with the \command{bdist\_wininst} command. For example: \begin{verbatim} python setup.py bdist_wininst \end{verbatim} will create an executable installer, \file{foo-1.0.win32.exe}, in the current directory. Other useful built distribution formats are RPM, implemented by the \command{bdist\_rpm} command, Solaris \program{pkgtool} (\command{bdist\_pkgtool}), and HP-UX \program{swinstall} (\command{bdist_sdux}). For example, the following command will create an RPM file called \file{foo-1.0.noarch.rpm}: \begin{verbatim} python setup.py bdist_rpm \end{verbatim} (The \command{bdist\_rpm} command uses the \command{rpm} executable, therefore this has to be run on an RPM-based system such as Red Hat Linux, SuSE Linux, or Mandrake Linux.) You can find out what distribution formats are available at any time by running \begin{verbatim} python setup.py bdist --help-formats \end{verbatim} \subsection{General Python terminology} \label{python-terms} If you're reading this document, you probably have a good idea of what modules, extensions, and so forth are. Nevertheless, just to be sure that everyone is operating from a common starting point, we offer the following glossary of common Python terms: \begin{description} \item[module] the basic unit of code reusability in Python: a block of code imported by some other code. Three types of modules concern us here: pure Python modules, extension modules, and packages. \item[pure Python module] a module written in Python and contained in a single \file{.py} file (and possibly associated \file{.pyc} and/or \file{.pyo} files). Sometimes referred to as a ``pure module.'' \item[extension module] a module written in the low-level language of the Python implementation: C/C++ for Python, Java for Jython. Typically contained in a single dynamically loadable pre-compiled file, e.g. a shared object (\file{.so}) file for Python extensions on \UNIX, a DLL (given the \file{.pyd} extension) for Python extensions on Windows, or a Java class file for Jython extensions. (Note that currently, the Distutils only handles C/C++ extensions for Python.) \item[package] a module that contains other modules; typically contained in a directory in the filesystem and distinguished from other directories by the presence of a file \file{\_\_init\_\_.py}. \item[root package] the root of the hierarchy of packages. (This isn't really a package, since it doesn't have an \file{\_\_init\_\_.py} file. But we have to call it something.) The vast majority of the standard library is in the root package, as are many small, standalone third-party modules that don't belong to a larger module collection. Unlike regular packages, modules in the root package can be found in many directories: in fact, every directory listed in \code{sys.path} contributes modules to the root package. \end{description} \subsection{Distutils-specific terminology} \label{distutils-term} The following terms apply more specifically to the domain of distributing Python modules using the Distutils: \begin{description} \item[module distribution] a collection of Python modules distributed together as a single downloadable resource and meant to be installed \emph{en masse}. Examples of some well-known module distributions are Numeric Python, PyXML, PIL (the Python Imaging Library), or mxBase. (This would be called a \emph{package}, except that term is already taken in the Python context: a single module distribution may contain zero, one, or many Python packages.) \item[pure module distribution] a module distribution that contains only pure Python modules and packages. Sometimes referred to as a ``pure distribution.'' \item[non-pure module distribution] a module distribution that contains at least one extension module. Sometimes referred to as a ``non-pure distribution.'' \item[distribution root] the top-level directory of your source tree (or source distribution); the directory where \file{setup.py} exists. Generally \file{setup.py} will be run from this directory. \end{description} \section{Writing the Setup Script} \label{setup-script} The setup script is the centre of all activity in building, distributing, and installing modules using the Distutils. The main purpose of the setup script is to describe your module distribution to the Distutils, so that the various commands that operate on your modules do the right thing. As we saw in section~\ref{simple-example} above, the setup script consists mainly of a call to \function{setup()}, and most information supplied to the Distutils by the module developer is supplied as keyword arguments to \function{setup()}. Here's a slightly more involved example, which we'll follow for the next couple of sections: the Distutils' own setup script. (Keep in mind that although the Distutils are included with Python 1.6 and later, they also have an independent existence so that Python 1.5.2 users can use them to install other module distributions. The Distutils' own setup script, shown here, is used to install the package into Python 1.5.2.) \begin{verbatim} #!/usr/bin/env python from distutils.core import setup setup(name="Distutils", version="1.0", description="Python Distribution Utilities", author="Greg Ward", author_email="gward@python.net", url="http://www.python.org/sigs/distutils-sig/", packages=['distutils', 'distutils.command'], ) \end{verbatim} There are only two differences between this and the trivial one-file distribution presented in section~\ref{simple-example}: more metadata, and the specification of pure Python modules by package, rather than by module. This is important since the Distutils consist of a couple of dozen modules split into (so far) two packages; an explicit list of every module would be tedious to generate and difficult to maintain. For more information on the additional meta-data, see section~\ref{meta-data}. Note that any pathnames (files or directories) supplied in the setup script should be written using the \UNIX{} convention, i.e. slash-separated. The Distutils will take care of converting this platform-neutral representation into whatever is appropriate on your current platform before actually using the pathname. This makes your setup script portable across operating systems, which of course is one of the major goals of the Distutils. In this spirit, all pathnames in this document are slash-separated. (MacOS programmers should keep in mind that the \emph{absence} of a leading slash indicates a relative path, the opposite of the MacOS convention with colons.) This, of course, only applies to pathnames given to Distutils functions. If you, for example, use standard python functions such as \function{glob.glob} or \function{os.listdir} to specify files, you should be careful to write portable code instead of hardcoding path separators: \begin{verbatim} glob.glob(os.path.join('mydir', 'subdir', '*.html')) os.listdir(os.path.join('mydir', 'subdir')) \end{verbatim} \subsection{Listing whole packages} \label{listing-packages} The \option{packages} option tells the Distutils to process (build, distribute, install, etc.) all pure Python modules found in each package mentioned in the \option{packages} list. In order to do this, of course, there has to be a correspondence between package names and directories in the filesystem. The default correspondence is the most obvious one, i.e. package \module{distutils} is found in the directory \file{distutils} relative to the distribution root. Thus, when you say \code{packages = ['foo']} in your setup script, you are promising that the Distutils will find a file \file{foo/\_\_init\_\_.py} (which might be spelled differently on your system, but you get the idea) relative to the directory where your setup script lives. If you break this promise, the Distutils will issue a warning but still process the broken package anyways. If you use a different convention to lay out your source directory, that's no problem: you just have to supply the \option{package\_dir} option to tell the Distutils about your convention. For example, say you keep all Python source under \file{lib}, so that modules in the ``root package'' (i.e., not in any package at all) are in \file{lib}, modules in the \module{foo} package are in \file{lib/foo}, and so forth. Then you would put \begin{verbatim} package_dir = {'': 'lib'} \end{verbatim} in your setup script. The keys to this dictionary are package names, and an empty package name stands for the root package. The values are directory names relative to your distribution root. In this case, when you say \code{packages = ['foo']}, you are promising that the file \file{lib/foo/\_\_init\_\_.py} exists. Another possible convention is to put the \module{foo} package right in \file{lib}, the \module{foo.bar} package in \file{lib/bar}, etc. This would be written in the setup script as \begin{verbatim} package_dir = {'foo': 'lib'} \end{verbatim} A \code{\var{package}: \var{dir}} entry in the \option{package\_dir} dictionary implicitly applies to all packages below \var{package}, so the \module{foo.bar} case is automatically handled here. In this example, having \code{packages = ['foo', 'foo.bar']} tells the Distutils to look for \file{lib/\_\_init\_\_.py} and \file{lib/bar/\_\_init\_\_.py}. (Keep in mind that although \option{package\_dir} applies recursively, you must explicitly list all packages in \option{packages}: the Distutils will \emph{not} recursively scan your source tree looking for any directory with an \file{\_\_init\_\_.py} file.) \subsection{Listing individual modules} \label{listing-modules} For a small module distribution, you might prefer to list all modules rather than listing packages---especially the case of a single module that goes in the ``root package'' (i.e., no package at all). This simplest case was shown in section~\ref{simple-example}; here is a slightly more involved example: \begin{verbatim} py_modules = ['mod1', 'pkg.mod2'] \end{verbatim} This describes two modules, one of them in the ``root'' package, the other in the \module{pkg} package. Again, the default package/directory layout implies that these two modules can be found in \file{mod1.py} and \file{pkg/mod2.py}, and that \file{pkg/\_\_init\_\_.py} exists as well. And again, you can override the package/directory correspondence using the \option{package\_dir} option. \subsection{Describing extension modules} \label{describing-extensions} % XXX read over this section Just as writing Python extension modules is a bit more complicated than writing pure Python modules, describing them to the Distutils is a bit more complicated. Unlike pure modules, it's not enough just to list modules or packages and expect the Distutils to go out and find the right files; you have to specify the extension name, source file(s), and any compile/link requirements (include directories, libraries to link with, etc.). All of this is done through another keyword argument to \function{setup()}, the \option{extensions} option. \option{extensions} is just a list of \class{Extension} instances, each of which describes a single extension module. Suppose your distribution includes a single extension, called \module{foo} and implemented by \file{foo.c}. If no additional instructions to the compiler/linker are needed, describing this extension is quite simple: \begin{verbatim} uExtension("foo", ["foo.c"]) \end{verbatim} The \class{Extension} class can be imported from \module{distutils.core} along with \function{setup()}. Thus, the setup script for a module distribution that contains only this one extension and nothing else might be: \begin{verbatim} from distutils.core import setup, Extension setup(name="foo", version="1.0", ext_modules=[Extension("foo", ["foo.c"])]) \end{verbatim} The \class{Extension} class (actually, the underlying extension-building machinery implemented by the \command{build\_ext} command) supports a great deal of flexibility in describing Python extensions, which is explained in the following sections. \subsubsection{Extension names and packages} The first argument to the \class{Extension} constructor is always the name of the extension, including any package names. For example, \begin{verbatim} Extension("foo", ["src/foo1.c", "src/foo2.c"]) \end{verbatim} describes an extension that lives in the root package, while \begin{verbatim} Extension("pkg.foo", ["src/foo1.c", "src/foo2.c"]) \end{verbatim} describes the same extension in the \module{pkg} package. The source files and resulting object code are identical in both cases; the only difference is where in the filesystem (and therefore where in Python's namespace hierarchy) the resulting extension lives. If you have a number of extensions all in the same package (or all under the same base package), use the \option{ext\_package} keyword argument to \function{setup()}. For example, \begin{verbatim} setup(... ext_package="pkg", ext_modules=[Extension("foo", ["foo.c"]), Extension("subpkg.bar", ["bar.c"])] ) \end{verbatim} will compile \file{foo.c} to the extension \module{pkg.foo}, and \file{bar.c} to \module{pkg.subpkg.bar}. \subsubsection{Extension source files} The second argument to the \class{Extension} constructor is a list of source files. Since the Distutils currently only support C, \Cpp, and Objective-C extensions, these are normally C/\Cpp/Objective-C source files. (Be sure to use appropriate extensions to distinguish \Cpp\ source files: \file{.cc} and \file{.cpp} seem to be recognized by both \UNIX{} and Windows compilers.) However, you can also include SWIG interface (\file{.i}) files in the list; the \command{build\_ext} command knows how to deal with SWIG extensions: it will run SWIG on the interface file and compile the resulting C/C++ file into your extension. \XXX{SWIG support is rough around the edges and largely untested; especially SWIG support of C++ extensions! Explain in more detail here when the interface firms up.} On some platforms, you can include non-source files that are processed by the compiler and included in your extension. Currently, this just means Windows message text (\file{.mc}) files and resource definition (\file{.rc}) files for Visual C++. These will be compiled to binary resource (\file{.res}) files and linked into the executable. \subsubsection{Preprocessor options} Three optional arguments to \class{Extension} will help if you need to specify include directories to search or preprocessor macros to define/undefine: \code{include\_dirs}, \code{define\_macros}, and \code{undef\_macros}. For example, if your extension requires header files in the \file{include} directory under your distribution root, use the \code{include\_dirs} option: \begin{verbatim} Extension("foo", ["foo.c"], include_dirs=["include"]) \end{verbatim} You can specify absolute directories there; if you know that your extension will only be built on \UNIX{} systems with X11R6 installed to \file{/usr}, you can get away with \begin{verbatim} Extension("foo", ["foo.c"], include_dirs=["/usr/include/X11"]) \end{verbatim} You should avoid this sort of non-portable usage if you plan to distribute your code: it's probably better to write C code like \begin{verbatim} #include \end{verbatim} If you need to include header files from some other Python extension, you can take advantage of the fact that header files are installed in a consistent way by the Distutils \command{install\_header} command. For example, the Numerical Python header files are installed (on a standard Unix installation) to \file{/usr/local/include/python1.5/Numerical}. (The exact location will differ according to your platform and Python installation.) Since the Python include directory---\file{/usr/local/include/python1.5} in this case---is always included in the search path when building Python extensions, the best approach is to write C code like \begin{verbatim} #include \end{verbatim} If you must put the \file{Numerical} include directory right into your header search path, though, you can find that directory using the Distutils \module{sysconfig} module: \begin{verbatim} from distutils.sysconfig import get_python_inc incdir = os.path.join(get_python_inc(plat_specific=1), "Numerical") setup(..., Extension(..., include_dirs=[incdir])) \end{verbatim} Even though this is quite portable---it will work on any Python installation, regardless of platform---it's probably easier to just write your C code in the sensible way. You can define and undefine pre-processor macros with the \code{define\_macros} and \code{undef\_macros} options. \code{define\_macros} takes a list of \code{(name, value)} tuples, where \code{name} is the name of the macro to define (a string) and \code{value} is its value: either a string or \code{None}. (Defining a macro \code{FOO} to \code{None} is the equivalent of a bare \code{\#define FOO} in your C source: with most compilers, this sets \code{FOO} to the string \code{1}.) \code{undef\_macros} is just a list of macros to undefine. For example: \begin{verbatim} Extension(..., define_macros=[('NDEBUG', '1')], ('HAVE_STRFTIME', None), undef_macros=['HAVE_FOO', 'HAVE_BAR']) \end{verbatim} is the equivalent of having this at the top of every C source file: \begin{verbatim} #define NDEBUG 1 #define HAVE_STRFTIME #undef HAVE_FOO #undef HAVE_BAR \end{verbatim} \subsubsection{Library options} You can also specify the libraries to link against when building your extension, and the directories to search for those libraries. The \code{libraries} option is a list of libraries to link against, \code{library\_dirs} is a list of directories to search for libraries at link-time, and \code{runtime\_library\_dirs} is a list of directories to search for shared (dynamically loaded) libraries at run-time. For example, if you need to link against libraries known to be in the standard library search path on target systems \begin{verbatim} Extension(..., libraries=["gdbm", "readline"]) \end{verbatim} If you need to link with libraries in a non-standard location, you'll have to include the location in \code{library\_dirs}: \begin{verbatim} Extension(..., library_dirs=["/usr/X11R6/lib"], libraries=["X11", "Xt"]) \end{verbatim} (Again, this sort of non-portable construct should be avoided if you intend to distribute your code.) \XXX{Should mention clib libraries here or somewhere else!} \subsubsection{Other options} There are still some other options which can be used to handle special cases. The \option{extra\_objects} option is a list of object files to be passed to the linker. These files must not have extensions, as the default extension for the compiler is used. \option{extra\_compile\_args} and \option{extra\_link\_args} can be used to specify additional command line options for the respective compiler and linker command lines. \option{export\_symbols} is only useful on Windows. It can contain a list of symbols (functions or variables) to be exported. This option is not needed when building compiled extensions: Distutils will automatically add \code{initmodule} to the list of exported symbols. \subsection{Installing Scripts} So far we have been dealing with pure and non-pure Python modules, which are usually not run by themselves but imported by scripts. Scripts are files containing Python source code, intended to be started from the command line. Scripts don't require Distutils to do anything very complicated. The only clever feature is that if the first line of the script starts with \code{\#!} and contains the word ``python'', the Distutils will adjust the first line to refer to the current interpreter location. The \option{scripts} option simply is a list of files to be handled in this way. From the PyXML setup script: \begin{verbatim} setup (... scripts = ['scripts/xmlproc_parse', 'scripts/xmlproc_val'] ) \end{verbatim} \subsection{Installing Additional Files} The \option{data\_files} option can be used to specify additional files needed by the module distribution: configuration files, message catalogs, data files, anything which doesn't fit in the previous categories. \option{data\_files} specifies a sequence of (\var{directory}, \var{files}) pairs in the following way: \begin{verbatim} setup(... data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']), ('config', ['cfg/data.cfg']), ('/etc/init.d', ['init-script'])] ) \end{verbatim} Note that you can specify the directory names where the data files will be installed, but you cannot rename the data files themselves. Each (\var{directory}, \var{files}) pair in the sequence specifies the installation directory and the files to install there. If \var{directory} is a relative path, it is interpreted relative to the installation prefix (Python's \code{sys.prefix} for pure-Python packages, \code{sys.exec_prefix} for packages that contain extension modules). Each file name in \var{files} is interpreted relative to the \file{setup.py} script at the top of the package source distribution. No directory information from \var{files} is used to determine the final location of the installed file; only the name of the file is used. You can specify the \option{data\_files} options as a simple sequence of files without specifying a target directory, but this is not recommended, and the \command{install} command will print a warning in this case. To install data files directly in the target directory, an empty string should be given as the directory. \subsection{Additional meta-data} \label{meta-data} The setup script may include additional meta-data beyond the name and version. This information includes: \begin{tableiii}{l|l|c}{code}% {Meta-Data}{Description}{Notes} \lineiii{name}{the name of the package}{(1)} \lineiii{version}{the version of this release}{(1)} \lineiii{author}{package author's name}{(2)} \lineiii{author_email}{email address of the package author}{(2)} \lineiii{maintainer}{package maintainer's name}{(2)} \lineiii{maintainer_email}{email address of the package maintainer}{(2)} \lineiii{url}{URL of the package's home page}{(1)} \lineiii{license}{the terms the package is released under}{} \lineiii{description}{a short, summary description of the package}{} \lineiii{long_description}{a longer description of the package}{} \lineiii{keywords}{some keywords appropriate to the package}{} \lineiii{platforms}{a list of the target platforms}{} \lineiii{classifiers}{a list of Trove classifiers}{(3),(4)} \lineiii{download_url}{a single URL containing the download location for this version of the package}{(3)} \end{tableiii} \noindent Notes: \begin{description} \item[(1)] these fields are required \item[(2)] either the author or the maintainer must be nominated \item[(3)] should not be used if your package is to be compatible with Python versions prior to 2.2.3 or 2.3. \item[(4)] The list of classifiers is available from the PyPI website (\url{http://www.python.org/pypi}). \end{description} \option{classifiers} are specified in a python list: \begin{verbatim} setup(... classifiers = [ 'Development Status :: 4 - Beta', 'Environment :: Console', 'Environment :: Web Environment', 'Intended Audience :: End Users/Desktop', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'License :: OSI Approved :: Python Software Foundation License', 'Operating System :: MacOS :: MacOS X', 'Operating System :: Microsoft :: Windows', 'Operating System :: POSIX', 'Programming Language :: Python', 'Topic :: Communications :: Email', 'Topic :: Office/Business', 'Topic :: Software Development :: Bug Tracking', ], ... ) \end{verbatim} If you wish to include classifiers in your \file{setup.py} file and also wish to remain backwards-compatible with Python releases prior to 2.2.3, then you can include the following code fragment in your \file{setup.py} before the \code{setup()} call. \begin{verbatim} # patch distutils if it can't cope with the "classifiers" keyword if sys.version < '2.2.3': from distutils.dist import DistributionMetadata DistributionMetadata.classifiers = None \end{verbatim} \section{Writing the Setup Configuration File} \label{setup-config} Often, it's not possible to write down everything needed to build a distribution \emph{a priori}: you may need to get some information from the user, or from the user's system, in order to proceed. As long as that information is fairly simple---a list of directories to search for C header files or libraries, for example---then providing a configuration file, \file{setup.cfg}, for users to edit is a cheap and easy way to solicit it. Configuration files also let you provide default values for any command option, which the installer can then override either on the command-line or by editing the config file. % (If you have more advanced needs, such as determining which extensions % to build based on what capabilities are present on the target system, % then you need the Distutils ``auto-configuration'' facility. This % started to appear in Distutils 0.9 but, as of this writing, isn't mature % or stable enough yet for real-world use.) The setup configuration file is a useful middle-ground between the setup script---which, ideally, would be opaque to installers\footnote{This ideal probably won't be achieved until auto-configuration is fully supported by the Distutils.}---and the command-line to the setup script, which is outside of your control and entirely up to the installer. In fact, \file{setup.cfg} (and any other Distutils configuration files present on the target system) are processed after the contents of the setup script, but before the command-line. This has several useful consequences: \begin{itemize} \item installers can override some of what you put in \file{setup.py} by editing \file{setup.cfg} \item you can provide non-standard defaults for options that are not easily set in \file{setup.py} \item installers can override anything in \file{setup.cfg} using the command-line options to \file{setup.py} \end{itemize} The basic syntax of the configuration file is simple: \begin{verbatim} [command] option=value ... \end{verbatim} where \var{command} is one of the Distutils commands (e.g. \command{build\_py}, \command{install}), and \var{option} is one of the options that command supports. Any number of options can be supplied for each command, and any number of command sections can be included in the file. Blank lines are ignored, as are comments, which run from a \character{\#} character until the end of the line. Long option values can be split across multiple lines simply by indenting the continuation lines. You can find out the list of options supported by a particular command with the universal \longprogramopt{help} option, e.g. \begin{verbatim} > python setup.py --help build_ext [...] Options for 'build_ext' command: --build-lib (-b) directory for compiled extension modules --build-temp (-t) directory for temporary files (build by-products) --inplace (-i) ignore build-lib and put compiled extensions into the source directory alongside your pure Python modules --include-dirs (-I) list of directories to search for header files --define (-D) C preprocessor macros to define --undef (-U) C preprocessor macros to undefine [...] \end{verbatim} Note that an option spelled \longprogramopt{foo-bar} on the command-line is spelled \option{foo\_bar} in configuration files. For example, say you want your extensions to be built ``in-place''---that is, you have an extension \module{pkg.ext}, and you want the compiled extension file (\file{ext.so} on \UNIX, say) to be put in the same source directory as your pure Python modules \module{pkg.mod1} and \module{pkg.mod2}. You can always use the \longprogramopt{inplace} option on the command-line to ensure this: \begin{verbatim} python setup.py build_ext --inplace \end{verbatim} But this requires that you always specify the \command{build\_ext} command explicitly, and remember to provide \longprogramopt{inplace}. An easier way is to ``set and forget'' this option, by encoding it in \file{setup.cfg}, the configuration file for this distribution: \begin{verbatim} [build_ext] inplace=1 \end{verbatim} This will affect all builds of this module distribution, whether or not you explcitly specify \command{build\_ext}. If you include \file{setup.cfg} in your source distribution, it will also affect end-user builds---which is probably a bad idea for this option, since always building extensions in-place would break installation of the module distribution. In certain peculiar cases, though, modules are built right in their installation directory, so this is conceivably a useful ability. (Distributing extensions that expect to be built in their installation directory is almost always a bad idea, though.) Another example: certain commands take a lot of options that don't change from run to run; for example, \command{bdist\_rpm} needs to know everything required to generate a ``spec'' file for creating an RPM distribution. Some of this information comes from the setup script, and some is automatically generated by the Distutils (such as the list of files installed). But some of it has to be supplied as options to \command{bdist\_rpm}, which would be very tedious to do on the command-line for every run. Hence, here is a snippet from the Distutils' own \file{setup.cfg}: \begin{verbatim} [bdist_rpm] release = 1 packager = Greg Ward doc_files = CHANGES.txt README.txt USAGE.txt doc/ examples/ \end{verbatim} Note that the \option{doc\_files} option is simply a whitespace-separated string split across multiple lines for readability. \begin{seealso} \seetitle[../inst/config-syntax.html]{Installing Python Modules}{More information on the configuration files is available in the manual for system administrators.} \end{seealso} \section{Creating a Source Distribution} \label{source-dist} As shown in section~\ref{simple-example}, you use the \command{sdist} command to create a source distribution. In the simplest case, \begin{verbatim} python setup.py sdist \end{verbatim} (assuming you haven't specified any \command{sdist} options in the setup script or config file), \command{sdist} creates the archive of the default format for the current platform. The default format is a gzip'ed tar file (\file{.tar.gz}) on \UNIX, and ZIP file on Windows. \XXX{no MacOS support here} You can specify as many formats as you like using the \longprogramopt{formats} option, for example: \begin{verbatim} python setup.py sdist --formats=gztar,zip \end{verbatim} to create a gzipped tarball and a zip file. The available formats are: \begin{tableiii}{l|l|c}{code}% {Format}{Description}{Notes} \lineiii{zip}{zip file (\file{.zip})}{(1),(3)} \lineiii{gztar}{gzip'ed tar file (\file{.tar.gz})}{(2),(4)} \lineiii{bztar}{bzip2'ed tar file (\file{.tar.bz2})}{(4)} \lineiii{ztar}{compressed tar file (\file{.tar.Z})}{(4)} \lineiii{tar}{tar file (\file{.tar})}{(4)} \end{tableiii} \noindent Notes: \begin{description} \item[(1)] default on Windows \item[(2)] default on \UNIX \item[(3)] requires either external \program{zip} utility or \module{zipfile} module (part of the standard Python library since Python~1.6) \item[(4)] requires external utilities: \program{tar} and possibly one of \program{gzip}, \program{bzip2}, or \program{compress} \end{description} \subsection{Specifying the files to distribute} \label{manifest} If you don't supply an explicit list of files (or instructions on how to generate one), the \command{sdist} command puts a minimal default set into the source distribution: \begin{itemize} \item all Python source files implied by the \option{py\_modules} and \option{packages} options \item all C source files mentioned in the \option{ext\_modules} or \option{libraries} options (\XXX{getting C library sources currently broken -- no get\_source\_files() method in build\_clib.py!}) \item anything that looks like a test script: \file{test/test*.py} (currently, the Distutils don't do anything with test scripts except include them in source distributions, but in the future there will be a standard for testing Python module distributions) \item \file{README.txt} (or \file{README}), \file{setup.py} (or whatever you called your setup script), and \file{setup.cfg} \end{itemize} Sometimes this is enough, but usually you will want to specify additional files to distribute. The typical way to do this is to write a \emph{manifest template}, called \file{MANIFEST.in} by default. The manifest template is just a list of instructions for how to generate your manifest file, \file{MANIFEST}, which is the exact list of files to include in your source distribution. The \command{sdist} command processes this template and generates a manifest based on its instructions and what it finds in the filesystem. If you prefer to roll your own manifest file, the format is simple: one filename per line, regular files (or symlinks to them) only. If you do supply your own \file{MANIFEST}, you must specify everything: the default set of files described above does not apply in this case. The manifest template has one command per line, where each command specifies a set of files to include or exclude from the source distribution. For an example, again we turn to the Distutils' own manifest template: \begin{verbatim} include *.txt recursive-include examples *.txt *.py prune examples/sample?/build \end{verbatim} The meanings should be fairly clear: include all files in the distribution root matching \code{*.txt}, all files anywhere under the \file{examples} directory matching \code{*.txt} or \code{*.py}, and exclude all directories matching \code{examples/sample?/build}. All of this is done \emph{after} the standard include set, so you can exclude files from the standard set with explicit instructions in the manifest template. (Or, you can use the \longprogramopt{no-defaults} option to disable the standard set entirely.) There are several other commands available in the manifest template mini-language; see section~\ref{sdist-cmd}. The order of commands in the manifest template matters: initially, we have the list of default files as described above, and each command in the template adds to or removes from that list of files. Once we have fully processed the manifest template, we remove files that should not be included in the source distribution: \begin{itemize} \item all files in the Distutils ``build'' tree (default \file{build/}) \item all files in directories named \file{RCS} or \file{CVS} \end{itemize} Now we have our complete list of files, which is written to the manifest for future reference, and then used to build the source distribution archive(s). You can disable the default set of included files with the \longprogramopt{no-defaults} option, and you can disable the standard exclude set with \longprogramopt{no-prune}. Following the Distutils' own manifest template, let's trace how the \command{sdist} command builds the list of files to include in the Distutils source distribution: \begin{enumerate} \item include all Python source files in the \file{distutils} and \file{distutils/command} subdirectories (because packages corresponding to those two directories were mentioned in the \option{packages} option in the setup script---see section~\ref{setup-script}) \item include \file{README.txt}, \file{setup.py}, and \file{setup.cfg} (standard files) \item include \file{test/test*.py} (standard files) \item include \file{*.txt} in the distribution root (this will find \file{README.txt} a second time, but such redundancies are weeded out later) \item include anything matching \file{*.txt} or \file{*.py} in the sub-tree under \file{examples}, \item exclude all files in the sub-trees starting at directories matching \file{examples/sample?/build}---this may exclude files included by the previous two steps, so it's important that the \code{prune} command in the manifest template comes after the \code{recursive-include} command \item exclude the entire \file{build} tree, and any \file{RCS} or \file{CVS} directories \end{enumerate} Just like in the setup script, file and directory names in the manifest template should always be slash-separated; the Distutils will take care of converting them to the standard representation on your platform. That way, the manifest template is portable across operating systems. \subsection{Manifest-related options} \label{manifest-options} The normal course of operations for the \command{sdist} command is as follows: \begin{itemize} \item if the manifest file, \file{MANIFEST} doesn't exist, read \file{MANIFEST.in} and create the manifest \item if neither \file{MANIFEST} nor \file{MANIFEST.in} exist, create a manifest with just the default file set \item if either \file{MANIFEST.in} or the setup script (\file{setup.py}) are more recent than \file{MANIFEST}, recreate \file{MANIFEST} by reading \file{MANIFEST.in} \item use the list of files now in \file{MANIFEST} (either just generated or read in) to create the source distribution archive(s) \end{itemize} There are a couple of options that modify this behaviour. First, use the \longprogramopt{no-defaults} and \longprogramopt{no-prune} to disable the standard ``include'' and ``exclude'' sets. Second, you might want to force the manifest to be regenerated---for example, if you have added or removed files or directories that match an existing pattern in the manifest template, you should regenerate the manifest: \begin{verbatim} python setup.py sdist --force-manifest \end{verbatim} Or, you might just want to (re)generate the manifest, but not create a source distribution: \begin{verbatim} python setup.py sdist --manifest-only \end{verbatim} \longprogramopt{manifest-only} implies \longprogramopt{force-manifest}. \programopt{-o} is a shortcut for \longprogramopt{manifest-only}, and \programopt{-f} for \longprogramopt{force-manifest}. \section{Creating Built Distributions} \label{built-dist} A ``built distribution'' is what you're probably used to thinking of either as a ``binary package'' or an ``installer'' (depending on your background). It's not necessarily binary, though, because it might contain only Python source code and/or byte-code; and we don't call it a package, because that word is already spoken for in Python. (And ``installer'' is a term specific to the Windows world. \XXX{do Mac people use it?}) A built distribution is how you make life as easy as possible for installers of your module distribution: for users of RPM-based Linux systems, it's a binary RPM; for Windows users, it's an executable installer; for Debian-based Linux users, it's a Debian package; and so forth. Obviously, no one person will be able to create built distributions for every platform under the sun, so the Distutils are designed to enable module developers to concentrate on their specialty---writing code and creating source distributions---while an intermediary species called \emph{packagers} springs up to turn source distributions into built distributions for as many platforms as there are packagers. Of course, the module developer could be his own packager; or the packager could be a volunteer ``out there'' somewhere who has access to a platform which the original developer does not; or it could be software periodically grabbing new source distributions and turning them into built distributions for as many platforms as the software has access to. Regardless of who they are, a packager uses the setup script and the \command{bdist} command family to generate built distributions. As a simple example, if I run the following command in the Distutils source tree: \begin{verbatim} python setup.py bdist \end{verbatim} then the Distutils builds my module distribution (the Distutils itself in this case), does a ``fake'' installation (also in the \file{build} directory), and creates the default type of built distribution for my platform. The default format for built distributions is a ``dumb'' tar file on \UNIX, and a simple executable installer on Windows. (That tar file is considered ``dumb'' because it has to be unpacked in a specific location to work.) Thus, the above command on a \UNIX{} system creates \file{Distutils-1.0.\filevar{plat}.tar.gz}; unpacking this tarball from the right place installs the Distutils just as though you had downloaded the source distribution and run \code{python setup.py install}. (The ``right place'' is either the root of the filesystem or Python's \filevar{prefix} directory, depending on the options given to the \command{bdist\_dumb} command; the default is to make dumb distributions relative to \filevar{prefix}.) Obviously, for pure Python distributions, this isn't any simpler than just running \code{python setup.py install}---but for non-pure distributions, which include extensions that would need to be compiled, it can mean the difference between someone being able to use your extensions or not. And creating ``smart'' built distributions, such as an RPM package or an executable installer for Windows, is far more convenient for users even if your distribution doesn't include any extensions. The \command{bdist} command has a \longprogramopt{formats} option, similar to the \command{sdist} command, which you can use to select the types of built distribution to generate: for example, \begin{verbatim} python setup.py bdist --format=zip \end{verbatim} would, when run on a \UNIX{} system, create \file{Distutils-1.0.\filevar{plat}.zip}---again, this archive would be unpacked from the root directory to install the Distutils. The available formats for built distributions are: \begin{tableiii}{l|l|c}{code}% {Format}{Description}{Notes} \lineiii{gztar}{gzipped tar file (\file{.tar.gz})}{(1),(3)} \lineiii{ztar}{compressed tar file (\file{.tar.Z})}{(3)} \lineiii{tar}{tar file (\file{.tar})}{(3)} \lineiii{zip}{zip file (\file{.zip})}{(4)} \lineiii{rpm}{RPM}{(5)} \lineiii{pkgtool}{Solaris \program{pkgtool}}{} \lineiii{sdux}{HP-UX \program{swinstall}}{} \lineiii{rpm}{RPM}{(5)} % \lineiii{srpm}{source RPM}{(5) \XXX{to do!}} \lineiii{wininst}{self-extracting ZIP file for Windows}{(2),(4)} \end{tableiii} \noindent Notes: \begin{description} \item[(1)] default on \UNIX \item[(2)] default on Windows \XXX{to-do!} \item[(3)] requires external utilities: \program{tar} and possibly one of \program{gzip}, \program{bzip2}, or \program{compress} \item[(4)] requires either external \program{zip} utility or \module{zipfile} module (part of the standard Python library since Python~1.6) \item[(5)] requires external \program{rpm} utility, version 3.0.4 or better (use \code{rpm --version} to find out which version you have) \end{description} You don't have to use the \command{bdist} command with the \longprogramopt{formats} option; you can also use the command that directly implements the format you're interested in. Some of these \command{bdist} ``sub-commands'' actually generate several similar formats; for instance, the \command{bdist\_dumb} command generates all the ``dumb'' archive formats (\code{tar}, \code{ztar}, \code{gztar}, and \code{zip}), and \command{bdist\_rpm} generates both binary and source RPMs. The \command{bdist} sub-commands, and the formats generated by each, are: \begin{tableii}{l|l}{command}% {Command}{Formats} \lineii{bdist\_dumb}{tar, ztar, gztar, zip} \lineii{bdist\_rpm}{rpm, srpm} \lineii{bdist\_wininst}{wininst} \end{tableii} The following sections give details on the individual \command{bdist\_*} commands. \subsection{Creating dumb built distributions} \label{creating-dumb} \XXX{Need to document absolute vs. prefix-relative packages here, but first I have to implement it!} \subsection{Creating RPM packages} \label{creating-rpms} The RPM format is used by many popular Linux distributions, including Red Hat, SuSE, and Mandrake. If one of these (or any of the other RPM-based Linux distributions) is your usual environment, creating RPM packages for other users of that same distribution is trivial. Depending on the complexity of your module distribution and differences between Linux distributions, you may also be able to create RPMs that work on different RPM-based distributions. The usual way to create an RPM of your module distribution is to run the \command{bdist\_rpm} command: \begin{verbatim} python setup.py bdist_rpm \end{verbatim} or the \command{bdist} command with the \longprogramopt{format} option: \begin{verbatim} python setup.py bdist --formats=rpm \end{verbatim} The former allows you to specify RPM-specific options; the latter allows you to easily specify multiple formats in one run. If you need to do both, you can explicitly specify multiple \command{bdist\_*} commands and their options: \begin{verbatim} python setup.py bdist_rpm --packager="John Doe " \ bdist_wininst --target_version="2.0" \end{verbatim} Creating RPM packages is driven by a \file{.spec} file, much as using the Distutils is driven by the setup script. To make your life easier, the \command{bdist\_rpm} command normally creates a \file{.spec} file based on the information you supply in the setup script, on the command line, and in any Distutils configuration files. Various options and sections in the \file{.spec} file are derived from options in the setup script as follows: \begin{tableii}{l|l}{textrm}% {RPM \file{.spec} file option or section}{Distutils setup script option} \lineii{Name}{\option{name}} \lineii{Summary (in preamble)}{\option{description}} \lineii{Version}{\option{version}} \lineii{Vendor}{\option{author} and \option{author\_email}, or \\& \option{maintainer} and \option{maintainer\_email}} \lineii{Copyright}{\option{licence}} \lineii{Url}{\option{url}} \lineii{\%description (section)}{\option{long\_description}} \end{tableii} Additionally, there many options in \file{.spec} files that don't have corresponding options in the setup script. Most of these are handled through options to the \command{bdist\_rpm} command as follows: \begin{tableiii}{l|l|l}{textrm}% {RPM \file{.spec} file option or section}% {\command{bdist\_rpm} option}% {default value} \lineiii{Release}{\option{release}}{``1''} \lineiii{Group}{\option{group}}{``Development/Libraries''} \lineiii{Vendor}{\option{vendor}}{(see above)} \lineiii{Packager}{\option{packager}}{(none)} \lineiii{Provides}{\option{provides}}{(none)} \lineiii{Requires}{\option{requires}}{(none)} \lineiii{Conflicts}{\option{conflicts}}{(none)} \lineiii{Obsoletes}{\option{obsoletes}}{(none)} \lineiii{Distribution}{\option{distribution\_name}}{(none)} \lineiii{BuildRequires}{\option{build\_requires}}{(none)} \lineiii{Icon}{\option{icon}}{(none)} \end{tableiii} Obviously, supplying even a few of these options on the command-line would be tedious and error-prone, so it's usually best to put them in the setup configuration file, \file{setup.cfg}---see section~\ref{setup-config}. If you distribute or package many Python module distributions, you might want to put options that apply to all of them in your personal Distutils configuration file (\file{\textasciitilde/.pydistutils.cfg}). There are three steps to building a binary RPM package, all of which are handled automatically by the Distutils: \begin{enumerate} \item create a \file{.spec} file, which describes the package (analogous to the Distutils setup script; in fact, much of the information in the setup script winds up in the \file{.spec} file) \item create the source RPM \item create the ``binary'' RPM (which may or may not contain binary code, depending on whether your module distribution contains Python extensions) \end{enumerate} Normally, RPM bundles the last two steps together; when you use the Distutils, all three steps are typically bundled together. If you wish, you can separate these three steps. You can use the \longprogramopt{spec-only} option to make \command{bdist\_rpm} just create the \file{.spec} file and exit; in this case, the \file{.spec} file will be written to the ``distribution directory''---normally \file{dist/}, but customizable with the \longprogramopt{dist-dir} option. (Normally, the \file{.spec} file winds up deep in the ``build tree,'' in a temporary directory created by \command{bdist\_rpm}.) \XXX{this isn't implemented yet---is it needed?!} You can also specify a custom \file{.spec} file with the \longprogramopt{spec-file} option; used in conjunction with \longprogramopt{spec-only}, this gives you an opportunity to customize the \file{.spec} file manually: \begin{verbatim} > python setup.py bdist_rpm --spec-only # ...edit dist/FooBar-1.0.spec > python setup.py bdist_rpm --spec-file=dist/FooBar-1.0.spec \end{verbatim} (Although a better way to do this is probably to override the standard \command{bdist\_rpm} command with one that writes whatever else you want to the \file{.spec} file; see section~\ref{extending} for information on extending the Distutils.) \subsection{Creating Windows Installers} \label{creating-wininst} Executable installers are the natural format for binary distributions on Windows. They display a nice graphical user interface, display some information about the module distribution to be installed taken from the metadata in the setup script, let the user select a few options, and start or cancel the installation. Since the metadata is taken from the setup script, creating Windows installers is usually as easy as running: \begin{verbatim} python setup.py bdist_wininst \end{verbatim} or the \command{bdist} command with the \longprogramopt{formats} option: \begin{verbatim} python setup.py bdist --formats=wininst \end{verbatim} If you have a pure module distribution (only containing pure Python modules and packages), the resulting installer will be version independent and have a name like \file{foo-1.0.win32.exe}. These installers can even be created on \UNIX{} or MacOS platforms. If you have a non-pure distribution, the extensions can only be created on a Windows platform, and will be Python version dependent. The installer filename will reflect this and now has the form \file{foo-1.0.win32-py2.0.exe}. You have to create a separate installer for every Python version you want to support. The installer will try to compile pure modules into bytecode after installation on the target system in normal and optimizing mode. If you don't want this to happen for some reason, you can run the \command{bdist_wininst} command with the \longprogramopt{no-target-compile} and/or the \longprogramopt{no-target-optimize} option. By default the installer will display the cool ``Python Powered'' logo when it is run, but you can also supply your own bitmap which must be a Windows \file{.bmp} file with the \longprogramopt{bitmap} option. The installer will also display a large title on the desktop background window when it is run, which is constructed from the name of your distribution and the version number. This can be changed to another text by using the \longprogramopt{title} option. The installer file will be written to the ``distribution directory'' --- normally \file{dist/}, but customizable with the \longprogramopt{dist-dir} option. \subsubsection{The Postinstallation script} \label{postinstallation-script} Starting with Python 2.3, a postinstallation script can be specified which the \longprogramopt{install-script} option. The basename of the script must be specified, and the script filename must also be listed in the scripts argument to the setup function. This script will be run at installation time on the target system after all the files have been copied, with argv[1] set to '-install', and again at uninstallation time before the files are removed with argv[1] set to '-remove'. The installation script runs embedded in the windows installer, every output (sys.stdout, sys.stderr) is redirected into a buffer and will be displayed in the GUI after the script has finished. Some functions especially useful in this context are available in the installation script. \begin{verbatim} dir_created(pathname) file_created(pathname) \end{verbatim} These functions should be called when a directory or file is created by the postinstall script at installation time. It will register the pathname with the uninstaller, so that it will be removed when the distribution is uninstalled. To be safe, directories are only removed if they are empty. \begin{verbatim} get_special_folder_path(csidl_string) \end{verbatim} This function can be used to retrieve special folder locations on Windows like the Start Menu or the Desktop. It returns the full path to the folder. 'csidl_string' must be on of the following strings: \begin{verbatim} "CSIDL_APPDATA" "CSIDL_COMMON_STARTMENU" "CSIDL_STARTMENU" "CSIDL_COMMON_DESKTOPDIRECTORY" "CSIDL_DESKTOPDIRECTORY" "CSIDL_COMMON_STARTUP" "CSIDL_STARTUP" "CSIDL_COMMON_PROGRAMS" "CSIDL_PROGRAMS" "CSIDL_FONTS" \end{verbatim} If the folder cannot be retrieved, OSError is raised. Which folders are available depends on the exact Windows version, and probably also the configuration. For details refer to Microsoft's documentation of the \code{SHGetSpecialFolderPath} function. \begin{verbatim} create_shortcut(target, description, filename[, arguments[, workdir[, iconpath[, iconindex]]]]) \end{verbatim} This function creates a shortcut. \var{target} is the path to the program to be started by the shortcut. \var{description} is the description of the sortcut. \var{filename} is the title of the shortcut that the user will see. \var{arguments} specifies the command line arguments, if any. \var{workdir} is the working directory for the program. \var{iconpath} is the file containing the icon for the shortcut, and \var{iconindex} is the index of the icon in the file \var{iconpath}. Again, for details consult the Microsoft documentation for the \code{IShellLink} interface. \section{Registering with the Package Index} \label{package-index} The Python Package Index (PyPI) holds meta-data describing distributions packaged with distutils. The distutils command \command{register} is used to submit your distribution's meta-data to the index. It is invoked as follows: \begin{verbatim} python setup.py register \end{verbatim} Distutils will respond with the following prompt: \begin{verbatim} running register We need to know who you are, so please choose either: 1. use your existing login, 2. register as a new user, 3. have the server generate a new password for you (and email it to you), or 4. quit Your selection [default 1]: \end{verbatim} \noindent Note: if your username and password are saved locally, you will not see this menu. If you have not registered with PyPI, then you will need to do so now. You should choose option 2, and enter your details as required. Soon after submitting your details, you will receive an email which will be used to confirm your registration. Once you are registered, you may choose option 1 from the menu. You will be prompted for your PyPI username and password, and \command{register} will then submit your meta-data to the index. You may submit any number of versions of your distribution to the index. If you alter the meta-data for a particular version, you may submit it again and the index will be updated. PyPI holds a record for each (name, version) combination submitted. The first user to submit information for a given name is designated the Owner of that name. They may submit changes through the \command{register} command or through the web interface. They may also designate other users as Owners or Maintainers. Maintainers may edit the package information, but not designate other Owners or Maintainers. By default PyPI will list all versions of a given package. To hide certain versions, the Hidden property should be set to yes. This must be edited through the web interface. \section{Examples} \label{examples} \subsection{Pure Python distribution (by module)} \label{pure-mod} If you're just distributing a couple of modules, especially if they don't live in a particular package, you can specify them individually using the \option{py\_modules} option in the setup script. In the simplest case, you'll have two files to worry about: a setup script and the single module you're distributing, \file{foo.py} in this example: \begin{verbatim} / setup.py foo.py \end{verbatim} (In all diagrams in this section, \verb|| will refer to the distribution root directory.) A minimal setup script to describe this situation would be: \begin{verbatim} from distutils.core import setup setup(name = "foo", version = "1.0", py_modules = ["foo"]) \end{verbatim} Note that the name of the distribution is specified independently with the \option{name} option, and there's no rule that says it has to be the same as the name of the sole module in the distribution (although that's probably a good convention to follow). However, the distribution name is used to generate filenames, so you should stick to letters, digits, underscores, and hyphens. Since \option{py\_modules} is a list, you can of course specify multiple modules, eg. if you're distributing modules \module{foo} and \module{bar}, your setup might look like this: \begin{verbatim} / setup.py foo.py bar.py \end{verbatim} and the setup script might be \begin{verbatim} from distutils.core import setup setup(name = "foobar", version = "1.0", py_modules = ["foo", "bar"]) \end{verbatim} You can put module source files into another directory, but if you have enough modules to do that, it's probably easier to specify modules by package rather than listing them individually. \subsection{Pure Python distribution (by package)} \label{pure-pkg} If you have more than a couple of modules to distribute, especially if they are in multiple packages, it's probably easier to specify whole packages rather than individual modules. This works even if your modules are not in a package; you can just tell the Distutils to process modules from the root package, and that works the same as any other package (except that you don't have to have an \file{\_\_init\_\_.py} file). The setup script from the last example could also be written as \begin{verbatim} from distutils.core import setup setup(name = "foobar", version = "1.0", packages = [""]) \end{verbatim} (The empty string stands for the root package.) If those two files are moved into a subdirectory, but remain in the root package, e.g.: \begin{verbatim} / setup.py src/ foo.py bar.py \end{verbatim} then you would still specify the root package, but you have to tell the Distutils where source files in the root package live: \begin{verbatim} from distutils.core import setup setup(name = "foobar", version = "1.0", package_dir = {"": "src"}, packages = [""]) \end{verbatim} More typically, though, you will want to distribute multiple modules in the same package (or in sub-packages). For example, if the \module{foo} and \module{bar} modules belong in package \module{foobar}, one way to layout your source tree is \begin{verbatim} / setup.py foobar/ __init__.py foo.py bar.py \end{verbatim} This is in fact the default layout expected by the Distutils, and the one that requires the least work to describe in your setup script: \begin{verbatim} from distutils.core import setup setup(name = "foobar", version = "1.0", packages = ["foobar"]) \end{verbatim} If you want to put modules in directories not named for their package, then you need to use the \option{package\_dir} option again. For example, if the \file{src} directory holds modules in the \module{foobar} package: \begin{verbatim} / setup.py src/ __init__.py foo.py bar.py \end{verbatim} an appropriate setup script would be \begin{verbatim} from distutils.core import setup setup(name = "foobar", version = "1.0", package_dir = {"foobar" : "src"}, packages = ["foobar"]) \end{verbatim} Or, you might put modules from your main package right in the distribution root: \begin{verbatim} / setup.py __init__.py foo.py bar.py \end{verbatim} in which case your setup script would be \begin{verbatim} from distutils.core import setup setup(name = "foobar", version = "1.0", package_dir = {"foobar" : ""}, packages = ["foobar"]) \end{verbatim} (The empty string also stands for the current directory.) If you have sub-packages, they must be explicitly listed in \option{packages}, but any entries in \option{package\_dir} automatically extend to sub-packages. (In other words, the Distutils does \emph{not} scan your source tree, trying to figure out which directories correspond to Python packages by looking for \file{\_\_init\_\_.py} files.) Thus, if the default layout grows a sub-package: \begin{verbatim} / setup.py foobar/ __init__.py foo.py bar.py subfoo/ __init__.py blah.py \end{verbatim} then the corresponding setup script would be \begin{verbatim} from distutils.core import setup setup(name = "foobar", version = "1.0", packages = ["foobar", "foobar.subfoo"]) \end{verbatim} (Again, the empty string in \option{package\_dir} stands for the current directory.) \subsection{Single extension module} \label{single-ext} Extension modules are specified using the \option{ext\_modules} option. \option{package\_dir} has no effect on where extension source files are found; it only affects the source for pure Python modules. The simplest case, a single extension module in a single C source file, is: \begin{verbatim} / setup.py foo.c \end{verbatim} If the \module{foo} extension belongs in the root package, the setup script for this could be \begin{verbatim} from distutils.core import setup setup(name = "foobar", version = "1.0", ext_modules = [Extension("foo", ["foo.c"])]) \end{verbatim} If the extension actually belongs in a package, say \module{foopkg}, then With exactly the same source tree layout, this extension can be put in the \module{foopkg} package simply by changing the name of the extension: \begin{verbatim} from distutils.core import setup setup(name = "foobar", version = "1.0", ext_modules = [Extension("foopkg.foo", ["foo.c"])]) \end{verbatim} %\subsection{Multiple extension modules} %\label{multiple-ext} %\subsection{Putting it all together} %\section{Extending the Distutils} %\label{extending} %\subsection{Extending existing commands} %\label{extend-existing} %\subsection{Writing new commands} %\label{new-commands} %\XXX{Would an uninstall command be a good example here?} \section{Reference} \label{reference} %\subsection{Building modules: the \protect\command{build} command family} %\label{build-cmds} %\subsubsection{\protect\command{build}} %\label{build-cmd} %\subsubsection{\protect\command{build\_py}} %\label{build-py-cmd} %\subsubsection{\protect\command{build\_ext}} %\label{build-ext-cmd} %\subsubsection{\protect\command{build\_clib}} %\label{build-clib-cmd} \subsection{Installing modules: the \protect\command{install} command family} \label{install-cmd} The install command ensures that the build commands have been run and then runs the subcommands \command{install\_lib}, \command{install\_data} and \command{install\_scripts}. %\subsubsection{\protect\command{install\_lib}} %\label{install-lib-cmd} \subsubsection{\protect\command{install\_data}} \label{install-data-cmd} This command installs all data files provided with the distribution. \subsubsection{\protect\command{install\_scripts}} \label{install-scripts-cmd} This command installs all (Python) scripts in the distribution. %\subsection{Cleaning up: the \protect\command{clean} command} %\label{clean-cmd} \subsection{Creating a source distribution: the \protect\command{sdist} command} \label{sdist-cmd} \XXX{fragment moved down from above: needs context!} The manifest template commands are: \begin{tableii}{ll}{command}{Command}{Description} \lineii{include \var{pat1} \var{pat2} ... } {include all files matching any of the listed patterns} \lineii{exclude \var{pat1} \var{pat2} ... } {exclude all files matching any of the listed patterns} \lineii{recursive-include \var{dir} \var{pat1} \var{pat2} ... } {include all files under \var{dir} matching any of the listed patterns} \lineii{recursive-exclude \var{dir} \var{pat1} \var{pat2} ...} {exclude all files under \var{dir} matching any of the listed patterns} \lineii{global-include \var{pat1} \var{pat2} ...} {include all files anywhere in the source tree matching\\& any of the listed patterns} \lineii{global-exclude \var{pat1} \var{pat2} ...} {exclude all files anywhere in the source tree matching\\& any of the listed patterns} \lineii{prune \var{dir}}{exclude all files under \var{dir}} \lineii{graft \var{dir}}{include all files under \var{dir}} \end{tableii} The patterns here are \UNIX-style ``glob'' patterns: \code{*} matches any sequence of regular filename characters, \code{?} matches any single regular filename character, and \code{[\var{range}]} matches any of the characters in \var{range} (e.g., \code{a-z}, \code{a-zA-Z}, \code{a-f0-9\_.}). The definition of ``regular filename character'' is platform-specific: on \UNIX{} it is anything except slash; on Windows anything except backslash or colon; on MacOS anything except colon. \XXX{Windows and MacOS support not there yet} %\subsection{Creating a built distribution: the % \protect\command{bdist} command family} %\label{bdist-cmds} %\subsubsection{\protect\command{bdist}} %\subsubsection{\protect\command{bdist\_dumb}} %\subsubsection{\protect\command{bdist\_rpm}} %\subsubsection{\protect\command{bdist\_wininst}} \input{sysconfig} \end{document}