mirror of https://github.com/python/cpython
552 lines
23 KiB
ReStructuredText
552 lines
23 KiB
ReStructuredText
.. _tut-modules:
|
|
|
|
*******
|
|
Modules
|
|
*******
|
|
|
|
If you quit from the Python interpreter and enter it again, the definitions you
|
|
have made (functions and variables) are lost. Therefore, if you want to write a
|
|
somewhat longer program, you are better off using a text editor to prepare the
|
|
input for the interpreter and running it with that file as input instead. This
|
|
is known as creating a *script*. As your program gets longer, you may want to
|
|
split it into several files for easier maintenance. You may also want to use a
|
|
handy function that you've written in several programs without copying its
|
|
definition into each program.
|
|
|
|
To support this, Python has a way to put definitions in a file and use them in a
|
|
script or in an interactive instance of the interpreter. Such a file is called a
|
|
*module*; definitions from a module can be *imported* into other modules or into
|
|
the *main* module (the collection of variables that you have access to in a
|
|
script executed at the top level and in calculator mode).
|
|
|
|
A module is a file containing Python definitions and statements. The file name
|
|
is the module name with the suffix :file:`.py` appended. Within a module, the
|
|
module's name (as a string) is available as the value of the global variable
|
|
``__name__``. For instance, use your favorite text editor to create a file
|
|
called :file:`fibo.py` in the current directory with the following contents::
|
|
|
|
# Fibonacci numbers module
|
|
|
|
def fib(n): # write Fibonacci series up to n
|
|
a, b = 0, 1
|
|
while b < n:
|
|
print b,
|
|
a, b = b, a+b
|
|
|
|
def fib2(n): # return Fibonacci series up to n
|
|
result = []
|
|
a, b = 0, 1
|
|
while b < n:
|
|
result.append(b)
|
|
a, b = b, a+b
|
|
return result
|
|
|
|
Now enter the Python interpreter and import this module with the following
|
|
command::
|
|
|
|
>>> import fibo
|
|
|
|
This does not enter the names of the functions defined in ``fibo`` directly in
|
|
the current symbol table; it only enters the module name ``fibo`` there. Using
|
|
the module name you can access the functions::
|
|
|
|
>>> fibo.fib(1000)
|
|
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
|
|
>>> fibo.fib2(100)
|
|
[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
|
|
>>> fibo.__name__
|
|
'fibo'
|
|
|
|
If you intend to use a function often you can assign it to a local name::
|
|
|
|
>>> fib = fibo.fib
|
|
>>> fib(500)
|
|
1 1 2 3 5 8 13 21 34 55 89 144 233 377
|
|
|
|
|
|
.. _tut-moremodules:
|
|
|
|
More on Modules
|
|
===============
|
|
|
|
A module can contain executable statements as well as function definitions.
|
|
These statements are intended to initialize the module. They are executed only
|
|
the *first* time the module is imported somewhere. [#]_
|
|
|
|
Each module has its own private symbol table, which is used as the global symbol
|
|
table by all functions defined in the module. Thus, the author of a module can
|
|
use global variables in the module without worrying about accidental clashes
|
|
with a user's global variables. On the other hand, if you know what you are
|
|
doing you can touch a module's global variables with the same notation used to
|
|
refer to its functions, ``modname.itemname``.
|
|
|
|
Modules can import other modules. It is customary but not required to place all
|
|
:keyword:`import` statements at the beginning of a module (or script, for that
|
|
matter). The imported module names are placed in the importing module's global
|
|
symbol table.
|
|
|
|
There is a variant of the :keyword:`import` statement that imports names from a
|
|
module directly into the importing module's symbol table. For example::
|
|
|
|
>>> from fibo import fib, fib2
|
|
>>> fib(500)
|
|
1 1 2 3 5 8 13 21 34 55 89 144 233 377
|
|
|
|
This does not introduce the module name from which the imports are taken in the
|
|
local symbol table (so in the example, ``fibo`` is not defined).
|
|
|
|
There is even a variant to import all names that a module defines::
|
|
|
|
>>> from fibo import *
|
|
>>> fib(500)
|
|
1 1 2 3 5 8 13 21 34 55 89 144 233 377
|
|
|
|
This imports all names except those beginning with an underscore (``_``).
|
|
|
|
|
|
.. _tut-modulesasscripts:
|
|
|
|
Executing modules as scripts
|
|
----------------------------
|
|
|
|
When you run a Python module with ::
|
|
|
|
python fibo.py <arguments>
|
|
|
|
the code in the module will be executed, just as if you imported it, but with
|
|
the ``__name__`` set to ``"__main__"``. That means that by adding this code at
|
|
the end of your module::
|
|
|
|
if __name__ == "__main__":
|
|
import sys
|
|
fib(int(sys.argv[1]))
|
|
|
|
you can make the file usable as a script as well as an importable module,
|
|
because the code that parses the command line only runs if the module is
|
|
executed as the "main" file::
|
|
|
|
$ python fibo.py 50
|
|
1 1 2 3 5 8 13 21 34
|
|
|
|
If the module is imported, the code is not run::
|
|
|
|
>>> import fibo
|
|
>>>
|
|
|
|
This is often used either to provide a convenient user interface to a module, or
|
|
for testing purposes (running the module as a script executes a test suite).
|
|
|
|
|
|
.. _tut-searchpath:
|
|
|
|
The Module Search Path
|
|
----------------------
|
|
|
|
.. index:: triple: module; search; path
|
|
|
|
When a module named :mod:`spam` is imported, the interpreter searches for a file
|
|
named :file:`spam.py` in the current directory, and then in the list of
|
|
directories specified by the environment variable :envvar:`PYTHONPATH`. This
|
|
has the same syntax as the shell variable :envvar:`PATH`, that is, a list of
|
|
directory names. When :envvar:`PYTHONPATH` is not set, or when the file is not
|
|
found there, the search continues in an installation-dependent default path; on
|
|
Unix, this is usually :file:`.:/usr/local/lib/python`.
|
|
|
|
Actually, modules are searched in the list of directories given by the variable
|
|
``sys.path`` which is initialized from the directory containing the input script
|
|
(or the current directory), :envvar:`PYTHONPATH` and the installation- dependent
|
|
default. This allows Python programs that know what they're doing to modify or
|
|
replace the module search path. Note that because the directory containing the
|
|
script being run is on the search path, it is important that the script not have
|
|
the same name as a standard module, or Python will attempt to load the script as
|
|
a module when that module is imported. This will generally be an error. See
|
|
section :ref:`tut-standardmodules` for more information.
|
|
|
|
|
|
"Compiled" Python files
|
|
-----------------------
|
|
|
|
As an important speed-up of the start-up time for short programs that use a lot
|
|
of standard modules, if a file called :file:`spam.pyc` exists in the directory
|
|
where :file:`spam.py` is found, this is assumed to contain an
|
|
already-"byte-compiled" version of the module :mod:`spam`. The modification time
|
|
of the version of :file:`spam.py` used to create :file:`spam.pyc` is recorded in
|
|
:file:`spam.pyc`, and the :file:`.pyc` file is ignored if these don't match.
|
|
|
|
Normally, you don't need to do anything to create the :file:`spam.pyc` file.
|
|
Whenever :file:`spam.py` is successfully compiled, an attempt is made to write
|
|
the compiled version to :file:`spam.pyc`. It is not an error if this attempt
|
|
fails; if for any reason the file is not written completely, the resulting
|
|
:file:`spam.pyc` file will be recognized as invalid and thus ignored later. The
|
|
contents of the :file:`spam.pyc` file are platform independent, so a Python
|
|
module directory can be shared by machines of different architectures.
|
|
|
|
Some tips for experts:
|
|
|
|
* When the Python interpreter is invoked with the :option:`-O` flag, optimized
|
|
code is generated and stored in :file:`.pyo` files. The optimizer currently
|
|
doesn't help much; it only removes :keyword:`assert` statements. When
|
|
:option:`-O` is used, *all* bytecode is optimized; ``.pyc`` files are ignored
|
|
and ``.py`` files are compiled to optimized bytecode.
|
|
|
|
* Passing two :option:`-O` flags to the Python interpreter (:option:`-OO`) will
|
|
cause the bytecode compiler to perform optimizations that could in some rare
|
|
cases result in malfunctioning programs. Currently only ``__doc__`` strings are
|
|
removed from the bytecode, resulting in more compact :file:`.pyo` files. Since
|
|
some programs may rely on having these available, you should only use this
|
|
option if you know what you're doing.
|
|
|
|
* A program doesn't run any faster when it is read from a :file:`.pyc` or
|
|
:file:`.pyo` file than when it is read from a :file:`.py` file; the only thing
|
|
that's faster about :file:`.pyc` or :file:`.pyo` files is the speed with which
|
|
they are loaded.
|
|
|
|
* When a script is run by giving its name on the command line, the bytecode for
|
|
the script is never written to a :file:`.pyc` or :file:`.pyo` file. Thus, the
|
|
startup time of a script may be reduced by moving most of its code to a module
|
|
and having a small bootstrap script that imports that module. It is also
|
|
possible to name a :file:`.pyc` or :file:`.pyo` file directly on the command
|
|
line.
|
|
|
|
* It is possible to have a file called :file:`spam.pyc` (or :file:`spam.pyo`
|
|
when :option:`-O` is used) without a file :file:`spam.py` for the same module.
|
|
This can be used to distribute a library of Python code in a form that is
|
|
moderately hard to reverse engineer.
|
|
|
|
.. index:: module: compileall
|
|
|
|
* The module :mod:`compileall` can create :file:`.pyc` files (or :file:`.pyo`
|
|
files when :option:`-O` is used) for all modules in a directory.
|
|
|
|
.. %
|
|
|
|
|
|
.. _tut-standardmodules:
|
|
|
|
Standard Modules
|
|
================
|
|
|
|
.. index:: module: sys
|
|
|
|
Python comes with a library of standard modules, described in a separate
|
|
document, the Python Library Reference ("Library Reference" hereafter). Some
|
|
modules are built into the interpreter; these provide access to operations that
|
|
are not part of the core of the language but are nevertheless built in, either
|
|
for efficiency or to provide access to operating system primitives such as
|
|
system calls. The set of such modules is a configuration option which also
|
|
depends on the underlying platform For example, the :mod:`winreg` module is only
|
|
provided on Windows systems. One particular module deserves some attention:
|
|
:mod:`sys`, which is built into every Python interpreter. The variables
|
|
``sys.ps1`` and ``sys.ps2`` define the strings used as primary and secondary
|
|
prompts:
|
|
|
|
.. %
|
|
|
|
::
|
|
|
|
>>> import sys
|
|
>>> sys.ps1
|
|
'>>> '
|
|
>>> sys.ps2
|
|
'... '
|
|
>>> sys.ps1 = 'C> '
|
|
C> print 'Yuck!'
|
|
Yuck!
|
|
C>
|
|
|
|
|
|
These two variables are only defined if the interpreter is in interactive mode.
|
|
|
|
The variable ``sys.path`` is a list of strings that determines the interpreter's
|
|
search path for modules. It is initialized to a default path taken from the
|
|
environment variable :envvar:`PYTHONPATH`, or from a built-in default if
|
|
:envvar:`PYTHONPATH` is not set. You can modify it using standard list
|
|
operations::
|
|
|
|
>>> import sys
|
|
>>> sys.path.append('/ufs/guido/lib/python')
|
|
|
|
|
|
.. _tut-dir:
|
|
|
|
The :func:`dir` Function
|
|
========================
|
|
|
|
The built-in function :func:`dir` is used to find out which names a module
|
|
defines. It returns a sorted list of strings::
|
|
|
|
>>> import fibo, sys
|
|
>>> dir(fibo)
|
|
['__name__', 'fib', 'fib2']
|
|
>>> dir(sys)
|
|
['__displayhook__', '__doc__', '__excepthook__', '__name__', '__stderr__',
|
|
'__stdin__', '__stdout__', '_getframe', 'api_version', 'argv',
|
|
'builtin_module_names', 'byteorder', 'callstats', 'copyright',
|
|
'displayhook', 'exc_info', 'excepthook',
|
|
'exec_prefix', 'executable', 'exit', 'getdefaultencoding', 'getdlopenflags',
|
|
'getrecursionlimit', 'getrefcount', 'hexversion', 'maxint', 'maxunicode',
|
|
'meta_path', 'modules', 'path', 'path_hooks', 'path_importer_cache',
|
|
'platform', 'prefix', 'ps1', 'ps2', 'setcheckinterval', 'setdlopenflags',
|
|
'setprofile', 'setrecursionlimit', 'settrace', 'stderr', 'stdin', 'stdout',
|
|
'version', 'version_info', 'warnoptions']
|
|
|
|
Without arguments, :func:`dir` lists the names you have defined currently::
|
|
|
|
>>> a = [1, 2, 3, 4, 5]
|
|
>>> import fibo
|
|
>>> fib = fibo.fib
|
|
>>> dir()
|
|
['__builtins__', '__doc__', '__file__', '__name__', 'a', 'fib', 'fibo', 'sys']
|
|
|
|
Note that it lists all types of names: variables, modules, functions, etc.
|
|
|
|
.. index:: module: __builtin__
|
|
|
|
:func:`dir` does not list the names of built-in functions and variables. If you
|
|
want a list of those, they are defined in the standard module
|
|
:mod:`__builtin__`::
|
|
|
|
>>> import __builtin__
|
|
>>> dir(__builtin__)
|
|
['ArithmeticError', 'AssertionError', 'AttributeError', 'DeprecationWarning',
|
|
'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False',
|
|
'FloatingPointError', 'FutureWarning', 'IOError', 'ImportError',
|
|
'IndentationError', 'IndexError', 'KeyError', 'KeyboardInterrupt',
|
|
'LookupError', 'MemoryError', 'NameError', 'None', 'NotImplemented',
|
|
'NotImplementedError', 'OSError', 'OverflowError',
|
|
'PendingDeprecationWarning', 'ReferenceError', 'RuntimeError',
|
|
'RuntimeWarning', 'StopIteration', 'SyntaxError',
|
|
'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'True',
|
|
'TypeError', 'UnboundLocalError', 'UnicodeDecodeError',
|
|
'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError',
|
|
'UserWarning', 'ValueError', 'Warning', 'WindowsError',
|
|
'ZeroDivisionError', '_', '__debug__', '__doc__', '__import__',
|
|
'__name__', 'abs', 'basestring', 'bool', 'buffer',
|
|
'chr', 'classmethod', 'cmp', 'compile',
|
|
'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod',
|
|
'enumerate', 'eval', 'exec', 'exit', 'filter', 'float',
|
|
'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex',
|
|
'id', 'input', 'int', 'isinstance', 'issubclass', 'iter',
|
|
'len', 'license', 'list', 'locals', 'map', 'max', 'min',
|
|
'object', 'oct', 'open', 'ord', 'pow', 'property', 'quit', 'range',
|
|
'repr', 'reversed', 'round', 'set',
|
|
'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super',
|
|
'tuple', 'type', 'vars', 'zip']
|
|
|
|
|
|
.. _tut-packages:
|
|
|
|
Packages
|
|
========
|
|
|
|
Packages are a way of structuring Python's module namespace by using "dotted
|
|
module names". For example, the module name :mod:`A.B` designates a submodule
|
|
named ``B`` in a package named ``A``. Just like the use of modules saves the
|
|
authors of different modules from having to worry about each other's global
|
|
variable names, the use of dotted module names saves the authors of multi-module
|
|
packages like NumPy or the Python Imaging Library from having to worry about
|
|
each other's module names.
|
|
|
|
Suppose you want to design a collection of modules (a "package") for the uniform
|
|
handling of sound files and sound data. There are many different sound file
|
|
formats (usually recognized by their extension, for example: :file:`.wav`,
|
|
:file:`.aiff`, :file:`.au`), so you may need to create and maintain a growing
|
|
collection of modules for the conversion between the various file formats.
|
|
There are also many different operations you might want to perform on sound data
|
|
(such as mixing, adding echo, applying an equalizer function, creating an
|
|
artificial stereo effect), so in addition you will be writing a never-ending
|
|
stream of modules to perform these operations. Here's a possible structure for
|
|
your package (expressed in terms of a hierarchical filesystem)::
|
|
|
|
sound/ Top-level package
|
|
__init__.py Initialize the sound package
|
|
formats/ Subpackage for file format conversions
|
|
__init__.py
|
|
wavread.py
|
|
wavwrite.py
|
|
aiffread.py
|
|
aiffwrite.py
|
|
auread.py
|
|
auwrite.py
|
|
...
|
|
effects/ Subpackage for sound effects
|
|
__init__.py
|
|
echo.py
|
|
surround.py
|
|
reverse.py
|
|
...
|
|
filters/ Subpackage for filters
|
|
__init__.py
|
|
equalizer.py
|
|
vocoder.py
|
|
karaoke.py
|
|
...
|
|
|
|
When importing the package, Python searches through the directories on
|
|
``sys.path`` looking for the package subdirectory.
|
|
|
|
The :file:`__init__.py` files are required to make Python treat the directories
|
|
as containing packages; this is done to prevent directories with a common name,
|
|
such as ``string``, from unintentionally hiding valid modules that occur later
|
|
on the module search path. In the simplest case, :file:`__init__.py` can just be
|
|
an empty file, but it can also execute initialization code for the package or
|
|
set the ``__all__`` variable, described later.
|
|
|
|
Users of the package can import individual modules from the package, for
|
|
example::
|
|
|
|
import sound.effects.echo
|
|
|
|
This loads the submodule :mod:`sound.effects.echo`. It must be referenced with
|
|
its full name. ::
|
|
|
|
sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)
|
|
|
|
An alternative way of importing the submodule is::
|
|
|
|
from sound.effects import echo
|
|
|
|
This also loads the submodule :mod:`echo`, and makes it available without its
|
|
package prefix, so it can be used as follows::
|
|
|
|
echo.echofilter(input, output, delay=0.7, atten=4)
|
|
|
|
Yet another variation is to import the desired function or variable directly::
|
|
|
|
from sound.effects.echo import echofilter
|
|
|
|
Again, this loads the submodule :mod:`echo`, but this makes its function
|
|
:func:`echofilter` directly available::
|
|
|
|
echofilter(input, output, delay=0.7, atten=4)
|
|
|
|
Note that when using ``from package import item``, the item can be either a
|
|
submodule (or subpackage) of the package, or some other name defined in the
|
|
package, like a function, class or variable. The ``import`` statement first
|
|
tests whether the item is defined in the package; if not, it assumes it is a
|
|
module and attempts to load it. If it fails to find it, an :exc:`ImportError`
|
|
exception is raised.
|
|
|
|
Contrarily, when using syntax like ``import item.subitem.subsubitem``, each item
|
|
except for the last must be a package; the last item can be a module or a
|
|
package but can't be a class or function or variable defined in the previous
|
|
item.
|
|
|
|
|
|
.. _tut-pkg-import-star:
|
|
|
|
Importing \* From a Package
|
|
---------------------------
|
|
|
|
.. index:: single: __all__
|
|
|
|
Now what happens when the user writes ``from sound.effects import *``? Ideally,
|
|
one would hope that this somehow goes out to the filesystem, finds which
|
|
submodules are present in the package, and imports them all. Unfortunately,
|
|
this operation does not work very well on Windows platforms, where the
|
|
filesystem does not always have accurate information about the case of a
|
|
filename! On these platforms, there is no guaranteed way to know whether a file
|
|
:file:`ECHO.PY` should be imported as a module :mod:`echo`, :mod:`Echo` or
|
|
:mod:`ECHO`. (For example, Windows 95 has the annoying practice of showing all
|
|
file names with a capitalized first letter.) The DOS 8+3 filename restriction
|
|
adds another interesting problem for long module names.
|
|
|
|
.. % The \code{__all__} Attribute
|
|
|
|
The only solution is for the package author to provide an explicit index of the
|
|
package. The import statement uses the following convention: if a package's
|
|
:file:`__init__.py` code defines a list named ``__all__``, it is taken to be the
|
|
list of module names that should be imported when ``from package import *`` is
|
|
encountered. It is up to the package author to keep this list up-to-date when a
|
|
new version of the package is released. Package authors may also decide not to
|
|
support it, if they don't see a use for importing \* from their package. For
|
|
example, the file :file:`sounds/effects/__init__.py` could contain the following
|
|
code::
|
|
|
|
__all__ = ["echo", "surround", "reverse"]
|
|
|
|
This would mean that ``from sound.effects import *`` would import the three
|
|
named submodules of the :mod:`sound` package.
|
|
|
|
If ``__all__`` is not defined, the statement ``from sound.effects import *``
|
|
does *not* import all submodules from the package :mod:`sound.effects` into the
|
|
current namespace; it only ensures that the package :mod:`sound.effects` has
|
|
been imported (possibly running any initialization code in :file:`__init__.py`)
|
|
and then imports whatever names are defined in the package. This includes any
|
|
names defined (and submodules explicitly loaded) by :file:`__init__.py`. It
|
|
also includes any submodules of the package that were explicitly loaded by
|
|
previous import statements. Consider this code::
|
|
|
|
import sound.effects.echo
|
|
import sound.effects.surround
|
|
from sound.effects import *
|
|
|
|
In this example, the echo and surround modules are imported in the current
|
|
namespace because they are defined in the :mod:`sound.effects` package when the
|
|
``from...import`` statement is executed. (This also works when ``__all__`` is
|
|
defined.)
|
|
|
|
Note that in general the practice of importing ``*`` from a module or package is
|
|
frowned upon, since it often causes poorly readable code. However, it is okay to
|
|
use it to save typing in interactive sessions, and certain modules are designed
|
|
to export only names that follow certain patterns.
|
|
|
|
Remember, there is nothing wrong with using ``from Package import
|
|
specific_submodule``! In fact, this is the recommended notation unless the
|
|
importing module needs to use submodules with the same name from different
|
|
packages.
|
|
|
|
|
|
Intra-package References
|
|
------------------------
|
|
|
|
The submodules often need to refer to each other. For example, the
|
|
:mod:`surround` module might use the :mod:`echo` module. In fact, such
|
|
references are so common that the :keyword:`import` statement first looks in the
|
|
containing package before looking in the standard module search path. Thus, the
|
|
:mod:`surround` module can simply use ``import echo`` or ``from echo import
|
|
echofilter``. If the imported module is not found in the current package (the
|
|
package of which the current module is a submodule), the :keyword:`import`
|
|
statement looks for a top-level module with the given name.
|
|
|
|
When packages are structured into subpackages (as with the :mod:`sound` package
|
|
in the example), you can use absolute imports to refer to submodules of siblings
|
|
packages. For example, if the module :mod:`sound.filters.vocoder` needs to use
|
|
the :mod:`echo` module in the :mod:`sound.effects` package, it can use ``from
|
|
sound.effects import echo``.
|
|
|
|
Starting with Python 2.5, in addition to the implicit relative imports described
|
|
above, you can write explicit relative imports with the ``from module import
|
|
name`` form of import statement. These explicit relative imports use leading
|
|
dots to indicate the current and parent packages involved in the relative
|
|
import. From the :mod:`surround` module for example, you might use::
|
|
|
|
from . import echo
|
|
from .. import formats
|
|
from ..filters import equalizer
|
|
|
|
Note that both explicit and implicit relative imports are based on the name of
|
|
the current module. Since the name of the main module is always ``"__main__"``,
|
|
modules intended for use as the main module of a Python application should
|
|
always use absolute imports.
|
|
|
|
|
|
Packages in Multiple Directories
|
|
--------------------------------
|
|
|
|
Packages support one more special attribute, :attr:`__path__`. This is
|
|
initialized to be a list containing the name of the directory holding the
|
|
package's :file:`__init__.py` before the code in that file is executed. This
|
|
variable can be modified; doing so affects future searches for modules and
|
|
subpackages contained in the package.
|
|
|
|
While this feature is not often needed, it can be used to extend the set of
|
|
modules found in a package.
|
|
|
|
|
|
.. rubric:: Footnotes
|
|
|
|
.. [#] In fact function definitions are also 'statements' that are 'executed'; the
|
|
execution enters the function name in the module's global symbol table.
|
|
|