566 lines
20 KiB
TeX
566 lines
20 KiB
TeX
\documentclass{howto}
|
|
\usepackage{distutils}
|
|
% $Id$
|
|
|
|
|
|
\title{What's New in Python 2.5}
|
|
\release{0.0}
|
|
\author{A.M. Kuchling}
|
|
\authoraddress{\email{amk@amk.ca}}
|
|
|
|
\begin{document}
|
|
\maketitle
|
|
\tableofcontents
|
|
|
|
This article explains the new features in Python 2.5. No release date
|
|
for Python 2.5 has been set; it will probably be released in the
|
|
autumn of 2006.
|
|
|
|
% Compare with previous release in 2 - 3 sentences here.
|
|
|
|
This article doesn't attempt to provide a complete specification of
|
|
the new features, but instead provides a convenient overview. For
|
|
full details, you should refer to the documentation for Python 2.5.
|
|
% add hyperlink when the documentation becomes available online.
|
|
If you want to understand the complete implementation and design
|
|
rationale, refer to the PEP for a particular new feature.
|
|
|
|
|
|
%======================================================================
|
|
\section{PEP 309: Partial Function Application}
|
|
|
|
The \module{functional} module is intended to contain tools for
|
|
functional-style programming. Currently it only contains
|
|
\class{partial}, but new functions will probably be added in future
|
|
versions of Python.
|
|
|
|
For programs written in a functional style, it can be useful to
|
|
construct variants of existing functions that have some of the
|
|
parameters filled in. Consider a Python function \code{f(a, b, c)};
|
|
you could create a new function \code{g(b, c)} that was equivalent to
|
|
\code{f(1, b, c)}. This is called ``partial function application'',
|
|
and is provided by the \class{partial} class in the new
|
|
\module{functional} module.
|
|
|
|
The constructor for \class{partial} takes the arguments
|
|
\code{(\var{function}, \var{arg1}, \var{arg2}, ...
|
|
\var{kwarg1}=\var{value1}, \var{kwarg2}=\var{value2})}. The resulting
|
|
object is callable, so you can just call it to invoke \var{function}
|
|
with the filled-in arguments.
|
|
|
|
Here's a small but realistic example:
|
|
|
|
\begin{verbatim}
|
|
import functional
|
|
|
|
def log (message, subsystem):
|
|
"Write the contents of 'message' to the specified subsystem."
|
|
print '%s: %s' % (subsystem, message)
|
|
...
|
|
|
|
server_log = functional.partial(log, subsystem='server')
|
|
\end{verbatim}
|
|
|
|
Here's another example, from a program that uses PyGTk. Here a
|
|
context-sensitive pop-up menu is being constructed dynamically. The
|
|
callback provided for the menu option is a partially applied version
|
|
of the \method{open_item()} method, where the first argument has been
|
|
provided.
|
|
|
|
\begin{verbatim}
|
|
...
|
|
class Application:
|
|
def open_item(self, path):
|
|
...
|
|
def init (self):
|
|
open_func = functional.partial(self.open_item, item_path)
|
|
popup_menu.append( ("Open", open_func, 1) )
|
|
\end{verbatim}
|
|
|
|
|
|
\begin{seealso}
|
|
|
|
\seepep{309}{Partial Function Application}{PEP proposed and written by
|
|
Peter Harris; implemented by Hye-Shik Chang, with adaptations by
|
|
Raymond Hettinger.}
|
|
|
|
\end{seealso}
|
|
|
|
|
|
%======================================================================
|
|
\section{PEP 314: Metadata for Python Software Packages v1.1}
|
|
|
|
Some simple dependency support was added to Distutils. The
|
|
\function{setup()} function now has \code{requires},\code{provides},
|
|
and \code{obsoletes}. When you build a source distribution using the
|
|
\code{sdist} command, the dependency information will be recorded in
|
|
the \file{PKG-INFO} file.
|
|
|
|
Another new keyword is \code{download_url}, which should be set to a
|
|
URL for the package's source code. This means it's now possible to
|
|
look up an entry in the package index, determine the dependencies for
|
|
a package, and download the required packages.
|
|
|
|
% XXX put example here
|
|
|
|
\begin{seealso}
|
|
|
|
\seepep{314}{Metadata for Python Software Packages v1.1}{PEP proposed
|
|
and written by A.M. Kuchling, Richard Jones, and Fred Drake;
|
|
implemented by Richard Jones and Fred Drake.}
|
|
|
|
\end{seealso}
|
|
|
|
|
|
%======================================================================
|
|
\section{PEP 342: New Generator Features}
|
|
|
|
As introduced in Python 2.3, generators only produce output; once a
|
|
generator's code was invoked to create an iterator, there's no way to
|
|
pass new parameters into the function when its execution is resumed.
|
|
Hackish solutions to this include making the generator's code look at
|
|
a global variable and then changing the global variable's value, or
|
|
passing in some mutable object that callers then modify. Python
|
|
2.5 adds the ability to pass values \emph{into} a generator.
|
|
|
|
To refresh your memory of basic generators, here's a simple example:
|
|
|
|
\begin{verbatim}
|
|
def counter (maximum):
|
|
i = 0
|
|
while i < maximum:
|
|
yield i
|
|
i += 1
|
|
\end{verbatim}
|
|
|
|
When you call \code{counter(10)}, the result is an iterator that
|
|
returns the values from 0 up to 9. On encountering the
|
|
\keyword{yield} statement, the iterator returns the provided value and
|
|
suspends the function's execution, preserving the local variables.
|
|
Execution resumes on the following call to the iterator's
|
|
\method{next()} method, picking up after the \keyword{yield}.
|
|
|
|
In Python 2.3, \keyword{yield} was a statement; it didn't return any
|
|
value. In 2.5, \keyword{yield} is now an expression, returning a
|
|
value that can be assigned to a variable or otherwise operated on:
|
|
|
|
\begin{verbatim}
|
|
val = (yield i)
|
|
\end{verbatim}
|
|
|
|
I recommend that you always put parentheses around a \keyword{yield}
|
|
expression when you're doing something with the returned value, as in
|
|
the above example. The parentheses aren't always necessary, but it's
|
|
easier to always add them instead of having to remember when they're
|
|
needed. The exact rules are that a \keyword{yield}-expression must
|
|
always be parenthesized except when it occurs at the top-level
|
|
expression on the right-hand side of an assignment, meaning
|
|
you can to write \code{val = yield i} but \code{val = (yield i) + 12}.
|
|
|
|
Values are sent into a generator by calling its
|
|
\method{send(\var{value})} method. The generator's code is then
|
|
resumed and the \keyword{yield} expression produces \var{value}.
|
|
If the regular \method{next()} method is called, the \keyword{yield}
|
|
returns \constant{None}.
|
|
|
|
Here's the previous example, modified to allow changing the value of
|
|
the internal counter.
|
|
|
|
\begin{verbatim}
|
|
def counter (maximum):
|
|
i = 0
|
|
while i < maximum:
|
|
val = (yield i)
|
|
# If value provided, change counter
|
|
if val is not None:
|
|
i = val
|
|
else:
|
|
i += 1
|
|
\end{verbatim}
|
|
|
|
And here's an example of changing the counter:
|
|
|
|
\begin{verbatim}
|
|
>>> it = counter(10)
|
|
>>> print it.next()
|
|
0
|
|
>>> print it.next()
|
|
1
|
|
>>> print it.send(8)
|
|
8
|
|
>>> print it.next()
|
|
9
|
|
>>> print it.next()
|
|
Traceback (most recent call last):
|
|
File ``t.py'', line 15, in ?
|
|
print it.next()
|
|
StopIteration
|
|
\end{verbatim}
|
|
|
|
Because \keyword{yield} will often be returning \constant{None},
|
|
you shouldn't just use its value in expressions unless you're sure
|
|
that only the \method{send()} method will be used.
|
|
|
|
There are two other new methods on generators in addition to
|
|
\method{send()}:
|
|
|
|
\begin{itemize}
|
|
|
|
\item \method{throw(\var{type}, \var{value}=None,
|
|
\var{traceback}=None)} is used to raise an exception inside the
|
|
generator; the exception is raised by the \keyword{yield} expression
|
|
where the generator's execution is paused.
|
|
|
|
\item \method{close()} raises a new \exception{GeneratorExit}
|
|
exception inside the generator to terminate the iteration.
|
|
On receiving this
|
|
exception, the generator's code must either raise
|
|
\exception{GeneratorExit} or \exception{StopIteration}; catching the
|
|
exception and doing anything else is illegal and will trigger
|
|
a \exception{RuntimeError}. \method{close()} will also be called by
|
|
Python's garbage collection when the generator is garbage-collected.
|
|
|
|
If you need to run cleanup code in case of a \exception{GeneratorExit},
|
|
I suggest using a \code{try: ... finally:} suite instead of
|
|
catching \exception{GeneratorExit}.
|
|
|
|
\end{itemize}
|
|
|
|
The cumulative effect of these changes is to turn generators from
|
|
one-way producers of information into both producers and consumers.
|
|
Generators also become \emph{coroutines}, a more generalized form of
|
|
subroutines; subroutines are entered at one point and exited at
|
|
another point (the top of the function, and a \keyword{return
|
|
statement}), but coroutines can be entered, exited, and resumed at
|
|
many different points (the \keyword{yield} statements).science term
|
|
|
|
|
|
\begin{seealso}
|
|
|
|
\seepep{342}{Coroutines via Enhanced Generators}{PEP written by
|
|
Guido van Rossum and Phillip J. Eby;
|
|
implemented by Phillip J. Eby. Includes examples of
|
|
some fancier uses of generators as coroutines.}
|
|
|
|
\seeurl{http://en.wikipedia.org/wiki/Coroutine}{The Wikipedia entry for
|
|
coroutines.}
|
|
|
|
\seeurl{http://www.sidhe.org/~dan/blog/archives/000178.html}{An
|
|
explanation of coroutines from a Perl point of view, written by Dan
|
|
Sugalski.}
|
|
|
|
\end{seealso}
|
|
|
|
|
|
%======================================================================
|
|
\section{Other Language Changes}
|
|
|
|
Here are all of the changes that Python 2.5 makes to the core Python
|
|
language.
|
|
|
|
\begin{itemize}
|
|
|
|
\item The \function{min()} and \function{max()} built-in functions
|
|
gained a \code{key} keyword argument analogous to the \code{key}
|
|
argument for \method{sort()}. This argument supplies a function
|
|
that takes a single argument and is called for every value in the list;
|
|
\function{min()}/\function{max()} will return the element with the
|
|
smallest/largest return value from this function.
|
|
For example, to find the longest string in a list, you can do:
|
|
|
|
\begin{verbatim}
|
|
L = ['medium', 'longest', 'short']
|
|
# Prints 'longest'
|
|
print max(L, key=len)
|
|
# Prints 'short', because lexicographically 'short' has the largest value
|
|
print max(L)
|
|
\end{verbatim}
|
|
|
|
(Contributed by Steven Bethard and Raymond Hettinger.)
|
|
|
|
\item Two new built-in functions, \function{any()} and
|
|
\function{all()}, evaluate whether an iterator contains any true or
|
|
false values. \function{any()} returns \constant{True} if any value
|
|
returned by the iterator is true; otherwise it will return
|
|
\constant{False}. \function{all()} returns \constant{True} only if
|
|
all of the values returned by the iterator evaluate as being true.
|
|
|
|
% XXX who added?
|
|
|
|
|
|
\item The list of base classes in a class definition can now be empty.
|
|
As an example, this is now legal:
|
|
|
|
\begin{verbatim}
|
|
class C():
|
|
pass
|
|
\end{verbatim}
|
|
(Implemented by Brett Cannon.)
|
|
|
|
\end{itemize}
|
|
|
|
|
|
%======================================================================
|
|
\subsection{Optimizations}
|
|
|
|
\begin{itemize}
|
|
|
|
\item When they were introduced
|
|
in Python 2.4, the built-in \class{set} and \class{frozenset} types
|
|
were built on top of Python's dictionary type.
|
|
In 2.5 the internal data structure has been customized for implementing sets,
|
|
and as a result sets will use a third less memory and are somewhat faster.
|
|
(Implemented by Raymond Hettinger.)
|
|
|
|
\end{itemize}
|
|
|
|
The net result of the 2.5 optimizations is that Python 2.5 runs the
|
|
pystone benchmark around XX\% faster than Python 2.4.
|
|
|
|
|
|
%======================================================================
|
|
\section{New, Improved, and Deprecated Modules}
|
|
|
|
As usual, Python's standard library received a number of enhancements and
|
|
bug fixes. Here's a partial list of the most notable changes, sorted
|
|
alphabetically by module name. Consult the
|
|
\file{Misc/NEWS} file in the source tree for a more
|
|
complete list of changes, or look through the CVS logs for all the
|
|
details.
|
|
|
|
\begin{itemize}
|
|
|
|
% collections.deque now has .remove()
|
|
|
|
% the cPickle module no longer accepts the deprecated None option in the
|
|
% args tuple returned by __reduce__().
|
|
|
|
% csv module improvements
|
|
|
|
% datetime.datetime() now has a strptime class method which can be used to
|
|
% create datetime object using a string and format.
|
|
|
|
\item A new \module{hashlib} module has been added to replace the
|
|
\module{md5} and \module{sha} modules. \module{hashlib} adds support
|
|
for additional secure hashes (SHA-224, SHA-256, SHA-384, and SHA-512).
|
|
When available, the module uses OpenSSL for fast platform optimized
|
|
implementations of algorithms. The old \module{md5} and \module{sha}
|
|
modules still exist as wrappers around hashlib to preserve backwards
|
|
compatibility. (Contributed by Gregory P. Smith.)
|
|
|
|
\item The \function{nsmallest()} and
|
|
\function{nlargest()} functions in the \module{heapq} module
|
|
now support a \code{key} keyword argument similar to the one
|
|
provided by the \function{min()}/\function{max()} functions
|
|
and the \method{sort()} methods. For example:
|
|
Example:
|
|
|
|
\begin{verbatim}
|
|
>>> import heapq
|
|
>>> L = ["short", 'medium', 'longest', 'longer still']
|
|
>>> heapq.nsmallest(2, L) # Return two lowest elements, lexicographically
|
|
['longer still', 'longest']
|
|
>>> heapq.nsmallest(2, L, key=len) # Return two shortest elements
|
|
['short', 'medium']
|
|
\end{verbatim}
|
|
|
|
(Contributed by Raymond Hettinger.)
|
|
|
|
\item The \function{itertools.islice()} function now accepts
|
|
\code{None} for the start and step arguments. This makes it more
|
|
compatible with the attributes of slice objects, so that you can now write
|
|
the following:
|
|
|
|
\begin{verbatim}
|
|
s = slice(5) # Create slice object
|
|
itertools.islice(iterable, s.start, s.stop, s.step)
|
|
\end{verbatim}
|
|
|
|
(Contributed by Raymond Hettinger.)
|
|
|
|
\item The \module{operator} module's \function{itemgetter()}
|
|
and \function{attrgetter()} functions now support multiple fields.
|
|
A call such as \code{operator.attrgetter('a', 'b')}
|
|
will return a function
|
|
that retrieves the \member{a} and \member{b} attributes. Combining
|
|
this new feature with the \method{sort()} method's \code{key} parameter
|
|
lets you easily sort lists using multiple fields.
|
|
|
|
% XXX who added?
|
|
|
|
|
|
\item The \module{os} module underwent a number of changes. The
|
|
\member{stat_float_times} variable now defaults to true, meaning that
|
|
\function{os.stat()} will now return time values as floats. (This
|
|
doesn't necessarily mean that \function{os.stat()} will return times
|
|
that are precise to fractions of a second; not all systems support
|
|
such precision.)
|
|
|
|
Constants named \member{os.SEEK_SET}, \member{os.SEEK_CUR}, and
|
|
\member{os.SEEK_END} have been added; these are the parameters to the
|
|
\function{os.lseek()} function. Two new constants for locking are
|
|
\member{os.O_SHLOCK} and \member{os.O_EXLOCK}.
|
|
|
|
On FreeBSD, the \function{os.stat()} function now returns
|
|
times with nanosecond resolution, and the returned object
|
|
now has \member{st_gen} and \member{st_birthtime}.
|
|
The \member{st_flags} member is also available, if the platform supports it.
|
|
% XXX patch 1180695, 1212117
|
|
|
|
\item The \module{socket} module now supports \constant{AF_NETLINK}
|
|
sockets on Linux, thanks to a patch from Philippe Biondi.
|
|
Netlink sockets are a Linux-specific mechanism for communications
|
|
between a user-space process and kernel code; an introductory
|
|
article about them is at \url{http://www.linuxjournal.com/article/7356}.
|
|
In Python code, netlink addresses are represented as a tuple of 2 integers,
|
|
\code{(\var{pid}, \var{group_mask})}.
|
|
|
|
\item New module: \module{spwd} provides functions for accessing the
|
|
shadow password database on systems that support it.
|
|
% XXX give example
|
|
|
|
\item The \class{TarFile} class in the \module{tarfile} module now has
|
|
an \method{extractall()} method that extracts all members from the
|
|
archive into the current working directory. It's also possible to set
|
|
a different directory as the extraction target, and to unpack only a
|
|
subset of the archive's members.
|
|
|
|
A tarfile's compression can be autodetected by
|
|
using the mode \code{'r|*'}.
|
|
% patch 918101
|
|
(Contributed by Lars Gust\"abel.)
|
|
|
|
\item A new package \module{xml.etree} has been added, which contains
|
|
a subset of the ElementTree XML library. Available modules are
|
|
\module{ElementTree}, \module{ElementPath}, and
|
|
\module{ElementInclude}, from ElementTree 1.2.6. (Contributed by
|
|
Fredrik Lundh.)
|
|
|
|
\item The \module{xmlrpclib} module now supports returning
|
|
\class{datetime} objects for the XML-RPC date type. Supply
|
|
\code{use_datetime=True} to the \function{loads()} function
|
|
or the \class{Unmarshaller} class to enable this feature.
|
|
% XXX patch 1120353
|
|
|
|
|
|
\end{itemize}
|
|
|
|
|
|
|
|
%======================================================================
|
|
% whole new modules get described in \subsections here
|
|
|
|
% XXX new distutils features: upload
|
|
|
|
% XXX should hashlib perhaps be described here instead?
|
|
% XXX should xml.etree perhaps be described here instead?
|
|
|
|
|
|
|
|
% ======================================================================
|
|
\section{Build and C API Changes}
|
|
|
|
Changes to Python's build process and to the C API include:
|
|
|
|
\begin{itemize}
|
|
|
|
\item The design of the bytecode compiler has changed a great deal, no
|
|
longer generating bytecode by traversing the parse tree. Instead
|
|
the parse tree is converted to an abstract syntax tree (or AST), and it is
|
|
the abstract syntax tree that's traversed to produce the bytecode.
|
|
|
|
No documentation has been written for the AST code yet. To start
|
|
learning about it, read the definition of the various AST nodes in
|
|
\file{Parser/Python.asdl}. A Python script reads this file and
|
|
generates a set of C structure definitions in
|
|
\file{Include/Python-ast.h}. The \cfunction{PyParser_ASTFromString()}
|
|
and \cfunction{PyParser_ASTFromFile()}, defined in
|
|
\file{Include/pythonrun.h}, take Python source as input and return the
|
|
root of an AST representing the contents. This AST can then be turned
|
|
into a code object by \cfunction{PyAST_Compile()}. For more
|
|
information, read the source code, and then ask questions on
|
|
python-dev.
|
|
|
|
% List of names taken from Jeremy's python-dev post at
|
|
% http://mail.python.org/pipermail/python-dev/2005-October/057500.html
|
|
The AST code was developed under Jeremy Hylton's management, and
|
|
implemented by (in alphabetical order) Brett Cannon, Nick Coghlan,
|
|
Grant Edwards, John Ehresman, Kurt Kaiser, Neal Norwitz, Tim Peters,
|
|
Armin Rigo, and Neil Schemenauer, plus the participants in a number of
|
|
AST sprints at conferences such as PyCon.
|
|
|
|
\item The built-in set types now have an official C API. Call
|
|
\cfunction{PySet_New()} and \cfunction{PyFrozenSet_New()} to create a
|
|
new set, \cfunction{PySet_Add()} and \cfunction{PySet_Discard()} to
|
|
add and remove elements, and \cfunction{PySet_Contains} and
|
|
\cfunction{PySet_Size} to examine the set's state.
|
|
|
|
\item The \cfunction{PyRange_New()} function was removed. It was
|
|
never documented, never used in the core code, and had dangerously lax
|
|
error checking.
|
|
|
|
\end{itemize}
|
|
|
|
|
|
%======================================================================
|
|
\subsection{Port-Specific Changes}
|
|
|
|
Platform-specific changes go here.
|
|
|
|
|
|
%======================================================================
|
|
\section{Other Changes and Fixes \label{section-other}}
|
|
|
|
As usual, there were a bunch of other improvements and bugfixes
|
|
scattered throughout the source tree. A search through the CVS change
|
|
logs finds there were XXX patches applied and YYY bugs fixed between
|
|
Python 2.4 and 2.5. Both figures are likely to be underestimates.
|
|
|
|
Some of the more notable changes are:
|
|
|
|
\begin{itemize}
|
|
|
|
\item Details go here.
|
|
|
|
\end{itemize}
|
|
|
|
|
|
%======================================================================
|
|
\section{Porting to Python 2.5}
|
|
|
|
This section lists previously described changes that may require
|
|
changes to your code:
|
|
|
|
\begin{itemize}
|
|
|
|
\item Some old deprecated modules (\module{statcache}, \module{tzparse},
|
|
\module{whrandom}) have been moved to \file{Lib/lib-old}.
|
|
You can get access to these modules again by adding the directory
|
|
to your \code{sys.path}:
|
|
|
|
\begin{verbatim}
|
|
import os
|
|
from distutils import sysconfig
|
|
|
|
lib_dir = sysconfig.get_python_lib(standard_lib=True)
|
|
old_dir = os.path.join(lib_dir, 'lib-old')
|
|
sys.path.append(old_dir)
|
|
\end{verbatim}
|
|
|
|
Doing so is discouraged, however; it's better to update any code that
|
|
still uses these modules.
|
|
|
|
% the pickle module no longer uses the deprecated bin parameter.
|
|
|
|
\end{itemize}
|
|
|
|
|
|
%======================================================================
|
|
\section{Acknowledgements \label{acks}}
|
|
|
|
The author would like to thank the following people for offering
|
|
suggestions, corrections and assistance with various drafts of this
|
|
article: .
|
|
|
|
\end{document}
|