mirror of https://github.com/python/cpython
478 lines
20 KiB
TeX
478 lines
20 KiB
TeX
\section{Built-in Module \sectcode{re}}
|
|
\label{module-re}
|
|
|
|
\bimodindex{re}
|
|
|
|
% XXX Remove before 1.5final release.
|
|
{\large\bf The \code{re} module is still in the process of being
|
|
developed, and more features will be added in future 1.5 alphas and
|
|
betas. This documentation is also preliminary and incomplete. If you
|
|
find a bug or documentation error, or just find something unclear,
|
|
please send a message to
|
|
\code{string-sig@python.org}, and we'll fix it.}
|
|
|
|
This module provides regular expression matching operations similar to
|
|
those found in Perl. It's 8-bit
|
|
clean: both patterns and strings may contain null bytes and characters
|
|
whose high bit is set. It is always available.
|
|
|
|
Regular expressions use the backslash character (\code{\e}) to
|
|
indicate special forms or to allow special characters to be used
|
|
without invoking their special meaning. This collides with Python's
|
|
usage of the same character for the same purpose in string literals;
|
|
for example, to match a literal backslash, one might have to write
|
|
\code{\e\e\e\e} as the pattern string, because the regular expression must be \code{\e\e}, and each backslash must be expressed as \code{\e\e} inside a regular Python string literal.
|
|
|
|
The solution is to use Python's raw string notation for regular
|
|
expression patterns; backslashes are not handled in any special way in
|
|
a string literal prefixed with 'r'. So \code{r"\e n"} is a two
|
|
character string containing a backslash and the letter 'n', while
|
|
\code{"\e n"} is a one-character string containing a newline. Usually
|
|
patterns will be expressed in Python code using this raw string notation.
|
|
|
|
% XXX Can the following section be dropped, or should it be boiled down?
|
|
|
|
%\strong{Please note:} There is a little-known fact about Python string
|
|
%literals which means that you don't usually have to worry about
|
|
%doubling backslashes, even though they are used to escape special
|
|
%characters in string literals as well as in regular expressions. This
|
|
%is because Python doesn't remove backslashes from string literals if
|
|
%they are followed by an unrecognized escape character.
|
|
%\emph{However}, if you want to include a literal \dfn{backslash} in a
|
|
%regular expression represented as a string literal, you have to
|
|
%\emph{quadruple} it or enclose it in a singleton character class.
|
|
%E.g.\ to extract \LaTeX\ \code{\e section\{{\rm
|
|
%\ldots}\}} headers from a document, you can use this pattern:
|
|
%\code{'[\e ] section\{\e (.*\e )\}'}. \emph{Another exception:}
|
|
%the escape sequence \code{\e b} is significant in string literals
|
|
%(where it means the ASCII bell character) as well as in Emacs regular
|
|
%expressions (where it stands for a word boundary), so in order to
|
|
%search for a word boundary, you should use the pattern \code{'\e \e b'}.
|
|
%Similarly, a backslash followed by a digit 0-7 should be doubled to
|
|
%avoid interpretation as an octal escape.
|
|
|
|
\subsection{Regular Expressions}
|
|
|
|
A regular expression (or RE) specifies a set of strings that matches
|
|
it; the functions in this module let you check if a particular string
|
|
matches a given regular expression (or if a given regular expression
|
|
matches a particular string, which comes down to the same thing).
|
|
|
|
Regular expressions can be concatenated to form new regular
|
|
expressions; if \emph{A} and \emph{B} are both regular expressions,
|
|
then \emph{AB} is also an regular expression. If a string \emph{p}
|
|
matches A and another string \emph{q} matches B, the string \emph{pq}
|
|
will match AB. Thus, complex expressions can easily be constructed
|
|
from simpler primitive expressions like the ones described here. For
|
|
details of the theory and implementation of regular expressions,
|
|
consult the Friedl book referenced below, or almost any textbook about
|
|
compiler construction.
|
|
|
|
A brief explanation of the format of regular expressions follows. For
|
|
further information and a gentler presentation, consult XXX somewhere.
|
|
|
|
Regular expressions can contain both special and ordinary characters.
|
|
Most ordinary characters, like '\code{A}', '\code{a}', or '\code{0}',
|
|
are the simplest regular expressions; they simply match themselves.
|
|
You can concatenate ordinary characters, so '\code{last}' matches the
|
|
characters 'last'. (In the rest of this section, we'll write RE's in
|
|
\code{this special font}, usually without quotes, and strings to be
|
|
matched 'in single quotes'.)
|
|
|
|
Some characters, like \code{|} or \code{(}, are special. Special
|
|
characters either stand for classes of ordinary characters, or affect
|
|
how the regular expressions around them are interpreted.
|
|
|
|
The special characters are:
|
|
\begin{itemize}
|
|
\item[\code{.}] (Dot.) In the default mode, this matches any
|
|
character except a newline. If the \code{DOTALL} flag has been
|
|
specified, this matches any character including a newline.
|
|
\item[\code{\^}] (Caret.) Matches the start of the string, and in
|
|
\code{MULTILINE} mode also immediately after each newline.
|
|
\item[\code{\$}] Matches the end of the string.
|
|
\code{foo} matches both 'foo' and 'foobar', while the regular
|
|
expression '\code{foo\$}' matches only 'foo'.
|
|
%
|
|
\item[\code{*}] Causes the resulting RE to
|
|
match 0 or more repetitions of the preceding RE, as many repetitions
|
|
as are possible. \code{ab*} will
|
|
match 'a', 'ab', or 'a' followed by any number of 'b's.
|
|
%
|
|
\item[\code{+}] Causes the
|
|
resulting RE to match 1 or more repetitions of the preceding RE.
|
|
\code{ab+} will match 'a' followed by any non-zero number of 'b's; it
|
|
will not match just 'a'.
|
|
%
|
|
\item[\code{?}] Causes the resulting RE to
|
|
match 0 or 1 repetitions of the preceding RE. \code{ab?} will
|
|
match either 'a' or 'ab'.
|
|
\item[\code{*?}, \code{+?}, \code{??}] The \code{*}, \code{+}, and
|
|
\code{?} qualifiers are all \dfn{greedy}; they match as much text as
|
|
possible. Sometimes this behaviour isn't desired; if the RE
|
|
\code{<.*>} is matched against \code{<H1>title</H1>}, it will match the
|
|
entire string, and not just \code{<H1>}.
|
|
Adding \code{?} after the qualifier makes it perform the match in
|
|
\dfn{non-greedy} or \dfn{minimal} fashion; as few characters as
|
|
possible will be matched. Using \code{.*?} in the previous
|
|
expression, it will match only \code{<H1>}.
|
|
%
|
|
\item[\code{\e}] Either escapes special characters (permitting you to match
|
|
characters like '*?+\&\$'), or signals a special sequence; special
|
|
sequences are discussed below.
|
|
|
|
If you're not using a raw string to
|
|
express the pattern, remember that Python also uses the
|
|
backslash as an escape sequence in string literals; if the escape
|
|
sequence isn't recognized by Python's parser, the backslash and
|
|
subsequent character are included in the resulting string. However,
|
|
if Python would recognize the resulting sequence, the backslash should
|
|
be repeated twice. This is complicated and hard to understand, so
|
|
it's highly recommended that you use raw strings.
|
|
%
|
|
\item[\code{[]}] Used to indicate a set of characters. Characters can
|
|
be listed individually, or a range is indicated by giving two
|
|
characters and separating them by a '-'. Special characters are not
|
|
active inside sets. For example, \code{[akm\$]} will match any of the
|
|
characters 'a', 'k', 'm', or '\$'; \code{[a-z]} will match any
|
|
lowercase letter and \code{[a-zA-Z0-9]} matches any letter or digit.
|
|
Character classes of the form \code{\e \var{X}} defined below are also acceptable.
|
|
If you want to include a \code{]} or a \code{-} inside a
|
|
set, precede it with a backslash.
|
|
|
|
Characters \emph{not} within a range can be matched by including a
|
|
\code{\^} as the first character of the set; \code{\^} elsewhere will
|
|
simply match the '\code{\^}' character.
|
|
%
|
|
\item[\code{|}]\code{A|B}, where A and B can be arbitrary REs,
|
|
creates a regular expression that will match either A or B. This can
|
|
be used inside groups (see below) as well. To match a literal '|',
|
|
use \code{\e|}, or enclose it inside a character class, like \code{[|]}.
|
|
%
|
|
\item[\code{( ... )}] Matches whatever regular expression is inside the parentheses, and indicates the start and end of a group; the
|
|
contents of a group can be retrieved after a match has been performed,
|
|
and can be matched later in the string with the
|
|
\code{\e \var{number}} special sequence, described below. To match the
|
|
literals '(' or ')',
|
|
use \code{\e(} or \code{\e)}, or enclose them inside a character
|
|
class: \code{[(] [)]}.
|
|
%
|
|
\item[\code{(?:...)}] A non-grouping version of regular parentheses.
|
|
Matches whatever's inside the parentheses, but the text matched by the
|
|
group \emph{cannot} be retrieved after performing a match or
|
|
referenced later in the pattern.
|
|
%
|
|
\item[\code{(?P<\var{name}>...)}] Similar to regular parentheses, but
|
|
the text matched by the group is accessible via the symbolic group
|
|
name \var{name}. Group names must be valid Python identifiers. A
|
|
symbolic group is also a numbered group, just as if the group were not
|
|
named. So the group named 'id' in the example above can also be
|
|
referenced as the numbered group 1.
|
|
|
|
For example, if the pattern string is
|
|
\code{r'(?P<id>[a-zA-Z_]\e w*)'}, the group can be referenced by its
|
|
name in arguments to methods of match objects, such as \code{m.group('id')}
|
|
or \code{m.end('id')}, and also by name in pattern text (e.g. \code{(?P=id)}) and
|
|
replacement text (e.g. \code{\e g<id>}).
|
|
%
|
|
\item[\code{(?\#...)}] A comment; the contents of the parentheses are simply ignored.
|
|
%
|
|
\item[\code{(?=...)}] Matches if \code{RE} matches next. This is not
|
|
implemented as of Python 1.5a3.
|
|
%
|
|
\item[\code{(?!...)}] Matches if \code{...} doesn't match next. This is not
|
|
implemented as of Python 1.5a3.
|
|
\end{itemize}
|
|
|
|
The special sequences consist of '\code{\e}' and a character from the
|
|
list below. If the ordinary character is not on the list, then the
|
|
resulting RE will match the second character. For example,
|
|
\code{\e\$} matches the character '\$'. Ones where the backslash
|
|
should be doubled are indicated.
|
|
|
|
\begin{itemize}
|
|
|
|
%
|
|
\item[\code{\e \var{number}}] Matches the contents of the group of the
|
|
same number. For example, \code{(.+) \e 1} matches 'the the' or '55
|
|
55', but not 'the end' (note the space after the group). This special
|
|
sequence can only be used to match one of the first 99 groups. If the
|
|
first digit of \var{number} is 0, or \var{number} is 3 octal digits
|
|
long, it will not interpreted as a group match, but as the character
|
|
with octal value \var{number}.
|
|
%
|
|
\item[\code{\e A}] Matches only at the start of the string.
|
|
%
|
|
\item[\code{\e b}] Matches the empty string, but only at the
|
|
beginning or end of a word. A word is defined as a sequence of
|
|
alphanumeric characters, so the end of a word is indicated by
|
|
whitespace or a non-alphanumeric character.
|
|
%
|
|
\item[\code{\e B}] Matches the empty string, but only when it is \emph{not} at the
|
|
beginning or end of a word.
|
|
%
|
|
\item[\code{\e d}]Matches any decimal digit; this is
|
|
equivalent to the set \code{[0-9]}.
|
|
%
|
|
\item[\code{\e D}]Matches any non-digit character; this is
|
|
equivalent to the set \code{[{\^}0-9]}.
|
|
%
|
|
\item[\code{\e s}]Matches any whitespace character; this is
|
|
equivalent to the set \code{[ \e t\e n\e r\e f\e v]}.
|
|
%
|
|
\item[\code{\e S}]Matches any non-whitespace character; this is
|
|
equivalent to the set \code{[{\^} \e t\e n\e r\e f\e v]}.
|
|
%
|
|
\item[\code{\e w}]Matches any alphanumeric character; this is
|
|
equivalent to the set \code{[a-zA-Z0-9_]}.
|
|
%
|
|
\item[\code{\e W}] Matches any non-alphanumeric character; this is
|
|
equivalent to the set \code{[{\^}a-zA-Z0-9_]}.
|
|
|
|
\item[\code{\e Z}]Matches only at the end of the string.
|
|
%
|
|
|
|
\item[\code{\e \e}] Matches a literal backslash.
|
|
|
|
\end{itemize}
|
|
|
|
\subsection{Module Contents}
|
|
|
|
The module defines the following functions and constants, and an exception:
|
|
|
|
\renewcommand{\indexsubitem}{(in module re)}
|
|
|
|
\begin{funcdesc}{compile}{pattern\optional{\, flags}}
|
|
Compile a regular expression pattern into a regular expression
|
|
object, which can be used for matching using its \code{match} and
|
|
\code{search} methods, described below.
|
|
|
|
The sequence
|
|
%
|
|
\bcode\begin{verbatim}
|
|
prog = re.compile(pat)
|
|
result = prog.match(str)
|
|
\end{verbatim}\ecode
|
|
%
|
|
is equivalent to
|
|
%
|
|
\bcode\begin{verbatim}
|
|
result = re.match(pat, str)
|
|
\end{verbatim}\ecode
|
|
%
|
|
but the version using \code{compile()} is more efficient when multiple
|
|
regular expressions are used concurrently in a single program.
|
|
%(The compiled version of the last pattern passed to \code{regex.match()} or
|
|
%\code{regex.search()} is cached, so programs that use only a single
|
|
%regular expression at a time needn't worry about compiling regular
|
|
%expressions.)
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{escape}{string}
|
|
Return \var{string} with all non-alphanumerics backslashed; this is
|
|
useful if you want to match some variable string which may have
|
|
regular expression metacharacters in it.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{match}{pattern\, string\optional{\, flags}}
|
|
If zero or more characters at the beginning of \var{string} match
|
|
the regular expression \var{pattern}, return a corresponding
|
|
\code{Match} object. Return \code{None} if the string does not
|
|
match the pattern; note that this is different from a zero-length
|
|
match.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{search}{pattern\, string\optional{\, flags}}
|
|
Scan through \var{string} looking for a location where the regular
|
|
expression \var{pattern} produces a match. Return \code{None} if no
|
|
position in the string matches the pattern; note that this is
|
|
different from finding a zero-length match at some point in the string.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{split}{pattern\, string\, \optional{, maxsplit=0}}
|
|
Split \var{string} by the occurrences of \var{pattern}. If
|
|
capturing parentheses are used in pattern, then occurrences of
|
|
patterns or subpatterns are also returned.
|
|
%
|
|
\bcode\begin{verbatim}
|
|
>>> re.split('[\W]+', 'Words, words, words.')
|
|
['Words', 'words', 'words', '']
|
|
>>> re.split('([\W]+)', 'Words, words, words.')
|
|
['Words', ', ', 'words', ', ', 'words', '.', '']
|
|
\end{verbatim}\ecode
|
|
%
|
|
This function combines and extends the functionality of
|
|
\code{regex.split()} and \code{regex.splitx()}.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{sub}{pattern\, repl\, string\optional{, count=0}}
|
|
Return the string obtained by replacing the leftmost non-overlapping
|
|
occurrences of \var{pattern} in \var{string} by the replacement
|
|
\var{repl}. If the pattern isn't found, \var{string} is returned
|
|
unchanged. \var{repl} can be a string or a function; if a function,
|
|
it is called for every non-overlapping occurance of \var{pattern}.
|
|
The function takes a single match object argument, and
|
|
returns the replacement string. For example:
|
|
%
|
|
\bcode\begin{verbatim}
|
|
>>> def dashrepl(matchobj):
|
|
... if matchobj.group(0) == '-': return ' '
|
|
... else: return '-'
|
|
>>> re.sub('-{1,2}', dashrepl, 'pro----gram-files')
|
|
'pro--gram files'
|
|
\end{verbatim}\ecode
|
|
%
|
|
The pattern may be a string or a
|
|
regexp object; if you need to specify regular expression flags, you
|
|
must use a regexp object, or use embedded modifiers in a pattern
|
|
string; e.g.
|
|
%
|
|
\bcode\begin{verbatim}
|
|
sub("(?i)b+", "x", "bbbb BBBB") returns 'x x'.
|
|
\end{verbatim}\ecode
|
|
%
|
|
The optional argument \var{count} is the maximum number of pattern
|
|
occurrences to be replaced; count must be a non-negative integer, and
|
|
the default value of 0 means to replace all occurrences.
|
|
|
|
Empty matches for the pattern are replaced only when not adjacent to a
|
|
previous match, so \code{sub('x*', '-', 'abc')} returns '-a-b-c-'.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{subn}{pattern\, repl\, string\optional{, count=0}}
|
|
Perform the same operation as \code{sub()}, but return a tuple
|
|
\code{(new_string, number_of_subs_made)}.
|
|
\end{funcdesc}
|
|
|
|
\begin{excdesc}{error}
|
|
Exception raised when a string passed to one of the functions here
|
|
is not a valid regular expression (e.g., unmatched parentheses) or
|
|
when some other error occurs during compilation or matching. (It is
|
|
never an error if a string contains no match for a pattern.)
|
|
\end{excdesc}
|
|
|
|
\subsection{Regular Expression Objects}
|
|
Compiled regular expression objects support the following methods and
|
|
attributes:
|
|
|
|
\renewcommand{\indexsubitem}{(re method)}
|
|
\begin{funcdesc}{match}{string\optional{\, pos}}
|
|
If zero or more characters at the beginning of \var{string} match
|
|
this regular expression, return a corresponding
|
|
\code{Match} object. Return \code{None} if the string does not
|
|
match the pattern; note that this is different from a zero-length
|
|
match.
|
|
|
|
The optional second parameter \var{pos} gives an index in the string
|
|
where the search is to start; it defaults to \code{0}. This is not
|
|
completely equivalent to slicing the string; the \code{'\^'} pattern
|
|
character matches at the real begin of the string and at positions
|
|
just after a newline, not necessarily at the index where the search
|
|
is to start.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{search}{string\optional{\, pos}}
|
|
Scan through \var{string} looking for a location where this regular
|
|
expression produces a match. Return \code{None} if no
|
|
position in the string matches the pattern; note that this is
|
|
different from finding a zero-length match at some point in the string.
|
|
|
|
The optional second parameter has the same meaning as for the
|
|
\code{match} method.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{split}{string\, \optional{, maxsplit=0}}
|
|
Identical to the \code{split} function, using the compiled pattern.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{sub}{repl\, string\optional{, count=0}}
|
|
Identical to the \code{sub} function, using the compiled pattern.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{subn}{repl\, string\optional{, count=0}}
|
|
Identical to the \code{subn} function, using the compiled pattern.
|
|
\end{funcdesc}
|
|
|
|
\renewcommand{\indexsubitem}{(regex attribute)}
|
|
|
|
\begin{datadesc}{flags}
|
|
The flags argument used when the regex object was compiled, or 0 if no
|
|
flags were provided.
|
|
\end{datadesc}
|
|
|
|
\begin{datadesc}{groupindex}
|
|
A dictionary mapping any symbolic group names (defined by
|
|
\code{?P<\var{id}>}) to group numbers. The dictionary is empty if no
|
|
symbolic groups were used in the pattern.
|
|
\end{datadesc}
|
|
|
|
\begin{datadesc}{pattern}
|
|
The pattern string from which the regex object was compiled.
|
|
\end{datadesc}
|
|
|
|
\subsection{Match Objects}
|
|
Match objects support the following methods and attributes:
|
|
|
|
\begin{funcdesc}{span}{group}
|
|
Return the 2-tuple \code{(start(\var{group}), end(\var{group}))}.
|
|
Note that if \var{group} did not contribute to the match, this is \code{(None,
|
|
None)}.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{start}{group}
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{end}{group}
|
|
Return the indices of the start and end of the substring matched by
|
|
\var{group}. Return \code{None} if \var{group} exists but did not contribute to
|
|
the match. Note that for a match object \code{m}, and a group \code{g}
|
|
that did contribute to the match, the substring matched by group \code{g} is
|
|
\bcode\begin{verbatim}
|
|
m.string[m.start(g):m.end(g)]
|
|
\end{verbatim}\ecode
|
|
%
|
|
Note too that \code{m.start(\var{group})} will equal
|
|
\code{m.end(\var{group})} if \var{group} matched a null string. For example,
|
|
after \code{m = re.search('b(c?)', 'cba')}, \code{m.start(0)} is 1,
|
|
\code{m.end(0)} is 2, \code{m.start(1)} and \code{m.end(1)} are both
|
|
2, and \code{m.start(2)} raises an
|
|
\code{IndexError} exception.
|
|
\end{funcdesc}
|
|
|
|
\begin{funcdesc}{group}{\optional{g1, g2, ...})}
|
|
This method is only valid when the last call to the \code{match}
|
|
or \code{search} method found a match. It returns one or more
|
|
groups of the match. If there is a single \var{index} argument,
|
|
the result is a single string; if there are multiple arguments, the
|
|
result is a tuple with one item per argument. If the \var{index} is
|
|
zero, the corresponding return value is the entire matching string; if
|
|
it is in the inclusive range [1..99], it is the string matching the
|
|
the corresponding parenthesized group (using the default syntax,
|
|
groups are parenthesized using \code{\e (} and \code{\e )}). If no
|
|
such group exists, the corresponding result is \code{None}.
|
|
|
|
If the regular expression was compiled by \code{symcomp} instead of
|
|
\code{compile}, the \var{index} arguments may also be strings
|
|
identifying groups by their group name.
|
|
\end{funcdesc}
|
|
|
|
\begin{datadesc}{pos}
|
|
The index at which the search or match began.
|
|
\end{datadesc}
|
|
|
|
\begin{datadesc}{re}
|
|
The regular expression object whose match() or search() method
|
|
produced this match object.
|
|
\end{datadesc}
|
|
|
|
\begin{datadesc}{string}
|
|
The string passed to \code{match()} or \code{search()}.
|
|
\end{datadesc}
|
|
|
|
|
|
|
|
\begin{seealso}
|
|
\seetext Jeffrey Friedl, \emph{Mastering Regular Expressions}.
|
|
\end{seealso}
|
|
|