Added IGNORE_EXCEPTION_DETAIL comparison option. The need is explained
in the new docs. DocTestRunner.__run: Separate the determination of the example outcome from reporting that outcome, to squash brittle code duplication and excessive nesting.
This commit is contained in:
parent
ba6019691e
commit
1fbf9c5ec1
|
@ -307,6 +307,9 @@ Some details you should read once, but won't need to remember:
|
||||||
to be the start of the exception detail. Of course this does the
|
to be the start of the exception detail. Of course this does the
|
||||||
right thing for genuine tracebacks.
|
right thing for genuine tracebacks.
|
||||||
|
|
||||||
|
\item When the \constant{IGNORE_EXCEPTION_DETAIL} doctest option is
|
||||||
|
is specified, everything following the leftmost colon is ignored.
|
||||||
|
|
||||||
\end{itemize}
|
\end{itemize}
|
||||||
|
|
||||||
\versionchanged[The ability to handle a multi-line exception detail
|
\versionchanged[The ability to handle a multi-line exception detail
|
||||||
|
@ -365,6 +368,34 @@ example's expected output:
|
||||||
is prone to in regular expressions.
|
is prone to in regular expressions.
|
||||||
\end{datadesc}
|
\end{datadesc}
|
||||||
|
|
||||||
|
\begin{datadesc}{IGNORE_EXCEPTION_DETAIL}
|
||||||
|
When specified, an example that expects an exception passes if
|
||||||
|
an exception of the expected type is raised, even if the exception
|
||||||
|
detail does not match. For example, an example expecting
|
||||||
|
\samp{ValueError: 42} will pass if the actual exception raised is
|
||||||
|
\samp{ValueError: 3*14}, but will fail, e.g., if
|
||||||
|
\exception{TypeError} is raised.
|
||||||
|
|
||||||
|
Note that a similar effect can be obtained using \constant{ELLIPSIS},
|
||||||
|
and \constant{IGNORE_EXCEPTION_DETAIL} may go away when Python releases
|
||||||
|
prior to 2.4 become uninteresting. Until then,
|
||||||
|
\constant{IGNORE_EXCEPTION_DETAIL} is the only clear way to write a
|
||||||
|
doctest that doesn't care about the exception detail yet continues
|
||||||
|
to pass under Python releases prior to 2.4 (doctest directives
|
||||||
|
appear to be comments to them). For example,
|
||||||
|
|
||||||
|
\begin{verbatim}
|
||||||
|
>>> (1, 2)[3] = 'moo' #doctest: +IGNORE_EXCEPTION_DETAIL
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "<stdin>", line 1, in ?
|
||||||
|
TypeError: object doesn't support item assignment
|
||||||
|
\end{verbatim}
|
||||||
|
|
||||||
|
passes under Python 2.4 and Python 2.3. The detail changed in 2.4,
|
||||||
|
to say "does not" instead of "doesn't".
|
||||||
|
|
||||||
|
\end{datadesc}
|
||||||
|
|
||||||
\begin{datadesc}{COMPARISON_FLAGS}
|
\begin{datadesc}{COMPARISON_FLAGS}
|
||||||
A bitmask or'ing together all the comparison flags above.
|
A bitmask or'ing together all the comparison flags above.
|
||||||
\end{datadesc}
|
\end{datadesc}
|
||||||
|
@ -463,6 +494,7 @@ can be useful.
|
||||||
|
|
||||||
\versionchanged[Constants \constant{DONT_ACCEPT_BLANKLINE},
|
\versionchanged[Constants \constant{DONT_ACCEPT_BLANKLINE},
|
||||||
\constant{NORMALIZE_WHITESPACE}, \constant{ELLIPSIS},
|
\constant{NORMALIZE_WHITESPACE}, \constant{ELLIPSIS},
|
||||||
|
\constant{IGNORE_EXCEPTION_DETAIL},
|
||||||
\constant{REPORT_UDIFF}, \constant{REPORT_CDIFF},
|
\constant{REPORT_UDIFF}, \constant{REPORT_CDIFF},
|
||||||
\constant{REPORT_NDIFF}, \constant{REPORT_ONLY_FIRST_FAILURE},
|
\constant{REPORT_NDIFF}, \constant{REPORT_ONLY_FIRST_FAILURE},
|
||||||
\constant{COMPARISON_FLAGS} and \constant{REPORTING_FLAGS}
|
\constant{COMPARISON_FLAGS} and \constant{REPORTING_FLAGS}
|
||||||
|
|
|
@ -176,6 +176,7 @@ __all__ = [
|
||||||
'DONT_ACCEPT_BLANKLINE',
|
'DONT_ACCEPT_BLANKLINE',
|
||||||
'NORMALIZE_WHITESPACE',
|
'NORMALIZE_WHITESPACE',
|
||||||
'ELLIPSIS',
|
'ELLIPSIS',
|
||||||
|
'IGNORE_EXCEPTION_DETAIL',
|
||||||
'COMPARISON_FLAGS',
|
'COMPARISON_FLAGS',
|
||||||
'REPORT_UDIFF',
|
'REPORT_UDIFF',
|
||||||
'REPORT_CDIFF',
|
'REPORT_CDIFF',
|
||||||
|
@ -261,11 +262,13 @@ DONT_ACCEPT_TRUE_FOR_1 = register_optionflag('DONT_ACCEPT_TRUE_FOR_1')
|
||||||
DONT_ACCEPT_BLANKLINE = register_optionflag('DONT_ACCEPT_BLANKLINE')
|
DONT_ACCEPT_BLANKLINE = register_optionflag('DONT_ACCEPT_BLANKLINE')
|
||||||
NORMALIZE_WHITESPACE = register_optionflag('NORMALIZE_WHITESPACE')
|
NORMALIZE_WHITESPACE = register_optionflag('NORMALIZE_WHITESPACE')
|
||||||
ELLIPSIS = register_optionflag('ELLIPSIS')
|
ELLIPSIS = register_optionflag('ELLIPSIS')
|
||||||
|
IGNORE_EXCEPTION_DETAIL = register_optionflag('IGNORE_EXCEPTION_DETAIL')
|
||||||
|
|
||||||
COMPARISON_FLAGS = (DONT_ACCEPT_TRUE_FOR_1 |
|
COMPARISON_FLAGS = (DONT_ACCEPT_TRUE_FOR_1 |
|
||||||
DONT_ACCEPT_BLANKLINE |
|
DONT_ACCEPT_BLANKLINE |
|
||||||
NORMALIZE_WHITESPACE |
|
NORMALIZE_WHITESPACE |
|
||||||
ELLIPSIS)
|
ELLIPSIS |
|
||||||
|
IGNORE_EXCEPTION_DETAIL)
|
||||||
|
|
||||||
REPORT_UDIFF = register_optionflag('REPORT_UDIFF')
|
REPORT_UDIFF = register_optionflag('REPORT_UDIFF')
|
||||||
REPORT_CDIFF = register_optionflag('REPORT_CDIFF')
|
REPORT_CDIFF = register_optionflag('REPORT_CDIFF')
|
||||||
|
@ -1293,6 +1296,10 @@ class DocTestRunner:
|
||||||
# to modify them).
|
# to modify them).
|
||||||
original_optionflags = self.optionflags
|
original_optionflags = self.optionflags
|
||||||
|
|
||||||
|
SUCCESS, FAILURE, BOOM = range(3) # `outcome` state
|
||||||
|
|
||||||
|
check = self._checker.check_output
|
||||||
|
|
||||||
# Process each example.
|
# Process each example.
|
||||||
for examplenum, example in enumerate(test.examples):
|
for examplenum, example in enumerate(test.examples):
|
||||||
|
|
||||||
|
@ -1337,45 +1344,53 @@ class DocTestRunner:
|
||||||
|
|
||||||
got = self._fakeout.getvalue() # the actual output
|
got = self._fakeout.getvalue() # the actual output
|
||||||
self._fakeout.truncate(0)
|
self._fakeout.truncate(0)
|
||||||
|
outcome = FAILURE # guilty until proved innocent or insane
|
||||||
|
|
||||||
# If the example executed without raising any exceptions,
|
# If the example executed without raising any exceptions,
|
||||||
# then verify its output and report its outcome.
|
# verify its output.
|
||||||
if exception is None:
|
if exception is None:
|
||||||
if self._checker.check_output(example.want, got,
|
if check(example.want, got, self.optionflags):
|
||||||
self.optionflags):
|
outcome = SUCCESS
|
||||||
if not quiet:
|
|
||||||
self.report_success(out, test, example, got)
|
|
||||||
else:
|
|
||||||
if not quiet:
|
|
||||||
self.report_failure(out, test, example, got)
|
|
||||||
failures += 1
|
|
||||||
|
|
||||||
# If the example raised an exception, then check if it was
|
# The example raised an exception: check if it was expected.
|
||||||
# expected.
|
|
||||||
else:
|
else:
|
||||||
exc_info = sys.exc_info()
|
exc_info = sys.exc_info()
|
||||||
exc_msg = traceback.format_exception_only(*exc_info[:2])[-1]
|
exc_msg = traceback.format_exception_only(*exc_info[:2])[-1]
|
||||||
|
if not quiet:
|
||||||
|
got += _exception_traceback(exc_info)
|
||||||
|
|
||||||
# If `example.exc_msg` is None, then we weren't
|
# If `example.exc_msg` is None, then we weren't expecting
|
||||||
# expecting an exception.
|
# an exception.
|
||||||
if example.exc_msg is None:
|
if example.exc_msg is None:
|
||||||
if not quiet:
|
outcome = BOOM
|
||||||
self.report_unexpected_exception(out, test, example,
|
|
||||||
exc_info)
|
# We expected an exception: see whether it matches.
|
||||||
failures += 1
|
elif check(example.exc_msg, exc_msg, self.optionflags):
|
||||||
# If `example.exc_msg` matches the actual exception
|
outcome = SUCCESS
|
||||||
# message (`exc_msg`), then the example succeeds.
|
|
||||||
elif (self._checker.check_output(example.exc_msg, exc_msg,
|
# Another chance if they didn't care about the detail.
|
||||||
self.optionflags)):
|
elif self.optionflags & IGNORE_EXCEPTION_DETAIL:
|
||||||
if not quiet:
|
m1 = re.match(r'[^:]*:', example.exc_msg)
|
||||||
got += _exception_traceback(exc_info)
|
m2 = re.match(r'[^:]*:', exc_msg)
|
||||||
self.report_success(out, test, example, got)
|
if m1 and m2 and check(m1.group(0), m2.group(0),
|
||||||
# Otherwise, the example fails.
|
self.optionflags):
|
||||||
else:
|
outcome = SUCCESS
|
||||||
if not quiet:
|
|
||||||
got += _exception_traceback(exc_info)
|
# Report the outcome.
|
||||||
self.report_failure(out, test, example, got)
|
if outcome is SUCCESS:
|
||||||
failures += 1
|
if not quiet:
|
||||||
|
self.report_success(out, test, example, got)
|
||||||
|
elif outcome is FAILURE:
|
||||||
|
if not quiet:
|
||||||
|
self.report_failure(out, test, example, got)
|
||||||
|
failures += 1
|
||||||
|
elif outcome is BOOM:
|
||||||
|
if not quiet:
|
||||||
|
self.report_unexpected_exception(out, test, example,
|
||||||
|
exc_info)
|
||||||
|
failures += 1
|
||||||
|
else:
|
||||||
|
assert False, ("unknown outcome", outcome)
|
||||||
|
|
||||||
# Restore the option flags (in case they were modified)
|
# Restore the option flags (in case they were modified)
|
||||||
self.optionflags = original_optionflags
|
self.optionflags = original_optionflags
|
||||||
|
|
|
@ -837,6 +837,43 @@ message is raised, then it is reported as a failure:
|
||||||
ValueError: message
|
ValueError: message
|
||||||
(1, 1)
|
(1, 1)
|
||||||
|
|
||||||
|
However, IGNORE_EXCEPTION_DETAIL can be used to allow a mismatch in the
|
||||||
|
detail:
|
||||||
|
|
||||||
|
>>> def f(x):
|
||||||
|
... r'''
|
||||||
|
... >>> raise ValueError, 'message' #doctest: +IGNORE_EXCEPTION_DETAIL
|
||||||
|
... Traceback (most recent call last):
|
||||||
|
... ValueError: wrong message
|
||||||
|
... '''
|
||||||
|
>>> test = doctest.DocTestFinder().find(f)[0]
|
||||||
|
>>> doctest.DocTestRunner(verbose=False).run(test)
|
||||||
|
(0, 1)
|
||||||
|
|
||||||
|
But IGNORE_EXCEPTION_DETAIL does not allow a mismatch in the exception type:
|
||||||
|
|
||||||
|
>>> def f(x):
|
||||||
|
... r'''
|
||||||
|
... >>> raise ValueError, 'message' #doctest: +IGNORE_EXCEPTION_DETAIL
|
||||||
|
... Traceback (most recent call last):
|
||||||
|
... TypeError: wrong type
|
||||||
|
... '''
|
||||||
|
>>> test = doctest.DocTestFinder().find(f)[0]
|
||||||
|
>>> doctest.DocTestRunner(verbose=False).run(test)
|
||||||
|
... # doctest: +ELLIPSIS
|
||||||
|
**********************************************************************
|
||||||
|
Line 2, in f
|
||||||
|
Failed example:
|
||||||
|
raise ValueError, 'message' #doctest: +IGNORE_EXCEPTION_DETAIL
|
||||||
|
Expected:
|
||||||
|
Traceback (most recent call last):
|
||||||
|
TypeError: wrong type
|
||||||
|
Got:
|
||||||
|
Traceback (most recent call last):
|
||||||
|
...
|
||||||
|
ValueError: message
|
||||||
|
(1, 1)
|
||||||
|
|
||||||
If an exception is raised but not expected, then it is reported as an
|
If an exception is raised but not expected, then it is reported as an
|
||||||
unexpected exception:
|
unexpected exception:
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue