request. Tim says that "correct 'fuzzy' comparison of floats cannot
be automated." (The motivation behind adding the new option
was verifying interactive examples in Python's latex documentation;
several such examples use numbers that don't print consistently on
different platforms.)
all examples in a given text file. (analagous to "testmod")
- Minor docstring fixes.
- Added module_relative parameter to DocTestFile/DocTestSuite, which
controls whether paths are module-relative & os-independent, or
os-specific.
- Fixed bug in handling of absolute paths.
- If run from an interactive session, make paths relative to the
directory containing sys.argv[0] (since __main__ doesn't have
a __file__ attribute).
with default False for testmod(). The real point of introducing this was
so that output from doctest.master.summarize() would be the same as in
2.3, and doctest.master in 2.4 is a backward-compatability hack used only
by testmod().
in the new docs.
DocTestRunner.__run: Separate the determination of the example outcome
from reporting that outcome, to squash brittle code duplication and
excessive nesting.
to unittest, so make it official: new module constants COMPARISON_FLAGS
and REPORTING_FLAGS, which are bitmasks or'ing together the relevant
individual option flags.
set_unittest_reportflags(): Reworked to use REPORTING_FLAGS, and
simplified overly complicated flag logic.
class FakeModule: Removed this; neither documented nor used.
- Improvements to interactive debugging support:
- Changed the replacement pdb.set_trace to redirect stdout to the
real stdout *only* during interactive debugging; stdout from code
continues to go to the fake stdout.
- When the interactive debugger gets to the end of an example,
automatically continue.
- Use a replacement linecache.getlines that will return source lines
from doctest examples; this makes the source available to the
debugger for interactive debugging.
- In test_doctest, use a specialized _FakeOutput class instead of a
temporary file to fake stdin for the interactive interpreter.
and intervening text strings.
- Removed DocTestParser.get_program(): use script_from_examples()
instead.
- Fixed bug in DocTestParser._INDENT_RE
- Fixed bug in DocTestParser._min_indent
- Moved _want_comment() to the utility function section
actual output into lines created spurious empty lines at the ends of
each. Those matched, but the fancy diffs had surprising line counts (1
larger than expected), and tests kept having to slam <BLANKLINE> into the
expected output to account for this. Using the splitlines() string method
with keepends=True instead accomplishes what was intended directly.
While a fancy diff can be confusing in the presence of ellipses, so far
I'm finding (2-0-0) that it's much more a major aid in narrowing down the
possibilities when an ellipsis-slinging test fails. So we no longer
refuse to do a fancy diff just because of ellipses.
This isn't ideal; it's just better.
rather than an expected output string. This gives the
output_difference method access to more information, such as the
indentation of the example, which might be useful.
NDIFF_DIFF->REPORT_NDIFF. This establishes the naming convention that
all reporting options should begin with "REPORT_" (since reporting
options are a different class from output comparison options; but they
are both set in optionflags).
to be more consistent with report_failure()
- If `want` or `got` is empty, then print "Expected nothing\n" or
"Got nothing\n" rather than "Expected:\n" or "Got:\n"
- Got rid of _tag_msg
exception message, or None if no exception is expected); and moved
exception parsing from DocTestRunner to DocTestParser. This is
architecturally cleaner, since it moves all parsing work to
DocTestParser; and it should make it easier for code outside
DocTestRunner (notably debugging code) to properly handle expected
exceptions.
a traceback message. I.e., examples that raise exceptions may no
longer generate pre-exception output. This restores the behavior of
doctest in python 2.3. The ability to check pre-exception output is
being removed because it makes the documentation simpler; and because
there are very few use cases for it.
NamedTemporaryFile (which can't work for this function's purposes on
Windows). Leaving temp files behind wasn't a great idea either, though,
so try to clean up. At least the test suite no longer leaves any of
these guys behind now.
docstrings toward being a lot shorter, and telling the whole truth in
the manual instead. This change is an example: the manual has detailed
explanations of the option names now, so it's Bad to repeat them in
the docstring (two detailed descriptions are certain to get out of synch).
Just listing the names has memory-jogging benefits, though, so that's
still helpful in the docstring.
- Test filenames sometimes had trailing .pyc or .pyo sufixes
(when module __file__ did).
- Trailing spaces spaces in expected output were dropped.
New default failure format:
- Separation of examples from file info makes examples easier to see
- More vertical separation, improving readability
- Emacs-recognized file info (also closer to Python exception format)
are non-obvious either way because the newline character "is invisible",
but it's still there all the same, and it's easier to explain/predict
if that reality is left alone.
It's redundant, since no output is written anyway: DebugRunner doesn't
generate any output for failures and unexpected exceptions, and since
verbose=False, it won't generate any output for non-failures either.
appeared at the end of a line. Repaired that. Also noted that it's
too easy to provoke this implementation into requiring exponential
time, and especially when a test fails. I'll replace the implementation
with an always-efficient one later.
them (which they are now), I had to rewrite the code to understand
it. This has got to be the most DWIM part of doctest -- but in context
is really necessary.
visually distinguish the expected output from the comments (use
"##" to mark expected outputs, and "#" to mark comments).
- If the string given to DocTestParser.get_program() is indented, then
strip its indentation. (In particular, find the min indentation of
non-blank lines, and strip that indentation from all lines.)
- Added comments for some regexps
- If the traceback type/message don't match, then still print full
traceback in report_failure (not just the first & last lines)
- Renamed DocTestRunner.__failure_header -> _failure_header
modify option flags for a single example; they do not turn options
on or off.)
- Added "indent" and "options" attributes for Example
- Got rid of add_newlines param to DocTestParser._parse_example (it's
no longer needed; Example's constructor now takes care of it).
- Added some docstrings
responsible for parsing the string.
- Renamed Parser to DocTestParser
- DocTestParser.get_*() now accept the string & name as command-line
arguments; the parser's constructor is now empty.
- Added DocTestParser.get_doctest() method
- Replaced "doctest_factory" argument to DocTestFinder with a "parser"
argument (takes a DocTestParser).
- Changed _tag_msg to take an indentation string argument.
the set_trace fiddling didn't make sense to me, and I ended up reworking
that part of the code. We really do want to save and restore
pdb.set_trace, so that each dynamically nested level of doctest gets
sys.stdout fiddled to what's appropriate for *it*. The only "trick"
really needed is that these layers of set_trace wrappers each call the
original pdb.set_trace (instead of the current pdb.set_trace).
the string one line at a time. The resulting code is (in my opinion,
anyway), much easier to read. In the process, I found and fixed a
bug in the orginal parser's line numbering in error messages (it was
inconsistant between 0-based and 1-based). Also, check for missing
blank lines after the prompt on all prompt lines, not just PS1 lines
(test added).
This got slammed in when find() was fixed to stop grabbing doctests
from modules imported *by* the module being tested. Such tests cannot
be expected to succeed, since they'll be run with the current module's
globals. Dozens of Zope3 doctests were failing because of that.
It wasn't clear why ignore_imports got added then. Maybe it's because
some existing tests failed when the change was made. Whatever, it's
a Bad Idea so it's gone now.
The only use of it was exceedingly obscure, in test_doctest's "Duplicate
Removal" test. It was "needed" there because, as an artifact of running
a doctest inside a doctest, the func_globals of functions compiled in
the second-level doctest don't match the module globals, and so the
test-finder believed these functions were from a foreign module and
skipped them. But that took a long time to figure out, and I actually
understand some of this stuff <0.9 wink>.
That problem was resolved by moving the source code for the second-level
doctest into an actual module (test/doctest_aliases.py).
The only remaining difficulty was that the test for the deprecated
Tester.rundict() then failed, because the test finder doesn't take
module=None at face value, trying to guess which module the user really
intended then. Its guess wasn't appropriate for what Tester.rundict
needs when module=None is given to *it*, which is "no, there is no
module here, and I mean it". So now passing module=False means exactly
that. This is hokey, but ignore_imports=False was really a hack to worm
around that there was no way to tell the test-finder that module=None
*sometimes* means what it says. There was no use case for the combination
of passing a real module with ignore_imports=False.
Ripped out the docs for the new DocTestFinder's namefilter argument,
and renamed it to _namefilter; this only existed to support isprivate.
Removed the new DocTestFinder's objfilter argument. No point adding
more cruft to a broken filtering design.
the tim-doctest-merge-24a2 tag on the the tim-doctest-branch branch.
We did development on the branch in case it wouldn't land in time for
2.4a2, but the branch looked good: Edward's tests passed there, ditto
Python's tests, and ditto the Zope3 tests. Together, those hit doctest
heavily.
skip over functions with private names (as indicated by the underscore
naming convention). The old default created too much of a risk that
user tests were being skipped inadvertently. Note, this change could
break code in the unlikely case that someone had intentionally put
failing tests in the docstrings of private functions. The breakage
is easily fixable by specifying the old behavior when calling testmod()
or Tester(). The more likely case is that the silent failure was
unintended and that the user needed to be informed so the test could be
fixed.
I won't have time to write real docs, but spent a lot of time adding
comments to his code and fleshing out the exported functions' docstrings.
There's probably opportunity to consolidate how docstrings get extracted
too, and the new code for that is probably better than the old code for
that (which strained mightily to recover from 2.2's new class/type
gimmicks).
now accepts "True" when a test expects "1", and similarly for "False"
versus "0". This is un-doctest-like, but on balance makes it much
more pleasant to write doctests that pass under 2.2 and 2.3. I expect
it to go away again, when 2.2 is forgotten. In the meantime, there's
a new doctest module constant that can be passed to a new optional
argument, if you want to turn this behavior off.
Note that this substitution is very simple-minded: the expected and
actual outputs have to consist of single tokens. No attempt is made,
e.g., to accept [True, False] when a test expects [1, 0]. This is a
simple hack for simple tests, and I intend to keep it that way.
PEP 285. Everything described in the PEP is here, and there is even
some documentation. I had to fix 12 unit tests; all but one of these
were printing Boolean outcomes that changed from 0/1 to False/True.
(The exception is test_unicode.py, which did a type(x) == type(y)
style comparison. I could've fixed that with a single line using
issubtype(x, type(y)), but instead chose to be explicit about those
places where a bool is expected.
Still to do: perhaps more documentation; change standard library
modules to return False/True from predicates.
Taught doctest about static methods, class methods, and property docstrings
in new-style classes. As for inspect.py/pydoc.py before it, the new stuff
needed didn't really fit into the old architecture (but was less of a
strain to force-fit here).
New-style class docstrings still aren't found, but that's the subject
of a different bug and I want to fix that right instead of hacking around
it in doctest.
from Tim Hochberg. Also mucho fiddling to change the way doctest
determines whether a thing is a function, module or class. Under 2.2,
this really requires the functions in inspect.py (e.g., types.ClassType
is close to meaningless now, if not outright misleading).
somewhat less horrid hack <wink>: if a module does
from __future__ import X
then the module dict D is left in a state such that (viewing X as a
string)
D[X] is getattr(__future__, X)
So by examining D for all the names of future features, and making that
test for each, we can make a darned good guess as to which future-features
were imported by the module. The appropriate flags are then sucked out
of the __future__ module, and passed on to compile()'s new optional
arguments (PEP 264).
Also gave doctest a meaningful __all__, removed the history of changes
(CVS serves that purpose now), and removed the __version__ vrbl (similarly;
before CVS, it was a reasonable clue, but not anymore).
that 'yield' is a keyword. This doesn't help test_generators at all! I
don't know why not. These things do work now (and didn't before this
patch):
1. "from __future__ import generators" now works in a native shell.
2. Similarly "python -i xxx.py" now has generators enabled in the
shell if xxx.py had them enabled.
3. This program (which was my doctest proxy) works fine:
from __future__ import generators
source = """\
def f():
yield 1
"""
exec compile(source, "", "single") in globals()
print type(f())
break cycles, which are a special problem when running generator tests
that provoke exceptions by invoking the .next() method of a named
generator-iterator: then the iterator is named in globs, and the
iterator's frame gets a tracekback object pointing back to globs, and
gc doesn't chase these types so the cycle leaks.
Also changed _run_examples() to make a copy of globs itself, so its
callers (direct and indirect) don't have to (and changed the callers
to stop making their own copies); *that* much is a change I've been
meaning to make for a long time (it's more robust the new way).
Here's a way to provoke the symptom without doctest; it leaks at a
prodigious rate; if the last two "source" lines are replaced with
g().next()
the iterator isn't named and then there's no leak:
source = """\
def g():
yield 1/0
k = g()
k.next()
"""
code = compile(source, "<source>", "exec")
def f(globs):
try:
exec code in globs
except ZeroDivisionError:
pass
while 1:
f(globals().copy())
After this change, running test_generators in an infinite loop still leaks,
but reduced from a flood to a trickle.
This makes verbose-mode output easier to dig thru, and removes an accidental
dependence on the order of dict.items() (made visible by recent changes to
dictobject.c).
newline at the end if there isn't one already. Expected output has no
way to indicate that a trailing newline was not expected, and in the
interpreter shell *Python* supplies the trailing newline before printing
the next prompt.
Jim Fulton
Edward Loper
Tim Peters
Martin v. Löwis
Walter Dörwald
Raymond Hettinger
Guido van Rossum
Eric S. Raymond
Skip Montanaro