The shutils.rmtree() implementation uses an excessive amount of memory when
deleting large directory hierarchies. Before actually deleting any files, it
builds up a list of (function, filename) tuples for all the files that it is
going to remove.
Briefly (from the NEWS file):
- Updates for the email package:
+ All deprecated APIs that in email 2.x issued warnings have been removed:
_encoder argument to the MIMEText constructor, Message.add_payload(),
Utils.dump_address_pair(), Utils.decode(), Utils.encode()
+ New deprecations: Generator.__call__(), Message.get_type(),
Message.get_main_type(), Message.get_subtype(), the 'strict' argument to
the Parser constructor. These will be removed in email 3.1.
+ Support for Python earlier than 2.3 has been removed (see PEP 291).
+ All defect classes have been renamed to end in 'Defect'.
+ Some FeedParser fixes; also a MultipartInvariantViolationDefect will be
added to messages that claim to be multipart but really aren't.
+ Updates to documentation.
deque_item(): a performance bug: the linked list of blocks was followed
from the left in most cases, because the test (i < (deque->len >> 1)) was
after "i %= BLOCKLEN".
deque_clear(): replaced a call to deque_len() with deque->len; not sure what
this call was here for, nor if all compilers under the sun would inline it.
deque_traverse(): I belive that it could be called by the GC when the deque
has leftblock==rightblock==NULL, because it is tracked before the first block
is allocated (though closely before). Still, a C extension module subclassing
deque could provide its own tp_alloc that could trigger a GC collection after
the PyObject_GC_Track()...
deque_richcompare(): rewrote to cleanly check for end-of-iterations instead of
relying on deque.__iter__().next() to succeed exactly len(deque) times -- an
assumption which can break if deques are subclassed. Added a test.
I wonder if the length should be explicitely bounded to INT_MAX, with
OverflowErrors, as in listobject.c. On 64-bit machines, adding more than
INT_MAX in the deque will result in trouble. (Note to anyone/me fixing
this: carefully check for overflows if len is close to INT_MAX in the
following functions: deque_rotate(), deque_item(), deque_ass_item())
request. Tim says that "correct 'fuzzy' comparison of floats cannot
be automated." (The motivation behind adding the new option
was verifying interactive examples in Python's latex documentation;
several such examples use numbers that don't print consistently on
different platforms.)
Also, add a testcase.
Formerly, the list_extend() code used several local variables to remember
its state across iterations. Since an iteration could call arbitrary
Python code, it was possible for the list state to be changed. The new
code uses dynamic structure references instead of C locals. So, they
are always up-to-date.
After list_resize() is called, its size has been updated but the new
cells are filled with NULLs. These needed to be filled before arbitrary
iteration code was called; otherwise, that code could attempt to modify
a list that was in a semi-invalid state. The solution was to change
the ob->size field back to a value reflecting the actual number of valid
cells.
its documentation.
* Documented that the compiled re methods are supposed to be more full
featured than their simpilified function counterparts.
* Documented the existing start and stop position arguments for the
findall() and finditer() methods of compiled regular expression objects.
* Added an optional flags argument to the re.findall() and re.finditer()
functions. This aligns their API with that for re.search() and
re.match().
When an integer is compared to a float now, the int isn't coerced to float.
This avoids spurious overflow exceptions and insane results. This should
compute correct results, without raising spurious exceptions, in all cases
now -- although I expect that what happens when an int/long is compared to
a NaN is still a platform accident.
Note that we had potential problems here even with "short" ints, on boxes
where sizeof(long)==8. There's #ifdef'ed code here to handle that, but
I can't test it as intended. I tested it by changing the #ifdef to
trigger on my 32-bit box instead.
I suppose this is a bugfix candidate, but I won't backport it. It's
long-winded (for speed) and messy (because the problem is messy). Note
that this also depends on a previous 2.4 patch that introduced
_Py_SwappedOp[] as an extern.
displaying a set of classes from one module it doesn't matter, but if you
are displaying a large class tree from multiple modules it improves the
display to sort by module.name.
all examples in a given text file. (analagous to "testmod")
- Minor docstring fixes.
- Added module_relative parameter to DocTestFile/DocTestSuite, which
controls whether paths are module-relative & os-independent, or
os-specific.
(Contributed by Nick Coghlan.)
Various code cleanups and optimizations (saves about 40% on testsuite
execution time and on the telco benchmark).
* caches results of various operations on self (esp. checks for being
a special value).
* _WorkRep now uses ints and longs for intermediate computations.
- Fixed bug in handling of absolute paths.
- If run from an interactive session, make paths relative to the
directory containing sys.argv[0] (since __main__ doesn't have
a __file__ attribute).
of the Template.delimiter attribute, we make use of the delimiter in the
escaped group, and in the safe_substitute() method more robust.
Now, .delimiter should be the unescaped delimiter literal, e.g. '$' or '&', or
whatever. The _TemplateMetaclass will re.escape() this value when it builds
the pattern.
* The parameterization of "delimiter" was incomplete.
* safe_substitute's code for braced delimiters should only be executed
when braced is not None.
* Invalid pattern group names now raise a ValueError. Formerly, the
convert code would fall off the end and improperly return None.
Beefed-up tests.
* Test delimiter override for all paths in substitute and safe_substitute.
* Alter unittest invocation to match other modules (now it itemizes the
tests as they are run).
Renamed the new generator at Trevor's recommendation.
The name HardwareRandom suggested a bit more than it
delivered (no radioactive decay detectors or such).
with default False for testmod(). The real point of introducing this was
so that output from doctest.master.summarize() would be the same as in
2.3, and doctest.master in 2.4 is a backward-compatability hack used only
by testmod().
- Template no longer inherits from unicode.
- SafeTemplate is removed. Now Templates have both a substitute() and a
safe_substitute() method, so we don't need separate classes. No more
__mod__() operator.
- Adopt Tim Peter's idea for giving Template a metaclass, which makes the
delimiter, the identifier pattern, or the entire pattern easy to override
and document, while retaining efficiency of class-time compilation of the
regexp.
- More informative ValueError messages which will help a user narrow down the
bogus delimiter to the line and column in the original string (helpful for
long triple quoted strings).
decoding incomplete input (when the input stream is temporarily exhausted).
codecs.StreamReader now implements buffering, which enables proper
readline support for the UTF-16 decoders. codecs.StreamReader.read()
has a new argument chars which specifies the number of characters to
return. codecs.StreamReader.readline() and codecs.StreamReader.readlines()
have a new argument keepends. Trailing "\n"s will be stripped from the lines
if keepends is false. Added C APIs PyUnicode_DecodeUTF8Stateful and
PyUnicode_DecodeUTF16Stateful.
SF patch #1015989
The basic idea of this patch is to compute lineno attributes for all AST nodes. The actual
implementation lead to a lot of restructing and code cleanup.
The generated AST nodes now have an optional lineno argument to constructor. Remove the
top-level asList(), since it didn't seem to serve any purpose. Add an __iter__ to ast nodes.
Use isinstance() instead of explicit type tests.
Change transformer to use the new lineno attribute, which replaces three lines of code with one.
Use universal newlines so that we can get rid of special-case code for line endings. Use
lookup_node() in a few more frequently called, but simple com_xxx methods(). Change string
exception to class exception.
Several functions adopted the strategy of altering a full lengthed
string copy and resizing afterwards. That would fail if the initial
string was short enough (0 or 1) to be interned. Interning precluded
the subsequent resizing operation.
The solution was to make sure the initial string was at least two
characters long.
Added tests to verify that all binascii functions do not crater when
given an empty string argument.
"all or none" to "all or some".
This provides much greater test coverage without eating much time.
It also makes it more likely that routine regression testing will
unearth bugs.
in the new docs.
DocTestRunner.__run: Separate the determination of the example outcome
from reporting that outcome, to squash brittle code duplication and
excessive nesting.
ldexp. Both methods are exact, and return the same results. Turns out
multiplication is a few (but just a few) percent faster on my box.
They're both significantly faster than using struct with a Q format
to convert bytes to a 64-bit long (struct.unpack() appears to lose due
to the tuple creation/teardown overhead), and calling _hexlify is
significantly faster than doing bytes.encode('hex'). So we appear to
have hit a local minimum (wrt speed) here.
components without division and without roundoff error for properly
sized mantissas (i.e. on systems with 53 or more mantissa bits per
float). Eliminates the previous implementation's rounding bias as
aptly demonstrated by Tim Peters.
to unittest, so make it official: new module constants COMPARISON_FLAGS
and REPORTING_FLAGS, which are bitmasks or'ing together the relevant
individual option flags.
set_unittest_reportflags(): Reworked to use REPORTING_FLAGS, and
simplified overly complicated flag logic.
class FakeModule: Removed this; neither documented nor used.
1) When a breakpoint is set via a function name:
- the breakpoint gets the lineno of the def statement
- a new funcname attribute is attached to the breakpoint
2) bdb.effective() calls new function checkfuncname() to handle:
- def statement is executed: don't break.
- a first executable line of a function with a breakpoint on the lineno of the
def statement is reached: break.
This fixes bugs 976878, 926369 and 875404. Thanks Ilya Sandler.
this module imports itself explicitly from test (so the "file names"
current doctest synthesizes for examples don't vary depending on how
test_generators is run).
s.join([t]) is t
for (s, t) in (str, str), (unicode, unicode), and (str, unicode).
For (unicode, str), verify that it's *not* t (the result is promoted
to unicode instead). Also verify that when t is a subclass of str or
unicode that "the right thing" happens.
- Improvements to interactive debugging support:
- Changed the replacement pdb.set_trace to redirect stdout to the
real stdout *only* during interactive debugging; stdout from code
continues to go to the fake stdout.
- When the interactive debugger gets to the end of an example,
automatically continue.
- Use a replacement linecache.getlines that will return source lines
from doctest examples; this makes the source available to the
debugger for interactive debugging.
- In test_doctest, use a specialized _FakeOutput class instead of a
temporary file to fake stdin for the interactive interpreter.
and intervening text strings.
- Removed DocTestParser.get_program(): use script_from_examples()
instead.
- Fixed bug in DocTestParser._INDENT_RE
- Fixed bug in DocTestParser._min_indent
- Moved _want_comment() to the utility function section
actual output into lines created spurious empty lines at the ends of
each. Those matched, but the fancy diffs had surprising line counts (1
larger than expected), and tests kept having to slam <BLANKLINE> into the
expected output to account for this. Using the splitlines() string method
with keepends=True instead accomplishes what was intended directly.
While a fancy diff can be confusing in the presence of ellipses, so far
I'm finding (2-0-0) that it's much more a major aid in narrowing down the
possibilities when an ellipsis-slinging test fails. So we no longer
refuse to do a fancy diff just because of ellipses.
This isn't ideal; it's just better.
rather than an expected output string. This gives the
output_difference method access to more information, such as the
indentation of the example, which might be useful.
NDIFF_DIFF->REPORT_NDIFF. This establishes the naming convention that
all reporting options should begin with "REPORT_" (since reporting
options are a different class from output comparison options; but they
are both set in optionflags).
to be more consistent with report_failure()
- If `want` or `got` is empty, then print "Expected nothing\n" or
"Got nothing\n" rather than "Expected:\n" or "Got:\n"
- Got rid of _tag_msg
* Add comment bars segregating this code from the rest.
* Improve readability of the re pattern with indentation and comments on
the same line.
* Replace the groupdict() and get() pair with a direct call to group()
which does the same thing.
exception message, or None if no exception is expected); and moved
exception parsing from DocTestRunner to DocTestParser. This is
architecturally cleaner, since it moves all parsing work to
DocTestParser; and it should make it easier for code outside
DocTestRunner (notably debugging code) to properly handle expected
exceptions.
a traceback message. I.e., examples that raise exceptions may no
longer generate pre-exception output. This restores the behavior of
doctest in python 2.3. The ability to check pre-exception output is
being removed because it makes the documentation simpler; and because
there are very few use cases for it.
This patch includes test cases and documentation updates, as well as NEWS file
updates.
This patch also updates the sre modules so that they don't import the string
module, breaking direct circular imports.
happen in 2.3, but nobody noticed it still was getting generated (the
warning was disabled by default). OverflowWarning and
PyExc_OverflowWarning should be removed for 2.5, and left notes all over
saying so.
* Make a pass to eliminate NOPs. Produce code that is more readable,
more compact, and a tiny bit faster. Makes the peepholer more flexible
in the scope of allowable transformations.
* With Guido's okay, bumped up the magic number so that this patch gets
widely exercised before the alpha goes out.
(Patch contributed by Nick Coghlan.)
Now joining string subtypes will always return a string.
Formerly, if there were only one item, it was returned unchanged.
NamedTemporaryFile (which can't work for this function's purposes on
Windows). Leaving temp files behind wasn't a great idea either, though,
so try to clean up. At least the test suite no longer leaves any of
these guys behind now.
docstrings toward being a lot shorter, and telling the whole truth in
the manual instead. This change is an example: the manual has detailed
explanations of the option names now, so it's Bad to repeat them in
the docstring (two detailed descriptions are certain to get out of synch).
Just listing the names has memory-jogging benefits, though, so that's
still helpful in the docstring.
- Test filenames sometimes had trailing .pyc or .pyo sufixes
(when module __file__ did).
- Trailing spaces spaces in expected output were dropped.
New default failure format:
- Separation of examples from file info makes examples easier to see
- More vertical separation, improving readability
- Emacs-recognized file info (also closer to Python exception format)
a new file, don't raise a dialog. IDLEfork 954928.
2. Refactor EditorWindow.wakeup() to WindowList.ListedToplevel.wakeup() and
clarify that the Toplevel of an EditorWindow is a WindowList.ListedToplevel.
3. Make a number of improvements to keyboard focus binding. Improve window
raising, especially in the debugger. IDLEfork Bug 763524 (GvR list).
4. Bump idlever to 1.1a3
M Debugger.py
M EditorWindow.py
M FileList.py
M NEWS.txt
M PyShell.py
M WindowList.py
M idlever.py
are non-obvious either way because the newline character "is invisible",
but it's still there all the same, and it's easier to explain/predict
if that reality is left alone.
'\037\014\n' (see http://quimby.gnus.org/notes/BABYL) so look for that as well,
so that applications won't get '\037' as the last line of the last message.
truncate() left the stream position unchanged, which meant the
"truncated" data didn't go away:
>>> io.write('abc')
>>> io.truncate(0)
>>> io.write('xyz')
>>> io.getvalue()
'abcxyz'
Patch by Dima Dorfman.
test_queue has failed occasionally for years, and there's more than one
cause.
The primary cause in the SF report appears to be that the test driver
really needs entirely different code for thread tests that expect to
raise exceptions than for thread tests that are testing non-exceptional
blocking semantics. So gave them entirely different code, and added a
ton of explanation.
Another cause is that the blocking thread tests relied in several places
on the difference between sleep(.1) and sleep(.2) being long enough for
the trigger thread to do its stuff sot that the blocking thread could make
progress. That's just not reliable on a loaded machine. Boosted the 0.2's
to 10.0's instead, which should be long enough under any non-catastrophic
system conditions. That doesn't make the test take longer to run, the 10.0
is just how long the blocking thread is *willing* to wait for the trigger
thread to do something. But if the Queue module is plain broken, such
tests will indeed take 10 seconds to fail now.
For similar (heavy load) reasons, changed threaded-test termination to
be willing to wait 10 seconds for the signal thread to end too.
* Check the found object for a None value during a contains/has_key
lookup. Perhaps it will help the OP who is likely suffering from an
occassional GC or threading object deletion after self.data is checked.
* Complete the previous patch by removing the unnecessary indirection
for weak dict iterators. Makes the code cleaner and more readable.
It's redundant, since no output is written anyway: DebugRunner doesn't
generate any output for failures and unexpected exceptions, and since
verbose=False, it won't generate any output for non-failures either.
appeared at the end of a line. Repaired that. Also noted that it's
too easy to provoke this implementation into requiring exponential
time, and especially when a test fails. I'll replace the implementation
with an always-efficient one later.
error based on decorating with staticmethod too soon for the code to execute.
This meant that if the test didn't pass it just errored out. Now if the test
doesn't pass it leads to a failure instead.
[ 1009560 ] Fix @decorator evaluation order
From the description:
Changes in this patch:
- Change Grammar/Grammar to require
newlines between adjacent decorators.
- Fix order of evaluation of decorators
in the C (compile.c) and python
(Lib/compiler/pycodegen.py) compilers
- Add better order of evaluation check
to test_decorators.py (test_eval_order)
- Update the decorator documentation in
the reference manual (improve description
of evaluation order and update syntax
description)
and the comment:
Used Brett's evaluation order (see
http://mail.python.org/pipermail/python-dev/2004-August/047835.html)
(I'm checking this in for Anthony who was having problems getting SF to
talk to him)
normalize() in Draft 1.06 (9 October 2002):
The normalize operation has been added; it reduces a number to a
canonical form. (This replaces the trim operator, which only
removed trailing fractional zeros.)
(version 1.45):
The max and min operations follow the rules in the current IEEE 754 revision draft:
if one operand is a quiet NaN and the other is number, then the number is always returned
if both operands are finite and equal in numerical value then an ordering is applied:
if the signs differ then max returns the operand with the positive sign and
min returns the operand with the negative sign; if the signs are the same then
the exponent is used to select the result.
version 2.39 of dectest.zip adds some new test files and because
some existing test files were getting skipped).
* Remove two docstrings which cluttered unittest's output.
* Simplify a for-loop with a list comprehension.
Johannes Gijsbers
Brett Cannon
Vinay Sajip
Tim Peters
David Goodger
Barry Warsaw
Just van Rossum
Armin Rigo
Raymond Hettinger
Walter Dörwald
Edward Loper
Neil Schemenauer
Phillip J. Eby
Skip Montanaro
Martin v. Löwis
Sean Reifschneider
Andrew M. Kuchling
Fred Drake
Jeremy Hylton
Gustavo Niemeyer
Anthony Baxter
Jim Fulton
Dave Cole
Mark Hammond
Kurt B. Kaiser
Hye-Shik Chang
Michael W. Hudson
Thomas Heller