Lesson learned: kids should not be allowed to use API's starting
with an underscore :-/
zipimport in 2.3a1 is even more broken than I thought: I attemped
to _PyString_Resize a string created by PyString_FromStringAndSize,
which fails for strings with length 0 or 1 since the latter returns
an interned string in those cases. This would cause a SystemError
with empty source files (and no matching pyc) in the zip archive.
I rewrote the offending code to simply allocate a new buffer and
avoid _PyString_Resize altogether.
Added a test that would've caught the problem.
(or None) now. In 2.3a1 they could also return an int or long, but that
was an unhelpfully redundant leftover from an earlier version wherein
they couldn't return a timedelta. TOOWTDI.
the test set as it only tested with a zip archive in the current directory,
but it doesn't work at all for packages when the zip archive was specified
as an absolute path. It's a real embarrassing bug: a strchr call should
have been strrchr; fever apparently implies dyslexia.
Second stupid bug: the zipimport test failed with a name error
__importer__ (which I had renamed to __loader__ everywhere but here).
I would've sworn I ran the test after that change but that can't be true.
What I don't understand that noone reported a failing test_zipimport.py
before the release of 2.3a1.
suggestion from Guido, along with a formal correctness proof of the
trickiest bit. The intricacy of the proof reveals how delicate this
is, but also how robust the conclusion: correctness doesn't rely on
dst() returning +- one hour (not all real time zones do!), it only
relies on:
1. That dst() returns a (any) non-zero value if and only if daylight
time is in effect.
and
2. That the tzinfo subclass implements a consistent notion of time zone.
The meaning of "consistent" was a hidden assumption, which is now an
explicit requirement in the docs. Alas, it's an unverifiable (by the
datetime implementation) requirement, but so it goes.
understood now: it can't work. Added comments explaining why (it's "the
usual"-- unrepresentable hours in local time --but in a slightly different
guise).
an idea from Guido. This restores that the datetime implementation
never passes a datetime d to a tzinfo method unless d.tzinfo is the
tzinfo instance whose method is being called. That in turn allows
enormous simplifications in user-written tzinfo classes (see the Python
sandbox US.py and EU.py for fully fleshed-out examples).
d.astimezone(tz) also raises ValueError now if d lands in the one hour
of the year that can't be expressed in tz (this can happen iff tz models
both standard and daylight time). That it used to return a nonsense
result always ate at me, and it turned out that it seemed impossible to
force a consistent nonsense result under the new implementation (which
doesn't know anything about how tzinfo classes implement their methods --
it can only infer properties indirectly). Guido doesn't like this --
expect it to change.
New tests of conversion between adjacent DST-aware timezones don't pass
yet, and are commented out.
Running the datetime tests in a loop under a debug build leaks 9
references per test run, but I don't believe the datetime code is the
cause (it didn't leak the last time I changed the C code, and the leak
is the same if I disable all the tests that invoke the only function
that changed here). I'll pursue that next.
- new import hooks in import.c, exposed in the sys module
- new module called 'zipimport'
- various changes to allow bootstrapping from zip files
I hope I didn't break the Windows build (or anything else for that
matter), but then again, it's been sitting on sf long enough...
Regarding the latest discussions on python-dev: zipimport sets
pkg.__path__ as specified in PEP 273, and likewise, sys.path item such as
/path/to/Archive.zip/subdir/ are supported again.
of the timetz case. A tzinfo method will always see a datetimetz arg,
or None, now. In the former case, it's still possible that it will get
a datetimetz argument belonging to a different timezone. That will get
fixed next.
ignore tuple.
The line, "from _random import Random as CoreGenerator", fools the test
code which expects CoreGenerator.__name__ to be "CoreGenerator" instead
of "Random".
Guido has in mind an easier way for users to code this stuff, but the
only tests we have now are for fixed-offset tzinfo classes, and this
stuff is extremely delicate in the endcases (read the new test code
for why: there are holes in time <wink>).
operands have identical tzinfo members (meaning object identity -- "is").
I misunderstood the intent here, reading wrong conclusion into
conflicting clues.
subtraction, work as documented. In the Python implementation,
they weren't calling utcoffset() if both operands had the same
tzinfo object. That's fine if it so happens that the shared
tzinfo object returns a fixed offset (independent of operand),
but can give wrong results if that's not so, and the latter
obtains in a tzinfo subclass instance trying to model both
standard and daylight times. The C implementation was already
doing this "correctly", so we're just adding tests to verify it.
be trusted with years before 1900, so now we raise ValueError if a date or
datetime or datetimetz .strftime() method is called with a year before
1900.
{timetz,datetimetz}.{utcoffset,dst}() now return a timedelta (or None)
instead of an int (or None).
tzinfo.{utcoffset,dst)() can now return a timedelta (or an int, or None).
Curiously, this was much easier to do in the C implementation than in the
Python implementation (which lives in the Zope3 code tree) -- the C code
already had lots of hair to extract C ints from offset objects, and used
C ints internally.
used that.
wrap_strftime(): Removed the most irritating uses of buf.
TestDate.test_ordinal_conversions(): The C implementation is fast enough
that we can afford to check the endpoints of every year. Also added
tm_yday tests at the endpoints.
[ 643835 ] Set Next Statement for Python debuggers
with a few tweaks by me: adding an unsigned or two, mentioning that
not all jumps are allowed in the doc for pdb, adding a NEWS item and
a note to whatsnew, and AuCTeX doing something cosmetic to libpdb.tex.
containing class objects) are allowed as the second argument.
This makes issubclass() more similar to isinstance() where recursive
tuples are allowed too.
test_resource calls resource.setrlimit() to change the file size limits.
This fails on Cygwin, which supports setrlimit() and getrlimit(), just not
changing that particular setting. (The same would apply to any other
platform that has those functions but not that particular feature.)
Since getrlimit() works and setrlimit() can be used for other reasons, a
check for ValueError was added to that part of the test.
supported as the second argument. This has the same meaning as
for isinstance(), i.e. issubclass(X, (A, B)) is equivalent
to issubclass(X, A) or issubclass(X, B). Compared to isinstance(),
this patch does not search the tuple recursively for classes, i.e.
any entry in the tuple that is not a class, will result in a
TypeError.
This closes SF patch #649608.
Most of these patches are from Thomas Heller, with long lines folded
by Tim. The change to test_descr.py is from Guido. See the bug report.
Not a bugfix candidate -- METH_CLASS is new in 2.3.
this can result in significantly smaller files. All classes as well as the
open function now accept an optional binary parameter, which defaults to
False for backward compatibility. Added a small test suite, updated the
libref documentation (including documenting the exported classes and fixing
a few other nits) and added a note about the change to Misc/NEWS.
Although motived by Cygwin, this patch will prevent
test_commands from failing on Unixes that support
ACLs. For example, the following is an excerpt from
the Solaris ls manpage:
...
-rwxrwxrwx+ 1 smith dev 10876 May 16 9:42 part2
The plus sign indicates that there is an ACL associated
with the file.
...
This patch updates regrtest.py to understand which
tests are normally skipped under Cygwin. The list of
tests was verified with the Cygwin Python maintainer.
Just van Rossum showed a weird, but clever way for pure python code to
trigger the BadInternalCall. The C code had assumed that calling a class
constructor would return an instance of that class; however, classes that
abuse __new__ can invalidate that assumption.
real module, by filtering out aliased methods. This, combined with
the recent fixes to pyclbr, make it possible to enable more tests with
fewer exceptions.
- The _modules cache now uses the full module name.
- The meaning of the (internal!!!) inpackage argument is changed: it
now is the parent package name, or None. readmodule() doesn't
support this argument any more.
- The meaning of the path argument is changed: when inpackage is set,
the module *must* be found in this path (as is the case for the real
package search).
- Miscellaneous cleanup, e.g. fixed __all__, changed some comments and
doc strings, etc.
- Adapted the unit tests to the new semantics (nothing much changed,
really). Added some debugging code to the unit tests that print
helpful extra info to stderr when a test fails (interpreting the
test failures turned out to be hard without these).
see problems with my code that I didn't see before the checkin, but:
When a subtype .mro() fails, we need to reset the type whose __bases__
are being changed, too. Fix + test.
Constructor accepts optional keyword arguments after a optional items list.
Add fromkeys() as an alternate constructor from an iterable over keys.
Expand related unittests.
[ 635933 ] make some type attrs writable
Plus a couple of extra tests beyond what's up there.
It hasn't been as carefully reviewed as it perhaps should, so all readers
are encouraged, nay exhorted, to give this a close reading.
There are still a couple of oddities related to assigning to __name__,
but I intend to solicit python-dev's opinions on these.
The bsddb subproject is gone.
The _bsddb subproject is new.
There are problems here, but I'm out of time to work on this now. If
anyone can address an XXX comment or two in readme.txt, please do!
Already supported dict() and dict(mapping).
Now supports dict(itemsequence) and
Just van Rossum's new syntax for dict(keywordargs).
Also, added related unittests.
The docs already promise dict-like behavior
so no update is needed there.
Replaced docstring with comments. Prevents subclass contamination.
Added the missing __cmp__() method and a test for __cmp__().
Used try/except style in preference to has_key() followed by a look-up.
Used iteritem() where possible to save creating a long key list and
to save redundant lookups.
Expanded .update() to look for the most helpful methods first and gradually
work down to a mininum expected interface.
Expanded documentation to be more clear on how to use the class.
When mwh added extended slicing, strings and unicode became mappings.
Thus, dict was set which prevented an error when doing:
newstr = 'format without a percent' % string_value
This fix raises an exception again when there are no formats
and % with a string value.
Armin Rigo's Draconian but effective fix for
SF bug 453523: list.sort crasher
slightly fiddled to catch more cases of list mutation. The dreaded
internal "immutable list type" is gone! OTOH, if you look at a list
*while* it's being sorted now, it will appear to be empty. Better
than a core dump.
figure out what the code was doing. The fixes were a combination of
closing open files before deletion, opening files in binary mode, and
plain skipping things that can't work on Windows (BaseTest.decompress
uses a process gimmick that doesn't exist on Windows, and, even if it
did, assumes a "bunzip2" executable is on PATH).
The _update method detected mutable elements by trapping TypeErrors.
Unfortunately, this masked useful TypeErrors raised by the iterable
itself. For cases where it is possible for an iterable to raise
a TypeError, the iterable is pre-converted to a list outside the
try/except so that any TypeErrors propagate through.
from Greg Chapman.
* Modules/_sre.c
(lastmark_restore): New function, implementing algorithm to restore
a state to a given lastmark. In addition to the similar algorithm used
in a few places of SRE_MATCH, restore lastindex when restoring lastmark.
(SRE_MATCH): Replace lastmark inline restoring by lastmark_restore(),
function. Also include it where missing. In SRE_OP_MARK, set lastindex
only if i > lastmark.
* Lib/test/re_tests.py
* Lib/test/test_sre.py
Included regression tests for the fixed bugs.
* Misc/NEWS
Mention fixes.
/* this is harder to get right than you might think */
angered some God somewhere. After noticing
>>> range(5000000)[slice(96360, None, 439)]
[]
I found that my cute test for the slice being empty failed due to
overflow. Fixed, and added simple test (not the above!).
bug #596434. (Alas, I don't think this completely covers that bug.)
Remove 'wrapper' argument from BaseTestCase.check_split() -- it's not
actually needed.
Since properties are supported here, is possible that
instance_getattr2() raises an exception. Fix all code that made this
assumption.
Backport candidate.
classes was called with three arguments. This makes no sense, there's
no way to pass in the "modulo" 3rd argument as for __pow__, and
classic classes don't do this. [SF bug 620179]
I don't want to backport this to 2.2.2, because it could break
existing code that has developed a work-around. Code in 2.2.2 that
wants to use __ipow__ and wants to be forward compatible with 2.3
should be written like this:
def __ipow__(self, exponent, modulo=None):
...
initializing GNU readline, setlocale(LC_CTYPE, "") is called, which
changes the <ctype.h> macros to use the "default" locale (which isn't
the *initial* locale -- the initial locale is the "C" locale in which
only ASCII characters are printable). When the default locale is e.g.
Latin-1, the repr() of string objects can include 8-bit characters
with the high bit set; I believe this is due to the recent
PRINT_MULTIBYTE_STRING changes to stringobject.c. This in turn screws
up test_pyexpat and test_rotor, which depend on the repr() of 8-bit
strings with high bit characters.
The solution (for now) is to force the LC_CTYPE locale to "C" after
importing rlcompleter. This is the locale required by the test suite
anyway.
ths "should be" skipped depends on os.path.supports_unicode_filenames,
not really on the platform. Fiddled the expected-skip constructor
appropriately.
list(xrange(sys.maxint / 4))
test. Changed 4 to 2.
The belief is that this test intended to trigger a bit of code in
listobject.c's NRESIZE macro that's looking for arithmetic overflow. As
written, it doesn't achieve that, though, and leaves it up to the platform
realloc() as to whether it wants to allocate 2 gigabytes. Some platforms
say "sure!", although they don't appear to mean it, and disaster ensues.
Changing 4 to 2 (just barely) manages to trigger the arithmetic overflow
test instead, leaving the platform realloc() out of it.
I'll backport this to the 2.2 branch next.
sys.getwindowsversion() on Windows (new enahanced Tim-proof <wink>
version), and fix test_pep277.py in a few minor ways.
Including doc and NEWS entries.
patch #617312, both on the trunk and the 22-maint branch.
Also added a test case, and ported the test_trace I wrote for HEAD
to 2.2.2 (with all those horrible extra 'line' events ;-).
A possibility to deadlock (on the hidden import lock) was created here
in 2.3, seemingly when tempfile.py started to call functions in
random.py. The cure is "the usual": don't spawn threads as a side
effect of importing, when the spawned threads themselves do imports
(directly or indirectly), and the code that spawned the threads is
waiting for the threads to finish (they can't finish, because they're
waiting for the import lock the spawner still holds). Worming around
this is why the "test_main" mechanism was introduced in regrest, so
it's a straightforward fix.
NOT a bugfix candidate; the problem was introduced in 2.3.
module used in the Zope TAL implementation. The bug was already fixed
in the Python standard library, but the regression test would be good
to keep around.
Because ob_size is a 32-bit int but sys.maxint is LONG_MAX which is a
64-bit value, there's no way to make this test succeed on a 64-bit
platform. So just skip it when sys.maxint isn't 0x7fffffff.
Backport candidate.
exception occurred so it should only be closed in the else clause.
Without this change we can an UnboundLocalError on Linux:
Traceback (most recent call last):
File "Lib/test/test_mmap.py", line 304, in ?
test_both()
File "Lib/test/test_mmap.py", line 208, in test_both
m.close()
UnboundLocalError: local variable 'm' referenced before assignment
with a size larger than the underlying file worked on Windows. It
does <wink>. However, merely creating an mmap that way has the side
effect of growing the file on disk to match the specified size. A
*later* test assumed that the file on disk was still exactly as it was
before the new "size too big" test was added, but that's no longer true.
So added a hack at the end of the "size too big" test to truncate the
disk file back to its original size on Windows.
Unicode strings (with arbitrary length) are allowed
as entries in the unicode.translate mapping.
Add a test case for multicharacter replacements.
(Multicharacter replacements were enabled by the
PEP 293 patch)
to fix it. (It fails when the day of the month is a 1-digit number,
because %c produces space+digit there, while strptime seems to expect
zero+digit somehow.)
of PyString_DecodeEscape(). This prevents a call to
_PyString_Resize() for the empty string, which would
result in a PyErr_BadInternalCall(), because the
empty string has more than one reference.
This closes SF bug http://www.python.org/sf/603937
Use a slightly different strategy to determine when not to call the line
trace function. This removes the need for the RETURN_NONE opcode, so
that's gone again. Update docs and comments to match.
Thanks to Neal and Armin!
Also add a test suite. This should have come with the original patch...
localtime, which in -0400 is 12 noon GMT. The bug boiled down to
broken conversion of 12 PM to hour 12 for the '%I %p' format string.
Added a test for this specific condition: Strptime12AMPMTests. Fix to
_strptime.py coming momentarily.
the tokenize module by test_tokenize.py. The FutureWarnings only
appeared during installation, and I've figured out a way to suppress
those in a different way.
sure these are the best fixes.
- Test maxint-1 against the negative octal constant -020000000000
- Comment out the tests for oct -1 and hex -1, since 037777777777 and
0xffffffff raise FutureWarnings now and in Python 2.4 those
constants will produce positive values, not negative values. So the
existing test seems to test something that won't be true in 2.4.
underlying dictionaries, there were no reasonable use cases (lexicographic
sorting of a list of sets is somewhat esoteric). Frees the operators
for other uses (such as strict subset and superset comparisons).
Updated documentation and test suite accordingly.
the inplace operators. The strategy is to have the operator overloading
code do the work and then to define equivalent method calls which rely on
the operators. The changes facilitate proper application of TypeError
and NonImplementedErrors.
Added corresponding tests to the test suite to make sure both the operator
and method call versions get exercised.
Add missing tests for difference_update().
wrong thing for a unicode subclass when there were zero string
replacements. The example given in the SF bug report was only one way
to trigger this; replacing a string of length >= 2 that's not found is
another. The code would actually write outside allocated memory if
replacement string was longer than the search string.
(I wonder how many more of these are lurking? The unicode code base
is full of wonders.)
Bugfix candidate; this same bug is present in 2.2.1.
base class (WrapperTestCase) instead, and call it repeatedly in the
methods that used to have a loop-over-subcases. Much simpler.
Rename perennial temp variable 't' to 'text'.
<peter@engcorp.com> based on a test script that's been kicking around my
home directory for a couple of months now and only saw the light of day
because I included it when I sent textwrap.py to python-dev for review.
Also, don't call gettempdir() in the default expression for the 'dir'
argument to various functions; use 'dir=None' for the default and
insert 'if dir is None: dir = gettemptir()' in the bodies. That way
the work done by gettempdir is postponed until needed.
[ 587993 ] SET_LINENO killer
Remove SET_LINENO. Tracing is now supported by inspecting co_lnotab.
Many sundry changes to document and adapt to this change.
binary=True
to
text=False
by BDFL Pronouncement. All other changes follow from this. The change
to the docs is ready to go, but blocked by another JackMacLock in the
doc directory.
do bizarre things to get a temp file, I changed it to use mkstemp instead
of NamedTemporaryFile. This tried to leave the file open while passing
its name to execfile(). On Win2K (but not Win9X), though, a file created
with O_TEMPORARY cannot be opened again, so the test failed with a
permission error when execfile tried to open it. Closer to the truth:
a file created with O_TEMPORARY can be opened again, but only if the
file is also created with SHARE_DELETE access via the Win32 CreateFile()
function. There's no way to get at that from MS's version of libc, though
(we'd have to ditch the "std" C file functions in favor of Win32 API
calls).
Close the bug report again -- this time for Cygwin due to a newlib bug.
See the following for the details:
http://sources.redhat.com/ml/newlib/2002/msg00369.html
Note that this commit is only a documentation (i.e., comment) change.
test was written. So boosted the number of "digits" this generates, and
also beefed up the "* / divmod" test to tickle numbers big enough to
trigger the Karatsuba algorithm. It takes about 2 seconds now on my box.
rewrite, by Zack Weinberg). This replaces most code in tempfile.py
(please review!!!) and adds extensive unit tests for it.
This will cause some warnings in the test suite; I'll check those in
soon, and also the docs.
least on OS/2 (see note on SF patch 555085 by A I MacIntyre) but
looks like the test *could* fail on any other platform too -- there's
no guarantee that recv() reads all data.
prints function and module names, which is more informative now that
we repeat some tests in slightly modified subclasses.
Add a test for read() until EOF.
Add test suites for line-buffered (bufsize==1) and a small custom
buffer size (bufsize==2).
Restructure testUnbufferedRead() somewhat to avoid a potentially
infinite loop.
and this broke a Zope "pipelining" test which read multiple responses
from the same connection (this attaches a new file object to the
socket for each response). Added a test for this too.
(I want to do some code cleanup too, but I thought I'd first fix
the problem with as little code as possible, and add a unit test
for this case. So that's what this checkin is about.)
on Windows. The test_sequence() ERROR is easily repaired if we're
willing to add an os.unlink() line to mhlib's updateline(). The
test_listfolders FAIL I gave up on -- I don't remember enough about Unix
link esoterica to recall why a link count of 2 is something a well-
written program should be keenly interested in <wink>.
Added new heapify() function, which transforms an arbitrary list into a
heap in linear time; that's a fundamental tool for using heaps in real
life <wink>.
Added heapyify() test. Added a "less naive" N-best algorithm to the test
suite, and noted that this could actually go much faster (building on
heapify()) if we had max-heaps instead of min-heaps (the iterative method
is appropriate when all the data isn't known in advance, but when it is
known in advance the tradeoffs get murkier).
at random, and replaces the elements at those positions with new random
values. I was pleasantly surprised by how fast this goes! It's hard to
conceive of an algorithm that could special-case for this effectively.
Plus it's exactly what happens if a burst of gamma rays corrupts your
sorted database on disk <wink>.
i 2**i *sort ... %sort
15 32768 0.18 ... 0.03
16 65536 0.24 ... 0.04
17 131072 0.53 ... 0.08
18 262144 1.17 ... 0.16
19 524288 2.56 ... 0.35
20 1048576 5.54 ... 0.77
and age of rampant computer breakins I imagine there are plenty of systems
with telnet disabled. Successful check of at least one getservbyname() call
is required for success
in the stability tests.
Bizarre: this takes 11x longer to run if and only if test_longexp is
run before it, on my box. The bigger REPS is in test_longexp, the
slower this gets. What happens on your box? It's not gc on my box
(which is good, because gc isn't a plausible candidate here).
The slowdown is massive in the parts of test_sort that implicitly
invoke a new-style class's __lt__ or __cmp__ methods. If I boost
REPS large enough in test_longexp, even the test_sort tests on an array
of size 64 visibly c-r-a-w-l. The relative slowdown is even worse in
a debug build. And if I reduce REPS in test_longexp, the slowdown in
test_sort goes away.
test_longexp does do horrid things to Win98's management of user
address space, but I thought I had made that a whole lot better a month
or so ago (by overallocating aggressively in the parser).
Just van Rossum
Tim Peters
Skip Montanaro
Neal Norwitz
Andrew MacIntyre
Martin v. Löwis
Fred Drake
Jack Jansen
Guido van Rossum
Barry Warsaw
Neil Schemenauer
Marc-André Lemburg
Raymond Hettinger
Andrew M. Kuchling
Michael W. Hudson
Walter Dörwald
Jason Tishler
Greg Ward
Ka-Ping Yee
Jeremy Hylton
Gustavo Niemeyer
Finn Bock
Mark Hammond
Piers Lauder