The problem is in sre_compile.py: the call to
_compile_charset near the end of _compile_info forgets to
pass in the flags, so that the info charset is not compiled
with re.U. (The info charset is used when searching to find
the first character at which a match could start; it is not
generated for patterns beginning with a repeat like '\w{1}'.)
test_nonrecursive_deep(): Reduced nesting depth to 60.
Not a bugfix candidate. 2.3 increased the number of stack frames
needed to pickle a list (in order to get implement the "list
batching" unpickling memory optimization new in 2.3).
between str, unicode, UserString and the string module
as possible. This increases code coverage in stringobject.c
from 83% to 86% and should help keep the string classes
in sync in the future. From SF patch #662807
time.sleep(1) sometimes delays for fractionally less than a second
resulting in too short of an interval for C's time.time() function
to create a distinct seed.
prior to NT. EMX has a number of Posix emulation routines, including
geteuid() but lacks chown(), so silently skip trying to actually set
a file ownership when extracting a file from a tar archive.
There are some problems with this module, but the tool works for
simple tasks and no one else has volunteered a better code coverage
tool. Should cleanup and document before the beta release.
M PyShell.py
M ScriptBinding.py
M rpc.py
M run.py
Clean up the way IDLEfork handles termination of the subprocess, restore
ability to interrupt user code in Windows (so long as it's doing terminal
I/O).
1. Handle subprocess interrupts in Windows with an RPC message.
2. Run/F5 will restart the subprocess even if user code is running.
3. Restart the subprocess if the link is dropped.
4. Exit IDLE cleanly even during I/O.
4. In rpc.py, remove explicit calls to statelock, let the condition
variable handle acquire() and release().
Fix off-by-1 error in normalize_line_endings():
when *p == '\0' the NUL was copied into q and q was auto-incremented,
the loop was broken out of,
then a newline was appended followed by a NUL.
So the function, in effect, was strcpy() but added two extra chars
which was caught by obmalloc in debug mode, since there was only
room for 1 additional newline.
Get test working under regrtest (added test_main).
- 'os2' references in ntpath.py relate to the VACPP port, not the EMX port;
- the VACPP port uses the same defpath as all other ntpath.py supported
platforms except 'ce'.
the optional proto 2 slot state.
pickle.py, load_build(): CAUTION: Noted that cPickle's
load_build and pickle's load_build really don't do the same
things with the state, and didn't before this patch either.
cPickle never tries to do .update(), and has no backoff if
instance.__dict__ can't be retrieved. There are no tests
that can tell the difference, and part of what cPickle's
load_build() did looked accidental to me, so I don't know
what the true intent is here.
pickletester.py, test_pickle.py: Got rid of the hack for
exempting cPickle from running some of the proto 2 tests.
dictobject.c, PyDict_Next(): documented intended use.
test_linuxaudiodev.py) are no longer run by default. This is
because they don't always work, depending on your hardware and
software. To run these tests, you must use an invocation like
./python Lib/test/regrtest.py -u audio test_ossaudiodev
with an indented code block but no newline would raise SyntaxError.
This would have been a four-line change in parsetok.c... Except
codeop.py depends on this behavior, so a compilation flag had to be
invented that causes the tokenizer to revert to the old behavior;
this required extra changes to 2 .h files, 2 .c files, and 2 .py
files. (Fixes SF bug #501622.)
and loading them via the other, except for the special cases of this
Guido added to test_datetime.py for datetime module objects. The new
test_xpickle.py tries all of pickletester's AbstractPickleTests in
both x-module ways.
This changes the default __new__ to refuse arguments iff tp_init is the
default __init__ implementation -- thus making it a TypeError when you
try to pass arguments to a constructor if the class doesn't override at
least __init__ or __new__.
"Unsigned" (i.e., positive-looking, but really negative) hex/oct
constants with a leading minus sign are once again properly negated.
The micro-optimization for negated numeric constants did the wrong
thing for such hex/oct constants. The patch avoids the optimization
for all hex/oct constants.
This needs to be backported to Python 2.2!
- Catch stderr as well as stdout
- Fixed a bug with non-installable packages
- Parse .pth files after installing, so you don't have to restart Python (or
the IDE) after installing.
descr_check(); it wasn't useful. Change the type argument of the
various _get() methods to PyObject * because the call signature of
tp_descr_get doesn't guarantee its type.
object is not a real str or unicode but an instance
of a subclass, construct the output via looping
over __getitem__. This guarantees that the result
is the same for function==None and function==lambda x:x
This doesn't happen for tuples, because filtertuple()
uses PyTuple_GetItem().
(This was discussed on SF bug #665835).
* Removed the ifilter flag wart by splitting it into two simpler functions.
* Fixed comment tabbing in C code.
* Factored module start-up code into a loop.
Documentation:
* Re-wrote introduction.
* Addede examples for quantifiers.
* Simplified python equivalent for islice().
* Documented split of ifilter().
Sets.py:
* Replace old ifilter() usage with new.
tickle the 2.2.2 __cmp__ bug test_datetime used to tickle, so the
workarounds for that bug no longer make sense in the test suite (which I'm
still trying to keep as closely in synch as possible with Zope3's
version).
__ne__ no longer complain if they don't know how to compare to the other
thing. If no meaningful way to compare is known, saying "not equal" is
sensible. This allows things like
if adatetime in some_sequence:
and
somedict[adatetime] = whatever
to work as expected even if some_sequence contains non-datetime objects,
or somedict non-datetime keys, because they only call __eq__.
It still complains (raises TypeError) for mixed-type comparisons in
contexts that require a total ordering, such as list.sort(), use as a
key in a BTree-based data structure, and cmp().
the tests will remain in sync:
"""
Tres discovered a weird bug when a datetime is pickled, caused by the
shadowing of __year, __month, __day and the use of proxies.
Here's a quick fix and a quick unit test. I don't quite understand
why this wasn't caught by the pickling unit tests.
"""
atomically, but deepcopy() didn't support this at all.
I don't see any reason for this, so I'm adding ClassType
to the set of types that are deep-copied atomically.
eventProc (which simply drops all events on the floor). Also added a
method SetDefaultEventProc through which frameworks can set a global
event handler (which can still be overridden on a per-call basis
with the eventProc argument).
* Fixed typo in exception message for times()
* Filled in missing times_traverse()
* Document reasons that imap() did not adopt a None fill-in feature
* Document that count(sys.maxint) will wrap-around on overflow
* Add overflow test to islice()
* Check that starmap()'s argument returns a tuple
* Verify that imap()'s tuple re-use is safe
* Make a similar tuple re-use (with safety check) for izip()
Reverting one of those irritating "security fixes". fdopen() opens
files in binary mode. That makes pydoc skip the \r\n on Windows that's
need to make the output readable in the shell. Screw it.
Simply replace all uses of statcache with os.stat.
Should I add a DeprecationWarning triggered if the use_statcache argument
is supplied, so we can remove it in 2.4?
Right now the test cases create a files and a directory in the temp.
directory. Raymond suggested checking files in to the test/ directory,
simplifying the setup/teardown methods; is that worth doing?
Apparently MAC OS 9 doesn't have POSIX-conforming timestamps. A test
fails as a result, but at least for this specific test it's easy enough
to get the POSIX epoch out of it.
This patch adds stdin, stdout as optional arguments to the cmd.Cmd
constructor (defaulting to sys.stdin, sys.stdout), and changes the Cmd
methods throughout to use self.stdout.write() and self.stdin.foo for
output and input. This allows much greater flexibility for using cmd -
for instance, hooking it into a telnet server.
Patch for library module and for documentation.
gzip shouldn't raise ValueError on corrupt files
Currently the gzip module will raise a ValueError if the file was
corrupt (bad crc or bad size). I can't see how that applies to
reading a corrupt file. IOError seems better, and it's what code
will likely be looking for.
guarantee to keep valid pointers in its slots.
tests: Moved ExtensionSaver from test_copy_reg into pickletester, and
use it both places. Once extension codes get assigned, it won't be
safe to overwrite them willy nilly in test suites, and ExtensionSaver
does a thorough job of undoing any possible damage.
Beefed up the EXT[124] tests a bit, to check the smallest and largest
codes in each opcode's range too.
blindly assumed that tp_as_sequence->sq_item always returns
a str or unicode object. This might fail with str or unicode
subclasses.
This patch checks whether the object returned from __getitem__
is a str/unicode object and raises a TypeError if not (and
the filter function returned true).
Furthermore the result for __getitem__ can be more than one
character long, so checks for enough memory have to be done.
this clarifies that they are part of an internal API (albeit shared
between pickle.py, copy_reg.py and cPickle.c).
I'd like to do the same for copy_reg.dispatch_table, but worry that it
might be used by existing code. This risk doesn't exist for the
extension registry.
module. This increases code coverage of Python/sysmodule.c
from 68% to 77% (on Linux).
The script doesn't exercise the error branch that handles an evil
or lost sys.excepthook in Python/pythonrun.c::PyErr_PrintEx().
Also this script might not work on Jython in its current form.
From SF patch #662807.
extension implemented flush() was fixed. Scott also rewrite the
zlib test suite using the unittest module. (SF bug #640230 and
patch #678531.)
Backport candidate I think.
outcome as __slotnames__ on the class. (Like __slots__, it's not safe
to ask for this as an attribute -- you must look for it in the
specific class's __dict__. But it must be set using attribute
notation, because __dict__ is a read-only proxy.)
* Treat major, minor numbers of HTTP version as separate integers
* Fix errors if version string is "HTTP/1.2.3" or even simply "BLAH".
* send_error() checks if 'self.command' is a
HEAD. However, if there's an error parsing the first line of the
HTTP request the self.command wasn't set yet; force
self.command to be initialized to None.
are actually getting generated. Add helpered method
ensure_opcode_in_pickle to do a correct job checking for that. Changed
test_long1(), test_long4(), and test_short_tuples() to use it.
huge. On older Linux systems, the C library's strtod() apparently
gives up before seeing the end of the string when it sees so many
digits that it thinks the result must be Infinity. (It is wrong, BTW
-- there could be an "e-10000" hiding behind 10,000 digits.) The
shorter shuge still tests what it's testing, without relying on
strtod() doing a super job.
exception, ResourceDenied. This is used to distinguish between tests that
are skipped for other reasons (platform support, missing data, etc.) from
those that are skipped because a "resource" has not been enabled. This
prevents those tests from being reported as unexpected skips for the
platform; those should only be considered unexpected skips if the resource
were enabled.
[ 676342 ] after using pdb readline does not work correctly
using Michael Stone's patch so the completer functionality of
cmd is only setup between preloop and postloop.
loops. Renamed DATA and BINDATA to DATA0 and DATA1. Included
disassemblies, but noted why we can't test them. Added XXX comment to
cPickle about a mysterious comment, where pickle and cPickle diverge
in how they number PUT indices.
Assorted code cleanups; e.g., sizeof(char) is 1 by definition, so there's
no need to do things like multiply by sizeof(char) in hairy malloc
arguments. Fixed an undetected-overflow bug in readline_file().
longobject.c: Fixed a really stupid bug in the new _PyLong_NumBits.
pickle.py: Fixed stupid bug in save_long(): When proto is 2, it
wrote LONG1 or LONG4, but forgot to return then -- it went on to
append the proto 1 LONG opcode too.
Fixed equally stupid cancelling bugs in load_long1() and
load_long4(): they *returned* the unpickled long instead of pushing
it on the stack. The return values were ignored. Tests passed
before only because save_long() pickled the long twice.
Fixed bugs in encode_long().
Noted that decode_long() is quadratic-time despite our hopes,
because long(string, 16) is still quadratic-time in len(string).
It's hex() that's linear-time. I don't know a way to make decode_long()
linear-time in Python, short of maybe transforming the 256's-complement
bytes into marshal's funky internal format, and letting marshal decode
that. It would be more valuable to make long(string, 16) linear time.
pickletester.py: Added a global "protocols" vector so tests can try
all the protocols in a sane way. Changed test_ints() and test_unicode()
to do so. Added a new test_long(), but the tail end of it is disabled
because it "takes forever" under pickle.py (but runs very quickly under
cPickle: cPickle proto 2 for longs is linear-time).
anymore either, so don't. This also allows to get rid of obscure code
making __getnewargs__ identical to __getstate__ (hmm ... hope there
wasn't more to this than I realize!).
longer needs to be public, and shoudn't be public because all datetime
objects are immutable. The Python implementation has changed
accordingly, but still need to change the C implementation.
The 4th item can be None or an iterator yielding list items, which are
used to append() or extend() the object. The 5th item can be None or
an iterator yielding a dict's (key, value) pairs, which are stuffed
into the object using __setitem__.
Also (as a separate, though related, feature) add "batching" for list
and dict items. If you pickled a dict or list with a million items in
the past, it would push a million items onto the stack. It now pushes
only 1000 items at a time on the stack, using repeated APPENDS or
SETITEMS opcodes. (For lists, I hope that using many short extend()
calls doesn't exhibit quadratic behavior.)
__module__ is the string name of the module the function was defined
in, just like __module__ of classes. In some cases, particularly for
C functions, the __module__ may be None.
Change PyCFunction_New() from a function to a macro, but keep an
unused copy of the function around so that we don't change the binary
API.
Change pickle's save_global() to use whichmodule() if __module__ is
None, but add the __module__ logic to whichmodule() since it might be
used outside of pickle.
error handers in the Unicode codecs: Negative
positions are treated as being relative to the end of
the input and out of bounds positions result in an
IndexError.
Also update the PEP and include an explanation of
this in the documentation for codecs.register_error.
Fixes a small bug in iconv_codecs: if the position
from the callback is negative *add* it to the size
instead of substracting it.
From SF patch #677429.
M rpc.py
SF Bug 676398 Doesn't handle non-built-in exceptions
1. Move exception formatting to the subprocess; allows subclassing of
exceptions, including subclasses created in the shell without
introducing excessive complexity in the RPC mechanism.
2. Provide access to linecache from subprocess to support this.
classes have a __reduce__ that returns (self.__class__,
self.__getstate__()). tzinfo.__reduce__() is a bit smarter, calling
__getinitargs__ and __getstate__ if they exist, and falling back to
__dict__ if it exists and isn't empty.
for this iconv() implementation in the init function.
For encoding: use a byteswapped version of the input if
neccessary.
For decoding: byteswap every piece returned by iconv()
if neccessary (but not those pieces returned from the
callback)
Comment out test_sane() in the test script, because
whether this works depends on whether byte swapping
is neccessary or not (an on Py_UNICODE_SIZE)
on the type instead of self.save(t). This defeated the purpose of
NEWOBJ, because it didn't generate a BINGET opcode when t was already
memoized; but moreover, it would generate multiple BINPUT opcodes for
the same type! pickletools.dis() doesn't like this.
How I found this? I was playing with picklesize.py in the datetime
sandbox, and noticed that protocol 2 pickles for multiple objects were
in fact larger than protocol 1 pickles! That was suspicious, so I
decided to disassemble one of the pickles.
This really needs a unit test, but I'm exhausted. I'll be late for
work as it is. :-(
the same function, don't save the state or write a BUILD opcode. This
is so that a type (e.g. datetime :-) can support protocol 2 using
__getnewargs__ while also supporting protocol 0 and 1 using
__getstate__. (Without this, the state would be pickled twice with
protocol 2, unless __getstate__ is defined to return None, which
breaks protocol 0 and 1.)
popped a MARK, but without stack emulation the disassembler couldn't
know that, and subsequent indentation got hosed.
Now the disassembler does do enough stack emulation to catch this. While
I was at it, also added lots of sanity checks for other stack operations,
and correct use of the memo. This goes (I think) a long way toward being
a "pickle verifier" now too.
types. The special handling for these can now be removed from save_newobj().
Add some testing for this.
Also add support for setting the 'fast' flag on the Python Pickler class,
which suppresses use of the memo.
Jack Jansen
Guido van Rossum
Neal Norwitz
Raymond Hettinger
Tim Peters
Walter Dörwald
Andrew M. Kuchling
Andrew MacIntyre
Mark Hammond
Jeremy Hylton
Kurt B. Kaiser
Skip Montanaro
Just van Rossum
Barry Warsaw
Anthony Baxter
Jason Tishler
Marc-André Lemburg
Fred Drake
Greg Ward
Michael W. Hudson
Gregory P. Smith
Thomas Heller
Neil Schemenauer