- gbk and gb18030 codec now handle U+30FB KATAKANA MIDDLE DOT correctly.
- iso2022_jp_2 codec now encodes into G0 for KS X 1001, GB2312
codepoints to conform the standard.
- iso2022_jp_3 and iso2022_jp_2004 codec can encode JIS X 2013:2
codepoints now.
generator expressions (x for x, in ... ) works again.
Sigh, I only fixed for loops the first time, not list comps and genexprs too.
I couldn't find any more unpacking cases where there is a similar bug lurking.
This code should be refactored to eliminate the duplication. I'm sure
the listcomp/genexpr code can be refactored. I'm not sure if the for loop
can re-use any of the same code though.
Will backport to 2.5 (the only place it matters).
i_divmod(): As discussed on Python-Dev, changed the overflow
checking to live happily with recent gcc optimizations that
assume signed integer arithmetic never overflows.
This differs from the corresponding change on the 2.5 and 2.4
branches, using a less obscure approach, but one that /may/
tickle platform idiocies in their definitions of LONG_MIN.
The 2.4 + 2.5 change avoided introducing a dependence on
LONG_MIN, at the cost of substantially goofier code.
of the Python part of a callback function to C. If it cannot be
converted, call PyErr_WriteUnraisable with the exception we got.
Before, arbitrary data has been passed to the calling C code in this
case.
(I'm not really sure the NEWS entry is understandable, but I cannot
find better words)
before raising SystemExit, allowing IDLE to honor quit() and exit().
M Lib/site.py
M Lib/idlelib/PyShell.py
M Lib/idlelib/CREDITS.txt
M Lib/idlelib/NEWS.txt
M Misc/NEWS
sporadically on other platforms. This is really a band-aid that doesn't
fix the underlying issue in SocketServer. It's not clear if it's worth
it to fix SocketServer, however, I opened a bug to track it:
http://python.org/sf/1540386
ctypes instances no longer have the internal and undocumented
'_as_parameter_' attribute which was used to adapt them to foreign
function calls; this mechanism is replaced by a function pointer in
the type's stgdict.
In the 'from_param' class methods, try the _as_parameter_ attribute if
other conversions are not possible.
This makes the documented _as_parameter_ mechanism work as intended.
Change the ctypes version number to 1.0.1.
Replace UnicodeDecodeErrors raised during == and !=
compares of Unicode and other objects with a new
UnicodeWarning.
All other comparisons continue to raise exceptions.
Exceptions other than UnicodeDecodeErrors are also left
untouched.
were failing due to inappropriate clipping of numbers larger than 2**31
with new-style classes. (typeobject.c) In reviewing the code for classic
classes, there were 2 problems. Any negative value return could be returned.
Always return -1 if there was an error. Also make the checks similar
with the new-style classes. I believe this is correct for 32 and 64 bit
boxes, including Windows64.
Add a test of classic classes too.
I modified this patch some by fixing style, some error checking, and adding
XXX comments. This patch requires review and some changes are to be expected.
I'm checking in now to get the greatest possible review and establish a
baseline for moving forward. I don't want this to hold up release if possible.
The first hunk changes the colon to an ! like other Windows variants.
We need to always wait on the child so the lock gets released and
no other tests fail. This is the try/finally in the second hunk.
PyThreadState_SetAsyncExc(): internal correctness changes wrt
refcount safety and deadlock avoidance. Also added a basic test
case (relying on ctypes) and repaired the docs.
In the 2.5 development cycle, MAKE_CLOSURE as changed to take free
variables as a tuple rather than as individual items on the stack.
Closes patch #1534084.
on each iteration. I'm not positive this is the best way to handle
this. I'm also not sure that there aren't other cases where
the lnotab is generated incorrectly. It would be great if people
that use pdb or tracing could test heavily.
Also:
* Remove dead/duplicated code that wasn't used/necessary
because we already handled the docstring prior to entering the loop.
* add some debugging code into the compiler (#if 0'd out).
This provides the proper warning for struct.pack().
PyErr_Warn() is now deprecated in favor of PyErr_WarnEx().
As mentioned by Tim Peters on python-dev.
41667, 41668 - initial switch to xmlcore
47044 - mention of xmlcore in What's New
50687 - mention of xmlcore in the library reference
re-apply xmlcore changes to xml:
41674 - line ending changes (re-applied manually), directory props
41677 - add cElementTree wrapper
41678 - PSF licensing for etree
41812 - whitespace normalization
42724 - fix svn:eol-style settings
43681, 43682 - remove Python version-compatibility cruft from minidom
46773 - fix encoding of \r\n\t in attr values in saxutils
47269 - added XMLParser alias for cElementTree compatibility
additional tests were added in Lib/test/test_sax.py that failed with
the xmlcore changes; these relate to SF bugs #1511497, #1513611
with PEP 302. This was fixed by adding an ``imp.NullImporter`` type that is
used in ``sys.path_importer_cache`` to cache non-directory paths and avoid
excessive filesystem operations during imports.
inspect.py, and pydoc.py. Specifically, this allows for querying the type of
an object against these built-in C types and more importantly, for getting
their docstrings printed in the interactive interpreter's help() function.
This patch includes a new built-in module called _types which provides
definitions of getset and member descriptors for use by the types.py module.
These types are exposed as types.GetSetDescriptorType and
types.MemberDescriptorType. Query functions are provided as
inspect.isgetsetdescriptor() and inspect.ismemberdescriptor(). The
implementations of these are robust enough to work with Python implementations
other than CPython, which may not have these fundamental types.
The patch also includes documentation and test suite updates.
I commit these changes now under these guiding principles:
1. Silence is assent. The release manager has not said "no", and of the few
people that cared enough to respond to the thread, the worst vote was "0".
2. It's easier to ask for forgiveness than permission.
3. It's so dang easy to revert stuff in svn, that you could view this as a
forcing function. :)
Windows build patches will follow.
In general, C doesn't define anything about what happens when
an operation on a signed integral type overflows, and PyOS_strtol()
did several formally undefined things of that nature on signed
longs. Some version of gcc apparently tries to exploit that now,
and PyOS_strtol() could fail to detect overflow then.
Tried to repair all that, although it seems at least as likely to me
that we'll get screwed by bad platform definitions for LONG_MIN
and/or LONG_MAX now. For that reason, I don't recommend backporting
this.
Note that I have no box on which this makes a lick of difference --
can't really test it, except to note that it didn't break anything
on my boxes.
Silent change: PyOS_strtol() used to return the hard-coded 0x7fffffff
in case of overflow. Now it returns LONG_MAX. They're the same only on
32-bit boxes (although C doesn't guarantee that either ...).
This adds a new key definition for OSX, which is slightly different from the
classic mac definition.
Also add NEWS item for a couple of bugfixes I added recently.
points out there are really two types of continued headers defined in this
RFC (i.e. "encoded" parameters with the form "name*0*=" and unencoded
parameters with the form "name*0="), but we were were handling them both the
same way and that isn't correct.
This patch should be much more RFC compliant in that only encoded params are
%-decoded and the charset/language information is only extract if there are
any encoded params in the segments. If there are no encoded params then the
RFC says that there will be no charset/language parts.
Note however that this will change the return value for Message.get_param() in
some cases. For example, whereas before if you had all unencoded param
continuations you would have still gotten a 3-tuple back from this method
(with charset and language == None), you will now get just a string. I don't
believe this is a backward incompatible change though because the
documentation for this method already indicates that either return value is
possible and that you must do an isinstance(val, tuple) check to discriminate
between the two. (Yeah that API kind of sucks but we can't change /that/
without breaking code.)
Test cases, some documentation updates, and a NEWS item accompany this patch.
Moved the code for _PyThread_CurrentFrames() up, so it's no longer
in a huge "#ifdef WITH_THREAD" block (I didn't realize it /was/ in
one).
Changed test_sys's test_current_frames() so it passes with or without
thread supported compiled in.
Note that test_sys fails when Python is compiled without threads,
but for an unrelated reason (the old test_exit() fails with an
indirect ImportError on the `thread` module). There are also
other unrelated compilation failures without threads, in extension
modules (like ctypes); at least the core compiles again.
Do we really support --without-threads? If so, there are several
problems remaining.
of values in the time tuple passed in. Unfortunately people came to rely on
undocumented behaviour of setting unneeded values to 0, regardless of if it was
within the valid range. Now those values force the value internally to the
minimum value when 0 is passed in.
'Extension Modules' section where they belong, probably.
This destroyes the original order of the news entries, don't know
if that is important or not.
/F wrote the text docs, Englebert Gruber massaged it to latex and I
did some more massaging to try and improve the consistency and
fix some name mismatches between the declaration and text.
str() or repr()) would work, just not multi-value tuples. Probably not a
backport candidate, since it changes the behaviour of passing a
single-element tuple:
>>> string.Template("$foo").substitute(dict(foo=(1,)))
'(1,)'
versus
'1'
be called at the end of each test that spawns children (perhaps it
should be called from regrtest instead?). This will hopefully prevent
some of the unexplained failures in the buildbots (hppa and alpha)
during tests that spawn children. The problems were not reproducible.
There were many zombies that remained at the end of several tests.
In the worst case, this shouldn't cause any more problems,
though it may not help either. Time will tell.
omit a default "error" argument for NULL pointer. This allows
the parser to take a codec from cjkcodecs again.
(Reported by Taewook Kang and reviewed by Walter Doerwald)
both a subclass of Warning and a subclass of types.ClassType. The latter is no
longer true thanks to new-style exceptions.
Closes bug #1510580. Thanks to AMK for the test.
now works reliably. It has been updated to use automatic BerkeleyDB
deadlock detection and the bsddb.dbutils.DeadlockWrap wrapper to retry
database calls that would previously deadlock. [SF python bug #775414]
adjacent triples in the result list describe non-adjacent matching
blocks. That's _nice_ to have, and Guido said he wanted it.
Not a bugfix candidate: Guido or not ;-), this changes visible
endcase semantics (note that some tests had to change), and
nothing about this was documented before. Since it was working
as designed, and behavior was consistent with the docs, it wasn't
"a bug".
arguments in reverse, the interpreter would infinitely recourse trying to get a
coercion that worked. So put in a recursion check after a coercion is made and
the next call to attempt to use the coerced values.
Fixes bug #992017 and closes crashers/coerce.py .
Heavily revised, comprising revisions:
46640 - original trunk revision (backed out in r46655)
46647 - markup fix (backed out in r46655)
46692:46918 merged from branch aimacintyre-sf1454481
branch tested on buildbots (Windows buildbots had problems
not related to these changes).
the char buffer was requested. Now it actually returns the char buffer if
available or raises a TypeError if it isn't (as is raised for the other buffer
types if they are not present but requested).
Not a backport candidate since it does change semantics of the buffer object
(although it could be argued this is enough of a bug to bother backporting).
aborts the db transaction safely when a modifier callback fails.
Fixes SF python patch/bug #1408584.
Also cleans up the bsddb.dbtables docstrings since thats the only
documentation that exists for that unadvertised module. (people
really should really just use sqlite3)
46640 Patch #1454481: Make thread stack size runtime tunable.
46647 Markup fix
The first is causing many buildbots to fail test runs, and there
are multiple causes with seemingly no immediate prospects for
repairing them. See python-dev discussion.
Note that a branch can (and should) be created for resolving these
problems, like
svn copy svn+ssh://svn.python.org/python/trunk -r46640 svn+ssh://svn.python.org/python/branches/NEW_BRANCH
followed by merging rev 46647 to the new branch.
to each allocated block. This was using 4 bytes for each such
piece of info regardless of platform. This didn't really matter
before (proof: no bug reports, and the debug-build obmalloc would
have assert-failed if it was ever asked for a chunk of memory
>= 2**32 bytes), since container indices were plain ints. But after
the Py_ssize_t changes, it's at least theoretically possible to
allocate a list or string whose guts exceed 2**32 bytes, and the
PYMALLOC_DEBUG routines would fail then (having only 4 bytes
to record the originally requested size).
Now we use sizeof(size_t) bytes for each of a PYMALLOC_DEBUG
build's extra debugging fields. This won't make any difference
on 32-bit boxes, but will add 16 bytes to each allocation in
a debug build on a 64-bit box.
the output required more than one line. "Small" dicts got
displayed in seemingly random order (the hash-induced order
produced by dict.__repr__). None of this was documented.
Now pprint functions always sort dicts by key, and the docs
promise it.
This was proposed and agreed to during the PyCon 2006 core
sprint -- I just didn't have time for it before now.
Renames functional extension module to _functools and adds a Python
functools module so that utility functions like update_wrapper can be
added easily.
invalid file paths for the built-in import machinery which leads to
fewer open calls on startup.
Also fix issue with PEP 302 style import hooks which lead to more open()
calls than necessary.
QueryPerformanceCounter(), but we believe Win64 does
support it now. So use in time.clock().
It would be peachy if someone with a Win64 box tried
this ;-)
In rare cases of strings specifying true values near sys.maxint,
and oddball bases (not decimal or a power of 2), int(string, base)
could deliver insane answers. This repairs all such problems, and
also speeds string->int significantly. On my box, here are %
speedups for decimal strings of various lengths:
length speedup
------ -------
1 12.4%
2 15.7%
3 20.6%
4 28.1%
5 33.2%
6 37.5%
7 41.9%
8 46.3%
9 51.2%
10 19.5%
11 19.9%
12 23.9%
13 23.7%
14 23.3%
15 24.9%
16 25.3%
17 28.3%
18 27.9%
19 35.7%
Note that the difference between 9 and 10 is the difference between
short and long Python ints on a 32-bit box. The patch doesn't
actually do anything to speed conversion to long: the speedup is
due to detecting "unsigned long" overflow more quickly.
This is a bugfix candidate, but it's a non-trivial patch and it
would be painful to separate the "bug fix" from the "speed up" parts.
Applied patch zombie-frames-2.diff from sf patch 876206 with updates for
Python 2.5 and also modified to retain the free_list to avoid the 67%
slow-down in pybench recursion test. 5% speed up in function call pybench.
about "%u", "%lu" and "%zu" formats.
Since PyString_FromFormat and PyErr_Format have exactly the same rules
(both inherited from PyString_FromFormatV), it would be good if someone
with more LaTeX Fu changed one of them to just point to the other.
Their docs were way out of synch before this patch, and I just did a
mass copy+paste to repair that.
Not a backport candidate (this is a new feature).
tuples. Lots to be added, still, but this will give big-memory people
something to play with in 2.5 alpha 2, and hopefully get more people to
write these tests.
the 2005 Summer of Code).
The revision adds a number of new mailbox classes that support adding
and removing messages; these classes also support mailbox locking and
default to using email.Message instead of rfc822.Message.
The old mailbox classes are largely left alone for backward compatibility.
The exception is the Maildir class, which was present in the old module
and now inherits from the new classes. The Maildir class's interface
is pretty simple, though, so I think it'll be compatible with existing
code.
(The change to the NEWS file also adds a missing word to a different
news item, which unfortunately required rewrapping the line.)
Python 2.4 changed ntpath.abspath to do an import
inside the function. As a result, due to Python's
import lock, anything calling abspath on Windows
(directly, or indirectly like tempfile.TemporaryFile)
hung when it was called from a thread spawned as a
side effect of importing a module.
This is a depressingly frequent problem, and
deserves a more general fix. I'm settling for
a micro-fix here because this specific one accounts
for a report of Zope Corp's ZEO hanging on Windows,
and it was an odd way to change abspath to begin
with (ntpath needs a different implementation
depending on whether we're actually running on
Windows, and the _obvious_ way to arrange for that
is not to bury a possibly-failing import _inside_
the function).
Note that if/when other micro-fixes of this kind
get made, the new Lib/test/threaded_import_hangers.py
is a convenient place to add tests for them.
much convolution <0.5 wink>. Simplified to the point that it works,
and test_threading_local no longer reports leaks under -R. Thanks
to Thomas Wouters for initial analysis.
passing a string. Martin already fixed the actual crash by ensuring
Py_UNICODE is unsigned. As discussed on python-dev, this fix
removes the possibility of creating a unicode string from a raw buffer.
There is an outstanding question of how to fix the crash in 2.4.
prepends the exception's module name to non-builtin exceptions, like
the interpreter itself does.
broke a number of doctests. should be discussed before checking in (see
discussion on python-dev).
failing on one of the 32-bit buildbot boxes because of it,
due to tempting but always-wrong Python code. Users
probably have code like this too (I know I did ...).
This is based on pysqlite2.1.3, and provides a DB-API interface in
the standard library. You'll need sqlite 3.2.2 or later to build
this - if you have an earlier version, the C extension module will
not be built.
objimpl.h, pymem.h: Stop mapping PyMem_{Del, DEL} and PyMem_{Free, FREE}
to PyObject_{Free, FREE} in a release build. They're aliases for the
system free() now.
_subprocess.c/sp_handle_dealloc(): Since the memory was originally
obtained via PyObject_NEW, it must be released via PyObject_FREE (or
_DEL).
pythonrun.c, tokenizer.c, parsermodule.c: I lost count of the number of
PyObject vs PyMem mismatches in these -- it's like the specific
function called at each site was picked at random, sometimes even with
memory obtained via PyMem getting released via PyObject. Changed most
to use PyObject uniformly, since the blobs allocated are predictably
small in most cases, and obmalloc is generally faster than system
mallocs then.
If extension modules in real life prove as sloppy as Python's front
end, we'll have to revert the objimpl.h + pymem.h part of this patch.
Note that no problems will show up in a debug build (all calls still go
thru obmalloc then). Problems will show up only in a release build, most
likely segfaults.
This was a fair amount of rework of the patch. Refactored test_fork1 so it
could be reused by the new tests for wait3/4. Also made them into new style
unittests (derive from unittest.TestCase).
This patch adds a-LAW encoding to audioop and replaces the old
u-LAW encoding/decoding code with the current code from sox.
Possible issues: the code from sox uses int16_t.
Code by Lars Immisch
This is a heavily altered derivative of SF patch 1123430, Evan
Jones's heroic effort to make obmalloc return unused arenas to
the system free(), with some heuristic strategies to make it
more likley that arenas eventually _can_ be freed.
of tuple) that provides incremental decoders and encoders (a way to use
stateful codecs without the stream API). Functions
codecs.getincrementaldecoder() and codecs.getincrementalencoder() have
been added.
added message attribute compared to the previous version of Exception. It is
also a new-style class, making all exceptions now new-style. KeyboardInterrupt
and SystemExit inherit from BaseException directly. String exceptions now
raise DeprecationWarning.
Applies patch 1104669, and closes bugs 1012952 and 518846.
- New semantics for __exit__() -- it must re-raise the exception
if type is not None; the with-statement itself doesn't do this.
(See the updated PEP for motivation.)
- Added context managers to:
- file
- thread.LockType
- threading.{Lock,RLock,Condition,Semaphore,BoundedSemaphore}
- decimal.Context
- Added contextlib.py, which defines @contextmanager, nested(), closing().
- Unit tests all around; bot no docs yet.
This was started by Mike Bland and completed by Guido
(with help from Neal).
This still needs a __future__ statement added;
Thomas is working on Michael's patch for that aspect.
There's a small amount of code cleanup and refactoring
in ast.c, compile.c and ceval.c (I fixed the lltrace
behavior when EXT_POP is used -- however I had to make
lltrace a static global).
PyThreadState_Delete(): if the auto-GIL-state machinery knows about
the thread state, forget it (since the thread state is being deleted,
continuing to remember it can't help, but can hurt if another thread
happens to get created with the same thread id).
I'll backport to 2.4 next.
- The copy module now "copies" function objects (as atomic objects).
- dict.__getitem__ now looks for a __missing__ hook before raising
KeyError.
- Added a new type, defaultdict, to the collections module.
This uses the new __missing__ hook behavior added to dict (see above).
Based on lsprof (patch #1212837) by Brett Rosen and Ted Czotter.
With further editing by Michael Hudson and myself.
History in svn repo: http://codespeak.net/svn/user/arigo/hack/misc/lsprof
* Module/_lsprof.c is the internal C module, Lib/cProfile.py a wrapper.
* pstats.py updated to display cProfile's caller/callee timings if available.
* setup.py and NEWS updated.
* documentation updates in the profiler section:
- explain the differences between the three profilers that we have now
- profile and cProfile can use a unified documentation, like (c)Pickle
- mention that hotshot is "for specialized usage" now
- removed references to the "old profiler" that no longer exists
* test updates:
- extended test_profile to cover delicate cases like recursion
- added tests for the caller/callee displays
- added test_cProfile, performing the same tests for cProfile
* TO-DO:
- cProfile gives a nicer name to built-in, particularly built-in methods,
which could be backported to profile.
- not tested on Windows recently!
is larger than FD_SETSIZE.
This can only be acheived with ulimit -n SOME_NUMBER_BIGGER_THAN_FD_SETSIZE
which is typically only available to root. Since this wouldn't normally
be run in a test (ie, run as root), it doesn't seem too worthwhile to
add a normal test. The bug report has one version of a test. I've
written another. Not sure what the best thing to do is.
Do the check before calling internal_select() because we can't set
an error in between Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS.
This seemed the clearest solution, ie handle before calling internal_select()
rather than inside. Plus there is at least one place outside
of internal_select() that needed to be handled.
Will backport.
on both Unix (SVR4 and BSD) and Windows. Restores behaviour of passing -1
for anonymous memory on Unix. Use MAP_ANONYMOUS instead of _ANON since
the latter is deprecated according to Linux (gentoo) man pages.
Should we continue to allow mmap.mmap(0, length) to work on Windows?
0 is a valid fd.
Will backport bugfix portions.
This is how string objects work. u'%f' could use , instead of .
for the decimal point. Now both strings and unicode always use periods.
This is the code that would break:
import locale
locale.setlocale(locale.LC_NUMERIC, 'de_DE')
u'%.1f' % 1.0
assert '1.0' == u'%.1f' % 1.0
I couldn't create a test case which fails, but this fixes the problem.
Will backport.
Add C API function Py_GetBuildNumber(), add it to the interactive prompt
banner (i.e. Py_GetBuildInfo()), and add it as the sys.build_number
attribute. The build number is a string instead of an int because it may
contain a trailing 'M' if there are local modifications.
asked to read tens of megabytes of data. On my Mac, it hits MemoryErrors
when reading around 15Mb in one chunk. The fix is to read the body in several
parts, not as one big piece.
It would be nice to fix the underlying socket.read() problem, too.
2.4 bugfix candidate.
PyUnicode_DecodeCharmap() the accept a unicode string as the mapping
argument which is used as a mapping table.
This code isn't used by any of the codecs yet.
- SF Bug #772896, unknown encoding results in MemoryError, which is not helpful
I will only backport the segfault fix. I'll let Anthony decide if he wants
the other changes backported. I will do the backport if asked.
Fix segfault. I tried to write a test, but it wouldn't crash
when running regrtest. This really should have some sort of test.
Should definitely be backported.
represented as a C int, raise OverflowError.
(Forward port from 2.4.2; the patch to classobject.c was already in
but needed a correction in the error message text.)
about illegal code points. The codec now supports PEP 293 style error handlers.
(This is a variant of the Nik Haldimann's patch that detects truncated data)
even if there is an exception in later lines, resulting in
correct line numbers for decoding errors in source code. Fixes#1178484.
Will backport to 2.4.
Georg Brandl
Neal Norwitz
Gustavo Niemeyer
Hye-Shik Chang
Tim Peters
Nick Coghlan
Martin v. Löwis
Neil Schemenauer
Andrew M. Kuchling
Marc-André Lemburg
Thomas Heller
Anthony Baxter
Kurt B. Kaiser
Guido van Rossum
Andrew MacIntyre
Bob Ippolito
Ronald Oussoren
Fred Drake
Phillip J. Eby
Matt Fleming
Gregory P. Smith
Barry Warsaw
Brett Cannon
Greg Ward
Peter Astrand
Thomas Wouters
Gerhard Häring
Vinay Sajip
Trent Mick
Walter Dörwald
Armin Rigo
George Yoshida
Richard Jones
Skip Montanaro
Matthias Klose
Raymond Hettinger
Fredrik Lundh
Michael W. Hudson