The problem is that _DB_get_type() can't be called without the GIL
because it calls a bunch of PyErr_* APIs when an error occurs.
There were no other cases in this file that it was called without the GIL.
Removing the BEGIN/END THREAD around _DB_get_type() made everything work.
Will backport.
Two changes:
Where possible, use link()/remove() to move files into a directory; this
makes it easier to avoid overwriting an existing file.
Use _create_carefully() to create files in tmp/, which uses O_EXCL.
Backport candidate.
It seems like this should be a different error than SystemError, but
I don't have any great ideas and SystemError was raised in 2.4 and earlier.
Will backport.
Check this and raise an error when something else is used - before
this change ctypes would hang or crash when such a callback was
called. This is a partial fix for #1574584.
Will backport to release25-maint.
* unified the way intobject, longobject and mystrtoul handle
values around -sys.maxint-1.
* in general, trying to entierely avoid overflows in any computation
involving signed ints or longs is extremely involved. Fixed a few
simple cases where a compiler might be too clever (but that's all
guesswork).
* more overflow checks against bad data in marshal.c.
* 2.5 specific: fixed a number of places that were still confusing int
and Py_ssize_t. Some of them could potentially have caused
"real-world" breakage.
* list.pop(x): fixing overflow issues on x was messy. I just reverted
to PyArg_ParseTuple("n"), which does the right thing. (An obscure
test was trying to give a Decimal to list.pop()... doesn't make
sense any more IMHO)
* trying to write a few tests...
In addition to testing positive numbers, verify negative numbers work in configure.
In order to avoid compiler warnings on OS X 10.4, also change the order of the check
for the format character to use (PY_FORMAT_SIZE_T) in the sprintf format
for Py_ssize_t. This patch changes PY_FORMAT_SIZE_T from "" to "l" if it wasn't
defined at configure time. Need to verify the buildbot results.
Backport candidate (if everyone thinks this patch can't be improved).
Building with HP's cc on HP-UX turned up a couple of problems.
_PyGILState_NoteThreadState was declared as static inconsistently.
Make it static as it's not necessary outside of this module.
Some tests failed because errno was reset to 0. (I think the tests
that failed were at least: test_fcntl and test_mailbox).
Ensure that errno doesn't change after a call to Py_END_ALLOW_THREADS.
This only affected debug builds.
As mentioned on python-dev, reverting patch #1504333 because it introduced
an infinite loop in rev 47154.
This patch also adds a test to prevent the regression.
- gbk and gb18030 codec now handle U+30FB KATAKANA MIDDLE DOT correctly.
- iso2022_jp_2 codec now encodes into G0 for KS X 1001, GB2312
codepoints to conform the standard.
- iso2022_jp_3 and iso2022_jp_2004 codec can encode JIS X 2013:2
codepoints now.
generator expressions (x for x, in ... ) works again.
Sigh, I only fixed for loops the first time, not list comps and genexprs too.
I couldn't find any more unpacking cases where there is a similar bug lurking.
This code should be refactored to eliminate the duplication. I'm sure
the listcomp/genexpr code can be refactored. I'm not sure if the for loop
can re-use any of the same code though.
Will backport to 2.5 (the only place it matters).
i_divmod(): As discussed on Python-Dev, changed the overflow
checking to live happily with recent gcc optimizations that
assume signed integer arithmetic never overflows.
This differs from the corresponding change on the 2.5 and 2.4
branches, using a less obscure approach, but one that /may/
tickle platform idiocies in their definitions of LONG_MIN.
The 2.4 + 2.5 change avoided introducing a dependence on
LONG_MIN, at the cost of substantially goofier code.
of the Python part of a callback function to C. If it cannot be
converted, call PyErr_WriteUnraisable with the exception we got.
Before, arbitrary data has been passed to the calling C code in this
case.
(I'm not really sure the NEWS entry is understandable, but I cannot
find better words)
before raising SystemExit, allowing IDLE to honor quit() and exit().
M Lib/site.py
M Lib/idlelib/PyShell.py
M Lib/idlelib/CREDITS.txt
M Lib/idlelib/NEWS.txt
M Misc/NEWS
sporadically on other platforms. This is really a band-aid that doesn't
fix the underlying issue in SocketServer. It's not clear if it's worth
it to fix SocketServer, however, I opened a bug to track it:
http://python.org/sf/1540386
ctypes instances no longer have the internal and undocumented
'_as_parameter_' attribute which was used to adapt them to foreign
function calls; this mechanism is replaced by a function pointer in
the type's stgdict.
In the 'from_param' class methods, try the _as_parameter_ attribute if
other conversions are not possible.
This makes the documented _as_parameter_ mechanism work as intended.
Change the ctypes version number to 1.0.1.
Replace UnicodeDecodeErrors raised during == and !=
compares of Unicode and other objects with a new
UnicodeWarning.
All other comparisons continue to raise exceptions.
Exceptions other than UnicodeDecodeErrors are also left
untouched.
were failing due to inappropriate clipping of numbers larger than 2**31
with new-style classes. (typeobject.c) In reviewing the code for classic
classes, there were 2 problems. Any negative value return could be returned.
Always return -1 if there was an error. Also make the checks similar
with the new-style classes. I believe this is correct for 32 and 64 bit
boxes, including Windows64.
Add a test of classic classes too.
I modified this patch some by fixing style, some error checking, and adding
XXX comments. This patch requires review and some changes are to be expected.
I'm checking in now to get the greatest possible review and establish a
baseline for moving forward. I don't want this to hold up release if possible.
This is the first batch of fixes that should be easy to verify based on context.
This fixes problem numbers: 220 (ast), 323-324 (symtable),
321-322 (structseq), 215 (array), 210 (hotshot), 182 (codecs), 209 (etree).
The first hunk changes the colon to an ! like other Windows variants.
We need to always wait on the child so the lock gets released and
no other tests fail. This is the try/finally in the second hunk.
PyThreadState_SetAsyncExc(): internal correctness changes wrt
refcount safety and deadlock avoidance. Also added a basic test
case (relying on ctypes) and repaired the docs.
In the 2.5 development cycle, MAKE_CLOSURE as changed to take free
variables as a tuple rather than as individual items on the stack.
Closes patch #1534084.
on each iteration. I'm not positive this is the best way to handle
this. I'm also not sure that there aren't other cases where
the lnotab is generated incorrectly. It would be great if people
that use pdb or tracing could test heavily.
Also:
* Remove dead/duplicated code that wasn't used/necessary
because we already handled the docstring prior to entering the loop.
* add some debugging code into the compiler (#if 0'd out).
This provides the proper warning for struct.pack().
PyErr_Warn() is now deprecated in favor of PyErr_WarnEx().
As mentioned by Tim Peters on python-dev.
41667, 41668 - initial switch to xmlcore
47044 - mention of xmlcore in What's New
50687 - mention of xmlcore in the library reference
re-apply xmlcore changes to xml:
41674 - line ending changes (re-applied manually), directory props
41677 - add cElementTree wrapper
41678 - PSF licensing for etree
41812 - whitespace normalization
42724 - fix svn:eol-style settings
43681, 43682 - remove Python version-compatibility cruft from minidom
46773 - fix encoding of \r\n\t in attr values in saxutils
47269 - added XMLParser alias for cElementTree compatibility
additional tests were added in Lib/test/test_sax.py that failed with
the xmlcore changes; these relate to SF bugs #1511497, #1513611
with PEP 302. This was fixed by adding an ``imp.NullImporter`` type that is
used in ``sys.path_importer_cache`` to cache non-directory paths and avoid
excessive filesystem operations during imports.
inspect.py, and pydoc.py. Specifically, this allows for querying the type of
an object against these built-in C types and more importantly, for getting
their docstrings printed in the interactive interpreter's help() function.
This patch includes a new built-in module called _types which provides
definitions of getset and member descriptors for use by the types.py module.
These types are exposed as types.GetSetDescriptorType and
types.MemberDescriptorType. Query functions are provided as
inspect.isgetsetdescriptor() and inspect.ismemberdescriptor(). The
implementations of these are robust enough to work with Python implementations
other than CPython, which may not have these fundamental types.
The patch also includes documentation and test suite updates.
I commit these changes now under these guiding principles:
1. Silence is assent. The release manager has not said "no", and of the few
people that cared enough to respond to the thread, the worst vote was "0".
2. It's easier to ask for forgiveness than permission.
3. It's so dang easy to revert stuff in svn, that you could view this as a
forcing function. :)
Windows build patches will follow.
In general, C doesn't define anything about what happens when
an operation on a signed integral type overflows, and PyOS_strtol()
did several formally undefined things of that nature on signed
longs. Some version of gcc apparently tries to exploit that now,
and PyOS_strtol() could fail to detect overflow then.
Tried to repair all that, although it seems at least as likely to me
that we'll get screwed by bad platform definitions for LONG_MIN
and/or LONG_MAX now. For that reason, I don't recommend backporting
this.
Note that I have no box on which this makes a lick of difference --
can't really test it, except to note that it didn't break anything
on my boxes.
Silent change: PyOS_strtol() used to return the hard-coded 0x7fffffff
in case of overflow. Now it returns LONG_MAX. They're the same only on
32-bit boxes (although C doesn't guarantee that either ...).
This adds a new key definition for OSX, which is slightly different from the
classic mac definition.
Also add NEWS item for a couple of bugfixes I added recently.
points out there are really two types of continued headers defined in this
RFC (i.e. "encoded" parameters with the form "name*0*=" and unencoded
parameters with the form "name*0="), but we were were handling them both the
same way and that isn't correct.
This patch should be much more RFC compliant in that only encoded params are
%-decoded and the charset/language information is only extract if there are
any encoded params in the segments. If there are no encoded params then the
RFC says that there will be no charset/language parts.
Note however that this will change the return value for Message.get_param() in
some cases. For example, whereas before if you had all unencoded param
continuations you would have still gotten a 3-tuple back from this method
(with charset and language == None), you will now get just a string. I don't
believe this is a backward incompatible change though because the
documentation for this method already indicates that either return value is
possible and that you must do an isinstance(val, tuple) check to discriminate
between the two. (Yeah that API kind of sucks but we can't change /that/
without breaking code.)
Test cases, some documentation updates, and a NEWS item accompany this patch.
Moved the code for _PyThread_CurrentFrames() up, so it's no longer
in a huge "#ifdef WITH_THREAD" block (I didn't realize it /was/ in
one).
Changed test_sys's test_current_frames() so it passes with or without
thread supported compiled in.
Note that test_sys fails when Python is compiled without threads,
but for an unrelated reason (the old test_exit() fails with an
indirect ImportError on the `thread` module). There are also
other unrelated compilation failures without threads, in extension
modules (like ctypes); at least the core compiles again.
Do we really support --without-threads? If so, there are several
problems remaining.
of values in the time tuple passed in. Unfortunately people came to rely on
undocumented behaviour of setting unneeded values to 0, regardless of if it was
within the valid range. Now those values force the value internally to the
minimum value when 0 is passed in.
'Extension Modules' section where they belong, probably.
This destroyes the original order of the news entries, don't know
if that is important or not.
/F wrote the text docs, Englebert Gruber massaged it to latex and I
did some more massaging to try and improve the consistency and
fix some name mismatches between the declaration and text.
str() or repr()) would work, just not multi-value tuples. Probably not a
backport candidate, since it changes the behaviour of passing a
single-element tuple:
>>> string.Template("$foo").substitute(dict(foo=(1,)))
'(1,)'
versus
'1'
be called at the end of each test that spawns children (perhaps it
should be called from regrtest instead?). This will hopefully prevent
some of the unexplained failures in the buildbots (hppa and alpha)
during tests that spawn children. The problems were not reproducible.
There were many zombies that remained at the end of several tests.
In the worst case, this shouldn't cause any more problems,
though it may not help either. Time will tell.
copy. This will hopefully catch problems where directories are added
under Lib/ but not to Makefile.pre.in. This breaks out the 2 runs
of the test suite with and without -O which is also nicer.
omit a default "error" argument for NULL pointer. This allows
the parser to take a codec from cjkcodecs again.
(Reported by Taewook Kang and reviewed by Walter Doerwald)
both a subclass of Warning and a subclass of types.ClassType. The latter is no
longer true thanks to new-style exceptions.
Closes bug #1510580. Thanks to AMK for the test.
issues that crop up from time to time, but this change seems to have been
pretty stable (no spurious warnings) for about a week.
Other modules which use threads may require similar use of
threading_setup/threading_cleanup from test_support.
now works reliably. It has been updated to use automatic BerkeleyDB
deadlock detection and the bsddb.dbutils.DeadlockWrap wrapper to retry
database calls that would previously deadlock. [SF python bug #775414]
adjacent triples in the result list describe non-adjacent matching
blocks. That's _nice_ to have, and Guido said he wanted it.
Not a bugfix candidate: Guido or not ;-), this changes visible
endcase semantics (note that some tests had to change), and
nothing about this was documented before. Since it was working
as designed, and behavior was consistent with the docs, it wasn't
"a bug".
arguments in reverse, the interpreter would infinitely recourse trying to get a
coercion that worked. So put in a recursion check after a coercion is made and
the next call to attempt to use the coerced values.
Fixes bug #992017 and closes crashers/coerce.py .
Heavily revised, comprising revisions:
46640 - original trunk revision (backed out in r46655)
46647 - markup fix (backed out in r46655)
46692:46918 merged from branch aimacintyre-sf1454481
branch tested on buildbots (Windows buildbots had problems
not related to these changes).
the char buffer was requested. Now it actually returns the char buffer if
available or raises a TypeError if it isn't (as is raised for the other buffer
types if they are not present but requested).
Not a backport candidate since it does change semantics of the buffer object
(although it could be argued this is enough of a bug to bother backporting).
aborts the db transaction safely when a modifier callback fails.
Fixes SF python patch/bug #1408584.
Also cleans up the bsddb.dbtables docstrings since thats the only
documentation that exists for that unadvertised module. (people
really should really just use sqlite3)
46640 Patch #1454481: Make thread stack size runtime tunable.
46647 Markup fix
The first is causing many buildbots to fail test runs, and there
are multiple causes with seemingly no immediate prospects for
repairing them. See python-dev discussion.
Note that a branch can (and should) be created for resolving these
problems, like
svn copy svn+ssh://svn.python.org/python/trunk -r46640 svn+ssh://svn.python.org/python/branches/NEW_BRANCH
followed by merging rev 46647 to the new branch.
to each allocated block. This was using 4 bytes for each such
piece of info regardless of platform. This didn't really matter
before (proof: no bug reports, and the debug-build obmalloc would
have assert-failed if it was ever asked for a chunk of memory
>= 2**32 bytes), since container indices were plain ints. But after
the Py_ssize_t changes, it's at least theoretically possible to
allocate a list or string whose guts exceed 2**32 bytes, and the
PYMALLOC_DEBUG routines would fail then (having only 4 bytes
to record the originally requested size).
Now we use sizeof(size_t) bytes for each of a PYMALLOC_DEBUG
build's extra debugging fields. This won't make any difference
on 32-bit boxes, but will add 16 bytes to each allocation in
a debug build on a 64-bit box.
the output required more than one line. "Small" dicts got
displayed in seemingly random order (the hash-induced order
produced by dict.__repr__). None of this was documented.
Now pprint functions always sort dicts by key, and the docs
promise it.
This was proposed and agreed to during the PyCon 2006 core
sprint -- I just didn't have time for it before now.
Renames functional extension module to _functools and adds a Python
functools module so that utility functions like update_wrapper can be
added easily.
invalid file paths for the built-in import machinery which leads to
fewer open calls on startup.
Also fix issue with PEP 302 style import hooks which lead to more open()
calls than necessary.
QueryPerformanceCounter(), but we believe Win64 does
support it now. So use in time.clock().
It would be peachy if someone with a Win64 box tried
this ;-)
In rare cases of strings specifying true values near sys.maxint,
and oddball bases (not decimal or a power of 2), int(string, base)
could deliver insane answers. This repairs all such problems, and
also speeds string->int significantly. On my box, here are %
speedups for decimal strings of various lengths:
length speedup
------ -------
1 12.4%
2 15.7%
3 20.6%
4 28.1%
5 33.2%
6 37.5%
7 41.9%
8 46.3%
9 51.2%
10 19.5%
11 19.9%
12 23.9%
13 23.7%
14 23.3%
15 24.9%
16 25.3%
17 28.3%
18 27.9%
19 35.7%
Note that the difference between 9 and 10 is the difference between
short and long Python ints on a 32-bit box. The patch doesn't
actually do anything to speed conversion to long: the speedup is
due to detecting "unsigned long" overflow more quickly.
This is a bugfix candidate, but it's a non-trivial patch and it
would be painful to separate the "bug fix" from the "speed up" parts.
Applied patch zombie-frames-2.diff from sf patch 876206 with updates for
Python 2.5 and also modified to retain the free_list to avoid the 67%
slow-down in pybench recursion test. 5% speed up in function call pybench.
about "%u", "%lu" and "%zu" formats.
Since PyString_FromFormat and PyErr_Format have exactly the same rules
(both inherited from PyString_FromFormatV), it would be good if someone
with more LaTeX Fu changed one of them to just point to the other.
Their docs were way out of synch before this patch, and I just did a
mass copy+paste to repair that.
Not a backport candidate (this is a new feature).
discussion.
There are two places of documentation that still mention __context__:
Doc/lib/libstdtypes.tex -- I wasn't quite sure how to rewrite that without
spending a whole lot of time thinking about it; and whatsnew, which Andrew
usually likes to change himself.
tuples. Lots to be added, still, but this will give big-memory people
something to play with in 2.5 alpha 2, and hopefully get more people to
write these tests.
The new char-array used in ioctl calls wasn't explicitly NUL-terminated;
quite probably the cause for the test_pty failures on Solaris that we
circumvented earlier. (I wasn't able to reproduce it with this patch, but it
has been somewhat elusive to start with.)
the 2005 Summer of Code).
The revision adds a number of new mailbox classes that support adding
and removing messages; these classes also support mailbox locking and
default to using email.Message instead of rfc822.Message.
The old mailbox classes are largely left alone for backward compatibility.
The exception is the Maildir class, which was present in the old module
and now inherits from the new classes. The Maildir class's interface
is pretty simple, though, so I think it'll be compatible with existing
code.
(The change to the NEWS file also adds a missing word to a different
news item, which unfortunately required rewrapping the line.)
Python 2.4 changed ntpath.abspath to do an import
inside the function. As a result, due to Python's
import lock, anything calling abspath on Windows
(directly, or indirectly like tempfile.TemporaryFile)
hung when it was called from a thread spawned as a
side effect of importing a module.
This is a depressingly frequent problem, and
deserves a more general fix. I'm settling for
a micro-fix here because this specific one accounts
for a report of Zope Corp's ZEO hanging on Windows,
and it was an odd way to change abspath to begin
with (ntpath needs a different implementation
depending on whether we're actually running on
Windows, and the _obvious_ way to arrange for that
is not to bury a possibly-failing import _inside_
the function).
Note that if/when other micro-fixes of this kind
get made, the new Lib/test/threaded_import_hangers.py
is a convenient place to add tests for them.
much convolution <0.5 wink>. Simplified to the point that it works,
and test_threading_local no longer reports leaks under -R. Thanks
to Thomas Wouters for initial analysis.
passing a string. Martin already fixed the actual crash by ensuring
Py_UNICODE is unsigned. As discussed on python-dev, this fix
removes the possibility of creating a unicode string from a raw buffer.
There is an outstanding question of how to fix the crash in 2.4.
prepends the exception's module name to non-builtin exceptions, like
the interpreter itself does.
broke a number of doctests. should be discussed before checking in (see
discussion on python-dev).
report failures, we'll see.
Skip certain hopeless tests: compiler and logging.
compiler will likely always show varying leaks since it doesn't work
on a defined set of modules unless -u compiler is specified. But that
takes forever (we only run with -u network currently).
logging causes hangs when running with -R.
failing on one of the 32-bit buildbot boxes because of it,
due to tempting but always-wrong Python code. Users
probably have code like this too (I know I did ...).
This is based on pysqlite2.1.3, and provides a DB-API interface in
the standard library. You'll need sqlite 3.2.2 or later to build
this - if you have an earlier version, the C extension module will
not be built.
objimpl.h, pymem.h: Stop mapping PyMem_{Del, DEL} and PyMem_{Free, FREE}
to PyObject_{Free, FREE} in a release build. They're aliases for the
system free() now.
_subprocess.c/sp_handle_dealloc(): Since the memory was originally
obtained via PyObject_NEW, it must be released via PyObject_FREE (or
_DEL).
pythonrun.c, tokenizer.c, parsermodule.c: I lost count of the number of
PyObject vs PyMem mismatches in these -- it's like the specific
function called at each site was picked at random, sometimes even with
memory obtained via PyMem getting released via PyObject. Changed most
to use PyObject uniformly, since the blobs allocated are predictably
small in most cases, and obmalloc is generally faster than system
mallocs then.
If extension modules in real life prove as sloppy as Python's front
end, we'll have to revert the objimpl.h + pymem.h part of this patch.
Note that no problems will show up in a debug build (all calls still go
thru obmalloc then). Problems will show up only in a release build, most
likely segfaults.
This was a fair amount of rework of the patch. Refactored test_fork1 so it
could be reused by the new tests for wait3/4. Also made them into new style
unittests (derive from unittest.TestCase).
This patch adds a-LAW encoding to audioop and replaces the old
u-LAW encoding/decoding code with the current code from sox.
Possible issues: the code from sox uses int16_t.
Code by Lars Immisch
This is a heavily altered derivative of SF patch 1123430, Evan
Jones's heroic effort to make obmalloc return unused arenas to
the system free(), with some heuristic strategies to make it
more likley that arenas eventually _can_ be freed.
of tuple) that provides incremental decoders and encoders (a way to use
stateful codecs without the stream API). Functions
codecs.getincrementaldecoder() and codecs.getincrementalencoder() have
been added.
added message attribute compared to the previous version of Exception. It is
also a new-style class, making all exceptions now new-style. KeyboardInterrupt
and SystemExit inherit from BaseException directly. String exceptions now
raise DeprecationWarning.
Applies patch 1104669, and closes bugs 1012952 and 518846.
- New semantics for __exit__() -- it must re-raise the exception
if type is not None; the with-statement itself doesn't do this.
(See the updated PEP for motivation.)
- Added context managers to:
- file
- thread.LockType
- threading.{Lock,RLock,Condition,Semaphore,BoundedSemaphore}
- decimal.Context
- Added contextlib.py, which defines @contextmanager, nested(), closing().
- Unit tests all around; bot no docs yet.
Neal Norwitz
Gregory P. Smith
Martin v. Löwis
Brett Cannon
Lars Gustäbel
Andrew M. Kuchling
Raymond Hettinger
Georg Brandl
Vinay Sajip
Matthias Klose
Thomas Heller
Walter Dörwald
Hye-Shik Chang
Armin Rigo
Jack Diederich
Nick Coghlan
Gustavo Niemeyer
Marc-André Lemburg
Tim Peters
Thomas Wouters
Anthony Baxter
Neil Schemenauer
Kurt B. Kaiser
Guido van Rossum
Andrew MacIntyre
Bob Ippolito
Ronald Oussoren
Fred Drake
Phillip J. Eby
Matt Fleming
Barry Warsaw
Greg Ward
Peter Astrand
Gerhard Häring
Trent Mick
George Yoshida
Steve Holden
Richard Jones
David Goodger
Skip Montanaro