Commit Graph

2239 Commits

Author SHA1 Message Date
Tim Peters ead8b7ab30 SF 1055820: weakref callback vs gc vs threads
In cyclic gc, clear weakrefs to unreachable objects before allowing any
Python code (weakref callbacks or __del__ methods) to run.

This is a critical bugfix, affecting all versions of Python since weakrefs
were introduced.  I'll backport to 2.3.
2004-10-30 23:09:22 +00:00
Armin Rigo 89a39461bf Wrote down the invariants of some common objects whose structure is
exposed in header files.  Fixed a few comments in these headers.

As we might have expected, writing down invariants systematically exposed a
(minor) bug.  In this case, function objects have a writeable func_code
attribute, which could be set to code objects with the wrong number of
free variables.  Calling the resulting function segfaulted the interpreter.
Added a corresponding test.
2004-10-28 16:32:00 +00:00
Raymond Hettinger 561fbf138d SF bug #1054139: serious string hashing error in 2.4b1
_PyString_Resize() readied strings for mutation but did not invalidate
the cached hash value.
2004-10-26 01:52:37 +00:00
Marc-André Lemburg 204bd6d9d2 Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar.
Python 2.3.x candidate.
2004-10-15 07:45:05 +00:00
Raymond Hettinger fb09f0e85c Finalize the freelist of list objects. 2004-10-07 03:58:07 +00:00
Raymond Hettinger 6429a4727e Use Py_CLEAR(). Add unrelated test. 2004-09-28 01:51:35 +00:00
Raymond Hettinger aa241e0149 Checkin Tim's fix to an error discussed on python-dev.
Also, add a testcase.

Formerly, the list_extend() code used several local variables to remember
its state across iterations.  Since an iteration could call arbitrary
Python code, it was possible for the list state to be changed.  The new
code uses dynamic structure references instead of C locals.  So, they
are always up-to-date.

After list_resize() is called, its size has been updated but the new
cells are filled with NULLs.  These needed to be filled before arbitrary
iteration code was called; otherwise, that code could attempt to modify
a list that was in a semi-invalid state.  The solution was to change
the ob->size field back to a value reflecting the actual number of valid
cells.
2004-09-26 19:24:20 +00:00
Brett Cannon a5ca2e7220 Remove 'extern' declaration for _Py_SwappedOp. 2004-09-25 01:37:24 +00:00
Neil Schemenauer 927a57fbeb Ensure negative offsets cannot be passed to buffer(). When composing
buffers, compute the new buffer size based on the old buffer size.
Fixes SF bug #1034242.
2004-09-24 19:17:26 +00:00
Neil Schemenauer fb6ba07d9c Fix buffer offset calculation (need to compute it before changing
'base').  Fixes SF bug #1033720.  Move offset sanity checking to
buffer_from_memory().
2004-09-24 15:41:27 +00:00
Tim Peters e1c69b3f6f float_richcompare(): Use the new Py_IS_NAN macro to ensure that, on
platforms where that macro works, NaN compared to an int or long works
the same as NaN compared to a finite float.
2004-09-23 19:22:41 +00:00
Tim Peters 307fa78107 SF bug #513866: Float/long comparison anomaly.
When an integer is compared to a float now, the int isn't coerced to float.
This avoids spurious overflow exceptions and insane results.  This should
compute correct results, without raising spurious exceptions, in all cases
now -- although I expect that what happens when an int/long is compared to
a NaN is still a platform accident.

Note that we had potential problems here even with "short" ints, on boxes
where sizeof(long)==8.  There's #ifdef'ed code here to handle that, but
I can't test it as intended.  I tested it by changing the #ifdef to
trigger on my 32-bit box instead.

I suppose this is a bugfix candidate, but I won't backport it.  It's
long-winded (for speed) and messy (because the problem is messy).  Note
that this also depends on a previous 2.4 patch that introduced
_Py_SwappedOp[] as an extern.
2004-09-23 08:06:40 +00:00
Tim Peters f4aca755bc A static swapped_op[] array was defined in 3 different C files, & I think
I need to define it again.  Bite the bullet and define it once as an
extern, _Py_SwappedOp[].
2004-09-23 02:39:37 +00:00
Martin v. Löwis 729d47db09 Patch #1024670: Support int objects in PyLong_AsUnsignedLong[Mask]. 2004-09-20 06:17:46 +00:00
Raymond Hettinger 1be1a79ff9 SF bug #1030557: PyMapping_Check crashes when argument is NULL
Make PySequence_Check() and PyMapping_Check() handle NULL inputs.  This
goes beyond what most of the other checks do, but it is nice defensive
programming and solves the OP's problem.
2004-09-19 06:00:15 +00:00
Skip Montanaro 6543b45b0c Initialize sep and seplen to suppress warning from gcc. 2004-09-16 03:28:13 +00:00
Thomas Heller ca0d2cb66e Add a missing line continuation character. 2004-09-15 11:41:32 +00:00
Michael W. Hudson 1fd00a1b71 Make the word "module" appear in the error string for calling the
module type with silly arguments.  (The exact name can be quibbled
over, if you care).

This was partially inspired by bug #1014215 and so on, but is also
just a good idea.
2004-09-14 17:19:09 +00:00
Michael W. Hudson 1593f502e8 Move a comment back to its rightful location. 2004-09-14 17:09:47 +00:00
Walter Dörwald 065a32f550 Make the hint about the None default less ambiguous. 2004-09-14 09:45:10 +00:00
Walter Dörwald 782afc5927 Enhance the docstrings for unicode.split() and string.split()
to make it clear that it is possible to pass None as the
separator argument to get the default "any whitespace" separator.
2004-09-14 09:40:45 +00:00
Raymond Hettinger a84f3abb9e SF #1022910: Conserve memory with list.pop()
The list resizing scheme only downsized when more than 16 elements were
removed in a single step:  del a[100:120].   As a result, the list would
never shrink when popping elements off one at a time.

This patch makes it shrink whenever more than half of the space is unused.

Also, at Tim's suggestion, renamed _new_size to new_allocated.  This makes
the code easier to understand.
2004-09-12 19:53:07 +00:00
Andrew M. Kuchling 55be9eab38 Typo fix: 'comparisions' is not a word 2004-09-10 12:59:54 +00:00
Walter Dörwald 69652035bc SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
decoding incomplete input (when the input stream is temporarily exhausted).
codecs.StreamReader now implements buffering, which enables proper
readline support for the UTF-16 decoders. codecs.StreamReader.read()
has a new argument chars which specifies the number of characters to
return. codecs.StreamReader.readline() and codecs.StreamReader.readlines()
have a new argument keepends. Trailing "\n"s will be stripped from the lines
if keepends is false. Added C APIs PyUnicode_DecodeUTF8Stateful and
PyUnicode_DecodeUTF16Stateful.
2004-09-07 20:24:22 +00:00
Raymond Hettinger 75ccea3777 SF patch #1020188: Use Py_CLEAR where necessary to avoid crashes
(Contributed by Dima Dorfman)
2004-09-01 07:02:44 +00:00
Tim Peters cd97da3b1d long_pow(): Fix more instances of leaks in error cases.
Bugfix candidate -- although long_pow() is so different now I doubt a
patch would apply to 2.3.
2004-08-30 02:58:59 +00:00
Tim Peters 47e52ee0c5 SF patch 936813: fast modular exponentiation
This checkin is adapted from part 2 (of 3) of Trevor Perrin's patch set.

BACKWARD INCOMPATIBILITY:  SHIFT must now be divisible by 5.  AFAIK,
nobody will care.  long_pow() could be complicated to worm around that,
if necessary.

long_pow():
  - BUGFIX:  This leaked the base and power when the power was negative
    (and so the computation delegated to float pow).
  - Instead of doing right-to-left exponentiation, do left-to-right.  This
    is more efficient for small bases, which is the common case.
  - In addition, if the exponent is large (more than FIVEARY_CUTOFF
    digits), precompute [a**i % c for i in range(32)], and go left to
    right 5 bits at a time.
l_divmod():
  - The signature changed so that callers who don't want the quotient,
    or don't want the remainder, can pass NULL in the slot they don't
    want.  This saves them from having to declare a vrbl for unwanted
    stuff, and remembering to decref it.
long_mod(), long_div(), long_classic_div():
  - Adjust to new l_divmod() signature, and simplified as a result.
2004-08-30 02:44:38 +00:00
Tim Peters 0973b99e1c SF patch 936813: fast modular exponentiation
This checkin is adapted from part 1 (of 3) of Trevor Perrin's patch set.

x_mul()
  - sped a little by optimizing the C
  - sped a lot (~2X) if it's doing a square; note that long_pow() squares
    often
k_mul()
  - more cache-friendly now if it's doing a square
KARATSUBA_CUTOFF
  - boosted; gradeschool mult is quicker now, and it may have been too low
    for many platforms anyway
KARATSUBA_SQUARE_CUTOFF
  - new
  - since x_mul is a lot faster at squaring now, the point at which
    Karatsuba pays for squaring is much higher than for general mult
2004-08-29 22:16:50 +00:00
Tim Peters 91879ab8ea PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
need to convert str objects from the iterable to unicode.  So, if
someone set the system default encoding to something nasty enough,
the conversion process could mutate the input iterable as a side
effect, and PySequence_Fast doesn't hide that from us if the input was
a list.  IOW, can't assume the size of PySequence_Fast's result is
invariant across PyUnicode_FromObject() calls.
2004-08-27 22:35:44 +00:00
Tim Peters 05eba1fdc8 PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
much to reduce the size of the code, but greatly improves its clarity.
It's also quicker in what's probably the most common case (the argument
iterable is a list).  Against it, if the iterable isn't a list or a tuple,
a temp tuple is materialized containing the entire input sequence, and
that's a bigger temp memory burden.  Yawn.
2004-08-27 21:32:02 +00:00
Tim Peters 894c512c2f PyUnicode_Join(): Missed a spot where I intended a cast from size_t to
int.  I sure wish MS would gripe about that!  Whatever, note that the
statement above it guarantees that the cast loses no info.
2004-08-27 05:08:36 +00:00
Tim Peters 8ce9f16259 PyUnicode_Join(): Two primary aims:
1. u1.join([u2]) is u2
2. Be more careful about C-level int overflow.

Since PySequence_Fast() isn't needed to achieve #1, it's not used -- but
the code could sure be simpler if it were.
2004-08-27 01:49:32 +00:00
Raymond Hettinger d2afee47b1 Fix docstring typo. 2004-08-25 19:42:12 +00:00
Tim Peters c885443479 Stop producing or using OverflowWarning. PEP 237 thought this would
happen in 2.3, but nobody noticed it still was getting generated (the
warning was disabled by default).  OverflowWarning and
PyExc_OverflowWarning should be removed for 2.5, and left notes all over
saying so.
2004-08-25 02:14:08 +00:00
Raymond Hettinger 674f241e9c SF Patch #1007087: Return new string for single subclass joins (Bug #1001011)
(Patch contributed by Nick Coghlan.)

Now joining string subtypes will always return a string.
Formerly, if there were only one item, it was returned unchanged.
2004-08-23 23:23:54 +00:00
Martin v. Löwis 70aa1f2095 Fix repr for negative imaginary part. Fixes #1013908. 2004-08-22 21:09:15 +00:00
Martin v. Löwis bf608750ad Patch #980082: Missing INCREF in PyType_Ready. 2004-08-18 13:16:54 +00:00
Neal Norwitz f076953eb1 SF patch #1005778, Fix seg fault if list object is modified during list.index()
Backport candidate
2004-08-13 03:18:29 +00:00
Michael W. Hudson 5e897959db This is my patch
[ 1004703 ] Make func_name writable

plus fixing a couple of nits in the documentation changes spotted by MvL
and a Misc/NEWS entry.
2004-08-12 18:12:44 +00:00
Brett Cannon 651dd52b3a Previous commit was viewed as "perverse". Changed to just cast the unused
variable to void..

Thanks to Sjoerd Mullender for the suggested change.
2004-08-08 21:21:18 +00:00
Tim Peters feec4533e2 Bug 1003935: xrange overflows
Added XXX comment about why the undocumented PyRange_New() API function
is too broken to be worth the considerable pain of repairing.

Changed range_new() to stop using PyRange_New().  This fixes a variety
of bogus errors.  Nothing in the core uses PyRange_New() now.

Documented that xrange() is intended to be simple and fast, and that
CPython restricts its arguments, and length of its result sequence, to
native C longs.

Added some tests that failed before the patch, and repaired a test that
relied on a bogus OverflowError getting raised.
2004-08-08 07:17:39 +00:00
Tim Peters d976ab7caf Trimmed trailing whitespace. 2004-08-08 06:29:10 +00:00
Armin Rigo 618fbf5469 This was quite a dark bug in my recent in-place string concatenation
hack: it would resize *interned* strings in-place!  This occurred because
their reference counts do not have their expected value -- stringobject.c
hacks them.  Mea culpa.
2004-08-07 20:58:32 +00:00
Armin Rigo 79f7ad228b Fixed some compiler warnings. 2004-08-07 19:27:39 +00:00
Jeremy Hylton 4c989ddc9c Subclasses of string can no longer be interned. The semantics of
interning were not clear here -- a subclass could be mutable, for
example -- and had bugs.  Explicitly interning a subclass of string
via intern() will raise a TypeError.  Internal operations that attempt
to intern a string subclass will have no effect.

Added a few tests to test_builtin that includes the old buggy code and
verifies that calls like PyObject_SetAttr() don't fail.  Perhaps these
tests should have gone in test_string.
2004-08-07 19:20:05 +00:00
Raymond Hettinger 2a7dedef9e SF bug #1004669: Type returned from .keys() is not checked 2004-08-07 04:55:30 +00:00
Hye-Shik Chang e9ddfbb412 SF #989185: Drop unicode.iswide() and unicode.width() and add
unicodedata.east_asian_width().  You can still implement your own
simple width() function using it like this:
    def width(u):
        w = 0
        for c in unicodedata.normalize('NFC', u):
            cwidth = unicodedata.east_asian_width(c)
            if cwidth in ('W', 'F'): w += 2
            else: w += 1
        return w
2004-08-04 07:38:35 +00:00
Fred Drake 6d3265dab6 Be more careful about maintaining the invariants; it was actually
possible that the callback-less flavors of the ref or proxy could have
been added during GC, so we don't want to replace them.
2004-08-03 14:47:25 +00:00
Michael W. Hudson 3f3b66823f Repair the same thinko in two places about handling of _Py_RefTotal in
the case of __del__ resurrecting an object.
This makes the apparent reference leaks in test_descr go away (which I
expected) and also kills off those in test_gc (which is more surprising
but less so once you actually think about it a bit).
2004-08-03 10:21:03 +00:00
Brett Cannon 5ad28e14b6 Tweak previous patch to silence a warning about the unused left value in the
comma expression in listpop() that was being returned.  Still essentially
unused (as it is meant to be), but now the compiler thinks it is worth
*something* by having it incremented.
2004-08-03 04:53:29 +00:00