Commit Graph

2471 Commits

Author SHA1 Message Date
Michael W. Hudson 1593f502e8 Move a comment back to its rightful location. 2004-09-14 17:09:47 +00:00
Walter Dörwald 065a32f550 Make the hint about the None default less ambiguous. 2004-09-14 09:45:10 +00:00
Walter Dörwald 782afc5927 Enhance the docstrings for unicode.split() and string.split()
to make it clear that it is possible to pass None as the
separator argument to get the default "any whitespace" separator.
2004-09-14 09:40:45 +00:00
Raymond Hettinger a84f3abb9e SF #1022910: Conserve memory with list.pop()
The list resizing scheme only downsized when more than 16 elements were
removed in a single step:  del a[100:120].   As a result, the list would
never shrink when popping elements off one at a time.

This patch makes it shrink whenever more than half of the space is unused.

Also, at Tim's suggestion, renamed _new_size to new_allocated.  This makes
the code easier to understand.
2004-09-12 19:53:07 +00:00
Andrew M. Kuchling 55be9eab38 Typo fix: 'comparisions' is not a word 2004-09-10 12:59:54 +00:00
Walter Dörwald 69652035bc SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
decoding incomplete input (when the input stream is temporarily exhausted).
codecs.StreamReader now implements buffering, which enables proper
readline support for the UTF-16 decoders. codecs.StreamReader.read()
has a new argument chars which specifies the number of characters to
return. codecs.StreamReader.readline() and codecs.StreamReader.readlines()
have a new argument keepends. Trailing "\n"s will be stripped from the lines
if keepends is false. Added C APIs PyUnicode_DecodeUTF8Stateful and
PyUnicode_DecodeUTF16Stateful.
2004-09-07 20:24:22 +00:00
Raymond Hettinger 75ccea3777 SF patch #1020188: Use Py_CLEAR where necessary to avoid crashes
(Contributed by Dima Dorfman)
2004-09-01 07:02:44 +00:00
Tim Peters cd97da3b1d long_pow(): Fix more instances of leaks in error cases.
Bugfix candidate -- although long_pow() is so different now I doubt a
patch would apply to 2.3.
2004-08-30 02:58:59 +00:00
Tim Peters 47e52ee0c5 SF patch 936813: fast modular exponentiation
This checkin is adapted from part 2 (of 3) of Trevor Perrin's patch set.

BACKWARD INCOMPATIBILITY:  SHIFT must now be divisible by 5.  AFAIK,
nobody will care.  long_pow() could be complicated to worm around that,
if necessary.

long_pow():
  - BUGFIX:  This leaked the base and power when the power was negative
    (and so the computation delegated to float pow).
  - Instead of doing right-to-left exponentiation, do left-to-right.  This
    is more efficient for small bases, which is the common case.
  - In addition, if the exponent is large (more than FIVEARY_CUTOFF
    digits), precompute [a**i % c for i in range(32)], and go left to
    right 5 bits at a time.
l_divmod():
  - The signature changed so that callers who don't want the quotient,
    or don't want the remainder, can pass NULL in the slot they don't
    want.  This saves them from having to declare a vrbl for unwanted
    stuff, and remembering to decref it.
long_mod(), long_div(), long_classic_div():
  - Adjust to new l_divmod() signature, and simplified as a result.
2004-08-30 02:44:38 +00:00
Tim Peters 0973b99e1c SF patch 936813: fast modular exponentiation
This checkin is adapted from part 1 (of 3) of Trevor Perrin's patch set.

x_mul()
  - sped a little by optimizing the C
  - sped a lot (~2X) if it's doing a square; note that long_pow() squares
    often
k_mul()
  - more cache-friendly now if it's doing a square
KARATSUBA_CUTOFF
  - boosted; gradeschool mult is quicker now, and it may have been too low
    for many platforms anyway
KARATSUBA_SQUARE_CUTOFF
  - new
  - since x_mul is a lot faster at squaring now, the point at which
    Karatsuba pays for squaring is much higher than for general mult
2004-08-29 22:16:50 +00:00
Tim Peters 91879ab8ea PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
need to convert str objects from the iterable to unicode.  So, if
someone set the system default encoding to something nasty enough,
the conversion process could mutate the input iterable as a side
effect, and PySequence_Fast doesn't hide that from us if the input was
a list.  IOW, can't assume the size of PySequence_Fast's result is
invariant across PyUnicode_FromObject() calls.
2004-08-27 22:35:44 +00:00
Tim Peters 05eba1fdc8 PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
much to reduce the size of the code, but greatly improves its clarity.
It's also quicker in what's probably the most common case (the argument
iterable is a list).  Against it, if the iterable isn't a list or a tuple,
a temp tuple is materialized containing the entire input sequence, and
that's a bigger temp memory burden.  Yawn.
2004-08-27 21:32:02 +00:00
Tim Peters 894c512c2f PyUnicode_Join(): Missed a spot where I intended a cast from size_t to
int.  I sure wish MS would gripe about that!  Whatever, note that the
statement above it guarantees that the cast loses no info.
2004-08-27 05:08:36 +00:00
Tim Peters 8ce9f16259 PyUnicode_Join(): Two primary aims:
1. u1.join([u2]) is u2
2. Be more careful about C-level int overflow.

Since PySequence_Fast() isn't needed to achieve #1, it's not used -- but
the code could sure be simpler if it were.
2004-08-27 01:49:32 +00:00
Raymond Hettinger d2afee47b1 Fix docstring typo. 2004-08-25 19:42:12 +00:00
Tim Peters c885443479 Stop producing or using OverflowWarning. PEP 237 thought this would
happen in 2.3, but nobody noticed it still was getting generated (the
warning was disabled by default).  OverflowWarning and
PyExc_OverflowWarning should be removed for 2.5, and left notes all over
saying so.
2004-08-25 02:14:08 +00:00
Raymond Hettinger 674f241e9c SF Patch #1007087: Return new string for single subclass joins (Bug #1001011)
(Patch contributed by Nick Coghlan.)

Now joining string subtypes will always return a string.
Formerly, if there were only one item, it was returned unchanged.
2004-08-23 23:23:54 +00:00
Martin v. Löwis 70aa1f2095 Fix repr for negative imaginary part. Fixes #1013908. 2004-08-22 21:09:15 +00:00
Martin v. Löwis bf608750ad Patch #980082: Missing INCREF in PyType_Ready. 2004-08-18 13:16:54 +00:00
Neal Norwitz f076953eb1 SF patch #1005778, Fix seg fault if list object is modified during list.index()
Backport candidate
2004-08-13 03:18:29 +00:00
Michael W. Hudson 5e897959db This is my patch
[ 1004703 ] Make func_name writable

plus fixing a couple of nits in the documentation changes spotted by MvL
and a Misc/NEWS entry.
2004-08-12 18:12:44 +00:00
Brett Cannon 651dd52b3a Previous commit was viewed as "perverse". Changed to just cast the unused
variable to void..

Thanks to Sjoerd Mullender for the suggested change.
2004-08-08 21:21:18 +00:00
Tim Peters feec4533e2 Bug 1003935: xrange overflows
Added XXX comment about why the undocumented PyRange_New() API function
is too broken to be worth the considerable pain of repairing.

Changed range_new() to stop using PyRange_New().  This fixes a variety
of bogus errors.  Nothing in the core uses PyRange_New() now.

Documented that xrange() is intended to be simple and fast, and that
CPython restricts its arguments, and length of its result sequence, to
native C longs.

Added some tests that failed before the patch, and repaired a test that
relied on a bogus OverflowError getting raised.
2004-08-08 07:17:39 +00:00
Tim Peters d976ab7caf Trimmed trailing whitespace. 2004-08-08 06:29:10 +00:00
Armin Rigo 618fbf5469 This was quite a dark bug in my recent in-place string concatenation
hack: it would resize *interned* strings in-place!  This occurred because
their reference counts do not have their expected value -- stringobject.c
hacks them.  Mea culpa.
2004-08-07 20:58:32 +00:00
Armin Rigo 79f7ad228b Fixed some compiler warnings. 2004-08-07 19:27:39 +00:00
Jeremy Hylton 4c989ddc9c Subclasses of string can no longer be interned. The semantics of
interning were not clear here -- a subclass could be mutable, for
example -- and had bugs.  Explicitly interning a subclass of string
via intern() will raise a TypeError.  Internal operations that attempt
to intern a string subclass will have no effect.

Added a few tests to test_builtin that includes the old buggy code and
verifies that calls like PyObject_SetAttr() don't fail.  Perhaps these
tests should have gone in test_string.
2004-08-07 19:20:05 +00:00
Raymond Hettinger 2a7dedef9e SF bug #1004669: Type returned from .keys() is not checked 2004-08-07 04:55:30 +00:00
Hye-Shik Chang e9ddfbb412 SF #989185: Drop unicode.iswide() and unicode.width() and add
unicodedata.east_asian_width().  You can still implement your own
simple width() function using it like this:
    def width(u):
        w = 0
        for c in unicodedata.normalize('NFC', u):
            cwidth = unicodedata.east_asian_width(c)
            if cwidth in ('W', 'F'): w += 2
            else: w += 1
        return w
2004-08-04 07:38:35 +00:00
Fred Drake 6d3265dab6 Be more careful about maintaining the invariants; it was actually
possible that the callback-less flavors of the ref or proxy could have
been added during GC, so we don't want to replace them.
2004-08-03 14:47:25 +00:00
Michael W. Hudson 3f3b66823f Repair the same thinko in two places about handling of _Py_RefTotal in
the case of __del__ resurrecting an object.
This makes the apparent reference leaks in test_descr go away (which I
expected) and also kills off those in test_gc (which is more surprising
but less so once you actually think about it a bit).
2004-08-03 10:21:03 +00:00
Brett Cannon 5ad28e14b6 Tweak previous patch to silence a warning about the unused left value in the
comma expression in listpop() that was being returned.  Still essentially
unused (as it is meant to be), but now the compiler thinks it is worth
*something* by having it incremented.
2004-08-03 04:53:29 +00:00
Michael W. Hudson f8df9a89bc Add a missing decref. 2004-08-02 13:22:01 +00:00
Tim Peters 8fc4a91665 list_ass_slice(): Document the obscure new intent that deleting a slice
of no more than 8 elements cannot fail.

listpop():  Take advantage of that its calls to list_resize() and
list_ass_slice() can't fail.  This is assert'ed in a debug build now, but
in an icky way.  That is, you can't say:

	assert(some_call() >= 0);

because then some_call() won't occur at all in a release build.  So it
has to be a big pile of #ifdefs on Py_DEBUG (yuck), or the pleasant:

        status = some_call();
        assert(status >= 0);

But in that case, compilers may whine in a release build, because status
appears unused then.  I'm not certain the ugly trick I used here will
convince all compilers to shut up about status (status is always "used" now,
as the first (ignored) clause in a comma expression).
2004-07-31 21:53:19 +00:00
Tim Peters 7357222d0e list_ass_slice(): The difference between "recycle" and "recycled" was
impossible to remember, so renamed one to something obvious.  Headed
off potential signed-vs-unsigned compiler complaints I introduced by
changing the type of a vrbl to unsigned.  Removed the need for the
tedious explanation about "backward pointer loops" by looping on an
int instead.
2004-07-31 02:54:42 +00:00
Tim Peters 8d9eb10c29 Armin asked for a list_ass_slice review in his checkin, so here's the
result.

list_resize():  Document the intent.  Code is increasingly relying on
subtle aspects of its behavior, and they deserve to be spelled out.

list_ass_slice():  A bit more simplification, by giving it a common
error exit and initializing more values.

Be clearer in comments about what "size" means (# of elements?  # of
bytes?).

While the number of elements in a list slice must fit in an int, there's
no guarantee that the number of bytes occupied by the slice will.  That
malloc() and memmove() take size_t arguments is a hint about that <wink>.
So changed to use size_t where appropriate.

ihigh - ilow should always be >= 0, but we never asserted that.  We do
now.

The loop decref'ing the recycled slice had a subtle insecurity:  C doesn't
guarantee that a pointer one slot *before* an array will compare "less
than" to a pointer within the array (it does guarantee that a pointer
one beyond the end of the array compares as expected).  This was actually
an issue in KSR's C implementation, so isn't purely theoretical.  Python
probably has other "go backwards" loops with a similar glitch.
list_clear() is OK (it marches an integer backwards, not a pointer).
2004-07-31 02:24:20 +00:00
Armin Rigo 1dd04a02e0 This is a reorganization of list_ass_slice(). It should probably be reviewed,
though I tried to be very careful.  This is a slight simplification, and it
adds a new feature: a small stack-allocated "recycled" array for the cases
when we don't remove too many items.

It allows PyList_SetSlice() to never fail if:
* you are sure that the object is a list; and
* you either do not remove more than 8 items, or clear the list.

This makes a number of other places in the source code correct again -- there
are some places that delete a single item without checking for MemoryErrors
raised by PyList_SetSlice(), or that clear the whole list, and sometimes the
context doesn't allow an error to be propagated.
2004-07-30 11:38:22 +00:00
Armin Rigo a37bbf2e5b What if you call lst.__init__() while it is being sorted? :-)
The invariant checks would break.
2004-07-30 11:20:18 +00:00
Raymond Hettinger c0aaa2db4f * Simplify and speed-up list_resize(). Relying on the newly documented
invariants allows the ob_item != NULL check to be replaced with an
  assertion.

* Added assertions to list_init() which document and verify that the
  tp_new slot establishes the invariants.  This may preclude a future
  bug if a custom tp_new slot is written.
2004-07-29 23:31:29 +00:00
Armin Rigo 93677f075d * drop the unreasonable list invariant that ob_item should never come back
to NULL during the lifetime of the object.

* listobject.c nevertheless did not conform to the other invariants,
  either; fixed.

* listobject.c now uses list_clear() as the obvious internal way to clear
  a list, instead of abusing list_ass_slice() for that.  It makes it easier
  to enforce the invariant about ob_item == NULL.

* listsort() sets allocated to -1 during sort; any mutation will set it
  to a value >= 0, so it is a safe way to detect mutation.  A negative
  value for allocated does not cause a problem elsewhere currently.
  test_sort.py has a new test for this fix.

* listsort() leak: if items were added to the list during the sort, AND if
  these items had a __del__ that puts still more stuff into the list,
  then this more stuff (and the PyObject** array to hold them) were
  overridden at the end of listsort() and never released.
2004-07-29 12:40:23 +00:00
Armin Rigo f414fc4004 Minor memory leak. 2004-07-29 10:56:55 +00:00
Tim Peters 51b4ade306 Fix obscure breakage (relative to 2.3) in listsort: the test for list
mutation during list.sort() used to rely on that listobject.c always
NULL'ed ob_item when ob_size fell to 0.  That's no longer true, so the
test for list mutation during a sort is no longer reliable.  Changed the
test to rely instead on that listobject.c now never NULLs-out ob_item
after (if ever) ob_item gets a non-NULL value.  This new assumption is
also documented now, as a required invariant in listobject.h.

The new assumption allowed some real simplification to some of the
hairier code in listsort(), so is a Good Thing on that count.
2004-07-29 04:07:15 +00:00
Tim Peters b38e2b61b3 Trimmed trailing whitespace. 2004-07-29 02:29:26 +00:00
Tim Peters 3986d4e660 PyList_New(): we went to all the trouble of computing and bounds-checking
the size_t nbytes, and passed nbytes to malloc, so it was confusing to
effectively recompute the same thing from scratch in the memset call.
2004-07-29 02:28:42 +00:00
Marc-André Lemburg d25c650461 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__(). 2004-07-23 16:13:25 +00:00
Neil Schemenauer 3a313e3655 Check the type of values returned by __int__, __float__, __long__,
__oct__, and __hex__.  Raise TypeError if an invalid type is
returned.  Note that PyNumber_Int and PyNumber_Long can still
return ints or longs.  Fixes SF bug #966618.
2004-07-19 16:29:17 +00:00
Nicholas Bastin 9ba301e589 Moved SunPro warning suppression into pyport.h and out of individual
modules and objects.
2004-07-15 15:54:05 +00:00
Marc-André Lemburg 126b44cd41 Fix a copy&paste typo. 2004-07-10 12:04:20 +00:00
Marc-André Lemburg 1dffb120b7 .encode()/.decode() patch part 2. 2004-07-08 19:13:55 +00:00
Marc-André Lemburg d2d4598ec2 Allow string and unicode return types from .encode()/.decode()
methods on string and unicode objects. Added unicode.decode()
which was missing for no apparent reason.
2004-07-08 17:57:32 +00:00
Neal Norwitz 739a8f86d6 Fix a couple of signed/unsigned comparison warnings 2004-07-08 01:55:58 +00:00
Neal Norwitz 93468eac72 Remove unused macros in .c files 2004-07-08 01:49:00 +00:00
Neal Norwitz bdcb9410c2 SF bug #978308, Spurious errors taking bool of dead pro
Need to return -1 on error.

Needs backport.
2004-07-08 01:22:31 +00:00
Fred Drake 0a4dd390bf Make weak references subclassable:
- weakref.ref and weakref.ReferenceType will become aliases for each
  other

- weakref.ref will be a modern, new-style class with proper __new__
  and __init__ methods

- weakref.WeakValueDictionary will have a lighter memory footprint,
  using a new weakref.ref subclass to associate the key with the
  value, allowing us to have only a single object of overhead for each
  dictionary entry (currently, there are 3 objects of overhead per
  entry: a weakref to the value, a weakref to the dictionary, and a
  function object used as a weakref callback; the weakref to the
  dictionary could be avoided without this change)

- a new macro, PyWeakref_CheckRefExact(), will be added

- PyWeakref_CheckRef() will check for subclasses of weakref.ref

This closes SF patch #983019.
2004-07-02 18:57:45 +00:00
Raymond Hettinger 214b1c3aae SF Bug #215126: Over restricted type checking on eval() function
The builtin eval() function now accepts any mapping for the locals argument.
Time sensitive steps guarded by PyDict_CheckExact() to keep from slowing
down the normal case.  My timings so no measurable impact.
2004-07-02 06:41:07 +00:00
Tim Peters e7c053233f sizeof(char) is 1, by definition, so get rid of that expression in
places it's just noise.
2004-06-27 17:24:49 +00:00
Martin v. Löwis 8d97e33bb7 Patch #966493: Cleanup generator/eval_frame exposure. 2004-06-27 15:43:12 +00:00
Raymond Hettinger a006c37472 SF bug #980419: int left-shift causes memory leak 2004-06-26 23:22:57 +00:00
Raymond Hettinger 8d726eef96 Cosmetic spacing fix. 2004-06-25 22:24:35 +00:00
Raymond Hettinger d56cbe57b8 Fix leak found by Eric Huss. 2004-06-25 22:17:39 +00:00
Nicholas Bastin 9e1bfe7dd9 Disabling end-of-loop code not reached warning on SunPro 2004-06-18 19:57:13 +00:00
Nicholas Bastin 1ce9e4cfc1 Fixed end-of-loop code not reached warning when using SunPro C 2004-06-17 18:27:18 +00:00
Raymond Hettinger 148a63f1fc Remove a function no longer in use. 2004-06-14 04:24:41 +00:00
Raymond Hettinger 47edb4b09c Remove unnecessary GC support. Sets cannot have cycles. 2004-06-13 08:20:46 +00:00
Raymond Hettinger 6c7a00fbaa * Factor out PyObject_SelfIter().
* Change a XDECREF to DECREF (adding an assertion just to be sure).
2004-06-12 05:17:55 +00:00
Anthony Baxter 3ecdb250af Fix for bug #966623 - classes created with type() in an exec(, {}) don't
have a __module__. Test for this case.

Bugfix candidate, will backport.
2004-06-11 14:41:18 +00:00
Skip Montanaro 51ffac6db7 dump HAVE_FOPENRF stuff - obsolete 2004-06-11 04:49:03 +00:00
Raymond Hettinger c978633ec6 Futher improvements to frozenset hashing (based on Yitz Gale's battery of
tests which nicely highly highlight weaknesses).

* Initial value is now a large prime.
* Pre-multiply by the set length to add one more basis of differentiation.
* Work a bit harder inside the loop to scatter bits from sources that
  may have closely spaced hash values.

All of this is necessary to make up for keep the hash function commutative.
Fortunately, the hash value is cached so the call to frozenset_hash() will
only occur once per set.
2004-06-10 22:41:48 +00:00
Raymond Hettinger 27e403ebe9 Fixups to the hash function for frozensets.
* Non-zero initial value so that hash(frozenset()) != hash(0).
* Final permutation to differentiate nested sets.
* Add logic to make sure that -1 is not a possible hash value.
2004-06-10 21:38:41 +00:00
Raymond Hettinger 57c2d930f6 Add a final permutation step to the tuple hash function.
Prevents a collision pattern that occurs with nested tuples.
(Yitz Gale provided code that repeatably demonstrated the weakness.)
2004-06-10 18:42:15 +00:00
Martin v. Löwis 737ea82a5a Patch #774665: Make Python LC_NUMERIC agnostic. 2004-06-08 18:52:54 +00:00
Neal Norwitz b5d7702e5c whoops, I wanted that commented out by default, will add doc to Misc 2004-06-06 19:21:34 +00:00
Neal Norwitz 7eb3c9196d SF bug 881641, make it easier to use valgrind 2004-06-06 19:20:22 +00:00
Andrew M. Kuchling 41627bf78d Reword message 2004-06-05 19:49:12 +00:00
Andrew M. Kuchling 204d7861df Fix exception wording 2004-06-05 19:29:41 +00:00
Raymond Hettinger 4ec44e851d Replaced arbitrary addend in tuple_hash with one that is known to generate
many more prime multipliers and that performs well on collision tests.
2004-06-04 06:35:20 +00:00
Hye-Shik Chang 974ed7cfa5 - SF #962502: Add two more methods for unicode type; width() and
iswide() for east asian width manipulation. (Inspired by David
Goodger, Reviewed by Martin v. Loewis)
- Move _PyUnicode_TypeRecord.flags to the end of the struct so that
no padding is added for UCS-4 builds. (Suggested by Martin v. Loewis)
2004-06-02 16:49:17 +00:00
Martin v. Löwis e440e47e91 Patch #957398: Add public API for Generator Object/Type. 2004-06-01 15:22:42 +00:00
Raymond Hettinger 41bd02256f SF bug #942952: Weakness in tuple hash
(Basic approach and test concept by Tim Peters.)

* Improved the hash to reduce collisions.
* Added the torture test to the test suite.
2004-06-01 06:36:24 +00:00
Raymond Hettinger cb87bc8e7e Add weakref support to array.array and file objects. 2004-05-31 00:35:52 +00:00
Raymond Hettinger 691d80532b Make sets and deques weak referencable. 2004-05-30 07:26:47 +00:00
Walter Dörwald d70ad8a9d9 Update docstring for dict.update() to match the new realities. 2004-05-28 20:59:21 +00:00
Michael W. Hudson 08678a1055 Remove float_compare as per
[ 899109 ] 1==float('nan')

which can now finally be closed, I think.
2004-05-26 17:36:12 +00:00
Raymond Hettinger 10c660673e SF bug #952866: "can't multiply sequence *by* non-int"
Minor wording fix.
2004-05-12 21:35:06 +00:00
Raymond Hettinger fdfe618228 Nits:
- Neatened the braces in PyList_New().
- Made sure "indexerr" was initialized to NULL.
- Factored if blocks in PyList_Append().
- Made sure "allocated" is initialized in list_init().
2004-05-05 06:28:16 +00:00
Raymond Hettinger 0468e416c1 SF patch #947476: Apply freelist technique to lists
Re-use list object bodies.  Saves calls to malloc() and free() for
faster list instantiation and deallocation.
2004-05-05 05:37:53 +00:00
Thomas Heller 1328b52c6f Two new public API functions, Py_IncRef and Py_DecRef. Useful for
dynamic embedders of Python.
2004-04-22 17:23:49 +00:00
Raymond Hettinger 7892b1c651 * Add unittests for iterators that report their length
* Document the differences between them
* Fix corner cases covered by the unittests
* Use Py_RETURN_NONE where possible for dictionaries
2004-04-12 18:10:01 +00:00
Raymond Hettinger 45d0b5cc44 Use Py_RETURN_NONE macro where applicable. 2004-04-12 17:21:03 +00:00
Raymond Hettinger 501f02cd02 Small refactoring saving one function() and eliminating some indirection.
* Applied app1() to listappend().
* Inlined ins() into its one remaining caller.
2004-04-12 14:01:16 +00:00
Raymond Hettinger 40a03821ae * Specialize ins1() into app1() for appends. Saves several unnecessary
steps and further improves the speed of list append.

* Add guards to the list iterator length method to handle corner cases.
2004-04-12 13:05:09 +00:00
Hye-Shik Chang 4057483164 SF Patch #926375: Remove a useless UTF-16 support code that is never
been used. (Suggested by Martin v. Loewis)
2004-04-06 07:24:51 +00:00
Raymond Hettinger ed9192e2ae Improve previous checkin to use a slot check instead of equivalent
attribute name lookup.
2004-04-05 08:14:48 +00:00
Raymond Hettinger e2eda606a8 Improve accuracy of sequence and mapping checks. 2004-04-04 08:51:41 +00:00
Andrew MacIntyre 4e10ed3b86 If a file is opened with an explicit buffer size >= 1, repeated
close() calls would attempt to free() the buffer already free()ed on
the first close().     [bug introduced with patch #788249]

Making sure that the buffer is free()ed in file object deallocation is
a belt-n-braces bit of insurance against a memory leak.
2004-04-04 07:01:35 +00:00
Hye-Shik Chang ff365c931b Get rid of gcc warning. 2004-03-25 16:37:03 +00:00
Martin v. Löwis 093c100bd4 Correct code to advance ptr to be well-formed C. 2004-03-25 16:16:28 +00:00
Phillip J. Eby 91a968af76 Ensure super() lookup of descriptor from classmethod works (SF #743627) 2004-03-25 02:19:34 +00:00
Martin v. Löwis 321c9ab74c Intern __name__. 2004-03-23 18:40:15 +00:00
Armin Rigo 6fce78e07f Restored revision 2.87. 2004-03-21 22:29:05 +00:00
Tim Peters 1c3fd875b9 PyTuple_New(): vrbl i no longer referenced, so removed it (which kills
off a new compiler wng under MSVC6).
2004-03-21 21:35:41 +00:00
Armin Rigo 56716150e6 This is the fastest I could get on Intel GCC. I kept the memset() in to clear
the newly created tuples, but tuples added in the freelist are now cleared in
tupledealloc already (which is very cheap, because we are already
Py_XDECREF'ing all elements anyway).

Python should have a standard Py_ZAP macro like ZAP in pystate.c.
2004-03-21 20:27:49 +00:00
Nicholas Bastin abce8a681c Changed file.name to be the object passed as the 'name' argument to file()
Fixes SF Bug #773356
2004-03-21 20:24:07 +00:00
Raymond Hettinger 8183fa46a9 Fix typo in comment. 2004-03-21 17:35:06 +00:00
Raymond Hettinger 93d448198b Add identity shortcut to PyObject_RichCompareBool. 2004-03-21 17:01:44 +00:00
Tim Peters 5f112eb43b recursive_isinstance(), recursive_issubclass(): New code here returned
NULL in case of error, but the functions are declared to return int.
MSVC 6 properly complains about that.  Return -1 on error instead.
2004-03-21 16:59:09 +00:00
Brett Cannon 4f65331483 Limit the nesting depth of a tuple passed as the second argument to
isinstance() or issubclass() to the recursion limit of the interpreter.
2004-03-20 22:52:14 +00:00
Armin Rigo 70d172dda4 Get rid of listextend_internal() and explain why the special case
'a.extend(a)' isn't so special anyway.
2004-03-20 22:19:23 +00:00
Armin Rigo 7cdf3e8a8a memset() hunt continuing. This is a net win. 2004-03-20 21:35:09 +00:00
Armin Rigo 75be012cba memset() with small memory sizes just kill us. 2004-03-20 21:10:27 +00:00
Guido van Rossum 09240f65f8 GCC was complaining that 'value' in dictiter_iternextvalue() wasn't
necessarily always set before used.  Between Tim, Armin & me we
couldn't prove GCC wrong, so we decided to fix the algorithm.  This
version is Armin's.
2004-03-20 19:11:58 +00:00
Fred Drake 086a0f79cd PyFile_WriteObject(): some of the local variables are only used when
Py_USING_UNICODE is defined
2004-03-19 15:22:36 +00:00
Raymond Hettinger 0690512a7d Factor out a double lookup. 2004-03-19 10:30:00 +00:00
Raymond Hettinger 435bf58b7b Make iterators length transparent where possible. 2004-03-18 22:43:10 +00:00
Raymond Hettinger 0ce6dc8530 Make the new dictionary iterators transparent with respect to length.
This gives another 30% speedup for operations such as
map(func, d.iteritems()) or list(d.iteritems()) which can both take
advantage of length information when provided.
2004-03-18 08:38:00 +00:00
Raymond Hettinger 019a148c72 Optimize dictionary iterators.
* Split into three separate types that share everything except the
  code for iternext.  Saves run time decision making and allows
  each iternext function to be specialized.

* Inlined PyDict_Next().  In addition to saving a function call, this
  allows a redundant test to be eliminated and further specialization
  of the code for the unique needs of each iterator type.

* Created a reusable result tuple for iteritems().  Saves the malloc
  time for tuples when the previous result was not kept by client code
  (this is the typical use case for iteritems).  If the client code
  does keep the reference, then a new tuple is created.

Results in a 20% to 30% speedup depending on the size and sparsity
of the dictionary.
2004-03-18 02:41:19 +00:00
Raymond Hettinger 4344278250 Dictionary optimizations:
* Factored constant structure references out of the inner loops for
  PyDict_Next(), dict_keys(), dict_values(), and dict_items().
  Gave measurable speedups to each (the improvement varies depending
  on the sparseness of the dictionary being measured).

* Added a freelist scheme styled after that for tuples.  Saves around
  80% of the calls to malloc and free.  About 10% of the time, the
  previous dictionary was completely empty; in those cases, the
  dictionary initialization with memset() can be skipped.
2004-03-17 21:55:03 +00:00
Raymond Hettinger 969d8c0c8c Add missing decref 2004-03-17 05:24:23 +00:00
Raymond Hettinger 9d5c44307a Fix typos and add some elaborations 2004-03-15 15:52:22 +00:00
Raymond Hettinger d4ff741e78 Revert last change. Found an application that was worse off with resize
exact turned on.  The tiny space savings wasn't worth the additional time
and code.
2004-03-15 09:01:31 +00:00
Raymond Hettinger 325d169a54 Eliminate an unnecessary test on a common code path. 2004-03-15 00:16:34 +00:00
Raymond Hettinger 0e91643bd2 list_resize() now has an "exact" option for bypassing the overallocation
scheme in situations that likely won't benefit from it.  This further
improves memory utilization from Py2.3 which always over-allocates
except for PyList_New().

Situations expected to benefit from over-allocation:
    list.insert(), list.pop(), list.append(), and list.extend()

Situations deemed unlikely to benefit:
    list_inplace_repeat, list_ass_slice, list_ass_subscript

The most gray area was for listextend_internal() which only runs
when the argument is a list or a tuple.  This could be viewed as
a one-time fixed length addition or it could be viewed as wrapping
a series of appends.  I left its over-allocation turned on but
could be convinced otherwise.
2004-03-14 06:42:23 +00:00
Raymond Hettinger 42bec93e5c Make PySequence_Fast_ITEMS public. (Thanks Skip.) 2004-03-12 16:38:17 +00:00
Raymond Hettinger 6e058d70ef * Eliminate duplicate call to PyObject_Size().
(Spotted by Michael Hudson.)

* Now that "selflen" is no longer inside a loop, it should not be a
  register variable.
2004-03-12 15:30:38 +00:00
Raymond Hettinger c1e4f9dd92 Use a new macro, PySequence_Fast_ITEMS to factor out code common to
three recent optimizations.  Aside from reducing code volume, it
increases readability.
2004-03-12 08:04:00 +00:00
Raymond Hettinger 57c4542bcd Now that list.extend() is at the root of many list operations, it becomes
worth it to in-line the call to PyIter_Next().

Saves another 15% on most list operations that acceptable a general
iterable argument (such as the list constructor).
2004-03-11 09:48:18 +00:00
Raymond Hettinger 8ca92ae54c Eliminate a big block of duplicate code in PySequence_List() by
exposing _PyList_Extend().
2004-03-11 09:13:12 +00:00
Raymond Hettinger 97bc618229 list_inplace_concat() is now expressed in terms of list_extend() which
avoids creating an intermediate tuple for iterable arguments other than
lists or tuples.

In other words, a+=b no longer requires extra memory when b is not a
list or tuple.  The list and tuple cases are unchanged.
2004-03-11 07:34:19 +00:00
Neil Schemenauer 4252a7a5d1 Make buffer objects based on mutable objects (like array) safe. 2004-03-11 02:42:45 +00:00
Neil Schemenauer 0eadcd9cbb Document one of the many problems with the buffer object. 2004-03-11 01:00:44 +00:00
Neil Schemenauer 5e3a675b6d Rename static functions, they should not have the _Py prefix. 2004-03-11 00:44:54 +00:00
Raymond Hettinger 66d31f8f38 Use memcpy() instead of memmove() when the buffers are known to be distinct. 2004-03-10 11:44:04 +00:00
Raymond Hettinger ef9bf4031a Tidied up the implementations of reversed (including the custom ones
for xrange and list objects).

* list.__reversed__ now checks the length of the sequence object before
  calling PyList_GET_ITEM() because the mutable could have changed length.

* all three implementations are now tranparent with respect to length and
  maintain the invariant len(it) == len(list(it)) even when the underlying
  sequence mutates.

* __builtin__.reversed() now frees the underlying sequence as soon
  as the iterator is exhausted.

* the code paths were rearranged so that the most common paths
  do not require a jump.
2004-03-10 10:10:42 +00:00
Raymond Hettinger d2c36261a2 Eliminate the double reverse option. It's only use case
was academic and it was potentially confusing to use.
2004-03-10 08:32:47 +00:00
Raymond Hettinger a6366fe085 Optimize inner loops for subscript, repeat, and concat. 2004-03-09 13:05:22 +00:00
Raymond Hettinger f889e10c19 Optimize slice assignments.
* Replace sprintf message with a constant message string -- this error
  message ran on every invocation except straight deletions but it was
  only needed when the rhs was not iterable.  The message was also
  out-of-date and did not reflect that iterable arguments were allowed.

* For inner loops that do not make ref count adjustments, use memmove()
  for fast copying and better readability.

* For inner loops that do make ref count adjustments, speed them up by
  factoring out the constant structure reference and using vitem[] instead.
2004-03-09 08:04:33 +00:00
Raymond Hettinger 3fd500b4a5 The copy module now handles sets directly. The __copy__ methods are no
longer needed.
2004-03-08 18:31:10 +00:00
Raymond Hettinger b7d05db0be Optimize tuple_slice() and make further improvements to list_slice()
and list.extend().  Factoring the inner loops to remove the constant
structure references and fixed offsets gives speedups ranging from
20% to 30%.
2004-03-08 07:25:05 +00:00
Raymond Hettinger 99842b6534 Small optimizations for list_slice() and list_extend_internal().
* Using addition instead of substraction on array indices allows the
  compiler to use a fast addressing mode.  Saves about 10%.

* Using PyTuple_GET_ITEM and PyList_SET_ITEM is about 7% faster than
  PySequenceFast_GET_ITEM which has to make a list check on every pass.
2004-03-08 05:56:15 +00:00
Raymond Hettinger ebedb2f773 Factor out code common to PyDict_Copy() and PyDict_Merge(). 2004-03-08 04:19:01 +00:00
Raymond Hettinger 31017aed36 SF #904720: dict.update should take a 2-tuple sequence like dict.__init_
(Championed by Bob Ippolito.)

The update() method for mappings now accepts all the same argument forms
as the dict() constructor.  This includes item lists and/or keyword
arguments.
2004-03-04 08:25:44 +00:00
Michael W. Hudson 6bee23cdc3 Oops, didn't mean to commit the removal of float_compare! 2004-02-26 13:16:03 +00:00
Michael W. Hudson 957f9774b6 Pass a variable that actually exists to PyFPE_END_PROTECT in
float_richcompare.  Reported on c.l.py by Helmut Jarausch.
2004-02-26 12:33:09 +00:00
Michael W. Hudson d3b33b5f6f "Fix" (for certain configurations of the planets, including
recent gcc on Linux/x86)

[ 899109 ] 1==float('nan')

by implementing rich comparisons for floats.

Seems to make comparisons involving NaNs somewhat less surprising
when the underlying C compiler actually implements C99 semantics.
2004-02-19 19:35:22 +00:00
Raymond Hettinger fa6c6f8a73 Keep the list.pop() optimization while restoring the many possibility
for types other than PyInt being accepted for the optional argument.
(Spotted by Neal Norwitz.)
2004-02-19 06:12:06 +00:00
Jeremy Hylton 7083bb744a Oops. Return -1 to distinguish error from empty dict.
This change probably isn't work a bug fix.  It's unlikely that anyone
was calling this method without passing it a real dict.
2004-02-17 20:10:11 +00:00
Raymond Hettinger 9eb86b3c7c Double the speed of list.pop() which was spending most of its time parsing
arguments.
2004-02-17 11:36:16 +00:00
Raymond Hettinger 90a39bf12c Refactor list_extend() and list_fill() for gains in code size, memory
utilization, and speed:

* Moved the responsibility for emptying the previous list from list_fill
  to list_init.

* Replaced the code in list_extend with the superior code from list_fill.

* Eliminated list_fill.

Results:

* list.extend() no longer creates an intermediate tuple except to handle
  the special case of x.extend(x).  The saves memory and time.

* list.extend(x) runs
    5 to 10% faster when x is a list or tuple
    15% faster when x is an iterable not defining __len__
    twice as fast when x is an iterable defining __len__

* the code is about 15 lines shorter and no longer duplicates
  functionality.
2004-02-15 03:57:00 +00:00
Raymond Hettinger ab517d2eac Fine tune the speed/space trade-off for overallocating small lists.
The Py2.3 approach overallocated small lists by up to 8 elements.
The last checkin would limited this to one but slowed down (by 20 to 30%)
the creation of small lists between 3 to 8 elements.

This tune-up balances the two, limiting overallocation to 3 elements
(significantly reducing space consumption from Py2.3) and running faster
than the previous checkin.

The first part of the growth pattern (0, 4, 8, 16) neatly meshes with
allocators that trigger data movement only when crossing a power of two
boundary.  Also, then even numbers mesh well with common data alignments.
2004-02-14 18:34:46 +00:00
Raymond Hettinger 2731ae4d6d Fix missing return value. Spotted by Neal Norwitz 2004-02-14 03:07:21 +00:00
Raymond Hettinger cb3e580ebc Optimize list.pop() for the common special case of popping off the end.
More than doubles its speed.
2004-02-13 18:36:31 +00:00
Raymond Hettinger 4bb9540dd6 * Optimized list appends and pops by making fewer calls the underlying system
realloc().  This is achieved by tracking the overallocation size in a new
  field and using that information to skip calls to realloc() whenever
  possible.

* Simplified and tightened the amount of overallocation.  For larger lists,
  this overallocates by 1/8th (compared to the previous scheme which ranged
  between 1/4th to 1/32nd over-allocation).  For smaller lists (n<6), the
  maximum overallocation is one byte (formerly it could be upto eight bytes).
  This saves memory in applications with large numbers of small lists.

* Eliminated the NRESIZE macro in favor of a new, static list_resize function
  that encapsulates the resizing logic.  Coverting this back to macro would
  give a small (under 1%) speed-up.  This was too small to warrant the loss
  of readability, maintainability, and de-coupling.

* Some functions using NRESIZE had grown unnecessarily complex in their
  efforts to bend to the macro's calling pattern.  With the new list_resize
  function in place, those other functions could be simplified.  That is
  being saved for a separate patch.

* The ob_item==NULL check could be eliminated from the new list_resize
  function.  This would entail finding each piece of code that sets ob_item
  to NULL and adding a new line to invalidate the overallocation tracking
  field.  Rather than impose a new requirement on other pieces of list code,
  it was preferred to leave the NULL check in place and retain the benefits
  of decoupling, maintainability and information hiding (only PyList_New()
  and list_sort() need to know about the new field).  This approach also
  reduces the odds of breaking an extension module.

(Collaborative effort by Raymond Hettinger, Hye-Shik Chang, Tim Peters,
 and Armin Rigo.)
2004-02-13 11:36:39 +00:00
Raymond Hettinger 029dba5a40 Make reversed() transparent with respect to length. 2004-02-10 09:33:39 +00:00
Raymond Hettinger b32e640489 SF patch #875689: >100k alloc wasted on startup
(Contributed by Mike Pall.)

Make sure fill_free_list() is called only once rather than 106 times
when pre-allocating small ints.
2004-02-08 18:54:37 +00:00
Raymond Hettinger 06353f76be Let reversed() work with itself. 2004-02-08 10:49:42 +00:00
Jim Fulton 8a1a594590 Fixed a bug in object.__reduce_ex__ (reduce_2) when using protocol
2.  Failure to clear the error when attempts to get the __getstate__
  attribute fail caused intermittent errors and odd behavior.
2004-02-08 04:21:26 +00:00
Skip Montanaro db6080507d Remove support for --without-universal-newlines (see PEP 11). 2004-02-07 13:53:46 +00:00
Raymond Hettinger c058fd14a9 * Fix ref counting in extend() and extendleft().
* Let deques support reversed().
2004-02-07 02:45:22 +00:00
Walter Dörwald cd736e71a3 Fix reallocation bug in unicode.translate(): The code was comparing
characters instead of character pointers to determine space requirements.
2004-02-05 17:36:00 +00:00
Fred Drake bc875f5a36 Allocating a new weakref object can cause existing weakref objects for
the same object to be collected by the cyclic GC support if they are
only referenced by a cycle.  If the weakref being collected was one of
the weakrefs without callbacks, some local variables for the
constructor became invalid and have to be re-computed.

The test caused a segfault under a debug build without the fix applied.
2004-02-04 23:14:14 +00:00
Fred Drake 6a2852cd48 Fix bug in interpretation of the "callback" argument in the constructors for
weakref ref and proxy objects; None was not being treated as identical to
NULL, though it was documented as equivalent.
2004-02-03 19:52:56 +00:00
Brett Cannon fb5a4e33fb Removed two unneeded lines from PyObject_Compare().
Closes bug #885293 (thanks, Josiah Carlson).
2004-01-27 20:17:54 +00:00
Armin Rigo 76beca957f Two forgotten Py_DECREF() for two out-of-memory conditions. 2004-01-27 16:08:07 +00:00
Tim Peters 7049d816fb Revert change accidentally checked in as part of a whitespace normalization
patch.
2004-01-18 20:31:02 +00:00
Tim Peters 58eb11cf62 Whitespace normalization. 2004-01-18 20:29:55 +00:00
Skip Montanaro ce59c04127 Remove support for SunOS 4.
Remove BAD_EXEC_PROTOYPE (leftover from IRIX 4 demolition).
2004-01-17 14:19:44 +00:00
Raymond Hettinger 2fb702966c SF Patch #871704: Py_SequenceFast can mask errors
(Contributed by Greg Chapman.)

Since this only changes the error message, I doubt that it should be
backported.
2004-01-11 23:26:51 +00:00
Hye-Shik Chang 75c00efcc7 [SF #866875] Add a specialized routine for one character
separaters on str.split() and str.rsplit().
2004-01-05 00:29:51 +00:00
Raymond Hettinger b86269db45 Apply pre-sizing optimization to a broader class of objects.
Formerly, the length was only fetched from sequence objects.
Now, any object that reports its length can benefit from pre-sizing.
2004-01-04 11:00:08 +00:00
Raymond Hettinger 7832cd6141 Apply tuple/list pre-sizing optimization to a broader class of objects.
Formerly, length data fetched from sequence objects.
Now, any object that reports its length can benefit from pre-sizing.

On one sample timing, it gave a threefold speedup for list(s) where s
was a set object.
2004-01-04 06:08:16 +00:00
Hye-Shik Chang 1bc09b7c2a Cosmetic fix for wrongly indented tabs with ts=4. 2004-01-03 19:35:43 +00:00
Raymond Hettinger a3b11e7fb3 * Simplify and speedup logic for tp_print.
* Speed-up intersection whenever PyDict_Next can be used.
2003-12-31 14:08:58 +00:00
Hye-Shik Chang 7db07e6972 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE. 2003-12-29 01:36:01 +00:00
Andrew MacIntyre f1ca7f561c complete backout of listobject.c v2.171 2003-12-28 07:43:56 +00:00
Jeremy Hylton 30973414c5 Revert previous two checkins to repair test failure.
The special-case code that was removed could return a value indicating
success but leave an exception set.  test_fileinput failed in a debug
build as a result.
2003-12-26 19:05:04 +00:00
Andrew MacIntyre 694e3a4a9d use the correct macro to access list size 2003-12-26 00:09:04 +00:00
Andrew MacIntyre d57caed52c Performance of list([]) in 2.3 came up in a thread on comp.lang.python,
which can be reviewed via
http://coding.derkeiler.com/Archive/Python/comp.lang.python/2003-12/1011.html

Duncan Booth investigated, and discovered that an "optimisation" was
in fact a pessimisation for small numbers of elements in a source list,
compared to not having the optimisation, although with large numbers
of elements in the source list the optimisation was quite beneficial.

He posted his change to comp.lang.python (but not to SF).

Further research has confirmed his assessment that the optimisation only
becomes a net win when the source list has more than 100 elements.

I also found that the optimisation could apply to tuples as well,
but the gains only arrive with source tuples larger than about 320
elements and are nowhere near as significant as the gains with lists,
(~95% gain @ 10000 elements for lists, ~20% gain @ 10000 elements for
tuples) so I haven't proceeded with this.

The code as it was applied the optimisation to list subclasses as
well, and this also appears to be a net loss for all reasonable sized
sources (~80-100% for up to 100 elements, ~20% for more than 500
elements; I tested up to 10000 elements).

Duncan also suggested special casing empty lists, which I've extended
to all empty sequences.

On the basis that list_fill() is only ever called with a list for the
result argument, testing for the source being the destination has
now happens before testing source types.
2003-12-25 13:28:48 +00:00
Hye-Shik Chang 7fc4cf57b8 Fix unicode.rsplit()'s bug that ignores separater on the end of string when
using specialized splitter for 1 char sep.
2003-12-23 09:10:16 +00:00
Skip Montanaro ac4ea13a3a There are places in Python which assume bytes have 8-bits. Formalize that a
bit by checking the value of UCHAR_MAX in Include/Python.h.  There was a
check in Objects/stringobject.c.  Remove that.  (Note that we don't define
UCHAR_MAX if it's not defined as the old test did.)
2003-12-22 16:31:41 +00:00
Hye-Shik Chang 40e9509dc7 Fix broken xmlcharrefreplace by rev 2.204.
(Pointy hat goes to perky)
2003-12-22 01:31:13 +00:00
Hye-Shik Chang 4a264fb054 SF #859573: Reduce compiler warnings on gcc 3.2 and above. 2003-12-19 01:59:56 +00:00
Raymond Hettinger 64958a15d7 Guido grants a Christmas wish:
sorted() becomes a regular function instead of a classmethod.
2003-12-17 20:43:33 +00:00
Raymond Hettinger 81ad32e435 Speedup set.update by using the override mode for PyDict_Merge(). 2003-12-15 21:16:06 +00:00
Hye-Shik Chang 3ae811b57d Add rsplit method for str and unicode builtin types.
SF feature request #801847.
Original patch is written by Sean Reifschneider.
2003-12-15 18:49:53 +00:00
Raymond Hettinger fb4e33a8e2 Improve algorithm for set.difference when the input is not a set. 2003-12-15 13:23:55 +00:00
Raymond Hettinger 438e02dfc8 * Refactor set.__contains__()
* Use Py_RETURN_NONE everywhere.
* Fix-up the firstpass check for the tp_print slot.
2003-12-13 19:38:47 +00:00
Raymond Hettinger 0deab62704 Refactor set.discard() and set.remove(). 2003-12-13 18:53:18 +00:00
Raymond Hettinger 6a8bbdbe7b Improve argument checking speed. 2003-12-13 15:21:55 +00:00
Raymond Hettinger dc5ae11abf Use dictionary specific looping idiom where possible.
Simplifies and speeds-up the code.
2003-12-13 14:46:46 +00:00
Raymond Hettinger 0c66967e3d Simplify previous checkin -- a new function was not needed. 2003-12-13 13:31:55 +00:00
Raymond Hettinger d3ae6729e7 Use PyDict_Contains() instead of PySequence_Contains(). 2003-12-13 11:58:56 +00:00
Raymond Hettinger 8f5cdaa784 * Added a new method flag, METH_COEXIST.
* Used the flag to optimize set.__contains__(), dict.__contains__(),
  dict.__getitem__(), and list.__getitem__().
2003-12-13 11:26:12 +00:00
Hye-Shik Chang 19cb193244 Fix memory error treatment correctly. Going to dsu_fail causes
deallocating garbage pointers; saved_ob_item and empty_ob_item.
(Reviewed by Raymond Hettinger)
2003-12-10 07:31:08 +00:00
Michael W. Hudson 1df0f654e8 Fixes and tests for various "holding pointers when arbitrary Python code
can run" bugs as discussed in

[ 848856 ] couple of new list.sort bugs
2003-12-04 11:25:46 +00:00
Guido van Rossum 6c9e130524 - Removed FutureWarnings related to hex/oct literals and conversions
and left shifts.  (Thanks to Kalle Svensson for SF patch 849227.)
  This addresses most of the remaining semantic changes promised by
  PEP 237, except for repr() of a long, which still shows the trailing
  'L'.  The PEP appears to promise warnings for operations that
  changed semantics compared to Python 2.3, but this is not
  implemented; we've suffered through enough warnings related to
  hex/oct literals and I think it's best to be silent now.
2003-11-29 23:52:13 +00:00
Raymond Hettinger 37e136373e Make sure the list.sort's decorate step unwinds itself before returning
an exception raised by the key function.
(Suggested by Michael Hudson.)
2003-11-28 21:43:02 +00:00
Raymond Hettinger 4f8f976576 Add optional fillchar argument to ljust(), rjust(), and center() string methods. 2003-11-26 08:21:35 +00:00
Raymond Hettinger bc0f2ab9bb Expose dict_contains() and PyDict_Contains() with is about 10% faster
than PySequence_Contains() and more clearly applicable to dicts.

Apply the new function in setobject.c where __contains__ checking is
ubiquitous.
2003-11-25 21:12:14 +00:00
Raymond Hettinger a38123e2fa Factor out more duplicate code. 2003-11-24 22:18:49 +00:00
Guido van Rossum 5f4e45d66f Stop GCC warning about int literal that's so long that it becomes an
unsigned int (on a 32-bit machine), by adding an explicit 'u' to the
literal (a prime used to improve the hash function for frozenset).
2003-11-24 04:13:13 +00:00
Raymond Hettinger f5f41bf087 * Checkin remaining documentation
* Add more tests
* Refactor and neaten the code a bit.
* Rename union_update() to update().
* Improve the algorithms (making them a closer to sets.py).
2003-11-24 02:57:33 +00:00
Raymond Hettinger 49ba4c39c4 * Simplify hash function and add test to show effectiveness of the hash
function.

* Add a better test for deepcopying.

* Add tests to show the __init__() function works like it does for list
  and tuple.  Add related test.

* Have shallow copies of frozensets return self.  Add related test.

* Have frozenset(f) return f if f is already a frozenset. Add related test.

* Beefed-up some existing tests.
2003-11-23 02:49:05 +00:00
Guido van Rossum baf0f8f24d - When method objects have an attribute that can be satisfied either
by the function object or by the method object, the function
  object's attribute usually wins.  Christian Tismer pointed out that
  that this is really a mistake, because this only happens for special
  methods (like __reduce__) where the method object's version is
  really more appropriate than the function's attribute.  So from now
  on, all method attributes will have precedence over function
  attributes with the same name.
2003-11-22 23:55:50 +00:00
Raymond Hettinger bfd334a42d Extend temporary hashability to remove() and discard().
Brings the functionality back in line with sets.py.
2003-11-22 03:55:23 +00:00
Raymond Hettinger 19c2d77842 Allow temporary hashability for the __contains__ test.
(Requested by Alex Martelli.)
2003-11-21 18:36:54 +00:00
Raymond Hettinger 3fbec701ca issubset() and issuperset() to work with general iterables 2003-11-21 07:56:36 +00:00
Raymond Hettinger 82d73dd459 Three minor performance improvements:
* Improve the hash function to increase the chance that distinct sets will
  have distinct xor'd hash totals.

* Use PyDict_Merge where possible (it is faster than an equivalent iter/set
  pair).

* Don't rebuild dictionaries where the input already has one.
2003-11-20 22:54:33 +00:00
Tim Peters 403a203223 SF bug 839548: Bug in type's GC handling causes segfaults.
Also SF patch 843455.

This is a critical bugfix.
I'll backport to 2.3 maint, but not beyond that.  The bugs this fixes
have been there since weakrefs were introduced.
2003-11-20 21:21:46 +00:00
Jack Jansen eddc1449ba Getting rid of all the code inside #ifdef macintosh too. 2003-11-20 01:44:59 +00:00
Jack Jansen 4bae2d5e46 Getting rid of code dependent on GUSI or the MetroWerks compiler. 2003-11-19 22:52:23 +00:00
Jack Jansen fb2765666f Getting rid of support for the ancient Apple MPW compiler. 2003-11-19 15:24:47 +00:00
Guido van Rossum b61982bacb Implement straightforward suggestions from gcc warnings (remove unused
variable, add extra braces).
2003-11-18 19:27:19 +00:00
Raymond Hettinger 1b92fd5bca Use PySequence_Contains() instead of direct access macro. 2003-11-18 14:15:31 +00:00
Raymond Hettinger 50a4bb325c Various fixups (most suggested by Armin Rigo). 2003-11-17 16:42:33 +00:00
Raymond Hettinger e2c277a69f Fix output spacing typo 2003-11-16 16:36:58 +00:00
Raymond Hettinger a690a9967e * Migrate set() and frozenset() from the sandbox.
* Install the unittests, docs, newsitem, include file, and makefile update.
* Exercise the new functions whereever sets.py was being used.

Includes the docs for libfuncs.tex.  Separate docs for the types are
forthcoming.
2003-11-16 16:17:49 +00:00
Tim Peters 0bd743cee1 subtype_dealloc(): Simplified overly contorted retracking logic. With
this change, I think subtype_dealloc is actually a smidgen less obscure
than it was in 2.3 -- we got rid of a negation in an "if" <wink>.
2003-11-13 22:50:00 +00:00
Tim Peters f7f9e9966b subtype_dealloc(): A more complete fix for critical bug 840829 +
expanded the test case with a piece that needs the more-complete fix.

I'll backport this to 2.3 maint.
2003-11-13 21:59:32 +00:00
Tim Peters add09b4149 SF bug 840829: weakref callbacks and gc corrupt memory.
subtype_dealloc():  This left the dying object exposed to gc, so that
if cyclic gc triggered during the weakref callback, gc tried to delete
the dying object a second time.  That's a disaster.  subtype_dealloc()
had a (I hope!) unique problem here, as every normal dealloc routine
untracks the object (from gc) before fiddling with weakrefs etc.  But
subtype_dealloc has obscure technical reasons for re-registering the
dying object with gc (already explained in a large comment block at
the bottom of the function).

The fix amounts to simply refraining from reregistering the dying object
with gc until after the weakref callback (if any) has been called.

This is a critical bug (hard to predict, and causes seemingly random
memory corruption when it occurs).  I'll backport it to 2.3 later.
2003-11-12 20:43:28 +00:00
Raymond Hettinger 001f228f36 Improve the reverse list iterator to free memory as soon as the iterator
is exhausted.
2003-11-08 11:58:44 +00:00
Raymond Hettinger c24c9106e8 Minor code fixup. Make sure that len reflects the current list size. 2003-11-08 11:35:22 +00:00
Raymond Hettinger 1021c44b41 Optimize reversed(list) using a custom iterator. 2003-11-07 15:38:09 +00:00
Raymond Hettinger 85c20a41df Implement and apply PEP 322, reverse iteration 2003-11-06 14:06:48 +00:00
Jeremy Hylton ceac90aecb Fix compiler warning about possible use of n without assignment.
Also fix use of n for two different variables in two different blocks.
2003-11-03 20:58:28 +00:00
Raymond Hettinger 54a831bef7 Use PyTuple_Pack() to simplify enumerate(). 2003-11-02 05:37:44 +00:00
Raymond Hettinger 0a9b9da0c3 Add list.sorted() classmethod. 2003-10-29 06:54:43 +00:00
Armin Rigo 2b3eb4062c Deleting cyclic object comparison.
SF patch 825639
http://mail.python.org/pipermail/python-dev/2003-October/039445.html
2003-10-28 12:05:48 +00:00
Raymond Hettinger 98779e0e36 Fix Greg Ward's error message nit: PyObject_SetItem and PySequenceSetItem
had slightly different error messages.
2003-10-27 09:22:16 +00:00
Walter Dörwald 4894c30626 Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
charmaptranslate_makespace() allocated more memory than required for the
next replacement but didn't remember that fact, so memory size was growing
exponentially every time a replacement string is longer that one character.
This fixes SF bug #828737.
2003-10-24 14:25:28 +00:00
Fred Drake d22bb6584d Avoid confusing name for the 3rd argument to str.replace().
This closes SF bug #827260.
2003-10-22 02:56:40 +00:00
Jeremy Hylton e4b9d8c2ba Removing bogus Py_DECREF() reported by Armin Rigo (SF bug 812353).
Even if a new dict is generated for locals, it is stored in
f->f_locals.
2003-10-21 18:14:20 +00:00
Jeremy Hylton 174d276d8c Fix indentation. 2003-10-21 18:10:28 +00:00
Walter Dörwald f0dfc7ac5c Fix a bunch of typos in documentation, docstrings and comments.
(From SF patch #810751)
2003-10-20 14:01:56 +00:00
Martin v. Löwis 01a74b2fa1 Make CObjects mutable. Fixes #477441. 2003-10-19 18:30:01 +00:00
Martin v. Löwis 6828e18a6a Patch #825679: Clarify semantics of .isfoo on empty strings.
Backported to 2.3.
2003-10-18 09:55:08 +00:00
Martin v. Löwis 849a972f35 Patch #809535: Mention behaviour of seek on text files. Backported to 2.3. 2003-10-18 09:38:01 +00:00
Raymond Hettinger ae4a299a0d Fix typo found by Neal Norwitz. 2003-10-16 17:16:30 +00:00
Raymond Hettinger 42b1ba31af * list.sort() now supports three keyword arguments: cmp, key, and reverse.
key provides C support for the decorate-sort-undecorate pattern.
  reverse provide a stable sort of the list with the comparisions reversed.

* Amended the docs to guarantee sort stability.
2003-10-16 03:41:09 +00:00
Raymond Hettinger 8ae4689657 Simplify and speedup uses of Py_BuildValue():
* Py_BuildValue("(OOO)",a,b,c)  -->  PyTuple_Pack(3,a,b,c)
* Py_BuildValue("()",a)         -->  PyTuple_New(0)
* Py_BuildValue("O", a)         -->  Py_INCREF(a)
2003-10-12 19:09:37 +00:00
Raymond Hettinger cb2da43db8 Extended tuple's C API to include a new function, PyTuple_Pack() that is
useful for rapidly building argument tuples without having to invoke the
more sophisticated machinery of Py_BuildValue().
2003-10-12 18:24:34 +00:00
Raymond Hettinger 56bb16f1b3 Use the simpler and faster PyArg_UnpackTuple() instead of
PyArg_ParseTuple() where possible.
2003-10-11 19:32:18 +00:00
Raymond Hettinger f34f2646a1 SF bug #820397: __nonzero__() returns 1/0
Altered to return a PyBool instead of a PyInt.

Backport candidate.
2003-10-11 17:29:04 +00:00
Guido van Rossum 98c65bed91 Return a bool rather than an int from proxy_has_key(). 2003-10-09 03:47:08 +00:00
Guido van Rossum 22c3dda1e6 Fix leak introduced by previous typeobject.c checkin. 2003-10-09 03:46:35 +00:00
Guido van Rossum 02c58f865c SF patch #820195 by Wojtek Walczak (gminick at users.sourceforge.net):
make obj.__contains__() returns True/False instead of 1/0.
2003-10-08 21:08:29 +00:00
Jeremy Hylton 504de6bd2c Fix for SF bug [ 817156 ] invalid \U escape gives 0=length unistr. 2003-10-06 05:08:26 +00:00
Tim Peters ced69f8a20 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
so fiddle Jeremy's fix to live with that.  Also added more comments.

Bugfix candidate (this bug is in all versions of Python, at least since
2.1).
2003-09-16 20:30:58 +00:00
Jeremy Hylton d808279be3 Double-fix of crash in Unicode freelist handling.
If a length-1 Unicode string was in the freelist and it was
uninitialized or pointed to a very large (magnitude) negative number,
the check

	 unicode_latin1[unicode->str[0]] == unicode

could cause a segmentation violation, e.g. unicode->str[0] is 0xcbcbcbcb.

Fix this in two ways:

1. Change guard befor unicode_latin1[] to test against 256U.  If I
   understand correctly, the unsigned long used to store UCS4 on my
   box was getting converted to a signed long to compare with the
   signed constant 256.

2. Change _PyUnicode_New() to make sure the first element of str is
   always initialized to zero.  There are several places in the code
   where the caller can exit with an error before initializing any
   of str, which would leave junk in str[0].

Also, silence a compiler warning on pointer vs. int arithmetic.

Bug fix candidate.
2003-09-16 19:41:39 +00:00
Raymond Hettinger a9e14b7015 Fix leak in classobject.c. The leak surfaced on the error exit when
hashing a class that does not define __hash__ but does define a
comparison.
2003-09-16 07:11:46 +00:00
Jeremy Hylton deb2dc6658 Change checks of PyUnicode_Resize() return value for clarity.
The unicode_resize() family only returns -1 or 0 so simply checking
for != 0 is sufficient, but somewhat unclear.  Many Python API
functions return < 0 on error, reserving the right to return 0 or 1 on
success.  Change the call sites for consistency with these calls.
2003-09-16 03:41:45 +00:00
Martin v. Löwis 7bbcde70d1 Only release buffer after file has been closed. Fixes #800824.
Will backport to 2.2.
2003-09-07 20:42:29 +00:00
Tim Peters f1827cfaab SF bug 801631: file.truncate fault on windows.
file_truncate():  C doesn't define what fflush(fp) does if fp is open
for update, and the preceding I/O operation on fp was input.  On Windows,
fflush() actually changes the current file position then.  Because
Windows doesn't support ftruncate() directly, this not only caused
Python's file.truncate() to change the file position (contra our docs),
it also caused the file not to change size.

Repaired by getting the initial file position at the start, restoring
it at the end, and tossing all the complicated micro-efficiency checks
trying to avoid "provably unnecessary" seeks.  file.truncate() can't
be a frequent operation, and seeking to the current file position has
got to be cheap anyway.

Bugfix candidate.
2003-09-07 03:30:18 +00:00
Raymond Hettinger b859c070ef SF bug #800796: Difference between hash() and __hash__()
slice(5).__hash__() now raises a TypeError.
2003-09-05 14:27:30 +00:00
Martin v. Löwis 1e3bdf6c45 Patch #788249: Pass an explicit buffer to setvbuf in PyFile_SetBufSize().
Fixes #603724. Will backport to 2.3.
2003-09-04 19:01:46 +00:00
Raymond Hettinger 574aa32578 SF patch #798467: Update docstring of has_key for bool changes
(Contributed by George Yoshida.)
2003-09-01 22:12:08 +00:00
Raymond Hettinger 0970dbab97 Remove 'e.g.' from error message 2003-08-30 23:57:36 +00:00
Raymond Hettinger 9bfe533c69 SF bug #795506: Wrong handling of string format code for float values.
Adding missing support for '%F'.

Will backport to 2.3.1.
2003-08-27 04:55:52 +00:00
Neal Norwitz 98cad48171 Fix SF #789402, Memory leak on open()
If opening a directory, the exception would leak.
2003-08-15 20:05:45 +00:00
Walter Dörwald 150523efa5 Fix refcounting leak in charmaptranslate_lookup() 2003-08-15 16:52:19 +00:00
Walter Dörwald 9b30f206ee Fix another refcounting leak in PyUnicode_EncodeCharmap(). 2003-08-15 16:26:34 +00:00
Walter Dörwald d4ade0885c Fix another refcounting leak (in PyUnicode_DecodeUnicodeEscape()). 2003-08-15 15:00:26 +00:00
Michael W. Hudson b2c7de4667 Fix for
[ 784825 ] fix obscure crash in descriptor handling

Should be applied to release23-maint and in all likelyhood
release22-maint, too.

Certainly doesn't apply to release21-maint.
2003-08-15 13:07:47 +00:00
Michael W. Hudson da0a0673b1 My last fix left n used unitialized in tha a==b case.
Fix, by not using n at all in that case.

Needs to be applied to release23-maint, too.
2003-08-15 12:06:41 +00:00
Tim Peters 465fa3dac4 complex_new(): This could leak when the argument was neither string nor
number.  This accounts for the 2 refcount leaks per test_complex run
Michael Hudson discovered (I figured only I would have the stomach to
look for leaks in floating-point code <wink>).
2003-08-15 01:16:37 +00:00
Walter Dörwald e5402fb340 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
when an encoding error occurs and the callback name is unknown,
i.e. when the callback has to be called. The problem was that
the fact that the callback has already been looked up was only
recorded in a local variable in charmap_encoding_error(), because
charmap_encoding_error() got it's own copy of the errorHandler
pointer instead of a pointer to the pointer in
PyUnicode_EncodeCharmap().
2003-08-14 20:25:29 +00:00
Michael W. Hudson b4f49385a3 Fix reference leak noted in test_types:
Check for a[:] = a _before_ calling PySequence_Fast on a.
release23-maint candidate
Reference leak doesn't happen with head of release22-maint.
2003-08-14 17:04:28 +00:00
Michael W. Hudson 71665dc90d Add a couple of decrefs to error paths.
Now test_descr only appears to leak two references & I think this
are in fact illusory (it's to do with things getting resurrected in
__del__ methods & it's easy to be believe confusion occurs when that
happens <wink>).  Woohoo!
2003-08-11 17:32:02 +00:00
Michael W. Hudson bdc6ea1110 Fix silly typo in comment. 2003-08-11 16:14:06 +00:00
Michael W. Hudson a6a277d831 /* XXX From here until type is allocated, "return NULL" leaks bases! */
Sure looks like it to me! <wink>

When I run the leak2.py script I posted to python-dev, I only see
three reference leaks in all of test_descr.  When I run
test_descr.test_main, I still see 46 leaks.  This clearly demands
posting a yelp to python-dev :-)

This certainly should be applied to release23-maint, and in all
likelyhood release22-maint as well.
2003-08-08 13:57:22 +00:00
Michael W. Hudson e723e453a1 Repair refcounting on error return from type_set_bases.
Include a test case that failed for one of my efforts to repair this.
2003-08-07 14:58:10 +00:00
Neil Schemenauer 7555294576 Remove code that tried to warn about shadowing builtin names after a
module had been compiled.  It gives too many spurious warnings.
2003-07-16 22:04:11 +00:00
Jeremy Hylton f75d9fce16 Remove stray comments. 2003-07-16 16:17:57 +00:00
Jeremy Hylton 1c7a0ea056 Remove unnecessary check in tests for slots allowed.
The !PyType_Check(base) check snuck in as part of rev 2.215, but was
unrelated to the SF patch that is mentioned in the checkin comment.
The test is currently unnecessary because base is set to the return
value of best_bases(), which returns a type or NULL.
2003-07-16 16:08:23 +00:00
Fred Drake fe89cc186c Remove proxy_print(), since that caused an inconsistency between
"print repr(proxy(a))" and "proxy(a)" at an interactive prompt.
Closes SF bug #722763.
2003-07-14 21:46:23 +00:00
Jeremy Hylton 6d3e0186d6 Add whitespace. 2003-07-11 17:02:39 +00:00
Mark Hammond 0ccda1ee10 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
defering to the encodings package.
As described in [ 763111 ] mbcs encoding should skip encodings package
2003-07-01 00:13:27 +00:00
Raymond Hettinger d693a81595 Fix SF 762891: "del p[key]" on proxy object raises SystemError() 2003-06-30 04:18:48 +00:00
Raymond Hettinger f466793fcc SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Submitted By: Christopher A. Craig

Fillin some missing decrefs.
2003-06-28 20:04:25 +00:00
Jeremy Hylton 3e3159ce6a Require that __nonzero__() return a bool or exactly an int. 2003-06-27 17:38:27 +00:00
Jeremy Hylton 090a3495b3 Check return type of __nonzero__() method.
The language reference says you must return an int or a bool.  This
fix limits the scope of SF bug 759227 (infinite recursion) to
subclasses of int.
2003-06-27 16:46:45 +00:00
Walter Dörwald 03f6c54359 Whitespace normalization. 2003-06-25 13:12:18 +00:00
Walter Dörwald 9ff3f03c3e Fix whitespace. 2003-06-18 14:17:01 +00:00
Raymond Hettinger be9715398b SF bug #753451: classmethod abuse --> SystemError
Check the argument to classmethod for callability.

Backport candidate.
2003-06-18 01:13:41 +00:00
Walter Dörwald 5ecd6c4db2 Fix typo in comment. 2003-06-17 20:22:24 +00:00
Walter Dörwald e8049befdf Use _PyEval_SliceIndex to handle list.index() calls with
huge start and stop arguments. Add tests.
2003-06-17 19:27:39 +00:00
Walter Dörwald e0a1bb6341 Whitespace normalization. 2003-06-17 15:48:11 +00:00
Guido van Rossum 2743d87d79 Fix sloppy index() implementation:
- don't use min() and max()
- interpret negative start/stop argument like negative slice indices
2003-06-17 14:25:14 +00:00
Raymond Hettinger d05abdec7b SF #754014: list.index() should accept optional start, end arguments
Also, modified UserList.index() to match and expanded the related tests.
2003-06-17 05:05:49 +00:00
Guido van Rossum 59195fdf40 - SF patch 751998 fixes an unwanted side effect of the previous fix
for SF bug 742860 (the next item).
2003-06-13 20:54:40 +00:00
Brett Cannon 10147f7d13 Fixed a comment. 2003-06-11 20:50:33 +00:00
Neil Schemenauer 4e3363e884 Warn about creating global variables by __setattr__ that shadow builtin
names.  Unfortunately, this is not bulletproof since the module
dictionary can be modified directly.
2003-06-09 18:42:19 +00:00
Neal Norwitz e2fdc61004 Fix SF #749831, copy raises SystemError when getstate raises exception 2003-06-08 13:19:58 +00:00
Guido van Rossum 1987c6693b Fix for SF 742911. We now clear the weakrefs *before* calling __del__
or emptying __dict__, just as we do for classic classes.
2003-05-29 14:29:23 +00:00
Raymond Hettinger e509b2ad24 Add notes on use cases with paired accesses to the same key. 2003-05-28 14:10:46 +00:00
Raymond Hettinger e8b0f0461b * Beefed-up tests
* Allow tuple re-use
* Call tp_iternext directly
2003-05-28 14:05:34 +00:00
Tim Peters e87568dd9a SF bug 705231: Assertion failed, python aborts.
float_pow():  Don't let the platform pow() raise -1.0 to an integer power
anymore; at least glibc gets it wrong in some cases.  Note that
math.pow() will continue to deliver wrong (but platform-native) results
in such cases.
2003-05-24 20:18:24 +00:00
Tim Peters 3cfe75470d PyType_Ready(): Complain if the type is a base type, and gc'able, and
tp_free is NULL or PyObject_Del at the end.  Because it's a base type
it must call tp_free in its dealloc function, and because it's gc'able
it must not call PyObject_Del.

inherit_slots():  Don't inherit tp_free unless the type and its base
agree about whether they're gc'able.  If the type is gc'able and the
base is not, and the base uses the default PyObject_Del for its
tp_free, give the type PyObject_GC_Del for its tp_free (the appropriate
default for a gc'able type).

cPickle.c:  The Pickler and Unpickler types claim to be base classes
and gc'able, but their dealloc functions didn't call tp_free.
Repaired that.  Also call PyType_Ready() on these typeobjects, so
that the correct (PyObject_GC_Del) default memory-freeing function
gets plugged into these types' tp_free slots.
2003-05-21 21:29:48 +00:00
Raymond Hettinger 6624e68546 SF bug #604716: faster [None]*n or []*n
Fulfilled request to special case repetitions of lists of length 0 or 1.
2003-05-21 05:58:46 +00:00
Brett Cannon be67d87e4d Fixing the previous patch to have the changes be to the proper docstrings. 2003-05-20 02:40:12 +00:00
Brett Cannon 154da9b7e2 Fix docstrings for __(get|set|del)slice__ to mention that negative indices are not supported. 2003-05-20 02:30:04 +00:00