Commit Graph

273 Commits

Author SHA1 Message Date
Brett Cannon 5ad28e14b6 Tweak previous patch to silence a warning about the unused left value in the
comma expression in listpop() that was being returned.  Still essentially
unused (as it is meant to be), but now the compiler thinks it is worth
*something* by having it incremented.
2004-08-03 04:53:29 +00:00
Tim Peters 8fc4a91665 list_ass_slice(): Document the obscure new intent that deleting a slice
of no more than 8 elements cannot fail.

listpop():  Take advantage of that its calls to list_resize() and
list_ass_slice() can't fail.  This is assert'ed in a debug build now, but
in an icky way.  That is, you can't say:

	assert(some_call() >= 0);

because then some_call() won't occur at all in a release build.  So it
has to be a big pile of #ifdefs on Py_DEBUG (yuck), or the pleasant:

        status = some_call();
        assert(status >= 0);

But in that case, compilers may whine in a release build, because status
appears unused then.  I'm not certain the ugly trick I used here will
convince all compilers to shut up about status (status is always "used" now,
as the first (ignored) clause in a comma expression).
2004-07-31 21:53:19 +00:00
Tim Peters 7357222d0e list_ass_slice(): The difference between "recycle" and "recycled" was
impossible to remember, so renamed one to something obvious.  Headed
off potential signed-vs-unsigned compiler complaints I introduced by
changing the type of a vrbl to unsigned.  Removed the need for the
tedious explanation about "backward pointer loops" by looping on an
int instead.
2004-07-31 02:54:42 +00:00
Tim Peters 8d9eb10c29 Armin asked for a list_ass_slice review in his checkin, so here's the
result.

list_resize():  Document the intent.  Code is increasingly relying on
subtle aspects of its behavior, and they deserve to be spelled out.

list_ass_slice():  A bit more simplification, by giving it a common
error exit and initializing more values.

Be clearer in comments about what "size" means (# of elements?  # of
bytes?).

While the number of elements in a list slice must fit in an int, there's
no guarantee that the number of bytes occupied by the slice will.  That
malloc() and memmove() take size_t arguments is a hint about that <wink>.
So changed to use size_t where appropriate.

ihigh - ilow should always be >= 0, but we never asserted that.  We do
now.

The loop decref'ing the recycled slice had a subtle insecurity:  C doesn't
guarantee that a pointer one slot *before* an array will compare "less
than" to a pointer within the array (it does guarantee that a pointer
one beyond the end of the array compares as expected).  This was actually
an issue in KSR's C implementation, so isn't purely theoretical.  Python
probably has other "go backwards" loops with a similar glitch.
list_clear() is OK (it marches an integer backwards, not a pointer).
2004-07-31 02:24:20 +00:00
Armin Rigo 1dd04a02e0 This is a reorganization of list_ass_slice(). It should probably be reviewed,
though I tried to be very careful.  This is a slight simplification, and it
adds a new feature: a small stack-allocated "recycled" array for the cases
when we don't remove too many items.

It allows PyList_SetSlice() to never fail if:
* you are sure that the object is a list; and
* you either do not remove more than 8 items, or clear the list.

This makes a number of other places in the source code correct again -- there
are some places that delete a single item without checking for MemoryErrors
raised by PyList_SetSlice(), or that clear the whole list, and sometimes the
context doesn't allow an error to be propagated.
2004-07-30 11:38:22 +00:00
Armin Rigo a37bbf2e5b What if you call lst.__init__() while it is being sorted? :-)
The invariant checks would break.
2004-07-30 11:20:18 +00:00
Raymond Hettinger c0aaa2db4f * Simplify and speed-up list_resize(). Relying on the newly documented
invariants allows the ob_item != NULL check to be replaced with an
  assertion.

* Added assertions to list_init() which document and verify that the
  tp_new slot establishes the invariants.  This may preclude a future
  bug if a custom tp_new slot is written.
2004-07-29 23:31:29 +00:00
Armin Rigo 93677f075d * drop the unreasonable list invariant that ob_item should never come back
to NULL during the lifetime of the object.

* listobject.c nevertheless did not conform to the other invariants,
  either; fixed.

* listobject.c now uses list_clear() as the obvious internal way to clear
  a list, instead of abusing list_ass_slice() for that.  It makes it easier
  to enforce the invariant about ob_item == NULL.

* listsort() sets allocated to -1 during sort; any mutation will set it
  to a value >= 0, so it is a safe way to detect mutation.  A negative
  value for allocated does not cause a problem elsewhere currently.
  test_sort.py has a new test for this fix.

* listsort() leak: if items were added to the list during the sort, AND if
  these items had a __del__ that puts still more stuff into the list,
  then this more stuff (and the PyObject** array to hold them) were
  overridden at the end of listsort() and never released.
2004-07-29 12:40:23 +00:00
Armin Rigo f414fc4004 Minor memory leak. 2004-07-29 10:56:55 +00:00
Tim Peters 51b4ade306 Fix obscure breakage (relative to 2.3) in listsort: the test for list
mutation during list.sort() used to rely on that listobject.c always
NULL'ed ob_item when ob_size fell to 0.  That's no longer true, so the
test for list mutation during a sort is no longer reliable.  Changed the
test to rely instead on that listobject.c now never NULLs-out ob_item
after (if ever) ob_item gets a non-NULL value.  This new assumption is
also documented now, as a required invariant in listobject.h.

The new assumption allowed some real simplification to some of the
hairier code in listsort(), so is a Good Thing on that count.
2004-07-29 04:07:15 +00:00
Tim Peters b38e2b61b3 Trimmed trailing whitespace. 2004-07-29 02:29:26 +00:00
Tim Peters 3986d4e660 PyList_New(): we went to all the trouble of computing and bounds-checking
the size_t nbytes, and passed nbytes to malloc, so it was confusing to
effectively recompute the same thing from scratch in the memset call.
2004-07-29 02:28:42 +00:00
Nicholas Bastin 9ba301e589 Moved SunPro warning suppression into pyport.h and out of individual
modules and objects.
2004-07-15 15:54:05 +00:00
Nicholas Bastin 1ce9e4cfc1 Fixed end-of-loop code not reached warning when using SunPro C 2004-06-17 18:27:18 +00:00
Raymond Hettinger fdfe618228 Nits:
- Neatened the braces in PyList_New().
- Made sure "indexerr" was initialized to NULL.
- Factored if blocks in PyList_Append().
- Made sure "allocated" is initialized in list_init().
2004-05-05 06:28:16 +00:00
Raymond Hettinger 0468e416c1 SF patch #947476: Apply freelist technique to lists
Re-use list object bodies.  Saves calls to malloc() and free() for
faster list instantiation and deallocation.
2004-05-05 05:37:53 +00:00
Raymond Hettinger 45d0b5cc44 Use Py_RETURN_NONE macro where applicable. 2004-04-12 17:21:03 +00:00
Raymond Hettinger 501f02cd02 Small refactoring saving one function() and eliminating some indirection.
* Applied app1() to listappend().
* Inlined ins() into its one remaining caller.
2004-04-12 14:01:16 +00:00
Raymond Hettinger 40a03821ae * Specialize ins1() into app1() for appends. Saves several unnecessary
steps and further improves the speed of list append.

* Add guards to the list iterator length method to handle corner cases.
2004-04-12 13:05:09 +00:00
Armin Rigo 70d172dda4 Get rid of listextend_internal() and explain why the special case
'a.extend(a)' isn't so special anyway.
2004-03-20 22:19:23 +00:00
Raymond Hettinger 435bf58b7b Make iterators length transparent where possible. 2004-03-18 22:43:10 +00:00
Raymond Hettinger d4ff741e78 Revert last change. Found an application that was worse off with resize
exact turned on.  The tiny space savings wasn't worth the additional time
and code.
2004-03-15 09:01:31 +00:00
Raymond Hettinger 0e91643bd2 list_resize() now has an "exact" option for bypassing the overallocation
scheme in situations that likely won't benefit from it.  This further
improves memory utilization from Py2.3 which always over-allocates
except for PyList_New().

Situations expected to benefit from over-allocation:
    list.insert(), list.pop(), list.append(), and list.extend()

Situations deemed unlikely to benefit:
    list_inplace_repeat, list_ass_slice, list_ass_subscript

The most gray area was for listextend_internal() which only runs
when the argument is a list or a tuple.  This could be viewed as
a one-time fixed length addition or it could be viewed as wrapping
a series of appends.  I left its over-allocation turned on but
could be convinced otherwise.
2004-03-14 06:42:23 +00:00
Raymond Hettinger 42bec93e5c Make PySequence_Fast_ITEMS public. (Thanks Skip.) 2004-03-12 16:38:17 +00:00
Raymond Hettinger 6e058d70ef * Eliminate duplicate call to PyObject_Size().
(Spotted by Michael Hudson.)

* Now that "selflen" is no longer inside a loop, it should not be a
  register variable.
2004-03-12 15:30:38 +00:00
Raymond Hettinger c1e4f9dd92 Use a new macro, PySequence_Fast_ITEMS to factor out code common to
three recent optimizations.  Aside from reducing code volume, it
increases readability.
2004-03-12 08:04:00 +00:00
Raymond Hettinger 57c4542bcd Now that list.extend() is at the root of many list operations, it becomes
worth it to in-line the call to PyIter_Next().

Saves another 15% on most list operations that acceptable a general
iterable argument (such as the list constructor).
2004-03-11 09:48:18 +00:00
Raymond Hettinger 8ca92ae54c Eliminate a big block of duplicate code in PySequence_List() by
exposing _PyList_Extend().
2004-03-11 09:13:12 +00:00
Raymond Hettinger 97bc618229 list_inplace_concat() is now expressed in terms of list_extend() which
avoids creating an intermediate tuple for iterable arguments other than
lists or tuples.

In other words, a+=b no longer requires extra memory when b is not a
list or tuple.  The list and tuple cases are unchanged.
2004-03-11 07:34:19 +00:00
Raymond Hettinger 66d31f8f38 Use memcpy() instead of memmove() when the buffers are known to be distinct. 2004-03-10 11:44:04 +00:00
Raymond Hettinger ef9bf4031a Tidied up the implementations of reversed (including the custom ones
for xrange and list objects).

* list.__reversed__ now checks the length of the sequence object before
  calling PyList_GET_ITEM() because the mutable could have changed length.

* all three implementations are now tranparent with respect to length and
  maintain the invariant len(it) == len(list(it)) even when the underlying
  sequence mutates.

* __builtin__.reversed() now frees the underlying sequence as soon
  as the iterator is exhausted.

* the code paths were rearranged so that the most common paths
  do not require a jump.
2004-03-10 10:10:42 +00:00
Raymond Hettinger a6366fe085 Optimize inner loops for subscript, repeat, and concat. 2004-03-09 13:05:22 +00:00
Raymond Hettinger f889e10c19 Optimize slice assignments.
* Replace sprintf message with a constant message string -- this error
  message ran on every invocation except straight deletions but it was
  only needed when the rhs was not iterable.  The message was also
  out-of-date and did not reflect that iterable arguments were allowed.

* For inner loops that do not make ref count adjustments, use memmove()
  for fast copying and better readability.

* For inner loops that do make ref count adjustments, speed them up by
  factoring out the constant structure reference and using vitem[] instead.
2004-03-09 08:04:33 +00:00
Raymond Hettinger b7d05db0be Optimize tuple_slice() and make further improvements to list_slice()
and list.extend().  Factoring the inner loops to remove the constant
structure references and fixed offsets gives speedups ranging from
20% to 30%.
2004-03-08 07:25:05 +00:00
Raymond Hettinger 99842b6534 Small optimizations for list_slice() and list_extend_internal().
* Using addition instead of substraction on array indices allows the
  compiler to use a fast addressing mode.  Saves about 10%.

* Using PyTuple_GET_ITEM and PyList_SET_ITEM is about 7% faster than
  PySequenceFast_GET_ITEM which has to make a list check on every pass.
2004-03-08 05:56:15 +00:00
Raymond Hettinger fa6c6f8a73 Keep the list.pop() optimization while restoring the many possibility
for types other than PyInt being accepted for the optional argument.
(Spotted by Neal Norwitz.)
2004-02-19 06:12:06 +00:00
Raymond Hettinger 9eb86b3c7c Double the speed of list.pop() which was spending most of its time parsing
arguments.
2004-02-17 11:36:16 +00:00
Raymond Hettinger 90a39bf12c Refactor list_extend() and list_fill() for gains in code size, memory
utilization, and speed:

* Moved the responsibility for emptying the previous list from list_fill
  to list_init.

* Replaced the code in list_extend with the superior code from list_fill.

* Eliminated list_fill.

Results:

* list.extend() no longer creates an intermediate tuple except to handle
  the special case of x.extend(x).  The saves memory and time.

* list.extend(x) runs
    5 to 10% faster when x is a list or tuple
    15% faster when x is an iterable not defining __len__
    twice as fast when x is an iterable defining __len__

* the code is about 15 lines shorter and no longer duplicates
  functionality.
2004-02-15 03:57:00 +00:00
Raymond Hettinger ab517d2eac Fine tune the speed/space trade-off for overallocating small lists.
The Py2.3 approach overallocated small lists by up to 8 elements.
The last checkin would limited this to one but slowed down (by 20 to 30%)
the creation of small lists between 3 to 8 elements.

This tune-up balances the two, limiting overallocation to 3 elements
(significantly reducing space consumption from Py2.3) and running faster
than the previous checkin.

The first part of the growth pattern (0, 4, 8, 16) neatly meshes with
allocators that trigger data movement only when crossing a power of two
boundary.  Also, then even numbers mesh well with common data alignments.
2004-02-14 18:34:46 +00:00
Raymond Hettinger 2731ae4d6d Fix missing return value. Spotted by Neal Norwitz 2004-02-14 03:07:21 +00:00
Raymond Hettinger cb3e580ebc Optimize list.pop() for the common special case of popping off the end.
More than doubles its speed.
2004-02-13 18:36:31 +00:00
Raymond Hettinger 4bb9540dd6 * Optimized list appends and pops by making fewer calls the underlying system
realloc().  This is achieved by tracking the overallocation size in a new
  field and using that information to skip calls to realloc() whenever
  possible.

* Simplified and tightened the amount of overallocation.  For larger lists,
  this overallocates by 1/8th (compared to the previous scheme which ranged
  between 1/4th to 1/32nd over-allocation).  For smaller lists (n<6), the
  maximum overallocation is one byte (formerly it could be upto eight bytes).
  This saves memory in applications with large numbers of small lists.

* Eliminated the NRESIZE macro in favor of a new, static list_resize function
  that encapsulates the resizing logic.  Coverting this back to macro would
  give a small (under 1%) speed-up.  This was too small to warrant the loss
  of readability, maintainability, and de-coupling.

* Some functions using NRESIZE had grown unnecessarily complex in their
  efforts to bend to the macro's calling pattern.  With the new list_resize
  function in place, those other functions could be simplified.  That is
  being saved for a separate patch.

* The ob_item==NULL check could be eliminated from the new list_resize
  function.  This would entail finding each piece of code that sets ob_item
  to NULL and adding a new line to invalidate the overallocation tracking
  field.  Rather than impose a new requirement on other pieces of list code,
  it was preferred to leave the NULL check in place and retain the benefits
  of decoupling, maintainability and information hiding (only PyList_New()
  and list_sort() need to know about the new field).  This approach also
  reduces the odds of breaking an extension module.

(Collaborative effort by Raymond Hettinger, Hye-Shik Chang, Tim Peters,
 and Armin Rigo.)
2004-02-13 11:36:39 +00:00
Tim Peters 7049d816fb Revert change accidentally checked in as part of a whitespace normalization
patch.
2004-01-18 20:31:02 +00:00
Tim Peters 58eb11cf62 Whitespace normalization. 2004-01-18 20:29:55 +00:00
Raymond Hettinger 7832cd6141 Apply tuple/list pre-sizing optimization to a broader class of objects.
Formerly, length data fetched from sequence objects.
Now, any object that reports its length can benefit from pre-sizing.

On one sample timing, it gave a threefold speedup for list(s) where s
was a set object.
2004-01-04 06:08:16 +00:00
Andrew MacIntyre f1ca7f561c complete backout of listobject.c v2.171 2003-12-28 07:43:56 +00:00
Jeremy Hylton 30973414c5 Revert previous two checkins to repair test failure.
The special-case code that was removed could return a value indicating
success but leave an exception set.  test_fileinput failed in a debug
build as a result.
2003-12-26 19:05:04 +00:00
Andrew MacIntyre 694e3a4a9d use the correct macro to access list size 2003-12-26 00:09:04 +00:00
Andrew MacIntyre d57caed52c Performance of list([]) in 2.3 came up in a thread on comp.lang.python,
which can be reviewed via
http://coding.derkeiler.com/Archive/Python/comp.lang.python/2003-12/1011.html

Duncan Booth investigated, and discovered that an "optimisation" was
in fact a pessimisation for small numbers of elements in a source list,
compared to not having the optimisation, although with large numbers
of elements in the source list the optimisation was quite beneficial.

He posted his change to comp.lang.python (but not to SF).

Further research has confirmed his assessment that the optimisation only
becomes a net win when the source list has more than 100 elements.

I also found that the optimisation could apply to tuples as well,
but the gains only arrive with source tuples larger than about 320
elements and are nowhere near as significant as the gains with lists,
(~95% gain @ 10000 elements for lists, ~20% gain @ 10000 elements for
tuples) so I haven't proceeded with this.

The code as it was applied the optimisation to list subclasses as
well, and this also appears to be a net loss for all reasonable sized
sources (~80-100% for up to 100 elements, ~20% for more than 500
elements; I tested up to 10000 elements).

Duncan also suggested special casing empty lists, which I've extended
to all empty sequences.

On the basis that list_fill() is only ever called with a list for the
result argument, testing for the source being the destination has
now happens before testing source types.
2003-12-25 13:28:48 +00:00
Raymond Hettinger 64958a15d7 Guido grants a Christmas wish:
sorted() becomes a regular function instead of a classmethod.
2003-12-17 20:43:33 +00:00
Raymond Hettinger 8f5cdaa784 * Added a new method flag, METH_COEXIST.
* Used the flag to optimize set.__contains__(), dict.__contains__(),
  dict.__getitem__(), and list.__getitem__().
2003-12-13 11:26:12 +00:00
Hye-Shik Chang 19cb193244 Fix memory error treatment correctly. Going to dsu_fail causes
deallocating garbage pointers; saved_ob_item and empty_ob_item.
(Reviewed by Raymond Hettinger)
2003-12-10 07:31:08 +00:00
Michael W. Hudson 1df0f654e8 Fixes and tests for various "holding pointers when arbitrary Python code
can run" bugs as discussed in

[ 848856 ] couple of new list.sort bugs
2003-12-04 11:25:46 +00:00
Raymond Hettinger 37e136373e Make sure the list.sort's decorate step unwinds itself before returning
an exception raised by the key function.
(Suggested by Michael Hudson.)
2003-11-28 21:43:02 +00:00
Raymond Hettinger 001f228f36 Improve the reverse list iterator to free memory as soon as the iterator
is exhausted.
2003-11-08 11:58:44 +00:00
Raymond Hettinger c24c9106e8 Minor code fixup. Make sure that len reflects the current list size. 2003-11-08 11:35:22 +00:00
Raymond Hettinger 1021c44b41 Optimize reversed(list) using a custom iterator. 2003-11-07 15:38:09 +00:00
Jeremy Hylton ceac90aecb Fix compiler warning about possible use of n without assignment.
Also fix use of n for two different variables in two different blocks.
2003-11-03 20:58:28 +00:00
Raymond Hettinger 0a9b9da0c3 Add list.sorted() classmethod. 2003-10-29 06:54:43 +00:00
Raymond Hettinger ae4a299a0d Fix typo found by Neal Norwitz. 2003-10-16 17:16:30 +00:00
Raymond Hettinger 42b1ba31af * list.sort() now supports three keyword arguments: cmp, key, and reverse.
key provides C support for the decorate-sort-undecorate pattern.
  reverse provide a stable sort of the list with the comparisions reversed.

* Amended the docs to guarantee sort stability.
2003-10-16 03:41:09 +00:00
Michael W. Hudson da0a0673b1 My last fix left n used unitialized in tha a==b case.
Fix, by not using n at all in that case.

Needs to be applied to release23-maint, too.
2003-08-15 12:06:41 +00:00
Michael W. Hudson b4f49385a3 Fix reference leak noted in test_types:
Check for a[:] = a _before_ calling PySequence_Fast on a.
release23-maint candidate
Reference leak doesn't happen with head of release22-maint.
2003-08-14 17:04:28 +00:00
Walter Dörwald e8049befdf Use _PyEval_SliceIndex to handle list.index() calls with
huge start and stop arguments. Add tests.
2003-06-17 19:27:39 +00:00
Guido van Rossum 2743d87d79 Fix sloppy index() implementation:
- don't use min() and max()
- interpret negative start/stop argument like negative slice indices
2003-06-17 14:25:14 +00:00
Raymond Hettinger d05abdec7b SF #754014: list.index() should accept optional start, end arguments
Also, modified UserList.index() to match and expanded the related tests.
2003-06-17 05:05:49 +00:00
Raymond Hettinger 6624e68546 SF bug #604716: faster [None]*n or []*n
Fulfilled request to special case repetitions of lists of length 0 or 1.
2003-05-21 05:58:46 +00:00
Raymond Hettinger 686b14d7ad SF bug #730296: Unexpected Changes in list Iterator
Reverted a Py2.3b1 change to iterator in subclasses of list and tuple.
They had been changed to use __getitem__ whenever it had been overriden
in the subclass.

This caused some usabilty and performance problems.  Also, it was
inconsistent with the rest of python where many container methods
access the underlying object directly without first checking for
an overridden getter.  Users needing a change in iterator behavior
should override it directly.
2003-05-07 01:28:47 +00:00
Martin v. Löwis cd12bfc142 Patch #708604: Check more function results. Will backport to 2.2. 2003-05-03 10:53:08 +00:00
Tim Peters 2af713c2f7 Squashed new compiler wngs about trying to compare pointers to
functions with different signatures.
2003-04-24 20:59:52 +00:00
Raymond Hettinger 9928571f3f SF bug 665835: filter() treatment of str and tuple inconsistent
As a side issue on this bug, it was noted that list and tuple iterators
used macros to directly access containers and would not recognize
__getitem__ overrides.  If the method is overridden, the patch returns
a generic sequence iterator which calls the __getitem__ method; otherwise,
it returns a high custom iterator with direct access to container elements.
2003-04-24 16:52:47 +00:00
Guido van Rossum 3a3cca5b82 - list.insert(i, x) now interprets negative i as it would be
interpreted by slicing, so negative values count from the end of the
  list.  This was the only place where such an interpretation was not
  placed on a list index.
2003-04-14 20:58:14 +00:00
Raymond Hettinger 1da1dbf458 Renamed PyObject_GenericGetIter to PyObject_SelfIter
to more accurately describe what the function does.

Suggested by Thomas Wouters.
2003-03-17 19:46:11 +00:00
Raymond Hettinger 0153826964 Created PyObject_GenericGetIter().
Factors out the common case of returning self.
2003-03-17 08:24:35 +00:00
Skip Montanaro 4abd5f0fce Allow list sort's comparison function to explicitly be None. See SF patch
661092.
2003-01-02 20:51:08 +00:00
Raymond Hettinger ea3fdf44a2 SF patch #659536: Use PyArg_UnpackTuple where possible.
Obtain cleaner coding and a system wide
performance boost by using the fast, pre-parsed
PyArg_Unpack function instead of PyArg_ParseTuple
function which is driven by a format string.
2002-12-29 16:33:45 +00:00
Raymond Hettinger f8bcfb13f1 SF Bug 645777: list.extend() works with any iterable and is no longer
experimental.
2002-12-29 05:49:09 +00:00
Michael W. Hudson a69c030c15 The final tweaks before closing
[ 633152 ] list slice ass ignores subtypes of list

Allow arbitrary sequences on the RHS of extended slices.
2002-12-05 21:32:32 +00:00
Tim Peters b9099c3df4 SF patch 637176: list.sort crasher
Armin Rigo's Draconian but effective fix for

SF bug 453523: list.sort crasher

slightly fiddled to catch more cases of list mutation.  The dreaded
internal "immutable list type" is gone!  OTOH, if you look at a list
*while* it's being sorted now, it will appear to be empty.  Better
than a core dump.
2002-11-12 22:08:10 +00:00
Neal Norwitz 03b109afc0 Use PyOS_snprintf() instead of sprintf and wrap the long line 2002-11-05 22:41:37 +00:00
Michael W. Hudson 5da854fe51 This is Alex Martelli's patch
[ 633870 ] allow any seq assignment to a list slice

plus a very silly little test case of my own.
2002-11-05 17:38:05 +00:00
Guido van Rossum bfa5a14adb Darn! Don't divide by zero. Bad fix. :-) 2002-10-11 23:39:35 +00:00
Guido van Rossum a5c0e6d6c8 Add checks for size overflow on list*n, list+list, tuple+tuple.
Will backport.
2002-10-11 21:05:56 +00:00
Neal Norwitz bb9c5f5032 PyObject_RichCompareBool() already returns -1, 0, or 1, so return its value 2002-09-05 21:32:55 +00:00
Raymond Hettinger aae5999b44 Micro-optimization for list_contains. Factored double if test
out of the loop.
2002-09-05 14:23:49 +00:00
Tim Peters e05f65a0c6 1. Combined the base and length arrays into a single array of structs.
This is friendlier for caches.

2. Cut MIN_GALLOP to 7, but added a per-sort min_gallop vrbl that adapts
   the "get into galloping mode" threshold higher when galloping isn't
   paying, and lower when it is.  There's no known case where this hurts.
   It's (of course) neutral for /sort, \sort and =sort.  It also happens
   to be neutral for !sort.  It cuts a tiny # of compares in 3sort and +sort.
   For *sort, it reduces the # of compares to better than what this used to
   do when MIN_GALLOP was hardcoded to 10 (it did about 0.1% more *sort
   compares before, but given how close we are to the limit, this is "a
   lot"!).  %sort used to do about 1.5% more compares, and ~sort about
   3.6% more.  Here are exact counts:

 i    *sort    3sort    +sort    %sort    ~sort    !sort
15   449235    33019    33016    51328   188720    65534  before
     448885    33016    33007    50426   182083    65534  after
      0.08%    0.01%    0.03%    1.79%    3.65%    0.00%  %ch from after

16   963714    65824    65809   103409   377634   131070
     962991    65821    65808   101667   364341   131070
      0.08%    0.00%    0.00%    1.71%    3.65%    0.00%

17  2059092   131413   131362   209130   755476   262142
    2057533   131410   131361   206193   728871   262142
      0.08%    0.00%    0.00%    1.42%    3.65%    0.00%

18  4380687   262440   262460   421998  1511174   524286
    4377402   262437   262459   416347  1457945   524286
      0.08%    0.00%    0.00%    1.36%    3.65%    0.00%

19  9285709   524581   524634   848590  3022584  1048574
    9278734   524580   524633   837947  2916107  1048574
      0.08%    0.00%    0.00%    1.27%    3.65%    0.00%

20 19621118  1048960  1048942  1715806  6045418  2097150
   19606028  1048958  1048941  1694896  5832445  2097150
      0.08%    0.00%    0.00%    1.23%    3.65%    0.00%

3. Added some key asserts I overlooked before.

4. Updated the doc file.
2002-08-10 05:21:15 +00:00
Tim Peters 6063e2615f PyList_Reverse(): This was leaking a reference to Py_None on every call.
I believe I introduced this bug when I refactored the reversal code so
that the mergesort could use it too.  It's not a problem on the 2.2 branch.
2002-08-08 01:06:39 +00:00
Tim Peters 66860f6da4 Sped the usual case for sorting by calling PyObject_RichCompareBool
directly when no comparison function is specified.  This saves a layer
of function call on every compare then.  Measured speedups:

 i    2**i  *sort  \sort  /sort  3sort  +sort  %sort  ~sort  =sort  !sort
15   32768  12.5%   0.0%   0.0% 100.0%   0.0%  50.0% 100.0% 100.0% -50.0%
16   65536   8.7%   0.0%   0.0%   0.0%   0.0%   0.0%  12.5%   0.0%   0.0%
17  131072   8.0%  25.0%   0.0%  25.0%   0.0%  14.3%   5.9%   0.0%   0.0%
18  262144   6.3% -10.0%  12.5%  11.1%   0.0%   6.3%   5.6%  12.5%   0.0%
19  524288   5.3%   5.9%   0.0%   5.6%   0.0%   5.9%   5.4%   0.0%   2.9%
20 1048576   5.3%   2.9%   2.9%   5.1%   2.8%   1.3%   5.9%   2.9%   4.2%

The best indicators are those that take significant time (larger i), and
where sort doesn't do very few compares (so *sort and ~sort benefit most
reliably).  The large numbers are due to roundoff noise combined with
platform variability; e.g., the 14.3% speedup for %sort at i=17 reflects
a printed elapsed time of 0.18 seconds falling to 0.17, but a change in
the last digit isn't really meaningful (indeed, if it really took 0.175
seconds, one electron having a lazy nanosecond could shift it to either
value <wink>).  Similarly the 25% at 3sort i=17 was a meaningless change
from 0.05 to 0.04.  However, almost all the "meaningless changes" were
in the same direction, which is good.  The before-and-after times for
*sort are clearest:

before after
  0.18  0.16
  0.25  0.23
  0.54  0.50
  1.18  1.11
  2.57  2.44
  5.58  5.30
2002-08-04 17:47:26 +00:00
Tim Peters 6bdbc9e0b1 SF bug 590366: Small typo in listsort:ParseTuple
The PyArg_ParseTuple() error string still said "msort".  Changed to "sort".
2002-08-03 02:28:24 +00:00
Tim Peters a64dc245ac Replaced samplesort with a stable, adaptive mergesort. 2002-08-01 02:13:36 +00:00
Michael W. Hudson 56796f672f Fix for
[ 587875 ] crash on deleting extended slice

The array code got simpler, always a good thing!
2002-07-29 14:35:04 +00:00
Martin v. Löwis 673c0a2247 Patch #574867: Correct list.extend docstring. 2002-07-28 16:35:57 +00:00
Tim Peters 330f9e9581 More sort cleanup: Moved the special cases from samplesortslice into
listsort.  If the former calls itself recursively, they're a waste of
time, since it's called on a random permutation of a random subset of
elements.  OTOH, for exactly the same reason, they're an immeasurably
small waste of time (the odds of finding exploitable order in a random
permutation are ~= 0, so the special-case loops looking for order give
up quickly).  The point is more for conceptual clarity.
Also changed some "assert comments" into real asserts; when this code
was first written, Python.h didn't supply assert.h.
2002-07-19 07:05:44 +00:00
Tim Peters 0fe977c4a9 binarysort() cleanup: Documented the key invariants, explained why they
imply this is a stable sort, and added some asserts.
2002-07-19 06:12:32 +00:00
Tim Peters 326b44871e listreverse(): Don't call the new reverse_slice unless the list
has something in it (else ob_item may be a NULL pointer).
2002-07-19 04:04:16 +00:00
Tim Peters a8c974c157 Cleanup yielding a small speed boost: before rich comparisons were
introduced, list.sort() was rewritten to use only the "< or not <?"
distinction.  After rich comparisons were introduced, docompare() was
fiddled to translate a Py_LT Boolean result into the old "-1 for <,
0 for ==, 1 for >" flavor of outcome, and the sorting code was left
alone.  This left things more obscure than they should be, and turns
out it also cost measurable cycles.

So:  The old CMPERROR novelty is gone.  docompare() is renamed to islt(),
and now has the same return conditinos as PyObject_RichCompareBool.  The
SETK macro is renamed to ISLT, and is even weirder than before (don't
complain unless you want to maintain the sort code <wink>).

Overall, this yields a 1-2% speedup in the usual (no explicit function
passed to list.sort()) case when sorting arrays of floats (as sortperf.py
does).  The boost is higher for arrays of ints.
2002-07-19 03:30:57 +00:00
Tim Peters 3b01a1217f Trimmed trailing whitespace. 2002-07-19 02:35:45 +00:00
Tim Peters 8e2e7ca330 Cleanup: Define one internal utility for reversing a list slice, and
use that everywhere.
2002-07-19 02:33:08 +00:00
Jeremy Hylton 938ace69a0 staticforward bites the dust.
The staticforward define was needed to support certain broken C
compilers (notably SCO ODT 3.0, perhaps early AIX as well) botched the
static keyword when it was used with a forward declaration of a static
initialized structure.  Standard C allows the forward declaration with
static, and we've decided to stop catering to broken C compilers.  (In
fact, we expect that the compilers are all fixed eight years later.)

I'm leaving staticforward and statichere defined in object.h as
static.  This is only for backwards compatibility with C extensions
that might still use it.

XXX I haven't updated the documentation.
2002-07-17 16:30:39 +00:00
Guido van Rossum 6b6272c857 Whitespace normalization. 2002-07-16 20:10:23 +00:00