* GH-116554: Relax list.sort()'s notion of "descending" run
Rewrote `count_run()` so that sub-runs of equal elements no longer end a descending run. Both ascending and descending runs can have arbitrarily many sub-runs of arbitrarily many equal elements now. This is tricky, because we only use ``<`` comparisons, so checking for equality doesn't come "for free". Surprisingly, it turned out there's a very cheap (one comparison) way to determine whether an ascending run consisted of all-equal elements. That sealed the deal.
In addition, after a descending run is reversed in-place, we now go on to see whether it can be extended by an ascending run that just happens to be adjacent. This succeeds in finding at least one additional element to append about half the time, and so appears to more than repay its cost (the savings come from getting to skip a binary search, when a short run is artificially forced to length MIINRUN later, for each new element `count_run()` can add to the initial run).
While these have been in the back of my mind for years, a question on StackOverflow pushed it to action:
https://stackoverflow.com/questions/78108792/
They were wondering why it took about 4x longer to sort a list like:
[999_999, 999_999, ..., 2, 2, 1, 1, 0, 0]
than "similar" lists. Of course that runs very much faster after this patch.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
I have compared output between pre- and post-patch runs of these tests
to make sure there's nothing missing and nothing broken, on both
Windows and Linux. The only differences I found were actually tests
that were previously *not* run.
svn+ssh://pythondev@svn.python.org/python/branches/p3yk
........
r55329 | brett.cannon | 2007-05-14 16:36:56 -0700 (Mon, 14 May 2007) | 3 lines
Implement the removal of tuple parameter unpacking (PEP 3113).
Thanks, Tony Lownds for the patch.
........
r55331 | neal.norwitz | 2007-05-14 16:40:30 -0700 (Mon, 14 May 2007) | 1 line
Update to use Python 3.0
........
r55332 | brett.cannon | 2007-05-14 16:47:18 -0700 (Mon, 14 May 2007) | 2 lines
Mention PEP 3113. And thanks to Tony Lownds for the PEP 3113 patch.
........
r55333 | neal.norwitz | 2007-05-14 16:57:06 -0700 (Mon, 14 May 2007) | 1 line
Fix exception printing (no more exceptions module)
........
r55334 | neal.norwitz | 2007-05-14 17:11:10 -0700 (Mon, 14 May 2007) | 1 line
Remove popen* functions from os
........
r55335 | neal.norwitz | 2007-05-14 18:03:38 -0700 (Mon, 14 May 2007) | 1 line
Get rid of most of popen. There are still some uses I need to cleanup.
........
r55336 | neal.norwitz | 2007-05-14 21:11:34 -0700 (Mon, 14 May 2007) | 1 line
Remove a few more remnants of the compiler package
........
r55337 | neal.norwitz | 2007-05-14 22:28:27 -0700 (Mon, 14 May 2007) | 1 line
Get test_[cx]pickle working on 64-bit platforms (avoid overflow int/long)
........
svn+ssh://pythondev@svn.python.org/python/branches/p3yk
........
r55077 | guido.van.rossum | 2007-05-02 11:54:37 -0700 (Wed, 02 May 2007) | 2 lines
Use the new print syntax, at least.
........
r55142 | fred.drake | 2007-05-04 21:27:30 -0700 (Fri, 04 May 2007) | 1 line
remove old cruftiness
........
r55143 | fred.drake | 2007-05-04 21:52:16 -0700 (Fri, 04 May 2007) | 1 line
make this work with the new Python
........
r55162 | neal.norwitz | 2007-05-06 22:29:18 -0700 (Sun, 06 May 2007) | 1 line
Get asdl code gen working with Python 2.3. Should continue to work with 3.0
........
r55164 | neal.norwitz | 2007-05-07 00:00:38 -0700 (Mon, 07 May 2007) | 1 line
Verify checkins to p3yk (sic) branch go to 3000 list.
........
r55166 | neal.norwitz | 2007-05-07 00:12:35 -0700 (Mon, 07 May 2007) | 1 line
Fix this test so it runs again by importing warnings_test properly.
........
r55167 | neal.norwitz | 2007-05-07 01:03:22 -0700 (Mon, 07 May 2007) | 8 lines
So long xrange. range() now supports values that are outside
-sys.maxint to sys.maxint. floats raise a TypeError.
This has been sitting for a long time. It probably has some problems and
needs cleanup. Objects/rangeobject.c now uses 4-space indents since
it is almost completely new.
........
r55171 | guido.van.rossum | 2007-05-07 10:21:26 -0700 (Mon, 07 May 2007) | 4 lines
Fix two tests that were previously depending on significant spaces
at the end of a line (and before that on Python 2.x print behavior
that has no exact equivalent in 3.0).
........
There's one major and one minor category still unfixed:
doctests are the major category (and I hope to be able to augment the
refactoring tool to refactor bona fide doctests soon);
other code generating print statements in strings is the minor category.
(Oh, and I don't know if the compiler package works.)
*ordering* between objects; there is only a default equality test
(defined by an object being equal to itself only). Read the comment
in object.c. The current implementation never uses a three-way
comparison to compute a rich comparison, but it does use a rich
comparison to compute a three-way comparison. I'm not quite done
ripping out all the calls to PyObject_Compare/Cmp, or replacing
tp_compare implementations with tp_richcompare implementations;
but much of that has happened (to make most unit tests pass).
The following tests still fail, because I need help deciding
or understanding:
test_codeop -- depends on comparing code objects
test_datetime -- need Tim Peters' opinion
test_marshal -- depends on comparing code objects
test_mutants -- need help understanding it
The problem with test_codeop and test_marshal is this: these tests
compare two different code objects and expect them to be equal.
Is that still a feature we'd like to support? I've temporarily
removed the comparison and hash code from code objects, so they
use the default (equality by pointer only) comparison.
For the other two tests, run them to see for yourself.
(There may be more failing test with "-u all".)
A general problem with getting lots of these tests to pass is
the reality that for object types that have a natural total ordering,
implementing __cmp__ is much more convenient than implementing
__eq__, __ne__, __lt__, and so on. Should we go back to allowing
__cmp__ to provide a total ordering? Should we provide some other
way to implement rich comparison with a single method override?
Alex proposed a __key__() method; I've considered a __richcmp__()
method. Or perhaps __cmp__() just shouldn't be killed off...
to NULL during the lifetime of the object.
* listobject.c nevertheless did not conform to the other invariants,
either; fixed.
* listobject.c now uses list_clear() as the obvious internal way to clear
a list, instead of abusing list_ass_slice() for that. It makes it easier
to enforce the invariant about ob_item == NULL.
* listsort() sets allocated to -1 during sort; any mutation will set it
to a value >= 0, so it is a safe way to detect mutation. A negative
value for allocated does not cause a problem elsewhere currently.
test_sort.py has a new test for this fix.
* listsort() leak: if items were added to the list during the sort, AND if
these items had a __del__ that puts still more stuff into the list,
then this more stuff (and the PyObject** array to hold them) were
overridden at the end of listsort() and never released.
* Install the unittests, docs, newsitem, include file, and makefile update.
* Exercise the new functions whereever sets.py was being used.
Includes the docs for libfuncs.tex. Separate docs for the types are
forthcoming.
key provides C support for the decorate-sort-undecorate pattern.
reverse provide a stable sort of the list with the comparisions reversed.
* Amended the docs to guarantee sort stability.
Armin Rigo's Draconian but effective fix for
SF bug 453523: list.sort crasher
slightly fiddled to catch more cases of list mutation. The dreaded
internal "immutable list type" is gone! OTOH, if you look at a list
*while* it's being sorted now, it will appear to be empty. Better
than a core dump.
in the stability tests.
Bizarre: this takes 11x longer to run if and only if test_longexp is
run before it, on my box. The bigger REPS is in test_longexp, the
slower this gets. What happens on your box? It's not gc on my box
(which is good, because gc isn't a plausible candidate here).
The slowdown is massive in the parts of test_sort that implicitly
invoke a new-style class's __lt__ or __cmp__ methods. If I boost
REPS large enough in test_longexp, even the test_sort tests on an array
of size 64 visibly c-r-a-w-l. The relative slowdown is even worse in
a debug build. And if I reduce REPS in test_longexp, the slowdown in
test_sort goes away.
test_longexp does do horrid things to Win98's management of user
address space, but I thought I had made that a whole lot better a month
or so ago (by overallocating aggressively in the parser).
Tim Peters
Jacob Walls
Rémi Lapeyre
embg
Berker Peksag
Zachary Ware
Éric Araujo
Daniel Stutzbach
Mark Dickinson
Benjamin Peterson
Raymond Hettinger
Guido van Rossum
Neal Norwitz
Armin Rigo
Michael W. Hudson
Skip Montanaro