Commit Graph

136 Commits

Author SHA1 Message Date
Raymond Hettinger b7d05db0be Optimize tuple_slice() and make further improvements to list_slice()
and list.extend().  Factoring the inner loops to remove the constant
structure references and fixed offsets gives speedups ranging from
20% to 30%.
2004-03-08 07:25:05 +00:00
Raymond Hettinger cb2da43db8 Extended tuple's C API to include a new function, PyTuple_Pack() that is
useful for rapidly building argument tuples without having to invoke the
more sophisticated machinery of Py_BuildValue().
2003-10-12 18:24:34 +00:00
Raymond Hettinger 686b14d7ad SF bug #730296: Unexpected Changes in list Iterator
Reverted a Py2.3b1 change to iterator in subclasses of list and tuple.
They had been changed to use __getitem__ whenever it had been overriden
in the subclass.

This caused some usabilty and performance problems.  Also, it was
inconsistent with the rest of python where many container methods
access the underlying object directly without first checking for
an overridden getter.  Users needing a change in iterator behavior
should override it directly.
2003-05-07 01:28:47 +00:00
Tim Peters 2af713c2f7 Squashed new compiler wngs about trying to compare pointers to
functions with different signatures.
2003-04-24 20:59:52 +00:00
Raymond Hettinger 9928571f3f SF bug 665835: filter() treatment of str and tuple inconsistent
As a side issue on this bug, it was noted that list and tuple iterators
used macros to directly access containers and would not recognize
__getitem__ overrides.  If the method is overridden, the patch returns
a generic sequence iterator which calls the __getitem__ method; otherwise,
it returns a high custom iterator with direct access to container elements.
2003-04-24 16:52:47 +00:00
Raymond Hettinger 1da1dbf458 Renamed PyObject_GenericGetIter to PyObject_SelfIter
to more accurately describe what the function does.

Suggested by Thomas Wouters.
2003-03-17 19:46:11 +00:00
Raymond Hettinger 0153826964 Created PyObject_GenericGetIter().
Factors out the common case of returning self.
2003-03-17 08:24:35 +00:00
Guido van Rossum 5d9113d8be Implement appropriate __getnewargs__ for all immutable subclassable builtin
types.  The special handling for these can now be removed from save_newobj().
Add some testing for this.

Also add support for setting the 'fast' flag on the Python Pickler class,
which suppresses use of the memo.
2003-01-29 17:58:45 +00:00
Guido van Rossum a5c0e6d6c8 Add checks for size overflow on list*n, list+list, tuple+tuple.
Will backport.
2002-10-11 21:05:56 +00:00
Neal Norwitz bb9c5f5032 PyObject_RichCompareBool() already returns -1, 0, or 1, so return its value 2002-09-05 21:32:55 +00:00
Raymond Hettinger bd9adab138 Micro-optimization for list_contains. Factored double if test
out of the loop.
2002-09-05 20:18:08 +00:00
Guido van Rossum e3a8e7ed1d Call me anal, but there was a particular phrase that was speading to
comments everywhere that bugged me: /* Foo is inlined */ instead of
/* Inline Foo */.  Somehow the "is inlined" phrase always confused me
for half a second (thinking, "No it isn't" until I added the missing
"here").  The new phrase is hopefully unambiguous.
2002-08-19 19:26:42 +00:00
Raymond Hettinger 48923c5533 Moved special case for tuples from iterobject.c to
tupleobject.c. Makes the code in iterobject.c cleaner
and speeds-up the general case by not checking for
tuples everytime.   SF Patch #592065.
2002-08-09 01:30:17 +00:00
Jeremy Hylton 938ace69a0 staticforward bites the dust.
The staticforward define was needed to support certain broken C
compilers (notably SCO ODT 3.0, perhaps early AIX as well) botched the
static keyword when it was used with a forward declaration of a static
initialized structure.  Standard C allows the forward declaration with
static, and we've decided to stop catering to broken C compilers.  (In
fact, we expect that the compilers are all fixed eight years later.)

I'm leaving staticforward and statichere defined in object.h as
static.  This is only for backwards compatibility with C extensions
that might still use it.

XXX I haven't updated the documentation.
2002-07-17 16:30:39 +00:00
Tim Peters 3459251d5a object.h special-build macro minefield: renamed all the new lexical
helper macros to something saner, and used them appropriately in other
files too, to reduce #ifdef blocks.

classobject.c, instance_dealloc():  One of my worst Python Memories is
trying to fix this routine a few years ago when COUNT_ALLOCS was defined
but Py_TRACE_REFS wasn't.  The special-build code here is way too
complicated.  Now it's much simpler.  Difference:  in a Py_TRACE_REFS
build, the instance is no longer in the doubly-linked list of live
objects while its __del__ method is executing, and that may be visible
via sys.getobjects() called from a __del__ method.  Tough -- the object
is presumed dead while its __del__ is executing anyway, and not calling
_Py_NewReference() at the start allows enormous code simplification.

typeobject.c, call_finalizer():  The special-build instance_dealloc()
pain apparently spread to here too via cut-'n-paste, and this is much
simpler now too.  In addition, I didn't understand why this routine
was calling _PyObject_GC_TRACK() after a resurrection, since there's no
plausible way _PyObject_GC_UNTRACK() could have been called on the
object by this point.  I suspect it was left over from pasting the
instance_delloc() code.  Instead asserted that the object is still
tracked.  Caution:  I suspect we don't have a test that actually
exercises the subtype_dealloc() __del__-resurrected-me code.
2002-07-11 06:23:50 +00:00
Jeremy Hylton 8b47dffc93 Fix for SF bug 571885
When resizing a tuple, zero out the memory starting at the end of the
old tuple not at the beginning of the old tuple.
2002-06-20 23:13:17 +00:00
Neal Norwitz 35fc7606f0 SF #561244 Micro optimizations
Convert loops to memset()s.
2002-06-13 21:11:11 +00:00
Martin v. Löwis 14f8b4cfcb Patch #568124: Add doc string macros. 2002-06-13 20:33:02 +00:00
Michael W. Hudson 5efaf7eac8 This is my nearly two year old patch
[ 400998 ] experimental support for extended slicing on lists

somewhat spruced up and better tested than it was when I wrote it.

Includes docs & tests.  The whatsnew section needs expanding, and arrays
should support extended slices -- later.
2002-06-11 10:55:12 +00:00
Neil Schemenauer 626d774df6 PyObject_GC_Del can now be used as a function designator. 2002-04-12 03:05:52 +00:00
Guido van Rossum ff413af605 This is Neil's fix for SF bug 535905 (Evil Trashcan and GC interaction).
The fix makes it possible to call PyObject_GC_UnTrack() more than once
on the same object, and then move the PyObject_GC_UnTrack() call to
*before* the trashcan code is invoked.

BUGFIX CANDIDATE!
2002-03-28 20:34:59 +00:00
Guido van Rossum f70590f990 _PyTuple_Resize(): this dumped core on tuple(globals()) for me. Turns
out the for loop at the end intended to zero out new items wasn't
doing anything, because sv->ob_size was already equal to newsize.  The
fix slightly refactors the function, introducing a variable oldsize
and doing away with sizediff (which was used only once), and using
oldsize and newsize consistently.  I also added comments explaining
what the two for loops do.  (Looking at the CVS annotation of this
function, it's no miracle a bug crept in -- this has been patched by
many different folks! :-)
2001-12-07 20:00:04 +00:00
Guido van Rossum 9475a2310d Enable GC for new-style instances. This touches lots of files, since
many types were subclassable but had a xxx_dealloc function that
called PyObject_DEL(self) directly instead of deferring to
self->ob_type->tp_free(self).  It is permissible to set tp_free in the
type object directly to _PyObject_Del, for non-GC types, or to
_PyObject_GC_Del, for GC types.  Still, PyObject_DEL was a tad faster,
so I'm fearing that our pystone rating is going down again.  I'm not
sure if doing something like

void xxx_dealloc(PyObject *self)
{
	if (PyXxxCheckExact(self))
		PyObject_DEL(self);
	else
		self->ob_type->tp_free(self);
}

is any faster than always calling the else branch, so I haven't
attempted that -- however those types whose own dealloc is fancier
(int, float, unicode) do use this pattern.
2001-10-05 20:51:39 +00:00
Tim Peters 7b07a41e9f The endless 460020 bug.
Disable t[:], t*0, t*1 optimizations when t is of a tuple subclass type.
2001-09-11 19:48:03 +00:00
Tim Peters 1b8ca0d87a Rewrite the tuple() docstring to parallel the list() docstring. 2001-09-02 06:42:25 +00:00
Tim Peters 9577761337 Repair apparent cut'n'pasteo in tuple() docstring. 2001-09-02 06:29:48 +00:00
Guido van Rossum 4b8c0f6d7d More stuff discovered while writing the simplest of testcases:
tupledealloc(): only feed the free list when the type is really a
tuple, not a subtype.  Otherwise, use PyObject_GC_Del().

_PyTuple_Resize(): disallow using this for tuple subtypes.
2001-08-30 18:31:30 +00:00
Guido van Rossum ae960afb5e Make str and tuple types subclassable. 2001-08-30 03:11:59 +00:00
Neil Schemenauer e83c00efd0 Use new GC API. 2001-08-29 23:54:21 +00:00
Tim Peters 6d6c1a35e0 Merge of descr-branch back into trunk. 2001-08-02 04:15:00 +00:00
Tim Peters a7259597f1 SF bug 433228: repr(list) woes when len(list) big.
Gave Python linear-time repr() implementations for dicts, lists, strings.
This means, e.g., that repr(range(50000)) is no longer 50x slower than
pprint.pprint() in 2.2 <wink>.

I don't consider this a bugfix candidate, as it's a performance boost.

Added _PyString_Join() to the internal string API.  If we want that in the
public API, fine, but then it requires runtime error checks instead of
asserts.
2001-06-16 05:11:17 +00:00
Thomas Wouters 0dcea5973d _PyTuple_Resize: guard against PyTuple_New() returning NULL, using Tim's
suggestion (modulo style).
2001-05-29 07:58:45 +00:00
Tim Peters 4324aa3572 Cruft cleanup: Removed the unused last_is_sticky argument from the internal
_PyTuple_Resize().
2001-05-28 22:30:08 +00:00
Thomas Wouters 6a922372ad _PyTuple_Resize: take into account the empty tuple. There can be only one.
Instead of raising a SystemError, just create a new tuple of the desired
size.

This fixes (at least) SF bug #420343.
2001-05-28 13:11:02 +00:00
Tim Peters d7ed3bf552 Speed tuple comparisons in two ways:
1. Omit the early-out EQ/NE "lengths different?" test.  Was unable to find
   any real code where it triggered, but it always costs.  The same is not
   true of list richcmps, where different-size lists appeared to get
   compared about half the time.
2. Because tuples are immutable, there's no need to refetch the lengths of
   both tuples from memory again on each loop trip.

BUG ALERT:  The tuple (and list) richcmp algorithm is arguably wrong,
because it won't believe there's any difference unless Py_EQ returns false
for some corresponding elements:

>>> class C:
...     def __lt__(x, y): return 1
...     __eq__ = __lt__
...
>>> C() < C()
1
>>> (C(),) < (C(),)
0
>>>

That doesn't make sense -- provided you believe the defn. of C makes sense.
2001-05-15 20:12:59 +00:00
Guido van Rossum f77bc62e73 Same treatment as listobject.c:
- tuplecontains(): call RichCompare(Py_EQ).

- Get rid of tuplecompare(), in favor of new tuplerichcompare() (a
  clone of list_compare()).

- Aligned the comments for large struct initializers.
2001-01-18 00:00:53 +00:00
Neil Schemenauer 08b53e6a2a Simplify _PyTuple_Resize by not using the tuple free list and dropping
support for the last_is_sticky flag.  A few hard to find bugs may be
fixed by this patch since the old code was buggy.
2000-10-05 19:36:49 +00:00
Martin v. Löwis 3cd760425f Correctly cast the return value of realloc. 2000-09-15 07:32:39 +00:00
Martin v. Löwis c58dbebf4b Correctly use realloc return value. Fixes bug #114424. 2000-09-15 07:07:46 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Fred Drake ba09633e1e ANSI-fication of the sources. 2000-07-09 07:04:36 +00:00
Guido van Rossum 4cc6ac7b87 Neil Schemenauer: small fixes for GC 2000-07-01 01:00:38 +00:00
Guido van Rossum ffcc3813d8 Change copyright notice - 2nd try. 2000-06-30 23:58:06 +00:00
Guido van Rossum fd71b9e9d4 Change copyright notice. 2000-06-30 23:50:40 +00:00
Jeremy Hylton c5007aa5c3 final patches from Neil Schemenauer for garbage collection 2000-06-30 05:02:53 +00:00
Jeremy Hylton d08b4c4524 part 2 of Neil Schemenauer's GC patches:
This patch modifies the type structures of objects that
participate in GC.  The object's tp_basicsize is increased when
GC is enabled.  GC information is prefixed to the object to
maintain binary compatibility.  GC objects also define the
tp_flag Py_TPFLAGS_GC.
2000-06-23 19:37:02 +00:00
Jeremy Hylton 8caad49c30 Round 1 of Neil Schemenauer's GC patches:
This patch adds the type methods traverse and clear necessary for GC
implementation.
2000-06-23 14:18:11 +00:00
Marc-André Lemburg 29dc381ce0 Michael Hudson <mwh21@cam.ac.uk>:
The error message refers to "append", yet the operation in
question is "concat".
2000-06-16 17:05:57 +00:00
Fred Drake 56780257c6 Thomas Wouters <thomas@xs4all.net>:
The following patch adds "sq_contains" support to rangeobject, and enables
the already-written support for sq_contains in listobject and tupleobject.

The rangeobject "contains" code should be a bit more efficient than the
current default "in" implementation ;-) It might not get used much, but it's
not that much to add.

listobject.c and tupleobject.c already had code for sq_contains, and the
proper struct member was set, but the PyType structure was not extended to
include tp_flags, so the object-specific code was not getting called (Go
ahead, test it ;-). I also did this for the immutable_list_type in
listobject.c, eventhough it is probably never used. Symmetry and all that.
2000-06-15 14:50:20 +00:00
Fred Drake b6a9ada757 Michael Hudson <mwh21@cam.ac.uk>:
Removed PyErr_BadArgument() calls and replaced them with more useful
error messages.
2000-06-01 03:12:13 +00:00
Guido van Rossum b18618dab7 Vladimir Marangozov's long-awaited malloc restructuring.
For more comments, read the patches@python.org archives.
For documentation read the comments in mymalloc.h and objimpl.h.

(This is not exactly what Vladimir posted to the patches list; I've
made a few changes, and Vladimir sent me a fix in private email for a
problem that only occurs in debug mode.  I'm also holding back on his
change to main.c, which seems unnecessary to me.)
2000-05-03 23:44:39 +00:00
Jeremy Hylton 37b1a26c89 add list_contains and tuplecontains: efficient implementations of tp_contains 2000-04-27 21:41:03 +00:00
Guido van Rossum 5ce78f8e4e Patch by Charles G Waldman to avoid a sneaky memory leak in
_PyTuple_Resize().  In addition, a change suggested by Jeremy Hylton
to limit the size of the free lists is also merged into this patch.

Charles wrote initially:

"""
Test Case:  run the following code:

class Nothing:
    def __len__(self):
        return 5
    def __getitem__(self, i):
        if i < 3:
            return i
        else:
            raise IndexError, i

def g(a,*b,**c):
    return

for x in xrange(1000000):
    g(*Nothing())


and watch Python's memory use go up and up.


Diagnosis:

The analysis begins with the call to PySequence_Tuple at line 1641 in
ceval.c - the argument to g is seen to be a sequence but not a tuple,
so it needs to be converted from an abstract sequence to a concrete
tuple.  PySequence_Tuple starts off by creating a new tuple of length
5 (line 1122 in abstract.c).  Then at line 1149, since only 3 elements
were assigned, _PyTuple_Resize is called to make the 5-tuple into a
3-tuple.  When we're all done the 3-tuple is decrefed, but rather than
being freed it is placed on the free_tuples cache.

The basic problem is that the 3-tuples are being added to the cache
but never picked up again, since _PyTuple_Resize doesn't make use of
the free_tuples cache.  If you are resizing a 5-tuple to a 3-tuple and
there is already a 3-tuple in free_tuples[3], instead of using this
tuple, _PyTuple_Resize will realloc the 5-tuple to a 3-tuple.  It
would more efficient to use the existing 3-tuple and cache the
5-tuple.

By making _PyTuple_Resize aware of the free_tuples (just as
PyTuple_New), we not only save a few calls to realloc, but also
prevent this misbehavior whereby tuples are being added to the
free_tuples list but never properly "recycled".
"""

And later:

"""
This patch replaces my submission of Sun, 16 Apr and addresses Jeremy
Hylton's suggestions that we also limit the size of the free tuple
list.  I chose 2000 as the maximum number of tuples of any particular
size to save.

There was also a problem with the previous version of this patch
causing a core dump if Python was built with Py_TRACE_REFS.  This is
fixed in the below version of the patch, which uses tupledealloc
instead of _Py_Dealloc.
"""
2000-04-21 21:15:05 +00:00
Guido van Rossum d724b23420 Christian Tismer's "trashcan" patch:
Added wrapping macros to dictobject.c, listobject.c, tupleobject.c,
frameobject.c, traceback.c that safely prevends core dumps
on stack overflow. Macros and functions in object.c, object.h.
The method is an "elevator destructor" that turns cascading
deletes into tail recursive behavior when some limit is hit.
2000-03-13 16:01:29 +00:00
Guido van Rossum bffd683f73 The rest of the changes by Trent Mick and Dale Nagata for warning-free
compilation on NT Alpha.  Mostly added casts etc.
2000-01-20 22:32:56 +00:00
Guido van Rossum 0eb55ac912 Mark Favas was quick to note that the last checkin divides by zero
when n == 0...  So divide by a->ob_size instead which was already
tested for 0.
1999-07-13 05:41:12 +00:00
Guido van Rossum 5bc51f2f27 Appropriate overflow checks so that things like sys.maxint*(1,) can't
dump core.
1999-07-12 23:06:58 +00:00
Guido van Rossum 68055ce6fe When tracing references, reset the type and size of tuples allocated
from the fast free list -- the type (at least) is reset by
_Py_Dealloc().
1998-12-11 14:56:38 +00:00
Guido van Rossum 017f7780a8 Make tuples less hungry -- an extra item was allocated but never used.
Tip by Vladimir Marangozov.
1998-11-16 18:56:03 +00:00
Guido van Rossum 1bb26872f5 Slight rearrangement of some code to make it faster, by Vladimir
Marangozov.
1998-06-26 15:53:50 +00:00
Guido van Rossum 787bdd37a0 PyTuple_SetItem should require that the tuple's refcnt is one! 1997-08-17 16:25:45 +00:00
Guido van Rossum fbbd57e4ca Added _Fini() routines to free up some memory 1997-08-05 02:16:08 +00:00
Guido van Rossum c0b618a2cc Quickly renamed the last directory. 1997-05-02 03:12:38 +00:00
Guido van Rossum 0969ad213d Better tuple hash function. 1996-12-16 17:55:46 +00:00
Guido van Rossum d266eb460e New permission notice, includes CNRI. 1996-10-25 14:44:06 +00:00
Guido van Rossum 441e4ab802 new debugger symbol names 1996-05-23 22:46:51 +00:00
Guido van Rossum 055968c068 better err checks in resizetuple 1995-08-04 04:05:10 +00:00
Guido van Rossum 6f9e433ab3 fix dusty debugging macros 1995-03-29 16:57:48 +00:00
Guido van Rossum 5fe605889a a few peephole optimizations 1995-03-09 12:12:50 +00:00
Guido van Rossum 6610ad9d6b Added 1995 to copyright message.
floatobject.c: fix hash().
methodobject.c: support METH_FREENAME flag bit.
1995-01-04 19:07:38 +00:00
Guido van Rossum 1d5735e846 Merge back to main trunk 1994-08-30 08:27:36 +00:00
Guido van Rossum a3d78fb268 * posixmodule.c: added set{uid,gid}.
* {tuple,list,mapping,array}object.c: call printobject with 0 for flags
* compile.c (parsestr): use quote instead of '\'' at one crucial point
* arraymodule.c (array_getattr): Added __members__ attribute
1993-11-10 09:23:53 +00:00
Sjoerd Mullender 615194a352 Fixed bugs in resizetuple and extended the interface.
Added ifdefs in stringobject.c for shared strings of length 1.
Renamed free_list in tupleobject.c to free_tuples.
1993-11-01 13:46:50 +00:00
Guido van Rossum 12d12c5faf * compile.[ch]: support for lambda()
* PROTO.h, mymalloc.h: added #ifdefs for TURBOC and GNUC.
* allobjects.h: added #include "rangeobject.h"
* Grammar: added lambda_input; relaxed syntax for exec.
* bltinmodule.c: added bagof, map, reduce, lambda, xrange.
* tupleobject.[ch]: added resizetuple().
* rangeobject.[ch]: new object type to speed up range operations (not
  convinced this is needed!!!)
1993-10-26 17:58:25 +00:00
Sjoerd Mullender 842d2ccdcd intobject.c: Save references to small integers, so that they can be
shared.  The default is to save references to the integers in
	the range -1..99.  The lower limit can be set by defining
	NSMALLNEGINTS (absolute value of smallest integer to be saved)
	and NSMALLPOSINTS (1 more than the largest integer to be
	saved).
tupleobject.c: Save a reference to the empty tuple to be returned
	whenever a tuple of size 0 is requested.  Tuples of size 1
	upto, but not including, MAXSAVESIZE (default 20) are put in
	free lists when deallocated.  When MAXSAVESIZE equals 1, only
	share references to the empty tuple, when MAXSAVESIZE equals
	0, don't include the code at all and revert to the old
	behavior.
object.c: Print some more statistics when COUNT_ALLOCS is defined.
1993-10-15 16:18:48 +00:00
Sjoerd Mullender a9c3c22c33 * Extended X interface: pixmap objects, colormap objects visual objects,
image objects, and lots of new methods.
* Added counting of allocations and deallocations of builtin types if
  COUNT_ALLOCS is defined.  Had to move calls to NEWREF down in some
  files.
* Bug fix in sorting lists.
1993-10-11 12:54:31 +00:00
Guido van Rossum 234f942aef * Added gmtime/localtime/mktime and SYSV timezone globals to timemodule.c.
Added $(SYSDEF) to its build rule in Makefile.
* cgensupport.[ch], modsupport.[ch]: removed some old stuff.  Also
  changed files that still used it...  And made several things static
  that weren't but should have been...  And other minor cleanups...
* listobject.[ch]: add external interfaces {set,get}listslice
* socketmodule.c: fix bugs in new send() argument parsing.
* sunaudiodevmodule.c: added flush() and close().
1993-06-17 12:35:49 +00:00
Guido van Rossum 9bfef44d97 * Changed all copyright messages to include 1993.
* Stubs for faster implementation of local variables (not yet finished)
* Added function name to code object.  Print it for code and function
  objects.  THIS MAKES THE .PYC FILE FORMAT INCOMPATIBLE (the version
  number has changed accordingly)
* Print address of self for built-in methods
* New internal functions getattro and setattro (getattr/setattr with
  string object arg)
* Replaced "dictobject" with more powerful "mappingobject"
* New per-type functio tp_hash to implement arbitrary object hashing,
  and hashobject() to interface to it
* Added built-in functions hash(v) and hasattr(v, 'name')
* classobject: made some functions static that accidentally weren't;
  added __hash__ special instance method to implement hash()
* Added proper comparison for built-in methods and functions
1993-03-29 10:43:31 +00:00
Guido van Rossum bab9d03855 Copyright for 1992 added 1992-04-05 14:26:55 +00:00
Guido van Rossum 7c36ad7f44 New function gettupleslice(v, i, j). 1992-01-14 18:45:33 +00:00
Guido van Rossum 49e85146e2 printobject now returns an error code
Remove superfluous err_nomem() call
,
1991-06-07 22:59:30 +00:00
Guido van Rossum b8393da8f8 Finally implement tuple*number. From now on all sequence types
must (pretend to) support all operations except assignments;
if you don't want to support an operation you have to provide
a dummy function that raises an exception...
1991-06-04 19:35:24 +00:00
Guido van Rossum f70e43a073 Added copyright notice. 1991-02-19 12:39:46 +00:00
Guido van Rossum 3f5da24ea3 "Compiling" version 1990-12-20 15:06:42 +00:00
Guido van Rossum 2a9096b5f9 New errors. 1990-10-21 22:15:08 +00:00
Guido van Rossum 85a5fbbdfe Initial revision 1990-10-14 12:07:46 +00:00