Commit Graph

235 Commits

Author SHA1 Message Date
Tim Peters 3918fb2549 Repair typo in comment. 2001-05-10 18:58:31 +00:00
Tim Peters 95bf9390a4 SF bug #422121 Insecurities in dict comparison.
Fixed a half dozen ways in which general dict comparison could crash
Python (even cause Win98SE to reboot) in the presence of kay and/or
value comparison routines that mutate the dict during dict comparison.
Bugfix candidate.
2001-05-10 08:32:44 +00:00
Tim Peters e63415ead8 SF patch #421922: Implement rich comparison for dicts.
d1 == d2 and d1 != d2 now work even if the keys and values in d1 and d2
don't support comparisons other than ==, and testing dicts for equality
is faster now (especially when inequality obtains).
2001-05-08 04:38:29 +00:00
Guido van Rossum b1f35bffe5 Mchael Hudson pointed out that the code for detecting changes in
dictionary size was comparing ma_size, the hash table size, which is
always a power of two, rather than ma_used, wich changes on each
insertion or deletion.  Fixed this.
2001-05-02 15:13:44 +00:00
Guido van Rossum 09e563abb4 Add experimental iterkeys(), itervalues(), iteritems() to dict
objects.

Tests show that iteritems() is 5-10% faster than iterating over the
dict and extracting the value with dict[key].
2001-05-01 12:10:21 +00:00
Guido van Rossum 213c7a6aa5 Mondo changes to the iterator stuff, without changing how Python code
sees it (test_iter.py is unchanged).

- Added a tp_iternext slot, which calls the iterator's next() method;
  this is much faster for built-in iterators over built-in types
  such as lists and dicts, speeding up pybench's ForLoop with about
  25% compared to Python 2.1.  (Now there's a good argument for
  iterators. ;-)

- Renamed the built-in sequence iterator SeqIter, affecting the C API
  functions for it.  (This frees up the PyIter prefix for generic
  iterator operations.)

- Added PyIter_Check(obj), which checks that obj's type has a
  tp_iternext slot and that the proper feature flag is set.

- Added PyIter_Next(obj) which calls the tp_iternext slot.  It has a
  somewhat complex return condition due to the need for speed: when it
  returns NULL, it may not have set an exception condition, meaning
  the iterator is exhausted; when the exception StopIteration is set
  (or a derived exception class), it means the same thing; any other
  exception means some other error occurred.
2001-04-23 14:08:49 +00:00
Guido van Rossum 59d1d2b434 Iterators phase 1. This comprises:
new slot tp_iter in type object, plus new flag Py_TPFLAGS_HAVE_ITER
new C API PyObject_GetIter(), calls tp_iter
new builtin iter(), with two forms: iter(obj), and iter(function, sentinel)
new internal object types iterobject and calliterobject
new exception StopIteration
new opcodes for "for" loops, GET_ITER and FOR_ITER (also supported by dis.py)
new magic number for .pyc files
new special method for instances: __iter__() returns an iterator
iteration over dictionaries: "for x in dict" iterates over the keys
iteration over files: "for x in file" iterates over lines

TODO:

documentation
test suite
decide whether to use a different way to spell iter(function, sentinal)
decide whether "for key in dict" is a good idea
use iterators in map/filter/reduce, min/max, and elsewhere (in/not in?)
speed tuning (make next() a slot tp_next???)
2001-04-20 19:13:02 +00:00
Guido van Rossum 55ad67d74d Oops. Removed dictiter_new decl that wasn't supposed to go in yet. 2001-04-20 16:52:06 +00:00
Guido van Rossum 0dbb4fba4c Implement, test and document "key in dict" and "key not in dict".
I know some people don't like this -- if it's really controversial,
I'll take it out again.  (If it's only Alex Martelli who doesn't like
it, that doesn't count as "real controversial" though. :-)

That's why this is a separate checkin from the iterators stuff I'm
about to check in next.
2001-04-20 16:50:40 +00:00
Guido van Rossum e04eaec5b6 Tim pointed out a remaining vulnerability in popitem(): the
PyTuple_New() could *conceivably* clear the dict, so move the test for
an empty dict after the tuple allocation.  It means that we waste time
allocating and deallocating a 2-tuple when the dict is empty, but who
cares.  It also means that when the dict is empty *and* there's no
memory to allocate a 2-tuple, we raise MemoryError, not KeyError --
but that may actually a good idea: if there's no room for a lousy
2-tuple, what are the chances that there's room for a KeyError
instance?
2001-04-16 00:02:32 +00:00
Guido van Rossum a4dd011259 Tentative fix for a problem that Tim discovered at the last moment,
and reported to python-dev: because we were calling dict_resize() in
PyDict_Next(), and because GC's dict_traverse() uses PyDict_Next(),
and because PyTuple_New() can cause GC, and because dict_items() calls
PyTuple_New(), it was possible for dict_items() to have the dict
resized right under its nose.

The solution is convoluted, and touches several places: keys(),
values(), items(), popitem(), PyDict_Next(), and PyDict_SetItem().

There are two parts to it. First, we no longer call dict_resize() in
PyDict_Next(), which seems to solve the immediate problem.  But then
PyDict_SetItem() must have a different policy about when *it* calls
dict_resize(), because we want to guarantee (e.g. for an algorithm
that Jeremy uses in the compiler) that you can loop over a dict using
PyDict_Next() and make changes to the dict as long as those changes
are only value replacements for existing keys using PyDict_SetItem().
This is done by resizing *after* the insertion instead of before, and
by remembering the size before we insert the item, and if the size is
still the same, we don't bother to even check if we might need to
resize.  An additional detail is that if the dict starts out empty, we
must still resize it before the insertion.

That was the first part. :-)

The second part is to make keys(), values(), items(), and popitem()
safe against side effects on the dict caused by allocations, under the
assumption that if the GC can cause arbitrary Python code to run, it
can cause other threads to run, and it's not inconceivable that our
dict could be resized -- it would be insane to write code that relies
on this, but not all code is sane.

Now, I have this nagging feeling that the loops in lookdict probably
are blissfully assuming that doing a simple key comparison does not
change the dict's size.  This is not necessarily true (the keys could
be class instances after all).  But that's a battle for another day.
2001-04-15 22:16:26 +00:00
Tim Peters 6783070ebf Make PyDict_Next safe to use for loops that merely modify the values
associated with existing dict keys.
This is a variant of part of Michael Hudson's patch #409864 "lazy fix for
Pings bizarre scoping crash".
2001-03-21 19:23:56 +00:00
Guido van Rossum b932420cc7 Rich comparisons:
- Use PyObject_RichCompareBool() when comparing keys; this makes the
  error handling cleaner.

- There were two implementations for dictionary comparison, an old one
  (#ifdef'ed out) and a new one.  Got rid of the old one, which was
  abandoned years ago.

- In the characterize() function, part of dictionary comparison, use
  PyObject_RichCompareBool() to compare keys and values instead.  But
  continue to use PyObject_Compare() for comparing the final
  (deciding) elements.

- Align the comments in the type struct initializer.

Note: I don't implement rich comparison for dictionaries -- there
doesn't seem to be much to be gained.  (The existing comparison
already decides that shorter dicts are always smaller than longer
dicts.)
2001-01-18 00:39:02 +00:00
Jeremy Hylton 1fb6088e86 dict_update has two boundary conditions: a.update(a) and a.update({})
Added test for second one.
2001-01-03 22:34:59 +00:00
Tim Peters f7f88b11e4 Add long-overdue docstrings to dict methods. 2000-12-13 23:18:45 +00:00
Tim Peters f1c7c884b3 Typo repair in comments. Fell for GregS's .popitem() poke. 2000-12-13 19:58:25 +00:00
Tim Peters ea8f2bf9ca Bring comments up to date (e.g., they still said the table had to be
a prime size, which is in fact never true anymore ...).
2000-12-13 01:02:46 +00:00
Guido van Rossum ba6ab84e73 Add popitem() -- SF patch #102733. 2000-12-12 22:02:18 +00:00
Moshe Zadka 5725d1eb03 Backing out my changes.
Improved version coming soon to a Source Forge near you!
2000-11-30 19:30:21 +00:00
Moshe Zadka 1a62750eda Added .first{item,value,key}() to dictionaries.
Complete with docos and tests.
OKed by Guido.
2000-11-30 12:31:03 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Fred Drake 1bff34ab96 Slight performance hack that also avoids requiring the existence of thread
state for dictionaries that have only been indexed by string keys.

See the comments in SourceForge for more.

This closes SourceForge patch #101309.
2000-08-31 19:31:38 +00:00
Fred Drake c88b99ce06 Clear errors raised by PyObject_Compare() without losing any existing
exception context.  This avoids improperly propogating errors raised by
a user-defined __cmp__() by a subsequent lookup operation.

This patch does *not* include the performance enhancement patch for
dictionaries with string keys only; that will be checked in separately.

This closes SourceForge patch #101277 and bug #112558.
2000-08-31 19:04:07 +00:00
Guido van Rossum 164452cec4 Barry's patch to implement the new setdefault() method. 2000-08-08 16:12:54 +00:00
Thomas Wouters 7889010731 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
char**) and return an int even on PC platforms. If not, please fix
PC/utils/makesrc.c ;-P
2000-07-22 19:25:51 +00:00
Thomas Wouters 7e47402264 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
comments, docstrings or error messages. I fixed two minor things in
test_winreg.py ("didn't" -> "Didn't" and "Didnt" -> "Didn't").

There is a minor style issue involved: Guido seems to have preferred English
grammar (behaviour, honour) in a couple places. This patch changes that to
American, which is the more prominent style in the source. I prefer English
myself, so if English is preferred, I'd be happy to supply a patch myself ;)
2000-07-16 12:04:32 +00:00
Tim Peters 1f5871e834 Removed Py_PROTO and switched to ANSI C declarations in the dict
implementation.  This was really to test whether my new CVS+SSH
setup is more usable than the old one -- and turns out it is (for
whatever reason, it was impossible to do a commit before that
involved more than one directory).
2000-07-04 17:44:48 +00:00
Guido van Rossum 4cc6ac7b87 Neil Schemenauer: small fixes for GC 2000-07-01 01:00:38 +00:00
Guido van Rossum ffcc3813d8 Change copyright notice - 2nd try. 2000-06-30 23:58:06 +00:00
Guido van Rossum fd71b9e9d4 Change copyright notice. 2000-06-30 23:50:40 +00:00
Jeremy Hylton c5007aa5c3 final patches from Neil Schemenauer for garbage collection 2000-06-30 05:02:53 +00:00
Jeremy Hylton d08b4c4524 part 2 of Neil Schemenauer's GC patches:
This patch modifies the type structures of objects that
participate in GC.  The object's tp_basicsize is increased when
GC is enabled.  GC information is prefixed to the object to
maintain binary compatibility.  GC objects also define the
tp_flag Py_TPFLAGS_GC.
2000-06-23 19:37:02 +00:00
Jeremy Hylton 8caad49c30 Round 1 of Neil Schemenauer's GC patches:
This patch adds the type methods traverse and clear necessary for GC
implementation.
2000-06-23 14:18:11 +00:00
Guido van Rossum b18618dab7 Vladimir Marangozov's long-awaited malloc restructuring.
For more comments, read the patches@python.org archives.
For documentation read the comments in mymalloc.h and objimpl.h.

(This is not exactly what Vladimir posted to the patches list; I've
made a few changes, and Vladimir sent me a fix in private email for a
problem that only occurs in debug mode.  I'm also holding back on his
change to main.c, which seems unnecessary to me.)
2000-05-03 23:44:39 +00:00
Jeremy Hylton a12c7a7620 Add PyDict_Copy() function to C API for dicts. It returns a new
dictionary that contains the same key/value pairs as p.
2000-03-30 22:27:31 +00:00
Guido van Rossum d724b23420 Christian Tismer's "trashcan" patch:
Added wrapping macros to dictobject.c, listobject.c, tupleobject.c,
frameobject.c, traceback.c that safely prevends core dumps
on stack overflow. Macros and functions in object.c, object.h.
The method is an "elevator destructor" that turns cascading
deletes into tail recursive behavior when some limit is hit.
2000-03-13 16:01:29 +00:00
Fred Drake 52fccfda5b dict_has_key(): Accept only one parameter. PR#210 reported by
Andreas Jung <ajung@sz-sb.de>.
2000-02-23 15:47:16 +00:00
Guido van Rossum 2bc137909d Vladimir Marangozov contributed updated comments. 1999-03-24 19:06:42 +00:00
Guido van Rossum f05fc716d1 Remove dead code discovered by Vladimir Marangozov. 1998-11-16 22:46:30 +00:00
Guido van Rossum c1c7b1a699 Slight rearrangement of code in lookdict() by Vladimir Marangozov, to
make a common case slightly faster.
1998-10-06 16:01:14 +00:00
Guido van Rossum 0fd00334c6 Avoid using calloc(). This triggered an obscure bug on multiprocessor
Sparc Solaris 2.6 (fully patched!) that I don't want to dig into, but
which I suspect is a bug in the multithreaded malloc library that only
shows up when run on a multiprocessor.  (The program wasn't using
threads, it was just using the multithreaded C library.)
1998-07-16 15:06:13 +00:00
Guido van Rossum 474b19e2ab Make sure that PyDict_GetItem[String]() *never* raises an exception.
If the argument is not a dictionary, simply return NULL.  If the
hash() on the key fails, clear the error.
1998-05-14 01:00:51 +00:00
Guido van Rossum 255443b720 Use Py_Repr{Enter,Leave} to display recursive dictionaries in finite space.
(Jeremy will hardly recognize his patch :-)
1998-04-10 22:47:14 +00:00
Guido van Rossum 6fcfa72c63 Correct Barry's fix -- take care of {}.get(0). 1997-10-20 20:10:00 +00:00
Barry Warsaw 320ac331d1 dict_get(): Fixed a couple of stupid mistakes which caused crashes.
Also got rid of some unnecessary code.
1997-10-20 17:26:25 +00:00
Barry Warsaw c38c5da5d0 dict_get(): New method for item access with different semantics than
__getitem__().  This method never raises an exception; if the key is
not in the dictionary, the second (optional) argument is returned.  If
the second argument is not provided and the key is missing, None is
returned.

mapp_methods: added "get" method.
1997-10-06 17:49:20 +00:00
Guido van Rossum 4f3bf1e383 Don't intern the key string for getitem and delitem. 1997-09-29 23:31:11 +00:00
Guido van Rossum fd7a0b871f Made lookdict nearly twice as fast, resulting in a 5% overall
improvement of pystone.  Vladimir Marangozov.
1997-08-18 21:52:47 +00:00
Guido van Rossum 5d8123f34a Reordered list of methods to hopefully put the most frequently used
ones near the front.
1997-07-13 03:58:01 +00:00
Guido van Rossum a8d5131d57 Renamed dict.absorb() (too spungy) to dict.update(). 1997-06-02 17:13:37 +00:00
Guido van Rossum e3f5b9c8d1 Added dict.absorb() and dict.copy(). 1997-05-28 19:15:28 +00:00
Guido van Rossum 5b2121b25f PyObject_Compare can now return an error. Unfortunately, there are a
few places where we don't know how to test for them without losing
speed; don't know yet how to handle that.
1997-05-23 00:01:41 +00:00
Guido van Rossum 037b2205e8 Moved PyObject_{Get,Set}Attr to object.c.
Fixed two 'return NULL' that should be 'return -1'.
1997-05-20 18:35:19 +00:00
Guido van Rossum 3cca24570e Got rid of all the last_name_* bogosities. I don't think the
complexity saved much any more.  A simple benchmark (grail) showed
that there were 3 times as many misses as hits, and the same number of
times again the code was bypassed altogether due to the existence of
setattro/getattro.
1997-05-16 14:23:33 +00:00
Guido van Rossum a9e7a81137 Renamed from mappingobject.c to dictobject.c.
(Sorry Jack, all your projects will have to be changed again. :-( )
1997-05-13 21:02:11 +00:00
Guido van Rossum c0b618a2cc Quickly renamed the last directory. 1997-05-02 03:12:38 +00:00
Guido van Rossum 3648884490 (Jack:) Align mapping entries to 4-words if USE_CACHE_ALIGNED is defined. 1997-04-11 19:14:07 +00:00
Guido van Rossum 2095d24842 Tweaks to keep the Microsoft compiler quiet. 1997-04-09 19:41:24 +00:00
Guido van Rossum fb8f1cadb2 Add clear() method to dictionary objects. 1997-03-21 21:55:12 +00:00
Guido van Rossum efb4609c4a Small lookmapping nits:
- remove bogus initialization using uninitialized i
- derive initial incr from hash
- copy mp->ma_table into a local variable
1997-01-29 15:53:56 +00:00
Guido van Rossum 9e5656ca3f Final three poly table entries corrected by Tim Peters.
Reindented the whole table.
1997-01-29 04:45:16 +00:00
Guido van Rossum 16e93a8d59 Changed the lookup algorithm again, based on Reimer Behrends's post.
The table size is now constrained to be a power of two, and we use a
variable increment based on GF(2^n)-{0} (not that I have the faintest
idea what that is :-) which helps avoid the expensive '%' operation.

Some of the entries in the table of polynomials have been modified
according to a post by Tim Peters.
1997-01-28 00:00:11 +00:00
Guido van Rossum ca756f2a1d Forget keeping track of whether a dictionary contains all interned
string keys.  Just doing a pointer compare before the string compare
(in fact before the hash compare!) is just as fast.
1997-01-23 19:39:29 +00:00
Guido van Rossum 2a61e7428d String interning. 1997-01-18 07:55:05 +00:00
Guido van Rossum 7d18159614 Rewrote lookmapping() according to suggestions by Jyrki Alakuijala. 1997-01-16 21:06:45 +00:00
Guido van Rossum a0a69b8b42 Experimental new implementation of dictionary comparison. This
defines that a shorter dictionary is always smaller than a longer one.
For dictionaries of the same size, the smallest differing element
determines the outcome (which yields the same results as before,
without explicit sorting).
1996-12-05 21:55:55 +00:00
Guido van Rossum d266eb460e New permission notice, includes CNRI. 1996-10-25 14:44:06 +00:00
Guido van Rossum d8eb1b340f Support for tp_getattro, tp_setattro (Sjoerd) 1996-08-09 20:52:03 +00:00
Guido van Rossum 310968dc06 Speedup suggested by Sjoerd 1996-07-30 16:45:31 +00:00
Guido van Rossum 992ded8f12 fix free memory reads in dictlookup et al 1995-12-08 01:16:31 +00:00
Guido van Rossum 5fe605889a a few peephole optimizations 1995-03-09 12:12:50 +00:00
Guido van Rossum 6610ad9d6b Added 1995 to copyright message.
floatobject.c: fix hash().
methodobject.c: support METH_FREENAME flag bit.
1995-01-04 19:07:38 +00:00
Guido van Rossum efc8713428 * Objects/mappingobject.c (mappingremove): don't call
lookmapping() for empty dictionary
1995-01-02 19:42:39 +00:00
Guido van Rossum d7047b395e Lots of minor changes. Note for mappingobject.c: the hash table pointer
can now be NULL.
1995-01-02 19:07:15 +00:00
Guido van Rossum 1d5735e846 Merge back to main trunk 1994-08-30 08:27:36 +00:00
Guido van Rossum 8732d6aeea Fix lay-out of previous fix. 1993-11-23 17:54:03 +00:00
Guido van Rossum b376a4ad18 * timemodule.c: Add hack for Solaris 2.
* posixmodule.c: don't prototype getcwd() -- it's not portable...
* mappingobject.c: double-check validity of last_name_char in
  dict{lookup,insert,remove}.
* arraymodule.c: need memmove only for non-STDC Suns.
* Makefile: comment out HTML_LIBS and XT_USE by default
* pythonmain.c: don't prototype getopt() -- it's not standardized
* socketmodule.c: cast flags arg to {get,set}sockopt() and addrbuf arg to
  recvfrom() to (ANY*).
* pythonrun.c (initsigs): fix prototype, make it static
* intobject.c (LONG_BIT): only #define it if not already defined
* classobject.[ch]: remove all references to unused instance_convert()
* mappingobject.c (getmappingsize): Don't return NULL in int function.
1993-11-23 17:53:17 +00:00
Guido van Rossum 52f2c05401 * parsermodule.c, Makefile, config.c: rudimentary interface to the Python
parser.
* mappingobject.c (lookmapping): 'freeslot' was never used due to a bug in
  the code.
1993-11-10 12:53:24 +00:00
Guido van Rossum a3d78fb268 * posixmodule.c: added set{uid,gid}.
* {tuple,list,mapping,array}object.c: call printobject with 0 for flags
* compile.c (parsestr): use quote instead of '\'' at one crucial point
* arraymodule.c (array_getattr): Added __members__ attribute
1993-11-10 09:23:53 +00:00
Guido van Rossum 4199facacd Added getmappingsize(). (Needed by previous checkin of posixmodule.c) 1993-11-05 10:18:44 +00:00
Sjoerd Mullender 3bb8a05947 Several optimizations and speed improvements.
cstubs: Use Matrix type instead of float[4][4].
1993-10-22 12:04:32 +00:00
Guido van Rossum 1fc238a813 Minor fixes / changes for Mac compatibility. 1993-07-29 08:25:09 +00:00
Guido van Rossum 25831652fd Several changes in one:
(1) dictionaries/mappings now have attributes values() and items() as
well as keys(); at the C level, use the new function mappinggetnext()
to iterate over a dictionary.

(2) "class C(): ..." is now illegal; you must write "class C: ...".

(3) Class objects now know their own name (finally!); and minor
improvements to the way how classes, functions and methods are
represented as strings.

(4) Added an "access" statement and semantics.  (This is still
experimental -- as long as you don't use the keyword 'access' nothing
should be changed.)
1993-05-19 14:50:45 +00:00
Guido van Rossum 9bfef44d97 * Changed all copyright messages to include 1993.
* Stubs for faster implementation of local variables (not yet finished)
* Added function name to code object.  Print it for code and function
  objects.  THIS MAKES THE .PYC FILE FORMAT INCOMPATIBLE (the version
  number has changed accordingly)
* Print address of self for built-in methods
* New internal functions getattro and setattro (getattr/setattr with
  string object arg)
* Replaced "dictobject" with more powerful "mappingobject"
* New per-type functio tp_hash to implement arbitrary object hashing,
  and hashobject() to interface to it
* Added built-in functions hash(v) and hasattr(v, 'name')
* classobject: made some functions static that accidentally weren't;
  added __hash__ special instance method to implement hash()
* Added proper comparison for built-in methods and functions
1993-03-29 10:43:31 +00:00
Guido van Rossum 4b1302bd1d Generalized version of dictionaries, with compatibility hacks. 1993-03-27 18:11:32 +00:00