The real benefit of the unicode specialized function comes from
bypassing the overhead of PyObject_RichCompareBool() and not
from being in-lined (especially since there was almost no shared
data between the caller and callee). Also, the in-lining was
having a negative effect on code generation for the callee.
Simplifies the code a little bit and does the resize check
only when a new key is added (giving a small speed up in
the case where the key already exists).
Fixes possible bug in set_merge() where the set_insert_key()
call relies on a big resize at the start to make enough room
for the keys but is vulnerable to a comparision callback that
could cause the table to shrink in the middle of the merge.
Also, changed the resize threshold from two-thirds of the
mask+1 to just two-thirds. The plus one offset gave no
real benefit (afterall, the two-thirds mark is just a
heuristic and isn't a precise cut-off).
Elsewhere in the setobject.c code we do a bitwise-and with the mask
instead of using a conditional to reset to zero on wrap-around.
Using that same technique here use gives cleaner, faster, and more
consistent code.
* Move the test for an exact key match to after a hash match
* Use "used" as a loop counter instead of "fill"
* Minor improvements to variable names and code consistency
The setobject freelist was consuming memory but not providing much value.
Even when a freelisted setobject was available, most of the setobject
fields still needed to be initialized and the small table still required
a memset(). This meant that the custom freelisting scheme for sets was
providing almost no incremental benefit over the default Python freelist
scheme used by _PyObject_Malloc() in Objects/obmalloc.c.
Modern processors tend to make consecutive memory accesses cheaper than
random probes into memory.
Small sets can fit into L1 cache, so they get less benefit. But they do
come out ahead because the consecutive probes don't probe the same key
more than once and because the randomization step occurs less frequently
(or not at all).
For the open addressing step, putting the perturb shift before the index
calculation gets the upper bits into play sooner.
The Gdb prettyprint plugin depended on the dummy object being displayable.
Other solutions besides a unicode object are possible. For now, get it
back up and running.
The identity checks in lookkey() need to be there to prevent the dummy
object from leaking through Py_RichCompareBool() into user code in the
rare circumstance where the dummy's hash value exactly matches the hash
value of the actual key being looked up.
Letting the compiler decide how to optimize the multiply by five
gives it the freedom to make better choices for the best technique
for a given target machine.
For example, GCC on x86_64 produces a little bit better code:
Old-way (3 steps with a data dependency between each step):
shrq $5, %r13
leaq 1(%rbx,%r13), %rax
leaq (%rax,%rbx,4), %rbx
New-way (3 steps with no dependency between the first two steps
which can be run in parallel):
leaq (%rbx,%rbx,4), %rax # i*5
shrq $5, %r13 # perturb >>= PERTURB_SHIFT
leaq 1(%r13,%rax), %rbx # 1 + perturb + i*5
computation as the overflow behavior of signed integers is undefined.
NOTE: This change is smaller compared to 3.2 as much of this cleanup had
already been done. I added the comment that my change in 3.2 added so that the
code would match up. Otherwise this just adds or synchronizes appropriate UL
designations on some constants to be pedantic.
In practice we require compiling everything with -fwrapv which forces overflow
to be defined as twos compliment but this keeps the code cleaner for checkers
or in the case where someone has compiled it without -fwrapv or their
compiler's equivalent.
Found by Clang trunk's Undefined Behavior Sanitizer (UBSan).
Cleanup only - no functionality or hash values change.
computation as the overflow behavior of signed integers is undefined.
In practice we require compiling everything with -fwrapv which forces overflow
to be defined as twos compliment but this keeps the code cleaner for checkers
or in the case where someone has compiled it without -fwrapv or their
compiler's equivalent.
Found by Clang trunk's Undefined Behavior Sanitizer (UBSan).
Cleanup only - no functionality or hash values change.
The fix was already committed to 3.2, but I merged two small changes
recommended by Raymond while I was working on the 2.7 patch to ease
future merges.
svn+ssh://pythondev@svn.python.org/python/branches/py3k
................
r78541 | ezio.melotti | 2010-03-01 06:08:34 +0200 (Mon, 01 Mar 2010) | 17 lines
Merged revisions 78515-78516,78522 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r78515 | georg.brandl | 2010-02-28 20:19:17 +0200 (Sun, 28 Feb 2010) | 1 line
#8030: make builtin type docstrings more consistent: use "iterable" instead of "seq(uence)", use "new" to show that set() always returns a new object.
........
r78516 | georg.brandl | 2010-02-28 20:26:37 +0200 (Sun, 28 Feb 2010) | 1 line
The set types can also be called without arguments.
........
r78522 | ezio.melotti | 2010-03-01 01:59:00 +0200 (Mon, 01 Mar 2010) | 1 line
#8030: more docstring fix for builtin types.
........
................
svn+ssh://pythondev@svn.python.org/python/trunk
........
r78515 | georg.brandl | 2010-02-28 20:19:17 +0200 (Sun, 28 Feb 2010) | 1 line
#8030: make builtin type docstrings more consistent: use "iterable" instead of "seq(uence)", use "new" to show that set() always returns a new object.
........
r78516 | georg.brandl | 2010-02-28 20:26:37 +0200 (Sun, 28 Feb 2010) | 1 line
The set types can also be called without arguments.
........
r78522 | ezio.melotti | 2010-03-01 01:59:00 +0200 (Mon, 01 Mar 2010) | 1 line
#8030: more docstring fix for builtin types.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r68128 | antoine.pitrou | 2009-01-01 15:11:22 +0100 (jeu., 01 janv. 2009) | 3 lines
Issue #3680: Reference cycles created through a dict, set or deque iterator did not get collected.
........
PyUnicode_AsStringAndSize -> _PyUnicode_AsStringAndSize to mark
them for interpreter internal use only.
We'll have to rework these APIs or create new ones for the
purpose of accessing the UTF-8 representation of Unicode objects
for 3.1.
svn+ssh://pythondev@svn.python.org/python/trunk
........
r64089 | armin.ronacher | 2008-06-10 22:37:02 +0200 (mar., 10 juin 2008) | 3 lines
Fix a formatting error in the ast documentation.
........
r64098 | raymond.hettinger | 2008-06-11 02:25:29 +0200 (mer., 11 juin 2008) | 6 lines
Mini-PEP: Simplifying numbers.py
* Convert binary methods in Integral to mixin methods
* Remove three-arg __pow__ as a required method
* Make __int__ the root method instead of __long__.
........
r64100 | raymond.hettinger | 2008-06-11 02:28:51 +0200 (mer., 11 juin 2008) | 1 line
Update numbers doc for the Integral simplification.
........
r64101 | raymond.hettinger | 2008-06-11 02:44:47 +0200 (mer., 11 juin 2008) | 3 lines
Handle the case with zero arguments.
........
r64102 | benjamin.peterson | 2008-06-11 03:31:28 +0200 (mer., 11 juin 2008) | 4 lines
convert test_struct to a unittest thanks to Giampaolo Rodola
I had to disable one test because it was functioning incorrectly, see #1530559
I also removed the debugging prints
........
r64113 | thomas.heller | 2008-06-11 09:10:43 +0200 (mer., 11 juin 2008) | 2 lines
Fix markup.
Document the new 'offset' parameter for the 'ctypes.byref' function.
........
r64115 | raymond.hettinger | 2008-06-11 12:30:54 +0200 (mer., 11 juin 2008) | 1 line
Multi-arg form for set.difference() and set.difference_update().
........
r64116 | raymond.hettinger | 2008-06-11 14:06:49 +0200 (mer., 11 juin 2008) | 1 line
Issue 3051: Let heapq work with either __lt__ or __le__.
........
r64118 | raymond.hettinger | 2008-06-11 14:39:09 +0200 (mer., 11 juin 2008) | 1 line
Optimize previous checkin for heapq.
........
r64120 | raymond.hettinger | 2008-06-11 15:14:50 +0200 (mer., 11 juin 2008) | 1 line
Add test for heapq using both __lt__ and __le__.
........
r64132 | gregory.p.smith | 2008-06-11 20:00:52 +0200 (mer., 11 juin 2008) | 3 lines
Correct an incorrect comment about our #include of stddef.h.
(see Doug Evans' comment on python-dev 2008-06-10)
........
r64342 | guido.van.rossum | 2008-06-17 19:38:02 +0200 (mar., 17 juin 2008) | 3 lines
Roll back Raymond's -r64098 while we think of something better.
(See issue 3056 -- we're close to a resolution but need unittests.)
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r60568 | christian.heimes | 2008-02-04 19:48:38 +0100 (Mon, 04 Feb 2008) | 1 line
Increase debugging to investige failing tests on some build bots
........
r60570 | christian.heimes | 2008-02-04 20:30:05 +0100 (Mon, 04 Feb 2008) | 1 line
Small adjustments for test compact freelist test. It's no passing on Windows as well.
........
r60573 | amaury.forgeotdarc | 2008-02-04 21:53:14 +0100 (Mon, 04 Feb 2008) | 2 lines
Correct quotes in NEWS file
........
r60575 | amaury.forgeotdarc | 2008-02-04 22:45:05 +0100 (Mon, 04 Feb 2008) | 13 lines
#1750076: Debugger did not step on every iteration of a while statement.
The mapping between bytecode offsets and source lines (lnotab) did not contain
an entry for the beginning of the loop.
Now it does, and the lnotab can be a bit larger:
in particular, several statements on the same line generate several entries.
However, this does not bother the settrace function, which will trigger only
one 'line' event.
The lnotab seems to be exactly the same as with python2.4.
........
r60584 | amaury.forgeotdarc | 2008-02-05 01:26:21 +0100 (Tue, 05 Feb 2008) | 3 lines
Change r60575 broke test_compile:
there is no need to emit co_lnotab item when both offsets are zeros.
........
r60587 | skip.montanaro | 2008-02-05 03:32:16 +0100 (Tue, 05 Feb 2008) | 1 line
sync with most recent version from python-mode sf project
........
r60588 | lars.gustaebel | 2008-02-05 12:51:40 +0100 (Tue, 05 Feb 2008) | 5 lines
Issue #2004: Use mode 0700 for temporary directories and default
permissions for missing directories.
(will backport to 2.5)
........
r60590 | georg.brandl | 2008-02-05 13:01:24 +0100 (Tue, 05 Feb 2008) | 2 lines
Convert external links to internal links. Fixes#2010.
........
r60592 | marc-andre.lemburg | 2008-02-05 15:50:40 +0100 (Tue, 05 Feb 2008) | 3 lines
Keep distutils Python 2.1 compatible (or even Python 2.4 in this case).
........
r60593 | andrew.kuchling | 2008-02-05 17:06:57 +0100 (Tue, 05 Feb 2008) | 5 lines
Update PEP URL.
(This code is duplicated between pydoc and DocXMLRPCServer; maybe it
should be refactored as a GHOP project.)
2.5.2 backport candidate.
........
r60596 | guido.van.rossum | 2008-02-05 18:32:15 +0100 (Tue, 05 Feb 2008) | 2 lines
In the experimental 'Scanner' feature, the group count was set wrong.
........
r60602 | facundo.batista | 2008-02-05 20:03:32 +0100 (Tue, 05 Feb 2008) | 3 lines
Issue 1951. Converts wave test cases to unittest.
........
r60603 | georg.brandl | 2008-02-05 20:07:10 +0100 (Tue, 05 Feb 2008) | 2 lines
Actually run the test.
........
r60604 | skip.montanaro | 2008-02-05 20:24:30 +0100 (Tue, 05 Feb 2008) | 2 lines
correct object name
........
r60605 | georg.brandl | 2008-02-05 20:58:17 +0100 (Tue, 05 Feb 2008) | 7 lines
* Use the same code to profile for test_profile and test_cprofile.
* Convert both to unittest.
* Use the same unit testing code.
* Include the expected output in both test files.
* Make it possible to regenerate the expected output by running
the file as a script with an '-r' argument.
........
r60613 | raymond.hettinger | 2008-02-06 02:49:00 +0100 (Wed, 06 Feb 2008) | 1 line
Sync-up with Py3k work.
........
r60614 | christian.heimes | 2008-02-06 13:44:34 +0100 (Wed, 06 Feb 2008) | 1 line
Limit free list of method and builtin function objects to 256 entries each.
........
r60616 | christian.heimes | 2008-02-06 14:33:44 +0100 (Wed, 06 Feb 2008) | 7 lines
Unified naming convention for free lists and their limits. All free lists
in Object/ are named ``free_list``, the counter ``numfree`` and the upper
limit is a macro ``PyName_MAXFREELIST`` inside an #ifndef block.
The chances should make it easier to adjust Python for platforms with
less memory, e.g. mobile phones.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r60553 | neal.norwitz | 2008-02-03 17:53:09 +0100 (Sun, 03 Feb 2008) | 1 line
Ignore leaky warnings from test_asynchat
........
r60555 | christian.heimes | 2008-02-03 20:51:13 +0100 (Sun, 03 Feb 2008) | 1 line
Another int -> pid_t case
........
r60560 | amaury.forgeotdarc | 2008-02-03 23:51:43 +0100 (Sun, 03 Feb 2008) | 6 lines
Ensure that PySet_Add() operates on a newly created frozenset, like PyTuple_SetItem does.
Add PyFrozenSet_Check(), which was not needed before; The list of Py*Set_Check* macros seems to be complete now.
Add missing NEWS entries about all this.
........
r60563 | amaury.forgeotdarc | 2008-02-04 00:14:32 +0100 (Mon, 04 Feb 2008) | 2 lines
Nasty typo in setobject.h
........
r60564 | amaury.forgeotdarc | 2008-02-04 00:15:32 +0100 (Mon, 04 Feb 2008) | 3 lines
Correct test_mailbox on win32: since the test sets a custom 'colon' attribute
to the main mailbox, copy it to secondary mailbox instances.
........
r60565 | amaury.forgeotdarc | 2008-02-04 00:57:24 +0100 (Mon, 04 Feb 2008) | 2 lines
Let test_socketserver pass on win32, which does not have AF_UNIX sockets.
........
r60566 | jeffrey.yasskin | 2008-02-04 02:04:35 +0100 (Mon, 04 Feb 2008) | 2 lines
Make int() and long() fall back to __trunc__(). See issue 2002.
........
r60567 | christian.heimes | 2008-02-04 19:00:12 +0100 (Mon, 04 Feb 2008) | 3 lines
Patch #1953
I implemented the function sys._compact_freelists() and C API functions PyInt_/PyFloat_CompactFreeList() to compact the pre-allocated blocks of ints and floats. They allow the user to reduce the memory usage of a Python process that deals with lots of numbers.
The patch also renames sys._cleartypecache to sys._clear_type_cache
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r60388 | thomas.heller | 2008-01-28 09:44:13 +0100 (Mon, 28 Jan 2008) | 1 line
Revert rev. 59925, it breaks comtypes (I need to further examine this).
........
r60397 | raymond.hettinger | 2008-01-28 21:34:33 +0100 (Mon, 28 Jan 2008) | 5 lines
Make PySet_Add() work with frozensets.
Works like PyTuple_SetItem() to build-up values in a brand new frozenset.
Also, PyFrozenSet_New() is now guaranteed to produce a distinct new frozenset.
........
r60398 | raymond.hettinger | 2008-01-28 22:34:30 +0100 (Mon, 28 Jan 2008) | 1 line
Let marshal built-up sets and frozensets one element at a time (without creating an intermediate tuple).
........
r60399 | raymond.hettinger | 2008-01-28 22:47:42 +0100 (Mon, 28 Jan 2008) | 1 line
Factor-out common code with a new macro
........
r60400 | raymond.hettinger | 2008-01-28 22:48:07 +0100 (Mon, 28 Jan 2008) | 1 line
Factor-out common code with a new macro
........
r60401 | raymond.hettinger | 2008-01-28 22:51:25 +0100 (Mon, 28 Jan 2008) | 1 line
Removed unnecessary conditional (spotted by Neal Norwitz).
........
r60403 | gregory.p.smith | 2008-01-29 00:21:00 +0100 (Tue, 29 Jan 2008) | 4 lines
Disable use of BerkeleyDB 4.6.x to see what the odd platform buildbots
think. In particular, neal norwitz has traced an Ubuntu sparc64 crash
to the Lib/test/bsddb/test_basics.py test when opening a db with DB_THREAD.
........
r60405 | brett.cannon | 2008-01-29 05:13:07 +0100 (Tue, 29 Jan 2008) | 2 lines
Fix the reindent rule to use $(BUILDPYTHON).
........
r60406 | brett.cannon | 2008-01-29 05:18:04 +0100 (Tue, 29 Jan 2008) | 3 lines
Update Vim syntax highlighting to specify what revision was used to generate
the file.
........
r60407 | brett.cannon | 2008-01-29 05:20:56 +0100 (Tue, 29 Jan 2008) | 2 lines
Ignore .pyc and .pyo files.
........
Raymond Hettinger
Yury Selivanov
Serhiy Storchaka
Victor Stinner
Eric V. Smith
Antoine Pitrou
Gregory P. Smith
Ezio Melotti
David Malcolm
Kristján Valur Jónsson
Benjamin Peterson
Petri Lehtinen
Martin v. Löwis
Mark Dickinson
Brian Curtin
Victor Stinner
Éric Araujo
Georg Brandl
Daniel Stutzbach
Mark Dickinson
Marc-André Lemburg
Nick Coghlan
Robert Schuppenies
Amaury Forgeot d'Arc
Christian Heimes