Commit Graph

487 Commits

Author SHA1 Message Date
Victor Stinner 9204fb8623
bpo-35081: Cleanup pystate.c and pystate.h (GH-10240)
* Remove _PyThreadState_Current
* Replace GET_TSTATE() with PyThreadState_GET()
* Replace GET_INTERP_STATE() with _PyInterpreterState_GET_UNSAFE()
* Replace direct access to _PyThreadState_Current with
  PyThreadState_GET()
* Replace _PyThreadState_Current with
  _PyRuntime.gilstate.tstate_current
* Rename SET_TSTATE() to _PyThreadState_SET(), name more
  consistent with _PyThreadState_GET()
* Update outdated comments
2018-10-30 15:13:17 +01:00
Victor Stinner 50fe3f8913
bpo-9263: _PyXXX_CheckConsistency() use _PyObject_ASSERT() (GH-10108)
Use _PyObject_ASSERT() in:

* _PyDict_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()

_PyObject_ASSERT() dumps the faulty object if the assertion fails
to help debugging.
2018-10-26 18:47:15 +02:00
Sergey Fedoseev 6395844e6a bpo-34573: Simplify __reduce__() of set and dict iterators. (GH-9050)
Simplify the pickling of set and dictionary objects iterators by consuming
the iterator into a list with PySequence_List.
2018-10-20 01:43:33 +01:00
INADA Naoki 2aaf98c16a bpo-34320: Fix dict(o) didn't copy order of dict subclass (GH-8624)
When dict subclass overrides order (`__iter__()`, `keys()`, and `items()`), `dict(o)`
should use it instead of dict ordering.


https://bugs.python.org/issue34320
2018-09-25 20:59:00 -07:00
Yury Selivanov 0b75228700
bpo-34042: Fix dict.copy() to maintain correct total refcount (GH-8119) 2018-07-06 12:20:07 -04:00
Siddhesh Poyarekar 55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
Ville Skyttä 61f82e0e33 Spelling fixes to docs, docstrings, and comments (GH-6374) 2018-04-20 16:08:45 -04:00
Gregory P. Smith 397f1b28c4
bpo-33312: Fix clang ubsan out of bounds warnings in dict. (GH-6537)
Fix clang ubsan (undefined behavior sanitizer) warnings in dictobject.c by
adjusting how the internal struct _dictkeysobject shared keys structure is
declared.

This remains ABI compatible.  We get rid of the union at the end of the
struct being used for conveinence to avoid typecasting in favor of char[]
variable length array at the end of a struct. This is known to clang to be
used for variable sized objects and will not cause an undefined behavior
problem.  Similarly, char arrays do not have strict aliasing undefined
behavior when cast.

PEP-007 does not currently list variable length arrays (VLAs) as allowed
in our subset of C99.  If this turns out to be a problem, the fix to this is
to change the char `dk_indices[]` into `dk_indices[1]` and restore the
three size computation subtractions this change removes:
  `- Py_MEMBER_SIZE(PyDictKeysObject, dk_indices)`

If this works as is I'll make a separate PR to update PEP-007.
2018-04-19 22:41:19 -07:00
INADA Naoki 5fbc511f56
bpo-33205: dict: Change GROWTH_RATE to `used*3` (GH-6350) 2018-04-17 15:53:34 +09:00
INADA Naoki d1c82c5cc7
bpo-33199: Initialize ma_version_tag in PyDict_Copy (GH-6341) 2018-04-03 11:43:53 +09:00
bennorth d7773d92bd bpo-18533: Avoid RecursionError from repr() of recursive dictview (#4823)
dictview_repr(): Use a Py_ReprEnter() / Py_ReprLeave() pair to check
for recursion, and produce "..." if so.

test_recursive_repr(): Check for the string rather than a
RecursionError.  (Test cannot be any tighter as contents are
implementation-dependent.)

test_deeply_nested_repr(): Add new test, replacing the original
test_recursive_repr().  It checks that a RecursionError is raised in
the case of a non-recursive but deeply nested structure.  (Very
similar to what test_repr_deep() in test/test_dict.py does for a
normal dict.)

OrderedDictTests: Add new test case, to test behavior on OrderedDict
instances containing their own values() or items().
2018-01-26 07:46:01 -08:00
Serhiy Storchaka f320be77ff bpo-32571: Avoid raising unneeded AttributeError and silencing it in C code (GH-5222)
Add two new private APIs: _PyObject_LookupAttr() and _PyObject_LookupAttrId()
2018-01-25 17:49:40 +09:00
Yury Selivanov b0a7a037b8
bpo-31179: Make dict.copy() up to 5.5 times faster. (#3067) 2018-01-22 11:54:41 -05:00
Serhiy Storchaka 60c3d3551a
bpo-31572: Get rid of _PyObject_HasAttrId() in dict and OrderedDict. (#3728)
Silence only AttributeError when get "key" and "items" attributes in
the constructor and the update() method of dict and OrderedDict .
2017-11-11 16:19:56 +02:00
luzpaz a5293b4ff2 Fix miscellaneous typos (#4275) 2017-11-05 15:37:50 +02:00
Barry Warsaw b2e5794870 bpo-31338 (#3374)
* Add Py_UNREACHABLE() as an alias to abort().
* Use Py_UNREACHABLE() instead of assert(0)
* Convert more unreachable code to use Py_UNREACHABLE()
* Document Py_UNREACHABLE() and a few other macros.
2017-09-14 18:13:16 -07:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
INADA Naoki a6296d34a4 bpo-31095: fix potential crash during GC (GH-2974) 2017-08-24 14:55:17 +09:00
INADA Naoki 778928b0c7 bpo-29304: Simplify dict lookup functions (GH-2407)
* remove hashpos parameter from lookdict functions.
* remove many duplicated code from lookdict functions.
2017-08-03 23:45:15 +09:00
Serhiy Storchaka 6969eaf468 bpo-29464: Rename METH_FASTCALL to METH_FASTCALL|METH_KEYWORDS and make (#1955)
the bare METH_FASTCALL be used for functions with positional-only
parameters.
2017-07-03 21:20:15 +03:00
INADA Naoki 870c286e27 bp-29304: Simplify dictobject.c (GH-2347)
replace `(i << 2) + 1` with `i*5`
2017-06-24 09:03:19 +09:00
INADA Naoki 073ae487b3 bpo-29304: simplify lookdict_index() function. (GH-2273) 2017-06-23 15:22:50 +09:00
Serhiy Storchaka 753bca3934 bpo-27945: Fixed various segfaults with dict. (#1657)
Based on patches by Duane Griffin and Tim Mitchell.
2017-05-20 12:30:02 +03:00
T. Wouters a00c3fd12d bpo-29941: Assert fixes (#886)
Make a non-Py_DEBUG, asserts-enabled build of CPython possible. This means
making sure helper functions are defined when NDEBUG is not defined, not
just when Py_DEBUG is defined.

Also fix a division-by-zero in obmalloc.c that went unnoticed because in Py_DEBUG mode, elsize is never zero.
2017-03-31 09:14:41 -07:00
Serhiy Storchaka c61ac1642d Don't use Py_SIZE for dict object. (#747) 2017-03-21 08:52:38 +02:00
INADA Naoki 1b8df107f8 bpo-24274: fix erroneous comment in dictobject.c (GH-196)
lookdict_unicode() and lookdict_unicode_nodummy() may raise exception
when key is not unicode.
2017-02-20 22:48:10 +09:00
INADA Naoki 2294f3aee1 bpo-29438: fixed use-after-free in key sharing dict (#17) 2017-02-12 13:51:30 +09:00
Serhiy Storchaka 19d2597ed5 Issue #29311: Regenerate Argument Clinic. 2017-02-04 08:05:07 +02:00
Serhiy Storchaka 78d9e58f20 Issues #29311, #29289: Fixed and improved docstrings for dict and OrderedDict
methods.
2017-01-25 00:30:04 +02:00
Serhiy Storchaka 48088ee9ba Issue #29311: Argument Clinic generates reasonable name for the parameter "default". 2017-01-19 19:00:30 +02:00
Victor Stinner 91f0d4aa2f Add a note explaining why dict_update() doesn't use METH_FASTCALL
Issue #29312.
2017-01-19 12:45:06 +01:00
Victor Stinner 7dc6a5f50a dict.get() and dict.setdefault() now use AC
Issue #29311: dict.get() and dict.setdefault() methods now use Argument Clinic
to parse arguments. Their calling convention changes from METH_VARARGS to
METH_FASTCALL which avoids the creation of a temporary tuple.

The signature of docstrings is also enhanced. For example,
    get(...)
becomes:
    get(self, key, default=None, /)
2017-01-19 12:37:13 +01:00
Serhiy Storchaka 617c7753ce Issue #28969: Fixed race condition in C implementation of functools.lru_cache.
KeyError could be raised when cached function with full cache was
simultaneously called from differen threads with the same uncached arguments.
2017-01-12 19:42:20 +02:00
Serhiy Storchaka 42e1ea9a10 Issue #28969: Fixed race condition in C implementation of functools.lru_cache.
KeyError could be raised when cached function with full cache was
simultaneously called from differen threads with the same uncached arguments.
2017-01-12 19:12:21 +02:00
Serhiy Storchaka 67796521dd Issue #28969: Fixed race condition in C implementation of functools.lru_cache.
KeyError could be raised when cached function with full cache was
simultaneously called from differen threads with the same uncached arguments.
2017-01-12 18:34:33 +02:00
Antoine Pitrou c06ae208eb Issue #28427: old keys should not remove new values from
WeakValueDictionary when collecting from another thread.
2016-12-27 14:34:54 +01:00
Antoine Pitrou d741ed492f Issue #28427: old keys should not remove new values from
WeakValueDictionary when collecting from another thread.
2016-12-27 14:23:43 +01:00
Antoine Pitrou e10ca3a0fe Issue #28427: old keys should not remove new values from
WeakValueDictionary when collecting from another thread.
2016-12-27 14:19:20 +01:00
INADA Naoki 6165d55f13 Issue #28147: Fix a memory leak in split-table dictionaries
setattr() must not convert combined table into split table.
2016-12-20 09:54:24 +09:00
Victor Stinner 5cc70c9935 Merge 3.6 2016-12-15 17:23:24 +01:00
Victor Stinner 3d3f264849 Fix a memory leak in split-table dictionaries
Issue #28147: Fix a memory leak in split-table dictionaries: setattr() must not
convert combined table into split table.

Patch written by INADA Naoki.
2016-12-15 17:21:23 +01:00
INADA Naoki ba6097734d Issue #28818: Simplify lookdict functions 2016-12-07 20:41:42 +09:00
Victor Stinner a5ed5f000a Use _PyObject_CallNoArg()
Replace:
    PyObject_CallObject(callable, NULL)
with:
    _PyObject_CallNoArg(callable)
2016-12-06 18:45:50 +01:00
Victor Stinner de4ae3d486 Backed out changeset b9c9691c72c5
Issue #28858: The change b9c9691c72c5 introduced a regression. It seems like
_PyObject_CallArg1() uses more stack memory than
PyObject_CallFunctionObjArgs().
2016-12-04 22:59:09 +01:00
Victor Stinner 27580c1fb5 Replace PyObject_CallFunctionObjArgs() with fastcall
* PyObject_CallFunctionObjArgs(func, NULL) => _PyObject_CallNoArg(func)
* PyObject_CallFunctionObjArgs(func, arg, NULL) => _PyObject_CallArg1(func, arg)

PyObject_CallFunctionObjArgs() allocates 40 bytes on the C stack and requires
extra work to "parse" C arguments to build a C array of PyObject*.

_PyObject_CallNoArg() and _PyObject_CallArg1() are simpler and don't allocate
memory on the C stack.

This change is part of the fastcall project. The change on listsort() is
related to the issue #23507.
2016-12-01 14:43:22 +01:00
INADA Naoki 2c5a830f2a Issue #28731: Optimize _PyDict_NewPresized() to create correct size dict.
Improve speed of dict literal with constant keys up to 30%.
2016-12-07 18:34:44 +09:00
INADA Naoki 92c50eee52 Issue #28731: Optimize _PyDict_NewPresized() to create correct size dict
Improve speed of dict literal with constant keys up to 30%.
2016-11-22 00:57:02 +09:00
Victor Stinner c7a8f67411 Issue #28618: Mark dict lookup functions as hot
It's common to see these functions in the top 3 of "perf report".
2016-11-15 15:13:40 +01:00
Victor Stinner 0cae609847 Use PyThreadState_GET() in performance critical code
It seems like _PyThreadState_UncheckedGet() is not inlined as expected, even
when using gcc -O3.
2016-11-11 01:43:56 +01:00
Serhiy Storchaka f0b311bd73 Issue #28123: _PyDict_GetItem_KnownHash() now can raise an exception as
PyDict_GetItemWithError().  Patch by Xiang Zhang.
2016-11-06 13:18:24 +02:00
INADA Naoki 93f26f794d Issue #28583: PyDict_SetDefault didn't combine split table when needed.
Patch by Xiang Zhang.
2016-11-02 18:45:16 +09:00
Serhiy Storchaka 7f0514ad54 Backed out changeset 6b88dfc7b25d 2016-10-31 20:14:05 +02:00
Serhiy Storchaka e26e20db95 Issue #28199: Microoptimized dict resizing. Based on patch by Naoki Inada. 2016-10-29 10:50:00 +03:00
Serhiy Storchaka 04230c4087 Issue #28123: _PyDict_GetItem_KnownHash() now can raise an exception as
PyDict_GetItemWithError().  Patch by Xiang Zhang.
2016-11-06 13:19:38 +02:00
INADA Naoki ca2d8be4ba Issue #28580: Optimize iterating split table values.
Patch by Xiang Zhang.
2016-11-04 16:59:10 +09:00
INADA Naoki a3498c7569 Issue #28583: PyDict_SetDefault didn't combine split table when needed.
Patch by Xiang Zhang.
2016-11-02 18:47:24 +09:00
Serhiy Storchaka 5e325d9c41 Merge from 3.6. 2016-10-31 20:15:48 +02:00
Serhiy Storchaka 041794908b Issue #28199: Microoptimized dict resizing. Based on patch by Naoki Inada. 2016-10-29 10:50:00 +03:00
Serhiy Storchaka d76d8bfee1 Issue #28199: Microoptimized dict resizing. Based on patch by Naoki Inada. 2016-10-29 10:49:43 +03:00
Yury Selivanov f0bbee6228 Merge 3.6 (issue #28544) 2016-10-28 19:01:46 -04:00
Yury Selivanov 684ef2c888 Issue #28544: Pass `PyObject*` to _PyDict_Pop, not `PyDictObject*` 2016-10-28 19:01:21 -04:00
INADA Naoki b574e77122 Issue #28509: dict.update() no longer allocate unnecessary large memory 2016-10-27 19:30:10 +09:00
INADA Naoki b1152be2de Issue #28509: dict.update() no longer allocate unnecessary large memory 2016-10-27 19:26:50 +09:00
Raymond Hettinger b12785d456 Reference the original compact-and-ordered proposal 2016-10-22 09:58:14 -07:00
doko@ubuntu.com de69ee71e2 - dictobject.c: Make dict_merge symbol a static symbol 2016-10-11 08:04:02 +02:00
doko@ubuntu.com c96df68ea3 - dictobject.c: Make dict_merge symbol a static symbol 2016-10-11 08:04:02 +02:00
Serhiy Storchaka 49f5cdde1a Issue #28183: Optimize and cleanup dict iteration. 2016-10-09 23:08:05 +03:00
INADA Naoki 267941c675 Issue #28201: Dict reduces possibility of 2nd conflict in hash table.
Do perturb shift after first conflict.
2016-10-06 15:19:07 +09:00
Serhiy Storchaka e036ef8fa2 Issue #27358: Optimized merging var-keyword arguments and improved error
message when pass a non-mapping as a var-keyword argument.
2016-10-02 11:06:43 +03:00
Serhiy Storchaka 97932e4c4f issue #28144: Decrease empty_keys_struct's dk_refcnt
since there is no dummy_struct any more.
Patch by Xiang Zhang.
2016-09-26 23:01:23 +03:00
Serhiy Storchaka 46825d2399 Issue #28194: Clean up some checks in dict implementation.
Patch by Xiang Zhang.
2016-09-26 21:29:34 +03:00
Raymond Hettinger 6692f01c91 merge 2016-09-18 21:46:08 -07:00
Raymond Hettinger 7eb1becc25 Issue #28189: dictitems_contains no longer swallows compare errors.
(Patch by Xiang Zhang)
2016-09-18 21:45:11 -07:00
Victor Stinner 611b0fa94c Add _PyDict_CheckConsistency()
Issue #28127: Add a function to check that a dictionary remains consistent
after any change.

By default, tables are not checked, only basic attributes. Define DEBUG_PYDICT
(ex: gcc -D DEBUG_PYDICT) to also check dictionary "content".
2016-09-14 15:02:01 +02:00
Victor Stinner d0ad11f6b4 Fix _PyDict_Pop() on pending key
Issue #28120: Fix dict.pop() for splitted dictionary when trying to remove a
"pending key" (Not yet inserted in split-table).

Patch by Xiang Zhang.
2016-09-13 16:56:38 +02:00
Victor Stinner 9926480b6a Issue #28040: Cleanup find_empty_slot()
find_empty_slot() only supports combined dict
2016-09-13 09:38:29 +02:00
Victor Stinner 3c336c5915 Issue #28077: find_empty_slot() only supports combined dict 2016-09-12 14:17:40 +02:00
Serhiy Storchaka 473e0e4dba Fixed compiler warnings in compact dict implementation on 32-bit platforms. 2016-09-10 21:34:43 +03:00
Victor Stinner 990397ef1e dictobject.c: explain why stringlib is used 2016-09-09 20:22:59 -07:00
Victor Stinner 78601a38c2 Fix SystemError in compact dict
Issue #28040: Fix _PyDict_DelItem_KnownHash() and _PyDict_Pop(): convert
splitted table to combined table to be able to delete the item.

Write an unit test for the issue.

Patch by INADA Naoki.
2016-09-09 19:28:36 -07:00
Berker Peksag 71c01d43a7 Issue #28033: Fix typo in dictobject.c
Patch by Wesley Emeneker.
2016-09-09 03:57:23 +03:00
Benjamin Peterson 3c569295d6 do not worry about 64-bit dict sizes on 32-bit platforms 2016-09-08 13:16:41 -07:00
Victor Stinner 3b6a6b4215 Add a new private version to the builtin dict type
Issue #26058: Add a new private version to the builtin dict type, incremented
at each dictionary creation and at each dictionary change.

Implementation of the PEP 509.
2016-09-08 12:51:24 -07:00
Benjamin Peterson 186122ead2 access dk_indices through a union 2016-09-08 12:20:12 -07:00
Victor Stinner a4348cc1be Add documentation to the dict implementation
Issue #27350.
2016-09-08 12:01:25 -07:00
Victor Stinner 58f7c5a955 Reindeint DK_xxx macros
Issue #27350.
2016-09-08 11:37:36 -07:00
Victor Stinner 208857ec33 dk_get_index/dk_set_index uses a type indices variable
Issue #27350.
2016-09-08 11:35:46 -07:00
Victor Stinner dee6e252cc Split lookdict_unicode_nodummy() assertion to debug
Issue #27350.
2016-09-08 11:16:07 -07:00
Victor Stinner 71211e3386 Add assertions to dk_set_index()
Issue #27350.
2016-09-08 10:52:46 -07:00
Benjamin Peterson 003f059b3a link to canonical blogspot 2016-09-08 10:14:31 -07:00
Benjamin Peterson 7322225a57 use native inline instead of Py_LOCAL_INLINE 2016-09-08 09:58:47 -07:00
Benjamin Peterson f0acae2d3c rearrange to make gcc happy 2016-09-08 09:50:08 -07:00
Victor Stinner 98ee9d5b73 Add Py_MEMBER_SIZE macro
Issue #27350: use Py_MEMBER_SIZE() macro to get the size of
PyDictKeyEntry.dk_indices, rather than hardcoding 8.
2016-09-08 09:33:56 -07:00
Victor Stinner 742da040db Implement compact dict
Issue #27350: `dict` implementation is changed like PyPy. It is more compact
and preserves insertion order.

_PyDict_Dummy() function has been removed.

Disable test_gdb: python-gdb.py is not updated yet to the new structure of
compact dictionaries (issue #28023).

Patch written by INADA Naoki.
2016-09-07 17:40:12 -07:00
Benjamin Peterson 55f445282d remove memory indirections in dict_traverse (closes #27956) 2016-09-05 12:12:59 -07:00
R David Murray ce85acff3a Merge: #20647: Update dictobject.c comments to account for randomized string hashes. 2016-07-10 12:40:03 -04:00
R David Murray 537ad7ad9f #20647: Update dictobject.c comments to account for randomized string hashes.
Patch by Jaysinh Shukla.
2016-07-10 12:33:18 -04:00
Serhiy Storchaka ab479c49d3 Issue #26494: Fixed crash on iterating exhausting iterators.
Affected classes are generic sequence iterators, iterators of str, bytes,
bytearray, list, tuple, set, frozenset, dict, OrderedDict, corresponding
views and os.scandir() iterator.
2016-03-30 20:41:15 +03:00
Serhiy Storchaka fbb1c5ee06 Issue #26494: Fixed crash on iterating exhausting iterators.
Affected classes are generic sequence iterators, iterators of str, bytes,
bytearray, list, tuple, set, frozenset, dict, OrderedDict, corresponding
views and os.scandir() iterator.
2016-03-30 20:40:02 +03:00
Benjamin Peterson 0f04bc7959 merge 3.5 (closes #26478) 2016-03-03 22:10:52 -08:00