Commit Graph

422 Commits

Author SHA1 Message Date
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Victor Stinner a15e260b70
bpo-40170: Add _PyIndex_Check() internal function (GH-19426)
Add _PyIndex_Check() function to the internal C API: fast inlined
verson of PyIndex_Check().

Add Include/internal/pycore_abstract.h header file.

Replace PyIndex_Check() with _PyIndex_Check() in C files of Objects
and Python subdirectories.
2020-04-08 02:01:56 +02:00
Guido van Rossum 48b069a003
bpo-39481: Implementation for PEP 585 (#18239)
This implements things like `list[int]`,
which returns an object of type `types.GenericAlias`.
This object mostly acts as a proxy for `list`,
but has attributes `__origin__` and `__args__`
that allow recovering the parts (with values `list` and `(int,)`.

There is also an approximate notion of type variables;
e.g. `list[T]` has a `__parameters__` attribute equal to `(T,)`.
Type variables are objects of type `typing.TypeVar`.
2020-04-07 09:50:06 -07:00
Petr Viktorin ce105541f8
bpo-37207: Use vectorcall for list() (GH-18928)
Speed up calls to list() by using the PEP 590 vectorcall
calling convention. Patch by Mark Shannon.

Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2020-03-30 14:16:16 +02:00
Serhiy Storchaka 2fe815edd6
bpo-38373: Change list overallocating strategy. (GH-18952)
* Add padding to make the allocated size multiple of 4.
* Do not overallocate if the new size is closer to overalocated size
  than to the old size.
2020-03-17 23:46:00 +02:00
Andy Lester dffe4c0709
bpo-39573: Finish converting to new Py_IS_TYPE() macro (GH-18601) 2020-03-04 14:15:20 +01:00
sweeneyde be7ead62db
bpo-39737: Remove code repitition in list_richcompare (GH-18638)
I may speed up list comparison on some platforms.
2020-02-26 09:00:35 +02:00
Petr Viktorin ffd9753a94
bpo-39245: Switch to public API for Vectorcall (GH-18460)
The bulk of this patch was generated automatically with:

    for name in \
        PyObject_Vectorcall \
        Py_TPFLAGS_HAVE_VECTORCALL \
        PyObject_VectorcallMethod \
        PyVectorcall_Function \
        PyObject_CallOneArg \
        PyObject_CallMethodNoArgs \
        PyObject_CallMethodOneArg \
    ;
    do
        echo $name
        git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
    done

    old=_PyObject_FastCallDict
    new=PyObject_VectorcallDict
    git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"

and then cleaned up:

- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
2020-02-11 17:46:57 +01:00
Victor Stinner 60ac6ed557
bpo-39573: Use Py_SET_SIZE() function (GH-18402)
Replace direct acccess to PyVarObject.ob_size with usage of
the Py_SET_SIZE() function.
2020-02-07 23:18:08 +01:00
Victor Stinner 58ac700fb0
bpo-39573: Use Py_TYPE() macro in Objects directory (GH-18392)
Replace direct access to PyObject.ob_type with Py_TYPE().
2020-02-07 03:04:21 +01:00
Victor Stinner c6e5c1123b
bpo-39489: Remove COUNT_ALLOCS special build (GH-18259)
Remove:

* COUNT_ALLOCS macro
* sys.getcounts() function
* SHOW_ALLOC_COUNT code in listobject.c
* SHOW_TRACK_COUNT code in tupleobject.c
* PyConfig.show_alloc_count field
* -X showalloccount command line option
* @test.support.requires_type_collecting decorator
2020-02-03 15:17:15 +01:00
Dong-hee Na 9e1ed518a5 bpo-39453: Add testcase for bpo-39453 (GH-18202)
https://bugs.python.org/issue39453



Automerge-Triggered-By: @pablogsal

Automerge-Triggered-By: @pablogsal
2020-01-27 09:04:25 -08:00
Dong-hee Na 4dbf2d8c67 bpo-39453: Make list.__contains__ hold strong references to avoid crashes (GH-18181) 2020-01-27 15:02:23 +00:00
Dong-hee Na 14d80d0b60 bpo-39425: Fix list.count performance regression (GH-18119)
https://bugs.python.org/issue39425



Automerge-Triggered-By: @pablogsal
2020-01-22 09:36:54 -08:00
Inada Naoki dfef986f12
bpo-38588: Optimize list comparison. (GH-17766)
Mitigate performance regression of the list comparison caused by 2d5bf56.
2019-12-31 10:58:37 +09:00
Dong-hee Na 2d5bf568ea bpo-38588: Fix possible crashes in dict and list when calling PyObject_RichCompareBool (GH-17734)
Take strong references before calling PyObject_RichCompareBool to protect against the case
where the object dies during the call.
2019-12-31 01:04:22 +00:00
Zackery Spytz d9e561d23d bpo-38610: Fix possible crashes in several list methods (GH-17022)
Hold strong references to list elements while calling PyObject_RichCompareBool().
2019-12-30 19:32:58 +00:00
HongWeipeng 3c87a667bb bpo-36946:Fix possible signed integer overflow when handling slices. (GH-15639)
This is a complement to PR 13375.
2019-09-08 13:15:56 +03:00
Victor Stinner bed4817d52
Make PyXXX_Fini() functions private (GH-15531)
For example, rename PyTuple_Fini() to _PyTuple_Fini().

These functions are only declared in the internal C API.
2019-08-27 00:12:32 +02:00
Serhiy Storchaka 18b711c5a7
bpo-37648: Fixed minor inconsistency in some __contains__. (GH-14904)
The collection's item is now always at the left and
the needle is on the right of ==.
2019-08-04 14:12:48 +03:00
Jeroen Demeyer 196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Tim Hoffmann 5c22476c01 Improve docstring of list.sort (GH-8516) 2019-05-31 21:10:02 -07:00
Jeroen Demeyer 530f506ac9 bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async (GH-13464)
Automatically replace
tp_print -> tp_vectorcall_offset
tp_compare -> tp_as_async
tp_reserved -> tp_as_async
2019-05-30 19:13:39 -07:00
Victor Stinner 331a6a56e9
bpo-36763: Implement the PEP 587 (GH-13592)
* Add a whole new documentation page:
  "Python Initialization Configuration"
* PyWideStringList_Append() return type is now PyStatus,
  instead of int
* PyInterpreterState_New() now calls PyConfig_Clear() if
  PyConfig_InitPythonConfig() fails.
* Rename files:

  * Python/coreconfig.c => Python/initconfig.c
  * Include/cpython/coreconfig.h => Include/cpython/initconfig.h
  * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h

* Rename structures

  * _PyCoreConfig => PyConfig
  * _PyPreConfig => PyPreConfig
  * _PyInitError => PyStatus
  * _PyWstrList => PyWideStringList

* Rename PyConfig fields:

  * use_module_search_paths => module_search_paths_set
  * module_search_path_env => pythonpath_env

* Rename PyStatus field: _func => func
* PyInterpreterState: rename core_config field to config
* Rename macros and functions:

  * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv()
  * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv()
  * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString()
  * _PyInitError_Failed() => PyStatus_Exception()
  * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx
  * _Py_UnixMain() => Py_BytesMain()
  * _Py_ExitInitError() => Py_ExitStatusException()
  * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs()
  * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs()
  * _Py_PreInitialize() => Py_PreInitialize()
  * _Py_RunMain() => Py_RunMain()
  * _Py_InitializeFromConfig() => Py_InitializeFromConfig()
  * _Py_INIT_XXX() => _PyStatus_XXX()
  * _Py_INIT_FAILED() => _PyStatus_EXCEPTION()

* Rename 'err' PyStatus variables to 'status'
* Convert RUN_CODE() macro to config_run_code() static inline function
* Remove functions:

  * _Py_InitializeFromArgs()
  * _Py_InitializeFromWideArgs()
  * _PyInterpreterState_GetCoreConfig()
2019-05-27 16:39:22 +02:00
Alexey Izbyshev 6bc5917903 bpo-35091: Objects/listobject.c: Replace overflow checks in gallop fu… (GH-10202)
…nctions with asserts

The actual overflow can never happen because of the following:
* The size of a list can't be greater than PY_SSIZE_T_MAX / sizeof(PyObject*).
* The size of a pointer on all supported plaftorms is at least 4 bytes.
* ofs is positive and less than the list size at the beginning of each iteration.





https://bugs.python.org/issue35091
2019-05-22 17:01:08 -07:00
Jeroen Demeyer 351c67416b bpo-35983: skip trashcan for subclasses (GH-11841)
Add new trashcan macros to deal with a double deallocation that could occur when the `tp_dealloc` of a subclass calls the `tp_dealloc` of a base class and that base class uses the trashcan mechanism.

Patch by Jeroen Demeyer.
2019-05-10 19:21:10 +02:00
Rémi Lapeyre dd5417afcf bpo-36218: Fix handling of heterogeneous values in list.sort (GH-12209) 2019-03-25 00:25:37 -07:00
Sergey Fedoseev 234531b446 bpo-36030: Add _PyTuple_FromArray() function (GH-11954) 2019-02-25 17:59:12 +01:00
Zackery Spytz ebc793d6ac bpo-33989: Ensure that ms.key_compare is always initialized in list_sort_impl(). (GH-8710) 2019-02-21 09:47:14 +02:00
Sergey Fedoseev ef1b88bf57 bpo-36062: Minor speed-up for list slicing and copying. (GH-11967) 2019-02-21 08:51:52 +02:00
sth f8b534477a closes bpo-35623: Fix integer overflow when sorting large lists (GH-11380)
There is already a `Py_ssize_t i` defined at function scope that is used
for similar loops. By removing the local `int i` declaration that `i` is
used, which has the appropriate type.
2019-01-01 18:01:54 -08:00
Sergey Fedoseev 0e5f771f38 bpo-33234: Simplify list_preallocate_exact() (GH-11220) 2018-12-29 22:31:36 +00:00
Serhiy Storchaka bb86bf4c4e
bpo-35444: Unify and optimize the helper for getting a builtin object. (GH-11047)
This speeds up pickling of some iterators.

This fixes also error handling in pickling methods when fail to
look up builtin "getattr".
2018-12-11 08:28:18 +02:00
Victor Stinner bcda8f1d42
bpo-35081: Add Include/internal/pycore_object.h (GH-10640)
Move _PyObject_GC_TRACK() and _PyObject_GC_UNTRACK() from
Include/objimpl.h to Include/internal/pycore_object.h.
2018-11-21 22:27:47 +01:00
Victor Stinner 621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Victor Stinner e281f7d80c
bpo-35081: Move accu.h to Include/internal/pycore_accu.h (GH-10271)
The accu.h header is no longer part of the Python C API: it has been
moved to the "internal" headers which are restricted to Python
itself.

Replace #include "accu.h" with #include "pycore_accu.h".
2018-11-01 02:30:36 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
Pablo Galindo 372d705d95
bpo-33234 Improve list() pre-sizing for inputs with known lengths (GH-9846)
The list() constructor isn't taking full advantage of known input
lengths or length hints. This commit makes the constructor
pre-size and not over-allocate when the input size is known (the
input collection implements __len__). One on the main advantages is
that this provides 12% difference in memory savings due to the difference
between overallocating and allocating exactly the input size.

For efficiency purposes and to avoid a performance regression for small
generators and collections, the size of the input object is calculated using
__len__ and not __length_hint__, as the later is considerably slower.
2018-10-28 20:16:26 +00:00
Raymond Hettinger f1aa8aed4a
Micro-optimize list index range checks (GH-9784) 2018-10-10 20:37:28 -07:00
Sergey Fedoseev 2fc46979b8 bpo-34151: Improve performance of some list operations (GH-8332) 2018-08-11 21:12:07 +08:00
Victor Stinner caba55b3b7
bpo-34301: Add _PyInterpreterState_Get() helper function (GH-8592)
sys_setcheckinterval() now uses a local variable to parse arguments,
before writing into interp->check_interval.
2018-08-03 15:33:52 +02:00
Siddhesh Poyarekar 55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
Eddie Elizondo 745dc65b17 closes bpo-32898: Fix debug build crash with COUNT_ALLOCS (GH-5800) 2018-02-21 20:55:18 -08:00
Victor Stinner 8017b804a0
bpo-28685: Fix compiler warning (GH-5423) 2018-01-29 13:47:06 +01:00
embg 1e34da49ef bpo-28685: Optimize sorted() list.sort() with type-specialized comparisons (#582) 2018-01-28 19:03:23 -08:00
Serhiy Storchaka 1fb72d2ad2
bpo-32137: The repr of deeply nested dict now raises a RecursionError (#4570)
instead of crashing due to a stack overflow.

This perhaps will fix similar problems in other extension types.
2017-12-03 22:12:11 +02:00
Victor Stinner 25420fe290
bpo-32030: Add more options to _PyCoreConfig (#4485)
Py_Main() now handles two more -X options:

* -X showrefcount: new _PyCoreConfig.show_ref_count field
* -X showalloccount: new _PyCoreConfig.show_alloc_count field
2017-11-20 18:12:22 -08:00
stratakis e8b1965639 bpo-23699: Use a macro to reduce boilerplate code in rich comparison functions (GH-793) 2017-11-02 20:32:54 +10:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Serhiy Storchaka b879fe82e7 Expand the PySlice_GetIndicesEx macro. (#1023) 2017-04-08 09:53:51 +03:00