cpython/Include/internal
Sam Gross 412920a41e
gh-112532: Improve mimalloc page visiting (#114133)
This adds support for visiting abandoned pages in mimalloc and improves
the performance of the page visiting code. Abandoned pages contain
memory blocks from threads that have exited. At some point, they may be
later reclaimed by other threads. We still need to visit those pages in
the free-threaded GC because they contain live objects.

This also reduces the overhead of visiting mimalloc pages:

 * Special cases for full, empty, and pages containing only a single
   block.
 * Fix free_map to use one bit instead of one byte per block.
 * Use fast integer division by a constant algorithm when computing
   block offset from block size and index.
2024-01-22 13:10:21 -08:00
..
mimalloc gh-112532: Improve mimalloc page visiting (#114133) 2024-01-22 13:10:21 -08:00
pycore_abstract.h gh-106320: Remove private AC converter functions (#108505) 2023-08-26 04:05:17 +02:00
pycore_asdl.h
pycore_ast.h gh-108216: Cleanup #include in internal header files (#108228) 2023-08-21 18:05:59 +00:00
pycore_ast_state.h gh-106905: Use separate structs to track recursion depth in each PyAST_mod2obj call. (GH-113035) 2023-12-25 19:36:59 +02:00
pycore_atexit.h gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207) 2023-12-07 12:33:40 -07:00
pycore_bitutils.h gh-108216: Cleanup #include in internal header files (#108228) 2023-08-21 18:05:59 +00:00
pycore_blocks_output_buffer.h gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221) 2023-08-21 19:15:52 +02:00
pycore_bytes_methods.h
pycore_bytesobject.h gh-107211: No longer export internal functions (5) (#108423) 2023-08-24 16:06:53 +00:00
pycore_call.h gh-112026: Restore removed private C API (#112115) 2023-11-15 16:38:31 +00:00
pycore_capsule.h gh-108240: Add pycore_capsule.h internal header file (#108596) 2023-08-29 01:20:02 +00:00
pycore_ceval.h gh-112723: Call `PyThreadState_Clear()` from the correct interpreter (#112776) 2023-12-12 17:20:21 -07:00
pycore_ceval_state.h gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207) 2023-12-07 12:33:40 -07:00
pycore_code.h gh-106550: Fix sign conversion in pycore_code.h (#112613) 2023-12-04 11:42:58 +01:00
pycore_codecs.h gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221) 2023-08-21 19:15:52 +02:00
pycore_compile.h gh-107149: make new opcode util functions private rather than public and unstable (#112042) 2023-11-14 00:31:02 +00:00
pycore_complexobject.h gh-112026: Restore removed private C API (#112115) 2023-11-15 16:38:31 +00:00
pycore_condvar.h gh-110014: Fix _POSIX_THREADS and _POSIX_SEMAPHORES usage (#110139) 2023-09-30 19:25:54 +02:00
pycore_context.h gh-111968: Use per-thread freelists for PyContext in free-threading (gh-114122) 2024-01-16 16:14:56 +00:00
pycore_critical_section.h gh-111863: Rename `Py_NOGIL` to `Py_GIL_DISABLED` (#111864) 2023-11-20 15:52:00 +02:00
pycore_crossinterp.h gh-76785: Avoid Pickled TracebackException for Propagated Subinterpreter Exceptions (gh-113036) 2023-12-13 00:31:30 +00:00
pycore_descrobject.h gh-106320: Move _PyMethodWrapper_Type to internal C API (#107064) 2023-07-22 20:57:59 +00:00
pycore_dict.h gh-112026: Restore removed private C API (#112115) 2023-11-15 16:38:31 +00:00
pycore_dict_state.h gh-104252: Immortalize Py_EMPTY_KEYS (gh-104253) 2023-05-10 07:28:40 -06:00
pycore_dtoa.h gh-111962: Make dtoa thread-safe in `--disable-gil` builds. (#112049) 2023-12-07 13:47:55 +00:00
pycore_emscripten_signal.h GH-108614: Unbreak emscripten build (GH-109132) 2023-09-08 17:54:45 +01:00
pycore_emscripten_trampoline.h gh-106213: Make Emscripten trampolines work with JSPI (GH-106219) 2023-09-15 15:04:21 -07:00
pycore_exceptions.h
pycore_faulthandler.h gh-108216: Cleanup #include in internal header files (#108228) 2023-08-21 18:05:59 +00:00
pycore_fileutils.h gh-112567: Add _Py_GetTicksPerSecond() function (#112587) 2023-12-01 17:05:56 +01:00
pycore_fileutils_windows.h gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221) 2023-08-21 19:15:52 +02:00
pycore_floatobject.h gh-111968: Use per-thread freelists for float in free-threading (gh-113886) 2024-01-10 15:47:13 +00:00
pycore_flowgraph.h gh-106149: move CFG and basicblock definitions into flowgraph.c, use them as opaque types in compile.c (#107639) 2023-08-10 13:03:47 +01:00
pycore_format.h gh-104018: remove unused format "z" handling in string formatfloat() (#104107) 2023-05-07 10:11:42 +05:30
pycore_frame.h gh-111354: remove comparisons with enum values, variable reuse, unused imports in genobject.c (#111708) 2023-11-09 10:27:20 +00:00
pycore_freelist.h gh-111968: Use per-thread freelists for generator in free-threading (gh-114189) 2024-01-18 18:15:00 +00:00
pycore_function.h gh-106581: Project through calls (#108067) 2023-08-17 11:29:58 -07:00
pycore_gc.h gh-111968: Use per-thread freelists for generator in free-threading (gh-114189) 2024-01-18 18:15:00 +00:00
pycore_genobject.h gh-111968: Use per-thread freelists for generator in free-threading (gh-114189) 2024-01-18 18:15:00 +00:00
pycore_getopt.h
pycore_gil.h gh-109693: Update _gil_runtime_state.locked to use pyatomic.h (gh-110836) 2023-10-17 07:32:50 +09:00
pycore_global_objects.h gh-108216: Cleanup #include in internal header files (#108228) 2023-08-21 18:05:59 +00:00
pycore_global_objects_fini_generated.h gh-113626: Add allow_code parameter in marshal functions (GH-113648) 2024-01-16 18:05:15 +02:00
pycore_global_strings.h gh-113626: Add allow_code parameter in marshal functions (GH-113648) 2024-01-16 18:05:15 +02:00
pycore_hamt.h
pycore_hashtable.h gh-107211: No longer export internal functions (5) (#108423) 2023-08-24 16:06:53 +00:00
pycore_identifier.h gh-112026: Restore removed private C API (#112115) 2023-11-15 16:38:31 +00:00
pycore_import.h gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207) 2023-12-07 12:33:40 -07:00
pycore_importdl.h gh-111863: Rename `Py_NOGIL` to `Py_GIL_DISABLED` (#111864) 2023-11-20 15:52:00 +02:00
pycore_initconfig.h gh-108634: PyInterpreterState_New() no longer calls Py_FatalError() (#108748) 2023-09-01 12:43:30 +02:00
pycore_instruments.h gh-103615: Use local events for opcode tracing (GH-109472) 2023-11-03 16:39:50 +00:00
pycore_interp.h gh-111968: Use per-thread freelists for generator in free-threading (gh-114189) 2024-01-18 18:15:00 +00:00
pycore_intrinsics.h gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221) 2023-08-21 19:15:52 +02:00
pycore_list.h gh-111968: Introduce _PyFreeListState and _PyFreeListState_GET API (gh-113584) 2024-01-10 08:04:41 +09:00
pycore_llist.h gh-108724: Add PyMutex and _PyParkingLot APIs (gh-109344) 2023-09-19 09:54:29 -06:00
pycore_lock.h gh-111964: Add _PyRWMutex a "readers-writer" lock (gh-112859) 2023-12-15 18:56:55 -07:00
pycore_long.h gh-112026: Restore removed private C API (#112115) 2023-11-15 16:38:31 +00:00
pycore_memoryobject.h gh-106320: Remove private _PyManagedBuffer_Type (#108431) 2023-08-24 19:07:54 +02:00
pycore_mimalloc.h gh-112532: Isolate abandoned segments by interpreter (#113717) 2024-01-04 22:21:40 +00:00
pycore_modsupport.h gh-111956: Add thread-safe one-time initialization. (gh-111960) 2023-11-16 12:19:54 -07:00
pycore_moduleobject.h gh-106320: Remove private _PyModule API (#107070) 2023-07-22 21:41:11 +00:00
pycore_namespace.h gh-107211: No longer export internal functions (4) (#107217) 2023-07-25 03:16:28 +00:00
pycore_object.h gh-111863: Rename `Py_NOGIL` to `Py_GIL_DISABLED` (#111864) 2023-11-20 15:52:00 +02:00
pycore_object_alloc.h gh-112529: Use GC heaps for GC allocations in free-threaded builds (gh-114157) 2024-01-21 01:14:45 +09:00
pycore_object_state.h gh-108634: Py_TRACE_REFS uses a hash table (#108663) 2023-08-31 18:33:34 +02:00
pycore_obmalloc.h gh-112867: fix for WITH_PYMALLOC_RADIX_TREE=0 (GH-112885) 2023-12-09 13:50:48 -08:00
pycore_obmalloc_init.h gh-101659: Isolate "obmalloc" State to Each Interpreter (gh-101660) 2023-04-24 17:23:57 -06:00
pycore_opcode_metadata.h gh-113710: Add types to the interpreter DSL (#113711) 2024-01-13 01:30:27 +08:00
pycore_opcode_utils.h gh-111354: simplify detection of RESUME after YIELD_VALUE at except-depth 1 (#111459) 2023-11-02 10:18:43 +00:00
pycore_optimizer.h GH-113860: Get rid of `_PyUOpExecutorObject` (GH-113954) 2024-01-12 11:58:23 +00:00
pycore_parking_lot.h gh-108724: Add PyMutex and _PyParkingLot APIs (gh-109344) 2023-09-19 09:54:29 -06:00
pycore_parser.h gh-110805: Allow the repl to show source code and complete tracebacks (#110775) 2023-10-13 09:25:37 +00:00
pycore_pathconfig.h gh-107211: No longer export internal functions (6) (#108424) 2023-08-24 17:28:35 +02:00
pycore_pyarena.h gh-107211: Fix test_peg_generator (#108435) 2023-08-24 17:47:44 +00:00
pycore_pybuffer.h gh-76785: Add SendChannel.send_buffer() (#110246) 2023-10-09 07:39:51 -06:00
pycore_pyerrors.h gh-76785: Move _Py_excinfo Functions Out of the Internal C-API (gh-111715) 2023-11-06 11:09:22 -07:00
pycore_pyhash.h gh-111545: Add Py_HashPointer() function (#112096) 2023-12-06 15:09:22 +01:00
pycore_pylifecycle.h gh-112723: Call `PyThreadState_Clear()` from the correct interpreter (#112776) 2023-12-12 17:20:21 -07:00
pycore_pymath.h Remove unused internal macros (#102415) 2023-03-04 12:20:14 +00:00
pycore_pymem.h gh-112529: Track if debug allocator is used as underlying allocator (#113747) 2024-01-16 13:42:15 -08:00
pycore_pymem_init.h gh-112529: Track if debug allocator is used as underlying allocator (#113747) 2024-01-16 13:42:15 -08:00
pycore_pystate.h gh-111968: Introduce _PyFreeListState and _PyFreeListState_GET API (gh-113584) 2024-01-10 08:04:41 +09:00
pycore_pystats.h gh-108753: Enhance pystats (#108754) 2023-09-06 15:54:59 +00:00
pycore_pythonrun.h gh-106320: Remove private pythonrun API (#108599) 2023-08-29 04:18:52 +02:00
pycore_pythread.h GH-110829: Ensure Thread.join() joins the OS thread (#110848) 2023-11-04 13:59:24 +00:00
pycore_range.h gh-89189: More compact range iterator (GH-27986) 2022-11-30 23:04:30 +02:00
pycore_runtime.h gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207) 2023-12-07 12:33:40 -07:00
pycore_runtime_init.h gh-112529: Track if debug allocator is used as underlying allocator (#113747) 2024-01-16 13:42:15 -08:00
pycore_runtime_init_generated.h gh-113626: Add allow_code parameter in marshal functions (GH-113648) 2024-01-16 18:05:15 +02:00
pycore_semaphore.h gh-110014: Fix _POSIX_THREADS and _POSIX_SEMAPHORES usage (#110139) 2023-09-30 19:25:54 +02:00
pycore_setobject.h gh-107211: No longer export internal functions (7) (#108425) 2023-08-24 17:40:56 +02:00
pycore_signal.h gh-109693: Use pyatomic.h for signal module (gh-110480) 2023-10-10 08:26:29 +09:00
pycore_sliceobject.h gh-111968: Use per-thread slice_cache in free-threading (gh-113972) 2024-01-16 00:38:57 +09:00
pycore_strhex.h gh-107211: No longer export pycore_strhex.h functions (#108229) 2023-08-21 18:12:22 +00:00
pycore_structseq.h gh-107211: No longer export internal functions (4) (#107217) 2023-07-25 03:16:28 +00:00
pycore_symtable.h gh-107211: Rename PySymtable_Lookup() to _PySymtable_Lookup() (#107212) 2023-07-25 00:54:09 +00:00
pycore_sysmodule.h gh-106320: Remove private _PySys functions (#108452) 2023-08-24 20:02:09 +00:00
pycore_time.h gh-112567: Add _PyTimeFraction C API (#112568) 2023-12-01 19:50:10 +01:00
pycore_token.h gh-107211: No longer export internal functions (7) (#108425) 2023-08-24 17:40:56 +02:00
pycore_traceback.h gh-110721: Use the traceback module for PyErr_Display() and fallback to the C implementation (#110702) 2023-10-12 14:52:14 +00:00
pycore_tracemalloc.h gh-107211: No longer export internal functions (3) (#107215) 2023-07-25 02:25:45 +00:00
pycore_tstate.h gh-111968: Introduce _PyFreeListState and _PyFreeListState_GET API (gh-113584) 2024-01-10 08:04:41 +09:00
pycore_tuple.h gh-111968: Use per-thread freelists for tuple in free-threading (gh-113921) 2024-01-12 03:46:28 +09:00
pycore_typeobject.h gh-111178: Make slot functions in typeobject.c have compatible types (GH-112752) 2023-12-20 15:13:44 +01:00
pycore_typevarobject.h gh-104549: Set __module__ on TypeAliasType (#104550) 2023-05-18 15:56:15 -07:00
pycore_ucnhash.h gh-111972: Make Unicode name C APIcapsule initialization thread-safe (#112249) 2023-11-30 11:12:49 +01:00
pycore_unicodeobject.h gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207) 2023-12-07 12:33:40 -07:00
pycore_unicodeobject_generated.h gh-113626: Add allow_code parameter in marshal functions (GH-113648) 2024-01-16 18:05:15 +02:00
pycore_unionobject.h
pycore_uop_ids.h gh-113710: Add types to the interpreter DSL (#113711) 2024-01-13 01:30:27 +08:00
pycore_uop_metadata.h GH-112354: `_GUARD_IS_TRUE_POP` side-exits to target the next instruction, not themselves. (GH-114078) 2024-01-15 11:41:06 +00:00
pycore_warnings.h gh-107211: No longer export internal functions (3) (#107215) 2023-07-25 02:25:45 +00:00
pycore_weakref.h gh-111926: Set up basic sementics of weakref API for freethreading (gh-113621) 2024-01-03 13:25:27 +00:00