Commit Graph

4051 Commits

Author SHA1 Message Date
Eric Snow f78be59c83
bpo-45953: Preserve backward compatibility on some PyThreadState field names. (GH-31038)
The gevent project is using the two `PyThreadState` fields I renamed in gh-30590.  This PR fixes the names.  See #msg412046.
2022-02-01 09:02:25 -08:00
Mark Shannon 48be46ec1f
bpo-46072: Add some object layout and allocation stats (GH-31051) 2022-02-01 15:05:18 +00:00
Mark Shannon 90ab138bbd
bpo-46072: Add simple stats for Python calls. (GH-30989) 2022-01-28 15:20:33 +00:00
Mark Shannon 89fd7c3452
bpo-46329: Split calls into precall and call instructions. (GH-30855)
* Add PRECALL_FUNCTION opcode.

* Move 'call shape' varaibles into struct.

* Replace CALL_NO_KW and CALL_KW with KW_NAMES and CALL instructions.

* Specialize for builtin methods taking using the METH_FASTCALL | METH_KEYWORDS protocol.

* Allow kwnames for specialized calls to builtin types.

* Specialize calls to tuple(arg) and str(arg).
2022-01-28 12:42:30 +00:00
Victor Stinner 18ea973c21
bpo-40170: Remove PyHeapType_GET_MEMBERS() macro (GH-30942)
Remove the PyHeapType_GET_MEMBERS() macro. It was exposed in the
public C API by mistake, it must only be used by Python internally.
Use the PyTypeObject.tp_members member instead.

Rename PyHeapType_GET_MEMBERS() to _PyHeapType_GET_MEMBERS() and move
it to the internal C API.
2022-01-28 00:39:52 +01:00
Victor Stinner 0575551f69
bpo-40170: Move _Py_GetAllocatedBlocks() to pycore_pymem.h (GH-30943)
Move _Py_GetAllocatedBlocks() and _PyObject_DebugMallocStats()
declarations to pycore_pymem.h. These functions are related to memory
allocators, not to the PyObject structure.
2022-01-27 21:23:22 +01:00
Christian Heimes 26b0482393
bpo-46476: Simplify and fix _PyStaticCode_Dealloc (GH-30965) 2022-01-27 19:32:12 +00:00
Eric Snow 247480a21c
bpo-46541: Generate the global objects initializer. (gh-30941)
This change is a prerequisite for generating code for other global objects (like strings in gh-30928).

(We borrowed some code from Tools/scripts/deepfreeze.py.)

https://bugs.python.org/issue46541
2022-01-27 11:06:09 -07:00
Kumar Aditya c7f810b34d
bpo-46476: Fix memory leak in code objects generated by deepfreeze (GH-30853)
Add _Py_Deepfreeze_Fini() and _PyStaticCode_Dealloc() functions.
2022-01-27 14:03:47 +01:00
Victor Stinner 6b491b9dc0
bpo-40170: Remove _Py_GetAllocatedBlocks() function (GH-30940)
Move _Py_GetAllocatedBlocks() and _PyObject_DebugMallocStats()
private functions to the internal C API.
2022-01-27 03:35:51 +01:00
Victor Stinner af32b3ef1f
bpo-40170: PyType_SUPPORTS_WEAKREFS() becomes a regular function (GH-30938)
Convert the PyType_SUPPORTS_WEAKREFS() macro to a regular function.
It no longer access the PyTypeObject.tp_weaklistoffset member
directly.

Add _PyType_SUPPORTS_WEAKREFS() static inline functions, used
internally by Python for best performance.
2022-01-27 03:00:55 +01:00
Brandt Bucher 8548366864
bpo-46528: Simplify the VM's stack manipulations (GH-30902) 2022-01-26 12:47:45 -08:00
Victor Stinner d4a85f104b
bpo-35134: Add Include/cpython/descrobject.h (GH-30923)
Move Include/descrobject.h non-limited API to a new
Include/cpython/descrobject.h header file.
2022-01-26 17:32:47 +01:00
Mark Shannon 0367a36fdc
bpo-43683: Streamline YIELD_VALUE and SEND (GH-30723)
* Split YIELD_VALUE into ASYNC_GEN_WRAP; YIELD_VALUE for async generators.

* Split SEND into SEND; YIELD_VALUE.

* Document new opcodes.
2022-01-24 11:08:53 +00:00
Dong-hee Na 76dc047a0e
bpo-46481: Implement vectorcall for weakref.ref.__call__ method. (GH-30820) 2022-01-24 00:39:45 +09:00
Mark Dickinson 83a0ef2162
bpo-29882: Fix portability bug introduced in GH-30774 (#30794) 2022-01-23 09:59:34 +00:00
Victor Stinner 12f4ac3bc8
bpo-46417: Clear symtable identifiers at exit (GH-30809)
Add _PySymtable_Fini() function, called by finalize_interp_clear().

Update test_cmd_line.test_showrefcount() to tolerate negative
reference count.
2022-01-23 00:06:56 +01:00
Victor Stinner 1626bf4ac7
bpo-46417: Clear Unicode static types at exit (GH-30806)
Add _PyUnicode_FiniTypes() function, called by
finalize_interp_types(). It clears these static types:

* EncodingMapType
* PyFieldNameIter_Type
* PyFormatterIter_Type

_PyStaticType_Dealloc() now does nothing if tp_subclasses
is not NULL.
2022-01-22 22:55:39 +01:00
Victor Stinner f1bcdeaca6
bpo-46417: Factorize _PyExc_InitTypes() code (GH-30804)
Add 'static_exceptions' list to factorize code between
_PyExc_InitTypes() and _PyBuiltins_AddExceptions().

_PyExc_InitTypes() does nothing if it's not the main interpreter.

Sort exceptions in Lib/test/exception_hierarchy.txt.
2022-01-22 21:48:56 +01:00
Victor Stinner 500c146387
bpo-46417: Clear more static types (GH-30796)
* Move PyContext static types into object.c static_types list.
* Rename PyContextTokenMissing_Type to _PyContextTokenMissing_Type
  and declare it in pycore_context.h.
* _PyHamtItems types are no long exported: replace PyAPI_DATA() with
  extern.
2022-01-22 18:55:48 +01:00
Victor Stinner cd8de40b3b
bpo-29882: _Py_popcount32() doesn't need 64x64 multiply (GH-30774)
32x32 bits multiply is enough for _Py_popcount32().
2022-01-22 00:54:42 +01:00
Victor Stinner 8ee07dda13
bpo-46417: Add _PyType_GetSubclasses() function (GH-30761)
Add a new _PyType_GetSubclasses() function to get type's subclasses.

_PyType_GetSubclasses(type) returns a list which holds strong
refererences to subclasses. It is safer than iterating on
type->tp_subclasses which yields weak references and can be modified
in the loop.

_PyType_GetSubclasses(type) now holds a reference to the tp_subclasses
dict while creating the list of subclasses.

set_collection_flag_recursive() of _abc.c now uses
_PyType_GetSubclasses().
2022-01-21 23:29:10 +01:00
Victor Stinner bc67f189fd
bpo-46417: Add _PyType_CAST() macro (GH-30760)
In debug mode, the macro makes sure that its argument is a type using
an assertion.
2022-01-21 21:39:01 +01:00
Victor Stinner 595225e86d
bpo-46417: Py_Finalize() clears static types (GH-30743)
Add _PyTypes_FiniTypes() best-effort function to clear static types:
don't deallocate a type if it still has subclasses.

remove_subclass() now sets tp_subclasses to NULL when removing the
last subclass.
2022-01-21 13:06:34 +01:00
Victor Stinner 1781d55eb3
bpo-46417: _curses uses PyStructSequence_NewType() (GH-30736)
The _curses module now creates its ncurses_version type as a heap
type using PyStructSequence_NewType(), rather than using a static
type.

* Move _PyStructSequence_FiniType() definition to pycore_structseq.h.
* test.pythoninfo: log curses.ncurses_version.
2022-01-21 03:30:20 +01:00
Victor Stinner e9e3eab0b8
bpo-46417: Finalize structseq types at exit (GH-30645)
Add _PyStructSequence_FiniType() and _PyStaticType_Dealloc()
functions to finalize a structseq static type in Py_Finalize().
Currrently, these functions do nothing if Python is built in release
mode.

Clear static types:

* AsyncGenHooksType: sys.set_asyncgen_hooks()
* FlagsType: sys.flags
* FloatInfoType: sys.float_info
* Hash_InfoType: sys.hash_info
* Int_InfoType: sys.int_info
* ThreadInfoType: sys.thread_info
* UnraisableHookArgsType: sys.unraisablehook
* VersionInfoType: sys.version
* WindowsVersionType: sys.getwindowsversion()
2022-01-21 01:42:25 +01:00
Mark Shannon b04dfbbe4b
bpo-46409: Make generators in bytecode (GH-30633)
* Add RETURN_GENERATOR and JUMP_NO_INTERRUPT opcodes.

* Trim frame and generator by word each.

* Minor refactor of frame.c

* Update test.test_sys to account for smaller frames.

* Treat generator functions as normal functions when evaluating and specializing.
2022-01-20 11:46:39 +00:00
Pablo Galindo 2402f1e1f8
Merge remote-tracking branch 'upstream/main' 2022-01-14 21:20:56 +00:00
Pablo Galindo 01d4fe0976
Post 3.11.0a4 2022-01-14 21:20:41 +00:00
Mark Dickinson 025cbe7a9b
bpo-45569: Change PYLONG_BITS_IN_DIGIT default to 30 (GH-30497) 2022-01-14 18:54:56 +00:00
Eric Snow 322f962f3e
bpo-45953: Statically initialize all the non-object PyInterpreterState fields we can. (gh-30589)
https://bugs.python.org/issue45953
2022-01-13 17:17:28 -07:00
Eric Snow 324908ba93
bpo-45953: Statically initialize all the PyThreadState fields we can. (gh-30590)
https://bugs.python.org/issue45953
2022-01-13 17:09:24 -07:00
neonene d4e64cd4b0
bpo-46362: Ensure ntpath.abspath() uses the Windows API correctly (GH-30571)
This makes ntpath.abspath()/getpath_abspath() follow normpath(), since some WinAPIs such as PathCchSkipRoot() require backslashed paths.
2022-01-13 23:35:42 +00:00
Eric Snow b8ddf7e794
Statically initialize _PyRuntimeState fields. (gh-30588)
https://bugs.python.org/issue45953
2022-01-13 16:33:40 -07:00
Eric Snow bc02eac9d2
bpo-46370: Move the static initializer for _PyRuntime to its own header file. (gh-30587)
https://bugs.python.org/issue46370
2022-01-13 15:54:36 -07:00
Pablo Galindo 9471106fd5
Python 3.11.0a4 2022-01-13 19:38:15 +00:00
Christian Heimes a6ca8eee22
bpo-46315: Add ifdef HAVE_ feature checks for WASI compatibility (GH-30507) 2022-01-13 09:46:04 +01:00
Eric Snow ed57b36c32
bpo-45953: Statically allocate the main interpreter (and initial thread state). (gh-29883)
Previously, the main interpreter was allocated on the heap during runtime initialization.  Here we instead embed it into _PyRuntimeState, which means it is statically allocated as part of the _PyRuntime global.  The same goes for the initial thread state (of each interpreter, including the main one).  Consequently there are fewer allocations during runtime/interpreter init, fewer possible failures, and better memory locality.

FYI, this also helps efforts to consolidate globals, which in turns helps work on subinterpreter isolation.

https://bugs.python.org/issue45953
2022-01-12 16:28:46 -07:00
Eric Snow cf496d657a
bpo-45953: Statically allocate and initialize global bytes objects. (gh-30096)
The empty bytes object (b'') and the 256 one-character bytes objects were allocated at runtime init.  Now we statically allocate and initialize them.

https://bugs.python.org/issue45953
2022-01-11 09:37:24 -07:00
Victor Stinner ea1a54506b
bpo-46303: Move fileutils.h private functions to internal C API (GH-30484)
Move almost all private functions of Include/cpython/fileutils.h to
the internal C API Include/internal/pycore_fileutils.h.

Only keep _Py_fopen_obj() in Include/cpython/fileutils.h, since it's
used by _testcapi which must not use the internal C API.

Move EncodeLocaleEx() and DecodeLocaleEx() functions from _testcapi
to _testinternalcapi, since the C API moved to the internal C API.
2022-01-11 11:56:16 +01:00
Mark Shannon e028ae99ec
bpo-45923: Handle call events in bytecode (GH-30364)
* Add a RESUME instruction to handle "call" events.
2022-01-06 13:09:25 +00:00
penguin_wwy 3db762db72
bpo-46031: add POP_JUMP_IF_NOT_NONE and POP_JUMP_IF_NONE (GH-30019) 2022-01-06 11:38:35 +00:00
Victor Stinner 35d6540c90
bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085)" (GH-30422)
This reverts commit ea251806b8.

Keep "assert(interned == NULL);" in _PyUnicode_Fini(), but only for
the main interpreter.

Keep _PyUnicode_ClearInterned() changes avoiding the creation of a
temporary Python list object.
2022-01-06 08:53:44 +01:00
Brandt Bucher 332e6b9725
bpo-45256: Don't track the exact depth of each `InterpreterFrame` (GH-30372) 2022-01-05 11:30:26 +00:00
Brandt Bucher 31e43cbe5f
bpo-46009: Remove GEN_START (GH-30367) 2022-01-04 11:38:32 -08:00
Irit Katriel a94461d718
bpo-46202: Remove opcode POP_EXCEPT_AND_RERAISE (GH-30302)
* bpo-46202: remove opcode POP_EXCEPT_AND_RERAISE

* do not assume that an exception group is truthy
2022-01-04 10:37:12 +00:00
Irit Katriel 65e7c1f90e
bpo-46219, 46221: simplify except* implementation following exc_info changes. Move helpers to exceptions.c. Do not assume that exception groups are truthy. (GH-30289) 2022-01-02 23:22:42 +00:00
Tim Peters 863729e9c6
bpo-46218: Change long_pow() to sliding window algorithm (GH-30319)
* bpo-46218: Change long_pow() to sliding window algorithm

The primary motivation is to eliminate long_pow's reliance on that the number of bits in a long "digit" is a multiple of 5. Now it no longer cares how many bits are in a digit.

But the sliding window approach also allows cutting the precomputed table of small powers in half, which reduces initialization overhead enough that the approach pays off for smaller exponents too. Depending on exponent bit patterns, a sliding window may also be able to save some bigint multiplies (sometimes when at least 5 consecutive exponent bits are 0, regardless of their starting bit position modulo 5).

Note: boosting the window width to 6 didn't work well overall. It give marginal speed improvements for huge exponents, but the increased overhead (the small-power table needs twice as many entries) made it a loss for smaller exponents.

Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2022-01-02 13:18:20 -06:00
Brandt Bucher 77195cd44b
bpo-46090: Allow PyThreadState.datastack_* members to be NULL (GH-30234) 2021-12-28 09:49:48 -08:00
David Hewitt 31ff96712e
bpo-46140: take more Py_buffer arguments as const * (GH-30217) 2021-12-22 15:07:46 +02:00