Commit Graph

4190 Commits

Author SHA1 Message Date
Brandt Bucher 8548366864
bpo-46528: Simplify the VM's stack manipulations (GH-30902) 2022-01-26 12:47:45 -08:00
Victor Stinner d4a85f104b
bpo-35134: Add Include/cpython/descrobject.h (GH-30923)
Move Include/descrobject.h non-limited API to a new
Include/cpython/descrobject.h header file.
2022-01-26 17:32:47 +01:00
Mark Shannon 0367a36fdc
bpo-43683: Streamline YIELD_VALUE and SEND (GH-30723)
* Split YIELD_VALUE into ASYNC_GEN_WRAP; YIELD_VALUE for async generators.

* Split SEND into SEND; YIELD_VALUE.

* Document new opcodes.
2022-01-24 11:08:53 +00:00
Dong-hee Na 76dc047a0e
bpo-46481: Implement vectorcall for weakref.ref.__call__ method. (GH-30820) 2022-01-24 00:39:45 +09:00
Mark Dickinson 83a0ef2162
bpo-29882: Fix portability bug introduced in GH-30774 (#30794) 2022-01-23 09:59:34 +00:00
Victor Stinner 12f4ac3bc8
bpo-46417: Clear symtable identifiers at exit (GH-30809)
Add _PySymtable_Fini() function, called by finalize_interp_clear().

Update test_cmd_line.test_showrefcount() to tolerate negative
reference count.
2022-01-23 00:06:56 +01:00
Victor Stinner 1626bf4ac7
bpo-46417: Clear Unicode static types at exit (GH-30806)
Add _PyUnicode_FiniTypes() function, called by
finalize_interp_types(). It clears these static types:

* EncodingMapType
* PyFieldNameIter_Type
* PyFormatterIter_Type

_PyStaticType_Dealloc() now does nothing if tp_subclasses
is not NULL.
2022-01-22 22:55:39 +01:00
Victor Stinner f1bcdeaca6
bpo-46417: Factorize _PyExc_InitTypes() code (GH-30804)
Add 'static_exceptions' list to factorize code between
_PyExc_InitTypes() and _PyBuiltins_AddExceptions().

_PyExc_InitTypes() does nothing if it's not the main interpreter.

Sort exceptions in Lib/test/exception_hierarchy.txt.
2022-01-22 21:48:56 +01:00
Victor Stinner 500c146387
bpo-46417: Clear more static types (GH-30796)
* Move PyContext static types into object.c static_types list.
* Rename PyContextTokenMissing_Type to _PyContextTokenMissing_Type
  and declare it in pycore_context.h.
* _PyHamtItems types are no long exported: replace PyAPI_DATA() with
  extern.
2022-01-22 18:55:48 +01:00
Victor Stinner cd8de40b3b
bpo-29882: _Py_popcount32() doesn't need 64x64 multiply (GH-30774)
32x32 bits multiply is enough for _Py_popcount32().
2022-01-22 00:54:42 +01:00
Victor Stinner 8ee07dda13
bpo-46417: Add _PyType_GetSubclasses() function (GH-30761)
Add a new _PyType_GetSubclasses() function to get type's subclasses.

_PyType_GetSubclasses(type) returns a list which holds strong
refererences to subclasses. It is safer than iterating on
type->tp_subclasses which yields weak references and can be modified
in the loop.

_PyType_GetSubclasses(type) now holds a reference to the tp_subclasses
dict while creating the list of subclasses.

set_collection_flag_recursive() of _abc.c now uses
_PyType_GetSubclasses().
2022-01-21 23:29:10 +01:00
Victor Stinner bc67f189fd
bpo-46417: Add _PyType_CAST() macro (GH-30760)
In debug mode, the macro makes sure that its argument is a type using
an assertion.
2022-01-21 21:39:01 +01:00
Victor Stinner 595225e86d
bpo-46417: Py_Finalize() clears static types (GH-30743)
Add _PyTypes_FiniTypes() best-effort function to clear static types:
don't deallocate a type if it still has subclasses.

remove_subclass() now sets tp_subclasses to NULL when removing the
last subclass.
2022-01-21 13:06:34 +01:00
Victor Stinner 1781d55eb3
bpo-46417: _curses uses PyStructSequence_NewType() (GH-30736)
The _curses module now creates its ncurses_version type as a heap
type using PyStructSequence_NewType(), rather than using a static
type.

* Move _PyStructSequence_FiniType() definition to pycore_structseq.h.
* test.pythoninfo: log curses.ncurses_version.
2022-01-21 03:30:20 +01:00
Victor Stinner e9e3eab0b8
bpo-46417: Finalize structseq types at exit (GH-30645)
Add _PyStructSequence_FiniType() and _PyStaticType_Dealloc()
functions to finalize a structseq static type in Py_Finalize().
Currrently, these functions do nothing if Python is built in release
mode.

Clear static types:

* AsyncGenHooksType: sys.set_asyncgen_hooks()
* FlagsType: sys.flags
* FloatInfoType: sys.float_info
* Hash_InfoType: sys.hash_info
* Int_InfoType: sys.int_info
* ThreadInfoType: sys.thread_info
* UnraisableHookArgsType: sys.unraisablehook
* VersionInfoType: sys.version
* WindowsVersionType: sys.getwindowsversion()
2022-01-21 01:42:25 +01:00
Mark Shannon b04dfbbe4b
bpo-46409: Make generators in bytecode (GH-30633)
* Add RETURN_GENERATOR and JUMP_NO_INTERRUPT opcodes.

* Trim frame and generator by word each.

* Minor refactor of frame.c

* Update test.test_sys to account for smaller frames.

* Treat generator functions as normal functions when evaluating and specializing.
2022-01-20 11:46:39 +00:00
Pablo Galindo 2402f1e1f8
Merge remote-tracking branch 'upstream/main' 2022-01-14 21:20:56 +00:00
Pablo Galindo 01d4fe0976
Post 3.11.0a4 2022-01-14 21:20:41 +00:00
Mark Dickinson 025cbe7a9b
bpo-45569: Change PYLONG_BITS_IN_DIGIT default to 30 (GH-30497) 2022-01-14 18:54:56 +00:00
Eric Snow 322f962f3e
bpo-45953: Statically initialize all the non-object PyInterpreterState fields we can. (gh-30589)
https://bugs.python.org/issue45953
2022-01-13 17:17:28 -07:00
Eric Snow 324908ba93
bpo-45953: Statically initialize all the PyThreadState fields we can. (gh-30590)
https://bugs.python.org/issue45953
2022-01-13 17:09:24 -07:00
neonene d4e64cd4b0
bpo-46362: Ensure ntpath.abspath() uses the Windows API correctly (GH-30571)
This makes ntpath.abspath()/getpath_abspath() follow normpath(), since some WinAPIs such as PathCchSkipRoot() require backslashed paths.
2022-01-13 23:35:42 +00:00
Eric Snow b8ddf7e794
Statically initialize _PyRuntimeState fields. (gh-30588)
https://bugs.python.org/issue45953
2022-01-13 16:33:40 -07:00
Eric Snow bc02eac9d2
bpo-46370: Move the static initializer for _PyRuntime to its own header file. (gh-30587)
https://bugs.python.org/issue46370
2022-01-13 15:54:36 -07:00
Pablo Galindo 9471106fd5
Python 3.11.0a4 2022-01-13 19:38:15 +00:00
Christian Heimes a6ca8eee22
bpo-46315: Add ifdef HAVE_ feature checks for WASI compatibility (GH-30507) 2022-01-13 09:46:04 +01:00
Eric Snow ed57b36c32
bpo-45953: Statically allocate the main interpreter (and initial thread state). (gh-29883)
Previously, the main interpreter was allocated on the heap during runtime initialization.  Here we instead embed it into _PyRuntimeState, which means it is statically allocated as part of the _PyRuntime global.  The same goes for the initial thread state (of each interpreter, including the main one).  Consequently there are fewer allocations during runtime/interpreter init, fewer possible failures, and better memory locality.

FYI, this also helps efforts to consolidate globals, which in turns helps work on subinterpreter isolation.

https://bugs.python.org/issue45953
2022-01-12 16:28:46 -07:00
Eric Snow cf496d657a
bpo-45953: Statically allocate and initialize global bytes objects. (gh-30096)
The empty bytes object (b'') and the 256 one-character bytes objects were allocated at runtime init.  Now we statically allocate and initialize them.

https://bugs.python.org/issue45953
2022-01-11 09:37:24 -07:00
Victor Stinner ea1a54506b
bpo-46303: Move fileutils.h private functions to internal C API (GH-30484)
Move almost all private functions of Include/cpython/fileutils.h to
the internal C API Include/internal/pycore_fileutils.h.

Only keep _Py_fopen_obj() in Include/cpython/fileutils.h, since it's
used by _testcapi which must not use the internal C API.

Move EncodeLocaleEx() and DecodeLocaleEx() functions from _testcapi
to _testinternalcapi, since the C API moved to the internal C API.
2022-01-11 11:56:16 +01:00
Mark Shannon e028ae99ec
bpo-45923: Handle call events in bytecode (GH-30364)
* Add a RESUME instruction to handle "call" events.
2022-01-06 13:09:25 +00:00
penguin_wwy 3db762db72
bpo-46031: add POP_JUMP_IF_NOT_NONE and POP_JUMP_IF_NONE (GH-30019) 2022-01-06 11:38:35 +00:00
Victor Stinner 35d6540c90
bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085)" (GH-30422)
This reverts commit ea251806b8.

Keep "assert(interned == NULL);" in _PyUnicode_Fini(), but only for
the main interpreter.

Keep _PyUnicode_ClearInterned() changes avoiding the creation of a
temporary Python list object.
2022-01-06 08:53:44 +01:00
Brandt Bucher 332e6b9725
bpo-45256: Don't track the exact depth of each `InterpreterFrame` (GH-30372) 2022-01-05 11:30:26 +00:00
Brandt Bucher 31e43cbe5f
bpo-46009: Remove GEN_START (GH-30367) 2022-01-04 11:38:32 -08:00
Irit Katriel a94461d718
bpo-46202: Remove opcode POP_EXCEPT_AND_RERAISE (GH-30302)
* bpo-46202: remove opcode POP_EXCEPT_AND_RERAISE

* do not assume that an exception group is truthy
2022-01-04 10:37:12 +00:00
Irit Katriel 65e7c1f90e
bpo-46219, 46221: simplify except* implementation following exc_info changes. Move helpers to exceptions.c. Do not assume that exception groups are truthy. (GH-30289) 2022-01-02 23:22:42 +00:00
Tim Peters 863729e9c6
bpo-46218: Change long_pow() to sliding window algorithm (GH-30319)
* bpo-46218: Change long_pow() to sliding window algorithm

The primary motivation is to eliminate long_pow's reliance on that the number of bits in a long "digit" is a multiple of 5. Now it no longer cares how many bits are in a digit.

But the sliding window approach also allows cutting the precomputed table of small powers in half, which reduces initialization overhead enough that the approach pays off for smaller exponents too. Depending on exponent bit patterns, a sliding window may also be able to save some bigint multiplies (sometimes when at least 5 consecutive exponent bits are 0, regardless of their starting bit position modulo 5).

Note: boosting the window width to 6 didn't work well overall. It give marginal speed improvements for huge exponents, but the increased overhead (the small-power table needs twice as many entries) made it a loss for smaller exponents.

Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2022-01-02 13:18:20 -06:00
Brandt Bucher 77195cd44b
bpo-46090: Allow PyThreadState.datastack_* members to be NULL (GH-30234) 2021-12-28 09:49:48 -08:00
David Hewitt 31ff96712e
bpo-46140: take more Py_buffer arguments as const * (GH-30217) 2021-12-22 15:07:46 +02:00
Raymond Hettinger 29ea68bd1d
Revert "bpo-46131: add fastpath for PyFloat_Check() (GH-30200)" (GH-30208)
This reverts commit 2ef06d4125.
2021-12-19 15:05:46 -06:00
Matti Picus 2ef06d4125
bpo-46131: add fastpath for PyFloat_Check() (#30200) 2021-12-19 14:24:30 -06:00
Mark Shannon efd6236d36
bpo-46072: Add top level stats struct (GH-30169) 2021-12-17 14:48:01 +00:00
Irit Katriel 396b58345f
bpo-45711: Remove type and traceback from exc_info (GH-30122)
* Do not PUSH/POP traceback or type to the stack as part of exc_info

* Remove exc_traceback and exc_type from _PyErr_StackItem

* Add to what's new, because this change breaks things like Cython
2021-12-17 14:46:22 +00:00
Mark Shannon 342b93f9f2
bpo-46072: Add --with-pystats configure option to simplify gathering of VM stats (GH-30116)
* Simplify specialization stats collection macros.

* Add --enable-pystats option to configure.

* Update specialization summary script to handle larger number of kinds
2021-12-15 15:32:32 +00:00
Mark Shannon 3a60bfef49
bpo-44525: Specialize for calls to type and other builtin classes with 1 argument. (GH-29942) 2021-12-15 15:03:42 +00:00
Mark Shannon 0b50a4f0cd
bpo-46039: Split yield from in two (GH-30035)
* Split YIELD_FROM opcode into SEND and JUMP_ABSOLUTE.

* Remove YIELD_FROM opcode.
2021-12-15 10:30:09 +00:00
Mark Shannon 9f8f45144b
bpo-44525: Split calls into PRECALL and CALL (GH-30011)
* Add 3 new opcodes for calls: PRECALL_METHOD, CALL_NO_KW, CALL_KW.

* Update specialization to handle new CALL opcodes.

* Specialize call to method descriptors.

* Remove old CALL opcodes: CALL_FUNCTION, CALL_METHOD, CALL_METHOD_KW, CALL_FUNCTION_KW.
2021-12-14 18:22:44 +00:00
Irit Katriel d60457a667
bpo-45292: [PEP-654] add except* (GH-29581) 2021-12-14 16:48:15 +00:00
Eric Snow 121f1f893a
bpo-45953: Statically initialize the small ints. (gh-30092)
The array of small PyLong objects has been statically declared. Here I also statically initialize them. Consequently they are no longer initialized dynamically during runtime init.

I've also moved them under a new sub-struct in _PyRuntimeState, in preparation for static allocation and initialization of other global objects.

https://bugs.python.org/issue45953
2021-12-13 18:04:05 -07:00
Pablo Galindo Salgado 59435eea08
bpo-46042: Improve SyntaxError locations in the symbol table (GH-30059) 2021-12-11 21:28:24 +00:00
Gabriele N. Tornetta 50669083fe
bpo-43931: Export Python version as API data (GH-25577)
When Python is embedded in other applications, it is not easy to determine which version of Python is being used. This change exposes the Python version as part of the API data. Tools like Austin (https://github.com/P403n1x87/austin) can benefit from this data when targeting applications like uWSGI, as the Python version can then be inferred systematically by looking at the exported symbols rather than relying on unreliable pattern matching or other hacks (like remote code execution etc...).

Automerge-Triggered-By: GH:pablogsal
2021-12-09 17:52:05 -08:00
Eric Snow c8749b5783
bpo-46008: Make runtime-global object/type lifecycle functions and state consistent. (gh-29998)
This change is strictly renames and moving code around.  It helps in the following ways:

* ensures type-related init functions focus strictly on one of the three aspects (state, objects, types)
* passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter
* consistent naming conventions help make what's going on more clear
* keeping API related to a type in the corresponding header file makes it more obvious where to look for it

https://bugs.python.org/issue46008
2021-12-09 12:59:26 -07:00
Victor Stinner 73325bbe77
bpo-46007: Exclude PyUnicode_CHECK_INTERNED() from limited C API (GH-29987)
Exclude the PyUnicode_CHECK_INTERNED() macro from the limited C API,
because it uses the PyASCIIObject structure which is excluded from
the limited C API.

Automerge-Triggered-By: GH:encukou
2021-12-09 00:58:09 -08:00
Pablo Galindo 23d52fc8c1
Post 3.11.0a3 2021-12-08 23:40:52 +00:00
Pablo Galindo 2e91dba437
Python 3.11.0a3 2021-12-08 22:24:29 +00:00
Eric Snow 32a67246b0
bpo-46008: Move Py*State init into distinct functions. (gh-29977)
Previously, basic initialization of PyInterprterState happened in PyInterpreterState_New() (along with allocation and adding the new interpreter to the runtime state). This prevented us from initializing interpreter states that were allocated separately (e.g. statically or in a free list). We've addressed that here by factoring out a separate function just for initialization. We've done the same for PyThreadState. _PyRuntimeState was sorted out when we added it since _PyRuntime is statically allocated. However, here we update the existing init code to line up with the functions for PyInterpreterState and PyThreadState.

https://bugs.python.org/issue46008
2021-12-07 18:59:49 -07:00
Eric Snow 758b74e71e
bpo-46008: Add _PyInterpreterState_Main(). (gh-29978)
PyInterpreterState_Main() is a plain function exposed in the public C-API. For internal usage we can take the more efficient approach in this PR.

https://bugs.python.org/issue46008
2021-12-07 18:56:06 -07:00
Eric Snow 1f384e3184
bpo-46008: Stop calling _PyThreadState_Init() in new_threadstate(). (gh-29973)
This simplifies new_threadstate().  We also rename _PyThreadState_Init() to _PyThreadState_SetCurrent() to reflect what it actually does.

https://bugs.python.org/issue46008
2021-12-07 17:26:29 -07:00
Eric Snow 313f92a57b
bpo-46008: Move thread-related interpreter state into a sub-struct. (gh-29971)
This parallels _PyRuntimeState.interpreters.  Doing this helps make it more clear what part of PyInterpreterState relates to its threads.

https://bugs.python.org/issue46008
2021-12-07 14:03:47 -07:00
Eric Snow 8262c96bcc
bpo-46008: Return void from _PyEval_InitState(). (gh-29970)
This falls into the category of keep-allocation-and-initialization separate. It also allows us to use _PyEval_InitState() safely in functions that return void.

https://bugs.python.org/issue46008
2021-12-07 14:02:17 -07:00
Mark Shannon 8319114fee
bpo-45947: Place dict and values pointer at fixed (negative) offset just before GC header. (GH-29879)
* Place __dict__ immediately before GC header for plain Python objects.

* Fix up lazy dict creation logic to use managed dict pointers.

* Manage values pointer, placing them directly before managed dict pointers.

* Convert hint-based load/store attr specialization target managed dict classes.

* Specialize LOAD_METHOD for managed dict objects.

* Remove unsafe _PyObject_GC_Calloc function.

* Remove unsafe _PyObject_GC_Malloc() function.

* Add comment explaning use of Py_TPFLAGS_MANAGED_DICT.
2021-12-07 16:02:53 +00:00
Mark Shannon 299483c95d
bpo-45963: Make space for the InterpreterFrame of a generator in that generator. (GH-29891)
* Make generator, coroutine and async gen structs all the same size.

* Store interpreter frame in generator (and coroutine). Reduces the number of allocations neeeded for a generator from two to one.
2021-12-06 10:13:49 +00:00
Benjamin Peterson 0ae4e0c959
bpo-45582 Fix prototype of _Py_Get_Getpath_CodeObject. (GH-29907)
Automerge-Triggered-By: GH:tiran
2021-12-03 15:21:58 -08:00
Irit Katriel 5bb7ef2768
bpo-45607: Make it possible to enrich exception displays via setting their __note__ field (GH-29880) 2021-12-03 22:01:15 +00:00
Dennis Sweeney 03768c4d13
bpo-45885: Specialize COMPARE_OP (GH-29734)
* Add COMPARE_OP_ADAPTIVE adaptive instruction.

* Add COMPARE_OP_FLOAT_JUMP, COMPARE_OP_INT_JUMP and COMPARE_OP_STR_JUMP specialized instructions.

* Introduce and use _PyUnicode_Equal
2021-12-03 11:29:12 +00:00
Steve Dower 99fcf15052
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
2021-12-03 00:08:42 +00:00
Christian Heimes cb8f491f46
bpo-40280: Optimize ints and and startup on wasm (GH-29887) 2021-12-02 12:19:30 +01:00
Victor Stinner a6c3b0faa1
bpo-45954: Rename PyConfig.no_debug_ranges to code_debug_ranges (GH-29886)
Rename PyConfig.no_debug_ranges to PyConfig.code_debug_ranges and
invert the value.

Document -X no_debug_ranges and PYTHONNODEBUGRANGES env var in
PyConfig.code_debug_ranges documentation.
2021-12-02 11:43:37 +01:00
Mark Shannon 49444fb807
bpo-45753: Interpreter internal tweaks (GH-29575)
* Split exit paths into exceptional and non-exceptional.

* Move exit tracing code to individual bytecodes.

* Wrap all trace entry and exit events in macros to make them clearer and easier to enhance.

* Move return sequence into RETURN_VALUE, YIELD_VALUE and YIELD_FROM. Distinguish between normal trace events and dtrace events.
2021-12-01 12:09:36 +00:00
Victor Stinner 4b97d974ec
bpo-45476: Disallow using asdl_seq_GET() as l-value (GH-29866)
The following internal macros can not longer be used as l-value:

* asdl_seq_GET()
* asdl_seq_GET_UNTYPED()
* asdl_seq_LEN()

They are modified to use the _Py_RVALUE() macro.
2021-11-30 15:13:55 +01:00
Victor Stinner c19c3a0961
bpo-45476: Add _Py_RVALUE() macro (GH-29860)
Add a new _Py_RVALUE() macro to prevent using an expression as an
l-value.

Replace a "(void)" cast with the _Py_RVALUE() macro in the following
macros:

* PyCell_SET()
* PyList_SET_ITEM()
* PyTuple_SET_ITEM()
* _PyGCHead_SET_FINALIZED()
* _PyGCHead_SET_NEXT()
* asdl_seq_SET()
* asdl_seq_SET_UNTYPED()

Add also parentheses around macro arguments in PyCell_SET() and
PyTuple_SET_ITEM() macros.
2021-11-30 12:14:45 +01:00
Mark Shannon 60929576e4
bpo-45786: Allocate space for frame in frame object. (GH-29729) 2021-11-29 12:34:59 +00:00
Irit Katriel c456dfafe9
bpo-45711: use exc_value instead of exc_type to determine if exc_info is valid. Add more assertions. (GH-29627) 2021-11-25 09:41:28 +00:00
Christian Heimes f4afc53bf6
bpo-45893: Add missing extern C to initconfig.h (GH-29761)
Co-authored-by: Steve Dower <steve.dower@python.org>
2021-11-24 21:12:12 +01:00
Victor Stinner 4ae26b9c1d
bpo-39026: Fix Python.h when building with Xcode (GH-29488)
Fix Python.h to build C extensions with Xcode: remove a relative
include from Include/cpython/pystate.h.
2021-11-23 18:58:57 +01:00
Mark Shannon 135cabd328
bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
* Make internal APIs that take PyFrameConstructor take a PyFunctionObject instead.

* Add reference to function to frame, borrow references to builtins and globals.

* Add COPY_FREE_VARS instruction to allow specialization of calls to inner functions.
2021-11-23 09:53:24 +00:00
Pablo Galindo Salgado fdcc46d955
bpo-45848: Allow the parser to get error lines from encoded files (GH-29646) 2021-11-20 15:36:07 +01:00
Dennis Sweeney 036fead695
bpo-45609: Specialize STORE_SUBSCR (GH-29242)
* Specialize STORE_SUBSCR for list[int], and dict[object]

* Adds _PyDict_SetItem_Take2 which consumes references to the key and values.
2021-11-19 10:30:37 +00:00
Mark Shannon 21fa7a3e8f
bpo-45829: Specialize BINARY_SUBSCR for __getitem__ implemented in Python. (GH-29592) 2021-11-18 11:02:14 +00:00
Dong-hee Na 345ba3f080
bpo-45510: Specialize BINARY_SUBTRACT (GH-29523) 2021-11-18 09:19:58 +00:00
Brandt Bucher 6a84d61c55
bpo-45636: Simplify BINARY_OP (GH-29565) 2021-11-16 05:53:57 -08:00
Mark Shannon b931077375
bpo-45753: Make recursion checks more efficient. (GH-29524)
* Uses recursion remaining, instead of recursion depth to speed up check against recursion limit.
2021-11-16 11:01:57 +00:00
Brandt Bucher 9178f533ff
bpo-45636: Merge all numeric operators (GH-29482) 2021-11-10 22:56:22 -08:00
Guido van Rossum 1cbaa505d0
bpo-45696: Deep-freeze selected modules (GH-29118)
This gains 10% or more in startup time for `python -c pass` on UNIX-ish systems.

The Makefile.pre.in generating code builds on Eric's work for bpo-45020, but the .c file generator is new.

Windows version TBD.
2021-11-10 18:01:53 -08:00
Christian Heimes cbab997efb
bpo-45723: Prepare support for autoconf 2.71 (GH-29441) 2021-11-08 19:31:14 +01:00
Pablo Galindo 4d8f2e09b6
Post 3.11.0a2 2021-11-05 21:29:33 +00:00
Pablo Galindo e2b4e4bab9
Python 3.11.0a2 2021-11-05 19:04:04 +00:00
Irit Katriel 3509b26c91
bpo-45292: [PEP 654] Update traceback display code to work with exception groups (GH-29207) 2021-11-05 09:39:18 +00:00
Mark Shannon acc89db923
bpo-45691: Make array of small ints static to fix use-after-free error. (GH-29366) 2021-11-03 16:22:32 +00:00
Eric Snow 074fa57506
bpo-45395: Make custom frozen modules additions instead of replacements. (gh-28778)
Currently custom modules (the array set on PyImport_FrozenModules) replace all the frozen stdlib modules. That can be problematic and is unlikely to be what the user wants. This change treats the custom frozen modules as additions instead. They take precedence over all other frozen modules except for those needed to bootstrap the import system. If the "code" field of an entry in the custom array is NULL then that frozen module is treated as disabled, which allows a custom entry to disable a frozen stdlib module.

This change allows us to get rid of is_essential_frozen_module() and simplifies the logic for which frozen modules should be ignored.

https://bugs.python.org/issue45395
2021-10-28 15:04:33 -06:00
Mark Shannon 4fc68560ea
Store actual ints, not pointers to them in the interpreter state. (GH-29274) 2021-10-28 17:35:43 +01:00
Mark Shannon 7f61d9d848
bpo-45256: Rationalize code around Python-to-Python calls a bit. (GH-29235) 2021-10-28 16:14:59 +01:00
Mark Shannon f291404a80
bpo-45637: Store the frame pointer in the cframe (GH-29267)
* Rename 'frame' to 'current_frame'
2021-10-28 13:59:11 +01:00
Brandt Bucher 82a662e521
bpo-44511: Improve the bytecode for class and mapping patterns (GH-26922)
* Refactor mapping patterns and speed up class patterns.

* Simplify MATCH_KEYS and MATCH_CLASS.

* Add COPY opcode.
2021-10-27 10:45:35 +01:00
Eric Snow 17c61045c5
bpo-45506: Normalize _PyPathConfig.stdlib_dir when calculated. (#29040)
The recently added PyConfig.stdlib_dir was being set with ".." entries. When __file__ was added for from modules this caused a problem on out-of-tree builds. This PR fixes that by normalizing "stdlib_dir" when it is calculated in getpath.c.

https://bugs.python.org/issue45506
2021-10-22 17:20:03 -06:00
Irit Katriel f30ad65dbf
bpo-45292: [PEP 654] add the ExceptionGroup and BaseExceptionGroup classes (GH-28569) 2021-10-23 00:13:46 +01:00
Christian Heimes 9942f42a93
bpo-45522: Allow to disable freelists on build time (GH-29056)
Freelists for object structs can now be disabled. A new ``configure``
option ``--without-freelists`` can be used to disable all freelists
except empty tuple singleton. Internal Py*_MAXFREELIST macros can now
be defined as 0 without causing compiler warnings and segfaults.

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-21 06:12:20 -07:00
Petr Viktorin 8a310dd5f4
bpo-45315: PyType_FromSpec: Copy spec->name and have the type own the memory for its name (GH-29103) 2021-10-21 11:46:20 +02:00
Ken Jin 2cbf50e812
bpo-44220: Export PyStructSequence_UnnamedField in the limited API (GH-26331) 2021-10-21 10:46:48 +02:00
Mark Shannon bc85eb7a4f
bpo-45527: Don't count cache hits, just misses. (GH-29092) 2021-10-20 20:53:48 +02:00
Mark Shannon 8863a0fcc5
bpo-44525: Specialize simple Python calls. (GH-29033) 2021-10-20 16:08:28 +01:00
Petr Viktorin 98fa3b53e2
bpo-45474: Exclude all of marshal.h if Py_LIMITED_API is defined (GH-29061)
Also, reword the What's New messages: this doesn't change the limited API, it only brings the Py_LIMITED_API macro closer to the ideal of only allowing the limited API.

Automerge-Triggered-By: GH:encukou
2021-10-20 02:32:14 -07:00
Ken Jin 3163e68c34
bpo-44525: Specialize ``CALL_FUNCTION`` for C function calls (GH-26934) 2021-10-20 00:16:36 +01:00
Victor Stinner 52af0756b2
bpo-45434: Only exclude <stdlib.h> in Python 3.11 limited C API (GH-29027)
The Python 3.11 limited C API no longer includes stdlib.h, stdio.h,
string.h and errno.h.

* Exclude Py_MEMCPY() from Python 3.11 limited C API.
* xxlimited C extension is now built with Python 3.11 limited C API.
2021-10-19 12:10:22 +02:00
Victor Stinner 5f09bb021a
bpo-35134: Add Include/cpython/longobject.h (GH-29044)
Move Include/longobject.h non-limited API to a new
Include/cpython/longobject.h header file.

Move the following definitions to the internal C API:

* _PyLong_DigitValue
* _PyLong_FormatAdvancedWriter()
* _PyLong_FormatWriter()
2021-10-19 02:04:52 +02:00
Victor Stinner aad88d33d9
bpo-35134: Split warnings.h and weakrefobject.h (GH-29042)
Split header files to move the non-limited API to Include/cpython/:

* Include/warnings.h => Include/cpython/warnings.h
* Include/weakrefobject.h => Include/cpython/weakrefobject.h

Exclude PyWeakref_GET_OBJECT() from the limited C API. It never
worked since the PyWeakReference structure is opaque in the limited C
API.

Move _PyWarnings_Init() and _PyErr_WarnUnawaitedCoroutine() to the
internal C API.
2021-10-19 01:31:57 +02:00
Victor Stinner 034f607906
bpo-43760: Rename _PyThreadState_DisableTracing() (GH-29032)
* Rename _PyThreadState_DisableTracing()
  to _PyThreadState_PauseTracing()
* Rename _PyThreadState_ResetTracing()
  to _PyThreadState_ResumeTracing()
2021-10-18 18:40:43 +02:00
Eric Snow b9cdd0fb9c
bpo-45020: Default to using frozen modules unless running from source tree. (gh-28940)
The default was "off".  Switching it to "on" means users get the benefit of frozen stdlib modules without having to do anything.  There's a special-case for running-in-source-tree, so contributors don't get surprised when their stdlib changes don't get used.

https://bugs.python.org/issue45020
2021-10-16 13:16:08 -06:00
Victor Stinner 00ffc4513d
bpo-45440: Remove pymath.c fallbacks (GH-28977)
Remove fallbacks for missing round(), copysign() and hypot() in
Python/pymath.c. Python now requires these functions to build.

These fallbacks were needed on Visual Studio 2012 and older. They are
no longer needed since Visual Stuido 2013. Python is now built with
Visual Studio 2017 or newer since Python 3.6.
2021-10-15 19:45:34 +02:00
Victor Stinner 51f8196d05
bpo-30459: Use (void) in macros setting variables (GH-28982)
Convert the result of macros setting variables to void to avoid risks
of misusing them:

* _PyGCHead_SET_NEXT()
* asdl_seq_SET()
* asdl_seq_SET_UNTYPED()
2021-10-15 19:44:35 +02:00
Victor Stinner 547d26aa08
bpo-43760: Add PyThreadState_EnterTracing() (GH-28542)
Add PyThreadState_EnterTracing() and PyThreadState_LeaveTracing()
functions to the limited C API to suspend and resume tracing and
profiling.

Add an unit test on the PyThreadState C API to _testcapi.

Add also internal _PyThreadState_DisableTracing() and
_PyThreadState_ResetTracing().
2021-10-15 16:06:30 +02:00
Victor Stinner 354c35220d
bpo-45482: Rename namespaceobject.h to pycore_namespace.h (GH-28975)
Rename Include/namespaceobject.h to
Include/internal/pycore_namespace.h.

The _testmultiphase extension is now built with the
Py_BUILD_CORE_MODULE macro defined to access _PyNamespace_Type.

object.c: remove unused "pycore_context.h" include.
2021-10-15 15:21:21 +02:00
Victor Stinner 105582e74c
bpo-45434: Remove Include/eval.h header file (GH-28973)
Move Include/eval.h content into Include/ceval.h and
Include/cpython/ceval.h, and remove Include/eval.h.
2021-10-15 13:06:05 +02:00
Victor Stinner 063abd931f
bpo-35081: Move interpreteridobject.h to Include/internal/ (GH-28969)
Move the interpreteridobject.h header file from Include/ to
Include/internal/. It only provides private functions.
2021-10-15 11:56:34 +02:00
Victor Stinner 8e5de40f90
bpo-35134: Move classobject.h to Include/cpython/ (GH-28968)
Move classobject.h, context.h, genobject.h and longintrepr.h header
files from Include/ to Include/cpython/.

Remove redundant "#ifndef Py_LIMITED_API" in context.h.

Remove explicit #include "longintrepr.h" in C files. It's not needed,
Python.h already includes it.
2021-10-15 09:46:29 +02:00
Victor Stinner 77b24ba505
bpo-35134: Move Include/cellobject.h to Include/cpython/ (GH-28964) 2021-10-15 02:39:58 +02:00
Victor Stinner 37b1d607bf
po-35134: Move Include/funcobject.h to Include/cpython/ (GH-28958)
Remove redundant "#ifndef Py_LIMITED_API" in funcobject.h.
2021-10-15 01:50:28 +02:00
Victor Stinner 7076bef8ba
bpo-45434: Remove useless space in includes (GH-28963)
Micro-optimize spaces!
2021-10-15 01:50:04 +02:00
Victor Stinner 284994762d
bpo-45434: Limited Python.h no longer includes stdio.h (GH-28960)
The <Python.h> header file no longer includes <stdio.h> if the
Py_LIMITED_API macro is defined.
2021-10-15 01:09:06 +02:00
Victor Stinner af1083e975
bpo-45474: Fix the limited C API of marshal.h (GH-28956)
Remove two functions from the limited C API:

* PyMarshal_WriteLongToFile()
* PyMarshal_WriteObjectToFile()

The PEP 384 excludes functions expecting "FILE*" from the stable ABI.

Remove also the Py_MARSHAL_VERSION macro from the limited C API.
2021-10-15 00:20:33 +02:00
Victor Stinner 0a883a76cd
bpo-35134: Add Include/cpython/floatobject.h (GH-28957)
Split Include/floatobject.h into sub-files: add
Include/cpython/floatobject.h and
Include/internal/pycore_floatobject.h.
2021-10-14 23:41:06 +02:00
Victor Stinner 3cc56c828d
bpo-45439: Move _PyObject_VectorcallTstate() to pycore_call.h (GH-28893)
* Move _PyObject_VectorcallTstate() and _PyObject_FastCallTstate() to
  pycore_call.h (internal C API).
* Convert PyObject_CallOneArg(), PyObject_Vectorcall(),
  _PyObject_FastCall() and PyVectorcall_Function() static inline
  functions to regular functions.
* Add _PyVectorcall_FunctionInline() static inline function.
* PyObject_Vectorcall(), _PyObject_FastCall(), and
  PyObject_CallOneArg() now call _PyThreadState_GET() rather
  than PyThreadState_Get().
2021-10-14 21:53:04 +02:00
Serhiy Storchaka 39aa98346d
bpo-45467: Fix IncrementalDecoder and StreamReader in the "raw-unicode-escape" codec (GH-28944)
They support now splitting escape sequences between input chunks.

Add the third parameter "final" in codecs.raw_unicode_escape_decode().
It is True by default to match the former behavior.
2021-10-14 20:04:19 +03:00
Dennis Sweeney 3b3d30e8f7
bpo-45367: Specialize BINARY_MULTIPLY (GH-28727) 2021-10-14 15:56:33 +01:00
Serhiy Storchaka c96d1546b1
bpo-45461: Fix IncrementalDecoder and StreamReader in the "unicode-escape" codec (GH-28939)
They support now splitting escape sequences between input chunks.

Add the third parameter "final" in codecs.unicode_escape_decode().
It is True by default to match the former behavior.
2021-10-14 13:17:00 +03:00
Victor Stinner 194a9526d8
bpo-45440: Require math.h isinf() to build (GH-28894)
Building Python now requires a C99 <math.h> header file providing
isinf(), isnan() and isfinite() functions.

Remove the Py_FORCE_DOUBLE() macro. It was used by the
Py_IS_INFINITY() macro.

Changes:

* Remove Py_IS_NAN(), Py_IS_INFINITY() and Py_IS_FINITE()
  in PC/pyconfig.h.
* Remove the _Py_force_double() function.
* configure no longer checks if math.h defines isinf(), isnan() and
  isfinite().
2021-10-13 23:27:50 +02:00
Victor Stinner aac29af678
bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
Include <stdlib.h> explicitly in C files.

Python.h includes <wchar.h>.
2021-10-13 19:25:53 +02:00
Victor Stinner bbe7497c5a
bpo-45434: Remove pystrhex.h header file (GH-28923)
Move Include/pystrhex.h to Include/internal/pycore_strhex.h.
The header file only contains private functions.

The following C extensions are now built with Py_BUILD_CORE_MODULE
macro defined to get access to the internal C API:

* _blake2
* _hashopenssl
* _md5
* _sha1
* _sha3
* _ssl
* binascii
2021-10-13 15:22:35 +02:00
Mark Shannon a8b9350964
bpo-45340: Don't create object dictionaries unless actually needed (GH-28802)
* Never change types' cached keys. It could invalidate inline attribute objects.

* Lazily create object dictionaries.

* Update specialization of LOAD/STORE_ATTR.

* Don't update shared keys version for deletion of value.

* Update gdb support to handle instance values.

* Rename SPLIT_KEYS opcodes to INSTANCE_VALUE.
2021-10-13 14:19:34 +01:00
Victor Stinner 97308dfcdc
bpo-45434: Move _Py_BEGIN_SUPPRESS_IPH to pycore_fileutils.h (GH-28922) 2021-10-13 15:03:35 +02:00
Victor Stinner 7cdc2a0f4b
pycore_pystate.h no longer redefines PyThreadState_GET() (GH-28921)
Redefining the PyThreadState_GET() macro in pycore_pystate.h is
useless since it doesn't affect files not including it. Either use
_PyThreadState_GET() directly, or don't use pycore_pystate.h internal
C API. For example, the _testcapi extension don't use the internal C
API, but use the public PyThreadState_Get() function instead.

Replace PyThreadState_Get() with _PyThreadState_GET(). The
_PyThreadState_GET() macro is more efficient than PyThreadState_Get()
and PyThreadState_GET() function calls which call fail with a fatal
Python error.

posixmodule.c and _ctypes extension now include <windows.h> before
pycore header files (like pycore_call.h).

_PyTraceback_Add() now uses _PyErr_Fetch()/_PyErr_Restore() instead
of PyErr_Fetch()/PyErr_Restore().

The _decimal and _xxsubinterpreters extensions are now built with the
Py_BUILD_CORE_MODULE macro defined to get access to the internal C
API.
2021-10-13 14:09:13 +02:00
Victor Stinner c63623a0a6
bpo-45434: bytearrayobject.h no longer includes <stdarg.h> (GH-28913)
bytearrayobject.h and _lzmamodule.c don't use va_list and so don't
need to include <stdarg.h>.
2021-10-13 04:37:55 +02:00
Victor Stinner 489176e428
bpo-45434: Convert Py_GETENV() macro to a function (GH-28912)
Avoid calling directly getenv() in the header file.
2021-10-13 03:39:50 +02:00
Victor Stinner d943d19172
bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
  PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
  defined to get access to internal _PyObject_CallNoArgs().
2021-10-12 08:38:19 +02:00
Victor Stinner ce3489cfdb
bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
2021-10-12 00:42:23 +02:00
Victor Stinner fb8f208a4d
bpo-45439: _PyObject_Call() only checks tp_vectorcall_offset once (GH-28890)
Add _PyVectorcall_Call() helper function.

Add "assert(PyCallable_Check(callable));" to PyVectorcall_Call(),
similar check than PyVectorcall_Function().
2021-10-12 00:18:26 +02:00
Victor Stinner 61190e092b
bpo-45412: Move copysign() define to pycore_pymath.h (GH-28889)
Move definitions of copysign(), round(), hypot(), fmod(), etc. from
pymath.h to pycore_pymath.h. These functions are not exported by
libpython and so must not be part of the C API.
2021-10-12 00:12:00 +02:00
Victor Stinner 1f316ea3b4
bpo-41123: Remove Py_UNICODE_COPY() and Py_UNICODE_FILL() (GH-28887) 2021-10-11 23:36:37 +02:00
Victor Stinner 03ea862b8a
bpo-45434: Python.h no longer includes <stdlib.h> (GH-28888) 2021-10-11 23:30:00 +02:00
Victor Stinner 7103356455
bpo-45412: Move _Py_SET_53BIT_PRECISION_START to pycore_pymath.h (GH-28882)
Move the following macros , to pycore_pymath.h (internal C API):

* _Py_SET_53BIT_PRECISION_HEADER
* _Py_SET_53BIT_PRECISION_START
* _Py_SET_53BIT_PRECISION_END

PEP 7: add braces to if and "do { ... } while (0)" in these macros.

Move also _Py_get_387controlword() and _Py_set_387controlword()
definitions to pycore_pymath.h. These functions are no longer
exported.

pystrtod.c now includes pycore_pymath.h.
2021-10-11 23:09:40 +02:00
Victor Stinner a9fe1a8e5b
bpo-45412: Update _Py_ADJUST_ERANGE1() comment (GH-28884)
Copy the comment from the removed Py_OVERFLOWED() function.
2021-10-11 23:07:41 +02:00
Victor Stinner 47717d1186
bpo-45434: Cleanup Python.h header file (GH-28883)
* Move limits.h include and UCHAR_MAX checks to pyport.h.
* Move sanitizers macros to pyport.h.
* Remove comment about <assert.h>: C extensions are built with NDEBUG
  automatically by Python.
2021-10-11 22:51:32 +02:00
Victor Stinner 2f92e2a590
bpo-45412: Remove Py_SET_ERRNO_ON_MATH_ERROR() macro (GH-28820)
Remove the following math macros using the errno variable:

* Py_ADJUST_ERANGE1()
* Py_ADJUST_ERANGE2()
* Py_OVERFLOWED()
* Py_SET_ERANGE_IF_OVERFLOW()
* Py_SET_ERRNO_ON_MATH_ERROR()

Create pycore_pymath.h internal header file.

Rename Py_ADJUST_ERANGE1() and Py_ADJUST_ERANGE2() to
_Py_ADJUST_ERANGE1() and _Py_ADJUST_ERANGE2(), and convert these
macros to static inline functions.

Move the following macros to pycore_pymath.h:

* _Py_IntegralTypeSigned()
* _Py_IntegralTypeMax()
* _Py_IntegralTypeMin()
* _Py_InIntegralTypeRange()
2021-10-11 21:00:25 +02:00
Mark Shannon fcb3d2ff63
Restore PEP 523 functionality. (GH-28871) 2021-10-11 11:34:02 +01:00
Inada Naoki ad970e8623
bpo-29410: Change the default hash algorithm to SipHash13. (GH-28752)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
Co-authored-by: Christian Heimes <christian@python.org>
2021-10-10 17:29:46 +09:00
Pablo Galindo Salgado b4903afd4d
bpo-45256: Remove the usage of the C stack in Python to Python calls (GH-28488)
Ths commit inlines calls to Python functions in the eval loop and steals all the arguments in the call from the caller for
performance.
2021-10-09 16:51:30 +01:00
Christian Clauss 8e8f752217
Fix typos in the Include directory (GH-28745) 2021-10-06 11:32:38 -07:00
Mark Shannon a7252f88d3
bpo-40116: Add insertion order bit-vector to dict values to allow dicts to share keys more freely. (GH-28520) 2021-10-06 13:19:53 +01:00
Eric Snow 08285d563e
bpo-45020: Identify which frozen modules are actually aliases. (gh-28655)
In the list of generated frozen modules at the top of Tools/scripts/freeze_modules.py, you will find that some of the modules have a different name than the module (or .py file) that is actually frozen. Let's call each case an "alias". Aliases do not come into play until we get to the (generated) list of modules in Python/frozen.c. (The tool for freezing modules, Programs/_freeze_module, is only concerned with the source file, not the module it will be used for.)

Knowledge of which frozen modules are aliases (and the identity of the original module) normally isn't important. However, this information is valuable when we go to set __file__ on frozen stdlib modules. This change updates Tools/scripts/freeze_modules.py to map aliases to the original module name (or None if not a stdlib module) in Python/frozen.c. We also add a helper function in Python/import.c to look up a frozen module's alias and add the result of that function to the frozen info returned from find_frozen().

https://bugs.python.org/issue45020
2021-10-05 11:26:37 -06:00
Pablo Galindo 444429142c
Post 3.11.0a1 2021-10-05 18:08:06 +01:00
Pablo Galindo 7c12e4835e
Python 3.11.0a1 2021-10-05 13:44:05 +01:00
Mark Shannon bd627eb7ed
bpo-43760: Check for tracing using 'bitwise or' instead of branch in dispatch. (GH-28723) 2021-10-05 11:01:11 +01:00
Victor Stinner 833fdf126c
bpo-41710: Add private _PyDeadline_Get() function (GH-28674)
Add a private C API for deadlines: add _PyDeadline_Init() and
_PyDeadline_Get() functions.

* Add _PyTime_Add() and _PyTime_Mul() functions which compute t1+t2
  and t1*t2 and clamp the result on overflow.
* _PyTime_MulDiv() now uses _PyTime_Add() and _PyTime_Mul().
2021-10-01 13:29:25 +02:00
Victor Stinner 98d2827002
bpo-41710: Fix PY_TIMEOUT_MAX on Windows (GH-28673)
WaitForSingleObject() accepts timeout in milliseconds in the range
[0; 0xFFFFFFFE] (DWORD type). INFINITE value (0xFFFFFFFF) means no
timeout. 0xFFFFFFFE milliseconds is around 49.7 days.

PY_TIMEOUT_MAX is (0xFFFFFFFE * 1000) milliseconds on Windows, around
49.7 days.

Partially revert commit 37b8294d62.
2021-10-01 13:03:03 +02:00
Victor Stinner b34dd58fee
bpo-41710: Document _PyTime_t API in pytime.h (GH-28647) 2021-09-30 11:23:03 +02:00
Victor Stinner 37b8294d62
bpo-41710: PyThread_acquire_lock_timed() clamps the timout (GH-28643)
PyThread_acquire_lock_timed() now clamps the timeout into the
[_PyTime_MIN; _PyTime_MAX] range (_PyTime_t type) if it is too large,
rather than calling Py_FatalError() which aborts the process.

PyThread_acquire_lock_timed() no longer uses
MICROSECONDS_TO_TIMESPEC() to compute sem_timedwait() argument, but
_PyTime_GetSystemClock() and _PyTime_AsTimespec_truncate().

Fix _thread.TIMEOUT_MAX value on Windows: the maximum timeout is
0x7FFFFFFF milliseconds (around 24.9 days), not 0xFFFFFFFF
milliseconds (around 49.7 days).

Set PY_TIMEOUT_MAX to 0x7FFFFFFF milliseconds, rather than 0xFFFFFFFF
milliseconds.

Fix PY_TIMEOUT_MAX overflow test: replace (us >= PY_TIMEOUT_MAX) with
(us > PY_TIMEOUT_MAX).
2021-09-30 10:16:51 +02:00
Victor Stinner d62d925823
bpo-41710: Add pytime_add() and pytime_mul() (GH-28642)
Add pytime_add() and pytime_mul() functions to pytime.c to compute
t+t2 and t*k with clamping to [_PyTime_MIN; _PyTime_MAX].

Fix pytime.h: _PyTime_FromTimeval() is not implemented on Windows.
2021-09-30 03:07:11 +02:00
Victor Stinner 09796f2f14
bpo-41710: Add _PyTime_AsTimespec_clamp() (GH-28629)
Add the _PyTime_AsTimespec_clamp() function: similar to
_PyTime_AsTimespec(), but clamp to _PyTime_t min/max and don't raise
an exception.

PyThread_acquire_lock_timed() now uses _PyTime_AsTimespec_clamp() to
remove the Py_UNREACHABLE() code path.

* Add _PyTime_AsTime_t() function.
* Add PY_TIME_T_MIN and PY_TIME_T_MAX constants.
* Replace _PyTime_AsTimeval_noraise() with _PyTime_AsTimeval_clamp().
* Add pytime_divide_round_up() function.
* Fix integer overflow in pytime_divide().
* Add pytime_divmod() function.
2021-09-30 02:11:41 +02:00
Eric Snow 0c50b8c0b8
bpo-45211: Remember the stdlib dir during startup. (gh-28586)
During runtime startup we figure out the stdlib dir but currently throw that information away. This change preserves it and exposes it via PyConfig.stdlib_dir, _Py_GetStdlibDir(), and sys._stdlib_dir.

https://bugs.python.org/issue45211
2021-09-28 12:18:28 -06:00
Eric Snow ae7839bbe8
bpo-45211: Move helpers from getpath.c to internal API. (gh-28550)
This accomplishes 2 things:

* consolidates some common code between getpath.c and getpathp.c
* makes the helpers available to code in other files

FWIW, the signature of the join_relfile() function (in fileutils.c) intentionally mirrors that of Windows' PathCchCombineEx().

Note that this change is mostly moving code around. No behavior is meant to change.

https://bugs.python.org/issue45211
2021-09-27 10:00:32 -06:00
Victor Stinner 58f8adfda3
bpo-21302: time.sleep() uses waitable timer on Windows (GH-28483)
On Windows, time.sleep() now uses a waitable timer which has a
resolution of 100 ns (10^-7 sec). Previously, it had a solution of 1
ms (10^-3 sec).

* On Windows, time.sleep() now calls PyErr_CheckSignals() before
  resetting the SIGINT event.
* Add _PyTime_As100Nanoseconds() function.
* Complete and update time.sleep() documentation.

Co-authored-by: Livius <egyszeregy@freemail.hu>
2021-09-22 16:09:30 +02:00
Mohamad Mansour 8f943ca257
[codemod] Fix non-matching bracket pairs (GH-28473)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-09-22 01:09:00 +02:00
Victor Stinner 79a3148099
bpo-45061: Detect refcount bug on empty tuple singleton (GH-28503)
Detect refcount bugs in C extensions when the empty tuple singleton
is destroyed by mistake.

Add the _Py_FatalRefcountErrorFunc() function.
2021-09-21 23:04:34 +02:00
Victor Stinner e4044e9f89
bpo-45116: Py_DEBUG ignores Py_ALWAYS_INLINE (GH-28419)
If the Py_DEBUG macro is defined, the Py_ALWAYS_INLINE macro does
nothing.
2021-09-17 22:46:38 +02:00
Victor Stinner 6b41355128
bpo-45116: Add the Py_ALWAYS_INLINE macro (GH-28390)
Add the Py_ALWAYS_INLINE macro to ask the compiler to always inline a
static inline function. The compiler can ignore it and decides to not
inline the function.
2021-09-17 14:09:14 +02:00
Mark Shannon 064464fc38
bpo-45219: Factor dictkey indexing (GH-28389) 2021-09-17 12:20:51 +01:00
Victor Stinner b49263b698
bpo-21302: Add _PyTime_AsNanoseconds() (GH-28350)
Refactor pytime.c:

* Add pytime_from_nanoseconds() and pytime_as_nanoseconds(),
  and use explicitly these functions
* Add two empty lines between functions
* PEP 7: add braces { ... }
* C99: declare variables where they are set
* Rename private functions to lowercase
* Rename error_time_t_overflow() to pytime_time_t_overflow()
* Rename win_perf_counter_frequency() to py_win_perf_counter_frequency()
* py_get_monotonic_clock(): add an assertion to detect overflow when
  mach_absolute_time() unsigned uint64_t is casted to _PyTime_t
  (signed int64_t).

_testcapi: use _PyTime_FromNanoseconds().
2021-09-15 14:26:43 +02:00
Eric Snow a65c86889e
bpo-45020: Add -X frozen_modules=[on|off] to explicitly control use of frozen modules. (gh-28320)
Currently we freeze several modules into the runtime. For each of these modules it is essential to bootstrapping the runtime that they be frozen. Any other stdlib module that we later freeze into the runtime is not essential. We can just as well import from the .py file.  This PR lets users explicitly choose which should be used, with the new "-X frozen_modules=[on|off]" CLI flag. The default is "off" for now.

https://bugs.python.org/issue45020
2021-09-14 17:31:45 -06:00
Irit Katriel c2f1e95337
bpo-45152: Add HAS_CONST macro and get_const_value() function and use… (#28262) 2021-09-14 09:53:32 +01:00
Victor Stinner cb15afcccf
bpo-39573: Py_TYPE becomes a static inline function (GH-28128)
Convert the Py_TYPE() and Py_SIZE() macros to static inline
functions. The Py_SET_TYPE() and Py_SET_SIZE() functions must now be
used to set an object type and size.
2021-09-08 11:59:13 +02:00
Yury Selivanov 2c3474a637
bpo-45123: PyAiter_Check and PyObject_GetAiter fix & rename. (GH-28194)
Fix PyAiter_Check to only check for the `__anext__` presense (not for
`__aiter__`). Rename `PyAiter_Check()` to `PyAIter_Check()`,
`PyObject_GetAiter()` -> `PyObject_GetAIter()`.

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-09-07 11:52:30 +01:00
Victor Stinner 7974c30b9f
bpo-45094: Add Py_NO_INLINE macro (GH-28140)
* Rename _Py_NO_INLINE macro to Py_NO_INLINE: make it public and
  document it.
* Sort macros in the C API documentation.
2021-09-03 16:44:02 +02:00
Mark Shannon d3eaf0cc5b
bpo-44945: Specialize BINARY_ADD (GH-27967) 2021-08-27 09:21:01 +01:00
Mark Shannon f9242d50b1
bpo-44990: Change layout of evaluation frames. "Layout B" (GH-27933)
Places the locals between the specials and stack. This is the more "natural" layout for a C struct, makes the code simpler and gives a slight speedup (~1%)
2021-08-25 13:44:20 +01:00
Irit Katriel 31ee985db8
bpo-44874: deprecate Py_TRASHCAN_SAFE_BEGIN and Py_TRASHCAN_SAFE_END (GH-27693)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-08-18 21:50:19 +02:00
Dong-hee Na c2c857b40f
bpo-44895: Introduce PYTHONDUMPREFSFILE variable for refcount dumping (GH-27767) 2021-08-18 00:52:50 +09:00
Ken Jin 96346cb6d0
bpo-44889: Specialize LOAD_METHOD with PEP 659 adaptive interpreter (GH-27722)
Adds four new instructions:

* LOAD_METHOD_ADAPTIVE
* LOAD_METHOD_CACHED
* LOAD_METHOD_MODULE
* LOAD_METHOD_CLASS
2021-08-17 15:55:55 +01:00
Hai Shi 3e2c643ae0
bpo-42035: Add PyType_GetQualName() to get a type's qualified name. (GH-27551) 2021-08-17 15:39:34 +02:00
Mark Shannon 4f51fa9e2d
bpo-44900: Add five superinstructions. (GH-27741)
* LOAD_FAST LOAD_FAST
* STORE_FAST LOAD_FAST
* LOAD_FAST LOAD_CONST
* LOAD_CONST LOAD_FAST
* STORE_FAST STORE_FAST
2021-08-16 12:23:13 +01:00
Irit Katriel 789a6af29f
bpo-44890: Fix AMD build error (GH-27740) 2021-08-12 15:20:44 +01:00
Irit Katriel 8ac0886091
bpo-44890: collect specialization stats if Py_DEBUG (GH-27731) 2021-08-12 12:15:06 +01:00
Mark Shannon 9816777861
Classify specialization failures. Provides more useful stats, with lower overhead. (GH-27701) 2021-08-10 14:53:05 +01:00
Mark Shannon ac75f6bdd4
bpo-44826: Specialize STORE_ATTR (GH-27590)
* Generalize cache names for LOAD_ATTR to allow store and delete specializations.

* Factor out specialization of attribute dictionary access.

* Specialize STORE_ATTR.
2021-08-09 10:40:21 +01:00
Mark Shannon cee67fa661
bpo-44821: Eagerly assign __dict__ for new objects. (GH-27589) 2021-08-04 16:41:14 +01:00
Mark Shannon c83919bd63
Add option to write specialization stats to files and script to summarize. (GH-27575)
* Add option to write stats to random file in a directory.

* Add script to summarize stats.
2021-08-04 11:39:52 +01:00
Mark Shannon 2116909b3e
Minor fixes to specialization stats. (GH-27457)
* Use class, not value for fail stats for BINARY_SUBSCR.

* Fix counts for unquickened instructions.
2021-07-29 20:50:03 +01:00
Irit Katriel ddd1c418c0
bpo-44725 : expose specialization stats in python (GH-27192) 2021-07-29 17:26:53 +01:00
Inada Naoki ce5e1a6809
bpo-41103: Resurrect the old buffer protocol. (GH-27437)
Revert "bpo-41103: Remove old buffer protocol support (#21117)"

This reverts commit 6f8a6ee59c.
2021-07-29 12:46:47 +02:00
Hai Shi a390ebea17
bpo-42035: Add a PyType_GetName() to get type's short name. (GH-23903) 2021-07-29 09:57:02 +02:00
Geoffrey Thomas 196998e220
closes bpo-44751: Move crypt.h include from public header to _cryptmodule (GH-27394)
Automerge-Triggered-By: GH:benjaminp
2021-07-27 15:58:12 -07:00
Mark Shannon ae0a2b7562
bpo-44590: Lazily allocate frame objects (GH-27077)
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.

* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.

* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.

* Move ownership of InterpreterFrame in generator from frame object to generator object.

* Do not create frame objects for Python calls.

* Do not create frame objects for generators.
2021-07-26 11:22:16 +01:00
Petr Viktorin a4760cc32d
bpo-42747: Remove Py_TPFLAGS_HAVE_AM_SEND and make Py_TPFLAGS_HAVE_VERSION_TAG no-op (GH-27260)
* Remove code that checks Py_TPFLAGS_HAVE_VERSION_TAG
    
    The field is always present in the type struct, as explained
    in the added comment.

* Remove Py_TPFLAGS_HAVE_AM_SEND
    
    The flag is not needed, and since it was added in 3.10 it can be removed now.
2021-07-23 06:21:11 -07:00
Serhiy Storchaka 8f50f44592
bpo-44654: Do not export the union type related symbols (GH-27223) 2021-07-18 15:55:20 +03:00
Serhiy Storchaka 0fd27375ca
bpo-44654: Refactor and clean up the union type implementation (GH-27196) 2021-07-17 22:44:10 +03:00
Ken Jin e9cd47d0e5
Remove legacy opcache structs (GH-27164) 2021-07-17 00:49:35 +09:00
Batuhan Taskaya 9af34c9351
bpo-20201: variadic arguments support for AC (GH-18609)
Implement support for `*args` in AC, and port `print()` to use it.
2021-07-16 18:43:02 +03:00
Irit Katriel 641345d636
bpo-26280: Port BINARY_SUBSCR to PEP 659 adaptive interpreter (GH-27043) 2021-07-15 13:13:12 +01:00
Mark Shannon 9487a17e3c
bpo-44207: Add an internal version number to function objects. (GH-27078) 2021-07-12 10:01:01 +01:00
Ammar Askar 4823d9a512
bpo-43950: Add option to opt-out of PEP-657 (GH-27023)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
2021-07-07 20:07:12 +01:00
Gabriele N. Tornetta 2f180ce2cb
bpo-44530: Add co_qualname field to PyCodeObject (GH-26941) 2021-07-07 12:21:51 +01:00