Commit Graph

216 Commits

Author SHA1 Message Date
Pablo Galindo Salgado 6d791a9736
gh-96143: Allow Linux perf profiler to see Python calls (GH-96123)
⚠️  ⚠️ Note for reviewers, hackers and fellow systems/low-level/compiler engineers ⚠️ ⚠️ 

If you have a lot of experience with this kind of shenanigans and want to improve the **first** version, **please make a PR against my branch** or **reach out by email** or **suggest code changes directly on GitHub**. 

If you have any **refinements or optimizations** please, wait until the first version is merged before starting hacking or proposing those so we can keep this PR productive.
2022-08-30 10:11:18 -07:00
Mark Shannon a4a9f2e879
GH-96177: Move GIL and eval breaker code out of ceval.c into ceval_gil.c. (GH-96204) 2022-08-24 14:21:01 +01:00
Aivars Kalvāns 8c88e360e7
gh-95005: Replace PyAccu with PyUnicodeWriter (gh-95006) 2022-07-27 17:43:34 +09:00
Kumar Aditya 71697664d7
GH-90699: Move generated static initializer to pycore_runtime_generated.h (GH-94051) 2022-07-07 13:04:05 -07:00
Dennis Sweeney 5fcfdd87c9
GH-91432: Specialize FOR_ITER (GH-91713)
* Adds FOR_ITER_LIST and FOR_ITER_RANGE specializations.

* Adds _PyLong_AssignValue() internal function to avoid temporary boxing of ints.
2022-06-21 11:19:26 +01:00
Victor Stinner 27b9894033
gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938)
Move the follow functions and type from frameobject.h to pyframe.h,
so the standard <Python.h> provide frame getter functions:

* PyFrame_Check()
* PyFrame_GetBack()
* PyFrame_GetBuiltins()
* PyFrame_GetGenerator()
* PyFrame_GetGlobals()
* PyFrame_GetLasti()
* PyFrame_GetLocals()
* PyFrame_Type

Remove #include "frameobject.h" from many C files. It's no longer
needed.
2022-06-19 12:02:33 +02:00
Ken Jin a51742ab82
gh-93911: Specialize `LOAD_ATTR_PROPERTY` (GH-93912) 2022-06-17 23:13:17 +08:00
Mark Shannon fa2b8b75eb
Improve object stats (#92845)
* Add incref/decref stats

* Show ratios for allocation in summary
2022-05-16 14:35:11 +01:00
Victor Stinner da5727a120
gh-92651: Remove the Include/token.h header file (#92652)
Remove the token.h header file. There was never any public tokenizer
C API. The token.h header file was only designed to be used by Python
internals.

Move Include/token.h to Include/internal/pycore_token.h. Including
this header file now requires that the Py_BUILD_CORE macro is
defined. It no longer checks for the Py_LIMITED_API macro.

Rename functions:

* PyToken_OneChar() => _PyToken_OneChar()
* PyToken_TwoChars() => _PyToken_TwoChars()
* PyToken_ThreeChars() => _PyToken_ThreeChars()
2022-05-11 23:22:50 +02:00
Victor Stinner 20cc695286
gh-64783: Fix signal.NSIG value on FreeBSD (#91929)
Fix signal.NSIG value on FreeBSD to accept signal numbers greater
than 32, like signal.SIGRTMIN and signal.SIGRTMAX.

* Add Py_NSIG constant.
* Add pycore_signal.h internal header file.
* _Py_Sigset_Converter() now includes the range of valid signals in
  the error message.
2022-04-26 00:13:31 +02:00
Victor Stinner 8a4e519e78
gh-79315: Add Include/cpython/pythread.h header (#91798) 2022-04-21 23:00:42 +02:00
Victor Stinner 6f9addb5ba
gh-79315: Add Include/cpython/modsupport.h header (#91797) 2022-04-21 23:00:26 +02:00
Victor Stinner 85addfb9c6
bpo-35134: Remove the Include/code.h header file (GH-32385)
Remove the Include/code.h header file. C extensions should only
include the main <Python.h> header file.

Python.h includes directly Include/cpython/code.h instead.
2022-04-07 02:29:52 +02:00
Victor Stinner 5c4d1f6e0e
bpo-35134: Add Include/cpython/setobject.h header (GH-32384) 2022-04-07 01:26:24 +02:00
Victor Stinner ca219f6dfc
bpo-35134: Add Include/cpython/complexobject.h header (GH-32383)
Move the private _PyComplex_FormatAdvancedWriter() function to the
internal C API. This function is no longer exported.
2022-04-07 01:05:27 +02:00
Jeremy Kloth 944f09adfc
bpo-47230: Silence compiler warnings on Windows from zlib 1.2.12 (GH-32337) 2022-04-05 20:03:17 +01:00
Serhiy Storchaka 1578f06c1c
bpo-47152: Move sources of the _sre module into a subdirectory (GH-32290) 2022-04-04 10:53:26 +03:00
Gregory P. Smith 9d1c4d69db
bpo-38256: Fix binascii.crc32() when inputs are 4+GiB (GH-32000)
When compiled with `USE_ZLIB_CRC32` defined (`configure` sets this on POSIX systems), `binascii.crc32(...)` failed to compute the correct value when the input data was >= 4GiB. Because the zlib crc32 API is limited to a 32-bit length.

This lines it up with the `zlib.crc32(...)` implementation that doesn't have that flaw.

**Performance:** This also adopts the same GIL releasing for larger inputs logic that `zlib.crc32` has, and causes the Windows build to always use zlib's crc32 instead of our slow C code as zlib is a required build dependency on Windows.
2022-03-20 12:28:15 -07:00
Brandt Bucher 0f41aac109
bpo-46841: Use *inline* caching for `BINARY_OP` (GH-31543) 2022-02-25 12:11:34 +00:00
Victor Stinner ec091bd47e
bpo-45459: Add pytypedefs.h header file (GH-31527)
Move forward declarations of Python C API types to a new pytypedefs.h
header file to solve interdependency issues between header files.

pytypedefs.h contains forward declarations of the following types:

* PyCodeObject
* PyFrameObject
* PyGetSetDef
* PyInterpreterState
* PyLongObject
* PyMemberDef
* PyMethodDef
* PyModuleDef
* PyObject
* PyThreadState
* PyTypeObject
2022-02-24 17:07:12 +01:00
Victor Stinner 66b3cd7063
bpo-45459: Rename buffer.h to pybuffer.h (#31201)
Rename Include/buffer.h header file to Include/pybuffer.h to avoid
conflicts with projects having an existing "buffer.h" header file.

* Incude pybuffer.h before object.h in Python.h.
* Remove #include "buffer.h" from Include/cpython/object.h.
* Add a forward declaration of the PyObject type in pybuffer.h to fix
  an inter-dependency issue.
2022-02-22 23:11:48 +01:00
Victor Stinner d4a85f104b
bpo-35134: Add Include/cpython/descrobject.h (GH-30923)
Move Include/descrobject.h non-limited API to a new
Include/cpython/descrobject.h header file.
2022-01-26 17:32:47 +01:00
Kumar Aditya ef3ef6fa43
bpo-46429: Merge all deepfrozen files into one (GH-30572) 2022-01-20 08:38:39 -08:00
Eric Snow bc02eac9d2
bpo-46370: Move the static initializer for _PyRuntime to its own header file. (gh-30587)
https://bugs.python.org/issue46370
2022-01-13 15:54:36 -07:00
Eric Snow 121f1f893a
bpo-45953: Statically initialize the small ints. (gh-30092)
The array of small PyLong objects has been statically declared. Here I also statically initialize them. Consequently they are no longer initialized dynamically during runtime init.

I've also moved them under a new sub-struct in _PyRuntimeState, in preparation for static allocation and initialization of other global objects.

https://bugs.python.org/issue45953
2021-12-13 18:04:05 -07:00
Christian Heimes 16638a4bdb
bpo-45654: No need to freeze types (GH-30028) 2021-12-10 19:09:09 +01:00
neonene 3f398a77d3
bpo-45582: Fix test_embed failure during a PGO build on Windows (GH-30014)
This defines VPATH differently in PGO instrumentation builds, to account for a different default output directory. It also adds sys._vpath on Windows to make the value available to sysconfig so that it can be used in tests.
2021-12-10 17:13:55 +00:00
Eric Snow c8749b5783
bpo-46008: Make runtime-global object/type lifecycle functions and state consistent. (gh-29998)
This change is strictly renames and moving code around.  It helps in the following ways:

* ensures type-related init functions focus strictly on one of the three aspects (state, objects, types)
* passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter
* consistent naming conventions help make what's going on more clear
* keeping API related to a type in the corresponding header file makes it more obvious where to look for it

https://bugs.python.org/issue46008
2021-12-09 12:59:26 -07:00
Kumar Aditya 44b0e76f2a
bpo-45654: Freeze the runpy module and stuff it imports (GH-29903) 2021-12-09 08:51:09 -08:00
Crowthebird d9301703fb
bpo-45816: Support building with VS 2022 (v143 toolset) on Windows (GH-29577) 2021-12-03 19:47:38 +00:00
Steve Dower 99fcf15052
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
2021-12-03 00:08:42 +00:00
Mark Shannon 135cabd328
bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
* Make internal APIs that take PyFrameConstructor take a PyFunctionObject instead.

* Add reference to function to frame, borrow references to builtins and globals.

* Add COPY_FREE_VARS instruction to allow specialization of calls to inner functions.
2021-11-23 09:53:24 +00:00
Guido van Rossum 1037ca5a8e
bpo-45850: Implement deep-freeze on Windows (#29648)
Implement changes to build with deep-frozen modules on Windows.
Note that we now require Python 3.10 as the "bootstrap" or "host" Python.
This causes a modest startup speed (around 7%) on Windows.
2021-11-22 10:09:48 -08:00
Pablo Galindo Salgado c9c4444d9f
Refactor parser compilation units into specific components (GH-29676) 2021-11-21 01:08:50 +00:00
Steve Dower a4774f42e3
bpo-45720: Drop references to shlwapi.dll on Windows (GH-29417) 2021-11-05 23:06:45 +00:00
Christian Heimes fa26245a1c
bpo-45548: Remove _math.c workarounds for pre-C99 libm (GH-29179)
The :mod:`math` and :mod:`cmath` implementation now require a C99 compatible
``libm`` and no longer ship with workarounds for missing acosh, asinh,
expm1, and log1p functions.

The changeset also removes ``_math.c`` and moves the last remaining
workaround into ``_math.h``. This simplifies static builds with
``Modules/Setup`` and resolves symbol conflicts.

Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
Co-authored-by: Brett Cannon <brett@python.org>
Signed-off-by: Christian Heimes <christian@python.org>
2021-10-25 01:25:27 -07:00
Victor Stinner 5f09bb021a
bpo-35134: Add Include/cpython/longobject.h (GH-29044)
Move Include/longobject.h non-limited API to a new
Include/cpython/longobject.h header file.

Move the following definitions to the internal C API:

* _PyLong_DigitValue
* _PyLong_FormatAdvancedWriter()
* _PyLong_FormatWriter()
2021-10-19 02:04:52 +02:00
Victor Stinner aad88d33d9
bpo-35134: Split warnings.h and weakrefobject.h (GH-29042)
Split header files to move the non-limited API to Include/cpython/:

* Include/warnings.h => Include/cpython/warnings.h
* Include/weakrefobject.h => Include/cpython/weakrefobject.h

Exclude PyWeakref_GET_OBJECT() from the limited C API. It never
worked since the PyWeakReference structure is opaque in the limited C
API.

Move _PyWarnings_Init() and _PyErr_WarnUnawaitedCoroutine() to the
internal C API.
2021-10-19 01:31:57 +02:00
Victor Stinner 354c35220d
bpo-45482: Rename namespaceobject.h to pycore_namespace.h (GH-28975)
Rename Include/namespaceobject.h to
Include/internal/pycore_namespace.h.

The _testmultiphase extension is now built with the
Py_BUILD_CORE_MODULE macro defined to access _PyNamespace_Type.

object.c: remove unused "pycore_context.h" include.
2021-10-15 15:21:21 +02:00
Victor Stinner 105582e74c
bpo-45434: Remove Include/eval.h header file (GH-28973)
Move Include/eval.h content into Include/ceval.h and
Include/cpython/ceval.h, and remove Include/eval.h.
2021-10-15 13:06:05 +02:00
Victor Stinner 063abd931f
bpo-35081: Move interpreteridobject.h to Include/internal/ (GH-28969)
Move the interpreteridobject.h header file from Include/ to
Include/internal/. It only provides private functions.
2021-10-15 11:56:34 +02:00
Victor Stinner 8e5de40f90
bpo-35134: Move classobject.h to Include/cpython/ (GH-28968)
Move classobject.h, context.h, genobject.h and longintrepr.h header
files from Include/ to Include/cpython/.

Remove redundant "#ifndef Py_LIMITED_API" in context.h.

Remove explicit #include "longintrepr.h" in C files. It's not needed,
Python.h already includes it.
2021-10-15 09:46:29 +02:00
Victor Stinner 77b24ba505
bpo-35134: Move Include/cellobject.h to Include/cpython/ (GH-28964) 2021-10-15 02:39:58 +02:00
Victor Stinner 37b1d607bf
po-35134: Move Include/funcobject.h to Include/cpython/ (GH-28958)
Remove redundant "#ifndef Py_LIMITED_API" in funcobject.h.
2021-10-15 01:50:28 +02:00
Victor Stinner 0a883a76cd
bpo-35134: Add Include/cpython/floatobject.h (GH-28957)
Split Include/floatobject.h into sub-files: add
Include/cpython/floatobject.h and
Include/internal/pycore_floatobject.h.
2021-10-14 23:41:06 +02:00
Victor Stinner bbe7497c5a
bpo-45434: Remove pystrhex.h header file (GH-28923)
Move Include/pystrhex.h to Include/internal/pycore_strhex.h.
The header file only contains private functions.

The following C extensions are now built with Py_BUILD_CORE_MODULE
macro defined to get access to the internal C API:

* _blake2
* _hashopenssl
* _md5
* _sha1
* _sha3
* _ssl
* binascii
2021-10-13 15:22:35 +02:00
Pablo Galindo Salgado a24676bedc
Add tests for the C tokenizer and expose it as a private module (GH-27924) 2021-08-24 17:50:05 +01:00
Mark Shannon ae0a2b7562
bpo-44590: Lazily allocate frame objects (GH-27077)
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.

* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.

* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.

* Move ownership of InterpreterFrame in generator from frame object to generator object.

* Do not create frame objects for Python calls.

* Do not create frame objects for generators.
2021-07-26 11:22:16 +01:00
Dong-hee Na 906fe47083
bpo-44611: Use BCryptGenRandom instead of CryptGenRandom on Windows (GH-27168) 2021-07-23 23:04:30 +09:00
Yurii Karabas 96c4cbd96c
bpo-44353: Implement typing.NewType __call__ method in C (#27262)
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Denis Laxalde <denis@laxalde.org>
2021-07-22 23:06:54 +02:00