Commit Graph

82 Commits

Author SHA1 Message Date
Eric Snow 87154d8dd8
gh-94673: Add Per-Interpreter tp_subclasses for Static Builtin Types (gh-95301) 2022-08-04 19:26:59 -06:00
Eric Snow 4a1dd73431
gh-94673: Add _PyStaticType_InitBuiltin() (#95152)
This is the first of several precursors to storing tp_subclasses (and tp_weaklist) on the interpreter state for static builtin types.

We do the following:

* add `_PyStaticType_InitBuiltin()`
* add `_Py_TPFLAGS_STATIC_BUILTIN`
* set it on all static builtin types in `_PyStaticType_InitBuiltin()`
* shuffle some code around to be able to use _PyStaticType_InitBuiltin()
    * rename `_PyStructSequence_InitType()` to `_PyStructSequence_InitBuiltinWithFlags()`
    * add `_PyStructSequence_InitBuiltin()`.
2022-07-25 12:47:31 -06:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Victor Stinner 9a24127113
bpo-46417: _PyStructSequence_FiniType() updates _Py_RefTotal (GH-30988) 2022-01-28 14:08:32 +01:00
Victor Stinner ac1f152421
bpo-46417: Use _PyType_CAST() in Objects directory (GH-30764) 2022-01-21 23:33:43 +01:00
Victor Stinner 1781d55eb3
bpo-46417: _curses uses PyStructSequence_NewType() (GH-30736)
The _curses module now creates its ncurses_version type as a heap
type using PyStructSequence_NewType(), rather than using a static
type.

* Move _PyStructSequence_FiniType() definition to pycore_structseq.h.
* test.pythoninfo: log curses.ncurses_version.
2022-01-21 03:30:20 +01:00
Victor Stinner e9e3eab0b8
bpo-46417: Finalize structseq types at exit (GH-30645)
Add _PyStructSequence_FiniType() and _PyStaticType_Dealloc()
functions to finalize a structseq static type in Py_Finalize().
Currrently, these functions do nothing if Python is built in release
mode.

Clear static types:

* AsyncGenHooksType: sys.set_asyncgen_hooks()
* FlagsType: sys.flags
* FloatInfoType: sys.float_info
* Hash_InfoType: sys.hash_info
* Int_InfoType: sys.int_info
* ThreadInfoType: sys.thread_info
* UnraisableHookArgsType: sys.unraisablehook
* VersionInfoType: sys.version
* WindowsVersionType: sys.getwindowsversion()
2022-01-21 01:42:25 +01:00
Eric Snow c8749b5783
bpo-46008: Make runtime-global object/type lifecycle functions and state consistent. (gh-29998)
This change is strictly renames and moving code around.  It helps in the following ways:

* ensures type-related init functions focus strictly on one of the three aspects (state, objects, types)
* passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter
* consistent naming conventions help make what's going on more clear
* keeping API related to a type in the corresponding header file makes it more obvious where to look for it

https://bugs.python.org/issue46008
2021-12-09 12:59:26 -07:00
Pablo Galindo c2931d31f8
bpo-43916: Move the _PyStructSequence_InitType function to the internal API (GH-25854) 2021-05-03 15:50:24 +01:00
Victor Stinner 3bb09947ec
bpo-43916: Add Py_TPFLAGS_DISALLOW_INSTANTIATION type flag (GH-25721)
Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow
creating type instances: set tp_new to NULL and don't create the
"__new__" key in the type dictionary.

The flag is set automatically on static types if tp_base is NULL or
&PyBaseObject_Type and tp_new is NULL.

Use the flag on the following types:

* _curses.ncurses_version type
* _curses_panel.panel
* _tkinter.Tcl_Obj
* _tkinter.tkapp
* _tkinter.tktimertoken
* _xxsubinterpretersmodule.ChannelID
* sys.flags type
* sys.getwindowsversion() type
* sys.version_info type

Update MyStr example in the C API documentation to use
Py_TPFLAGS_DISALLOW_INSTANTIATION.

Add _PyStructSequence_InitType() function to create a structseq type
with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set.

type_new() calls _PyType_CheckConsistency() at exit.
2021-04-30 12:46:15 +02:00
Victor Stinner 442ad74fc2
bpo-43687: Py_Initialize() creates singletons earlier (GH-25147)
Reorganize pycore_interp_init() to initialize singletons before the
the first PyType_Ready() call. Fix an issue when Python is configured
using --without-doc-strings.
2021-04-02 15:28:13 +02:00
Pablo Galindo 0632b1012d
bpo-42128: Add __match_args__ to structseq-based classes (GH-24732) 2021-03-04 00:03:44 +00:00
Jozef Grajciar c994ffe695
bpo-11717: fix ssize_t redefinition error when targeting 32bit Windows app (GH-24479) 2021-03-01 11:18:33 +00:00
Victor Stinner 00d7abd7ef
bpo-42519: Replace PyMem_MALLOC() with PyMem_Malloc() (GH-23586)
No longer use deprecated aliases to functions:

* Replace PyMem_MALLOC() with PyMem_Malloc()
* Replace PyMem_REALLOC() with PyMem_Realloc()
* Replace PyMem_FREE() with PyMem_Free()
* Replace PyMem_Del() with PyMem_Free()
* Replace PyMem_DEL() with PyMem_Free()

Modify also the PyMem_DEL() macro to use directly PyMem_Free().
2020-12-01 09:56:42 +01:00
Serhiy Storchaka 686c203cd4
bpo-42423: Accept single base class in PyType_FromModuleAndSpec() (GH-23441) 2020-11-22 13:25:02 +02:00
Serhiy Storchaka fb5db7ec58
bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId. (GH-22648)
These functions are considered not safe because they suppress all internal errors
and can return wrong result.  PyDict_GetItemString and _PyDict_GetItemId can
also silence current exception in rare cases.

Remove no longer used _PyDict_GetItemId.
Add _PyDict_ContainsId and rename _PyDict_Contains into
_PyDict_Contains_KnownHash.
2020-10-26 08:43:39 +02:00
Victor Stinner 97d15ae1d8
bpo-40170: Use inline _PyType_HasFeature() function (GH-22375)
Use _PyType_HasFeature() in the _io module and in structseq
implementation. Replace PyType_HasFeature() opaque function call with
_PyType_HasFeature() inlined function.
2020-09-23 14:08:38 +02:00
Victor Stinner 384621c42f
bpo-41078: Rename pycore_tupleobject.h to pycore_tuple.h (GH-21056) 2020-06-22 17:27:35 +02:00
Pablo Galindo 1cf15af9a6 bpo-40217: Ensure Py_VISIT(Py_TYPE(self)) is always called for PyType_FromSpec types (reverts GH-19414) (GH-20264)
Heap types now always visit the type in tp_traverse. See added docs for details.

This reverts commit 0169d3003b.

Automerge-Triggered-By: @encukou
2020-05-27 02:03:38 -07:00
Victor Stinner 4a21e57fe5
bpo-40268: Remove unused structmember.h includes (GH-19530)
If only offsetof() is needed: include stddef.h instead.

When structmember.h is used, add a comment explaining that
PyMemberDef is used.
2020-04-15 02:35:41 +02:00
Victor Stinner 60ac6ed557
bpo-39573: Use Py_SET_SIZE() function (GH-18402)
Replace direct acccess to PyVarObject.ob_size with usage of
the Py_SET_SIZE() function.
2020-02-07 23:18:08 +01:00
Serhiy Storchaka bd44a7ead9
bpo-38650: Constify PyStructSequence_UnnamedField. (GH-17005)
Make it a constant and referring to a constant string.
2019-11-16 18:55:29 +02:00
Serhiy Storchaka 279f44678c
bpo-37206: Unrepresentable default values no longer represented as None. (GH-13933)
In ArgumentClinic, value "NULL" should now be used only for unrepresentable default values
(like in the optional third parameter of getattr). "None" should be used if None is accepted
as argument and passing None has the same effect as not passing the argument at all.
2019-09-14 12:24:05 +03:00
Raymond Hettinger 4210ad5ebd
bpo-38096: Complete the "structseq" and "named tuple" cleanup (GH-16010) 2019-09-12 07:56:28 -07:00
Pablo Galindo 3caf4de6f0
Call PyObject_GC_UnTrack in structseq dealloc (GH-13751) 2019-06-02 21:52:49 +01:00
Pablo Galindo 7ffcf848df
bpo-37126: Allow structseq objects to be tracked by the GC (GH-13729) 2019-06-02 15:45:13 +01:00
Eddie Elizondo 364f0b0f19 bpo-35810: Incref heap-allocated types in PyObject_Init (GH-11661)
* Incref heap-allocated types in PyObject_Init
* Add documentation and porting notes to What's New
2019-03-27 12:52:18 +01:00
Zackery Spytz 93e8012f2c bpo-36398: Fix a possible crash in structseq_repr(). (GH-12492)
If the first PyUnicode_DecodeUTF8() call fails in structseq_repr(),
_PyUnicodeWriter_Dealloc() will be called on an uninitialized
_PyUnicodeWriter.
2019-03-22 09:24:34 +02:00
Victor Stinner c70ab02df2
bpo-36365: Rewrite structseq_repr() using _PyUnicodeWriter (GH-12440)
No longer limit repr(structseq) to 512 bytes. Use _PyUnicodeWriter
for better performance and to write directly Unicode rather than
encoding repr() value to UTF-8 and then decoding from UTF-8.
2019-03-20 00:05:51 +01:00
Serhiy Storchaka d53fe5f407
bpo-36254: Fix invalid uses of %d in format strings in C. (GH-12264) 2019-03-13 22:59:55 +02:00
Sergey Fedoseev 234531b446 bpo-36030: Add _PyTuple_FromArray() function (GH-11954) 2019-02-25 17:59:12 +01:00
Victor Stinner 989052047e
Fix compiler warning in structseq_repr() (GH-10841)
Replace strncpy() with memcpy() in structseq_repr() to fix the
following compiler warning:

Objects/structseq.c:187:5: warning: 'strncpy' specified bound depends on the length of the source argument [-Wstringop-overflow=]
     strncpy(pbuf, typ->tp_name, len);
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Objects/structseq.c:185:11: note: length computed here
     len = strlen(typ->tp_name) > TYPE_MAXSIZE ? TYPE_MAXSIZE :

The function writes the terminating NUL byte later.
2018-12-01 02:46:40 +01:00
Pablo Galindo bfb855bef6
bpo-34784: Implement correct cleanup in PyStructSequence new implementation (GH-10536)
PyTuple_Pack can fail and return NULL. If this happens, then PyType_FromSpecWithBases will incorrectly create a new type without bases. Also, it will crash on the Py_DECREF that follows. Also free members and type in error conditions.
2018-11-15 02:03:57 +00:00
Eddie Elizondo 474eedfb3d bpo-34784: Fix PyStructSequence_NewType with heap-allocated StructSequence (GH-9665) 2018-11-13 13:09:31 +01:00
Siddhesh Poyarekar 55edd0c185 bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS. (GH-6030)
METH_NOARGS functions need only a single argument but they are cast
into a PyCFunction, which takes two arguments.  This triggers an
invalid function cast warning in gcc8 due to the argument mismatch.
Fix this by adding a dummy unused argument.
2018-04-29 21:59:33 +03:00
Serhiy Storchaka 18b250f844 bpo-29793: Convert some builtin types constructors to Argument Clinic. (#615) 2017-03-19 08:51:07 +02:00
Serhiy Storchaka 007d7ff73f Issue #28761: The fields name and doc of structures PyMemberDef, PyGetSetDef,
PyStructSequence_Field, PyStructSequence_Desc, and wrapperbase are now of
type "const char *" rather of "char *".
2016-11-22 07:58:08 +02:00
Serhiy Storchaka 85b0f5beb1 Added the const qualifier to char* variables that refer to readonly internal
UTF-8 represenatation of Unicode objects.
2016-11-20 10:16:47 +02:00
Serhiy Storchaka 06515833fe Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Serhiy Storchaka 2d06e84455 Issue #25923: Added the const qualifier to static constant arrays. 2015-12-25 19:53:18 +02:00
Serhiy Storchaka 56f6e76c68 Issue #15989: Fixed some scarcely probable integer overflows.
It is very unlikely that they can occur in real code for now.
2015-09-06 21:25:30 +03:00
Victor Stinner 1c8f059019 Issue #18520: Add a new PyStructSequence_InitType2() function, same than
PyStructSequence_InitType() except that it has a return value (0 on success,
-1 on error).

 * PyStructSequence_InitType2() now raises MemoryError on memory allocation failure
 * Fix also some calls to PyDict_SetItemString(): handle error
2013-07-22 22:24:54 +02:00
Victor Stinner 9a146eeadb Issue #18408: Fix structseq_reduce(), handle PyDict_SetItemString() failure 2013-07-17 13:41:39 +02:00
Victor Stinner 26f91999b4 Close #18469: Replace PyDict_GetItemString() with _PyDict_GetItemId() in structseq.c
_PyDict_GetItemId() is more efficient: it only builds the Unicode string once.
Identifiers (dictionary keys) are now created at Python initialization, and if
the creation failed, Python does exit with a fatal error.

Before, PyDict_GetItemString() failure was not handled: structseq_new() could
call PyObject_GC_NewVar() with a negative size, and structseq_dealloc() could
also crash.
2013-07-17 01:22:45 +02:00
Stefan Krah 6b962860e2 Check for NULL return value in PyStructSequence_NewType(). Found by Coverity. 2012-08-19 11:20:41 +02:00
Antoine Pitrou 4b3c7846c9 Fix indentation 2012-02-15 02:52:58 +01:00
Antoine Pitrou 37784ba5c0 Issue #13020: Fix a reference leak when allocating a structsequence object fails.
Patch by Suman Saha.
2012-02-15 02:51:43 +01:00
Martin v. Löwis 4d0d471a80 Merge branches/pep-0384. 2010-12-03 20:14:31 +00:00
Georg Brandl 71c23d4473 Include structseq.h in Python.h, and remove now-redundant includes in individual sources. 2010-11-30 09:30:54 +00:00
Benjamin Peterson d02441ea2f fix repr of complicated structseqs #9206 2010-07-08 22:33:03 +00:00