cpython/Tools
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
..
buildbot bpo-41173: Copy test results file from ARM worker before uploading (GH-21305) 2020-07-08 00:24:39 +01:00
c-analyzer bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928) 2022-02-08 13:39:07 -07:00
ccbench bpo-43723: Fix deprecation error caused by thread.setDaemon() (GH-25361) 2021-04-12 13:12:36 +02:00
clinic Remove an outdated comment. (GH-30010) 2021-12-09 20:44:36 +02:00
demo bpo-44740: Lowercase "internet" and "web" where appropriate. (#27378) 2021-07-27 00:11:55 +02:00
freeze bpo-46608: exclude marshalled-frozen data if deep-freezing to save 300 KB space (GH-31074) 2022-02-04 09:57:03 -08:00
gdb bpo-45256: Don't track the exact depth of each `InterpreterFrame` (GH-30372) 2022-01-05 11:30:26 +00:00
i18n Update URLs in comments and metadata to use HTTPS (GH-27458) 2021-07-30 15:54:46 +02:00
importbench Tools/importbench: Fix a misplaced stderr= (GH-12690) 2019-04-05 09:18:19 +02:00
iobench Replace KB unit with KiB (#4293) 2017-11-08 14:44:44 -08:00
msi Optimize images by IMGbot (GH-21348) 2022-02-04 15:49:43 +09:00
nuget bpo-41744: Package python.props with correct name in NuGet package (GH-22154) 2020-09-14 20:30:15 +01:00
peg_generator bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. (#31015) 2022-02-02 12:15:16 -08:00
pynche Fix typos in the Tools directory (GH-28769) 2021-10-06 10:55:16 -07:00
scripts bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928) 2022-02-08 13:39:07 -07:00
ssl bpo-46106: Update OpenSSL to 1.1.1m (GH-30211) 2021-12-21 21:20:16 -05:00
stringbench [codemod] Fix non-matching bracket pairs (GH-28473) 2021-09-22 01:09:00 +02:00
tz bpo-29919: Remove unused imports found by pyflakes (#137) 2017-03-27 16:05:26 +02:00
unicode closes bpo-45190: Update Unicode data to version 14.0.0. (GH-28336) 2021-09-14 11:00:38 -07:00
unittestgui Remove a redundant assignment in Tools/unittestgui/unittestgui.py (GH-21438) 2021-05-16 16:55:06 +01:00
wasm bpo-40280: Address more test failures on Emscripten (GH-31050) 2022-02-05 20:52:01 +01:00
README bpo-43425: Remove test2to3 from Tools (#26981) 2021-07-17 12:10:43 +02:00

README

This directory contains a number of Python programs that are useful
while building or extending Python.

buildbot        Batchfiles for running on Windows buildbot workers.

ccbench         A Python threads-based concurrency benchmark. (*)

demo            Several Python programming demos.

freeze          Create a stand-alone executable from a Python program.

gdb             Python code to be run inside gdb, to make it easier to
                debug Python itself (by David Malcolm).

i18n            Tools for internationalization. pygettext.py
                parses Python source code and generates .pot files,
                and msgfmt.py generates a binary message catalog
                from a catalog in text format.

iobench         Benchmark for the new Python I/O system. (*)

msi             Support for packaging Python as an MSI package on Windows.

parser          Un-parsing tool to generate code from an AST.

peg_generator   PEG-based parser generator (pegen) used for new parser.

pynche          A Tkinter-based color editor.

scripts         A number of useful single-file programs, e.g. tabnanny.py
                by Tim Peters, which checks for inconsistent mixing of
                tabs and spaces, and 2to3, which converts Python 2 code
                to Python 3 code.

stringbench     A suite of micro-benchmarks for various operations on
                strings (both 8-bit and unicode). (*)

unicode         Tools for generating unicodedata and codecs from unicode.org
                and other mapping files (by Fredrik Lundh, Marc-Andre Lemburg
                and Martin von Loewis).

unittestgui     A Tkinter based GUI test runner for unittest, with test
                discovery.


(*) A generic benchmark suite is maintained separately at https://github.com/python/performance