* Add an InternalDocs file describing how interning should work and how to use it.
* Add internal functions to *explicitly* request what kind of interning is done:
- `_PyUnicode_InternMortal`
- `_PyUnicode_InternImmortal`
- `_PyUnicode_InternStatic`
* Switch uses of `PyUnicode_InternInPlace` to those.
* Disallow using `_Py_SetImmortal` on strings directly.
You should use `_PyUnicode_InternImmortal` instead:
- Strings should be interned before immortalization, otherwise you're possibly
interning a immortalizing copy.
- `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
`SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
backports, as they are now part of public API and version-specific ABI.
* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.
* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
- `_Py_ID`
- `_Py_STR` (including the empty string)
- one-character latin-1 singletons
Now, when you intern a singleton, that exact singleton will be interned.
* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).
* Intern `_Py_STR` singletons at startup.
* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.
* Beef up the tests. Cover internal details (marked with `@cpython_only`).
* Add lots of assertions
Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
When the _Py_SINGLETON() is used, Argument Clinic now adds an
explicit "pycore_runtime.h" include to get the macro. Previously, the
macro may or may not be included indirectly by another include.
_PyArg_Parser holds static global data generated for modules by Argument Clinic. The _PyArg_Parser.kwtuple field is a tuple object, even though it's stored within a static global. In some cases the tuple is statically allocated and thus it's okay that it gets shared by multiple interpreters. However, in other cases the tuple is set lazily, allocated from the heap using the active interprepreter at the point the tuple is needed.
This is a problem once that interpreter is destroyed since _PyArg_Parser.kwtuple becomes at dangling pointer, leading to crashes. It isn't a problem if the tuple is allocated under the main interpreter, since its lifetime is bound to the lifetime of the runtime. The solution here is to temporarily switch to the main interpreter. The alternative would be to always statically allocate the tuple.
This change also fixes a bug where only the most recent parser was added to the global linked list.
* Move ifndef_symbols, includes and add_include() from Clinic to
Codegen. Add a 'codegen' (Codegen) attribute to Clinic.
* Remove libclinic.crenderdata module: move code to libclinic.codegen.
* BlockPrinter.print_block(): remove unused 'limited_capi' argument.
Remove also 'core_includes' parameter.
* Add get_includes() methods.
* Make Codegen.ifndef_symbols private.
* Make Codegen.includes private.
* Make CConverter.includes private.
Add libclinic.clanguage module and move the following classes and
functions there:
* CLanguage
* declare_parser()
Add libclinic.codegen and move the following classes there:
* BlockPrinter
* BufferSeries
* Destination
Move the following functions to libclinic.function:
* permute_left_option_groups()
* permute_optional_groups()
* permute_right_option_groups()
* Add a new create_parser_namespace() function for
PythonParser to pass objects to executed code.
* In run_clinic(), list converters using 'converters' and
'return_converters' dictionarties.
* test_clinic: add 'object()' return converter.
* Use also create_parser_namespace() in eval_ast_expr().
Co-authored-by: Erlend E. Aasland <erlend@python.org>
* Move Block and BlockParser classes to a new libclinic.block_parser
module.
* Move Language and PythonLanguage classes to a new
libclinic.language module.
The fildes converter of Argument Clinic now always call
PyObject_AsFileDescriptor(), not only for the limited C API.
The _PyLong_FileDescriptor_Converter() converter stays as a fallback
when PyObject_AsFileDescriptor() cannot be used.
* Move param guard to param state machine
* Override return converter during parsing
* Don't use a custom type slot return converter; instead
special case type slot functions during generation.
Refactor state_modulename_name() of the parsing state machine, by
adding helpers for the sections that deal with ...:
1. parsing the function name
2. normalizing "function kind"
3. dealing with cloned functions
4. resolving return converters
5. adding the function to the DSL parser
Make it possible for a converter to have multiple includes, by collecting
them in a list on the converter instance. This implies converter includes
are added during template generation, so we have to add them to the
clinic instance at the end of the template generation instead of in the
beginning.
Rework error handling in the C preprocessor helper. Instead of monkey-
patching the cpp.Monitor.fail() method from within clinic.py, rewrite
cpp.py to use a subclass of the ClinicError exception. As a side-effect,
ClinicError is moved into Tools/clinic/libclinic/errors.py.
Yak-shaving in preparation for putting cpp.py into libclinic.
Move the following global helpers into libclinic:
- format_escape()
- normalize_snippet()
- wrap_declarations()
Also move strip_leading_and_trailing_blank_lines() and make it internal to libclinic.
Split up clinic.py by establishing libclinic as a support package for
Argument Clinic. Get rid of clinic.py globals by either making them
class members, or by putting them into libclinic.
- Move INCLUDE_COMMENT_COLUMN to BlockPrinter
- Move NO_VARARG to CLanguage
- Move formatting helpers to libclinic
- Move some constants to libclinic (and annotate them as Final)
Replace the internal accumulator APIs by using lists of strings and join().
Yak-shaving for separating out formatting code into a separate file.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
dependabot[bot]
Petr Viktorin
Victor Stinner
Serhiy Storchaka
Xie Yanbo
Eric Snow
neonene
Erlend E. Aasland
Nikita Sobolev
Alex Waygood