Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity

Merge heads

This commit is contained in:
Serhiy Storchaka 2014-01-04 22:49:40 +02:00
parent 3079328d29 78cf85c669
commit 78194cd4e9
5 changed files with 954 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

900
Doc/howto/clinic.rst Normal file
View File

@ -0,0 +1,900 @@
======================
Argument Clinic How-To
======================
:author: Larry Hastings
.. topic:: Abstract
Argument Clinic is a preprocessor for CPython C files.
Its purpose is to automate all the boilerplate involved
with writing argument parsing code for "builtins".
This document shows you how to convert your first C
function to work with Argument Clinic, and then introduces
some advanced topics on Argument Clinic usage.
Argument Clinic is currently considered an internal
tool for the CPython code tree. Its use is not supported
for files outside the CPython code tree, and no guarantees
are made regarding backwards compatibility for future
versions. In other words: if you maintain an external C
extension for CPython, you're welcome to experiment with
Argument Clinic in your own code. But the version of Argument
Clinic that ships with CPython 3.5 *could* be totally
incompatible and break all your code.
========================
Basic Concepts And Usage
========================
Argument Clinic ships with CPython. You can find it in ``Tools/clinic/clinic.py``.
If you run that script, specifying a C file as an argument::
% python3 Tools/clinic/clinic.py foo.c
Argument Clinic will scan over the file looking for lines that
look exactly like this::
/*[clinic]
When it finds one, it reads everything up to a line that looks
like this::
[clinic]*/
Everything in between these two lines is input for Argument Clinic.
All of these lines, including the beginning and ending comment
lines, are collectively called an Argument Clinic "input block",
or "block" for short.
When Argument Clinic parses one of these blocks, it
generates output. This output is rewritten into the C file
immediately after the block, followed by a comment containing a checksum.
The resulting Argument Clinic block looks like this::
/*[clinic]
... clinic input goes here ...
[clinic]*/
... clinic output goes here ...
/*[clinic checksum:...]*/
If you run Argument Clinic on the same file a second time, Argument Clinic
will discard the old output and write out the new output with a fresh checksum
line. However, if the input hasn't changed, the output won't change either.
You should never modify the output portion of an Argument Clinic block. Instead,
change the input until it produces the output you want. (That's the purpose of the
checksum--to detect and warn you in case someone accidentally modifies the output.)
For the sake of clarity, here's the terminology we'll use with Argument Clinic:
* The first line of the comment (``/*[clinic]``) is the *start line*.
* The last line of the initial comment (``[clinic]*/``) is the *end line*.
* The last line (``/*[clinic checksum:...]*/``) is the *checksum line*.
* In between the start line and the end line is the *input*.
* In between the end line and the checksum line is the *output*.
* All the text collectively, from the start line to the checksum line inclusively,
is the *block*. (A block that hasn't been successfully processed by Argument
Clinic yet doesn't have output or a checksum line, but it's still considered
a block.)
==============================
Converting Your First Function
==============================
The best way to get a sense of how Argument Clinic works is to
convert a function to work with it. Let's dive in!
0. Make sure you're working with a freshly updated trunk.
1. Find a Python builtin that calls either ``PyArg_ParseTuple()``
or ``PyArg_ParseTupleAndKeywords()``, and hasn't been converted yet.
For my example I'm using ``pickle.Pickler.dump()``.
2. If the call to the ``PyArg_Parse`` function uses any of the
following format units::
O&
O!
es
es#
et
et#
or if it has multiple calls to ``PyArg_ParseTuple()``,
you should choose a different function. Argument Clinic *does*
support all of these scenarios. But these are advanced
topics--let's do something simpler for your first function.
3. Add the following boilerplate above the function, creating our block::
/*[clinic]
[clinic]*/
4. Cut the docstring and paste it in between the ``[clinic]`` lines,
removing all the junk that makes it a properly quoted C string.
When you're done you should have just the text, based at the left
margin, with no line wider than 80 characters.
(Argument Clinic will preserve indents inside the docstring.)
Sample::
/*[clinic]
Write a pickled representation of obj to the open file.
[clinic]*/
5. If your docstring doesn't have a "summary" line, Argument Clinic will
complain. So let's make sure it has one. The "summary" line should
be a paragraph consisting of a single 80-column line
at the beginning of the docstring.
(Our docstring consists solely of the summary line, so the sample
code doesn't have to change for this step.)
6. Above the docstring, enter the name of the function, followed
by a blank line. This should be the Python name of the function,
and should be the full dotted path
to the function--it should start with the name of the module,
include any sub-modules, and if the function is a method on
a class it should include the class name too.
Sample::
/*[clinic]
pickle.Pickler.dump
Write a pickled representation of obj to the open file.
[clinic]*/
7. If this is the first time that module or class has been used with Argument
Clinic in this C file,
you must declare the module and/or class. Proper Argument Clinic hygiene
prefers declaring these in a separate block somewhere near the
top of the C file, in the same way that include files and statics go at
the top. (In our sample code we'll just show the two blocks next to
each other.)
Sample::
/*[clinic]
module pickle
class pickle.Pickler
[clinic]*/
/*[clinic]
pickle.Pickler.dump
Write a pickled representation of obj to the open file.
[clinic]*/
8. Declare each of the parameters to the function. Each parameter
should get its own line. All the parameter lines should be
indented from the function name and the docstring.
The general form of these parameter lines is as follows::
name_of_parameter: converter
If the parameter has a default value, add that after the
converter::
name_of_parameter: converter = default_value
Add a blank line below the parameters.
What's a "converter"? It establishes both the type
of the variable used in C, and the method to convert the Python
value into a C value at runtime.
For now you're going to use what's called a "legacy converter"--a
convenience syntax intended to make porting old code into Argument
Clinic easier.
For each parameter, copy the "format unit" for that
parameter from the ``PyArg_Parse()`` format argument and
specify *that* as its converter, as a quoted
string. ("format unit" is the formal name for the one-to-three
character substring of the ``format`` parameter that tells
the argument parsing function what the type of the variable
is and how to convert it.)
For multicharacter format units like ``z#``, use the
entire two-or-three character string.
Sample::
/*[clinic]
module pickle
class pickle.Pickler
[clinic]*/
/*[clinic]
pickle.Pickler.dump
obj: 'O'
Write a pickled representation of obj to the open file.
[clinic]*/
9. If your function has ``|`` in the format string, meaning some
parameters have default values, you can ignore it. Argument
Clinic infers which parameters are optional based on whether
or not they have default values.
If your function has ``$`` in the format string, meaning it
takes keyword-only arguments, specify ``*`` on a line by
itself before the first keyword-only argument, indented the
same as the parameter lines.
(``pickle.Pickler.dump`` has neither, so our sample is unchanged.)
10. If the existing C function uses ``PyArg_ParseTuple()``
(instead of ``PyArg_ParseTupleAndKeywords()``), then all its
arguments are positional-only.
To mark all parameters as positional-only in Argument Clinic,
add a ``/`` on a line by itself after the last parameter,
indented the same as the parameter lines.
Sample::
/*[clinic]
module pickle
class pickle.Pickler
[clinic]*/
/*[clinic]
pickle.Pickler.dump
obj: 'O'
/
Write a pickled representation of obj to the open file.
[clinic]*/
11. It's helpful to write a per-parameter docstring, indented
another level past the parameter declaration. But per-parameter
docstrings are optional; you can skip this step if you prefer.
Here's how per-parameter docstrings work. The first line
of the per-parameter docstring must be indented further than the
parameter definition. This left margin establishes the left margin
for the whole per-parameter docstring; all the text you write will
be outdented by this amount. You can write as much as you like,
across multiple lines if you wish.
Sample::
/*[clinic]
module pickle
class pickle.Pickler
[clinic]*/
/*[clinic]
pickle.Pickler.dump
obj: 'O'
The object to be pickled.
/
Write a pickled representation of obj to the open file.
[clinic]*/
12. Save and close the file, then run ``Tools/clinic/clinic.py`` on it.
With luck everything worked and your block now has output! Reopen
the file in your text editor to see::
/*[clinic]
module pickle
class pickle.Pickler
[clinic]*/
/*[clinic checksum: da39a3ee5e6b4b0d3255bfef95601890afd80709]*/
/*[clinic]
pickle.Pickler.dump
obj: 'O'
The object to be pickled.
/
Write a pickled representation of obj to the open file.
[clinic]*/
PyDoc_STRVAR(pickle_Pickler_dump__doc__,
"Write a pickled representation of obj to the open file.\n"
"\n"
...
static PyObject *
pickle_Pickler_dump_impl(PyObject *self, PyObject *obj)
/*[clinic checksum: 3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/
13. Double-check that the argument-parsing code Argument Clinic generated
looks basically the same as the existing code.
First, ensure both places use the same argument-parsing function.
The existing code must call either
``PyArg_ParseTuple()`` or ``PyArg_ParseTupleAndKeywords()``;
ensure that the code generated by Argument Clinic calls the
same function.
Second, the format string passed in to ``PyArg_ParseTuple()`` or
``PyArg_ParseTupleAndKeywords()`` should be *exactly* the same
as the hand-written one in the existing function.
Well, there's one way that Argument Clinic's output is permitted
to be different. Argument Clinic always generates a format string
ending with ``:`` followed by the name of the function. If the
format string originally ended with ``;`` (to specify usage help),
this is harmless--don't worry about this difference.
Apart from that, if either of these things differ in *any way*,
fix your input to Argument Clinic and rerun ``Tools/clinic/clinic.py``
until they are the same.
14. Notice that the last line of its output is the declaration
of your "impl" function. This is where the builtin's implementation goes.
Delete the existing prototype of the function you're modifying, but leave
the opening curly brace. Now delete its argument parsing code and the
declarations of all the variables it dumps the arguments into.
Notice how the Python arguments are now arguments to this impl function;
if the implementation used different names for these variables, fix it.
The result should be a function that handles just the implementation
of the Python function without any argument-parsing code.
Sample::
/*[clinic]
module pickle
class pickle.Pickler
[clinic]*/
/*[clinic checksum: da39a3ee5e6b4b0d3255bfef95601890afd80709]*/
/*[clinic]
pickle.Pickler.dump
obj: 'O'
The object to be pickled.
/
Write a pickled representation of obj to the open file.
[clinic]*/
PyDoc_STRVAR(pickle_Pickler_dump__doc__,
"Write a pickled representation of obj to the open file.\n"
"\n"
...
static PyObject *
pickle_Pickler_dump_impl(PyObject *self, PyObject *obj)
/*[clinic checksum: 3bd30745bf206a48f8b576a1da3d90f55a0a4187]*/
{
/* Check whether the Pickler was initialized correctly (issue3664).
Developers often forget to call __init__() in their subclasses, which
would trigger a segfault without this check. */
if (self->write == NULL) {
PyErr_Format(PicklingError,
"Pickler.__init__() was not called by %s.__init__()",
Py_TYPE(self)->tp_name);
return NULL;
}
if (_Pickler_ClearBuffer(self) < 0)
return NULL;
...
15. Compile and run the relevant portions of the regression-test suite.
This change should not introduce any new compile-time warnings or errors,
and there should be no externally-visible change to Python's behavior.
Well, except for one difference: ``inspect.signature()`` run on your function
should now provide a valid signature!
Congratulations, you've ported your first function to work with Argument Clinic!
===============
Advanced Topics
===============
Renaming the C functions generated by Argument Clinic
-----------------------------------------------------
Argument Clinic automatically names the functions it generates for you.
Occasionally this may cause a problem, if the generated name collides with
the name of an existing C function. There's an easy solution: you can explicitly
specify the base name to use for the C functions. Just add the keyword ``"as"``
to your function declaration line, followed by the function name you wish to use.
Argument Clinic will use the function name you use for the base (generated) function,
and then add ``"_impl"`` to the end for the name of the impl function.
For example, if we wanted to rename the C function names generated for
``pickle.Pickler.dump``, it'd look like this::
/*[clinic]
pickle.Pickler.dump as pickler_dumper
...
The base function would now be named ``pickler_dumper()``,
and the impl function would be named ``pickler_dumper_impl()``.
Optional Groups
---------------
Some legacy functions have a tricky approach to parsing their arguments:
they count the number of positional arguments, then use a ``switch`` statement
to call one of several different ``PyArg_ParseTuple()`` calls depending on
how many positional arguments there are. (These functions cannot accept
keyword-only arguments.) This approach was used to simulate optional
arguments back before ``PyArg_ParseTupleAndKeywords()`` was created.
Functions using this approach can often be converted to
use ``PyArg_ParseTupleAndKeywords()``, optional arguments, and default values.
But it's not always possible, because some of these legacy functions have
behaviors ``PyArg_ParseTupleAndKeywords()`` can't directly support.
The most obvious example is the builtin function ``range()``, which has
an optional argument on the *left* side of its required argument!
Another example is ``curses.window.addch()``, which has a group of two
arguments that must always be specified together. (The arguments are
called ``x`` and ``y``; if you call the function passing in ``x``,
you must also pass in ``y``--and if you don't pass in ``x`` you may not
pass in ``y`` either.)
For the sake of backwards compatibility, Argument Clinic supports this
alternate approach to parsing, using what are called *optional groups*.
Optional groups are groups of arguments that can only be specified together.
They can be to the left or the right of the required arguments. They
can *only* be used with positional-only parameters.
To specify an optional group, add a ``[`` on a line by itself before
the parameters you wish to be
in a group together, and a ``]`` on a line by itself after the
parameters. As an example, here's how ``curses.window.addch``
uses optional groups to make the first two parameters and the last
parameter optional::
/*[clinic]
curses.window.addch
[
x: int
X-coordinate.
y: int
Y-coordinate.
]
ch: object
Character to add.
[
attr: long
Attributes for the character.
]
/
...
Notes:
* For every optional group, one additional parameter will be passed into the
impl function representing the group. The parameter will be an int, and it will
be named ``group_{direction}_{number}``,
where ``{direction}`` is either ``right`` or ``left`` depending on whether the group
is before or after the required parameters, and ``{number}`` is a monotonically
increasing number (starting at 1) indicating how far away the group is from
the required parameters. When the impl is called, this parameter will be set
to zero if this group was unused, and set to non-zero if this group was used.
(By used or unused, I mean whether or not the parameters received arguments
in this invocation.)
* If there are no required arguments, the optional groups will behave
as if they are to the right of the required arguments.
* In the case of ambiguity, the argument parsing code
favors parameters on the left (before the required parameters).
* Optional groups are *only* intended for legacy code. Please do not
use optional groups for new code.
Using real Argument Clinic converters, instead of "legacy converters"
---------------------------------------------------------------------
To save time, and to minimize how much you need to learn
to achieve your first port to Argument Clinic, the walkthrough above tells
you to use the "legacy converters". "Legacy converters" are a convenience,
designed explicitly to make porting existing code to Argument Clinic
easier. And to be clear, their use is entirely acceptable when porting
code for Python 3.4.
However, in the long term we probably want all our blocks to
use Argument Clinic's real syntax for converters. Why? A couple
reasons:
* The proper converters are far easier to read and clearer in their intent.
* There are some format units that are unsupported as "legacy converters",
because they require arguments, and the legacy converter syntax doesn't
support specifying arguments.
* In the future we may have a new argument parsing library that isn't
restricted to what ``PyArg_ParseTuple()`` supports.
So if you want
to go that extra effort, you should consider using normal
converters instead of the legacy converters.
In a nutshell, the syntax for Argument Clinic (non-legacy) converters
looks like a Python function call. However, if there are no explicit
arguments to the function (all functions take their default values),
you may omit the parentheses. Thus ``bool`` and ``bool()`` are exactly
the same. All parameters to Argument Clinic converters are keyword-only.
All Argument Clinic converters accept the following arguments:
``doc_default``
If the parameter takes a default value, normally this value is also
provided in the ``inspect.Signature`` metadata, and displayed in the
docstring. ``doc_default`` lets you override the value used in these
two places: pass in a string representing the Python value you wish
to use in these two contexts.
``required``
If a parameter takes a default value, Argument Clinic infers that the
parameter is optional. However, you may want a parameter to take a
default value in C, but not behave in Python as if the parameter is
optional. Passing in ``required=True`` to a converter tells Argument
Clinic that this parameter is not optional, even if it has a default
value.
``annotation``
The annotation value for this parameter. Not currently supported,
because PEP 8 mandates that the Python library may not use
annotations.
Below is a table showing the mapping of legacy converters into real
Argument Clinic converters. On the left is the legacy converter,
on the right is the text you'd replace it with.
========= =================================================================================
``'B'`` ``byte(bitwise=True)``
``'b'`` ``byte``
``'c'`` ``char``
``'C'`` ``int(types='str')``
``'d'`` ``double``
``'D'`` ``Py_complex``
``'es#'`` ``str(encoding='name_of_encoding', length=True, zeroes=True)``
``'es'`` ``str(encoding='name_of_encoding')``
``'et#'`` ``str(encoding='name_of_encoding', types='bytes bytearray str', length=True)``
``'et'`` ``str(encoding='name_of_encoding', types='bytes bytearray str')``
``'f'`` ``float``
``'h'`` ``short``
``'H'`` ``unsigned_short``
``'i'`` ``int``
``'I'`` ``unsigned_int``
``'K'`` ``unsigned_PY_LONG_LONG``
``'L'`` ``PY_LONG_LONG``
``'n'`` ``Py_ssize_t``
``'O!'`` ``object(type='name_of_Python_type')``
``'O&'`` ``object(converter='name_of_c_function')``
``'O'`` ``object``
``'p'`` ``bool``
``'s#'`` ``str(length=True)``
``'S'`` ``PyBytesObject``
``'s'`` ``str``
``'s*'`` ``Py_buffer(types='str bytes bytearray buffer')``
``'u#'`` ``Py_UNICODE(length=True)``
``'u'`` ``Py_UNICODE``
``'U'`` ``unicode``
``'w*'`` ``Py_buffer(types='bytearray rwbuffer')``
``'y#'`` ``str(type='bytes', length=True)``
``'Y'`` ``PyByteArrayObject``
``'y'`` ``str(type='bytes')``
``'y*'`` ``Py_buffer``
``'Z#'`` ``Py_UNICODE(nullable=True, length=True)``
``'z#'`` ``str(nullable=True, length=True)``
``'Z'`` ``Py_UNICODE(nullable=True)``
``'z'`` ``str(nullable=True)``
``'z*'`` ``Py_buffer(types='str bytes bytearray buffer', nullable=True)``
========= =================================================================================
As an example, here's our sample ``pickle.Pickler.dump`` using the proper
converter::
/*[clinic]
pickle.Pickler.dump
obj: object
The object to be pickled.
/
Write a pickled representation of obj to the open file.
[clinic]*/
Argument Clinic will show you all the converters it has
available. For each converter it'll show you all the parameters
it accepts, along with the default value for each parameter.
Just run ``Tools/clinic/clinic.py --converters`` to see the full list.
Advanced converters
-------------------
Remeber those format units you skipped for your first
time because they were advanced? Here's how to handle those too.
The trick is, all those format units take arguments--either
conversion functions, or types, or strings specifying an encoding.
(But "legacy converters" don't support arguments. That's why we
skipped them for your first function.) The argument you specified
to the format unit is now an argument to the converter; this
argument is either ``converter`` (for ``O&``), ``type`` (for ``O!``),
or ``encoding`` (for all the format units that start with ``e``).
Note that ``object()`` must explicitly support each Python type you specify
for the ``type`` argument. Currently it only supports ``str``. It should be
easy to add more, just edit ``Tools/clinic/clinic.py``, search for ``O!`` in
the text, and add more entries to the dict mapping types to strings just above it.
Note also that this approach takes away some possible flexibility for the format
units starting with ``e``. It used to be possible to decide at runtime what
encoding string to pass in to ``PyArg_ParseTuple()``. But now this string must
be hard-coded at compile-time. This limitation is deliberate; it made supporting
this format unit much easier, and may allow for future compile-time optimizations.
This restriction does not seem unreasonable; CPython itself always passes in static
hard-coded strings when using format units starting with ``e``.
Using a return converter
------------------------
By default the impl function Argument Clinic generates for you returns ``PyObject *``.
But your C function often computes some C type, then converts it into the ``PyObject *``
at the last moment. Argument Clinic handles converting your inputs from Python types
into native C types--why not have it convert your return value from a native C type
into a Python type too?
That's what a "return converter" does. It changes your impl function to return
some C type, then adds code to the generated (non-impl) function to handle converting
that value into the appropriate ``PyObject *``.
The syntax for return converters is similar to that of parameter converters.
You specify the return converter like it was a return annotation on the
function itself. Return converters behave much the same as parameter converters;
they take arguments, the arguments are all keyword-only, and if you're not changing
any of the default arguments you can omit the parentheses.
(If you use both ``"as"`` *and* a return converter for your function,
the ``"as"`` should come before the return converter.)
There's one additional complication when using return converters: how do you
indicate an error has occured? Normally, a function returns a valid (non-``NULL``)
pointer for success, and ``NULL`` for failure. But if you use an integer return converter,
all integers are valid. How can Argument Clinic detect an error? Its solution: each return
converter implicitly looks for a special value that indicates an error. If you return
that value, and an error has been set (``PyErr_Occurred()`` returns a true
value), then the generated code will propogate the error. Otherwise it will
encode the value you return like normal.
Currently Argument Clinic supports only a few return converters::
int
long
Py_ssize_t
DecodeFSDefault
None of these take parameters. For the first three, return -1 to indicate
error. For ``DecodeFSDefault``, the return type is ``char *``; return a NULL
pointer to indicate an error.
Calling Python code
-------------------
The rest of the advanced topics require you to write Python code
which lives inside your C file and modifies Argument Clinic at
runtime. This is simple; you simply define a Python block.
A Python block uses different delimiter lines than an Argument
Clinic function block. It looks like this::
/*[python]
# python code goes here
[python]*/
All the code inside the Python block is executed at the
time it's parsed. All text written to stdout inside the block
is redirected into the "output" after the block.
As an example, here's a Python block that adds a static integer
variable to the C code::
/*[python]
print('static int __ignored_unused_variable__ = 0;')
[python]*/
static int __ignored_unused_variable__ = 0;
/*[python checksum:...]*/
Using a "self converter"
------------------------
Argument Clinic automatically adds a "self" parameter for you
using a default converter. However, you can override
Argument Clinic's converter and specify one yourself.
Just add your own ``self`` parameter as the first parameter in a
block, and ensure that its converter is an instance of
``self_converter`` or a subclass thereof.
What's the point? This lets you automatically cast ``self``
from ``PyObject *`` to a custom type.
How do you specify the custom type you want to cast ``self`` to?
If you only have one or two functions with the same type for ``self``,
you can directly use Argument Clinic's existing ``self`` converter,
passing in the type you want to use as the ``type`` parameter::
/*[clinic]
_pickle.Pickler.dump
self: self(type="PicklerObject *")
obj: object
/
Write a pickled representation of the given object to the open file.
[clinic]*/
On the other hand, if you have a lot of functions that will use the same
type for ``self``, it's best to create your own converter, subclassing
``self_converter`` but overwriting the ``type`` member::
/*[clinic]
class PicklerObject_converter(self_converter):
type = "PicklerObject *"
[clinic]*/
/*[clinic]
_pickle.Pickler.dump
self: PicklerObject
obj: object
/
Write a pickled representation of the given object to the open file.
[clinic]*/
Writing a custom converter
--------------------------
As we hinted at in the previous section... you can write your own converters!
A converter is simply a Python class that inherits from ``CConverter``.
The main purpose of a custom converter is if you have a parameter using
the ``O&`` format unit--parsing this parameter means calling
a ``PyArg_ParseTuple()`` "converter function".
Your converter class should be named ``*something*_converter``.
If the name follows this convention, then your converter class
will be automatically registered with Argument Clinic; its name
will be the name of your class with the ``_converter`` suffix
stripped off. (This is done automatically for you with a metaclass.)
You shouldn't subclass ``CConverter.__init__``. Instead, you should
write a ``converter_init()`` function. ``converter_init()``
always accepts a ``self`` parameter; after that, all additional
parameters *must* be keyword-only. Any arguments passed in to
the converter in Argument Clinic will be passed along to your
``converter_init()``.
There are some additional members of ``CConverter`` you may wish
to specify in your subclass. Here's the current list:
``type``
The C type to use for this variable.
``type`` should be a Python string specifying the type, e.g. ``int``.
If this is a pointer type, the type string should end with ``' *'``.
``default``
The Python default value for this parameter, as a Python value.
Or the magic value ``unspecified`` if there is no default.
``doc_default``
``default`` as it should appear in the documentation,
as a string.
Or ``None`` if there is no default.
This string, when run through ``eval()``, should produce
a Python value.
``py_default``
``default`` as it should appear in Python code,
as a string.
Or ``None`` if there is no default.
``c_default``
``default`` as it should appear in C code,
as a string.
Or ``None`` if there is no default.
``c_ignored_default``
The default value used to initialize the C variable when
there is no default, but not specifying a default may
result in an "uninitialized variable" warning. This is
easily happen when using option groups--although
properly-written code won't actually use the variable,
the variable does get passed in to the _impl, and the
C compiler will complain about the "use" of the uninitialized
value. This value should be a string.
``converter``
The name of the C converter function, as a string.
``impl_by_reference``
A boolean value. If true,
Argument Clinic will add a ``&`` in front of the name of
the variable when passing it into the impl function.
``parse_by_reference``
A boolean value. If true,
Argument Clinic will add a ``&`` in front of the name of
the variable when passing it into ``PyArg_ParseTuple()``.
Here's the simplest example of a custom converter, from ``Modules/zlibmodule.c``::
/*[python]
class uint_converter(CConverter):
type = 'unsigned int'
converter = 'uint_converter'
[python]*/
/*[python checksum: da39a3ee5e6b4b0d3255bfef95601890afd80709]*/
This block adds a ``uint`` converter to Argument Clinic. Parameters
declared as ``uint`` will be declared as type ``unsigned int``, and will
be parsed by calling the ``uint_converter`` converter function in C.
``uint`` variables automatically support default values.
More sophisticated custom converters can insert custom C code to
handle initialization and cleanup.
You can see more examples of custom converters in the CPython
source tree; grep the C files for the string ``CConverter``.
Writing a custom return converter
---------------------------------
Writing a custom return converter is much like writing
a custom converter. Except it's much simpler, because return
converters are themselves much simpler.
Return converters must subclass ``CReturnConverter``.
There are no examples yet of custom return converters,
because they are not widely used yet. If you wish to
write your own return converter, please read ``Tools/clinic/clinic.py``,
specifically the implementation of ``CReturnConverter`` and
all its subclasses.
Using Argument Clinic in Python files
-------------------------------------
It's actually possible to use Argument Clinic to preprocess Python files.
There's no point to using Argument Clinic blocks, of course, as the output
wouldn't make any sense to the Python interpreter. But using Argument Clinic
to run Python blocks lets you use Python as a Python preprocessor!
Since Python comments are different from C comments, Argument Clinic
blocks embedded in Python files look slightly different. They look like this::
#/*[python]
#print("def foo(): pass")
#[python]*/
def foo(): pass
#/*[python checksum:...]*/

1
Doc/howto/index.rst
View File

@ -28,4 +28,5 @@ Currently, the HOWTOs are:
webservers.rst
argparse.rst
ipaddress.rst
clinic.rst

2
Misc/NEWS
View File

@ -343,6 +343,8 @@ Documentation
Tools/Demos
-----------
- Issue #19659: Added documentation for Argument Clinic.
- Issue #19976: Argument Clinic METH_NOARGS functions now always
take two parameters.

7
Modules/zlibmodule.c
View File

@ -312,9 +312,6 @@ class uint_converter(CConverter):
type = 'unsigned int'
converter = 'uint_converter'
class compobject_converter(self_converter):
type = "compobject *"
[python]*/
/*[python checksum: da39a3ee5e6b4b0d3255bfef95601890afd80709]*/
@ -750,7 +747,7 @@ save_unconsumed_input(compobject *self, int err)
zlib.Decompress.decompress
self: compobject
self: self(type="compobject *")
data: Py_buffer
The binary data to decompress.
@ -1032,7 +1029,7 @@ PyZlib_flush(compobject *self, PyObject *args)
/*[clinic]
zlib.Compress.copy
self: compobject
self: self(type="compobject *")
Return a copy of the compression object.
[clinic]*/

54
Tools/clinic/clinic.py
View File

@ -1296,11 +1296,13 @@ class CConverter(metaclass=CConverterAutoRegister):
must be keyword-only.
"""
# The C type to use for this variable.
# 'type' should be a Python string specifying the type, e.g. "int".
# If this is a pointer type, the type string should end with ' *'.
type = None
format_unit = 'O&'
# The Python default value for this parameter, as a Python value.
# Or "unspecified" if there is no default.
# Or the magic value "unspecified" if there is no default.
default = unspecified
# "default" as it should appear in the documentation, as a string.
@ -1330,9 +1332,32 @@ class CConverter(metaclass=CConverterAutoRegister):
# (If this is not None, format_unit must be 'O&'.)
converter = None
encoding = None
# Should Argument Clinic add a '&' before the name of
# the variable when passing it into the _impl function?
impl_by_reference = False
# Should Argument Clinic add a '&' before the name of
# the variable when passing it into PyArg_ParseTuple (AndKeywords)?
parse_by_reference = True
#############################################################
#############################################################
## You shouldn't need to read anything below this point to ##
## write your own converter functions. ##
#############################################################
#############################################################
# The "format unit" to specify for this variable when
# parsing arguments using PyArg_ParseTuple (AndKeywords).
# Custom converters should always use the default value of 'O&'.
format_unit = 'O&'
# What encoding do we want for this variable? Only used
# by format units starting with 'e'.
encoding = None
# Do we want an adjacent '_length' variable for this variable?
# Only used by format units ending with '#'.
length = False
def __init__(self, name, function, default=unspecified, *, doc_default=None, required=False, annotation=unspecified, **kwargs):
@ -1751,7 +1776,7 @@ class self_converter(CConverter):
this is the default converter used for "self".
"""
type = "PyObject *"
def converter_init(self):
def converter_init(self, *, type=None):
f = self.function
if f.kind == CALLABLE:
if f.cls:
@ -1766,6 +1791,9 @@ class self_converter(CConverter):
self.name = "cls"
self.type = "PyTypeObject *"
if type:
self.type = type
def render(self, parameter, data):
fail("render() should never be called on self_converter instances")
@ -1787,7 +1815,13 @@ class CReturnConverterAutoRegister(type):
class CReturnConverter(metaclass=CReturnConverterAutoRegister):
# The C type to use for this variable.
# 'type' should be a Python string specifying the type, e.g. "int".
# If this is a pointer type, the type string should end with ' *'.
type = 'PyObject *'
# The Python default value for this parameter, as a Python value.
# Or the magic value "unspecified" if there is no default.
default = None
def __init__(self, *, doc_default=None, **kwargs):
@ -1826,6 +1860,16 @@ class CReturnConverter(metaclass=CReturnConverterAutoRegister):
add_c_return_converter(CReturnConverter, 'object')
class NoneType_return_converter(CReturnConverter):
def render(self, function, data):
self.declare(data)
data.return_conversion.append('''
if (_return_value != Py_None)
goto exit;
return_value = Py_None;
Py_INCREF(Py_None);
'''.strip())
class int_return_converter(CReturnConverter):
type = 'int'
@ -2680,7 +2724,7 @@ def main(argv):
# print(" ", short_name + "".join(parameters))
print()
print("All converters also accept (doc_default=None, required=False).")
print("All converters also accept (doc_default=None, required=False, annotation=None).")
print("All return converters also accept (doc_default=None).")
sys.exit(0)