Various improvements to the docs of the buffer API
This commit is contained in:
Antoine Pitrou
parent
b78b4893a9
commit
99a00a455c
|
|
@ -2,8 +2,8 @@
|
||||||
|
|
||||||
.. _bufferobjects:
|
.. _bufferobjects:
|
||||||
|
|
||||||
Buffer Objects
|
Buffer API
|
||||||
--------------
|
----------
|
||||||
|
|
||||||
.. sectionauthor:: Greg Stein <gstein@lyra.org>
|
.. sectionauthor:: Greg Stein <gstein@lyra.org>
|
||||||
.. sectionauthor:: Benjamin Peterson
|
.. sectionauthor:: Benjamin Peterson
|
||||||
|
|
@ -17,30 +17,56 @@ functions can be used by an object to expose its data in a raw, byte-oriented
|
||||||
format. Clients of the object can use the buffer interface to access the
|
format. Clients of the object can use the buffer interface to access the
|
||||||
object data directly, without needing to copy it first.
|
object data directly, without needing to copy it first.
|
||||||
|
|
||||||
Two examples of objects that support the buffer interface are bytes and
|
Examples of objects that support the buffer interface are :class:`bytes`,
|
||||||
arrays. The bytes object exposes the character contents in the buffer
|
:class:`bytearray` and :class:`array.array`. The bytes and bytearray objects
|
||||||
interface's byte-oriented form. An array can also expose its contents, but it
|
exposes their bytes contents in the buffer interface's byte-oriented form.
|
||||||
should be noted that array elements may be multi-byte values.
|
An :class:`array.array` can also expose its contents, but it should be noted
|
||||||
|
that array elements may be multi-byte values.
|
||||||
|
|
||||||
|
An example consumer of the buffer interface is the :meth:`~io.BufferedIOBase.write`
|
||||||
|
method of file objects: any object that can export a series of bytes through
|
||||||
|
the buffer interface can be written to a file. While :meth:`write` only
|
||||||
|
needs read-only access to the internal contents of the object passed to it,
|
||||||
|
other methods such as :meth:`~io.BufferedIOBase.readinto` need write access
|
||||||
|
to the contents of their argument. The buffer interface allows objects to
|
||||||
|
selectively allow or reject exporting of read-write and read-only buffers.
|
||||||
|
|
||||||
|
There are two ways for a consumer of the buffer interface to acquire a buffer
|
||||||
|
over a target object:
|
||||||
|
|
||||||
|
* call :cfunc:`PyObject_GetBuffer` with the right parameters;
|
||||||
|
|
||||||
|
* call :cfunc:`PyArg_ParseTuple` (or one of its siblings) with one of the
|
||||||
|
``y*``, ``w*`` or ``s*`` :ref:`format codes <arg-parsing>`.
|
||||||
|
|
||||||
|
In both cases, :cfunc:`PyBuffer_Release` must be called when the buffer
|
||||||
|
isn't needed anymore. Failure to do so could lead to various issues such as
|
||||||
|
resource leaks.
|
||||||
|
|
||||||
An example user of the buffer interface is the file object's :meth:`write`
|
|
||||||
method. Any object that can export a series of bytes through the buffer
|
|
||||||
interface can be written to a file. There are a number of format codes to
|
|
||||||
:cfunc:`PyArg_ParseTuple` that operate against an object's buffer interface,
|
|
||||||
returning data from the target object.
|
|
||||||
|
|
||||||
.. index:: single: PyBufferProcs
|
.. index:: single: PyBufferProcs
|
||||||
|
|
||||||
More information on the buffer interface is provided in the section
|
How the buffer interface is exposed by a type object is described in the
|
||||||
:ref:`buffer-structs`, under the description for :ctype:`PyBufferProcs`.
|
section :ref:`buffer-structs`, under the description for :ctype:`PyBufferProcs`.
|
||||||
|
|
||||||
Buffer objects are useful as a way to expose the data from another object's
|
|
||||||
buffer interface to the Python programmer. They can also be used as a zero-copy
|
Buffer objects
|
||||||
|
==============
|
||||||
|
|
||||||
|
Buffer objects are useful as a way to expose the binary data from another
|
||||||
|
object to the Python programmer. They can also be used as a zero-copy
|
||||||
slicing mechanism. Using their ability to reference a block of memory, it is
|
slicing mechanism. Using their ability to reference a block of memory, it is
|
||||||
possible to expose any data to the Python programmer quite easily. The memory
|
possible to expose any data to the Python programmer quite easily. The memory
|
||||||
could be a large, constant array in a C extension, it could be a raw block of
|
could be a large, constant array in a C extension, it could be a raw block of
|
||||||
memory for manipulation before passing to an operating system library, or it
|
memory for manipulation before passing to an operating system library, or it
|
||||||
could be used to pass around structured data in its native, in-memory format.
|
could be used to pass around structured data in its native, in-memory format.
|
||||||
|
|
||||||
|
Contrary to most data types exposed by the Python interpreter, buffer objects
|
||||||
|
are not :ctype:`PyObject` pointers but rather simple C structures. This
|
||||||
|
allows them to be created and copied very simply. When a generic wrapper
|
||||||
|
around a buffer object is needed, a :ref:`memoryview <memoryviewobjects>` object
|
||||||
|
can be created.
|
||||||
|
|
||||||
|
|
||||||
.. ctype:: Py_buffer
|
.. ctype:: Py_buffer
|
||||||
|
|
||||||
|
|
@ -133,18 +159,23 @@ Buffer related functions
|
||||||
|
|
||||||
.. cfunction:: int PyObject_CheckBuffer(PyObject *obj)
|
.. cfunction:: int PyObject_CheckBuffer(PyObject *obj)
|
||||||
|
|
||||||
Return 1 if *obj* supports the buffer interface otherwise 0.
|
Return 1 if *obj* supports the buffer interface otherwise 0. When 1 is
|
||||||
|
returned, it doesn't guarantee that :cfunc:`PyObject_GetBuffer` will
|
||||||
|
succeed.
|
||||||
|
|
||||||
|
|
||||||
.. cfunction:: int PyObject_GetBuffer(PyObject *obj, Py_buffer *view, int flags)
|
.. cfunction:: int PyObject_GetBuffer(PyObject *obj, Py_buffer *view, int flags)
|
||||||
|
|
||||||
Export *obj* into a :ctype:`Py_buffer`, *view*. These arguments must
|
Export a view over some internal data from the target object *obj*.
|
||||||
never be *NULL*. The *flags* argument is a bit field indicating what
|
*obj* must not be NULL, and *view* must point to an existing
|
||||||
kind of buffer the caller is prepared to deal with and therefore what
|
:ctype:`Py_buffer` structure allocated by the caller (most uses of
|
||||||
kind of buffer the exporter is allowed to return. The buffer interface
|
this function will simply declare a local variable of type
|
||||||
allows for complicated memory sharing possibilities, but some caller may
|
:ctype:`Py_buffer`). The *flags* argument is a bit field indicating
|
||||||
not be able to handle all the complexity but may want to see if the
|
what kind of buffer is requested. The buffer interface allows
|
||||||
exporter will let them take a simpler view to its memory.
|
for complicated memory layout possibilities; however, some callers
|
||||||
|
won't want to handle all the complexity and instead request a simple
|
||||||
|
view of the target object (using :cmacro:`PyBUF_SIMPLE` for a read-only
|
||||||
|
view and :cmacro:`PyBUF_WRITABLE` for a read-write view).
|
||||||
|
|
||||||
Some exporters may not be able to share memory in every possible way and
|
Some exporters may not be able to share memory in every possible way and
|
||||||
may need to raise errors to signal to some consumers that something is
|
may need to raise errors to signal to some consumers that something is
|
||||||
|
|
@ -154,26 +185,31 @@ Buffer related functions
|
||||||
:cdata:`Py_buffer` structure is filled in with non-default values and/or
|
:cdata:`Py_buffer` structure is filled in with non-default values and/or
|
||||||
raise an error if the object can't support a simpler view of its memory.
|
raise an error if the object can't support a simpler view of its memory.
|
||||||
|
|
||||||
0 is returned on success and -1 on error.
|
On success, 0 is returned and the *view* structure is filled with useful
|
||||||
|
values. On error, -1 is returned and an exception is raised; the *view*
|
||||||
|
is left in an undefined state.
|
||||||
|
|
||||||
The following table gives possible values to the *flags* arguments.
|
The following table gives possible values to the *flags* arguments.
|
||||||
|
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| Flag | Description |
|
| Flag | Description |
|
||||||
+==============================+===================================================+
|
+==============================+===================================================+
|
||||||
| :cmacro:`PyBUF_SIMPLE` | This is the default flag state. The returned |
|
| .. cmacro:: PyBUF_SIMPLE | This is the default flag. The returned buffer |
|
||||||
| | buffer may or may not have writable memory. The |
|
| | exposes a read-only memory area. The format of |
|
||||||
| | format of the data will be assumed to be unsigned |
|
| | data is assumed to be raw unsigned bytes, without |
|
||||||
| | bytes. This is a "stand-alone" flag constant. It |
|
| | any particular structure. This is a "stand-alone"|
|
||||||
|
| | flag constant. It |
|
||||||
| | never needs to be '|'d to the others. The exporter|
|
| | never needs to be '|'d to the others. The exporter|
|
||||||
| | will raise an error if it cannot provide such a |
|
| | will raise an error if it cannot provide such a |
|
||||||
| | contiguous buffer of bytes. |
|
| | contiguous buffer of bytes. |
|
||||||
| | |
|
| | |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_WRITABLE` | The returned buffer must be writable. If it is |
|
| .. cmacro:: PyBUF_WRITABLE | Like :cmacro:`PyBUF_SIMPLE`, but the returned |
|
||||||
| | not writable, then raise an error. |
|
| | buffer is writable. If the exporter doesn't |
|
||||||
|
| | support |
|
||||||
|
| | writable buffers, an error is raised. |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_STRIDES` | This implies :cmacro:`PyBUF_ND`. The returned |
|
| .. cmacro:: PyBUF_STRIDES | This implies :cmacro:`PyBUF_ND`. The returned |
|
||||||
| | buffer must provide strides information (i.e. the |
|
| | buffer must provide strides information (i.e. the |
|
||||||
| | strides cannot be NULL). This would be used when |
|
| | strides cannot be NULL). This would be used when |
|
||||||
| | the consumer can handle strided, discontiguous |
|
| | the consumer can handle strided, discontiguous |
|
||||||
|
|
@ -183,19 +219,17 @@ Buffer related functions
|
||||||
| | not possible (i.e. without the suboffsets). |
|
| | not possible (i.e. without the suboffsets). |
|
||||||
| | |
|
| | |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_ND` | The returned buffer must provide shape |
|
| .. cmacro:: PyBUF_ND | The returned buffer must provide shape |
|
||||||
| | information. The memory will be assumed C-style |
|
| | information. The memory will be assumed C-style |
|
||||||
| | contiguous (last dimension varies the |
|
| | contiguous (last dimension varies the |
|
||||||
| | fastest). The exporter may raise an error if it |
|
| | fastest). The exporter may raise an error if it |
|
||||||
| | cannot provide this kind of contiguous buffer. If |
|
| | cannot provide this kind of contiguous buffer. If |
|
||||||
| | this is not given then shape will be *NULL*. |
|
| | this is not given then shape will be *NULL*. |
|
||||||
| | |
|
| | |
|
||||||
| | |
|
|
||||||
| | |
|
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
|:cmacro:`PyBUF_C_CONTIGUOUS` | These flags indicate that the contiguity returned |
|
|.. cmacro:: PyBUF_C_CONTIGUOUS| These flags indicate that the contiguity returned |
|
||||||
|:cmacro:`PyBUF_F_CONTIGUOUS` | buffer must be respectively, C-contiguous (last |
|
| PyBUF_F_CONTIGUOUS| buffer must be respectively, C-contiguous (last |
|
||||||
|:cmacro:`PyBUF_ANY_CONTIGUOUS`| dimension varies the fastest), Fortran contiguous |
|
| PyBUF_ANY_CONTIGUOUS| dimension varies the fastest), Fortran contiguous |
|
||||||
| | (first dimension varies the fastest) or either |
|
| | (first dimension varies the fastest) or either |
|
||||||
| | one. All of these flags imply |
|
| | one. All of these flags imply |
|
||||||
| | :cmacro:`PyBUF_STRIDES` and guarantee that the |
|
| | :cmacro:`PyBUF_STRIDES` and guarantee that the |
|
||||||
|
|
@ -203,7 +237,7 @@ Buffer related functions
|
||||||
| | correctly. |
|
| | correctly. |
|
||||||
| | |
|
| | |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_INDIRECT` | This flag indicates the returned buffer must have |
|
| .. cmacro:: PyBUF_INDIRECT | This flag indicates the returned buffer must have |
|
||||||
| | suboffsets information (which can be NULL if no |
|
| | suboffsets information (which can be NULL if no |
|
||||||
| | suboffsets are needed). This can be used when |
|
| | suboffsets are needed). This can be used when |
|
||||||
| | the consumer can handle indirect array |
|
| | the consumer can handle indirect array |
|
||||||
|
|
@ -213,7 +247,7 @@ Buffer related functions
|
||||||
| | |
|
| | |
|
||||||
| | |
|
| | |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_FORMAT` | The returned buffer must have true format |
|
| .. cmacro:: PyBUF_FORMAT | The returned buffer must have true format |
|
||||||
| | information if this flag is provided. This would |
|
| | information if this flag is provided. This would |
|
||||||
| | be used when the consumer is going to be checking |
|
| | be used when the consumer is going to be checking |
|
||||||
| | for what 'kind' of data is actually stored. An |
|
| | for what 'kind' of data is actually stored. An |
|
||||||
|
|
@ -223,28 +257,28 @@ Buffer related functions
|
||||||
| | returned as *NULL* (which means ``'B'``, or |
|
| | returned as *NULL* (which means ``'B'``, or |
|
||||||
| | unsigned bytes) |
|
| | unsigned bytes) |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_STRIDED` | This is equivalent to ``(PyBUF_STRIDES | |
|
| .. cmacro:: PyBUF_STRIDED | This is equivalent to ``(PyBUF_STRIDES | |
|
||||||
| | PyBUF_WRITABLE)``. |
|
| | PyBUF_WRITABLE)``. |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_STRIDED_RO` | This is equivalent to ``(PyBUF_STRIDES)``. |
|
| .. cmacro:: PyBUF_STRIDED_RO | This is equivalent to ``(PyBUF_STRIDES)``. |
|
||||||
| | |
|
| | |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_RECORDS` | This is equivalent to ``(PyBUF_STRIDES | |
|
| .. cmacro:: PyBUF_RECORDS | This is equivalent to ``(PyBUF_STRIDES | |
|
||||||
| | PyBUF_FORMAT | PyBUF_WRITABLE)``. |
|
| | PyBUF_FORMAT | PyBUF_WRITABLE)``. |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_RECORDS_RO` | This is equivalent to ``(PyBUF_STRIDES | |
|
| .. cmacro:: PyBUF_RECORDS_RO | This is equivalent to ``(PyBUF_STRIDES | |
|
||||||
| | PyBUF_FORMAT)``. |
|
| | PyBUF_FORMAT)``. |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_FULL` | This is equivalent to ``(PyBUF_INDIRECT | |
|
| .. cmacro:: PyBUF_FULL | This is equivalent to ``(PyBUF_INDIRECT | |
|
||||||
| | PyBUF_FORMAT | PyBUF_WRITABLE)``. |
|
| | PyBUF_FORMAT | PyBUF_WRITABLE)``. |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_FULL_RO` | This is equivalent to ``(PyBUF_INDIRECT | |
|
| .. cmacro:: PyBUF_FULL_RO | This is equivalent to ``(PyBUF_INDIRECT | |
|
||||||
| | PyBUF_FORMAT)``. |
|
| | PyBUF_FORMAT)``. |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_CONTIG` | This is equivalent to ``(PyBUF_ND | |
|
| .. cmacro:: PyBUF_CONTIG | This is equivalent to ``(PyBUF_ND | |
|
||||||
| | PyBUF_WRITABLE)``. |
|
| | PyBUF_WRITABLE)``. |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
| :cmacro:`PyBUF_CONTIG_RO` | This is equivalent to ``(PyBUF_ND)``. |
|
| .. cmacro:: PyBUF_CONTIG_RO | This is equivalent to ``(PyBUF_ND)``. |
|
||||||
| | |
|
| | |
|
||||||
+------------------------------+---------------------------------------------------+
|
+------------------------------+---------------------------------------------------+
|
||||||
|
|
||||||
|
|
@ -299,6 +333,7 @@ Buffer related functions
|
||||||
.. index::
|
.. index::
|
||||||
object: memoryview
|
object: memoryview
|
||||||
|
|
||||||
|
.. _memoryviewobjects:
|
||||||
|
|
||||||
MemoryView objects
|
MemoryView objects
|
||||||
==================
|
==================
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue