Second round of updates to the descriptor howto guide (GH-22946) (GH-22958)

Doc/howto/descriptor.rst

@@ -29,8 +29,8 @@ This HowTo guide has three major sections:
 Primer
 ^^^^^^
 
-In this primer, we start with most basic possible example and then we'll add
+In this primer, we start with the most basic possible example and then we'll
-new capabilities one by one.
+add new capabilities one by one.
 
 
 Simple example: A descriptor that returns a constant
@@ -197,7 +197,7 @@ be recorded, giving each descriptor its own *public_name* and *private_name*::
 
     import logging
 
-    logging.basicConfig(level=logging.INFO)
+    logging.basicConfig(level=logging.INFO, force=True)
 
     class LoggedAccess:
 
@@ -258,6 +258,10 @@ Closing thoughts
 A :term:`descriptor` is what we call any object that defines :meth:`__get__`,
 :meth:`__set__`, or :meth:`__delete__`.
 
+Optionally, descriptors can have a :meth:`__set_name__` method.  This is only
+used in cases where a descriptor needs to know either the class where it is
+created or the name of class variable it was assigned to.
+
 Descriptors get invoked by the dot operator during attribute lookup.  If a
 descriptor is accessed indirectly with ``vars(some_class)[descriptor_name]``,
 the descriptor instance is returned without invoking it.
@@ -291,7 +295,7 @@ Validator class
 A validator is a descriptor for managed attribute access.  Prior to storing
 any data, it verifies that the new value meets various type and range
 restrictions.  If those restrictions aren't met, it raises an exception to
-prevents data corruption at its source.
+prevent data corruption at its source.
 
 This :class:`Validator` class is both an :term:`abstract base class` and a
 managed attribute descriptor::
@@ -438,12 +442,12 @@ In general, a descriptor is an object attribute with "binding behavior", one
 whose attribute access has been overridden by methods in the descriptor
 protocol.  Those methods are :meth:`__get__`, :meth:`__set__`, and
 :meth:`__delete__`.  If any of those methods are defined for an object, it is
-said to be a descriptor.
+said to be a :term:`descriptor`.
 
 The default behavior for attribute access is to get, set, or delete the
 attribute from an object's dictionary.  For instance, ``a.x`` has a lookup chain
 starting with ``a.__dict__['x']``, then ``type(a).__dict__['x']``, and
-continuing through the base classes of ``type(a)`` excluding metaclasses. If the
+continuing through the base classes of ``type(a)``. If the
 looked-up value is an object defining one of the descriptor methods, then Python
 may override the default behavior and invoke the descriptor method instead.
 Where this occurs in the precedence chain depends on which descriptor methods
@@ -492,60 +496,76 @@ Invoking Descriptors
 A descriptor can be called directly by its method name.  For example,
 ``d.__get__(obj)``.
 
-Alternatively, it is more common for a descriptor to be invoked automatically
+But it is more common for a descriptor to be invoked automatically from
-upon attribute access.  For example, ``obj.d`` looks up ``d`` in the dictionary
+attribute access.  The expression ``obj.d`` looks up ``d`` in the dictionary of
-of ``obj``.  If ``d`` defines the method :meth:`__get__`, then ``d.__get__(obj)``
+``obj``.  If ``d`` defines the method :meth:`__get__`, then ``d.__get__(obj)``
 is invoked according to the precedence rules listed below.
 
-The details of invocation depend on whether ``obj`` is an object or a class.
+The details of invocation depend on whether ``obj`` is an object, class, or
+instance of super.
 
-For objects, the machinery is in :meth:`object.__getattribute__` which
+**Objects**:  The machinery is in :meth:`object.__getattribute__`.
-transforms ``b.x`` into ``type(b).__dict__['x'].__get__(b, type(b))``.  The
+
-implementation works through a precedence chain that gives data descriptors
+It transforms ``b.x`` into ``type(b).__dict__['x'].__get__(b, type(b))``.
+
+The implementation works through a precedence chain that gives data descriptors
 priority over instance variables, instance variables priority over non-data
 descriptors, and assigns lowest priority to :meth:`__getattr__` if provided.
+
 The full C implementation can be found in :c:func:`PyObject_GenericGetAttr()` in
 :source:`Objects/object.c`.
 
-For classes, the machinery is in :meth:`type.__getattribute__` which transforms
+**Classes**:  The machinery is in :meth:`type.__getattribute__`.
-``B.x`` into ``B.__dict__['x'].__get__(None, B)``.  In pure Python, it looks
-like::
 
-    def __getattribute__(self, key):
+It transforms ``A.x`` into ``A.__dict__['x'].__get__(None, A)``.
+
+In pure Python, it looks like this::
+
+    def __getattribute__(cls, key):
         "Emulate type_getattro() in Objects/typeobject.c"
-        v = object.__getattribute__(self, key)
+        v = object.__getattribute__(cls, key)
         if hasattr(v, '__get__'):
-            return v.__get__(None, self)
+            return v.__get__(None, cls)
         return v
 
-The important points to remember are:
+**Super**:  The machinery is in the custom :meth:`__getattribute__` method for
+object returned by :class:`super()`.
 
-* descriptors are invoked by the :meth:`__getattribute__` method
+The attribute lookup ``super(A, obj).m`` searches ``obj.__class__.__mro__`` for
-* overriding :meth:`__getattribute__` prevents automatic descriptor calls
+the base class ``B`` immediately following ``A`` and then returns
-* :meth:`object.__getattribute__` and :meth:`type.__getattribute__` make
+``B.__dict__['m'].__get__(obj, A)``.
-  different calls to :meth:`__get__`.
-* data descriptors always override instance dictionaries.
-* non-data descriptors may be overridden by instance dictionaries.
 
-The object returned by ``super()`` also has a custom :meth:`__getattribute__`
+If not a descriptor, ``m`` is returned unchanged.  If not in the dictionary,
-method for invoking descriptors.  The attribute lookup ``super(B, obj).m`` searches
+``m`` reverts to a search using :meth:`object.__getattribute__`.
-``obj.__class__.__mro__`` for the base class ``A`` immediately following ``B``
-and then returns ``A.__dict__['m'].__get__(obj, B)``.  If not a descriptor,
-``m`` is returned unchanged.  If not in the dictionary, ``m`` reverts to a
-search using :meth:`object.__getattribute__`.
 
 The implementation details are in :c:func:`super_getattro()` in
-:source:`Objects/typeobject.c`.  and a pure Python equivalent can be found in
+:source:`Objects/typeobject.c`.  A pure Python equivalent can be found in
 `Guido's Tutorial`_.
 
 .. _`Guido's Tutorial`: https://www.python.org/download/releases/2.2.3/descrintro/#cooperation
 
-The details above show that the mechanism for descriptors is embedded in the
+**Summary**:  The details listed above show that the mechanism for descriptors is
-:meth:`__getattribute__()` methods for :class:`object`, :class:`type`, and
+embedded in the :meth:`__getattribute__()` methods for :class:`object`,
-:func:`super`.  Classes inherit this machinery when they derive from
+:class:`type`, and :func:`super`.
-:class:`object` or if they have a metaclass providing similar functionality.
+
-Likewise, classes can turn-off descriptor invocation by overriding
+The important points to remember are:
-:meth:`__getattribute__()`.
+
+* Descriptors are invoked by the :meth:`__getattribute__` method.
+
+* Classes inherit this machinery from :class:`object`, :class:`type`, or
+  :func:`super`.
+
+* Overriding :meth:`__getattribute__` prevents automatic descriptor calls
+  because all the descriptor logic is in that method.
+
+* :meth:`object.__getattribute__` and :meth:`type.__getattribute__` make
+  different calls to :meth:`__get__`.  The first includes the instance and may
+  include the class.  The second puts in ``None`` for the instance and always
+  includes the class.
+
+* Data descriptors always override instance dictionaries.
+
+* Non-data descriptors may be overridden by instance dictionaries.
 
 
 Automatic Name Notification
@@ -569,47 +589,70 @@ afterwards, :meth:`__set_name__` will need to be called manually.
 Descriptor Example
 ------------------
 
-The following code creates a class whose objects are data descriptors which
+The following code is simplified skeleton showing how data descriptors could
-print a message for each get or set.  Overriding :meth:`__getattribute__` is
+be used to implement an `object relational mapping
-alternate approach that could do this for every attribute.  However, this
+<https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping>`_.
-descriptor is useful for monitoring just a few chosen attributes::
 
-    class RevealAccess:
+The essential idea is that instances only hold keys to a database table.  The
-        """A data descriptor that sets and returns values
+actual data is stored in an external table that is being dynamically updated::
-           normally and prints a message logging their access.
-        """
 
-        def __init__(self, initval=None, name='var'):
+    class Field:
-            self.val = initval
+
-            self.name = name
+        def __set_name__(self, owner, name):
+            self.fetch = f'SELECT {name} FROM {owner.table} WHERE {owner.key}=?;'
+            self.store = f'UPDATE {owner.table} SET {name}=? WHERE {owner.key}=?;'
 
         def __get__(self, obj, objtype=None):
-            print('Retrieving', self.name)
+            return conn.execute(self.fetch, [obj.key]).fetchone()[0]
-            return self.val
 
-        def __set__(self, obj, val):
+        def __set__(self, obj, value):
-            print('Updating', self.name)
+            conn.execute(self.store, [value, obj.key])
-            self.val = val
+            conn.commit()
 
-    class B:
+We can use the :class:`Field` to define "models" that describe the schema for
-        x = RevealAccess(10, 'var "x"')
+each table in a database::
-        y = 5
 
-    >>> m = B()
+    class Movie:
-    >>> m.x
+        table = 'Movies'                    # Table name
-    Retrieving var "x"
+        key = 'title'                       # Primary key
-    10
+        director = Field()
-    >>> m.x = 20
+        year = Field()
-    Updating var "x"
-    >>> m.x
-    Retrieving var "x"
-    20
-    >>> m.y
-    5
 
-The protocol is simple and offers exciting possibilities.  Several use cases are
+        def __init__(self, key):
-so common that they have been packaged into individual function calls.
+            self.key = key
-Properties, bound methods, static methods, and class methods are all
+
+    class Song:
+        table = 'Music'
+        key = 'title'
+        artist = Field()
+        year = Field()
+        genre = Field()
+
+        def __init__(self, key):
+            self.key = key
+
+An interactive session shows how data is retrieved from the database and how
+it can be updated::
+
+    >>> import sqlite3
+    >>> conn = sqlite3.connect('entertainment.db')
+
+    >>> Movie('Star Wars').director
+    'George Lucas'
+    >>> jaws = Movie('Jaws')
+    >>> f'Released in {jaws.year} by {jaws.director}'
+    'Released in 1975 by Steven Spielberg'
+
+    >>> Song('Country Roads').artist
+    'John Denver'
+
+    >>> Movie('Star Wars').director = 'J.J. Abrams'
+    >>> Movie('Star Wars').director
+    'J.J. Abrams'
+
+The descriptor protocol is simple and offers exciting possibilities.  Several
+use cases are so common that they have been packaged into individual function
+calls.  Properties, bound methods, static methods, and class methods are all
 based on the descriptor protocol.
 
 
@@ -619,7 +662,7 @@ Properties
 Calling :func:`property` is a succinct way of building a data descriptor that
 triggers function calls upon access to an attribute.  Its signature is::
 
-    property(fget=None, fset=None, fdel=None, doc=None) -> property attribute
+    property(fget=None, fset=None, fdel=None, doc=None) -> property
 
 The documentation shows a typical use to define a managed attribute ``x``::
 
@@ -695,17 +738,30 @@ Functions and Methods
 Python's object oriented features are built upon a function based environment.
 Using non-data descriptors, the two are merged seamlessly.
 
-Class dictionaries store methods as functions.  In a class definition, methods
+Functions stored in class dictionaries get turned into methods when invoked.
-are written using :keyword:`def` or :keyword:`lambda`, the usual tools for
+Methods only differ from regular functions in that the object instance is
-creating functions.  Methods only differ from regular functions in that the
+prepended to the other arguments.  By convention, the instance is called
-first argument is reserved for the object instance.  By Python convention, the
+*self* but could be called *this* or any other variable name.
-instance reference is called *self* but may be called *this* or any other
-variable name.
 
-To support method calls, functions include the :meth:`__get__` method for
+Methods can be created manually with :class:`types.MethodType` which is
-binding methods during attribute access.  This means that all functions are
+roughly equivalent to::
-non-data descriptors which return bound methods when they are invoked from an
+
-object.  In pure Python, it works like this::
+    class Method:
+        "Emulate Py_MethodType in Objects/classobject.c"
+
+        def __init__(self, func, obj):
+            self.__func__ = func
+            self.__self__ = obj
+
+        def __call__(self, *args, **kwargs):
+            func = self.__func__
+            obj = self.__self__
+            return func(obj, *args, **kwargs)
+
+To support automatic creation of methods, functions include the
+:meth:`__get__` method for binding methods during attribute access.  This
+means that functions are non-data descriptors which return bound methods
+during dotted lookup from an instance.  Here's how it works::
 
     class Function:
         ...
@@ -716,15 +772,20 @@ object.  In pure Python, it works like this::
                 return self
             return types.MethodType(self, obj)
 
-Running the following in class in the interpreter shows how the function
+Running the following class in the interpreter shows how the function
 descriptor works in practice::
 
     class D:
         def f(self, x):
              return x
 
-Access through the class dictionary does not invoke :meth:`__get__`.  Instead,
+The function has a :term:`qualified name` attribute to support introspection::
-it just returns the underlying function object::
+
+    >>> D.f.__qualname__
+    'D.f'
+
+Accessing the function through the class dictionary does not invoke
+:meth:`__get__`.  Instead, it just returns the underlying function object::
 
     >>> D.__dict__['f']
     <function D.f at 0x00C45070>
@@ -735,13 +796,8 @@ underlying function unchanged::
     >>> D.f
     <function D.f at 0x00C45070>
 
-The function has a :term:`qualified name` attribute to support introspection::
+The interesting behavior occurs during dotted access from an instance.  The
-
+dotted lookup calls :meth:`__get__` which returns a bound method object::
-    >>> D.f.__qualname__
-    'D.f'
-
-Dotted access from an instance calls :meth:`__get__` which returns a bound
-method object::
 
     >>> d = D()
     >>> d.f
@@ -752,9 +808,13 @@ instance::
 
     >>> d.f.__func__
     <function D.f at 0x1012e5ae8>
+
     >>> d.f.__self__
     <__main__.D object at 0x1012e1f98>
 
+If you have ever wondered where *self* comes from in regular methods or where
+*cls* comes from in class methods, this is it!
+
 
 Static Methods and Class Methods
 --------------------------------
@@ -798,8 +858,8 @@ in statistical work but does not directly depend on a particular dataset.
 It can be called either from an object or the class:  ``s.erf(1.5) --> .9332`` or
 ``Sample.erf(1.5) --> .9332``.
 
-Since staticmethods return the underlying function with no changes, the example
+Since static methods return the underlying function with no changes, the
-calls are unexciting::
+example calls are unexciting::
 
     class E:
         @staticmethod