traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Close #14588: added a PEP 3115 compliant dynamic type creation mechanism

This commit is contained in:
Nick Coghlan 2012-05-20 02:34:13 +10:00
parent 7c5ba513b9
commit 7fc570a51e
7 changed files with 486 additions and 51 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
Doc/library/functions.rst
View File

@ -1324,10 +1324,12 @@ are always available. They are listed here in alphabetical order.
Accordingly, :func:`super` is undefined for implicit lookups using statements or
operators such as ``super()[name]``.
Also note that :func:`super` is not limited to use inside methods. The two
argument form specifies the arguments exactly and makes the appropriate
references. The zero argument form automatically searches the stack frame
for the class (``__class__``) and the first argument.
Also note that, aside from the zero argument form, :func:`super` is not
limited to use inside methods. The two argument form specifies the
arguments exactly and makes the appropriate references. The zero
argument form only works inside a class definition, as the compiler fills
in the necessary details to correctly retrieve the class being defined,
as well as accessing the current instance for ordinary methods.
For practical suggestions on how to design cooperative classes using
:func:`super`, see `guide to using super()

67
Doc/library/types.rst
View File

@ -1,5 +1,5 @@
:mod:`types` --- Names for built-in types
=========================================
:mod:`types` --- Dynamic type creation and names for built-in types
===================================================================
.. module:: types
:synopsis: Names for built-in types.
@ -8,20 +8,69 @@
--------------
This module defines names for some object types that are used by the standard
This module defines utility function to assist in dynamic creation of
new types.
It also defines names for some object types that are used by the standard
Python interpreter, but not exposed as builtins like :class:`int` or
:class:`str` are. Also, it does not include some of the types that arise
transparently during processing such as the ``listiterator`` type.
:class:`str` are.
Typical use is for :func:`isinstance` or :func:`issubclass` checks.
The module defines the following names:
Dynamic Type Creation
---------------------
.. function:: new_class(name, bases=(), kwds=None, exec_body=None)
Creates a class object dynamically using the appropriate metaclass.
The arguments are the components that make up a class definition: the
class name, the base classes (in order), the keyword arguments (such as
``metaclass``) and the callback function to populate the class namespace.
The *exec_body* callback should accept the class namespace as its sole
argument and update the namespace directly with the class contents.
.. function:: prepare_class(name, bases=(), kwds=None)
Calculates the appropriate metaclass and creates the class namespace.
The arguments are the components that make up a class definition: the
class name, the base classes (in order) and the keyword arguments (such as
``metaclass``).
The return value is a 3-tuple: ``metaclass, namespace, kwds``
*metaclass* is the appropriate metaclass
*namespace* is the prepared class namespace
*kwds* is an updated copy of the passed in *kwds* argument with any
``'metaclass'`` entry removed. If no *kwds* argument is passed in, this
will be an empty dict.
.. seealso::
:pep:`3115` - Metaclasses in Python 3000
Introduced the ``__prepare__`` namespace hook
Standard Interpreter Types
--------------------------
This module provides names for many of the types that are required to
implement a Python interpreter. It deliberately avoids including some of
the types that arise only incidentally during processing such as the
``listiterator`` type.
Typical use is of these names is for :func:`isinstance` or
:func:`issubclass` checks.
Standard names are defined for the following types:
.. data:: FunctionType
LambdaType
The type of user-defined functions and functions created by :keyword:`lambda`
expressions.
The type of user-defined functions and functions created by
:keyword:`lambda` expressions.
.. data:: GeneratorType

136
Doc/reference/datamodel.rst
View File

@ -1550,53 +1550,115 @@ Notes on using *__slots__*
Customizing class creation
--------------------------
By default, classes are constructed using :func:`type`. A class definition is
read into a separate namespace and the value of class name is bound to the
result of ``type(name, bases, dict)``.
By default, classes are constructed using :func:`type`. The class body is
executed in a new namespace and the class name is bound locally to the
result of ``type(name, bases, namespace)``.
When the class definition is read, if a callable ``metaclass`` keyword argument
is passed after the bases in the class definition, the callable given will be
called instead of :func:`type`. If other keyword arguments are passed, they
will also be passed to the metaclass. This allows classes or functions to be
written which monitor or alter the class creation process:
The class creation process can be customised by passing the ``metaclass``
keyword argument in the class definition line, or by inheriting from an
existing class that included such an argument. In the following example,
both ``MyClass`` and ``MySubclass`` are instances of ``Meta``::
* Modifying the class dictionary prior to the class being created.
class Meta(type):
pass
* Returning an instance of another class -- essentially performing the role of a
factory function.
class MyClass(metaclass=Meta):
pass
These steps will have to be performed in the metaclass's :meth:`__new__` method
-- :meth:`type.__new__` can then be called from this method to create a class
with different properties. This example adds a new element to the class
dictionary before creating the class::
class MySubclass(MyClass):
pass
class metacls(type):
def __new__(mcs, name, bases, dict):
dict['foo'] = 'metacls was here'
return type.__new__(mcs, name, bases, dict)
Any other keyword arguments that are specified in the class definition are
passed through to all metaclass operations described below.
You can of course also override other class methods (or add new methods); for
example defining a custom :meth:`__call__` method in the metaclass allows custom
behavior when the class is called, e.g. not always creating a new instance.
When a class definition is executed, the following steps occur:
If the metaclass has a :meth:`__prepare__` attribute (usually implemented as a
class or static method), it is called before the class body is evaluated with
the name of the class and a tuple of its bases for arguments. It should return
an object that supports the mapping interface that will be used to store the
namespace of the class. The default is a plain dictionary. This could be used,
for example, to keep track of the order that class attributes are declared in by
returning an ordered dictionary.
* the appropriate metaclass is determined
* the class namespace is prepared
* the class body is executed
* the class object is created
The appropriate metaclass is determined by the following precedence rules:
Determining the appropriate metaclass
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* If the ``metaclass`` keyword argument is passed with the bases, it is used.
The appropriate metaclass for a class definition is determined as follows:
* Otherwise, if there is at least one base class, its metaclass is used.
* if no bases and no explicit metaclass are given, then :func:`type` is used
* if an explicit metaclass is given and it is *not* an instance of
:func:`type`, then it is used directly as the metaclass
* if an instance of :func:`type` is given as the explicit metaclass, or
bases are defined, then the most derived metaclass is used
* Otherwise, the default metaclass (:class:`type`) is used.
The most derived metaclass is selected from the explicitly specified
metaclass (if any) and the metaclasses (i.e. ``type(cls)``) of all specified
base classes. The most derived metaclass is one which is a subtype of *all*
of these candidate metaclasses. If none of the candidate metaclasses meets
that criterion, then the class definition will fail with ``TypeError``.
Preparing the class namespace
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Once the appropriate metaclass has been identified, then the class namespace
is prepared. If the metaclass has a ``__prepare__`` attribute, it is called
as ``namespace = metaclass.__prepare__(name, bases, **kwds)`` (where the
additional keyword arguments, if any, come from the class definition).
If the metaclass has no ``__prepare__`` attribute, then the class namespace
is initialised as an empty :func:`dict` instance.
.. seealso::
:pep:`3115` - Metaclasses in Python 3000
Introduced the ``__prepare__`` namespace hook
Executing the class body
^^^^^^^^^^^^^^^^^^^^^^^^
The class body is executed (approximately) as
``exec(body, globals(), namespace)``. The key difference from a normal
call to :func:`exec` is that lexical scoping allows the class body (including
any methods) to reference names from the current and outer scopes when the
class definition occurs inside a function.
However, even when the class definition occurs inside the function, methods
defined inside the class still cannot see names defined at the class scope.
Class variables must be accessed through the first parameter of instance or
class methods, and cannot be accessed at all from static methods.
Creating the class object
^^^^^^^^^^^^^^^^^^^^^^^^^
Once the class namespace has been populated by executing the class body,
the class object is created by calling
``metaclass(name, bases, namespace, **kwds)`` (the additional keywords
passed here are the same as those passed to ``__prepate__``).
This class object is the one that will be referenced by the zero-argument
form of :func:`super`. ``__class__`` is an implicit closure reference
created by the compiler if any methods in a class body refer to either
``__class__`` or ``super``. This allows the zero argument form of
:func:`super` to correctly identify the class being defined based on
lexical scoping, while the class or instance that was used to make the
current call is identified based on the first argument passed to the method.
After the class object is created, any class decorators included in the
function definition are invoked and the resulting object is bound in the
local namespace to the name of the class.
.. seealso::
:pep:`3135` - New super
Describes the implicit ``__class__`` closure reference
Metaclass example
^^^^^^^^^^^^^^^^^
The potential uses for metaclasses are boundless. Some ideas that have been
explored including logging, interface checking, automatic delegation, automatic
explored include logging, interface checking, automatic delegation, automatic
property creation, proxies, frameworks, and automatic resource
locking/synchronization.
@ -1609,9 +1671,9 @@ to remember the order that class members were defined::
def __prepare__(metacls, name, bases, **kwds):
return collections.OrderedDict()
def __new__(cls, name, bases, classdict):
result = type.__new__(cls, name, bases, dict(classdict))
result.members = tuple(classdict)
def __new__(cls, name, bases, namespace, **kwds):
result = type.__new__(cls, name, bases, dict(namespace))
result.members = tuple(namespace)
return result
class A(metaclass=OrderedClass):

4
Doc/whatsnew/3.3.rst
View File

@ -1239,6 +1239,10 @@ Add a new :class:`types.MappingProxyType` class: Read-only proxy of a mapping.
(:issue:`14386`)
The new functions `types.new_class` and `types.prepare_class` provide support
for PEP 3115 compliant dynamic type creation. (:issue:`14588`)
urllib
------

251
Lib/test/test_types.py
View File

@ -747,8 +747,257 @@ class MappingProxyTests(unittest.TestCase):
self.assertEqual(copy['key1'], 27)
class ClassCreationTests(unittest.TestCase):
class Meta(type):
def __init__(cls, name, bases, ns, **kw):
super().__init__(name, bases, ns)
@staticmethod
def __new__(mcls, name, bases, ns, **kw):
return super().__new__(mcls, name, bases, ns)
@classmethod
def __prepare__(mcls, name, bases, **kw):
ns = super().__prepare__(name, bases)
ns["y"] = 1
ns.update(kw)
return ns
def test_new_class_basics(self):
C = types.new_class("C")
self.assertEqual(C.__name__, "C")
self.assertEqual(C.__bases__, (object,))
def test_new_class_subclass(self):
C = types.new_class("C", (int,))
self.assertTrue(issubclass(C, int))
def test_new_class_meta(self):
Meta = self.Meta
settings = {"metaclass": Meta, "z": 2}
# We do this twice to make sure the passed in dict isn't mutated
for i in range(2):
C = types.new_class("C" + str(i), (), settings)
self.assertIsInstance(C, Meta)
self.assertEqual(C.y, 1)
self.assertEqual(C.z, 2)
def test_new_class_exec_body(self):
Meta = self.Meta
def func(ns):
ns["x"] = 0
C = types.new_class("C", (), {"metaclass": Meta, "z": 2}, func)
self.assertIsInstance(C, Meta)
self.assertEqual(C.x, 0)
self.assertEqual(C.y, 1)
self.assertEqual(C.z, 2)
def test_new_class_exec_body(self):
#Test that keywords are passed to the metaclass:
def meta_func(name, bases, ns, **kw):
return name, bases, ns, kw
res = types.new_class("X",
(int, object),
dict(metaclass=meta_func, x=0))
self.assertEqual(res, ("X", (int, object), {}, {"x": 0}))
def test_new_class_defaults(self):
# Test defaults/keywords:
C = types.new_class("C", (), {}, None)
self.assertEqual(C.__name__, "C")
self.assertEqual(C.__bases__, (object,))
def test_new_class_meta_with_base(self):
Meta = self.Meta
def func(ns):
ns["x"] = 0
C = types.new_class(name="C",
bases=(int,),
kwds=dict(metaclass=Meta, z=2),
exec_body=func)
self.assertTrue(issubclass(C, int))
self.assertIsInstance(C, Meta)
self.assertEqual(C.x, 0)
self.assertEqual(C.y, 1)
self.assertEqual(C.z, 2)
# Many of the following tests are derived from test_descr.py
def test_prepare_class(self):
# Basic test of metaclass derivation
expected_ns = {}
class A(type):
def __new__(*args, **kwargs):
return type.__new__(*args, **kwargs)
def __prepare__(*args):
return expected_ns
B = types.new_class("B", (object,))
C = types.new_class("C", (object,), {"metaclass": A})
# The most derived metaclass of D is A rather than type.
meta, ns, kwds = types.prepare_class("D", (B, C), {"metaclass": type})
self.assertIs(meta, A)
self.assertIs(ns, expected_ns)
self.assertEqual(len(kwds), 0)
def test_metaclass_derivation(self):
# issue1294232: correct metaclass calculation
new_calls = [] # to check the order of __new__ calls
class AMeta(type):
def __new__(mcls, name, bases, ns):
new_calls.append('AMeta')
return super().__new__(mcls, name, bases, ns)
@classmethod
def __prepare__(mcls, name, bases):
return {}
class BMeta(AMeta):
def __new__(mcls, name, bases, ns):
new_calls.append('BMeta')
return super().__new__(mcls, name, bases, ns)
@classmethod
def __prepare__(mcls, name, bases):
ns = super().__prepare__(name, bases)
ns['BMeta_was_here'] = True
return ns
A = types.new_class("A", (), {"metaclass": AMeta})
self.assertEqual(new_calls, ['AMeta'])
new_calls.clear()
B = types.new_class("B", (), {"metaclass": BMeta})
# BMeta.__new__ calls AMeta.__new__ with super:
self.assertEqual(new_calls, ['BMeta', 'AMeta'])
new_calls.clear()
C = types.new_class("C", (A, B))
# The most derived metaclass is BMeta:
self.assertEqual(new_calls, ['BMeta', 'AMeta'])
new_calls.clear()
# BMeta.__prepare__ should've been called:
self.assertIn('BMeta_was_here', C.__dict__)
# The order of the bases shouldn't matter:
C2 = types.new_class("C2", (B, A))
self.assertEqual(new_calls, ['BMeta', 'AMeta'])
new_calls.clear()
self.assertIn('BMeta_was_here', C2.__dict__)
# Check correct metaclass calculation when a metaclass is declared:
D = types.new_class("D", (C,), {"metaclass": type})
self.assertEqual(new_calls, ['BMeta', 'AMeta'])
new_calls.clear()
self.assertIn('BMeta_was_here', D.__dict__)
E = types.new_class("E", (C,), {"metaclass": AMeta})
self.assertEqual(new_calls, ['BMeta', 'AMeta'])
new_calls.clear()
self.assertIn('BMeta_was_here', E.__dict__)
def test_metaclass_override_function(self):
# Special case: the given metaclass isn't a class,
# so there is no metaclass calculation.
class A(metaclass=self.Meta):
pass
marker = object()
def func(*args, **kwargs):
return marker
X = types.new_class("X", (), {"metaclass": func})
Y = types.new_class("Y", (object,), {"metaclass": func})
Z = types.new_class("Z", (A,), {"metaclass": func})
self.assertIs(marker, X)
self.assertIs(marker, Y)
self.assertIs(marker, Z)
def test_metaclass_override_callable(self):
# The given metaclass is a class,
# but not a descendant of type.
new_calls = [] # to check the order of __new__ calls
prepare_calls = [] # to track __prepare__ calls
class ANotMeta:
def __new__(mcls, *args, **kwargs):
new_calls.append('ANotMeta')
return super().__new__(mcls)
@classmethod
def __prepare__(mcls, name, bases):
prepare_calls.append('ANotMeta')
return {}
class BNotMeta(ANotMeta):
def __new__(mcls, *args, **kwargs):
new_calls.append('BNotMeta')
return super().__new__(mcls)
@classmethod
def __prepare__(mcls, name, bases):
prepare_calls.append('BNotMeta')
return super().__prepare__(name, bases)
A = types.new_class("A", (), {"metaclass": ANotMeta})
self.assertIs(ANotMeta, type(A))
self.assertEqual(prepare_calls, ['ANotMeta'])
prepare_calls.clear()
self.assertEqual(new_calls, ['ANotMeta'])
new_calls.clear()
B = types.new_class("B", (), {"metaclass": BNotMeta})
self.assertIs(BNotMeta, type(B))
self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta'])
prepare_calls.clear()
self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta'])
new_calls.clear()
C = types.new_class("C", (A, B))
self.assertIs(BNotMeta, type(C))
self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta'])
prepare_calls.clear()
self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta'])
new_calls.clear()
C2 = types.new_class("C2", (B, A))
self.assertIs(BNotMeta, type(C2))
self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta'])
prepare_calls.clear()
self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta'])
new_calls.clear()
# This is a TypeError, because of a metaclass conflict:
# BNotMeta is neither a subclass, nor a superclass of type
with self.assertRaises(TypeError):
D = types.new_class("D", (C,), {"metaclass": type})
E = types.new_class("E", (C,), {"metaclass": ANotMeta})
self.assertIs(BNotMeta, type(E))
self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta'])
prepare_calls.clear()
self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta'])
new_calls.clear()
F = types.new_class("F", (object(), C))
self.assertIs(BNotMeta, type(F))
self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta'])
prepare_calls.clear()
self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta'])
new_calls.clear()
F2 = types.new_class("F2", (C, object()))
self.assertIs(BNotMeta, type(F2))
self.assertEqual(prepare_calls, ['BNotMeta', 'ANotMeta'])
prepare_calls.clear()
self.assertEqual(new_calls, ['BNotMeta', 'ANotMeta'])
new_calls.clear()
# TypeError: BNotMeta is neither a
# subclass, nor a superclass of int
with self.assertRaises(TypeError):
X = types.new_class("X", (C, int()))
with self.assertRaises(TypeError):
X = types.new_class("X", (int(), C))
def test_main():
run_unittest(TypesTests, MappingProxyTests)
run_unittest(TypesTests, MappingProxyTests, ClassCreationTests)
if __name__ == '__main__':
test_main()

58
Lib/types.py
View File

@ -40,3 +40,61 @@ GetSetDescriptorType = type(FunctionType.__code__)
MemberDescriptorType = type(FunctionType.__globals__)
del sys, _f, _g, _C, # Not for export
# Provide a PEP 3115 compliant mechanism for class creation
def new_class(name, bases=(), kwds=None, exec_body=None):
"""Create a class object dynamically using the appropriate metaclass."""
meta, ns, kwds = prepare_class(name, bases, kwds)
if exec_body is not None:
exec_body(ns)
return meta(name, bases, ns, **kwds)
def prepare_class(name, bases=(), kwds=None):
"""Call the __prepare__ method of the appropriate metaclass.
Returns (metaclass, namespace, kwds) as a 3-tuple
*metaclass* is the appropriate metaclass
*namespace* is the prepared class namespace
*kwds* is an updated copy of the passed in kwds argument with any
'metaclass' entry removed. If no kwds argument is passed in, this will
be an empty dict.
"""
if kwds is None:
kwds = {}
else:
kwds = dict(kwds) # Don't alter the provided mapping
if 'metaclass' in kwds:
meta = kwds.pop('metaclass')
else:
if bases:
meta = type(bases[0])
else:
meta = type
if isinstance(meta, type):
# when meta is a type, we first determine the most-derived metaclass
# instead of invoking the initial candidate directly
meta = _calculate_meta(meta, bases)
if hasattr(meta, '__prepare__'):
ns = meta.__prepare__(name, bases, **kwds)
else:
ns = {}
return meta, ns, kwds
def _calculate_meta(meta, bases):
"""Calculate the most derived metaclass."""
winner = meta
for base in bases:
base_meta = type(base)
if issubclass(winner, base_meta):
continue
if issubclass(base_meta, winner):
winner = base_meta
continue
# else:
raise TypeError("metaclass conflict: "
"the metaclass of a derived class "
"must be a (non-strict) subclass "
"of the metaclasses of all its bases")
return winner

11
Misc/NEWS
View File

@ -42,6 +42,10 @@ Core and Builtins
Library
-------
- Issue #14588: The types module now provide new_class() and prepare_class()
functions to support PEP 3115 compliant dynamic class creation. Patch by
Daniel Urban and Nick Coghlan.
- Issue #13152: Allow to specify a custom tabsize for expanding tabs in
textwrap. Patch by John Feuerstein.
@ -166,6 +170,13 @@ Build
- Issue #13210: Windows build now uses VS2010, ported from VS2008.
Documentation
-------------
- Issue #14588: The language reference now accurately documents the Python 3
class definition process. Patch by Nick Coghlan.
What's New in Python 3.3.0 Alpha 3?
===================================