traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Issue #11085: Moved collections abstract base classes into a separate module 

called collections.abc, following the pattern used by importlib.abc.  For
backwards compatibility, the names continue to also be imported into the
collections module.
This commit is contained in:
Raymond Hettinger 2011-02-22 00:41:50 +00:00
parent ecc26923cd
commit 158c9c26fc
8 changed files with 163 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

139
Doc/library/collections.abc.rst Normal file
View File

@ -0,0 +1,139 @@
:mod:`collections.abc` --- Abstract Base Classes for Containers
===============================================================
.. module:: collections.abc
:synopsis: Abstract base classes for containers
.. moduleauthor:: Raymond Hettinger <python at rcn.com>
.. sectionauthor:: Raymond Hettinger <python at rcn.com>
.. testsetup:: *
from collections import *
import itertools
__name__ = '<doctest>'
**Source code:** :source:`Lib/collections/abc.py`
--------------
This module provides :term:`abstract base classes <abstract base class>` that
can be used to test whether a class provides a particular interface; for
example, whether it is hashable or whether it is a mapping.
.. versionchanged:: 3.3
Formerly, this module was part of the :mod:`collections` module.
.. _abstract-base-classes:
Collections Abstract Base Classes
---------------------------------
The collections module offers the following ABCs:
========================= ===================== ====================== ====================================================
ABC Inherits Abstract Methods Mixin Methods
========================= ===================== ====================== ====================================================
:class:`Container` ``__contains__``
:class:`Hashable` ``__hash__``
:class:`Iterable` ``__iter__``
:class:`Iterator` :class:`Iterable` ``__next__`` ``__iter__``
:class:`Sized` ``__len__``
:class:`Callable` ``__call__``
:class:`Sequence` :class:`Sized`, ``__getitem__`` ``__contains__``, ``__iter__``, ``__reversed__``,
:class:`Iterable`, ``index``, and ``count``
:class:`Container`
:class:`MutableSequence` :class:`Sequence` ``__setitem__`` Inherited Sequence methods and
``__delitem__``, ``append``, ``reverse``, ``extend``, ``pop``,
and ``insert`` ``remove``, and ``__iadd__``
:class:`Set` :class:`Sized`, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``,
:class:`Iterable`, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``,
:class:`Container` ``__sub__``, ``__xor__``, and ``isdisjoint``
:class:`MutableSet` :class:`Set` ``add`` and Inherited Set methods and
``discard`` ``clear``, ``pop``, ``remove``, ``__ior__``,
``__iand__``, ``__ixor__``, and ``__isub__``
:class:`Mapping` :class:`Sized`, ``__getitem__`` ``__contains__``, ``keys``, ``items``, ``values``,
:class:`Iterable`, ``get``, ``__eq__``, and ``__ne__``
:class:`Container`
:class:`MutableMapping` :class:`Mapping` ``__setitem__`` and Inherited Mapping methods and
``__delitem__`` ``pop``, ``popitem``, ``clear``, ``update``,
and ``setdefault``
:class:`MappingView` :class:`Sized` ``__len__``
:class:`KeysView` :class:`MappingView`, ``__contains__``,
:class:`Set` ``__iter__``
:class:`ItemsView` :class:`MappingView`, ``__contains__``,
:class:`Set` ``__iter__``
:class:`ValuesView` :class:`MappingView` ``__contains__``, ``__iter__``
========================= ===================== ====================== ====================================================
These ABCs allow us to ask classes or instances if they provide
particular functionality, for example::
size = None
if isinstance(myvar, collections.Sized):
size = len(myvar)
Several of the ABCs are also useful as mixins that make it easier to develop
classes supporting container APIs. For example, to write a class supporting
the full :class:`Set` API, it only necessary to supply the three underlying
abstract methods: :meth:`__contains__`, :meth:`__iter__`, and :meth:`__len__`.
The ABC supplies the remaining methods such as :meth:`__and__` and
:meth:`isdisjoint` ::
class ListBasedSet(collections.Set):
''' Alternate set implementation favoring space over speed
and not requiring the set elements to be hashable. '''
def __init__(self, iterable):
self.elements = lst = []
for value in iterable:
if value not in lst:
lst.append(value)
def __iter__(self):
return iter(self.elements)
def __contains__(self, value):
return value in self.elements
def __len__(self):
return len(self.elements)
s1 = ListBasedSet('abcdef')
s2 = ListBasedSet('defghi')
overlap = s1 & s2 # The __and__() method is supported automatically
Notes on using :class:`Set` and :class:`MutableSet` as a mixin:
(1)
Since some set operations create new sets, the default mixin methods need
a way to create new instances from an iterable. The class constructor is
assumed to have a signature in the form ``ClassName(iterable)``.
That assumption is factored-out to an internal classmethod called
:meth:`_from_iterable` which calls ``cls(iterable)`` to produce a new set.
If the :class:`Set` mixin is being used in a class with a different
constructor signature, you will need to override :meth:`from_iterable`
with a classmethod that can construct new instances from
an iterable argument.
(2)
To override the comparisons (presumably for speed, as the
semantics are fixed), redefine :meth:`__le__` and
then the other operations will automatically follow suit.
(3)
The :class:`Set` mixin provides a :meth:`_hash` method to compute a hash value
for the set; however, :meth:`__hash__` is not defined because not all sets
are hashable or immutable. To add set hashabilty using mixins,
inherit from both :meth:`Set` and :meth:`Hashable`, then define
``__hash__ = Set._hash``.
.. seealso::
* `OrderedSet recipe <http://code.activestate.com/recipes/576694/>`_ that uses
:class:`MutableSet`.
* For more about ABCs, see the :mod:`abc` module and :pep:`3119`.

127
Doc/library/collections.rst
View File

@ -12,7 +12,7 @@
import itertools
__name__ = '<doctest>'
**Source code:** :source:`Lib/collections.py` and :source:`Lib/_abcoll.py`
**Source code:** :source:`Lib/collections/__init__.py`
--------------
@ -31,9 +31,10 @@ Python's general purpose built-in containers, :class:`dict`, :class:`list`,
:class:`UserString` wrapper around string objects for easier string subclassing
===================== ====================================================================
In addition to the concrete container classes, the collections module provides
:ref:`abstract-base-classes` that can be used to test whether a class provides a
particular interface, for example, whether it is hashable or a mapping.
.. versionchanged:: 3.3
Moved :ref:`abstract-base-classes` to the :mod:`collections.abc` module.
For backwards compatibility, they continue to be visible in this module
as well.
:class:`Counter` objects
@ -957,121 +958,3 @@ attribute.
be an instance of :class:`bytes`, :class:`str`, :class:`UserString` (or a
subclass) or an arbitrary sequence which can be converted into a string using
the built-in :func:`str` function.
.. _abstract-base-classes:
ABCs - abstract base classes
----------------------------
The collections module offers the following ABCs:
========================= ===================== ====================== ====================================================
ABC Inherits Abstract Methods Mixin Methods
========================= ===================== ====================== ====================================================
:class:`Container` ``__contains__``
:class:`Hashable` ``__hash__``
:class:`Iterable` ``__iter__``
:class:`Iterator` :class:`Iterable` ``__next__`` ``__iter__``
:class:`Sized` ``__len__``
:class:`Callable` ``__call__``
:class:`Sequence` :class:`Sized`, ``__getitem__`` ``__contains__``, ``__iter__``, ``__reversed__``,
:class:`Iterable`, ``index``, and ``count``
:class:`Container`
:class:`MutableSequence` :class:`Sequence` ``__setitem__`` Inherited Sequence methods and
``__delitem__``, ``append``, ``reverse``, ``extend``, ``pop``,
and ``insert`` ``remove``, and ``__iadd__``
:class:`Set` :class:`Sized`, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``,
:class:`Iterable`, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``,
:class:`Container` ``__sub__``, ``__xor__``, and ``isdisjoint``
:class:`MutableSet` :class:`Set` ``add`` and Inherited Set methods and
``discard`` ``clear``, ``pop``, ``remove``, ``__ior__``,
``__iand__``, ``__ixor__``, and ``__isub__``
:class:`Mapping` :class:`Sized`, ``__getitem__`` ``__contains__``, ``keys``, ``items``, ``values``,
:class:`Iterable`, ``get``, ``__eq__``, and ``__ne__``
:class:`Container`
:class:`MutableMapping` :class:`Mapping` ``__setitem__`` and Inherited Mapping methods and
``__delitem__`` ``pop``, ``popitem``, ``clear``, ``update``,
and ``setdefault``
:class:`MappingView` :class:`Sized` ``__len__``
:class:`KeysView` :class:`MappingView`, ``__contains__``,
:class:`Set` ``__iter__``
:class:`ItemsView` :class:`MappingView`, ``__contains__``,
:class:`Set` ``__iter__``
:class:`ValuesView` :class:`MappingView` ``__contains__``, ``__iter__``
========================= ===================== ====================== ====================================================
These ABCs allow us to ask classes or instances if they provide
particular functionality, for example::
size = None
if isinstance(myvar, collections.Sized):
size = len(myvar)
Several of the ABCs are also useful as mixins that make it easier to develop
classes supporting container APIs. For example, to write a class supporting
the full :class:`Set` API, it only necessary to supply the three underlying
abstract methods: :meth:`__contains__`, :meth:`__iter__`, and :meth:`__len__`.
The ABC supplies the remaining methods such as :meth:`__and__` and
:meth:`isdisjoint` ::
class ListBasedSet(collections.Set):
''' Alternate set implementation favoring space over speed
and not requiring the set elements to be hashable. '''
def __init__(self, iterable):
self.elements = lst = []
for value in iterable:
if value not in lst:
lst.append(value)
def __iter__(self):
return iter(self.elements)
def __contains__(self, value):
return value in self.elements
def __len__(self):
return len(self.elements)
s1 = ListBasedSet('abcdef')
s2 = ListBasedSet('defghi')
overlap = s1 & s2 # The __and__() method is supported automatically
Notes on using :class:`Set` and :class:`MutableSet` as a mixin:
(1)
Since some set operations create new sets, the default mixin methods need
a way to create new instances from an iterable. The class constructor is
assumed to have a signature in the form ``ClassName(iterable)``.
That assumption is factored-out to an internal classmethod called
:meth:`_from_iterable` which calls ``cls(iterable)`` to produce a new set.
If the :class:`Set` mixin is being used in a class with a different
constructor signature, you will need to override :meth:`from_iterable`
with a classmethod that can construct new instances from
an iterable argument.
(2)
To override the comparisons (presumably for speed, as the
semantics are fixed), redefine :meth:`__le__` and
then the other operations will automatically follow suit.
(3)
The :class:`Set` mixin provides a :meth:`_hash` method to compute a hash value
for the set; however, :meth:`__hash__` is not defined because not all sets
are hashable or immutable. To add set hashabilty using mixins,
inherit from both :meth:`Set` and :meth:`Hashable`, then define
``__hash__ = Set._hash``.
.. seealso::
* Latest version of the :source:`Python source code for the collections
abstract base classes <Lib/_abcoll.py>`
* `OrderedSet recipe <http://code.activestate.com/recipes/576694/>`_ for an
example built on :class:`MutableSet`.
* For more about ABCs, see the :mod:`abc` module and :pep:`3119`.

1
Doc/library/datatypes.rst
View File

@ -21,6 +21,7 @@ The following modules are documented in this chapter:
datetime.rst
calendar.rst
collections.rst
collections.abc.rst
heapq.rst
bisect.rst
array.rst

11
Lib/collections.py → Lib/collections/__init__.py
View File

@ -1,10 +1,11 @@
__all__ = ['deque', 'defaultdict', 'namedtuple', 'UserDict', 'UserList',
'UserString', 'Counter', 'OrderedDict']
# For bootstrapping reasons, the collection ABCs are defined in _abcoll.py.
# They should however be considered an integral part of collections.py.
from _abcoll import *
import _abcoll
__all__ += _abcoll.__all__
# For backwards compatability, continue to make the collections ABCs
# available through the collections module.
from collections.abc import *
import collections.abc
__all__ += collections.abc.__all__
from _collections import deque, defaultdict
from operator import itemgetter as _itemgetter

4
Lib/_abcoll.py → Lib/collections/abc.py
View File

@ -3,9 +3,7 @@
"""Abstract Base Classes (ABCs) for collections, according to PEP 3119.
DON'T USE THIS MODULE DIRECTLY! The classes here should be imported
via collections; they are defined here only to alleviate certain
bootstrapping issues. Unit tests are in test_collections.
Unit tests are in test_collections.
"""
from abc import ABCMeta, abstractmethod

2
Lib/os.py
View File

@ -434,7 +434,7 @@ def get_exec_path(env=None):
# Change environ to automatically call putenv(), unsetenv if they exist.
from _abcoll import MutableMapping # Can't use collections (bootstrap)
from collections.abc import MutableMapping
class _Environ(MutableMapping):
def __init__(self, data, encodekey, decodekey, encodevalue, decodevalue, putenv, unsetenv):

9
Lib/test/regrtest.py
View File

@ -1056,7 +1056,8 @@ def dash_R(the_module, test, indirect_test, huntrleaks):
False if the test didn't leak references; True if we detected refleaks.
"""
# This code is hackish and inelegant, but it seems to do the job.
import copyreg, _abcoll
import copyreg
import collections.abc
if not hasattr(sys, 'gettotalrefcount'):
raise Exception("Tracking reference leaks requires a debug build "
@ -1073,7 +1074,7 @@ def dash_R(the_module, test, indirect_test, huntrleaks):
else:
zdc = zipimport._zip_directory_cache.copy()
abcs = {}
for abc in [getattr(_abcoll, a) for a in _abcoll.__all__]:
for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]:
if not isabstract(abc):
continue
for obj in abc.__subclasses__() + [abc]:
@ -1119,7 +1120,7 @@ def dash_R_cleanup(fs, ps, pic, zdc, abcs):
import gc, copyreg
import _strptime, linecache
import urllib.parse, urllib.request, mimetypes, doctest
import struct, filecmp, _abcoll
import struct, filecmp, collections.abc
from distutils.dir_util import _path_created
from weakref import WeakSet
@ -1146,7 +1147,7 @@ def dash_R_cleanup(fs, ps, pic, zdc, abcs):
sys._clear_type_cache()
# Clear ABC registries, restoring previously saved ABC registries.
for abc in [getattr(_abcoll, a) for a in _abcoll.__all__]:
for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]:
if not isabstract(abc):
continue
for obj in abc.__subclasses__() + [abc]:

5
Misc/NEWS
View File

@ -27,6 +27,11 @@ Core and Builtins
Library
-------
- Issue #11085: Moved collections abstract base classes into a separate
module called collections.abc, following the pattern used by importlib.abc.
For backwards compatibility, the names are imported into the collections
module.
- Issue #4681: Allow mmap() to work on file sizes and offsets larger than
4GB, even on 32-bit builds. Initial patch by Ross Lagerwall, adapted for
32-bit Windows.