Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity

gh-97781: Apply changes from importlib_metadata 5. (GH-97785) 

* gh-97781: Apply changes from importlib_metadata 5.

* Apply changes from upstream

* Apply changes from upstream.
This commit is contained in:
Jason R. Coombs 2022-10-06 15:25:24 -04:00 committed by GitHub
parent 2b5f1360ea
commit 8af04cdef2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 88 additions and 302 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

108
Doc/library/importlib.metadata.rst
View File

@ -13,21 +13,39 @@
**Source code:** :source:`Lib/importlib/metadata/__init__.py`
``importlib.metadata`` is a library that provides access to installed
package metadata, such as its entry points or its
top-level name. Built in part on Python's import system, this library
``importlib_metadata`` is a library that provides access to
the metadata of an installed `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
such as its entry points
or its top-level names (`Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_\s, modules, if any).
Built in part on Python's import system, this library
intends to replace similar functionality in the `entry point
API`_ and `metadata API`_ of ``pkg_resources``. Along with
:mod:`importlib.resources`,
this package can eliminate the need to use the older and less efficient
``pkg_resources`` package.
By "installed package" we generally mean a third-party package installed into
Python's ``site-packages`` directory via tools such as `pip
<https://pypi.org/project/pip/>`_. Specifically,
it means a package with either a discoverable ``dist-info`` or ``egg-info``
directory, and metadata defined by :pep:`566` or its older specifications.
By default, package metadata can live on the file system or in zip archives on
``importlib_metadata`` operates on third-party *distribution packages*
installed into Python's ``site-packages`` directory via tools such as
`pip <https://pypi.org/project/pip/>`_.
Specifically, it works with distributions with discoverable
``dist-info`` or ``egg-info`` directories,
and metadata defined by the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_.
.. important::
These are *not* necessarily equivalent to or correspond 1:1 with
the top-level *import package* names
that can be imported inside Python code.
One *distribution package* can contain multiple *import packages*
(and single modules),
and one top-level *import package*
may map to multiple *distribution packages*
if it is a namespace package.
You can use :ref:`package_distributions() <package-distributions>`
to get a mapping between them.
By default, distribution metadata can live on the file system
or in zip archives on
:data:`sys.path`. Through an extension mechanism, the metadata can live almost
anywhere.
@ -37,12 +55,19 @@ anywhere.
https://importlib-metadata.readthedocs.io/
The documentation for ``importlib_metadata``, which supplies a
backport of ``importlib.metadata``.
This includes an `API reference
<https://importlib-metadata.readthedocs.io/en/latest/api.html>`__
for this module's classes and functions,
as well as a `migration guide
<https://importlib-metadata.readthedocs.io/en/latest/migration.html>`__
for existing users of ``pkg_resources``.
Overview
========
Let's say you wanted to get the version string for a package you've installed
Let's say you wanted to get the version string for a
`Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ you've installed
using ``pip``. We start by creating a virtual environment and installing
something into it:
@ -151,11 +176,10 @@ for more information on entry points, their definition, and usage.
The "selectable" entry points were introduced in ``importlib_metadata``
3.6 and Python 3.10. Prior to those changes, ``entry_points`` accepted
no parameters and always returned a dictionary of entry points, keyed
by group. For compatibility, if no parameters are passed to entry_points,
a ``SelectableGroups`` object is returned, implementing that dict
interface. In the future, calling ``entry_points`` with no parameters
will return an ``EntryPoints`` object. Users should rely on the selection
interface to retrieve entry points by group.
by group. With ``importlib_metadata`` 5.0 and Python 3.12,
``entry_points`` always returns an ``EntryPoints`` object. See
`backports.entry_points_selectable <https://pypi.org/project/backports.entry_points_selectable>`_
for compatibility options.
.. _metadata:
@ -163,7 +187,8 @@ interface to retrieve entry points by group.
Distribution metadata
---------------------
Every distribution includes some metadata, which you can extract using the
Every `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ includes some metadata,
which you can extract using the
``metadata()`` function::
>>> wheel_metadata = metadata('wheel') # doctest: +SKIP
@ -201,7 +226,8 @@ all the metadata in a JSON-compatible form per :PEP:`566`::
Distribution versions
---------------------
The ``version()`` function is the quickest way to get a distribution's version
The ``version()`` function is the quickest way to get a
`Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_'s version
number, as a string::
>>> version('wheel') # doctest: +SKIP
@ -214,7 +240,8 @@ Distribution files
------------------
You can also get the full set of files contained within a distribution. The
``files()`` function takes a distribution package name and returns all of the
``files()`` function takes a `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ name
and returns all of the
files installed by this distribution. Each file object returned is a
``PackagePath``, a :class:`pathlib.PurePath` derived object with additional ``dist``,
``size``, and ``hash`` properties as indicated by the metadata. For example::
@ -259,19 +286,24 @@ distribution is not known to have the metadata present.
Distribution requirements
-------------------------
To get the full set of requirements for a distribution, use the ``requires()``
To get the full set of requirements for a `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
use the ``requires()``
function::
>>> requires('wheel') # doctest: +SKIP
["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"]
Package distributions
---------------------
.. _package-distributions:
.. _import-distribution-package-mapping:
A convenience method to resolve the distribution or
distributions (in the case of a namespace package) for top-level
Python packages or modules::
Mapping import to distribution packages
---------------------------------------
A convenience method to resolve the `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_
name (or names, in the case of a namespace package)
that provide each importable top-level
Python module or `Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_::
>>> packages_distributions()
{'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'], 'jaraco': ['jaraco.classes', 'jaraco.functools'], ...}
@ -285,7 +317,8 @@ Distributions
While the above API is the most common and convenient usage, you can get all
of that information from the ``Distribution`` class. A ``Distribution`` is an
abstract object that represents the metadata for a Python package. You can
abstract object that represents the metadata for
a Python `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_. You can
get the ``Distribution`` instance::
>>> from importlib.metadata import distribution # doctest: +SKIP
@ -305,14 +338,16 @@ instance::
>>> dist.metadata['License'] # doctest: +SKIP
'MIT'
The full set of available metadata is not described here. See :pep:`566`
for additional details.
The full set of available metadata is not described here.
See the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_ for additional details.
Distribution Discovery
======================
By default, this package provides built-in support for discovery of metadata for file system and zip file packages. This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular:
By default, this package provides built-in support for discovery of metadata
for file system and zip file `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_\s.
This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular:
- ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
- ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.
@ -321,15 +356,18 @@ By default, this package provides built-in support for discovery of metadata for
Extending the search algorithm
==============================
Because package metadata is not available through :data:`sys.path` searches, or
package loaders directly, the metadata for a package is found through import
system :ref:`finders <finders-and-loaders>`. To find a distribution package's metadata,
Because `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ metadata
is not available through :data:`sys.path` searches, or
package loaders directly,
the metadata for a distribution is found through import
system `finders`_. To find a distribution package's metadata,
``importlib.metadata`` queries the list of :term:`meta path finders <meta path finder>` on
:data:`sys.meta_path`.
The default ``PathFinder`` for Python includes a hook that calls into
``importlib.metadata.MetadataPathFinder`` for finding distributions
loaded from typical file-system-based paths.
By default ``importlib_metadata`` installs a finder for distribution packages
found on the file system.
This finder doesn't actually find any *distributions*,
but it can find their metadata.
The abstract class :py:class:`importlib.abc.MetaPathFinder` defines the
interface expected of finders by Python's import system.
@ -358,4 +396,4 @@ a custom finder, return instances of this derived ``Distribution`` in the
.. _`entry point API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
.. _`metadata API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api
.. _`importlib_resources`: https://importlib-resources.readthedocs.io/en/latest/index.html
.. _`finders`: https://docs.python.org/3/reference/import.html#finders-and-loaders

211
Lib/importlib/metadata/__init__.py
View File

@ -24,7 +24,7 @@ from contextlib import suppress
from importlib import import_module
from importlib.abc import MetaPathFinder
from itertools import starmap
from typing import List, Mapping, Optional, Union
from typing import List, Mapping, Optional
__all__ = [
@ -134,6 +134,7 @@ class DeprecatedTuple:
1
"""
# Do not remove prior to 2023-05-01 or Python 3.13
_warn = functools.partial(
warnings.warn,
"EntryPoint tuple interface is deprecated. Access members by name.",
@ -184,6 +185,10 @@ class EntryPoint(DeprecatedTuple):
following the attr, and following any extras.
"""
name: str
value: str
group: str
dist: Optional['Distribution'] = None
def __init__(self, name, value, group):
@ -218,17 +223,6 @@ class EntryPoint(DeprecatedTuple):
vars(self).update(dist=dist)
return self
def __iter__(self):
"""
Supply iter so one may construct dicts of EntryPoints by name.
"""
msg = (
"Construction of dict of EntryPoints is deprecated in "
"favor of EntryPoints."
)
warnings.warn(msg, DeprecationWarning)
return iter((self.name, self))
def matches(self, **params):
"""
EntryPoint matches the given parameters.
@ -274,77 +268,7 @@ class EntryPoint(DeprecatedTuple):
return hash(self._key())
class DeprecatedList(list):
"""
Allow an otherwise immutable object to implement mutability
for compatibility.
>>> recwarn = getfixture('recwarn')
>>> dl = DeprecatedList(range(3))
>>> dl[0] = 1
>>> dl.append(3)
>>> del dl[3]
>>> dl.reverse()
>>> dl.sort()
>>> dl.extend([4])
>>> dl.pop(-1)
4
>>> dl.remove(1)
>>> dl += [5]
>>> dl + [6]
[1, 2, 5, 6]
>>> dl + (6,)
[1, 2, 5, 6]
>>> dl.insert(0, 0)
>>> dl
[0, 1, 2, 5]
>>> dl == [0, 1, 2, 5]
True
>>> dl == (0, 1, 2, 5)
True
>>> len(recwarn)
1
"""
__slots__ = ()
_warn = functools.partial(
warnings.warn,
"EntryPoints list interface is deprecated. Cast to list if needed.",
DeprecationWarning,
stacklevel=2,
)
def _wrap_deprecated_method(method_name: str): # type: ignore
def wrapped(self, *args, **kwargs):
self._warn()
return getattr(super(), method_name)(*args, **kwargs)
return method_name, wrapped
locals().update(
map(
_wrap_deprecated_method,
'__setitem__ __delitem__ append reverse extend pop remove '
'__iadd__ insert sort'.split(),
)
)
def __add__(self, other):
if not isinstance(other, tuple):
self._warn()
other = tuple(other)
return self.__class__(tuple(self) + other)
def __eq__(self, other):
if not isinstance(other, tuple):
self._warn()
other = tuple(other)
return tuple(self).__eq__(other)
class EntryPoints(DeprecatedList):
class EntryPoints(tuple):
"""
An immutable collection of selectable EntryPoint objects.
"""
@ -355,14 +279,6 @@ class EntryPoints(DeprecatedList):
"""
Get the EntryPoint in self matching name.
"""
if isinstance(name, int):
warnings.warn(
"Accessing entry points by index is deprecated. "
"Cast to tuple if needed.",
DeprecationWarning,
stacklevel=2,
)
return super().__getitem__(name)
try:
return next(iter(self.select(name=name)))
except StopIteration:
@ -386,10 +302,6 @@ class EntryPoints(DeprecatedList):
def groups(self):
"""
Return the set of all groups of all entry points.
For coverage while SelectableGroups is present.
>>> EntryPoints().groups
set()
"""
return {ep.group for ep in self}
@ -405,101 +317,6 @@ class EntryPoints(DeprecatedList):
)
class Deprecated:
"""
Compatibility add-in for mapping to indicate that
mapping behavior is deprecated.
>>> recwarn = getfixture('recwarn')
>>> class DeprecatedDict(Deprecated, dict): pass
>>> dd = DeprecatedDict(foo='bar')
>>> dd.get('baz', None)
>>> dd['foo']
'bar'
>>> list(dd)
['foo']
>>> list(dd.keys())
['foo']
>>> 'foo' in dd
True
>>> list(dd.values())
['bar']
>>> len(recwarn)
1
"""
_warn = functools.partial(
warnings.warn,
"SelectableGroups dict interface is deprecated. Use select.",
DeprecationWarning,
stacklevel=2,
)
def __getitem__(self, name):
self._warn()
return super().__getitem__(name)
def get(self, name, default=None):
self._warn()
return super().get(name, default)
def __iter__(self):
self._warn()
return super().__iter__()
def __contains__(self, *args):
self._warn()
return super().__contains__(*args)
def keys(self):
self._warn()
return super().keys()
def values(self):
self._warn()
return super().values()
class SelectableGroups(Deprecated, dict):
"""
A backward- and forward-compatible result from
entry_points that fully implements the dict interface.
"""
@classmethod
def load(cls, eps):
by_group = operator.attrgetter('group')
ordered = sorted(eps, key=by_group)
grouped = itertools.groupby(ordered, by_group)
return cls((group, EntryPoints(eps)) for group, eps in grouped)
@property
def _all(self):
"""
Reconstruct a list of all entrypoints from the groups.
"""
groups = super(Deprecated, self).values()
return EntryPoints(itertools.chain.from_iterable(groups))
@property
def groups(self):
return self._all.groups
@property
def names(self):
"""
for coverage:
>>> SelectableGroups().names
set()
"""
return self._all.names
def select(self, **params):
if not params:
return self
return self._all.select(**params)
class PackagePath(pathlib.PurePosixPath):
"""A reference to a path in a package"""
@ -1013,27 +830,19 @@ Wrapper for ``distributions`` to return unique distributions by name.
"""
def entry_points(**params) -> Union[EntryPoints, SelectableGroups]:
def entry_points(**params) -> EntryPoints:
"""Return EntryPoint objects for all installed packages.
Pass selection parameters (group or name) to filter the
result to entry points matching those properties (see
EntryPoints.select()).
For compatibility, returns ``SelectableGroups`` object unless
selection parameters are supplied. In the future, this function
will return ``EntryPoints`` instead of ``SelectableGroups``
even when no selection parameters are supplied.
For maximum future compatibility, pass selection parameters
or invoke ``.select`` with parameters on the result.
:return: EntryPoints or SelectableGroups for all installed packages.
:return: EntryPoints for all installed packages.
"""
eps = itertools.chain.from_iterable(
dist.entry_points for dist in _unique(distributions())
)
return SelectableGroups.load(eps).select(**params)
return EntryPoints(eps).select(**params)
def files(distribution_name):

10
Lib/test/test_importlib/test_main.py
View File

@ -1,8 +1,6 @@
import re
import json
import pickle
import unittest
import warnings
import importlib.metadata
try:
@ -260,14 +258,6 @@ class TestEntryPoints(unittest.TestCase):
"""EntryPoints should be hashable"""
hash(self.ep)
def test_json_dump(self):
"""
json should not expect to be able to dump an EntryPoint
"""
with self.assertRaises(Exception):
with warnings.catch_warnings(record=True):
json.dumps(self.ep)
def test_module(self):
assert self.ep.module == 'value'

56
Lib/test/test_importlib/test_metadata_api.py
View File

@ -124,62 +124,6 @@ class APITests(
def test_entry_points_missing_group(self):
assert entry_points(group='missing') == ()
def test_entry_points_dict_construction(self):
"""
Prior versions of entry_points() returned simple lists and
allowed casting those lists into maps by name using ``dict()``.
Capture this now deprecated use-case.
"""
with suppress_known_deprecation() as caught:
eps = dict(entry_points(group='entries'))
assert 'main' in eps
assert eps['main'] == entry_points(group='entries')['main']
# check warning
expected = next(iter(caught))
assert expected.category is DeprecationWarning
assert "Construction of dict of EntryPoints is deprecated" in str(expected)
def test_entry_points_by_index(self):
"""
Prior versions of Distribution.entry_points would return a
tuple that allowed access by index.
Capture this now deprecated use-case
See python/importlib_metadata#300 and bpo-44246.
"""
eps = distribution('distinfo-pkg').entry_points
with suppress_known_deprecation() as caught:
eps[0]
# check warning
expected = next(iter(caught))
assert expected.category is DeprecationWarning
assert "Accessing entry points by index is deprecated" in str(expected)
def test_entry_points_groups_getitem(self):
"""
Prior versions of entry_points() returned a dict. Ensure
that callers using '.__getitem__()' are supported but warned to
migrate.
"""
with suppress_known_deprecation():
entry_points()['entries'] == entry_points(group='entries')
with self.assertRaises(KeyError):
entry_points()['missing']
def test_entry_points_groups_get(self):
"""
Prior versions of entry_points() returned a dict. Ensure
that callers using '.get()' are supported but warned to
migrate.
"""
with suppress_known_deprecation():
entry_points().get('missing', 'default') == 'default'
entry_points().get('entries', 'default') == entry_points()['entries']
entry_points().get('missing', ()) == ()
def test_entry_points_allows_no_attributes(self):
ep = entry_points().select(group='entries', name='main')
with self.assertRaises(AttributeError):

5
Misc/NEWS.d/next/Library/2022-10-03-13-25-19.gh-issue-97781.gCLLef.rst Normal file
View File

@ -0,0 +1,5 @@
Removed deprecated interfaces in ``importlib.metadata`` (entry points
accessed as dictionary, implicit dictionary construction of sequence of
``EntryPoint`` objects, mutablility of ``EntryPoints`` result, access of
entry point by index). ``entry_points`` now has a simpler, more
straightforward API (returning ``EntryPoints``).