bpo-34632: Add importlib.metadata (GH-12547)

Add importlib.metadata module as forward port of the standalone importlib_metadata.
This commit is contained in:
Jason R. Coombs 2019-05-24 19:59:01 -04:00 committed by Barry Warsaw
parent 6dbbe748e1
commit 1bbf7b661f
15 changed files with 2049 additions and 639 deletions

View File

@ -0,0 +1,257 @@
.. _using:
==========================
Using importlib.metadata
==========================
.. note::
This functionality is provisional and may deviate from the usual
version semantics of the standard library.
``importlib.metadata`` is a library that provides for access to installed
package metadata. Built in part on Python's import system, this library
intends to replace similar functionality in the `entry point
API`_ and `metadata API`_ of ``pkg_resources``. Along with
``importlib.resources`` in `Python 3.7
and newer`_ (backported as `importlib_resources`_ for older versions of
Python), this can eliminate the need to use the older and less efficient
``pkg_resources`` package.
By "installed package" we generally mean a third-party package installed into
Python's ``site-packages`` directory via tools such as `pip
<https://pypi.org/project/pip/>`_. Specifically,
it means a package with either a discoverable ``dist-info`` or ``egg-info``
directory, and metadata defined by `PEP 566`_ or its older specifications.
By default, package metadata can live on the file system or in zip archives on
``sys.path``. Through an extension mechanism, the metadata can live almost
anywhere.
Overview
========
Let's say you wanted to get the version string for a package you've installed
using ``pip``. We start by creating a virtual environment and installing
something into it::
.. highlight:: none
$ python3 -m venv example
$ source example/bin/activate
(example) $ pip install wheel
You can get the version string for ``wheel`` by running the following::
.. highlight:: none
(example) $ python
>>> from importlib.metadata import version # doctest: +SKIP
>>> version('wheel') # doctest: +SKIP
'0.32.3'
You can also get the set of entry points keyed by group, such as
``console_scripts``, ``distutils.commands`` and others. Each group contains a
sequence of :ref:`EntryPoint <entry-points>` objects.
You can get the :ref:`metadata for a distribution <metadata>`::
>>> list(metadata('wheel')) # doctest: +SKIP
['Metadata-Version', 'Name', 'Version', 'Summary', 'Home-page', 'Author', 'Author-email', 'Maintainer', 'Maintainer-email', 'License', 'Project-URL', 'Project-URL', 'Project-URL', 'Keywords', 'Platform', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Classifier', 'Requires-Python', 'Provides-Extra', 'Requires-Dist', 'Requires-Dist']
You can also get a :ref:`distribution's version number <version>`, list its
:ref:`constituent files <files>`, and get a list of the distribution's
:ref:`requirements`.
Functional API
==============
This package provides the following functionality via its public API.
.. _entry-points:
Entry points
------------
The ``entry_points()`` function returns a dictionary of all entry points,
keyed by group. Entry points are represented by ``EntryPoint`` instances;
each ``EntryPoint`` has a ``.name``, ``.group``, and ``.value`` attributes and
a ``.load()`` method to resolve the value.
>>> eps = entry_points() # doctest: +SKIP
>>> list(eps) # doctest: +SKIP
['console_scripts', 'distutils.commands', 'distutils.setup_keywords', 'egg_info.writers', 'setuptools.installation']
>>> scripts = eps['console_scripts'] # doctest: +SKIP
>>> wheel = [ep for ep in scripts if ep.name == 'wheel'][0] # doctest: +SKIP
>>> wheel # doctest: +SKIP
EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')
>>> main = wheel.load() # doctest: +SKIP
>>> main # doctest: +SKIP
<function main at 0x103528488>
The ``group`` and ``name`` are arbitrary values defined by the package author
and usually a client will wish to resolve all entry points for a particular
group. Read `the setuptools docs
<https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discovery-of-services-and-plugins>`_
for more information on entrypoints, their definition, and usage.
.. _metadata:
Distribution metadata
---------------------
Every distribution includes some metadata, which you can extract using the
``metadata()`` function::
>>> wheel_metadata = metadata('wheel') # doctest: +SKIP
The keys of the returned data structure [#f1]_ name the metadata keywords, and
their values are returned unparsed from the distribution metadata::
>>> wheel_metadata['Requires-Python'] # doctest: +SKIP
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
.. _version:
Distribution versions
---------------------
The ``version()`` function is the quickest way to get a distribution's version
number, as a string::
>>> version('wheel') # doctest: +SKIP
'0.32.3'
.. _files:
Distribution files
------------------
You can also get the full set of files contained within a distribution. The
``files()`` function takes a distribution package name and returns all of the
files installed by this distribution. Each file object returned is a
``PackagePath``, a `pathlib.Path`_ derived object with additional ``dist``,
``size``, and ``hash`` properties as indicated by the metadata. For example::
>>> util = [p for p in files('wheel') if 'util.py' in str(p)][0] # doctest: +SKIP
>>> util # doctest: +SKIP
PackagePath('wheel/util.py')
>>> util.size # doctest: +SKIP
859
>>> util.dist # doctest: +SKIP
<importlib.metadata._hooks.PathDistribution object at 0x101e0cef0>
>>> util.hash # doctest: +SKIP
<FileHash mode: sha256 value: bYkw5oMccfazVCoYQwKkkemoVyMAFoR34mmKBx8R1NI>
Once you have the file, you can also read its contents::
>>> print(util.read_text()) # doctest: +SKIP
import base64
import sys
...
def as_bytes(s):
if isinstance(s, text_type):
return s.encode('utf-8')
return s
.. _requirements:
Distribution requirements
-------------------------
To get the full set of requirements for a distribution, use the ``requires()``
function. Note that this returns an iterator::
>>> list(requires('wheel')) # doctest: +SKIP
["pytest (>=3.0.0) ; extra == 'test'"]
Distributions
=============
While the above API is the most common and convenient usage, you can get all
of that information from the ``Distribution`` class. A ``Distribution`` is an
abstract object that represents the metadata for a Python package. You can
get the ``Distribution`` instance::
>>> from importlib.metadata import distribution # doctest: +SKIP
>>> dist = distribution('wheel') # doctest: +SKIP
Thus, an alternative way to get the version number is through the
``Distribution`` instance::
>>> dist.version # doctest: +SKIP
'0.32.3'
There are all kinds of additional metadata available on the ``Distribution``
instance::
>>> d.metadata['Requires-Python'] # doctest: +SKIP
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'
>>> d.metadata['License'] # doctest: +SKIP
'MIT'
The full set of available metadata is not described here. See `PEP 566
<https://www.python.org/dev/peps/pep-0566/>`_ for additional details.
Extending the search algorithm
==============================
Because package metadata is not available through ``sys.path`` searches, or
package loaders directly, the metadata for a package is found through import
system `finders`_. To find a distribution package's metadata,
``importlib.metadata`` queries the list of `meta path finders`_ on
`sys.meta_path`_.
By default ``importlib.metadata`` installs a finder for distribution packages
found on the file system. This finder doesn't actually find any *packages*,
but it can find the packages' metadata.
The abstract class :py:class:`importlib.abc.MetaPathFinder` defines the
interface expected of finders by Python's import system.
``importlib.metadata`` extends this protocol by looking for an optional
``find_distributions`` callable on the finders from
``sys.meta_path``. If the finder has this method, it must return
an iterator over instances of the ``Distribution`` abstract class. This
method must have the signature::
def find_distributions(name=None, path=None):
"""Return an iterable of all Distribution instances capable of
loading the metadata for packages matching the name
(or all names if not supplied) along the paths in the list
of directories ``path`` (defaults to sys.path).
"""
What this means in practice is that to support finding distribution package
metadata in locations other than the file system, you should derive from
``Distribution`` and implement the ``load_metadata()`` method. This takes a
single argument which is the name of the package whose metadata is being
found. This instance of the ``Distribution`` base abstract class is what your
finder's ``find_distributions()`` method should return.
.. _`entry point API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
.. _`metadata API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api
.. _`Python 3.7 and newer`: https://docs.python.org/3/library/importlib.html#module-importlib.resources
.. _`importlib_resources`: https://importlib-resources.readthedocs.io/en/latest/index.html
.. _`PEP 566`: https://www.python.org/dev/peps/pep-0566/
.. _`finders`: https://docs.python.org/3/reference/import.html#finders-and-loaders
.. _`meta path finders`: https://docs.python.org/3/glossary.html#term-meta-path-finder
.. _`sys.meta_path`: https://docs.python.org/3/library/sys.html#sys.meta_path
.. _`pathlib.Path`: https://docs.python.org/3/library/pathlib.html#pathlib.Path
.. rubric:: Footnotes
.. [#f1] Technically, the returned distribution metadata object is an
`email.message.Message
<https://docs.python.org/3/library/email.message.html#email.message.EmailMessage>`_
instance, but this is an implementation detail, and not part of the
stable API. You should only use dictionary-like methods and syntax
to access the metadata contents.

View File

@ -17,3 +17,4 @@ The full list of modules described in this chapter is:
modulefinder.rst
runpy.rst
importlib.rst
importlib.metadata.rst

View File

@ -350,3 +350,6 @@ whatsnew/3.7,,::,error::BytesWarning
whatsnew/changelog,,::,error::BytesWarning
whatsnew/changelog,,::,default::BytesWarning
whatsnew/changelog,,::,default::DeprecationWarning
library/importlib.metadata,,.. highlight:,.. highlight:: none
library/importlib.metadata,,:main,"EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')"
library/importlib.metadata,,`,of directories ``path`` (defaults to sys.path).

1 c-api/arg :ref PyArg_ParseTuple(args, "O|O:ref", &object, &callback)
350 whatsnew/changelog :: error::BytesWarning
351 whatsnew/changelog :: default::BytesWarning
352 whatsnew/changelog :: default::DeprecationWarning
353 library/importlib.metadata .. highlight: .. highlight:: none
354 library/importlib.metadata :main EntryPoint(name='wheel', value='wheel.cli:main', group='console_scripts')
355 library/importlib.metadata ` of directories ``path`` (defaults to sys.path).

View File

@ -1363,6 +1363,58 @@ class PathFinder:
return None
return spec.loader
search_template = r'(?:{pattern}(-.*)?\.(dist|egg)-info|EGG-INFO)'
@classmethod
def find_distributions(cls, name=None, path=None):
"""
Find distributions.
Return an iterable of all Distribution instances capable of
loading the metadata for packages matching the ``name``
(or all names if not supplied) along the paths in the list
of directories ``path`` (defaults to sys.path).
"""
import re
from importlib.metadata import PathDistribution
if path is None:
path = sys.path
pattern = '.*' if name is None else re.escape(name)
found = cls._search_paths(pattern, path)
return map(PathDistribution, found)
@classmethod
def _search_paths(cls, pattern, paths):
"""Find metadata directories in paths heuristically."""
import itertools
return itertools.chain.from_iterable(
cls._search_path(path, pattern)
for path in map(cls._switch_path, paths)
)
@staticmethod
def _switch_path(path):
from contextlib import suppress
import zipfile
from pathlib import Path
with suppress(Exception):
return zipfile.Path(path)
return Path(path)
@classmethod
def _predicate(cls, pattern, root, item):
import re
return re.match(pattern, str(item.name), flags=re.IGNORECASE)
@classmethod
def _search_path(cls, root, pattern):
if not root.is_dir():
return ()
normalized = pattern.replace('-', '_')
matcher = cls.search_template.format(pattern=normalized)
return (item for item in root.iterdir()
if cls._predicate(matcher, root, item))
class FileFinder:

View File

@ -0,0 +1,394 @@
import io
import re
import abc
import csv
import sys
import email
import pathlib
import operator
import functools
import itertools
import collections
from configparser import ConfigParser
from contextlib import suppress
from importlib import import_module
from importlib.abc import MetaPathFinder
from itertools import starmap
__all__ = [
'Distribution',
'PackageNotFoundError',
'distribution',
'distributions',
'entry_points',
'files',
'metadata',
'requires',
'version',
]
class PackageNotFoundError(ModuleNotFoundError):
"""The package was not found."""
class EntryPoint(collections.namedtuple('EntryPointBase', 'name value group')):
"""An entry point as defined by Python packaging conventions.
See `the packaging docs on entry points
<https://packaging.python.org/specifications/entry-points/>`_
for more information.
"""
pattern = re.compile(
r'(?P<module>[\w.]+)\s*'
r'(:\s*(?P<attr>[\w.]+))?\s*'
r'(?P<extras>\[.*\])?\s*$'
)
"""
A regular expression describing the syntax for an entry point,
which might look like:
- module
- package.module
- package.module:attribute
- package.module:object.attribute
- package.module:attr [extra1, extra2]
Other combinations are possible as well.
The expression is lenient about whitespace around the ':',
following the attr, and following any extras.
"""
def load(self):
"""Load the entry point from its definition. If only a module
is indicated by the value, return that module. Otherwise,
return the named object.
"""
match = self.pattern.match(self.value)
module = import_module(match.group('module'))
attrs = filter(None, (match.group('attr') or '').split('.'))
return functools.reduce(getattr, attrs, module)
@property
def extras(self):
match = self.pattern.match(self.value)
return list(re.finditer(r'\w+', match.group('extras') or ''))
@classmethod
def _from_config(cls, config):
return [
cls(name, value, group)
for group in config.sections()
for name, value in config.items(group)
]
@classmethod
def _from_text(cls, text):
config = ConfigParser()
try:
config.read_string(text)
except AttributeError: # pragma: nocover
# Python 2 has no read_string
config.readfp(io.StringIO(text))
return EntryPoint._from_config(config)
def __iter__(self):
"""
Supply iter so one may construct dicts of EntryPoints easily.
"""
return iter((self.name, self))
class PackagePath(pathlib.PurePosixPath):
"""A reference to a path in a package"""
def read_text(self, encoding='utf-8'):
with self.locate().open(encoding=encoding) as stream:
return stream.read()
def read_binary(self):
with self.locate().open('rb') as stream:
return stream.read()
def locate(self):
"""Return a path-like object for this path"""
return self.dist.locate_file(self)
class FileHash:
def __init__(self, spec):
self.mode, _, self.value = spec.partition('=')
def __repr__(self):
return '<FileHash mode: {} value: {}>'.format(self.mode, self.value)
class Distribution:
"""A Python distribution package."""
@abc.abstractmethod
def read_text(self, filename):
"""Attempt to load metadata file given by the name.
:param filename: The name of the file in the distribution info.
:return: The text if found, otherwise None.
"""
@abc.abstractmethod
def locate_file(self, path):
"""
Given a path to a file in this distribution, return a path
to it.
"""
@classmethod
def from_name(cls, name):
"""Return the Distribution for the given package name.
:param name: The name of the distribution package to search for.
:return: The Distribution instance (or subclass thereof) for the named
package, if found.
:raises PackageNotFoundError: When the named package's distribution
metadata cannot be found.
"""
for resolver in cls._discover_resolvers():
dists = resolver(name)
dist = next(dists, None)
if dist is not None:
return dist
else:
raise PackageNotFoundError(name)
@classmethod
def discover(cls):
"""Return an iterable of Distribution objects for all packages.
:return: Iterable of Distribution objects for all packages.
"""
return itertools.chain.from_iterable(
resolver()
for resolver in cls._discover_resolvers()
)
@staticmethod
def _discover_resolvers():
"""Search the meta_path for resolvers."""
declared = (
getattr(finder, 'find_distributions', None)
for finder in sys.meta_path
)
return filter(None, declared)
@property
def metadata(self):
"""Return the parsed metadata for this Distribution.
The returned object will have keys that name the various bits of
metadata. See PEP 566 for details.
"""
text = (
self.read_text('METADATA')
or self.read_text('PKG-INFO')
# This last clause is here to support old egg-info files. Its
# effect is to just end up using the PathDistribution's self._path
# (which points to the egg-info file) attribute unchanged.
or self.read_text('')
)
return email.message_from_string(text)
@property
def version(self):
"""Return the 'Version' metadata for the distribution package."""
return self.metadata['Version']
@property
def entry_points(self):
return EntryPoint._from_text(self.read_text('entry_points.txt'))
@property
def files(self):
file_lines = self._read_files_distinfo() or self._read_files_egginfo()
def make_file(name, hash=None, size_str=None):
result = PackagePath(name)
result.hash = FileHash(hash) if hash else None
result.size = int(size_str) if size_str else None
result.dist = self
return result
return file_lines and starmap(make_file, csv.reader(file_lines))
def _read_files_distinfo(self):
"""
Read the lines of RECORD
"""
text = self.read_text('RECORD')
return text and text.splitlines()
def _read_files_egginfo(self):
"""
SOURCES.txt might contain literal commas, so wrap each line
in quotes.
"""
text = self.read_text('SOURCES.txt')
return text and map('"{}"'.format, text.splitlines())
@property
def requires(self):
"""Generated requirements specified for this Distribution"""
return self._read_dist_info_reqs() or self._read_egg_info_reqs()
def _read_dist_info_reqs(self):
spec = self.metadata['Requires-Dist']
return spec and filter(None, spec.splitlines())
def _read_egg_info_reqs(self):
source = self.read_text('requires.txt')
return source and self._deps_from_requires_text(source)
@classmethod
def _deps_from_requires_text(cls, source):
section_pairs = cls._read_sections(source.splitlines())
sections = {
section: list(map(operator.itemgetter('line'), results))
for section, results in
itertools.groupby(section_pairs, operator.itemgetter('section'))
}
return cls._convert_egg_info_reqs_to_simple_reqs(sections)
@staticmethod
def _read_sections(lines):
section = None
for line in filter(None, lines):
section_match = re.match(r'\[(.*)\]$', line)
if section_match:
section = section_match.group(1)
continue
yield locals()
@staticmethod
def _convert_egg_info_reqs_to_simple_reqs(sections):
"""
Historically, setuptools would solicit and store 'extra'
requirements, including those with environment markers,
in separate sections. More modern tools expect each
dependency to be defined separately, with any relevant
extras and environment markers attached directly to that
requirement. This method converts the former to the
latter. See _test_deps_from_requires_text for an example.
"""
def make_condition(name):
return name and 'extra == "{name}"'.format(name=name)
def parse_condition(section):
section = section or ''
extra, sep, markers = section.partition(':')
if extra and markers:
markers = '({markers})'.format(markers=markers)
conditions = list(filter(None, [markers, make_condition(extra)]))
return '; ' + ' and '.join(conditions) if conditions else ''
for section, deps in sections.items():
for dep in deps:
yield dep + parse_condition(section)
class DistributionFinder(MetaPathFinder):
"""
A MetaPathFinder capable of discovering installed distributions.
"""
@abc.abstractmethod
def find_distributions(self, name=None, path=None):
"""
Find distributions.
Return an iterable of all Distribution instances capable of
loading the metadata for packages matching the ``name``
(or all names if not supplied) along the paths in the list
of directories ``path`` (defaults to sys.path).
"""
class PathDistribution(Distribution):
def __init__(self, path):
"""Construct a distribution from a path to the metadata directory."""
self._path = path
def read_text(self, filename):
with suppress(FileNotFoundError, NotADirectoryError, KeyError):
return self._path.joinpath(filename).read_text(encoding='utf-8')
read_text.__doc__ = Distribution.read_text.__doc__
def locate_file(self, path):
return self._path.parent / path
def distribution(package):
"""Get the ``Distribution`` instance for the given package.
:param package: The name of the package as a string.
:return: A ``Distribution`` instance (or subclass thereof).
"""
return Distribution.from_name(package)
def distributions():
"""Get all ``Distribution`` instances in the current environment.
:return: An iterable of ``Distribution`` instances.
"""
return Distribution.discover()
def metadata(package):
"""Get the metadata for the package.
:param package: The name of the distribution package to query.
:return: An email.Message containing the parsed metadata.
"""
return Distribution.from_name(package).metadata
def version(package):
"""Get the version string for the named package.
:param package: The name of the distribution package to query.
:return: The version string for the package as defined in the package's
"Version" metadata key.
"""
return distribution(package).version
def entry_points():
"""Return EntryPoint objects for all installed packages.
:return: EntryPoint objects for all installed packages.
"""
eps = itertools.chain.from_iterable(
dist.entry_points for dist in distributions())
by_group = operator.attrgetter('group')
ordered = sorted(eps, key=by_group)
grouped = itertools.groupby(ordered, by_group)
return {
group: tuple(eps)
for group, eps in grouped
}
def files(package):
return distribution(package).files
def requires(package):
"""
Return a list of requirements for the indicated distribution.
:return: An iterator of requirements, suitable for
packaging.requirement.Requirement.
"""
return distribution(package).requires

View File

Binary file not shown.

View File

@ -0,0 +1,199 @@
from __future__ import unicode_literals
import os
import sys
import shutil
import tempfile
import textwrap
import contextlib
try:
from contextlib import ExitStack
except ImportError:
from contextlib2 import ExitStack
try:
import pathlib
except ImportError:
import pathlib2 as pathlib
__metaclass__ = type
@contextlib.contextmanager
def tempdir():
tmpdir = tempfile.mkdtemp()
try:
yield pathlib.Path(tmpdir)
finally:
shutil.rmtree(tmpdir)
@contextlib.contextmanager
def save_cwd():
orig = os.getcwd()
try:
yield
finally:
os.chdir(orig)
@contextlib.contextmanager
def tempdir_as_cwd():
with tempdir() as tmp:
with save_cwd():
os.chdir(str(tmp))
yield tmp
class SiteDir:
def setUp(self):
self.fixtures = ExitStack()
self.addCleanup(self.fixtures.close)
self.site_dir = self.fixtures.enter_context(tempdir())
class OnSysPath:
@staticmethod
@contextlib.contextmanager
def add_sys_path(dir):
sys.path[:0] = [str(dir)]
try:
yield
finally:
sys.path.remove(str(dir))
def setUp(self):
super(OnSysPath, self).setUp()
self.fixtures.enter_context(self.add_sys_path(self.site_dir))
class DistInfoPkg(OnSysPath, SiteDir):
files = {
"distinfo_pkg-1.0.0.dist-info": {
"METADATA": """
Name: distinfo-pkg
Author: Steven Ma
Version: 1.0.0
Requires-Dist: wheel >= 1.0
Requires-Dist: pytest; extra == 'test'
""",
"RECORD": "mod.py,sha256=abc,20\n",
"entry_points.txt": """
[entries]
main = mod:main
"""
},
"mod.py": """
def main():
print("hello world")
""",
}
def setUp(self):
super(DistInfoPkg, self).setUp()
build_files(DistInfoPkg.files, self.site_dir)
class DistInfoPkgOffPath(SiteDir):
def setUp(self):
super(DistInfoPkgOffPath, self).setUp()
build_files(DistInfoPkg.files, self.site_dir)
class EggInfoPkg(OnSysPath, SiteDir):
files = {
"egginfo_pkg.egg-info": {
"PKG-INFO": """
Name: egginfo-pkg
Author: Steven Ma
License: Unknown
Version: 1.0.0
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries
""",
"SOURCES.txt": """
mod.py
egginfo_pkg.egg-info/top_level.txt
""",
"entry_points.txt": """
[entries]
main = mod:main
""",
"requires.txt": """
wheel >= 1.0; python_version >= "2.7"
[test]
pytest
""",
"top_level.txt": "mod\n"
},
"mod.py": """
def main():
print("hello world")
""",
}
def setUp(self):
super(EggInfoPkg, self).setUp()
build_files(EggInfoPkg.files, prefix=self.site_dir)
class EggInfoFile(OnSysPath, SiteDir):
files = {
"egginfo_file.egg-info": """
Metadata-Version: 1.0
Name: egginfo_file
Version: 0.1
Summary: An example package
Home-page: www.example.com
Author: Eric Haffa-Vee
Author-email: eric@example.coms
License: UNKNOWN
Description: UNKNOWN
Platform: UNKNOWN
""",
}
def setUp(self):
super(EggInfoFile, self).setUp()
build_files(EggInfoFile.files, prefix=self.site_dir)
def build_files(file_defs, prefix=pathlib.Path()):
"""Build a set of files/directories, as described by the
file_defs dictionary. Each key/value pair in the dictionary is
interpreted as a filename/contents pair. If the contents value is a
dictionary, a directory is created, and the dictionary interpreted
as the files within it, recursively.
For example:
{"README.txt": "A README file",
"foo": {
"__init__.py": "",
"bar": {
"__init__.py": "",
},
"baz.py": "# Some code",
}
}
"""
for name, contents in file_defs.items():
full_name = prefix / name
if isinstance(contents, dict):
full_name.mkdir()
build_files(contents, prefix=full_name)
else:
if isinstance(contents, bytes):
with full_name.open('wb') as f:
f.write(contents)
else:
with full_name.open('w') as f:
f.write(DALS(contents))
def DALS(str):
"Dedent and left-strip"
return textwrap.dedent(str).lstrip()

View File

@ -0,0 +1,158 @@
# coding: utf-8
import re
import textwrap
import unittest
import importlib.metadata
from . import fixtures
from importlib.metadata import (
Distribution, EntryPoint,
PackageNotFoundError, distributions,
entry_points, metadata, version,
)
class BasicTests(fixtures.DistInfoPkg, unittest.TestCase):
version_pattern = r'\d+\.\d+(\.\d)?'
def test_retrieves_version_of_self(self):
dist = Distribution.from_name('distinfo-pkg')
assert isinstance(dist.version, str)
assert re.match(self.version_pattern, dist.version)
def test_for_name_does_not_exist(self):
with self.assertRaises(PackageNotFoundError):
Distribution.from_name('does-not-exist')
def test_new_style_classes(self):
self.assertIsInstance(Distribution, type)
class ImportTests(fixtures.DistInfoPkg, unittest.TestCase):
def test_import_nonexistent_module(self):
# Ensure that the MetadataPathFinder does not crash an import of a
# non-existant module.
with self.assertRaises(ImportError):
importlib.import_module('does_not_exist')
def test_resolve(self):
entries = dict(entry_points()['entries'])
ep = entries['main']
self.assertEqual(ep.load().__name__, "main")
def test_resolve_without_attr(self):
ep = EntryPoint(
name='ep',
value='importlib.metadata',
group='grp',
)
assert ep.load() is importlib.metadata
class NameNormalizationTests(
fixtures.OnSysPath, fixtures.SiteDir, unittest.TestCase):
@staticmethod
def pkg_with_dashes(site_dir):
"""
Create minimal metadata for a package with dashes
in the name (and thus underscores in the filename).
"""
metadata_dir = site_dir / 'my_pkg.dist-info'
metadata_dir.mkdir()
metadata = metadata_dir / 'METADATA'
with metadata.open('w') as strm:
strm.write('Version: 1.0\n')
return 'my-pkg'
def test_dashes_in_dist_name_found_as_underscores(self):
"""
For a package with a dash in the name, the dist-info metadata
uses underscores in the name. Ensure the metadata loads.
"""
pkg_name = self.pkg_with_dashes(self.site_dir)
assert version(pkg_name) == '1.0'
@staticmethod
def pkg_with_mixed_case(site_dir):
"""
Create minimal metadata for a package with mixed case
in the name.
"""
metadata_dir = site_dir / 'CherryPy.dist-info'
metadata_dir.mkdir()
metadata = metadata_dir / 'METADATA'
with metadata.open('w') as strm:
strm.write('Version: 1.0\n')
return 'CherryPy'
def test_dist_name_found_as_any_case(self):
"""
Ensure the metadata loads when queried with any case.
"""
pkg_name = self.pkg_with_mixed_case(self.site_dir)
assert version(pkg_name) == '1.0'
assert version(pkg_name.lower()) == '1.0'
assert version(pkg_name.upper()) == '1.0'
class NonASCIITests(fixtures.OnSysPath, fixtures.SiteDir, unittest.TestCase):
@staticmethod
def pkg_with_non_ascii_description(site_dir):
"""
Create minimal metadata for a package with non-ASCII in
the description.
"""
metadata_dir = site_dir / 'portend.dist-info'
metadata_dir.mkdir()
metadata = metadata_dir / 'METADATA'
with metadata.open('w', encoding='utf-8') as fp:
fp.write('Description: pôrˈtend\n')
return 'portend'
@staticmethod
def pkg_with_non_ascii_description_egg_info(site_dir):
"""
Create minimal metadata for an egg-info package with
non-ASCII in the description.
"""
metadata_dir = site_dir / 'portend.dist-info'
metadata_dir.mkdir()
metadata = metadata_dir / 'METADATA'
with metadata.open('w', encoding='utf-8') as fp:
fp.write(textwrap.dedent("""
Name: portend
pôrˈtend
""").lstrip())
return 'portend'
def test_metadata_loads(self):
pkg_name = self.pkg_with_non_ascii_description(self.site_dir)
meta = metadata(pkg_name)
assert meta['Description'] == 'pôrˈtend'
def test_metadata_loads_egg_info(self):
pkg_name = self.pkg_with_non_ascii_description_egg_info(self.site_dir)
meta = metadata(pkg_name)
assert meta.get_payload() == 'pôrˈtend\n'
class DiscoveryTests(fixtures.EggInfoPkg,
fixtures.DistInfoPkg,
unittest.TestCase):
def test_package_discovery(self):
dists = list(distributions())
assert all(
isinstance(dist, Distribution)
for dist in dists
)
assert any(
dist.metadata['Name'] == 'egginfo-pkg'
for dist in dists
)
assert any(
dist.metadata['Name'] == 'distinfo-pkg'
for dist in dists
)

View File

@ -0,0 +1,151 @@
import re
import textwrap
import unittest
import itertools
from collections.abc import Iterator
from . import fixtures
from importlib.metadata import (
Distribution, PackageNotFoundError, distribution,
entry_points, files, metadata, requires, version,
)
class APITests(
fixtures.EggInfoPkg,
fixtures.DistInfoPkg,
fixtures.EggInfoFile,
unittest.TestCase):
version_pattern = r'\d+\.\d+(\.\d)?'
def test_retrieves_version_of_self(self):
pkg_version = version('egginfo-pkg')
assert isinstance(pkg_version, str)
assert re.match(self.version_pattern, pkg_version)
def test_retrieves_version_of_distinfo_pkg(self):
pkg_version = version('distinfo-pkg')
assert isinstance(pkg_version, str)
assert re.match(self.version_pattern, pkg_version)
def test_for_name_does_not_exist(self):
with self.assertRaises(PackageNotFoundError):
distribution('does-not-exist')
def test_for_top_level(self):
self.assertEqual(
distribution('egginfo-pkg').read_text('top_level.txt').strip(),
'mod')
def test_read_text(self):
top_level = [
path for path in files('egginfo-pkg')
if path.name == 'top_level.txt'
][0]
self.assertEqual(top_level.read_text(), 'mod\n')
def test_entry_points(self):
entries = dict(entry_points()['entries'])
ep = entries['main']
self.assertEqual(ep.value, 'mod:main')
self.assertEqual(ep.extras, [])
def test_metadata_for_this_package(self):
md = metadata('egginfo-pkg')
assert md['author'] == 'Steven Ma'
assert md['LICENSE'] == 'Unknown'
assert md['Name'] == 'egginfo-pkg'
classifiers = md.get_all('Classifier')
assert 'Topic :: Software Development :: Libraries' in classifiers
@staticmethod
def _test_files(files_iter):
assert isinstance(files_iter, Iterator), files_iter
files = list(files_iter)
root = files[0].root
for file in files:
assert file.root == root
assert not file.hash or file.hash.value
assert not file.hash or file.hash.mode == 'sha256'
assert not file.size or file.size >= 0
assert file.locate().exists()
assert isinstance(file.read_binary(), bytes)
if file.name.endswith('.py'):
file.read_text()
def test_file_hash_repr(self):
assertRegex = self.assertRegex
util = [
p for p in files('distinfo-pkg')
if p.name == 'mod.py'
][0]
assertRegex(
repr(util.hash),
'<FileHash mode: sha256 value: .*>')
def test_files_dist_info(self):
self._test_files(files('distinfo-pkg'))
def test_files_egg_info(self):
self._test_files(files('egginfo-pkg'))
def test_version_egg_info_file(self):
self.assertEqual(version('egginfo-file'), '0.1')
def test_requires_egg_info_file(self):
requirements = requires('egginfo-file')
self.assertIsNone(requirements)
def test_requires(self):
deps = requires('egginfo-pkg')
assert any(
dep == 'wheel >= 1.0; python_version >= "2.7"'
for dep in deps
)
def test_requires_dist_info(self):
deps = list(requires('distinfo-pkg'))
assert deps and all(deps)
def test_more_complex_deps_requires_text(self):
requires = textwrap.dedent("""
dep1
dep2
[:python_version < "3"]
dep3
[extra1]
dep4
[extra2:python_version < "3"]
dep5
""")
deps = sorted(Distribution._deps_from_requires_text(requires))
expected = [
'dep1',
'dep2',
'dep3; python_version < "3"',
'dep4; extra == "extra1"',
'dep5; (python_version < "3") and extra == "extra2"',
]
# It's important that the environment marker expression be
# wrapped in parentheses to avoid the following 'and' binding more
# tightly than some other part of the environment expression.
assert deps == expected
class OffSysPathTests(fixtures.DistInfoPkgOffPath, unittest.TestCase):
def test_find_distributions_specified_path(self):
dists = itertools.chain.from_iterable(
resolver(path=[str(self.site_dir)])
for resolver in Distribution._discover_resolvers()
)
assert any(
dist.metadata['Name'] == 'distinfo-pkg'
for dist in dists
)

View File

@ -0,0 +1,56 @@
import sys
import unittest
from contextlib import ExitStack
from importlib.metadata import distribution, entry_points, files, version
from importlib.resources import path
class TestZip(unittest.TestCase):
root = 'test.test_importlib.data'
def setUp(self):
# Find the path to the example-*.whl so we can add it to the front of
# sys.path, where we'll then try to find the metadata thereof.
self.resources = ExitStack()
self.addCleanup(self.resources.close)
wheel = self.resources.enter_context(
path(self.root, 'example-21.12-py3-none-any.whl'))
sys.path.insert(0, str(wheel))
self.resources.callback(sys.path.pop, 0)
def test_zip_version(self):
self.assertEqual(version('example'), '21.12')
def test_zip_entry_points(self):
scripts = dict(entry_points()['console_scripts'])
entry_point = scripts['example']
self.assertEqual(entry_point.value, 'example:main')
def test_missing_metadata(self):
self.assertIsNone(distribution('example').read_text('does not exist'))
def test_case_insensitive(self):
self.assertEqual(version('Example'), '21.12')
def test_files(self):
for file in files('example'):
path = str(file.dist.locate_file(file))
assert '.whl/' in path, path
class TestEgg(TestZip):
def setUp(self):
# Find the path to the example-*.egg so we can add it to the front of
# sys.path, where we'll then try to find the metadata thereof.
self.resources = ExitStack()
self.addCleanup(self.resources.close)
egg = self.resources.enter_context(
path(self.root, 'example-21.12-py3.6.egg'))
sys.path.insert(0, str(egg))
self.resources.callback(sys.path.pop, 0)
def test_files(self):
for file in files('example'):
path = str(file.dist.locate_file(file))
assert '.egg/' in path, path

View File

@ -0,0 +1 @@
Introduce the ``importlib.metadata`` module with (provisional) support for reading metadata from third-party packages.

1
Python.framework/Resources Symbolic link
View File

@ -0,0 +1 @@
Versions/Current/Resources

File diff suppressed because it is too large Load Diff