Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity

bpo-32248 - Implement importlib.resources (#4911) 

Port importlib_resources to importlib.resources
This commit is contained in:
Barry Warsaw 2017-12-30 15:18:06 -05:00 committed by GitHub
parent ffcb4c0165
commit deae6b451f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
28 changed files with 912 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

127
Doc/library/importlib.rst
View File

@ -484,7 +484,7 @@ ABC hierarchy::
versus on the file system.
For any of methods of this class, a *resource* argument is
expected to be a :term:`file-like object` which represents
expected to be a :term:`path-like object` which represents
conceptually just a file name. This means that no subdirectory
paths should be included in the *resource* argument. This is
because the location of the package that the loader is for acts
@ -775,6 +775,131 @@ ABC hierarchy::
itself does not end in ``__init__``.
:mod:`importlib.resources` -- Resources
---------------------------------------
.. module:: importlib.resources
:synopsis: Package resource reading, opening, and access
**Source code:** :source:`Lib/importlib/resources.py`
--------------
.. versionadded:: 3.7
This module leverages Python's import system to provide access to *resources*
within *packages*. If you can import a package, you can access resources
within that package. Resources can be opened or read, in either binary or
text mode.
Resources are roughly akin to files inside directories, though it's important
to keep in mind that this is just a metaphor. Resources and packages **do
not** have to exist as physical files and directories on the file system.
Loaders can support resources by implementing the :class:`ResourceReader`
abstract base class.
The following types are defined.
.. data:: Package
The ``Package`` type is defined as ``Union[str, ModuleType]``. This means
that where the function describes accepting a ``Package``, you can pass in
either a string or a module. Module objects must have a resolvable
``__spec__.submodule_search_locations`` that is not ``None``.
.. data:: Resource
This type describes the resource names passed into the various functions
in this package. This is defined as ``Union[str, os.PathLike]``.
The following functions are available.
.. function:: open_binary(package, resource)
Open for binary reading the *resource* within *package*.
*package* is either a name or a module object which conforms to the
``Package`` requirements. *resource* is the name of the resource to open
within *package*; it may not contain path separators and it may not have
sub-resources (i.e. it cannot be a directory). This function returns a
``typing.BinaryIO`` instance, a binary I/O stream open for reading.
.. function:: open_text(package, resource, encoding='utf-8', errors='strict')
Open for text reading the *resource* within *package*. By default, the
resource is opened for reading as UTF-8.
*package* is either a name or a module object which conforms to the
``Package`` requirements. *resource* is the name of the resource to open
within *package*; it may not contain path separators and it may not have
sub-resources (i.e. it cannot be a directory). *encoding* and *errors*
have the same meaning as with built-in :func:`open`.
This function returns a ``typing.TextIO`` instance, a text I/O stream open
for reading.
.. function:: read_binary(package, resource)
Read and return the contents of the *resource* within *package* as
``bytes``.
*package* is either a name or a module object which conforms to the
``Package`` requirements. *resource* is the name of the resource to open
within *package*; it may not contain path separators and it may not have
sub-resources (i.e. it cannot be a directory). This function returns the
contents of the resource as :class:`bytes`.
.. function:: read_text(package, resource, encoding='utf-8', errors='strict')
Read and return the contents of *resource* within *package* as a ``str``.
By default, the contents are read as strict UTF-8.
*package* is either a name or a module object which conforms to the
``Package`` requirements. *resource* is the name of the resource to open
within *package*; it may not contain path separators and it may not have
sub-resources (i.e. it cannot be a directory). *encoding* and *errors*
have the same meaning as with built-in :func:`open`. This function
returns the contents of the resource as :class:`str`.
.. function:: path(package, resource)
Return the path to the *resource* as an actual file system path. This
function returns a context manager for use in a :keyword:`with` statement.
The context manager provides a :class:`pathlib.Path` object.
Exiting the context manager cleans up any temporary file created when the
resource needs to be extracted from e.g. a zip file.
*package* is either a name or a module object which conforms to the
``Package`` requirements. *resource* is the name of the resource to open
within *package*; it may not contain path separators and it may not have
sub-resources (i.e. it cannot be a directory).
.. function:: is_resource(package, name)
Return ``True`` if there is a resource named *name* in the package,
otherwise ``False``. Remember that directories are *not* resources!
*package* is either a name or a module object which conforms to the
``Package`` requirements.
.. function:: contents(package)
Return an iterator over the named items within the package. The iterator
returns :class:`str` resources (e.g. files) and non-resources
(e.g. directories). The iterator does not recurse into subdirectories.
*package* is either a name or a module object which conforms to the
``Package`` requirements.
:mod:`importlib.machinery` -- Importers and path hooks
------------------------------------------------------

9
Doc/whatsnew/3.7.rst
View File

@ -282,7 +282,14 @@ Other Language Changes
New Modules
===========
* None yet.
importlib.resources
-------------------
This module provides several new APIs and one new ABC for access to, opening,
and reading *resources* inside packages. Resources are roughly akin to files
inside of packages, but they needn't be actual files on the physical file
system. Module loaders can implement the
:class:`importlib.abc.ResourceReader` ABC to support this new module's API.
Improved Modules

303
Lib/importlib/resources.py Normal file
View File

@ -0,0 +1,303 @@
import os
import tempfile
from . import abc as resources_abc
from builtins import open as builtins_open
from contextlib import contextmanager, suppress
from importlib import import_module
from importlib.abc import ResourceLoader
from io import BytesIO, TextIOWrapper
from pathlib import Path
from types import ModuleType
from typing import Iterator, Optional, Set, Union # noqa: F401
from typing import cast
from typing.io import BinaryIO, TextIO
from zipfile import ZipFile
Package = Union[str, ModuleType]
Resource = Union[str, os.PathLike]
def _get_package(package) -> ModuleType:
"""Take a package name or module object and return the module.
If a name, the module is imported. If the passed or imported module
object is not a package, raise an exception.
"""
if hasattr(package, '__spec__'):
if package.__spec__.submodule_search_locations is None:
raise TypeError('{!r} is not a package'.format(
package.__spec__.name))
else:
return package
else:
module = import_module(package)
if module.__spec__.submodule_search_locations is None:
raise TypeError('{!r} is not a package'.format(package))
else:
return module
def _normalize_path(path) -> str:
"""Normalize a path by ensuring it is a string.
If the resulting string contains path separators, an exception is raised.
"""
str_path = str(path)
parent, file_name = os.path.split(str_path)
if parent:
raise ValueError('{!r} must be only a file name'.format(path))
else:
return file_name
def _get_resource_reader(
package: ModuleType) -> Optional[resources_abc.ResourceReader]:
# Return the package's loader if it's a ResourceReader. We can't use
# a issubclass() check here because apparently abc.'s __subclasscheck__()
# hook wants to create a weak reference to the object, but
# zipimport.zipimporter does not support weak references, resulting in a
# TypeError. That seems terrible.
if hasattr(package.__spec__.loader, 'open_resource'):
return cast(resources_abc.ResourceReader, package.__spec__.loader)
return None
def open_binary(package: Package, resource: Resource) -> BinaryIO:
"""Return a file-like object opened for binary reading of the resource."""
resource = _normalize_path(resource)
package = _get_package(package)
reader = _get_resource_reader(package)
if reader is not None:
return reader.open_resource(resource)
absolute_package_path = os.path.abspath(package.__spec__.origin)
package_path = os.path.dirname(absolute_package_path)
full_path = os.path.join(package_path, resource)
try:
return builtins_open(full_path, mode='rb')
except OSError:
# Just assume the loader is a resource loader; all the relevant
# importlib.machinery loaders are and an AttributeError for
# get_data() will make it clear what is needed from the loader.
loader = cast(ResourceLoader, package.__spec__.loader)
data = None
if hasattr(package.__spec__.loader, 'get_data'):
with suppress(OSError):
data = loader.get_data(full_path)
if data is None:
package_name = package.__spec__.name
message = '{!r} resource not found in {!r}'.format(
resource, package_name)
raise FileNotFoundError(message)
else:
return BytesIO(data)
def open_text(package: Package,
resource: Resource,
encoding: str = 'utf-8',
errors: str = 'strict') -> TextIO:
"""Return a file-like object opened for text reading of the resource."""
resource = _normalize_path(resource)
package = _get_package(package)
reader = _get_resource_reader(package)
if reader is not None:
return TextIOWrapper(reader.open_resource(resource), encoding, errors)
absolute_package_path = os.path.abspath(package.__spec__.origin)
package_path = os.path.dirname(absolute_package_path)
full_path = os.path.join(package_path, resource)
try:
return builtins_open(
full_path, mode='r', encoding=encoding, errors=errors)
except OSError:
# Just assume the loader is a resource loader; all the relevant
# importlib.machinery loaders are and an AttributeError for
# get_data() will make it clear what is needed from the loader.
loader = cast(ResourceLoader, package.__spec__.loader)
data = None
if hasattr(package.__spec__.loader, 'get_data'):
with suppress(OSError):
data = loader.get_data(full_path)
if data is None:
package_name = package.__spec__.name
message = '{!r} resource not found in {!r}'.format(
resource, package_name)
raise FileNotFoundError(message)
else:
return TextIOWrapper(BytesIO(data), encoding, errors)
def read_binary(package: Package, resource: Resource) -> bytes:
"""Return the binary contents of the resource."""
resource = _normalize_path(resource)
package = _get_package(package)
with open_binary(package, resource) as fp:
return fp.read()
def read_text(package: Package,
resource: Resource,
encoding: str = 'utf-8',
errors: str = 'strict') -> str:
"""Return the decoded string of the resource.
The decoding-related arguments have the same semantics as those of
bytes.decode().
"""
resource = _normalize_path(resource)
package = _get_package(package)
with open_text(package, resource, encoding, errors) as fp:
return fp.read()
@contextmanager
def path(package: Package, resource: Resource) -> Iterator[Path]:
"""A context manager providing a file path object to the resource.
If the resource does not already exist on its own on the file system,
a temporary file will be created. If the file was created, the file
will be deleted upon exiting the context manager (no exception is
raised if the file was deleted prior to the context manager
exiting).
"""
resource = _normalize_path(resource)
package = _get_package(package)
reader = _get_resource_reader(package)
if reader is not None:
try:
yield Path(reader.resource_path(resource))
return
except FileNotFoundError:
pass
# Fall-through for both the lack of resource_path() *and* if
# resource_path() raises FileNotFoundError.
package_directory = Path(package.__spec__.origin).parent
file_path = package_directory / resource
if file_path.exists():
yield file_path
else:
with open_binary(package, resource) as fp:
data = fp.read()
# Not using tempfile.NamedTemporaryFile as it leads to deeper 'try'
# blocks due to the need to close the temporary file to work on
# Windows properly.
fd, raw_path = tempfile.mkstemp()
try:
os.write(fd, data)
os.close(fd)
yield Path(raw_path)
finally:
try:
os.remove(raw_path)
except FileNotFoundError:
pass
def is_resource(package: Package, name: str) -> bool:
"""True if 'name' is a resource inside 'package'.
Directories are *not* resources.
"""
package = _get_package(package)
_normalize_path(name)
reader = _get_resource_reader(package)
if reader is not None:
return reader.is_resource(name)
try:
package_contents = set(contents(package))
except (NotADirectoryError, FileNotFoundError):
return False
if name not in package_contents:
return False
# Just because the given file_name lives as an entry in the package's
# contents doesn't necessarily mean it's a resource. Directories are not
# resources, so let's try to find out if it's a directory or not.
path = Path(package.__spec__.origin).parent / name
if path.is_file():
return True
if path.is_dir():
return False
# If it's not a file and it's not a directory, what is it? Well, this
# means the file doesn't exist on the file system, so it probably lives
# inside a zip file. We have to crack open the zip, look at its table of
# contents, and make sure that this entry doesn't have sub-entries.
archive_path = package.__spec__.loader.archive # type: ignore
package_directory = Path(package.__spec__.origin).parent
with ZipFile(archive_path) as zf:
toc = zf.namelist()
relpath = package_directory.relative_to(archive_path)
candidate_path = relpath / name
for entry in toc:
try:
relative_to_candidate = Path(entry).relative_to(candidate_path)
except ValueError:
# The two paths aren't relative to each other so we can ignore it.
continue
# Since directories aren't explicitly listed in the zip file, we must
# infer their 'directory-ness' by looking at the number of path
# components in the path relative to the package resource we're
# looking up. If there are zero additional parts, it's a file, i.e. a
# resource. If there are more than zero it's a directory, i.e. not a
# resource. It has to be one of these two cases.
return len(relative_to_candidate.parts) == 0
# I think it's impossible to get here. It would mean that we are looking
# for a resource in a zip file, there's an entry matching it in the return
# value of contents(), but we never actually found it in the zip's table of
# contents.
raise AssertionError('Impossible situation')
def contents(package: Package) -> Iterator[str]:
"""Return the list of entries in 'package'.
Note that not all entries are resources. Specifically, directories are
not considered resources. Use `is_resource()` on each entry returned here
to check if it is a resource or not.
"""
package = _get_package(package)
reader = _get_resource_reader(package)
if reader is not None:
yield from reader.contents()
return
# Is the package a namespace package? By definition, namespace packages
# cannot have resources.
if (package.__spec__.origin == 'namespace' and
not package.__spec__.has_location):
return []
package_directory = Path(package.__spec__.origin).parent
try:
yield from os.listdir(str(package_directory))
except (NotADirectoryError, FileNotFoundError):
# The package is probably in a zip file.
archive_path = getattr(package.__spec__.loader, 'archive', None)
if archive_path is None:
raise
relpath = package_directory.relative_to(archive_path)
with ZipFile(archive_path) as zf:
toc = zf.namelist()
subdirs_seen = set() # type: Set
for filename in toc:
path = Path(filename)
# Strip off any path component parts that are in common with the
# package directory, relative to the zip archive's file system
# path. This gives us all the parts that live under the named
# package inside the zip file. If the length of these subparts is
# exactly 1, then it is situated inside the package. The resulting
# length will be 0 if it's above the package, and it will be
# greater than 1 if it lives in a subdirectory of the package
# directory.
#
# However, since directories themselves don't appear in the zip
# archive as a separate entry, we need to return the first path
# component for any case that has > 1 subparts -- but only once!
if path.parts[:len(relpath.parts)] != relpath.parts:
continue
subparts = path.parts[len(relpath.parts):]
if len(subparts) == 1:
yield subparts[0]
elif len(subparts) > 1:
subdir = subparts[0]
if subdir not in subdirs_seen:
subdirs_seen.add(subdir)
yield subdir

0
Lib/test/test_importlib/data01/__init__.py Normal file
View File

BIN
Lib/test/test_importlib/data01/binary.file Normal file
View File

Binary file not shown.

0
Lib/test/test_importlib/data01/subdirectory/__init__.py Normal file
View File

BIN
Lib/test/test_importlib/data01/subdirectory/binary.file Normal file
View File

Binary file not shown.

BIN
Lib/test/test_importlib/data01/utf-16.file Normal file
View File

Binary file not shown.

1
Lib/test/test_importlib/data01/utf-8.file Normal file
View File

@ -0,0 +1 @@
Hello, UTF-8 world!

0
Lib/test/test_importlib/data02/__init__.py Normal file
View File

0
Lib/test/test_importlib/data02/one/__init__.py Normal file
View File

1
Lib/test/test_importlib/data02/one/resource1.txt Normal file
View File

@ -0,0 +1 @@
one resource

0
Lib/test/test_importlib/data02/two/__init__.py Normal file
View File

1
Lib/test/test_importlib/data02/two/resource2.txt Normal file
View File

@ -0,0 +1 @@
two resource

0
Lib/test/test_importlib/data03/__init__.py Normal file
View File

0
Lib/test/test_importlib/data03/namespace/portion1/__init__.py Normal file
View File

0
Lib/test/test_importlib/data03/namespace/portion2/__init__.py Normal file
View File

0
Lib/test/test_importlib/data03/namespace/resource1.txt Normal file
View File

72
Lib/test/test_importlib/test_open.py Normal file
View File

@ -0,0 +1,72 @@
import unittest
from importlib import resources
from . import data01
from . import util
class CommonBinaryTests(util.CommonResourceTests, unittest.TestCase):
def execute(self, package, path):
with resources.open_binary(package, path):
pass
class CommonTextTests(util.CommonResourceTests, unittest.TestCase):
def execute(self, package, path):
with resources.open_text(package, path):
pass
class OpenTests:
def test_open_binary(self):
with resources.open_binary(self.data, 'utf-8.file') as fp:
result = fp.read()
self.assertEqual(result, b'Hello, UTF-8 world!\n')
def test_open_text_default_encoding(self):
with resources.open_text(self.data, 'utf-8.file') as fp:
result = fp.read()
self.assertEqual(result, 'Hello, UTF-8 world!\n')
def test_open_text_given_encoding(self):
with resources.open_text(
self.data, 'utf-16.file', 'utf-16', 'strict') as fp:
result = fp.read()
self.assertEqual(result, 'Hello, UTF-16 world!\n')
def test_open_text_with_errors(self):
# Raises UnicodeError without the 'errors' argument.
with resources.open_text(
self.data, 'utf-16.file', 'utf-8', 'strict') as fp:
self.assertRaises(UnicodeError, fp.read)
with resources.open_text(
self.data, 'utf-16.file', 'utf-8', 'ignore') as fp:
result = fp.read()
self.assertEqual(
result,
'H\x00e\x00l\x00l\x00o\x00,\x00 '
'\x00U\x00T\x00F\x00-\x001\x006\x00 '
'\x00w\x00o\x00r\x00l\x00d\x00!\x00\n\x00')
def test_open_binary_FileNotFoundError(self):
self.assertRaises(
FileNotFoundError,
resources.open_binary, self.data, 'does-not-exist')
def test_open_text_FileNotFoundError(self):
self.assertRaises(
FileNotFoundError,
resources.open_text, self.data, 'does-not-exist')
class OpenDiskTests(OpenTests, unittest.TestCase):
def setUp(self):
self.data = data01
class OpenZipTests(OpenTests, util.ZipSetup, unittest.TestCase):
pass
if __name__ == '__main__':
unittest.main()

39
Lib/test/test_importlib/test_path.py Normal file
View File

@ -0,0 +1,39 @@
import unittest
from importlib import resources
from . import data01
from . import util
class CommonTests(util.CommonResourceTests, unittest.TestCase):
def execute(self, package, path):
with resources.path(package, path):
pass
class PathTests:
def test_reading(self):
# Path should be readable.
# Test also implicitly verifies the returned object is a pathlib.Path
# instance.
with resources.path(self.data, 'utf-8.file') as path:
# pathlib.Path.read_text() was introduced in Python 3.5.
with path.open('r', encoding='utf-8') as file:
text = file.read()
self.assertEqual('Hello, UTF-8 world!\n', text)
class PathDiskTests(PathTests, unittest.TestCase):
data = data01
class PathZipTests(PathTests, util.ZipSetup, unittest.TestCase):
def test_remove_in_context_manager(self):
# It is not an error if the file that was temporarily stashed on the
# file system is removed inside the `with` stanza.
with resources.path(self.data, 'utf-8.file') as path:
path.unlink()
if __name__ == '__main__':
unittest.main()

53
Lib/test/test_importlib/test_read.py Normal file
View File

@ -0,0 +1,53 @@
import unittest
from importlib import resources
from . import data01
from . import util
class CommonBinaryTests(util.CommonResourceTests, unittest.TestCase):
def execute(self, package, path):
resources.read_binary(package, path)
class CommonTextTests(util.CommonResourceTests, unittest.TestCase):
def execute(self, package, path):
resources.read_text(package, path)
class ReadTests:
def test_read_binary(self):
result = resources.read_binary(self.data, 'binary.file')
self.assertEqual(result, b'\0\1\2\3')
def test_read_text_default_encoding(self):
result = resources.read_text(self.data, 'utf-8.file')
self.assertEqual(result, 'Hello, UTF-8 world!\n')
def test_read_text_given_encoding(self):
result = resources.read_text(
self.data, 'utf-16.file', encoding='utf-16')
self.assertEqual(result, 'Hello, UTF-16 world!\n')
def test_read_text_with_errors(self):
# Raises UnicodeError without the 'errors' argument.
self.assertRaises(
UnicodeError, resources.read_text, self.data, 'utf-16.file')
result = resources.read_text(self.data, 'utf-16.file', errors='ignore')
self.assertEqual(
result,
'H\x00e\x00l\x00l\x00o\x00,\x00 '
'\x00U\x00T\x00F\x00-\x001\x006\x00 '
'\x00w\x00o\x00r\x00l\x00d\x00!\x00\n\x00')
class ReadDiskTests(ReadTests, unittest.TestCase):
data = data01
class ReadZipTests(ReadTests, util.ZipSetup, unittest.TestCase):
pass
if __name__ == '__main__':
unittest.main()

143
Lib/test/test_importlib/test_resource.py Normal file
View File

@ -0,0 +1,143 @@
import sys
import unittest
from importlib import resources
from . import data01
from . import zipdata02
from . import util
class ResourceTests:
# Subclasses are expected to set the `data` attribute.
def test_is_resource_good_path(self):
self.assertTrue(resources.is_resource(self.data, 'binary.file'))
def test_is_resource_missing(self):
self.assertFalse(resources.is_resource(self.data, 'not-a-file'))
def test_is_resource_subresource_directory(self):
# Directories are not resources.
self.assertFalse(resources.is_resource(self.data, 'subdirectory'))
def test_contents(self):
contents = set(resources.contents(self.data))
# There may be cruft in the directory listing of the data directory.
# Under Python 3 we could have a __pycache__ directory, and under
# Python 2 we could have .pyc files. These are both artifacts of the
# test suite importing these modules and writing these caches. They
# aren't germane to this test, so just filter them out.
contents.discard('__pycache__')
contents.discard('__init__.pyc')
contents.discard('__init__.pyo')
self.assertEqual(contents, {
'__init__.py',
'subdirectory',
'utf-8.file',
'binary.file',
'utf-16.file',
})
class ResourceDiskTests(ResourceTests, unittest.TestCase):
def setUp(self):
self.data = data01
class ResourceZipTests(ResourceTests, util.ZipSetup, unittest.TestCase):
pass
class ResourceLoaderTests(unittest.TestCase):
def test_resource_contents(self):
package = util.create_package(
file=data01, path=data01.__file__, contents=['A', 'B', 'C'])
self.assertEqual(
set(resources.contents(package)),
{'A', 'B', 'C'})
def test_resource_is_resource(self):
package = util.create_package(
file=data01, path=data01.__file__,
contents=['A', 'B', 'C', 'D/E', 'D/F'])
self.assertTrue(resources.is_resource(package, 'B'))
def test_resource_directory_is_not_resource(self):
package = util.create_package(
file=data01, path=data01.__file__,
contents=['A', 'B', 'C', 'D/E', 'D/F'])
self.assertFalse(resources.is_resource(package, 'D'))
def test_resource_missing_is_not_resource(self):
package = util.create_package(
file=data01, path=data01.__file__,
contents=['A', 'B', 'C', 'D/E', 'D/F'])
self.assertFalse(resources.is_resource(package, 'Z'))
class ResourceCornerCaseTests(unittest.TestCase):
def test_package_has_no_reader_fallback(self):
# Test odd ball packages which:
# 1. Do not have a ResourceReader as a loader
# 2. Are not on the file system
# 3. Are not in a zip file
module = util.create_package(
file=data01, path=data01.__file__, contents=['A', 'B', 'C'])
# Give the module a dummy loader.
module.__loader__ = object()
# Give the module a dummy origin.
module.__file__ = '/path/which/shall/not/be/named'
if sys.version_info >= (3,):
module.__spec__.loader = module.__loader__
module.__spec__.origin = module.__file__
self.assertFalse(resources.is_resource(module, 'A'))
class ResourceFromZipsTest(util.ZipSetupBase, unittest.TestCase):
ZIP_MODULE = zipdata02 # type: ignore
def test_unrelated_contents(self):
# https://gitlab.com/python-devs/importlib_resources/issues/44
#
# Here we have a zip file with two unrelated subpackages. The bug
# reports that getting the contents of a resource returns unrelated
# files.
self.assertEqual(
set(resources.contents('ziptestdata.one')),
{'__init__.py', 'resource1.txt'})
self.assertEqual(
set(resources.contents('ziptestdata.two')),
{'__init__.py', 'resource2.txt'})
class NamespaceTest(unittest.TestCase):
def test_namespaces_cant_have_resources(self):
contents = set(resources.contents(
'test.test_importlib.data03.namespace'))
self.assertEqual(len(contents), 0)
# Even though there is a file in the namespace directory, it is not
# considered a resource, since namespace packages can't have them.
self.assertFalse(resources.is_resource(
'test.test_importlib.data03.namespace',
'resource1.txt'))
# We should get an exception if we try to read it or open it.
self.assertRaises(
FileNotFoundError,
resources.open_text,
'test.test_importlib.data03.namespace', 'resource1.txt')
self.assertRaises(
FileNotFoundError,
resources.open_binary,
'test.test_importlib.data03.namespace', 'resource1.txt')
self.assertRaises(
FileNotFoundError,
resources.read_text,
'test.test_importlib.data03.namespace', 'resource1.txt')
self.assertRaises(
FileNotFoundError,
resources.read_binary,
'test.test_importlib.data03.namespace', 'resource1.txt')
if __name__ == '__main__':
unittest.main()

163
Lib/test/test_importlib/util.py
View File

@ -1,17 +1,24 @@
import abc
import builtins
import contextlib
import errno
import functools
import importlib
from importlib import machinery, util, invalidate_caches
from importlib.abc import ResourceReader
import io
import os
import os.path
from pathlib import Path, PurePath
from test import support
import unittest
import sys
import tempfile
import types
from . import data01
from . import zipdata01
BUILTINS = types.SimpleNamespace()
BUILTINS.good_name = None
@ -386,3 +393,159 @@ class CASEOKTestBase:
if any(x in self.importlib._bootstrap_external._os.environ
for x in possibilities) != should_exist:
self.skipTest('os.environ changes not reflected in _os.environ')
def create_package(file, path, is_package=True, contents=()):
class Reader(ResourceReader):
def open_resource(self, path):
self._path = path
if isinstance(file, Exception):
raise file
else:
return file
def resource_path(self, path_):
self._path = path_
if isinstance(path, Exception):
raise path
else:
return path
def is_resource(self, path_):
self._path = path_
if isinstance(path, Exception):
raise path
for entry in contents:
parts = entry.split('/')
if len(parts) == 1 and parts[0] == path_:
return True
return False
def contents(self):
if isinstance(path, Exception):
raise path
# There's no yield from in baseball, er, Python 2.
for entry in contents:
yield entry
name = 'testingpackage'
# Unforunately importlib.util.module_from_spec() was not introduced until
# Python 3.5.
module = types.ModuleType(name)
loader = Reader()
spec = machinery.ModuleSpec(
name, loader,
origin='does-not-exist',
is_package=is_package)
module.__spec__ = spec
module.__loader__ = loader
return module
class CommonResourceTests(abc.ABC):
@abc.abstractmethod
def execute(self, package, path):
raise NotImplementedError
def test_package_name(self):
# Passing in the package name should succeed.
self.execute(data01.__name__, 'utf-8.file')
def test_package_object(self):
# Passing in the package itself should succeed.
self.execute(data01, 'utf-8.file')
def test_string_path(self):
# Passing in a string for the path should succeed.
path = 'utf-8.file'
self.execute(data01, path)
@unittest.skipIf(sys.version_info < (3, 6), 'requires os.PathLike support')
def test_pathlib_path(self):
# Passing in a pathlib.PurePath object for the path should succeed.
path = PurePath('utf-8.file')
self.execute(data01, path)
def test_absolute_path(self):
# An absolute path is a ValueError.
path = Path(__file__)
full_path = path.parent/'utf-8.file'
with self.assertRaises(ValueError):
self.execute(data01, full_path)
def test_relative_path(self):
# A reative path is a ValueError.
with self.assertRaises(ValueError):
self.execute(data01, '../data01/utf-8.file')
def test_importing_module_as_side_effect(self):
# The anchor package can already be imported.
del sys.modules[data01.__name__]
self.execute(data01.__name__, 'utf-8.file')
def test_non_package_by_name(self):
# The anchor package cannot be a module.
with self.assertRaises(TypeError):
self.execute(__name__, 'utf-8.file')
def test_non_package_by_package(self):
# The anchor package cannot be a module.
with self.assertRaises(TypeError):
module = sys.modules['test.test_importlib.util']
self.execute(module, 'utf-8.file')
@unittest.skipIf(sys.version_info < (3,), 'No ResourceReader in Python 2')
def test_resource_opener(self):
bytes_data = io.BytesIO(b'Hello, world!')
package = create_package(file=bytes_data, path=FileNotFoundError())
self.execute(package, 'utf-8.file')
self.assertEqual(package.__loader__._path, 'utf-8.file')
@unittest.skipIf(sys.version_info < (3,), 'No ResourceReader in Python 2')
def test_resource_path(self):
bytes_data = io.BytesIO(b'Hello, world!')
path = __file__
package = create_package(file=bytes_data, path=path)
self.execute(package, 'utf-8.file')
self.assertEqual(package.__loader__._path, 'utf-8.file')
def test_useless_loader(self):
package = create_package(file=FileNotFoundError(),
path=FileNotFoundError())
with self.assertRaises(FileNotFoundError):
self.execute(package, 'utf-8.file')
class ZipSetupBase:
ZIP_MODULE = None
@classmethod
def setUpClass(cls):
data_path = Path(cls.ZIP_MODULE.__file__)
data_dir = data_path.parent
cls._zip_path = str(data_dir / 'ziptestdata.zip')
sys.path.append(cls._zip_path)
cls.data = importlib.import_module('ziptestdata')
@classmethod
def tearDownClass(cls):
try:
sys.path.remove(cls._zip_path)
except ValueError:
pass
try:
del sys.path_importer_cache[cls._zip_path]
del sys.modules[cls.data.__name__]
except KeyError:
pass
try:
del cls.data
del cls._zip_path
except AttributeError:
pass
class ZipSetup(ZipSetupBase):
ZIP_MODULE = zipdata01 # type: ignore

0
Lib/test/test_importlib/zipdata01/__init__.py Normal file
View File

BIN
Lib/test/test_importlib/zipdata01/ziptestdata.zip Normal file
View File

Binary file not shown.

0
Lib/test/test_importlib/zipdata02/__init__.py Normal file
View File

BIN
Lib/test/test_importlib/zipdata02/ziptestdata.zip Normal file
View File

Binary file not shown.

3
Misc/NEWS.d/next/Library/2017-12-15-15-34-12.bpo-32248.zmO8G2.rst
View File

@ -1,2 +1,3 @@
Add :class:`importlib.abc.ResourceReader` as an ABC for loaders to provide a
unified API for reading resources contained within packages.
unified API for reading resources contained within packages. Also add
:mod:`importlib.resources` as the port of ``importlib_resources``.