bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).
This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:
1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)
Differences from the reference implementation:
- The module is arranged slightly differently: the accelerated module is
`_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
some changes in the test support function. (Suggested by Victor
Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
because we may do some refactoring in the future that would likely
require this separation anyway; we may:
- include the property tests
- automatically run all the tests against both pure Python and C,
rather than manually constructing C and Python test classes (similar
to the way this works with test_datetime.py, which generates C
and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
CI, though only on the coverage jobs. Introducing a PyPI dependency as
part of the CI build was controversial, and this is seen as less of a
major change, since the coverage jobs already depend on pip and PyPI.
Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:
- Fixed reference and memory leaks
With much debugging help from Pablo Galindo
- Added smoke tests ensuring that the C and Python modules are built
The import machinery can be somewhat fragile, and the "seamlessly falls
back to pure Python" nature of this module makes it so that a problem
building the C extension or a failure to import the pure Python version
might easily go unnoticed.
- Adjustments to zoneinfo.__dir__
Suggested by Petr Viktorin.
- Slight refactorings as suggested by Steve Dower.
- Removed unnecessary if check on std_abbr
Discovered this because of a missing line in branch coverage.
2020-05-16 05:20:06 -03:00
|
|
|
import os
|
|
|
|
import sysconfig
|
|
|
|
|
|
|
|
|
|
|
|
def reset_tzpath(to=None):
|
|
|
|
global TZPATH
|
|
|
|
|
|
|
|
tzpaths = to
|
|
|
|
if tzpaths is not None:
|
|
|
|
if isinstance(tzpaths, (str, bytes)):
|
|
|
|
raise TypeError(
|
|
|
|
f"tzpaths must be a list or tuple, "
|
|
|
|
+ f"not {type(tzpaths)}: {tzpaths!r}"
|
|
|
|
)
|
2020-05-29 10:34:30 -03:00
|
|
|
|
|
|
|
if not all(map(os.path.isabs, tzpaths)):
|
bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).
This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:
1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)
Differences from the reference implementation:
- The module is arranged slightly differently: the accelerated module is
`_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
some changes in the test support function. (Suggested by Victor
Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
because we may do some refactoring in the future that would likely
require this separation anyway; we may:
- include the property tests
- automatically run all the tests against both pure Python and C,
rather than manually constructing C and Python test classes (similar
to the way this works with test_datetime.py, which generates C
and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
CI, though only on the coverage jobs. Introducing a PyPI dependency as
part of the CI build was controversial, and this is seen as less of a
major change, since the coverage jobs already depend on pip and PyPI.
Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:
- Fixed reference and memory leaks
With much debugging help from Pablo Galindo
- Added smoke tests ensuring that the C and Python modules are built
The import machinery can be somewhat fragile, and the "seamlessly falls
back to pure Python" nature of this module makes it so that a problem
building the C extension or a failure to import the pure Python version
might easily go unnoticed.
- Adjustments to zoneinfo.__dir__
Suggested by Petr Viktorin.
- Slight refactorings as suggested by Steve Dower.
- Removed unnecessary if check on std_abbr
Discovered this because of a missing line in branch coverage.
2020-05-16 05:20:06 -03:00
|
|
|
raise ValueError(_get_invalid_paths_message(tzpaths))
|
|
|
|
base_tzpath = tzpaths
|
|
|
|
else:
|
|
|
|
env_var = os.environ.get("PYTHONTZPATH", None)
|
|
|
|
if env_var is not None:
|
|
|
|
base_tzpath = _parse_python_tzpath(env_var)
|
|
|
|
else:
|
|
|
|
base_tzpath = _parse_python_tzpath(
|
|
|
|
sysconfig.get_config_var("TZPATH")
|
|
|
|
)
|
|
|
|
|
|
|
|
TZPATH = tuple(base_tzpath)
|
|
|
|
|
|
|
|
|
|
|
|
def _parse_python_tzpath(env_var):
|
|
|
|
if not env_var:
|
|
|
|
return ()
|
|
|
|
|
|
|
|
raw_tzpath = env_var.split(os.pathsep)
|
|
|
|
new_tzpath = tuple(filter(os.path.isabs, raw_tzpath))
|
|
|
|
|
|
|
|
# If anything has been filtered out, we will warn about it
|
|
|
|
if len(new_tzpath) != len(raw_tzpath):
|
|
|
|
import warnings
|
|
|
|
|
|
|
|
msg = _get_invalid_paths_message(raw_tzpath)
|
|
|
|
|
|
|
|
warnings.warn(
|
|
|
|
"Invalid paths specified in PYTHONTZPATH environment variable."
|
|
|
|
+ msg,
|
|
|
|
InvalidTZPathWarning,
|
|
|
|
)
|
|
|
|
|
|
|
|
return new_tzpath
|
|
|
|
|
|
|
|
|
|
|
|
def _get_invalid_paths_message(tzpaths):
|
|
|
|
invalid_paths = (path for path in tzpaths if not os.path.isabs(path))
|
|
|
|
|
|
|
|
prefix = "\n "
|
|
|
|
indented_str = prefix + prefix.join(invalid_paths)
|
|
|
|
|
|
|
|
return (
|
|
|
|
"Paths should be absolute but found the following relative paths:"
|
|
|
|
+ indented_str
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
def find_tzfile(key):
|
|
|
|
"""Retrieve the path to a TZif file from a key."""
|
|
|
|
_validate_tzfile_path(key)
|
|
|
|
for search_path in TZPATH:
|
|
|
|
filepath = os.path.join(search_path, key)
|
|
|
|
if os.path.isfile(filepath):
|
|
|
|
return filepath
|
|
|
|
|
|
|
|
return None
|
|
|
|
|
|
|
|
|
|
|
|
_TEST_PATH = os.path.normpath(os.path.join("_", "_"))[:-1]
|
|
|
|
|
|
|
|
|
|
|
|
def _validate_tzfile_path(path, _base=_TEST_PATH):
|
|
|
|
if os.path.isabs(path):
|
|
|
|
raise ValueError(
|
|
|
|
f"ZoneInfo keys may not be absolute paths, got: {path}"
|
|
|
|
)
|
|
|
|
|
|
|
|
# We only care about the kinds of path normalizations that would change the
|
|
|
|
# length of the key - e.g. a/../b -> a/b, or a/b/ -> a/b. On Windows,
|
|
|
|
# normpath will also change from a/b to a\b, but that would still preserve
|
|
|
|
# the length.
|
|
|
|
new_path = os.path.normpath(path)
|
|
|
|
if len(new_path) != len(path):
|
|
|
|
raise ValueError(
|
|
|
|
f"ZoneInfo keys must be normalized relative paths, got: {path}"
|
|
|
|
)
|
|
|
|
|
|
|
|
resolved = os.path.normpath(os.path.join(_base, new_path))
|
|
|
|
if not resolved.startswith(_base):
|
|
|
|
raise ValueError(
|
|
|
|
f"ZoneInfo keys must refer to subdirectories of TZPATH, got: {path}"
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
|
|
del _TEST_PATH
|
|
|
|
|
|
|
|
|
2020-05-17 22:55:11 -03:00
|
|
|
def available_timezones():
|
|
|
|
"""Returns a set containing all available time zones.
|
|
|
|
|
|
|
|
.. caution::
|
|
|
|
|
|
|
|
This may attempt to open a large number of files, since the best way to
|
|
|
|
determine if a given file on the time zone search path is to open it
|
|
|
|
and check for the "magic string" at the beginning.
|
|
|
|
"""
|
|
|
|
from importlib import resources
|
|
|
|
|
|
|
|
valid_zones = set()
|
|
|
|
|
|
|
|
# Start with loading from the tzdata package if it exists: this has a
|
|
|
|
# pre-assembled list of zones that only requires opening one file.
|
|
|
|
try:
|
|
|
|
with resources.open_text("tzdata", "zones") as f:
|
|
|
|
for zone in f:
|
|
|
|
zone = zone.strip()
|
|
|
|
if zone:
|
|
|
|
valid_zones.add(zone)
|
|
|
|
except (ImportError, FileNotFoundError):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def valid_key(fpath):
|
|
|
|
try:
|
|
|
|
with open(fpath, "rb") as f:
|
|
|
|
return f.read(4) == b"TZif"
|
|
|
|
except Exception: # pragma: nocover
|
|
|
|
return False
|
|
|
|
|
|
|
|
for tz_root in TZPATH:
|
|
|
|
if not os.path.exists(tz_root):
|
|
|
|
continue
|
|
|
|
|
|
|
|
for root, dirnames, files in os.walk(tz_root):
|
|
|
|
if root == tz_root:
|
|
|
|
# right/ and posix/ are special directories and shouldn't be
|
|
|
|
# included in the output of available zones
|
|
|
|
if "right" in dirnames:
|
|
|
|
dirnames.remove("right")
|
|
|
|
if "posix" in dirnames:
|
|
|
|
dirnames.remove("posix")
|
|
|
|
|
|
|
|
for file in files:
|
|
|
|
fpath = os.path.join(root, file)
|
|
|
|
|
|
|
|
key = os.path.relpath(fpath, start=tz_root)
|
|
|
|
if os.sep != "/": # pragma: nocover
|
|
|
|
key = key.replace(os.sep, "/")
|
|
|
|
|
|
|
|
if not key or key in valid_zones:
|
|
|
|
continue
|
|
|
|
|
|
|
|
if valid_key(fpath):
|
|
|
|
valid_zones.add(key)
|
|
|
|
|
|
|
|
if "posixrules" in valid_zones:
|
|
|
|
# posixrules is a special symlink-only time zone where it exists, it
|
|
|
|
# should not be included in the output
|
|
|
|
valid_zones.remove("posixrules")
|
|
|
|
|
|
|
|
return valid_zones
|
|
|
|
|
|
|
|
|
bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).
This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:
1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)
Differences from the reference implementation:
- The module is arranged slightly differently: the accelerated module is
`_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
some changes in the test support function. (Suggested by Victor
Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
because we may do some refactoring in the future that would likely
require this separation anyway; we may:
- include the property tests
- automatically run all the tests against both pure Python and C,
rather than manually constructing C and Python test classes (similar
to the way this works with test_datetime.py, which generates C
and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
CI, though only on the coverage jobs. Introducing a PyPI dependency as
part of the CI build was controversial, and this is seen as less of a
major change, since the coverage jobs already depend on pip and PyPI.
Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:
- Fixed reference and memory leaks
With much debugging help from Pablo Galindo
- Added smoke tests ensuring that the C and Python modules are built
The import machinery can be somewhat fragile, and the "seamlessly falls
back to pure Python" nature of this module makes it so that a problem
building the C extension or a failure to import the pure Python version
might easily go unnoticed.
- Adjustments to zoneinfo.__dir__
Suggested by Petr Viktorin.
- Slight refactorings as suggested by Steve Dower.
- Removed unnecessary if check on std_abbr
Discovered this because of a missing line in branch coverage.
2020-05-16 05:20:06 -03:00
|
|
|
class InvalidTZPathWarning(RuntimeWarning):
|
|
|
|
"""Warning raised if an invalid path is specified in PYTHONTZPATH."""
|
|
|
|
|
|
|
|
|
|
|
|
TZPATH = ()
|
|
|
|
reset_tzpath()
|