Commit Graph

98 Commits

Author SHA1 Message Date
Desmond Cheong 1fb9bd222b
gh-103200: Fix performance issues with `zipimport.invalidate_caches()` (GH-103208)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2023-07-07 22:02:13 +00:00
Barry Warsaw 326997829d
gh-98040: Remove find_loader, find_module and other deprecated APIs (#98059)
* Remove deprecated classes from pkgutil
* Remove some other PEP 302 obsolescence
* Use find_spec instead of load_module
* Remove more tests of PEP 302 obsolete APIs
* Remove another bunch of tests using obsolete load_modules()
* Remove deleted names from __all__
* Remove obsolete footnote
* imp is removed
* Remove `imp` from generated stdlib names
* What's new and blurb
* Update zipimport documentation for the removed methods
* Fix some Windows tests
* Remove any test (or part of a test) that references `find_module()`.
* Use assertIsNone() / assertIsNotNone() consistently.
* Update Doc/reference/import.rst
* We don't need pkgutil._get_spec() any more either
*  test.test_importlib.fixtures.NullFinder
* ...BadLoaderFinder.find_module
* ...test_api.InvalidatingNullFinder.find_module
* ...test.test_zipimport test of z.find_module
* Suppress cross-references to find_loader and find_module
* Suppress cross-references to Finder
* Suppress cross-references to pkgutil.ImpImporter and pkgutil.ImpLoader

---------

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Adam Turner <9087854+aa-turner@users.noreply.github.com>
2023-05-03 04:55:22 -07:00
Thomas Grainger 6da988a46c
gh-91181: drop support for bytes on sys.path (GH-31934)
Support for bytes broke sometime between Python 3.2 and 3.6 and has been broken ever since. Trying to bring back supports is surprisingly difficult in the face of -b and checking for keys in sys.path_importer_cache. Since the support was broken for so long, trying to overcome the difficulty of bringing back the support has been deemed not worth it.

Co-authored-by: Eryk Sun <eryksun@gmail.com>
Co-authored-by: Brett Cannon <brett@python.org>
2022-07-16 18:07:53 -07:00
Victor Stinner 92bcb26d00
gh-94379: Remove zipimport find_loader() and find_module() methods (#94380)
zipimport: Remove find_loader() and find_module() methods, deprecated
in Python 3.10: use the find_spec() method instead. See PEP 451 for
the rationale.
2022-07-05 12:11:42 +02:00
jackh-ncl 484a2357c8
gh-92886: Fix tests that fail when running with optimizations (`-O`) in `test_zipimport.py` (GH-93236) 2022-06-10 15:57:04 -07:00
Christian Heimes 137fd3d88a
gh-90473: Decrease recursion limit and skip tests on WASI (GH-92803) 2022-05-19 12:43:16 +02:00
Irit Katriel a287b31bcb
bpo-46411: Remove unnecessary calls to sys.exc_info() in tests (GH-30638) 2022-01-18 07:05:16 +00:00
Serhiy Storchaka 40348acc18
bpo-45229: Remove test_main in many tests (GH-28405)
Instead of explicitly enumerate test classes for run_unittest()
use the unittest ability to discover tests. This also makes these
tests discoverable and runnable with unittest.

load_tests() can be used for dynamic generating tests and adding
doctests. setUpModule(), tearDownModule() and addModuleCleanup()
can be used for running code before and after all module tests.
2021-09-19 15:27:33 +03:00
Brett Cannon 209b7035f7
bpo-45183: don't raise an exception when calling zipimport.zipimporter.find_spec() when the zip file is missing and the internal cache has been reset (GH-28435)
This can occur when the zip file gets deleted, you call zipimport.zipimporter.invalidate_cache(), and then try to use zipimport.zipimporter.find_spec() (i.e. you left the zip file path on sys.path).
2021-09-17 16:48:17 -07:00
Serhiy Storchaka 851811f577
bpo-5846: Do not use obsolete unittest functions. (GH-28303)
Get rid of use of makeSuite() and findTestCases().
Also make test_math and test_threading_local discoverable.
2021-09-13 10:49:53 +03:00
Ammar Askar bb21e28fd0
bpo-34990: Treat the pyc header's mtime in compileall as an unsigned int (GH-19708)
Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>
2021-08-24 12:13:32 +03:00
Ammar Askar 4823d9a512
bpo-43950: Add option to opt-out of PEP-657 (GH-27023)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
2021-07-07 20:07:12 +01:00
Ammar Askar 5644c7b3ff
bpo-43950: Print columns in tracebacks (PEP 657) (GH-26958)
The traceback.c and traceback.py mechanisms now utilize the newly added code.co_positions and PyCode_Addr2Location
to print carets on the specific expressions involved in a traceback.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2021-07-05 00:14:33 +01:00
Desmond Cheong 3abf6f0102
bpo-14678: Update zipimport to support importlib.invalidate_caches() (GH-24159)
Added an invalidate_caches() method to the zipimport.zipimporter class based on the implementation of importlib.FileFinder.invalidate_caches(). This was done by adding a get_files() method and an _archive_mtime attribute to zipimport.zipimporter to check for updates or cache invalidation whenever the cache of files and toc entry information in the zipimporter is accessed.
2021-03-08 12:06:02 -08:00
Irit Katriel fb34096140
bpo-24792: Fix zipimporter masking the cause of import errors (GH-22204)
zipimport's _unmarshal_code swallows import errors and then _get_module_code doesn't know the cause of the error, and returns the generic, and sometimes incorrect, 'could not find...'.

Automerge-Triggered-By: GH:brettcannon
2020-12-18 16:09:54 -08:00
Brett Cannon 2de5097ba4
bpo-26131: Deprecate usage of load_module() (GH-23469)
Raise an ImportWarning when the import system falls back on load_module(). As for implementations of load_module(), raise a DeprecationWarning.
2020-12-04 15:39:21 -08:00
Brett Cannon d2e94bb084
bpo-42131: Add PEP 451-related methods to zipimport (GH-23187)
Specifically, find_spec(), create_module(), and exec_module().

Co-authored-by: Nick Coghlan <ncoghlan@gmail.com>
2020-11-13 15:14:58 -08:00
Hai Shi 96a6a6d42b
bpo-40275: Use new test.support helper submodules in tests (GH-21412) 2020-07-09 15:25:10 +02:00
Hai Shi a3ec3ad9e2
bpo-40275: More lazy imports in test.support (GH-20131)
Make the the following imports lazy in test.support:

* bz2
* gzip
* lzma
* resource
* zlib

The following test.support decorators now need to be called
with parenthesis:

* @support.requires_bz2
* @support.requires_gzip
* @support.requires_lzma
* @support.requires_zlib

For example, "@requires_zlib" becomes "@requires_zlib()".
2020-05-19 00:02:57 +02:00
Xtreak 79f02fee1a bpo-39033: Fix NameError in zipimport during hash validation (GH-17588)
Patch by Karthikeyan Singaravelan.
2019-12-16 09:34:12 +10:00
Serhiy Storchaka 5b10b98247
bpo-22831: Use "with" to avoid possible fd leaks in tests (part 2). (GH-10929) 2019-03-05 10:06:26 +02:00
Zackery Spytz 5a5ce064b3 bpo-5950: Support reading zips with comments in zipimport (#9548)
* bpo-5950: Support reading zips with comments in zipimport
2018-09-25 15:15:47 -04:00
Serhiy Storchaka b2984ab9a7
bpo-25711: Remove outdated zipimport tests. (GH-9404)
They were specific to the C implementation.
2018-09-19 17:43:33 +03:00
Serhiy Storchaka 79d1c2e6c9
bpo-25711: Rewrite zipimport in pure Python. (GH-6809) 2018-09-18 22:22:29 +03:00
Benjamin Peterson 42aa93b8ff
closes bpo-31650: PEP 552 (Deterministic pycs) implementation (#4575)
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

- The core changes to importlib to understand how to read, validate, and
  regenerate hash-based pycs.

- Support for generating hash-based pycs in py_compile and compileall.

- Modifications to our siphash implementation to support passing a custom
  key. We then expose it to importlib through _imp.

- Updates to all places in the interpreter, standard library, and tests that
  manually generate or parse pyc files to grok the new format.

- Support in the interpreter command line code for long options like
  --check-hash-based-pycs.

- Tests and documentation for all of the above.
2017-12-09 10:26:52 -08:00
Oren Milman db60a5bfa5 bpo-31781: Prevent crashes when calling methods of an uninitialized zipimport.zipimporter object (GH-3986) 2017-10-20 13:42:35 -07:00
Oren Milman 01c6a8859e bpo-31602: Fix an assertion failure in zipimporter.get_source() in case of a bad zlib.decompress() (GH-3784)
While a rare potential failure (it requires swapping out zlib.decompress() itself and forcing it to return a non-bytes object), this change prevents a potential C-level assertion failure and instead substitutes it with an exception.

Thanks to Oren Milman for the patch.
2017-09-29 11:34:31 -07:00
Oren Milman 631fdee6e6 bpo-31291: Fixed an assertion failure in zipimport.zipimporter.get_data() (#3226)
if pathname.replace('/', '\\') returns non-string.
2017-08-29 20:40:15 +03:00
Berker Peksag 8ee9edbf45 Issue #28131: Merge from 3.5 2016-09-14 08:10:45 +03:00
Berker Peksag 4aa74c429c Issue #28131: Fix a regression in zipimport's compile_source()
zipimport should use the same optimization level as the interpreter.
2016-09-14 08:09:48 +03:00
Steve Dower ff70fc2206 Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch by Eryk Sun) 2016-09-09 17:33:37 -07:00
Steve Dower 8dcc48ee3b Issue #25758: Prevents zipimport from unnecessarily encoding a filename (patch by Eryk Sun) 2016-09-09 17:27:33 -07:00
Serhiy Storchaka febc332056 Issue #26754: Undocumented support of general bytes-like objects
as path in compile() and similar functions is now deprecated.
2016-08-06 23:29:29 +03:00
Serhiy Storchaka f95de0e8cc Issue #26754: PyUnicode_FSDecoder() accepted a filename argument encoded as
an iterable of integers. Now only strings and byte-like objects are accepted.
2016-06-18 13:56:16 +03:00
Serhiy Storchaka 9305d83425 Issue #26754: PyUnicode_FSDecoder() accepted a filename argument encoded as
an iterable of integers. Now only strings and byte-like objects are accepted.
2016-06-18 13:53:36 +03:00
Brett Cannon 43cfd8240c Merge for issue #17633 2016-01-15 11:23:46 -08:00
Brett Cannon 56aae8f304 Issue #17633: Improve support for namespace packages with zipimport.
Previously zipimport mistakenly limited namespace support to only the
top-level of the zipfile when it should have supported an arbitrary
depth.

Thanks to Phil Connel for the bug report and initial patch and Mike
Romberg for the final patch.
2016-01-15 11:22:19 -08:00
Robert Collins 9e1bf4968f Issue #8585: improved tests for zipimporter2. Patch from Mark Lawrence. 2015-07-23 02:57:56 +12:00
Brett Cannon f299abdafa Issue #23731: Implement PEP 488.
The concept of .pyo files no longer exists. Now .pyc files have an
optional `opt-` tag which specifies if any extra optimizations beyond
the peepholer were applied.
2015-04-13 14:21:02 -04:00
Victor Stinner 57d516bd6b Issue #23696: Remove test on ZipImportError.__context__ because the context is
None on Windows.

When the file is not readable, the error occurs at open on UNIX. On Windows,
the error only occurs at the first operation on the open file. It would require
to many changes to set __context__ to an OSError for all file operations, for a
little benefit (__context__ is almost never used).
2015-03-20 13:48:36 +01:00
Victor Stinner fbd6f9ed12 Issue #23696: Chain ZipImportError to the OSError 2015-03-20 10:52:25 +01:00
Benjamin Peterson ec91cd6040 merge backout for #20621 2014-02-16 14:17:28 -05:00
Benjamin Peterson eb43736de2 backout 2807a5f011e4 for causing #20621 2014-02-16 14:12:57 -05:00
Gregory P. Smith 2e385e2592 Fixes issue #19081: When a zipimport .zip file in sys.path being imported from
is modified during the lifetime of the Python process after zipimport has
already cached the zip's table of contents we detect this and recover
rather than read bad data from the .zip (causing odd import errors).
2014-01-07 18:34:23 -08:00
Gregory P. Smith 2bcbc14117 Fixes Issue #19081: When a zipimport .zip file in sys.path being imported from
is modified during the lifetime of the Python process after zipimport has
already cached the zip's table of contents we detect this and recover
rather than read bad data from the .zip (causing odd import errors).
2014-01-07 18:30:07 -08:00
Zachary Ware 101d9e7250 Issue 19572: More silently skipped tests explicitly skipped. 2013-12-08 00:44:27 -06:00
Zachary Ware 9fe6d86709 Issue 19572: More silently skipped tests explicitly skipped. 2013-12-08 00:20:35 -06:00
Brett Cannon 9529fbfd36 Issue #17177: Stop using imp in a bunch of tests 2013-06-15 17:11:25 -04:00
Brett Cannon 638ce0779b Move code from test_importhooks into test_zipimport. 2013-06-12 15:57:01 -04:00
Serhiy Storchaka 0e6b7b5cd2 Issue #8745: Small speed up zipimport on Windows. Patch by Catalin Iacob. 2013-02-16 17:43:45 +02:00