* bpo-44893: Implement EntryPoint as simple class and deprecate tuple access in favor of attribute access. Syncs with importlib_metadata 4.8.1.
* Apply refactorings found in importlib_metadata 4.8.2.
* bpo-45514: Apply changes from importlib_resources@a3ef4128c6
* Mark legacy functions as deprecated in the docs and link to the migration docs in importlib_resources docs.
* Apply changes from importlib_resources@329ae9d5f2c.
* Indicate importlib.resources as a module.
Co-authored-by: Filipe Laíns <lains@riseup.net>
Rename namespace package __loader__ class to be public.
Make the old name, i.e. _NamespaceLoader, an alias for the public name, for backward compatibility.
Currently frozen modules do not have __file__ set. In their spec, origin is set to "frozen" and they are marked as not having a location. (Similarly, for frozen packages __path__ is set to an empty list.) However, for frozen stdlib modules we are able to extrapolate __file__ as long as we can determine the stdlib directory at runtime. (We now do so since gh-28586.) Having __file__ set is helpful for a number of reasons. Likewise, having a non-empty __path__ means we can import submodules of a frozen package from the filesystem (e.g. we could partially freeze the encodings module).
This change sets __file__ (and adds to __path__) for frozen stdlib modules. It uses sys._stdlibdir (from gh-28586) and the frozen module alias information (from gh-28655). All that work is done in FrozenImporter (in Lib/importlib/_bootstrap.py).
Also, if a frozen module is imported before importlib is bootstrapped (during interpreter initialization) then we fix up that module and its spec during the importlib bootstrapping step (i.e. imporlib._bootstrap._setup()) to match what gets set by FrozenImporter, including setting the file info (if the stdlib dir is known). To facilitate this, modules imported using PyImport_ImportFrozenModule() have __origname__ set using the frozen module alias info. __origname__ is popped off during importlib bootstrap.
(To be clear, even with this change the new code to set __file__ during fixups in imporlib._bootstrap._setup() doesn't actually get triggered yet. This is because sys._stdlibdir hasn't been set yet in interpreter initialization at the point importlib is bootstrapped. However, we do fix up such modules at that point to otherwise match the result of importing through FrozenImporter, just not the __file__ and __path__ parts. Doing so will require changes in the order in which things happen during interpreter initialization. That can be addressed separately. Once it is, the file-related fixup code from this PR will kick in.)
Here are things this change does not do:
* set __file__ for non-stdlib modules (no way of knowing the parent dir)
* set __file__ if the stdlib dir is not known (nor assume the expense of finding it)
* relatedly, set __file__ if the stdlib is in a zip file
* verify that the filename set to __file__ actually exists (too expensive)
* update __path__ for frozen packages that alias a non-package (since there is no package dir)
Other things this change skips, but we may do later:
* set __file__ on modules imported using PyImport_ImportFrozenModule()
* set co_filename when we unmarshal the frozen code object while importing the module (e.g. in FrozenImporter.exec_module()) -- this would allow tracebacks to show source lines
* implement FrozenImporter.get_filename() and FrozenImporter.get_source()
https://bugs.python.org/issue21736
In the list of generated frozen modules at the top of Tools/scripts/freeze_modules.py, you will find that some of the modules have a different name than the module (or .py file) that is actually frozen. Let's call each case an "alias". Aliases do not come into play until we get to the (generated) list of modules in Python/frozen.c. (The tool for freezing modules, Programs/_freeze_module, is only concerned with the source file, not the module it will be used for.)
Knowledge of which frozen modules are aliases (and the identity of the original module) normally isn't important. However, this information is valuable when we go to set __file__ on frozen stdlib modules. This change updates Tools/scripts/freeze_modules.py to map aliases to the original module name (or None if not a stdlib module) in Python/frozen.c. We also add a helper function in Python/import.c to look up a frozen module's alias and add the result of that function to the frozen info returned from find_frozen().
https://bugs.python.org/issue45020
Before this change we end up duplicating effort and throwing away data in FrozenImporter.find_spec(). Now we do the work once in find_spec() and the only thing we do in FrozenImporter.exec_module() is turn the raw frozen data into a code object and then exec it.
We've added _imp.find_frozen(), add an arg to _imp.get_frozen_object(), and updated FrozenImporter. We've also moved some code around to reduce duplication, get a little more consistency in outcomes, and be more efficient.
Note that this change is mostly necessary if we want to set __file__ on frozen stdlib modules. (See https://bugs.python.org/issue21736.)
https://bugs.python.org/issue45324
I've added a number of test-only modules. Some of those cases are covered by the recently frozen stdlib modules (and some will be once we add encodings back in). However, I figured we'd play it safe by having a set of modules guaranteed to be there during tests.
https://bugs.python.org/issue45020
Instead of explicitly enumerate test classes for run_unittest()
use the unittest ability to discover tests. This also makes these
tests discoverable and runnable with unittest.
load_tests() can be used for dynamic generating tests and adding
doctests. setUpModule(), tearDownModule() and addModuleCleanup()
can be used for running code before and after all module tests.
Here's one more small cleanup that should have been in PR gh-28319. We eliminate stdout side-effects from importing the frozen __hello__ module, and update tests accordingly. We also move the module's source file into Lib/ from Toos/freeze/flag.py.
https://bugs.python.org/issue45019
Currently we freeze several modules into the runtime. For each of these modules it is essential to bootstrapping the runtime that they be frozen. Any other stdlib module that we later freeze into the runtime is not essential. We can just as well import from the .py file. This PR lets users explicitly choose which should be used, with the new "-X frozen_modules=[on|off]" CLI flag. The default is "off" for now.
https://bugs.python.org/issue45020
This avoids the following error if DeprecationWarnings are ignored.
======================================================================
ERROR: test_entry_points_by_index (test.test_importlib.test_metadata_api.APITests)
Prior versions of Distribution.entry_points would return a
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/Python-3.10.0b3/Lib/test/test_importlib/test_metadata_api.py", line 145, in test_entry_points_by_index
expected = next(iter(caught))
StopIteration
----------------------------------------------------------------------
Ran 1402 tests in 2.125s
FAILED (errors=1, skipped=18, expected failures=1)
* bpo-43926: Cleaner metadata with PEP 566 JSON support.
* Add blurb
* Add versionchanged and versionadded declarations for changes to metadata.
* Use descriptor for PEP 566
* Unify behavior in ResourceReaderDefaultsTests and align with the behavior found in importlib_resources.
* Equip NamespaceLoader with a NamespaceReader.
* Apply changes from importlib_resources 5.0.4
* bpo-42382: In importlib.metadata, `EntryPoint` objects now expose a `.dist` object referencing the `Distribution` when constructed from a `Distribution`.
Also, sync importlib_metadata 3.3:
- Add support for package discovery under package normalization rules.
- The object returned by `metadata()` now has a formally-defined protocol called `PackageMetadata` with declared support for the `.get_all()` method.
* Add blurb
* Remove latent footnote.
* bpo-41490: ``path`` method to aggressively close handles
* Add blurb
* In ZipReader.contents, eagerly evaluate the contents to release references to the zipfile.
* Instead use _ensure_sequence to ensure any iterable from a reader is eagerly converted to a list if it's not already a sequence.
Make the the following imports lazy in test.support:
* bz2
* gzip
* lzma
* resource
* zlib
The following test.support decorators now need to be called
with parenthesis:
* @support.requires_bz2
* @support.requires_gzip
* @support.requires_lzma
* @support.requires_zlib
For example, "@requires_zlib" becomes "@requires_zlib()".
* bpo-39791: Update importlib.resources to support files() API (importlib_resources 1.5).
* 📜🤖 Added by blurb_it.
* Add some documentation about the new objects added.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Extension modules: m_traverse, m_clear and m_free functions of
PyModuleDef are no longer called if the module state was requested
but is not allocated yet. This is the case immediately after the
module is created and before the module is executed (Py_mod_exec
function). More precisely, these functions are not called if m_size is
greater than 0 and the module state (as returned by
PyModule_GetState()) is NULL.
Extension modules without module state (m_size <= 0) are not affected.
Co-Authored-By: Petr Viktorin <encukou@gmail.com>
Fix compileall.compile_dir() ddir= behavior on sub-packages.
Fixes compileall.compile_dir's ddir parameter and compileall command
line flag `-d` to no longer write the wrong pathname to the generated
pyc file for submodules beneath the root of the directory tree being
compiled. This fixes a regression introduced with Python 3.5.
Also marks the _new_ in 3.9 from PR #16012 parameters to compile_dir as keyword only (as that is the only way they will be used) and fixes an omission of them in one place from the docs.
* Improve zipfile.Path performance on zipfiles with a large number of entries.
* 📜🤖 Added by blurb_it.
* Add bpo to blurb
* Sync with importlib_metadata 1.5 (6fe70ca)
* Update blurb.
* Remove compatibility code
* Add stubs module, omitted from earlier commit
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
* bpo-39022, bpo-38594: Sync with importlib_metadata 1.3 including improved docs for custom finders and better serialization support in EntryPoints.
* 📜🤖 Added by blurb_it.
* Correct module reference
There are plenty of legitimate scripts in the tree that begin with a
`#!`, but also a few that seem to be marked executable by mistake.
Found them with this command -- it gets executable files known to Git,
filters to the ones that don't start with a `#!`, and then unmarks
them as executable:
$ git ls-files --stage \
| perl -lane 'print $F[3] if (!/^100644/)' \
| while read f; do
head -c2 "$f" | grep -qxF '#!' \
|| chmod a-x "$f"; \
done
Looking at the list by hand confirms that we didn't sweep up any
files that should have the executable bit after all. In particular
* The `.psd` files are images from Photoshop.
* The `.bat` files sure look like things that can be run.
But we have lots of other `.bat` files, and they don't have
this bit set, so it must not be needed for them.
Automerge-Triggered-By: @benjaminp
Imports now raise `TypeError` instead of `ValueError` for relative import failures. This makes things consistent between `builtins.__import__` and `importlib.__import__` as well as using a more natural import for the failure.
https://bugs.python.org/issue37444
Automerge-Triggered-By: @brettcannon
Move `threaded_import_hangers`, a dependency of `test_threaded_import`, to the directory `test_importlib/`. Also update the import references for `threaded_import_hangers` in `test_threaded_import`.
https://bugs.python.org/issue19696
Replacing the deprecated method "random.choose" to "random.choice" was technically not part of the original issue. However, it was discussed in the talk page and involved one of the files being moved. I assumed this was too minor to justify the creation of a separate issue.
Also, I added my name to the contributors list in Misc/ACKS. This will be my third PR to cpython, forgot to do it in the previous ones.
https://bugs.python.org/issue19696
This commit contains the implementation of PEP570: Python positional-only parameters.
* Update Grammar/Grammar with new typedarglist and varargslist
* Regenerate grammar files
* Update and regenerate AST related files
* Update code object
* Update marshal.c
* Update compiler and symtable
* Regenerate importlib files
* Update callable objects
* Implement positional-only args logic in ceval.c
* Regenerate frozen data
* Update standard library to account for positional-only args
* Add test file for positional-only args
* Update other test files to account for positional-only args
* Add News entry
* Update inspect module and related tests
Unconditional forcing of ``CHECKED_HASH`` invalidation was introduced in
3.7.0 in bpo-29708. The change is bad, as it unconditionally overrides
*invalidation_mode*, even if it was passed as an explicit argument to
``py_compile.compile()`` or ``compileall``. An environment variable
should *never* override an explicit argument to a library function.
That change leads to multiple test failures if the ``SOURCE_DATE_EPOCH``
environment variable is set.
This changes ``py_compile.compile()`` to only look at
``SOURCE_DATE_EPOCH`` if no explicit *invalidation_mode* was specified.
I also made various relevant tests run with explicit control over the
value of ``SOURCE_DATE_EPOCH``.
While looking at this, I noticed that ``zipimport`` does not work
with hash-based .pycs _at all_, though I left the fixes for
subsequent commits.
In some development setups it is inconvenient or impossible to write bytecode
caches to the code tree, but the bytecode caches are still useful. The
PYTHONPYCACHEPREFIX environment variable allows specifying an alternate
location for cached bytecode files, within which a directory tree mirroring the code
tree will be created. This cache tree is then used (for both reading and writing)
instead of the local `__pycache__` subdirectory within each source directory.
Exposed at runtime as sys.pycache_prefix (defaulting to None), and can
be set from the CLI as "-X pycache_prefix=path".
Patch by Carl Meyer.
Multi-phase initialized modules allow m_traverse to be called while the
module is still being initialized, so module authors may need to account
for that.
Jason R. Coombs
Nikita Sobolev
Miro Hrončok
Barry Warsaw
Eric Snow
Mohamad Mansour
Serhiy Storchaka
Shreyan Avigyan
Brett Cannon
Steve Dower
Inada Naoki
Antoine Pitrou
Neil Schemenauer
Hai Shi
Victor Stinner
Roman Yurchak
idomic
Gregory P. Smith
Adam Johnson
Benjamin Peterson
Min ho Kim
Greg Price
Ngalim Siregar
Kyle Stanley
Victor Stinner
Kyle Stanley
Joannah Nanjekye
Anthony Sottile
penguindustin
Pablo Galindo
Windson yang
Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి)
Elvis Pranskevichus
Ned Deily
Carl Meyer
Marcel Plch
Brett Cannon