This makes it more clear that a given test is definitely testing against a single-phase init (legacy) extension module. The new module is a companion to _testmultiphase.
https://github.com/python/cpython/issues/98627
test_tools.test_sundry() now uses an unittest mock to prevent the
logging module to register a real "atfork" function which kept the
logging module dictionary alive. So the logging module can be
properly unloaded. Previously, the logging module was loaded before
test_sundry(), but it's no longer the case since recent test_tools
sub-tests removals.
In `_warnings.c`, in the C equivalent of `warnings.warn_explicit()`, if the module globals are given (and not None), the warning will attempt to get the source line for the issued warning. To do this, it needs the module's loader.
Previously, it would only look up `__loader__` in the module globals. In https://github.com/python/cpython/issues/86298 we want to defer to the `__spec__.loader` if available.
The first step on this journey is to check that `loader == __spec__.loader` and issue another warning if it is not. This commit does that.
Since this is a PoC, only manual testing for now.
```python
# /tmp/foo.py
import warnings
import bar
warnings.warn_explicit(
'warning!',
RuntimeWarning,
'bar.py', 2,
module='bar knee',
module_globals=bar.__dict__,
)
```
```python
# /tmp/bar.py
import sys
import os
import pathlib
# __loader__ = pathlib.Path()
```
Then running this: `./python.exe -Wdefault /tmp/foo.py`
Produces:
```
bar.py:2: RuntimeWarning: warning!
import os
```
Uncomment the `__loader__ = ` line in `bar.py` and try it again:
```
sys:1: ImportWarning: Module bar; __loader__ != __spec__.loader (<_frozen_importlib_external.SourceFileLoader object at 0x109f7dfa0> != PosixPath('.'))
bar.py:2: RuntimeWarning: warning!
import os
```
Automerge-Triggered-By: GH:warsaw
* gh-95218: Move tests for importlib.resources into test_importlib.resources.
* Also update makefile
* Include test_importlib/resources in code ownership rule.
Previously, we were blocking the frozen imports and forcing the source
version to be used, but we did not fix up sys.meta_path or
sys.path_hooks, causing the frozen importers to leak into the source
version of the test.
Add methods enterContext() and enterClassContext() in TestCase.
Add method enterAsyncContext() in IsolatedAsyncioTestCase.
Add function enterModuleContext().
- Add requires_fork and requires_subprocess to more tests
- Skip extension import tests if dlopen is not available
- Don't assume that _testcapi is a shared extension
- Skip a lot of socket tests that don't work on Emscripten
- Skip mmap tests, mmap emulation is incomplete
- venv does not work yet
- Cannot get libc from executable
The "entire" test suite is now passing on Emscripten with EMSDK from git head (91 suites are skipped).
Confirmed with @jaraco that this indeed needs a fix.
A question that came up while I was digging into the code: I think `SelectableGroups` could similarly use `__slots__ = ()`, since its purpose seems only for convenience around `dict`, not to have attributes of its own.
Automerge-Triggered-By: GH:jaraco
* bpo-44893: Implement EntryPoint as simple class and deprecate tuple access in favor of attribute access. Syncs with importlib_metadata 4.8.1.
* Apply refactorings found in importlib_metadata 4.8.2.
* bpo-45514: Apply changes from importlib_resources@a3ef4128c6
* Mark legacy functions as deprecated in the docs and link to the migration docs in importlib_resources docs.
* Apply changes from importlib_resources@329ae9d5f2c.
* Indicate importlib.resources as a module.
Co-authored-by: Filipe Laíns <lains@riseup.net>
Rename namespace package __loader__ class to be public.
Make the old name, i.e. _NamespaceLoader, an alias for the public name, for backward compatibility.
Currently frozen modules do not have __file__ set. In their spec, origin is set to "frozen" and they are marked as not having a location. (Similarly, for frozen packages __path__ is set to an empty list.) However, for frozen stdlib modules we are able to extrapolate __file__ as long as we can determine the stdlib directory at runtime. (We now do so since gh-28586.) Having __file__ set is helpful for a number of reasons. Likewise, having a non-empty __path__ means we can import submodules of a frozen package from the filesystem (e.g. we could partially freeze the encodings module).
This change sets __file__ (and adds to __path__) for frozen stdlib modules. It uses sys._stdlibdir (from gh-28586) and the frozen module alias information (from gh-28655). All that work is done in FrozenImporter (in Lib/importlib/_bootstrap.py).
Also, if a frozen module is imported before importlib is bootstrapped (during interpreter initialization) then we fix up that module and its spec during the importlib bootstrapping step (i.e. imporlib._bootstrap._setup()) to match what gets set by FrozenImporter, including setting the file info (if the stdlib dir is known). To facilitate this, modules imported using PyImport_ImportFrozenModule() have __origname__ set using the frozen module alias info. __origname__ is popped off during importlib bootstrap.
(To be clear, even with this change the new code to set __file__ during fixups in imporlib._bootstrap._setup() doesn't actually get triggered yet. This is because sys._stdlibdir hasn't been set yet in interpreter initialization at the point importlib is bootstrapped. However, we do fix up such modules at that point to otherwise match the result of importing through FrozenImporter, just not the __file__ and __path__ parts. Doing so will require changes in the order in which things happen during interpreter initialization. That can be addressed separately. Once it is, the file-related fixup code from this PR will kick in.)
Here are things this change does not do:
* set __file__ for non-stdlib modules (no way of knowing the parent dir)
* set __file__ if the stdlib dir is not known (nor assume the expense of finding it)
* relatedly, set __file__ if the stdlib is in a zip file
* verify that the filename set to __file__ actually exists (too expensive)
* update __path__ for frozen packages that alias a non-package (since there is no package dir)
Other things this change skips, but we may do later:
* set __file__ on modules imported using PyImport_ImportFrozenModule()
* set co_filename when we unmarshal the frozen code object while importing the module (e.g. in FrozenImporter.exec_module()) -- this would allow tracebacks to show source lines
* implement FrozenImporter.get_filename() and FrozenImporter.get_source()
https://bugs.python.org/issue21736
In the list of generated frozen modules at the top of Tools/scripts/freeze_modules.py, you will find that some of the modules have a different name than the module (or .py file) that is actually frozen. Let's call each case an "alias". Aliases do not come into play until we get to the (generated) list of modules in Python/frozen.c. (The tool for freezing modules, Programs/_freeze_module, is only concerned with the source file, not the module it will be used for.)
Knowledge of which frozen modules are aliases (and the identity of the original module) normally isn't important. However, this information is valuable when we go to set __file__ on frozen stdlib modules. This change updates Tools/scripts/freeze_modules.py to map aliases to the original module name (or None if not a stdlib module) in Python/frozen.c. We also add a helper function in Python/import.c to look up a frozen module's alias and add the result of that function to the frozen info returned from find_frozen().
https://bugs.python.org/issue45020
Before this change we end up duplicating effort and throwing away data in FrozenImporter.find_spec(). Now we do the work once in find_spec() and the only thing we do in FrozenImporter.exec_module() is turn the raw frozen data into a code object and then exec it.
We've added _imp.find_frozen(), add an arg to _imp.get_frozen_object(), and updated FrozenImporter. We've also moved some code around to reduce duplication, get a little more consistency in outcomes, and be more efficient.
Note that this change is mostly necessary if we want to set __file__ on frozen stdlib modules. (See https://bugs.python.org/issue21736.)
https://bugs.python.org/issue45324
I've added a number of test-only modules. Some of those cases are covered by the recently frozen stdlib modules (and some will be once we add encodings back in). However, I figured we'd play it safe by having a set of modules guaranteed to be there during tests.
https://bugs.python.org/issue45020
Instead of explicitly enumerate test classes for run_unittest()
use the unittest ability to discover tests. This also makes these
tests discoverable and runnable with unittest.
load_tests() can be used for dynamic generating tests and adding
doctests. setUpModule(), tearDownModule() and addModuleCleanup()
can be used for running code before and after all module tests.
Here's one more small cleanup that should have been in PR gh-28319. We eliminate stdout side-effects from importing the frozen __hello__ module, and update tests accordingly. We also move the module's source file into Lib/ from Toos/freeze/flag.py.
https://bugs.python.org/issue45019
Currently we freeze several modules into the runtime. For each of these modules it is essential to bootstrapping the runtime that they be frozen. Any other stdlib module that we later freeze into the runtime is not essential. We can just as well import from the .py file. This PR lets users explicitly choose which should be used, with the new "-X frozen_modules=[on|off]" CLI flag. The default is "off" for now.
https://bugs.python.org/issue45020
This avoids the following error if DeprecationWarnings are ignored.
======================================================================
ERROR: test_entry_points_by_index (test.test_importlib.test_metadata_api.APITests)
Prior versions of Distribution.entry_points would return a
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/Python-3.10.0b3/Lib/test/test_importlib/test_metadata_api.py", line 145, in test_entry_points_by_index
expected = next(iter(caught))
StopIteration
----------------------------------------------------------------------
Ran 1402 tests in 2.125s
FAILED (errors=1, skipped=18, expected failures=1)
* bpo-43926: Cleaner metadata with PEP 566 JSON support.
* Add blurb
* Add versionchanged and versionadded declarations for changes to metadata.
* Use descriptor for PEP 566
* Unify behavior in ResourceReaderDefaultsTests and align with the behavior found in importlib_resources.
* Equip NamespaceLoader with a NamespaceReader.
* Apply changes from importlib_resources 5.0.4
* bpo-42382: In importlib.metadata, `EntryPoint` objects now expose a `.dist` object referencing the `Distribution` when constructed from a `Distribution`.
Also, sync importlib_metadata 3.3:
- Add support for package discovery under package normalization rules.
- The object returned by `metadata()` now has a formally-defined protocol called `PackageMetadata` with declared support for the `.get_all()` method.
* Add blurb
* Remove latent footnote.
* bpo-41490: ``path`` method to aggressively close handles
* Add blurb
* In ZipReader.contents, eagerly evaluate the contents to release references to the zipfile.
* Instead use _ensure_sequence to ensure any iterable from a reader is eagerly converted to a list if it's not already a sequence.
Make the the following imports lazy in test.support:
* bz2
* gzip
* lzma
* resource
* zlib
The following test.support decorators now need to be called
with parenthesis:
* @support.requires_bz2
* @support.requires_gzip
* @support.requires_lzma
* @support.requires_zlib
For example, "@requires_zlib" becomes "@requires_zlib()".
* bpo-39791: Update importlib.resources to support files() API (importlib_resources 1.5).
* 📜🤖 Added by blurb_it.
* Add some documentation about the new objects added.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Extension modules: m_traverse, m_clear and m_free functions of
PyModuleDef are no longer called if the module state was requested
but is not allocated yet. This is the case immediately after the
module is created and before the module is executed (Py_mod_exec
function). More precisely, these functions are not called if m_size is
greater than 0 and the module state (as returned by
PyModule_GetState()) is NULL.
Extension modules without module state (m_size <= 0) are not affected.
Co-Authored-By: Petr Viktorin <encukou@gmail.com>
Fix compileall.compile_dir() ddir= behavior on sub-packages.
Fixes compileall.compile_dir's ddir parameter and compileall command
line flag `-d` to no longer write the wrong pathname to the generated
pyc file for submodules beneath the root of the directory tree being
compiled. This fixes a regression introduced with Python 3.5.
Also marks the _new_ in 3.9 from PR #16012 parameters to compile_dir as keyword only (as that is the only way they will be used) and fixes an omission of them in one place from the docs.
* Improve zipfile.Path performance on zipfiles with a large number of entries.
* 📜🤖 Added by blurb_it.
* Add bpo to blurb
* Sync with importlib_metadata 1.5 (6fe70ca)
* Update blurb.
* Remove compatibility code
* Add stubs module, omitted from earlier commit
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
* bpo-39022, bpo-38594: Sync with importlib_metadata 1.3 including improved docs for custom finders and better serialization support in EntryPoints.
* 📜🤖 Added by blurb_it.
* Correct module reference
Jason R. Coombs
Dong-hee Na
Sebastian Berg
Eric Snow
Anh71me
Jeong YunWon
Victor Stinner
Barry Warsaw
Nikita Sobolev
Brett Cannon
Irit Katriel
Pablo Galindo Salgado
Christian Heimes
Florian Bruhin
Filipe Laíns
Serhiy Storchaka
Arie Bovenberg
Steve Dower
Miro Hrončok
Mohamad Mansour
Shreyan Avigyan
Inada Naoki
Antoine Pitrou
Neil Schemenauer
Hai Shi
Roman Yurchak
idomic
Gregory P. Smith
Adam Johnson
Benjamin Peterson