Unconditional forcing of ``CHECKED_HASH`` invalidation was introduced in
3.7.0 in bpo-29708. The change is bad, as it unconditionally overrides
*invalidation_mode*, even if it was passed as an explicit argument to
``py_compile.compile()`` or ``compileall``. An environment variable
should *never* override an explicit argument to a library function.
That change leads to multiple test failures if the ``SOURCE_DATE_EPOCH``
environment variable is set.
This changes ``py_compile.compile()`` to only look at
``SOURCE_DATE_EPOCH`` if no explicit *invalidation_mode* was specified.
I also made various relevant tests run with explicit control over the
value of ``SOURCE_DATE_EPOCH``.
While looking at this, I noticed that ``zipimport`` does not work
with hash-based .pycs _at all_, though I left the fixes for
subsequent commits.
(cherry picked from commit a6b3ec5b6d)
Co-authored-by: Elvis Pranskevichus <elvis@magic.io>
Importing ProcessPoolExecutor may hang or cause an error when the import
accesses urandom on a low resource platform
https://bugs.python.org/issue29877
(cherry picked from commit 1d817e4c82)
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.
While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:
- The core changes to importlib to understand how to read, validate, and
regenerate hash-based pycs.
- Support for generating hash-based pycs in py_compile and compileall.
- Modifications to our siphash implementation to support passing a custom
key. We then expose it to importlib through _imp.
- Updates to all places in the interpreter, standard library, and tests that
manually generate or parse pyc files to grok the new format.
- Support in the interpreter command line code for long options like
--check-hash-based-pycs.
- Tests and documentation for all of the above.
Issue #26100:
* Add subprocess._optim_args_from_interpreter_flags()
* Add test.support.optim_args_from_interpreter_flags()
* Use new functions in distutils, test_cmd_line_script, test_compileall and
test_inspect
The change enables test_details() test of test_inspect when -O or -OO command
line option is used.
Issue #26101: Exclude Lib/test/ from sys.path in test_compilepath(). The
directory contains invalid Python files like Lib/test/badsyntax_pep3120.py,
whereas the test ensures that all files can be compiled.
Before the argparse conversion, compileall would (sometimes) accept multiple
paths when -d was specified. Afterward, it does not. The corresponding check
in the original code claimed to prevent multiple *directories* from being
specified...but it didn't really work even to do that. So this patch fixes
the regression by invoking the consenting adults rule: if you specify a
combination of arguments to compileall that produces files with inconsistent
destdirs (which you could do before), it is on you.
Patch by Jake Garver.
sys.path has an unwritable __pycache__ directory.
This typically comes up when someone runs the test suite from an
administrative install of Python on Windows where the user does not
have write permissions to the stdlib's directory.
Thanks to Zachary Ware and Matthias Klose for reporting bugs related
to this issue.
(grafted from 34bbd537b3e688dfbb6498e9083445a6a72fc4b1)
sys.path has an unwritable __pycache__ directory.
This typically comes up when someone runs the test suite from an
administrative install of Python on Windows where the user does not
have write permissions to the stdlib's directory.
Thanks to Zachary Ware and Matthias Klose for reporting bugs related
to this issue.
The concept of .pyo files no longer exists. Now .pyc files have an
optional `opt-` tag which specifies if any extra optimizations beyond
the peepholer were applied.
quiet parameters of compile_{dir, file, path} functions now have
a multilevel value.
Also, -q option of the CLI now have a multilevel value.
Patch by Thomas Kluyver.
Both compileall.compile_dir() and the CLI for compileall now allow for
specifying how many workers to use (or 0 to use all CPUs).
Thanks to Claudiu Popa for the patch.
Try this again, hopefully the right way this time.
Arfrever Taifersar Arahesis reported that test_compileall failed during Gentoo
install because it was tyring to write .pyc files to a read-only system
directory during test_no_args_compiles_path. Having the tests call python
with -S should eliminate the system directories from the path.
Arfrever Taifersar Arahesis reported that test_compileall failed during Gentoo
install because it was tyring to write .pyc files to a read-only system
directory during test_no_args_compiles_path. Having subprocess call python
with -S should eliminate the system directories from the path.
Before the introduction of filename arguments to compileall it gave semi useful
messages about not being able to 'list' names that weren't valid directories.
This fix restores that behavior. In addition to the test for this case, the
patch also adds a test for the default behavior of compileall when no arguments
are provided, and fixes a bug in one of the previously added tests.
* Patch contributed by Arfrever Frehtes Taifersar Arahesis.
* Test added by Barry
Also, improve Makefile's deletion of __pycache__ directories so e.g. 'make
distclean' doesn't fail if no __pycache__ directories exist.
Miss Islington (bot)
Benjamin Peterson
Berker Peksag
Christian Heimes
Victor Stinner
Brett Cannon
R David Murray
Serhiy Storchaka
Victor Stinner
Georg Brandl
Éric Araujo
Barry Warsaw
Martin v. Löwis
Matthias Klose