Commit Graph

39 Commits

Author SHA1 Message Date
Lumír 'Frenzy' Balhar adc72bb2f9
bpo-38112: Document that compileall.compile_[dir,file] also accept multiple opt levels (GH-20174) 2020-05-18 15:23:37 +02:00
Shantanu a2b3cdd661
bpo-40445: Update compileall.compile_dir docs (GH-19806) 2020-05-15 14:28:23 -07:00
Lumír 'Frenzy' Balhar e77d428856
bpo-40495: compileall option to hardlink duplicate pyc files (GH-19901)
compileall is now able to use hardlinks to prevent duplicates in a
case when .pyc files for different optimization levels have the same content.

Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Co-authored-by: Victor Stinner <vstinner@python.org>
2020-05-14 16:17:22 +02:00
Gregory P. Smith 02673352b5
bpo-39769: Fix compileall ddir for subpkgs. (GH-18676)
Fix compileall.compile_dir() ddir= behavior on sub-packages.

Fixes compileall.compile_dir's ddir parameter and compileall command
line flag `-d` to no longer write the wrong pathname to the generated
pyc file for submodules beneath the root of the directory tree being
compiled.  This fixes a regression introduced with Python 3.5.

Also marks the _new_ in 3.9 from PR #16012 parameters to compile_dir as keyword only (as that is the only way they will be used) and fixes an omission of them in one place from the docs.
2020-02-28 17:28:37 -08:00
Lumír 'Frenzy' Balhar 8e7bb991de bpo-38112: Compileall improvements (GH-16012)
* Raise the limit of maximum path depth to actual  recursion limit

* Add posibilities to adjust a path compiled in .pyc  file.

Now, you can:
- Strip a part of path from a beggining of path into compiled file
   example "-s /test /test/build/real/test.py" → "build/real/test.py"
- Append some new path to a beggining of path into compiled file
   example "-p /boo real/test.py" → "/boo/real/test.py"

You can also use both options in the same time. In that case,
striping is done before appending.

* Add a possibility to specify multiple optimization levels

Each optimization level then leads to separated compiled file.
Use `action='append'` instead of `nargs='+'` for the -o option.
Instead of `-o 0 1 2`, specify `-o 0 -o 1 -o 2`. It's more to type,
but much more explicit.

* Add a symlinks limitation feature

This feature allows us to limit byte-compilation of symbolic
links if they are pointing outside specified dir (build root
for example).
2019-09-26 08:28:26 +02:00
Hai Shi 68e495df90 bpo-37775: Update compileall doc for invalidation_mode parameter (GH-15148) 2019-08-15 00:03:11 +02:00
Antoine Pitrou 1a2dd82f56
bpo-36786: Run compileall in parallel during "make install" (GH-13078) 2019-05-15 23:45:18 +02:00
NAKAMURA Osamu 0983fcd0d5 Doc: Use `option` word for command line interface. (GH-12142)
For command line option, `option` is better than `parameter`.
2019-03-04 20:43:43 -08:00
Elvis Pranskevichus a6b3ec5b6d bpo-34022: Stop forcing of hash-based invalidation with SOURCE_DATE_EPOCH (GH-9607)
Unconditional forcing of ``CHECKED_HASH`` invalidation was introduced in
3.7.0 in bpo-29708.  The change is bad, as it unconditionally overrides
*invalidation_mode*, even if it was passed as an explicit argument to
``py_compile.compile()`` or ``compileall``.  An environment variable
should *never* override an explicit argument to a library function.
That change leads to multiple test failures if the ``SOURCE_DATE_EPOCH``
environment variable is set.

This changes ``py_compile.compile()`` to only look at
``SOURCE_DATE_EPOCH`` if no explicit *invalidation_mode* was specified.
I also made various relevant tests run with explicit control over the
value of ``SOURCE_DATE_EPOCH``.

While looking at this, I noticed that ``zipimport`` does not work
with hash-based .pycs _at all_, though I left the fixes for
subsequent commits.
2018-10-10 18:43:14 +02:00
Carl Meyer b193fa996a bpo-33499: Add PYTHONPYCACHEPREFIX env var for alt bytecode cache location. (GH-6834)
In some development setups it is inconvenient or impossible to write bytecode
caches to the code tree, but the bytecode caches are still useful. The
PYTHONPYCACHEPREFIX environment variable allows specifying an alternate
location for cached bytecode files, within which a directory tree mirroring the code
tree will be created. This cache tree is then used (for both reading and writing)
instead of the local `__pycache__` subdirectory within each source directory.

Exposed at runtime as sys.pycache_prefix (defaulting to None), and can
be set from the CLI as "-X pycache_prefix=path".

Patch by Carl Meyer.
2018-06-16 14:40:56 +10:00
Benjamin Peterson 42aa93b8ff
closes bpo-31650: PEP 552 (Deterministic pycs) implementation (#4575)
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

- The core changes to importlib to understand how to read, validate, and
  regenerate hash-based pycs.

- Support for generating hash-based pycs in py_compile and compileall.

- Modifications to our siphash implementation to support passing a custom
  key. We then expose it to importlib through _imp.

- Updates to all places in the interpreter, standard library, and tests that
  manually generate or parse pyc files to grok the new format.

- Support in the interpreter command line code for long options like
  --check-hash-based-pycs.

- Tests and documentation for all of the above.
2017-12-09 10:26:52 -08:00
Berker Peksag 812a2b65af Issue #28226: compileall now supports pathlib 2016-10-01 00:54:18 +03:00
Terry Jan Reedy 4da945f361 Merge Issue #22558. 2016-06-11 15:06:08 -04:00
Terry Jan Reedy fa089b9b0b Issue #22558: Add remaining doc links to source code for Python-coded modules.
Reformat header above separator line (added if missing) to a common format.
Patch by Yoni Lavi.
2016-06-11 15:02:54 -04:00
Brett Cannon 1e3c3e906c Issue #25768: Make compileall functions return booleans and document
the return values as well as test them.

Thanks to Nicholas Chammas for the bug report and initial patch.
2015-12-27 13:17:04 -08:00
Serhiy Storchaka 2446eab230 Fixed a typo in the -b option. 2015-09-13 21:09:17 +03:00
Yury Selivanov f03d50c5da docs.compileall: Fix markup (rendering was off, noticed by Berker Peksag) 2015-09-09 09:32:07 -04:00
Yury Selivanov 100fc3fbc2 whatsnew/3.5: Fix library news till Py3.5a1. Update other docs. 2015-09-08 22:40:30 -04:00
Berker Peksag d86ef05a02 Issue #23917: Fall back to sequential compilation when ProcessPoolExecutor doesn't exist.
Patch by Claudiu Popa.
2015-04-22 09:39:19 +03:00
Brett Cannon f299abdafa Issue #23731: Implement PEP 488.
The concept of .pyo files no longer exists. Now .pyc files have an
optional `opt-` tag which specifies if any extra optimizations beyond
the peepholer were applied.
2015-04-13 14:21:02 -04:00
Berker Peksag 6554b86b1f Issue #21338: Add silent mode for compileall.
quiet parameters of compile_{dir, file, path} functions now have
a multilevel value.

Also, -q option of the CLI now have a multilevel value.

Patch by Thomas Kluyver.
2014-10-15 11:10:57 +03:00
Georg Brandl b83f56f915 merge with 3.4 2014-10-02 08:39:01 +02:00
Georg Brandl c0a8f8ce84 closes #22528: add source links to symtable and compileall 2014-10-02 08:38:39 +02:00
Brett Cannon f1a8df0ac9 Issue #16104: Allow compileall to do parallel bytecode compilation.
Both compileall.compile_dir() and the CLI for compileall now allow for
specifying how many workers to use (or 0 to use all CPUs).

Thanks to Claudiu Popa for the patch.
2014-09-12 10:39:48 -04:00
Benjamin Peterson 344ff4ab2b allow recursion depth to be specified (closes #19628)
Patch from Claudiu Popa.
2014-08-19 16:13:26 -05:00
Georg Brandl f5c801fdca Fix option description that is a warning in new Sphinx versions. 2014-03-18 07:44:07 +01:00
Georg Brandl 1aca953a40 Fix example ignoring ".svn" directories in compileall. 2013-04-14 12:02:43 +02:00
Éric Araujo 01606dea3d #10553: Explain why compileall has no command-line argument to control optimization 2011-03-26 03:22:55 +01:00
R. David Murray 8b24aac936 Fix argument name typo in compileall docs. 2011-02-11 22:37:16 +00:00
R. David Murray 94f58c3a65 #10454: clarify the compileall docs and help messages. 2010-12-17 16:29:07 +00:00
Éric Araujo 930df31987 Add missing docs and directives related to PEP 3147 and byte-compilation 2010-12-16 06:28:48 +00:00
Éric Araujo f68fa05f7b Add versionadded directive missing from r78983. 2010-12-16 02:10:11 +00:00
Georg Brandl 8334fd9285 Add an "optimize" parameter to compile() to control the optimization level, and provide an interface to it in py_compile, compileall and PyZipFile. 2010-12-04 10:26:46 +00:00
Éric Araujo 713d3039dc Fix usage of :option: in the docs (#9312).
:option: is used to create a link to an option of python, not to mark
up any instance of any arbitrary command-line option.  These were
changed to ````.

For modules which do have a command-line interface, lists of options
have been properly marked up with the program/cmdoption directives
combo.  Options defined in such blocks can be linked to with :option:
later in the same file, they won’t link to an option of python.

Finally, the markup of command-line fragments in optparse.rst has
been cleaned to use ``x`` instead of ``"x"``, keeping that latter
form for actual Python strings.

Patch by Eli Bendersky and Éric Araujo.
2010-11-18 16:38:46 +00:00
Barry Warsaw 28a691b7fd PEP 3147 2010-04-17 00:19:56 +00:00
Georg Brandl 71515ca191 Remove surplus empty lines and convert more files to new optional arg style. 2009-05-17 12:29:12 +00:00
Georg Brandl c2a4f4fb67 Update signature style for optional arguments, part 3. 2009-04-10 09:03:43 +00:00
Benjamin Peterson 5478b47355 Merged revisions 66457-66459,66465-66468,66483-66485,66487-66491 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r66457 | antoine.pitrou | 2008-09-13 15:30:30 -0500 (Sat, 13 Sep 2008) | 5 lines

  Issue #3850: Misc/find_recursionlimit.py was broken.

  Reviewed by A.M. Kuchling.
........
  r66458 | benjamin.peterson | 2008-09-13 17:54:43 -0500 (Sat, 13 Sep 2008) | 1 line

  fix a name issue; note all doc files should be encoded in utf8
........
  r66459 | benjamin.peterson | 2008-09-14 11:02:22 -0500 (Sun, 14 Sep 2008) | 1 line

  clarify that radix for int is not 'guessed'
........
  r66465 | skip.montanaro | 2008-09-14 21:03:05 -0500 (Sun, 14 Sep 2008) | 3 lines

  Review usage.  Fix a mistake in the new-style class definition.  Add a
  couple new definitions (CPython and virtual machine).
........
  r66466 | skip.montanaro | 2008-09-14 21:19:53 -0500 (Sun, 14 Sep 2008) | 2 lines

  Pick up a few more definitions from the glossary on the wiki.
........
  r66467 | benjamin.peterson | 2008-09-14 21:53:23 -0500 (Sun, 14 Sep 2008) | 1 line

  mention that object.__init__ no longer takes arbitrary args and kwargs
........
  r66468 | andrew.kuchling | 2008-09-15 08:08:32 -0500 (Mon, 15 Sep 2008) | 1 line

  Rewrite item a bit
........
  r66483 | georg.brandl | 2008-09-16 05:17:45 -0500 (Tue, 16 Sep 2008) | 2 lines

  Fix typo.
........
  r66484 | benjamin.peterson | 2008-09-16 16:20:28 -0500 (Tue, 16 Sep 2008) | 2 lines

  be less wordy
........
  r66485 | georg.brandl | 2008-09-17 03:45:54 -0500 (Wed, 17 Sep 2008) | 2 lines

  #3888: add some deprecated modules in whatsnew.
........
  r66487 | skip.montanaro | 2008-09-17 06:50:36 -0500 (Wed, 17 Sep 2008) | 2 lines

  usage
........
  r66488 | andrew.kuchling | 2008-09-17 07:57:04 -0500 (Wed, 17 Sep 2008) | 1 line

  Markup fixes
........
  r66489 | andrew.kuchling | 2008-09-17 07:58:22 -0500 (Wed, 17 Sep 2008) | 2 lines

  Remove comment about improvement: pystone is about the same, and
  the improvements seem to be difficult to quantify
........
  r66490 | andrew.kuchling | 2008-09-17 08:04:53 -0500 (Wed, 17 Sep 2008) | 1 line

  Note sqlite3 version; move item
........
  r66491 | benjamin.peterson | 2008-09-17 16:54:56 -0500 (Wed, 17 Sep 2008) | 1 line

  document compileall command flags
........
2008-09-17 22:25:09 +00:00
Georg Brandl 116aa62bf5 Move the 3k reST doc tree in place. 2007-08-15 14:28:22 +00:00