2021-04-08 17:32:21 -03:00
|
|
|
****************
|
|
|
|
Configure Python
|
|
|
|
****************
|
|
|
|
|
2022-12-05 10:41:44 -04:00
|
|
|
Build Requirements
|
|
|
|
==================
|
|
|
|
|
|
|
|
Features required to build CPython:
|
|
|
|
|
|
|
|
* A `C11 <https://en.cppreference.com/w/c/11>`_ compiler. `Optional C11
|
|
|
|
features
|
|
|
|
<https://en.wikipedia.org/wiki/C11_(C_standard_revision)#Optional_features>`_
|
|
|
|
are not required.
|
|
|
|
|
|
|
|
* Support for `IEEE 754 <https://en.wikipedia.org/wiki/IEEE_754>`_ floating
|
|
|
|
point numbers and `floating point Not-a-Number (NaN)
|
|
|
|
<https://en.wikipedia.org/wiki/NaN#Floating_point>`_.
|
|
|
|
|
|
|
|
* Support for threads.
|
|
|
|
|
|
|
|
* OpenSSL 1.1.1 or newer for the :mod:`ssl` and :mod:`hashlib` modules.
|
|
|
|
|
|
|
|
* On Windows, Microsoft Visual Studio 2017 or later is required.
|
|
|
|
|
|
|
|
.. versionchanged:: 3.11
|
|
|
|
C11 compiler, IEEE 754 and NaN support are now required.
|
2022-12-06 09:35:32 -04:00
|
|
|
On Windows, Visual Studio 2017 or later is required.
|
2022-12-05 10:41:44 -04:00
|
|
|
|
|
|
|
.. versionchanged:: 3.10
|
|
|
|
OpenSSL 1.1.1 is now required.
|
|
|
|
|
|
|
|
.. versionchanged:: 3.7
|
|
|
|
Thread support and OpenSSL 1.0.2 are now required.
|
|
|
|
|
|
|
|
.. versionchanged:: 3.6
|
|
|
|
Selected C99 features are now required, like ``<stdint.h>`` and ``static
|
|
|
|
inline`` functions.
|
|
|
|
|
|
|
|
.. versionchanged:: 3.5
|
|
|
|
On Windows, Visual Studio 2015 or later is required.
|
|
|
|
|
|
|
|
See also :pep:`7` "Style Guide for C Code" and :pep:`11` "CPython platform
|
|
|
|
support".
|
|
|
|
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. _configure-options:
|
|
|
|
|
|
|
|
Configure Options
|
|
|
|
=================
|
|
|
|
|
|
|
|
List all ``./configure`` script options using::
|
|
|
|
|
|
|
|
./configure --help
|
|
|
|
|
|
|
|
See also the :file:`Misc/SpecialBuilds.txt` in the Python source distribution.
|
|
|
|
|
|
|
|
General Options
|
|
|
|
---------------
|
|
|
|
|
|
|
|
.. cmdoption:: --enable-loadable-sqlite-extensions
|
|
|
|
|
|
|
|
Support loadable extensions in the :mod:`_sqlite` extension module (default
|
2021-04-12 18:27:35 -03:00
|
|
|
is no).
|
|
|
|
|
|
|
|
See the :meth:`sqlite3.Connection.enable_load_extension` method of the
|
|
|
|
:mod:`sqlite3` module.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
|
|
|
.. cmdoption:: --disable-ipv6
|
|
|
|
|
|
|
|
Disable IPv6 support (enabled by default if supported), see the
|
|
|
|
:mod:`socket` module.
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
.. cmdoption:: --enable-big-digits=[15|30]
|
2021-04-08 17:32:21 -03:00
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Define the size in bits of Python :class:`int` digits: 15 or 30 bits.
|
|
|
|
|
2022-01-14 14:54:56 -04:00
|
|
|
By default, the digit size is 30.
|
2021-04-08 20:58:20 -03:00
|
|
|
|
|
|
|
Define the ``PYLONG_BITS_IN_DIGIT`` to ``15`` or ``30``.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
See :data:`sys.int_info.bits_per_digit <sys.int_info>`.
|
|
|
|
|
|
|
|
.. cmdoption:: --with-suffix=SUFFIX
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Set the Python executable suffix to *SUFFIX*.
|
|
|
|
|
|
|
|
The default suffix is ``.exe`` on Windows and macOS (``python.exe``
|
2022-04-04 14:31:31 -03:00
|
|
|
executable), ``.js`` on Emscripten node, ``.html`` on Emscripten browser,
|
|
|
|
``.wasm`` on WASI, and an empty string on other platforms (``python``
|
|
|
|
executable).
|
2021-11-29 11:01:55 -04:00
|
|
|
|
|
|
|
.. versionchanged:: 3.11
|
2022-04-04 14:31:31 -03:00
|
|
|
The default suffix on WASM platform is one of ``.js``, ``.html``
|
|
|
|
or ``.wasm``.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. cmdoption:: --with-tzpath=<list of absolute paths separated by pathsep>
|
|
|
|
|
2023-07-21 06:40:37 -03:00
|
|
|
Select the default time zone search path for :const:`zoneinfo.TZPATH`.
|
2021-04-12 18:27:35 -03:00
|
|
|
See the :ref:`Compile-time configuration
|
|
|
|
<zoneinfo_data_compile_time_config>` of the :mod:`zoneinfo` module.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Default: ``/usr/share/zoneinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo``.
|
|
|
|
|
|
|
|
See :data:`os.pathsep` path separator.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. versionadded:: 3.9
|
|
|
|
|
|
|
|
.. cmdoption:: --without-decimal-contextvar
|
|
|
|
|
|
|
|
Build the ``_decimal`` extension module using a thread-local context rather
|
|
|
|
than a coroutine-local context (default), see the :mod:`decimal` module.
|
|
|
|
|
2023-07-21 06:40:37 -03:00
|
|
|
See :const:`decimal.HAVE_CONTEXTVAR` and the :mod:`contextvars` module.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. versionadded:: 3.9
|
|
|
|
|
2021-12-07 18:47:24 -04:00
|
|
|
.. cmdoption:: --with-dbmliborder=<list of backend names>
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
Override order to check db backends for the :mod:`dbm` module
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
A valid value is a colon (``:``) separated string with the backend names:
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
* ``ndbm``;
|
|
|
|
* ``gdbm``;
|
|
|
|
* ``bdb``.
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
.. cmdoption:: --without-c-locale-coercion
|
|
|
|
|
|
|
|
Disable C locale coercion to a UTF-8 based locale (enabled by default).
|
2021-04-08 17:32:21 -03:00
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Don't define the ``PY_COERCE_C_LOCALE`` macro.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
See :envvar:`PYTHONCOERCECLOCALE` and the :pep:`538`.
|
|
|
|
|
2022-12-05 10:41:44 -04:00
|
|
|
.. cmdoption:: --without-freelists
|
|
|
|
|
|
|
|
Disable all freelists except the empty tuple singleton.
|
|
|
|
|
|
|
|
.. versionadded:: 3.11
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. cmdoption:: --with-platlibdir=DIRNAME
|
|
|
|
|
|
|
|
Python library directory name (default is ``lib``).
|
|
|
|
|
|
|
|
Fedora and SuSE use ``lib64`` on 64-bit platforms.
|
|
|
|
|
|
|
|
See :data:`sys.platlibdir`.
|
|
|
|
|
|
|
|
.. versionadded:: 3.9
|
|
|
|
|
|
|
|
.. cmdoption:: --with-wheel-pkg-dir=PATH
|
|
|
|
|
|
|
|
Directory of wheel packages used by the :mod:`ensurepip` module
|
|
|
|
(none by default).
|
|
|
|
|
|
|
|
Some Linux distribution packaging policies recommend against bundling
|
|
|
|
dependencies. For example, Fedora installs wheel packages in the
|
|
|
|
``/usr/share/python-wheels/`` directory and don't install the
|
|
|
|
:mod:`ensurepip._bundled` package.
|
|
|
|
|
|
|
|
.. versionadded:: 3.10
|
|
|
|
|
2021-11-10 18:02:19 -04:00
|
|
|
.. cmdoption:: --with-pkg-config=[check|yes|no]
|
|
|
|
|
|
|
|
Whether configure should use :program:`pkg-config` to detect build
|
|
|
|
dependencies.
|
|
|
|
|
|
|
|
* ``check`` (default): :program:`pkg-config` is optional
|
|
|
|
* ``yes``: :program:`pkg-config` is mandatory
|
|
|
|
* ``no``: configure does not use :program:`pkg-config` even when present
|
|
|
|
|
|
|
|
.. versionadded:: 3.11
|
|
|
|
|
2021-12-16 09:40:54 -04:00
|
|
|
.. cmdoption:: --enable-pystats
|
|
|
|
|
|
|
|
Turn on internal statistics gathering.
|
|
|
|
|
|
|
|
The statistics will be dumped to a arbitrary (probably unique) file in
|
2022-11-04 07:15:54 -03:00
|
|
|
``/tmp/py_stats/``, or ``C:\temp\py_stats\`` on Windows. If that directory
|
|
|
|
does not exist, results will be printed on stdout.
|
2021-12-16 09:40:54 -04:00
|
|
|
|
2022-04-28 09:02:45 -03:00
|
|
|
Use ``Tools/scripts/summarize_stats.py`` to read the stats.
|
2021-12-16 09:40:54 -04:00
|
|
|
|
|
|
|
.. versionadded:: 3.11
|
|
|
|
|
2022-05-01 22:24:41 -03:00
|
|
|
WebAssembly Options
|
|
|
|
-------------------
|
2022-04-04 14:31:31 -03:00
|
|
|
|
|
|
|
.. cmdoption:: --with-emscripten-target=[browser|node]
|
|
|
|
|
|
|
|
Set build flavor for ``wasm32-emscripten``.
|
|
|
|
|
|
|
|
* ``browser`` (default): preload minimal stdlib, default MEMFS.
|
|
|
|
* ``node``: NODERAWFS and pthread support.
|
|
|
|
|
|
|
|
.. versionadded:: 3.11
|
|
|
|
|
|
|
|
.. cmdoption:: --enable-wasm-dynamic-linking
|
|
|
|
|
|
|
|
Turn on dynamic linking support for WASM.
|
|
|
|
|
|
|
|
Dynamic linking enables ``dlopen``. File size of the executable
|
|
|
|
increases due to limited dead code elimination and additional features.
|
|
|
|
|
|
|
|
.. versionadded:: 3.11
|
2022-04-23 09:59:33 -03:00
|
|
|
|
|
|
|
.. cmdoption:: --enable-wasm-pthreads
|
|
|
|
|
|
|
|
Turn on pthreads support for WASM.
|
|
|
|
|
|
|
|
.. versionadded:: 3.11
|
2022-04-04 14:31:31 -03:00
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
Install Options
|
|
|
|
---------------
|
|
|
|
|
2023-03-15 12:10:03 -03:00
|
|
|
.. cmdoption:: --prefix=PREFIX
|
|
|
|
|
|
|
|
Install architecture-independent files in PREFIX. On Unix, it
|
|
|
|
defaults to :file:`/usr/local`.
|
|
|
|
|
|
|
|
This value can be retrived at runtime using :data:`sys.prefix`.
|
|
|
|
|
|
|
|
As an example, one can use ``--prefix="$HOME/.local/"`` to install
|
|
|
|
a Python in its home directory.
|
|
|
|
|
|
|
|
.. cmdoption:: --exec-prefix=EPREFIX
|
|
|
|
|
|
|
|
Install architecture-dependent files in EPREFIX, defaults to :option:`--prefix`.
|
|
|
|
|
|
|
|
This value can be retrived at runtime using :data:`sys.exec_prefix`.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. cmdoption:: --disable-test-modules
|
|
|
|
|
|
|
|
Don't build nor install test modules, like the :mod:`test` package or the
|
|
|
|
:mod:`_testcapi` extension module (built and installed by default).
|
|
|
|
|
|
|
|
.. versionadded:: 3.10
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
.. cmdoption:: --with-ensurepip=[upgrade|install|no]
|
2021-04-08 17:32:21 -03:00
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Select the :mod:`ensurepip` command run on Python installation:
|
|
|
|
|
|
|
|
* ``upgrade`` (default): run ``python -m ensurepip --altinstall --upgrade``
|
|
|
|
command.
|
|
|
|
* ``install``: run ``python -m ensurepip --altinstall`` command;
|
|
|
|
* ``no``: don't run ensurepip;
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
|
|
|
|
|
|
|
Performance options
|
|
|
|
-------------------
|
|
|
|
|
|
|
|
Configuring Python using ``--enable-optimizations --with-lto`` (PGO + LTO) is
|
gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908)
* Add support for the BOLT post-link binary optimizer
Using [bolt](https://github.com/llvm/llvm-project/tree/main/bolt)
provides a fairly large speedup without any code or functionality
changes. It provides roughly a 1% speedup on pyperformance, and a
4% improvement on the Pyston web macrobenchmarks.
It is gated behind an `--enable-bolt` configure arg because not all
toolchains and environments are supported. It has been tested on a
Linux x86_64 toolchain, using llvm-bolt built from the LLVM 14.0.6
sources (their binary distribution of this version did not include bolt).
Compared to [a previous attempt](https://github.com/faster-cpython/ideas/issues/224),
this commit uses bolt's preferred "instrumentation" approach, as well as adds some non-PIE
flags which enable much better optimizations from bolt.
The effects of this change are a bit more dependent on CPU microarchitecture
than other changes, since it optimizes i-cache behavior which seems
to be a bit more variable between architectures. The 1%/4% numbers
were collected on an Intel Skylake CPU, and on an AMD Zen 3 CPU I
got a slightly larger speedup (2%/4%), and on a c6i.xlarge EC2 instance
I got a slightly lower speedup (1%/3%).
The low speedup on pyperformance is not entirely unexpected, because
BOLT improves i-cache behavior, and the benchmarks in the pyperformance
suite are small and tend to fit in i-cache.
This change uses the existing pgo profiling task (`python -m test --pgo`),
though I was able to measure about a 1% macrobenchmark improvement by
using the macrobenchmarks as the training task. I personally think that
both the PGO and BOLT tasks should be updated to use macrobenchmarks,
but for the sake of splitting up the work this PR uses the existing pgo task.
* Simplify the build flags
* Add a NEWS entry
* Update Makefile.pre.in
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Update configure.ac
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Add myself to ACKS
* Add docs
* Other review comments
* fix tab/space issue
* Make it more clear that --enable-bolt is experimental
* Add link to bolt's github page
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2022-08-18 18:33:54 -03:00
|
|
|
recommended for best performance. The experimental ``--enable-bolt`` flag can
|
|
|
|
also be used to improve performance.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. cmdoption:: --enable-optimizations
|
|
|
|
|
|
|
|
Enable Profile Guided Optimization (PGO) using :envvar:`PROFILE_TASK`
|
|
|
|
(disabled by default).
|
|
|
|
|
2021-04-09 09:35:10 -03:00
|
|
|
The C compiler Clang requires ``llvm-profdata`` program for PGO. On
|
|
|
|
macOS, GCC also requires it: GCC is just an alias to Clang on macOS.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
Disable also semantic interposition in libpython if ``--enable-shared`` and
|
|
|
|
GCC is used: add ``-fno-semantic-interposition`` to the compiler and linker
|
|
|
|
flags.
|
|
|
|
|
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
|
|
|
.. versionchanged:: 3.10
|
|
|
|
Use ``-fno-semantic-interposition`` on GCC.
|
|
|
|
|
|
|
|
.. envvar:: PROFILE_TASK
|
|
|
|
|
|
|
|
Environment variable used in the Makefile: Python command line arguments for
|
|
|
|
the PGO generation task.
|
|
|
|
|
|
|
|
Default: ``-m test --pgo --timeout=$(TESTTIMEOUT)``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.8
|
|
|
|
|
2021-07-19 07:52:56 -03:00
|
|
|
.. cmdoption:: --with-lto=[full|thin|no|yes]
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
Enable Link Time Optimization (LTO) in any build (disabled by default).
|
|
|
|
|
2021-05-25 07:25:22 -03:00
|
|
|
The C compiler Clang requires ``llvm-ar`` for LTO (``ar`` on macOS), as well
|
|
|
|
as an LTO-aware linker (``ld.gold`` or ``lld``).
|
2021-04-09 09:35:10 -03:00
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
2021-07-19 07:52:56 -03:00
|
|
|
.. versionadded:: 3.11
|
|
|
|
To use ThinLTO feature, use ``--with-lto=thin`` on Clang.
|
|
|
|
|
2022-09-16 07:40:05 -03:00
|
|
|
.. versionchanged:: 3.12
|
|
|
|
Use ThinLTO as the default optimization policy on Clang if the compiler accepts the flag.
|
|
|
|
|
gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908)
* Add support for the BOLT post-link binary optimizer
Using [bolt](https://github.com/llvm/llvm-project/tree/main/bolt)
provides a fairly large speedup without any code or functionality
changes. It provides roughly a 1% speedup on pyperformance, and a
4% improvement on the Pyston web macrobenchmarks.
It is gated behind an `--enable-bolt` configure arg because not all
toolchains and environments are supported. It has been tested on a
Linux x86_64 toolchain, using llvm-bolt built from the LLVM 14.0.6
sources (their binary distribution of this version did not include bolt).
Compared to [a previous attempt](https://github.com/faster-cpython/ideas/issues/224),
this commit uses bolt's preferred "instrumentation" approach, as well as adds some non-PIE
flags which enable much better optimizations from bolt.
The effects of this change are a bit more dependent on CPU microarchitecture
than other changes, since it optimizes i-cache behavior which seems
to be a bit more variable between architectures. The 1%/4% numbers
were collected on an Intel Skylake CPU, and on an AMD Zen 3 CPU I
got a slightly larger speedup (2%/4%), and on a c6i.xlarge EC2 instance
I got a slightly lower speedup (1%/3%).
The low speedup on pyperformance is not entirely unexpected, because
BOLT improves i-cache behavior, and the benchmarks in the pyperformance
suite are small and tend to fit in i-cache.
This change uses the existing pgo profiling task (`python -m test --pgo`),
though I was able to measure about a 1% macrobenchmark improvement by
using the macrobenchmarks as the training task. I personally think that
both the PGO and BOLT tasks should be updated to use macrobenchmarks,
but for the sake of splitting up the work this PR uses the existing pgo task.
* Simplify the build flags
* Add a NEWS entry
* Update Makefile.pre.in
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Update configure.ac
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Add myself to ACKS
* Add docs
* Other review comments
* fix tab/space issue
* Make it more clear that --enable-bolt is experimental
* Add link to bolt's github page
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2022-08-18 18:33:54 -03:00
|
|
|
.. cmdoption:: --enable-bolt
|
|
|
|
|
|
|
|
Enable usage of the `BOLT post-link binary optimizer
|
2022-08-20 12:11:35 -03:00
|
|
|
<https://github.com/llvm/llvm-project/tree/main/bolt>`_ (disabled by
|
gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908)
* Add support for the BOLT post-link binary optimizer
Using [bolt](https://github.com/llvm/llvm-project/tree/main/bolt)
provides a fairly large speedup without any code or functionality
changes. It provides roughly a 1% speedup on pyperformance, and a
4% improvement on the Pyston web macrobenchmarks.
It is gated behind an `--enable-bolt` configure arg because not all
toolchains and environments are supported. It has been tested on a
Linux x86_64 toolchain, using llvm-bolt built from the LLVM 14.0.6
sources (their binary distribution of this version did not include bolt).
Compared to [a previous attempt](https://github.com/faster-cpython/ideas/issues/224),
this commit uses bolt's preferred "instrumentation" approach, as well as adds some non-PIE
flags which enable much better optimizations from bolt.
The effects of this change are a bit more dependent on CPU microarchitecture
than other changes, since it optimizes i-cache behavior which seems
to be a bit more variable between architectures. The 1%/4% numbers
were collected on an Intel Skylake CPU, and on an AMD Zen 3 CPU I
got a slightly larger speedup (2%/4%), and on a c6i.xlarge EC2 instance
I got a slightly lower speedup (1%/3%).
The low speedup on pyperformance is not entirely unexpected, because
BOLT improves i-cache behavior, and the benchmarks in the pyperformance
suite are small and tend to fit in i-cache.
This change uses the existing pgo profiling task (`python -m test --pgo`),
though I was able to measure about a 1% macrobenchmark improvement by
using the macrobenchmarks as the training task. I personally think that
both the PGO and BOLT tasks should be updated to use macrobenchmarks,
but for the sake of splitting up the work this PR uses the existing pgo task.
* Simplify the build flags
* Add a NEWS entry
* Update Makefile.pre.in
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Update configure.ac
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Add myself to ACKS
* Add docs
* Other review comments
* fix tab/space issue
* Make it more clear that --enable-bolt is experimental
* Add link to bolt's github page
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2022-08-18 18:33:54 -03:00
|
|
|
default).
|
|
|
|
|
|
|
|
BOLT is part of the LLVM project but is not always included in their binary
|
|
|
|
distributions. This flag requires that ``llvm-bolt`` and ``merge-fdata``
|
|
|
|
are available.
|
|
|
|
|
|
|
|
BOLT is still a fairly new project so this flag should be considered
|
|
|
|
experimental for now. Because this tool operates on machine code its success
|
|
|
|
is dependent on a combination of the build environment + the other
|
|
|
|
optimization configure args + the CPU architecture, and not all combinations
|
|
|
|
are supported.
|
2023-05-22 08:45:20 -03:00
|
|
|
BOLT versions before LLVM 16 are known to crash BOLT under some scenarios.
|
|
|
|
Use of LLVM 16 or newer for BOLT optimization is strongly encouraged.
|
|
|
|
|
|
|
|
The :envvar:`!BOLT_INSTRUMENT_FLAGS` and :envvar:`!BOLT_APPLY_FLAGS`
|
|
|
|
:program:`configure` variables can be defined to override the default set of
|
|
|
|
arguments for :program:`llvm-bolt` to instrument and apply BOLT data to
|
|
|
|
binaries, respectively.
|
gh-90536: Add support for the BOLT post-link binary optimizer (gh-95908)
* Add support for the BOLT post-link binary optimizer
Using [bolt](https://github.com/llvm/llvm-project/tree/main/bolt)
provides a fairly large speedup without any code or functionality
changes. It provides roughly a 1% speedup on pyperformance, and a
4% improvement on the Pyston web macrobenchmarks.
It is gated behind an `--enable-bolt` configure arg because not all
toolchains and environments are supported. It has been tested on a
Linux x86_64 toolchain, using llvm-bolt built from the LLVM 14.0.6
sources (their binary distribution of this version did not include bolt).
Compared to [a previous attempt](https://github.com/faster-cpython/ideas/issues/224),
this commit uses bolt's preferred "instrumentation" approach, as well as adds some non-PIE
flags which enable much better optimizations from bolt.
The effects of this change are a bit more dependent on CPU microarchitecture
than other changes, since it optimizes i-cache behavior which seems
to be a bit more variable between architectures. The 1%/4% numbers
were collected on an Intel Skylake CPU, and on an AMD Zen 3 CPU I
got a slightly larger speedup (2%/4%), and on a c6i.xlarge EC2 instance
I got a slightly lower speedup (1%/3%).
The low speedup on pyperformance is not entirely unexpected, because
BOLT improves i-cache behavior, and the benchmarks in the pyperformance
suite are small and tend to fit in i-cache.
This change uses the existing pgo profiling task (`python -m test --pgo`),
though I was able to measure about a 1% macrobenchmark improvement by
using the macrobenchmarks as the training task. I personally think that
both the PGO and BOLT tasks should be updated to use macrobenchmarks,
but for the sake of splitting up the work this PR uses the existing pgo task.
* Simplify the build flags
* Add a NEWS entry
* Update Makefile.pre.in
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Update configure.ac
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Add myself to ACKS
* Add docs
* Other review comments
* fix tab/space issue
* Make it more clear that --enable-bolt is experimental
* Add link to bolt's github page
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2022-08-18 18:33:54 -03:00
|
|
|
|
|
|
|
.. versionadded:: 3.12
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. cmdoption:: --with-computed-gotos
|
|
|
|
|
|
|
|
Enable computed gotos in evaluation loop (enabled by default on supported
|
|
|
|
compilers).
|
|
|
|
|
|
|
|
.. cmdoption:: --without-pymalloc
|
|
|
|
|
|
|
|
Disable the specialized Python memory allocator :ref:`pymalloc <pymalloc>`
|
|
|
|
(enabled by default).
|
|
|
|
|
|
|
|
See also :envvar:`PYTHONMALLOC` environment variable.
|
|
|
|
|
|
|
|
.. cmdoption:: --without-doc-strings
|
|
|
|
|
|
|
|
Disable static documentation strings to reduce the memory footprint (enabled
|
|
|
|
by default). Documentation strings defined in Python are not affected.
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Don't define the ``WITH_DOC_STRINGS`` macro.
|
|
|
|
|
|
|
|
See the ``PyDoc_STRVAR()`` macro.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. cmdoption:: --enable-profiling
|
|
|
|
|
|
|
|
Enable C-level code profiling with ``gprof`` (disabled by default).
|
|
|
|
|
2023-03-04 17:39:52 -04:00
|
|
|
.. cmdoption:: --with-strict-overflow
|
|
|
|
|
|
|
|
Add ``-fstrict-overflow`` to the C compiler flags (by default we add
|
|
|
|
``-fno-strict-overflow`` instead).
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. _debug-build:
|
|
|
|
|
2021-04-29 08:06:59 -03:00
|
|
|
Python Debug Build
|
|
|
|
------------------
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
A debug build is Python built with the :option:`--with-pydebug` configure
|
|
|
|
option.
|
|
|
|
|
|
|
|
Effects of a debug build:
|
|
|
|
|
2021-04-29 08:06:59 -03:00
|
|
|
* Display all warnings by default: the list of default warning filters is empty
|
|
|
|
in the :mod:`warnings` module.
|
2021-04-08 17:32:21 -03:00
|
|
|
* Add ``d`` to :data:`sys.abiflags`.
|
|
|
|
* Add :func:`sys.gettotalrefcount` function.
|
|
|
|
* Add :option:`-X showrefcount <-X>` command line option.
|
2022-05-24 18:23:01 -03:00
|
|
|
* Add :option:`-d` command line option and :envvar:`PYTHONDEBUG` environment
|
|
|
|
variable to debug the parser.
|
2022-04-16 19:57:00 -03:00
|
|
|
* Add support for the ``__lltrace__`` variable: enable low-level tracing in the
|
2021-04-08 20:58:20 -03:00
|
|
|
bytecode evaluation loop if the variable is defined.
|
2021-04-29 08:06:59 -03:00
|
|
|
* Install :ref:`debug hooks on memory allocators <default-memory-allocators>`
|
|
|
|
to detect buffer overflow and other memory errors.
|
|
|
|
* Define ``Py_DEBUG`` and ``Py_REF_DEBUG`` macros.
|
2021-05-25 18:01:30 -03:00
|
|
|
* Add runtime checks: code surrounded by ``#ifdef Py_DEBUG`` and ``#endif``.
|
2021-04-29 08:06:59 -03:00
|
|
|
Enable ``assert(...)`` and ``_PyObject_ASSERT(...)`` assertions: don't set
|
|
|
|
the ``NDEBUG`` macro (see also the :option:`--with-assertions` configure
|
|
|
|
option). Main runtime checks:
|
|
|
|
|
|
|
|
* Add sanity checks on the function arguments.
|
|
|
|
* Unicode and int objects are created with their memory filled with a pattern
|
|
|
|
to detect usage of uninitialized objects.
|
|
|
|
* Ensure that functions which can clear or replace the current exception are
|
|
|
|
not called with an exception raised.
|
2022-04-21 18:04:01 -03:00
|
|
|
* Check that deallocator functions don't change the current exception.
|
2021-04-29 08:06:59 -03:00
|
|
|
* The garbage collector (:func:`gc.collect` function) runs some basic checks
|
|
|
|
on objects consistency.
|
|
|
|
* The :c:macro:`Py_SAFE_DOWNCAST()` macro checks for integer underflow and
|
|
|
|
overflow when downcasting from wide types to narrow types.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
See also the :ref:`Python Development Mode <devmode>` and the
|
|
|
|
:option:`--with-trace-refs` configure option.
|
|
|
|
|
|
|
|
.. versionchanged:: 3.8
|
|
|
|
Release builds and debug builds are now ABI compatible: defining the
|
2021-04-29 08:06:59 -03:00
|
|
|
``Py_DEBUG`` macro no longer implies the ``Py_TRACE_REFS`` macro (see the
|
|
|
|
:option:`--with-trace-refs` option), which introduces the only ABI
|
|
|
|
incompatibility.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
|
|
|
|
Debug options
|
|
|
|
-------------
|
|
|
|
|
|
|
|
.. cmdoption:: --with-pydebug
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
:ref:`Build Python in debug mode <debug-build>`: define the ``Py_DEBUG``
|
|
|
|
macro (disabled by default).
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. cmdoption:: --with-trace-refs
|
|
|
|
|
|
|
|
Enable tracing references for debugging purpose (disabled by default).
|
|
|
|
|
|
|
|
Effects:
|
|
|
|
|
|
|
|
* Define the ``Py_TRACE_REFS`` macro.
|
|
|
|
* Add :func:`sys.getobjects` function.
|
|
|
|
* Add :envvar:`PYTHONDUMPREFS` environment variable.
|
|
|
|
|
|
|
|
This build is not ABI compatible with release build (default build) or debug
|
2021-04-08 20:58:20 -03:00
|
|
|
build (``Py_DEBUG`` and ``Py_REF_DEBUG`` macros).
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. versionadded:: 3.8
|
|
|
|
|
|
|
|
.. cmdoption:: --with-assertions
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Build with C assertions enabled (default is no): ``assert(...);`` and
|
|
|
|
``_PyObject_ASSERT(...);``.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
If set, the ``NDEBUG`` macro is not defined in the :envvar:`OPT` compiler
|
|
|
|
variable.
|
|
|
|
|
|
|
|
See also the :option:`--with-pydebug` option (:ref:`debug build
|
|
|
|
<debug-build>`) which also enables assertions.
|
|
|
|
|
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
|
|
|
.. cmdoption:: --with-valgrind
|
|
|
|
|
|
|
|
Enable Valgrind support (default is no).
|
|
|
|
|
|
|
|
.. cmdoption:: --with-dtrace
|
|
|
|
|
|
|
|
Enable DTrace support (default is no).
|
|
|
|
|
2021-04-12 18:27:35 -03:00
|
|
|
See :ref:`Instrumenting CPython with DTrace and SystemTap
|
|
|
|
<instrumentation>`.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
|
|
|
.. cmdoption:: --with-address-sanitizer
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Enable AddressSanitizer memory error detector, ``asan`` (default is no).
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
|
|
|
.. cmdoption:: --with-memory-sanitizer
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Enable MemorySanitizer allocation error detector, ``msan`` (default is no).
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
|
|
|
.. cmdoption:: --with-undefined-behavior-sanitizer
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Enable UndefinedBehaviorSanitizer undefined behaviour detector, ``ubsan``
|
2021-04-08 17:32:21 -03:00
|
|
|
(default is no).
|
|
|
|
|
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
|
|
|
|
|
|
|
Linker options
|
|
|
|
--------------
|
|
|
|
|
|
|
|
.. cmdoption:: --enable-shared
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Enable building a shared Python library: ``libpython`` (default is no).
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. cmdoption:: --without-static-libpython
|
|
|
|
|
|
|
|
Do not build ``libpythonMAJOR.MINOR.a`` and do not install ``python.o``
|
|
|
|
(built and enabled by default).
|
|
|
|
|
|
|
|
.. versionadded:: 3.10
|
|
|
|
|
|
|
|
|
|
|
|
Libraries options
|
|
|
|
-----------------
|
|
|
|
|
|
|
|
.. cmdoption:: --with-libs='lib1 ...'
|
|
|
|
|
|
|
|
Link against additional libraries (default is no).
|
|
|
|
|
|
|
|
.. cmdoption:: --with-system-expat
|
|
|
|
|
|
|
|
Build the :mod:`pyexpat` module using an installed ``expat`` library
|
|
|
|
(default is no).
|
|
|
|
|
|
|
|
.. cmdoption:: --with-system-libmpdec
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Build the ``_decimal`` extension module using an installed ``mpdec``
|
2021-04-08 17:32:21 -03:00
|
|
|
library, see the :mod:`decimal` module (default is no).
|
|
|
|
|
|
|
|
.. versionadded:: 3.3
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
.. cmdoption:: --with-readline=editline
|
2021-04-08 17:32:21 -03:00
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Use ``editline`` library for backend of the :mod:`readline` module.
|
|
|
|
|
|
|
|
Define the ``WITH_EDITLINE`` macro.
|
|
|
|
|
|
|
|
.. versionadded:: 3.10
|
|
|
|
|
|
|
|
.. cmdoption:: --without-readline
|
|
|
|
|
|
|
|
Don't build the :mod:`readline` module (built by default).
|
|
|
|
|
|
|
|
Don't define the ``HAVE_LIBREADLINE`` macro.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. versionadded:: 3.10
|
|
|
|
|
|
|
|
.. cmdoption:: --with-libm=STRING
|
|
|
|
|
|
|
|
Override ``libm`` math library to *STRING* (default is system-dependent).
|
|
|
|
|
|
|
|
.. cmdoption:: --with-libc=STRING
|
|
|
|
|
|
|
|
Override ``libc`` C library to *STRING* (default is system-dependent).
|
|
|
|
|
|
|
|
.. cmdoption:: --with-openssl=DIR
|
|
|
|
|
|
|
|
Root of the OpenSSL directory.
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
.. versionadded:: 3.7
|
|
|
|
|
|
|
|
.. cmdoption:: --with-openssl-rpath=[no|auto|DIR]
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
Set runtime library directory (rpath) for OpenSSL libraries:
|
|
|
|
|
|
|
|
* ``no`` (default): don't set rpath;
|
|
|
|
* ``auto``: auto-detect rpath from :option:`--with-openssl` and
|
|
|
|
``pkg-config``;
|
|
|
|
* *DIR*: set an explicit rpath.
|
|
|
|
|
|
|
|
.. versionadded:: 3.10
|
|
|
|
|
|
|
|
|
|
|
|
Security Options
|
|
|
|
----------------
|
|
|
|
|
2021-10-10 05:29:46 -03:00
|
|
|
.. cmdoption:: --with-hash-algorithm=[fnv|siphash13|siphash24]
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
Select hash algorithm for use in ``Python/pyhash.c``:
|
|
|
|
|
2021-10-10 05:29:46 -03:00
|
|
|
* ``siphash13`` (default);
|
|
|
|
* ``siphash24``;
|
|
|
|
* ``fnv``.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. versionadded:: 3.4
|
|
|
|
|
2021-10-10 05:29:46 -03:00
|
|
|
.. versionadded:: 3.11
|
|
|
|
``siphash13`` is added and it is the new default.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. cmdoption:: --with-builtin-hashlib-hashes=md5,sha1,sha256,sha512,sha3,blake2
|
|
|
|
|
|
|
|
Built-in hash modules:
|
|
|
|
|
|
|
|
* ``md5``;
|
|
|
|
* ``sha1``;
|
|
|
|
* ``sha256``;
|
|
|
|
* ``sha512``;
|
|
|
|
* ``sha3`` (with shake);
|
|
|
|
* ``blake2``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.9
|
|
|
|
|
|
|
|
.. cmdoption:: --with-ssl-default-suites=[python|openssl|STRING]
|
|
|
|
|
|
|
|
Override the OpenSSL default cipher suites string:
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
* ``python`` (default): use Python's preferred selection;
|
2021-04-08 17:32:21 -03:00
|
|
|
* ``openssl``: leave OpenSSL's defaults untouched;
|
2021-05-01 15:53:10 -03:00
|
|
|
* *STRING*: use a custom string
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
See the :mod:`ssl` module.
|
|
|
|
|
|
|
|
.. versionadded:: 3.7
|
|
|
|
|
2021-05-01 15:53:10 -03:00
|
|
|
.. versionchanged:: 3.10
|
|
|
|
|
|
|
|
The settings ``python`` and *STRING* also set TLS 1.2 as minimum
|
|
|
|
protocol version.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
macOS Options
|
|
|
|
-------------
|
|
|
|
|
|
|
|
See ``Mac/README.rst``.
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
.. cmdoption:: --enable-universalsdk
|
|
|
|
.. cmdoption:: --enable-universalsdk=SDKDIR
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
Create a universal binary build. *SDKDIR* specifies which macOS SDK should
|
|
|
|
be used to perform the build (default is no).
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
.. cmdoption:: --enable-framework
|
|
|
|
.. cmdoption:: --enable-framework=INSTALLDIR
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
Create a Python.framework rather than a traditional Unix install. Optional
|
|
|
|
*INSTALLDIR* specifies the installation path (default is no).
|
|
|
|
|
|
|
|
.. cmdoption:: --with-universal-archs=ARCH
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Specify the kind of universal binary that should be created. This option is
|
2021-04-08 17:32:21 -03:00
|
|
|
only valid when :option:`--enable-universalsdk` is set.
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Options:
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
* ``universal2``;
|
|
|
|
* ``32-bit``;
|
|
|
|
* ``64-bit``;
|
|
|
|
* ``3-way``;
|
|
|
|
* ``intel``;
|
|
|
|
* ``intel-32``;
|
|
|
|
* ``intel-64``;
|
|
|
|
* ``all``.
|
|
|
|
|
|
|
|
.. cmdoption:: --with-framework-name=FRAMEWORK
|
|
|
|
|
|
|
|
Specify the name for the python framework on macOS only valid when
|
|
|
|
:option:`--enable-framework` is set (default: ``Python``).
|
|
|
|
|
|
|
|
|
2021-11-29 12:23:29 -04:00
|
|
|
Cross Compiling Options
|
|
|
|
-----------------------
|
|
|
|
|
|
|
|
Cross compiling, also known as cross building, can be used to build Python
|
|
|
|
for another CPU architecture or platform. Cross compiling requires a Python
|
2021-12-13 15:48:46 -04:00
|
|
|
interpreter for the build platform. The version of the build Python must match
|
|
|
|
the version of the cross compiled host Python.
|
2021-11-29 12:23:29 -04:00
|
|
|
|
|
|
|
.. cmdoption:: --build=BUILD
|
|
|
|
|
|
|
|
configure for building on BUILD, usually guessed by :program:`config.guess`.
|
|
|
|
|
|
|
|
.. cmdoption:: --host=HOST
|
|
|
|
|
|
|
|
cross-compile to build programs to run on HOST (target platform)
|
|
|
|
|
2021-12-13 15:48:46 -04:00
|
|
|
.. cmdoption:: --with-build-python=path/to/python
|
2021-11-29 12:23:29 -04:00
|
|
|
|
|
|
|
path to build ``python`` binary for cross compiling
|
|
|
|
|
|
|
|
.. versionadded:: 3.11
|
|
|
|
|
|
|
|
.. cmdoption:: CONFIG_SITE=file
|
|
|
|
|
|
|
|
An environment variable that points to a file with configure overrides.
|
|
|
|
|
|
|
|
Example *config.site* file::
|
|
|
|
|
|
|
|
# config.site-aarch64
|
|
|
|
ac_cv_buggy_getaddrinfo=no
|
|
|
|
ac_cv_file__dev_ptmx=yes
|
|
|
|
ac_cv_file__dev_ptc=no
|
|
|
|
|
|
|
|
|
|
|
|
Cross compiling example::
|
|
|
|
|
|
|
|
CONFIG_SITE=config.site-aarch64 ../configure \
|
|
|
|
--build=x86_64-pc-linux-gnu \
|
|
|
|
--host=aarch64-unknown-linux-gnu \
|
|
|
|
--with-build-python=../x86_64/python
|
|
|
|
|
|
|
|
|
2021-04-09 09:35:10 -03:00
|
|
|
Python Build System
|
|
|
|
===================
|
|
|
|
|
|
|
|
Main files of the build system
|
|
|
|
------------------------------
|
|
|
|
|
|
|
|
* :file:`configure.ac` => :file:`configure`;
|
|
|
|
* :file:`Makefile.pre.in` => :file:`Makefile` (created by :file:`configure`);
|
|
|
|
* :file:`pyconfig.h` (created by :file:`configure`);
|
|
|
|
* :file:`Modules/Setup`: C extensions built by the Makefile using
|
|
|
|
:file:`Module/makesetup` shell script;
|
2022-07-25 10:50:46 -03:00
|
|
|
* :file:`setup.py`: C extensions built using the ``setuptools`` package.
|
2021-04-09 09:35:10 -03:00
|
|
|
|
|
|
|
Main build steps
|
|
|
|
----------------
|
|
|
|
|
|
|
|
* C files (``.c``) are built as object files (``.o``).
|
|
|
|
* A static ``libpython`` library (``.a``) is created from objects files.
|
|
|
|
* ``python.o`` and the static ``libpython`` library are linked into the
|
|
|
|
final ``python`` program.
|
|
|
|
* C extensions are built by the Makefile (see :file:`Modules/Setup`)
|
|
|
|
and ``python setup.py build``.
|
|
|
|
|
|
|
|
Main Makefile targets
|
|
|
|
---------------------
|
|
|
|
|
|
|
|
* ``make``: Build Python with the standard library.
|
|
|
|
* ``make platform:``: build the ``python`` program, but don't build the
|
|
|
|
standard library extension modules.
|
|
|
|
* ``make profile-opt``: build Python using Profile Guided Optimization (PGO).
|
|
|
|
You can use the configure :option:`--enable-optimizations` option to make
|
|
|
|
this the default target of the ``make`` command (``make all`` or just
|
|
|
|
``make``).
|
|
|
|
* ``make buildbottest``: Build Python and run the Python test suite, the same
|
|
|
|
way than buildbots test Python. Set ``TESTTIMEOUT`` variable (in seconds)
|
|
|
|
to change the test timeout (1200 by default: 20 minutes).
|
|
|
|
* ``make install``: Build and install Python.
|
|
|
|
* ``make regen-all``: Regenerate (almost) all generated files;
|
|
|
|
``make regen-stdlib-module-names`` and ``autoconf`` must be run separately
|
|
|
|
for the remaining generated files.
|
|
|
|
* ``make clean``: Remove built files.
|
|
|
|
* ``make distclean``: Same than ``make clean``, but remove also files created
|
|
|
|
by the configure script.
|
|
|
|
|
|
|
|
C extensions
|
|
|
|
------------
|
|
|
|
|
|
|
|
Some C extensions are built as built-in modules, like the ``sys`` module.
|
|
|
|
They are built with the ``Py_BUILD_CORE_BUILTIN`` macro defined.
|
|
|
|
Built-in modules have no ``__file__`` attribute::
|
|
|
|
|
|
|
|
>>> import sys
|
|
|
|
>>> sys
|
|
|
|
<module 'sys' (built-in)>
|
|
|
|
>>> sys.__file__
|
|
|
|
Traceback (most recent call last):
|
|
|
|
File "<stdin>", line 1, in <module>
|
|
|
|
AttributeError: module 'sys' has no attribute '__file__'
|
|
|
|
|
2021-10-05 18:48:44 -03:00
|
|
|
Other C extensions are built as dynamic libraries, like the ``_asyncio`` module.
|
2021-04-09 09:35:10 -03:00
|
|
|
They are built with the ``Py_BUILD_CORE_MODULE`` macro defined.
|
|
|
|
Example on Linux x86-64::
|
|
|
|
|
|
|
|
>>> import _asyncio
|
|
|
|
>>> _asyncio
|
|
|
|
<module '_asyncio' from '/usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so'>
|
|
|
|
>>> _asyncio.__file__
|
|
|
|
'/usr/lib64/python3.9/lib-dynload/_asyncio.cpython-39-x86_64-linux-gnu.so'
|
|
|
|
|
|
|
|
:file:`Modules/Setup` is used to generate Makefile targets to build C extensions.
|
|
|
|
At the beginning of the files, C extensions are built as built-in modules.
|
|
|
|
Extensions defined after the ``*shared*`` marker are built as dynamic libraries.
|
|
|
|
|
|
|
|
The :file:`setup.py` script only builds C extensions as shared libraries using
|
|
|
|
the :mod:`distutils` module.
|
|
|
|
|
|
|
|
The :c:macro:`PyAPI_FUNC()`, :c:macro:`PyAPI_API()` and
|
|
|
|
:c:macro:`PyMODINIT_FUNC()` macros of :file:`Include/pyport.h` are defined
|
|
|
|
differently depending if the ``Py_BUILD_CORE_MODULE`` macro is defined:
|
|
|
|
|
|
|
|
* Use ``Py_EXPORTED_SYMBOL`` if the ``Py_BUILD_CORE_MODULE`` is defined
|
|
|
|
* Use ``Py_IMPORTED_SYMBOL`` otherwise.
|
|
|
|
|
|
|
|
If the ``Py_BUILD_CORE_BUILTIN`` macro is used by mistake on a C extension
|
|
|
|
built as a shared library, its ``PyInit_xxx()`` function is not exported,
|
|
|
|
causing an :exc:`ImportError` on import.
|
|
|
|
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
Compiler and linker flags
|
|
|
|
=========================
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Options set by the ``./configure`` script and environment variables and used by
|
|
|
|
``Makefile``.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Preprocessor flags
|
|
|
|
------------------
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. envvar:: CONFIGURE_CPPFLAGS
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Value of :envvar:`CPPFLAGS` variable passed to the ``./configure`` script.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. versionadded:: 3.6
|
|
|
|
|
|
|
|
.. envvar:: CPPFLAGS
|
|
|
|
|
|
|
|
(Objective) C/C++ preprocessor flags, e.g. ``-I<include dir>`` if you have
|
|
|
|
headers in a nonstandard directory ``<include dir>``.
|
|
|
|
|
|
|
|
Both :envvar:`CPPFLAGS` and :envvar:`LDFLAGS` need to contain the shell's
|
|
|
|
value for setup.py to be able to build extension modules using the
|
|
|
|
directories specified in the environment variables.
|
|
|
|
|
|
|
|
.. envvar:: BASECPPFLAGS
|
|
|
|
|
|
|
|
.. versionadded:: 3.4
|
|
|
|
|
|
|
|
.. envvar:: PY_CPPFLAGS
|
|
|
|
|
|
|
|
Extra preprocessor flags added for building the interpreter object files.
|
|
|
|
|
|
|
|
Default: ``$(BASECPPFLAGS) -I. -I$(srcdir)/Include $(CONFIGURE_CPPFLAGS) $(CPPFLAGS)``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
|
|
|
|
Compiler flags
|
|
|
|
--------------
|
|
|
|
|
|
|
|
.. envvar:: CC
|
|
|
|
|
|
|
|
C compiler command.
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Example: ``gcc -pthread``.
|
|
|
|
|
|
|
|
.. envvar:: CXX
|
|
|
|
|
|
|
|
C++ compiler command.
|
|
|
|
|
|
|
|
Example: ``g++ -pthread``.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. envvar:: CFLAGS
|
|
|
|
|
|
|
|
C compiler flags.
|
|
|
|
|
|
|
|
.. envvar:: CFLAGS_NODIST
|
|
|
|
|
|
|
|
:envvar:`CFLAGS_NODIST` is used for building the interpreter and stdlib C
|
|
|
|
extensions. Use it when a compiler flag should *not* be part of the
|
|
|
|
distutils :envvar:`CFLAGS` once Python is installed (:issue:`21121`).
|
|
|
|
|
2022-06-20 10:58:02 -03:00
|
|
|
In particular, :envvar:`CFLAGS` should not contain:
|
|
|
|
|
2022-10-06 22:01:30 -03:00
|
|
|
* the compiler flag ``-I`` (for setting the search path for include files).
|
|
|
|
The ``-I`` flags are processed from left to right, and any flags in
|
|
|
|
:envvar:`CFLAGS` would take precedence over user- and package-supplied ``-I``
|
2022-06-20 10:58:02 -03:00
|
|
|
flags.
|
|
|
|
|
2022-10-06 22:01:30 -03:00
|
|
|
* hardening flags such as ``-Werror`` because distributions cannot control
|
2022-06-20 10:58:02 -03:00
|
|
|
whether packages installed by users conform to such heightened
|
|
|
|
standards.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. versionadded:: 3.5
|
|
|
|
|
2022-11-14 08:43:45 -04:00
|
|
|
.. envvar:: COMPILEALL_OPTS
|
|
|
|
|
|
|
|
Options passed to the :mod:`compileall` command line when building PYC files
|
|
|
|
in ``make install``. Default: ``-j0``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.12
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. envvar:: EXTRA_CFLAGS
|
|
|
|
|
|
|
|
Extra C compiler flags.
|
|
|
|
|
|
|
|
.. envvar:: CONFIGURE_CFLAGS
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Value of :envvar:`CFLAGS` variable passed to the ``./configure``
|
|
|
|
script.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. versionadded:: 3.2
|
|
|
|
|
|
|
|
.. envvar:: CONFIGURE_CFLAGS_NODIST
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Value of :envvar:`CFLAGS_NODIST` variable passed to the ``./configure``
|
|
|
|
script.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. versionadded:: 3.5
|
|
|
|
|
|
|
|
.. envvar:: BASECFLAGS
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Base compiler flags.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. envvar:: OPT
|
|
|
|
|
|
|
|
Optimization flags.
|
|
|
|
|
|
|
|
.. envvar:: CFLAGS_ALIASING
|
|
|
|
|
|
|
|
Strict or non-strict aliasing flags used to compile ``Python/dtoa.c``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.7
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
.. envvar:: CCSHARED
|
|
|
|
|
|
|
|
Compiler flags used to build a shared library.
|
|
|
|
|
|
|
|
For example, ``-fPIC`` is used on Linux and on BSD.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. envvar:: CFLAGSFORSHARED
|
|
|
|
|
|
|
|
Extra C flags added for building the interpreter object files.
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Default: ``$(CCSHARED)`` when :option:`--enable-shared` is used, or an empty
|
|
|
|
string otherwise.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. envvar:: PY_CFLAGS
|
|
|
|
|
|
|
|
Default: ``$(BASECFLAGS) $(OPT) $(CONFIGURE_CFLAGS) $(CFLAGS) $(EXTRA_CFLAGS)``.
|
|
|
|
|
|
|
|
.. envvar:: PY_CFLAGS_NODIST
|
|
|
|
|
|
|
|
Default: ``$(CONFIGURE_CFLAGS_NODIST) $(CFLAGS_NODIST) -I$(srcdir)/Include/internal``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.5
|
|
|
|
|
|
|
|
.. envvar:: PY_STDMODULE_CFLAGS
|
|
|
|
|
|
|
|
C flags used for building the interpreter object files.
|
|
|
|
|
|
|
|
Default: ``$(PY_CFLAGS) $(PY_CFLAGS_NODIST) $(PY_CPPFLAGS) $(CFLAGSFORSHARED)``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.7
|
|
|
|
|
|
|
|
.. envvar:: PY_CORE_CFLAGS
|
|
|
|
|
|
|
|
Default: ``$(PY_STDMODULE_CFLAGS) -DPy_BUILD_CORE``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
|
|
|
|
.. envvar:: PY_BUILTIN_MODULE_CFLAGS
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Compiler flags to build a standard library extension module as a built-in
|
|
|
|
module, like the :mod:`posix` module.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
Default: ``$(PY_STDMODULE_CFLAGS) -DPy_BUILD_CORE_BUILTIN``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.8
|
|
|
|
|
2021-04-09 09:35:10 -03:00
|
|
|
.. envvar:: PURIFY
|
|
|
|
|
|
|
|
Purify command. Purify is a memory debugger program.
|
|
|
|
|
|
|
|
Default: empty string (not used).
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
Linker flags
|
|
|
|
------------
|
|
|
|
|
2021-04-09 09:35:10 -03:00
|
|
|
.. envvar:: LINKCC
|
|
|
|
|
|
|
|
Linker command used to build programs like ``python`` and ``_testembed``.
|
|
|
|
|
2022-08-05 08:26:58 -03:00
|
|
|
Default: ``$(PURIFY) $(CC)``.
|
2021-04-09 09:35:10 -03:00
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. envvar:: CONFIGURE_LDFLAGS
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Value of :envvar:`LDFLAGS` variable passed to the ``./configure`` script.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
Avoid assigning :envvar:`CFLAGS`, :envvar:`LDFLAGS`, etc. so users can use
|
|
|
|
them on the command line to append to these values without stomping the
|
|
|
|
pre-set values.
|
|
|
|
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
|
|
|
|
.. envvar:: LDFLAGS_NODIST
|
|
|
|
|
|
|
|
:envvar:`LDFLAGS_NODIST` is used in the same manner as
|
|
|
|
:envvar:`CFLAGS_NODIST`. Use it when a linker flag should *not* be part of
|
|
|
|
the distutils :envvar:`LDFLAGS` once Python is installed (:issue:`35257`).
|
|
|
|
|
2022-06-20 10:58:02 -03:00
|
|
|
In particular, :envvar:`LDFLAGS` should not contain:
|
|
|
|
|
2022-10-06 22:01:30 -03:00
|
|
|
* the compiler flag ``-L`` (for setting the search path for libraries).
|
|
|
|
The ``-L`` flags are processed from left to right, and any flags in
|
|
|
|
:envvar:`LDFLAGS` would take precedence over user- and package-supplied ``-L``
|
2022-06-20 10:58:02 -03:00
|
|
|
flags.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. envvar:: CONFIGURE_LDFLAGS_NODIST
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Value of :envvar:`LDFLAGS_NODIST` variable passed to the ``./configure``
|
|
|
|
script.
|
|
|
|
|
2021-04-08 17:32:21 -03:00
|
|
|
.. versionadded:: 3.8
|
|
|
|
|
|
|
|
.. envvar:: LDFLAGS
|
|
|
|
|
|
|
|
Linker flags, e.g. ``-L<lib dir>`` if you have libraries in a nonstandard
|
|
|
|
directory ``<lib dir>``.
|
|
|
|
|
|
|
|
Both :envvar:`CPPFLAGS` and :envvar:`LDFLAGS` need to contain the shell's
|
|
|
|
value for setup.py to be able to build extension modules using the
|
|
|
|
directories specified in the environment variables.
|
|
|
|
|
|
|
|
.. envvar:: LIBS
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Linker flags to pass libraries to the linker when linking the Python
|
|
|
|
executable.
|
|
|
|
|
|
|
|
Example: ``-lrt``.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
.. envvar:: LDSHARED
|
|
|
|
|
|
|
|
Command to build a shared library.
|
|
|
|
|
|
|
|
Default: ``@LDSHARED@ $(PY_LDFLAGS)``.
|
|
|
|
|
|
|
|
.. envvar:: BLDSHARED
|
|
|
|
|
2021-04-08 20:58:20 -03:00
|
|
|
Command to build ``libpython`` shared library.
|
2021-04-08 17:32:21 -03:00
|
|
|
|
|
|
|
Default: ``@BLDSHARED@ $(PY_CORE_LDFLAGS)``.
|
|
|
|
|
|
|
|
.. envvar:: PY_LDFLAGS
|
|
|
|
|
|
|
|
Default: ``$(CONFIGURE_LDFLAGS) $(LDFLAGS)``.
|
|
|
|
|
|
|
|
.. envvar:: PY_LDFLAGS_NODIST
|
|
|
|
|
|
|
|
Default: ``$(CONFIGURE_LDFLAGS_NODIST) $(LDFLAGS_NODIST)``.
|
|
|
|
|
|
|
|
.. versionadded:: 3.8
|
|
|
|
|
|
|
|
.. envvar:: PY_CORE_LDFLAGS
|
|
|
|
|
|
|
|
Linker flags used for building the interpreter object files.
|
|
|
|
|
|
|
|
.. versionadded:: 3.8
|