On content update, builds `python` and the docs. Also adds a Dockerfile that should include everything but autoconf 2.69 that's necessary to build CPython and the entire stdlib on Fedora.
Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>
Co-authored-by: Dusty Phillips <dusty@phillips.codes>
There are some warnings if build python via clang:
Parser/pegen.c:812:31: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
_PyPegen_clear_memo_statistics()
^
void
Parser/pegen.c:820:29: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
_PyPegen_get_memo_statistics()
^
void
Fix it to make clang happy.
Signed-off-by: Chenxi Mao <chenxi.mao@suse.com>
In commit 254b309c80 a previous change to avoid linking to libpython was partially reverted for Android (and later Cygwin as well), to add back the link flags. This was applied to distutils and to python-config.sh, but not to python.pc.
Add it back to python.pc as well.
Automerge-Triggered-By: GH:gpshead
Fix creating install directories in `make sharedinstall` if they exist already outside `DESTDIR`. The previous make rules assumed that the directories would be created via a dependency on a rule for `$(DESTSHARED)` that did not fire if the directory did exist outside `$(DESTDIR)`.
While technically `$(DESTDIR)` could be prepended to the rule name, moving the rules for creating directories straight into the `sharedinstall` rule seems to fit the common practices better. Since the rule explicitly checks whether the individual directories exist anyway, there seems to be no reason to rely on make determining that implicitly as well.
To use this, ensure that clang support was selected in Visual Studio Installer, then set the PlatformToolset environment variable to "ClangCL" and build as normal from the command line.
It remains unsupported, but at least is possible now for experimentation.
Fix the gdbm_compat library detection logic to actually check for
-lgdbm_compat independently of the ndbm detection.
This fixes the build failure with `--with-dbmliborder=gdbm`,
and implicit fallback to ndbm with the default value.
When cross-compiling, the compile/run test for -pthread always fails so -pthread
will never be automatically set without an override from the cache. ac_cv_pthread
can already be overridden, so do the same thing for ac_cv_cxx_thread.
It has had no effect on non-macOS platforms for a long time, and has had
the non-obvious effect of invoking `pkg_config` and not setting
`-DUSING_APPLE_OS_LIBFFI` on macOS since GH-22855.
Fix a number of compile errors with GCC-12 on macOS:
1. In pylifecycle.c the compile rejects _Pragma within a declaration
2. posixmodule.c was missing a number of ..._RUNTIME macros for non-clang on macOS
3. _ctypes assumed that __builtin_available is always present on macOS
Add COMPILEALL_OPTS variable in Makefile to override compileall
options (default: -j0) in "make install". Also merge the compileall
commands into a single command building PYC files for the all
optimization levels (0, 1, 2) at once.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
This got introduced in commit 5884449539
to determine if readline is already linked against curses or tinfo in
the setup.py, which is no longer present.
The switch cases (really TARGET(opcode) macros) have been moved from ceval.c to generated_cases.c.h. That file is generated from instruction definitions in bytecodes.c (which impersonates a C file so the C code it contains can be edited without custom support in e.g. VS Code).
The code generator lives in Tools/cases_generator (it has a README.md explaining how it works). The DSL used to describe the instructions is a work in progress, described in https://github.com/faster-cpython/ideas/blob/main/3.12/interpreter_definition.md.
This is surely a work-in-progress. An easy next step could be auto-generating super-instructions.
**IMPORTANT: Merge Conflicts**
If you get a merge conflict for instruction implementations in ceval.c, your best bet is to port your changes to bytecodes.c. That file looks almost the same as the original cases, except instead of `TARGET(NAME)` it uses `inst(NAME)`, and the trailing `DISPATCH()` call is omitted (the code generator adds it automatically).
* Add support for the BOLT post-link binary optimizer
Using [bolt](https://github.com/llvm/llvm-project/tree/main/bolt)
provides a fairly large speedup without any code or functionality
changes. It provides roughly a 1% speedup on pyperformance, and a
4% improvement on the Pyston web macrobenchmarks.
It is gated behind an `--enable-bolt` configure arg because not all
toolchains and environments are supported. It has been tested on a
Linux x86_64 toolchain, using llvm-bolt built from the LLVM 14.0.6
sources (their binary distribution of this version did not include bolt).
Compared to [a previous attempt](https://github.com/faster-cpython/ideas/issues/224),
this commit uses bolt's preferred "instrumentation" approach, as well as adds some non-PIE
flags which enable much better optimizations from bolt.
The effects of this change are a bit more dependent on CPU microarchitecture
than other changes, since it optimizes i-cache behavior which seems
to be a bit more variable between architectures. The 1%/4% numbers
were collected on an Intel Skylake CPU, and on an AMD Zen 3 CPU I
got a slightly larger speedup (2%/4%), and on a c6i.xlarge EC2 instance
I got a slightly lower speedup (1%/3%).
The low speedup on pyperformance is not entirely unexpected, because
BOLT improves i-cache behavior, and the benchmarks in the pyperformance
suite are small and tend to fit in i-cache.
This change uses the existing pgo profiling task (`python -m test --pgo`),
though I was able to measure about a 1% macrobenchmark improvement by
using the macrobenchmarks as the training task. I personally think that
both the PGO and BOLT tasks should be updated to use macrobenchmarks,
but for the sake of splitting up the work this PR uses the existing pgo task.
* Simplify the build flags
* Add a NEWS entry
* Update Makefile.pre.in
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Update configure.ac
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
* Add myself to ACKS
* Add docs
* Other review comments
* fix tab/space issue
* Make it more clear that --enable-bolt is experimental
* Add link to bolt's github page
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
Remove the "configure --with-cxx-main" build option: it didn't work
for many years. Remove the MAINCC variable from configure and
Makefile.
The MAINCC variable was added by the issue gh-42471: commit
0f48d98b74. Previously, --with-cxx-main
was named --with-cxx.
Keep CXX and LDCXXSHARED variables, even if they are no longer used
by Python build system.
- check for ``dup()`` libc function
- handle missing ``F_DUPFD`` in ``dup2()`` replacement function
- add workaround for WASI libc bug in MSG_TRUNC
- ESHUTDOWN is missing, use EPIPE instead
- POLLPRI is missing, define as 0 (no-op)
All install targets use the "all" target as synchronization point to
prevent race conditions with PGO builds. PGO builds use recursive make,
which can lead to two parallel `./python setup.py build` processes that
step on each others toes.
"test" targets now correctly compile PGO build in a clean repo.
Python now always use the ``%zu`` and ``%zd`` printf formats to
format a size_t or Py_ssize_t number. Building Python 3.12 requires a
C11 compiler, so these printf formats are now always supported.
* PyObject_Print() and _PyObject_Dump() now use the printf %zd format
to display an object reference count.
* Update PY_FORMAT_SIZE_T comment.
* Remove outdated notes about the %zd format in PyBytes_FromFormat()
and PyUnicode_FromFormat() documentations.
* configure no longer checks for the %zd format and no longer defines
PY_FORMAT_SIZE_T macro in pyconfig.h.
* pymacconfig.h no longer undefines PY_FORMAT_SIZE_T: macOS 10.4 is
no longer supported. Python 3.12 now requires macOS 10.6 (Snow
Leopard) or newer.
At compile time, '+z' is already properly used with HP aCC, and shared
libraries are correctly linked with '+b'. The '-fPIC' switch can safely be
dropped.
This makes macOS gdbm provided by Homebrew not segfault through correct
selection of the linked library (-lgdbm_compat) *AND* the correct ndbm-style
header (gdbm-ndbm.h instead of the invalid ndbm.h).
* Move the code for generating Modules/_sre/sre_constants.h from
Lib/re/_constants.py into a separate script
Tools/scripts/generate_sre_constants.py.
* Add target `regen-sre` in the makefile.
* Make target `regen-all` depending on `regen-sre`.
Shared extension on Emscripten now have suffix
``.cpython-311-wasm32-emscripten.so`` (JS loader) and
``.cpython-311-wasm32-emscripten.wasm`` (WebAssembly code).
`main` branch is failing, see https://dev.azure.com/python/cpython/_build/results?buildId=96616&view=logs&j=4db1505a-29e5-5cc0-240b-53a8a2681f75&t=a975920c-8356-5388-147c-613d5fab0171
Logs:
```
PATH=./venv/bin:$PATH sphinx-lint -i tools -i ./venv -i README.rst
No problems found.
PATH=./venv/bin:$PATH sphinx-lint ../Misc/NEWS.d/next/
[1] ../Misc/NEWS.d/next/Library/2022-02-09-00-53-23.[bpo-45863]().zqQXVv.rst:0: No newline at end of file (no-newline-at-end-of-file).
[1] ../Misc/NEWS.d/next/Build/2022-01-19-11-08-32.[bpo-46430]().k403m_.rst:0: No newline at end of file (no-newline-at-end-of-file).
2 problems with severity 1 found.
```
This PR fixes these two problems, so `main` is green again.
Related PR: https://github.com/python/cpython/pull/31097
CC @JulienPalard
Automerge-Triggered-By: GH:JulienPalard
Building Python now requires a C99 <math.h> header file providing a
NAN constant, or the __builtin_nan() built-in function. If a platform
does not support Not-a-Number (NaN), the Py_NO_NAN macro can be
defined in the pyconfig.h file.
This reduces the size of the data segment by **300 KB** of the executable because if the modules are deep-frozen then the marshalled frozen data just wastes space. This was inspired by comment by @gvanrossum in https://github.com/python/cpython/pull/29118#issuecomment-958521863. Note: There is a new option `--deepfreeze-only` in `freeze_modules.py` to change this behavior, it is on be default to save disk space.
```console
# du -s ./python before
27892 ./python
# du -s ./python after
27524 ./python
```
Automerge-Triggered-By: GH:ericsnowcurrently
Brett Cannon
Chenxi Mao
Thomas Wouters
Matthias Görgens
Eli Schwartz
Michał Górny
Dong-hee Na
Steve Dower
Ayappan Perumal
Gregory Szorc
ram vikram singh
Kushal Das
Dustin Spicuzza
Zachary Ware
Christian Rendina
Sam James
SQLPATCH
Skip Montanaro
Ronald Oussoren
Victor Stinner
Miro Hrončok
Hugo van Kemenade
Guido van Rossum
Sam James
serge-sans-paille
Serhiy Storchaka
Jelle Zijlstra
Christian Heimes
adang1345
Pablo Galindo Salgado
Kevin Modzelewski
Oleg Iarygin
neonene
Shixian Li
Kumar Aditya
Michael Osipov
Łukasz Langa
Erlend Egeberg Aasland
Nikita Sobolev