There were about 14 files that are actually in the repo but that are
covered by the rules in .gitignore.
Git itself takes no notice of what .gitignore says about files that
it's already tracking... but the discrepancy can be confusing to a
human that adds a new file unexpectedly covered by these rules, as
well as to non-Git software that looks at .gitignore but doesn't
implement this wrinkle in its semantics. (E.g., `rg`.)
Several of these are from rules that apply more broadly than
intended: for example, `Makefile` applies to `Doc/Makefile` and
`Tools/freeze/test/Makefile`, whereas `/Makefile` means only the
`Makefile` at the repo's root.
And the `Modules/Setup` rule simply wasn't updated after 961d54c5c.
https://bugs.python.org/issue37936
(cherry picked from commit 5e5e951502)
Co-authored-by: Greg Price <gnprice@gmail.com>
Mark some individual tests to skip when --pgo is used. The tests
marked increase the PGO task time significantly and likely don't
help improve optimization of the final executable.
(cherry picked from commit 52a48e62c6)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
Reduce the number of unit tests run for the PGO generation task. This
speeds up the task by a factor of about 15x. Running the full unit test
suite is slow. This change may result in a slightly less optimized build
since not as many code branches will be executed. If you are willing to
wait for the much slower build, the old behavior can be restored using
'./configure [..] PROFILE_TASK="-m test --pgo-extended"'. We make no
guarantees as to which PGO task set produces a faster build. Users who
care should run their own relevant benchmarks as results can depend on
the environment, workload, and compiler tool chain.
(cherry picked from commit 4e16a4a311)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
Many PyRun_XXX() functions like PyRun_String() were no longer
exported in libpython38.dll by mistake. Export them again to fix the
ABI compatibiliy.
(cherry picked from commit 343ed0ffe0)
Co-authored-by: Victor Stinner <vstinner@redhat.com>
To embed Python into an application, a new --embed option must be
passed to "python3-config --libs --embed" to get "-lpython3.8" (link
the application to libpython). To support both 3.8 and older, try
"python3-config --libs --embed" first and fallback to "python3-config
--libs" (without --embed) if the previous command fails.
Add a pkg-config "python-3.8-embed" module to embed Python into an
application: "pkg-config python-3.8-embed --libs" includes
"-lpython3.8". To support both 3.8 and older, try "pkg-config
python-X.Y-embed --libs" first and fallback to "pkg-config python-X.Y
--libs" (without --embed) if the previous command fails (replace
"X.Y" with the Python version).
On the other hand, "pkg-config python3.8 --libs" no longer contains
"-lpython3.8". C extensions must not be linked to libpython (except
on Android, case handled by the script); this change is backward
incompatible on purpose.
"make install" now also installs "python-3.8-embed.pc".
On Unix, C extensions are no longer linked to libpython.
It is now possible to load a C extension built using a shared library
Python with a statically linked Python.
When Python is embedded, libpython must not be loaded with
RTLD_LOCAL, but RTLD_GLOBAL instead. Previously, using RTLD_LOCAL, it
was already not possible to load C extensions which were not linked
to libpython, like C extensions of the standard library built by the
"*shared*" section of Modules/Setup.
distutils, python-config and python-config.py have been modified.
Release build and debug build are now ABI compatible: the Py_DEBUG
define no longer implies Py_TRACE_REFS define which introduces the
only ABI incompatibility.
A new "./configure --with-trace-refs" build option is now required to
get Py_TRACE_REFS define which adds sys.getobjects() function and
PYTHONDUMPREFS environment variable.
Changes:
* Add ./configure --with-trace-refs
* Py_DEBUG no longer implies Py_TRACE_REFS
"./configure --with-pymalloc" no longer adds the "m" flag to SOABI
(sys.implementation.cache_tag).
Enabling or disabling pymalloc has no impact on the ABI.
Change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined.
The Py_BUILD_CORE_MODULE define must be now be used to build a C
extension as a dynamic library accessing Python internals: export the
PyInit_xxx() function in DLL exports on Windows.
Changes:
* Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE now imply
Py_BUILD_CORE directy in pyport.h.
* ceval.c compilation now fails with an error if Py_BUILD_CORE is not
defined, just to ensure that Python is build with the correct
defines.
* setup.py now compiles _pickle.c with Py_BUILD_CORE_MODULE define.
* setup.py compiles _json.c with Py_BUILD_CORE_MODULE define, rather
than Py_BUILD_CORE_BUILTIN define
* PCbuild/pythoncore.vcxproj: Add Py_BUILD_CORE_BUILTIN define.
Add -fmax-type-align=8 to CFLAGS when clang compiler is detected.
The pymalloc memory allocator aligns memory on 8 bytes. On x86-64,
clang expects alignment on 16 bytes by default and so uses MOVAPS
instruction which can lead to segmentation fault. Instruct clang that
Python is limited to alignemnt on 8 bytes to use MOVUPS instruction
instead: slower but don't trigger a SIGSEGV if the memory is not
aligned on 16 bytes.
Sadly, the flag must be expected to CFLAGS and not just
CFLAGS_NODIST, since third party C extensions can have the same
issue.
[bpo-36146](https://bugs.python.org/issue36146) introduced another regression. In case of missing OpenSSL
libraries or headers, setup.py no longer reported _hashlib and _ssl to
be missing.
Signed-off-by: Christian Heimes <christian@python.org>
https://bugs.python.org/issue36577
Add TEST_EXTENSIONS constant to setup.py to allow to not build test
extensions like _testcapi.
Changes:
* Add add_ldflags_cppflags() subfunction
* Rename add_compiler_directories() to configure_compiler().
* Remove unused COMPILED_WITH_PYDEBUG constant.
* Use self.add() rather than accessing directly self.extensions.
* Remove module_enabled() function: check differently if curses
extension is built or not.
When compiling 3rd party C extensions, the linker flags used by the
compiler for the interpreter and the stdlib modules, will get
leaked into distutils. In order to avoid that, the PY_CORE_LDFLAGS
and PY_LDFLAGS_NODIST are introduced to keep those flags separated.
"make profile-opt" no longer replaces CFLAGS_NODIST with CFLAGS. It
now adds profile-guided optimization (PGO) flags to CFLAGS_NODIST,
existing CFLAGS_NODIST flags are kept.
When using link time optimizations, the -flto flag is passed to
BASECFLAGS, which makes it propagate to distutils. Those flags
should be reserved for the interpreter and the stdlib extension
modules only, thus moving those flags to CFLAGS_NODIST.
Fix an undefined behaviour in the pthread implementation of
PyThread_start_new_thread(): add a function wrapper to always return
NULL.
Add pythread_callback struct and pythread_wrapper() to thread_pthread.h.
Visual Studio solution: Set InlineFunctionExpansion to
OnlyExplicitInline ("/Ob1" option) on all projects (in
pyproject.props) in Debug mode on Win32 and x64 platforms to expand
functions marked as inline.
This change should make Python compiled in Debug mode a little bit
faster on Windows. On Unix, GCC uses -Og optimization level for
./configure --with-pydebug.
.o generated by clang in LTO mode actually are LLVM bitcode files, which
leads to a few errors during configure/build step:
- add lto flags to the BASECFLAGS instead of CFLAGS, as CFLAGS are used
to build autoconf test case, and some are not compatible with clang LTO
(they assume binary in the .o, not bitcode)
- force llvm-ar instead of ar, as ar is not aware of .o files generated
by clang -flto
Restores the use of pyexpatns.h to isolate our embedded copy of the expat C
library so that its symbols do not conflict at link or dynamic loading time
with an embedding application or other extension modules with their own
version of libexpat.
5dc3f23b5f (diff-3afaf7274c90ce1b7405f75ad825f545) inadvertently removed it when upgrading expat.
Currently configure.ac uses AC_RUN_IFELSE to determine the byte order of doubles, but this silently fails under cross compilation and Python doesn't do floats properly.
Instead, steal a macro from autoconf-archive which compiles code using magic doubles (which encode to ASCII) and grep for the representation in the binary.
RFC because this doesn't yet handle the weird ancient ARMv4 OABI 'mixed-endian' encoding properly. This encoding is ancient and I don't believe the union of "Python 3.8 users" and "OABI users" has anything in. Should the support for this just be dropped too? Alternatively, someone will need to find an OABI toolchain to verify the encoding of the magic double.
Miss Islington (bot)
Łukasz Langa
Victor Stinner
Antoine Pitrou
xdegaye
Paul Ganssle
pxinwr
Christian Heimes
Michael Felt
btharper
Steve Dower
Gregory Szorc
Jakub Kulík
stratakis
jdemeyer
Siddhesh Poyarekar
Max Bélanger
serge-sans-paille
Xtreak
Gregory P. Smith
Ross Burton
Alexandru Ardelean