Commit Graph

706 Commits

Author SHA1 Message Date
adanhawth b1288964e3
bpo-46602: Do not append conftest.c (GH-31062)
The heredoc creation statements use >> to append conftest.c.  This can cause
tricky build issues if the file is not correctly removed prior to its
name being reused (such name is reused several times for different
contextual tests during the build).  One such result from appending may
cause #include <ac_nonexistent.h> to persist when testing to acquire
PLATFORM_TRIPLET.  This can then lead to downstream issues concerning SOABI.
2022-02-01 21:38:15 -05:00
Victor Stinner 0515eafe55
bpo-46600: ./configure --with-pydebug uses -Og with clang (GH-31052)
Fix the test checking if the C compiler supports -Og option in the
./configure script to also use -Og on clang which supports it.
2022-02-01 14:47:12 +01:00
Natanael Copa 1f036ede59
bpo-43112: detect musl as a separate SOABI (GH-24502)
musl libc and gnu libc are not ABI compatible so we need set different
SOABI for musl and not simply assume that all linux is linux-gnu.

Replace linux-gnu with the detected os for the build from config.guess
for linux-musl*.
2022-01-28 15:02:54 -08:00
Christian Heimes 6e5a193816
bpo-46513: Remove AC_C_CHAR_UNSIGNED / __CHAR_UNSIGNED__ (GH-30851) 2022-01-26 01:03:49 -08:00
Christian Heimes d5fd438b38
bpo-40280: Get help() working and more (GH-30858) 2022-01-24 22:02:01 +01:00
Thomas Klausner 40fcd16889
bpo-30512: Add CAN Socket support for NetBSD (GH-30066) 2022-01-21 09:44:05 +02:00
Christian Heimes c02e860ee7
bpo-40280: Misc fixes for wasm32-emscripten (GH-30722) 2022-01-20 18:56:33 +01:00
Thomas Klausner 60ceedbdd5
bpo-46045: Do not use POSIX semaphores on NetBSD (GH-30047)
This fixes hanging tests test_compileall,, test_multiprocessing_fork and test_concurrent_futures.
2022-01-18 22:38:35 +02:00
Mark Dickinson 025cbe7a9b
bpo-45569: Change PYLONG_BITS_IN_DIGIT default to 30 (GH-30497) 2022-01-14 18:54:56 +00:00
Christian Heimes c8319f7921
bpo-40280: Build WASM stdlib bundle and more modules for node (GH-30597) 2022-01-14 12:11:49 +01:00
Victor Stinner 6be848922b
bpo-44133: Link Python executable with object files (GH-30556)
When Python is built without --enable-shared, the "python" program is
now linked to object files, rather than being linked to the Python
library (libpython.a), to make sure that all symbols are exported.
Previously, the linker omitted some symbols like the Py_FrozenMain()
function.

When Python is configured with --without-static-libpython, the Python
static library (libpython.a) is no longer built.

* Check --without-static-libpython earlier in configure.ac
* Add LINK_PYTHON_OBJS and LINK_PYTHON_DEPS variables to Makefile.
* test_capi now ensures that the "Py_FrozenMain" symbol is exported.
2022-01-13 19:24:28 +01:00
Christian Heimes a6ca8eee22
bpo-46315: Add ifdef HAVE_ feature checks for WASI compatibility (GH-30507) 2022-01-13 09:46:04 +01:00
Christian Heimes e34c9367f8
bpo-40280: Allow to compile _testcapi as builtin module (GH-30559) 2022-01-12 20:27:37 +01:00
Christian Heimes 43839ba438
bpo-40280: Add --with-emscripten-target to build for browser or node (GH-30552)
Co-authored-by: Ethan Smith <ethan@ethanhs.me>
2022-01-12 16:08:19 +01:00
Thomas Klausner 3d11c1b8b4
bpo-46308: Fix unportable test(1) operator in configure script (GH-30490) 2022-01-08 19:54:13 -05:00
Christian Heimes 994f90c077
bpo-45723: Fix detection of epoll (#30449) 2022-01-07 09:15:20 +01:00
Christian Heimes cae55542d2
bpo-46263: Don't use MULTIARCH on FreeBSD (#30410) 2022-01-05 10:54:17 +01:00
Christian Heimes 0339434835
bpo-40280: Add Tools/wasm with helpers for cross building (GH-29984)
Co-authored-by: Ethan Smith <ethan@ethanhs.me>
Co-authored-by: Brett Cannon <brett@python.org>
2021-12-18 15:54:02 +01:00
Mark Shannon 342b93f9f2
bpo-46072: Add --with-pystats configure option to simplify gathering of VM stats (GH-30116)
* Simplify specialization stats collection macros.

* Add --enable-pystats option to configure.

* Update specialization summary script to handle larger number of kinds
2021-12-15 15:32:32 +00:00
Christian Heimes eb483c46d6
bpo-45949: Pure Python freeze module for cross builds (GH-29899) 2021-12-13 20:48:46 +01:00
Erlend Egeberg Aasland 74b23c97cd
bpo-45723: Normalise configure user communication (GH-30024) 2021-12-10 12:27:38 +01:00
Christian Heimes 0461c68cdf
bpo-45847: Don't override user overrides for CFLAGS/LIBS (GH-29967) 2021-12-08 11:52:08 +01:00
Christian Heimes 91b59a3fcd
bpo-45847: Revert Port _ctypes partly to PY_STDLIB_MOD (GH-29747) (GH-29969) 2021-12-07 20:56:41 +01:00
Ned Deily ddbab69b6d
bpo-45798: Let libmpdec decide which archs to build on macOS as done previously. (GH-29949) 2021-12-06 21:35:50 -05:00
Christian Heimes fc012d8012
bpo-45847: Fix uuid detection on macOS (GH-29946) 2021-12-06 21:43:44 +01:00
Christian Heimes 612e59b53f
bpo-45950: Fix macOS framework builds of _bootstrap_python (GH-29936) 2021-12-06 17:13:53 +01:00
Christian Heimes 64be8d369b
bpo-45847: Update whatsnew and add place holder entries for missing extensions (GH-29914) 2021-12-04 15:14:48 +01:00
Christian Heimes 4045392e0e
bpo-45847: Port _ctypes partly to PY_STDLIB_MOD (GH-29747)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-12-04 11:21:43 +01:00
Christian Heimes 84ca1232b0
bpo-45950: Introduce Bootstrap Python again (#29859)
The build system now uses a :program:`_bootstrap_python` interpreter for
freezing and deepfreezing again. To speed up build process the build tools
:program:`_bootstrap_python` and :program:`_freeze_module` are no longer
build with LTO.

Cross building depends on a build Python interpreter, which must have same
version and bytecode as target host Python.
2021-12-03 16:01:11 +01:00
Christian Heimes 309110f37c
bpo-40280: Emscripten with_ensurepip=no, second attempt (GH-29884) 2021-12-01 23:16:27 +01:00
Christian Heimes 9deb83468c
bpo-40280: Emscripten defaults to --with-ensurepip=no (GH-29873) 2021-12-01 20:43:07 +01:00
Christian Heimes 992565f7f7
bpo-45881: configure --with-freeze-module --with-build-python (GH-29835)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
Co-authored-by: Ethan Smith <ethan@ethanhs.me>
2021-11-29 17:23:29 +01:00
Christian Heimes b394af13f6
bpo-45847: PY_STDLIB_MOD_SIMPLE now checks py_stdlib_not_available (GH-29844) 2021-11-29 16:34:16 +01:00
Christian Heimes 6ac3c8a314
bpo-40280: Emscripten systems use .wasm suffix by default (GH-29842) 2021-11-29 16:01:55 +01:00
Erlend Egeberg Aasland c1dec9540a
bpo-45723: Sort the grand AC_CHECK_HEADERS check (GH-29846)
Automerge-Triggered-By: GH:tiran
2021-11-29 06:41:11 -08:00
Erlend Egeberg Aasland f7a62f2405
bpo-45847: Make socket module conditional (GH-29769) 2021-11-29 14:57:36 +01:00
Erlend Egeberg Aasland b3f443a35e
bpo-40280: Disable unusable core extension modules on emscripten (GH-29834)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-29 12:39:14 +01:00
Erlend Egeberg Aasland aaf42222cf
bpo-45847: Port _multiprocessing to PY_STDLIB_MOD (GH-29768) 2021-11-28 13:00:51 +01:00
Pablo Galindo Salgado e71c12efcd
bpo-42268: Fail the configure step if the selected compiler doesn't support memory sanitizer (GH-29806) 2021-11-27 18:04:06 +00:00
Christian Heimes 4ebde73b8e
bpo-40280: Move hard-coded feature checks to configure (GH-29789)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-26 19:36:48 +01:00
Christian Heimes 8caceb7a47
bpo-40280: Add configure check for socket shutdown (GH-29795) 2021-11-26 15:16:54 +01:00
Christian Heimes ee1e2c604c
bpo-40280: Use Setup.stdlib static for wasm builds (GH-29784)
``Modules/Setup.stdlib`` contains ``Setup`` lines for all stdlib extension modules for which ``configure`` has detected their dependencies. The file is not used yet and still under development. To use the file, do ``ln -sfr Modules/Setup.stdlib Modules/Setup.local``.
2021-11-26 08:26:49 +01:00
Christian Heimes 1052a39b76
bpo-40280: Add wasm cross build targets (GH-29771) 2021-11-25 21:24:40 +01:00
Christian Heimes 64c3807da9
bpo-45847: Fix _crypt detection on Ubuntu (GH-29743) 2021-11-24 10:47:22 +01:00
Christian Heimes 8af6481f6b
bpo-45847: Port _uuid to PY_STDLIB_MOD (GH-29741) 2021-11-24 10:20:37 +01:00
Erlend Egeberg Aasland 324527012f
bpo-45847: Port _posixshmem to PY_STDLIB_MOD (GH-29738) 2021-11-24 08:19:17 +01:00
Christian Heimes 9cf5646bb4
bpo-45847: Port _gdbm to PY_STDLIB_MOD (GH-29720) 2021-11-23 22:58:38 +01:00
Christian Heimes b9e9292d75
bpo-45847: Port _ssl and _hashlib to PY_STDLIB_MOD (GH-29727) 2021-11-23 22:58:13 +01:00
Christian Heimes 095bc8f0d6
bpo-45847: Port _crypt to PY_STDLIB_MOD (GH-29725) 2021-11-23 22:26:50 +01:00
Christian Heimes f840398a5f
bpo-45873: Restore Python 3.6 compatibility (GH-29730)
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2021-11-23 21:36:40 +01:00
Guido van Rossum 5be98e57b3
bpo-45873: Get rid of bootstrap_python (#29717)
Instead we use $(PYTHON_FOR_REGEN) .../deepfreeze.py with the
frozen .h file as input, as we did for Windows in bpo-45850.

We also get rid of the code that generates the .h files
when make regen-frozen is run (i.e., .../make_frozen.py),
and the MANIFEST file.

Restore Python 3.8 and 3.9 as Windows host Python again

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2021-11-23 08:56:06 -08:00
Erlend Egeberg Aasland d82f2caf94
bpo-45847: Port _socket to PY_STDLIB_MOD (GH-29713) 2021-11-23 08:52:05 +01:00
Erlend Egeberg Aasland 758a23d1c4
bpo-45847: Fix xxlimited and xxlimited_35 build conditions (GH-29715) 2021-11-23 08:51:30 +01:00
Erlend Egeberg Aasland 2dc7d3dda6
bpo-45847: Port xxlimited and xxlimited_35 to PY_STDLIB_MOD (GH-29707) 2021-11-22 21:27:05 +01:00
Christian Heimes 0e1c2f3ef8
bpo-45847: port _struct to PY_STDLIB_MOD (GH-29706) 2021-11-22 15:58:43 +01:00
Erlend Egeberg Aasland eee683cbde
bpo-45847: Port audioop, _csv, and _posixsubprocess to PY_STDLIB_MOD_SIMPLE (GH-29705)
Automerge-Triggered-By: GH:tiran
2021-11-22 06:37:25 -08:00
Christian Heimes c6dec7e27a
bpo-45847: Port nis module to PY_STDLIB_MOD (GH-29699) 2021-11-22 15:18:41 +01:00
Christian Heimes d9cedabeba
bpo-45847: Port compression libs to PY_STDLIB_MOD (GH-29702) 2021-11-22 14:52:29 +01:00
Erlend Egeberg Aasland b451673f93
bpo-45847: Port mmap, select, and _xxsubinterpreters to Py_STDLIB_MOD (GH-29703) 2021-11-22 14:49:58 +01:00
Erlend Egeberg Aasland 5b946cadaa
bpo-45847: Port fcntl to Py_STDLIB_MOD (GH-29696)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-22 14:02:27 +01:00
Christian Heimes 29699a2a2a
bpo-45847: Various PY_STDLIB_MOD cleanups (GH-29697) 2021-11-22 12:09:14 +01:00
Erlend Egeberg Aasland 39f7d2ff01
bpo-45847: Port _lfprof, _opcode, _asyncio, _queue, _statistics, and _typing to PY_STDLIB_MOD_SIMPLE (GH-29690)
Automerge-Triggered-By: GH:tiran
2021-11-22 01:57:50 -08:00
Erlend Egeberg Aasland 718cee08cc
bpo-45847: Port _bisect, _heapq, _json, _pickle, _random, and _zoneinfo to PY_STDLIB_MOD_SIMPLE (GH-29689)
Automerge-Triggered-By: GH:tiran
2021-11-22 00:45:41 -08:00
Erlend Egeberg Aasland 133c65a870
bpo-45847: Port array, _contextvars, math, and cmath to PY_STDLIB_MOD_SIMPLE (GH-29688) 2021-11-22 09:05:36 +01:00
Erlend Egeberg Aasland db2277a114
bpo-45723: Add helpers for save/restore env (GH-29637) 2021-11-22 09:05:06 +01:00
Christian Heimes 2afa1a1266
bpo-45847: Port codecs and unicodedata to PY_STDLIB_MOD (GH-29685) 2021-11-21 14:08:47 +01:00
Christian Heimes f201d261cf
bpo-45847: Port grp, spwd, termios, resource, syslog to PY_STDLIB_MOD (GH-29668) 2021-11-21 10:45:31 +01:00
Christian Heimes f36c69a26e
bpo-45847: Port test modules to PY_STDLIB_MOD (GH-29660) 2021-11-20 16:43:10 +01:00
Erlend Egeberg Aasland 6d430ef5ab
bpo-45774: Fix SQLite load extension autodetection (GH-29659) 2021-11-20 15:02:52 +01:00
Christian Heimes 5596909eac
bpo-45847: Port _scproxy to PY_STDLIB_MOD (GH-29644) 2021-11-20 10:18:48 +01:00
Christian Heimes c8c21bdd19
bpo-45847: Port builtin hashlib extensions to PY_STDLIB_MOD (GH-29642) 2021-11-19 20:20:32 +01:00
Christian Heimes 7e44dc0ba7
bpo-45573: Move mandatory core modules to Modules/Setup.bootstrap (GH-29616) 2021-11-19 16:40:57 +01:00
Erlend Egeberg Aasland 29e5874d5a
bpo-45774: Autoconfiscate SQLite detection (GH-29507)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-19 15:10:41 +01:00
Christian Heimes e4bb22fabb
bpo-45573: Add Modules/Setup.stdlib with conditional modules (GH-29615) 2021-11-18 14:40:01 +01:00
Christian Heimes 5275e59c0c
bpo-45573: check for ossaudiodev in configure (GH-29614) 2021-11-18 10:56:26 +01:00
Christian Heimes 25ecc040d0
bpo-45573: Introduce extension module flags in Makefile (GH-29594)
``configure`` now uses a standardized format to forward state, compiler
flags, and linker flags to ``Makefile``, ``setup.py``, and
``Modules/Setup``. ``makesetup`` use the new variables by default if a
module line does not contain any compiler or linker flags. ``setup.py``
has a new function ``addext()``.

For a module ``egg``, configure adds:

* ``MODULE_EGG`` with value yes, missing, disabled, or n/a
* ``MODULE_EGG_CFLAGS``
* ``MODULE_EGG_LDFLAGS``

``Makefile.pre.in`` may also provide ``MODULE_EGG_DEPS`` that lists
dependencies such as header files and static libs.

Signed-off-by: Christian Heimes <christian@python.org>
2021-11-18 09:18:44 +01:00
Christian Heimes 464e6616be
bpo-45800: Move pyexpat build setup into configure (GH-29547)
Settings for :mod:`pyexpat` C extension are now detected by ``configure``.
The bundled ``expat`` library is built in ``Makefile``.

Signed-off-by: Christian Heimes <christian@python.org>
2021-11-14 10:02:24 +01:00
Christian Heimes 0486570f7b
bpo-45798: Move _decimal build setup into configure (GH-29541)
Settings for :mod:`decimal` internal C extension are now detected by
:program:`configure`. The bundled `libmpdec` library is built in
``Makefile``.

Signed-off-by: Christian Heimes <christian@python.org>
2021-11-13 13:56:16 +01:00
Guido van Rossum 1cbaa505d0
bpo-45696: Deep-freeze selected modules (GH-29118)
This gains 10% or more in startup time for `python -c pass` on UNIX-ish systems.

The Makefile.pre.in generating code builds on Eric's work for bpo-45020, but the .c file generator is new.

Windows version TBD.
2021-11-10 18:01:53 -08:00
Christian Heimes fc9b622819
bpo-45723: Add --with-pkg-config to configure (GH-29517)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-10 23:02:19 +01:00
Christian Heimes 0a9f69539b
bpo-45747: Detect gdbm/dbm dependencies in configure (GH-29467)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-10 20:26:55 +01:00
Erlend Egeberg Aasland 76d14fac72
bpo-45723: Improve and simplify more configure.ac checks (GH-29485) 2021-11-10 16:30:18 +01:00
Erlend Egeberg Aasland 49171aa91a
bpo-45723: Remove dead code for obsolete `--with-dyld` option (GH-29500)
Was commented out by Jack Jansen in 2001-08-15 by commit
b6e9cad34ce46a6a733d8aa5bf5b9d389fa1316f:

"Ripped out Next/OpenStep support, which was broken anyway"

Automerge-Triggered-By: GH:tiran
2021-11-09 13:05:58 -08:00
Christian Heimes 5b7c7cb104
bpo-45763: Detect compression build deps in configure (GH-29483) 2021-11-09 16:42:53 +01:00
Erlend Egeberg Aasland 185533639d
bpo-45723: Remove obsolete AC_EXEEXT from configure.ac (GH-29486)
From the autoconf docs *Obsolete Macros* section:

    Defined the output variable EXEEXT based on the output of the 
    compiler, which is now done automatically. Typically set to empty
    string if Posix and ‘.exe’ if a DOS variant.
2021-11-09 07:32:59 -08:00
Christian Heimes 8fefaad242
bpo-45743: -Wl,-search_paths_first is no longer needed (GH-29464) 2021-11-09 09:06:41 +01:00
Christian Heimes cbab997efb
bpo-45723: Prepare support for autoconf 2.71 (GH-29441) 2021-11-08 19:31:14 +01:00
Erlend Egeberg Aasland 9bd0cf5970
bpo-45723: Add macro for disabling/enabling CC warnings (GH-29466)
Co-authored-by: Christian Heimes <christian@python.org>
2021-11-08 18:58:22 +01:00
Christian Heimes 57c50c9c7c
bpo-45723: Add helper macros and more caching to configure.ac (GH-29429)
Almost all checks are now cached by AC_CACHE_CHECK().

Common patterns are replaced by helper macros.

Variable names now use naming scheme ``ac_cv_func_$funcname``,
``ac_cv_lib_$library_$funcname``, or ``ac_cv_header_$headername_h``.

``SYS_SELECT_WITH_SYS_TIME`` is no longer used.

``uuid_create`` and ``uuid_enc_be`` are provided by libc on BSD. It is
safe to use ``AC_CHECK_FUNCS`` here.

Caching speeds up ./configure -C from ~ 4s to 2.6s on my system.

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-11-08 08:06:31 +01:00
Christian Heimes be3cd5c05d
bpo-45723: Detect missing pkg-config (GH-29442) 2021-11-07 11:18:45 +01:00
Christian Heimes e9594f6747
bpo-45731: Handle --enable-loadable-sqlite-extensions in configure (GH-29434) 2021-11-06 10:30:37 +01:00
Christian Heimes 77a1f8d94c
bpo-43158: Use configure values for building _uuid extension (GH-29353) 2021-11-02 12:25:13 +01:00
Christian Heimes 4c95fb4640
bpo-45548: Fix out-of-tree and Debian builds (GH-29263)
Add Modules subdirs to SRCDIRS to generate directories for out-of-tree
object files.

Debian wants ncurses lib. Works on Fedora, too.

Debian also needs pkg-config to detect correct flags.

Remove more outdated comments. Makefile now tracks header dependencies
-lintl is injected by configure when needed. Build _dbm with
gdbm-compat.

Group some modules by purpose. socket, select, and mmap work on Windows,
too.
2021-10-28 11:57:38 +02:00
Christian Heimes 77e3f224d6
bpo-45548: Remove checks for finite and gamma (GH-29206) 2021-10-25 18:25:43 +02:00
Christian Heimes fa26245a1c
bpo-45548: Remove _math.c workarounds for pre-C99 libm (GH-29179)
The :mod:`math` and :mod:`cmath` implementation now require a C99 compatible
``libm`` and no longer ship with workarounds for missing acosh, asinh,
expm1, and log1p functions.

The changeset also removes ``_math.c`` and moves the last remaining
workaround into ``_math.h``. This simplifies static builds with
``Modules/Setup`` and resolves symbol conflicts.

Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
Co-authored-by: Brett Cannon <brett@python.org>
Signed-off-by: Christian Heimes <christian@python.org>
2021-10-25 01:25:27 -07:00
Christian Heimes 9942f42a93
bpo-45522: Allow to disable freelists on build time (GH-29056)
Freelists for object structs can now be disabled. A new ``configure``
option ``--without-freelists`` can be used to disable all freelists
except empty tuple singleton. Internal Py*_MAXFREELIST macros can now
be defined as 0 without causing compiler warnings and segfaults.

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-21 06:12:20 -07:00
Christian Heimes 81520fe677
bpo-45536: Check OpenSSL APIs in configure (GH-29088) 2021-10-20 17:18:34 +02:00
Victor Stinner 00ffc4513d
bpo-45440: Remove pymath.c fallbacks (GH-28977)
Remove fallbacks for missing round(), copysign() and hypot() in
Python/pymath.c. Python now requires these functions to build.

These fallbacks were needed on Visual Studio 2012 and older. They are
no longer needed since Visual Stuido 2013. Python is now built with
Visual Studio 2017 or newer since Python 3.6.
2021-10-15 19:45:34 +02:00
Victor Stinner 194a9526d8
bpo-45440: Require math.h isinf() to build (GH-28894)
Building Python now requires a C99 <math.h> header file providing
isinf(), isnan() and isfinite() functions.

Remove the Py_FORCE_DOUBLE() macro. It was used by the
Py_IS_INFINITY() macro.

Changes:

* Remove Py_IS_NAN(), Py_IS_INFINITY() and Py_IS_FINITE()
  in PC/pyconfig.h.
* Remove the _Py_force_double() function.
* configure no longer checks if math.h defines isinf(), isnan() and
  isfinite().
2021-10-13 23:27:50 +02:00
David Bohman 9c4766772c
bpo-45405: Prevent ``internal configure error`` when running ``configure`` with recent versions of non-Apple clang. (#28845)
Change the configure logic to function properly on macOS when the compiler
outputs a platform triplet for option --print-multiarch.

Co-authored-by: Ned Deily <nad@python.org>
2021-10-12 20:10:26 -04:00
Mike Gilbert be21706f37
bpo-45433: Do not link libpython against libcrypt (GH-28881)
Save/restore LIBS when calling AC_SEARCH_LIBS(..., crypt). This avoid
linking libpython with libcrypt.
2021-10-12 01:24:03 +02:00
Inada Naoki ad970e8623
bpo-29410: Change the default hash algorithm to SipHash13. (GH-28752)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
Co-authored-by: Christian Heimes <christian@python.org>
2021-10-10 17:29:46 +09:00
Ned Deily 0af08f343a
bpo-45163: Restrict added libnetwork check to builds on Haiku. (GH-28729)
For example, without the guard the check could cause macOS
installer builds to fail to install on older supported macOS
releases where libnetwork is not available and is not needed
on any release.
2021-10-05 11:02:57 +01:00
Victor Stinner 1ee0f94d16
bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() (GH-28662)
On Unix, if the sem_clockwait() function is available in the C
library (glibc 2.30 and newer), the threading.Lock.acquire() method
now uses the monotonic clock (time.CLOCK_MONOTONIC) for the timeout,
rather than using the system clock (time.CLOCK_REALTIME), to not be
affected by system clock changes.

configure now checks if the sem_clockwait() function is available.
2021-10-01 09:55:28 +02:00
Victor Stinner 7834ff26cb
bpo-21302: Add nanosleep() implementation for time.sleep() in Unix (GH-28545)
Co-authored-by: Livius <egyszeregy@freemail.hu>
2021-09-25 14:36:26 +02:00
Livius 85a4748118
bpo-21302: Add clock_nanosleep() implementation for time.sleep() (GH-28111)
In Unix operating systems, time.sleep() now uses the clock_nanosleep() function,
if available, which allows to sleep for an interval specified with nanosecond precision.

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-09-13 14:37:38 +02:00
David CARLIER 5f5b7d0c65
bpo-45163: Haiku build fix. (GH-28269)
linkage issues mainly for shared libs and missing system library,
also little nit into the signal extension as strsignal returns
a constant in this platform.
2021-09-11 11:46:22 +03:00
Łukasz Langa be9de8721d
bpo-34602: Quadruple stack size on macOS when compiling with UBSAN (GH-27309) 2021-09-03 09:32:19 +02:00
Dong-hee Na b2cf2513f9
bpo-44340: Add support for building with clang full/thin lto (GH-27231) 2021-07-19 19:52:56 +09:00
Binbin 17b16e13bb
Fix typos in multiple files (GH-26689)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-06-12 22:47:44 -04:00
Ned Deily 59acfd4a09
bpo-43109: Fix --with-lto configure option on macOS (GH-26341)
Allow --with-lto configure option to work with Apple-supplied Xcode or
Command Line Tools which do not provide llvm-ar.
2021-05-24 23:00:04 -04:00
Dong-hee Na a9e43615c2
Fix typo for configure.ac (GH-26305) 2021-05-22 08:40:59 +09:00
Pablo Galindo bb3e0c240b
Python 3.11.0a0 2021-05-03 21:25:35 +01:00
Ned Deily 8703178258
bpo-43568: Drop support for MACOSX_DEPLOYMENT_TARGET < 10.3 (GH-25827)
Only complain if the config target is >= 10.3 and the current target is
< 10.3. The check was originally added to ensure that incompatible
LDSHARED flags are not used, because -undefined dynamic_lookup is
used when building for 10.3 and later, and is not supported on older OS
versions. Apart from that, there should be no problem in general
with using an older target.

Authored-by: Joshua Root <jmr@macports.org>
2021-05-02 20:28:43 -04:00
Ned Deily 0cb33da1cc
bpo-44009: Provide "python3.x-intel64" for Apple Silicon Macs (GH-25804)
This allows reliably forcing macOS universal2 framework builds
to run under Rosetta 2 Intel-64 emulation on Apple Silicon Macs
if needed for testing or when universal2 wheels are not yet
available.
2021-05-02 04:48:29 -04:00
Christian Heimes e983252b51
bpo-43998: Default to TLS 1.2 and increase cipher suite security (GH-25778)
The ssl module now has more secure default settings. Ciphers without forward
secrecy or SHA-1 MAC are disabled by default. Security level 2 prohibits
weak RSA, DH, and ECC keys with less than 112 bits of security.
:class:`~ssl.SSLContext` defaults to minimum protocol version TLS 1.2.
Settings are based on Hynek Schlawack's research.

```
$ openssl version
OpenSSL 1.1.1k  FIPS 25 Mar 2021
$ openssl ciphers -v '@SECLEVEL=2:ECDH+AESGCM:ECDH+CHACHA20:ECDH+AES:DHE+AES:!aNULL:!eNULL:!aDSS:!SHA1:!AESCCM'
TLS_AES_256_GCM_SHA384  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 Kx=any      Au=any  Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(128) Mac=AEAD
TLS_AES_128_CCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESCCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
```

Signed-off-by: Christian Heimes <christian@python.org>
2021-05-01 20:53:10 +02:00
Jakub Kulík 9032cf5cb1
bpo-43667: Fix broken Unicode encoding in non-UTF locales on Solaris (GH-25096) 2021-04-30 15:21:42 +02:00
Christian Heimes 39258d3595
bpo-43669: PEP 644: Require OpenSSL 1.1.1 or newer (GH-23014)
- Remove HAVE_X509_VERIFY_PARAM_SET1_HOST check
- Update hashopenssl to require OpenSSL 1.1.1
- multissltests only OpenSSL > 1.1.0
- ALPN is always supported
- SNI is always supported
- Remove deprecated NPN code. Python wrappers are no-op.
- ECDH is always supported
- Remove OPENSSL_VERSION_1_1 macro
- Remove locking callbacks
- Drop PY_OPENSSL_1_1_API macro
- Drop HAVE_SSL_CTX_CLEAR_OPTIONS macro
- SSL_CTRL_GET_MAX_PROTO_VERSION is always defined now
- security level is always available now
- get_num_tickets is available with TLS 1.3
- X509_V_ERR MISMATCH is always available now
- Always set SSL_MODE_RELEASE_BUFFERS
- X509_V_FLAG_TRUSTED_FIRST is always available
- get_ciphers is always supported
- SSL_CTX_set_keylog_callback is always available
- Update Modules/Setup with static link example
- Mention PEP in whatsnew
- Drop 1.0.2 and 1.1.0 from GHA tests
2021-04-17 11:36:35 +02:00
Jessica Clarke dec0757549
bpo-43179: Generalise alignment for optimised string routines (GH-24624)
* Remove m68k-specific hack from ascii_decode

On m68k, alignments of primitives is more relaxed, with 4-byte and
8-byte types only requiring 2-byte alignment, thus using sizeof(size_t)
does not work. Instead, use the portable alternative.

Note that this is a minimal fix that only relaxes the assertion and the
condition for when to use the optimised version remains overly strict.
Such issues will be fixed tree-wide in the next commit.

NB: In C11 we could use _Alignof(size_t) instead, but for compatibility
we use autoconf.

* Optimise string routines for architectures with non-natural alignment

C only requires that sizeof(x) is a multiple of alignof(x), not that the
two are equal. Thus anywhere where we optimise based on alignment we
should be using alignof(x) not sizeof(x).

This is more annoying than it would be in C11 where we could just use
_Alignof(x) (and alignof(x) in C++11), but since we still require only
C99 we must plumb the information all the way from autoconf through the
various typedefs and defines.
2021-03-31 12:12:39 +02:00
Christian Heimes 5d6e8c1c1a
bpo-43617: Check autoconf-archive package in configure.ac (GH-25016)
Signed-off-by: Christian Heimes <christian@python.org>
2021-03-27 14:44:04 +01:00
Victor Stinner 94faa0724f
bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers (GH-24933)
These functions were undocumented and excluded from the limited C
API.

Most names defined by these header files were not prefixed by "Py"
and so could create names conflicts. For example, Python-ast.h
defined a "Yield" macro which was conflict with the "Yield" name used
by the Windows <winbase.h> header.

Use the Python ast module instead.

* Move Include/asdl.h to Include/internal/pycore_asdl.h.
* Move Include/Python-ast.h to Include/internal/pycore_ast.h.
* Remove ast.h header file.
* pycore_symtable.h no longer includes Python-ast.h.
2021-03-23 20:47:40 +01:00
Christian Heimes 32eba61ea4
bpo-43466: Add --with-openssl-rpath configure option (GH-24820) 2021-03-19 10:29:25 +01:00
Ned Deily 0608425944
bpo-43103: Fix build failure with macOS framework builds. (GH-24676) 2021-03-01 02:04:02 -05:00
Manolis Stamatogiannakis d20279494a
bpo-42603: Use pkg-config to get TCL/TK paths for tkinter. (GH-23721) 2021-02-28 22:29:57 -05:00
Victor Stinner 801bb0b503
bpo-43103: Add configure --without-static-libpython (GH-24418)
Add a new configure --without-static-libpython option to not build
the libpythonMAJOR.MINOR.a static library and not install the
python.o object file.

Fix smelly.py and stable_abi.py tools when libpython3.10.a is
missing.
2021-02-17 11:14:42 +01:00
Roland Hieber e1f7769513
bpo-13501: allow choosing between readline and libedit (GH-24189)
In contrast to macOS, libedit is available as its own include file and
library on Linux systems to prevent file name clashes. So if both
libraries are available on the system, readline is currently chosen by
default; and if only libedit is available, it is not found at all. This
patch adds a way to link against libedit by adding the following
arguments to configure:

  --with-readline           link against libreadline (the default)
  --with-readline=editline  link against libeditline
  --with-readline=no        disable building the readline module
  --without-readline        (same)

The runtime detection of libedit vs. readline was already done in commit
7105319ada (2019-12-04, serge-sans-paille: "bpo-38634: Allow
non-apple build to cope with libedit (GH-16986)").

Fixes: GH-12076 ("bpo-13501 Build or disable readline with Editline")
Fixes: bpo-13501 ("Make libedit support more generic; port readline / libedit to FreeBSD")
Co-authored-by: Enji Cooper (ngie-eign)
Co-authored-by: Martin Panter (vadmium)
Co-authored-by: Robert Marshall (kellinm)
2021-02-08 17:05:25 -08:00
Victor Stinner 6790005a9a
bpo-43031: Set a timeout when running tests in PGO build (GH-24339)
Pass --timeout=$(TESTTIMEOUT) option to the default profile task
"./python -m test --pgo" command.
2021-01-27 11:16:15 +01:00
Victor Stinner 75e59a97f5
bpo-42856: Add --with-wheel-pkg-dir=PATH configure option (GH-24210)
Add --with-wheel-pkg-dir=PATH option to the ./configure script. If
specified, the ensurepip module looks for setuptools and pip wheel
packages in this directory: if both are present, these wheel packages
are used instead of ensurepip bundled wheel packages.

Some Linux distribution packaging policies recommend against bundling
dependencies. For example, Fedora installs wheel packages in the
/usr/share/python-wheels/ directory and don't install the
ensurepip._bundled package.

ensurepip: Remove unused runpy import.
2021-01-20 17:07:21 +01:00
Paul Ganssle 0f66498fd8
bpo-42874: Remove grep -qE options for Solaris 10 compatibility (GH-24200)
According to [bpo-42874](), some versions of grep do not support the `-q` and `-E` options. Although both options are used elsewhere in the configure script, this particular bit of validation can be achieved without them,
so there's no real harm in using a grep call with no flags.

Would be good to get some people taking advantage of the `--with-tzpath` arguments in the wild to try this out.. Local testing seems to indicate that this does the same thing, but I don't know that we have any buildbots using this option. Maybe @pablogsal?

[bpo-42874]():
2021-01-12 10:17:52 -08:00
pxinwr 277ce3060b
bpo-27640: Add --disable-test-modules configure option (GH-23886)
Added --disable-test-modules option to the configure script:
don't build nor install test modules.

Patch by Xavier de Gaye, Thomas Petazzoni and Peixing Xin.

Co-Authored-By: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Co-Authored-By: Xavier de Gaye <xdegaye@gmail.com>
2020-12-30 13:50:39 +01:00
Victor Stinner 52a327c1cb
bpo-39465: Add pycore_atomic_funcs.h header (GH-20766)
Add pycore_atomic_funcs.h internal header file: similar to
pycore_atomic.h but don't require to declare variables as atomic.

Add _Py_atomic_size_get() and _Py_atomic_size_set() functions.
2020-12-23 03:41:08 +01:00
Matti Picus a44ce6c9f7
bpo-42604: always set EXT_SUFFIX=${SOABI}${SHLIB_SUFFIX} when using configure (GH-23708)
Now all platforms use a value for the "EXT_SUFFIX" build variable derived
from SOABI (for instance in FreeBSD, "EXT_SUFFIX" is now ".cpython-310d.so"
instead of ".so"). Previously only Linux, Mac and VxWorks were using a value
for "EXT_SUFFIX" that included "SOABI".

Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2020-12-20 02:56:57 +00:00
pxinwr c117426bf8
bpo-31904: Enable libpython3.so shared library for VxWorks (GH-23741) 2020-12-14 23:14:43 +01:00
Joshua Root 674fa0a740
bpo-42598: Fix implicit function declarations in configure (GH-23690)
This is invalid in C99 and later and is an error with some compilers
(e.g. clang in Xcode 12), and can thus cause configure checks to
produce incorrect results.
2020-12-13 15:56:34 -05:00
Ronald Oussoren 0f20bd9042
bpo-38443: Check that the specified universal architectures work (GH-22910)
As [bpo-38443]() says the error message from configure when specifying --enable-universalsdk with a set of architectures that is not supported by the compiler is not very helpful.   This PR explicitly checks if the compiler works and bails out if it doesn't.
2020-11-21 17:13:11 -08:00
Pablo Galindo a57b3d30f6
bpo-41625: Expose the splice() system call in the os module (GH-21947) 2020-11-17 00:00:38 +00:00
Kevin Adler c79667ff79
bpo-42087: Remove support for AIX 5.3 and below (GH-22830)
As AIX 5.3 and below do not support thread_cputime, it was decided in
https://bugs.python.org/issue40680 to require AIX 6.1 and above. This
commit removes workarounds for — and references to — older, unsupported
AIX versions.
2020-11-16 16:16:10 +01:00
Christian Heimes cd9fed6afb
bpo-41001: Add os.eventfd() (#20930)
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
2020-11-13 19:48:52 +01:00
Ronald Oussoren 41761933c1
bpo-41100: Support macOS 11 and Apple Silicon (GH-22855)
Co-authored-by:  Lawrence D’Anna <lawrence_danna@apple.com>

* Add support for macOS 11 and Apple Silicon (aka arm64)
   
  As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy

* Support building on recent versions of macOS while deploying to older versions

  This allows building installers on macOS 11 while still supporting macOS 10.9.
2020-11-08 10:05:27 +01:00
Alexey Izbyshev 976da903a7
bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe (GH-11671)
* bpo-35823: subprocess: Use vfork() instead of fork() on Linux when safe

When used to run a new executable image, fork() is not a good choice
for process creation, especially if the parent has a large working set:
fork() needs to copy page tables, which is slow, and may fail on systems
where overcommit is disabled, despite that the child is not going to
touch most of its address space.

Currently, subprocess is capable of using posix_spawn() instead, which
normally provides much better performance. However, posix_spawn() does not
support many of child setup operations exposed by subprocess.Popen().
Most notably, it's not possible to express `close_fds=True`, which
happens to be the default, via posix_spawn(). As a result, most users
can't benefit from faster process creation, at least not without
changing their code.

However, Linux provides vfork() system call, which creates a new process
without copying the address space of the parent, and which is actually
used by C libraries to efficiently implement posix_spawn(). Due to sharing
of the address space and even the stack with the parent, extreme care
is required to use vfork(). At least the following restrictions must hold:

* No signal handlers must execute in the child process. Otherwise, they
  might clobber memory shared with the parent, potentially confusing it.

* Any library function called after vfork() in the child must be
  async-signal-safe (as for fork()), but it must also not interact with any
  library state in a way that might break due to address space sharing
  and/or lack of any preparations performed by libraries on normal fork().
  POSIX.1 permits to call only execve() and _exit(), and later revisions
  remove vfork() specification entirely. In practice, however, almost all
  operations needed by subprocess.Popen() can be safely implemented on
  Linux.

* Due to sharing of the stack with the parent, the child must be careful
  not to clobber local variables that are alive across vfork() call.
  Compilers are normally aware of this and take extra care with vfork()
  (and setjmp(), which has a similar problem).

* In case the parent is privileged, special attention must be paid to vfork()
  use, because sharing an address space across different privilege domains
  is insecure[1].

This patch adds support for using vfork() instead of fork() on Linux
when it's possible to do safely given the above. In particular:

* vfork() is not used if credential switch is requested. The reverse case
  (simple subprocess.Popen() but another application thread switches
  credentials concurrently) is not possible for pure-Python apps because
  subprocess.Popen() and functions like os.setuid() are mutually excluded
  via GIL. We might also consider to add a way to opt-out of vfork() (and
  posix_spawn() on platforms where it might be implemented via vfork()) in
  a future PR.

* vfork() is not used if `preexec_fn != None`.

With this change, subprocess will still use posix_spawn() if possible, but
will fallback to vfork() on Linux in most cases, and, failing that,
to fork().

[1] https://ewontfix.com/7

Co-authored-by: Gregory P. Smith [Google LLC] <gps@google.com>
2020-10-23 17:47:01 -07:00
Petr Viktorin c6d7e82d19
bpo-38980: Only apply -fno-semantic-interposition if available (GH-22892) 2020-10-22 18:11:53 +02:00
Pablo Galindo b451b0e9a7
bpo-38980: Add -fno-semantic-interposition when building with optimizations (GH-22862) 2020-10-21 22:46:52 +01:00
Kevin Adler 1dd6d956a3
closes bpo-42030: Remove legacy AIX dynload support (GH-22717)
Since c19c5a6, AIX builds have defaulted to using dynload_shlib over
dynload_aix when dlopen is available. This function has been available
since AIX 4.3, which went out of support in 2003, the same year the
previously referenced commit was made. It has been nearly 20 years
since a version of AIX has been supported which has not used
dynload_shlib so there's no reason to keep this legacy code around.
2020-10-16 13:03:28 -05:00
Kyle Evans 1800c60080
bpo-40423: Optimization: use close_range(2) if available (GH-22651)
close_range(2) should be preferred at all times if it's available, otherwise we'll use closefrom(2) if available with a fallback to fdwalk(3) or plain old loop over fd range in order of most efficient to least.

[note that this version does check for ENOSYS, but currently ignores all other errors]

Automerge-Triggered-By: @pablogsal
2020-10-11 13:18:53 -07:00
Stefan Krah 84a7917b4c
bpo-41721: Add xlc options (GH-22096) 2020-09-04 22:33:17 +02:00
Stefan Krah e6dcd371b2
bpo-19521: Fix parallel build race condition on AIX (GH-21997)
Patch by Michael Haubenwallner.
2020-08-29 17:00:08 +02:00
wasiher 5798f78777
bpo-41340: Removed fallback implementation for strdup (GH-21634) 2020-07-27 12:28:45 +09:00
Ronald Oussoren 8ea6353f60
BPO-41100: Support macOS 11 when building (GH-21113) 2020-06-25 10:55:48 -04:00
Lysandros Nikolaou 314858e276
bpo-40939: Remove the old parser (Part 2) (GH-21005)
Remove some remaining files and Makefile targets for the old parser
2020-06-20 19:07:25 +01:00
Pablo Galindo 1ed83adb0e
bpo-40939: Remove the old parser (GH-20768)
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
2020-06-11 17:30:46 +01:00
Victor Stinner 51ae31e5b9
bpo-40684: Fix make install for platlibdir=lib64 (GH-20736)
"make install" now uses the PLATLIBDIR variable for the destination
lib-dynload/ directory when ./configure --with-platlibdir is used.

Update --with-platlibdir comment in configure.
2020-06-09 15:32:43 +02:00
Pablo Galindo d4fe098d1e
Python 3.10.0a0 (GH-20198) 2020-05-19 03:33:01 +01:00
Ned Deily 951ab58024
Revert "bpo-26317: Support OBJC and OBJCXX configure command line variables (GH-20176)" (GH-20182)
This reverts commit 0da5466650.

The commit is causing make failures on a FreeBSD buildbot.
Due to the imminent 3.9.0b1 cutoff, revert this commit for
now pending further investigation.
2020-05-18 11:31:21 -04:00
Ned Deily 0da5466650
bpo-26317: Support OBJC and OBJCXX configure command line variables (GH-20176)
Add support to the configure script for OBJC and OBJCXX command line options so that the macOS builds can use the clang compiler for the macOS-specific Objective C source files. This allows third-party compilers, like GNU gcc, to be used to build the rest of the project since some of the Objective C system header files are not compilable by GNU gcc.

Co-authored-by: Jeffrey Kintscher <websurfer@surf2c.net>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2020-05-18 09:17:22 -04:00
Paul Ganssle 62972d9d73
bpo-40503: PEP 615: Tests and implementation for zoneinfo (GH-19909)
This is the initial implementation of PEP 615, the zoneinfo module,
ported from the standalone reference implementation (see
https://www.python.org/dev/peps/pep-0615/#reference-implementation for a
link, which has a more detailed commit history).

This includes (hopefully) all functional elements described in the PEP,
but documentation is found in a separate PR. This includes:

1. A pure python implementation of the ZoneInfo class
2. A C accelerated implementation of the ZoneInfo class
3. Tests with 100% branch coverage for the Python code (though C code
   coverage is less than 100%).
4. A compile-time configuration option on Linux (though not on Windows)

Differences from the reference implementation:

- The module is arranged slightly differently: the accelerated module is
  `_zoneinfo` rather than `zoneinfo._czoneinfo`, which also necessitates
  some changes in the test support function. (Suggested by Victor
  Stinner and Steve Dower.)
- The tests are arranged slightly differently and do not include the
  property tests. The tests live at test/test_zoneinfo/test_zoneinfo.py
  rather than test/test_zoneinfo.py or test/test_zoneinfo/__init__.py
  because we may do some refactoring in the future that would likely
  require this separation anyway; we may:
        - include the property tests
        - automatically run all the tests against both pure Python and C,
          rather than manually constructing C and Python test classes (similar
          to the way this works with test_datetime.py, which generates C
          and Python test cases from datetimetester.py).
- This includes a compile-time configuration option on Linux (though not
  on Windows); added with much help from Thomas Wouters.
- Integration into the CPython build system is obviously different from
  building a standalone zoneinfo module wheel.
- This includes configuration to install the tzdata package as part of
  CI, though only on the coverage jobs. Introducing a PyPI dependency as
  part of the CI build was controversial, and this is seen as less of a
  major change, since the coverage jobs already depend on pip and PyPI.

Additional changes that were introduced as part of this PR, most / all of
which were backported to the reference implementation:

- Fixed reference and memory leaks

    With much debugging help from Pablo Galindo

- Added smoke tests ensuring that the C and Python modules are built

    The import machinery can be somewhat fragile, and the "seamlessly falls
    back to pure Python" nature of this module makes it so that a problem
    building the C extension or a failure to import the pure Python version
    might easily go unnoticed.

- Adjustments to zoneinfo.__dir__

    Suggested by Petr Viktorin.

- Slight refactorings as suggested by Steve Dower.

- Removed unnecessary if check on std_abbr

    Discovered this because of a missing line in branch coverage.
2020-05-16 10:20:06 +02:00
Christian Heimes 9b60e55db2
bpo-40637: Add option to disable builtin hashes (GH-20121)
Signed-off-by: Christian Heimes <christian@python.org>

Automerge-Triggered-By: @tiran
2020-05-15 14:54:53 -07:00
Victor Stinner c5fa364f4e
bpo-40514: Add --with-experimental-isolated-subinterpreters (GH-19926)
Add --with-experimental-isolated-subinterpreters build option to
configure: better isolate subinterpreters, experimental build mode.

When used, force the usage of the libc malloc() memory allocator,
since pymalloc relies on the unique global interpreter lock (GIL).
2020-05-05 16:41:11 +02:00
karl ding 360371f79c
bpo-40291: Add support for CAN_J1939 sockets (GH-19538)
Add support for CAN_J1939 sockets that wrap SAE J1939 protocol
functionality provided by Linux 5.4+ kernels.
2020-04-29 15:31:19 -07:00
Pablo Galindo a25f3c4c8f
bpo-40334: Fix builds outside the source directory and regenerate autoconf files (GH-19667) 2020-04-23 01:38:11 +01:00
Zackery Spytz 97e0de04b8
bpo-25780: Expose CAN_RAW_JOIN_FILTERS in the socket module (GH-19190)
Co-Authored-By: Stefan Tatschner <stefan@rumpelsepp.org>
2020-04-09 13:03:49 +01:00
Petr Viktorin 3c97e1e457
bpo-39761: Fix dtrace build with empty $DFLAGS (GH-18766)
This fixes a regression introduced in bpo-38960.

When DFLAGS was empty, "$DFLAGS" results in an empty argument ("").
Without the quotes, an empty variable will be ignored by the shell.
2020-03-11 14:27:42 +01:00
Victor Stinner 8510f43078
bpo-1294959: Add sys.platlibdir attribute (GH-18381)
Add --with-platlibdir option to the configure script: name of the
platform-specific library directory, stored in the new sys.platlitdir
attribute. It is used to build the path of platform-specific dynamic
libraries and the path of the standard library.

It is equal to "lib" on most platforms. On Fedora and SuSE, it is
equal to "lib64" on 64-bit systems.

Co-Authored-By: Jan Matějek <jmatejek@suse.com>
Co-Authored-By: Matěj Cepl <mcepl@cepl.eu>
Co-Authored-By: Charalampos Stratakis <cstratak@redhat.com>
2020-03-10 09:53:09 +01:00
Stefan Krah 815280eb16
bpo-39794: Add --without-decimal-contextvar (#18702) 2020-02-29 19:43:42 +01:00
Ian Norton a9edf44a2d
closes bpo-39619 Fix os.chroot on HP-UX 11.31 (GH-18495)
Setting `-D_XOPEN_SOURCE=700` on HP-UX causes system functions such as chroot to be undefined.  This change stops `_XOPEN_SOURCE` begin set on HP-UX

Co-authored-by: Benjamin Peterson <benjamin@python.org>
2020-02-13 19:09:11 -08:00
David Carlier aabdeb766b bpo-38960: DTrace build fix for FreeBSD. (GH-17451)
DTrace build fix for FreeBSD.

- allowing passing an extra flag as it need to define the arch size.
- casting some probe's arguments.
2020-01-28 13:53:32 +01:00
Victor Stinner b8d1262e8a
bpo-39395: putenv() and unsetenv() always available (GH-18135)
The os.putenv() and os.unsetenv() functions are now always available.

On non-Windows platforms, Python now requires setenv() and unsetenv()
functions to build.

Remove putenv_dict from posixmodule.c: it's not longer needed.
2020-01-24 14:05:48 +01:00
Victor Stinner b477d19a6b
bpo-39406: Implement os.putenv() with setenv() if available (GH-18128)
If setenv() C function is available, os.putenv() is now implemented
with setenv() instead of putenv(), so Python doesn't have to handle
the environment variable memory.
2020-01-22 22:48:16 +01:00
Anthony Shaw 2de064e630 bpo-39160 Align the verbs, grammar and defaults for `./configure --help` (GH-17747) 2020-01-14 01:40:10 -05:00
Michael Felt 39afa2d314 bpo-38021: Modify AIX platform_tag so it covers PEP 425 needs (GH-17303)
Provides a richer platform tag for AIX that we expect to be sufficient for PEP 425
binary distribution identification. Any backports to earlier Python versions will be
handled via setuptools.

Patch by Michael Felt.
2019-12-16 00:17:53 +10:00
Benjamin Peterson 5c0c325453 closes bpo-38713: Expose P_PIDFD in os if it's defined. (GH-17071)
https://bugs.python.org/issue38713
2019-11-05 21:58:31 -08:00
Victor Stinner 028f7349a0
bpo-37415: Fix stdatomic.h header check for ICC compiler (GH-16717)
Fix stdatomic.h header check for ICC compiler: the ICC implementation
lacks atomic_uintptr_t type which is needed by Python.

Test:

* atomic_int and atomic_uintptr_t types
* atomic_load_explicit() and atomic_store_explicit()
* memory_order_relaxed and memory_order_seq_cst constants

But don't test ATOMIC_VAR_INIT(): it's not used in Python.
2019-10-22 21:53:50 +02:00
Vinay Sajip 0b60f64e43
bpo-11410: Standardize and use symbol visibility attributes across POSIX and Windows. (GH-16347) 2019-10-15 08:26:12 +01:00
Jesús Cea 52d1b86bde
bpo-38301: In Solaris family, we must be sure to use '-D_REENTRANT' (#16446) 2019-09-28 03:44:32 +02:00
Doyle Rowland 0519d497b0 closes bpo-36002: Use AC_PATH_TOOL to find llvm-profdata and llvm-ar. (GH-14998) 2019-09-13 14:38:07 +01:00
Jakub Kulík e20134f889 bpo-38110: Use fdwalk for os.closerange() when available. (GH-15224)
Use fdwalk() on platforms that support it to implement os.closerange().
2019-09-11 10:11:57 -05:00
Benjamin Peterson f1c19031fd bpo-38068: Clean up gettimeofday configure logic. (GH-15775)
Assume gettimeofday exists and takes two arguments.
2019-09-10 03:37:59 -07:00
Benjamin Peterson bed04b6647
bpo-34652 again: Remove lchmod from the default AC_CHECK_FUNCS list. (GH-15758) 2019-09-09 05:13:00 -07:00
Sergey Fedoseev 52c1a6a15a Remove leftovers from the times when long long wasn't required (GH-15501)
In a38e9d1399 pyconfig.h.in was
manually edited and that edit was overwritten when running autoreconf.
2019-08-26 16:12:47 +01:00
Ronald Oussoren 1a057bab0f
bpo-18049: Sync thread stack size to main thread size on macOS (GH-14748)
This changeset increases the default size of the stack
for threads on macOS to the size of the stack
of the main thread and reenables the relevant
recursion test.
2019-08-01 07:43:07 +02:00
Neil Schemenauer 4e16a4a311
bpo-36044: Reduce number of unit tests run for PGO build (GH-14702)
Reduce the number of unit tests run for the PGO generation task.  This
speeds up the task by a factor of about 15x.  Running the full unit test
suite is slow.  This change may result in a slightly less optimized build
since not as many code branches will be executed.  If you are willing to
wait for the much slower build, the old behavior can be restored using
'./configure [..] PROFILE_TASK="-m test --pgo-extended"'.  We make no
guarantees as to which PGO task set produces a faster build.  Users who
care should run their own relevant benchmarks as results can depend on
the environment, workload, and compiler tool chain.
2019-07-22 12:54:25 -07:00
Ned Deily 5bbbc733e6
bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)
Under some conditions the earlier fix for bpo-18075, "Infinite recursion
tests triggering a segfault on Mac OS X", now causes failures on macOS
when attempting to change stack limit with resource.setrlimit
resource.RLIMIT_STACK, like regrtest does when running the test suite.
The reverted change had specified a non-default stack size when linking
the python executable on macOS.  As of macOS 10.14.4, the previous
code causes a hard failure when running tests, although similar
failures had been seen under some conditions under some earlier
systems.  Reverting the change to the interpreter stack size at link
time helped for release builds but caused some tests to fail when
built --with-pydebug.  Try the opposite approach: continue to build
the interpreter with an increased stack size on macOS and remove
the failing setrlimit call in regrtest initialization.  This will
definitely avoid the resource.RLIMIT_STACK error and should have
no, or fewer, side effects.
2019-07-02 03:12:18 -04:00
Łukasz Langa 9ab2fb1c68
Bump to 3.9.0a0 2019-06-04 22:12:32 +02:00
Pablo Galindo aac4d0342c
bpo-26826: Expose copy_file_range in the os module (GH-7255) 2019-05-31 19:39:47 +01:00
Zackery Spytz 43fdbd2729 bpo-26836: Add os.memfd_create() (#13567)
* bpo-26836: Add os.memfd_create()

* Use the glibc wrapper for memfd_create()

Co-Authored-By: Christian Heimes <christian@python.org>

* Fix deletions caused by autoreconf.

* Use MFD_CLOEXEC as the default value for *flags*.

* Add memset_s to configure.ac.

* Revert memset_s changes.

* Apply the requested changes.

* Tweak the docs.
2019-05-29 21:57:03 +02:00
David Carlier d8b7551672 bpo-33164: blake2 fix for HP-UX (GH-13633) 2019-05-29 19:58:11 +09:00
Zackery Spytz 02db696732 bpo-32941: Add madvise() for mmap objects (GH-6172)
Allow mmap objects to access the madvise() system call.
2019-05-27 18:48:16 +02:00
E. M. Bray b1fc417841 bpo-21536: Fix configure.ac for LIBPYTHON on Android/Cygwin (GH-13552)
Add also missing AC_MSG_RESULT for AC_MSG_CHECKING(MACHDEP).
2019-05-24 18:39:38 +02:00
E. M. Bray c994c8fc19 bpo-21536: On Cygwin, C extensions must be linked with libpython (GH-13549)
It is also possible to link against a library or executable with a
statically linked libpython, but not both with the same DLL.  In fact
building a statically linked python is currently broken on Cygwin
for other (related) reasons.

The same problem applies to other POSIX-like layers over Windows
(MinGW, MSYS) but Python's build system does not seem to attempt
to support those platforms at the moment.
2019-05-24 17:33:47 +02:00
David Carlier 51aa35e9e1 bpo-33164: update blake2 implementation (GH-6286) 2019-05-23 13:32:44 +09:00
Victor Stinner 0a8e57248b
bpo-36721: Add --embed option to python-config (GH-13500)
To embed Python into an application, a new --embed option must be
passed to "python3-config --libs --embed" to get "-lpython3.8" (link
the application to libpython). To support both 3.8 and older, try
"python3-config --libs --embed" first and fallback to "python3-config
--libs" (without --embed) if the previous command fails.

Add a pkg-config "python-3.8-embed" module to embed Python into an
application: "pkg-config python-3.8-embed --libs" includes
"-lpython3.8".  To support both 3.8 and older, try "pkg-config
python-X.Y-embed --libs" first and fallback to "pkg-config python-X.Y
--libs" (without --embed) if the previous command fails (replace
"X.Y" with the Python version).

On the other hand, "pkg-config python3.8 --libs" no longer contains
"-lpython3.8". C extensions must not be linked to libpython (except
on Android, case handled by the script); this change is backward
incompatible on purpose.

"make install" now also installs "python-3.8-embed.pc".
2019-05-23 03:30:23 +02:00
pxinwr f2d7ac7e5b bpo-31904: Add posix module support for VxWorks (GH-12118) 2019-05-21 12:46:37 +02:00
Victor Stinner d97adfb409
bpo-36618: Don't add -fmax-type-align=8 flag for clang (GH-13320)
Python 3.8 now respects the x86-64 ABI: memory allocations are
aligned on 16 bytes. The clang flag was only used as a temporary
workaround.
2019-05-14 19:29:53 +02:00
Ned Deily 883dfc668f
bpo-34602: Avoid failures setting macOS stack resource limit (GH-13011)
Under some conditions the earlier fix for bpo-18075, "Infinite recursion
tests triggering a segfault on Mac OS X", now causes failures on macOS
when attempting to change stack limit with resource.setrlimit
resource.RLIMIT_STACK, like regrtest does when running the test suite.
The reverted change had specified a non-default stack size when linking
the python executable on macOS.  As of macOS 10.14.4, the previous
code causes a hard failure when running tests, although similar
failures had been seen under some conditions under some earlier
systems.  For now, revert the original change and resume using
the default stack size when linking the interpreter.
2019-04-29 15:07:39 -04:00
xdegaye 254b309c80 bpo-21536: On Android, C extensions are linked to libpython (GH-12989) 2019-04-29 09:27:40 +02:00
Victor Stinner 6d13e5b35b
bpo-36722: Don't define ALT_SOABI for Py_TRACE_REFS build (GH-12973)
Py_TRACE_REFS ABI is incompatible with release and debug (Py_DEBUG)
ABI.
2019-04-26 18:56:19 +02:00
Victor Stinner 5422e3cfb7
bpo-36722: Debug build loads libraries built in release mode (GH-12952)
In debug build, import now also looks for C extensions compiled in
release mode and for C extensions compiled in the stable ABI.
2019-04-26 01:40:00 +02:00
Victor Stinner f4e4703e74
bpo-36465: Make release and debug ABI compatible (GH-12615)
Release build and debug build are now ABI compatible: the Py_DEBUG
define no longer implies Py_TRACE_REFS define which introduces the
only ABI incompatibility.

A new "./configure --with-trace-refs" build option is now required to
get Py_TRACE_REFS define which adds sys.getobjects() function and
PYTHONDUMPREFS environment variable.

Changes:

* Add ./configure --with-trace-refs
* Py_DEBUG no longer implies Py_TRACE_REFS
2019-04-25 00:56:28 +02:00
Victor Stinner 6c44fde3e0
bpo-36707: Remove the "m" flag (pymalloc) from SOABI (GH-12931)
"./configure --with-pymalloc" no longer adds the "m" flag to SOABI
(sys.implementation.cache_tag).

Enabling or disabling pymalloc has no impact on the ABI.
2019-04-24 16:10:09 +02:00
Victor Stinner a304b136ad
bpo-36618: Don't add -fmax-type-align flag to old clang (GH-12811) 2019-04-13 00:51:07 +02:00
Victor Stinner 23a683adf8
bpo-36618: Add -fmax-type-align=8 flag for clang (GH-12809)
Add -fmax-type-align=8 to CFLAGS when clang compiler is detected.

The pymalloc memory allocator aligns memory on 8 bytes. On x86-64,
clang expects alignment on 16 bytes by default and so uses MOVAPS
instruction which can lead to segmentation fault. Instruct clang that
Python is limited to alignemnt on 8 bytes to use MOVUPS instruction
instead: slower but don't trigger a SIGSEGV if the memory is not
aligned on 16 bytes.

Sadly, the flag must be expected to CFLAGS and not just
CFLAGS_NODIST, since third party C extensions can have the same
issue.
2019-04-12 21:27:37 +02:00